Rockbox

This is the bug/patch tracker for Rockbox. Click here for more information.

Quick links: Bugs · Patches · Rockbox frontpage

Tasklist

FS#11429 - Improved sector cache / Remove on-stack and in-struct sector buffers / Align storage accesses

Attached to Project: Rockbox
Opened by Michael Sparmann (TheSeven) - Monday, 21 June 2010, 21:11 GMT+2
Last edited by Michael Sparmann (TheSeven) - Monday, 21 June 2010, 21:14 GMT+2
Task Type Patches
Category Operating System/Drivers
Status New
Assigned To No-one
Player Type All players
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Private No

Details

This patch adds a sector-based read/write cache between the storage and fat drivers.
It has a static pool of 64 sector buffers which can be locked by the file handling code to replace the sector buffers in the file and dir structs, and serve as cache as long as they aren't used. It can also be used to enforce exclusive access to a sector.

If requested using the STORAGE_NEEDS_ALIGN define, all storage accesses will be aligned to cacheline boundaries.
   sectorcache-r27014.patch (73.3 KiB) 
 apps/main.c                                          |    3 
 firmware/common/dir_uncached.c                       |    7 
 firmware/common/dircache.c                           |    5 
 firmware/common/disk.c                               |   22 
 firmware/common/file.c                               |  154 ++---
 firmware/common/sectorcache.c                        |  418 +++++++++++++
 firmware/drivers/fat.c                               |  571 +++++++++++--------
 firmware/export/fat.h                                |    9 
 firmware/export/sectorcache.h                        |   91 +++
 firmware/powermgmt.c                                 |    8 
 firmware/rolo.c                                      |    5 
 firmware/SOURCES                                     |    1 
 firmware/target/arm/s5l8700/app.lds                  |    2 
 firmware/target/arm/s5l8700/ipodnano2g/ftl-nano2g.c  |   10 
 firmware/target/arm/s5l8700/ipodnano2g/nand-nano2g.c |    8 
 firmware/target/arm/s5l8700/system-target.h          |    1 
 firmware/usb.c                                       |    5 
 firmware/usbstack/usb_storage.c                      |    9 
 18 files changed, 980 insertions(+), 349 deletions(-)

This task depends upon

Comment by Rafaël Carré (funman) - Tuesday, 22 June 2010, 07:41 GMT+2
doesn't build for MULTIDRIVE/MULTIVOLUME
Comment by Michael Sparmann (TheSeven) - Tuesday, 22 June 2010, 09:18 GMT+2
funman: I almost expected that. There seems to be quite a lot of inconsistent use of IF_MD vs. IF_MV even in the old FAT code, and to be honest, I haven't really understood what this is all about and what the difference between volumes and drives actually is.
Comment by amaury pouly (pamaury) - Tuesday, 22 June 2010, 10:18 GMT+2
Easy: drives are drives ! Volumes are logical drives (e.g a FAT partition)
If you have multidrive, then multivolume is enabled.
But yeah, the code is quitte different depending on MD or mot, this is a shame.
Comment by Michael Sparmann (TheSeven) - Tuesday, 22 June 2010, 20:39 GMT+2
hopefully works on multivolume/multidrive now
   fs11429-sectorcache-r27014.patch (74.4 KiB) 
 apps/main.c                                          |    3 
 firmware/common/dir_uncached.c                       |    6 
 firmware/common/dircache.c                           |    5 
 firmware/common/disk.c                               |   22 
 firmware/common/file.c                               |  155 ++---
 firmware/common/sectorcache.c                        |  418 +++++++++++++
 firmware/drivers/fat.c                               |  582 +++++++++++--------
 firmware/export/fat.h                                |   13 
 firmware/export/sectorcache.h                        |   91 ++
 firmware/powermgmt.c                                 |    8 
 firmware/rolo.c                                      |    5 
 firmware/target/arm/s5l8700/app.lds                  |    2 
 firmware/target/arm/s5l8700/ipodnano2g/ftl-nano2g.c  |   10 
 firmware/target/arm/s5l8700/ipodnano2g/nand-nano2g.c |    8 
 firmware/target/arm/s5l8700/system-target.h          |    1 
 firmware/usb.c                                       |    9 
 firmware/usbstack/usb_storage.c                      |    9 
 17 files changed, 998 insertions(+), 349 deletions(-)

Comment by Michael Marley (mamarley) - Thursday, 29 July 2010, 20:57 GMT+2
I am trying to build Rockbox with this patch for my Nano2G, but I get the following error when attempting to link:

LD rockbox.elf
/home/michael/Downloads/rockbox/build/apps/main.o: In function `main':
main.c:(.text+0x1c8): undefined reference to `sectorcache_init'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(powermgmt.o): In function `shutdown_hw':
powermgmt.c:(.text+0x2d8): undefined reference to `sectorcache_clean_all'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(rolo.o): In function `rolo_load':
rolo.c:(.text+0x1bc): undefined reference to `sectorcache_clean_all'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(file.o): In function `lseek':
file.c:(.text+0x184): undefined reference to `sectorcache_clean'
file.c:(.text+0x1a8): undefined reference to `sectorcache_unlock'
file.c:(.text+0x20c): undefined reference to `sectorcache_lock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(file.o): In function `readwrite':
file.c:(.text+0x364): undefined reference to `sectorcache_markdirty'
file.c:(.text+0x39c): undefined reference to `sectorcache_clean'
file.c:(.text+0x3d0): undefined reference to `sectorcache_unlock'
file.c:(.text+0x53c): undefined reference to `sectorcache_lock'
file.c:(.text+0x5ac): undefined reference to `sectorcache_markdirty'
file.c:(.text+0x5fc): undefined reference to `sectorcache_lock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(file.o): In function `fsync':
file.c:(.text+0xc14): undefined reference to `sectorcache_clean'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(file.o): In function `close':
file.c:(.text+0xd5c): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(disk.o): In function `disk_init':
disk.c:(.text+0x5c): undefined reference to `sectorcache_lock'
disk.c:(.text+0x108): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(disk.o): In function `disk_mount':
disk.c:(.text+0x13c): undefined reference to `sectorcache_invalidate'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `fat_closedir':
fat.c:(.text+0x134): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `flush_fat_sector':
fat.c:(.text+0x16c): undefined reference to `sectorcache_writethrough'
fat.c:(.text+0x1ac): undefined reference to `sectorcache_writethrough'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `cache_fat_sector':
fat.c:(.text+0x288): undefined reference to `sectorcache_readthrough'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `fat_getnext':
fat.c:(.text+0x75c): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x784): undefined reference to `sectorcache_lock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `transfer':
fat.c:(.text+0xc84): undefined reference to `sectorcache_writethrough'
fat.c:(.text+0xc9c): undefined reference to `sectorcache_readthrough'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `free_direntries':
fat.c:(.text+0xfc8): undefined reference to `sectorcache_lock'
fat.c:(.text+0x1028): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x1030): undefined reference to `sectorcache_clean'
fat.c:(.text+0x1054): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x1080): undefined reference to `sectorcache_lock'
fat.c:(.text+0x10b4): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x10bc): undefined reference to `sectorcache_clean'
fat.c:(.text+0x10e0): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `update_fsinfo':
fat.c:(.text+0x111c): undefined reference to `sectorcache_lock'
fat.c:(.text+0x115c): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x1164): undefined reference to `sectorcache_clean'
fat.c:(.text+0x1188): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `fat_mount':
fat.c:(.text+0x1378): undefined reference to `sectorcache_lock'
fat.c:(.text+0x1528): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x1540): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x15ac): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x16a0): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x16dc): undefined reference to `sectorcache_lock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `write_long_name':
fat.c:(.text+0x19b4): undefined reference to `sectorcache_lock'
fat.c:(.text+0x1ab0): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x1ab8): undefined reference to `sectorcache_clean'
fat.c:(.text+0x1adc): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x1b1c): undefined reference to `sectorcache_lock'
fat.c:(.text+0x1dc0): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x1dc8): undefined reference to `sectorcache_clean'
fat.c:(.text+0x1dec): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `add_dir_entry':
fat.c:(.text+0x2100): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x2110): undefined reference to `sectorcache_lock'
fat.c:(.text+0x21f4): undefined reference to `sectorcache_unlock'
fat.c:(.text+0x2240): undefined reference to `sectorcache_lock'
fat.c:(.text+0x2270): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x2278): undefined reference to `sectorcache_clean'
fat.c:(.text+0x22a0): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `update_short_entry':
fat.c:(.text+0x2494): undefined reference to `sectorcache_lock'
fat.c:(.text+0x2520): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x2528): undefined reference to `sectorcache_clean'
fat.c:(.text+0x254c): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `fat_rename':
fat.c:(.text+0x26c4): undefined reference to `sectorcache_lock'
fat.c:(.text+0x2730): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x2738): undefined reference to `sectorcache_clean'
fat.c:(.text+0x275c): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(fat.o): In function `fat_create_dir':
fat.c:(.text+0x28a8): undefined reference to `sectorcache_lock'
fat.c:(.text+0x28d8): undefined reference to `sectorcache_markdirty'
fat.c:(.text+0x28e0): undefined reference to `sectorcache_clean'
fat.c:(.text+0x2904): undefined reference to `sectorcache_unlock'
/home/michael/Downloads/rockbox/build/firmware/libfirmware.a(usb_storage.o): In function `usb_storage_init_connection':
usb_storage.c:(.text+0x1198): undefined reference to `sectorcache_invalidate'
usb_storage.c:(.text+0x11a8): undefined reference to `sectorcache_lock'
usb_storage.c:(.text+0x11bc): undefined reference to `sectorcache_unlock'
collect2: ld returned 1 exit status
make: *** [/home/michael/Downloads/rockbox/build/rockbox.elf] Error 1
rm /home/michael/Downloads/rockbox/build/apps/bitmaps/native/usblogo.128x37x16.c /home/michael/Downloads/rockbox/build/apps/bitmaps/native/default_icons.6x8x16.c /home/michael/Downloads/rockbox/build/apps/bitmaps/native/rockboxlogo.176x54x16.c
Comment by Michael Marley (mamarley) - Thursday, 29 July 2010, 21:13 GMT+2
I found the problem. Your second patch does not include the necessary modifications to the SOURCES file.
Comment by Michael Marley (mamarley) - Saturday, 31 July 2010, 12:19 GMT+2
This patch causes Rockbox not to work correctly. When it boots, it says that it couldn't find the Rockbox directory and that the installation is incomplete. It then displays an unthemed version of the normal menu, and attempting to access the "Files" list results in much garbage being displayed for the file names instead of the correct ones.

Loading...