This is the bug/patch tracker for Rockbox. Click here for more information.
Quick links: Bugs · Patches · Rockbox frontpage
FS#12216 - Dircache shuts down when closing a file that was opened prior to reloading the dircache
|
DetailsThe dircache shuts down (resets dircache_initialized to false) when it encounters files it does not know about, assuming a dircache inconsistency. One of the reasons for shutting down the dircache is when it encounters a file handle it doesn't know about. To prevent this from happening, the dircache maintains a mapping between file handles and file names, which is maintained even when the dircache is not fully operating yet during the boot phase (by way of a small temporary cache array).
The problem is that this scheme has a few holes that allow files to be open without the dircache knowing about it: 1. When dircache_initializing is not set, the temporary cache is not maintained and newly opened files will slip through. 2. When the dircache is disabled and later re-enabled, open-file mappings that were active during the first dircache instance are not carried over to the new instance. Both cases occur frequently when reloading the dircache after updating the database, which defaults to disabling the dircache and using the dircache buffer for its commit phase, and then reloading the dircache. The most common case I see is when playback is active during the automatic DB update (Resume Playback is my startup screen). When I wait for the DB update and dircache reload to complete and then start a new playlist from the database, the playlist-control file (I believe) is closed. The new dircache instance does not know that this file was open before and shuts down (in update_filesize). (With current SVN, this later causes the DB browser to crash, but that's a different story.) I'm wondering what the best way to keep the dircache online would be. Some ideas: * When the dircache shuts down, save the current list of bindings to the temporary cache, to be reloaded when the new instance starts up. Also set dircache_initializing = true, and reset it only when the temporary cache overflows). This would require a few more entries in the temporary cache than the current two. * Recover the bindings from the FAT filesystem layer, or maybe even generally store them there. This layer already has a list of open files, but currently does not store a file-descriptor binding. Unfortunately, this solution does not work for hosted platforms. * Make the dircache resilient against unknown file-descriptor bindings. It's somewhat unclear to me how that could work, as each handle-based update that we ignore and just pass through to the filesystem creates a dircache inconsistency. Thoughts? |
This task depends upon
For files, I believe it is indeed possible to rebuild the open file list in dircache.c from the open list file in file.c with information from the FAT layer. However, this is not exactly a free operation. The main advantage is that then you don't need to manage a queue.
I don't think making dircache resilient against unknown fd is a good idea, it will only hide the problem.
According to an IRC conversation with Slasheri, another related problem with dircache startup is that dircache entries may become stale due to races with file-modifying code executed during the startup phase (after the dircache entry has been created, but before the dircache goes online). A possible solution could be to delay write operations while dircache_initializing is true, either with a delay loop or with a semaphore.
Following up on how to prevent dircache from shutting down when it finds an unknown file handle:
One idea could be to replace the temporary cache with a complete mapping of file names to open file handles, kept up to date at all times regardless of whether dircache is online or not. This mapping could be stored for all open files in open() or in dircache_bind(), or we can rely on the FAT layer (on native platforms).
This information could be inspected when initializing the dircache. Alternatively, it could be used to recover the filename for an unknown handle late, when we would normally shut down the dircache; once the filename is known, the dircache entry can be updated from the real directory entry.