FS#10537 - Multifile Buffering and Wavpack hybrid support
Opened by Bryan Jacobs (BryanJacobs) - Monday, 17 August 2009, 19:33 GMT
Last edited by Mustapha Senhaji (DrMoos) - Monday, 17 August 2009, 19:35 GMT
The attached patch enables simultaneous buffering of multiple files in a minimally intrusive way. It also contains code to allow the playback of Wavpack hybrid files, both lossy and lossless. In the event that a Wavpack file is not a hybrid file, only original code paths are used. This patch will not affect users who do not use this file type.
How it works:
A handle may now be marked as "chunky". If a handle is so marked, adding a new handle will NOT reserve space in the buffer for the remaining portion of the file. Instead only a small amount of space (currently 2*BUFFERING_DEFAULT_FILECHUNK) is reserved. Additional checking is also added to ensure that buffer_handle will not allow one handle to overflow into another; in the event that a handle's widx would cross into the next handle's header, the handle is "extended" by calling add_handle a second time and linking from the original via file_next. Thus, there are two paths to iterate from a "chunky" handle - the next handle in the buffer layout (h->next), or the next handle containing data from the same file (h->file_next).
Key new methods:
extend_handle: used to create more space for a handle which is active but would overlap its successor if widx were advanced
clean_handle: remove pieces of a handle marked as "invalid", but don't remove the whole handle (as close_handle does)
bufopenchunky: as bufopen, but for a chunky handle
finish_handle: reserves either some or all space for a handle in the buffer
finish_chunky_handles: finishes all chunky handles
- Playback of lossless hybrid mono and stereo files in the sim functions mostly as expected. Occasional segfaults may occur - these are probably a thread safety issue. Trying on a unicore target would be productive.
- The functionality of any of rockbox other than hybrid wavpack files is not impaired
- Playback on an iPod 5.5G works as well, but is not yet real-time (apparently the hard disk isn't fast enough to buffer both files and keep up with playback, so priority should be given to the .wv file to fix this)
- Buffer space utilization is very efficient, the only waste is caused by the 300 bytes per struct memory_handle created. This could be minimized by only storing ridx, widx, and data in a minimal memory handle and copying the rest of the information to file_next as necessary (this is how open file descriptors are currently handled)
What doesn't work:
- Seeking does not always work, although seeks not requiring a rebuffer are usually OK (even those which must cross into file_next!). Especially unlikely to work is backwards seeking. This is because the Wavpack codec uses approximate seeking, which is fine when you don't have to worry about only one file but falls apart when you need to have the same block in both the wv and wvc files.
- Segmentation faults, especially at the end of very large hybrid files. These are almost all caused by the Wavpack codec in apps/codecs/libwavpack/words.c's response to reading extra 0 bytes. Wavpack does not check the validity of wvbits.ptr before reading from it! When combined with any error at all in the buffering code, or even any corruption in the input files, this leads to a segfault! There are also some issues in the original Rockbox buffering code. For example, in rebuffer_handle, h->next is used without first holding llist_mutex. This is a thread safety issue. On top of these, there's a bounds check I've missed somewhere in apps/buffering.c which can cause buffer corruption.
Most of the Wavpack decoding methods here were ported from those on http://wavpack.com with minor modifications. These files are under the GPL.
This patch contains commented printf statements as partial documentation and in case someone wishes to trace the flow of execution in the simulator.