Rockbox

This is the bug/patch tracker for Rockbox. Click here for more information.

Quick links: Bugs · Patches · Rockbox frontpage

Tasklist

FS#8923 - m4a playback bugs

Attached to Project: Rockbox
Opened by Paul D (paschadee) - Monday, 21 April 2008, 16:51 GMT+2
Last edited by Andree Buschmann (Buschel) - Monday, 18 April 2011, 21:17 GMT+2
Task Type Bugs
Category Codecs
Status Closed
Assigned To No-one
Player Type iPod Nano
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Private No

Details

been using rockbox for years, but recently m4a playback has become more frequent as i encode more with it.

-long mixes e.g. 1hr dont load well (long wait times with the DB switched off)
-dont resume at all, sometimes. system crash if system restart and resumes playback in said mixes.
-Somewhere between 1 and two hours long or greater length continuous mixes in m4a simply don't load at all.

-bookmarks or resumed playback in extremely low bitrate files e.g. 40kbps doesn't resume correctly or at all in those files.

For reference, I use winamps 'format converter' plugin --> MP4/LC-AAC

can document more if need be.
This task depends upon

Closed by  Andree Buschmann (Buschel)
Monday, 18 April 2011, 21:17 GMT+2
Reason for closing:  Fixed
Additional comments about closing:  Finally fixed with r29745.
Comment by Paul Louden (Llorean) - Tuesday, 22 April 2008, 01:56 GMT+2
These are more or less known issues with the AAC codec and MP4 parser. The best solution is just to use a different codec. This problem has been around for a long time, and isn't likely to be an easy fix even if someone starts working on it.
Comment by Andree Buschmann (Buschel) - Tuesday, 01 February 2011, 09:33 GMT+2
With r29175 and r29178 the resume should work properly now.
Comment by Andree Buschmann (Buschel) - Wednesday, 02 February 2011, 12:27 GMT+2
I tested with very long AAC (not AAC-HE) files with playtime of 1-2h and did not encounter any problems with playback, seek and resume anymore. There is a delay before a long song starts to play. This is caused by parsing the up to 2 MB long header.
Comment by Andree Buschmann (Buschel) - Thursday, 03 February 2011, 10:05 GMT+2
Info from IRC chat: The residual isse with m4a files is that the different seek-tables that are needed by the codec are becoming quite large on files with long duration. A more sophisticated way to create tables with reduced size has not been implemented yet.
Comment by Andree Buschmann (Buschel) - Friday, 04 February 2011, 16:56 GMT+2
Kind of proof-of-concept that smaller table sizes can be achieved in general.

The attached patch reduces the size of chunk_offset[] by a factor of 8. This works for most of my files. All files with num_sample_to_chunks>2 do not work. This patch does not check for this and will -- in this state -- not play such file.

Main problem for future work in this area is that our codecs (e.g. aac.c) call get_sample_offset() _each_ frame. When this is removed most files do not play anymore... As long as we need this we cannot drop the precision of the array that is interesting for memory consuption (sample_byte_size[]).

Note: The patch has still several printf()'s in there.
   m4a_seek_opt_v02.patch (6.6 KiB) 
 apps/codecs/aac.c          |    2 +-
 apps/codecs/libm4a/m4a.c   |   35 +++++++++++++++++++++++++----------
 apps/codecs/libm4a/demux.c |   25 ++++++++++++++++++++++++-
 apps/codecs/libm4a/m4a.h   |    4 +++-
 4 files changed, 53 insertions(+), 13 deletions(-)

Comment by Magnus Holmgren (learman) - Saturday, 05 March 2011, 10:36 GMT+2
Most of the time it should be possible to replace get_sample_offset() with increasing the read offset by frame_info.bytesconsumed after each frame. In some files, these two don't yield the same result though, as chunks don't always immediately follow each other.

One way to solve this could be to covert the seek tables to "byte-offset, num-frames" entries (and time offset, if needed), making sure that no entry "spans" a chunk gap. Also, it'd be nice to do this during buffering. Haven't done much coding along these lines though, as dealing with the MP4 seek tables isn't particularly fun, IMO. :)
Comment by Andree Buschmann (Buschel) - Saturday, 05 March 2011, 11:27 GMT+2
Is there a way to easily detect those frames where (read offset + frame_info.bytesconsumed) != get_sample_offset()? Can we extract this from the seek tables?

Idea: Only save special entries for those frames. The entry contains "frame idx" and "next byte pos" (or "bytes to skip"). Instead of reconstructing the byte offset with get_sample_offset() each time we can then check whether we can simply use frame_info.bytesconsumed or need to read from the "special case" array. This check is done via checking if the current frame has an entry in this table. I would expect that such table would be much smaller than the current ones. Such table would need uint32 for "frame idx" and uint16 for "bytes to skip" per entry.
Disadvantage: This might require searching for a match of current frame to the "frame idx" over large arrays in case we need such table.
Comment by Magnus Holmgren (learman) - Saturday, 05 March 2011, 13:33 GMT+2
Depends on what you mean by easily. :) It should be easy enough to extract from the seek tables, e.g., by iterating over each frame size and at the end of each chunk check if the next chunk starts immediately afterwards or not.

My idea is to decode numframes frames using frameinfo.bytesconsumed to advance the read offset, then seek to the next offset if needed. No special cases or long arrays to check during decode. Also, the entire frame size table wouldn't be needed then (usually the biggest table). 5-10,000 seek entries should give good enough seek resolution, I think.
Comment by sideral (sideral) - Monday, 07 March 2011, 16:53 GMT+2
I'm closing  FS#11991  as a duplicate. In it I reported that I could not play back the following files on my ClipV2 (with various revisions between r28308 and r29506):
http://www.club-autonomic.com/podcast/autonomic_podcast_layer_11.m4a
http://www.club-autonomic.com/podcast/autonomic_podcast_layer_12.m4a

Buschel and saratoga commented that these files' TOC/seek table won't fit into the decoder's RAM, and Buschel pointed to this task for a discussion of how to fix this problem.
Comment by Richard Ash (RichardAsh1981) - Monday, 04 April 2011, 22:55 GMT+2
For anyone who is looking for a work-around now that BBC radio only comes as AAC or WMA files through get_iplayer, you can get long programs that won't play (like Radio 3's Late Junction) to work by splitting them into shorter sections (with smaller seek tables) using MP4Box (which you'll need to use anyway). On my Samsung YH-925, 1 hour works so I can do

ffmpeg -i download.flv -vn -acodec copy raw.aac
MP4Box -new -add raw.aac -split 3600 outfile.mp4

Which generates outfile_001.mp4, outfile_002.mp4 and so on until it runs out of file. Said files play nicely (and gaplessly!) in Rockbox. If they are too big for lower-memory players, the 3600 is the file length in seconds, which could be lowered down to 1800 for 30 mins and so on.

Hope this helps if getting the playback fixed properly is going to take a while!
For the record this is MP4Box from GPac 0.4.5-r2 and FFmpeg 0.6_p25767, both from Gentoo on amd64
Comment by Andree Buschmann (Buschel) - Friday, 15 April 2011, 21:25 GMT+2
The attached patch is another proof-of-concept. It shows that the main decoding loop can be implemented without the need to call get_sample_offset() each frame. This is done via building up a lookup-table (called m4a_sync_tab[][]) which holds the neccessary data. Advantage is that this look-up table needs *far* less entries than sample_byte_size[].
For now the size of m4a_sync_tab[][] is fixed and the build-up is done after allocating and reading sample_byte_size[]. To be able to reduce the memory consumption it is needed to build up m4a_sync_tab[][] *instead* sample_byte_size[]. For the seek functionality we are then able to allocate and build up sample_byte_size[] with a far smaller resolution (e.g. not for each frame, but for each 10th frame).
   m4a_buffer_advance_v07.patch (6.3 KiB) 
 apps/codecs/aac.c        |   59 +++++++++++++++--------------------------------
 apps/codecs/libm4a/m4a.c |   54 +++++++++++++++++++++++++++++++++++++++++++
 apps/codecs/libm4a/m4a.h |    6 ++++
 3 files changed, 79 insertions(+), 40 deletions(-)

Comment by Andree Buschmann (Buschel) - Friday, 15 April 2011, 23:34 GMT+2
This v08 version implements another way of building up the lookup table m4a_sync_tab[][]. This implementation does not need the memory consuming sample_byte_size[] at all and is a lot faster. On the other hand the lookup table itself is bigger (~1/10 entries compared to sample_byte_size[]).
   m4a_buffer_advance_v08.patch (8 KiB) 
 apps/codecs/aac.c        |   59 +++++++++--------------------
 apps/codecs/libm4a/m4a.c |   94 +++++++++++++++++++++++++++++++++++++++++++++++
 apps/codecs/libm4a/m4a.h |    6 +++
 3 files changed, 119 insertions(+), 40 deletions(-)

Comment by Andree Buschmann (Buschel) - Saturday, 16 April 2011, 15:51 GMT+2
The v10 version does not need sample_byte_size[] anymore for playback, seek and resume. Next step is to also remove the need for chunk_offset[] which is only used to build up the lookup-table.

Savings for some of my samples here (measured is malloc-usage in Byte):
sample / svn / v10
autonomic / 826.236 / 560.452
nrj / 118.260 / 116.140
radiozu / 142.484 / 139.644
Schubdüse / 700.312 / 190.108 (!)
twip_002 / 338.948 / 224.348
twip_003 / 561.952 / 370.880
   m4a_buffer_advance_v10.patch (15.9 KiB) 
 apps/codecs/aac.c          |   59 +++---------
 apps/codecs/alac.c         |    2 
 apps/codecs/libm4a/m4a.c   |  219 ++++++++++++++++++++++++++++++++++++++++++++-
 apps/codecs/libm4a/demux.c |   26 +++--
 apps/codecs/libm4a/m4a.h   |   18 +++
 5 files changed, 274 insertions(+), 50 deletions(-)

Comment by Andree Buschmann (Buschel) - Saturday, 16 April 2011, 16:37 GMT+2
v11 version builds the lookup table when reading chunk_offset w/o allocating or saving chunk_offset[].

The savings for the samples files from above:
sample / svn / v11
autonomic / 826.236 / 450.360 (!) -> duration 1:34h
nrj / 118.260 / 115.528 -> duration 1:11m
radiozu / 142.484 / 138.692 -> duration 1:50m
Schubdüse / 700.312 / 130.080 (!) -> duration 2:02h
twip_002 / 338.948 / 181.480 (!) -> duration 38:45m
twip_003 / 561.952 / 298.704 (!) -> duration 1:05h

As expected the files with long duration will benefit a lot.
   m4a_buffer_advance_v11.patch (16.5 KiB) 
 apps/codecs/aac.c          |   59 ++++---------
 apps/codecs/alac.c         |    2 
 apps/codecs/libm4a/m4a.c   |  194 ++++++++++++++++++++++++++++++++++++++++++++-
 apps/codecs/libm4a/demux.c |   64 ++++++++++++--
 apps/codecs/libm4a/m4a.h   |   20 ++++
 5 files changed, 284 insertions(+), 55 deletions(-)

Comment by Andree Buschmann (Buschel) - Saturday, 16 April 2011, 21:00 GMT+2
Now alac works again. I will submit the alac changes independently as those are just a clean-up of the alac-decoder and are not directly connected to the m4a changes.
   m4a_buffer_advance_v12.patch (18.4 KiB) 
 apps/codecs/aac.c            |   49 +---------
 apps/codecs/alac.c           |   21 +---
 apps/codecs/libalac/alac.c   |    9 +
 apps/codecs/libalac/decomp.h |    1 
 apps/codecs/libm4a/m4a.c     |  194 ++++++++++++++++++++++++++++++++++++++++++-
 apps/codecs/libm4a/demux.c   |   64 +++++++++++---
 apps/codecs/libm4a/m4a.h     |   19 +++-
 7 files changed, 286 insertions(+), 71 deletions(-)

Comment by Andree Buschmann (Buschel) - Saturday, 16 April 2011, 22:00 GMT+2
Further clean-up after submitting changes for alac and aac decoder. No functional changes.

Next steps:
- Speed up building up the lookup table.
   m4a_buffer_advance_v14.patch (14.5 KiB) 
 apps/codecs/aac.c          |    4 
 apps/codecs/alac.c         |    4 
 apps/codecs/libm4a/m4a.c   |  210 +++++++--------------------------------------
 apps/codecs/libm4a/demux.c |   65 +++++++------
 apps/codecs/libm4a/m4a.h   |   13 ++
 5 files changed, 87 insertions(+), 209 deletions(-)

Comment by Andree Buschmann (Buschel) - Saturday, 16 April 2011, 23:17 GMT+2
Much faster now, internal loops were reduced by a factor of >10000 for long files.
   m4a_buffer_advance_v16.patch (16.1 KiB) 
 apps/codecs/aac.c          |    4 
 apps/codecs/alac.c         |    4 
 apps/codecs/libm4a/m4a.c   |  231 +++++++++------------------------------------
 apps/codecs/libm4a/demux.c |   74 ++++++++------
 apps/codecs/libm4a/m4a.h   |   13 +-
 5 files changed, 107 insertions(+), 219 deletions(-)

Comment by Andree Buschmann (Buschel) - Sunday, 17 April 2011, 12:43 GMT+2
This version removes the need to read the sample_byte_size[] data. The patches before needed this to gather the maximum required buffer size. For libfaad the maximum allowed buffer size is 2048 byte, for alac I am not sure what value to use... As a first approach I defined the maximum alac buffer 32768 -- this works for my few files. As a results the loading times of large songs go down by several seconds on my iPod Video 30GB. The following measurements shows the seconds from selecting a new file until the first sound is played.

sample / svn / v16 / v17
autonomic / 8s / 7s / 5s
Schubdüse / 6s / 6s / 4s
twip_002 / 4s / 4s / 3s
twip_003 / 5s / 6s / 4s

I might submit the changes in aac/alac decoder independently.
   m4a_buffer_advance_v17.patch (18 KiB) 
 apps/codecs/libfaad/bits.c |    4 
 apps/codecs/aac.c          |   14 ++
 apps/codecs/alac.c         |    8 +
 apps/codecs/libm4a/m4a.c   |  231 +++++++++------------------------------------
 apps/codecs/libm4a/demux.c |   83 ++++++++--------
 apps/codecs/libm4a/m4a.h   |   12 +-
 6 files changed, 120 insertions(+), 232 deletions(-)

Comment by sideral (sideral) - Monday, 18 April 2011, 00:27 GMT+2
The two files I reported (autonomic_podcast_layer_{11,12}.m4a) work fine with patch v17; thanks Buschel! In addition to playback, I also tried some skipping, spooling, and mid-track resuming, and they all worked fine without any noticeable imprecision.

I'm not sure what you mean with your 5 s measurement from select to playback – on my ClipV2 the files start almost instantly.
Comment by Andree Buschmann (Buschel) - Monday, 18 April 2011, 07:51 GMT+2
Thanks for testing, and good to hear those files play now :o)

The 5s were measured on my iPod Video. It needs ~2-3s for disc spinup and has a slower CPU. On Your ClipV2 I would expect to only have max. 1-2s before playback starts.

Loading...