Rockbox

The problem is gone with the latest Opus release:
https://people.xiph.org/~jm/opus/opus-1.3/

Tested with 32 and 48 kBit.

I am still experiencing this problem on Clip+ with the latest firmware 4ed5727

*PANIC*
Stkov codec
pc:30067390 sp:30803a A: 3008f734

It happens when I seek to the middle of this file, about 40 minutes in (24Kbit ogg opus): http://discogenie.dyndns.org/Cloudsdale/Music/The%20Adventure%20Zone/The%20Adventure%20Zone%20-%20Amnesty/Justin,%20Travis,%20Clint%20and%20Griffin%20M%20-%2026%20-%20Amnesty%20Arc%204%20Part%205.opus

In the meantime is there a last known good version of the firmware where this worked? I had another Clip+ about 2-3 years ago that I never saw any errors on.
I’m listening to speeches and podcasts constantly and this is a major pain to have to tiptoe around this error, so if this doesn’t get fixed I will probably do some bisection to see if there’s a firmware version where this doesn’t happen.

What opusenv version have you used for creating that file?

opusenc

ffmpeg-20190107-038d291-win64-static.
Hard to tell exactly what version of libopus is used in there (libavcodec is version 58.43.100…), but the release is dated 2.5 months after opus 1.3 so I can only guess that it’s up-to-date.

I can try re-encoding the file with latest opus 1.3.1 to see if that makes a difference though.

Reencoding the file with the latest opusenc 1.3.1 doesn’t make any difference.

Hmmm. I just tried flashing different versions of the firmware to see if I can identify the regression (bisecting is actually really difficult because I can’t effectively browse all of the dev build history, but I digress…)
I couldn’t repro the problem on 3.13 firmware, but I can repro with 3.14 and the latest dev version.

One strange thing is that when I restored my original backup afterwards (to the latest dev 4ed5727), I could still repro the panic but it was much rarer. All I could think of is that if this is an out-of-memory error because of stack space or something, maybe some setting changed (database caching? max playlist size?) that affected background memory use.

I was hoping I could just tell you “here is the commit that caused the regression” but for now that’s all the info I have.

Can someone test this with the latest dev versions?

I am going to test it today …

I tested with clip+ dev build 190810 and the opus files I posted above.
No problems found on resume playback and launching bookmarks.

However, on the second file (48kbps) I can trigger a stack overflow by seeking in the file and releasing the > button.

And again, encoding the flac file with libopus 1.3, libopusenc 0.2.1 @48kbps also that problem is gone.

Yep, I can still repro. Sansa clip+, build ID 3b75f98, 24Kbps .opus file encoded with libopus 1.3. Seeking about 45 minutes into a 1.5 hour file triggers a panic:
Stkov codec pc:3007c73c sp:30803a A: 30094838

john or logan can you get me that file so I can do some testing?

Sorry, the link above doesn’t work anymore since I keep moving things around. This one should work:
http://discogenie.dyndns.org/Cloudsdale/Podcasts/The%20Adventure%20Zone/The%20Adventure%20Zone%20-%20Amnesty/Justin,%20Travis,%20Clint%20and%20Griffin%20M%20-%2026%20-%20Amnesty%20Arc%204%20Part%205.opus

I used the file linked above: http://www.mediafire.com/file/bt0llwlyaeucxl7/obama_speech.48.opus

Good news everyone I was able to reproduce the error
bad news is that I haven’t figured out how to fix the issue yet

followed it here:
https://github.com/Rockbox/rockbox/blob/master/lib/rbcodec/codecs/opus.c#L502 ci→pcmbuf_insert(&output[skip * header.channels], NULL, ret);

https://github.com/Rockbox/rockbox/blob/master/apps/codec_thread.c#L223

and then lo and behold I find this: https://github.com/Rockbox/rockbox/blob/master/apps/codec_thread.c#L94 /* Workaround stack overflow in opus codec on highmem devices (see  FS#13060 ). */
#if !defined(CPU_COLDFIRE) && (MEMORYSIZE >= 8) && defined(IRAMSIZE) && IRAMSIZE > (32 * 1024)
#define WORKAROUND_FS13060 0×800 #else
#define WORKAROUND_FS13060 0
#endif

https://www.rockbox.org/tracker/task/13060

So a fix had already been submitted and you’ve found a case where it still occurs so the search continues..

ok try this clip+ build
http://www.mediafire.com/file/lism0bn3edhi8wh/Clip%252BOpus_8-12-19rockbox-full.zip

This fixes it for me hopefully you as well
http://gerrit.rockbox.org/r/#/c/2231/ (build only includes the above patch)

speachy also has a patch that moves some opus structures off the stack that may help
http://gerrit.rockbox.org/r/#/c/1932/

Hallelujah, that fixes it for me! I went seeking back and forth through a dozen opus files and couldn’t repro it once.

It seems like a strange fix though - I would have thought that resetting the decoder state wouldn’t actually free any memory, only zero out the buffers that were already in place. Still, I can’t argue with the results v0v.

You really don’t know how thankful I am for this. I was hitting this bug on a near-daily basis and it was the one thing keeping me from fully enjoying Rockbox. I owe you a coffee or something.

At first I thought it might be the effect of disabling optimizations for ARM since one of the changelogs mentions
‘Resetting the encoder or decoder state with OPUS_RESET_STATE would disable some run-time selected architecture-specific optimizations;’ but disabling optimizations did not stop the stackoverflow

OPUS_RESET_STATE removes all the previous info opus uses to optimize the audio so I figure there might be some loss of quality
but since preskip is still intact I figure it is about the same as streaming opus and I honestly hear no extra artifacts (not to say there aren’t any…)
this will still need some testing and a bit of error handling too before i push it and close this FS

Asking about the consequences of OPUS_RESET_STATE in the #opus IRC channel it sounds like its a good idea to reset the decoder after a seek anyways

mark4o> The packets overlap and may reuse state set by other recent packets, so if you seek to a different position,

      resetting the state helps to ensure that the subsequent packets won't use the state set by the unrelated packets that were processed before the seek

I did some more testing on the RESET_STATE patch. I encoded a 2-hour long file at various bitrates and modes, from 16Kbps - 64Kbps and with flags such as FORCE_MODE_SILK, FORCE_MODE_HYBRID, to try and produce files that would require varying complexity for the decoder to process. Seeking around through all of them, bookmarking and restarting, playing and pausing - none of them could trigger the stack overflow.

I still can’t explain how resetting the state fixes the issue, but I agree that it’s definitely best to do the reset after seeking, along with the 80ms preroll and the other stuff that Mark mentioned in IRC

we already do the preroll its a bit convoluted but it’s there so yeah we’ll call this fixed, going to close this as its been pushed to the dev versions earlier today