Comparison of Available Decoders Across Targets

Things that would help this process along

• We need to choose a Rockbox approved testfile, encode it in every (Rockbox) decodable format, zip it into a huge archive and make it available for download.

Description

• This test stemmed from this forum post by Senab and spawned a curiosity about how different decoders perform across different targets.
• Example: How well does Wavpack decode on a Coldfire 5249 (iRiver H1x0) vs a PP5020 dual ARM7 (4G iPod ).
• To do this we need a standardized set of test files, a standard testing procedure and a few volunteers with different targets willing to do the test.

Requirements

Note: These instructions are Windows only (for now).

• Create a directory to save files to. I called mine "codec_test".
• Download the sample song here to your working directory and rename it to "sample.flac".
• Download the encoder pack here which contains CLI encoders, a debug menu patch and a batch file for converting the sample song into various (supported) formats.
• Download the (free) Nero AAC encoders here
• Download and install Windows Media Encoder 9 from here (required windows validation)

Generating Sample Set

• Unzip the entire contents of "codec_pack.zip" to your working directory. In my case "codec_test".
• Unzip the "neroaacenc.exe" file from "NeroDigitalAudio.zip" into the "codecs" directory.
• Make sure you have renamed "fishbone1999-05-14d2t06.flac" to "sample.flac" and have put it in the directory. (any file named sample.wav or sample.flac can be used)
• Double-click the "encode.bat" file. Press "8" (encode all) and hit "enter". Once it's done (it will take some time) type "9" and "enter" to exit.
• This produces a subfolder called "files" with 32 encodings of the sample WAV. (The FLAC was first decoded to WAV)
• Copy the "files" folder to your player.

Note: These files are VBR. (where applicable)

Compile including the test_codec plugin

• In apps/plugins/SOURCES include a line at the end of the "CONFIG_CODEC == SWCODEC" section with "test_codec.c" in it. (no quotes)
• Compile and install. (If you don't know what all of this is about see SimpleGuideToCompiling)
• After install you might want to reset (or clear) all of your settings!

Method

• File Browse to the "files" folder and long-select the first sample file.
• Scroll down to "Open With..." and select test_codec. Select "Speed Test Folder".
• Let the plugin do it's thing. Once it's done you will have a file called "test_codec_log_XX.txt" in the root of your player.

Note: We're missing Speex, ALAC (Chronon: We have a single data point now) and APE in the batch file.

Please post your results right here including the target name and the rockbox revision.

Results:

Stripwax: Done on my ipod 5g 60GB on 2007.07.08:

Chronon: Done on my Gigabeat F40 on 2007.08.01

• I included ALAC [enc. w/ iTunes] and HE-AAC [enc. w/ Nero] in this run also.

przemhb: Done on my iriver H10 5GB on 2007.08.24

• I included HE-AAC and HEv2-AAC (enc. with Nero)
• Update (2007.08.29): APE results added

• (1) There is only one file encoded as AAC HEv2 due to continuous load error during the test.
• (2) AAC HEv2 file (encoded with Nero "-hev2" switch) was decoded over 3 times faster then AAC HE files.

TimPickersgill: Done on my H140 with r15250-071021

• I included APE in this run.

ThomasMartitz: Sansa e260, on 3rd of October 2008; r18698; most right column contains comparison to the previous test by MichaelAndrew on 30th of December 2007
*-- ThomasMartitz - 03 Oct 2008

DavideGentile : Done on my iRiver H1x0 on 2008.03.03

MatthewHaun : Done on my iRiver H3x0 on 2008.07.17

DavideGentile : Done on my iRiver H1x0 on 2008.08.21