This is the bug/patch tracker for Rockbox. Click here for more information.
Quick links: Bugs · Patches · Rockbox frontpage
FS#4984 - Fixed point WMA decoder lib
Attached to Project:
Rockbox
Opened by Paul Jones (marsdaddy) - Saturday, 01 April 2006, 11:50 GMT+2
Last edited by Dave Chapman (linuxstb) - Tuesday, 03 July 2007, 12:36 GMT+2
Opened by Paul Jones (marsdaddy) - Saturday, 01 April 2006, 11:50 GMT+2
Last edited by Dave Chapman (linuxstb) - Tuesday, 03 July 2007, 12:36 GMT+2
|
DetailsI have coded up a fixed point version of the WMA decoder from the ffmpeg project and have attached it here.
I would like it to be included in rockbox. Currently I am working on integrating this into the rockbox architecture (ie ASF format handling). I have tested it standalone on a number of WMA files without error though I think there are code paths that have not been exercised. Enjoy! |
This task depends upon
Closed by Dave Chapman (linuxstb)
Tuesday, 03 July 2007, 12:36 GMT+2
Reason for closing: Accepted
Additional comments about closing: Codec now in SVN
Tuesday, 03 July 2007, 12:36 GMT+2
Reason for closing: Accepted
Additional comments about closing: Codec now in SVN
For files taken from ffmpeg, you should keep the original (C) message from the original source files. As you've obviously made non-trivial changes to the files yourself, you could add an extra (C) line with your name indicating your changes.
But nice work so far. I have a couple of questions - do you have a test program (either PC-based or a Rockbox plugin) to test the decoder, and (assuming yes to the second part of the previous question), do you have any idea how fast the decoder is running on a real Rockbox device?
No, pinched the headers from elsewhere...
>But nice work so far. I have a couple of questions - do you have a test program (either PC-based or a Rockbox plugin) to test >the decoder, and (assuming yes to the second part of the previous question), do you have any idea how fast the decoder is >running on a real Rockbox device?
I have a PC based test bed yes, not updated it for a little while. No idea how fast it runs on a real device yet, functionality
first etc.
Also IMO, it will be much easier to measure the speed of the decoder and check the accuracy of the output if you wrote a "wma2wav" plugin that ran on the device instead of integrating it immediately into the codec system - there are no mechanisms in Rockbox to measure the speed of codecs or check their output. If the plugin is written in a similar style to the codec system (i.e. the plugin reads the WMA file into RAM, and pass the pointers to WMA data in this buffer to the WMA decode function), then porting it to the codec system is relatively trivial.
If you need any help in this respect, I would be happy to try to port your PC-based test program to a Rockbox plugin. I'm sure that there are others who would also be willing to try to optimise the decoder once it is running on real Rockbox devices.
Interesting - I hadn't considered this and will look into it. I'll have a crack at getting the plugin working. I'll let
you know if I need a hand.
I've ported the wma decoder from ffmpeg project using fixed point like you did before, and it runs well at 96MHz now.
But now I encounter a quality issue that it has noise of shock wave like "po po" sound when playing songs.
Because I don't have any technical document about wma algorithm, I don't have any idea about how to improve the quality.
Can any of you give me some hints or advice to do this job or just tell me where to find the related documents?
I'll be pleased to inform you all if I have progress.
Thanks for your kindly reply.
Henry
I've ported the wma decoder from ffmpeg project using fixed point like you did before, and it runs well at 96MHz now.
But now I encounter a quality issue that it has noise of shock wave like "po po" sound when playing songs.
Because I don't have any technical document about wma algorithm, I don't have any idea about how to improve the quality.
Can any of you give me some hints or advice to do this job or just tell me where to find the related documents?
I'll be pleased to inform you all if I have progress.
Thanks for your kindly reply.
Henry
As far as I'm aware there are no technical docs on WMA available. The ffmpeg effort was I believe reverse engineered.
My own work on the fixed point decoder has issues at the moment - if you would be happy to share your code with me, I would be happy
to run it in my test bed and together we may be able to sort this out.
Thanks.
Please both consider posting patches here with the current state of your code - there are a few people involved with Rockbox who would be able to help with any issues you're experiencing. But it's impossible for us to help without seeing the code.
Good luck to everybody. Let's make it.
Henry
What's the current status of your wma codec? Are you planning on submitting a patch for inclusion in CVS in the near future?
As you know, lots of people want to see WMA support in Rockbox, and there are also developers apart from yourself willing to work on it, including me - but we can't wait forever for you to release code. As it's been more than 5 months since you posted your first code, I'm at the stage of giving up waiting and starting a port myself - with the intention of committing to CVS as soon as I can so that others can start helping with the conversion to fixed-point and optimisation.
But obviously I don't want to waste time duplicating the work you've already done, so would be very happy instead to see a patch from you which we could then commit to CVS and finally get to a working state.
Regards,
Dave.
The simulator that this source builds is capable of opening a WMA and decoding it to an output file. For example, there is a wma file
named "in.wam" in the build-sim/archos which can be selected in the simulator and "x.wav" is produced from it. The WMA decoder is accessed
through a plugin rather than being an integrated decoder. Note that I had to get it to accept ".wam" files rather than ".wma" as there is
already an attempt to play ".wma" somewhere in the rockbox code.
Notes on the WMA decoder --
1) It's not (quite) complete - I recently switched to 64 bit fixed point from 32 and there is at least one issue - the "fixmul64" in fixedpoint.c
is inaccurate - temporarily, I have modified it to use floats internally to show that it is indeed this method that is causing the problem.
2) Even with the "float" usage in the 64 bit mul, the amplitude of the output varies over time. The file "in.wam" is a sine wave. Run the decoder
and look at "x.wav" -- you'll see what I mean.
3) The decoder isn't quick. There's plenty of room for optimisation but I haven't got that far.
4) Some of the code in wmadeci.c may look mad but that's because when doing the coldfire compilation there were numerous compiler related
errors. There are fudges in the code because of this.
I can see that the other decoders use 64 bit fixed maths in assembler and I think someone braver than me needs to swap this code over to using it.
I hope someone can work with this code to get it fast and fully working.
Any questions, I'll be happy to help.
I made a patch against current cvs with the conflicts still in place (because I might have made some mistakes fixing those) for people who want to give it a shot.
update: ok, I got sound working after all, it's just very very slow.
Attached is a patch which compiles with current CVS (in the sim only - targets complain about missing sin and cos functions). There are plenty of warnings, but it compiles and the wma2wav plugin works in the sim.
I haven't included all the changes in Paul's original patch (I ignored changes to add wma to the playback code), and I made a small number of minor changes to fix a few errors. I just tried to limit the changes to be enough to get wma2wav working.
I also changed the plugin to use the wma extension - you need to use the "open with..." option to make Rockbox open the file with the wma2wav plugin instead of attempting to play it.
Finally also disabled the debug output in wma2wav.c - this was creating a huge logfile and slowing down the decoding by an extraordinary amount.
make complains that libwma/wmadeci.c needs button-target.h for some reason.
Also, wma should be added to codecs in tools/configure and not to OUTPUT in apps/codecs/Makefile.
The first problem is solved by adding $(TARGET_INC) to CFLAGS in the Makefile.
I believe that as well as debug output I left in a temporary use of sin() and cos() - there are
fixed point versions of these functions in use and the sin() and cos() are just there again for
diagnostic/debug purposes. You'll see what I mean.
I won't be able to work on this again in the short term - my house burnt down :(
Enjoy!
I also made a small change on line 43 of wma.c to get rid of the compiling error it had. Now there are only warnings (but there are a lot).
wma2wav.c still doesn't compile.
wma.c: In function ‘wma_decode_data’:
wma.c:86: warning: overflow in implicit constant conversion
wma.c:87: warning: unused variable ‘bias’
wma.c:86: warning: unused variable ‘level’
wma.c:110: warning: label ‘error’ defined but not used
wma.c:58: warning: unused variable ‘bit_rate’
wma.c:57: warning: unused variable ‘flags’
wma.c:75: warning: ‘length’ is used uninitialized in this function
wma.c: In function ‘codec_main’:
wma.c:142: warning: passing argument 2 of ‘ci->configure’ makes integer from pointer without a cast
wma.c:143: warning: passing argument 2 of ‘ci->configure’ makes integer from pointer without a cast
wma.c:144: warning: passing argument 2 of ‘ci->configure’ makes integer from pointer without a cast
wma.c:155: warning: passing argument 2 of ‘ci->configure’ makes integer from pointer without a cast
CC wmadeci.c
wmadeci.c:78: warning: function declaration isn’t a prototype
wmadeci.c: In function ‘Debug64’:
wmadeci.c:303: warning: function returns address of local variable
wmadeci.c: In function ‘WmaInitForPlugin’:
wmadeci.c:705: warning: assignment from incompatible pointer type
wmadeci.c: In function ‘fft_inits’:
wmadeci.c:903: warning: assignment discards qualifiers from pointer target type
wmadeci.c:906: warning: assignment discards qualifiers from pointer target type
wmadeci.c:909: warning: assignment discards qualifiers from pointer target type
wmadeci.c:912: warning: assignment discards qualifiers from pointer target type
wmadeci.c:915: warning: assignment discards qualifiers from pointer target type
wmadeci.c:890: warning: unused variable ‘s2’
wmadeci.c:889: warning: unused variable ‘s1’
wmadeci.c:889: warning: unused variable ‘c1’
wmadeci.c:889: warning: unused variable ‘alpha’
wmadeci.c: In function ‘get_vlc’:
wmadeci.c:1184: warning: unused parameter ‘ln’
wmadeci.c: In function ‘wma_decode_block’:
wmadeci.c:2109: warning: initialization from incompatible pointer type
wmadeci.c:2109: warning: unused variable ‘tP’
wmadeci.c:2137: warning: unused variable ‘tmp’
wmadeci.c: In function ‘wma_decode_init_fixed’:
wmadeci.c:2504: warning: integer constant is too large for ‘long’ type
wmadeci.c:2743: warning: assignment discards qualifiers from pointer target type
wmadeci.c:2746: warning: assignment discards qualifiers from pointer target type
wmadeci.c:2749: warning: assignment discards qualifiers from pointer target type
wmadeci.c:2752: warning: assignment discards qualifiers from pointer target type
wmadeci.c:2755: warning: assignment discards qualifiers from pointer target type
wmadeci.c:2768: warning: assignment from incompatible pointer type
wmadeci.c:2773: warning: assignment from incompatible pointer type
wmadeci.c:2428: warning: unused variable ‘window’
wmadeci.c: In function ‘wma_decode_superframe’:
wmadeci.c:2820: warning: unused variable ‘name_cache’
wmadeci.c:2819: warning: unused variable ‘res’
wmadeci.c: At top level:
wmadeci.c:2965: warning: missing initializer
wmadeci.c:2965: warning: (near initialization for ‘wmav1i_decoder.capabilities’)
wmadeci.c:2978: warning: missing initializer
wmadeci.c:2978: warning: (near initialization for ‘wmav2i_decoder.capabilities’)
wmadeci.c:68: warning: ‘bno’ defined but not used
wmadeci.c:2396: warning: ‘Crc32’ defined but not used
wmadeci.c:2410: warning: ‘Crc16’ defined but not used
wmadeci.c:1388: warning: ‘firstTime’ defined but not used
CC asf.c
In file included from avcodec.h:14,
from avformat.h:19,
from asf.c:22:
common.h: In function ‘put_bits’:
common.h:399: warning: implicit declaration of function ‘bswap_32’
asf.c: At top level:
asf.c:797: warning: function declaration isn’t a prototype
asf.c: In function ‘tag_recode’:
asf.c:810: warning: unused variable ‘length’
asf.c:809: warning: unused variable ‘len1’
asf.c:808: warning: unused variable ‘ansaptr’
asf.c:807: warning: unused variable ‘ansbptr’
asf.c:803: warning: unused variable ‘result’
asf.c:801: warning: unused parameter ‘before’
asf.c: In function ‘asf_read_header’:
asf.c:1062: warning: implicit declaration of function ‘atoi’
asf.c:878: warning: unused parameter ‘ap’
asf.c: In function ‘asf_read_seek’:
asf.c:1491: warning: comparison between signed and unsigned
asf.c:1494: warning: comparison between signed and unsigned
asf.c:1496: warning: comparison between signed and unsigned
asf.c:1511: warning: unused variable ‘bigPtsMax’
asf.c: At top level:
asf.c:1569: warning: missing initializer
asf.c:1569: warning: (near initialization for ‘asf_iformat.flags’)
asf.c:797: warning: ‘getlocale’ defined but not used
CC futils.c
In file included from avcodec.h:14,
from avformat.h:19,
from futils.c:19:
common.h: In function ‘put_bits’:
common.h:399: warning: implicit declaration of function ‘bswap_32’
futils.c:26:1: warning: "isdigit" redefined
In file included from common.h:32,
from avcodec.h:14,
from avformat.h:19,
from futils.c:19:
/home/crispy/rockbox/firmware/include/ctype.h:49:1: warning: this is the location of the previous definition
futils.c:27:1: warning: "isalpha" redefined
/home/crispy/rockbox/firmware/include/ctype.h:46:1: warning: this is the location of the previous definition
futils.c:28:1: warning: "isalnum" redefined
/home/crispy/rockbox/firmware/include/ctype.h:53:1: warning: this is the location of the previous definition
futils.c: In function ‘av_open_input_file’:
futils.c:405: warning: implicit declaration of function ‘codec_malloc’
futils.c:405: warning: assignment makes pointer from integer without a cast
futils.c:467: warning: implicit declaration of function ‘codec_free’
futils.c: In function ‘convert_timestamp_units’:
futils.c:509: warning: comparison between signed and unsigned
futils.c:512: warning: comparison between signed and unsigned
futils.c: In function ‘compute_frame_duration’:
futils.c:585: warning: unused parameter ‘s’
futils.c: In function ‘compute_pkt_fields’:
futils.c:643: warning: comparison between signed and unsigned
futils.c:659: warning: comparison between signed and unsigned
futils.c: In function ‘av_has_timings’:
futils.c:1056: warning: comparison between signed and unsigned
futils.c:1057: warning: comparison between signed and unsigned
futils.c: In function ‘av_update_stream_timings’:
futils.c:1075: warning: comparison between signed and unsigned
futils.c:1078: warning: comparison between signed and unsigned
futils.c: In function ‘fill_all_stream_timings’:
futils.c:1106: warning: comparison between signed and unsigned
futils.c: In function ‘av_estimate_timings_from_bit_rate’:
futils.c:1130: warning: comparison between signed and unsigned
futils.c:1138: warning: comparison between signed and unsigned
futils.c:1139: warning: comparison between signed and unsigned
futils.c: In function ‘try_decode_frame’:
futils.c:1199: warning: unused variable ‘picture’
futils.c: In function ‘dump_format’:
futils.c:1551: warning: unused parameter ‘ic’
futils.c:1552: warning: unused parameter ‘index’
futils.c:1553: warning: unused parameter ‘url’
futils.c:1554: warning: unused parameter ‘is_output’
futils.c: In function ‘parse_image_size’:
futils.c:1595: warning: implicit declaration of function ‘strtol’
futils.c: In function ‘find_info_tag’:
futils.c:1620: warning: comparison between signed and unsigned
futils.c: In function ‘url_split’:
futils.c:1749: warning: implicit declaration of function ‘strtoul’
wma2wav.c:395: error: conflicting types for ‘creat’
/home/crispy/rockbox/firmware/include/file.h:72: error: previous declaration of ‘creat’ was here
wma2wav.c: In function ‘creat’:
wma2wav.c:396: error: too many arguments to function ‘rb->creat’
wma2wav.c: At top level:
wma2wav.c:405: warning: function declaration isn’t a prototype
wma2wav.c:410: warning: function declaration isn’t a prototype
wma2wav.c: In function ‘wma2wav’:
wma2wav.c:439: warning: implicit declaration of function ‘WmaInitForPlugin’
wma2wav.c:439: warning: unused variable ‘res’
wma2wav.c:419: warning: unused variable ‘save_a’
wma2wav.c:418: warning: unused variable ‘samples_remaining’
wma2wav.c:418: warning: unused variable ‘total_samples’
wma2wav.c:417: warning: unused variable ‘total_bytes_written’
wma2wav.c:417: warning: unused variable ‘total_bytes_read’
wma2wav.c:416: warning: unused variable ‘num_chans’
wma2wav.c:416: warning: unused variable ‘out_fd’
wma2wav.c:416: warning: unused variable ‘in_fd’
wma2wav.c: At top level:
wma2wav.c:478: warning: function declaration isn’t a prototype
wma2wav.c: In function ‘tag_recode’:
wma2wav.c:503: warning: implicit declaration of function ‘strcpy’
wma2wav.c: At top level:
wma2wav.c:498: warning: unused parameter ‘codep’
wma2wav.c: In function ‘write_prelim_header’:
wma2wav.c:516: warning: implicit declaration of function ‘memcpy’
wma2wav.c: At top level:
wma2wav.c:558: warning: return type defaults to ‘int’
wma2wav.c: In function ‘BufferToLE’:
wma2wav.c:559: warning: initialization from incompatible pointer type
wma2wav.c:568: warning: control reaches end of non-void function
wma2wav.c: In function ‘audio_decode’:
wma2wav.c:586: warning: implicit declaration of function ‘codec_malloc’
wma2wav.c:586: warning: assignment makes pointer from integer without a cast
wma2wav.c:581: warning: unused variable ‘tmp’
Here's the full set if it helps any developers. Seems to break the compile after the rocks
1) Fixed overflow bug that causes MDCT blocks with very large gain fields to overflow and give garbage out
2) Removed a lot of the ffmpeg baggage. Plugin now invokes a number of the decode routines directly. Plugin no long compiles a library.
3) Fixed a few issues that prevented clean compiling on linux, but probably broke cygwin. If it doesn't compile in cygwin, check comments in dtypes.h.
4) Added a debug mode. Add "dbprintf('a'...) to only print messages when 'a' is supplied as a command line switch (example: add 'm' and you'll get the MDCT coefficients printed).
On the other hand, the code is literally with my dbprintf statements and commented out junk. Line 151-152 of wmatest2 also has a commented out command that lets you seek and decode short blocks of the file (useful for debugging).
A rather serious bug still occurs in some samples that leads to nasty clipping on short transients. Will look into that now.
To use, type make, then './wmaplay file.wma file.wav [debug char]'
1) Fixed overflow bug that causes MDCT blocks with very large gain fields to overflow and give garbage out
2) Removed a lot of the ffmpeg baggage. Plugin now invokes a number of the decode routines directly. Plugin no long compiles a library.
3) Fixed a few issues that prevented clean compiling on linux, but probably broke cygwin. If it doesn't compile in cygwin, check comments in dtypes.h.
4) Added a debug mode. Add "dbprintf('a'...) to only print messages when 'a' is supplied as a command line switch (example: add 'm' and you'll get the MDCT coefficients printed).
On the other hand, the code is literally with my dbprintf statements and commented out junk. Line 151-152 of wmatest2 also has a commented out command that lets you seek and decode short blocks of the file (useful for debugging).
A rather serious bug still occurs in some samples that leads to nasty clipping on short transients. Will look into that now.
To use, type make, then './wmaplay file.wma file.wav [debug char]'
Fix overflows in fft by adjusting precision as needed.
Use more accurate COS and SIN estimate.
Properly decodes my 128k test file (error signal is at -67dB), performance should be identical to the original.
But isn't wma now in svn rockbox? Is it diffirent from this patch?
Nope.
>>Is it diffirent from this patch?
As far as I know, this is the only fixed point open source WMA decoder in existence.
In addition to the plugin API, I made the following changes:
Removed all calls to malloc from the wmadecoder itself, and some from the ASF parser.
Removed all calls to realloc.
Added a simple (and probably broken) malloc implementation so that the ASF parser can run in Rockbox
Removed some code that wasn't needed.
Fixed a lot of warnings.
I suspect the reason it doesn't run on any devices yet is because my malloc is broken, but thats just a guess.
To install, unzip into your plugins directory.
/apps/codecs/
libfaad (faad_malloc, see common.c)
Tremor (ogg_malloc, ogg_tmpmalloc, see oggmalloc.c)
/apps/plugins/
zxbox (my_malloc)
chessbox (pl_malloc)
mpegplayer (mpeg2_malloc)
midi (my_malloc)
doom (Z_Malloc, see z_bmalloc.[ch])
rockboy (my_malloc)
dict (pl_malloc)
This doesn't seem right to me. Shouldn't it be a ".c" file?
Are you using nono standard zip files?
Maybe that's the problem?
BTW, the archive also contains a 512KB .wma sample...
End-of-central-directory signature not found. Either this file is not a Zip file, or it constitutes one disk of a multi-part Zip file.
Got it! Thanks.
It should be realtime on Gigabeat and PP targets, but is currently extremely slow (around 20% realtime) on Coldfire.