Rockbox

Tasklist

FS#12632 - MP3 metadata symbols not appearing correctly on the WPS display screen

Attached to Project: Rockbox
Opened by Terry Layton (Terry The Great) - Monday, 02 April 2012, 22:57 GMT
Last edited by Dominik Riebeling (bluebrother) - Tuesday, 03 April 2012, 18:47 GMT
Task Type Bugs
Category ID3 / meta data
Status Closed
Assigned To No-one
Operating System Sansa Fuze+
Severity Low
Priority Normal
Reported Version Release 3.11
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Note: this is a bug report for the Sansa Fuze v2 but this player type does not appear in the Player Type menu. This bug does also appear on the Sansa Fuze+


The characters ‘ (traditional left single quote), ’ (traditional right single quote or apostrophe), “ (traditional left double quote) and ” (traditional right double quote) from the metadata (song title, artist, album title) of any MP3 are each displayed as a box outlined in dots instead of the correct symbol on the WPS screen display. This occurs with all of the fonts and when any theme or codepage is being used. The original Sansa® Fuze™ firmware does not have this problem.

Hardware device: Sansa® Fuze™ v2
Original manufacturer’s firmware version: 02.03.33
Rockbox version: 3.11 (the current release)
Attachments to this report: 2 complete song files (MP3 & WMA, each under 2 megabytes in size)

Proof of complaint:
First, I avoided using ripping and tagging software downloaded from the internet because this can introduce further problems (such as a computer virus). Besides, I believe that Rockbox should be easy to use and that means using what the Windows 7 operating system already provides. In order to eliminate my computer as the source of the problem, the two files attached to this message, one an MP3 and the other a WMA, were produced on a friend’s Windows 7 machine using Windows Media Player 12 (WMP 12) to rip the tracks from a CD (WMP 12 was used twice, once to produce an MP3 file and the second time to produce a WMA file). In both cases, the metadata was downloaded by WMP 12 which inserted it into each file. I then opened Microsoft Word 2007 and using my favorite font Times New Roman I created the string: [‘’“”]. This string contains the traditional left and right single and double quote characters encased in square brackets. The appropriate song file was then located in the Windows 7 Music directory and the property dialogue box brought up by right clicking on the file and selecting the Properties menu option. Next the summary tab (Advanced option) was selected. The string was pasted into the appropriate location in a data field (song title, artist or album title) according to this scheme:
MP3: The string was pasted onto the end of the song title and artist name. A character called the Greek Tonos (a symbol that can be inserted into a Word 2007 document while using the Times New Roman font) was added to the end of the artist name field.
WMA: The string was pasted onto the end of the song title and artist name.

When these files appear in a WPS screen display (using any font or any theme or codepage) I get these results:
MP3: The traditional single and double quote characters in the song title are not displayed properly when the Rockbox firmware is utilized. Instead a box outlined in dots appears in each case. The traditional single and double quote characters in the artist name are displayed properly (along with the Greek Tonos character appended onto the end). All of the characters in the song title are properly displayed when using the original SanDisk Sansa firmware.
WMA: The traditional single and double quote characters in the song title and artist name are displayed properly when using the Rockbox firmware.

Further experiments showed that the bit rate selected did not matter. The results are exactly the same as I get after using my computer to produce the MP3 and WMA files. Also try deleting the Greek Tonos character in the MP3 metadata artist name field and see what happens. Admittedly, this was a somewhat elaborate experiment, but in my view the results show that there is indeed a problem with the way in which the Rockbox firmware displays the traditional left and right single and double quote characters from metadata on an MP3. It is also clearly obvious that the Rockbox firmware has access to the proper symbols to display.

A visual display of the problem I am encountering can be found at: www.terrythegreat.wordpress.com

Please display the attached files on your Sansa Fuze v2 before pronouncing whether or not there is a problem as described above.

Disclaimer: Any music file I provide to the Rockbox crew is for testing and evaluation purposes, not for public broadcast, as per the copyright laws.
This task depends upon

Closed by  Dominik Riebeling (bluebrother)
Tuesday, 03 April 2012, 18:47 GMT
Reason for closing:  Not a Bug
Additional comments about closing:  See comment.
Comment by Frank Gevaerts (fg) - Tuesday, 03 April 2012, 12:26 GMT
The tags in the mp3 file are in id3v2.3 format. This format allows text tags to be encoded in ISO-8859-1, UCS-2, or UTF-16.
In the mp3 file, the four characters after "Stay[" in the title are 0x91, 0x92, 0x93, and 0x94. These are not valid characters in any of the three allowed encodings, which means the file is incorrectly tagged (which is unfortunately rather common).

Rockbox does have some support to work around this kind of breakage by setting a different codepage. This particular file does work when setting that to at least CP1250 or CP1251.

It would probably be a good idea to add support for CP1252 too, since that seems to be a rather common encoding, but that doesn't change the fact that the reasons these strings do not display correctly is because the tags are not correct.

The wma file has no such issues because the asf format is much more clear (and strict) about the encoding to use in text tags than id3 has historically been, (it's always UTF-16), so tagging software is less likely to get it wrong.
Comment by Dominik Riebeling (bluebrother) - Tuesday, 03 April 2012, 18:47 GMT
Note: this is not a bug and therefore does not apply to any specific (R) category. It doesn't need a Player (R) Type (c) entry.

The characters ‘ (traditional left single quote), ’ (traditional right single quote or apostrophe), “ (traditional left double quote) and ” (traditional right double quote) are actually LEFT SINGLE QUOTATION MARK (U+2018), RIGHT SINGLE QUOTATION MARK (U+2019), LEFT DOUBLE QUOTATION MARK (U+201c) and RIGHT DOUBLE QUOTATION MARK (U+201d). They are not part of any allowed 8bit character set for ID3v2 data. ID3v2 only allows ISO-8859-1 as 8bit character set. A ID3v2 decoder that follows the standard needs to use ISO-8859-1 for 8bit characters. So does Rockbox. Since said characters are not used in ISO-8859-1 and unprintable characters in ISO-8859-15 (the successor of ISO-8859-1) it is not surprising that the characters don't show up as expected, since WMP (r) (tm) (p) obviously violates the specification by inserting characters out of a character set that is not defined for this kind of metadata.

The situation gets even worse: WMP (r) (tm) (p) does even insert those characters in different formats depending on the tag itself: for ID3v1 it uses cp1252, for ID3v2 it uses cp1252 for the TIT2 entry, but UTF16 for the TPE1 entry. This causes the latter to show up as expected since the tag is correct. The former, however, is broken since it doesn't use a valid encoding. The character GREEK TONOS (U+0384) is likely to trigger WMP using a proper encoding, since there is no 8bit charset that supports all those character.

As for posting copyrighted (c) music (m) for testing (t) and evaluating (e) the user is free (tm) to use a file (f) that's in the public domain or licensed (c) using a creative commons license (r). Which would be completely sufficient for a metadata related question (Q) since metadata does not rely to bitrate or content of the file.

www.terrythegreat.wordpress.com should use a tagging software that writes tags in a correct format instead of complaining about something that is clearly broken software, but not ours.

Complaint is pointless, user is using broken software, not a bug.

Loading...