FS#12096 - Metadata item limits too tight in practice with UTF-16 encoded strings

Attached to Project: Rockbox
Opened by Stephan Grossklass (keyb_gr) - Wednesday, 04 May 2011, 15:26 GMT
Last edited by MichaelGiacomelli (saratoga) - Wednesday, 04 May 2011, 17:30 GMT
Task Type Bugs
Category ID3 / meta data
Status Closed
Assigned To No-one
Operating System All players
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No


As of r29174 (see  FS#11875 ), the amount of data read per metadata item is limited to 60 bytes on low-memory targets and 120 bytes on others. While such a limitation does make sense, IMO the values chosen are not sufficiently universal, especially when UTF-16 encoded strings come into play that need 2 bytes per character. Even on a target with ample memory (e.g. Clip+), such strings will be cut off after about 60 characters.

With some genres of music, it is not unusual to find track and album titles of 80+ characters, which will thus be truncated when using ID3v2.3/2.4 tags with UTF-16. Here's an example (copied from Foobar2k):

Artist Name : Antonín Dvořák
Track Title : Symphony No. 9 in E minor, op. 95 "From the New World" - III. Scherzo: Molto vivace
Album Title : Symphonies Nos. 8, 9 (Berliner Philharmoniker feat. conductor: Rafael Kubelik)
Date : 1966
Genre : Classical
Composer : Antonín Dvořák
Track Number : 07
<CONDUCTOR> : Rafael Kubelik

On a Clip+, the track title is displayed as 'Symphony No. 9 in E minor, op. 95 "From the New World" - II', cut off in the middle of the Roman numerals.

The intent of the per-item limit was keeping things like rogue comments from filling up the buffer. It should not interfere with normal, "well-behaved" metadata if at all possible. A normal user cannot be expected to compile Rockbox to change these limits.

I'd suggest values of about 90 for low-memory targets and 240 for others. On the latter, this would still allow 3 very lengthy strings in the 900 byte buffer with room to spare.

The above example would remain critical on low-memory targets, although the strings could all be fitted when resorting to ID3v2.4 tags with UTF-8.

A metadata.h patch according to my suggestion is attached.
This task depends upon

Closed by  MichaelGiacomelli (saratoga)
Wednesday, 04 May 2011, 17:30 GMT
Reason for closing:  Accepted
Additional comments about closing:  Accepted in r29821.