Rockbox

Tasklist

FS#2289 - Feature: browse directories using id3 tags

Attached to Project: Rockbox
Opened by J. Deonarine (deonarin) - Sunday, 01 August 2004, 00:52 GMT
Last edited by J. Deonarine (deonarin) - Thursday, 30 June 2005, 13:20 GMT
Task Type Patches
Category
Status Closed
Assigned To No-one
Operating System
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No

Details

This would at least partially fill feature requests:
http://sourceforge.net/tracker/index.php?func=detail&group_id=44306&atid=439121&aid=720223
http://sourceforge.net/tracker/index.php?func=detail&group_id=44306&atid=439121&aid=912482
http://sourceforge.net/tracker/index.php?func=detail&group_id=44306&atid=439121&aid=915806
http://sourceforge.net/tracker/index.php?func=detail&group_id=44306&atid=439121&aid=915816

(and probably some others)

With this modification, you can select whether or not
you want to sort files by their names, or via id3
information (right now all that is supported is name,
track, and artist). It does this by giving you control
over what is displayed in the tree view for mp3 files.

To use this, first turn the feature on in
(main menu)->General Settings->File View->Use id3 Tags
in Browser

Next, you will need to create .display files. There is
a 'global' file called .display_global in the .rockbox
directory. If this file is missing, then by default
the browser will show only filenames. Otherwise, this
file will specify the default browse configuration.

Browse configuration is overridden by the .display file
in the directory you are viewing. If the .display file
is present but invalid or empty, then the folder will
display file names instead of using the id3 tags.

A 'valid' .display file contains one or more %
characters. The meaningful escape sequences are:

%a - replace with Artist name
%t - replace with Track name
%r - replace with tRack number

If the file lacks a id3 tag, or if no meaningful escape
sequences are present in the .display file, the file
name is used.

Since I have to allocate memory for this function, and
I do this at startup, turning on this function for the
first time since you've restarted the box may require
you to reboot. Turning this feature off is immediate.

Changes to .display_global require a reboot (I read
this file at startup once and not again) while changes
to .display files take effect on the next directory read.

As we have to read each file in the directory in order
to get the id3 information, using this function causes
additional disk activity. It also can make browsing
directories slower if there are large numbers of files
in that directory.

If you have no .display_global file, or you have a
blank .display file in a directory, no additional disk
access takes place. Turning this feature off
immediately removes the additional disk accesses from
future directory reads.

This is just a first run at this feature... probably we
would want to also support more escape sequences, and
maybe conditional escape sequences (much like the while
playing screen). I probably could have allocated the
memory for this feature without a reboot, but I was
afraid of memory fragmentation (not sure if this is a
problem for you guys or not...) Obviously, a mp3 id3
database would be more useful, but frankly, I can't
imagine that we would get very good performance based
on the limited hardware in the Archos. That and this
seemed doable in a weekend :)

This version is workable, and seems to run without
issue on my box.
This task depends upon

Closed by  J. Deonarine (deonarin)
Thursday, 30 June 2005, 13:20 GMT
Reason for closing:  Out of Date
Comment by Anonymous Submitter - Sunday, 01 August 2004, 10:00 GMT

nice! ID3 browsing can look a lot better :)
but too bad it's really slow. maybe you can add some sort of
save function. or a rename-to-this-format tool.
anyway, keep up the good work :)
Comment by zeekoe (zeekoe) - Sunday, 01 August 2004, 10:37 GMT

nice! ID3 browsing can look a lot better :)
but too bad it's really slow. maybe you can add some sort of
save function. or a rename-to-this-format tool.
anyway, keep up the good work :)
Comment by J. Deonarine (deonarin) - Sunday, 01 August 2004, 17:09 GMT

Yeah, having a cache would be nice, but then you run into
two problems: one is consistancy, (what triggers a
refresh?) and the other is search time on the cache. (does
it take longer to look up the file name than to read the id3
info again?)

Plus, that is again starting to edge towards the id3
database. I will look into it, but I think this might be as
good as we can do, given our resources.
Comment by Anonymous Submitter - Sunday, 01 August 2004, 18:17 GMT

what is wrong with a ID3 database? Why not implement an
ipod database style browser?

It would be nice to have a menu item "database" that would
be akin to browsing like an ipod. You could either update the
database on a computer, or have a plugin to do it on the
device.

If you were to store something like a checksum along with
each item in the database maybe you would not have to
rescan _EVERY_ file for ID3 information?

The Archos gmini series implements something like this, and I
think it would be really useful.
Comment by J. Deonarine (deonarin) - Monday, 02 August 2004, 01:48 GMT

Just uploaded a new copy of the patch, against the latest
CVS, adding caching support. Caches are stored in
/.rockbox/cache, with a name of .display_cache_DIRNAMEHASH.

This should run very much quicker once you've built the
caches...

The problem with the database approach is that for browsing
at least, you need to be able to quickly select the id3
information based on some easily accessable info from the
file (in most cases, a checksum of sorts, as you say) In
order to do this quickly, we would need hashtables in RAM.
We would need to store as much as possible in RAM in order
to avoid reading the disk (which is sequential, and slow)
I'm just thinking that with the limited ram in this unit
(2mb, and every time we allocate any of it, we detract from
mp3 buffer and therefore battery life) there isn't much
chance of doing better than going and reading each tag every
time. And that would be abysmally slow.

I've implemented this so that it could be possible to butt
up to a database if we can come up with a way of creating
one within the requirements of the unit. But I'd rather do
what I think can be done, rather than wait around until the
database materializes.

The cache is fairly simple: it takes a hash of the full
directory path, and uses that to choose the filename from
the cache. Each cache file contains a hash of the format
used to generate it, and then the generated display name for
each file (along with a hash of the file's name).
Basically, I'm not saving the id3 tags, instead, I'm saving
the display name generated from the tags.

Comment by J. Deonarine (deonarin) - Monday, 02 August 2004, 01:53 GMT

I can think of one way where you could mess things up, if
you update the id3 tags on a file and then don't clear the
cache or change the file name or directory in any way.

You would just be looking at stale tag information.

I could probably fix this, but at the moment I'd like to see
if this gets accepted first. If it doesn't, then what's the
point in fixing the corner cases :) (probably by adding the
modified date to the input to the hash function, I could
avoid this situation)
Comment by zeekoe (zeekoe) - Monday, 02 August 2004, 21:46 GMT

hmm.. just did a quick test...
the root dir loads in about 5-10 seconds now, and subdirs load
in about 2 seconds. this happens too the second and third
time i enter them.
id3 browsing has really speeded up now though, also about 2
seconds per dir.
/me tests again tomorrow
Comment by J. Deonarine (deonarin) - Tuesday, 03 August 2004, 00:54 GMT

Here are some real numbers:

Test setup was a recorder v2, directory containing 44 files.
I entered other directories to make sure the disk was spun
up before doing the time trials.

id3 on:
no cache: ~4.5 seconds
cached: ~1.6 seconds
no id3: less than 1 second (somewhere around .7-.9 sec)
id3 off:
same result as id3 on, no id3 tag usage

Buffer size:
daily build for 8/2 with my settings: 1.601MB
patched build, id3 off: 1.597MB
patched build, id3 on: 1.582MB
Comment by Linus Nielsen Feltzing (linusnielsen) - Tuesday, 03 August 2004, 22:40 GMT

I like thisQ I will test this as soon as I can, but I am
currently on vacation, so don't hold your breath... :-)
Comment by J. Deonarine (deonarin) - Tuesday, 03 August 2004, 23:44 GMT

Thanks Linus! Glad to hear there's some interest in taking
this :)

I've updated the patch again. There were some special cases
where caches could get created or read when we didn't need
to. He was having a performance problem with his root
directory. We will see if this fixes the problem.
Comment by J. Deonarine (deonarin) - Saturday, 07 August 2004, 23:53 GMT

Updated the patch again. Found another small bug.

Also, the track number token has been changed from %r to %n.
Zeekoe and amiconn suggested this, and I had already
thought about changing it.

There is a Wiki page for this feature now as well:

http://rockbox.haxx.se/twiki/bin/view/Main/UseDisplayName
Comment by J. Deonarine (deonarin) - Tuesday, 10 August 2004, 02:21 GMT

More updates. Markus Braun pointed out some errors in the
display name generation. Specifically, mp3's without tags
or with blank tags were not being properly skipped. I also
made a couple of other modifications.

First, the hash generation takes the file time/date into
consideration. So if you modify the id3 tags, the cache
should properly regenerate. Second, I cache the fact that
we couldn't create a display name, so that we don't re-read
the tags each time we encounter a file without tags.
Comment by J. Deonarine (deonarin) - Friday, 13 August 2004, 01:27 GMT

Another update. Markus again pointed out a bug, I wasn't
properly using the id3 v1/v2 preference. That has been fixed.

I also added an ON+PLAY option to directories to clear the
cache on a directory-by-directory basis.
Comment by J. Deonarine (deonarin) - Tuesday, 14 September 2004, 20:15 GMT

I guess this is dead, or at least dormant. I've tried to
get this accepted, I've talked to the maintainers, I've
created documentation... not really much of a response. I
guess there isn't any interest.

So ends my involvement with rockbox. I guess filling
feature requests isn't a productive contribution to the project.
Comment by Daniel Stenberg (bagder) - Tuesday, 14 September 2004, 21:00 GMT

We appreciate your work, your time, your patch and your
patience.
Comment by Linus Nielsen Feltzing (linusnielsen) - Wednesday, 15 September 2004, 05:49 GMT

Unfortunately, you filed the patch in the summer, when the
devs were on vacation, and then the patch collected some dust.
Comment by J. Deonarine (deonarin) - Thursday, 16 September 2004, 01:52 GMT

So, as I offered in an email to Linus on 9/1, I have updated
the patch.

As you appreciate my work, I likewise appreciate your
consideration.
Comment by J. Deonarine (deonarin) - Thursday, 30 June 2005, 13:20 GMT

Closing this as out of date. It was a cool idea, I thought,
but it doesn't seem like any of the devs agree. Sigh. With
my other patch rejected after a year, I figure this ends my
involvement with the rockbox project. Have fun guys.

Loading...