Wiki > Main > DataBase (compare)
Difference: DataBase (r135 vs. r134)
Rockbox contains a music an audio database system. Using the information contained in the tags (ID3v1, ID3v2, Vorbis Comments, Apev2 etc) in your audio files, Rockbox builds and maintains a database of the these music files on your player and allows you to browse them by Artist, Album and Genre. It can also collect runtime data allowing for most played, unplayed and most recently played tracks to be logged, along with per-track ratings..
To enable the database it must first be Initialized (see Initialize Now below)
Note: Whereas normally the term "tag" means the metadata container of an audio file, in this topic, the term "tag" refers to one field thereof.
The database options can be accessed in Settings -> General Settings -> Database. The following attempts to explain the sub menus functions:
Setting this to 'Yes' loads the database to the players RAM, allowing faster browsing and searching. Setting 'No' keeps the database on the disk, meaning slower browsing but it does not use extra RAM and saves some battery on boot up. However if you frequently use the database you should load to RAM as this will reduce the overall battery consumption as the disk does not need to spin on each search.
If "automatic update" has been set to on, it automatically does everything that "update now" does at every boot. However, without dircache (General Settings -> System -> Disk -> Directory Cache) enabled, auto update may be very slow while it performs a search for deleted files.
Initialize now removes all database files (removing runtimedb data also) and then it builds the database from scratch. This takes approximately 5 to 10 minutes for most units (though quite a bit longer on Archos models except Ondios), and requires a restart to allow the files to be committed to the database. While initialize is running you can still listen to music as it runs in a background thread. You can even shut down your player and resume the database building process next time you turn it on.
Once the database has been built and committed, you can browse your music using this database from the main menu.
You may need to increase the value of the "Max files in dir browser" setting (General Settings -> System -> Limits) in order to view long lists of tracks in the ID3 database browser.
Update now only detects new and deleted files regardless of Directory Cache settings, and updates the database without removing runtimedb data.
When enabled this allows for most played, unplayed and most recent played tracks to be logged and scored. To use it the database must be initialized, and gather runtime data enabled. Runtime data is stored into the database every time rebuffering happens and old track entries are unbuffered from memory.
This allows for the runtime data to be exported into /.rockbox/database_changelog.txt, which backs up the runtime data in ascii format. This is needed when the database structures change, because new code can't read old database code. But all modifications exported to ascii format should be readable by all database versions.
Allows the /.rockbox/database_changelog.txt backup to be conveniently loaded into the database. This is performed automatically when the database is initialized.
The Rockbox database works by scanning the file system for new files, parsing them using the tag parsers built into each codec, and then storing that data for display. Unfortunately this means that if serious errors are present in the file system (due to corruption, etc) or file tags, the database update can fail. In this event, the following steps may help:
Use your PC to check that the FAT32 file system on your device contains no errors.
The database can log files as it parses them. Go to system > debug > metadata log. Then rebuild the database. As each file is parsed, it will be logged to /metadata.log. If the database parser fails, the last file logged is probably responsible. Try removing it and rebuilding the database. If you find a file that crashes the database, please upload it to our bug tracker.
The /.rockbox/tagnavi.config file and its user-editable counterpart /.rockbox/tagnavi_custom.config control the structure and layout of the ID3 File View. It is strongly recommended to modify only the /.rockbox/tagnavi_custom.config file, as the default file will be overwritten every time you install a new build.
In tagnavi.config you can define your own layout for the database navigation using powerful filters and conditionals. It is possible to logically include other files (see
The order of the different sections in the file is crucial because the file is interpreted whilst loading in the order in which it is written. The order should generally follow the list below. The important point is that for a line to display, all of its contents should have previously been defined. E.g., if a main menu item points to a sub-menu, then the sub-menu must be defined in the file prior to the main menu.
A tagnavi file can contain the following types of lines:
Note: You can only use a %format string for formatting the title of a track, it cannot be used for formatting within the album hierarchy http://forums.rockbox.org/index.php?topic=9570.0
The following are provided to give an idea of what can be achieved using the flexibility of a custom tagnavi file. To try any of these, either download the example directly using the links (and rename to the correct name format), or copy and paste the code into a text editor and save the file as tagnavi_custom.config in the format ANSI without BOM (Byte Order Mark). UTF-8 format will not work. Podcasts have been used in the examples, but the rules apply equally to any other differentiation that the user would like to make.
It is quite likely that a user would like to differentiate podcasts from music files on their player. The following code would simply add a 'Custom...' menu item to the end of the default Database File View, selecting which would show all files genre tagged as 'podcast'.
#! rockbox/tagbrowser/2.0 %menu_start "custom" "Podcasts" "Podcasts" -> title = "fmt_title" ? genre = "podcast" #EOF
Note that because a standard formatting is used, a tracknumber value is shown.
It is unusual for all podcasts to be tagged correctly, so the following code uses the file location to ensure that all the files are shown. For this to work, it is assumed that all podcasts are in a folder named 'podcast' or 'podcasts' somewhere on the device.
#! rockbox/tagbrowser/2.0 %format "fmt_podcast" "%s (%s)" title filename %menu_start "custom" "Podcasts" "All podcasts" -> title = "fmt_podcast" ? filename ~ "/podcast" #EOF
This works because the 'filename' attribute is actually full path and filename, so the query 'title contains "/podcast"' captures all files with that in their path, regardless of tagging. The custom formatting also shows both title (from tag) and filename in case the file is untagged.
Building on the previous example, it is possible to filter information based on the runtime information (playcount and lastplayed). Using this, podcasts can be split into 'new' and 'old' (i.e., already listened to) and sorted in listening order. For this to work, the runtime database has to be enabled -
#! rockbox/tagbrowser/2.0 # Un-listened to files sorted by duration, shortest files at the top %format "fmt_podcast" "%08d%s (%s)" length title filename %strip = "8" # Listened-to files sorted by playing order, most recently played at # the top %format "fmt_podcast_old" "%06d%s (%s)" lastplayed title filename %sort = "inverse" %strip = "6" %menu_start "custom" "Podcasts" "New Podcasts" -> title = "fmt_podcast" ? filename ~ "/podcast" & playcount == "0" "Old Podcasts" -> title = "fmt_podcast_old" ? filename ~ "/podcast" & playcount != "0" #EOF
As the full tagnavi_custom file is long, only exerpts are shown below. The best way to understand the syntax is to download the file and study it in a text editor or, even better, install it and just play with the functionality!
# The main menu is mostly calls to sub-menus with the exception of the # podcast/radio selections that filter only on whether the track has # been listened to or not (playcount), the rest of the hard work is # done by the formatting %menu_start "bascule" "Rockbox Browser" "Album Artists" ==> "custom_artist_initial" "All Artists" ==> "custom_artist" "Albums" ==> "custom_album" "Mood" ==> "mood_tree" "New Podcasts/Radio" -> title = "f_radio" ? filename ~ "/radio/" & playcount == "0" "Old Podcasts/Radio" -> title = "f_radio_old" ? filename ~ "/radio/" & playcount != "0" "Tracks by usage" ==> "custom_runtime" "Search" ==> "custom_search" "Year" -> year -> genre -> artist -> album -> title = "f_browse" "All tracks A-Z" -> title = "f_direct"
Note the 'Rockbox Browser' title. Using these well adds a touch of class to the look of the screen.
# Condition 'filename !~ "_cmpl"' excludes all compliation # album tracks from this menu,by virtue of the fact that all such # tracks have '_cmpl' appended to the filename i.e., # 06-missing_cmpl.mp3 'Non-alphabetic' category created as a catch-all # for tracks beginning with punctuation marks etc. 'T' artists # specially formatted to exclude names beginning with 'The' and a # 'The...' category separately defined %menu_start "custom_artist_initial" "Browse album artists" "Non-alphabetic" -> artist ? artist < "A" & filename !~ "_cmpl" -> album -> title = "f_browse" "A" -> artist ? artist ^ "A" & filename !~ "_cmpl" -> album -> title = "f_browse" "B" -> artist ? artist ^ "B" & filename !~ "_cmpl" -> album -> title = "f_browse" ... "T" -> artist ? artist ^ "T" & artist !^ "The " & filename !~ "_cmpl" -> album -> title = "f_browse" "The..." -> artist ? artist ^ "The " & filename !~ "_cmpl" -> album -> title = "f_browse" ... "Z" -> artist ? artist ^ "Z" & filename !~ "_cmpl" -> album -> title = "f_browse"
# A-Z of all artists, including compilation artists # Special selection solely for artists on compilation albums %menu_start "custom_artist" "Browse all artists" "Artist by genre" -> genre -> artist -> album -> title = "f_browse" "Compilations by artist" -> artist ? filename ~ "_cmpl" -> album -> title = "f_browse" "Non-alphabetic" -> artist ? artist < "A" -> album -> title = "f_browse" "A" -> artist ? artist ^ "A" -> album -> title = "f_browse" "B" -> artist ? artist ^ "B" -> album -> title = "f_browse"
# Most search requirements are covered here! %menu_start "custom_search" "Search for..." "Artist starting with..." -> artist ? artist ^ "" -> album -> title = "f_browse" "Artist containing..." -> artist ? artist ~ "" -> album -> title = "f_browse" "Artist between years..." -> artist ? year >= "" & year <= "" -> album -> title = "f_chrono" "Album starting with..." -> album ? album ^ "" -> title = "f_browse" "Album containing..." -> album ? album ~ "" -> title = "f_browse" "Album between years..." -> album ? year >= "" & year <= "" -> title = "f_chrono" "Track starting with..." -> album ? title ^ "" -> title = "f_direct" "Track containing..." -> album ? title ~ "" -> title = "f_direct" "Filename starting with..." -> album ? filename ^ "" -> title = "f_direct" "Filename containing..." -> album ? filename ~ "" -> title = "f_direct"
# The runtime entities populated by the Database are used to # derive the filters and conditions for the 'Usage database' menu # 'Most played' and 'Recently played tracks' display only tracks that # have been played (playcount > "0") # A 'favourite' is classified here as having been played more than # three times and with an average of 85% of the track having been # listened to. %menu_start "custom_runtime" "Usage database" "Most played (Plays|Score)" -> title = "f_runtime" ? playcount > "0" "Recently played tracks" -> title = "f_lastplayed" ? playcount > "0" "Favourite albums" -> album ? playcount > "3" & autoscore > "85" -> title = "f_browse" "Recent favourites" -> title = "f_lastplayed" ? playcount > "3" & autoscore > "85" "Forgotten favourites" -> title = "f_forgotten" ? playcount > "3" & autoscore > "85" "Never played tracks" -> artist ? playcount == "0" -> album -> title = "f_browse"
The runtime database information could be used creatively to construct quite complicated conditional statements if the user wished
# The following mood menus are similar in that each mood (upbeat party, # downbeat rock, etc.) is purely a list of tracks tagged with any of # the genres in the list (e.g., "acoustic|ambient|new age|easy # listening") This can be used to quickly produce a substantial # playlist of similar music when you just know what 'sort' of music you # want ... %menu_start "mood_2" "Party" "All tracks" -> artist ? genre @ "dance|disco|electronica|pop|techno|trance|funk|soul" -> title = "f_mood" "Upbeat" -> artist ? genre @ "dance|disco|electronica|pop|techno|trance" -> title = "f_mood" "Downbeat" -> artist ? genre @ "funk|soul" -> title = "f_mood" ...
# This ensures that, rather than being accessed via the 'Custom...' # entry in the default Database view, this custom menu *is* the default # menu %root_menu "bascule"
An in depth inspection of the file-format of the files created while the database initialization can be found here: TagcacheDBFormat.
Tracks do not sort correctly when using disc numbers with more than one digit or tracknumbers with more than 2 digits
The database uses the format string for sorting which is set to one digit for discnumber and 2 digits for tracknumber. If you want more digits to sort correctly, you can adapt the string in tagnavi.config (see http://www.rockbox.org/twiki/bin/view/Main/DataBase#tagnavi_config_v2_0_Syntax )
For example change
%format "fmt_title" "%d.%02d. %s" discnum tracknum title ? discnum > "0"
%format "fmt_title" "%02d.%03d. %s" discnum tracknum title ? discnum > "0"
for 2 digit discnumbers and 3 digit track numbers. Note that tagnavi.config will get overwritten with every update, so the better option is to create your own tagnavi_custom.config.
The setting "Max Entries In File Browser" must be larger than the search result for the sort to succeed.
The database was added to the Rockbox CVS on 26 March 2006 (then called Tag Cache). Development continues, but as of late 2006 the functionality is very stable and comprehensive.
Here you can list all features you would like to see implemented in the database in future:
If your database crashes during building, you can enable logging under System > Debug > metadata log. This will generate a log file listing each file parsed by the database. If parsing crashes without completing, the last file listed may be the cause. If you find a file that crashes the database, upload it and file a bug report so that it gets fixed.
To delete the database hook your device up to a computer. Navigate to the device drive. Delete all files in the /.rockbox partition the begin with "database_"
r138 - 01 Feb 2014 - 17:08:14 - ChrisJordanRevision r135 - 27 Nov 2013 - 00:14 - ChrisJordan
Revision r134 - 10 Jul 2013 - 17:23 - MichaelGiacomelli
Copyright © by the contributing authors.