Rockbox.org home
release
dev builds
extras
themes manual
wiki
device status forums
mailing lists
IRC bugs
patches
dev guide



Rockbox mail archive

Subject: Re: Tag Cache DB File Format Question

Re: Tag Cache DB File Format Question

From: Charles Mason <charlie.mas_at_gmail.com>
Date: Wed, 15 Oct 2008 16:10:18 +0100

On Tue, Oct 14, 2008 at 8:32 PM, Charles Mason <charlie.mas_at_gmail.com> wrote:
> On Tue, Oct 14, 2008 at 6:19 PM, Thomas Martitz <thomas47_at_arcor.de> wrote:
>> Charles Mason schrieb:
>>>
>>> I presume this means there's a problem with the DB my extension has
>>> written. I presume that's a memory address, is there any easy way to
>>> find out what that relates to?
>>>
>>> Comparing the output of a DB generated with Rock Box and one with my
>>> Extension I can't see any major differences. The only one I have found
>>> is with the entry in the index file after the first 4 version bytes
>>> (offset bytes 4-7). According to the Tag Cache DB Spec in the Wiki
>>> that is supposed to be the number of non header bytes in the index
>>> file however in the version created by Rock Box its about 2.5 times
>>> larger than the actual number of non header bytes. Is the wiki wrong
>>> and if so what does this field relate to ?
>>
>> I have, together with ArenOlson worked on that Wikipage. I've pretty
>> familiar with teh Tagcache file format.
>>
>> I have to admin though, that I've never written "selfmade" db files, nor I
>> double checked the bytes field that's bugging you.
>>
>> The bytes field should be actually state the exact filesize of the file,
>> minus the header (12bytes), if I'm not wrong.
>>
>> Looking at my parser, the bytes stated in the db index file also tells a
>> much higher (1.6x higher) value than the file size is actually.
>>
>> However, it's correct for the tag (0-8) files.
>>
>> Looking in tagcache.c, this particular part looks interesting:
>>
>> /* Update the master index headers. */
>> if ( (masterfd = open_master_fd(&tcmh, true)) < 0)
>> {
>> read_lock--;
>> return false;
>> }
>> tcmh.tch.entry_count += tch.entry_count;
>> tcmh.tch.datasize = sizeof(struct master_header)
>> + sizeof(struct index_entry) * tcmh.tch.entry_count
>> + tch.datasize;
>>
>> The bytes field for the index file seems to mean something different
>> actually, including the size of the header
>
> That's what I have been seeing the Wiki seems to be correct for the 8
> tag files but not the master index. The one thought I had was perhaps
> the the index file size includes the size of the data in the tag
> files. It strikes me that would be handy thing to know for a parser
> running on a device with limited resources, but that's purely
> speculation..
>
> I guess the answers in the source code somewhere. I will keep digging
> thorugh it.

Ok I think I have figured it out. As far as I can tell the code below
is called for each tag type.

    tch.datasize = lseek(fd, 0, SEEK_END) - sizeof(struct tagcache_header);

    if (index_type != tag_filename)
        h->datasize += tch.datasize;

Once all the tag indexes are built the tch.datasize below equals the
total of all the datasize headers of all the tag files except the file
name tags file.

 tcmh.tch.datasize = sizeof(struct master_header)
       + sizeof(struct index_entry) * tcmh.tch.entry_count
       + tch.datasize;

That is added to the size all all the non header bytes in the index.
However unlike all the tag files the size of the header is also added
in to value indexes contrary to what the wiki says.

Before I update the wiki it would be handy if some one could quickly
take a look and check I am correct. All the above samples are from
tagcache.c.

The problem is that doesn't seem to solve my problem with the
DataAbort I am getting. Could someone tell me what the memory address
refers to and if there is a way to find out which chunk of code is
generating it? Is there a "debug" build I need to be using?

Charlie M
Received on 2008-10-15


Page was last modified "Jan 10 2012" The Rockbox Crew
aaa