Rockbox

Another choice would be to increase the size of the font buffer on players with large amounts of RAM... it's already conditional on ram (60000 bytes for players with more than 2MB ram, 10000 otherwise) so we could add higher levels.

> I recommend that we either strip glyphs from 15-Adobe-Helvetica until it fits in the buffer, or change Cabbie to use a different font instead.

I strongly object the rational of limiting Unicode coverage of a default font. I would say the opposite - increase coverage as much as possible so that new users will find that Rockbox just works for them. Increasing Unicode coverage increases the users base - adding support for Chinese language makes Rockbox an appealing product for all Chinese users, etc.
I think users expect Rockbox to work out-of-the-box, and that includes the ability to display their music tracks ID3 info properly (provided it is encoded as UTF-8, otherwise they will have to change default codepage).
I think we shouldn't expect users to know that they can resolve the issue of having text displayed as gibberish by changing the theme's font. They just think that Rockbox doesn't support their language and that it sucks (pardon me for that).

If I have to choose a temporary solution for this issue I would choose longer boot time over narrower Unicode coverage.
I think we should rather have users that might think Rockbox boots a bit slower than they expect it to, but is a great software that works as they expect it to work, than to have users who experience a super-fast loading unusable piece of s...oftware.

> Fixing the font caching to not be so slow would be nice, but a lot more work ;)

I think we should go towards fixing font caching. I suggest loading only a minimal set of glyphs upon boot, e.g. loading only English, and leaving other glyphs to be loaded upon first use. We don't have to fill the cache at boot time, we can have it grow over time.

Why don't we either introduce a cabbiev2_unicode theme (that uses the current font) or explicitly mention in the manual that the default theme provides limited unicode coverage but that it can be resolved by changing to this font, then change the default loaded theme to a faster font (such as a version of this font with stripped character set).

I don't agree with the rationale of "we should punish users with 5 seconds of extra boot time to prevent some users from having to change the theme." The cost of wider unicode support is a decision that should be left up to the user, and it's more likely they'll know that decision exists if they see something that doesn't work at all (characters missing) that leads them to check the manual, than if they just have a slow boot and think "Oh, Rockbox takes a while to start I guess."

> I strongly object the rational of limiting Unicode coverage of a default font. I would say the opposite - increase coverage as much as possible so that new users will find that Rockbox just works for them.

I disagree here. Unicode coverage vs. use is always a trade off. For me, I don't need asian characters and the like, and I prefer faster loading than loading glyps that I never need. I also prefer having a smaller font buffer if the space is simply not needed. As far as I can see the current font buffer size is a good tradeoff for a lot of cases. It might not be for targets with larger screen size (like Beast and 5G). So this raises the (somewhat) important question: which glyphs are actually used by most users?

> and that includes the ability to display their music tracks ID3 info properly (provided it is encoded as UTF-8, otherwise they will have to change default codepage).

(unrelated side note: there are tag types around that don't require setting a code page. Vorbis comments for example are always utf-8).

> If I have to choose a temporary solution for this issue I would choose longer boot time over narrower Unicode coverage.

I completely disagree here. Increasing the font buffer on devices defaulting to 15-Adobe-Helvetica would be the better solution (those devices usually have more RAM anyway). In any way, I want my player to power up *fast*, and given the fact that we *always* boot the player which is slower than Apples sleep mode we *need* to be fast. An Ipod user experiencing boot times with 6 seconds only for loading the font (which I guess will get at least around 10 seconds with the apple bootloader and other loading stuff that has to happen) is likely to turn away because the Apple firmware "boots" almost instantly.
A short check with stopwatch timing: nano2g: 2 sec (boot from iloader menu), mini2g with CF mod: 6sec (complete boot with Rockbox bootloader), OF: instantly. And that is with a font that fits into the buffer.

Fixing font cache to be faster is of course the best solution; however I disagree with making boot slower just for loading glyphs most likely the majority of users never need. We need to make booting faster, not slower (that's also a reason why a graphical bootloader is bad)

Completely agree - fast boot time is more important. People that need the glyphs can change the font and then suffer the slowdowns. I haven't checked if this is the cause, but the Gigabeat S takes 11 seconds to boot from seeing the Rockbox bootloader - this is an eternity.

If we *really* wanted to ensure that broad coverage of Unicode was available, we wouldn't be using a way smaller font on most targets. If that's your argument, then we ought to fix the *other* versions of Helvetica to include equally as many characters, no? (which will make *all* targets boot slower).

Caching is not slow because it loads too many glyphs; it's slow because it does it in a completely naieve way. The most recently used glyphs are loaded, *one at a time*, in *order of how recently used they were*. This causes hundreds/thousands of seeks, and provokes absolute worst case behaviour from the 1024-byte-sector emulation code used on the iPod 5.5G. Just sorting the list by index before saving it would likely improve the speed substantially.

If this was just a matter of 100-200ms on disk targets (and less on flash) then I wouldn't really care *too* much, but the fact that it adds 6 entire seconds on 5.5G is just insane and I don't think any argument about keeping users with non-ascii characters happy justifies it.

I would say that in the long term we should both fix the font caching code to work less horribly, and decide *which* character sets should be included by default, and then adjust *all* the versions of the font to match: add characters if there are missing ones on some versions, and remove characters if there are gratuitously too many on others... but I really think we should do something in the short term as well.

I dont tinhk increasing the buffer is a good optino because it just means we need to up it later when more glyphs are added.
Fixing glyphcache to actually be a proper on disk cache instead of just a list of indicies would probaly speed font loading up alot.

I tihnk the best option is maybe to just load a cut down version by default and if a glyph is needed then at that point we load the full version of the font. (or when disk is inactive even)

I always assumed that the main holdup on load had something to do with... well, no idea, really. I just tried switching to the helvetica bold (size 14) from the extras pack, and WOW. I wish I'd known about this ages ago. Boot time is very impressive with a smaller font, and 14 bold is comparable in physical size to the default 15, so the visual difference wasn't incredibly jarring. I'd personally recommend it to users who don't need a ton of glyphs, while other solutions are being discussed.

A while back rockbox started to boot very slow, and now I know why. My iPod 5.5G 80GB loads in about 20 seconds from after the apple logo. I use the biggest font in my own theme (ter-u32b.fnt).

hrm. so, I won't pretend to know anything about the rockbox font format, but (thinking about the idea mentioned above about loading the most common/English glyphs first), can we not reorder the glyphs in the file so that the most common/English glyphs are at the beginning of the font file and just load the first (X) bytes to fill the font cache? where (X) is 60000 bytes or, frankly, whatever.

Okay, here's a proposed *interim* solution to this: use buffer_alloc to allocate the font buffer, according to the size of the first font loaded. There is both a maximum and minimum, defined according to total player memory. If the font is bigger than the maximum, the maximum is allocated and the font cache used. Otherwise, the size of the font is allocated (but not less than the minimum), and the whole font is loaded. This means that if the user changes font after boot behaviour may get worse, but on next reboot the buffer size will adjust to the new font.

I think this is a reasonable compromise until we work out how to fix the font cache: users who currently use small fonts will see no difference. I'd like some testing and review of the patch, and I'd also like comments on what a suitable max/min size should be for the different memory sizes: at the moment the values I have used just increase the maximum to 300000 bytes for players with 32MB or more of memory, which I believe covers the problematic ones here, but we should probably think about decent choices across the whole range of memory size.

Note that the above patch causes things to go horribly wrong if you ever boot the player up with the font set to sysfont and then set the font to something else, since this causes buffer_alloc to get called after the playback system has been initialised :)

It works fine in "normal" use cases though. I'll fix it at some point.

I've been testing this patch for three days, and it works very well for me, although I've had to reduce the font size a little bit from ter-u32b.fnt to ter-u28b.fnt for the "big info", and also I've unified the "other" fonts between wps and sbs (only xtal-14.fnt , instead of various fonts)

Hi. What about this patch? Lately the file firmware/font.c get changed and I think it's not compatible with this. Can you take a look to it, please? I need this patch because I use a big font.
Thanks!

If you just need to be able to fit a bigger font, just increase MAX_FONT_SIZE to something big enough for the font you use.

The patch is buggy and probably not a great solution anyway; there's no need to make a change that complicated just to make the font buffer bigger on your player.

This is a quick and dirty patch to load the glyph_cache in disk order, making the sector cache work more efficiently. I really think this is the right path to improve font performance. Total boot time on my iPod Video with an anti-aliased DejaVuSans 20 is under 9 seconds. There is no more wait to load a huge font than doing anything else on the iPod.

Definately the right approach, but there are some problems with your patch: it only loads 256 glyphs unless I misunderstand it.. and it destroys the LRU order in the process (which will, in theory, cause cache misses to increase after boot).

You can measure boot time and all its stages precisely using bootchart: pick it from the advanced settings during configure and it will logf() a series of boot time measurements, and this can be viewed or dumped to disk in the debug menu. There are some scripts to graph it as well on the wiki, but it should be easy enoguh to pick the font_load out.

I reworked this a bit. This runs through the entire glyph cache in groups of 32. i.e. it reads 32, sorts them, and reads their bitmaps then on to the next 32. The trade off is that the drive will have to make N/32 passes but the LRU order is still somewhat preserved.

Changes from V2:
Now skips loading the part of the glyph cache that doesn't fit in memory
Increased the block size to 256
Reload the glyphs in LRU order after loading in disk order to preserve LRU order.

Has anyone looked into just dumping the whole gylph cache (along with appropriate information to invalidate when needed)? That still seems the most simply solution (and probably the fastest) to me.

You'd have to keep a separate .glyphcache file for each font.

I stumbled on a bug in the glyph file write out while I was working on this.
The logic to skip writing out invalid glyphs is broken when the first character is not zero.

Hi.
If it helps, today I've tried to increase MAX_FONT_SIZE to 300000 (like in your patch) in a fresh font.c file, and it loads VERY slow like in the standard build (25 to 30 seconds). After that, I've re-synched your entire patch with the actual font.c and it loads that slow too.
With the other patches (glyph_xxxxxx.diff) I haven't noticed any speedup.

This confuses me. At MAX_FONT_SIZE 300,000 15-Adobe-Helvetica (232k) should load straight into memory, bypassing my changes. A bigger buffer will slow down the loading of fonts that don't fit because it has room to preload more glyphs. Are you loading Cabbie_v2 with 15 px Adobe-Helvetica? How big is your .rockbox/.glyphcache file?

Please note that glyph_writeout_fix.diff is just to stop the creation of .glyphcache with thousands of extra 0x001f's. You want glyph_cache_diskorder_v3.diff for the loading speed up.

If loading takes 25-30 seconds you have another problem, I'm afraid; the font issue here causes font loading to take ~6 seconds, bringing the total boot time on an ipodvideo to about 8-9 seconds. If it takes 30 seconds then something else is probably the culprit. You may want to create a bootchart to see what it is which takes all that time...

Fixed a bug in the size counter.

Until my last update from svn before august 2th, with the initial patch applied, my iPod 5.5G 80Gb was loading at 8-10 seconds.

I use in my wps very large fonts, although the fnt files sizes are between 130-170kb.
%Fl(2,ter-u28b.fnt)
%Fl(3,16-Terminus-Bold.fnt)
(and the system font)

In my sbs I use
%Fl(3,16-Terminus-Bold.fnt)
(and the system font)

I've tried the glyph_cache_diskorder_v4.diff patch, after re-synching it, and it still loads in the 25-30 seconds range.

***SOLVED before submit this post, after notice that the glyphcache files are very old.
1304 2010-07-10 14:16 .glyphcache
150 2009-09-15 17:40 .glyphcache-brwsr
8 2009-09-15 17:40 .glyphcache-menu
0 2009-09-15 17:40 .glyphcache-record
0 2009-09-15 17:40 .glyphcache-tuner
58 2009-09-15 17:40 .glyphcache-wps

I've deleted all of them and it loads fast again in about 10 seconds. I was very frustrated with this, but now it's ok.

Thank you all.

Re-sync to SVN (paths changed)
Included the glyph file write fix.

I've recently noticed that on my 80GB iPod Video default font takes enormous time to load and asked to reopen this bug.
This bug was closed as it was supposed to be fixed in r27864, but according to my tests with bootchart r27864 and r27863 on my device takes exactly same amount of time to load 15-Adobe-Helvetica:
-On fresh install font takes 1.0s to load
-After viewing contents of my music folder, which has filenames with cyrillic and grek characters, loading font start to take 3.8s

Then after around r27920 behaviour changes and keeps the same until latest svn (r28850):
-On fresh install 15-Adobe-Helvetica loads slightly over 6 seconds
-After viewing music library contents the font start to take 7.5s to load

I've been using my custom build with MAX_FONT_SIZE increased to 300000 for a couple of weeks and it loads much faster: font loading take only 0.15s. Much more smoother :)

Can anyone else reproduce this? I've tried several big fonts after looking at the Greek alphabet and I don't have the issue.

freddyb: on what target was this? The original problem was only really visible on a 80GB ipod video.

Sorry, iPod Video with 30GB drive. But I did have the slow loading before the patch.