Rockbox

  • Status Closed
  • Percent Complete
    100%
  • Task Type Bugs
  • Category Operating System/Drivers
  • Assigned To No-one
  • Operating System Sansa e200
  • Severity Low
  • Priority Very Low
  • Reported Version Daily build (which?)
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: Rockbox
Opened by mc2739 - 2010-02-20
Last edited by pamaury - 2010-04-23

FS#11043 - High dircache stack usage after usb connection

I am seeing high dircache stack usage after USB connection on my e280v1.

I tested as far back as r24419 and up to current (r24794).

Steps to reproduce:

1. Clear all settings on boot
2. Check dircache usage on debug screen
3. Attach USB cable
4. Eject or safely remove
5. Remove USB cable
6. Check dircache usage on debug screen
7. Enable dircache
8. Reboot
9. Check dircache usage on debug screen
10. Attach USB cable
11. Eject or safely remove
12. Remove USB cable
13. Check dircache usage on debug screen

With r24419, the stack usage for steps 2, 6, 9, and 10 were 2%, 3%, 2%, and 82%. This 82% level was also present after rebooting.

With r24657, there was an increase in stack usage. The levels recorded were: 2%, 3%, 2%, and 93%.

With r24708, there was another increase in stack usage. The levels recorded were: 4%, 7%, 4%, and 97%.

Current svn gives the same values as those recorded for r24708.

As I have enabled other features in Rockbox, I have started to see “Stkov dircache (0)” panics. These panics continue after reboot and require editing the config.cfg file in USB mode to disable dircache. After disabling dircache, it will reboot without problems.

Closed by  pamaury
2010-04-23 08:08
Reason for closing:  Fixed
Additional comments about closing:   Warning: Undefined array key "typography" in /home/rockbox/flyspray/plugins/dokuwiki/inc/parserutils.php on line 371 Warning: Undefined array key "camelcase" in /home/rockbox/flyspray/plugins/dokuwiki/inc/parserutils.php on line 407

r25696, r25697

The high stack usage is normal because I reduced the stack size of dircache but I also heavily modified the way dircache generate its internal structure.
The stack usage of dircache is supposed to be completely independent of all other features of rockbox and depends only the maximum directory depth.
The current code should accomodate a depth of 20 approximately.

What is the maximum depth of your directory tree ?
I will look at the commit you quote however.

I fix a NULL-pointer deference in r24835. I don't really expect it to solve a stkov problem but could you try with this fix to be sure ?

That fix did not help.

I was able to get the dircache stack back to 97% from 99% by deleting my .rockbox directory. This helps some, but I will still get a stkov after a disconnect and rolo of a new firmware.

Does this bug still exists ?

Yes, the dircache stack usage is still high after USB connection (97% to 99%). Tested with r25217.

Could you please check your file system for errors ?
I really can't understand how this is possible. I myself have a e280v1 and I get 62% usage with pretty deep structures.

If it continues, you add this line in firmware/common/dircache.c:
#define LOGF_ENABLE
just before
#include "logf.h"

Recompile and paste logf output after usb connection.

These are stats from my e270r after formatting and reloading Rockbox (r25259)
dircache os stack usage (dircache disabled)

  before USB connection: 4%
  after USB connection: 7%

dircache os stack usage (dircache enabled)

  before USB connection: 4%
  after USB connection: 63%

directory depth: 4

from stats plugin

  Files: 319
  Music: 0
  Dirs: 36
  Max files in Dir: 43

from Windows Vista dir listing:

  Total Files Listed:
      319 File(s)     15,135,905 bytes
      104 Dir(s)   5,982,404,608 bytes free

These are stats from my e270r after restoring music (r25259)
dircache os stack usage (dircache disabled)

  before USB connection: 4%
  after USB connection: 7%

dircache os stack usage (dircache enabled)

  before USB connection: 4%
  after USB connection: 97%

directory depth: 4

from stats plugin

  Files: 671
  Music: 298
  Dirs: 64
  Max files in Dir: 43

from Windows Vista dir listing:

  Total Files Listed:
      667 File(s)  5,698,753,068 bytes
      188 Dir(s)     297,766,912 bytes free

These are stats from my e270r using logf build (r25259M)
dircache os stack usage (dircache disabled)

  before USB connection: 4%
  after USB connection: 7%

dircache os stack usage (dircache enabled)

  before USB connection: 4%
  after USB connection: 93%

directory depth: 4

from stats plugin

  Files: 672
  Music: 298
  Dirs: 64
  Max files in Dir: 43

from Windows Vista dir listing:

  Total Files Listed:
      668 File(s)  5,698,753,345 bytes
      188 Dir(s)     297,762,816 bytes free
   logf.txt (0.3 KiB)

I looked at the generated code for an e200, compiled using gcc 4.4.3. The call chain for the dircache scanning looks something like this, in a worst case scenario, stack wise (with the stack usage for each function after it):

dircache.c:
dircache_thread 44
dircache_do_rebuild 28
dircache_scan_and_build 304

sab_process_dir 60

fat.c:
fat_getnext 108
fat_readwrite 92
transfer 28

ata-sd-pp.c:
sd_read_sectors 76
sd_select_bank 580
sd_command 60

The recursive function is sab_process_dir. The non-recursive parts need 1320 bytes. Given that the stack is currently set to 1536 bytes, that leaves some 200 bytes for sab_process_dir. Enough for 3 levels of recursion… If the stack was increased to 2.5 kB (i.e., add 0xA00), it would allow for some 20 levels (a bit overkill perhaps, but better to be on the safe side).

Thank you very much for your findings. I think know understand why the bug happen. There is an ugly buffer in ata-sd-pp.c:sd_select_bank which seems useless to me but I need more information before going on.
The last point I'm still not sure to understand understand is why on my sd card I can go up to 20, the reason for this seems to be that sd_select_bank is only called on big sd cards (code suggest on SDHC cards).

So the solution is:
* either increase stack size as you propose
* try to reduce sd_select_bank buffer size. The buffer size is 512 bytes but a quick look suggest it can be reduced to 32 bytes. Then we could increase stack size if it's still necessary.

Hm, the comment I saw says "Only switch banks with non-SDHC cards". And I do have an SDHC-card plugged (and 8 GB "internally"). In any case, that buffer should be reduced, or write_buf increased in the loop. It does seem a bit odd to need to write 512 bytes, when only one byte is of interest - but then I don't know anything about the protocol.

But adding about 500 bytes would only be enough for 11 levels, at least with gcc 4.4.3, so some stack increase is probably needed too.

Yes that's my point. We'll probably end up increasing the stack size a bit (don't know if 11 levels are sufficients) but I'm waiting for the answer of someone who knows the SD code to shrink this buffer.

Hello,
sorry for the delay, I have been busy and forgot this task for a moment :)
I have commited the fix in r25696 and r25697, I would be nice if you could test it because I can't check the ata-sd-pp change, even though I'm pretty sure it works.

Thanks for the fixes.

Here are the stack levels I get after USB connection with dircache enabled:

r25695 - 90%
r25696 - 61%
r25697 - 44%

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing