Rockbox

Did you tried this with uSDHC card inserted?

For me it doesn't happened for Fuze v1 r26353 with and without uSDHC card inserted.
For Clip+ it happened only with uSDHC card inserted for r26330 and r26353 as well.

I have this problem as well. I downgraded to OF 2.2.26 to test and see if it had something to do with newer OF.
In my tests it happens only when a SDHC card is inserted.
Remove card and OF refreshes media. Reboot into OF. No refresh. Reboot to RB. Do nothing but shutdown.
Reboot into OF. No refresh.
Now insert card and repeat above sequence. Final reboot into OF will refresh again.
This is repeatable and with newer OF 2.3.33.
It seems to imply that RB is now altering the SDHC media in some way that OF detects as requiring a media refresh, even when nothing is changed while in RB. Tested with svn r26251, so change must be before this.
Hope these details help track it down. With 4GB + 8GB it means waiting quite a bit during development testing.

I also experience this on Fuzev2. I also don't get any problems if I take out the µSD card (not sure if it being SDHC has anything do with it).

I can verify that for me too this only happens with the card inserted (now using build r26428), so yes, it does suggest to me that RB is adding/changing some data on the SD card, which is then detected by the Sansa OS.

You can check this by dumping the content of the card before & after rockbox insertion, e.g. using dd, and check the difference between the 2 dumps

I did a series of tests. I started with an empty 512MB SD card since this wouldn't take as long as my 8GB one. It has no files but has root folders.
I used dd to make image files after each of a sequence of steps and then used "cmp -lb A B" to compare them.
What I found is that the card contents remain identical even when the Sansa firmware detects changes.
If I remove the SD card when the power is off it knows that it has been removed and inserted again and this always forces a rebuild.
If I just turn power off and on with Sansa firmware then it doesn't rebuild.
If I turn power off, start RB, turn power off, start Sansa then it always does a rebuild - as though the card was removed but it wasn't.
All 5 of the image files I dumped at various steps never changed even one byte.
From this I expect that the "change detect" flag must be either on the system flash drive or other internal flash state memory.
Hope this helps.

I did do some more comparisons between USB insertions looking at the SYS files and binary images of the first 1MB on the system drive.
It appears the MTABLE.SYS changes when a rebuild occurs so I presume that contains the Sansa database.
But also at byte offsets 1001, 401037, 881293 in the drive images there are changes that only seem to occur when RB is run between insertions.
I don't know what files these are actually in. I know the image is FAT format but I don't know how to view it as FAT to see what files map those bytes.

Tested it on clip+. It looks like this bug was introduced with r26243, I've got media refresh with it.
No media refresh with r26241 and r26142

Perhaps the OF doesn't like us writing to RTC alarm registers

Yeah, It's looks like. But still only when memory card is inserted. I don't understand how/why those two things are related to each other.