FS#7657 - Split user data from .rockbox

Attached to Project: Rockbox
Opened by Henrik Backe (hbacke) - Saturday, 25 August 2007, 22:02 GMT
Last edited by Jonathan Gordon (jdgordon) - Sunday, 06 June 2010, 12:39 GMT
Task Type Patches
Category Configuration
Status Closed
Assigned To No-one
Operating System All players
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No


This patch is an all out take on separating user data from the /.rockbox structure.
I've called the the separate directory /rockbox (easily changed with a define).

Some of the directories in .rockbox has been duplicated here (backdrops, fonts, themes etc.)
So a user created theme should be placed in /rockbox/themes .
When a theme is loaded from a config file (on boot for example) it first searches in /.rockbox/themes and if not found there /rockbox/themes is searched.

All browse menues, where applicable, has been split in a system (/.rockbox) and user (/rockbox) part.

There is also a plugin (rockbox_upgrade_files) that creates the required structure under /rockbox and then copies files from there old places under /.rockbox. The plugin never copies a file that already exists under the new structure.

Here is the new structure:

rockbox/backdrops (duplicated)
rockbox/data (save files)
rockbox/eqs (duplicated)
rockbox/fonts (duplicated)
rockbox/games/sokoban (save files for sokoban)
rockbox/games/sudoku (save files for sudoku)
rockbox/icons (duplicated)
rockbox/themes (duplicated)
rockbox/wps (duplicated)

All save files execep config.cfg are stored under /rockbox/data
This patch has been tested on an ipod video.

Maybe this patch tries to do to much but I got a little bit carried away since I've not had the opportunity to do som rockboxing in almost two years.

Thee plugin rockbox_files now comes a a separet diff
This task depends upon

Closed by  Jonathan Gordon (jdgordon)
Sunday, 06 June 2010, 12:39 GMT
Reason for closing:  Rejected
Additional comments about closing:  if you managed to break a rockbox install deleteing config.cfg and the .tcd files should be enough to get it working again so there is no real issue in backing up the whoel .rockbox folder
Comment by Daniel Stenberg (bagder) - Saturday, 25 August 2007, 22:11 GMT
Now we're only waiting for the actual patch too! ;-)
Comment by Jonathan Gordon (jdgordon) - Saturday, 25 August 2007, 23:50 GMT
see now ths sort of patch is much more useable than the patch which just moves the save files or playlist..
however, I think the plugin o update the file system isnt needed, and especially not needed to ru on startup every boot.
And what happens with the default files? (the eq's, wps', icons, viewers.config, tagnavi.config which are shipped int h
Comment by Henrik Backe (hbacke) - Sunday, 26 August 2007, 21:59 GMT
Well autorun of the plugin is probably an overkill, but I think that saving the score files from jewels and bubbles
is essential. Also it might keep down the support load a bit. Another thing that will keep down the support load is the
folder structure is pre generated which the plugin does, the folder structure can of course be generated by the build system.

The default files continue to live in /.rockbox where eqs, fonts, icons, themes and wps gets duplicate folders
in /rockbox (but not duplicated content). The user add his own themes and stuff to the /rockbox folders. If there is a conflict of names the one from the /.rockbox structure is used.

viewers.config and tagnavi.config are currently untouched and I think that should remain so for viewers.config.
tagnavi.config I completely forgot about but i think this file should be able to reside in both /.rockbox and /rockbox
with the /rockbox/tagnavi.config as the default and /.rockbox/tagnvai.config as the fall back. This scheme could be used on
viewers.config also but is there anyone besides developers that fiddles with it?

Comment by Dominik Riebeling (bluebrother) - Sunday, 26 August 2007, 22:09 GMT
I really dislike the two almost-identical paths "/.rockbox" and "/rockbox". IMO the user configuration (it's still configuration) should be hidden, i.e. a dot-folder. I'd suggest "/.rb-data" or "/.rockbox-config" or something similar.

Also, I don't think we need a plugin to (auto-)migrate. There hasn't been a release yet, so users should expect things to move. We previously had this (move of /rockbox.* to /.rockbox/, move of the plugins etc.) so I really don't see a reason why a plugin would help. Saving some score files is absolutely not necessary IMO as Rockbox is about playing music. Users who want to keep those files can move them manually as the won't be deleted. We had frequent resets of the configuration until we moved to text configuration anyway ...

For tagnavi.config, there is this optional user-file tagnavi_custom.config (iirc) which should go to the user folder. Perhaps just call it tagnavi.config too and add it the same way tagnavi_custom.config does now?
Comment by Henrik Backe (hbacke) - Sunday, 26 August 2007, 22:52 GMT
The naming is done thru a single define so its easy to change. Personally I don't care if its a dot file or not, but I think it should begin with rockbox and have some kind of a suffix, user or config is probably better than data but I don't really have any strong opinions about that.

The plugin is there take it or leave it, but why leave it out because we have'nt released in a while, if it helps keeping the support load down keep it, if it does nothing good for the support load skip it.

Oh, i didn't know about tagnavi_custom.config. I'll move it to the rockbox folder and I think that keeping the name tagnavi_custom.config is best to avoid any confusion on which file the user should edit.

Btw wasn't there a discussion a while ago on renaming .rockbox to rockbox to avoid the problems on Mac OS/X?
If there still are any plans to that sometime in the future /rockbox is a really bad choice and suffixing the name with .
is not such a good idea. But like I said personally I don't have a strong opinion. Another minuscule question that might erupt in vi/emacs type of flamefest, should the name have a - or a _ as delimiter? Is .rockbox_user the better than .rockbox-user ? :-)

I'll probably update the patch with tagnavi_custom and without autorun tomorrow.

Comment by Henrik Backe (hbacke) - Monday, 27 August 2007, 21:11 GMT
The patch is now updated and rockbox_upgrade_files is moved to a separate patch for easier exclusion.
The root directory for user files is changed to /rockbox_user (my favourite so far). but the question is should it be a hidden directory or not?
Comment by Dominik Riebeling (bluebrother) - Monday, 27 August 2007, 21:32 GMT
All common OSes hide user configuration data (linux by using dot-files, windows by using a different folder / the registry, etc.) so IMO the configuration folder should be a dot-folder.
Comment by Jonathan Gordon (jdgordon) - Tuesday, 28 August 2007, 02:21 GMT
Im not sure weather I agree with Dominik or not :p
they config folder has all the theme and wps files as well so its not your typical config folder and so shouldnt be hidden... but... these are small screens and real-estate is expensive, so they should be hidden in supported... (i.e make it a .folder)
Comment by Dominik Riebeling (bluebrother) - Tuesday, 28 August 2007, 07:08 GMT
Jonathan: unsurprising you don't agree with me ;-)

To strenghten my point of view, even themes are configuration data -- the user will only access them through the "browse themes" (and similar) menus. While playing music (i.e. operating the player) the contents of that folder are completely irrelevant to the user so there is no need to show it. When installing themes and similar: average users should use rbutil. More advanced users should be able coping with the fact that the folder is hidden (as the only consequence is that it's not shown as default). This worked quite fine in the past ... The main reason I see behind splitting up into a rockbox system folder and a user folder is to be able wiping the system folder completely without loosing your settings. Still it's a typical configuration folder (just think of ~/.vim -- that folder also holds user-installed plugins etc. The same applies for quite a lot of other *nix programs, so I disagree that in Rockbox' case it wouldn't be a typical configuration folder. In fact, IMO this is a typical configuration folder.)
Comment by Henrik Backe (hbacke) - Friday, 31 August 2007, 20:27 GMT
The browse cfg files change got lost somewhere but now it is back.

Since everybody seems to argue for dotting the the user folder I feel that a few arguments for not dotting are needed.

- Dot files and folders causes problems for some Mac users unzip utility.
- It is still a lot faster/easier to find the correct configuration file/font/rock by browsing the .rockbox folder than using the setting menus (I have local patch applied that unhides .rockbox). The settings structure is a lot better nowadays but I still browse the .rockbox folder, for most things, instead of using the menus.
I use the fat "hidden" attribute a lot to save screen space and make the browsing easier (I would love to be able to toggle it in the onplay menu).

Btw, somebody else needs to commit this patch (and change the folder name to whatever is agreed upon) because I have not reapplied for commit access (and I'm not the man to do the changes that are needed in build system, history has taught me that I wreak havoc every time I try to change a Makefile)
Comment by Dominik Riebeling (bluebrother) - Saturday, 01 September 2007, 09:56 GMT
Ok, so once more arguments from me ;-)

1. problems with dot files / folders on mac are a problem of broken tools on mac. Using a non-dot name might even make the solution worse, as when unzipping the build (which I assume contains the folder structure required for the user settings) will only show the user settings folder as default. This will confuse users even more (especially if that folder is called "rockbox") as they might (and I'm pretty sure a lot will) think that this is the build folder. So it's heavily error-prone that users doing a manual install will only copy the user settings (but claim to have done a correct install when asking for help). This screams for support headaches. Also, in the future Rockbox Utility should handle installation, thus removing any needs for the user to mess with that folders.
2. You might consider it faster to look at files from the PC; average users won't do that. Besides that there is no difference how a folder is called when browsing it from the PC.

Also, as already has been said, about all OSes usually hide the user configuration data (but you can browse them if you know where it's stored). Not following this might confuse users, I could even imagine users removing the /rockbox folder by thinking it's unneccesary (as the binary files live in /.rockbox) ... Additionally, I don't think it's possible to default-set the hidden flag for a folder with our current distribution format (i.e. the zip file) and I don't know of any other format that could do so. I also don't know of any way to manipulate the hidden flag from linux / macox.

To sum it up, making the configuration folder visible and naming it too similarly (rockbox vs. .rockbox) will cause user confusion and support trouble IMO. Plus I don't think we shouldn't disregard the usual way of hiding the user configuration. There are no drawbacks for experts who want to access files directly. (on windows, you could even create a shortcut "rockbox" to the hidden folder ".rb-config" or whatever that is to be called)
Comment by Jonathan Gordon (jdgordon) - Saturday, 01 September 2007, 10:31 GMT
We could of course just have <rockbox directory>/userdata/<various subfolders>... that would have the same outcome, but 1 less root folder to worry about
Comment by Henrik Backe (hbacke) - Saturday, 01 September 2007, 11:29 GMT
First i know (and knew from the beginning) that rockbox was a bad name, but it got the debate going :)

1 I consider it very arrogant to say that because the tools are broken on Mac, we should not try to make life easier for them.
But the rest of your reasoning is golden. And if we still, for some reason, do not want to hide the rockbox_user folder we must consider changing the name .rockbox to rockbox .

2 Do you mean that i should edit config.cfg on the pc to change the font for example? I meant that it is easier for me to browse the .rockbox folder to change font than to do it thru the settings menu since the .rockbox folder is not hidden in my patched version of rockbox.
Comment by Henrik Backe (hbacke) - Saturday, 01 September 2007, 11:33 GMT
Being able to remove the .rockbox folder if you somehow has screwed up your installation and still retain your users settings is
one of the main reasons for this patch, so putting it under the .rockbox folder is not a very good idea.
Comment by Henrik Backe (hbacke) - Saturday, 01 September 2007, 11:53 GMT
Also, if this change is desirable I think that applying sooner than later will cause less problems for the new themes site and rbutil-qt.
Comment by Dominik Riebeling (bluebrother) - Saturday, 01 September 2007, 12:24 GMT
1. Well, feel free to consider me arrogant, but it's a fact that there are broken tools around and while I agree it would be good to make life easier for those people (I never said we shouldn't make life easier for those! But it's not up to Rockbox to work around broken _host_ tools) you won't make it easier for them at all -- there is _still_ the .rockbox folder, and IMO by creating a second, non-hidden folder you make life even _harder_ for them: if a user extracts the zip file and doesn't see _any_ resulting folder he will most likely assume something went wrong. If he sees one folder he'll miss .rockbox and most likely not suspect that something went wrong, giving support questions like "I installed correctly and it still gives me 'rockbox.ipod not found!'". So IMO using a non-hidden folder makes life even harder, thus your attempt for making it easier is the completely wrong approach.
Besides, we are working on Rockbox Utility to make life even _much_ easier for those (and a bunch of other) users as it also installs fonts, themes, etc. We had an initial test release of Rockbox Utility just two days ago (and hopefully can release it stable soon).

1b. I absolutely disagree that the .rockbox folder should be visible by default. We had a similar issue with the file rockbox.ipod (or other extension, depending on the player) where lots of users considered this a folder and tried to change into it. Rockbox' system folder should be hidden per default to make life _easier_ for the majority of users. Showing the system files will confuse the average user and is therefore not done by about all original firmwares. Finally, changing the folder name will require a bootloader update and cause much grief. The last required change in the bootloader (coprocessor support on PP tagets) was really painful to support.

2. Ok, I misunderstood what you meant. In that case things are different but you could still use the shortcuts plugin to change to that hidden folder. Another option would be to simply introduce a "browse system folder" entry in the menu ...

3. I agree that putting the data below .rockbox defeats the main purpose of this patch.
Comment by Henrik Backe (hbacke) - Saturday, 01 September 2007, 22:08 GMT
1. I do not think you are arrogant and I apologize for coming on a little bit to strong there, It's just that I've spent far to many years involved in some kind of user support that I get going when anyone argues that because we didn't break it so we wont do anything to help (you really didn't say so, I took it out of context). And I still don't like that part of your argument just because taken out of context it gets people like me and touchy users a bit ruffled. But like I said before the rest of your reasoning is golden and you've added more golden arguments here. But I believe that Rockbox really should work around anything that causes problems for Rockbox users and that definitely includes broken host tools, it's just coming up with the best workaround. Providing rbutil is the best workaround for this problem. I realise that rbutil does a lot more and solving this Mac problem is not the main driving force for developing rbutil, but amongst other things _it_ _is_ a workaround for the broken Mac tools.

1b. I'm just saying that we should be consistent, both folders should be hidden or both folder should be visible and your arguments to keep .rockbox hidden are pretty hard to argue against so I won't.

2. I like the "browse system folder" idea (it really should be "browse user folder" with this patch applied) but I was
only truing to argue that browsing a folder structure is still a lot easier than finding things via the settings menu (plugins and maybe themes are the exception) and having rockbox_user visible would provide us with this benefit.

I'll probably write a "browse user folder" patch for myself and scrap my "visible .rockbox" patch but there needs to be bigger demand than one person (me) before I post this patch :-)
Comment by Henrik Backe (hbacke) - Tuesday, 04 September 2007, 20:46 GMT
Added chessbox.log and chessbox.png to new structure.
Changed user root folder to .rockbox_user
Comment by Henrik Backe (hbacke) - Tuesday, 11 September 2007, 20:41 GMT
R14662 broke a few bits an revealed plenty of pieces I missed so I had to update the patch.
Comment by Henrik Backe (hbacke) - Sunday, 21 October 2007, 12:32 GMT
Comment by Rosso Maltese (asettico) - Sunday, 02 March 2008, 11:58 GMT
Could you resync with current build?