Rockbox

Tasklist

FS#7369 - Go/Igo/Weiqi/Baduk game recorder and viewer (sgfbox)

Attached to Project: Rockbox
Opened by Joshua Simmons (mud) - Thursday, 28 June 2007, 23:40 GMT
Last edited by Mustapha Senhaji (DrMoos) - Wednesday, 11 February 2009, 16:47 GMT
Task Type Patches
Category Plugins
Status Closed
Assigned To No-one
Operating System All players
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

This patch adds the sgfbox plugin, a game recorder and viewer for Go/Igo/Weiqi/Baduk. It saves and views SGF ( http://senseis.xmp.net/?SmartGameFormat) files which are pretty much standard in the Go world.

It was developed and tested ONLY on a sansa e280, and there is no specific handling of any other platforms. In particular the button mapping is specific to the e200, and I doubt it will work well on other players (comments on what keys will work well one what players are welcome).

If you don't know what Go is, try here: http://senseis.xmp.net/?Go

I did not include any SGF files with this patch, but you can easily find some on the internet. Here are some random ones if you're lazy:
http://www.andromeda.com/people/ddyer/age-summer-94/companion.html

The bulk of this patch is ports of needed parts of GNU Go (libsgf and libboard). GNU Go is released under GPL, so there should be no issues with licensing.

Release notes:

This plugin is very much a work in progress and is /not/ ready for SVN inclusion. The porting of GNU Go code was done very sloppily and should probably be redone. A lot of code/data that isn't being used is probably included in the ported parts, making this plugin use up almost all of the 512 KB available on most ports. It will definitely not build/run on targets with less than this available for plugins.

- No rigorous testing has been done. It seems rather stable, but
I would not be at all surprised if there are huge crashes and
file corruption bugs. Don't depend on this to record games that
you care about, and don't come crying if it magically crashes your
player, deletes all your music, and burns your house down.
- Game results are not handled in any way (neither read nor written)
- Game comments and board marks are not handled
- Player and place information is not output
- The plugin reads and writes files to / for now.
- The plugin assumes that /default.sgf is an SGF file that it can use
to store the current game state if no other file is specified
- It has only been used/tested on a sansa e280
- The controls are almost certainly not going to work well on any
other player, but I really have no idea what will.
- It assumes that you have a color bitmap display, which shouldn't be
necessary
- The code is really ugly and probably breaks every guideline. If
you are brave, the main plugin code is in sgfbox.c, the rest being
mainly ports of GNU Go code. The malloc code seems to be in public
domain.

This plugin could be slimmed down quite a bit, but it's never going to be small. A board library for Go is fundamentally a non-trivial problem, if you want to be able to undo moves in a timely fashion, for several reasons including:
- One move can change a large portion of the board and you have to either keep track of what changed, or replay through the whole game from the beginning
- Keeping track of strings of stones is really annoying to do efficiently (which is necessary to handle captures)
- There's a lot of board state that is a little annoying (ko positions, captured stones)
This task depends upon

Closed by  Mustapha Senhaji (DrMoos)
Wednesday, 11 February 2009, 16:47 GMT
Reason for closing:  Accepted
Additional comments about closing:  Commited as r19972+19974. Many thanks for your works.
Comment by Joshua Simmons (mud) - Friday, 29 June 2007, 01:46 GMT
Oh I forgot to mention, there's no way to pass and there's no indication of pass moves in an SGF you're viewing.
Comment by Joshua Simmons (mud) - Friday, 29 June 2007, 12:19 GMT
New version, adds support for way more targets. Basically every target should work except for any of the archos.

Also added "outlining" of the white stones, which I think i'm going to remove and only use on monochrome players if I ever port it to them.
Comment by Joshua Simmons (mud) - Friday, 29 June 2007, 12:38 GMT
I forgot to mention: on some targets, you hold the 'play' button to reach the menu. I think that's the only really non-obvious button at this point.
Comment by Joshua Simmons (mud) - Saturday, 30 June 2007, 01:31 GMT
Renaming this plugin to 'goban'.

Added option to mark variations available from the current move (there is a toggle button on everything but ipod).

Change SUBDIRS to only build goban on targets with LCD_DEPTH >= 2, which is all of them except for the archos players.


There aren't many major features left that I want to implement. The main one is support for game results (reading and writing). After that, I will stop major development unless people request features, and will focus on code cleanup and bug fixes.

Please give me feedback on this plugin, or else any features you want will probably never be implemented (at least by me).

I am not at this time planning on fully porting GNU Go, because it is too much effort (and I hate playing Go against computers anyway, they are really bad at it). If someone would like to take up the torch in that regard, feel free. I warn you that it does not look like an easy task.
Comment by Fed (Fed) - Sunday, 22 July 2007, 17:55 GMT
I think you are missing a 'void' in the brackets for sgf_draw_all_variations (line 68 and 421).

I like the layout. I was wondering about play with a computer opponent? Are you planning on implementing this?
Comment by Joshua Simmons (mud) - Tuesday, 07 August 2007, 02:06 GMT
Fed: Thanks, yes I did forget the void. Glad to know you like the layout.

I am tentatively planning on porting the rest of GNUGo (playing against the computer), but it will be a lot of work and it won't happen quickly. I did a pretty poor job of porting the parts I already used as well, and most of that will need to be redone. In short, it will probably happen eventually but it won't be soon.
Comment by naftali (mirfle) - Saturday, 01 September 2007, 23:34 GMT
Any chance for a compiled h120 version so I can try this out?

Thanks
Comment by Fed (Fed) - Sunday, 23 September 2007, 19:24 GMT
It looks to me that your plugin no longer works with the revamped plugin setup. Can you fix this?

Thanks,

Fed
Comment by Joshua Simmons (mud) - Sunday, 23 September 2007, 23:21 GMT
I will work on this plugin in the future, but it might be a while. The reason is that I'm planning on redoing it the correct way so that I can fully port GNUgo (so you'll be able to play against the computer, as well as record games). This will require quite a bit of work, and I don't have the motivation right now because there's no tournaments I'm attending any time soon where I could show it off (and I have another plugin I want to write that's more urgent)... Look for something around February or March of 2008, I won't have much free time until then.

If someone wants to change it enough to get it to work on the new plugin system, feel free of course (that shouldn't be too difficult, I just don't have time right now).
Comment by Seamus Allan (quantummoose) - Thursday, 04 October 2007, 00:44 GMT
Hey dude, I was looking to port GnuGo to Rockbox myself until I found this thread. Are you still keen to? Do you want to do some combined work? I have 2 months where my girlfriend is overseas, so have plenty of spare time to work at something like this, and would love to.

Cheers

Seamus
Comment by Joshua Simmons (mud) - Thursday, 04 October 2007, 23:26 GMT
Seamus,

Ah, that's great that someone else is interested. The only problem is I'm really busy until after the new year, so I wouldn't have time until then at least. If you really want to do it now, feel free to use anything I've written if you find it useful.

The board drawing and user interface I thought turned out alright, that would save you that work at least. I wouldn't recommend trying to use the parts of GNUGo I've ported though (they're really messy and hackish).

Alternatively I would love to combine effort with you, if you'd like to wait until say, February of 2008.
Comment by Seamus Allan (quantummoose) - Friday, 05 October 2007, 03:08 GMT
Hrm,
well I'll have a more in depth look at the GnuGo source and see just what is involved. I don't know too much about the plugin system for Rockbox, or much about the rockbox architecture at all, so I'll have a look at all that too and see just how serious this will be. I'd just like to be able to play Go when I'm sitting on the bus/plane etc.
Cheers
S
Comment by Joshua Simmons (mud) - Tuesday, 13 November 2007, 00:27 GMT
I had a few minutes, so I made a couple of small changes.

Changes:
-Now compiles against newest svn revision (r15606), 2007-11-12.
-Fixed bad bug that broke variations
-Added an icon

Feedback welcome.

Note: not very well tested, but the old version didn't work at all.
Comment by Joshua Simmons (mud) - Wednesday, 14 November 2007, 09:07 GMT
By request of a forum user:

This is a build of rockbox with this plugin included, for the sansa e200 series players. Based on svn revision 15613, 2007-11-14.

[Attachment deleted - flyspray isn't the place to post unofficial Rockbox builds, please keep it to source code patches only]
Comment by Joshua Simmons (mud) - Tuesday, 01 April 2008, 07:39 GMT
Been a long time since the last update. Fixed the long-standing bug with save-as, and updated the patch to apply against the most recent svn revision (r16908 at time of writing). I know of no substantial bugs at this point, if anyone sees one I would appreciate it if you'd get in touch.

Soon I will be completing this plugin and then moving on to a new rockbox related project. The list of features I will finish before moving on is:
- Game metadata support (player names, game result, etc) Viewing and editing. May not be exhaustive (there is a lot of metadata you can add to SGF which I find useless, and it would be annoying to support).
- Adding support for board "marks" such as triangle and square. They probably won't be all different shapes and such, just different colors on the targets that support color. Will support drawing them as well as displaying.
- Possibly adding some support for game comments, but I don't see this feature being used very much...and it's going to be fairly annoying, so I may choose not to do this unless someone can come up with a good reason. Might only add support for reading comments.
- Removing unnecessary parts of the ported GNUGo files to save space (both RAM and hard disk)

If anyone has requests for features that they really need, now would be the time to speak up.

I've decided not to fully port GNUGo (so that you would be able to play against the computer). If someone else wants to, they are free to go ahead. I would be willing to provide assistance, but I'm just not motivated enough to do it myself (I hate playing computers anyway, and on a limited platform it's going to be even worse).
Comment by Tibor Fekete (feka) - Saturday, 25 October 2008, 15:19 GMT
Hi. Do you still support this patch? Can you make it work on the iPod nano recent build? I tried to compile it but did not succeed in doing so. Got the following error:

make[1]: *** [rocks] Error 2
make: *** [build] Error 2
Comment by Tibor Fekete (feka) - Sunday, 26 October 2008, 00:10 GMT
Any coproduction with this project? -> http://www.rockbox.org/tracker/task/7718
Comment by Joshua Simmons (mud) - Tuesday, 09 December 2008, 02:35 GMT
@feka: Nope, I assume that that author didn't know this patch existed (or didn't want to deal with my code when adding pente).

Attached is an updated copy of the plugin to work with the new build system and with a fixed prototype of plugin_start. Everything else is the same as the last patch, including the bugs. I only have an e200, but I assume it will work on the nano as well.



I'm currently doing a lot of refactoring on this plugin, in particular getting rid of all of the GNUGo code (or at least the vast majority). It's just not that suited to use on rockbox (it assumes too much memory and dynamic memory). Also, I ported it sloppily and I'd have to redo the porting regardless, so I'd rather just fix it once and for all by making my own board and SGF library.

When I'm done, any reasonable SGF file should be loadable (including very large ones such as Kogo's Joseki Dictionary).

Also I hope that this plugin will finally be ready for inclusion in the svn once I make these changes. We'll see I suppose...it will certainly be closer to ready than it is now.
Comment by Joshua Simmons (mud) - Monday, 12 January 2009, 10:07 GMT
I've gotten pretty far in my rewrite. It's still not done, but it's getting close. I believe that I have implemented all features that were in the previous version so far, plus a couple more. Here's a very very rough version if anyone is interested. It's barely been tested at all, so there are guaranteed to be large bugs and probably crashes/freezes. Don't use this on any files that you care about yet. Only tested to build/run for e200 series players, although it should theoretically work on anything with a color bitmap, and possibly on greyscale as well (not on monochrome).

Status:
- Basic game saving/loading/playing working quite well
- Large games are loadable, but take a looonnnnngg time. Do not load Kogo's Joseki Dictionary or anything of that size unless you expect your player to freeze for at least an hour, probably more. I am not kidding. Also, there is no way to abort long file loads yet. I haven't profiled to find out what the culprit in these long load times is, it's possible that it can be greatly reduced.
- Game info can be edited (wasn't available in previous version)
- No GNUGo code exists anymore, although I did copy some basic ideas from them for the board and position handling code. It was just easier to start from scratch for the board and SGF handling code.
- No malloc needed anymore (although i still have a kind of a memory pool implementation for SGF nodes and properties, it's much much simpler)

Todo list:
- Mess with the "undo" system a bit to make it possible to implement stone adding/removing
- Change sgf parsing to not use file_caches, they never should have been used there in the first place (except possibly in the parse_stack, undecided)
- Change button handling to the new action system
- Redo the board display functins to use bitmaps instead of manually drawn stones (the current code is rather a mess)
- Implement adding and removing stones
- Implement board marks
- Implement comment adding/editing (possibly, not sure how much I want to do this)
- Lots of refactoring to remove typedefs and C++ comments and such things to fit the rockbox style guide
- Testing (some tests exist already, not included in this patch)
- Create a manual and add code documentation
- Probably something else that I'm forgetting
Comment by Joshua Simmons (mud) - Monday, 12 January 2009, 10:09 GMT
well that didn't work. trying again.
Comment by Tibor Fekete (feka) - Thursday, 15 January 2009, 17:26 GMT
That is great! When I have some time and power to do so, I will patch and try it out!
Comment by Joshua Simmons (mud) - Friday, 16 January 2009, 14:14 GMT
New version:
- Add Stone and Remove Stone 99% done (you can add and remove stones fine, but they create variations which there's no way to go to if you create another variation). Need to work on variation selection for them to be very useful.
- Keypads are redone for every target which I could. Some are complete guesses, so no guarantees. (i only own an e200)
- The menu system is better, and a context menu is now available on most targets if you hold the stone placement button (usually "select" or something similar). On ipods, there aren't enough buttons so there can't be a context menu, the normal menu is instead available by holding select.
- The undo/redo/play move logic is greatly cleaned up and about a third of the size that it was
- Some buffer sizes were set to debug values in the previous build, they should be more reasonable now

almost everything is completely untested, hence the "prealpha"

Todo:
- board marks
- sgf comments
- variation selection/marking
- cleaning up display code
- removing typedefs etc.
- manual

This will probably be my last update until alpha version (all features implemented).

@feka: Glad you're still interested! Let me know if you run into any problems.
Comment by Joshua Simmons (mud) - Wednesday, 21 January 2009, 09:12 GMT
I've reached what I believe to be "feature complete", so I'm calling this the alpha version. Here's a complete list of features and such, as this flyspray entry has gotten rather confusing with many changes over the last year or two.

- Targets supported:
- Basically everything with a bitmap display mono/grayscale/color
- Small screens make viewing difficult with a 19x19 board, but anything with at least a minimum LCD dimension of 19 * 3 and a little bit of size somewhere for a footer/sidebar should work to some extent (it's up to you if it's usable or not, and small boards look great even on small displays)

- Keys set up for all targets (I think, let me know if yours isn't)
- New targets only need a keymap and the plugin should work perfectly. There is very little to no target-specific code
- Support for tall, wide, and square screens alike
- Everything is drawn dynamically, so no bitmaps are needed (and therefore none need to be updated for new targets)


- Basic features:
- Playing moves (obviously)
- Adding/removing stones (setting up go problems)
- Adding/removing/viewing common board marks
- Full variation support, including marking child variations
- ko and illegal moves
- Game info setting and reading (most common and not-so-common records supported)
- Comment adding/viewing/editing
- Full SGF support, with minimal known deviations from the specification
- Does pretty well with preserving SGF properties which are not supported (they are saved back to the file, but otherwise ignored
- Theoretically able to load any SGF file, Kogo's Joseki Dictionary has been tested to some extent and seems to work perfectly (load time is 15-30 seconds on e280, save time approximately the same)

Known defects:
- SGF board label properties (LB) with labels greater than one character long are clobbered (only the first character is retained). In practice, I've /never/ seen these used, so I'm not sure I'm going to bother to fix this. I may add a warning if this is detected.
- Variation and 'last move' marks take priority over any others. This seems like a bad choice and will be changed
- There is no way to cancel file saving/loading, and it's possible that large files may take quite a long time on some targets
- On wide screens, the "footer" text is drawn vertically even if there would be room to draw it horizontally (which is preferable)
- Some minor configurabilty would be good, such as the ability to turn on/off variation drawing, board marks, etc.

Other big todos:
- Many keymaps are not tested at all. The e200 family and ipod family keymaps I have fairly confident of, but the rest really need testing by someone who has the actual device.
- The "file caches" that I came up with are...less than perfect (mostly in performance and simplicity, no corruption bugs known). They might need to be simplified greatly, or possibly even entirely removed.
- "Test harness" is not finished yet (not included in patch), and little rigorous integration testing has been done
- Need to write a manual
- Need to put the code documentation into the code (it's separate right now and not included)
- Some rearrangement is needed to split sgf.c into separate files
- Error checking could use some looking at, mostly in sgf parsing/outputting

Attached patch is known to build/run on e200, e200 sim, and ipod sim. Any others /should/ work as well, but haven't been tested very well or at all. I can provide custom builds if anyone requests them (please build yourself if you have the ability to).
Comment by Tibor Fekete (feka) - Friday, 23 January 2009, 22:09 GMT
Hi! Unfortunately I can not compile the project. I get tons of error messages about static and non-static declarations. :-( If you could post an ipod nano version, that would be really great. Thanks!
Comment by Joshua Simmons (mud) - Saturday, 24 January 2009, 06:47 GMT
@feka Here you go. This is built against r19835 (for ipod nano), but if I understand correctly it should work until the rockbox plugin API changes. If you can't get this to work, I can post the whole .zip, but I can't do it here so try this first.

By the way, if I recall correctly the controls on the nano are as follows:
Scrolling moves forward/back in the game tree
Left/Right and Menu/Play move the cursor
Short Select is play (or whatever tool is selected)
Long Select brings up the menu.

Enjoy. There are almost definitely bugs left, so don't rely on it too much, but I've already used the alpha build a few times on e280 with no catastrophic problems.
Comment by Tibor Fekete (feka) - Saturday, 31 January 2009, 00:17 GMT
Hi!

Thanks for the plugin. It works for me. Controls as you wrote them. Nano display is very small though. :-(

A few comments:
- Can't find a way to open an existing sgf file. Since I am lazy, I prefer downloading an existing one over creating it. Can you help me? (I am sure you can. :-))

The menu looks like this:
- New
- Save
- Save as
- Game Info
- Context Menu
- Quit


- I would use the Context Menu option as the default when pressing Select. If anybody wants to use heavily the editing (that is not me), he/she should have at least this help. Stil has to press twice and choose from the larger second menu. Anyone lazy as I am (even if this is the 99 percent of users) will use the menu so rarely that it does not matter if we have to scroll to select New, or Open.

- When displaying a 19x19 board the display is not used up all the way down. 13x13 board is larger than the 19x19. Is this some kind of pixel ratio problem? I mean there are, let's say, 5 pixels for drawing a square on a smaller board then changing to a larger board the number of pixels per square drops one and then the board is much smaller then possible, because it decreases 1 pixel times the number of the squares (board size). Unfortunately 19x19 sized board does not use up the nano display space optimally...

What do you think about virtual display? Is there a technique for that in rockbox? But I guess it wouldn't be much easier to follow a game with lots of tennuki on a virtual display...

Cheers,
Feka
Comment by Joshua Simmons (mud) - Sunday, 01 February 2009, 14:37 GMT
Thanks for the comments.

"Can't find a way to open an existing sgf file. Since I am lazy, I prefer downloading an existing one over creating it. Can you help me? (I am sure you can. :-))"

There isn't really an easy way to browse for a file to load /inside/ of a plugin, so to load SGF files just use the normal Rockbox file browser and select a .sgf file. It will be opened by this plugin :). If you are already inside the plugin, unfortunately you have to exit out and go back to the file browser to load a different SGF file.

If you are using the .rock I provided you might have to use the "Open With..." functionality of the context menu (see the rockbox manual) in the file browser and select "goban". I didn't really think about that...I probably should have just given you the whole rockbox.zip, but I can't attach those here. Once I get a little further in my testing, I'll make a build for each target and link them here.

"I would use the Context Menu option as the default when pressing Select. If anybody wants to use heavily the editing (that is not me), he/she should have at least this help. Stil has to press twice and choose from the larger second menu. Anyone lazy as I am (even if this is the 99 percent of users) will use the menu so rarely that it does not matter if we have to scroll to select New, or Open."

I don't think I'd like making the "Context Menu" the default (most people probably won't use it), I'll have to think about what I can do (I agree that the current menu system isn't perfect). I've been sort of considering merging those two menus into one, but that might be rather ugly and somewhat confusing.

"When displaying a 19x19 board the display is not used up all the way down. 13x13 board is larger than the 19x19. Is this some kind of pixel ratio problem? I mean there are, let's say, 5 pixels for drawing a square on a smaller board then changing to a larger board the number of pixels per square drops one and then the board is much smaller then possible, because it decreases 1 pixel times the number of the squares (board size). Unfortunately 19x19 sized board does not use up the nano display space optimally..."

I believe what you're talking about is that the actual pixel size of the 13x13 board is larger than the 19x19 board? There is unfortunately no way around this unless I implement zooming (aka, "virtual display" which your last comment mentions). The reason is simply that the nano LCD screen height isn't a multiple of 19, so 19x19 boards can't fit perfectly. The same is true of 13x13 I believe, but 13 comes closer to dividing the LCD height perfectly so more of the screen is used. Usually, more of the screen is wasted the larger the board gets. It's also complicated by the fact that currently only odd intersection sizes are allowed (even sizes look really ugly and are annoying to draw).

In your opinion, is the plugin usable on nano (in terms of the 19x19 board being viewable at that size)? It looks usable on the nano simulator, but that's probably doesn't mean much. I haven't yet decided if I'm going to implement zooming (the downsides are code complexity and difficulty in coming up with controls if I make it user-controllable).
Comment by Joshua Simmons (mud) - Tuesday, 03 February 2009, 23:37 GMT
Alright, I implemented board zooming. The plugin now looks a ton better on targets with small screens. Zooming is controlled via the menu, so I pretty much skipped adding new keys or anything.

I'm also providing builds of rockbox with this patch included, so you can try it out if you like (@feka: this should fix any weirdness with opening SGF files with just using the .rock).
The builds can be found in the forums here: http://forums.rockbox.org/index.php?topic=20475.0 .
Notice: Do NOT report any bugs in these to the IRC channel or elsewhere in the forums, as they are UNSUPPORTED.

The patch itself is attached, please build from that yourself if you're able to.


Comment by Joshua Simmons (mud) - Thursday, 05 February 2009, 12:32 GMT
Here's a new version. Summary:
- Several small bug fixes and cleanups
- Now completely in line with the rockbox coding style (as far as I know)
- Quite a bit more testing has been done, and the associated bugs fixed
- Test procedure is finished I believe, it has only been run on e200 target and all supported sims
-

I'm calling this v1.0 beta. Other than more testing (especially of some key configurations) I don't know of anything that needs to be done at the moment.

Attached are the patch itself, the manual screenshot images (to be extracted in svn root), and some sgf files used in the test procedure (look for GBN_TEST in goban.h if you want to run the test on your target). This zip (test_sgfs.zip) is to be extracted to the target root.

As before, unofficial builds are linked to in the forum at http://forums.rockbox.org/index.php?topic=20475.0 (the manual .pdfs are now there as well if you want to check it out, or look up the controls for your player).
Comment by Tibor Fekete (feka) - Thursday, 05 February 2009, 17:15 GMT
Nice! It is much more usable indeed. I could follow three professional games already! It is really great that you could implement it so quickly.
- With the .rock version I was not able to open any files. Now I can do it.
- The best zoom level allows me to see the 99 percent of the board. I loose only the view of half of the stones on the edge of the board. Still, a bit small, so on a bus I really have to concentrate, but that is the price of such a small device as the ipod nano.

I will have my comments for the sake of perfection but that does not mean that I don't like or use the plugin as it is. :-)

- The zooming feature might be better of implemented in a bit lazier fashion. So if a zoom level is chosen next time that could be the default...
- Can you direct me to the spec that describes the proper sgf format that is needed for goban plugin? I have a lot of title games but some of them seem to be in bad sgf format. The error message is: "Unable to parse SGF file. Will overwrite." I could look into the file and check them or write a simple script to convert into the proper format.
- Game info is really broken into little parts. It is quite hard to get all the info about a game: players, their ranks, event, score, and so on each under a separate menu item.
- Might be reasonable to distinguish between editing mode and replay mode. I know it is just me, but I usually want to replay already played games from their sgf file. From this point of view is a bit annoying that each time I do this I can modify them, moreover it saves the last state when I quit the plugin.

Next thing I will try is to download some of my own and others' games from kgs and try to replay them. I will look for a revised game with comments and branches.

Keep up the good work!

Feka
Comment by Joshua Simmons (mud) - Thursday, 05 February 2009, 17:39 GMT
"The zooming feature might be better of implemented in a bit lazier fashion. So if a zoom level is chosen next time that could be the default..."

That should be possible, I'll have to think if I can do it in a consistent way (if the user loads a different size game, the old zoom probably doesn't make sense any more the way I'm defining zoom values). It shouldn't be too hard though, at worst I might just have to go back to the default in that case. There's probably another setting or two that could be saved as well, variation drawing off the top of my head.

"Can you direct me to the spec that describes the proper sgf format that is needed for goban plugin? I have a lot of title games but some of them seem to be in bad sgf format. The error message is: "Unable to parse SGF file. Will overwrite." I could look into the file and check them or write a simple script to convert into the proper format."

The specification that I've been going by is http://www.red-bean.com/sgf/ . There is also a syntax checker linked on that page, which is pretty helpful. I don't think I deviate from it /too/ much from that spec, except for some properties that I don't handle (those shouldn't result in that error ever). Would it be at all possible for you to send me the files that fail? (if they're okay to attach here, that would probably be the easiest thing, otherwise I can get you my email). I haven't found any files recently that fail, and anything I can get to break is a great chance to fix bugs.

By the way, there isn't necessarily something wrong with your files. I wouldn't be too shocked if I have parsing bugs left, although I've tried to work them all out.

"Game info is really broken into little parts. It is quite hard to get all the info about a game: players, their ranks, event, score, and so on each under a separate menu item."

Hmm, that's not a bad point. I could probably add a simple option to just show the really important stuff like player names, rules, handicap, komi, and game result on one screen.

"Might be reasonable to distinguish between editing mode and replay mode. I know it is just me, but I usually want to replay already played games from their sgf file. From this point of view is a bit annoying that each time I do this I can modify them, moreover it saves the last state when I quit the plugin."

Actually, it won't save over a loaded file unless you directly tell it to (by selecting "Save" or "Save As" and using the same name). When you exit the plugin with unsaved changes, those are saved to /sgf/gbn_def.sgf which is considered a "scratch" file by the plugin. So as long as you don't use that filename or manually save a game, it should never overwrite any files you're viewing.

With that in mind, I don't know if there's much benefit to having separate editing and viewing modes. I should definitely add something about that to the manual though, it had totally slipped my mind.

Thanks again for the comments, they really help. I have a few other people trying this out, but they haven't said too much yet. :)
Comment by Tibor Fekete (feka) - Thursday, 05 February 2009, 23:25 GMT
I really like to do this. Something is coming out of this project that I like. Please be aware of the fact that users seem to need way too many features. I am a user. So read my comments cautiously. I even have evil thoughts of hijacking the project in two ways:
- get you to implement means to solve go problems,
- get you to re-implement the previously removed gnu go features.

;-)

Saving the zoom level: Could it have a default zoom level per board size?

Sgf file problem: I'd swear I had problems with lots of my files. I attach one I had problems with but now I can open any of them. I even tried to delete the directory and copied there again, but (un?)fortunately works now. Unfortunate in a way since there seems to be some inconsistent behaviour. Either on mydevice or in my head. None is good. I will try to reproduce the problem. I experience strange things with my nano anyways. Sometimes it does not go to disk mode when I connect to the computer, for example.

Game info: Sorry for getting to the same thing again, but this can also be a place for distinguishing between play or edit mode. In play mode you might want to see the whole info on one window. In editing mode you want to add these informations and as far I can see the rockbox platform does not allow you an easy way to input text, so you might need to keep this granulated menu for adding the informations. All infos are separate text input fields the rockbox way.

Play and edit modes: Nano might be the weak point, but the navigation through the whole game is a bit hard. Here is how I use and will use the plugin: Upload a game to my rockbox and watch it on the way. Trying to find out what would I play for the next. In this scenario it is really annoying to overnavigate (to the second next move) since then I lose the joy of finding my best move. And the wheel is hard to navigate in small amounts. So I could use some "timing + autoplay" options with the possibility to stop go back and continue the autoplay again.

Saving: Good to hear it. I've just seen the 'Saving' label during exit and was lazy to check if modifications went into the game or not.
ething about that to the manual though, it had totally slipped my mind.

Just to make things clear: I am not anywhere near a good go player. Yet... :-)

I attach a game I had problems with but now I do not. Just for the record.

Feka
Comment by Joshua Simmons (mud) - Friday, 06 February 2009, 03:17 GMT
"I really like to do this. Something is coming out of this project that I like. Please be aware of the fact that users seem to need way too many features. I am a user. So read my comments cautiously. I even have evil thoughts of hijacking the project in two ways:
- get you to implement means to solve go problems,
- get you to re-implement the previously removed gnu go features."

Heh, thankfully I don't think I'll do any of those. :) They're both quite difficult, especially on the devices that rockbox supports. I'm wary of adding any more big features in general actually, but you've suggested some other good stuff. :)

Actually, it's kind of unclear from reading this FS entry, but the computer playing go was /never/ implemented at all. At some point I planned it, but quickly realized that it wasn't something that I wanted to do. The only difference in the previous version was that I was using GNUGo code for parsing and some other things. To the user, the current version contains all features of the old version and quite a few more.

"Saving the zoom level: Could it have a default zoom level per board size?"

Possibly. I don't want to make it too complicated, but that's one possibility. Right now I'm leaning towards either only using the saved setting on 19x19 boards (since that's where zoom seems to be the most important anyway), or just saving the size of stones, in pixels and then trying to match that with loaded games. The second one might be fairly nice, but I can't tell if it will sometimes give horrible behavior so I'll have to try it out.

"Sgf file problem: I'd swear I had problems with lots of my files. I attach one I had problems with but now I can open any of them."

Hmm, that is odd. I'll have a look through the code and see if anything obvious could be causing that. Off the top of my head, if it can't read the file all of the way through that can happen...or if the file format is wrong (but that would be repeatable and I think there's a more specific message for that).

Thanks for attaching that file as well. There is one obvious error in the file, but it should never cause parse failure. (The "error" is that the time limit property, "TM[]" is supposed to contain the time limit in seconds. You're not strictly allowed to use any other units so TM[13h] is technically not valid.) Otherwise the file seems mostly standard and well formed.

"Play and edit modes: Nano might be the weak point, but the navigation through the whole game is a bit hard. Here is how I use and will use the plugin: Upload a game to my rockbox and watch it on the way. Trying to find out what would I play for the next. In this scenario it is really annoying to overnavigate (to the second next move) since then I lose the joy of finding my best move. And the wheel is hard to navigate in small amounts. So I could use some "timing + autoplay" options with the possibility to stop go back and continue the autoplay again."

Hmm, that seems like more of a control system problem than a need for a new feature. It's quite difficult on the various ipods because there really aren't that many buttons to use. The only thing I could do, offhand, would be to change it so that moving the cursor left and right is handled with the scroll whell, which moving in the game tree is on the left and right buttons. Would that be preferable for you? I have been considering something like it for a while, but am wary because it gets rid of any symmetry between moving the cursor left/right and up/down. I can make a build with that control system for you if you'd like to try it out. (On my e200 I've tried it, but it's not really worth it since the e200's scroll wheel is much easier to move one space at a time).
Comment by Joshua Simmons (mud) - Saturday, 07 February 2009, 10:23 GMT
Here's v1.0-beta3 (beta2 was not released). Changes:
- Configuration file implemented
- Saving zoom level (only saves manual changes to the zoom level, not automatic)
- Saving "draw variations" toggle
- Added small "metadata summary" function to get the basic info for a game. It is shown automatically on manual file load, if there is anything interesting to show. It can also be accessed in the Game Info menu of course.

Note: I keep forgetting to mention, thanks for Llorean (Paul Louden) for some great suggestions on the IRC channel with regard to move-undo handling. These suggestions were implemented in the nocache version of v1.0 alpha.

Builds are linked to at http://forums.rockbox.org/index.php?topic=20475.0 as well as the manuals (the manual is the same as v1.0-beta, no changes...although some small ones are pending).

Have fun, and any comments are always more than welcome.
Comment by Tibor Fekete (feka) - Saturday, 07 February 2009, 22:51 GMT
You are fast man.
Info is nice.
Zoom level is saved, and you are right: it is quite enough to save generally since the 19x19 would be modified anyways by it.

You were also right when you said that I have control problems. Many of my problems now seem to be control problems arising from the poor set of available controls on the nano. The game control is best as it is now I admit. No need for special build.
With the use of variation marks I can now choose between branches. This is nice, since it uses an already used set of controls for game variations. I was about to ask for a control to switch between variations but this is a much better way to do it.

My next concern is the comment:
- I'd like to read the comments easier than it is now. Is that possible? Another control (maybe double key control) for doing this on a nano? Or could it be as a part of the game tree? I mean: One move is the move on the board, I turn the wheel over and the comment appears (up and down controls can be used to go up and down) the turn the wheel and get back to the board and the move (or the next move). This might be stupid just emphasizes the fact that sometimes the comments are as much important as the game itself.
- KJD and lots of commented games have discussions on one move. There is poor support for reading them in here. Parsing the whole stuff in one line and possibly trimming it down to some fixed length makes it hard to read them.

BTW: When opening KJD there was a message stating that: 'Stopping music playback to get more space.' What does that make? Is rockbox using some kind of swap and both applications needed swap space? How can I have music on and still work on the josekis? Can I configure something to allow this? Should I have just free disk space on may device?

When is the right time to make a plugin part of the main builds? What is the criteria? I know there are lots of plugins being much more buggy than this one woks now.

When leaving a game on, my device wont go to sleep mode. So it just keeps sucking the batteries down, That might be a major issue to work on right? Going to sleep(?), probably saving the state, then as a key is pressed coming online again maybe loading the game with its state...

PS: And I downloaded games from KGS and they work. :-)

Feka
Comment by Joshua Simmons (mud) - Saturday, 07 February 2009, 23:44 GMT
"You are fast man.
Info is nice.
Zoom level is saved, and you are right: it is quite enough to save generally since the 19x19 would be modified anyways by it."

Glad that all works :). Thankfully it was pretty easy to add.

"You were also right when you said that I have control problems. Many of my problems now seem to be control problems arising from the poor set of available controls on the nano. The game control is best as it is now I admit. No need for special build."

It's possible that I could slow down the scrolling a bit somehow on ipods, but it would be somewhat difficult in that I can't test it since I don't have an ipod at all, and also that there's two types of ipod scroll wheels I believe.

"My next concern is the comment:
- I'd like to read the comments easier than it is now. Is that possible? Another control (maybe double key control) for doing this on a nano? Or could it be as a part of the game tree? I mean: One move is the move on the board, I turn the wheel over and the comment appears (up and down controls can be used to go up and down) the turn the wheel and get back to the board and the move (or the next move). This might be stupid just emphasizes the fact that sometimes the comments are as much important as the game itself.
- KJD and lots of commented games have discussions on one move. There is poor support for reading them in here. Parsing the whole stuff in one line and possibly trimming it down to some fixed length makes it hard to read them."

Would it be enough to have comments automatically displayed when you view a move with comments? (almost certainly with a setting to turn this off). It wouldn't be perfect, as there's not a really a great way to display a lot of text in rockbox, so very long comments would simply be cut off. I'll probably implement that in the next build, it should be fairly simple.

I don't think that there's much I can do about the fixed length...it's a combination of not wanting to devote a ton of RAM to that and there not really existing a nice way to display a lot of text in a plugin... I've been considering either modifying the keyboard input thing (what's used to display/edit the comments) to use multiple lines, or making some kind of library for plugins to display a lot of text...but they're really separate from this plugin and I have no idea if I'll ever get to it.

"BTW: When opening KJD there was a message stating that: 'Stopping music playback to get more space.' What does that make? Is rockbox using some kind of swap and both applications needed swap space? How can I have music on and still work on the josekis? Can I configure something to allow this? Should I have just free disk space on may device?"

Unfortunately it's just not possible to play music at the same time as viewing SGF files with a very large amount of moves/branches/nodes/whatever (such as KJD). It's a limitation on RAM, not on hard drive space. With different algorithms than I'm using it could be possible, but it'd be quite a bit more complicated and the performance of the plugin in a lot of cases would be pretty bad I believe. What happens is that rockbox plugins have a certain amount of RAM that they can play with and leave music running, or they can steal the buffer rockbox uses to play music if they need more space. So with large SGF files I need to steal the audio buffer, and music playback isn't possible any more (while the plugin is running at least).

"When is the right time to make a plugin part of the main builds? What is the criteria? I know there are lots of plugins being much more buggy than this one woks now."

I don't think that there is a specific set of criteria. Not too long from now I plan to ask on the mailing list to try to get a developer to consider putting it in svn (meaning it'd be in the main builds). Probably after my next set of changes or two, and a little bit more testing of some SGF files I've been meaning to throw at the plugin again (I tested them once, but I've made changes since).

"When leaving a game on, my device wont go to sleep mode. So it just keeps sucking the batteries down, That might be a major issue to work on right? Going to sleep(?), probably saving the state, then as a key is pressed coming online again maybe loading the game with its state..."

Actually right now I'm purposely disabling the player from going to sleep or it would do that automatically. I could make a setting for that quite easily, which I'll do. The reason I chose not to have it go to sleep is that I mostly think of the plugin as a game recorder (simply because that's what I generally use it for), and it's quite common that a move isn't played in a tournament game for several minutes (so the device would power off between moves, which would be quite annoying). I may also add an autosave function if I do that, to automatically save the game every X minutes (probably to the gbn_def.sgf file, not the actual file)
Comment by Joshua Simmons (mud) - Sunday, 08 February 2009, 14:22 GMT
Here's v1.0-beta4. From here I'm going to redo my full test suite (the included automatic test I've already run, but I'll throw a lot of SGF files at it and see if I can break anything) and call whatever is left the 1.0 release (may or may not be the same as beta4, depending on if I find any bugs or not). After that I'm going to focus on trying to get this submitted into subversion. Any further changes, except bug fixes, will probably have to wait until after it gets submitted (assuming it ever does).

The change log is as follows:
- Added options menu, enabled poweroff timer, autosave, etc.
- Cleaned up menus to use enums instead of raw numbers.
- Fixed bug with save_game file logic.
- Got rid of TODOs which were answered questions and others that were useless.
- Fixed up game_dirty logic with regard to autosaves.
- Updated manual to reflect recent plugin changes.

The short version is that I implemented everything that I said I was going to in my last post, did some cleanups and bug fixes, and updated the manual.

Builds available at http://forums.rockbox.org/index.php?topic=20475.0 and the updated manual as well (only the .pdfs).

Have fun.
Comment by Joshua Simmons (mud) - Monday, 09 February 2009, 15:56 GMT
I split sgf.c and sgf.h into different files, it seems a bit cleaner that way. Otherwise there's little different than last version. I've done a bunch more testing (random SGF files) and still no bugs found. No new builds this time since they should be identical to the beta4 ones.
Comment by Joshua Simmons (mud) - Tuesday, 10 February 2009, 12:21 GMT
Alright, I've finished all of the testing that I can think of and I haven't found any new bugs. I'm going to call the current plugin version 1.0. No new builds, since it's the same as the last one, but I'll go ahead and rename them on the build site :).

Some files known to work:
http://waterfire.us/joseki.htm (Kogo's Joseki Dictionary)
http://xmp.net/arno/fuseki.html (Arno's Fuseki Database) Note: he's using the "N" (Node name) SGF property instead of comments, so the comments aren't visible (in my opinion that's a problem with his file, not the plugin). If you want to view the comments, replace every instance of "N[" with "C[" in the file, skipping the "GN[" near the top.
Various games saved from KGS (http://www.gokgs.com)
Various games saved from IGS (http://www.pandanet.co.jp/English/)
Pro games from http://gobase.org
A bunch of other games I have in my collection which I don't have links to.
Comment by Joshua Simmons (mud) - Tuesday, 10 February 2009, 13:25 GMT
Here's a new patch, fixed whitespace errors in a lot of places (whitespace at end of line mostly).

Also fixed a double "the" in the manual pointed out by BigBambi and a bug in my \opts pointed out by pixelma which prevented some buttons from showing up for certain targets.
Comment by Tibor Fekete (feka) - Tuesday, 10 February 2009, 20:32 GMT
Hi!

I am using beta4.

Tried to let the device power off with the plugin running displaying a game from KGS. As I turned on the device this morning and tried to start the plugin I got the same message as I would have opened the Kogo's Joseki Dictionary about stopping the music to free up space, and indeed the KJD was opened. I guess there must be a problem at saving the state. I opened a game clicking on the sgf again then quit from the plugin and opened the plugin itself. I thought if anything were to be opened it should have been the game I browsed last time. It was still the KJD. I quit again and went to the /sgf/gbn_def.sgf and opened it directly. It was the KJD. Could you make the poweroff to save the browsed game automatically before power off? Can you save the browsing details like the branch and the exact step you were at when power off happened?

Another smaller problem is that all menus under Options (the questions) are displayed higher than the previous menu title. So after choosing one of the options, the Options menu listing shows up and a few rows of pixels still have the previous question since they are not overwritten. (Hmm sounds confusing. Sorry for that.)

Hope I helped. Did not try newer patches as you said they are generally the same.

Feka
Comment by Joshua Simmons (mud) - Tuesday, 10 February 2009, 22:17 GMT
Hmm, the first paragraph that you mentioned is actually pretty expected, but maybe the behavior should change. The gbn_def.sgf file only stores any /unsaved/ state. So if you loaded another game after that but didn't change anything, even the autosaves won't change gbn_def.sgf. I will think how I should deal with that...I guess it might make sense to clear gbn_def.sgf when a new file is loaded but I am not sure. It will require a little thought. By the way, state is not saved on poweroff, it is only ever saved if you exit, or on the autosaves (which are periodic). I don't think rockbox gives my plugin a chance to save state before it powers off when the poweroff timer fires.

It's difficult to get it to load the previous state of the file (such as the branch and node number)...I could save them to the .sgf file, but I'm not in love with adding unofficial SGF properties to files (that's the only way I could do it easily I believe). The problem is that there's no "registry" for unofficial SGF properties, so I could very well conflict with other SGF readers, and I'd have no way to know...and it'd be very difficult for me to test.

You second thing is a bug, but it's a bug with rockbox's menu handling as far as I can tell. I'll look if there is a bug report for it already and add one if not.

Comment by Tibor Fekete (feka) - Tuesday, 10 February 2009, 23:30 GMT
Hi!

Yes I've found the autosave timer and set it to a short (1 minute) time. And still, the actual game is not saved only if I change something in it. I understand the reason behind it. Since you mean the plugin to be used primarily as a game recorder. As a game replaying facility it could need the state saving.

From the game replay view there is a need of saving the state of the plugin only if you quit or it is closed by the rockbox poweroff and it consists of:
- the game viewed
- the branch and node.

I did not mean some kind of bookmarking for games. As I get it, you would need to add something to the sgf file if you were to implement sgf bookmarking.

The above mentioned infos could be in a separate file, not necessarily inside the sgf. Just one file since there is enough to have one state saved at a time, the last one.

And I understand there could be problems ad errors to handle in this case.

Feka
Comment by Joshua Simmons (mud) - Tuesday, 10 February 2009, 23:46 GMT
Well, storing the state of "the branch and node" in a separate file is not really as trivial as it sounds, because you need to start a chain of "user chose variation 1, user chose variation 0, user chose variation 3", for every possibly branch point up to the place you were when you exited. The actual code for it wouldn't be too bad, but testing it would be kind of annoying. It's a lot easier to just add something to the SGF file, but I'm not sure I want to do that (adding something to the sgf file). So it's between that, which I'd rather not do, and having a separate file to keep track of it. The separate file would work, but it'd be fairly bug prone and would require a decent amount of testing. (the corner cases are pretty annoying).

Anyway, that might be a nice feature, I'm not completely against it at all, but I think it will probably wait until I can get this in svn (assuming I can interest a developer). There's just too much room for bugs in it, and I'm not totally convinced that it's worth it.

I can easily have it save /a copy of/ whatever file you were looking at in gbn_def.sgf when you exit or autosave, that might make sense (and would an extremely simple change with little testing needed).

I don't think I want it to store the actual filename that was loaded though and reuse that, because I think that that would often not be what the user wants. The gbn_def.sgf is mostly designed so that it's difficult to lose unsaved data, like if you make changes to a file or start a new one and forget to save it. But if you load the plugin without loading a file, I'm not sure it makes sense to go back to the loaded file again...the user might not want to edit that. They may have completely forgotten that they ever loaded a file before, and if they make new changes it could silently modify their old file.
Comment by Joshua Simmons (mud) - Wednesday, 11 February 2009, 15:05 GMT
Here's a new version which makes it simpler to understand what gbn_def.sgf does. It now only exists when there is data from the most recent .sgf file which is not saved anywhere else. I added a blurb to the manual as well.

New builds and manuals going up soon:
http://forums.rockbox.org/index.php?topic=20475.0

Loading...