Notice: A non well formed numeric value encountered in /sites/ on line 96 Notice: A non well formed numeric value encountered in /sites/ on line 96 Notice: A non well formed numeric value encountered in /sites/ on line 96 Deprecated: Function create_function() is deprecated in /sites/ on line 104 Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /sites/ on line 845 Deprecated: Function create_function() is deprecated in /sites/ on line 111 FS#4826 : RFC: New WPS Parser



FS#4826 - RFC: New WPS Parser

Attached to Project: Rockbox
Opened by Dan Everton (safetydan) - Monday, 13 March 2006, 20:54 GMT
Last edited by Matthias Mohr (aka Massa) (mmohr) - Tuesday, 22 August 2006, 21:50 GMT
Task Type Patches
Category Themes
Status Closed
Assigned To Dan Everton (safetydan)
Operating System All players
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No


Attached is the start of a new WPS parser. It's not totally functional yet and has issues. It's currently written to be compiled standalone. If you do

gcc -Wall -g -o gwps-tokenizer gwps-tokenizer.c && ./gwps-tokenizer

It should print out a "parse tree" of sorts.

Essentially I'm posting this here to get some feedback on whether the approach I've chosen is the right way to go. My current goals are

1. Make the WPS parsing code easier to understand and extend.
2. Parse the WPS only once and use the tokenised stream for display.

Some ideas have been taken from the WPS tokenising patch here  FS#2898 
This task depends upon

Closed by  Nicolas Pennequin (nicolas_p)
Tuesday, 20 March 2007, 19:22 GMT
Reason for closing:  Out of Date
Additional comments about closing:  See  FS#6862 
Comment by Matthias Mohr (aka Massa) (mmohr) - Tuesday, 22 August 2006, 21:57 GMT
safetydan - are you still working on this?

I'm currently in the progress in looking deeper and deeper into the current wps code.
And the deeper I look into, the more do I think that the WPS parsing code needs complete reworking ;-)

I had a look at your code and it looks much cleaner.
It seems to be a much better approach than the current code!

I have some additional ideas which we could discuss.
When are you online in IRC?
Or would a forum's talk be better for you?
Comment by Dan Everton (safetydan) - Saturday, 26 August 2006, 08:59 GMT
I haven't looked at it since I posted this task, but I agree it's a cleaner approach :) I'll be on IRC once I'm fully back from holidays.
Comment by Matthias Mohr (aka Massa) (mmohr) - Monday, 28 August 2006, 17:20 GMT
Here is my first version of a token based parser.
I used your code as base and changed it a lot :-)

I used less token types and tried to put all available tokens in an array
which also contain fucntion pointers for special parsing (if needed - there's
also a default parsing function which should work for 90% of all tags) and
there are action functions which later will be called by the wps execution
code to do the "work" for the tag (e.g. load an image and display it).

My code is able to parse basic WPS including conditionals and also conditionals
inside conditionals - but only if the conditional arguments consists only
of one token!
e.g. "%?ps<Shuffle Off|%?mh<Shuffle On|Shuffle and Hold On>>" will work
"%?ps<Shuffle Off|Shuffle%?mh<| and Hold> On>>" will not work
(second argument off %?ps contains two strings and a sub-conditional)

I didn't had a look at sublines - so I assume they will not work ;)

There are still a _lot_ TODO's, but my plan is to finish it and create
a working patch for the CVS - but this may take some time...

safetydan, I would be glad if you (or anybody else who wants to) could
have a look at my code and maybe have some ideas for a better handling
of the conditionals/subconditionals and their arguments.

I'm sure you'll find a lot of other things to improve (at least I do ;)
- it'll not be the latest incarnation of a token based parsing code :-D
Comment by Matthias Mohr (aka Massa) (mmohr) - Sunday, 10 September 2006, 17:10 GMT
No one wants to test and comment it?

That's really sad :-(
Comment by Nicolas Pennequin (nicolas_p) - Sunday, 14 January 2007, 21:07 GMT
I'm currently in the process of reworking this, using bits of both Dan's and Matthias's code. I think I'll have something presentable quite soon.
Comment by Nicolas Pennequin (nicolas_p) - Sunday, 14 January 2007, 23:55 GMT
So here's my code. I used the ideas I found most interesting in both the files above to try to make everything as simple as possible.
Basically it works like Dan's but should print a correct parse tree, and it uses Matthias's idea of an array containing all the different possible tags. This makes the code much shorter and much easier to maintain.
Also I added support for alternating sublines.
When I have time I'll try to make this into a usable patch.

Please tell me what you think about it :)