• Status Closed
  • Percent Complete
  • Task Type Patches
  • Category User Interface → Themes
  • Assigned To No-one
  • Operating System
  • Severity Low
  • Priority Very Low
  • Reported Version Unstable (example)
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: Rockbox
Opened by nathanh - 2006-01-09

FS#2898 - token parser for wps code

The token parser converts the tags from a WPS file to a
structured representation of the WPS. The display code
then renders the tokens rather than parsing the tags to
refresh the display. The purpose of the token parser is
to simplify the WPS code and pave the way for features
that are difficult (perhaps impossible) to implement
with the existing WPS code.

Closed by  nicolas_p
2007-03-20 19:25
Reason for closing:  Out of Date
Additional comments about closing:   Warning: Undefined array key "typography" in /home/rockbox/flyspray/plugins/dokuwiki/inc/parserutils.php on line 371 Warning: Undefined array key "camelcase" in /home/rockbox/flyspray/plugins/dokuwiki/inc/parserutils.php on line 407

See  FS#6862 

First version of the patch is for comment and feedback only.
The code does not yet display properly so don’t expect to
apply the patch and run the binary. However the skeleton of
what I want has been written so I’m throwing it out for

The tags from the WPS file are parsed into tokens only once.
The tokens are stored in an array using a bytecode format.
Each token is either a static element (eg, a text string or
an image) or a dynamic element (eg, an ID3 tag or the time

The tokens are stored in z-order in the array. This means
the rendering code simply runs from 0 to num_tokens. In the
future I intend to implement bounding boxes and use dirty
regions to ensure that only elements that have been dirtied
are redrawn. That comes later, after tokens work.

Considerations that are still being worked out:

  1. sublines, possibly implement as conditionals with time

constraints, collapses two special cases into one.

  1. conditionals, I will implement each option as its own

token, the display code decides which to render.

  1. refresh frequency, some tokens don’t need to refresh as

often, dirty regions will minimise this problem.

Comments are welcome.

Anonymous Submitter commented on 2006-01-09 20:09

“Do you aware of this:

Yes, of course. Stephan and I have discussed the token based
algorithm on IRC (though we both independently came to the
conclusion that it was necessary before the discussion). The
code here is similar in some ways to Stephan’s algorithm,
different in other ways. The most significant difference is
that I use tokens for everything, including static text. I
see on the wiki that MatthiasM also suggested that approach.

Anonymous Submitter commented on 2006-01-12 21:05

“The most significant difference is
that I use tokens for everything, including static text. I
see on the wiki that MatthiasM also suggested that approach.” That was me :-) And yes, I think it’s good to handle everything as token.

I think the goal should be that every token can have attributes.
attribute possibility could dependend on the type of the
token (text, image, special object, are there more?). It
could be e.g. foreground color, background color, font
attributes (e.g. bold, …), size, position, …

So I suggest that your defined tokens (WPS_TOKEN_…) all
get one of the types (e.g. id3tag-tokens are of type text,
battery level tokens are images, …)
Then define all possible attributes (WPS_ATTR_…) and group
them to type families (e.g. one enum per tokentype) so that
it’s easy to find out which attributes are allowed for which

When parsing the WPS, you have then to look for wps tags. A
wps tag is either a token itself (e.g. an id3 tag) or an
attribute. I think it would be best, when attributes
surround tokens (like in HTML).
With that we will form tokens for normal text by attributes
(e.g. when the first part of the text shall be bold and red
, the rest of the text shall be normal and green they will
form two tokens of “normal text”)
Conditionals could also be attributes for tokens.

So when first parsing the WPS the list of tokens with their
attributes could be build.
Displaying the WPS just means to go through the array entry
for entry (that’s what your code already does if I interpret
it correct), first looks at the conditional attributes and
then calls the correspondend display routine for the token type.

What do you think about my thoughts?
Are they just garbage or is something useful in them?

Matthias (aka Massa)

P.S.: I really appreciate that you started it - keep on
going! It’s really time for a new and flexible WPS parsing
and displaying!

Anonymous Submitter commented on 2006-01-20 10:16

Are you still working on it?

Anonymous Submitter commented on 2006-01-24 00:37

Is this pretty cool project aborted? or still alive?

“Is this pretty cool project aborted? or still alive?”

Not aborted. I’ve been busy with paid work and unable to
make any progress for the past few weeks. I am still
interested in completing the patch and submitting, assuming
nobody beats me to the punch.

Anonymous Submitter commented on 2006-01-24 12:55

No one is working on it, just go ahead and finish this
promising stuff please

Anonymous Submitter commented on 2006-02-07 23:12

Any news about this patch :(


Available keyboard shortcuts


Task Details

Task Editing