This page is outdated. See WpsTokenizerDEPRECATED instead.
Not all wps-tags are handled in this paper.
This is only an idea, maybe this is the wrong way to do it if the wps-code should be rewritten.
this file describes a possible new methode for the wps-handling (token-based) currently this only describes the used data-struct and how the wps gets updated the biggest part will be to generate the tokens from the wps-format-file. All non-"static"-tags and text (static-tags = which needs only be parsed on wps-file-loading) will be converted to tokens. the text-token holds all non-tag text-string until the line ends or a tag was found each found tag (also "normal" text) in the wps-format will be represented by an token each token points to the next token. When an tag is used in an conditional then the token (1)points to the next token after the conditional via "next", (2) points to the first token in the conditional itself via "next_conditional" known problems in this design: - how to handle the creation of the tokens and how to manage then needed memory-space to hold all tokens (because we can't know how many tokens will be needed upon loading) - how to handle when between two tags are only a few spaces(<5) to avoid to use an extra token(if possible). pseudo-code: var strings // buffer which holds all "normal" text-string each string is null-terminated define wps_token: var type // which tag it is e.g. image-loading-tag either the tag-string(without the %/%?) or a number representation union pvar value // the value(as an pointer to the string) of the tag if type = text otherwise null var id // image id if type = image otherwise -1 var line // line-number var conditional // if found as an conditional pvar next // points to the next token if conditional = 1 it points // to the token after the conditional pvar next_conditional // points to first token in the conditional if conditional = 1 struct-size: 4 Bytes (type) + 4 Bytes (value/id) + 4 Bytes (line) + 1 Byte (conditional) + 4 Bytes (next) + 4 Bytes (next_conditional) ------------------------------ 19 Bytes function update_wps: var token = start_token var oldcharpos = 0 var after_con = null var oldline = 0 do if token.conditional // is token an conditional value = get_value(token) // get conditional value from the tag represented by the token if value != -1 // check if the tag was valid to be used in an conditional if(!after_con) // is this token the first conditional (when an conditional is in an conditional) after_con = token.next // save the token after the conditional // go to the result-token of the conditional i = 0 while i < value token = token.next_conditional i++ // check if the token is an empty one (the result isn't used) // when empty then jump to the token after the (first) conditional if(token.type == empty) token = after_con // check if the new token is not conditional-one else if(!token.conditional) handle_token(token) // handle the token token = after_con // jump after the (first) conditional after_con = null else token = token.next // go to the next token else handle_token(token) // handle the token token = token.next // go to the next token if token.line != oldline oldcharpos = 0 while token.next != null // loop until all tokens are handled update_lcd() // update the lcd with the new frame-buffer function handle_token(token): // handles non conditional tokens if token.type != text and token.type != image // if the token is not a simple-text-string or an image value = get_value(token) // get the value of the token as an string print(oldcharpos,token.line,value) // draw the value-string on the given position oldcharpos = string_length(value) // save the new string position for new to display values/text else if token.type == text // if token is an simple-text-string print(oldcharpos,token.line,token.value) // draw the text oldcharpos = string_length(token.value) // save the new string position for new to display values else if token.type == image // if token is an image draw_image(token.id) // draw the image function get_value(token) //returns the value for the wps-tag // if token.conditional is set then the "normal" value is mapped // to a conditional-value if the tag can be used in an conditional // otherwise -1 (when token.conditional = 1)NicolasPennequin - 15 Jan 2007
There are two flyspray entries about this : FS#2898 and FS#4826.
I've started work on trying to implement this, though not exactly in the way described above (I posted some preliminary code on FS#4826). IMHO it can be much simpler than what's suggested, i.e. no need to keep track of the next conditional and the next token. -- NicolasPennequin - 20 Mar 2007
I have made good progress on this. I have created a new tracker entry for my work : FS#6862. I will try to find time to edit this page to explain the concepts of the patch, and maybe add information and ideas about a possible parser plugin and bytecode WPS format.
r6 - 06 Sep 2010 - 12:04:07 - JonathanGordon
Copyright © by the contributing authors.