FS#11343 - last Arabic character in a line appears as the first for some cases

Attached to Project: Rockbox
Opened by Amr Medhat (amr) - Monday, 31 May 2010, 20:24 GMT
Task Type Bugs
Category User Interface
Status Unconfirmed
Assigned To No-one
Operating System All players
Severity Low
Priority Normal
Reported Version Release 3.4
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No


- write a word like "مسمار" in a text file, or just use the word as the file name

- the last character "ر" will appear as the first one like this: "رمسما"

- this bug appeared only since r26019

- when reverting to r26018 the bug won't show up

- for some combination of letters like "دليل" the issue happens for the last 2 characters

- for some other words the problem doesn't happen at all

- the only common case is that this happens only with last word in a line appearing on the viewport
This task depends upon

Comment by Amr Medhat (amr) - Monday, 31 May 2010, 21:26 GMT
Note: I'm testing with simulator on Linux host
Comment by benedikt goos (biengo) - Saturday, 12 June 2010, 17:37 GMT
Am I right to assume this is only related to the text_editor and text_viewer plugin, or does it show up in wps and sbs as well? Could you perhaps provide a screenshot or even a screendump from an actual device?
Comment by Amr Medhat (amr) - Sunday, 13 June 2010, 06:45 GMT
It is not related to viewing file content in a plugin only, as it shows up for file names as well.
Comment by Amr Medhat (amr) - Wednesday, 16 June 2010, 15:20 GMT
here is a screenshot for the file name "مسمار" appearing as "رمسما"
Comment by Rafaël Carré (funman) - Friday, 25 June 2010, 23:12 GMT
r26019 only affects simulator, does it work fine on target ?
Comment by Rafaël Carré (funman) - Friday, 25 June 2010, 23:30 GMT
looks fine in r27134 Clipv1 simulator, perhaps it got fixed by other Simulator updates.

Amr, please confirm that the issue is fixed and we can close this bug
Comment by Amr Medhat (amr) - Saturday, 26 June 2010, 16:09 GMT
not fixed yet for me, I test with sansa e200 simulator on Kubuntu 9.04 - kernel 2.6.28-11

I didn't test the latest build on sansa target yet , but I guess , yes, it shows up on simulator only

according to kugel, the reason might be the use of non-standard library, but I couldn't figure out which one or where in the firmware code
Comment by Rafaël Carré (funman) - Saturday, 26 June 2010, 21:50 GMT
works fine for e200 simulator too

my $LANG variable is "fr_FR.UTF-8" and i'm using ubuntu 10.04
Comment by Amr Medhat (amr) - Saturday, 26 June 2010, 22:23 GMT
how to configure this $LANG variable ?

my settings are:

settings / general settings / language = english
settings / general settings / display / default codepage = unicode (and sometimes I switch it to cp1256)
settings / theme settings / font = 16 gnu unifont
Comment by Rafaël Carré (funman) - Sunday, 27 June 2010, 00:03 GMT
LANG is a UNIX environment variable, for example:

$ echo $LANG$
$ export LANG=C
$ ./rockboxui

I didn't modify simulator settings so I'm using the default
Comment by Amr Medhat (amr) - Sunday, 27 June 2010, 18:31 GMT
my $LANG variable is en_US.UTF-8 and nothing new happened after export LANG=C

how do you test the case with the default simulator settings ?

I have to set font to GNU Unifont to view Arabic filenames,

and to view content of text file containing arabic characters in text viewer, I have to make sure codepage is either unicode or cp1256 depending on the file
Comment by Rafaël Carré (funman) - Sunday, 27 June 2010, 18:47 GMT
To reset to default settings:

$ rm simdisk/.rockbox/config.cfg
$ ./rockboxui

After that I only change font to be GNU Unifont
Comment by Amr Medhat (amr) - Sunday, 27 June 2010, 22:00 GMT
I gave up ! !

I've just built the code on an old system running fedora core 9, and it worked great and the problem didn't show up ..

I still cannot understand the reason behind the issue with me on kubuntu, and why after r26019 only .. , Could it be an update to libsdl-dev ?
Comment by Amr Medhat (amr) - Monday, 28 June 2010, 13:35 GMT
sorry, multimedia library has nothing to do with that, but it is weird issue and there is no clue why it happens .. and whether it is related to a library implementation or a system host configuration on ubuntu or what ..
Comment by Amr Medhat (amr) - Monday, 28 June 2010, 17:40 GMT
still existing !

I thought it's ok on fedora when I tried with the word " مسمار " which appeared well; but when I tried with another case, the word : دليل , the problem showed up again, and it appeared like : ل ليد (i.e. inserting the last character at the beginning of the word) ; while on ubutnu the case was replacing the last two characters not the last one only

i reviewed the joining algorithm implementation for rendering arabic text and nothing has change before or after r26019 !
Comment by Rafaël Carré (funman) - Monday, 28 June 2010, 17:54 GMT
On Ubuntu 10.04, filenames look normal in the file browser

[fun@kru ~/sim-e/simdisk]% ls
دليل مسمار
[fun@kru ~/sim-e/simdisk]% for i in *;do echo $i | xxd;done
0000000: d8af d984 d98a d984 0a .........
0000000: d985 d8b3 d985 d8a7 d8b1 0a ...........

Can you run the command with xxd to check if character encoding is the same ?
for i in *;do echo $i | xxd ; done
Comment by Amr Medhat (amr) - Monday, 28 June 2010, 20:51 GMT
your screen capture is the same as mine on fedora which displays the word دليل in a wrong way (misplacing 2 letters not 1 as i described before)

here is the command output, which is the same on both fedora and ubuntu (maybe the difference over yours is the ".txt" at the end of my files)

0000000: d8af d984 d98a d984 2e74 7874 0a .........txt.
0000000: d985 d8b3 d985 d8a7 d8b1 2e74 7874 0a ...........txt.

attached 3 screendumps :
1 - from sansa showing the correct rendering of the 2 file names (which is the same on simulator before r26019 )
2 - from simulator on fedora (one file name is correct and the other is wrong) (the same as your result)
3 - from simulator on ubuntu (both filenames are wrong)