Rockbox

This is the bug/patch tracker for Rockbox. Click here for more information.

Quick links: Bugs · Patches · Rockbox frontpage

Tasklist

FS#9546 - UTF-8 BOM "visible" in viewer

Attached to Project: Rockbox
Opened by Robert (inrobert) - Thursday, 13 November 2008, 19:47 GMT+2
Last edited by Yoshihisa Uchida (Uchida) - Wednesday, 17 March 2010, 13:20 GMT+2
Task Type Bugs
Category Plugins
Status Closed
Assigned To No-one
Player Type All players
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Version 3.1
Due Date Undecided
Percent Complete 100%
Private No

Details

I have created some .txt files in UTF-8 format with Byte Order Marks (BOMs), using Windows XP SP3 Notepad and OpenOffice 3.0 Writer. In the viewer options I have set UTF-8 encoding. All the characters are correctly displayed, but at the beginning of each file there is strange rectangle that represents BOM. I have done a research and found out that BOM handling in Rockbox core was corrected by this patch – http://www.rockbox.org/tracker/task/6203?pagenum=3, but described issue is present in viewer plugin.

I know one can save without BOM, using Notepad2, but BOM allows Windows to recognize the UTF-8 file automatically, which is useful - for example it allows Wordpad and Lister to correctly open UTF-8 .txt files. So, as somebody is using .txt files both with Rockbox and Windows, the proper BOM handling by Rockbox is useful.
This task depends upon

Closed by  Yoshihisa Uchida (Uchida)
Wednesday, 17 March 2010, 13:20 GMT+2
Reason for closing:  Accepted
Additional comments about closing:  patch commits.
Comment by Yoshihisa Uchida (Uchida) - Wednesday, 21 January 2009, 10:20 GMT+2
I create the patch file.

Please confirm it.
   viewer_utf8.patch (5.3 KiB) 
 b/apps/misc.c           |   45 ++++++++++++++++++++++++++++++++++++++++++---
 b/apps/misc.h           |    3 +++
 b/apps/plugin.c         |    3 +++
 b/apps/plugin.h         |    7 +++++--
 b/apps/plugins/viewer.c |    6 +++---
 5 files changed, 56 insertions(+), 8 deletions(-)

Comment by Jonathan Gordon (jdgordon) - Wednesday, 21 January 2009, 10:39 GMT+2
I'm not sure that is the best way to go about fixing this....
I cant think of many places in the core where the text file position is as important for seeking as it is in viewer so I tihnk its better to fix viewer to be smarter than to waste code in the core for something which will only be used in a plugin
Comment by Dominik Riebeling (bluebrother) - Wednesday, 21 January 2009, 17:51 GMT+2
I put together a patch for this as well a bit ago (but forgot to post it here). The main problem I have with this is the fact that the viewer tries to align reads with the disk sectors. Both the above patch and my version (which is slightly more compact as it doesn't introduce a specific filesize function -- you can easily do this by using open_utf8() and then substracting the current byte position) break this functionality.
Comment by Yoshihisa Uchida (Uchida) - Thursday, 22 January 2009, 08:35 GMT+2
Gordon, Riebeling comment thanks.

My patch file found some problems.
When the problem is solved, the patch file will upload.

When I correct the patch file, *_utf8() functions will not use. (Then, viewer.c changes only)

Riebeling, because it might take time to correct the patch file, I think that you may upload your patch file where you are made without waiting for my correction.


Comment by Yoshihisa Uchida (Uchida) - Thursday, 22 January 2009, 11:25 GMT+2
Because the correction of problems was completed, a new patch file upload.
   viewer_utf8_2.patch (3.3 KiB) 
 b/apps/plugins/viewer.c |   67 +++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 61 insertions(+), 6 deletions(-)

Comment by Yoshihisa Uchida (Uchida) - Friday, 13 February 2009, 04:32 GMT+2
When the patch of  FS#9855 ,  FS#9892 ,  FS#9893 , or  FS#9898  is applied, the patch of this task cannot be applied.

Even if you forcibly appｌｙ the patch file, it doesn't operate correctly.
Oppositely, even if you apply these patches after you apply the patch of this task, it doesn't operate correctly.

Ｉｆ you want to apply the patch of  FS#9855  (or  FS#9892 ,  FS#9893 ,  FS#9898 ), you must use the patch file of  FS#9899  (this task is closed).
Comment by Yoshihisa Uchida (Uchida) - Saturday, 21 February 2009, 04:50 GMT+2
Because the patch file of  FS#9899  is old, I send a new patch.
Please apply the patch in order of  FS#9855 ,  FS#9892 ,  FS#9893 ,  FS#9898 ,  FS#9902 ,  FS#9853  and this patch.

If you do not apply these patch files, this patch need not be applied.
   viewer_utf8_3.patch (4.6 KiB) 
 b/apps/plugins/viewer.c |   81 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 71 insertions(+), 10 deletions(-)

Comment by Yoshihisa Uchida (Uchida) - Thursday, 05 March 2009, 14:36 GMT+2
sync the patch file.
   viewer_utf8_4.patch (4.6 KiB) 
 b/apps/plugins/viewer.c |   81 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 71 insertions(+), 10 deletions(-)

Comment by Yoshihisa Uchida (Uchida) - Wednesday, 17 June 2009, 11:18 GMT+2
sync r21316
   viewer_utf8_5.patch (4.6 KiB) 
 b/apps/plugins/viewer.c |   81 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 71 insertions(+), 10 deletions(-)

Comment by Yoshihisa Uchida (Uchida) - Saturday, 20 February 2010, 06:57 GMT+2
sync 24782
   viewer_utf8_6.patch (4.6 KiB) 
 b/apps/plugins/viewer.c |   82 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 72 insertions(+), 10 deletions(-)

Loading...