Touchscreen implementation progress

Why?

Several upcoming targets feature a touchscreen as their main input method, and some of these have no (useful) physical buttons at all. The Rockbox UI needs to be adapted to make use of the touchscreen on these ports.

The touchscreen is currently used as a 3x3 grid of emulated "buttons". This allows screens' keymappings to be defined in the usual manner, thus providing access to most Rockbox functionality. While this method does work (and may have advantages for some users) it is not an intuitive way to use a touchscreen, and does not allow the full potential of these devices to be used.

Users have the option to switch to "Absolute Point" mode, which is our name for a "real touchscreen" mode. This page discusses progress towards making Rockbox usable in this mode. The goal is for all screens to work in either grid or absolute mode, without forcing the user into a particular touchscreen mode.

NOTE: "Absolute Point" mode is a poor name, other suggestions are welcome!

What needs to be done

Unfortunately, every screen in rockbox will need some amount of reworking/fiddling to make it work, and there are lots of screens in rockbox, so this will take a while.

Each button loop needs to be fixed so if the returned action from action_get() is ACTION_TOUCHSCREEN it needs to figure out where the user pressed so it can decide which action the user wanted to do. On most screens this shouldnt be too hard, but some will be very tricky, (my goal is to be able to use the images on the WPS as the buttons, so they would be themeable)

Screens adapted

So, im going to try to keep this page updated instead of putting this on the tracker because it wont be ready for svn for ages.

Here are the screens which currently work with the mouse (and some help on the screen "points")

• Generic list widget (i.e any screen which uses the list, i.e settings, menus, browsers)
  • if the title is shown, clicking the title will do the cancel action, if there is no title and the statusbar is enabled, clicking the stausbar will do cancel
  • clicking an item will select it and do the ok action
  • clicking the very top left corner (battery icon if statusbar is shown) will do the menu action (usually go to the main menu)
  • clicking the scroll bar will try to scroll the screen (this needs lots of work!)
  • long press the selected item will open up the context menu for the item.
• Main menu (same as list widget)
• Settings (same as list widget)
• Menus (same as list widget)
• Browsers (same as list widget)
• Colour chooser
  • press the color box to accept the colour
  • press the title to cancel
  • touching the sliders change their values
• WPS (see below)
• QuickScreen
  • pressing the text of the 4 options should change that option.
  • any other area is exit

Screens NOT yet adapted

The following is a list of some of the screens that still need to be adapted:

• On-screen keyboard (FS#10925)
• EQ screen (FS#10639)
• Pitch screen
• FM Radio
• Yes/No screen (FS#10806)
• ...

Plugins

Currently the plugin loader forces the user into 3x3 grid mode (without any warning) before entering a plugin, and restores the user's preference on exit. This is far from ideal, since the user should not have to adjust to 3x3 grid mode to navigate a plugin's menu....

One solution would be to remove this 'forced switch' on plugin entry, so that the user's preference is maintained as far as possible (especially while browsing menus). Each plugin that goes "full screen" should be modified to explicitly set Grid or Absolute mode as appropriate, but when returning to a menu the user's preference should always be restored.

WPS

A description of the touchscreen WPS tags can be found on the CustomWPS page. A version of the CabbieV2 WPS in SVN (cabbiev2.320x240x16.mrobe500.wps) contains an example of how to use the touchscreen tags.

There is a separate question of creating a default WPS for touchscreen targets. This is particularly a problem on the Cowon D2, where CabbieV2 uses a font that is too small to be useful on its 320x240 touchscreen.

How to help

Pick a screen to adapt, then remember to upload the patch to FlySpray.

look at apps/gui/list.c for a full example on how to fix a screen, but quickly...

if (action == ACTION_TOUCHSCREEN)
{
    int x,y,button;
    get_action_statuscode(&button);
    x = (button_get_data()&0xffff0000)>>16; // 0 is left
    y = (button_get_data()&0xffff); // 0 is top
   // figure out what the user pressed
   action = ACTION_STD_?
}

Changes

• got long press working on the mouse so we can use BUTTON_REPEAT for actions. most actions should only be done on the BUTTON_REL event, so remember to check for the previous button or nasty stuff happens.
• (v3) clicking in the scroll wheel (and dragging it up and down) will now jump around in the list, it mostly works, but needs some tweaking.
• (v4) try to not do actions when the screen changes and the button is held down, add the wps control
• (v5) use Nico's v4 version. change the list to a single click interface which feels more natural
• (v6) quickscreen is working.
• (vX) comitted to SVN?
• some improvements are done (switching between TOUCHSCREEN_POINT & TOUCHSCREEN_BUTTON is done by pressing F4 in the simulator) [MaurusCuelenaere]

Comments?