release
dev builds
extras
themes manual
wiki
device status forums
mailing lists
IRC bugs
patches
dev guide



Search | Go
Wiki > Main > SummerOfCode > SummerOfCode2010

Google Summer of Code 2010

What?

Rockbox has been accepted to participate in GSoC 2010: the project page is here.

Accepted projects for GSoC 2010

Project name Student Mentor
Rockbox As An Application Thomas Martitz Frank Gevaerts
New WMA audio codecs Mohamed Tarek Michael Giacomelli
TTS in core Rockbox Delyan Kratunov Dominik Wenger
Graphical Rockbox Theme Editor Robert Bieber Dominik Riebeling

Information for students

If you are considering applying to Rockbox (or any other organization, in fact), you may want to have a look at this wiki page: http://code.google.com/p/google-summer-of-code/wiki/AdviceforStudents . It contains a lot of good advice about what you should expect during the Summer of Code and helps you create a good proposal.

Please use the template from GSoCApplicationTemplate2010 as a guideline when submitting your application to google.

If you're serious about working on Rockbox for SoC, then the best way you can show us is to start getting involved in the project before you make your application - i.e. download the code, hang out in our IRC channel, subscribe to our mailing lists, and start submitting patches.

Primary project ideas

The following list are the main project ideas the Rockbox project would like to see completed for SoC 2010. Applications for these projects are more likely to be accepted.

If you are a student planning on applying, please come into our IRC channel on FreeNode and talk to us about it! Please do not contact possible mentors personally at this stage.


Rockbox as an Application

Overview

Rockbox is a replacement firmware for portable audio players - it consists of a kernel, hardware drivers and a music-playing application and completely replaces the existing software on a device (although dual-booting is normally available). Portable devices which allow third party applications to run within the retail OS are becoming increasingly popular and this project would take the "application" part of Rockbox and run it on such a device.

Rockbox can currently be compiled to run as a "UI Simulator" on a PC using SDL for the UI. This can be used as a basis of this project, but requires restructuring the Rockbox code to use create a clean "Application" target, instead of a simulator.

The student is free to choose any such device to port Rockbox to, but the following are potential targets:

  • Android
  • Windows mobile
  • Embedded OS/X (iPod Touch, iPhone)
  • Plain Linux targets - e.g. Maemo or Motorola's smartphones
Some further background information is available at http://daniel.haxx.se/blog/2010/02/15/the-rockbox-future-is-an-app/

Skills involved

  • GNU Make/Perl/POSIX Shell (for the Rockbox build system)
  • C
  • SDL
  • ability to understand and refactor existing code
  • knowledge of an existing embedded environment (Android, Windows Mobile etc)

Suggested goals

A mid-term goal could be to undertake the refactoring of the existing Rockbox code and produce a Rockbox application capable of running in a dexktop environment using SDL

The remaining part of the summer could be spent porting this Rockbox application to a portable device.

Potential mentors


New WMA audio codecs

Overview

Windows Media Audio (WMA) is a set of four audio codecs:

  • WMA Standard
  • WMA Professional
  • WMA Voice
  • WMA Lossless
Rockbox currently just supports WMA Standard, as well as a parser for the ASF container format WMA audio is usually stored in.

This project would extend Rockbox's WMA support by porting existing open source WMA Professional and WMA Voice decoders from [[http://www.ffmpeg.org][ffmpeg]. There is no existing open source decoder for WMA Lossless.

During GSOC2008 a floating point WMA Pro codec was implemented for ffmpeg. This project would port this codec to Rockbox by removing all floating point code, streamlining memory usage, and adding assembler optimisations for at least some Rockbox devices (ARM, Coldfire and MIPS).

Similarly, there is a floating point WMA Voice codec in ffmpeg which would need the same treatment.

Skills needed

  • C
  • Assembler (ARM, Coldfire, MIPs)
  • Fixed-point arithmetic
  • Audio codec theory

Suggested goals

A student could either propose to port both codecs by mid-term, and then spend the rest of the summer optimising and bug-fixing, or to completely implement one codec by mid-term, and then the second by the end of summer.

We would expect by the end of the summer to have two new working codecs in Rockbox, with some degree of optimisation

Potential mentors


Audio codec optimisation

Overview

Rockbox's core function is audio playback, and the efficiency at which Rockbox can do that has a direct impact on the battery run-time of the portable devices Rockbox runs on - codecs can never run too efficiently.

A project to work on codec optimisation assumes a student with some knowledge of either assembly programming, digital signal processing, or processor optimization, but not necessarily an extensive knowledge of audio compression. Learning about fixed-point arithmetic maybe needed for some of the codecs projects too. Mentoring will be provided to bring a student's knowledge up to par during the project. If in doubt, ask in IRC about any of these proposals, or come up with one that suits you.

Possible optimisation work includes:

MP3 Codec Optimization

Rockbox has a fixed point MP3 decoder based on libmad. Unfortunately libmad is not particularly fast even after extensive optimization. For example, on ARM7TDMI, Rockbox uses 38MHz for 128k MP3. The official ARM decoder uses 29MHz, a difference of almost 25%. Other libraries exist for decoding mp3, each with their own strengths and weaknesses. In this project, a student would explore the performance of each of the major components of our decoder and then evaluate possible replacements in other codec libraries (ffmpeg, etc) or implement their own code based on algorithms in the literature. The goal would be to produce a decoders using either the best known algorithm or best known open source code for each step in the decoding process. The resulting improvements to Rockbox would reduce power consumption and provide the open source community with substantially more efficient embedded MPEG codecs.

AAC Codec Optimization

Rockbox has a fixed point AAC decoder based on libfaad. Unfortunately libfaad is not particularly fast even after extensive optimization. For example, on ARM7TDMI, Rockbox uses over 120MHz for 64kbps AAC-HE. The Helix ARM decoder uses only 60 MHz - a difference of over 100%. FFmpeg provides a much more efficient, but floating point decoder. In this project, a student would explore combining some or all of the ffmpeg decoder into Rockbox, either replacing or augmenting libfaad, but it's much preferable to replace libfaad completely. The ffmpeg decoder would be converted to fixed point and optimized for embedded decoding. The resulting improvements to Rockbox would reduce power consumption and provide the open source community with substantially more efficient embedded MPEG codecs.

Optimization for the MIPS CPU

Some new rockbox targets run on MIPS CPUs. In this project, a student would review various ARM assembly optimizations added to MP3, Vorbis, AAC, WMA and FLAC and then port them to MIPS. This would ensure that rockbox's exceptionally fast codec library can be used efficiently on MIPS CPUs, and also provide fast embedded MIPS codecs for formats that currently lack them.

Codec Library and Optimization

Create a stand alone command line version of the rockbox codec api functions so that rockbox codecs can be compiled and tested outside of rockbox, much as ffmpeg codecs can be tested from the command line. Then optimize codecs in order to fit into certain memory or CPU usage goals. Introduce standard, optimized library functions for common tasks to complement the existing IMDCT library (such as bitstream parsing, huffman decoding, etc). The goal of this project would be to improve the performance and memory usage of rockbox codecs, while making them accessible to other embedded projects.

Skills needed

  • C
  • assembly programming
  • digital signal processing
  • processor optimization
  • fixed-point arithmetic
  • audio codec theory

Suggested goals

Due to the open scope of this project idea we would rely on the student to provide project goals.

Potential mentors


Improved video playback

Overview

Rockbox currently has limited video playback via the mpegplayer plugin. This can play MPEG program streams containing MPEG-1 or MPEG-2 video and MPEG audio.

There are many areas where Rockbox's video playing capabilities can be improved, including support for more video/audio codecs, support for different container formats, making use of hardware video acceleration or integrating video playback into Rockbox's core audio playback engine (allowing mixed audio/video playlists).

An initial step that any project to enhance video playback in Rockbox would be to refactor the existing mpegplayer plugin to implement an infrastructure for multiple video/audio codec or container format support.

Some specific proposals:

  • MPEG4 (A)SP Video - many rockbox targets are probably capable of decoding MPEG4 SP video. This format is more commonly used for low resolution video than mpeg1/2, and so is a logical choice to include in rockbox.
  • VP3/Theora Video - Recently there has been interest in using Theora as a general purpose free video codec. In this project the codec would be optimized for embedded use on ARM and DSP cores providing the community with a fast embedded VP3 decoder suitable for integration into embedded systems as well as rockbox. The resulting decoder would serve as a proof of principle ARM optimized codec which could be used to evaluate Theora as an embedded codec.
  • AC-3 audio (aka Dolby Digital) - AC-3 is already supported in Rockbox and almost a factor of two faster then mp3 audio on many targets. It is also very widely combined with mpeg1/2 video. Adding support for it would enable faster decoding on slower targets.
  • AAC audio - Another audio format already supported in Rockbox and commonly used in video files.
  • MP4 container - Rockbox already has an MP4 container parser (for audio playback) which could be used as the basis of mp4 video playback.
  • Hardware Video Scaling - Some devices have hardware for resizing video. Adding an interface for hardware video scaling to rockbox would improve decoding on these targets.
A student undertaking this project could propose to implement a number of the smaller features proposed above, or a single large feature.

Skills involved

  • C
  • Assembler (ARM/Coldfire/MIPs) for target-specific optimisations
  • Fixed-point arithmetic
  • Knowledge of video/audio codec theory
The precise skills depend on the tasks the student decides to undertake for this project.

Suggested goals

The goals would depend on the precise enhancements the student proposes to implement.

However, we would expect a summer's work to involve implementing the infracture for multiple audio/video codec and container support and then adding a new video codec. In addition, existing Rockbox audio codecs (e.g. AC-3 and AAC) could be implemented as part of this project.

Potential mentors


Secondary project ideas

The following list are ideas for other projects that are wanted, but less likely to be accepted. These are only ideas and suggestions, students are free to make up and describe their own project when they apply.

NOTE: This is not a general wish list for Rockbox features. Don't add a project here without discussing it with the Rockbox developers first.

Text To Speech

Overview

Rockbox currently supports speaking menus and filenames by premade voice clips. While this feature is especially for visually impaired people it has the major drawback that it doesn't work with texts that can't get prerendered easily. Examples for such texts are the database view (metadata from the files' tags) and filenames the user changed on the player (until a new voice clip is generated).

Adding a lightweight Text To Speech (TTS) engine would allow to overcome these problems. This will make more parts of Rockbox accessible to people that can't use the usual, graphical interface. An enginge that might be suited is espeak (the last version that is GPLv2+ is 1.26). Another possiblity could be flite.

Skills involved

  • C
  • porting
  • ability to understand and refactor existing code
  • optimising code for constrained environments

Suggested goals

  • analyze the available Open Source TTS engines and identify the one to port
  • port the TTS engine to a simple Rockbox plugin. Such a plugin could render a text from a selected text file.
  • depending on the time left the TTS engine could be extended to be able of getting integrated into core Rockbox. This would also require adjusting Rockbox to allow usage of voice clips and the TTS system in parallel.

Potential mentors

  • tbd


Rockbox Playback Test Driver

Overview

Rockbox has an extremely complicated playback engine that can be run on many different devices as well as a PC simulator. Unfortunately, the PC simulator simulates the entire rockbox GUI, making it extremely difficult to reproduce some playback bugs due to timing issues and preventing automated testing. At the same time, the enormous number of features in the playback engine means that most changes seem to break obscure use cases.

Adding a simple interface for scripting rockbox actions would allow more easily reproducing problems identified on target, and allow for automated testing of changes to the playback engine. In this project a student would add the ability to compile rockbox's playback engine on PC without a GUI. Instead commands would be implemented by a test driver, a simple script that sequentially issued actions to the playback engine and recorded the result. Finally, the student would introduce a number of example tests for the playback engine involving common tasks such as changing tracks, seeking through an audio file, and building a playlist.

Skills involved

  • C
  • ability to understand and refactor existing code

Suggested goals

  • introduce a test case that replicates at least one well known but currently difficult to reproduce rockbox playback bug
  • integrate the test driver into the rockbox build system, so that each change to the rockbox source automatically tests playback for potential regressions

Potential mentors

  • Michael Giacomelli


Port buflib from the pluginlib over to the core and use it to allocate big buffers in a more dynamic way.

Overview

Rockbox has no malloc() for various reasons (WhyNoMalloc). However, time has shown that fixed size static buffers limit us a lot: Either they're too big and waste precious RAM which often directly translates into a loss of battery life or they are too small and limit our users when it comes to customizability. As Rockbox runs on a wide range of targets, some with large hard disks and some with small flash based memory, some with small greyscale screens and others with large color screens, choosing appropriate buffer sizes gets harder and harder. The pluginlib has a powerful buflib library which is handle-based able to move/resize/compact allocated and unused memory.

This project would involve porting it over to the core and adapt some parts of Rockbox to use it so that we can get rid of many fixed size buffers. Most work would be spend on making the code base aware of the fact that memory may move and to make the audio buffering engine handle an audio buffer that changes its size and location at runtime.

Skills involved

  • C
  • Refactoring existing code
  • Understanding complex code (the rockbox buffering code is not trivial)

Suggested goals

The student would be expected to port the buflib, and adapt a simple part of rockbox (e.g : a codec) to use it, as a proof-of-concept. Then work would continue by adapting more parts of rockbox as time allows, starting with the playback code.

Potential mentors


Clean up the radio code

Overview

The current radio code is mostly in apps/recorder/radio.c and is a big mess of tuning/ presets handling/ the actual radio screen/ etc. The ideal outcome of this project would be to split that one file into separate files handling radio presets, the driver layer, and a GUI layer.

http://www.rockbox.org/tracker/task/9573 might be a good starting point.

Skills involved

  • C
  • ability to understand and refactor existing code

Suggested goals

The student would be expected to extract the different parts of the existing code into separate modules and come up with a clean radio interface (building on the existing tuner_get/set() system). Another major goal would be to update the GUI code (radio_screen() ) to update it to fully use the action system and get out of the #ifdef hell which is still there. Lastly the presets should be moved out and made cleaner.

Potential Mentors


Make multi-volume targets look like a single volume (like unionfs on linux)

Overview

Many targets feature both internal storage and a slot for external storage to be inserted (SD card slot, etc). Currently Rockbox shows the SD/MMC/... card as a separate volume (on one currently in development device, it even shows an external card as the main file system and the internal one as a separate storage, because it's can't be writteb for now).

Ideally, Rockbox could (optionally) show both the internal storage and the external card(s) as a single filesystem. The volumes would of course need to have certain priorities (for filename collisions, writes, etc), however it should be presented to the user and behave as one filesystem. They are several reasons to use such a system, some of them are:

  • People often store their music files in an author/album/files hierarchy. Such a system would allow a clean and elegant merge for such users, especially if they don't use the database
  • get rid of the ugly hack currently in use (which has its uses but also its drawbacks)
  • It could perhaps allow settings override, even booting from an external card, they are lots of potential uses here...
This is not a trivial task, particularly because it probably means fiddling with different code layers like the storage API, directory and tag cache.

Skills involved

  • C
  • ability to understand complex code (the storage API is not the trivial, the directory cache one is tricky is several plaves and the tag cache is really complex, the interaction between them is important and they are even a few bugs remaining)
  • ability to understand one already submitted solution (it's a patch on the bug tracker), take the remarks into account and even reuse parts of it if possible

Suggested goals

A mid term goal could be to implement the union file system for the storage API part, not taking into account caching and not investigating too much which parts of the rockbox code have to be updated. The focus would be on functionality, user settings can be implemented later. Special care will be taken about the fact that this is dynamic: the user can eject an external card and the code has to handle that !

The remaining part of the summer could be spent on fixing the directory cache if needed and updating the rest of the code to take the unionfs into account (or not !). Updating the tag cache can also be needed (this part is really tricky, there are decisions to take).

Potential mentors


Multiple/Relocatable Plugins

Overview

Rockbox uses a plugin architecture for some of its features, mainly codecs and what are actually called "Plugins" in the user interface. Currently, only one plugin of each type can be loaded at a time. There is a demand to be able to run multiple plugins in parallel. This is not possible currently, because plugins are compiled at a fixed address at the start of the plugin buffer. The project would mean to create a framework which allows plugins to be compiled and linked with relocation information, to be resolved at load time. It would probably also mean finding new file format for the plugins, such as coff or elf (or look at elf2flt). Of course part of the job is to adapt the core plugin loader to handle more than a single plugin and a new file format too.

Reasons for this ability to run multiple plugins are:
  • Some code could be removed from the core binary, and moved into external executables (USB drivers, maybe metadata parsers as well)
  • Multiple normal plugins could run at the same time. The main case would be the ability to keep running battery_bench while still using plugins.
  • Visualisation plugins in the WPS would be possible using this.
  • Later on, this could make things like animated screen transitions (e.g. as described in http://bjorn.haxx.se/blog/2010/01/29/pluggable-rockbox-screen-transitions/) a lot easier to implement.
  • If the buflib-in-core rework (see higher) is also done, this would allow reducing the rockbox memory usage significantly
This project requires a basic understanding of the compile-assemble-link chain and to get known to various alternative binary formats. It may even require changing an existing format to fit our needs.

Skills involved

  • GNU Make (for the Rockbox build system)
  • C
  • GNU ld
  • possibly other tools such as elf2flt

Suggested goals

The mid-term goal would be to have the new loader working and demonstrated, but not necessarily fully integrated yet.

By the end of the summer, we'd expect to have both plugins and codecs working with the new system, as well as working visualisation plugins for the WPS, and possibly USB plugins.

Potential mentors


ARM Emulator

Overview

Since virtually all new DAPs coming to market contain an ARM core, a well working ARM emulator would be extremely useful many aspects of Rockbox development.

An emulator could be used to help reverse hash/encryption algorithms used in many retail firmwares (such as Apple, Sandisk, Creative players), to help determine addresses of memory mapped hardware, and to optimize codecs by allowing more detailed analysis of memory/IRAM/etc.

A simple but only semi-functional Windows emulator exists for the Sansa E200 series which emulates much of the PortalPlayer hardware. It currently can boot some builds of rockbox, but is not stable enough to be very useful.

This project could resume work on that emulator (or pick another to use), first getting it to run rockbox or a retail (Sandisk, Apple, etc) firmware without instability. This would involve a process a lot like porting rockbox, but in reverse. Starting with the drivers in Rockbox, one would need to implement the hardware devices they interact with until rockbox (and hopefully the OF) could run as if on real hardware. These would include LCD controller, various internal memories, buttons, and audio hardware.

Additional goals could include emulating the pipeline latencies of various ARM cores, emulating cache to allow memory profiling and introducing a general framework that would allow for easy adaptation to the differing memory maps of various ARM cores.

Another approach would be to have a look at SkyEye to see if we can build upon that. It is designed to emulate various ARM SOCs and could likely be improved to support a SOC used by Rockbox. As an added bonus, it can already run Linux for ARM. Given the advanced state of SkyEye, this may be a better place to start.

Skills involved

  • ARM assembler
  • C

Suggested goals

An mid-term goal would be for an emulator capable of booting and running one Rockbox target build. By the end of the summer, multiple Rockbox devices would be emulated, proving the multi-SoC capabilities of the emulator.

Potential mentors


Sansa's Various Docks

Overview

Sansa has some docks that allow remote control of the device. Some of the higher profile ones that are able to be sourced from your favorite used goods sites are: Sandisk Sansa Docking Station, Timex Clock Radio for Sansa, and Altec-Lansing inMotion im413.

The goal of this project is to allow those docks to be used to control rockbox.

Suggested first steps would be to learn the method and protocol that the dock communicates with the Sansa after receiving remote events (with substeps of possibly figuring out the proper pins to put under a scope). The probable communication methods are Serial, I2C, or USB.

Once the communication method is known, the project would progress onto understanding and implementing the protocol much like the clever folks who did the IPod Accessory Protocol (IAP). There are various details that can be learned from the forum and irc logs about previous research into this and into the IAP.

Skills involved

  • C
  • reverse engineering

Suggested goals

By mid-term, it would be good to understand at least the basic communication protocol.

At the end of the summer, rockbox should be controllable using one of the docks.

Potential mentors


Online Services

Overview

There are already a number of online services on the rockbox webpage to help the community. For example there is the Themepage to host and automatically check all those user generated themes for rockbox various devices. And there is a page to allow people to easily translate rockbox into their own language.

Improving and or adding additional services could greatly help the community. Possible tasks are:
  • Add a service to manage and handle user submitted fmpresets or generate fmpresets from a global radiofrequency database
  • Add a service to generate voicefiles using free tts systems.
  • Improving the themepage.
  • Improving the translation page, and extending it to rbutil.
  • Integrating those new services into rbutil.
Ofcouse students can also suggest other usefull services.

Skills involved

Current rockbox webservices are written with php, but other languages are also possible for new services.

  • PHP
  • SQL
  • HTML
  • ability to understand and refactor existing code

Suggested goals

A possible midterm goal could be to create one service new from scratch.

The remaining part of the summer could be spent developing a second service, or makeing big improvements on existing services.

Potential mentors


Rockbox Theme Editor

Overview

Rockbox is customizable with themes. Themes are created using a tag-based language and contributed by users (see http://themes.rockbox.org). With these tags a wide variety of aspects can get controlled. The part of Rockbox that's been customizable the longest time is the While Playing Screen (WPS). However, this language is specific to Rockbox -- you might consider it some kind of "compiled markup". This requires artists to learn that language before creating a theme is possible.

A graphical editor for theme files would greatly simplify the process of creating themes. The simplest form of such an editor would be a text editor with syntax hilighting, syntax checking of the input, immediately rendering of the theme file and shortcut buttons to insert available tags.

Skills involved

The current Theme Editor is based on Qt and written in C++. Using a different toolkit and / or programming language can be discussed, though this list assumes keeping the current ones.

  • C++
  • C
  • Qt
  • ability to understand and refactor existing code
  • Parsers and code generators
  • knowledge of Rockbox theme language might be helpful

Suggested goals

  • Create an application that checks an existing theme (using the already existing checkwps tool) and display its screens.
  • Provide a way to handle the differences between the supported players. Optimally this should allow the easy addition of new targets.
  • Extend the application to allow editing the theme file using an editor component.
  • Depending on the time left, add support for drag and drop of visual elements.

Potential mentors


Mentors

  • MichaelGiacomelli - Codec Projects
  • FrankGevaerts - Rockbox as an application, relocatable plugins, possibly other stuff
  • BryanChilds - RaaA (Android ideally) - mainly as backup mentor
  • DominikWenger - Theme editor/Online Services and backup mentor
  • PeterDHoye - FM radio and recording screen code and skinning- or backup mentor if required
  • DanielStenberg - I am not very active in the project atm, but I've been around for a while to know stuff and I'll happily assist at least as a backup mentor
  • DominikRiebeling - Theme Editor, possibly others as backup mentor

Backup mentors

r77 - 02 Apr 2021 - 20:46:07 - UnknownUser


Parents: SummerOfCode
Copyright © by the contributing authors.