dev builds
themes manual
device status forums
mailing lists
IRC bugs
dev guide

Rockbox mail archive

Subject: Re: Pre-release testing framework

Re: Pre-release testing framework

From: Frank Gevaerts <>
Date: Tue, 29 May 2012 20:42:21 +0200

On Tue, May 29, 2012 at 06:58:53PM +0200, Bertrik Sikken wrote:
> With respect to test strategy: In my opinion, we can start with
> a set of rather basic tests to verify high-level behaviour, rather
> than to go for test completeness. A test suite taking about (say)
> one hour should keep the barrier low for people who want to
> participate in testing.

I agree in general, but I think one hour for the basic set of tests is
too much.

I stronly believe we need to have a *very* resstricted set of basic
tests that only tests basic playback, including things like playing a
file from the file browser or database, pause, skip, volume change, and
not much more. After that, we can (should) have more advanced tests,
again divided into small sets, that ideally cover all functionality.

The reasin I think this is better than larger chunks (at least for the
basics) is that I think we mainly need *wide* testing. We apparently
have targets (and I'd guess these days this goes for at least half our
stable targets) that see no testing at all during a release cycle, and
on some of those targets we then only get two or three comments after
the release about major issues (such as sound not working at all). We
*need* someone to test these targets, and if there are only (say) five
people who might be tempted to do that, fifteen minutes (for starters, I
hope they'll continue after the basic bit) is easier to sell than an


> Kind regards,
> Bertrik

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Received on 2012-05-29

Page was last modified "Jan 10 2012" The Rockbox Crew