Rockbox mail archiveSubject: Re: Pre-release testing framework
Re: Pre-release testing framework
From: Lorenzo Miori <memorys60_at_gmail.com>
Date: Tue, 29 May 2012 20:48:42 +0200
Yes. Atomic tests. Not just saying playback, you're right, it's too wide.
And yes, plugin or other high level functionalities have to be
debugged later: it's a music player after all :D
2012/5/29 Frank Gevaerts <frank_at_gevaerts.be>:
> On Tue, May 29, 2012 at 06:58:53PM +0200, Bertrik Sikken wrote:
>> With respect to test strategy: In my opinion, we can start with
>> a set of rather basic tests to verify high-level behaviour, rather
>> than to go for test completeness. A test suite taking about (say)
>> one hour should keep the barrier low for people who want to
>> participate in testing.
> I agree in general, but I think one hour for the basic set of tests is
> too much.
> I stronly believe we need to have a *very* resstricted set of basic
> tests that only tests basic playback, including things like playing a
> file from the file browser or database, pause, skip, volume change, and
> not much more. After that, we can (should) have more advanced tests,
> again divided into small sets, that ideally cover all functionality.
> The reasin I think this is better than larger chunks (at least for the
> basics) is that I think we mainly need *wide* testing. We apparently
> have targets (and I'd guess these days this goes for at least half our
> stable targets) that see no testing at all during a release cycle, and
> on some of those targets we then only get two or three comments after
> the release about major issues (such as sound not working at all). We
> *need* someone to test these targets, and if there are only (say) five
> people who might be tempted to do that, fifteen minutes (for starters, I
> hope they'll continue after the basic bit) is easier to sell than an
>> Kind regards,
> "Debugging is twice as hard as writing the code in the first place.
> Therefore, if you write the code as cleverly as possible, you are,
> by definition, not smart enough to debug it." - Brian W. Kernighan
Received on 2012-05-29