On Tue, May 29, 2012 at 08:48:42PM +0200, Lorenzo Miori wrote:
> Yes. Atomic tests. Not just saying playback, you're right, it's too wide.

Yes, but that wasn't really my point :) What I mean is that we should
have a *small* basic test suite (which, yes, consists of clear
unambiguous and simple tests) that users can go through quickly in a few
minutes (ok, that's probably overoptimistic). As soon as they're through
that, we have usable results (not many, but today we have nothing) and
the user may well go on to the next (also reasonably short) set of less
basic tests (e.g. "DSP effects", or "bookmarking", or "the demo
plugins").

Keeping the sets short will avoid people getting discouraged before they
even get started, both on actual testing and on adding new tests, while
I don't see any disadvantages.

Of course some tests will take longer ("battery life not worse than in
previous release", "Zork is fully playable on the frotz plugin", ...)
think that's a reason not to try to keep the rest short.

Frank

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan