We've now done five releases in the modern era. I think that in general
the release process has worked well, but I do feel that there are some
A number of people spent large amounts of time to get 3.0 as good as
possible. This included work on the manual, basic functionality testing,
going through the bug list, trying to fix obvious bugs and possibly
disabling features if they were too buggy. There were of course
problems, and maybe the release process took too long (five weeks from
freeze to release), but I think that in general this was a nice release
that we can be proud of.
In my opinion, our latest release, 3.4, was very different. Some of the
people who had been involved in the release process were busy with other
things, and we kind of forgot about it. For a while it wasn't too clear
if the freeze had already started, and we didn't do any RC builds. The
result was a very short release process (one week). Quality-wise
however, I think (although I don't really have proof, this is more of a
general feeling) that this release was not very good, and that this is
just the low point in a longer trend, i.e. releases are gradually
So what's been going on?
I think there are two main factors involved:
- We've been getting used to doing releases, and working on them is just
not as exciting as it once was
- More people use the releases, so bugs in relatively rarely used parts
of the code don't get noticed immediately anymore
Can we reverse the trend?
I suspect that we can. The main thing that has been missing recently is
testing of the soon-to-be release. This testing does not need to be very
in-depth, often a quick check is enough to spot problems. It has to be
done on every target though.
My proposal is to involve our users here, and to do the following:
- Starting from the day of the freeze, we provide RC builds.
- We provide a feedback template that helps people figure out what needs
testing, and to ensure that the resulting reports are reasonably easy
- Someone collects these reports and consolidates them to flyspray
tasks for the actual bugs.
If we do this, having a week (or more, although I don't think more is
needed) of RC time after the freeze makes sense again. If we have two
weeks between the start of the freeze (and the start of the reporting
season) and the release, I suspect that we can fix most of the obvious
bugs in time, and if not actually know about them so we can mention them
in the release notes.
For the next release, I can volunteer for providing the RC builds, for
proposing the feedback template, and for collecting and analysing the
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Received on 2009-11-19