Re: [Libreoffice] minutes of tech steering call ...

On Sat, Jun 25, 2011 at 4:52 PM, Cor Nouws <oolst@nouenoff.nl> wrote:
> > [...]
>
> ( Now I snip a whole lot of words from you with valuable aspects of
> QA<>developmet etc.

Yeah, sometime I tend to to suffer from logorrhea :-)

> > > The more QA, people, the faster it goes, of course. But that is another
> > > matter.
> > > And of course, QA has to be a continuous state. So a possible longer time
> > > from freeze to release, would IMO not mean that we advise people to start
> > > later with testing :-)
> >
> > Then the question is How would 'freezing' improve that (motivate
> > poeple to do early testing)
>
> Not earlier compared to the moment of freezing. But earlier compared to the
> moment of releasing. Thus more time available for that specific testing.

Ok, let me try to reformulate the problem:

Formal testing of a Beta/Rc require an large amount of work just to
regression test things... A lot of these tests are not expected to
yield bug, especially after the first Beta (that is, regression would
be mostly found at that point, and hopefully very few would be
introduced by subsequent fixes)
So a test window of 1 week, will be use mostly doing that and not as
much used to discover new bugs and test new function exhaustively.

Hence the proposal become: make the release rate of Beta/RC slower.
and 5 beta/RC at two week interval would yield better result than 10
Beta/RC at one week interval.

Do I get that right ?

If that is indeed the crux of the argument, then I would suggest:

Expand the time between Beta/RC to two weeks (maybe 3 for the first
beta?) (I would also suggest release on Thursday so that feedback from
the second week-end have a chance to be fixed before the next release)
But then, QA still need to use Daily Build, as much as possible,
during the period to confirm that bugs found in the current Beta/RC
are indeed fixed to satisfaction _before_ the next Beta/RC comes out.
In other words, use a continuous QA process on top of a formal
Release-Based QA, to limit the number of formal Release-QA while
maintaining a good pace of fixing...


Norbert

PS: with the merging of the git repos, it will become fairly trivial
to identify a 'build' using the git-sha of the commit that was used to
build it.
So if, when we close a bug in Bugzilla, we mention the commit-sha that
is associated with the fix, it should be relatively easy to determine
if a given daily build is supposed to include the fix.
Heck we could even make that a web-service: give the sha of your
version and the sha associated with a bug in bugzilla and it tells you
if that bug was meant to be fixed on that version of not....

(for example:  if [ "($git merge-base $build_sha $bugfix_sha)" =
"$bugfix_sha" ] ; then echo "$bugfix_sha Fixed in $build_sha" ; fi )