Date: prev next · Thread: first prev next last
2011 Archives by date, by thread · List index


Hi Norbert, all,

(sorry for the delay, but of course we all have our mail archives in order ;-) )

Norbert Thiebaud wrote (26-06-11 00:59)
On Sat, Jun 25, 2011 at 4:52 PM, Cor Nouws<oolst@nouenoff.nl>  wrote:

The more QA, people, the faster it goes, of course. But that is another
matter.
And of course, QA has to be a continuous state. So a possible longer time
from freeze to release, would IMO not mean that we advise people to start
later with testing :-)

Then the question is How would 'freezing' improve that (motivate
poeple to do early testing)

Not earlier compared to the moment of freezing. But earlier compared to the
moment of releasing. Thus more time available for that specific testing.


Ok, let me try to reformulate the problem:

Formal testing of a Beta/Rc require an large amount of work just to
regression test things... A lot of these tests are not expected to
yield bug, especially after the first Beta (that is, regression would
be mostly found at that point, and hopefully very few would be
introduced by subsequent fixes)
So a test window of 1 week, will be use mostly doing that and not as
much used to discover new bugs and test new function exhaustively.

Hence the proposal become: make the release rate of Beta/RC slower.
and 5 beta/RC at two week interval would yield better result than 10
Beta/RC at one week interval.

Do I get that right ?

Hmm, that is indeed an interesting and important extending of my comments.
My comment simply is: when you have not enough people to do the QA-work in n weeks, make sure you have 2*n weeks (by releasing first beta earlier, not by releasing final later).

What you write is extremely important too:
if you need more than one week to do a lot of release tests, a new beta in just one week is not useful. (Even worse, it will frustrate testing.)

If that is indeed the crux of the argument, then I would suggest:

Expand the time between Beta/RC to two weeks (maybe 3 for the first
beta?)

When I look at:
http://wiki.documentfoundation.org/ReleasePlan/3.5
indeed there are 5 time frames of one week (\ Xmas).
So a first frame of 3 weeks, 2nd and 3rd of 2 weeks and then two of one week, looks much better. Maybe the same is valuable for the RC too: allow two weeks for the first cycle. Both for Beta and RC this larger first period is important too, since both with Beta and RC new groups of users (i.e. testers) will get involved. And setting the new environment up the first time, just takes longer.

But hey, we still did not even sum up the criteria that we need to decide what we have to do to play a rather safe game this time.

(I would also suggest release on Thursday so that feedback from
the second week-end have a chance to be fixed before the next release)

Marketing wise, release on Thursday is risky, because one day later (what happens) makes it Friday..

But then, QA still need to use Daily Build, as much as possible,
during the period to confirm that bugs found in the current Beta/RC
are indeed fixed to satisfaction _before_ the next Beta/RC comes out.
In other words, use a continuous QA process on top of a formal
Release-Based QA, to limit the number of formal Release-QA while
maintaining a good pace of fixing...

Fully agree.

Regards,

--
 - Cor
 - http://nl.libreoffice.org


Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.