Re: [Libreoffice] 3.5 release from QA to point-zero

Hi Norbert,

Norbert Thiebaud wrote (13-09-11 23:48)
> On Tue, Sep 13, 2011 at 2:42 PM, Cor Nouws<oolst@nouenoff.nl>  wrote:
> > Thus the solution is: earlier feature freeze.
>
> I disagree that this is _the_ solution. we could feature freeze
> today... and that would not change a things unless QA is actually
> done... and reciprocally, there is no need for a feature freeze to
> start doing serious QA work... that is, even if a couple of feature
> were to be integrated in the 2 or 3 weeks you may want to extend the
> freeze, that still would translate in 95%+ of the code untouched in
> these 2 to 3 weeks.

I agree with you.

> That is why I mentioned what mmeeks translated into a 'slushy freeze',
> that is a period pre-official freeze where we self-impose an elevated
> attention to commit breakages or a 'quiet period' if you will, where
> emphasis is on bug fixing... in counterpart we need to get QA used to
> QA master. Heck I even thing we could function that wall all the way
> to RC1 (and use the notion o 'beta' just as a 'state of affair'
> buzzword, but still on master) If someone really want to commit
> invasive/dangerous things in the pre-RC period, then they should do a
> feature branch until we branch for RC1. at RC1 you get 2 full weeks to
> do a formal QA campaign, and then go on weekly RC... Again with QA
> using daily build to verify asap that bug are closed... If continuous
> QA was paid attention to, the formal RC1 QA should be mostly a matter
> of verifying that things do still work.

OK.

> I strongly believe that the best way to have a quality release is to
> have a process that tend to make RC1 QA an acceptance test rather than
> a 'let's start finding the bugs' starting point, which I feel is
> pretty much what happened for 3.4... Granted for the 3.4 release,
> master has seen pretty invasive change quite shortly prior to the
> official branching for beta ( due to m106 merge mostly), so certainly
> the dev side of the house made it very hard for QA to kick in gear in
> a timely manner.... but we will _not_ have that again for 3.5.

Indeed, that is a difference on the positive side.
On the other side, at this stage more fundamental changes in the code 
base are going on (at least that is my impression).

> What the QA team can do this time around though, is not hold a grunge
> against the mistake that were done in 3.4 and em-brass the goal that
> master be 'should deliverable every day' (1).

I do not have the idea that there is a negative sentiment in the current 
work, due to the un-lucky circumstances with 3.4

> Of course that is an extreme requirement... but the idea behind this
> motto is that we should aim to be in a position such that at any given
> day, we could decide to branch a RC1 for the next version.

That would be a sort of perfect balance between development and QA.
And of course the whole discussion is about that: the balance.

That of course was part of the initial discussion too (1).
We can't however trust that just saying "we need more QA" etc is enough.
And I know the good work that is going on on that front, and I also 
trust that we grow better gradually. Maybe I am a bit conservative in my 
expectations. And that also has to do with the balance in my feeling 
what I know from working many years with early versions, what I see 
under my fingers etc.

And because of balance: indeed a longer time for QA, or earlier feature 
freeze of course can not be _the_ solution.
In previous mails in this thread I already mentioned some things that we 
can do to try to get more QA at the right time.
But again repeating my self: I know how hard it is to have enough time 
available for that work.
And with more daily releases present, we can do better testing, but it 
also demands for more available time.
(So when Ubuntu's Scot James expects better results with monthly 
releases, he probably puts a heavy burden on the QA community (2)

I really start to wonder what Rainers indea is on this subject.

> The elephant in the room to make that (and quite frankly any other QA
> approach) a viable reality is automated testing. These are hard, and
> even harder when GUI is involved, but they are well worth it.  Stephan
> Bergmann just took up the goal of getting subsequent tests running
> again into daily build.. that is a great step in the right
> direction...

One of the many good things that we can see that is going on :-)

Regards,
Cor

> (1) that is why I do not give to much weight to the objection about
> 'moving the freeze date'. We choose a fairly rapid release schedule,
> if a feature cannot make it into one release... it will make the next
> one... six month later... it is not like missing the cut-off limit
> means 2 to 3 years delay anymore...

1) http://lists.freedesktop.org/archives/libreoffice/2011-June/014201.html
2) http://netsplit.com/2011/09/08/new-ubuntu-release-process/

--
 - Cor
 - http://nl.libreoffice.org