Re: [Libreoffice] minutes of tech steering call ...

On Fri, Jun 24, 2011 at 6:00 PM, Cor Nouws <oolst@nouenoff.nl> wrote:
> >        Indeed - by freezing earlier we could potentially make things worse
> > ;-)
>
> Warning: you're going to loose me completely in the next paragraph.
>
> > because QA will not have started at all, and thus there will apparently
> > be no bugs to fix, and hackers will then get really stuck into working
> > on another branch, which will diverge far more from the code we're
> > releasing, such that they have less interest / ability to fix bugs in
> > the stable version by the time QA arrives in earnest so ...
>
> Do misunderstand the meaning of 'freezing'? I understand it as the point
> from where only bug fixes are allowed to the branch.
> So a longer time frame without large clean-up, re-factoring, other smart
> improvements and new features. Definitely this gives more time to find and
> fix bugs.

Well true except that it freeze _that_ branch... but it does not
_force_ people to stop working on master.

So what I understand Michael saying is:

1/ you freeze and create the 3.X branch
2/ little test is done on 3.X branch for the reasons aforementioned
3/ with little bug reporting, in the mean-time most dev go back to
hacking on master (without freeze)
4/ Just before or just after release there is a rush of QA activity
that uncover all these latent bug...
5/ by this time the dev have been working on something else for 3-6-9
weeks (pick the number of week you want the 'freeze to be')
6/ the level or interest to go back in time (code-wise) is vanishing...

iow: QA should really be a continuous process, just as dev. that is QA
should (also) be done on master, with an emphasis on pre-freeze
period, so that bugs are uncovered pre-freeze as much as possible.
Then the freeze period is the time when: we branch the code and limit
the change to it to fix-bug and QA _intensify_ right when we freeze so
that bug report pour in soon after the freeze, not 3 -4 weeks after
it.

Freezing, in my mind means: we have something close enough of being
good. we freeze and spend some time to make it release-good. but for
that to work, it need a concerted effort of all, not just dev stopping
to code on that branch. By moving the testing more toward master, we
could minimize these epic 'rush' to RC that I have noticed so far,
when all the full time dev are swamped, rushing to fix RC in time...
It is not good for them, and it is not good for the rest of us nor for
master as we are both essentially orphaned for a 3-4 week period, when
our 'champions' have no time and little patience to help and guide...

As caolan said in another post, 'master' should no be seen as
'unstable'. 'master' should be seen as the latest version about to be
released... Sure there will be time when master breaks... but that
should be rare and short-lived. And it is the Dev's responsibility to
take master breakage seriously, it is a 'culture' that we need to
develop and engrain at the core of our 'community'.
We are making progress toward that goal, notably by improving the
automation and tooling around master. Right now Linux and Mac build
breakage are typically detected in less than 30 minutes... Windows is
still a problem, but it is not hopeless... There is a lot of work to
be done to automate testing, and QA can help with that. In the end, as
master become more and more 'reliable' we should be able to convince
QA volunteers that testing Dailies is useful, that the sooner a bug is
caught the cheaper it is to fix... and if Dailies get some coverage
then the 'frozen' version will be that much more stable and there
won't need such a long and epic 'stabilization' period.

I think that one problem may also be an approach to QA: testing
dailies doe not mean 'run the formal test procedure that is needed for
the _validation_ of a Release, it means download it regularly and
'play' with it... run some test, run some work-flow you like or you
think can be tricky, run different test on the next dailies (unless
you try to verify if a bug is fixed)... heck even randomly pick a
dozen or what-many you have time for today and run these on that
daily... tomorrow, or next week-end do another set on the _then_
daily...
Again the point of the exercise is not to 'validate' a version, but to
try to stumble upon bugs as early as possible.... as a bonus side
effect that can also provide 'usability' feed-back on feature while
they are developed... again:the sooner a problem is detected the
cheaper it is to fix.

We could also 'formalize' that a bit by formally producing 'weeklies'
(i.e pick a dailies and 'promote' it) and setup Litmus so that people
could log-in and test the 'release of the week'. Of course in that
case the goal is not necessarily to cover everything every week at all
cost..


> The more QA, people, the faster it goes, of course. But that is another
> matter.
> And of course, QA has to be a continuous state. So a possible longer time
> from freeze to release, would IMO not mean that we advise people to start
> later with testing :-)

Then the question is How would 'freezing' improve that (motivate
poeple to do early testing)

> Any change that the time from freeze to release can be too long, and thus a
> waste of developer time? I can hardly imagine: if less time is needed for
> fixing bugs, more time is left for major work on the master.

Changing the freeze date has no impact what-so-ever on the amount of
bug or the time it takes to fix them...
I must be missing your point here...


> >        So, this all needs to be discussed explained; that is best done in
> > person I think.
> >
> > > However, we do not have to decide that exactly right now, do we?!
> >
> >        Quite. So - what I think we should do is to propose a joint talk on
> > the
> > topic at the LibreOffice conference in Paris in October - preferably
> > with some good assessment of the quality of master at that time; then we
> > can decide whether to move the existing (December) freeze date forward
> > or back at our leisure - and over beer.
>
> OK, half October .. till December 5 is only 7 weeks.
> Deciding at that moment, holds the risk that developers suddenly have say
> only 3 or 4 weeks left for finishing major work, in stead of 7.
> IMO, that is not fair.

That is the 'norm'... the point of continuous dev is that everyday
could be release day... but since we release fairly often the
consequence of not being ready for
a given release is not that dire...

'Major work' is typically done in a feature-branch.. and that feature
branch is merged when it is ready not when the schedule say so. The
whole idea behind 'time-based-released' is that you release what is
ready at the time you've chosen. not cram the development to squeeze
it in time....

Norbert