Re: New test to automatically importing all bugzilla documents

2013/2/19 Lubos Lunak <l.lunak@suse.cz>:
> On Monday 18 of February 2013, Markus Mohrhard wrote:
> > Currently I plan to run the script every 2 or 3 weeks on a TDF server
> > and report the documents back. Hopefully some of these tasks can be
> > automated by time and we only need someone to start the build. There
> > are several open tasks and some things I noticed would be nice to
> > have. I list them here to make is easier for others to step in and
> > implement some of them or add their own ideas:
> >
> > TODOS:
> > * Calc and Draw/Impress document import
> > * Script allowing to manage running several scripts in parallel: In
> > therory this is a task that can be done in perfect parallelisation and
> > it would help with the time needed as we have alone more than 15000
> > writer documents, at least more than 8000 calc documents ...
>
>  How long does it take to run the complete test?

The writer test files are nearly completed now after about 20 hours
with a dbgutil build. But I have been running the test in parallel
with up to 5 instances with one for each of the tested file formats.

>  I'm asking because the practice with tinderboxes is often that if one of them
> collects a larger number of commits between two rebuilds, then if a breakage
> shows up, nobody seems to feel responsible. So, if possible, I think it'd be
> better if this was run more often than 2-3 weeks, possibly as another
> tinderbox running the check.

At least for writer this should be the smaller problem right now. What
I can see from the long list of crashes that have been reproduced
already there is for now enough work on fixing the existing crashes.
Additionally this script still needs a bit of manual love and I
suspect it will finally need about 2 days for a full run with Calc,
Writer and Draw/Impress. All this limited to ODF, OOXML, the old
binary Microsoft formats and a few more formats like RTF.

The experience we have in Calc where we did such tests manually
already before the 3.5 and 4.0 release is that if the number of
documents is small enough (in our case we had about 15) and all core
developers are interested in it you can get all crashes fixed in
reasonable time. I hope that everybody feels at least responsible
enough to help clean that list before a release.

Regards,
Markus