Re: [Libreoffice] comprehensive binfilter tests?

On 09/26/2011 11:45 AM, Michael Meeks wrote:
> On Mon, 2011-09-26 at 10:23 +0200, Stephan Bergmann wrote:
> > With the apparently somewhat semi-automatic code clean-up/removal in
> > binfilter (removing dead code, noticing compiler warnings about thus
> > newly unused variables, thus removing more dead code, ...), I wonder
> > whether this does not introduce regressions.
>
>         Probably it does; all code change has that potential :-) but this
> should -never- stop us doing it. Arguably code that has the highest rate
> of change coupled with people working on and caring about it also has
> the best quality, certainly after a few minor point releases.
>
>         The LibreOffice approach has been to encourage people to change the
> code, and fix any knock-on problems later. True, some parts of the code
> are horribly fragile - but this is a great way of identifying and
> hardening /  fixing those pieces. The OpenOffice approach of
> discouraging code change was IMHO a short-sighted fast-track to
> obscurity, discouraging the volunteer developers crucial to progress
> from contributing.

Micheal, I do not agree with what you write above (at least not in its 
IMO exaggerated form), but I think its also irrelevant for this thread. 
 My concern here is with code that is changed semi-automatically and 
rather aggressively, with probably hardly anybody checking that the 
results are still working as intended.  So the chance of regressions 
remaining undetected for quite a while is IMO higher here than for other 
typical code changes.  (And the cost of analysing the regressions, if 
they are eventually found, will also be rather high, given the 
aggressive pruning of allegedly dead code in the meantime).

> >    Do we have some comprehensive test suite for binfilter?
>
>         No. Having said that, we don't have a -comprehensive- test suite for
> other, more useful code that people are actively using day by day; say
> the ODF import / export support ;-) or the Microsoft binary / OOXML
> filters, all of which are also under active development.
>
>         I imagine that something useful for testing binfilter's import (we
> killed export) may be quite simple - particularly since the style /
> content of the ODF it produces is expected to never change again :-)
> Potentially we could have a during-build unit-test modelled on the sc/
> filters-test that hooked the XML out of the binfilter filter directly
> [without going through a nasty re-save-as-ODF that'd introduce regular
> constant change] that would be robust, cause ~no false positives and be
> rather helpful. At least the extra confidence might help accelerate the
> rate of change and boldness with which it is made there.
>
>         Is that what you were thinking ? :-)

I wasn't thinking anything in particular, just wanted to make sure there 
is not some good regression test mechanism hidden somewhere that would 
come in handy here and of which I was not yet aware.

-Stephan