Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 23:52 GMT+02:00 Zolnai Tamás <zolnaitamas2000@gmail.com>:
> 2015-06-16 9:15 GMT+02:00 Stephan Bergmann <sbergman@redhat.com>:
> > On 06/15/2015 03:51 PM, Zolnai Tamás wrote:
> > > I've got an idea too: automatically generated unit tests.
> > > I think with a clang plugin we can generate unit tests for all
> > > class/struct in the source code in build time. Build system can link
> > > it as a simple unit test and run it directly after generation. If it
> > > returns with no error it can be removed. If it returns an error the
> > > test should be untouched to allow running it directly (without
> > > regeneration).
> > >
> > > These kind of tests gets in my mind now:
> > > - Create an object and remove it (check no problem with destruction)
> > > - Create and remove an object thousand of times (memory leak)
> > > - Copy operator and copy constructor result is equivalent (operator== is
> > > needed)
> > > - Copy an object, remove the original object (shallow copy)
> > > - Create a const object, call all const method on it and check it remains
> > > equal.
> > > - Getter and setter method: set a value and check the corresponding
> > > getter method returns with the same value
> > > -- Set values depending on the parameter type
> > > --- int: end points of the value range, negative value, 0, positive value
> > > --- string: empty string, dummy string, string with wild characters.
> >
> >
> > I'm not sure how exciting an approach that is.  Especially for the later
> > part of your list, it builds on the assumption that an object is merely a
> > conglomeration of public member functions, that can be called in arbitrary
> > sequence and with arbitrary arguments, and that is often just not the case.
> > The dreaded JunitTest_*_unoapi are modeled somewhat like that.
>
> Yeap, that's right. There are cases when they are not arbitrary, but
> when that's the case I think it's always a good idea to add assertions
> (or warnings or comments or anything) to the code to make assumptions
> explicit about the arguments or order of the member functions. The
> unit test generator should use these information.

I've got a good example how useful this unit test generation could be.
Next to the general test we can create tests for specific classes for
example for those which inherited from SfxPoolItem. We could test
whether setting attributes via PutValue will result in the same value
when get it back with QueryValue (with and without CONVERT_TWIPS
flag). Assertions can be added to specify the range of member ids. To
find out which member id means what kind of attributes unomap.cxx file
can be parsed (or more assertions can be added :D). With that we would
have unit test not just for existing classes, but for the future ones
too.
I'm still not sure what a clang plugin can do, so I'm speaking about
the way how it should work in theory.

Best Regards,
Tamás