Re: yet another unit test framework? -- was fdo#55814: unit test is missing

2013/4/4 David Ostrovsky <d.ostrovsky@gmx.de>

> [Moving this discussion to ML to have better visibility]
>
> with https://gerrit.libreoffice.org/#/c/3128/ we have support for unit
> tests written in python. (We have even found two bugs with it
> already ... and fixed).
>
> I am not going to provide the huge advantages of dynamic type languages
> in general here, but while python is very impressive it *is* truly
> read-write language compare to number of write-only languages, that used
> in LO ecosystem.

Do we really want to start a discussion on this level?


> Yes, it is probably true that you can not easily debug these unit tests.
> But is the debuggability the only argument here? I doubt it. We have
> logging framework and in the end one can still migrate python unit test
> to C++ (if needed) to debug it.
Yes debugging is the main argument for maintainability of our tests. While
it has not the nicest syntax it is at least easy for the person debugging a
failing test. And often (just according to math) the person debugging a
failing unit test is not the one who wrote it so the argument rewrite it
when you need to debug it is a bit lame.
IMO even debugging the c++ should be easier as we can see with random
people running into test failures that the common advice is to disable the
test instead of debugging it. I fear that we see this effect much more in
the python tests as more people will follow that path when a test randomly
fails (and yes every test will fail randomly at some point on a strange
platform). To some degree we have the same problem in c++ but until now we
were able to limit this behavior mainly to disabled test cases for BSD.

Also I'm not the biggest fan of the argumentation that it allows more
people to write unit tests. I still believe that tests are mainly written
after a bug has been fixed which means that the developer knows at least a
bit of C++ and with the existing testing infrastructure adding a test case
to one of the existing tests is hopefully easy enough. If it is not we
should work on making it easier to write the C++ based test. Additionally
an example out of Calc/Impress that the argumentation "make writing test
easier and magically people will show up writing them" is not true: We have
for Calc and Impress existing test frameworks that require no coding from
the person writing the test and I had exactly two persons after long
advocating at conferences who contributed one test case to Calc.

I'll stop here because I think I made my point but I have a few other
arguments. Personally I'm against python based tests especially if they are
out-of-process but as long as someone agrees to maintain them (that also
means that this person must feel responsible when a test fails in some rare
circumstances and nobody else cares) I can live with them.

Regards,
Markus