Date: prev next · Thread: first prev next last
2015 Archives by date, by thread · List index


On 10/06/15 20:22, Norbert Thiebaud wrote:
- a test auto-induced one: when a test is unstable and produce random
failures based on circumstances... the infamous 'heisenbugs'
and heisenbug can be a systemic/design problem or can be a real bug
that is hard to trigger. either way these are not useful, and in fact
harmful in a ci context; because the human nature is 'If you can't
reproduce it is not a bug'
so the later category of real hard to trigger bug is always labeled
'systemic error' and ignored anyway... and it make people numb to
errors...
For automated testing, trust is paramount: heisenbug test failure are
the enemy, false non-failure is bad but actually less painful

Would it make sense to have a server dedicated to Heisenbugs? If a test
triggers a heisenbug, disable it on most of them but try to instrument
the heisenbug tester up the wazoo so that when it fails, there's a pile
of logs to try and work out what went wrong.

And of course, it doesn't spam failure reports generally unless someone
asks for them (the person to whom the bug is assigned should get them,
of course), but it does save those logs for forensic analysis.

Cheers,
Wol

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.