Gerrit: Reliability and fidelity of verification results

On Tue, 2015-07-21 at 07:18 -0500, Norbert Thiebaud wrote:
> On Tue, Jul 21, 2015 at 1:54 AM, David Ostrovsky <d.ostrovsky@gmx.de> wrote:
> > On Mon, 2015-07-13 at 16:51 -0500, Norbert Thiebaud wrote:
> > > On Mon, Jul 13, 2015 at 1:51 PM, David Ostrovsky <d.ostrovsky@gmx.de> wrote:
> > > > > And your patch 8 would have failed the same way on tb58/tb59/tb60
> > [...]
> >
> > > The release builder have the so-called 'stale' tool chain.
> > > Just like in real-life and like other platform there is a diversity of
> > > platform and we usually do not drop
> > > support for 'older' platform unless there is an imperative motivation.
> >
> > I have another discrepancy with verification: [1].

[...]

> > On patch set 8 of this change, it's issuing -Werror=return-type: [3]
> > even though the last statement is assert(false);
>
> The last statement is (void) (which is what assert resolve to in non
> debug build...
> hence the -Werror=return-type
> The run you pointed too was not a debug one:

Thanks. It was too early in the morning and I was still sleeping,
obviously.

$ gcc -Wall -c -E foo.cxx

expands to:

int bar() {
  ((false) ? static_cast<void> (0) : __assert_fail ("false", "foo.cxx",
3, __PRETTY_FUNCTION__));
}

and in release mode (this time correctly) it expands to

$ gcc -Wall -DNDEBUG -c -E foo.cxx

int bar() {
  (static_cast<void> (0));
}

thus the warning is correct here and Boost 1.59 beta is broken in this
regard.

I hope we don't have other tinderboxes, that are verifying Gerrit
changes in debug mode, so that the breakage above wouldn't be catched,
do we? To catch such subtleties we should multiply verification platform
with 2:

* linux_rel
* linux_dbg
* mac_rel
* mac_dbg
* win_rel
* win_dbg

Oh, wait, in this thread we learned subtle differences between Mac clang
3.5 (Mavericks) and 3.6 (Yosemite), so that it wouldn't hurt to have:

mac_3.5
mac_3.6

and of course we (still?) support old 32 bit architecture on windows, so
we should have these too:

win_32bit
win_64bit

and all these with release|debug permutations. Quite a lot resources,
but we don't care, we should have enough resources for now, right?