Re: can we get more parallelism in make check?

On 05/07/2013 11:33 AM, Noel Grandin wrote:
> On 2013-05-07 10:59, Michael Stahl wrote:
> > On 07/05/13 10:52, Noel Grandin wrote:
> > > Any build experts feel like speeding up "make check".
> > > I will definitely owe you a beverage of your choice :-)
> > >
> > > The "subsequentcheck" part of "make check" spends most of it's life
> > > executing one test at a time,
> > huh, really?  it's clearly running very much in parallel here.
> >
> > what does "grep PARALLELISM config_host.mk" say?
> >
> > cd ..apparently there are 2 variables, one is CHECK_PARALLELISM and
> > both have
> > the same value here.
>
> [noel@hexrex libo]$ grep PARALLELISM config_host.mk
> export CHECK_PARALLELISM?=6
> export PARALLELISM?=6
>
> Yes, it currently runs somewhat in parallel.
> But if you use top or a system monitor during the "subsequenctcheck"
> phase, you will note that there is very little system activity going on.
> This is because the only kind of parallelism it currently has is that it
> runs modules in parallel.
> Since (a) the bulk of the tests are in very few modules and (b) the
> tests are quite lightweight, it leads to very low system utilization.

In my experience, the long subsequentcheck phase is due to the 
JunitTest_*_{complex,unoapi} tests, that just take very long to execute 
(including various sleeps, see the qadevOOo code).  While CPU 
utilization is low, memory consumption is high due to the many (sleepy) 
soffice.bin and java processes running in parallel.

Stephan