Re: Tracing where build time is spent

On 16.02.20 15:33, Luboš Luňák wrote:
>   Hello,
>
>   I've pushed https://gerrit.libreoffice.org/c/core/+/88754 , which allows
> viewing what actually happens during building. Exact instructions are in
> https://cgit.freedesktop.org/libreoffice/core/tree/solenv/gbuild/Trace.mk .
>
>   At http://ge.tt/9z4s0J13 I've uploaded a trace
> of './autogen.sh --enable-dbgutil --disable-java && make' Windows build on 16
> HT core Ryzen 7 that takes about 70 minutes, viewable using the
> chrome://tracing URL in Chromium.
>
>   A couple of observations:
>
> - The build gets to building actual LO sources only after half the build time.
> This means that the default of building most externals ourselves is kind of
> silly, especially on Windows without ccache. It would make sense to have
> some --with-system-libs=auto which would try to use as much system stuff as
> possible and print a summary, and --enable-debug/dbgutil could default to it.
> Even on Windows, e.g. python3 is an optional component installed by MSVC, I
> don't know why we don't even try to use it.

the problem is that the MSVC's python binaries won't contain any patches 
required for LO, and may be configured with a different set of optional 
modules enabled...  probably it's good enough to pass all the unit 
tests, but i wouldn't bet on it.  oh, and will it be built with debug 
runtimes for --enable-dbgutil?

Norbert once added some support to gbuild for downloading "binary 
tarballs" but i don't think anybody used it in years.

> - About 9 minutes are spent building only nss and firebird, both apparently
> doing -j1 builds, and nss blocks most of LO sources, and nss itself is
> blocked by python3. IIRC somebody has already tried to do something about
> these and didn't manage. Nss's build page says that the recommended build is
> actually using some Mozilla build tool instead of autotools, has somebody
> tried that?

there's a humongous patch in our gerrit to make it build in parallel 
with make that really should be upstreamed because it would be quite 
unmaintainable as LO downstream patch... and also NSS has grown a 2nd 
build system using "gyp" in the meantime, which i don't know anything 
about but would presumably introduce new build dependencies.

https://gerrit.libreoffice.org/c/core/+/74751/70

> - Unittests in dbgutil take as much as half the time it takes to compile .cxx
> sources (not counting externals not built by gbuild). It could make sense to
> revisit using -O1/-Og, at least for Jenkins builds. Also, now that we do have

jenkins would very likely benefit from -Og with few if any downside.

> Jenkins builds, maybe finally plain 'make' could be sensible and not run
> tests all the time.

i have never liked the number of tests that are run via "make", 
particularly since the vast majority of them are not unit tests, but 
integration tests.  i'd like to move all integration tests to "make 
subsequentcheck", but a previous attempt at that found 2 problems:

* most jenkins builders only run "make" and not "make check"

* it introduced an additional delay while linking the serialized large 
libraries where currently some integration tests run in parallel (this 
would require some tweaks to avoid)

https://gerrit.libreoffice.org/c/core/+/31075