Re: fdo#63315: sign windows binaries during build

Hi David, *,

On Wed, May 7, 2014 at 10:17 AM, David Tardon <dtardon@redhat.com> wrote:
> On Tue, May 06, 2014 at 02:24:15PM +0200, Christian Lohmaier wrote:
> > On Tue, May 6, 2014 at 11:23 AM, Andras Timar <timar74@gmail.com> wrote:
> > > On Tue, May 6, 2014 at 1:56 AM, Mathias MICHEL
> > > <mathias-michel@laposte.net> wrote:
> > > > See https://bugs.freedesktop.org/show_bug.cgi?id=63315
> > > > From the Cloph's comment it seems the issue is fixed but with room for
> > > > improvement.
> >
> > The room for improvement is in the general build, not specific to
> > signing (i.e. copying files individually instead of using for example
> > tar, or at least copy with multiple files at once/recursive copy,
> > touching lots and lots of files,...) calling cp x times (creating a
> > process that often), instead of once for x files is slow as hell on
> > windows/cygwin.
>
> I would strongly object to any change that improves performance of the
> release builds at the expense of incremental builds, e.g., by dropping
> precise dependencies in order to copy more files at once.

Building SDK or Helpcontent or translations is not affected by
incremental builds usually. And of course you won't lose dependencies
by copying multiple stuff at once. You might waste some time doing i/o
by shoving around files that weren't changed, but for stuff like sdk
documentation that is created by doxygen that injects timestamps into
every file (i.e. stuff is always changed) this doesn't matter from my
POV.
Same for help. Changing help means regenerating index, so everything
will be looked up anyway, so having it in the filesystem cache doesn't
hurt for linux. But copying this in one go will help the build on
linux.

> I really do
> not want to go back to the world of "compatible" vs. "incompatible"
> builds...

That won't be the case. Copying more stuff at once for those will not
mess with dependendencies. It might result in more disk-activity, but
the areas where it matters most (sdk, help, translations) aren't
touched in everyday development, aren't even enabled in everyday
development builds.

> > LO's patched version of make did make some touch and cp builtin
> > functionality, but since gbuildification those rules don't match the
> > majority of cases.
>
> I do not really understand that... Does that mean there is something
> that actively prevents make to use the builtin functions?

No, not preventing. The code that identifies candidates that could be
handled by patched make just need adjustments to match.

https://gerrit.libreoffice.org/gitweb?p=gnu-make-lo.git;a=blob;f=runbuiltin.c;hb=refs/heads/gnu-make-lo-3.82#l454

But this is only a workaround to hide the problem with the many
processes. Removing the tons of "touch" calls would be real
improvement. And I don't see a reason why a stuff need to depend on a
touch-pseudo-target the created object itself.

> [...]
> > (see 
> > http://cgit.freedesktop.org/libreoffice/core/commit/?id=ea391abb3f2eb0ffaa892f9d7437dcf33bda6440
> > that made singing depend on slowchecks being done, as otherwise
> > signing would fail)
>
> And I strongly suspect the next stop will be gallery creation, as gengal
> is internal tool which needs a zillion libraries...

Well, even if rhetorical: If that turns out to be a problem: Yes.
Still preferred over spawning zillions of processes on windows.

> If we really want to
> continue to use this script for signing, IMHO it should be done as a
> separate step after build.

Which in effect it is. After slowcheck it is more or less separate.
Having it after the build as separate /manual step is pointless, as
then you woudn't only need to sign, but also rebuild the installsets.

And building installests on windows also is slow. So I object to
making it an "after make finished" thing.

ciao
Christian