Re: fdo#63315: sign windows binaries during build

Hi,

On Wed, May 07, 2014 at 10:41:59AM +0200, Christian Lohmaier wrote:
> Hi David, *,
>
> On Wed, May 7, 2014 at 10:17 AM, David Tardon <dtardon@redhat.com> wrote:
> > On Tue, May 06, 2014 at 02:24:15PM +0200, Christian Lohmaier wrote:
> > > On Tue, May 6, 2014 at 11:23 AM, Andras Timar <timar74@gmail.com> wrote:
> > > > On Tue, May 6, 2014 at 1:56 AM, Mathias MICHEL
> > > > <mathias-michel@laposte.net> wrote:
> > > > > See https://bugs.freedesktop.org/show_bug.cgi?id=63315
> > > > > From the Cloph's comment it seems the issue is fixed but with room for
> > > > > improvement.
> > >
> > > The room for improvement is in the general build, not specific to
> > > signing (i.e. copying files individually instead of using for example
> > > tar, or at least copy with multiple files at once/recursive copy,
> > > touching lots and lots of files,...) calling cp x times (creating a
> > > process that often), instead of once for x files is slow as hell on
> > > windows/cygwin.
> >
> > I would strongly object to any change that improves performance of the
> > release builds at the expense of incremental builds, e.g., by dropping
> > precise dependencies in order to copy more files at once.
>
> Building SDK or Helpcontent or translations is not affected by
> incremental builds usually. And of course you won't lose dependencies
> by copying multiple stuff at once. You might waste some time doing i/o
> by shoving around files that weren't changed, but for stuff like sdk
> documentation that is created by doxygen that injects timestamps into
> every file (i.e. stuff is always changed) this doesn't matter from my
> POV.

SDK docs are already delivered using gb_GeneratedPackage, which copies
whole directories, because the file names are random.

> Same for help. Changing help means regenerating index, so everything
> will be looked up anyway, so having it in the filesystem cache doesn't
> hurt for linux. But copying this in one go will help the build on
> linux.

Actually copying all the generated help files at once would even make
the gbuild code simpler... I will add this to my TODO list.

> > [...]
> > > (see 
> > > http://cgit.freedesktop.org/libreoffice/core/commit/?id=ea391abb3f2eb0ffaa892f9d7437dcf33bda6440
> > > that made singing depend on slowchecks being done, as otherwise
> > > signing would fail)
> >
> > And I strongly suspect the next stop will be gallery creation, as gengal
> > is internal tool which needs a zillion libraries...
>
> Well, even if rhetorical: If that turns out to be a problem: Yes.
> Still preferred over spawning zillions of processes on windows.
>
> > If we really want to
> > continue to use this script for signing, IMHO it should be done as a
> > separate step after build.
>
> Which in effect it is. After slowcheck it is more or less separate.

Except that it isn't. There are still tasks that are not needed by unit
tests, so they could end up running in parallel with the signing. I have
already mentioned gallery creation, but there is generation of help, UI
file translations and possibly others. All of these use internal tools,
therefore they use some internal libraries.

The only way to ensure there will be no problems in this area is to move
the signing to instsetoo_native and let the target depend on
$(call gb_Postprocess_get_target,AllModulesButInstsetNative)
Which puts it outside of the "normal" build anyway.

> Having it after the build as separate /manual step is pointless, as
> then you woudn't only need to sign, but also rebuild the installsets.
>
> And building installests on windows also is slow. So I object to
> making it an "after make finished" thing.

Well, there is no reason why the installsets must be created directly as
part of build. There could be a separate make target for that.

D.