Re: Speeding up CI builds? (Re: ESC meeting minutes: 2020-01-23)

On Thursday 30 of January 2020, Thorsten Behrens wrote:
> Luboš Luňák wrote:
> >  So what is the actual need there, if any? Not just Mac, but CI
> > builds in general.
>
> Discussion was prompted by the news item that Firefox CI went for
> cross-building on Mac. And since we were facing similar scalability
> issues (that are solved at this time), the ESC thought about if (and
> how) that could be an option for us.
>
> > For example, there should be another way to speed Mac builds by
> > cross-compiling, namely using icecream to handle that. Icecream has
> > a very simple-to-use support for cross-compiling, it'd require only
> > setting up Linux build node(s) and Mac would still run the build
> > itself, so there'd be no need to alter the build system besides
> > setting CC/CXX.
>
> That's an absolute brilliant idea, as it also transparently solves the
> (non-tiny) catch that for running _tests_, Firefox was still relying
> on physical Mac hardware (though of course fewer of them needed).
>
> Would that work more or less out of the box?

 More or less, yes. The Linux machines don't need any special setup besides 
being icecream nodes. The Mac machine would need compiler setup, which 
besides setting CC/CXX for icecream would also mean creating cross-compiling 
compiler environment tarball.

 That is (poorly) described at 
https://github.com/icecc/icecream/#cross-compiling-using-icecream . In a 
nutshell, since Clang is a cross-compiler out of the box, it should require 
only:
- run 'create-icecc-env /clang/binary' on a Linux box , where the Clang binary 
would be a Linux build of Clang that would be used by the Linux nodes for the 
cross-compiling (it should preferably be exactly the same Clang version as 
the one used on the Mac).
- move the resulting tarball to the Mac machine and set something 
like "ICECC_VERSION=x86_64:/compiler/tarball.tar.gz" (I think that'd prevent 
building on the Mac itself, so it really should need running create-icecc-env 
also on the Mac itself and then actually 
setting "ICECC_VERSION=/compiler/mac.tar.gz,x86_64:/compiler/linux.tar.gz").

 I did this myself just once somewhen in the past (and with Cygwin/gcc), but 
this should be more or less it. I can help with the details or any problems 
(I should finally update the icecream docs for cross-compiling anyway).

-- 
 Luboš Luňák
 l.lunak@collabora.com