Date: prev next · Thread: first prev next last
2013 Archives by date, by thread · List index


On 04/03/13 12:36, Bjoern Michaelsen wrote:
On Mon, Mar 04, 2013 at 10:31:12AM +0000, Michael Meeks wrote:
Hi guys,

     Norbert pointed out to me that one of the slowest pieces of our install
is the:

     'checking files with ARCHIVE flags' ...

     Phase in the make_installer.pl foo, which burns a ton of CPU time for
no aparently good reason.

     Last I looked it seemed that we were using the perl Zip functionality
to list files in a .zip and/or to un-compress it to /tmp for some
reason.

http://opengrok.libreoffice.org/xref/core/solenv/bin/modules/installer/archivefiles.pm#145

     I believe we use .zip files as an easy way to package up a chunk of
source code directory - say containing a template structure, into a
bundle; and then install it en-masse in a simple rule and (more to the
point) a single scp2 entry :-)

that's the problem.

     With the new make infrastructure / changes to scp2 - it would be just
wonderful if we could stop: zipping that stuff during the build, and
then unzipping it to /tmp during some very slow phase of
make_installer.pl - and then copying it into the install-set; and simply
copy it straight in ;->

     Hopefully that no longer busts some conceptual flow ?

So I had a quick look at the above perl-foo and a grep on ARCHIVE in scp2 -- it
seems this is mostly done for external stuff like dictionaries etc. -- where we
do not have insight into what those builds do. And we also do not _want_ to
have insight into that. Doing a 'whatever is inside this tarball is the product
of external project foo' isnt that stupid of an approach.

actually that doesn't appear to be the case - none of the
dictionaries/Dictionary*.mk or */Zip*.mk use wildcard * to add files,
all list the files explicitly.

there are some zip files built by CustomTargets that just pack some
recursive directory though, e.g. the 2 different python zip files in
pyuno and python3.

i'd assume that in many cases zip files are entirely superfluous now;
they are a remnant from an age where partial builds of single modules
into a pre-built solver were a common workflow (and of course to avoid
duplication of lots of files in scp2).  we should be able to replace
almost all of them by converting them to Package*.mk files, and using
these directly to define the content of the installation set.

in the end it should look something like this:

1) instead of solver/ imagine some installation/ dir that is a runnable
   installation (LO and SDK and any help/l10n etc.)

2) */Package*.mk copies the files into the right place in installation/
   without any zipping or other nonsense

3) the RPM/DEB/archive/whatever instset creation basically works by
   taking the list of files from the */Package*.mk and putting the files
   into the proper RPM/... package
   (not sure whether it's best to reach this by writing some
   list-of-content file when building the Package, or reading all
   modules' Packages, or defining Packages more centrally...)

not quite sure what the best migration path to get there is; currently
most Package_*.mk just copy internal headers around which is quite
pointless in any case and probably should be fixed first.

then i guess we could start with a "list-of-contents" approach, and a
new "feature" in scp2 to replace ARCHIVE with this
(Package.mk-generated) list-of-contents file (i.e. gbuild writes the
list-of-contents file and scp2 reads it), and then incrementally migrate
things over.

or would it be easier to generate scp2's "output" files "directly" from
gbuild?  (i never looked at what is going on there anyway)



Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.