Re: [Libreoffice] Windows installer size reduction effort - the compression theory

Hi Michael and Friends of LibreOffice,

I ran few test to figure out what is the best method for installset
compressing (se the attached document and diagram). I cheated a bit,
because I used the tools (7z, zip, cabmake) and I didn't modified build
environment. So I downloaded LibO_3.3.1rc2_Win_x86_install_multi.exe and
extracted it. I found that unpacked cab file is the best if we repack it
with LZMA but it is unbalanced because the preinstalled installation kit
is more than 700MB.
So I went to other direction, whatif I increase the efficiency of LZM
compression of makecab. I found that we can use .Set CompressionMemory=
21 setting. This setting produces 83,91% of original installer size and
if we combine it with a simple zip compression the download size can
reduce to 83,54%. This scenario is represented by blue color and the
gain was 36 MB! I think the compression time is not much higher and the
installation time is not increased critically. The preinstallation time
decreased so I think this is an interesting opportunity to reduce the
windows installer's size. What is your opinion? Of course I tried
several other method, but this compilation of compression algorithms
produces the best overall appearance. In the gray section I tried a
special way when I uncompressed every zip container (ODF, JAR, ZIP, etc)
in the installset and every file contains only stored data without
compression. In this way I was able to gain more 15 MB, but this require
zip recompressing at the end of installation process that may make it to
complex and time consuming. Please check the attached document, and if
you want to go with it apply the attached patches.

All the best,
KAMI

2011-02-21 10:42 keltezéssel, Michael Meeks írta:
> Hi Kalman,
>
> On Sun, 2011-02-20 at 15:15 +0100, Kálmán „KAMI” Szalai wrote:
> > I am sure we can shrink the installer more with better compression
>       Hah - so, of course Fridrich and Tor have looked into this - and
> naturally you are right :-) there is a lot we can do. Clearly
> compressing things badly first, and then well later (eg. zip, then lzma)
> doesn't give the best results. That is particularly so when there are
> lots of similarities in the eg. un-compressed ODF files we have
> internally for eg. templates but minor differences that will have
> hard-to-compress knock-on-effects in the compressed stream.
>
>       The ideal would be to use only one level of compression - using the
> best algorithm (LZMA) ie. NSIS, and nothing in the .cab file (which is
> limited to various lame algorithms, and perhaps per-contained-file
> compression, rather than per-whole-cab-file).
>
>       Unfortunately, with the currently level of eg. template duplication,
> this would give us a vast .cab file that would chew lots of space on the
> target machine - though it might shrink our download nicely :-)
>
> > What is your opinion? Can somebody test it on a real Windows build
> > system? 
>       The current balance is based on testing; quite possibly there is a
> better set of compression options - but we need to work out what it is.
> Our current settings are optimised for a multi-lang install as we ship
> it. To change that I'd like to see a table:
>
> New options   Download/kb     Install/kb
>
>       with the relevant sizes of both of these guys. That is just a matter of
> running lots of long builds and comparing the output I guess, and I
> suspect we will get something like a trade-off between these two values.
>
>       Clearly, the 'real' solution is engineering to stop us having so much
> pointless duplication ;-)
>
>       HTH,
>
>               Michael.

Attachment: LibO - Compression tests.ods
Description: application/vnd.oasis.opendocument.spreadsheet

From 068adb8f3900f8612092b6cbfd8c2ea8c2403988 Mon Sep 17 00:00:00 2001
From: Kalman Szalai - KAMI <kami911@gmail.com>
Date: Tue, 22 Feb 2011 17:28:37 +0100
Subject: [PATCH 92/92] Windows installer compression optimization

---
 solenv/bin/modules/installer/windows/msiglobal.pm |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/solenv/bin/modules/installer/windows/msiglobal.pm 
b/solenv/bin/modules/installer/windows/msiglobal.pm
index 735c6e5..19ce8f3 100644
--- a/solenv/bin/modules/installer/windows/msiglobal.pm
+++ b/solenv/bin/modules/installer/windows/msiglobal.pm
@@ -64,7 +64,7 @@ sub write_ddf_file_header
     push(@{$ddffileref} ,$oneline);
     $oneline = ".Set Compress=ON\n";
     push(@{$ddffileref} ,$oneline);
-    $oneline = ".Set CompressionLevel=$installer::globals::cabfilecompressionlevel\n";
+    $oneline = ".Set CompressionMemory= 21\n";
     push(@{$ddffileref} ,$oneline);
     $oneline = ".Set Cabinet=ON\n";
     push(@{$ddffileref} ,$oneline);
-- 
1.7.1

From 5a7dd93332ac29e4c37dead92d7fd5bc5e31adbc Mon Sep 17 00:00:00 2001
From: Kalman Szalai - KAMI <kami911@gmail.com>
Date: Tue, 22 Feb 2011 17:21:53 +0100
Subject: [PATCH] Windows installer compression optimization

---
 .../source/win32/nsis/downloadtemplate.nsi         |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/setup_native/source/win32/nsis/downloadtemplate.nsi 
b/setup_native/source/win32/nsis/downloadtemplate.nsi
index 226c3ca..6484ef9 100644
--- a/setup_native/source/win32/nsis/downloadtemplate.nsi
+++ b/setup_native/source/win32/nsis/downloadtemplate.nsi
@@ -3,8 +3,9 @@
 !define PRODUCT_PUBLISHER "PUBLISHERPLACEHOLDER"
 !define PRODUCT_WEB_SITE "WEBSITEPLACEHOLDER"
 
-SetCompressor lzma
-; SetCompressor zlib
+SetCompressor zlib
+SetDatablockOptimize On
+
 ; Helpful for debugging, disable for products
 ; RequestExecutionLevel user
 
-- 
1.7.1

Attachment: signature.asc
Description: OpenPGP digital signature