Date: prev next · Thread: first prev next last
2011 Archives by date, by thread · List index


On Wed, 2011-01-12 at 10:47 -0600, Norbert Thiebaud wrote:
indeed it is a dog. it took 15h20.
conservatively at least 13 hours of it were in helcontent2, mostly I/O
bound creating temp files and copying things around over and over.

        Wow; that is bad.

based on profiling while this was running... I did another attempt at
building just helpcontent2 but with the follwing tweak:

        :-)

export TMPDIR to a directory in a ramdisk and link unxmacxi.pro of
helpcontent2 to another directory in the same ramdisk (3.5GB needed)

        It is interesting that that gives a big speedup.

the build of helcontent2 in these condition dropped to 26 minutes elapsed.

        A huge win. Personally, I find it hard to believe that the problem is
not I/O related - ie. to do something -that- slow, a file-system has to
fsync (ie. physically move the disk heads) I imagine, I don't think much
else can explain it. Out of interest how fast is:

mkdir /tmp/small ; time for (( a=1; a<=10000; a++ )) do touch /tmp/small/$a; rm /tmp/small/$a; done

        for you ? I get 22 to 26 seconds (on my two test machines), one has
an SSD, the other a hard-disk, but you can see that neither write anything
to the disk in that time.

        So - I'm simply surprised that a ramdisk makes much difference.

        I was suspicious of the database / lucene building piece - which may
well do fsyncs, and write a journal to try to ensure data integrity, I
can believe that is horribly slow.

Note: that if you do not have enough memory to do that, you could at
least move the TMPDIR do a ramdisk. you don't need a big one (not sure
exactly how much, but I bet 500M should probably do)
there are some step in helpcontent2 that do a LOT of creation/delete on TMP.

        Right; I guess it might be nice to have an 'strace -f -ttt' log (or Mac
equivalent) of a few minutes of the build - to see what is going on.

        Thanks !

                Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot


Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.