On Wed, 2011-01-12 at 10:47 -0600, Norbert Thiebaud wrote:
indeed it is a dog. it took 15h20.
conservatively at least 13 hours of it were in helcontent2, mostly I/O
bound creating temp files and copying things around over and over.
Wow; that is bad.
based on profiling while this was running... I did another attempt at
building just helpcontent2 but with the follwing tweak:
:-)
export TMPDIR to a directory in a ramdisk and link unxmacxi.pro of
helpcontent2 to another directory in the same ramdisk (3.5GB needed)
It is interesting that that gives a big speedup.
the build of helcontent2 in these condition dropped to 26 minutes elapsed.
A huge win. Personally, I find it hard to believe that the problem is
not I/O related - ie. to do something -that- slow, a file-system has to
fsync (ie. physically move the disk heads) I imagine, I don't think much
else can explain it. Out of interest how fast is:
mkdir /tmp/small ; time for (( a=1; a<=10000; a++ )) do touch /tmp/small/$a; rm /tmp/small/$a; done
for you ? I get 22 to 26 seconds (on my two test machines), one has
an SSD, the other a hard-disk, but you can see that neither write anything
to the disk in that time.
So - I'm simply surprised that a ramdisk makes much difference.
I was suspicious of the database / lucene building piece - which may
well do fsyncs, and write a journal to try to ensure data integrity, I
can believe that is horribly slow.
Note: that if you do not have enough memory to do that, you could at
least move the TMPDIR do a ramdisk. you don't need a big one (not sure
exactly how much, but I bet 500M should probably do)
there are some step in helpcontent2 that do a LOT of creation/delete on TMP.
Right; I guess it might be nice to have an 'strace -f -ttt' log (or Mac
equivalent) of a few minutes of the build - to see what is going on.
Thanks !
Michael.
--
michael.meeks@novell.com <><, Pseudo Engineer, itinerant idiot
Context
Privacy Policy |
Impressum (Legal Info) |
Copyright information: Unless otherwise specified, all text and images
on this website are licensed under the
Creative Commons Attribution-Share Alike 3.0 License.
This does not include the source code of LibreOffice, which is
licensed under the Mozilla Public License (
MPLv2).
"LibreOffice" and "The Document Foundation" are
registered trademarks of their corresponding registered owners or are
in actual use as trademarks in one or more countries. Their respective
logos and icons are also subject to international copyright laws. Use
thereof is explained in our
trademark policy.