Re: Performance issues with our internal memory allocator

Hey,

On Mon, Sep 28, 2015 at 8:25 AM, Markus Mohrhard <
markus.mohrhard@googlemail.com> wrote:

> Hey,
>
> On Sat, Sep 26, 2015 at 11:39 PM, Michael Meeks <
> michael.meeks@collabora.com> wrote:
>
> > Hi Markus,
> >
> > On Sat, 2015-09-26 at 20:51 +0200, Markus Mohrhard wrote:
> > > so we have been running our in-build performance tests now for a few
> > > weeks and recently discovered that our internal memory allocator is
> > > causing spikes in the runtime.
> >
> >         What fun =) the irony is that it was written to avoid exactly such
> > spikes (which were primarily on Windows) ;-) Thanks for finding this
> > one !
> >
> > >  It became even worse during the weekend with the tests taking 200
> > > times the instructions. Most of it seems to be spend in our memory
> > > handling code and not really in the actual code. (see for example
> > http://perf.libreoffice.org/perf_html/ftest_of_cppu_sc_on_vm139.details.html
> > with the annotated callgrind ouput at http://pastebin.com/ELC64s1n).
> > > We had a profile that showed the issue inside of the memory allocator
> > > much better but I have to find it again.
> >
> >         Interesting. We found a really silly one in the ::Interpret just
> > recently, and have a simple fix - that could cause some excessive
> > allocation - but I forget if Tor merged that to master (yet).
> >
> >         TBH - I find using kcachegrind -incredibly- more useful than the
> > annotated output above.
>
> Me too. The annotated output is what we currently get from the performance
> testing in jenkins. So they are much better than nothing as they allow us
> to look into the past by just inspecting the build logs. I'm already
> incredibly thankful to Norbert for making them available as it allows me to
> see what is going on the VM.
>
>
> > > Is the internal memory allocator really still useful despite showing
> > > sometimes really bad behavior ?
> >
> >         I'd say not myself. My hope is that the windows allocator has
> > also had
> > some work done on it since ~2005? when the issue was worked around by
> > mhu.
> >
> > >  Personally I would just fall back to the system memory allocator
> > > except for the few cases where we know that it makes a difference
> > > (small memory blocks in calc formula tokens, ...)
> >
> >         Right - it should be far quicker, particularly on Linux.
> >
> >         I'd love to see how a change like that impact the profiles; worth
> > a
> > commit to master and a quick revert later if there is a visible issue
> > anywhere I guess =)
>
> I have just committed such a change. We only have "reliable" data for
> linux but if we see some huge improvement there we should at least consider
> keeping it enabled on linux. The other idea that I had is that it is
> related to using swap as we were surprisingly close to the RAM limit on the
> VM. However after discussing this idea with Norbert I'm no longer sure if
> using the swap would result in a changed callgrind IR count.


So it seems that the system allocator is actually even worse as can be seen
at http://perf.libreoffice.org/perf_html/suite_cppu_sc.html


I'll leave it in for a bit as I want to see what happens if we increase the
memory for the vm but I fear that the numbers show that the internal
allocator is actually useful.

> >         Then again - I think we're going to need a custom allocator of
> > some
> > kind (though prolly rather slow & dumb) for LibreOfficeKit
> > pre-initialization for cloudy bits - so; perhaps that allocator may be
> > useful in the end temporarily for that.
>
> Of course there are a few places where we need a custom allocator but if
> it really performs badly we might want to limit these places.
>
> Markus
>
> >         ATB,
> >
> >                 Michael.
> >
> > --
> >  michael.meeks@collabora.com  <><, Pseudo Engineer, itinerant idiot