Re: [Libreoffice] ccache support

On Friday 09 of September 2011, Norbert Thiebaud wrote:
> On Fri, Sep 9, 2011 at 11:10 AM, Terrence Enger <tenger@iseries-guru.com> 
wrote:
> > On Tue, 2011-08-30 at 12:56 -0500, Norbert Thiebaud wrote:
> > > 2/ I had a spike in IRC questions/confusions from new tentative
> > > contributors on that topic, so I wanted to make it as newbie friendly
> > > as possible. iow a 'just fscking work' approach.
> >
> > Cache compression is advertised as saving space in the cache in return
> > for a modest increase in compilation time.  For me, it is the clear
> > choice: I see the improved cache usage, and I do not notice the
> > increased time.
>
> Well, my boxes are usually cpu bound (even with a fully hot ccache)
> and I have plenty of space for my ccache, so that is not that clear a
> choice to me.

 Since there (AFAIR) haven't been any actual data presented in the discussion 
about enabling ccache, let me present some. I admit they are somewhat 
special, but they are real nevertheless. So, the build setup is Intel Xeon 
@2.53GHz (4 cores), 12G RAM, quite powerful icecream network 
(--with-max-jobs=40) and HDD with 120MB/s speed reported by hdparm (but it's 
not SSD, so the seek time is awful as it is with HDDs). I also build 
with --enable-debug.

- cd svx; make clean; make  (just to make the disk caches hot)
- make clean; time make  (i.e. no ccache)
real    0m52.378s
user    1m37.430s
sys     0m21.630s
- make clean; ccache -C; time cmakeset make (cmakeset enables ccache use here)
real    1m1.170s
user    1m58.422s
sys     0m27.661s
- make clean; time cmakeset make
real    0m14.562s
user    0m28.270s
sys     0m6.972s

 So if ccache has 0% hit ratio, there is 17% overhead in this specific case. 
With 100% hit ratio, 73% is saved. That, if my math is right, means at least 
19% ccache hit ratio is required for breaking even.

 My ccache -s reports
cache hit (direct)                173894
cache hit (preprocessed)           43418
cache miss                        323627

 which is 40% hit ratio, but this is with using ccache only explicitly for 
full rebuilds (i.e. when I develop e.g. in sw/ and build just there, it's 
without ccache). I'm also not sure how many hits come from rebuilding 
everything from scratch, which is not something people without this powerful 
build setup would do that easily (I sometimes just give up on a problem and 
rebuilding everything, and after pull -r I usually also rebuild from scratch 
in another dir before switching to it). Finally, it doesn't say anything 
about how many of those hits are tiny .c files and how many of the misses are 
huge .cxx files, which are more likely to be misses because of having more 
dependencies.

 Could somebody else, preferably with a somewhat more common system :), post 
their results, so that we can compare? I assume the ccache overhead (caused 
by I/O I'd say) is exceptionally high here, but I still doubt ccache warrants 
being enabled by default just like that.

> I don't do that in configure for the same reason we don't change
> CCACHE_DIR or the cache dir size.
> Using ccache if available is one thing... but trying to
> 'auto-magically' optimize ccache is another can of worms altogether...

 I think the case above shows that there are some worms even in the one thing.

-- 
 Lubos Lunak
 l.lunak@suse.cz