Re: qemu CI & ccache: cache size is too small

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, May 27, 2024 at 02:38:08PM +0300, Michael Tokarev wrote:
> 27.05.2024 14:19, Thomas Huth wrote:
> > On 27/05/2024 12.49, Michael Tokarev wrote:
> > > Hi!
> > > 
> > > Noticed today that a rebuild of basically the same tree (a few commits apart)
> > > in CI result in just 11% hit rate of ccache:
> > > 
> > > https://gitlab.com/mjt0k/qemu/-/jobs/6947445337#L5054
> > 
> > For me, the results look better:
> > 
> >   https://gitlab.com/thuth/qemu/-/jobs/6918599017#L4954
> 
> Yeah, it's a bit better, but still not good enough.
> I dunno how much changes the source had between the two runs.
> It still had 11 cleanups, and the cache size is at the same level.
> (It is an older ccache, too).
> 
> > > while it should be near 100%.  What's interesting in there is:
> > > 
> > > 1) cache size is close to max cache size,
> > > and more important,
> > > 2) cleanups performed 78
> > > 
> > > so it has to remove old entries before it finished the build.
> > 
> > Did you maybe switch between master and stable branches before that run?
> > ... I guess that could have invalidated most of the cached files since
> > we switched from CentOS 8 to 9 recently...?
> 
> Nope, nothing else ran between the two and it was just a few
> source-level commits (stable-8.2 pick ups), without changing
> giltab/containers/etc configuration.
> 
> I increased cache size to 900M and did another test run, here are
> the results: https://gitlab.com/mjt0k/qemu/-/jobs/6947894974#L5054
> 
> cache directory                     /builds/mjt0k/qemu/ccache
> primary config                      /builds/mjt0k/qemu/ccache/ccache.conf
> secondary config      (readonly)    /etc/ccache.conf
> stats updated                       Mon May 27 11:17:44 2024
> stats zeroed                        Mon May 27 11:10:22 2024
> cache hit (direct)                  1862
> cache hit (preprocessed)             274
> cache miss                          1219
> cache hit rate                     63.67 %
> called for link                      285
> called for preprocessing              71
> compiler produced empty output         5
> preprocessor error                     2
> no input file                          6
> cleanups performed                     0
> files in cache                      9948
> cache size                         654.6 MB
> max cache size                     900.0 MB
> 
> This is having in mind that the previous run was with CCACHE_SIZE=500M
> and had multiple cleanups, so 63% is actually more than I'd expect already.

Given your original job had cache of 447 MB, and new cache is 654 MB, the
old cache is 68% of size of the new cache. So effectively your 63% is
high 90's cache hit rate of what was present.

This would suggest a cache size of 700 MB is more appropriate, unless some
other jobs have even high usage needs.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|