Re: [RFC PATCH 5/6] include/qemu/compiler: Fix problem with gcc_struct and Clang

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, Jul 31, 2023 at 02:04:42PM +0200, Thomas Huth wrote:
> On 31/07/2023 11.32, Daniel P. Berrangé wrote:
> > On Mon, Jul 31, 2023 at 11:10:58AM +0200, Thomas Huth wrote:
> ...
> > > Or do you see another possibility how we could fix that timeout problem in
> > > the 64-bit MSYS2 job? Still switching to clang, but compiling with
> > > --extra-cflags="-Wno-unknown-attributes" maybe?
> > 
> > I was surprised to see that we're not using ccache in gitlab CI. It wouldn't
> > help the from-clean compile time, but thereafter it ought to help. I'm doing
> > some tests with that to see the impact.
> 
> I tried that two years ago, but the results were rather disappointing:
> 
>  https://lists.nongnu.org/archive/html/qemu-devel/2021-04/msg02189.html
> 
> ... but that was only with the Linux runners. Maybe it helps with the MSYS
> runners?

I've done some tests over the last few days and my results are
rather different.  The speed up is *spectacular* if the hit rate
is high.

eg, consider you are retriggering broadly the same pipeline over &
over with iterations of your patches. eg 99% of the QEMU code
probably doesn't change as you're just re-trying to fix some edge
case in 1 file, and thus you'll get near 100% hit rate.

With such a case I see:

The msys64 job can complete in 24 minutes, instead of 50-60 minutes

The build-system-fedora job can complete in 6 minutes instead of 20
minutes

The build-user-hexagon job can cmplete in 3 minutes instead of 6

Basically a 50% cut in time on all compile jobs I've looked at
so far.

NB, these are both shared runners, since the pipeline is in my
fork. I've no idea what the impact will be on the Azure private
runners for upstream.

Given the CI limitations these days, such a speedup is massively
beneficial for our contributors though.

The downside is storage for the cache. In theory gitlab has defined
a quota for storage per account. In practice they are still yet to
enforce that. If they do ever enforce it, we're likely doomed as
our container images alone will exceed it several times over, let
alone leave room for build logs, job caches, artifacts, etc. While
storage quota isn't enforced though, we might as well use ccache.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|