Re: [PATCH] gitlab: purge msys pacman cache

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Oct 10, 2025 at 06:23:22PM +0100, Peter Maydell wrote:
> On Fri, 10 Oct 2025 at 17:05, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > For the Windows msys2 CI job we install many packages using pacman
> > and use the GitLab cache to preserve the pacman cache across CI
> > runs. While metadata still needs downloading, this avoids pacman
> > re-downloading packages from msys2 if they have not changed.
> >
> > The problem is that pacman never automatically purges anything
> > from its package cache. Thus the GitLab cache is growing without
> > bound and packing/unpacking the cache is consuming an increasing
> > amount of time in the CI job.
> >
> > If we run 'pacman -Sc' /after/ installing our desired package set,
> > it will purge any cached downloaded packages that are not matching
> > any installed package.
> >
> > This will (currently) cap the pacman download cache at approx
> > 256 MB.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >  .gitlab-ci.d/windows.yml | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > See a test job with this change, plus a find across the msys
> > pacman cache, showing the cleanup effects....
> >
> > Before cleanup:
> >
> >   https://gitlab.com/berrange/qemu/-/jobs/11679136531#L34
> >
> > After cleanup:
> >
> >   https://gitlab.com/berrange/qemu/-/jobs/11679136531#L1126
> >
> > diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> > index 1e6a01bd9a..6e1135d8b8 100644
> > --- a/.gitlab-ci.d/windows.yml
> > +++ b/.gitlab-ci.d/windows.yml
> > @@ -87,6 +87,7 @@ msys2-64bit:
> >        mingw-w64-x86_64-pkgconf
> >        mingw-w64-x86_64-python
> >        mingw-w64-x86_64-zstd"
> > +  - .\msys64\usr\bin\bash -lc "pacman -Sc --noconfirm"
> >    - Write-Output "Running build at $(Get-Date -Format u)"
> >    - $env:JOBS = $(.\msys64\usr\bin\bash -lc nproc)
> >    - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
> > --
> 
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> 
> Hopefully this will make the CI job take less time (for the
> record, currently in the main CI run it takes about 70 minutes,
> of which 10 minutes is "unpack the cache" at the start and
> 20 minutes is "repack the cache" at the end).

Based on how long it takes to pack/unpack the cache in my job, I'd
be expecting approx a 25 minute win.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|