[PATCH v2] gitlab-ci: Decrease the size of the compiler cache

[PATCH v2] gitlab-ci: Decrease the size of the compiler cache

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

Uploading the cache from the runner takes a long time in the MSYS2
job, mostly due to the size of the compiler cache.
However, looking at runs with a non-poluted cache, it seems like
you can get a build with a 99% hit rate already with ~ 160 MiB cache
size, so the compiler cache with 500 MiB certainly contains a lot of
stale files. Thus decrease the size of the ccache to a more reasonable
value to speed up the MSYS2 job in our CI.

While at it, also add a "du -sh" for the build folder to get a better
feeling for the amount of object code that is required for the build,
and publish the list of files in /var/cache to be able to better
analyze what is really clogging our cache here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 v2: Decrease cache size to 180M and save cache-log.txt as artifact

 .gitlab-ci.d/windows.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 6e1135d8b86..5dbdabfbec0 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -25,6 +25,7 @@ msys2-64bit:
     expire_in: 7 days
     paths:
       - build/meson-logs
+      - build/cache-log.txt
     reports:
       junit: build/meson-logs/*.junit.xml
   before_script:
@@ -94,7 +95,7 @@ msys2-64bit:
   - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - $env:CCACHE_BASEDIR = "$env:CI_PROJECT_DIR"
   - $env:CCACHE_DIR = "$env:CCACHE_BASEDIR/ccache"
-  - $env:CCACHE_MAXSIZE = "500M"
+  - $env:CCACHE_MAXSIZE = "180M"
   - $env:CCACHE_DEPEND = 1 # cache misses are too expensive with preprocessor mode
   - $env:CC = "ccache gcc"
   - mkdir build
@@ -103,5 +104,7 @@ msys2-64bit:
   - ..\msys64\usr\bin\bash -lc "../configure $CONFIGURE_ARGS"
   - ..\msys64\usr\bin\bash -lc "make -j$env:JOBS"
   - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;"
+  - ..\msys64\usr\bin\bash -lc "ls -lR /var/cache > cache-log.txt"
+  - ..\msys64\usr\bin\bash -lc "du -sh ."
   - ..\msys64\usr\bin\bash -lc "ccache --show-stats"
   - Write-Output "Finished build at $(Get-Date -Format u)"
-- 
2.51.0

Re: [PATCH v2] gitlab-ci: Decrease the size of the compiler cache

[Daniel P. Berrangé]

On Mon, Oct 20, 2025 at 06:17:59PM +0200, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
> 
> Uploading the cache from the runner takes a long time in the MSYS2
> job, mostly due to the size of the compiler cache.
> However, looking at runs with a non-poluted cache, it seems like
> you can get a build with a 99% hit rate already with ~ 160 MiB cache
> size, so the compiler cache with 500 MiB certainly contains a lot of
> stale files. Thus decrease the size of the ccache to a more reasonable
> value to speed up the MSYS2 job in our CI.
> 
> While at it, also add a "du -sh" for the build folder to get a better
> feeling for the amount of object code that is required for the build,
> and publish the list of files in /var/cache to be able to better
> analyze what is really clogging our cache here.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  v2: Decrease cache size to 180M and save cache-log.txt as artifact
> 
>  .gitlab-ci.d/windows.yml | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2] gitlab-ci: Decrease the size of the compiler cache

[Philippe Mathieu-Daudé]

On 20/10/25 18:17, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
> 
> Uploading the cache from the runner takes a long time in the MSYS2
> job, mostly due to the size of the compiler cache.
> However, looking at runs with a non-poluted cache, it seems like
> you can get a build with a 99% hit rate already with ~ 160 MiB cache
> size, so the compiler cache with 500 MiB certainly contains a lot of
> stale files. Thus decrease the size of the ccache to a more reasonable
> value to speed up the MSYS2 job in our CI.
> 
> While at it, also add a "du -sh" for the build folder to get a better
> feeling for the amount of object code that is required for the build,
> and publish the list of files in /var/cache to be able to better
> analyze what is really clogging our cache here.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   v2: Decrease cache size to 180M and save cache-log.txt as artifact
> 
>   .gitlab-ci.d/windows.yml | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>