From: Tamir Duberstein <tamird@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>,
Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
Tamir Duberstein <tamird@gmail.com>
Subject: [PATCH v3 2/2] ci: use twice the CPU count on both providers
Date: Wed, 30 Sep 2026 10:20:35 -0400 [thread overview]
Message-ID: <20260930-ci-large-test-resources-v3-2-d65ac7c21b5f@gmail.com> (raw)
In-Reply-To: <20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com>
GitHub Actions sets JOBS to ten regardless of runner size, while
GitLab CI uses the detected CPU count. Use twice the CPU count for
Make and prove on both providers, doubling GitLab's job count.
Five GitHub Actions attempts per policy, with the long tests enabled,
gave these sums of per-job median successful build/test-step times
(minutes; four or five samples per job) [1-3]:
Fixed 10 CPU count 2x CPU count
Linux Make 278.9 273.9 259.9
macOS Make 94.5 119.1 99.8
Windows Make 102.2 103.8 100.8
Workflow overhead is excluded; Windows runner images varied.
Use twice the CPU count to scale concurrency with runner size while
avoiding the larger macOS slowdown observed with one job per CPU.
Compared with ten jobs, this trades a lower Linux total for a higher
macOS total.
Keep GitLab's Linux and Windows CPU queries. On macOS, use the native
sysctl command on both providers so CPU detection does not depend on
GNU coreutils.
Link: https://github.com/tamird/git/actions/runs/36070869894/attempts/1 [1]
Link: https://github.com/tamird/git/actions/runs/36070867524/attempts/1 [2]
Link: https://github.com/tamird/git/actions/runs/36070867647/attempts/1 [3]
Assisted-by: LLM
Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
ci/lib.sh | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/ci/lib.sh b/ci/lib.sh
index c6ccbf8c17..f0b7f35850 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -227,7 +227,6 @@ then
cache_dir="$HOME/none"
GIT_TEST_OPTS="--github-workflow-markup"
- JOBS=10
distro=$(echo "$CI_JOB_IMAGE" | tr : -)
elif test true = "$GITLAB_CI"
@@ -250,7 +249,6 @@ then
case "$OS,$CI_JOB_IMAGE" in
Windows_NT,*)
CI_OS_NAME=windows
- JOBS=$NUMBER_OF_PROCESSORS
;;
*,macos-*)
# GitLab CI has Python installed via multiple package managers,
@@ -260,11 +258,9 @@ then
export PATH="$(brew --prefix)/bin:$PATH"
CI_OS_NAME=osx
- JOBS=$(nproc)
;;
*,almalinux:*|*,alpine:*|*,debian:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*)
CI_OS_NAME=linux
- JOBS=$(nproc)
;;
*)
echo "Could not identify OS image" >&2
@@ -291,6 +287,19 @@ else
exit 1
fi
+case "$CI_OS_NAME" in
+windows|windows_nt)
+ JOBS=$NUMBER_OF_PROCESSORS
+ ;;
+osx)
+ JOBS=$(sysctl -n hw.logicalcpu)
+ ;;
+*)
+ JOBS=$(nproc)
+ ;;
+esac
+JOBS=$((2 * JOBS))
+
MAKEFLAGS="$MAKEFLAGS --jobs=$JOBS"
GIT_PROVE_OPTS="--timer --jobs $JOBS"
--
2.56.0.rc2.851.g50a151a6e9.frankengit
next prev parent reply other threads:[~2026-09-30 14:20 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-23 17:13 [PATCH 0/2] ci: reduce pressure from large test fixtures Tamir Duberstein
2026-09-23 17:13 ` [PATCH 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-24 6:13 ` Patrick Steinhardt
2026-09-24 20:41 ` Tamir Duberstein
2026-09-25 0:13 ` Jeff King
2026-09-25 6:41 ` Junio C Hamano
2026-09-28 6:36 ` Patrick Steinhardt
2026-09-23 17:13 ` [PATCH 2/2] ci: match Linux jobs to available CPUs Tamir Duberstein
2026-09-24 6:13 ` Patrick Steinhardt
2026-09-25 15:03 ` Tamir Duberstein
2026-09-23 18:36 ` [PATCH 0/2] ci: reduce pressure from large test fixtures Tamir Duberstein
2026-09-25 16:35 ` [PATCH v2 0/2] ci: use cmp and align job-count selection Tamir Duberstein
2026-09-25 16:35 ` [PATCH v2 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-28 6:36 ` Patrick Steinhardt
2026-09-25 16:35 ` [PATCH v2 2/2] ci: align job counts across CI providers Tamir Duberstein
2026-09-28 6:36 ` Patrick Steinhardt
2026-09-28 10:16 ` Tamir Duberstein
2026-09-28 11:14 ` Patrick Steinhardt
2026-09-30 14:20 ` [PATCH v3 0/2] ci: use cmp and align job-count selection Tamir Duberstein
2026-09-30 14:20 ` [PATCH v3 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-30 14:20 ` Tamir Duberstein [this message]
2026-09-30 14:45 ` [PATCH v3 2/2] ci: use twice the CPU count on both providers Patrick Steinhardt
2026-09-30 14:45 ` [PATCH v3 0/2] ci: use cmp and align job-count selection Patrick Steinhardt
2026-09-30 14:58 ` Tamir Duberstein
2026-09-30 18:04 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260930-ci-large-test-resources-v3-2-d65ac7c21b5f@gmail.com \
--to=tamird@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox