Git development
 help / color / mirror / Atom feed
From: Tamir Duberstein <tamird@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>,
	Junio C Hamano <gitster@pobox.com>,  Jeff King <peff@peff.net>,
	Tamir Duberstein <tamird@gmail.com>
Subject: [PATCH v2 0/2] ci: use cmp and align job-count selection
Date: Fri, 25 Sep 2026 12:35:37 -0400	[thread overview]
Message-ID: <20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com> (raw)
In-Reply-To: <20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com>

The first patch uses cmp for the huge commit-message output comparison.
On this input, GNU diffutils 3.8 on Linux arm64 takes 5.276 seconds with
4,199,924 KiB peak RSS for diff, versus 0.506 seconds and 1,264 KiB for
cmp. Patch 1 includes the fixture and measurement details.

The second patch aligns GitHub Actions' Make and prove job counts with
GitLab CI's CPU-count policy, replacing GitHub's fixed ten jobs.

Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
Changes in v2:
- Replace the unavailable CI failure reference with comparison runtime
  and peak RSS measurements.
- Drop the file removal; following tests overwrite expect and actual.
- Share job-count selection between GitHub Actions and GitLab CI.
- Use native CPU-count queries on macOS and Windows.
- Link to v1: https://patch.msgid.link/20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com

---
Tamir Duberstein (2):
      t4205: compare huge output without diff
      ci: align job counts across CI providers

 ci/lib.sh                     | 16 ++++++++++++----
 t/t4205-log-pretty-formats.sh |  2 +-
 2 files changed, 13 insertions(+), 5 deletions(-)

Range-diff versus v1:

1:  5babc36eb5 ! 1:  b48c86e164 t4205: compare huge output without diff
    @@ Metadata
      ## Commit message ##
         t4205: compare huge output without diff
     
    -    The huge-commit test compares two files with a line larger than 2 GiB.
    -    In Linux GitHub Actions jobs, git log produces its huge output but
    -    its subsequent diff process is killed with SIGKILL.
    +    The huge-commit test compares output containing a line larger than 2 GiB.
    +    For two identical files containing 2,147,483,649 "1" bytes followed by
    +    "0\n", GNU diffutils 3.8 on Linux arm64 gives these measurements:
     
    -    Use test_cmp_bin to compare the output byte for byte without constructing
    -    a line-oriented diff. Remove the two large files after a successful
    -    comparison, releasing more than 4 GiB before subsequent tests.
    +      Command               Mean +/- stddev       Maximum RSS (KiB)
    +      diff -u expect actual  5.276 +/- 0.572 s              4199924
    +      cmp expect actual      0.506 +/- 0.099 s                 1264
     
    +    The test needs only an equality check. Use test_cmp_bin, which runs cmp,
    +    to compare the output byte for byte with less time and memory.
    +
    +    Assisted-by: LLM
         Signed-off-by: Tamir Duberstein <tamird@gmail.com>
     
      ## t/t4205-log-pretty-formats.sh ##
    @@ t/t4205-log-pretty-formats.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'se
      	git log -1 --format="%B%<(1)%x30" $huge_commit >actual &&
      	echo 0 >>expect &&
     -	test_cmp expect actual
    -+	test_cmp_bin expect actual &&
    -+	rm expect actual
    ++	test_cmp_bin expect actual
      '
      
      test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '
2:  0836f6b372 < -:  ---------- ci: match Linux jobs to available CPUs
-:  ---------- > 2:  8ec0308f6a ci: align job counts across CI providers

---
base-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7
change-id: 20260923-ci-large-test-resources-349cdc95f7f8


  parent reply	other threads:[~2026-09-25 16:35 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-23 17:13 [PATCH 0/2] ci: reduce pressure from large test fixtures Tamir Duberstein
2026-09-23 17:13 ` [PATCH 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-24  6:13   ` Patrick Steinhardt
2026-09-24 20:41     ` Tamir Duberstein
2026-09-25  0:13       ` Jeff King
2026-09-25  6:41         ` Junio C Hamano
2026-09-28  6:36       ` Patrick Steinhardt
2026-09-23 17:13 ` [PATCH 2/2] ci: match Linux jobs to available CPUs Tamir Duberstein
2026-09-24  6:13   ` Patrick Steinhardt
2026-09-25 15:03     ` Tamir Duberstein
2026-09-23 18:36 ` [PATCH 0/2] ci: reduce pressure from large test fixtures Tamir Duberstein
2026-09-25 16:35 ` Tamir Duberstein [this message]
2026-09-25 16:35   ` [PATCH v2 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-28  6:36     ` Patrick Steinhardt
2026-09-25 16:35   ` [PATCH v2 2/2] ci: align job counts across CI providers Tamir Duberstein
2026-09-28  6:36     ` Patrick Steinhardt
2026-09-28 10:16       ` Tamir Duberstein
2026-09-28 11:14         ` Patrick Steinhardt
2026-09-30 14:20   ` [PATCH v3 0/2] ci: use cmp and align job-count selection Tamir Duberstein
2026-09-30 14:20     ` [PATCH v3 1/2] t4205: compare huge output without diff Tamir Duberstein
2026-09-30 14:20     ` [PATCH v3 2/2] ci: use twice the CPU count on both providers Tamir Duberstein
2026-09-30 14:45       ` Patrick Steinhardt
2026-09-30 14:45     ` [PATCH v3 0/2] ci: use cmp and align job-count selection Patrick Steinhardt
2026-09-30 14:58       ` Tamir Duberstein
2026-09-30 18:04         ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com \
    --to=tamird@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox