Re: [PATCH 6/6] t5608: add regression test for >4GB object clone

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

Hi Stolee & Jeff,

On Sun, 3 May 2026, Derrick Stolee wrote:

> On 5/1/2026 2:38 AM, Jeff King wrote:
> > On Wed, Apr 29, 2026 at 09:34:21AM -0400, Derrick Stolee wrote:
> 
> >>> +test_expect_success SIZE_T_IS_64BIT 'set up repo with >4GB object' '
> >>
> >> Your prereq here prevents it from running on 32-bit builds, which is
> >> good. However, I wonder if it would be worth also specifying these
> >> tests as expensive. It's less likely that these layers will be touched
> >> often, so it should be enough to run these on major occasions, such as
> >> testing a release candidate.
> > 
> > I think it is already skipped in most cases, because t5608 requires the
> > GIT_TEST_CLONE_2GB environment variable be set. Arguably it should just
> > be using EXPENSIVE, too, as I do not think there is much value in having
> > individual flags for all of the expensive tests. I think that test just
> > predates the modern prereq system entirely.
> 
> Thanks for the extra details here! That helps avoid the issues that I
> was thinking about, but maybe doubling-down and adding EXPENSIVE is
> still worth it. 

Indeed. The `GIT_TEST_CLONE_2GB` flag is set for all CI jobs:
https://gitlab.com/git-scm/git/-/blob/v2.54.0/ci/lib.sh#L314

So I've tried to accelerate it.

First, by using the "unsafe" SHA-1 (because we're in no danger here, we
generate the data ourselves). That helped some, but unfortunately the most
efficient implementation (OpenSSL) is out of bounds for us because we
cannot enable its use in the default configuration for licensing reasons:
OpenSSL's license and Git's GPLv2 are fundamentally incompatible with each
other, and even though Linus' intention with
https://gitlab.com/git-scm/git/-/blob/e83c5163316f89bfbde7d9ab23ca2e25604af290/Makefile#L11
was clearly to allow linking to OpenSSL (actually, requiring it), there is
one Git contributor I spoke to who flatly stated that they'd block every
attempt to add an exception to Git's license retroactively to allow
linking to OpenSSL (at least for distribution, which is when the GPLv2
would kick in). So that's that.

Second, I added shortcuts for the 4GiB+1 size exercised in the added test
cases. That did help! But only the generation time of those packfiles is
helpd by that. The `git clone` that is tested is still awfully slow, and
that _cannot_ be worked around in the same ways.

Therefore, I ended up marking the test cases as `EXPENSIVE`.

To allow them to be run regularly anyway, I added a final patch to the
series that lets the CI runs on the integration branches (other than
`seen`) run all `EXPENSIVE` test cases. One could argue that this patch
should be split out, and I'm open to it, but in the interest of keeping
the time I am working on this patch series _somewhat_ closer to what could
be called reasonable, I'll just keep it in the patch series for now,
hedging for the possibility that maybe nobody objects?

Ciao,
Johannes

> >> I suppose this also is a question for Junio and our process for
> >> validating releases. Do we have a certain cadence where we run the
> >> expensive tests? What has been our threshold for hiding a test case
> >> behind the expensive label?
> > 
> > AFAIK the labeling of expensive things is mostly ad-hoc, and nobody is
> > systematically running them. Likewise for the t/perf tests, which are
> > super expensive but do (very occasionally) turn up interesting
> > regressions.
> I used to be more diligent about running the performance tests myself
> around release windows. The EXPENSIVE tests would also be good to do
> on rc0. I will contemplate how to put this into my routine.
> 
> Thanks,
> -Stolee
> 
> 
>