[PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Taylor Blau]

There are a handful of related test breakages which are found when
running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in
your environment.

Both test failures are the result of something like:

    git repack --write-midx --write-bitmap-index [...] &&

    test_path_is_file $midx &&
    test_path_is_file $midx-$(midx_checksum $objdir).bitmap

, where we repack instructing Git to write a new MIDX and corresponding
MIDX bitamp.

The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the
enviornment. This causes Git to write out a second MIDX (after
processing the builtin's `--write-midx` argument) which is identical to
the first, but does not request a bitmap (since we did not set the
GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment).

Since c528e179662 (pack-bitmap: write multi-pack bitmaps, 2021-08-31),
the MIDX machinery will drop an existing MIDX bitmap when rewriting an
identical MIDX which does not itself request a corresponding bitmap,
which is similar to the way repack itself behaves in the pack-bitmap
case.

Correct these issues (which date back to [1] and [2], respectively) by
explicitly setting GIT_TEST_MULTI_PACK_INDEX to zero before running each
command.

In the future, we should consider removing GIT_TEST_MULTI_PACK_INDEX,
and in general clean up unused GIT_TEST_-variables. But that is a larger
effort, and this ensures that we can cleanly run:

    $ GIT_TEST_MULTI_PACK_INDEX=1 make test

in the meantime.

[1]: 324efc90d1b (builtin/repack.c: pass `--refs-snapshot` when writing
  bitmaps, 2021-10-01)

[2]: 197443e80ab (repack: don't remove .keep packs with
  `--pack-kept-objects`, 2022-10-17).

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 t/t7700-repack.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
index 94f9f4a1da..127efe99f8 100755
--- a/t/t7700-repack.sh
+++ b/t/t7700-repack.sh
@@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' '
 		git log --format="create refs/tags/%s/%s %H" HEAD >refs &&
 		git update-ref --stdin <refs &&
 
+		GIT_TEST_MULTI_PACK_INDEX=0 \
 		git repack --write-midx --write-bitmap-index &&
 		test_path_is_file $midx &&
 		test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&
@@ -749,6 +750,7 @@ test_expect_success '--write-midx with --pack-kept-objects' '
 		keep="$objdir/pack/pack-$one.keep" &&
 		touch "$keep" &&
 
+		GIT_TEST_MULTI_PACK_INDEX=0 \
 		git repack --write-midx --write-bitmap-index --geometric=2 -d \
 			--pack-kept-objects &&
 
-- 
2.44.0.414.g7e8d435d58e

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

> There are a handful of related test breakages which are found when
> running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in
> your environment.
>
> Both test failures are the result of something like:
>
>     git repack --write-midx --write-bitmap-index [...] &&
>
>     test_path_is_file $midx &&
>     test_path_is_file $midx-$(midx_checksum $objdir).bitmap
>
> , where we repack instructing Git to write a new MIDX and corresponding
> MIDX bitamp.
>
> The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the
> enviornment. This causes Git to write out a second MIDX (after
> processing the builtin's `--write-midx` argument) which is identical to
> the first, but does not request a bitmap (since we did not set the
> GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment).

Doesn't it sound more like a bug, though?  If a command line option
requests something, should we still be honoring a contradicting
instruction given by environment variable(s)?

But anyway.

> diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
> index 94f9f4a1da..127efe99f8 100755
> --- a/t/t7700-repack.sh
> +++ b/t/t7700-repack.sh
> @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' '
>  		git log --format="create refs/tags/%s/%s %H" HEAD >refs &&
>  		git update-ref --stdin <refs &&
>  
> +		GIT_TEST_MULTI_PACK_INDEX=0 \
>  		git repack --write-midx --write-bitmap-index &&
>  		test_path_is_file $midx &&
>  		test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&

Is it a viable alternative approach to skip this check (and the
other one) when GIT_TEST_MULTI_PACK_INDEX is set (i.e., lazy
prereq).  It will give us a better documentation value, e.g.,

	test_lazy_prereq FORCED_MIDX '
                # Features that are broken when GIT_TEST_* forces it
                # to enable are protexted with this prerequisite.
		test "$GIT_TEST_MULTI_PACK_INDEX" = 1;
	'

	test_expect_success !FORCED_MIDX '--write-midx with ...' '
		...
	'

With a single comment, we can annotate any future tests that relies
on features working correctly even with GIT_TEST_MULTI_PACK_INDEX.

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Taylor Blau]

On Tue, Apr 02, 2024 at 11:37:10AM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
>
> > There are a handful of related test breakages which are found when
> > running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in
> > your environment.
> >
> > Both test failures are the result of something like:
> >
> >     git repack --write-midx --write-bitmap-index [...] &&
> >
> >     test_path_is_file $midx &&
> >     test_path_is_file $midx-$(midx_checksum $objdir).bitmap
> >
> > , where we repack instructing Git to write a new MIDX and corresponding
> > MIDX bitamp.
> >
> > The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the
> > enviornment. This causes Git to write out a second MIDX (after
> > processing the builtin's `--write-midx` argument) which is identical to
> > the first, but does not request a bitmap (since we did not set the
> > GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment).
>
> Doesn't it sound more like a bug, though?  If a command line option
> requests something, should we still be honoring a contradicting
> instruction given by environment variable(s)?
>
> But anyway.

I have generally considered the `--write-midx` and
`GIT_TEST_MULTI_PACK_INDEX` options to be orthogonal to each other. The
latter is a developer-oriented option that forces Git to write a MIDX
post-repack regardless of the command-line option.

It predates the `--write-midx` option by a number of years, and IIUC was
introduced to enhance test coverage while the MIDX was being originally
developed.

I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of
GIT_TEST_-variables to get rid of as it has served its purpose.

> > diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
> > index 94f9f4a1da..127efe99f8 100755
> > --- a/t/t7700-repack.sh
> > +++ b/t/t7700-repack.sh
> > @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' '
> >  		git log --format="create refs/tags/%s/%s %H" HEAD >refs &&
> >  		git update-ref --stdin <refs &&
> >
> > +		GIT_TEST_MULTI_PACK_INDEX=0 \
> >  		git repack --write-midx --write-bitmap-index &&
> >  		test_path_is_file $midx &&
> >  		test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&
>
> Is it a viable alternative approach to skip this check (and the
> other one) when GIT_TEST_MULTI_PACK_INDEX is set (i.e., lazy
> prereq).  It will give us a better documentation value, e.g.,
>
> 	test_lazy_prereq FORCED_MIDX '
>                 # Features that are broken when GIT_TEST_* forces it
>                 # to enable are protexted with this prerequisite.
> 		test "$GIT_TEST_MULTI_PACK_INDEX" = 1;
> 	'
>
> 	test_expect_success !FORCED_MIDX '--write-midx with ...' '
> 		...
> 	'
>
> With a single comment, we can annotate any future tests that relies
> on features working correctly even with GIT_TEST_MULTI_PACK_INDEX.

It's possible, but not something that I have seen done elsewhere for
this or any of the other GIT_TEST- variables.

Like I said, I'd like to get rid of this (and many other)
GIT_TEST-related variables, but that is a larger effort than this single
patch.

Thanks,
Taylor

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

> I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of
> GIT_TEST_-variables to get rid of as it has served its purpose.
>
> Like I said, I'd like to get rid of this (and many other)
> GIT_TEST-related variables, but that is a larger effort than this single
> patch.
Yup, that sounds like a good longer-term goals.  While it does not
smell like it is consistent with that goal to add more instances of
use of it to the test, it may inevitably be a "few steps backward in
preparation to jump big forward", perhaps.

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Taylor Blau]

On Tue, Apr 02, 2024 at 12:55:39PM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
>
> > I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of
> > GIT_TEST_-variables to get rid of as it has served its purpose.
> >
> > Like I said, I'd like to get rid of this (and many other)
> > GIT_TEST-related variables, but that is a larger effort than this single
> > patch.
>
> Yup, that sounds like a good longer-term goals.  While it does not
> smell like it is consistent with that goal to add more instances of
> use of it to the test, it may inevitably be a "few steps backward in
> preparation to jump big forward", perhaps.

Yep, exactly.

Thanks,
Taylor

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Jeff King]

On Tue, Apr 02, 2024 at 02:45:28PM -0400, Taylor Blau wrote:

> I have generally considered the `--write-midx` and
> `GIT_TEST_MULTI_PACK_INDEX` options to be orthogonal to each other. The
> latter is a developer-oriented option that forces Git to write a MIDX
> post-repack regardless of the command-line option.
> 
> It predates the `--write-midx` option by a number of years, and IIUC was
> introduced to enhance test coverage while the MIDX was being originally
> developed.
> 
> I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of
> GIT_TEST_-variables to get rid of as it has served its purpose.

Hmm. Obviously it is of little value in this explicit --write-midx
test, but I thought the main value was just exercising all of the
_other_ tests with a midx in place. Doesn't that potentially have value
(just like testing with SPLIT_INDEX, etc, gets more coverage)?

If it is worth keeping (and I do not really have a strong opinion
there), the real issue seems to me that it does not behave like
--write-midx. That is the source of the problem here, but also makes
test runs with it unrealistic, since the command-line option is how
real-world users would trigger it.

I.e., I would have expected something like this, so that the variables
takes precedence over config but under command-line options:

diff --git a/builtin/repack.c b/builtin/repack.c
index 15e4cccc45..4b02d9cb77 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1197,6 +1197,11 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 
 	git_config(repack_config, &cruft_po_args);
 
+	write_midx = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, write_midx);
+	if (write_midx)
+		write_bitmaps = git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP,
+					     write_bitmaps);
+
 	argc = parse_options(argc, argv, prefix, builtin_repack_options,
 				git_repack_usage, 0);
 
@@ -1214,10 +1219,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		if (!write_midx &&
 		    (!(pack_everything & ALL_INTO_ONE) || !is_bare_repository()))
 			write_bitmaps = 0;
-	} else if (write_bitmaps &&
-		   git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0) &&
-		   git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP, 0)) {
-		write_bitmaps = 0;
 	}
 	if (pack_kept_objects < 0)
 		pack_kept_objects = write_bitmaps > 0 && !write_midx;
@@ -1515,13 +1516,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	if (run_update_server_info)
 		update_server_info(0);
 
-	if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0)) {
-		unsigned flags = 0;
-		if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP, 0))
-			flags |= MIDX_WRITE_BITMAP | MIDX_WRITE_REV_INDEX;
-		write_midx_file(get_object_directory(), NULL, NULL, flags);
-	}
-
 cleanup:
 	string_list_clear(&names, 1);
 	existing_packs_release(&existing);

But it gets weird because some tests (like t7700.2) explicitly ask for
bitmaps on the command line and want pack bitmaps but _not_ midx
bitmaps.

So I dunno. Maybe this is a can of worms that is not worth falling into.
After all, these are not "real" environment variables that we expect
users to use. I just wonder if the ci runs with them are buying us
anything for all of the tests outside of t7700.

-Peff

Re: [PATCH] t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1`

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> So I dunno. Maybe this is a can of worms that is not worth falling into.
> After all, these are not "real" environment variables that we expect
> users to use. I just wonder if the ci runs with them are buying us
> anything for all of the tests outside of t7700.

Yeah, I agree with the general direction of reducing the role
GIT_TEST_* environment variables take.