Re: [PATCHv2] write_index: optionally allow broken null sha1s

[Junio C Hamano <gitster@pobox.com>]

Jeff King <peff@peff.net> writes:

> I'd be very surprised if this works in practice on most of our current
> test scripts. There are often subtle dependencies on the state left over
> from previous tests. Running the script below up through t3800 (at which
> point I lost patience) reveals 37 test scripts that are broken. Which is
> only about 17%, but we're clearly not quite there yet.
> ...

Yes, I agree that in an ideal world with infinite amount of time, we
should strive to make sure that each test is independent from
previous steps, and if we do not have infinite amount of time, we
should make sure we find some (say 5%) time to various code clean-up
effort, in both writing and reviewing such, including tests.

The attached is cute.  Thanks for a food for thought.

> -- >8 --
> #!/usr/bin/perl
> #
> # Run as "perl foo.pl t0000-basic.sh" (or whatever script you want to
> # check).
>
> my $script = shift;
> my ($script_num) = $script =~ /^(t\d+)/;
>
> # run once to get the list of tests
> my @tests = run_tests($script);
>
> # mark some tests as "setup" tests that will always be run
> foreach my $test (@tests) {
> 	if ($test->{desc} =~ /set.?up/i) {
> 		print STDERR "marking $test->{num} - $test->{desc} as setup\n";
> 		$test->{setup} = 1;
> 	}
> }
>
> # now try each test in isolation, but including setup tests
> foreach my $test (@tests) {
> 	$ENV{GIT_SKIP_TESTS} = skip_all_except($script_num, $test, @tests);
> 	run_tests($script) or die "failed $test->{num} ($test->{desc})\n";
> }
>
> sub run_tests {
> 	my @r;
> 	open(my $fh, '-|', qw(sh), $script);
> 	while (<$fh>) {
> 		/^ok (\d+) - (.*)/ and push @r, { num => $1, desc => $2 };
> 	}
> 	$? and return ();
> 	return @r;
> }
>
> sub skip_all_except {
> 	my $prefix = shift;
> 	my $want = shift;
>
> 	return join(' ',
> 		map { "$prefix.$_->{num}" }
> 		grep { !$_->{setup} && $_->{num} != $want->{num} }
> 		@_);
> }