debugging git tests, was: Re: [PATCH v4 2/8] t5520: test no merge candidates cases

[Jeff King <peff@peff.net>]

On Mon, May 18, 2015 at 10:46:50AM -0700, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >> +test_expect_success 'fail if wildcard spec does not match any refs' '
> >> +	git checkout -b test copy^ &&
> >> +	test_when_finished "git checkout -f copy && git branch -D test" &&
> >
> > When I read this line, I immediately asked myself whether the
> > branch would be deleted even if the test case failed. I then
> > tested this theory by editing the first test case ("setup") like
> > this:
> > ...
> > and indeed, the file "file" was gone, even if the test case
> > failed. I therefore believe that this "test_when_finished" cleanup
> > might make debugging substantially harder. Maybe we can drop these
> > lines from this patch?
> 
> The test framework is aware of the fact that it needs to help the
> people who are debugging the scripts.  The support is limited to the
> case in which you run it under the -i option, i.e.
> 
> 	$ cd t
>         $ sh ./t5520-pull.sh -i -v
> 
> will refrain from running test_when_finished scripts when the test
> piece fails.  Even though this is only limited to -i, I found it
> often sufficient for debugging.

If you don't use "-i", you are pretty much screwed anyway, because the
subsequent tests will stomp all over the state of the test directory.
Many a head-scratching session has been caused by looking at the wrong
state, and these days my go-to options for debugging a test are "-v -i".
But since we are talking about it in a related thread, I will advertise
the new "-x" here, too.  :)

As a side note, I've also considered better support for running the
debugger on git commands inside a test (right now, I usually stick a
"gdb --args" in the pipeline, but you have to remember to run with "-v",
and to redirect stdin appropriately). Do other people have this
annoyance, too?

I'm vaguely thinking of something like putting debug support into
bin-wrappers/git, but activating it only for certain tests (so you could
say "t5520-pull.sh --gdb=10", and git would start under the debugger
only for test 10). I think we'd also have to use gdbserver for I/O
sanity, and maybe provide short script to do:

   gdb -ex "target remote localhost:$some_port" "$TEST_DIRECTORY"/../git

That still doesn't cover all cases (when git spawns an external command,
you probably want to run the debugger on that; likewise, I have a
git-remote-debug hack for debugging remote-curl). I suspect with clever
use of gdb options that you could convince the original gdb invocation
to end up tracing the process you care about, though.

-Peff