Re: [PATCH 3/6] Facilitate debugging Git executables in tests with gdb

[Jeff King <peff@peff.net>]

On Tue, Oct 27, 2015 at 04:39:37PM -0700, Stefan Beller wrote:

> On Tue, Oct 27, 2015 at 4:28 PM, Jeff King <peff@peff.net> wrote:
> > I agree doing so would be crazy. But would:
> >
> >   ./t1234-frotz.sh --gdb=17
> >
> > be sane to run gdb only inside test 17?
> 
> OT:
> We have two ways of addressing tests, by number and by name.

Yeah. The numbers are not stable if the script gets new test, but they
are usually fine for within a debugging session. Names are annoying to
type (and also not guaranteed unique).

> Usually when a test fails ("Foo gobbles the bar correctly" failed),
> I want to run tests 1,17 (1 is the correct setup and 17 is the failing test)
> But coming up with that tuple is hard.
>   * How do I know we need to run 1 as the setup ? (usually we do,
>     sometimes we don't and other times we also need 2,3 to completely
> setup the tests)

I think trying to deduce that tuple is a fool's errand. It takes a lot
of manual work, and even if you _think_ you have it, sometimes state
left from earlier tests is accidentally important. But it's usually not
that expensive to run earlier tests at all; it's just expensive to run
them with extra debugging. That's why we have options like
"--valgrind-only=17". We still _run_ tests 1..16, but we do it quickly,
and then execute the expensive and slow valgrind git only on the
suspicious one.

And I'd propose --gdb to work the same way (run all the other tests, but
only kick in gdb for the suspicious one).

If you had multiple "git" invocations inside test 17, you could even do
something like "--gdb=17:4" to kick in only for the 4th git invocation
or something. But counting up git invocations is probably too irritating
to be worth doing manually.

>   * How do I know it's test 17 which is failing? My workflow up to now
>     I just searched the test title in the file, such that I'd be there anyway
>     to inspect it further. But still I found it inconvenient to
> mentally map between
>     17 and the test title.

I usually just run the test script and look at the output. Here's a
failure (which I obviously induced with an extra line):

  $ ./t4103-apply-binary.sh -v -i
  [...]
  ok 5 - check binary diff -- should fail.
  
  expecting success: 
          git checkout master &&
          echo whoops, we fail here && false &&
          test_must_fail git apply --check C.diff
  
  Already on 'master'
  whoops, we fail here
  not ok 6 - check binary diff (copy) -- should fail.
  #
  #               git checkout master &&
  #               echo whoops, we fail here && false &&
  #               test_must_fail git apply --check C.diff
  #

I'd pull the test number from the "not ok" above (it's actually even
easier to see if you drop the "-v", but I usually start my debugging
with "-v" anyway, since error messages often make the problem obvious).

-Peff