Re: [PATCH 3/6] t5516 (fetch-push): introduce mk_test_with_name()

[Jonathan Nieder <jrnieder@gmail.com>]

Jeff King wrote:

>                                                                    I
> tend to read the tests in a top-down manner: a test is interesting
> (usually because it fails), and then I want to see what it is doing, so
> I look at any functions it calls, and so forth.
>
> What I usually find _much_ harder to debug is when there is hidden state
> leftover from other tests.

Thanks for articulating this.  I agree that keeping track of state is
the hardest part of working with git's tests.

To clarify my earlier comment, I was reading the test script from the
point of view of someone who wants to add an additional test, rather
than someone debugging an existing one.  That person has a difficult
task: she needs to understand

 * What do the existing tests in the script do?  This information
   is important in deciding whether the new test will be redundant.

 * How do I work with the particular dialect used in the script,
   including helpers?  A new test should fit in with the style of its
   surroundings to avoid contributing to an unmaintainable mess.

 * What is the intended scope of the test script?  Does my new test
   belong in this script?

 * What is the logical progression of the script?  What story does this
   script tell?  Where should I insert my test to maintain a logical
   ordering?

 * What state that later tests rely on do I need to avoid clobbering?

Generally the best way to help such a person is to make the script
very simple.  In particular, using standard helpers instead of custom
functions when appropriate and documenting helpers used to give new
readers a quick introduction to the dialect can help a lot.