Re: sh -x -i -v with continuous integration, was Re: [PATCH 1/4] t7601: test for pull.ff=true overrides merge.ff=false

[Jeff King <peff@peff.net>]

On Sat, May 16, 2015 at 12:07:23PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Do we object to having to sprinkle the "verbose" throughout the test
> > scripts?
> 
> Yes.  
> 
> An unconstrained "verbose" that applies to anything would make
> people less careful to come up with more useful abstractions,
> e.g. test_line_count, which I view as a bigger problem.

I don't think that is true at all. They serve two different purposes.

We need "test_line_count" because of portability considerations. But we
do not need "test_eq" for that; the only reason to add it is for
debuggable output. We _could_ add a bunch of little helpers to cover
each situation, but empirically we have not[1], because it's annoying to
do so. Instead we just write a bare "test". I had hoped that "verbose"
would make it sufficiently easy to improve upon the status quo that
people would actually use it[2].

So I don't really buy the argument that "verbose" makes things worse. I
think I _do_ buy the argument that it is not worth the extra effort
given that I implemented "-x" around the same time. I think the
"verbose" output is a little nicer, as I said earlier, but having the
test-writer do literally zero work may be worth it.

-Peff

[1] The exception, AFAIK, is test_path_is_*.

[2] For the most part, actually, I think the status quo is using
    test_cmp, and most of the uses of "test" are old. Using test_cmp is
    more verbose, but it does have the advantage of checking the exit
    code of the sub-programs (and also its output is much saner for
    multi-line comparisons, but that isn't relevant to Paul's patches).

    I'd be in favor of helpers that made it shorter to use test_cmp.
    E.g.:

      # "actual" contains "foo\n"; we save one line of "echo"
      git foo >actual &&
      test_cmp_str foo actual

      # same, but take multiline input on stdin. Saves one line of "cat"
      git foo >actual &&
      test_cmp_str - actual <<-\EOF
      foo
      EOF

      # similar, but it runs the command for us and compares its
      # stdout, saving another line. Note that we take stdout as-is,
      # so you get no opportunity to post-process it (but you
      # can still do it the "long" way as above if you need to).
      test_cmp_cmd foo git foo

      # same, with stdin; obviously this doesn't work if you need to
      # send something over stdin to the command, but again, you can
      # fallback to the old way.
      test_cmp_cmd - git foo <<-\EOF
      foo
      EOF

   These also go in the direction of "verbose", by providing generic
   solutions rather than specific ones. So by the argument you made
   above, this is a bad thing.