[PATCH] t/README: mention test files are make targets

[PATCH] t/README: mention test files are make targets

[Philippe Blain via GitGitGadget]

From: Philippe Blain <levraiphilippeblain@gmail.com>

Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each
test file defines a target in the test Makefile, such that one can
invoke:

	make *checkout*

to run all tests with 'checkout' in their filename. This is useful to
run a subset of tests when you have a good idea of what part of the code
is touched by the changes your are testing.

Document that in t/README to help new (or more seasoned) contributors
that might not be aware.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
---
    t/README: mention test files are make targets

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1701%2Fphil-blain%2Ftests-makefile-targets-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1701/phil-blain/tests-makefile-targets-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1701

 t/README | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/t/README b/t/README
index 621d3b8c095..71211109338 100644
--- a/t/README
+++ b/t/README
@@ -32,6 +32,13 @@ the tests.
     ok 2 - plain with GIT_WORK_TREE
     ok 3 - plain bare
 
+t/Makefile defines a target for each test file, such that you can also use
+shell pattern matching to run a subset of the tests:
+
+    make *checkout*
+
+will run all tests with 'checkout' in their filename.
+
 Since the tests all output TAP (see https://testanything.org) they can
 be run with any TAP harness. Here's an example of parallel testing
 powered by a recent version of prove(1):

base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b
-- 
gitgitgadget

Re: [PATCH] t/README: mention test files are make targets

[Junio C Hamano]

"Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Philippe Blain <levraiphilippeblain@gmail.com>
>
> Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each
> test file defines a target in the test Makefile, such that one can
> invoke:
>
> 	make *checkout*
>
> to run all tests with 'checkout' in their filename. This is useful to
> run a subset of tests when you have a good idea of what part of the code
> is touched by the changes your are testing.

While I agree with the patch that this is a useful "feature" of
t/Makefile, I've always felt it was ugly to use a file itself that
we do not consider a build product, rather a source, as the target
to trigger some action.  Are we comfortable casting this behaviour
in stone by documenting it here?  Just checking by asking others, as
you are obviously comfortable enough to write this patch ;-)

Thanks.

> Document that in t/README to help new (or more seasoned) contributors
> that might not be aware.
>
> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>     t/README: mention test files are make targets
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1701%2Fphil-blain%2Ftests-makefile-targets-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1701/phil-blain/tests-makefile-targets-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1701
>
>  t/README | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/t/README b/t/README
> index 621d3b8c095..71211109338 100644
> --- a/t/README
> +++ b/t/README
> @@ -32,6 +32,13 @@ the tests.
>      ok 2 - plain with GIT_WORK_TREE
>      ok 3 - plain bare
>  
> +t/Makefile defines a target for each test file, such that you can also use
> +shell pattern matching to run a subset of the tests:
> +
> +    make *checkout*
> +
> +will run all tests with 'checkout' in their filename.
> +
>  Since the tests all output TAP (see https://testanything.org) they can
>  be run with any TAP harness. Here's an example of parallel testing
>  powered by a recent version of prove(1):
>
> base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b

Re: [PATCH] t/README: mention test files are make targets

[Philippe Blain]

Hi Junio,

Le 2024-03-24 à 12:10, Junio C Hamano a écrit :
> "Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> From: Philippe Blain <levraiphilippeblain@gmail.com>
>>
>> Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each
>> test file defines a target in the test Makefile, such that one can
>> invoke:
>>
>> 	make *checkout*
>>
>> to run all tests with 'checkout' in their filename. This is useful to
>> run a subset of tests when you have a good idea of what part of the code
>> is touched by the changes your are testing.
> 
> While I agree with the patch that this is a useful "feature" of
> t/Makefile, I've always felt it was ugly to use a file itself that
> we do not consider a build product, rather a source, as the target
> to trigger some action.  Are we comfortable casting this behaviour
> in stone by documenting it here?

Since '$(T)' is listed at the bottom of the Makefile as .PHONY,
I think it is OK and not that ugly since this uses a documented feature
of make.

Cheers,
Philippe.

Re: [PATCH] t/README: mention test files are make targets

[Junio C Hamano]

Philippe Blain <levraiphilippeblain@gmail.com> writes:

> Since '$(T)' is listed at the bottom of the Makefile as .PHONY,
> I think it is OK and not that ugly since this uses a documented feature
> of make.

You're prehaps right.

I've always felt that the documented .PHONY feature was to mark
targets that do not correspond to any filename on the filesystem,
e.g., "all", "clean", "install".  Of course these can exist as
filenames as well, and .PHONY works as an instruction that says "the
existence or freshness of these targets do not matter at all".  For
our use, it is OK.

I however wonder if marking $(T) as .PHONY is the right thing to
begin with.  Declaring that the existence or freshness of
t0000-basic.sh does not matter means we will not be able to later
write rules other than "just run it!" that does depend on the
freshness of t0000-basic.sh file, no?  IOW, if we wanted to add a
target like this at the end of the t/Makefile:

        t-combined.sh: t0000-basic.sh t0001-init.sh
                cat t0000-basic.sh t0001-init.sh >"$@"

then "make -C t t-combined.sh" would end up running these two test
scripts (because they are .PHONY) and then leave the concatenation
in t-combined.sh file.  Without changing anything, doing the same
"make -C t t-combined.sh" again will again run these two tests and
recreate the same t-combined.sh file, even though there is no need
to.

So I think that is what I felt ugly.

As long as we do not use these $(T) files as an input to some other
thing and list them as the dependencies, we are OK, though.

Re: [PATCH] t/README: mention test files are make targets

[Chris Torek]

On Mon, Mar 25, 2024 at 2:49 AM Junio C Hamano <gitster@pobox.com> wrote:
> As long as we do not use these $(T) files as an input to some other
> thing and list them as the dependencies, we are OK, though.

You could (maybe later / at need) stop listing them as `.PHONY` and
instead use:

    $(T)::
        sh -c ./$@

or similar, so that some $(T) *can* be an input. Note that this requires
using double-colon rules earlier to build the test.

I wouldn't do this without a pretty strong reason though.

Chris

Re: [PATCH] t/README: mention test files are make targets

[Junio C Hamano]

Chris Torek <chris.torek@gmail.com> writes:

> On Mon, Mar 25, 2024 at 2:49 AM Junio C Hamano <gitster@pobox.com> wrote:
>> As long as we do not use these $(T) files as an input to some other
>> thing and list them as the dependencies, we are OK, though.
>
> You could (maybe later / at need) stop listing them as `.PHONY` and
> instead use:
>
>     $(T)::
>         sh -c ./$@
>
> or similar, so that some $(T) *can* be an input. Note that this requires
> using double-colon rules earlier to build the test.
>
> I wouldn't do this without a pretty strong reason though.

Me neither.

I personally think a target that is marked as .PHONY and does not
use double-colon rule is a bug by itself but that is a separate
story.

In any case, just to avoid leaving the thread hanging, I'll take the
patch as is, as it documents a useful trick in the status quo.

Thanks, all.