[PATCH] makefile: add ability to run specific test files

[PATCH] makefile: add ability to run specific test files

[Jacob Keller]

Running a specific test file manually does not obtain the exact
environment setup by the Makefile. Add ability to run any of the tests
in the t/ directory so that a user can more quickly debug a failing
test. Otherwise, the entire test suite needs to be run, which can take a
vary long time.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 Makefile | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 07ea1058379a..86bdc4ed1ee9 100644
--- a/Makefile
+++ b/Makefile
@@ -2262,13 +2262,18 @@ export TEST_NO_MALLOC_CHECK
 
 ### Testing rules
 
+T = $(sort $(wildcard t/t[0-9][0-9][0-9][0-9]-*.sh))
+
+$(T):
+	$(MAKE) -C t $(notdir $@)
+
 test: all
 	$(MAKE) -C t/ all
 
 perf: all
 	$(MAKE) -C t/perf/ all
 
-.PHONY: test perf
+.PHONY: test perf $(T)
 
 test-ctype$X: ctype.o
 
-- 
2.0.1.475.g9b8d714

Re: [PATCH] makefile: add ability to run specific test files

[Junio C Hamano]

Jacob Keller <jacob.e.keller@intel.com> writes:

> Running a specific test file manually does not obtain the exact
> environment setup by the Makefile.

What kind of things are missing, exactly?  Perhaps that is something
you need to fix, instead of mucking with the top-level Makefile.

I recall last time when I did a patch like this I was told to look
into "make -C t" ;-)  What is different this round?

Re: [PATCH] makefile: add ability to run specific test files

[Keller, Jacob E]

On Wed, 2014-07-09 at 15:59 -0700, Junio C Hamano wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
> 
> > Running a specific test file manually does not obtain the exact
> > environment setup by the Makefile.
> 
> What kind of things are missing, exactly?  Perhaps that is something
> you need to fix, instead of mucking with the top-level Makefile.
> 
> I recall last time when I did a patch like this I was told to look
> into "make -C t" ;-)  What is different this round?

It uses the git from my environment instead of the git I have built,
which is bad since I don't really want to run make install.

Thanks,
Jake

Re: [PATCH] makefile: add ability to run specific test files

[Junio C Hamano]

On Wed, Jul 9, 2014 at 4:49 PM, Keller, Jacob E
<jacob.e.keller@intel.com> wrote:
> On Wed, 2014-07-09 at 15:59 -0700, Junio C Hamano wrote:
>>
>> What kind of things are missing, exactly?  Perhaps that is something
>> you need to fix, instead of mucking with the top-level Makefile.
>
> It uses the git from my environment instead of the git I have built,
> which is bad since I don't really want to run make install.

Are you sure about that?  Try adding something like

  die("I am broken");

at the very beginning of main() in git.c, rebuild your git (i.e.
"make", not "make install")
and then

  $ cd t
  $ sh ./t1234-test.sh -v

for any of the test scripts. You should see any test piece that runs "git" sees
"git" dying with that message.

Otherwise, there is something wrong with git you are building. Unless you have
a patch or two to t/test-lib.sh or something that breaks the test framework, you
should be able to test what you just have built without getting affected by what
is installed in your $PATH. After all, that is how we bootstrap git
from a tarball
without any installed version, and friends do not force friends install without
testing first ;-)

Re: [PATCH] makefile: add ability to run specific test files

[Keller, Jacob E]

On Wed, 2014-07-09 at 21:14 -0700, Junio C Hamano wrote:
> On Wed, Jul 9, 2014 at 4:49 PM, Keller, Jacob E
> <jacob.e.keller@intel.com> wrote:
> > On Wed, 2014-07-09 at 15:59 -0700, Junio C Hamano wrote:
> >>
> >> What kind of things are missing, exactly?  Perhaps that is something
> >> you need to fix, instead of mucking with the top-level Makefile.
> >
> > It uses the git from my environment instead of the git I have built,
> > which is bad since I don't really want to run make install.
> 
> Are you sure about that?  Try adding something like
> 
>   die("I am broken");
> 
> at the very beginning of main() in git.c, rebuild your git (i.e.
> "make", not "make install")
> and then
> 
>   $ cd t
>   $ sh ./t1234-test.sh -v
> 
> for any of the test scripts. You should see any test piece that runs "git" sees
> "git" dying with that message.
> 
> Otherwise, there is something wrong with git you are building. Unless you have
> a patch or two to t/test-lib.sh or something that breaks the test framework, you
> should be able to test what you just have built without getting affected by what
> is installed in your $PATH. After all, that is how we bootstrap git
> from a tarball
> without any installed version, and friends do not force friends install without
> testing first ;-)

Ok, I'll give it a shot. All I know for sure right now is running the
test directly passed and running from "make test" it failed.

I'll see if that makes any difference.

Thanks,
Jake

Re: [PATCH] makefile: add ability to run specific test files

[Jeff King]

On Thu, Jul 10, 2014 at 08:39:57PM +0000, Keller, Jacob E wrote:

> Ok, I'll give it a shot. All I know for sure right now is running the
> test directly passed and running from "make test" it failed.

When you say directly, I assume you mean "cd t && ./1234-xxx.sh".
You can also run a single-shot test like:

  cd t && make t1234-...

which may make the environment more like "make test".

-Peff

Re: [PATCH] makefile: add ability to run specific test files

[Keller, Jacob E]

On Thu, 2014-07-10 at 04:14 +0000, Junio C Hamano wrote:
> On Wed, Jul 9, 2014 at 4:49 PM, Keller, Jacob E
> <jacob.e.keller@intel.com> wrote:
> > On Wed, 2014-07-09 at 15:59 -0700, Junio C Hamano wrote:
> >>
> >> What kind of things are missing, exactly?  Perhaps that is something
> >> you need to fix, instead of mucking with the top-level Makefile.
> >
> > It uses the git from my environment instead of the git I have built,
> > which is bad since I don't really want to run make install.
> 
> Are you sure about that?  Try adding something like
> 
>   die("I am broken");
> 
> at the very beginning of main() in git.c, rebuild your git (i.e.
> "make", not "make install")
> and then
> 
>   $ cd t
>   $ sh ./t1234-test.sh -v
> 
> for any of the test scripts. You should see any test piece that runs "git" sees
> "git" dying with that message.
> 
> Otherwise, there is something wrong with git you are building. Unless you have
> a patch or two to t/test-lib.sh or something that breaks the test framework, you
> should be able to test what you just have built without getting affected by what
> is installed in your $PATH. After all, that is how we bootstrap git
> from a tarball
> without any installed version, and friends do not force friends install without
> testing first ;-)

This is even more interesting. I tried your die check, and it definitely
runs the correct version of git.

However, if I run the test directly:

cd t ; sh t3200-branch.sh -v

it passes.

if I run:

make test

that particular test fails. If I have this patch applied, and I run

make t/t3200-branch.sh

it also fails.

I have done this directly on current master branch. So something is
differing between the two test runs.

Also, if I run:

make -C t t3200-branch.sh

that passes, so it really *is* something setup by the main makefile.

Any more suggestions?

Thanks,
Jake