profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

[Paul Smith]

Is anyone aware of this?  It seems that profile-fast fails when invoked
from a downloaded tarball, if you are in a Git repository when you
unpack it.

So, for example, I have:

  $ cd $HOME/src
  $ git status
  On branch master
  Your branch is up-to-date with 'origin/master'.

(this is NOT the Git source repo, this is my personal repo containing
scripts used to build various source packages)

Then I download git-2.2.1.tar.xz, unpack it, configure, and run "make
profile-fast".

After it builds everything it starts trying to run the tests, and the
t/perf/run script fails (I added some -x flags in the scripts):

  === Running 9 tests in this tree ===
  cp: cannot stat '/home/psmith/src/git-2.2.1/t/..//home/psmith/src/.git/objects': No such file or directory
  error: failed to copy repository '/home/psmith/src/git-2.2.1/t/..' to '/home/psmith/src/git-2.2.1/t/perf/trash directory.p0000-perf-lib-sanity'

The problem is in the t/perf/perf-lib.sh:test_perf_create_repo_from()
function, where we see this:

  repo="$1"
  source="$2"
  source_git=$source/$(cd "$source" && git rev-parse --git-dir)

The function is invoked as:

  test_perf_create_repo_from '/home/psmith/src/git-2.2.1/t/perf/trash directory.p5302-pack-index' /home/psmith/src/git-2.2.1/t/..

but this doesn't work, because "$source" is not a Git repository; it's
the root of the unpacked tarball.  But when we run "git rev-parse" in
it, it finds the parent Git directory ($HOME/git) and uses that, so the
value of source_git becomes:

  /home/psmith/src/git-2.2.1/t/..//home/psmith/src/.git/objects

which is obviously invalid.  I think you want source_git to be set
something like this:

  source_git=$(cd "$source" && git rev-parse --git-dir || echo "$source")

instead.  There are other ways to do this of course, but this worked for
me... basically we want to use either git rev-parse OR $source but not
both.  I think...?

Re: profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

[Junio C Hamano]

Paul Smith <paul@mad-scientist.net> writes:

> Is anyone aware of this?  It seems that profile-fast fails when invoked
> from a downloaded tarball, if you are in a Git repository when you
> unpack it.

Not me (I don't do profile build), but I am not surprised.

> The problem is in the t/perf/perf-lib.sh:test_perf_create_repo_from()
> function, where we see this:
>
>   repo="$1"
>   source="$2"
>   source_git=$source/$(cd "$source" && git rev-parse --git-dir)
>
> The function is invoked as:
>
>   test_perf_create_repo_from '/home/psmith/src/git-2.2.1/t/perf/trash
> directory.p5302-pack-index' /home/psmith/src/git-2.2.1/t/..
>
> but this doesn't work, because "$source" is not a Git repository; it's
> the root of the unpacked tarball.

Yup, that is exactly why I said I am not surprised.

I do not think profile build is prepared to be run without having
our history (after all, it is not test_perf_create_REPO_from, not
test_perf_create_source_directory_of_git_from).  It wants to create
a repository that hosts a reasonably sized but not too big a
project, i.e. us.

The safest and cleanest fix would be to make sure that the said
function checks if $source/.git is a repository, perhaps?

Re: profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

[Jeff King]

On Tue, Jan 06, 2015 at 03:13:30PM -0800, Junio C Hamano wrote:

> > The problem is in the t/perf/perf-lib.sh:test_perf_create_repo_from()
> > function, where we see this:
> >
> >   repo="$1"
> >   source="$2"
> >   source_git=$source/$(cd "$source" && git rev-parse --git-dir)
> >
> > The function is invoked as:
> >
> >   test_perf_create_repo_from '/home/psmith/src/git-2.2.1/t/perf/trash
> > directory.p5302-pack-index' /home/psmith/src/git-2.2.1/t/..
> >
> > but this doesn't work, because "$source" is not a Git repository; it's
> > the root of the unpacked tarball.
> 
> Yup, that is exactly why I said I am not surprised.
> 
> I do not think profile build is prepared to be run without having
> our history (after all, it is not test_perf_create_REPO_from, not
> test_perf_create_source_directory_of_git_from).  It wants to create
> a repository that hosts a reasonably sized but not too big a
> project, i.e. us.
> 
> The safest and cleanest fix would be to make sure that the said
> function checks if $source/.git is a repository, perhaps?

Hmph. We already dealt with this once in 93b5393 (Makefile: make perf
tests optional for profile build, 2014-08-19). That check explicitly
looks for ".git" (and not checking that we are in an outer git repo), so
I am not sure why it is kicking in at all. But probably the fix should
go there.

-Peff

Re: profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

[Jeff King]

On Mon, Jan 12, 2015 at 04:08:28PM -0500, Jeff King wrote:

> > I do not think profile build is prepared to be run without having
> > our history (after all, it is not test_perf_create_REPO_from, not
> > test_perf_create_source_directory_of_git_from).  It wants to create
> > a repository that hosts a reasonably sized but not too big a
> > project, i.e. us.
> > 
> > The safest and cleanest fix would be to make sure that the said
> > function checks if $source/.git is a repository, perhaps?
> 
> Hmph. We already dealt with this once in 93b5393 (Makefile: make perf
> tests optional for profile build, 2014-08-19). That check explicitly
> looks for ".git" (and not checking that we are in an outer git repo), so
> I am not sure why it is kicking in at all. But probably the fix should
> go there.

Oh, I see. It is because Paul is using profile-fast, which does not have
that check. And I did not add it as part of 93b5393, because it would
not make sense. If you do not do the perf tests, then profile-fast is
literally doing nothing. :)

I think the right solution is either:

  1. Switch to using "make profile", which will use the regular test
     suite to gather data, and skip the perf tests.

  2. Set GIT_PERF_REPO to some representative repo of your choice for
     gathering profiling data (it does not have to be git.git at all,
     but it is probably more useful if it is not a tiny toy repo).

We _could_ teach the perf code to look upwards for a surrounding repo
with `git rev-parse` rather than looking for ../../.git. But there is no
guarantee that the repo we find would actually be a useful one for doing
perf tests. I'd just as soon leave it as-is, and have people specify a
useful repo via GIT_PERF_REPO if they have one.

-Peff