git-clone: path or ssh problem with git-upload-pack in 1.6.0?

git-clone: path or ssh problem with git-upload-pack in 1.6.0?

[Paul Johnston]

I'm having trouble with git-clone and I'm wondering if there's
something I'm doing wrong or something wrong with git.  Probably the
former, hopefully someone can set me straight.

I'm an admittedly novice git user. I'm trying to clone a repository
over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
installed from a macports package.  This installs into /opt/local/bin.
I also cloned git from HEAD and 'make; make install'ed into ~/bin,
this is the version shown below.  Either way, these are
nonstandard/non-system-wide installation locations, and it requires
that my shell PATH reflect this, obviously.

----------------------------------------------

# About my installation
imac:~ paul$ which git
/Users/paul/bin/git

imac:~ paul$ git --version
git version 1.6.0.1.285.g1070

imac:~ paul$ ls /opt/local/bin/git*
/opt/local/bin/git
/opt/local/bin/git-upload-archive       /opt/local/bin/gitk
/opt/local/bin/git-receive-pack         /opt/local/bin/git-upload-pack

imac:~ paul$ ls ~/bin/git*
/Users/paul/bin/git                     /Users/paul/bin/git-shell
         /Users/paul/bin/git-upload-pack
/Users/paul/bin/git-receive-pack        /Users/paul/bin/git-upload-archive

# Make a simple reposirtory /tmp/test/.git
imac:~ paul$ cd /tmp
imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
init; git add README; git commit -a -m'Initial import'; cd ..
Initialized empty Git repository in /private/tmp/test/.git/
Created initial commit 49c10e6: Initial import
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README

# OK, this fails expectedly
imac:/tmp paul$ cd ~
imac:~ paul$ git clone ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
bash: line 1: git-upload-pack: command not found
fatal: The remote end hung up unexpectedly

# This has been killing me...  Have tried numerous variants of the
same with similar results...
imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack
ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
remote: fatal: exec pack-objects failed.
error: git-upload-pack: git-pack-objects died with error.
fatal: git-upload-pack: aborting due to possible repository corruption
on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOF
fatal: index-pack failed

# Fix it with symlinks in /usr/bin
imac:~ paul$ cd /usr/bin/
imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* .
imac:/usr/bin paul$ ls -al git*
lrwxr-xr-x   1 root  wheel  18 Sep  9 19:18 git -> /opt/local/bin/git
lrwxr-xr-x   1 root  wheel  31 Sep  9 19:18 git-receive-pack ->
/opt/local/bin/git-receive-pack
lrwxr-xr-x   1 root  wheel  33 Sep  9 19:18 git-upload-archive ->
/opt/local/bin/git-upload-archive
lrwxr-xr-x   1 root  wheel  30 Sep  9 19:18 git-upload-pack ->
/opt/local/bin/git-upload-pack
lrwxr-xr-x   1 root  wheel  19 Sep  9 19:18 gitk -> /opt/local/bin/gitk

# And now git-clone works as expected...
imac:/usr/bin paul$ cd
imac:~ paul$ git clone ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

imac:~ paul$ find test/
test/
test//.git
   { file listing removed for brevity... }
test//README

-------------------------------------------

OK, great, now it works. I suspect I'm not using the --upload-pack
option correctly or my ssh installation is suboptimal, but as I could
find no existing documentation on the subject, I am posting this
message.  Any thoughts?

Paul Johnston

git-clone: path or ssh problem with git-upload-pack in 1.6.0?

[Paul Johnston]

I'm having trouble with git-clone and I'm wondering if there's
something I'm doing wrong or something wrong with git.  Probably the
former, hopefully someone can set me straight.

I'm an admittedly novice git user. I'm trying to clone a repository
over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
installed from a macports package.  This installs into /opt/local/bin.
I also cloned git from HEAD and 'make; make install'ed into ~/bin,
this is the version shown below.  Either way, these are
nonstandard/non-system-wide installation locations, and it requires
that my shell PATH reflect this, obviously.

----------------------------------------------

# About my installation
imac:~ paul$ which git
/Users/paul/bin/git

imac:~ paul$ git --version
git version 1.6.0.1.285.g1070

imac:~ paul$ ls /opt/local/bin/git*
/opt/local/bin/git
/opt/local/bin/git-upload-archive       /opt/local/bin/gitk
/opt/local/bin/git-receive-pack         /opt/local/bin/git-upload-pack

imac:~ paul$ ls ~/bin/git*
/Users/paul/bin/git
/Users/paul/bin/git-shell
/Users/paul/bin/git-upload-pack
/Users/paul/bin/git-receive-pack
/Users/paul/bin/git-upload-archive

# Make a simple reposirtory /tmp/test/.git
imac:~ paul$ cd /tmp
imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
init; git add README; git commit -a -m'Initial import'; cd ..
Initialized empty Git repository in /private/tmp/test/.git/
Created initial commit 49c10e6: Initial import
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README

# OK, this fails expectedly
imac:/tmp paul$ cd ~
imac:~ paul$ git clone ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
bash: line 1: git-upload-pack: command not found
fatal: The remote end hung up unexpectedly

# This has been killing me...  Have tried numerous variants of the
same with similar results...
imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack
ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
remote: fatal: exec pack-objects failed.
error: git-upload-pack: git-pack-objects died with error.
fatal: git-upload-pack: aborting due to possible repository corruption
on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOF
fatal: index-pack failed

# Fix it with symlinks in /usr/bin
imac:~ paul$ cd /usr/bin/
imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* .
imac:/usr/bin paul$ ls -al git*
lrwxr-xr-x   1 root  wheel  18 Sep  9 19:18 git -> /opt/local/bin/git
lrwxr-xr-x   1 root  wheel  31 Sep  9 19:18 git-receive-pack ->
/opt/local/bin/git-receive-pack
lrwxr-xr-x   1 root  wheel  33 Sep  9 19:18 git-upload-archive ->
/opt/local/bin/git-upload-archive
lrwxr-xr-x   1 root  wheel  30 Sep  9 19:18 git-upload-pack ->
/opt/local/bin/git-upload-pack
lrwxr-xr-x   1 root  wheel  19 Sep  9 19:18 gitk -> /opt/local/bin/gitk

# And now git-clone works as expected...
imac:/usr/bin paul$ cd
imac:~ paul$ git clone ssh://localhost/tmp/test/.git
Initialized empty Git repository in /Users/paul/test/.git/
Password:
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

imac:~ paul$ find test/
test/
test//.git
  { file listing removed for brevity... }
test//README

-------------------------------------------

OK, great, now it works. I suspect I'm not using the --upload-pack
option correctly or my ssh installation is suboptimal, but as I could
find no existing documentation on the subject, I am posting this
message.  Any thoughts?

Paul Johnston

[PATCH] help git-upload-pack find git

[Michael J Gruber]

The --upload-pack option to clone raises the expectation that finding
git-upload-pack on the server side is enough. But git-upload-pack needs
to find git (for forking "git pack-objects") which fails if git is not
in $PATH.

This patch makes git-upload-pack use git_set_argv0_path() so that an
explicitely specified path for git-upload-pack is added to the path
which is used when fork()ing.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 upload-pack.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

The corresponding bug was reported by Paul Johnston who wondered why "git clone"
failed when specifying --upload-pack for an out-of-$PATH installation of git.
I'm not sure whether we should encourage this, but the --upload-pack option
clearly gives the impression that git-upload-pack is all that is needed.

I haven't looked at other server side programmes yet regarding this issue,
but "git clone" works fine with this patch in the situation where it
would not without.

diff --git a/upload-pack.c b/upload-pack.c
index e5adbc0..1fb59de 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -615,6 +615,21 @@ int main(int argc, char **argv)
 	char *dir;
 	int i;
 	int strict = 0;
+	const char *cmd = argv[0] && *argv[0] ? argv[0] : "git-upload-pack";
+	char *slash = (char *)cmd + strlen(cmd);
+
+	/*
+	 * Take the basename of argv[0] as the command
+	 * name, and the dirname as the default exec_path
+	 * if we don't have anything better.
+	 */
+	do
+		--slash;
+	while (cmd <= slash && !is_dir_sep(*slash));
+	if (cmd <= slash) {
+		*slash++ = 0;
+		git_set_argv0_path(cmd);
+	}
 
 	for (i = 1; i < argc; i++) {
 		char *arg = argv[i];
-- 
1.6.0.1.308.gede4c

Re: [PATCH] help git-upload-pack find git

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> The corresponding bug was reported by Paul Johnston who wondered why
> "git clone" failed when specifying --upload-pack for an out-of-$PATH
> installation of git.  I'm not sure whether we should encourage this, but
> the --upload-pack option clearly gives the impression that
> git-upload-pack is all that is needed.

Another possibility would be to teach exec_cmd.c:setup_path() to add the
directory specified by $(bindir) to PATH after we add GIT_EXEC_PATH to
it.  That should cover the case David Miller reported, shouldn't it?

Re: [PATCH] help git-upload-pack find git

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 15.09.2008 21:34:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> The corresponding bug was reported by Paul Johnston who wondered why
>> "git clone" failed when specifying --upload-pack for an out-of-$PATH
>> installation of git.  I'm not sure whether we should encourage this, but
>> the --upload-pack option clearly gives the impression that
>> git-upload-pack is all that is needed.
> 
> Another possibility would be to teach exec_cmd.c:setup_path() to add the
> directory specified by $(bindir) to PATH after we add GIT_EXEC_PATH to
> it.  That should cover the case David Miller reported, shouldn't it?

Probably. I read that thread only after submitting my patch for
upload-pack but suspected that other server side incarnations may suffer
from the same problem.

I was actually surprised that setup_path() uses argv0_path without
setting it, same as with argv_exec_path. I assumed this is for a good
reason, I'm lacking the code base overview to judge this myself.

In any case, git.c sets argv0_path early, messes a bit with argv[0] and
calls setup_path afterwards anyways. So adding the path in setup_path()
should not hurt any "git foo" command.

One could construe situations where even that wouldn't help, because
git-upload-pack can't pass --exec-dir to git and they can be in
different locations - but I think that's crazy.

git.c, upload-pack.c, receive.pack.c and shell.c are the only callers.
setup_path() needs to get a parameter. If shell.c should profit from the
change then it needs to be taught how to pass an absolute path to
do_{generic,cvs}_cmd().

So, I guess the general approach (change setup() path and have every
caller profit) is OK. OK with you?

Michael

Re: [PATCH] help git-upload-pack find git

[Johannes Sixt]

Michael J Gruber schrieb:
> I was actually surprised that setup_path() uses argv0_path without
> setting it, same as with argv_exec_path. I assumed this is for a good
> reason, I'm lacking the code base overview to judge this myself.

It argv_exec_path comes from git --exec-path=..., hence, git (via git.c)
is the only caller of setup_path() that is able to set it. We can leave
that one out of the equation.

> In any case, git.c sets argv0_path early, messes a bit with argv[0] and
> calls setup_path afterwards anyways. So adding the path in setup_path()
> should not hurt any "git foo" command.
> 
> One could construe situations where even that wouldn't help, because
> git-upload-pack can't pass --exec-dir to git and they can be in
> different locations - but I think that's crazy.
> 
> git.c, upload-pack.c, receive.pack.c and shell.c are the only callers.
> setup_path() needs to get a parameter. If shell.c should profit from the
> change then it needs to be taught how to pass an absolute path to
> do_{generic,cvs}_cmd().
> 
> So, I guess the general approach (change setup() path and have every
> caller profit) is OK. OK with you?

Have you studied the commit message of e1464ca7bb0d (Record the command
invocation path early) and the context in which this commit occurs? It's
about relocatable git installations and how system_path() derives various
other paths from argv[0].

Please show how you think you could change setup_path(), but keep in mind
that in git.c you neither can do the equivalent of git_set_argv0_path()
later nor setup_path() earlier.

-- Hannes

Re: [PATCH] help git-upload-pack find git

[Michael J Gruber]

Johannes Sixt venit, vidit, dixit 16.09.2008 08:17:
> Michael J Gruber schrieb:
>> I was actually surprised that setup_path() uses argv0_path without
>> setting it, same as with argv_exec_path. I assumed this is for a good
>> reason, I'm lacking the code base overview to judge this myself.
> 
> It argv_exec_path comes from git --exec-path=..., hence, git (via git.c)
> is the only caller of setup_path() that is able to set it. We can leave
> that one out of the equation.
> 
>> In any case, git.c sets argv0_path early, messes a bit with argv[0] and
>> calls setup_path afterwards anyways. So adding the path in setup_path()
>> should not hurt any "git foo" command.
>>
>> One could construe situations where even that wouldn't help, because
>> git-upload-pack can't pass --exec-dir to git and they can be in
>> different locations - but I think that's crazy.
>>
>> git.c, upload-pack.c, receive.pack.c and shell.c are the only callers.
>> setup_path() needs to get a parameter. If shell.c should profit from the
>> change then it needs to be taught how to pass an absolute path to
>> do_{generic,cvs}_cmd().
>>
>> So, I guess the general approach (change setup() path and have every
>> caller profit) is OK. OK with you?
> 
> Have you studied the commit message of e1464ca7bb0d (Record the command
> invocation path early) and the context in which this commit occurs? It's
> about relocatable git installations and how system_path() derives various
> other paths from argv[0].

That is what I meant by my lack of overview ;)
I might have implemented what amounts to a revert of e1464ca7bb0d otherwise.

> Please show how you think you could change setup_path(), but keep in mind
> that in git.c you neither can do the equivalent of git_set_argv0_path()
> later nor setup_path() earlier.

Well, I'd be happy with the change in upload-pack as submitted. Junio
suggested the more general approach. In fact I'd prefer everyone set
their path, but not everyone seems to be allowed to in their restricted
environments.

Why is it that setup_path() cannot be done before commit_pager_choice()?
We need the pager in case list_common_cmds_help() is triggered, but why
can't setup_path() be before that? Not because of "--", really.

Michael

Re: [PATCH] help git-upload-pack find git

[Johannes Sixt]

Michael J Gruber schrieb:
> Why is it that setup_path() cannot be done before commit_pager_choice()?
> We need the pager in case list_common_cmds_help() is triggered, but why
> can't setup_path() be before that? Not because of "--", really.

I think that setup_path() must be called *after* handle_options() because
of this use-case:

  $ git --exec-path=/some/where foo

and git_set_argv0_path() must be called *before* handle_options() because
of this use-case:

  $ git --exec-path

-- Hannes

Re: [PATCH] help git-upload-pack find git

[Michael J Gruber]

Johannes Sixt venit, vidit, dixit 16.09.2008 15:43:
> Michael J Gruber schrieb:
>> Why is it that setup_path() cannot be done before commit_pager_choice()?
>> We need the pager in case list_common_cmds_help() is triggered, but why
>> can't setup_path() be before that? Not because of "--", really.
> 
> I think that setup_path() must be called *after* handle_options() because
> of this use-case:
> 
>   $ git --exec-path=/some/where foo
> 
> and git_set_argv0_path() must be called *before* handle_options() because
> of this use-case:
> 
>   $ git --exec-path
> 
> -- Hannes

Tricky, I see. But still I'm getting more and more confused:
handle_options() calls git_exec_path() in this case, which does (unless
there is argv_exec_path or the variable):
return system_path(GIT_EXEC_PATH)
Duh, that comes from the Makefile, -DGIT_EXEC_PATH... Back to somewhat
less confused state.

OK, now how about

A: path as submitted

or

B: leave git.c as is, except that setup_path(NULL) is used; have
setup_path(argv0path) call git_set_argv0_path() unless the arg is NULL;
tell upload-pack etc. to stuff (other setup_path() callers) their
argv[0] into setup_path(). Junio, is that what you meant?

Michael

Re: git-clone: path or ssh problem with git-upload-pack in 1.6.0?

[Michael J Gruber]

Paul Johnston venit, vidit, dixit 15.09.2008 10:30:
> I'm having trouble with git-clone and I'm wondering if there's
> something I'm doing wrong or something wrong with git.  Probably the
> former, hopefully someone can set me straight.
> 
> I'm an admittedly novice git user. I'm trying to clone a repository
> over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
> installed from a macports package.  This installs into /opt/local/bin.
> I also cloned git from HEAD and 'make; make install'ed into ~/bin,
> this is the version shown below.  Either way, these are
> nonstandard/non-system-wide installation locations, and it requires
> that my shell PATH reflect this, obviously.
> 
> ----------------------------------------------
> 
> # About my installation
> imac:~ paul$ which git
> /Users/paul/bin/git
> 
> imac:~ paul$ git --version
> git version 1.6.0.1.285.g1070
> 
> imac:~ paul$ ls /opt/local/bin/git*
> /opt/local/bin/git
> /opt/local/bin/git-upload-archive       /opt/local/bin/gitk
> /opt/local/bin/git-receive-pack         /opt/local/bin/git-upload-pack
> 
> imac:~ paul$ ls ~/bin/git*
> /Users/paul/bin/git
> /Users/paul/bin/git-shell
> /Users/paul/bin/git-upload-pack
> /Users/paul/bin/git-receive-pack
> /Users/paul/bin/git-upload-archive
> 
> # Make a simple reposirtory /tmp/test/.git
> imac:~ paul$ cd /tmp
> imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
> init; git add README; git commit -a -m'Initial import'; cd ..
> Initialized empty Git repository in /private/tmp/test/.git/
> Created initial commit 49c10e6: Initial import
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 README
> 
> # OK, this fails expectedly
> imac:/tmp paul$ cd ~
> imac:~ paul$ git clone ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> bash: line 1: git-upload-pack: command not found
> fatal: The remote end hung up unexpectedly
> 
> # This has been killing me...  Have tried numerous variants of the
> same with similar results...
> imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack
> ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> remote: fatal: exec pack-objects failed.
> error: git-upload-pack: git-pack-objects died with error.
> fatal: git-upload-pack: aborting due to possible repository corruption
> on the remote side.
> remote: aborting due to possible repository corruption on the remote side.
> fatal: early EOF
> fatal: index-pack failed
> 
> # Fix it with symlinks in /usr/bin
> imac:~ paul$ cd /usr/bin/
> imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* .
> imac:/usr/bin paul$ ls -al git*
> lrwxr-xr-x   1 root  wheel  18 Sep  9 19:18 git -> /opt/local/bin/git
> lrwxr-xr-x   1 root  wheel  31 Sep  9 19:18 git-receive-pack ->
> /opt/local/bin/git-receive-pack
> lrwxr-xr-x   1 root  wheel  33 Sep  9 19:18 git-upload-archive ->
> /opt/local/bin/git-upload-archive
> lrwxr-xr-x   1 root  wheel  30 Sep  9 19:18 git-upload-pack ->
> /opt/local/bin/git-upload-pack
> lrwxr-xr-x   1 root  wheel  19 Sep  9 19:18 gitk -> /opt/local/bin/gitk
> 
> # And now git-clone works as expected...
> imac:/usr/bin paul$ cd
> imac:~ paul$ git clone ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> remote: Counting objects: 3, done.
> remote: Total 3 (delta 0), reused 0 (delta 0)
> Receiving objects: 100% (3/3), done.
> 
> imac:~ paul$ find test/
> test/
> test//.git
>   { file listing removed for brevity... }
> test//README
> 
> -------------------------------------------
> 
> OK, great, now it works. I suspect I'm not using the --upload-pack
> option correctly or my ssh installation is suboptimal, but as I could
> find no existing documentation on the subject, I am posting this
> message.  Any thoughts?

re ssh: I'm surprised your PATH doesn't include $HOME/bin when sshing.
That should make everything work.

re upload-pack: That indeed sounds as if upload-pack can't execute
pack-objects. I checked by moving my bin away, and clone failed. Seems
everyone is using proper PATHS ;)

In fact, upload-pack tries to call "git pack-objects" and fails if it
can't find git. Patch upcoming...

Michael

Re: git-clone: path or ssh problem with git-upload-pack in 1.6.0?

[Michael Wookey]

> I'm having trouble with git-clone and I'm wondering if there's
> something I'm doing wrong or something wrong with git.  Probably the
> former, hopefully someone can set me straight.
>
> I'm an admittedly novice git user. I'm trying to clone a repository
> over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
> installed from a macports package.  This installs into /opt/local/bin.
> I also cloned git from HEAD and 'make; make install'ed into ~/bin,
> this is the version shown below.  Either way, these are
> nonstandard/non-system-wide installation locations, and it requires
> that my shell PATH reflect this, obviously.

The following might help..

http://marc.info/?l=git&m=121378876831164&w=2