[PATCH] Fix problem with authentification on http repository.

[PATCH] Fix problem with authentification on http repository.

[jean.guyader]

From: Jean Guyader <jean.guyader@gmail.com>

Curl uses the option -u user:passwd and not the user:password
given in the url.
The solution was to extract user:password from the url and set
the option.

Here the regex used :
        sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'
---
 git-clone.sh     |    3 ++-
 git-fetch.sh     |    8 ++++----
 git-ls-remote.sh |    3 ++-
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/git-clone.sh b/git-clone.sh
index 5e582fe..57206ac 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -34,7 +34,8 @@ fi
 
 http_fetch () {
 	# $1 = Remote, $2 = Local
-	curl -nsfL $curl_extra_args "$1" >"$2" ||
+        curl_userpw=`echo "$1" | sed -re 's-http://([^:]*):([^@]+)@.*-\1:\2-g'`
+	curl -u "$curl_userpw" -nsfL $curl_extra_args "$1" >"$2" ||
 		case $? in
 		126|127) exit ;;
 		*)	 return $? ;;
diff --git a/git-fetch.sh b/git-fetch.sh
index e44af2c..0b7d055 100755
--- a/git-fetch.sh
+++ b/git-fetch.sh
@@ -268,13 +268,13 @@ fetch_per_ref () {
 		"`git config --bool http.noEPSV`" = true ]; then
 	      noepsv_opt="--disable-epsv"
 	  fi
-
-	  # Find $remote_name from ls-remote output.
+          pretty_remote=$(echo "$remote" | sed -re 's-http://([^:]+):([^@]+)@-http://\1:******@-g')
+          # Find $remote_name from ls-remote output.
 	  head=$(echo "$ls_remote_result" | \
 		git fetch--tool -s pick-rref "$remote_name" "-")
 	  expr "z$head" : "z$_x40\$" >/dev/null ||
-		die "No such ref $remote_name at $remote"
-	  echo >&2 "Fetching $remote_name from $remote using $proto"
+		die "No such ref $remote_name at $pretty_remote"
+	  echo >&2 "Fetching $remote_name from $pretty_remote using $proto"
 	  case "$quiet" in '') v=-v ;; *) v= ;; esac
 	  git-http-fetch $v -a "$head" "$remote" || exit
 	  ;;
diff --git a/git-ls-remote.sh b/git-ls-remote.sh
index d56cf92..d6e9906 100755
--- a/git-ls-remote.sh
+++ b/git-ls-remote.sh
@@ -61,7 +61,8 @@ http://* | https://* | ftp://* )
 		"`git config --bool http.noEPSV`" = true ]; then
 		curl_extra_args="${curl_extra_args} --disable-epsv"
 	fi
-	curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
+        curl_userpw=`echo "$peek_repo" | sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'`
+	curl -u "$curl_userpw" -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
 		echo "failed	slurping"
 	;;
 
-- 
1.5.2.4

Re: [PATCH] Fix problem with authentification on http repository.

[Johannes Schindelin]

Hi,

On Sun, 30 Sep 2007, jean.guyader@linkea.org wrote:

>  git-clone.sh     |    3 ++-
>  git-fetch.sh     |    8 ++++----
>  git-ls-remote.sh |    3 ++-

git-fetch is already a builtin in "next", and -clone and -ls-remote are 
likely to follow suit.

Ciao,
Dscho

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

jean.guyader@linkea.org writes:

> From: Jean Guyader <jean.guyader@gmail.com>
>
> Curl uses the option -u user:passwd and not the user:password
> given in the url.
> The solution was to extract user:password from the url and set
> the option.
>
> Here the regex used :
>         sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'

This is more like "allowing embedded authentication credentials
in URL for http transport".

We never supported URLs with embedded credentials (see
Documentation/urls.txt), partly because nobody asked for it, but
more importantly because giving -n to curl to have it read from
user's .netrc is generally much more preferred approach.

Re: [PATCH] Fix problem with authentification on http repository.

[Johannes Schindelin]

Hi,

On Sat, 29 Sep 2007, Junio C Hamano wrote:

> jean.guyader@linkea.org writes:
> 
> > From: Jean Guyader <jean.guyader@gmail.com>
> >
> > Curl uses the option -u user:passwd and not the user:password
> > given in the url.
> > The solution was to extract user:password from the url and set
> > the option.
> >
> > Here the regex used :
> >         sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'
> 
> This is more like "allowing embedded authentication credentials
> in URL for http transport".
> 
> We never supported URLs with embedded credentials (see
> Documentation/urls.txt), partly because nobody asked for it, but
> more importantly because giving -n to curl to have it read from
> user's .netrc is generally much more preferred approach.

To elaborate on that: if you fetch from somewhere, your url, username and 
password can be read from the output of "ps ax | grep http" very easily.

Ciao,
Dscho

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Sat, 29 Sep 2007, Junio C Hamano wrote:
>
>> We never supported URLs with embedded credentials (see
>> Documentation/urls.txt), partly because nobody asked for it, but
>> more importantly because giving -n to curl to have it read from
>> user's .netrc is generally much more preferred approach.
>
> To elaborate on that: if you fetch from somewhere, your url, username and 
> password can be read from the output of "ps ax | grep http" very easily.

Actually Documentation/howto/setup-git-server-over-http.txt
talks about http://user@server/path/ format.  How well does this
work in practice?  If it does, we should update Documentation/urls.txt
to allow optional user@ there like...

---

 Documentation/urls.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index e67f914..c9aab86 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -6,8 +6,8 @@ to name the remote repository:
 
 ===============================================================
 - rsync://host.xz/path/to/repo.git/
-- http://host.xz/path/to/repo.git/
-- https://host.xz/path/to/repo.git/
+- http://{startsb}user@{endsb}host.xz/path/to/repo.git/
+- https://{startsb}user@{endsb}host.xz/path/to/repo.git/
 - git://host.xz/path/to/repo.git/
 - git://host.xz/~user/path/to/repo.git/
 - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Sat, 29 Sep 2007, Junio C Hamano wrote:
>
>> We never supported URLs with embedded credentials (see
>> Documentation/urls.txt), partly because nobody asked for it, but
>> more importantly because giving -n to curl to have it read from
>> user's .netrc is generally much more preferred approach.
>
> To elaborate on that: if you fetch from somewhere, your url, username and 
> password can be read from the output of "ps ax | grep http" very easily.

Actually Documentation/howto/setup-git-server-over-http.txt
talks about http://user@server/path/ format.  How well does this
work in practice?  If it does, we should update Documentation/urls.txt
to allow optional user@ there like...

---

 Documentation/urls.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index e67f914..c9aab86 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -6,8 +6,8 @@ to name the remote repository:
 
 ===============================================================
 - rsync://host.xz/path/to/repo.git/
-- http://host.xz/path/to/repo.git/
-- https://host.xz/path/to/repo.git/
+- http://{startsb}user@{endsb}host.xz/path/to/repo.git/
+- https://{startsb}user@{endsb}host.xz/path/to/repo.git/
 - git://host.xz/path/to/repo.git/
 - git://host.xz/~user/path/to/repo.git/
 - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/

Re: [PATCH] Fix problem with authentification on http repository.

[Mike Hommey]

On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
> >
> >> We never supported URLs with embedded credentials (see
> >> Documentation/urls.txt), partly because nobody asked for it, but
> >> more importantly because giving -n to curl to have it read from
> >> user's .netrc is generally much more preferred approach.
> >
> > To elaborate on that: if you fetch from somewhere, your url, username and 
> > password can be read from the output of "ps ax | grep http" very easily.
> 
> Actually Documentation/howto/setup-git-server-over-http.txt
> talks about http://user@server/path/ format.  How well does this
> work in practice?  If it does, we should update Documentation/urls.txt
> to allow optional user@ there like...

AFAIK, there is nothing to deal with asking a password to the user in git
in this case, so that doesn't work.

Mike

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
>> >
>> >> We never supported URLs with embedded credentials (see
>> >> Documentation/urls.txt), partly because nobody asked for it, but
>> >> more importantly because giving -n to curl to have it read from
>> >> user's .netrc is generally much more preferred approach.
>> >
>> > To elaborate on that: if you fetch from somewhere, your url, username and 
>> > password can be read from the output of "ps ax | grep http" very easily.
>> 
>> Actually Documentation/howto/setup-git-server-over-http.txt
>> talks about http://user@server/path/ format.  How well does this
>> work in practice?  If it does, we should update Documentation/urls.txt
>> to allow optional user@ there like...
>
> AFAIK, there is nothing to deal with asking a password to the user in git
> in this case, so that doesn't work.

Then perhaps the howto needs updating?

Re: [PATCH] Fix problem with authentification on http repository.

[Mike Hommey]

On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> Mike Hommey <mh@glandium.org> writes:
> 
> > On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> 
> >> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
> >> >
> >> >> We never supported URLs with embedded credentials (see
> >> >> Documentation/urls.txt), partly because nobody asked for it, but
> >> >> more importantly because giving -n to curl to have it read from
> >> >> user's .netrc is generally much more preferred approach.
> >> >
> >> > To elaborate on that: if you fetch from somewhere, your url, username and 
> >> > password can be read from the output of "ps ax | grep http" very easily.
> >> 
> >> Actually Documentation/howto/setup-git-server-over-http.txt
> >> talks about http://user@server/path/ format.  How well does this
> >> work in practice?  If it does, we should update Documentation/urls.txt
> >> to allow optional user@ there like...
> >
> > AFAIK, there is nothing to deal with asking a password to the user in git
> > in this case, so that doesn't work.
> 
> Then perhaps the howto needs updating?

Actually, the howto tells to setup a ~/.netrc file, which should make
the http://user@server/path/ format work. I don't know if dropping user@
would still work, that depends how libcurl works with it.

Mike

Re: [PATCH] Fix problem with authentification on http repository.

[Daniel Stenberg]

On Wed, 9 Jul 2008, Mike Hommey wrote:

> Actually, the howto tells to setup a ~/.netrc file, which should make
> the http://user@server/path/ format work. I don't know if dropping user@
> would still work, that depends how libcurl works with it.

git has the netrc parser set to "optional", which makes libcurl use the login 
name given in the URL (if present) and search for the associated password in 
the .netrc file.

Typical users probably only have one user per host name in the .netrc file and 
then the name part in the URL provides no particular extra goodness but can 
just as well be excluded.

-- 

  / daniel.haxx.se

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> >> Actually Documentation/howto/setup-git-server-over-http.txt
>> >> talks about http://user@server/path/ format.  How well does this
>> >> work in practice?  If it does, we should update Documentation/urls.txt
>> >> to allow optional user@ there like...
>> >
>> > AFAIK, there is nothing to deal with asking a password to the user in git
>> > in this case, so that doesn't work.
>> 
>> Then perhaps the howto needs updating?
>
> Actually, the howto tells to setup a ~/.netrc file, which should make
> the http://user@server/path/ format work. I don't know if dropping user@
> would still work, that depends how libcurl works with it.

Ok, so the documentation update I suggested earlier may actually be
valid, right?

Re: [PATCH] Fix problem with authentification on http repository.

[Mike Hommey]

On Wed, Jul 09, 2008 at 01:18:22AM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> Mike Hommey <mh@glandium.org> writes:
> 
> > On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> > ...
> >> >> Actually Documentation/howto/setup-git-server-over-http.txt
> >> >> talks about http://user@server/path/ format.  How well does this
> >> >> work in practice?  If it does, we should update Documentation/urls.txt
> >> >> to allow optional user@ there like...
> >> >
> >> > AFAIK, there is nothing to deal with asking a password to the user in git
> >> > in this case, so that doesn't work.
> >> 
> >> Then perhaps the howto needs updating?
> >
> > Actually, the howto tells to setup a ~/.netrc file, which should make
> > the http://user@server/path/ format work. I don't know if dropping user@
> > would still work, that depends how libcurl works with it.
> 
> Ok, so the documentation update I suggested earlier may actually be
> valid, right?

The problem is that these urls won't work out of the box if the user
doesn't setup a .netrc file. I'd say it would be better not to add these
forms in Documentation/urls.txt until git prompts for the password
when it is missing.

Note that http://user:pass@server/path/ /should/ work (but that would
need validation), though not a good idea on command line.

Mike

Re: [PATCH] Fix problem with authentification on http repository.

[Johannes Schindelin]

Hi,

On Wed, 9 Jul 2008, Mike Hommey wrote:

> Note that http://user:pass@server/path/ /should/ work (but that would 
> need validation), though not a good idea on command line.

Well, now that the programs using URLs are all builtins, we can actually 
do something about it.  We can edit out the "user[:pass]@" part out of 
argv, which usually means that "ps" will not see it anymore.

Ciao,
Dscho

Re: [PATCH] Fix problem with authentification on http repository.

[Jeff King]

On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:

> > Note that http://user:pass@server/path/ /should/ work (but that would 
> > need validation), though not a good idea on command line.
> 
> Well, now that the programs using URLs are all builtins, we can actually 
> do something about it.  We can edit out the "user[:pass]@" part out of 
> argv, which usually means that "ps" will not see it anymore.

Wouldn't there still be a race condition for publicly broadcasting your
password via ps?

-Peff

Re: [PATCH] Fix problem with authentification on http repository.

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
>
>> > Note that http://user:pass@server/path/ /should/ work (but that would 
>> > need validation), though not a good idea on command line.
>> 
>> Well, now that the programs using URLs are all builtins, we can actually 
>> do something about it.  We can edit out the "user[:pass]@" part out of 
>> argv, which usually means that "ps" will not see it anymore.
>
> Wouldn't there still be a race condition for publicly broadcasting your
> password via ps?

In addition, I think on some Unices overwriting argv[0] does not have the
effect Dscho is aiming for.

I thought we do discourage [:pass] part in our documentation.

Re: [PATCH] Fix problem with authentification on http repository.

[Jeff King]

On Thu, Jul 10, 2008 at 12:32:27AM -0700, Junio C Hamano wrote:

> > Wouldn't there still be a race condition for publicly broadcasting your
> > password via ps?
> 
> In addition, I think on some Unices overwriting argv[0] does not have the
> effect Dscho is aiming for.

I thought so, too, but I couldn't find any mention of it online.  For
curiosity's sake, do you happen to know offhand any platform where this
doesn't work (of course, Dscho is really proposing overwriting argv[i],
but I don't know if that makes a difference to the portability)?

-Peff

Re: [PATCH] Fix problem with authentification on http repository.

[Johannes Schindelin]

Hi,

On Thu, 10 Jul 2008, Jeff King wrote:

> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
> 
> > > Note that http://user:pass@server/path/ /should/ work (but that 
> > > would need validation), though not a good idea on command line.
> > 
> > Well, now that the programs using URLs are all builtins, we can 
> > actually do something about it.  We can edit out the "user[:pass]@" 
> > part out of argv, which usually means that "ps" will not see it 
> > anymore.
> 
> Wouldn't there still be a race condition for publicly broadcasting your 
> password via ps?

Yes.  For a brief amount of time, but yes.

BTW I thought I remembered one program xxx'ing out the password via argv, 
but unfortunately I do not remember which one it was.

Anyway, was just an idea.

Ciao,
Dscho

Re: [PATCH] Fix problem with authentification on http repository.

[Andreas Ericsson]

Johannes Schindelin wrote:
> Hi,
> 
> On Thu, 10 Jul 2008, Jeff King wrote:
> 
>> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
>>
>>>> Note that http://user:pass@server/path/ /should/ work (but that 
>>>> would need validation), though not a good idea on command line.
>>> Well, now that the programs using URLs are all builtins, we can 
>>> actually do something about it.  We can edit out the "user[:pass]@" 
>>> part out of argv, which usually means that "ps" will not see it 
>>> anymore.
>> Wouldn't there still be a race condition for publicly broadcasting your 
>> password via ps?
> 
> Yes.  For a brief amount of time, but yes.
> 
> BTW I thought I remembered one program xxx'ing out the password via argv, 
> but unfortunately I do not remember which one it was.
> 

The mysql client does (or did) it, although perhaps only on systems that
support it.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231