From 200 to 404 to 407.

From 200 to 404 to 407.

[Peter Odéus]

Hi,

Doing a "git clone..." using authenticated proxy (set in $http_proxy):

Upon initial success (HTTP 200) and receiving a single HTTP 404 (not
found), every GET after that renders a HTTP 407 (Proxy authentication
required).

curl -I "url_giving_http_407" comes out just fine as a HTTP 200.

Bug or just me?

Re: From 200 to 404 to 407.

[Avery Pennarun]

On Tue, Oct 27, 2009 at 6:18 AM, Peter Odéus <peter.odeus@gmail.com> wrote:
> Doing a "git clone..." using authenticated proxy (set in $http_proxy):
>
> Upon initial success (HTTP 200) and receiving a single HTTP 404 (not
> found), every GET after that renders a HTTP 407 (Proxy authentication
> required).
>
> curl -I "url_giving_http_407" comes out just fine as a HTTP 200.

The easiest way to track this down is to log it using ethereal, and
compare the http session dumps.  There may be differences in headers
or something that git is providing vs. the command-line curl.

If you don't know how to analyze http dumps, then just log them to a
file using tcpdump's -w option and email them to me, and I'll take a
look.  (I probably won't fix the bug in git, but I can at least tell
you what/if the bug is :))  Please don't send the binary dumpfiles to
the mailing list, though.

Have fun,

Avery

Re: From 200 to 404 to 407.

[Peter]

Don't have root access at work == no wireshark.

Re: From 200 to 404 to 407.

[Peter Odéus]

Peter <peter.odeus <at> gmail.com> writes:

> 
> 
> Don't have root access at work == no wireshark.
> 
> 

I have tried to attach rich console output (thanks to env variable 
GIT_CURL_VERBOSE=1) into a gmane-follow-up-message.

No dice. 

Are there restrictions on what characters are permitted?  

Re: From 200 to 404 to 407.

[Avery Pennarun]

On Wed, Oct 28, 2009 at 4:38 AM, Peter Odéus <peter.odeus@gmail.com> wrote:
> * Re-using existing connection! (#0) with host proxyserver.acme.com
> * Connected to proxyserver.acme.com (192.71.145.9) port 8080
> > GET http://gitrepo.outside.com/git/gitrepo.git/objects/6b/132a9e81161e58812902d7f735a38bf5ee1583 HTTP/1.1
> Proxy-Authorization: Basic cmQva3F3Zzc2MjptYW1tYW1pYQ==
> User-Agent: git/1.6.5.2
> Host: gitrepo.outside.com
> Accept: */*
>
> * The requested URL returned error: 404

So this git object didn't exist, apparently.  Can you confirm that the
object shouldn't be there?  (on the server: git cat-file -p
6b132a9e81161e58812902d7f735a38bf5ee1583)  Does git-fsck report
anything weird on the server repository?

> * Closing connection #0
> * Couldn't find host gitrepo.outside.com in the .netrc file, using defaults
> * About to connect() to proxyserver.acme.com port 8080
> *   Trying 192.71.145.9... * connected
> * Connected to proxyserver.acme.com (192.71.145.9) port 8080
> > GET http://gitrepo.outside.com/git/gitrepo.git/objects/info/http-alternates HTTP/1.1
> User-Agent: git/1.6.5.2
> Host: gitrepo.outside.com
> Accept: */*
> Pragma: no-cache
>
> < HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires
> authorization to fulfill the request. Access to the Web Proxy service
> is denied.  )

This error seems to happen because the Proxy-Authorization line was
not included in this request like it was included in prior ones.
Probably the authorization key was forgotten when the first connection
closed.  If there hadn't been a 404, the connection wouldn't have
closed and this wouldn't have happened, which is presumably why you
haven't seen this problem before.

This is where my expertise ends, since I've never messed with either
libcurl or git's usage of it.  I couldn't tell you if this is a
libcurl bug or a git bug.  (Proxies are relatively rare nowadays, so
this code path is likely to be rarely tested.)

Hopefully someone else on the list can assist.

** WARNING: the username/password sent in the Proxy-Authorization line
is not encrypted and you've posted a trace of it to a public mailing
list.  You need to change your password immediately. **

Good luck,

Avery

Re: From 200 to 404 to 407.

[Peter Odéus]

Avery Pennarun <apenwarr <at> gmail.com> writes:

> 
> On Wed, Oct 28, 2009 at 4:38 AM, Peter Odéus <peter.odeus <at> gmail.com> 
wrote:
> > * Re-using existing connection! (#0) with host proxyserver.acme.com
> > * Connected to proxyserver.acme.com (192.71.145.9) port 8080
> > > GET
> 
http://gitrepo.outside.com/git/gitrepo.git/objects/6b/132a9e81161e58812902d7f735
a38bf5ee1583 HTTP/1.1
> > Proxy-Authorization: Basic cmQva3F3Zzc2MjptYW1tYW1pYQ==
> > User-Agent: git/1.6.5.2
> > Host: gitrepo.outside.com
> > Accept: */*
> >
> > * The requested URL returned error: 404
> 
> So this git object didn't exist, apparently.  Can you confirm that the
> object shouldn't be there?  (on the server: git cat-file -p
> 6b132a9e81161e58812902d7f735a38bf5ee1583)  Does git-fsck report
> anything weird on the server repository?
> 
> > * Closing connection #0
> > * Couldn't find host gitrepo.outside.com in the .netrc file, using defaults
> > * About to connect() to proxyserver.acme.com port 8080
> > *   Trying 192.71.145.9... * connected
> > * Connected to proxyserver.acme.com (192.71.145.9) port 8080
> > > GET http://gitrepo.outside.com/git/gitrepo.git/objects/info/http-
alternates HTTP/1.1
> > User-Agent: git/1.6.5.2
> > Host: gitrepo.outside.com
> > Accept: */*
> > Pragma: no-cache
> >
> > < HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires
> > authorization to fulfill the request. Access to the Web Proxy service
> > is denied.  )
> 
> This error seems to happen because the Proxy-Authorization line was
> not included in this request like it was included in prior ones.
> Probably the authorization key was forgotten when the first connection
> closed.  If there hadn't been a 404, the connection wouldn't have
> closed and this wouldn't have happened, which is presumably why you
> haven't seen this problem before.
> 
> This is where my expertise ends, since I've never messed with either
> libcurl or git's usage of it.  I couldn't tell you if this is a
> libcurl bug or a git bug.  (Proxies are relatively rare nowadays, so
> this code path is likely to be rarely tested.)
> 
> Hopefully someone else on the list can assist.
> 
> ** WARNING: the username/password sent in the Proxy-Authorization line
> is not encrypted and you've posted a trace of it to a public mailing
> list.  You need to change your password immediately. **
> 
> Good luck,
> 
> Avery
> 

If you wouldn't have posted the base64-encoded password it would have never hit 
the gmane list ;-) 

Don't worry about it, because I tried to submit the console output myself all 
day. But to no avail.

Is it enough to replace greater-than-signs to something else, btw?

Anyway. Behind these corporate walls, the proxy is a reality to deal with, so 
hopefully someone else can bring us closer to a solutiuon.

Thanks for your input.

:D

/Peter 

Re: From 200 to 404 to 407.

[Avery Pennarun]

On Wed, Oct 28, 2009 at 12:10 PM, Peter Odéus <peter.odeus@gmail.com> wrote:
> Avery Pennarun <apenwarr <at> gmail.com> writes:
> If you wouldn't have posted the base64-encoded password it would have never hit
> the gmane list ;-)

Sorry about that.  The To: line indicated that the message had been
sent to the list, so I didn't realize that your copy was rejected.  I
have no idea what made gmane reject it for you (or whether my own copy
was rejected, for that matter :)).

Avery

Re: From 200 to 404 to 407.

[Avery Pennarun]

On Wed, Oct 28, 2009 at 12:10 PM, Peter Odéus <peter.odeus@gmail.com> wrote:
> Anyway. Behind these corporate walls, the proxy is a reality to deal with, so
> hopefully someone else can bring us closer to a solutiuon.

I was just looking at the source and it seems all the curl-related
stuff is isolated in git/http.c, and is pretty easy to understand.
Since I have no way to reproduce your problem, I won't attempt to look
into it myself.  But if you look for "proxy" (and especially
CURLOPT_PROXY) in that file, you might find something.

Specifically, I would suggest resetting the CURLOPT_PROXY setting
before each URL request (eg. in new_http_object_request).

Hope this helps.

Avery