https_proxy and libcurl

https_proxy and libcurl

[Phil Hord]

I have been unable to clone via http proxy because of a wrongly
configured proxy setup in my lab.

I had this env:

    http_proxy=http://proxy.myco.com
    https_proxy=https://proxy.myco.com

The problem is that libcurl ignores the protocol part of the proxy
url, and it defaults to port 1080. wget honors the protocol specifier,
but it defaults to port 80 if none is given.

So, I thought my env was equivalent to this:

    http_proxy=proxy.myco.com:80
    https_proxy=proxy.myco.com:443

So it is with wget.  But with curl, it is equivalent to this:

    http_proxy=proxy.myco.com:1080
    https_proxy=proxy.myco.com:1080

Git clone gave a confusing (but correct) error message:

    $ export https_proxy=https://proxy.myco.com
    $ git clone https://github.com/git/git
    Cloning into 'git'...
    error: The requested URL returned error: 500 while accessing
https://github.com/git/git/info/refs?service=git-upload-pack
    fatal: HTTP request failed


If I didn't have https_proxy set at all, I got a long timeout as it
tried to connect directly and ran into our firewall.

    $ unset https_proxy
    $ git clone https://github.com/git/git
    Cloning into 'git'...
    error: Failed connect to github.com:443; Connection timed out
while accessing
https://github.com/git/git/info/refs?service=git-upload-pack
    fatal: HTTP request failed

This also did not help, of course:
    'git config http.proxy https://proxy.myco.com'

But this did:
    'git config http.proxy https://proxy.myco.com:443'


The fix was to specify the port explicitly, like this:

    http_proxy=proxy.myco.com:80
    https_proxy=proxy.myco.com:443

Phil

[1] An added wrinkle is that there is a proxy server listening on
1080, but it does not support encrypted connections.  A listener on
443 does handle https requests correctly.

Re: https_proxy and libcurl

[Junio C Hamano]

Phil Hord <phil.hord@gmail.com> writes:

> I have been unable to clone via http proxy because of a wrongly
> configured proxy setup in my lab.
>
> I had this env:
>
>     http_proxy=http://proxy.myco.com
>     https_proxy=https://proxy.myco.com
>
> The problem is that libcurl ignores the protocol part of the proxy
> url, and it defaults to port 1080. wget honors the protocol specifier,
> but it defaults to port 80 if none is given.

IIRC, the historical norm is to set these to <host>:<port>.

So many people mistakenly write them with <method>:// that some
tools over time learned to strip and ignore that prefix, though.

> The fix was to specify the port explicitly, like this:
>
>     http_proxy=proxy.myco.com:80
>     https_proxy=proxy.myco.com:443

Yeah, that is the correct syntax to use.  Is there anything you want
Git to do to be more helpful?

Re: https_proxy and libcurl

[Daniel Stenberg]

On Fri, 22 Feb 2013, Junio C Hamano wrote:

>>     http_proxy=http://proxy.myco.com
>>     https_proxy=https://proxy.myco.com
>>
>> The problem is that libcurl ignores the protocol part of the proxy
>> url, and it defaults to port 1080. wget honors the protocol specifier,
>> but it defaults to port 80 if none is given.
>
> IIRC, the historical norm is to set these to <host>:<port>.
>
> So many people mistakenly write them with <method>:// that some tools over 
> time learned to strip and ignore that prefix, though.

You're right, but also what exactly is the https:// _to_ the proxy supposed to 
mean? The standard procedure to connect to a proxy when communicating with 
either HTTP or HTTPS is using plain HTTP.

If you would want port 443 for a HTTPS connection to a proxy, you'd use:

   https_proxy=http://proxy.myco.com:443

Or without the ignore protocol prefix:

   https_proxy=proxy.myco.com:443

-- 

  / daniel.haxx.se

Re: https_proxy and libcurl

[Phil Hord]

On Fri, Feb 22, 2013 at 4:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Phil Hord <phil.hord@gmail.com> writes:
>
>> I have been unable to clone via http proxy because of a wrongly
>> configured proxy setup in my lab.
>>
>> I had this env:
>>
>>     http_proxy=http://proxy.myco.com
>>     https_proxy=https://proxy.myco.com
>>
>> The problem is that libcurl ignores the protocol part of the proxy
>> url, and it defaults to port 1080. wget honors the protocol specifier,
>> but it defaults to port 80 if none is given.
>
> IIRC, the historical norm is to set these to <host>:<port>.
>
> So many people mistakenly write them with <method>:// that some
> tools over time learned to strip and ignore that prefix, though.

On the server in question, I use wget to retrieve packages in another
script.  When our firewall was tightened in the past, this script
broke.  I fixed it by setting the https_proxy as I listed above.  This
satisfied wget and I got on with life.

I do not need git connectivity to remotes from this server very often.
 But when I tried, it failed with an error message that did not
implicate the proxy server.  But it was the proxy server that returned
the error 500.

  error: The requested URL returned error: 500 while accessing
       https://github.com/git/git/info/refs?service=git-upload-pack

>
>> The fix was to specify the port explicitly, like this:
>>
>>     http_proxy=proxy.myco.com:80
>>     https_proxy=proxy.myco.com:443
>
> Yeah, that is the correct syntax to use.  Is there anything you want
> Git to do to be more helpful?

Maybe Git can tell me more about proxy failures.  Or maybe Git doesn't
know because curl hides the knowledge.  Something like this:

  error: The proxy server at proxy.myco.com:1080
       returned error: 500 while accessing
       https://github.com/git/git/info/refs?service=git-upload-pack

Maybe Git can warn me that the protocol prefix (if provided) is going
to be ignored or that I have failed to provide a port.  Either of
these would be "above and beyond" the norm. But since curl is internal
to Git and its errors are digested by Git, it is less obvious where to
begin looking to solve such a problem; git doesn't provide the
plethora of connection-debugging output options that curl itself does.

Fwiw, the proxy info originated with our IT department who mostly had
to answer the question for Windows users and servers.  Windows appears
to default to port 8080 when no port is specified.  Port 8080 would
also have worked, but curl defaults to 1080.  It seems that the curl
default of 1080 is the only one that caused me trouble.

My basest hope is that someone suffering similar troubles will find
this thread in a search for "git proxy error".

Phil