git 2.8.1 not working with socks5h https proxy anymore

git 2.8.1 not working with socks5h https proxy anymore

[Felix Ruess]

Hi all,

I just encountered a problem with the latest git version (2.8.1) that
looks like a regression to me:
When trying to clone a repo via a https socks5 proxy the connection times out:

$ git config --global 'http.proxy=socks5h://127.0.0.1:1080'
$ export GIT_CURL_VERBOSE=1
$ git clone https://foo.de/bar.git
Cloning into 'bar'...
* Couldn't find host foo.de in the .netrc file; using defaults
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Hostname was NOT found in DNS cache
* 80
* 147
* 201
* 194
channel 2: open failed: connect failed: Connection timed out
channel 4: open failed: connect failed: Connection timed out
* Failed to receive SOCKS5 connect request ack.
* Closing connection 0
fatal: unable to access 'https://foo.de/bar.git/': Failed to receive
SOCKS5 connect request ack.

I'm on Ubuntu 14.04 64bit and it works perfectly fine with git 1.9.1
(and was also working with older git 2.x versions, although not sure
any more what the last working version was).

Cheers, Felix

Re: git 2.8.1 not working with socks5h https proxy anymore

[Dennis Kaarsemaker]

On vr, 2016-04-08 at 20:46 +0200, Felix Ruess wrote:
> Hi all,
> 
> I just encountered a problem with the latest git version (2.8.1) that
> looks like a regression to me:
> When trying to clone a repo via a https socks5 proxy the connection
> times out:
> 
> $ git config --global 'http.proxy=socks5h://127.0.0.1:1080'
> $ export GIT_CURL_VERBOSE=1
> $ git clone https://foo.de/bar.git
> Cloning into 'bar'...
> * Couldn't find host foo.de in the .netrc file; using defaults
> * Hostname was NOT found in DNS cache
> *   Trying 127.0.0.1...
> * Hostname was NOT found in DNS cache
> * 80
> * 147
> * 201
> * 194
> channel 2: open failed: connect failed: Connection timed out
> channel 4: open failed: connect failed: Connection timed out
> * Failed to receive SOCKS5 connect request ack.
> * Closing connection 0
> fatal: unable to access 'https://foo.de/bar.git/': Failed to receive
> SOCKS5 connect request ack.
> 
> I'm on Ubuntu 14.04 64bit and it works perfectly fine with git 1.9.1
> (and was also working with older git 2.x versions, although not sure
> any more what the last working version was).

I think that's 6d7afe07f29df75f831a46fb0f657fa37e561779, which
interprets that as a socks5 proxy.

I think this should fix it (but haven't tested it beyond 'it
compiles'):

diff --git a/http.c b/http.c
index 69da445..4304b80 100644
--- a/http.c
+++ b/http.c
@@ -605,7 +605,10 @@ static CURL *get_curl_handle(void)
 	if (curl_http_proxy) {
 		curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
 #if LIBCURL_VERSION_NUM >= 0x071800
-		if (starts_with(curl_http_proxy, "socks5"))
+		if (starts_with(curl_http_proxy, "socks5h"))
+			curl_easy_setopt(result,
+				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
+		else if (starts_with(curl_http_proxy, "socks5"))
 			curl_easy_setopt(result,
 				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
 		else if (starts_with(curl_http_proxy, "socks4a"))

But I have no socks5h proxy to test with. Can you give this patch a spin?
-- 
Dennis Kaarsemaker
www.kaarsemaker.net

Re: git 2.8.1 not working with socks5h https proxy anymore

[Junio C Hamano]

Felix Ruess <felix.ruess@gmail.com> writes:

> I just encountered a problem with the latest git version (2.8.1) that
> looks like a regression to me:
> When trying to clone a repo via a https socks5 proxy the connection times out:
>
> $ git config --global 'http.proxy=socks5h://127.0.0.1:1080'

The first version of Git that has code that explicitly supports
socks proxy is 2.6.4, it seems.  Since then we have always used
CURLPROXY_SOCKS5 for curl_http_proxy that begins with a string
"socks5".

Checking

    https://github.com/curl/curl/blob/master/docs/libcurl/symbols-in-versions

I find that that at curl 7.18.0, curl added a separate
CURLPROXY_SOCKS5_HOSTNAME that can be triggered by "socks5h:" that
is separate from CURLPROXY_SOCKS5, and I am guessing that the
differences in behaviour between these two is what is causing you
trouble.  https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html says

  ..., socks5h:// (the last one to enable socks5 and asking the proxy
  to do the resolving, also known as CURLPROXY_SOCKS5_HOSTNAME type)

and because not using _HOSTNAME variant would likely not ask the
proxy to do the resolving, it explains the stall on your end, if
your box cannot resolve external hostname.

Perhaps the attached patch may help?

I do not know if Pat is still active as a developer, but I am
wondering what the reason was to use starts_with(..., "socks5")
instead of "socks5:", "socks4a:", etc. when identifying the proxy
type.

 http.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/http.c b/http.c
index 69da445..4304b80 100644
--- a/http.c
+++ b/http.c
@@ -605,7 +605,10 @@ static CURL *get_curl_handle(void)
 	if (curl_http_proxy) {
 		curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
 #if LIBCURL_VERSION_NUM >= 0x071800
-		if (starts_with(curl_http_proxy, "socks5"))
+		if (starts_with(curl_http_proxy, "socks5h"))
+			curl_easy_setopt(result,
+				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
+		else if (starts_with(curl_http_proxy, "socks5"))
 			curl_easy_setopt(result,
 				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
 		else if (starts_with(curl_http_proxy, "socks4a"))

Re: git 2.8.1 not working with socks5h https proxy anymore

[Felix Ruess]

Thanks a lot!
Tested the patch and it works as expected :-)

On Fri, Apr 8, 2016 at 9:16 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Felix Ruess <felix.ruess@gmail.com> writes:
>
>> I just encountered a problem with the latest git version (2.8.1) that
>> looks like a regression to me:
>> When trying to clone a repo via a https socks5 proxy the connection times out:
>>
>> $ git config --global 'http.proxy=socks5h://127.0.0.1:1080'
>
> The first version of Git that has code that explicitly supports
> socks proxy is 2.6.4, it seems.  Since then we have always used
> CURLPROXY_SOCKS5 for curl_http_proxy that begins with a string
> "socks5".
>
> Checking
>
>     https://github.com/curl/curl/blob/master/docs/libcurl/symbols-in-versions
>
> I find that that at curl 7.18.0, curl added a separate
> CURLPROXY_SOCKS5_HOSTNAME that can be triggered by "socks5h:" that
> is separate from CURLPROXY_SOCKS5, and I am guessing that the
> differences in behaviour between these two is what is causing you
> trouble.  https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html says
>
>   ..., socks5h:// (the last one to enable socks5 and asking the proxy
>   to do the resolving, also known as CURLPROXY_SOCKS5_HOSTNAME type)
>
> and because not using _HOSTNAME variant would likely not ask the
> proxy to do the resolving, it explains the stall on your end, if
> your box cannot resolve external hostname.
>
> Perhaps the attached patch may help?
>
> I do not know if Pat is still active as a developer, but I am
> wondering what the reason was to use starts_with(..., "socks5")
> instead of "socks5:", "socks4a:", etc. when identifying the proxy
> type.
>
>  http.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/http.c b/http.c
> index 69da445..4304b80 100644
> --- a/http.c
> +++ b/http.c
> @@ -605,7 +605,10 @@ static CURL *get_curl_handle(void)
>         if (curl_http_proxy) {
>                 curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
>  #if LIBCURL_VERSION_NUM >= 0x071800
> -               if (starts_with(curl_http_proxy, "socks5"))
> +               if (starts_with(curl_http_proxy, "socks5h"))
> +                       curl_easy_setopt(result,
> +                               CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
> +               else if (starts_with(curl_http_proxy, "socks5"))
>                         curl_easy_setopt(result,
>                                 CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
>                 else if (starts_with(curl_http_proxy, "socks4a"))

Re: git 2.8.1 not working with socks5h https proxy anymore

[Junio C Hamano]

Felix Ruess <felix.ruess@gmail.com> writes:

> Thanks a lot!
> Tested the patch and it works as expected :-)

Thanks, lets do this, then.

-- >8 --
Subject: [PATCH] http: differentiate socks5:// and socks5h://

Felix Ruess <felix.ruess@gmail.com> noticed that with configuration

    $ git config --global 'http.proxy=socks5h://127.0.0.1:1080'

connections to remote sites time out, waiting for DNS resolution.

The logic to detect various flavours of SOCKS proxy and ask the
libcurl layer to use appropriate one understands the proxy string
that begin with socks5, socks4a, etc., but does not know socks5h,
and we end up using CURLPROXY_SOCKS5.  The correct one to use is
CURLPROXY_SOCKS5_HOSTNAME.

https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html says

  ..., socks5h:// (the last one to enable socks5 and asking the
  proxy to do the resolving, also known as CURLPROXY_SOCKS5_HOSTNAME
  type).

which is consistent with the way the breakage was reported.

Tested-by: Felix Ruess <felix.ruess@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 http.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/http.c b/http.c
index c29ce81..b560c13 100644
--- a/http.c
+++ b/http.c
@@ -466,7 +466,10 @@ static CURL *get_curl_handle(void)
 	if (curl_http_proxy) {
 		curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
 #if LIBCURL_VERSION_NUM >= 0x071800
-		if (starts_with(curl_http_proxy, "socks5"))
+		if (starts_with(curl_http_proxy, "socks5h"))
+			curl_easy_setopt(result,
+				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
+		else if (starts_with(curl_http_proxy, "socks5"))
 			curl_easy_setopt(result,
 				CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
 		else if (starts_with(curl_http_proxy, "socks4a"))
-- 
2.8.1-339-gc925d85