[PATCH] Allow HTTP proxy to be overridden in config

[PATCH] Allow HTTP proxy to be overridden in config

[Sam Vilain]

The http_proxy / HTTPS_PROXY variables used by curl to control
proxying may not be suitable for git.  Allow the user to override them
in the configuration file.
---
  In particular, privoxy will block directories called /ad/ ... d'oh!

 Documentation/config.txt |    4 ++++
 http.c                   |   11 +++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7ee97df..859a7f3 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -515,6 +515,10 @@ specified as 'gitcvs.<access_method>.<varname>' (where 'access_method'
 is one of "ext" and "pserver") to make them apply only for the given
 access method.
 
+http.proxy::
+	Override the HTTP proxy, normally configured using the 'http_proxy'
+	environment variable (see gitlink:curl[1]).
+
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
 	over HTTPS. Can be overridden by the 'GIT_SSL_NO_VERIFY' environment
diff --git a/http.c b/http.c
index c6fb8ac..8f60d89 100644
--- a/http.c
+++ b/http.c
@@ -24,6 +24,7 @@ char *ssl_cainfo = NULL;
 long curl_low_speed_limit = -1;
 long curl_low_speed_time = -1;
 int curl_ftp_no_epsv = 0;
+char *curl_http_proxy = NULL;
 
 struct curl_slist *pragma_header;
 
@@ -160,6 +161,13 @@ static int http_options(const char *var, const char *value)
 		curl_ftp_no_epsv = git_config_bool(var, value);
 		return 0;
 	}
+	if (!strcmp("http.proxy", var)) {
+		if (curl_http_proxy == NULL) {
+			curl_http_proxy = xmalloc(strlen(value)+1);
+			strcpy(curl_http_proxy, value);
+		}
+		return 0;
+	}
 
 	/* Fall back on the default ones */
 	return git_default_config(var, value);
@@ -205,6 +213,9 @@ static CURL* get_curl_handle(void)
 	if (curl_ftp_no_epsv)
 		curl_easy_setopt(result, CURLOPT_FTP_USE_EPSV, 0);
 
+	if (curl_http_proxy)
+		curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
+
 	return result;
 }
 
-- 
1.5.3.5

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Junio C Hamano]

Sam Vilain <sam.vilain@catalyst.net.nz> writes:

> The http_proxy / HTTPS_PROXY variables used by curl to control
> proxying may not be suitable for git.  Allow the user to override them
> in the configuration file.
> ---
>   In particular, privoxy will block directories called /ad/ ... d'oh!

> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 7ee97df..859a7f3 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -515,6 +515,10 @@ specified as 'gitcvs.<access_method>.<varname>' (where 'access_method'
>  is one of "ext" and "pserver") to make them apply only for the given
>  access method.
>  
> +http.proxy::
> +	Override the HTTP proxy, normally configured using the 'http_proxy'
> +	environment variable (see gitlink:curl[1]).
> +

This may work around the issue you cited, but it makes me wonder
if it is a road to insanity.  Does the curl library expect that
(1) each and every HTTP talking application that uses the
library offer this kind of knob for its users to tweak, and (2)
users set the knob for each and every one of such application?

I would say if privoxy cannot be tweaked to allow /ad/ in chosen
context (e.g. /ad/ in general is rejected but /objects/ad/ is
Ok), that is what needs to be fixed.

Or it would be the use of such a broken proxy by the user.  That
can be fixed and much easily.

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Sam Vilain]

Junio C Hamano wrote:
>> The http_proxy / HTTPS_PROXY variables used by curl to control
>> proxying may not be suitable for git.  Allow the user to override them
>> in the configuration file.
>> ---
>>   In particular, privoxy will block directories called /ad/ ... d'oh!
>> +++ b/Documentation/config.txt
>> +http.proxy::
>> +	Override the HTTP proxy, normally configured using the 'http_proxy'
>> +	environment variable (see gitlink:curl[1]).
> 
> This may work around the issue you cited, but it makes me wonder
> if it is a road to insanity.  Does the curl library expect that
> (1) each and every HTTP talking application that uses the
> library offer this kind of knob for its users to tweak, and (2)
> users set the knob for each and every one of such application?

This is true.  However I still think that it is a useful feature for
many users, with few side effects.  If nothing else the bit on the man
page will prompt them to think, "oh, I should set that in the environment".

> I would say if privoxy cannot be tweaked to allow /ad/ in chosen
> context (e.g. /ad/ in general is rejected but /objects/ad/ is
> Ok), that is what needs to be fixed.
> Or it would be the use of such a broken proxy by the user.  That
> can be fixed and much easily.

Yes - but consider the dilemma of the user.  They've apt-get installed
this privoxy thing and figured out how to set their applications to use
it.  Now, it doesn't work and they think it is the proxy in the way, and
they've no idea that they might be able to reconfigure it.  This way
they can tell git to bypass it.

I don't know, I see your point and pretty much agree with it.  It just
seems like something that might come in handy (as well as be another
vector for something you need to check when you get HTTP fetch issues -
so maybe a command-line option would be better).

Actually something that would really have helped is more documentation
of the various GIT_CURL_* environment variables available for debugging.
-- 
Sam Vilain, Chief Yak Shaver, Catalyst IT (NZ) Ltd.
phone: +64 4 499 2267        PGP ID: 0x66B25843

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Junio C Hamano]

Sam Vilain <sam.vilain@catalyst.net.nz> writes:

> Junio C Hamano wrote:
>>> The http_proxy / HTTPS_PROXY variables used by curl to control
>>> proxying may not be suitable for git.  Allow the user to override them
>>> in the configuration file.
>>> ---
>>>   In particular, privoxy will block directories called /ad/ ... d'oh!
>>> +++ b/Documentation/config.txt
>>> +http.proxy::
>>> +	Override the HTTP proxy, normally configured using the 'http_proxy'
>>> +	environment variable (see gitlink:curl[1]).
>> 
>> This may work around the issue you cited, but it makes me wonder
>> if it is a road to insanity.  Does the curl library expect that
>> (1) each and every HTTP talking application that uses the
>> library offer this kind of knob for its users to tweak, and (2)
>> users set the knob for each and every one of such application?
>
> This is true.  However I still think that it is a useful feature for
> many users, with few side effects.  If nothing else the bit on the man
> page will prompt them to think, "oh, I should set that in the environment".
>
>> I would say if privoxy cannot be tweaked to allow /ad/ in chosen
>> context (e.g. /ad/ in general is rejected but /objects/ad/ is
>> Ok), that is what needs to be fixed.
>> Or it would be the use of such a broken proxy by the user.  That
>> can be fixed and much easily.
>
> Yes - but consider the dilemma of the user.  They've apt-get installed
> this privoxy thing and figured out how to set their applications to use
> it.  Now, it doesn't work and they think it is the proxy in the way, and
> they've no idea that they might be able to reconfigure it.  This way
> they can tell git to bypass it.
>
> I don't know, I see your point and pretty much agree with it.  It just
> seems like something that might come in handy (as well as be another
> vector for something you need to check when you get HTTP fetch issues -
> so maybe a command-line option would be better).
>
> Actually something that would really have helped is more documentation
> of the various GIT_CURL_* environment variables available for debugging.

Having thought about this a bit more, I changed my mind.  From the
beginning, I did not mind adding a dozen or so lines if the change helps
the user.  I was not _fundamentally_ opposed to your patch.

However.

There may be other environment variables that the users may want to set
differently when running git from the settings they use in their
interactive sessions.  We have precedentsto support such situations in
the form of git-specific environment variables (e.g. GIT_EDITOR and
GIT_PAGER).  These might have been a road to insanity already, and what
we should have done may be to introduce multivalued core.environment
variables in $HOME/.gitconfig, like so:

	[core.environment]
        	EDITOR = vim
		PAGER = less

without introducing GIT_foo environment variables.  We can then change
the git potty[*1*] start-up sequence to add them to the enviornment.  But
this is a bit like water under the bridge now.

While that approach could work for environment variables that apply
globally to any and all git sessions the user may run, I suspect it
would not work well for things like http_proxy.  If we really wanted to
help users, I think it should be tied to which remote we are going to,
just like we made git-send-email to use different mailpath depending on
which project you are communicating with.

In that sense, I think http.proxy configuration variable does not go far
enough, even though it might be a step in the right direction.  Perhaps
use your configuration variable http.proxy (or "core.environment") to
define the global default, with remote.$name.httpproxy to override it?


[Footnote]

*1* git(7) calls "git" itself as "git potty".  Is this word still used?
I also notice that Andreas's name is misspelled there.

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Andreas Ericsson]

Junio C Hamano wrote:
> 
> [Footnote]
> 
> *1* git(7) calls "git" itself as "git potty".  Is this word still used?
> I also notice that Andreas's name is misspelled there.


AFAIR I sort of invented that to disambiguate git-the-program from
git-the-project while working on writing the program in C. I haven't
heard/seen it a lot since then, except on a few occasions where it
helped serve that same purpose.

I also misspelled my own name in that patch. Perhaps it's time I
corrected that.

---%<---%<---%<---
From: Andreas Ericsson <ae@op5.se>
Subject: [PATCH] Correct typo in Documentation/git.txt

Signed-off-by: Andreas Ericsson <ae@op5.se>
---

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 9ff4659..6c98ab2 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -536,7 +536,7 @@ Authors
 -------
 * git's founding father is Linus Torvalds <torvalds@osdl.org>.
 * The current git nurse is Junio C Hamano <gitster@pobox.com>.
-* The git potty was written by Andres Ericsson <ae@op5.se>.
+* The git potty was written by Andreas Ericsson <ae@op5.se>.
 * General upbringing is handled by the git-list <git@vger.kernel.org>.
 
 Documentation
---%<---%<---%<---
-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Johannes Schindelin]

Hi,

On Mon, 3 Dec 2007, Andreas Ericsson wrote:

> Junio C Hamano wrote:
> > 
> > [Footnote]
> > 
> > *1* git(7) calls "git" itself as "git potty".  Is this word still used?
> > I also notice that Andreas's name is misspelled there.
> 
> 
> AFAIR I sort of invented that to disambiguate git-the-program from 
> git-the-project while working on writing the program in C. I haven't 
> heard/seen it a lot since then, except on a few occasions where it 
> helped serve that same purpose.

I always spell it "git wrapper".

Ciao,
Dscho

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Sam Vilain]

Junio C Hamano wrote:
> In that sense, I think http.proxy configuration variable does not go far
> enough, even though it might be a step in the right direction.  Perhaps
> use your configuration variable http.proxy (or "core.environment") to
> define the global default, with remote.$name.httpproxy to override it?

Sure, why not.

Subject: [PATCH] Add remote.<name>.proxy

As well as allowing a default proxy option, allow it to be set
per-remote.

Signed-off-by: Sam Vilain <sam.vilain@catalyst.net.nz>
---
 Documentation/config.txt |    8 +++++++-
 remote.c                 |    2 ++
 remote.h                 |    5 +++++
 transport.c              |    3 +++
 4 files changed, 17 insertions(+), 1 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7e6c02a..831df58 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -545,7 +545,8 @@ access method.
 
 http.proxy::
 	Override the HTTP proxy, normally configured using the 'http_proxy'
-	environment variable (see gitlink:curl[1]).
+	environment variable (see gitlink:curl[1]).  This can be overridden
+	on a per-remote basis; see remote.<name>.proxy
 
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
@@ -695,6 +696,11 @@ remote.<name>.url::
 	The URL of a remote repository.  See gitlink:git-fetch[1] or
 	gitlink:git-push[1].
 
+remote.<name>.proxy::
+	For remotes that require curl (http, https and ftp), the URL to
+	the proxy to use for that remote.  Set to the empty string to
+	disable proxying for that remote.
+
 remote.<name>.fetch::
 	The default set of "refspec" for gitlink:git-fetch[1]. See
 	gitlink:git-fetch[1].
diff --git a/remote.c b/remote.c
index bec2ba1..94df314 100644
--- a/remote.c
+++ b/remote.c
@@ -278,6 +278,8 @@ static int handle_config(const char *key, const char *value)
 	} else if (!strcmp(subkey, ".tagopt")) {
 		if (!strcmp(value, "--no-tags"))
 			remote->fetch_tags = -1;
+	} else if (!strcmp(subkey, ".proxy")) {
+		remote->fetch_refspec = xstrdup(value);
 	}
 	return 0;
 }
diff --git a/remote.h b/remote.h
index 878b4ec..2268558 100644
--- a/remote.h
+++ b/remote.h
@@ -25,6 +25,11 @@ struct remote {
 
 	const char *receivepack;
 	const char *uploadpack;
+
+	/*
+	 * for curl remotes only
+	 */
+	const char *proxy;
 };
 
 struct remote *remote_get(const char *name);
diff --git a/transport.c b/transport.c
index 43b9e7c..c62ec88 100644
--- a/transport.c
+++ b/transport.c
@@ -463,6 +463,9 @@ static struct ref *get_refs_via_curl(const struct transport *transport)
 	curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
 	curl_easy_setopt(slot->curl, CURLOPT_URL, refs_url);
 	curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
+	if (transport->remote->proxy) {
+		curl_easy_setopt(slot->curl, CURLOPT_PROXY, transport->remote->proxy);
+	}
 	if (start_active_slot(slot)) {
 		run_active_slot(slot);
 		if (results.curl_result != CURLE_OK) {
-- 
1.5.3.5

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Junio C Hamano]

Sam Vilain <sam.vilain@catalyst.net.nz> writes:

> Subject: [PATCH] Add remote.<name>.proxy
>
> As well as allowing a default proxy option, allow it to be set
> per-remote.

Thanks.

I was very tempted to rename them to remote.*.httpproxy, but ended up
keeping the name of the variable.  We might want to allow tunnelling the
git native transport over the named proxy when remote.*.url is native
transport.

Re: [PATCH] Allow HTTP proxy to be overridden in config

[Sam Vilain]

Junio C Hamano wrote:
>> Subject: [PATCH] Add remote.<name>.proxy
>> As well as allowing a default proxy option, allow it to be set
>> per-remote.
> 
> Thanks.
> 
> I was very tempted to rename them to remote.*.httpproxy, but ended up
> keeping the name of the variable.  We might want to allow tunnelling the
> git native transport over the named proxy when remote.*.url is native
> transport.

Also, it controls proxy for all of the curl-based git transports, not
just http.

Sam.