Fwd: [OT] Re: Git via a proxy server?

Fwd: [OT] Re: Git via a proxy server?

[Sam Song]

Hello,

Petr Vandrovec <petr@vmware.com> wrote:
> Date:	Mon, 27 Feb 2006 00:35:00 +0100
> From:	Petr Vandrovec <petr@vmware.com>
> To:	Sergey Vlasov <vsu@altlinux.ru>
> CC:	"Salyzyn, Mark" <mark_salyzyn@adaptec.com>,
> 	Kernel Mailing List 
> <linux-kernel@vger.kernel.org>
> > [snip]
> > I have successfully used transconnect
> > (http://sourceforge.net/projects/transconnect) for
> > tunnelling git
> > protocol through a HTTP proxy (squid in my case)
> > supporting the CONNECT method.
> >
> > [snip] 
> > Note: most HTTP proxy servers allow CONNECT method
> > to a very limited range of ports, and 
> > administrators will need to enable the git port
> > (9418) explicitly.
> > 
> I know I'm coming kinda late, but I'm using:
> 
> export GIT_PROXY_COMMAND=/usr/local/bin/proxy-cmd.sh
> 
> and proxy-cmd.sh is just single-line command glued
> from what I found available in /bin:
> 
> #! /bin/bash
> 
> (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket
> 
> proxy.ourcompany.com 3128 | (read a; read a; cat )
> 
> Replace socket's arguments 'proxy.ourcompany.com
> 3128' with your http proxy.  Fortunately our proxy
> does not see anything wrong with git's port.
> 		Best regards,
> 			Petr Vandrovec

With above usage on GIT_PROXY_COMMAND, I still have
problem on connection with remote git repository.

I also tried setting http_proxy directly but the same
result. It's first usage of git in our network. Well, 
need I enable the git port 9418 at proxy server? Or 
did I miss sth or what?

I use git-2006-05-14.tar.gz package on FC3. 

Method I : Use GIT_PROXY_COMMAND

[root@sam u-boot]# git clone \
git://www.denx.de/git/u-boot.git u-boot-denx.git

fatal: exec failed
fetch-pack from 'git://www.denx.de/git/u-boot.git' 
failed.

[root@sam u-boot]# git clone \
http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6

Cannot get remote repository information.
Perhaps git-update-server-info needs to be run there?
[root@sam u-boot]#

/usr/local/bin/proxy-cmd.sh 

#! /bin/bash

(echo "CONNECT $1:$2 HTTP/1.0";echo;cat) | socket
<um> <pwd> 192.168.40.99 80 | (read a;read a;cat)

Method II : Use http_proxy directly

[root@sam u-boot]# export \
http_proxy="http://<username>:<pwd>@192.168.40.99:80"
[root@sam u-boot]# git clone \ 
http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6

Cannot get remote repository information.
Perhaps git-update-server-info needs to be run there?

[root@sam u-boot]# git clone \
git://www.denx.de/git/u-boot.git u-boot-denx.git
fatal: unable to connect a socket (Connection timed 
out)
fetch-pack from 'git://www.denx.de/git/u-boot.git' 
failed.
[root@sam u-boot]#

Thanks in advance,

Sam

P.S. I forward this thread from LKML for better 
discussion. Hope Petr wouldn't mind.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam
protection around 
http://mail.yahoo.com 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Fwd: [OT] Re: Git via a proxy server?

[Petr Vandrovec]

Sam Song wrote:
> Hello,
> 
> Petr Vandrovec <petr@vmware.com> wrote:
> 
>>Date:	Mon, 27 Feb 2006 00:35:00 +0100
>>From:	Petr Vandrovec <petr@vmware.com>
>>To:	Sergey Vlasov <vsu@altlinux.ru>
>>CC:	"Salyzyn, Mark" <mark_salyzyn@adaptec.com>,
>>	Kernel Mailing List 
>><linux-kernel@vger.kernel.org>
>>
>>>[snip]
>>>I have successfully used transconnect
>>>(http://sourceforge.net/projects/transconnect) for
>>>tunnelling git
>>>protocol through a HTTP proxy (squid in my case)
>>>supporting the CONNECT method.
>>>
>>>[snip] 
>>>Note: most HTTP proxy servers allow CONNECT method
>>>to a very limited range of ports, and 
>>>administrators will need to enable the git port
>>>(9418) explicitly.
>>>
>>
>>I know I'm coming kinda late, but I'm using:
>>
>>export GIT_PROXY_COMMAND=/usr/local/bin/proxy-cmd.sh
>>
>>and proxy-cmd.sh is just single-line command glued
>>from what I found available in /bin:
>>
>>#! /bin/bash
>>
>>(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket
>>
>>proxy.ourcompany.com 3128 | (read a; read a; cat )
>>
>>Replace socket's arguments 'proxy.ourcompany.com
>>3128' with your http proxy.  Fortunately our proxy
>>does not see anything wrong with git's port.
>>		Best regards,
>>			Petr Vandrovec
> 
> 
> With above usage on GIT_PROXY_COMMAND, I still have
> problem on connection with remote git repository.
> 
> I also tried setting http_proxy directly but the same
> result. It's first usage of git in our network. Well, 
> need I enable the git port 9418 at proxy server? Or 
> did I miss sth or what?

Yes.  Try running 'socket 192.168.40.99 80', and type
CONNECT 204.152.191.37:9418 HTTP/1.0
Proxy-Authorization: Basic <yoursecret,f.e.wget -d should reveal this to you>
<empty line>

You should get back user readable diagnostics what went wrong.  Yes, your admin 
must allow CONNECT method for target port 9418.

> I use git-2006-05-14.tar.gz package on FC3. 
> 
> Method I : Use GIT_PROXY_COMMAND
> 
> [root@sam u-boot]# git clone \
> git://www.denx.de/git/u-boot.git u-boot-denx.git
> 
> fatal: exec failed
> fetch-pack from 'git://www.denx.de/git/u-boot.git' 
> failed.

Is $GIT_PROXY_COMMAND executable? (just in case...)  Try 'strace -f git clone 
...', it should tell you what's going on.

> #! /bin/bash
> 
> (echo "CONNECT $1:$2 HTTP/1.0";echo;cat) | socket
> <um> <pwd> 192.168.40.99 80 | (read a;read a;cat)

What is '<um>' and '<pwd>' ?  socket just connects somewhere, so if you are 
supposed to use <username>:<pwd> to connect to your proxy, you must add 
Proxy-Authorization header yourself:

(echo "CONNECT $1:$2 HTTP/1.0";
  echo "Proxy-Authorization: Basic <base64encoded um:pwd>";
  echo;
  cat ) | socket 192.168.40.99 80 | (read a; read a; cat)

Best to test this is to start 'socket 192.168.40.99 80' from command line and 
then type these two lines above, plus one empty line.  You should get back '200 
OK', empty line, and then you can start communicating using git protocol - if 
you can do that...

> Method II : Use http_proxy directly
> 
> [root@sam u-boot]# export \
> http_proxy="http://<username>:<pwd>@192.168.40.99:80"
> [root@sam u-boot]# git clone \ 
> http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6

As far as I can tell, http_proxy is ignored (Debian's git 1.3.2-1/cogito 0.17.2-1).
								Petr

Re: Fwd: [OT] Re: Git via a proxy server?

[Sam Song]

Petr Vandrovec <petr@vmware.com> wrote:
> Best to test this is to start 'socket 192.168.40.99
> 80' from command line and 
> then type these two lines above, plus one empty
> line.  You should get back '200 
> OK', empty line, and then you can start
> communicating using git protocol - if 
> you can do that...

I cannot run "socket" and "CONNECT" on Fedora Core 3.
It simply told me that no such command. How could I 
do this task in my case?

> As far as I can tell, http_proxy is ignored
> (Debian's git 1.3.2-1/cogito 0.17.2-1).

Seems you tried proxy-cmd.sh on Debian. Which 
distribution did you use? 

Thanks a lot,

Sam


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Fwd: [OT] Re: Git via a proxy server?

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

On Tue, 2006-05-16 20:56:39 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
> Petr Vandrovec <petr@vmware.com> wrote:
> > Best to test this is to start 'socket 192.168.40.99
> > 80' from command line and 
> > then type these two lines above, plus one empty
> > line.  You should get back '200 
> > OK', empty line, and then you can start
> > communicating using git protocol - if 
> > you can do that...
> 
> I cannot run "socket" and "CONNECT" on Fedora Core 3.
> It simply told me that no such command. How could I 
> do this task in my case?

Well, install some package to have `socket' available? Debian calls
the packet `socket', too, so I guess Fedora may have something
similar.

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Fwd: [OT] Re: Git via a proxy server?

[Petr Vandrovec]

Jan-Benedict Glaw wrote:
> On Tue, 2006-05-16 20:56:39 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
> 
>>Petr Vandrovec <petr@vmware.com> wrote:
>>
>>>Best to test this is to start 'socket 192.168.40.99
>>>80' from command line and 
>>>then type these two lines above, plus one empty
>>>line.  You should get back '200 
>>>OK', empty line, and then you can start
>>>communicating using git protocol - if 
>>>you can do that...
>>
>>I cannot run "socket" and "CONNECT" on Fedora Core 3.
>>It simply told me that no such command. How could I 
>>do this task in my case?
> 
> 
> Well, install some package to have `socket' available? Debian calls
> the packet `socket', too, so I guess Fedora may have something
> similar.

Surprisingly they do not...  You should be able to replace 'socket' with 
'netcat' - and I believe that netcat/nc package is available for Fedora.  For 
this purpose they have same command line & behavior.
							Petr

Re: Fwd: [OT] Re: Git via a proxy server?

[Sam Song]

Petr Vandrovec <petr@vmware.com> wrote:
> Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> > Well, install some package to have `socket'
> > available? Debian calls
> > the packet `socket', too, so I guess Fedora may
> > have something similar.
> 
> Surprisingly they do not...  You should be able to
> replace 'socket' with 
> 'netcat' - and I believe that netcat/nc package is
> available for Fedora.  For 
> this purpose they have same command line & behavior.

Ummm, I am trying on that. nc is avaiable for Fedora.
But what could be the replacement for CONNECT in
Fedora? :-)

Thanks for your kind support,

Sam



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Fwd: [OT] Re: Git via a proxy server?

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2797 bytes --]

On Wed, 2006-05-17 20:44:28 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
> Petr Vandrovec <petr@vmware.com> wrote:
> > Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> > > Well, install some package to have `socket'
> > > available? Debian calls
> > > the packet `socket', too, so I guess Fedora may
> > > have something similar.
> > 
> > Surprisingly they do not...  You should be able to
> > replace 'socket' with 
> > 'netcat' - and I believe that netcat/nc package is
> > available for Fedora.  For 
> > this purpose they have same command line & behavior.
> 
> Ummm, I am trying on that. nc is avaiable for Fedora.
> But what could be the replacement for CONNECT in
> Fedora? :-)

Erm, you haven't understood what you're doing there, have you?

With the GIT_PROXY_COMMAND helper, you're expected to create a clean
tunnel which in turn git can use to transfer its data.

You've only got some limited internet connectivity via a HTTP proxy
available, so you need to use this. This means:

  * The proxy administrator needs to allos outgoing connections for
    the CONNECT method with git's TCP port.
  * You need to have some minimalistic program to initially speak HTTP
    with the proxy and later on just stream the raw git protocol
    through the link.
  * You may or may not need to strip anything that came into the git
    stream by accident because you tunnled it through a HTTP proxy. A
    reply message from the proxy server is an example for this.

So this little script (using "CONNECT" and netcat or socket) does the
first part: it talks in the language HTTP with the proxy server. It
may be enough to just use CONNECT, but you may need to speak some more
lines, eg. for proxy authorization.

The first `cat' in there is just for pushing the git protocol though the
HTTP proxy connection later on (hopefully after the proxy was made to
accept the the CONNECT request.)  Once the proxy accepted it, it'll
send you a HTTP/200 message (or something like that) and an empty
line. This is what the two reads are for; the next `cat' simply again
transfers all the rest (the git protocol).

To draw the line, there's not _one_ solution to HTTP proxy tunneling,
there are many, and you'll need to design one that fits your network.
It should be quite simple, given that you've got nice tools like
`strace' and `tcpdump', which will help you to understand how the
proxy reacts and so on.

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]