Help! My ISP blocks repo.or.cz. How to push changes?

Help! My ISP blocks repo.or.cz. How to push changes?

[Jakub Narebski]

The ISP I use (Telekomunikacja Polska S.A., aka TP) made some 
unannounced changes for ADSL service (Neostrada) which made it block 
repo.or.cz (and of course its aliases, including git.or.cz where git 
wiki resides). It blocks also gimp.org and some Polish IRC servers 
(irc.freenode.org on which #git resides works O.K.). People speculate 
that this blocking was based on MAPS (Mail Abuse Prevention System, 
which is SPAM backwards) lists to fight SPAM and/or to block botnets,
and uses null routing (IP based) blocking. I have no idea why repo.or.cz 
is blocked: gimp.org is supposedly blocked because it hosts 
irc.gimp.org on the same IP. By block I mean that even ping doesn't 
work (no reply at all).

I can access git wiki via one of many free HTTP proxies; currently I use 
http://www.4proxy.de so there are only slight problems there.  


The problems is with fetching (via git:// protocol) of forks of git 
repository on repo.or.cz, and pushing (via SSH) to a few of my git 
repositories hosted on repo.or.cz.

Do you have any suggestions to bypass this block for git? I have access 
to Linux shell account (no root access, though) which doesn't have 
problems with repo.or.cz, so I think I could set up SSH tunnel: but 
how? And what to do with access via git:// - move to SSH too?

Thanks in advance
-- 
Jakub Narebski
Poland

http://forums.thedailywtf.com/forums/t/10789.aspx

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Alex Riesen]

2009/1/12 Jakub Narebski <jnareb@gmail.com>:
> Do you have any suggestions to bypass this block for git? I have access
> to Linux shell account (no root access, though) which doesn't have
> problems with repo.or.cz, so I think I could set up SSH tunnel: but
> how? And what to do with access via git:// - move to SSH too?

See man ssh, look for -L. It works for arbitrary ports, so you can redirect
git:// port to anywhere. Same for push over ssh, just give another port when
connecting.

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Jakub Narebski]

Alex Riesen wrote:
> 2009/1/12 Jakub Narebski <jnareb@gmail.com>:

> > Do you have any suggestions to bypass this block for git? I have access
> > to Linux shell account (no root access, though) which doesn't have
> > problems with repo.or.cz, so I think I could set up SSH tunnel: but
> > how? And what to do with access via git:// - move to SSH too?
> 
> See man ssh, look for -L. It works for arbitrary ports, so you can redirect
> git:// port to anywhere. Same for push over ssh, just give another port when
> connecting.

Currently I have the folowing in my ~/.ssh/config:

  # TP S.A. blocks repo.or.cz
  Host repo.or.cz
	NoHostAuthenticationForLocalhost yes
	HostName localhost
	Port 2222

and I can simply use "git push repo" without any changes.
But I have to run 

 $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com

first. Is there any way to automate this?

-- 
Jakub Narebski
Poland

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Luciano Rocha]

[-- Attachment #1.1: Type: text/plain, Size: 1333 bytes --]

On Mon, Jan 12, 2009 at 12:13:44PM +0100, Jakub Narebski wrote:
> Alex Riesen wrote:
> > 2009/1/12 Jakub Narebski <jnareb@gmail.com>:
> 
> > > Do you have any suggestions to bypass this block for git? I have access
> > > to Linux shell account (no root access, though) which doesn't have
> > > problems with repo.or.cz, so I think I could set up SSH tunnel: but
> > > how? And what to do with access via git:// - move to SSH too?
> > 
> > See man ssh, look for -L. It works for arbitrary ports, so you can redirect
> > git:// port to anywhere. Same for push over ssh, just give another port when
> > connecting.
> 
> Currently I have the folowing in my ~/.ssh/config:
> 
>   # TP S.A. blocks repo.or.cz
>   Host repo.or.cz
> 	NoHostAuthenticationForLocalhost yes
> 	HostName localhost
> 	Port 2222
> 
> and I can simply use "git push repo" without any changes.
> But I have to run 
> 
>  $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
> 
> first. Is there any way to automate this?

For those cases I use the attached script. Read the first lines, with
information on how to install.

The end result is that I can do:
ssh host1--host2--host3

Or scp, rsync, etc.

Regards,
Luciano Rocha

-- 
Luciano Rocha <luciano@eurotux.com>
Eurotux Informática, S.A. <http://www.eurotux.com/>

[-- Attachment #1.2: ssh_p --]
[-- Type: text/plain, Size: 2166 bytes --]

#!/bin/bash
## SSH connection chaining
##
## (C) 2007, Luciano Rocha <strange@nsk.pt>
##
## This program is free software; you can redistribute it and/or
## modify it under the terms of the GNU General Public License
## as published by the Free Software Foundation; either version 2
## of the License, or (at your option) any later version.
## 
## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.
## 
## You should have received a copy of the GNU General Public License
## along with this program; if not, write to the Free Software
## Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
## MA 02110-1301, USA.
## 
## Or check the webpage: http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
##
## Documentation:
##
## Include in your .ssh/config:
## host *--*
## proxycommand ssh_p %h %p
##
## And then you can do:
## ssh hosta--hostb--hostc
## scp hosta--hostb--hostc:file .
##
## A user specification is also possible, but not for the last host, for
## that one, you'll have to use ssh's standard user specification (user@host
## or -luser):
## ssh usera_hosta--userb_hostb--hostc -luserc
## scp userc@usera_hosta--userb_hostb--hostc:file .
##
## Compression is disable on all proxy sshs, as traffic is already encrypted.
## Activate compression in the master ssh command, if desired.
##
## Environment variable SSH_CONFIG_FILE can point to a ssh configuration
## file for use in the proxy sshs.

## missing hops
h="${1%--*}"
## nc target
rh="${1##*--}"
p="${2:-22}"


## user for next hop
u=
n="${h##*--}"
if [ -z "${n##*_*}" ]; then
	u=${n%%_*}
	if [ "$h" != "$n" ]; then
		h="${h%--*}--${n##*_}"
	else
		h="${n##*_}"
	fi
fi

echo "${SSH_INDENT}+ connecting to $h for $rh" >&2


exec env SSH_INDENT="${SSH_INDENT} " ssh -o "Compression no" \
	${SSH_CONFIG_FILE:+-F$SSH_CONFIG_FILE} \
	${u:+$u@}$h \
	"echo \"${SSH_INDENT}- connected to $h, forwarding to $rh:$p\" >&2 ; nc -w 3600 $rh $p"

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Asheesh Laroia]

On Mon, 12 Jan 2009, Jakub Narebski wrote:

> But I have to run
>
> $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
>
> first. Is there any way to automate this?

Check out 'gstm' or 'autossh'.

-- Asheesh.

-- 
You will get what you deserve.

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Jakub Narebski]

Asheesh Laroia wrote:
> On Mon, 12 Jan 2009, Jakub Narebski wrote:
> 
> > But I have to run
> >
> > $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
> >
> > first. Is there any way to automate this?
> 
> Check out 'gstm' or 'autossh'.

I don't know about gSTM (Gnome SSH Tunnel Manager), but autossh
does only provide reconnect in the case the gateway host closes
connection. I still have to run it, perhaps from startup script.

-- 
Jakub Narebski
Poland

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Asheesh Laroia]

On Tue, 13 Jan 2009, Jakub Narebski wrote:

> Asheesh Laroia wrote:
>> On Mon, 12 Jan 2009, Jakub Narebski wrote:
>>
>>> But I have to run
>>>
>>> $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
>>>
>>> first. Is there any way to automate this?
>>
>> Check out 'gstm' or 'autossh'.
>
> I don't know about gSTM (Gnome SSH Tunnel Manager), but autossh
> does only provide reconnect in the case the gateway host closes
> connection. I still have to run it, perhaps from startup script.

Yeah, gSTM is pretty manual in that regard. autossh from a startup script 
makes sense.

-- Asheesh.

-- 
Q:	How many marketing people does it take to change a light bulb?
A:	I'll have to get back to you on that.

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Mike Hommey]

On Mon, Jan 12, 2009 at 12:13:44PM +0100, Jakub Narebski <jnareb@gmail.com> wrote:
> Alex Riesen wrote:
> > 2009/1/12 Jakub Narebski <jnareb@gmail.com>:
> 
> > > Do you have any suggestions to bypass this block for git? I have access
> > > to Linux shell account (no root access, though) which doesn't have
> > > problems with repo.or.cz, so I think I could set up SSH tunnel: but
> > > how? And what to do with access via git:// - move to SSH too?
> > 
> > See man ssh, look for -L. It works for arbitrary ports, so you can redirect
> > git:// port to anywhere. Same for push over ssh, just give another port when
> > connecting.
> 
> Currently I have the folowing in my ~/.ssh/config:
> 
>   # TP S.A. blocks repo.or.cz
>   Host repo.or.cz
> 	NoHostAuthenticationForLocalhost yes
> 	HostName localhost
> 	Port 2222
> 
> and I can simply use "git push repo" without any changes.
> But I have to run 
> 
>  $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
> 
> first. Is there any way to automate this?

Something like the following should do the trick:
Host repo.or.cz
	ProxyCommand ssh jnareb@host.example.com nc %h %p

You will need nc (netcat) installed on the host.example.com server, though.

Mike

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Jakub Narebski]

On Mon, 12 January 2009, Mike Hommey wrote:
> On Mon, Jan 12, 2009 at 12:13:44PM +0100, Jakub Narebski <jnareb@gmail.com> wrote:

>> Currently I have the folowing in my ~/.ssh/config:
>> 
>>   # TP S.A. blocks repo.or.cz
>>   Host repo.or.cz
>> 	NoHostAuthenticationForLocalhost yes
>> 	HostName localhost
>> 	Port 2222
>> 
>> and I can simply use "git push repo" without any changes.
>> But I have to run 
>> 
>>  $ ssh -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
>> 
>> first. Is there any way to automate this?
> 
> Something like the following should do the trick:
> Host repo.or.cz
> 	ProxyCommand ssh jnareb@host.example.com nc %h %p
> 
> You will need nc (netcat) installed on the host.example.com server, though.

I assume that is both in place of above ~/.ssh/config configuration,
and making unnecessary port forwarding (ssh -L) invocation, isn't it?

P.S. What should I put in core.gitProxy to make it possible to fetch
via git:// protocol from repo.or.cz?

-- 
Jakub Narebski
Poland

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Markus Heidelberg]

Jakub Narebski, 13.01.2009:
> P.S. What should I put in core.gitProxy to make it possible to fetch
> via git:// protocol from repo.or.cz?

I'm not sure if this is what you need, but I use this at work for
fetching via git protocol:

[core]
	gitProxy = /etc/gitproxy.sh for kernel.org
	gitProxy = /etc/gitproxy.sh for or.cz
	# and several others ...

gitproxy.sh:
#! /bin/sh
(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket <company proxy host> <port> | (read a; read a; cat )

Markus

Re: Help! My ISP blocks repo.or.cz. How to push changes? (a solution)

[Jakub Narebski]

Jakub Narebski wrote:

> The ISP I use (Telekomunikacja Polska S.A., aka TP) made some
> unannounced changes for ADSL service (Neostrada) which made it block
> repo.or.cz (and of course its aliases, including git.or.cz where git
> wiki resides).

Thank you all for your help with arriving at solution. I'll describe it 
below; perhaps it would help somebody else (now that it is in mailing 
list archive).


First, let me explain what I am working with:

I have access to shell account with set up SSH key access; let's name 
this machine host.example.com. I don't have admin rights there, and 
quota is quite tight; I have installed netcat (nc) in ~/bin - it is 
only 22 kB.

I don't know where to find SOCKS5 proxy, and I don't have 'tsocks'
installed either on my computer, or on shell account... I think.


Now, solutions:

1. For reading gitweb at repo.or.cz, and for reading and editing git 
   wiki at http://git.or.cz/gitwiki/ I use one of free HTTP proxies:
   http://www.4proxy.de (first such proxy I have found that has an
   option to _not_ obfuscate URLs; it still unnecessary escapes some
   things like '/' in the query argument).

2. For pushing changes to repo.or.cz I use SSH tunnel (I could have
   used ProxyCommand solution with netcat instead[1]). I run:

   $ autossh -M 2000 -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com

   at startup, and I have set the following in ~/.ssh/config:

   # TP S.A. blocks repo.or.cz
   Host repo.or.cz
   	#ssh -f -N -L 2222:repo.or.cz:22 host.example.com
   	NoHostAuthenticationForLocalhost yes
   	HostName localhost
   	Port 2222

   [1] Alternate solution:

   # TP S.A. blocks repo.or.cz
   Host repo.or.cz
	ProxyCommand ssh host.example.com exec /home/jnareb/bin/nc %h %p

3. For fetching changes via git:// protocol from repo.or.cz I use the
   following setup in git config:

   [core]
   	gitProxy = ssh-proxy for "repo.or.cz"

   Unfortunately example from Documentation/config.txt with "ssh" as
   git proxy command doesn't work, and neither putting command with
   options (e.g. "ssh host.example.com /home/jnareb/bin/nc") doesn't
   work: the command is _not_ split on whitespace. So I had to use
   helper script ~/bin/ssh-proxy:

   #!/bin/sh

   ssh host.example.com /home/jnareb/bin/nc "$1" "$2"


I hope that would help somebody... and if somebody notices better 
solution, hs/she would provide me with it :-)

-- 
Jakub Narebski
Poland

Re: Help! My ISP blocks repo.or.cz. How to push changes? (a solution)

[Asheesh Laroia]

On Wed, 14 Jan 2009, Jakub Narebski wrote:

> Jakub Narebski wrote:
>
>> The ISP I use (Telekomunikacja Polska S.A., aka TP) made some
>> unannounced changes for ADSL service (Neostrada) which made it block
>> repo.or.cz (and of course its aliases, including git.or.cz where git
>> wiki resides).
>
> Thank you all for your help with arriving at solution. I'll describe it
> below; perhaps it would help somebody else (now that it is in mailing
> list archive).

I'll just add some comments that explain what I suggested with a little 
more clarity. I'm not saying you have to use it or anything, I just want 
to make sure I was clear!

And I'm glad you have a solution!

> First, let me explain what I am working with:
>
> I have access to shell account with set up SSH key access; let's name
> this machine host.example.com. I don't have admin rights there, and
> quota is quite tight; I have installed netcat (nc) in ~/bin - it is
> only 22 kB.

Yes, that is great!

> I don't know where to find SOCKS5 proxy, and I don't have 'tsocks' 
> installed either on my computer, or on shell account... I think.

tsocks is packaged in Debian, and surely in other distributions as well. 
You don't run it on the shell account, but on your own workstation (which 
I call "laptop" below for clarity).

A SOCKS5 proxy can be generated by ssh by running:

 	[user@laptop] $ ssh -D 1080 shelluser@shellbox

Now:
 	[user@laptop] $ telnet localhost 1080

will demonstrate that local port 1080 is listening. Because you created it 
with -D to ssh, that local port 1080 *is* a SOCKS5 proxy, created by the 
local SSH client.

> Now, solutions:
>
> 1. For reading gitweb at repo.or.cz, and for reading and editing git
>   wiki at http://git.or.cz/gitwiki/ I use one of free HTTP proxies:
>   http://www.4proxy.de (first such proxy I have found that has an
>   option to _not_ obfuscate URLs; it still unnecessary escapes some
>   things like '/' in the query argument).

Great! (Though note that configuring the web browser to use your 
SSH-created SOCKS5 proxy would let you avoid this escaping since the whole 
system would be under your control.)

> 2. For pushing changes to repo.or.cz I use SSH tunnel (I could have
>   used ProxyCommand solution with netcat instead[1]). I run:
>
>   $ autossh -M 2000 -f -N -L 2222:repo.or.cz:22 jnareb@host.example.com
>
>   at startup, and I have set the following in ~/.ssh/config:
>
>   # TP S.A. blocks repo.or.cz
>   Host repo.or.cz
>   	#ssh -f -N -L 2222:repo.or.cz:22 host.example.com
>   	NoHostAuthenticationForLocalhost yes
>   	HostName localhost
>   	Port 2222
>
>   [1] Alternate solution:
>
>   # TP S.A. blocks repo.or.cz
>   Host repo.or.cz
> 	ProxyCommand ssh host.example.com exec /home/jnareb/bin/nc %h %p

Right-o.

> 3. For fetching changes via git:// protocol from repo.or.cz I use the
>   following setup in git config:
>
>   [core]
>   	gitProxy = ssh-proxy for "repo.or.cz"
>
>   Unfortunately example from Documentation/config.txt with "ssh" as
>   git proxy command doesn't work, and neither putting command with
>   options (e.g. "ssh host.example.com /home/jnareb/bin/nc") doesn't
>   work: the command is _not_ split on whitespace. So I had to use
>   helper script ~/bin/ssh-proxy:
>
>   #!/bin/sh
>
>   ssh host.example.com /home/jnareb/bin/nc "$1" "$2"

Great!

> I hope that would help somebody... and if somebody notices better 
> solution, hs/she would provide me with it :-)

(-:

-- Asheesh.

-- 
A visit to a fresh place will bring strange work.

Re: Help! My ISP blocks repo.or.cz. How to push changes?

[Asheesh Laroia]

This is a little off-topic, but you CC:d the git list first. (-;

On Mon, 12 Jan 2009, Jakub Narebski wrote:

> The ISP I use (Telekomunikacja Polska S.A., aka TP) made some
> unannounced changes for ADSL service (Neostrada) which made it block
> repo.or.cz (and of course its aliases, including git.or.cz where git
> wiki resides). It blocks also gimp.org and some Polish IRC servers
> (irc.freenode.org on which #git resides works O.K.). People speculate
> that this blocking was based on MAPS (Mail Abuse Prevention System,
> which is SPAM backwards) lists to fight SPAM and/or to block botnets,
> and uses null routing (IP based) blocking. I have no idea why repo.or.cz
> is blocked: gimp.org is supposedly blocked because it hosts
> irc.gimp.org on the same IP. By block I mean that even ping doesn't
> work (no reply at all).

That's horrifying.

> I can access git wiki via one of many free HTTP proxies; currently I use
> http://www.4proxy.de so there are only slight problems there.
>
> The problems is with fetching (via git:// protocol) of forks of git
> repository on repo.or.cz, and pushing (via SSH) to a few of my git
> repositories hosted on repo.or.cz.
>
> Do you have any suggestions to bypass this block for git? I have access
> to Linux shell account (no root access, though) which doesn't have
> problems with repo.or.cz, so I think I could set up SSH tunnel: but
> how? And what to do with access via git:// - move to SSH too?

$ ssh -D 1080 user@host

In a a separate terminal:

$ cat > /tmp/tsocks.conf
# Here we have a config for tsocks that uses localhost:1080 as SOCKS5.
server = 127.0.0.1
# Server type defaults to 4 so we need to specify it as 5 for this one
server_type = 5
# The port defaults to 1080 but I've stated it here for clarity
server_port = 1080
^D
$ export TSOCKS_CONF_FILE=/tmp/tsocks.conf
$ tsocks lynx http://repo.or.cz/

You would need the 'tsocks' tool for your client system. Note that the 
remote system doesn't need any configuration this way. Just prefix any 
command-line operations that involve repo.or.cz with 'tsocks' and be sure 
to set TSOCKS_CONF_FILE. (On a Windows machine, use putty's graphical SSH 
client and FreeCap <http://www.freecap.ru/eng/>.

Once you have the 'ssh -D' tunnel running, you can use it in Firefox as a 
SOCKS proxy: host = localhost, port = 1080.

-- Asheesh.

-- 
Don't relax!  It's only your tension that's holding you together.