Transparent Remote Proxy Server

Transparent Remote Proxy Server

[ms419]

I'm setting up a transparent proxy to a remote proxy server & can't 
figure out why it doesn't work.

My gateway is tor; it's running Debian unstable, iptables 1.2.11-2, & a 
custom 2.4.27 kernel. My proxy server is wum; it's also running Debian 
unstable, squid 2.5.6-8, iptables 1.2.11-2, & a custom 2.6.7 kernel.

Physically, tor is connected by each of 3 NICs to an ADSL modem, to wum 
by a crossover cable, & to the rest of the network by a hub.

Logically, tor & wum are in 192.168.103.0/24; tor & the rest of the 
network are in 192.168.179.0/24.

The proxy server is working; clients can be configured to use wum as a 
web proxy. Transparent proxying was also working, using DNAT & 
MASQUERADE targets on tor. I'm now setting up a more general solution, 
using policy routing.

All web traffic, except web traffic from wum to external networks, 
should be routed through wum. All other traffic, except traffic between 
local networks, should be masqueraded. To this end, I'm using the 
following rules on tor:

ip route add default via wum table 2

ip rule add fwmark 3 table 2

iptables -t filter -F

iptables -t mangle -F

iptables -t mangle -A PREROUTING -s 192.168.103.0/24 -d 
192.168.103.0/24 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.168.103.0/24 -d 
192.168.179.0/24 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.168.179.0/24 -d 
192.168.103.0/24 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.168.179.0/24 -d 
192.168.179.0/24 -j ACCEPT
iptables -t mangle -A PREROUTING -s wum -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.168.103.0/24 -p tcp --dport 
http -j MARK --set-mark 3
iptables -t mangle -A PREROUTING -s 192.168.179.0/24 -p tcp --dport 
http -j MARK --set-mark 3

iptables -t nat -F

iptables -t nat -A POSTROUTING -s 192.168.103.0/24 -d 192.168.103.0/24 
-j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.103.0/24 -d 192.168.179.0/24 
-j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.179.0/24 -d 192.168.103.0/24 
-j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.179.0/24 -d 192.168.179.0/24 
-j ACCEPT
iptables -t nat -A POSTROUTING -m mark --mark 3 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.103.0/24 -j MASQUERADE
iptables -t nat -A POSTROUTING -s 192.168.179.0/24 -j MASQUERADE

To pass web traffic to the proxy server, I'm using the following rules 
on wum:

iptables -t filter -F

iptables -t mangle -F

iptables -t nat -F

iptables -t nat -A PREROUTING -s 192.168.103.0/24 -d 192.168.103.0/24 
-j ACCEPT
iptables -t nat -A PREROUTING -s 192.168.103.0/24 -d 192.168.179.0/24 
-j ACCEPT
iptables -t nat -A PREROUTING -s 192.168.179.0/24 -d 192.168.103.0/24 
-j ACCEPT
iptables -t nat -A PREROUTING -s 192.168.179.0/24 -d 192.168.179.0/24 
-j ACCEPT
iptables -t nat -A PREROUTING -s 192.168.103.0/24 -p tcp --dport http 
-j REDIRECT --to-port 3128
iptables -t nat -A PREROUTING -s 192.168.179.0/24 -p tcp --dport http 
-j REDIRECT --to-port 3128

Unfortunately, I can't figure out why it doesn't work. Clients 
configured to do so can still use wum as a web proxy, but otherwise, 
they can't access the web.

Running tcpdump on wum seemingly confirms that web traffic is being 
routed through it:

09:37:51.732695 IP 192.168.179.43.60313 > 209.197.112.171.80: S 
1595892543:1595892543(0) win 65535 <mss 1460,nop,wscale 
0,nop,nop,timestamp 1791956815 0>
09:37:51.735092 IP 209.197.112.171.80 > 192.168.179.43.60313: S 
3071158741:3071158741(0) ack 1595892544 win 5792 <mss 
1460,nop,nop,timestamp 500666180 1791956815,nop,wscale 0>

But squid's logs report no activity & clients can't access the web.

Can anyone else suggest what might be wrong with this configuration?

Thanks for your help!

Jack

Re: Transparent Remote Proxy Server

[Aleksandar Milivojevic]

ms419@freezone.co.uk wrote:
> I'm setting up a transparent proxy to a remote proxy server & can't 
> figure out why it doesn't work.
> 
> My gateway is tor; it's running Debian unstable, iptables 1.2.11-2, & a 
> custom 2.4.27 kernel. My proxy server is wum; it's also running Debian 
> unstable, squid 2.5.6-8, iptables 1.2.11-2, & a custom 2.6.7 kernel.
> 
> Physically, tor is connected by each of 3 NICs to an ADSL modem, to wum 
> by a crossover cable, & to the rest of the network by a hub.
> 
> Logically, tor & wum are in 192.168.103.0/24; tor & the rest of the 
> network are in 192.168.179.0/24.

Haven't done something like that myself, but wouldn't it be much simpler 
to implement it like this (using filter table for filtering, and nat 
table for NATing, as they were intended to be used):

replace:
   int_if with tor's internal interface
   ext_if with tor's external interface
   wum_if with tor's interface to wum
   tor_if with wum's interface to tor

On tor:
iptables -t nat -A PREROUTING -i int_if -s 192.168.179.0/24 \
    -p tcp --dport 80 -j DNAT --to-destination wum:3128
iptables -t nat -A POSTROUTING -o ext_if -s 192.168.0.0/16 -j MASQUERADE
iptables -A FORWARD -i int_if -o wum_if -s 192.168.179.0/24 -d wum \
    -p tcp --dport 3128 -m state --state NEW -j ACCEPT

On wum:
iptables -A INPUT -i tor_if -s 192.168.179.0/24 -d wum \
    -p tcp --dport 3128 -m state --state NEW -j ACCEPT

Assuming chains in filter table have default policy set to DROP, and 
chains in nat&mangle tables to ACCEPT.

These are just hints, you are advised not to cut&paste them into your 
configuration (tm).

-- 
Aleksandar Milivojevic <amilivojevic@pbl.ca>    Pollard Banknote Limited
Systems Administrator                           1499 Buffalo Place
Tel: (204) 474-2323 ext 276                     Winnipeg, MB  R3T 1L7

Re: Transparent Remote Proxy Server

[ms419]

On Sep 24, 2004, at 12:01 PM, Aleksandar Milivojevic wrote:

> ms419@freezone.co.uk wrote:
>> I'm setting up a transparent proxy to a remote proxy server & can't 
>> figure out why it doesn't work.
>> My gateway is tor; it's running Debian unstable, iptables 1.2.11-2, & 
>> a custom 2.4.27 kernel. My proxy server is wum; it's also running 
>> Debian unstable, squid 2.5.6-8, iptables 1.2.11-2, & a custom 2.6.7 
>> kernel.
>> Physically, tor is connected by each of 3 NICs to an ADSL modem, to 
>> wum by a crossover cable, & to the rest of the network by a hub.
>> Logically, tor & wum are in 192.168.103.0/24; tor & the rest of the 
>> network are in 192.168.179.0/24.
>
> Haven't done something like that myself, but wouldn't it be much 
> simpler to implement it like this (using filter table for filtering, 
> and nat table for NATing, as they were intended to be used):

Thank you sincerely for your suggestion! You are correct: It is simpler 
to use DNAT & MASQUERADE. I tried it & it works. But now, I am trying 
to avoid using NAT.

The problem with NAT (as I understand it) is it rewrites the 
destination address, which breaks HTTP/1.0 requests without a Host: 
header. By using policy routing, I hope to route traffic through wum 
without rewriting the destination address.

I'm using as my guide the Transparent Proxy HOWTO by Daniel Kiracofe.

I'm still not sure what is wrong - thank you very much for any help!

Jack

Re: Transparent Remote Proxy Server

[Jason Opperisano]

On Mon, 2004-09-27 at 14:25, ms419@freezone.co.uk wrote:
> Thank you sincerely for your suggestion! You are correct: It is simpler 
> to use DNAT & MASQUERADE. I tried it & it works. But now, I am trying 
> to avoid using NAT.
> 
> The problem with NAT (as I understand it) is it rewrites the 
> destination address, which breaks HTTP/1.0 requests without a Host: 
> header. By using policy routing, I hope to route traffic through wum 
> without rewriting the destination address.

your "-j REDIRECT" on wum rewrites the destination IP address as well.

-j

-- 
Jason Opperisano <opie@817west.com>

Re: Transparent Remote Proxy Server

[Aleksandar Milivojevic]

ms419@freezone.co.uk wrote:
> Thank you sincerely for your suggestion! You are correct: It is simpler 
> to use DNAT & MASQUERADE. I tried it & it works. But now, I am trying to 
> avoid using NAT.
> 
> The problem with NAT (as I understand it) is it rewrites the destination 
> address, which breaks HTTP/1.0 requests without a Host: header. By using 
> policy routing, I hope to route traffic through wum without rewriting 
> the destination address.
> 
> I'm using as my guide the Transparent Proxy HOWTO by Daniel Kiracofe.

As Jason wrote, REDIRECT will also rewrite destination IP address. 
Squid is using woodoo magic to find out what was the original 
destination address before rewriting (that's why you need to also change 
Squid configuration).  Squid can do that only if it runs on the same box 
where the address was rewritten.  Even the woodoo magic has limitations ;-)

Daniel's document is using nat table for both filtering and NATing, 
which is the approach I don't particulary like.  Plus the rules are very 
open (made to demonstrate the concept, not to be used on a real firewall).

Anyhow, what you might try out is a bit of debugging to see what is 
going on the wire.  Your tcpdump from wum shows that it got SYN packet, 
and that it sent out SYN ACK.  I don't see third packet with ACK going 
back to wum, so it might be that it got dropped somewhere.  My next step 
would be to move to tor.  I'd guess if you run tcpdump on tor's 
interface to internal network, you'd see only SYN packet, and not SYN 
ACK.  And if you run it on interface wheer wum is connected that you 
would see both SYN and SYN ACK.  If that is the case, than you might 
have something else on tor that is causing it to drop return packets 
from wum.  As usual, without having an insight into your entire rule 
set, it might be hard to tell why (something like output of iptables-save).

-- 
Aleksandar Milivojevic <amilivojevic@pbl.ca>    Pollard Banknote Limited
Systems Administrator                           1499 Buffalo Place
Tel: (204) 474-2323 ext 276                     Winnipeg, MB  R3T 1L7

Re: Transparent Remote Proxy Server

[Jason Opperisano]

On Mon, 2004-09-27 at 15:30, Aleksandar Milivojevic wrote:
> As Jason wrote, REDIRECT will also rewrite destination IP address. 
> Squid is using woodoo magic to find out what was the original 
> destination address before rewriting (that's why you need to also change 
> Squid configuration).  Squid can do that only if it runs on the same box 
> where the address was rewritten.  Even the woodoo magic has limitations ;-)
> 
> Daniel's document is using nat table for both filtering and NATing, 
> which is the approach I don't particulary like.  Plus the rules are very 
> open (made to demonstrate the concept, not to be used on a real firewall).
> 
> Anyhow, what you might try out is a bit of debugging to see what is 
> going on the wire.  Your tcpdump from wum shows that it got SYN packet, 
> and that it sent out SYN ACK.  I don't see third packet with ACK going 
> back to wum, so it might be that it got dropped somewhere.  My next step 
> would be to move to tor.  I'd guess if you run tcpdump on tor's 
> interface to internal network, you'd see only SYN packet, and not SYN 
> ACK.  And if you run it on interface wheer wum is connected that you 
> would see both SYN and SYN ACK.  If that is the case, than you might 
> have something else on tor that is causing it to drop return packets 
> from wum.  

keep in mind that a side-effect of the REDIRECT "magic" is that the SYN
does not go to; nor does the SYN-ACK come from, wum's IP address (as was
evident from the tcpdump snippet provided).

with the REDIRECT target, wum will pretend to be the destination web
server from the perspective of the client and tor.

all this being said--i couldn't tell from the OP why things weren't
working.

tcpdump anywhere and everywhere and you'll find the culprit.

-j

-- 
Jason Opperisano <opie@817west.com>

Re: Transparent Remote Proxy Server

[Arthur Meyer]

Am Montag, 27. September 2004 21:37 schrieb Jason Opperisano:
> On Mon, 2004-09-27 at 15:30, Aleksandar Milivojevic wrote:
> > As Jason wrote, REDIRECT will also rewrite destination IP address.
> > Squid is using woodoo magic to find out what was the original
> > destination address before rewriting (that's why you need to also change
> > Squid configuration).  Squid can do that only if it runs on the same box
> > where the address was rewritten.  Even the woodoo magic has limitations
> > ;-)
> >
> > Daniel's document is using nat table for both filtering and NATing,
> > which is the approach I don't particulary like.  Plus the rules are very
> > open (made to demonstrate the concept, not to be used on a real
> > firewall).
> >
> > Anyhow, what you might try out is a bit of debugging to see what is
> > going on the wire.  Your tcpdump from wum shows that it got SYN packet,
> > and that it sent out SYN ACK.  I don't see third packet with ACK going
> > back to wum, so it might be that it got dropped somewhere.  My next step
> > would be to move to tor.  I'd guess if you run tcpdump on tor's
> > interface to internal network, you'd see only SYN packet, and not SYN
> > ACK.  And if you run it on interface wheer wum is connected that you
> > would see both SYN and SYN ACK.  If that is the case, than you might
> > have something else on tor that is causing it to drop return packets
> > from wum.
>
> keep in mind that a side-effect of the REDIRECT "magic" is that the SYN
> does not go to; nor does the SYN-ACK come from, wum's IP address (as was
> evident from the tcpdump snippet provided).
>
> with the REDIRECT target, wum will pretend to be the destination web
> server from the perspective of the client and tor.
>
> all this being said--i couldn't tell from the OP why things weren't
> working.
>
> tcpdump anywhere and everywhere and you'll find the culprit.
>
> -j

If you are using squid and netfilter to setup a firewall, you need to modify 
configure and squid.conf accordingly.
Compile squid mit configure parameter --enable-linux-netfilter.
Set then in the squid.conf die following statements
httpd_accel_host to virtual
httpd_accel_with_proxy to on
httpd_accel_uses_host_header to on

With these settings you can use nat.
Arthur Meyer
TBZ

Re: Transparent Remote Proxy Server

[Jason Opperisano]

On Mon, 2004-09-27 at 15:37, Jason Opperisano wrote:
> keep in mind that a side-effect of the REDIRECT "magic" is that the SYN
> does not go to; nor does the SYN-ACK come from, wum's IP address (as was
> evident from the tcpdump snippet provided).
> 
> with the REDIRECT target, wum will pretend to be the destination web
> server from the perspective of the client and tor.
> 
> all this being said--i couldn't tell from the OP why things weren't
> working.
> 
> tcpdump anywhere and everywhere and you'll find the culprit.
> 
> -j

hmmm...along those same lines--i just had a thought.  is it possible
that "rp_filter" on tor is blocking the packets from wum because "-j
REDIRECT" is making wum spoof the address of the destination web server?

on tor, check:

        sysctl -a | grep '\.rp_filter'
        sysctl -a | grep log_martians
        
try either (a) setting rp_filter=0 on the interface on tor that is
connected to wum, or (b) leave rp_filter=1 and set "log_martians=1" on
that interface on tor to see if rp_filter is; in fact, the culprit.

just a thought.

-j 

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
I'd horsewhip you if I had a horse. -- Groucho Marx
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~

FIXED: Transparent Remote Proxy Server

[ms419]

On Sep 28, 2004, at 6:44 AM, Jason Opperisano wrote:
> On Mon, 2004-09-27 at 15:37, Jason Opperisano wrote:
>> keep in mind that a side-effect of the REDIRECT "magic" is that the 
>> SYN
>> does not go to; nor does the SYN-ACK come from, wum's IP address (as 
>> was
>> evident from the tcpdump snippet provided).
>>
>> with the REDIRECT target, wum will pretend to be the destination web
>> server from the perspective of the client and tor.
>>
>> all this being said--i couldn't tell from the OP why things weren't
>> working.
>>
>> tcpdump anywhere and everywhere and you'll find the culprit.
>>
>> -j
>
> hmmm...along those same lines--i just had a thought.  is it possible
> that "rp_filter" on tor is blocking the packets from wum because "-j
> REDIRECT" is making wum spoof the address of the destination web 
> server?
>
> on tor, check:
>
>         sysctl -a | grep '\.rp_filter'
>         sysctl -a | grep log_martians
>
> try either (a) setting rp_filter=0 on the interface on tor that is
> connected to wum, or (b) leave rp_filter=1 and set "log_martians=1" on
> that interface on tor to see if rp_filter is; in fact, the culprit.
>
> just a thought.
>
> -j

Oh my gosh, you are so brilliant!

I doubted this was the problem, since squid's logs showed no activity - 
I thought my problem was in reaching wum, not in receiving returning 
packets. Shows how much I know : P

Strangely, sysctl -w net.ipv4.conf.eth2.log_martians=1 doesn't log a 
thing. I don't understand why - I tailed every file in /var/log.

I set sysctl -w net.ipv4.conf.eth2.rp_filter=0 anyway, & it works! 
Thank you so much!

I'd like to send you a token of my appreciation, if you'll give me your 
postal address. (It's homemade, so please don't get too excited : )

Sincere thanks for everyone's help!

Jack