snat to multiple source ip

snat to multiple source ip

[Marco Berizzi]

Hello everybody.

I have a system with Slackware linux 2.4.30 and squid 2.5s10.
Squid is running as an http proxy to control internet access.
This system is connected to the internet with 1 hdsl and
2 adsl connections.
My firewall script mark all the squid packet and I route them
through the two adsl connections (I have patched the kernel
with the equalize patch).
The problem is that Squid set the packet source ip with the
hdsl ip, so I need to snat them to the ADSLs ip:

iptables -t nat -I POSTROUTING -s HDSL_ip
--protocol tcp -m multiport --dports SQUIDports
-j SNAT --to first_adsl_ip --to second_adsl_ip

As stated by man:

"You can add several --to-source option. If you specify
more than one source address, either via an address range
or multiple --to-source options, a simple round-robin  (one
after another in cycle) takes place between these addresses."

Everytime I connect to the internet the ip address is *always*
the first_adsl_ip and after few refresh it change to second_adsl_ip
(I know there are chaching issue with proxy/browser).
Usually I connect to http://www.dnsstuff.com/ to control which is
my ip address. My company has about 150 pc that connect through this
proxy. Is this behaviour fine or am I missing anything?

Re: snat to multiple source ip

[/dev/rob0]

On Thursday 2005-September-15 04:59, Marco Berizzi wrote:
> My firewall script mark all the squid packet and I route them

You did not post your rules.

> through the two adsl connections (I have patched the kernel
> with the equalize patch).

Which patch is this? I have used Julian Anastasov's patches
    http://www.ssi.bg/~ja/#multigw
at numerous sites. The NANO-HOWTO linked therefrom is a step by step 
recipe for how to set up the routing and firewall rules.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

snat to multiple source ip

[Marco Berizzi]

Thanks for the reply Rob0.

On Thursday 2005-September-15 04:59, Marco Berizzi wrote:
>> My firewall script mark all the squid packet and I route them

>You did not post your rules.

This is not the real problem.
The problem is the SNAT rule. It is doesn't work as stated
by man. How does it work? Per socket? Or per socket per host?

iptables -t nat -I POSTROUTING -s HDSL_ip
--protocol tcp -m multiport --dports SQUIDports
-j SNAT --to first_adsl_ip --to second_adsl_ip

This rules snat all packets created by this (squid) host, but
everytime I connect to the internet *always* the first_adsl_ip
is chosen. My company lan has about 150 pc that connect to
the internet by this proxy, so I don't understand why *everytime*
I open my browser and I connect to www.dnsstuff.com always the
same ip is displayed. It isn't a cache problem because I reset them
(both on squid and browser side).

>> through the two adsl connections (I have patched the kernel
>> with the equalize patch).

> Which patch is this? I have used Julian Anastasov's patches

Here is http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/1314.html
However this is only for routing not for nat.

Re: snat to multiple source ip

[Edmundo Carmona]

You are using multiroute path, right?

what is the output of

ip route show default

(on the router, of course).

On 9/20/05, Marco Berizzi <pupilla@hotmail.com> wrote:
> Thanks for the reply Rob0.
> 
> On Thursday 2005-September-15 04:59, Marco Berizzi wrote:
> >> My firewall script mark all the squid packet and I route them
> 
> >You did not post your rules.
> 
> This is not the real problem.
> The problem is the SNAT rule. It is doesn't work as stated
> by man. How does it work? Per socket? Or per socket per host?
> 
> iptables -t nat -I POSTROUTING -s HDSL_ip
> --protocol tcp -m multiport --dports SQUIDports
> -j SNAT --to first_adsl_ip --to second_adsl_ip
> 
> This rules snat all packets created by this (squid) host, but
> everytime I connect to the internet *always* the first_adsl_ip
> is chosen. My company lan has about 150 pc that connect to
> the internet by this proxy, so I don't understand why *everytime*
> I open my browser and I connect to www.dnsstuff.com always the
> same ip is displayed. It isn't a cache problem because I reset them
> (both on squid and browser side).
> 
> >> through the two adsl connections (I have patched the kernel
> >> with the equalize patch).
> 
> > Which patch is this? I have used Julian Anastasov's patches
> 
> Here is http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/1314.html
> However this is only for routing not for nat.
> 
> 
> 
>

Re: snat to multiple source ip

[Marco Berizzi]

Edmundo Carmona wrote:

> You are using multiroute path, right?

This is my firewall/proxy schema:


        +---------+
        |         |
        |         |              ---HDSL
        |         |             /
Private |firewall |            /
--------+         +---eth0----+-----adslA
network |   &     |            \
        | Squid   |             \
        |         |              ---adslB
        |         |
        +---------+


> what is the output of

> ip route show default

> (on the router, of course).

Here is:

hdsl_default_gateway_ip dev eth0  scope link
adslB_network dev eth0  proto kernel  scope link  src adslB_ip
adslA_network dev eth0  proto kernel  scope link  src adslA_ip
hdsl_network dev eth1  scope link
my_private_network dev eth2  proto kernel  scope link  src
linux_private_ip
127.0.0.0/8 dev lo  scope link
default via hdsl_default_gateway_ip dev eth0  metric 1

Then I run this script to add ip & gw for adsl connections:

$IP address add $ip_adsl_a/$netmask_adsl_a brd + dev eth0 label
eth0:adslA
$IP address add $ip_adsl_b/$netmask_adsl_b brd + dev eth0 label
eth0:adslB

$IP route add $adsl_network_a dev eth0 src $ip_adsl_a table adslA
$IP route add default via $GW_adsl_a table adslA
$IP route add $adsl_network_b dev eth0 src $ip_adsl_b table adslB
$IP route add default via $GW_adsl_b table adslB

$IP route add $adsl_network_a dev eth0 src $ip_adsl_a table adsl
$IP route add $adsl_network_b dev eth0 src $ip_adsl_b table adsl
$IP route add default equalize table adsl \
             nexthop dev eth0 via $GW_adsl_a weight 1 \
             nexthop dev eth0 via $GW_adsl_b weight 1

$IP rule add fwmark 1 table adsl priority 400
[firewall mark packets so they are going to this routing table]

However routing and SNAT should not be related.
My question is the following.
How does SNAT work when multiple ip are specified?

iptables -t nat -I POSTROUTING -s ipX --protocol tcp
-j SNAT --to first_ip --to second_ip

Why everytime I connect to www.dnsstuff.com I get always the
first_ip? And after refreshing show me the second ip?
I *think* that the round robin cycle works on a per socket/per host
couple.
Example:

1st socket opened to a.a.a.a  -->> SNAT with first_ip
2nd socket opened to b.b.b.b  -->> SNAT with first_ip
3rd socket opened to c.c.c.c  -->> SNAT with first_ip
4th socket opened to b.b.b.b  -->> SNAT with second_ip
5th socket opened to d.d.d.d  -->> SNAT with first_ip
6th socket opened to a.a.a.a  -->> SNAT with second_ip

Can anyone confirm this? Wrong or correct?

Re: snat to multiple source ip

[/dev/rob0]

On Tuesday 20 September 2005 07:39, Marco Berizzi wrote:
> > Which patch is this? I have used Julian Anastasov's patches
>
> Here is
> http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/1314.html
> However this is only for routing not for nat.

I really do not know, sorry. All I can do is suggest that you try it 
using Julian's routes patch and the NANO-HOWTO. IWFM. I understand some 
of the "how" but very little of the "why". Eventually I hope to know 
more about it.

My non-technical explanation of how it works: the source IP is set by 
iproute2 in the routing tables. This is where the load balancing takes 
place.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: snat to multiple source ip

[Marco Berizzi]

/dev/rob0 wrote:

> On Tuesday 20 September 2005 07:39, Marco Berizzi wrote:

> the source IP is set by
> iproute2 in the routing tables.

Ok, I'm not clearly explained.
I'm using SNAT to change the source ip address because iproute2 set an
ip address that I don't like.
So, I have inserted the following line in my firewall script:

iptables -t nat -I POSTROUTING -s iproute2_chosen_ip --protocol tcp
-j SNAT --to i_want_this_first_ip --to i_want_this_second_ip

But everytime I connect to the internet always "i_want_this_first_ip" is
chosen to
do the SNAT.

RE: snat to multiple source ip

[Derick Anderson]

 

> -----Original Message-----
> From: netfilter-bounces@lists.netfilter.org 
> [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of 
> Marco Berizzi
> Sent: Tuesday, September 20, 2005 10:24 AM
> To: netfilter@lists.netfilter.org
> Subject: Re: snat to multiple source ip
> 
> /dev/rob0 wrote:
> 
> > On Tuesday 20 September 2005 07:39, Marco Berizzi wrote:
> 
> > the source IP is set by
> > iproute2 in the routing tables.
> 
> Ok, I'm not clearly explained.
> I'm using SNAT to change the source ip address because 
> iproute2 set an ip address that I don't like.
> So, I have inserted the following line in my firewall script:
> 
> iptables -t nat -I POSTROUTING -s iproute2_chosen_ip 
> --protocol tcp -j SNAT --to i_want_this_first_ip --to 
> i_want_this_second_ip
> 
> But everytime I connect to the internet always 
> "i_want_this_first_ip" is chosen to do the SNAT.

Uneducated, lazy guess: the SNAT target supports only one "--to [IP]"
and the command parser ignores the extraneous "--to [IP2]". You can test
this by switching the two IPs and see if you go out on a different
address.

Derick Anderson

Re: snat to multiple source ip

[Marco Berizzi]

Derick Anderson wrote:

> Uneducated, lazy guess: the SNAT target supports only one "--to [IP]"
> and the command parser ignores the extraneous "--to [IP2]". You can
test
> this by switching the two IPs and see if you go out on a different
> address.

Nope!

From "man iptables":

"...You  can  add  several --to-source options.  If you
specify more than one source address, either
via an address range or multiple --to-source options, a
simple round-robin (one after another in
cycle) takes place between these adresses."

Re: snat to multiple source ip

[Edmundo Carmona]

The problem for you is that you have a single default gateway:
default via hdsl_default_gateway_ip dev eth0  metric 1

In case you want to use more than one device to route traffic, here's
ours at the office (output taken from ip route show default):

default
        nexthop via 201.249.120.1  dev eth1 weight 2
        nexthop via 200.109.64.1  dev eth2 weight 1

It requires more than just adding one multipath default route. You
have to set different routing tables for each interface you want to
use... and please, make sure none of them is present in the same
subnetwork (that cause the multipath to NOT war at all... I'm waiting
to one grateful day be accepted in the LARTC mail list to ask about
this).

Anyway.... here's the multipath simplest guide:
http://lartc.org/howto/lartc.rpdb.multiple-links.html

and also, look here:
http://linux-ip.net/html/adv-multi-internet.html

On 9/20/05, Derick Anderson <danderson@vikus.com> wrote:
> 
> 
> > -----Original Message-----
> > From: netfilter-bounces@lists.netfilter.org
> > [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of
> > Marco Berizzi
> > Sent: Tuesday, September 20, 2005 10:24 AM
> > To: netfilter@lists.netfilter.org
> > Subject: Re: snat to multiple source ip
> >
> > /dev/rob0 wrote:
> >
> > > On Tuesday 20 September 2005 07:39, Marco Berizzi wrote:
> >
> > > the source IP is set by
> > > iproute2 in the routing tables.
> >
> > Ok, I'm not clearly explained.
> > I'm using SNAT to change the source ip address because
> > iproute2 set an ip address that I don't like.
> > So, I have inserted the following line in my firewall script:
> >
> > iptables -t nat -I POSTROUTING -s iproute2_chosen_ip
> > --protocol tcp -j SNAT --to i_want_this_first_ip --to
> > i_want_this_second_ip
> >
> > But everytime I connect to the internet always
> > "i_want_this_first_ip" is chosen to do the SNAT.
> 
> Uneducated, lazy guess: the SNAT target supports only one "--to [IP]"
> and the command parser ignores the extraneous "--to [IP2]". You can test
> this by switching the two IPs and see if you go out on a different
> address.
> 
> Derick Anderson
> 
>

RE: snat to multiple source ip

[Harrison, Bruce (CXO)]

Hello Marco,

    You're assuming that the man page accurately describes the behavior
of the code (which normally is the case).  But how about doing a quick
test as suggested by Derick just to be sure.  Could be a bug?

Have a Great Day!


Bruce...

-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Marco
Berizzi
Sent: Tuesday, September 20, 2005 9:17 AM
To: netfilter@lists.netfilter.org
Subject: Re: snat to multiple source ip

Derick Anderson wrote:

> Uneducated, lazy guess: the SNAT target supports only one "--to [IP]"
> and the command parser ignores the extraneous "--to [IP2]". You can
test
> this by switching the two IPs and see if you go out on a different
> address.

Nope!

From "man iptables":

"...You  can  add  several --to-source options.  If you
specify more than one source address, either
via an address range or multiple --to-source options, a
simple round-robin (one after another in
cycle) takes place between these adresses."

Re: snat to multiple source ip

[Marco Berizzi]

Harrison, Bruce (CXO) wrote:

> You're assuming that the man page accurately describes the behavior
> of the code (which normally is the case).  But how about doing a quick
> test as suggested by Derick just to be sure.  Could be a bug?

The second ip address sometimes *is* used. My guess is that SNAT
choose the second IP when the system open a second socket to the same
host.
My guess is that SNAT is working as described in the previous post:
1st socket opened to a.a.a.a  -->> SNAT with first_ip
2nd socket opened to b.b.b.b  -->> SNAT with first_ip
3rd socket opened to c.c.c.c  -->> SNAT with first_ip
4th socket opened to b.b.b.b  -->> SNAT with second_ip
5th socket opened to d.d.d.d  -->> SNAT with first_ip
6th socket opened to a.a.a.a  -->> SNAT with second_ip

It should work in this manner:

1st socket opened to a.a.a.a  -->> SNAT with first_ip
2nd socket opened to b.b.b.b  -->> SNAT with second_ip
3rd socket opened to c.c.c.c  -->> SNAT with first_ip
4th socket opened to b.b.b.b  -->> SNAT with second_ip
5th socket opened to d.d.d.d  -->> SNAT with first_ip
6th socket opened to a.a.a.a  -->> SNAT with second_ip

Right?

> Have a Great Day!

thanks!

RE: snat to multiple source ip

[Derick Anderson]

 

> -----Original Message-----
> From: netfilter-bounces@lists.netfilter.org 
> [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of 
> Marco Berizzi
> Sent: Tuesday, September 20, 2005 11:17 AM
> To: netfilter@lists.netfilter.org
> Subject: Re: snat to multiple source ip
> 
> Derick Anderson wrote:
> 
> > Uneducated, lazy guess: the SNAT target supports only one 
> "--to [IP]"
> > and the command parser ignores the extraneous "--to [IP2]". You can
> test
> > this by switching the two IPs and see if you go out on a different 
> > address.
> 
> Nope!
> 
> >From "man iptables":
> 
> "...You  can  add  several --to-source options.  If you 
> specify more than one source address, either via an address 
> range or multiple --to-source options, a simple round-robin 
> (one after another in
> cycle) takes place between these adresses."

Well that's neat, I'll have to try that some time. Might work nicely as
a load balancer where I work.

Derick

snat to multiple source ip

[Marco Berizzi]

Hello everybody.
I would like to better understand the SNAT target when
multiple --to-source are specified.

Man states:

"You can add several --to-source option. If you specify
more than one source address, either via an address range
or multiple --to-source options, a simple round-robin  (one
after another in cycle) takes place between these addresses."

iptables -t nat -I POSTROUTING -s ipX --protocol tcp
-j SNAT --to first_ip --to second_ip

This round robin cycle is per socket. Correct?

But, after a little test, I have concluded that SNAT
choose the second IP when the system open a second
socket to the same host.

SNAT test results:

1st socket opened to a.a.a.a  -->> SNAT with first_ip
2nd socket opened to b.b.b.b  -->> SNAT with first_ip
3rd socket opened to c.c.c.c  -->> SNAT with first_ip
4th socket opened to b.b.b.b  -->> SNAT with second_ip
5th socket opened to d.d.d.d  -->> SNAT with first_ip
6th socket opened to a.a.a.a  -->> SNAT with second_ip

I think it should work in this manner:

1st socket opened to a.a.a.a  -->> SNAT with first_ip
2nd socket opened to b.b.b.b  -->> SNAT with second_ip
3rd socket opened to c.c.c.c  -->> SNAT with first_ip
4th socket opened to b.b.b.b  -->> SNAT with second_ip
5th socket opened to d.d.d.d  -->> SNAT with first_ip
6th socket opened to a.a.a.a  -->> SNAT with second_ip

May anyone confirm this? Wrong or correct?

Re: snat to multiple source ip

[Phil Oester]

On Mon, Sep 26, 2005 at 05:38:48PM +0200, Marco Berizzi wrote:
> I would like to better understand the SNAT target when
> multiple --to-source are specified.

This capability has been removed from newer (2.6.11+) kernels,
and was reported as being broken in kernels prioer to that.

Phil

Re: snat to multiple source ip

[Marco Berizzi]

> On Mon, Sep 26, 2005 at 05:38:48PM +0200, Marco Berizzi wrote:
> > I would like to better understand the SNAT target when
> > multiple --to-source are specified.
>
> This capability has been removed from newer (2.6.11+) kernels,
> and was reported as being broken in kernels prioer to that.

Thanks for the reply. FYI tests were done with linux 2.4.31
Is there any other equivalent under 2.4 or 2.6?

Re: snat to multiple source ip

[Phil Oester]

On Mon, Sep 26, 2005 at 06:39:13PM +0200, Marco Berizzi wrote:
> > On Mon, Sep 26, 2005 at 05:38:48PM +0200, Marco Berizzi wrote:
> > > I would like to better understand the SNAT target when
> > > multiple --to-source are specified.
> >
> > This capability has been removed from newer (2.6.11+) kernels,
> > and was reported as being broken in kernels prior to that.
> 
> Thanks for the reply. FYI tests were done with linux 2.4.31
> Is there any other equivalent under 2.4 or 2.6?

You could try the random match from pom, but to do this without
patching the kernel, probably could use multiple -s rules:

-s 10.0.0.0/24 -j SNAT x.x.x.x
-s 10.1.0.0/24 -j SNAT y.y.y.y

or somesuch.

Phil