Odd issue with two SNATed Firewalls and Wireless router

Odd issue with two SNATed Firewalls and Wireless router

[Andrew Gargan]

Hi All

have an issue with a network setup. 

the issue is this some client machines on my network (10 of +- 150) keep
losing their connection to our mail server or any other mail server we
set up their accounts on.

This is an issue with my network because as soon as they are on dialup
or iburst or ADSL elsewhere they don't have this issue.

I have a 10.0.0.0/8 network which is my internal net.
this net routes traffic through 10.0.2.1 (SNAT) which is connected to my
DMZ 172.16.0.0/16.
the traffic is then sent out to the world via 172.16.0.1(SNAT) over
sentech mywireless.

I have isolated the issue to being on the 172.16.0.1 machine since a
ISDN leased line in the DMZ works 100%.

Has anyone experienced similar issues using a shared NATed mywireless
.... 

most of the mail comes down .... it seems to break when transmissions
are over +-600 KB)

I was told that changing the MTU for the ppp0 device to 1300 would help
but no luck there.

Also one strange thing I noticed (though I don't know TCP/IP that well)

was the txqueuelength value of 3 for the ppp0 interface

here is my kernel: 2.6.9-1.667 (FC3)

here is ifconfig:
eth0      Link encap:Ethernet  HWaddr 00:03:47:71:7B:36
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::203:47ff:fe71:7b36/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27879849 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26767743 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:669378689 (638.3 MiB)  TX bytes:3461641354 (3.2 GiB)

eth1      Link encap:Ethernet  HWaddr 00:03:47:71:7B:37
          inet addr:10.0.7.2  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::203:47ff:fe71:7b37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27333550 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28013971 errors:1 dropped:0 overruns:0 carrier:1
          collisions:614337 txqueuelen:1000
          RX bytes:3798771770 (3.5 GiB)  TX bytes:845067479 (805.9 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:13651 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13651 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:556916 (543.8 KiB)  TX bytes:556916 (543.8 KiB)

ppp0      Link encap:Point-to-Point Protocol
          inet addr:196.35.170.61  P-t-P:66.18.87.50 
Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1400  Metric:1
          RX packets:7595399 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7622079 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:1453866192 (1.3 GiB)  TX bytes:2513716398 (2.3 GiB)


and iptables -L:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DROP       tcp  --  anywhere             anywhere            tcp
flags:SYN,RST,ACK/SYN
DROP       tcp  --  anywhere             anywhere            tcp
flags:SYN,RST,ACK/SYN

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
TCPMSS     tcp  --  anywhere             anywhere            tcp
flags:SYN,RST/SYN TCPMSS clamp to PMTU
ACCEPT     all  --  anywhere             anywhere            state
RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination


and mii-tool:
eth0: negotiated 100baseTx-FD, link ok
eth1: no autonegotiation, 10baseT-HD, link ok

eth1 is the ether used to connect to the mywireless ... for routing
purposes the ip it has assigned isn't really used ... 

I am using rp-pppoe I think ... 

Andrew Gargan
Developer
Interface Media (PTY) Ltd.
Tel: 011 507 3003

Re: Odd issue with two SNATed Firewalls and Wireless router

[Taylor, Grant]

> most of the mail comes down .... it seems to break when transmissions
> are over +-600 KB)

Try using the TCPMSS target to limit the MSS (Maximum (TCP) Segment Size) value to be at least as small as your path MSS.

Hint:

iptables -t filter -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu



Grant. . . .

Re: Odd issue with two SNATed Firewalls and Wireless router

[ISC Jorge Ceron Galvan]

I've been dealing with the same problem for a while and I thougth it was not a routing 
problem, but after I read your mail I don't know.

I'll tell you what I've done just to see if we can find out a solution.

I'm not doing nat because I want a real IP at my wireless client.

cisco eth0  200.0.0.1/24

linux eth0  200.0.0.10/24  gw 200.0.0.1/24

linux eth1  200.0.0.129/27

remote fortinet router eth0 200.0.0.130/27   gw 200.0.0.129

I thougth it could be a routing problem because I'm using subnet 0 at eth0, and at eth1 I 
set up a subnet from eth0. It's not an iptables issue because I'm not using it at all. The 
configuration is quite simple, but I don't know whether you can subnet a class C net this 
way. The mail server we are trying to reach is somewhere in the internet. 

I'm using a wireless AP at my side and a wireless bridge at the other side; the bridge is 
connected directly to the fortinet router. Could it be a protocol bridge problem?

Any Idea?  ( I apologise for my english )

-----Original Message-----
From: Andrew Gargan <andrew@iface.co.za>
To: netfilter@lists.netfilter.org
Date: Wed, 24 Aug 2005 14:30:44 +0200
Subject: Odd issue with two SNATed Firewalls and Wireless router

> Hi All
> 
> have an issue with a network setup. 
> 
> the issue is this some client machines on my network (10 of +- 150)
> keep
> losing their connection to our mail server or any other mail server
> we
> set up their accounts on.
> 
> This is an issue with my network because as soon as they are on
> dialup
> or iburst or ADSL elsewhere they don't have this issue.
> 
> I have a 10.0.0.0/8 network which is my internal net.
> this net routes traffic through 10.0.2.1 (SNAT) which is connected to
> my
> DMZ 172.16.0.0/16.
> the traffic is then sent out to the world via 172.16.0.1(SNAT) over
> sentech mywireless.
> 
> I have isolated the issue to being on the 172.16.0.1 machine since a
> ISDN leased line in the DMZ works 100%.
> 
> Has anyone experienced similar issues using a shared NATed mywireless
> .... 
> 
> most of the mail comes down .... it seems to break when transmissions
> are over +-600 KB)
> 
> I was told that changing the MTU for the ppp0 device to 1300 would
> help
> but no luck there.
> 
> Also one strange thing I noticed (though I don't know TCP/IP that
> well)
> 
> was the txqueuelength value of 3 for the ppp0 interface
> 
> here is my kernel: 2.6.9-1.667 (FC3)
> 
> here is ifconfig:
> eth0      Link encap:Ethernet  HWaddr 00:03:47:71:7B:36
>           inet addr:172.16.0.1  Bcast:172.16.0.255 
> Mask:255.255.255.0
>           inet6 addr: fe80::203:47ff:fe71:7b36/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:27879849 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:26767743 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:669378689 (638.3 MiB)  TX bytes:3461641354 (3.2
> GiB)
> 
> eth1      Link encap:Ethernet  HWaddr 00:03:47:71:7B:37
>           inet addr:10.0.7.2  Bcast:10.255.255.255  Mask:255.0.0.0
>           inet6 addr: fe80::203:47ff:fe71:7b37/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:27333550 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:28013971 errors:1 dropped:0 overruns:0 carrier:1
>           collisions:614337 txqueuelen:1000
>           RX bytes:3798771770 (3.5 GiB)  TX bytes:845067479 (805.9
> MiB)
> 
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:13651 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:13651 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:556916 (543.8 KiB)  TX bytes:556916 (543.8 KiB)
> 
> ppp0      Link encap:Point-to-Point Protocol
>           inet addr:196.35.170.61  P-t-P:66.18.87.50 
> Mask:255.255.255.255
>           UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1400  Metric:1
>           RX packets:7595399 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:7622079 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:3
>           RX bytes:1453866192 (1.3 GiB)  TX bytes:2513716398 (2.3
> GiB)
> 
> 
> and iptables -L:
> 
> Chain INPUT (policy ACCEPT)
> target     prot opt source               destination
> ACCEPT     all  --  anywhere             anywhere
> ACCEPT     all  --  anywhere             anywhere
> DROP       tcp  --  anywhere             anywhere            tcp
> flags:SYN,RST,ACK/SYN
> DROP       tcp  --  anywhere             anywhere            tcp
> flags:SYN,RST,ACK/SYN
> 
> Chain FORWARD (policy ACCEPT)
> target     prot opt source               destination
> TCPMSS     tcp  --  anywhere             anywhere            tcp
> flags:SYN,RST/SYN TCPMSS clamp to PMTU
> ACCEPT     all  --  anywhere             anywhere            state
> RELATED,ESTABLISHED
> 
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
> 
> 
> and mii-tool:
> eth0: negotiated 100baseTx-FD, link ok
> eth1: no autonegotiation, 10baseT-HD, link ok
> 
> eth1 is the ether used to connect to the mywireless ... for routing
> purposes the ip it has assigned isn't really used ... 
> 
> I am using rp-pppoe I think ... 
> 
> Andrew Gargan
> Developer
> Interface Media (PTY) Ltd.
> Tel: 011 507 3003

Re: Odd issue with two SNATed Firewalls and Wireless router

[/dev/rob0]

On Wednesday 2005-August-24 13:43, ISC Jorge Ceron Galvan wrote:
> I'm not doing nat because I want a real IP at my wireless client.
>
> cisco eth0  200.0.0.1/24

What a great netblock that is! Uh, this *is* your real IP, or did you 
munge it for posting? It's not the same as what appears in your mail 
headers.

If you're going to munge IP addresses, you should not use a live 
netblock. Pick something from RFC 1918 or an unassigned (bogon) range.

> linux eth0  200.0.0.10/24  gw 200.0.0.1/24

/24 covers 200.0.0.0 (the network address) through 200.0.0.255 (the 
broadcast address.)

> linux eth1  200.0.0.129/27

And this is included in the eth0 network. Perhaps you should use /25 
netmask or greater on eth0.

> remote fortinet router eth0 200.0.0.130/27   gw 200.0.0.129

I don't know what this means.

> I thougth it could be a routing problem because I'm using subnet 0 at
> eth0, and at eth1 I set up a subnet from eth0.

Yes, that is a part of the problem, I would think.

> It's not an iptables 
> issue because I'm not using it at all. The configuration is quite

This is the netfilter list, so you're off topic here.

> simple, but I don't know whether you can subnet a class C net this

I don't know either. I generally find that doing things the right way 
works better. ;)

> way. The mail server we are trying to reach is somewhere in the
> internet.
>
> I'm using a wireless AP at my side and a wireless bridge at the other
> side; the bridge is connected directly to the fortinet router. Could
> it be a protocol bridge problem?

1. Check the routing
2. Check the routing
3. Check the routing
4. Look at packet counters, is eth1 being used at all?

replying to the OP as well:
> -----Original Message-----
> From: Andrew Gargan <andrew@iface.co.za>
snip
> > Has anyone experienced similar issues using a shared NATed
> > mywireless ....
> >
> > most of the mail comes down .... it seems to break when
> > transmissions are over +-600 KB)
> >
> > I was told that changing the MTU for the ppp0 device to 1300 would
> > help but no luck there.

It does sound like a possible router MTU issue. It does not sound like 
iptables/netfilter is involved.

> > eth1      Link encap:Ethernet  HWaddr 00:03:47:71:7B:37
> >           inet addr:10.0.7.2  Bcast:10.255.255.255  Mask:255.0.0.0
> >           inet6 addr: fe80::203:47ff:fe71:7b37/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:27333550 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:28013971 errors:1 dropped:0 overruns:0
> > carrier:1 collisions:614337 txqueuelen:1000

That is a lot of collisions. It might not indicate a problem, but 
likewise, it might.

> > and iptables -L:

... is utterly useless. "iptables -vL" is better, but iptables-save(8) 
is greatly preferred.

That said, nothing indicates the likelihood of a problem with your 
iptables rules.

> > I am using rp-pppoe I think ...

You think?
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: Odd issue with two SNATed Firewalls and Wireless router

[ISC Jorge Ceron Galvan]

No, that's not my real net.

let's say my real net is 1.2.3.0/24, so I have this

cisco sync0    5.6.7.9/30    (internet)
cisco eth0     1.2.3.1/24
linux eth0     1.2.3.10/24   gw 1.2.3.1

linux eth1     1.2.3.129/27
remote router  1.2.3.130/27  gw 1.2.3.129

as I understood I should be doing something like this

cisco sync0    5.6.7.9/30    (internet)
cisco eth0     1.2.3.1/27
linux eth0     1.2.3.10/27   gw 1.2.3.1

linux eth1     1.2.3.129/27
remote router  1.2.3.130/27  gw 1.2.3.129

I told you I was not using iptables because I didn't think it was part of the problem but, 
as a matter of fact, I do. I'm doing nat on eth0 using (again) 1.2.3.17/24 so linux eth0 was

linux eth0     1.2.3.10/24
linux eth0     1.2.3.17/24 secondary

and as I you say, this worst things, thouhgt; finally the right configuration would be:

cisco sync0    5.6.7.8/30    (internet)
cisco eth0     1.2.3.1/27
linux eth0     1.2.3.10/27   gw 1.2.3.1
linux eth0     1.2.3.17/27   secondary

linux eth1     1.2.3.129/27
remote router  1.2.3.130/27  gw 1.2.3.129

iptables -t nat -A POSTROUTING -o eth0 -s 10.10.2.0/24 -j SNAT --to 1.2.3.17


right? is this simplier? do you think using /24 at eth0 could be affecting the smtp?

the main problem is at 1.2.3.130 : this is a small router doing nat using 1.2.3.130, but the 
pc's behind it could´t reach an internet mail server


thx !!!!


-----Original Message-----
From: /dev/rob0 <rob0@gmx.co.uk>
To: netfilter@lists.netfilter.org
Date: Wed, 24 Aug 2005 15:47:18 -0500
Subject: Re: Odd issue with two SNATed Firewalls and Wireless router

> On Wednesday 2005-August-24 13:43, ISC Jorge Ceron Galvan wrote:
> > I'm not doing nat because I want a real IP at my wireless client.
> >
> > cisco eth0  200.0.0.1/24
> 
> What a great netblock that is! Uh, this *is* your real IP, or did you
> munge it for posting? It's not the same as what appears in your mail 
> headers.
> 
> If you're going to munge IP addresses, you should not use a live 
> netblock. Pick something from RFC 1918 or an unassigned (bogon)
> range.
> 
> > linux eth0  200.0.0.10/24  gw 200.0.0.1/24
> 
> /24 covers 200.0.0.0 (the network address) through 200.0.0.255 (the 
> broadcast address.)
> 
> > linux eth1  200.0.0.129/27
> 
> And this is included in the eth0 network. Perhaps you should use /25 
> netmask or greater on eth0.
> 
> > remote fortinet router eth0 200.0.0.130/27   gw 200.0.0.129
> 
> I don't know what this means.
> 
> > I thougth it could be a routing problem because I'm using subnet 0
> at
> > eth0, and at eth1 I set up a subnet from eth0.
> 
> Yes, that is a part of the problem, I would think.
> 
> > It's not an iptables 
> > issue because I'm not using it at all. The configuration is quite
> 
> This is the netfilter list, so you're off topic here.
> 
> > simple, but I don't know whether you can subnet a class C net this
> 
> I don't know either. I generally find that doing things the right way
> works better. ;)
> 
> > way. The mail server we are trying to reach is somewhere in the
> > internet.
> >
> > I'm using a wireless AP at my side and a wireless bridge at the
> other
> > side; the bridge is connected directly to the fortinet router.
> Could
> > it be a protocol bridge problem?
> 
> 1. Check the routing
> 2. Check the routing
> 3. Check the routing
> 4. Look at packet counters, is eth1 being used at all?
> 
> replying to the OP as well:
> > -----Original Message-----
> > From: Andrew Gargan <andrew@iface.co.za>
> snip
> > > Has anyone experienced similar issues using a shared NATed
> > > mywireless ....
> > >
> > > most of the mail comes down .... it seems to break when
> > > transmissions are over +-600 KB)
> > >
> > > I was told that changing the MTU for the ppp0 device to 1300
> would
> > > help but no luck there.
> 
> It does sound like a possible router MTU issue. It does not sound
> like 
> iptables/netfilter is involved.
> 
> > > eth1      Link encap:Ethernet  HWaddr 00:03:47:71:7B:37
> > >           inet addr:10.0.7.2  Bcast:10.255.255.255 
> Mask:255.0.0.0
> > >           inet6 addr: fe80::203:47ff:fe71:7b37/64 Scope:Link
> > >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> > >           RX packets:27333550 errors:0 dropped:0 overruns:0
> frame:0
> > >           TX packets:28013971 errors:1 dropped:0 overruns:0
> > > carrier:1 collisions:614337 txqueuelen:1000
> 
> That is a lot of collisions. It might not indicate a problem, but 
> likewise, it might.
> 
> > > and iptables -L:
> 
> ... is utterly useless. "iptables -vL" is better, but
> iptables-save(8) 
> is greatly preferred.
> 
> That said, nothing indicates the likelihood of a problem with your 
> iptables rules.
> 
> > > I am using rp-pppoe I think ...
> 
> You think?
> -- 
>     mail to this address is discarded unless "/dev/rob0"
>     or "not-spam" is in Subject: header