snat and ICMP question

snat and ICMP question

[Ciaran Deignan]

Hi All,

I have a problem with ICMP (destination-unreachable /
fragmentation needed) packets not bieng NAT-ed correctly
with a specific SNAT configuration.

I have an IPsec tunnel (frees/wan), and I need to source-nat
everything that comes out of the tunnel (strange routing
problem).

I'm using the following configuration

iptables -t mangle -A PREROUTING --in-interface ipsec+ \
                        -j MARK --set-mark 1
iptables -t nat -A POSTROUTING -m mark --mark 1 \
                                -j MASQUERADE

However the IPsec tunnel had an MTU of 1400, slightly less 
than the ehternet packet. When a user requests a large
web page (for example), the web server send big packets,
and an ICMP error is generated by the NAT-ing node. However
the ICMP packet contains the real destination address, not
the address of the NAT device... The web server ignores the
ICMP error, which is normal.

I'm using iptables v1.2.5. I've noticed that there is
a DNAT / ICMP correction in 1.2.7a. I'm going to download
and test this new version, but I suspect that the behaviour
will be the same.

Has anybody previously encountered this sort of problem?
How can I tell iptables to NAT inside ICMP packets that are
generated locally but that concern connections coming
from the tunnel? I suppose it would work if I just
masquraded everything going through any interface,
but that seems a bit drastic...

Thanks for any pointers,
Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

Re: snat and ICMP question

[Ciaran Deignan]

Hi again,

> I'm using iptables v1.2.5. I've noticed that there is
> a DNAT / ICMP correction in 1.2.7a. I'm going to download
> and test this new version, but I suspect that the behaviour
> will be the same.

I tried with the latest version, but there is no difference.

I still can't see any way of configuring iptables to do
what I need. Even if the original source address was
known, how could I specifically tell iptables to mangle
the address inside the ICMP packet? Maybe its a bug?

Thanks for any advice,
Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

snat and ICMP question

[Ciaran Deignan]

Hi All,

I asked this question on the normal netfilter mailing-list.
Either I outlined the problem badly, or I'm beyond help :)

I have a problem with ICMP (destination-unreachable /
fragmentation needed) packets not bieng NAT-ed correctly
with a specific SNAT configuration.

I have an IPsec tunnel (frees/wan), and I need to source-nat
everything that comes out of the tunnel (strange routing
problem).

I'm using the following configuration

iptables -t mangle -A PREROUTING --in-interface ipsec+ \
                        -j MARK --set-mark 1
iptables -t nat -A POSTROUTING -m mark --mark 1 \
                                -j MASQUERADE

However the IPsec tunnel had an MTU of 1400, slightly less 
than the ethernet packet. When a user requests a large
web page (for example), the web server send big packets,
and an ICMP error is generated by the NAT-ing node. However
the ICMP packet contains the real destination address, not
the address of the NAT device... The web server ignores the
ICMP error, which is normal.

I'm using iptables v1.2.7a with a 2.4.17 kernel. I haven't
tried anding any additional patches.

Has anybody previously encountered this sort of problem?
How can I tell iptables to NAT inside ICMP packets that are
generated locally but that concern connections coming
from the tunnel? I suppose it would work if I just
masquraded everything going through any interface,
but that seems a bit drastic...

Thanks for any pointers,
Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

Re: snat and ICMP question

[Marian Stagarescu]

> I'm using the following configuration
>
> iptables -t mangle -A PREROUTING --in-interface ipsec+ \
>                         -j MARK --set-mark 1
> iptables -t nat -A POSTROUTING -m mark --mark 1 \
>                                 -j MASQUERADE

i think that the icmp packet is not marked hence not masqueraded 
(it is locally generated by the ip stack (above ipsec anyways) hence
does not follow rule 1 mangle above).

is this your config ?

      
                    SNAT
             ------------------->
ipsec-interface         public_interface-------WEB-SERVER
     mtu 1400             mtu 1500
                                       |               1500
                                       |         <-------------      
                                       |                       
        ICMP frag required     |
        (OUTPUT) ------------+

> However the IPsec tunnel had an MTU of 1400, slightly less
> than the ethernet packet. When a user requests a large
> web page (for example), the web server send big packets,
> and an ICMP error is generated by the NAT-ing node. However
> the ICMP packet contains the real destination address, not
> the address of the NAT device... The web server ignores the
> ICMP error, which is normal.
>
> I'm using iptables v1.2.7a with a 2.4.17 kernel. I haven't
> tried anding any additional patches.
>
> Has anybody previously encountered this sort of problem?
> How can I tell iptables to NAT inside ICMP packets that are
> generated locally but that concern connections coming
> from the tunnel? I suppose it would work if I just
> masquraded everything going through any interface,
> but that seems a bit drastic...
>
> Thanks for any pointers,
> Ciaran

Re: snat and ICMP question

[Ciaran Deignan]

Marian Stagarescu a écrit :
> 
> > I'm using the following configuration
> >
> > iptables -t mangle -A PREROUTING --in-interface ipsec+ \
> >                         -j MARK --set-mark 1
> > iptables -t nat -A POSTROUTING -m mark --mark 1 \
> >                                 -j MASQUERADE
> 
> i think that the icmp packet is not marked hence not masqueraded
> (it is locally generated by the ip stack (above ipsec anyways) hence
> does not follow rule 1 mangle above).
> 
> is this your config ?
> 
> 
>                     SNAT
>              ------------------->
> ipsec-interface         public_interface-------WEB-SERVER
>      mtu 1400             mtu 1500
>                                        |               1500
>                                        |         <-------------
>                                        |
>         ICMP frag required     |
>         (OUTPUT) ------------+

The ASCII art is a bit messes up, but yes, this is my
config. Actually, the web server is on the private interface.

The ICMP error message contains the real address of the
remote user. Do you think I should mark ICMP messages
on the OUTPUT chain? I must try that...

Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

Re: snat and ICMP question

[Ciaran Deignan]

> > > iptables -t mangle -A PREROUTING --in-interface ipsec+ \
> > >                         -j MARK --set-mark 1
> > > iptables -t nat -A POSTROUTING -m mark --mark 1 \
> > >                                 -j MASQUERADE
> >

> config. Actually, the web server is on the private interface.
> 
> The ICMP error message contains the real address of the
> remote user. Do you think I should mark ICMP messages
> on the OUTPUT chain? I must try that...

I tried adding the following rule, but that didn't change
anything:

	iptables -t mangle -A OUTPUT -p ICMP -j MARK --set-mark 1

Is there any way I could cause the ICMP error to be
generated before the packet is de-nated?

Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

Re: snat and ICMP question

[Marian Stagarescu]

for a quick test you can try to masquerade all and see if the error persists:

iptables -t nat -A POSTROUTING -o outgoing-interface -j MASQUERADE.

can you post the icmp error message ?

an ICMP does have an outter ip hdr and an inner ip dhr
the inner ip hdr is the ip hdr of the dgram that caused the error, 
hence should be the web server ip.
the outer ip hdr should be NATed address if you do the above masquerading.

marian



On Wednesday 09 October 2002 07:16 am, Ciaran Deignan wrote:
> > > > iptables -t mangle -A PREROUTING --in-interface ipsec+ \
> > > >                         -j MARK --set-mark 1
> > > > iptables -t nat -A POSTROUTING -m mark --mark 1 \
> > > >                                 -j MASQUERADE
> >
> > config. Actually, the web server is on the private interface.
> >
> > The ICMP error message contains the real address of the
> > remote user. Do you think I should mark ICMP messages
> > on the OUTPUT chain? I must try that...
>
> I tried adding the following rule, but that didn't change
> anything:
>
> 	iptables -t mangle -A OUTPUT -p ICMP -j MARK --set-mark 1
>
> Is there any way I could cause the ICMP error to be
> generated before the packet is de-nated?
>
> Ciaran

Re: snat and ICMP question

[Ciaran Deignan]

Hi Marian,

Marian Stagarescu a écrit :
> 
> for a quick test you can try to masquerade all and see if the error persists:
> 
> iptables -t nat -A POSTROUTING -o outgoing-interface -j MASQUERADE.

OK I tried with
	iptables -t nat -A POSTROUTING --out-interface eth0 \
                       -j MASQUERADE
but the result is the same.


> can you post the icmp error message ?

OK, first some details...
The remote user is a Windows-2000 PC, whose IP address is
192.168.1.4. The IPsec gateway has an internal address of
10.15.1.111. The remote user is accessing an internal
web server with the address 10.15.1.105. The TCP connection
is being S-NAT-ed to take the internal address of the
gateway. 

Here's a fragment of the trace generated by
	# tcpdump -i eth0 -lnt host 10.15.1.105

10.15.1.105.80 > 10.15.1.111.2483: tcp 399 (DF)
10.15.1.111.2483 > 10.15.1.105.80: tcp 393 (DF)
10.15.1.105.80 > 10.15.1.111.2483: tcp 0 (DF)
10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
(mtu 1400) [tos 0xc0] 
10.15.1.105.80 > 10.15.1.111.2483: tcp 1223 (DF)
10.15.1.111.2483 > 10.15.1.105.80: tcp 0 (DF)
10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
(mtu 1400) [tos 0xc0] 
10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
(mtu 1400) [tos 0xc0] 
10.15.1.111.2483 > 10.15.1.105.80: tcp 0 (DF)

The remote user requests a large page from the web server,
which sends it back with the "don't fragment" bit set, in
1500-byte blocks. This packet is de-NAT-ed and routed to the
IPsec tunnel. The IPsec tunnel has an MTU size of 1400, for 
compatibility with an IPsec feature called "NAT Traversal".
The ICMP error is generated by the gateway, but contains the 
IP address of the remote PC...

The web server (linux) sees an ICMP-destinataion unreachable, 
coming from the gateway but concerning the remote PC's address.
Since it does not believe that it has a connection with 192.168.1.4,
it ignores the error...

I can send you a powerpoint presentation if you're interested in 
the general IPsec networking environments...

> an ICMP does have an outter ip hdr and an inner ip dhr
> the inner ip hdr is the ip hdr of the dgram that caused the error,
> hence should be the web server ip.

Humm, maybe you wanted an iptables-style "LOG" to see the
full contents of the icmp error? Is the tcpdump info not enough?

> the outer ip hdr should be NATed address if you do the above masquerading.

its the address on the header inside the ICMP "payload"
that needs to be re-NAT-ed.

I'm surprised this problem hasn't appeared in a more
normal environment, or with other tunneling environments.

any ideas?
Ciaran

-- 
+---------------------------------------------------------+
Ciaran Deignan                              04 38 49 87 27

Netcelo SA - IPsec VPN Solutions    http://www.netcelo.com/
18-20 rue Henri Barbusse - BP 2501, 38035 Grenoble Cedex 2
+---------------------------------------------------------+

Re: snat and ICMP question

[Balazs Scheidler]

On Thu, Oct 10, 2002 at 12:23:00PM +0200, Ciaran Deignan wrote:
> Marian Stagarescu a écrit :
> > 
> > for a quick test you can try to masquerade all and see if the error persists:
> > 
> > iptables -t nat -A POSTROUTING -o outgoing-interface -j MASQUERADE.
> 
> OK I tried with
> 	iptables -t nat -A POSTROUTING --out-interface eth0 \
>                        -j MASQUERADE
> but the result is the same.
> 
> 
> > can you post the icmp error message ?
> 
> OK, first some details...
> The remote user is a Windows-2000 PC, whose IP address is
> 192.168.1.4. The IPsec gateway has an internal address of
> 10.15.1.111. The remote user is accessing an internal
> web server with the address 10.15.1.105. The TCP connection
> is being S-NAT-ed to take the internal address of the
> gateway. 
> 
> Here's a fragment of the trace generated by
> 	# tcpdump -i eth0 -lnt host 10.15.1.105
> 
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 399 (DF)
> 10.15.1.111.2483 > 10.15.1.105.80: tcp 393 (DF)
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 0 (DF)
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
> 10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
> (mtu 1400) [tos 0xc0] 
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 1223 (DF)
> 10.15.1.111.2483 > 10.15.1.105.80: tcp 0 (DF)
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
> 10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
> (mtu 1400) [tos 0xc0] 
> 10.15.1.105.80 > 10.15.1.111.2483: tcp 1460 (DF)
> 10.15.1.111 > 10.15.1.105: icmp: 192.168.1.4 unreachable - need to frag
> (mtu 1400) [tos 0xc0] 
> 10.15.1.111.2483 > 10.15.1.105.80: tcp 0 (DF)
> 
> The remote user requests a large page from the web server,
> which sends it back with the "don't fragment" bit set, in
> 1500-byte blocks. This packet is de-NAT-ed and routed to the
> IPsec tunnel. The IPsec tunnel has an MTU size of 1400, for 
> compatibility with an IPsec feature called "NAT Traversal".
> The ICMP error is generated by the gateway, but contains the 
> IP address of the remote PC...
> 
> The web server (linux) sees an ICMP-destinataion unreachable, 
> coming from the gateway but concerning the remote PC's address.
> Since it does not believe that it has a connection with 192.168.1.4,
> it ignores the error...
> 
> I can send you a powerpoint presentation if you're interested in 
> the general IPsec networking environments...
> 
> > an ICMP does have an outter ip hdr and an inner ip dhr
> > the inner ip hdr is the ip hdr of the dgram that caused the error,
> > hence should be the web server ip.
> 
> Humm, maybe you wanted an iptables-style "LOG" to see the
> full contents of the icmp error? Is the tcpdump info not enough?
> 
> > the outer ip hdr should be NATed address if you do the above masquerading.
> 
> its the address on the header inside the ICMP "payload"
> that needs to be re-NAT-ed.
> 
> I'm surprised this problem hasn't appeared in a more
> normal environment, or with other tunneling environments.
> 
> any ideas?
> Ciaran

I think the issue you are having is the same that I came accross with my
transparent proxy patches. ICMP nat is done in two steps, once the external
header is translated and later the internal one. (IIRC: the external IP
headers is translated in PREROUTING time, and the internal in POSTROUTING
time).

I've sent a patch, I include below which fixed the problem, but has not been
accepted (nor commented) by Netfilter people.

(the patch is slightly edited, due to a name change which I removed so that
you can apply to plain 2.4.18, you may have some offsets though)

diff -urN --exclude-from kernel-exclude linux-2.4.18-vanilla/net/ipv4/netfilter/ip_nat_core.c linux-2.4.18-cttproxy/net/ipv4/netfilter/ip_nat_core.c
--- linux-2.4.18-vanilla/net/ipv4/netfilter/ip_nat_core.c       Mon Aug  5 16:08:06 2002
+++ linux-2.4.18-cttproxy/net/ipv4/netfilter/ip_nat_core.c      Mon Aug  5 17:22:26 2002
@@ -830,27 +845,27 @@
                   packet, except it was never src/dst reversed, so
                   where we would normally apply a dst manip, we apply
                   a src, and vice versa. */
-               if (info->manips[i].hooknum == opposite_hook[hooknum]) {
+               if (info->manips[i].hooknum == hooknum) {
                        DEBUGP("icmp_reply: inner %s -> %u.%u.%u.%u %u\n",
                               info->manips[i].maniptype == IP_NAT_MANIP_SRC
                               ? "DST" : "SRC",
                               NIPQUAD(info->manips[i].manip.ip),
                               ntohs(info->manips[i].manip.u.udp.port));
-                       manip_pkt(inner->protocol, inner,
+                       manip_pkt(inner->protocol, inner,
                                  skb->len - ((void *)inner - (void *)iph),
                                  &info->manips[i].manip,
                                  !info->manips[i].maniptype,
                                  &skb->nfcache);
-               /* Outer packet needs to have IP header NATed like
-                   it's a reply. */
-               } else if (info->manips[i].hooknum == hooknum) {
+                       /* Outer packet needs to have IP header NATed like
+                          it's a reply. */
+
                        /* Use mapping to map outer packet: 0 give no
                            per-proto mapping */
                        DEBUGP("icmp_reply: outer %s -> %u.%u.%u.%u\n",
                               info->manips[i].maniptype == IP_NAT_MANIP_SRC
                               ? "SRC" : "DST",
                               NIPQUAD(info->manips[i].manip.ip));
-                       manip_pkt(0, iph, skb->len,
+                       manip_pkt(0, iph, skb->len,
                                  &info->manips[i].manip,
                                  info->manips[i].maniptype,
                                  &skb->nfcache);


-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1