IPsecv6 tunnel mode fragmentation

IPsecv6 tunnel mode fragmentation

[Joy Latten]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

We have come across an ipsec problem that I think was
noted a while back in the following link.
http://www.mail-archive.com/netdev@vger.kernel.org/msg61659.html

When an icmpv6 pkt-too-big message for a destination
is received, it is processed and the route's mtu is adjusted.

Transport mode uses "adjusted" mtu and works ok, but tunnel-mode
which has inner and outer iphdrs has problems.

ah and esp leave it to the ipv6 layer to fragment.
So, it seems esp/ah tunnel mode can produce an outgoing ipsec tunnel
mode pkt whose inner pkthdr has the dst with the adjusted mtu,
but inner pkt size larger than the adjusted mtu.

The outer pkthdr has tunnel's dst mtu  which has not been 
adjusted, since the pkt-too-big message was not for it.
So, ipv6 layer will use outer mtu to decide whether to fragment or not.
It doesn't appear to use the inner, "adjusted" mtu.
In the tcpdump attached below, since outer mtu is larger than the
outgoing pkt's size, it is not fragmented.

Thus lies the problem. So when the link with the "adjusted" mtu
gets the decrypted packet, the decrypted pkt may be too large for the
link's mtu. The "adjusted" mtu was never regarded when creating the pkt.

Hopefully, I have explained this clearly, if not
let me know. Should esp/ah pre-fragment... or should mtu of 
inner pkt's dst be used for outer pkt? What's the best way to approach
this? Thanks for any info.

regards,
Joy 

ipsec config:

target <-------> Secuity gateway <-----> host
        (tunnel)

attachment is tcpdump from target.


[-- Attachment #2: target.bin --]
[-- Type: application/octet-stream, Size: 10236 bytes --]

Re: IPsecv6 tunnel mode fragmentation

[Herbert Xu]

Joy Latten <latten@austin.ibm.com> wrote:
> 
> We have come across an ipsec problem that I think was
> noted a while back in the following link.
> http://www.mail-archive.com/netdev@vger.kernel.org/msg61659.html

Looks like a configuration issue to me.  One end is using the
same IP address (*::1234) both within and outside the tunnel.
Thus when the ICMP error message is sent it ends up outside the
tunnel causing it to be discarded by the other side.

So if you're using tunnel mode you really should use distinct
IP addresses.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: IPsecv6 tunnel mode fragmentation

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Wed, 8 Dec 2010 15:11:09 +0800

> Joy Latten <latten@austin.ibm.com> wrote:
>> 
>> We have come across an ipsec problem that I think was
>> noted a while back in the following link.
>> http://www.mail-archive.com/netdev@vger.kernel.org/msg61659.html
> 
> Looks like a configuration issue to me.  One end is using the
> same IP address (*::1234) both within and outside the tunnel.
> Thus when the ICMP error message is sent it ends up outside the
> tunnel causing it to be discarded by the other side.
> 
> So if you're using tunnel mode you really should use distinct
> IP addresses.

Agreed.

Re: IPsecv6 tunnel mode fragmentation

[Joy Latten]

Hi Herbert,

On Wed, 2010-12-08 at 15:11 +0800, Herbert Xu wrote:
> Joy Latten <latten@austin.ibm.com> wrote:
> > 
> > We have come across an ipsec problem that I think was
> > noted a while back in the following link.
> > http://www.mail-archive.com/netdev@vger.kernel.org/msg61659.html
> 
> Looks like a configuration issue to me.  One end is using the
> same IP address (*::1234) both within and outside the tunnel.
> Thus when the ICMP error message is sent it ends up outside the
> tunnel causing it to be discarded by the other side.
> 
> So if you're using tunnel mode you really should use distinct
> IP addresses.
> 

Do you mean the tunnel mode packet has (*::1234) for both the 
inner and outer headers?

I was actually attempting testcase 5.3.11 from
http://w3.antd.nist.gov/usgv6/TSTs/tests/Phase2_IPsec_Interoperability_v1.1.pdf

My net config varied a little though...

2001:db8:1:1::1234       2001:db8:1:1::56    2033:1:...:35  2033:1:...95
TARGET(eth0) <--ipsectnl--> (eth1)SecurityGateway(eth0) <---> (eth0)HOST
       1500                     1500                 1280        1280

To me the test seems a host-to-securitygateway tunnel mode scenario.

In the 5.3.11 testcase, the tunnel endpoint (eth0 of TARGET) is also
endpoint for the pkt.

i.e. SPD entry from 5.3.11 testcase using above config:

Security Policy Database (SPD) for HOST1_SA-1
tunnel source address           SGW_eth1
tunnel destination address      TGT_eth0
source address                  HOST_eth0
destination address             TGT_eth0
upperspec                       any
direction                       in
protocol                        ESP
mode                            tunnel 

We did attempt to debug in kernel to see whether the pkt-too-big 
was being discarded... from what we saw, it did not seem to be. 
Also, an "ip -6  r l c" on TARGET showed the adjusted mtu for the route 
to HOST, which the pkt-too-big was for. tunnel's mtu is untouched.

2001:db8:1:1::56 via 2001:db8:1:1::56 dev eth0  metric 0 
    cache  mtu 1500 advmss 1440 hoplimit 4294967295
2033:1:2:3:209:6bf:fe6:95 via 2001:db8:1:1::56 dev eth0  metric 0 
    cache  expires 592sec mtu 1280 advmss 1440 hoplimit 4294967295

Thus why I reached my original conclusion, that perhaps 
esp/ah disregard inner hdr mtu since does not fragment.
Thus inner pkt will be too big when it reaches link for 
inner/original pkt to be sent on.

If I have misunderstood your response, my apologies. 
Please let me know.

regards,
Joy 

Re: IPsecv6 tunnel mode fragmentation

[Herbert Xu]

On Wed, Dec 08, 2010 at 08:20:42PM -0600, Joy Latten wrote:
>
> We did attempt to debug in kernel to see whether the pkt-too-big 
> was being discarded... from what we saw, it did not seem to be. 
> Also, an "ip -6  r l c" on TARGET showed the adjusted mtu for the route 
> to HOST, which the pkt-too-big was for. tunnel's mtu is untouched.

Oh I see what you're saying.  Sorry I misunderstood the issue
earlier.

> Thus why I reached my original conclusion, that perhaps 
> esp/ah disregard inner hdr mtu since does not fragment.
> Thus inner pkt will be too big when it reaches link for 
> inner/original pkt to be sent on.

Indeed we don't have any fragmentation capbility at all on the
IPsec output path inside the tunnel.  For IPv4 this isn't an issue
because the other end can perform fragmentation.  This is clearly
not an option with IPv6.

I'll look into this.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt