TCPMSS clamp-mss-to-pmtu breaks packets that have no MSS set (incl. solution)

TCPMSS clamp-mss-to-pmtu breaks packets that have no MSS set (incl. solution)

[Daniel Gibson]

Hi,

clamp-mss-to-pmtu (as used in "iptables -A FORWARD -p tcp --tcp-flags 
SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu") sets a possibly much to high 
MSS (the one it got from PMTU) in packets that had no MSS set before.
RFC 879 says "HOSTS MUST NOT SEND DATAGRAMS LARGER THAN 576 OCTETS 
UNLESS THEY HAVE SPECIFIC KNOWLEDGE THAT THE DESTINATION HOST IS 
PREPARED TO ACCEPT LARGER DATAGRAMS."
That means that if no MSS was set by a server, one must not send packets 
with a MSS > 536 to that server.
clamp-mss-to-pmtu however ignores this and, if no MSS was set in a 
packet, sets the MSS determined by PMTU, often something like 1452, so 
much to big packets are sent to the host that conforms to the RFC but, 
braindead as it is, blocks "ICMP Fragmentation needed" packets.
So the connection times out after some retries that are all dropped on 
the remote firewall or whatever because the packets are to big.
By the way, this bug seems to be present since kernel 2.4.3 (when TCPMSS 
was introduced).

The solution: If no MSS was set in the packet and "newmss", as 
determined by PMTU, is > 536, set "newmss" to 536 before applying it to 
the packet.
I think this is the only sane fix:
* Leaving it as it is now obviously breaks the RFC
* I realized that not setting a MSS for a packet that had none set 
before (or always setting 536) might cause trouble if the MSS determined 
by PMTU is (for whatever reason) < 536.

See http://bugzilla.netfilter.org/show_bug.cgi?id=662 for further 
information and a (really simple) patch.

Cheers,
- Daniel

PS: The bugreport mentions that http://research.microsoft.com doesn't 
work because of this bug - it seem like Microsoft has fixed this 
recently. They still don't send a MSS, but they accept larger packets now.

Re: TCPMSS clamp-mss-to-pmtu breaks packets that have no MSS set (incl. solution)

[Jan Engelhardt]

On Friday 2010-07-30 03:01, Daniel Gibson wrote:

> Hi,
>
> clamp-mss-to-pmtu (as used in "iptables -A FORWARD -p tcp
> --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu") sets a
> possibly much to high MSS (the one it got from PMTU) in packets
> that had no MSS set before.

If a packet has no MSS option, none will be added no more.
(commit v2.6.34-rc1~233^2~154^2~44)

Re: TCPMSS clamp-mss-to-pmtu breaks packets that have no MSS set (incl. solution)

[Nepenthes Development Team]

Hi,

On Mon, Aug 2, 2010 at 5:16 PM, Jan Engelhardt <jengelh@medozas.de> wrote:
> If a packet has no MSS option, none will be added no more.
> (commit v2.6.34-rc1~233^2~154^2~44)

git log -1 -p v2.6.34-rc1~233^2~154^2~44

I found some changes to prevent setting an MSS if the SYN packet has a
payload already, and to prevent it from increasing the MSS, but
nothing to prevent if from setting the MSS if there was no MSS set
before.

The code to add an MSS option if none exists still exists,
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=net/netfilter/xt_TCPMSS.c;h=62ec021fbd50ddd278da5f9f53756587d83fb786;hb=HEAD#l116


Markus

Re: TCPMSS clamp-mss-to-pmtu breaks packets that have no MSS set (incl. solution)

[Daniel Gibson]

Nepenthes Development Team schrieb:
> Hi,
> 
> On Mon, Aug 2, 2010 at 5:16 PM, Jan Engelhardt <jengelh@medozas.de> wrote:
>> If a packet has no MSS option, none will be added no more.
>> (commit v2.6.34-rc1~233^2~154^2~44)
> 
> git log -1 -p v2.6.34-rc1~233^2~154^2~44
> 
> I found some changes to prevent setting an MSS if the SYN packet has a
> payload already, and to prevent it from increasing the MSS, but
> nothing to prevent if from setting the MSS if there was no MSS set
> before.
> 
> The code to add an MSS option if none exists still exists,
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=net/netfilter/xt_TCPMSS.c;h=62ec021fbd50ddd278da5f9f53756587d83fb786;hb=HEAD#l116
> 
> 
> Markus
> --

Hi,

any news on this?

- Daniel