[LARTC] fwmark+iproute2 routing policy, CIPE, tcpmss woes

[Myles Uyema myles@puck.nether.net]

<PRE>CIPE between two Linux systems on the Internet has been established.
Now I want to route all tcp traffic between the two linux systems over
the CIPE tunnel, so I've set up fwmark and iproute2 policies.

However, the TCPMSS rule only applies to SYN packets.  I'm able to
set the TCPMSS to 1400 for client TCP connections, but on the server,
the kernel message &quot;sending pkt_too_big to self&quot; repeatedly gets
logged, and tcp connections constantly get hung.

CIPE address    Internet IP
192.168.250.33  209.249.19.46  linux-dsl
192.168.250.34  65.0.152.158   athome

Commands run on &quot;athome&quot; (relevant sections reversed on &quot;linux-dsl&quot;)
echo 201 &gt;&gt; /etc/iproute2/rt_tables cipe0.vpn

iptables -t mangle -A OUTPUT -p tcp --syn -j TCPMSS --set-mss 1460
iptables -t mangle -A OUTPUT -p tcp -d 209.249.19.46 -j MARK --set-mark 1

ip rule add fwmark 1 table cipe0.vpn
ip route add default via 192.168.250.34 dev cipcb0 table cipe0.vpn


Here's a snippet from &quot;tcpdump -ni cipcb0 tcp port 25&quot;
Kernel filter, protocol ALL, TURBO mode (575 frames), datagram packet socket
tcpdump: listening on cipcb0
12:15:58.996627 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: S 2412475983:2412475983(0) win 5840 &lt;mss 1400,sackOK,timestamp 817624 0,nop,wscale 0&gt; (DF)
12:16:01.991788 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: S 2412475983:2412475983(0) win 5840 &lt;mss 1400,sackOK,timestamp 817924 0,nop,wscale 0&gt; (DF)
12:16:02.011327 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: S 2416837938:2416837938(0) ack 2412475984 win 5792 &lt;mss 1460,sackOK,timestamp 40493706 817624,nop,wscale 0&gt; (DF)
12:16:02.011445 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: . 1:1(0) ack 1 win 5840 &lt;nop,nop,timestamp 817925 40493706&gt; (DF)
12:16:02.039513 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 1:62(61) ack 1 win 5792 &lt;nop,nop,timestamp 40493709 817925&gt; (DF)
12:16:02.039591 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: . 1:1(0) ack 62 win 5840 &lt;nop,nop,timestamp 817928 40493709&gt; (DF)
12:16:02.059384 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 1:38(37) ack 62 win 5840 &lt;nop,nop,timestamp 817930 40493709&gt; (DF)
12:16:02.093587 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: . 62:62(0) ack 38 win 5792 &lt;nop,nop,timestamp 40493714 817930&gt; (DF)
12:16:02.096197 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 62:238(176) ack 38 win 5792 &lt;nop,nop,timestamp 40493714 817930&gt; (DF)
12:16:02.096603 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 38:44(6) ack 238 win 6500 &lt;nop,nop,timestamp 817934 40493714&gt; (DF)
12:16:02.136428 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 238:261(23) ack 44 win 5792 &lt;nop,nop,timestamp 40493717 817934&gt; (DF)
12:16:02.136625 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 44:79(35) ack 261 win 6500 &lt;nop,nop,timestamp 817938 40493717&gt; (DF)
12:16:02.165945 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 261:309(48) ack 79 win 5792 &lt;nop,nop,timestamp 40493722 817938&gt; (DF)
12:16:02.166140 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 79:114(35) ack 309 win 6500 &lt;nop,nop,timestamp 817941 40493722&gt; (DF)
12:16:02.191079 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 309:362(53) ack 114 win 5792 &lt;nop,nop,timestamp 40493724 817941&gt; (DF)
12:16:02.191251 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 114:120(6) ack 362 win 6500 &lt;nop,nop,timestamp 817943 40493724&gt; (DF)
12:16:02.210254 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: P 362:412(50) ack 120 win 5792 &lt;nop,nop,timestamp 40493726 817943&gt; (DF)
12:16:02.210537 &gt; 65.0.152.158.10046 &gt; 209.249.19.46.smtp: P 120:165(45) ack 412 win 6500 &lt;nop,nop,timestamp 817945 40493726&gt; (DF)
12:16:02.275256 &lt; 209.249.19.46.smtp &gt; 65.0.152.158.10046: . 412:412(0) ack 165 win 5792 &lt;nop,nop,timestamp 40493733 817945&gt; (DF)

Notice how 209.249.19.46 responds with a 1460 TCPMSS.  This is too large
when being tunneled through cipcb0.  It appears to me that
fwmark+iproute2 policy routing does not honor the gateway device's MTU.
(CIPE tunnel devices have an MTU of 1442).

Can anyone volunteer some suggestions?



</PRE>