[BUG] XFRM is not updating RTAX_ADVMSS metric

[BUG] XFRM is not updating RTAX_ADVMSS metric

[Eduardo Panisset]

Hi All,

I have been testing DSMIPv6 code which uses all kind of advanced
features of XFRM framework and I believe I have found a bug related to
update RTAX_ADVMSS route metric.
The XFRM code on net/xfrm/xfrm_policy.c by its functions
xfrm_init_pmtu and xfrm_bundle_ok updates RTAX_MTU route caching
metric however I believe it must update RTAX_ADVMSS as this later is
used by tcp connect function for adverting the MSS value on SYN
messages.

As MSS is not being updated by XFRM the TCP SYN messages (e.g.
originated from a internet browser)  is erroneously informing its MSS
(without taking into account the overhead added to IP packet size by
XFRM transformations).  One result of that is the browser gets
"frozen" after starts a TCP connection because TCP messages sent by
TCP server will never get to it (TCP server is sending too large
segments to browser).

Below I describe the changes I have done (on xfrm_init_pmtu and
xfrm_bundle_ok) and that seem to fix this problem:

xfrm_init_pmtu:
                 .
                 .
                 .

        dst->metrics[RTAX_MTU-1] = pmtu; // original code, below my changes

        if (dst->xfrm->props.mode == XFRM_MODE_TUNNEL)
                 switch (dst->xfrm->props.family)
                 {
                 case AF_INET:
                 dst->metrics[RTAX_ADVMSS-1] = max_t(unsigned int,
pmtu - sizeof(struct iphdr) - sizeof(struct tcphdr), 256);
                 break;

                 case AF_INET6:
                 dst->metrics[RTAX_ADVMSS-1] = max_t(unsigned int,
pmtu - sizeof(struct ipv6hdr) - sizeof(struct tcphdr),
                            dev_net(dst->dev)->ipv6.
sysctl.ip6_rt_min_advmss);
                 break;
                 }

xfrm_bundle_ok:

               .
               .
               .

        dst->metrics[RTAX_MTU-1] = mtu; // original code, below my changes

        if (dst->xfrm->props.mode == XFRM_MODE_TUNNEL)
                switch (dst->xfrm->props.family)
                {
                case AF_INET:
                        dst->metrics[RTAX_ADVMSS-1] = max_t(unsigned
int, mtu - sizeof(struct iphdr) - sizeof(struct tcphdr), 256);
                break;

                case AF_INET6:
                        dst->metrics[RTAX_ADVMSS-1] = max_t(unsigned
int, mtu - sizeof(struct ipv6hdr) - sizeof(struct tcphdr),

dev_net(dst->dev)->ipv6.sysctl.ip6_rt_min_advmss);
                break;
                }

Regards,
Eduardo Panisset.