Re: [bisected] xfrm: TCP connection initiating PMTU discovery stalls on v3.

Wolfgang Walter
2014-12-10T18:34:52Z

Am Dienstag, 9. Dezember 2014, 13:40:19 schrieb Eric Dumazet:
> On Tue, 2014-12-09 at 21:36 +0100, Wolfgang Walter wrote:
> > How would that be done? I found no way to disable it especially for xfrm.
> > I
> > disabled gso for the interface which serves the ipsec traffic but this
> > does
> > not help. tcp still uses gso for the esp tunnel.
> >
> > I put a view printk's in net/xfrm/xfrm_output.c and net/ipv4/tcp_output.c.
> > (I try to understand where in the xfrm transformation gso is handeled).
> >
> > What I can say yet is:
> >
> > xfrm_output() is used with ipsec (esp) tunnel mode but at I never see gso
> > packets here. xfrm_output_gso() is never called.
> >
> > Everytime tcp_set_skb_tso_segs() is called for a tcp connection over the
> > esp- tunnel and it is a gso case then the tcp connection hangs. Those
> > packets always have skb->len 1398 and mss_now is 1374. I see a call of
> > xfrm_output() afterwards but for a packet of skb->len 52 (maybe ACK from
> > other direction?).
> >
> > As long as the tcp-connection over the ipsec-tunnel works and if I send
> > bulk traffic xfrm_output() is called 3 times with packet skb->len 1426
> > and then one time with 78 (maybe other direction?), don't know if that is
> > of any interest.
> >
> >
> > With non-ipsec-traffic gso works fine: in this case the skb->len() varies
> > a
> > lot and mss_now is always 1288.
>
> Presumably something happens so that sk_can_gso() returns false.
>
> But apparently, this happens _after_ tcp_sendmsg(), ie a bit too late.
>
> Note that after https://patchwork.ozlabs.org/patch/418506/ is applied,
> fix would simply be :
>
> diff --git a/net/core/sock.c b/net/core/sock.c
> index
> 9a56b2000c3f374fb95aedada3327447816a9512..678ef8393680dc781445c5f121719ea8e
> a7bb7c1 100644 --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -1585,6 +1585,8 @@ void sk_setup_caps(struct sock *sk, struct dst_entry
> *dst) sk->sk_gso_max_size = dst->dev->gso_max_size;
> sk->sk_gso_max_segs = dst->dev->gso_max_segs;
> }
> + } else {
> + sk->sk_gso_max_segs = 1;
> }
> }
> EXPORT_SYMBOL_GPL(sk_setup_caps);

Ok. I tried that. I backported https://patchwork.ozlabs.org/patch/418506/ to
3.14.26 and then applied this patch. Still hangs. At that stage sk_can_gso(sk)
is true. The reason is that the interface of *dst in sk_setup_caps() is the
interface which serves the ipsec traffic.

There is a difference, though. If I disable gso for the interface which serves
the ipsec traffic then the hang disappears now. This seems to be because of
the patch above: sk_can_gso(sk) in sk_setup_caps then returns false and the ne
else branch is taken. So there may be a hidden bug in the non-gso case.

Back to xfrm case with hang:

I see: a call of sk_setup_caps() which takes the path
sk->sk_route_caps |= NETIF_F_SG | NETIF_F_HW_CSUM;
...

So gso is on. When the hang happens sk_setup_caps is called from
inet_sk_rebuild_header(). Now the path

sk->sk_route_caps &= ~NETIF_F_GSO_MASK;

is taken as dst->header_len is now non zero.

This is the reason why later calls of sk_can_gso() return false.

I'll try to change the above patch to

@@ -1585,6 +1585,8 @@ void sk_setup_caps(struct sock *sk, struct dst_entry
*dst) sk->sk_gso_max_size = dst->dev->gso_max_size;
sk->sk_gso_max_segs = dst->dev->gso_max_segs;
}
}
+ if (sk_can_gso(sk)) {
+ sk->sk_gso_max_segs = 1;
}
}
EXPORT_SYMBOL_GPL(sk_setup_caps);

so that the case that GSO is disabled because of dst->header_len != 0 sets
sk_gso_max_segs, too.

Regards,
--
Wolfgang Walter
Studentenwerk München
Anstalt des öffentlichen Rechts