Re: Problems with fragments since gso skb forwarding changes in virtual environment

[Tobias Brunner <tobias@strongswan.org>]

Hi Florian,

> Do I interpret this correctly:
> 
> Host A - br1 - Router R - br2 - Host B
>   Mtu >1500               Mtu 1500
> 
> 1. host A sends GSO packet, DF not set
> 2. packet arrives at R, still GSO packet
> 3. forward on R fragments packet since it won't fit
>    outgoing interface (which is normal virtio ethernet) mtu
> 4. fragmented packets leave R
> 5. fragmented packets arrive on host system (not pictured above) br2
> interface
> 
> 6. packets are being bridged on host system, call_iptables sysctl on
> 7. packets are defragmented by netfilter on host due to call_iptables
> sysctl on
> 8. packets are tossed on host in br_dev_queue_push_xmit because
>    is_skb_forwardable() returns false
> 
> Is that correct?

Exactly.  The MTU is 1500 on all interfaces though.

>> Without the commit, and between A and R even with it (because it only
>> affects forwarding), the skbs are GSO throughout and transmitted from A
>> to B without ever actually being fragmented.
> 
> I see why this change makes it trip over GSO skbs, but I fail to
> see why it would work with larger-than-1500-mtu-and-fragmentation-allowed
> packets being sent from A to B. (or with fragments generated locally
> on R).
> 
> To the host system it should make no difference at all if the fragments
> came into existence in R's forwarding path, or being sent by A, or if
> the fragments were generated locally on R (i.e. ping -s $bignum $hosta
> on R with DF off).

In our test scenarios the packets are UDP and GSO and without the commit
(or between A and R) they travel unchanged between guest and host
kernels without ever touching a physical interface that would actually
cause them to get fragmented (I wasn't aware of this, until I looked
into this issue).

For ICMP it's interesting to note that 'ping -s $bignum $hostb' from A
works even with the commit.  The packet is already fragmented when it
leaves A and these fragments are forwarded properly by the host bridges.
 They are defragmented by the nf_defrag_ipv4 module, but are fragmented
again in br_nf_dev_queue_xmit() because skb->nfct is non-null as pointed
out by you and David.

I tried removing the skb->nfct check, and while that fixes the
forwarding issue on the host, for some reason the UDP socket on B does
not receive the packet (the guest kernel does, even defragments it and
queues it to the socket, but the userland program never receives the
datagram).

Regards,
Tobias