nftables flow offload possible mtu handling issue

[<marcmicalizzi@gmail.com>]

With flow offload between devices of differing mtus, there seems to be an
issue sending from through higher mtu to the lower mtu device.
I’m currently on 4.20 from the linux-arm mcbin branch, as it’s all I can get
running on my specific embedded platform.

In my case I have

table ip nat {
        chain POSTROUTING {
                type nat hook postrouting priority 100; policy accept;
                oifname "ppp0" ip saddr 192.168.10.0/24 snat to
xx.xxx.xx.xxx

        }
}
table ip filter {
        flowtable f1 {
                hook ingress priority 0
                devices = { eth0.1, ppp0 }
        }

        chain FORWARD {
                type filter hook forward priority 0; policy accept;
                ip protocol tcp flow offload @f1
                counter
                tcp flags syn tcp option maxseg size set rt mtu
        }
}

ppp0 has an mtu of 1500.
Running a speedtest from a device connected to eth0.1, download seems to
always be fine, but if eth0.1 has an mtu higher than 1492, upload fails
completely most of the time. (Prior to flow offloading, eth0/eth0.1 has an
mtu of 9000, and this is the mtu I would prefer to have it running at.

An interesting observation, as well, is that with eth0.1 mtu at 9000, if
during the upload portion of a speed test I issue `ip link set ppp0 mtu
(1492/1500, whichever it currently is not)`, the upload will start working
for the remainder of the upload test.

This also manifests outside speedtest, but less predictably, with dropped
connections due to outgoing packet failures.