r8169: RTL8126A (XID 64a) TX packets lost with TSO enabled

[Zach Schneider <me@zachschneider.com>]

Hi,

On an RTL8126A (PCI 10ec:8126, XID 64a, firmware rtl8126a-3_0.0.5
08/30/24) TCP uploads collapse when r8169 has TSO enabled, which is the
default for this chip. Disabling TSO with ethtool fixes it.

Kernel: 7.2.9-arch1-1 (Arch Linux stock kernel; also seen on 7.2.9-zen).
Board: ASUS ROG STRIX X870E-E GAMING WIFI (onboard RTL8126).
Link: 2.5Gbps/Full, flow control rx/tx.

Test: 100 MB HTTPS upload (IPv4) over the RTL8126, measuring TCP
RetransSegs from /proc/net/snmp:

  r8169, TSO on (default)   upload stalls, >120 s,  25-32% retransmitted
  r8169, TSO off            45.8 MB/s, 2.2 s,       0.0% retransmitted

The stalled connection shows bytes_retrans of about a quarter of
bytes_sent and cwnd collapsing to 4. TX error counters in ethtool -S
stay at zero. Downloads are unaffected (on 7.2.9-zen they were faster
than with the vendor driver, with rx_missed 0).

Reducing gso_max_size does not help. It fails the same way at 64000,
32000, 16000, 8000, 4000 and 2896, so any segmentation offload seems to
produce packets that get lost, not only large bursts.

For comparison, Realtek's out-of-tree r8126 10.018.00 driver ends up
with TSO off by default on this setup: rtl8126_fix_features() clears
NETIF_F_ALL_CSUM unless built with CONFIG_R8126_VLAN, so TSO is dropped
too. That driver also builds TSO descriptors differently for this chip
(GiantSendEn_V3 bit 16, GiantSendv4_V3 bit 29 and GiantSendTCP_V3 bit 30
in opts[1]), while rtl8169_tso_csum_v2() sets TD1_GTSENV4 (bit 26) in
opts[0]. I don't know whether that difference matters in the mode r8169
runs the chip in.

RTL_GIGA_MAC_VER_70 gets SG/TSO enabled by default since commit
3907f1ffc0ec ("r8169: add support for RTL8126A").

Relevant dmesg:

  r8169 0000:0e:00.0 eth0: RTL8126A, <MAC>, XID 64a, IRQ 200
  r8169 0000:0e:00.0 eth0: jumbo features [frames: 16362 bytes, tx checksumming: ko]
  Realtek Internal NBASE-T PHY r8169-0-e00:00: attached PHY driver (mii_bus:phy_addr=r8169-0-e00:00, irq=MAC)
  r8169 0000:0e:00.0 enp14s0: Link is Up - 2.5Gbps/Full - flow control rx/tx

I'm happy to test patches or collect more data. I only have this one
machine on the LAN, so I can't capture on the receiving side.

The analysis of the driver sources was done with help from an LLM
coding assistant; the measurements are from my machine.

Thanks,
Zach Schneider