Re: [Intel-wired-lan] i40e XDP program stops transmitting after link down/up

Paul Menzel
2024-01-30T16:15:09Z

Dear Seth,

Thank you for bring this up.

Am 30.01.24 um 15:17 schrieb Seth Forshee:
> I got a inquiry from a colleague about a behavior he's seeing with i40e
> but not with other NICs. The interfaces are bonded with a XDP
> load-balancer program attached to them. After 'ip link set ethX down; ip
> link set ethX up' on one of the interfaces the XDP program on that
> interface is no longer transmitting packets. He found that tx starts
> again after running 'sudo ethtool -t ethX'.
>
> There's a 'i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable
> timeout' message in dmesg when disabling the interface. I've included
> the relevant portions from dmesg below.
>
> This was first observed with a 6.1 kernel, but we've confirmed that the
> behavior is the same in 6.7. I realize the firmware is pretty old, so
> far our attempts to update the NVM have failed.

Does that mean, the problem didn’t happen before Linux 6.1? If so, if
you have the reproducer and the time, bisecting the issue is normally
the fastest way to solve the issue.

Kind regards,

Paul

> [ 0.000000] Linux version 6.7.0 (root@616a530b3729) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #2 SMP Thu Jan 25 10:37:21 EST 2024
> ...
> [ 9.038171] i40e: Intel(R) Ethernet Connection XL710 Network Driver
> [ 9.044447] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
> ...
> [ 9.064833] i40e 0000:d8:00.0: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [ 9.320886] i40e 0000:d8:00.0: MAC address: xx:xx:xx:xx:xx:xx
> [ 9.327030] i40e 0000:d8:00.0: FW LLDP is enabled
> [ 9.344331] i40e 0000:d8:00.0 eth0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [ 9.355552] i40e 0000:d8:00.0: PCI-Express: Speed 8.0GT/s Width x8
> [ 9.374074] i40e 0000:d8:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
> ...
> [ 9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [ 9.652066] i40e 0000:d8:00.1: MAC address: xx:xx:xx:xx:xx:xx
> [ 9.658040] i40e 0000:d8:00.1: FW LLDP is enabled
> [ 9.688622] i40e 0000:d8:00.1 eth1: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [ 9.699822] i40e 0000:d8:00.1: PCI-Express: Speed 8.0GT/s Width x8
> [ 9.719259] i40e 0000:d8:00.1: Features: PF-id[1] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
> [ 9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [ 13.948968] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [ 13.956753] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [ 13.970005] i40e 0000:d8:00.0: FW LLDP is disabled
> ...
> [ 14.113004] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [ 14.120800] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [ 14.133940] i40e 0000:d8:00.1: FW LLDP is disabled
> ...
> [ 14.267874] bonding: bond0 is being created...
> [ 14.979070] i40e 0000:d8:00.1 eth1: set new mac address xx:xx:xx:xx:xx:xx
> [ 14.994445] bond0: (slave eth1): Enslaving as a backup interface with an up link
> [ 14.994681] i40e 0000:d8:00.0 eth0: set new mac address xx:xx:xx:xx:xx:xx
> [ 15.010359] bond0: (slave eth0): Enslaving as a backup interface with an up link
> [ 15.912874] i40e 0000:d8:00.0: Stop LLDP AQ command failed =0x1
> [ 15.944842] i40e 0000:d8:00.1: Stop LLDP AQ command failed =0x1
> [ 41.262871] 8021q: 802.1Q VLAN Support v1.8
> [ 41.262890] 8021q: adding VLAN 0 to HW filter on device eth0
> [ 41.262902] 8021q: adding VLAN 0 to HW filter on device eth1
> [ 41.262914] 8021q: adding VLAN 0 to HW filter on device bond0
> [ 48.272456] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [ 48.280233] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [ 48.307415] i40e 0000:d8:00.0: User requested queue count/HW max RSS count: 12/32
> [ 48.440266] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [ 48.448025] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [ 48.475051] i40e 0000:d8:00.1: User requested queue count/HW max RSS count: 12/32
> [ 58.935900] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [ 58.945123] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [ 59.131772] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [ 59.139560] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [ 336.363825] i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable timeout
> [ 336.603619] bond0: (slave eth1): link status definitely down, disabling slave
> [ 345.464976] 8021q: adding VLAN 0 to HW filter on device eth1
> [ 345.547358] bond0: (slave eth1): link status definitely up, 40000 Mbps full duplex
> [ 345.547379] bond0: active interface up!