All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH net v7 0/3] igb: only strip the Rx timestamp header from the first buffer of a frame
@ 2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters, Piotr Kwapulinski,
	Aleksandr Loktionov, Alexander Nowlin

With Rx hardware timestamping enabled (e.g. by ptp4l), igb strips the
16-byte inline timestamp header from every Rx buffer of a frame instead
of only from the first one. A continuation buffer that starts with 8
zero bytes loses 16 bytes, so jumbo frames with zero payload arrive
short and are dropped.

v3 used skb == NULL to find the first buffer of a frame. The Sashiko
AI review pointed out that this does not hold when an XDP program
consumes the first buffer of a frame that spans several buffers, so v3
was sent back for rework.

The same question came up for igc. There, Maciej suggested not to track
frame boundaries in the data path, but to make sure that frames never
span several buffers while XDP is attached [1]. Paul Moses' igc v3 does
that [2]. This version follows the same approach for igb:

  1/3 limits the receive length (RLPML, or the PF pool's VMOLR with VFs)
      to the configured frame size while XDP is attached, and leaves
      room for the timestamp header in the XDP buffer size checks for
      3 KiB, legacy-rx and PAGE_SIZE >= 8192 buffers. It also refuses
      XDP with rx-all and legacy-rx changes while XDP is attached, as
      both would break the limit. The limit is now set from
      igb_configure() instead of igb_set_rx_mode(). With VFs, the PF
      pool limit is also written on i354.
  2/3 is v3, unchanged in code, with a note that it relies on 1/3.
      The Reviewed-by and Tested-by tags of v3 are kept for that reason.
  3/3 gives AF_XDP zero-copy rings a 2 KiB or 3 KiB hardware buffer
      instead of rounding the XSK frame size down to 1 KiB. It refuses
      XSK pools that cannot hold the current maximum frame plus the
      timestamp header, and MTU changes that would no longer fit a bound
      pool. Unlike the igc patch in [2], which checks against the
      largest XDP MTU, this checks the current MTU, as igb allows XDP up
      to MTU 3030 and a fixed check would rule out 2 KiB chunks.
      Enabling VFs, which leaves the PF a single Tx queue, is refused
      with -EBUSY while an XSK pool is bound to another queue; the core
      refuses ethtool -L in the same case. The check runs before any VF
      state is set up. A pool on a queue hidden in another way, for
      example after a fallback to MSI, is checked again when its ring
      is set up.

As in igc, 1/3 moves the RLPML programming out of igb_set_rx_mode(),
which can run without RTNL, into igb_configure(), which runs when the
interface is (re)configured, before Rx is enabled. This includes the PF
pool limit in VMOLR, which igb_set_rx_mode() now leaves as it is.
Attaching or removing XDP on a running interface goes through
igb_close() and igb_open(), and an MTU change through igb_down() and
igb_up(), so the limit is updated before Rx is enabled again.

The Sashiko reviews of v4 to v6 also point out older issues that this
series does not change:

- On 2 KiB build_skb rings, igb_skb_pad() leaves exactly
  IGB_MAX_FRAME_BUILD_SKB bytes before skb_shared_info, with no room for
  the 16-byte timestamp header. With Rx timestamping on, frame and
  header can overlap skb_shared_info by up to 12 bytes, or 2 bytes at
  the XDP limit at MTU 1500. This dates from 9cbc948b5a20 ("igb: add XDP
  support"). I left it out of this series: it is a separate problem from
  the one fixed here, and a fix changes the buffer layout and the XDP
  headroom for all igb users, so it needs its own review and testing.
- igb_set_vf_rlpml() let a VF set other VMOLR bits of its own pool. That
  is fixed separately: https://lore.kernel.org/all/20261104-igb-vf-rlpml-v1-1-729621b60eee@aweta.nl/
- igb_alloc_rx_buffers_zc() does not clear the length of the next
  descriptor after a partial refill at the end of the ring, so
  igb_clean_rx_irq_zc() can read a stale descriptor. igc clears it in
  that case. This is a separate zero-copy problem as well.
- __igb_open() sets up the rings before it updates the real queue
  counts, so a queue that comes back is used in zero-copy mode again
  only from the next reconfiguration; see 3/3.
- When the queue setup or pci_enable_sriov() fails, igb_enable_sriov()
  frees the VF state without a lock, and after a failed
  pci_enable_sriov() the PF keeps the VF queue layout. The new -EBUSY
  return does not go through that path.
- igb_set_rx_mode() is also called without the address lock, for
  example from the VF mailbox interrupt, and then walks the address
  lists.
- The ethtool offline self-test sets up its test rings with queue index
  0, so igb_xsk_pool() gives them the pool bound to queue 0. That needs
  a separate fix.

How to reproduce
----------------
Receiver with igb (eth0), peer with MTU 8228. hwstamp_ctl turns on Rx
timestamping with filter ALL, no ptp4l needed; this ping method is
taken from Paul's igc report. One host with two ports also works if
the sending port is moved into its own network namespace.

1) Truncated jumbo frames (patch 2)

  receiver:  ip link set eth0 mtu 8228
             hwstamp_ctl -i eth0 -r 1
  peer:      ping -c 5 -M do -s 8200 -p 00 <receiver>

  Without the series there are no replies, and IpExtInTruncatedPkts
  goes up by one per frame. With the series all requests are answered.

2) XDP consumes the first buffer (patch 1)

  receiver:  ip link set eth0 mtu 3000
             hwstamp_ctl -i eth0 -r 1
             xdp-filter load -m native eth0
             xdp-filter ip <peer> -m src
             tcpdump -nn -e -i eth0 'ether src 00:00:00:00:00:00'
  peer:      ping -c 3 -M do -s 8200 -p 00 <receiver>

  At MTU 3000 XDP can attach, but without patch 1 the MAC still accepts
  the 8242-byte frames, which then span three Rx buffers. xdp-filter
  drops the first one, and the rest reaches the stack as a separate
  frame with all-zero MAC addresses:

    without the series    5154 bytes (header stripped twice)
    v3 alone              5170 bytes (stripped once)
    this series           no frame; rx_long_length_errors +1 per frame

3) AF_XDP zero-copy with 2 KiB chunks (patch 3)

  receiver:  ip link set eth0 mtu 1500
             ethtool -L eth0 combined 1
             hwstamp_ctl -i eth0 -r 1
             xdpsock -i eth0 -q 0 -r -z -N -f 2048
  peer:      ping -c 50 -i 0.02 -s 1472 <receiver>

  xdpsock is from xdp-project/bpf-examples (AF_XDP-example). It also
  takes ARP, so the peer needs a fresh ARP entry before it starts.
  Compare rx_packets (MAC) with rx_queue_0_packets in ethtool -S:

    without the series    68 frames, 118 descriptors
    this series           64 frames, 64 descriptors

  At MTU 3000, -f 4096 receives 3014-byte frames in one buffer. Binding
  with -f 2048 at MTU 3000 was refused with EOPNOTSUPP by the new check
  on the 6.14.9 test kernel; current kernels refuse it earlier, in
  xp_assign_dev(), with EINVAL.

Tested on an I350 with these patches applied to the igb driver of
kernel 6.14.9 (the series itself is against v7.1-rc7), with the steps
above: results as listed. The v3 line is from an earlier run with an
XDP program dropping ICMP. Also checked: XDP attach at MTU 1500 and
3030 works, at MTU 3046 it is refused, as is an MTU change to 3046
with XDP attached. With two VFs allocated (booted with pci=realloc),
step 2 gives the same picture: three leftover frames without the
series, none with it, as the PF pool limit in VMOLR drops the frames.
In step 3, 98-byte frames and 4 KiB chunks take one descriptor per
frame with and without the series. With XDP attached, rx-all stays off
and legacy-rx cannot be changed, and XDP attach is refused with rx-all
on, or with legacy-rx on at MTU 3000, also when it was set while the
interface was down. RLPML and the PF pool's VMOLR limit read back as
expected with and without XDP and VFs. In repeated runs of about 200 MTU
changes each with XDP and two VFs, the limit was right after every
change, except for one read in one run, where the value was not logged;
eleven reruns showed no mismatch.

The steps above were repeated on kernel 7.3.1 with the series applied to
the v7.3 igb driver, with the same results, including step 2 with VFs.
There, enabling two VFs while xdpsock was bound to queue 1 was refused
with EBUSY; with the socket on queue 0 it worked. With 31 macvlans on
the PF, which fill all 32 receive address registers, all 31 addresses
stayed valid after the refused VF enable; with v6, two of them were
overwritten by the VFs' empty MAC addresses. With the driver built so
that the I350 keeps two Rx queues with VFs, as an 82576 does, binding a
zero-copy socket to queue 1 after VFs were enabled on the down PF was
refused with EINVAL; without the new check it was accepted. Removing XDP
from a down, runtime-suspended interface with rx-all requested resumed
the device and turned rx-all on. The i354 path was checked with the
driver built so that the I350 takes the i354 branches for VMOLR: with
two VFs and XDP attached, the PF pool limit kept its old value of 9728
with v5, which skipped i354, and read back as 3026 at MTU 3000 and 1526
at MTU 1500 since v6. Only the registers were checked; I have no i354 to
test on.
Not tested: PAGE_SIZE >= 8192.

The patches, their descriptions and this cover letter were prepared
with the help of an AI coding assistant (Claude). I reviewed them and
ran the tests above on my test bench.

[1] https://lore.kernel.org/all/20261025065154.2037149-1-p@1g4.org/
[2] https://lore.kernel.org/all/20261028083344.3047350-1-p@1g4.org/

Changes in v7:
- Patch 1: when XDP is removed and rx-all is turned back on, resume a
  runtime-suspended device first, as ethtool does, and skip it if the
  device could not be resumed or the interface could not be opened
  again (Sashiko).
- Patch 3: check for bound XSK pools at the start of igb_enable_sriov(),
  before any VF state is set up, and hold RTNL until the PF queues are
  set up again (Sashiko). In v6 the check ran after the VFs' MAC
  filters were written, which could disable PF unicast filters until
  the next reset, and the -EBUSY return went through the existing error
  path, which frees VF state without a lock. The check no longer runs
  when VFs are disabled, and the DMA coalescing flag is no longer saved
  and restored. The older error paths are unchanged.
- Patch 3: igb_xsk_pool_enable() also checks the PF's Tx queue count, so
  that no zero-copy pool is bound to a queue without a Tx ring on 82576
  with VFs.
- Patch 3: the commit message now says that a queue that comes back is
  used in zero-copy mode again only from the next reconfiguration
  (Sashiko); no code change for that.
- Patch 2: no change.
- The list of older issues above names four more.
- Link to v6: https://patch.msgid.link/20261111-igb-rx-ts-fix-v6-0-ba5b9903859c@aweta.nl

Changes in v6:
- Patch 1: with VFs, also write the PF pool limit on i354 (Sashiko).
- Patch 3: refuse to enable VFs with -EBUSY while an XSK pool is bound
  to a queue the PF would lose (Maciej). Restore the DMA coalescing flag
  on that error. Keep the frame size check in igb_xsk_pool() only for
  queues hidden in another way, such as a fallback to MSI.
- Patch 4 dropped: with patch 3 the hardware does not split frames in
  zero-copy mode, so it only added dead code to the hot path (Maciej).
- Link to v5: https://patch.msgid.link/20261104-igb-rx-ts-fix-v5-0-bf4b4367129a@aweta.nl

Changes in v5:
- Patch 1: v4 also subtracted the timestamp header from the build_skb
  frame limit, so XDP no longer attached at MTU 1495-1510 on x86
  (1493-1508 where NET_IP_ALIGN is 2), including the default MTU 1500.
  Only subtract the header where the limit is the hardware buffer size,
  and compute it from the adapter state instead of the ring flags. Skip
  the check when XDP is removed, so a program can always be detached.
- Patch 1: refuse XDP with rx-all and legacy-rx changes with XDP
  attached, and turn rx-all back on after XDP removal if it was
  requested meanwhile. Program RLPML and the PF pool limit from
  igb_configure() instead of igb_set_rx_mode(). With XDP attached, an
  MTU change can now move from 2 KiB to 3 KiB buffers. Added
  Suggested-by: Maciej.
- Patch 1: the issues above were also reported by the Sashiko review
  of v4.
- Patch 2: no code change. Commit message: igc is fixed by Paul's
  series, not by this patch; the loss is per continuation buffer, not
  only at the tail; the possible size wrap is described (Sashiko).
- New patch 3: AF_XDP zero-copy Rx buffer size, as asked by Maciej.
- New patch 4: drop zero-copy frames that span several descriptors;
  depends on patch 3.
- Link to v4: https://patch.msgid.link/20261102-igb-rx-ts-fix-v4-0-8785124c5d23@aweta.nl

Changes in v4:
- New patch 1: limit the receive length while XDP is attached, so that
  skb == NULL reliably marks the first buffer (Sashiko review), as
  done for igc.
- Patch 2: no code change; added a note that it relies on patch 1.
- Link to v3: https://patch.msgid.link/20260625-igb-rx-ts-fix-v3-1-99b3efa08dca@aweta.nl

---
Tjerk Kusters (3):
      igb: do not let XDP frames span more than one Rx buffer
      igb: only strip Rx timestamp header on the first buffer of a frame
      igb: do not split AF_XDP zero-copy frames over several Rx buffers

 drivers/net/ethernet/intel/igb/igb.h         |   9 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c |   9 +
 drivers/net/ethernet/intel/igb/igb_main.c    | 244 +++++++++++++++++++++------
 drivers/net/ethernet/intel/igb/igb_xsk.c     |  27 ++-
 4 files changed, 233 insertions(+), 56 deletions(-)
---
base-commit: 2d3090a8aeb596a26935db0955d46c9a5db5c6ce
change-id: 20260619-igb-rx-ts-fix-cd70585ee316

Best regards,
--  
Tjerk Kusters <tkusters@aweta.nl>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH net v7 0/3] igb: only strip the Rx timestamp header from the first buffer of a frame
@ 2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters via B4 Relay @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters, Piotr Kwapulinski,
	Aleksandr Loktionov, Alexander Nowlin

With Rx hardware timestamping enabled (e.g. by ptp4l), igb strips the
16-byte inline timestamp header from every Rx buffer of a frame instead
of only from the first one. A continuation buffer that starts with 8
zero bytes loses 16 bytes, so jumbo frames with zero payload arrive
short and are dropped.

v3 used skb == NULL to find the first buffer of a frame. The Sashiko
AI review pointed out that this does not hold when an XDP program
consumes the first buffer of a frame that spans several buffers, so v3
was sent back for rework.

The same question came up for igc. There, Maciej suggested not to track
frame boundaries in the data path, but to make sure that frames never
span several buffers while XDP is attached [1]. Paul Moses' igc v3 does
that [2]. This version follows the same approach for igb:

  1/3 limits the receive length (RLPML, or the PF pool's VMOLR with VFs)
      to the configured frame size while XDP is attached, and leaves
      room for the timestamp header in the XDP buffer size checks for
      3 KiB, legacy-rx and PAGE_SIZE >= 8192 buffers. It also refuses
      XDP with rx-all and legacy-rx changes while XDP is attached, as
      both would break the limit. The limit is now set from
      igb_configure() instead of igb_set_rx_mode(). With VFs, the PF
      pool limit is also written on i354.
  2/3 is v3, unchanged in code, with a note that it relies on 1/3.
      The Reviewed-by and Tested-by tags of v3 are kept for that reason.
  3/3 gives AF_XDP zero-copy rings a 2 KiB or 3 KiB hardware buffer
      instead of rounding the XSK frame size down to 1 KiB. It refuses
      XSK pools that cannot hold the current maximum frame plus the
      timestamp header, and MTU changes that would no longer fit a bound
      pool. Unlike the igc patch in [2], which checks against the
      largest XDP MTU, this checks the current MTU, as igb allows XDP up
      to MTU 3030 and a fixed check would rule out 2 KiB chunks.
      Enabling VFs, which leaves the PF a single Tx queue, is refused
      with -EBUSY while an XSK pool is bound to another queue; the core
      refuses ethtool -L in the same case. The check runs before any VF
      state is set up. A pool on a queue hidden in another way, for
      example after a fallback to MSI, is checked again when its ring
      is set up.

As in igc, 1/3 moves the RLPML programming out of igb_set_rx_mode(),
which can run without RTNL, into igb_configure(), which runs when the
interface is (re)configured, before Rx is enabled. This includes the PF
pool limit in VMOLR, which igb_set_rx_mode() now leaves as it is.
Attaching or removing XDP on a running interface goes through
igb_close() and igb_open(), and an MTU change through igb_down() and
igb_up(), so the limit is updated before Rx is enabled again.

The Sashiko reviews of v4 to v6 also point out older issues that this
series does not change:

- On 2 KiB build_skb rings, igb_skb_pad() leaves exactly
  IGB_MAX_FRAME_BUILD_SKB bytes before skb_shared_info, with no room for
  the 16-byte timestamp header. With Rx timestamping on, frame and
  header can overlap skb_shared_info by up to 12 bytes, or 2 bytes at
  the XDP limit at MTU 1500. This dates from 9cbc948b5a20 ("igb: add XDP
  support"). I left it out of this series: it is a separate problem from
  the one fixed here, and a fix changes the buffer layout and the XDP
  headroom for all igb users, so it needs its own review and testing.
- igb_set_vf_rlpml() let a VF set other VMOLR bits of its own pool. That
  is fixed separately: https://lore.kernel.org/all/20261104-igb-vf-rlpml-v1-1-729621b60eee@aweta.nl/
- igb_alloc_rx_buffers_zc() does not clear the length of the next
  descriptor after a partial refill at the end of the ring, so
  igb_clean_rx_irq_zc() can read a stale descriptor. igc clears it in
  that case. This is a separate zero-copy problem as well.
- __igb_open() sets up the rings before it updates the real queue
  counts, so a queue that comes back is used in zero-copy mode again
  only from the next reconfiguration; see 3/3.
- When the queue setup or pci_enable_sriov() fails, igb_enable_sriov()
  frees the VF state without a lock, and after a failed
  pci_enable_sriov() the PF keeps the VF queue layout. The new -EBUSY
  return does not go through that path.
- igb_set_rx_mode() is also called without the address lock, for
  example from the VF mailbox interrupt, and then walks the address
  lists.
- The ethtool offline self-test sets up its test rings with queue index
  0, so igb_xsk_pool() gives them the pool bound to queue 0. That needs
  a separate fix.

How to reproduce
----------------
Receiver with igb (eth0), peer with MTU 8228. hwstamp_ctl turns on Rx
timestamping with filter ALL, no ptp4l needed; this ping method is
taken from Paul's igc report. One host with two ports also works if
the sending port is moved into its own network namespace.

1) Truncated jumbo frames (patch 2)

  receiver:  ip link set eth0 mtu 8228
             hwstamp_ctl -i eth0 -r 1
  peer:      ping -c 5 -M do -s 8200 -p 00 <receiver>

  Without the series there are no replies, and IpExtInTruncatedPkts
  goes up by one per frame. With the series all requests are answered.

2) XDP consumes the first buffer (patch 1)

  receiver:  ip link set eth0 mtu 3000
             hwstamp_ctl -i eth0 -r 1
             xdp-filter load -m native eth0
             xdp-filter ip <peer> -m src
             tcpdump -nn -e -i eth0 'ether src 00:00:00:00:00:00'
  peer:      ping -c 3 -M do -s 8200 -p 00 <receiver>

  At MTU 3000 XDP can attach, but without patch 1 the MAC still accepts
  the 8242-byte frames, which then span three Rx buffers. xdp-filter
  drops the first one, and the rest reaches the stack as a separate
  frame with all-zero MAC addresses:

    without the series    5154 bytes (header stripped twice)
    v3 alone              5170 bytes (stripped once)
    this series           no frame; rx_long_length_errors +1 per frame

3) AF_XDP zero-copy with 2 KiB chunks (patch 3)

  receiver:  ip link set eth0 mtu 1500
             ethtool -L eth0 combined 1
             hwstamp_ctl -i eth0 -r 1
             xdpsock -i eth0 -q 0 -r -z -N -f 2048
  peer:      ping -c 50 -i 0.02 -s 1472 <receiver>

  xdpsock is from xdp-project/bpf-examples (AF_XDP-example). It also
  takes ARP, so the peer needs a fresh ARP entry before it starts.
  Compare rx_packets (MAC) with rx_queue_0_packets in ethtool -S:

    without the series    68 frames, 118 descriptors
    this series           64 frames, 64 descriptors

  At MTU 3000, -f 4096 receives 3014-byte frames in one buffer. Binding
  with -f 2048 at MTU 3000 was refused with EOPNOTSUPP by the new check
  on the 6.14.9 test kernel; current kernels refuse it earlier, in
  xp_assign_dev(), with EINVAL.

Tested on an I350 with these patches applied to the igb driver of
kernel 6.14.9 (the series itself is against v7.1-rc7), with the steps
above: results as listed. The v3 line is from an earlier run with an
XDP program dropping ICMP. Also checked: XDP attach at MTU 1500 and
3030 works, at MTU 3046 it is refused, as is an MTU change to 3046
with XDP attached. With two VFs allocated (booted with pci=realloc),
step 2 gives the same picture: three leftover frames without the
series, none with it, as the PF pool limit in VMOLR drops the frames.
In step 3, 98-byte frames and 4 KiB chunks take one descriptor per
frame with and without the series. With XDP attached, rx-all stays off
and legacy-rx cannot be changed, and XDP attach is refused with rx-all
on, or with legacy-rx on at MTU 3000, also when it was set while the
interface was down. RLPML and the PF pool's VMOLR limit read back as
expected with and without XDP and VFs. In repeated runs of about 200 MTU
changes each with XDP and two VFs, the limit was right after every
change, except for one read in one run, where the value was not logged;
eleven reruns showed no mismatch.

The steps above were repeated on kernel 7.3.1 with the series applied to
the v7.3 igb driver, with the same results, including step 2 with VFs.
There, enabling two VFs while xdpsock was bound to queue 1 was refused
with EBUSY; with the socket on queue 0 it worked. With 31 macvlans on
the PF, which fill all 32 receive address registers, all 31 addresses
stayed valid after the refused VF enable; with v6, two of them were
overwritten by the VFs' empty MAC addresses. With the driver built so
that the I350 keeps two Rx queues with VFs, as an 82576 does, binding a
zero-copy socket to queue 1 after VFs were enabled on the down PF was
refused with EINVAL; without the new check it was accepted. Removing XDP
from a down, runtime-suspended interface with rx-all requested resumed
the device and turned rx-all on. The i354 path was checked with the
driver built so that the I350 takes the i354 branches for VMOLR: with
two VFs and XDP attached, the PF pool limit kept its old value of 9728
with v5, which skipped i354, and read back as 3026 at MTU 3000 and 1526
at MTU 1500 since v6. Only the registers were checked; I have no i354 to
test on.
Not tested: PAGE_SIZE >= 8192.

The patches, their descriptions and this cover letter were prepared
with the help of an AI coding assistant (Claude). I reviewed them and
ran the tests above on my test bench.

[1] https://lore.kernel.org/all/20261025065154.2037149-1-p@1g4.org/
[2] https://lore.kernel.org/all/20261028083344.3047350-1-p@1g4.org/

Changes in v7:
- Patch 1: when XDP is removed and rx-all is turned back on, resume a
  runtime-suspended device first, as ethtool does, and skip it if the
  device could not be resumed or the interface could not be opened
  again (Sashiko).
- Patch 3: check for bound XSK pools at the start of igb_enable_sriov(),
  before any VF state is set up, and hold RTNL until the PF queues are
  set up again (Sashiko). In v6 the check ran after the VFs' MAC
  filters were written, which could disable PF unicast filters until
  the next reset, and the -EBUSY return went through the existing error
  path, which frees VF state without a lock. The check no longer runs
  when VFs are disabled, and the DMA coalescing flag is no longer saved
  and restored. The older error paths are unchanged.
- Patch 3: igb_xsk_pool_enable() also checks the PF's Tx queue count, so
  that no zero-copy pool is bound to a queue without a Tx ring on 82576
  with VFs.
- Patch 3: the commit message now says that a queue that comes back is
  used in zero-copy mode again only from the next reconfiguration
  (Sashiko); no code change for that.
- Patch 2: no change.
- The list of older issues above names four more.
- Link to v6: https://patch.msgid.link/20261111-igb-rx-ts-fix-v6-0-ba5b9903859c@aweta.nl

Changes in v6:
- Patch 1: with VFs, also write the PF pool limit on i354 (Sashiko).
- Patch 3: refuse to enable VFs with -EBUSY while an XSK pool is bound
  to a queue the PF would lose (Maciej). Restore the DMA coalescing flag
  on that error. Keep the frame size check in igb_xsk_pool() only for
  queues hidden in another way, such as a fallback to MSI.
- Patch 4 dropped: with patch 3 the hardware does not split frames in
  zero-copy mode, so it only added dead code to the hot path (Maciej).
- Link to v5: https://patch.msgid.link/20261104-igb-rx-ts-fix-v5-0-bf4b4367129a@aweta.nl

Changes in v5:
- Patch 1: v4 also subtracted the timestamp header from the build_skb
  frame limit, so XDP no longer attached at MTU 1495-1510 on x86
  (1493-1508 where NET_IP_ALIGN is 2), including the default MTU 1500.
  Only subtract the header where the limit is the hardware buffer size,
  and compute it from the adapter state instead of the ring flags. Skip
  the check when XDP is removed, so a program can always be detached.
- Patch 1: refuse XDP with rx-all and legacy-rx changes with XDP
  attached, and turn rx-all back on after XDP removal if it was
  requested meanwhile. Program RLPML and the PF pool limit from
  igb_configure() instead of igb_set_rx_mode(). With XDP attached, an
  MTU change can now move from 2 KiB to 3 KiB buffers. Added
  Suggested-by: Maciej.
- Patch 1: the issues above were also reported by the Sashiko review
  of v4.
- Patch 2: no code change. Commit message: igc is fixed by Paul's
  series, not by this patch; the loss is per continuation buffer, not
  only at the tail; the possible size wrap is described (Sashiko).
- New patch 3: AF_XDP zero-copy Rx buffer size, as asked by Maciej.
- New patch 4: drop zero-copy frames that span several descriptors;
  depends on patch 3.
- Link to v4: https://patch.msgid.link/20261102-igb-rx-ts-fix-v4-0-8785124c5d23@aweta.nl

Changes in v4:
- New patch 1: limit the receive length while XDP is attached, so that
  skb == NULL reliably marks the first buffer (Sashiko review), as
  done for igc.
- Patch 2: no code change; added a note that it relies on patch 1.
- Link to v3: https://patch.msgid.link/20260625-igb-rx-ts-fix-v3-1-99b3efa08dca@aweta.nl

---
Tjerk Kusters (3):
      igb: do not let XDP frames span more than one Rx buffer
      igb: only strip Rx timestamp header on the first buffer of a frame
      igb: do not split AF_XDP zero-copy frames over several Rx buffers

 drivers/net/ethernet/intel/igb/igb.h         |   9 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c |   9 +
 drivers/net/ethernet/intel/igb/igb_main.c    | 244 +++++++++++++++++++++------
 drivers/net/ethernet/intel/igb/igb_xsk.c     |  27 ++-
 4 files changed, 233 insertions(+), 56 deletions(-)
---
base-commit: 2d3090a8aeb596a26935db0955d46c9a5db5c6ce
change-id: 20260619-igb-rx-ts-fix-cd70585ee316

Best regards,
--  
Tjerk Kusters <tkusters@aweta.nl>



^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH net v7 1/3] igb: do not let XDP frames span more than one Rx buffer
  2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  -1 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters

igb does not support XDP multi-buffer: igb_xdp_setup() only accepts an
MTU whose frame fits in one Rx buffer, and the XDP program only sees
the first buffer of a frame. The maximum receive length is not limited
accordingly, though. RLPML is only lowered when the frame fits a
build_skb buffer on systems with PAGE_SIZE < 8192 and no VFs. Otherwise
the MAC still accepts frames of up to MAX_JUMBO_FRAME_SIZE while XDP is
attached, for example at MTU 3000 on 3 KiB buffers, and at any MTU when
PAGE_SIZE >= 8192. With VFs allocated, RLPML always stays at
MAX_JUMBO_FRAME_SIZE and the PF pool limit in VMOLR is only lowered for
frames that fit a build_skb buffer, so the same happens at MTU 3000.

Such frames span several descriptors. When the program does not return
XDP_PASS for the first buffer, no skb is built, and igb_clean_rx_irq()
handles the next buffer of the same frame as the start of a new frame:
it runs the program on it and may pass it to the stack as a bogus frame
made from the middle of the original payload.

Limit the receive length to the configured maximum frame size while XDP
is attached: in RLPML when no VFs are allocated, and in the PF pool's
VMOLR otherwise, as RLPML then has to stay large for the VFs. Larger
frames are dropped by the MAC.

Also make sure that the limit holds:

- Leave room for the inline Rx timestamp header when checking that the
  frame fits in one Rx buffer, on XDP setup and on MTU change, where the
  limit is the hardware buffer size. This lowers the largest MTU
  accepted with XDP by 16 bytes: from 3046 to 3030 on 3 KiB buffers,
  and from 2022 to 2006 with legacy-rx or PAGE_SIZE >= 8192. Setups that
  use XDP at those MTUs will have to lower the MTU. The build_skb frame
  limit used with 2 KiB buffers is kept, so XDP still attaches up to MTU
  1510 on x86. The header is only prepended by 82580 and later MACs; the
  room is reserved on all MACs for simplicity.
- Base that check on the buffer size that igb_set_rx_buffer_len() will
  select, not on the ring flags, which are only updated when the rings
  are configured and can be stale on an interface that is down. As a
  result, an MTU change with XDP attached can now also move from 2 KiB
  to 3 KiB buffers. The check is skipped when XDP is removed.
- Refuse XDP while rx-all is enabled, and keep rx-all off while XDP is
  attached, as the MAC then also stores frames longer than RLPML. If
  rx-all was requested meanwhile, it is turned on when XDP is removed,
  unless the interface could not be opened again. A runtime-suspended
  device is resumed for that first, as ethtool does, and the features
  are left as they are if the resume fails.
- Refuse to change the legacy-rx private flag while XDP is attached, as
  that changes the Rx buffer size without checking it against the MTU.
- Program both limits from igb_configure(), which runs when the
  interface is (re)configured, before Rx is enabled, and no longer from
  igb_set_rx_mode(), which can also run without RTNL, for example on an
  IPv6 multicast join or a VF mailbox request. Otherwise such a call
  during an XDP removal or an MTU change could apply the new limit while
  the rings still use the old buffers.
  Attaching or removing an XDP program on a running interface goes
  through igb_close() and igb_open(), and an MTU change through
  igb_down() and igb_up(), so igb_configure() runs in both cases.
  igb_set_rx_mode() still updates other bits of the PF pool's VMOLR. The
  core calls it under the address lock, so the limit is changed under
  that lock. The VF mailbox path calls it from the interrupt handler
  without the lock, but interrupts are masked while igb_configure()
  runs.

This follows the same approach as the igc fix that limits RLPML while
XDP is attached. igb_set_rx_mode() does not touch VMOLR on i354, but
i354 can have VFs, and then the PF pool limit is the only one. So it is
also written on i354 when VFs are allocated, with or without XDP.

Tested on an I350 at MTU 3000 with Rx timestamping on and xdp-filter
dropping all IPv4 from the peer: 8242-byte frames are now dropped by the
MAC (rx_long_length_errors), instead of their last two buffers being
delivered as a separate frame. XDP attach at MTU 1500 and 3030 still
works, at MTU 3046 it is refused, as is an MTU change from 3030 to 3046
with XDP attached; an MTU change from 1500 to 3000 with XDP attached
now works. With XDP attached, rx-all stays off and the legacy-rx flag
cannot be changed. XDP attach is refused with rx-all on, and at MTU
3000 with legacy-rx on, also when the flag was set while the interface
was down. MTU changes between 1500 and 3000 with XDP attached, while
the peer sent 3000-byte zero-payload frames, let no frame through as a
separate frame. On 7.3.1, removing XDP from a down, runtime-suspended
interface with rx-all requested resumed the device and turned rx-all
on, and the link came up normally afterwards.

With two VFs allocated, the same frames are counted by the MAC but
dropped by the PF pool limit in VMOLR before they reach the Rx queue;
without this patch their last two buffers are again delivered as a
separate frame. Read back from the registers, RLPML then stays at 9728
and the PF pool limit is 3026 at MTU 3000 and 1526 at MTU 1500 with XDP
attached. In repeated runs of about 200 MTU changes each, the limit was
right after every change, except for one read in one run, where the
value was not logged; eleven reruns showed no mismatch.

The i354 path was checked with the driver built so that the I350 takes
the i354 branches for VMOLR. With two VFs and XDP attached, the PF pool
limit kept its old value of 9728 when the VMOLR write was skipped on
i354, and read back as 3026 at MTU 3000 and 1526 at MTU 1500 with this
patch. Only the registers were checked; I have no i354 to test on.

Link: https://lore.kernel.org/all/20261028083344.3047350-3-p@1g4.org/
Fixes: 9cbc948b5a20 ("igb: add XDP support")
Cc: stable@vger.kernel.org
Suggested-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb_ethtool.c |   9 ++
 drivers/net/ethernet/intel/igb/igb_main.c    | 161 ++++++++++++++++++++-------
 2 files changed, 127 insertions(+), 43 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_ethtool.c b/drivers/net/ethernet/intel/igb/igb_ethtool.c
index f7938c1da835..4f2891fca282 100644
--- a/drivers/net/ethernet/intel/igb/igb_ethtool.c
+++ b/drivers/net/ethernet/intel/igb/igb_ethtool.c
@@ -3438,6 +3438,15 @@ static int igb_set_priv_flags(struct net_device *netdev, u32 priv_flags)
 		flags |= IGB_FLAG_RX_LEGACY;
 
 	if (flags != adapter->flags) {
+		/* the Rx buffer size depends on this flag, see
+		 * igb_xdp_setup()
+		 */
+		if (igb_xdp_is_enabled(adapter)) {
+			netdev_warn(netdev,
+				    "Cannot change legacy-rx while XDP is attached\n");
+			return -EOPNOTSUPP;
+		}
+
 		adapter->flags = flags;
 
 		/* reset interface to repopulate queues */
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index ce91dda00ec0..5f1a5e2240e6 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1964,6 +1964,57 @@ static void igb_setup_tx_mode(struct igb_adapter *adapter)
 		   "enabled" : "disabled");
 }
 
+/**
+ *  igb_set_rlpml - set the receive length limits
+ *  @adapter: private board structure
+ *
+ *  Called from igb_configure(), before Rx is enabled. igb_set_rx_mode() can
+ *  also run without RTNL, so it does not change these limits.
+ **/
+static void igb_set_rlpml(struct igb_adapter *adapter)
+{
+	u32 rlpml = MAX_JUMBO_FRAME_SIZE, pool_rlpml = MAX_JUMBO_FRAME_SIZE;
+	unsigned int vfn = adapter->vfs_allocated_count;
+	struct e1000_hw *hw = &adapter->hw;
+	u32 vmolr;
+
+#if (PAGE_SIZE < 8192)
+	/* restrict as needed to support build_skb */
+	if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB) {
+		pool_rlpml = IGB_MAX_FRAME_BUILD_SKB;
+		if (!vfn)
+			rlpml = IGB_MAX_FRAME_BUILD_SKB;
+	}
+#endif
+	/* XDP does not support frames that span more than one Rx buffer.
+	 * With VFs the global limit stays large and the PF pool is limited
+	 * through VMOLR.
+	 */
+	if (igb_xdp_is_enabled(adapter)) {
+		pool_rlpml = min(pool_rlpml, adapter->max_frame_size);
+		if (!vfn)
+			rlpml = min(rlpml, adapter->max_frame_size);
+	}
+	wr32(E1000_RLPML, rlpml);
+
+	/* the PF pool's VMOLR is used on these MACs, see igb_set_rx_mode(),
+	 * and with VFs it carries the only limit for the PF, also on i354
+	 */
+	if (hw->mac.type < e1000_82576 ||
+	    (hw->mac.type > e1000_i350 && !vfn))
+		return;
+
+	/* igb_set_rx_mode() also updates this register, and the core calls
+	 * it under the address lock
+	 */
+	netif_addr_lock_bh(adapter->netdev);
+	vmolr = rd32(E1000_VMOLR(vfn));
+	vmolr &= ~E1000_VMOLR_RLPML_MASK;
+	vmolr |= pool_rlpml | E1000_VMOLR_LPE;
+	wr32(E1000_VMOLR(vfn), vmolr);
+	netif_addr_unlock_bh(adapter->netdev);
+}
+
 /**
  *  igb_configure - configure the hardware for RX and TX
  *  @adapter: private board structure
@@ -1981,6 +2032,7 @@ static void igb_configure(struct igb_adapter *adapter)
 
 	igb_setup_tctl(adapter);
 	igb_setup_mrqc(adapter);
+	igb_set_rlpml(adapter);
 	igb_setup_rctl(adapter);
 
 	igb_nfc_filter_restore(adapter);
@@ -2468,6 +2520,14 @@ void igb_reset(struct igb_adapter *adapter)
 static netdev_features_t igb_fix_features(struct net_device *netdev,
 	netdev_features_t features)
 {
+	struct igb_adapter *adapter = netdev_priv(netdev);
+
+	/* with rx-all the MAC also stores frames longer than RLPML, which
+	 * XDP cannot handle
+	 */
+	if (igb_xdp_is_enabled(adapter))
+		features &= ~NETIF_F_RXALL;
+
 	/* Since there is no support for separate Rx/Tx vlan accel
 	 * enable/disable make sure Tx flag is always in same state as Rx.
 	 */
@@ -2886,6 +2946,27 @@ static int igb_setup_tc(struct net_device *dev, enum tc_setup_type type,
 	}
 }
 
+/* Check that a frame fits in one Rx buffer together with the inline Rx
+ * timestamp header, for the buffer size igb_set_rx_buffer_len() selects.
+ * The ring flags are only updated when the rings are configured, so they
+ * are not used here. For build_skb rings the frame is limited to
+ * IGB_MAX_FRAME_BUILD_SKB, which is below the 2 KiB hardware buffer size
+ * even with the header.
+ */
+static bool igb_xdp_frame_fits(struct igb_adapter *adapter, int max_frame)
+{
+#if (PAGE_SIZE < 8192)
+	if (!(adapter->flags & IGB_FLAG_RX_LEGACY)) {
+		if (max_frame <= IGB_MAX_FRAME_BUILD_SKB &&
+		    !IGB_2K_TOO_SMALL_WITH_PADDING)
+			return true;
+
+		return max_frame + IGB_TS_HDR_LEN <= IGB_RXBUFFER_3072;
+	}
+#endif
+	return max_frame + IGB_TS_HDR_LEN <= IGB_RXBUFFER_2048;
+}
+
 static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 {
 	int i, frame_size = dev->mtu + IGB_ETH_PKT_HDR_PAD;
@@ -2893,18 +2974,23 @@ static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 	struct bpf_prog *prog = bpf->prog, *old_prog;
 	bool running = netif_running(dev);
 	bool need_reset;
+	int err = 0;
 
-	/* verify igb ring attributes are sufficient for XDP */
-	for (i = 0; i < adapter->num_rx_queues; i++) {
-		struct igb_ring *ring = adapter->rx_ring[i];
+	/* with rx-all the MAC also stores frames longer than RLPML */
+	if (prog && (dev->features & NETIF_F_RXALL)) {
+		NL_SET_ERR_MSG_MOD(bpf->extack, "XDP is not supported with rx-all");
+		return -EOPNOTSUPP;
+	}
 
-		if (frame_size > igb_rx_bufsz(ring)) {
-			NL_SET_ERR_MSG_MOD(bpf->extack,
-					   "The RX buffer size is too small for the frame size");
-			netdev_warn(dev, "XDP RX buffer size %d is too small for the frame size %d\n",
-				    igb_rx_bufsz(ring), frame_size);
-			return -EINVAL;
-		}
+	/* verify igb ring attributes are sufficient for XDP, leaving room
+	 * for the inline Rx timestamp header
+	 */
+	if (prog && !igb_xdp_frame_fits(adapter, frame_size)) {
+		NL_SET_ERR_MSG_MOD(bpf->extack,
+				   "The RX buffer size is too small for the frame size");
+		netdev_warn(dev, "XDP RX buffer size is too small for the frame size %d\n",
+			    frame_size);
+		return -EINVAL;
 	}
 
 	old_prog = xchg(&adapter->xdp_prog, prog);
@@ -2933,7 +3019,20 @@ static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 	}
 
 	if (running)
-		igb_open(dev);
+		err = igb_open(dev);
+
+	/* rx-all may have been requested while XDP was attached. A failed
+	 * igb_open() leaves NAPI disabled, so do not reconfigure then. A down
+	 * interface can be runtime suspended; resume it first, as ethtool
+	 * does before it changes features.
+	 */
+	if (!prog && !err && (dev->wanted_features & NETIF_F_RXALL)) {
+		pm_runtime_get_sync(&adapter->pdev->dev);
+		/* still detached if the resume failed */
+		if (netif_device_present(dev))
+			netdev_update_features(dev);
+		pm_runtime_put(&adapter->pdev->dev);
+	}
 
 	return 0;
 }
@@ -5323,7 +5422,7 @@ static void igb_set_rx_mode(struct net_device *netdev)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	struct e1000_hw *hw = &adapter->hw;
 	unsigned int vfn = adapter->vfs_allocated_count;
-	u32 rctl = 0, vmolr = 0, rlpml = MAX_JUMBO_FRAME_SIZE;
+	u32 rctl = 0, vmolr = 0;
 	int count;
 
 	/* Check for Promiscuous and All Multicast modes */
@@ -5380,14 +5479,6 @@ static void igb_set_rx_mode(struct net_device *netdev)
 				     E1000_RCTL_VFE);
 	wr32(E1000_RCTL, rctl);
 
-#if (PAGE_SIZE < 8192)
-	if (!adapter->vfs_allocated_count) {
-		if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB)
-			rlpml = IGB_MAX_FRAME_BUILD_SKB;
-	}
-#endif
-	wr32(E1000_RLPML, rlpml);
-
 	/* In order to support SR-IOV and eventually VMDq it is necessary to set
 	 * the VMOLR to enable the appropriate modes.  Without this workaround
 	 * we will have issues with VLAN tag stripping not being done for frames
@@ -5399,19 +5490,10 @@ static void igb_set_rx_mode(struct net_device *netdev)
 	/* set UTA to appropriate mode */
 	igb_set_uta(adapter, !!(vmolr & E1000_VMOLR_ROPE));
 
+	/* keep the receive length limit set by igb_set_rlpml() */
 	vmolr |= rd32(E1000_VMOLR(vfn)) &
 		 ~(E1000_VMOLR_ROPE | E1000_VMOLR_MPME | E1000_VMOLR_ROMPE);
 
-	/* enable Rx jumbo frames, restrict as needed to support build_skb */
-	vmolr &= ~E1000_VMOLR_RLPML_MASK;
-#if (PAGE_SIZE < 8192)
-	if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB)
-		vmolr |= IGB_MAX_FRAME_BUILD_SKB;
-	else
-#endif
-		vmolr |= MAX_JUMBO_FRAME_SIZE;
-	vmolr |= E1000_VMOLR_LPE;
-
 	wr32(E1000_VMOLR(vfn), vmolr);
 
 	igb_restore_vf_multicasts(adapter);
@@ -6707,19 +6789,12 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int max_frame = new_mtu + IGB_ETH_PKT_HDR_PAD;
 
-	if (igb_xdp_is_enabled(adapter)) {
-		int i;
-
-		for (i = 0; i < adapter->num_rx_queues; i++) {
-			struct igb_ring *ring = adapter->rx_ring[i];
-
-			if (max_frame > igb_rx_bufsz(ring)) {
-				netdev_warn(adapter->netdev,
-					    "Requested MTU size is not supported with XDP. Max frame size is %d\n",
-					    max_frame);
-				return -EINVAL;
-			}
-		}
+	if (igb_xdp_is_enabled(adapter) &&
+	    !igb_xdp_frame_fits(adapter, max_frame)) {
+		netdev_warn(adapter->netdev,
+			    "Requested MTU size is not supported with XDP. Frame size is %d\n",
+			    max_frame);
+		return -EINVAL;
 	}
 
 	/* adjust max frame to be at least the size of a standard frame */

-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH net v7 1/3] igb: do not let XDP frames span more than one Rx buffer
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters via B4 Relay @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters

From: Tjerk Kusters <tkusters@aweta.nl>

igb does not support XDP multi-buffer: igb_xdp_setup() only accepts an
MTU whose frame fits in one Rx buffer, and the XDP program only sees
the first buffer of a frame. The maximum receive length is not limited
accordingly, though. RLPML is only lowered when the frame fits a
build_skb buffer on systems with PAGE_SIZE < 8192 and no VFs. Otherwise
the MAC still accepts frames of up to MAX_JUMBO_FRAME_SIZE while XDP is
attached, for example at MTU 3000 on 3 KiB buffers, and at any MTU when
PAGE_SIZE >= 8192. With VFs allocated, RLPML always stays at
MAX_JUMBO_FRAME_SIZE and the PF pool limit in VMOLR is only lowered for
frames that fit a build_skb buffer, so the same happens at MTU 3000.

Such frames span several descriptors. When the program does not return
XDP_PASS for the first buffer, no skb is built, and igb_clean_rx_irq()
handles the next buffer of the same frame as the start of a new frame:
it runs the program on it and may pass it to the stack as a bogus frame
made from the middle of the original payload.

Limit the receive length to the configured maximum frame size while XDP
is attached: in RLPML when no VFs are allocated, and in the PF pool's
VMOLR otherwise, as RLPML then has to stay large for the VFs. Larger
frames are dropped by the MAC.

Also make sure that the limit holds:

- Leave room for the inline Rx timestamp header when checking that the
  frame fits in one Rx buffer, on XDP setup and on MTU change, where the
  limit is the hardware buffer size. This lowers the largest MTU
  accepted with XDP by 16 bytes: from 3046 to 3030 on 3 KiB buffers,
  and from 2022 to 2006 with legacy-rx or PAGE_SIZE >= 8192. Setups that
  use XDP at those MTUs will have to lower the MTU. The build_skb frame
  limit used with 2 KiB buffers is kept, so XDP still attaches up to MTU
  1510 on x86. The header is only prepended by 82580 and later MACs; the
  room is reserved on all MACs for simplicity.
- Base that check on the buffer size that igb_set_rx_buffer_len() will
  select, not on the ring flags, which are only updated when the rings
  are configured and can be stale on an interface that is down. As a
  result, an MTU change with XDP attached can now also move from 2 KiB
  to 3 KiB buffers. The check is skipped when XDP is removed.
- Refuse XDP while rx-all is enabled, and keep rx-all off while XDP is
  attached, as the MAC then also stores frames longer than RLPML. If
  rx-all was requested meanwhile, it is turned on when XDP is removed,
  unless the interface could not be opened again. A runtime-suspended
  device is resumed for that first, as ethtool does, and the features
  are left as they are if the resume fails.
- Refuse to change the legacy-rx private flag while XDP is attached, as
  that changes the Rx buffer size without checking it against the MTU.
- Program both limits from igb_configure(), which runs when the
  interface is (re)configured, before Rx is enabled, and no longer from
  igb_set_rx_mode(), which can also run without RTNL, for example on an
  IPv6 multicast join or a VF mailbox request. Otherwise such a call
  during an XDP removal or an MTU change could apply the new limit while
  the rings still use the old buffers.
  Attaching or removing an XDP program on a running interface goes
  through igb_close() and igb_open(), and an MTU change through
  igb_down() and igb_up(), so igb_configure() runs in both cases.
  igb_set_rx_mode() still updates other bits of the PF pool's VMOLR. The
  core calls it under the address lock, so the limit is changed under
  that lock. The VF mailbox path calls it from the interrupt handler
  without the lock, but interrupts are masked while igb_configure()
  runs.

This follows the same approach as the igc fix that limits RLPML while
XDP is attached. igb_set_rx_mode() does not touch VMOLR on i354, but
i354 can have VFs, and then the PF pool limit is the only one. So it is
also written on i354 when VFs are allocated, with or without XDP.

Tested on an I350 at MTU 3000 with Rx timestamping on and xdp-filter
dropping all IPv4 from the peer: 8242-byte frames are now dropped by the
MAC (rx_long_length_errors), instead of their last two buffers being
delivered as a separate frame. XDP attach at MTU 1500 and 3030 still
works, at MTU 3046 it is refused, as is an MTU change from 3030 to 3046
with XDP attached; an MTU change from 1500 to 3000 with XDP attached
now works. With XDP attached, rx-all stays off and the legacy-rx flag
cannot be changed. XDP attach is refused with rx-all on, and at MTU
3000 with legacy-rx on, also when the flag was set while the interface
was down. MTU changes between 1500 and 3000 with XDP attached, while
the peer sent 3000-byte zero-payload frames, let no frame through as a
separate frame. On 7.3.1, removing XDP from a down, runtime-suspended
interface with rx-all requested resumed the device and turned rx-all
on, and the link came up normally afterwards.

With two VFs allocated, the same frames are counted by the MAC but
dropped by the PF pool limit in VMOLR before they reach the Rx queue;
without this patch their last two buffers are again delivered as a
separate frame. Read back from the registers, RLPML then stays at 9728
and the PF pool limit is 3026 at MTU 3000 and 1526 at MTU 1500 with XDP
attached. In repeated runs of about 200 MTU changes each, the limit was
right after every change, except for one read in one run, where the
value was not logged; eleven reruns showed no mismatch.

The i354 path was checked with the driver built so that the I350 takes
the i354 branches for VMOLR. With two VFs and XDP attached, the PF pool
limit kept its old value of 9728 when the VMOLR write was skipped on
i354, and read back as 3026 at MTU 3000 and 1526 at MTU 1500 with this
patch. Only the registers were checked; I have no i354 to test on.

Link: https://lore.kernel.org/all/20261028083344.3047350-3-p@1g4.org/
Fixes: 9cbc948b5a20 ("igb: add XDP support")
Cc: stable@vger.kernel.org
Suggested-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb_ethtool.c |   9 ++
 drivers/net/ethernet/intel/igb/igb_main.c    | 161 ++++++++++++++++++++-------
 2 files changed, 127 insertions(+), 43 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_ethtool.c b/drivers/net/ethernet/intel/igb/igb_ethtool.c
index f7938c1da835..4f2891fca282 100644
--- a/drivers/net/ethernet/intel/igb/igb_ethtool.c
+++ b/drivers/net/ethernet/intel/igb/igb_ethtool.c
@@ -3438,6 +3438,15 @@ static int igb_set_priv_flags(struct net_device *netdev, u32 priv_flags)
 		flags |= IGB_FLAG_RX_LEGACY;
 
 	if (flags != adapter->flags) {
+		/* the Rx buffer size depends on this flag, see
+		 * igb_xdp_setup()
+		 */
+		if (igb_xdp_is_enabled(adapter)) {
+			netdev_warn(netdev,
+				    "Cannot change legacy-rx while XDP is attached\n");
+			return -EOPNOTSUPP;
+		}
+
 		adapter->flags = flags;
 
 		/* reset interface to repopulate queues */
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index ce91dda00ec0..5f1a5e2240e6 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1964,6 +1964,57 @@ static void igb_setup_tx_mode(struct igb_adapter *adapter)
 		   "enabled" : "disabled");
 }
 
+/**
+ *  igb_set_rlpml - set the receive length limits
+ *  @adapter: private board structure
+ *
+ *  Called from igb_configure(), before Rx is enabled. igb_set_rx_mode() can
+ *  also run without RTNL, so it does not change these limits.
+ **/
+static void igb_set_rlpml(struct igb_adapter *adapter)
+{
+	u32 rlpml = MAX_JUMBO_FRAME_SIZE, pool_rlpml = MAX_JUMBO_FRAME_SIZE;
+	unsigned int vfn = adapter->vfs_allocated_count;
+	struct e1000_hw *hw = &adapter->hw;
+	u32 vmolr;
+
+#if (PAGE_SIZE < 8192)
+	/* restrict as needed to support build_skb */
+	if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB) {
+		pool_rlpml = IGB_MAX_FRAME_BUILD_SKB;
+		if (!vfn)
+			rlpml = IGB_MAX_FRAME_BUILD_SKB;
+	}
+#endif
+	/* XDP does not support frames that span more than one Rx buffer.
+	 * With VFs the global limit stays large and the PF pool is limited
+	 * through VMOLR.
+	 */
+	if (igb_xdp_is_enabled(adapter)) {
+		pool_rlpml = min(pool_rlpml, adapter->max_frame_size);
+		if (!vfn)
+			rlpml = min(rlpml, adapter->max_frame_size);
+	}
+	wr32(E1000_RLPML, rlpml);
+
+	/* the PF pool's VMOLR is used on these MACs, see igb_set_rx_mode(),
+	 * and with VFs it carries the only limit for the PF, also on i354
+	 */
+	if (hw->mac.type < e1000_82576 ||
+	    (hw->mac.type > e1000_i350 && !vfn))
+		return;
+
+	/* igb_set_rx_mode() also updates this register, and the core calls
+	 * it under the address lock
+	 */
+	netif_addr_lock_bh(adapter->netdev);
+	vmolr = rd32(E1000_VMOLR(vfn));
+	vmolr &= ~E1000_VMOLR_RLPML_MASK;
+	vmolr |= pool_rlpml | E1000_VMOLR_LPE;
+	wr32(E1000_VMOLR(vfn), vmolr);
+	netif_addr_unlock_bh(adapter->netdev);
+}
+
 /**
  *  igb_configure - configure the hardware for RX and TX
  *  @adapter: private board structure
@@ -1981,6 +2032,7 @@ static void igb_configure(struct igb_adapter *adapter)
 
 	igb_setup_tctl(adapter);
 	igb_setup_mrqc(adapter);
+	igb_set_rlpml(adapter);
 	igb_setup_rctl(adapter);
 
 	igb_nfc_filter_restore(adapter);
@@ -2468,6 +2520,14 @@ void igb_reset(struct igb_adapter *adapter)
 static netdev_features_t igb_fix_features(struct net_device *netdev,
 	netdev_features_t features)
 {
+	struct igb_adapter *adapter = netdev_priv(netdev);
+
+	/* with rx-all the MAC also stores frames longer than RLPML, which
+	 * XDP cannot handle
+	 */
+	if (igb_xdp_is_enabled(adapter))
+		features &= ~NETIF_F_RXALL;
+
 	/* Since there is no support for separate Rx/Tx vlan accel
 	 * enable/disable make sure Tx flag is always in same state as Rx.
 	 */
@@ -2886,6 +2946,27 @@ static int igb_setup_tc(struct net_device *dev, enum tc_setup_type type,
 	}
 }
 
+/* Check that a frame fits in one Rx buffer together with the inline Rx
+ * timestamp header, for the buffer size igb_set_rx_buffer_len() selects.
+ * The ring flags are only updated when the rings are configured, so they
+ * are not used here. For build_skb rings the frame is limited to
+ * IGB_MAX_FRAME_BUILD_SKB, which is below the 2 KiB hardware buffer size
+ * even with the header.
+ */
+static bool igb_xdp_frame_fits(struct igb_adapter *adapter, int max_frame)
+{
+#if (PAGE_SIZE < 8192)
+	if (!(adapter->flags & IGB_FLAG_RX_LEGACY)) {
+		if (max_frame <= IGB_MAX_FRAME_BUILD_SKB &&
+		    !IGB_2K_TOO_SMALL_WITH_PADDING)
+			return true;
+
+		return max_frame + IGB_TS_HDR_LEN <= IGB_RXBUFFER_3072;
+	}
+#endif
+	return max_frame + IGB_TS_HDR_LEN <= IGB_RXBUFFER_2048;
+}
+
 static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 {
 	int i, frame_size = dev->mtu + IGB_ETH_PKT_HDR_PAD;
@@ -2893,18 +2974,23 @@ static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 	struct bpf_prog *prog = bpf->prog, *old_prog;
 	bool running = netif_running(dev);
 	bool need_reset;
+	int err = 0;
 
-	/* verify igb ring attributes are sufficient for XDP */
-	for (i = 0; i < adapter->num_rx_queues; i++) {
-		struct igb_ring *ring = adapter->rx_ring[i];
+	/* with rx-all the MAC also stores frames longer than RLPML */
+	if (prog && (dev->features & NETIF_F_RXALL)) {
+		NL_SET_ERR_MSG_MOD(bpf->extack, "XDP is not supported with rx-all");
+		return -EOPNOTSUPP;
+	}
 
-		if (frame_size > igb_rx_bufsz(ring)) {
-			NL_SET_ERR_MSG_MOD(bpf->extack,
-					   "The RX buffer size is too small for the frame size");
-			netdev_warn(dev, "XDP RX buffer size %d is too small for the frame size %d\n",
-				    igb_rx_bufsz(ring), frame_size);
-			return -EINVAL;
-		}
+	/* verify igb ring attributes are sufficient for XDP, leaving room
+	 * for the inline Rx timestamp header
+	 */
+	if (prog && !igb_xdp_frame_fits(adapter, frame_size)) {
+		NL_SET_ERR_MSG_MOD(bpf->extack,
+				   "The RX buffer size is too small for the frame size");
+		netdev_warn(dev, "XDP RX buffer size is too small for the frame size %d\n",
+			    frame_size);
+		return -EINVAL;
 	}
 
 	old_prog = xchg(&adapter->xdp_prog, prog);
@@ -2933,7 +3019,20 @@ static int igb_xdp_setup(struct net_device *dev, struct netdev_bpf *bpf)
 	}
 
 	if (running)
-		igb_open(dev);
+		err = igb_open(dev);
+
+	/* rx-all may have been requested while XDP was attached. A failed
+	 * igb_open() leaves NAPI disabled, so do not reconfigure then. A down
+	 * interface can be runtime suspended; resume it first, as ethtool
+	 * does before it changes features.
+	 */
+	if (!prog && !err && (dev->wanted_features & NETIF_F_RXALL)) {
+		pm_runtime_get_sync(&adapter->pdev->dev);
+		/* still detached if the resume failed */
+		if (netif_device_present(dev))
+			netdev_update_features(dev);
+		pm_runtime_put(&adapter->pdev->dev);
+	}
 
 	return 0;
 }
@@ -5323,7 +5422,7 @@ static void igb_set_rx_mode(struct net_device *netdev)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	struct e1000_hw *hw = &adapter->hw;
 	unsigned int vfn = adapter->vfs_allocated_count;
-	u32 rctl = 0, vmolr = 0, rlpml = MAX_JUMBO_FRAME_SIZE;
+	u32 rctl = 0, vmolr = 0;
 	int count;
 
 	/* Check for Promiscuous and All Multicast modes */
@@ -5380,14 +5479,6 @@ static void igb_set_rx_mode(struct net_device *netdev)
 				     E1000_RCTL_VFE);
 	wr32(E1000_RCTL, rctl);
 
-#if (PAGE_SIZE < 8192)
-	if (!adapter->vfs_allocated_count) {
-		if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB)
-			rlpml = IGB_MAX_FRAME_BUILD_SKB;
-	}
-#endif
-	wr32(E1000_RLPML, rlpml);
-
 	/* In order to support SR-IOV and eventually VMDq it is necessary to set
 	 * the VMOLR to enable the appropriate modes.  Without this workaround
 	 * we will have issues with VLAN tag stripping not being done for frames
@@ -5399,19 +5490,10 @@ static void igb_set_rx_mode(struct net_device *netdev)
 	/* set UTA to appropriate mode */
 	igb_set_uta(adapter, !!(vmolr & E1000_VMOLR_ROPE));
 
+	/* keep the receive length limit set by igb_set_rlpml() */
 	vmolr |= rd32(E1000_VMOLR(vfn)) &
 		 ~(E1000_VMOLR_ROPE | E1000_VMOLR_MPME | E1000_VMOLR_ROMPE);
 
-	/* enable Rx jumbo frames, restrict as needed to support build_skb */
-	vmolr &= ~E1000_VMOLR_RLPML_MASK;
-#if (PAGE_SIZE < 8192)
-	if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB)
-		vmolr |= IGB_MAX_FRAME_BUILD_SKB;
-	else
-#endif
-		vmolr |= MAX_JUMBO_FRAME_SIZE;
-	vmolr |= E1000_VMOLR_LPE;
-
 	wr32(E1000_VMOLR(vfn), vmolr);
 
 	igb_restore_vf_multicasts(adapter);
@@ -6707,19 +6789,12 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int max_frame = new_mtu + IGB_ETH_PKT_HDR_PAD;
 
-	if (igb_xdp_is_enabled(adapter)) {
-		int i;
-
-		for (i = 0; i < adapter->num_rx_queues; i++) {
-			struct igb_ring *ring = adapter->rx_ring[i];
-
-			if (max_frame > igb_rx_bufsz(ring)) {
-				netdev_warn(adapter->netdev,
-					    "Requested MTU size is not supported with XDP. Max frame size is %d\n",
-					    max_frame);
-				return -EINVAL;
-			}
-		}
+	if (igb_xdp_is_enabled(adapter) &&
+	    !igb_xdp_frame_fits(adapter, max_frame)) {
+		netdev_warn(adapter->netdev,
+			    "Requested MTU size is not supported with XDP. Frame size is %d\n",
+			    max_frame);
+		return -EINVAL;
 	}
 
 	/* adjust max frame to be at least the size of a standard frame */

-- 
2.27.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH net v7 2/3] igb: only strip Rx timestamp header on the first buffer of a frame
  2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  -1 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Piotr Kwapulinski, Aleksandr Loktionov,
	Alexander Nowlin, Tjerk Kusters

When Rx hardware timestamping is enabled (e.g. ptp4l, which configures
HWTSTAMP_FILTER_ALL), the NIC prepends a 16-byte timestamp header to the
first Rx buffer of every received frame. igb_clean_rx_irq() strips this
header inside its per-buffer loop:

	if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
		ts_hdr_len = igb_ptp_rx_pktstamp(rx_ring->q_vector,
						 pktbuf, &timestamp);
		pkt_offset += ts_hdr_len;
		size -= ts_hdr_len;
	}

For a frame that spans more than one Rx buffer (e.g. a jumbo frame), this
block runs once per buffer. The timestamp header only exists at the start
of the first buffer, but igb_ptp_rx_pktstamp() is called for every buffer.

On a continuation buffer the data is packet payload, not a timestamp
header. igb_ptp_rx_pktstamp() already has two guards against acting on a
non-header buffer: it returns 0 if PTP is disabled, and returns 0 if the
reserved dwords (the first 8 bytes) are non-zero. Neither is sufficient
here: PTP is enabled, and a continuation buffer whose payload happens to
begin with 8 zero bytes passes the reserved-dword check. In that case the
payload is mistaken for a valid timestamp header and igb_ptp_rx_pktstamp()
returns IGB_TS_HDR_LEN, and the caller reduces that buffer's size by 16.
igb_add_rx_frag() adds a continuation buffer from its original offset, so
the buffer loses its last 16 bytes and the data of the next buffer
follows directly. A frame spanning N buffers loses 16 bytes per affected
continuation buffer, in the middle of the payload when N > 2. If the last
buffer of a frame holds 8 to 15 bytes, all zero, the unsigned size wraps
and a fragment of about 4 GiB is added to the skb. The sender chooses the
frame length and content, so this can be triggered remotely.

This is easily triggered by a GigE Vision camera streaming dark frames
(mostly 0x00 pixel data) over jumbo UDP with PTP active on the receiver:
the all-zero frames are dropped as truncated while frames with non-zero
content are fine. The driver reports no error; only the IP layer counts
the frames in IpExtInTruncatedPkts.

No content-based check can reliably tell a continuation buffer that begins
with zero bytes from a real timestamp header, because both are all zero.
Fix it structurally instead: only attempt the strip on the first buffer of
a frame, which is the only buffer that can contain a timestamp header. In
igb_clean_rx_irq() skb is NULL until the first buffer has been processed,
so guarding the strip with !skb restricts it to the first buffer
regardless of payload content.

This relies on the previous patch. Without it, an XDP program that does
not return XDP_PASS for the first buffer of a multi-buffer frame leaves
skb NULL for the rest of that frame. With the receive length limited
while XDP is attached, such frames no longer reach the XDP path.

igc has the same bug; Paul Moses' igc series, linked below, fixes it the
same way.

Link: https://lore.kernel.org/all/20261028083344.3047350-2-p@1g4.org/
Fixes: 5379260852b0 ("igb: Fix XDP with PTP enabled")
Cc: stable@vger.kernel.org
Reviewed-by: Piotr Kwapulinski <piotr.kwapulinski@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Reviewed-by: Kurt Kanzenbach <kurt@linutronix.de>
Tested-by: Alexander Nowlin <alexander.nowlin@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb_main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 5f1a5e2240e6..a59ee6575c56 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -9135,8 +9135,11 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
 		rx_buffer = igb_get_rx_buffer(rx_ring, size, &rx_buf_pgcnt);
 		pktbuf = page_address(rx_buffer->page) + rx_buffer->page_offset;
 
-		/* pull rx packet timestamp if available and valid */
-		if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
+		/* pull rx packet timestamp if available and valid; it is only
+		 * present on the first buffer of a frame
+		 */
+		if (!skb &&
+		    igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
 			int ts_hdr_len;
 
 			ts_hdr_len = igb_ptp_rx_pktstamp(rx_ring->q_vector,

-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH net v7 2/3] igb: only strip Rx timestamp header on the first buffer of a frame
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters via B4 Relay @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Piotr Kwapulinski, Aleksandr Loktionov,
	Alexander Nowlin, Tjerk Kusters

From: Tjerk Kusters <tkusters@aweta.nl>

When Rx hardware timestamping is enabled (e.g. ptp4l, which configures
HWTSTAMP_FILTER_ALL), the NIC prepends a 16-byte timestamp header to the
first Rx buffer of every received frame. igb_clean_rx_irq() strips this
header inside its per-buffer loop:

	if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
		ts_hdr_len = igb_ptp_rx_pktstamp(rx_ring->q_vector,
						 pktbuf, &timestamp);
		pkt_offset += ts_hdr_len;
		size -= ts_hdr_len;
	}

For a frame that spans more than one Rx buffer (e.g. a jumbo frame), this
block runs once per buffer. The timestamp header only exists at the start
of the first buffer, but igb_ptp_rx_pktstamp() is called for every buffer.

On a continuation buffer the data is packet payload, not a timestamp
header. igb_ptp_rx_pktstamp() already has two guards against acting on a
non-header buffer: it returns 0 if PTP is disabled, and returns 0 if the
reserved dwords (the first 8 bytes) are non-zero. Neither is sufficient
here: PTP is enabled, and a continuation buffer whose payload happens to
begin with 8 zero bytes passes the reserved-dword check. In that case the
payload is mistaken for a valid timestamp header and igb_ptp_rx_pktstamp()
returns IGB_TS_HDR_LEN, and the caller reduces that buffer's size by 16.
igb_add_rx_frag() adds a continuation buffer from its original offset, so
the buffer loses its last 16 bytes and the data of the next buffer
follows directly. A frame spanning N buffers loses 16 bytes per affected
continuation buffer, in the middle of the payload when N > 2. If the last
buffer of a frame holds 8 to 15 bytes, all zero, the unsigned size wraps
and a fragment of about 4 GiB is added to the skb. The sender chooses the
frame length and content, so this can be triggered remotely.

This is easily triggered by a GigE Vision camera streaming dark frames
(mostly 0x00 pixel data) over jumbo UDP with PTP active on the receiver:
the all-zero frames are dropped as truncated while frames with non-zero
content are fine. The driver reports no error; only the IP layer counts
the frames in IpExtInTruncatedPkts.

No content-based check can reliably tell a continuation buffer that begins
with zero bytes from a real timestamp header, because both are all zero.
Fix it structurally instead: only attempt the strip on the first buffer of
a frame, which is the only buffer that can contain a timestamp header. In
igb_clean_rx_irq() skb is NULL until the first buffer has been processed,
so guarding the strip with !skb restricts it to the first buffer
regardless of payload content.

This relies on the previous patch. Without it, an XDP program that does
not return XDP_PASS for the first buffer of a multi-buffer frame leaves
skb NULL for the rest of that frame. With the receive length limited
while XDP is attached, such frames no longer reach the XDP path.

igc has the same bug; Paul Moses' igc series, linked below, fixes it the
same way.

Link: https://lore.kernel.org/all/20261028083344.3047350-2-p@1g4.org/
Fixes: 5379260852b0 ("igb: Fix XDP with PTP enabled")
Cc: stable@vger.kernel.org
Reviewed-by: Piotr Kwapulinski <piotr.kwapulinski@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Reviewed-by: Kurt Kanzenbach <kurt@linutronix.de>
Tested-by: Alexander Nowlin <alexander.nowlin@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb_main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 5f1a5e2240e6..a59ee6575c56 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -9135,8 +9135,11 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
 		rx_buffer = igb_get_rx_buffer(rx_ring, size, &rx_buf_pgcnt);
 		pktbuf = page_address(rx_buffer->page) + rx_buffer->page_offset;
 
-		/* pull rx packet timestamp if available and valid */
-		if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
+		/* pull rx packet timestamp if available and valid; it is only
+		 * present on the first buffer of a frame
+		 */
+		if (!skb &&
+		    igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
 			int ts_hdr_len;
 
 			ts_hdr_len = igb_ptp_rx_pktstamp(rx_ring->q_vector,

-- 
2.27.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH net v7 3/3] igb: do not split AF_XDP zero-copy frames over several Rx buffers
  2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  -1 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters

SRRCTL.BSIZEPKT has a granularity of 1 KiB. For AF_XDP zero-copy rings,
igb_setup_srrctl() programs xsk_pool_get_rx_frame_size(), which is
rounded down. With 2 KiB UMEM chunks and the default headroom this is
1792 bytes, so the hardware buffer becomes 1 KiB and a full-size frame
spans two Rx buffers. igb_clean_rx_irq_zc() handles every descriptor as
a complete frame, so the AF_XDP socket gets such a frame as two
packets, the second one without headers. With Rx timestamping enabled,
the timestamp header is also stripped from the second buffer, and a
short final buffer makes the size wrap.

Program the same buffer size as for the other rings, 2 KiB or 3 KiB.
This depends on "igb: do not let XDP frames span more than one Rx
buffer": the receive length limit set there while XDP is attached keeps
the hardware from writing more than the frame plus the timestamp header
into the larger buffer. Without it, the hardware can write past the XSK
frame.

Refuse an XSK pool whose frame size cannot hold that, and an MTU change
that would no longer fit a bound pool, also while no XDP program is
attached. The current MTU is checked rather than the largest XDP MTU,
as igb allows XDP up to MTU 3030 and that would rule out 2 KiB chunks.

With VFs, the PF keeps a single Tx queue on all MACs, and one Rx queue
on i350 or two on 82576. A pool bound to another queue would lose its
Tx queue, which breaks the socket, and possibly its Rx queue, which the
MTU check then no longer sees. The core refuses ethtool -L in that case
(with -EINVAL). Likewise, refuse to enable VFs with -EBUSY while an XSK
pool is bound to a queue other than 0. igb_enable_sriov() checks this
before it sets up any VF state, so nothing has to be undone on that
error, and holds RTNL until the PF queues are set up again, so that no
pool can be bound in between. For that, igb_sriov_reinit() is split
into a part that runs under RTNL and a wrapper that takes it.
pci_enable_sriov() still runs without RTNL, and disabling VFs is not
refused. The older error paths of igb_enable_sriov(), for a failed queue
setup or a failed pci_enable_sriov(), are not changed.

The real queue counts are only updated when the interface is opened.
After VFs were enabled on a down 82576 PF, a pool could still be bound
to queue 1, which then has no Tx ring, and closing the socket on the
running PF would dereference the missing ring. igb_xsk_pool_enable()
now also checks the PF's Tx queue count.

A queue can also be hidden without such a request, for example when the
driver falls back from MSI-X to MSI and keeps one queue. The MTU check
does not see a pool on that queue, so igb_xsk_pool() checks the frame
size again when the ring is set up, and does not use zero-copy if the
frame does not fit. The socket then receives through the copy path,
and its Tx wakeup fails with EINVAL, until the ring is set up again
with an MTU that fits. __igb_open() sets up the rings before it updates
the queue counts, so a queue that comes back is used in zero-copy mode
again only from the next reconfiguration; that is not changed here.
This path was not tested.

This is the igb counterpart of the igc XSK buffer change.

Tested on an I350 with xdpsock from bpf-examples in rxdrop zero-copy
mode on one queue, Rx timestamping on and 1514-byte frames from the
peer. With 2 KiB chunks, the receive descriptors went from about two
per frame to one per frame; 98-byte frames and 4 KiB chunks were one
per frame with and without this patch. At MTU 3000, 4 KiB chunks
receive 3014-byte frames in one buffer. A 2 KiB chunk pool at MTU 3000
was refused by the new check on the 6.14.9 test kernel; current kernels
already refuse it in xp_assign_dev(). There, with 2 KiB chunks and the
default headroom, the new check refuses MTU 1751 to 1766, which the core
accepts.

On 7.3.1, with xdpsock bound to queue 1 of an I350, enabling two VFs is
now refused with EBUSY and the socket keeps its queue; with the socket on
queue 0, which the PF keeps, the VFs are enabled. With 31 macvlans on
the PF, which fill all 32 receive address registers, all 31 addresses
stay valid after the refused VF enable, read back from the registers.
With the driver built so that the I350 keeps two Rx queues with VFs, as
an 82576 does, binding a zero-copy socket to queue 1 after VFs were
enabled on the down PF is now refused with EINVAL; before, it was
accepted. The socket was closed while the PF was down, so the missing
ring was not dereferenced.

Link: https://lore.kernel.org/all/20261028083344.3047350-3-p@1g4.org/
Fixes: 2c6196013f84 ("igb: Add AF_XDP zero-copy Rx support")
Cc: stable@vger.kernel.org
Suggested-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb.h      |  9 ++++
 drivers/net/ethernet/intel/igb/igb_main.c | 76 +++++++++++++++++++++++++++----
 drivers/net/ethernet/intel/igb/igb_xsk.c  | 27 ++++++++++-
 3 files changed, 101 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 0fff1df81b7b..0495c02544d6 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -856,6 +856,15 @@ int igb_add_mac_steering_filter(struct igb_adapter *adapter,
 int igb_del_mac_steering_filter(struct igb_adapter *adapter,
 				const u8 *addr, u8 queue, u8 flags);
 
+/* XDP does not support frames that span more than one Rx buffer, so a
+ * frame plus the inline Rx timestamp header must fit in one XSK buffer.
+ */
+static inline bool igb_xsk_frame_fits(struct xsk_buff_pool *pool,
+				      int max_frame)
+{
+	return max_frame + IGB_TS_HDR_LEN <= xsk_pool_get_rx_frame_size(pool);
+}
+
 struct xsk_buff_pool *igb_xsk_pool(struct igb_adapter *adapter,
 				   struct igb_ring *ring);
 int igb_xsk_pool_setup(struct igb_adapter *adapter,
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index a59ee6575c56..d3165ae97b00 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3782,13 +3782,13 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 }
 
 #ifdef CONFIG_PCI_IOV
-static int igb_sriov_reinit(struct pci_dev *dev)
+static int igb_sriov_reinit_locked(struct pci_dev *dev)
 {
 	struct net_device *netdev = pci_get_drvdata(dev);
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	struct pci_dev *pdev = adapter->pdev;
 
-	rtnl_lock();
+	ASSERT_RTNL();
 
 	if (netif_running(netdev))
 		igb_close(netdev);
@@ -3800,7 +3800,6 @@ static int igb_sriov_reinit(struct pci_dev *dev)
 	igb_init_queue_configuration(adapter);
 
 	if (igb_init_interrupt_scheme(adapter, true)) {
-		rtnl_unlock();
 		dev_err(&pdev->dev, "Unable to allocate memory for queues\n");
 		return -ENOMEM;
 	}
@@ -3808,9 +3807,18 @@ static int igb_sriov_reinit(struct pci_dev *dev)
 	if (netif_running(netdev))
 		igb_open(netdev);
 
+	return 0;
+}
+
+static int igb_sriov_reinit(struct pci_dev *dev)
+{
+	int err;
+
+	rtnl_lock();
+	err = igb_sriov_reinit_locked(dev);
 	rtnl_unlock();
 
-	return 0;
+	return err;
 }
 
 static int igb_disable_sriov(struct pci_dev *pdev, bool reinit)
@@ -3856,6 +3864,7 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int old_vfs = pci_num_vf(pdev);
 	struct vf_mac_filter *mac_list;
+	unsigned int q, end;
 	int err = 0;
 	int num_vf_mac_filters, i;
 
@@ -3866,6 +3875,27 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	if (!num_vfs)
 		goto out;
 
+	/* With VFs the PF keeps a single Tx queue, see
+	 * igb_set_interrupt_capability(). Refuse VFs while an XSK pool is
+	 * bound to another queue; the core refuses ethtool -L in the same
+	 * case. Check before any VF state is set up, and hold RTNL until the
+	 * queues are set up again, so that no pool can be bound in between.
+	 */
+	if (reinit) {
+		rtnl_lock();
+		end = max(netdev->real_num_rx_queues,
+			  netdev->real_num_tx_queues);
+		for (q = 1; q < end; q++) {
+			if (xsk_get_pool_from_qid(netdev, q)) {
+				dev_err(&pdev->dev,
+					"Cannot change the number of queues, an AF_XDP socket is bound to queue %u\n",
+					q);
+				err = -EBUSY;
+				goto err_unlock;
+			}
+		}
+	}
+
 	if (old_vfs) {
 		dev_info(&pdev->dev, "%d pre-allocated VFs found - override max_vfs setting of %d\n",
 			 old_vfs, max_vfs);
@@ -3880,7 +3910,7 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	if (!adapter->vf_data) {
 		adapter->vfs_allocated_count = 0;
 		err = -ENOMEM;
-		goto out;
+		goto err_unlock;
 	}
 
 	/* Due to the limited number of RAR entries calculate potential
@@ -3923,7 +3953,8 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	adapter->flags &= ~IGB_FLAG_DMAC;
 
 	if (reinit) {
-		err = igb_sriov_reinit(pdev);
+		err = igb_sriov_reinit_locked(pdev);
+		rtnl_unlock();
 		if (err)
 			goto err_out;
 	}
@@ -3943,6 +3974,11 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	kfree(adapter->vf_data);
 	adapter->vf_data = NULL;
 	adapter->vfs_allocated_count = 0;
+	goto out;
+
+err_unlock:
+	if (reinit)
+		rtnl_unlock();
 out:
 	return err;
 }
@@ -4846,9 +4882,13 @@ void igb_setup_srrctl(struct igb_adapter *adapter, struct igb_ring *ring)
 	u32 srrctl = 0;
 	u32 buf_size;
 
-	if (ring->xsk_pool)
-		buf_size = xsk_pool_get_rx_frame_size(ring->xsk_pool);
-	else if (ring_uses_large_buffer(ring))
+	/* SRRCTL.BSIZEPKT has a 1 KiB granularity, so programming the XSK
+	 * frame size would round it down and split frames over several
+	 * buffers. XSK rings use the same buffer size as the other rings.
+	 * While XDP is attached, the receive length limit keeps frames
+	 * within the XSK frame, see igb_xsk_frame_fits().
+	 */
+	if (ring_uses_large_buffer(ring))
 		buf_size = IGB_RXBUFFER_3072;
 	else
 		buf_size = IGB_RXBUFFER_2048;
@@ -6788,6 +6828,7 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 {
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int max_frame = new_mtu + IGB_ETH_PKT_HDR_PAD;
+	int i;
 
 	if (igb_xdp_is_enabled(adapter) &&
 	    !igb_xdp_frame_fits(adapter, max_frame)) {
@@ -6801,6 +6842,23 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 	if (max_frame < (ETH_FRAME_LEN + ETH_FCS_LEN))
 		max_frame = ETH_FRAME_LEN + ETH_FCS_LEN;
 
+	/* the frame must fit the buffers of a bound XSK pool, also while no
+	 * XDP program is attached, as attaching one does not check it again
+	 */
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		struct xsk_buff_pool *pool = xsk_get_pool_from_qid(netdev, i);
+
+		if (!pool || !pool->dev)
+			continue;
+
+		if (!igb_xsk_frame_fits(pool, max_frame)) {
+			netdev_warn(adapter->netdev,
+				    "Requested MTU size is not supported with AF_XDP. Frame size is %d\n",
+				    max_frame);
+			return -EINVAL;
+		}
+	}
+
 	while (test_and_set_bit(__IGB_RESETTING, &adapter->state))
 		usleep_range(1000, 2000);
 
diff --git a/drivers/net/ethernet/intel/igb/igb_xsk.c b/drivers/net/ethernet/intel/igb/igb_xsk.c
index ce4a7b58cad2..43b33a536efe 100644
--- a/drivers/net/ethernet/intel/igb/igb_xsk.c
+++ b/drivers/net/ethernet/intel/igb/igb_xsk.c
@@ -91,7 +91,21 @@ struct xsk_buff_pool *igb_xsk_pool(struct igb_adapter *adapter,
 	if (!igb_xdp_is_enabled(adapter))
 		return NULL;
 
-	return (pool && pool->dev) ? pool : NULL;
+	if (!pool || !pool->dev)
+		return NULL;
+
+	/* igb_change_mtu() does not see a pool on a queue that is hidden, for
+	 * example after a fallback from MSI-X to MSI. The hardware buffer can
+	 * be larger than the XSK frame, so check again when the ring is set up.
+	 */
+	if (!igb_xsk_frame_fits(pool, adapter->max_frame_size)) {
+		netdev_warn_once(adapter->netdev,
+				 "XSK frame size is too small for the MTU, queue %d is not used in zero-copy mode\n",
+				 qid);
+		return NULL;
+	}
+
+	return pool;
 }
 
 static int igb_xsk_pool_enable(struct igb_adapter *adapter,
@@ -103,13 +117,22 @@ static int igb_xsk_pool_enable(struct igb_adapter *adapter,
 	bool if_running;
 	int err;
 
-	if (qid >= adapter->num_rx_queues)
+	/* the real queue counts are only updated when the interface is
+	 * opened, so also check the PF's own queues, see
+	 * igb_set_interrupt_capability()
+	 */
+	if (qid >= adapter->num_rx_queues ||
+	    qid >= adapter->num_tx_queues)
 		return -EINVAL;
 
 	if (qid >= netdev->real_num_rx_queues ||
 	    qid >= netdev->real_num_tx_queues)
 		return -EINVAL;
 
+	/* checked again on MTU changes and when the ring is set up */
+	if (!igb_xsk_frame_fits(pool, adapter->max_frame_size))
+		return -EOPNOTSUPP;
+
 	err = xsk_pool_dma_map(pool, &adapter->pdev->dev, IGB_RX_DMA_ATTR);
 	if (err)
 		return err;

-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH net v7 3/3] igb: do not split AF_XDP zero-copy frames over several Rx buffers
@ 2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Tjerk Kusters via B4 Relay @ 2026-11-20 11:10 UTC (permalink / raw)
  To: Tony Nguyen, Przemek Kitszel, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Alexei Starovoitov,
	Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend,
	Stanislav Fomichev, Sven Auhagen, Richard Cochran,
	Kurt Kanzenbach, Sriram Yagnaraman
  Cc: intel-wired-lan, netdev, linux-kernel, bpf, Maciej Fijalkowski,
	Paul Moses, stable, Tjerk Kusters

From: Tjerk Kusters <tkusters@aweta.nl>

SRRCTL.BSIZEPKT has a granularity of 1 KiB. For AF_XDP zero-copy rings,
igb_setup_srrctl() programs xsk_pool_get_rx_frame_size(), which is
rounded down. With 2 KiB UMEM chunks and the default headroom this is
1792 bytes, so the hardware buffer becomes 1 KiB and a full-size frame
spans two Rx buffers. igb_clean_rx_irq_zc() handles every descriptor as
a complete frame, so the AF_XDP socket gets such a frame as two
packets, the second one without headers. With Rx timestamping enabled,
the timestamp header is also stripped from the second buffer, and a
short final buffer makes the size wrap.

Program the same buffer size as for the other rings, 2 KiB or 3 KiB.
This depends on "igb: do not let XDP frames span more than one Rx
buffer": the receive length limit set there while XDP is attached keeps
the hardware from writing more than the frame plus the timestamp header
into the larger buffer. Without it, the hardware can write past the XSK
frame.

Refuse an XSK pool whose frame size cannot hold that, and an MTU change
that would no longer fit a bound pool, also while no XDP program is
attached. The current MTU is checked rather than the largest XDP MTU,
as igb allows XDP up to MTU 3030 and that would rule out 2 KiB chunks.

With VFs, the PF keeps a single Tx queue on all MACs, and one Rx queue
on i350 or two on 82576. A pool bound to another queue would lose its
Tx queue, which breaks the socket, and possibly its Rx queue, which the
MTU check then no longer sees. The core refuses ethtool -L in that case
(with -EINVAL). Likewise, refuse to enable VFs with -EBUSY while an XSK
pool is bound to a queue other than 0. igb_enable_sriov() checks this
before it sets up any VF state, so nothing has to be undone on that
error, and holds RTNL until the PF queues are set up again, so that no
pool can be bound in between. For that, igb_sriov_reinit() is split
into a part that runs under RTNL and a wrapper that takes it.
pci_enable_sriov() still runs without RTNL, and disabling VFs is not
refused. The older error paths of igb_enable_sriov(), for a failed queue
setup or a failed pci_enable_sriov(), are not changed.

The real queue counts are only updated when the interface is opened.
After VFs were enabled on a down 82576 PF, a pool could still be bound
to queue 1, which then has no Tx ring, and closing the socket on the
running PF would dereference the missing ring. igb_xsk_pool_enable()
now also checks the PF's Tx queue count.

A queue can also be hidden without such a request, for example when the
driver falls back from MSI-X to MSI and keeps one queue. The MTU check
does not see a pool on that queue, so igb_xsk_pool() checks the frame
size again when the ring is set up, and does not use zero-copy if the
frame does not fit. The socket then receives through the copy path,
and its Tx wakeup fails with EINVAL, until the ring is set up again
with an MTU that fits. __igb_open() sets up the rings before it updates
the queue counts, so a queue that comes back is used in zero-copy mode
again only from the next reconfiguration; that is not changed here.
This path was not tested.

This is the igb counterpart of the igc XSK buffer change.

Tested on an I350 with xdpsock from bpf-examples in rxdrop zero-copy
mode on one queue, Rx timestamping on and 1514-byte frames from the
peer. With 2 KiB chunks, the receive descriptors went from about two
per frame to one per frame; 98-byte frames and 4 KiB chunks were one
per frame with and without this patch. At MTU 3000, 4 KiB chunks
receive 3014-byte frames in one buffer. A 2 KiB chunk pool at MTU 3000
was refused by the new check on the 6.14.9 test kernel; current kernels
already refuse it in xp_assign_dev(). There, with 2 KiB chunks and the
default headroom, the new check refuses MTU 1751 to 1766, which the core
accepts.

On 7.3.1, with xdpsock bound to queue 1 of an I350, enabling two VFs is
now refused with EBUSY and the socket keeps its queue; with the socket on
queue 0, which the PF keeps, the VFs are enabled. With 31 macvlans on
the PF, which fill all 32 receive address registers, all 31 addresses
stay valid after the refused VF enable, read back from the registers.
With the driver built so that the I350 keeps two Rx queues with VFs, as
an 82576 does, binding a zero-copy socket to queue 1 after VFs were
enabled on the down PF is now refused with EINVAL; before, it was
accepted. The socket was closed while the PF was down, so the missing
ring was not dereferenced.

Link: https://lore.kernel.org/all/20261028083344.3047350-3-p@1g4.org/
Fixes: 2c6196013f84 ("igb: Add AF_XDP zero-copy Rx support")
Cc: stable@vger.kernel.org
Suggested-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Signed-off-by: Tjerk Kusters <tkusters@aweta.nl>
---
 drivers/net/ethernet/intel/igb/igb.h      |  9 ++++
 drivers/net/ethernet/intel/igb/igb_main.c | 76 +++++++++++++++++++++++++++----
 drivers/net/ethernet/intel/igb/igb_xsk.c  | 27 ++++++++++-
 3 files changed, 101 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 0fff1df81b7b..0495c02544d6 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -856,6 +856,15 @@ int igb_add_mac_steering_filter(struct igb_adapter *adapter,
 int igb_del_mac_steering_filter(struct igb_adapter *adapter,
 				const u8 *addr, u8 queue, u8 flags);
 
+/* XDP does not support frames that span more than one Rx buffer, so a
+ * frame plus the inline Rx timestamp header must fit in one XSK buffer.
+ */
+static inline bool igb_xsk_frame_fits(struct xsk_buff_pool *pool,
+				      int max_frame)
+{
+	return max_frame + IGB_TS_HDR_LEN <= xsk_pool_get_rx_frame_size(pool);
+}
+
 struct xsk_buff_pool *igb_xsk_pool(struct igb_adapter *adapter,
 				   struct igb_ring *ring);
 int igb_xsk_pool_setup(struct igb_adapter *adapter,
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index a59ee6575c56..d3165ae97b00 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3782,13 +3782,13 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 }
 
 #ifdef CONFIG_PCI_IOV
-static int igb_sriov_reinit(struct pci_dev *dev)
+static int igb_sriov_reinit_locked(struct pci_dev *dev)
 {
 	struct net_device *netdev = pci_get_drvdata(dev);
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	struct pci_dev *pdev = adapter->pdev;
 
-	rtnl_lock();
+	ASSERT_RTNL();
 
 	if (netif_running(netdev))
 		igb_close(netdev);
@@ -3800,7 +3800,6 @@ static int igb_sriov_reinit(struct pci_dev *dev)
 	igb_init_queue_configuration(adapter);
 
 	if (igb_init_interrupt_scheme(adapter, true)) {
-		rtnl_unlock();
 		dev_err(&pdev->dev, "Unable to allocate memory for queues\n");
 		return -ENOMEM;
 	}
@@ -3808,9 +3807,18 @@ static int igb_sriov_reinit(struct pci_dev *dev)
 	if (netif_running(netdev))
 		igb_open(netdev);
 
+	return 0;
+}
+
+static int igb_sriov_reinit(struct pci_dev *dev)
+{
+	int err;
+
+	rtnl_lock();
+	err = igb_sriov_reinit_locked(dev);
 	rtnl_unlock();
 
-	return 0;
+	return err;
 }
 
 static int igb_disable_sriov(struct pci_dev *pdev, bool reinit)
@@ -3856,6 +3864,7 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int old_vfs = pci_num_vf(pdev);
 	struct vf_mac_filter *mac_list;
+	unsigned int q, end;
 	int err = 0;
 	int num_vf_mac_filters, i;
 
@@ -3866,6 +3875,27 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	if (!num_vfs)
 		goto out;
 
+	/* With VFs the PF keeps a single Tx queue, see
+	 * igb_set_interrupt_capability(). Refuse VFs while an XSK pool is
+	 * bound to another queue; the core refuses ethtool -L in the same
+	 * case. Check before any VF state is set up, and hold RTNL until the
+	 * queues are set up again, so that no pool can be bound in between.
+	 */
+	if (reinit) {
+		rtnl_lock();
+		end = max(netdev->real_num_rx_queues,
+			  netdev->real_num_tx_queues);
+		for (q = 1; q < end; q++) {
+			if (xsk_get_pool_from_qid(netdev, q)) {
+				dev_err(&pdev->dev,
+					"Cannot change the number of queues, an AF_XDP socket is bound to queue %u\n",
+					q);
+				err = -EBUSY;
+				goto err_unlock;
+			}
+		}
+	}
+
 	if (old_vfs) {
 		dev_info(&pdev->dev, "%d pre-allocated VFs found - override max_vfs setting of %d\n",
 			 old_vfs, max_vfs);
@@ -3880,7 +3910,7 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	if (!adapter->vf_data) {
 		adapter->vfs_allocated_count = 0;
 		err = -ENOMEM;
-		goto out;
+		goto err_unlock;
 	}
 
 	/* Due to the limited number of RAR entries calculate potential
@@ -3923,7 +3953,8 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	adapter->flags &= ~IGB_FLAG_DMAC;
 
 	if (reinit) {
-		err = igb_sriov_reinit(pdev);
+		err = igb_sriov_reinit_locked(pdev);
+		rtnl_unlock();
 		if (err)
 			goto err_out;
 	}
@@ -3943,6 +3974,11 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit)
 	kfree(adapter->vf_data);
 	adapter->vf_data = NULL;
 	adapter->vfs_allocated_count = 0;
+	goto out;
+
+err_unlock:
+	if (reinit)
+		rtnl_unlock();
 out:
 	return err;
 }
@@ -4846,9 +4882,13 @@ void igb_setup_srrctl(struct igb_adapter *adapter, struct igb_ring *ring)
 	u32 srrctl = 0;
 	u32 buf_size;
 
-	if (ring->xsk_pool)
-		buf_size = xsk_pool_get_rx_frame_size(ring->xsk_pool);
-	else if (ring_uses_large_buffer(ring))
+	/* SRRCTL.BSIZEPKT has a 1 KiB granularity, so programming the XSK
+	 * frame size would round it down and split frames over several
+	 * buffers. XSK rings use the same buffer size as the other rings.
+	 * While XDP is attached, the receive length limit keeps frames
+	 * within the XSK frame, see igb_xsk_frame_fits().
+	 */
+	if (ring_uses_large_buffer(ring))
 		buf_size = IGB_RXBUFFER_3072;
 	else
 		buf_size = IGB_RXBUFFER_2048;
@@ -6788,6 +6828,7 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 {
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	int max_frame = new_mtu + IGB_ETH_PKT_HDR_PAD;
+	int i;
 
 	if (igb_xdp_is_enabled(adapter) &&
 	    !igb_xdp_frame_fits(adapter, max_frame)) {
@@ -6801,6 +6842,23 @@ static int igb_change_mtu(struct net_device *netdev, int new_mtu)
 	if (max_frame < (ETH_FRAME_LEN + ETH_FCS_LEN))
 		max_frame = ETH_FRAME_LEN + ETH_FCS_LEN;
 
+	/* the frame must fit the buffers of a bound XSK pool, also while no
+	 * XDP program is attached, as attaching one does not check it again
+	 */
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		struct xsk_buff_pool *pool = xsk_get_pool_from_qid(netdev, i);
+
+		if (!pool || !pool->dev)
+			continue;
+
+		if (!igb_xsk_frame_fits(pool, max_frame)) {
+			netdev_warn(adapter->netdev,
+				    "Requested MTU size is not supported with AF_XDP. Frame size is %d\n",
+				    max_frame);
+			return -EINVAL;
+		}
+	}
+
 	while (test_and_set_bit(__IGB_RESETTING, &adapter->state))
 		usleep_range(1000, 2000);
 
diff --git a/drivers/net/ethernet/intel/igb/igb_xsk.c b/drivers/net/ethernet/intel/igb/igb_xsk.c
index ce4a7b58cad2..43b33a536efe 100644
--- a/drivers/net/ethernet/intel/igb/igb_xsk.c
+++ b/drivers/net/ethernet/intel/igb/igb_xsk.c
@@ -91,7 +91,21 @@ struct xsk_buff_pool *igb_xsk_pool(struct igb_adapter *adapter,
 	if (!igb_xdp_is_enabled(adapter))
 		return NULL;
 
-	return (pool && pool->dev) ? pool : NULL;
+	if (!pool || !pool->dev)
+		return NULL;
+
+	/* igb_change_mtu() does not see a pool on a queue that is hidden, for
+	 * example after a fallback from MSI-X to MSI. The hardware buffer can
+	 * be larger than the XSK frame, so check again when the ring is set up.
+	 */
+	if (!igb_xsk_frame_fits(pool, adapter->max_frame_size)) {
+		netdev_warn_once(adapter->netdev,
+				 "XSK frame size is too small for the MTU, queue %d is not used in zero-copy mode\n",
+				 qid);
+		return NULL;
+	}
+
+	return pool;
 }
 
 static int igb_xsk_pool_enable(struct igb_adapter *adapter,
@@ -103,13 +117,22 @@ static int igb_xsk_pool_enable(struct igb_adapter *adapter,
 	bool if_running;
 	int err;
 
-	if (qid >= adapter->num_rx_queues)
+	/* the real queue counts are only updated when the interface is
+	 * opened, so also check the PF's own queues, see
+	 * igb_set_interrupt_capability()
+	 */
+	if (qid >= adapter->num_rx_queues ||
+	    qid >= adapter->num_tx_queues)
 		return -EINVAL;
 
 	if (qid >= netdev->real_num_rx_queues ||
 	    qid >= netdev->real_num_tx_queues)
 		return -EINVAL;
 
+	/* checked again on MTU changes and when the ring is set up */
+	if (!igb_xsk_frame_fits(pool, adapter->max_frame_size))
+		return -EOPNOTSUPP;
+
 	err = xsk_pool_dma_map(pool, &adapter->pdev->dev, IGB_RX_DMA_ATTR);
 	if (err)
 		return err;

-- 
2.27.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH net v7 1/3] igb: do not let XDP frames span more than one Rx buffer
  2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
  (?)
@ 2026-11-21 11:11   ` sashiko-bot
  -1 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-21 11:11 UTC (permalink / raw)
  To: Tjerk Kusters; +Cc: intel-wired-lan, bpf

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] igb: missing IGB_TS_HDR_LEN room in 2 KiB build_skb buffers
  with XDP
--

commit 91c56c1d76e1bd0d76622dd294c28474abad9f0e
Author: Tjerk Kusters <tkusters@aweta.nl>

igb: do not let XDP frames span more than one Rx buffer

This commit limits RLPML and VMOLR to the configured maximum frame size while
XDP is attached, accounts for the inline Rx timestamp header when checking if
a frame fits in one Rx buffer, and prevents rx-all or legacy-rx changes from
bypassing those limits.

Link: https://lore.kernel.org/all/20261028083344.3047350-3-p@1g4.org/

> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index ce91dda00ec0e..5f1a5e2240e64 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c

[ ... ]

> @@ -2886,6 +2946,27 @@ static int igb_setup_tc(struct net_device *dev, enum tc_setup_type type,
>  	}
>  }
>  
> +/* Check that a frame fits in one Rx buffer together with the inline Rx
> + * timestamp header, for the buffer size igb_set_rx_buffer_len() selects.
> + * The ring flags are only updated when the rings are configured, so they
> + * are not used here. For build_skb rings the frame is limited to
> + * IGB_MAX_FRAME_BUILD_SKB, which is below the 2 KiB hardware buffer size
> + * even with the header.
> + */
> +static bool igb_xdp_frame_fits(struct igb_adapter *adapter, int max_frame)
> +{
> +#if (PAGE_SIZE < 8192)
> +	if (!(adapter->flags & IGB_FLAG_RX_LEGACY)) {
> +		if (max_frame <= IGB_MAX_FRAME_BUILD_SKB &&
> +		    !IGB_2K_TOO_SMALL_WITH_PADDING)
> +			return true;

[Severity: High]
Can a frame with an inline Rx timestamp overflow into skb_shared_info on a
2 KiB build_skb buffer?

Although IGB_MAX_FRAME_BUILD_SKB + IGB_TS_HDR_LEN fits inside the 2048-byte
hardware buffer size in SRRCTL, DMA on a build_skb buffer starts at
IGB_SKB_PAD and must leave SKB_DATA_ALIGN(sizeof(struct skb_shared_info))
bytes at the end of the 2 KiB slice:

drivers/net/ethernet/intel/igb/igb.h:igb_skb_pad() {
    ...
	else
		rx_buf_len = IGB_RXBUFFER_1536;

	/* if needed make room for NET_IP_ALIGN */
	rx_buf_len -= NET_IP_ALIGN;

	return igb_compute_pad(rx_buf_len);
}

Because igb_skb_pad() sets IGB_SKB_PAD to
SKB_WITH_OVERHEAD(IGB_RXBUFFER_2048) - IGB_MAX_FRAME_BUILD_SKB without
reserving IGB_TS_HDR_LEN (16 bytes), only IGB_MAX_FRAME_BUILD_SKB bytes are
available between the DMA start offset and skb_shared_info.

When hardware Rx timestamping is enabled (E1000_RXDADV_STAT_TSIP) and RLPML
or VMOLR allows frames up to max_frame_size (or IGB_MAX_FRAME_BUILD_SKB), the
MAC strips the 4-byte FCS and prepends the 16-byte timestamp, writing
wire_len + 12 bytes from IGB_SKB_PAD.  A frame with wire_len between
IGB_MAX_FRAME_BUILD_SKB - 11 and IGB_MAX_FRAME_BUILD_SKB (for example at MTU
1500..1510) pushes xdp.data_end up to 12 bytes past
SKB_WITH_OVERHEAD(IGB_RXBUFFER_2048).  On XDP_PASS, igb_build_skb() ->
napi_build_skb() zeroes skb_shared_info over the end of the payload and
__skb_put() advances skb->tail past skb->end, triggering skb_over_panic() on
a later skb_put() or skb_copy(); on XDP_TX or XDP_REDIRECT,
xdp_update_frame_from_buff() hits XDP_WARN and drops the frame.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-igb-rx-ts-fix-v7-0-63b03045978e@aweta.nl?part=1

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-11-21 11:11 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 11:10 [PATCH net v7 0/3] igb: only strip the Rx timestamp header from the first buffer of a frame Tjerk Kusters
2026-11-20 11:10 ` Tjerk Kusters via B4 Relay
2026-11-20 11:10 ` [PATCH net v7 1/3] igb: do not let XDP frames span more than one Rx buffer Tjerk Kusters
2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
2026-11-21 11:11   ` sashiko-bot
2026-11-20 11:10 ` [PATCH net v7 2/3] igb: only strip Rx timestamp header on the first buffer of a frame Tjerk Kusters
2026-11-20 11:10   ` Tjerk Kusters via B4 Relay
2026-11-20 11:10 ` [PATCH net v7 3/3] igb: do not split AF_XDP zero-copy frames over several Rx buffers Tjerk Kusters
2026-11-20 11:10   ` Tjerk Kusters via B4 Relay

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.