From: Tianyi Gao <tianyi@cloudflare.com>
To: netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org,
Tony Nguyen <anthony.l.nguyen@intel.com>,
Przemek Kitszel <przemyslaw.kitszel@intel.com>
Cc: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Simon Horman <horms@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Jesper Dangaard Brouer <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Paul Greenwalt <paul.greenwalt@intel.com>,
Alice Michael <alice.michael@intel.com>,
Eric Joyner <eric.joyner@intel.com>,
Jesse Brandeburg <jbrandeb@kernel.org>,
Lorenzo Bianconi <lorenzo@kernel.org>,
bpf@vger.kernel.org, kernel-team@cloudflare.com,
tianyi.gao@linux.dev, Tianyi Gao <tianyi@cloudflare.com>
Subject: [PATCH net] ice: do not report CHECKSUM_COMPLETE when an XDP program is attached
Date: Sat, 21 Nov 2026 01:24:02 +0000 [thread overview]
Message-ID: <20261121012402.84848-1-tianyi@cloudflare.com> (raw)
On E830, ice reports CHECKSUM_COMPLETE with the generic checksum (GCS)
that the hardware computes over the received frame. If an XDP program
changes the packet and returns XDP_PASS, that checksum no longer
matches. The stack then checks the packet again in software and calls
netdev_rx_csum_fault().
On Cloudflare servers, an XDP load balancer removes the encapsulation
of the packets that it accepts locally. Our production traffic hits
netdev_rx_csum_fault() all the time.
Do not use GCS when the ring has an XDP program, as mlx5 has done since
commit 5d0bb3bac4b9 ("net/mlx5e: XDP, Avoid checksum complete when XDP
prog is loaded").
E830 also sets the L3L4P and checksum error bits, so the packet gets
CHECKSUM_UNNECESSARY or CHECKSUM_NONE, as on E810. For example, a zero
UDP checksum over IPv6 is marked as bad and gets CHECKSUM_NONE. A zero
UDP checksum over IPv4 gets CHECKSUM_UNNECESSARY.
Two pending series [1][2] give XDP access to the checksum status.
Neither of them supports CHECKSUM_COMPLETE.
ice_rx_csum() reads rx_ring->xdp_prog again after the poll loop ran XDP.
While ICE_VSI_REBUILD_PENDING is set, ice_xdp_setup_prog() can replace
the program without stopping NAPI, so the two reads can differ. At
worst, a packet in that poll gets CHECKSUM_COMPLETE as before this
patch. The pointer is only compared with NULL, so this cannot crash.
Link: https://lore.kernel.org/all/20260715-bpf-xdp-meta-rxcksum-v5-0-623d5c0d0ab7@kernel.org/ [1]
Link: https://lore.kernel.org/all/cover.1794281983.git.tianyi@cloudflare.com/ [2]
Fixes: 905d1a220e8d ("ice: Add E830 checksum offload support")
Suggested-by: Jesse Brandeburg <jbrandeb@kernel.org>
Signed-off-by: Tianyi Gao <tianyi@cloudflare.com>
---
Tested on an E830 (8086:12d2, board N31483-007, fw.mgmt 7.9.5, DDP ICE
COMMS 1.3.63.0) with an XDP program that removes a UDP encapsulation
header with bpf_xdp_adjust_head() and returns XDP_PASS. The kernel was
based on 6.18.64, where ice_rx_csum() is the same as in net:
- Before: 63322069 netdev_rx_csum_fault() calls in 30 s.
- After: 0.
- Without an XDP program, the port still reports CHECKSUM_COMPLETE.
On an E810, which has no GCS, drivers/net/xdp.py, ping.py and
hw/csum.py give the same results with and without this patch.
Request for comment: I also prepared a drv-net selftest for net-next.
It removes 16 bytes from the UDP payload with bpf_xdp_adjust_head()
and fails if the packet reaches the stack with CHECKSUM_COMPLETE. I am
not sure we want to enforce this for all NICs, so asking here.
drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
index e695a664e53d..ad22d855436e 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
@@ -123,6 +123,7 @@ ice_rx_csum(struct ice_rx_ring *ring, struct sk_buff *skb,
rx_status1 = le16_to_cpu(rx_desc->wb.status_error1);
if ((ring->flags & ICE_RX_FLAGS_RING_GCS) &&
+ !READ_ONCE(ring->xdp_prog) &&
rx_desc->wb.rxdid == ICE_RXDID_FLEX_NIC &&
(decoded.inner_prot == LIBETH_RX_PT_INNER_TCP ||
decoded.inner_prot == LIBETH_RX_PT_INNER_UDP ||
base-commit: 92e97d7cffc7ec6adfa4699203c18b1ab0c8b314
--
2.56.0
reply other threads:[~2026-11-21 1:24 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121012402.84848-1-tianyi@cloudflare.com \
--to=tianyi@cloudflare.com \
--cc=alice.michael@intel.com \
--cc=andrew+netdev@lunn.ch \
--cc=anthony.l.nguyen@intel.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=eric.joyner@intel.com \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jbrandeb@kernel.org \
--cc=john.fastabend@gmail.com \
--cc=kernel-team@cloudflare.com \
--cc=kuba@kernel.org \
--cc=lorenzo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=paul.greenwalt@intel.com \
--cc=przemyslaw.kitszel@intel.com \
--cc=sdf@fomichev.me \
--cc=tianyi.gao@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox