[PATCH v5 net-next 0/2] ECN offload handling series

[PATCH v5 net-next 0/2] ECN offload handling series

[chia-yu.chang]

From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>

Hello,

Please find the v5 ECN offload handling series.
It clarifies the flags of SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN and fixs
the usage of SKB_GSO_TCP_ECN of hns3 in the RX path.

This corresponds to discussions in virtio mailing list:
https://lore.kernel.org/all/20250814120118.81787-1-chia-yu.chang@nokia-bell-labs.com/

Best regards,
Chia-Yu

---
Chia-Yu Chang (2):
  net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN
  net: hns3: fix GSO_ECN flag setting in the RX path

 drivers/net/ethernet/hisilicon/hns3/hns3_enet.c |  2 --
 include/linux/skbuff.h                          | 15 ++++++++++++++-
 2 files changed, 14 insertions(+), 3 deletions(-)

-- 
2.34.1

[PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN

[chia-yu.chang]

From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>

This patch updates the documentation of ECN‑related GSO flags, it
clarifies the limitations of SKB_GSO_TCP_ECN and explains how to preserve
the CWR flag (part of the ACE signal) in the Rx path.

For Tx, SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN are used respectively for
RFC3168 ECN and AccECN (RFC9768). SKB_GSO_TCP_ECN indicates that the
first segment has CWR set, while subsequent segments have CWR cleared.
In contrast, SKB_GSO_TCP_ACCECN means that the segment uses AccECN and
therefore its CWR flag must not be modified during segmentation.

For RX, SKB_GSO_TCP_ECN shall NOT be used, because the stack cannot know
whether the connection uses RFC3168 ECN or AccECN, whereas RFC3168 ECN
offload may clear CWR flag and thus corrupts the ACE signal. Instead, any
segment that arrives with CWR set must use the SKB_GSO_TCP_ACCECN flag
to prevent RFC3168 ECN offload logic from clearing the CWR flag.

This corresponds to discussions in virtio mailing list:
https://lore.kernel.org/all/20250814120118.81787-1-chia-yu.chang@nokia-bell-labs.com/
And it was suggested to clarify SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN.

Co-developed-by: Ilpo Järvinen <ij@kernel.org>
Signed-off-by: Ilpo Järvinen <ij@kernel.org>
Signed-off-by: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>
Cc: Parav Pandit <parav@nvidia.com>

---
v4:
- Update commit message

v3:
- Update commit messages and documentation for clarity
---
 include/linux/skbuff.h | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 22eda1d54a0e..6c0d725facf0 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -671,7 +671,13 @@ enum {
 	/* This indicates the skb is from an untrusted source. */
 	SKB_GSO_DODGY = 1 << 1,
 
-	/* This indicates the tcp segment has CWR set. */
+	/* For TX, this indicates that the first TCP segment has CWR set, and
+	 * any subsequent segment in the same skb has CWR cleared. This flag
+	 * must not be used in RX, because the connection to which the segment
+	 * belongs is not tracked to use RFC3168 or AccECN. Using RFC3168 ECN
+	 * offload may clear CWR and corrupt ACE signal (CWR is part of it).
+	 * Instead, SKB_GSO_TCP_ACCECN shall be used to avoid CWR corruption.
+	 */
 	SKB_GSO_TCP_ECN = 1 << 2,
 
 	__SKB_GSO_TCP_FIXEDID = 1 << 3,
@@ -706,6 +712,13 @@ enum {
 
 	SKB_GSO_FRAGLIST = 1 << 18,
 
+	/* For TX, this indicates that the TCP segment uses the CWR flag as part
+	 * of the ACE signal, and the CWR flag must not be modified in the skb.
+	 * For RX, any incoming segment with CWR set must use this flag so that
+	 * no RFC3168 ECN offload can clear the CWR flag. This is required to
+	 * preserve ACE signal correctness (CWR is part of it) in a forwarding
+	 * scenario, e.g., from one netdevice RX to other netdevice TX
+	 */
 	SKB_GSO_TCP_ACCECN = 1 << 19,
 
 	/* These indirectly map onto the same netdev feature.
-- 
2.34.1

[PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path

[chia-yu.chang]

From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>

Currently, the hns3 Rx path sets the SKB_GSO_TCP_ECN flag when a TCP
segment with the CWR flag set is received. This is incorrect because
the hns3 hardware GRO resets the IP ToS byte to 0 during packet
aggregation. Consequently, no valid ECN signals are carreid, meaning
that it cannot support RFC3168 ECN or Accurate ECN protocols. Setting
SKB_GSO_TCP_ECN or SKB_GSO_TCP_ACCECN under these conditions misleads
the upper network stack.

Fix this by removing the incorrect SKB_GSO_TCP_ECN flag assignment
in hns3_gro_complete().

Fixes: d474d88f88261 ("net: hns3: add hns3_gro_complete for HW GRO process")

Signed-off-by: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>

---
v5:
- Remove SKB_GSO_TCP_ECN flag and update the commit message

v3:
- Rewrite the commit message for clarity
---
 drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
index 6ecb32e28e79..5149eca0104a 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
@@ -3916,8 +3916,6 @@ static int hns3_gro_complete(struct sk_buff *skb, u32 l234info)
 	}
 
 	skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count;
-	if (th->cwr)
-		skb_shinfo(skb)->gso_type |= SKB_GSO_TCP_ECN;
 
 	if (l234info & BIT(HNS3_RXD_GRO_FIXID_B))
 		skb_shinfo(skb)->gso_type |= SKB_GSO_TCP_FIXEDID;
-- 
2.34.1

Re: [PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path

[Jijie Shao]

on 2026/8/5 5:35, chia-yu.chang@nokia-bell-labs.com wrote:
> From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>
>
> Currently, the hns3 Rx path sets the SKB_GSO_TCP_ECN flag when a TCP
> segment with the CWR flag set is received. This is incorrect because
> the hns3 hardware GRO resets the IP ToS byte to 0 during packet
> aggregation. Consequently, no valid ECN signals are carreid, meaning
> that it cannot support RFC3168 ECN or Accurate ECN protocols. Setting
> SKB_GSO_TCP_ECN or SKB_GSO_TCP_ACCECN under these conditions misleads
> the upper network stack.
>
> Fix this by removing the incorrect SKB_GSO_TCP_ECN flag assignment
> in hns3_gro_complete().
>
> Fixes: d474d88f88261 ("net: hns3: add hns3_gro_complete for HW GRO process")

One note:
since this patch fixes a bug (incorrect SKB_GSO_TCP_ECN flag set in hns3_gro_complete),
it should target net instead of net-next.

I'm running HW tests on hns3 to verify the fix. Will share results
based on what I observe.

Thanks,
Jijie Shao

Re: [PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path

[Jijie Shao]

on 2026/8/5 5:35, chia-yu.chang@nokia-bell-labs.com wrote:
> From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>
>
> Currently, the hns3 Rx path sets the SKB_GSO_TCP_ECN flag when a TCP
> segment with the CWR flag set is received. This is incorrect because
> the hns3 hardware GRO resets the IP ToS byte to 0 during packet
> aggregation. Consequently, no valid ECN signals are carreid, meaning
> that it cannot support RFC3168 ECN or Accurate ECN protocols. Setting
> SKB_GSO_TCP_ECN or SKB_GSO_TCP_ACCECN under these conditions misleads
> the upper network stack.
>
> Fix this by removing the incorrect SKB_GSO_TCP_ECN flag assignment
> in hns3_gro_complete().


Tested on hns3 HW (2x 100G, direct cable, openEuler 24.03, kernel 7.2.0-rc6).

tcpdump confirms HW GRO zeroes IP ToS: aggregated (>MTU) packets carry
tos 0x0, while non-aggregated packets keep their ToS/ECN marks — so the
zeroing is done by HW GRO, not the sender. Before patch, CWR packets had
SKB_GSO_TCP_ECN set (0x5 = SKB_GSO_TCPV4 | SKB_GSO_TCP_ECN). After patch,
all GRO events gso_type=0x1 (SKB_GSO_TCPV4 only), no 0x5 observed.

Tested-by: Jijie Shao <shaojijie@huawei.com>

Re: [PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN

[Jakub Kicinski]

On Tue,  4 Aug 2026 23:35:09 +0200 chia-yu.chang@nokia-bell-labs.com
wrote:
> This corresponds to discussions in virtio mailing list:
> https://lore.kernel.org/all/20250814120118.81787-1-chia-yu.chang@nokia-bell-labs.com/
> And it was suggested to clarify SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN.

> -	/* This indicates the tcp segment has CWR set. */
> +	/* For TX, this indicates that the first TCP segment has CWR set, and
> +	 * any subsequent segment in the same skb has CWR cleared. This flag
> +	 * must not be used in RX, because the connection to which the segment
> +	 * belongs is not tracked to use RFC3168 or AccECN. Using RFC3168 ECN
> +	 * offload may clear CWR and corrupt ACE signal (CWR is part of it).
> +	 * Instead, SKB_GSO_TCP_ACCECN shall be used to avoid CWR corruption.
> +	 */

I still can't wrap my head around this TBH.

SKB_GSO_TCP_ECN means RFC3168
SKB_GSO_TCP_ACCECN means AccECN

If the HW can correctly detect cwr on first frame and then no cwr 
and report that as ECN/RFC3168 - what's the problem? TSO will produce
the exact expected segment sequence.

Is the program that if we re-GRO that frame in SW we end up with
ECN+ACCECN on the same skb?

RE: [PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN

[Chia-Yu Chang (Nokia)]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org> 
> Sent: Wednesday, August 12, 2026 2:30 AM
> To: Chia-Yu Chang (Nokia) <chia-yu.chang@nokia-bell-labs.com>
> Cc: shaojijie@huawei.com; shenjian15@huawei.com; linux-rdma@vger.kernel.org; eperezma@redhat.com; jasowang@redhat.com; virtualization@lists.linux.dev; mst@redhat.com; xuanzhuo@linux.alibaba.com; pabeni@redhat.com; edumazet@google.com; linux-doc@vger.kernel.org; corbet@lwn.net; horms@kernel.org; dsahern@kernel.org; kuniyu@google.com; bpf@vger.kernel.org; netdev@vger.kernel.org; dave.taht@gmail.com; jhs@mojatatu.com; stephen@networkplumber.org; xiyou.wangcong@gmail.com; jiri@resnulli.us; davem@davemloft.net; andrew+netdev@lunn.ch; donald.hunter@gmail.com; ast@fiberby.net; liuhangbin@gmail.com; shuah@kernel.org; linux-kselftest@vger.kernel.org; ij@kernel.org; ncardwell@google.com; Koen De Schepper (Nokia) <koen.de_schepper@nokia-bell-labs.com>; g.white@cablelabs.com; ingemar.s.johansson@ericsson.com; mirja.kuehlewind@ericsson.com; cheshire@apple.com; rs.ietf@gmx.at; Jason_Livingood@comcast.com; vidhi_goel@apple.com; Parav Pandit <parav@nvidia.com>
> Subject: Re: [PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN
> 
> 
> CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> On Tue,  4 Aug 2026 23:35:09 +0200 chia-yu.chang@nokia-bell-labs.com
> wrote:
> > This corresponds to discussions in virtio mailing list:
> > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > .kernel.org%2Fall%2F20250814120118.81787-1-chia-yu.chang%40nokia-bell-
> > labs.com%2F&data=05%7C02%7Cchia-yu.chang%40nokia-bell-labs.com%7C150b5
> > 1c3733640dd2cde08def808e48f%7C5d4717519675428d917b70f44f9630b0%7C0%7C0
> > %7C639220914236399114%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUs
> > IlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7
> > C0%7C%7C%7C&sdata=jGSBo1wXJOnjulVOJKW3PAEbrUAq7X30Zax4SBJ6VRQ%3D&reser
> > ved=0 And it was suggested to clarify SKB_GSO_TCP_ECN and 
> > SKB_GSO_TCP_ACCECN.
> 
> > -     /* This indicates the tcp segment has CWR set. */
> > +     /* For TX, this indicates that the first TCP segment has CWR set, and
> > +      * any subsequent segment in the same skb has CWR cleared. This flag
> > +      * must not be used in RX, because the connection to which the segment
> > +      * belongs is not tracked to use RFC3168 or AccECN. Using RFC3168 ECN
> > +      * offload may clear CWR and corrupt ACE signal (CWR is part of it).
> > +      * Instead, SKB_GSO_TCP_ACCECN shall be used to avoid CWR corruption.
> > +      */
> 
> I still can't wrap my head around this TBH.
> 
> SKB_GSO_TCP_ECN means RFC3168
> SKB_GSO_TCP_ACCECN means AccECN
> 
> If the HW can correctly detect cwr on first frame and then no cwr and report that as ECN/RFC3168 - what's the problem? TSO will produce the exact expected segment sequence.
> 
> Is the program that if we re-GRO that frame in SW we end up with
> ECN+ACCECN on the same skb?

Yes, this is the problem.
The HW does not know whether the received packets belong to an RFC3168 ECN flow or an AccECN flow on the RX path.
For example, HW GRO may set SKB_GSO_TCP_ECN after observing that the first packet has CWR=1:

+===================+==========+=================+================+
|     Packet id     | CWR flag |       Flag      | Flushed as SKB |
+===================+==========+=================+================+
|         0         |     1    | SKB_GSO_TCP_ECN |        0       |
|         1         |     0    |         -       |        0       |
|         2         |     1    |         -       |        0       |
|         3         |     1    |         -       |        1       |
+===================+==========+=================+================+

If the aggregated skb is forwarded through a device using GSO, e.g., HW RX (GRO) -> veth TX (GSO), the SKB_GSO_TCP_ECN applies RFC3168 semantics.
This means that only the 1st segment keeps the CWR flag while all subsequent segments have CWR cleared:

+===================+==========+
|     Packet id     | CWR flag |
+===================+==========+
|         0         |     1    |
|         1         |     0    |
|         2         |     0    |
|         3         |     0    |
+===================+==========+

This behavior is ok for RFC3168, since CWR is expected to appear only once.
However, for AccECN, CWR is part of the ACE signal and must be preserved across all segments.

In the example above, the original CWR sequence was 1,0,1,1.
But after re-segmentation it becomes: 1,0,0,0.
This is why SKB_GSO_TCP_ECN should not be used in RX/GRO paths.

--
Chia-Yu

RE: [PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path

[Chia-Yu Chang (Nokia)]

> -----Original Message-----
> From: Jijie Shao <shaojijie@huawei.com> 
> Sent: Tuesday, August 11, 2026 3:32 PM
> To: Chia-Yu Chang (Nokia) <chia-yu.chang@nokia-bell-labs.com>; shenjian15@huawei.com; linux-rdma@vger.kernel.org; eperezma@redhat.com; jasowang@redhat.com; virtualization@lists.linux.dev; mst@redhat.com; xuanzhuo@linux.alibaba.com; pabeni@redhat.com; edumazet@google.com; linux-doc@vger.kernel.org; corbet@lwn.net; horms@kernel.org; dsahern@kernel.org; kuniyu@google.com; bpf@vger.kernel.org; netdev@vger.kernel.org; dave.taht@gmail.com; jhs@mojatatu.com; kuba@kernel.org; stephen@networkplumber.org; xiyou.wangcong@gmail.com; jiri@resnulli.us; davem@davemloft.net; andrew+netdev@lunn.ch; donald.hunter@gmail.com; ast@fiberby.net; liuhangbin@gmail.com; shuah@kernel.org; linux-kselftest@vger.kernel.org; ij@kernel.org; ncardwell@google.com; Koen De Schepper (Nokia) <koen.de_schepper@nokia-bell-labs.com>; g.white@cablelabs.com; ingemar.s.johansson@ericsson.com; mirja.kuehlewind@ericsson.com; cheshire@apple.com; rs.ietf@gmx.at; Jason_Livingood@comcast.com; vidhi_goel@apple.com
> Cc: shaojijie@huawei.com
> Subject: Re: [PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path
> 
> 
> on 2026/8/5 5:35, chia-yu.chang@nokia-bell-labs.com wrote:
> > From: Chia-Yu Chang <chia-yu.chang@nokia-bell-labs.com>
> >
> > Currently, the hns3 Rx path sets the SKB_GSO_TCP_ECN flag when a TCP 
> > segment with the CWR flag set is received. This is incorrect because 
> > the hns3 hardware GRO resets the IP ToS byte to 0 during packet 
> > aggregation. Consequently, no valid ECN signals are carreid, meaning 
> > that it cannot support RFC3168 ECN or Accurate ECN protocols. Setting 
> > SKB_GSO_TCP_ECN or SKB_GSO_TCP_ACCECN under these conditions misleads 
> > the upper network stack.
> >
> > Fix this by removing the incorrect SKB_GSO_TCP_ECN flag assignment in 
> > hns3_gro_complete().
> 
> 
> Tested on hns3 HW (2x 100G, direct cable, openEuler 24.03, kernel 7.2.0-rc6).
> 
> tcpdump confirms HW GRO zeroes IP ToS: aggregated (>MTU) packets carry tos 0x0, while non-aggregated packets keep their ToS/ECN marks — so the zeroing is done by HW GRO, not the sender. Before patch, CWR packets had SKB_GSO_TCP_ECN set (0x5 = SKB_GSO_TCPV4 | SKB_GSO_TCP_ECN). After patch, all GRO events gso_type=0x1 (SKB_GSO_TCPV4 only), no 0x5 observed.
> 
> Tested-by: Jijie Shao <shaojijie@huawei.com>

Thanks Jijie for confirmation, I will submit to net in the next round.