Linux virtualization list
 help / color / mirror / Atom feed
From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
To: "Chia-Yu Chang (Nokia)" <chia-yu.chang@nokia-bell-labs.com>,
	 Willem de Bruijn <willemdebruijn.kernel@gmail.com>,
	 Jakub Kicinski <kuba@kernel.org>
Cc: Mirja Kuehlewind <mirja.kuehlewind@ericsson.com>,
	 "shaojijie@huawei.com" <shaojijie@huawei.com>,
	 "shenjian15@huawei.com" <shenjian15@huawei.com>,
	 "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	 "eperezma@redhat.com" <eperezma@redhat.com>,
	 "jasowang@redhat.com" <jasowang@redhat.com>,
	 "virtualization@lists.linux.dev"
	<virtualization@lists.linux.dev>,
	 "mst@redhat.com" <mst@redhat.com>,
	 "xuanzhuo@linux.alibaba.com" <xuanzhuo@linux.alibaba.com>,
	 "pabeni@redhat.com" <pabeni@redhat.com>,
	 "edumazet@google.com" <edumazet@google.com>,
	 "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	 "corbet@lwn.net" <corbet@lwn.net>,
	 "horms@kernel.org" <horms@kernel.org>,
	 "dsahern@kernel.org" <dsahern@kernel.org>,
	 "kuniyu@google.com" <kuniyu@google.com>,
	 "bpf@vger.kernel.org" <bpf@vger.kernel.org>,
	 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	 "dave.taht@gmail.com" <dave.taht@gmail.com>,
	 "jhs@mojatatu.com" <jhs@mojatatu.com>,
	 "stephen@networkplumber.org" <stephen@networkplumber.org>,
	 "xiyou.wangcong@gmail.com" <xiyou.wangcong@gmail.com>,
	 "jiri@resnulli.us" <jiri@resnulli.us>,
	 "davem@davemloft.net" <davem@davemloft.net>,
	 "andrew+netdev@lunn.ch" <andrew+netdev@lunn.ch>,
	 "donald.hunter@gmail.com" <donald.hunter@gmail.com>,
	 "ast@fiberby.net" <ast@fiberby.net>,
	 "liuhangbin@gmail.com" <liuhangbin@gmail.com>,
	 "shuah@kernel.org" <shuah@kernel.org>,
	 "linux-kselftest@vger.kernel.org"
	<linux-kselftest@vger.kernel.org>,
	 "ij@kernel.org" <ij@kernel.org>,
	 "ncardwell@google.com" <ncardwell@google.com>,
	 "Koen De Schepper (Nokia)"
	<koen.de_schepper@nokia-bell-labs.com>,
	 "g.white@cablelabs.com" <g.white@cablelabs.com>,
	 Ingemar Johansson S <ingemar.s.johansson@ericsson.com>,
	 "cheshire@apple.com" <cheshire@apple.com>,
	 "rs.ietf@gmx.at" <rs.ietf@gmx.at>,
	 "Jason_Livingood@comcast.com" <Jason_Livingood@comcast.com>,
	 "vidhi_goel@apple.com" <vidhi_goel@apple.com>,
	 Parav Pandit <parav@nvidia.com>,
	 Willem de Bruijn <willemb@google.com>,
	 ij@kernel.org
Subject: RE: [PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN
Date: Tue, 25 Aug 2026 12:12:24 -0400	[thread overview]
Message-ID: <willemdebruijn.kernel.28ce4bc2544df@gmail.com> (raw)
In-Reply-To: <PAXPR07MB7984F0C072220066856DD8ACA3AF2@PAXPR07MB7984.eurprd07.prod.outlook.com>

> > > On Fri, 14 Aug 2026 09:34:01 -0400 Willem de Bruijn wrote:
> > > > > Current SW GRO sets SKB_GSO_TCP_ACCECN when the flushed skb carries CWR in tcp_gro_complete():
> > > > > if (th->cwr)
> > > > >     shinfo->gso_type |= SKB_GSO_TCP_ACCECN;
> > > >
> > > > And I suppose it follows correct AccECN rules for coalescing.
> > > >
> > > > That is a performance regression from RFC 3168 ECN, as it allows for 
> > > > less effective coalescing. I have no intuition how much it will 
> > > > differ in practice.
> > > >
> > > > > For HW GRO of a legacy device that implementing RFC3168 semantics, setting SKB_GSO_TCP_ECN seems reasonable.
> > > > > However, such a device would not be able to preserve ACCECN signaling across the GRO/GSO.
> > > > > In that case, if preserving AccECN signaling is required, disabling HW GRO may indeed be necessary.
> > > >
> > > > Right.
> > >
> > > I'm still not following.. Maybe Willem can ELI5 what the problem is.
> > >
> > > _SW_ GRO follows only the AccECN rules.
> > > But if HW GRO follows RFC 3168 and we mark the aggregate as 
> > > SKB_GSO_TCP_ECN - TSO will also abide, and segmented output will be 
> > > identical to pre-GRO input.
> > 
> > +1
> > 
> > > Are we trying to ban RFC 3168 behavior in HW purely to match SW?
> > 
> > I think that's the intent here?
> Hi Willem,
> 
> I think we can still change SKB_GSO_TCP_ECN into SKB_GSO_TCP_ACCECN on the RX path, even if HW GRO and SW GRO use different aggregation rules.
> Currently, SW GRO flushes when the CWR state changes and sets SKB_GSO_TCP_ACCECN when the resulting skb carries CWR=1.
>
> 
> For HW GRO, if the device flushes immediately when a CWR=1 packet arrives, there is no need to set either SKB_GSO_TCP_ECN or SKB_GSO_TCP_ACCECN.
> In that case, each CWR=1 packet is emitted separately, and there is no need to preserve multiple CWR indications through GSO.
> So, if the HW can be confirmed to always flush CWR-marked packets without coalescing them, preserving multiple CWR indications through GSO is not required.
> In that case, using SKB_GSO_TCP_ACCECN would not provide any additional benefit.

I agree.

The difference on transmit is that SKB_GSO_TCP_ACCECN will copy the
ECN bits to every segment, whereas SKB_GSO_TCP_ECN will only set the
bits on the first segment.

From commit 023af5a72ab1 ("gso: AccECN support") that introduced
SKB_GSO_TCP_ACCECN:

    With RFC 3168 ECN aware TSO (NETIF_F_TSO_ECN) CWR flag is cleared
    starting from 2nd segment which is incompatible how AccECN handles
    the CWR flag. Such super-segments are indicated by SKB_GSO_TCP_ECN.
    With AccECN, CWR flag (or more accurately, the ACE field that also
    includes ECE & AE flags) changes only when new packet(s) with CE
    mark arrives so the flag should not be changed within a super-skb.

Makes me wonder what SKB_GS_TCP_ACCECN adds. Copying bits
from the GSO skb to all segments is the default. SKB_GSO_TCP_ECN is
an indication that the NIC knows how to diverge from this default for
these specific bits.

That commit confirms this:

    If NIC is completely unaware of RFC3168 ECN (doesn't support
    NETIF_F_TSO_ECN) or its TSO engine can be set to not touch CWR flag
    despite supporting also NETIF_F_TSO_ECN, TSO could be safely used
    with AccECN on such NIC. This should be evaluated per NIC basis
    (not done in this patch series for any NICs).`

> 
> The problematic case is when a HW GRO aggregates multiple packets carrying CWR=1 and only sets SKB_GSO_TCP_ECN.

The driver of such a device could be updated to set SKB_GSO_TCP_ACCECN
or not set any such ECN GSO flag.

The driver of other devices that do follow RFC 3168 semantics will
continue to have to set SKB_GSO_TCP_ECN on the GSO skb. It is quite
plausible that few or no devices currently support ECN in their HW-GRO
coalescing.

> During GSO, only the first output segment would carry CWR=1, which loses the remaining CWR signaling information required by AccECN.
> In that case, changing from SKB_GSO_TCP_ECN to SKB_GSO_TCP_ACCECN would preserve the original signaling by ensuring that the segmented packets carry the correct CWR information.
> As I understand RFC3168, receiving additional CWR-marked packets is harmless, because once a valid CWR has been received, the receiver already stops echoing ECE.

Agreed.

> Additional CWR indications do not change the receiver state.
> This would have a performance impact since it disables HW TSO for the skb and falls back to software segmentation.

You mean if the device advertises NETIF_F_TSO_ECN, and GSO skbs may
contain AccECN signals, the host must downgrade from TSO to GSO to
avoid corrupting the signal?

The cost of that would be significant. Not something to do for
established environments. But technically seemingly correct, at least
for packets with non-zero ECN bits.

> But IMO it preserves correctness for both RFC3168 ECN and AccECN.

Side-note: this thread probably has way too many Cc: for this narrow topic.

  reply	other threads:[~2026-08-25 16:12 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04 21:35 [PATCH v5 net-next 0/2] ECN offload handling series chia-yu.chang
2026-08-04 21:35 ` [PATCH v5 net-next 1/2] net: update comments for SKB_GSO_TCP_ECN and SKB_GSO_TCP_ACCECN chia-yu.chang
2026-08-12  0:30   ` Jakub Kicinski
2026-08-12 10:33     ` Chia-Yu Chang (Nokia)
2026-08-12 22:54       ` Jakub Kicinski
2026-08-13 10:28         ` Chia-Yu Chang (Nokia)
2026-08-13 15:58           ` Willem de Bruijn
     [not found]             ` <47BE8A8A-F80C-4C30-BC95-C85FEBFFB606@ericsson.com>
2026-08-14 10:29               ` Chia-Yu Chang (Nokia)
2026-08-14 13:34                 ` Willem de Bruijn
2026-08-14 19:01                   ` Jakub Kicinski
2026-08-14 19:14                     ` Willem de Bruijn
2026-08-25 12:08                       ` Chia-Yu Chang (Nokia)
2026-08-25 16:12                         ` Willem de Bruijn [this message]
2026-08-04 21:35 ` [PATCH v5 net-next 2/2] net: hns3: fix GSO_ECN flag setting in the RX path chia-yu.chang
2026-08-06 11:47   ` Jijie Shao
2026-08-11 13:31   ` Jijie Shao
2026-08-12 10:35     ` Chia-Yu Chang (Nokia)
2026-08-12 22:59       ` Jakub Kicinski
2026-08-19  2:51         ` Jijie Shao
2026-08-24 17:24           ` Chia-Yu Chang (Nokia)
2026-08-18 13:32     ` Jijie Shao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=willemdebruijn.kernel.28ce4bc2544df@gmail.com \
    --to=willemdebruijn.kernel@gmail.com \
    --cc=Jason_Livingood@comcast.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=ast@fiberby.net \
    --cc=bpf@vger.kernel.org \
    --cc=cheshire@apple.com \
    --cc=chia-yu.chang@nokia-bell-labs.com \
    --cc=corbet@lwn.net \
    --cc=dave.taht@gmail.com \
    --cc=davem@davemloft.net \
    --cc=donald.hunter@gmail.com \
    --cc=dsahern@kernel.org \
    --cc=edumazet@google.com \
    --cc=eperezma@redhat.com \
    --cc=g.white@cablelabs.com \
    --cc=horms@kernel.org \
    --cc=ij@kernel.org \
    --cc=ingemar.s.johansson@ericsson.com \
    --cc=jasowang@redhat.com \
    --cc=jhs@mojatatu.com \
    --cc=jiri@resnulli.us \
    --cc=koen.de_schepper@nokia-bell-labs.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=liuhangbin@gmail.com \
    --cc=mirja.kuehlewind@ericsson.com \
    --cc=mst@redhat.com \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=parav@nvidia.com \
    --cc=rs.ietf@gmx.at \
    --cc=shaojijie@huawei.com \
    --cc=shenjian15@huawei.com \
    --cc=shuah@kernel.org \
    --cc=stephen@networkplumber.org \
    --cc=vidhi_goel@apple.com \
    --cc=virtualization@lists.linux.dev \
    --cc=willemb@google.com \
    --cc=xiyou.wangcong@gmail.com \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox