Netdev List
 help / color / mirror / Atom feed
From: Kuniyuki Iwashima <kuniyu@google.com>
To: Eric Dumazet <edumazet@kernel.org>
Cc: Neal Cardwell <ncardwell@google.com>,
	"David S . Miller" <davem@davemloft.net>,
	 Jakub Kicinski <kuba@kernel.org>,
	Paolo Abeni <pabeni@redhat.com>,
	 Matthieu Baerts <matttbe@kernel.org>,
	Mat Martineau <martineau@kernel.org>,
	 Simon Horman <horms@kernel.org>,
	Kuniyuki Iwashima <kuni1840@gmail.com>,
	netdev@vger.kernel.org,  mptcp@lists.linux.dev
Subject: Re: [PATCH v3 net-next 5/7] tcp: Allocate all new skb before collapsing.
Date: Thu, 19 Nov 2026 20:02:59 +0000	[thread overview]
Message-ID: <av9U9w4wGpO7HmDY@google.com> (raw)
In-Reply-To: <CAL4WiioNduj1AybypLPDs6HuW2F9Y9zw3B2ycLSogRMnfzBk9g@mail.gmail.com>

On Thu, 19 Nov 2026 08:44:54 +0100, Eric Dumazet wrote:
> Le jeu. 19 nov. 2026 à 08:17, Kuniyuki Iwashima <kuniyu@google.com> a écrit :
> >
> > At the final stage, tcp_collapse() iterates over collapsible
> > candidate skbs and repacks them to new skbs.
> >
> > In each iteration, a new skb is allocated and inserted to the
> > receive queue or a temporary list for RB tree.
> >
> > If alloc_skb() fails while splitting a large bloated skb across
> > multiple new skbs, the partially-copied skb is not removed and
> > new skbs remain in the queue as duplicates, consuming more memory.
> >
> > It is only when all candidates are properly collapsed that
> > we can save memory.
> >
> > To avoid complicated rollback, let's preallocate skbs before
> > collapsing.
> >
> > Now all skb in [start, sub_end) must be allocated in a batch,
> > so the section size is capped at TCP_COLLAPSE_MAX_TOTAL_SIZE
> > (16 skbs), except for the first single skb.
> >
> > Once the size exceeds the limit, the remaining skbs are
> > re-evaluated from restart:.
> >
> > Legitimate overlaps (retransmits with a different segmentation)
> > only involve a few skbs, so if an overlapping OOO chain reaches
> > the limit, we simply skip it.
> >
> > Note that the next skb can be skipped if its density already meets
> > tp->scaling_ratio, and each cut adds at most one remainder skb
> > (~3.4% worst-case truesize overhead on 4 KiB pages), which is a
> > small tradeoff to bound the temporary allocation.
> >
> > Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
> > ---
> > v3: Make TCP_COLLAPSE_MAX_TOTAL_SIZE hard-limit for multiple skbs
> > ---
> >  net/ipv4/tcp_input.c | 41 ++++++++++++++++++++++++++++++++---------
> >  1 file changed, 32 insertions(+), 9 deletions(-)
> >
> > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> > index 27ed9ba11ec9..990e34896373 100644
> > --- a/net/ipv4/tcp_input.c
> > +++ b/net/ipv4/tcp_input.c
> > @@ -5822,6 +5822,11 @@ static struct sk_buff *tcp_skb_next(struct sk_buff *skb, struct sk_buff_head *li
> >  #define TCP_COLLAPSE_SCALING_RATIO                             \
> >         ((TCP_COLLAPSE_MAX_SIZE << TCP_RMEM_TO_WIN_SCALE) /     \
> >          (SKB_TRUESIZE(TCP_COLLAPSE_MAX_SIZE)))
> > +/* Cap pool preallocation per sub-range to at most 16 skbs in seq
> > + * space.  A single first skb exceeding this is still collapsed,
> > + * whereas an exceeding overlapping ofo range is skipped.
> > + */
> > +#define TCP_COLLAPSE_MAX_TOTAL_SIZE (TCP_COLLAPSE_MAX_SIZE * 16)
> >
> >  static bool tcp_collapse_skb_bloated(const struct sk_buff *skb,
> >                                      u32 start, u8 scaling_ratio)
> > @@ -5862,8 +5867,9 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
> >  {
> >         unsigned int old_truesize, new_truesize;
> >         struct sk_buff *skb = head, *n;
> > -       struct sk_buff_head tmp;
> > +       struct sk_buff_head tmp, pool;
> >         int remainder, nskbs;
> > +       struct sk_buff *nskb;
> >         u8 scaling_ratio;
> >         u32 sub_end;
> >
> > @@ -5871,6 +5877,7 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
> >                             TCP_COLLAPSE_SCALING_RATIO);
> >
> >         __skb_queue_head_init(&tmp);
> > +       __skb_queue_head_init(&pool);
> >
> >         /* First, check that queue is collapsible and find
> >          * the point where collapsing can be useful.
> > @@ -5926,8 +5933,9 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
> >                 }
> >
> >                 if (!overlap &&
> > -                   !tcp_collapse_skb_bloated(n, sub_end,
> > -                                             TCP_COLLAPSE_SCALING_RATIO))
> > +                   (TCP_SKB_CB(n)->end_seq - start >= TCP_COLLAPSE_MAX_TOTAL_SIZE ||
> > +                    !tcp_collapse_skb_bloated(n, sub_end,
> > +                                              TCP_COLLAPSE_SCALING_RATIO)))
> >                         break;
> >
> >                 sub_end = TCP_SKB_CB(n)->end_seq;
> > @@ -5935,25 +5943,38 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
> >                 n = tcp_skb_next(n, list);
> >         }
> >
> > +       if (sub_end != TCP_SKB_CB(skb)->end_seq &&
> > +           sub_end - start >= TCP_COLLAPSE_MAX_TOTAL_SIZE)
> > +               goto skip;
> 
> This is fine for now, but note that a single 64K GRO skb overlapping
> a bloated skb is enough to reach the limit, and the bloated skb is
> then never collapsed. The range is walked again at every
> tcp_collapse_ofo_queue() call, for no gain.
> 
> A possible followup would be to cut the range instead of skipping it.
> 
> When @n overlaps and we do not want (or can not) merge it, we know
> that @n already contains [TCP_SKB_CB(n)->seq, sub_end), because
> skbs with end_seq <= sub_end were removed earlier in the loop.
> 
> So we could do :
> 
>         sub_end = TCP_SKB_CB(n)->seq;
> 
> and collapse [start, sub_end) only, as long as sub_end is after start.
> 
> All new skbs would then have a seq smaller than TCP_SKB_CB(n)->seq,
> so the rbtree ordering is preserved, and we would not need to drag
> @n into the range.
> 
> After the copy, the skbs of the range that were not fully consumed
> would have to be freed, since the rest of their payload is in @n.
> 
> The same trick would also work for the unmergeable overlapping skb
> case added in patch 3.
> 
> I have not tested this, and this can be done later.

Ah, this sounds good too, I'll follow up after your frag series.

Thank you !

  reply	other threads:[~2026-11-19 20:03 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  7:15 [PATCH v3 net-next 0/7] tcp: Prevent tcp_collapse() from bloating queue Kuniyuki Iwashima
2026-11-19  7:15 ` [PATCH v3 net-next 1/7] tcp: Refactor the first loop in tcp_collapse() Kuniyuki Iwashima
2026-11-19  7:15 ` [PATCH v3 net-next 2/7] tcp: Don't collapse high density skb Kuniyuki Iwashima
2026-11-19  7:15 ` [PATCH v3 net-next 3/7] tcp: Stop collapsing if not bloated Kuniyuki Iwashima
2026-11-19  7:39   ` Eric Dumazet
2026-11-19  7:15 ` [PATCH v3 net-next 4/7] tcp: Skip collapsing if collapsed skb is less-dense Kuniyuki Iwashima
2026-11-19  7:41   ` Eric Dumazet
2026-11-19  7:15 ` [PATCH v3 net-next 5/7] tcp: Allocate all new skb before collapsing Kuniyuki Iwashima
2026-11-19  7:44   ` Eric Dumazet
2026-11-19 20:02     ` Kuniyuki Iwashima [this message]
2026-11-19  7:15 ` [PATCH v3 net-next 6/7] tcp: Batch memory accounting in tcp_collapse() Kuniyuki Iwashima
2026-11-19  7:15 ` [PATCH v3 net-next 7/7] selftest: packetdrill: Add test for tcp_collapse() Kuniyuki Iwashima
2026-11-19  7:45   ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av9U9w4wGpO7HmDY@google.com \
    --to=kuniyu@google.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=horms@kernel.org \
    --cc=kuba@kernel.org \
    --cc=kuni1840@gmail.com \
    --cc=martineau@kernel.org \
    --cc=matttbe@kernel.org \
    --cc=mptcp@lists.linux.dev \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox