[PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[Eric Dumazet]

tcp_gro_pull_header() is used in GRO fast path, inline it.

Signed-off-by: Eric Dumazet <edumazet@google.com>
---
 include/net/gro.h      | 27 +++++++++++++++++++++++++++
 include/net/tcp.h      |  1 -
 net/ipv4/tcp_offload.c | 27 ---------------------------
 3 files changed, 27 insertions(+), 28 deletions(-)

diff --git a/include/net/gro.h b/include/net/gro.h
index e3affb2e2ca8b2c89a9ffd6f7fb1219a42200ac3..b65f631c521d7d9741ef86781add0038c9ce4055 100644
--- a/include/net/gro.h
+++ b/include/net/gro.h
@@ -593,4 +593,31 @@ static inline void inet6_get_iif_sdif(const struct sk_buff *skb, int *iif, int *
 struct packet_offload *gro_find_receive_by_type(__be16 type);
 struct packet_offload *gro_find_complete_by_type(__be16 type);
 
+static inline struct tcphdr *tcp_gro_pull_header(struct sk_buff *skb)
+{
+	unsigned int thlen, hlen, off;
+	struct tcphdr *th;
+
+	off = skb_gro_offset(skb);
+	hlen = off + sizeof(*th);
+	th = skb_gro_header(skb, hlen, off);
+	if (unlikely(!th))
+		return NULL;
+
+	thlen = th->doff * 4;
+	if (unlikely(thlen < sizeof(*th)))
+		return NULL;
+
+	hlen = off + thlen;
+	if (!skb_gro_may_pull(skb, hlen)) {
+		th = skb_gro_header_slow(skb, hlen, off);
+		if (unlikely(!th))
+			return NULL;
+	}
+
+	skb_gro_pull(skb, thlen);
+
+	return th;
+}
+
 #endif /* _NET_GRO_H */
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4833ec7903eca45cd31bb4ab023a8d326ed8b436..0deb5e9dd9114641468f26696b086b8261a67f60 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2313,7 +2313,6 @@ void tcp_v4_destroy_sock(struct sock *sk);
 
 struct sk_buff *tcp_gso_segment(struct sk_buff *skb,
 				netdev_features_t features);
-struct tcphdr *tcp_gro_pull_header(struct sk_buff *skb);
 struct sk_buff *tcp_gro_lookup(struct list_head *head, struct tcphdr *th);
 struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb,
 				struct tcphdr *th);
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index 2cb93da93abc2922fc347f4d62093741eb17c9d8..fdda18b1abda0f69226f6b59cec6d7d51b53555b 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -282,33 +282,6 @@ struct sk_buff *tcp_gro_lookup(struct list_head *head, struct tcphdr *th)
 	return NULL;
 }
 
-struct tcphdr *tcp_gro_pull_header(struct sk_buff *skb)
-{
-	unsigned int thlen, hlen, off;
-	struct tcphdr *th;
-
-	off = skb_gro_offset(skb);
-	hlen = off + sizeof(*th);
-	th = skb_gro_header(skb, hlen, off);
-	if (unlikely(!th))
-		return NULL;
-
-	thlen = th->doff * 4;
-	if (thlen < sizeof(*th))
-		return NULL;
-
-	hlen = off + thlen;
-	if (!skb_gro_may_pull(skb, hlen)) {
-		th = skb_gro_header_slow(skb, hlen, off);
-		if (unlikely(!th))
-			return NULL;
-	}
-
-	skb_gro_pull(skb, thlen);
-
-	return th;
-}
-
 struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb,
 				struct tcphdr *th)
 {
-- 
2.51.2.1041.gc1ab5b90ca-goog

Re: [PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[Alexander Lobakin]

From: Eric Dumazet <edumazet@google.com>
Date: Thu, 13 Nov 2025 14:03:57 +0000

> tcp_gro_pull_header() is used in GRO fast path, inline it.

Looks reasonable, but... any perf numbers? bloat-o-meter stats?

> 
> Signed-off-by: Eric Dumazet <edumazet@google.com>

Thanks,
Olek

Re: [PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[Eric Dumazet]

On Fri, Nov 14, 2025 at 7:56 AM Alexander Lobakin
<aleksander.lobakin@intel.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
> Date: Thu, 13 Nov 2025 14:03:57 +0000
>
> > tcp_gro_pull_header() is used in GRO fast path, inline it.
>
> Looks reasonable, but... any perf numbers? bloat-o-meter stats?

This is used two times, one from IPv4, one from IPv6.

FDO usually embeds this function in the two callers, this patch
reduces the gap between FDO and non FDO kernels.
Non FDO builds get a ~0.5 % performance increase with this patch, for
a cost of less than 192 bytes on x86_64.

It might sound small, but adding all these changes together is not small.

Re: [PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[Alexander Lobakin]

From: Eric Dumazet <edumazet@google.com>
Date: Fri, 14 Nov 2025 08:09:31 -0800

> On Fri, Nov 14, 2025 at 7:56 AM Alexander Lobakin
> <aleksander.lobakin@intel.com> wrote:
>>
>> From: Eric Dumazet <edumazet@google.com>
>> Date: Thu, 13 Nov 2025 14:03:57 +0000
>>
>>> tcp_gro_pull_header() is used in GRO fast path, inline it.
>>
>> Looks reasonable, but... any perf numbers? bloat-o-meter stats?
> 
> This is used two times, one from IPv4, one from IPv6.
> 
> FDO usually embeds this function in the two callers, this patch
> reduces the gap between FDO and non FDO kernels.
> Non FDO builds get a ~0.5 % performance increase with this patch, for
> a cost of less than 192 bytes on x86_64.

I asked for these as you usually provide detailed stats with `perf top`
output etc, but not this time.

(although you always require to provide perf stats when someone else
 sends an optimization patch)

> 
> It might sound small, but adding all these changes together is not small.

A couple weeks ago you wrote that 1% of perf improvement is "a noise".

I'm not against this patch (if you add everything from the above to the
commit message + maybe your usual detailed stats).
+0.5% for 192 bytes sounds good to me (I don't call it "a noise").

But from my PoV this just feels like "my 0.5% is bigger than your 1%"
and "you have to show me the numbers, and I don't".

The rules are the same for everyone.

Thanks,
Olek

Re: [PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[Eric Dumazet]

On Fri, Nov 14, 2025 at 8:39 AM Alexander Lobakin
<aleksander.lobakin@intel.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
> Date: Fri, 14 Nov 2025 08:09:31 -0800
>
> > On Fri, Nov 14, 2025 at 7:56 AM Alexander Lobakin
> > <aleksander.lobakin@intel.com> wrote:
> >>
> >> From: Eric Dumazet <edumazet@google.com>
> >> Date: Thu, 13 Nov 2025 14:03:57 +0000
> >>
> >>> tcp_gro_pull_header() is used in GRO fast path, inline it.
> >>
> >> Looks reasonable, but... any perf numbers? bloat-o-meter stats?
> >
> > This is used two times, one from IPv4, one from IPv6.
> >
> > FDO usually embeds this function in the two callers, this patch
> > reduces the gap between FDO and non FDO kernels.
> > Non FDO builds get a ~0.5 % performance increase with this patch, for
> > a cost of less than 192 bytes on x86_64.
>
> I asked for these as you usually provide detailed stats with `perf top`
> output etc, but not this time.

Because I am currently chasing many small optimizations, and they hit
various subsystems.

>
> (although you always require to provide perf stats when someone else
>  sends an optimization patch)
>
> >
> > It might sound small, but adding all these changes together is not small.
>
> A couple weeks ago you wrote that 1% of perf improvement is "a noise".
>
> I'm not against this patch (if you add everything from the above to the
> commit message + maybe your usual detailed stats).
> +0.5% for 192 bytes sounds good to me (I don't call it "a noise").
>
> But from my PoV this just feels like "my 0.5% is bigger than your 1%"
> and "you have to show me the numbers, and I don't".
>
> The rules are the same for everyone.

Sure, thank you so much !

Re: [PATCH net-next] tcp: gro: inline tcp_gro_pull_header()

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Thu, 13 Nov 2025 14:03:57 +0000 you wrote:
> tcp_gro_pull_header() is used in GRO fast path, inline it.
> 
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---
>  include/net/gro.h      | 27 +++++++++++++++++++++++++++
>  include/net/tcp.h      |  1 -
>  net/ipv4/tcp_offload.c | 27 ---------------------------
>  3 files changed, 27 insertions(+), 28 deletions(-)

Here is the summary with links:
  - [net-next] tcp: gro: inline tcp_gro_pull_header()
    https://git.kernel.org/netdev/net-next/c/6d650ae9282b

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html