[PATCH net-next] seg6: reallocate the skb head on L2 encapsulation only when needed

[PATCH net-next] seg6: reallocate the skb head on L2 encapsulation only when needed

[Yuya Kusakabe]

The L2 encapsulation modes of the seg6 lwtunnel reallocate the skb head
on every packet, where the IPv6 encapsulation modes reallocate only when
they have to. This site only has to fit skb->mac_len before its
skb_push(); the outer header that follows is already covered by
__seg6_do_srh_encap()'s own skb_cow_head(). On a forwarding path neither
reallocation is necessary: ixgbe leaves 206 bytes of headroom, against

  14 (skb->mac_len) + 40 (ipv6hdr) + 24 (SRH with one segment)
  + 16 (LL_RESERVED_SPACE, via dst_dev_overhead()) = 94

for the whole encapsulation.

The cost is amplified by CONFIG_INIT_ON_ALLOC_DEFAULT_ON, which many
distributions enable: every new head is zeroed in full, and that memset
alone accounts for 16% of the datapath profile.

Use skb_cow_head() instead, matching the IPv6 encapsulation modes.

Throughput at 0.5% packet loss, 64-byte frames forwarded through one
2.30 GHz core (Xeon E5-2650 v3, ixgbe 82599ES), offered by TRex and
binary-searched over 10 runs of 10 s:

  Before: 660.7 kpps
  After:  991.3 kpps

Assisted-by: Claude:claude-opus-5
Signed-off-by: Yuya Kusakabe <yuya.kusakabe@gmail.com>
---
 net/ipv6/seg6_iptunnel.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/net/ipv6/seg6_iptunnel.c b/net/ipv6/seg6_iptunnel.c
index 61c6a27bf202..a6556efd8e0b 100644
--- a/net/ipv6/seg6_iptunnel.c
+++ b/net/ipv6/seg6_iptunnel.c
@@ -446,8 +446,9 @@ static int seg6_do_srh(struct sk_buff *skb, struct dst_entry *cache_dst)
 		if (!skb_mac_header_was_set(skb))
 			return -EINVAL;
 
-		if (pskb_expand_head(skb, skb->mac_len, 0, GFP_ATOMIC) < 0)
-			return -ENOMEM;
+		err = skb_cow_head(skb, skb->mac_len);
+		if (unlikely(err))
+			return err;
 
 		skb_mac_header_rebuild(skb);
 		skb_push(skb, skb->mac_len);

---
base-commit: c8ea08ba34f2a2e9bfb18ff3d69eb2d69b324f49
change-id: 20260902-seg6-l2cow-77dc3ba41232

Best regards,
--  
Yuya Kusakabe <yuya.kusakabe@gmail.com>

Re: [PATCH net-next] seg6: reallocate the skb head on L2 encapsulation only when needed

[Eric Dumazet]

On Wed, Sep 2, 2026 at 10:34 AM Yuya Kusakabe <yuya.kusakabe@gmail.com> wrote:
>
> The L2 encapsulation modes of the seg6 lwtunnel reallocate the skb head
> on every packet, where the IPv6 encapsulation modes reallocate only when
> they have to. This site only has to fit skb->mac_len before its
> skb_push(); the outer header that follows is already covered by
> __seg6_do_srh_encap()'s own skb_cow_head(). On a forwarding path neither
> reallocation is necessary: ixgbe leaves 206 bytes of headroom, against
>
>   14 (skb->mac_len) + 40 (ipv6hdr) + 24 (SRH with one segment)
>   + 16 (LL_RESERVED_SPACE, via dst_dev_overhead()) = 94
>
> for the whole encapsulation.
>
> The cost is amplified by CONFIG_INIT_ON_ALLOC_DEFAULT_ON, which many
> distributions enable: every new head is zeroed in full, and that memset
> alone accounts for 16% of the datapath profile.
>
> Use skb_cow_head() instead, matching the IPv6 encapsulation modes.
>
> Throughput at 0.5% packet loss, 64-byte frames forwarded through one
> 2.30 GHz core (Xeon E5-2650 v3, ixgbe 82599ES), offered by TRex and
> binary-searched over 10 runs of 10 s:
>
>   Before: 660.7 kpps
>   After:  991.3 kpps
>
> Assisted-by: Claude:claude-opus-5
> Signed-off-by: Yuya Kusakabe <yuya.kusakabe@gmail.com>
> ---
>  net/ipv6/seg6_iptunnel.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/net/ipv6/seg6_iptunnel.c b/net/ipv6/seg6_iptunnel.c
> index 61c6a27bf202..a6556efd8e0b 100644
> --- a/net/ipv6/seg6_iptunnel.c
> +++ b/net/ipv6/seg6_iptunnel.c
> @@ -446,8 +446,9 @@ static int seg6_do_srh(struct sk_buff *skb, struct dst_entry *cache_dst)
>                 if (!skb_mac_header_was_set(skb))
>                         return -EINVAL;
>
> -               if (pskb_expand_head(skb, skb->mac_len, 0, GFP_ATOMIC) < 0)
> -                       return -ENOMEM;
> +               err = skb_cow_head(skb, skb->mac_len);
> +               if (unlikely(err))
> +                       return err;
>
>                 skb_mac_header_rebuild(skb);
>                 skb_push(skb, skb->mac_len);
>

This is a nice improvement for forwarded traffic, but locally
generated TCP traffic
is still hitting two expensive reallocations (second one in
__seg6_do_srh_encap())

Can we combine needed headrooms so that only a single re-alloc occurs?

 headroom = skb->mac_len + sizeof(struct ipv6hdr) +
  ipv6_optlen(tinfo->srh) +
 dst_dev_overhead(cache_dst, skb);

err = skb_cow_head(skb, headroom);
if (unlikely(err))
        return err;

Re: [PATCH net-next] seg6: reallocate the skb head on L2 encapsulation only when needed

[Yuya Kusakabe]

On Wed, Sep 2, 2026 at 11:00 AM Eric Dumazet <edumazet@google.com> wrote:
> This is a nice improvement for forwarded traffic, but locally
> generated TCP traffic
> is still hitting two expensive reallocations (second one in
> __seg6_do_srh_encap())
>
> Can we combine needed headrooms so that only a single re-alloc occurs?
Thanks for looking at this.  I will send a v2 with the combined
headroom.

One clarification on the locally generated case: seg6_build_state()
does not redirect the output path for the L2 modes, so locally
generated traffic never reaches this branch:

	newts->flags |= LWTUNNEL_STATE_INPUT_REDIRECT;

	if (tuninfo->mode != SEG6_IPTUN_MODE_L2ENCAP &&
	    tuninfo->mode != SEG6_IPTUN_MODE_L2ENCAP_RED)
		newts->flags |= LWTUNNEL_STATE_OUTPUT_REDIRECT;

Two reallocations do happen on the forwarded path, though: with a
header-cloned skb whose headroom is below the total, asking only for
skb->mac_len leaves the second reallocation to __seg6_do_srh_encap(),
because the cow that unclones the skb does not also make room for the
outer header.  So the combined headroom is the right shape either way.