From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=209.85.218.49 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1795052325; b=kNtKLb5nCqmmNXPtWnhimIUY+f4RRmvWDsbCGVb8NUEQukWeOx+1DxwpwJtB6ScRSmBZ bFQXwJDoZcIV/5Sa/H6yADTgDG5kAhF5bLAe9AHpfeWKo3J+2ryV8IizyGpIBLOvLgLH9 0UEtvx+FvEAN5pPBnZb6LMXiCAkBNToMTYUEpmHPgfv25pZ6ekzP+nlEo/mhvcnUd0AIP MS02uwIZpgnAUd5OmqQQSWj7B9JNWUJpCxDTzJkS4+AOaRYq4sAan++QovGAQUUejzv/S LHmnEV5ovM5b5uVQukV/6FGsemNr9afDdEhf8LOvW7SqpcNUskJvgyxWaWWDg2t4cFA== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795052325; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Gm-Gg:X-Received:Received:From:To:Cc:Subject: Date:Message-ID:X-Mailer:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding; bh=Gt22CS9KurBhOaCwllbQzvFCleXjrzTFFkshOQCBzQs=; b=JBowba5G5Biv/z6VR/WZSIG8WnO2Bkq8mgPPZ/O+Fcr/VycRTb8e2LFxuPEh243zcShC MaLd9pCYgKfGzLNSGNSCUEWtazmGMoAw+0ALrvi8IfUW7kVfeiaC5PhVNETT8bJRkEFF0 D9HutskbNtb0I2/5nxqff+QUUDsDqruDTiGatG5vI2LrHHI0HYqvKzksce7+gze26hXXg 07lz+ktEqpwNcx3iwiRIh5hteRwwB5qFGzJEKsHZT3kQlJI9RdULGOiZZV4GTJZbydXRr 6AGZPB8g+NAcfSZGKQ2vSva30Q8YF5bz13C/fI/JIbrjKVFtcmPc5U/y+2e11g9Sbng== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google09082023 header.b=FqjNUR0p; arc=none smtp.remote-ip=209.85.218.49 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA494606A338 for ; Thu, 19 Nov 2026 01:38:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org EA494606A338 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google09082023 header.b=FqjNUR0p Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-c4cb6834eaaso276217866b.2 for ; Wed, 18 Nov 2026 17:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1795052322; x=1795657122; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Gt22CS9KurBhOaCwllbQzvFCleXjrzTFFkshOQCBzQs=; b=FqjNUR0pj/V1OQcCToct5qmQlcnfTIuMF3YJLs3SLeiXwOogpBpF9zrc2qap++7cQZ CqUgQcntRbQC8o0eOrBIfjgldAaa/N9tsLyGFWvaFwiS1oMZ5zY/e/nYWeI2XQNsmz9x iTd9Objy7p6pMimgGhJZxGXAmmBdKJ2mOgl7hTssvte0/F+h7NlI+d6oomZpr3LyzsEW QERcPEhre6rtIl/Xcx+fbEg/XYP0h1rzRp3ujeqPbu8554aijADJln85rL9nvJx5rI1u cN+PtcAQYivWMnvIJmLOrfYkzbl8y30hzOqMfdJcWaztMsE2tTuV3rnfmNxKACUq4tvD 7Slw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795052322; x=1795657122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Gt22CS9KurBhOaCwllbQzvFCleXjrzTFFkshOQCBzQs=; b=RlXUeOltNu2uYyy9T+EyuMhzA2S3T94E9wCont6NcftdmqxRqqhQTj0IWFhi7BRyA2 EWUid8K4HHFvUYt0P3OANitbkPMsO6SdmyNbl0F+lItwNzP6OuD5DDJCS1TAhcCVAY7J dRj0Q9k+iFgzCQoWOT9REP6+X1Kn0qeVz+nT+t7p2QWWqLBqdf6Tho0kK34kBRYlrynS 0WUcKf5e2HB9XIQ1keezhkqmeruxObnrISjP9TZqkIq6POL8hLYGCeAshpou12YapLND Bit69XSUI2cTehhsHO6CY6VIJwQvPQaxSlBhoBDGYsT+wiecI8LNVr5yxLT1oMCrpAx9 wFfA== X-Gm-Message-State: APuU4Z8DRmI6FhkozQv+WKdYU7LCD9n4ecW4ux2JKp8xnVrpqfu1elcy KtohbF/Dc8FlLqIuyh3CwMKWGPkH5yjELcqx0XBMvWdoixc1OtDjnoXAy27EgkHMFGjcQCOrbY5 CqaTj48w= X-Gm-Gg: AYBFou32j9I/hVOU4mroIaLzydrPzn1MvUqpf5/kMxePqOH9ayVf1DFc5Psg6Xt8Qh8 Tb/dz9diDz16RnWojVoF0NRX17KX0vpr+AHMKGEW17UVdKuLAiQWnG1okXT+/Zwx0r7QtZIyO3X +qGE4dNXAguTd7rH1jnRypWHNTHRxDWrXpEnwGwPPUUbMrzwjmu+26k5W6AR809qmozzZTFMntE rw8t5fOmW8JfI3fR3xvHry2uUFwz1iz9UOEGTobAIpY+6tNg/CH8wyGzc/qo4xiNSt7yZQZp3kt 1LMIRD5iIamxFsJInEvNXHcGRfPi6YE/DWWJugsqpkvleQMdZlsbQoi0uYXavfO3T25A+ws40aq K2DJ4Xk1BL2fg/VJhx7poOzn7FNQ6bExLB88qvyDhdY1jd5ugZmMZYYrHnkqbNC84Y7zitDyyWM rPHey8Pxl0OjLX8SPSsN1x5zXSv54O64c67ZcaKuk4pZrGoy4sWxHiVONePAGWhx1SPBo5C8HwY qznIbTEPQkIkK8X7R+4jzWCsFqC6tDQKbXhPSLIhLf3SxhI4+JVrI3aphOafvY0GToxP3d8i1k3 bEyCL6OHGfwezWIYJeb8rpik03///pkjXIibv8x+3sJG5q3tMwMkF4xJBog= X-Received: by 2002:a17:907:da17:b0:c50:4c04:36e3 with SMTP id a640c23a62f3a-c510e349068mr350958766b.37.1795052322240; Wed, 18 Nov 2026 17:38:42 -0800 (PST) Received: from L9JK9NPFH9 ([104.28.20.56]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c51233c4080sm59785966b.16.2026.11.18.17.38.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 18 Nov 2026 17:38:40 -0800 (PST) From: Tianyi Gao To: netdev@vger.kernel.org, Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: "David S. Miller" , Simon Horman , David Keogh , Jesper Dangaard Brouer , Daniel Borkmann , David Woodhouse , Dragos Tatulea , Nimrod Oren , Alasdair McWilliam , Willem de Bruijn , Maciej Fijalkowski , Stanislav Fomichev , Magnus Karlsson , David Ahern , Ido Schimmel , Alexei Starovoitov , John Fastabend , Donald Hunter , Andrew Lunn , Jonathan Corbet , Shuah Khan , Shuah Khan , Randy Dunlap , Breno Leitao , =?UTF-8?q?Asbj=C3=B8rn=20Sloth=20T=C3=B8nnesen?= , Danielle Ratson , Kuniyuki Iwashima , bpf@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, nh-open-source@amazon.com, Stephen Hemminger , "Gustavo A. R. Silva" , "Remy D. Farley" , Amery Hung , Andrii Nakryiko , Andy Roulin , Anil Kaushik , Demos Ra , Eduard Zingerman , Emil Tsalapatis , Hangbin Liu , Ihor Solodrai , Jiayuan Chen , Jiri Olsa , Joe Damato , Kumar Kartikeya Dwivedi , Li RongQing , Louis Scalbert , Marc Kleine-Budde , Martin KaFai Lau , Martin Karsten , Matt Bobrowski , Michael Bommarito , Nikolay Aleksandrov , Runyu Xiao , Song Liu , Thorsten Leemhuis , Wang Zhan , Yonghong Song , Zhou Qiuyue , Tianyi Gao , Tianyi Gao Subject: [PATCH net-next v2 02/11] net: reserve extra TX headroom for local egress Date: Thu, 19 Nov 2026 01:38:21 +0000 Message-ID: <20261119013830.36935-3-tianyi@cloudflare.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261119013830.36935-1-tianyi@cloudflare.com> References: <20261119013830.36935-1-tianyi@cloudflare.com> X-Mailing-List: netdev@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: David Keogh Use netdev_get_alloc_headroom() and netdev_get_alloc_headroom_hlen() instead of LL_RESERVED_SPACE() and LL_RESERVED_SPACE_EX() where the stack allocates packets on behalf of a local sender and sizes them for the output device: - __ip_append_data() and __ip6_append_data(), covering UDP, ICMP, ping and raw sockets without IP_HDRINCL; - raw_send_hdrinc() and rawv6_send_hdrinc(); - the three AF_PACKET transmit paths; - AF_XDP copy mode, which open-codes its own calculation from needed_headroom and now uses netdev_get_alloc_needed_headroom(). These are the main paths where a packet's headroom is decided by the device's advertised requirement and nothing else. Adding the extra reservation here means such packets reach the TC egress hook with room for additional headers already in place. Paths that check an existing packet against LL_RESERVED_SPACE() and expand it on demand, such as ip_finish_output2() and the tunnel transmit functions, are deliberately left alone. Using the larger value there would force a reallocation of every packet that lacks the full extra reservation, including TCP segments that already have enough room for the headers an egress program actually pushes. ip_do_fragment() and ip6_fragment() are also left alone. They use the same value to check the headroom of existing fragments and to size new ones. A larger value would send more packets to the slow path, which copies them. A local sender's fragments normally come from __ip_append_data() or __ip6_append_data(). Signed-off-by: David Keogh Co-developed-by: Tianyi Gao Signed-off-by: Tianyi Gao --- v2: - Follow the renames in patch 1. --- net/ipv4/ip_output.c | 2 +- net/ipv4/raw.c | 2 +- net/ipv6/ip6_output.c | 2 +- net/ipv6/raw.c | 2 +- net/packet/af_packet.c | 7 ++++--- net/xdp/xsk.c | 6 ++++-- 6 files changed, 12 insertions(+), 9 deletions(-) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 5d6f5f5fc0ed..dedea01c6adc 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -988,7 +988,7 @@ static int __ip_append_data(struct sock *sk, mtu = cork->gso_size ? IP_MAX_MTU : cork->fragsize; paged = !!cork->gso_size; - hh_len = LL_RESERVED_SPACE(rt->dst.dev); + hh_len = netdev_get_alloc_headroom(rt->dst.dev); fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0); maxfraglen = ((mtu - fragheaderlen) & ~7) + fragheaderlen; diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c index f33a9dfc5309..85e2178f36d6 100644 --- a/net/ipv4/raw.c +++ b/net/ipv4/raw.c @@ -356,7 +356,7 @@ static int raw_send_hdrinc(struct sock *sk, struct flowi4 *fl4, if (flags&MSG_PROBE) goto out; - hlen = LL_RESERVED_SPACE(rt->dst.dev); + hlen = netdev_get_alloc_headroom(rt->dst.dev); tlen = rt->dst.dev->needed_tailroom; skb = sock_alloc_send_skb(sk, length + hlen + tlen + 15, diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 3cc911f6618e..50393d9a81ee 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1493,7 +1493,7 @@ static int __ip6_append_data(struct sock *sk, mtu = cork->gso_size ? IP6_MAX_MTU : cork->fragsize; orig_mtu = mtu; - hh_len = LL_RESERVED_SPACE(rt->dst.dev); + hh_len = netdev_get_alloc_headroom(rt->dst.dev); fragheaderlen = sizeof(struct ipv6hdr) + rt->rt6i_nfheader_len + (opt ? opt->opt_nflen : 0); diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index 00d832c7cdae..00cf7475462e 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -602,7 +602,7 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length, struct sk_buff *skb; int err; struct rt6_info *rt = dst_rt6_info(*dstp); - int hlen = LL_RESERVED_SPACE(rt->dst.dev); + int hlen = netdev_get_alloc_headroom(rt->dst.dev); int tlen = rt->dst.dev->needed_tailroom; if (length > rt->dst.dev->mtu) { diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 6ace2a5b2d17..a1932a0e5e42 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -2032,7 +2032,8 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, goto out_unlock; if (!skb) { - size_t reserved = LL_RESERVED_SPACE_EX(dev, hard_header_len); + size_t reserved = + netdev_get_alloc_headroom_hlen(dev, hard_header_len); int tlen = dev->needed_tailroom; unsigned int hhlen = dev->header_ops ? hard_header_len : 0; @@ -2860,7 +2861,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto tpacket_error; status = TP_STATUS_SEND_REQUEST; - hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); + hlen = netdev_get_alloc_headroom_hlen(dev, hard_header_len); tlen = dev->needed_tailroom; if (vnet_hdr_sz) { data += vnet_hdr_sz; @@ -3087,7 +3088,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock; err = -ENOBUFS; - hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); + hlen = netdev_get_alloc_headroom_hlen(dev, hard_header_len); tlen = dev->needed_tailroom; linear = __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len); linear = max(linear, min_t(int, len, hard_header_len)); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index f22f9a2cdbbe..5932add2879f 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -1047,7 +1047,8 @@ static struct sk_buff *xsk_build_skb_zerocopy(struct xdp_sock *xs, buffer = xsk_buff_raw_get_data(pool, addr); if (!skb) { - hr = max(NET_SKB_PAD, L1_CACHE_ALIGN(xs->dev->needed_headroom)); + hr = netdev_get_alloc_needed_headroom(xs->dev); + hr = max(NET_SKB_PAD, L1_CACHE_ALIGN(hr)); skb = sock_alloc_send_skb(&xs->sk, hr, 1, &err); if (unlikely(!skb)) @@ -1130,7 +1131,8 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, len = desc->len; if (!skb) { - hr = max(NET_SKB_PAD, L1_CACHE_ALIGN(dev->needed_headroom)); + hr = netdev_get_alloc_needed_headroom(dev); + hr = max(NET_SKB_PAD, L1_CACHE_ALIGN(hr)); tr = dev->needed_tailroom; skb = sock_alloc_send_skb(&xs->sk, hr + len + tr, 1, &err); if (unlikely(!skb)) -- 2.56.0