From: Jiebin Sun <jiebin.sun@intel.com>
To: netdev@vger.kernel.org
Cc: edumazet@kernel.org, davem@davemloft.net, kuba@kernel.org,
pabeni@redhat.com, horms@kernel.org,
linux-kernel@vger.kernel.org, tim.c.chen@linux.intel.com,
tianyou.li@intel.com, wangyang.guo@intel.com
Subject: [PATCH net-next v2] net: dst: avoid false sharing between tclassid and __rcuref
Date: Fri, 20 Nov 2026 14:16:04 +0800 [thread overview]
Message-ID: <20261120061604.3676605-1-jiebin.sun@intel.com> (raw)
On x86-64 with CONFIG_IP_ROUTE_CLASSID=y, ip_rcv_finish_core() reads
dst_entry::tclassid from the cache line containing __rcuref. Concurrent
reference updates cause false sharing with this load.
Swap expires with {error, __pad, tclassid} to move tclassid and error
into the first cache line without growing dst_entry. The structures
that embed it, including rtable, rt6_info, and metadata_dst, keep
their sizes.
pahole on x86-64 (sizes and offsets in bytes)
+----------------------+--------+-------+
| Struct or field | Before | After |
+----------------------+--------+-------+
| sizeof(dst_entry) | 136 | 136 |
| dst_entry.expires | 24 | 96 |
| dst_entry.__rcuref | 64 | 64 |
| dst_entry.tclassid | 100 | 28 |
+----------------------+--------+-------+
The IPv4 loopback paths are shown below.
TX: __dev_queue_xmit() -> skb_dst_force() -> dst_hold_safe()
-> rcuref_get()
RX: ip_rcv_finish_core() -> skb_dst(skb)->tclassid
At the tclassid load, separate 30-second perf c2c runs recorded:
+----------+------------+-------------+
| Kernel | Local HITM | Remote HITM |
+----------+------------+-------------+
| Baseline | 2624 | 2611 |
| Swap | 1 | 0 |
+----------+------------+-------------+
Memcached over IPv4 loopback (64 threads each, 5 x 60s,
20s warmup, mean ops/s)
Baseline: commit 6628e7d4c6b2 ("Linux 7.4-rc1")
+-----------+-----------------+
| Memcached | Improvement (%) |
+-----------+-----------------+
| default | +32.55 |
| hit | +45.79 |
| balanced | -0.12 |
| large | +45.62 |
+-----------+-----------------+
Default client
memtier_benchmark -s 127.0.0.1 -p 11221 -P memcache_text
-t 64 -c 1 --pipeline=16 --test-time=60 --ratio=1:100
--data-size=32 --key-maximum=10000000
Server
memcached -c 4096 -t 64 -u pnp -p 11221 -l 127.0.0.1
Client variants
hit: --key-maximum=1000
balanced: --key-maximum=1000 --ratio=1:1
large: --key-maximum=1000 --data-size=1024
Suggested-by: Eric Dumazet <edumazet@kernel.org>
Suggested-by: Tim Chen <tim.c.chen@linux.intel.com>
Reviewed-by: Tianyou Li <tianyou.li@intel.com>
Reviewed-by: Wangyang Guo <wangyang.guo@intel.com>
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
---
Changes in v2:
- Swap expires with error/__pad/tclassid and show the pahole results.
- Clarify the loopback path and keep the memcached and c2c tables.
include/net/dst.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/net/dst.h b/include/net/dst.h
index dbedfe72e1fd..a692fe85f4bc 100644
--- a/include/net/dst.h
+++ b/include/net/dst.h
@@ -30,7 +30,9 @@ struct dst_entry {
};
struct dst_ops *ops;
unsigned long _metrics;
- unsigned long expires;
+ short error;
+ short __pad;
+ __u32 tclassid;
#ifdef CONFIG_XFRM
struct xfrm_state *xfrm;
#else
@@ -74,9 +76,7 @@ struct dst_entry {
int __use;
unsigned long lastuse;
struct rcu_head rcu_head;
- short error;
- short __pad;
- __u32 tclassid;
+ unsigned long expires;
#ifndef CONFIG_64BIT
struct lwtunnel_state *lwtstate;
rcuref_t __rcuref; /* 32-bit offset 64 */
base-commit: 66726ba54a01a2951688a56f5eb4d13d063acbea
--
2.52.0
reply other threads:[~2026-11-20 6:11 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120061604.3676605-1-jiebin.sun@intel.com \
--to=jiebin.sun@intel.com \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=horms@kernel.org \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=tianyou.li@intel.com \
--cc=tim.c.chen@linux.intel.com \
--cc=wangyang.guo@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox