From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oa1-f54.google.com (mail-oa1-f54.google.com [209.85.160.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F16B483806 for ; Wed, 26 Aug 2026 20:20:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787775618; cv=none; b=KDwfjoSFkJ3y2cuAsrwiJwDOrXHK6zac7KyevaSGH65DgF20aoLV/mBZMzykTMGTqW/tUK/Di35oylPLkRQhT8cBaiu0+P7fw/1Ng+8JkTdo/xrVmcCnOTS6FsqWieRECEYnXXuAREMBZfq90S0HDjyu5pGRs5YNQutmr/qwnMU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787775618; c=relaxed/simple; bh=Ut/PjvIImnY64JnqnynvMuxcOBqH+zLzueSVJw4Qbq0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=c9YvD36AhOAVf33iV9M0tmLt/Os2mMJVrQX0ua8HKyysTFXEfneXWCM4Ya9YFCZ7sNvCJFqQED4fPcFxnNbekTXni7OP98pfvLXIbHP2ibweOy8pZImgmJ8mrun5iXL/o9q7uiBFzSVxqZZ+O5vVigpKdP95lWBIm6vv5/oWCzk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=bg8ptw4z; arc=none smtp.client-ip=209.85.160.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="bg8ptw4z" Received: by mail-oa1-f54.google.com with SMTP id 586e51a60fabf-464fb5c1ea5so1001388fac.3 for ; Wed, 26 Aug 2026 13:20:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1787775616; x=1788380416; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=2ADL2kWhxZtGy4zWAARuC/jsldPk9uJ7GZCsaK5GLJ8=; b=bg8ptw4zdV307CDaKRqryOfIqZ27JF6bnNWT1D/BbOBJYxQKnq/oRLgZWZoazyY5rl USlRCLkyO1snPzGq2kjbKwVGJmX8kK0HAeWbcN4J/2P+04t0zgyE0tvJjy1jMVrN6/Ss qEb0WHkDlCx2/6C3Ish4/iC9z51clWvHVRNKOxaIZpkJJIyMxmUwOVZq5fy/yvLeU5ft Rp19Ryp5HVRwddGj+8YgMO4aOSL4YvN5GZGE04UVo4/g7Q6+vnCuuM/s4beMWgNYmPAd iur9wAM/0iacivIoyT3tKkKEbJpxSVVSSuPyGynvYyraQXp4zOemgw4vlZQIq15El9jF Xsog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787775616; x=1788380416; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=2ADL2kWhxZtGy4zWAARuC/jsldPk9uJ7GZCsaK5GLJ8=; b=MqH4rZIfdb98uq3IPn28r4RkiU7pWVyG4F+CSNUsDhqrdwK2JcVCJpciV5Rtedefwx MbVTVkJrDYOGvtzDYPWdNF4MkHn68hbQAs230+AEXJkkOW23FhqL6ktFJeWnM4csGkFy 8MTaMuKkq6VITzyCGkvc4P3mrwmyg7KkJB/B+rf1SESXKcZ16sdpOqtyP035b0wXZRYO EVNep6Yf4hkMhUn1tu4YSZwjm1PTk/ePAMZz/mDGCEUy9wRXdBchA4fwOYO0Tz+wgXAg qjAqy8W7lh2jaIJ+oaz3h3iqo7aB2dJe3LYWCrwNAyXYMqiUH5wNm2beDJ50626IMhjG fYrg== X-Gm-Message-State: AFuF++nDClKjzvt8uHDr+cxWhPQZXh2hhy9d0uG9gOGifEwkNdmU9AD3 71ERTaKZkAbvF275CSFoWFTTXxpcRh7+zzupl8MtNA/ag32f8ItNJOAvvy8J6XK908Y= X-Gm-Gg: AR+sD121/O9GlP1jxtGnP/+zNpGKIKxYH8xt76BKM4cHxlEuMyXt3Nnyx37+VTslzuy IActVQvXHp5rgReEgswMGh0FZPdzUm/+ehR6s+m50CIb9hVMlqcYBcYhKSaYcQJLjJcm0bUU3iq fCt1B/aPybYqY43in5xWFek/cuizhY092qRy+brcvYNQJhp740mg6yTl/yKGmwHlHJjDr6krGle 7BBa0gQyhtO7kefewhONwaeFWEcDneDsa8g8ffy1aavcxjE2jUf3bESv3dr8Pe8eT5h1RzlZnG+ TFSIbgaqhjMzsMOsmehzSRY2zdZGbFKdpbFdaCV71xllSz0zE96PajdDghAndOjg939J5zegJkz 3mAVO9uzB4gKR8/J+zvR7De1pszDPcWraEBemSsBv84K6iK3TA1mKuGpDsu19mgu3c9dbbx5sXR NwKWOE6J3yUq0LSIhMoe2ng3c6nb/qVpptRE+NFYasUFmnsea8rS8= X-Received: by 2002:a05:6820:220d:b0:6aa:e70f:301e with SMTP id 006d021491bc7-6b1a03af4aamr8729903eaf.8.1787775615783; Wed, 26 Aug 2026 13:20:15 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2e46::49c:30]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7f4c83d09d1sm2474928a34.14.2026.08.26.13.20.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2026 13:20:15 -0700 (PDT) From: Chris J Arges Date: Wed, 26 Aug 2026 15:20:07 -0500 Subject: [PATCH RFC net-next 1/3] ipv4: hash uncached routes by device Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260826-hash-bucket-route-lists-v1-1-fa9b9f30eb74@cloudflare.com> References: <20260826-hash-bucket-route-lists-v1-0-fa9b9f30eb74@cloudflare.com> In-Reply-To: <20260826-hash-bucket-route-lists-v1-0-fa9b9f30eb74@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1787775612; l=3229; i=carges@cloudflare.com; h=from:subject:message-id; bh=Ut/PjvIImnY64JnqnynvMuxcOBqH+zLzueSVJw4Qbq0=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QGKtMglFWb2eF4aZn13tPCiD5edLWKEp4C+U9eq3qkK/bBhhmXCdy3tyUWdIV1sw9yi0PuzZCl3 jMELPC0GRkwU= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M rt_flush_dev() currently walks every per-CPU uncached route list for each device being removed. This repeatedly examines unrelated routes and makes teardown increasingly expensive as the number of devices grows. Replace each per-CPU list with 64 hash buckets keyed by the route's netdevice. Keep the owning-list pointer in dst_entry so route removal remains unchanged, while device teardown only walks the matching bucket on each CPU. Hash collisions are filtered by the existing device comparison. Signed-off-by: Chris J Arges --- net/ipv4/route.c | 36 +++++++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 7 deletions(-) diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 604cc51dfd9b..3f9bc1ec72cc 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -1552,11 +1553,22 @@ struct uncached_list { struct list_head head; }; -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt_uncached_list); +#define RT_UNCACHED_HASH_BITS 6 +#define RT_UNCACHED_HASH_SIZE BIT(RT_UNCACHED_HASH_BITS) + +struct uncached_table { + struct uncached_list buckets[RT_UNCACHED_HASH_SIZE]; +}; + +static DEFINE_PER_CPU_ALIGNED(struct uncached_table, rt_uncached_table); void rt_add_uncached_list(struct rtable *rt) { - struct uncached_list *ul = raw_cpu_ptr(&rt_uncached_list); + struct uncached_table *table = raw_cpu_ptr(&rt_uncached_table); + struct uncached_list *ul; + + ul = &table->buckets[hash_ptr(dst_dev(&rt->dst), + RT_UNCACHED_HASH_BITS)]; rt->dst.rt_uncached_list = ul; @@ -1588,14 +1600,18 @@ void rt_flush_dev(struct net_device *dev) int cpu; for_each_possible_cpu(cpu) { - struct uncached_list *ul = &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + struct uncached_list *ul; + + table = per_cpu_ptr(&rt_uncached_table, cpu); + ul = &table->buckets[hash_ptr(dev, RT_UNCACHED_HASH_BITS)]; if (list_empty(&ul->head)) continue; spin_lock_bh(&ul->lock); list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { - if (rt->dst.dev != dev) + if (dst_dev(&rt->dst) != dev) continue; rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev); netdev_ref_replace(dev, blackhole_netdev, @@ -3771,10 +3787,16 @@ int __init ip_rt_init(void) ip_tstamps = idents_hash + (ip_idents_mask + 1) * sizeof(*ip_idents); for_each_possible_cpu(cpu) { - struct uncached_list *ul = &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + int bucket; + + table = per_cpu_ptr(&rt_uncached_table, cpu); + for (bucket = 0; bucket < RT_UNCACHED_HASH_SIZE; bucket++) { + struct uncached_list *ul = &table->buckets[bucket]; - INIT_LIST_HEAD(&ul->head); - spin_lock_init(&ul->lock); + INIT_LIST_HEAD(&ul->head); + spin_lock_init(&ul->lock); + } } #ifdef CONFIG_IP_ROUTE_CLASSID ip_rt_acct = __alloc_percpu(256 * sizeof(struct ip_rt_acct), __alignof__(struct ip_rt_acct)); -- 2.43.0