From: Simon Horman <horms@kernel.org>
To: Chuang Wang <nashuiliang@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Stanislav Fomichev <sdf.kernel@gmail.com>,
Kuniyuki Iwashima <kuniyu@google.com>,
Samiullah Khawaja <skhawaja@google.com>,
Hangbin Liu <liuhangbin@gmail.com>,
Neal Cardwell <ncardwell@google.com>,
Shakeel Butt <shakeel.butt@linux.dev>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH net-next v6] net: reduce RFS/ARFS flow updates by checking LLC affinity
Date: Thu, 4 Jun 2026 20:10:05 +0100 [thread overview]
Message-ID: <20260604191005.GG3920875@horms.kernel.org> (raw)
In-Reply-To: <20260602115323.3502-1-nashuiliang@gmail.com>
On Tue, Jun 02, 2026 at 07:53:10PM +0800, Chuang Wang wrote:
> The current implementation of rps_record_sock_flow() updates the flow
> table every time a socket is processed on a different CPU. In high-load
> scenarios, especially with Accelerated RFS (ARFS), this triggers
> frequent flow steering updates via ndo_rx_flow_steer.
>
> For drivers like mlx5 that implement hardware flow steering, these
> constant updates lead to significant contention on internal driver locks
> (e.g., arfs_lock). This contention often becomes a performance
> bottleneck that outweighs the steering benefits.
>
> This patch introduces a cache-aware update strategy: the flow record is
> only updated if the flow migrates across Last Level Cache (LLC)
> boundaries. This minimizes expensive hardware reconfigurations while
> preserving cache locality for the application. A new sysctl,
> net.core.rps_feat_llc_affinity, is added to toggle this feature.
>
> Additionally, export sock_rps_record_flow_hash() and
> sock_rps_record_flow(). This resolves a symbol visibility compilation
> error triggered by 'tun' using sock_rps_record_flow_hash() in
> tun_flow_update() when CONFIG_TUN is built as a module. The same logic
> is applied to SCTP, allowing it to use sock_rps_record_flow() safely
> when built as a module.
>
> Performance Test Results:
> The patch was tested in a K8s environment (AMD CPU 128*2, 16-core Pod
> with CPU pinning, mlx5 NIC) using brpc[1] echo_server and rpc_press.
>
> rpc_press Commands:
>
> for i in {1..8}; do
> ./rpc_press -proto=./echo.proto -method=example.EchoService.Echo
> -server=<IP>:8000 -input='{"message":"hello"}'
> -qps=0 -thread_num=512 -connection_type=pooled &
> done
>
> Monitor mlx5e_rx_flow_steer frequency:
>
> /usr/share/bcc/tools/funccount -i 1 mlx5e_rx_flow_steer
>
> Frequency of mlx5e_rx_flow_steer (via funccount[2]):
>
> Before: ~335,000 counts/sec
> After: ~23,000 counts/sec (reduced by ~93%)
>
> System Metrics (after enabling rps_feat_llc_affinity):
>
> CPU Utilization: 38% -> 32%
> CPU PSI (Pressure Stall Information): 20% -> 10%
>
> These results demonstrate that filtering updates by LLC affinity
> significantly reduces driver lock contention and improves overall
> CPU efficiency under heavy network load.
>
> [1] https://github.com/apache/brpc/
> [2] https://github.com/iovisor/bcc/blob/master/tools/funccount.py
>
> Signed-off-by: Chuang Wang <nashuiliang@gmail.com>
Hi,
There is AI-generated review of this patch-set available on both
https://sashiko.dev and https://netdev-ai.bots.linux.dev/sashiko/
I would appreciate it if you could look over that with a view
to addressing any issues that directly effect this patch-set.
prev parent reply other threads:[~2026-06-04 19:10 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-02 11:53 [PATCH net-next v6] net: reduce RFS/ARFS flow updates by checking LLC affinity Chuang Wang
2026-06-04 19:10 ` Simon Horman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260604191005.GG3920875@horms.kernel.org \
--to=horms@kernel.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=liuhangbin@gmail.com \
--cc=nashuiliang@gmail.com \
--cc=ncardwell@google.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sdf.kernel@gmail.com \
--cc=shakeel.butt@linux.dev \
--cc=skhawaja@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox