From: Tejun Heo <tj@kernel.org>
To: Chuck Lever III <chuck.lever@oracle.com>
Cc: open list <linux-kernel@vger.kernel.org>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: contention on pwq->pool->lock under heavy NFS workload
Date: Thu, 22 Jun 2023 09:23:30 -1000 [thread overview]
Message-ID: <ZJSfsvFtC_d265M1@slm.duckdns.org> (raw)
In-Reply-To: <CF5AF0E6-5213-489D-87CD-8E8325A6560F@oracle.com>
Hello,
On Thu, Jun 22, 2023 at 03:45:18PM +0000, Chuck Lever III wrote:
> The good news:
>
> On stock 6.4-rc7:
>
> fio 8k [r=108k,w=46.9k IOPS]
>
> On the affinity-scopes-v2 branch (with no other tuning):
>
> fio 8k [r=130k,w=55.9k IOPS]
Ah, okay, that's probably coming from per-cpu pwq. Didn't expect that to
make that much difference but that's nice.
> The bad news:
>
> pool->lock is still the hottest lock on the system during the test.
>
> I'll try some of the alternate scope settings this afternoon.
Yeah, in your system, there's still gonna be one pool shared across all
CPUs. SMT or CPU may behave better but it might make sense to add a way to
further segment the scope so that e.g. one can split a cache domain N-ways.
Thanks.
--
tejun
next prev parent reply other threads:[~2023-06-22 19:23 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-21 15:26 contention on pwq->pool->lock under heavy NFS workload Chuck Lever III
2023-06-21 21:28 ` Tejun Heo
2023-06-22 14:38 ` Chuck Lever III
2023-06-22 15:45 ` Chuck Lever III
2023-06-22 19:23 ` Tejun Heo [this message]
2023-06-22 19:39 ` Chuck Lever III
2023-06-23 14:37 ` Chuck Lever III
2023-06-24 1:44 ` Tejun Heo
2023-06-25 16:01 ` Chuck Lever III
2023-06-26 20:21 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZJSfsvFtC_d265M1@slm.duckdns.org \
--to=tj@kernel.org \
--cc=chuck.lever@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox