From: Bugspray Bot <bugbot@kernel.org>
To: trondmy@kernel.org, cel@kernel.org, jlayton@kernel.org,
linux-nfs@vger.kernel.org, anna@kernel.org
Subject: NFSD callback operations block everything when clients are unresponsive
Date: Fri, 13 Sep 2024 20:05:07 +0000 [thread overview]
Message-ID: <20240913-b218735c0-12465843fe68@bugzilla.kernel.org> (raw)
cel writes via Kernel.org Bugzilla:
Several reporters note that after commit c1ccfcf1a9bf ("NFSD: Reschedule CB operations when backchannel rpc_clnt is shut down"), NFSD's callback work queue is blocked when one of the clients is unresponsive.
We know that NFSD's callback_wq is single-threaded (ordered), and that there is only one WQ for all of the NFS server's clients.
What blocks callback operations is the retry loop in nfsd4_run_cb_work(). It was added to ensure that CB_OFFLOAD operations are delivered reliably, but it causes head-of-queue blocking when any NFS client becomes unresponsive when a callback operation is pending.
We've partially addressed this by giving each lease its own callback_wq.
However it's clear that retrying callback operations from within the callback WQ is going to be problematic to some extent. The solution is to hoist the responsibility for retrying higher up into the individual implementations of the callback operations (CB_RECALL, CB_NOTIFY_LOCK, CB_OFFLOAD, and so on), since each of these operations has their own needs in terms of recourse when a callback operation cannot be sent.
View: https://bugzilla.kernel.org/show_bug.cgi?id=218735#c0
You can reply to this message to join the discussion.
--
Deet-doot-dot, I am a bot.
Kernel.org Bugzilla (bugspray 0.1-dev)
next reply other threads:[~2024-09-13 20:05 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-13 20:05 Bugspray Bot [this message]
2024-09-13 20:05 ` NFSD callback operations block everything when clients are unresponsive Bugspray Bot
2024-09-13 20:05 ` Bugspray Bot
2024-09-13 20:05 ` Bugspray Bot
2024-09-13 20:05 ` Bugspray Bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240913-b218735c0-12465843fe68@bugzilla.kernel.org \
--to=bugbot@kernel.org \
--cc=anna@kernel.org \
--cc=cel@kernel.org \
--cc=jlayton@kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox