From: NeilBrown <neilb@suse.de>
To: Chuck Lever <chuck.lever@oracle.com>, Jeff Layton <jlayton@kernel.org>
Cc: linux-nfs@vger.kernel.org,
Olga Kornievskaia <okorniev@redhat.com>,
Dai Ngo <Dai.Ngo@oracle.com>, Tom Talpey <tom@talpey.com>
Subject: [PATCH 2/4] nfsd: allocate new session-based DRC slots on demand.
Date: Wed, 13 Nov 2024 16:38:35 +1100 [thread overview]
Message-ID: <20241113055345.494856-3-neilb@suse.de> (raw)
In-Reply-To: <20241113055345.494856-1-neilb@suse.de>
If a client ever uses the highest available slot for a given session,
attempt to allocate another slot so there is room for the client to use
more slots if wanted. GFP_NOWAIT is used so if there is not plenty of
free memory, failure is expected - which is what we want. It also
allows the allocation while holding a spinlock.
We would expect to stablise with one more slot available than the client
actually uses.
Now that we grow the slot table on demand we can start with a smaller
allocation. Define NFSD_MAX_UNUSED_SLOTS and allocate at most that many
when session is created.
Signed-off-by: NeilBrown <neilb@suse.de>
---
fs/nfsd/nfs4state.c | 22 ++++++++++++++++------
fs/nfsd/state.h | 2 ++
2 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 2dcba0c83c10..683bb908039b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1953,7 +1953,7 @@ static struct nfsd4_session *alloc_session(struct nfsd4_channel_attrs *fattrs,
if (!new->se_slots[0])
goto out_free;
- for (i = 1; i < numslots; i++) {
+ for (i = 1; i < numslots && i < NFSD_MAX_UNUSED_SLOTS; i++) {
new->se_slots[i] = kzalloc(slotsize, GFP_KERNEL | __GFP_NORETRY);
if (!new->se_slots[i])
break;
@@ -4187,11 +4187,6 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
slot = session->se_slots[seq->slotid];
dprintk("%s: slotid %d\n", __func__, seq->slotid);
- /* We do not negotiate the number of slots yet, so set the
- * maxslots to the session maxreqs which is used to encode
- * sr_highest_slotid and the sr_target_slot id to maxslots */
- seq->maxslots = session->se_fchannel.maxreqs;
-
trace_nfsd_slot_seqid_sequence(clp, seq, slot);
status = check_slot_seqid(seq->seqid, slot->sl_seqid,
slot->sl_flags & NFSD4_SLOT_INUSE);
@@ -4241,6 +4236,21 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
cstate->session = session;
cstate->clp = clp;
+ /*
+ * If the client ever uses the highest available slot,
+ * gently try to allocate another one.
+ */
+ if (seq->slotid == session->se_fchannel.maxreqs - 1 &&
+ session->se_fchannel.maxreqs < NFSD_MAX_SLOTS_PER_SESSION) {
+ int s = session->se_fchannel.maxreqs;
+
+ session->se_slots[s] = kzalloc(slot_bytes(&session->se_fchannel),
+ GFP_NOWAIT);
+ if (session->se_slots[s])
+ session->se_fchannel.maxreqs += 1;
+ }
+ seq->maxslots = session->se_fchannel.maxreqs;
+
out:
switch (clp->cl_cb_state) {
case NFSD4_CB_DOWN:
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index c052e9eea81b..012b68a0bafa 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -215,6 +215,8 @@ static inline struct nfs4_delegation *delegstateid(struct nfs4_stid *s)
/* Maximum number of slots per session. 160 is useful for long haul TCP */
#define NFSD_MAX_SLOTS_PER_SESSION 160
+/* Maximum number of slots per session that are allocated by unused */
+#define NFSD_MAX_UNUSED_SLOTS 6
/* Maximum session per slot cache size */
#define NFSD_SLOT_CACHE_SIZE 2048
/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
--
2.47.0
next prev parent reply other threads:[~2024-11-13 5:54 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-13 5:38 [PATCH 0/4 RFC] nfsd: allocate/free session-based DRC slots on demand NeilBrown
2024-11-13 5:38 ` [PATCH 1/4] nfsd: remove artificial limits on the session-based DRC NeilBrown
2024-11-13 16:48 ` Jeff Layton
2024-11-15 4:01 ` NeilBrown
2024-11-15 14:07 ` Jeff Layton
2024-11-13 5:38 ` NeilBrown [this message]
2024-11-13 5:38 ` [PATCH 3/4] nfsd: free unused session-DRC slots NeilBrown
2024-11-13 16:52 ` Jeff Layton
2024-11-13 5:38 ` [PATCH 4/4] nfsd: add shrinker to reduce number of slots allocated per session NeilBrown
2024-11-13 11:23 ` [PATCH 0/4 RFC] nfsd: allocate/free session-based DRC slots on demand Daire Byrne
2024-11-15 4:56 ` NeilBrown
2024-11-15 14:18 ` Chuck Lever
2024-11-18 15:43 ` Daire Byrne
2024-11-13 14:48 ` Chuck Lever III
2024-11-15 9:03 ` NeilBrown
2024-11-15 14:27 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241113055345.494856-3-neilb@suse.de \
--to=neilb@suse.de \
--cc=Dai.Ngo@oracle.com \
--cc=chuck.lever@oracle.com \
--cc=jlayton@kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=okorniev@redhat.com \
--cc=tom@talpey.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox