[PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Dai Ngo]

The NFS server should ask clients to voluntarily return unused
delegations when the number of granted delegations reaches the
max_delegations. This is so that the server can continue to
grant delegations for new requests.

Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
---
 fs/nfsd/nfs4state.c | 3 +++
 1 file changed, 3 insertions(+)

v2: move declaration of deleg_reaper() up to other forward
    declarations in the file.

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index fdc95bfbfbb6..961000261b3e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -87,6 +87,7 @@ static void nfs4_free_ol_stateid(struct nfs4_stid *stid);
 void nfsd4_end_grace(struct nfsd_net *nn);
 static void _free_cpntf_state_locked(struct nfsd_net *nn, struct nfs4_cpntf_state *cps);
 static void nfsd4_file_hash_remove(struct nfs4_file *fi);
+static void deleg_reaper(struct nfsd_net *nn);
 
 /* Locking: */
 
@@ -6550,6 +6551,8 @@ nfs4_laundromat(struct nfsd_net *nn)
 	/* service the server-to-server copy delayed unmount list */
 	nfsd4_ssc_expire_umount(nn);
 #endif
+	if (atomic_long_read(&num_delegations) >= max_delegations)
+		deleg_reaper(nn);
 out:
 	return max_t(time64_t, lt.new_timeo, NFSD_LAUNDROMAT_MINTIMEOUT);
 }
-- 
2.39.3

回复: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Hanxiao Chen (Fujitsu)]

> -----邮件原件-----
> 发件人: Dai Ngo <dai.ngo@oracle.com>
> 发送时间: 2024年3月4日 6:23
> 收件人: chuck.lever@oracle.com; jlayton@kernel.org
> 抄送: linux-nfs@vger.kernel.org
> 主题: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number
> of delegations reaches its limit
> 
> The NFS server should ask clients to voluntarily return unused
> delegations when the number of granted delegations reaches the
> max_delegations. This is so that the server can continue to
> grant delegations for new requests.
> 
> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> ---
>
Tested-by: Chen Hanxiao <chenhx.fnst@fujitsu.com>

Regards,
- Chen

Re: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Jeff Layton]

On Sun, 2024-03-03 at 14:23 -0800, Dai Ngo wrote:
> The NFS server should ask clients to voluntarily return unused
> delegations when the number of granted delegations reaches the
> max_delegations. This is so that the server can continue to
> grant delegations for new requests.
> 
> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> ---
>  fs/nfsd/nfs4state.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> v2: move declaration of deleg_reaper() up to other forward
>     declarations in the file.
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index fdc95bfbfbb6..961000261b3e 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -87,6 +87,7 @@ static void nfs4_free_ol_stateid(struct nfs4_stid *stid);
>  void nfsd4_end_grace(struct nfsd_net *nn);
>  static void _free_cpntf_state_locked(struct nfsd_net *nn, struct nfs4_cpntf_state *cps);
>  static void nfsd4_file_hash_remove(struct nfs4_file *fi);
> +static void deleg_reaper(struct nfsd_net *nn);
>  
>  /* Locking: */
>  
> @@ -6550,6 +6551,8 @@ nfs4_laundromat(struct nfsd_net *nn)
>  	/* service the server-to-server copy delayed unmount list */
>  	nfsd4_ssc_expire_umount(nn);
>  #endif
> +	if (atomic_long_read(&num_delegations) >= max_delegations)
> +		deleg_reaper(nn);
>  out:
>  	return max_t(time64_t, lt.new_timeo, NFSD_LAUNDROMAT_MINTIMEOUT);
>  }

Seems like a reasonable thing to do:

Reviewed-by: Jeff Layton <jlayton@kernel.org>

Re: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Chuck Lever]

On Sun, Mar 03, 2024 at 05:23:02PM -0500, Dai Ngo wrote:
> The NFS server should ask clients to voluntarily return unused
> delegations when the number of granted delegations reaches the
> max_delegations. This is so that the server can continue to
> grant delegations for new requests.
> 
> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> ---
>  fs/nfsd/nfs4state.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> v2: move declaration of deleg_reaper() up to other forward
>     declarations in the file.
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index fdc95bfbfbb6..961000261b3e 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -87,6 +87,7 @@ static void nfs4_free_ol_stateid(struct nfs4_stid *stid);
>  void nfsd4_end_grace(struct nfsd_net *nn);
>  static void _free_cpntf_state_locked(struct nfsd_net *nn, struct nfs4_cpntf_state *cps);
>  static void nfsd4_file_hash_remove(struct nfs4_file *fi);
> +static void deleg_reaper(struct nfsd_net *nn);
>  
>  /* Locking: */
>  
> @@ -6550,6 +6551,8 @@ nfs4_laundromat(struct nfsd_net *nn)
>  	/* service the server-to-server copy delayed unmount list */
>  	nfsd4_ssc_expire_umount(nn);
>  #endif
> +	if (atomic_long_read(&num_delegations) >= max_delegations)
> +		deleg_reaper(nn);
>  out:
>  	return max_t(time64_t, lt.new_timeo, NFSD_LAUNDROMAT_MINTIMEOUT);
>  }
> -- 
> 2.39.3
> 

Applied to nfsd-next. Thanks, Dai!

-- 
Chuck Lever

Re: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Cedric Blancher]

On Sun, 3 Mar 2024 at 23:23, Dai Ngo <dai.ngo@oracle.com> wrote:
>
> The NFS server should ask clients to voluntarily return unused
> delegations when the number of granted delegations reaches the
> max_delegations. This is so that the server can continue to
> grant delegations for new requests.

What is this limit max_delegations? Where is it set, and where can an
admin alter it at runtime?

Are you aware that for example the msnfs41client Windows NFSv4.1
driver easily uses a few hundred delegations, as required by the
highly multithreaded nature (i.e. every Win32 syscall is async) of the
Windows kernel?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: [PATCH v2 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reaches its limit

[Dai Ngo]

On 3/5/24 12:49 AM, Cedric Blancher wrote:
> On Sun, 3 Mar 2024 at 23:23, Dai Ngo <dai.ngo@oracle.com> wrote:
>> The NFS server should ask clients to voluntarily return unused
>> delegations when the number of granted delegations reaches the
>> max_delegations. This is so that the server can continue to
>> grant delegations for new requests.
> What is this limit max_delegations?

4 delegations per megabyte of available RAM.

>   Where is it set,

when nfsd starts up.

>   and where can an
> admin alter it at runtime?

There is no admin tool to change this setting. But if you want to
play with it then you can poke the memory and change its value,
The code uses max_delegations dynamically.

-Dai

>
> Are you aware that for example the msnfs41client Windows NFSv4.1
> driver easily uses a few hundred delegations, as required by the
> highly multithreaded nature (i.e. every Win32 syscall is async) of the
> Windows kernel?
>
> Ced