Re: [PATCH 4/4] NFSD: Send a meaningful CB_RECALL_ANY keep count

["Chuck Lever" <cel@kernel.org>]

On Wed, Aug 12, 2026, at 2:17 AM, Cedric Blancher wrote:
> On Tue, 11 Aug 2026 at 21:57, Chuck Lever <cel@kernel.org> wrote:
>>
>> deleg_reaper() sets craa_objects_to_keep to zero on every
>> CB_RECALL_ANY. Per RFC 8881 Section 20.6.3, that asks the client to
>> retain no read or write delegation at all, whether or not the
>> delegation backs an open file.
>>
>> The field names a count the client may keep. The client picks which
>> objects to return, because the server cannot read lack of recent
>> use as lack of usefulness. Zero leaves nothing to choose among. A
>> client that complies returns the delegations backing its open files
>> and reopens each one with CLAIM_DELEGATE_CUR, so NFSD trades a
>> delegation for an open stateid and recovers nothing. A client that
>> reads the zero as "unspecified" does nothing instead, and
>> nfsd4_cb_recall_any_done() inspects only the reply status, so NFSD
>> cannot tell the two apart.
>>
>> Derive the keep count from cl_deleg_count and ask each client for a
>> single delegation. A larger request reaches delegations an
>> application still has open, and both callers re-arm while their
>> condition lasts. Skip a client holding one delegation rather than
>> send the zero again. That gate subsumes the list_empty() test above
>> it, and it belongs above the NFSD4_CALLBACK_RUNNING test_and_set. A
>> continue below that point latches the bit with no callback in
>> flight to clear it.
>>
>> The Linux client ignores craa_objs_to_keep and returns unused
>> delegations from the type mask alone, so the count changes nothing
>> for it. The gate does. The reaper goes quiet for a client walked
>> down to one delegation.
>>
>> Fixes: 44df6f439a17 ("NFSD: add delegation reaper to react to low memory condition")
>> Signed-off-by: Chuck Lever <cel@kernel.org>
>
> ms-nfs41-client hit that bug when implementing CB_RECALL_ANY
> (https://github.com/kofemann/ms-nfs41-client/commit/3abe73c8ab0d924450a1c1c480f0b14f963f9531)
> with Linux 7.0 nfsd.
> What should existing NFSv4.1 clients do if they encounter a
> objects_to_keep value of 0? Right now it recalls ALL delegations,
> which basically is a "reset" of all delegations.

After studying this issue for a few days... and sleeping on it a
bit...

RFC 8881 Section 20.6.3 does not normatively mandate any particular
client response to CB_RECALL_ANY other than returning NFS4ERR_INVAL
when the craa_type_mask bitmask is invalid. The client-facing verbs
in that section are "is to return", "chooses", and "it is the job
of". All descriptive language, no BCP14 keywords.

Thus, according to spec, there are no interoperability consequences
if a client ignores the value of the craa_objects_to_keep argument.
The spec gives client implementers considerable flexibility here.

A server has no visibility of which delegations are actively in use
on clients, since the point of delegation is to reduce client-to-
server traffic. That's why the client gets to choose which to
return.

A good quality client implementation, IMHO, should choose idle or
currently unused delegations, but protect state that is still in
active use. To return a DELEG stateid that is still in use, the
client would need to first ensure it has an OPEN stateid to
continue using. That would result in no real change in the server
memory footprint for that file, so there is no benefit.

AFAICS a craa_objects_to_keep value of zero is not a bug; it is
in fact a valid value for that argument. But NFSD shouldn't ask
clients to toss out their entire working set at the first sign of
memory pressure. The new behavior is a little more graceful.

(So I realize the patches don't actually say this much, but my
thinking is still evolving).

-- 
Chuck Lever