Confused by pnfs LAYOUTRETURN - seeking clarity.

[NeilBrown <neilb@suse.com>]

[-- Attachment #1: Type: text/plain, Size: 2334 bytes --]


I've been trying to understand how LAYOUTRETURN is used in pNFS,
primarily because our SLE12-SP1 kernel (based on 3.12) appears
to have a very different opinion than some Netapp filers.

My reading of RFC-5661 suggests that the client needs to call
LAYOUTRETURN for every layout that it received from the server.  A
single LAYOUTRETURN can cover a whole file or a whole filesystem, so it
doesn't need to be 1-for-1, but there is no implicit return.

However RFC-5663 contains the text

   A LAYOUTRETURN operation represents an explicit release of resources
   by the client, usually done for the purpose of avoiding unnecessary
   CB_LAYOUTRECALL operations in the future.

This seems to imply that LAYOUTRETURN is only an optimisation.  If you
don't want to avoid CB_LAYOUTRECALL, there is not much call for
LAYOUTRETURN.  It seems to suggest (without explicitly saying) that the
CB_LAYOUTRECALL will effect the return of a layout without the client
explicitly sending LAYOUTRETURN in response.  RFC-5661 says LAYOUTRETURN
does need to be sent in response.

The code in 3.12 doesn't send LAYOUTRETURN in response to
CB_LAYOUTRECALL, nor does it send LAYOUTRETURN when it closes a file
marked as "return layouts on close".  The one place I have seen evidence
of it returning layouts is when a file is unlinked, though I think there
are others (chmod, IO error).

The current upstream code seems to call LAYOUTRETURN more correctly, but
it is hard to be sure because I couldn't find a commit which acknowledged
the specific problem and corrected it - just commits that claim to be
making improvements and avoiding races and things like that.

Questions:
 - Am I correct that all layouts need to be explicitly returned by the
   client, and so the text from RFC-5663 is misleading?

 - If so, what is the earliest kernel that is believed to correctly
   return layouts in response to CB_LAYOUTRECALL, or a 'roc' file being
   closed?


I was advised that Netapp are considering a change (netapp issue
955835):
  An enhancement will be added in future versions of Ontap to clear out
  the corresponding layout states after a file has been closed in the
  event the client does not return them.

This sounds like a mistake, unless "clear out" means "send
CB_LAYOUTRECALL for".  Should we advice Netapp against this?

Thanks,
NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]