[PATCH v2 0/3] pNFS/flexfiles: mirror instance error handling

[Benjamin Coddington <ben.coddington@hammerspace.com>]

When a flexfiles data server fails, the client marks the layout for return
and cancels the layout segment's in-flight I/O.  ff_layout_cancel_io()
cancels I/O to every mirror instance rather than just the failed one, and
cancelling a request that has already been transmitted does not stop the
data server from executing it.  Those writes may complete after the client
has sent its LAYOUTRETURN, and the metadata server will see writes to a file
for which no write layout is outstanding.

Patch 1 reports the operation the client was preparing to send when a
data server connection fails, instead of OP_ILLEGAL.

Patches 2 and 3 let a layout driver cancel I/O to a single device, and
pass the failed device down from the flexfiles error paths, so that I/O
to the remaining mirrors drains before the LAYOUTRETURN is sent.

Layout recalls that set clora_changed, bulk returns and revocations are
unchanged: they continue to cancel I/O to every device.

This version applies on top of the flexfiles device-notification and
caching series [1], which rewrites the data-server setup and
I/O-cancellation paths these patches touch.

v1 posting:
  https://lore.kernel.org/linux-nfs/cover.1786636529.git.bcodding@hammerspace.com/

Changes since v1:

 - Rebase onto [1].  No functional change from v1; every conflict was
   in context that series rewrites:

   - nfs4_ff_layout_prepare_ds() now takes the device node the caller
     has already resolved and referenced, so patch 1 replaces the
     trailing fail_return argument of a five-argument signature, and
     three of the five call sites wrap to stay inside 80 columns.

   - ff_layout_cancel_io() now walks mirror->dss[].mirror_ds under RCU
     and takes a reference on the node, so patch 2 applies its
     device-ID filter ahead of the RCU dereference rather than in
     place of a plain pointer read.  The filter still compares the raw
     device ID from the layout, which is immutable across the in-place
     device re-resolve that series adds.

   - ff_layout_resend_pnfs_read() now yields a device node rather than
     a data server and puts it on the success path, so patch 2's NULL
     device-ID argument moves into the braced error arm.

[1] https://lore.kernel.org/linux-nfs/cover.1788530385.git.bcodding@hammerspace.com/

Benjamin Coddington (3):
  NFSv4/flexfiles: report the intended opnum when DS connection setup
    fails
  pNFS: allow layout drivers to cancel I/O to a single device
  NFSv4/flexfiles: only cancel I/O to a failed mirror instance

 fs/nfs/blocklayout/blocklayout.c          |  6 +++--
 fs/nfs/callback_proc.c                    |  2 +-
 fs/nfs/filelayout/filelayout.c            |  4 +--
 fs/nfs/flexfilelayout/flexfilelayout.c    | 32 +++++++++++++++--------
 fs/nfs/flexfilelayout/flexfilelayout.h    |  2 +-
 fs/nfs/flexfilelayout/flexfilelayoutdev.c | 17 +++++++-----
 fs/nfs/pnfs.c                             | 32 ++++++++++++++---------
 fs/nfs/pnfs.h                             | 14 ++++++----
 8 files changed, 67 insertions(+), 42 deletions(-)

-- 
2.53.0