[PATCH v9 00/19] nfs/nfsd: add support for localio

[Mike Snitzer <snitzer@kernel.org>]

Hi,

I'd prefer to see these changes land upstream for 6.11 if possible.
They are adequately Kconfig'd to certainly pose no risk if disabled.
And even if localio enabled it has proven to work well with increased
testing.

Worked with Kent Overstreet to enable testing integration with ktest
running xfstests, the dashboard is here:
https://evilpiepirate.org/~testdashboard/ci?branch=snitm-nfs
(it is running way more xfstests tests than is usual for nfs, would be
good to reconcile that with the listing provided here:
https://wiki.linux-nfs.org/wiki/index.php/Xfstests )

Changes since v8:
- Fixed xfstests generic/355 (clear suid after write) as a side-effect
  of dropping the "nfs/localio: use dedicated workqueues for
  filesystem read and write" patch (XFS is looking at the security
  context of the task... which is really odd!)
- Refactored and fixed nfs_local_vfs_getattr() to support NFS v4 as
  requested by Neil.
- Fixed potential for localio file opens to prevent nfsd from shutting
  down (as caught by Jeff's helpful review) by switching to using
  percpu_ref_tryget_live (and renamed nfsd_serv_get to
  nfsd_serv_try_get).
- Removed all dprintk() from fs/nfsd/localio.c
- Removed one dprintk() from fs/nfs/localio.c, left others because the
  nfs client maintainers don't seem so against them (they do require
  explicit enablement after all).

TODO:
- Hopefully get a favorable response to this patch from XFS engineers:
  https://marc.info/?l=linux-xfs&m=171959152810706&w=2
  (otherwise, will need to revisit using dedicated workqueue patch)

All review and comments are welcome!

Thanks,
Mike

My git tree is here:
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/

This v9 is both branch nfs-localio-for-6.11 (always tracks latest)
and nfs-localio-for-6.11.v9

Mike Snitzer (11):
  nfs_common: add NFS LOCALIO auxiliary protocol enablement
  nfs/localio: fix nfs_localio_vfs_getattr() to properly support v4
  nfs: factor out {encode,decode}_opaque_fixed to nfs_xdr.h
  SUNRPC: remove call_allocate() BUG_ON if p_arglen=0 to allow RPC with void arg
  nfs: implement client support for NFS_LOCALIO_PROGRAM
  nfsd: add "localio" support
  nfsd/localio: manage netns reference in nfsd_open_local_fh
  nfsd: use percpu_ref to interlock nfsd_destroy_serv and nfsd_open_local_fh
  nfsd: add Kconfig options to allow localio to be enabled
  nfsd: implement server support for NFS_LOCALIO_PROGRAM
  nfs: add Documentation/filesystems/nfs/localio.rst

NeilBrown (1):
  SUNRPC: replace program list with program array

Trond Myklebust (2):
  nfs: enable localio for non-pNFS I/O
  pnfs/flexfiles: Enable localio for flexfiles I/O

Weston Andros Adamson (5):
  nfs: pass nfs_client to nfs_initiate_pgio
  nfs: pass descriptor thru nfs_initiate_pgio path
  nfs: pass struct file to nfs_init_pgio and nfs_init_commit
  sunrpc: add rpcauth_map_to_svc_cred_local
  nfs: add "localio" support

 Documentation/filesystems/nfs/localio.rst | 135 ++++
 fs/Kconfig                                |   3 +
 fs/nfs/Kconfig                            |  14 +
 fs/nfs/Makefile                           |   1 +
 fs/nfs/blocklayout/blocklayout.c          |   6 +-
 fs/nfs/client.c                           |  15 +-
 fs/nfs/filelayout/filelayout.c            |  16 +-
 fs/nfs/flexfilelayout/flexfilelayout.c    | 131 +++-
 fs/nfs/flexfilelayout/flexfilelayout.h    |   2 +
 fs/nfs/flexfilelayout/flexfilelayoutdev.c |   6 +
 fs/nfs/inode.c                            |   4 +
 fs/nfs/internal.h                         |  60 +-
 fs/nfs/localio.c                          | 827 ++++++++++++++++++++++
 fs/nfs/nfs4xdr.c                          |  13 -
 fs/nfs/nfstrace.h                         |  61 ++
 fs/nfs/pagelist.c                         |  32 +-
 fs/nfs/pnfs.c                             |  24 +-
 fs/nfs/pnfs.h                             |   6 +-
 fs/nfs/pnfs_nfs.c                         |   2 +-
 fs/nfs/write.c                            |  13 +-
 fs/nfs_common/Makefile                    |   3 +
 fs/nfs_common/nfslocalio.c                |  74 ++
 fs/nfsd/Kconfig                           |  14 +
 fs/nfsd/Makefile                          |   1 +
 fs/nfsd/filecache.c                       |   2 +-
 fs/nfsd/localio.c                         | 319 +++++++++
 fs/nfsd/netns.h                           |  12 +-
 fs/nfsd/nfsctl.c                          |   2 +-
 fs/nfsd/nfsd.h                            |   2 +-
 fs/nfsd/nfssvc.c                          | 116 ++-
 fs/nfsd/trace.h                           |   3 +-
 fs/nfsd/vfs.h                             |   9 +
 include/linux/nfs.h                       |   9 +
 include/linux/nfs_fs.h                    |   2 +
 include/linux/nfs_fs_sb.h                 |  10 +
 include/linux/nfs_xdr.h                   |  20 +-
 include/linux/nfslocalio.h                |  41 ++
 include/linux/sunrpc/auth.h               |   4 +
 include/linux/sunrpc/svc.h                |   7 +-
 net/sunrpc/auth.c                         |  15 +
 net/sunrpc/clnt.c                         |   1 -
 net/sunrpc/svc.c                          |  68 +-
 net/sunrpc/svc_xprt.c                     |   2 +-
 net/sunrpc/svcauth_unix.c                 |   3 +-
 44 files changed, 1975 insertions(+), 135 deletions(-)
 create mode 100644 Documentation/filesystems/nfs/localio.rst
 create mode 100644 fs/nfs/localio.c
 create mode 100644 fs/nfs_common/nfslocalio.c
 create mode 100644 fs/nfsd/localio.c
 create mode 100644 include/linux/nfslocalio.h

-- 
2.44.0