[PATCH v5 00/51] Add Flexfile Layout Module

[Tom Haynes <thomas.haynes@primarydata.com>]

This patchset introduces the Flexfile Layout Module for the
client.

It will correspond to draft 5
(http://tools.ietf.org/id/draft-ietf-nfsv4-flex-files-05.txt)
of the Parallel NFS (pNFS) Flexible File Layout
(https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/).

Note: The major change from version 4 was the XDR change
from a cred to a stringified user and group principal
in the layout.

This version has been rebased to v3.19-rc5 and is ready
for inclusion in 3.20.

The [flexfiles] branch of

git://git.linux-nfs.org/projects/loghyr/linux-nfs.git

also has this code.

Peng Tao (36):
  nfs41: pull data server cache from file layout to generic pnfs
  nfs41: pull decode_ds_addr from file layout to generic pnfs
  nfs41: pull nfs4_ds_connect from file layout to generic pnfs
  nfs41: allow LD to choose DS connection auth flavor
  nfs41: move file layout macros to generic pnfs
  nfsv3: introduce nfs3_set_ds_client
  nfs41: allow LD to choose DS connection version/minor_version
  nfs41: create NFSv3 DS connection if specified
  nfs: allow different protocol in nfs_initiate_commit
  nfs4: pass slot table to nfs40_setup_sequence
  nfs4: export nfs4_sequence_done
  nfs: allow to specify cred in nfs_initiate_pgio
  nfs: set hostname when creating nfsv3 ds connection
  nfs/flexclient: export pnfs_layoutcommit_inode
  nfs41: close a small race window when adding new layout to global list
  nfs41: serialize first layoutget of a file
  nfs: save server READ/WRITE/COMMIT status
  nfs41: pass iomode through layoutreturn args
  nfs41: make a helper function to send layoutreturn
  nfs41: add a helper to mark layout for return
  nfs41: don't use a layout if it is marked for returning
  nfs41: send layoutreturn in last put_lseg
  nfs41: clear NFS_LAYOUT_RETURN if layoutreturn is sent or failed to
    send
  nfs/filelayout: use pnfs_error_mark_layout_for_return
  nfs41: add a debug warning if we destroy an unempty layout
  nfs: only reset desc->pg_mirror_idx when mirroring is supported
  nfs: add nfs_pgio_current_mirror helper
  pnfs: allow LD to ask to resend read through pnfs
  nfs41: add range to layoutreturn args
  nfs41: allow async version layoutreturn
  nfs41: introduce NFS_LAYOUT_RETURN_BEFORE_CLOSE
  nfs/flexfiles: send layoutreturn before freeing lseg
  nfs41: add NFS_LAYOUT_RETRY_LAYOUTGET to layout header flags
  nfs: add a helper to set NFS_ODIRECT_RESCHED_WRITES to direct writes
  nfs41: wait for LAYOUTRETURN before retrying LAYOUTGET
  nfs: count DIO good bytes correctly with mirroring

Tom Haynes (5):
  pnfs: Prepare for flexfiles by pulling out common code
  pnfs: Do not grab the commit_info lock twice when rescheduling writes
  pnfs: Add nfs_rpc_ops in calls to nfs_initiate_pgio
  pnfs/flexfiles: Add the FlexFile Layout Driver
  pnfs: Update documentation on the Layout Drivers

Trond Myklebust (1):
  NFSv4.1/NFSv3: Add pNFS callbacks for nfs3_(read|write|commit)_done()

Weston Andros Adamson (9):
  sunrpc: add rpc_count_iostats_idx
  nfs: introduce pg_cleanup op for pgio descriptors
  pnfs: release lseg in pnfs_generic_pg_cleanup
  nfs: handle overlapping reqs in lock_and_join
  nfs: rename pgio header ds_idx to ds_commit_idx
  pnfs: pass ds_commit_idx through the commit path
  nfs: add mirroring support to pgio layer
  nfs: mirroring support for direct io
  pnfs: fail comparison when bucket verifier not set

 Documentation/filesystems/nfs/pnfs.txt    |   13 +-
 fs/nfs/Kconfig                            |    5 +
 fs/nfs/Makefile                           |    3 +-
 fs/nfs/blocklayout/blocklayout.c          |    2 +
 fs/nfs/direct.c                           |  112 +-
 fs/nfs/filelayout/filelayout.c            |  315 +-----
 fs/nfs/filelayout/filelayout.h            |   40 -
 fs/nfs/filelayout/filelayoutdev.c         |  469 +--------
 fs/nfs/flexfilelayout/Makefile            |    5 +
 fs/nfs/flexfilelayout/flexfilelayout.c    | 1571 +++++++++++++++++++++++++++++
 fs/nfs/flexfilelayout/flexfilelayout.h    |  155 +++
 fs/nfs/flexfilelayout/flexfilelayoutdev.c |  552 ++++++++++
 fs/nfs/idmap.c                            |    3 +-
 fs/nfs/internal.h                         |   31 +-
 fs/nfs/nfs2xdr.c                          |   10 +-
 fs/nfs/nfs3_fs.h                          |    2 +
 fs/nfs/nfs3client.c                       |   41 +
 fs/nfs/nfs3proc.c                         |    9 +
 fs/nfs/nfs3super.c                        |    2 +-
 fs/nfs/nfs3xdr.c                          |    3 +
 fs/nfs/nfs4_fs.h                          |    6 +
 fs/nfs/nfs4client.c                       |    7 +-
 fs/nfs/nfs4proc.c                         |   48 +-
 fs/nfs/nfs4xdr.c                          |    9 +-
 fs/nfs/objlayout/objio_osd.c              |    5 +-
 fs/nfs/pagelist.c                         |  294 +++++-
 fs/nfs/pnfs.c                             |  419 ++++++--
 fs/nfs/pnfs.h                             |  120 ++-
 fs/nfs/pnfs_nfs.c                         |  808 +++++++++++++++
 fs/nfs/read.c                             |   33 +-
 fs/nfs/write.c                            |   52 +-
 include/linux/nfs4.h                      |    1 +
 include/linux/nfs_fs_sb.h                 |    9 +-
 include/linux/nfs_idmap.h                 |    2 +
 include/linux/nfs_page.h                  |   22 +-
 include/linux/nfs_xdr.h                   |    6 +-
 include/linux/sunrpc/metrics.h            |    2 +
 net/sunrpc/stats.c                        |   26 +-
 38 files changed, 4194 insertions(+), 1018 deletions(-)
 create mode 100644 fs/nfs/flexfilelayout/Makefile
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.c
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.h
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayoutdev.c
 create mode 100644 fs/nfs/pnfs_nfs.c

-- 
1.9.3