Re: [PATCH v2 00/10] server-side support for "inter" SSC copy

Olga Kornievskaia
2018-12-20T18:42:41Z

On Fri, Nov 30, 2018 at 3:03 PM Olga Kornievskaia
<olga.kornievskaia@gmail.com> wrote:
>
> This patch series adds support for NFSv4.2 copy offload feature
> allowing copy between two different NFS servers.
>
> This functionality depends on the VFS ability to support generic
> copy_file_range() where a copy is done between an NFS file and
> a local file system.
>
> This feature is enabled by the kernel module parameter --
> inter_copy_offload_enable -- and by default is disabled. There is
> also a kernel compile configuration of NFSD_V4_2_INTER_SSC that
> adds dependency on the NFS client side functions called from the
> server.
>
> These patches work on top of existing async intra copy offload
> patches. For the "inter" SSC, the implementation only supports
> asynchronous inter copy.
>
> On the source server, upon receiving a COPY_NOTIFY, it generate a
> unique stateid that's kept in the global list. Upon receiving a READ
> with a stateid, the code checks the normal list of open stateid and
> now additionally, it'll check the copy state list as well before
> deciding to either fail with BAD_STATEID or find one that matches.
> The stored stateid is only valid to be used for the first time
> with a choosen lease period (90s currently). When the source server
> received an OFFLOAD_CANCEL, it will remove the stateid from the
> global list. Otherwise, the copy stateid is removed upon the removal
> of its "parent" stateid (open/lock/delegation stateid).
>
> On the destination server, upon receiving a COPY request, the server
> establishes the necessary clientid/session with the source server.
> It calls into the NFS client code to establish the necessary
> open stateid, filehandle, file description (without doing an NFS open).
> Then the server calls into the copy_file_range() to preform the copy
> where the source file will issue NFS READs and then do local file
> system writes (this depends on the VFS ability to do cross device
> copy_file_range().
>
> v2:
> -- in on top of 4.20-rc4 + client side inter patch series
> -- VFS changes to do enable generic copy_file_range() and then NFS
> falls back on generic_copy_file_range() for previous EXDEV/OPNOTSUPP
> errors
> -- hopefully addressed Bruce's review comments (highlights are):
> --- copy_notify patch: addressed naming, sc_cp_list access is
> now protected by s2s_cp_lock
> --- fillin netloc4 patch: address the size and added WARN_ON
> --- add ca_source to COPY: decode only 1 address, dont allocate
> memory (the rest into dummy)
> --- check stateid against stored: moved the refcount under lock
> --- allow stale filehandle: adding a loop to go thru the ops in
> the compound, store/manage puttfh if copy is present in the compound
> mark the source putfh as "no verify".
>
> All the patches (client inter) and this patch series is available
> from git://linux-nfs.org/projects/aglo/linux.git under the "linux-ssc"
> branch
>

Bruce,

Do you have comments on this v2? Once VFS has the patches for the
generic copy_file_range() functionality, NFS should be all set to just
used it.

> Olga Kornievskaia (10):
> VFS generic copy_file_range() support
> NFS fallback to generic_copy_file_range
> NFSD fill-in netloc4 structure
> NFSD add ca_source_server<> to COPY
> NFSD return nfs4_stid in nfs4_preprocess_stateid_op
> NFSD add COPY_NOTIFY operation
> NFSD check stateids against copy stateids
> NFSD generalize nfsd4_compound_state flag names
> NFSD: allow inter server COPY to have a STALE source server fh
> NFSD add nfs4 inter ssc to nfsd4_copy
>
> fs/nfs/nfs4file.c | 9 +-
> fs/nfsd/Kconfig | 10 ++
> fs/nfsd/nfs4proc.c | 406 ++++++++++++++++++++++++++++++++++++++++++++++-----
> fs/nfsd/nfs4state.c | 124 ++++++++++++++--
> fs/nfsd/nfs4xdr.c | 166 ++++++++++++++++++++-
> fs/nfsd/nfsd.h | 32 ++++
> fs/nfsd/nfsfh.h | 5 +-
> fs/nfsd/nfssvc.c | 6 +
> fs/nfsd/state.h | 21 ++-
> fs/nfsd/xdr4.h | 37 ++++-
> fs/read_write.c | 66 +++++++--
> include/linux/fs.h | 7 +
> include/linux/nfs4.h | 1 +
> mm/filemap.c | 6 +-
> 14 files changed, 810 insertions(+), 86 deletions(-)
>
> --
> 1.8.3.1
>