Re: [PATCH v2 4/4] NFSD: handle unaligned DIO for NFS reexport

[Mike Snitzer <snitzer@kernel.org>]

On Fri, Aug 01, 2025 at 10:33:02AM -0400, Jeff Layton wrote:
> On Fri, 2025-08-01 at 10:07 -0400, Chuck Lever wrote:
> > On 7/31/25 5:48 PM, Mike Snitzer wrote:
> > > On Thu, Jul 31, 2025 at 05:28:13PM -0400, Mike Snitzer wrote:
> > > > On Thu, Jul 31, 2025 at 04:58:00PM -0400, Jeff Layton wrote:
> > > > > On Thu, 2025-07-31 at 15:44 -0400, Mike Snitzer wrote:
> > > > > > NFS doesn't have any DIO alignment constraints but it doesn't support
> > > > > > STATX_DIOALIGN, so update NFSD such that it doesn't disable the use of
> > > > > > NFSD_IO_DIRECT if it is reexporting NFS.
> > > > > > 
> > > > > > Signed-off-by: Mike Snitzer <snitzer@kernel.org>
> > > > > > ---
> > > > > >  fs/nfs/export.c          |  3 ++-
> > > > > >  fs/nfsd/filecache.c      | 11 +++++++++++
> > > > > >  include/linux/exportfs.h | 13 +++++++++++++
> > > > > >  3 files changed, 26 insertions(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/fs/nfs/export.c b/fs/nfs/export.c
> > > > > > index e9c233b6fd209..2cae75ba6b35d 100644
> > > > > > --- a/fs/nfs/export.c
> > > > > > +++ b/fs/nfs/export.c
> > > > > > @@ -155,5 +155,6 @@ const struct export_operations nfs_export_ops = {
> > > > > >  		 EXPORT_OP_REMOTE_FS		|
> > > > > >  		 EXPORT_OP_NOATOMIC_ATTR	|
> > > > > >  		 EXPORT_OP_FLUSH_ON_CLOSE	|
> > > > > > -		 EXPORT_OP_NOLOCKS,
> > > > > > +		 EXPORT_OP_NOLOCKS		|
> > > > > > +		 EXPORT_OP_NO_DIOALIGN_NEEDED,
> > > > > >  };
> > > > > > diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c
> > > > > > index 5601e839a72da..ea489dd44fd9a 100644
> > > > > > --- a/fs/nfsd/filecache.c
> > > > > > +++ b/fs/nfsd/filecache.c
> > > > > > @@ -1066,6 +1066,17 @@ nfsd_file_getattr(const struct svc_fh *fhp, struct nfsd_file *nf)
> > > > > >  	     nfsd_io_cache_write != NFSD_IO_DIRECT))
> > > > > >  		return nfs_ok;
> > > > > >  
> > > > > > +	if (exportfs_handles_unaligned_dio(nf->nf_file->f_path.mnt->mnt_sb->s_export_op)) {
> > > > > > +		/* Underlying filesystem doesn't support STATX_DIOALIGN
> > > > > > +		 * but it can handle all unaligned DIO, so establish
> > > > > > +		 * DIO alignment that is accommodating.
> > > > > > +		 */
> > > > > > +		nf->nf_dio_mem_align = 4;
> > > > > > +		nf->nf_dio_offset_align = PAGE_SIZE;
> > > > > > +		nf->nf_dio_read_offset_align = nf->nf_dio_offset_align;
> > > > > > +		return nfs_ok;
> > > > > > +	}
> > > > > > +
> > > > > >  	status = fh_getattr(fhp, &stat);
> > > > > >  	if (status != nfs_ok)
> > > > > >  		return status;
> > > > > > diff --git a/include/linux/exportfs.h b/include/linux/exportfs.h
> > > > > > index 9369a607224c1..626b8486dd985 100644
> > > > > > --- a/include/linux/exportfs.h
> > > > > > +++ b/include/linux/exportfs.h
> > > > > > @@ -247,6 +247,7 @@ struct export_operations {
> > > > > >  						*/
> > > > > >  #define EXPORT_OP_FLUSH_ON_CLOSE	(0x20) /* fs flushes file data on close */
> > > > > >  #define EXPORT_OP_NOLOCKS		(0x40) /* no file locking support */
> > > > > > +#define EXPORT_OP_NO_DIOALIGN_NEEDED	(0x80) /* fs can handle unaligned DIO */
> > > > > >  	unsigned long	flags;
> > > > > >  };
> > > > > >  
> > > > > > @@ -262,6 +263,18 @@ exportfs_cannot_lock(const struct export_operations *export_ops)
> > > > > >  	return export_ops->flags & EXPORT_OP_NOLOCKS;
> > > > > >  }
> > > > > >  
> > > > > > +/**
> > > > > > + * exportfs_handles_unaligned_dio() - check if export can handle unaligned DIO
> > > > > > + * @export_ops:	the nfs export operations to check
> > > > > > + *
> > > > > > + * Returns true if the export can handle unaligned DIO.
> > > > > > + */
> > > > > > +static inline bool
> > > > > > +exportfs_handles_unaligned_dio(const struct export_operations *export_ops)
> > > > > > +{
> > > > > > +	return export_ops->flags & EXPORT_OP_NO_DIOALIGN_NEEDED;
> > > > > > +}
> > > > > > +
> > > > > >  extern int exportfs_encode_inode_fh(struct inode *inode, struct fid *fid,
> > > > > >  				    int *max_len, struct inode *parent,
> > > > > >  				    int flags);
> > > > > 
> > > > > 
> > > > > Would it not be simpler (better?) to add support for STATX_DIOALIGN to
> > > > > NFS, and just have it report '1' for both values?
> > > > 
> > > > I suppose adding NFS support for STATX_DIOALIGN, that doesn't actually
> > > > go over the wire, does make sense.
> > > > 
> > > > But I wouldn't think setting them to 1 valid.  Pretty sure they need
> > > > to be a power-of-2 (since they are used as masks passed to
> > > > iov_iter_is_aligned).
> > > > 
> > > > In addition, we want to make sure NFS's default DIO alignment (which
> > > > isn't informed by actual DIO alignment advertised by NFSD's underlying
> > > > filesystem and hardware, e.g. XFS and NVMe) is able to be compatible
> > > > with both finer (512b) and coarser (4096b) grained DIO alignment.
> > > > Only way to achieve that would be to skew toward the coarse-grained
> > > > end of the spectrum, right?
> > > > 
> > > > More conservative but more likely to work with everything.
> > > 
> > > Thinking/looking further: I really do prefer the approach I took in
> > > this patch (over the suggestion to implement STATX_DIOALIGN for NFS).
> > > 
> > > Otherwise NFS would forced to needlessly issue an RPC via fh_getattr()
> > > even though we're talking about NFS faking its STATX_DIOALIGN response
> > > (so it doesn't need to do the work to do a full-blown GETATTR).
> > > 
> > > This would be wasteful for the NFS reexport usecase.
> > 
> > Jeff's point is that applications (and in particular, user space NFS
> > servers) will use statx() to discover these values. The NFS client has
> > to implement STATX_DIOALIGN.

I agree, I'll add it to my TODO.

Adding that support is now very important for the NFS reexport case.
But joining them is a choice. They don't _need_ to be tightly coupled.

We could just have NFS and NFSD call the same helper _and_ set the
EXPORT_OP_NO_DIOALIGN_NEEDED I proposed.

> > I don't buy the idea that using vfs_getattr() to call into the NFS
> > client is wasteful here. Isn't this done once when the nfsd_file
> > is opened? And, since these are emulated values that are not fetched
> > via the NFS protocol, wouldn't that mean the NFS client could respond
> > without sending an RPC?

Not if the request_mask includes more than simply asking for
(STATX_DIOALIGN | STATX_DIO_READ_ALIGN) -- which currently will always
be the case for fh_getattr().  But yes, fair point that it only
happens for nfsd_file open.

> > I prefer to not add the exception processing to NFSD if, in the medium
> > to long run, the NFS client has to get support for DIOALIGN anyway.
> > 
> 
> I too think this would be a better approach. We have other exportable
> filesystems that have no DIO alignment restrictions too (Ceph comes to
> mind, but there are others). It would be nice if they "just worked" and
> didn't have to do special EXPORT_* flag shenanigans.

Not trying to make this harder than it is, just stating my peace and
then acquiescing...

Each filesystem where this applicable just setting a single well
defined export flag is considerably simpler than the alternative (of
needing to construct an emulated STATX_DIOALIGN response that will
accomplish the goal of NFSD transforming misaligned DIO such that it
isn't too fine-grained to be useful once the rubber meets the road and
the DIO is issued to the local filesystem exported by the ultimate
NFSD endpoint).

Maybe for bonus points, not holding breath, I'll see about adding a
EXPORT_SYMBOL helper <handwave>somewhere</handwave> that will allow
other filesystems like Ceph to benefit too. But to my eyes, an export
flag provides that ;)

Anyway, we have agreement that:
1) You'll drop this 4th patch for NFS reexport
2) I'll work on implementing STATX_DIOALIGN support for NFS

Thanks,
Mike