Re: [PATCH v7 05/14] NFSD: @stable for direct writes is always NFS_FILE_SYNC

[Mike Snitzer <snitzer@kernel.org>]

On Mon, Oct 27, 2025 at 06:27:47AM -0700, Christoph Hellwig wrote:
> On Mon, Oct 27, 2025 at 09:23:27AM -0400, Chuck Lever wrote:
> > Promoting all NFSD_IO_DIRECT writes to FILE_SYNC was my idea, based on
> > the assumption that IOCB_DIRECT writes to local file systems left
> > nothing to be done by a later commit. My assumption is based on the
> > behavior of O_DIRECT on NFS files.
> > 
> > If that assumption is not true, then I agree there is no technical
> > reason to promote NFSD_IO_DIRECT writes to FILE_SYNC, and I can remove
> > that built-in assumption for v8 of this series.
> 
> It is not true, or rather only true for a tiny subset of use cases
> (which NFS can't even query a head of time).
> 
> For devices that advertise a volatile write cache, commit has to flush
> that.  High-end enough device won't have one, but a lot of devices that
> people NFS-export do.  For pure overwrites the file system could
> optimize this way by using the FUA flag, and at least the iomap direct
> I/O code does implementation that optimization for that particular case.

NFSD_IO_DIRECT isn't meant to be uniformly better for all types of
storage.  Any storage that has a volatile write cache is probably best
served by existing NFSD default (NFSD_IO_BUFFERED).

> For any write that is not purely an overwrite, commit has to write out
> the metadata to track the newly allocated blocks.  Applications that
> do overwrite fully allocated blocks typically do that using the O_DSYNC
> flag to fully benefit from optimizations for that case.

Think there is a bit of disconnect here in that this new
NFSD_IO_DIRECT mode is admin configured purely on NFSD's side and
always-on once enabled.

The client doesn't control if/when NFSD would make use of O_DIRECT
(other than if it sends misaligned IO and NFSD must do what it can to
ensure it safely hits stable storage).

In addition, the use of NFSD_IO_DIRECT is intended to allow for
systems large _and_ small to get the advantage of lower memory
utilization.  Buffered IO is one extreme, but even using a model where
NFSD were to not impose NFS_FILE_SYNC would create a situation where
more memory needed batch IO and then wait for client to send COMMIT.

The current approach of using IOCB_DSYNC|IOCB_SYNC have performed
really well on modern NVMe servers.