Re: [PATCH v11 2/3] NFSD: Implement NFSD_IO_DIRECT for NFS WRITE

[Mike Snitzer <snitzer@kernel.org>]

On Fri, Nov 07, 2025 at 10:40:56AM -0500, Chuck Lever wrote:
> On 11/7/25 10:39 AM, Christoph Hellwig wrote:
> > On Fri, Nov 07, 2025 at 10:34:21AM -0500, Chuck Lever wrote:
> >> +no_dio:
> >> +	/*
> >> +	 * No DIO alignment possible - pack into single non-DIO segment.
> >> +	 * IOCB_DONTCACHE preserves the intent of NFSD_IO_DIRECT.
> >> +	 */
> >> +	nfsd_write_dio_seg_init(&segments[0], bvec, nvecs, total, 0,
> >> +				total, iocb);
> > 
> > I'd like to sort out the discussion on why to set IOCB_DONTCACHE when
> > nothing is aligned, but not for the non-aligned parts as that is
> > extremely counter-intuitive.  From the other thread it might be because
> > the test case used to justify it is very unaligned and caching partial
> > pages is helpful, but if that is indeed the case the right check would
> > be for writes that are file offset unaligned vs the page or max folio
> > size and not about being able to do parts of it as direct I/O.

NFSD_IO_DIRECT's splitting an unaligned WRITE into 3 segments is a
hybrid IO approach that consciously (_from initial design_) leverages
the page cache for the unaligned ends.

1: The ends of an unaligned write are certainly file offset unaligned.
2: The ends are also less than a page so implies unaligned in memory.

Anyway, categorizing the misalignment further enables you to do what
differently?  Any misalignment requires use of buffered IO.

At issue if whether we use DONTCACHE's drop-behind semantic or not.
That drop-behind is clearly unhelpful for the ends of each streaming
unaligned WRITE.

The page cache is beneficial for NFSD_IO_DIRECT's subpage writes on
either end of a DIO aligned middle (at most 2 pages left in page cache
for unaligned IO).  So _not_ using DONTCACHE is a key point of the
hybrid IO model NFSD_IO_DIRECT provides for handling unaligned IO.

> > Either way a tweak to suddenly use cached buffered I/O when the mode
> > asks for direct should have a comment explaining the justification
> > and explain the rationale instead of rushing it in.

Use of cached buffered wasn't sudden or rushing -- it restores a key
original design point of NFSD_IO_DIRECT (from back in July/August).

While I can appreciate your confusion, given NFSD_IO_DIRECT's intent
of avoiding caching, DONTCACHE's ability to preserve the general
"avoid caching" semantic of O_DIRECT is really besides the point
because... NFSD_IO_DIRECT does need to use the page cache for its
unaligned end segments _and_ it actually benefits from page caching.

And I completely agree this needs a comment. Hopefully adding
something like the collowing comment before "return nsegs" in
nfsd_write_dio_iters_init() helps:

/*
 * Normal buffered IO is used for the unaligned prefix/suffix
 * because these subpage writes can benefit from the page cache
 * (e.g. for optimally handling streaming unaligned WRITEs).
 */

> Agreed. The cover letter noted that this is still controversial.

Only see this, must be missing what you're referring to:

  Changes since v9:
  * Unaligned segments no longer use IOCB_DONTCACHE