Re: [PATCH v11 2/3] NFSD: Implement NFSD_IO_DIRECT for NFS WRITE

[Mike Snitzer <snitzer@kernel.org>]

On Fri, Nov 07, 2025 at 03:28:06PM -0500, Chuck Lever wrote:
> On 11/7/25 10:39 AM, Christoph Hellwig wrote:
> > On Fri, Nov 07, 2025 at 10:34:21AM -0500, Chuck Lever wrote:
> >> +no_dio:
> >> +	/*
> >> +	 * No DIO alignment possible - pack into single non-DIO segment.
> >> +	 * IOCB_DONTCACHE preserves the intent of NFSD_IO_DIRECT.
> >> +	 */
> >> +	nfsd_write_dio_seg_init(&segments[0], bvec, nvecs, total, 0,
> >> +				total, iocb);
> > 
> > I'd like to sort out the discussion on why to set IOCB_DONTCACHE when
> > nothing is aligned, but not for the non-aligned parts as that is
> > extremely counter-intuitive.  From the other thread it might be because
> > the test case used to justify it is very unaligned and caching partial
> > pages is helpful, but if that is indeed the case the right check would
> > be for writes that are file offset unaligned vs the page or max folio
> > size and not about being able to do parts of it as direct I/O.
> > 
> > Either way a tweak to suddenly use cached buffered I/O when the mode
> > asks for direct should have a comment explaining the justification
> > and explain the rationale instead of rushing it in.
> 
> +1 for an explanatory comment, but I'm not getting what is "counter-
> intuitive" about leaving the content of the end segments in the page
> cache. The end segments are small and simply cannot be handled by direct
> I/O.
> 
> I raised a similar concern about whether NFSD needs to care about highly
> unaligned NFS WRITE requests performing well. I'm not convinced that a
> performance argument is an appropriate rationale for not using
> DONTCACHE on the ends.

Those ends lend themselves to benefitting from more capable RMW
if/when needed.  All it takes to justify not using DONTCACHE is one
workload that benefits (even if suboptimal by nature) given there is
no apparent downside (other than requiring we document/comment the
behavior).

Or is this something we make tunable?
NFSD_IO_DIRECT_DONTCACHE_UNALIGNED?

I don't think it needed but maybe ultra small systems with next to no
memory that must not use memory at all costs (including performance)?
Maybe we wait until someone asks for that? ;)

> Upshot is I'm on the fence here; I can see both sides of this
> controversy.

NFSD_IO_DIRECT using cached buffered in this subpage write case makes
enabling NFSD_IO_DIRECT by default more acceptable _because_ it
doesn't cause needless performance problems (that we know of).

Streaming misaligned WRITEs is the only workload I'm aware of where
NFSD_IO_DIRECT can be made noticably worse in comparison to
NFSD_IO_BUFFERED (but I haven't done a bucnh of small IO testing).
That's a pretty good place for NFSD_IO_DIRECT given the fix is a
really straightforward tradeoff decision.

The MM subsystem handles order-0 page allocation and reclaim really
well, making NFSD_IO_DIRECT's 3 segment hybrid IO model quite capable
even though we hope applications don't force NFSD to use it (meaning
the client application takes care to send DIO-aligned IO).