[PATCH v4 01/10] NFSD: cut a misaligned direct-mode WRITE at page boundaries

[Mike Snitzer <snitzer@kernel.org>]

nfsd_write_dio_iters_init() splits a misaligned direct-mode WRITE into a
direct middle and buffered ends so that as little of it as possible goes
through the page cache.  That only works if direct and buffered I/O never
meet inside a page: the direct middle invalidates the page cache over its
range, so a middle that starts or stops inside the page a buffered end
writes has that page dropped, and whichever of the two neighbouring
WRITEs comes second has to read it back from the device.

The middle was cut at the file system's direct I/O offset alignment:

	prefix_end = round_up(offset, offset_align);
	middle_end = round_down(orig_end, offset_align);

On 4 KiB pages over 4096-byte blocks that alignment is the page size,
the cut is a page boundary, and the problem cannot arise.  It is broken
wherever the page is larger than the alignment: 16 and 64 KiB pages over
4096-byte blocks, and 4 KiB pages over 512-byte blocks.  There a
misaligned WRITE costs a read per shared page on top of its three I/Os.

Cut each end of a WRITE on its own:

- an end that is already an offset_align multiple needs no buffered
  segment, and the middle starts or stops there, mid-page or not.  A
  WRITE aligned to the device stays entirely direct at every page size;
- a misaligned end needs a buffered segment whatever is done, and that
  segment now runs to the nearest page boundary inside the WRITE.

That is one rule: a misaligned end is cut at a multiple of the larger of
offset_align and PAGE_SIZE, which are both powers of two.

The segments still tile exactly the bytes of the WRITE: a buffered end
grows inward, at the middle's expense, never outward.  A misaligned WRITE
that holds no whole page has no middle and is issued as one buffered
segment.  Where the offset alignment is a page-size multiple nothing
changes.

nfsd-io-modes.rst said that DONTCACHE is not used for the misaligned
segments, which the write loop already contradicted: every segment
without IOCB_DIRECT gets IOCB_DONTCACHE when the file system supports
it.  Replace that sentence with one that says so, next to one that
describes the new cut.

Fixes: 06c5c97293e3 ("NFSD: Implement NFSD_IO_DIRECT for NFS WRITE")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
---
 Documentation/filesystems/nfs/nfsd-io-modes.rst |  9 ++++++---
 fs/nfsd/vfs.c                                   | 13 +++++++++++++
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/nfs/nfsd-io-modes.rst b/Documentation/filesystems/nfs/nfsd-io-modes.rst
index 0fd6e82478fe6..1916b2eee58ad 100644
--- a/Documentation/filesystems/nfs/nfsd-io-modes.rst
+++ b/Documentation/filesystems/nfs/nfsd-io-modes.rst
@@ -126,9 +126,12 @@ Misaligned WRITE:
     middle and end as needed. The large middle segment is DIO-aligned
     and the start and/or end are misaligned. Buffered IO is used for the
     misaligned segments and O_DIRECT is used for the middle DIO-aligned
-    segment. DONTCACHE buffered IO is _not_ used for the misaligned
-    segments because using normal buffered IO offers significant RMW
-    performance benefit when handling streaming misaligned WRITEs.
+    segment. A misaligned start or end segment runs to the nearest page
+    boundary inside the WRITE, not just to the nearest DIO-aligned
+    offset, so the O_DIRECT segment never shares a page with a buffered
+    one; a start or end that is already DIO-aligned gets no buffered
+    segment at all. If the filesystem supports FOP_DONTCACHE, the
+    misaligned segments are marked DONTCACHE.
 
 Tracing:
     The nfsd_read_direct trace event shows how NFSD expands any
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index f4069da853725..ed5903c3becac 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -1324,6 +1324,7 @@ nfsd_write_dio_iters_init(struct nfsd_file *nf, struct bio_vec *bvec,
 	size_t prefix, middle, suffix;
 	loff_t offset = iocb->ki_pos;
 	unsigned int nsegs = 0;
+	u32 cut;
 
 	/*
 	 * Check if direct I/O is feasible for this write request.
@@ -1338,6 +1339,18 @@ nfsd_write_dio_iters_init(struct nfsd_file *nf, struct bio_vec *bvec,
 	orig_end = offset + total;
 	middle_end = round_down(orig_end, offset_align);
 
+	/*
+	 * The direct middle invalidates the page cache over its range, so it
+	 * must not share a page with a buffered end.  An aligned end needs no
+	 * buffered segment; a misaligned one is cut on a page boundary even
+	 * where the offset alignment is finer than a page.
+	 */
+	cut = max_t(u32, offset_align, PAGE_SIZE);
+	if (prefix_end != offset)
+		prefix_end = min(round_up(offset, cut), orig_end);
+	if (middle_end != orig_end)
+		middle_end = max(round_down(orig_end, cut), prefix_end);
+
 	prefix = prefix_end - offset;
 	middle = middle_end - prefix_end;
 	suffix = orig_end - middle_end;
-- 
2.52.0