All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/4] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS
@ 2026-11-18 18:36 Benjamin Coddington
  2026-11-18 18:36 ` [PATCH 1/4] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: Benjamin Coddington @ 2026-11-18 18:36 UTC (permalink / raw)
  To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs

From: Benjamin Coddington <bcodding@hammerspace.com>

When a file's flexfiles layouts carry FF_FLAGS_NO_IO_THRU_MDS, the client
keeps reads and writes off the MDS when it has no layout segment for
them.  Reads and writes handle that differently:

 - Reads test the inode's NFS_INO_NO_IO_THRU_MDS bit, wait a second
   between attempts, and honour io_maxretrans on soft mounts.

 - Writes test a bit on the layout header and return -EAGAIN.
   nfs_writepages() and the O_DIRECT reschedule path resend at once, so
   writers spin for as long as there is no layout.  nfs_wb_folio()
   returns the -EAGAIN, so a read(2) or write(2) that has to flush a
   folio another open context dirtied fails with EAGAIN.  Once the
   header is freed the bit is gone and writes go through the MDS, and
   ff_layout_pg_init_write() sends them there whenever
   pnfs_generic_pg_check_layout() drops an invalidated segment.

Neither direction uses the MDS when the server has said there is no
layout to give.  After NFS4ERR_LAYOUTUNAVAILABLE, RFC 8435 says the I/O
is to go to the MDS (section 5.1.1), and the flag is a hint (section
5.1.2).  Instead, reads wait and writes spin until the 120 second fail
bit expires, then send a LAYOUTGET that can fail the same way.

Patch 1 sends writeback of a single locked folio (nfs_wb_folio(),
reclaim, migration) through the MDS instead of refusing it.  Resending
with the folio locked could deadlock against a bulk recall, which can't
complete while nfs_writepages() holds a layout segment, and
nfs_writepages() would be waiting for that folio.  Patch 2 has the
remaining writes test the inode bit, and has nfs_writepages() and the
O_DIRECT reschedule path wait a second before resending a refused
write, after their folios and requests are unlocked.  Patch 3 removes
the header bit, which nothing tests after patch 2.  Patch 4 sends
reads, writes and the resend after a data server error through the MDS
while the fail bit for their iomode is set.

That last change is also the path for a data server the client can't
use, such as a version mismatch: the client reports the failure with
LAYOUTERROR or LAYOUTRETURN, and the server decides whether to answer
the next LAYOUTGET with NFS4ERR_LAYOUTUNAVAILABLE.  The missing-nfsv3
case is covered separately by Roberto's "NFSv4/flexfiles: couple back
with NFSv3".

These patches are independent of the series "pNFS: don't reply
NOMATCHING to a layout recall with I/O in flight", but both change
ff_layout_pg_get_mirror_count_write(), so whichever goes second will
need a small rebase.  The series is based on Anna's linux-next.

Tested against a reffs MDS and DS, injecting LAYOUTGET errors after a
file recall.  With NFS4ERR_LAYOUTUNAVAILABLE, reads and writes go through
the MDS at once.  With a retryable error (NFS4ERR_SERVERFAULT) held for
8 seconds, writeback sends one LAYOUTGET a second and no I/O to the MDS,
SIGKILL ends a writer blocked in fsync, and a soft mount with
io_maxretrans=3 returns EIO after about 4 seconds.  During the hold, a
read of a folio that another open context had dirtied flushes it
through the MDS and returns without EAGAIN.

Benjamin Coddington (4):
  pNFS/flexfiles: use the MDS for locked-folio writeback without a
    layout
  pNFS/flexfiles: pace writes that have no layout and can't use the MDS
  pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit
  pNFS/flexfiles: use the MDS when the server has no layout to give

 fs/nfs/direct.c                        |   2 +
 fs/nfs/flexfilelayout/flexfilelayout.c | 113 +++++++++++++++++--------
 fs/nfs/flexfilelayout/flexfilelayout.h |  16 ----
 fs/nfs/internal.h                      |  11 +--
 fs/nfs/nfs4trace.c                     |   1 +
 fs/nfs/nfs4trace.h                     |   1 +
 fs/nfs/pagelist.c                      |   2 +
 fs/nfs/pnfs.c                          |  22 +++++
 fs/nfs/pnfs.h                          |   1 +
 fs/nfs/write.c                         |  21 ++++-
 include/linux/nfs_page.h               |  10 +++
 11 files changed, 139 insertions(+), 61 deletions(-)

-- 
2.53.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-18 23:56 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 18:36 [PATCH 0/4] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
2026-11-18 18:36 ` [PATCH 1/4] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
2026-11-18 23:22   ` Anna Schumaker
2026-11-18 23:56     ` Benjamin Coddington
2026-11-18 18:36 ` [PATCH 2/4] pNFS/flexfiles: pace writes that have no layout and can't use the MDS Benjamin Coddington
2026-11-18 23:52   ` Anna Schumaker
2026-11-18 18:36 ` [PATCH 3/4] pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit Benjamin Coddington
2026-11-18 18:36 ` [PATCH 4/4] pNFS/flexfiles: use the MDS when the server has no layout to give Benjamin Coddington

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.