* [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS
@ 2026-11-20 16:09 Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 1/5] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
When a file's flexfiles layouts carry FF_FLAGS_NO_IO_THRU_MDS, the client
keeps reads and writes off the MDS when it has no layout segment for
them. Reads and writes handle that differently:
- Reads test the inode's NFS_INO_NO_IO_THRU_MDS bit, wait a second
between attempts, and honour io_maxretrans on soft mounts.
- Writes test a bit on the layout header and return -EAGAIN.
nfs_writepages() and the O_DIRECT reschedule path resend at once, so
writers spin for as long as there is no layout. nfs_wb_folio()
returns the -EAGAIN, so a read(2) or write(2) that has to flush a
folio another open context dirtied fails with EAGAIN. Once the
header is freed the bit is gone and writes go through the MDS, and
ff_layout_pg_init_write() sends them there whenever
pnfs_generic_pg_check_layout() drops an invalidated segment.
Neither direction uses the MDS when the server has said there is no
layout to give. After NFS4ERR_LAYOUTUNAVAILABLE, RFC 8435 says the I/O
is to go to the MDS (section 5.1.1), and the flag is a hint (section
5.1.2). Instead, reads wait and writes spin until the 120 second fail
bit expires, then send a LAYOUTGET that can fail the same way.
Patch 1 sends writeback of a single locked folio (nfs_wb_folio(),
reclaim, migration) through the MDS instead of refusing it. Resending
with the folio locked could deadlock against a bulk recall, which can't
complete while nfs_writepages() holds a layout segment, and
nfs_writepages() would be waiting for that folio. Patch 2 makes the
O_DIRECT write work a delayed_work. Patch 3 has the remaining writes
test the inode bit. After the folios and requests of a refused pass
are unlocked, data-integrity nfs_writepages() waits a second before it
resends, and the O_DIRECT reschedule queues its next pass a second
later; other writeback leaves the folios dirty. Patch 4 removes the
header bit, which nothing tests after patch 3. Patch 5 sends reads,
writes and the resend after a data server error through the MDS while
the fail bit for their iomode is set.
That last change is also the path for a data server the client can't
use, such as a version mismatch: the client reports the failure with
LAYOUTERROR or LAYOUTRETURN, and the server decides whether to answer
the next LAYOUTGET with NFS4ERR_LAYOUTUNAVAILABLE. The missing-nfsv3
case is covered separately by Roberto's "NFSv4/flexfiles: couple back
with NFSv3".
These patches are independent of the series "pNFS: don't reply
NOMATCHING to a layout recall with I/O in flight", but both change
ff_layout_pg_get_mirror_count_write(), so whichever goes second will
need a small rebase.
Tested against a reffs MDS and DS with a six-way mirrored layout,
injecting LAYOUTGET errors after a file recall. With
NFS4ERR_LAYOUTUNAVAILABLE, reads and writes go through the MDS at once,
and the data written through the MDS is on every mirror. With a
retryable error (NFS4ERR_SERVERFAULT) held for 6 to 8 seconds, fsync
and the O_DIRECT reschedule retry once a second with no I/O to the MDS,
SIGKILL ends a writer blocked in fsync or an O_DIRECT write,
sync_file_range(SYNC_FILE_RANGE_WRITE) returns 0, and a soft mount with
io_maxretrans=3 returns EIO after about 3 seconds. During the hold, a
read of a folio that another open context had dirtied flushes it
through the MDS and returns without EAGAIN.
The series is based on v7.4-rc3 with three flexfiles fixes that were
posted separately for stable:
pNFS/flexfiles: read the layout hdr under i_lock in pg_get_mirror_count_write
https://lore.kernel.org/linux-nfs/23435d398481f51ce2680053cd3a151fe673f91c.1795108831.git.bcodding@hammerspace.com/
pNFS/flexfiles: don't delay or restart an rpc_task that LOCALIO never ran
https://lore.kernel.org/linux-nfs/9c46fda0f2cd6d3843b198dac304b4722514b8e3.1795108831.git.bcodding@hammerspace.com/
pNFS/flexfiles: restart a write that the DS error handler retries
https://lore.kernel.org/linux-nfs/8ee107c9d0c68b6d0ee7305605ff7edbe0ffa160.1795108832.git.bcodding@hammerspace.com/
Patch 3 replaces the header bit test that the first of these fixes.
Changes since v2:
https://lore.kernel.org/linux-nfs/cover.1795117974.git.bcodding@hammerspace.com/
- Patch 3: on a soft mount, fail the refused write once the next pass
would exceed io_maxretrans, instead of waiting a second for a pass
that fails anyway. The first O_DIRECT pass queues its reschedule a
second later too, instead of at once (sashiko).
Changes since v1:
https://lore.kernel.org/linux-nfs/cover.1795024981.git.bcodding@hammerspace.com/
- Rebased onto the three fixes above.
- Patch 1: shorter pg_allow_mds comment; the nfs_wb_folio() comment
says the refused folio was redirtied (Anna).
- New patch 2, so that patch 3 no longer sleeps on nfsiod.
- Patch 3 (was 2): only data-integrity nfs_writepages() waits, and a
fatal signal ends the wait; other writeback returns 0 instead of
waiting a second per folio. The O_DIRECT reschedule requeues after a
second instead of sleeping. The tracepoints that read the layout
header without an lseg are emitted under i_lock.
Benjamin Coddington (5):
pNFS/flexfiles: use the MDS for locked-folio writeback without a
layout
NFS: make the O_DIRECT write work a delayed_work
pNFS/flexfiles: pace writes that have no layout and can't use the MDS
pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit
pNFS/flexfiles: use the MDS when the server has no layout to give
fs/nfs/direct.c | 13 ++-
fs/nfs/flexfilelayout/flexfilelayout.c | 138 ++++++++++++++++---------
fs/nfs/flexfilelayout/flexfilelayout.h | 16 ---
fs/nfs/internal.h | 14 +--
fs/nfs/nfs4trace.c | 1 +
fs/nfs/nfs4trace.h | 1 +
fs/nfs/pagelist.c | 2 +
fs/nfs/pnfs.c | 22 ++++
fs/nfs/pnfs.h | 1 +
fs/nfs/write.c | 31 +++++-
include/linux/nfs_page.h | 2 +
11 files changed, 164 insertions(+), 77 deletions(-)
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
prerequisite-patch-id: 71097d575f7ba74684462ab64b028ff8815dff63
prerequisite-patch-id: 5a73a1fb9d47a31d4ec337ec0264eadbcaa9fd4e
prerequisite-patch-id: 2663ab8e1949d6a11d957cc2784161bf2816cac2
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v3 1/5] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
@ 2026-11-20 16:09 ` Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 2/5] NFS: make the O_DIRECT write work a delayed_work Benjamin Coddington
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
nfs_writepage_locked() writes back one folio that its caller holds
locked, for nfs_wb_folio(), nfs_wb_folio_reclaim() and migration. When
there is no layout segment and the file's layout header carries
FF_FLAGS_NO_IO_THRU_MDS, ff_layout_pg_get_mirror_count_write() refuses
the write with -EAGAIN. nfs_do_writepage() redirties the folio and
returns the -EAGAIN, and nfs_wb_folio() passes it to nfs_updatepage(),
nfs_flush_incompatible(), nfs_read_folio() and nfs_launder_folio(). A
read(2) or write(2) on a blocking file then fails with EAGAIN, for
example when another open context has dirtied the same folio while a
layout is being recalled.
Resending from nfs_wb_folio(), as nfs_writepages() does, would hold the
folio locked across the resends. A bulk recall can't complete while
nfs_writepages() holds a layout segment, and nfs_writepages() would be
waiting for that folio's lock.
FF_FLAGS_NO_IO_THRU_MDS is a hint (RFC 8435, section 5.1.2). Mark the
descriptor that nfs_writepage_locked() uses with a new pg_allow_mds
bit, and send its write through the MDS when there is no layout
segment. When ff_layout_pg_init_write() can't use a data server, send
such a write to the MDS too, instead of retrying with the folio locked.
ff_layout_pg_init_write() switches a write to the MDS by resetting the
descriptor and returning -EAGAIN, which nfs_wb_folio() also passes up,
with or without the flag. When nfs_wb_folio() gets -EAGAIN, wait for
any part of the folio already under writeback, as its loop does, and
write the folio once more with a descriptor that goes straight to the
MDS. nfs_wb_folio_reclaim() still returns the error, and
nfs_release_folio() keeps the folio.
Fixes: 8b0401373734 ("pNFS/flexfiles: Fix list corruption if the mirror count changes")
Fixes: 1d62e659c0bf ("NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS in pg_get_mirror_count_write")
Signed-off-by: Benjamin Coddington <bcodding@hammerspace.com>
---
fs/nfs/flexfilelayout/flexfilelayout.c | 5 +++--
fs/nfs/pagelist.c | 1 +
fs/nfs/write.c | 20 ++++++++++++++++----
include/linux/nfs_page.h | 1 +
4 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index 3adfcc7d0cfc..56527054d87b 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -1176,7 +1176,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio,
mirror_ds, dss_id, OP_WRITE);
if (IS_ERR(ds)) {
nfs4_ff_layout_put_deviceid(mirror_ds);
- if (!ff_layout_no_fallback_to_mds(pgio->pg_lseg))
+ if (pgio->pg_allow_mds ||
+ !ff_layout_no_fallback_to_mds(pgio->pg_lseg))
goto out_mds;
pnfs_generic_pg_cleanup(pgio);
/* Sleep for 1 second before retrying */
@@ -1239,7 +1240,7 @@ ff_layout_pg_get_mirror_count_write(struct nfs_pageio_descriptor *pgio,
lo = NFS_I(inode)->layout;
no_fallback = lo && ff_layout_hdr_no_fallback_to_mds(lo);
spin_unlock(&inode->i_lock);
- if (no_fallback) {
+ if (!pgio->pg_allow_mds && no_fallback) {
/*
* FF_FLAGS_NO_IO_THRU_MDS: no current lseg but the server's
* policy forbids MDS fallback. Surface -EAGAIN so writeback
diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c
index 71f0ce2bc4ea..aa6d7b658d79 100644
--- a/fs/nfs/pagelist.c
+++ b/fs/nfs/pagelist.c
@@ -871,6 +871,7 @@ void nfs_pageio_init(struct nfs_pageio_descriptor *desc,
int io_flags)
{
desc->pg_moreio = 0;
+ desc->pg_allow_mds = 0;
desc->pg_inode = inode;
desc->pg_ops = pg_ops;
desc->pg_completion_ops = compl_ops;
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 0b42d31f7a75..0b2b262b0dde 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -627,15 +627,16 @@ static int nfs_do_writepage(struct folio *folio, struct writeback_control *wbc,
* Write an mmapped page to the server.
*/
static int nfs_writepage_locked(struct folio *folio,
- struct writeback_control *wbc)
+ struct writeback_control *wbc, bool force_mds)
{
struct nfs_pageio_descriptor pgio;
struct inode *inode = folio->mapping->host;
int err;
nfs_inc_stats(inode, NFSIOS_VFSWRITEPAGE);
- nfs_pageio_init_write(&pgio, inode, 0, false,
+ nfs_pageio_init_write(&pgio, inode, 0, force_mds,
&nfs_async_write_completion_ops);
+ pgio.pg_allow_mds = 1;
err = nfs_do_writepage(folio, wbc, &pgio);
pgio.pg_error = 0;
nfs_pageio_complete(&pgio);
@@ -2057,7 +2058,7 @@ int nfs_wb_folio_reclaim(struct inode *inode, struct folio *folio)
return -EBUSY;
if (folio_clear_dirty_for_io(folio)) {
trace_nfs_writeback_folio_reclaim(inode, range_start, len);
- ret = nfs_writepage_locked(folio, &wbc);
+ ret = nfs_writepage_locked(folio, &wbc, false);
trace_nfs_writeback_folio_reclaim_done(inode, range_start, len,
ret);
return ret;
@@ -2084,6 +2085,7 @@ int nfs_wb_folio(struct inode *inode, struct folio *folio)
.range_start = range_start,
.range_end = range_start + len - 1,
};
+ bool force_mds = false;
int ret;
trace_nfs_writeback_folio(inode, range_start, len);
@@ -2091,7 +2093,17 @@ int nfs_wb_folio(struct inode *inode, struct folio *folio)
for (;;) {
folio_wait_writeback(folio);
if (folio_clear_dirty_for_io(folio)) {
- ret = nfs_writepage_locked(folio, &wbc);
+ ret = nfs_writepage_locked(folio, &wbc, force_mds);
+ /*
+ * -EAGAIN redirtied the folio and asks for a resend,
+ * which can't wait for a layout with the folio locked:
+ * resend it through the MDS once any part of it
+ * already sent has completed.
+ */
+ if (ret == -EAGAIN && !force_mds) {
+ force_mds = true;
+ continue;
+ }
if (ret < 0)
goto out_error;
continue;
diff --git a/include/linux/nfs_page.h b/include/linux/nfs_page.h
index c38e4b380be5..d79bea9d4f90 100644
--- a/include/linux/nfs_page.h
+++ b/include/linux/nfs_page.h
@@ -120,6 +120,7 @@ struct nfs_pageio_descriptor {
u32 pg_mirror_idx; /* current mirror */
unsigned short pg_maxretrans;
unsigned char pg_moreio : 1;
+ unsigned char pg_allow_mds : 1; /* MDS if no layout */
};
/* arbitrarily selected limit to number of mirrors */
--
2.53.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 2/5] NFS: make the O_DIRECT write work a delayed_work
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 1/5] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
@ 2026-11-20 16:09 ` Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 3/5] pNFS/flexfiles: pace writes that have no layout and can't use the MDS Benjamin Coddington
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
nfs_direct_write_complete() queues dreq->work on nfsiod_workqueue to
commit, reschedule or complete an O_DIRECT write. Make it a
delayed_work, queued with no delay, so that a reschedule can be paced
without sleeping in the worker.
Signed-off-by: Benjamin Coddington <bcodding@hammerspace.com>
---
fs/nfs/direct.c | 8 +++++---
fs/nfs/internal.h | 2 +-
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index a3c6e8f4ea06..9ef8031a68ad 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
@@ -219,7 +219,7 @@ static inline struct nfs_direct_req *nfs_direct_req_alloc(void)
init_completion(&dreq->completion);
INIT_LIST_HEAD(&dreq->mds_cinfo.list);
pnfs_init_ds_commit_info(&dreq->ds_cinfo);
- INIT_WORK(&dreq->work, nfs_direct_write_schedule_work);
+ INIT_DELAYED_WORK(&dreq->work, nfs_direct_write_schedule_work);
spin_lock_init(&dreq->lock);
return dreq;
@@ -739,7 +739,8 @@ static void nfs_direct_write_clear_reqs(struct nfs_direct_req *dreq)
static void nfs_direct_write_schedule_work(struct work_struct *work)
{
- struct nfs_direct_req *dreq = container_of(work, struct nfs_direct_req, work);
+ struct nfs_direct_req *dreq = container_of(to_delayed_work(work),
+ struct nfs_direct_req, work);
int flags = dreq->flags;
dreq->flags = 0;
@@ -760,7 +761,8 @@ static void nfs_direct_write_schedule_work(struct work_struct *work)
static void nfs_direct_write_complete(struct nfs_direct_req *dreq)
{
trace_nfs_direct_write_complete(dreq);
- queue_work(nfsiod_workqueue, &dreq->work); /* Calls nfs_direct_write_schedule_work */
+ /* Calls nfs_direct_write_schedule_work */
+ queue_delayed_work(nfsiod_workqueue, &dreq->work, 0);
}
static void nfs_direct_write_completion(struct nfs_pgio_header *hdr)
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 5af6a032f145..8f1361397f8f 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -1035,7 +1035,7 @@ struct nfs_direct_req {
/* commit state */
struct nfs_mds_commit_info mds_cinfo; /* Storage for cinfo */
struct pnfs_ds_commit_info ds_cinfo; /* Storage for cinfo */
- struct work_struct work;
+ struct delayed_work work;
int flags;
/* for write */
#define NFS_ODIRECT_DO_COMMIT (1) /* an unstable reply was received */
--
2.53.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 3/5] pNFS/flexfiles: pace writes that have no layout and can't use the MDS
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 1/5] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 2/5] NFS: make the O_DIRECT write work a delayed_work Benjamin Coddington
@ 2026-11-20 16:09 ` Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 4/5] pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 5/5] pNFS/flexfiles: use the MDS when the server has no layout to give Benjamin Coddington
4 siblings, 0 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
When a write has no layout segment and its file's layouts carry
FF_FLAGS_NO_IO_THRU_MDS, ff_layout_pg_get_mirror_count_write() returns
-EAGAIN. nfs_writepages() and the O_DIRECT reschedule path resend it at
once, so writers spin for as long as pnfs_update_layout() has no layout
to give: while NFS_LAYOUT_BULK_RECALL is set, while LAYOUTGETs are
blocked, and for the 120 seconds after NFS4ERR_LAYOUTUNAVAILABLE.
The check also tests a bit on the layout header. Once that header has
been freed, a new one starts without the bit and the write goes through
the MDS. ff_layout_pg_init_write() has no check at all: when
pnfs_generic_pg_check_layout() drops an invalidated segment, the rest of
the descriptor's writes go through the MDS.
Handle both places the way ff_layout_pg_init_read() handles reads: test
the inode's NFS_INO_NO_IO_THRU_MDS bit, return -ERESTARTSYS when the
task sending the write is killed or exiting, and on soft mounts give up
once the next pass would exceed io_maxretrans. Mark a refusal with a
new pg_nolayout descriptor bit. After the pass that was refused has
unlocked its folios and requests, data-integrity nfs_writepages() waits
a second before it resends, and a fatal signal ends the wait. Other
writeback stops there and leaves the refused folios dirty, instead of
waiting a second for each one. The O_DIRECT path queues its reschedule
a second later instead of resending at once, both from
nfs_direct_write_schedule_iovec() and from
nfs_direct_write_reschedule(). That pass runs on nfsiod rather than in
the writer, so a fatal signal to the writer does not end its retries;
io_maxretrans on soft mounts does. Writes marked pg_allow_mds by
nfs_writepage_locked() still go through the MDS.
The test is ff_layout_inode_no_fallback_to_mds(), which
ff_layout_pg_init_read() now uses too, and the refusal is
ff_layout_pg_refuse_write(). Emit the MDS-fallback tracepoint only when
the write does go through the MDS, and a new
pnfs_mds_fallback_write_refused tracepoint when it is refused. No lseg
pins the layout header that either tracepoint reads, so emit them under
i_lock.
Fixes: 1d62e659c0bf ("NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS in pg_get_mirror_count_write")
Signed-off-by: Benjamin Coddington <bcodding@hammerspace.com>
---
fs/nfs/direct.c | 7 +-
fs/nfs/flexfilelayout/flexfilelayout.c | 91 ++++++++++++++++++--------
fs/nfs/internal.h | 1 +
fs/nfs/nfs4trace.c | 1 +
fs/nfs/nfs4trace.h | 1 +
fs/nfs/pagelist.c | 1 +
fs/nfs/write.c | 11 ++++
include/linux/nfs_page.h | 1 +
8 files changed, 87 insertions(+), 27 deletions(-)
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index 9ef8031a68ad..2f355f7c8fc7 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
@@ -626,6 +626,8 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq)
}
}
+ if (desc.pg_error == -EAGAIN && desc.pg_nolayout)
+ dreq->work_delay = HZ;
if (put_dreq(dreq))
nfs_direct_write_complete(dreq);
}
@@ -744,6 +746,7 @@ static void nfs_direct_write_schedule_work(struct work_struct *work)
int flags = dreq->flags;
dreq->flags = 0;
+ dreq->work_delay = 0;
switch (flags) {
case NFS_ODIRECT_DO_COMMIT:
nfs_direct_commit_schedule(dreq);
@@ -762,7 +765,7 @@ static void nfs_direct_write_complete(struct nfs_direct_req *dreq)
{
trace_nfs_direct_write_complete(dreq);
/* Calls nfs_direct_write_schedule_work */
- queue_delayed_work(nfsiod_workqueue, &dreq->work, 0);
+ queue_delayed_work(nfsiod_workqueue, &dreq->work, dreq->work_delay);
}
static void nfs_direct_write_completion(struct nfs_pgio_header *hdr)
@@ -928,6 +931,8 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
nfs_init_cinfo_from_dreq(&cinfo, dreq);
spin_lock(&dreq->lock);
dreq->flags = NFS_ODIRECT_RESCHED_WRITES;
+ if (desc.pg_nolayout)
+ dreq->work_delay = HZ;
spin_unlock(&dreq->lock);
nfs_unlock_request(req);
nfs_mark_request_commit(req, NULL, &cinfo, 0);
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index 56527054d87b..ef8715de6474 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -982,6 +982,51 @@ static bool ff_layout_pg_retry_wait(struct nfs_pageio_descriptor *pgio)
return true;
}
+/*
+ * Returns true if the inode's layouts forbid I/O through the MDS.
+ */
+static bool ff_layout_inode_no_fallback_to_mds(struct inode *inode, u32 iomode)
+{
+ return nfs_no_io_thru_mds(inode);
+}
+
+/*
+ * Refuse a write that has no layout segment and can't go through the MDS:
+ * -EAGAIN so that the caller resends it, or a hard error when the caller
+ * should give up. pg_nolayout tells a caller that resends at once to
+ * wait first, once it has dropped its locks.
+ */
+static void ff_layout_pg_refuse_write(struct nfs_pageio_descriptor *pgio,
+ struct nfs_page *req)
+{
+ struct inode *inode = pgio->pg_inode;
+ struct nfs_server *server = NFS_SERVER(inode);
+
+ /* No lseg pins the layout hdr, and it is detached under i_lock */
+ if (trace_pnfs_mds_fallback_write_refused_enabled()) {
+ spin_lock(&inode->i_lock);
+ trace_pnfs_mds_fallback_write_refused(inode, req_offset(req),
+ req->wb_bytes, IOMODE_RW,
+ NFS_I(inode)->layout,
+ NULL);
+ spin_unlock(&inode->i_lock);
+ }
+ if (server->flags & (NFS_MOUNT_SOFT | NFS_MOUNT_SOFTERR) &&
+ io_maxretrans && req->wb_nio >= io_maxretrans) {
+ if (server->flags & NFS_MOUNT_SOFTERR)
+ pgio->pg_error = -ETIMEDOUT;
+ else
+ pgio->pg_error = -EIO;
+ return;
+ }
+ if (fatal_signal_pending(current) || nfs_current_task_exiting()) {
+ pgio->pg_error = -ERESTARTSYS;
+ return;
+ }
+ pgio->pg_nolayout = 1;
+ pgio->pg_error = -EAGAIN;
+}
+
static void
ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio,
struct nfs_page *req,
@@ -1108,7 +1153,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,
* FF_FLAGS_NO_IO_THRU_MDS may itself be gone by now, which is why
* this asks the inode and not the hdr.
*/
- if (!nfs_no_io_thru_mds(pgio->pg_inode))
+ if (!ff_layout_inode_no_fallback_to_mds(pgio->pg_inode, IOMODE_READ))
goto out_mds;
retry_nolseg:
/* Retry getting layout segment if lower layer returned -EAGAIN */
@@ -1156,9 +1201,13 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio,
return;
}
}
- /* If no lseg, fall back to write through mds */
- if (pgio->pg_lseg == NULL)
- goto out_mds;
+ if (!pgio->pg_lseg) {
+ if (pgio->pg_allow_mds ||
+ !ff_layout_inode_no_fallback_to_mds(pgio->pg_inode, IOMODE_RW))
+ goto out_mds;
+ ff_layout_pg_refuse_write(pgio, req);
+ return;
+ }
/* Use a direct mapping of ds_idx to pgio mirror_idx */
if (pgio->pg_mirror_count != FF_LAYOUT_MIRROR_COUNT(pgio->pg_lseg))
@@ -1212,10 +1261,6 @@ static unsigned int
ff_layout_pg_get_mirror_count_write(struct nfs_pageio_descriptor *pgio,
struct nfs_page *req)
{
- struct inode *inode = pgio->pg_inode;
- struct pnfs_layout_hdr *lo;
- bool no_fallback;
-
if (!pgio->pg_lseg) {
pgio->pg_lseg =
pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req),
@@ -1229,26 +1274,20 @@ ff_layout_pg_get_mirror_count_write(struct nfs_pageio_descriptor *pgio,
}
if (pgio->pg_lseg)
return FF_LAYOUT_MIRROR_COUNT(pgio->pg_lseg);
-
- trace_pnfs_mds_fallback_pg_get_mirror_count(pgio->pg_inode,
- 0, NFS4_MAX_UINT64, IOMODE_RW,
- NFS_I(pgio->pg_inode)->layout,
- pgio->pg_lseg);
-
- /* No lseg pins the layout hdr, and it is detached under i_lock */
- spin_lock(&inode->i_lock);
- lo = NFS_I(inode)->layout;
- no_fallback = lo && ff_layout_hdr_no_fallback_to_mds(lo);
- spin_unlock(&inode->i_lock);
- if (!pgio->pg_allow_mds && no_fallback) {
- /*
- * FF_FLAGS_NO_IO_THRU_MDS: no current lseg but the server's
- * policy forbids MDS fallback. Surface -EAGAIN so writeback
- * retries rather than silently issuing the WRITE via MDS.
- */
- pgio->pg_error = -EAGAIN;
+ if (!pgio->pg_allow_mds &&
+ ff_layout_inode_no_fallback_to_mds(pgio->pg_inode, IOMODE_RW)) {
+ ff_layout_pg_refuse_write(pgio, req);
goto out;
}
+
+ if (trace_pnfs_mds_fallback_pg_get_mirror_count_enabled()) {
+ spin_lock(&pgio->pg_inode->i_lock);
+ trace_pnfs_mds_fallback_pg_get_mirror_count(pgio->pg_inode,
+ 0, NFS4_MAX_UINT64, IOMODE_RW,
+ NFS_I(pgio->pg_inode)->layout,
+ pgio->pg_lseg);
+ spin_unlock(&pgio->pg_inode->i_lock);
+ }
/* no lseg means that pnfs is not in use, so no mirroring here */
nfs_pageio_reset_write_mds(pgio);
out:
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 8f1361397f8f..831f0e80206a 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -1036,6 +1036,7 @@ struct nfs_direct_req {
struct nfs_mds_commit_info mds_cinfo; /* Storage for cinfo */
struct pnfs_ds_commit_info ds_cinfo; /* Storage for cinfo */
struct delayed_work work;
+ unsigned long work_delay; /* jiffies */
int flags;
/* for write */
#define NFS_ODIRECT_DO_COMMIT (1) /* an unstable reply was received */
diff --git a/fs/nfs/nfs4trace.c b/fs/nfs/nfs4trace.c
index 3fdc013f56d8..93a84f546de8 100644
--- a/fs/nfs/nfs4trace.c
+++ b/fs/nfs/nfs4trace.c
@@ -25,6 +25,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_read_done);
EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_write_done);
EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_read_pagelist);
EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_write_pagelist);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_write_refused);
EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_ds_connect);
EXPORT_TRACEPOINT_SYMBOL_GPL(ff_layout_read_error);
diff --git a/fs/nfs/nfs4trace.h b/fs/nfs/nfs4trace.h
index b5c89eeef2bc..94ed6b833a28 100644
--- a/fs/nfs/nfs4trace.h
+++ b/fs/nfs/nfs4trace.h
@@ -2115,6 +2115,7 @@ DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_read_done);
DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_write_done);
DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_read_pagelist);
DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_write_pagelist);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_write_refused);
DECLARE_EVENT_CLASS(nfs4_deviceid_event,
TP_PROTO(
diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c
index aa6d7b658d79..9a8731de6f5b 100644
--- a/fs/nfs/pagelist.c
+++ b/fs/nfs/pagelist.c
@@ -872,6 +872,7 @@ void nfs_pageio_init(struct nfs_pageio_descriptor *desc,
{
desc->pg_moreio = 0;
desc->pg_allow_mds = 0;
+ desc->pg_nolayout = 0;
desc->pg_inode = inode;
desc->pg_ops = pg_ops;
desc->pg_completion_ops = compl_ops;
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 0b2b262b0dde..fbb7df3320de 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -693,6 +693,17 @@ int nfs_writepages(struct address_space *mapping, struct writeback_control *wbc)
nfs_pageio_complete(&pgio);
if (err == -EAGAIN && mntflags & NFS_MOUNT_SOFTERR)
break;
+ if (err == -EAGAIN && pgio.pg_nolayout) {
+ if (wbc->sync_mode != WB_SYNC_ALL) {
+ err = 0;
+ break;
+ }
+ if (schedule_timeout_killable(HZ) &&
+ fatal_signal_pending(current)) {
+ err = -ERESTARTSYS;
+ break;
+ }
+ }
} while (err < 0 && !nfs_error_is_fatal(err));
nfs_io_completion_put(ioc);
diff --git a/include/linux/nfs_page.h b/include/linux/nfs_page.h
index d79bea9d4f90..10b45434c335 100644
--- a/include/linux/nfs_page.h
+++ b/include/linux/nfs_page.h
@@ -121,6 +121,7 @@ struct nfs_pageio_descriptor {
unsigned short pg_maxretrans;
unsigned char pg_moreio : 1;
unsigned char pg_allow_mds : 1; /* MDS if no layout */
+ unsigned char pg_nolayout : 1; /* refused, no layout */
};
/* arbitrarily selected limit to number of mirrors */
--
2.53.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 4/5] pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
` (2 preceding siblings ...)
2026-11-20 16:09 ` [PATCH v3 3/5] pNFS/flexfiles: pace writes that have no layout and can't use the MDS Benjamin Coddington
@ 2026-11-20 16:09 ` Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 5/5] pNFS/flexfiles: use the MDS when the server has no layout to give Benjamin Coddington
4 siblings, 0 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
Nothing tests NFS4_FF_HDR_NO_IO_THRU_MDS since writes moved to the
inode's NFS_INO_NO_IO_THRU_MDS bit. Remove it, its helper, and
nfs4_flexfile_layout::flags, which held only that bit.
Signed-off-by: Benjamin Coddington <bcodding@hammerspace.com>
---
fs/nfs/flexfilelayout/flexfilelayout.c | 8 ++------
fs/nfs/flexfilelayout/flexfilelayout.h | 16 ----------------
fs/nfs/internal.h | 9 ++++-----
3 files changed, 6 insertions(+), 27 deletions(-)
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index ef8715de6474..4d3ad04775d6 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -641,12 +641,8 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh,
if (!p)
goto out_sort_mirrors;
fls->flags = be32_to_cpup(p);
- if (fls->flags & FF_FLAGS_NO_IO_THRU_MDS) {
- set_bit(NFS4_FF_HDR_NO_IO_THRU_MDS,
- &FF_LAYOUT_FROM_HDR(lh)->flags);
- /* Outlives this layout hdr; see NFS_INO_NO_IO_THRU_MDS */
+ if (fls->flags & FF_FLAGS_NO_IO_THRU_MDS)
nfs_set_no_io_thru_mds(lh->plh_inode);
- }
p = xdr_inline_decode(&stream, 4);
if (!p)
@@ -1151,7 +1147,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,
* MDS there is no fallback to take, so wait for a layout on the
* same terms as the -EAGAIN above. The layout hdr that carried
* FF_FLAGS_NO_IO_THRU_MDS may itself be gone by now, which is why
- * this asks the inode and not the hdr.
+ * the flag is kept on the inode.
*/
if (!ff_layout_inode_no_fallback_to_mds(pgio->pg_inode, IOMODE_READ))
goto out_mds;
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h
index 5d5587fded76..bb60537b2b93 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.h
+++ b/fs/nfs/flexfilelayout/flexfilelayout.h
@@ -113,16 +113,12 @@ struct nfs4_ff_layout_segment {
struct nfs4_ff_layout_mirror *mirror_array[] __counted_by(mirror_array_cnt);
};
-/* nfs4_flexfile_layout::flags bit indices */
-#define NFS4_FF_HDR_NO_IO_THRU_MDS 0 /* any lseg has had FF_FLAGS_NO_IO_THRU_MDS */
-
struct nfs4_flexfile_layout {
struct pnfs_layout_hdr generic_hdr;
struct pnfs_ds_commit_info commit_info;
struct list_head mirrors;
struct list_head error_list; /* nfs4_ff_layout_ds_err */
ktime_t last_report_time; /* Layoutstat report times */
- unsigned long flags;
};
/*
@@ -186,18 +182,6 @@ ff_layout_no_fallback_to_mds(struct pnfs_layout_segment *lseg)
return FF_LAYOUT_LSEG(lseg)->flags & FF_FLAGS_NO_IO_THRU_MDS;
}
-/*
- * Sticky hdr-level mirror of FF_FLAGS_NO_IO_THRU_MDS so callers that have
- * no current lseg (e.g. between LAYOUTRETURN and the next LAYOUTGET) can
- * still honor the no-MDS-fallback policy.
- */
-static inline bool
-ff_layout_hdr_no_fallback_to_mds(struct pnfs_layout_hdr *lo)
-{
- return test_bit(NFS4_FF_HDR_NO_IO_THRU_MDS,
- &FF_LAYOUT_FROM_HDR(lo)->flags);
-}
-
static inline bool
ff_layout_no_read_on_rw(struct pnfs_layout_segment *lseg)
{
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 831f0e80206a..9819c4db3d85 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -556,11 +556,10 @@ static inline bool nfs_file_io_is_buffered(struct nfs_inode *nfsi)
* FF_FLAGS_NO_IO_THRU_MDS in ff_layout_alloc_lseg()). The RFC 5661
* mdsthreshold hint must not be acted on for such a file, since the only
* thing pnfs_within_mdsthreshold() can ask for is the one thing the
- * layout forbids. Servers are assumed to be consistent in their
- * no-fallback policy per file, the same assumption
- * ff_layout_hdr_no_fallback_to_mds() already makes; if one were not, the
- * only effect is that its mdsthreshold hint - a SHOULD - stops being
- * honored for an inode that is already in core.
+ * layout forbids. Flexfiles also retries reads and writes that have no
+ * layout segment instead of sending them through the MDS, except
+ * writeback of a single locked folio (pg_allow_mds). Servers are
+ * assumed to be consistent in their no-fallback policy per file.
*/
#define NFS_INO_NO_IO_THRU_MDS (30)
--
2.53.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 5/5] pNFS/flexfiles: use the MDS when the server has no layout to give
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
` (3 preceding siblings ...)
2026-11-20 16:09 ` [PATCH v3 4/5] pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit Benjamin Coddington
@ 2026-11-20 16:09 ` Benjamin Coddington
4 siblings, 0 replies; 6+ messages in thread
From: Benjamin Coddington @ 2026-11-20 16:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Mike Snitzer; +Cc: linux-nfs
From: Benjamin Coddington <bcodding@hammerspace.com>
When LAYOUTGET fails with NFS4ERR_LAYOUTUNAVAILABLE, or with
NFS4ERR_TOOSMALL after the retry at the session's maximum reply size,
pnfs_update_layout() sets the layout's fail bit for that iomode and
returns no layout without sending another LAYOUTGET for the next 120
seconds. If the file's layouts carried FF_FLAGS_NO_IO_THRU_MDS, reads
and writes wait for a layout the whole time, and then send a LAYOUTGET
that can fail the same way. On a hard mount that repeats for as long
as the server keeps answering that way.
RFC 8435 says that after NFS4ERR_LAYOUTUNAVAILABLE "the I/O is to go to
the metadata server" (section 5.1.1), and that FF_FLAGS_NO_IO_THRU_MDS
is a hint (section 5.1.2). Send the I/O through the MDS while the fail
bit is set, in ff_layout_inode_no_fallback_to_mds(), which reads,
writes and the resend after a data server error share. This also lets
a server deal with a data server the client can't use: the client
reports the failure with LAYOUTERROR (or in LAYOUTRETURN) and returns
the layout, and the server can answer the next LAYOUTGET with
NFS4ERR_LAYOUTUNAVAILABLE.
Fixes: 2f007ba074ad ("pNFS/flexfiles: don't read through MDS when previous layout forbid it")
Fixes: 1d62e659c0bf ("NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS in pg_get_mirror_count_write")
Fixes: ec906cf7bee6 ("pNFS/flexfiles: don't reset to MDS for v4 error when previous layout forbid it")
Signed-off-by: Benjamin Coddington <bcodding@hammerspace.com>
---
fs/nfs/flexfilelayout/flexfilelayout.c | 40 +++++++++++++++-----------
fs/nfs/internal.h | 6 ++--
fs/nfs/pnfs.c | 22 ++++++++++++++
fs/nfs/pnfs.h | 1 +
4 files changed, 50 insertions(+), 19 deletions(-)
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index 4d3ad04775d6..6fc082e97c86 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -980,10 +980,14 @@ static bool ff_layout_pg_retry_wait(struct nfs_pageio_descriptor *pgio)
/*
* Returns true if the inode's layouts forbid I/O through the MDS.
+ * FF_FLAGS_NO_IO_THRU_MDS is a hint (RFC 8435, section 5.1.2), so set it
+ * aside while a failed LAYOUTGET for @iomode (NFS4ERR_LAYOUTUNAVAILABLE or
+ * NFS4ERR_TOOSMALL) keeps pnfs_update_layout() from sending another.
*/
static bool ff_layout_inode_no_fallback_to_mds(struct inode *inode, u32 iomode)
{
- return nfs_no_io_thru_mds(inode);
+ return nfs_no_io_thru_mds(inode) &&
+ !pnfs_layout_get_failed(inode, iomode);
}
/*
@@ -1141,13 +1145,12 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,
}
/*
* No segment, and no error to report either: pnfs_update_layout()
- * simply has nothing to give (NFS_LAYOUT_BULK_RECALL, a failed
- * pnfs_layout_io_test, blocked LAYOUTGETs, a layout being
- * returned). If the server forbids reading this file through the
- * MDS there is no fallback to take, so wait for a layout on the
- * same terms as the -EAGAIN above. The layout hdr that carried
- * FF_FLAGS_NO_IO_THRU_MDS may itself be gone by now, which is why
- * the flag is kept on the inode.
+ * simply has nothing to give (NFS_LAYOUT_BULK_RECALL, blocked
+ * LAYOUTGETs, a layout being returned). If the server forbids
+ * reading this file through the MDS there is no fallback to take,
+ * so wait for a layout on the same terms as the -EAGAIN above. The
+ * layout hdr that carried FF_FLAGS_NO_IO_THRU_MDS may itself be
+ * gone by now, which is why the flag is kept on the inode.
*/
if (!ff_layout_inode_no_fallback_to_mds(pgio->pg_inode, IOMODE_READ))
goto out_mds;
@@ -1412,7 +1415,8 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,
struct nfs4_state *state,
struct nfs_client *clp,
struct pnfs_layout_segment *lseg,
- struct nfs4_deviceid_node *devid)
+ struct nfs4_deviceid_node *devid,
+ u32 iomode)
{
struct pnfs_layout_hdr *lo = lseg->pls_layout;
struct inode *inode = lo->plh_inode;
@@ -1501,8 +1505,9 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,
return -NFS4ERR_RESET_TO_PNFS;
reset:
/*
- * FF_FLAGS_NO_IO_THRU_MDS: never resend through the MDS. The
- * caller would do so with force_mds set (ff_layout_reset_read(),
+ * FF_FLAGS_NO_IO_THRU_MDS: don't resend through the MDS (see
+ * ff_layout_inode_no_fallback_to_mds()). The caller would do so
+ * with force_mds set (ff_layout_reset_read(),
* ff_layout_reset_write(hdr, false) -> pnfs_*_done_resend_to_mds()),
* which builds a descriptor out of the plain MDS page ops and so
* consults no layout at all -- this is the last point at which the
@@ -1512,7 +1517,7 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,
* pnfs_destroy_layout(). Having done so they can no longer ask the
* layout hdr about the flag, hence the inode.
*/
- if (nfs_no_io_thru_mds(inode)) {
+ if (ff_layout_inode_no_fallback_to_mds(inode, iomode)) {
dprintk("%s Retry through pNFS, no MDS fallback. Error %d\n",
__func__, task->tk_status);
return -NFS4ERR_RESET_TO_PNFS;
@@ -1597,7 +1602,8 @@ static int ff_layout_async_handle_error(struct rpc_task *task,
struct nfs4_state *state,
struct nfs_client *clp,
struct pnfs_layout_segment *lseg,
- struct nfs4_deviceid_node *devid)
+ struct nfs4_deviceid_node *devid,
+ u32 iomode)
{
int vers = clp->cl_nfs_mod->rpc_vers->number;
@@ -1617,7 +1623,7 @@ static int ff_layout_async_handle_error(struct rpc_task *task,
lseg, devid);
case 4:
return ff_layout_async_handle_error_v4(task, op_status, state,
- clp, lseg, devid);
+ clp, lseg, devid, iomode);
default:
/* should never happen */
WARN_ON_ONCE(1);
@@ -1751,7 +1757,7 @@ static int ff_layout_read_done_cb(struct rpc_task *task,
err = ff_layout_async_handle_error(task, hdr->res.op_status,
hdr->args.context->state,
hdr->ds_clp, hdr->lseg,
- hdr->ds_dev);
+ hdr->ds_dev, IOMODE_READ);
trace_nfs4_pnfs_read(hdr, err);
clear_bit(NFS_IOHDR_RESEND_PNFS, &hdr->flags);
@@ -1961,7 +1967,7 @@ static int ff_layout_write_done_cb(struct rpc_task *task,
err = ff_layout_async_handle_error(task, hdr->res.op_status,
hdr->args.context->state,
hdr->ds_clp, hdr->lseg,
- hdr->ds_dev);
+ hdr->ds_dev, IOMODE_RW);
trace_nfs4_pnfs_write(hdr, err);
clear_bit(NFS_IOHDR_RESEND_PNFS, &hdr->flags);
@@ -2013,7 +2019,7 @@ static int ff_layout_commit_done_cb(struct rpc_task *task,
err = ff_layout_async_handle_error(task, data->res.op_status,
NULL, data->ds_clp, data->lseg,
- data->ds_dev);
+ data->ds_dev, IOMODE_RW);
trace_nfs4_pnfs_commit_ds(data, err);
switch (err) {
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 9819c4db3d85..bcbb5996c55c 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -558,8 +558,10 @@ static inline bool nfs_file_io_is_buffered(struct nfs_inode *nfsi)
* thing pnfs_within_mdsthreshold() can ask for is the one thing the
* layout forbids. Flexfiles also retries reads and writes that have no
* layout segment instead of sending them through the MDS, except
- * writeback of a single locked folio (pg_allow_mds). Servers are
- * assumed to be consistent in their no-fallback policy per file.
+ * writeback of a single locked folio (pg_allow_mds), and unless a
+ * LAYOUTGET for that iomode has failed with NFS4ERR_LAYOUTUNAVAILABLE or
+ * NFS4ERR_TOOSMALL. Servers are assumed to be consistent in their
+ * no-fallback policy per file.
*/
#define NFS_INO_NO_IO_THRU_MDS (30)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 14f4a4b12ffd..582fe81fc2d1 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -551,6 +551,28 @@ pnfs_layout_io_test_failed(struct pnfs_layout_hdr *lo, u32 iomode)
return true;
}
+/*
+ * Returns true if a LAYOUTGET for @iomode failed with an error, such as
+ * NFS4ERR_LAYOUTUNAVAILABLE, in the last PNFS_LAYOUTGET_RETRY_TIMEOUT.
+ * pnfs_layout_io_test_failed() clears an expired fail bit only when
+ * pnfs_update_layout() gets that far, so check the time here too.
+ */
+bool pnfs_layout_get_failed(struct inode *inode, u32 iomode)
+{
+ struct pnfs_layout_hdr *lo;
+ unsigned long now = jiffies;
+ bool ret = false;
+
+ spin_lock(&inode->i_lock);
+ lo = NFS_I(inode)->layout;
+ if (lo && test_bit(pnfs_iomode_to_fail_bit(iomode), &lo->plh_flags))
+ ret = time_in_range(lo->plh_retry_timestamp,
+ now - PNFS_LAYOUTGET_RETRY_TIMEOUT, now);
+ spin_unlock(&inode->i_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(pnfs_layout_get_failed);
+
static void
pnfs_init_lseg(struct pnfs_layout_hdr *lo, struct pnfs_layout_segment *lseg,
const struct pnfs_layout_range *range,
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 2774d4adf4c3..0007a4ca3da9 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -296,6 +296,7 @@ int pnfs_generic_pg_writepages(struct nfs_pageio_descriptor *desc);
size_t pnfs_generic_pg_test(struct nfs_pageio_descriptor *pgio,
struct nfs_page *prev, struct nfs_page *req);
void pnfs_set_lo_fail(struct pnfs_layout_segment *lseg);
+bool pnfs_layout_get_failed(struct inode *inode, u32 iomode);
struct pnfs_layout_segment *pnfs_layout_process(struct nfs4_layoutget *lgp);
void pnfs_layoutget_free(struct nfs4_layoutget *lgp);
void pnfs_free_lseg_list(struct list_head *tmp_list);
--
2.53.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-20 16:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 16:09 [PATCH v3 0/5] pNFS/flexfiles: I/O without a layout under FF_FLAGS_NO_IO_THRU_MDS Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 1/5] pNFS/flexfiles: use the MDS for locked-folio writeback without a layout Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 2/5] NFS: make the O_DIRECT write work a delayed_work Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 3/5] pNFS/flexfiles: pace writes that have no layout and can't use the MDS Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 4/5] pNFS/flexfiles: remove the layout-header NO_IO_THRU_MDS bit Benjamin Coddington
2026-11-20 16:09 ` [PATCH v3 5/5] pNFS/flexfiles: use the MDS when the server has no layout to give Benjamin Coddington
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.