* [PATCH 0/3] FF_FLAGS_NO_READ_IO
@ 2016-05-18 21:07 Tom Haynes
2016-05-18 21:07 ` [PATCH 1/3] nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO Tom Haynes
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Tom Haynes @ 2016-05-18 21:07 UTC (permalink / raw)
To: Trond Myklebust; +Cc: Linux NFS Mailing list, Anna Schumaker
FF_FLAGS_NO_READ_IO is a hint the mds can use to inform the
client that an IOMODE_RW layout segment should not be used
for READ traffic and that the client should instead request
an IOMODE_READ layout segment. This can be used to isolate
only WRITE traffic onto the IOMODE_RW layout segment.
Tom Haynes (3):
nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO
nfs/flexfiles: Use the layout segment for reading unless it a
IOMODE_RW and reading is disabled
pnfs: pnfs_update_layout needs to consider if strict iomode checking
is on
fs/nfs/filelayout/filelayout.c | 2 ++
fs/nfs/flexfilelayout/flexfilelayout.c | 50 +++++++++++++++++++++++--------
fs/nfs/flexfilelayout/flexfilelayout.h | 10 ++++++-
fs/nfs/flexfilelayout/flexfilelayoutdev.c | 6 ++++
fs/nfs/pnfs.c | 34 +++++++++++++--------
fs/nfs/pnfs.h | 1 +
6 files changed, 77 insertions(+), 26 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH 1/3] nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO 2016-05-18 21:07 [PATCH 0/3] FF_FLAGS_NO_READ_IO Tom Haynes @ 2016-05-18 21:07 ` Tom Haynes 2016-05-18 21:07 ` [PATCH 2/3] nfs/flexfiles: Use the layout segment for reading unless it a IOMODE_RW and reading is disabled Tom Haynes ` (2 subsequent siblings) 3 siblings, 0 replies; 5+ messages in thread From: Tom Haynes @ 2016-05-18 21:07 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing list, Anna Schumaker The mds can inform the client not to use the IOMODE_RW layout segment for doing READs. I.e., it is basically a IOMODE_WRITE layout segment. It would do this to not interfere with the WRITEs. Signed-off-by: Tom Haynes <loghyr@primarydata.com> --- fs/nfs/flexfilelayout/flexfilelayout.h | 10 +++++++++- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 6 ++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h index 8dabf64..c66c6c1 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.h +++ b/fs/nfs/flexfilelayout/flexfilelayout.h @@ -10,7 +10,8 @@ #define FS_NFS_NFS4FLEXFILELAYOUT_H #define FF_FLAGS_NO_LAYOUTCOMMIT 1 -#define FF_FLAGS_NO_IO_THRU_MDS 2 +#define FF_FLAGS_NO_IO_THRU_MDS 2 +#define FF_FLAGS_NO_READ_IO 4 #include "../pnfs.h" @@ -155,6 +156,12 @@ ff_layout_no_fallback_to_mds(struct pnfs_layout_segment *lseg) } static inline bool +ff_layout_no_read_on_rw(struct pnfs_layout_segment *lseg) +{ + return FF_LAYOUT_LSEG(lseg)->flags & FF_FLAGS_NO_READ_IO; +} + +static inline bool ff_layout_test_devid_unavailable(struct nfs4_deviceid_node *node) { return nfs4_test_deviceid_unavailable(node); @@ -194,6 +201,7 @@ struct rpc_cred *ff_layout_get_ds_cred(struct pnfs_layout_segment *lseg, u32 ds_idx, struct rpc_cred *mdscred); bool ff_layout_has_available_ds(struct pnfs_layout_segment *lseg); bool ff_layout_avoid_mds_available_ds(struct pnfs_layout_segment *lseg); +bool ff_layout_avoid_read_on_rw(struct pnfs_layout_segment *lseg); struct file *ff_local_open_fh(struct pnfs_layout_segment *lseg, u32 ds_idx, struct nfs_client *clp, struct rpc_cred *cred, diff --git a/fs/nfs/flexfilelayout/flexfilelayoutdev.c b/fs/nfs/flexfilelayout/flexfilelayoutdev.c index 1a603d4..b90c2bd 100644 --- a/fs/nfs/flexfilelayout/flexfilelayoutdev.c +++ b/fs/nfs/flexfilelayout/flexfilelayoutdev.c @@ -623,6 +623,12 @@ bool ff_layout_avoid_mds_available_ds(struct pnfs_layout_segment *lseg) ff_layout_has_available_ds(lseg); } +bool ff_layout_avoid_read_on_rw(struct pnfs_layout_segment *lseg) +{ + return lseg->pls_range.iomode == IOMODE_RW && + ff_layout_no_read_on_rw(lseg); +} + module_param(dataserver_retrans, uint, 0644); MODULE_PARM_DESC(dataserver_retrans, "The number of times the NFSv4.1 client " "retries a request before it attempts further " -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/3] nfs/flexfiles: Use the layout segment for reading unless it a IOMODE_RW and reading is disabled 2016-05-18 21:07 [PATCH 0/3] FF_FLAGS_NO_READ_IO Tom Haynes 2016-05-18 21:07 ` [PATCH 1/3] nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO Tom Haynes @ 2016-05-18 21:07 ` Tom Haynes 2016-05-18 21:07 ` [PATCH 3/3] pnfs: pnfs_update_layout needs to consider if strict iomode checking is on Tom Haynes 2016-05-24 20:20 ` [PATCH 0/3] FF_FLAGS_NO_READ_IO Anna Schumaker 3 siblings, 0 replies; 5+ messages in thread From: Tom Haynes @ 2016-05-18 21:07 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing list, Anna Schumaker Signed-off-by: Tom Haynes <loghyr@primarydata.com> --- fs/nfs/flexfilelayout/flexfilelayout.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 981b39a..ea24b0a 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -852,7 +852,8 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, int ds_idx; /* Use full layout for now */ - if (!pgio->pg_lseg) { + if (!pgio->pg_lseg || ff_layout_avoid_read_on_rw(pgio->pg_lseg)) { + pnfs_put_lseg(pgio->pg_lseg); pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, req->wb_context, 0, @@ -862,9 +863,9 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); pgio->pg_lseg = NULL; - return; } } + /* If no lseg, fall back to read through mds */ if (pgio->pg_lseg == NULL) goto out_mds; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/3] pnfs: pnfs_update_layout needs to consider if strict iomode checking is on 2016-05-18 21:07 [PATCH 0/3] FF_FLAGS_NO_READ_IO Tom Haynes 2016-05-18 21:07 ` [PATCH 1/3] nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO Tom Haynes 2016-05-18 21:07 ` [PATCH 2/3] nfs/flexfiles: Use the layout segment for reading unless it a IOMODE_RW and reading is disabled Tom Haynes @ 2016-05-18 21:07 ` Tom Haynes 2016-05-24 20:20 ` [PATCH 0/3] FF_FLAGS_NO_READ_IO Anna Schumaker 3 siblings, 0 replies; 5+ messages in thread From: Tom Haynes @ 2016-05-18 21:07 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing list, Anna Schumaker As flexfiles has FF_FLAGS_NO_READ_IO, there is a need to generically support enforcing that a IOMODE_RW segment will not allow READ I/O. Signed-off-by: Tom Haynes <loghyr@primarydata.com> --- fs/nfs/filelayout/filelayout.c | 2 ++ fs/nfs/flexfilelayout/flexfilelayout.c | 49 +++++++++++++++++++++++++--------- fs/nfs/pnfs.c | 34 ++++++++++++++--------- fs/nfs/pnfs.h | 1 + 4 files changed, 61 insertions(+), 25 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 7cda834..8f9ab33 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -898,6 +898,7 @@ filelayout_pg_init_read(struct nfs_pageio_descriptor *pgio, 0, NFS4_MAX_UINT64, IOMODE_READ, + false, GFP_KERNEL); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); @@ -923,6 +924,7 @@ filelayout_pg_init_write(struct nfs_pageio_descriptor *pgio, 0, NFS4_MAX_UINT64, IOMODE_RW, + false, GFP_NOFS); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index ea24b0a..6b62035 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -843,6 +843,36 @@ ff_layout_choose_best_ds_for_read(struct pnfs_layout_segment *lseg, } static void +ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio, + struct nfs_page *req, + bool strict_iomode) +{ +retry_strict: + pnfs_put_lseg(pgio->pg_lseg); + pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, + req->wb_context, + 0, + NFS4_MAX_UINT64, + IOMODE_READ, + strict_iomode, + GFP_KERNEL); + if (IS_ERR(pgio->pg_lseg)) { + pgio->pg_error = PTR_ERR(pgio->pg_lseg); + pgio->pg_lseg = NULL; + } + + /* If we don't have checking, do get a IOMODE_RW + * segment, and the server wants to avoid READs + * there, then retry! + */ + if (pgio->pg_lseg && !strict_iomode && + ff_layout_avoid_read_on_rw(pgio->pg_lseg)) { + strict_iomode = true; + goto retry_strict; + } +} + +static void ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) { @@ -852,19 +882,10 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, int ds_idx; /* Use full layout for now */ - if (!pgio->pg_lseg || ff_layout_avoid_read_on_rw(pgio->pg_lseg)) { - pnfs_put_lseg(pgio->pg_lseg); - pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, - req->wb_context, - 0, - NFS4_MAX_UINT64, - IOMODE_READ, - GFP_KERNEL); - if (IS_ERR(pgio->pg_lseg)) { - pgio->pg_error = PTR_ERR(pgio->pg_lseg); - pgio->pg_lseg = NULL; - } - } + if (!pgio->pg_lseg) + ff_layout_pg_get_read(pgio, req, false); + else if (ff_layout_avoid_read_on_rw(pgio->pg_lseg)) + ff_layout_pg_get_read(pgio, req, true); /* If no lseg, fall back to read through mds */ if (pgio->pg_lseg == NULL) @@ -916,6 +937,7 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, 0, NFS4_MAX_UINT64, IOMODE_RW, + false, GFP_NOFS); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); @@ -974,6 +996,7 @@ ff_layout_pg_get_mirror_count_write(struct nfs_pageio_descriptor *pgio, 0, NFS4_MAX_UINT64, IOMODE_RW, + false, GFP_NOFS); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 8bbaeaa..376cedf 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -1321,23 +1321,28 @@ out_existing: /* * iomode matching rules: - * iomode lseg match - * ----- ----- ----- - * ANY READ true - * ANY RW true - * RW READ false - * RW RW true - * READ READ true - * READ RW true + * iomode lseg strict match + * iomode + * ----- ----- ------ ----- + * ANY READ N/A true + * ANY RW N/A true + * RW READ N/A false + * RW RW N/A true + * READ READ N/A true + * READ RW true false + * READ RW false true */ static bool pnfs_lseg_range_match(const struct pnfs_layout_range *ls_range, - const struct pnfs_layout_range *range) + const struct pnfs_layout_range *range, + bool strict_iomode) { struct pnfs_layout_range range1; if ((range->iomode == IOMODE_RW && ls_range->iomode != IOMODE_RW) || + (range->iomode != ls_range->iomode && + strict_iomode == true) || !pnfs_lseg_range_intersecting(ls_range, range)) return 0; @@ -1352,7 +1357,8 @@ pnfs_lseg_range_match(const struct pnfs_layout_range *ls_range, */ static struct pnfs_layout_segment * pnfs_find_lseg(struct pnfs_layout_hdr *lo, - struct pnfs_layout_range *range) + struct pnfs_layout_range *range, + bool strict_iomode) { struct pnfs_layout_segment *lseg, *ret = NULL; @@ -1361,7 +1367,8 @@ pnfs_find_lseg(struct pnfs_layout_hdr *lo, list_for_each_entry(lseg, &lo->plh_segs, pls_list) { if (test_bit(NFS_LSEG_VALID, &lseg->pls_flags) && !test_bit(NFS_LSEG_LAYOUTRETURN, &lseg->pls_flags) && - pnfs_lseg_range_match(&lseg->pls_range, range)) { + pnfs_lseg_range_match(&lseg->pls_range, range, + strict_iomode)) { ret = pnfs_get_lseg(lseg); break; } @@ -1478,6 +1485,7 @@ pnfs_update_layout(struct inode *ino, loff_t pos, u64 count, enum pnfs_iomode iomode, + bool strict_iomode, gfp_t gfp_flags) { struct pnfs_layout_range arg = { @@ -1539,7 +1547,7 @@ lookup_again: goto out_unlock; } - lseg = pnfs_find_lseg(lo, &arg); + lseg = pnfs_find_lseg(lo, &arg, strict_iomode); if (lseg) { trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg, PNFS_UPDATE_LAYOUT_FOUND_CACHED); @@ -1858,6 +1866,7 @@ pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *r req_offset(req), rd_size, IOMODE_READ, + false, GFP_KERNEL); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); @@ -1882,6 +1891,7 @@ pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio, req_offset(req), wb_size, IOMODE_RW, + false, GFP_NOFS); if (IS_ERR(pgio->pg_lseg)) { pgio->pg_error = PTR_ERR(pgio->pg_lseg); diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 30d94d1e..ce1b5bc 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -299,6 +299,7 @@ struct pnfs_layout_segment *pnfs_update_layout(struct inode *ino, loff_t pos, u64 count, enum pnfs_iomode iomode, + bool strict_iomode, gfp_t gfp_flags); void pnfs_clear_layoutreturn_waitbit(struct pnfs_layout_hdr *lo); -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 0/3] FF_FLAGS_NO_READ_IO 2016-05-18 21:07 [PATCH 0/3] FF_FLAGS_NO_READ_IO Tom Haynes ` (2 preceding siblings ...) 2016-05-18 21:07 ` [PATCH 3/3] pnfs: pnfs_update_layout needs to consider if strict iomode checking is on Tom Haynes @ 2016-05-24 20:20 ` Anna Schumaker 3 siblings, 0 replies; 5+ messages in thread From: Anna Schumaker @ 2016-05-24 20:20 UTC (permalink / raw) To: Tom Haynes, Trond Myklebust; +Cc: Linux NFS Mailing list Hi Tom, I'm having trouble applying these patches: Applying: nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO fatal: sha1 information is lacking or useless (fs/nfs/flexfilelayout/flexfilelayout.h). error: could not build fake ancestor Patch failed at 0001 nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO And attempting to apply using patch(1) has failed chunks. Do you have an updated version of these? Thanks, Anna On 05/18/2016 05:07 PM, Tom Haynes wrote: > FF_FLAGS_NO_READ_IO is a hint the mds can use to inform the > client that an IOMODE_RW layout segment should not be used > for READ traffic and that the client should instead request > an IOMODE_READ layout segment. This can be used to isolate > only WRITE traffic onto the IOMODE_RW layout segment. > > Tom Haynes (3): > nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO > nfs/flexfiles: Use the layout segment for reading unless it a > IOMODE_RW and reading is disabled > pnfs: pnfs_update_layout needs to consider if strict iomode checking > is on > > fs/nfs/filelayout/filelayout.c | 2 ++ > fs/nfs/flexfilelayout/flexfilelayout.c | 50 +++++++++++++++++++++++-------- > fs/nfs/flexfilelayout/flexfilelayout.h | 10 ++++++- > fs/nfs/flexfilelayout/flexfilelayoutdev.c | 6 ++++ > fs/nfs/pnfs.c | 34 +++++++++++++-------- > fs/nfs/pnfs.h | 1 + > 6 files changed, 77 insertions(+), 26 deletions(-) > ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-05-24 20:20 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-05-18 21:07 [PATCH 0/3] FF_FLAGS_NO_READ_IO Tom Haynes 2016-05-18 21:07 ` [PATCH 1/3] nfs/flexfiles: Helper function to detect FF_FLAGS_NO_READ_IO Tom Haynes 2016-05-18 21:07 ` [PATCH 2/3] nfs/flexfiles: Use the layout segment for reading unless it a IOMODE_RW and reading is disabled Tom Haynes 2016-05-18 21:07 ` [PATCH 3/3] pnfs: pnfs_update_layout needs to consider if strict iomode checking is on Tom Haynes 2016-05-24 20:20 ` [PATCH 0/3] FF_FLAGS_NO_READ_IO Anna Schumaker
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).