Re: [PATCH v3 4/4] nfsd: start non-blocking writeback after adding nfsd_file to the LRU

[Jeff Layton <jlayton@kernel.org>]

On Fri, 2022-10-28 at 19:50 +0000, Chuck Lever III wrote:
> 
> > On Oct 28, 2022, at 2:57 PM, Jeff Layton <jlayton@kernel.org> wrote:
> > 
> > When a GC entry gets added to the LRU, kick off SYNC_NONE writeback
> > so that we can be ready to close it when the time comes. This should
> > help minimize delays when freeing objects reaped from the LRU.
> > 
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
> > ---
> > fs/nfsd/filecache.c | 23 +++++++++++++++++++++--
> > 1 file changed, 21 insertions(+), 2 deletions(-)
> > 
> > diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c
> > index 47cdc6129a7b..c43b6cff03e2 100644
> > --- a/fs/nfsd/filecache.c
> > +++ b/fs/nfsd/filecache.c
> > @@ -325,6 +325,20 @@ nfsd_file_fsync(struct nfsd_file *nf)
> > 		nfsd_reset_write_verifier(net_generic(nf->nf_net, nfsd_net_id));
> > }
> > 
> > +static void
> > +nfsd_file_flush(struct nfsd_file *nf)
> > +{
> > +	struct file *file = nf->nf_file;
> > +	struct address_space *mapping;
> > +
> > +	if (!file || !(file->f_mode & FMODE_WRITE))
> > +		return;
> > +
> > +	mapping = file->f_mapping;
> > +	if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
> > +		filemap_flush(mapping);
> > +}
> > +
> > static int
> > nfsd_file_check_write_error(struct nfsd_file *nf)
> > {
> > @@ -484,9 +498,14 @@ nfsd_file_put(struct nfsd_file *nf)
> > 
> > 		/* Try to add it to the LRU.  If that fails, decrement. */
> > 		if (nfsd_file_lru_add(nf)) {
> > -			/* If it's still hashed, we're done */
> > -			if (test_bit(NFSD_FILE_HASHED, &nf->nf_flags))
> > +			/*
> > +			 * If it's still hashed, we can just return now,
> > +			 * after kicking off SYNC_NONE writeback.
> > +			 */
> > +			if (test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) {
> > +				nfsd_file_flush(nf);
> > 				return;
> > +			}
> 
> nfsd_write() calls nfsd_file_put() after every nfsd_vfs_write(). In some
> cases, this new logic adds an async flush after every UNSTABLE NFSv3 WRITE.
> 
> I'll need to see performance measurements demonstrating no negative
> impact on throughput or latency of NFSv3 WRITEs with large payloads.
> 
> 

In your earlier mail, you mentioned that you wanted the writeback work
to be done in the context of nfsd threads. nfsd_file_put is how nfsd
threads put their references so this seems like the right place to do
it.

If you're concerned about calling filemap_flush too often because we
have an entry that's cycling onto and off of the LRU, then another
(maybe better) idea might be to kick off writeback when we clear the
REFERENCED flag. That would need to be done in the gc thread context, 
however.

> > 			/*
> > 			 * We're racing with unhashing, so try to remove it from
> > -- 
> > 2.37.3
> > 
> 
> --
> Chuck Lever
> 
> 
> 

-- 
Jeff Layton <jlayton@kernel.org>