From: Trond Myklebust <trondmy@hammerspace.com>
To: "schumaker.anna@gmail.com" <schumaker.anna@gmail.com>,
"trondmy@kernel.org" <trondmy@kernel.org>
Cc: "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v7 10/21] NFS: Reduce use of uncached readdir
Date: Fri, 25 Feb 2022 04:07:16 +0000 [thread overview]
Message-ID: <189fe459d0ab3ed851790c2cdd86a9c71d3bedee.camel@hammerspace.com> (raw)
In-Reply-To: <CAFX2JfmLvuh2KROa_tP=_RBXbUg68E--JHutgh1xKPV9en5sfg@mail.gmail.com>
On Thu, 2022-02-24 at 11:55 -0500, Anna Schumaker wrote:
> Hi Trond,
>
> On Wed, Feb 23, 2022 at 8:25 PM <trondmy@kernel.org> wrote:
> >
> > From: Trond Myklebust <trond.myklebust@hammerspace.com>
> >
> > When reading a very large directory, we want to try to keep the
> > page
> > cache up to date if doing so is inexpensive. With the change to
> > allow
> > readdir to continue reading even when the cache is incomplete, we
> > no
> > longer need to fall back to uncached readdir in order to scale to
> > large
> > directories.
> >
> > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
>
> As of this patch, cthon tests are passing again.
>
I'm going to push out a v8 patchset. I'm just waiting for a few more
comments, etc.
Anyhow, I wonder if the fact that we're not initialising the verifier
in nfs_opendir() is part of the problem here. I've added a patch to do
this.
I also found an issue with the nfs2/nfs3_decode_dirent() return values
(and have a fix for that), however I'm assuming that is not the problem
you're seeing, since you were reporting issues with NFSv4, which is
unaffected.
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com
next prev parent reply other threads:[~2022-02-25 4:07 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-23 21:12 [PATCH v7 00/21] Readdir improvements trondmy
2022-02-23 21:12 ` [PATCH v7 01/21] NFS: constify nfs_server_capable() and nfs_have_writebacks() trondmy
2022-02-23 21:12 ` [PATCH v7 02/21] NFS: Trace lookup revalidation failure trondmy
2022-02-23 21:12 ` [PATCH v7 03/21] NFS: Use kzalloc() to avoid initialising the nfs_open_dir_context trondmy
2022-02-23 21:12 ` [PATCH v7 04/21] NFS: Calculate page offsets algorithmically trondmy
2022-02-23 21:12 ` [PATCH v7 05/21] NFS: Store the change attribute in the directory page cache trondmy
2022-02-23 21:12 ` [PATCH v7 06/21] NFS: If the cookie verifier changes, we must invalidate the " trondmy
2022-02-23 21:12 ` [PATCH v7 07/21] NFS: Don't re-read the entire page cache to find the next cookie trondmy
2022-02-23 21:12 ` [PATCH v7 08/21] NFS: Adjust the amount of readahead performed by NFS readdir trondmy
2022-02-23 21:12 ` [PATCH v7 09/21] NFS: Simplify nfs_readdir_xdr_to_array() trondmy
2022-02-23 21:12 ` [PATCH v7 10/21] NFS: Reduce use of uncached readdir trondmy
2022-02-23 21:12 ` [PATCH v7 11/21] NFS: Improve heuristic for readdirplus trondmy
2022-02-23 21:12 ` [PATCH v7 12/21] NFS: Don't ask for readdirplus unless it can help nfs_getattr() trondmy
2022-02-23 21:12 ` [PATCH v7 13/21] NFSv4: Ask for a full XDR buffer of readdir goodness trondmy
2022-02-23 21:12 ` [PATCH v7 14/21] NFS: Readdirplus can't help lookup for case insensitive filesystems trondmy
2022-02-23 21:12 ` [PATCH v7 15/21] NFS: Don't request readdirplus when revalidation was forced trondmy
2022-02-23 21:13 ` [PATCH v7 16/21] NFS: Add basic readdir tracing trondmy
2022-02-23 21:13 ` [PATCH v7 17/21] NFS: Trace effects of readdirplus on the dcache trondmy
2022-02-23 21:13 ` [PATCH v7 18/21] NFS: Trace effects of the readdirplus heuristic trondmy
2022-02-23 21:13 ` [PATCH v7 19/21] NFS: Convert readdir page cache to use a cookie based index trondmy
2022-02-23 21:13 ` [PATCH v7 20/21] NFS: Fix up forced readdirplus trondmy
2022-02-23 21:13 ` [PATCH v7 21/21] NFS: Remove unnecessary cache invalidations for directories trondmy
2022-02-24 17:31 ` [PATCH v7 19/21] NFS: Convert readdir page cache to use a cookie based index Benjamin Coddington
2022-02-25 2:33 ` Trond Myklebust
2022-02-25 3:17 ` NeilBrown
2022-02-25 4:25 ` Trond Myklebust
2022-02-25 12:33 ` Benjamin Coddington
2022-02-25 13:11 ` Trond Myklebust
2022-02-24 15:53 ` [PATCH v7 16/21] NFS: Add basic readdir tracing Benjamin Coddington
2022-02-25 2:35 ` Trond Myklebust
2022-02-24 16:55 ` [PATCH v7 10/21] NFS: Reduce use of uncached readdir Anna Schumaker
2022-02-25 4:07 ` Trond Myklebust [this message]
2022-02-24 16:30 ` [PATCH v7 08/21] NFS: Adjust the amount of readahead performed by NFS readdir Anna Schumaker
2022-02-24 16:18 ` [PATCH v7 06/21] NFS: If the cookie verifier changes, we must invalidate the page cache Anna Schumaker
2022-02-24 14:53 ` [PATCH v7 05/21] NFS: Store the change attribute in the directory " Benjamin Coddington
2022-02-25 2:26 ` Trond Myklebust
2022-02-25 3:51 ` Trond Myklebust
2022-02-25 11:38 ` Benjamin Coddington
2022-02-25 13:10 ` Trond Myklebust
2022-02-25 13:26 ` Trond Myklebust
2022-02-25 14:44 ` Benjamin Coddington
2022-02-25 15:18 ` Trond Myklebust
2022-02-25 15:34 ` Benjamin Coddington
2022-02-25 20:23 ` Benjamin Coddington
2022-02-25 20:28 ` Benjamin Coddington
2022-02-25 20:41 ` Trond Myklebust
2022-02-25 22:04 ` Benjamin Coddington
2022-02-25 22:29 ` Trond Myklebust
2022-02-24 14:15 ` [PATCH v7 04/21] NFS: Calculate page offsets algorithmically Benjamin Coddington
2022-02-25 2:11 ` Trond Myklebust
2022-02-25 11:28 ` Benjamin Coddington
2022-02-25 12:44 ` Trond Myklebust
2022-02-24 14:14 ` [PATCH v7 02/21] NFS: Trace lookup revalidation failure Benjamin Coddington
2022-02-25 2:09 ` Trond Myklebust
2022-02-24 12:25 ` [PATCH v7 00/21] Readdir improvements David Wysochanski
2022-02-25 4:00 ` Trond Myklebust
2022-02-24 15:07 ` David Wysochanski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=189fe459d0ab3ed851790c2cdd86a9c71d3bedee.camel@hammerspace.com \
--to=trondmy@hammerspace.com \
--cc=linux-nfs@vger.kernel.org \
--cc=schumaker.anna@gmail.com \
--cc=trondmy@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox