Re: [PATCH v4 00/21] Readdir enhancements

David Wysochanski
2020-11-09T13:26:32Z

On Sat, Nov 7, 2020 at 9:13 AM <trondmy@kernel.org> wrote:
>
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
>
> The following patch series performs a number of cleanups on the readdir
> code.
> It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
> the caching code to ensure that we cache the entire contents of that
> 1MB call (instead of discarding the data that doesn't fit into a single
> page).
> For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
> optimises searching for cookies in the client's page cache by skipping
> over pages that contain cookie values that are not in the range we are
> searching for.
> Finally, it improves scalability when dealing with very large
> directories by turning off caching when those directories are changing,
> so as to avoid the need for a linear search on the client of the entire
> directory when looking for the first entry pointed to by the current
> file descriptor offset.
>
> v2: Fix the handling of the NFSv3/v4 directory verifier.
> v3: Optimise searching when the readdir cookies are seen to be ordered.
> v4: Optimise performance for large directories that are changing.
> Add in llseek dependency patches.
>
> Trond Myklebust (21):
> NFS: Remove unnecessary inode locking in nfs_llseek_dir()
> NFS: Remove unnecessary inode lock in nfs_fsync_dir()
> NFS: Ensure contents of struct nfs_open_dir_context are consistent
> NFS: Clean up readdir struct nfs_cache_array
> NFS: Clean up nfs_readdir_page_filler()
> NFS: Clean up directory array handling
> NFS: Don't discard readdir results
> NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
> NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
> NFS: Simplify struct nfs_cache_array_entry
> NFS: Support larger readdir buffers
> NFS: More readdir cleanups
> NFS: nfs_do_filldir() does not return a value
> NFS: Reduce readdir stack usage
> NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
> NFS: Allow the NFS generic code to pass in a verifier to readdir
> NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
> NFS: Improve handling of directory verifiers
> NFS: Optimisations for monotonically increasing readdir cookies
> NFS: Reduce number of RPC calls when doing uncached readdir
> NFS: Do uncached readdir when we're seeking a cookie in an empty page
> cache
>

Thanks for these last couple patches to handle changing directories /
uncached scenario better. I'm testing with this set now and should
have results later today.