Re: [PATCH v2 00/19] Prepare to lift lookup out of exclusive lock for directory ops

[Al Viro <viro@zeniv.linux.org.uk>]

On Mon, Apr 27, 2026 at 01:29:33PM +1000, NeilBrown wrote:
> This patch set progresses my effort to improve concurrency of
> directory operations and specifically to allow concurrent updates
> in a given directory.
> 
> It is a selection of patches from the 53-patch set I posted in March
> which got relatively little response.  Maybe a shorter set will be more
> approachable.

Got it, will post a review tomorrow.

> This set:
>  - prepares the VFS in various ways
>  - make use of these preparations in ovl and NFS (the most challenging
>    filesystems for lookup as they do the most interesting things)
>  - make use in efivars and shmem which for different reasons need a small 
>    change that seemed worth including here.
> 
> The goal that these patch work towards is moving lookup out of i_rwsem
> on the directory - except for the actual ->lookup call.  This is itself
> a step towards allowing broad concurrency of operations in a given
> directory.
> 
> There are two particular requirements before lookup can move outside the lock:
> 1/ d_drop() mustn't be used before an operation completes: the dentry being present
>    in the dcache becomes part of the locking protocol.  This in turn requires
>    d_splice_alias() to work with hashed negative dentries.
> 2/ d_alloc_parallel() mustn't be called while i_rw_sem is held, as this would
>    result in a lock inversion.  So d_alloc_noblock and others are introduced
>    to handle the various cases.
>    In a few cases we need to drop and re-take i_rw_sem inside ->lookup.
>    As lookup might be called with a shared or exclusive lock this requires
>    a new LOOKUP_SHARED flag which is ugly but can be removed after the
>    lookup is moved out of the lock (then ->lookup will only ever be called
>    with a shared lock).
> 
> The full set of patches including these 19 and the rest to complete the
> lifting of lookup out of the exclusive lock can be found at
>    github/neilbrown/linux in branch pdirops
> 
> Significant changes since last time are:
>  - use wait_var_event for d_alloc_parallel() rather than effectively
>    duplicating that infrastructure - as suggested by Christop
>  - changes to ovl_readdir handling as discussed with Amir.
> 
> Thanks,
> NeilBrown
> 
> 
>  [PATCH v2 01/19] VFS: fix various typos in documentation for
>  [PATCH v2 02/19] VFS: enhance d_splice_alias() to handle in-lookup
>  [PATCH v2 03/19] VFS: allow d_alloc_name() to be used with ->d_hash
>  [PATCH v2 04/19] VFS: use wait_var_event for waiting in
>  [PATCH v2 05/19] VFS: introduce d_alloc_noblock()
>  [PATCH v2 06/19] VFS: add d_duplicate()
>  [PATCH v2 07/19] VFS: Add LOOKUP_SHARED flag.
>  [PATCH v2 08/19] VFS/xfs/ntfs: drop parent lock across
>  [PATCH v2 09/19] ovl: stop using lookup_one() in iterate_shared()
>  [PATCH v2 10/19] VFS/ovl: add d_alloc_noblock_return()
>  [PATCH v2 11/19] efivarfs: use d_alloc_name()
>  [PATCH v2 12/19] shmem: use d_duplicate()
>  [PATCH v2 13/19] nfs: remove d_drop()/d_alloc_parallel() from
>  [PATCH v2 14/19] nfs: use d_splice_alias() in nfs_link()
>  [PATCH v2 15/19] nfs: don't d_drop() before d_splice_alias()
>  [PATCH v2 16/19] nfs: don't d_drop() before d_splice_alias() in
>  [PATCH v2 17/19] nfs: Use d_alloc_noblock() in nfs_prime_dcache()
>  [PATCH v2 18/19] nfs: use d_alloc_noblock() in silly-rename
>  [PATCH v2 19/19] nfs: use d_duplicate()