[PATCH 0/9] fs: unmount without a grace period when nothing else holds the mounts

[PATCH 0/9] fs: unmount without a grace period when nothing else holds the mounts

[Christian Brauner]

namespace_unlock() waits for an expedited grace period every time it
puts a batch of unmounted mounts. The RCU delay was added for lockless
path walk in 2013. The fast path of mntput_no_expire() started relying
on it in 2018 and Neil made it expedited the same year because 5000
umounts took 100 seconds otherwise.

An expedited grace period IPIs every CPU. On a 64 vcpu VM that's 20 to
30 microseconds per namespace_unlock(), two thirds of tearing down a
single detached mount. On PREEMPT_RT or with rcupdate.rcu_normal=1 it's
a normal grace period of 6 milliseconds and podman run --rm pays 14 of
them per container.

People have tried to get rid of it before. There was a series to hand
lazy umounts to a workqueue. It was dropped from 6.16 again after LTP
found loop devices busy right after umount -l and mkfs corrupted an ext4
image that a kworker was still tearing down. Returning from umount(2)
before the filesystem is gone is a uapi change, lazy or not. I also
tried sampling the grace period state in umount_tree() and polling it in
namespace_unlock() but the two are only microseconds apart. No gain.

So let's elide the grace period when it isn't needed. Since commit
7eb84d54fac5 ("fs: don't let a migrating task hide its reference from
do_umount()") the per-cpu count is split into gets and puts and the
puts are summed first. A count of exactly one under mount_lock after
the mount has left its namespace can only be the mount's own reference.

A holder's get is counted whenever its put is. A walker's increment is
either counted or the walker sees the seqcount change and backs out
under mount_lock where it finds MNT_DOOMED. And nobody can find the
mount anymore. So we drop the reference and finish the mount off right
there. Everything else waits for the grace period as before. The fast
paths of mntput_no_expire() and __legitimize_mnt() don't change and
nothing changes for userspace.

For that the caller's umount(2) reference has to be dropped under the
mount_lock hold that unmounted the mount so the peek finds the mount's
own reference alone. The peek also has to be the last put.

Taking the grace period away exposes filesystems that unknowingly relied
on it for whatever they touch in rcu path walk (but rest assured that
the bug is reproducible outside of umount(2)). The first patch fixes
overlayfs. ext4, ntfs3 and the unicode map of casefolded filesystems are
broken without this series already because a chdir() out of a lazily
unmounted directory drops the last reference without a grace period too.

The ipc namespace is now freed with kfree_rcu() because free_ipc()'s
grace period also covered the lockless lookups of the namespace tree.

Dropping the explicit smp_mb()s that mnt_get_count() made redundant is
already in vfs.fixes (cf. [2]). This series doesn't depend on it.

A reproducer can't really exercise this so the formal side is heavier
than usual:

- a TLA+ model of a two-mount batch in 62 configurations where every
  mutation found its bug
- 20 LKMM litmus tests, 18 of them added under
  Documentation/litmus-tests/fs/. Every shape the series relies on is
  Never and every barrier mutation is Sometimes.
- the same tests under the AArch64 and POWER models and klitmus7 on
  x86, except the three late walker tests
- a Dartagnan harness of the C that agrees with herd7
- TLA+ trace validation of ftrace traces from the selftest and stress
  runs

All of it is in:

    https://github.com/brauner/linux-formal
    kernel/mount/gp-on-demand-proposal/

umount-bench in kops/s, median of 15 runs on a 64 vcpu VM, base against
the series, with 1 and 8 workers:

    mode               base            series            delta
    mountumount        26.1 /  75.4     57.1 / 125.5    +118% / +66%
    dissolveone        38.2 / 108.5    170.6 / 173.4    +347% / +60%
    dissolveflat       15.5             22.4             +45%
    dissolvechain      15.7             22.4             +43%
    nsdeath             6.15             7.10            +15%
    detachlocked        2.72             3.22            +18%
    lazytree            2.93             3.11             +6%
    held modes         level, +1 to +4%

And scaling with the vcpu count, single worker, median of 9:

    vcpus            1     8    16    32    64   128   256   512
    mountumount    +30%  +44%  +63%  +92%  +83%  +93%  +71%  +75%
    dissolveone    +86% +130% +183% +293% +224% +242% +155% +149%

Above 64 vcpus the per-cpu sums start to dominate a namespace death so
the multi-mount modes only gain 10 to 27% there.

With rcupdate.rcu_normal=1, the RT case, podman run --rm in an 8 vcpu
VM goes from 124 to 93 milliseconds and the 14 grace periods per
container become 9. With expedited grace periods it doesn't move.

[1]: https://lore.kernel.org/r/20261009-work-fixes-rcu-v1-0-7446d2d335e6@kernel.org
[2]: https://lore.kernel.org/r/20261011-work-mount-fixes-5-v1-0-a9b9a8ecf565@kernel.org

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
Christian Brauner (9):
      ovl: free the ovl_fs after a grace period
      fs: split the final put out of mntput_no_expire_slowpath()
      fs: drop the caller's umount(2) reference under mount_lock
      fs: unmount without a grace period when nothing else holds the mounts
      fs: port kern_unmount{_array}() to mntput_unmounted()
      selftests/filesystems: check that unheld mounts are gone after umount
      Documentation/litmus-tests: add the mount reference count tests
      ipc: free the namespace after a grace period
      ipc: unmount the mqueue mount with kern_unmount()

 Documentation/litmus-tests/README                  |  95 ++++++++
 .../fs/mnt-gp-sees-fastpath-put-nogp.litmus        |  75 ++++++
 .../fs/mnt-gp-sees-fastpath-put-sync-rcu.litmus    |  73 ++++++
 .../fs/mnt-gp-sees-fastpath-put.litmus             |  79 +++++++
 .../fs/mnt-holder-get-before-put-getsfirst.litmus  |  63 +++++
 .../fs/mnt-holder-get-before-put-nomb.litmus       |  62 +++++
 .../fs/mnt-holder-get-before-put-nowmb.litmus      |  62 +++++
 .../fs/mnt-holder-get-before-put.litmus            |  73 ++++++
 .../mnt-kern-unmount-gp-sees-fastpath-put.litmus   |  70 ++++++
 .../mnt-kern-unmount-holder-get-before-put.litmus  |  60 +++++
 .../fs/mnt-sum-misses-fastpath-put.litmus          |  69 ++++++
 .../fs/mnt-walker-bail-sees-doomed-nolock.litmus   |  89 +++++++
 .../fs/mnt-walker-bail-sees-doomed.litmus          |  95 ++++++++
 .../mnt-walker-get-vs-seqcount-no-outer-mb.litmus  |  92 ++++++++
 .../fs/mnt-walker-get-vs-seqcount-nomb.litmus      |  92 ++++++++
 .../fs/mnt-walker-get-vs-seqcount.litmus           | 102 ++++++++
 .../fs/mnt-walker-late-sees-new-pwd-nomb.litmus    |  89 +++++++
 .../fs/mnt-walker-late-sees-new-pwd-nowmb.litmus   |  88 +++++++
 .../fs/mnt-walker-late-sees-new-pwd.litmus         | 103 ++++++++
 fs/namespace.c                                     | 219 +++++++++++------
 fs/overlayfs/ovl_entry.h                           |   2 +
 fs/overlayfs/params.c                              |  29 ++-
 include/linux/mount.h                              |   1 -
 ipc/namespace.c                                    |  17 +-
 .../selftests/filesystems/mount_cycle/.gitignore   |   1 +
 .../selftests/filesystems/mount_cycle/Makefile     |   2 +-
 .../filesystems/mount_cycle/unheld_umount_test.c   | 262 +++++++++++++++++++++
 27 files changed, 1974 insertions(+), 90 deletions(-)
---
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
change-id: 20261009-work-mount-gp_on_demand-proposal-77800244d92a

[PATCH 1/9] ovl: free the ovl_fs after a grace period

[Christian Brauner]

During rcu lookup ovl_permission() accesses the underyling inode through
ovl_i_path_real(). It accesses lowerpath->layer->mnt out of
ofs->layers. It also accesses ofs->creator_cred to switch creds.
ovl_free_fs() ends up freeing ofs->layers, dropping the creds and
freeing ofs itself after kern_unmount_array().

Commit a535116d8033 ("ovl: make use of ->layers safe in rcu pathwalk")
made that safe by piggy-backing onto the grace period in
kern_unmount_array(). That ties overlayfs to an implementation detail
of kern_unmount_array() and we want to make that grace period optional
in follow-up patches.

So free ofs and everything rcu lookup needs from an rcu callback.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/overlayfs/ovl_entry.h |  2 ++
 fs/overlayfs/params.c    | 29 +++++++++++++++++++++--------
 2 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 80cad4ea96a3..8ce962ed158e 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -92,6 +92,8 @@ struct ovl_fs {
 	/* r/o snapshot of upperdir sb's only taken on volatile mounts */
 	errseq_t errseq;
 	bool casefold;
+	/* ->layers, ->fs and ->creator_cred are read by rcu pathwalks */
+	struct rcu_head rcu;
 };
 
 /* Number of lower layers, not including data-only layers */
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index c93fcaa45d4a..36fa3a3274a6 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -837,6 +837,26 @@ int ovl_init_fs_context(struct fs_context *fc)
 
 }
 
+/*
+ * ovl_permission() and ovl_dentry_revalidate_common() run in rcu pathwalk
+ * mode and read ->layers[].mnt, ->fs and ->creator_cred through the
+ * superblock, so those go only after a grace period. The mounts themselves
+ * are rcu-freed by kern_unmount_array().
+ */
+static void ovl_free_fs_rcu(struct rcu_head *head)
+{
+	struct ovl_fs *ofs = container_of(head, struct ovl_fs, rcu);
+
+	kfree(ofs->layers);
+	kfree(ofs->fs);
+	kfree(ofs->config.lowerdirs);
+	kfree(ofs->config.upperdir);
+	kfree(ofs->config.workdir);
+	if (ofs->creator_cred)
+		put_cred(ofs->creator_cred);
+	kfree(ofs);
+}
+
 void ovl_free_fs(struct ovl_fs *ofs)
 {
 	struct vfsmount **mounts;
@@ -860,17 +880,10 @@ void ovl_free_fs(struct ovl_fs *ofs)
 		mounts[i] = ofs->layers[i].mnt;
 	}
 	kern_unmount_array(mounts, ofs->numlayer);
-	kfree(ofs->layers);
 	for (i = 0; i < ofs->numfs; i++)
 		free_anon_bdev(ofs->fs[i].pseudo_dev);
-	kfree(ofs->fs);
 
-	kfree(ofs->config.lowerdirs);
-	kfree(ofs->config.upperdir);
-	kfree(ofs->config.workdir);
-	if (ofs->creator_cred)
-		put_cred(ofs->creator_cred);
-	kfree(ofs);
+	call_rcu(&ofs->rcu, ovl_free_fs_rcu);
 }
 
 int ovl_fs_params_verify(const struct ovl_fs_context *ctx,

-- 
2.53.0

[PATCH 2/9] fs: split the final put out of mntput_no_expire_slowpath()

[Christian Brauner]

Split mntput_no_expire_slowpath() into mntput_final_locked() and
mntput_queue_cleanup() so we can reuse them.

No functional changes.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/namespace.c | 64 ++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 40 insertions(+), 24 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index 3ed3766f3a18..eacf181298d9 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1410,10 +1410,47 @@ static void delayed_mntput(struct work_struct *unused)
 }
 static DECLARE_DELAYED_WORK(delayed_mntput_work, delayed_mntput);
 
-static void noinline mntput_no_expire_slowpath(struct mount *mnt)
+/*
+ * The count reached zero under mount_lock so we doom the mount and
+ * cleanup everything.
+ */
+static void mntput_final_locked(struct mount *mnt, struct list_head *shrink)
 {
 	struct mnt_cover *cover;
 	struct hlist_node *n;
+
+	VFS_WARN_ON_ONCE(mnt->mnt.mnt_flags & MNT_DOOMED);
+	mnt->mnt.mnt_flags |= MNT_DOOMED;
+
+	mnt_del_instance(mnt);
+	if (unlikely(!list_empty(&mnt->mnt_expire)))
+		list_del(&mnt->mnt_expire);
+
+	/* nothing stays attached to an unmounted mount */
+	VFS_WARN_ON_ONCE(!list_empty(&mnt->mnt_mounts));
+	hlist_for_each_entry_safe(cover, n, &mnt->mnt_covers, node)
+		drop_cover(cover, shrink);
+}
+
+/* cleanup_mnt() sleeps: from task work, or the workqueue for kernel threads */
+static void mntput_queue_cleanup(struct mount *mnt)
+{
+	if (likely(!(mnt->mnt.mnt_flags & MNT_INTERNAL))) {
+		struct task_struct *task = current;
+		if (likely(!(task->flags & PF_KTHREAD))) {
+			init_task_work(&mnt->mnt_rcu, __cleanup_mnt);
+			if (!task_work_add(task, &mnt->mnt_rcu, TWA_RESUME))
+				return;
+		}
+		if (llist_add(&mnt->mnt_llist, &delayed_mntput_list))
+			schedule_delayed_work(&delayed_mntput_work, 1);
+		return;
+	}
+	cleanup_mnt(mnt);
+}
+
+static void noinline mntput_no_expire_slowpath(struct mount *mnt)
+{
 	LIST_HEAD(list);
 	int count;
 
@@ -1437,32 +1474,11 @@ static void noinline mntput_no_expire_slowpath(struct mount *mnt)
 		unlock_mount_hash();
 		return;
 	}
-	mnt->mnt.mnt_flags |= MNT_DOOMED;
+	mntput_final_locked(mnt, &list);
 	rcu_read_unlock();
-
-	mnt_del_instance(mnt);
-	if (unlikely(!list_empty(&mnt->mnt_expire)))
-		list_del(&mnt->mnt_expire);
-
-	/* nothing stays attached to an unmounted mount */
-	VFS_WARN_ON_ONCE(!list_empty(&mnt->mnt_mounts));
-	hlist_for_each_entry_safe(cover, n, &mnt->mnt_covers, node)
-		drop_cover(cover, &list);
 	unlock_mount_hash();
 	shrink_dentry_list(&list);
-
-	if (likely(!(mnt->mnt.mnt_flags & MNT_INTERNAL))) {
-		struct task_struct *task = current;
-		if (likely(!(task->flags & PF_KTHREAD))) {
-			init_task_work(&mnt->mnt_rcu, __cleanup_mnt);
-			if (!task_work_add(task, &mnt->mnt_rcu, TWA_RESUME))
-				return;
-		}
-		if (llist_add(&mnt->mnt_llist, &delayed_mntput_list))
-			schedule_delayed_work(&delayed_mntput_work, 1);
-		return;
-	}
-	cleanup_mnt(mnt);
+	mntput_queue_cleanup(mnt);
 }
 
 static void mntput_no_expire(struct mount *mnt)

-- 
2.53.0

[PATCH 3/9] fs: drop the caller's umount(2) reference under mount_lock

[Christian Brauner]

The mount umount(2) was called on has at least two references after
umount_tree() has finished:

(1) the caller's path reference
(2) the mounts own reference

Currently path_umount() drops (1) after do_umount() has returned and
namespace_unlock() has already put (2). Instead, let do_umount() take
over the caller's reference and drop it under mount_lock right after
umount_tree(). In the common case this brings the count down to the
mount's own reference and namespace_unlock() can take advantage of that
in follow-up patches. If do_umount() fails the reference is dropped
after the locks as before.

While at it, convert do_umount() to guards.

No functional changes.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/namespace.c | 43 +++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 20 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index eacf181298d9..8c6673bd8c03 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1502,6 +1502,8 @@ static void mntput_no_expire(struct mount *mnt)
 	mntput_no_expire_slowpath(mnt);
 }
 
+DEFINE_FREE(mntput_no_expire, struct mount *, if (_T) mntput_no_expire(_T))
+
 void mntput(struct vfsmount *mnt)
 {
 	if (mnt) {
@@ -1935,8 +1937,10 @@ static int do_umount_root(struct super_block *sb)
 	return ret;
 }
 
+/* Unmount @mnt, dropping the reference the caller holds to it on every path. */
 static int do_umount(struct mount *mnt, int flags)
 {
+	struct mount *ref __free(mntput_no_expire) = mnt;
 	struct super_block *sb = mnt->mnt.mnt_sb;
 	int retval;
 
@@ -2003,37 +2007,36 @@ static int do_umount(struct mount *mnt, int flags)
 		return do_umount_root(sb);
 	}
 
-	namespace_lock();
-	lock_mount_hash();
+	guard(namespace_excl)();
+	guard(mount_writer)();
 
 	/* Repeat the earlier racy checks, now that we are holding the locks */
-	retval = -EINVAL;
 	if (!check_mnt(mnt))
-		goto out;
+		return -EINVAL;
 
 	if (mnt->mnt.mnt_flags & MNT_LOCKED)
-		goto out;
+		return -EINVAL;
 
 	if (!mnt_has_parent(mnt)) /* not the absolute root */
-		goto out;
+		return -EINVAL;
 
 	event++;
 	if (flags & MNT_DETACH) {
 		umount_tree(mnt, UMOUNT_PROPAGATE);
-		retval = 0;
 	} else {
 		smp_mb(); // paired with __legitimize_mnt()
 		shrink_submounts(mnt);
-		retval = -EBUSY;
-		if (!propagate_mount_busy(mnt, 2)) {
-			umount_tree(mnt, UMOUNT_PROPAGATE|UMOUNT_SYNC);
-			retval = 0;
-		}
+		if (propagate_mount_busy(mnt, 2))
+			return -EBUSY;
+		umount_tree(mnt, UMOUNT_PROPAGATE|UMOUNT_SYNC);
 	}
-out:
-	unlock_mount_hash();
-	namespace_unlock();
-	return retval;
+	/*
+	 * By definition the caller's reference isn't the last
+	 * one, so drop it.
+	 */
+	mnt_dec_count(mnt);
+	retain_and_null_ptr(ref);
+	return 0;
 }
 
 /*
@@ -2117,12 +2120,12 @@ int path_umount(const struct path *path, int flags)
 	int ret;
 
 	ret = can_umount(path, flags);
-	if (!ret)
-		ret = do_umount(mnt, flags);
-
 	/* we mustn't call path_put() as that would clear mnt_expiry_mark */
 	dput(path->dentry);
-	mntput_no_expire(mnt);
+	if (ret)
+		mntput_no_expire(mnt);
+	else
+		ret = do_umount(mnt, flags);
 	return ret;
 }
 

-- 
2.53.0

[PATCH 4/9] fs: unmount without a grace period when nothing else holds the mounts

[Christian Brauner]

Currently namespace_unlock() always waits for an RCU grace period
before it drops the references of the mounts that umount_tree()
unmounted. The grace period is needed for two reasons:

(1) mntput_no_expire() decrements the count without mount_lock as long
    as it sees a non-NULL ->mnt_ns under rcu_read_lock(). That only
    works because the mount's own reference isn't dropped until an RCU
    delay after ->mnt_ns has been cleared.

(2) __legitimize_mnt() speculatively bumps the count of a mount during
    rcu lookup and drops it again under mount_lock when it realizes the
    seqcount has changed or the mount is doomed.

Without the grace period a decrement or an increment of an rcu lookup
could land after the final mount put has summed up the counters and end
up leaking the mount. Or the task in rcu lookup would end up shutting
the filesystem down instead of task which called umount(2) causing
spurious EBUSY errors.

The grace period is rather expensive though. Neil switched it to
synchronize_rcu_expedited() in commit 22cb7405fada ("VFS: use
synchronize_rcu_expedited() in namespace_unlock()") because 5000
unmounts took 100 seconds with normal grace periods. But an expedited
grace period still IPIs every CPU. On a reasonably beefy 64 vcpu VM
that's 20 to 31 microseconds for every namespace_unlock(). That's two
thirds of the teardown of a detached single mount and half of a mount
and unmount pair.

We've tried a few things to get rid of it. For example, handing a lazy
unmount to a workqueue after the grace period (cf. [1]). But that
changes uapi. Today every mount that nobody else references is torn
down before umount(2) returns, lazy or not, because the puts are done
from the calling task. The deferred variant broke LTP which reuses its
loop devices right after a lazy unmount, and mkfs corrupted an ext4
image while the filesystem was still being torn down from the
workqueue. I also tried sampling the grace period state in
umount_tree() and polling it in namespace_unlock() but that didn't help
as the two are only microseconds apart.

But we can elide the grace period in a lot of cases. In commit
7eb84d54fac5 ("fs: don't let a migrating task hide its reference from
do_umount()") I split the per-cpu reference count into gets and puts
and the puts are summed before the gets. So the increment from rcu
lookup in __legitimize_mnt() is either visible to the sum or rcu lookup
sees the seqcount bump of the unmount and drops its reference under
mount_lock.

IOW, if the sum under mount_lock comes out as one after the mount has
left its namespace then it's the mount's own reference. No holder can
be on the fast path of mntput_no_expire() with a reference to it, no
rcu lookup holds a reference to it, and nobody can find it anymore. So
it is safe to drop that reference and put the mount without waiting for
a grace period. Mounts that are still held wait for the grace period as
before.

The fast paths of mntput_no_expire() and __legitimize_mnt() don't need
to be changed and nothing changes for path lookup. Nothing changes for
userspace either.

The unheld mounts of a batch are finished under mount_lock and the held
ones share a single grace period. Finishing all of them under one
mount_lock hold took a namespace death from 62.9 to 49.9 us in
namespace_unlock() and gave the 8 worker variants of the dissolve modes
another 4 to 7%. But every peek sums up the counters of all possible
cpus and path walks spin on mount_lock while it's held. So we finish at
most 16 mounts per hold.

umount-bench, kops/s, median of 15 runs on a 64 vcpu VM, base against
the full series, 1 and 8 workers:

    mode                  base            full            delta
    mountumount 1w / 8w   26.1 / 75.4     57.1 / 125.5    +118% / +66%
    dissolveone 1w / 8w   38.2 / 108.5    170.6 / 173.4   +347% / +60%
    dissolveflat 1w       15.5            22.4            +45%
    dissolvechain 1w      15.7            22.4            +43%
    nsdeath 1w            6.15            7.10            +15%
    detachlocked 1w       2.72            3.22            +18%
    lazytree 1w           2.93            3.11            +6%
    held modes            14.8-17.4       +1 to +4%       level

For a namespace death the synchronize_rcu_expedited() (30.9 us) is gone:

- namespace_unlock() goes from 93 to 61 us
- the slow path's 54 us are replaced by mntput_final_locked() at 5.8 us

Propagating a tree of 128 mounts into 1000 mount namespaces, median of
14 runs:

- mount and umount propagation stay level within 1%
- releasing the namespaces is 5-10% faster

Mounting, statting and unmounting in a loop with 8 threads for 20s:

- overlay is 82-87% faster (292k -> 528-544k ops)
- casefolded tmpfs is 24-26% faster (433k -> 539-545k ops)

The gain grows with the number of cpus the expedited grace period has to
IPI. Single worker, base against this series, median of 9:

  vcpus            1     8    16    32    64   128   256   512
  mountumount    +30%  +44%  +63%  +92%  +83%  +93%  +71%  +75%
  dissolveone    +86% +130% +183% +293% +224% +242% +155% +149%

Above 64 vcpus the per-cpu sums start to dominate a namespace death. At
512 vcpus 521 of the 676 us it spends in namespace_unlock() are the
sixty mnt_get_count() sums and 118 us are the grace period. So the
multi-mount modes only gain 10 to 27% there. Held mounts stay within a
few percent everywhere because they still wait for their grace period.

[1]: https://lore.kernel.org/all/20250408210350.749901-12-echanude@redhat.com

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/namespace.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 79 insertions(+), 13 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index 8c6673bd8c03..435e5f6db880 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1489,10 +1489,12 @@ static void mntput_no_expire(struct mount *mnt)
 		 * Since we don't do lock_mount_hash() here,
 		 * ->mnt_ns can change under us.  However, if it's
 		 * non-NULL, then there's a reference that won't
-		 * be dropped until after an RCU delay done after
-		 * turning ->mnt_ns NULL.  So if we observe it
-		 * non-NULL under rcu_read_lock(), the reference
-		 * we are dropping is not the final one.
+		 * be dropped until after turning ->mnt_ns NULL and
+		 * either an RCU delay or a look under mount_lock
+		 * that found it to be the only one left, which it
+		 * isn't while ours is counted (mntput_unheld()).
+		 * So if we observe it non-NULL under rcu_read_lock(),
+		 * the reference we are dropping is not the final one.
 		 */
 		smp_wmb();	/* pairs with the smp_mb() in mnt_get_count() */
 		mnt_dec_count(mnt);
@@ -1504,6 +1506,75 @@ static void mntput_no_expire(struct mount *mnt)
 
 DEFINE_FREE(mntput_no_expire, struct mount *, if (_T) mntput_no_expire(_T))
 
+/*
+ * Drop the own reference of a mount that has left its namespace without
+ * waiting for a grace period. mnt_get_count() is biased towards gets,
+ * i.e., it overcounts rather than undercounts. So if the count under
+ * mount_lock is one we know its ours and we're going to drop it.
+ *
+ * Called under mount_lock, the cleanup of the mount is left to the caller.
+ * Returns false if somebody else still references the mount.
+ */
+static bool mntput_unheld(struct mount *mnt, struct list_head *shrink)
+{
+	VFS_BUG_ON(mnt->mnt_ns);
+	if (mnt_get_count(mnt) != 1)
+		return false;
+	mnt_dec_count(mnt);
+	mntput_final_locked(mnt, shrink);
+	return true;
+}
+
+/* peeks per mount_lock hold, pathwalks spin while it is held */
+#define MNTPUT_UNMOUNTED_BATCH 16
+
+/*
+ * Put the mounts umount_tree() collected. Ideally, every mount just has a
+ * single reference count which means that we can elide the grace period.
+ * We batch mntput_unheld() as mnt_get_count() needs to sum over every cpu.
+ * On big machines it's not that fast. If there are mounts with held references
+ * we need the grace period.
+ */
+static void mntput_unmounted(struct hlist_head *head)
+{
+	struct hlist_node *p;
+	struct mount *m;
+	HLIST_HEAD(held);
+
+	while (!hlist_empty(head)) {
+		unsigned int n = 0;
+		HLIST_HEAD(unheld);
+		LIST_HEAD(shrink);
+
+		lock_mount_hash();
+		hlist_for_each_entry_safe(m, p, head, mnt_umount) {
+			hlist_del(&m->mnt_umount);
+			if (mntput_unheld(m, &shrink))
+				hlist_add_head(&m->mnt_umount, &unheld);
+			else
+				hlist_add_head(&m->mnt_umount, &held);
+			if (++n == MNTPUT_UNMOUNTED_BATCH)
+				break;
+		}
+		unlock_mount_hash();
+		shrink_dentry_list(&shrink);
+
+		hlist_for_each_entry_safe(m, p, &unheld, mnt_umount) {
+			hlist_del(&m->mnt_umount);
+			mntput_queue_cleanup(m);
+		}
+	}
+
+	if (hlist_empty(&held))
+		return;
+
+	synchronize_rcu_expedited();
+	hlist_for_each_entry_safe(m, p, &held, mnt_umount) {
+		hlist_del(&m->mnt_umount);
+		mntput(&m->mnt);
+	}
+}
+
 void mntput(struct vfsmount *mnt)
 {
 	if (mnt) {
@@ -1527,7 +1598,9 @@ EXPORT_SYMBOL(mntget);
 /*
  * Make a mount point inaccessible to new lookups.
  * Because there may still be current users, the caller MUST WAIT
- * for an RCU grace period before destroying the mount point.
+ * for an RCU grace period before destroying the mount point, unless
+ * it finds under mount_lock that it holds the only reference left,
+ * see mntput_unheld().
  */
 void mnt_make_shortterm(struct vfsmount *mnt)
 {
@@ -1782,8 +1855,6 @@ static void free_mnt_ns(struct mnt_namespace *);
 static void namespace_unlock(void)
 {
 	struct hlist_head head;
-	struct hlist_node *p;
-	struct mount *m;
 	struct mnt_namespace *ns = emptied_ns;
 	LIST_HEAD(list);
 
@@ -1814,12 +1885,7 @@ static void namespace_unlock(void)
 	if (likely(hlist_empty(&head)))
 		return;
 
-	synchronize_rcu_expedited();
-
-	hlist_for_each_entry_safe(m, p, &head, mnt_umount) {
-		hlist_del(&m->mnt_umount);
-		mntput(&m->mnt);
-	}
+	mntput_unmounted(&head);
 }
 
 static inline void namespace_lock(void)

-- 
2.53.0

[PATCH 5/9] fs: port kern_unmount{_array}() to mntput_unmounted()

[Christian Brauner]

kern_unmount() and kern_unmount_array() follow the same protocol as
namespace_unlock(). They clear ->mnt_ns, wait for a grace period and
then put the mounts. So port them to mntput_unmounted() as well and let
kern_unmount() go through kern_unmount_array(). A mount that nobody
else holds is now put without waiting for a grace period.

Note that a mount that is still held now waits for an expedited grace
period. kern_unmount() used to wait for a normal one. Callers of
kern_unmount_array() may pass NULL entries so skip them explicitly.

Overlayfs unmounts its layers through kern_unmount_array() on every
umount. With the previous patch and this one mounting, statting and
unmounting an overlay in a loop (8 threads, 20 s) goes from 292k to
528k-544k operations.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/namespace.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index 435e5f6db880..af36cce922a2 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -6600,23 +6600,23 @@ EXPORT_SYMBOL_GPL(kern_mount);
 void kern_unmount(struct vfsmount *mnt)
 {
 	/* release long term mount so mount point can be released */
-	if (!IS_ERR(mnt)) {
-		mnt_make_shortterm(mnt);
-		synchronize_rcu();	/* yecchhh... */
-		mntput(mnt);
-	}
+	if (!IS_ERR(mnt))
+		kern_unmount_array(&mnt, 1);
 }
 EXPORT_SYMBOL(kern_unmount);
 
 void kern_unmount_array(struct vfsmount *mnt[], unsigned int num)
 {
+	HLIST_HEAD(head);
 	unsigned int i;
 
-	for (i = 0; i < num; i++)
+	for (i = 0; i < num; i++) {
+		if (!mnt[i])
+			continue;
 		mnt_make_shortterm(mnt[i]);
-	synchronize_rcu_expedited();
-	for (i = 0; i < num; i++)
-		mntput(mnt[i]);
+		hlist_add_head(&real_mount(mnt[i])->mnt_umount, &head);
+	}
+	mntput_unmounted(&head);
 }
 EXPORT_SYMBOL(kern_unmount_array);
 

-- 
2.53.0

[PATCH 6/9] selftests/filesystems: check that unheld mounts are gone after umount

[Christian Brauner]

Use the exclusive open of a loop device to check when the filesystem on
it is shut down:

- synchronous umount(2), before it returns
- lazy umount with nothing else holding the mount, before it returns
- lazy umount of a mount held by a file descriptor, by the last close()
- synchronous umount(2) of a held mount, refused
- death of a mount namespace, before its last task can be reaped

None of the tests wait or retry. They check what has happened by the
time the call returns.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 .../selftests/filesystems/mount_cycle/.gitignore   |   1 +
 .../selftests/filesystems/mount_cycle/Makefile     |   2 +-
 .../filesystems/mount_cycle/unheld_umount_test.c   | 262 +++++++++++++++++++++
 3 files changed, 264 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/filesystems/mount_cycle/.gitignore b/tools/testing/selftests/filesystems/mount_cycle/.gitignore
index 30f7dd071bcb..7b9c4f79053c 100644
--- a/tools/testing/selftests/filesystems/mount_cycle/.gitignore
+++ b/tools/testing/selftests/filesystems/mount_cycle/.gitignore
@@ -6,3 +6,4 @@ mount_cover_test
 locked_handle_test
 nsfs_rbind_loop_test
 overmount_ns_file_test
+unheld_umount_test
diff --git a/tools/testing/selftests/filesystems/mount_cycle/Makefile b/tools/testing/selftests/filesystems/mount_cycle/Makefile
index fbe8c5e19d42..333b7225d89e 100644
--- a/tools/testing/selftests/filesystems/mount_cycle/Makefile
+++ b/tools/testing/selftests/filesystems/mount_cycle/Makefile
@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 TEST_GEN_PROGS := loop_cycle_test unmounted_tree_test overmount_reparent_test mount_cover_test locked_handle_test
-TEST_GEN_PROGS += nsfs_rbind_loop_test overmount_ns_file_test
+TEST_GEN_PROGS += nsfs_rbind_loop_test overmount_ns_file_test unheld_umount_test
 
 CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES)
 
diff --git a/tools/testing/selftests/filesystems/mount_cycle/unheld_umount_test.c b/tools/testing/selftests/filesystems/mount_cycle/unheld_umount_test.c
new file mode 100644
index 000000000000..96e3db277e49
--- /dev/null
+++ b/tools/testing/selftests/filesystems/mount_cycle/unheld_umount_test.c
@@ -0,0 +1,262 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A mount that nothing else references is torn down before umount(2)
+ * returns, lazy or not, before the last close of the file that kept it
+ * alive returns, and before the task that took a mount namespace down
+ * with it can be reaped. A loop device shows when that has happened: the
+ * filesystem claims it, and an exclusive open of the device succeeds
+ * only once the filesystem has let go.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <sched.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/mount.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include <linux/loop.h>
+
+#include "../../kselftest_harness.h"
+
+#define IMAGE_SIZE	(1440 * 1024)
+#define SECTOR		512
+
+/* A blank FAT12 floppy image: boot sector, two FATs, an empty root directory. */
+static int write_fat12(int fd)
+{
+	unsigned char sector[SECTOR] = {
+		0xeb, 0x3c, 0x90, 'M', 'S', 'W', 'I', 'N', '4', '.', '1',
+		[11] = 0x00, 0x02,	/* bytes per sector: 512 */
+		[13] = 1,		/* sectors per cluster */
+		[14] = 1, 0,		/* reserved sectors */
+		[16] = 2,		/* FATs */
+		[17] = 0xe0, 0x00,	/* root directory entries: 224 */
+		[19] = 0x40, 0x0b,	/* total sectors: 2880 */
+		[21] = 0xf0,		/* media descriptor */
+		[22] = 9, 0,		/* sectors per FAT */
+		[24] = 18, 0,		/* sectors per track */
+		[26] = 2, 0,		/* heads */
+		[38] = 0x29,		/* extended boot signature */
+		[39] = 0x12, 0x34, 0x56, 0x78,
+		[43] = 'N', 'O', ' ', 'N', 'A', 'M', 'E', ' ', ' ', ' ', ' ',
+		[54] = 'F', 'A', 'T', '1', '2', ' ', ' ', ' ',
+		[510] = 0x55, 0xaa,
+	};
+	unsigned char fat[SECTOR] = { 0xf0, 0xff, 0xff };
+
+	if (pwrite(fd, sector, SECTOR, 0) != SECTOR)
+		return -1;
+	/* the first FAT and the second one, one sector each is enough */
+	if (pwrite(fd, fat, SECTOR, 1 * SECTOR) != SECTOR ||
+	    pwrite(fd, fat, SECTOR, 10 * SECTOR) != SECTOR)
+		return -1;
+	return ftruncate(fd, IMAGE_SIZE);
+}
+
+/* Bind a free loop device to the open image @ifd; the device number. */
+static int loop_bind(int ifd)
+{
+	int cfd, lfd, n;
+	char dev[32];
+
+	cfd = open("/dev/loop-control", O_RDWR);
+	if (cfd < 0)
+		return -1;
+	n = ioctl(cfd, LOOP_CTL_GET_FREE);
+	close(cfd);
+	if (n < 0)
+		return -1;
+	snprintf(dev, sizeof(dev), "/dev/loop%d", n);
+	lfd = open(dev, O_RDWR);
+	if (lfd < 0)
+		return -1;
+	if (ioctl(lfd, LOOP_SET_FD, ifd))
+		n = -1;
+	close(lfd);
+	return n;
+}
+
+/* Mount the loop device at @mp: vfat, or msdos. */
+static int loop_mount(const char *dev, const char *mp)
+{
+	if (!mount(dev, mp, "vfat", 0, NULL))
+		return 0;
+	return mount(dev, mp, "msdos", 0, NULL);
+}
+
+/*
+ * An exclusive open of the device fails with EBUSY while a filesystem
+ * holds it. No retries: the point is that it has been let go by now.
+ */
+static bool loop_free(const char *dev)
+{
+	int lfd = open(dev, O_RDONLY | O_EXCL);
+
+	if (lfd < 0)
+		return false;
+	close(lfd);
+	return true;
+}
+
+/* Tell the loop device to let go of its image. */
+static void loop_clear(const char *dev)
+{
+	int lfd = open(dev, O_RDWR);
+
+	if (lfd >= 0) {
+		ioctl(lfd, LOOP_CLR_FD);
+		close(lfd);
+	}
+}
+
+FIXTURE(unheld_umount) {
+	char dir[64];
+	char mp[80];
+	char dev[32];
+};
+
+FIXTURE_SETUP(unheld_umount)
+{
+	char img[80];
+	int ifd, n;
+
+	if (geteuid() != 0)
+		SKIP(return, "test requires CAP_SYS_ADMIN");
+	if (access("/dev/loop-control", R_OK | W_OK))
+		SKIP(return, "test requires loop devices");
+
+	ASSERT_EQ(unshare(CLONE_NEWNS), 0);
+	ASSERT_EQ(mount("", "/", NULL, MS_REC | MS_PRIVATE, NULL), 0);
+
+	snprintf(self->dir, sizeof(self->dir), "/tmp/unheld_umount.XXXXXX");
+	ASSERT_NE(mkdtemp(self->dir), NULL);
+	ASSERT_EQ(mount("tmpfs", self->dir, "tmpfs", 0, NULL), 0);
+
+	snprintf(img, sizeof(img), "%s/img", self->dir);
+	ifd = open(img, O_RDWR | O_CREAT | O_EXCL, 0600);
+	ASSERT_GE(ifd, 0);
+	ASSERT_EQ(write_fat12(ifd), 0);
+	n = loop_bind(ifd);
+	close(ifd);	/* the loop device holds the file from now on */
+	ASSERT_GE(n, 0);
+	snprintf(self->dev, sizeof(self->dev), "/dev/loop%d", n);
+
+	snprintf(self->mp, sizeof(self->mp), "%s/mnt", self->dir);
+	ASSERT_EQ(mkdir(self->mp, 0755), 0);
+	/* the harness skips the teardown after a SKIP() here */
+	if (loop_mount(self->dev, self->mp)) {
+		loop_clear(self->dev);
+		SKIP(return, "test requires a FAT filesystem");
+	}
+	umount2(self->mp, 0);
+}
+
+FIXTURE_TEARDOWN(unheld_umount)
+{
+	umount2(self->mp, MNT_DETACH);
+	loop_clear(self->dev);
+	umount2(self->dir, MNT_DETACH);
+	rmdir(self->dir);
+}
+
+/* A synchronous umount(2) returns with the filesystem shut down. */
+TEST_F(unheld_umount, sync_umount_releases_before_return)
+{
+	ASSERT_EQ(loop_mount(self->dev, self->mp), 0);
+	ASSERT_FALSE(loop_free(self->dev));
+	ASSERT_EQ(umount2(self->mp, 0), 0);
+	EXPECT_TRUE(loop_free(self->dev))
+		TH_LOG("%s is still claimed after umount(2) returned", self->dev);
+}
+
+/* So does a lazy one when nothing else references the mount. */
+TEST_F(unheld_umount, lazy_umount_idle_releases_before_return)
+{
+	ASSERT_EQ(loop_mount(self->dev, self->mp), 0);
+	ASSERT_FALSE(loop_free(self->dev));
+	ASSERT_EQ(umount2(self->mp, MNT_DETACH), 0);
+	EXPECT_TRUE(loop_free(self->dev))
+		TH_LOG("%s is still claimed after umount2(MNT_DETACH) returned", self->dev);
+}
+
+/*
+ * A lazy umount of a mount that a file descriptor holds leaves the
+ * filesystem alive; the close that drops the last reference shuts it
+ * down before it returns.
+ */
+TEST_F(unheld_umount, lazy_umount_held_releases_on_close)
+{
+	int fd;
+
+	ASSERT_EQ(loop_mount(self->dev, self->mp), 0);
+	fd = open(self->mp, O_PATH | O_DIRECTORY | O_CLOEXEC);
+	ASSERT_GE(fd, 0);
+	ASSERT_EQ(umount2(self->mp, MNT_DETACH), 0);
+	EXPECT_FALSE(loop_free(self->dev))
+		TH_LOG("%s was released while a file on it was still open", self->dev);
+	ASSERT_EQ(close(fd), 0);
+	EXPECT_TRUE(loop_free(self->dev))
+		TH_LOG("%s is still claimed after the last file on it was closed", self->dev);
+}
+
+/* A synchronous umount(2) of a held mount is refused, nothing changes. */
+TEST_F(unheld_umount, sync_umount_held_refused)
+{
+	int fd;
+
+	ASSERT_EQ(loop_mount(self->dev, self->mp), 0);
+	fd = open(self->mp, O_PATH | O_DIRECTORY | O_CLOEXEC);
+	ASSERT_GE(fd, 0);
+	EXPECT_EQ(umount2(self->mp, 0), -1);
+	EXPECT_EQ(errno, EBUSY);
+	EXPECT_FALSE(loop_free(self->dev));
+	ASSERT_EQ(close(fd), 0);
+	ASSERT_EQ(umount2(self->mp, 0), 0);
+	EXPECT_TRUE(loop_free(self->dev));
+}
+
+/*
+ * The mounts of a mount namespace go with its last task: by the time
+ * that task can be reaped the filesystems it had mounted are shut down.
+ */
+TEST_F(unheld_umount, namespace_death_releases_before_reap)
+{
+	int status, to_parent[2], to_child[2];
+	pid_t pid;
+	char c;
+
+	ASSERT_EQ(pipe(to_parent), 0);
+	ASSERT_EQ(pipe(to_child), 0);
+	pid = fork();
+	ASSERT_GE(pid, 0);
+	if (pid == 0) {
+		close(to_parent[0]);
+		close(to_child[1]);
+		if (unshare(CLONE_NEWNS) || loop_mount(self->dev, self->mp))
+			_exit(1);
+		if (write(to_parent[1], "", 1) != 1)
+			_exit(2);
+		/* leave once the parent has looked, with the mount in place */
+		if (read(to_child[0], &c, 1) != 0)
+			_exit(3);
+		_exit(0);
+	}
+	close(to_parent[1]);
+	close(to_child[0]);
+	ASSERT_EQ(read(to_parent[0], &c, 1), 1);
+	EXPECT_FALSE(loop_free(self->dev))
+		TH_LOG("%s is not claimed by the child's mount", self->dev);
+	close(to_child[1]);
+	ASSERT_EQ(waitpid(pid, &status, 0), pid);
+	ASSERT_EQ(status, 0);
+	close(to_parent[0]);
+	EXPECT_TRUE(loop_free(self->dev))
+		TH_LOG("%s is still claimed after the namespace's last task was reaped", self->dev);
+}
+
+TEST_HARNESS_MAIN

-- 
2.53.0

[PATCH 7/9] Documentation/litmus-tests: add the mount reference count tests

[Christian Brauner]

The reference count of a mount is split into per-cpu gets and puts and
not everyone who touches it takes mount_lock:

(1) A holder drops a reference on the fast path of mntput_no_expire()
    under rcu_read_lock() as long as it sees a non-NULL ->mnt_ns.

(2) A walker in __legitimize_mnt() speculatively takes a reference
    during rcu path walk and drops it again under mount_lock if the
    seqcount has changed.

(3) The unmounter sums up the counters under mount_lock with
    mnt_get_count() and finishes the mount off without a grace period
    if the sum says that nothing else holds it.

That relies on three barriers and one grace period:

- the smp_wmb() before the fast-path put in mntput_no_expire()
- the smp_mb() between the increment and the re-read of the seqcount
  in __legitimize_mnt()
- the smp_mb() between the puts pass and the gets pass in
  mnt_get_count()
- the synchronize_rcu_expedited() that mntput_unmounted() waits for if
  the sum came out higher than one

Finishing an unmounted mount off without a grace period when the count
under mount_lock is one relies on all of this. A holder's get is
counted whenever its put is. A walker's increment is either counted or
the walker sees the seqcount change and drops its reference under
mount_lock. So the one reference the sum found is the mount's own.

That's not enough on its own though. A walker that only starts after
the sum must not be able to find the mount at all. Its seqcount sample
would already be current and __legitimize_mnt() would hand it a
reference to a mount we're tearing down. The mount isn't in the mount
tree anymore and every other way to get at it goes through a reference
the sum would have counted. The tricky case is a holder that had its
cwd on the mount and called chdir() right before the sum. set_fs_pwd()
stores the new fs->pwd before it puts the old one and both the smp_wmb()
in write_sequnlock() and the one on the fast path of mntput_no_expire()
order that. The smp_mb() in mnt_get_count() then orders the load of the
put before the seqcount store of the unlock. So a walker that sees that
seqcount also sees the new fs->pwd.

Add litmus tests so we can check that argument against the kernel
memory model. Each test is a two-process cut of fs/namespace.c except
for the mnt-walker-late-sees-new-pwd tests which need a third process
for the late walker. The per-cpu counters are one variable per cpu,
write_seqlock() is spelled out as a spinlock, a seqcount store and an
smp_wmb(), and read_seqretry() is an smp_rmb() and a READ_ONCE().

The tests of the code as it is are "Never". The exception is
mnt-sum-misses-fastpath-put which shows that the sum can miss a put on
the fast path. That's why mntput_unmounted() still waits for a grace
period when the count isn't one. Next to them are mutations with one
ingredient removed and those are "Sometimes": the holder's smp_wmb(),
the smp_mb() in mnt_get_count() or the order of its two passes, the
walker's smp_mb(), the lock around the walker's look at ->mnt_flags,
and the grace period. So we know which barrier carries which pairing.

Run them with herd7 against tools/memory-model:

    cd tools/memory-model
    scripts/checklitmus.sh ../../Documentation/litmus-tests/fs/<test>

Note that every architecture's smp_mb(), smp_wmb(), READ_ONCE(),
WRITE_ONCE(), spinlock and RCU implementation has to follow the memory
model. So a "Never" there holds on all architectures.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 Documentation/litmus-tests/README                  |  95 +++++++++++++++++++
 .../fs/mnt-gp-sees-fastpath-put-nogp.litmus        |  75 +++++++++++++++
 .../fs/mnt-gp-sees-fastpath-put-sync-rcu.litmus    |  73 +++++++++++++++
 .../fs/mnt-gp-sees-fastpath-put.litmus             |  79 ++++++++++++++++
 .../fs/mnt-holder-get-before-put-getsfirst.litmus  |  63 +++++++++++++
 .../fs/mnt-holder-get-before-put-nomb.litmus       |  62 +++++++++++++
 .../fs/mnt-holder-get-before-put-nowmb.litmus      |  62 +++++++++++++
 .../fs/mnt-holder-get-before-put.litmus            |  73 +++++++++++++++
 .../mnt-kern-unmount-gp-sees-fastpath-put.litmus   |  70 ++++++++++++++
 .../mnt-kern-unmount-holder-get-before-put.litmus  |  60 ++++++++++++
 .../fs/mnt-sum-misses-fastpath-put.litmus          |  69 ++++++++++++++
 .../fs/mnt-walker-bail-sees-doomed-nolock.litmus   |  89 ++++++++++++++++++
 .../fs/mnt-walker-bail-sees-doomed.litmus          |  95 +++++++++++++++++++
 .../mnt-walker-get-vs-seqcount-no-outer-mb.litmus  |  92 ++++++++++++++++++
 .../fs/mnt-walker-get-vs-seqcount-nomb.litmus      |  92 ++++++++++++++++++
 .../fs/mnt-walker-get-vs-seqcount.litmus           | 102 ++++++++++++++++++++
 .../fs/mnt-walker-late-sees-new-pwd-nomb.litmus    |  89 ++++++++++++++++++
 .../fs/mnt-walker-late-sees-new-pwd-nowmb.litmus   |  88 ++++++++++++++++++
 .../fs/mnt-walker-late-sees-new-pwd.litmus         | 103 +++++++++++++++++++++
 19 files changed, 1531 insertions(+)

diff --git a/Documentation/litmus-tests/README b/Documentation/litmus-tests/README
index 6c666f3422ea..47fca037c6e1 100644
--- a/Documentation/litmus-tests/README
+++ b/Documentation/litmus-tests/README
@@ -38,6 +38,101 @@ cmpxchg-fail-unordered-2.litmus
     acquire operation.
 
 
+fs (/fs directory)
+------------------
+
+The reference count of a mount is split into per-CPU gets and puts.  A
+holder drops a reference on the fast path of mntput_no_expire() without
+mount_lock, a walker in __legitimize_mnt() takes one during an RCU path
+walk, and the unmounter sums the counters under mount_lock with
+mnt_get_count() to decide whether it can finish the mount off without
+waiting for a grace period.  The tests spell write_seqlock() out as a
+spinlock, a seqcount store and smp_wmb(), read_seqbegin() and
+read_seqretry() as READ_ONCE() and smp_rmb(), and the per-CPU counters
+as one variable per CPU.
+
+mnt-sum-misses-fastpath-put.litmus
+    Demonstrates that the sum under mount_lock can miss a put on the
+    fast path of mntput_no_expire(), which is why mntput_unmounted()
+    falls back to a grace period when the count is not one.
+
+mnt-holder-get-before-put.litmus
+    Demonstrates that a fast-path put the sum counts has its get counted
+    too: the smp_wmb() in mntput_no_expire() pairs with the smp_mb()
+    between the puts pass and the gets pass of mnt_get_count().
+
+mnt-holder-get-before-put-nowmb.litmus
+    The same without the holder's smp_wmb(); the sum can count the put
+    but not the get.
+
+mnt-holder-get-before-put-nomb.litmus
+    The same without the smp_mb() inside mnt_get_count(); an smp_mb()
+    ahead of the puts pass does not stand in for it.
+
+mnt-holder-get-before-put-getsfirst.litmus
+    The same with the gets summed before the puts; the order of the two
+    passes is part of the pairing.
+
+mnt-gp-sees-fastpath-put.litmus
+    Demonstrates that after the grace period of mntput_unmounted() the
+    plain mntput() sees the fast-path put the sum missed, because the
+    holder's read-side critical section ended before the grace period.
+
+mnt-gp-sees-fastpath-put-sync-rcu.litmus
+    The same with synchronize_rcu() in place of
+    synchronize_rcu_expedited().
+
+mnt-gp-sees-fastpath-put-nogp.litmus
+    The same with an smp_mb() in place of the grace period; the put can
+    still be missed.
+
+mnt-walker-get-vs-seqcount.litmus
+    Demonstrates the store buffering pairing between the increment and
+    smp_mb() of __legitimize_mnt() and the seqcount store and the
+    smp_mb() of mnt_get_count() under mount_lock: the walker cannot
+    keep a reference to a mount the sum finishes off.
+
+mnt-walker-get-vs-seqcount-no-outer-mb.litmus
+    The same with the sum as the tree has it, without an smp_mb()
+    between taking mount_lock and the sum; the smp_mb() inside
+    mnt_get_count() is the one that pairs.
+
+mnt-walker-get-vs-seqcount-nomb.litmus
+    The same without the walker's smp_mb(); the walker can keep a
+    reference to a freed mount.
+
+mnt-walker-bail-sees-doomed.litmus
+    Demonstrates that a walker which sees the seqcount change and reads
+    ->mnt_flags under mount_lock finds MNT_DOOMED on a mount the sum
+    finished off; mount_lock orders the two critical sections.
+
+mnt-walker-bail-sees-doomed-nolock.litmus
+    The same with ->mnt_flags read outside mount_lock; the walker can
+    miss MNT_DOOMED.
+
+mnt-walker-late-sees-new-pwd.litmus
+    Demonstrates that a walker which starts after the sum finished a
+    mount off cannot reach it through the old cwd of a holder whose put
+    the sum counted: the holder orders the new fs->pwd before the put
+    and the smp_mb() of mnt_get_count() orders the load of the put
+    before the seqcount store the walker samples.
+
+mnt-walker-late-sees-new-pwd-nowmb.litmus
+    The same without the smp_wmb()s that order the new fs->pwd before
+    the put; the walker can read the old cwd.
+
+mnt-walker-late-sees-new-pwd-nomb.litmus
+    The same without the smp_mb() inside mnt_get_count(); the walker can
+    read the old cwd.
+
+mnt-kern-unmount-holder-get-before-put.litmus
+    The holder test for kern_unmount_array(), where mnt_make_shortterm()
+    clears ->mnt_ns with a plain WRITE_ONCE() outside mount_lock.
+
+mnt-kern-unmount-gp-sees-fastpath-put.litmus
+    The grace period test for kern_unmount_array().
+
+
 locking (/locking directory)
 ----------------------------
 
diff --git a/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-nogp.litmus b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-nogp.litmus
new file mode 100644
index 000000000000..b0b4037217a4
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-nogp.litmus
@@ -0,0 +1,75 @@
+C mnt-gp-sees-fastpath-put-nogp
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-gp-sees-fastpath-put.litmus with an smp_mb() in place of the
+ * grace period.  A barrier orders the unmounter's own accesses and
+ * waits for nothing: the holder's read-side critical section may still
+ * be running and the second sum misses the put like the first one.
+ * Only a grace period covers a fast-path put the sum missed.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r3;
+	int r5;
+	int r6;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+
+	smp_mb();
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 5);
+	smp_wmb();
+	smp_mb();
+	WRITE_ONCE(*puts_u, 1);
+	r5 = READ_ONCE(*puts_h);
+	smp_mb();
+	r6 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 6);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r5=0)
diff --git a/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-sync-rcu.litmus b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-sync-rcu.litmus
new file mode 100644
index 000000000000..e8d4b17f1426
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put-sync-rcu.litmus
@@ -0,0 +1,73 @@
+C mnt-gp-sees-fastpath-put-sync-rcu
+
+(*
+ * Result: Never
+ *
+ * mnt-gp-sees-fastpath-put.litmus with synchronize_rcu() in place of
+ * synchronize_rcu_expedited().  To the memory model the two are the
+ * same fence.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r3;
+	int r5;
+	int r6;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+
+	synchronize_rcu();
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 5);
+	smp_wmb();
+	smp_mb();
+	WRITE_ONCE(*puts_u, 1);
+	r5 = READ_ONCE(*puts_h);
+	smp_mb();
+	r6 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 6);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r5=0)
diff --git a/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put.litmus b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put.litmus
new file mode 100644
index 000000000000..b60100fdab5e
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-gp-sees-fastpath-put.litmus
@@ -0,0 +1,79 @@
+C mnt-gp-sees-fastpath-put
+
+(*
+ * Result: Never
+ *
+ * The sum of mntput_unheld() can miss the put of a holder on the fast
+ * path of mntput_no_expire(), see mnt-sum-misses-fastpath-put.litmus.
+ * mntput_unmounted() then waits in synchronize_rcu_expedited() and
+ * drops the own reference with a plain mntput(), which sums again in
+ * mntput_no_expire_slowpath() under mount_lock.  The holder read
+ * ->mnt_ns non-NULL inside rcu_read_lock() before umount_tree() stored
+ * NULL, so its read-side critical section, and the put inside it, ends
+ * before the grace period does.  The first sum may miss the put
+ * (1:r1), the second one never does (1:r5=1).
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r3;
+	int r5;
+	int r6;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+
+	synchronize_rcu_expedited();
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 5);
+	smp_wmb();
+	smp_mb();
+	WRITE_ONCE(*puts_u, 1);
+	r5 = READ_ONCE(*puts_h);
+	smp_mb();
+	r6 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 6);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r5=0)
diff --git a/Documentation/litmus-tests/fs/mnt-holder-get-before-put-getsfirst.litmus b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-getsfirst.litmus
new file mode 100644
index 000000000000..d20470a276fe
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-getsfirst.litmus
@@ -0,0 +1,63 @@
+C mnt-holder-get-before-put-getsfirst
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-holder-get-before-put.litmus with mnt_get_count() summing the
+ * gets before the puts, the smp_mb() still between the two passes.
+ * The order of the passes is part of the pairing: read first, the
+ * gets pass can predate a get whose put is then counted.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	WRITE_ONCE(*gets_h, 2);
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=1 /\ 1:r3=1)
diff --git a/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nomb.litmus b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nomb.litmus
new file mode 100644
index 000000000000..2da4353bc087
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nomb.litmus
@@ -0,0 +1,62 @@
+C mnt-holder-get-before-put-nomb
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-holder-get-before-put.litmus without the smp_mb() between the
+ * puts pass and the gets pass of mnt_get_count().  The smp_mb() ahead
+ * of the puts pass is still there and does not help: the barrier that
+ * pairs with the holder's smp_wmb() has to sit between the two loads.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	WRITE_ONCE(*gets_h, 2);
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=1 /\ 1:r3=1)
diff --git a/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nowmb.litmus b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nowmb.litmus
new file mode 100644
index 000000000000..d9070e813e47
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-holder-get-before-put-nowmb.litmus
@@ -0,0 +1,62 @@
+C mnt-holder-get-before-put-nowmb
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-holder-get-before-put.litmus without the smp_wmb() between the
+ * get and the fast-path put in mntput_no_expire().  Nothing orders the
+ * two stores: the sum counts the put but not the get and finishes the
+ * mount off under the holder.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	WRITE_ONCE(*gets_h, 2);
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=1 /\ 1:r3=1)
diff --git a/Documentation/litmus-tests/fs/mnt-holder-get-before-put.litmus b/Documentation/litmus-tests/fs/mnt-holder-get-before-put.litmus
new file mode 100644
index 000000000000..414c303dcbb6
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-holder-get-before-put.litmus
@@ -0,0 +1,73 @@
+C mnt-holder-get-before-put
+
+(*
+ * Result: Never
+ *
+ * A holder with a reference of its own (gets_h=1) does a transient
+ * mntget() and drops it on the fast path of mntput_no_expire(): the
+ * store to its gets counter, READ_ONCE(->mnt_ns) non-NULL, smp_wmb(),
+ * the store to its puts counter.  umount_tree() clears ->mnt_ns under
+ * write_seqlock() and mntput_unheld() sums with mnt_get_count(): the
+ * puts pass, smp_mb(), the gets pass.  The smp_wmb() pairs with that
+ * smp_mb(), so a put the sum counts has its get counted too.  The bad
+ * outcome is the sum counting the put (1:r1=1) but not the get
+ * (1:r3=1): a count of one, and the mount finished off under the
+ * reference the holder still has.  Message passing, smp_wmb() on the
+ * holder's side and smp_mb() on the sum's.  The sum is modelled with
+ * an smp_mb() ahead of the puts pass as well; it takes no part in
+ * this, see the -nomb variant.  A put that lands on another CPU's
+ * counter after a migration is the same test, the model has no CPUs.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	WRITE_ONCE(*gets_h, 2);
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=1 /\ 1:r3=1)
diff --git a/Documentation/litmus-tests/fs/mnt-kern-unmount-gp-sees-fastpath-put.litmus b/Documentation/litmus-tests/fs/mnt-kern-unmount-gp-sees-fastpath-put.litmus
new file mode 100644
index 000000000000..59dd434e6a76
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-kern-unmount-gp-sees-fastpath-put.litmus
@@ -0,0 +1,70 @@
+C mnt-kern-unmount-gp-sees-fastpath-put
+
+(*
+ * Result: Never
+ *
+ * mnt-gp-sees-fastpath-put.litmus for kern_unmount_array(), where
+ * mnt_make_shortterm() clears ->mnt_ns with a plain WRITE_ONCE()
+ * outside mount_lock.  The holder still read it non-NULL inside
+ * rcu_read_lock() before the store, so its read-side critical section
+ * ends before the grace period of mntput_unmounted() does and the
+ * plain mntput() that follows sees its put.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r3;
+	int r5;
+	int r6;
+
+	WRITE_ONCE(*mnt_ns, 0);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+
+	synchronize_rcu_expedited();
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 5);
+	smp_wmb();
+	smp_mb();
+	WRITE_ONCE(*puts_u, 1);
+	r5 = READ_ONCE(*puts_h);
+	smp_mb();
+	r6 = READ_ONCE(*gets_h);
+	smp_wmb();
+	WRITE_ONCE(*seq, 6);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r5=0)
diff --git a/Documentation/litmus-tests/fs/mnt-kern-unmount-holder-get-before-put.litmus b/Documentation/litmus-tests/fs/mnt-kern-unmount-holder-get-before-put.litmus
new file mode 100644
index 000000000000..dd7976f06b93
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-kern-unmount-holder-get-before-put.litmus
@@ -0,0 +1,60 @@
+C mnt-kern-unmount-holder-get-before-put
+
+(*
+ * Result: Never
+ *
+ * mnt-holder-get-before-put.litmus for kern_unmount_array(), where
+ * mnt_make_shortterm() clears ->mnt_ns with a plain WRITE_ONCE(),
+ * outside mount_lock and without a seqcount write, before
+ * mntput_unmounted() sums under the lock.  The pairing is unchanged:
+ * the holder's smp_wmb() against the smp_mb() of mnt_get_count(),
+ * neither of which needs the lock or the seqcount.  Only holders are
+ * modelled.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	WRITE_ONCE(*gets_h, 2);
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	WRITE_ONCE(*mnt_ns, 0);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=1 /\ 1:r3=1)
diff --git a/Documentation/litmus-tests/fs/mnt-sum-misses-fastpath-put.litmus b/Documentation/litmus-tests/fs/mnt-sum-misses-fastpath-put.litmus
new file mode 100644
index 000000000000..34347c6a2b25
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-sum-misses-fastpath-put.litmus
@@ -0,0 +1,69 @@
+C mnt-sum-misses-fastpath-put
+
+(*
+ * Result: Sometimes
+ *
+ * A holder drops its reference on the fast path of mntput_no_expire():
+ * READ_ONCE(->mnt_ns) non-NULL under rcu_read_lock(), smp_wmb(), the
+ * put.  umount_tree() clears ->mnt_ns under write_seqlock() and
+ * mntput_unheld() sums the counters with mnt_get_count() under the
+ * lock.  The holder read ->mnt_ns before it was cleared (0:r0=1) and
+ * the sum still does not count its put (1:r1=0): the put is simply
+ * later.  The sum comes out at two, so mntput_unmounted() has to wait
+ * for the holder's read-side critical section before the plain
+ * mntput() that drops the own reference, see
+ * mnt-gp-sees-fastpath-put.litmus.  This is why the grace period
+ * branch exists, not a bug.
+ *)
+
+{
+	mnt_ns=1;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+}
+
+P0(int *mnt_ns, int *gets_h, int *puts_h)
+{
+	int r0;
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+exists (0:r0=1 /\ 1:r1=0)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed-nolock.litmus b/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed-nolock.litmus
new file mode 100644
index 000000000000..233f2e65d25d
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed-nolock.litmus
@@ -0,0 +1,89 @@
+C mnt-walker-bail-sees-doomed-nolock
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-walker-bail-sees-doomed.litmus with the walker reading
+ * ->mnt_flags outside mount_lock.  Nothing orders that read against
+ * the sum's critical section: the walker can read the flags before
+ * MNT_DOOMED is set and keep a reference to a mount the sum has
+ * finished off.  This is why __legitimize_mnt() looks under the lock.
+ *)
+
+{
+	mnt_ns=1;
+	gets_w=0;
+	puts_w=0;
+	gets_u=1;
+	puts_u=0;
+	flags=0;
+	seq=0;
+	freed=0;
+}
+
+P0(spinlock_t *mlock, int *seq, int *gets_w, int *puts_w, int *flags)
+{
+	int r0;
+	int r1;
+	int r2;
+	int r3;
+
+	r0 = READ_ONCE(*seq);
+	smp_rmb();
+	smp_rmb();
+	r1 = READ_ONCE(*seq);
+	if (r1 == 0) {
+		WRITE_ONCE(*gets_w, 1);
+		smp_mb();
+		smp_rmb();
+		r2 = READ_ONCE(*seq);
+		if (r2 != 0) {
+			r3 = READ_ONCE(*flags);
+			if (r3 != 0) {
+				WRITE_ONCE(*puts_w, 1);
+			}
+		}
+	}
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *flags, int *gets_w, int *puts_w, int *gets_u, int *puts_u, int *freed)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_w);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_w);
+	r4 = READ_ONCE(*gets_u);
+	if (r3 == 0) {
+		WRITE_ONCE(*flags, 3);
+		WRITE_ONCE(*freed, 1);
+	}
+	if (r3 == 1) {
+		if (r1 == 1) {
+			WRITE_ONCE(*flags, 3);
+			WRITE_ONCE(*freed, 1);
+		}
+	}
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+filter (0:r0=0)
+exists (0:r1=0 /\ ~(0:r2=0) /\ 0:r3=0 /\ freed=1)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed.litmus b/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed.litmus
new file mode 100644
index 000000000000..187619db7fb6
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-bail-sees-doomed.litmus
@@ -0,0 +1,95 @@
+C mnt-walker-bail-sees-doomed
+
+(*
+ * Result: Never
+ *
+ * A lazy unmount: umount_tree() clears ->mnt_ns under write_seqlock()
+ * without setting MNT_SYNC_UMOUNT, and mntput_unheld() sets MNT_DOOMED
+ * (flags=3) when the sum says nothing else holds the mount.  A walker
+ * in __legitimize_mnt() that sees the seqcount change takes mount_lock
+ * and reads ->mnt_flags there.  Not both the walker finding no flag
+ * set (0:r3=0, it keeps the reference) and the mount being finished
+ * off (freed=1).  mount_lock orders the two critical sections: either
+ * the walker's comes first and the sum counts its increment, or the
+ * sum's does and the walker finds MNT_DOOMED.
+ *)
+
+{
+	mnt_ns=1;
+	gets_w=0;
+	puts_w=0;
+	gets_u=1;
+	puts_u=0;
+	flags=0;
+	seq=0;
+	freed=0;
+}
+
+P0(spinlock_t *mlock, int *seq, int *gets_w, int *puts_w, int *flags)
+{
+	int r0;
+	int r1;
+	int r2;
+	int r3;
+
+	r0 = READ_ONCE(*seq);
+	smp_rmb();
+	smp_rmb();
+	r1 = READ_ONCE(*seq);
+	if (r1 == 0) {
+		WRITE_ONCE(*gets_w, 1);
+		smp_mb();
+		smp_rmb();
+		r2 = READ_ONCE(*seq);
+		if (r2 != 0) {
+			spin_lock(mlock);
+			r3 = READ_ONCE(*flags);
+			if (r3 != 0) {
+				WRITE_ONCE(*puts_w, 1);
+			}
+			spin_unlock(mlock);
+		}
+	}
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *flags, int *gets_w, int *puts_w, int *gets_u, int *puts_u, int *freed)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_w);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_w);
+	r4 = READ_ONCE(*gets_u);
+	if (r3 == 0) {
+		WRITE_ONCE(*flags, 3);
+		WRITE_ONCE(*freed, 1);
+	}
+	if (r3 == 1) {
+		if (r1 == 1) {
+			WRITE_ONCE(*flags, 3);
+			WRITE_ONCE(*freed, 1);
+		}
+	}
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+filter (0:r0=0)
+exists (0:r1=0 /\ ~(0:r2=0) /\ 0:r3=0 /\ freed=1)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-no-outer-mb.litmus b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-no-outer-mb.litmus
new file mode 100644
index 000000000000..66e65920e491
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-no-outer-mb.litmus
@@ -0,0 +1,92 @@
+C mnt-walker-get-vs-seqcount-no-outer-mb
+
+(*
+ * Result: Never
+ *
+ * mnt-walker-get-vs-seqcount.litmus with the sum as mntput_unheld()
+ * has it: write_seqlock(), the puts pass, the smp_mb() of
+ * mnt_get_count(), the gets pass, and no smp_mb() between taking the
+ * lock and the sum.  The inner smp_mb() comes after the seqcount store
+ * and before the gets loads in program order, which is all store
+ * buffering asks of the unmounter's side.
+ *)
+
+{
+	mnt_ns=1;
+	gets_w=0;
+	puts_w=0;
+	gets_u=1;
+	puts_u=0;
+	flags=0;
+	seq=0;
+	freed=0;
+}
+
+P0(spinlock_t *mlock, int *seq, int *gets_w, int *puts_w, int *flags)
+{
+	int r0;
+	int r1;
+	int r2;
+	int r3;
+
+	r0 = READ_ONCE(*seq);
+	smp_rmb();
+	smp_rmb();
+	r1 = READ_ONCE(*seq);
+	if (r1 == 0) {
+		WRITE_ONCE(*gets_w, 1);
+		smp_mb();
+		smp_rmb();
+		r2 = READ_ONCE(*seq);
+		if (r2 != 0) {
+			spin_lock(mlock);
+			r3 = READ_ONCE(*flags);
+			if (r3 != 0) {
+				WRITE_ONCE(*puts_w, 1);
+			}
+			spin_unlock(mlock);
+		}
+	}
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *flags, int *gets_w, int *puts_w, int *gets_u, int *puts_u, int *freed)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	WRITE_ONCE(*flags, 1);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	r1 = READ_ONCE(*puts_w);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_w);
+	r4 = READ_ONCE(*gets_u);
+	if (r3 == 0) {
+		WRITE_ONCE(*flags, 3);
+		WRITE_ONCE(*freed, 1);
+	}
+	if (r3 == 1) {
+		if (r1 == 1) {
+			WRITE_ONCE(*flags, 3);
+			WRITE_ONCE(*freed, 1);
+		}
+	}
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+filter (0:r0=0)
+exists (0:r1=0 /\ 0:r2=0 /\ freed=1)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-nomb.litmus b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-nomb.litmus
new file mode 100644
index 000000000000..e3297207e760
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount-nomb.litmus
@@ -0,0 +1,92 @@
+C mnt-walker-get-vs-seqcount-nomb
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-walker-get-vs-seqcount.litmus without the smp_mb() between the
+ * increment and the re-read of the seqcount in __legitimize_mnt().
+ * The smp_rmb() of read_seqretry() does not order a store against a
+ * later load: the walker sees the seqcount unchanged and keeps its
+ * reference while the sum misses the increment and finishes the mount
+ * off.
+ *)
+
+{
+	mnt_ns=1;
+	gets_w=0;
+	puts_w=0;
+	gets_u=1;
+	puts_u=0;
+	flags=0;
+	seq=0;
+	freed=0;
+}
+
+P0(spinlock_t *mlock, int *seq, int *gets_w, int *puts_w, int *flags)
+{
+	int r0;
+	int r1;
+	int r2;
+	int r3;
+
+	r0 = READ_ONCE(*seq);
+	smp_rmb();
+	smp_rmb();
+	r1 = READ_ONCE(*seq);
+	if (r1 == 0) {
+		WRITE_ONCE(*gets_w, 1);
+		smp_rmb();
+		r2 = READ_ONCE(*seq);
+		if (r2 != 0) {
+			spin_lock(mlock);
+			r3 = READ_ONCE(*flags);
+			if (r3 != 0) {
+				WRITE_ONCE(*puts_w, 1);
+			}
+			spin_unlock(mlock);
+		}
+	}
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *flags, int *gets_w, int *puts_w, int *gets_u, int *puts_u, int *freed)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	WRITE_ONCE(*flags, 1);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_w);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_w);
+	r4 = READ_ONCE(*gets_u);
+	if (r3 == 0) {
+		WRITE_ONCE(*flags, 3);
+		WRITE_ONCE(*freed, 1);
+	}
+	if (r3 == 1) {
+		if (r1 == 1) {
+			WRITE_ONCE(*flags, 3);
+			WRITE_ONCE(*freed, 1);
+		}
+	}
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+filter (0:r0=0)
+exists (0:r1=0 /\ 0:r2=0 /\ freed=1)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount.litmus b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount.litmus
new file mode 100644
index 000000000000..46ca500f8d50
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-get-vs-seqcount.litmus
@@ -0,0 +1,102 @@
+C mnt-walker-get-vs-seqcount
+
+(*
+ * Result: Never
+ *
+ * A walker in __legitimize_mnt() increments its gets counter, does
+ * smp_mb() and re-reads the seqcount of mount_lock with
+ * read_seqretry(); if it changed, the walker takes mount_lock and
+ * drops the reference again when it finds MNT_SYNC_UMOUNT or
+ * MNT_DOOMED.  umount_tree() stores the seqcount when it takes
+ * write_seqlock(), clears ->mnt_ns and sets MNT_SYNC_UMOUNT;
+ * mntput_unheld() takes the lock again and sums with mnt_get_count():
+ * the puts pass, smp_mb(), the gets pass.  Store buffering between the
+ * walker's increment and the unmounter's seqcount store: not both the
+ * walker seeing the seqcount unchanged (0:r1=0, 0:r2=0, it keeps the
+ * reference) and the sum missing the increment (freed=1, the mount is
+ * finished off).  The walker's smp_mb() and the smp_mb() of
+ * mnt_get_count() are the two full barriers that takes.  The sum is
+ * modelled with a further smp_mb() ahead of the puts pass, which the
+ * -no-outer-mb variant drops.
+ *)
+
+{
+	mnt_ns=1;
+	gets_w=0;
+	puts_w=0;
+	gets_u=1;
+	puts_u=0;
+	flags=0;
+	seq=0;
+	freed=0;
+}
+
+P0(spinlock_t *mlock, int *seq, int *gets_w, int *puts_w, int *flags)
+{
+	int r0;
+	int r1;
+	int r2;
+	int r3;
+
+	r0 = READ_ONCE(*seq);
+	smp_rmb();
+	smp_rmb();
+	r1 = READ_ONCE(*seq);
+	if (r1 == 0) {
+		WRITE_ONCE(*gets_w, 1);
+		smp_mb();
+		smp_rmb();
+		r2 = READ_ONCE(*seq);
+		if (r2 != 0) {
+			spin_lock(mlock);
+			r3 = READ_ONCE(*flags);
+			if (r3 != 0) {
+				WRITE_ONCE(*puts_w, 1);
+			}
+			spin_unlock(mlock);
+		}
+	}
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *flags, int *gets_w, int *puts_w, int *gets_u, int *puts_u, int *freed)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	WRITE_ONCE(*flags, 1);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	smp_mb();
+	r1 = READ_ONCE(*puts_w);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_w);
+	r4 = READ_ONCE(*gets_u);
+	if (r3 == 0) {
+		WRITE_ONCE(*flags, 3);
+		WRITE_ONCE(*freed, 1);
+	}
+	if (r3 == 1) {
+		if (r1 == 1) {
+			WRITE_ONCE(*flags, 3);
+			WRITE_ONCE(*freed, 1);
+		}
+	}
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+filter (0:r0=0)
+exists (0:r1=0 /\ 0:r2=0 /\ freed=1)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nomb.litmus b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nomb.litmus
new file mode 100644
index 000000000000..38d52c31f0b8
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nomb.litmus
@@ -0,0 +1,89 @@
+C mnt-walker-late-sees-new-pwd-nomb
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-walker-late-sees-new-pwd.litmus without the smp_mb() between the
+ * puts pass and the gets pass of mnt_get_count().  The smp_wmb() of
+ * write_sequnlock() orders stores only: the load of the put can be
+ * satisfied after the seqcount store of the unlock and a walker that
+ * samples the seqcount after the sum still reads the old cwd.
+ *)
+
+{
+	mnt_ns=1;
+	pwd=0;
+	fseq=0;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+	seq=0;
+}
+
+P0(spinlock_t *fslock, int *fseq, int *pwd, int *mnt_ns, int *puts_h)
+{
+	int r0;
+
+	spin_lock(fslock);
+	WRITE_ONCE(*fseq, 1);
+	smp_wmb();
+	WRITE_ONCE(*pwd, 1);
+	smp_wmb();
+	WRITE_ONCE(*fseq, 2);
+	spin_unlock(fslock);
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+P2(int *seq, int *fseq, int *pwd)
+{
+	int r5;
+	int r6;
+	int r7;
+	int r8;
+
+	r5 = READ_ONCE(*seq);
+	smp_rmb();
+	r7 = READ_ONCE(*fseq);
+	smp_rmb();
+	r6 = READ_ONCE(*pwd);
+	smp_rmb();
+	r8 = READ_ONCE(*fseq);
+}
+
+filter (2:r7=0 /\ 2:r8=0 \/ 2:r7=2 /\ 2:r8=2)
+exists (1:r1=1 /\ 2:r5=4 /\ 2:r6=0)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nowmb.litmus b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nowmb.litmus
new file mode 100644
index 000000000000..d3d68619311e
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd-nowmb.litmus
@@ -0,0 +1,88 @@
+C mnt-walker-late-sees-new-pwd-nowmb
+
+(*
+ * Result: Sometimes
+ *
+ * mnt-walker-late-sees-new-pwd.litmus without the smp_wmb() of
+ * write_sequnlock() after the store of the new fs->pwd and without the
+ * smp_wmb() on the fast path of mntput_no_expire().  Nothing orders the
+ * new fs->pwd before the put: the sum can count the put while a walker
+ * that samples the seqcount after the sum still reads the old cwd.
+ *)
+
+{
+	mnt_ns=1;
+	pwd=0;
+	fseq=0;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+	seq=0;
+}
+
+P0(spinlock_t *fslock, int *fseq, int *pwd, int *mnt_ns, int *puts_h)
+{
+	int r0;
+
+	spin_lock(fslock);
+	WRITE_ONCE(*fseq, 1);
+	smp_wmb();
+	WRITE_ONCE(*pwd, 1);
+	WRITE_ONCE(*fseq, 2);
+	spin_unlock(fslock);
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+P2(int *seq, int *fseq, int *pwd)
+{
+	int r5;
+	int r6;
+	int r7;
+	int r8;
+
+	r5 = READ_ONCE(*seq);
+	smp_rmb();
+	r7 = READ_ONCE(*fseq);
+	smp_rmb();
+	r6 = READ_ONCE(*pwd);
+	smp_rmb();
+	r8 = READ_ONCE(*fseq);
+}
+
+filter (2:r7=0 /\ 2:r8=0 \/ 2:r7=2 /\ 2:r8=2)
+exists (1:r1=1 /\ 2:r5=4 /\ 2:r6=0)
diff --git a/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd.litmus b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd.litmus
new file mode 100644
index 000000000000..3e195dd4317a
--- /dev/null
+++ b/Documentation/litmus-tests/fs/mnt-walker-late-sees-new-pwd.litmus
@@ -0,0 +1,103 @@
+C mnt-walker-late-sees-new-pwd
+
+(*
+ * Result: Never
+ *
+ * A holder has its cwd on the mount and chdir()s away right before
+ * mntput_unheld() sums.  set_fs_pwd() stores the new fs->pwd under
+ * write_seqlock() of fs->seq and then drops the reference to the old cwd
+ * on the fast path of mntput_no_expire(): smp_wmb(), the put.
+ * umount_tree() clears ->mnt_ns under write_seqlock() of mount_lock and
+ * mntput_unheld() sums with mnt_get_count() under the lock: the puts
+ * pass, smp_mb(), the gets pass.  The sum counts the put (1:r1=1), comes
+ * out at one and finishes the mount off.  A walker whose path_init()
+ * samples the seqcount of mount_lock after that critical section
+ * (2:r5=4) and then reads fs->pwd must not read the old cwd (2:r6=0).
+ * Its seqcount sample would never change again and __legitimize_mnt()
+ * would hand it a reference to the finished mount.  The smp_wmb() of
+ * write_sequnlock() orders the new fs->pwd before the put and so does
+ * the smp_wmb() of the fast path, either one is enough.  The smp_mb() of
+ * mnt_get_count() orders the load of the put before the seqcount store
+ * of the unlock, and the walker orders its loads with smp_rmb().  A
+ * holder that already sees ->mnt_ns NULL puts under mount_lock, which
+ * orders the put anyway.
+ *)
+
+{
+	mnt_ns=1;
+	pwd=0;
+	fseq=0;
+	gets_h=1;
+	puts_h=0;
+	gets_u=1;
+	puts_u=0;
+	seq=0;
+}
+
+P0(spinlock_t *fslock, int *fseq, int *pwd, int *mnt_ns, int *puts_h)
+{
+	int r0;
+
+	spin_lock(fslock);
+	WRITE_ONCE(*fseq, 1);
+	smp_wmb();
+	WRITE_ONCE(*pwd, 1);
+	smp_wmb();
+	WRITE_ONCE(*fseq, 2);
+	spin_unlock(fslock);
+
+	rcu_read_lock();
+	r0 = READ_ONCE(*mnt_ns);
+	if (r0) {
+		smp_wmb();
+		WRITE_ONCE(*puts_h, 1);
+	}
+	rcu_read_unlock();
+}
+
+P1(spinlock_t *mlock, int *seq, int *mnt_ns, int *gets_h, int *puts_h, int *gets_u, int *puts_u)
+{
+	int r1;
+	int r2;
+	int r3;
+	int r4;
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 1);
+	smp_wmb();
+	WRITE_ONCE(*mnt_ns, 0);
+	smp_wmb();
+	WRITE_ONCE(*seq, 2);
+	spin_unlock(mlock);
+
+	spin_lock(mlock);
+	WRITE_ONCE(*seq, 3);
+	smp_wmb();
+	r1 = READ_ONCE(*puts_h);
+	r2 = READ_ONCE(*puts_u);
+	smp_mb();
+	r3 = READ_ONCE(*gets_h);
+	r4 = READ_ONCE(*gets_u);
+	smp_wmb();
+	WRITE_ONCE(*seq, 4);
+	spin_unlock(mlock);
+}
+
+P2(int *seq, int *fseq, int *pwd)
+{
+	int r5;
+	int r6;
+	int r7;
+	int r8;
+
+	r5 = READ_ONCE(*seq);
+	smp_rmb();
+	r7 = READ_ONCE(*fseq);
+	smp_rmb();
+	r6 = READ_ONCE(*pwd);
+	smp_rmb();
+	r8 = READ_ONCE(*fseq);
+}
+
+filter (2:r7=0 /\ 2:r8=0 \/ 2:r7=2 /\ 2:r8=2)
+exists (1:r1=1 /\ 2:r5=4 /\ 2:r6=0)

-- 
2.53.0

[PATCH 8/9] ipc: free the namespace after a grace period

[Christian Brauner]

Since commit 74b24a582e1f ("ipc: support ns lookup") put_ipc_ns()
removes the namespace from the namespace tree before it hands it to
free_ipc(). But free_ipc_ns() still frees it with a plain kfree(). The
namespace tree is walked under rcu_read_lock() though. lookup_ns_id(),
listns() and the decoding of nsfs file handles find namespaces
locklessly and read their reference counts.

Every other namespace type frees with kfree_rcu() or call_rcu() after
ns_tree_remove(). The ipc namespace got away with kfree() because
free_ipc() waits for a grace period for the mqueue mount before it frees
the batch. That grace period goes away in the next patch.

So free the namespace after a grace period of its own.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 ipc/namespace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ipc/namespace.c b/ipc/namespace.c
index 1e71353bdb4a..76df11e0b134 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -164,7 +164,7 @@ static void free_ipc_ns(struct ipc_namespace *ns)
 	dec_ipc_namespaces(ns->ucounts);
 	put_user_ns(ns->user_ns);
 	ns_common_free(ns);
-	kfree(ns);
+	kfree_rcu(ns, ns.ns_rcu);
 }
 
 static LLIST_HEAD(free_ipc_list);

-- 
2.53.0

[PATCH 9/9] ipc: unmount the mqueue mount with kern_unmount()

[Christian Brauner]

free_ipc() open-codes kern_unmount() for the whole batch of dying
namespaces. Just let free_ipc_ns() call kern_unmount() instead. A
namespace whose queues are still open elsewhere now waits for an
expedited grace period of its own instead of the normal one for the
whole batch.

Currently free_ipc() pays a normal grace period for every batch of
dying ipc namespaces. That's one of the two normal grace periods a
podman run --rm costs per container. The other one is for the network
namespace.

That leaves kern_unmount_array() as the only user of
mnt_make_shortterm() so make it static. While at it, update the comment
above the work queue. It isn't there to avoid synchronize_rcu() anymore.

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/namespace.c        |  2 +-
 include/linux/mount.h |  1 -
 ipc/namespace.c       | 15 +++------------
 3 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index af36cce922a2..e6b3a3206f3a 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1602,7 +1602,7 @@ EXPORT_SYMBOL(mntget);
  * it finds under mount_lock that it holds the only reference left,
  * see mntput_unheld().
  */
-void mnt_make_shortterm(struct vfsmount *mnt)
+static void mnt_make_shortterm(struct vfsmount *mnt)
 {
 	if (mnt)
 		WRITE_ONCE(real_mount(mnt)->mnt_ns, NULL);
diff --git a/include/linux/mount.h b/include/linux/mount.h
index b16ba5d45605..2342929e2fff 100644
--- a/include/linux/mount.h
+++ b/include/linux/mount.h
@@ -74,7 +74,6 @@ extern void mnt_drop_write(struct vfsmount *mnt);
 extern void mnt_drop_write_file(struct file *file);
 extern void mntput(struct vfsmount *mnt);
 extern struct vfsmount *mntget(struct vfsmount *mnt);
-extern void mnt_make_shortterm(struct vfsmount *mnt);
 extern struct vfsmount *mnt_clone_internal(const struct path *path);
 extern bool __mnt_is_readonly(const struct vfsmount *mnt);
 extern bool mnt_may_suid(struct vfsmount *mnt);
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 76df11e0b134..24706b910654 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -21,7 +21,8 @@
 #include "util.h"
 
 /*
- * The work queue is used to avoid the cost of synchronize_rcu in kern_unmount.
+ * The work queue keeps the teardown of a namespace out of the callers of
+ * put_ipc_ns(), the eviction of an mqueue inode among them.
  */
 static void free_ipc(struct work_struct *unused);
 static DECLARE_WORK(free_ipc_work, free_ipc);
@@ -149,11 +150,7 @@ void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
 
 static void free_ipc_ns(struct ipc_namespace *ns)
 {
-	/*
-	 * Caller needs to wait for an RCU grace period to have passed
-	 * after making the mount point inaccessible to new accesses.
-	 */
-	mntput(ns->mq_mnt);
+	kern_unmount(ns->mq_mnt);
 	sem_exit_ns(ns);
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
@@ -173,12 +170,6 @@ static void free_ipc(struct work_struct *unused)
 	struct llist_node *node = llist_del_all(&free_ipc_list);
 	struct ipc_namespace *n, *t;
 
-	llist_for_each_entry_safe(n, t, node, mnt_llist)
-		mnt_make_shortterm(n->mq_mnt);
-
-	/* Wait for any last users to have gone away. */
-	synchronize_rcu();
-
 	llist_for_each_entry_safe(n, t, node, mnt_llist)
 		free_ipc_ns(n);
 }

-- 
2.53.0

Re: [PATCH 3/9] fs: drop the caller's umount(2) reference under mount_lock

[Al Viro]

On Mon, Oct 12, 2026 at 05:39:16PM +0200, Christian Brauner wrote:

> +DEFINE_FREE(mntput_no_expire, struct mount *, if (_T) mntput_no_expire(_T))

You probably want to make it clear in commit message that this is _not_ the
decrement-under-lock referred to by the subject...

>  static int do_umount(struct mount *mnt, int flags)
>  {
> +	struct mount *ref __free(mntput_no_expire) = mnt;

... since it's outside of guard(mount_writer()) scope, thankfully (it would
obviously break under mount_lock).

Re: [PATCH 4/9] fs: unmount without a grace period when nothing else holds the mounts

[Al Viro]

On Mon, Oct 12, 2026 at 05:39:17PM +0200, Christian Brauner wrote:

>  fs/namespace.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++---------
>  1 file changed, 79 insertions(+), 13 deletions(-)
> 
> diff --git a/fs/namespace.c b/fs/namespace.c
> index 8c6673bd8c03..435e5f6db880 100644
> --- a/fs/namespace.c
> +++ b/fs/namespace.c
> @@ -1489,10 +1489,12 @@ static void mntput_no_expire(struct mount *mnt)
>  		 * Since we don't do lock_mount_hash() here,
>  		 * ->mnt_ns can change under us.  However, if it's
>  		 * non-NULL, then there's a reference that won't
> -		 * be dropped until after an RCU delay done after
> -		 * turning ->mnt_ns NULL.  So if we observe it
> -		 * non-NULL under rcu_read_lock(), the reference
> -		 * we are dropping is not the final one.
> +		 * be dropped until after turning ->mnt_ns NULL and
> +		 * either an RCU delay or a look under mount_lock
> +		 * that found it to be the only one left, which it
> +		 * isn't while ours is counted (mntput_unheld()).

Umm....   something like "either an RCU delay or having mntput_unheld()
observe the refcount to be 1 in a mount_lock exclusive scope.  The latter
can't happen until the mnt_dec_count() below", perhaps?  Probably still
too convoluted...

I'm not trying to nitpick - the place _is_ subtle and we are pretty
much guaranteed to look at it quite a few times down the road...

So basically you are relying upon the following:
1) RCU references to mounts are only used inside RCU read-side
critical scopes that have
	* mount_lock sampled prior to having observed the mount
in a live state
	* __legitimize_mnt() with the sampled value of mount_lock
done to obtain a counting reference
2) there's an RCU delay between having been observed live and freeing.
3) ->mnt_parent of a something that had been live in an RCU scope will
point to something that had been live in the same RCU scope.
4) ->root.mnt and ->pwd.mnt observed under ->seq are live.
5) anything observed in mount hash under mount_lock is live.

Might work...  FWIW, I wonder (again) if we should replace the check for
mnt->mnt.mnt_flags & MNT_DOOMED in the slow path of mntput_no_expire()
with WARN_ON() - it really should never happen.