Linux driver-core infrastructure
 help / color / mirror / Atom feed
* [PATCH 0/6] kernfs: stop taking kernfs_iattr_rwsem for writing in mkdir/rmdir
@ 2026-11-18 22:01 Shakeel Butt
  2026-11-18 22:01 ` [PATCH 1/6] kernfs: report times and link count from the node in getattr Shakeel Butt
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Shakeel Butt @ 2026-11-18 22:01 UTC (permalink / raw)
  To: Christian Brauner
  Cc: Greg Kroah-Hartman, Tejun Heo, Jori Koolstra, Ian Kent,
	T . J . Mercier, Meta kernel team, linux-fsdevel, driver-core,
	linux-kernel

In the Meta fleet, cgroup mkdir and rmdir can stall cgroup operations
on the whole host.  They hold cgroup_mutex and kernfs_rwsem while they
wait for kernfs_iattr_rwsem, which every path walk takes for reading,
and they take it once or twice for every file of the cgroup.  systemd,
clone3() into a cgroup, BPF queries and the OOM daemon all wait behind
them.

- Our lock monitor saw a kernfs_rwsem writer blocked on
  kernfs_iattr_rwsem several thousand times on a large number of hosts
  in 29 days (in mkdir and rmdir), for up to 1.8 s at a time.

- Under memory and CPU pressure the waits become host-wide stalls.  In
  one week, we observed several cases where the OOM daemon and the
  monitoring agent were stuck for up to 22 minutes.  In October, one
  batch job caused these stalls on a large number of hosts.  On the
  sampled hosts with current kernels, the cgroup_mutex owner was always
  waiting for kernfs_iattr_rwsem (in mkdir and rmdir).  One specific
  stall lasted 90 minutes and ended in a reboot.

The write lock is not needed for most of what it covers:

- kernfs_rwsem already serialises the parent's two counters and
  KERNFS_REMOVING (patches 3 and 4).
- The parent's times get a seqlock of their own (patch 5).
- Only inode setup needs to set a directory's i_nlink, so removal can
  clear it without the lock (patches 2 and 6).

The lock also kept times and link counts stable while getattr and
permission copied them into the shared inode.  Patches 1 and 2 make
getattr report them from the node and stop copying them, so stat()
cannot return a stale or torn value.

After the series, mkdir and rmdir take no kernfs_iattr_rwsem for
writing.  Write acquisitions, from lock_stat:

                                          before   after
  5 dummy netdevs created and removed       1850       0
  100 cgroup mkdirs and 100 rmdirs          7500       0

Latency in a VM with 8 vCPUs, while 32 threads walk cgroupfs paths
(worst of three 20 s runs; mkdir and rmdir by a SCHED_FIFO thread):

                         before                  after
                    p50     p99     max     p50    p99    max
  cgroup mkdir     2.8ms   9.0ms  20.1ms    25us   37us   84us
  cgroup rmdir     4.2ms  12.7ms  26.6ms    28us   38us  107us

A write that only needs cgroup_mutex waited up to 23 ms before and up
to 34 us after.  With 8 threads and no writer, stat() and access() on
a path whose directories have iattrs are 12-14% faster, since the
refresh copies less.

Tested with the kernfs selftests on lockdep + KASAN and KCSAN kernels;
with extra checks of link counts, times and IN_DELETE_SELF; with
concurrent mkdir/rmdir, setattr, stat, readdir and netdev churn; and
with test-only delays that force the stale stat(), lost time update,
backwards time, overwritten setattr and stat()-during-removal races.
KCSAN reports nothing new, and its set_nlink() reports from the refresh
are gone.

Based on vfs-7.5.kernfs.  It also applies to v7.4-rc3.

Shakeel Butt (6):
  kernfs: report times and link count from the node in getattr
  kernfs: don't copy times and link count into the inode on refresh
  kernfs: don't take kernfs_iattr_rwsem for the directory counters
  kernfs: don't take kernfs_iattr_rwsem to publish KERNFS_REMOVING
  kernfs: update a directory's times without kernfs_iattr_rwsem
  kernfs: don't take kernfs_iattr_rwsem in __kernfs_remove()

 fs/kernfs/dir.c             |  48 +++++---------
 fs/kernfs/inode.c           | 125 ++++++++++++++++++++++++++++++------
 fs/kernfs/kernfs-internal.h |   2 +
 3 files changed, 124 insertions(+), 51 deletions(-)


base-commit: ccf2a4692fed0fa77751d32d2690585127a152c5
-- 
2.53.1-Meta


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-18 22:02 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 22:01 [PATCH 0/6] kernfs: stop taking kernfs_iattr_rwsem for writing in mkdir/rmdir Shakeel Butt
2026-11-18 22:01 ` [PATCH 1/6] kernfs: report times and link count from the node in getattr Shakeel Butt
2026-11-18 22:01 ` [PATCH 2/6] kernfs: don't copy times and link count into the inode on refresh Shakeel Butt
2026-11-18 22:01 ` [PATCH 3/6] kernfs: don't take kernfs_iattr_rwsem for the directory counters Shakeel Butt
2026-11-18 22:01 ` [PATCH 4/6] kernfs: don't take kernfs_iattr_rwsem to publish KERNFS_REMOVING Shakeel Butt
2026-11-18 22:01 ` [PATCH 5/6] kernfs: update a directory's times without kernfs_iattr_rwsem Shakeel Butt
2026-11-18 22:01 ` [PATCH 6/6] kernfs: don't take kernfs_iattr_rwsem in __kernfs_remove() Shakeel Butt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox