From: Shakeel Butt <shakeel.butt@linux.dev>
To: Christian Brauner <brauner@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Tejun Heo <tj@kernel.org>, Jori Koolstra <jkoolstra@xs4all.nl>,
Meta kernel team <kernel-team@meta.com>,
cgroups@vger.kernel.org, linux-fsdevel@vger.kernel.org,
driver-core@lists.linux.dev, linux-kernel@vger.kernel.org
Subject: [PATCH] kernfs: allocate in open before taking the active reference
Date: Thu, 19 Nov 2026 11:19:22 -0800 [thread overview]
Message-ID: <20261119191923.2223993-1-shakeel.butt@linux.dev> (raw)
kernfs_fop_open() allocates the open file, its seq_file and the open
node while holding the file's active reference. The seq_file is
charged to the opener's memory cgroup, so in a cgroup at its limit the
allocation reclaims. Removing the file waits for active references.
cgroup rmdir does that with cgroup_mutex held, so every cgroup
operation on the host waits for that reclaim. Deleting a netdev waits
the same way with rtnl_lock held.
Allocate them before taking the active reference. kn->flags already
says whether the file has ->seq_show, which is all seq_open() needs.
If the file is removed in the meantime, open fails with -ENODEV as
before. Only the prealloc buffer, whose size comes from the ops, and
->open() still allocate under the active reference.
In a VM, eight tasks in a 256 MB memory cgroup at its limit, with swap
writes taking 500 ms, open and read small files of a cgroup that
another task removes and recreates every 20 ms (60 s, worst of four
runs):
before after
reclaim with an active reference held 5.0 s 0
rmdir waiting for it, cgroup_mutex held 377 ms 6.3 ms
mkdir/rmdir of other cgroups 382 ms 125 ms
The rest of the mkdir/rmdir time is waiting for kernfs_rwsem. Opening
and closing cgroup files in a loop is as fast as before.
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
---
This applies on vfs-7.5.kernfs, with or without my kernfs_iattr_rwsem
series [1].
[1] https://lore.kernel.org/all/20261118220154.2965621-1-shakeel.butt@linux.dev/
fs/kernfs/file.c | 106 ++++++++++++++++++++++++++---------------------
1 file changed, 59 insertions(+), 47 deletions(-)
diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c
index 10ee357b388e..08c29eb3f435 100644
--- a/fs/kernfs/file.c
+++ b/fs/kernfs/file.c
@@ -512,9 +512,11 @@ static int kernfs_fop_mmap(struct file *file, struct vm_area_struct *vma)
* kernfs_get_open_node - get or create kernfs_open_node
* @kn: target kernfs_node
* @of: kernfs_open_file for this instance of open
+ * @new_on: kernfs_open_node allocated by the caller, or %NULL
*
* If @kn->attr.open exists, increment its reference count; otherwise,
- * create one. @of is chained to the files list.
+ * create one, from @new_on if given. @of is chained to the files list.
+ * @new_on is consumed.
*
* Locking:
* Kernel thread context (may sleep).
@@ -523,19 +525,12 @@ static int kernfs_fop_mmap(struct file *file, struct vm_area_struct *vma)
* %0 on success, -errno on failure.
*/
static int kernfs_get_open_node(struct kernfs_node *kn,
- struct kernfs_open_file *of)
+ struct kernfs_open_file *of,
+ struct kernfs_open_node *new_on)
{
- struct kernfs_open_node *on, *new_on = NULL;
+ struct kernfs_open_node *on;
struct mutex *mutex;
- /*
- * Peek without the mutex: if nothing has this open, we will need a
- * node and can allocate before taking a mutex shared by every node
- * hashing to it.
- */
- if (!rcu_access_pointer(kn->attr.open))
- new_on = kzalloc_obj(*new_on);
-
mutex = kernfs_open_file_mutex_lock(kn);
on = kernfs_deref_open_node_locked(kn);
@@ -544,7 +539,7 @@ static int kernfs_get_open_node(struct kernfs_node *kn,
on = new_on;
new_on = NULL;
if (!on) {
- /* the peek raced; rare, so allocate here */
+ /* the caller's peek raced; rare, so allocate here */
on = kzalloc_obj(*on);
if (!on) {
mutex_unlock(mutex);
@@ -618,13 +613,52 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)
{
struct kernfs_node *kn = inode->i_private;
struct kernfs_root *root = kernfs_root(kn);
+ struct kernfs_open_node *new_on = NULL;
const struct kernfs_ops *ops;
struct kernfs_open_file *of;
bool has_read, has_write, has_mmap, no_splice_write;
- int error = -EACCES;
+ int error;
+
+ /*
+ * Allocate before taking the active reference. Removing @kn waits
+ * for active references, often with a global lock held (cgroup rmdir
+ * holds cgroup_mutex), and the seq_file is charged to the opener's
+ * memory cgroup, so allocating can wait for reclaim. Only the
+ * prealloc buffer, which needs @ops, and ->open() allocate under the
+ * active reference.
+ */
+ of = kzalloc_obj(struct kernfs_open_file);
+ if (!of)
+ return -ENOMEM;
+
+ /*
+ * Always instantiate seq_file even if read access doesn't use
+ * seq_file or is not requested. This unifies private data access
+ * and readable regular files are the vast majority anyway.
+ * kn->flags tells whether there is ->seq_show, without @ops. Other
+ * bits of it can change meanwhile, hence READ_ONCE().
+ */
+ if (READ_ONCE(kn->flags) & KERNFS_HAS_SEQ_SHOW)
+ error = seq_open(file, &kernfs_seq_ops);
+ else
+ error = seq_open(file, NULL);
+ if (error)
+ goto err_free;
+ of->seq_file = file->private_data;
+ of->seq_file->private = of;
+
+ /* seq_file clears PWRITE unconditionally, restore it if WRITE */
+ if (file->f_mode & FMODE_WRITE)
+ file->f_mode |= FMODE_PWRITE;
+
+ /* if nothing has @kn open, we will need a new open node too */
+ if (!rcu_access_pointer(kn->attr.open))
+ new_on = kzalloc_obj(*new_on);
+
+ error = -ENODEV;
if (!kernfs_get_active(kn))
- return -ENODEV;
+ goto err_seq_release;
ops = kernfs_ops(kn);
@@ -633,22 +667,17 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)
has_mmap = ops->mmap;
/* see the flag definition for details */
+ error = -EACCES;
if (root->flags & KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK) {
if ((file->f_mode & FMODE_WRITE) &&
(!(inode->i_mode & S_IWUGO) || !has_write))
- goto err_out;
+ goto err_put_active;
if ((file->f_mode & FMODE_READ) &&
(!(inode->i_mode & S_IRUGO) || !has_read))
- goto err_out;
+ goto err_put_active;
}
- /* allocate a kernfs_open_file for the file */
- error = -ENOMEM;
- of = kzalloc_obj(struct kernfs_open_file);
- if (!of)
- goto err_out;
-
/*
* Splice callbacks need this before the iter paths acquire an active
* reference, so cache it while @kn is active.
@@ -707,31 +736,16 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)
* ->read must be used instead.
*/
if (ops->prealloc && ops->seq_show)
- goto err_free;
+ goto err_put_active;
if (ops->prealloc) {
int len = of->atomic_write_len ?: PAGE_SIZE;
of->prealloc_buf = kmalloc(len + 1, GFP_KERNEL);
error = -ENOMEM;
if (!of->prealloc_buf)
- goto err_free;
+ goto err_put_active;
mutex_init(&of->prealloc_mutex);
}
- /*
- * Always instantiate seq_file even if read access doesn't use
- * seq_file or is not requested. This unifies private data access
- * and readable regular files are the vast majority anyway.
- */
- if (ops->seq_show)
- error = seq_open(file, &kernfs_seq_ops);
- else
- error = seq_open(file, NULL);
- if (error)
- goto err_free;
-
- of->seq_file = file->private_data;
- of->seq_file->private = of;
-
/*
* seq_read_iter() takes @of->mutex under the seq_file lock, whose key
* is shared with seq_files that can be spliced. Separate it too.
@@ -742,14 +756,11 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)
lockdep_set_class(&of->seq_file->lock, &seq_no_splice_key);
}
- /* seq_file clears PWRITE unconditionally, restore it if WRITE */
- if (file->f_mode & FMODE_WRITE)
- file->f_mode |= FMODE_PWRITE;
-
/* make sure we have open node struct */
- error = kernfs_get_open_node(kn, of);
+ error = kernfs_get_open_node(kn, of, new_on);
+ new_on = NULL;
if (error)
- goto err_seq_release;
+ goto err_put_active;
if (ops->open) {
/* nobody has access to @of yet, skip @of->mutex */
@@ -764,13 +775,14 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)
err_put_node:
kernfs_unlink_open_file(kn, of, true);
+err_put_active:
+ kernfs_put_active(kn);
err_seq_release:
+ kfree(new_on);
seq_release(inode, file);
err_free:
kfree(of->prealloc_buf);
kfree(of);
-err_out:
- kernfs_put_active(kn);
return error;
}
base-commit: ccf2a4692fed0fa77751d32d2690585127a152c5
--
2.53.1-Meta
reply other threads:[~2026-11-19 19:19 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119191923.2223993-1-shakeel.butt@linux.dev \
--to=shakeel.butt@linux.dev \
--cc=brauner@kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=driver-core@lists.linux.dev \
--cc=gregkh@linuxfoundation.org \
--cc=jkoolstra@xs4all.nl \
--cc=kernel-team@meta.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox