From: "Harry Yoo (Meta)" <harry@kernel.org>
To: Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@kernel.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Shakeel Butt <shakeel.butt@linux.dev>,
Muchun Song <muchun.song@linux.dev>,
Andrew Morton <akpm@linux-foundation.org>,
Harry Yoo <harry@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Vlastimil Babka <vbabka@kernel.org>, Hao Li <hao.li@linux.dev>,
Christoph Lameter <cl@gentwo.org>,
David Rientjes <rientjes@google.com>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Clark Williams <clrkwllms@kernel.org>,
Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@meta.com, cgroups@vger.kernel.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
linux-rt-devel@lists.linux.dev
Subject: [PATCH for-next v3 1/7] mm/memcontrol: defer obj_cgroup_release() to irq_work
Date: Sun, 15 Nov 2026 15:09:35 +0000 [thread overview]
Message-ID: <20261115-kmalloc-nolock-fixes-v3-1-9034dd4ecaab@kernel.org> (raw)
In-Reply-To: <20261115-kmalloc-nolock-fixes-v3-0-9034dd4ecaab@kernel.org>
obj_cgroup_put() can be called from any context.
__memcg_slab_free_hook() puts a per-object reference on
every accounted free, and draining a per-cpu objcg stock slot puts
the cached reference. When it drops the last reference,
obj_cgroup_release() takes objcg_lock, frees percpu memory
(pcpu_lock, via percpu_ref_exit()) and calls kfree_rcu(). Each of
these can deadlock when the current context interrupted a holder of
the respective lock.
Defer the release to irq_work. llist_add() and irq_work_queue() are
NMI-safe, and the irq_work runs in a context where acquiring a
spinlock is allowed.
Add a separate llist_node instead of reusing the list/rcu union
because the list stays linked until the deferred release. struct
obj_cgroup grows from 56 to 64 bytes, but both are served from
kmalloc-64, so no memory is additionally consumed.
Fixes: af92793e52c3 ("slab: Introduce kmalloc_nolock() and kfree_nolock().")
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Signed-off-by: Harry Yoo (Meta) <harry@kernel.org>
---
include/linux/memcontrol.h | 2 ++
mm/memcontrol.c | 27 ++++++++++++++++++++++++---
2 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 69e1bcc67c58..4d5c634daef8 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -185,6 +185,8 @@ struct obj_cgroup {
struct list_head list; /* protected by objcg_lock */
struct rcu_head rcu;
};
+ /* Used to defer obj_cgroup_release() to irq_work */
+ struct llist_node release_node;
bool is_root;
};
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index a05d9a8baa1b..c53147a88580 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -147,9 +147,8 @@ static void memcg_uncharge_kmem(struct mem_cgroup *memcg, unsigned int nr_pages)
memcg_uncharge(memcg, nr_pages);
}
-static void obj_cgroup_release(struct percpu_ref *ref)
+static void __obj_cgroup_release(struct obj_cgroup *objcg)
{
- struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt);
unsigned int nr_bytes;
unsigned int nr_pages;
unsigned long flags;
@@ -190,10 +189,32 @@ static void obj_cgroup_release(struct percpu_ref *ref)
list_del(&objcg->list);
spin_unlock_irqrestore(&objcg_lock, flags);
- percpu_ref_exit(ref);
+ percpu_ref_exit(&objcg->refcnt);
kfree_rcu(objcg, rcu);
}
+static LLIST_HEAD(objcg_release_llist);
+
+static void objcg_release_workfn(struct irq_work *work)
+{
+ struct llist_node *head = llist_del_all(&objcg_release_llist);
+ struct obj_cgroup *objcg, *tmp;
+
+ llist_for_each_entry_safe(objcg, tmp, head, release_node)
+ __obj_cgroup_release(objcg);
+}
+
+static DEFINE_IRQ_WORK(objcg_release_work, objcg_release_workfn);
+
+static void obj_cgroup_release(struct percpu_ref *ref)
+{
+ struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt);
+
+ /* The last put may come from a context that cannot take spinlocks */
+ if (llist_add(&objcg->release_node, &objcg_release_llist))
+ irq_work_queue(&objcg_release_work);
+}
+
static struct obj_cgroup *obj_cgroup_alloc(void)
{
struct obj_cgroup *objcg;
--
2.54.0
next prev parent reply other threads:[~2026-11-15 15:10 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-15 15:09 [PATCH for-next v3 0/7] memcg,slab: kmalloc_nolock() fixes Harry Yoo (Meta)
2026-11-15 15:09 ` Harry Yoo (Meta) [this message]
2026-11-15 15:09 ` [PATCH for-next v3 2/7] mm/slab: pass free_flags to slab_free_hook() Harry Yoo (Meta)
2026-11-15 15:09 ` [PATCH for-next v3 3/7] mm/slab: handle SLAB_FREE_NOLOCK in slab_free_hook() instead of open coding Harry Yoo (Meta)
2026-11-16 8:26 ` sashiko-bot
2026-11-15 15:09 ` [PATCH for-next v3 4/7] mm/slab: fix a deadlock in memcg_alloc_abort_single() Harry Yoo (Meta)
2026-11-15 15:09 ` [PATCH for-next v3 5/7] mm/slab: serialize deferred_work_barrier() Harry Yoo (Meta)
2026-11-15 15:09 ` [PATCH for-next v3 6/7] mm/slab: refactor free_to_pcs() Harry Yoo (Meta)
2026-11-15 15:09 ` [PATCH for-next v3 7/7] mm/slab: call debug_locks and KCSAN checks in kfree_nolock() Harry Yoo (Meta)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261115-kmalloc-nolock-fixes-v3-1-9034dd4ecaab@kernel.org \
--to=harry@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=ast@kernel.org \
--cc=bigeasy@linutronix.de \
--cc=cgroups@vger.kernel.org \
--cc=cl@gentwo.org \
--cc=clrkwllms@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=hao.li@linux.dev \
--cc=kernel-team@meta.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=mhocko@kernel.org \
--cc=muchun.song@linux.dev \
--cc=rientjes@google.com \
--cc=roman.gushchin@linux.dev \
--cc=rostedt@goodmis.org \
--cc=shakeel.butt@linux.dev \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.