[PATCH] cgroup: Move cgrp_dead_ task+iwork into its own struct

[PATCH] cgroup: Move cgrp_dead_ task+iwork into its own struct

[Sebastian Andrzej Siewior]

cgrp_dead_tasks and cgrp_dead_tasks_iwork are independent per-CPU
variables and are independently accessed via a this_cpu_ptr().

Having a custom struct with those two members makes it possible to have
only one per-CPU accessor and access the second member via an offset.
This makes the code a bit more compact and is micro-optimization.

Besides that, it allows cgrp_dead_tasks_iwork_fn() to access the list
pointer via the passed iwork pointer. This not only eliminates the
this_cpu_ptr() but also makes it possible to be invoked on a different
CPU. This does not happen as of today but might if I can flush the
irq_work items from another CPU during CPU-hotplug events.

Create struct cgroup_dead with the per-CPU variables as members. Access
the struct with a per-CPU accessor and use the cointer_of() in the
irq_work callback.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/cgroup/cgroup.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 2d532bf2c0c7a..5277274f48b54 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -7194,15 +7194,19 @@ static void do_cgroup_task_dead(struct task_struct *tsk)
  * the cgroup and task_struct can be pinned indefinitely. Bounce through lazy
  * irq_work to allow batching while ensuring timely completion.
  */
-static DEFINE_PER_CPU(struct llist_head, cgrp_dead_tasks);
-static DEFINE_PER_CPU(struct irq_work, cgrp_dead_tasks_iwork);
+struct cgroup_dead {
+	struct irq_work		iwork;
+	struct llist_head	tasks;
+};
+static DEFINE_PER_CPU(struct cgroup_dead, cgroup_dead);
 
 static void cgrp_dead_tasks_iwork_fn(struct irq_work *iwork)
 {
+	struct cgroup_dead *cgrp_dead = container_of(iwork, struct cgroup_dead, iwork);
 	struct llist_node *lnode;
 	struct task_struct *task, *next;
 
-	lnode = llist_del_all(this_cpu_ptr(&cgrp_dead_tasks));
+	lnode = llist_del_all(&cgrp_dead->tasks);
 	llist_for_each_entry_safe(task, next, lnode, cg_dead_lnode) {
 		do_cgroup_task_dead(task);
 		put_task_struct(task);
@@ -7214,17 +7218,20 @@ static void __init cgroup_rt_init(void)
 	int cpu;
 
 	for_each_possible_cpu(cpu) {
-		init_llist_head(per_cpu_ptr(&cgrp_dead_tasks, cpu));
-		per_cpu(cgrp_dead_tasks_iwork, cpu) =
-			IRQ_WORK_INIT_LAZY(cgrp_dead_tasks_iwork_fn);
+		struct cgroup_dead *cgrp_dead = per_cpu_ptr(&cgroup_dead, cpu);
+
+		init_llist_head(&cgrp_dead->tasks);
+		cgrp_dead->iwork = IRQ_WORK_INIT_LAZY(cgrp_dead_tasks_iwork_fn);
 	}
 }
 
 void cgroup_task_dead(struct task_struct *task)
 {
+	struct cgroup_dead *cgrp_dead = this_cpu_ptr(&cgroup_dead);
+
 	get_task_struct(task);
-	llist_add(&task->cg_dead_lnode, this_cpu_ptr(&cgrp_dead_tasks));
-	irq_work_queue(this_cpu_ptr(&cgrp_dead_tasks_iwork));
+	llist_add(&task->cg_dead_lnode, &cgrp_dead->tasks);
+	irq_work_queue(&cgrp_dead->iwork);
 }
 #else	/* CONFIG_PREEMPT_RT */
 static void __init cgroup_rt_init(void) {}
-- 
2.55.0

Re: [PATCH] cgroup: Move cgrp_dead_ task+iwork into its own struct

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

On Fri, Sep 11, 2026 at 12:19:00PM +0200, Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:
> cgrp_dead_tasks and cgrp_dead_tasks_iwork are independent per-CPU
> variables and are independently accessed via a this_cpu_ptr().
> 
> Having a custom struct with those two members makes it possible to have
> only one per-CPU accessor and access the second member via an offset.
> This makes the code a bit more compact and is micro-optimization.
> 
> Besides that, it allows cgrp_dead_tasks_iwork_fn() to access the list
> pointer via the passed iwork pointer. This not only eliminates the
> this_cpu_ptr() but also makes it possible to be invoked on a different
> CPU. This does not happen as of today but might if I can flush the
> irq_work items from another CPU during CPU-hotplug events.
> 
> Create struct cgroup_dead with the per-CPU variables as members. Access
> the struct with a per-CPU accessor and use the cointer_of() in the
> irq_work callback.
> 
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
>  kernel/cgroup/cgroup.c | 23 +++++++++++++++--------
>  1 file changed, 15 insertions(+), 8 deletions(-)

Acked-by: Michal Koutný <mkoutny@suse.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]

Re: [PATCH] cgroup: Move cgrp_dead_ task+iwork into its own struct

[Tejun Heo]

Applied to cgroup/for-7.4 with Michal's Acked-by and description typo fixes.

Thanks.
--
tejun