[PATCH v4 0/5] sched/debug: Introduce per-CPU debugfs files

[PATCH v4 0/5] sched/debug: Introduce per-CPU debugfs files

[Aaron Tomlin]

Hi Peter, Juri, Ingo, Vincent,

This patch series addresses a few pre-existing memory safety and list
traversal concurrency issues in scheduler debugfs handlers, and introduces
per-CPU debugfs files under /sys/kernel/debug/sched/cpu/cpu<N>/debug.

Patch 1 fixes a potential use-after-free in print_cpu() where rq->curr is
dereferenced locklessly to output the running task's PID. If the task exits
concurrently and its reference count drops to zero, put_task_struct()
schedules __put_task_struct_rcu_cb() via call_rcu(). Without holding an RCU
read lock, an RCU grace period can elapse concurrently and free the task
structure via free_task(), leading to a use-after-free race condition. This
patch protects rq->curr access using rcu_dereference() inside an RCU
read-side critical section, ensuring sparse compliance.

Patch 2 fixes a use-after-free in print_dl_rq() where cpu_rq(cpu)->rd is
dereferenced locklessly to display deadline bandwidth statistics. During
CPU hot-unplug or cgroup cpuset repartitioning events,
partition_sched_domains() calls rq_attach_root() to detach the CPU from its
root_domain and schedules free_rootdomain() via call_rcu(). Without an RCU
read lock, an RCU grace period can resolve concurrently while debugfs reads
the file, allowing free_rootdomain() to execute kfree() and causing a UAF
when reading dl_bw->bw. This patch adds rcu_assign_pointer() on the writer
side in rq_attach_root() and fetches rq->rd using READ_ONCE() inside an RCU
read-side critical section in print_dl_rq().

Patch 3 fixes a time-of-check to time-of-use race condition in
sched_show_numa(), where p->mm is checked locklessly and then passed to
P(mm->numa_scan_seq), causing a double-evaluation of p->mm. If the task
exits concurrently (exit_mm(p)), p->mm can be set to NULL on another CPU
between the check and the macro expansion, causing a NULL-pointer
dereference crash. This patch reads p->mm once into a local variable using
READ_ONCE(p->mm) before checking and dereferencing its numa_scan_seq field.

Patch 4 fixes an RCU traversal violation in print_cfs_stats() where
rq->leaf_cfs_rq_list is traversed locklessly using
for_each_leaf_cfs_rq_safe(), which expands to list_for_each_entry_safe().
Although leaf_cfs_rq_list is modified using list_add_rcu(),
list_for_each_entry_safe() lacks READ_ONCE(), allowing compiler reordering
or re-fetching that can cause readers to observe newly inserted cfs_rq
nodes before internal fields are initialised. This patch introduces
for_each_leaf_cfs_rq_rcu() using list_for_each_entry_rcu() and adds a hard
iteration ceiling to prevent RCU stalls under list churn.

Patch 5 introduces per-CPU debugfs entries under
/sys/kernel/debug/sched/cpu/, allowing targeted inspection of an individual
CPU's runqueue on demand. If the target CPU is currently offline, reading
its file returns -ENODEV.


Changes since v3:

 - Updated Patch 1 to use rcu_dereference(rq->curr) instead of READ_ONCE()
   to preserve __rcu

 - Added missing writer-side RCU publication barrier (rcu_assign_pointer())
   in rq_attach_root() for Patch 2

 - Added Patch 3 to fix a TOCTOU condition in sched_show_numa() using
   READ_ONCE(p->mm)

 - Added a safety iteration ceiling in print_cfs_stats() for Patch 4 to
   prevent unbounded list iteration and RCU stalls under heavy
   leaf_cfs_rq_list churn

 - Linked to v3: https://lore.kernel.org/lkml/20260808235522.380038-1-atomlin@atomlin.com/

Changes since v2:

 - Protected lockless rq->curr dereferencing in print_cpu() with
   rcu_read_lock() and READ_ONCE()

 - Protected lockless rq->rd dereferencing in print_dl_rq() against CPU
   hot-unplug and cgroup cpuset repartitioning races

 - Introduced for_each_leaf_cfs_rq_rcu() using list_for_each_entry_rcu()
   for lockless leaf_cfs_rq_list iteration

 - Linked to v2: https://lore.kernel.org/lkml/20260728205238.18447-1-atomlin@atomlin.com/

Changes since v1:

 - Reframed commit message motivation around targeted interactive
   debugging on large SMP topologies (Peter Zijlstra and Zhan Xusheng)

 - Gated sched_debug_cpu_show() with a cpu_online(cpu) check
   returning -ENODEV when target CPU is offline (Zhan Xusheng)

 - Linked to v1: https://lore.kernel.org/lkml/20260728020309.6169-1-atomlin@atomlin.com/

Aaron Tomlin (5):
  sched/debug: Protect lockless rq->rd access in print_dl_rq()
  sched/debug: Protect lockless rq->curr access in print_cpu()
  sched/debug: Protect lockless p->mm access in sched_show_numa()
  sched/fair: Use list_for_each_entry_rcu() in print_cfs_stats()
  sched/debug: Introduce per-CPU debugfs files

 kernel/sched/debug.c    | 66 +++++++++++++++++++++++++++++++++++++----
 kernel/sched/fair.c     | 17 +++++++++--
 kernel/sched/topology.c |  2 +-
 3 files changed, 76 insertions(+), 9 deletions(-)

-- 
2.55.0

[PATCH v4 1/5] sched/debug: Protect lockless rq->rd access in print_dl_rq()

[Aaron Tomlin]

In print_dl_rq(), cpu_rq(cpu)->rd is dereferenced locklessly to display
deadline bandwidth statistics.

During CPU hot-unplug or cgroup cpuset repartitioning events,
partition_sched_domains() calls cpu_attach_domain(), which executes
rq_attach_root() to detach the CPU from its root_domain. When the
reference count of the detached root_domain drops to zero,
rq_attach_root() calls call_rcu(&old_rd->rcu, free_rootdomain) to
schedule memory teardown after an RCU grace period.

However, rq_attach_root() previously updated rq->rd using a plain C store
without an RCU publication barrier (i.e., rcu_assign_pointer()). Without a
release memory barrier on the writer side, CPU or compiler reordering could
allow the new rq->rd pointer store to become visible to other CPUs before
the initialization writes to rd->dl_bw are committed.

Furthermore, because print_dl_rq() did not hold an RCU read lock while
dereferencing cpu_rq(cpu)->rd, an RCU grace period could elapse
concurrently while debugfs is reading the file, allowing
free_rootdomain() to execute kfree(old_rd) and causing a use-after-free
race condition when print_dl_rq() reads dl_bw->bw.

Resolve this by using rcu_assign_pointer(rq->rd, rd) in rq_attach_root() to
guarantee a release memory barrier when publishing a root_domain.
Finally, fetch rq->rd using READ_ONCE() inside an RCU read-side critical
section in print_dl_rq().

Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/debug.c    | 12 +++++++++---
 kernel/sched/topology.c |  2 +-
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 40584b27ea0c..2c2156dfab00 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -1081,6 +1081,7 @@ void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq)
 void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
 {
 	struct dl_bw *dl_bw;
+	struct root_domain *rd;
 
 	SEQ_printf(m, "\n");
 	SEQ_printf(m, "dl_rq[%d]:\n", cpu);
@@ -1089,9 +1090,14 @@ void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
 	SEQ_printf(m, "  .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x))
 
 	PU(dl_nr_running);
-	dl_bw = &cpu_rq(cpu)->rd->dl_bw;
-	SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
-	SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
+	rcu_read_lock();
+	rd = READ_ONCE(cpu_rq(cpu)->rd);
+	if (rd) {
+		dl_bw = &rd->dl_bw;
+		SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
+		SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
+	}
+	rcu_read_unlock();
 
 #undef PU
 }
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 622e2e01974c..b411cc00029c 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -496,7 +496,7 @@ void rq_attach_root(struct rq *rq, struct root_domain *rd)
 	}
 
 	atomic_inc(&rd->refcount);
-	rq->rd = rd;
+	rcu_assign_pointer(rq->rd, rd);
 
 	cpumask_set_cpu(rq->cpu, rd->span);
 	if (cpumask_test_cpu(rq->cpu, cpu_active_mask))
-- 
2.55.0

Re: [PATCH v4 1/5] sched/debug: Protect lockless rq->rd access in print_dl_rq()

[K Prateek Nayak]

Hello Aaron,

On 8/10/2026 7:28 AM, Aaron Tomlin wrote:
> @@ -1089,9 +1090,14 @@ void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
>  	SEQ_printf(m, "  .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x))
>  
>  	PU(dl_nr_running);
> -	dl_bw = &cpu_rq(cpu)->rd->dl_bw;
> -	SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
> -	SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
> +	rcu_read_lock();
> +	rd = READ_ONCE(cpu_rq(cpu)->rd);

nit. Why not rcu_dereference_all() here to keep symmetry with the
update side that now uses rcu_assign_pointer()?

> +	if (rd) {

I don't think rq->rd can ever be NULL after the scheduler has
initialized.

> +		dl_bw = &rd->dl_bw;
> +		SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
> +		SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
> +	}
> +	rcu_read_unlock();
>  
>  #undef PU
>  }

-- 
Thanks and Regards,
Prateek

Re: [PATCH v4 1/5] sched/debug: Protect lockless rq->rd access in print_dl_rq()

[Daniel Vacek]

On Mon, 10 Aug 2026 at 03:58, Aaron Tomlin <atomlin@atomlin.com> wrote:
> In print_dl_rq(), cpu_rq(cpu)->rd is dereferenced locklessly to display
> deadline bandwidth statistics.
>
> During CPU hot-unplug or cgroup cpuset repartitioning events,
> partition_sched_domains() calls cpu_attach_domain(), which executes
> rq_attach_root() to detach the CPU from its root_domain. When the
> reference count of the detached root_domain drops to zero,
> rq_attach_root() calls call_rcu(&old_rd->rcu, free_rootdomain) to
> schedule memory teardown after an RCU grace period.
>
> However, rq_attach_root() previously updated rq->rd using a plain C store
> without an RCU publication barrier (i.e., rcu_assign_pointer()). Without a
> release memory barrier on the writer side, CPU or compiler reordering could
> allow the new rq->rd pointer store to become visible to other CPUs before
> the initialization writes to rd->dl_bw are committed.
>
> Furthermore, because print_dl_rq() did not hold an RCU read lock while
> dereferencing cpu_rq(cpu)->rd, an RCU grace period could elapse
> concurrently while debugfs is reading the file, allowing
> free_rootdomain() to execute kfree(old_rd) and causing a use-after-free
> race condition when print_dl_rq() reads dl_bw->bw.
>
> Resolve this by using rcu_assign_pointer(rq->rd, rd) in rq_attach_root() to
> guarantee a release memory barrier when publishing a root_domain.
> Finally, fetch rq->rd using READ_ONCE() inside an RCU read-side critical
> section in print_dl_rq().
>
> Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file")
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
> ---
>  kernel/sched/debug.c    | 12 +++++++++---
>  kernel/sched/topology.c |  2 +-
>  2 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
> index 40584b27ea0c..2c2156dfab00 100644
> --- a/kernel/sched/debug.c
> +++ b/kernel/sched/debug.c
> @@ -1081,6 +1081,7 @@ void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq)
>  void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
>  {
>         struct dl_bw *dl_bw;
> +       struct root_domain *rd;
>
>         SEQ_printf(m, "\n");
>         SEQ_printf(m, "dl_rq[%d]:\n", cpu);
> @@ -1089,9 +1090,14 @@ void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
>         SEQ_printf(m, "  .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x))
>
>         PU(dl_nr_running);
> -       dl_bw = &cpu_rq(cpu)->rd->dl_bw;
> -       SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
> -       SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
> +       rcu_read_lock();
> +       rd = READ_ONCE(cpu_rq(cpu)->rd);
> +       if (rd) {
> +               dl_bw = &rd->dl_bw;
> +               SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
> +               SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
> +       }
> +       rcu_read_unlock();

Hey Aaron. How about simple

@@ -1089,7 +1089,8 @@ void print_dl_rq(struct seq_file *m, int cpu,
struct dl_rq *dl_rq)
        SEQ_printf(m, "  .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x))

        PU(dl_nr_running);
-       dl_bw = &cpu_rq(cpu)->rd->dl_bw;
+       guard(rcu)();
+       dl_bw = &rcu_dereference(cpu_rq(cpu)->rd)->dl_bw;
        SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
        SEQ_printf(m, "  .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);


>
>  #undef PU
>  }
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 622e2e01974c..b411cc00029c 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -496,7 +496,7 @@ void rq_attach_root(struct rq *rq, struct root_domain *rd)
>         }
>
>         atomic_inc(&rd->refcount);
> -       rq->rd = rd;
> +       rcu_assign_pointer(rq->rd, rd);
>
>         cpumask_set_cpu(rq->cpu, rd->span);
>         if (cpumask_test_cpu(rq->cpu, cpu_active_mask))
> --
> 2.55.0
>

[PATCH v4 2/5] sched/debug: Protect lockless rq->curr access in print_cpu()

[Aaron Tomlin]

In print_cpu(), rq->curr is dereferenced locklessly to print the current
task's PID via task_pid_nr(rq->curr).

While accessing /sys/kernel/debug/sched/debug is inherently best-effort
only; rq->curr is indeed expected to change dynamically while
print_cpu() is executing. However, if the task currently running on the
CPU exits concurrently and its reference count drops to zero,
put_task_struct() calls call_rcu() to schedule
__put_task_struct_rcu_cb(). Because print_cpu() does not hold the RCU
read lock while dereferencing rq->curr, an RCU grace period can complete
concurrently and free the task structure via free_task(), creating a
potential use-after-free race condition.

Resolve this by reading rq->curr using rcu_dereference(rq->curr) inside an
RCU read-side critical section. Holding the RCU read lock delays the
invocation of __put_task_struct_rcu_cb() until after rcu_read_unlock(),
ensuring that the struct task_struct memory remains valid while being
accessed.

Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/debug.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 2c2156dfab00..c3409166a288 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -1132,7 +1132,10 @@ do {									\
 	P(nr_switches);
 	P(nr_uninterruptible);
 	PN(next_balance);
-	SEQ_printf(m, "  .%-30s: %ld\n", "curr->pid", (long)(task_pid_nr(rq->curr)));
+	rcu_read_lock();
+	SEQ_printf(m, "  .%-30s: %ld\n", "curr->pid",
+		   (long)(task_pid_nr(rcu_dereference(rq->curr))));
+	rcu_read_unlock();
 	PN(clock);
 	PN(clock_task);
 #undef P
-- 
2.55.0

[PATCH v4 3/5] sched/debug: Protect lockless p->mm access in sched_show_numa()

[Aaron Tomlin]

In sched_show_numa(), p->mm is checked locklessly and then passed to the
P(mm->numa_scan_seq) macro. This results in a double-evaluation of p->mm.

If the task exits concurrently via exit_mm(p) between the check and the
macro expansion, p->mm can be set to NULL on another CPU. The second
evaluation then dereferences a NULL pointer.

Fix this TOCTOU race by reading p->mm once into a local variable using
READ_ONCE(p->mm) before checking and dereferencing its numa_scan_seq
field.

Fixes: b32e86b4301e ("sched/numa: Add debugging")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/debug.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index c3409166a288..f5494e02f600 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -1317,8 +1317,10 @@ void print_numa_stats(struct seq_file *m, int node, unsigned long tsf,
 static void sched_show_numa(struct task_struct *p, struct seq_file *m)
 {
 #ifdef CONFIG_NUMA_BALANCING
-	if (p->mm)
-		P(mm->numa_scan_seq);
+	struct mm_struct *mm = READ_ONCE(p->mm);
+
+	if (mm)
+		__PS("mm->numa_scan_seq", mm->numa_scan_seq);
 
 	P(numa_pages_migrated);
 	P(numa_preferred_nid);
-- 
2.55.0

[PATCH v4 4/5] sched/fair: Use list_for_each_entry_rcu() in print_cfs_stats()

[Aaron Tomlin]

In print_cfs_stats(), rq->leaf_cfs_rq_list is traversed using
for_each_leaf_cfs_rq_safe(), which expands to list_for_each_entry_safe().
Although rq->leaf_cfs_rq_list is RCU-protected, list_for_each_entry_safe()
is a non-RCU iteration macro. It dereferences pointer links without
READ_ONCE() and pre-fetches the next pointer.

When a writer concurrently adds a new cfs_rq to the list using
list_add_rcu(), a reader traversing with list_for_each_entry_safe()
lacks READ_ONCE() protection. Without READ_ONCE(), the compiler is free
to re-fetch pointers or reorder instructions. As a result, the reader
can observe a newly inserted cfs_rq's pointer before its internal fields
are fully visible, leading to reading uninitialised data or
dereferencing invalid pointers.

Additionally, because print_cfs_rq() drops rq->lock during seq_file I/O,
concurrent cfs_rq list removals and re-insertions can modify
cfs_rq->next. If cfs_rq is re-inserted near the head of the list while
rq->lock is dropped, lockless readers can jump backward in the list.
Under sufficient load it could lead to unbounded list traversal inside
the RCU read-side critical section and trigger an RCU stall.

Fix this by introducing for_each_leaf_cfs_rq_rcu(), which expands to
list_for_each_entry_rcu(). This uses READ_ONCE() during list traversal.
Finally, capping print_cfs_stats() lockless list traversal with a hard
iteration ceiling to guarantee loop termination and prevent RCU stalls
under continuous list churn.

Fixes: 0601267ca673 ("sched/fair: Rewrite list_add_leaf_cfs_rq()")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/fair.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d78467ec6ee1..eb03323332b0 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -412,6 +412,10 @@ static inline void assert_list_leaf_cfs_rq(struct rq *rq)
 	list_for_each_entry_safe(cfs_rq, pos, &rq->leaf_cfs_rq_list,	\
 				 leaf_cfs_rq_list)
 
+#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq)				\
+	list_for_each_entry_rcu(cfs_rq, &(rq)->leaf_cfs_rq_list,	\
+				leaf_cfs_rq_list)
+
 /* Do the two (enqueued) entities belong to the same group ? */
 static inline struct cfs_rq *
 is_same_group(struct sched_entity *se, struct sched_entity *pse)
@@ -497,6 +501,9 @@ static inline void assert_list_leaf_cfs_rq(struct rq *rq)
 #define for_each_leaf_cfs_rq_safe(rq, cfs_rq, pos)	\
 		for (cfs_rq = &rq->cfs, pos = NULL; cfs_rq; cfs_rq = pos)
 
+#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq)	\
+		for (cfs_rq = &rq->cfs; cfs_rq; cfs_rq = NULL)
+
 static inline struct sched_entity *parent_entity(struct sched_entity *se)
 {
 	return NULL;
@@ -15399,13 +15406,19 @@ DEFINE_SCHED_CLASS(fair) = {
 #endif
 };
 
+#define SCHED_DEBUG_MAX_ITER 1024
+
 void print_cfs_stats(struct seq_file *m, int cpu)
 {
-	struct cfs_rq *cfs_rq, *pos;
+	struct cfs_rq *cfs_rq;
+	int max_iter = SCHED_DEBUG_MAX_ITER;
 
 	rcu_read_lock();
-	for_each_leaf_cfs_rq_safe(cpu_rq(cpu), cfs_rq, pos)
+	for_each_leaf_cfs_rq_rcu(cpu_rq(cpu), cfs_rq) {
+		if (--max_iter < 0)
+			break;
 		print_cfs_rq(m, cpu, cfs_rq);
+	}
 	rcu_read_unlock();
 }
 
-- 
2.55.0

[PATCH v4 5/5] sched/debug: Introduce per-CPU debugfs files

[Aaron Tomlin]

Currently, accessing scheduler debugging details for a specific CPU
requires reading /sys/kernel/debug/sched/debug, which outputs
information for all online CPUs. When investigating a latency anomaly or
scheduling issue isolated to a specific CPU, accessing
/sys/kernel/debug/sched/cpu/cpu<N>/debug provides an immediate, targeted
view of that runqueue.

Add support for per-CPU debug files under:
/sys/kernel/debug/sched/cpu/cpu<N>/debug. Reading
/sys/kernel/debug/sched/cpu/cpu<N>/debug calls print_cpu() specifically
for CPU <N>, exposing CPU-specific runqueue details on demand. If the
target CPU is currently offline, reading its file returns -ENODEV.

Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/debug.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index f5494e02f600..22e917db86fa 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -356,6 +356,7 @@ static const struct file_operations sched_verbose_fops = {
 };
 
 static const struct seq_operations sched_debug_sops;
+static void print_cpu(struct seq_file *m, int cpu);
 
 static int sched_debug_open(struct inode *inode, struct file *filp)
 {
@@ -633,6 +634,47 @@ static void debugfs_fair_server_init(void)
 	}
 }
 
+static int sched_debug_cpu_show(struct seq_file *m, void *v)
+{
+	unsigned long cpu = (unsigned long) m->private;
+
+	if (!cpu_online(cpu))
+		return -ENODEV;
+
+	print_cpu(m, cpu);
+	return 0;
+}
+
+static int sched_debug_cpu_open(struct inode *inode, struct file *filp)
+{
+	return single_open(filp, sched_debug_cpu_show, inode->i_private);
+}
+
+static const struct file_operations sched_debug_cpu_fops = {
+	.open		= sched_debug_cpu_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
+static __init void debugfs_cpu_init(void)
+{
+	struct dentry *d_cpu_dir;
+	unsigned long cpu;
+	char buf[16];
+
+	d_cpu_dir = debugfs_create_dir("cpu", debugfs_sched);
+
+	for_each_possible_cpu(cpu) {
+		struct dentry *d_cpu;
+
+		snprintf(buf, sizeof(buf), "cpu%lu", cpu);
+		d_cpu = debugfs_create_dir(buf, d_cpu_dir);
+
+		debugfs_create_file("debug", 0444, d_cpu, (void *) cpu, &sched_debug_cpu_fops);
+	}
+}
+
 static __init int sched_init_debug(void)
 {
 	struct dentry __maybe_unused *numa, *llc;
@@ -690,6 +732,7 @@ static __init int sched_init_debug(void)
 #ifdef CONFIG_SCHED_CLASS_EXT
 	debugfs_ext_server_init();
 #endif
+	debugfs_cpu_init();
 
 	return 0;
 }
-- 
2.55.0