Re: [PATCH 0/4] powerpc/topology: Introduce llc_override debugfs

[Tim Chen <tim.c.chen@linux.intel.com>]

On Mon, 2026-10-12 at 15:50 +0530, Shrikanth Hegde wrote:
> There is a new feature in scheduler called as Cache Aware Scheduling [1].
> It tries to keep related tasks within the same LLC domain while
> keeping the system reasonably balanced.
> 
> Cache Aware Scheduling assumes that the scheduler LLC domain is large enough
> to be a useful placement target. This is true on many systems where an LLC spans
> several cores, but not on PowerPC systems where the scheduler-visible LLC is at
> the SMT level. In that case, sd_llc_size is effectively the SMT width.
> 
> This becomes visible in the capacity check:
>         return !fits_capacity((mm->sc_stat.nr_running_avg * cpu_smt_num_threads),
>                               scale * per_cpu(sd_llc_size, cpu));
> 
> As a result, Cache aware scheduling may be enabled but has no room to operate.
> 
> Hence provide a debugfs option to set LLC at specific domain so that
> one can try experiment their workload with cache aware scheduling.
> This also provides ways to explore it on system with SMT=4 to see if
> there are any opportunities to improve or uncover bugs if there is any.
> 
> Note it is a debug option and it is recommended to keep it disabled.
> There is no performance guarantees across different values.
> 
> [1]: https://lore.kernel.org/all/cover.1778703694.git.tim.c.chen@linux.intel.com/

Shrikanth,

This is an interesting idea. One question from a quick look at the code.

The override moves sd_llc, sd_llc_size and sd_llc_shared, but not
sd_llc_id. The id comes from arch_llc_mask(), which is
cpu_l2_cache_mask() on powerpc, so after your patches it is still
unique per core. It is also sticky: build_sched_domains() assigns it
only when it is -1, and it is reset only on CPU offline. So a rebuild
from the knob keeps the old ids.

Cache aware scheduling uses this id as the identity of an LLC:
p->preferred_llc, nr_pref_dst_llc/llc_counts[], and the
llc_id(cpu) == llc_id(grp->cpu) check in task_cache_work(). In that
loop the occupancy is now summed over the MC span, but the id check
rarely matches and is broken.

To make this work, arch_llc_mask() needs to follow the override, and
the llc ids need to be reset when the topology is forced to rebuild.

> 
> Tim, Chenyu, Peter,
> Are there any selftest or patches which is used for
> functional validation of cache aware scheduling?

We haven't created one. We validate mostly with workloads such as
hackbench, schbench and yogini (which can vary the number of threads
over time and track CPU assignment). I agree they are not ideal for
functional checks of the load balance path.

> per-LLC occupancy in /proc/{PID}/schedstat may not tell the full
> story when there are multiple processes and multiple LLC's.
> 
> Atleast for functional verification, we might need one if there is
> not one already. The reason being,
> - At present, we may need a custom workload. hackbench/schbench are
>   wakeup intensive and typically bail out even before load balance.
> - stress-ng is per process. So doesn't get cache aware scheduling.
> - A selftest could spwan specific number of threads till llc size etc
>   and ensure packing happens till core_count(llc_size) by default.
>   This could also be used for tunable verfifications.
> - Provide an ability to observe how much of the workload was contained
>   within the LLC due to cache aware scheduling or number of load
>   balances stopped due to preferred llc.

A selftest would be great and much welcomed.  

Tim

> 
> These are some initial thoughts I had. If it sounds ok, I can try to add
> a new selftest/kunit test. 
> 
> Shrikanth Hegde (4):
>   powerpc/topology: Introduce llc_override debug knob
>   powerpc/topology: Add docs for llc_override
>   powerpc/topology: Update mc/pkg flags based on llc_override
>   powerpc/topology: Rebuild sched domains on llc_override change
> 
>  Documentation/arch/powerpc/index.rst    |   1 +
>  Documentation/arch/powerpc/topology.rst |  27 +++++
>  arch/powerpc/kernel/smp.c               | 128 +++++++++++++++++++++++-
>  3 files changed, 151 insertions(+), 5 deletions(-)
>  create mode 100644 Documentation/arch/powerpc/topology.rst