Re: [PATCH 1/2] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection

[Andrea Righi <arighi@nvidia.com>]

Hi Prateek,

On Mon, Apr 20, 2026 at 03:09:17PM +0530, K Prateek Nayak wrote:
> On 4/20/2026 2:06 PM, Andrea Righi wrote:
> >> With your changes, only two places actually care about test_idle_cores():
> >>
> >> - select_idle_capacity()
> >> - select_idle_cpu()
> >>
> >> If we go into select_idle_capacity(), we don't do select_idle_cpu() so
> >> the two paths are mutually exclusive.
> >>
> >> In nohz_balancer_kick(), if we find, sd_asym_cpucapacity, we simply
> >> don't care about the sd_llc_shared->nr_busy_cpus during balancing so
> >> that begs the question if we can simply track idle_cores at
> >> sd_asym_cpucapacity for these systems?
> > 
> > Yeah, makes sense to me. I was planning to test something similar, so thanks for
> > sharing this patch. :) I'll give it a try and report back.
> 
> Thank you for taking it for a spin!

I've tested this extensively on Vera and haven't encountered any issues.
Performance wise I get similar results (with vs without), which was expected, as
sd_llc matches sd_asym_cpucapacity in my case.

> 
> >> I still have one question: Can first SD_ASYM_CPUCAPACITY_FULL be set at
> >> a SD_NUMA?
> >>
> >> We'll need to deal with overlapping domains then but seems like it could
> >> be possible with weird cpusets :-(
> >>
> >> But in that case, do we even want to search CPUs outside the NUMA in
> >> select_idle_capacity()? I don't think anything stops this currently but
> >> I might be wrong.
> > 
> > My $0.02 on this.
> > 
> > In theory it could happen with unusual topologies or constrained cpusets,
> > although it should be quite rare. That said, select_idle_capacity() already
> > operates on the span of sd_asym_cpucapacity, so if that domain crosses NUMA
> > boundaries, we're already scanning across NUMA today. This patch doesn't
> > fundamentally alter this behavior.
> 
> Ack! I was just thinking loud from the topology standpoint since
> sd->shared is not designed to handle the overlapping domains like
> sg->sgc does but we can probably figure some way to make it work.
> 
> Using the ring topology example from topology.c:
> 
>   0 ----- 1
>   |       |
>   |       |
>   |       |
>   3 ----- 2
> 
> Consider NUMA-1 below gets the SD_ASYM_CPUCAPACITY_FULL flag:
> 
>   NUMA-2       0-3             0-3             0-3             0-3
>    groups:     {0-1,3},{1-3}   {0-2},{0,2-3}   {1-3},{0-1,3}   {0,2-3},{0-2}
> 
>   NUMA-1       0-1,3           0-2             1-3             0,2-3
>    groups:     {0},{1},{3}     {0},{1},{2}     {1},{2},{3}     {0},{2},{3}
> 
>   NUMA-0       0               1               2               3
> 
> 
> The "sd->shared" assignments at NUMA-1 will put first, second, and the
> last domain in the same "shared" range by today's logic since the first
> CPU in their span is the same although their spans are slightly
> different.
> 
> The third will be standalone since the first CPU of the domain span
> will be different.

Yeah, makes sense. I'm wondering if we should attach the shared blob to
sd_asym_cpucapacity only when asym is a non-overlapping domain, otherwise
fallback to sd_llc and, in this case, ignore has_idle_cores in
select_idle_capacity(). This might be not the best in terms of efficiency on
those exotic topologies, but it'd eliminate the overlap/aliasing risk, while
still being correct. What do you think?

Thanks,
-Andrea