Re: [PATCH 1/2] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection

Andrea Righi
2026-04-20T21:42:27Z

Hi Prateek,

On Mon, Apr 20, 2026 at 03:09:17PM +0530, K Prateek Nayak wrote:
> On 4/20/2026 2:06 PM, Andrea Righi wrote:
> >> With your changes, only two places actually care about test_idle_cores():
> >>
> >> - select_idle_capacity()
> >> - select_idle_cpu()
> >>
> >> If we go into select_idle_capacity(), we don't do select_idle_cpu() so
> >> the two paths are mutually exclusive.
> >>
> >> In nohz_balancer_kick(), if we find, sd_asym_cpucapacity, we simply
> >> don't care about the sd_llc_shared->nr_busy_cpus during balancing so
> >> that begs the question if we can simply track idle_cores at
> >> sd_asym_cpucapacity for these systems?
> >
> > Yeah, makes sense to me. I was planning to test something similar, so thanks for
> > sharing this patch. :) I'll give it a try and report back.
>
> Thank you for taking it for a spin!

I've tested this extensively on Vera and haven't encountered any issues.
Performance wise I get similar results (with vs without), which was expected, as
sd_llc matches sd_asym_cpucapacity in my case.

>
> >> I still have one question: Can first SD_ASYM_CPUCAPACITY_FULL be set at
> >> a SD_NUMA?
> >>
> >> We'll need to deal with overlapping domains then but seems like it could
> >> be possible with weird cpusets :-(
> >>
> >> But in that case, do we even want to search CPUs outside the NUMA in
> >> select_idle_capacity()? I don't think anything stops this currently but
> >> I might be wrong.
> >
> > My $0.02 on this.
> >
> > In theory it could happen with unusual topologies or constrained cpusets,
> > although it should be quite rare. That said, select_idle_capacity() already
> > operates on the span of sd_asym_cpucapacity, so if that domain crosses NUMA
> > boundaries, we're already scanning across NUMA today. This patch doesn't
> > fundamentally alter this behavior.
>
> Ack! I was just thinking loud from the topology standpoint since
> sd->shared is not designed to handle the overlapping domains like
> sg->sgc does but we can probably figure some way to make it work.
>
> Using the ring topology example from topology.c:
>
> 0 ----- 1
> | |
> | |
> | |
> 3 ----- 2
>
> Consider NUMA-1 below gets the SD_ASYM_CPUCAPACITY_FULL flag:
>
> NUMA-2 0-3 0-3 0-3 0-3
> groups: {0-1,3},{1-3} {0-2},{0,2-3} {1-3},{0-1,3} {0,2-3},{0-2}
>
> NUMA-1 0-1,3 0-2 1-3 0,2-3
> groups: {0},{1},{3} {0},{1},{2} {1},{2},{3} {0},{2},{3}
>
> NUMA-0 0 1 2 3
>
>
> The "sd->shared" assignments at NUMA-1 will put first, second, and the
> last domain in the same "shared" range by today's logic since the first
> CPU in their span is the same although their spans are slightly
> different.
>
> The third will be standalone since the first CPU of the domain span
> will be different.

Yeah, makes sense. I'm wondering if we should attach the shared blob to
sd_asym_cpucapacity only when asym is a non-overlapping domain, otherwise
fallback to sd_llc and, in this case, ignore has_idle_cores in
select_idle_capacity(). This might be not the best in terms of efficiency on
those exotic topologies, but it'd eliminate the overlap/aliasing risk, while
still being correct. What do you think?

Thanks,
-Andrea