Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi <arighi@nvidia.com>]

Hi Dietmar,

On Thu, Sep 10, 2026 at 12:11:06PM +0200, Dietmar Eggemann wrote:
> On 09.09.26 18:17, Andrea Righi wrote:
> > On Wed, Sep 09, 2026 at 05:19:08PM +0200, Dietmar Eggemann wrote:
> >> On 09.09.26 08:26, Andrea Righi wrote:
> >>
> >> [...]
> >>
> >>> +void __init arm64_init_sched_topology(void)
> >>> +{
> >>> +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> >>> +		return;
> >>> +
> >>> +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> >>> +		return;
> >>> +
> >>> +	if (!topology_core_has_smt(smp_processor_id()))
> >>> +		return;
> >>> +
> >>> +	set_sched_topology(arm64_asym_smt_topology);
> >>> +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
> >>
> >> I'm not really a big fan of using this arm64 setup for NVIDIA Olympus
> >> alone here.
> >>
> >>> +}
> >>> +
> >>> +int arch_asym_cpu_priority(int cpu)
> >>> +{
> >>> +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> >>> +}
> >>> +
> >>
> >> This is done since it will only be called for NVIDIA Olympus since it's
> >> the only CPU model enabling this. I think it will work but it's still
> >> ugly :-)
> >>
> >> [...]
> >>
> > 
> > Agreed on both points. As mentioned in the other email, I'll coordinate with the
> > firmware team on a proper ABI for describing the PE0 preference. For the systems
> > with the current firmware, I don't have many ideas on how to enable this... a
> > special scheduler facility enabled via kernel boot option? Other ideas?
> With the explicit kernel boot time option discussed in the other thread,
> how do you want to code arch_asym_cpu_priority()?

Good point.

I think we can avoid an arm64 override entirely and rely on the default
implementation:

int __weak arch_asym_cpu_priority(int cpu)
{
	return -cpu;
}

The boot option would only add SD_ASYM_PACKING to the SMT domain. A rough
prototype could look like this in kernel/sched/topology.c:

static bool sched_smt_asym_packing __read_mostly;

static int __init setup_sched_smt_asym_packing(char *str)
{
	sched_smt_asym_packing = true;
	return 1;
}
__setup("sched_smt_asym_packing", setup_sched_smt_asym_packing);

int cpu_smt_flags(void)
{
	int flags = SD_SHARE_CPUCAPACITY | SD_SHARE_LLC;

	if (sched_smt_asym_packing)
		flags |= SD_ASYM_PACKING;

	return flags;
}

This would make the option generic rather than specific to arm64 or Olympus.

> 
> AFAIK, Arm architecture does not require the first/primary HW thread of
> an SMT core to have MPIDR_EL1.Aff0 == 0.

Agreed. Using the default -cpu ordering avoids making any assumptions about
MPIDR and consistently prefers the lowest-numbered logical CPU in each SMT
domain.

> 
> The closest I came to you setup in Arm64 is a ThunderX2 SMT-4 machine on
> which I can run 'OpenBLAS benchmark/sgemm.goto'. This machine has
> symmetric CPU capacity though like Power7.
> It has MPIDR_EL1.Aff0 == {0,1,2,3} for HW threads but '== 0' wouldn't
> work here.
> 
> [...]

Right, -cpu provides a total ordering for all threads, so this should provide
the expected sibling ordering on the ThunderX2 machine as well. If you're
willing to test the next version there, that would be very useful!

Thanks,
-Andrea