* [RFC PATCH] powerpc: Implement hotplug smt control
@ 2022-02-17 7:04 Joel Stanley
2022-03-02 15:40 ` Christophe Leroy
0 siblings, 1 reply; 2+ messages in thread
From: Joel Stanley @ 2022-02-17 7:04 UTC (permalink / raw)
To: linuxppc-dev
x86 added a control for turning SMT on and off in commit 05736e4ac13c
("cpu/hotplug: Provide knobs to control SMT").
Implement this for powerpc as an alternative to the currently method of
iterating through /sys/devices/system/cpu/cpuN/online for every CPU.
# ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4* 5* 6* 7*
Core 1: 8* 9* 10* 11* 12* 13* 14* 15*
# grep . /sys/devices/system/cpu/smt/*
/sys/devices/system/cpu/smt/active:1
/sys/devices/system/cpu/smt/control:on
# echo off > /sys/devices/system/cpu/smt/control
# ppc64_cpu --info
Core 0: 0* 1 2 3 4 5 6 7
Core 1: 8* 9 10 11 12 13 14 15
# grep . /sys/devices/system/cpu/smt/*
/sys/devices/system/cpu/smt/active:0
/sys/devices/system/cpu/smt/control:off
Signed-off-by: Joel Stanley <joel@jms.id.au>
---
This is a RFC as there are bugs:
- Booting with nosmt results in a WARN for every sibling thread
smp: Bringing up secondary CPUs ...
CPU UP failed (-125) CPU 1 state (null) (151)
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at kernel/cpu.c:711 _cpu_up+0x304/0x310
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.17.0-rc4-00055-g2db720040f59-dirty #8
...
NIP [c000000000136ac4] _cpu_up+0x304/0x310
LR [c000000000136ac4] _cpu_up+0x304/0x310
Call Trace:
[c000000008503ab0] [c000000000136ac4] _cpu_up+0x304/0x310 (unreliable)
[c000000008503b60] [c000000000136be4] cpu_up+0x114/0x1b0
[c000000008503c00] [c000000000137538] bringup_nonboot_cpus+0xb8/0x110
[c000000008503c60] [c000000002031838] smp_init+0x48/0xd4
[c000000008503cc0] [c0000000020048f4] kernel_init_freeable+0x20c/0x3dc
[c000000008503da0] [c0000000000127a4] kernel_init+0x34/0x1a0
[c000000008503e10] [c00000000000cd64] ret_from_kernel_thread+0x5c/0x64
Instruction dump:
1d270028 7cca482a 48089689 60000000 7ec4b378 7f03c378 4bffd45d 7ec6b378
7f05c378 7f84e378 38600000 4bffe919 <0b030000> eac10060 4bfffe64 3c4c0271
---[ end trace 0000000000000000 ]---
and then you can get a BUG at runtime if you fiddle with the online
state:
# ppc64_cpu --smt=on
# ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4* 5* 6* 7*
Core 1: 8* 9* 10* 11* 12* 13* 14* 15*
# ppc64_cpu --smt=off
[ 95.643467][ T203] ------------[ cut here ]------------
[ 95.643556][ T203] kernel BUG at kernel/irq_work.c:235!
[ 95.643633][ T203] Oops: Exception in kernel mode, sig: 5 [#1]
- Using the smt control to turn off SMT means you cannot online those
CPUs with /sys/devices/system/cpu/cpuN/online (returns EPERM)
# ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4* 5* 6* 7*
Core 1: 8* 9* 10* 11* 12* 13* 14* 15*
# echo off > /sys/devices/system/cpu/smt/control
# ppc64_cpu --info
Core 0: 0* 1 2 3 4 5 6 7
Core 1: 8* 9 10 11 12 13 14 15
# grep . /sys/devices/system/cpu/smt/*
/sys/devices/system/cpu/smt/active:0
/sys/devices/system/cpu/smt/control:off
# ppc64_cpu --smt=on
One or more cpus could not be on/offlined
# strace ppc64_cpu --smt=on
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/online", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=65536, ...}, AT_EMPTY_PATH) = 0
read(3, "1\n", 8192) = 2
close(3) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/online", O_WRONLY) = 3
write(3, "1", 1) = 1
close(3) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu1/online", O_WRONLY) = 3
write(3, "1", 1) = -1 EPERM (Operation not permitted)
close(3) = 0
access("/sys/devices/system/cpu/cpu8/online", F_OK) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu8/online", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=65536, ...}, AT_EMPTY_PATH) = 0
read(3, "1\n", 8192) = 2
close(3) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu8/online", O_WRONLY) = 3
write(3, "1", 1) = 1
close(3) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu9/online", O_WRONLY) = 3
write(3, "1", 1) = -1 EPERM (Operation not permitted)
close(3) = 0
write(2, "One or more cpus could not be on"..., 42One or more cpus could not be on/offlined
- Setting smt off with the online interface doesn't update the status
of the smt control sysfs file (but does update the active file):
# ppc64_cpu --smt=off
# grep . /sys/devices/system/cpu/smt/*
/sys/devices/system/cpu/smt/active:0
/sys/devices/system/cpu/smt/control:on
Signed-off-by: Joel Stanley <joel@jms.id.au>
---
arch/powerpc/include/asm/topology.h | 8 ++++++++
arch/powerpc/kernel/smp.c | 10 ++++++++++
arch/powerpc/Kconfig | 1 +
3 files changed, 19 insertions(+)
diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
index 36fcafb1fd6d..58280ca4321c 100644
--- a/arch/powerpc/include/asm/topology.h
+++ b/arch/powerpc/include/asm/topology.h
@@ -147,5 +147,13 @@ static inline int cpu_to_coregroup_id(int cpu)
#endif
#endif
+#ifdef CONFIG_HOTPLUG_CPU
+bool topology_is_primary_thread(unsigned int cpu);
+bool topology_smt_supported(void);
+#else
+static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
+static inline bool topology_smt_supported(void) { return false; }
+#endif
+
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_TOPOLOGY_H */
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index b7fd6a72aa76..24bd98401c91 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1781,4 +1781,14 @@ void arch_cpu_idle_dead(void)
start_secondary_resume();
}
+bool topology_smt_supported(void)
+{
+ return cpu_has_feature(CPU_FTR_SMT);
+}
+
+bool topology_is_primary_thread(unsigned int cpu)
+{
+ return cpu_thread_in_core(cpu) == 0;
+}
+
#endif
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index b779603978e1..a3e3d5de4d39 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -239,6 +239,7 @@ config PPC
select HAVE_STATIC_CALL if PPC32
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING
+ select HOTPLUG_SMT if SMP
select HUGETLB_PAGE_SIZE_VARIABLE if PPC_BOOK3S_64 && HUGETLB_PAGE
select IOMMU_HELPER if PPC64
select IRQ_DOMAIN
--
2.34.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [RFC PATCH] powerpc: Implement hotplug smt control
2022-02-17 7:04 [RFC PATCH] powerpc: Implement hotplug smt control Joel Stanley
@ 2022-03-02 15:40 ` Christophe Leroy
0 siblings, 0 replies; 2+ messages in thread
From: Christophe Leroy @ 2022-03-02 15:40 UTC (permalink / raw)
To: Joel Stanley, linuxppc-dev@lists.ozlabs.org
Le 17/02/2022 à 08:04, Joel Stanley a écrit :
> x86 added a control for turning SMT on and off in commit 05736e4ac13c
> ("cpu/hotplug: Provide knobs to control SMT").
>
> Implement this for powerpc as an alternative to the currently method of
> iterating through /sys/devices/system/cpu/cpuN/online for every CPU.
>
> # ppc64_cpu --info
> Core 0: 0* 1* 2* 3* 4* 5* 6* 7*
> Core 1: 8* 9* 10* 11* 12* 13* 14* 15*
> # grep . /sys/devices/system/cpu/smt/*
> /sys/devices/system/cpu/smt/active:1
> /sys/devices/system/cpu/smt/control:on
> # echo off > /sys/devices/system/cpu/smt/control
> # ppc64_cpu --info
> Core 0: 0* 1 2 3 4 5 6 7
> Core 1: 8* 9 10 11 12 13 14 15
> # grep . /sys/devices/system/cpu/smt/*
> /sys/devices/system/cpu/smt/active:0
> /sys/devices/system/cpu/smt/control:off
>
> Signed-off-by: Joel Stanley <joel@jms.id.au>
Build fails with corenet64_smp_defconfig:
CC kernel/cpu.o
kernel/cpu.c: In function 'cpuhp_smt_disable':
kernel/cpu.c:2220:23: error: implicit declaration of function
'cpu_down_maps_locked' [-Werror=implicit-function-declaration]
2220 | ret = cpu_down_maps_locked(cpu, CPUHP_OFFLINE);
| ^~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make[1]: *** [scripts/Makefile.build:288: kernel/cpu.o] Error 1
Christophe
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-03-02 15:41 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-02-17 7:04 [RFC PATCH] powerpc: Implement hotplug smt control Joel Stanley
2022-03-02 15:40 ` Christophe Leroy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).