* Re: [PATCH v2] powerpc/kexec: Enable SMT before waking offline CPUs
2025-10-28 10:55 ` [PATCH v2] " Nysal Jan K.A.
@ 2025-10-28 14:31 ` Samir Alamshaha Mulani
2025-10-28 17:26 ` Shrikanth Hegde
` (2 subsequent siblings)
3 siblings, 0 replies; 10+ messages in thread
From: Samir Alamshaha Mulani @ 2025-10-28 14:31 UTC (permalink / raw)
To: Nysal Jan K.A., Sourabh Jain, Madhavan Srinivasan,
Christophe Leroy
Cc: Srikar Dronamraju, Sachin P Bappalige, stable, Michael Ellerman,
Nicholas Piggin, Laurent Dufour, Thomas Gleixner, linuxppc-dev,
linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2907 bytes --]
On 28/10/25 4:25 pm, Nysal Jan K.A. wrote:
> If SMT is disabled or a partial SMT state is enabled, when a new kernel
> image is loaded for kexec, on reboot the following warning is observed:
>
> kexec: Waking offline cpu 228.
> WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc
> [snip]
> NIP kexec_prepare_cpus+0x1b0/0x1bc
> LR kexec_prepare_cpus+0x1a0/0x1bc
> Call Trace:
> kexec_prepare_cpus+0x1a0/0x1bc (unreliable)
> default_machine_kexec+0x160/0x19c
> machine_kexec+0x80/0x88
> kernel_kexec+0xd0/0x118
> __do_sys_reboot+0x210/0x2c4
> system_call_exception+0x124/0x320
> system_call_vectored_common+0x15c/0x2ec
>
> This occurs as add_cpu() fails due to cpu_bootable() returning false for
> CPUs that fail the cpu_smt_thread_allowed() check or non primary
> threads if SMT is disabled.
>
> Fix the issue by enabling SMT and resetting the number of SMT threads to
> the number of threads per core, before attempting to wake up all present
> CPUs.
>
> Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()")
> Reported-by: Sachin P Bappalige<sachinpb@linux.ibm.com>
> Cc:stable@vger.kernel.org # v6.6+
> Reviewed-by: Srikar Dronamraju<srikar@linux.ibm.com>
> Signed-off-by: Nysal Jan K.A.<nysal@linux.ibm.com>
> ---
> arch/powerpc/kexec/core_64.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c
> index 222aa326dace..825ab8a88f18 100644
> --- a/arch/powerpc/kexec/core_64.c
> +++ b/arch/powerpc/kexec/core_64.c
> @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state)
> mb();
> }
>
> +
> +/*
> + * The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
> + * returns false for CPUs that fail the cpu_smt_thread_allowed() check
> + * or non primary threads if SMT is disabled. Re-enable SMT and set the
> + * number of SMT threads to threads per core.
> + */
> +static void kexec_smt_reenable(void)
> +{
> +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
> + lock_device_hotplug();
> + cpu_smt_num_threads = threads_per_core;
> + cpu_smt_control = CPU_SMT_ENABLED;
> + unlock_device_hotplug();
> +#endif
> +}
> +
> /*
> * We need to make sure each present CPU is online. The next kernel will scan
> * the device tree and assume primary threads are online and query secondary
> @@ -216,6 +233,8 @@ static void wake_offline_cpus(void)
> {
> int cpu = 0;
>
> + kexec_smt_reenable();
> +
> for_each_present_cpu(cpu) {
> if (!cpu_online(cpu)) {
> printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
Hi Nysal,
I have verified this patch with both *kexec* and the *smt_enabled=on/off
or 4 mode* bootline parameters, and no warnings or issues were observed
during testing.
Test Passed.
Thank you for fix !!
Tested-by: Samir M <samir@linux.ibm.com>
[-- Attachment #2: Type: text/html, Size: 3743 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v2] powerpc/kexec: Enable SMT before waking offline CPUs
2025-10-28 10:55 ` [PATCH v2] " Nysal Jan K.A.
2025-10-28 14:31 ` Samir Alamshaha Mulani
@ 2025-10-28 17:26 ` Shrikanth Hegde
2025-10-29 3:36 ` Nysal Jan K.A.
2025-12-19 9:58 ` Sourabh Jain
2025-12-27 4:24 ` Madhavan Srinivasan
3 siblings, 1 reply; 10+ messages in thread
From: Shrikanth Hegde @ 2025-10-28 17:26 UTC (permalink / raw)
To: Nysal Jan K.A.
Cc: Srikar Dronamraju, Sachin P Bappalige, stable, Michael Ellerman,
Nicholas Piggin, Laurent Dufour, Thomas Gleixner, linuxppc-dev,
linux-kernel, Sourabh Jain, Madhavan Srinivasan, Christophe Leroy
Hi Nysal.
On 10/28/25 4:25 PM, Nysal Jan K.A. wrote:
> If SMT is disabled or a partial SMT state is enabled, when a new kernel
> image is loaded for kexec, on reboot the following warning is observed:
>
> kexec: Waking offline cpu 228.
> WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc
> [snip]
> NIP kexec_prepare_cpus+0x1b0/0x1bc
> LR kexec_prepare_cpus+0x1a0/0x1bc
> Call Trace:
> kexec_prepare_cpus+0x1a0/0x1bc (unreliable)
> default_machine_kexec+0x160/0x19c
> machine_kexec+0x80/0x88
> kernel_kexec+0xd0/0x118
> __do_sys_reboot+0x210/0x2c4
> system_call_exception+0x124/0x320
> system_call_vectored_common+0x15c/0x2ec
>
> This occurs as add_cpu() fails due to cpu_bootable() returning false for
> CPUs that fail the cpu_smt_thread_allowed() check or non primary
> threads if SMT is disabled.
>
> Fix the issue by enabling SMT and resetting the number of SMT threads to
> the number of threads per core, before attempting to wake up all present
> CPUs.
>
> Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()")
> Reported-by: Sachin P Bappalige <sachinpb@linux.ibm.com>
> Cc: stable@vger.kernel.org # v6.6+
> Reviewed-by: Srikar Dronamraju <srikar@linux.ibm.com>
> Signed-off-by: Nysal Jan K.A. <nysal@linux.ibm.com>
> ---
> arch/powerpc/kexec/core_64.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c
> index 222aa326dace..825ab8a88f18 100644
> --- a/arch/powerpc/kexec/core_64.c
> +++ b/arch/powerpc/kexec/core_64.c
> @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state)
> mb();
> }
>
> +
> +/*
> + * The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
> + * returns false for CPUs that fail the cpu_smt_thread_allowed() check
> + * or non primary threads if SMT is disabled. Re-enable SMT and set the
> + * number of SMT threads to threads per core.
> + */
> +static void kexec_smt_reenable(void)
> +{
> +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
> + lock_device_hotplug();
I was looking at usage of lock_device_hotplug, looks like a good candidate for
guard() use case. Could be done on its own patch/series.
> + cpu_smt_num_threads = threads_per_core;
> + cpu_smt_control = CPU_SMT_ENABLED;
> + unlock_device_hotplug();
> +#endif
> +}
Will this work too? It might be better since we anyway going to bring that CPU up
by doing add_cpu afterwords.
cpu_smt_num_threads = threads_per_core;
cpuhp_smt_enable()
> +
> /*
> * We need to make sure each present CPU is online. The next kernel will scan
> * the device tree and assume primary threads are online and query secondary
> @@ -216,6 +233,8 @@ static void wake_offline_cpus(void)
> {
> int cpu = 0;
>
> + kexec_smt_reenable();
> +
If we do above, just change the below logic to complain if any present CPU is offline.
> for_each_present_cpu(cpu) {
> if (!cpu_online(cpu)) {
> printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v2] powerpc/kexec: Enable SMT before waking offline CPUs
2025-10-28 17:26 ` Shrikanth Hegde
@ 2025-10-29 3:36 ` Nysal Jan K.A.
0 siblings, 0 replies; 10+ messages in thread
From: Nysal Jan K.A. @ 2025-10-29 3:36 UTC (permalink / raw)
To: Shrikanth Hegde
Cc: Srikar Dronamraju, Sachin P Bappalige, stable, Michael Ellerman,
Nicholas Piggin, Laurent Dufour, Thomas Gleixner, linuxppc-dev,
linux-kernel, Sourabh Jain, Madhavan Srinivasan, Christophe Leroy
hi Shrikanth,
On Tue, Oct 28, 2025 at 10:56:05PM +0530, Shrikanth Hegde wrote:
> Hi Nysal.
>
> On 10/28/25 4:25 PM, Nysal Jan K.A. wrote:
[snip]
> > --- a/arch/powerpc/kexec/core_64.c
> > +++ b/arch/powerpc/kexec/core_64.c
> > @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state)
> > mb();
> > }
> > +
> > +/*
> > + * The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
> > + * returns false for CPUs that fail the cpu_smt_thread_allowed() check
> > + * or non primary threads if SMT is disabled. Re-enable SMT and set the
> > + * number of SMT threads to threads per core.
> > + */
> > +static void kexec_smt_reenable(void)
> > +{
> > +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
> > + lock_device_hotplug();
>
> I was looking at usage of lock_device_hotplug, looks like a good candidate for
> guard() use case. Could be done on its own patch/series.
>
Agree, we can look at it as a separate patch.
> > + cpu_smt_num_threads = threads_per_core;
> > + cpu_smt_control = CPU_SMT_ENABLED;
> > + unlock_device_hotplug();
> > +#endif
> > +}
>
>
> Will this work too? It might be better since we anyway going to bring that CPU up
> by doing add_cpu afterwords.
>
> cpu_smt_num_threads = threads_per_core;
> cpuhp_smt_enable()
>
There is some reasoning in 4d37cc2dc3df, which made the switch to use the core
device API, against calling cpu_up() directly. The other issue is
cpuhp_smt_enable() can skip bringing up a CPU in certain cases, for example
when a core is offline.
> > +
> > /*
> > * We need to make sure each present CPU is online. The next kernel will scan
> > * the device tree and assume primary threads are online and query secondary
> > @@ -216,6 +233,8 @@ static void wake_offline_cpus(void)
> > {
> > int cpu = 0;
> > + kexec_smt_reenable();
> > +
>
> If we do above, just change the below logic to complain if any present CPU is offline.
>
> > for_each_present_cpu(cpu) {
> > if (!cpu_online(cpu)) {
> > printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
>
Thanks for the review.
--Nysal
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] powerpc/kexec: Enable SMT before waking offline CPUs
2025-10-28 10:55 ` [PATCH v2] " Nysal Jan K.A.
2025-10-28 14:31 ` Samir Alamshaha Mulani
2025-10-28 17:26 ` Shrikanth Hegde
@ 2025-12-19 9:58 ` Sourabh Jain
2025-12-27 4:24 ` Madhavan Srinivasan
3 siblings, 0 replies; 10+ messages in thread
From: Sourabh Jain @ 2025-12-19 9:58 UTC (permalink / raw)
To: Nysal Jan K.A., Madhavan Srinivasan, Christophe Leroy
Cc: Srikar Dronamraju, Sachin P Bappalige, stable, Michael Ellerman,
Nicholas Piggin, Laurent Dufour, Thomas Gleixner, linuxppc-dev,
linux-kernel
On 28/10/25 16:25, Nysal Jan K.A. wrote:
> If SMT is disabled or a partial SMT state is enabled, when a new kernel
> image is loaded for kexec, on reboot the following warning is observed:
>
> kexec: Waking offline cpu 228.
> WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc
> [snip]
> NIP kexec_prepare_cpus+0x1b0/0x1bc
> LR kexec_prepare_cpus+0x1a0/0x1bc
> Call Trace:
> kexec_prepare_cpus+0x1a0/0x1bc (unreliable)
> default_machine_kexec+0x160/0x19c
> machine_kexec+0x80/0x88
> kernel_kexec+0xd0/0x118
> __do_sys_reboot+0x210/0x2c4
> system_call_exception+0x124/0x320
> system_call_vectored_common+0x15c/0x2ec
>
> This occurs as add_cpu() fails due to cpu_bootable() returning false for
> CPUs that fail the cpu_smt_thread_allowed() check or non primary
> threads if SMT is disabled.
>
> Fix the issue by enabling SMT and resetting the number of SMT threads to
> the number of threads per core, before attempting to wake up all present
> CPUs.
>
> Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()")
> Reported-by: Sachin P Bappalige <sachinpb@linux.ibm.com>
> Cc: stable@vger.kernel.org # v6.6+
> Reviewed-by: Srikar Dronamraju <srikar@linux.ibm.com>
> Signed-off-by: Nysal Jan K.A. <nysal@linux.ibm.com>
> ---
> arch/powerpc/kexec/core_64.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c
> index 222aa326dace..825ab8a88f18 100644
> --- a/arch/powerpc/kexec/core_64.c
> +++ b/arch/powerpc/kexec/core_64.c
> @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state)
> mb();
> }
>
> +
> +/*
> + * The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
> + * returns false for CPUs that fail the cpu_smt_thread_allowed() check
> + * or non primary threads if SMT is disabled. Re-enable SMT and set the
> + * number of SMT threads to threads per core.
> + */
> +static void kexec_smt_reenable(void)
> +{
> +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
> + lock_device_hotplug();
> + cpu_smt_num_threads = threads_per_core;
> + cpu_smt_control = CPU_SMT_ENABLED;
> + unlock_device_hotplug();
> +#endif
> +}
> +
> /*
> * We need to make sure each present CPU is online. The next kernel will scan
> * the device tree and assume primary threads are online and query secondary
> @@ -216,6 +233,8 @@ static void wake_offline_cpus(void)
> {
> int cpu = 0;
>
> + kexec_smt_reenable();
> +
> for_each_present_cpu(cpu) {
> if (!cpu_online(cpu)) {
> printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
The fix looks good to me. I also tested it with QEMU using different
SMT values with the kexec_load and kexec_file_load system calls, and no
warnings were observed during kexec.
Feel free to add:
Reviewed-by: Sourabh Jain <sourabhjain@linux.ibm.com>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v2] powerpc/kexec: Enable SMT before waking offline CPUs
2025-10-28 10:55 ` [PATCH v2] " Nysal Jan K.A.
` (2 preceding siblings ...)
2025-12-19 9:58 ` Sourabh Jain
@ 2025-12-27 4:24 ` Madhavan Srinivasan
3 siblings, 0 replies; 10+ messages in thread
From: Madhavan Srinivasan @ 2025-12-27 4:24 UTC (permalink / raw)
To: Sourabh Jain, Christophe Leroy, Nysal Jan K.A.
Cc: Srikar Dronamraju, Sachin P Bappalige, stable, Michael Ellerman,
Nicholas Piggin, Laurent Dufour, Thomas Gleixner, linuxppc-dev,
linux-kernel
On Tue, 28 Oct 2025 16:25:12 +0530, Nysal Jan K.A. wrote:
> If SMT is disabled or a partial SMT state is enabled, when a new kernel
> image is loaded for kexec, on reboot the following warning is observed:
>
> kexec: Waking offline cpu 228.
> WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc
> [snip]
> NIP kexec_prepare_cpus+0x1b0/0x1bc
> LR kexec_prepare_cpus+0x1a0/0x1bc
> Call Trace:
> kexec_prepare_cpus+0x1a0/0x1bc (unreliable)
> default_machine_kexec+0x160/0x19c
> machine_kexec+0x80/0x88
> kernel_kexec+0xd0/0x118
> __do_sys_reboot+0x210/0x2c4
> system_call_exception+0x124/0x320
> system_call_vectored_common+0x15c/0x2ec
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/kexec: Enable SMT before waking offline CPUs
https://git.kernel.org/powerpc/c/c2296a1e42418556efbeb5636c4fa6aa6106713a
cheers
^ permalink raw reply [flat|nested] 10+ messages in thread