[PATCH v2 0/2] panic: Reduce CPU consumption after panic

[PATCH v2 0/2] panic: Reduce CPU consumption after panic

[carlos.bilbao]

From: Carlos Bilbao <carlos.bilbao@kernel.org>

Provide a priority-based mechanism to set the behavior of the kernel at
the post-panic stage -- the current default is a waste of CPU except for
cases with console that generate insightful output.

In v1 [1], I illustrated the potential to reduce unnecessary CPU usage
with an experiment with VMs. The main delta is that, instead of a weak
function that archs can overwrite, we provide a flexible priority-based
mechanism (following suggestions by Sean Christopherson),
panic_set_handling().

[1] https://lore.kernel.org/all/20250326151204.67898-1-carlos.bilbao@kernel.org/

Carlos:
  panic: Allow for dynamic custom behavior after panic
  x86/panic: Add x86_panic_handler as default post-panic behavior

---
 arch/x86/kernel/setup.c | 12 ++++++++++++
 include/linux/panic.h   |  2 ++
 kernel/panic.c          | 27 +++++++++++++++++++++++++++
 3 files changed, 41 insertions(+)

[PATCH v2 1/2] panic: Allow for dynamic custom behavior after panic

[carlos.bilbao]

From: Carlos Bilbao <carlos.bilbao@kernel.org>

Introduce panic_set_handling() to allow overriding the default post-panic
behavior.

Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
---
 include/linux/panic.h |  2 ++
 kernel/panic.c        | 27 +++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/include/linux/panic.h b/include/linux/panic.h
index 2494d51707ef..cf8d4a944407 100644
--- a/include/linux/panic.h
+++ b/include/linux/panic.h
@@ -98,4 +98,6 @@ extern void add_taint(unsigned flag, enum lockdep_ok);
 extern int test_taint(unsigned flag);
 extern unsigned long get_taint(void);
 
+void panic_set_handling(void (*fn)(void), int priority);
+
 #endif	/* _LINUX_PANIC_H */
diff --git a/kernel/panic.c b/kernel/panic.c
index a3889f38153d..2cdd83b4afb6 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -276,6 +276,30 @@ static void panic_other_cpus_shutdown(bool crash_kexec)
 		crash_smp_send_stop();
 }
 
+/*
+ * This is the default function called after a kernel panic has been
+ * handled. Higher priority alternatives can be set with function
+ * panic_set_handling()
+ *
+ */
+static void after_panic_handling(void)
+{
+	mdelay(PANIC_TIMER_STEP);
+}
+
+static void (*panic_halt)(void) = after_panic_handling;
+static int panic_hlt_priority;
+
+void panic_set_handling(void (*fn)(void), int priority)
+{
+	if (priority <= panic_hlt_priority)
+		return;
+
+	panic_hlt_priority = priority;
+	panic_halt = fn;
+}
+EXPORT_SYMBOL_GPL(panic_set_handling);
+
 /**
  *	panic - halt the system
  *	@fmt: The text string to print
@@ -467,6 +491,9 @@ void panic(const char *fmt, ...)
 	console_flush_on_panic(CONSOLE_FLUSH_PENDING);
 	nbcon_atomic_flush_unsafe();
 
+	if (panic_halt)
+		panic_halt();
+
 	local_irq_enable();
 	for (i = 0; ; i += PANIC_TIMER_STEP) {
 		touch_softlockup_watchdog();
-- 
2.47.1

Re: [PATCH v2 1/2] panic: Allow for dynamic custom behavior after panic

[Sean Christopherson]

On Mon, Apr 28, 2025, carlos.bilbao@kernel.org wrote:
> diff --git a/kernel/panic.c b/kernel/panic.c
> index a3889f38153d..2cdd83b4afb6 100644
> --- a/kernel/panic.c
> +++ b/kernel/panic.c
> @@ -276,6 +276,30 @@ static void panic_other_cpus_shutdown(bool crash_kexec)
>  		crash_smp_send_stop();
>  }
>  
> +/*
> + * This is the default function called after a kernel panic has been
> + * handled. Higher priority alternatives can be set with function
> + * panic_set_handling()
> + *
> + */
> +static void after_panic_handling(void)
> +{
> +	mdelay(PANIC_TIMER_STEP);
> +}
> +
> +static void (*panic_halt)(void) = after_panic_handling;

The default implementation clearly doesn't halt, which makes this unnecessarily
confusing.  And if you're going to provide a default implementation, why bother
checking for NULL in panic()?  Just leave panic_halt NULL.

> +static int panic_hlt_priority;

Uber nit, pick one of halt or hlt.

> +
> +void panic_set_handling(void (*fn)(void), int priority)
> +{
> +	if (priority <= panic_hlt_priority)

If panic_halt is NULL by default, maybe do?

	if (panic_halt && priority <= panic_halt_priority)

> +		return;
> +
> +	panic_hlt_priority = priority;
> +	panic_halt = fn;
> +}
> +EXPORT_SYMBOL_GPL(panic_set_handling);

This doesn't seem like something that should be exported unless it's absolutely
necessary, and it shouldn't be necessary as of this series.

> +
>  /**
>   *	panic - halt the system
>   *	@fmt: The text string to print
> @@ -467,6 +491,9 @@ void panic(const char *fmt, ...)
>  	console_flush_on_panic(CONSOLE_FLUSH_PENDING);
>  	nbcon_atomic_flush_unsafe();
>  
> +	if (panic_halt)
> +		panic_halt();
> +
>  	local_irq_enable();
>  	for (i = 0; ; i += PANIC_TIMER_STEP) {
>  		touch_softlockup_watchdog();
> -- 
> 2.47.1
> 

Re: [PATCH v2 1/2] panic: Allow for dynamic custom behavior after panic

[Carlos Bilbao]

Hello,


Sean, thanks for getting back to me so fast!


On 4/29/25 09:54, Sean Christopherson wrote:
> On Mon, Apr 28, 2025, carlos.bilbao@kernel.org wrote:
>> diff --git a/kernel/panic.c b/kernel/panic.c
>> index a3889f38153d..2cdd83b4afb6 100644
>> --- a/kernel/panic.c
>> +++ b/kernel/panic.c
>> @@ -276,6 +276,30 @@ static void panic_other_cpus_shutdown(bool crash_kexec)
>>  		crash_smp_send_stop();
>>  }
>>  
>> +/*
>> + * This is the default function called after a kernel panic has been
>> + * handled. Higher priority alternatives can be set with function
>> + * panic_set_handling()
>> + *
>> + */
>> +static void after_panic_handling(void)
>> +{
>> +	mdelay(PANIC_TIMER_STEP);
>> +}
>> +
>> +static void (*panic_halt)(void) = after_panic_handling;
> The default implementation clearly doesn't halt, which makes this unnecessarily
> confusing.  And if you're going to provide a default implementation, why bother
> checking for NULL in panic()?  Just leave panic_halt NULL.


Agreed.


>
>> +static int panic_hlt_priority;
> Uber nit, pick one of halt or hlt.


True, will use halt.


>> +
>> +void panic_set_handling(void (*fn)(void), int priority)
>> +{
>> +	if (priority <= panic_hlt_priority)
> If panic_halt is NULL by default, maybe do?
>
> 	if (panic_halt && priority <= panic_halt_priority)
>
>> +		return;
>> +
>> +	panic_hlt_priority = priority;
>> +	panic_halt = fn;
>> +}
>> +EXPORT_SYMBOL_GPL(panic_set_handling);
> This doesn't seem like something that should be exported unless it's absolutely
> necessary, and it shouldn't be necessary as of this series.
>
>> +
>>  /**
>>   *	panic - halt the system
>>   *	@fmt: The text string to print
>> @@ -467,6 +491,9 @@ void panic(const char *fmt, ...)
>>  	console_flush_on_panic(CONSOLE_FLUSH_PENDING);
>>  	nbcon_atomic_flush_unsafe();
>>  
>> +	if (panic_halt)
>> +		panic_halt();
>> +
>>  	local_irq_enable();
>>  	for (i = 0; ; i += PANIC_TIMER_STEP) {
>>  		touch_softlockup_watchdog();
>> -- 
>> 2.47.1
>>

Agree with all. Will fix in v3.


Thanks,

Carlos

[PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[carlos.bilbao]

From: Carlos Bilbao <carlos.bilbao@kernel.org>

Add function x86_panic_handler() as the default behavior for x86 for
post-panic stage via panic_set_handling(). Instead of busy-wait loop, it
will halt if there's no console to save CPU cycles.

Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
---
 arch/x86/kernel/setup.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 9d2a13b37833..3bfef55e9adb 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -16,6 +16,7 @@
 #include <linux/initrd.h>
 #include <linux/iscsi_ibft.h>
 #include <linux/memblock.h>
+#include <linux/panic.h>
 #include <linux/panic_notifier.h>
 #include <linux/pci.h>
 #include <linux/root_dev.h>
@@ -837,6 +838,15 @@ static void __init x86_report_nx(void)
 	}
 }
 
+
+static void x86_panic_handler(void)
+{
+	if (console_trylock()) {
+		console_unlock();
+		safe_halt();
+	}
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures
@@ -1252,6 +1262,8 @@ void __init setup_arch(char **cmdline_p)
 #endif
 
 	unwind_init();
+
+	panic_set_handling(x86_panic_handler, 1);
 }
 
 #ifdef CONFIG_X86_32
-- 
2.47.1

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[Sean Christopherson]

On Mon, Apr 28, 2025, carlos.bilbao@kernel.org wrote:
> From: Carlos Bilbao <carlos.bilbao@kernel.org>
> 
> Add function x86_panic_handler() as the default behavior for x86 for
> post-panic stage via panic_set_handling(). Instead of busy-wait loop, it
> will halt if there's no console to save CPU cycles.
> 
> Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
> ---
>  arch/x86/kernel/setup.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 9d2a13b37833..3bfef55e9adb 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -16,6 +16,7 @@
>  #include <linux/initrd.h>
>  #include <linux/iscsi_ibft.h>
>  #include <linux/memblock.h>
> +#include <linux/panic.h>
>  #include <linux/panic_notifier.h>
>  #include <linux/pci.h>
>  #include <linux/root_dev.h>
> @@ -837,6 +838,15 @@ static void __init x86_report_nx(void)
>  	}
>  }
>  
> +

Spurious newline.

> +static void x86_panic_handler(void)
> +{
> +	if (console_trylock()) {

A comment here would be very helpful.  Even with the changelog saying "if there's
no console", as an unfamiliar reader of console code, I have zero idea why being
able to lock the console is an effective test for no console.

> +		console_unlock();
> +		safe_halt();
> +	}
> +}

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[Carlos Bilbao]

Hello,

On 4/29/25 09:57, Sean Christopherson wrote:
> On Mon, Apr 28, 2025, carlos.bilbao@kernel.org wrote:
>> From: Carlos Bilbao <carlos.bilbao@kernel.org>
>>
>> Add function x86_panic_handler() as the default behavior for x86 for
>> post-panic stage via panic_set_handling(). Instead of busy-wait loop, it
>> will halt if there's no console to save CPU cycles.
>>
>> Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
>> ---
>>  arch/x86/kernel/setup.c | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 9d2a13b37833..3bfef55e9adb 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -16,6 +16,7 @@
>>  #include <linux/initrd.h>
>>  #include <linux/iscsi_ibft.h>
>>  #include <linux/memblock.h>
>> +#include <linux/panic.h>
>>  #include <linux/panic_notifier.h>
>>  #include <linux/pci.h>
>>  #include <linux/root_dev.h>
>> @@ -837,6 +838,15 @@ static void __init x86_report_nx(void)
>>  	}
>>  }
>>  
>> +
> Spurious newline.
>
>> +static void x86_panic_handler(void)
>> +{
>> +	if (console_trylock()) {
> A comment here would be very helpful.  Even with the changelog saying "if there's
> no console", as an unfamiliar reader of console code, I have zero idea why being
> able to lock the console is an effective test for no console.


Agree, will fix in v3.


>
>> +		console_unlock();
>> +		safe_halt();
>> +	}
>> +}


Thanks,

Carlos

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[John Ogness]

On 2025-04-28, carlos.bilbao@kernel.org wrote:
> From: Carlos Bilbao <carlos.bilbao@kernel.org>
>
> Add function x86_panic_handler() as the default behavior for x86 for
> post-panic stage via panic_set_handling(). Instead of busy-wait loop, it
> will halt if there's no console to save CPU cycles.
>
> Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
> ---
>  arch/x86/kernel/setup.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 9d2a13b37833..3bfef55e9adb 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -16,6 +16,7 @@
>  #include <linux/initrd.h>
>  #include <linux/iscsi_ibft.h>
>  #include <linux/memblock.h>
> +#include <linux/panic.h>
>  #include <linux/panic_notifier.h>
>  #include <linux/pci.h>
>  #include <linux/root_dev.h>
> @@ -837,6 +838,15 @@ static void __init x86_report_nx(void)
>  	}
>  }
>  
> +
> +static void x86_panic_handler(void)
> +{
> +	if (console_trylock()) {
> +		console_unlock();
> +		safe_halt();
> +	}

I do not understand what you are trying to accomplish with the
console_trylock(). At this point in the panic, all the messages are
already output. The console lock is totally irrelevant.

Also, the console lock is only valid for legacy consoles.

I see no reason why you don't just use safe_halt() as your panic
handler.

John Ogness

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[Carlos Bilbao]

Hello John,

On 4/29/25 11:53, John Ogness wrote:
> On 2025-04-28, carlos.bilbao@kernel.org wrote:
>> From: Carlos Bilbao <carlos.bilbao@kernel.org>
>>
>> Add function x86_panic_handler() as the default behavior for x86 for
>> post-panic stage via panic_set_handling(). Instead of busy-wait loop, it
>> will halt if there's no console to save CPU cycles.
>>
>> Signed-off-by: Carlos Bilbao (DigitalOcean) <carlos.bilbao@kernel.org>
>> ---
>>  arch/x86/kernel/setup.c | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 9d2a13b37833..3bfef55e9adb 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -16,6 +16,7 @@
>>  #include <linux/initrd.h>
>>  #include <linux/iscsi_ibft.h>
>>  #include <linux/memblock.h>
>> +#include <linux/panic.h>
>>  #include <linux/panic_notifier.h>
>>  #include <linux/pci.h>
>>  #include <linux/root_dev.h>
>> @@ -837,6 +838,15 @@ static void __init x86_report_nx(void)
>>  	}
>>  }
>>  
>> +
>> +static void x86_panic_handler(void)
>> +{
>> +	if (console_trylock()) {
>> +		console_unlock();
>> +		safe_halt();
>> +	}
> I do not understand what you are trying to accomplish with the
> console_trylock(). At this point in the panic, all the messages are
> already output. The console lock is totally irrelevant.
>
> Also, the console lock is only valid for legacy consoles.
>
> I see no reason why you don't just use safe_halt() as your panic
> handler.


Yes, in my original implementation I simply used halt, but I was trying to
be cautious in case any remaining messages hadn't flushed. I wonder, can we
be certain that, as you said, all output (e.g., a backtrace) has already
been displayed on screen at this point? I'm not sure.


>
> John Ogness


Thanks,

Carlos

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[John Ogness]

On 2025-04-29, Carlos Bilbao <carlos.bilbao.osdev@gmail.com> wrote:
>> I see no reason why you don't just use safe_halt() as your panic
>> handler.
>
> Yes, in my original implementation I simply used halt, but I was
> trying to be cautious in case any remaining messages hadn't flushed. I
> wonder, can we be certain that, as you said, all output (e.g., a
> backtrace) has already been displayed on screen at this point? I'm not
> sure.

Well, without this series, the kernel goes into an infinite loop. So if
the messages are not all out, they won't magically appear during the
infinite loop either.

John

Re: [PATCH v2 2/2] x86/panic: Add x86_panic_handler as default post-panic behavior

[Carlos Bilbao]

Hello,

On 4/30/25 02:58, John Ogness wrote:
> On 2025-04-29, Carlos Bilbao <carlos.bilbao.osdev@gmail.com> wrote:
>>> I see no reason why you don't just use safe_halt() as your panic
>>> handler.
>> Yes, in my original implementation I simply used halt, but I was
>> trying to be cautious in case any remaining messages hadn't flushed. I
>> wonder, can we be certain that, as you said, all output (e.g., a
>> backtrace) has already been displayed on screen at this point? I'm not
>> sure.
> Well, without this series, the kernel goes into an infinite loop. So if
> the messages are not all out, they won't magically appear during the
> infinite loop either.


I thought there may be some async stuff going on :)


>
> John


Thanks,

Carlos