[PATCH v2 0/4] x86/boot: provide better diagnostics in AP boot failure

[PATCH v2 0/4] x86/boot: provide better diagnostics in AP boot failure

[Roger Pau Monne]

Hello,

This series attempts to improve AP boot failure diagnosis by improving
the printed failure messages (patch 1) and detecting AP getting stuck
during bringup (patch 4).  Patches 2 and 3 are preparatory changes for
the work done in patch 4.

They should be non-functional changes for systems working correctly.

Thanks, Roger.

Roger Pau Monne (4):
  x86/boot: print CPU and APIC ID in bring up failure
  x86/traps: remove smp_mb() ahead of IPI dispatch
  x86/traps: split code to dump execution state to a separate helper
  x86/boot: attempt to print trace and panic on AP bring up stall

 xen/arch/x86/include/asm/processor.h |  1 +
 xen/arch/x86/smpboot.c               | 14 +++++-
 xen/arch/x86/traps.c                 | 64 +++++++++++++++++-----------
 3 files changed, 51 insertions(+), 28 deletions(-)

-- 
2.49.0

[PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Roger Pau Monne]

Print the CPU and APIC ID that fails to respond to the init sequence, or
that didn't manage to reach the "callin" state.  Expand a bit the printed
error messages.  Otherwise the "Not responding." message is not easy to
understand by users.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Also print APIC ID.
---
 xen/arch/x86/smpboot.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 0189d6c332a4..dbc2f2f1d411 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
             smp_mb();
             if ( bootsym(trampoline_cpu_started) == 0xA5 )
                 /* trampoline started but...? */
-                printk("Stuck ??\n");
+                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
+                       cpu, apicid);
             else
                 /* trampoline code not run */
-                printk("Not responding.\n");
+                printk("CPU%u/APICID%u: Not responding to startup\n",
+                       cpu, apicid);
         }
     }
 
-- 
2.49.0

Re: [PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Jan Beulich]

On 22.05.2025 09:54, Roger Pau Monne wrote:
> Print the CPU and APIC ID that fails to respond to the init sequence, or
> that didn't manage to reach the "callin" state.  Expand a bit the printed
> error messages.  Otherwise the "Not responding." message is not easy to
> understand by users.
> 
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - Also print APIC ID.
> ---
>  xen/arch/x86/smpboot.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> index 0189d6c332a4..dbc2f2f1d411 100644
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
>              smp_mb();
>              if ( bootsym(trampoline_cpu_started) == 0xA5 )
>                  /* trampoline started but...? */
> -                printk("Stuck ??\n");
> +                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
> +                       cpu, apicid);
>              else
>                  /* trampoline code not run */
> -                printk("Not responding.\n");
> +                printk("CPU%u/APICID%u: Not responding to startup\n",
> +                       cpu, apicid);
>          }
>      }
>  

Elsewhere I think we print AIC IDs in hex; may be better to do so here, too.
That may then want some text re-arrangement though, e.g.

"CPU%u: APICID %#x not responding to startup\n"

Thoughts?

Jan

Re: [PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Andrew Cooper]

On 22/05/2025 10:10 am, Jan Beulich wrote:
> On 22.05.2025 09:54, Roger Pau Monne wrote:
>> Print the CPU and APIC ID that fails to respond to the init sequence, or
>> that didn't manage to reach the "callin" state.  Expand a bit the printed
>> error messages.  Otherwise the "Not responding." message is not easy to
>> understand by users.
>>
>> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> ---
>> Changes since v1:
>>  - Also print APIC ID.
>> ---
>>  xen/arch/x86/smpboot.c | 6 ++++--
>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
>> index 0189d6c332a4..dbc2f2f1d411 100644
>> --- a/xen/arch/x86/smpboot.c
>> +++ b/xen/arch/x86/smpboot.c
>> @@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
>>              smp_mb();
>>              if ( bootsym(trampoline_cpu_started) == 0xA5 )
>>                  /* trampoline started but...? */
>> -                printk("Stuck ??\n");
>> +                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
>> +                       cpu, apicid);
>>              else
>>                  /* trampoline code not run */
>> -                printk("Not responding.\n");
>> +                printk("CPU%u/APICID%u: Not responding to startup\n",
>> +                       cpu, apicid);
>>          }
>>      }
>>  
> Elsewhere I think we print AIC IDs in hex; may be better to do so here, too.
> That may then want some text re-arrangement though, e.g.
>
> "CPU%u: APICID %#x not responding to startup\n"
>
> Thoughts?

Definitely hex.  Elsewhere APIC_ID always has an underscore.

I'd suggest:

"APIC_ID %#x (CPU%u) didn't respond to SIPI\n"

The APIC_ID is the critical detail, and the CPU number is fairly incidental.

Also as we're changing things, lets not retain the STARTUP name seeing
as it only exists on pre-Pentium APICs.  SIPI is what we use these days.

~Andrew

Re: [PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Roger Pau Monné]

On Thu, May 22, 2025 at 03:39:57PM +0100, Andrew Cooper wrote:
> On 22/05/2025 10:10 am, Jan Beulich wrote:
> > On 22.05.2025 09:54, Roger Pau Monne wrote:
> >> Print the CPU and APIC ID that fails to respond to the init sequence, or
> >> that didn't manage to reach the "callin" state.  Expand a bit the printed
> >> error messages.  Otherwise the "Not responding." message is not easy to
> >> understand by users.
> >>
> >> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >> ---
> >> Changes since v1:
> >>  - Also print APIC ID.
> >> ---
> >>  xen/arch/x86/smpboot.c | 6 ++++--
> >>  1 file changed, 4 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> >> index 0189d6c332a4..dbc2f2f1d411 100644
> >> --- a/xen/arch/x86/smpboot.c
> >> +++ b/xen/arch/x86/smpboot.c
> >> @@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
> >>              smp_mb();
> >>              if ( bootsym(trampoline_cpu_started) == 0xA5 )
> >>                  /* trampoline started but...? */
> >> -                printk("Stuck ??\n");
> >> +                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
> >> +                       cpu, apicid);
> >>              else
> >>                  /* trampoline code not run */
> >> -                printk("Not responding.\n");
> >> +                printk("CPU%u/APICID%u: Not responding to startup\n",
> >> +                       cpu, apicid);
> >>          }
> >>      }
> >>  
> > Elsewhere I think we print AIC IDs in hex; may be better to do so here, too.
> > That may then want some text re-arrangement though, e.g.
> >
> > "CPU%u: APICID %#x not responding to startup\n"
> >
> > Thoughts?
> 
> Definitely hex.  Elsewhere APIC_ID always has an underscore.

Maybe I'm confused, but I don't think Xen uses an underscore, it's
always 'APIC ID' when printed.  I don't mind adding it here, I assume
what you mean with elsewhere is other projects like Linux?

Thanks, Roger.

Re: [PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Andrew Cooper]

On 22/05/2025 5:50 pm, Roger Pau Monné wrote:
> On Thu, May 22, 2025 at 03:39:57PM +0100, Andrew Cooper wrote:
>> On 22/05/2025 10:10 am, Jan Beulich wrote:
>>> On 22.05.2025 09:54, Roger Pau Monne wrote:
>>>> Print the CPU and APIC ID that fails to respond to the init sequence, or
>>>> that didn't manage to reach the "callin" state.  Expand a bit the printed
>>>> error messages.  Otherwise the "Not responding." message is not easy to
>>>> understand by users.
>>>>
>>>> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> ---
>>>> Changes since v1:
>>>>  - Also print APIC ID.
>>>> ---
>>>>  xen/arch/x86/smpboot.c | 6 ++++--
>>>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
>>>> index 0189d6c332a4..dbc2f2f1d411 100644
>>>> --- a/xen/arch/x86/smpboot.c
>>>> +++ b/xen/arch/x86/smpboot.c
>>>> @@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
>>>>              smp_mb();
>>>>              if ( bootsym(trampoline_cpu_started) == 0xA5 )
>>>>                  /* trampoline started but...? */
>>>> -                printk("Stuck ??\n");
>>>> +                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
>>>> +                       cpu, apicid);
>>>>              else
>>>>                  /* trampoline code not run */
>>>> -                printk("Not responding.\n");
>>>> +                printk("CPU%u/APICID%u: Not responding to startup\n",
>>>> +                       cpu, apicid);
>>>>          }
>>>>      }
>>>>  
>>> Elsewhere I think we print AIC IDs in hex; may be better to do so here, too.
>>> That may then want some text re-arrangement though, e.g.
>>>
>>> "CPU%u: APICID %#x not responding to startup\n"
>>>
>>> Thoughts?
>> Definitely hex.  Elsewhere APIC_ID always has an underscore.
> Maybe I'm confused, but I don't think Xen uses an underscore, it's
> always 'APIC ID' when printed.  I don't mind adding it here, I assume
> what you mean with elsewhere is other projects like Linux?

It's apic_id in plenty of smpboot.c, but fine - lets do it with a space.

We do need to reduce from $N ways of rendering this down to 1.

~Andrew

Re: [PATCH v2 1/4] x86/boot: print CPU and APIC ID in bring up failure

[Roger Pau Monné]

On Thu, May 22, 2025 at 06:22:19PM +0100, Andrew Cooper wrote:
> On 22/05/2025 5:50 pm, Roger Pau Monné wrote:
> > On Thu, May 22, 2025 at 03:39:57PM +0100, Andrew Cooper wrote:
> >> On 22/05/2025 10:10 am, Jan Beulich wrote:
> >>> On 22.05.2025 09:54, Roger Pau Monne wrote:
> >>>> Print the CPU and APIC ID that fails to respond to the init sequence, or
> >>>> that didn't manage to reach the "callin" state.  Expand a bit the printed
> >>>> error messages.  Otherwise the "Not responding." message is not easy to
> >>>> understand by users.
> >>>>
> >>>> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >>>> ---
> >>>> Changes since v1:
> >>>>  - Also print APIC ID.
> >>>> ---
> >>>>  xen/arch/x86/smpboot.c | 6 ++++--
> >>>>  1 file changed, 4 insertions(+), 2 deletions(-)
> >>>>
> >>>> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> >>>> index 0189d6c332a4..dbc2f2f1d411 100644
> >>>> --- a/xen/arch/x86/smpboot.c
> >>>> +++ b/xen/arch/x86/smpboot.c
> >>>> @@ -618,10 +618,12 @@ static int do_boot_cpu(int apicid, int cpu)
> >>>>              smp_mb();
> >>>>              if ( bootsym(trampoline_cpu_started) == 0xA5 )
> >>>>                  /* trampoline started but...? */
> >>>> -                printk("Stuck ??\n");
> >>>> +                printk("CPU%u/APICID%u: Didn't finish startup sequence\n",
> >>>> +                       cpu, apicid);
> >>>>              else
> >>>>                  /* trampoline code not run */
> >>>> -                printk("Not responding.\n");
> >>>> +                printk("CPU%u/APICID%u: Not responding to startup\n",
> >>>> +                       cpu, apicid);
> >>>>          }
> >>>>      }
> >>>>  
> >>> Elsewhere I think we print AIC IDs in hex; may be better to do so here, too.
> >>> That may then want some text re-arrangement though, e.g.
> >>>
> >>> "CPU%u: APICID %#x not responding to startup\n"
> >>>
> >>> Thoughts?
> >> Definitely hex.  Elsewhere APIC_ID always has an underscore.
> > Maybe I'm confused, but I don't think Xen uses an underscore, it's
> > always 'APIC ID' when printed.  I don't mind adding it here, I assume
> > what you mean with elsewhere is other projects like Linux?
> 
> It's apic_id in plenty of smpboot.c, but fine - lets do it with a space.
> 
> We do need to reduce from $N ways of rendering this down to 1.

Oh, so it's lowercase with underscore.  Anyway, will use uppercase and
space as agreed.

Thanks, Roger.

[PATCH v2 2/4] x86/traps: remove smp_mb() ahead of IPI dispatch

[Roger Pau Monne]

The IPI dispatch functions should already have the required barriers to
ensure correct memory ordering.

Note other callers of send_IPI_mask() don't use any barriers.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
 xen/arch/x86/traps.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index c94779b4ad4f..22f20629327d 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -785,8 +785,6 @@ void fatal_trap(const struct cpu_user_regs *regs, bool show_remote)
             cpumask_andnot(&show_state_mask, &cpu_online_map,
                            cpumask_of(smp_processor_id()));
             set_nmi_callback(nmi_show_execution_state);
-            /* Ensure new callback is set before sending out the NMI. */
-            smp_wmb();
             smp_send_nmi_allbutself();
 
             /* Wait at most 10ms for some other CPU to respond. */
-- 
2.49.0

Re: [PATCH v2 2/4] x86/traps: remove smp_mb() ahead of IPI dispatch

[Jan Beulich]

On 22.05.2025 09:54, Roger Pau Monne wrote:
> The IPI dispatch functions should already have the required barriers to
> ensure correct memory ordering.

To be quite honest, "should" isn't sufficient here for my taste. Either
they are there or they aren't. According to my check they are, so ...

> Note other callers of send_IPI_mask() don't use any barriers.
> 
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with the word dropped.

Jan

[PATCH v2 3/4] x86/traps: split code to dump execution state to a separate helper

[Roger Pau Monne]

Split the code that triggers remote CPUs to dump stacks into a separate
function.  Also introduce a parameter that can be set by the caller of the
newly introduced function to force CPUs to dump the full stack, rather than
just dumping the current function name.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/include/asm/processor.h |  1 +
 xen/arch/x86/traps.c                 | 62 +++++++++++++++++-----------
 2 files changed, 39 insertions(+), 24 deletions(-)

diff --git a/xen/arch/x86/include/asm/processor.h b/xen/arch/x86/include/asm/processor.h
index eacd425c5350..10d8078cc1ca 100644
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -371,6 +371,7 @@ void show_registers(const struct cpu_user_regs *regs);
 #define dump_execution_state() run_in_exception_handler(show_execution_state)
 void show_page_walk(unsigned long addr);
 void noreturn fatal_trap(const struct cpu_user_regs *regs, bool show_remote);
+void show_execution_state_nmi(const cpumask_t *mask, bool show_all);
 
 extern void mtrr_ap_init(void);
 extern void mtrr_bp_init(void);
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 22f20629327d..f6646d505644 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -714,13 +714,15 @@ static cpumask_t show_state_mask;
 static bool opt_show_all;
 boolean_param("async-show-all", opt_show_all);
 
+static bool force_show_all;
+
 static int cf_check nmi_show_execution_state(
     const struct cpu_user_regs *regs, int cpu)
 {
     if ( !cpumask_test_cpu(cpu, &show_state_mask) )
         return 0;
 
-    if ( opt_show_all )
+    if ( opt_show_all || force_show_all )
         show_execution_state(regs);
     else if ( guest_mode(regs) )
         printk(XENLOG_ERR "CPU%d\t%pv\t%04x:%p in guest\n",
@@ -734,6 +736,38 @@ static int cf_check nmi_show_execution_state(
     return 1;
 }
 
+void show_execution_state_nmi(const cpumask_t *mask, bool show_all)
+{
+    unsigned int msecs, pending;
+
+    force_show_all = show_all;
+
+    watchdog_disable();
+    console_start_sync();
+
+    cpumask_copy(&show_state_mask, mask);
+    set_nmi_callback(nmi_show_execution_state);
+    send_IPI_mask(mask, APIC_DM_NMI);
+
+    /* Wait at most 10ms for some other CPU to respond. */
+    msecs = 10;
+    pending = cpumask_weight(&show_state_mask);
+    while ( pending && msecs-- )
+    {
+        unsigned int left;
+
+        mdelay(1);
+        left = cpumask_weight(&show_state_mask);
+        if ( left < pending )
+        {
+            pending = left;
+            msecs = 10;
+        }
+    }
+    if ( pending )
+        printk("Non-responding CPUs: {%*pbl}\n", CPUMASK_PR(&show_state_mask));
+}
+
 const char *vector_name(unsigned int vec)
 {
     static const char names[][4] = {
@@ -780,31 +814,11 @@ void fatal_trap(const struct cpu_user_regs *regs, bool show_remote)
 
         if ( show_remote )
         {
-            unsigned int msecs, pending;
+            cpumask_t *scratch = this_cpu(scratch_cpumask);
 
-            cpumask_andnot(&show_state_mask, &cpu_online_map,
+            cpumask_andnot(scratch, &cpu_online_map,
                            cpumask_of(smp_processor_id()));
-            set_nmi_callback(nmi_show_execution_state);
-            smp_send_nmi_allbutself();
-
-            /* Wait at most 10ms for some other CPU to respond. */
-            msecs = 10;
-            pending = cpumask_weight(&show_state_mask);
-            while ( pending && msecs-- )
-            {
-                unsigned int left;
-
-                mdelay(1);
-                left = cpumask_weight(&show_state_mask);
-                if ( left < pending )
-                {
-                    pending = left;
-                    msecs = 10;
-                }
-            }
-            if ( pending )
-                printk("Non-responding CPUs: {%*pbl}\n",
-                       CPUMASK_PR(&show_state_mask));
+            show_execution_state_nmi(scratch, false);
         }
     }
 
-- 
2.49.0

Re: [PATCH v2 3/4] x86/traps: split code to dump execution state to a separate helper

[Jan Beulich]

On 22.05.2025 09:54, Roger Pau Monne wrote:
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -714,13 +714,15 @@ static cpumask_t show_state_mask;
>  static bool opt_show_all;
>  boolean_param("async-show-all", opt_show_all);
>  
> +static bool force_show_all;
> +
>  static int cf_check nmi_show_execution_state(
>      const struct cpu_user_regs *regs, int cpu)
>  {
>      if ( !cpumask_test_cpu(cpu, &show_state_mask) )
>          return 0;
>  
> -    if ( opt_show_all )
> +    if ( opt_show_all || force_show_all )
>          show_execution_state(regs);
>      else if ( guest_mode(regs) )
>          printk(XENLOG_ERR "CPU%d\t%pv\t%04x:%p in guest\n",
> @@ -734,6 +736,38 @@ static int cf_check nmi_show_execution_state(
>      return 1;
>  }
>  
> +void show_execution_state_nmi(const cpumask_t *mask, bool show_all)
> +{
> +    unsigned int msecs, pending;
> +
> +    force_show_all = show_all;

Prior (v1) comments here still apply.

Jan

[PATCH v2 4/4] x86/boot: attempt to print trace and panic on AP bring up stall

[Roger Pau Monne]

With the current AP bring up code, Xen can get stuck indefinitely if an AP
freezes during boot after the 'callin' step.  Introduce a 5s timeout while
waiting for APs to finish startup.

On failure of an AP to complete startup, send an NMI to trigger the
printing of a stack backtrace on the stuck AP and panic on the BSP.

This patch was done while investigating the issue caused by Intel erratum
ICX143.  It wasn't helpful in that case, but it's still and improvement
when debugging AP bring up related issues.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Use 5s timeout.
 - Print APICID.
 - Split NMI dispatch code movement to a pre-patch.
 - Reorder timeout check condition.
---
 xen/arch/x86/smpboot.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index dbc2f2f1d411..50c5674555e4 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -1372,6 +1372,7 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint32_t pxm)
 int __cpu_up(unsigned int cpu)
 {
     int apicid, ret;
+    s_time_t start;
 
     if ( (apicid = x86_cpu_to_apicid[cpu]) == BAD_APICID )
         return -ENODEV;
@@ -1390,10 +1391,17 @@ int __cpu_up(unsigned int cpu)
     time_latch_stamps();
 
     set_cpu_state(CPU_STATE_ONLINE);
+    start = NOW();
     while ( !cpu_online(cpu) )
     {
         cpu_relax();
         process_pending_softirqs();
+        if ( (NOW() - start) > SECONDS(5) )
+        {
+            /* AP is stuck, send NMI and panic. */
+            show_execution_state_nmi(cpumask_of(cpu), true);
+            panic("CPU%u/APICID%u: Stuck while starting up\n", cpu, apicid);
+        }
     }
 
     return 0;
-- 
2.49.0

Re: [PATCH v2 4/4] x86/boot: attempt to print trace and panic on AP bring up stall

[Andrew Cooper]

On 22/05/2025 8:54 am, Roger Pau Monne wrote:
> With the current AP bring up code, Xen can get stuck indefinitely if an AP
> freezes during boot after the 'callin' step.  Introduce a 5s timeout while
> waiting for APs to finish startup.
>
> On failure of an AP to complete startup, send an NMI to trigger the
> printing of a stack backtrace on the stuck AP and panic on the BSP.
>
> This patch was done while investigating the issue caused by Intel erratum
> ICX143.  It wasn't helpful in that case, but it's still and improvement
> when debugging AP bring up related issues.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>