Re: [PATCH] x86/ucode: Remove MICROCODE_UPDATE_TIMEOUT and associated panic()

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 18/08/2026 3:31 pm, Jan Beulich wrote:
> On 18.08.2026 15:07, Andrew Cooper wrote:
>> Panicing in the case of a timeout turns out to be about the worst possible
>> action Xen can take.  It leaves all other APs waiting on the condition
>> variable, some in NMI context.  As a result, they fail to be shot down and
>> dump state for kexec crash analysis.
> At the same time there likely isn't much to be learned from a kexec dump, as
> the source of the issue is in the CPU, not in Xen.

The single most valuable print message I've ever added to Xen is the one
which reports which CPUs didn't respond to NMIs.  Up until now, it has
always highlighted hardware issues.

Despite my wish to remove this panic specifically, there is still
information to be gained from kexec in a similar scenario.

> Further, this code runs with the watchdog disabled. If there's truly no
> progress anymore, how would one know from the outside whether the system is
> dead altogether vs the control CPU still kicking around?

The scenario you describe can only occur if the BSP accepts the
microcode successfully, and one of the APs locks up properly.

If the BSP locks up, we never get as far as deciding to panic(), and the
system hangs already.

If we have a bad microcode, it is far more likely for the BSP to hang
than for the BSP to work one of the APs hang.

>> Microcode Loading on Granite Rapids takes about 4.5s of wallclock time, far in
>> excess of the of the arbitrary 1s Xen allows.  This time is spent in the WRMSR
>> to load the blob, and there's nothing the system can do but to sit and wait.
>> Despite the delay, the system as a whole does survive.
> For this I wonder whether the log message issues after 1 sec is adequate. If
> we know things can take this long, wouldn't we better issue the log message
> no earlier than, say, 5s * nr_sockets?
>

I have some work there not submitted yet, which would periodically print
a message.  That at least gives you slight signs of life.

But nothing involving nr_sockets.  It's far more often wrong than it is
right, and that's not how our algorithm scales.

GNR has gone from milliseconds to 4.5s.  Putting the limit at 5s is just
going to need another change in a year or two.

>> Microcode loading occures through admin operation only, so get rid of the
>> timeout completely.  It does nothing but make a bad sitaution worse.
> Worse when taking one perspective, yes, yet a silent hang also is worse than
> a panic() telling you what was wrong.
>
> This is a tough one, with - likely - no really good options. And establishing
> what's "least bad" may also be difficult.

As indicated, there's only one narrow and unlikely case where the
panic() is a true positive.

The likely true-hang case don't reach the panic(), and that only leaves
"timeout too short" which has proved to be the case on GNR.

~Andrew