Re: NMI problems with Dell SMP Xeons

Andi Kleen
2006-06-07T07:20:31Z

On Wednesday 07 June 2006 06:49, Keith Owens wrote:
> Following a suggestion by Brendan Trotter, I ran some more tests to
> track down the problem with sending NMI IPI on Dell Xeons.
>
> BIOS Logical OS ACPI Cpus IPI 2 NMI IPI
> Processor BIOS OS (APIC_DM_NMI)
>
> Enabled Enabled 4 4 Not delivered Delivered as NMI
> Enabled Disabled 4 2 Machine reset Machine reset
> Disabled Enabled 2 2 Not delivered Delivered as NMI
> Disabled Disabled 2 2 Not delivered Delivered as NMI
>
> So the killer combination with this motherboard is when the BIOS knows
> about logical processors but the OS does not. Sending IPI 2 or NMI IPI
> with that combination kills the machine. Brendan suggested that the
> BIOS is seeing the broadcast NMI on the logical processors which are
> not under OS control and that the BIOS cannot cope.

How did you manage that? Normally the OS should use all CPUs
known to BIOS. Or did you boot with special boot options to limit it?

> Should we change the x86_64 send_IPI_allbutself() so it is only
> delivered to cpus that the OS knows about, instead of doing a general
> broadcast.

Hmm, we should be doing that already to avoid races for CPU hotplug. But
maybe it's not working correctly for KDB. Does it go away when you
enable CPU hotplug? Anyways, should be a SMOP to force it. I wouldn't
have a problem to use sequence ipis always and get rid of the broadcasts.
There were benchmarks at some point and there wasn't a noticeable
difference.

-Andi