Re: NMI stack overflow during resume of PCIe bridge with CONFIG_HARDLOCKUP_DETECTOR=y

Bert Karwatzki
2026-01-13T22:20:57Z

Am Dienstag, dem 13.01.2026 um 20:30 +0100 schrieb Thomas Gleixner:
> On Tue, Jan 13 2026 at 18:50, Bert Karwatzki wrote:
> > Am Dienstag, dem 13.01.2026 um 16:24 +0100 schrieb Thomas Gleixner:
> >
> > > What's more likely is that after a while _ALL_ CPUs are hung up in
> > > the NMI handler after they tripped over the HPET read.
> >
> > I'm not sure about that, my latest testrun (with v6.18) crashed with
> > only one message from exc_nmi().
>
> What means crashed? Did it actually crash and output something or does
> the machine just go dead? I assume the latter as you have no output.
>
Crashed means, rebooting and trying to reboot into the new kernel, which fails
(i.e. drops to rescue mode)
because one of my 2 nvme devices (not the root fs) is missing from the PCI bus
(the missing device reappears after a shutdown).
When sound is running while such a crash occurs the sound loops for about ~2s,
before such a reboot occurs (a behaviour I've seen during an unrelated NULL ptr deref in an
interrupt handler once). There's no additional error message output when such
a crash occurs, only my printk()s.

> > > along with the full output of /proc/iomem
> >
> > The physical address is 0xf0100000
> >
> > $ cat /proc/iomem
> > f0000000-fcffffff : PCI Bus 0000:00
> > f0000000-f7ffffff : PCI ECAM 0000 [bus 00-7f]
> > f0000000-f7ffffff : pnp 00:00
>
> That's the memory mapped PCI config space and this tries to access:
>
> MMIO_START 0xf0000000
> BUSNUM 0x01 << 20 0x00100000
> SLOT/FN 0x00 << 12 0x00000000
> OFFSET 0x00 << 0 0x00000000
> -----------
> 0xf0100000
>
> Offset 0 is vendor/device ID IIRC.
>
> Anyway if that access does not complete because of a hardware issue,
> then any subsequent access to the MMIO mapped HPET goes stale as well.
>
> As the HPET is the active clocksource on your machine, this obviously
> does not only affect the NMI watchdog readout, it affects the regular
> timekeeper accesses too and all other MMIO accesses all over the place.
> So gradually your machine just stalls on outstanding MMIO transactions
> w/o further notice... The NMI is just a red herring.
>
I've already tried different clocsources, with mixed results. tsc does not
work on my system as it is unstable, while acpi_pm also crashed. With jiffies
I had mixed results. It might have avoided one crash, but crashed on another
testrun in the same situation.

> You need to figure out why that MMIO access to that device's
> configuration space stalls as anything else is just subsequent
> damage.
>
> There is not much what can be done about that unless the PCI bus raises
> a failure interrupt and some magic reset sequence aborts the outstanding
> stalled transactions.
>
> Whether that's feasible or not, I don't know. The failure mechanism
> might run into the same stall scenario when accessing the PCI muck for
> reset...

In the two testruns without CONFIG_HARDLOCKUP_DETECTOR I simply lost the discrete
GPU without a crash, i.e. after 4 and 8.5h the GPU would not resume any more, and
accessing the GPU with DRI_PRIME=1 glxinfo just gives a (user) segfault. But I need
more testruns to see if this behaviour is consistent.

>
> Sorry for not being helpful.
>
> Thanks,
>
> tglx
>

Thanks you

Bert Karwatzki