[REGRESSION] High interrupt jitter on PowerNV VFIO INTx passthrough

[REGRESSION] High interrupt jitter on PowerNV VFIO INTx passthrough

[Timothy Pearson]

On POWER9/PowerNV, when passing through an Atcom AXE400PL PCIe DAHDI card which uses INTx signalling with the wctdm driver [1], we are now seeing high interrupt jitter / lost interrupts after upgrading to kernel 6.12.  This jitter and interrupt loss makes the card unusable, and persists in both XIVE and XICS mode.  The card used to work correctly on kernel 6.1.x and earlier.

The easiest way to demonstrate the problem is to install a TigerJet-base DAHDI card that uses the wctdm driver, and run the dahdi_test utility.  If it shows anything less than 99.99%, interrupts are being lost or are arriving at improper times.  Example on the affected hardware:

dahdi_test
Opened pseudo dahdi interface, measuring accuracy...
99.122% 99.915% 98.833% 99.704% 99.423% 100.000% 99.418% 99.995%
97.654% 99.030% 99.707% 98.831% 99.671% 99.457% 99.612% 99.512%
97.371% 99.999% 99.808% 96.005% 99.909% 99.999% 99.811% 97.755%
99.030% 99.314% 97.953% 99.998% 99.908% 99.127% 98.244% 99.804%
<Ctrl+C>
--- Results after 32 passes ---
Best: 100.000% -- Worst: 96.005% -- Average: 99.184939%
Cumulative Accuracy (not per pass): 99.191

Unfortunately, given IBM's continued moves away from open firmware systems and OpenPOWER in general (see Power10/Power11 and binary firmware requirements), we are moving this workload onto open firmware amd64 systems.  As a result, I will not be able to provide further information or assist with debugging.  However, I expect this issue to affect other hardware, including common Ethernet controllers that don't support MSI; the DAHDI hardware just exposes very visible symptoms due to its real-time nature vs. other affected cards.

[1] https://github.com/asterisk/dahdi-linux/blob/master/drivers/dahdi/wctdm.c