Crashdump and IOMMU problems

[Andrew Cooper <andrew.cooper3@citrix.com>]

Hello,

I have been debugging kexec interaction problems with XenServer and 
found that the problem lies in how Xen tares down the computer in a crash.

The Xen kexec path does not touch IOMMU at all, which leaves the kexec 
native kernel with interrupt remapping enabled without realizing it.  
This leads to the kexec kernel failing to understand why its interrupts 
aren't working.

As a debugging measure, I have put iommu_ops->suspend() and 
iommu_disable_IR() on the kexec path and this 'fixes' the problem, 
although it is far from safe.

 From a correctness point of view, Xen really does need to shutdown all 
IOMMU remapping before it jumps to the crash kernel.  I know that kdump 
is a "seat of the pants best effort" in the best case, but there is more 
which Xen needs to do to help it along.  I was considering adding a 
crash_shutdown function to iommu_ops which goes and twiddles the 
relevant disable bits, without saving state.

However, disabling DMA remapping while transfers are still ongoing is 
likely asking for trouble.  Seeing as people on here are likely to know 
far more than me on this subject:

1) Is there a systematic way to find and disable active DMA transfers, 
or indeed a systematic way to shut down PCI (etc) devices which is safe 
for the kexec path.
2) Are there any other PC subsystems which could do with being shut down 
in a sensible manor to make life easier for the kdump kernel?

Thanks in advance,

~Andrew

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com