Re: [PATCH] KEXEC: disconnect all PCI devices from the PCI bus on crash

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 07/07/11 10:53, Ian Campbell wrote:
> On Thu, 2011-07-07 at 10:42 +0100, Jan Beulich wrote:
>>>>> On 07.07.11 at 11:12, Ian Campbell <Ian.Campbell@eu.citrix.com> wrote:
>>> On Thu, 2011-07-07 at 10:10 +0100, Jan Beulich wrote:
>>>>>>> On 07.07.11 at 10:53, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>>>> On Wed, 2011-07-06 at 19:42 +0100, Konrad Rzeszutek Wilk wrote:
>>>>>> On Wed, Jul 06, 2011 at 01:39:12PM +0100, Andrew Cooper wrote:
>>>>>>> +/* Disconnect a PCI device from the PCI bus.  From the PCI spec:
>>>>>>> + *     "When a 0 is written to [the COMMAND] register, the device is
>>>>>>> + *     logically disconnected from the PCI bus for all accesses except
>>>>>>> + *     configuration accesses. All devices are required to support
>>>>>>> + *     this base level of functionality."
>>>>>>> + */
>>>>>>> +void disconnect_pci_device(struct pci_dev *pdev)
>>>>>>> +{
>>>>>>> +    pci_conf_write16(pdev->bus, PCI_SLOT(pdev->devfn),
>>>>>>> +                     PCI_FUNC(pdev->devfn), PCI_COMMAND, 0);
>>>>>> So if you have a PCI serial card (or Intel AMT) and you are using that for
>>>>>> serial output on the hypervisor line, this will turn it off. There should
>>>>>> be some whitelist capability to not do it for PCI serial devices that are
>>>>>> owned (used) by the hypervisor.
>>>>> That would be useful for debugging the kexec process itself but in the
>>>>> general case there won't be any further output from the hypervisor and
>>>>> if the kexec'd kernel wants to use the device it is going to have to set
>>>>> it up again anyways.
>>>> No, not generally. Just look at Linux' early-printk code: The device
>>>> is assumed to be enabled (by the BIOS), as the PCI subsystem can't
>>>> possibly be initialized at this point already.
>>> That's arguably a debugging facility as well though.
>>>
>>>> This also means that white-listing just devices Xen uses may not be
>>>> enough: If Xen doesn't use a serial console (or the secondary kernel
>>>> wants to use some other device Xen doesn't care about - VGA or
>>>> other kind of console devices come to mind), it must not find it fully
>>>> disconnected from the bus. Consequently I would think that while
>>>> interrupt and DMA activity should be forced off, decoding I/O and
>>>> memory addresses by the devices shouldn't be.
>>> The problem is that this can't be done without device specific
>>> knowledge, which the hypervisor generally doesn't have and we can't get
>>> the device's owning domain to do anything because we are crashing.
>> Why would there be any device specific knowledge needed? It's
>> all done through the command word, just that writing zero isn't
>> really appropriate.
> So presumably if you disable bus mastering you've effectively disabled
> DMA but how do you disable interrupts via the command word?
>
> Ian.
>
Bit 10 of the control word is "disable assertion of INTx# pins"  (so set
bit to 1 to disable interrupts).  That should cover legacy interrupts. 
For MSI and above, disabling DMA should prevent the bus writing to the
magic local APIC addresses.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com