From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: [PATCH] KEXEC: disconnect all PCI devices from the PCI bus on crash Date: Thu, 7 Jul 2011 11:02:45 +0100 Message-ID: <4E158445.6050209@citrix.com> References: <7ea606c5ce8c8fcfc577.1309955952@andrewcoop.uk.xensource.com> <20110706184233.GA30520@dumpdata.com> <1310028827.634.199.camel@zakaz.uk.xensource.com> <4E159421020000780004C8EA@nat28.tlf.novell.com> <1310029954.634.204.camel@zakaz.uk.xensource.com> <4E159B94020000780004C91E@nat28.tlf.novell.com> <1310032414.634.208.camel@zakaz.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1310032414.634.208.camel@zakaz.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Campbell Cc: "xen-devel@lists.xensource.com" , Jan Beulich , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org On 07/07/11 10:53, Ian Campbell wrote: > On Thu, 2011-07-07 at 10:42 +0100, Jan Beulich wrote: >>>>> On 07.07.11 at 11:12, Ian Campbell wrote: >>> On Thu, 2011-07-07 at 10:10 +0100, Jan Beulich wrote: >>>>>>> On 07.07.11 at 10:53, Ian Campbell wrote: >>>>> On Wed, 2011-07-06 at 19:42 +0100, Konrad Rzeszutek Wilk wrote: >>>>>> On Wed, Jul 06, 2011 at 01:39:12PM +0100, Andrew Cooper wrote: >>>>>>> +/* Disconnect a PCI device from the PCI bus. From the PCI spec: >>>>>>> + * "When a 0 is written to [the COMMAND] register, the device is >>>>>>> + * logically disconnected from the PCI bus for all accesses except >>>>>>> + * configuration accesses. All devices are required to support >>>>>>> + * this base level of functionality." >>>>>>> + */ >>>>>>> +void disconnect_pci_device(struct pci_dev *pdev) >>>>>>> +{ >>>>>>> + pci_conf_write16(pdev->bus, PCI_SLOT(pdev->devfn), >>>>>>> + PCI_FUNC(pdev->devfn), PCI_COMMAND, 0); >>>>>> So if you have a PCI serial card (or Intel AMT) and you are using that for >>>>>> serial output on the hypervisor line, this will turn it off. There should >>>>>> be some whitelist capability to not do it for PCI serial devices that are >>>>>> owned (used) by the hypervisor. >>>>> That would be useful for debugging the kexec process itself but in the >>>>> general case there won't be any further output from the hypervisor and >>>>> if the kexec'd kernel wants to use the device it is going to have to set >>>>> it up again anyways. >>>> No, not generally. Just look at Linux' early-printk code: The device >>>> is assumed to be enabled (by the BIOS), as the PCI subsystem can't >>>> possibly be initialized at this point already. >>> That's arguably a debugging facility as well though. >>> >>>> This also means that white-listing just devices Xen uses may not be >>>> enough: If Xen doesn't use a serial console (or the secondary kernel >>>> wants to use some other device Xen doesn't care about - VGA or >>>> other kind of console devices come to mind), it must not find it fully >>>> disconnected from the bus. Consequently I would think that while >>>> interrupt and DMA activity should be forced off, decoding I/O and >>>> memory addresses by the devices shouldn't be. >>> The problem is that this can't be done without device specific >>> knowledge, which the hypervisor generally doesn't have and we can't get >>> the device's owning domain to do anything because we are crashing. >> Why would there be any device specific knowledge needed? It's >> all done through the command word, just that writing zero isn't >> really appropriate. > So presumably if you disable bus mastering you've effectively disabled > DMA but how do you disable interrupts via the command word? > > Ian. > Bit 10 of the control word is "disable assertion of INTx# pins" (so set bit to 1 to disable interrupts). That should cover legacy interrupts. For MSI and above, disabling DMA should prevent the bus writing to the magic local APIC addresses. -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com