From mboxrd@z Thu Jan 1 00:00:00 1970 From: Boris Ostrovsky Subject: Re: Xen 4.2.1 boot failure with IOMMU enabled Date: Thu, 14 Feb 2013 09:55:16 -0500 Message-ID: <511CFAD4.9040905@oracle.com> References: <7d6022b9-fff5-4cca-b091-347d3e869909@default> <511A2FA402000078000BDAF7@nat28.tlf.novell.com> <511A33F302000078000BDB2B@nat28.tlf.novell.com> <511A35C402000078000BDB47@nat28.tlf.novell.com> <511A5DC3.4010106@oracle.com> <511A763302000078000BDC82@nat28.tlf.novell.com> <511B574902000078000BDE41@nat28.tlf.novell.com> <511B5EE102000078000BDE8A@nat28.tlf.novell.com> <511CD8C202000078000BE280@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <511CD8C202000078000BE280@nat28.tlf.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: povder , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 02/14/2013 06:29 AM, Jan Beulich wrote: >>>> On 13.02.13 at 19:21, povder wrote: >> I don't see 06:00.1 device in IOMMU enabling process on which Xen crashes. >> >> lspci output: http://pastebin.com/raw.php?i=3wpKPQT9 > This is really odd: The "iommu=debug" output you made available > shows that while there are further devices that have no associated > IOMMU, the bus scan done in the hypervisor didn't even find a > device at 06:00.1. Which I see possible only in two ways: Either > the device becomes visible on the bus only when the driver for > 06:00.0 loads (and is otherwise detectable only by other means, > e.g. ACPI), or 06:00.0 doesn't have the multi function device flag > properly set. That latter aspect could be checked by looking at > the raw (hex) config space dump of 06:00.0. If I read this correctly, Linux enables multi-functionness (?): http://lxr.linux.no/#linux+v3.7.7/drivers/pci/quirks.c#L1494 So you are probably right. BIOS does not enumerate 06:00.1 in IVRS because it doesn't see it enabled yet. > > Boris, one other thought I had in this context: Is it really possible > for functions on the same (non-bridge) device to be serviced > by different IOMMUs? I can't see how this may be possible: IOMMU is PCIe root complex and any downstream device can only send transactions through its root. (I hope I am using right terminology). > If not, find_iommu_for_device() could simply > look for function 0 if nothing is known about the passed in function. Yes, this could work. But with a warning in the log. -boris