From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gordan Bobic Subject: Re: Multi-bridged PCIe devices (Was: Re: iommuu/vt-d issues with LSI MegaSAS (PERC5i)) Date: Wed, 11 Sep 2013 14:10:26 +0100 Message-ID: References: <523075C502000078000F2617@nat28.tlf.novell.com> <9bd7c39084e6b264dda5b6ac256ea97b@mail.shatteredsilicon.net> <52307EAA02000078000F26B1@nat28.tlf.novell.com> <3f1e678224a3f94125b5050b794882a8@mail.shatteredsilicon.net> <5230863202000078000F2712@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VJkBh-0008At-NM for xen-devel@lists.xenproject.org; Wed, 11 Sep 2013 13:10:29 +0000 Received: from mail.shatteredsilicon.net (localhost [127.0.0.1]) by external.sentinel2 (Postfix) with ESMTP id 6D4AD220294 for ; Wed, 11 Sep 2013 14:10:26 +0100 (BST) In-Reply-To: <5230863202000078000F2712@nat28.tlf.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On Wed, 11 Sep 2013 14:03:14 +0100, "Jan Beulich" wrote: >>>> On 11.09.13 at 14:45, Gordan Bobic wrote: >> dmesg, xl dmesg, lspci -vvvnn and lspci -tvnn output is attached. >> >> I'll try adding one of my LSI cards and see the comparative >> behaviour. Right now I don't even know if the phantom device >> is on the SAS card or the motherboard. > > The Adaptec card being the only thing on bus 0f makes it pretty > likely that this other device also is on that card. > > I guess the issue is mainly because the device itself is a PCI one, > while the immediately upstream bridge (where I mean only the > visible one) is PCIe. There _must_ be a PCIe-PCI bridge between > them. And as long as firmware doesn't know about that bridge > and the bridge doesn't properly handle config space accesses to > it, such a device just can't be used with an IOMMU (without some > yet to be invented workaround). I'm actually thinking about Konrad's proposed hack in that thread from 3 years ago. If the device IDs are parameterized out rather than hard-coded, then this could work in nearly the same was as xen-pciback in terms of usage. Pass the phantom device IDs as parameters to the module. Done that way it might even be considered clean enough to be fit for public consumption. Gordan