From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: q35 in xen? vfio in xen? Date: Mon, 24 Feb 2014 14:43:01 -0500 Message-ID: <20140224194301.GA8089@phenom.dumpdata.com> References: <201402220031.s1M0Vcxm023037@aserz7021.oracle.com> <3B22ECA2D19A3D408C83F4F15A9CB7D4507B7AA0@G9W0737.americas.hpqcorp.net> <20140224164004.GO816@phenom.dumpdata.com> <3B22ECA2D19A3D408C83F4F15A9CB7D4507B945A@G9W0737.americas.hpqcorp.net> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <3B22ECA2D19A3D408C83F4F15A9CB7D4507B945A@G9W0737.americas.hpqcorp.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Zhang, Eniac" , anthony.perard@citrix.com, Stefano Stabellini Cc: "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Mon, Feb 24, 2014 at 07:28:32PM +0000, Zhang, Eniac wrote: > Hi Konrad, > = > Thanks for the info. Your guest sees the virtual function as pci device= just like I had suspected. Unfortunately that won't work for me. I guess= I have to take a hard look at implement pci-e passthrough using pciback th= en. You won't have to do it with pciback. Keep in mind that pciback just "holds" the device so that other drivers (like ixbgvf) don't use it. 'xl' ends up doing the proper hypercall to assign the device to the guest. And QEMU also does it set of calls to setup the BARS, interrupts, deal with MSI-X, etc. What you are going to have to look at is QEMU - and how to make it work with the newer emulated chipset. Stefano (CC-ed) here is the maintainer of QEMU Xen pieces. Anthony (CC-ed as well) had backported the proper pieces in QEMU to do PCI passthrough. Looking forward to your patches and we will be more than happy to help you upstream them! > = > Regards/Eniac > = > -----Original Message----- > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] = > Sent: Monday, February 24, 2014 9:40 AM > To: Zhang, Eniac > Cc: xen-devel@lists.xen.org > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? > = > On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote: > > Hi Konrad, > > = > > Here's what I see when start a VM under xen using pciback to pass a pci= -e device into domU. The device can be seen by guest, and also functioning= fine. But it's not seen as a pci-e device, rather, it looks just like an = ordinary pci device because only the first 0x100 bytes of its configuration= space is accessible. So if a driver needs to use data in the extended con= figuration space for certain features, it will fail. > > = > > When you say you "did PCIe pass through of an VF of an SR-IOV device". = Are you actually using it as a pci-e device or have throttled it back to p= ci mode without aware of the difference? If you did see the pci-e device i= n guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xx= xx output from guest? > = > # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev = 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton= II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (re= v 01) # lspci -t > -[0000:00]-+-00.0 > +-01.0 > +-01.1 > +-01.3 > +-02.0 > +-03.0 > \-04.0 > # lspci -s 00:04.0 -xxxxx > 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (re= v 01) > 00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00 > 10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3 > 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0 > 30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00 > 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00 > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00 > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00 > d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > = > -bash-4.1# more /vm-pci.cfg > builder=3D'hvm' > disk =3D [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r'] > memory =3D 2048 > boot=3D"d" > maxvcpus=3D32 > vcpus=3D1 > serial=3D'pty' > vnclisten=3D"0.0.0.0" > name=3D"latest" > pci =3D ["0000:02:10.0"] > = > > = > > Also to echo your second comment: I might still be a newbie in qemu fi= eld (I started working on this 4 months ago). I thought the chipset limits= what you can see/do in vm. Ie. If you have 440fx emulations then you can= 't have any pci-e devices (fake or passthru) in the same system. Is that n= ot true? > > = > > Regards/Eniac > > = > > -----Original Message----- > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] > > Sent: Friday, February 21, 2014 5:32 PM > > To: Zhang, Eniac > > Cc: xen-devel@lists.xen.org > > Subject: RE: [Xen-devel] q35 in xen? vfio in xen? > > = > > = > > On Feb 21, 2014 4:58 PM, "Zhang, Eniac" wrote: > > > > > > Hi Konrad, > > > > > > Thanks for your reply. > > > > > > Yes, I am aware of the pciback.=A0 Unfortunately it doesn't seem to = > > > support pci-e passthrough. (I could be wrong here) > > = > > I just did PCIe pass through of an VF of an SR-IOV device. It certainly= is PCIe. > > = > > > > > > There are two reason that I am interested in this.=A0 For one, my pro= ject calls for pci-e device passthrough, which can't be accomplished with 4= 40fx chipset emulation.=A0 Secondly, I feel we ought to move on with the te= chnology.=A0 440fx is ancient in computer terms.=A0 Qemu is good and all th= at, but if it refuses to support pci-e natively then it's just a matter of = time that it will become obsoleted.=A0 The trend is clear that pci-e is tak= ing over the world. = > > > > > = > > I am not sure what you are saying but it does not matter whether QEMU e= mulates 440fx or q35 for PCI pass through . > > = > > > Regards/Eniac > > > > > > -----Original Message----- > > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] > > > Sent: Friday, February 21, 2014 2:50 PM > > > To: Zhang, Eniac > > > Cc: xen-devel@lists.xen.org > > > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? = > > > > > > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: = > > > > Hi all, > > > > = > > > > I am playing with q35 chipset in qemu (1.6.1).=A0 It seems we can't= enable q35 machine under xen yet.=A0 I made a few quick hacks which all fa= il miserably (linux kernel oops and window BSOD).=A0 I was wondering why th= is hasn't been done (q35 was introduced into qemu in 2009). = > > > > = > > > > Next question, vfio works very well for me in standalone qemu (with= Linux host handling iommu), but is that supported under xen?=A0 I haven't = tried anything there yet because my gut-feeling is that it won't work.=A0 B= ecause passing vfio device to qemu can only be done on qemu commandline, an= d xen is not aware of this passing through device, thus not able to make io= mmu arrangement for this device.=A0 Am I on the right track here? = > > > > > > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under = Xen. It uses a different mechanism (and you need to bind the device to pcib= ack). = > > > > > > > = > > > > I am interested in implementing both these two features.=A0 I'd lik= e to connect with anyone who's already on this so we don't duplicate the ef= forts. = > > > > > > What do you need Q35 for? = > > > > > > > = > > > > Regards/Eniac > > > > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@lists.xen.org > > > > http://lists.xen.org/xen-devel > > >