From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: GPU passthrough issue when VM is configured with 4G memory Date: Wed, 6 Mar 2013 14:45:45 -0500 Message-ID: <20130306194545.GA13533@phenom.dumpdata.com> References: <33183CC9F5247A488A2544077AF1902038628410@szxeml538-mbx.china.huawei.com> <513739DD.8050507@eu.citrix.com> <20130306140439.GQ8912@reaktio.net> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <20130306140439.GQ8912@reaktio.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= Cc: Hanweidong , George Dunlap , Yanqiangjun , Luonengjun , Wangzhenguo , Yangxiaowei , "Gonglei (Arei)" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Wed, Mar 06, 2013 at 04:04:39PM +0200, Pasi K=E4rkk=E4inen wrote: > On Wed, Mar 06, 2013 at 12:43:09PM +0000, George Dunlap wrote: > > On 06/03/13 11:38, Hanweidong wrote: > > >> -----Original Message----- > > >> From: dunlapg@gmail.com [mailto:dunlapg@gmail.com] On Behalf Of Geor= ge > > >> Dunlap > > >> Sent: 2013??3??5?? 20:59 > > >> To: Gonglei (Arei) > > >> Cc: xen-devel@lists.xen.org; Yangxiaowei; Yanqiangjun; Luonengjun; > > >> Wangzhenguo; Hanweidong > > >> Subject: Re: [Xen-devel] GPU passthrough issue when VM is configured > > >> with 4G memory > > >> > > >> On Mon, Mar 4, 2013 at 8:10 AM, Gonglei (Arei) > > >> wrote: > > >>> Hi,all > > >>> > > >>> I have tried to passthrough GPU card(Nvidia quadro 4000) on the > > >> latest Xen > > >>> unstable version (QEMU is using Qemu-upsteam-unstable, not > > >> traditional > > >>> Qemu). This issue as below: > > >>> > > >>> Windows7 64-bit guest will blue screen when GPU passthrough > > >> configure > > >>> 4g memory,blue screen code is 50, and SUSE 11 64-bit guest will > > >> always stay > > >>> at the grub screen. I noticed that it will relocate RAM that > > >> overlaps PCI > > >>> space in pci_setup()(tools/hvmloader/pci.c). If VM memory is > > >> configured with > > >>> 3G, it won't cause relocate RAM that overlaps PCI space in > > >> pci_setup(), and > > >>> GPU pass-through is no problem. So it seems this issue is related to > > >>> "relocate RAM" in pci_setup(). > > >> So one issue XenServer found with passing through GPUs is that there > > >> are bugs in some PCI bridges that completely break VT-d. The issue > > >> was that if the *guest* physical address space overlapped the *host* > > >> physical address of a different device, that the PCI bridges would > > >> send traffic from the passed-through card intended for the guest to > > >> another card instead. The work-around was to make the hole in the > > >> guest MMIO space the same size as the host MMIO hole. I'm not sure = if > > >> that made it upstream or not -- let me check... > > >> > > > Hi George, > > > > > > Could you post your patch and let us have a try with it? Thanks! > > = > > So the patch got checked in, but there still may be some more work if > > you want to use it. :-) > > = > > The patch modifies xc_hvm_build_args structure to include a field called > > "mmio_size". If this is set to zero, it will default to > > HVM_BELOW_4G_MMIO_LENGTH; otherwise, it will be the size of the default > > MMIO hole set during the build process. The guest BIOS may modify this > > at boot time to make it bigger, but it doesn't make it smaller. > > = > > Since this was designed for xapi, however, which calls libxc directly, > > we didn't add any options to xend / xl / libxl to set this option. > > = > > The easiest way to test it probably is just to hard-code > > HVM_BELOW_4G_MMIO_LENGTH to a new value (from the description, setting > > it to 1GiB should be sufficient). > > = > > Then if you want to use it in production, you probably want to either: > > 1. Try it with the latest version of XCP (which I think has an option > > you can set) > > 2. Implement a config option for xl that allows you to set the MMIO hole > > size. > > = > > #2 should be a relatively straightforward matter of "plumbing", and > > would be a welcome contribution. :-) > > = > > If you do implement #2, it might be nice to have an option of > > "mmio_hole_size=3Dhost", which will set the guest mmio hole to the same > > size as the host. That's what we implemented for XenServer, to make sure > > there would never be any collisions. > > = > = > does the e820_host=3D option affect this? = No. That is for PV guests only.