From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Mon, 19 May 2014 12:29:46 +0200 Message-ID: <5379DD1A.6050106@gmail.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> <5374DFCA.10207@gmail.com> <5375CD4F0200007800012E27@mail.emea.novell.com> <5375F410.2060406@gmail.com> <537614F30200007800013139@mail.emea.novell.com> <53763E9B0200007800013260@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmKpL-0002dC-7A for xen-devel@lists.xenproject.org; Mon, 19 May 2014 10:29:51 +0000 Received: by mail-wg0-f45.google.com with SMTP id m15so7509987wgh.28 for ; Mon, 19 May 2014 03:29:49 -0700 (PDT) In-Reply-To: <53763E9B0200007800013260@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 05/16/2014 04:36 PM, Jan Beulich wrote: >>>> On 16.05.14 at 13:38, wrote: >>>>> On 16.05.14 at 13:18, wrote: >>>> If I coded up a patch to deal with this on -unstable, would you be >>>> able to test that? >>> Willing to give it a go (xen major version updates are often problematic >>> to do though so can't promise success). What would your patch be doing? >>> Adding entries to MTRR for the relocated regions? >> This and properly declare the region in ACPI's _CRS. For starters I'll >> probably try keeping the WB default overlaid with UC variable ranges, >> as that's going to be the less intrusive change. > Okay here are two patches - the first to deal with the above mentioned > items, and the second to further increase correctness and at once > shrink the number of MTRR regions needed. > > Afaict they apply equally well to stable-4.3, master, and staging. > > But to be honest I don't expect any performance improvement, all > I'd expect is that BARs relocated above 4Gb would now get treated > equally to such below 4Gb - UC in all cases. Thanks Jan. I've tried the patches and you're correct, putting UC in MTRR for the relocated region didn't help the issue. However, I had to hack that manually - the codepaths to do that in your hvmloader patch were not activating. The hvmloader is not programming guest pci bars to 64bit regions at all, rather still programming them with 32 bit regions... upon a look this seems because using_64bar conditon, as well as bar64_relocate in hvmloader/pci.c is always false. So bar relocation to 64bit is not happening, but ram relocation as per the code tagged as /* Relocate RAM that overlaps PCI space (in 64k-page chunks). */ is happening. This maybe is correct (?), although I think the fact that RAM is relocated but not the BAR causes the tools (i.e. qemu) to lose sight of what memory is used for mmio and as you mentioned in one of the previous posts, the calls which would set it to mmio_direct in p2m table are not happening. Our qemu is pretty ancient and doesn't support 64bit bars so its not super trivial to verify whether relocating bars to 64bit would help. Trying to make sense out of this..