From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jan Beulich" Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Mon, 19 May 2014 13:06:41 +0100 Message-ID: <537A0FF10200007800013973@mail.emea.novell.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> <5374DFCA.10207@gmail.com> <5375CD4F0200007800012E27@mail.emea.novell.com> <5375F410.2060406@gmail.com> <537614F30200007800013139@mail.emea.novell.com> <53763E9B0200007800013260@mail.emea.novell.com> <5379DD1A.6050106@gmail.com> <5379FB2902000078000137EC@mail.emea.novell.com> <5379E149.1020504@gmail.com> <537A020B0200007800013865@mail.emea.novell.com> <5379EBC1.5050902@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmML7-0001nl-6S for xen-devel@lists.xenproject.org; Mon, 19 May 2014 12:06:45 +0000 In-Reply-To: <5379EBC1.5050902@gmail.com> Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Tomasz Wroblewski Cc: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org >>> On 19.05.14 at 13:32, wrote: > On 05/19/2014 01:07 PM, Jan Beulich wrote: >>>>> On 19.05.14 at 12:47, wrote: >>> On 05/19/2014 12:38 PM, Jan Beulich wrote: >>>> So perhaps time for sending complete logs, plus suitable information >>>> from inside the guest of how things (RAM, MMIO, MTRRs) end up being >>>> set up? >>> Could be, though please read the explanation I came up in the other post >>> whether its enough, I think it makes sense... 64bit guest BARs are >>> indeed not in use (confirmed from guest). MTRR is setup such that only >>> the low region is UC, which is correct. >> Yes, that's a very sensible theory, which - as just said in the other >> reply - can be easily verified. >> >>> But the RAM relocation code causes the caching on relocated region to be >>> UC instead of WB due to the timing (very early, MTRR disabled) at which >>> it runs, which is incorrect. I am thinking enabling MTRR during that >>> relocation would probably fix it on 4.3 >> Except that this is a chicken and egg problem then: In order to >> populate the variable range MTRRs, the BAR assignment (and hence >> the prerequisite RAM relocation) need to be done already. > I am not sure; looking at hvmloader code, wouldn't it be possible to > calculate the BAR locations first, then update the MTRR var ranges and > enable it, and only then actually write the BAR registers (from > precalculated info)? Presumably it's only the write part which needs to > be done after relocation as it causes qemu to setup mmio etc. Leaving aside that this would require splitting pci_setup(), and hence communicating state from its main part (RAM relocation and resource allocation) to the final one (BAR writing), which by itself is already not as simple a change as one would like for something that is intended to go _only_ into the stable trees, you also already imply with the above that we'd add a pre-enabling step for the MTRRs. I.e. we'd end up with - enable fixed-range MTRRs and set default to WB (no var ranges) - pci_setup_early() - set variable range MTRRs - pci_setup_late() - set MTRRs in one go on APs Yes, that ought to work. But do we want this much diverging from -unstable on 4.3 and 4.4? Are we certain that namely the two-stage MTRR setup won't have any unintended side effects? > Yeah I gave about a day of effort to port us onto unstable and test > there but it sadly looks to be a bigger job, so leaving that as a last > resort (though planning to spend couple more days on it soon). Then as an alternative did you try pulling over the EPT changes from -unstable? Jan