From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Mon, 19 May 2014 13:32:17 +0200 Message-ID: <5379EBC1.5050902@gmail.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> <5374DFCA.10207@gmail.com> <5375CD4F0200007800012E27@mail.emea.novell.com> <5375F410.2060406@gmail.com> <537614F30200007800013139@mail.emea.novell.com> <53763E9B0200007800013260@mail.emea.novell.com> <5379DD1A.6050106@gmail.com> <5379FB2902000078000137EC@mail.emea.novell.com> <5379E149.1020504@gmail.com> <537A020B0200007800013865@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmLnr-0006cM-R7 for xen-devel@lists.xenproject.org; Mon, 19 May 2014 11:32:24 +0000 Received: by mail-wi0-f174.google.com with SMTP id r20so3939270wiv.7 for ; Mon, 19 May 2014 04:32:20 -0700 (PDT) In-Reply-To: <537A020B0200007800013865@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 05/19/2014 01:07 PM, Jan Beulich wrote: >>>> On 19.05.14 at 12:47, wrote: >> On 05/19/2014 12:38 PM, Jan Beulich wrote: >>> So perhaps time for sending complete logs, plus suitable information >>> from inside the guest of how things (RAM, MMIO, MTRRs) end up being >>> set up? >> Could be, though please read the explanation I came up in the other post >> whether its enough, I think it makes sense... 64bit guest BARs are >> indeed not in use (confirmed from guest). MTRR is setup such that only >> the low region is UC, which is correct. > Yes, that's a very sensible theory, which - as just said in the other > reply - can be easily verified. > >> But the RAM relocation code causes the caching on relocated region to be >> UC instead of WB due to the timing (very early, MTRR disabled) at which >> it runs, which is incorrect. I am thinking enabling MTRR during that >> relocation would probably fix it on 4.3 > Except that this is a chicken and egg problem then: In order to > populate the variable range MTRRs, the BAR assignment (and hence > the prerequisite RAM relocation) need to be done already. I am not sure; looking at hvmloader code, wouldn't it be possible to calculate the BAR locations first, then update the MTRR var ranges and enable it, and only then actually write the BAR registers (from precalculated info)? Presumably it's only the write part which needs to be done after relocation as it causes qemu to setup mmio etc. > What we > might be able to do (after careful evaluation whether backporting > what we have on -unstable wouldn't be the better option, which > first of all implies you'd need to test things there) is enable MTRRs > with WB default, but without any variable ranges set up _before_ > doing the RAM relocation/BAR assignment. The main problem to solve > there, however, would still be to make sure the EPT entries get > re-created for the regions that we don't want to be WB (after the > variable ranges got set up). That, I'm afraid, would still lead us to > considerations on backporting at least some of the changes from > -unstable. Yeah I gave about a day of effort to port us onto unstable and test there but it sadly looks to be a bigger job, so leaving that as a last resort (though planning to spend couple more days on it soon). > Jan >