From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Mon, 19 May 2014 14:17:30 +0200 Message-ID: <5379F65A.7020609@gmail.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> <5374DFCA.10207@gmail.com> <5375CD4F0200007800012E27@mail.emea.novell.com> <5375F410.2060406@gmail.com> <537614F30200007800013139@mail.emea.novell.com> <53763E9B0200007800013260@mail.emea.novell.com> <5379DD1A.6050106@gmail.com> <5379FB2902000078000137EC@mail.emea.novell.com> <5379E149.1020504@gmail.com> <537A020B0200007800013865@mail.emea.novell.com> <5379EBC1.5050902@gmail.com> <537A0FF10200007800013973@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmMVe-0002lo-7n for xen-devel@lists.xenproject.org; Mon, 19 May 2014 12:17:38 +0000 Received: by mail-wi0-f181.google.com with SMTP id n15so4041856wiw.8 for ; Mon, 19 May 2014 05:17:36 -0700 (PDT) In-Reply-To: <537A0FF10200007800013973@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 05/19/2014 02:06 PM, Jan Beulich wrote: >>>> On 19.05.14 at 13:32, wrote: >> On 05/19/2014 01:07 PM, Jan Beulich wrote: >>>>>> On 19.05.14 at 12:47, wrote: >>>> On 05/19/2014 12:38 PM, Jan Beulich wrote: >>>>> So perhaps time for sending complete logs, plus suitable information >>>>> from inside the guest of how things (RAM, MMIO, MTRRs) end up being >>>>> set up? >>>> Could be, though please read the explanation I came up in the other post >>>> whether its enough, I think it makes sense... 64bit guest BARs are >>>> indeed not in use (confirmed from guest). MTRR is setup such that only >>>> the low region is UC, which is correct. >>> Yes, that's a very sensible theory, which - as just said in the other >>> reply - can be easily verified. >>> >>>> But the RAM relocation code causes the caching on relocated region to be >>>> UC instead of WB due to the timing (very early, MTRR disabled) at which >>>> it runs, which is incorrect. I am thinking enabling MTRR during that >>>> relocation would probably fix it on 4.3 >>> Except that this is a chicken and egg problem then: In order to >>> populate the variable range MTRRs, the BAR assignment (and hence >>> the prerequisite RAM relocation) need to be done already. >> I am not sure; looking at hvmloader code, wouldn't it be possible to >> calculate the BAR locations first, then update the MTRR var ranges and >> enable it, and only then actually write the BAR registers (from >> precalculated info)? Presumably it's only the write part which needs to >> be done after relocation as it causes qemu to setup mmio etc. > Leaving aside that this would require splitting pci_setup(), and > hence communicating state from its main part (RAM relocation and > resource allocation) to the final one (BAR writing), which by itself is > already not as simple a change as one would like for something that > is intended to go _only_ into the stable trees, you also already > imply with the above that we'd add a pre-enabling step for the > MTRRs. I.e. we'd end up with > > - enable fixed-range MTRRs and set default to WB (no var ranges) > - pci_setup_early() > - set variable range MTRRs > - pci_setup_late() > - set MTRRs in one go on APs > > Yes, that ought to work. But do we want this much diverging from > -unstable on 4.3 and 4.4? Are we certain that namely the two-stage > MTRR setup won't have any unintended side effects? > >> Yeah I gave about a day of effort to port us onto unstable and test >> there but it sadly looks to be a bigger job, so leaving that as a last >> resort (though planning to spend couple more days on it soon). > Then as an alternative did you try pulling over the EPT changes > from -unstable? That would be indeed preferable, I've looked over them but couldn't figure out which particular change would fix the EPT update after MTRR enable. Do you remember which that was? I could test it and try to narrow any other commits it'd require (seems there were a lot of ept related changes) > Jan >