From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Thu, 15 May 2014 17:39:54 +0200 Message-ID: <5374DFCA.10207@gmail.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Wkyi1-0002iD-0T for xen-devel@lists.xenproject.org; Thu, 15 May 2014 16:40:41 +0000 Received: by mail-ee0-f46.google.com with SMTP id t10so819461eei.5 for ; Thu, 15 May 2014 09:40:39 -0700 (PDT) In-Reply-To: <537502710200007800012C7E@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: Jinsong Liu , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 05/15/2014 06:07 PM, Jan Beulich wrote: >>>> On 15.05.14 at 16:56, wrote: >> On 05/15/2014 04:34 PM, Tomasz Wroblewski wrote: >>> On 05/15/2014 03:39 PM, Tomasz Wroblewski wrote: >>>> On 05/15/2014 03:23 PM, Jan Beulich wrote: >>>>>>>> On 15.05.14 at 14:10, wrote: >>>>>> Not really sure why it only affects 64bit vms but I've just noticed >>>>>> the >>>>>> pci BARs for the card are being relocated by hvmloader as per some >>>>>> logs: >>>>>> >>>>>> (XEN) HVM3: Relocating guest memory for lowmem MMIO space enabled >>>>>> (XEN) HVM3: Relocating 0xffff pages from 0e0001000 to 14dc00000 for >>>>>> lowmem MMIO hole >>>>>> (XEN) HVM3: Relocating 0x1 pages from 0e0000000 to 15dbff000 for >>>>>> lowmem >>>>>> MMIO hole >>>>>> >>>>>> So it might be also related to that. >>>>> Indeed it might - what are the (guest) MTRR types for those regions? >>>> It's writeback for both the 32bit and 64bit above ranges. >>> ... however, after a bit more debugging its uncached at the time >>> hvmloader does the relocation so that's why it ends up like that in >>> EPT tables. It does go to writeback only soon after. Haven't >>> pinpointed the exact time point for that yet nor why it's being >>> updated to writeback, but it seems to be before the guest starts >>> booting (i.e. still on bios screens). >> ... and after even more I see that the type is uncached at the time the >> relocation is happening because mtrr is disabled at that time and >> get_mtrr_type() function exits with uncached value in the first few >> lines of it. Later when guests enabled MTRR, ept is not updated. So >> maybe the EPTs should be updated in some way at that time, > Which is what -unstable is now doing. Right. > But the question remains why this region doesn't get marked UC or > WC, but WB. The region doesn't seem to be marked in any way in mtrr so it just goes off the default type for that mtrr ((struct mtrr_state*)->def_type) which seems to be WB. > Jan >