From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes Date: Fri, 16 May 2014 13:18:40 +0200 Message-ID: <5375F410.2060406@gmail.com> References: <537484A9.9000001@gmail.com> <5374CFF80200007800012A53@mail.emea.novell.com> <5374AEBD.7090403@gmail.com> <5374DBFD0200007800012AEE@mail.emea.novell.com> <5374C389.507@gmail.com> <5374D08F.2050202@gmail.com> <5374D5B0.2080808@gmail.com> <537502710200007800012C7E@mail.emea.novell.com> <5374DFCA.10207@gmail.com> <5375CD4F0200007800012E27@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WlGA0-0002m8-Rc for xen-devel@lists.xenproject.org; Fri, 16 May 2014 11:18:45 +0000 Received: by mail-ee0-f49.google.com with SMTP id e53so1433074eek.22 for ; Fri, 16 May 2014 04:18:43 -0700 (PDT) In-Reply-To: <5375CD4F0200007800012E27@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: Jinsong Liu , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org >>>> ... and after even more I see that the type is uncached at the time the >>>> relocation is happening because mtrr is disabled at that time and >>>> get_mtrr_type() function exits with uncached value in the first few >>>> lines of it. Later when guests enabled MTRR, ept is not updated. So >>>> maybe the EPTs should be updated in some way at that time, >>> Which is what -unstable is now doing. >> Right. >>> But the question remains why this region doesn't get marked UC or >>> WC, but WB. >> The region doesn't seem to be marked in any way in mtrr so it just goes >> off the default type for that mtrr ((struct mtrr_state*)->def_type) >> which seems to be WB. > I was expecting this for the relocated (above 4Gb) region, but is this > also the case for the one below? You're right, I had a silly typo in the pfn number I was testing, the lower region indeed does have entry in the MTRR and it is set to UC. The relocated region doesn't. > In any event - all MMIO regions of passed through devices absolutely > have to be represented in the MTRRs as long as the regions' types > in the host MTRRs differ from the default type in the guest ones, > though in the end this makes me (once again) question whether > defaulting to WB and setting up exceptions for MMIO isn't the wrong > approach especially when pass-through is being used. This used to > be the other way around until April 2008 (commit a6a82232: > "x86, hvm: Lots of MTRR/PAT emulation cleanup"). > > If I coded up a patch to deal with this on -unstable, would you be > able to test that? Willing to give it a go (xen major version updates are often problematic to do though so can't promise success). What would your patch be doing? Adding entries to MTRR for the relocated regions? I do fear it might not be enough since thru recent debugging it seems there are more wrongs done by the relocation code - the p2m entries for the relocated region are not setup with p2m_mmio_direct for example but rather default p2m_ram_rw (unless this has changed again past 4.3.2). > Jan >