From mboxrd@z Thu Jan  1 00:00:00 1970
From: Gianluca Guida <gianluca.guida@eu.citrix.com>
Subject: Re: Re: Poor performance on HVM (kernbench)
Date: Thu, 11 Sep 2008 19:26:32 +0100
Message-ID: <48C962D8.5030102@eu.citrix.com>
References: <1e16a9ed0809101123m71a12030v7d06501f6467f93@mail.gmail.com>	
	<1e16a9ed0809101422p6a58304dxaa1a92847109a518@mail.gmail.com>	
	<48C8EC48.6080507@eu.citrix.com> <48C9367B.6090003@eu.citrix.com>	
	<1e16a9ed0809110835p73165150t3f78d75b2f2ca8eb@mail.gmail.com>	
	<48C9548C.4060709@eu.citrix.com>
	<de76405a0809111107r387487b5s7aa6d87d0401ce5@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
In-Reply-To: <de76405a0809111107r387487b5s7aa6d87d0401ce5@mail.gmail.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: George Dunlap <dunlapg@umich.edu>
Cc: Muli Ben-Yehuda <MULI@il.ibm.com>, deshantm@gmail.com, Anthony Liguori <aliguori@us.ibm.com>, xen-devel mailing list <xen-devel@lists.xensource.com>
List-Id: xen-devel@lists.xenproject.org

George Dunlap wrote:
> So, the problem appears to be with a ton of brute-force searches to
> remove writable mappings, both during resync and promotion.  My
> analysis tool is reporting that of the 30 seconds or so in the trace
> from xen-unstable, the guest spent a whopping 67% in the hypervisor:
>  * 26% doing resyncs as a result of marking another page out-of-sync
>  * 9% promoting pages
>  * 27% resyncing as a result of cr3 switches
> And almost the entirety of all of those can be attributed to
> brute-force searches to remove writable mappings.

Fantastic (well, sort of)!

If I understand it correctly, Todd is using PV drivers in Linux HVM 
guests, so the reason for brute-force search is due to former L1 
page-tables being used as I/O pages, not being unshadowed because they 
can get writable mappings out of it.
It is, shortly, an unshadowing problem. Should be `easy` to fix. I 
wasn't using PV drivers, so I was not experiencing this behaviour.

Or, it could be a fixup table bug, but I doubt it.

George, did you saw excessive fixup faults in the trace?

Todd, could you try without PV drivers (plain qemu emulation) and see if 
the results get better?

Thanks,
Gianluca