From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave McCracken Subject: RFC: Superpage/hugepage performance improvement Date: Mon, 5 Apr 2010 12:52:29 -0500 Message-ID: <201004051252.29210.dcm@mccr.org> Mime-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser , Jeremy Fitzhardinge Cc: Xen Developers List List-Id: xen-devel@lists.xenproject.org In our testing we found that the superpage/hugepage mapping code is seriously bogged down by the need to maintain the reference count on each of the underlying pages every time a hugepage is mapped. I came up with a fix where a guest can call into the hypervisor to mark a set of pages as a superpage, thus locking that set of pages to be read/write data pages until the corresponding unmark is call is made. To make this work I added two mmuext ops, one to mark a superpage and one to unmark it. This change makes a huge performance difference in the hugepage mapping (on the order of 50 times faster). On the Linux side, the hugepages are marked at the time they are put into the hugepage pool, and unmarked when they are taken out of the pool. This commonly happens very infrequently. Does this mechanism sound reasonable to you all? If so, I'd like to make sure the numbers we use for the new mmuext ops are reserved upstream so we won't have to change them in the future. I will port the actual patch forward to mainline shortly and send it off, but I wanted to get an early indication of how you feel about the design. Thanks, Dave McCracken Oracle Corp.