From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: [PATCH] Add hypercall to mark superpages to improve performance Date: Fri, 30 Apr 2010 15:03:22 -0700 Message-ID: References: <201004301643.01580.dcm@mccr.org> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <201004301643.01580.dcm@mccr.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dave McCracken Cc: Jeremy Fitzhardinge , Xen Developers List List-Id: xen-devel@lists.xenproject.org On 30/04/2010 14:43, "Dave McCracken" wrote: >> That could be implemented with no extra hypercalls, and I reckon it's >> probably easier to make this race-free too. Obviously it does have extra >> code complexity to construct this array (which I suppose needs to be >> sparse, just like page_info array, in the face of very sparse memory >> maps). The space overhead (about 8 bytes per 2MB, or 0.0004% of total >> system memory) would be trivial. Compared with an extra reference count in >> every page_info, which would have a much higher 0.2% overhead. > > I like this idea. I'll look into it. The algorithm for acquiring a superpage refcount would be something like: y = superpage_info->count do { x = y if ( x == 0 ) for (each page in super_page) if (!get_page(page)) goto undo_and_fail; } while ((y = cmpxchg(&superpage_info->count, x, x+1)) != x); For destroying a superpage refcount: y = superpage_info->count do { x = y } while ((y = cmpxchg(..,x,x-1)) != x); if (x==1) for (each page in super_page) put_page(page) I'd actually have two refcounts in superpage struct: one for read-only mappings and one for read-write mappings. The latter would be updated as above except for the use of {get,put}_page_and_type() instead of {get,put}_page(). The other thing to note is that this approach is cheap when updating superpage refcounts between non-zero values. If regularly constructing/destroying the *only* superpage mapping of a page, then obviously you are going to be continually taking the slow path. In that case, pinning a superpage with new hypercalls as you suggested may have to be done. It kind of depends on how your workloads interact with the reference-counting. In any case, you could implement the basic version as described here, and add hypercalls as a second stage if they turn out to be needed. I can help with further details and advice if need be. -- Keir