From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: slow live magration / xc_restore on xen4 pvops Date: Thu, 3 Jun 2010 08:12:17 +0100 Message-ID: References: <20100603065542.GC52378@zanzibar.kublai.com> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20100603065542.GC52378@zanzibar.kublai.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Brendan Cully , "andreas.olsowski@uni.leuphana.de" , "xen-devel@lists.xensource.com" , Ian List-Id: xen-devel@lists.xenproject.org On 03/06/2010 07:55, "Brendan Cully" wrote: >> kernel, min call time, max call time >> 2.6.18, 4 us, 72 us >> pvops, 202 us, 10696 us (!) >> >> It looks like pvops is dramatically slower to perform the >> xc_domain_memory_populate_physmap call! > > Looking at changeset 20841: > > Allow certain performance-critical hypercall wrappers to register data > buffers via a new interface which allows them to be 'bounced' into a > pre-mlock'ed page-sized per-thread data area. This saves the cost of > mlock/munlock on every such hypercall, which can be very expensive on > modern kernels. > > ...maybe the lock_pages call in xc_memory_op (called from > xc_domain_memory_populate_physmap) has gotten very expensive? > Especially considering this hypercall is now issued once per page. Maybe there are two issues here then. I mean, there's slow, and there's 10ms for a presumably in-core kernel operation, which is rather mad. Getting our batching back for 4k allocations is the most critical thing though, of course. -- Keir