From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34239) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WKnMK-00018p-D5 for qemu-devel@nongnu.org; Tue, 04 Mar 2014 06:18:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WKnME-0006LI-EW for qemu-devel@nongnu.org; Tue, 04 Mar 2014 06:18:04 -0500 Received: from mx1.redhat.com ([209.132.183.28]:17067) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WKnME-0006L9-4Y for qemu-devel@nongnu.org; Tue, 04 Mar 2014 06:17:58 -0500 Message-ID: <5315B65F.8070400@redhat.com> Date: Tue, 04 Mar 2014 12:17:51 +0100 From: Paolo Bonzini MIME-Version: 1.0 References: <33183CC9F5247A488A2544077AF19020815D4452@SZXEMA503-MBS.china.huawei.com> In-Reply-To: <33183CC9F5247A488A2544077AF19020815D4452@SZXEMA503-MBS.china.huawei.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Do memory mappings need be rebuilt when deleting ioeventfds? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Gonglei (Arei)" , "qemu-devel@nongnu.org" Cc: "Herongguang (Stephen)" , Juan Quintela Il 04/03/2014 11:49, Gonglei (Arei) ha scritto: > Hi, > Recently I found that when doing migration on a VM with many Virtio NICs, > a lot down time was consuming in vm_state_notify(). Further investigation > shows major consumption is in function memory_region_del_eventfd(). When deletes an > ioeventfd, in address space transactions commit, it begins with deleting > all memory mappings in all address spaces, and add each memory mapping again. > This is time consuming. > > In my test, each ioeventfd deleing needs about 5ms, within which memory > mapping rebuilding needs about 4ms. With many Nics and vmchannel in a VM > doing migrating, there can be many ioeventfds deleting which increasing > downtime remarkably. > > As far as I can see, memory mappings don't rely on ioeventfds, there is > no need to destroy and rebuild them when manipulating ioeventfds. I think > this is for decoupling consideration. Although this simplifies coding and > design, it scarifies performance. > > So I'd want to know if you have any improve plan on it? Here's my > assumption: in memory_region_del_eventfd() and memory_region_add_eventfd(), > we mark that this is a pure ioeventfd operation, so in begin and region_nop > memory listeners, we eliminate memory mapping destroy and rebuild, thus > decrease migration down time and improve migration performance. Precisely, what you suggest won't work because you can have memory_region_add/del_eventfd as a part of a larger transaction. But overall this is a good idea. You can split memory_region_update_pending in two parts, like memory_region_update_pending and memory_region_ioeventfd_change_pending. Then, if memory_region_update_pending is false but memory_region_ioeventfd_change_pending is true, you can activate an optimized path. Paolo