From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40661) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boU0w-0000Eb-2D for qemu-devel@nongnu.org; Mon, 26 Sep 2016 07:24:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1boU0s-0003iY-Fb for qemu-devel@nongnu.org; Mon, 26 Sep 2016 07:24:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40848) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boU0s-0003iI-9A for qemu-devel@nongnu.org; Mon, 26 Sep 2016 07:23:58 -0400 Date: Mon, 26 Sep 2016 12:23:50 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20160926112349.GF2029@work-vm> References: <5feb15.7e53.1576070ae2d.Coremail.lichunguang@hust.edu.cn> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5feb15.7e53.1576070ae2d.Coremail.lichunguang@hust.edu.cn> Subject: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as dirty after they have been sent List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Chunguang Li Cc: qemu-devel@nongnu.org, amit.shah@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, quintela@redhat.com * Chunguang Li (lichunguang@hust.edu.cn) wrote: > Hi all! > I have some confusion about the dirty bitmap during migration. I have digged into the code. I figure out that every now and then during migration, the dirty bitmap will be grabbed from the kernel space through ioctl(KVM_GET_DIRTY_LOG), and then be used to update qemu's dirty bitmap. However I think this mechanism leads to resendness of some NON-dirty pages. > > Take the first iteration of precopy for instance, during which all the pages will be sent. Before that during the migration setup, the ioctl(KVM_GET_DIRTY_LOG) is called once, so the kernel begins to produce the dirty bitmap from this moment. When the pages "that haven't been sent" are written, the kernel space marks them as dirty. However I don't think this is correct, because these pages will be sent during this and the next iterations with the same content (if they are not written again after they are sent). It only makes sense to mark the pages which have already been sent during one iteration as dirty when they are written. > > > Am I right about this consideration? If I am right, is there some advice to improve this? I think you're right that this can happen; to clarify I think the case you're talking about is: Iteration 1 sync bitmap start sending pages page 'n' is modified - but hasn't been sent yet page 'n' gets sent Iteration 2 sync bitmap 'page n is shown as modified' send page 'n' again So you're right that is wasteful; I guess it's more wasteful on big VMs with slow networks where the length of each iteration is large. Fixing it is not easy, because you have to be really careful never to miss a page modification, even if the page is sent about the same time it's dirtied. One way would be to sync the dirty log from the kernel in smaller chunks. Dave > > Thanks, > Chunguang Li -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK