From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46496) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj2Qw-0005Fd-As for qemu-devel@nongnu.org; Thu, 24 Mar 2016 06:24:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aj2Qr-0003v4-SI for qemu-devel@nongnu.org; Thu, 24 Mar 2016 06:24:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60604) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj2Qr-0003v0-KY for qemu-devel@nongnu.org; Thu, 24 Mar 2016 06:24:01 -0400 Date: Thu, 24 Mar 2016 10:23:55 +0000 From: "Dr. David Alan Gilbert" Message-ID: <20160324102354.GB2230@work-vm> References: <1458632629-4649-1-git-send-email-liang.z.li@intel.com> <20160322190530.GI2216@work-vm> <20160324012424.GB14956@linux-gk3p> <20160324090004.GA2230@work-vm> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC Design Doc]Speed up live migration by skipping free pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Liang Z" Cc: "rkagan@virtuozzo.com" , "linux-kernel@vger.kenel.org" , "ehabkost@redhat.com" , "kvm@vger.kernel.org" , "mst@redhat.com" , "simhan@hpe.com" , "quintela@redhat.com" , "qemu-devel@nongnu.org" , "jitendra.kolhe@hpe.com" , "mohan_parthasarathy@hpe.com" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , Wei Yang , "rth@twiddle.net" * Li, Liang Z (liang.z.li@intel.com) wrote: > > * Wei Yang (richard.weiyang@huawei.com) wrote: > > > On Wed, Mar 23, 2016 at 06:48:22AM +0000, Li, Liang Z wrote: > > > [...] > > > >> > 8. Pseudo code > > > >> > Dirty page logging should be enabled before getting the free p= age > > > >> > information from guest, this is important because during the > > > >> > process of getting free pages, some free pages may be used and > > > >> > written by the guest, dirty page logging can trace these pages= . > > > >> > The pseudo code=A0is like below: > > > >> > > > > >> > ----------------------------------------------- > > > >> > MigrationState *s =3D migrate_get_current(); > > > >> > ... > > > >> > > > > >> > memory_global_dirty_log_start(); > > > >> > > > > >> > if (get_guest_mem_info(&info)) { > > > >> > while (!get_free_page_bmap(free_page_bitmap, > > > >> > drop_page_cache) > > > >> && > > > >> > s->state !=3D MIGRATION_STATUS_CANCELLING) { > > > >> > usleep(1000) // sleep for 1 ms > > > >> > } > > > >> > > > > >> > tighten_free_page_bmap =3D > > > >> tighten_guest_free_pages(free_page_bitmap); > > > >> > filter_out_guest_free_pages(tighten_free_page_bmap); > > > >> > } > > > >> > > > >> Given the typical speed of networks; it wouldn't do too much har= m > > > >> to start sending assuming all pages are dirty and then when the > > > >> guest finally gets around to finishing the bitmap then update, s= o > > > >> it's asynchronous - and then if the guest never responds we don'= t really > > care. > > > > > > > >Indeed, thanks! > > > > > > > > > > This is interesting. By doing so, the threshold I mentioned in anot= her > > > mail is not necessary, since we can do it in parallel. > >=20 > > Actually I just realised it's a little more complex; we can't sync th= e dirty > > bitmap again from the guest until after we've received the guests 'fr= ee' > > bitmap; that's because we wouldn't know if a 'dirty' page reflected t= hat a > > page declared as 'free' had now been reused - so there is still an or= dering > > there. > >=20 > > Dave >=20 > Not very complex, we can implement like this: >=20 > 1. Set all the bits in the migration_bitmap_rcu->bmap to 1 > 2. Clear all the bits in ram_list. dirty_memory[DIRTY_MEMORY_MIGRATION] > 3. Send the get_free_page_bitmap request > 4. Start to send pages to destination and check if the free_page_bitmap= is ready > if (is_ready) { > filter out the free pages from migration_bitmap_rcu->bmap; > migration_bitmap_sync(); > }=20 > continue until live migration complete.=20 > =20 >=20 > Is that right? The order I'm trying to understand is something like: a) Send the get_free_page_bitmap request b) Start sending pages c) Reach the end of memory [ is_ready is false - guest hasn't made free map yet ] d) normal migration_bitmap_sync() at end of first pass e) Carry on sending dirty pages f) is_ready is true f.1) filter out free pages? f.2) migration_bitmap_sync() It's f.1 I'm worried about. If the guest started generating the free bitmap before (d), then a page marked as 'free' in f.1=20 might have become dirty before (d) and so (f.2) doesn't set the dirty again, and so we can't filter out pages in f.1. Dave >=20 > Liang > >=20 > > > > > > >Liang -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK