From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53301) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XGtGN-0003Z1-Q8 for qemu-devel@nongnu.org; Mon, 11 Aug 2014 13:20:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XGtGH-00078y-HX for qemu-devel@nongnu.org; Mon, 11 Aug 2014 13:20:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56994) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XGtGH-00078u-99 for qemu-devel@nongnu.org; Mon, 11 Aug 2014 13:19:57 -0400 Message-ID: <53E8FB38.1060903@redhat.com> Date: Mon, 11 Aug 2014 11:19:52 -0600 From: Eric Blake MIME-Version: 1.0 References: <1407767399-3030-1-git-send-email-dgilbert@redhat.com> <1407767399-3030-44-git-send-email-dgilbert@redhat.com> In-Reply-To: <1407767399-3030-44-git-send-email-dgilbert@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="EdA0CB3tjHquifgEixf4O7InKTqqWvSkG" Subject: Re: [Qemu-devel] [PATCH v2 43/43] Start documenting how postcopy works. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert (git)" , qemu-devel@nongnu.org Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, lilei@linux.vnet.ibm.com, quintela@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --EdA0CB3tjHquifgEixf4O7InKTqqWvSkG Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 08/11/2014 08:29 AM, Dr. David Alan Gilbert (git) wrote: > From: "Dr. David Alan Gilbert" >=20 > Signed-off-by: Dr. David Alan Gilbert > --- > docs/migration.txt | 150 +++++++++++++++++++++++++++++++++++++++++++++= ++++++++ > 1 file changed, 150 insertions(+) I personally like docs early in the series (see if they make sense in isolation, then whether the implementation matched the docs; rather than being tainted by implementation and overlooking design flaws). >=20 > diff --git a/docs/migration.txt b/docs/migration.txt > index 0492a45..fec2d46 100644 > --- a/docs/migration.txt > +++ b/docs/migration.txt > @@ -294,3 +294,153 @@ save/send this state when we are in the middle of= a pio operation > (that is what ide_drive_pio_state_needed() checks). If DRQ_STAT is > not enabled, the values on that fields are garbage and don't need to > be sent. > + > +=3D Return path =3D > + > +In most migration scenarios there is only a single data path that runs= > +from the source VM to the destination, typically along a single fd (al= though > +possibly with another fd or similar for some fast way of throwing page= s across). > + > +However, some uses need two way comms; in particular the Postcopy dest= ination s/comms/communication/ > +needs to be able to request pages on demand from the source. > + > +For these scenarios there is a 'return path' from the destination to t= he source; > +qemu_file_get_return_path(QEMUFile* fwdpath) gives the QEMUFile* for t= he return > +path. > + > + Source side > + Forward path - written by migration thread > + Return path - opened by main thread, read by fd_handler on main = thread > + > + Destination side > + Forward path - read by main thread > + Return path - opened by main thread, written by main thread AND = postcopy > + thread (protected by rp_mutex) > + > +Opening the return path generally sets the fd to be non-blocking so th= at a > +failed destination can't block the source; and since the non-blockingn= ess seems > +to follow both directions it does alter the semantics of the forward p= ath. Once you've opened a return path, haven't you already committed to the migration? That is, since we already document that postcopy loses the VM if something goes wrong, what point is there in trying to protect from a failed destination? Once the source knows that the destination has opened the return path, the source should never run the VM again. > + > +=3D Postcopy =3D > +'Postcopy' migration is a way to deal with migrations that refuse to c= onverge; > +it's plus side is that there is an upper bound on the amount of migrat= ion traffic s/it's/its/ > +and time it takes, the down side is that during the postcopy phase, a = failure of > +*either* side or the network connection causes the guest to be lost. > + =2E.. > +=3D=3D=3D Postcopy states =3D=3D=3D > +Postcopy moves through a series of states (see postcopy_ram_state) > +from ADVISE->LISTEN->RUNNING->END > + > + Advise: Set at the start of migration if postcopy is enabled, even > + if it hasn't passed the start-time threshold; here the desti= nation > + checks it's OS has the support needed for postcopy, and perf= orms s/it's/that its/ > + setup to ensure the RAM mappings are suitable for later post= copy. > + (Triggered by reception of POSTCOPY_RAM_ADVISE command) > + > +Normal precopy now carries on as normal, until the point that the sour= ce > +hits the start-time threshold and transitions to postcopy. The source= > +stops it's CPUs and transmits a 'discard bitmap' indicating pages that= s/it's/its/ (now that you've made the same mistake multiple times: remember, "it's" is correct ONLY if you can say "it is" (or "it has") in the same context; possession uses "its"). > +have been previously sent but are now dirty again and hence are out of= > +date on the destination. > + > +The migration stream now contains a 'package' containing it's own chun= k s/it's/its/ > +of migration stream, followed by a return to a normal stream containin= g > +page data. The package (sent as CMD_PACKAGED) contains the commands t= o > +cycle the states on the destination, followed by all of the device > +state excluding RAM. This lets the destination request pages from the= > +source in parallel with loading device state, this is required since > +some devices (virtio) access guest memory during device initialisation= =2E I don't know if we have a strong preference for US (initialization) or UK (initialisation) spelling in our docs. =2E.. > + > +Note that once in postcopy mode, the sent map is still updated, howeve= r it's s/it's/its/ > +contents are not-consistent as a local view of what's been sent since = it's This use of "it's" is correct, but sounds awkward. > +only got the masked result. Maybe: Note that once in postcopy mode, the sent map is still updated; however, its contents are not necessarily consistent with the pages already sent due to the masking with the migration bitmap. --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --EdA0CB3tjHquifgEixf4O7InKTqqWvSkG Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg iQEcBAEBCAAGBQJT6Ps4AAoJEKeha0olJ0Nqb6QIAKKdQ/B3x8TcxyyK2+ZhYSU3 LQj+9NmlDExTLim92cHlPIjhNTA/evzHsDOvq26mIafYrIXJImmsE5doayeC7Q6R XA5LYh2qEatBXY01dIDNOtQIQddwuRMUrZI80V6fmLmR4HJj9xDAWMQpjo8U6XXC uv51WLVr1DUni676/fjkqFuotO0Hhlqnnp9/qgCztcm9vls5C7Mbc4cDSn+WjZeL UzLr+TjCfRgY+Hcwpw1Xnz1l704BsvghaBne/7Wy4isy5DiUjvOmv9HF2X40hzSD lgfqGqTN5oipSxVod9+pcwMtpYT3epFSg5jPyvoY/dJNbUve4fiOXAavmUa3WNg= =Nx9+ -----END PGP SIGNATURE----- --EdA0CB3tjHquifgEixf4O7InKTqqWvSkG--