From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51533) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c8fIL-00067g-Np for qemu-devel@nongnu.org; Sun, 20 Nov 2016 22:29:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c8fIK-0003dt-DD for qemu-devel@nongnu.org; Sun, 20 Nov 2016 22:29:25 -0500 Date: Mon, 21 Nov 2016 10:58:56 +1100 From: David Gibson Message-ID: <20161120235856.GD18153@umbus.fritz.box> References: <1479433227-29238-1-git-send-email-mdroth@linux.vnet.ibm.com> <20161118054505.GD31640@umbus.fritz.box> <20161118163949.3756.44682@loki> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="76DTJ5CE0DCVQemd" Content-Disposition: inline In-Reply-To: <20161118163949.3756.44682@loki> Subject: Re: [Qemu-devel] [PATCH for-2.8 0/3] spapr: fix breakage of memory unplug after migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Michael Roth Cc: qemu-devel@nongnu.org, qemu-ppc@nongnu.org, duanj@linux.vnet.ibm.com, bharata@linux.vnet.ibm.com, dgilbert@redhat.com, quintela@redhat.com, amit.shah@redhat.com --76DTJ5CE0DCVQemd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 18, 2016 at 10:39:49AM -0600, Michael Roth wrote: > Quoting David Gibson (2016-11-17 23:45:05) > > On Thu, Nov 17, 2016 at 07:40:24PM -0600, Michael Roth wrote: > > > These patches are based on David's ppc-for-2.8 tree, and are also > > > available from: > > >=20 > > > https://github.com/mdroth/qemu/commits/spapr-cas-migration > > >=20 > > > Currently, memory hotplugged to a pseries guest cannot be removed aft= er > > > the guest has been migrated. This is due to 2 issues: > > >=20 > > > 1) The coldplugged state of memory on the target side is one where the > > > corresponding DRC's allocation state is: > > >=20 > > > allocation_state =3D=3D unallocated, > > > awaiting_allocation =3D=3D true, > > >=20 > > > When the guest attempts to unplug memory on the target side, it fi= rst > > > checks that allocation_state =3D=3D allocated. If we fix this, the= guest > > > can successfully notify QEMU of completion on it's end, but then t= he > > > DRC code sees that awaiting_allocation =3D=3D true, so it defers t= he > > > finalizing of the LMB and corresponding DIMM since it assumes that > > > the DIMM must have been previously allocated before it can be remo= ved. > > >=20 > > > To address this, we pull in patches 1-2 from Jian Jun's DRC migrat= ion > > > series: > > >=20 > > > https://lists.gnu.org/archive/html/qemu-ppc/2016-10/msg00048.html > > >=20 > > > with some minor changes relating to prior review comments, and > > > the addition of migrating the DRC's awaiting_allocation value, whi= ch > > > wasn't part of the original patch. This doesn't address the full s= cope > > > of the issues Jian Jun was looking at (involving synchronizing sta= te > > > when migration occurs during fairly small race windows), just this > > > particular case, which is more user visible since the time window = is > > > indefinite. > > >=20 > > > 2) The ability to unplug memory is gated on the QEMU side by a check = as > > > to whether or not support for newer-style hotplug events was negot= iated > > > via CAS during boot. The check is performed by checking the corres= ponding > > > entry in the sPAPROptionVector structure. However, since this valu= e isn't > > > migrated currently, we are unable to unplug until after the guest = reboots. > > >=20 > > > We address that here by adding migration support for sPAPROptionVe= ctors, > > > and including the CAS-negotiated vector as part of the migration s= tream > > > for any cases where we advertise newer-style hotplug event support= to > > > the guest. > > >=20 > > > David, > > >=20 > > > These fixes ended up going out much later than planned. I'm not sure > > > if you're planning another pull for 2.8 or not, and realize there are > > > some patches here not specifically pseries-related so it's > > > understandable if we opt to pursue these for 2.9/2.8.1 instead. But if > > > possible I'm hoping to get these in so that the memory unplug > > > support is fully functional for 2.8. > >=20 > > Yeah, I'm still expecting to push a few bugfixes in before 2.8. So, > > I've merged these patches into ppc-for-2.8 (fixing a couple of trivial > > style nits along the way). I have a couple of comments that I'll make > > on the patches, but they're not important enough to stop these going > > in ASAP. > >=20 > > Unfortunately, of course, this is not the only migration breakage we > > have at the moment. I'm presently wrestling with both breakage due to > > changes in the insns_flags masks, and due to the reworking of the mmio > > windows for the PHB. >=20 > Ok, thanks for the heads up. FYI I'm still hoping to get the insns_flags > fix in for 2.7.1 (which is a bit behind at this point, should have schedu= le > and initial tree posted next week though), so I will keep an eye out for > those. Yeah, in addition to being sick, I've had to rethink how to fix these migration problems, including how to address this for 2.7.1. I'm working on it right now. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --76DTJ5CE0DCVQemd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYMjjAAAoJEGw4ysog2bOSjRAP/3bhqkwKe5A3sS/nscSOxBIG 3wIq2wUhmXmB6mGItmwqD7nqoZYjdcxugLQG8Q50dKzeBE3GE0vRkigL2uHZZrbX GKQ/AWnM1J/BXArFyOEcEis0ds1hvyZVMACa41gnamCyAzI3DRn5X2wZ/gNckSh7 +dPPZz+X3WlNdWFDw7MQU3TAKVQ33NQ0mutGM5YHXec9KK4h6X8Ky6IFv5uWKzGA FSwI5HOE2mmCF7t6fXg8P6uScl4y33NCaKrft1cNnIM6hd1X9+CBR1RV2nbzFkTd XFMsaHxDdKjZLrNdtwzeG+pZhZegkykiRcwGH0IM07vH1H7b4UifCsXv0o4Pji13 Az5CENwWWpLvI3RbgZ7uSjn75TZUBmRsKVSP5p2ohQFUJGTq0GzjbukEaCEUFhwb a27EtPQazlZLfVkzjKWXfTwgok5bTuCXU74wKhp/lFXPZFPiA2D9tdDDSTo+VKxJ Zag/EKoMD6Lw+9AUPGbkte2JJH17DDcbbpXwt0o4HvDvVn/onwWxAfO5naG/SkI+ oOTid8j4saNd0A83cWuhlwZCzk63NwnSq6A9BFAwbjFyuUPEGfyO48Ek1kH6GObN FzuLONjOt4DxLHTHY7iBwgeVP903Bwu7cAGTCPsfrlTZiuKIOOV79Jv+lhwaXPJu cMO+TtBv6Nks7cMd8JRA =NLt2 -----END PGP SIGNATURE----- --76DTJ5CE0DCVQemd--