From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 0/3] VMX: Properly handle pi descriptor and per-cpu blocking list Date: Mon, 23 May 2016 14:39:12 +0200 Message-ID: <1464007152.21930.55.camel@citrix.com> References: <1463734431-22353-1-git-send-email-feng.wu@intel.com> <573F02B102000078000ED304@prv-mh.provo.novell.com> <5742D6BB02000078000EDA57@prv-mh.provo.novell.com> <5742E0BE02000078000EDABD@prv-mh.provo.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8091080227723159364==" Return-path: In-Reply-To: <5742E0BE02000078000EDABD@prv-mh.provo.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Jan Beulich , Feng Wu Cc: Kevin Tian , "keir@xen.org" , "george.dunlap@eu.citrix.com" , "andrew.cooper3@citrix.com" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org --===============8091080227723159364== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-h2y1Jt5TAt9DF+XQhAky" --=-h2y1Jt5TAt9DF+XQhAky Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2016-05-23 at 02:51 -0600, Jan Beulich wrote: > > > > On 23.05.16 at 10:44, wrote: > > >=C2=A0 > > > vCPU-s currently having their v->processor set to the pCPU being > > > hot removed would simply get migrated elsewhere. If that's not > > > accompanied by respective PI blocking list adjustments, then I > > > can > > > only refer you back to the original series' review process, where > > > it was (iirc) more than once that it was pointed out that getting > > > out > > > of sync v->processor and the PI list a vCPU sits on may casue > > > issues. And if there is an issue here, I'm not sure what a good > > > solution would be. > > Okay, here is my understanding about the blocking and vCPU > > migration things, if vCPU is blocking and its v->processor is pCPU > > 0, > > at some point, pCPU0 is removed from the system, it will also > > migrate the vCPU to another pCPU, say, pCPU1 as the response > > to pCPU0 being unplugged, hence, v->processor =3D pCPU1, right? > > Yes. >=20 See, for instance, cpu_disable_scheduler() in schedule.c. What we do is go over all the vcpus of all domains of either the system or the cpupool, and force the ones that we found with v->processor set to the pCPU that is going down, to perform migration (system_state will be different than SYS_STATE_suspend, and we hence take the 'else' branch). Considering that the pCPU is no longer part of the relevant bitmask-s during the migration, the vCPUs will figure out that they just can't stay there, and move somewhere else. Note, however, that this happens for running and runnable vCPUs. If a vCPU is blocker, there is nothing to do, and in fact, nothing happens (as vcpu_sleep_nosync() and vcpu_wake() are NOP in that case). For those vCPUs, as soon as they wake up, they'll figure out that their own v->processor is not there any longer, and will move somewhere else. > > If that is the case, I think the current code may have issues in > > the > > above case, since 'NDST' filed in PI descriptor is still vCPU0 even > > it is removed from the system. I need to consider how to handle > > this case.=20 > Exactly. I don't think you can do that, for instance, from within cpu_disable_scheduler(), or anythign that gets called from there, as it basically deals with running vCPUs, while you're interested in blocked ones. I wonder whether you can register your own notifier, and, inside it, scan the blocked list and fixup things... (just the first idea that came up in my mind) > > But this is not an issue=C2=A0=C2=A0in non pCPU hotplug scenario. > >=20 It's probably an issue even if you remove a cpu from a cpupool (and even a more "interesting" one, if you also manage to add it to another pool, in the meantime) isn't it? Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-h2y1Jt5TAt9DF+XQhAky Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJXQvnwAAoJEBZCeImluHPu7oQQAKs0q/MLgNBda+uFvXOJr+Fv +Wn+Q5v6Wo0+avrlzRp/WkMYFbl0kt1NXozTz8OTX9oSpFgBIklJ1hYv3qcp9C/x RDgV+65CJ8fFg1GJoDZVu5etUQ/fBy+aPhFZVItLdA53RENptBV46LPaO0XjQR+i VLazuX3+awaI1zkZB7DiVoBX9YeXmgKJnLUIdjjcHcd6jm06KHAKybFRSmy9F2/C JhNZEaKthuYtnNDsqR7Kpnvb8WCNjWLtSX48Jj6PSd0PSoDxHjzHeA+m/hIsLx57 pslBorSwnbn1MZCndUkJsGmnZUi/655VHFlTTAGe/pfNRwxykhiUV2R6xsLGrDrs wq+K0nCnxe1hamj5Dhp9odTr3DlwMgGyq+DydG2oucSWhxePR4t6Z1RpVKPFRUfH HGWzvyS/hGGYXvdSawnyxvGNrWGp88MtESrZSLIFxfzfky2quGlPG9CLyNBjtLHy aqSK931OSeuRqDyxDQQ92/D/Vpgu+hXW3xjcign2KuREzyh0fkWSHP0QBiNvzA/k haH7TQ+c3sr0iFWIzXtngTF50X1xaGvxaCQVrXdGxwzkR9QUZM2JgfixxOoeQjZa gdAzlkeJmu1XWJ1DAfdIWHPBJiaOXM+0zrdCc6XDqPmeuZN3QiSDMqVsqIcClOEX fTatiYpqEjL+vxLm0krh =Ur53 -----END PGP SIGNATURE----- --=-h2y1Jt5TAt9DF+XQhAky-- --===============8091080227723159364== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9y Zy94ZW4tZGV2ZWwK --===============8091080227723159364==--