From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: xen/arm: Domain not fully destroyed when using credit2 Date: Tue, 24 Jan 2017 14:19:34 +0100 Message-ID: <1485263974.32103.61.camel@citrix.com> References: <58871C790200007800133268@prv-mh.provo.novell.com> <1485262392.32103.52.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7435911113566887401==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cW10n-0004wB-Dp for xen-devel@lists.xenproject.org; Tue, 24 Jan 2017 13:19:49 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Julien Grall , Jan Beulich Cc: Stefano Stabellini , Wei Liu , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , xen-devel List-Id: xen-devel@lists.xenproject.org --===============7435911113566887401== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-A3Op5Uvti+wgiXNvy5Hn" --=-A3Op5Uvti+wgiXNvy5Hn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2017-01-24 at 13:04 +0000, Julien Grall wrote: > On 24/01/17 12:53, Dario Faggioli wrote: > > Do you have a script and/or some more info for letting me try to > > reproduce it (e.g., you say some otf the vCPUs are pinned, which > > one? > > etc)? >=20 > That was mentioned in my first e-mail :). My configuration is: > - ARM platform with 6 cores > - staging Xen with credit2 enabled by default > - DOM0 using 2 pinned vCPUs > - Guest using 2 vCPUs (not pinned) >=20 Yeah, but some of the details were either missing, or not clear to me... Sorry for bothering and thanks for re-stating this here. :-) How are Dom0 vCPUs pinned, exclusively (i.e., there are 2 pCPUs on which _only_ Dom0 and _no_ DomU can run)? > The script is really simple: >=20 > for i in `seq 1 10`; do > sudo xl create ~/works/guest/guest.cfg; > sudo xl destroy guest; > done >=20 Ok. > > I'm a bit curious about why you're saying this is being exposed by > > using Credit2. >=20 > It is been exposed by Credit2 because compared to Credit1 there is > no=C2=A0 > interrupt traffic made by the scheduler.=20 > So, when you say "no interrupt traffic", do you perhaps mean that SCHEDULE_SOFTIRQ is rarely (never!) raised for idle pCPUs? Or are you really talking about actual interrupts (either inter-processor or not)? > On ARM with credit2 the=C2=A0 > interrupt traffic is reduced to none for idle pCPU. >=20 Yes, but _iff_ we're talking about SCHEDULE_SOFTIRQ events, for a truly idle pCPU (e.g., if I use vcpu-pin to *forbid* every vCPU to execute there), that's _zero_ also for Credit1, at least on x86 (I've just tried)! Perhaps this is too extreme/unrealistic of an idle situation, but I'm trying to understand the problem. :-) > In fact: > >=20 > > =C2=A01) I've power-cycled quite a few domains in these last months, > > while > > =C2=A0=C2=A0=C2=A0=C2=A0under Credit2, and I don't think I have encount= ered it on x86; >=20 > AFAIU, IPI is often the only way to broadcast some instruction on > x86.=C2=A0 > So compare to ARM, you have likely an higher interrupt traffic. >=20 Right. > Also, the problem is not obvious to spot unless you look at the free=C2= =A0 > memory (via xl info) before and after. Another solution is printing > a=C2=A0 > message in both domain_destroy and complete_domain_destroy. >=20 > You will spot the first message directly. The latter may never be > printed. >=20 Yep, I was already instrumenting the code like this... I'll let you know. Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-A3Op5Uvti+wgiXNvy5Hn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYh1RnAAoJEBZCeImluHPuxLYQAOEQAA7fLcrIAkGlYRNv/d7l 8SCvrIM3TXh/a28U14PCaKxEqtErlrF2F08MQeNo9ltNdWkvSJfvz+vcI9JKgBs/ 3zamZ8rjVesD9Yb2dz3m/FtIGRbZWNzuoG00OHJDwZMWmh5B//gXCgsIEv0Z/6Ht FC5kpQQ2Ii2IqMrSaoFuOltYEFP0bqEnmZBowmOsvu9uGt86/u2TsAWSQ2b7CxUv hMS9N9w9Ye17SRkNkocCKM7Dffy/pyXV7t96j9zPr3P8owmCwa1YXca7YeOfLBkl hjy55kbNqi8C0ZB/ZwU3XFYXb0zeIjPQN0pK2Lcf8OoHqExIRuXsF6OtiHFYK2zf LEQEtosVaXr10/244SzEf9by3FdmehDVEMVuVVnMyqHTMrF3MTMybZy92RF13DHa 2hLWbfpixHcELkPgJ9N1HzX/82krDgNlb6YI19Xmt0/pEiOgludOxR5QGyqrsHf1 qcvkYa6+OPpquzmE3Dmk33xj2ZLmEkayGIzuLfN2OsHjHgFrHOSmoaNYZcBrKVfn QSKUbP/4KfJOjPQxtDOHSA7TxKVozQ/BLN0GLTQK6+GdbHVLTcrPJSFPCZwqqYyk FwDh1jEoEdLzNLVBLNFLhFBz4kidahfOdFKsqeI6zr9si+mAKIKSUAzN+OMnP1bL drqGsvh+T6us/ih13F1d =WwFF -----END PGP SIGNATURE----- --=-A3Op5Uvti+wgiXNvy5Hn-- --===============7435911113566887401== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============7435911113566887401==--