From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: xen/arm: Domain not fully destroyed when using credit2 Date: Tue, 24 Jan 2017 13:53:12 +0100 Message-ID: <1485262392.32103.52.camel@citrix.com> References: <58871C790200007800133268@prv-mh.provo.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6845652938076644791==" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cW0bC-0002xF-Km for xen-devel@lists.xenproject.org; Tue, 24 Jan 2017 12:53:22 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Julien Grall , Jan Beulich Cc: Stefano Stabellini , Wei Liu , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , xen-devel List-Id: xen-devel@lists.xenproject.org --===============6845652938076644791== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-pdExOgG5St0MutVUk96i" --=-pdExOgG5St0MutVUk96i Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2017-01-24 at 10:50 +0000, Julien Grall wrote: > On 24/01/2017 08:20, Jan Beulich wrote: > > > > > On 23.01.17 at 20:42, wrote: > > > The function domain_destroy will setup the RCU callback > > > (complete_domain_destroy) by calling call_rcu. call_rcu will add > > > the > > > callback into the RCU list and then will may send an IPI (see > > > force_quiescent_state) if the threshold reached. This IPI is here > > > to > > > make sure all CPUs are quiescent before calling the callbacks > > > (e.g > > > complete_domain_destroy). In my case, the threshold has not > > > reached and > > > therefore an IPI is not sent. > >=20 > > But wait - isn't it the nature of RCU that it may take arbitrary > > time > > until the actual call(s) happen(s)? >=20 > Today this arbitrary time could be infinite if an idle pCPU does not=C2= =A0 > receive an interrupt. So some part of domain resource will never be > freed. >=20 > If I am power-cycling a domain in loop, after some time the > toolstack=C2=A0 > will fail to allocate memory because of exhausted resources. > Previous=C2=A0 > instance of the domain was not yet fully destroyed (e.g=C2=A0 > complete_domain_destroy was not called). >=20 Do you have a script and/or some more info for letting me try to reproduce it (e.g., you say some otf the vCPUs are pinned, which one? etc)? I'm a bit curious about why you're saying this is being exposed by using Credit2. In fact: =C2=A01) I've power-cycled quite a few domains in these last months, while= =C2=A0 =C2=A0 =C2=A0 under Credit2, and I don't think I have encountered it on x86= ; =C2=A02) I see how it may be related to Credit2 being more deterministic=C2= =A0 =C2=A0 =C2=A0 and not trying to schedule stuff around pseudo-randomly like= =C2=A0 =C2=A0 =C2=A0 Credit1 does... but I'd like to try investigating a bit more. Thanks and Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-pdExOgG5St0MutVUk96i Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYh044AAoJEBZCeImluHPu/5EQANOymFCg02bTLjxycXeEJntM Do0nDAF/YLS4aEe4cT7pMMCHsZa1cD7hfMST9mNLi8Y77VbGhbJxXRyD4fCkB5z8 +zY6sKtz2sTkmx59nfE2hgHI8g0DNt/sgCzZvpeFE4THmlkdtXVrt6SsafLRetkV jlcIczvLRaBVR5mRk8qlM59xWmVL1QbXlCHh3GMDOvYaw+lmp8wyjVWxhDubU38c fu+m5gg1FSjRWmjF9NBOI13Xl0bjwmaiYViZlh1Sdlp5tKetAadutDuLRQcWgeFB U/clTD0PR8KWG6TdGwjQkrDDkXmQDj7QNSuaNfrQ0PtXs01rJbr2XIgrhGoqyulD qgcRGl6gzJycgB/1EnRdS/6zr8J1VLtM4k3wWtbtq1hX02NGL8pP8IrKR7V2Sdzc QaoPNYr3tjruD1SIxe3E+WihwpwKCtF98C74y7Vd0OzsaA1avphrseJwLzW3Z1+X O3VlS5bWGX9NGzLGS/ogKE4rkxTKBQoOTnOxw77ksQdYMpkYtP7slOMvFCFOOjc/ 6tS1pNPrUjm3Am50S9PSYnHAixoMufi2aJViqxwvSlPJj75oLvELWYN2cV8mCZSC Xnu2bf61r2ebB0RuCRWKpk84ctC9VkP/vUZqQfYAQJ/Nvl/sp77f6Z3Bwv2ZFNJD GL9v1ASOyjAMmz7bG5Ei =VM75 -----END PGP SIGNATURE----- --=-pdExOgG5St0MutVUk96i-- --===============6845652938076644791== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============6845652938076644791==--