From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: xen/arm: Domain not fully destroyed when using credit2 Date: Wed, 25 Jan 2017 17:00:09 +0100 Message-ID: <1485360009.32103.125.camel@citrix.com> References: <58871C790200007800133268@prv-mh.provo.novell.com> <1485262392.32103.52.camel@citrix.com> <1485263974.32103.61.camel@citrix.com> <1485265231.32103.65.camel@citrix.com> <1485267419.32103.71.camel@citrix.com> <1485342610.32103.96.camel@citrix.com> <0c19de52-e7f5-b97a-ec31-41b4284add32@arm.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0615874134335626592==" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cWQ06-0007aI-Tg for xen-devel@lists.xenproject.org; Wed, 25 Jan 2017 16:00:47 +0000 In-Reply-To: <0c19de52-e7f5-b97a-ec31-41b4284add32@arm.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Julien Grall , Jan Beulich Cc: Stefano Stabellini , Wei Liu , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , xen-devel List-Id: xen-devel@lists.xenproject.org --===============0615874134335626592== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-eZE4B5Hqbmn7AZCepN24" --=-eZE4B5Hqbmn7AZCepN24 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2017-01-25 at 12:38 +0000, Julien Grall wrote: > Hi Dario, >=20 Hey, > On 25/01/17 11:10, Dario Faggioli wrote: > > My point was that, still from scheduling perspective, neither > > Credit1 > > nor Credit2 sets a wakeup timer for idle pCPUs. > >=20 > > Well, in Credit1, the master_ticker timer is never stopped (while, > > e.g., the per-pCPU tick is stopped before entering deep sleep, > > via sched_tick_suspend(), see commit 964fae8ac), but that's only 1 > > pCPU. >=20 > The function sched_tick_suspend is never called on ARM. The power > saving=C2=A0 > in Xen ARM is still very limited and this would need to be updated > in=C2=A0 > the future. >=20 > So I guess that's why I still see interrupt coming on the idle pCPU > when=C2=A0 > credit1 is used.=20 > Yes. If you don't suspend the tick before going to wfi/hlt/whatever, there will be a timer firing --and AFAICT waking you up from the low power state-- every 10ms (with default Credit1 timeslice), even for idle pCPUs. > Looking at credit2, the callback tick_suspend is not=C2=A0 > called. Does it mean there is no per-pCPU timer? >=20 Exactly, we (happily) don't need that in Credit2. :-) > Now, from my understanding, if we decide to call sched_tick_suspend > on=C2=A0 > ARM before idling. We will likely have the same problem with credit1=C2= =A0 > because there is no more interrupt to wake-up the pCPU. >=20 Basing on what you've said so far in this thread, I tend to think that, yes, that would be the case. > But I don't think this is an issue in the scheduler.=20 > Agreed. > IHMO, the problem=C2=A0 > is in the RCU. Indeed a CPU in lower power mode (i.e=C2=A0=C2=A0wfi on AR= M or=C2=A0 > pm_idle on x86 is been executed) will never get out to tell to the > RCU :=C2=A0 > "I am quiet, go ahead". So the RCU will never be able to reclaim the=C2= =A0 > memory and will result on a memory exhaustion if the pCPU never > receive=C2=A0 > an interrupt (this could happen if pCPU has never ran a guest). >=20 > The question now, is how to fix it? >=20 And a good one. I may be wrong (I certainly wasn't around at the time), but ISTR out RCU code is imported/inspired by Linux... Looking there again may help, but, nowadays, Linux RCU subsystem is a=C2=A0Lernaean Hydra monster, with 100 heads and sharpen claws! :-O And, while, in there, it has to be like that, I don't think we need all such complexity, and hence we can't just re-sync. :-/ Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-eZE4B5Hqbmn7AZCepN24 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYiMuJAAoJEBZCeImluHPu2roQAMyxBF9oiV+Ko5YAEYp4SKrX J1le0qQG7VYdbapVCRgAcYHhoFgM4tBAb530rqBX6cmixbdPgXI+Zs37SNQylhso aPeDTLepMu4x4mJXUQ/5VqYa635QJM0ixHvKI5NbJZnFoWoilG0eKOj0YbFGGnay igUHRH7BWYl/nGIvV6NN0rHYNOebIZiMXn0dFoxBV7wVg3HydgOUSdSHJ/L97hZl IcuXZ7wWDIU7LirzREQpWVsyyMEZxXHiLGDJT2yyqFnKJZaU9IFRHLkzHMQjdt+r quKLdbeh8nmaWXSfR3C6C0UUme/bfrQjB1Uej4MXyJ3TSNm08Q9ql1BO2Hcgn4qW dmjwDBqIF8ubWNKL8mQ2kQohUfMAdY/QaerkrAYjBSP/uo9uyZYMYuMyMLYtrb+U 5cw8I9L+Zd50rDWF2Rxev54N8Xupz7Qxu6AtrCi6wnJGA6keM3oUP+5uE6rDneL2 1LYwgH5sYXcVfmkE1SWMSXRqYA9fVn0nfHnBEwcfFy9SOmbjT+3BdIgJOXT/KyVo 2M4HSjebMcPxS0lP2mrZ2PhjmyU/KxXGT5t1WQBVxIjqCq4YLNyHjTUMQw0MqU0F MgpCUFAt01vYf5TMDqRKTMqxsiIP5268iEmQcsCCMHkJJ6vyrp1u652EnRJ7QaR5 uolxI/lKCrrprGcIWR6m =fPjS -----END PGP SIGNATURE----- --=-eZE4B5Hqbmn7AZCepN24-- --===============0615874134335626592== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============0615874134335626592==--