From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH] xen/arm: introduce vwfi parameter Date: Tue, 21 Feb 2017 17:51:39 +0100 Message-ID: <1487695899.6732.420.camel@citrix.com> References: <1487286292-29502-1-git-send-email-sstabellini@kernel.org> <484f2a39-1322-f40e-de01-97746e5de280@arm.com> <1487618436.6732.230.camel@citrix.com> <1845466e-c6a3-bcb9-0813-80ecdf267f03@arm.com> <1487631215.6732.266.camel@citrix.com> <06f20feb-0e9e-c757-b245-ebb55b422c67@arm.com> <1487668158.6732.310.camel@citrix.com> <14575011-0042-8940-c19f-2482136ff91c@foss.arm.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0395339420349740502==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cgDfb-0006GH-7D for xen-devel@lists.xenproject.org; Tue, 21 Feb 2017 16:52:07 +0000 In-Reply-To: <14575011-0042-8940-c19f-2482136ff91c@foss.arm.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Julien Grall , Stefano Stabellini Cc: edgar.iglesias@xilinx.com, george.dunlap@eu.citrix.com, nd@arm.com, Punit Agrawal , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org --===============0395339420349740502== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-JPA0Q1FzRTuwE6FmYprQ" --=-JPA0Q1FzRTuwE6FmYprQ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2017-02-21 at 12:30 +0000, Julien Grall wrote: > On 21/02/2017 09:09, Dario Faggioli wrote: > > Well, TBH, we still are not entirely sure who the culprit is for > > high > > latency. There are spikes in Credit2, and I'm investigating that. > > But > > apart from them? I think we need other numbers with which we can > > compare the numbers that Stefano has collected. >=20 > I think the problem is because we save/restore the vCPU state when=C2=A0 > switching to the idle vCPU. >=20 That may well be. Or at least, that may well be part of the problem. I don't know enough of ARM to know whether it's the predominant cause of high latencies or not. On x86, on Linux, polling is used to prevent the CPU to go in deep C-states. I was assuming things to be similar on ARM, and that's the reason why I thought introducing a polling mode could have been useful (although wasteful). But you guys are the ones that know whether or not that is the case (and in another email, you seem to say it's not). > Let say the only 1 vCPU can run on the pCPU, when the vCPU is issuing > a=C2=A0 > WFI the following steps will happen: > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* WFI trapped and vcpu blocked > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* save vCPU state > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* run idle_loop > -> Interrupt incoming for the guest > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* restore vCPU state > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* back to the guest >=20 > Saving/restoring on ARM requires to context switch all the state of > the=C2=A0 > VM (this is not saved in memory when entering in the hypervisor). > This=C2=A0 > include things like system register, interrupt controller state, > FPU... >=20 Yes. In fact, on x86, we have what we call 'lazy context switch', which deals specifically with some aspect of this situation. Indeed it seems like implementing the same on ARM --if you don't have it already, and if possible-- would be useful in this case too. Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-JPA0Q1FzRTuwE6FmYprQ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYrHAdAAoJEBZCeImluHPuAhoP/RY6d5pcET0VfOZLRQqopYiL IPlyrc+KB7eHQsvH54TqNUsOaE+1YsBCSMWEySvZiJAwbTFP8KSwN3lVhBJa2Q+B ouUJYFklAoKP7AHx/6uj1/5I/Ws/1JAYnuZwoMCtYOknOd3JkUYB85JaDRboHKv+ UryhDkYnnyQDWmJEi78qCSKPIDolI/9W1Ne/JC/IZEHmtpsVx+MW9Qm2iU7+GIaI Tv1/S9hvsHeeSvuU1mcqJcKUC0YP8t6HyXLf77kvhgKCnXy1etOb0l/5x1f8fuRN jpz69GwZK3tqG3qFdmhaYaqAOqTxvIeFbDJjQ8KmsYEnZoC3ccQlGB1KB8o++dYP CcAhVZGdysmDp5oNl8jQS1aWMBEypuX+nxjQ3cqmlYARyxM4iOnWX47+nsFJVg4j PQO5J0sX5kVzqg6F2bUwVaLm5VdH/5MOvqi/v59zMLmdlEbDqL0gW4zHlGceFDeg iDAiLz9VAyq9bebowWc+X1K6IaSITrYsDuZMZzABXn82G7Hpv+hjR8oxoliEJQMF bnXgDDyjdrciyNOmtG8VvN+zyEYknNZRnZZVL2CpZp1IyND4X2z6QvnFt1rXfcv4 JZM2/nyJaMEiVa31yr2xs4I9CQKIzDf4kNpn97Xpk6iBls3KRPdwMZB4cbOi7jzU uAhIi8ixQIMyGvVq7EVL =igPN -----END PGP SIGNATURE----- --=-JPA0Q1FzRTuwE6FmYprQ-- --===============0395339420349740502== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============0395339420349740502==--