From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: arm64: Approach for DT based NUMA and issues Date: Mon, 28 Nov 2016 13:30:12 +0100 Message-ID: <1480336212.3178.7.camel@citrix.com> References: <1480208512.2712.241.camel@citrix.com> <6dd921fa-eef2-08ba-565e-e0dfa1f0ab4e@arm.com> <1480279892.2712.261.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0470577232228648314==" Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cBL52-00060a-UR for xen-devel@lists.xenproject.org; Mon, 28 Nov 2016 12:30:45 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Vijay Kilari Cc: Andre Przywara , Julien Grall , Stefano Stabellini , "prasun.kapoor" , xen-devel List-Id: xen-devel@lists.xenproject.org --===============0470577232228648314== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-apN6RsfuKirtbyg2mZWX" --=-apN6RsfuKirtbyg2mZWX Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2016-11-28 at 16:32 +0530, Vijay Kilari wrote: > On Mon, Nov 28, 2016 at 2:21 AM, Dario Faggioli > wrote: > >=C2=A0 > > That makes perfect sense to me, and FWIW, is also what I'd do. In > > fact, > > the whole point of what I was saying was not to confuse Xen NUMA > > support and Dom0 NUMA support; if we want to do both of them, the > > latter right after the former, fine, but they're separate things > > indeed. > =C2=A0 Yes, agreed. Whatever the existing Xen NUMA-Aware code is > completely kept > under x86, which can be used for arm as well. So needs cleanup and > make common > for both archs. >=20 Sure. > =C2=A0 Regarding Dom0 NUMA-aware, in arm Dom0 is completely not NUMA- > aware, not even > to the extent supported in x86. >=20 Well, Dom0 is ~0% NUMA aware on x86. But it's not important whose Dom0 is _less_ NUMA aware. What I (and also Julien, AFAICT) am talking about is that we should start make Xen NUMA aware for ARM, before looking at Dom0. > > This means that, by default, Dom0 memory is indeed spread among > > various > > existing nodes. Eg., on my NUMA test box here at home, here's how > > things are for Dom0: >=20 > This default behaviour of spreading memory across existing nodes is > better to some > extent compared to ARM.. On ARM, All the allocation is based on > allocator. > All it assumes all the memory is on single node. >=20 Again, I don't know much about ARM, but my point is this: look at the differences between xen/include/asm-arm/numa.h and xen/include/asm-x86/numa.h. E.g., from the ARM one: #define cpu_to_node(cpu) 0 This is what I'm saying we should deal with first. > > dom0_nodes=3Dx is a way to tell Xen to (try as hard as it can) to > > only > > allocate the memory for dom0 only from NUMA node x but, even if > > more > > than one node is specified, that does not include giving to him a > > virtual NUMA topology, nor making it aware of the underline NUMA > > topology of the host in any way. > >=20 >=20 > =C2=A0=C2=A0=C2=A0AFAIK, dom0_nodes is implemented only in x86 not in arm= . >=20 Well --given, for instance, the example above-- of course it is! :-) Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-apN6RsfuKirtbyg2mZWX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYPCNWAAoJEBZCeImluHPu4U4QAKk7y8h/bn7ckzuQnPXfBL0D xw0pEVIZZXEVkh5lI7DiRHgnSZ5I80MyGUIfjLO1qEu9bFtVEWkdGazE+clE3XBw uXJjs1Bdvr3z+/SlTSUvIhk2zUseZXVtAOBI6PrTkCel0Pz/Of0TUV/RQTkOFvo4 6ZWnW+YN64X73xzgSP6XbWPbz8msGhmHERF/zbmb3L4iolmETq195lmmPjeTj2LT 2lZ5KwbLqZ0Dn8zEDScpJN8f6xJcd/N9Y4p/YrsJ8OWlSpDJUVvXMiH4H0Kv3YLK MuZ2RUk8lhmv06HpVqnfDQ6np4AQ+AHGTg+PHg96pAk/PIe2GsjCVuORAfzxp5xH oPsUD+9xp/hZc1RBnJuC1izqoJTAXbSc5xBxh/x6PhhxT4YwO5VWb3D8yWrx6f8c bUGaN/cjD0X0GORfjHID3hrGPwMJewwPSLLpS13VtK7m64Bbs74nmUagrD7NWiNq sikgs5pSf7fOIwxbE2Jc2ua+pmmmYfTRtfy2lCJvYBaJVbE0tdUcersic2gA0Nqk sKZxpIGs1HnXt6+1786hRUbPsaQmVgeZZ1Q+/KGn1z+kv+UGfJmBA3+5KCREW8NZ oFrz1PVPKU9XNGO3/z5w+bSOrzp7IHq+Jkav0nxh0s07b/3SQoj6n0x3HHHC8PDn O9M2ZHlcOhMCulg7H4iW =VEbU -----END PGP SIGNATURE----- --=-apN6RsfuKirtbyg2mZWX-- --===============0470577232228648314== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============0470577232228648314==--