From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: arm64: Approach for DT based NUMA and issues Date: Sun, 27 Nov 2016 02:01:52 +0100 Message-ID: <1480208512.2712.241.camel@citrix.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0085537099497016218==" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cAnr5-00046I-Uf for xen-devel@lists.xenproject.org; Sun, 27 Nov 2016 01:02:08 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Vijay Kilari , Stefano Stabellini , Julien Grall , Andre Przywara Cc: xen-devel , "prasun.kapoor" List-Id: xen-devel@lists.xenproject.org --===============0085537099497016218== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-bAsIiEIYiWAXs9zNXcxE" --=-bAsIiEIYiWAXs9zNXcxE Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, 2016-11-26 at 12:29 +0530, Vijay Kilari wrote: > 4) Generating memory nodes for DOM0 > --------------------------------------------------------- > Linux kernel device drivers that uses devm_zalloc(), tries to > allocate memory > from local memory node. So Dom0 needs to have memory allocated on all > the > available nodes of the system. >=20 So, first of all, I know next to nothing about ARM, NUMA or non-NUMA. That being said, providing a guest with a NUMA layout is what we call "virtual NUMA" (vNUMA). It is implemented (although it still has some issues) for x86 already (so you can have a look), but only for DomU. I agree that we need to support vNUMA for Dom0 sooner rather than later, and I agree that Dom0 is a bit special, so some tricks may be necessary. But until we don't implement vNUMA for Dom0, Dom0 is just a non-NUMA virtual machine, and the kernel running inside that should just behave like it behaves on a non-NUMA box. Again, I don't know much about ARM, but I think that, until we don't have vNUMA for Dom0, that devm_zalloc() thing will just see 1 and only 1 NUMA node from which to allocate memory. That being said, and for what it's worth... > Ex: SMMU driver of device on node 1 tries to allocate memory > on node 1. >=20 > ISSUE: > =C2=A0- Dom0's memory should be split across all the available memory > nodes > =C2=A0=C2=A0=C2=A0of the system and memory nodes should be generated acco= rdingly. > ...This is the default behavior, at least on x86. > =C2=A0- Memory DT node generated by Xen for Dom0 should populate numa- > node-id > =C2=A0=C2=A0=C2=A0information. >=20 Generating DT nodes for Dom0 is exactly what I mean when I say "implementing / enabling vNUMA for Dom0" (in this case on ARM). So, yes, let's do it, but let's discuss how to do it properly (e.g., if there's anything that can be common between archs, such as some bits of the interface). Thanks and Regards, Dario --=C2=A0 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-bAsIiEIYiWAXs9zNXcxE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYOjCAAAoJEBZCeImluHPuNg4P/0Dn7rTPzFI3zz4POUwzDF/y hyy3kYYbF/bEqKVa3NzPO3mzm8+mYDewuSv75r2jglJEy2Yb0uPJZqcrJKQqQMHp pqWVvaL7wv1GtV+46FmCeB8xbnLjXo41rFjAiZth5FoVLhWMbkYs+tTq68M+WjO4 3Zzj9gYE9VAJTrEkIIkRzYYoAFgnP9lf33FwFYK+JkHu/FgtuJ3Yssd9y7NvfBI2 1ZmEpgsZ7AgPzbVOKIq0toUKNUzfYl97xxmzBgR7DdqtRuCQl2wo4DRhEs5QTKnq axMW50hHBl+TonKxODJpE1fFKGG4EQU1VvtjgmCD73v9c9aYRI1n3/DNWNGf0lEQ KDh3HbO0RpMIum1zLvyypdVmD2Ks2zKTDXiTmDGIohuAOgm7HI1KT5uGXZnFAjI+ 5Kfzefo3GRGD2v16wDlDofWMxRAOQoUloOiPqAnNXO5IZRI56WLpaDW3ZGcJIF64 NfmLMXAJRUMmG2dv48nPYbXJ7L/6x5qV/UezkIaN/swuO1SlMbonaEZT6YSzR5Yw GdriDE9zj1P9XUGFZ9Pr24IfsVrJTGr99vz3Mqdh0MgXMc6mCim6tL69l+Bfy2dS Qi2dlu9Sy/8kPRcjWAAN3jxoHk4uslO4W9VDdlmOQXqByVdcILOFcLtto/6xPhvf 0FYFJopEEetoYvmZT3bg =KeXe -----END PGP SIGNATURE----- --=-bAsIiEIYiWAXs9zNXcxE-- --===============0085537099497016218== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============0085537099497016218==--