From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: RFC: Still TODO for 4.2? xl domain numa memory allocation vs xm/xend Date: Fri, 20 Jan 2012 12:44:34 +0100 Message-ID: <1327059874.2337.38.camel@Abyss> References: <1325694562.25206.304.camel@zakaz.uk.xensource.com> <20120119211430.GT12984@reaktio.net> <1327046368.21391.29.camel@dagon.hellion.org.uk> <1327058562.17599.134.camel@zakaz.uk.xensource.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8639657199683423197==" Return-path: In-Reply-To: <1327058562.17599.134.camel@zakaz.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Campbell Cc: xen-devel , "Keir (Xen.org)" , Stefano Stabellini , George Dunlap , "Tim (Xen.org)" , Ian Jackson , Jan Beulich List-Id: xen-devel@lists.xenproject.org --===============8639657199683423197== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-x3E40jczZXXTjW0U40lM" --=-x3E40jczZXXTjW0U40lM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, 2012-01-20 at 11:22 +0000, Ian Campbell wrote:=20 > > It seems that xend is retrieving numa info about the platform, see > > pyxc_numainfo, then using those info to pin vcpus to pcpus, see > > _setCPUAffinity. > > Still it seems to me more of an hack than the right way to solve the > > problem. >=20 > Right, so in the absence of any explicit configuration it basically > picks a NUMA node (via some heuristic) and automatically puts the guest > into it. >=20 Seems so. As Stefano is saying I don't think this is something that should be done at the toolstack level, or at least not at the xl-level of the toolstack. :-) > It seems to me that xl's behaviour isn't wrong as such, it's just > different. >=20 Indeed. > I think the important thing is that xl should honour user's explicit > requests to use a particular node, either via vcpu pinning or cpupools > etc. >=20 And I agree again, honouring explicit user requests is key point. I think the issue here is what should be dona, say by default, i.e., if the user doesn't say anything about CPU/memory allocation. My idea was to have Xen supporting a "NUMA-aware operational mode" where (and this will actually be the first step!) it does exactly what xend is doing right now --- that is, choosing a node and putting the new guest there, both memory and CPU-wise. However, having this logic in the hypervisor would allow Xen itself, for example, while investigating which node to use for a new guest, or during a sort of periodic load balancing or whatever, to change its mind and move a guest to a different node from where it was put in the first place, as well as a bunch of other things. I'm not sure the same can be done within the toolstack but I think I can say that if it can, it would be way more complex and probably less effective... Am I wrong? Of course, even in such mode, if the user explicitly tells us what he wants, e.g., by means of cpupools, pinning, etc., we should still honour such request. Then the question is whether or nod this mode would be the default, or would need to be explicitly requested (boot parameter or something), but that would become important only when we will have it up and running... :-) What do you think? Does this look reasonable? As the topic has been raised, I'd very much enjoy some early feedback! :-P Regards, Dario --=20 <> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, PhD, Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-x3E40jczZXXTjW0U40lM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEABECAAYFAk8ZU6MACgkQk4XaBE3IOsQKNQCgknwYi+ceqzO0mVvoV+QgHYc3 ggYAn3ifcmHJjcLkZtEdRIKt/vtcm8Gh =SsRw -----END PGP SIGNATURE----- --=-x3E40jczZXXTjW0U40lM-- --===============8639657199683423197== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============8639657199683423197==--