From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 1 of 3 v5/leftover] libxl: enable automatic placement of guests on NUMA nodes Date: Fri, 20 Jul 2012 10:20:29 +0200 Message-ID: <1342772429.19530.247.camel@Solace> References: <5fa66c8b9093399e5bc3.1342458792@Solace> <5007FBCE.6000201@amd.com> <1342707771.19530.235.camel@Solace> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2778514099867428075==" Return-path: In-Reply-To: <1342707771.19530.235.camel@Solace> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andre Przywara Cc: Ian Campbell , Stefano Stabellini , George Dunlap , Andrew Cooper , Juergen Gross , Ian Jackson , xen-devel List-Id: xen-devel@lists.xenproject.org --===============2778514099867428075== Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-T2hkAwQ4Nnyhm9JZQ+e7" --=-T2hkAwQ4Nnyhm9JZQ+e7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, 2012-07-19 at 16:22 +0200, Dario Faggioli wrote: > Interesting. That's really the kind of testing we need in order to > fine-tune the details. Thanks for doing this. >=20 > > Then I started 32 guests, each 4 vCPUs and 1 GB of RAM. > > Now since the code prefers free memory so much over free CPUs, the=20 > > placement was the following: > > node0: guests 2,5,8,11,14,17,20,25,30 > > node1: guests 21,27 > > node2: none > > node3: none > > node4: guests 1,4,7,10,13,16,19,23,29 > > node5: guests 24,31 > > node6: guests 3,6,9,12,15,18,22,28 > > node7: guests 26,32 > >=20 > > As you can see, the nodes with more memory are _way_ overloaded, while= =20 > > the lower memory ones are underutilized. In fact the first 20 guests= =20 > > didn't use the other nodes at all. > > I don't care so much about the two memory-less nodes, but I'd like to= =20 > > know how you came to the magic "3" in the formula: > >=20 > > > + > > > + return sign(3*freememkb_diff + nrdomains_diff); > > > +} > >=20 > > That all being said, this is the first time the patchset had the chance > to run on such a big system, so I'm definitely open to suggestion on how > to make that formula better in reflecting what we think it's The Right > Thing! >=20 Thinking more about this, I realize that I was implicitly assuming some symmetry in the amount of memory each nodes comes with, which is probably something I shouldn't have done... I really am not sure what to do here, perhaps treating the two metrics more evenly? Or maybe even reverse the logic and give nr_domains more weight? I was also thinking whether it could be worthwhile to consider the total number of vcpus on a node instead than the number of domain, but again, that's not guaranteed to be any more meaningful (suppose there are a lot of idle vcpus)... Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-T2hkAwQ4Nnyhm9JZQ+e7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEABECAAYFAlAJFM0ACgkQk4XaBE3IOsRgUgCeMA3uDnQ6YZ5ek9zRJxvoO0bA +boAmwYpRWuOOPATFy+tEQ3I21biCj9S =UTDo -----END PGP SIGNATURE----- --=-T2hkAwQ4Nnyhm9JZQ+e7-- --===============2778514099867428075== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============2778514099867428075==--