From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 07 of 10 [RFC]] sched_credit: Let the scheduler know about `node affinity` Date: Fri, 13 Apr 2012 01:06:35 +0200 Message-ID: <1334271995.2417.49.camel@Abyss> References: <1f4b55806de9e7109ff6.1334150274@Solace> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4242401788805767873==" Return-path: In-Reply-To: <1f4b55806de9e7109ff6.1334150274@Solace> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org Cc: Andre Przywara , Ian Campbell , Stefano Stabellini , George Dunlap , Juergen Gross , Ian Jackson , Jan Beulich List-Id: xen-devel@lists.xenproject.org --===============4242401788805767873== Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-LM2ZLqsghGK+GGRy7bn+" --=-LM2ZLqsghGK+GGRy7bn+ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2012-04-11 at 15:17 +0200, Dario Faggioli wrote:=20 > Basic idea is: cpu-affinity tells where a vcpu can run, > while node-affinity tells where (in terms of on what NUMA nodes, > but that of course imply a set of CPUs) all the vcpus of the > domain should/prefer to run... Problems starts when you have > both of them speaking at the same time! > > >=20 > XXX I'm benchmarking this just right now, and peeking at the results > they don't seem too bad. Numbers are mostly close to the case where > the VM is already pinned to a node when created. I'll post the > results as soon as I can, and I'll be happy to investigate any issue, > if you feel like the approach could be the right one. >=20 Some of them are ready, and here they come: http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/index.ht= ml Look, for example, at this one here: http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/kernbench= _avgstd.png Where the red line is the default Xen behavior (no pinning, no affinity, nothing!), the dark blue line is the best case (domain pinned at creation =3D=3D> all memory accesses are local) and the light blue line is this one patch in action, i.e., domain _with_affinity_ to node #0 and no pinning at all. It's very nice (at least for me! :-D) to see it almost always does better than default, and it manage in getting quite close to the best case scenario when load increases. Still on the "nice things" side, there doesn't appear to be significative regressions introduced by this patch, if the new node-affinity feature is not stressed, as it shows by comparing the "before" and "after" the patch graphs (looking at matching curves): (before) http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005/kernbe= nch_avgstd.png (after) http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/= kernbench_avgstd.png Unfortunately, there seems to be at least some issues with the cases where load is not so high. In fact, given VMs have 4 vcpus and host has 16 pcpus, there are less vcpus than pcpus up to the run involving 4 VMs, which is actually where things starts to work well, while before they're near or even worst than default! :-O Any thoughts? Again, if you think the approach taken here could be a sensible one, I'll start digging what's going on (by some tracing etc.), and I'm sure I can work out the low-load cases and "persuade" them to behave better. :-D Thanks and Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-LM2ZLqsghGK+GGRy7bn+ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEABECAAYFAk+HX/sACgkQk4XaBE3IOsRZ3ACeI3HRaMCTj1thjVBvqYr60fte fIYAnAxk/qKzo8bOaoY99B0GTDcGY0Q5 =Kbxl -----END PGP SIGNATURE----- --=-LM2ZLqsghGK+GGRy7bn+-- --===============4242401788805767873== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4242401788805767873==--