From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [DOC RFC] Heterogeneous Multi Processing Support in Xen Date: Thu, 15 Dec 2016 19:41:56 +0100 Message-ID: <1481827316.3445.354.camel@citrix.com> References: <1481135379.3445.142.camel@citrix.com> <3f9a7da1-7c05-e082-99d2-302dbeee61b9@suse.com> <1481192825.3445.157.camel@citrix.com> <8ee2f981-fdad-63e2-5779-02fedc7d137d@suse.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6772270334331093737==" Return-path: In-Reply-To: <8ee2f981-fdad-63e2-5779-02fedc7d137d@suse.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Juergen Gross , Xen Devel Cc: Peng Fan , Stefano Stabellini , George Dunlap , Andrew Cooper , anastassios.nanos@onapp.com, Jan Beulich , Peng Fan List-Id: xen-devel@lists.xenproject.org --===============6772270334331093737== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-zU79tnZkUJKzHaviViPH" --=-zU79tnZkUJKzHaviViPH Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, 2016-12-08 at 11:38 +0100, Juergen Gross wrote: > So you really solved the following problem in credit2? >=20 > You have three domains with 2 vcpus each and different weights. Run > them > on 3 physical cpus with following pinning: >=20 > dom1: pcpu 1 and 2 > dom2: pcpu 2 and 3 > dom3: pcpu 1 and 3 >=20 > How do you decide which vcpu to run on which pcpu for how long? >=20 Ok, back to this (sorry, a bit later than how I'd hoped). So, I tried to think a bit at the described scenario, but could not figure out what you are hinting at. There are missing pieces of information, such as what the vcpus do, and what exactly are the weights (besides than being different). Therefore, I decided to put together a quick eperiment. I've created the domains, sat up all their vcpus to run cpu-hog tasks, picked up a configuration of my choice for the weights, and run them under both Credit1 and Credit2. It's a very simple tests, but it will hopefully be helpful in understanding the situation better. Here's the result. On Credit1, equal weigths, unpinned (i.e., plenty of pCPUs available): =C2=A0NAME =C2=A0CPU(%) [1] =C2=A0vm1 =C2=A0=C2=A0199.9 =C2=A0vm2 =C2=A0=C2=A0199.9 =C2=A0vm3 =C2=A0=C2=A0199.9 Pinning as you suggest (i.e., to 3 pCPUs): =C2=A0NAME =C2=A0CPU(%) [2] =C2=A0vm1 =C2=A0149.0 =C2=A0vm2 =C2=A0=C2=A0=C2=A066.2 =C2=A0vm3 =C2=A0=C2=A084.8 Changing the weights: =C2=A0Name =C2=A0ID Weight=C2=A0=C2=A0Cap [3] =C2=A0vm1 =C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0256=C2=A0=C2=A0=C2=A0=C2=A00 =C2=A0vm2 =C2=A0 9=C2=A0=C2=A0=C2=A0=C2=A0512=C2=A0=C2=A0=C2=A0=C2=A00 =C2=A0vm3 =C2=A0 6=C2=A0=C2=A0=C2=A01024=C2=A0=C2=A0=C2=A0=C2=A00 NAME CPU(%) vm1 100.0 vm2 100.0 vm3 100.0 So, here in Credit1, things are ok when there's no pinning in place [1]. As= soon as we pin, _even_without_ touching the weights [2], things become *cr= azy*. In fact, there's absolutely no reason why CPU% numbers would look lik= e how they look in [2]. This does not surprise me much, though. Credit1's load balancer basically m= oves vcpus around in a pseudo random fashion, and having to enforce pinning= constraints make things even more unpredictable. Then it comes the amusing part. At this point, I wonder if I haven't done s= omething wrong in setting up the experiments... Because things really looks= too funny. :-O In fact, for some reasons, changing the weights as shown [3] cause CPU% num= bers to fluctuate a bit (not visible above) and then to stabilize at 100%. = That may look like an improvement, but certainly does not reflect the chose= n set of weights. So, I'd say you were right. Or, actually, things are even worse than what y= ou said: in Credit1, it's not only that pinning and weights does not play w= ell together, it's that even pinning alone works pretty bad. Now, on Credit2, equal weigths, unpinned (i.e., plenty of pCPUs available): =C2=A0NAME =C2=A0CPU(%) [4] =C2=A0vm1 =C2=A0 199.9 vm2 199.9 vm3 199.9 Pinning as you suggest (i.e., to 3 pCPUs): NAME=C2=A0=C2=A0CPU(%) [5] vm1 100.0 vm2 100.1 vm3 100.0 Changing the weights: Name ID Weight [6] vm1 2 256 vm2 3 512 vm3 6 1024 NAME CPU(%) vm1 44.1 vm2 87.2 vm3 168.7 Which looks nearly *perfect* to me. :-) In fact, with no constraints [4], each VM gets the 200% share it's asking for. When only 3 pCPUs can be used, by means of pinning [5], each VM gets its fair share of 100%. When setting up weights in such a way that vm2 should get 2x CPU time than vm1 and vm3 should get 2x CPU time than vm2 [6], things looks, well, exactly like that! :-P So, since I did not fully understand the problem, I'm not sure whether this really answers your question, but it look to me like it actually could! :-D For sure, it puts Credit2 in rather a good light :-P. Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-zU79tnZkUJKzHaviViPH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYUuP1AAoJEBZCeImluHPurqkQAI3uzQzfGLJ2BHy0vjtlJp6G rgFqnVCnibhuoWGhyHMDl7qDzwpe8u1XKV3Oku4UFy61lt2nq44Kr7MUegj3LNMN act86xRUeJlvlegcSM8ckCsE23OLUgBQi07MfJdfV7iBQ3jdsK6/hhU8tMhAiNtr c0B9B/H2sXeyBjCjbcUL3BcFSr2ifqUcxrm0s4fsW6lrgfxeRC5fiaGRsLZSfgfc bGIH6mTflRGo3Fnk6yw1YgN/kbQV/jhr3rcI/J8F8vYaMC4owZ7hyNpO/34VAwHt 9XuDflKcTvzuo7NKey3m6KQxfDI51HzA88mhDAH1AHKFyNfouPPLXyJMw/RYmtSC e9j3nOftbc1JUwxhZcExketwwrwcvDJ1C0SreX2LS6kcOjFA9MtwHkX2U3NxRdR/ tneqHKlBXy9rdDJvk3J+fyXTzXpwDk2QZrmNSdyaMKqLRx2UpFVf/8foUkarbhBq V1rbPYM2Z28StLUuuRpL+jLkmlExzBLFwtyfQwEV/I+V3i0bsIDgHaHdUmBEq/oI 3lfwOeWOBqldw/MqgjcGj7o4/mNDi6RY7kJWtxbN72ySspK527f4YOsddShptAcn SQXSv+Z3ZfxwEMWbLa5gzJIhG39wggVXXzFLaylP0ExCV8yzL55ikU8E2uE0Kn/c XPN/6l/Ku0FrmjJ8jkKW =mtHR -----END PGP SIGNATURE----- --=-zU79tnZkUJKzHaviViPH-- --===============6772270334331093737== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============6772270334331093737==--