From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: PV-vNUMA issue: topology is misinterpreted by the guest Date: Tue, 28 Jul 2015 19:13:06 +0200 Message-ID: <1438103586.2889.143.camel@citrix.com> References: <1437042762.28251.18.camel@citrix.com> <55A78DF2.1060709@citrix.com> <20150716152513.GU12455@zion.uk.xensource.com> <55A7D17C.5060602@citrix.com> <55A7D2CC.1050708@oracle.com> <55A7F7F40200007800092152@mail.emea.novell.com> <55A7DE45.4040804@citrix.com> <55A7E2D8.3040203@oracle.com> <55A8B83802000078000924AE@mail.emea.novell.com> <1437118075.23656.25.camel@citrix.com> <55A946C6.8000002@oracle.com> <1437401354.5036.19.camel@citrix.com> <55AD08F7.7020105@oracle.com> <55AEA4DD.7080406@oracle.com> <1437572160.5036.39.camel@citrix.com> <55AF9F8F.7030200@suse.com> <55AFA16B.3070103@oracle.com> <55AFA41E.1080101@suse.com> <55AFAC34.1060606@oracle.com> <55B070ED.2040200@suse.com> <1437660433.5036.96.camel@citrix.com> <55B21364.5040906@suse.com> <1437749076.4682.47.camel@citrix.com> <55B25650.4030402@suse.com> <55B258C9.4040400@suse.com> <1437753509.4682.78.camel@citrix.com> <55B26377.4060807@suse.com> <1438006166.5036.156.camel@citrix.com> <55B7052F.8090804@suse.com> <55B79B9C.6030505@suse.com> <1438100238.2889.135.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4453965175456319241==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZK8Rn-0003J6-H1 for xen-devel@lists.xenproject.org; Tue, 28 Jul 2015 17:13:47 +0000 In-Reply-To: <1438100238.2889.135.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Juergen Gross Cc: Elena Ufimtseva , Wei Liu , Andrew Cooper , David Vrabel , Jan Beulich , "xen-devel@lists.xenproject.org" , Boris Ostrovsky List-Id: xen-devel@lists.xenproject.org --===============4453965175456319241== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-J0MArgspD43OVqdc0w/g" --=-J0MArgspD43OVqdc0w/g Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2015-07-28 at 18:17 +0200, Dario Faggioli wrote: > So, my test box looks like this: > cpu_topology : > cpu: core socket node > 0: 0 1 0 > 1: 0 1 0 > 2: 1 1 0 > 3: 1 1 0 > 4: 9 1 0 > 5: 9 1 0 > 6: 10 1 0 > 7: 10 1 0 > 8: 0 0 1 > 9: 0 0 1 > 10: 1 0 1 > 11: 1 0 1 > 12: 9 0 1 > 13: 9 0 1 > 14: 10 0 1 > 15: 10 0 1 >=20 > In Dom0, here's what I see _without_ any pinning: >=20 And now I've tried with dom0_vcpus_pin: root@Zhaman:~# xl vcpu-list=20 Name ID VCPU CPU State Time(s) Affinity= (Hard / Soft) Domain-0 0 0 0 -b- 6.8 0 / all Domain-0 0 1 1 -b- 1.6 1 / all Domain-0 0 2 2 -b- 2.3 2 / all Domain-0 0 3 3 -b- 1.5 3 / all Domain-0 0 4 4 -b- 3.2 4 / all Domain-0 0 5 5 -b- 1.5 5 / all Domain-0 0 6 6 -b- 2.0 6 / all Domain-0 0 7 7 -b- 2.2 7 / all Domain-0 0 8 8 -b- 1.6 8 / all Domain-0 0 9 9 -b- 1.6 9 / all Domain-0 0 10 10 r-- 2.1 10 / all Domain-0 0 11 11 -b- 1.5 11 / all Domain-0 0 12 12 -b- 2.4 12 / all Domain-0 0 13 13 -b- 1.1 13 / all Domain-0 0 14 14 -b- 2.4 14 / all Domain-0 0 15 15 -b- 2.4 15 / all > root@Zhaman:~# for i in `seq 0 15`;do cat /sys/devices/system/cpu/cpu$i/t= opology/thread_siblings_list ;done > 0-1 > 0-1 > 2-3 > 2-3 > 4-5 > 4-5 > 6-7 > 6-7 > 8-9 > 8-9 > 10-11 > 10-11 > 12-13 > 12-13 > 14-15 > 14-15 >=20 Same result. > root@Zhaman:~# cat /proc/cpuinfo |grep "physical id" > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 1 > physical id : 0 > physical id : 0 > physical id : 0 > physical id : 0 > physical id : 0 > physical id : 0 > physical id : 0 > physical id : 0 >=20 Same result. > root@Zhaman:~# cat /proc/cpuinfo |grep "core id" > core id : 0 > core id : 0 > core id : 1 > core id : 1 > core id : 9 > core id : 9 > core id : 10 > core id : 10 > core id : 0 > core id : 0 > core id : 1 > core id : 1 > core id : 9 > core id : 9 > core id : 10 > core id : 10 >=20 Same result. > root@Zhaman:~# cat /proc/cpuinfo |grep "cpu cores" > cpu cores : 4 > >=20 Same result. > root@Zhaman:~# cat /proc/cpuinfo |grep "siblings"=20 > siblings : 8 > >=20 And same result here as well. So, for Dom0, pinning does not make much difference as far as what topology is guessed, which actually makes sense, considering how pcpus are brought up in Xen, and assuming that Linux does something similar (no, I'm not familiar with that code). It should be making a difference in terms of whether and how much this topology, although matching the host one, mislead the guest scheduler. Actually, I still think it does, it's just harder to identify than we expected, which may be seen as a good thing (it does not look like it is a big deal, after all :-D), or a bad thing (it may start biting us anytime, without us noticing promptly :-/). Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-J0MArgspD43OVqdc0w/g Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlW3uCIACgkQk4XaBE3IOsS54wCdHELdclkCm7Y3QxyrxWoOblik AHkAn31CnJ7cs1I9IJlmQUhx/nM4Ryz7 =e6Ks -----END PGP SIGNATURE----- --=-J0MArgspD43OVqdc0w/g-- --===============4453965175456319241== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4453965175456319241==--