From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: Xen on ARM IRQ latency and scheduler overhead Date: Fri, 17 Feb 2017 12:02:41 +0100 Message-ID: <1487329361.6732.76.camel@citrix.com> References: <1486716022.3042.112.camel@citrix.com> <1487247614.6732.45.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1030558948960127617==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Stefano Stabellini Cc: george.dunlap@eu.citrix.com, edgar.iglesias@xilinx.com, julien.grall@arm.com, xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org --===============1030558948960127617== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-DUZ2m2MaihFqq4UGasYV" --=-DUZ2m2MaihFqq4UGasYV Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Just very quickly... On Thu, 2017-02-16 at 15:07 -0800, Stefano Stabellini wrote: > (XEN) Active queues: 1 > (XEN)=C2=A0=C2=A0 default-weight=C2=A0=C2=A0=C2=A0=C2=A0 =3D 256 > (XEN) Runqueue 0: > (XEN)=C2=A0=C2=A0 ncpus=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 4 > (XEN)=C2=A0=C2=A0 cpus=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0-3 > (XEN)=C2=A0=C2=A0 max_weight=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 =3D 256 > (XEN)=C2=A0=C2=A0 instload=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 =3D 1 > (XEN)=C2=A0=C2=A0 aveload=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 =3D 3208 (~1%) > (XEN) l(XEN)=C2=A0=C2=A0=C2=A0 idlers: 00000000,00000000,00000000,0000000= a > a(XEN)=C2=A0 tickled: 00000000,00000000,00000000,00000000 > t(XEN)=C2=A0 fully idle cores: 00000000,00000000,00000000,0000000a > e(XEN) Domain info: > n(XEN)=C2=A0 Domain: 0 w 256 v 4 > c(XEN)=C2=A0=C2=A0=C2=A0 1: [0.0] flags=3D2 cpu=3D0 credit=3D10500000 [w= =3D256] load=3D3170 (~1%) > (XEN)=C2=A0=C2=A0=C2=A0=C2=A0 2: y[0.1] flags=3D0 cpu=3D1=C2=A0 credit=3D= 10500000 [w=3D256]( load=3D131072 (~50%) > (XEN)=C2=A0=C2=A0=C2=A0=C2=A0 3: n[0.2] flags=3D0 cpu=3D2s credit=3D10500= 000 [w=3D256]) load=3D131072 (~50%): > (XEN)=C2=A0=C2=A0=C2=A0=C2=A0 4:=C2=A0 [0.3] flags=3D0 cpu=3D3m credit=3D= 10500000 [w=3D256]a load=3D131072 (~50%)x > Status of vcpus 2, 3 and 4 is a bit weird. I'll think about it. > =3D(XEN)=C2=A0 Domain: 1 w 256 v 1 > 1(XEN)=C2=A0=C2=A0=C2=A0 5: 1[1.0] flags=3D2 cpu=3D2 credit=3D9713074 [w= =3D256] load=3D56 (~0%) > (XEN) Runqueue info: > 6(XEN) runqueue 0: > 9(XEN) CPUs info: > 0(XEN) CPU[00]=C2=A0=C2=A0 runq=3D0, sibling=3D00000000,00000000,00000000= ,00000001, wcore=3D00000000,00000000,00000000,00000001 > This tells me that nr_cpu_ids is very big (I think it tells it is 128, i.e., ARM default), which means cpumask_t-s are huge. What does `xl info' says. On my (x86) test box, it's like this: =C2=A0... =C2=A0nr_cpus=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 16 =C2=A0max_cpu_id=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0: 63 =C2=A0... (and I have=C2=A0NR_CPUS=3D256, i.e., x86 the default). Cpumasks being bigger also means cpumask operation being slower, and this matters quite a bit in Credit2, because we use cpumasks a lot (but also in Credit1, because we use cpumasks a little less than in Credit2, but still quite a bit). Isn't there a way, on ARM, to figure out online that you're not going to have 128 cpus in the platform? Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-DUZ2m2MaihFqq4UGasYV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJYpthTAAoJEBZCeImluHPuGFoQAMjrLYNJFz8diiwo8HVWMAxP 6xv/yEmfMTMEK1zpk+8eCiGlXXcON4Z+o0NPiTQw6qZYN3RjfzJVpw1Sd5fav1wq FGemmcGhSeLPsDxJq3I+jyAqZ7vSf2JvgIjYqFBVcor3raaA++Ak0XvKTnhw2bMj mIZ9WxIdUDYouZYbag44tKNwpVY2laPIlmX04A5lzojAfZPi9Fma0rspclrPFh73 Hn7JoBfYZIqMeaJIyagD2v1KeprQ4I7Gx585T+rk6b1B6AXLHa+7+NyDkggzeFYa i1MHKKHHgPB77y760n62AmlNZCQDdWtNOeY+RSnC6ZXmAl2HDOWS7+/Ty1NSe/ZY 1vrdMojndK+5HNBWU+pddDaRZPEVVcFsY4NZMFD/VqJNKbCgnxuCgskAnP4JNfcC NvU9rT+t/A/+QOmGGJNcMMo8cT5HSrVkJnP3Z1EHFuZyc6eUdy9EJDBl+KVz2cfE 1q/t1I1iWrxykShq/MqQ6Q9XxHaE3GaAMJKe0hpEr360XWmaqxCC5t397nEBAZGT LH4t1YVv0B9M1AFLz5C5ZR49xXBv8Wpi8cKYWjvbB0WHwEl8Q8Q9J3RJI8pRpaRp HiUbyQnZzREc7GE8qioShj4Ay7KRkzgULq2o939LwMAaQ2ugWLr94aqgNRKlowew Rb1VSWZGK4Ynx0LsQXSm =MEtr -----END PGP SIGNATURE----- --=-DUZ2m2MaihFqq4UGasYV-- --===============1030558948960127617== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============1030558948960127617==--