From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 18/24] xen: credit2: soft-affinity awareness fallback_cpu() and cpu_pick() Date: Mon, 5 Sep 2016 15:26:42 +0200 Message-ID: <1473082002.19612.45.camel@citrix.com> References: <147145358844.25877.7490417583264534196.stgit@Solace.fritz.box> <147145438023.25877.4564307455358470316.stgit@Solace.fritz.box> <57C80C30.8060205@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2188085939447214603==" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bgtvu-0001ZA-J9 for xen-devel@lists.xenproject.org; Mon, 05 Sep 2016 13:27:30 +0000 In-Reply-To: <57C80C30.8060205@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: anshul makkar , xen-devel@lists.xenproject.org Cc: "Justin T. Weaver" , George Dunlap List-Id: xen-devel@lists.xenproject.org --===============2188085939447214603== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-8qk98Hz1gJFzfWAwS2a+" --=-8qk98Hz1gJFzfWAwS2a+ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, 2016-09-01 at 12:08 +0100, anshul makkar wrote: > On 17/08/16 18:19, Dario Faggioli wrote: > >=C2=A0 > > diff --git a/xen/common/sched_credit2.c > > b/xen/common/sched_credit2.c > >=C2=A0 > > @@ -506,34 +506,68 @@ void smt_idle_mask_clear(unsigned int cpu, > > cpumask_t *mask) > > =C2=A0 } > >=20 > > =C2=A0 /* > > - * When a hard affinity change occurs, we may not be able to check > > some > > - * (any!) of the other runqueues, when looking for the best new > > processor > > - * for svc (as trylock-s in csched2_cpu_pick() can fail). If that > > happens, we > > - * pick, in order of decreasing preference: > > - *=C2=A0=C2=A0- svc's current pcpu; > > - *=C2=A0=C2=A0- another pcpu from svc's current runq; > > - *=C2=A0=C2=A0- any cpu. > > + * In csched2_cpu_pick(), it may not be possible to actually look > > at remote > > + * runqueues (the trylock-s on their spinlocks can fail!). If that > > happens, > With remote runqueues , do you mean runqs on remote socket ?=20 > I mean runqueues different from the runq the vcpu is currently assigned to (as per runq_assign()/runq_deassing()). If you have runqueues configured to be per-socket, yes, it will try to lock runqueues in which there are pcpus that are on a different socket wrt svc->vcpu->processor. > Can't we=C2=A0 > just read their workload or we can change the locktype to allow > reading ? > Reading without taking the lock would race against the load value being updated. And updating the load is done by __update_runq_load(), which, with all it's shifts and maths, by no means is an atomic operation. So it's not just a matter of risking to read a slightly outdated value, which, I agree, may not be that bad, it's that we risk reading something inconsistent. :-/ About "changing the locktype", I guess you mean that we can turn also the runqueue lock into an rw-lock? If yes, that's indeed interesting, and I've also thought about it, but, for now, always deferred trying to actually do that. It's technically non trivial, as it would involve changing schedule_data->schedule_lock and all the {v,p}cpu_schedule_lock*() functions. Also, it's a lock that will almost all the times be taken for writing, which usually means what you want is a proper spinlock. So, IMO, before embarking in doing something like that, it'd be good to figure out how frequently we actually fail to take the remote runqueue lock, and what's the real impact of having to deal the consequence of that. I'm not saying it's not worth a try, but I'm saying that's it's something at high risk of being a lot of work for a very small gain, and that there are more important things to focus on. > > + * we pick, in order of decreasing preference: > > + *=C2=A0=C2=A01) svc's current pcpu, if it is part of svc's soft affin= ity; > > + *=C2=A0=C2=A02) a pcpu in svc's current runqueue that is also in svc'= s soft > > affinity; > svc's current runqueue. Do you mean the runq in which svc is > currently=C2=A0 > queued ? > I mean the runqueue to which svc is currently assigned (again, as per runq_assign()/runq_deassing()), which in turns mean that, if svc is queued in a runqueue, it's queues there (so, I guess the short answer to your question is "yes" :-D). Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-8qk98Hz1gJFzfWAwS2a+ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJXzXKTAAoJEBZCeImluHPuSlcP/irZwurOwo4VUje7N7n4f6D/ FxLYGRMtZHGWgxNHLqRO82C+7N6xrwhxKfga/6Z3Ylqej2Vb4MMLjsLAKdGmDHZ2 Z7bJhmTBlWjH0ME9MoM9l5pwVbgelpGM71K4A4uu5r1jncPyOMf0nHH6+RnXG0gh WCdM2E2aCKRR4evQ0GrsIWCFGsMqnQHwWnabkXzsPW8bRXS6HQlrWMYlCw2piEMK poUNKTbaMSXiByufdtvzg0H7MJbStAOrSPG1iUk9kT71ONrnVqCOFuGsyz17UtcW u7o6Uy71/mUkoPjcCeqJGdojgqmMxcvhE2ZOVBJlQEtfop/3y2xOf8JvLIN91mMy h1jRmli05c1VBsen1epcLnHIjXFz8l8422rL9sijYnQLJfv0tgZqstzjQtIMxYpk QQKX7fpxqeh5eK71QyPntHdT50WKQYOTZ+FIm9Gz8Di9+qXcjy5awnKri4wU04fM HmVnDp3YoQC9iA2BdIQGFL/+XjwGE1x5lSZ/yXEwDOJ1BbsP4zzIAcoKT9DlT4dD qpZJY6upiPS0l5v2tlxUZdlYZWcn5U7vJyRvHu76qS+xLRVlmUxGpl7rh/kZju+k 5jKF7wE8bniKd3aUKMYMbXeOkToAqYc0umFiSNIiK5YRUruM2jP0Z9gH89x0F1Sv bKDOZiGr/v6tG/PSCGn+ =sqVV -----END PGP SIGNATURE----- --=-8qk98Hz1gJFzfWAwS2a+-- --===============2188085939447214603== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============2188085939447214603==--