From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: About vcpu wakeup and runq tickling in credit Date: Wed, 24 Oct 2012 18:48:53 +0200 Message-ID: <1351097333.18330.15.camel@Solace> References: <1350999260.5064.56.camel@Solace> <5086B4DF.6060701@eu.citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1651033115773789615==" Return-path: In-Reply-To: <5086B4DF.6060701@eu.citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: George Dunlap Cc: Keir Fraser , Jan Beulich , xen-devel List-Id: xen-devel@lists.xenproject.org --===============1651033115773789615== Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-svvlkOFhygR0X8Od8Q4Z" --=-svvlkOFhygR0X8Od8Q4Z Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2012-10-23 at 16:16 +0100, George Dunlap wrote: > > If yes, here's my question. Is that right to always tickle v_W's affine > > CPUs and only them? > > > > I'm asking because a possible scenario, at least according to me, is > > that P schedules very quickly after this and, as prio(v_W)>prio(v_C), i= t > > selects v_W and leaves v_C in its runq. At that point, one of the > > tickled CPU (say P') enters schedule, sees that P is not idle, and trie= s > > to steal a vcpu from its runq. Now we know that P' has affinity with > > v_W, but v_W is not there, while v_C is, and if P' is not in its > > affinity, we've forced P' to reschedule for nothing. > > Also, there now might be another (or even a number of) CPU where v_C > > could run that stays idle, as it has not being tickled. >=20 > Yes -- the two clauses look a bit like they were conceived=20 > independently, and maybe no one thought about how they might interact. >=20 Yep, it looked the same to me. > > As it comes to possible solution, I think that, for instance, tickling > > all the CPUs in both v_W's and v_C's affinity masks could solve this, > > but that would also potentially increase the overhead (by asking _a_lot= _ > > of CPUs to reschedule), and again, it's hard to say if/when it's > > worth... >=20 > Well in my code, opt_tickle_idle_one is on by default, which means only= =20 > one other cpu will be woken up. If there were an easy way to make it=20 > wake up a CPU in v_C's affinity as well (supposing that there was no=20 > overlap), that would probably be a win. >=20 Yes, default is to tickle only 1 idler. However, as we offer that as a command line option, I think we should consider what could happen if one disables it. I double checked this on Linux and, mutatis mutandis, they sort of go the way I was suggesting, i.e., "pinging" the CPUs with affinity to the task that will likely stay in the runq rather than being picked up locally. However, there are of course big difference between the two schedulers and different assumptions being made, thus I'm not really sure of that being the best thing to do for us. So, yes, it probably make sense to think about something clever to try to involve CPUs from both the masks without causing too much overhead. I'll put that in my TODO List. :-) > Of course, that's only necessary if: > * v_C is lower priority than v_W > * There are no idlers that intersect both v_C and v_W's affinity mask. >=20 Sure, I said that in the first place, and I don't think checking for that is too hard... Just a couple of more bitmap ops. But again, I'll give it some thinking. > It's probably a good idea though to try to set up a scenario where this= =20 > might be an issue and see how often it actually happens. >=20 Definitely, before trying to "fix" it, I'm interested in finding out what I'd be actually fixing. Will do that. Thanks for taking time to check and answer this! :-) Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-svvlkOFhygR0X8Od8Q4Z Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEABECAAYFAlCIG/UACgkQk4XaBE3IOsQ9NwCfVUC/m2bq0P3eUGiJ9nPfmVp+ LmYAnRlw7XBotD+2F3VGsFkzPcHBI4sI =Bvht -----END PGP SIGNATURE----- --=-svvlkOFhygR0X8Od8Q4Z-- --===============1651033115773789615== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1651033115773789615==--