From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 3/4] xen: sched: reorganize cpu_disable_scheduler() Date: Wed, 8 Jul 2015 17:13:36 +0200 Message-ID: <1436368416.22672.220.camel@citrix.com> References: <20150703152743.23194.15530.stgit@Solace.station> <20150703154930.23194.20319.stgit@Solace.station> <559BB4FC.1070106@suse.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7919274333823216871==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZCr3x-0006El-5Q for xen-devel@lists.xenproject.org; Wed, 08 Jul 2015 15:15:05 +0000 In-Reply-To: <559BB4FC.1070106@suse.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Juergen Gross Cc: George Dunlap , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org --===============7919274333823216871== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-1QHQPBGO1GEPcWOXdek/" --=-1QHQPBGO1GEPcWOXdek/ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2015-07-07 at 13:16 +0200, Juergen Gross wrote: > On 07/03/2015 05:49 PM, Dario Faggioli wrote: > > Solution is to distinguish, inside cpu_disable_scheduler(), > > the two cases of cpupool manipulation and teardown. For > > cpupool manipulation, it is correct to ask the scheduler to > > take an action, as pathological situation (like there not > > being any cpu in the pool where to send vcpus) are taken > > care of (i.e., forbidden!) already. For suspend and shutdown, > > we don't want the scheduler to be involved at all, as the > > final goal is pretty simple: "send all the vcpus to the > > boot cpu ASAP", so we just go for it. > > > > Signed-off-by: Dario Faggioli >=20 > Just 2 minor nits (see below), otherwise: >=20 > Acked-by: Juergen Gross >=20 Thanks. > > --- a/xen/common/schedule.c > > +++ b/xen/common/schedule.c > > @@ -645,25 +675,72 @@ int cpu_disable_scheduler(unsigned int cpu) > > cpumask_setall(v->cpu_hard_affinity); > > } > > > > - if ( v->processor =3D=3D cpu ) > > + if ( v->processor !=3D cpu ) > > { > > - set_bit(_VPF_migrating, &v->pause_flags); > > + /* This vcpu is not on cpu, so we can move on. */ > > vcpu_schedule_unlock_irqrestore(lock, flags, v); > > - vcpu_sleep_nosync(v); > > - vcpu_migrate(v); > > + continue; > > } > > - else > > - vcpu_schedule_unlock_irqrestore(lock, flags, v); > > > > /* > > - * A vcpu active in the hypervisor will not be migratable. > > - * The caller should try again after releasing and reaquir= ing > > - * all locks. > > + * If we're here, it means that the vcpu is on cpu. Let's = see how > > + * it's best to send it away, depending on whether we are = shutting > > + * down/suspending, or doing cpupool manipulations. > > */ > > - if ( v->processor =3D=3D cpu ) > > - ret =3D -EAGAIN; > > - } > > + set_bit(_VPF_migrating, &v->pause_flags); > > + vcpu_schedule_unlock_irqrestore(lock, flags, v); > > + vcpu_sleep_nosync(v); > > + > > + /* > > + * In case of shutdown/suspend, it is not necessary to ask= the > > + * scheduler to chime in. In fact: > > + * * there is no reason for it: the end result we are aft= er is > > + * just 'all the vcpus on the boot pcpu, and no vcpu an= ywhere > > + * else', so let's just go for it; > > + * * it's wrong, when dealing a cpupool with only non-boo= t pcpus, > > + * as the scheduler will always fail to send the vcpus = away > > + * from the last online (non boot) pcpu! >=20 > I'd add a comment that in shutdown/suspend case all domains are being > paused, so we can be active in dom0/Pool-0 only. >=20 Sure, I'll add this. > > + * > > + * Therefore, in the shutdown/suspend case, let's just pic= k one > > + * of the still online pcpus, and send everyone there. Ide= ally, > > + * we would pick up the boot pcpu directly, but we don't k= now > > + * which one it is. > > + * > > + * OTOH, if the system is still live, and we are here beca= use of > > + * cpupool manipulations: > > + * * it would be best to call the scheduler, as that woul= d serve > > + * as a re-evaluation of the placement of the vcpu, tak= ing into > > + * account the modified cpupool configuration; > > + * * the scheduler will always fine a suitable solution, = or > > + * things would have failed before getting in here. > > + * > > + * Therefore, in the cpupool manipulation case, let's just= ask the > > + * scheduler to do its job, via calling vcpu_migrate(). > > + */ > > + if ( unlikely(system_state =3D=3D SYS_STATE_suspend) ) > > + { > > + /* > > + * The boot pcpu is, usually, pcpu #0, so using cpumas= k_first() > > + * actually helps us to achieve our ultimate goal quic= ker. > > + */ > > + cpumask_andnot(&online_affinity, &cpu_online_map, cpum= ask_of(cpu)); >=20 > What about an ASSERT/BUG regarding a non-empty online_affinity? >=20 Sounds good. Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-1QHQPBGO1GEPcWOXdek/ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlWdPiAACgkQk4XaBE3IOsTcTgCdE78HVLE86UT8F1P1t8M6w5/F 6esAnRozrvGLopYk4NNXQJpgDb+cLmdG =yvdt -----END PGP SIGNATURE----- --=-1QHQPBGO1GEPcWOXdek/-- --===============7919274333823216871== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============7919274333823216871==--