From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [RFC 0/5] xen/arm: support big.little SoC Date: Thu, 22 Sep 2016 10:43:03 +0200 Message-ID: <1474533783.4393.312.camel@citrix.com> References: <20160919083619.GA16854@linux-7smt.suse> <97d77bdb-2f4e-e89a-95b9-8aacb56eebc0@suse.com> <1474305482.4393.42.camel@citrix.com> <1474325742.4393.78.camel@citrix.com> <1474332846.4393.153.camel@citrix.com> <20160920100331.GB8084@linux-u7w5.ap.freescale.net> <4c52141f-a6a4-a0b1-dced-f799b592481e@arm.com> <61196660-df7c-7324-2fb6-cfb11f44ea1e@arm.com> <39623498-bb30-4ff7-f075-219487a5afbb@arm.com> <6bd7d587-f9ba-c3bf-db96-46a2958d9e5b@arm.com> <1474472740.4393.281.camel@citrix.com> <75979980-985e-51c8-9331-ad9657e4d612@arm.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2526875085822701305==" Return-path: In-Reply-To: <75979980-985e-51c8-9331-ad9657e4d612@arm.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Julien Grall , George Dunlap , Stefano Stabellini Cc: Juergen Gross , Peng Fan , Steve Capper , George Dunlap , Andrew Cooper , Punit Agrawal , "xen-devel@lists.xen.org" , Jan Beulich , Peng Fan List-Id: xen-devel@lists.xenproject.org --===============2526875085822701305== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-t2Serh0rwl0Zv78QieYt" --=-t2Serh0rwl0Zv78QieYt Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2016-09-21 at 20:28 +0100, Julien Grall wrote: > On 21/09/2016 16:45, Dario Faggioli wrote: > > This does not seem to match with what has been said at some point > > in > > this thread... And if it's like that, how's that possible, if the > > pcpus' ISAs are (even only slightly) different? >=20 > Right, at some point I mentioned that the set of errata and features=C2= =A0 > will be different between processor. >=20 Yes, I read that, but wasn't (and still am not) sure about whether or not that meant a vcpu can move freely between classes or not, in the way that the scheduler does that. In fact, you say: > With a bit of work in Xen, it would be possible to do move the vCPU=C2=A0 > between big and LITTLE cpus. As mentioned above, we could sanitize > the=C2=A0 > features to only enable a common set.=20 > You can view the big.LITTLE=C2=A0 > problem as a local live migration between two kind of CPUs. >=20 Local migration basically --from the vcpu perspective-- means create a new vcpu, stop the original vcpu, copy the state from original to new, destroy the original vcpu and start the new one. My point is that this is not something that can be done within nor initiated by the scheduler, e.g., during a context switch or a vcpu wakeup! And I'm saying this because... > In your suggestion you don't mention what would happen if the guest=C2=A0 > configuration does not contain the affinity. Does it mean the vCPU > will=C2=A0 > be scheduled anywhere? A pCPU/class will be chosen randomly? >=20 ...in my example there were vcpus for which no set of classes was specified, and I said that it meant those vcpus can run on any pcpu of any class. And this would be what I think we should do even in cases where no "vcpuclass" parameter is specified at all. *BUT* that is only possible if moving a vcpu from a pcpu of class A to a pcpu of class B does *NOT* require the steps described above, similar to local migration. IOW, this is only possible if moving a vcpu from a pcpu of class A to a pcpu of class B *ONLY* requires a context switch. If changing class requires local migration, the scheduler must be told that he should never move vcpus between classes (or set of classes made by homogeneous enough vcpus for which a context switch is sufficient). If changing class is --or can be made to be, with some work in Xen-- just a context switch, then we can have the scheduler moving vcpus between (set of) classes. It's probably not too big of a deal, wrt the end result (see below), but it changes the implementation a lot. But, yeah, if changing class can be made simple with some work in Xen, but is not simple/possible **right now**, then this means that, _for_now_, vcpus for which a class is not specified must be assigned to a class (or a set of classes within which the scheduler can freely move vcpus). In future, we can change this, broadening the "default class" as much as seamless migration within its pcpus allows that. Hope I made myself clear enough. :-D > To be honest, I quite like this idea.=20 > :-) > It could be used as soft/hard=C2=A0 > affinity for the moment. But can be extended in the future if/when > the=C2=A0 > scheduler gain knowledge of power efficiency and vCPU can migrate=C2=A0 > between big and LITTLE. >=20 Yes, exactly, and this is, I think, true in both of the above outlined cases. What I meant when I said it is the implementation, rather than the end result that changes, is that: =C2=A0- if complex migration-alike operations are necessary for changing=C2= =A0 =C2=A0 =C2=A0class, migrating between classes (e.g., between big and LITTLE= )=C2=A0 =C2=A0 =C2=A0will have to happen, e.g., in a load and energy management and =C2=A0 =C2=A0balancing component implemented above the scheduler itself =C2=A0- if just plain context switch is enough, the scheduler can do =C2=A0 =C2=A0everything by itself. But yes, in any case, the model we're coming up with looks to be a very good starting point, because it is orthogonal to and independent from other components and solution (e.g., cpupools) and is pretty simple and basic, and leaves room for future extensions. Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-t2Serh0rwl0Zv78QieYt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJX45mYAAoJEBZCeImluHPuh/0QAJrZMpw7/3prQ1AkXKUD9pSr NbNFiPxtkF/kF9B+Do6lefF83gnwv8uLamGkGXhpHjlCtVXDHWEl4Y2+wQFHwUrW M9AMMorKLPl9QFn4xoJhR99pXtSZN/IZ4H5yTpqMKpN5k/dSbLPs06LHoVdLy+mt pS4IunNpiera1MbfD+WO9Sv48SpqPXhnVdSTf5DHKIpO773S31ZTQV+tHuEdywF0 HRvAzrXfOVf7Y4e6AiZeWQK7JHpKCFidbgJqPBSo5QgR6RDt2XipI3uxEl0gaSVs tjzcmDYG4yrkuA+X4E84POlzmufORAQ5VmCKOIz8Z9sTlByagTHXosdWgANze0OA TqXtYKIaIz/jhlYcb5+v0q+LejXjoocqR/hM+IQl2RY+WbcwTJCJWrD86uraMaO8 ySG0h/Ime+Vv1gLH9JEBw3XMrbWICmzCk6sbR1DTuDCuhnN5n9CJEUgaHobBEyju R91My/DHUEjt1Y7PT7oY0CU35YMk0mZ3vm7vb+F7XW8BMZ3GXXyy3VFgJEwvbcgb H4xXTdeB7rdE+krO87P9t6h2idYjVX/ftFyEo8PSejMFdV4j3wWTltqT7lss8We/ M5L0FwwvKSd152UUtqooKmMmD+rJCVxRf8Jwvy2qQltckdyMcgI27HmNYhOHMDax wFla8ZqyZ1McAF375Mts =CU1V -----END PGP SIGNATURE----- --=-t2Serh0rwl0Zv78QieYt-- --===============2526875085822701305== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============2526875085822701305==--