From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [RFC PATCH 3/7] xen: psr: reserve an RMID for each core Date: Wed, 8 Apr 2015 14:03:34 +0000 Message-ID: <1428501089.5671.150.camel@citrix.com> References: <20150404020423.22875.23590.stgit@Solace.station> <20150404021441.22875.9924.stgit@Solace.station> <55252D09.6070307@eu.citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3585942311268009259==" Return-path: In-Reply-To: <55252D09.6070307@eu.citrix.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: George Dunlap Cc: Wei Liu , Ian Campbell , Andrew Cooper , "xen-devel@lists.xen.org" , "dongxiao.xu@intel.com" , "JBeulich@suse.com" , "chao.p.peng@linux.intel.com" List-Id: xen-devel@lists.xenproject.org --===============3585942311268009259== Content-Language: en-US Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-8bHCUwl2LU10nIXcWKIC" --=-8bHCUwl2LU10nIXcWKIC Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2015-04-08 at 14:28 +0100, George Dunlap wrote: > On 04/04/2015 03:14 AM, Dario Faggioli wrote: > > This allows for a new item to be passed as part of the psr=3D > > boot option: "percpu_cmt". If that is specified, Xen tries, > > at boot time, to associate an RMID to each core. > >=20 > > XXX This all looks rather straightforward, if it weren't > > for the fact that it is, apparently, more common than > > I though to run out of RMID. For example, on a dev box > > we have in Cambridge, there are 144 pCPUs and only 71 > > RMIDs. >=20 > Is that because you have 2 sockets? >=20 It has 4 sockets. Chao explained in one of his mails that there usually is 2 or more RMIDs per hardware thread. > There's no need to keep RMIDs unique across sockets, is there? E.g., > socket 0 cpu 0 and socket 1 cpu 0 can have the same RMID, because cache > and the MSRs are per-socket. >=20 Exactly. And in fact, I just added to my TODO list improving Xen's current PSR support to take per-socketness of RMIDs correctly into account. > If we're doing things on a per-domain basis, having the same RMID > allocated for each socket sort of makes sense; but even then, if you > know a domain is only going to run on a given socket, there's no reason > in theory we couldn't use same RMID for a different domain on the other > socket (assuming it was only going to run on the other socket). >=20 Right now CMT is per-domain and, on a box with 71 available RMIDs _per_each_socket_, we have (in Xen) an array of 71 possible RMIDs (72, but RMID 0 is treated specially) to be assigned to domains. Independently on where a domain will ever run, we can use separate arrays, and each domain can have an RMID on each socket. If, as you say, there are well known restrictions, a domain can avoid having RMIDs for certain sockets. In the worst case (all domains can run everywhere), we use the same amount of RMIDs, in better/best cases, we use less of them. This is even more important when looking at per-pCPU CMT configurations. In fact, right now, on that box, I have more than 71 pCPUs, so I can't associate an RMID to all the pCPUs. However, with the proper per-socket support implemented, I not only will be able to associate an RMID to all the pCPUs, but I'll have 35 RMIDs on each socket free. :-) > One advantage of doing things of a per-vcpu level is that you wouldn't > have to worry about inter-socket RMID issues. > Sorry, but I'm lost again, I'm afraid. What do you mean with this? Thanks and Regards, Dario --=-8bHCUwl2LU10nIXcWKIC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlUlMmEACgkQk4XaBE3IOsTyLwCglqe9rrYz1NYw64jyMpxCWPpN A44AnRGxBQ5Le2aHud6TUUcb+tdVo+Tv =7n9j -----END PGP SIGNATURE----- --=-8bHCUwl2LU10nIXcWKIC-- --===============3585942311268009259== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============3585942311268009259==--