From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [Hackathon minutes] PV frontends/backends and NUMA machines Date: Tue, 21 May 2013 11:53:27 +0200 Message-ID: <1369130007.12423.22.camel@Solace> References: <20130521083251.GD9626@ocelot.phlegethon.org> <20130521092003.GE9626@ocelot.phlegethon.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3809598539475649573==" Return-path: In-Reply-To: <20130521092003.GE9626@ocelot.phlegethon.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Tim Deegan Cc: George Dunlap , "xen-devel@lists.xensource.com" , Stefano Stabellini List-Id: xen-devel@lists.xenproject.org --===============3809598539475649573== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-b3cnFb5Td9EuxSUcHv6C" --=-b3cnFb5Td9EuxSUcHv6C Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On mar, 2013-05-21 at 10:20 +0100, Tim Deegan wrote: > At 09:47 +0100 on 21 May (1369129629), George Dunlap wrote: > > The second is to make the pfn -> NUMA node layout reasonable. At the > > moment, as I understand it, pfns will be striped across nodes. In > > theory dom0 could deal with this, but it seems like in practice it's > > going to be nasty trying to sort that stuff out. It would be much > > better, if you have (say) 4 nodes and 4GiB of memory assigned to dom0, > > to have pfn 0-1G on node 0, 1-2G on node 2, &c. >=20 > Yeah, I can see that fixing that post-hoc would be a PITA. =20 > Indeed! :-P > I guess if > you figure out the vcpu assignments at dom0-build time, the normal NUMA > memory allocation code will just DTRT (since that's what you'd want for > a comparable domU)? >=20 Well, we need to check what actually happens deeper, but I don't think that is going to be enough, and that is true for DomUs as well. In fact, what we have right now (for DomUs) is: memory is allocated from a subset of the host nodes and vCPUs pefers to be scheduled on the pCPUs of those nodes. However, what a true 'guest NUMA awareness' prescribes is that you know what memory "belongs to" (i.e., is accessed quicker from) each vCPU, and this is something we don't have. So, yes, I think creating a node-affinity for Dom0 early enough would be a reasonable first step, and would already help quite a bit. However, that would just mean that we'll (going back to George's example) get 1G of memory from node0, 1G of memory from node1, etc. What we want is to force pfns 0-1G to be actually allocated out of node0, and so on and so forth... And that is something that I don't think the current code can guarantee. Dario --=-b3cnFb5Td9EuxSUcHv6C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) iEYEABECAAYFAlGbRBcACgkQk4XaBE3IOsTWyACdHrrjzcuG3yJJ3bjze95eCflg FkYAnRkss/5mqjblKDsMVPeax0oKYqX7 =/bhr -----END PGP SIGNATURE----- --=-b3cnFb5Td9EuxSUcHv6C-- --===============3809598539475649573== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============3809598539475649573==--