From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH 1/7] xen: vNUMA support for PV guests Date: Fri, 22 Nov 2013 19:29:34 +0100 Message-ID: <1385144974.21426.64.camel@Solace> References: <1381963072-7703-1-git-send-email-ufimtseva@gmail.com> <525FC46D02000078000FBB6A@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6352128726617580248==" Return-path: Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VjvU4-0001HL-TO for xen-devel@lists.xenproject.org; Fri, 22 Nov 2013 18:29:41 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Li Yechen Cc: Keir Fraser , Jan Beulich , Stefano Stabellini , George Dunlap , Matt Wilson , Elena Ufimtseva , xen-devel List-Id: xen-devel@lists.xenproject.org --===============6352128726617580248== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-U70k/nbM587W7OCh5Box" --=-U70k/nbM587W7OCh5Box Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On gio, 2013-11-21 at 18:00 +0800, Li Yechen wrote: > Dario, > I just reply to this email again in case you haven't seen it :) >=20 No, don't worry, I haven't forgot. :-) > On Tue, Oct 22, 2013 at 10:31 PM, Li Yechen > wrote: > Hi Elena, > =20 > Congratulations to your work again! > =20 > =20 > Have you considered the other memory operations in > xen/common/memory.c? > =20 > There are two important function: decrease_reservation(&args) > and populate_physmap(&args) > =20 > decrease_reservation(&args) remove pages from domain. > populate_physmap(&args) alloc pages for domain. > Yes, that's definitely something we need to adress, probably in this patch series, even before thinking about NUMA aware ballooning. > Guest domain pass the mask of nodes to xen by these two > hypercalls.=20 > =20 > For decrease_reservation, xen will also receive a number of > pages. We just free them from domain. Here, we should update > the memory size of vnodes and pnodes > (I think you keep a counter for the page numbers of each vnode > and pnode, something as vnuma_memszs, but please forgive me > that you have submitted such a huge patch that I could not > understand everything in time : - | ) > =20 > For populate_physmap, xen will allocate blank pages from its > heap for domain guest, from specific nodes, according to the > nodemask. Here we should update your counters too! > =20 Well, I haven't gone re-check the code, but that does make sense. In Edinburgh, Elena told me that she did some tests of ballooning with her series applied, and nothing exploded (which is already something. :-D). We definitely should double check what happens, from where the pages came /are taken from, and ensure the accounting is done right. > And as I see, we don't have a protocol here on whether the > nodemask in (&args ) is pnode or vnode. > =20 > I think it should be vnode, since guest domain knows nothing > about the node affinity. > =20 > So my idea could be: we communicate with guest domain using > vnode IDs. If we need to change the memory size of guest > domain, for example, memory increase/decrease on pnode[0], > we . And in the two functions of memory.c mentioned above, we > received the vnode_mask, transfer it back to pnode_mask, thus > it will work perfectly! And we don't need an extra hypercall > for guest domain any more! > =20 Mmm.. it may be me, but I'm not sure I follow. I agree that the guest should speak vnode-s, but I'm not sure I get what you mean when you say "use your node affinity to change pnode[0] to vnodes_mask, pass it to guest domain". Anyway, I was thinking, you did a pretty nice job on hacking something together when for NUMA aware ballooning when vNUMA wasn't even released. Now that Elena's patchset is out, how about you try to adapt what you had at the time, plus the outcome of all that nice discussion we also had, on top of it, and show us what happens? :-) Elena's patches are not in the final form, but that should constitute a fairly decent basis for another, and this time easier to understand and to review, proof of concept implementation, isn't that so? Of course, there's no hurry, this will definitely be something we'll consider for the next release of Xen (so 4.5, not 4.4 which will hopefully be released in January), i.e., there should be plenty of time. :-D What do you think? Thanks and Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-U70k/nbM587W7OCh5Box Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) iEYEABECAAYFAlKPoo4ACgkQk4XaBE3IOsTGNgCgj0G1O7POJhku2WAa9Tf2dlCD BnQAn0QlikmQDQvfEL1yFh/CaaWSDK8+ =O7AV -----END PGP SIGNATURE----- --=-U70k/nbM587W7OCh5Box-- --===============6352128726617580248== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============6352128726617580248==--