From mboxrd@z Thu Jan 1 00:00:00 1970 From: George Dunlap Subject: Re: [Hackathon minutes] PV frontends/backends and NUMA machines Date: Fri, 24 May 2013 17:00:32 +0100 Message-ID: <519F8EA0.8080301@eu.citrix.com> References: <20130521083251.GD9626@ocelot.phlegethon.org> <519B632902000078000D79C4@nat28.tlf.novell.com> <1369132208.12423.41.camel@Solace> <519B6BFE02000078000D7A5A@nat28.tlf.novell.com> <1369133894.12423.52.camel@Solace> <519B7ADF02000078000D7AF9@nat28.tlf.novell.com> <1369143809.12423.85.camel@Solace> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1369143809.12423.85.camel@Solace> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Dario Faggioli Cc: "xen-devel@lists.xensource.com" , Tim Deegan , Jan Beulich , Stefano Stabellini List-Id: xen-devel@lists.xenproject.org On 21/05/13 14:43, Dario Faggioli wrote: > On mar, 2013-05-21 at 12:47 +0100, Jan Beulich wrote: >>>>> On 21.05.13 at 12:58, Dario Faggioli wrote: >>> Well, sure, but then, again, how do you control which (and not only how >>> much) memory is taken from which node? >> Hmm, I may not have followed, but why is "which" important here at >> all? The only (usual) restriction should apply regarding preservation of >> memory below 4G. >> > It is if you want Dom0 to think it is running, say, on 2 nodes and > actually have the memory, say, in the range 0-1G accessed quicker from > d0v0 (vcpu0 of Dom0), and the vice versa with memory within 1-2G and > d0v1. > > That enables NUMA optimization _inside_ Dom0, like the pinning of the > backends and all the other stuff discussed (during the Hackathon and) in > this thread. > > However, to do that, we, I think, need to be able not only to specify > that we want 1G worth of memory on one specific node, but also to > request explicitly for some of Dom0's PFN to be here and for some others > to be there, as we were saying earlier in the thread with Tim. One thing that I wanted to add to this discussion -- unless there's some way for the toolstack to figure out, for each node, how much memory is currently free *and* how much memory could be freed by dom0 on that node, and a way to ask dom0 to free memory from a specific a node, then booting with dom0 having all the memory is basically going to make all of our NUMA work a noop. We may end up having to switch from defaulting to giving dom0 and autoballooning to giving dom0 a fixed amount. -George