Re: [Hackathon minutes] PV frontends/backends and NUMA machines

[George Dunlap <george.dunlap@eu.citrix.com>]

On 21/05/13 14:43, Dario Faggioli wrote:
> On mar, 2013-05-21 at 12:47 +0100, Jan Beulich wrote:
>>>>> On 21.05.13 at 12:58, Dario Faggioli <raistlin@linux.it> wrote:
>>> Well, sure, but then, again, how do you control which (and not only how
>>> much) memory is taken from which node?
>> Hmm, I may not have followed, but why is "which" important here at
>> all? The only (usual) restriction should apply regarding preservation of
>> memory below 4G.
>>
> It is if you want Dom0 to think it is running, say, on 2 nodes and
> actually have the memory, say, in the range 0-1G accessed quicker from
> d0v0 (vcpu0 of Dom0), and the vice versa with memory within 1-2G and
> d0v1.
>
> That enables NUMA optimization _inside_ Dom0, like the pinning of the
> backends and all the other stuff discussed (during the Hackathon and) in
> this thread.
>
> However, to do that, we, I think, need to be able not only to specify
> that we want 1G worth of memory on one specific node, but also to
> request explicitly for some of Dom0's PFN to be here and for some others
> to be there, as we were saying earlier in the thread with Tim.

One thing that I wanted to add to this discussion -- unless there's some 
way for the toolstack to figure out, for each node, how much memory is 
currently free *and* how much memory could be freed by dom0 on that 
node, and a way to ask dom0 to free memory from a specific a node, then 
booting with dom0 having all the memory is basically going to make all 
of our NUMA work a noop.

We may end up having to switch from defaulting to giving dom0 and 
autoballooning to giving dom0 a fixed amount.

  -George