Re: [PATCH 0/3] KVM-userspace: add NUMA support for guests

[Avi Kivity <avi@redhat.com>]

Andi Kleen wrote:
>> Please explain.  When would you call getcpu() and what would you do at 
>> that time?
>>     
>
> When the guest allocates on the node of its current CPU get memory on
> the node pool getcpu() tells you it is running on. More tricky 
> is handling guest explicitely accessing other node for NUMA policy
> purposes, but in this case you can access the cache of the getcpu
> information of other vcpus. 
>   

The guest allocates when it touches the page for the first time.  This 
means very little since all of memory may be touched during guest bootup 
or shortly afterwards.  Even if not, it is still a one-time operation, 
and any choices we make based on it will last the lifetime of the guest.

> This is roughly equivalent of getting a fresh new demand fault page,
> but doesn't require to unmap/free/remap.
>   

Lost again, sorry.

> The tricky bit is probably figuring out what is a fresh new page for
> the guest. That might need some paravirtual help.
>   

The guest typically recycles its own pages (exception is ballooning).  
Also it doesn't make sense to manage this on a per page basis as the 
guest won't do that.  We need to mimic real hardware.

The static case is simple.  We allocate memory from a few nodes (for 
small guests, only one) and establish a guest_node -> host_node 
mapping.  vcpus on guest node X are constrained to host node according 
to this mapping.

The dynamic case is really complicated.  We can allow vcpus to wander to 
other cpus on cpu overcommit, but need to pull them back soonish, or 
alternatively migrate the entire node, taking into account the cost of 
the migration, cpu availability on the target node, and memory 
availability on the target node.  Since the cost is so huge, this needs 
to be done on a very coarse scale.

I don't see this happening in the kernel anytime soon.

-- 
error compiling committee.c: too many arguments to function