Re: Memory-hotplug support for x86_64 domUs?

Re: Memory-hotplug support for x86_64 domUs?

[Jeremy Fitzhardinge]

Radhakrishnan, Prashanth wrote:
> Hi,
>
> I wanted to know if the following would work:
> - memory-hotplug on x86_64 domUs
> - memory-hotplug + ballooning interplay (e.g.
> http://lkml.org/lkml/2008/4/3/449), again on x86_64 domUs
>
> For starts, if I read the Xen linux src (at
> http://xenbits.xensource.com/linux-2.6.18-xen.hg) right, it says that
> SPARSEMEM cannot be enabled with X86_64_XEN. Is that right?
>   

No, sparsemem is supported in pvops Xen kernels (both 32 and 64 bit).  
But it isn't very useful at present.

I have some patches to allow balloon expansion by using memory hotplug 
to add new memory to the system.  But unfortunately it requires some 
changes to the memory hotplug subsystem, and last time I posted them we 
(the memory hotplug maintainers and I) failed to converge on a mutually 
acceptable set of patches.  I put it aside to work on some higher value 
things (x86-64 support, mainly), and it's been sitting in the TODO state 
since.

    J

Re: Memory-hotplug support for x86_64 domUs?

[Jeremy Fitzhardinge]

Radhakrishnan, Prashanth wrote:
> Thanks for the info!
>
>   
>> But it isn't very useful at present.
>>     
>
> Can you please elaborate why. Is it not possible to add memory to VMs
> without rebooting them?
>   

Sorry,  I was unclear.  It's definitely possible in principle to add 
memory to VMs on the fly, but it just isn't implemented in the pvops 
kernels at the moment.  The patch I posted (that you referred to) makes 
the balloon driver call into the memory hotplug subsystem to add new 
memory to the system.  But the new memory is not populated all at once; 
it's just a set of unbacked pageframes to start with.  The memory 
hotplug system automatically puts a file in /sys referring to the newly 
added "memory", and allows you to "online" it all.  Unfortunately, since 
there's no real memory behind the pages, this will almost immediately 
cause your domain to crash.

I'd like to make it so that the balloon driver can control onlining 
pages on a page-by-page basis, and there should be no usermode interface 
for this memory.  But so far we haven't worked out mutually acceptible 
way to do this.

I could post the straightforward balloon/hotplug patches, but I'm not 
comfortable doing so when they leave such an dangerous pit for people to 
fall into.

    J