Re: [PATCH 6 of 8 [RFC]] libxc: introduce xc_domain_move_memory

[Juergen Gross <juergen.gross@ts.fujitsu.com>]

On 09.04.2013 08:56, Dario Faggioli wrote:
> On mar, 2013-04-09 at 06:23 +0100, Juergen Gross wrote:
>> On 09.04.2013 04:49, Dario Faggioli wrote:
>>> as a mechanism of deallocating and reallocating (immediately!) _all_
>>> the memory of a domain. Notice it relies on the guest being suspended
>>> already, before the function is invoked.
>>
>> Is this solution intended to be the final one?
>>
> Well, the idea of sharing the patches, even at this stage, was right
> about discussing that! :-P
>
>> This might be okay for a domain with less than 1GB of memory, but I see
>> problems for really huge domains. The needed time to copy the memory might
>> result in long offline times.
>>
> I see what you mean. I thought about approaches that copy only a
> specific part of the memory, perhaps according to some usage statistics.
>
> I've not yet abandoned that idea, but I honestly think that, if we go
> through the suspend-copy-resume (which is pretty much the only thing I
> can do with PV guests, isn't it?), that can't be for a page or two, or
> the impact of the overhead would be even higher!
>
>> For this case something like live migration
>> (optional?) would be a better solution, I think.
>>
> Well, I thought about that too, and I'm open to
> thinking/discussing/hearing suggestions about how to implement a "live
> phase" for this.
>
> The problem is, with a more migration-alike approach, I'll end up
> doubling the memory requirements of, potentially, all the domains (since
> I'd need space for storing the full RAM image of each one!), which I
> don't think it is an acceptable requirement either, _especially_ for big
> guests, is it? :-(

What about the following approach:

- do the migration in chunks (like 1GB, may be configurable)
- don't move pages which are already on one of the target nodes
- try to allocate memory on the target node while the domain is still running.
   If this fails, there is no need to move that chunk. Depending on the page
   size requirements (huge pages) decide whether the move is aborted or done
   partially.
- in case of successful allocation suspend the domain, do the copy and update
   page tables for the copied pages, then resume the domain
- free the memory chunk on the old node(s)
- repeat until either no memory obtained or move is finished

This will have higher overhead, but the domain will be suspended for only
short periods of time. The memory requirements don't matter, as the additional
memory will be allocated only for a short period of time. Additionally this
approach is more secure, as the domain can't end in suspended state without
memory (you don't have to avoid ballooning or creation of other domains during
the move).


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
PBG PDG ES&S SWE OS6                   Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html