Re: [PATCH 6 of 8 [RFC]] libxc: introduce xc_domain_move_memory

[Juergen Gross <juergen.gross@ts.fujitsu.com>]

On 09.04.2013 10:51, Dario Faggioli wrote:
> On mar, 2013-04-09 at 09:13 +0100, Juergen Gross wrote:
>> What about the following approach:
>>
> In general, I like it... More details below.
>
>> - do the migration in chunks (like 1GB, may be configurable)
>>
> Yes, provided these chunks are big enough, I think the overhead of is
> acceptable.
>
>> - don't move pages which are already on one of the target nodes
>>
> Yep, that is definitely sane, and was already on my TODO list (although,
> you're right, I forgot to mention it in the cover or in the various
> changelogs). It's not there yet because I'm missing a way of knowing on
> what node a page is, but I'm already working of putting it together.
>
> Anyway, I agree on this too, and thanks for pointing that out. :-)
>
>> - try to allocate memory on the target node while the domain is still running.
>>     If this fails, there is no need to move that chunk. Depending on the page
>>     size requirements (huge pages) decide whether the move is aborted or done
>>     partially.
>> - in case of successful allocation suspend the domain, do the copy and update
>>     page tables for the copied pages, then resume the domain
>>
> This is also fine, the only issue being that I'd probably need to fiddle
> with the domain max_mem, and stuff like that, wouldn't I? I'm saying
> this because, when testing the few that I sent already, I run right into
> this when I was trying to do it in the allocate-copy-deallocate order
> (of course, depending on how big a chunk is, but this is going to be
> much less than 1GB!).

There might be 1GB huge pages which have to be copied at once (especially for
PV-domains). Doing a migration to another node for performance reasons and
losing huge page advantages at the same time seems to be a bad idea. :-)

> Do you see what I mean? Do you think it would be nice to increase the
> domain's "memory allowance" (temporarily, of course) for this to be
> possible?

Would make sense, I think. :-)

>
>> - free the memory chunk on the old node(s)
>> - repeat until either no memory obtained or move is finished
>>
>> This will have higher overhead, but the domain will be suspended for only
>> short periods of time. The memory requirements don't matter, as the additional
>> memory will be allocated only for a short period of time.
>>
> Yep, this all makes sense, with the only nit being the max_mem issue
> above.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
PBG PDG ES&S SWE OS6                   Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html