From mboxrd@z Thu Jan 1 00:00:00 1970 From: Juergen Gross Subject: Re: [PATCH 6 of 8 [RFC]] libxc: introduce xc_domain_move_memory Date: Tue, 09 Apr 2013 11:16:30 +0200 Message-ID: <5163DC6E.4090800@ts.fujitsu.com> References: <5163A5E4.6030608@ts.fujitsu.com> <1365490591.17022.14.camel@Abyss> <5163CDA2.7030206@ts.fujitsu.com> <1365497480.17022.81.camel@Abyss> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1365497480.17022.81.camel@Abyss> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 09.04.2013 10:51, Dario Faggioli wrote: > On mar, 2013-04-09 at 09:13 +0100, Juergen Gross wrote: >> What about the following approach: >> > In general, I like it... More details below. > >> - do the migration in chunks (like 1GB, may be configurable) >> > Yes, provided these chunks are big enough, I think the overhead of is > acceptable. > >> - don't move pages which are already on one of the target nodes >> > Yep, that is definitely sane, and was already on my TODO list (although, > you're right, I forgot to mention it in the cover or in the various > changelogs). It's not there yet because I'm missing a way of knowing on > what node a page is, but I'm already working of putting it together. > > Anyway, I agree on this too, and thanks for pointing that out. :-) > >> - try to allocate memory on the target node while the domain is still running. >> If this fails, there is no need to move that chunk. Depending on the page >> size requirements (huge pages) decide whether the move is aborted or done >> partially. >> - in case of successful allocation suspend the domain, do the copy and update >> page tables for the copied pages, then resume the domain >> > This is also fine, the only issue being that I'd probably need to fiddle > with the domain max_mem, and stuff like that, wouldn't I? I'm saying > this because, when testing the few that I sent already, I run right into > this when I was trying to do it in the allocate-copy-deallocate order > (of course, depending on how big a chunk is, but this is going to be > much less than 1GB!). There might be 1GB huge pages which have to be copied at once (especially for PV-domains). Doing a migration to another node for performance reasons and losing huge page advantages at the same time seems to be a bad idea. :-) > Do you see what I mean? Do you think it would be nice to increase the > domain's "memory allowance" (temporarily, of course) for this to be > possible? Would make sense, I think. :-) > >> - free the memory chunk on the old node(s) >> - repeat until either no memory obtained or move is finished >> >> This will have higher overhead, but the domain will be suspended for only >> short periods of time. The memory requirements don't matter, as the additional >> memory will be allocated only for a short period of time. >> > Yep, this all makes sense, with the only nit being the max_mem issue > above. Juergen -- Juergen Gross Principal Developer Operating Systems PBG PDG ES&S SWE OS6 Telephone: +49 (0) 89 3222 2967 Fujitsu Technology Solutions e-mail: juergen.gross@ts.fujitsu.com Domagkstr. 28 Internet: ts.fujitsu.com D-80807 Muenchen Company details: ts.fujitsu.com/imprint.html