From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: [PATCH 0/4] fix freemem loop Date: Fri, 6 Mar 2015 10:45:01 +0000 Message-ID: <1425638701.14353.38.camel@citrix.com> References: <1425577775.14353.1.camel@citrix.com> <1425618783.5Ob3InrGpG@mlatimer1.dnsdhcp.provo.novell.com> <1425635204.14353.22.camel@citrix.com> <1425636822.14353.32.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Stefano Stabellini Cc: xen-devel@lists.xensource.com, Wei Liu , Mike Latimer , Ian Jackson , xen-devel@lists.xen.org, Jim Fehlig List-Id: xen-devel@lists.xenproject.org On Fri, 2015-03-06 at 10:22 +0000, Stefano Stabellini wrote: > On Fri, 6 Mar 2015, Ian Campbell wrote: > > On Fri, 2015-03-06 at 09:52 +0000, Stefano Stabellini wrote: > > > On Fri, 6 Mar 2015, Ian Campbell wrote: > > > > On Thu, 2015-03-05 at 21:08 -0700, Mike Latimer wrote: > > > > > On Thursday, March 05, 2015 05:49:35 PM Ian Campbell wrote: > > > > > > On Tue, 2015-03-03 at 11:08 +0000, Stefano Stabellini wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > this patch series fixes the freemem loop on machines with very large > > > > > > > amount of memory, where the current wait time is not enough. > > > > > > > > > > > > > > In order to be able to handle arbitrarly large amount of ram, we > > > > > > > implement in libxl_wait_for_memory_target a policy of waiting until dom0 > > > > > > > is making progress. > > > > > > > > > > > > What is the impact of the libxl API change made here on other callers, > > > > > > in particular libvirt? > > > > > > > > I should have CCd Jim when I asked the question. Now done. > > > > > > > > > This change will have one interesting effect on libvirt. Currently, > > > > > libxlDomainFreeMem loops 3 times, then returns 0 (if no errors are > > > > > encountered). This means that domain creation starts before dom0 finishes > > > > > ballooning (unlike xl's previous behavior, which would fail). > > > > > > > > > > With this change, domain creation through virsh will wait (in > > > > > libxl_wait_for_memory_target) until dom0 finishes ballooning. This should > > > > > result in an increase in the speed of the domain starting, as there will not > > > > > be memory contention between the two processes. > > > > > > > > > > The libvirt side calls libxl_wait_for_memory_target with a 1 second timeout - > > > > > which doesn't leave a huge amount of room for slow memory allocation. This > > > > > timeout, as well as the logic in general, should be changed to match the new > > > > > xl behavior (IMO). I expect this to really only matter when dealing with large > > > > > domains. > > > > > > > > For libvirt we also need to consider what happens when a libvirt which > > > > is modified along these lines uses an older libxl (I believe back to 4.2 > > > > is supported). > > > > > > In the libvirt case even if libvirt calls libxl_wait_for_memory_target > > > with just 1 second timeout, it is likely going to end up waiting longer > > > than that due to the change in behaviour of the function. > > > > > > What I am saying is that even if we don't modify libvirt, we are going to > > > get a more reliable and more stable behaviour than what we have now. > > > > > > That said, I agree that it would be good to improve the freemem loop in > > > libvirt in a similar way to what we are doing with xl. > > > > This is missing my point. > > > > We need to consider the case of a modified libvirt with the old, broken, > > libxl too. > > For that case, we might want to make sure that the new timeout passed to > libxl_wait_for_memory_target is not lower than the overall libvirt > timeout today (libxl_wait_for_free_memory+libxl_wait_for_memory_target). OK, sounds like we may not need a #define then. But perhaps you could update the comment above the prototypes of these functions in libxl.h which explains how to use the interface such that it works well with new libxl and at least tolerably with old libxl and (if it is different) how to use it so it works super-well with new libxl if you don't care about old libxl. Ian.