From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Dingwall Subject: Re: Kernel 3.11 / 3.12 OOM killer and Xen ballooning Date: Wed, 29 Jan 2014 14:45:24 +0000 Message-ID: <52E91404.30602@zynstra.com> References: <52A602E5.3080300@zynstra.com> <20131209214816.GA3000@phenom.dumpdata.com> <52A72AB8.9060707@zynstra.com> <20131210152746.GF3184@phenom.dumpdata.com> <52A812B0.6060607@oracle.com> <52A89334.3090007@zynstra.com> <52B18F44.2030500@oracle.com> <52B3443F.5060704@zynstra.com> <52B3B6D7.50606@oracle.com> <52BBEBEF.8040509@zynstra.com> <52C50661.7060900@oracle.com> <52CBC700.1060602@zynstra.com> <52CE7E67.5080708@oracle.com> <52D64B87.6000400@zynstra.com> <52D69E0B.5020006@oracle.com> <52D6B8B6.5070302@zynstra.com> <52D7346A.3000300@oracle.com> <52E7E594.2050104@zynstra.com> <52E911CA.9020700@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <52E911CA.9020700@oracle.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Bob Liu Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org Bob Liu wrote: > On 01/29/2014 01:15 AM, James Dingwall wrote: >> Bob Liu wrote: >>> I have made a patch by reserving extra 10% of original total memory, by >>> this way I think we can make the system much more reliably in all cases. >>> Could you please have a test? You don't need to set >>> selfballoon_reserved_mb by yourself any more. >> I have to say that with this patch the situation has definitely >> improved. I have been running it with 3.12.[78] and 3.13 and pushing it >> quite hard for the last 10 days or so. Unfortunately yesterday I got an > Good news! > >> OOM during a compile (link) of webkit-gtk. I think your patch is part >> of the solution but I'm not sure if the other bit is simply to be more >> generous with the guest memory allocation or something else. Having >> tested with memory = 512 and no tmem I get an OOM with the same >> compile, with memory = 1024 and no tmem the compile completes ok (both >> cases without maxmem). As my domains are usually started with memory = >> 512 and maxmem = 1024 it seems that there should be sufficient with my > But I think from the beginning tmem/balloon driver can't expand guest > memory from size 'memory' to 'maxmem' automatically. I am carrying this patch for libxl (4.3.1) because maxmem wasn't being honoured. diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 356f920..fb7965d 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap); libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap); - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT); xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); state->store_domid = xs_domid ? atoi(xs_domid) : 0; free(xs_domid); > >> default parameters. Also for an experiment I set memory=1024 and removed >> maxmem and when tmem is activated I see "[ 3393.884105] xen:balloon: >> reserve_additional_memory: add_memory() failed: -17" printed many times >> in the guest kernel log. >> > I'll take a look at it. It seems possible that this could be the same cause as for the message being printed in dom0 and which I reported in http://lists.xen.org/archives/html/xen-devel/2012-12/msg01607.html and for which no fix seems to have made it to the kernel. I'm still working around this with: #!/bin/sh CURRENT_KB="/sys/bus/xen_memory/devices/xen_memory0/info/current_kb" TARGET_KB="/sys/bus/xen_memory/devices/xen_memory0/target_kb" CKB=$(< "${CURRENT_KB}") TKB=$(< "${TARGET_KB}") if [ "${TKB}" -gt "${CKB}" ] ; then echo "Resizing dom0 memory balloon target to ${CKB}" echo "${CKB}" > "${TARGET_KB}" fi Thanks, James -- *James Dingwall* Script Monkey zynstra-signature-logo twitter-black linkedin-black Zynstra is a private limited company registered in England and Wales (registered number 07864369). Our registered office is 5 New Street Square, London, EC4A 3TW and our headquarters are at Bath Ventures, Broad Quay, Bath, BA1 1UD.