[PATCH] libxl: document memory parameters and behaviour

[PATCH] libxl: document memory parameters and behaviour

[Ian Jackson]

commit a6b0f1e47447b7da2e838f9f68caf2019b177c36
Author: Ian Jackson <ian.jackson@eu.citrix.com>
Date:   Wed Sep 14 10:34:54 2011 +0100

    libxl: document memory parameters and behaviour
    
    Introduce a new comment in libxl.h, describing the various guest
    memory usage parameters.
    
    Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index d146575..6999d54 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -349,8 +349,62 @@ int libxl_domain_unpause(libxl_ctx *ctx, uint32_t domid);
 
 int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char *filename);
 
+/*
+ * Domains have three memory parameters:
+ *
+ *   * Target ("target").  This is the request from the toolstack to
+ *     the guest's balloon system.  With a cooperative guest, the
+ *     guest will adjust its memory usage towards this target (by
+ *     requesting more memory from Xen, or returning memory to Xen, as
+ *     applicable).
+ *
+ *   * Limit ("maxmem" while the domain is running; set to the same as
+ *     "target_memkb" on creation).  Xen will not allow the guest to
+ *     increase its allocation (by requesting more memory from Xen) if
+ *     the guest would then be using more memory than this.  However,
+ *     getting memory back from the guest requires the cooperation of
+ *     the guest, so if the limit has previously been reduced below
+ *     the guest's current usage, the guest's actual usage may remain
+ *     above the limit.
+ *
+ *   * Boot-time memory maximum (the "max_memkb" domain creation
+ *     parameter).  This is the absolute maximum amount of memory the
+ *     domain will ever be able to allocate.  (In theory a balloon
+ *     driver capable of memory hotplug might be able to surpass this
+ *     limit but currently the toolstack does not support this.)
+ *
+ * Domains actually use a bit more memory than they see directly; all
+ * of these parameters in the libxl API are specified in terms of
+ * guest-visible memory.  The actual guest memory usage will be
+ * somewhat higher because some is needed for infrastructure purposes
+ * related to the domain (Xen heap entries, video memory offered to
+ * hvm guests, memory for a device model stub domain, etc.).  The
+ * amount of memory which would actually be necessary for a particular
+ * guest can be discovered with the libxl_domain_need_memory call.
+ *
+ * Note that when a guest's target is changed, while in principle the
+ * guest's actual memory usage is supposed to eventually reach the new
+ * target, this may not ever happen.  The guest may not have a
+ * suitable balloon driver, or it may not be sufficiently cooperative
+ * (eg, the guest admin may have arranged for it not to honour
+ * requests to use less memory), or it may be under severe memory
+ * pressure and not able to comply.  Even requests to increase memory
+ * usage may not be honoured, for example because the increase would
+ * take the guest beyond a compiled-in or boot-time limit.
+ *
+ * It is not proper for the limit to be less than the target.
+ *
+ * When creating an HVM guest, if target is less than max_memkb,
+ * populate-on-demand is used.  In this situation the guest must have
+ * a balloon driver which must reclaim enough p-o-d guest memory
+ * quickly enough, or the guest will crash during boot.
+ */
+
 int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb);
 int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, int32_t target_memkb, int relative, int enforce);
+  /* if enforce is set, sets the target too.  NB that there is a bug here,
+   * in that if you reduce the target with enforce=1 and then increase it
+   * again with enforce=0, the limit will now be less than the target. */
 int libxl_get_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t *out_target);
 /* how much free memory in the system a domain needs to be built */
 int libxl_domain_need_memory(libxl_ctx *ctx, libxl_domain_build_info *b_info,

Re: [PATCH] libxl: document memory parameters and behaviour

[Olaf Hering]

On Wed, Sep 14, Ian Jackson wrote:

> commit a6b0f1e47447b7da2e838f9f68caf2019b177c36
> Author: Ian Jackson <ian.jackson@eu.citrix.com>
> Date:   Wed Sep 14 10:34:54 2011 +0100
> 
>     libxl: document memory parameters and behaviour
>     
>     Introduce a new comment in libxl.h, describing the various guest
>     memory usage parameters.

Thanks for documenting this.

How do the three describe values relate to xc_dominfo_t->nr_pages?
The comment there suggests that nr_pages could be even higher at some
point. Is the max number of pages a domain can allocate
(xc_dominfo_t->max_memkb / 1024)?
If so, then xenpaging should not use nr_pages but max_memkb.

Olaf

Re: [PATCH] libxl: document memory parameters and behaviour

[Olaf Hering]

On Wed, Sep 14, Olaf Hering wrote:

> On Wed, Sep 14, Ian Jackson wrote:
> 
> > commit a6b0f1e47447b7da2e838f9f68caf2019b177c36
> > Author: Ian Jackson <ian.jackson@eu.citrix.com>
> > Date:   Wed Sep 14 10:34:54 2011 +0100
> > 
> >     libxl: document memory parameters and behaviour
> >     
> >     Introduce a new comment in libxl.h, describing the various guest
> >     memory usage parameters.
> 
> Thanks for documenting this.
> 
> How do the three describe values relate to xc_dominfo_t->nr_pages?

Sorry.  I did not notice that xc_domaininfo_t is actually
xen_domctl_getdomaininfo_t, not xc_dominfo_t from tools/libxc/xenctrl.h.


Olaf