Re: [PATCH for 4.6 v3 2/3] xl/libxl: disallow saving a guest with vNUMA configured

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2015-09-10 at 15:50 +0100, Wei Liu wrote:
> This is because the migration stream does not preserve node information.
> 
> Note this is not a regression for migration v2 vs legacy migration
> because neither of them preserve node information.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> Cc: andrew.cooper3@citrix.com
> 
> v3:
> 1. Update manpage, code comment and commit message.
> 2. *Don't* check if nomigrate is set.
> ---
>  docs/man/xl.cfg.pod.5   |  2 ++
>  tools/libxl/libxl_dom.c | 14 ++++++++++++++
>  2 files changed, 16 insertions(+)
> 
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index 80e51bb..555f8ba 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -263,6 +263,8 @@ virtual node.
>  
>  Note that virtual NUMA for PV guest is not yet supported, because
>  there is an issue with cpuid handling that affects PV virtual NUMA.
> +Further more, guest with virtual NUMA cannot be saved or migrated
> +because migration stream does not preserve node information.
>  
>  Each B<VNODE_SPEC> is a list, which has a form of
>  "[VNODE_CONFIG_OPTION,VNODE_CONFIG_OPTION, ... ]"  (without quotes).
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index c2518a3..a4d37dc 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -24,6 +24,7 @@
>  #include <xen/hvm/hvm_info_table.h>
>  #include <xen/hvm/hvm_xs_strings.h>
>  #include <xen/hvm/e820.h>
> +#include <xen/errno.h>
>  
>  libxl_domain_type libxl__domain_type(libxl__gc *gc, uint32_t domid)
>  {
> @@ -1612,6 +1613,7 @@ void libxl__domain_save(libxl__egc *egc,
> libxl__domain_suspend_state *dss)
>      const libxl_domain_remus_info *const r_info = dss->remus;
>      libxl__srm_save_autogen_callbacks *const callbacks =
>          &dss->sws.shs.callbacks.save.a;
> +    unsigned int nr_vnodes = 0, nr_vmemranges = 0, nr_vcpus = 0;
>  
>      dss->rc = 0;
>      logdirty_init(&dss->logdirty);
> @@ -1636,6 +1638,18 @@ void libxl__domain_save(libxl__egc *egc,
> libxl__domain_suspend_state *dss)
>            | (debug ? XCFLAGS_DEBUG : 0)
>            | (dss->hvm ? XCFLAGS_HVM : 0);
>  
> +    /* Disallow saving a guest with vNUMA configured because migration
> +     * stream does not preserve node information.
> +     */
> +    rc = xc_domain_getvnuma(CTX->xch, domid, &nr_vnodes, &nr_vmemranges,
> +                            &nr_vcpus, NULL, NULL, NULL);
> +    assert(rc == -1 && (errno == XEN_ENOBUFS || errno == XEN_EOPNOTSUPP));

Has this been tested with a domain _without_ vnuma config.

Specifically if there is no vnuma config and therefore 0 vnodes and 0
vmemranges will the hypervisor actually return XEN_ENOBUFS rather than
success (because it succeeded to put 0 things into a zero length array).

It looks like the non-zero number of vcpus in the domain will indeed
trigger the ENOBUFS case, but I wanted to check.

Ian.

> +    if (errno == XEN_ENOBUFS && nr_vnodes) {
> +        LOG(ERROR, "Cannot save a guest with vNUMA configured");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
>      dss->guest_evtchn.port = -1;
>      dss->guest_evtchn_lockfd = -1;
>      dss->guest_responded = 0;