Re: Problem with IOMEM and domain reboot

[Wei Liu <wei.liu2@citrix.com>]

On Tue, Feb 06, 2018 at 02:44:56PM +0200, Oleksandr Andrushchenko wrote:
>  From aa1f20af73a5a3c8f2c904b857a79334d18d41ff Mon Sep 17 00:00:00 2001
> > > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > Date: Wed, 20 Dec 2017 17:51:18 +0200
> > > Subject: [PATCH] [HACK] Reset iomem's gfn to LIBXL_INVALID_GFN on reboot
> > > 
> > > During domain reboot its configuration is partially reused
> > > to re-create a new domain, but iomem's GFN field for the
> > > iomem is only restored for those memory ranges, which are
> > > configured in form of [IOMEM_START,NUM_PAGES[@GFN], but not for
> > > those in form of [IOMEM_START,NUM_PAGES], e.g. without GFN.
> > > For the latter GFN is reset to 0, but while mapping ranges
> > > to a domain during reboot there is a check that GFN treated
> > > as valid if it is not equal to LIBXL_INVALID_GFN, thus making
> > > Xen to map IOMEM_START to address 0 in the guest's address space.
> > > 
> > > Workaround it by resseting GFN to LIBXL_INVALID_GFN, so xl
> > > can set proper values for mapping on reboot.
> > > 
> > > Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > ---
> > >   tools/libxl/libxl_domain.c | 9 +++++++++
> > >   1 file changed, 9 insertions(+)
> > > 
> > > diff --git a/tools/libxl/libxl_domain.c b/tools/libxl/libxl_domain.c
> > > index ef1a0927b00d..2678ad2ad54f 100644
> > > --- a/tools/libxl/libxl_domain.c
> > > +++ b/tools/libxl/libxl_domain.c
> > > @@ -1647,6 +1647,15 @@ int libxl_retrieve_domain_configuration(libxl_ctx *ctx, uint32_t domid,
> > >           }
> > >       }
> > > +    /* reset IOMEM's GFN to initial value */
> > > +    {
> > > +        int i;
> > > +
> > > +        for (i = 0; i < d_config->b_info.num_iomem; i++)
> > > +            if (d_config->b_info.iomem[i].gfn == 0)
> > > +                d_config->b_info.iomem[i].gfn = LIBXL_INVALID_GFN;
> > > +    }
> > > +
> > I don't think this is necessary. Instead we should tell libxl to save
> > the generated value into the template. Add an update_config hook for the
> > iomem type should be better.
> Agree, this is why I tagged the patch as [HACK]
> Unfortunately, I have little knowledge of libxl and not sure
> how to properly fix it. Can you tell a bit more on what
> a proper fix could be?

See libxl__update_domain_configuration, which is called after domain
construction is completed. It will call the update_config hook for a
device type to save anything that is generated in the process of domain
creation. One example is in libxl_nic. You can do the same to iomem I
think.

The end result is the generated values you care about are saved into the
template. When the domain is migrated / rebooted libxl will use the
saved values instead.

Strictly speaking your patch of adding the snippet to
libxl_retrieve_domain_configuration isn't wrong, but I would prefer that
function to only contain code to fetch states that can be changed during
domain runtime. The iomem range isn't one of those states AIUI.

Wei.


> > Wei.
> Thank you,
> Oleksandr

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel