[PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Yosuke Iwamatsu]

Hi,

This series of patches is an attempt to support live migration of hvm
domains with stubdom-dm.

For hvm domains to live-migrate, we should create a log-dirty shared
memory segment between xc_save and qemu-dm. for a normal qemu-dm,
memory sharing is done by inter process communication interface
(shmget/shmat/shmctl), but it obviously doesn't work for a stubdom-dm
because it is not a dom0 process but a domain itself.

The idea here is to allocate log-dirty the shared memory segment in
the stubdom-dm and to map the memory region into the xc_save process
by using grant table interface.

[PATCH 1/4] mini-os: Make utility function get_self_id() public
[PATCH 2/4] Log-dirty shared memory between stubdom-dm and xc_save
[PATCH 3/4] Log-dirty shared memory between stubdom-dm and xc_save
            (ioemu-remote part)
[PATCH 4/4] xend: Give stubdom flag to xc_save

Thanks,
-----------------------
Yosuke Iwamatsu
        NEC Corporation

Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Samuel Thibault]

Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit :
> This series of patches is an attempt to support live migration of hvm
> domains with stubdom-dm.

Err, I'm a bit surprised. I haven't tested it since then, but I'm sure
live migration was working with stubdoms a few months ago, without
the need for a logdirty shared buffer.  This works thanks to the few
lines at the end of the cpu_physical_memory_rw() function that tell the
hypervisor about modified memory.  Actually, the plan was even to use
that bit of code in all cases (not only non-stubdom), so as to get rid
of the logdirty shared buffer completely.

Samuel

Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Samuel Thibault]

Samuel Thibault, le Tue 16 Dec 2008 11:02:31 +0100, a écrit :
> Actually, the plan was even to use that bit of code in all cases (not
> only non-stubdom), so as to get rid of the logdirty shared buffer
> completely.

(That even _helps_ the shadow code, which can thus know that a pagetable
was replaced by data from the disk, for instance.)

Samuel

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Ian Jackson]

Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm"):
> Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit :
> > This series of patches is an attempt to support live migration of hvm
> > domains with stubdom-dm.
> 
> Err, I'm a bit surprised. I haven't tested it since then, but I'm sure
> live migration was working with stubdoms a few months ago, without
> the need for a logdirty shared buffer.  This works thanks to the few
> lines at the end of the cpu_physical_memory_rw() function that tell the
> hypervisor about modified memory.  Actually, the plan was even to use
> that bit of code in all cases (not only non-stubdom), so as to get rid
> of the logdirty shared buffer completely.

I take it you're referring to this:

#ifdef CONFIG_STUBDOM
    if (logdirty_bitmap != NULL)
        xc_hvm_modified_memory(xc_handle, domid, _addr >> TARGET_PAGE_BITS,
                ((_addr + _len + TARGET_PAGE_SIZE - 1) >> TARGET_PAGE_BITS)
                    - (_addr >> TARGET_PAGE_BITS));
#endif

Yosuke Iwamatsu: is this not working for you then ?  It's possible
that I've broken it somehow.

Ian.

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Samuel Thibault]

Ian Jackson, le Tue 16 Dec 2008 14:04:53 +0000, a écrit :
> Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm"):
> > This works thanks to the few lines at the end of the
> > cpu_physical_memory_rw() function that tell the hypervisor about
> > modified memory.  Actually, the plan was even to use that bit of
> > code in all cases (not only non-stubdom), so as to get rid of the
> > logdirty shared buffer completely.
> 
> I take it you're referring to this:
> 
> #ifdef CONFIG_STUBDOM
>     if (logdirty_bitmap != NULL)
>         xc_hvm_modified_memory(xc_handle, domid, _addr >> TARGET_PAGE_BITS,
>                 ((_addr + _len + TARGET_PAGE_SIZE - 1) >> TARGET_PAGE_BITS)
>                     - (_addr >> TARGET_PAGE_BITS));
> #endif

Yes, precisely.

Samuel

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Yosuke Iwamatsu]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

Ian Jackson wrote:
> Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm"):
>> Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit :
>>> This series of patches is an attempt to support live migration of hvm
>>> domains with stubdom-dm.
>> Err, I'm a bit surprised. I haven't tested it since then, but I'm sure
>> live migration was working with stubdoms a few months ago, without
>> the need for a logdirty shared buffer.  This works thanks to the few
>> lines at the end of the cpu_physical_memory_rw() function that tell the
>> hypervisor about modified memory.  Actually, the plan was even to use
>> that bit of code in all cases (not only non-stubdom), so as to get rid
>> of the logdirty shared buffer completely.
> 
> I take it you're referring to this:
> 
> #ifdef CONFIG_STUBDOM
>     if (logdirty_bitmap != NULL)
>         xc_hvm_modified_memory(xc_handle, domid, _addr >> TARGET_PAGE_BITS,
>                 ((_addr + _len + TARGET_PAGE_SIZE - 1) >> TARGET_PAGE_BITS)
>                     - (_addr >> TARGET_PAGE_BITS));
> #endif

Ah, I didn't notice this.

> Yosuke Iwamatsu: is this not working for you then ?  It's possible
> that I've broken it somehow.

Live-migration fails because stubdom-dm doesn't respond to the request
from xc_save for the logdirty bitmap segment. The attached patch
simply generated from old ioemu tree will fix the issue, then.

Derived from old xen internal tools/ioemu tree.
Signed-off-by: Yosuke Iwamatsu <y-iwamatsu@ab.jp.nec.com>

Thanks,
-- Yosuke

[-- Attachment #2: stubdom_respond_to_logdirty_event.patch --]
[-- Type: all/allfiles, Size: 1360 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Samuel Thibault]

Yosuke Iwamatsu, le Wed 17 Dec 2008 10:06:10 +0900, a écrit :
> The attached patch
> simply generated from old ioemu tree will fix the issue, then.

> Derived from old xen internal tools/ioemu tree.
> Signed-off-by: Yosuke Iwamatsu <y-iwamatsu@ab.jp.nec.com>

Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Ian Jackson]

Yosuke Iwamatsu writes ("Re: [Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support live migration of	HVM domain with stubdom-dm"):
> Live-migration fails because stubdom-dm doesn't respond to the request
> from xc_save for the logdirty bitmap segment. The attached patch
> simply generated from old ioemu tree will fix the issue, then.

Thanks, applied.  I take it this should be backported to 3.3 ?

Ian.

Re: Re: [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

[Samuel Thibault]

Ian Jackson, le Wed 17 Dec 2008 11:39:46 +0000, a écrit :
> Yosuke Iwamatsu writes ("Re: [Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support live migration of	HVM domain with stubdom-dm"):
> > Live-migration fails because stubdom-dm doesn't respond to the request
> > from xc_save for the logdirty bitmap segment. The attached patch
> > simply generated from old ioemu tree will fix the issue, then.
> 
> Thanks, applied.  I take it this should be backported to 3.3 ?

Yes (though in 3.3 the in-tree qemu is used by default).

Samuel