VM Migration on ARM

VM Migration on ARM

[Vijay Kilari]

Hi Andrew,

    From Ian & Stefano, I came to know that you have introduced
Migration framework v2
under below patch series

http://marc.info/?l=xen-devel&m=141036915311145

I have few queries:

1) Is there any plans/timeline to support for ARM32/ARM64?.
2) Also does the Domain Image format specification for ARM is complete?

Your insights on this topic for arm64 would be helpful.

Regards
Vijay

Re: VM Migration on ARM

[Andrew Cooper]

On 11/03/15 05:32, Vijay Kilari wrote:
> Hi Andrew,
>
>     From Ian & Stefano, I came to know that you have introduced
> Migration framework v2
> under below patch series
>
> http://marc.info/?l=xen-devel&m=141036915311145
>
> I have few queries:
>
> 1) Is there any plans/timeline to support for ARM32/ARM64?.

I will not personally be doing any ARM migration support, not even as
part of getting migration v2 accepted upstream.

> 2) Also does the Domain Image format specification for ARM is complete?

The spec has certain sentinel values for ARM already where it made sense
to just include them while writing.

It has no specific thought to ARM migration, but an ARM virtual machine
is exactly like an x86 HVM domain (from the toolstacks point of view),
other than the Qemu record.  With any luck, no new additions to the spec
will be needed, but the spec is trivially extendible if required.

~Andrew

Re: VM Migration on ARM

[Julien Grall]

Hi,

On 11/03/2015 10:51, Andrew Cooper wrote:
> On 11/03/15 05:32, Vijay Kilari wrote:
>> Hi Andrew,
>>
>>      From Ian & Stefano, I came to know that you have introduced
>> Migration framework v2
>> under below patch series
>>
>> http://marc.info/?l=xen-devel&m=141036915311145
>>
>> I have few queries:
>>
>> 1) Is there any plans/timeline to support for ARM32/ARM64?.
>
> I will not personally be doing any ARM migration support, not even as
> part of getting migration v2 accepted upstream.

FWIW, there is a patch series on the ML for supporting migration v1 [1].
But the author doesn't work on it anymore, so AFAIK nobody touched it 
since this version.

Regards,

[1] http://marc.info/?l=xen-devel&m=139958417713245

-- 
Julien Grall

Re: VM Migration on ARM

[Ian Campbell]

On Wed, 2015-03-11 at 11:04 +0000, Julien Grall wrote:
> Hi,
> 
> On 11/03/2015 10:51, Andrew Cooper wrote:
> > On 11/03/15 05:32, Vijay Kilari wrote:
> >> Hi Andrew,
> >>
> >>      From Ian & Stefano, I came to know that you have introduced
> >> Migration framework v2
> >> under below patch series
> >>
> >> http://marc.info/?l=xen-devel&m=141036915311145
> >>
> >> I have few queries:
> >>
> >> 1) Is there any plans/timeline to support for ARM32/ARM64?.
> >
> > I will not personally be doing any ARM migration support, not even as
> > part of getting migration v2 accepted upstream.

Just to be clear, I don't expect you to do anything of the sort either,
other than keeping ARM as is compiling of course.

> FWIW, there is a patch series on the ML for supporting migration v1 [1].
> But the author doesn't work on it anymore, so AFAIK nobody touched it 
> since this version.

Right.

Some parts of that series could still be a useful starting point, and
for the rest I expect that the (re)implementation on top of Andrew's
work should be almost trivial.

IMO it ought to be reasonably easy to get save/restore and
dead-migration working based on the combination of those two series, and
I think this should be done first before move on to looking at all the
log-dirty stuff and p2m frobbing which is needed to perform a live
migration, which is going to be far more time consuming to get right.

Ian.

Re: VM Migration on ARM

[Vijay Kilari]

Hi Andrew,

On Wed, Mar 11, 2015 at 4:21 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 11/03/15 05:32, Vijay Kilari wrote:
>> Hi Andrew,
>>
>>     From Ian & Stefano, I came to know that you have introduced
>> Migration framework v2
>> under below patch series
>>
>> http://marc.info/?l=xen-devel&m=141036915311145
>>
>> I have few queries:
>>
>> 1) Is there any plans/timeline to support for ARM32/ARM64?.
>
> I will not personally be doing any ARM migration support, not even as
> part of getting migration v2 accepted upstream.
>
>> 2) Also does the Domain Image format specification for ARM is complete?
>
> The spec has certain sentinel values for ARM already where it made sense
> to just include them while writing.
>
> It has no specific thought to ARM migration, but an ARM virtual machine
> is exactly like an x86 HVM domain (from the toolstacks point of view),
> other than the Qemu record.  With any luck, no new additions to the spec
> will be needed, but the spec is trivially extendible if required.

IMO, ARM uses PVH domain.
Can you please let me know how to use your patches, I mean xl/xc commands
to do migration (non-live/live) using your patches.
Any debugging and testing support information would be helpful.

Regards
Vijay

Re: VM Migration on ARM

[Andrew Cooper]

On 20/03/15 13:17, Vijay Kilari wrote:
> Hi Andrew,
>
> On Wed, Mar 11, 2015 at 4:21 PM, Andrew Cooper
> <andrew.cooper3@citrix.com>  wrote:
>> On 11/03/15 05:32, Vijay Kilari wrote:
>>> Hi Andrew,
>>>
>>>      From Ian & Stefano, I came to know that you have introduced
>>> Migration framework v2
>>> under below patch series
>>>
>>> http://marc.info/?l=xen-devel&m=141036915311145
>>>
>>> I have few queries:
>>>
>>> 1) Is there any plans/timeline to support for ARM32/ARM64?.
>> I will not personally be doing any ARM migration support, not even as
>> part of getting migration v2 accepted upstream.
>>
>>> 2) Also does the Domain Image format specification for ARM is complete?
>> The spec has certain sentinel values for ARM already where it made sense
>> to just include them while writing.
>>
>> It has no specific thought to ARM migration, but an ARM virtual machine
>> is exactly like an x86 HVM domain (from the toolstacks point of view),
>> other than the Qemu record.  With any luck, no new additions to the spec
>> will be needed, but the spec is trivially extendible if required.
> IMO, ARM uses PVH domain.
> Can you please let me know how to use your patches, I mean xl/xc commands
> to do migration (non-live/live) using your patches.
> Any debugging and testing support information would be helpful.

Hi - apologises for the late reply.  I have been on holiday.

The patches series as available does make `xl save/restore/migrate` 
work, although the error handling leaves a lot to be desired.

We (XenServer) are in the process of attempting to get everything 
upstream.  The libxc side of things is definitely working and already 
shipped in XenServer 6.5 (We absolutely needed to support migrating 
between 32 and 64bit toolstacks, which was the primary reason for the 
rewrite).

What sort of timescale are you looking for on this?  I hope to have a 
new series posted soon, now that the datacopier subseries has been accepted.

~Andrew

Re: VM Migration on ARM

[Vijay Kilari]

Hi Andrew,

On Wed, Mar 25, 2015 at 9:32 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 20/03/15 13:17, Vijay Kilari wrote:
>>
>> Hi Andrew,
>>
>> On Wed, Mar 11, 2015 at 4:21 PM, Andrew Cooper
>> <andrew.cooper3@citrix.com>  wrote:
>>>
>>> On 11/03/15 05:32, Vijay Kilari wrote:
>>>>
>>>> Hi Andrew,
>>>>
>>>>      From Ian & Stefano, I came to know that you have introduced
>>>> Migration framework v2
>>>> under below patch series
>>>>
>>>> http://marc.info/?l=xen-devel&m=141036915311145
>>>>
>>>> I have few queries:
>>>>
>>>> 1) Is there any plans/timeline to support for ARM32/ARM64?.
>>>
>>> I will not personally be doing any ARM migration support, not even as
>>> part of getting migration v2 accepted upstream.
>>>
>>>> 2) Also does the Domain Image format specification for ARM is complete?
>>>
>>> The spec has certain sentinel values for ARM already where it made sense
>>> to just include them while writing.
>>>
>>> It has no specific thought to ARM migration, but an ARM virtual machine
>>> is exactly like an x86 HVM domain (from the toolstacks point of view),
>>> other than the Qemu record.  With any luck, no new additions to the spec
>>> will be needed, but the spec is trivially extendible if required.
>>
>> IMO, ARM uses PVH domain.
>> Can you please let me know how to use your patches, I mean xl/xc commands
>> to do migration (non-live/live) using your patches.
>> Any debugging and testing support information would be helpful.
>
>
> Hi - apologises for the late reply.  I have been on holiday.
>
> The patches series as available does make `xl save/restore/migrate` work,
> although the error handling leaves a lot to be desired.
>
> We (XenServer) are in the process of attempting to get everything upstream.
> The libxc side of things is definitely working and already shipped in
> XenServer 6.5 (We absolutely needed to support migrating between 32 and
> 64bit toolstacks, which was the primary reason for the rewrite).
>
> What sort of timescale are you looking for on this?  I hope to have a new
> series posted soon, now that the datacopier subseries has been accepted.
>

  I am looking for Xen 4.6 release. If not, atleast a prototype on
ARM64 would be
good enough. I would be grateful if you could provide your support on xl/xc
stuff for arm64. I can take care of other things.

Regards
Vijay

Re: VM Migration on ARM

[Andrew Cooper]

On 27/03/15 13:32, Vijay Kilari wrote:
>
>>>>> 2) Also does the Domain Image format specification for ARM is complete?
>>>> The spec has certain sentinel values for ARM already where it made sense
>>>> to just include them while writing.
>>>>
>>>> It has no specific thought to ARM migration, but an ARM virtual machine
>>>> is exactly like an x86 HVM domain (from the toolstacks point of view),
>>>> other than the Qemu record.  With any luck, no new additions to the spec
>>>> will be needed, but the spec is trivially extendible if required.
>>> IMO, ARM uses PVH domain.
>>> Can you please let me know how to use your patches, I mean xl/xc commands
>>> to do migration (non-live/live) using your patches.
>>> Any debugging and testing support information would be helpful.
>>
>> Hi - apologises for the late reply.  I have been on holiday.
>>
>> The patches series as available does make `xl save/restore/migrate` work,
>> although the error handling leaves a lot to be desired.
>>
>> We (XenServer) are in the process of attempting to get everything upstream.
>> The libxc side of things is definitely working and already shipped in
>> XenServer 6.5 (We absolutely needed to support migrating between 32 and
>> 64bit toolstacks, which was the primary reason for the rewrite).
>>
>> What sort of timescale are you looking for on this?  I hope to have a new
>> series posted soon, now that the datacopier subseries has been accepted.
>>
>    I am looking for Xen 4.6 release. If not, atleast a prototype on
> ARM64 would be
> good enough. I would be grateful if you could provide your support on xl/xc
> stuff for arm64. I can take care of other things.

I will try to remember to CC you on the next posting of the series. Most 
of the code is architecture agnostic.  The arm bits should hopefully be 
a case of copying the x86 hvm code and perhaps tweaking a hypercall or two.

It is certainly my intention to have migration v2 for x86 fully upstream 
and complete in the 4.6 time frame (and that includes not merging it at 
the 11th hour).

~Andrew