Re: Migration between different bitness toolstacks

["Roger Pau Monné" <roger.pau@citrix.com>]

On 14/01/14 17:18, Ian Campbell wrote:
> On Tue, 2014-01-14 at 16:05 +0000, Jan Beulich wrote:
>>>>> On 14.01.14 at 15:57, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>> As part of XenServer's attempt to move to a 64bit dom0, we have
>>> encountered a sizeable flaw in xc_domain_{save,restore}().
>>>
>>> Migration of a VM from a 32bit toolstack to a 64bit toolstackfails with:
>>>
>>> xc: detail: xc_domain_restore: starting restore of new domid 1
>>> xc: detail: xc_domain_restore: p2m_size = ffffffff00010000
>>> xc: error: Couldn't allocate p2m_frame_list array: Internal error
>>> xc: detail: Restore exit of domid 1 with rc=1
>>>
>>> This is caused because of
>>>
>>> RDEXACT(io_fd, &dinfo->p2m_size, sizeof(unsigned long))
>>>
>>> where sizeof(unsigned long) is different between the source and destination.
>>>
>>>
>>> It is unreasonable for the format of the migration stream to rely on the
>>> bitness of the toolstack, which should be completely transparent as far
>>> as "motion of a VM" is concerned.  Furthermore, the same issue occurs
>>> with suspend/resume where the stream gets written to a file in the meantime.
>>>
>>> A quick grep across the code shows several other items in the migration
>>> stream which depend on toolstack bitness.
>>>
>>> There is no way to divine whether the far side of the migration stream
>>> is 32 or 64 bit, which is now vital information required to read the
>>> stream correctly.
>>
>> And I think, even if x86 doesn't care, differing endianness should
>> be dealt with at the same time.
> 
> FWIW I'm not currently expecting ARM to reuse
> tools/libxc/xc_domain_{save,restore}.c.
> 
> It might be worth putting the effort into making the ARM code be cleaner
> and supportable with a sensible protocol so that other future ports can
> reuse it. Potentially even x86 could one day switch, although the old
> code would have to remain for compat purposes.

If we only guarantee migration support between n and n+1 (so for example
4.2 to 4.3, but not 4.2 to 4.4), the old code could go away at some point.

http://wiki.xen.org/wiki/Xen_Version_Compatibility