Migration of a 32-bit PV-Guest

Migration of a 32-bit PV-Guest

[Vasco Fachin]

Hi everybody,

I am currently working on a object-oriented paravirtual operating
system as a university project.

We were able to use a MiniOS core in order to build a relative stable
guest, or at least stable enough for our purposes. :-D

However for one thing we could not proceed. We need to migrate our
guest to another machine, so after some research we came by this
hypercall: HYPERVISOR_update_va_mapping(...), which is apparently
needed in order to unmap the shared info page before we could suspend
our system.

At this point, our system crashes everytime with the following output
from "xm dmesg":
(XEN) mm.c:918:d320 Error getting mfn 4636 (pfn 5555555555555555) from
L1 entry 0000000004636023 for l1e_owner=320, pg_owner=320

I tested the same hypercall with a 32bit version of MiniOS and it
crashes as well with the same output. Then I tried it with a 64bit
version of MiniOS and seems to work just fine.

Somehow I cannot find any 32bit paravirtual machine that could be
migrated. Does anybody know if such a system exists?

Our configuration at the moment is this:
Dom0: Ubuntu 12.10 64bit
XEN: xen 4.1.4 64bit
DomU: 32bit PV-Guest (Object-oriented derived from a 32bit version of MiniOS).

We tested our system, with the same behavior, also on the following platform:
Dom0: Ubuntu 12.04 32bit
XEN: xen 4.1.4 32bit

My question is: does anybody know if a migration of a 32bit PV-System
is possible (in Xen 4.1.4/any newer release)? Or has anyone an idea of
why this particular hypercall fails to execute on a 32bit guest, while
it works on a 64bit guest?

I of course don't ask for a solution of our problem, but maybe someone
came by the same kind of problem and can point me in the right
direction?

Thank you very much in advance!

Best regards,

Vasco Fachin

Re: Migration of a 32-bit PV-Guest

[Andrew Cooper]

On 21/01/15 15:17, Vasco Fachin wrote:
> Hi everybody,
>
> I am currently working on a object-oriented paravirtual operating
> system as a university project.

Sounds like an interesting project.

>
> We were able to use a MiniOS core in order to build a relative stable
> guest, or at least stable enough for our purposes. :-D
>
> However for one thing we could not proceed. We need to migrate our
> guest to another machine, so after some research we came by this
> hypercall: HYPERVISOR_update_va_mapping(...), which is apparently
> needed in order to unmap the shared info page before we could suspend
> our system.

Yes - the shared info page must be removed from the pagetables before
suspend, and must be remapped after resume.  The shared info page is
actually a Xen page mapped into the guest, so will be in a different
location.

>
> At this point, our system crashes everytime with the following output
> from "xm dmesg":
> (XEN) mm.c:918:d320 Error getting mfn 4636 (pfn 5555555555555555) from
> L1 entry 0000000004636023 for l1e_owner=320, pg_owner=320

pfn 5555555555555555 looks like a deliberately poisoned value, which
suggests that you are not umapping the page that you are intending to.

Can you observe what the shared info mfn is from boot?  I presume it
isn't 0x4636 given the above error.

>
> I tested the same hypercall with a 32bit version of MiniOS and it
> crashes as well with the same output. Then I tried it with a 64bit
> version of MiniOS and seems to work just fine.
>
> Somehow I cannot find any 32bit paravirtual machine that could be
> migrated. Does anybody know if such a system exists?

Any 32bit Linux guest will successfully migrate.

>
> Our configuration at the moment is this:
> Dom0: Ubuntu 12.10 64bit
> XEN: xen 4.1.4 64bit
> DomU: 32bit PV-Guest (Object-oriented derived from a 32bit version of MiniOS).
>
> We tested our system, with the same behavior, also on the following platform:
> Dom0: Ubuntu 12.04 32bit
> XEN: xen 4.1.4 32bit
>
> My question is: does anybody know if a migration of a 32bit PV-System
> is possible (in Xen 4.1.4/any newer release)? Or has anyone an idea of
> why this particular hypercall fails to execute on a 32bit guest, while
> it works on a 64bit guest?

I can tell you for absolute certain that 32bit PV guests can migrate. 
We test this extensively in XenServer, and I was quite careful to make
sure it continued to work even after rewriting the migration toolstack
code for migration v2.

~Andrew

Re: Migration of a 32-bit PV-Guest

[Ian Campbell]

On Wed, 2015-01-21 at 15:29 +0000, Andrew Cooper wrote:
> > I tested the same hypercall with a 32bit version of MiniOS and it
> > crashes as well with the same output. Then I tried it with a 64bit
> > version of MiniOS and seems to work just fine.
> >
> > Somehow I cannot find any 32bit paravirtual machine that could be
> > migrated. Does anybody know if such a system exists?
> 
> Any 32bit Linux guest will successfully migrate.

It's seems pretty likely that the OP was talking about 32bit *minios*
domains, not Linux ones.

I'm not sure if mini-os supports migration at all. I do recall the
openmirage people fixing some stuff in mini-os in that regard, but I'd
expect that was for 64-bit since I expect that is what they use.

Vasco, you might look at what the mirage folks did (look in xen.git and
there trees) and see if it applies to 32-bit, otherwise I'm afraid you
may be stuck implementing (or at least fixing) migration in mini-os
yourself.

Ian.