Re: [PATCH RFC LINUX v1] xen: arm: enable migration on ARM.

[Ian Campbell <ian.campbell@citrix.com>]

On Wed, 2016-01-06 at 17:47 +0000, Stefano Stabellini wrote:
> Please CC linux-arm for the non-RFC patches
> 
> On Wed, 9 Dec 2015, Ian Campbell wrote:
> > Replace various stub functions with real functionality, including
> > reestablishing the shared info page and the per-vcpu info pages on
> > restore.
> > 
> > Reestablishing the vcpu info page is a little subtle. The
> > VCPUOP_register_vcpu_info hypercall can only be called on either the
> > current VCPU or on an offline different VCPU. Since migration occurs
> > with all VCPUS online they are all therefore online at the point of
> > resume.
> > 
> > Therefore we must perform a cross VCPU call to each non-boot VCPU,
> > which cannot be done in the xen_arch_post_suspend() callback since
> > that is run from stop_machine() with interrupts disabled.
> > 
> > Furthermore VCPUOP_register_vcpu_info can only be called once per-VCPU
> > in a given domain, so it must not be called after a cancelled suspend
> > (which resumes in the same domain).
> > 
> > Therefore xen_arch_resume() gains a suspend_cancelled parameter and we
> > resume the secondary VCPUs there only if needed.
> 
> This is a bit complex, maybe we could use a cpu notifier like we do on
> x86?

I('m pretty sure I) tried that first but CPUs don't go offline in this kind
of migration, so there is no notification event.

The ARM VCPUs fall somewhere between the behaviour of x86/PV and x86/HVM
VCPUS and sadly AFAICT neither of those approaches works. In particular
x86/HVM doesn't do VCPU placement for >LEGACY_MAX_VCPUS which is needed by
ARM, I think the fix for x86/HVM would involve changes a lot like this one.

Ian.