Re: [PATCH v8 05/13] tools/libxc: support to resume uncooperative HVM guests

[Wei Liu <wei.liu2@citrix.com>]

On Fri, Feb 19, 2016 at 11:20:08AM -0500, Konrad Rzeszutek Wilk wrote:
[...]
> > > > > > ===
> > > > > > 
> > > > > > Note that I fix one place in this version from "guest state" to
> > > > > > "guest
> > > > > > return code" in the second paragraph. And that sentence is a big big
> > > > > > assumption that I don't know whether it is true or not --
> > > > > > reverse-engineer from comment before xc_domain_resume and what Linux
> > > > > > does.
> > > > > > 
> > > > > > But the more I think the more I'm not sure if I'm writing the right
> > > > > > thing. I also can't judge what is the right behaviour on the Linux
> > > > > > side.
> > > > > > 
> > > > > > Konrad, can you fact-check the commit message a bit? And maybe you
> > > > > > can
> > > > > > help answer the following questions?
> > > > > > 
> > > > > > 1. If we use fast=0 on PVHVM guest, will it work?
> > > > > 
> > > > > Yes.
> > > > > > 2. If we use fast=0 on HVM guest, will it work?
> > > > > 
> > > > > Yes.
> > > > > 
> > > > > > 
> > > > > > What is worse, when I say "work" I actually have no clear definition
> > > > > > of
> > > > > > it. There doesn't seem to be a defined state that the guest needs to
> > > > > > be.
> > > > > 
> > > > > For PVHVM guests, fast = 0, requires that the guest makes an hypercall
> > > > > to  SCHEDOP_shutdown(SHUTDOWN_suspend). After the hypercall has
> > > > > completed (so Xen has suspended the guest then later resumed it), it
> > > > > would be the guest responsibility to setup Xen infrastructure. As in
> > > > > retrieve the shared_info (XENMAPSPACE_shared_info), setup XenBus, etc.
> > > > > 
> > > > > For HVM guests, fast = 0, suspends the guests without the guest making
> > > > > any hypercalls. It is in effect the hypervisor injecting an S3 suspend.
> > > > > Afterwards the guest is resumed and continues as usual. No PV drivers -
> > > > > hence no need to re-establish Xen PV infrastructure.
> > > > > 
> > > > 
> > > > Wait, isn't this function about resuming a guest? I'm confused because
> > > > you talk about HV injecting S3 suspend. I guess you wrote the wrong
> > > > thing?
> 
> I was writing the whole chain - suspend, and then resume. This patch is
> about resume - but to get to resume you need to suspend first.
> 

Yes, of course. I was thinking more about writing it down as comment for
xc_domain_resume, so I wrote something from the perspective of resuming.

If you don't disagree with my extrapolation in previous email we don't
need to quibble about the wording anymore.

> > > > 
> > > > My guess is below, from the perspective of resuming a guest
> > > > 
> > > >   PVHVM guest would have used SCHEDOP_shutdown(SHUTDOWN_suspend) to
> > > >   suspend. So when toolstack uses fast=0, the guest resumes from the
> > > >   hypercall with return code unmodified. Guest then re-setup Xen
> > > >   infrastructure.
> > > 
> > > Who or what has torn down the existing infrastructure from the guest's life
> > > before the suspend in this case? AFAI Remember a guest expects to return
> 
> The guest. Or it can ignore it and and just re-init all its settings.
> 
> > > from SCHEDOP_shutdown(SHUTDOWN_suspend) with return code == 0 in a freshly
> > > minted new domain, but in the resume case it is actually resuming in the
> > > original domain, complete with any evtchn's and grant tables mappings etc
> > > still intact from before it slept.
> > > 
> > > Perhaps I'm misremembering and the guest is expected to deal with the
> > > possibility of resources already being in place when it re-sets up the
> > > infra?
> 
> Correct - albeit all of them are stale. Thought on some off-chance they may
> be set correctly.
> 
> > > 
> > 
> > Sigh, this is that sort of things that get to my nerves. I should try to
> > write something down when we come to a conclusion.  I would be happy to
> > have any definite answer to the expected behaviour of guest.
> > Extrapolation is not very helpful in the face of some many different
> > versions of Linux'es and BSDs.
> > 
> > But, if the confusion is only about PVHVM guest with fast=0, we can
> > forbid that specific combination for now. That should be enough to move
> > COLO forward.
> 
> .. forbid what? PVHVM resuming with fast=0? Why?  Because the guest may
> fall on its face?

Yes, forbid resuming PVHVM with fast=0 if we have no clear definition of
how it works. It's not because guest would fall, it's because we can't
tell which side (the guest or the toolstack) is buggy when the guest
falls.

But it looks like we (you ;-) ) have clear idea of how it works, we
(you) just need to write it down.

Wei.

> > 
> > Wei.
> > 
> > > Ian.
> > >