Re: PV driver domains and S3 sleep

[Jeremy Fitzhardinge <jeremy@goop.org>]

 On 09/16/2010 12:04 PM, Joanna Rutkowska wrote:
> On 09/16/10 13:52, Keir Fraser wrote:
>> On 16/09/2010 12:44, "Rafal Wojtczuk" <rafal@invisiblethingslab.com> wrote:
>>
>>> The topic is self-explanatory: how to ensure that a PV driver domain correctly
>>> prepares its PCI devices for S3 sleep?
>>> If I do "pm-suspend" in dom0, and the driver domain has active network
>>> interfaces, 
>>> suspend hangs the system. Yes, in case of this particular machine, suspend
>>> works
>>> fine when there is no driver domain.
>>> It is possible to manually invoke scripts from /usr/lib64/pm-utils/sleep.d/ in
>>> driver 
>>> domain. In the test case, "ifconfig down wlan0" in the driver domain allows
>>> the suspend to go smoothly. But generally, is it enough ? The kernel device
>>> driver should 
>>> prepare the PCI device properly for S3, shouldn't it ?
>>> Would it be more proper to [somehow] notify a driver domain _kernel_ that we
>>> are 
>>> going to S3 (just like dom0 kernel is notified), and let it execute all
>>> necessary actions
>>> (including, but not only, launching of usermode pm-utils scripts), just like
>>> dom0 kernel 
>>> does ? Would it work at all, considering that driver domain kernel has no
>>> access to 
>>> ACPI tables ? 
>>> Currently, how are these issues taken care of in the mainstream Xen?
>> I don't think it currently is handled. HVM driver domains (using VT-d or
>> equivalent) can be put into virtual S3. We would need an equivalent concept
>> for PV driver domains. Or for devices to be hot-unplugged from the driver
>> domain, and re-plugged on resume?
>>
> But, can you explain how Xen notifies Dom0 when the system enters S3,
> and if the same mechanism could be (easily) used to do the same for a
> driver PV domain?

The dom0 kernel initiates S3 itself (possibly in response to a
lid-switch or the like), so it knows it is going into S3.  As part of
that it can do something to notify all the other domains that they in
turn need to do something.

I think the simplest thing to do is just do a regular PV save/restore on
the domains, but without needing to save their pages to disk.  That way
their regular device model suspend/resume will do the right thing to the
hardware devices.  The only change that may be needed is to make sure
that the normal PCI suspend/resume bus calls are done on the Xen
save/restore path.

Or perhaps the domains will need to know whether its a normal
save/restore, vs S3, vs S5.  In that case we could add a xenstore key
which they could watch to know they need to do something.  But it would
need a bit of thought to handshake the whole process.

    J