Re: PV driver domains and S3 sleep

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Sep 16, 2010 at 01:44:24PM +0200, Rafal Wojtczuk wrote:
> Hello,
> 
> The topic is self-explanatory: how to ensure that a PV driver domain correctly 
> prepares its PCI devices for S3 sleep?
> If I do "pm-suspend" in dom0, and the driver domain has active network interfaces, 
> suspend hangs the system. Yes, in case of this particular machine, suspend works
> fine when there is no driver domain. 
> It is possible to manually invoke scripts from /usr/lib64/pm-utils/sleep.d/ in driver 
> domain. In the test case, "ifconfig down wlan0" in the driver domain allows
> the suspend to go smoothly. But generally, is it enough ? The kernel device driver should 

The pci_disable calls that are made do put the devices in the D3 (or is it D0 state).
However those calls are not made when you do 'ifconfig X down' (I think). You need
to do 'rmmod ipw2100' to trigger those calls, or trigger the drivers' suspend call
invocation.

The drivers' suspend call invocation is a twisty maze of dependency (ie, must first
suspend the driver, and only after that you can suspend the PCI bus).

The S3 suspend on Linux also freezez the user space, cgroups, and whole bunch
of other stuff. But you don't care about that.

What I think you care about is to put the device in the appropiate D state.

> prepare the PCI device properly for S3, shouldn't it ?  
> Would it be more proper to [somehow] notify a driver domain _kernel_ that we are 
> going to S3 (just like dom0 kernel is notified), and let it execute all necessary actions 
> (including, but not only, launching of usermode pm-utils scripts), just like dom0 kernel 
> does ? Would it work at all, considering that driver domain kernel has no access to 
> ACPI tables ? 

I think that depends on the PCI device. In laptop world, the wireless card can
do some weird stuff when you press Ctrl-F5 for example - it would invoke some 
ACPI code (well, the Linux kernel AML driver would invoke it), which then
disables/unloads the driver as appropiate. With DomU having no ACPI support, it means
that the Dom0 would yank the PCI device away from the DomU - which actually considering
that we are using pciback as the owner, would mean you could pass a request to the
DomU saying: "Hey, reconfigure now. Device going away." And I think that might
work today actually.

But back to putting the device in the appropiate D state. You could
pass in DomU a call akin to doing "suspend" > /sys/power/state
which should do the appropiate PCI move.

> Currently, how are these issues taken care of in the mainstream Xen? 

Never explored I fear.
> 
> Thanks in advance,
> Rafal Wojtczuk
> 
> 
>  
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel