Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

[Jeremy Maitin-Shepard <jbms@cmu.edu>]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

[snip]

> Okay, I have thought it through and I think that, as an initial step, we can do
> something like this:

> - preload the image-saving kernel before hibernation
> - in the hibernation code path replace device_suspend() with the shutting down
> of
>   all devices without unregistering them (not very nice, but should be
> sufficient
>   for a while)

It seems that the effect of what is done by the current hibernate
implementations is to shutdown all of the devices, but according to
kernel data structures, have it look like the devices were merely
suspended (i.e. device_suspend).  Then in the resume path, the "restore
image" kernel also calls device_suspend just before jumping to the
hibernated kernel, so all of the devices the "restore image" kernel knew
about are in the state device_suspend expects them to be in, except that
they were actually suspended by a different kernel, so they might not be
in quite the right state.  There is also the issue that the "restore
image" kernel might not know about all of the devices; for instance, if
USB support is modular, and, as is likely to be the case, the user
didn't load the USB modules in the "restore image" kernel from an initrd
or something, then the USB devices will actually be powered off, rather
than "suspended".  Despite these apparent discrepancies, it seems that
for many devices (I'm not sure USB devices are included, though),
device_restore happens to do the right thing so that the device is
placed back in the state it needs to be so that the driver can begin
talking to it as it did before, and the device is recognized as the same
device as was there before (since otherwise mounted filesystems backed
by block devices that came back as a different device would cause great
havoc).

Since I recall there being issues with USB devices being recognized as
the same devices post-hibernate-resume, without looking at the code I'm
inclined to believe that the USB drivers still don't end up resuming
from hibernate correctly.

Note that I am describing what is done currently, not what is planned to
be done (i.e. change device_suspend to quiesce and device_resume to
unquiesce).  It seems that ironically, despite everyone believing that
device_suspend/device_resume is incorrect for hibernate, many of the
things that those functions do (like saving the PCI configuration,
perhaps, and then restoring it later, or re-initializing the device) are
actually necessary, especially for modular drivers that won't be loaded
by the "restore image" kernel.

What needs to be done is for the devices to be shut down (or possibly
just quiesced for a select few, but we won't worry about that
complication until later; in the case of the current implementations,
they should all be quiesced rather than shut down), but whatever
information that will be needed later to reinitialize the device
(ideally the reinitialization should be able to handle the device either
being in a quiesced state or completely off) and recognize it as the
same device must be saved.  This probably means they cannot be
"unregistered", as otherwise there would be nothing with which to
associate the saved information.

The resume path needs to use the saved state to reinitialize the device
and recognize it as the same device.  It seems that the existing
reprobing code may not be sufficient for this.  Note that exactly the
same thing must be done on resume for both the current hibernate
implementations and the kexec approach.  It seems that properly
restoring the devices should be relatively easy for the devices that
already get this correct, like IDE devices and basic PCI devices (and
SATA and SCSI devices as well perhaps?), and possibly harder for
Firewire or USB devices.

> - when we've called device_power_down() and save_processor_state(), jump to
>   the image-saving kernel and let it run
> - make the image-saving kernel set up everything, save the image without
>   starting any user space (we may use the existing image-saving code for this
>   purpose, with some modifications) and power off the system (or make it enter
>   S4)

I suppose this has the advantage of not requiring that a
kernel-to-userspace interface be created for this purpose.

> - use the existing restoration code to load the image and jump to the
>   hibernated kernel

This would again avoid the need for a separate userspace-kernelspace
interface for the purpose, so I agree it could be a useful thing to do
initially.

> - in the restore code patch replace device_resume() with the reprobing of all
>   devices.

See my comments above.

-- 
Jeremy Maitin-Shepard