Hibernation considerations

Hibernation considerations

[Rafael J. Wysocki]

Hi,

Since many alternative approaches to hibernation are now being considered and
discussed, I thought it might be a good idea to list some things that in my not
so humble opinion should be taken care of by any hibernation framework.  They
are listed below, not in any particular order, because I think they all are
important.  Still, I might have forgotten something, so everyone with
experience in implementing hibernation, especially Pavel and Nigel, please
check if the list is complete.

(1) Filesystems mounted before the hibernation are untouchable

    When there's a memory snapshot, either in the form of a hibernation image,
    or in the form of the "old" kernel and processes available to the "new"
    kexeced kernel responsible for saving their memory, the filesystems mounted
    before the hibernation should not be accessed, even for reading, because
    that would cause their on-disk state to be inconsistent with the snapshot
    and might lead to a filesystem corruption.

(2) Swap space in use before the hibernation must be handled with care

    If swap space is used for saving the memory snapshot, the snapshot-saving
    application (or kernel) must be careful enough not to overwrite swap pages
    that contain valid memory contents stored in there before the hibernation.

(3) There are memory regions that must not be saved or restored

    Some memory regions contain data that shouldn't be overwritten during the
    restore, because that might lead to the system not working correctly
    afterwards.  Also, on some systems there are valid 'struct pages'
    structures that in fact corresond to memory holes and we should not attempt
    to save those pages.

(4) The user should be able to limit the size of a hibernation image

    There are a couple of reasons of that.  For example, the storage space
    used for saving the image may be smaller than the entire RAM or the user
    may want the image to be saved quickier.

(5) Hibernation should be transparent from the applications' point of view

    Generally, applications should not notice that hibernation took place.
    [Note that I don't regard all processes as applications and I think that
    there may be processes which need to handle the hibernation in a special
    way.]  Ideally, for example, if some audio is being played when a
    hibernation starts, the audio player should be able to continue playing the
    same audio after the restore from the point in which it has been
    interrupted by the hibernation.  Also, the CPU affinities and similar
    settings requested by the applications before a hibernation should be
    binding after the restore.

(6) State of devices from before hibernation should be restored, if possible

    If possible, during a restore devices should be brought back to the same
    state in which they were before the corresponding hibernation.  Of course
    in some situations it might be impossible to do that (eg. the user
    connected the hibernated system to a different IP subnet and then
    restored), but as a general rule, we should do our best to restore the
    state of devices, which is directly related to point (5) above.

(7) On ACPI systems special platform-related actions have to be carried out at
    the right points, so that the platform works correctly after the restore

    The ACPI specification requires us to invoke some global ACPI methods
    during the hibernation and during the restore.  Moreover, the ordering of
    code related to these ACPI methods may not be arbitrary (eg. some of
    them have to be executed after devices are put into low power states etc.).

(8) Hibernation and restore should not be too slow

    In my opinion, if more than one minute is needed to hibernate the system
    with the help of certain hibernation framework, then this framework is not
    very useful in practice.  It might be useful to perform some special tasks
    (eg. moving a server to another place without taking it down), but it is
    not very useful, for example, to notebook users.

(9) Hibernation framework should not be too difficult to set up

    It follows from my experience that if the users are required to do too much
    work to set up a hibernation framework, they will not use it as long as
    there are simpler alternatives (some of them will not use hibernation at
    all if it's too difficult to get to work).  On the other hand, if the users
    are provided with a working hibernation framework by their distribution
    and they find it useful, they are not likely to use kernel.org kernels if
    t's too difficult to replace the distribution kernel with a generic one due
    to the hibernation framework's requirements.

All of the existing hibernation frameworks have been written with the above
points in mind and that's why they are what they are.  In particular, the
existence of the tasks freezer, hated by some people to the point of insanity,
follows directly from points (1), (4) and (5).

In my opinion any hibernation framework that doesn't take the above
requirements into account in any way will be a failure.  Moreover, the existing
frameworks fail to follow some of them too, so I consider all of these
frameworks as a work in progress.  For this reason, I will much more appreciate
ideas allowing us to improve the existing frameworks in a more or less
evolutionary way, then attempts to replace them all with something entirely
new.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Nigel Cunningham]

[-- Attachment #1.1: Type: text/plain, Size: 6039 bytes --]

Hi.

On Sunday 15 July 2007 22:33:32 Rafael J. Wysocki wrote:
> Hi,
> 
> Since many alternative approaches to hibernation are now being considered 
and
> discussed, I thought it might be a good idea to list some things that in my 
not
> so humble opinion should be taken care of by any hibernation framework.  
They
> are listed below, not in any particular order, because I think they all are
> important.  Still, I might have forgotten something, so everyone with
> experience in implementing hibernation, especially Pavel and Nigel, please
> check if the list is complete.
> 
> (1) Filesystems mounted before the hibernation are untouchable
> 
>     When there's a memory snapshot, either in the form of a hibernation 
image,
>     or in the form of the "old" kernel and processes available to the "new"
>     kexeced kernel responsible for saving their memory, the filesystems 
mounted
>     before the hibernation should not be accessed, even for reading, because
>     that would cause their on-disk state to be inconsistent with the 
snapshot
>     and might lead to a filesystem corruption.
> 
> (2) Swap space in use before the hibernation must be handled with care
> 
>     If swap space is used for saving the memory snapshot, the 
snapshot-saving
>     application (or kernel) must be careful enough not to overwrite swap 
pages
>     that contain valid memory contents stored in there before the 
hibernation.
> 
> (3) There are memory regions that must not be saved or restored
> 
>     Some memory regions contain data that shouldn't be overwritten during 
the
>     restore, because that might lead to the system not working correctly
>     afterwards.  Also, on some systems there are valid 'struct pages'
>     structures that in fact corresond to memory holes and we should not 
attempt
>     to save those pages.
> 
> (4) The user should be able to limit the size of a hibernation image
> 
>     There are a couple of reasons of that.  For example, the storage space
>     used for saving the image may be smaller than the entire RAM or the user
>     may want the image to be saved quickier.
> 
> (5) Hibernation should be transparent from the applications' point of view
> 
>     Generally, applications should not notice that hibernation took place.
>     [Note that I don't regard all processes as applications and I think that
>     there may be processes which need to handle the hibernation in a special
>     way.]  Ideally, for example, if some audio is being played when a
>     hibernation starts, the audio player should be able to continue playing 
the
>     same audio after the restore from the point in which it has been
>     interrupted by the hibernation.  Also, the CPU affinities and similar
>     settings requested by the applications before a hibernation should be
>     binding after the restore.
> 
> (6) State of devices from before hibernation should be restored, if possible
> 
>     If possible, during a restore devices should be brought back to the same
>     state in which they were before the corresponding hibernation.  Of 
course
>     in some situations it might be impossible to do that (eg. the user
>     connected the hibernated system to a different IP subnet and then
>     restored), but as a general rule, we should do our best to restore the
>     state of devices, which is directly related to point (5) above.
> 
> (7) On ACPI systems special platform-related actions have to be carried out 
at
>     the right points, so that the platform works correctly after the restore
> 
>     The ACPI specification requires us to invoke some global ACPI methods
>     during the hibernation and during the restore.  Moreover, the ordering 
of
>     code related to these ACPI methods may not be arbitrary (eg. some of
>     them have to be executed after devices are put into low power states 
etc.).
> 
> (8) Hibernation and restore should not be too slow
> 
>     In my opinion, if more than one minute is needed to hibernate the system
>     with the help of certain hibernation framework, then this framework is 
not
>     very useful in practice.  It might be useful to perform some special 
tasks
>     (eg. moving a server to another place without taking it down), but it is
>     not very useful, for example, to notebook users.
> 
> (9) Hibernation framework should not be too difficult to set up
> 
>     It follows from my experience that if the users are required to do too 
much
>     work to set up a hibernation framework, they will not use it as long as
>     there are simpler alternatives (some of them will not use hibernation at
>     all if it's too difficult to get to work).  On the other hand, if the 
users
>     are provided with a working hibernation framework by their distribution
>     and they find it useful, they are not likely to use kernel.org kernels 
if
>     t's too difficult to replace the distribution kernel with a generic one 
due
>     to the hibernation framework's requirements.
> 
> All of the existing hibernation frameworks have been written with the above
> points in mind and that's why they are what they are.  In particular, the
> existence of the tasks freezer, hated by some people to the point of 
insanity,
> follows directly from points (1), (4) and (5).
> 
> In my opinion any hibernation framework that doesn't take the above
> requirements into account in any way will be a failure.  Moreover, the 
existing
> frameworks fail to follow some of them too, so I consider all of these
> frameworks as a work in progress.  For this reason, I will much more 
appreciate
> ideas allowing us to improve the existing frameworks in a more or less
> evolutionary way, then attempts to replace them all with something entirely
> new.

Sounds good to me. Nothing extra occurs immediately.

Regards,

Nigel
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Hibernation considerations

[Dr. David Alan Gilbert]

* Rafael J. Wysocki (rjw@sisk.pl) wrote:

> (5) Hibernation should be transparent from the applications' point of view
> 
>     Generally, applications should not notice that hibernation took place.
>     [Note that I don't regard all processes as applications and I think that
>     there may be processes which need to handle the hibernation in a special
>     way.]  Ideally, for example, if some audio is being played when a
>     hibernation starts, the audio player should be able to continue playing the
>     same audio after the restore from the point in which it has been
>     interrupted by the hibernation.  Also, the CPU affinities and similar

That would be _so_ embarrassing in a library; I'd rather the audio
player had the opportunity to consider whether restarting was a good idea.

> (6) State of devices from before hibernation should be restored, if possible
> 
>     If possible, during a restore devices should be brought back to the same
>     state in which they were before the corresponding hibernation.  Of course
>     in some situations it might be impossible to do that (eg. the user
>     connected the hibernated system to a different IP subnet and then
>     restored), but as a general rule, we should do our best to restore the
>     state of devices, which is directly related to point (5) above.

Or the user unplugs their flash drive after hibernation rather than before.

Two things which I think would be nice to consider are:
   1) Encryption - I'd actually prefer if my luks device did not
       remember the key accross a hibernation; I want to be forced to
       reenter the phrase.  However I don't know what the best thing
       to do to partitions/applications using the luks device is.

   2) Some level of debugging needs to be available so that users can
      provide something so you can see why something hasn't hibernated
      or why (as in the case of this tosh laptop) it still takes power
      during hibernation.

Dave
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    | Running GNU/Linux on Alpha,68K| Happy  \ 
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Re: Hibernation considerations

[jimmy bahuleyan]

Al Boldi wrote:
> 
> This should be the responsibility of the kexec'd hibernating kernel.  Note 
> though in (6), the normal kernel takes care of preparing devices, then the 
> hibernating kernel dumps the image and either calls S4 or S3.  On resume 
> from S3 it can immediately switch over to the normal kernel, and from S4 the 
> known bootup would occur.
> 
>> (8) Hibernation and restore should not be too slow
>>
>>     In my opinion, if more than one minute is needed to hibernate the
>> system with the help of certain hibernation framework, then this framework
>> is not very useful in practice.  It might be useful to perform some
>> special tasks (eg. moving a server to another place without taking it
>> down), but it is not very useful, for example, to notebook users.
> 
> The latest hibernating kexec patches boot a kexec'd modular kernel with 
> initramfs into crashkernel=16M@16M in less than one second.  Switch-back is 
> almost instant.  Add to this the time required to either store or restore 
> the image, and it may be obvious that this approach isn't slower, but maybe 
> even faster than the current swsusp.
> 

What about (9)? Would it be that a user choosing to build a kernel with
hibernate support gets a additional modular kernel built (which he
should then use for resumption) or he should configure & build the
modular kernel independent of main kernel?

Or will the Linux boot procedure change so that it always goes thru a
modular part followed by kexec (just to be uniform)?

Although the kexec approach seems interesting, the final user-scenario
seems a bit complex (or confusing).

-jb
-- 
Tact is the art of making a point without making an enemy.

Re: Hibernation considerations

[Alan Stern]

On Sun, 15 Jul 2007, Al Boldi wrote:

> >     If possible, during a restore devices should be brought back to the
> > same state in which they were before the corresponding hibernation.  Of
> > course in some situations it might be impossible to do that (eg. the user
> > connected the hibernated system to a different IP subnet and then
> > restored), but as a general rule, we should do our best to restore the
> > state of devices, which is directly related to point (5) above.
> 
> This part could easily be handled by the normal kernel before and after 
> resume.

I agree with you except for the word "easily".  And there are some 
things the kernel simply punts on (I'm thinking of the current VGA 
font).

> > (7) On ACPI systems special platform-related actions have to be carried
> > out at the right points, so that the platform works correctly after the
> > restore
> >
> >     The ACPI specification requires us to invoke some global ACPI methods
> >     during the hibernation and during the restore.  Moreover, the ordering
> > of code related to these ACPI methods may not be arbitrary (eg. some of
> > them have to be executed after devices are put into low power states
> > etc.).
> 
> This should be the responsibility of the kexec'd hibernating kernel.  Note 
> though in (6), the normal kernel takes care of preparing devices, then the 
> hibernating kernel dumps the image and either calls S4 or S3.  On resume 
> from S3 it can immediately switch over to the normal kernel, and from S4 the 
> known bootup would occur.

Is it really that simple?  Somehow I doubt it.  In order for some 
devices to remain available for the kexec'd kernel to use, they cannot 
be suspended at the ACPI level.  So the kexec'd kernel will have to 
handle the ACPI requirements for those devices.  Likewise, it would 
have to handle the ACPI interactions which need to be done after all 
devices are prepared for the transition to S3 or S4.

> > (8) Hibernation and restore should not be too slow
> >
> >     In my opinion, if more than one minute is needed to hibernate the
> > system with the help of certain hibernation framework, then this framework
> > is not very useful in practice.  It might be useful to perform some
> > special tasks (eg. moving a server to another place without taking it
> > down), but it is not very useful, for example, to notebook users.
> 
> The latest hibernating kexec patches boot a kexec'd modular kernel with 
> initramfs into crashkernel=16M@16M in less than one second.  Switch-back is 
> almost instant.  Add to this the time required to either store or restore 
> the image, and it may be obvious that this approach isn't slower, but maybe 
> even faster than the current swsusp.

Does that include the time required for probing PCI buses?  On my
desktop system, PCI probing incurs a five-second timeout delay because
of a bug in the BIOS's USB firmware.  Don't be so sure that kexec will
always be lightning fast; it is always better to avoid unnecessary
boots.

Alan Stern

Re: Hibernation considerations

[Al Boldi]

jimmy bahuleyan wrote:
> Al Boldi wrote:
> > This should be the responsibility of the kexec'd hibernating kernel. 
> > Note though in (6), the normal kernel takes care of preparing devices,
> > then the hibernating kernel dumps the image and either calls S4 or S3. 
> > On resume from S3 it can immediately switch over to the normal kernel,
> > and from S4 the known bootup would occur.
> >
> >> (8) Hibernation and restore should not be too slow
> >>
> >>     In my opinion, if more than one minute is needed to hibernate the
> >> system with the help of certain hibernation framework, then this
> >> framework is not very useful in practice.  It might be useful to
> >> perform some special tasks (eg. moving a server to another place
> >> without taking it down), but it is not very useful, for example, to
> >> notebook users.
> >
> > The latest hibernating kexec patches boot a kexec'd modular kernel with
> > initramfs into crashkernel=16M@16M in less than one second.  Switch-back
> > is almost instant.  Add to this the time required to either store or
> > restore the image, and it may be obvious that this approach isn't
> > slower, but maybe even faster than the current swsusp.
>
> What about (9)? Would it be that a user choosing to build a kernel with
> hibernate support gets a additional modular kernel built (which he
> should then use for resumption) or he should configure & build the
> modular kernel independent of main kernel?
>
> Or will the Linux boot procedure change so that it always goes thru a
> modular part followed by kexec (just to be uniform)?
>
> Although the kexec approach seems interesting, the final user-scenario
> seems a bit complex (or confusing).

Well, it may sound confusing because it is so unexpectedly simple.  I didn't 
answer to (9) because from a user pov nothing should change, and everything 
should be scriptable such that the user wouldn't even notice the kernel 
using a new hibernation approach.


Thanks!

--
Al

Re: Hibernation considerations

[Al Boldi]

Alan Stern wrote:
> On Sun, 15 Jul 2007, Al Boldi wrote:
> > >     If possible, during a restore devices should be brought back to
> > > the same state in which they were before the corresponding
> > > hibernation.  Of course in some situations it might be impossible to
> > > do that (eg. the user connected the hibernated system to a different
> > > IP subnet and then restored), but as a general rule, we should do our
> > > best to restore the state of devices, which is directly related to
> > > point (5) above.
> >
> > This part could easily be handled by the normal kernel before and after
> > resume.
>
> I agree with you except for the word "easily".  And there are some
> things the kernel simply punts on (I'm thinking of the current VGA
> font).

Why; can you explain?

> > > (7) On ACPI systems special platform-related actions have to be
> > > carried out at the right points, so that the platform works correctly
> > > after the restore
> > >
> > >     The ACPI specification requires us to invoke some global ACPI
> > > methods during the hibernation and during the restore.  Moreover, the
> > > ordering of code related to these ACPI methods may not be arbitrary
> > > (eg. some of them have to be executed after devices are put into low
> > > power states etc.).
> >
> > This should be the responsibility of the kexec'd hibernating kernel. 
> > Note though in (6), the normal kernel takes care of preparing devices,
> > then the hibernating kernel dumps the image and either calls S4 or S3. 
> > On resume from S3 it can immediately switch over to the normal kernel,
> > and from S4 the known bootup would occur.
>
> Is it really that simple?  Somehow I doubt it.  In order for some
> devices to remain available for the kexec'd kernel to use, they cannot
> be suspended at the ACPI level.  So the kexec'd kernel will have to
> handle the ACPI requirements for those devices.  Likewise, it would
> have to handle the ACPI interactions which need to be done after all
> devices are prepared for the transition to S3 or S4.

Ok, after applying the latest kexec patches, I was able to use the kexec'd 
kernel to suspend to ram and resume to the normal kernel, while working 
under a full-blown X session.  It went without a hitch.  All that is needed 
now are the dump/restore hibernation-image routines.

> > > (8) Hibernation and restore should not be too slow
> > >
> > >     In my opinion, if more than one minute is needed to hibernate the
> > > system with the help of certain hibernation framework, then this
> > > framework is not very useful in practice.  It might be useful to
> > > perform some special tasks (eg. moving a server to another place
> > > without taking it down), but it is not very useful, for example, to
> > > notebook users.
> >
> > The latest hibernating kexec patches boot a kexec'd modular kernel with
> > initramfs into crashkernel=16M@16M in less than one second.  Switch-back
> > is almost instant.  Add to this the time required to either store or
> > restore the image, and it may be obvious that this approach isn't
> > slower, but maybe even faster than the current swsusp.
>
> Does that include the time required for probing PCI buses?  On my
> desktop system, PCI probing incurs a five-second timeout delay because
> of a bug in the BIOS's USB firmware.

Using a modular kernel you would only insmod those modules that you need to 
dump the image, which is mainly the diskdriver.  If you wanted to dump it 
onto USB flash, then you would insmod that driver, and if that driver is 
slow due to a bug, then a fix should be in order.

> Don't be so sure that kexec will
> always be lightning fast; it is always better to avoid unnecessary
> boots.

Agreed, but what we want to achieve right know is a proof of concept.  Later, 
I could imagine a specially stub'd device driver to be kexec'd instead of 
the full kernel.


Thanks!

--
Al

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Cornelius Riemenschneider wrote:

> Hi,
> I think a (10) is needed to be clear:
>
> (10) This depends directly on (9)
> Easy Userinterface for the average user, no one wants to edit several
> files and use a long commandline for a suspend (not all know about
> alias(P)), so keeping the effort to suspend your system low, please.(eg
> for my TuxOnIce i just type hibernate as root/ sudo hibernate as normal
> user and the suspend begin, the rest is automatically done by the
> configuration(for me this is shutting down my wireless interface)
>
> Cornelius Riemenschneider

for both #9 and #10, if there are complex command lines that need to be 
used, they can be put into scripts (including things like 'sudo 
hibernate')

this doesn't prevent people from useing kernel.org kernels, in fact the 
seperate between the main system and what's used for hibernation may make 
it easier to change the main kernel (all that you would need is to enable 
kexec in your main kernel and 'everything else would just work')

now if you need/want to change out your hibernate kernel you will have to 
change the hibernate scripts to tell them to use the new kernel. yes this 
is an extra step, but it can still be simple. for that matter, it may be 
possible for the distros to agree enough to add 'make install_hibernate' 
to the kernel makefiles like they have 'make install' that will just 'do 
the right thing' for each particular distro.

makeing things explicit and flexible may require complex command lines to 
implement, but nothing says that the user (even a moderatly advanced user) 
needs to deal with those command lines directly.

David Lang

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Alan Stern wrote:

>>> (7) On ACPI systems special platform-related actions have to be carried
>>> out at the right points, so that the platform works correctly after the
>>> restore
>>>
>>>     The ACPI specification requires us to invoke some global ACPI methods
>>>     during the hibernation and during the restore.  Moreover, the ordering
>>> of code related to these ACPI methods may not be arbitrary (eg. some of
>>> them have to be executed after devices are put into low power states
>>> etc.).
>>
>> This should be the responsibility of the kexec'd hibernating kernel.  Note
>> though in (6), the normal kernel takes care of preparing devices, then the
>> hibernating kernel dumps the image and either calls S4 or S3.  On resume
>> from S3 it can immediately switch over to the normal kernel, and from S4 the
>> known bootup would occur.
>
> Is it really that simple?  Somehow I doubt it.  In order for some
> devices to remain available for the kexec'd kernel to use, they cannot
> be suspended at the ACPI level.  So the kexec'd kernel will have to
> handle the ACPI requirements for those devices.  Likewise, it would
> have to handle the ACPI interactions which need to be done after all
> devices are prepared for the transition to S3 or S4.

so if there are two states for hardware

1. able to be detected and initialized with the standard boot routines

2. ACPI suspend mode, requireing ACPI specifics to wake up

is it possible to detect which is needed? cna you try the ACPI wakeup and 
if it doesn't work go to the standard boot routine (or vice-versa)?

>>> (8) Hibernation and restore should not be too slow
>>>
>>>     In my opinion, if more than one minute is needed to hibernate the
>>> system with the help of certain hibernation framework, then this framework
>>> is not very useful in practice.  It might be useful to perform some
>>> special tasks (eg. moving a server to another place without taking it
>>> down), but it is not very useful, for example, to notebook users.
>>
>> The latest hibernating kexec patches boot a kexec'd modular kernel with
>> initramfs into crashkernel=16M@16M in less than one second.  Switch-back is
>> almost instant.  Add to this the time required to either store or restore
>> the image, and it may be obvious that this approach isn't slower, but maybe
>> even faster than the current swsusp.
>
> Does that include the time required for probing PCI buses?  On my
> desktop system, PCI probing incurs a five-second timeout delay because
> of a bug in the BIOS's USB firmware.  Don't be so sure that kexec will
> always be lightning fast; it is always better to avoid unnecessary
> boots.

some drivers will have delays on some machines (especially when you 
consider hardware bugs that have to be worked around), but at the moment 
nothing is remotely close to the 'limit' of 60 seconds being talked about 
here.

although with some machines the mear trasnfer times for the data could 
cause noticable delays (if you have a server with 128G of ram it may take 
a while to get it to a drive)

David Lang

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:

> Hi,
>
> Since many alternative approaches to hibernation are now being considered and
> discussed, I thought it might be a good idea to list some things that in my not
> so humble opinion should be taken care of by any hibernation framework.  They
> are listed below, not in any particular order, because I think they all are
> important.  Still, I might have forgotten something, so everyone with
> experience in implementing hibernation, especially Pavel and Nigel, please
> check if the list is complete.
>
> (1) Filesystems mounted before the hibernation are untouchable
>
>    When there's a memory snapshot, either in the form of a hibernation image,
>    or in the form of the "old" kernel and processes available to the "new"
>    kexeced kernel responsible for saving their memory, the filesystems mounted
>    before the hibernation should not be accessed, even for reading, because
>    that would cause their on-disk state to be inconsistent with the snapshot
>    and might lead to a filesystem corruption.

AFAIK this is only the case with ext3, all other filesystems could be 
accessed read-only safely

this is arguably a bug with ext3 (and has been discussed as such), but 
right now the ext3 team has decided not to change this bahavior so 
hibernate needs to work around it. but don't mistake a work-around for a 
single (admittedly very popular) filesystem with a hard and fast 
directive.

> (2) Swap space in use before the hibernation must be handled with care
>
>    If swap space is used for saving the memory snapshot, the snapshot-saving
>    application (or kernel) must be careful enough not to overwrite swap pages
>    that contain valid memory contents stored in there before the hibernation.

true, in fact, given that many distros and live-CD's autodetect swap 
partitions and consider them fair game, I would argue that the best thing 
to do would be to have the main system free up it's swap partitions before 
going into hibernation.

however, this could be a decision of the particular hibernate routines.

for the kexec approach the mapping of what swap pages are in use is one 
more chunk of data that needs to be assembled and made available through a 
defined interface.

> (4) The user should be able to limit the size of a hibernation image
>
>    There are a couple of reasons of that.  For example, the storage space
>    used for saving the image may be smaller than the entire RAM or the user
>    may want the image to be saved quickier.

it may make sense for this to be split into hard and soft limits.

if you try to save more then the storage space can hold you cannot 
continue, but if you are just a little over the arbatrary size limit that 
was set to make things fast you are better off saving things as-is then 
punting, going back to the system, trying to free more ram, and trying a 
hibernate again.

with the kexec approach the enforcment of these limits is also split into 
two sections.

when the hibernate command is given in the main kernel, it's userspace 
needs to follow some policy to decide how much (if any) memory to free.

this could be anything from 'none, try and save all caches' to 'anything 
you can to minimize the amount of data to be saved, trash all caches' to 
something in between like 'try and free up enough memory to get the saved 
data below 1G, but save caches beyond that point'

then when the second kernel runs, it's userspace tools get the list of 
what memory should be saved that the main kernel handed to it, and then 
decides if this is acceptable (probably mostly the hard limits of 'can 
this work') and proceeds to save it somewhere.

but since the kexec command and the preporation of the devices can change 
the memory, the estimates done by the first kernel's userspace are just 
that, estimates.

> (7) On ACPI systems special platform-related actions have to be carried out at
>    the right points, so that the platform works correctly after the restore
>
>    The ACPI specification requires us to invoke some global ACPI methods
>    during the hibernation and during the restore.  Moreover, the ordering of
>    code related to these ACPI methods may not be arbitrary (eg. some of
>    them have to be executed after devices are put into low power states etc.).

for a pure hibernate mode, you will be powering off the box after saving 
the suspend image. why are there any special ACPI modes involved?

now, for suspend-to-ram you need to be aware of every possible power 
saving option you have and the cost of each of them, and here the ACPI 
modes are heavily used.

I think this is mixing suspend and hibernate still.

David Lang

Re: Hibernation considerations

[Cornelius Riemenschneider]

Hi,
I think a (10) is needed to be clear:

(10) This depends directly on (9)
Easy Userinterface for the average user, no one wants to edit several
files and use a long commandline for a suspend (not all know about
alias(P)), so keeping the effort to suspend your system low, please.(eg
for my TuxOnIce i just type hibernate as root/ sudo hibernate as normal
user and the suspend begin, the rest is automatically done by the
configuration(for me this is shutting down my wireless interface)

Cornelius Riemenschneider

My source of power: www.humppa.com

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:

> On Sunday, 15 July 2007 14:58, Dr. David Alan Gilbert wrote:
>> * Rafael J. Wysocki (rjw@sisk.pl) wrote:
>>
>>> (5) Hibernation should be transparent from the applications' point of view
>>>
>>>     Generally, applications should not notice that hibernation took place.
>>>     [Note that I don't regard all processes as applications and I think that
>>>     there may be processes which need to handle the hibernation in a special
>>>     way.]  Ideally, for example, if some audio is being played when a
>>>     hibernation starts, the audio player should be able to continue playing the
>>>     same audio after the restore from the point in which it has been
>>>     interrupted by the hibernation.  Also, the CPU affinities and similar
>>
>> That would be _so_ embarrassing in a library; I'd rather the audio
>> player had the opportunity to consider whether restarting was a good idea.
>>
>>> (6) State of devices from before hibernation should be restored, if possible
>>>
>>>     If possible, during a restore devices should be brought back to the same
>>>     state in which they were before the corresponding hibernation.  Of course
>>>     in some situations it might be impossible to do that (eg. the user
>>>     connected the hibernated system to a different IP subnet and then
>>>     restored), but as a general rule, we should do our best to restore the
>>>     state of devices, which is directly related to point (5) above.
>>
>> Or the user unplugs their flash drive after hibernation rather than before.
>>
>> Two things which I think would be nice to consider are:
>>    1) Encryption - I'd actually prefer if my luks device did not
>>        remember the key accross a hibernation; I want to be forced to
>>        reenter the phrase.  However I don't know what the best thing
>>        to do to partitions/applications using the luks device is.
>
> Encryption is possible with both the userland hibernation (aka uswsusp) and
> TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a "must
> have" feature for a framework to be generally useful (many users don't use it
> anyway).

he's talking about the main system useing an encrypted device/partition, 
not the hibernate image being stored encrypted.

This would require the main system 'forget' the keys when it does the 
hinbernate and prompt for it again during the wake-up phase.

David Lang

Re: Hibernation considerations

[Rafael J. Wysocki]

On Sunday, 15 July 2007 14:58, Dr. David Alan Gilbert wrote:
> * Rafael J. Wysocki (rjw@sisk.pl) wrote:
> 
> > (5) Hibernation should be transparent from the applications' point of view
> > 
> >     Generally, applications should not notice that hibernation took place.
> >     [Note that I don't regard all processes as applications and I think that
> >     there may be processes which need to handle the hibernation in a special
> >     way.]  Ideally, for example, if some audio is being played when a
> >     hibernation starts, the audio player should be able to continue playing the
> >     same audio after the restore from the point in which it has been
> >     interrupted by the hibernation.  Also, the CPU affinities and similar
> 
> That would be _so_ embarrassing in a library; I'd rather the audio
> player had the opportunity to consider whether restarting was a good idea.
> 
> > (6) State of devices from before hibernation should be restored, if possible
> > 
> >     If possible, during a restore devices should be brought back to the same
> >     state in which they were before the corresponding hibernation.  Of course
> >     in some situations it might be impossible to do that (eg. the user
> >     connected the hibernated system to a different IP subnet and then
> >     restored), but as a general rule, we should do our best to restore the
> >     state of devices, which is directly related to point (5) above.
> 
> Or the user unplugs their flash drive after hibernation rather than before.
> 
> Two things which I think would be nice to consider are:
>    1) Encryption - I'd actually prefer if my luks device did not
>        remember the key accross a hibernation; I want to be forced to
>        reenter the phrase.  However I don't know what the best thing
>        to do to partitions/applications using the luks device is.

Encryption is possible with both the userland hibernation (aka uswsusp) and
TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a "must
have" feature for a framework to be generally useful (many users don't use it
anyway).

>    2) Some level of debugging needs to be available so that users can
>       provide something so you can see why something hasn't hibernated
>       or why (as in the case of this tosh laptop) it still takes power
>       during hibernation.

I agree.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:

> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> Hi,
>>>
>>> Since many alternative approaches to hibernation are now being considered and
>>> discussed, I thought it might be a good idea to list some things that in my not
>>> so humble opinion should be taken care of by any hibernation framework.  They
>>> are listed below, not in any particular order, because I think they all are
>>> important.  Still, I might have forgotten something, so everyone with
>>> experience in implementing hibernation, especially Pavel and Nigel, please
>>> check if the list is complete.
>>>
>>> (1) Filesystems mounted before the hibernation are untouchable
>>>
>>>    When there's a memory snapshot, either in the form of a hibernation image,
>>>    or in the form of the "old" kernel and processes available to the "new"
>>>    kexeced kernel responsible for saving their memory, the filesystems mounted
>>>    before the hibernation should not be accessed, even for reading, because
>>>    that would cause their on-disk state to be inconsistent with the snapshot
>>>    and might lead to a filesystem corruption.
>>
>> AFAIK this is only the case with ext3, all other filesystems could be
>> accessed read-only safely
>>
>> this is arguably a bug with ext3 (and has been discussed as such), but
>> right now the ext3 team has decided not to change this bahavior so
>> hibernate needs to work around it. but don't mistake a work-around for a
>> single (admittedly very popular) filesystem with a hard and fast
>> directive.
>>
>>> (2) Swap space in use before the hibernation must be handled with care
>>>
>>>    If swap space is used for saving the memory snapshot, the snapshot-saving
>>>    application (or kernel) must be careful enough not to overwrite swap pages
>>>    that contain valid memory contents stored in there before the hibernation.
>>
>> true, in fact, given that many distros and live-CD's autodetect swap
>> partitions and consider them fair game, I would argue that the best thing
>> to do would be to have the main system free up it's swap partitions before
>> going into hibernation.
>>
>> however, this could be a decision of the particular hibernate routines.
>>
>> for the kexec approach the mapping of what swap pages are in use is one
>> more chunk of data that needs to be assembled and made available through a
>> defined interface.
>>
>>> (4) The user should be able to limit the size of a hibernation image
>>>
>>>    There are a couple of reasons of that.  For example, the storage space
>>>    used for saving the image may be smaller than the entire RAM or the user
>>>    may want the image to be saved quickier.
>>
>> it may make sense for this to be split into hard and soft limits.
>>
>> if you try to save more then the storage space can hold you cannot
>> continue, but if you are just a little over the arbatrary size limit that
>> was set to make things fast you are better off saving things as-is then
>> punting, going back to the system, trying to free more ram, and trying a
>> hibernate again.
>>
>> with the kexec approach the enforcment of these limits is also split into
>> two sections.
>>
>> when the hibernate command is given in the main kernel, it's userspace
>> needs to follow some policy to decide how much (if any) memory to free.
>
> How are you going to achieve this without (a) having hibernation-aware
> user space or (b) the freezer?

the hibernate command is a userspace command, but the fact that other 
things in userspace are running at the same time is exactly why this is 
only an estimate and best-effort as I said in the paragraph below.

>> but since the kexec command and the preporation of the devices can change
>> the memory, the estimates done by the first kernel's userspace are just
>> that, estimates.
>>
>>> (7) On ACPI systems special platform-related actions have to be carried out at
>>>    the right points, so that the platform works correctly after the restore
>>>
>>>    The ACPI specification requires us to invoke some global ACPI methods
>>>    during the hibernation and during the restore.  Moreover, the ordering of
>>>    code related to these ACPI methods may not be arbitrary (eg. some of
>>>    them have to be executed after devices are put into low power states etc.).
>>
>> for a pure hibernate mode, you will be powering off the box after saving
>> the suspend image. why are there any special ACPI modes involved?
>
> Because, for example, on my machine the status of power supply (present
> vs not present) is not updated correctly after the restore if ACPI callbacks
> aren't used during the hibernation.  That's just experience and it's in line
> with the ACPI spec.

so if a machine is actually powered off the /dev/suspend process won't 
work?

remember that the system may run a different OS between the hibernate and 
the resume, makeing any assumptions about what state the hardware is in 
when you start the resume is a problem.

David Lang

>> now, for suspend-to-ram you need to be aware of every possible power
>> saving option you have and the cost of each of them, and here the ACPI
>> modes are heavily used.
>>
>> I think this is mixing suspend and hibernate still.
>
> Yes, it is, but that's not we who's mixing.  We just need to handle some
> systems built with ACPI in mind.
>
> Greetings,
> Rafael
>
>
>

Re: Hibernation considerations

[Rafael J. Wysocki]

On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
> 
> > Hi,
> >
> > Since many alternative approaches to hibernation are now being considered and
> > discussed, I thought it might be a good idea to list some things that in my not
> > so humble opinion should be taken care of by any hibernation framework.  They
> > are listed below, not in any particular order, because I think they all are
> > important.  Still, I might have forgotten something, so everyone with
> > experience in implementing hibernation, especially Pavel and Nigel, please
> > check if the list is complete.
> >
> > (1) Filesystems mounted before the hibernation are untouchable
> >
> >    When there's a memory snapshot, either in the form of a hibernation image,
> >    or in the form of the "old" kernel and processes available to the "new"
> >    kexeced kernel responsible for saving their memory, the filesystems mounted
> >    before the hibernation should not be accessed, even for reading, because
> >    that would cause their on-disk state to be inconsistent with the snapshot
> >    and might lead to a filesystem corruption.
> 
> AFAIK this is only the case with ext3, all other filesystems could be 
> accessed read-only safely
> 
> this is arguably a bug with ext3 (and has been discussed as such), but 
> right now the ext3 team has decided not to change this bahavior so 
> hibernate needs to work around it. but don't mistake a work-around for a 
> single (admittedly very popular) filesystem with a hard and fast 
> directive.
> 
> > (2) Swap space in use before the hibernation must be handled with care
> >
> >    If swap space is used for saving the memory snapshot, the snapshot-saving
> >    application (or kernel) must be careful enough not to overwrite swap pages
> >    that contain valid memory contents stored in there before the hibernation.
> 
> true, in fact, given that many distros and live-CD's autodetect swap 
> partitions and consider them fair game, I would argue that the best thing 
> to do would be to have the main system free up it's swap partitions before 
> going into hibernation.
> 
> however, this could be a decision of the particular hibernate routines.
> 
> for the kexec approach the mapping of what swap pages are in use is one 
> more chunk of data that needs to be assembled and made available through a 
> defined interface.
> 
> > (4) The user should be able to limit the size of a hibernation image
> >
> >    There are a couple of reasons of that.  For example, the storage space
> >    used for saving the image may be smaller than the entire RAM or the user
> >    may want the image to be saved quickier.
> 
> it may make sense for this to be split into hard and soft limits.
> 
> if you try to save more then the storage space can hold you cannot 
> continue, but if you are just a little over the arbatrary size limit that 
> was set to make things fast you are better off saving things as-is then 
> punting, going back to the system, trying to free more ram, and trying a 
> hibernate again.
> 
> with the kexec approach the enforcment of these limits is also split into 
> two sections.
> 
> when the hibernate command is given in the main kernel, it's userspace 
> needs to follow some policy to decide how much (if any) memory to free.

How are you going to achieve this without (a) having hibernation-aware
user space or (b) the freezer?

> this could be anything from 'none, try and save all caches' to 'anything 
> you can to minimize the amount of data to be saved, trash all caches' to 
> something in between like 'try and free up enough memory to get the saved 
> data below 1G, but save caches beyond that point'
> 
> then when the second kernel runs, it's userspace tools get the list of 
> what memory should be saved that the main kernel handed to it, and then 
> decides if this is acceptable (probably mostly the hard limits of 'can 
> this work') and proceeds to save it somewhere.
> 
> but since the kexec command and the preporation of the devices can change 
> the memory, the estimates done by the first kernel's userspace are just 
> that, estimates.
> 
> > (7) On ACPI systems special platform-related actions have to be carried out at
> >    the right points, so that the platform works correctly after the restore
> >
> >    The ACPI specification requires us to invoke some global ACPI methods
> >    during the hibernation and during the restore.  Moreover, the ordering of
> >    code related to these ACPI methods may not be arbitrary (eg. some of
> >    them have to be executed after devices are put into low power states etc.).
> 
> for a pure hibernate mode, you will be powering off the box after saving 
> the suspend image. why are there any special ACPI modes involved?

Because, for example, on my machine the status of power supply (present
vs not present) is not updated correctly after the restore if ACPI callbacks
aren't used during the hibernation.  That's just experience and it's in line
with the ACPI spec.

> now, for suspend-to-ram you need to be aware of every possible power 
> saving option you have and the cost of each of them, and here the ACPI 
> modes are heavily used.
> 
> I think this is mixing suspend and hibernate still.

Yes, it is, but that's not we who's mixing.  We just need to handle some
systems built with ACPI in mind.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Alan Stern]

On Sun, 15 Jul 2007 david@lang.hm wrote:

> >> for a pure hibernate mode, you will be powering off the box after saving
> >> the suspend image. why are there any special ACPI modes involved?
> >
> > Because, for example, on my machine the status of power supply (present
> > vs not present) is not updated correctly after the restore if ACPI callbacks
> > aren't used during the hibernation.  That's just experience and it's in line
> > with the ACPI spec.
> 
> so if a machine is actually powered off the /dev/suspend process won't 
> work?
> 
> remember that the system may run a different OS between the hibernate and 
> the resume, makeing any assumptions about what state the hardware is in 
> when you start the resume is a problem.

As I understand it, running a different OS between the hibernate and 
the resume would violate the ACPI spec.

Alan Stern

Re: Hibernation considerations

[Alan Stern]

On Sun, 15 Jul 2007 david@lang.hm wrote:

> > (1) Filesystems mounted before the hibernation are untouchable
> >
> >    When there's a memory snapshot, either in the form of a hibernation image,
> >    or in the form of the "old" kernel and processes available to the "new"
> >    kexeced kernel responsible for saving their memory, the filesystems mounted
> >    before the hibernation should not be accessed, even for reading, because
> >    that would cause their on-disk state to be inconsistent with the snapshot
> >    and might lead to a filesystem corruption.
> 
> AFAIK this is only the case with ext3, all other filesystems could be 
> accessed read-only safely
> 
> this is arguably a bug with ext3 (and has been discussed as such), but 
> right now the ext3 team has decided not to change this bahavior so 
> hibernate needs to work around it. but don't mistake a work-around for a 
> single (admittedly very popular) filesystem with a hard and fast 
> directive.

Isn't is possible to avoid this problem by mounting an ext3 filesystem 
as readonly ext2?  Provided the filesystem isn't dirty it should be 
doable.  (And provided the filesystem doesn't use any ext3 extensions 
that are incompatible with ext2.)

Alan Stern

Re: Hibernation considerations

[Rafael J. Wysocki]

On Monday, 16 July 2007 00:42, david@lang.hm wrote:
> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
> >> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> Hi,
> >>>
> >>> Since many alternative approaches to hibernation are now being considered and
> >>> discussed, I thought it might be a good idea to list some things that in my not
> >>> so humble opinion should be taken care of by any hibernation framework.  They
> >>> are listed below, not in any particular order, because I think they all are
> >>> important.  Still, I might have forgotten something, so everyone with
> >>> experience in implementing hibernation, especially Pavel and Nigel, please
> >>> check if the list is complete.
> >>>
> >>> (1) Filesystems mounted before the hibernation are untouchable
> >>>
> >>>    When there's a memory snapshot, either in the form of a hibernation image,
> >>>    or in the form of the "old" kernel and processes available to the "new"
> >>>    kexeced kernel responsible for saving their memory, the filesystems mounted
> >>>    before the hibernation should not be accessed, even for reading, because
> >>>    that would cause their on-disk state to be inconsistent with the snapshot
> >>>    and might lead to a filesystem corruption.
> >>
> >> AFAIK this is only the case with ext3, all other filesystems could be
> >> accessed read-only safely
> >>
> >> this is arguably a bug with ext3 (and has been discussed as such), but
> >> right now the ext3 team has decided not to change this bahavior so
> >> hibernate needs to work around it. but don't mistake a work-around for a
> >> single (admittedly very popular) filesystem with a hard and fast
> >> directive.
> >>
> >>> (2) Swap space in use before the hibernation must be handled with care
> >>>
> >>>    If swap space is used for saving the memory snapshot, the snapshot-saving
> >>>    application (or kernel) must be careful enough not to overwrite swap pages
> >>>    that contain valid memory contents stored in there before the hibernation.
> >>
> >> true, in fact, given that many distros and live-CD's autodetect swap
> >> partitions and consider them fair game, I would argue that the best thing
> >> to do would be to have the main system free up it's swap partitions before
> >> going into hibernation.
> >>
> >> however, this could be a decision of the particular hibernate routines.
> >>
> >> for the kexec approach the mapping of what swap pages are in use is one
> >> more chunk of data that needs to be assembled and made available through a
> >> defined interface.
> >>
> >>> (4) The user should be able to limit the size of a hibernation image
> >>>
> >>>    There are a couple of reasons of that.  For example, the storage space
> >>>    used for saving the image may be smaller than the entire RAM or the user
> >>>    may want the image to be saved quickier.
> >>
> >> it may make sense for this to be split into hard and soft limits.
> >>
> >> if you try to save more then the storage space can hold you cannot
> >> continue, but if you are just a little over the arbatrary size limit that
> >> was set to make things fast you are better off saving things as-is then
> >> punting, going back to the system, trying to free more ram, and trying a
> >> hibernate again.
> >>
> >> with the kexec approach the enforcment of these limits is also split into
> >> two sections.
> >>
> >> when the hibernate command is given in the main kernel, it's userspace
> >> needs to follow some policy to decide how much (if any) memory to free.
> >
> > How are you going to achieve this without (a) having hibernation-aware
> > user space or (b) the freezer?
> 
> the hibernate command is a userspace command, but the fact that other 
> things in userspace are running at the same time is exactly why this is 
> only an estimate and best-effort as I said in the paragraph below.
> 
> >> but since the kexec command and the preporation of the devices can change
> >> the memory, the estimates done by the first kernel's userspace are just
> >> that, estimates.
> >>
> >>> (7) On ACPI systems special platform-related actions have to be carried out at
> >>>    the right points, so that the platform works correctly after the restore
> >>>
> >>>    The ACPI specification requires us to invoke some global ACPI methods
> >>>    during the hibernation and during the restore.  Moreover, the ordering of
> >>>    code related to these ACPI methods may not be arbitrary (eg. some of
> >>>    them have to be executed after devices are put into low power states etc.).
> >>
> >> for a pure hibernate mode, you will be powering off the box after saving
> >> the suspend image. why are there any special ACPI modes involved?
> >
> > Because, for example, on my machine the status of power supply (present
> > vs not present) is not updated correctly after the restore if ACPI callbacks
> > aren't used during the hibernation.  That's just experience and it's in line
> > with the ACPI spec.
> 
> so if a machine is actually powered off the /dev/suspend process won't 
> work?

No, it sort of works as usual, but after the restore the platform is not in the
correct state.

> remember that the system may run a different OS between the hibernate and 
> the resume, makeing any assumptions about what state the hardware is in 
> when you start the resume is a problem.

True, that's problematic.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Alan Stern]

On Sun, 15 Jul 2007, Al Boldi wrote:

> Alan Stern wrote:
> > On Sun, 15 Jul 2007, Al Boldi wrote:
> > > >     If possible, during a restore devices should be brought back to
> > > > the same state in which they were before the corresponding
> > > > hibernation.  Of course in some situations it might be impossible to
> > > > do that (eg. the user connected the hibernated system to a different
> > > > IP subnet and then restored), but as a general rule, we should do our
> > > > best to restore the state of devices, which is directly related to
> > > > point (5) above.
> > >
> > > This part could easily be handled by the normal kernel before and after
> > > resume.
> >
> > I agree with you except for the word "easily".  And there are some
> > things the kernel simply punts on (I'm thinking of the current VGA
> > font).
> 
> Why; can you explain?

>From personal experience I can assure you that it hasn't been easy
getting the USB subsystem to restore devices following a hibernate.  
(In fact the current implementation goes against the spirit, if not the
letter, of the USB spec.)  And making it work requires user
intervention.

As for the VGA font, the effect is easy to see: Run setfont before 
hibernating; when you resume the original font will be back.  The 
kernel simply does not bother to save the VGA font information across a 
hibernate.

> > > This should be the responsibility of the kexec'd hibernating kernel. 
> > > Note though in (6), the normal kernel takes care of preparing devices,
> > > then the hibernating kernel dumps the image and either calls S4 or S3. 
> > > On resume from S3 it can immediately switch over to the normal kernel,
> > > and from S4 the known bootup would occur.
> >
> > Is it really that simple?  Somehow I doubt it.  In order for some
> > devices to remain available for the kexec'd kernel to use, they cannot
> > be suspended at the ACPI level.  So the kexec'd kernel will have to
> > handle the ACPI requirements for those devices.  Likewise, it would
> > have to handle the ACPI interactions which need to be done after all
> > devices are prepared for the transition to S3 or S4.
> 
> Ok, after applying the latest kexec patches, I was able to use the kexec'd 
> kernel to suspend to ram and resume to the normal kernel, while working 
> under a full-blown X session.  It went without a hitch.  All that is needed 
> now are the dump/restore hibernation-image routines.

That's exactly my point.  While doing suspend-to-RAM from a kexec'd 
kernel may be simple, saving the hibernation image will add 
complications.

> > > The latest hibernating kexec patches boot a kexec'd modular kernel with
> > > initramfs into crashkernel=16M@16M in less than one second.  Switch-back
> > > is almost instant.  Add to this the time required to either store or
> > > restore the image, and it may be obvious that this approach isn't
> > > slower, but maybe even faster than the current swsusp.
> >
> > Does that include the time required for probing PCI buses?  On my
> > desktop system, PCI probing incurs a five-second timeout delay because
> > of a bug in the BIOS's USB firmware.
> 
> Using a modular kernel you would only insmod those modules that you need to 
> dump the image, which is mainly the diskdriver.  If you wanted to dump it 
> onto USB flash, then you would insmod that driver, and if that driver is 
> slow due to a bug, then a fix should be in order.

I said nothing about dumping onto USB flash.  I was referring to PCI 
probing; presumably any reasonable kernel for desktop/laptop systems 
will include PCI support.

And the bug isn't in Linux; it is in the firmware.  There's no way to 
fix it short of a BIOS upgrade (and this particular BIOS is no longer 
supported).

Alan Stern

Re: Hibernation considerations

[Nigel Cunningham]

[-- Attachment #1.1: Type: text/plain, Size: 1267 bytes --]

Hi.

On Monday 16 July 2007 09:15:47 Alan Stern wrote:
> On Sun, 15 Jul 2007 david@lang.hm wrote:
> 
> > >> for a pure hibernate mode, you will be powering off the box after 
saving
> > >> the suspend image. why are there any special ACPI modes involved?
> > >
> > > Because, for example, on my machine the status of power supply (present
> > > vs not present) is not updated correctly after the restore if ACPI 
callbacks
> > > aren't used during the hibernation.  That's just experience and it's in 
line
> > > with the ACPI spec.
> > 
> > so if a machine is actually powered off the /dev/suspend process won't 
> > work?
> > 
> > remember that the system may run a different OS between the hibernate and 
> > the resume, makeing any assumptions about what state the hardware is in 
> > when you start the resume is a problem.
> 
> As I understand it, running a different OS between the hibernate and 
> the resume would violate the ACPI spec.

Well then, I know one or two people who would argue that the ACPI spec is 
faulty. :\

Regards,

Nigel
-- 
Nigel Cunningham
Christian Reformed Church of Cobden
103 Curdie Street, Cobden 3266, Victoria, Australia
Ph. +61 3 5595 1185 / +61 417 100 574
Communal Worship: 11 am Sunday.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Alan Stern wrote:

> On Sun, 15 Jul 2007 david@lang.hm wrote:
>
>>>> for a pure hibernate mode, you will be powering off the box after saving
>>>> the suspend image. why are there any special ACPI modes involved?
>>>
>>> Because, for example, on my machine the status of power supply (present
>>> vs not present) is not updated correctly after the restore if ACPI callbacks
>>> aren't used during the hibernation.  That's just experience and it's in line
>>> with the ACPI spec.
>>
>> so if a machine is actually powered off the /dev/suspend process won't
>> work?
>>
>> remember that the system may run a different OS between the hibernate and
>> the resume, makeing any assumptions about what state the hardware is in
>> when you start the resume is a problem.
>
> As I understand it, running a different OS between the hibernate and
> the resume would violate the ACPI spec.

then we need a third mode of operation.

mode 1: Suspend-to-ram

   the system is paused and put into a low-power mode but data remains in 
memory and the system stays awake enough to keep the memory refreshed.

mode 2: new

   the system is paused, data is stored to permanent media, and the system 
is put into a ultra-low power mode.

mode 3: hibernate

   the system is paused, data is stored to permanent media, and the system 
is powered off

with mode 3 there are no requirements or limitations about what can be 
done with the hardware before a resume (the resume could even take place 
on a different piece of identical hardware)

mode 2 could be what you are talking about doing, although I don't see any 
advantage of creating it in additon to mode 3, it doesn't use any less 
power and it locks the system so that it can't be used for anything else 
in the meantime. I guess if it was significantly faster to do then mode 3 
there may be _some_ reason to consider it, but I don't see the speed 
difference.

David Lang

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:

> On Monday, 16 July 2007 00:42, david@lang.hm wrote:
>> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
>>>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
>>>>
>>>>>    The ACPI specification requires us to invoke some global ACPI methods
>>>>>    during the hibernation and during the restore.  Moreover, the ordering of
>>>>>    code related to these ACPI methods may not be arbitrary (eg. some of
>>>>>    them have to be executed after devices are put into low power states etc.).
>>>>
>>>> for a pure hibernate mode, you will be powering off the box after saving
>>>> the suspend image. why are there any special ACPI modes involved?
>>>
>>> Because, for example, on my machine the status of power supply (present
>>> vs not present) is not updated correctly after the restore if ACPI callbacks
>>> aren't used during the hibernation.  That's just experience and it's in line
>>> with the ACPI spec.
>>
>> so if a machine is actually powered off the /dev/suspend process won't
>> work?
>
> No, it sort of works as usual, but after the restore the platform is not in the
> correct state.

this is not hibernate as I and many others are thinking of it.

hibernate as we are thinking would work on basicly any hardware, including 
things with no ACPI or power savings support. and the system could be in 
hibernate mode for any time period.

for that matter, after a system is put into hibernate mode the system 
could be completely disassembled and any components replaced and the 
system would work after a resume (assuming you still have access to the 
suspend image)

>> remember that the system may run a different OS between the hibernate and
>> the resume, makeing any assumptions about what state the hardware is in
>> when you start the resume is a problem.
>
> True, that's problematic.

putting it mildly.

David Lang

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Alan Stern wrote:

> On Sun, 15 Jul 2007 david@lang.hm wrote:
>
>>> (1) Filesystems mounted before the hibernation are untouchable
>>>
>>>    When there's a memory snapshot, either in the form of a hibernation image,
>>>    or in the form of the "old" kernel and processes available to the "new"
>>>    kexeced kernel responsible for saving their memory, the filesystems mounted
>>>    before the hibernation should not be accessed, even for reading, because
>>>    that would cause their on-disk state to be inconsistent with the snapshot
>>>    and might lead to a filesystem corruption.
>>
>> AFAIK this is only the case with ext3, all other filesystems could be
>> accessed read-only safely
>>
>> this is arguably a bug with ext3 (and has been discussed as such), but
>> right now the ext3 team has decided not to change this bahavior so
>> hibernate needs to work around it. but don't mistake a work-around for a
>> single (admittedly very popular) filesystem with a hard and fast
>> directive.
>
> Isn't is possible to avoid this problem by mounting an ext3 filesystem
> as readonly ext2?  Provided the filesystem isn't dirty it should be
> doable.  (And provided the filesystem doesn't use any ext3 extensions
> that are incompatible with ext2.)

from the last discussion I saw on the kernel mailing list, no. the act of 
mounting the ext3 filesystem as ext2 read-only will change it as the 
unsupported extentions get turned off (and I think the journal contents at 
least are lost as part of this)

David Lang

Re: Hibernation considerations

[david]

On Sun, 15 Jul 2007, Alan Stern wrote:

> On Sun, 15 Jul 2007, Al Boldi wrote:
>
>> Alan Stern wrote:
>>> On Sun, 15 Jul 2007, Al Boldi wrote:
>
>>>> This should be the responsibility of the kexec'd hibernating kernel.
>>>> Note though in (6), the normal kernel takes care of preparing devices,
>>>> then the hibernating kernel dumps the image and either calls S4 or S3.
>>>> On resume from S3 it can immediately switch over to the normal kernel,
>>>> and from S4 the known bootup would occur.
>>>
>>> Is it really that simple?  Somehow I doubt it.  In order for some
>>> devices to remain available for the kexec'd kernel to use, they cannot
>>> be suspended at the ACPI level.  So the kexec'd kernel will have to
>>> handle the ACPI requirements for those devices.  Likewise, it would
>>> have to handle the ACPI interactions which need to be done after all
>>> devices are prepared for the transition to S3 or S4.
>>
>> Ok, after applying the latest kexec patches, I was able to use the kexec'd
>> kernel to suspend to ram and resume to the normal kernel, while working
>> under a full-blown X session.  It went without a hitch.  All that is needed
>> now are the dump/restore hibernation-image routines.
>
> That's exactly my point.  While doing suspend-to-RAM from a kexec'd
> kernel may be simple, saving the hibernation image will add
> complications.

suspend-to-RAM should not involve kexec, the only reason for doing the 
kexec to to get a seperate userspace to use for suspend-to-disk operations 
instead of trying to partially freeze the sustem and keep useing it.

David Lang

Re: Hibernation considerations

[Matthew Garrett]

On Sun, Jul 15, 2007 at 02:33:32PM +0200, Rafael J. Wysocki wrote:

(snip)

Many of these assumptions are based on the assumption that we want to 
save a full image of RAM. I'm not convinced that this is true. The two 
things that we need are application state and hardware state. 
Application state can clearly be saved without kernel involvement 
(though restoring some of it may need some help from the kernel...), so 
hardware state is a more interesting question.

The obvious argument for saving the entirity of memory is that we have 
no mechanism for picking apart hardware state from any other part of the 
kernel. In reality, we're looking at implementing a set of hibernation 
operations anyway - it would be possible to utilise those to save as 
much state as needed. You also get fringe benefits, like being able to 
freeze a process that's accessing a piece of flaky hardware, swap the 
card out (assuming hotplug PCI), restore some amount of state and then 
let the process continue.

I appreciate that this suggestion sounds kind of fragile and 
complicated, but I think that's true of most descriptions of suspend to 
disk :) The main benefit is that it means we can use the hibernation 
infrastructure for other purposes (checkpointing, swapping hardware, 
that kind of thing) and reduce the damage caused by users doing 
seemingly reasonable things (like suspending Linux, booting Windows and 
then writing to a shared partition...). 

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Matthew Garrett wrote:

> On Sun, Jul 15, 2007 at 02:33:32PM +0200, Rafael J. Wysocki wrote:
>
> (snip)
>
> Many of these assumptions are based on the assumption that we want to
> save a full image of RAM. I'm not convinced that this is true. The two
> things that we need are application state and hardware state.
> Application state can clearly be saved without kernel involvement
> (though restoring some of it may need some help from the kernel...), so
> hardware state is a more interesting question.

one other reason for saving all the ram is that some people want to save 
all the system caches so that the machine is just as responsive immediatly 
after the resume as it was before the hibernate.

> The obvious argument for saving the entirity of memory is that we have
> no mechanism for picking apart hardware state from any other part of the
> kernel. In reality, we're looking at implementing a set of hibernation
> operations anyway - it would be possible to utilise those to save as
> much state as needed. You also get fringe benefits, like being able to
> freeze a process that's accessing a piece of flaky hardware, swap the
> card out (assuming hotplug PCI), restore some amount of state and then
> let the process continue.
>
> I appreciate that this suggestion sounds kind of fragile and
> complicated, but I think that's true of most descriptions of suspend to
> disk :) The main benefit is that it means we can use the hibernation
> infrastructure for other purposes (checkpointing, swapping hardware,
> that kind of thing) and reduce the damage caused by users doing
> seemingly reasonable things (like suspending Linux, booting Windows and
> then writing to a shared partition...).

I see the order being a little different.

if anyone implements a reliable checkpoint/restore of applications then 
that could be used as to pause specific applicaitons, move applications 
from one machine to another, move applications from one kernel to another, 
and as a side effect, as a form of hibernation when you are willing to 
loose your cache in favor of storing as little info as possible (since you 
wouldn't need to store the cache memory or any kernel memory/state)

David Lang

Re: Hibernation considerations

[Al Boldi]

Alan Stern wrote:
> As for the VGA font, the effect is easy to see: Run setfont before
> hibernating; when you resume the original font will be back.  The
> kernel simply does not bother to save the VGA font information across a
> hibernate.

This could probably be handled by a device suspend/resume call; but this 
problem is not specific to kexec.

> > Ok, after applying the latest kexec patches, I was able to use the
> > kexec'd kernel to suspend to ram and resume to the normal kernel, while
> > working under a full-blown X session.  It went without a hitch.  All
> > that is needed now are the dump/restore hibernation-image routines.
>
> That's exactly my point.  While doing suspend-to-RAM from a kexec'd
> kernel may be simple, saving the hibernation image will add
> complications.

>From a kexec'd hibernation kernel pov, both S3 and S4 look conceptually 
exactly the same.  The only difference is, in S3 the memory is in memory and 
in S4 the memory is on storage.  All device handling is exactly the same, so 
if there is a problem with device handling between the kexec'd hibernation 
kernel and the normal kernel, then that would have made itself visible.

david@lang.hm wrote:
> suspend-to-RAM should not involve kexec, the only reason for doing the
> kexec to to get a seperate userspace to use for suspend-to-disk operations
> instead of trying to partially freeze the sustem and keep useing it.

Or you could do suspend-to-disk-and-RAM.  But in the above case, it was meant 
to test kexec compatibility with device suspend/resume calls.


Thanks!

--
Al

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]

>> Isn't is possible to avoid this problem by mounting an ext3 filesystem
>> as readonly ext2?  Provided the filesystem isn't dirty it should be
>> doable.  (And provided the filesystem doesn't use any ext3 extensions
>> that are incompatible with ext2.)

> from the last discussion I saw on the kernel mailing list, no. the act of
> mounting the ext3 filesystem as ext2 read-only will change it as the unsupported
> extentions get turned off (and I think the journal contents at least are lost as
> part of this)

The fact of the matter is that it really doesn't matter whether mounting
it read-only actually corrupts the data on disk or not.  Regardless, it
should not be done, because you are accessing a dirty filesystem that is
still in use, and consequently there are no guarantees that either the
metadata or the file contents are consistent.  It isn't necessary for
hibernation to be able to access mounted partitions anyway.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Al Boldi wrote:

>
> david@lang.hm wrote:
>> suspend-to-RAM should not involve kexec, the only reason for doing the
>> kexec to to get a seperate userspace to use for suspend-to-disk operations
>> instead of trying to partially freeze the sustem and keep useing it.
>
> Or you could do suspend-to-disk-and-RAM.  But in the above case, it was meant
> to test kexec compatibility with device suspend/resume calls.

the point I am trying to make here is that there is no reason that the 
kexec approach needs to do _any_ suspend/resume calls.

all that is needed is the ability of the new kernel to initialize the 
devices it needs.

suspend-to-disk-and-ram could be implemented as three 
seperate steps

1. suspend-to-disk

2. resume-from-disk

3. suspend-to-ram

followed by either

4. resume-from-ram

or

4. battery dies and loptop powers off completely

5. power-on boot.

6. resume-from-disk

all that you need to do is to make sure that the system doesn't run 
anything that would affect permanent media or the outside world between 
steps #2 and #3

yes it's more steps, but each step is logicly seperate, and each step will 
be excercised on a regular basis, so the combination of the steps will 
also be reliable.

this is far better then creating yet another way to pause the system that 
only a few people will use.

David Lang

Re: Hibernation considerations

[Rafael J. Wysocki]

On Monday, 16 July 2007 01:49, david@lang.hm wrote:
> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Monday, 16 July 2007 00:42, david@lang.hm wrote:
> >> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
> >>>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
> >>>>
> >>>>>    The ACPI specification requires us to invoke some global ACPI methods
> >>>>>    during the hibernation and during the restore.  Moreover, the ordering of
> >>>>>    code related to these ACPI methods may not be arbitrary (eg. some of
> >>>>>    them have to be executed after devices are put into low power states etc.).
> >>>>
> >>>> for a pure hibernate mode, you will be powering off the box after saving
> >>>> the suspend image. why are there any special ACPI modes involved?
> >>>
> >>> Because, for example, on my machine the status of power supply (present
> >>> vs not present) is not updated correctly after the restore if ACPI callbacks
> >>> aren't used during the hibernation.  That's just experience and it's in line
> >>> with the ACPI spec.
> >>
> >> so if a machine is actually powered off the /dev/suspend process won't
> >> work?
> >
> > No, it sort of works as usual, but after the restore the platform is not in the
> > correct state.
> 
> this is not hibernate as I and many others are thinking of it.
> 
> hibernate as we are thinking would work on basicly any hardware, including 
> things with no ACPI or power savings support. and the system could be in 
> hibernate mode for any time period.
> 
> for that matter, after a system is put into hibernate mode the system 
> could be completely disassembled and any components replaced and the 
> system would work after a resume (assuming you still have access to the 
> suspend image)

Well, this is not how ACPI defines the S4 sleep state.  If the system is in
S4, that corresponds to our hibernation, you are _not_ allowed to disassemble
it.

I've just done an experiment on my test desktop.  I had enabled suspend support
in the CMOS setup and afterwards I made Linux hibernate in the "platform" mode.
Then, when the system was powred on, the BIOS showed me a nice "Resume from
hibernation" screen that is not normally displayed during boot.  This clearly
means that some information has been preserved by the platform across the
hibernate/restore cycle.  We are supposed to handle that.

Greetngs,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Jim Crilly]

On 07/16/07 02:06:27PM +0200, Rafael J. Wysocki wrote:
> On Monday, 16 July 2007 01:49, david@lang.hm wrote:
> > On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> > 
> > > On Monday, 16 July 2007 00:42, david@lang.hm wrote:
> > >> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> > >>
> > >>> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
> > >>>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
> > >>>>
> > >>>>>    The ACPI specification requires us to invoke some global ACPI methods
> > >>>>>    during the hibernation and during the restore.  Moreover, the ordering of
> > >>>>>    code related to these ACPI methods may not be arbitrary (eg. some of
> > >>>>>    them have to be executed after devices are put into low power states etc.).
> > >>>>
> > >>>> for a pure hibernate mode, you will be powering off the box after saving
> > >>>> the suspend image. why are there any special ACPI modes involved?
> > >>>
> > >>> Because, for example, on my machine the status of power supply (present
> > >>> vs not present) is not updated correctly after the restore if ACPI callbacks
> > >>> aren't used during the hibernation.  That's just experience and it's in line
> > >>> with the ACPI spec.
> > >>
> > >> so if a machine is actually powered off the /dev/suspend process won't
> > >> work?
> > >
> > > No, it sort of works as usual, but after the restore the platform is not in the
> > > correct state.
> > 
> > this is not hibernate as I and many others are thinking of it.
> > 
> > hibernate as we are thinking would work on basicly any hardware, including 
> > things with no ACPI or power savings support. and the system could be in 
> > hibernate mode for any time period.
> > 
> > for that matter, after a system is put into hibernate mode the system 
> > could be completely disassembled and any components replaced and the 
> > system would work after a resume (assuming you still have access to the 
> > suspend image)
> 
> Well, this is not how ACPI defines the S4 sleep state.  If the system is in
> S4, that corresponds to our hibernation, you are _not_ allowed to disassemble
> it.
> 
> I've just done an experiment on my test desktop.  I had enabled suspend support
> in the CMOS setup and afterwards I made Linux hibernate in the "platform" mode.
> Then, when the system was powred on, the BIOS showed me a nice "Resume from
> hibernation" screen that is not normally displayed during boot.  This clearly
> means that some information has been preserved by the platform across the
> hibernate/restore cycle.  We are supposed to handle that.
> 

What I believe he's getting at is that Linux hibernation shouldn't be tied
to any ACPI states. Yes, when available and working most people will want
to enter ACPI S4 but we should still have the option of doing a normal
poweroff. With the latter method it would look just like regular power off/on
cycle to the firmware. And that would definitely be useful for things like
working around buggy ACPI implementations or supporting platforms that don't
do ACPI at all. That is the difference between the platform and shutdown
options in /sys/power/disk, isn't it?

> Greetngs,
> Rafael
> 

Jim.

Re: Hibernation considerations

[Al Boldi]

david@lang.hm wrote:
> On Mon, 16 Jul 2007, Al Boldi wrote:
> > david@lang.hm wrote:
> >> suspend-to-RAM should not involve kexec, the only reason for doing the
> >> kexec to to get a seperate userspace to use for suspend-to-disk
> >> operations instead of trying to partially freeze the sustem and keep
> >> useing it.
> >
> > Or you could do suspend-to-disk-and-RAM.  But in the above case, it was
> > meant to test kexec compatibility with device suspend/resume calls.
>
> the point I am trying to make here is that there is no reason that the
> kexec approach needs to do _any_ suspend/resume calls.

When you go through ACPI, then that implies calling suspend/resume calls.

> all that is needed is the ability of the new kernel to initialize the
> devices it needs.

We have to go through ACPI, for wakeup functions to succeed.  A simple 
power-off won't do.

> suspend-to-disk-and-ram could be implemented as three
> seperate steps
>
> 1. suspend-to-disk
>
> 2. resume-from-disk
>
> 3. suspend-to-ram
>
> followed by either
>
> 4. resume-from-ram
>
> or
>
> 4. battery dies and loptop powers off completely
>
> 5. power-on boot.
>
> 6. resume-from-disk
>
> all that you need to do is to make sure that the system doesn't run
> anything that would affect permanent media or the outside world between
> steps #2 and #3

Exactly, which is why your scheme would break down on #3, and that's why you 
need to call S3 from within the kexec'd hibernation kernel after saving the 
hibernation image.


Thanks!

--
Al

Re: Hibernation considerations

[Alan Stern]

On Mon, 16 Jul 2007, Nigel Cunningham wrote:

> > As I understand it, running a different OS between the hibernate and 
> > the resume would violate the ACPI spec.
> 
> Well then, I know one or two people who would argue that the ACPI spec is 
> faulty. :\

It's hard to argue against that.  In fact there are people who would 
argue that the ACPI spec is incomplete and inconsistent period, 
regardless of any questions about what happens between hibernate and 
resume.  :-)

Alan Stern

Re: Hibernation considerations

[Alan Stern]

On Sun, 15 Jul 2007 david@lang.hm wrote:

> then we need a third mode of operation.
> 
> mode 1: Suspend-to-ram
> 
>    the system is paused and put into a low-power mode but data remains in 
> memory and the system stays awake enough to keep the memory refreshed.
> 
> mode 2: new
> 
>    the system is paused, data is stored to permanent media, and the system 
> is put into a ultra-low power mode.
> 
> mode 3: hibernate
> 
>    the system is paused, data is stored to permanent media, and the system 
> is powered off
> 
> with mode 3 there are no requirements or limitations about what can be 
> done with the hardware before a resume (the resume could even take place 
> on a different piece of identical hardware)
> 
> mode 2 could be what you are talking about doing, although I don't see any 
> advantage of creating it in additon to mode 3, it doesn't use any less 
> power and it locks the system so that it can't be used for anything else 
> in the meantime. I guess if it was significantly faster to do then mode 3 
> there may be _some_ reason to consider it, but I don't see the speed 
> difference.

Part of the problem here is that ACPI already has its own terminology, 
and you're trying to invent a new one instead of using the existing 
one.

I agree, it would be good to have a non-ACPI-specific hibernation mode,
something which would look to ACPI like a normal shutdown.  But I'm not 
so sure this is possible.

You have to understand that the ACPI spec is weird and complex.  The
mere fact that you have written a system image to disk changes the way
ACPI regards the shutdown procedure.  Even though you may treat all the
devices and the rest of the hardware exactly the same, it's a different
operation as far as ACPI is concerned, with different requirements.

Yes, it's bizarre.  Why do you think so many people have complained so 
vehemently about ACPI for all these years?

Alan Stern

Re: Hibernation considerations

[Alan Stern]

On Mon, 16 Jul 2007, Al Boldi wrote:

> From a kexec'd hibernation kernel pov, both S3 and S4 look conceptually 
> exactly the same.  The only difference is, in S3 the memory is in memory and 
> in S4 the memory is on storage.  All device handling is exactly the same, so 
> if there is a problem with device handling between the kexec'd hibernation 
> kernel and the normal kernel, then that would have made itself visible.

You have contradicted yourself.  "In S3 the memory is in memory and in
S4 the memory is on storage".  How does the memory get onto storage?  
The kexec'd hibernation kernel writes it there.  To do so it accesses a 
storage device.

Consequently the device handling _cannot_ be exactly the same in S3 and
S4.

Alan Stern

Re: Hibernation considerations

[Rafael J. Wysocki]

On Monday, 16 July 2007 16:15, Alan Stern wrote:
> On Mon, 16 Jul 2007, Nigel Cunningham wrote:
> 
> > > As I understand it, running a different OS between the hibernate and 
> > > the resume would violate the ACPI spec.
> > 
> > Well then, I know one or two people who would argue that the ACPI spec is 
> > faulty. :\
> 
> It's hard to argue against that.  In fact there are people who would 
> argue that the ACPI spec is incomplete and inconsistent period, 
> regardless of any questions about what happens between hibernate and 
> resume.  :-)

That's true, but OTOH it's difficult to argue with my notebook's firmware
about that. ;-)

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Monday, 16 July 2007 14:38, Jim Crilly wrote:
> On 07/16/07 02:06:27PM +0200, Rafael J. Wysocki wrote:
> > On Monday, 16 July 2007 01:49, david@lang.hm wrote:
> > > On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> > > 
> > > > On Monday, 16 July 2007 00:42, david@lang.hm wrote:
> > > >> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
> > > >>
> > > >>> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
> > > >>>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
> > > >>>>
> > > >>>>>    The ACPI specification requires us to invoke some global ACPI methods
> > > >>>>>    during the hibernation and during the restore.  Moreover, the ordering of
> > > >>>>>    code related to these ACPI methods may not be arbitrary (eg. some of
> > > >>>>>    them have to be executed after devices are put into low power states etc.).
> > > >>>>
> > > >>>> for a pure hibernate mode, you will be powering off the box after saving
> > > >>>> the suspend image. why are there any special ACPI modes involved?
> > > >>>
> > > >>> Because, for example, on my machine the status of power supply (present
> > > >>> vs not present) is not updated correctly after the restore if ACPI callbacks
> > > >>> aren't used during the hibernation.  That's just experience and it's in line
> > > >>> with the ACPI spec.
> > > >>
> > > >> so if a machine is actually powered off the /dev/suspend process won't
> > > >> work?
> > > >
> > > > No, it sort of works as usual, but after the restore the platform is not in the
> > > > correct state.
> > > 
> > > this is not hibernate as I and many others are thinking of it.
> > > 
> > > hibernate as we are thinking would work on basicly any hardware, including 
> > > things with no ACPI or power savings support. and the system could be in 
> > > hibernate mode for any time period.
> > > 
> > > for that matter, after a system is put into hibernate mode the system 
> > > could be completely disassembled and any components replaced and the 
> > > system would work after a resume (assuming you still have access to the 
> > > suspend image)
> > 
> > Well, this is not how ACPI defines the S4 sleep state.  If the system is in
> > S4, that corresponds to our hibernation, you are _not_ allowed to disassemble
> > it.
> > 
> > I've just done an experiment on my test desktop.  I had enabled suspend support
> > in the CMOS setup and afterwards I made Linux hibernate in the "platform" mode.
> > Then, when the system was powred on, the BIOS showed me a nice "Resume from
> > hibernation" screen that is not normally displayed during boot.  This clearly
> > means that some information has been preserved by the platform across the
> > hibernate/restore cycle.  We are supposed to handle that.
> > 
> 
> What I believe he's getting at is that Linux hibernation shouldn't be tied
> to any ACPI states. Yes, when available and working most people will want
> to enter ACPI S4 but we should still have the option of doing a normal
> poweroff. With the latter method it would look just like regular power off/on
> cycle to the firmware. And that would definitely be useful for things like
> working around buggy ACPI implementations or supporting platforms that don't
> do ACPI at all. That is the difference between the platform and shutdown
> options in /sys/power/disk, isn't it?

Yes, but this is not my point.

The point is that there are systems that _require_ the ACPI handling to work
correctly after the restore and we need to that _that_ into consideration.

IOW, there are poeple for whom the non-ACPI framework won't work as expected.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Al Boldi]

Alan Stern wrote:
> On Mon, 16 Jul 2007, Al Boldi wrote:
> > From a kexec'd hibernation kernel pov, both S3 and S4 look conceptually
> > exactly the same.  The only difference is, in S3 the memory is in memory
> > and in S4 the memory is on storage.  All device handling is exactly the
> > same, so if there is a problem with device handling between the kexec'd
> > hibernation kernel and the normal kernel, then that would have made
> > itself visible.
>
> You have contradicted yourself.  "In S3 the memory is in memory and in
> S4 the memory is on storage".  How does the memory get onto storage?
> The kexec'd hibernation kernel writes it there.  To do so it accesses a
> storage device.
>
> Consequently the device handling _cannot_ be exactly the same in S3 and
> S4.

Ok, you should have read this in the context of suspending/resuming from/to 
the normal kernel, and in that case they are exactly the same, i.e. kexec -e 
for suspend and kexec -j for resume.

BTW, it would be really helpful if people would actually try the kexec 
hibernation patches, as this may yield a much more constructive discussion.


Thanks!

--
Al

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:

> On Monday, 16 July 2007 14:38, Jim Crilly wrote:
>> On 07/16/07 02:06:27PM +0200, Rafael J. Wysocki wrote:
>>> On Monday, 16 July 2007 01:49, david@lang.hm wrote:
>>>> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
>>>>
>>>>> On Monday, 16 July 2007 00:42, david@lang.hm wrote:
>>>>>> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
>>>>>>
>>>>>>> On Sunday, 15 July 2007 22:13, david@lang.hm wrote:
>>>>>>>> On Sun, 15 Jul 2007, Rafael J. Wysocki wrote:
>>>>>>>>
>>>>>>>>>    The ACPI specification requires us to invoke some global ACPI methods
>>>>>>>>>    during the hibernation and during the restore.  Moreover, the ordering of
>>>>>>>>>    code related to these ACPI methods may not be arbitrary (eg. some of
>>>>>>>>>    them have to be executed after devices are put into low power states etc.).
>>>>>>>>
>>>>>>>> for a pure hibernate mode, you will be powering off the box after saving
>>>>>>>> the suspend image. why are there any special ACPI modes involved?
>>>>>>>
>>>>>>> Because, for example, on my machine the status of power supply (present
>>>>>>> vs not present) is not updated correctly after the restore if ACPI callbacks
>>>>>>> aren't used during the hibernation.  That's just experience and it's in line
>>>>>>> with the ACPI spec.
>>>>>>
>>>>>> so if a machine is actually powered off the /dev/suspend process won't
>>>>>> work?
>>>>>
>>>>> No, it sort of works as usual, but after the restore the platform is not in the
>>>>> correct state.
>>>>
>>>> this is not hibernate as I and many others are thinking of it.
>>>>
>>>> hibernate as we are thinking would work on basicly any hardware, including
>>>> things with no ACPI or power savings support. and the system could be in
>>>> hibernate mode for any time period.
>>>>
>>>> for that matter, after a system is put into hibernate mode the system
>>>> could be completely disassembled and any components replaced and the
>>>> system would work after a resume (assuming you still have access to the
>>>> suspend image)
>>>
>>> Well, this is not how ACPI defines the S4 sleep state.  If the system is in
>>> S4, that corresponds to our hibernation, you are _not_ allowed to disassemble
>>> it.
>>>
>>> I've just done an experiment on my test desktop.  I had enabled suspend support
>>> in the CMOS setup and afterwards I made Linux hibernate in the "platform" mode.
>>> Then, when the system was powred on, the BIOS showed me a nice "Resume from
>>> hibernation" screen that is not normally displayed during boot.  This clearly
>>> means that some information has been preserved by the platform across the
>>> hibernate/restore cycle.  We are supposed to handle that.
>>>
>>
>> What I believe he's getting at is that Linux hibernation shouldn't be tied
>> to any ACPI states. Yes, when available and working most people will want
>> to enter ACPI S4 but we should still have the option of doing a normal
>> poweroff. With the latter method it would look just like regular power off/on
>> cycle to the firmware. And that would definitely be useful for things like
>> working around buggy ACPI implementations or supporting platforms that don't
>> do ACPI at all. That is the difference between the platform and shutdown
>> options in /sys/power/disk, isn't it?
>
> Yes, but this is not my point.
>
> The point is that there are systems that _require_ the ACPI handling to work
> correctly after the restore and we need to that _that_ into consideration.
>
> IOW, there are poeple for whom the non-ACPI framework won't work as expected.

why would the type of hibernate that I'm talking about (power off, not S4 
mode) not work on a box that has ACPI?

you are saying that it wouldn't work right after a restore, but that's 
only if the restored image is expecting to have it's devices in ACPI sleep 
modes. my answer is "don't do that then" S4 mode does not save power 
compared to powering off, and when doing a hibernate you should be 
prepared to run out of battery before you wake up, which would then look 
like a power-on

the S4 mode sounds like a deeper version of suspend-to-ram, with all of 
the limitations of STR becouse you still require power continuously and 
must remain in complete control of the machine.

if you want to do a suspend to S4 mode, go for it, but we need to have a 
different name for that then for the suspend-to-disk-and-poer-off mode 
that I thought was hibernate

David Lang

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Al Boldi wrote:

> david@lang.hm wrote:
>> On Mon, 16 Jul 2007, Al Boldi wrote:
>>> david@lang.hm wrote:
>>>> suspend-to-RAM should not involve kexec, the only reason for doing the
>>>> kexec to to get a seperate userspace to use for suspend-to-disk
>>>> operations instead of trying to partially freeze the sustem and keep
>>>> useing it.
>>>
>>> Or you could do suspend-to-disk-and-RAM.  But in the above case, it was
>>> meant to test kexec compatibility with device suspend/resume calls.
>>
>> the point I am trying to make here is that there is no reason that the
>> kexec approach needs to do _any_ suspend/resume calls.
>
> When you go through ACPI, then that implies calling suspend/resume calls.

then don't go through ACPI

>> all that is needed is the ability of the new kernel to initialize the
>> devices it needs.
>
> We have to go through ACPI, for wakeup functions to succeed.  A simple
> power-off won't do.

the kexec switch being posted requires ACPI be disabled, so it's clearly 
possible to switch kernels and initialize devices without ACPI

>> suspend-to-disk-and-ram could be implemented as three
>> seperate steps
>>
>> 1. suspend-to-disk
>>
>> 2. resume-from-disk
>>
>> 3. suspend-to-ram
>>
>> followed by either
>>
>> 4. resume-from-ram
>>
>> or
>>
>> 4. battery dies and loptop powers off completely
>>
>> 5. power-on boot.
>>
>> 6. resume-from-disk
>>
>> all that you need to do is to make sure that the system doesn't run
>> anything that would affect permanent media or the outside world between
>> steps #2 and #3
>
> Exactly, which is why your scheme would break down on #3, and that's why you
> need to call S3 from within the kexec'd hibernation kernel after saving the
> hibernation image.

when a kexec is called, how does the kernel know what to execute? 
something needs to tell it what to do, and I think that something is 
either something in the kexec image, or it's something passed as a 
parameter to that image.

all that would be needed to do #3 safely is to have the kernel that you 
restarted on #2 do a suspend-to-ram before it does anything else.

David Lang

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Al Boldi wrote:

> Alan Stern wrote:
>> On Mon, 16 Jul 2007, Al Boldi wrote:
>>> From a kexec'd hibernation kernel pov, both S3 and S4 look conceptually
>>> exactly the same.  The only difference is, in S3 the memory is in memory
>>> and in S4 the memory is on storage.  All device handling is exactly the
>>> same, so if there is a problem with device handling between the kexec'd
>>> hibernation kernel and the normal kernel, then that would have made
>>> itself visible.
>>
>> You have contradicted yourself.  "In S3 the memory is in memory and in
>> S4 the memory is on storage".  How does the memory get onto storage?
>> The kexec'd hibernation kernel writes it there.  To do so it accesses a
>> storage device.
>>
>> Consequently the device handling _cannot_ be exactly the same in S3 and
>> S4.
>
> Ok, you should have read this in the context of suspending/resuming from/to
> the normal kernel, and in that case they are exactly the same, i.e. kexec -e
> for suspend and kexec -j for resume.
>
> BTW, it would be really helpful if people would actually try the kexec
> hibernation patches, as this may yield a much more constructive discussion.

I would love to, but so far I don't see the nessasary pieces

once I kexec to the new kernel, how can it find out what pages of memory 
(and swap) need to be saved?

If i knew that then I could write a trivial perl program to save those 
pages with the appropriate headers to make a suspend file.

however, I'm now being told that that suspend file won't work if the 
machine is actually powered off, so there's a need to something more for 
the wake-up side of things as well.

David Lang

Re: Hibernation considerations

[david]

On Mon, 16 Jul 2007, Alan Stern wrote:

> On Sun, 15 Jul 2007 david@lang.hm wrote:
>
>> then we need a third mode of operation.
>>
>> mode 1: Suspend-to-ram
>>
>>    the system is paused and put into a low-power mode but data remains in
>> memory and the system stays awake enough to keep the memory refreshed.
>>
>> mode 2: new
>>
>>    the system is paused, data is stored to permanent media, and the system
>> is put into a ultra-low power mode.
>>
>> mode 3: hibernate
>>
>>    the system is paused, data is stored to permanent media, and the system
>> is powered off
>>
>> with mode 3 there are no requirements or limitations about what can be
>> done with the hardware before a resume (the resume could even take place
>> on a different piece of identical hardware)
>>
>> mode 2 could be what you are talking about doing, although I don't see any
>> advantage of creating it in additon to mode 3, it doesn't use any less
>> power and it locks the system so that it can't be used for anything else
>> in the meantime. I guess if it was significantly faster to do then mode 3
>> there may be _some_ reason to consider it, but I don't see the speed
>> difference.
>
> Part of the problem here is that ACPI already has its own terminology,
> and you're trying to invent a new one instead of using the existing
> one.
>
> I agree, it would be good to have a non-ACPI-specific hibernation mode,
> something which would look to ACPI like a normal shutdown.  But I'm not
> so sure this is possible.

why would it not be possible?

> You have to understand that the ACPI spec is weird and complex.  The
> mere fact that you have written a system image to disk changes the way
> ACPI regards the shutdown procedure.  Even though you may treat all the
> devices and the rest of the hardware exactly the same, it's a different
> operation as far as ACPI is concerned, with different requirements.
>
> Yes, it's bizarre.  Why do you think so many people have complained so
> vehemently about ACPI for all these years?

so let's act as if ACPI doesn't exist and make a suspend-to-disk that 
works without it and looks to ACPI like a complete power off/on cycle (but 
looks to the user like a suspend/resume cycle)

this should avoid all the headaches about ACPI completely becouse you just 
don't make any ACPI calls at all.

this will also work on any type of system, and it will work in the 
presence of dead batteries, failed components, booting other OS's, moveing 
the image to different hardware, etc.

I can't think of anything much more frustrating then thinking that I 
suspended a system and then discovering that becouse the battery went dead 
(a complete power loss) that the system wouldn't boot up properly. to me 
this would be a fairly common condition (when I'm mobile I use the machine 
until I am out of battery, then stop and it may be a long time (days) 
before I can charge the thing up again) this would not be a reliable 
suspend as far as I'm concerned.

for suspend-to-ram you have to worry about ACPI states and what you are 
doing with them, for suspend-to-disk you can ignore them and completely 
power the system off instead.

David Lang

Re: Hibernation considerations

[Matthew Garrett]

On Mon, Jul 16, 2007 at 09:28:13PM -0700, david@lang.hm wrote:

> why would the type of hibernate that I'm talking about (power off, not S4 
> mode) not work on a box that has ACPI?

Powering off rather than using S4 means you lose most wakeup device 
support. That would be a functional regression compared to the current 
code.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: Hibernation considerations

[Al Boldi]

david@lang.hm wrote:
> On Mon, 16 Jul 2007, Al Boldi wrote:
> > We have to go through ACPI, for wakeup functions to succeed.  A simple
> > power-off won't do.
>
> the kexec switch being posted requires ACPI be disabled, so it's clearly
> possible to switch kernels and initialize devices without ACPI

It's a given that kexec works in the absence of ACPI; what we have to handle 
is the ACPI states across kernel invocations, to ensure wakeup functions 
succeed.  If you don't need this, then just power off.

> >> suspend-to-disk-and-ram could be implemented as three
> >> seperate steps
> >>
> >> 1. suspend-to-disk
> >>
> >> 2. resume-from-disk
> >>
> >> 3. suspend-to-ram
> >>
> >> followed by either
> >>
> >> 4. resume-from-ram
> >>
> >> or
> >>
> >> 4. battery dies and loptop powers off completely
> >>
> >> 5. power-on boot.
> >>
> >> 6. resume-from-disk
> >>
> >> all that you need to do is to make sure that the system doesn't run
> >> anything that would affect permanent media or the outside world between
> >> steps #2 and #3
> >
> > Exactly, which is why your scheme would break down on #3, and that's why
> > you need to call S3 from within the kexec'd hibernation kernel after
> > saving the hibernation image.
>
> when a kexec is called, how does the kernel know what to execute?
> something needs to tell it what to do, and I think that something is
> either something in the kexec image, or it's something passed as a
> parameter to that image.
>
> all that would be needed to do #3 safely is to have the kernel that you
> restarted on #2 do a suspend-to-ram before it does anything else.

If you mean by kernel 'the normal kernel', then this won't work, because it 
would imply a change of state after saving its image.

If you mean by kernel 'the kexec'd hibernation kernel', then you wouldn't 
need to do #2, but rather do #3 right after dumping the image in #1.

[...insert from another post...]
> > BTW, it would be really helpful if people would actually try the kexec
> > hibernation patches, as this may yield a much more constructive
> > discussion.
>
> I would love to, but so far I don't see the nessasary pieces
>
> once I kexec to the new kernel, how can it find out what pages of memory
> (and swap) need to be saved?

No need to save the swap, all you need to do is to dump /dev/oldmem onto 
storage, and if that dump image is compatible with swsusp, then a normal 
kernel should be able to resume from this image via /dev/snapshot.


Thanks!

--
Al

Re: Hibernation considerations

[Alan Stern]

On Mon, 16 Jul 2007 david@lang.hm wrote:

> > I agree, it would be good to have a non-ACPI-specific hibernation mode,
> > something which would look to ACPI like a normal shutdown.  But I'm not
> > so sure this is possible.
> 
> why would it not be possible?

> I can't think of anything much more frustrating then thinking that I 
> suspended a system and then discovering that becouse the battery went dead 
> (a complete power loss) that the system wouldn't boot up properly. to me 
> this would be a fairly common condition (when I'm mobile I use the machine 
> until I am out of battery, then stop and it may be a long time (days) 
> before I can charge the thing up again) this would not be a reliable 
> suspend as far as I'm concerned.
> 
> for suspend-to-ram you have to worry about ACPI states and what you are 
> doing with them, for suspend-to-disk you can ignore them and completely 
> power the system off instead.

If the only problem with doing this would be lack of wakeup support
then I'm all for it.  There must be a lot of people who would like
their computers to hibernate with power drain as close to 0 as possible
and who don't care about remote wakeup.  In fact they might even prefer
not to have wakeup support, so the computer doesn't resume at
unexpected times.

Alan Stern

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 14:08, Al Boldi wrote:
> david@lang.hm wrote:
> > On Mon, 16 Jul 2007, Al Boldi wrote:
> > > We have to go through ACPI, for wakeup functions to succeed.  A simple
> > > power-off won't do.
> >
> > the kexec switch being posted requires ACPI be disabled, so it's clearly
> > possible to switch kernels and initialize devices without ACPI
> 
> It's a given that kexec works in the absence of ACPI; what we have to handle 
> is the ACPI states across kernel invocations, to ensure wakeup functions 
> succeed.  If you don't need this, then just power off.
> 
> > >> suspend-to-disk-and-ram could be implemented as three
> > >> seperate steps
> > >>
> > >> 1. suspend-to-disk
> > >>
> > >> 2. resume-from-disk
> > >>
> > >> 3. suspend-to-ram
> > >>
> > >> followed by either
> > >>
> > >> 4. resume-from-ram
> > >>
> > >> or
> > >>
> > >> 4. battery dies and loptop powers off completely
> > >>
> > >> 5. power-on boot.
> > >>
> > >> 6. resume-from-disk
> > >>
> > >> all that you need to do is to make sure that the system doesn't run
> > >> anything that would affect permanent media or the outside world between
> > >> steps #2 and #3
> > >
> > > Exactly, which is why your scheme would break down on #3, and that's why
> > > you need to call S3 from within the kexec'd hibernation kernel after
> > > saving the hibernation image.
> >
> > when a kexec is called, how does the kernel know what to execute?
> > something needs to tell it what to do, and I think that something is
> > either something in the kexec image, or it's something passed as a
> > parameter to that image.
> >
> > all that would be needed to do #3 safely is to have the kernel that you
> > restarted on #2 do a suspend-to-ram before it does anything else.
> 
> If you mean by kernel 'the normal kernel', then this won't work, because it 
> would imply a change of state after saving its image.
> 
> If you mean by kernel 'the kexec'd hibernation kernel', then you wouldn't 
> need to do #2, but rather do #3 right after dumping the image in #1.
> 
> [...insert from another post...]
> > > BTW, it would be really helpful if people would actually try the kexec
> > > hibernation patches, as this may yield a much more constructive
> > > discussion.
> >
> > I would love to, but so far I don't see the nessasary pieces
> >
> > once I kexec to the new kernel, how can it find out what pages of memory
> > (and swap) need to be saved?
> 
> No need to save the swap,

Correct.

> all you need to do is to dump /dev/oldmem onto storage,

I'm not sure of that.

> and if that dump image is compatible with swsusp,

No, it's not.  swusp additionally needs to know PFNs to restore the pages into.

> then a normal kernel should be able to resume from this image via
> /dev/snapshot. 

Nope.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
> On Mon, 16 Jul 2007 david@lang.hm wrote:
> 
> > > I agree, it would be good to have a non-ACPI-specific hibernation mode,
> > > something which would look to ACPI like a normal shutdown.  But I'm not
> > > so sure this is possible.
> > 
> > why would it not be possible?
> 
> > I can't think of anything much more frustrating then thinking that I 
> > suspended a system and then discovering that becouse the battery went dead 
> > (a complete power loss) that the system wouldn't boot up properly. to me 
> > this would be a fairly common condition (when I'm mobile I use the machine 
> > until I am out of battery, then stop and it may be a long time (days) 
> > before I can charge the thing up again) this would not be a reliable 
> > suspend as far as I'm concerned.
> > 
> > for suspend-to-ram you have to worry about ACPI states and what you are 
> > doing with them, for suspend-to-disk you can ignore them and completely 
> > power the system off instead.
> 
> If the only problem with doing this would be lack of wakeup support
> then I'm all for it.  There must be a lot of people who would like
> their computers to hibernate with power drain as close to 0 as possible
> and who don't care about remote wakeup.  In fact they might even prefer
> not to have wakeup support, so the computer doesn't resume at
> unexpected times.

I'm afraid of one thing, though.

If we create a framework without ACPI (well, ACPI needs to be enabled in the
kernel anyway for other reasons, like the ability to suspend to RAM) and then
it turns out that we have to add some ACPI hooks to it, that might be difficult
to do cleanly.

Thus, it seems reasonable to think of the ACPI handling in advance.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Matthew Garrett wrote:

> On Mon, Jul 16, 2007 at 09:28:13PM -0700, david@lang.hm wrote:
>
>> why would the type of hibernate that I'm talking about (power off, not S4
>> mode) not work on a box that has ACPI?
>
> Powering off rather than using S4 means you lose most wakeup device
> support. That would be a functional regression compared to the current
> code.

only if the kexec isn't able to initialize those devices.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Al Boldi wrote:

> david@lang.hm wrote:
>> On Mon, 16 Jul 2007, Al Boldi wrote:
>>> We have to go through ACPI, for wakeup functions to succeed.  A simple
>>> power-off won't do.
>>
>> the kexec switch being posted requires ACPI be disabled, so it's clearly
>> possible to switch kernels and initialize devices without ACPI
>
> It's a given that kexec works in the absence of ACPI; what we have to handle
> is the ACPI states across kernel invocations, to ensure wakeup functions
> succeed.  If you don't need this, then just power off.
>
>>>> suspend-to-disk-and-ram could be implemented as three
>>>> seperate steps
>>>>
>>>> 1. suspend-to-disk
>>>>
>>>> 2. resume-from-disk
>>>>
>>>> 3. suspend-to-ram
>>>>
>>>> followed by either
>>>>
>>>> 4. resume-from-ram
>>>>
>>>> or
>>>>
>>>> 4. battery dies and loptop powers off completely
>>>>
>>>> 5. power-on boot.
>>>>
>>>> 6. resume-from-disk
>>>>
>>>> all that you need to do is to make sure that the system doesn't run
>>>> anything that would affect permanent media or the outside world between
>>>> steps #2 and #3
>>>
>>> Exactly, which is why your scheme would break down on #3, and that's why
>>> you need to call S3 from within the kexec'd hibernation kernel after
>>> saving the hibernation image.
>>
>> when a kexec is called, how does the kernel know what to execute?
>> something needs to tell it what to do, and I think that something is
>> either something in the kexec image, or it's something passed as a
>> parameter to that image.
>>
>> all that would be needed to do #3 safely is to have the kernel that you
>> restarted on #2 do a suspend-to-ram before it does anything else.
>
> If you mean by kernel 'the normal kernel', then this won't work, because it
> would imply a change of state after saving its image.

yes, it would change the state, but if it only changes the state in ways 
that aren't visable to the outside world why would it matter?

if power dies you restore from the disk image (useing the non ACPI 
approach), and the changes that you make are just lost

> If you mean by kernel 'the kexec'd hibernation kernel', then you wouldn't
> need to do #2, but rather do #3 right after dumping the image in #1.
>
> [...insert from another post...]
>>> BTW, it would be really helpful if people would actually try the kexec
>>> hibernation patches, as this may yield a much more constructive
>>> discussion.
>>
>> I would love to, but so far I don't see the nessasary pieces
>>
>> once I kexec to the new kernel, how can it find out what pages of memory
>> (and swap) need to be saved?
>
> No need to save the swap, all you need to do is to dump /dev/oldmem onto
> storage, and if that dump image is compatible with swsusp, then a normal
> kernel should be able to resume from this image via /dev/snapshot.

Rafael is saying that there's more involved, you can't just dump 
/dev/oldmem, you have to avoid specific pages.

as for swap, saving that may be required, depending on how clean you want 
toleave the box for other OS's in the meantime.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
>> On Mon, 16 Jul 2007 david@lang.hm wrote:
>>
>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
>>>> something which would look to ACPI like a normal shutdown.  But I'm not
>>>> so sure this is possible.
>>>
>>> why would it not be possible?
>>
>>> I can't think of anything much more frustrating then thinking that I
>>> suspended a system and then discovering that becouse the battery went dead
>>> (a complete power loss) that the system wouldn't boot up properly. to me
>>> this would be a fairly common condition (when I'm mobile I use the machine
>>> until I am out of battery, then stop and it may be a long time (days)
>>> before I can charge the thing up again) this would not be a reliable
>>> suspend as far as I'm concerned.
>>>
>>> for suspend-to-ram you have to worry about ACPI states and what you are
>>> doing with them, for suspend-to-disk you can ignore them and completely
>>> power the system off instead.
>>
>> If the only problem with doing this would be lack of wakeup support
>> then I'm all for it.  There must be a lot of people who would like
>> their computers to hibernate with power drain as close to 0 as possible
>> and who don't care about remote wakeup.  In fact they might even prefer
>> not to have wakeup support, so the computer doesn't resume at
>> unexpected times.
>
> I'm afraid of one thing, though.
>
> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> it turns out that we have to add some ACPI hooks to it, that might be difficult
> to do cleanly.

doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some 
people will want both, some won't.

at the moment kexec doesn't work with ACPI, that is a limitation that 
should be fixed, but makeing it able to work with ACPI enabled doesn't 
mean that it needs to be changed to depend on ACPI and it especially 
doesn't mean that it should pick up the limitations of the existing ACPI 
based hibernation approaches.

if there is no ACPI on the system it should work, if ther is ACPI on the 
system it should still work.

> Thus, it seems reasonable to think of the ACPI handling in advance.

but don't become dependant on ACPI.

David Lang

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
> >> On Mon, 16 Jul 2007 david@lang.hm wrote:
> >>
> >>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
> >>>> something which would look to ACPI like a normal shutdown.  But I'm not
> >>>> so sure this is possible.
> >>>
> >>> why would it not be possible?
> >>
> >>> I can't think of anything much more frustrating then thinking that I
> >>> suspended a system and then discovering that becouse the battery went dead
> >>> (a complete power loss) that the system wouldn't boot up properly. to me
> >>> this would be a fairly common condition (when I'm mobile I use the machine
> >>> until I am out of battery, then stop and it may be a long time (days)
> >>> before I can charge the thing up again) this would not be a reliable
> >>> suspend as far as I'm concerned.
> >>>
> >>> for suspend-to-ram you have to worry about ACPI states and what you are
> >>> doing with them, for suspend-to-disk you can ignore them and completely
> >>> power the system off instead.
> >>
> >> If the only problem with doing this would be lack of wakeup support
> >> then I'm all for it.  There must be a lot of people who would like
> >> their computers to hibernate with power drain as close to 0 as possible
> >> and who don't care about remote wakeup.  In fact they might even prefer
> >> not to have wakeup support, so the computer doesn't resume at
> >> unexpected times.
> >
> > I'm afraid of one thing, though.
> >
> > If we create a framework without ACPI (well, ACPI needs to be enabled in the
> > kernel anyway for other reasons, like the ability to suspend to RAM) and then
> > it turns out that we have to add some ACPI hooks to it, that might be difficult
> > to do cleanly.
> 
> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some 
> people will want both, some won't.
> 
> at the moment kexec doesn't work with ACPI, that is a limitation that 
> should be fixed, but makeing it able to work with ACPI enabled doesn't 
> mean that it needs to be changed to depend on ACPI and it especially 
> doesn't mean that it should pick up the limitations of the existing ACPI 
> based hibernation approaches.
> 
> if there is no ACPI on the system it should work, if ther is ACPI on the 
> system it should still work.
> 
> > Thus, it seems reasonable to think of the ACPI handling in advance.
> 
> but don't become dependant on ACPI.

Not dependent, but with the possibility of ACPI support taken into account.

Arguably you can create a framework that, for example, will not allow the user
to adjust the size of the image, but then adding such a functionality may
require you to change the entire design.  Same thing with ACPI.

I would rather avoid such pitfalls, if I could.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

[snip]

> I'm afraid of one thing, though.

> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> it turns out that we have to add some ACPI hooks to it, that might be difficult
> to do cleanly.

> Thus, it seems reasonable to think of the ACPI handling in advance.

As far as I understand, ACPI support is only useful for hibernate to the
extent that it allows some or all of the following features:

 - possibly shows a nice looking "hibernate" LED
 - possibly allows the BIOS to show something about hibernate
 - possibly allows the lid or keyboard to "wake up" (turn on) the system

Note that properly restoring device state (or even properly determining
whether on external/mains power vs. battery) on resume is not something
that should require special hibernate ACPI support, since it should be
possible to make hibernate (and in general it will be the case that
hibernate will) look exactly like a reboot to the BIOS/ACPI/devices.
The problem that you mentioned on your system regarding power source
information would seem to just be a problem with how ACPI is
reinitialized after resuming from hibernation, which is not at all
surprising since we know it (the use of driver calls for hibernate) is
currently broken in many ways.

It seems that enabling S4 mode should just be treated as a special
shutdown mode, independent of hibernate.  In practice, it may likely
only be useful in conjunction with hibernate, but there doesn't seem to
be any reason it needs to be coupled.

It would be useful to determine whether it is necessary to initialize
ACPI specially after "resuming" from S4 mode, though, or whether they
can be initialized normally (i.e. by a normal kernel for instance,
completely unaware of hibernate).  If they can be initialized normally,
then it seems that it is unnecessary to have any ACPI S4 mode support in
the resume path, and it can merely exist as a special shutdown mode.
Note that it seems a bit odd if ACPI can't be initialized normally after
resume from S4 (and still work), since the "load image" kernel
initializes everything normally before attempting to resume the
hibernated system.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
>>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
>>>>
>>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
>>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
>>>>>> so sure this is possible.
>>>>>
>>>>> why would it not be possible?
>>>>
>>>>> I can't think of anything much more frustrating then thinking that I
>>>>> suspended a system and then discovering that becouse the battery went dead
>>>>> (a complete power loss) that the system wouldn't boot up properly. to me
>>>>> this would be a fairly common condition (when I'm mobile I use the machine
>>>>> until I am out of battery, then stop and it may be a long time (days)
>>>>> before I can charge the thing up again) this would not be a reliable
>>>>> suspend as far as I'm concerned.
>>>>>
>>>>> for suspend-to-ram you have to worry about ACPI states and what you are
>>>>> doing with them, for suspend-to-disk you can ignore them and completely
>>>>> power the system off instead.
>>>>
>>>> If the only problem with doing this would be lack of wakeup support
>>>> then I'm all for it.  There must be a lot of people who would like
>>>> their computers to hibernate with power drain as close to 0 as possible
>>>> and who don't care about remote wakeup.  In fact they might even prefer
>>>> not to have wakeup support, so the computer doesn't resume at
>>>> unexpected times.
>>>
>>> I'm afraid of one thing, though.
>>>
>>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
>>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
>>> it turns out that we have to add some ACPI hooks to it, that might be difficult
>>> to do cleanly.
>>
>> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
>> people will want both, some won't.
>>
>> at the moment kexec doesn't work with ACPI, that is a limitation that
>> should be fixed, but makeing it able to work with ACPI enabled doesn't
>> mean that it needs to be changed to depend on ACPI and it especially
>> doesn't mean that it should pick up the limitations of the existing ACPI
>> based hibernation approaches.
>>
>> if there is no ACPI on the system it should work, if ther is ACPI on the
>> system it should still work.
>>
>>> Thus, it seems reasonable to think of the ACPI handling in advance.
>>
>> but don't become dependant on ACPI.
>
> Not dependent, but with the possibility of ACPI support taken into account.
>
> Arguably you can create a framework that, for example, will not allow the user
> to adjust the size of the image, but then adding such a functionality may
> require you to change the entire design.  Same thing with ACPI.
>
> I would rather avoid such pitfalls, if I could.

Ok, what is it that you think ACPI fundamentally changes in this process?

keep in mind that we are not makeing the assumption that the hardware 
will remain powered (even a little bit), or the assumption that nothing 
else will run on the hardware (eliminating any possibility that the 
hardware is in a known ACPI state)

David Lang

Re: Hibernation considerations

[Dr. David Alan Gilbert]

* david@lang.hm (david@lang.hm) wrote:
> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:

> >Encryption is possible with both the userland hibernation (aka uswsusp) and
> >TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a 
> >"must
> >have" feature for a framework to be generally useful (many users don't use 
> >it
> >anyway).
> 
> he's talking about the main system useing an encrypted device/partition, 
> not the hibernate image being stored encrypted.
> 
> This would require the main system 'forget' the keys when it does the 
> hinbernate and prompt for it again during the wake-up phase.

Indeed - although as I say I really don't know what you would do with
apps using the mounts at that point.   Still it seems like a 
sensible requrest from the security side.

Dave
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    | Running GNU/Linux on Alpha,68K| Happy  \ 
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Dr. David Alan Gilbert wrote:

> * david@lang.hm (david@lang.hm) wrote:
>> On Mon, 16 Jul 2007, Rafael J. Wysocki wrote:
>
>>> Encryption is possible with both the userland hibernation (aka uswsusp) and
>>> TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a
>>> "must
>>> have" feature for a framework to be generally useful (many users don't use
>>> it
>>> anyway).
>>
>> he's talking about the main system useing an encrypted device/partition,
>> not the hibernate image being stored encrypted.
>>
>> This would require the main system 'forget' the keys when it does the
>> hinbernate and prompt for it again during the wake-up phase.
>
> Indeed - although as I say I really don't know what you would do with
> apps using the mounts at that point.   Still it seems like a
> sensible requrest from the security side.

along the same lines, it would probably be a good idea to have the ability 
for a system to re-ask for the pass phrase periodicly while the system is 
running.

I see two possible approaches to these issues.

1. implement the periodic re-request capability, and when going into 
hibernate time-out any known pass phrases.

this is a lot of work overall, but the suspend portion is trivial so there 
would not be any suspend surprises.

2. flush the keyring on hibernate and have the resume process re-populate 
it (either by pokeing directly into the memory, or by providing a table 
that the resuming kernel reads from during wake-up to re-populate it)

this is less work, but it's all suspend related so it will get less 
testing.

David Lang

Re: Hibernation considerations

[Alan Stern]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> I'm afraid of one thing, though.
> 
> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> it turns out that we have to add some ACPI hooks to it, that might be difficult
> to do cleanly.
> 
> Thus, it seems reasonable to think of the ACPI handling in advance.

Absolutely.  This needs to be done in such a way that it will work:

	On platforms without ACPI;

	On platforms with ACPI where we do a non-ACPI type of shutdown
	to whatever extent it is possible (or perhaps an ACPI-aware
	shutdown rather than change to S4);

	On platforms with ACPI where we do an ACPI-aware transition
	to S4.

Rafael, for those of us who aren't thoroughly familiar with all the ins
and outs of the ACPI spec, could you please summarize a list of the
ACPI calls needed in the second and third cases above?  Indicate which
ones need to be done from within the original kernel and which should
be done from within a kexec'd hibernation kernel.


I'm still not entirely clear on how "suspend-to-both" ought to be
handled.  Presumably it will start off as a normal hibernation.  But
instead of shutting down, wouldn't the kexec'd kernel return to the
original kernel?  After all, the original kernel knows about all the
devices and can put them into a low-power state, while the kexec'd
kernel might not have sufficient information.

But what about the freezer?  The original reason for using kexec was to
avoid the need for the freezer.  With no freezer, while the original
kernel is busy powering down its devices, user tasks will be free to
carry out I/O -- which will make the memory snapshot inconsistent with
the on-disk data structures.

Alan Stern

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 19:06, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
> >> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
> >>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
> >>>>
> >>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
> >>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
> >>>>>> so sure this is possible.
> >>>>>
> >>>>> why would it not be possible?
> >>>>
> >>>>> I can't think of anything much more frustrating then thinking that I
> >>>>> suspended a system and then discovering that becouse the battery went dead
> >>>>> (a complete power loss) that the system wouldn't boot up properly. to me
> >>>>> this would be a fairly common condition (when I'm mobile I use the machine
> >>>>> until I am out of battery, then stop and it may be a long time (days)
> >>>>> before I can charge the thing up again) this would not be a reliable
> >>>>> suspend as far as I'm concerned.
> >>>>>
> >>>>> for suspend-to-ram you have to worry about ACPI states and what you are
> >>>>> doing with them, for suspend-to-disk you can ignore them and completely
> >>>>> power the system off instead.
> >>>>
> >>>> If the only problem with doing this would be lack of wakeup support
> >>>> then I'm all for it.  There must be a lot of people who would like
> >>>> their computers to hibernate with power drain as close to 0 as possible
> >>>> and who don't care about remote wakeup.  In fact they might even prefer
> >>>> not to have wakeup support, so the computer doesn't resume at
> >>>> unexpected times.
> >>>
> >>> I'm afraid of one thing, though.
> >>>
> >>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> >>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> >>> it turns out that we have to add some ACPI hooks to it, that might be difficult
> >>> to do cleanly.
> >>
> >> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
> >> people will want both, some won't.
> >>
> >> at the moment kexec doesn't work with ACPI, that is a limitation that
> >> should be fixed, but makeing it able to work with ACPI enabled doesn't
> >> mean that it needs to be changed to depend on ACPI and it especially
> >> doesn't mean that it should pick up the limitations of the existing ACPI
> >> based hibernation approaches.
> >>
> >> if there is no ACPI on the system it should work, if ther is ACPI on the
> >> system it should still work.
> >>
> >>> Thus, it seems reasonable to think of the ACPI handling in advance.
> >>
> >> but don't become dependant on ACPI.
> >
> > Not dependent, but with the possibility of ACPI support taken into account.
> >
> > Arguably you can create a framework that, for example, will not allow the user
> > to adjust the size of the image, but then adding such a functionality may
> > require you to change the entire design.  Same thing with ACPI.
> >
> > I would rather avoid such pitfalls, if I could.
> 
> Ok, what is it that you think ACPI fundamentally changes in this process?
> 
> keep in mind that we are not makeing the assumption that the hardware 
> will remain powered (even a little bit), or the assumption that nothing 
> else will run on the hardware (eliminating any possibility that the 
> hardware is in a known ACPI state)

Well, first, the fact is that _some_ systems _will_ be powered while in
hibernation (the majority of notebooks, for example) and you should assume
that the platform _may_ retain some information accross the hibernation/restore
cycle.  In that case you _should_ _not_ trash the information retained by the
platform.

Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
state as a result of the hibernation procedure.  In my opinion this may be done
after saving the image, but still this means, for example, that the
image-saving kernel needs to support ACPI.

Next, during the restore, we should first check if the image is present (and
valid) _without_ turning ACPI on (note that this is not done by the current
hibernation code and that leads to strange problems on some systems).  Then,
if the image is present (and valid), we should first load it, jump to the
hibernated kernel and _then_ turn ACPI on and execute the _BFS and
_WAK ACPI global methods (again, this is not done by the current code in that
order, which is wrong).  Only after that is the hibernated kernel supposed to
continue.

[Please refer to section 15.3 of the 3.0b ACPI spec for details.]

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 18:09, Jeremy Maitin-Shepard wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> [snip]
> 
> > I'm afraid of one thing, though.
> 
> > If we create a framework without ACPI (well, ACPI needs to be enabled in the
> > kernel anyway for other reasons, like the ability to suspend to RAM) and then
> > it turns out that we have to add some ACPI hooks to it, that might be difficult
> > to do cleanly.
> 
> > Thus, it seems reasonable to think of the ACPI handling in advance.
> 
> As far as I understand, ACPI support is only useful for hibernate to the
> extent that it allows some or all of the following features:
> 
>  - possibly shows a nice looking "hibernate" LED
>  - possibly allows the BIOS to show something about hibernate
>  - possibly allows the lid or keyboard to "wake up" (turn on) the system
> 
> Note that properly restoring device state (or even properly determining
> whether on external/mains power vs. battery) on resume is not something
> that should require special hibernate ACPI support, since it should be
> possible to make hibernate (and in general it will be the case that
> hibernate will) look exactly like a reboot to the BIOS/ACPI/devices.
> The problem that you mentioned on your system regarding power source
> information would seem to just be a problem with how ACPI is
> reinitialized after resuming from hibernation, which is not at all
> surprising since we know it (the use of driver calls for hibernate) is
> currently broken in many ways.
> 
> It seems that enabling S4 mode should just be treated as a special
> shutdown mode, independent of hibernate.  In practice, it may likely
> only be useful in conjunction with hibernate, but there doesn't seem to
> be any reason it needs to be coupled.
> 
> It would be useful to determine whether it is necessary to initialize
> ACPI specially after "resuming" from S4 mode, though, or whether they
> can be initialized normally (i.e. by a normal kernel for instance,
> completely unaware of hibernate).  If they can be initialized normally,
> then it seems that it is unnecessary to have any ACPI S4 mode support in
> the resume path, and it can merely exist as a special shutdown mode.
> Note that it seems a bit odd if ACPI can't be initialized normally after
> resume from S4 (and still work), since the "load image" kernel
> initializes everything normally before attempting to resume the
> hibernated system.

Unfortunately, this is more complicated (please see my recent reply to David).

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 20:32, Alan Stern wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > I'm afraid of one thing, though.
> > 
> > If we create a framework without ACPI (well, ACPI needs to be enabled in the
> > kernel anyway for other reasons, like the ability to suspend to RAM) and then
> > it turns out that we have to add some ACPI hooks to it, that might be difficult
> > to do cleanly.
> > 
> > Thus, it seems reasonable to think of the ACPI handling in advance.
> 
> Absolutely.  This needs to be done in such a way that it will work:
> 
> 	On platforms without ACPI;
> 
> 	On platforms with ACPI where we do a non-ACPI type of shutdown
> 	to whatever extent it is possible (or perhaps an ACPI-aware
> 	shutdown rather than change to S4);
> 
> 	On platforms with ACPI where we do an ACPI-aware transition
> 	to S4.
> 
> Rafael, for those of us who aren't thoroughly familiar with all the ins
> and outs of the ACPI spec, could you please summarize a list of the
> ACPI calls needed in the second and third cases above?  Indicate which
> ones need to be done from within the original kernel and which should
> be done from within a kexec'd hibernation kernel.

Sure.

In the third case (ie. transition to S4) we are supposed to do the following:

(1) Upon entering the sleep state, which IMO can be done _after_ the image
    has been saved:
  * figure out which devices can wake up
  * put devices into low power states (wake-up devices are placed in the Dx
    states compatible with the wake capability, the others are powered off)
  * execute the _PTS global control method
  * switch off the nonlocal CPUs (eg. nonboot CPUs on x86)
  * execute the _GTS global control method
  * set the GPE enable registers corresponding to the wake-up devices)
  * make the platform enter S4 (there's a well defined procedure for that)
  I think that this should be done by the image-saving kernel.

(2) Upon start-up (by which I mean what happens after the user has pressed
    the power button or something like that):
  * check if the image is present (and valid) _without_ enabling ACPI (we don't
    do that now, but I see no reason for not doing it in the new framework)
  * if the image is present (and valid), load it
  * turn on ACPI (unless already turned on by the BIOS, that is)
  * execute the _BFS global control method
  * execute the _WAK global control method
  * continue
  Here, the first two things should be done by the image-loading kernel, but
  the remaining operations have to be carried out by the restored kernel.

In the remaining two cases we generally don't need to bother with the global
ACPI handling.

> I'm still not entirely clear on how "suspend-to-both" ought to be
> handled.  Presumably it will start off as a normal hibernation.  But
> instead of shutting down, wouldn't the kexec'd kernel return to the
> original kernel?

No, I think the image-saving kernel should suspend.  Then, on resume the
platform will go back to it and it will jump back to the hibernated kernel.

> After all, the original kernel knows about all the devices and can put them
> into a low-power state, while the kexec'd kernel might not have sufficient
> information.

That's correct, but ...

> But what about the freezer?  The original reason for using kexec was to
> avoid the need for the freezer.  With no freezer, while the original
> kernel is busy powering down its devices, user tasks will be free to
> carry out I/O -- which will make the memory snapshot inconsistent with
> the on-disk data structures.

... we can't return to the hibernated kernel unless we are going to cancel the
hibernation.

That's why I think that for the suspend-to-both the image-saving kernel will
need to support the same set of devices as the hibernated kernel.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 19:06, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
>>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>>>
>>>>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
>>>>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
>>>>>>
>>>>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
>>>>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
>>>>>>>> so sure this is possible.
>>>>>>>
>>>>>>> why would it not be possible?
>>>>>>
>>>>>>> I can't think of anything much more frustrating then thinking that I
>>>>>>> suspended a system and then discovering that becouse the battery went dead
>>>>>>> (a complete power loss) that the system wouldn't boot up properly. to me
>>>>>>> this would be a fairly common condition (when I'm mobile I use the machine
>>>>>>> until I am out of battery, then stop and it may be a long time (days)
>>>>>>> before I can charge the thing up again) this would not be a reliable
>>>>>>> suspend as far as I'm concerned.
>>>>>>>
>>>>>>> for suspend-to-ram you have to worry about ACPI states and what you are
>>>>>>> doing with them, for suspend-to-disk you can ignore them and completely
>>>>>>> power the system off instead.
>>>>>>
>>>>>> If the only problem with doing this would be lack of wakeup support
>>>>>> then I'm all for it.  There must be a lot of people who would like
>>>>>> their computers to hibernate with power drain as close to 0 as possible
>>>>>> and who don't care about remote wakeup.  In fact they might even prefer
>>>>>> not to have wakeup support, so the computer doesn't resume at
>>>>>> unexpected times.
>>>>>
>>>>> I'm afraid of one thing, though.
>>>>>
>>>>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
>>>>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
>>>>> it turns out that we have to add some ACPI hooks to it, that might be difficult
>>>>> to do cleanly.
>>>>
>>>> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
>>>> people will want both, some won't.
>>>>
>>>> at the moment kexec doesn't work with ACPI, that is a limitation that
>>>> should be fixed, but makeing it able to work with ACPI enabled doesn't
>>>> mean that it needs to be changed to depend on ACPI and it especially
>>>> doesn't mean that it should pick up the limitations of the existing ACPI
>>>> based hibernation approaches.
>>>>
>>>> if there is no ACPI on the system it should work, if ther is ACPI on the
>>>> system it should still work.
>>>>
>>>>> Thus, it seems reasonable to think of the ACPI handling in advance.
>>>>
>>>> but don't become dependant on ACPI.
>>>
>>> Not dependent, but with the possibility of ACPI support taken into account.
>>>
>>> Arguably you can create a framework that, for example, will not allow the user
>>> to adjust the size of the image, but then adding such a functionality may
>>> require you to change the entire design.  Same thing with ACPI.
>>>
>>> I would rather avoid such pitfalls, if I could.
>>
>> Ok, what is it that you think ACPI fundamentally changes in this process?
>>
>> keep in mind that we are not makeing the assumption that the hardware
>> will remain powered (even a little bit), or the assumption that nothing
>> else will run on the hardware (eliminating any possibility that the
>> hardware is in a known ACPI state)
>
> Well, first, the fact is that _some_ systems _will_ be powered while in
> hibernation (the majority of notebooks, for example) and you should assume
> that the platform _may_ retain some information accross the hibernation/restore
> cycle.  In that case you _should_ _not_ trash the information retained by the
> platform.

no, systems that remain powered while asleep are a different type of 
suspend then ones that don't.

> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
> state as a result of the hibernation procedure.  In my opinion this may be done
> after saving the image, but still this means, for example, that the
> image-saving kernel needs to support ACPI.
>
> Next, during the restore, we should first check if the image is present (and
> valid) _without_ turning ACPI on (note that this is not done by the current
> hibernation code and that leads to strange problems on some systems).  Then,
> if the image is present (and valid), we should first load it, jump to the
> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
> _WAK ACPI global methods (again, this is not done by the current code in that
> order, which is wrong).  Only after that is the hibernated kernel supposed to
> continue.
>
> [Please refer to section 15.3 of the 3.0b ACPI spec for details.]

you are starting from the assumption that ACPI S4 mode should be used.

I'm saying that a suspend that uses ACPI S4 mode is fundamentally 
different from one that does a power off instead.

from my point of view the ACPI S4 sleep mode has far more in common with 
suspend-to-ram then with the suspend-to-disk that I'm talking about

non-ACPI hibernate

   since the box powers off
     it uses zero power while suspended
     another OS could be run before a resume
     hardware can be swapped, suspend image could be sent around the world to be restored on another system.
     restore makes no assumptions about the state of the hardware when it is restored
     restore is slower (full BIOS boot is required)
   should be able to work on just about any hardware (the limit is the ability to initialize the devices)


ACPI suspends

   since the box never completely powers off:
     a complete power failure breaks the suspend
     the OS must remain in control so other uses must be prevented.
     hardware must remain in the ACPI state from suspend until restore.
     restore can be faster (some initialization may be able to be skipped)
   requires ACPI hardware support

under the catagory of ACPI suspends you have

   fast suspend-to-ram (stop scheduling, put the CPU to sleep, as long as 
the memory keeps getting refreshed)
   slow suspend-to-ram (stop scheduling, put as much of the hardware as 
possible to sleep, including spinning down disks and other things that 
take a while to undo)
   suspend-to-disk (stop scheduleing, copy the ram somewhere so that it 
doesn't need to be refreshed, put everything into low-power mode)

   and there are probably quite a few others as well. but they are all in 
the same family in that you have to worry about ACPI states, and they all 
have the same restrictions on what can happen between suspend and resume

the non-ACPI hibernate behaves very differently, and for some people (and 
I think I am one of them) it will meet their needs better then _any_ of 
the ACPI suspends.

David Lang

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

[snip]

> Well, first, the fact is that _some_ systems _will_ be powered while in
> hibernation (the majority of notebooks, for example) and you should assume
> that the platform _may_ retain some information accross the hibernation/restore
> cycle.  In that case you _should_ _not_ trash the information retained by the
> platform.

I'm not sure the majority of notebook users will want wakeup support in
exchange for some power consumption while the system is off.  I think
many people would not consider the trouble of having to press the power
button instead of merely opening the lid too great.

Furthermore, S4 mode is of course also not suitable if you intend to
replace the battery while the system is hibernated.

It does seem that it is useful to provide S4 as an option, but certainly
just shutting down should also be an option on all systems.

> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
> state as a result of the hibernation procedure.  In my opinion this may be done
> after saving the image, but still this means, for example, that the
> image-saving kernel needs to support ACPI.

It seems that it most certainly must be done AFTER saving the image, as
the image obviously cannot be saved after entering S4 state, since S4
state is nearly the same as powering off completely and all memory will
be lost.

> Next, during the restore, we should first check if the image is present (and
> valid) _without_ turning ACPI on (note that this is not done by the current
> hibernation code and that leads to strange problems on some systems).  Then,
> if the image is present (and valid), we should first load it, jump to the
> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
> _WAK ACPI global methods (again, this is not done by the current code in that
> order, which is wrong).  Only after that is the hibernated kernel supposed to
> continue.

It seems that the implementation of that behavior for Linux cannot be
quite so simple, since resume from hibernation is driven (in general)
from an initrd/initramfs rather than directly from the kernel
initialization sequence, in order to support modular drivers and
features like DM and LVM.

Thus, there would have to be a new "delay_acpi_initialization" kernel
command-line option.  Additionally, there would be a sysfs interface to
tell the kernel to proceed with the ACPI initialization as normal.  This
would be used by an initrd/initramfs after determining that a resume
from hibernate will not be done.  If a resume from hibernate is done,
this hook won't be used, and instead the resumed kernel will call the
ACPI hibernate resume stuff if S4 state was used; otherwise, the resumed
kernel will just re-initialize ACPI as normal.  Also, if the in-kernel
code for checking if a resume can be done does not find a hibernate
image, it will also invoke the delayed ACPI initialization.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Alan Stern wrote:

> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>
>> I'm afraid of one thing, though.
>>
>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
>> it turns out that we have to add some ACPI hooks to it, that might be difficult
>> to do cleanly.
>>
>> Thus, it seems reasonable to think of the ACPI handling in advance.
>
> Absolutely.  This needs to be done in such a way that it will work:
>
> 	On platforms without ACPI;
>
> 	On platforms with ACPI where we do a non-ACPI type of shutdown
> 	to whatever extent it is possible (or perhaps an ACPI-aware
> 	shutdown rather than change to S4);
>
> 	On platforms with ACPI where we do an ACPI-aware transition
> 	to S4.
>
> Rafael, for those of us who aren't thoroughly familiar with all the ins
> and outs of the ACPI spec, could you please summarize a list of the
> ACPI calls needed in the second and third cases above?  Indicate which
> ones need to be done from within the original kernel and which should
> be done from within a kexec'd hibernation kernel.
>

there was just a link on slashdot toa primer on the subject of power 
management

http://www.techarp.com/showarticle.aspx?artno=420

>
> I'm still not entirely clear on how "suspend-to-both" ought to be
> handled.  Presumably it will start off as a normal hibernation.  But
> instead of shutting down, wouldn't the kexec'd kernel return to the
> original kernel?  After all, the original kernel knows about all the
> devices and can put them into a low-power state, while the kexec'd
> kernel might not have sufficient information.

this is what I'm thinking, but the issue here is that the original kernel 
needs to go into suspend-to-ram mode instead of resuming operation. per 
the e-mail I got from Ying last night this should not be hard to 
implement.

> But what about the freezer?  The original reason for using kexec was to
> avoid the need for the freezer.  With no freezer, while the original
> kernel is busy powering down its devices, user tasks will be free to
> carry out I/O -- which will make the memory snapshot inconsistent with
> the on-disk data structures.

no, user tasks just don't get scheduled during shutdown.

the big problem with the freezer isn't stopping anything from happening, 
it's _selectivly_ stopping things.

with kexec you don't need to let any portion of the origional kernel or 
userspace operate so you don't have a problem.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 20:32, Alan Stern wrote:
>
>> I'm still not entirely clear on how "suspend-to-both" ought to be
>> handled.  Presumably it will start off as a normal hibernation.  But
>> instead of shutting down, wouldn't the kexec'd kernel return to the
>> original kernel?
>
> No, I think the image-saving kernel should suspend.  Then, on resume the
> platform will go back to it and it will jump back to the hibernated kernel.
>
>> After all, the original kernel knows about all the devices and can put them
>> into a low-power state, while the kexec'd kernel might not have sufficient
>> information.
>
> That's correct, but ...
>
>> But what about the freezer?  The original reason for using kexec was to
>> avoid the need for the freezer.  With no freezer, while the original
>> kernel is busy powering down its devices, user tasks will be free to
>> carry out I/O -- which will make the memory snapshot inconsistent with
>> the on-disk data structures.
>
> ... we can't return to the hibernated kernel unless we are going to cancel the
> hibernation.

this is where we disagree.

why not? if all that the hibernated kernel does is to suspend-to-ram and 
makes no changes to disks or TCP connections anything that it does do 
would be lost if power were to fail and you instead did a restore from 
disk.

there is only a problem if something takes place that would prevent the 
restore-from-disk from working. if this is done in a non-ACPI way that 
will work across a power cycle you don't have to worry about the hardware 
state not matching anyway.

> That's why I think that for the suspend-to-both the image-saving kernel will
> need to support the same set of devices as the hibernated kernel.

suspend-to-both doesn't really make sense if the suspend-to-disk portion 
is useing the ACPI S4 mode.

if you don't run out of power you will restore-from-ram

if you do run out of power the restore-from-disk won't work either becouse 
devices are not in the right ACPI states.

David Lang

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

[snip]

>> Rafael, for those of us who aren't thoroughly familiar with all the ins
>> and outs of the ACPI spec, could you please summarize a list of the
>> ACPI calls needed in the second and third cases above?  Indicate which
>> ones need to be done from within the original kernel and which should
>> be done from within a kexec'd hibernation kernel.

> Sure.

> In the third case (ie. transition to S4) we are supposed to do the following:

> (1) Upon entering the sleep state, which IMO can be done _after_ the image
>     has been saved:

I assume you mean "in order to enter the sleep state", rather than "upon
entering the sleep state".  I still don't understand what you mean by
"which IMO can be done _after_ the image has been saved"; as far as I
understand, the last step of this process, "make the platform enter S4",
is almost like a shutdown as far as the kernel is concerned (except for
the tiny detail of having to call those special ACPI methods on resume);
consequently, it would seem that nothing can be done after that step.

>   * figure out which devices can wake up
>   * put devices into low power states (wake-up devices are placed in the Dx
>     states compatible with the wake capability, the others are powered off)
>   * execute the _PTS global control method
>   * switch off the nonlocal CPUs (eg. nonboot CPUs on x86)
>   * execute the _GTS global control method
>   * set the GPE enable registers corresponding to the wake-up devices)
>   * make the platform enter S4 (there's a well defined procedure for that)
>   I think that this should be done by the image-saving kernel.

I agree.

> (2) Upon start-up (by which I mean what happens after the user has pressed
>     the power button or something like that):
>   * check if the image is present (and valid) _without_ enabling ACPI (we don't
>     do that now, but I see no reason for not doing it in the new framework)
>   * if the image is present (and valid), load it
>   * turn on ACPI (unless already turned on by the BIOS, that is)
>   * execute the _BFS global control method
>   * execute the _WAK global control method
>   * continue
>   Here, the first two things should be done by the image-loading kernel, but
>   the remaining operations have to be carried out by the restored
>     kernel.

It doesn't seem like a problem for that to be the case, but out of
curiosity why do those methods need to be executed by the "restored"
kernel, rather than the "image loading" kernel.  Do they require some
information from ACPI-related kernel data structures that were populated
by the normal ACPI initialization?

[snip]

> ... we can't return to the hibernated kernel unless we are going to cancel the
> hibernation.

I agree.

> That's why I think that for the suspend-to-both the image-saving kernel will
> need to support the same set of devices as the hibernated kernel.

If all of the devices that the image writing kernel doesn't know about
have already been shut down/powered off by the hibernated kernel, then
does the "image writing" kernel still need to know about them in order
to suspend to RAM properly (i.e. without leaving some devices on wasting
power)?

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
>
> [snip]
>
>>> Rafael, for those of us who aren't thoroughly familiar with all the ins
>>> and outs of the ACPI spec, could you please summarize a list of the
>>> ACPI calls needed in the second and third cases above?  Indicate which
>>> ones need to be done from within the original kernel and which should
>>> be done from within a kexec'd hibernation kernel.
>
>> Sure.
>
>> In the third case (ie. transition to S4) we are supposed to do the following:
>
>> (1) Upon entering the sleep state, which IMO can be done _after_ the image
>>     has been saved:
>
> I assume you mean "in order to enter the sleep state", rather than "upon
> entering the sleep state".  I still don't understand what you mean by
> "which IMO can be done _after_ the image has been saved"; as far as I
> understand, the last step of this process, "make the platform enter S4",
> is almost like a shutdown as far as the kernel is concerned (except for
> the tiny detail of having to call those special ACPI methods on resume);
> consequently, it would seem that nothing can be done after that step.
>
>>   * figure out which devices can wake up
>>   * put devices into low power states (wake-up devices are placed in the Dx
>>     states compatible with the wake capability, the others are powered off)

this can't be done by the image-saving kernel if that kernel doesn't know 
about the device.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> david@lang.hm writes:
>
> [snip]
>
>> the non-ACPI hibernate behaves very differently, and for some people (and I
>> think I am one of them) it will meet their needs better then _any_ of the ACPI
>> suspends.
>
> It may have certain differences from the user point of view, but from
> the implementation view, it seems that it is nearly exactly the same.
> The only differences seem to be:
>
> - rather than shutting down, do whatever is necessary to stick the
>   system in S4 state.
>
> - make sure ACPI isn't initialized by the "load image" kernel
>
> - rather than "resume from hibernate" ACPI by initializing it normally,
>   issue the special hibernate-related methods.
>
> Thus, it seems that supporting ACPI S4 will have a very minimal affect
> on the hibernate implementation.

from what Rafael is saying supporting ACPI S4 mode requires a very 
fundamentally different restore approach, and in addition imposes very 
different restrictions on what can be done with the machine while it's 
suspended.

David Lang

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]

> the non-ACPI hibernate behaves very differently, and for some people (and I
> think I am one of them) it will meet their needs better then _any_ of the ACPI
> suspends.

It may have certain differences from the user point of view, but from
the implementation view, it seems that it is nearly exactly the same.
The only differences seem to be: 

 - rather than shutting down, do whatever is necessary to stick the
   system in S4 state.

 - make sure ACPI isn't initialized by the "load image" kernel

 - rather than "resume from hibernate" ACPI by initializing it normally,
   issue the special hibernate-related methods.

Thus, it seems that supporting ACPI S4 will have a very minimal affect
on the hibernate implementation.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
>
> [snip]
>
>> Well, first, the fact is that _some_ systems _will_ be powered while in
>> hibernation (the majority of notebooks, for example) and you should assume
>> that the platform _may_ retain some information accross the hibernation/restore
>> cycle.  In that case you _should_ _not_ trash the information retained by the
>> platform.
>
> I'm not sure the majority of notebook users will want wakeup support in
> exchange for some power consumption while the system is off.  I think
> many people would not consider the trouble of having to press the power
> button instead of merely opening the lid too great.

I think you mean to say that they would be willing to trade wakeup support 
in exchange for lower power consumption...

> Furthermore, S4 mode is of course also not suitable if you intend to
> replace the battery while the system is hibernated.

exactly, and how many users realize that replacing the battery, or 
allowing the battery to die completely will cause the restore to fail?

> It does seem that it is useful to provide S4 as an option, but certainly
> just shutting down should also be an option on all systems.

absolutly.

>> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
>> state as a result of the hibernation procedure.  In my opinion this may be done
>> after saving the image, but still this means, for example, that the
>> image-saving kernel needs to support ACPI.
>
> It seems that it most certainly must be done AFTER saving the image, as
> the image obviously cannot be saved after entering S4 state, since S4
> state is nearly the same as powering off completely and all memory will
> be lost.

and the image-saving kernel could kexec back to the main kernel if the 
main kernel then knows that it should execute the S4 mode instead of 
restoring.

>> Next, during the restore, we should first check if the image is present (and
>> valid) _without_ turning ACPI on (note that this is not done by the current
>> hibernation code and that leads to strange problems on some systems).  Then,
>> if the image is present (and valid), we should first load it, jump to the
>> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
>> _WAK ACPI global methods (again, this is not done by the current code in that
>> order, which is wrong).  Only after that is the hibernated kernel supposed to
>> continue.
>
> It seems that the implementation of that behavior for Linux cannot be
> quite so simple, since resume from hibernation is driven (in general)
> from an initrd/initramfs rather than directly from the kernel
> initialization sequence, in order to support modular drivers and
> features like DM and LVM.
>
> Thus, there would have to be a new "delay_acpi_initialization" kernel
> command-line option.  Additionally, there would be a sysfs interface to
> tell the kernel to proceed with the ACPI initialization as normal.  This
> would be used by an initrd/initramfs after determining that a resume
> from hibernate will not be done.  If a resume from hibernate is done,
> this hook won't be used, and instead the resumed kernel will call the
> ACPI hibernate resume stuff if S4 state was used; otherwise, the resumed
> kernel will just re-initialize ACPI as normal.  Also, if the in-kernel
> code for checking if a resume can be done does not find a hibernate
> image, it will also invoke the delayed ACPI initialization.

remember, one of the thoughts for a good hibernate implementation is that 
the image may be over the network, not on the local disk. you don't want 
the kernel trying to implment everything nessasary to get the image back.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 22:18, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Tuesday, 17 July 2007 19:06, david@lang.hm wrote:
>>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>>>
>>>>> On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
>>>>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>>>>>
>>>>>>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
>>>>>>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
>>>>>>>>
>>>>>>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
>>>>>>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
>>>>>>>>>> so sure this is possible.
>>>>>>>>>
>>>>>>>>> why would it not be possible?
>>>>>>>>
>>>>>>>>> I can't think of anything much more frustrating then thinking that I
>>>>>>>>> suspended a system and then discovering that becouse the battery went dead
>>>>>>>>> (a complete power loss) that the system wouldn't boot up properly. to me
>>>>>>>>> this would be a fairly common condition (when I'm mobile I use the machine
>>>>>>>>> until I am out of battery, then stop and it may be a long time (days)
>>>>>>>>> before I can charge the thing up again) this would not be a reliable
>>>>>>>>> suspend as far as I'm concerned.
>>>>>>>>>
>>>>>>>>> for suspend-to-ram you have to worry about ACPI states and what you are
>>>>>>>>> doing with them, for suspend-to-disk you can ignore them and completely
>>>>>>>>> power the system off instead.
>>>>>>>>
>>>>>>>> If the only problem with doing this would be lack of wakeup support
>>>>>>>> then I'm all for it.  There must be a lot of people who would like
>>>>>>>> their computers to hibernate with power drain as close to 0 as possible
>>>>>>>> and who don't care about remote wakeup.  In fact they might even prefer
>>>>>>>> not to have wakeup support, so the computer doesn't resume at
>>>>>>>> unexpected times.
>>>>>>>
>>>>>>> I'm afraid of one thing, though.
>>>>>>>
>>>>>>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
>>>>>>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
>>>>>>> it turns out that we have to add some ACPI hooks to it, that might be difficult
>>>>>>> to do cleanly.
>>>>>>
>>>>>> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
>>>>>> people will want both, some won't.
>>>>>>
>>>>>> at the moment kexec doesn't work with ACPI, that is a limitation that
>>>>>> should be fixed, but makeing it able to work with ACPI enabled doesn't
>>>>>> mean that it needs to be changed to depend on ACPI and it especially
>>>>>> doesn't mean that it should pick up the limitations of the existing ACPI
>>>>>> based hibernation approaches.
>>>>>>
>>>>>> if there is no ACPI on the system it should work, if ther is ACPI on the
>>>>>> system it should still work.
>>>>>>
>>>>>>> Thus, it seems reasonable to think of the ACPI handling in advance.
>>>>>>
>>>>>> but don't become dependant on ACPI.
>>>>>
>>>>> Not dependent, but with the possibility of ACPI support taken into account.
>>>>>
>>>>> Arguably you can create a framework that, for example, will not allow the user
>>>>> to adjust the size of the image, but then adding such a functionality may
>>>>> require you to change the entire design.  Same thing with ACPI.
>>>>>
>>>>> I would rather avoid such pitfalls, if I could.
>>>>
>>>> Ok, what is it that you think ACPI fundamentally changes in this process?
>>>>
>>>> keep in mind that we are not makeing the assumption that the hardware
>>>> will remain powered (even a little bit), or the assumption that nothing
>>>> else will run on the hardware (eliminating any possibility that the
>>>> hardware is in a known ACPI state)
>>>
>>> Well, first, the fact is that _some_ systems _will_ be powered while in
>>> hibernation (the majority of notebooks, for example) and you should assume
>>> that the platform _may_ retain some information accross the hibernation/restore
>>> cycle.  In that case you _should_ _not_ trash the information retained by the
>>> platform.
>>
>> no, systems that remain powered while asleep are a different type of
>> suspend then ones that don't.
>>
>>> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
>>> state as a result of the hibernation procedure.  In my opinion this may be done
>>> after saving the image, but still this means, for example, that the
>>> image-saving kernel needs to support ACPI.
>>>
>>> Next, during the restore, we should first check if the image is present (and
>>> valid) _without_ turning ACPI on (note that this is not done by the current
>>> hibernation code and that leads to strange problems on some systems).  Then,
>>> if the image is present (and valid), we should first load it, jump to the
>>> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
>>> _WAK ACPI global methods (again, this is not done by the current code in that
>>> order, which is wrong).  Only after that is the hibernated kernel supposed to
>>> continue.
>>>
>>> [Please refer to section 15.3 of the 3.0b ACPI spec for details.]
>>
>> you are starting from the assumption that ACPI S4 mode should be used.
>>
>> I'm saying that a suspend that uses ACPI S4 mode is fundamentally
>> different from one that does a power off instead.
>
> It is different, but not fundamentally.
>
>> from my point of view the ACPI S4 sleep mode has far more in common with
>> suspend-to-ram then with the suspend-to-disk that I'm talking about
>>
>> non-ACPI hibernate
>>
>>    since the box powers off
>>      it uses zero power while suspended
>>      another OS could be run before a resume
>>      hardware can be swapped, suspend image could be sent around the world to be restored on another system.
>>      restore makes no assumptions about the state of the hardware when it is restored
>>      restore is slower (full BIOS boot is required)
>>    should be able to work on just about any hardware (the limit is the ability to initialize the devices)
>>
>>
>> ACPI suspends
>>
>>    since the box never completely powers off:
>>      a complete power failure breaks the suspend
>>      the OS must remain in control so other uses must be prevented.
>>      hardware must remain in the ACPI state from suspend until restore.
>>      restore can be faster (some initialization may be able to be skipped)
>>    requires ACPI hardware support
>>
>> under the catagory of ACPI suspends you have
>>
>>    fast suspend-to-ram (stop scheduling, put the CPU to sleep, as long as
>> the memory keeps getting refreshed)
>>    slow suspend-to-ram (stop scheduling, put as much of the hardware as
>> possible to sleep, including spinning down disks and other things that
>> take a while to undo)
>>    suspend-to-disk (stop scheduleing, copy the ram somewhere so that it
>> doesn't need to be refreshed, put everything into low-power mode)
>>
>>    and there are probably quite a few others as well. but they are all in
>> the same family in that you have to worry about ACPI states, and they all
>> have the same restrictions on what can happen between suspend and resume
>>
>> the non-ACPI hibernate behaves very differently, and for some people (and
>> I think I am one of them) it will meet their needs better then _any_ of
>> the ACPI suspends.
>
> OTOH, there are many people who would want the ACPI suspends to be handled
> and they don't really care for the power-off-only hibernation.
>
> If you aren't going to support the ACPI hibernation, your framework will be
> incomplete and therefore not generally useful.

if you make the framework limited by the ACPI requriement, your framework 
will not be able to be used in all cases and is therefor incomplete and 
not generally useful.

see, I can make authoritative sounding declarations too. :-)

I agree that some people want ACPI suspends, but you don't seem to allow 
the fact that some people don't, and those people don't want to have the 
ACPI based limits. they _especially_ don't want those limits when it 
appears as if supporting those limits is what's preventing their much 
simpler case from working reliably.

I strongly suspect that the majority of users don't care about ACPI, they 
want to be able to pause and resume their machine. they may want a couple 
options for how fast the resume is (trading resume speed against how much 
power the system eats), but the deep sleep modes (suspend-to-disk, 
hibernate) probably have restore times that are close enough to each other 
that very few people would care enough to opt for a ACPI S4 mode that 
won't survive a loss of battery power over a non-ACPI mode that would.

David Lang

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]

> this is where we disagree.

> why not? if all that the hibernated kernel does is to suspend-to-ram and makes
> no changes to disks or TCP connections anything that it does do would be lost if
> power were to fail and you instead did a restore from disk.

It would be okay to switch the "hibernated" kernel in order to
e.g. initiate a suspend to ram provided that everything is done
atomically with interrupts off, for instance.  It is not clear, though,
that it is possible to suspend to ram atomically like that.

There is also the question of what state the devices will be in when
switching back from the "save image" kernel to the "hibernated" kernel.

[snip]

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]

>>> * figure out which devices can wake up
>>> * put devices into low power states (wake-up devices are placed in the Dx
>>> states compatible with the wake capability, the others are powered off)

> this can't be done by the image-saving kernel if that kernel doesn't know about
> the device.

The image-saving kernel can be made to know about all of the "wake up"
devices; all other devices should have already been powered off by the
"hibernated" kernel.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:18, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 19:06, david@lang.hm wrote:
> >> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
> >>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>>>
> >>>>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
> >>>>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
> >>>>>>
> >>>>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
> >>>>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
> >>>>>>>> so sure this is possible.
> >>>>>>>
> >>>>>>> why would it not be possible?
> >>>>>>
> >>>>>>> I can't think of anything much more frustrating then thinking that I
> >>>>>>> suspended a system and then discovering that becouse the battery went dead
> >>>>>>> (a complete power loss) that the system wouldn't boot up properly. to me
> >>>>>>> this would be a fairly common condition (when I'm mobile I use the machine
> >>>>>>> until I am out of battery, then stop and it may be a long time (days)
> >>>>>>> before I can charge the thing up again) this would not be a reliable
> >>>>>>> suspend as far as I'm concerned.
> >>>>>>>
> >>>>>>> for suspend-to-ram you have to worry about ACPI states and what you are
> >>>>>>> doing with them, for suspend-to-disk you can ignore them and completely
> >>>>>>> power the system off instead.
> >>>>>>
> >>>>>> If the only problem with doing this would be lack of wakeup support
> >>>>>> then I'm all for it.  There must be a lot of people who would like
> >>>>>> their computers to hibernate with power drain as close to 0 as possible
> >>>>>> and who don't care about remote wakeup.  In fact they might even prefer
> >>>>>> not to have wakeup support, so the computer doesn't resume at
> >>>>>> unexpected times.
> >>>>>
> >>>>> I'm afraid of one thing, though.
> >>>>>
> >>>>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> >>>>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> >>>>> it turns out that we have to add some ACPI hooks to it, that might be difficult
> >>>>> to do cleanly.
> >>>>
> >>>> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
> >>>> people will want both, some won't.
> >>>>
> >>>> at the moment kexec doesn't work with ACPI, that is a limitation that
> >>>> should be fixed, but makeing it able to work with ACPI enabled doesn't
> >>>> mean that it needs to be changed to depend on ACPI and it especially
> >>>> doesn't mean that it should pick up the limitations of the existing ACPI
> >>>> based hibernation approaches.
> >>>>
> >>>> if there is no ACPI on the system it should work, if ther is ACPI on the
> >>>> system it should still work.
> >>>>
> >>>>> Thus, it seems reasonable to think of the ACPI handling in advance.
> >>>>
> >>>> but don't become dependant on ACPI.
> >>>
> >>> Not dependent, but with the possibility of ACPI support taken into account.
> >>>
> >>> Arguably you can create a framework that, for example, will not allow the user
> >>> to adjust the size of the image, but then adding such a functionality may
> >>> require you to change the entire design.  Same thing with ACPI.
> >>>
> >>> I would rather avoid such pitfalls, if I could.
> >>
> >> Ok, what is it that you think ACPI fundamentally changes in this process?
> >>
> >> keep in mind that we are not makeing the assumption that the hardware
> >> will remain powered (even a little bit), or the assumption that nothing
> >> else will run on the hardware (eliminating any possibility that the
> >> hardware is in a known ACPI state)
> >
> > Well, first, the fact is that _some_ systems _will_ be powered while in
> > hibernation (the majority of notebooks, for example) and you should assume
> > that the platform _may_ retain some information accross the hibernation/restore
> > cycle.  In that case you _should_ _not_ trash the information retained by the
> > platform.
> 
> no, systems that remain powered while asleep are a different type of 
> suspend then ones that don't.
> 
> > Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
> > state as a result of the hibernation procedure.  In my opinion this may be done
> > after saving the image, but still this means, for example, that the
> > image-saving kernel needs to support ACPI.
> >
> > Next, during the restore, we should first check if the image is present (and
> > valid) _without_ turning ACPI on (note that this is not done by the current
> > hibernation code and that leads to strange problems on some systems).  Then,
> > if the image is present (and valid), we should first load it, jump to the
> > hibernated kernel and _then_ turn ACPI on and execute the _BFS and
> > _WAK ACPI global methods (again, this is not done by the current code in that
> > order, which is wrong).  Only after that is the hibernated kernel supposed to
> > continue.
> >
> > [Please refer to section 15.3 of the 3.0b ACPI spec for details.]
> 
> you are starting from the assumption that ACPI S4 mode should be used.
> 
> I'm saying that a suspend that uses ACPI S4 mode is fundamentally 
> different from one that does a power off instead.

It is different, but not fundamentally.

> from my point of view the ACPI S4 sleep mode has far more in common with 
> suspend-to-ram then with the suspend-to-disk that I'm talking about
> 
> non-ACPI hibernate
> 
>    since the box powers off
>      it uses zero power while suspended
>      another OS could be run before a resume
>      hardware can be swapped, suspend image could be sent around the world to be restored on another system.
>      restore makes no assumptions about the state of the hardware when it is restored
>      restore is slower (full BIOS boot is required)
>    should be able to work on just about any hardware (the limit is the ability to initialize the devices)
> 
> 
> ACPI suspends
> 
>    since the box never completely powers off:
>      a complete power failure breaks the suspend
>      the OS must remain in control so other uses must be prevented.
>      hardware must remain in the ACPI state from suspend until restore.
>      restore can be faster (some initialization may be able to be skipped)
>    requires ACPI hardware support
> 
> under the catagory of ACPI suspends you have
> 
>    fast suspend-to-ram (stop scheduling, put the CPU to sleep, as long as 
> the memory keeps getting refreshed)
>    slow suspend-to-ram (stop scheduling, put as much of the hardware as 
> possible to sleep, including spinning down disks and other things that 
> take a while to undo)
>    suspend-to-disk (stop scheduleing, copy the ram somewhere so that it 
> doesn't need to be refreshed, put everything into low-power mode)
> 
>    and there are probably quite a few others as well. but they are all in 
> the same family in that you have to worry about ACPI states, and they all 
> have the same restrictions on what can happen between suspend and resume
> 
> the non-ACPI hibernate behaves very differently, and for some people (and 
> I think I am one of them) it will meet their needs better then _any_ of 
> the ACPI suspends.

OTOH, there are many people who would want the ACPI suspends to be handled
and they don't really care for the power-off-only hibernation.

If you aren't going to support the ACPI hibernation, your framework will be
incomplete and therefore not generally useful.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:39, Jeremy Maitin-Shepard wrote:
> david@lang.hm writes:
> 
> [snip]
> 
> > the non-ACPI hibernate behaves very differently, and for some people (and I
> > think I am one of them) it will meet their needs better then _any_ of the ACPI
> > suspends.
> 
> It may have certain differences from the user point of view, but from
> the implementation view, it seems that it is nearly exactly the same.
> The only differences seem to be: 
> 
>  - rather than shutting down, do whatever is necessary to stick the
>    system in S4 state.
> 
>  - make sure ACPI isn't initialized by the "load image" kernel
> 
>  - rather than "resume from hibernate" ACPI by initializing it normally,
>    issue the special hibernate-related methods.
> 
> Thus, it seems that supporting ACPI S4 will have a very minimal affect
> on the hibernate implementation.

Still, you need to take it into account.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:24, Jeremy Maitin-Shepard wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> [snip]
> 
> > Well, first, the fact is that _some_ systems _will_ be powered while in
> > hibernation (the majority of notebooks, for example) and you should assume
> > that the platform _may_ retain some information accross the hibernation/restore
> > cycle.  In that case you _should_ _not_ trash the information retained by the
> > platform.
> 
> I'm not sure the majority of notebook users will want wakeup support in
> exchange for some power consumption while the system is off.  I think
> many people would not consider the trouble of having to press the power
> button instead of merely opening the lid too great.
> 
> Furthermore, S4 mode is of course also not suitable if you intend to
> replace the battery while the system is hibernated.
> 
> It does seem that it is useful to provide S4 as an option, but certainly
> just shutting down should also be an option on all systems.
> 
> > Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
> > state as a result of the hibernation procedure.  In my opinion this may be done
> > after saving the image, but still this means, for example, that the
> > image-saving kernel needs to support ACPI.
> 
> It seems that it most certainly must be done AFTER saving the image, as
> the image obviously cannot be saved after entering S4 state, since S4
> state is nearly the same as powering off completely and all memory will
> be lost.
> 
> > Next, during the restore, we should first check if the image is present (and
> > valid) _without_ turning ACPI on (note that this is not done by the current
> > hibernation code and that leads to strange problems on some systems).  Then,
> > if the image is present (and valid), we should first load it, jump to the
> > hibernated kernel and _then_ turn ACPI on and execute the _BFS and
> > _WAK ACPI global methods (again, this is not done by the current code in that
> > order, which is wrong).  Only after that is the hibernated kernel supposed to
> > continue.
> 
> It seems that the implementation of that behavior for Linux cannot be
> quite so simple, since resume from hibernation is driven (in general)
> from an initrd/initramfs rather than directly from the kernel
> initialization sequence, in order to support modular drivers and
> features like DM and LVM.

That's correct.

> Thus, there would have to be a new "delay_acpi_initialization" kernel
> command-line option.  Additionally, there would be a sysfs interface to
> tell the kernel to proceed with the ACPI initialization as normal.  This
> would be used by an initrd/initramfs after determining that a resume
> from hibernate will not be done.  If a resume from hibernate is done,
> this hook won't be used, and instead the resumed kernel will call the
> ACPI hibernate resume stuff if S4 state was used; otherwise, the resumed
> kernel will just re-initialize ACPI as normal.  Also, if the in-kernel
> code for checking if a resume can be done does not find a hibernate
> image, it will also invoke the delayed ACPI initialization.

Yes, something like this.

My point is, though, that it really requires some thought and needs to be
remebered about.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> david@lang.hm writes:
>
> [snip]
>
>> this is where we disagree.
>
>> why not? if all that the hibernated kernel does is to suspend-to-ram and makes
>> no changes to disks or TCP connections anything that it does do would be lost if
>> power were to fail and you instead did a restore from disk.
>
> It would be okay to switch the "hibernated" kernel in order to
> e.g. initiate a suspend to ram provided that everything is done
> atomically with interrupts off, for instance.  It is not clear, though,
> that it is possible to suspend to ram atomically like that.

why would it neeed to be with interrupts off?

I am arguing that it wouldn't matter if the "hibernated" kernel changed 
every bit of ram, as long as it didn't change anything that would be 
visable when the ram is overwritten by the saved image.

> There is also the question of what state the devices will be in when
> switching back from the "save image" kernel to the "hibernated" kernel.

yes, this is a key factor.

if the saved image assumes that the hardware is in some ACPI mode instead 
of re-initializeing the hardware then the suspend-to-ram operation could 
leave them in a different mode.

but if the saved image doesn't make assumptions about the hardware modes 
and initializes the hardware then it shouldn't matter.

David Lang

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> david@lang.hm writes:
>
> [snip]
>
>>>> * figure out which devices can wake up
>>>> * put devices into low power states (wake-up devices are placed in the Dx
>>>> states compatible with the wake capability, the others are powered off)
>
>> this can't be done by the image-saving kernel if that kernel doesn't know about
>> the device.
>
> The image-saving kernel can be made to know about all of the "wake up"
> devices; all other devices should have already been powered off by the
> "hibernated" kernel.

not nessasarily.

for example, you don't want the "hibernated" kernel to spin down your 
disks in the general case, but your image-writing kernel may not have 
drivers in it to talk to some of the disks. when things are powered 
off this just isn't an issue, but if you use S4 mode instead it is.

David Lang

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:34, Jeremy Maitin-Shepard wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> [snip]
> 
> >> Rafael, for those of us who aren't thoroughly familiar with all the ins
> >> and outs of the ACPI spec, could you please summarize a list of the
> >> ACPI calls needed in the second and third cases above?  Indicate which
> >> ones need to be done from within the original kernel and which should
> >> be done from within a kexec'd hibernation kernel.
> 
> > Sure.
> 
> > In the third case (ie. transition to S4) we are supposed to do the following:
> 
> > (1) Upon entering the sleep state, which IMO can be done _after_ the image
> >     has been saved:
> 
> I assume you mean "in order to enter the sleep state", rather than "upon
> entering the sleep state".

Yes, that seems to be more accurate.

> I still don't understand what you mean by 
> "which IMO can be done _after_ the image has been saved"; as far as I
> understand, the last step of this process, "make the platform enter S4",
> is almost like a shutdown as far as the kernel is concerned (except for
> the tiny detail of having to call those special ACPI methods on resume);
> consequently, it would seem that nothing can be done after that step.

Well, the ACPI spec suggests to save the image somewhere after the devices
have been but into low power states, which is kind of unreasonable. :-)

> >   * figure out which devices can wake up
> >   * put devices into low power states (wake-up devices are placed in the Dx
> >     states compatible with the wake capability, the others are powered off)
> >   * execute the _PTS global control method
> >   * switch off the nonlocal CPUs (eg. nonboot CPUs on x86)
> >   * execute the _GTS global control method
> >   * set the GPE enable registers corresponding to the wake-up devices)
> >   * make the platform enter S4 (there's a well defined procedure for that)
> >   I think that this should be done by the image-saving kernel.
> 
> I agree.
> 
> > (2) Upon start-up (by which I mean what happens after the user has pressed
> >     the power button or something like that):
> >   * check if the image is present (and valid) _without_ enabling ACPI (we don't
> >     do that now, but I see no reason for not doing it in the new framework)
> >   * if the image is present (and valid), load it
> >   * turn on ACPI (unless already turned on by the BIOS, that is)
> >   * execute the _BFS global control method
> >   * execute the _WAK global control method
> >   * continue
> >   Here, the first two things should be done by the image-loading kernel, but
> >   the remaining operations have to be carried out by the restored
> >     kernel.
> 
> It doesn't seem like a problem for that to be the case, but out of
> curiosity why do those methods need to be executed by the "restored"
> kernel, rather than the "image loading" kernel.  Do they require some
> information from ACPI-related kernel data structures that were populated
> by the normal ACPI initialization?

Well, there are some complications.  For example, _BFS and _WAK should be
executed with interrupts on (I'm told that the AML interpreter might not work
with interrupts disabled) and the nonboot CPUs should be offline while they
are being executed.  Perhaps we should also avoid playing with APICs and
things like that after executing _WAK, so it's better to execute them from the
restored kernel.

> [snip]
> 
> > ... we can't return to the hibernated kernel unless we are going to cancel the
> > hibernation.
> 
> I agree.
> 
> > That's why I think that for the suspend-to-both the image-saving kernel will
> > need to support the same set of devices as the hibernated kernel.
> 
> If all of the devices that the image writing kernel doesn't know about
> have already been shut down/powered off by the hibernated kernel, then
> does the "image writing" kernel still need to know about them in order
> to suspend to RAM properly (i.e. without leaving some devices on wasting
> power)?

For some devices it may be illegal to leave them powered on while entering
S3.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 22:34, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Tuesday, 17 July 2007 20:32, Alan Stern wrote:
>>>
>>>> I'm still not entirely clear on how "suspend-to-both" ought to be
>>>> handled.  Presumably it will start off as a normal hibernation.  But
>>>> instead of shutting down, wouldn't the kexec'd kernel return to the
>>>> original kernel?
>>>
>>> No, I think the image-saving kernel should suspend.  Then, on resume the
>>> platform will go back to it and it will jump back to the hibernated kernel.
>>>
>>>> After all, the original kernel knows about all the devices and can put them
>>>> into a low-power state, while the kexec'd kernel might not have sufficient
>>>> information.
>>>
>>> That's correct, but ...
>>>
>>>> But what about the freezer?  The original reason for using kexec was to
>>>> avoid the need for the freezer.  With no freezer, while the original
>>>> kernel is busy powering down its devices, user tasks will be free to
>>>> carry out I/O -- which will make the memory snapshot inconsistent with
>>>> the on-disk data structures.
>>>
>>> ... we can't return to the hibernated kernel unless we are going to cancel the
>>> hibernation.
>>
>> this is where we disagree.
>>
>> why not? if all that the hibernated kernel does is to suspend-to-ram and
>> makes no changes to disks or TCP connections anything that it does do
>> would be lost if power were to fail and you instead did a restore from
>> disk.
>
> How do you guarantee that no tasks are scheduled when you get back to the
> hibernated kernel?

just don't schedule any userspace tasks. all you need to do is to execute 
the ACPI sleep functions. you normally do that after stopping userspace 
anyway.

>> there is only a problem if something takes place that would prevent the
>> restore-from-disk from working. if this is done in a non-ACPI way that
>> will work across a power cycle you don't have to worry about the hardware
>> state not matching anyway.
>>
>>> That's why I think that for the suspend-to-both the image-saving kernel will
>>> need to support the same set of devices as the hibernated kernel.
>>
>> suspend-to-both doesn't really make sense if the suspend-to-disk portion
>> is useing the ACPI S4 mode.
>
> Well, not exactly.  If your battery runs out of power while you're suspended,
> but you have the image saved, it's still better to restore from the image, even
> if something may not work correctly after the restore, than to risk a loss of
> data.

if things don't work correctly you are still risking the loss of data, the 
user just doesn't know it.

>> if you don't run out of power you will restore-from-ram
>>
>> if you do run out of power the restore-from-disk won't work either becouse
>> devices are not in the right ACPI states.
>
> See above.
>
> Greetings,
> Rafael
>
>
>

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:27, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Alan Stern wrote:
> 
> > On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >
> >> I'm afraid of one thing, though.
> >>
> >> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> >> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> >> it turns out that we have to add some ACPI hooks to it, that might be difficult
> >> to do cleanly.
> >>
> >> Thus, it seems reasonable to think of the ACPI handling in advance.
> >
> > Absolutely.  This needs to be done in such a way that it will work:
> >
> > 	On platforms without ACPI;
> >
> > 	On platforms with ACPI where we do a non-ACPI type of shutdown
> > 	to whatever extent it is possible (or perhaps an ACPI-aware
> > 	shutdown rather than change to S4);
> >
> > 	On platforms with ACPI where we do an ACPI-aware transition
> > 	to S4.
> >
> > Rafael, for those of us who aren't thoroughly familiar with all the ins
> > and outs of the ACPI spec, could you please summarize a list of the
> > ACPI calls needed in the second and third cases above?  Indicate which
> > ones need to be done from within the original kernel and which should
> > be done from within a kexec'd hibernation kernel.
> >
> 
> there was just a link on slashdot toa primer on the subject of power 
> management
> 
> http://www.techarp.com/showarticle.aspx?artno=420
> 
> >
> > I'm still not entirely clear on how "suspend-to-both" ought to be
> > handled.  Presumably it will start off as a normal hibernation.  But
> > instead of shutting down, wouldn't the kexec'd kernel return to the
> > original kernel?  After all, the original kernel knows about all the
> > devices and can put them into a low-power state, while the kexec'd
> > kernel might not have sufficient information.
> 
> this is what I'm thinking, but the issue here is that the original kernel 
> needs to go into suspend-to-ram mode instead of resuming operation. per 
> the e-mail I got from Ying last night this should not be hard to 
> implement.
> 
> > But what about the freezer?  The original reason for using kexec was to
> > avoid the need for the freezer.  With no freezer, while the original
> > kernel is busy powering down its devices, user tasks will be free to
> > carry out I/O -- which will make the memory snapshot inconsistent with
> > the on-disk data structures.
> 
> no, user tasks just don't get scheduled during shutdown.
> 
> the big problem with the freezer isn't stopping anything from happening, 
> it's _selectivly_ stopping things.

It's selectively stopping kernel threads, which is just about right.  If you
that _this_ is a main problem with the freezer, then think again.

> with kexec you don't need to let any portion of the origional kernel or 
> userspace operate so you don't have a problem.

In fact, the main problem with the freezer is that it is a coarse-grained
solution.  Therefore, what I believe we should do is to evolve in the directoin
of more fine-grained solutions and gradually phase out the freezer.

The kexec-based approach is an attempt to replace one coarse-grained solution
(the freezer) with even more coarse-grained solution (stopping the entire
kernel with everything), which IMO doesn't address the main problem.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:34, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 20:32, Alan Stern wrote:
> >
> >> I'm still not entirely clear on how "suspend-to-both" ought to be
> >> handled.  Presumably it will start off as a normal hibernation.  But
> >> instead of shutting down, wouldn't the kexec'd kernel return to the
> >> original kernel?
> >
> > No, I think the image-saving kernel should suspend.  Then, on resume the
> > platform will go back to it and it will jump back to the hibernated kernel.
> >
> >> After all, the original kernel knows about all the devices and can put them
> >> into a low-power state, while the kexec'd kernel might not have sufficient
> >> information.
> >
> > That's correct, but ...
> >
> >> But what about the freezer?  The original reason for using kexec was to
> >> avoid the need for the freezer.  With no freezer, while the original
> >> kernel is busy powering down its devices, user tasks will be free to
> >> carry out I/O -- which will make the memory snapshot inconsistent with
> >> the on-disk data structures.
> >
> > ... we can't return to the hibernated kernel unless we are going to cancel the
> > hibernation.
> 
> this is where we disagree.
> 
> why not? if all that the hibernated kernel does is to suspend-to-ram and 
> makes no changes to disks or TCP connections anything that it does do 
> would be lost if power were to fail and you instead did a restore from 
> disk.

How do you guarantee that no tasks are scheduled when you get back to the
hibernated kernel?

> there is only a problem if something takes place that would prevent the 
> restore-from-disk from working. if this is done in a non-ACPI way that 
> will work across a power cycle you don't have to worry about the hardware 
> state not matching anyway.
> 
> > That's why I think that for the suspend-to-both the image-saving kernel will
> > need to support the same set of devices as the hibernated kernel.
> 
> suspend-to-both doesn't really make sense if the suspend-to-disk portion 
> is useing the ACPI S4 mode.

Well, not exactly.  If your battery runs out of power while you're suspended,
but you have the image saved, it's still better to restore from the image, even
if something may not work correctly after the restore, than to risk a loss of
data.

> if you don't run out of power you will restore-from-ram
> 
> if you do run out of power the restore-from-disk won't work either becouse 
> devices are not in the right ACPI states.

See above.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:37, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:
> 
> > "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> >
> > [snip]
> >
> >>> Rafael, for those of us who aren't thoroughly familiar with all the ins
> >>> and outs of the ACPI spec, could you please summarize a list of the
> >>> ACPI calls needed in the second and third cases above?  Indicate which
> >>> ones need to be done from within the original kernel and which should
> >>> be done from within a kexec'd hibernation kernel.
> >
> >> Sure.
> >
> >> In the third case (ie. transition to S4) we are supposed to do the following:
> >
> >> (1) Upon entering the sleep state, which IMO can be done _after_ the image
> >>     has been saved:
> >
> > I assume you mean "in order to enter the sleep state", rather than "upon
> > entering the sleep state".  I still don't understand what you mean by
> > "which IMO can be done _after_ the image has been saved"; as far as I
> > understand, the last step of this process, "make the platform enter S4",
> > is almost like a shutdown as far as the kernel is concerned (except for
> > the tiny detail of having to call those special ACPI methods on resume);
> > consequently, it would seem that nothing can be done after that step.
> >
> >>   * figure out which devices can wake up
> >>   * put devices into low power states (wake-up devices are placed in the Dx
> >>     states compatible with the wake capability, the others are powered off)
> 
> this can't be done by the image-saving kernel if that kernel doesn't know 
> about the device.

Good observation. :-)

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:

> david@lang.hm writes:
>
> [snip]
>
>>> How do you guarantee that no tasks are scheduled when you get back to the
>>> hibernated kernel?
>
>> just don't schedule any userspace tasks. all you need to do is to execute the
>> ACPI sleep functions. you normally do that after stopping userspace
>> anyway.
>
> What does "stopping userspace" mean?  You already said it does not mean
> disabling interrupts.  But using the freezer is also not an option,
> since the avoidance of that is the main reason for the kexec approach in
> the first place.

just don't schedule any non-kernel threads.

remember that the normal shutdown/suspend procedure is (from another 
related thread)

> >>sys_reboot(LINUX_REBOOT_CMD_KEXEC)
> >>     kernel_kexec
> >>         kernel_restart_prepare
> >>             device_shutdown
> >>         machine_shutdown
> >>         machine_kexec

I'm just saying that instead of going back to the normal operation of the 
kernel you just go directly to the new shutdown routine instead.

> [snip]
>
>>> Well, not exactly.  If your battery runs out of power while you're suspended,
>>> but you have the image saved, it's still better to restore from the image,
>> even
>>> if something may not work correctly after the restore, than to risk a loss of
>>> data.
>
>> if things don't work correctly you are still risking the loss of data, the user
>> just doesn't know it.
>
> It should be possible on any system to do a hibernate followed by a
> shutdown (and then resume properly, without any problems).  Thus, for
> handling suspend to both, you resume as if the system had been shutdown,
> rather than resuming as if the system came from S4.

I agree with this, but according to Rafael if the "hibernated" image is 
assuming that the devices were put into low-power mode by ACPI and you 
boot up instead the system doesn't work right.

David Lang

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]

>> How do you guarantee that no tasks are scheduled when you get back to the
>> hibernated kernel?

> just don't schedule any userspace tasks. all you need to do is to execute the
> ACPI sleep functions. you normally do that after stopping userspace
> anyway.

What does "stopping userspace" mean?  You already said it does not mean
disabling interrupts.  But using the freezer is also not an option,
since the avoidance of that is the main reason for the kexec approach in
the first place.

[snip]

>> Well, not exactly.  If your battery runs out of power while you're suspended,
>> but you have the image saved, it's still better to restore from the image,
> even
>> if something may not work correctly after the restore, than to risk a loss of
>> data.

> if things don't work correctly you are still risking the loss of data, the user
> just doesn't know it.

It should be possible on any system to do a hibernate followed by a
shutdown (and then resume properly, without any problems).  Thus, for
handling suspend to both, you resume as if the system had been shutdown,
rather than resuming as if the system came from S4.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 22:53, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 22:18, david@lang.hm wrote:
> >> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> On Tuesday, 17 July 2007 19:06, david@lang.hm wrote:
> >>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>>>
> >>>>> On Tuesday, 17 July 2007 17:29, david@lang.hm wrote:
> >>>>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>>>>>
> >>>>>>> On Tuesday, 17 July 2007 16:15, Alan Stern wrote:
> >>>>>>>> On Mon, 16 Jul 2007 david@lang.hm wrote:
> >>>>>>>>
> >>>>>>>>>> I agree, it would be good to have a non-ACPI-specific hibernation mode,
> >>>>>>>>>> something which would look to ACPI like a normal shutdown.  But I'm not
> >>>>>>>>>> so sure this is possible.
> >>>>>>>>>
> >>>>>>>>> why would it not be possible?
> >>>>>>>>
> >>>>>>>>> I can't think of anything much more frustrating then thinking that I
> >>>>>>>>> suspended a system and then discovering that becouse the battery went dead
> >>>>>>>>> (a complete power loss) that the system wouldn't boot up properly. to me
> >>>>>>>>> this would be a fairly common condition (when I'm mobile I use the machine
> >>>>>>>>> until I am out of battery, then stop and it may be a long time (days)
> >>>>>>>>> before I can charge the thing up again) this would not be a reliable
> >>>>>>>>> suspend as far as I'm concerned.
> >>>>>>>>>
> >>>>>>>>> for suspend-to-ram you have to worry about ACPI states and what you are
> >>>>>>>>> doing with them, for suspend-to-disk you can ignore them and completely
> >>>>>>>>> power the system off instead.
> >>>>>>>>
> >>>>>>>> If the only problem with doing this would be lack of wakeup support
> >>>>>>>> then I'm all for it.  There must be a lot of people who would like
> >>>>>>>> their computers to hibernate with power drain as close to 0 as possible
> >>>>>>>> and who don't care about remote wakeup.  In fact they might even prefer
> >>>>>>>> not to have wakeup support, so the computer doesn't resume at
> >>>>>>>> unexpected times.
> >>>>>>>
> >>>>>>> I'm afraid of one thing, though.
> >>>>>>>
> >>>>>>> If we create a framework without ACPI (well, ACPI needs to be enabled in the
> >>>>>>> kernel anyway for other reasons, like the ability to suspend to RAM) and then
> >>>>>>> it turns out that we have to add some ACPI hooks to it, that might be difficult
> >>>>>>> to do cleanly.
> >>>>>>
> >>>>>> doing suspend-to-ram should be orthoginal to doing hibernate-to-disk. some
> >>>>>> people will want both, some won't.
> >>>>>>
> >>>>>> at the moment kexec doesn't work with ACPI, that is a limitation that
> >>>>>> should be fixed, but makeing it able to work with ACPI enabled doesn't
> >>>>>> mean that it needs to be changed to depend on ACPI and it especially
> >>>>>> doesn't mean that it should pick up the limitations of the existing ACPI
> >>>>>> based hibernation approaches.
> >>>>>>
> >>>>>> if there is no ACPI on the system it should work, if ther is ACPI on the
> >>>>>> system it should still work.
> >>>>>>
> >>>>>>> Thus, it seems reasonable to think of the ACPI handling in advance.
> >>>>>>
> >>>>>> but don't become dependant on ACPI.
> >>>>>
> >>>>> Not dependent, but with the possibility of ACPI support taken into account.
> >>>>>
> >>>>> Arguably you can create a framework that, for example, will not allow the user
> >>>>> to adjust the size of the image, but then adding such a functionality may
> >>>>> require you to change the entire design.  Same thing with ACPI.
> >>>>>
> >>>>> I would rather avoid such pitfalls, if I could.
> >>>>
> >>>> Ok, what is it that you think ACPI fundamentally changes in this process?
> >>>>
> >>>> keep in mind that we are not makeing the assumption that the hardware
> >>>> will remain powered (even a little bit), or the assumption that nothing
> >>>> else will run on the hardware (eliminating any possibility that the
> >>>> hardware is in a known ACPI state)
> >>>
> >>> Well, first, the fact is that _some_ systems _will_ be powered while in
> >>> hibernation (the majority of notebooks, for example) and you should assume
> >>> that the platform _may_ retain some information accross the hibernation/restore
> >>> cycle.  In that case you _should_ _not_ trash the information retained by the
> >>> platform.
> >>
> >> no, systems that remain powered while asleep are a different type of
> >> suspend then ones that don't.
> >>
> >>> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
> >>> state as a result of the hibernation procedure.  In my opinion this may be done
> >>> after saving the image, but still this means, for example, that the
> >>> image-saving kernel needs to support ACPI.
> >>>
> >>> Next, during the restore, we should first check if the image is present (and
> >>> valid) _without_ turning ACPI on (note that this is not done by the current
> >>> hibernation code and that leads to strange problems on some systems).  Then,
> >>> if the image is present (and valid), we should first load it, jump to the
> >>> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
> >>> _WAK ACPI global methods (again, this is not done by the current code in that
> >>> order, which is wrong).  Only after that is the hibernated kernel supposed to
> >>> continue.
> >>>
> >>> [Please refer to section 15.3 of the 3.0b ACPI spec for details.]
> >>
> >> you are starting from the assumption that ACPI S4 mode should be used.
> >>
> >> I'm saying that a suspend that uses ACPI S4 mode is fundamentally
> >> different from one that does a power off instead.
> >
> > It is different, but not fundamentally.
> >
> >> from my point of view the ACPI S4 sleep mode has far more in common with
> >> suspend-to-ram then with the suspend-to-disk that I'm talking about
> >>
> >> non-ACPI hibernate
> >>
> >>    since the box powers off
> >>      it uses zero power while suspended
> >>      another OS could be run before a resume
> >>      hardware can be swapped, suspend image could be sent around the world to be restored on another system.
> >>      restore makes no assumptions about the state of the hardware when it is restored
> >>      restore is slower (full BIOS boot is required)
> >>    should be able to work on just about any hardware (the limit is the ability to initialize the devices)
> >>
> >>
> >> ACPI suspends
> >>
> >>    since the box never completely powers off:
> >>      a complete power failure breaks the suspend
> >>      the OS must remain in control so other uses must be prevented.
> >>      hardware must remain in the ACPI state from suspend until restore.
> >>      restore can be faster (some initialization may be able to be skipped)
> >>    requires ACPI hardware support
> >>
> >> under the catagory of ACPI suspends you have
> >>
> >>    fast suspend-to-ram (stop scheduling, put the CPU to sleep, as long as
> >> the memory keeps getting refreshed)
> >>    slow suspend-to-ram (stop scheduling, put as much of the hardware as
> >> possible to sleep, including spinning down disks and other things that
> >> take a while to undo)
> >>    suspend-to-disk (stop scheduleing, copy the ram somewhere so that it
> >> doesn't need to be refreshed, put everything into low-power mode)
> >>
> >>    and there are probably quite a few others as well. but they are all in
> >> the same family in that you have to worry about ACPI states, and they all
> >> have the same restrictions on what can happen between suspend and resume
> >>
> >> the non-ACPI hibernate behaves very differently, and for some people (and
> >> I think I am one of them) it will meet their needs better then _any_ of
> >> the ACPI suspends.
> >
> > OTOH, there are many people who would want the ACPI suspends to be handled
> > and they don't really care for the power-off-only hibernation.
> >
> > If you aren't going to support the ACPI hibernation, your framework will be
> > incomplete and therefore not generally useful.
> 
> if you make the framework limited by the ACPI requriement, your framework 
> will not be able to be used in all cases and is therefor incomplete and 
> not generally useful.
> 
> see, I can make authoritative sounding declarations too. :-)

Well, being able to support ACPI need not imply being unable to work in the
other cases.  Conversely, being able to work in non-ACPI cases need not imply
being unable to support ACPI.  Hence, you can support ACPI and be able to
work in the other cases at the same time.

Try again. ;-)

> I agree that some people want ACPI suspends, but you don't seem to allow 
> the fact that some people don't,

No, I do.

What I'm trying to say is that we should support the ACPI S4 transition and
resume, which DOES NOT MEAN that we should support ONLY that.

> and those people don't want to have the ACPI based limits.

There are NO ACPI LIMITS!  There only are things that you need to implement
if you're going to support ACPI, but they need not be used ALWAYS, no?

> they _especially_ don't want those limits when it appears as if supporting
> those limits is what's preventing their much simpler case from working
> reliably. 
> 
> I strongly suspect that the majority of users don't care about ACPI, they 
> want to be able to pause and resume their machine. they may want a couple 
> options for how fast the resume is (trading resume speed against how much 
> power the system eats), but the deep sleep modes (suspend-to-disk, 
> hibernate) probably have restore times that are close enough to each other 
> that very few people would care enough to opt for a ACPI S4 mode that 
> won't survive a loss of battery power over a non-ACPI mode that would.

Please don't argue like that, these are not arguments for avoiding the ACPI
support.  Arguably, you can support _both_ ACPI and non-ACPI hibernation, so
design for being able to support both.  End of story (as far as I'm concerned).

Greetings,
Rafael
 

-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 23:06, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:
> 
> > david@lang.hm writes:
> >
> > [snip]
> >
> >>>> * figure out which devices can wake up
> >>>> * put devices into low power states (wake-up devices are placed in the Dx
> >>>> states compatible with the wake capability, the others are powered off)
> >
> >> this can't be done by the image-saving kernel if that kernel doesn't know about
> >> the device.
> >
> > The image-saving kernel can be made to know about all of the "wake up"
> > devices; all other devices should have already been powered off by the
> > "hibernated" kernel.
> 
> not nessasarily.

More than that.  The hibernated kernel should not power off any devices,
because that is _wasteful_ (unless, of course, the powering off a device is the
only way to quiesce it).

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:

> On Tuesday, 17 July 2007 22:53, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>
>>> On Tuesday, 17 July 2007 22:18, david@lang.hm wrote:
>>>> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
>>>>
>>>>> Now, with that in mind, ACPI requires us to make the system enter the S4 sleep
>>>>> state as a result of the hibernation procedure.  In my opinion this may be done
>>>>> after saving the image, but still this means, for example, that the
>>>>> image-saving kernel needs to support ACPI.
>>>>>
>>>>> Next, during the restore, we should first check if the image is present (and
>>>>> valid) _without_ turning ACPI on (note that this is not done by the current
>>>>> hibernation code and that leads to strange problems on some systems).  Then,
>>>>> if the image is present (and valid), we should first load it, jump to the
>>>>> hibernated kernel and _then_ turn ACPI on and execute the _BFS and
>>>>> _WAK ACPI global methods (again, this is not done by the current code in that
>>>>> order, which is wrong).  Only after that is the hibernated kernel supposed to
>>>>> continue.
>>>>>
>>>>> [Please refer to section 15.3 of the 3.0b ACPI spec for details.]
>>>>
>>>> you are starting from the assumption that ACPI S4 mode should be used.
>>>>
>>>> I'm saying that a suspend that uses ACPI S4 mode is fundamentally
>>>> different from one that does a power off instead.
>>>
>>> It is different, but not fundamentally.
>>>
>>>> from my point of view the ACPI S4 sleep mode has far more in common with
>>>> suspend-to-ram then with the suspend-to-disk that I'm talking about
>>>>
>>>> non-ACPI hibernate
>>>>
>>>>    since the box powers off
>>>>      it uses zero power while suspended
>>>>      another OS could be run before a resume
>>>>      hardware can be swapped, suspend image could be sent around the world to be restored on another system.
>>>>      restore makes no assumptions about the state of the hardware when it is restored
>>>>      restore is slower (full BIOS boot is required)
>>>>    should be able to work on just about any hardware (the limit is the ability to initialize the devices)
>>>>
>>>>
>>>> ACPI suspends
>>>>
>>>>    since the box never completely powers off:
>>>>      a complete power failure breaks the suspend
>>>>      the OS must remain in control so other uses must be prevented.
>>>>      hardware must remain in the ACPI state from suspend until restore.
>>>>      restore can be faster (some initialization may be able to be skipped)
>>>>    requires ACPI hardware support
>>>>
>>>> under the catagory of ACPI suspends you have
>>>>
>>>>    fast suspend-to-ram (stop scheduling, put the CPU to sleep, as long as
>>>> the memory keeps getting refreshed)
>>>>    slow suspend-to-ram (stop scheduling, put as much of the hardware as
>>>> possible to sleep, including spinning down disks and other things that
>>>> take a while to undo)
>>>>    suspend-to-disk (stop scheduleing, copy the ram somewhere so that it
>>>> doesn't need to be refreshed, put everything into low-power mode)
>>>>
>>>>    and there are probably quite a few others as well. but they are all in
>>>> the same family in that you have to worry about ACPI states, and they all
>>>> have the same restrictions on what can happen between suspend and resume
>>>>
>>>> the non-ACPI hibernate behaves very differently, and for some people (and
>>>> I think I am one of them) it will meet their needs better then _any_ of
>>>> the ACPI suspends.
>>>
>>> OTOH, there are many people who would want the ACPI suspends to be handled
>>> and they don't really care for the power-off-only hibernation.
>>>
>>> If you aren't going to support the ACPI hibernation, your framework will be
>>> incomplete and therefore not generally useful.
>>
>> if you make the framework limited by the ACPI requriement, your framework
>> will not be able to be used in all cases and is therefor incomplete and
>> not generally useful.
>>
>> see, I can make authoritative sounding declarations too. :-)
>
> Well, being able to support ACPI need not imply being unable to work in the
> other cases.  Conversely, being able to work in non-ACPI cases need not imply
> being unable to support ACPI.  Hence, you can support ACPI and be able to
> work in the other cases at the same time.
>
> Try again. ;-)

I thought it was you who was saying that the restore operation requried 
ACPI if you used ACPI at all.

>> I agree that some people want ACPI suspends, but you don't seem to allow
>> the fact that some people don't,
>
> No, I do.
>
> What I'm trying to say is that we should support the ACPI S4 transition and
> resume, which DOES NOT MEAN that we should support ONLY that.

it has sounded like you are saying that if you support ACPI S4 sleep then 
it's incompatible with powering off.

>> and those people don't want to have the ACPI based limits.
>
> There are NO ACPI LIMITS!  There only are things that you need to implement
> if you're going to support ACPI, but they need not be used ALWAYS, no?

yes there are limits. the fact that you can't remove the battery in S4 
mode without messing things up is a limit, the fact that it's against ACPI 
specs to boot another OS before resume is a limit. you seem to keep 
pointing out requirements of ACPI that limit what can be done. I do not 
disagree about these being requirements _if you are useing ACPI_, but if 
you don't need ACPI they should not limit you.

>> they _especially_ don't want those limits when it appears as if supporting
>> those limits is what's preventing their much simpler case from working
>> reliably.
>>
>> I strongly suspect that the majority of users don't care about ACPI, they
>> want to be able to pause and resume their machine. they may want a couple
>> options for how fast the resume is (trading resume speed against how much
>> power the system eats), but the deep sleep modes (suspend-to-disk,
>> hibernate) probably have restore times that are close enough to each other
>> that very few people would care enough to opt for a ACPI S4 mode that
>> won't survive a loss of battery power over a non-ACPI mode that would.
>
> Please don't argue like that, these are not arguments for avoiding the ACPI
> support.  Arguably, you can support _both_ ACPI and non-ACPI hibernation, so
> design for being able to support both.  End of story (as far as I'm concerned).

I wish I understood how you are saying this, becouse it does not match 
what I thought I have been reading for the last few days.

David Lang

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 23:17, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> 
> > On Tuesday, 17 July 2007 22:34, david@lang.hm wrote:
> >> On Tue, 17 Jul 2007, Rafael J. Wysocki wrote:
> >>
> >>> On Tuesday, 17 July 2007 20:32, Alan Stern wrote:
> >>>
> >>>> I'm still not entirely clear on how "suspend-to-both" ought to be
> >>>> handled.  Presumably it will start off as a normal hibernation.  But
> >>>> instead of shutting down, wouldn't the kexec'd kernel return to the
> >>>> original kernel?
> >>>
> >>> No, I think the image-saving kernel should suspend.  Then, on resume the
> >>> platform will go back to it and it will jump back to the hibernated kernel.
> >>>
> >>>> After all, the original kernel knows about all the devices and can put them
> >>>> into a low-power state, while the kexec'd kernel might not have sufficient
> >>>> information.
> >>>
> >>> That's correct, but ...
> >>>
> >>>> But what about the freezer?  The original reason for using kexec was to
> >>>> avoid the need for the freezer.  With no freezer, while the original
> >>>> kernel is busy powering down its devices, user tasks will be free to
> >>>> carry out I/O -- which will make the memory snapshot inconsistent with
> >>>> the on-disk data structures.
> >>>
> >>> ... we can't return to the hibernated kernel unless we are going to cancel the
> >>> hibernation.
> >>
> >> this is where we disagree.
> >>
> >> why not? if all that the hibernated kernel does is to suspend-to-ram and
> >> makes no changes to disks or TCP connections anything that it does do
> >> would be lost if power were to fail and you instead did a restore from
> >> disk.
> >
> > How do you guarantee that no tasks are scheduled when you get back to the
> > hibernated kernel?
> 
> just don't schedule any userspace tasks. all you need to do is to execute 
> the ACPI sleep functions. you normally do that after stopping userspace 
> anyway.

This is plain reinventing the freezer under another guise, sorry. ;-)

> >> there is only a problem if something takes place that would prevent the
> >> restore-from-disk from working. if this is done in a non-ACPI way that
> >> will work across a power cycle you don't have to worry about the hardware
> >> state not matching anyway.
> >>
> >>> That's why I think that for the suspend-to-both the image-saving kernel will
> >>> need to support the same set of devices as the hibernated kernel.
> >>
> >> suspend-to-both doesn't really make sense if the suspend-to-disk portion
> >> is useing the ACPI S4 mode.
> >
> > Well, not exactly.  If your battery runs out of power while you're suspended,
> > but you have the image saved, it's still better to restore from the image, even
> > if something may not work correctly after the restore, than to risk a loss of
> > data.
> 
> if things don't work correctly you are still risking the loss of data, the 
> user just doesn't know it.

The experience shows that this is not the case.  The disks usually work (you
have to initialize them without ACPI anyway to load the image, no?).

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 23:27, Jeremy Maitin-Shepard wrote:
> david@lang.hm writes:
> 
> [snip]
> 
> >> How do you guarantee that no tasks are scheduled when you get back to the
> >> hibernated kernel?
> 
> > just don't schedule any userspace tasks. all you need to do is to execute the
> > ACPI sleep functions. you normally do that after stopping userspace
> > anyway.
> 
> What does "stopping userspace" mean?  You already said it does not mean
> disabling interrupts.  But using the freezer is also not an option,
> since the avoidance of that is the main reason for the kexec approach in
> the first place.
> 
> [snip]
> 
> >> Well, not exactly.  If your battery runs out of power while you're suspended,
> >> but you have the image saved, it's still better to restore from the image,
> > even
> >> if something may not work correctly after the restore, than to risk a loss of
> >> data.
> 
> > if things don't work correctly you are still risking the loss of data, the user
> > just doesn't know it.
> 
> It should be possible on any system to do a hibernate followed by a
> shutdown (and then resume properly, without any problems).  Thus, for
> handling suspend to both, you resume as if the system had been shutdown,
> rather than resuming as if the system came from S4.

Exactly.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

david@lang.hm writes:

[snip]
 
>> There are NO ACPI LIMITS!  There only are things that you need to implement
>> if you're going to support ACPI, but they need not be used ALWAYS, no?

> yes there are limits. the fact that you can't remove the battery in S4 mode
> without messing things up is a limit,

You won't mess things up as long as the resuming kernel knows that it
should resume as if the system were shutdown, rather than sent to S4
state.  Maybe it is even possible to detect what type of resuming is
needed automatically.  Similarly, booting another OS shouldn't be a
problem, except that if you do it without powering off the system first,
some devices might not work under the other OS if the other OS doesn't
initialize them properly.

[snip]

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[Rafael J. Wysocki]

On Tuesday, 17 July 2007 23:27, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Jeremy Maitin-Shepard wrote:
> 
> > david@lang.hm writes:
> >
> > [snip]
> >
> >>> How do you guarantee that no tasks are scheduled when you get back to the
> >>> hibernated kernel?
> >
> >> just don't schedule any userspace tasks. all you need to do is to execute the
> >> ACPI sleep functions. you normally do that after stopping userspace
> >> anyway.
> >
> > What does "stopping userspace" mean?  You already said it does not mean
> > disabling interrupts.  But using the freezer is also not an option,
> > since the avoidance of that is the main reason for the kexec approach in
> > the first place.
> 
> just don't schedule any non-kernel threads.
> 
> remember that the normal shutdown/suspend procedure is (from another 
> related thread)
> 
> > >>sys_reboot(LINUX_REBOOT_CMD_KEXEC)
> > >>     kernel_kexec
> > >>         kernel_restart_prepare
> > >>             device_shutdown
> > >>         machine_shutdown
> > >>         machine_kexec
> 
> I'm just saying that instead of going back to the normal operation of the 
> kernel you just go directly to the new shutdown routine instead.
> 
> > [snip]
> >
> >>> Well, not exactly.  If your battery runs out of power while you're suspended,
> >>> but you have the image saved, it's still better to restore from the image,
> >> even
> >>> if something may not work correctly after the restore, than to risk a loss of
> >>> data.
> >
> >> if things don't work correctly you are still risking the loss of data, the user
> >> just doesn't know it.
> >
> > It should be possible on any system to do a hibernate followed by a
> > shutdown (and then resume properly, without any problems).  Thus, for
> > handling suspend to both, you resume as if the system had been shutdown,
> > rather than resuming as if the system came from S4.
> 
> I agree with this, but according to Rafael if the "hibernated" image is 
> assuming that the devices were put into low-power mode by ACPI and you 
> boot up instead the system doesn't work right.

That's correct, some devices don't work right, but these are not disks.  They
usually are devices closely related to the platform, like the embedded
controller.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[david]

On Tue, 17 Jul 2007, Alan Stern wrote:

> On Tue, 17 Jul 2007 david@lang.hm wrote:
>
>>> But what about the freezer?  The original reason for using kexec was to
>>> avoid the need for the freezer.  With no freezer, while the original
>>> kernel is busy powering down its devices, user tasks will be free to
>>> carry out I/O -- which will make the memory snapshot inconsistent with
>>> the on-disk data structures.
>>
>> no, user tasks just don't get scheduled during shutdown.
>
> But a user task may be holding a lock which is needed for putting some
> device into low-power mode.  It can't release that lock if it doesn't
> get scheduled.

then you can't suspend that box. if you schedule it, it could get another 
lock (or another process gets another lock)

if you can't power down or put hardware into low-power mode without the 
approval of userspace, you are in serious trouble.

David Lang

Re: Hibernation considerations

[Alan Stern]

On Tue, 17 Jul 2007 david@lang.hm wrote:

> > But what about the freezer?  The original reason for using kexec was to
> > avoid the need for the freezer.  With no freezer, while the original
> > kernel is busy powering down its devices, user tasks will be free to
> > carry out I/O -- which will make the memory snapshot inconsistent with
> > the on-disk data structures.
> 
> no, user tasks just don't get scheduled during shutdown.

But a user task may be holding a lock which is needed for putting some 
device into low-power mode.  It can't release that lock if it doesn't 
get scheduled.

Alan Stern

Re: Hibernation considerations

[Matthew Garrett]

On Tue, Jul 17, 2007 at 08:19:32AM -0700, david@lang.hm wrote:
> On Tue, 17 Jul 2007, Matthew Garrett wrote:
> >Powering off rather than using S4 means you lose most wakeup device
> >support. That would be a functional regression compared to the current
> >code.
> 
> only if the kexec isn't able to initialize those devices.

If you aren't using ACPI, you probably don't know how to.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: Hibernation considerations

[david]

On Wed, 18 Jul 2007, Matthew Garrett wrote:

> On Tue, Jul 17, 2007 at 08:19:32AM -0700, david@lang.hm wrote:
>> On Tue, 17 Jul 2007, Matthew Garrett wrote:
>>> Powering off rather than using S4 means you lose most wakeup device
>>> support. That would be a functional regression compared to the current
>>> code.
>>
>> only if the kexec isn't able to initialize those devices.
>
> If you aren't using ACPI, you probably don't know how to.

the current kexec patch to allow you to move back to the original kernel 
requires ACPI be disabled to work.

that states pretty strongly that kexec isn't dependant on ACPI to 
initialize devices.

David Lang

Re: Hibernation considerations

[Matthew Garrett]

On Tue, Jul 17, 2007 at 08:54:24PM -0700, david@lang.hm wrote:
> On Wed, 18 Jul 2007, Matthew Garrett wrote:
> 
> >On Tue, Jul 17, 2007 at 08:19:32AM -0700, david@lang.hm wrote:
> >>On Tue, 17 Jul 2007, Matthew Garrett wrote:
> >>>Powering off rather than using S4 means you lose most wakeup device
> >>>support. That would be a functional regression compared to the current
> >>>code.
> >>
> >>only if the kexec isn't able to initialize those devices.
> >
> >If you aren't using ACPI, you probably don't know how to.
> 
> the current kexec patch to allow you to move back to the original kernel 
> requires ACPI be disabled to work.

Which means it isn't putting the hardware into S4, which means that you 
don't get the platform wakeup events.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: Hibernation considerations

[david]

On Wed, 18 Jul 2007, Matthew Garrett wrote:

> On Tue, Jul 17, 2007 at 08:54:24PM -0700, david@lang.hm wrote:
>> On Wed, 18 Jul 2007, Matthew Garrett wrote:
>>
>>> On Tue, Jul 17, 2007 at 08:19:32AM -0700, david@lang.hm wrote:
>>>> On Tue, 17 Jul 2007, Matthew Garrett wrote:
>>>>> Powering off rather than using S4 means you lose most wakeup device
>>>>> support. That would be a functional regression compared to the current
>>>>> code.
>>>>
>>>> only if the kexec isn't able to initialize those devices.
>>>
>>> If you aren't using ACPI, you probably don't know how to.
>>
>> the current kexec patch to allow you to move back to the original kernel
>> requires ACPI be disabled to work.
>
> Which means it isn't putting the hardware into S4, which means that you
> don't get the platform wakeup events.

Ok, I was misunderstanding what you meant by wakeup device support. if you 
are meaning things like wake-on-lan then yes, you do loose that with no 
ACPI support, but that is acceptable to many people. I understand it's not 
for some, and I'm not saying that this should be the only type of suspend, 
but I'm also saying that this should be one option.

David Lang

Re: Hibernation considerations

[Alan Stern]

On Tue, 17 Jul 2007 david@lang.hm wrote:

> On Tue, 17 Jul 2007, Alan Stern wrote:
> 
> > On Tue, 17 Jul 2007 david@lang.hm wrote:
> >
> >>> But what about the freezer?  The original reason for using kexec was to
> >>> avoid the need for the freezer.  With no freezer, while the original
> >>> kernel is busy powering down its devices, user tasks will be free to
> >>> carry out I/O -- which will make the memory snapshot inconsistent with
> >>> the on-disk data structures.
> >>
> >> no, user tasks just don't get scheduled during shutdown.
> >
> > But a user task may be holding a lock which is needed for putting some
> > device into low-power mode.  It can't release that lock if it doesn't
> > get scheduled.
> 
> then you can't suspend that box. if you schedule it, it could get another 
> lock (or another process gets another lock)
> 
> if you can't power down or put hardware into low-power mode without the 
> approval of userspace, you are in serious trouble.

You don't seem to appreciate the issues involved here.  Part of the 
justification for the freezer is that it doesn't need userspace 
approval and it freezes tasks at controlled points where they don't 
hold any locks.

Never mind.  It seems clear that this approach will suffer the same 
drawback as the proposal for removing the freezer from the 
suspend-to-RAM pathway.  Namely, device drivers will have to be changed 
to prevent user I/O requests from proceeding while devices are supposed 
to be quiescent or in a low-power state.

If a driver fails to handle this properly, its device could be 
reactivated in order to service a user request before the memory 
snapshot is made.  This could easily ruin the snapshot.

Alan Stern

Re: Hibernation considerations

[Rafael J. Wysocki]

On Wednesday, 18 July 2007 16:29, Alan Stern wrote:
> On Tue, 17 Jul 2007 david@lang.hm wrote:
> 
> > On Tue, 17 Jul 2007, Alan Stern wrote:
> > 
> > > On Tue, 17 Jul 2007 david@lang.hm wrote:
> > >
> > >>> But what about the freezer?  The original reason for using kexec was to
> > >>> avoid the need for the freezer.  With no freezer, while the original
> > >>> kernel is busy powering down its devices, user tasks will be free to
> > >>> carry out I/O -- which will make the memory snapshot inconsistent with
> > >>> the on-disk data structures.
> > >>
> > >> no, user tasks just don't get scheduled during shutdown.
> > >
> > > But a user task may be holding a lock which is needed for putting some
> > > device into low-power mode.  It can't release that lock if it doesn't
> > > get scheduled.
> > 
> > then you can't suspend that box. if you schedule it, it could get another 
> > lock (or another process gets another lock)
> > 
> > if you can't power down or put hardware into low-power mode without the 
> > approval of userspace, you are in serious trouble.
> 
> You don't seem to appreciate the issues involved here.  Part of the 
> justification for the freezer is that it doesn't need userspace 
> approval and it freezes tasks at controlled points where they don't 
> hold any locks.
> 
> Never mind.  It seems clear that this approach will suffer the same 
> drawback as the proposal for removing the freezer from the 
> suspend-to-RAM pathway.  Namely, device drivers will have to be changed 
> to prevent user I/O requests from proceeding while devices are supposed 
> to be quiescent or in a low-power state.

I agree.
 
> If a driver fails to handle this properly, its device could be 
> reactivated in order to service a user request before the memory 
> snapshot is made.  This could easily ruin the snapshot.

That's why I've been saying for quite some time that we first need to take care
of the drivers. :-)

IMO we've reached the point at which, whatever we want to do next, the drivers
are in the way.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Al Boldi]

Rafael J. Wysocki wrote:
> On Wednesday, 18 July 2007 16:29, Alan Stern wrote:
> >
> > Never mind.  It seems clear that this approach will suffer the same
> > drawback as the proposal for removing the freezer from the
> > suspend-to-RAM pathway.  Namely, device drivers will have to be changed
> > to prevent user I/O requests from proceeding while devices are supposed
> > to be quiescent or in a low-power state.
>
> I agree.
>
> > If a driver fails to handle this properly, its device could be
> > reactivated in order to service a user request before the memory
> > snapshot is made.  This could easily ruin the snapshot.
>
> That's why I've been saying for quite some time that we first need to take
> care of the drivers. :-)
>
> IMO we've reached the point at which, whatever we want to do next, the
> drivers are in the way.

Correct, but only if we want ACPI support.  Granted, we need a separation of 
the hibernate/suspend PM functions, but in the absence of ACPI, all we need 
right now are dump/restore routines for the crashkernel.

Next, we should be looking into reducing the kexec'd kernel environment size, 
which currently, at 16MB, is way too big, and even at 1MB would be 
problematic for small systems.

So, ACPI should really be the least of our worries, and the reason why people 
are fixating on ACPI is probably because they have nothing else to fixate 
on.  They probably haven't even tried the kexec patches to appreciate how 
easy the kexec approach is and how little interference APCI represents when 
suspending and resuming.


Thanks!

--
Al

Re: Hibernation considerations

[Rafael J. Wysocki]

On Friday, 20 July 2007 06:40, Al Boldi wrote:
> Rafael J. Wysocki wrote:
> > On Wednesday, 18 July 2007 16:29, Alan Stern wrote:
> > >
> > > Never mind.  It seems clear that this approach will suffer the same
> > > drawback as the proposal for removing the freezer from the
> > > suspend-to-RAM pathway.  Namely, device drivers will have to be changed
> > > to prevent user I/O requests from proceeding while devices are supposed
> > > to be quiescent or in a low-power state.
> >
> > I agree.
> >
> > > If a driver fails to handle this properly, its device could be
> > > reactivated in order to service a user request before the memory
> > > snapshot is made.  This could easily ruin the snapshot.
> >
> > That's why I've been saying for quite some time that we first need to take
> > care of the drivers. :-)
> >
> > IMO we've reached the point at which, whatever we want to do next, the
> > drivers are in the way.
> 
> Correct, but only if we want ACPI support.

No, in general.

> Granted, we need a separation of  
> the hibernate/suspend PM functions, but in the absence of ACPI, all we need 
> right now are dump/restore routines for the crashkernel.

IMO you aren't right, but I guess there's no point in trying to convince you.

> Next, we should be looking into reducing the kexec'd kernel environment size, 
> which currently, at 16MB, is way too big, and even at 1MB would be 
> problematic for small systems.
> 
> So, ACPI should really be the least of our worries, and the reason why people 
> are fixating on ACPI is probably because they have nothing else to fixate 
> on.

Yeah, right.  Please read the $subject message again.  And sorry, but IMO your
previous replies to it haven't addressed any of the original points.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Pavel Machek]

Hi!

> >Part of the problem here is that ACPI already has its own terminology,
> >and you're trying to invent a new one instead of using the existing
> >one.
> >
> >I agree, it would be good to have a non-ACPI-specific hibernation mode,
> >something which would look to ACPI like a normal shutdown.  But I'm not
> >so sure this is possible.
> 
> why would it not be possible?
> 
> >You have to understand that the ACPI spec is weird and complex.  The
> >mere fact that you have written a system image to disk changes the way
> >ACPI regards the shutdown procedure.  Even though you may treat all the
> >devices and the rest of the hardware exactly the same, it's a different
> >operation as far as ACPI is concerned, with different requirements.
> >
> >Yes, it's bizarre.  Why do you think so many people have complained so
> >vehemently about ACPI for all these years?
> 
> so let's act as if ACPI doesn't exist and make a suspend-to-disk that 
> works without it and looks to ACPI like a complete power off/on cycle (but 
> looks to the user like a suspend/resume cycle)

...if you act as if ACPI does not exist, you'll loose AC/battery power
status upon resume, and maybe more, because ACPI handles that.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation considerations

[Pavel Machek]

Hi!

> from my point of view the ACPI S4 sleep mode has far more in common with 
> suspend-to-ram then with the suspend-to-disk that I'm talking about
> 
> non-ACPI hibernate
> 
>   since the box powers off
>     it uses zero power while suspended
>     another OS could be run before a resume
>     hardware can be swapped, suspend image could be sent around the world 
>     to be restored on another system.
>     restore makes no assumptions about the state of the hardware when it is 
>     restored
>     restore is slower (full BIOS boot is required)
>   should be able to work on just about any hardware (the limit is the 
>   ability to initialize the devices)

So it will be break at least battery status and "AC plugged in"
status, because those are handled by ACPI and we do not know how to
control them by hand.
								Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation considerations

[Jeremy Maitin-Shepard]

Pavel Machek <pavel@ucw.cz> writes:

[snip]

> So it will be break at least battery status and "AC plugged in"
> status, because those are handled by ACPI and we do not know how to
> control them by hand.

It seems that it should be possible to initialize ACPI as if the system
just booted up normally.  Then battery status and such should be
correct, since they are correct after normal initialization.

It should be possible to make hibernate look just like a reboot to all
of the devices, including ACPI stuff.

-- 
Jeremy Maitin-Shepard

Re: Hibernation considerations

[Pavel Machek]

Hi!

> Pavel Machek <pavel@ucw.cz> writes:
> 
> [snip]
> 
> > So it will be break at least battery status and "AC plugged in"
> > status, because those are handled by ACPI and we do not know how to
> > control them by hand.
> 
> It seems that it should be possible to initialize ACPI as if the system
> just booted up normally.  Then battery status and such should be
> correct, since they are correct after normal initialization.
> 
> It should be possible to make hibernate look just like a reboot to all
> of the devices, including ACPI stuff.

Patch to make that work with swsusp/shutdown method would be indeed
welcome. It does not work today.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation considerations

[david]

On Sat, 21 Jul 2007, Pavel Machek wrote:

> Hi!
>
>> Pavel Machek <pavel@ucw.cz> writes:
>>
>> [snip]
>>
>>> So it will be break at least battery status and "AC plugged in"
>>> status, because those are handled by ACPI and we do not know how to
>>> control them by hand.
>>
>> It seems that it should be possible to initialize ACPI as if the system
>> just booted up normally.  Then battery status and such should be
>> correct, since they are correct after normal initialization.
>>
>> It should be possible to make hibernate look just like a reboot to all
>> of the devices, including ACPI stuff.
>
> Patch to make that work with swsusp/shutdown method would be indeed
> welcome. It does not work today.

is this a problem in the restore path?

with the kexec approach (and ignoring suspend to ram and disk for the 
moment) the system will actually get shutdown completely after the image 
is written. on resume it gets cold booted. at this point the ACPI stuff 
should have no problem

now if the ACPI drivers are storing something in ram about the battery 
status and AC power status, but don't re-check after the resume, it seems 
to me that they can't possibly be reliable anyway. if they do re-check 
after the resume then where's the problem?

David Lang

Re: Hibernation considerations

[Pavel Machek]

Hi!

> >>>So it will be break at least battery status and "AC plugged in"
> >>>status, because those are handled by ACPI and we do not know how to
> >>>control them by hand.
> >>
> >>It seems that it should be possible to initialize ACPI as if the system
> >>just booted up normally.  Then battery status and such should be
> >>correct, since they are correct after normal initialization.
> >>
> >>It should be possible to make hibernate look just like a reboot to all
> >>of the devices, including ACPI stuff.
> >
> >Patch to make that work with swsusp/shutdown method would be indeed
> >welcome. It does not work today.
> 
> is this a problem in the restore path?
> 
> with the kexec approach (and ignoring suspend to ram and disk for the 
> moment) the system will actually get shutdown completely after the image 
> is written. on resume it gets cold booted. at this point the ACPI stuff 
> should have no problem
> 
> now if the ACPI drivers are storing something in ram about the battery 
> status and AC power status, but don't re-check after the resume, it
>seems

That seems to be the problem. They store something in ram, and we
don't tell them that we resumed. That's why platform mode is
important, and way to go on ACPI systems.

								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation considerations

[david]

On Sat, 21 Jul 2007, Pavel Machek wrote:

>>>>> So it will be break at least battery status and "AC plugged in"
>>>>> status, because those are handled by ACPI and we do not know how to
>>>>> control them by hand.
>>>>
>>>> It seems that it should be possible to initialize ACPI as if the system
>>>> just booted up normally.  Then battery status and such should be
>>>> correct, since they are correct after normal initialization.
>>>>
>>>> It should be possible to make hibernate look just like a reboot to all
>>>> of the devices, including ACPI stuff.
>>>
>>> Patch to make that work with swsusp/shutdown method would be indeed
>>> welcome. It does not work today.
>>
>> is this a problem in the restore path?
>>
>> with the kexec approach (and ignoring suspend to ram and disk for the
>> moment) the system will actually get shutdown completely after the image
>> is written. on resume it gets cold booted. at this point the ACPI stuff
>> should have no problem
>>
>> now if the ACPI drivers are storing something in ram about the battery
>> status and AC power status, but don't re-check after the resume, it
>> seems
>
> That seems to be the problem. They store something in ram, and we
> don't tell them that we resumed. That's why platform mode is
> important, and way to go on ACPI systems.

this sounds like the few drivers that do this sort of thing (and this 
should only be the things that report status, drivers that enable devices 
should be taken care of) need a 'forget what you think you know, check the 
reality of the hardware' function.

even without suspend I've sen these drivers get out of sync with reality, 
and so such a function call would be useful to get them back in sync in 
any case.

if such a 'check reality' function was available it should be 
straightforward to call it after a non-ACPI hibrnate/resume

David Lang

Re: Hibernation considerations

[Vojtech Pavlik]

On Mon, Jul 16, 2007 at 12:38:11AM +0200, Rafael J. Wysocki wrote:

> > Or the user unplugs their flash drive after hibernation rather than before.
> > 
> > Two things which I think would be nice to consider are:
> >    1) Encryption - I'd actually prefer if my luks device did not
> >        remember the key accross a hibernation; I want to be forced to
> >        reenter the phrase.  However I don't know what the best thing
> >        to do to partitions/applications using the luks device is.
> 
> Encryption is possible with both the userland hibernation (aka uswsusp) and
> TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a "must
> have" feature for a framework to be generally useful (many users don't use it
> anyway).

If a user uses an encrypted filesystem, then he also needs an encrypted
swap and encrypted hibernation image: Otherwise the fileystem encryption
is not very useful.

Forgetting the filesystem/swap decryption keys before hibernation is
probably harder to do - there may be sensitive data in the kernel memory
image that weren't cleared - even if the key itself is not there.

In my opinion, encrypted hibernation is what every notebook user should
want - that's the only way how to make sure data from the notebook
aren't available when the notebook is physically stolen.

-- 
Vojtech Pavlik
Director SuSE Labs

Re: Hibernation considerations

[Rafael J. Wysocki]

On Sunday, 29 July 2007 08:53, Vojtech Pavlik wrote:
> On Mon, Jul 16, 2007 at 12:38:11AM +0200, Rafael J. Wysocki wrote:
> 
> > > Or the user unplugs their flash drive after hibernation rather than before.
> > > 
> > > Two things which I think would be nice to consider are:
> > >    1) Encryption - I'd actually prefer if my luks device did not
> > >        remember the key accross a hibernation; I want to be forced to
> > >        reenter the phrase.  However I don't know what the best thing
> > >        to do to partitions/applications using the luks device is.
> > 
> > Encryption is possible with both the userland hibernation (aka uswsusp) and
> > TuxOnIce (formerly known as suspend2).  Still, I don't consider it as a "must
> > have" feature for a framework to be generally useful (many users don't use it
> > anyway).
> 
> If a user uses an encrypted filesystem, then he also needs an encrypted
> swap and encrypted hibernation image: Otherwise the fileystem encryption
> is not very useful.

I was talking about hibernation image encryption.  Arguably, if the image is
encrypted, you don't need to worry about its contents, including the keys for
other kinds of encryption (eg. fs encryption).
 
> Forgetting the filesystem/swap decryption keys before hibernation is
> probably harder to do - there may be sensitive data in the kernel memory
> image that weren't cleared - even if the key itself is not there.

If the image is encrypted, its contents are not available to anyone
unauthorized and that includes the filesystem/swap decryption keys.

> In my opinion, encrypted hibernation is what every notebook user should
> want - that's the only way how to make sure data from the notebook
> aren't available when the notebook is physically stolen.

Provided that there are any sensitive (to the user or her employer etc.) data
in the notebook.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation considerations

[Stefan Seyfried]

Hi,

Sorry for joining late, just a small annotation:

On Tue, Jul 17, 2007 at 01:18:13PM -0700, david@lang.hm wrote:
 
> non-ACPI hibernate
> 
>   since the box powers off
>     it uses zero power while suspended
>     another OS could be run before a resume
>     hardware can be swapped, suspend image could be sent around the world to be restored on another system.
>     restore makes no assumptions about the state of the hardware when it is restored
>     restore is slower (full BIOS boot is required)
>   should be able to work on just about any hardware (the limit is the ability to initialize the devices)
> 
> 
> ACPI suspends
> 
>   since the box never completely powers off:A

wrong

>     a complete power failure breaks the suspend

wrong

>     the OS must remain in control so other uses must be prevented.
>     hardware must remain in the ACPI state from suspend until restore.
>     restore can be faster (some initialization may be able to be skipped)
>   requires ACPI hardware support
> 
> under the catagory of ACPI suspends you have

ACPI S4 turns off the machine completely and you can remove the battery (this
is even required somewhere in the spec). Any state saving is done in CMOS RAM
or flash.

But for example many Notebooks resume much faster if they go through the
ACPI S4 hooks during suspend (less than one second from "lid open" to "grub"
while they need ~10 seconds through the BIOS on a "normal" boot.
My Toughbook resumes on "Lid Opened" after S4, it doesn't after a shutdown.

So there will be differences.
I'm not saying that they are too important, but 20% faster resume still is
a good saving for me.

No need to restart this thread btw ;-)

Have fun,

    Stefan
-- 
Stefan Seyfried
QA / R&D Team Mobile Devices        |              "Any ideas, John?"
SUSE LINUX Products GmbH, Nürnberg  | "Well, surrounding them's out." 

This footer brought to you by insane German lawmakers:
SUSE Linux Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)