Hibernation Redesign (was: malicious filesystems (was Re: [linux-pm] Re: [PATCH] Remove process freezer from suspend to RAM)

Hibernation Redesign (was: malicious filesystems (was Re: [linux-pm] Re: [PATCH] Remove process freezer from suspend to RAM)

[Al Boldi]

Pavel Machek wrote:
> We are stuck with refrigerator for now, and at least for hibernation,
> I don't see any feasible alternative.

Feasible alternative?

Freezing is the only way to successfully suspend, in kernel space that is.  
The problem here is:  Why do we freeze in kernel space?

APM didn't freeze in kernel space, and neither should ACPI.  The only reason 
you would want to freeze in kernel space is because your system does not 
have APM or ACPI or anything else to freeze from BIOS level.

So the problem is really easy to solve: either call the APM suspend functions 
or install ACPI soft-extensions to implement hibernation from BIOS level.

But there is a catch:  Hibernation from BIOS level probably requires a 
diskdriver, which isn't always available from BIOS level or is really slow.  

So you are back to freezing in kernel space, but this time you know why, and 
that you need a system that simply halts to disk after issuing sync-out 
requests to all peripheral devices involved.  Which sounds like the freezer 
in mainline, does it not?  And if it does, then what's the problem?


Thanks!

--
Al

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Al Boldi <a1426z@gawab.com> writes:

> Pavel Machek wrote:
>> We are stuck with refrigerator for now, and at least for hibernation,
>> I don't see any feasible alternative.

> Feasible alternative?

I posted such an alternative to the list a short time ago: hibenrating
from a *new* kernel space/user space that is created by loading a new
kernel in a manner similar to what is done for kexec crashdumps.  Unlike
kexec crashdumps, however, it would not require reserving any memory at
boot, because the necessary memory (maybe 16MB or 64MB) can be freed
just before hibernating, and device drivers can be properly stopped so
that DMAs don't stomp over certain memory.

This approach eliminates the need for the freezer, as it would make
hibernate look a lot a bit like suspend to ram from the perspective of
the "old" kernel (the kernel being hibernated), as the hibernate
operation itself would be completely atomic from the perspective of the
"old" kernel.  That is not to say, of course, that any code paths would
actually be shared, or that the drivers would do the same things
(because they probably would not).

[snip]

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Nick Piggin]

Jeremy Maitin-Shepard wrote:
> Al Boldi <a1426z@gawab.com> writes:
> 
> 
>>Pavel Machek wrote:
>>
>>>We are stuck with refrigerator for now, and at least for hibernation,
>>>I don't see any feasible alternative.
> 
> 
>>Feasible alternative?
> 
> 
> I posted such an alternative to the list a short time ago: hibenrating
> from a *new* kernel space/user space that is created by loading a new
> kernel in a manner similar to what is done for kexec crashdumps.  Unlike
> kexec crashdumps, however, it would not require reserving any memory at
> boot, because the necessary memory (maybe 16MB or 64MB) can be freed
> just before hibernating, and device drivers can be properly stopped so
> that DMAs don't stomp over certain memory.

This is the Morton method, isn't it? :) I remember it sounding like a
very good idea when he brought it up, but I can't remember the details
of why it was rejected or what the problems were.


> This approach eliminates the need for the freezer, as it would make
> hibernate look a lot a bit like suspend to ram from the perspective of
> the "old" kernel (the kernel being hibernated), as the hibernate
> operation itself would be completely atomic from the perspective of the
> "old" kernel.  That is not to say, of course, that any code paths would
> actually be shared, or that the drivers would do the same things
> (because they probably would not).

Well it basically is suspend to RAM with the additional step that a
new kernel gets booted and writes out the data from RAM to disk then
shuts down. I suspect that freeing memory on the fly for the new kernel
would be non-trivial (but possible), however simply having a reserve
RAM region for the new kernel would be fine for a first step.

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Nick Piggin <nickpiggin@yahoo.com.au> writes:

> Jeremy Maitin-Shepard wrote:
>> Al Boldi <a1426z@gawab.com> writes:
>> 
>> 
>>> Pavel Machek wrote:
>>> 
>>>> We are stuck with refrigerator for now, and at least for hibernation,
>>>> I don't see any feasible alternative.
>> 
>> 
>>> Feasible alternative?
>> 
>> 
>> I posted such an alternative to the list a short time ago: hibenrating
>> from a *new* kernel space/user space that is created by loading a new
>> kernel in a manner similar to what is done for kexec crashdumps.  Unlike
>> kexec crashdumps, however, it would not require reserving any memory at
>> boot, because the necessary memory (maybe 16MB or 64MB) can be freed
>> just before hibernating, and device drivers can be properly stopped so
>> that DMAs don't stomp over certain memory.

> This is the Morton method, isn't it? :) I remember it sounding like a
> very good idea when he brought it up, but I can't remember the details
> of why it was rejected or what the problems were.

Perhaps he did bring it up before I did.  Please forward me a link to
the thread or other reference if you can find it, as I'd be interested
in reading it.


>> This approach eliminates the need for the freezer, as it would make
>> hibernate look a lot a bit like suspend to ram from the perspective of
>> the "old" kernel (the kernel being hibernated), as the hibernate
>> operation itself would be completely atomic from the perspective of the
>> "old" kernel.  That is not to say, of course, that any code paths would
>> actually be shared, or that the drivers would do the same things
>> (because they probably would not).

> Well it basically is suspend to RAM with the additional step that a
> new kernel gets booted and writes out the data from RAM to disk then
> shuts down.

There is the key difference, though, that the drivers should do rather
different things.  In particular, rather than place the hardware in a
low-power mode, it should place it in some state such that the new
kernel being loaded can handle it.

> I suspect that freeing memory on the fly for the new kernel
> would be non-trivial (but possible), however simply having a reserve
> RAM region for the new kernel would be fine for a first step.

Freeing memory on the fly should be extremely easy for the kernel (this
is precisely what it does when it needs to satisfy an allocation).  Note
that the memory allocated need not be contiguous.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Nick Piggin]

Jeremy Maitin-Shepard wrote:
> Nick Piggin <nickpiggin@yahoo.com.au> writes:

>>This is the Morton method, isn't it? :) I remember it sounding like a
>>very good idea when he brought it up, but I can't remember the details
>>of why it was rejected or what the problems were.
> 
> 
> Perhaps he did bring it up before I did.  Please forward me a link to
> the thread or other reference if you can find it, as I'd be interested
> in reading it.

Sent in the next mail.


>>I suspect that freeing memory on the fly for the new kernel
>>would be non-trivial (but possible), however simply having a reserve
>>RAM region for the new kernel would be fine for a first step.
> 
> 
> Freeing memory on the fly should be extremely easy for the kernel (this
> is precisely what it does when it needs to satisfy an allocation).  Note
> that the memory allocated need not be contiguous.

Yes, I have a rough idea about how page reclaim works. But I just
mean it would not be trivial to load the new kernel into physically
discontiguous memory. Possible of course, but I don't think kexec or
the setup code could quite cope ATM.

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Nick Piggin <nickpiggin@yahoo.com.au> writes:

> Jeremy Maitin-Shepard wrote:
>> Nick Piggin <nickpiggin@yahoo.com.au> writes:

>>> This is the Morton method, isn't it? :) I remember it sounding like a
>>> very good idea when he brought it up, but I can't remember the details
>>> of why it was rejected or what the problems were.
>> 
>> 
>> Perhaps he did bring it up before I did.  Please forward me a link to
>> the thread or other reference if you can find it, as I'd be interested
>> in reading it.

> Sent in the next mail.

Thanks.  I've started reading over the thread.

>>> I suspect that freeing memory on the fly for the new kernel
>>> would be non-trivial (but possible), however simply having a reserve
>>> RAM region for the new kernel would be fine for a first step.
>> 
>> 
>> Freeing memory on the fly should be extremely easy for the kernel (this
>> is precisely what it does when it needs to satisfy an allocation).  Note
>> that the memory allocated need not be contiguous.

> Yes, I have a rough idea about how page reclaim works. But I just
> mean it would not be trivial to load the new kernel into physically
> discontiguous memory. Possible of course, but I don't think kexec or
> the setup code could quite cope ATM.

It would indeed be a pain for the new kernel to be loaded and have to
use discontiguous memory.  The trick is, though, that this is not
necessary.  Immediately before jumping to the new kernel, the first X
bytes (where X is the amount of memory the new kernel will get,
typically 16MB or 64MB) of physical memory are backed up into the
arbitrary discontiguous pages that are made available.  This will not
take very long, because copying even 64MB of memory is extremely fast.
Then the new kernel is free to use the first X bytes of contiguous
physical memory.  Problem solved.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Nick Piggin]

Jeremy Maitin-Shepard wrote:
> Nick Piggin <nickpiggin@yahoo.com.au> writes:

>>Yes, I have a rough idea about how page reclaim works. But I just
>>mean it would not be trivial to load the new kernel into physically
>>discontiguous memory. Possible of course, but I don't think kexec or
>>the setup code could quite cope ATM.
> 
> 
> It would indeed be a pain for the new kernel to be loaded and have to
> use discontiguous memory.  The trick is, though, that this is not
> necessary.  Immediately before jumping to the new kernel, the first X
> bytes (where X is the amount of memory the new kernel will get,
> typically 16MB or 64MB) of physical memory are backed up into the
> arbitrary discontiguous pages that are made available.  This will not
> take very long, because copying even 64MB of memory is extremely fast.
> Then the new kernel is free to use the first X bytes of contiguous
> physical memory.  Problem solved.

Ah, that sounds like it would be the right way to go. Good thinking.

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Nick Piggin]

Nick Piggin wrote:
> Jeremy Maitin-Shepard wrote:
> 
>> Nick Piggin <nickpiggin@yahoo.com.au> writes:
> 
> 
>>> Yes, I have a rough idea about how page reclaim works. But I just
>>> mean it would not be trivial to load the new kernel into physically
>>> discontiguous memory. Possible of course, but I don't think kexec or
>>> the setup code could quite cope ATM.
>>
>>
>>
>> It would indeed be a pain for the new kernel to be loaded and have to
>> use discontiguous memory.  The trick is, though, that this is not
>> necessary.  Immediately before jumping to the new kernel, the first X
>> bytes (where X is the amount of memory the new kernel will get,
>> typically 16MB or 64MB) of physical memory are backed up into the
>> arbitrary discontiguous pages that are made available.  This will not
>> take very long, because copying even 64MB of memory is extremely fast.
>> Then the new kernel is free to use the first X bytes of contiguous
>> physical memory.  Problem solved.
> 
> 
> Ah, that sounds like it would be the right way to go. Good thinking.

Hmm, considering it is not a crash situation, it might even be
better again to simply reuse the exising kernel text and kernel
page table and memory map information if possible. That would
probably only require a meg or two to reinit drivers, load a
suspend-to-disk-init, and do the IO.

OTOH it would likely be more complex than just freeing up a bit
of memory and relocating it.

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Jeremy Fitzhardinge]

Jeremy Maitin-Shepard wrote:
> It would indeed be a pain for the new kernel to be loaded and have to
> use discontiguous memory.  The trick is, though, that this is not
> necessary.  Immediately before jumping to the new kernel, the first X
> bytes (where X is the amount of memory the new kernel will get,
> typically 16MB or 64MB) of physical memory are backed up into the
> arbitrary discontiguous pages that are made available.  This will not
> take very long, because copying even 64MB of memory is extremely fast.
> Then the new kernel is free to use the first X bytes of contiguous
> physical memory.  Problem solved.
>   

You could also use the paravirt_ops pte hooks to create a mapping from 
linear "physical" addresses to actual machine pages.  This is what a Xen 
kernel needs to do, and all the infrastructure will be in place shortly.

    J

Re: Hibernation Redesign

[Pavel Machek]

Hi!

> >> This approach eliminates the need for the freezer, as it would make
> >> hibernate look a lot a bit like suspend to ram from the perspective of
> >> the "old" kernel (the kernel being hibernated), as the hibernate
> >> operation itself would be completely atomic from the perspective of the
> >> "old" kernel.  That is not to say, of course, that any code paths would
> >> actually be shared, or that the drivers would do the same things
> >> (because they probably would not).
> 
> > Well it basically is suspend to RAM with the additional step that a
> > new kernel gets booted and writes out the data from RAM to disk then
> > shuts down.
> 
> There is the key difference, though, that the drivers should do rather
> different things.  In particular, rather than place the hardware in a
> low-power mode, it should place it in some state such that the new
> kernel being loaded can handle it.

Actually, when current kernel restores the snapshot... driver
requirements seem to be pretty similar. So that should not be a big
problem.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation Redesign

[Oliver Neukum]

Am Montag, 9. Juli 2007 schrieb Pavel Machek:
> Hi!
> 
> > >> This approach eliminates the need for the freezer, as it would make
> > >> hibernate look a lot a bit like suspend to ram from the perspective of
> > >> the "old" kernel (the kernel being hibernated), as the hibernate
> > >> operation itself would be completely atomic from the perspective of the
> > >> "old" kernel.  That is not to say, of course, that any code paths would
> > >> actually be shared, or that the drivers would do the same things
> > >> (because they probably would not).
> > 
> > > Well it basically is suspend to RAM with the additional step that a
> > > new kernel gets booted and writes out the data from RAM to disk then
> > > shuts down.
> > 
> > There is the key difference, though, that the drivers should do rather
> > different things.  In particular, rather than place the hardware in a
> > low-power mode, it should place it in some state such that the new
> > kernel being loaded can handle it.
> 
> Actually, when current kernel restores the snapshot... driver
> requirements seem to be pretty similar. So that should not be a big
> problem.

Hm, once the new kernel is booted, this decision is irrevocable, isn't it?
Is there any way to deal with errors by handing control back?

	Regards
		Oliver

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Oliver Neukum <oliver@neukum.org> writes:

[snip]

> Hm, once the new kernel is booted, this decision is irrevocable, isn't it?
> Is there any way to deal with errors by handing control back?

Returning to the old kernel can be done by telling drivers to set the
hardware to the appropriate state, then copying the backed up memory
back to the beginning of physical memory, and finally jumping to the old
kernel.  It would be much like what is done to resume from hibernation.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Oliver Neukum]

Am Montag, 9. Juli 2007 schrieb Jeremy Maitin-Shepard:
> Oliver Neukum <oliver@neukum.org> writes:
> 
> [snip]
> 
> > Hm, once the new kernel is booted, this decision is irrevocable, isn't it?
> > Is there any way to deal with errors by handing control back?
> 
> Returning to the old kernel can be done by telling drivers to set the
> hardware to the appropriate state, then copying the backed up memory
> back to the beginning of physical memory, and finally jumping to the old
> kernel.  It would be much like what is done to resume from hibernation.

If you can do that, why load a new kernel image?

	Regards
		Oliver

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Oliver Neukum <oliver@neukum.org> writes:

> Am Montag, 9. Juli 2007 schrieb Jeremy Maitin-Shepard:
>> Oliver Neukum <oliver@neukum.org> writes:
>> 
>> [snip]
>> 
>> > Hm, once the new kernel is booted, this decision is irrevocable, isn't it?
>> > Is there any way to deal with errors by handing control back?
>> 
>> Returning to the old kernel can be done by telling drivers to set the
>> hardware to the appropriate state, then copying the backed up memory
>> back to the beginning of physical memory, and finally jumping to the old
>> kernel.  It would be much like what is done to resume from hibernation.

> If you can do that, why load a new kernel image?

The challenges in doing that are analogous to the challenges in
suspending to RAM, for which it has been agreed that drivers should be
fixed such that the freezer is not necessary.

The hard part of hibernate is not creating the snapshot; rather, the
hard part is writing the snapshot, and allowing the user some
flexibility in how and where the snapshot is written.  The kdump
approach allows complete flexibility in writing the snapshot
(essentially any kernel or user space facility can be used), while not
interfering at all with the snapshot state.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Nick Piggin]

Nick Piggin wrote:
> Jeremy Maitin-Shepard wrote:
> 
>> Al Boldi <a1426z@gawab.com> writes:
>>
>>
>>> Pavel Machek wrote:
>>>
>>>> We are stuck with refrigerator for now, and at least for hibernation,
>>>> I don't see any feasible alternative.
>>
>>
>>
>>> Feasible alternative?
>>
>>
>>
>> I posted such an alternative to the list a short time ago: hibenrating
>> from a *new* kernel space/user space that is created by loading a new
>> kernel in a manner similar to what is done for kexec crashdumps.  Unlike
>> kexec crashdumps, however, it would not require reserving any memory at
>> boot, because the necessary memory (maybe 16MB or 64MB) can be freed
>> just before hibernating, and device drivers can be properly stopped so
>> that DMAs don't stomp over certain memory.
> 
> 
> This is the Morton method, isn't it? :) I remember it sounding like a
> very good idea when he brought it up, but I can't remember the details
> of why it was rejected or what the problems were.

Hmm, and it seems like I won't get to know without reliving what
looks like an epic flamewar starting.... here:

   http://thread.gmane.org/gmane.linux.kernel/374889

However from a quick look it seems like the only reason is the RAM
overhead of a reserve area. It seems unfortunate that it was
dismissed so quickly because of that problem (which could be
improved).

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Pavel Machek]

Hi!

> >>from a *new* kernel space/user space that is created by loading a new
> >>kernel in a manner similar to what is done for kexec crashdumps.  Unlike
> >>kexec crashdumps, however, it would not require reserving any memory at
> >>boot, because the necessary memory (maybe 16MB or 64MB) can be freed
> >>just before hibernating, and device drivers can be properly stopped so
> >>that DMAs don't stomp over certain memory.
> >
> >
> >This is the Morton method, isn't it? :) I remember it sounding like a
> >very good idea when he brought it up, but I can't remember the details
> >of why it was rejected or what the problems were.
> 
> Hmm, and it seems like I won't get to know without reliving what
> looks like an epic flamewar starting.... here:
> 
>   http://thread.gmane.org/gmane.linux.kernel/374889
> 
> However from a quick look it seems like the only reason is the RAM
> overhead of a reserve area. It seems unfortunate that it was
> dismissed so quickly because of that problem (which could be
> improved).

Well, the kexec thingy should be able to hibernate a machine, but it
will not be "software-suspend" any more, and it may be quite a lot of
work to get it going.

In the end, you'll get rid of freezer problems, but will have two
kernels to care about, and certainly more conventional design. I do
not think I have time to try that (and don't think freezer problems
are _that_ bad in the first place), but some interested soul could
certainly try it.

If we deprecate swsusp in 2.6.45 or so, I do not think I'll be crying.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation Redesign

[Al Boldi]

Pavel Machek wrote:
> > >>from a *new* kernel space/user space that is created by loading a new
> > >>kernel in a manner similar to what is done for kexec crashdumps. 
> > >> Unlike kexec crashdumps, however, it would not require reserving any
> > >> memory at boot, because the necessary memory (maybe 16MB or 64MB) can
> > >> be freed just before hibernating, and device drivers can be properly
> > >> stopped so that DMAs don't stomp over certain memory.

Interesting proposition, which may actually work.  Thanks!

> > >This is the Morton method, isn't it? :) I remember it sounding like a
> > >very good idea when he brought it up, but I can't remember the details
> > >of why it was rejected or what the problems were.
> >
> > Hmm, and it seems like I won't get to know without reliving what
> > looks like an epic flamewar starting.... here:
> >
> >   http://thread.gmane.org/gmane.linux.kernel/374889
> >
> > However from a quick look it seems like the only reason is the RAM
> > overhead of a reserve area. It seems unfortunate that it was
> > dismissed so quickly because of that problem (which could be
> > improved).
>
> Well, the kexec thingy should be able to hibernate a machine, but it
> will not be "software-suspend" any more, and it may be quite a lot of
> work to get it going.
>
> In the end, you'll get rid of freezer problems, but will have two
> kernels to care about, and certainly more conventional design. I do
> not think I have time to try that (and don't think freezer problems
> are _that_ bad in the first place), but some interested soul could
> certainly try it.

Who said we need two kernels?  You could inline it like Xen, which would give 
you one kernel with two modes:  normal and hibernate.


Thanks!

--
Al
  

Re: Hibernation Redesign

[Nick Piggin]

Al Boldi wrote:
> Pavel Machek wrote:

>>In the end, you'll get rid of freezer problems, but will have two
>>kernels to care about, and certainly more conventional design. I do
>>not think I have time to try that (and don't think freezer problems
>>are _that_ bad in the first place), but some interested soul could
>>certainly try it.
> 
> 
> Who said we need two kernels?  You could inline it like Xen, which would give 
> you one kernel with two modes:  normal and hibernate.


kexec can boot the currently running kernel image. There is no
need for two kernels.

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Al Boldi <a1426z@gawab.com> writes:

[snip]

> Who said we need two kernels?  You could inline it like Xen, which would give 
> you one kernel with two modes:  normal and hibernate.

I don't know a whole lot about xen, but it seems that one issue with
this approach is that it requires you run your system under a hypervisor
at all times, which may introduce some overhead.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Jeremy Fitzhardinge]

Jeremy Maitin-Shepard wrote:
> I don't know a whole lot about xen, but it seems that one issue with
> this approach is that it requires you run your system under a hypervisor
> at all times, which may introduce some overhead.
>   

No, I don't think that's what Al is proposing.  The kernel-internal 
interfaces we've put in place to make Xen work could be reused to do 
some of the things you're talking about.  In particular, a kernel 
running under Xen has to be able to deal with non-contiguous physical 
pages, and reusing the same pagetable hooks would allow a kexeced kernel 
to run happily out of any random assortment of pages you manage to 
allocate for it.

    J

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Jeremy Fitzhardinge <jeremy@goop.org> writes:

> Jeremy Maitin-Shepard wrote:
>> I don't know a whole lot about xen, but it seems that one issue with
>> this approach is that it requires you run your system under a hypervisor
>> at all times, which may introduce some overhead.
>> 

> No, I don't think that's what Al is proposing.  The kernel-internal interfaces
> we've put in place to make Xen work could be reused to do some of the things
> you're talking about.  In particular, a kernel running under Xen has to be able
> to deal with non-contiguous physical pages, and reusing the same pagetable hooks
> would allow a kexeced kernel to run happily out of any random assortment of
> pages you manage to allocate for it.

I suppose that would be an interesting thing to look into.  Another
possible approach for having the kernel run in non-contiguous memory is
to specify a memmap exactly to the kernel on the command-line, as I
believe is done for the crashdump kernels currently.  It would, of
course, require an extremely long and complicated memmap specification
in general.  I recall reading, though, that even with the relocatable
kernel support, there are still significant alignment requirements for
loading the kernel.  In particular, I seem to recall that it is
necessary to load an x86 kernel at maybe a 16MB boundary, and on other
platforms the alignment requirements may be even more restrictive.  In
addition, I recall that the Linux boot procedure on x86 and on some
other platforms necessarily uses certain low-address memory, like the
first 640K, which must be backed up regardless.

For these reasons, it seems that it would be easiest to simply backup
the first e.g. 16 or 64 MB of memory, and not have to worry about
loading the kernel at a non-standard address and specifying a
complicated exact memmap.  Someone might prove me wrong, though.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Jeremy Fitzhardinge]

Jeremy Maitin-Shepard wrote:
> I suppose that would be an interesting thing to look into.  Another
> possible approach for having the kernel run in non-contiguous memory is
> to specify a memmap exactly to the kernel on the command-line, as I
> believe is done for the crashdump kernels currently. 

That sounds very fragile.  It would be better to extend the bootparams 
to contain that information.

> I recall reading, though, that even with the relocatable
> kernel support, there are still significant alignment requirements for
> loading the kernel.  In particular, I seem to recall that it is
> necessary to load an x86 kernel at maybe a 16MB boundary, and on other
> platforms the alignment requirements may be even more restrictive.

2MB for x86, I think.  But that's not really an issue if you use a 
P(seudo-physical) to M(achine) mapping, since you can choose any 
arrangement you like for the kernel.  The only restriction is that you 
can't use large pages any more, but I don't think that's an issue for a 
dump/hibernation kernel.

>   In
> addition, I recall that the Linux boot procedure on x86 and on some
> other platforms necessarily uses certain low-address memory, like the
> first 640K, which must be backed up regardless.
>   

Well, the traditional framebuffer/ISA space between 640k and 1M probably 
needs to be identity mapped, but I don't think there's anything in there 
which specifically needs to be save/restored (except framebuffer 
contents, maybe?).

> For these reasons, it seems that it would be easiest to simply backup
> the first e.g. 16 or 64 MB of memory, and not have to worry about
> loading the kernel at a non-standard address and specifying a
> complicated exact memmap.  Someone might prove me wrong, though.
>   

Yes, I suppose.  You're certain the old kernel's devices are completely 
quiescent at that point?

    J

Re: Hibernation Redesign

[Al Boldi]

Jeremy Fitzhardinge wrote:
> Jeremy Maitin-Shepard wrote:
> >   In
> > addition, I recall that the Linux boot procedure on x86 and on some
> > other platforms necessarily uses certain low-address memory, like the
> > first 640K, which must be backed up regardless.
>
> Well, the traditional framebuffer/ISA space between 640k and 1M probably
> needs to be identity mapped, but I don't think there's anything in there
> which specifically needs to be save/restored (except framebuffer
> contents, maybe?).
>
> > For these reasons, it seems that it would be easiest to simply backup
> > the first e.g. 16 or 64 MB of memory, and not have to worry about
> > loading the kernel at a non-standard address and specifying a
> > complicated exact memmap.  Someone might prove me wrong, though.
>
> Yes, I suppose.  You're certain the old kernel's devices are completely
> quiescent at that point?

That's exactly the problem; trying to save a state from within the kernel 
would probably necessitate a freezer hack, which we are trying so dearly to 
avoid.

The only way the kexec approach may be successful, is by completely relying 
on the kexec'd hibernate-mode kernel to save state, otherwise we would be 
back to square one.  And if that required a special boot procedure for the 
kexec'd hibernate-mode kernel, then why not?


Thanks!

--
Al

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 06:11, Al Boldi wrote:
> Jeremy Fitzhardinge wrote:
> > Jeremy Maitin-Shepard wrote:
> > >   In
> > > addition, I recall that the Linux boot procedure on x86 and on some
> > > other platforms necessarily uses certain low-address memory, like the
> > > first 640K, which must be backed up regardless.
> >
> > Well, the traditional framebuffer/ISA space between 640k and 1M probably
> > needs to be identity mapped, but I don't think there's anything in there
> > which specifically needs to be save/restored (except framebuffer
> > contents, maybe?).
> >
> > > For these reasons, it seems that it would be easiest to simply backup
> > > the first e.g. 16 or 64 MB of memory, and not have to worry about
> > > loading the kernel at a non-standard address and specifying a
> > > complicated exact memmap.  Someone might prove me wrong, though.
> >
> > Yes, I suppose.  You're certain the old kernel's devices are completely
> > quiescent at that point?
> 
> That's exactly the problem; trying to save a state from within the kernel 
> would probably necessitate a freezer hack, which we are trying so dearly to 
> avoid.

Well, I don't think that avoiding the freezer whatever it takes would be a good
idea.  There needs to be some balance. ;-)

> The only way the kexec approach may be successful, is by completely relying 
> on the kexec'd hibernate-mode kernel to save state, otherwise we would be 
> back to square one.  And if that required a special boot procedure for the 
> kexec'd hibernate-mode kernel, then why not?

Because such things are very difficult for many users.  My experience with
the userland suspend shows that clearly.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Miklos Szeredi]

> > > Yes, I suppose.  You're certain the old kernel's devices are completely
> > > quiescent at that point?
> > 
> > That's exactly the problem; trying to save a state from within the
> > kernel would probably necessitate a freezer hack, which we are
> > trying so dearly to avoid.
> 
> Well, I don't think that avoiding the freezer whatever it takes
> would be a good idea.  There needs to be some balance. ;-)

Well, it takes some extra locking in the drivers.  Which is needed
_anyway_ if we want to have a working s2ram without the freezer.

With the kexec approach, I don't see any extra requirements from the
kernel to be able to drop the freezer.

Miklos

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 12:42, Miklos Szeredi wrote:
> > > > Yes, I suppose.  You're certain the old kernel's devices are completely
> > > > quiescent at that point?
> > > 
> > > That's exactly the problem; trying to save a state from within the
> > > kernel would probably necessitate a freezer hack, which we are
> > > trying so dearly to avoid.
> > 
> > Well, I don't think that avoiding the freezer whatever it takes
> > would be a good idea.  There needs to be some balance. ;-)
> 
> Well, it takes some extra locking in the drivers.  Which is needed
> _anyway_ if we want to have a working s2ram without the freezer.
> 
> With the kexec approach, I don't see any extra requirements from the
> kernel to be able to drop the freezer.

This is not my point.  I think that if what it takes to implement the kexec
approach, as a complete working solution, is much more complicated than
what we have now, then the current soultion is favorable.

Anyway, to implement the kexec approach we must separate the hibernation
from the suspend at the drivers level, which I'm still going to do, but I need to
take part in endless discussions regarding the freezer, how it is bad and
how we should drop it, because it breaks things (which NB is not true, because
it doesn't).

>From a practical point of view, the freezer is not the most problematic part
of the infrastructure.  For example, none of the bug reports that we have
registered in the bugzilla is related to the freezer.  Moreover, to drop it,
we first need to redesign the other things.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Miklos Szeredi]

> Anyway, to implement the kexec approach we must separate the
> hibernation from the suspend at the drivers level, which I'm still
> going to do, but I need to take part in endless discussions

Discussions are good.  We understand the problem better.  Now I still
think we don't understand every aspect completely, so continuing the
discussion makes sense.

> regarding the freezer, how it is bad and how we should drop it,
> because it breaks things (which NB is not true, because it doesn't).

This thread started out from a bug, that seemed to be caused by the
freezer (we still don't exactly know what it was caused by), and the
discussion uncovered various problems _with_ the freezer, that up to
now no other _proper_ solutions have been propsed than to remove the
freezer.

Yes, we can live with hacks, but we don't want to live with them
forever.  Even if they seem to be simpler than a big rewrite, the long
term headaches are not worth it.

Miklos

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 13:11, Miklos Szeredi wrote:
> > Anyway, to implement the kexec approach we must separate the
> > hibernation from the suspend at the drivers level, which I'm still
> > going to do, but I need to take part in endless discussions
> 
> Discussions are good.  We understand the problem better.  Now I still
> think we don't understand every aspect completely, so continuing the
> discussion makes sense.

OK

I can't do A until I do B, but when I do B, then the entire problem will
change, so what's the point in discussing A before doing B?

> > regarding the freezer, how it is bad and how we should drop it,
> > because it breaks things (which NB is not true, because it doesn't).
> 
> This thread started out from a bug, that seemed to be caused by the
> freezer (we still don't exactly know what it was caused by), and the
> discussion uncovered various problems _with_ the freezer, that up to
> now no other _proper_ solutions have been propsed than to remove the
> freezer.

First, please list those problems for completness.  Moreover, I think they
should be documented if they haven't been yet.

Second, there were many more problems with the freezer in the past and we
were able to eliminate them over time.  It sometimes is difficult to invent a
viable solution on demand ...

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Miklos Szeredi]

> > > regarding the freezer, how it is bad and how we should drop it,
> > > because it breaks things (which NB is not true, because it doesn't).
> > 
> > This thread started out from a bug, that seemed to be caused by the
> > freezer (we still don't exactly know what it was caused by), and the
> > discussion uncovered various problems _with_ the freezer, that up to
> > now no other _proper_ solutions have been propsed than to remove the
> > freezer.
> 
> First, please list those problems for completness.  Moreover, I think they
> should be documented if they haven't been yet.

Suspend failing due to ordering requirements from fuse, I think you
documented this one.

Suspend failing due to malicious/buggy fuse filesystem.

Suspend failing if network filesystem is mounted, and network or
server is down.  This is one of those "we can live with it" things
that only happen rarely, and it's annoying then, but soon forgotten.

Freezing of tasks is slowing down suspend.  Don't know how serious
this is, suspend is pretty fast, but could possibly be even faster.

Don't know what the OLPC guys do to make suspending extremely quick,
are they freezing any tasks?

> Second, there were many more problems with the freezer in the past and we
> were able to eliminate them over time.  It sometimes is difficult to invent a
> viable solution on demand ...

OK, if you elimitate all of the above problems with the freezer, I'll
say no more against it.

Miklos

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

Hi.

On Wednesday 11 July 2007 21:54:58 Miklos Szeredi wrote:
> Freezing of tasks is slowing down suspend.  Don't know how serious
> this is, suspend is pretty fast, but could possibly be even faster.

It's FUD. Freezing of tasks normally takes next to no time. I've never 
understood the rediculously long timeout it has. If freezing succeeds, all 
processes are frozen within 1/2 a second tops. If it fails, nothing is going 
to change in the following 19.5 seconds (or whatever it is if I don't 
remember the value properly).
 
Regards,

Nigel


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Miklos Szeredi]

> > Freezing of tasks is slowing down suspend.  Don't know how serious
> > this is, suspend is pretty fast, but could possibly be even faster.
> 
> It's FUD. Freezing of tasks normally takes next to no time. I've never 
> understood the rediculously long timeout it has. If freezing succeeds, all 
> processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> to change in the following 19.5 seconds (or whatever it is if I don't 
> remember the value properly).

Right.  The 20s timeout is again a sign of brokenness.

If we expect something to fail, it should fail immediately, without
waiting for arbitrary timeouts.

And if we don't expect it to fail, why the timeout?

Of course we know it can fail (network problems, etc), so it's wrong
whatever way we look at it.

Miklos

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

Hi.

On Wednesday 11 July 2007 22:09:39 Miklos Szeredi wrote:
> > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > this is, suspend is pretty fast, but could possibly be even faster.
> > 
> > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > understood the rediculously long timeout it has. If freezing succeeds, all 
> > processes are frozen within 1/2 a second tops. If it fails, nothing is 
going 
> > to change in the following 19.5 seconds (or whatever it is if I don't 
> > remember the value properly).
> 
> Right.  The 20s timeout is again a sign of brokenness.
> 
> If we expect something to fail, it should fail immediately, without
> waiting for arbitrary timeouts.
> 
> And if we don't expect it to fail, why the timeout?

Two reasons:

1) The processing does take time. We're sending (pseudo-)signals to other 
processes, and the amount of time it will take for them to enter the 
refrigerator will depend upon the number of processes there are, the 
scheduling algorithm being used, the rate at which they're forking and what 
other activity is occuring. That's why I said "normally" above.

2) Things do get broken. New functionality gets added which sometimes doesn't 
immediately play well with the freezer. In such cases, we want to fail nicely 
rather than wrongly assuming it will always work and waiting indefinitely.

Regards,

Nigel
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 14:09, Miklos Szeredi wrote:
> > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > this is, suspend is pretty fast, but could possibly be even faster.
> > 
> > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > understood the rediculously long timeout it has. If freezing succeeds, all 
> > processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> > to change in the following 19.5 seconds (or whatever it is if I don't 
> > remember the value properly).
> 
> Right.  The 20s timeout is again a sign of brokenness.

Are you still serious?

> If we expect something to fail, it should fail immediately, without
> waiting for arbitrary timeouts.

I don't agree.  If you think so, then please tell me what the softlockup
infrastructure is for.

> And if we don't expect it to fail, why the timeout?

We know that it can fail, so we use the timeout to detect failures.
 
> Of course we know it can fail (network problems, etc), so it's wrong
> whatever way we look at it.

Are you trying to say that whatever can fail is wrong?

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Miklos Szeredi]

> > > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > > this is, suspend is pretty fast, but could possibly be even faster.
> > > 
> > > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > > understood the rediculously long timeout it has. If freezing succeeds, all 
> > > processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> > > to change in the following 19.5 seconds (or whatever it is if I don't 
> > > remember the value properly).
> > 
> > Right.  The 20s timeout is again a sign of brokenness.
> 
> Are you still serious?
> 
> > If we expect something to fail, it should fail immediately, without
> > waiting for arbitrary timeouts.
> 
> I don't agree.  If you think so, then please tell me what the softlockup
> infrastructure is for.
> 
> > And if we don't expect it to fail, why the timeout?
> 
> We know that it can fail, so we use the timeout to detect failures.
>  
> > Of course we know it can fail (network problems, etc), so it's wrong
> > whatever way we look at it.
> 
> Are you trying to say that whatever can fail is wrong?

No. Sorry about the sloppy sentence.

What I was trying to say, is that if we _know_ that the suspend can
fail, it is wrong to have a timeout to determine that it will fail.

Miklos

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 14:29, Miklos Szeredi wrote:
> > > > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > > > this is, suspend is pretty fast, but could possibly be even faster.
> > > > 
> > > > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > > > understood the rediculously long timeout it has. If freezing succeeds, all 
> > > > processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> > > > to change in the following 19.5 seconds (or whatever it is if I don't 
> > > > remember the value properly).
> > > 
> > > Right.  The 20s timeout is again a sign of brokenness.
> > 
> > Are you still serious?
> > 
> > > If we expect something to fail, it should fail immediately, without
> > > waiting for arbitrary timeouts.
> > 
> > I don't agree.  If you think so, then please tell me what the softlockup
> > infrastructure is for.
> > 
> > > And if we don't expect it to fail, why the timeout?
> > 
> > We know that it can fail, so we use the timeout to detect failures.
> >  
> > > Of course we know it can fail (network problems, etc), so it's wrong
> > > whatever way we look at it.
> > 
> > Are you trying to say that whatever can fail is wrong?
> 
> No. Sorry about the sloppy sentence.
> 
> What I was trying to say, is that if we _know_ that the suspend can
> fail, it is wrong to have a timeout to determine that it will fail.

Hmm, I'm not sure what you mean ...

If there are conditions in which it's not a good idea to hibernate (let's
consider hibernation only for clarity), I think we can use a timeout.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Pavel Machek]

Hi!

> > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > this is, suspend is pretty fast, but could possibly be even faster.
> > 
> > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > understood the rediculously long timeout it has. If freezing succeeds, all 
> > processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> > to change in the following 19.5 seconds (or whatever it is if I don't 
> > remember the value properly).
> 
> Right.  The 20s timeout is again a sign of brokenness.

Well, "scenario 1/2 - simple/tricky deadlock" is a sign of brokenness,
too.

> If we expect something to fail, it should fail immediately, without
> waiting for arbitrary timeouts.

Agreed that freezer is not nice, but core issue here is that fuse
allows userspace tasks to hold kernel locks... bad.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Hibernation Redesign

[Miklos Szeredi]

> > > > Freezing of tasks is slowing down suspend.  Don't know how serious
> > > > this is, suspend is pretty fast, but could possibly be even faster.
> > > 
> > > It's FUD. Freezing of tasks normally takes next to no time. I've never 
> > > understood the rediculously long timeout it has. If freezing succeeds, all 
> > > processes are frozen within 1/2 a second tops. If it fails, nothing is going 
> > > to change in the following 19.5 seconds (or whatever it is if I don't 
> > > remember the value properly).
> > 
> > Right.  The 20s timeout is again a sign of brokenness.
> 
> Well, "scenario 1/2 - simple/tricky deadlock" is a sign of brokenness,
> too.

Sure.  But that one hasn't generated a single bug report (and never
will), because it's so insanely difficult to produce, even if one
explicitly wants to do that.  It will just never happen accidentally.

And even if it happens, there's absolutely no side affects on the rest
of the system.  You managed to make a process not react to SIGKILL,
well hurray.  You can do that more easily with ptrace().

Miklos

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 13:54, Miklos Szeredi wrote:
> > > > regarding the freezer, how it is bad and how we should drop it,
> > > > because it breaks things (which NB is not true, because it doesn't).
> > > 
> > > This thread started out from a bug, that seemed to be caused by the
> > > freezer (we still don't exactly know what it was caused by), and the
> > > discussion uncovered various problems _with_ the freezer, that up to
> > > now no other _proper_ solutions have been propsed than to remove the
> > > freezer.
> > 
> > First, please list those problems for completness.  Moreover, I think they
> > should be documented if they haven't been yet.
> 
> Suspend failing due to ordering requirements from fuse, I think you
> documented this one.
> 
> Suspend failing due to malicious/buggy fuse filesystem.
> 
> Suspend failing if network filesystem is mounted, and network or
> server is down.  This is one of those "we can live with it" things
> that only happen rarely, and it's annoying then, but soon forgotten.

All of the above boil down to the freezer limitation regarding uninterruptible
tasks.  And yes, I think it should be explicitly documented.

> Freezing of tasks is slowing down suspend.  Don't know how serious
> this is, suspend is pretty fast, but could possibly be even faster.

Hmm, I don't know either.  I'm not seeing this in my tests, but well, more
data will be appreciated.

The sync probably slows down more than the freezing itself ...

> Don't know what the OLPC guys do to make suspending extremely quick,
> are they freezing any tasks?

They used to, but I'm not sure if they do.

> > Second, there were many more problems with the freezer in the past and we
> > were able to eliminate them over time.  It sometimes is difficult to invent a
> > viable solution on demand ...
> 
> OK, if you elimitate all of the above problems with the freezer, I'll
> say no more against it.

I think the ordering issue may be eliminated, to some extent.  I have a patch
for that, but you were CCed on that patch series, so you should know. ;-)

The other ones are more difficult, so time will tell.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

Hi.

On Wednesday 11 July 2007 22:19:10 Rafael J. Wysocki wrote:
> The sync probably slows down more than the freezing itself ...

Absolutely. Waaaay more. Maybe it would help if you popped in some printks 
that help people see that it's not the freezer taking a long time, but 
syncing?
 
Nigel

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Rafael J. Wysocki]

Hi,

On Wednesday, 11 July 2007 14:49, Nigel Cunningham wrote:
> Hi.
> 
> On Wednesday 11 July 2007 22:19:10 Rafael J. Wysocki wrote:
> > The sync probably slows down more than the freezing itself ...
> 
> Absolutely. Waaaay more. Maybe it would help if you popped in some printks 
> that help people see that it's not the freezer taking a long time, but 
> syncing?

In a while, I'll be posting a freezer update to linux-pm.  It contains
something like this among other things.  Please have a look.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 1858 bytes --]

Hi.

On Wednesday 11 July 2007 21:11:34 Miklos Szeredi wrote:
> > Anyway, to implement the kexec approach we must separate the
> > hibernation from the suspend at the drivers level, which I'm still
> > going to do, but I need to take part in endless discussions
> 
> Discussions are good.  We understand the problem better.  Now I still
> think we don't understand every aspect completely, so continuing the
> discussion makes sense.
> 
> > regarding the freezer, how it is bad and how we should drop it,
> > because it breaks things (which NB is not true, because it doesn't).
> 
> This thread started out from a bug, that seemed to be caused by the
> freezer (we still don't exactly know what it was caused by), and the
> discussion uncovered various problems _with_ the freezer, that up to
> now no other _proper_ solutions have been propsed than to remove the
> freezer.

No other _proper_ solutions have been proposed. Everyone who suggests removing 
the freezer also suggests implementing it all over again. It might be sending 
SIGSTOP to everything. It might be shifting the desk chairs around and 
creating a completely new kernel context, but they always have the same 
goal - stopping the existing activity, and they all come with their own 
issues (even if they're not obvious yet because the alternatives are 
currently vapourware to one extent or another).

IMHO, the real solution is to go back to the original issue and fix it 
properly. Make fuse filesystems play nicely with the existing freezer. I've 
just gone back and looked at the point where you started talking 
about "malicious filesystems". You talk about fuse imposing certain ordering 
in the userspace tasks being frozen. Please, say more. What ordering issues? 
Why? How can such ordering be determined programmatically?
 
Regards,

Nigel

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Miklos Szeredi]

> No other _proper_ solutions have been proposed. Everyone who suggests removing 
> the freezer also suggests implementing it all over again. It might be sending 
> SIGSTOP to everything. It might be shifting the desk chairs around and 
> creating a completely new kernel context, but they always have the same 
> goal - stopping the existing activity, and they all come with their own 
> issues (even if they're not obvious yet because the alternatives are 
> currently vapourware to one extent or another).

Hey, I know precious little about drivers and power management, but I
can clarly see, that

 a) stopping all tasks and requiring them to finish any syscall they
 are in

 b) stopping tasks _in the driver_ that are trying to access the
 harware during/after suspend

are completely different solutions, the later being much more fine
grained and not having all of the mentioned problems that the former
exhibits.

> IMHO, the real solution is to go back to the original issue and fix it 
> properly. Make fuse filesystems play nicely with the existing freezer. I've 
> just gone back and looked at the point where you started talking 
> about "malicious filesystems". You talk about fuse imposing certain ordering 
> in the userspace tasks being frozen. Please, say more. What ordering issues? 
> Why? How can such ordering be determined programmatically?

It can't.  If you are interested, please read through that thread.  If
something's still not clear, let's discuss it further.

Miklos

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 2284 bytes --]

Hi.

On Wednesday 11 July 2007 22:24:04 Miklos Szeredi wrote:
> > No other _proper_ solutions have been proposed. Everyone who suggests 
removing 
> > the freezer also suggests implementing it all over again. It might be 
sending 
> > SIGSTOP to everything. It might be shifting the desk chairs around and 
> > creating a completely new kernel context, but they always have the same 
> > goal - stopping the existing activity, and they all come with their own 
> > issues (even if they're not obvious yet because the alternatives are 
> > currently vapourware to one extent or another).
> 
> Hey, I know precious little about drivers and power management, but I
> can clarly see, that
> 
>  a) stopping all tasks and requiring them to finish any syscall they
>  are in
> 
>  b) stopping tasks _in the driver_ that are trying to access the
>  harware during/after suspend
> 
> are completely different solutions, the later being much more fine
> grained and not having all of the mentioned problems that the former
> exhibits.

The point to freezing tasks isn't just to stop drivers doing work. It's also 
to stop userspace doing work and thereby increase reliability. The more work 
that is occuring while we're trying to write a hibernation image, the less 
reliable the hibernation will be (competing for memory and so on) and the 
slower it will be (competing for cycles etc).
 
> > IMHO, the real solution is to go back to the original issue and fix it 
> > properly. Make fuse filesystems play nicely with the existing freezer. 
I've 
> > just gone back and looked at the point where you started talking 
> > about "malicious filesystems". You talk about fuse imposing certain 
ordering 
> > in the userspace tasks being frozen. Please, say more. What ordering 
issues? 
> > Why? How can such ordering be determined programmatically?
> 
> It can't.  If you are interested, please read through that thread.  If
> something's still not clear, let's discuss it further.

You can say it "imposes certain ordering" but you can say what that ordering 
is?! How about mount order? That must be a good start.

Regards,

Nigel (off to bed now).
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Miklos Szeredi]

> The point to freezing tasks isn't just to stop drivers doing work. It's also 
> to stop userspace doing work and thereby increase reliability. The more work 
> that is occuring while we're trying to write a hibernation image, the less 
> reliable the hibernation will be (competing for memory and so on) and the 
> slower it will be (competing for cycles etc).

We are talking about a kexec based hibernate.  All of the above just
doesn't apply in that case.

I'm not contending, that the current hibernation (epecially Pavel's
new userspace based one) needs a frozen userland.

> > > Why? How can such ordering be determined programmatically?
> > 
> > It can't.  If you are interested, please read through that thread.  If
> > something's still not clear, let's discuss it further.
> 
> You can say it "imposes certain ordering" but you can say what that ordering 
> is?! How about mount order? That must be a good start.

The order in which tasks are relying on each other to be able to
finish a syscall.

Mount order says _nothing_ about tasks.

Miklos

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Nigel Cunningham <nigel@nigel.suspend2.net> writes:

[snip]

> No other _proper_ solutions have been proposed. Everyone who suggests removing 
> the freezer also suggests implementing it all over again. It might be sending 
> SIGSTOP to everything. It might be shifting the desk chairs around and 
> creating a completely new kernel context, but they always have the same 
> goal - stopping the existing activity, and they all come with their own 
> issues (even if they're not obvious yet because the alternatives are 
> currently vapourware to one extent or another).

I'll certainly admit the kexec idea is vaporware currently, but it does
differ in a significant way from freezer-based approaches, such that I
don't think it should be referred to as just another implementation of a
freezer.  Specifically, it doesn't require that the "old kernel" be in a
"consistent" state to a greater extent than suspend to ram; it is the
case that all of the devices must be quiesced or shut down to some
extent, but doing this without races and deadlocks (and without the
freezer) is certainly very, very similar to what needs to be done for
suspend to ram, which will need to be solved anyway.  Unlike the
existing hibernate approaches, however, it will not be necessary to use
any of the driver infrastructure once switched to the "save image"
kernel, and thus it will not matter what locks are held, for instance.

[snip]

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Mark Lord]

Jeremy Maitin-Shepard wrote:
>>
> I'll certainly admit the kexec idea is vaporware currently, but it does
> differ in a significant way from freezer-based approaches, such that I
> don't think it should be referred to as just another implementation of a
> freezer.  Specifically, it doesn't require that the "old kernel" be in a
> "consistent" state to a greater extent than suspend to ram; it is the
> case that all of the devices must be quiesced or shut down to some
> extent, but doing this without races and deadlocks (and without the
> freezer) is certainly very, very similar to what needs to be done for
> suspend to ram, which will need to be solved anyway.  Unlike the
> existing hibernate approaches, however, it will not be necessary to use
> any of the driver infrastructure once switched to the "save image"
> kernel, and thus it will not matter what locks are held, for instance.

I really doubt that kexec(a special kernel) is going to solve anything here.
The new kernel will have to initialize, probe for devices, etc.
Which will take time.
Which will slow down hibernate to an unacceptable degree.
Right now, it (TuxOnIce) is *very* fast.
Adding 10 seconds or so for reprobing/resetting/reiniting devices
is not going to be useful.
And modifying all of the drivers to *not* do their usual probe sequence
sounds rather intrusive and is likely also a non-starter here.

Or is it?

Cheers

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Wednesday, 11 July 2007 22:48, Mark Lord wrote:
> Jeremy Maitin-Shepard wrote:
> >>
> > I'll certainly admit the kexec idea is vaporware currently, but it does
> > differ in a significant way from freezer-based approaches, such that I
> > don't think it should be referred to as just another implementation of a
> > freezer.  Specifically, it doesn't require that the "old kernel" be in a
> > "consistent" state to a greater extent than suspend to ram; it is the
> > case that all of the devices must be quiesced or shut down to some
> > extent, but doing this without races and deadlocks (and without the
> > freezer) is certainly very, very similar to what needs to be done for
> > suspend to ram, which will need to be solved anyway.  Unlike the
> > existing hibernate approaches, however, it will not be necessary to use
> > any of the driver infrastructure once switched to the "save image"
> > kernel, and thus it will not matter what locks are held, for instance.
> 
> I really doubt that kexec(a special kernel) is going to solve anything here.
> The new kernel will have to initialize, probe for devices, etc.
> Which will take time.
> Which will slow down hibernate to an unacceptable degree.
> Right now, it (TuxOnIce) is *very* fast.
> Adding 10 seconds or so for reprobing/resetting/reiniting devices
> is not going to be useful.
> And modifying all of the drivers to *not* do their usual probe sequence
> sounds rather intrusive and is likely also a non-starter here.
> 
> Or is it?

Well, we're going to change what the drivers do during hibernation
for other reasons, so it may well be non-issue.  Still, we can only speculate
until it's done.

Which is why I think that the present discussion is premature.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Al Boldi]

Mark Lord wrote:
> Jeremy Maitin-Shepard wrote:
> > I'll certainly admit the kexec idea is vaporware currently,

Your idea is starting to become a reality with this thread:
"[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"

> > but it does
> > differ in a significant way from freezer-based approaches, such that I
> > don't think it should be referred to as just another implementation of a
> > freezer.  Specifically, it doesn't require that the "old kernel" be in a
> > "consistent" state to a greater extent than suspend to ram; it is the
> > case that all of the devices must be quiesced or shut down to some
> > extent, but doing this without races and deadlocks (and without the
> > freezer) is certainly very, very similar to what needs to be done for
> > suspend to ram, which will need to be solved anyway.  Unlike the
> > existing hibernate approaches, however, it will not be necessary to use
> > any of the driver infrastructure once switched to the "save image"
> > kernel, and thus it will not matter what locks are held, for instance.
>
> I really doubt that kexec(a special kernel) is going to solve anything
> here. The new kernel will have to initialize, probe for devices, etc.
> Which will take time.
> Which will slow down hibernate to an unacceptable degree.
> Right now, it (TuxOnIce) is *very* fast.
> Adding 10 seconds or so for reprobing/resetting/reiniting devices
> is not going to be useful.
> And modifying all of the drivers to *not* do their usual probe sequence
> sounds rather intrusive and is likely also a non-starter here.
>
> Or is it?

Well, it's definitely less intrusive than readying drivers for the freezer.  
And, it's not even necessary for a modular kernel, where you would only 
build in the drivers you need to hibernate.


Thanks!

--
Al

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Thursday, 12 July 2007 00:17, Al Boldi wrote:
> Mark Lord wrote:
> > Jeremy Maitin-Shepard wrote:
> > > I'll certainly admit the kexec idea is vaporware currently,
> 
> Your idea is starting to become a reality with this thread:
> "[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"
> 
> > > but it does
> > > differ in a significant way from freezer-based approaches, such that I
> > > don't think it should be referred to as just another implementation of a
> > > freezer.  Specifically, it doesn't require that the "old kernel" be in a
> > > "consistent" state to a greater extent than suspend to ram; it is the
> > > case that all of the devices must be quiesced or shut down to some
> > > extent, but doing this without races and deadlocks (and without the
> > > freezer) is certainly very, very similar to what needs to be done for
> > > suspend to ram, which will need to be solved anyway.  Unlike the
> > > existing hibernate approaches, however, it will not be necessary to use
> > > any of the driver infrastructure once switched to the "save image"
> > > kernel, and thus it will not matter what locks are held, for instance.
> >
> > I really doubt that kexec(a special kernel) is going to solve anything
> > here. The new kernel will have to initialize, probe for devices, etc.
> > Which will take time.
> > Which will slow down hibernate to an unacceptable degree.
> > Right now, it (TuxOnIce) is *very* fast.
> > Adding 10 seconds or so for reprobing/resetting/reiniting devices
> > is not going to be useful.
> > And modifying all of the drivers to *not* do their usual probe sequence
> > sounds rather intrusive and is likely also a non-starter here.
> >
> > Or is it?
> 
> Well, it's definitely less intrusive than readying drivers for the freezer.

There's nothing like this!

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Al Boldi]

Rafael J. Wysocki wrote:
> On Thursday, 12 July 2007 00:17, Al Boldi wrote:
> > Mark Lord wrote:
> > > Jeremy Maitin-Shepard wrote:
> > > > I'll certainly admit the kexec idea is vaporware currently,
> >
> > Your idea is starting to become a reality with this thread:
> > "[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"
> >
> > > > but it does
> > > > differ in a significant way from freezer-based approaches, such that
> > > > I don't think it should be referred to as just another
> > > > implementation of a freezer.  Specifically, it doesn't require that
> > > > the "old kernel" be in a "consistent" state to a greater extent than
> > > > suspend to ram; it is the case that all of the devices must be
> > > > quiesced or shut down to some extent, but doing this without races
> > > > and deadlocks (and without the freezer) is certainly very, very
> > > > similar to what needs to be done for suspend to ram, which will need
> > > > to be solved anyway.  Unlike the existing hibernate approaches,
> > > > however, it will not be necessary to use any of the driver
> > > > infrastructure once switched to the "save image" kernel, and thus it
> > > > will not matter what locks are held, for instance.
> > >
> > > I really doubt that kexec(a special kernel) is going to solve anything
> > > here. The new kernel will have to initialize, probe for devices, etc.
> > > Which will take time.
> > > Which will slow down hibernate to an unacceptable degree.
> > > Right now, it (TuxOnIce) is *very* fast.
> > > Adding 10 seconds or so for reprobing/resetting/reiniting devices
> > > is not going to be useful.
> > > And modifying all of the drivers to *not* do their usual probe
> > > sequence sounds rather intrusive and is likely also a non-starter
> > > here.
> > >
> > > Or is it?
> >
> > Well, it's definitely less intrusive than readying drivers for the
> > freezer.
>
> There's nothing like this!

Are you sure?

# grep -i freeze drivers/*/* | wc -l

gives:  297

Maybe you can clarify?


Thanks!

--
Al

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 3983 bytes --]

Hi.

On Thursday 12 July 2007 09:12:24 Al Boldi wrote:
> Rafael J. Wysocki wrote:
> > On Thursday, 12 July 2007 00:17, Al Boldi wrote:
> > > Mark Lord wrote:
> > > > Jeremy Maitin-Shepard wrote:
> > > > > I'll certainly admit the kexec idea is vaporware currently,
> > >
> > > Your idea is starting to become a reality with this thread:
> > > "[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"
> > >
> > > > > but it does
> > > > > differ in a significant way from freezer-based approaches, such that
> > > > > I don't think it should be referred to as just another
> > > > > implementation of a freezer.  Specifically, it doesn't require that
> > > > > the "old kernel" be in a "consistent" state to a greater extent than
> > > > > suspend to ram; it is the case that all of the devices must be
> > > > > quiesced or shut down to some extent, but doing this without races
> > > > > and deadlocks (and without the freezer) is certainly very, very
> > > > > similar to what needs to be done for suspend to ram, which will need
> > > > > to be solved anyway.  Unlike the existing hibernate approaches,
> > > > > however, it will not be necessary to use any of the driver
> > > > > infrastructure once switched to the "save image" kernel, and thus it
> > > > > will not matter what locks are held, for instance.
> > > >
> > > > I really doubt that kexec(a special kernel) is going to solve anything
> > > > here. The new kernel will have to initialize, probe for devices, etc.
> > > > Which will take time.
> > > > Which will slow down hibernate to an unacceptable degree.
> > > > Right now, it (TuxOnIce) is *very* fast.
> > > > Adding 10 seconds or so for reprobing/resetting/reiniting devices
> > > > is not going to be useful.
> > > > And modifying all of the drivers to *not* do their usual probe
> > > > sequence sounds rather intrusive and is likely also a non-starter
> > > > here.
> > > >
> > > > Or is it?
> > >
> > > Well, it's definitely less intrusive than readying drivers for the
> > > freezer.
> >
> > There's nothing like this!
> 
> Are you sure?
> 
> # grep -i freeze drivers/*/* | wc -l
> 
> gives:  297
> 
> Maybe you can clarify?

Rafael's probably just gone to sleep, so let me be so bold as to give an 
answer.

First, you need to understand that there is a difference between the process 
freezer and driver support for suspending and resuming. The two are 
completely unrelated. Process freezing happens at the start of a hibernation 
cycle. Driver suspend and resume calls happen prior to and after an atomic 
copy or restore. Process freezing stops processes. Driver suspend and resume 
calls tell drivers to save their state and possibly enter low power modes. 
Process freezing isn't a necessary prerequisite for calling driver 
suspend/resume routines. (That has been the point to discussions about 
suspend to ram).

Since this is the case, doing grep -i freeze drivers/*/* is a bogus test. If 
you want to look for drivers using the freezer, you need to grep for 
try_to_freeze and PF_NOFREEZE. PF_NOFREEZE says the related kernel thread 
shouldn't be frozen (11 hits in the source tree I'm looking at). Grepping for 
try_to_freeze gets 29. By the way, find drivers/ -name '*.[ch]' | xargs grep 
try_to_freeze (for example) is a better search - it isn't dependant upon the 
number of subdirectories.

You'll see from the above numbers that the freezer is not nearly as intrusive 
as you were thinking (~10% of what you wrote above). It is limited to code 
related to kernel threads, and then to either setting a flag when the thread 
is started to say "I don't need to be frozen" or to the appropriate point at 
which to check whether to freeze.

Hope that helps.

Nigel
-- 
Nigel Cunningham
Christian Reformed Church of Cobden
103 Curdie Street, Cobden 3266, Victoria, Australia
Ph. +61 3 5595 1185 / +61 417 100 574
Communal Worship: 11 am Sunday.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Al Boldi]

Nigel Cunningham wrote:
> You'll see from the above numbers that the freezer is not nearly as
> intrusive as you were thinking (~10% of what you wrote above). It is
> limited to code related to kernel threads, and then to either setting a
> flag when the thread is started to say "I don't need to be frozen" or to
> the appropriate point at which to check whether to freeze.

That's what I thought.

> Hope that helps.
> 
> Nigel
> -- 
> Nigel Cunningham
> Christian Reformed Church of Cobden
> 103 Curdie Street, Cobden 3266, Victoria, Australia
> Ph. +61 3 5595 1185 / +61 417 100 574
> Communal Worship: 11 am Sunday.


Try this: http://www.islam-guide.com


Thanks!

--
Al

Re: Hibernation Redesign

[Rafael J. Wysocki]

On Thursday, 12 July 2007 01:12, Al Boldi wrote:
> Rafael J. Wysocki wrote:
> > On Thursday, 12 July 2007 00:17, Al Boldi wrote:
> > > Mark Lord wrote:
> > > > Jeremy Maitin-Shepard wrote:
> > > > > I'll certainly admit the kexec idea is vaporware currently,
> > >
> > > Your idea is starting to become a reality with this thread:
> > > "[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"
> > >
> > > > > but it does
> > > > > differ in a significant way from freezer-based approaches, such that
> > > > > I don't think it should be referred to as just another
> > > > > implementation of a freezer.  Specifically, it doesn't require that
> > > > > the "old kernel" be in a "consistent" state to a greater extent than
> > > > > suspend to ram; it is the case that all of the devices must be
> > > > > quiesced or shut down to some extent, but doing this without races
> > > > > and deadlocks (and without the freezer) is certainly very, very
> > > > > similar to what needs to be done for suspend to ram, which will need
> > > > > to be solved anyway.  Unlike the existing hibernate approaches,
> > > > > however, it will not be necessary to use any of the driver
> > > > > infrastructure once switched to the "save image" kernel, and thus it
> > > > > will not matter what locks are held, for instance.
> > > >
> > > > I really doubt that kexec(a special kernel) is going to solve anything
> > > > here. The new kernel will have to initialize, probe for devices, etc.
> > > > Which will take time.
> > > > Which will slow down hibernate to an unacceptable degree.
> > > > Right now, it (TuxOnIce) is *very* fast.
> > > > Adding 10 seconds or so for reprobing/resetting/reiniting devices
> > > > is not going to be useful.
> > > > And modifying all of the drivers to *not* do their usual probe
> > > > sequence sounds rather intrusive and is likely also a non-starter
> > > > here.
> > > >
> > > > Or is it?
> > >
> > > Well, it's definitely less intrusive than readying drivers for the
> > > freezer.
> >
> > There's nothing like this!
> 
> Are you sure?
> 
> # grep -i freeze drivers/*/* | wc -l
> 
> gives:  297
> 
> Maybe you can clarify?

Yes, I can.  For example, please have a look at 'struct ata_port_operations'. :-)

The only direct relationship between the freezer and drivers is that some of
them use kernel threads that call try_to_freeze() (and other freezer-related
functions).

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: Hibernation Redesign

[Nick Piggin]

Rafael J. Wysocki wrote:

> The only direct relationship between the freezer and drivers is that some of
> them use kernel threads that call try_to_freeze() (and other freezer-related
> functions).

If removing thoset was the only benefit of getting rid of the freezer
with kexec, it would almost be worth it, IMO :)

-- 
SUSE Labs, Novell Inc.

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Al Boldi <a1426z@gawab.com> writes:

> Mark Lord wrote:
>> Jeremy Maitin-Shepard wrote:
>> > I'll certainly admit the kexec idea is vaporware currently,

> Your idea is starting to become a reality with this thread:
> "[PATCH 0/2] Kexec jump: The first step to kexec base hibernation"

Someone else pointed out that the idea was actually proposed by Andrew
Morton over a year ago, but it didn't get very much consideration then.

It is good to see that quite a few people are thinking about it now,
though, and that Ying Huang has started writing some code.

-- 
Jeremy Maitin-Shepard

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 2685 bytes --]

Hi.

On Wednesday 11 July 2007 23:16:41 Jeremy Maitin-Shepard wrote:
> Nigel Cunningham <nigel@nigel.suspend2.net> writes:
> 
> [snip]
> 
> > No other _proper_ solutions have been proposed. Everyone who suggests 
removing 
> > the freezer also suggests implementing it all over again. It might be 
sending 
> > SIGSTOP to everything. It might be shifting the desk chairs around and 
> > creating a completely new kernel context, but they always have the same 
> > goal - stopping the existing activity, and they all come with their own 
> > issues (even if they're not obvious yet because the alternatives are 
> > currently vapourware to one extent or another).
> 
> I'll certainly admit the kexec idea is vaporware currently, but it does
> differ in a significant way from freezer-based approaches, such that I
> don't think it should be referred to as just another implementation of a
> freezer.  Specifically, it doesn't require that the "old kernel" be in a
> "consistent" state to a greater extent than suspend to ram; it is the
> case that all of the devices must be quiesced or shut down to some
> extent, but doing this without races and deadlocks (and without the
> freezer) is certainly very, very similar to what needs to be done for
> suspend to ram, which will need to be solved anyway.  Unlike the
> existing hibernate approaches, however, it will not be necessary to use
> any of the driver infrastructure once switched to the "save image"
> kernel, and thus it will not matter what locks are held, for instance.

Locks are not nearly the issue that you're making them out to be, and neither 
are most processes. It's only fuse that has caused this whole ding.

The lack of context from the original kernel is also going to be a problem. If 
you want to store the image on a local hard disk, the kernel being hibernated 
is going to need to do that for you, because it will need to use the 
information only it has regarding what swap is in use, how files on mounted 
filesystems map to devices and sectors and so on. It will need to somehow 
transfer that information from itself to the 'saving kernel'. In the end, the 
only advantage you'll get is that you don't have to worry about fuse 
processes anymore.

Frankly, I wish kexec was a viable alternative. I'm tired of working on kernel 
code, and would be perfectly happy to stop working on Suspend2 if kexec would 
work or if Rafael got swsusp to a point where the difference in features was 
minimal. But I just can't see that kexec is a viable alternative. Sorry.

Nigel
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[david]

On Wed, 11 Jul 2007, Nigel Cunningham wrote:

> Hi.
>
> On Wednesday 11 July 2007 21:11:34 Miklos Szeredi wrote:
>>> Anyway, to implement the kexec approach we must separate the
>>> hibernation from the suspend at the drivers level, which I'm still
>>> going to do, but I need to take part in endless discussions
>>
>> Discussions are good.  We understand the problem better.  Now I still
>> think we don't understand every aspect completely, so continuing the
>> discussion makes sense.
>>
>>> regarding the freezer, how it is bad and how we should drop it,
>>> because it breaks things (which NB is not true, because it doesn't).
>>
>> This thread started out from a bug, that seemed to be caused by the
>> freezer (we still don't exactly know what it was caused by), and the
>> discussion uncovered various problems _with_ the freezer, that up to
>> now no other _proper_ solutions have been propsed than to remove the
>> freezer.
>
> No other _proper_ solutions have been proposed. Everyone who suggests removing
> the freezer also suggests implementing it all over again. It might be sending
> SIGSTOP to everything. It might be shifting the desk chairs around and
> creating a completely new kernel context, but they always have the same
> goal - stopping the existing activity, and they all come with their own
> issues (even if they're not obvious yet because the alternatives are
> currently vapourware to one extent or another).

I think the big problem with the existing freezer is that you want to stop 
everything, except X, except Y, except Z.....

the advantage of the new approaches being proposed is that they don't 
require _anything_ from the origional system continue to run so you avoid 
all the exceptions.

freezing everything is easy, figuring out what you don't want to freeze is 
where everyone is seeing problems.

> IMHO, the real solution is to go back to the original issue and fix it
> properly. Make fuse filesystems play nicely with the existing freezer. I've
> just gone back and looked at the point where you started talking
> about "malicious filesystems". You talk about fuse imposing certain ordering
> in the userspace tasks being frozen. Please, say more. What ordering issues?
> Why? How can such ordering be determined programmatically?

I think most people just see this as a symptom of the problem, not the 
core problem itself.

David Lang

Re: Hibernation Redesign

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 3034 bytes --]

Hi.

On Thursday 12 July 2007 03:55:40 david@lang.hm wrote:
> On Wed, 11 Jul 2007, Nigel Cunningham wrote:
> 
> > Hi.
> >
> > On Wednesday 11 July 2007 21:11:34 Miklos Szeredi wrote:
> >>> Anyway, to implement the kexec approach we must separate the
> >>> hibernation from the suspend at the drivers level, which I'm still
> >>> going to do, but I need to take part in endless discussions
> >>
> >> Discussions are good.  We understand the problem better.  Now I still
> >> think we don't understand every aspect completely, so continuing the
> >> discussion makes sense.
> >>
> >>> regarding the freezer, how it is bad and how we should drop it,
> >>> because it breaks things (which NB is not true, because it doesn't).
> >>
> >> This thread started out from a bug, that seemed to be caused by the
> >> freezer (we still don't exactly know what it was caused by), and the
> >> discussion uncovered various problems _with_ the freezer, that up to
> >> now no other _proper_ solutions have been propsed than to remove the
> >> freezer.
> >
> > No other _proper_ solutions have been proposed. Everyone who suggests 
removing
> > the freezer also suggests implementing it all over again. It might be 
sending
> > SIGSTOP to everything. It might be shifting the desk chairs around and
> > creating a completely new kernel context, but they always have the same
> > goal - stopping the existing activity, and they all come with their own
> > issues (even if they're not obvious yet because the alternatives are
> > currently vapourware to one extent or another).
> 
> I think the big problem with the existing freezer is that you want to stop 
> everything, except X, except Y, except Z.....
> 
> the advantage of the new approaches being proposed is that they don't 
> require _anything_ from the origional system continue to run so you avoid 
> all the exceptions.
> 
> freezing everything is easy, figuring out what you don't want to freeze is 
> where everyone is seeing problems.
> 
> > IMHO, the real solution is to go back to the original issue and fix it
> > properly. Make fuse filesystems play nicely with the existing freezer. 
I've
> > just gone back and looked at the point where you started talking
> > about "malicious filesystems". You talk about fuse imposing certain 
ordering
> > in the userspace tasks being frozen. Please, say more. What ordering 
issues?
> > Why? How can such ordering be determined programmatically?
> 
> I think most people just see this as a symptom of the problem, not the 
> core problem itself.

Sure, but before we go out and buy a new car, let's figure out properly what 
the problems with the old one are. It might just be that the horrendous sound 
that's making us want a new car isn't as bad as we initially think. Even if 
we do decide we want a new version, we can at least learn something from the 
old one.

Regards,

Nigel
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Hibernation Redesign

[Al Boldi]

Jeremy Fitzhardinge wrote:
> Jeremy Maitin-Shepard wrote:
> > I don't know a whole lot about xen, but it seems that one issue with
> > this approach is that it requires you run your system under a hypervisor
> > at all times, which may introduce some overhead.
>
> No, I don't think that's what Al is proposing.  The kernel-internal
> interfaces we've put in place to make Xen work could be reused to do
> some of the things you're talking about.  In particular, a kernel
> running under Xen has to be able to deal with non-contiguous physical
> pages, and reusing the same pagetable hooks would allow a kexeced kernel
> to run happily out of any random assortment of pages you manage to
> allocate for it.

Exactly, there may well be overlap between Xen and the kexec hibernate 
approach, for which code structures should definitely be leveraged.

And, I wasn't suggesting to use Xen as an HV, which wouldn't really solve 
anything, but was trying to point out that there is no need to maintain two 
separate kernels, much like Xen, which inlines two modes into the kernel:  
host and guest.

So kexec really seems the way to go, which mimics the way APM used to do it, 
which is known to work flawlessly with minimal OS involvement.


Thanks!

--
Al

Re: Hibernation Redesign

[Jeremy Maitin-Shepard]

Al Boldi <a1426z@gawab.com> writes:

[snip]

> Exactly, there may well be overlap between Xen and the kexec hibernate 
> approach, for which code structures should definitely be leveraged.

> And, I wasn't suggesting to use Xen as an HV, which wouldn't really solve 
> anything, but was trying to point out that there is no need to maintain two 
> separate kernels, much like Xen, which inlines two modes into the kernel:  
> host and guest.

With relocatable kernels, or by simply using the "backup the first 16 or
64 MB of physical memory" approach, the same kernel image could be used
both as the normal kernel as the "save image" kernel.  The actual
behavior of the system would likely depend on kernel command-line
parameters or an initrd, rather than being hard-coded into the kernel
image.  If it is made a requirement that the same kernel be used, then
as is done currently, the text sections need not be touched at all.
There is a significant advantage, however, to using a different kernel:
unneeded drivers can be compiled out, leading to faster load times.

> So kexec really seems the way to go, which mimics the way APM used to do it, 
> which is known to work flawlessly with minimal OS involvement.

Now all that is needed is someone with enough time and interest to
implement it.  :)

-- 
Jeremy Maitin-Shepard