Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

Some general comments on the suspend blockers/wakelock/opportunistic
suspend v6 patch series, posted here:

    https://lists.linux-foundation.org/pipermail/linux-pm/2010-April/025146.html

The comments below are somewhat telegraphic in the interests of 
readability - more specific comments to follow in later E-mails. I am 
indebted to those of us who discussed these issues at LPC last year and 
ELC this year for several stimulating discussions.

There are several general problems with the design of opportunistic
suspend and suspend-blocks.

1. The opportunistic suspend code bypasses existing Linux kernel code,
   such as timers and the scheduler, that indicates when code
   needs to run, and when the system is idle.  This causes two problems:

   a. When opportunistic suspend is enabled, the default mode is to
      break all timers and scheduling on the system.  This isn't
      right: the default mode should be to preserve standard Linux
      behavior.  Exceptions can then be added for process groups that
      should run with the non-standard timer and scheduler behavior.

   b. The series introduces a de novo kernel API and userspace API
      that are unrelated to timers and the scheduler, but if the point
      is to modify the behavior of timers or the scheduler, the
      existing timer or scheduler APIs should be extended.  Any new
      APIs will need to be widely spread throughout the kernel and
      userspace.

2. The suspend-block kernel API tells the kernel _how_ to accomplish a
   goal, rather than telling the kernel _what_ the goal is.  This
   results in layering violations, unstated assumptions, and is too
   coarse-grained.  These problems in turn will cause fragile kernel
   code, kernel code with userspace dependencies, and power management
   problems on modern hardware.  Code should ask for what it wants.
   For example, if a driver needs to place an upper bound on its
   device wakeup latency, or if it needs to place an upper bound on
   interrupt response latency, that is what it should request.  Driver
   and subsystem code should not care how the kernel implements those
   requests, since the implementation can differ on different hardware
   and even on different use-cases with the same hardware.
   
3. Similarly, the suspend-block userspace API tells the kernel how to
   accomplish a goal, rather than telling the kernel what the goal is.
   Userspace processes should ask the kernel for what they really
   want.  If a process' timers should be disabled upon entering
   suspend, or the timer durations should have a lower bound, that's
   what the API should request.

Merging this series as currently designed and implemented will cause 
problems.  Suspend-blocks introduce a second, separate idle management 
approach in the Linux kernel.  The existing approach is the familiar timer 
and scheduler based approach.  The new approach is one where timers and 
runqueues no longer matter: the system is always at risk of entering 
suspend at any moment, with only suspend-blocks to stop it. Driver authors 
will effectively have to implement both approaches in their code.

Once merged, it will be nearly impossible to remove this code in favor 
of a cleaner approach.  Suspend-block calls are likely to spread 
throughout the kernel and drivers.  Patches 6, 7, and 8 are the leading 
edge of this - a quick grep through the Android common kernel at

    git://android.git.kernel.org/kernel/common.git 

shows wakelocks in the following drivers:

    drivers/input/evdev.c
    drivers/input/misc/gpio_input.c
    drivers/input/misc/gpio_matrix.c
    drivers/mmc/core/core.c
    drivers/rtc/alarm.c
    drivers/usb/gadget/f_mass_storage.c

Suspend-blocks will be difficult to convert to a finer-grained approach 
later.  The API design problems, mentioned above in points 2 and 3, will 
make it very difficult to determine what a driver author's or modifier's 
intention was when adding the suspend-block.  Also, patches 2 and 7 
introduce userspace APIs.  We will undoubtedly wish to avoid removing a 
userspace API once it is merged.  It will be quite difficult to implement 
such a general directive ("block system suspend") on a future kernel that 
may have a much finer-grained notion of low-power system modes, indeed 
that may have no useful notion of "system suspend."

...

The opportunistic suspend patches try to solve at least two real problems, 
that should be resolved in some way.  First, some types of userspace 
processes can unintentionally block system power management.  Second, the 
kernel is missing a system-wide form of CPUIdle.  This patch series, 
though, isn't the right way to solve either of these problems.  Let's 
figure out a different approach.

Figuring out a different way to do this should not limit Android at all, 
since Google can do what other Linux distributions do and continue to 
patch opportunistic suspend/suspend-block calls into their kernels as 
needed to ship devices, while contributing towards a different solution to 
the problem.

 
regards,

- Paul

(Linux-OMAP co-maintainer, focusing mostly on power management and 
software architecture issues)

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Wed, May 12, 2010 at 09:35:30PM -0600, Paul Walmsley wrote:
> 
> Figuring out a different way to do this should not limit Android at all, 
> since Google can do what other Linux distributions do and continue to 
> patch opportunistic suspend/suspend-block calls into their kernels as 
> needed to ship devices, while contributing towards a different solution to 
> the problem.

I basically agree, except that despite having a year to do so none of us 
have come up with a different way that would actually work. Google have 
done this work. Who's going to prove that there is actually a different 
way to do this?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Wed, 12 May 2010, Paul Walmsley wrote:

> Hello,
> 
> Some general comments on the suspend blockers/wakelock/opportunistic
> suspend v6 patch series, posted here:
> 
>     https://lists.linux-foundation.org/pipermail/linux-pm/2010-April/025146.html
> 
> The comments below are somewhat telegraphic in the interests of 
> readability - more specific comments to follow in later E-mails. I am 
> indebted to those of us who discussed these issues at LPC last year and 
> ELC this year for several stimulating discussions.
> 
> There are several general problems with the design of opportunistic
> suspend and suspend-blocks.
> 
> 1. The opportunistic suspend code bypasses existing Linux kernel code,
>    such as timers and the scheduler, that indicates when code
>    needs to run, and when the system is idle.

Whoa!  That's not my understanding at all.

As I see it, opportunistic suspend doesn't bypass any code that isn't 
already bypassed by the existing suspend code.  Users can do

	echo mem >/sys/power/state

whenever they want, without regard to kernel timers and the scheduler 
(other than the fact that the user's thread must be running in order to 
carry out the write, of course).

>  This causes two problems:
> 
>    a. When opportunistic suspend is enabled, the default mode is to
>       break all timers and scheduling on the system.  This isn't
>       right: the default mode should be to preserve standard Linux
>       behavior.  Exceptions can then be added for process groups that
>       should run with the non-standard timer and scheduler behavior.

I don't understand this at all.  What gets broken, and how?  In 
particular, what gets broken that isn't also broken by "echo mem 
>/sys/power/state"?

>    b. The series introduces a de novo kernel API and userspace API
>       that are unrelated to timers and the scheduler, but if the point
>       is to modify the behavior of timers or the scheduler, the
>       existing timer or scheduler APIs should be extended.  Any new
>       APIs will need to be widely spread throughout the kernel and
>       userspace.

But the point _isn't_ to modify the behavior of timers and the
scheduler.  The point is to provide a way for the system to enter a
very low-power state as soon as possible while safely handling races.

> 2. The suspend-block kernel API tells the kernel _how_ to accomplish a
>    goal, rather than telling the kernel _what_ the goal is.  This
>    results in layering violations, unstated assumptions, and is too
>    coarse-grained.  These problems in turn will cause fragile kernel
>    code, kernel code with userspace dependencies, and power management
>    problems on modern hardware.  Code should ask for what it wants.
>    For example, if a driver needs to place an upper bound on its
>    device wakeup latency, or if it needs to place an upper bound on
>    interrupt response latency, that is what it should request.  Driver
>    and subsystem code should not care how the kernel implements those
>    requests, since the implementation can differ on different hardware
>    and even on different use-cases with the same hardware.

Although the first sentence is true, I don't find it useful.  The goal 
of suspend blockers is to prevent the system from entering a low-power 
state until some important task is finished.  It has little to do with 
interrupt response latency or device wakeup latency.

As far as I can tell, suspend blockers are more or less a direct
implementation of the desired goal.

> 3. Similarly, the suspend-block userspace API tells the kernel how to
>    accomplish a goal, rather than telling the kernel what the goal is.
>    Userspace processes should ask the kernel for what they really
>    want.  If a process' timers should be disabled upon entering
>    suspend, or the timer durations should have a lower bound, that's
>    what the API should request.

The userspace API has essentially the same goal as the kernel API.

> Merging this series as currently designed and implemented will cause 
> problems.  Suspend-blocks introduce a second, separate idle management 
> approach in the Linux kernel.  The existing approach is the familiar timer 
> and scheduler based approach.  The new approach is one where timers and 
> runqueues no longer matter: the system is always at risk of entering 
> suspend at any moment, with only suspend-blocks to stop it. Driver authors 
> will effectively have to implement both approaches in their code.

That's true.  Where's the problem?  The system is _already_ at risk of
entering suspend at any moment, as I described above.  If the "timer
and scheduler based" approach can be adapted to do what the Android 
people want, then all the better -- but I rather suspect it can't.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 13:17 +0100, Matthew Garrett wrote:
> On Wed, May 12, 2010 at 09:35:30PM -0600, Paul Walmsley wrote:
> > 
> > Figuring out a different way to do this should not limit Android at all, 
> > since Google can do what other Linux distributions do and continue to 
> > patch opportunistic suspend/suspend-block calls into their kernels as 
> > needed to ship devices, while contributing towards a different solution to 
> > the problem.
> 
> I basically agree, except that despite having a year to do so none of us 
> have come up with a different way that would actually work. Google have 
> done this work. Who's going to prove that there is actually a different 
> way to do this?

We all feel the pain of inelegance right? I think it's clear that this
system will not last (but there's no other immediate option) .. That
doesn't mean we should reject it, but we need to be clear that this
system will get replaced. So we should format the patches appropriately.
To me the userspace aspect is a permanent change .. If we could drop
that (or put it into debugfs) then it would make this a lot easy to
accept as a stepping stone.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Thu, May 13, 2010 at 10:33 AM, Daniel Walker <dwalker@fifo99.com> wrote:
> On Thu, 2010-05-13 at 13:17 +0100, Matthew Garrett wrote:
>> On Wed, May 12, 2010 at 09:35:30PM -0600, Paul Walmsley wrote:
>> >
>> > Figuring out a different way to do this should not limit Android at all,
>> > since Google can do what other Linux distributions do and continue to
>> > patch opportunistic suspend/suspend-block calls into their kernels as
>> > needed to ship devices, while contributing towards a different solution to
>> > the problem.
>>
>> I basically agree, except that despite having a year to do so none of us
>> have come up with a different way that would actually work. Google have
>> done this work. Who's going to prove that there is actually a different
>> way to do this?
>
> We all feel the pain of inelegance right? I think it's clear that this
> system will not last (but there's no other immediate option) .. That
> doesn't mean we should reject it, but we need to be clear that this
> system will get replaced. So we should format the patches appropriately.
> To me the userspace aspect is a permanent change .. If we could drop
> that (or put it into debugfs) then it would make this a lot easy to
> accept as a stepping stone.

I'm in agreement on the first half of this -- from the Google/Android
point of view, if we can someday accomplish everything we need with
some different facility, we'll happily shift over to that.  That seems
like a normal operating mode for mainline -- new solutions arise,
drivers are migrated to those new solutions, old solutions fall to the
wayside.  We fully expect that the world will change over time and one
of our largest goals with trying to get work upstream is to reduce the
pain of those changes for everyone, while trying to get to "you can
build a kernel out of the box from mainline that Just Works with an
android userspace".

I'm not sure this necessitates using only debugfs for the userspace
interface.  A userspace interface is necessary to accomplish what
we're trying to do here, otherwise we have only half a solution, and
our hope is that it'd be a stable interface (as userspace interfaces
are supposed to be) for as long as its needed.  I could totally
imagine the userspace interface eventually becoming a no-op or
punching through to some other facility, depending on how this problem
is solved long-term in the ideal post-suspend-block future.

Brian

Re: [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 11:17 -0700, Brian Swetland wrote:

> 
> I'm not sure this necessitates using only debugfs for the userspace
> interface.  A userspace interface is necessary to accomplish what
> we're trying to do here, otherwise we have only half a solution, and
> our hope is that it'd be a stable interface (as userspace interfaces
> are supposed to be) for as long as its needed.  I could totally
> imagine the userspace interface eventually becoming a no-op or
> punching through to some other facility, depending on how this problem
> is solved long-term in the ideal post-suspend-block future.

The problem is that once this userspace interface is exposed, it's
nearly permanent and has to be support for a long long time .. It might
seen trivial to just remove something your not using, but we never know
who is using what once the kernel is released.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 11:25:57AM -0700, Daniel Walker wrote:

> The problem is that once this userspace interface is exposed, it's
> nearly permanent and has to be support for a long long time .. It might
> seen trivial to just remove something your not using, but we never know
> who is using what once the kernel is released.

Deprecating sysfs interfaces can be done within 6 months or so, 
especially if there's only one real consumer.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 19:36 +0100, Matthew Garrett wrote:
> On Thu, May 13, 2010 at 11:25:57AM -0700, Daniel Walker wrote:
> 
> > The problem is that once this userspace interface is exposed, it's
> > nearly permanent and has to be support for a long long time .. It might
> > seen trivial to just remove something your not using, but we never know
> > who is using what once the kernel is released.
> 
> Deprecating sysfs interfaces can be done within 6 months or so, 
> especially if there's only one real consumer.

I'll assume your right (can you give an example of this?), but why
should we even add it if we know it's already going to get replaced.
It's like it's pre-deprecated ..

Daniel

Re: [PATCH 1/8] PM: Add suspend block api.

[Paul Walmsley]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 6285 bytes --]

Hi,

Some comments on the opportunistic suspend portion of this patch.

On Fri, 30 Apr 2010, Arve Hjønnevåg wrote:

> Adds /sys/power/policy that selects the behaviour of /sys/power/state.
> After setting the policy to opportunistic, writes to /sys/power/state
> become non-blocking requests that specify which suspend state to enter
> when no suspend blockers are active. A special state, "on", stops the
> process by activating the "main" suspend blocker.
> 
> Signed-off-by: Arve Hjønnevåg <arve@android.com>
> ---
>  Documentation/power/opportunistic-suspend.txt |  119 +++++++++++
>  include/linux/suspend_blocker.h               |   64 ++++++
>  kernel/power/Kconfig                          |   16 ++
>  kernel/power/Makefile                         |    1 +
>  kernel/power/main.c                           |   92 ++++++++-
>  kernel/power/power.h                          |    9 +
>  kernel/power/suspend.c                        |    4 +-
>  kernel/power/suspend_blocker.c                |  261 +++++++++++++++++++++++++
>  8 files changed, 559 insertions(+), 7 deletions(-)
>  create mode 100644 Documentation/power/opportunistic-suspend.txt
>  create mode 100755 include/linux/suspend_blocker.h
>  create mode 100644 kernel/power/suspend_blocker.c


While reading the patch, it seemed that this opportunistic suspend
mechanism is a degenerate case of a system-wide form of CPUIdle.  I'll
call it "SystemIdle," for lack of a better term.  The main difference
between this and the CPUIdle design is that this patch's SystemIdle is
not integrated with any of the existing Linux mechanisms for limiting
idle duration or depth.  This is a major problem with the current
patch and should prevent it from being merged in its current state.
But if some SystemIdle-type code were to be written that did work with
the rest of the Linux kernel, it would be very useful.

To take an example from the current Linux-OMAP kernels: right now, the
Linux-OMAP code handles system-level idle through CPUIdle.[1] To
borrow ACPI terminology, some of the S-states are implemented as
C-states.  That isn't right, for several reasons:

- There are other constraints on system-level idle modes beyond those
  that apply to the CPU.  To take an OMAP example, on boards that
  support it, OMAP2+ chips can cut power to the external SoC
  high-frequency clock oscillator ("HF oscillator").[2] This is the
  clock that is later used to drive the CPU clock, bus clock, etc., on
  the SoC, but can also be used to drive other chips on the board,
  such as external audio codec chips, GPS receivers, etc.)  If power
  is cut to the HF oscillator, it can take several milliseconds to
  stabilize once power is reapplied.  Part of the decision as to
  whether to cut power to the HF oscillator is a classic idle
  balancing test between power economy and wakeup latency.  But this
  occurs on a system level - the CPU may not even be involved.

  Consider a low-power audio playback use-case.  The CPU may be only
  rarely involved, but other devices on the SoC may need to be active.
  For example, if a large number of audio samples are loaded into main
  memory, the CPU can go into a very deep sleep state while the DMA
  controller transfers the samples to the audio serial interface
  device.  But the DMA controller and audio serial interface need to
  run occasionally, so depending on FIFO depths in the system, the HF
  oscillator clock may need to be kept on, even if the CPU idle level
  would suggest that it could be disabled.

  Additionally, other external chips outside the SoC, may be clocked
  by that HF oscillator.  Continuing the low-power audio playback
  use-case, the external audio codec chip may also be clocked from the
  HF oscillator.  If the HF oscillator is cut when the SoC is idle,
  but the audio codec has samples in its buffer, audio playback will
  be disrupted.

- There should be only one system-level governor.  Since all of the
  production OMAPs so far have been single-CPU systems, we've been
  able to get away with this.  But on multi-CPU systems, there is one
  CPUIdle governor per CPU.  So reusing the CPUIdle governor to do this
  won't work the same way as it has for us in the past.

So some good SystemIdle-type code would be really useful for us.  But a 
significant part of what "good" means is that such a SystemIdle needs to 
be driven from the bottom up, rather than from the top down. In other 
words, both the 'policy' (i.e., the choice of metrics to use to determine 
the system idle state), and the 'implementation of that policy' (i.e., the 
way to enter those idle states), should be left up to the underlying chip 
architecture code that would implement SystemIdle client drivers.  (In 
this regard, it would be even more different than CPUIdle, which hardcodes 
the policy.)

This way, at least initially, there will be minimal top-down
restrictions on what architectures need to do to implement to their
chip's power management features.  As common features are identified,
by individual architectures implementing working code, then the
architecture developers can collaborate, and common components can be
proposed for merging into the top-level SystemIdle code.

Such a SystemIdle implementation should also work for Android's
opportunistic suspend code.  Google developers could choose the system
metrics that they wish to use to control system idle levels.  If they
want to ignore timers and the scheduler, that's fine - that design
decision can be confined to their policy driver, and the suspend-block
code can be activated the moment that some driver sets a PM
constraint, rather than doing anything more refined.  Similarly,
Google can change the way that the policy decisions are implemented.
Google could use a driver that does not take any advantage of
fine-grained power management aside from CPUIdle.  This driver could
simply enter full system suspend whenever the policy driver authorizes
it to do.

This approach - or some similar approach - should allow Android to do what 
it needs, while still allowing other, more finely-grained power management 
approaches to do something different.


regards,

- Paul

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 11:59:37AM -0700, Daniel Walker wrote:
> On Thu, 2010-05-13 at 19:36 +0100, Matthew Garrett wrote:
> > Deprecating sysfs interfaces can be done within 6 months or so, 
> > especially if there's only one real consumer.
> 
> I'll assume your right (can you give an example of this?), but why
> should we even add it if we know it's already going to get replaced.
> It's like it's pre-deprecated ..

See feature-removal-schedule.txt. So far we have no indication that it's 
going to be replaced, because nobody has actually suggested a working 
way to do this better. If we had a concrete implementation proposal for 
that then we'd be in a pretty different position right now.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Alan Stern <stern@rowland.harvard.edu> [100513 07:11]:
> On Wed, 12 May 2010, Paul Walmsley wrote:
> 
> > Hello,
> > 
> > Some general comments on the suspend blockers/wakelock/opportunistic
> > suspend v6 patch series, posted here:
> > 
> >     https://lists.linux-foundation.org/pipermail/linux-pm/2010-April/025146.html
> > 
> > The comments below are somewhat telegraphic in the interests of 
> > readability - more specific comments to follow in later E-mails. I am 
> > indebted to those of us who discussed these issues at LPC last year and 
> > ELC this year for several stimulating discussions.
> > 
> > There are several general problems with the design of opportunistic
> > suspend and suspend-blocks.
> > 
> > 1. The opportunistic suspend code bypasses existing Linux kernel code,
> >    such as timers and the scheduler, that indicates when code
> >    needs to run, and when the system is idle.
> 
> Whoa!  That's not my understanding at all.
> 
> As I see it, opportunistic suspend doesn't bypass any code that isn't 
> already bypassed by the existing suspend code.  Users can do
> 
> 	echo mem >/sys/power/state
> 
> whenever they want, without regard to kernel timers and the scheduler 
> (other than the fact that the user's thread must be running in order to 
> carry out the write, of course).

The difference between echo mem > /sys/power/state and suspend blocks
is that with suspend blocks the system keeps running. And that's why 
it should be handled by runtime power management instead.

The suspend blocks seems like a hack to spam filter good and bad
apps from timer usage point of view. Applications are categorized
as good or bad depending if they grab a susped blocker or not.

I believe categorizing the apps should be instead done with some
timer flags or cgroups instead.

Cheers,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 12:17:17PM -0700, Tony Lindgren wrote:
> The suspend blocks seems like a hack to spam filter good and bad
> apps from timer usage point of view. Applications are categorized
> as good or bad depending if they grab a susped blocker or not.
> 
> I believe categorizing the apps should be instead done with some
> timer flags or cgroups instead.

I agree, but we have no mechanism for implementing that in a race-free 
way. We don't even have a realistical proposal for what that mechanism 
would look like. Should we refuse bread today for the promise of cake 
tomorrow?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 20:11 +0100, Matthew Garrett wrote:
> On Thu, May 13, 2010 at 11:59:37AM -0700, Daniel Walker wrote:
> > On Thu, 2010-05-13 at 19:36 +0100, Matthew Garrett wrote:
> > > Deprecating sysfs interfaces can be done within 6 months or so, 
> > > especially if there's only one real consumer.
> > 
> > I'll assume your right (can you give an example of this?), but why
> > should we even add it if we know it's already going to get replaced.
> > It's like it's pre-deprecated ..
> 
> See feature-removal-schedule.txt. So far we have no indication that it's 
> going to be replaced, because nobody has actually suggested a working 
> way to do this better. If we had a concrete implementation proposal for 
> that then we'd be in a pretty different position right now.

Ok, feature-removal-schedule.txt applies to everything tho. What your
saying is that if this interface only last a short time it might take 6
months, if it last for a long time it would take longer. There's no easy
way to know that Google is the only user after some amount of time
passes.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Matthew Garrett <mjg@redhat.com> [100513 12:20]:
> On Thu, May 13, 2010 at 12:17:17PM -0700, Tony Lindgren wrote:
> > The suspend blocks seems like a hack to spam filter good and bad
> > apps from timer usage point of view. Applications are categorized
> > as good or bad depending if they grab a susped blocker or not.
> > 
> > I believe categorizing the apps should be instead done with some
> > timer flags or cgroups instead.
> 
> I agree, but we have no mechanism for implementing that in a race-free 
> way. We don't even have a realistical proposal for what that mechanism 
> would look like. Should we refuse bread today for the promise of cake 
> tomorrow?

Well this is an interesting problem, and once solved will be handy
for all kind of things. My worry is that if it's integrated in it's
current form it will be totally out of control all over the place :(

Still hoping we can come up with some clean way that avoid the patching
all over the place part.. How about the following, can you please check
if it would help with your example of guaranteed handling of event:

1. In the kernel, we add one more timer queue for critical timers.
   The current timer queue(s) stay as it is.

2. We allow selecting the timer based on some flag, the default
   behaviour being the current default timer queue.

3. Then we add next_timer_interupt_critical() to only query the
   critical timers along the lines of the current next_timer_interrupt().

4. We implement a custom pm_idle that suspends the system based on
   some logic and checking if next_timer_interrupt_critical() is
   empty. If the next_timer_interrupt_critical() does not return
   anything, we assume it's OK to suspend the system.

Now to me it sounds if your the input layer and userspace handle
both grab the timers with the critical flags, it should be guaranteed
that the events get handled before the system is suspended.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 12:36:34PM -0700, Daniel Walker wrote:
> On Thu, 2010-05-13 at 20:11 +0100, Matthew Garrett wrote:
> > See feature-removal-schedule.txt. So far we have no indication that it's 
> > going to be replaced, because nobody has actually suggested a working 
> > way to do this better. If we had a concrete implementation proposal for 
> > that then we'd be in a pretty different position right now.
> 
> Ok, feature-removal-schedule.txt applies to everything tho. What your
> saying is that if this interface only last a short time it might take 6
> months, if it last for a long time it would take longer. There's no easy
> way to know that Google is the only user after some amount of time
> passes.

If the interface is there for a long time, it's because we haven't come 
up with anything better. And if we haven't come up with anything better, 
the interface deserves to be there.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 12:42:05PM -0700, Tony Lindgren wrote:

> 1. In the kernel, we add one more timer queue for critical timers.
>    The current timer queue(s) stay as it is.
> 
> 2. We allow selecting the timer based on some flag, the default
>    behaviour being the current default timer queue.
> 
> 3. Then we add next_timer_interupt_critical() to only query the
>    critical timers along the lines of the current next_timer_interrupt().
> 
> 4. We implement a custom pm_idle that suspends the system based on
>    some logic and checking if next_timer_interrupt_critical() is
>    empty. If the next_timer_interrupt_critical() does not return
>    anything, we assume it's OK to suspend the system.

Ok. So we stick the untrusted bits of userspace on the critical timer 
list. Now we get a network packet that generates a wakeup event and gets 
read by an application. What happens if that application can't fully 
process the packet in a single timeslice?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Matthew Garrett <mjg@redhat.com> [100513 12:49]:
> On Thu, May 13, 2010 at 12:42:05PM -0700, Tony Lindgren wrote:
> 
> > 1. In the kernel, we add one more timer queue for critical timers.
> >    The current timer queue(s) stay as it is.
> > 
> > 2. We allow selecting the timer based on some flag, the default
> >    behaviour being the current default timer queue.
> > 
> > 3. Then we add next_timer_interupt_critical() to only query the
> >    critical timers along the lines of the current next_timer_interrupt().
> > 
> > 4. We implement a custom pm_idle that suspends the system based on
> >    some logic and checking if next_timer_interrupt_critical() is
> >    empty. If the next_timer_interrupt_critical() does not return
> >    anything, we assume it's OK to suspend the system.
> 
> Ok. So we stick the untrusted bits of userspace on the critical timer 
> list.

I guess you mean the trusted instead of untrusted apps in the userspace
above, the ones that are critical to keep running.

> Now we get a network packet that generates a wakeup event and gets 
> read by an application. What happens if that application can't fully 
> process the packet in a single timeslice?

The system stays running because there's something to do. The system
won't suspend until all the processors hit the kernel idle loop and
the next_timer_interrupt_critical() returns nothing.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:

> The system stays running because there's something to do. The system
> won't suspend until all the processors hit the kernel idle loop and
> the next_timer_interrupt_critical() returns nothing.

At which point an application in a busy loop cripples you. I think we 
could implement your suggestion more easily by just giving untrusted 
applications an effectively infinite amount of timer slack, but it still 
doesn't handle the case where an app behaves excrutiatingly badly.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> 
> > The system stays running because there's something to do. The system
> > won't suspend until all the processors hit the kernel idle loop and
> > the next_timer_interrupt_critical() returns nothing.
> 
> At which point an application in a busy loop cripples you.

Maybe you could deal with the misbehaving untrusted apps in the userspace
by sending kill -STOP to them when the screen blanks? Then continue
when some event wakes up the system again.

> I think we could implement your suggestion more easily by just giving
> untrusted applications an effectively infinite amount of timer slack,
> but it still doesn't handle the case where an app behaves excrutiatingly
> badly.

Hmm, if you use timer slack then you still need to search through
the whole timer list instead of a smaller critical timer list.
Both ways sound doable though.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 01:23:20PM -0700, Tony Lindgren wrote:
> * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > 
> > > The system stays running because there's something to do. The system
> > > won't suspend until all the processors hit the kernel idle loop and
> > > the next_timer_interrupt_critical() returns nothing.
> > 
> > At which point an application in a busy loop cripples you.
> 
> Maybe you could deal with the misbehaving untrusted apps in the userspace
> by sending kill -STOP to them when the screen blanks? Then continue
> when some event wakes up the system again.

And if that's the application that's listening to the network socket 
that you want to get a wakeup event from? This problem is hard. I'd love 
there to be an elegant solution based on using the scheduler, but I 
really don't know what it is.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 13:23 -0700, Tony Lindgren wrote:
> * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > 
> > > The system stays running because there's something to do. The system
> > > won't suspend until all the processors hit the kernel idle loop and
> > > the next_timer_interrupt_critical() returns nothing.
> > 
> > At which point an application in a busy loop cripples you.
> 
> Maybe you could deal with the misbehaving untrusted apps in the userspace
> by sending kill -STOP to them when the screen blanks? Then continue
> when some event wakes up the system again.

Couldn't you just use priorities (nice), or cgroups to deal with that?
I'm sure there is a way to limit an apps runtime, so the system does go
idle sometimes.

> > I think we could implement your suggestion more easily by just giving
> > untrusted applications an effectively infinite amount of timer slack,
> > but it still doesn't handle the case where an app behaves excrutiatingly
> > badly.
> 
> Hmm, if you use timer slack then you still need to search through
> the whole timer list instead of a smaller critical timer list.
> Both ways sound doable though.

There are deferrable timers already in Linux.. It seems like it would
just be an extension of that.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Matthew Garrett <mjg@redhat.com> [100513 13:29]:
> On Thu, May 13, 2010 at 01:23:20PM -0700, Tony Lindgren wrote:
> > * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > > 
> > > > The system stays running because there's something to do. The system
> > > > won't suspend until all the processors hit the kernel idle loop and
> > > > the next_timer_interrupt_critical() returns nothing.
> > > 
> > > At which point an application in a busy loop cripples you.
> > 
> > Maybe you could deal with the misbehaving untrusted apps in the userspace
> > by sending kill -STOP to them when the screen blanks? Then continue
> > when some event wakes up the system again.
> 
> And if that's the application that's listening to the network socket 
> that you want to get a wakeup event from? This problem is hard. I'd love 
> there to be an elegant solution based on using the scheduler, but I 
> really don't know what it is.

Your system should wake up to an interrupt in that case. Then you have
the trusted apps running that can decide if the untrusted apps should
be continued or not.

The key would be to have the basic apps behave and use the critical
timers as needed. The advantage is that then you can do all the
policy in the userspace and in a custom pm_idle function.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Matthew Garrett wrote:
> On Thu, May 13, 2010 at 12:36:34PM -0700, Daniel Walker wrote:
> > On Thu, 2010-05-13 at 20:11 +0100, Matthew Garrett wrote:
> > > See feature-removal-schedule.txt. So far we have no indication that it's 
> > > going to be replaced, because nobody has actually suggested a working 
> > > way to do this better. If we had a concrete implementation proposal for 
> > > that then we'd be in a pretty different position right now.
> > 
> > Ok, feature-removal-schedule.txt applies to everything tho. What your
> > saying is that if this interface only last a short time it might take 6
> > months, if it last for a long time it would take longer. There's no easy
> > way to know that Google is the only user after some amount of time
> > passes.
> 
> If the interface is there for a long time, it's because we haven't come 
> up with anything better. And if we haven't come up with anything better, 
> the interface deserves to be there.

Moreover, the interface is already in use out-of-tree and that usage is
actually _growing_, so we have a growing number of out-of-tree drivers that
aren't megrgeable for this very reason.

I don't see any _realistic_ way of solving this problem other than merging
the opportunistic suspend.  If anyone sees one, and I mean _realistic_ and
_practically_ _feasible_, please tell me.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Alan Stern <stern@rowland.harvard.edu> [100513 07:11]:
> > On Wed, 12 May 2010, Paul Walmsley wrote:
> > 
> > > Hello,
> > > 
> > > Some general comments on the suspend blockers/wakelock/opportunistic
> > > suspend v6 patch series, posted here:
> > > 
> > >     https://lists.linux-foundation.org/pipermail/linux-pm/2010-April/025146.html
> > > 
> > > The comments below are somewhat telegraphic in the interests of 
> > > readability - more specific comments to follow in later E-mails. I am 
> > > indebted to those of us who discussed these issues at LPC last year and 
> > > ELC this year for several stimulating discussions.
> > > 
> > > There are several general problems with the design of opportunistic
> > > suspend and suspend-blocks.
> > > 
> > > 1. The opportunistic suspend code bypasses existing Linux kernel code,
> > >    such as timers and the scheduler, that indicates when code
> > >    needs to run, and when the system is idle.
> > 
> > Whoa!  That's not my understanding at all.
> > 
> > As I see it, opportunistic suspend doesn't bypass any code that isn't 
> > already bypassed by the existing suspend code.  Users can do
> > 
> > 	echo mem >/sys/power/state
> > 
> > whenever they want, without regard to kernel timers and the scheduler 
> > (other than the fact that the user's thread must be running in order to 
> > carry out the write, of course).
> 
> The difference between echo mem > /sys/power/state and suspend blocks
> is that with suspend blocks the system keeps running.

Care to elaborate?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 23:11 +0200, Rafael J. Wysocki wrote:
> On Thursday 13 May 2010, Matthew Garrett wrote:
> > On Thu, May 13, 2010 at 12:36:34PM -0700, Daniel Walker wrote:
> > > On Thu, 2010-05-13 at 20:11 +0100, Matthew Garrett wrote:
> > > > See feature-removal-schedule.txt. So far we have no indication that it's 
> > > > going to be replaced, because nobody has actually suggested a working 
> > > > way to do this better. If we had a concrete implementation proposal for 
> > > > that then we'd be in a pretty different position right now.
> > > 
> > > Ok, feature-removal-schedule.txt applies to everything tho. What your
> > > saying is that if this interface only last a short time it might take 6
> > > months, if it last for a long time it would take longer. There's no easy
> > > way to know that Google is the only user after some amount of time
> > > passes.
> > 
> > If the interface is there for a long time, it's because we haven't come 
> > up with anything better. And if we haven't come up with anything better, 
> > the interface deserves to be there.
> 
> Moreover, the interface is already in use out-of-tree and that usage is
> actually _growing_, so we have a growing number of out-of-tree drivers that
> aren't megrgeable for this very reason.

Why can't we merge the drivers? Drivers are just drivers, they don't
need this to get merged. (They need it to work with Android)

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 02:10:06PM -0700, Tony Lindgren wrote:
> * Matthew Garrett <mjg@redhat.com> [100513 13:29]:
> > And if that's the application that's listening to the network socket 
> > that you want to get a wakeup event from? This problem is hard. I'd love 
> > there to be an elegant solution based on using the scheduler, but I 
> > really don't know what it is.
> 
> Your system should wake up to an interrupt in that case. Then you have
> the trusted apps running that can decide if the untrusted apps should
> be continued or not.

What race-free mechanism do you use to ensure that? It's very easy to 
handwave these problems away. It's very difficult to actually write an 
implementation that works.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Matthew Garrett wrote:
> On Thu, May 13, 2010 at 01:23:20PM -0700, Tony Lindgren wrote:
> > * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > > 
> > > > The system stays running because there's something to do. The system
> > > > won't suspend until all the processors hit the kernel idle loop and
> > > > the next_timer_interrupt_critical() returns nothing.
> > > 
> > > At which point an application in a busy loop cripples you.
> > 
> > Maybe you could deal with the misbehaving untrusted apps in the userspace
> > by sending kill -STOP to them when the screen blanks? Then continue
> > when some event wakes up the system again.
> 
> And if that's the application that's listening to the network socket 
> that you want to get a wakeup event from? This problem is hard. I'd love 
> there to be an elegant solution based on using the scheduler, but I 
> really don't know what it is.

I agree and I don't understand the problem that people have with the
opportunistic suspend feature.

It solves a practical issue that _at_ _the_ _moment_ cannot be solved
differently, while there's a growing number of out-of-tree drivers depending
on this framework.  We need those drivers in and because we don't have any
viable alternative at hand, we have no good reason to reject it.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
> On Thursday 13 May 2010, Matthew Garrett wrote:
> > On Thu, May 13, 2010 at 01:23:20PM -0700, Tony Lindgren wrote:
> > > * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > > > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > > > 
> > > > > The system stays running because there's something to do. The system
> > > > > won't suspend until all the processors hit the kernel idle loop and
> > > > > the next_timer_interrupt_critical() returns nothing.
> > > > 
> > > > At which point an application in a busy loop cripples you.
> > > 
> > > Maybe you could deal with the misbehaving untrusted apps in the userspace
> > > by sending kill -STOP to them when the screen blanks? Then continue
> > > when some event wakes up the system again.
> > 
> > And if that's the application that's listening to the network socket 
> > that you want to get a wakeup event from? This problem is hard. I'd love 
> > there to be an elegant solution based on using the scheduler, but I 
> > really don't know what it is.
> 
> I agree and I don't understand the problem that people have with the
> opportunistic suspend feature.

It seems to be picking quite a few comments for one.
 
> It solves a practical issue that _at_ _the_ _moment_ cannot be solved
> differently, while there's a growing number of out-of-tree drivers depending
> on this framework.  We need those drivers in and because we don't have any
> viable alternative at hand, we have no good reason to reject it.

Nothing is preventing merging the drivers can be merged without
these calls.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Daniel Walker wrote:
> On Thu, 2010-05-13 at 23:11 +0200, Rafael J. Wysocki wrote:
> > On Thursday 13 May 2010, Matthew Garrett wrote:
> > > On Thu, May 13, 2010 at 12:36:34PM -0700, Daniel Walker wrote:
> > > > On Thu, 2010-05-13 at 20:11 +0100, Matthew Garrett wrote:
> > > > > See feature-removal-schedule.txt. So far we have no indication that it's 
> > > > > going to be replaced, because nobody has actually suggested a working 
> > > > > way to do this better. If we had a concrete implementation proposal for 
> > > > > that then we'd be in a pretty different position right now.
> > > > 
> > > > Ok, feature-removal-schedule.txt applies to everything tho. What your
> > > > saying is that if this interface only last a short time it might take 6
> > > > months, if it last for a long time it would take longer. There's no easy
> > > > way to know that Google is the only user after some amount of time
> > > > passes.
> > > 
> > > If the interface is there for a long time, it's because we haven't come 
> > > up with anything better. And if we haven't come up with anything better, 
> > > the interface deserves to be there.
> > 
> > Moreover, the interface is already in use out-of-tree and that usage is
> > actually _growing_, so we have a growing number of out-of-tree drivers that
> > aren't megrgeable for this very reason.
> 
> Why can't we merge the drivers? Drivers are just drivers, they don't
> need this to get merged. (They need it to work with Android)

Because someone would have to remove suspend blockers (or rather wakelocks)
from the drivers, test that they work correctly without suspend blockers and
submit the modified versions.  Going forward, every party responsible for such
a driver would have to maintain an out-of-tree version with suspend blockers
(or wakelocks) anyway, so the incentive to do that is zero.

Practically, as long as the opportunistic suspend is out of tree, there will be
a _growing_ number of out-of-tree drivers out there, which is not acceptable
in the long run.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Rafael J. Wysocki <rjw@sisk.pl> [100513 14:08]:
> On Thursday 13 May 2010, Tony Lindgren wrote:
> > 
> > The difference between echo mem > /sys/power/state and suspend blocks
> > is that with suspend blocks the system keeps running.
> 
> Care to elaborate?

The suspend blocks affects the runtime behaviour of the system by
categorizing apps into good and bad apps in a weird way.

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 23:27 +0200, Rafael J. Wysocki wrote:

> Because someone would have to remove suspend blockers (or rather wakelocks)
> from the drivers, test that they work correctly without suspend blockers and
> submit the modified versions.  Going forward, every party responsible for such
> a driver would have to maintain an out-of-tree version with suspend blockers
> (or wakelocks) anyway, so the incentive to do that is zero.

They should work without wakelock since wakelock are optional .. I mean
there's nothing in suspend blockers I've seen that indicates it's
required for some drivers to work. So it's just a matter of patching out
the wakelocks, with no need to re-test anything.

You get the driver mainlined, then maintain a small patch to add
wakelocks. Not hard at all , with lots of incentive to do so since you
don't have to maintain such a large block of code out of tree.

> Practically, as long as the opportunistic suspend is out of tree, there will be
> a _growing_ number of out-of-tree drivers out there, which is not acceptable
> in the long run.

I don't see why your saying that. These driver should work with out all
of this, which means they can get mainlined right now.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Matthew Garrett <mjg@redhat.com> [100513 14:16]:
> On Thu, May 13, 2010 at 02:10:06PM -0700, Tony Lindgren wrote:
> > * Matthew Garrett <mjg@redhat.com> [100513 13:29]:
> > > And if that's the application that's listening to the network socket 
> > > that you want to get a wakeup event from? This problem is hard. I'd love 
> > > there to be an elegant solution based on using the scheduler, but I 
> > > really don't know what it is.
> > 
> > Your system should wake up to an interrupt in that case. Then you have
> > the trusted apps running that can decide if the untrusted apps should
> > be continued or not.
> 
> What race-free mechanism do you use to ensure that? It's very easy to 
> handwave these problems away. It's very difficult to actually write an 
> implementation that works.

Can you describe where do you see the race now?

Your trusted apps should be have in quite a normal way except the system
suspends if no critical timers are on the list. For the untrusted apps
you assume you can stop them at whatever userspace policy you set.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Daniel Walker <dwalker@fifo99.com> [100513 14:28]:
> On Thu, 2010-05-13 at 23:27 +0200, Rafael J. Wysocki wrote:
> 
> > Because someone would have to remove suspend blockers (or rather wakelocks)
> > from the drivers, test that they work correctly without suspend blockers and
> > submit the modified versions.  Going forward, every party responsible for such
> > a driver would have to maintain an out-of-tree version with suspend blockers
> > (or wakelocks) anyway, so the incentive to do that is zero.
> 
> They should work without wakelock since wakelock are optional .. I mean
> there's nothing in suspend blockers I've seen that indicates it's
> required for some drivers to work. So it's just a matter of patching out
> the wakelocks, with no need to re-test anything.
> 
> You get the driver mainlined, then maintain a small patch to add
> wakelocks. Not hard at all , with lots of incentive to do so since you
> don't have to maintain such a large block of code out of tree.
> 
> > Practically, as long as the opportunistic suspend is out of tree, there will be
> > a _growing_ number of out-of-tree drivers out there, which is not acceptable
> > in the long run.
> 
> I don't see why your saying that. These driver should work with out all
> of this, which means they can get mainlined right now.

I agree with Daniel here. We should keep merging the drivers separate
from the suspend blocks issues.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Thu, 13 May 2010, Tony Lindgren wrote:

> * Alan Stern <stern@rowland.harvard.edu> [100513 07:11]:
> > On Wed, 12 May 2010, Paul Walmsley wrote:
> > 
> > > Hello,
> > > 
> > > Some general comments on the suspend blockers/wakelock/opportunistic
> > > suspend v6 patch series, posted here:
> > > 
> > >     https://lists.linux-foundation.org/pipermail/linux-pm/2010-April/025146.html
> > > 
> > > The comments below are somewhat telegraphic in the interests of 
> > > readability - more specific comments to follow in later E-mails. I am 
> > > indebted to those of us who discussed these issues at LPC last year and 
> > > ELC this year for several stimulating discussions.
> > > 
> > > There are several general problems with the design of opportunistic
> > > suspend and suspend-blocks.
> > > 
> > > 1. The opportunistic suspend code bypasses existing Linux kernel code,
> > >    such as timers and the scheduler, that indicates when code
> > >    needs to run, and when the system is idle.
> > 
> > Whoa!  That's not my understanding at all.
> > 
> > As I see it, opportunistic suspend doesn't bypass any code that isn't 
> > already bypassed by the existing suspend code.  Users can do
> > 
> > 	echo mem >/sys/power/state
> > 
> > whenever they want, without regard to kernel timers and the scheduler 
> > (other than the fact that the user's thread must be running in order to 
> > carry out the write, of course).
> 
> The difference between echo mem > /sys/power/state and suspend blocks
> is that with suspend blocks the system keeps running.

Irrelevant.  Paul wasn't talking about the suspend blockers; he was 
talking about opportunistic suspend.  So what's the difference between 
opportunistic suspend and "echo mem >/sys/power/state"?  Especially 
when suspend blockers aren't being used?

> And that's why 
> it should be handled by runtime power management instead.

Runtime PM is not capable of freezing userspace and shutting down CPUs.  
More or less by definition -- if it could then it wouldn't be "runtime" 
any more, since the processor wouldn't be running.

> The suspend blocks seems like a hack to spam filter good and bad
> apps from timer usage point of view. Applications are categorized
> as good or bad depending if they grab a susped blocker or not.

You're referring just to the userspace part of the suspend blocker 
API.  What about the kernel part?

> I believe categorizing the apps should be instead done with some
> timer flags or cgroups instead.

That would be compatible with using suspend blockers.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Thu, 13 May 2010, Tony Lindgren wrote:

> Well this is an interesting problem, and once solved will be handy
> for all kind of things. My worry is that if it's integrated in it's
> current form it will be totally out of control all over the place :(
> 
> Still hoping we can come up with some clean way that avoid the patching
> all over the place part.. How about the following, can you please check
> if it would help with your example of guaranteed handling of event:
> 
> 1. In the kernel, we add one more timer queue for critical timers.
>    The current timer queue(s) stay as it is.
> 
> 2. We allow selecting the timer based on some flag, the default
>    behaviour being the current default timer queue.
> 
> 3. Then we add next_timer_interupt_critical() to only query the
>    critical timers along the lines of the current next_timer_interrupt().
> 
> 4. We implement a custom pm_idle that suspends the system based on
>    some logic and checking if next_timer_interrupt_critical() is
>    empty. If the next_timer_interrupt_critical() does not return
>    anything, we assume it's OK to suspend the system.
> 
> Now to me it sounds if your the input layer and userspace handle
> both grab the timers with the critical flags, it should be guaranteed
> that the events get handled before the system is suspended.

Why do you want this to be tied to timers?  Many of the events in 
question are asynchronous with no specific timing relations.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Greg KH]

On Thu, May 13, 2010 at 02:33:29PM -0700, Daniel Walker wrote:
> On Thu, 2010-05-13 at 23:27 +0200, Rafael J. Wysocki wrote:
> 
> > Because someone would have to remove suspend blockers (or rather wakelocks)
> > from the drivers, test that they work correctly without suspend blockers and
> > submit the modified versions.  Going forward, every party responsible for such
> > a driver would have to maintain an out-of-tree version with suspend blockers
> > (or wakelocks) anyway, so the incentive to do that is zero.
> 
> They should work without wakelock since wakelock are optional .. I mean
> there's nothing in suspend blockers I've seen that indicates it's
> required for some drivers to work. So it's just a matter of patching out
> the wakelocks, with no need to re-test anything.
> 
> You get the driver mainlined, then maintain a small patch to add
> wakelocks. Not hard at all , with lots of incentive to do so since you
> don't have to maintain such a large block of code out of tree.

Sorry, but it doesn't seem to work that way.  Look at the large number
of out-of-tree android device drivers that remain sitting there because
of the lack of this interface being in the kernel.

Also note that such a driver, without wakelocks, would never get tested,
and so, things start quickly diverging.

thanks,

greg k-h

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Alan Stern <stern@rowland.harvard.edu> [100513 14:32]:
> On Thu, 13 May 2010, Tony Lindgren wrote:
> 
> > The difference between echo mem > /sys/power/state and suspend blocks
> > is that with suspend blocks the system keeps running.
> 
> Irrelevant.  Paul wasn't talking about the suspend blockers; he was 
> talking about opportunistic suspend.  So what's the difference between 
> opportunistic suspend and "echo mem >/sys/power/state"?  Especially 
> when suspend blockers aren't being used?

Opportunistic suspend is really trying to do runtime PM, see below.
 
> > And that's why 
> > it should be handled by runtime power management instead.
> 
> Runtime PM is not capable of freezing userspace and shutting down CPUs.  
> More or less by definition -- if it could then it wouldn't be "runtime" 
> any more, since the processor wouldn't be running.

Not true. We are already powering off CPUs and rebooting them for
at least omaps in every idle loop using cpuidle. The memory stays on.
 
> > The suspend blocks seems like a hack to spam filter good and bad
> > apps from timer usage point of view. Applications are categorized
> > as good or bad depending if they grab a susped blocker or not.
> 
> You're referring just to the userspace part of the suspend blocker 
> API.  What about the kernel part?

IMHO some timer flags should be used in the kernel too. Currently
there's no way of knowing which timers are good or bad from suspend
point of view.

Regards,

Tony 

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Alan Stern <stern@rowland.harvard.edu> [100513 14:36]:
> On Thu, 13 May 2010, Tony Lindgren wrote:
> 
> > Well this is an interesting problem, and once solved will be handy
> > for all kind of things. My worry is that if it's integrated in it's
> > current form it will be totally out of control all over the place :(
> > 
> > Still hoping we can come up with some clean way that avoid the patching
> > all over the place part.. How about the following, can you please check
> > if it would help with your example of guaranteed handling of event:
> > 
> > 1. In the kernel, we add one more timer queue for critical timers.
> >    The current timer queue(s) stay as it is.
> > 
> > 2. We allow selecting the timer based on some flag, the default
> >    behaviour being the current default timer queue.
> > 
> > 3. Then we add next_timer_interupt_critical() to only query the
> >    critical timers along the lines of the current next_timer_interrupt().
> > 
> > 4. We implement a custom pm_idle that suspends the system based on
> >    some logic and checking if next_timer_interrupt_critical() is
> >    empty. If the next_timer_interrupt_critical() does not return
> >    anything, we assume it's OK to suspend the system.
> > 
> > Now to me it sounds if your the input layer and userspace handle
> > both grab the timers with the critical flags, it should be guaranteed
> > that the events get handled before the system is suspended.
> 
> Why do you want this to be tied to timers?  Many of the events in 
> question are asynchronous with no specific timing relations.

To me it seems that the non-timer related events can be dealt
with toggling the opportunistic suspend idle flag in sysfs.
That should depend on the device and use specific policy.

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Daniel Walker <dwalker@fifo99.com> [100513 14:28]:
> > On Thu, 2010-05-13 at 23:27 +0200, Rafael J. Wysocki wrote:
> > 
> > > Because someone would have to remove suspend blockers (or rather wakelocks)
> > > from the drivers, test that they work correctly without suspend blockers and
> > > submit the modified versions.  Going forward, every party responsible for such
> > > a driver would have to maintain an out-of-tree version with suspend blockers
> > > (or wakelocks) anyway, so the incentive to do that is zero.
> > 
> > They should work without wakelock since wakelock are optional .. I mean
> > there's nothing in suspend blockers I've seen that indicates it's
> > required for some drivers to work. So it's just a matter of patching out
> > the wakelocks, with no need to re-test anything.
> > 
> > You get the driver mainlined, then maintain a small patch to add
> > wakelocks. Not hard at all , with lots of incentive to do so since you
> > don't have to maintain such a large block of code out of tree.
> > 
> > > Practically, as long as the opportunistic suspend is out of tree, there will be
> > > a _growing_ number of out-of-tree drivers out there, which is not acceptable
> > > in the long run.
> > 
> > I don't see why your saying that. These driver should work with out all
> > of this, which means they can get mainlined right now.
> 
> I agree with Daniel here. We should keep merging the drivers separate
> from the suspend blocks issues.

Unfortunately, that's completely unrealistic.  Please refer to the Greg's reply
for details.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
> > On Thursday 13 May 2010, Matthew Garrett wrote:
> > > On Thu, May 13, 2010 at 01:23:20PM -0700, Tony Lindgren wrote:
> > > > * Matthew Garrett <mjg@redhat.com> [100513 13:03]:
> > > > > On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> > > > > 
> > > > > > The system stays running because there's something to do. The system
> > > > > > won't suspend until all the processors hit the kernel idle loop and
> > > > > > the next_timer_interrupt_critical() returns nothing.
> > > > > 
> > > > > At which point an application in a busy loop cripples you.
> > > > 
> > > > Maybe you could deal with the misbehaving untrusted apps in the userspace
> > > > by sending kill -STOP to them when the screen blanks? Then continue
> > > > when some event wakes up the system again.
> > > 
> > > And if that's the application that's listening to the network socket 
> > > that you want to get a wakeup event from? This problem is hard. I'd love 
> > > there to be an elegant solution based on using the scheduler, but I 
> > > really don't know what it is.
> > 
> > I agree and I don't understand the problem that people have with the
> > opportunistic suspend feature.
> 
> It seems to be picking quite a few comments for one.
>  
> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
> > differently, while there's a growing number of out-of-tree drivers depending
> > on this framework.  We need those drivers in and because we don't have any
> > viable alternative at hand, we have no good reason to reject it.
> 
> Nothing is preventing merging the drivers can be merged without
> these calls.

And yet, there _is_ a growing nuber of drivers that don't get merge because
of that.  That's _reality_.  Are you going to discuss with facts, or what?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:08]:
> > On Thursday 13 May 2010, Tony Lindgren wrote:
> > > 
> > > The difference between echo mem > /sys/power/state and suspend blocks
> > > is that with suspend blocks the system keeps running.
> > 
> > Care to elaborate?
> 
> The suspend blocks affects the runtime behaviour of the system by
> categorizing apps into good and bad apps in a weird way.

I don't see this, can you please explain?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Thu, 13 May 2010, Tony Lindgren wrote:

> > > And that's why 
> > > it should be handled by runtime power management instead.
> > 
> > Runtime PM is not capable of freezing userspace and shutting down CPUs.  
> > More or less by definition -- if it could then it wouldn't be "runtime" 
> > any more, since the processor wouldn't be running.
> 
> Not true. We are already powering off CPUs and rebooting them for
> at least omaps in every idle loop using cpuidle. The memory stays on.

Okay, that's a valid point.  But is that approach usable in general 
(i.e., on non-OMAP systems)?

How do you handle situations where the CPU is currently idle but an 
event (such as I/O completion) is expected to occur in the near future?  
You don't want to power-off and reboot then, do you?

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Alan Stern <stern@rowland.harvard.edu> [100513 14:32]:
> > On Thu, 13 May 2010, Tony Lindgren wrote:
> > 
> > > The difference between echo mem > /sys/power/state and suspend blocks
> > > is that with suspend blocks the system keeps running.
> > 
> > Irrelevant.  Paul wasn't talking about the suspend blockers; he was 
> > talking about opportunistic suspend.  So what's the difference between 
> > opportunistic suspend and "echo mem >/sys/power/state"?  Especially 
> > when suspend blockers aren't being used?
> 
> Opportunistic suspend is really trying to do runtime PM, see below.

NO, IT IS NOT!  What it does is to detect situations in which it is desirable
to put the _entire_ _system_ to sleep, while runtime PM works on a per-device
basis.

> > > And that's why 
> > > it should be handled by runtime power management instead.
> > 
> > Runtime PM is not capable of freezing userspace and shutting down CPUs.  
> > More or less by definition -- if it could then it wouldn't be "runtime" 
> > any more, since the processor wouldn't be running.
> 
> Not true. We are already powering off CPUs and rebooting them for
> at least omaps in every idle loop using cpuidle. The memory stays on.

What about user space, though?  Do you freeze it?

> > > The suspend blocks seems like a hack to spam filter good and bad
> > > apps from timer usage point of view. Applications are categorized
> > > as good or bad depending if they grab a susped blocker or not.
> > 
> > You're referring just to the userspace part of the suspend blocker 
> > API.  What about the kernel part?
> 
> IMHO some timer flags should be used in the kernel too. Currently
> there's no way of knowing which timers are good or bad from suspend
> point of view.

How is that answering the Alan's question?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Thursday 13 May 2010, Tony Lindgren wrote:
> * Alan Stern <stern@rowland.harvard.edu> [100513 14:36]:
> > On Thu, 13 May 2010, Tony Lindgren wrote:
> > 
> > > Well this is an interesting problem, and once solved will be handy
> > > for all kind of things. My worry is that if it's integrated in it's
> > > current form it will be totally out of control all over the place :(
> > > 
> > > Still hoping we can come up with some clean way that avoid the patching
> > > all over the place part.. How about the following, can you please check
> > > if it would help with your example of guaranteed handling of event:
> > > 
> > > 1. In the kernel, we add one more timer queue for critical timers.
> > >    The current timer queue(s) stay as it is.
> > > 
> > > 2. We allow selecting the timer based on some flag, the default
> > >    behaviour being the current default timer queue.
> > > 
> > > 3. Then we add next_timer_interupt_critical() to only query the
> > >    critical timers along the lines of the current next_timer_interrupt().
> > > 
> > > 4. We implement a custom pm_idle that suspends the system based on
> > >    some logic and checking if next_timer_interrupt_critical() is
> > >    empty. If the next_timer_interrupt_critical() does not return
> > >    anything, we assume it's OK to suspend the system.
> > > 
> > > Now to me it sounds if your the input layer and userspace handle
> > > both grab the timers with the critical flags, it should be guaranteed
> > > that the events get handled before the system is suspended.
> > 
> > Why do you want this to be tied to timers?  Many of the events in 
> > question are asynchronous with no specific timing relations.
> 
> To me it seems that the non-timer related events can be dealt
> with toggling the opportunistic suspend idle flag in sysfs.
> That should depend on the device and use specific policy.

OK, that's all hand waving.  Do you have any patches, please?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Tony Lindgren]

* Alan Stern <stern@rowland.harvard.edu> [100513 14:56]:
> On Thu, 13 May 2010, Tony Lindgren wrote:
> 
> > > > And that's why 
> > > > it should be handled by runtime power management instead.
> > > 
> > > Runtime PM is not capable of freezing userspace and shutting down CPUs.  
> > > More or less by definition -- if it could then it wouldn't be "runtime" 
> > > any more, since the processor wouldn't be running.
> > 
> > Not true. We are already powering off CPUs and rebooting them for
> > at least omaps in every idle loop using cpuidle. The memory stays on.
> 
> Okay, that's a valid point.  But is that approach usable in general 
> (i.e., on non-OMAP systems)?

Yes if your system wakes to interrupts at least. If your system does
not wake to timer events, then you'll get missed timers. So it should
work on PC too with CONFIG_NO_HZ and if RTC was used for the timer
wake event.
 
> How do you handle situations where the CPU is currently idle but an 
> event (such as I/O completion) is expected to occur in the near future?  
> You don't want to power-off and reboot then, do you?

The idle code looks at next_timer_interrupt() value, then if the
next timer event if far enough ahead, the system powers down and
wakes to the timer interrupt. It also wakes to device interrupts.

For the drivers to block hitting off-idle custom idle functions
can be used. Paul Walmsley and Kevin Hilman have some ideas on
having generic latency information available for that. So there
are similar issues to opportunistic suspend + suspend blocks
that should be sorted out in a generic way.

For the drivers to power down, some timeout value seems to work
best.

Regards,

Tony

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[tytso]

On Thu, May 13, 2010 at 02:25:56PM -0700, Tony Lindgren wrote:
> > I agree and I don't understand the problem that people have with the
> > opportunistic suspend feature.
> 
> It seems to be picking quite a few comments for one.

It's picking up a lot of comments because *someone* seems to be trying
to use a "last post wins" style of argumentation.  One of the things
that is hard for people who aren't regular denizens of LKML is to
understand whose comments can be ignored.....

Seriously, you are posting a lot, but it's not clear you're making any
clear sense...

					- Ted

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Thu, May 13, 2010 at 12:42 PM, Tony Lindgren <tony@atomide.com> wrote:
> * Matthew Garrett <mjg@redhat.com> [100513 12:20]:
>> On Thu, May 13, 2010 at 12:17:17PM -0700, Tony Lindgren wrote:
>> > The suspend blocks seems like a hack to spam filter good and bad
>> > apps from timer usage point of view. Applications are categorized
>> > as good or bad depending if they grab a susped blocker or not.
>> >
>> > I believe categorizing the apps should be instead done with some
>> > timer flags or cgroups instead.
>>
>> I agree, but we have no mechanism for implementing that in a race-free
>> way. We don't even have a realistical proposal for what that mechanism
>> would look like. Should we refuse bread today for the promise of cake
>> tomorrow?
>
> Well this is an interesting problem, and once solved will be handy
> for all kind of things. My worry is that if it's integrated in it's
> current form it will be totally out of control all over the place :(
>
> Still hoping we can come up with some clean way that avoid the patching
> all over the place part.. How about the following, can you please check
> if it would help with your example of guaranteed handling of event:
>
> 1. In the kernel, we add one more timer queue for critical timers.
>   The current timer queue(s) stay as it is.
>
> 2. We allow selecting the timer based on some flag, the default
>   behaviour being the current default timer queue.
>
> 3. Then we add next_timer_interupt_critical() to only query the
>   critical timers along the lines of the current next_timer_interrupt().
>
> 4. We implement a custom pm_idle that suspends the system based on
>   some logic and checking if next_timer_interrupt_critical() is
>   empty. If the next_timer_interrupt_critical() does not return
>   anything, we assume it's OK to suspend the system.
>

I think trying to solve this at the timer level requires more patching
than using suspend blockers. With suspend blocker you block suspend
until some work has finished, or an event has been consumed. You can
safely use any existing synchronous api.

> Now to me it sounds if your the input layer and userspace handle
> both grab the timers with the critical flags, it should be guaranteed
> that the events get handled before the system is suspended.
>

There are no timers used in this path. However any critical section
used from this path could cause the system to enter idle. Instead of
blocking suspend while there are unprocessed input event you now have
to make sure all code that can block any of the threads involved in
processing those event don't wait on a non critical timer.

-- 
Arve Hjønnevåg

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Thu, May 13, 2010 at 02:46:53PM -0700, Greg KH wrote:
> On Thu, May 13, 2010 at 02:33:29PM -0700, Daniel Walker wrote:

> > You get the driver mainlined, then maintain a small patch to add
> > wakelocks. Not hard at all , with lots of incentive to do so since you
> > don't have to maintain such a large block of code out of tree.

> Sorry, but it doesn't seem to work that way.  Look at the large number
> of out-of-tree android device drivers that remain sitting there because
> of the lack of this interface being in the kernel.

Is that really the issue?  The ones I've looked at have mostly suffered
from being built on 2.6.29 and needing refreshes for current kernel APIs
or from general code quality issues - I don't recall ever looking at one
and thinking that the wakelocks were the one issue.

> Also note that such a driver, without wakelocks, would never get tested,
> and so, things start quickly diverging.

Chances are that if the driver is useful people will start using it in
non-Android systems anyway.  As people have already observed wakelocks
needn't have any practical effect on the running system so if the
drivers are broken without wakelocks they'd be broken anyway.  

If this really is a big concern with getting drivers into mainline then
surely we could just add some noop wakelock functions for the drivers to
call?  It's not particularly pretty but it'd deal with the driver merge
side of things.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Friday 14 May 2010, Tony Lindgren wrote:
> * Alan Stern <stern@rowland.harvard.edu> [100513 14:56]:
> > On Thu, 13 May 2010, Tony Lindgren wrote:
> > 
> > > > > And that's why 
> > > > > it should be handled by runtime power management instead.
> > > > 
> > > > Runtime PM is not capable of freezing userspace and shutting down CPUs.  
> > > > More or less by definition -- if it could then it wouldn't be "runtime" 
> > > > any more, since the processor wouldn't be running.
> > > 
> > > Not true. We are already powering off CPUs and rebooting them for
> > > at least omaps in every idle loop using cpuidle. The memory stays on.
> > 
> > Okay, that's a valid point.  But is that approach usable in general 
> > (i.e., on non-OMAP systems)?
> 
> Yes if your system wakes to interrupts at least. If your system does
> not wake to timer events, then you'll get missed timers. So it should
> work on PC too with CONFIG_NO_HZ and if RTC was used for the timer
> wake event.
>  
> > How do you handle situations where the CPU is currently idle but an 
> > event (such as I/O completion) is expected to occur in the near future?  
> > You don't want to power-off and reboot then, do you?
> 
> The idle code looks at next_timer_interrupt() value, then if the
> next timer event if far enough ahead, the system powers down and
> wakes to the timer interrupt. It also wakes to device interrupts.

For the record, waking to interrupts doesn't work on quite some systems
(like ACPI-based PCs for one example).

Rafael

RE: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Woodruff, Richard]

> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> owner@vger.kernel.org] On Behalf Of Greg KH
> Sent: Thursday, May 13, 2010 4:47 PM

> Also note that such a driver, without wakelocks, would never get tested,
> and so, things start quickly diverging.

Do wakelock enabled drivers require a wakelock aware user space to function properly?  If the driver is added you want to make sure the benefit is there and testable for all userspaces.

Early Android service managers did take/release userspace locks to ensure the handoff worked.

Getting all these drivers is positive.  Getting to some constraint mechanism is positive.  It does need to exist end to end to make a difference at the battery.

Regards,
Richard W.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Greg KH]

On Thu, May 13, 2010 at 11:27:49PM +0100, Mark Brown wrote:
> On Thu, May 13, 2010 at 02:46:53PM -0700, Greg KH wrote:
> > On Thu, May 13, 2010 at 02:33:29PM -0700, Daniel Walker wrote:
> 
> > > You get the driver mainlined, then maintain a small patch to add
> > > wakelocks. Not hard at all , with lots of incentive to do so since you
> > > don't have to maintain such a large block of code out of tree.
> 
> > Sorry, but it doesn't seem to work that way.  Look at the large number
> > of out-of-tree android device drivers that remain sitting there because
> > of the lack of this interface being in the kernel.
> 
> Is that really the issue?  The ones I've looked at have mostly suffered
> from being built on 2.6.29 and needing refreshes for current kernel APIs
> or from general code quality issues - I don't recall ever looking at one
> and thinking that the wakelocks were the one issue.

Yes, it is an issue.  See the mess that we have had in trying to get
some of the original G1 drivers merged in the staging tree for proof of
this.

> > Also note that such a driver, without wakelocks, would never get tested,
> > and so, things start quickly diverging.
> 
> Chances are that if the driver is useful people will start using it in
> non-Android systems anyway.

No, not usually.  Do you really think that someone else is going to use
the G1 camera driver for anything else?  :)

> As people have already observed wakelocks needn't have any practical
> effect on the running system so if the drivers are broken without
> wakelocks they'd be broken anyway.  
> 
> If this really is a big concern with getting drivers into mainline then
> surely we could just add some noop wakelock functions for the drivers to
> call?  It's not particularly pretty but it'd deal with the driver merge
> side of things.

I fail to see why getting the real functionality of the wakelocks into
the kernel is a problem.  Especially as they fit a real need, and work
well for that.

thanks,

greg k-h

Re: [PATCH 0/8] Suspend block api (version 6)

[Greg KH]

On Thu, May 13, 2010 at 05:33:58PM -0500, Woodruff, Richard wrote:
> 
> > From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> > owner@vger.kernel.org] On Behalf Of Greg KH
> > Sent: Thursday, May 13, 2010 4:47 PM
> 
> > Also note that such a driver, without wakelocks, would never get tested,
> > and so, things start quickly diverging.
> 
> Do wakelock enabled drivers require a wakelock aware user space to
> function properly?

Not that I can tell, but others might know more.

> If the driver is added you want to make sure the benefit is there and
> testable for all userspaces.

Agreed.

thanks,

greg k-h

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Friday 14 May 2010, Mark Brown wrote:
> On Thu, May 13, 2010 at 02:46:53PM -0700, Greg KH wrote:
> > On Thu, May 13, 2010 at 02:33:29PM -0700, Daniel Walker wrote:
> 
> > > You get the driver mainlined, then maintain a small patch to add
> > > wakelocks. Not hard at all , with lots of incentive to do so since you
> > > don't have to maintain such a large block of code out of tree.
> 
> > Sorry, but it doesn't seem to work that way.  Look at the large number
> > of out-of-tree android device drivers that remain sitting there because
> > of the lack of this interface being in the kernel.
> 
> Is that really the issue?  The ones I've looked at have mostly suffered
> from being built on 2.6.29 and needing refreshes for current kernel APIs
> or from general code quality issues - I don't recall ever looking at one
> and thinking that the wakelocks were the one issue.
> 
> > Also note that such a driver, without wakelocks, would never get tested,
> > and so, things start quickly diverging.
> 
> Chances are that if the driver is useful people will start using it in
> non-Android systems anyway.

You're missing the point IMO.  Even if they are only used on Android, there
still is a problem if they don't go into the mainline, because that leads to
code divergence that's growing over time and will ultimately create an entire
ecosystem that's independent of the mainline.

We've been successful in avoiding that for quite some time and I don't think
we should allow that to happen right now because of the opportunistic suspend
feature.

I'm not a big fan of suspend blockers myself, but let's face it, _currently_
there's no alternative and we need to stop the trend, the sooner the better.

> As people have already observed wakelocks
> needn't have any practical effect on the running system so if the
> drivers are broken without wakelocks they'd be broken anyway.  

You need to prove the reverse, ie. that a driver working correctly with
wakelocks will also work correctly without them, which is not a given.

> If this really is a big concern with getting drivers into mainline then
> surely we could just add some noop wakelock functions for the drivers to
> call?

Well, I don't see a big difference between that and adding the feature
depending on a .config option.  Really.

> It's not particularly pretty but it'd deal with the driver merge
> side of things.

Again, I don't see why you hate that feature so much.  What is there to worry
about?

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Thu, May 13, 2010 at 3:46 PM, Greg KH <gregkh@suse.de> wrote:
> On Thu, May 13, 2010 at 05:33:58PM -0500, Woodruff, Richard wrote:
>>
>> > From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
>> > owner@vger.kernel.org] On Behalf Of Greg KH
>> > Sent: Thursday, May 13, 2010 4:47 PM
>>
>> > Also note that such a driver, without wakelocks, would never get tested,
>> > and so, things start quickly diverging.
>>
>> Do wakelock enabled drivers require a wakelock aware user space to
>> function properly?
>
> Not that I can tell, but others might know more.
>

Some of our drivers may not work correctly with forced suspend, but if
you don't use suspend at all, the wakelocks have no effect and all the
drivers will work correctly.

>> If the driver is added you want to make sure the benefit is there and
>> testable for all userspaces.
>
> Agreed.
>
> thanks,
>
> greg k-h
>



-- 
Arve Hjønnevåg

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Thu, May 13, 2010 at 3:46 PM, Greg KH <gregkh@suse.de> wrote:
> On Thu, May 13, 2010 at 05:33:58PM -0500, Woodruff, Richard wrote:
>>
>> > From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
>> > owner@vger.kernel.org] On Behalf Of Greg KH
>> > Sent: Thursday, May 13, 2010 4:47 PM
>>
>> > Also note that such a driver, without wakelocks, would never get tested,
>> > and so, things start quickly diverging.
>>
>> Do wakelock enabled drivers require a wakelock aware user space to
>> function properly?
>
> Not that I can tell, but others might know more.

Drivers with correct wakelock usage will play nice with opportunistic suspend.

If you're not using opportunistic suspend, you probably don't need
wakelocks at all, and the driver (unless it's broken) should work just
fine for you.

>> If the driver is added you want to make sure the benefit is there and
>> testable for all userspaces.
>
> Agreed.

Definitely.  The fact that say the dsp audio driver or serial driver
on MSM use wakelocks to play nice with opportunistic suspend should
have no impact on, say, Debian-on-G1 which is not using that feature.
Debian would still be able to play audio or write to the serial port.

With wakelock support in the kernel, I'm able to maintain drivers that
(provided they meet the normal style, correctness, etc requirements)
that both can be submitted to mainline (yay!) and can ship on
production hardware as-is (yay!).  Porting other linux based
environments to hardware like G1, N1, etc becomes that much easier
too, which hopefully makes various folks happy.

This helps get us ever closer to being able to build a
production-ready kernel for various android devices "out of the box"
from the mainline tree and gets me ever closer to not being in the
business of maintaining a bunch of SoC-specific
android-something-2.6.# trees, which seriously is not a business I
particularly want to be in.

Brian

Re: [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Fri, May 14, 2010 at 12:46:33AM +0200, Rafael J. Wysocki wrote:
> On Friday 14 May 2010, Mark Brown wrote:

> > Is that really the issue?  The ones I've looked at have mostly suffered
> > from being built on 2.6.29 and needing refreshes for current kernel APIs
> > or from general code quality issues - I don't recall ever looking at one
> > and thinking that the wakelocks were the one issue.

...

> > Chances are that if the driver is useful people will start using it in
> > non-Android systems anyway.

> You're missing the point IMO.  Even if they are only used on Android, there
> still is a problem if they don't go into the mainline, because that leads to
> code divergence that's growing over time and will ultimately create an entire
> ecosystem that's independent of the mainline.

See my first paragraph there.  My point here is that we appear to have
the standard vendor BSP ecosystem problem here rather than a wakelock
problem.

It's fairly common in the embedded space to get a whole bunch of work
done which doesn't make its way into mainline due to a SoC vendor having
produced a BSP for their SoC which is based around a particular kernel
version which never gets updated.  This means users with that SoC can't
boot anything newer until someone does the work of mainlining support
for the system, meaning that development on systems using that SoC gets
stuck on an old kernel which mainline drifts away from.  Users find it
hard to contribute back since they can't run current code easily and
often have to jump through serious hoops to backport drivers from newer
kernels.  If the SoC is successful enough then you do get something of
an ecosystem around the BSP, though eventually that usually results in
the community doing the mainline merge.

> We've been successful in avoiding that for quite some time and I don't think
> we should allow that to happen right now because of the opportunistic suspend
> feature.

This is still a work in progress in the embedded space (where wakelocks
are primarily relevant).  Many of the major vendors are working in the
right direction here, but it's far from universal and it's not something
that it's easy for vendors to change overnight.

> I'm not a big fan of suspend blockers myself, but let's face it, _currently_
> there's no alternative and we need to stop the trend, the sooner the better.

I don't think this is a major part of the trend - like I say, the fact
that people have been working with an old kernel version is generally
a much more substantial issue than the wakelocks in the code I've seen.

> > As people have already observed wakelocks
> > needn't have any practical effect on the running system so if the
> > drivers are broken without wakelocks they'd be broken anyway.  

> You need to prove the reverse, ie. that a driver working correctly with
> wakelocks will also work correctly without them, which is not a given.

If they can be compiled out then the updates really ought to be trivial.
If not I really need to go back and reexamine what's going on here to
make sure I understand what drivers are supposed to do, I have to
confess that I haven't looked too closely at the driver side API yet.

> > It's not particularly pretty but it'd deal with the driver merge
> > side of things.

> Again, I don't see why you hate that feature so much.  What is there to worry
> about?

As I have said previously I'm not actively opposed to merging wakelocks
at this point.  My major concern has been addressed, I now agree with
most of what the PM guys are saying - it's not the nicest thing ever but
it works for now.

The issue was that when I originally noticed the patch series was being
considered for mainline again was that one effect of using it in a
mobile phone with the standard Linux embedded audio subsystem would be
to break use cases such as audio on voice calls, which isn't really
desirable, and that there was no roadmap for how to fix that or any
other subsystems with similar issues.  This didn't seem like it would
have been a good situation given that the major user is expected to be
Android, which is mainly for mobile phones.

Since we seemed to all agree that no other subsystems were affected,
meaning nothing general was required, I've now implemented support in
the subsystem for ignoring suspends for some audio paths (this should
appear In the next merge window).  This should mesh well with wakelocks.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Thu, May 13, 2010 at 4:36 PM, Mark Brown
<broonie@opensource.wolfsonmicro.com> wrote:
>
>> I'm not a big fan of suspend blockers myself, but let's face it, _currently_
>> there's no alternative and we need to stop the trend, the sooner the better.
>
> I don't think this is a major part of the trend - like I say, the fact
> that people have been working with an old kernel version is generally
> a much more substantial issue than the wakelocks in the code I've seen.

Current published trees are based on .32 (used for the coming-soon
froyo release that's been in late QA for a while now) and forward
development is moving to .34 post final (or in the case of tegra2,
tracking .34-rc series as it happens).  We've been actively snapping
up to track mainline since we started doing this around 2.6.16.  We'd
*love* to be able to get more stuff sanely upstream instead of
maintaining branches and rebasing every other mainline release or so.

> The issue was that when I originally noticed the patch series was being
> considered for mainline again was that one effect of using it in a
> mobile phone with the standard Linux embedded audio subsystem would be
> to break use cases such as audio on voice calls, which isn't really
> desirable, and that there was no roadmap for how to fix that or any
> other subsystems with similar issues.  This didn't seem like it would
> have been a good situation given that the major user is expected to be
> Android, which is mainly for mobile phones.

I'd love to have a separate discussion on using standard linux
embedded audio for mobile devices -- one of my goals for 2010 is to
try to migrate from our "one off" approach on MSM to making use of
ALSA and standard interfaces.  I have a lot of questions about handing
encoded data (mp3/aac/etc) that will be processed by the DSP, how to
approach routing control, and how to best interact with the
user/kernel interface, etc.

Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Thu, May 13, 2010 at 03:45:03PM -0700, Greg KH wrote:
> On Thu, May 13, 2010 at 11:27:49PM +0100, Mark Brown wrote:
> > On Thu, May 13, 2010 at 02:46:53PM -0700, Greg KH wrote:

> > Is that really the issue?  The ones I've looked at have mostly suffered
> > from being built on 2.6.29 and needing refreshes for current kernel APIs
> > or from general code quality issues - I don't recall ever looking at one
> > and thinking that the wakelocks were the one issue.

> Yes, it is an issue.  See the mess that we have had in trying to get
> some of the original G1 drivers merged in the staging tree for proof of
> this.

Ah, that's surprising - I had thought most of the issues there were due
to the substantial MSM architecture core code which most of the G1 stuff
depended on (things like the DSP interface and so on) and the general
need for staging-style updates which churned against the non-mainline
versions rather than the wakelocks.  It's true that the wakelocks
weren't helping, though.
 
Most of the cases I've seen have been off-CPU drivers that were either
working to outdated APIs or having to jump through hoops because they
really wanted to use new kernel features rather than CPU things.

> > Chances are that if the driver is useful people will start using it in
> > non-Android systems anyway.

> No, not usually.  Do you really think that someone else is going to use
> the G1 camera driver for anything else?  :)

Not if it's genuinely G1 specific, although presumably most of that
driver is actually chip drivers for the various components of a camera
subsystem which may well appear in other systems (not that I have the
slightest bit of familiarity with how camera hardware is organised).

> > If this really is a big concern with getting drivers into mainline then
> > surely we could just add some noop wakelock functions for the drivers to
> > call?  It's not particularly pretty but it'd deal with the driver merge
> > side of things.

> I fail to see why getting the real functionality of the wakelocks into
> the kernel is a problem.  Especially as they fit a real need, and work
> well for that.

Well, like I've said I personally don't object to merging them now that
the audio use case has been sorted.  I suggested this because it would
allow something to be put in place to facilitate driver merging which
would avoid the core and userspace issues that people are still raising
with the implementation and let people get on with at least the driver
side.

If wakelocks don't make the next merge window and there is a problem
with drivers then it'd be nice to get the stubs in so that the APIs are
present in subsystem trees for drivers to merge.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Thu, May 13, 2010 at 04:48:40PM -0700, Brian Swetland wrote:
> On Thu, May 13, 2010 at 4:36 PM, Mark Brown

> Current published trees are based on .32 (used for the coming-soon
> froyo release that's been in late QA for a while now) and forward
> development is moving to .34 post final (or in the case of tegra2,
> tracking .34-rc series as it happens).  We've been actively snapping
> up to track mainline since we started doing this around 2.6.16.  We'd

Yeah, I had noticed the development stuff was more up to date - it'll be
good once it gets out of QA and into general use since system
integrators do seem to pick up your new releases fairly quickly.

> *love* to be able to get more stuff sanely upstream instead of
> maintaining branches and rebasing every other mainline release or so.

Having greatly cut down the number of out of tree drivers I'm carrying I
can only say that it's great when it happens :)

> I'd love to have a separate discussion on using standard linux
> embedded audio for mobile devices -- one of my goals for 2010 is to
> try to migrate from our "one off" approach on MSM to making use of

Probably best to do that, this thread is already quite big enough
without going off on a tangent.  FWIW we can usually be found in
#alsa-soc on freenode as well as on the lists.  A couple of pointers
which might help your research, though:

> ALSA and standard interfaces.  I have a lot of questions about handing
> encoded data (mp3/aac/etc) that will be processed by the DSP, how to

This usually doesn't go through ALSA at all (the ALSA APIs can't cope
with variable bitrate data), the data currently goes via DSP specific
APIs and gets injected into the ALSA domain in much the same way as data
from the baseband.

> approach routing control, and how to best interact with the
> user/kernel interface, etc.

Routing control for embedded systems is done by exposing the routing
control to userspace via ALSA controls which can be set by apps - using
a GUI to configure the routes interactively is a massive usability win
in development.  Abstraction for generic user-visible apps is intended
to be handled by a layer on top of that:

	http://www.slimlogic.co.uk/?p=40

which is currently in development but I believe is expected to come to
fruition this year (Liam is driving this one).

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Magnus Damm]

On Fri, May 14, 2010 at 6:47 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Alan Stern <stern@rowland.harvard.edu> [100513 14:32]:
>> On Thu, 13 May 2010, Tony Lindgren wrote:
>>
>> > The difference between echo mem > /sys/power/state and suspend blocks
>> > is that with suspend blocks the system keeps running.
>>
>> Irrelevant.  Paul wasn't talking about the suspend blockers; he was
>> talking about opportunistic suspend.  So what's the difference between
>> opportunistic suspend and "echo mem >/sys/power/state"?  Especially
>> when suspend blockers aren't being used?
>
> Opportunistic suspend is really trying to do runtime PM, see below.
>
>> > And that's why
>> > it should be handled by runtime power management instead.
>>
>> Runtime PM is not capable of freezing userspace and shutting down CPUs.
>> More or less by definition -- if it could then it wouldn't be "runtime"
>> any more, since the processor wouldn't be running.
>
> Not true. We are already powering off CPUs and rebooting them for
> at least omaps in every idle loop using cpuidle. The memory stays on.

I agree with you Tony. I thought shutting down CPUs for power
managment purposes could be done without freezing user space. At least
that's what we do today with SH-Mobile.

I don't dislike the idea of freezing a misbehaing user space app, but
I wonder if hardware platforms really need this. I think hardware
requirements and software requirements should be kept separated.

There seem to be some confusion regarding what Runtime PM can and can
not do. For SH-Mobile we use Runtime PM to manage the clocks and power
supply to on-chip I/O devices, and from CPU idle context we check the
state of the Runtime PM devices and decide what level of CPU deep
sleep we can enter. You can call this system CPU deep sleep if you'd
like depending on the dependencies are layed out on your hardware
platform.

For CPU deep sleep we more or less always stop the clock so we need to
put the memory in self-refresh to avoid loosing memory contents. In
some cases the deep sleep means that the power to the CPU core will be
cut, so a more advanced context save/restore path needs to be used.

Still not sure how the system wide suspend is different from Runtime
PM and CPUidle from the hardware perspective...

/ magnus

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Matthew Garrett <mjg@redhat.com> writes:

> On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
>
>> The system stays running because there's something to do. The system
>> won't suspend until all the processors hit the kernel idle loop and
>> the next_timer_interrupt_critical() returns nothing.
>
> At which point an application in a busy loop cripples you. I think we 
> could implement your suggestion more easily by just giving untrusted 
> applications an effectively infinite amount of timer slack, 
>
> but it still doesn't handle the case where an app behaves
> excrutiatingly badly.

Is design for exruciatingly bad apps a design requirement?

If so, opportunistic suspend + suspend blockers fails as well.  An app
could easily hold a suspend blocker during its entire execution
crippling PM.

Using opportunistic suspend may possibly allow you contain bad
apps/drivers, but at the cost of having to patch already working and
trusted apps and known-working kernel code with suspend blockers.

IMO, rather than accepting a solution that allows bad apps to run
wild, it would be much better to _continue_ focus on tools for finding
and containing bad apps.  This approach has the added bonus of solving
problems on *every* linux-based system, not just Android.

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Magnus Damm <magnus.damm@gmail.com> writes:

> Still not sure how the system wide suspend is different from Runtime
> PM and CPUidle from the hardware perspective...

At least on OMAP and other embedded SoCs I'm familiar with, there is
no difference from the hardware perspecitve.

However, I understand that on ACPI-based systems, there are low-power
that are only reachable via system-wide suspsend since ACPI does not
provide the kernel with fine-grained control to hit those states
during idle.

That being said, I don't think this should be an issue since
opportunistic suspend is currently targetted primarily at embedded HW
which has much more fine-grained power control than traditional
ACPI-based systems.

To me the only real difference between system-wide suspend and runtime
PM + CPUidle is the freezing of userspace.

As has been discussed elsewhere in this discussion, any alternative
solution must address the freezing/idling of userspace.  

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Thu, 2010-05-13 at 14:46 -0700, Greg KH wrote:
> On Thu, May 13, 2010 at 02:33:29PM -0700, Daniel Walker wrote:
> > On Thu, 2010-05-13 at 23:27 +0200, Rafael J. Wysocki wrote:
> > 
> > > Because someone would have to remove suspend blockers (or rather wakelocks)
> > > from the drivers, test that they work correctly without suspend blockers and
> > > submit the modified versions.  Going forward, every party responsible for such
> > > a driver would have to maintain an out-of-tree version with suspend blockers
> > > (or wakelocks) anyway, so the incentive to do that is zero.
> > 
> > They should work without wakelock since wakelock are optional .. I mean
> > there's nothing in suspend blockers I've seen that indicates it's
> > required for some drivers to work. So it's just a matter of patching out
> > the wakelocks, with no need to re-test anything.
> > 
> > You get the driver mainlined, then maintain a small patch to add
> > wakelocks. Not hard at all , with lots of incentive to do so since you
> > don't have to maintain such a large block of code out of tree.
> 
> Sorry, but it doesn't seem to work that way.  Look at the large number
> of out-of-tree android device drivers that remain sitting there because
> of the lack of this interface being in the kernel.

I don't think that's due to this interface tho. During your CELF
presentation you noted several bits of code that could go in right now
but haven't (and still haven't as far as I've seen). I'm actively
pushing code related to Android (with wakelocks removed).. Putting a
wakelock contingency on everything to me doesn't make much sense.

> Also note that such a driver, without wakelocks, would never get tested,
> and so, things start quickly diverging.

That's not totally true. For example the MMC driver had wakelocks (I
think, or for sure mmc core does), and the MMC driver has been tested
for G1 and works fine so far without them. I have code that is queued
for my tree that will enable MMC on G1. I can merge Nexus one support
through my tree also which would allow all the drivers for that device
to eventually be used.

With that device support in mainline then those drivers become nice
things to have with or with out wakelocks. You don't need wakelocks to
run Debian or anything else except Android.

Daniel

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

Please forgive the delay in commenting - I am just now catching up on 
these huge E-mail threads;

On Mon, 3 May 2010, Alan Stern wrote:

> So for example, runtime PM cannot be used to put the system into an S3 
> sleep state.

This is not quite true.  Runtime PM, *in conjunction with CPUIdle,* can 
put many current embedded Linux systems into not just S3, but actually S5.

This is done right now on OMAP in mainline Linux[1].  (S5 is called "C6" 
in that codebase, since the CPUIdle governor is used - this is a 
misnomer.)

This approach allows the system to wake up, or stay running, as requested 
by the timer subsystem, or the scheduler.  No suspend-block-like API is 
needed.

Several consumer-market Linux devices have already shipped using this 
mechanism.

> The simplest example is that suspend blockers can be used to control 
> whether or not the CPU shuts down, whereas runtime PM can't.

Runtime PM still needs to be combined with some sort of governor to 
control not only CPU power states, but also system power states.  CPUIdle 
works pretty well as a system power governor for a single-CPU system, but 
is admittedly something of a hack[2].

Going forward, some sort of basic, system-wide governor is a cleaner 
design[3].  But since the "opportunistic suspend" governor proposed in 
patch 1 ignores timer expiries and the scheduler runqueue, and then hacks 
around that decision by inserting suspend-block calls into code that would 
otherwise work normally, it's not a good approach.


- Paul



1. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/cpuidle34xx.c;h=3d3d035db9aff62ce522e0080e570cfdbf8e70cc;hb=4462dc02842698f173f518c1f5ce79c0fb89395a#l292

2. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu, 13
   May 2010 13:01:33 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18592

3. ibid.

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

On Mon, 3 May 2010, Mark Brown wrote:

> I fully agree with this.  We do need to ensure that a runtime PM based
> system can suspend the CPU core and RAM as well as system suspend can
> but that seems doable.

It is already possible on OMAP-based systems running mainline Linux[1].

Consumer-market Linux devices that do this have been available since at 
least 2007[2].

Intel is pursing a similar approach with Moorestown[3], since "echo mem > 
/sys/power/state" is much more inflexible than an idle-loop-based system 
"suspend."  

Going forward, ACPI-style centralized power management is dead.  Like 
"echo mem > /sys/power/state", ACPI PM is too inflexible for most 
use-cases, and so wastes power because the system must be kept in a 
higher-power state than necessary.  Even Intel is abandoning ACPI[4].


- Paul


1. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri May 14 
   10:22:05 PDT 2010: 
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025530.html

2. http://en.wikipedia.org/wiki/Nokia_N800

3. German Monroy's presentation at ELC 2010, "Wake-ups Effect on Idle 
   Power for Intel's Moorestown MID and Smartphone Platform":
   http://elinux.org/images/0/07/Effect_of_wakeups_on_Moorestown_power.pdf

4. Jacob Pan's presentation at ELC 2010, "Porting the Linux Kernel to x86 
   MID Platforms": http://elinux.org/images/e/ee/Jacob-Pan-x86MID-elc2010.pdf

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hi,

On Mon, 3 May 2010, Rafael J. Wysocki wrote:

> However, the real question is whether or not the opportunistic suspend 
> feature is worth adding to the kernel as such and I think it is.
> 
> To me, it doesn't duplicate the runtime PM framework which is aimed at 
> the power management of individual devices rather than the system as a 
> whole.

Runtime PM is also aimed at system power management as a whole. The 
difference is that it approaches the problem from the bottom up (i.e., the 
activity level of the actual hardware devices) rather than from the top 
down (i.e., "echo mem > /sys/power/state").

In bottom-up approaches, like runtime PM, system-level power management 
becomes a very small fraction of the total power management problem.  It 
is only possible to power down that small fraction once all devices, and 
the clock and power domains/islands that contain those devices, become 
idle.  ("Devices" here is used in a hardware sense, and thus includes the 
CPU.)

A bottom-up approach like runtime PM allows parts of the system can be 
powered off when they are not in use.  This is important on systems that 
have more levels of power management hierarchy than just "device" and 
"system," as is the case on modern SoCs like Renesas SH-Mobile, Intel 
Moorestown[1], and TI OMAP[2].  These chips group devices into sets of 
multiple power domains/islands, which can be controlled independently.

Once the devices, including the CPU, are idle, the system idle loop can be 
entered to determine what power level the system should be programmed to 
enter.  At this point, system-level power management only controls a small 
fraction, maybe 1% or 2%, of the chipset that is not associated with any 
devices; along with any board-level resources that the chipset depends on 
to function that are power-controllable, such as the system high-frequency 
clock oscillator[3].

No heavyweight process of iterating through the device tree to suspend 
them is needed at this point, as with a top-down power management approach 
like Android opportunistic suspend, since the system already knows that 
the devices are idle.

Of course, to determine what system-level power state to enter, there 
needs to be some sort of governor to handle these system-level power 
decisions.  On mainline Linux OMAP, we use the CPUIdle governor to do 
this[4].  This is not the cleanest possible choice[5], but works pretty 
well in the absence of a system-level governor.

The Linux OMAP CPUIdle code considers the next timer expiration and any PM 
constraints[6].  Based on the required wakeup latency and constraints, the 
PM code[7][8] programs the power-controllable parts of the system to enter 
one of several power states.  The CPU then enters WFI (wait-for-interrupt, 
like x86 HLT), and the SoC power management controller implements the 
system power transition, now that all on-chip devices are idle.

All of this is already implemented in mainline Linux code[9].  Devices 
based on this code have already shipped from multiple 
vendors[10][11][12][13]. These devices have small batteries and long 
use-times, which attests to the performance of this approach.

Since this approach does not "echo mem > /sys/power/state", it honors the 
existing Linux timer and scheduler subsystems, and so this is all possible 
without systemic modification of the kernel code tree or device drivers, 
in contrast to the current Android proposal under consideration.



- Paul


1. Jacob Pan's presentation at ELC 2010, "Porting the Linux Kernel to x86 
   MID Platforms": 
   http://elinux.org/images/e/ee/Jacob-Pan-x86MID-elc2010.pdf

2. OMAP35x Technical Reference Manual Rev. F, Figure 4-16 "Device Power 
   Domains": http://www.ti.com/litv/pdf/spruf98f

3. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu, 13
   May 2010 13:01:33 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18592

4. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/cpuidle34xx.c;h=3d3d035db9aff62ce522e0080e570cfdbf8e70cc;hb=4462dc02842698f173f518c1f5ce79c0fb89395a#l292

5. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu, 13
   May 2010 13:01:33 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18592

6. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/plat-omap/include/plat/omap-pm.h;h=3ee41d7114929d771cadbb9f02191fd16c5b5abe;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e

7. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/pm24xx.c;h=374299ea7aded92999b5e54439e43f017806ce4d;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e

8. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/pm34xx.c;h=ea0000bc5358e196df58e88da3f54dd71f0a4706;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e

9. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=tree;f=arch/arm/mach-omap2;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e

10. http://en.wikipedia.org/wiki/Palm_Pre

11. http://en.wikipedia.org/wiki/N800

12. http://en.wikipedia.org/wiki/N810

13. http://en.wikipedia.org/wiki/N900

Re: [PATCH 1/8] PM: Add suspend block api.

[Paul Walmsley]

On Thu, 13 May 2010, Paul Walmsley wrote:

> Some comments on the opportunistic suspend portion of this patch.

...

> While reading the patch, it seemed that this opportunistic suspend
> mechanism is a degenerate case of a system-wide form of CPUIdle.  I'll
> call it "SystemIdle," for lack of a better term.

The footnotes for this E-mail were inadvertently omitted.  Sorry about 
that.  They are:


1. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/cpuidle34xx.c;h=3d3d035db9aff62ce522e0080e570cfdbf8e70cc;hb=4462dc02842698f173f518c1f5ce79c0fb89395a#l292

2. OMAP35x Technical Reference Manual Rev. F, section 4.7.5.2 "System
   Clock Oscillator Control": http://www.ti.com/litv/pdf/spruf98f

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

On Mon, 3 May 2010, Matthew Garrett wrote:

> I agree that the runtime scenario is a far more appealing one from an 
> aesthetic standpoint, but so far we don't have a very compelling 
> argument for dealing with the starting and stopping of userspace.

The problem of how to start and stop (some) userspace is not specifically 
system power management-related, nor top-down, /sys/power/state-suspend 
related.  PM is just one potential user.

It's hard to see how the Android opportunistic suspend approach would be 
useful for the other use-cases (e.g., checkpoint/restart).  On the other 
hand, it's easier to see how something like freezer cgroups could be 
useful for system power management and checkpoint/restart.

And the Android opportunistic suspend patches have design problems - they 
definitely do not come for free, as noted in [1][2][3] (among others' 
comments).


- Paul


1. Paul Walmsley E-mail to the linux-pm mailing list, dated Wed, 12
   May 2010 21:35:30 -0600: http://lkml.org/lkml/2010/5/12/528

2. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:13:50 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18657

3. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:27:56 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18658

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 963 bytes --]


Hello,

On Mon, 3 May 2010, Arve Hjønnevåg wrote:

> No, suspend blockers are mostly used to ensure wakeup events are not
> ignored, and to ensure tasks triggered by these wakeup events
> complete.

Standard Linux systems don't need these, because the scheduler just keeps 
the system running as long as there is work to be done.

Suspend-blocks are only needed because patch 1's opportunistic suspend 
governor tries to suspend the system even when the scheduler indicates 
that there is work to be done.  That decision requires all kinds of hacks 
throughout the codebase [1][2].


- Paul


1. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:27:56 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18658

2. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:13:50 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18657


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Fri, May 14, 2010 at 02:20:43PM -0600, Paul Walmsley wrote:
> Hello,
> 
> On Mon, 3 May 2010, Matthew Garrett wrote:
> 
> > I agree that the runtime scenario is a far more appealing one from an 
> > aesthetic standpoint, but so far we don't have a very compelling 
> > argument for dealing with the starting and stopping of userspace.
> 
> The problem of how to start and stop (some) userspace is not specifically 
> system power management-related, nor top-down, /sys/power/state-suspend 
> related.  PM is just one potential user.
>
> It's hard to see how the Android opportunistic suspend approach would be 
> useful for the other use-cases (e.g., checkpoint/restart).  On the other 
> hand, it's easier to see how something like freezer cgroups could be 
> useful for system power management and checkpoint/restart.

And difficult to see how to implement something using freezer cgroups 
that actually works in this case. Look, I don't want to sound like I 
have a one-track mind or anything, but all of these arguments would be 
significantly more compelling if someone would actually provide a 
concrete implementation proposal that deals with the set of use-cases 
that Google's implementation does and which doesn't make anyone cry. 
Otherwise the immeasurably most likely outcome is that this code gets 
merged and we get to live with it.
 
-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

On Tue, 4 May 2010, Matthew Garrett wrote:

> On Mon, May 03, 2010 at 04:37:22PM -0700, Kevin Hilman wrote:
> 
> > Please forgive the ignorance of ACPI (in embedded, we thankfully live
> > in magical world without ACPI) but doesn't that already happen with
> > CPUidle and C-states?  I think of CPUidle as basically runtime PM for
> > the CPU.  IOW, runtime PM manages the devices, CPUidle manages the CPU
> > (via C-states), resulting in dynaimc PM for the entire system.  What
> > am I missing?
> 
> ACPI doesn't provide any functionality for cutting power to most devices 
> other than shifting into full system suspend.

ACPI may not, but the vast majority of ACPI systems are paired with PCI 
buses, or buses that emulate PCI buses; and PCI devices can implement 
device power control through the PCI Power Management capability. On my 
2-year old ThinkPad, 13 of the 24 devices do, according to lspci.

Many Linux device drivers already use PCI D-states to support a 
rudimentary form of runtime PM:

[paul@twilight current]$ fgrep -r pci_set_power_state drivers/ | cut -d: 
-f1 | sort -u
drivers/ata/ata_piix.c
drivers/ata/libata-core.c
drivers/ata/pata_macio.c
drivers/block/cciss.c
drivers/char/agp/amd64-agp.c
drivers/char/agp/amd-k7-agp.c
drivers/char/agp/ati-agp.c
drivers/char/agp/nvidia-agp.c
drivers/char/agp/sis-agp.c
drivers/char/agp/via-agp.c
drivers/firewire/ohci.c
drivers/gpio/bt8xxgpio.c
drivers/gpu/drm/i915/i915_dma.c
drivers/gpu/drm/i915/i915_drv.c
drivers/gpu/drm/nouveau/nouveau_drv.c
drivers/gpu/drm/radeon/radeon_device.c
drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
drivers/i2c/busses/i2c-i801.c
drivers/ide/delkin_cb.c
drivers/ide/sc1200.c
drivers/ide/setup-pci.c
drivers/ieee1394/ohci1394.c
drivers/media/video/bt8xx/bttv-driver.c
drivers/media/video/cx88/cx88-mpeg.c
drivers/media/video/cx88/cx88-video.c
drivers/media/video/saa7134/saa7134-core.c
drivers/memstick/host/jmb38x_ms.c
drivers/message/fusion/mptbase.c
drivers/misc/cb710/core.c
drivers/misc/tifm_7xx1.c
drivers/mmc/host/sdhci-pci.c
drivers/mmc/host/via-sdmmc.c
drivers/net/3c59x.c
drivers/net/8139cp.c
drivers/net/8139too.c
drivers/net/amd8111e.c
drivers/net/atl1c/atl1c_main.c
drivers/net/atl1e/atl1e_main.c
drivers/net/atlx/atl1.c
drivers/net/atlx/atl2.c
drivers/net/benet/be_main.c
drivers/net/e1000/e1000_main.c
drivers/net/e1000e/netdev.c
drivers/net/e100.c
drivers/net/forcedeth.c
drivers/net/igb/igb_main.c
drivers/net/irda/vlsi_ir.c
drivers/net/ixgbe/ixgbe_main.c
drivers/net/jme.c
drivers/net/ksz884x.c
drivers/net/myri10ge/myri10ge.c
drivers/net/ne2k-pci.c
drivers/net/netxen/netxen_nic_main.c
drivers/net/pci-skeleton.c
drivers/net/pcnet32.c
drivers/net/qlcnic/qlcnic_main.c
drivers/net/qlge/qlge_main.c
drivers/net/r8169.c
drivers/net/sc92031.c
drivers/net/sfc/efx.c
drivers/net/sis900.c
drivers/net/skge.c
drivers/net/sky2.c
drivers/net/smsc9420.c
drivers/net/starfire.c
drivers/net/tc35815.c
drivers/net/tg3.c
drivers/net/tulip/dmfe.c
drivers/net/tulip/tulip_core.c
drivers/net/tulip/uli526x.c
drivers/net/typhoon.c
drivers/net/via-rhine.c
drivers/net/via-velocity.c
drivers/net/vmxnet3/vmxnet3_drv.c
drivers/net/wireless/adm8211.c
drivers/net/wireless/airo.c
drivers/net/wireless/ath/ath9k/pci.c
drivers/net/wireless/hostap/hostap_pci.c
drivers/net/wireless/ipw2x00/ipw2100.c
drivers/net/wireless/ipw2x00/ipw2200.c
drivers/net/wireless/iwlwifi/iwl-core.c
drivers/net/wireless/orinoco/orinoco_pci.h
drivers/net/wireless/p54/p54pci.c
drivers/net/wireless/prism54/islpci_hotplug.c
drivers/net/wireless/rt2x00/rt2x00pci.c
drivers/net/wireless/rtl818x/rtl8180_dev.c
drivers/parport/parport_serial.c
drivers/pci/pci-acpi.c
drivers/pci/pci.c
drivers/pci/pci-driver.c
drivers/pcmcia/yenta_socket.c
drivers/scsi/3w-9xxx.c
drivers/scsi/3w-sas.c
drivers/scsi/aic7xxx/aic79xx_osm_pci.c
drivers/scsi/aic7xxx/aic7xxx_osm_pci.c
drivers/scsi/aic7xxx/aic7xxx_pci.c
drivers/scsi/hpsa.c
drivers/scsi/lpfc/lpfc_init.c
drivers/scsi/megaraid/megaraid_sas.c
drivers/scsi/mpt2sas/mpt2sas_scsih.c
drivers/scsi/nsp32.c
drivers/scsi/pm8001/pm8001_init.c
drivers/scsi/pmcraid.c
drivers/serial/8250_pci.c
drivers/serial/serial_txx9.c
drivers/spi/dw_spi_pci.c
drivers/ssb/pcihost_wrapper.c
drivers/staging/crystalhd/crystalhd_lnx.c
drivers/staging/rt2860/pci_main_dev.c
drivers/staging/rtl8187se/r8180_core.c
drivers/staging/rtl8192e/r8192_pm.c
drivers/staging/sm7xx/smtcfb.c
drivers/staging/vt6655/device_main.c
drivers/usb/gadget/langwell_udc.c
drivers/video/arkfb.c
drivers/video/aty/aty128fb.c
drivers/video/aty/atyfb_base.c
drivers/video/aty/radeon_pm.c
drivers/video/i810/i810_main.c
drivers/video/nvidia/nvidia.c
drivers/video/s3fb.c
drivers/video/savage/savagefb_driver.c
drivers/video/vt8623fb.c
drivers/virtio/virtio_pci.c


- Paul

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

> On Thursday 13 May 2010, Tony Lindgren wrote:
>> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:

[...]

>>  
>> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
>> > differently, while there's a growing number of out-of-tree drivers depending
>> > on this framework.  We need those drivers in and because we don't have any
>> > viable alternative at hand, we have no good reason to reject it.
>> 
>> Nothing is preventing merging the drivers can be merged without
>> these calls.
>
> And yet, there _is_ a growing nuber of drivers that don't get merge because
> of that.  That's _reality_.  Are you going to discuss with facts, or what?

It may be reality, but IMO, "preventing other drivers" isn't a good
*technical* argument for merging a feature.  It feels like these "for
the 'good' of the community" arguments are being used to trump the
technical arguments.  Maybe we need to keep the separate.

Distros (especially embedded ones) have long had out of tree features
that create barriers to getting other drivers upstream.  While it
might be nice to see all those features upstream, no one has argued
that they should get merged simply because they create a barrier.  Each
feature should be merged on its own technical merit.

Kevin

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello,

On Tue, 4 May 2010, Alan Stern wrote:

> Another thing that Kevin is missing: There is more to the system than
> the devices and the CPU.  For example: RAM, an embedded controller (on
> modern desktop/laptop systems), a power supply, and so on.  Dynamic PM
> for the CPU and the devices won't power-down these things, but system
> PM will.

No.  A dynamic PM-based kernel can, and in the case of the mainline Linux 
code running on OMAP, does, power down RAM[1] and the power supply[2].

Since the CPU is already powered off when it is time for this to happen, 
so this is done under autonomous hardware control.  We currently use 
CPUIdle[3] to choose whether to program the SoC hardware to enable this 
autonomous power control before it goes off, depending on the required 
wakeup latency for timers and PM constraints.


- Paul


1. (technically, put the RAM into self-refresh mode): 
   http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/sleep34xx.S/sleep34xx.S;h=d522cd70bf53485d0a3af2453845777e3a89edd7;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e#l204

2. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/sleep34xx.S/sleep34xx.S;h=d522cd70bf53485d0a3af2453845777e3a89edd7;hb=4fc4c3ce0dc1096cbd0daa3fe8f6905cbec2b87e#l204

3. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/mach-omap2/cpuidle34xx.c;h=3d3d035db9aff62ce522e0080e570cfdbf8e70cc;hb=4462dc02842698f173f518c1f5ce79c0fb89395a#l292

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Friday 14 May 2010, Kevin Hilman wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> > On Thursday 13 May 2010, Tony Lindgren wrote:
> >> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
> 
> [...]
> 
> >>  
> >> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
> >> > differently, while there's a growing number of out-of-tree drivers depending
> >> > on this framework.  We need those drivers in and because we don't have any
> >> > viable alternative at hand, we have no good reason to reject it.
> >> 
> >> Nothing is preventing merging the drivers can be merged without
> >> these calls.
> >
> > And yet, there _is_ a growing nuber of drivers that don't get merge because
> > of that.  That's _reality_.  Are you going to discuss with facts, or what?
> 
> It may be reality, but IMO, "preventing other drivers" isn't a good
> *technical* argument for merging a feature.  It feels like these "for
> the 'good' of the community" arguments are being used to trump the
> technical arguments.  Maybe we need to keep the separate.
> 
> Distros (especially embedded ones) have long had out of tree features
> that create barriers to getting other drivers upstream.  While it
> might be nice to see all those features upstream, no one has argued
> that they should get merged simply because they create a barrier.  Each
> feature should be merged on its own technical merit.

Well, this is very much like the selinux vs apparmour (& friends) issue.
One can argue we need only one of them, but in fact we're not worse off having
both in.

The feature is not technically unacceptable to me and since having it in
would potentially make it easier to merge quite a few drivers, I regard that as
a good enough argument for.  YMMV.

Thanks,
Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Kevin Hilman <khilman@deeprootsystems.com> writes:

> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
>
>> On Thursday 13 May 2010, Tony Lindgren wrote:
>>> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
>
> [...]
>
>>>  
>>> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
>>> > differently, while there's a growing number of out-of-tree drivers depending
>>> > on this framework.  We need those drivers in and because we don't have any
>>> > viable alternative at hand, we have no good reason to reject it.
>>> 
>>> Nothing is preventing merging the drivers can be merged without
>>> these calls.
>>
>> And yet, there _is_ a growing nuber of drivers that don't get merge because
>> of that.  That's _reality_.  Are you going to discuss with facts, or what?
>
> It may be reality, but IMO, "preventing other drivers" isn't a good
> *technical* argument for merging a feature.  It feels like these "for
> the 'good' of the community" arguments are being used to trump the
> technical arguments.  Maybe we need to keep the separate.

To continue along the "for the good of the community" path...

If it truly is the lack of a suspend blocker API that is preventing
the merge of these out of tree drivers, I second Mark's proposal[1] to
merge a noop version of the API while the technical issues continue to
be discussed.  Then we would see how many drivers get submitted and
merged.

Personally, I suspect that lack of this feature is not the real
obstacle to getting these out-of-tree drivers upstream.  Having this
API upstream will not change the product schedules and corporate
cultures that have prevented code from making its way upstream.

Kevin

[1] https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025501.html

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Friday 14 May 2010, Kevin Hilman wrote:
> Kevin Hilman <khilman@deeprootsystems.com> writes:
> 
> > "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> >
> >> On Thursday 13 May 2010, Tony Lindgren wrote:
> >>> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
> >
> > [...]
> >
> >>>  
> >>> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
> >>> > differently, while there's a growing number of out-of-tree drivers depending
> >>> > on this framework.  We need those drivers in and because we don't have any
> >>> > viable alternative at hand, we have no good reason to reject it.
> >>> 
> >>> Nothing is preventing merging the drivers can be merged without
> >>> these calls.
> >>
> >> And yet, there _is_ a growing nuber of drivers that don't get merge because
> >> of that.  That's _reality_.  Are you going to discuss with facts, or what?
> >
> > It may be reality, but IMO, "preventing other drivers" isn't a good
> > *technical* argument for merging a feature.  It feels like these "for
> > the 'good' of the community" arguments are being used to trump the
> > technical arguments.  Maybe we need to keep the separate.
> 
> To continue along the "for the good of the community" path...
> 
> If it truly is the lack of a suspend blocker API that is preventing
> the merge of these out of tree drivers, I second Mark's proposal[1] to
> merge a noop version of the API while the technical issues continue to
> be discussed.

I'm against that, sorry.

> Then we would see how many drivers get submitted and merged.
>
> Personally, I suspect that lack of this feature is not the real
> obstacle to getting these out-of-tree drivers upstream.  Having this
> API upstream will not change the product schedules and corporate
> cultures that have prevented code from making its way upstream.

But apparently it is considered as a suitable excuse.

Rafael

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
>
> Hello,
>
> On Mon, 3 May 2010, Arve Hjønnevåg wrote:
>
>> No, suspend blockers are mostly used to ensure wakeup events are not
>> ignored, and to ensure tasks triggered by these wakeup events
>> complete.
>
> Standard Linux systems don't need these,

If you don't want to lose wakeup events they do. Standard Linux
systems support suspend, but since they usually don't have a lot of
wakeup events you don't run into a lot of problems.

> because the scheduler just keeps
> the system running as long as there is work to be done.
>

That is only true if you never use suspend.

> Suspend-blocks are only needed because patch 1's opportunistic suspend
> governor tries to suspend the system even when the scheduler indicates
> that there is work to be done.  That decision requires all kinds of hacks
> throughout the codebase [1][2].
>
>
> - Paul
>
>
> 1. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
>   May 2010 00:27:56 -0600:
>   http://permalink.gmane.org/gmane.linux.power-management.general/18658
>
> 2. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
>   May 2010 00:13:50 -0600:
>   http://permalink.gmane.org/gmane.linux.power-management.general/18657
>
>


-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Matthew Garrett <mjg@redhat.com> writes:

> On Fri, May 14, 2010 at 02:20:43PM -0600, Paul Walmsley wrote:
>> Hello,
>> 
>> On Mon, 3 May 2010, Matthew Garrett wrote:
>> 
>> > I agree that the runtime scenario is a far more appealing one from an 
>> > aesthetic standpoint, but so far we don't have a very compelling 
>> > argument for dealing with the starting and stopping of userspace.
>> 
>> The problem of how to start and stop (some) userspace is not specifically 
>> system power management-related, nor top-down, /sys/power/state-suspend 
>> related.  PM is just one potential user.
>>
>> It's hard to see how the Android opportunistic suspend approach would be 
>> useful for the other use-cases (e.g., checkpoint/restart).  On the other 
>> hand, it's easier to see how something like freezer cgroups could be 
>> useful for system power management and checkpoint/restart.
>
> And difficult to see how to implement something using freezer cgroups 
> that actually works in this case. Look, I don't want to sound like I 
> have a one-track mind or anything, but all of these arguments would be 
> significantly more compelling if someone would actually provide a 
> concrete implementation proposal that deals with the set of use-cases 
> that Google's implementation does and which doesn't make anyone cry. 

That might be possible if this "set of use-cases" was available
someplace.  At least I haven't seen it, and would expect it to be in
the docs included with patch 1.

Another likely reason that that there hasn't been an alternate
proposal (at least from some of us that are raising concerns) is
because we already have a working solution to dynamic, system-wide PM
that is 1) already in mainline and 2) shipping on consumer devices
with very strict power budgets (as already pointed out in detail by
Paul[2].)

Yes, "excruciatingly bad" apps can kill PM on these systems since
anyone can write apps, but the same is true on an opporunistic-suspend
based system since any app could hold a suspend blocker whenever it
wants.

Kevin

[1] https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025415.html
[2] https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025532.html

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Fri, May 14, 2010 at 3:32 PM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:
> Matthew Garrett <mjg@redhat.com> writes:
>
>> On Fri, May 14, 2010 at 02:20:43PM -0600, Paul Walmsley wrote:
>>> Hello,
>>>
>>> On Mon, 3 May 2010, Matthew Garrett wrote:
>>>
>>> > I agree that the runtime scenario is a far more appealing one from an
>>> > aesthetic standpoint, but so far we don't have a very compelling
>>> > argument for dealing with the starting and stopping of userspace.
>>>
>>> The problem of how to start and stop (some) userspace is not specifically
>>> system power management-related, nor top-down, /sys/power/state-suspend
>>> related.  PM is just one potential user.
>>>
>>> It's hard to see how the Android opportunistic suspend approach would be
>>> useful for the other use-cases (e.g., checkpoint/restart).  On the other
>>> hand, it's easier to see how something like freezer cgroups could be
>>> useful for system power management and checkpoint/restart.
>>
>> And difficult to see how to implement something using freezer cgroups
>> that actually works in this case. Look, I don't want to sound like I
>> have a one-track mind or anything, but all of these arguments would be
>> significantly more compelling if someone would actually provide a
>> concrete implementation proposal that deals with the set of use-cases
>> that Google's implementation does and which doesn't make anyone cry.
>
> That might be possible if this "set of use-cases" was available
> someplace.  At least I haven't seen it, and would expect it to be in
> the docs included with patch 1.
>
> Another likely reason that that there hasn't been an alternate
> proposal (at least from some of us that are raising concerns) is
> because we already have a working solution to dynamic, system-wide PM
> that is 1) already in mainline and 2) shipping on consumer devices
> with very strict power budgets (as already pointed out in detail by
> Paul[2].)
>
> Yes, "excruciatingly bad" apps can kill PM on these systems since
> anyone can write apps, but the same is true on an opporunistic-suspend
> based system since any app could hold a suspend blocker whenever it
> wants.
>

No, apps need permission to block suspend.

-- 
Arve Hjønnevåg

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

> On Friday 14 May 2010, Kevin Hilman wrote:
>> Kevin Hilman <khilman@deeprootsystems.com> writes:
>> 
>> > "Rafael J. Wysocki" <rjw@sisk.pl> writes:
>> >
>> >> On Thursday 13 May 2010, Tony Lindgren wrote:
>> >>> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
>> >
>> > [...]
>> >
>> >>>  
>> >>> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
>> >>> > differently, while there's a growing number of out-of-tree drivers depending
>> >>> > on this framework.  We need those drivers in and because we don't have any
>> >>> > viable alternative at hand, we have no good reason to reject it.
>> >>> 
>> >>> Nothing is preventing merging the drivers can be merged without
>> >>> these calls.
>> >>
>> >> And yet, there _is_ a growing nuber of drivers that don't get merge because
>> >> of that.  That's _reality_.  Are you going to discuss with facts, or what?
>> >
>> > It may be reality, but IMO, "preventing other drivers" isn't a good
>> > *technical* argument for merging a feature.  It feels like these "for
>> > the 'good' of the community" arguments are being used to trump the
>> > technical arguments.  Maybe we need to keep the separate.
>> 
>> To continue along the "for the good of the community" path...
>> 
>> If it truly is the lack of a suspend blocker API that is preventing
>> the merge of these out of tree drivers, I second Mark's proposal[1] to
>> merge a noop version of the API while the technical issues continue to
>> be discussed.
>
> I'm against that, sorry.

OK, I'll bite... Why?

>> Then we would see how many drivers get submitted and merged.
>>
>> Personally, I suspect that lack of this feature is not the real
>> obstacle to getting these out-of-tree drivers upstream.  Having this
>> API upstream will not change the product schedules and corporate
>> cultures that have prevented code from making its way upstream.
>
> But apparently it is considered as a suitable excuse.

No, it is not a _technical_ excuse.  Just a healthy, experience-based
dose of skepticism.

It was expressed because I find the arguments above for merging
because it prevents out-of-tree drivers from merging quite
unconvincing.  This is not just about opportunistic suspend + suspend
blockers specifically but comes from several years experience in the
embedded Linux world.

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Fri, May 14, 2010 at 3:45 PM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:
>>> Personally, I suspect that lack of this feature is not the real
>>> obstacle to getting these out-of-tree drivers upstream.  Having this
>>> API upstream will not change the product schedules and corporate
>>> cultures that have prevented code from making its way upstream.
>>
>> But apparently it is considered as a suitable excuse.
>
> No, it is not a _technical_ excuse.  Just a healthy, experience-based
> dose of skepticism.
>
> It was expressed because I find the arguments above for merging
> because it prevents out-of-tree drivers from merging quite
> unconvincing.  This is not just about opportunistic suspend + suspend
> blockers specifically but comes from several years experience in the
> embedded Linux world.

It provides useful functionality -- you apparently disagree, but the
wakelock/suspendblock model is in use, shipping, and solving problems
for quite a lot of android devices that have been shipping for a while
now.  We actively go to lowest power state in idle (on omap, msm,
etc), and use drivers that aggressively declock and depower modules
(similar to runtime pm), but we have found that using the
opportunistic suspend model combined with wakelocks allows us to
attain even lower average power consumption in always-connected,
actively-syncing devices.

It has been claimed that because Android userspace makes use of this
functionality a number of silicon vendors who want to submit code
upstream are inconvenienced by having to maintain "android" and
"mainline" versions of their drivers.  I can't speak for them, since
nobody has identified the particular inconvenienced vendors to me, nor
have they spoken with me directly, but personally I do find that
having to maintain two different versions of drivers (one version for
upstream, one for shipping products) is inconvenient.

Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/8] Suspend block api (version 6)

[tytso]

On Fri, May 14, 2010 at 03:32:58PM -0700, Kevin Hilman wrote:
> Another likely reason that that there hasn't been an alternate
> proposal (at least from some of us that are raising concerns) is
> because we already have a working solution to dynamic, system-wide PM
> that is 1) already in mainline and 2) shipping on consumer devices
> with very strict power budgets (as already pointed out in detail by
> Paul[2].)

The examples cited where the things like the Palm Pre, and the Nokia
N770/800/810 series.  #1, what works on one embedded
chipset/architecture might not work on another, and #2, the battery
lifetime on the N770 and N800 (both of which I have) is **appalling**
**bad**.

I really don't understand why people are so opposed to merging code
that works well for a very large set of devices and products.  Just
because *you* don't need it is not a sufficiently good reason to argue
for it not be merged.  If you don't want to use it, then don't CONFIG
it in.

						- Ted

Re: [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Arve Hjønnevåg wrote:

> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
> >
> > Hello,
> >
> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
> >
> >> No, suspend blockers are mostly used to ensure wakeup events are not
> >> ignored, and to ensure tasks triggered by these wakeup events
> >> complete.
> >
> > Standard Linux systems don't need these,
> 
> If you don't want to lose wakeup events they do. Standard Linux
> systems support suspend, but since they usually don't have a lot of
> wakeup events you don't run into a lot of problems.

The primary client for opportunistic suspend and suspend blockers
appears to be embedded systems.  These systems are typically capable of
powering the CPU down and up again with low latency, and they typically
have very aggressive runtime PM support capable of powering down each
device when it's not in active use.  Given this ability, it does seem
that opportunistic suspend and suspend blockers might be unnecessary.

I'd like to explore this avenue a little farther.  In particular, what 
is the issue involving loss of wakeup events?  Can you describe this in 
more detail?

> > because the scheduler just keeps
> > the system running as long as there is work to be done.
> >
> 
> That is only true if you never use suspend.

Why would you want to use system suspend if runtime PM can do
everything you need?

Sure, I can see that an ACPI-based system needs something more.  But 
that's not the real issue here.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Rafael J. Wysocki wrote:

> > > How do you handle situations where the CPU is currently idle but an 
> > > event (such as I/O completion) is expected to occur in the near future?  
> > > You don't want to power-off and reboot then, do you?
> > 
> > The idle code looks at next_timer_interrupt() value, then if the
> > next timer event if far enough ahead, the system powers down and
> > wakes to the timer interrupt. It also wakes to device interrupts.
> 
> For the record, waking to interrupts doesn't work on quite some systems
> (like ACPI-based PCs for one example).

Ironically, it appears that ACPI-based PCs are in a position to benefit 
more from opportunistic suspend and suspend blockers than are embedded 
systems -- and yet they are being proposed for use on cell phones 
rather than on desktops.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Magnus Damm wrote:

> I agree with you Tony. I thought shutting down CPUs for power
> managment purposes could be done without freezing user space. At least
> that's what we do today with SH-Mobile.

If you can shut down and restart CPUs in the time periods between 
transitions into and out of the idle loop, then yes -- userspace 
doesn't need to be suspended.  After all, if the idle loop is running 
then no user threads are runnable.  (Not necessarily true on SMP 
systems, but you know what I mean.)

> Still not sure how the system wide suspend is different from Runtime
> PM and CPUidle from the hardware perspective...

For embedded systems, apparently the difference is minimal.  For other
systems, like ACPI-based PCs, there is a big difference: Powering down
devices and CPUs still leaves large parts of the system running.

In addition, these larger systems generally don't have aggressive 
runtime PM support, so a significant fraction (maybe more than 50%) of 
devices won't be powered down when they are idle -- whereas system 
suspend powers virtually everything down.

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Brian Swetland wrote:

> It provides useful functionality -- you apparently disagree, but the
> wakelock/suspendblock model is in use, shipping, and solving problems
> for quite a lot of android devices that have been shipping for a while
> now.  We actively go to lowest power state in idle (on omap, msm,
> etc), and use drivers that aggressively declock and depower modules
> (similar to runtime pm), but we have found that using the
> opportunistic suspend model combined with wakelocks allows us to
> attain even lower average power consumption in always-connected,
> actively-syncing devices.

Can you explain this in more detail?  Are you saying that some devices
go on generating interrupts and causing timers to be scheduled, even 
though what they're doing isn't important enough to prevent the system 
from powering down?

Alan Stern

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Fri, May 14, 2010 at 7:58 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Fri, 14 May 2010, Brian Swetland wrote:
>
>> It provides useful functionality -- you apparently disagree, but the
>> wakelock/suspendblock model is in use, shipping, and solving problems
>> for quite a lot of android devices that have been shipping for a while
>> now.  We actively go to lowest power state in idle (on omap, msm,
>> etc), and use drivers that aggressively declock and depower modules
>> (similar to runtime pm), but we have found that using the
>> opportunistic suspend model combined with wakelocks allows us to
>> attain even lower average power consumption in always-connected,
>> actively-syncing devices.
>
> Can you explain this in more detail?  Are you saying that some devices
> go on generating interrupts and causing timers to be scheduled, even
> though what they're doing isn't important enough to prevent the system
> from powering down?

In tickless mode, the time until next timer is a signed int, so the
longest the kernel will ever sleep is ~2 seconds at a go.  In
practice, userspace entities often have polling behavior that can
trigger more often than that, and I've observed some kernel periodic
timers (haven't cataloged them recently) that happen more often than
once a second.

When we go to full suspend, we know that only specific wakeup sources
(keyboard gpios, baseband voice/ip events, rtc alarms, etc) are going
to wake us up.

Brian

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Fri, May 14, 2010 at 7:25 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Fri, 14 May 2010, Arve Hjønnevåg wrote:
>
>> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> >
>> > Hello,
>> >
>> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
>> >
>> >> No, suspend blockers are mostly used to ensure wakeup events are not
>> >> ignored, and to ensure tasks triggered by these wakeup events
>> >> complete.
>> >
>> > Standard Linux systems don't need these,
>>
>> If you don't want to lose wakeup events they do. Standard Linux
>> systems support suspend, but since they usually don't have a lot of
>> wakeup events you don't run into a lot of problems.
>
> The primary client for opportunistic suspend and suspend blockers
> appears to be embedded systems.  These systems are typically capable of
> powering the CPU down and up again with low latency, and they typically
> have very aggressive runtime PM support capable of powering down each
> device when it's not in active use.  Given this ability, it does seem
> that opportunistic suspend and suspend blockers might be unnecessary.
>

With the current kernel and user-space code we run, we get
significantly better battery life on the msm plaform using suspend
compared to entering the same power state from idle. On some devices
it takes only five wakeup events per second to cut the best case
battery life in half.

> I'd like to explore this avenue a little farther.  In particular, what
> is the issue involving loss of wakeup events?  Can you describe this in
> more detail?
>

On the desktop systems I have used I only wake the up the system by
pressing a button/key or with an rtc alarm. Losing a button or key
wakeup event is not usually a problem on a desktop since the user will
press it again. Losing an alarm however could be a problem and this
can be avoided by using opportunistic suspend and suspend blockers.

>> > because the scheduler just keeps
>> > the system running as long as there is work to be done.
>> >
>>
>> That is only true if you never use suspend.
>
> Why would you want to use system suspend if runtime PM can do
> everything you need?
>
Because it stops threads that wakeup every second to check if they
have work to do (this includes standard kernel threads), and it
prevents bad apps that never go idle from completely destroying our
battery life.

> Sure, I can see that an ACPI-based system needs something more.  But
> that's not the real issue here.
>

The system we started with entered a much lower power state from
suspend than idle so we needed wakelocks to get more than 24 hours
battery life. We kept wakelocks when we moved to the msm platform
since it reduces our power consumption.

-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Fri, May 14, 2010 at 7:35 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Fri, 14 May 2010, Rafael J. Wysocki wrote:
>
>> > > How do you handle situations where the CPU is currently idle but an
>> > > event (such as I/O completion) is expected to occur in the near future?
>> > > You don't want to power-off and reboot then, do you?
>> >
>> > The idle code looks at next_timer_interrupt() value, then if the
>> > next timer event if far enough ahead, the system powers down and
>> > wakes to the timer interrupt. It also wakes to device interrupts.
>>
>> For the record, waking to interrupts doesn't work on quite some systems
>> (like ACPI-based PCs for one example).
>
> Ironically, it appears that ACPI-based PCs are in a position to benefit
> more from opportunistic suspend and suspend blockers than are embedded
> systems -- and yet they are being proposed for use on cell phones
> rather than on desktops.
>

I would also use it on a desktop.

-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Saturday 15 May 2010, Arve Hjønnevåg wrote:
> On Fri, May 14, 2010 at 3:32 PM, Kevin Hilman
> <khilman@deeprootsystems.com> wrote:
> > Matthew Garrett <mjg@redhat.com> writes:
> >
> >> On Fri, May 14, 2010 at 02:20:43PM -0600, Paul Walmsley wrote:
> >>> Hello,
> >>>
> >>> On Mon, 3 May 2010, Matthew Garrett wrote:
> >>>
> >>> > I agree that the runtime scenario is a far more appealing one from an
> >>> > aesthetic standpoint, but so far we don't have a very compelling
> >>> > argument for dealing with the starting and stopping of userspace.
> >>>
> >>> The problem of how to start and stop (some) userspace is not specifically
> >>> system power management-related, nor top-down, /sys/power/state-suspend
> >>> related.  PM is just one potential user.
> >>>
> >>> It's hard to see how the Android opportunistic suspend approach would be
> >>> useful for the other use-cases (e.g., checkpoint/restart).  On the other
> >>> hand, it's easier to see how something like freezer cgroups could be
> >>> useful for system power management and checkpoint/restart.
> >>
> >> And difficult to see how to implement something using freezer cgroups
> >> that actually works in this case. Look, I don't want to sound like I
> >> have a one-track mind or anything, but all of these arguments would be
> >> significantly more compelling if someone would actually provide a
> >> concrete implementation proposal that deals with the set of use-cases
> >> that Google's implementation does and which doesn't make anyone cry.
> >
> > That might be possible if this "set of use-cases" was available
> > someplace.  At least I haven't seen it, and would expect it to be in
> > the docs included with patch 1.
> >
> > Another likely reason that that there hasn't been an alternate
> > proposal (at least from some of us that are raising concerns) is
> > because we already have a working solution to dynamic, system-wide PM
> > that is 1) already in mainline and 2) shipping on consumer devices
> > with very strict power budgets (as already pointed out in detail by
> > Paul[2].)
> >
> > Yes, "excruciatingly bad" apps can kill PM on these systems since
> > anyone can write apps, but the same is true on an opporunistic-suspend
> > based system since any app could hold a suspend blocker whenever it
> > wants.
> >
> 
> No, apps need permission to block suspend.

Are you referring to the fact the permissions of the special device file or
something different?

Rafael

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Saturday 15 May 2010, tytso@mit.edu wrote:
> On Fri, May 14, 2010 at 03:32:58PM -0700, Kevin Hilman wrote:
> > Another likely reason that that there hasn't been an alternate
> > proposal (at least from some of us that are raising concerns) is
> > because we already have a working solution to dynamic, system-wide PM
> > that is 1) already in mainline and 2) shipping on consumer devices
> > with very strict power budgets (as already pointed out in detail by
> > Paul[2].)
> 
> The examples cited where the things like the Palm Pre, and the Nokia
> N770/800/810 series.  #1, what works on one embedded
> chipset/architecture might not work on another, and #2, the battery
> lifetime on the N770 and N800 (both of which I have) is **appalling**
> **bad**.
> 
> I really don't understand why people are so opposed to merging code
> that works well for a very large set of devices and products.  Just
> because *you* don't need it is not a sufficiently good reason to argue
> for it not be merged.  If you don't want to use it, then don't CONFIG
> it in.

Violently agreed.

And really, the only semi-technical argument against the opportunistic suspend
feature I've seen so far in this thread is that it _may_ _be_ possible to
achieve the same goal in a different way.  If I don't see anything more serious
than that, I will take the patchset and push it to Linus (patches [1-6/8] from
version 7 to be precise, as soon as the changelogs are improved as
requested).

Thanks,
Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Thu, May 13, 2010 at 02:34:55PM -0700, Tony Lindgren wrote:
> * Matthew Garrett <mjg@redhat.com> [100513 14:16]:
> > What race-free mechanism do you use to ensure that? It's very easy to 
> > handwave these problems away. It's very difficult to actually write an 
> > implementation that works.
> 
> Can you describe where do you see the race now?

1) Trusted app decides to suspend
2) Network packet that would otherwise wake the system is received
3) Trusted app sends SIGSTOP to untrusted userspace
4) Network packet sits waiting for stopped userspace to process it

Unless the trusted userspace gets woken up on every event that would 
potentially cause a wakeup, you're racy. And the alternative involves an 
extra userspace wakeup for every network packet - which is expensive.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Sat, May 15, 2010 at 12:47 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>> >
>> > Yes, "excruciatingly bad" apps can kill PM on these systems since
>> > anyone can write apps, but the same is true on an opporunistic-suspend
>> > based system since any app could hold a suspend blocker whenever it
>> > wants.
>>
>> No, apps need permission to block suspend.
>
> Are you referring to the fact the permissions of the special device file or
> something different?

The special device file will have filesystem permissions that limit
access to system services.

Arbitrary userspace apps do not have direct access to the device --
they use a service (provided by binder rpc), and the app must declare
its intent (and the user accept this on install) in order to use that
service. The service keeps track of usage stats so if a user
experiences poor battery life they can look at the battery usage thing
and see which apps are keeping the device awake, etc.

Brian

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Saturday 15 May 2010, Kevin Hilman wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> > On Friday 14 May 2010, Kevin Hilman wrote:
> >> Kevin Hilman <khilman@deeprootsystems.com> writes:
> >> 
> >> > "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> >> >
> >> >> On Thursday 13 May 2010, Tony Lindgren wrote:
> >> >>> * Rafael J. Wysocki <rjw@sisk.pl> [100513 14:16]:
> >> >
> >> > [...]
> >> >
> >> >>>  
> >> >>> > It solves a practical issue that _at_ _the_ _moment_ cannot be solved
> >> >>> > differently, while there's a growing number of out-of-tree drivers depending
> >> >>> > on this framework.  We need those drivers in and because we don't have any
> >> >>> > viable alternative at hand, we have no good reason to reject it.
> >> >>> 
> >> >>> Nothing is preventing merging the drivers can be merged without
> >> >>> these calls.
> >> >>
> >> >> And yet, there _is_ a growing nuber of drivers that don't get merge because
> >> >> of that.  That's _reality_.  Are you going to discuss with facts, or what?
> >> >
> >> > It may be reality, but IMO, "preventing other drivers" isn't a good
> >> > *technical* argument for merging a feature.  It feels like these "for
> >> > the 'good' of the community" arguments are being used to trump the
> >> > technical arguments.  Maybe we need to keep the separate.
> >> 
> >> To continue along the "for the good of the community" path...
> >> 
> >> If it truly is the lack of a suspend blocker API that is preventing
> >> the merge of these out of tree drivers, I second Mark's proposal[1] to
> >> merge a noop version of the API while the technical issues continue to
> >> be discussed.
> >
> > I'm against that, sorry.
> 
> OK, I'll bite... Why?

Because in that case the real feature will always be opposed as "unnecessary"
and never merged.  I very much prefer to decide whether to merge it or reject
it right now.

> >> Then we would see how many drivers get submitted and merged.
> >>
> >> Personally, I suspect that lack of this feature is not the real
> >> obstacle to getting these out-of-tree drivers upstream.  Having this
> >> API upstream will not change the product schedules and corporate
> >> cultures that have prevented code from making its way upstream.
> >
> > But apparently it is considered as a suitable excuse.
> 
> No, it is not a _technical_ excuse.  Just a healthy, experience-based
> dose of skepticism.

I didn't mean that, actually.  What I wanted to say is that people use the
lack of "wakelocks" in the mainline as an excuse not to push their drivers
upstream (which is kind of understandable, because there's a zero benefit to
them from mainlining their code, as they will have to maintain a separate
"Android" version of it anyway).

Sorry for the confusion.

> It was expressed because I find the arguments above for merging
> because it prevents out-of-tree drivers from merging quite
> unconvincing.  This is not just about opportunistic suspend + suspend
> blockers specifically but comes from several years experience in the
> embedded Linux world.

In this particular case, the lack of the mainline's support for opportunistic
suspend (in the form of "wakelocks" to be precise) has been given as a main
obstacle against merging of several drivers at least.

So, let's not just easily generalize, please.

And this is not the only reason to push the opportunistic suspend feature
upstream IMO.

First, I think it is a legitimate approach to power management, whether you
like it or not.  I haven't seen anyone seriously arguing against that yet.

Second, (as said above) there is a number of drivers _already_ depending
on it and that number is only going to grow given the popularity of Android.
They are not mainlined in part because their authors don't see a benefit from
doing so (usually the benefit is that you don't have to maintain your code out
of the tree, because the mainline does it for you to some extent, so if you
need to maintain a separate version yourself, the benefit is zero).

Next, the only thing the Arve's patches do is to give people an _option_ to use
opportunistic suspend if they need it or want it.  It's not mandatory and not
even enabled by default, so I don't really see what the problem is.  Quite on
the contrary, I'd like people to be able to use the mainline on Android systems
without major modifications, because potentially that can increase our tester
base (and developer base too in consequence) and including opportunistic
suspend in the mainline would be a step in that direction.

Finally, it appears to address some issues that at the moment we don't
seriously know how to address in a different way.

Thanks,
Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Brian Swetland wrote:

> In tickless mode, the time until next timer is a signed int, so the
> longest the kernel will ever sleep is ~2 seconds at a go.  In
> practice, userspace entities often have polling behavior that can
> trigger more often than that, and I've observed some kernel periodic
> timers (haven't cataloged them recently) that happen more often than
> once a second.

Paul and Kevin, how does the OMAP implementation handle these 
difficulties?

Also, how does it handle the issue of ill-behaved apps?

Alan Stern

Re: [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Fri, 14 May 2010, Arve Hjønnevåg wrote:

> > I'd like to explore this avenue a little farther.  In particular, what
> > is the issue involving loss of wakeup events?  Can you describe this in
> > more detail?
> >
> 
> On the desktop systems I have used I only wake the up the system by
> pressing a button/key or with an rtc alarm. Losing a button or key
> wakeup event is not usually a problem on a desktop since the user will
> press it again. Losing an alarm however could be a problem and this
> can be avoided by using opportunistic suspend and suspend blockers.

How can runtime PM combined with CPUidle cause an alarm to be lost?

> > Why would you want to use system suspend if runtime PM can do
> > everything you need?
> >
> Because it stops threads that wakeup every second to check if they
> have work to do (this includes standard kernel threads), and it
> prevents bad apps that never go idle from completely destroying our
> battery life.

Ah, the ill-behaved apps problem.  I think everybody agrees that they 
are hard to deal with.

The kernel-threads problem might better be addressed by fixing those
threads than by adding a new API.

> > Sure, I can see that an ACPI-based system needs something more.  But
> > that's not the real issue here.
> >
> 
> The system we started with entered a much lower power state from
> suspend than idle so we needed wakelocks to get more than 24 hours
> battery life. We kept wakelocks when we moved to the msm platform
> since it reduces our power consumption.

Is it generally true among embedded systems nowadays that idle is 
capable of reaching essentially the same power states as suspend?

Alan Stern

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Saturday 15 May 2010, Brian Swetland wrote:
> On Sat, May 15, 2010 at 12:47 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> >> >
> >> > Yes, "excruciatingly bad" apps can kill PM on these systems since
> >> > anyone can write apps, but the same is true on an opporunistic-suspend
> >> > based system since any app could hold a suspend blocker whenever it
> >> > wants.
> >>
> >> No, apps need permission to block suspend.
> >
> > Are you referring to the fact the permissions of the special device file or
> > something different?
> 
> The special device file will have filesystem permissions that limit
> access to system services.
> 
> Arbitrary userspace apps do not have direct access to the device --
> they use a service (provided by binder rpc), and the app must declare
> its intent (and the user accept this on install) in order to use that
> service.

On Android, right?

> The service keeps track of usage stats so if a user
> experiences poor battery life they can look at the battery usage thing
> and see which apps are keeping the device awake, etc.

That sounds like it should go into a doc or at least into the changelog of
patch [2/8] IMnsHO.

The thing you said about timers in one of the previous messages should be
documented too, BTW, because it's one of the motivation factors for this
work.

In general, I think the motivation is not explained sufficienty in the patches'
changelogs and that's one of the reasons why people have so many doubts about
the feature's usefulness.

Thanks,
Rafael

Re: [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Sat, May 15, 2010 at 4:51 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> On Saturday 15 May 2010, Brian Swetland wrote:
>> The special device file will have filesystem permissions that limit
>> access to system services.
>>
>> Arbitrary userspace apps do not have direct access to the device --
>> they use a service (provided by binder rpc), and the app must declare
>> its intent (and the user accept this on install) in order to use that
>> service.
>
> On Android, right?

Yup -- other userspace environments may choose different ways to
grant/deny access.  At the bottom we gate it by having a group that
processes which may access the device are part of (with the device
node being 0660, etc).

>> The service keeps track of usage stats so if a user
>> experiences poor battery life they can look at the battery usage thing
>> and see which apps are keeping the device awake, etc.
>
> That sounds like it should go into a doc or at least into the changelog of
> patch [2/8] IMnsHO.
>
> The thing you said about timers in one of the previous messages should be
> documented too, BTW, because it's one of the motivation factors for this
> work.
>
> In general, I think the motivation is not explained sufficienty in the patches'
> changelogs and that's one of the reasons why people have so many doubts about
> the feature's usefulness.

Seems reasonable -- I know Arve's working on making these a bit more
verbose for you.

Brian

Re: [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Sat, 2010-05-15 at 21:52 +0200, Rafael J. Wysocki wrote:
> On Saturday 15 May 2010, tytso@mit.edu wrote:

> > The examples cited where the things like the Palm Pre, and the Nokia
> > N770/800/810 series.  #1, what works on one embedded
> > chipset/architecture might not work on another, and #2, the battery

Pretty much all of the architectures going after this sort of market
seem to be taking very similar design decisions here. The high degree of
integration on the SoCs kind of forces it since you're always going to
end up with at least some things on the CPU sitting around idle so
something needs to be done to drive their runtime power cost down.
However, this does all require software support and the level of
implementation of that you see does vary.

> > lifetime on the N770 and N800 (both of which I have) is **appalling**
> > **bad**.

Hrm. I've got those devices as well and I don't recall the battery life
being particularly awful when they weren't actively being used which is
the interesting case there. It's been a while but ISTR they compared
well enough with the smartphones of their generation. YMMV, of course.

I guess the other direction to look at this from is that Android devices
seem to have battery lifetimes that are comparable to other smartphones,
including Linux based ones, rather than remarkably better than them.
There's a reasonable number out there that I've seen, using a range of
approaches including both pure runtime PM and also normal suspends
initiated from userspace with varying degrees of control over the
customisation end users can do on their system.

Wakelocks aren't totally going out on a limb in their use of suspend,
general systems in this area aren't actually relying purely on runtime
PM yet. Where wakelocks do stand out from the other solutions I have
seen is that they push the management of the decision to suspend into
the kernel.

I think it's probably most accurate to say that the thing that what
wakelocks are designed to deliver is to make the system more forgiving
of power inefficient application code (providing it doesn't just request
permission to burn the battery, of course). I can't think of any actual
implementations I have seen which do anything special to deal with them.
The argument there would I guess be that the impact on battery life
provides a fairly strong incentive to the application developers to not
do silly things. There's a fairly clear tradeoff there, and neither
approach is going to be universally right.

> And really, the only semi-technical argument against the opportunistic suspend
> feature I've seen so far in this thread is that it _may_ _be_ possible to
> achieve the same goal in a different way.  If I don't see anything more serious
> than that, I will take the patchset and push it to Linus (patches [1-6/8] from
> version 7 to be precise, as soon as the changelogs are improved as
> requested).

Well, the other concerns that *might* be considered technical are the
requirement for driver modifications to take wakelocks (personally I'm
fairly OK with that - it's not especially pretty but they seem trivial
enough so meh) and the userspace ABI that's being created to manage
suspend blocking (which I've not even looked at).

Like I say, I don't personally object to merging the code at this point - I
mostly wanted to provide a broader view on the approaches people are using
here.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Sat, 2010-05-15 at 22:14 +0200, Rafael J. Wysocki wrote:
> On Saturday 15 May 2010, Kevin Hilman wrote:
> > "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> > > On Friday 14 May 2010, Kevin Hilman wrote:

> > >> If it truly is the lack of a suspend blocker API that is preventing
> > >> the merge of these out of tree drivers, I second Mark's proposal[1] to
> > >> merge a noop version of the API while the technical issues continue to
> > >> be discussed.

> > > I'm against that, sorry.

> > OK, I'll bite... Why?

> Because in that case the real feature will always be opposed as "unnecessary"
> and never merged.  I very much prefer to decide whether to merge it or reject
> it right now.

FWIW (as the person who made the suggestion) I do think that it is
something one might call expedient rather than actually good.

> > It was expressed because I find the arguments above for merging
> > because it prevents out-of-tree drivers from merging quite
> > unconvincing.  This is not just about opportunistic suspend + suspend
> > blockers specifically but comes from several years experience in the
> > embedded Linux world.

> In this particular case, the lack of the mainline's support for opportunistic
> suspend (in the form of "wakelocks" to be precise) has been given as a main
> obstacle against merging of several drivers at least.

Where are these objections coming from? The only example I've seen cited
is the G1 stuff, which is a fairly special case for a number of reasons
(including the underlying MSM BSP, which was pretty substantial itself).

> And this is not the only reason to push the opportunistic suspend feature
> upstream IMO.

Agreed, my purpose here is mostly to push back on what sound like
unrealistic expectations about what we're getting.

> Second, (as said above) there is a number of drivers _already_ depending
> on it and that number is only going to grow given the popularity of Android.
> They are not mainlined in part because their authors don't see a benefit from
> doing so (usually the benefit is that you don't have to maintain your code out
> of the tree, because the mainline does it for you to some extent, so if you
> need to maintain a separate version yourself, the benefit is zero).

Wakelocks are going to be a fairly minor part of any decision here -
it'd be pretty surprising if they take much effort to remove from a
driver. What's much more of an issue is that you've got essentially the
same situation as you have with the enterprise Linux distributions, a
fixed kernel version that vendors need to target. The differences that
implies are far more substantial than wakelocks for many areas of the
kernel, especially at the points in the cycle where the fixed kernel has
drifted furthest from the current mainline.

Things aren't quite the same as with the enterprise distributions,
though - the lifecycles for many parts in the consumer space are much,
much shorter than those in the enterprise markets. They can be
sufficiently short to mean that a mainline driver won't show up where
customers need it soon enough. For example, Google is currently
preparing an Android release based on 2.6.32 the merge window for which
was in September last year, over six months ago. This is an extremely
long latency if you're working on something in the region of a twelve
month cycle. The lack of standardisation for register interfaces in the
embedded space means that the generation to generation differences can
easily be sufficiently substantial to make a new driver the only
sensible option.

This isn't to say that the old parts just suddenly vanish, and clearly
there's an advantage from ongoing mainline inclusion, but the tradeoffs
are a bit different to those in other markets.

There's also a less pressure from end users towards mainline inclusion -
even on Linux people in the embedded space are used to having to get
code from multiple vendors working together so the lack of mainline
support isn't the sort of issue it would be with something like server
class hardware. This is changing over time as more and more vendors buy
into mainline but there's a way to go yet. If the part requires changes
outside the driver itself (a new or updated kernel interface, for
example) there's a pressure to just deal with it in the driver in a way
which is going to be unacceptable for mainline, possibly even involving
per-system code modifications in the driver. Sometimes ongoing mainline
development will mean that the driver needs updating anyway in ways that
are much more substantial than ripping out wakelocks would be.

Having wakelocks in does make things easier for drivers using them but
we need to recognise that all sorts of other things that are much harder
to deal with will also come up.

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Sat, May 15, 2010 at 2:25 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Fri, 14 May 2010, Arve Hjønnevåg wrote:
>
>> > I'd like to explore this avenue a little farther.  In particular, what
>> > is the issue involving loss of wakeup events?  Can you describe this in
>> > more detail?
>> >
>>
>> On the desktop systems I have used I only wake the up the system by
>> pressing a button/key or with an rtc alarm. Losing a button or key
>> wakeup event is not usually a problem on a desktop since the user will
>> press it again. Losing an alarm however could be a problem and this
>> can be avoided by using opportunistic suspend and suspend blockers.
>
> How can runtime PM combined with CPUidle cause an alarm to be lost?
>
It doesn't. My desktop systems only gets to a low power state (<3W)
from suspend.

>> > Why would you want to use system suspend if runtime PM can do
>> > everything you need?
>> >
>> Because it stops threads that wakeup every second to check if they
>> have work to do (this includes standard kernel threads), and it
>> prevents bad apps that never go idle from completely destroying our
>> battery life.
>
> Ah, the ill-behaved apps problem.  I think everybody agrees that they
> are hard to deal with.
>
> The kernel-threads problem might better be addressed by fixing those
> threads than by adding a new API.
>
>> > Sure, I can see that an ACPI-based system needs something more.  But
>> > that's not the real issue here.
>> >
>>
>> The system we started with entered a much lower power state from
>> suspend than idle so we needed wakelocks to get more than 24 hours
>> battery life. We kept wakelocks when we moved to the msm platform
>> since it reduces our power consumption.
>
> Is it generally true among embedded systems nowadays that idle is
> capable of reaching essentially the same power states as suspend?

Embedded system in general, no, but all the recent SOCs I've seen that
target phones do.

-- 
Arve Hjønnevåg

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Alan Stern <stern@rowland.harvard.edu> writes:

> On Fri, 14 May 2010, Brian Swetland wrote:
>
>> In tickless mode, the time until next timer is a signed int, so the
>> longest the kernel will ever sleep is ~2 seconds at a go.  In
>> practice, userspace entities often have polling behavior that can
>> trigger more often than that, and I've observed some kernel periodic
>> timers (haven't cataloged them recently) that happen more often than
>> once a second.
>
> Paul and Kevin, how does the OMAP implementation handle these 
> difficulties?

just a minor clarification... these aren't OMAP-specific issues, but
generic issues to all power-sensitive kernel users.

The ~2 second limit was fixed by Jon Hunter (TI) and is in mainline
since 2.6.32[1].

For other timers, there has been active work (mostly by Intel folks)
on deferrable timers, coalescing timers, timer slack etc. that has
greatly reduced the kernel timer impact on wakeups.

> Also, how does it handle the issue of ill-behaved apps?

For userspace, apps that have polling behavior or are ill-behaved must
be found and fixed.  Thanks to tools like powertop, this is a farily
easy task.

But really, I don't consider the "ill-behaved app" problem to be a
real-world problem.  Both in maemo/meego and Android, if someone
writes an app that kills battery life, it will get reported as a bug,
or get bad ratings etc.  On these kinds of devices, there is a *stong*
developer incentive to not write battery sucking apps.

Kevin


[1]
commit 97813f2fe77804a4464564c75ba8d8826377feea
Author: Jon Hunter <jon-hunter@ti.com>
Date:   Tue Aug 18 12:45:11 2009 -0500

    nohz: Allow 32-bit machines to sleep for more than 2.15 seconds
    
    In the dynamic tick code, "max_delta_ns" (member of the
    "clock_event_device" structure) represents the maximum sleep time
    that can occur between timer events in nanoseconds.
    
    The variable, "max_delta_ns", is defined as an unsigned long
    which is a 32-bit integer for 32-bit machines and a 64-bit
    integer for 64-bit machines (if -m64 option is used for gcc).
    The value of max_delta_ns is set by calling the function
    "clockevent_delta2ns()" which returns a maximum value of LONG_MAX.
    For a 32-bit machine LONG_MAX is equal to 0x7fffffff and in
    nanoseconds this equates to ~2.15 seconds. Hence, the maximum
    sleep time for a 32-bit machine is ~2.15 seconds, where as for
    a 64-bit machine it will be many years.
    
    This patch changes the type of max_delta_ns to be "u64" instead of
    "unsigned long" so that this variable is a 64-bit type for both 32-bit
    and 64-bit machines. It also changes the maximum value returned by
    clockevent_delta2ns() to KTIME_MAX.  Hence this allows a 32-bit
    machine to sleep for longer than ~2.15 seconds. Please note that this
    patch also changes "min_delta_ns" to be "u64" too and although this is
    unnecessary, it makes the patch simpler as it avoids to fixup all
    callers of clockevent_delta2ns().
    
    [ tglx: changed "unsigned long long" to u64 as we use this data type
        through out the time code ]
    
    Signed-off-by: Jon Hunter <jon-hunter@ti.com>
    Cc: John Stultz <johnstul@us.ibm.com>
    LKML-Reference: <1250617512-23567-3-git-send-email-jon-hunter@ti.com>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

tytso@mit.edu writes:

> On Fri, May 14, 2010 at 03:32:58PM -0700, Kevin Hilman wrote:
>> Another likely reason that that there hasn't been an alternate
>> proposal (at least from some of us that are raising concerns) is
>> because we already have a working solution to dynamic, system-wide PM
>> that is 1) already in mainline and 2) shipping on consumer devices
>> with very strict power budgets (as already pointed out in detail by
>> Paul[2].)
>
> The examples cited where the things like the Palm Pre, and the Nokia
> N770/800/810 series.  #1, what works on one embedded
> chipset/architecture might not work on another, 

In my experience with embedded SoCs they are all remarkably similar in
power management capabilities and control.

> and #2, the battery lifetime on the N770 and N800 (both of which I have)
> is **appalling** **bad**.

Appalling bad compared to what?  

What's probably more interesting in terms of rough comparisons is
comparing similar devices with and without opportunistic suspend.  The
Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
OMAP3) and roughly the same kernel (2.6.2[89], although both are
heavily patched from mainline.)

The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)

I don't know of any more objective comparison of the two, but as a
user of both devices I can say that the active usage is basically the
same (around a day) and the idle use is similar as well, even though
the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
own usage suggests the n900 is a bit better in idle time, but I have
not done any measuring or objective tests.  I'm guessing the
difference is probably because the Droid does not use the deepest
off-mode power states either in idle or suspend (IIRC) where the n900
does.  I suspect that if both were using off-mode and had the same
battery, these differences would go away.

While this is not really a scientific comparison, it at least gives a
rough idea.  If using opportunistic suspend was adding noticably
better battery life, I think this would be a different discussion.

> I really don't understand why people are so opposed to merging code
> that works well for a very large set of devices and products.  Just
> because *you* don't need it is not a sufficiently good reason to argue
> for it not be merged.  If you don't want to use it, then don't CONFIG
> it in.

At least for OMAP, I don't consider "config it out" a viable option.

We would like one kernel (in particular one PM core) to be usable for
a broad range of devices: maemo/meego, Android, webOS, Archos, and
whatever else we haven't seen yet.  Having to config in/out something
as important as core PM functionality has other consequences.  It
makes PM-aware driver and subsystem writers (and maintainers) to have
to develop and validate against two different types of PM
functionality.

As co-maintainer of the PM subsystem for an architecture (OMAP) where
people *really* care about power, I don't consider that a viable or
scalable option.

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[James Bottomley]

On Mon, 2010-05-17 at 08:40 -0700, Kevin Hilman wrote:
> > Also, how does it handle the issue of ill-behaved apps?
> 
> For userspace, apps that have polling behavior or are ill-behaved must
> be found and fixed.  Thanks to tools like powertop, this is a farily
> easy task.

That's a bit glib ... powertop can detect power consumption stats on a
running system ... if you have a polling app preventing your system from
suspending, powertop isn't necessarily going to find it ... especially
if the polling interval is of the order of powertop's.  Powertop can
find the bad tens of wakeups per second, but it only takes one wakup
every few seconds or so to drain the battery significantly when
operating on suspend from idle.

> But really, I don't consider the "ill-behaved app" problem to be a
> real-world problem.  Both in maemo/meego and Android, if someone
> writes an app that kills battery life, it will get reported as a bug,
> or get bad ratings etc.  On these kinds of devices, there is a *stong*
> developer incentive to not write battery sucking apps.

I'm not sure this is real world, either.  Developers can fire up
powertop from the command line when their phone isn't idling for as long
as it should.  But a phone is a consumer device: the average smart phone
user just wants to browse the web, get email, go to facebook and play
with some cool apps.  If one of those cool apps is rogue, they're not
really going to know which one or how to find it (and firing up powertop
from the command line isn't something which will occur to them as a
matter of routine).

One of the nice things that suspend blockers actually does is to give
the kernel a clear name for the process blocking suspend (and thus
consuming power).  This allows a nice way to assign power budget to the
application and present who's using what in a nice visible form, which
does facilitate the reporting of bad apps, even for the non-developer
user.

James

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Mon, May 17, 2010 at 01:04:45PM -0400, James Bottomley wrote:
> > For userspace, apps that have polling behavior or are ill-behaved must
> > be found and fixed.  Thanks to tools like powertop, this is a farily
> > easy task.
> 
> That's a bit glib ... powertop can detect power consumption stats on a
> running system ... if you have a polling app preventing your system from
> suspending, powertop isn't necessarily going to find it ... especially
> if the polling interval is of the order of powertop's.  Powertop can
> find the bad tens of wakeups per second, but it only takes one wakup
> every few seconds or so to drain the battery significantly when
> operating on suspend from idle.

you can always increase powertop's interval through command line and
once you went down to 1 wakeup every two seconds, you increase
powertop's interval and try to cut down 10 more ill-behaved apps. And
you keep going until you have e.g. 1 wakeup per minute or whatever your
target is.

> > But really, I don't consider the "ill-behaved app" problem to be a
> > real-world problem.  Both in maemo/meego and Android, if someone
> > writes an app that kills battery life, it will get reported as a bug,
> > or get bad ratings etc.  On these kinds of devices, there is a *stong*
> > developer incentive to not write battery sucking apps.
> 
> I'm not sure this is real world, either.  Developers can fire up
> powertop from the command line when their phone isn't idling for as long
> as it should.  But a phone is a consumer device: the average smart phone
> user just wants to browse the web, get email, go to facebook and play
> with some cool apps.  If one of those cool apps is rogue, they're not
> really going to know which one or how to find it (and firing up powertop
> from the command line isn't something which will occur to them as a
> matter of routine).

Agree with you here.

> One of the nice things that suspend blockers actually does is to give
> the kernel a clear name for the process blocking suspend (and thus
> consuming power).  This allows a nice way to assign power budget to the
> application and present who's using what in a nice visible form, which
> does facilitate the reporting of bad apps, even for the non-developer
> user.

if that's the only thing we want suspend_blockers for, there are other
simpler ways to do it. Just add a kernel debugging option for anyone
doing poll() or keeping a device open() or whatever and you have the
name the of the processes consuming power and preventing system from
going into sleep.

IMO, suspend_blocker is trying to fix application problems in kernel
space by unconditionaly (well sort of) freezing userspace if there are
no suspen_blockers held. So even if application is doing
poll(pfds, ARRAY_SIZE(pfds), 2); that won't be noticed because as long as
the suspend_blocker is released, that poll() will be frozen, no ?

IMO the real fix would be on that particular poll(), changing the
timeout e.g. based on cpufreq notifications or even relying completely
on IRQs with poll(pdfs, ARRAY_SIZE(pfds), -1); Of course, this is only a
crude example trying to show that the real issue lies on the application
rather than on kernel.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Mon, 2010-05-17 at 13:04 -0400, James Bottomley wrote:

> I'm not sure this is real world, either.  Developers can fire up
> powertop from the command line when their phone isn't idling for as long
> as it should.  But a phone is a consumer device: the average smart phone
> user just wants to browse the web, get email, go to facebook and play
> with some cool apps.  If one of those cool apps is rogue, they're not
> really going to know which one or how to find it (and firing up powertop
> from the command line isn't something which will occur to them as a
> matter of routine).
> 
> One of the nice things that suspend blockers actually does is to give
> the kernel a clear name for the process blocking suspend (and thus
> consuming power).  This allows a nice way to assign power budget to the
> application and present who's using what in a nice visible form, which
> does facilitate the reporting of bad apps, even for the non-developer
> user.

If you have an idle based PM system you could get the same information
from having scheduler statistics. Since the "bad" apps are the ones that
are always either running or ready to run, and they would hardly ever
sleep. I don't know if there are specific scheduler statistics for that,
but it doesn't seem like it would be hard to make. It's even much more
natural than getting statistics from suspend blockers, since it requires
the app to be custom made to use suspend blockers.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 08:47:31PM +0300, Felipe Balbi wrote:

> IMO the real fix would be on that particular poll(), changing the
> timeout e.g. based on cpufreq notifications or even relying completely
> on IRQs with poll(pdfs, ARRAY_SIZE(pfds), -1); Of course, this is only a
> crude example trying to show that the real issue lies on the application
> rather than on kernel.

We know that this problem is mostly uninteresting if your userland is 
well written. The sad truth is that it's impossible to trust that your 
userland is well written, and broadly impossible to communicate to users 
that the reason that their battery life is miserable is because of the 
applications and not because of the platform. If you don't believe that 
that's a worthwhile use case to deal with then suspend blockers buy you 
pretty much nothing. But if you do, then nobody's yet demonstrated 
another workable way for this to be handled.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[James Bottomley]

On Mon, 2010-05-17 at 20:47 +0300, Felipe Balbi wrote:
> On Mon, May 17, 2010 at 01:04:45PM -0400, James Bottomley wrote:
> > > For userspace, apps that have polling behavior or are ill-behaved must
> > > be found and fixed.  Thanks to tools like powertop, this is a farily
> > > easy task.
> > 
> > That's a bit glib ... powertop can detect power consumption stats on a
> > running system ... if you have a polling app preventing your system from
> > suspending, powertop isn't necessarily going to find it ... especially
> > if the polling interval is of the order of powertop's.  Powertop can
> > find the bad tens of wakeups per second, but it only takes one wakup
> > every few seconds or so to drain the battery significantly when
> > operating on suspend from idle.
> 
> you can always increase powertop's interval through command line and
> once you went down to 1 wakeup every two seconds, you increase
> powertop's interval and try to cut down 10 more ill-behaved apps. And
> you keep going until you have e.g. 1 wakeup per minute or whatever your
> target is.

Have you actually tried this?  On my N1 with CM5.0.6 just running
powertop requires me to keep the USB system up (debugging cable) and
paths into the usb console ... all of this produces significant wakeup
distortion, mostly in the msm i2c subsystem.  But in all the noise it's
hard to find rogue applications.

> > > But really, I don't consider the "ill-behaved app" problem to be a
> > > real-world problem.  Both in maemo/meego and Android, if someone
> > > writes an app that kills battery life, it will get reported as a bug,
> > > or get bad ratings etc.  On these kinds of devices, there is a *stong*
> > > developer incentive to not write battery sucking apps.
> > 
> > I'm not sure this is real world, either.  Developers can fire up
> > powertop from the command line when their phone isn't idling for as long
> > as it should.  But a phone is a consumer device: the average smart phone
> > user just wants to browse the web, get email, go to facebook and play
> > with some cool apps.  If one of those cool apps is rogue, they're not
> > really going to know which one or how to find it (and firing up powertop
> > from the command line isn't something which will occur to them as a
> > matter of routine).
> 
> Agree with you here.
> 
> > One of the nice things that suspend blockers actually does is to give
> > the kernel a clear name for the process blocking suspend (and thus
> > consuming power).  This allows a nice way to assign power budget to the
> > application and present who's using what in a nice visible form, which
> > does facilitate the reporting of bad apps, even for the non-developer
> > user.
> 
> if that's the only thing we want suspend_blockers for, there are other
> simpler ways to do it.

Nice straw man slide.

The technical reason for wanting suspend blockers (as has been stated
more times than I can be bothered to go back and count) is that no-one
can currently produce a working model for race free kernel to user work
handoff and, in the face of open app stores, rogue applications are a
significant problem.  The fact that suspend blockers enables easy
identification of power hogging apps is just a very useful side effect.

James


>  Just add a kernel debugging option for anyone
> doing poll() or keeping a device open() or whatever and you have the
> name the of the processes consuming power and preventing system from
> going into sleep.
> 
> IMO, suspend_blocker is trying to fix application problems in kernel
> space by unconditionaly (well sort of) freezing userspace if there are
> no suspen_blockers held. So even if application is doing
> poll(pfds, ARRAY_SIZE(pfds), 2); that won't be noticed because as long as
> the suspend_blocker is released, that poll() will be frozen, no ?
> 
> IMO the real fix would be on that particular poll(), changing the
> timeout e.g. based on cpufreq notifications or even relying completely
> on IRQs with poll(pdfs, ARRAY_SIZE(pfds), -1); Of course, this is only a
> crude example trying to show that the real issue lies on the application
> rather than on kernel.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Mon, May 17, 2010 at 01:59:39PM -0400, James Bottomley wrote:
> Have you actually tried this?  On my N1 with CM5.0.6 just running
> powertop requires me to keep the USB system up (debugging cable) and
> paths into the usb console ... all of this produces significant wakeup
> distortion, mostly in the msm i2c subsystem.  But in all the noise it's
> hard to find rogue applications.

Well, I use serial console, but in the worst case scenario I would make
powertop save the output to a memory buffer a big one, and after
finished flush to some mmc or anything like that.

> The technical reason for wanting suspend blockers (as has been stated
> more times than I can be bothered to go back and count) is that no-one
> can currently produce a working model for race free kernel to user work
> handoff and, in the face of open app stores, rogue applications are a
> significant problem.  The fact that suspend blockers enables easy
> identification of power hogging apps is just a very useful side effect.

I still can't get over the fact that suspend_blockers are dealing with
userland problems in kernel space. If we can't really trust apps, I'm
sorry but companies like Google and Nokia (which I work for) will have
to setup better application acceptance on their stores. The fact that we
can get statistics of power usage from kernel space is actually really
good and could be easily automated on an AppStore environment. If it's
cause too many wakeups you report that to the developer of the app
before accepting. The same feature could be shipped on the SDK, so
developer has also early feedback about how good (or bad) his/her
application really is to the system.

IMO we should be celebrating good apps, not dealing in kernel space with
bad ones. And on top of all that, we would still need custom
applications with suspend_blockers support built into them.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Mon, May 17, 2010 at 06:58:20PM +0100, Matthew Garrett wrote:
> We know that this problem is mostly uninteresting if your userland is 
> well written. The sad truth is that it's impossible to trust that your 
> userland is well written, and broadly impossible to communicate to users 
> that the reason that their battery life is miserable is because of the 
> applications and not because of the platform. If you don't believe that 
> that's a worthwhile use case to deal with then suspend blockers buy you 
> pretty much nothing. But if you do, then nobody's yet demonstrated 
> another workable way for this to be handled.

don't get me wrong, I have faced similar problems of use time targets
due to ill-behaved applications, we just decided to deal with it with
the help of bugzilla. File a bug to that application and get the
developer to fix it. At least you are teaching the guy to fish. I
understand when you open an AppStore the problem grows bigger but, like
I replied to James, build an automated system to check average power
usage on the SDK and AppStore acceptance process and you get developers
to fix their apps before they reach the device.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Mon, May 17, 2010 at 11:12 AM, Felipe Balbi <me@felipebalbi.com> wrote:
>
>> The technical reason for wanting suspend blockers (as has been stated
>> more times than I can be bothered to go back and count) is that no-one
>> can currently produce a working model for race free kernel to user work
>> handoff and, in the face of open app stores, rogue applications are a
>> significant problem.  The fact that suspend blockers enables easy
>> identification of power hogging apps is just a very useful side effect.
>
> I still can't get over the fact that suspend_blockers are dealing with
> userland problems in kernel space. If we can't really trust apps, I'm
> sorry but companies like Google and Nokia (which I work for) will have
> to setup better application acceptance on their stores.

We (Google) would like to allow completely open app distribution with
minimal hurdles, and avoid the walled garden approach.  Toward this
goal we're not even requiring the use of a central app store for
distribution.

Obviously, given the ability to run *any* app, users will run into bad
(or perhaps just less-than-optimal-powerwise) apps.  Being able to
provide the best possible battery life (in spite of
sometimes-nonoptimal userspace apps) and simultaneously informing
users about which apps are better/worse for their battery life is a
goal here.

> IMO we should be celebrating good apps, not dealing in kernel space with
> bad ones. And on top of all that, we would still need custom
> applications with suspend_blockers support built into them.

For a large majority of apps, running in the background while the
device is asleep (screen off) is not essential, they don't request the
"keep device awake" permission, never hold a wakelock, etc.  Those
that do need to do this have the permission, may hold suspend
blockers, and are accounted for.

Unrelated to apps, the ability to say "please enter suspend as soon as
there's no more work (kernel or userspace) preventing it", in a
simple, non-racy way is useful.

Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Mon, May 17, 2010 at 11:26:59AM -0700, Brian Swetland wrote:
> We (Google) would like to allow completely open app distribution with
> minimal hurdles, and avoid the walled garden approach.  Toward this
> goal we're not even requiring the use of a central app store for
> distribution.

I understand that, but still we should be telling developers what
they're doing wrong so that they can improve themselves as professionals
and still make the final device better.

> Obviously, given the ability to run *any* app, users will run into bad
> (or perhaps just less-than-optimal-powerwise) apps.  Being able to
> provide the best possible battery life (in spite of
> sometimes-nonoptimal userspace apps) and simultaneously informing
> users about which apps are better/worse for their battery life is a
> goal here.

I see. Just hope MeeGo doesn't venture on the same waters :-s

> For a large majority of apps, running in the background while the
> device is asleep (screen off) is not essential, they don't request the
> "keep device awake" permission, never hold a wakelock, etc.  Those
> that do need to do this have the permission, may hold suspend
> blockers, and are accounted for.

but can anyone write an app that holds a suspend_blocker ?? If so, then
your goal is already broken, right ? I mean, if anyone can keep a
suspend_blocker held forever, you'll never ever sleep, right ? While
with runtime, if you keep the keypad open, only the keypad and the paths
directly related to it (probably the i2c controller and the power domain
where the i2c controller sits) will be kept alive, no ?

> Unrelated to apps, the ability to say "please enter suspend as soon as
> there's no more work (kernel or userspace) preventing it", in a
> simple, non-racy way is useful.

I just tend to agree with Kevin on questioning how different how
different this actually is from runtime_pm. I guess I would need to dig
through some documentation in order to understand but it seems really
similar.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Mon, May 17, 2010 at 11:39 AM, Felipe Balbi <me@felipebalbi.com> wrote:
> Hi,
>
> On Mon, May 17, 2010 at 11:26:59AM -0700, Brian Swetland wrote:
>> We (Google) would like to allow completely open app distribution with
>> minimal hurdles, and avoid the walled garden approach.  Toward this
>> goal we're not even requiring the use of a central app store for
>> distribution.
>
> I understand that, but still we should be telling developers what
> they're doing wrong so that they can improve themselves as professionals
> and still make the final device better.

I agree.  Which is why we develop tools to help developers understand
what their apps are doing.

>> For a large majority of apps, running in the background while the
>> device is asleep (screen off) is not essential, they don't request the
>> "keep device awake" permission, never hold a wakelock, etc.  Those
>> that do need to do this have the permission, may hold suspend
>> blockers, and are accounted for.
>
> but can anyone write an app that holds a suspend_blocker ?? If so, then
> your goal is already broken, right ? I mean, if anyone can keep a
> suspend_blocker held forever, you'll never ever sleep, right ? While
> with runtime, if you keep the keypad open, only the keypad and the paths
> directly related to it (probably the i2c controller and the power domain
> where the i2c controller sits) will be kept alive, no ?

No, you'll never suspend, which is different from never going to the
lowest CPU power state.  On shipping Android devices we aggressively
completely power down the CPU in idle whenever we can (based on
latency requirements generally).  We power off peripherals whenever
they're not in use.

This is why I've stated previously that I don't think runtime PM and
opportunistic suspend are competitive features.  Everyone who cares
about minimizing power should want runtime pm or at least similar
functionality (our drivers have always powered down peripherals when
not in use, even while the device is open, etc, prior to the existence
of runtime PM).

If your environment is such that going to full suspend will not gain
you anything, then don't use opportunistic suspend.  We find that
there are savings to be had with this model in Android which is why we
use it.  If you are going to use opportunistic suspend,
suspend_blockers provide useful functionality.

Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mark Brown]

On Mon, May 17, 2010 at 09:39:05PM +0300, Felipe Balbi wrote:
> On Mon, May 17, 2010 at 11:26:59AM -0700, Brian Swetland wrote:

> > For a large majority of apps, running in the background while the
> > device is asleep (screen off) is not essential, they don't request the
> > "keep device awake" permission, never hold a wakelock, etc.  Those
> > that do need to do this have the permission, may hold suspend
> > blockers, and are accounted for.

> but can anyone write an app that holds a suspend_blocker ?? If so, then
> your goal is already broken, right ? I mean, if anyone can keep a
> suspend_blocker held forever, you'll never ever sleep, right ? While
> with runtime, if you keep the keypad open, only the keypad and the paths
> directly related to it (probably the i2c controller and the power domain
> where the i2c controller sits) will be kept alive, no ?

The Android UI provides a list of which applications have been
preventing suspend and how long for, the idea being that if something
has been holding suspend blockers for ever you can at least look at the
list and see what it was.

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mike Chan]

On Mon, May 17, 2010 at 11:39 AM, Felipe Balbi <me@felipebalbi.com> wrote:
> Hi,
>
> On Mon, May 17, 2010 at 11:26:59AM -0700, Brian Swetland wrote:
>> We (Google) would like to allow completely open app distribution with
>> minimal hurdles, and avoid the walled garden approach.  Toward this
>> goal we're not even requiring the use of a central app store for
>> distribution.
>
> I understand that, but still we should be telling developers what
> they're doing wrong so that they can improve themselves as professionals
> and still make the final device better.
>

We currently do track power usage per-application which is displayed
in the phone UI. (Settings -> About Phone -> Battery Usage).  We also
provide several (although not perfect) command line utilities for
developers to see their power impact system.

We are constantly working on ways to improve tracking and make such
power data more accessible to developers so they can see how their
applications are impacting battery life.

>> Obviously, given the ability to run *any* app, users will run into bad
>> (or perhaps just less-than-optimal-powerwise) apps.  Being able to
>> provide the best possible battery life (in spite of
>> sometimes-nonoptimal userspace apps) and simultaneously informing
>> users about which apps are better/worse for their battery life is a
>> goal here.
>
> I see. Just hope MeeGo doesn't venture on the same waters :-s
>
>> For a large majority of apps, running in the background while the
>> device is asleep (screen off) is not essential, they don't request the
>> "keep device awake" permission, never hold a wakelock, etc.  Those
>> that do need to do this have the permission, may hold suspend
>> blockers, and are accounted for.
>
> but can anyone write an app that holds a suspend_blocker ?? If so, then
> your goal is already broken, right ? I mean, if anyone can keep a
> suspend_blocker held forever, you'll never ever sleep, right ? While
> with runtime, if you keep the keypad open, only the keypad and the paths
> directly related to it (probably the i2c controller and the power domain
> where the i2c controller sits) will be kept alive, no ?
>

Any app can grab a suspend blocker, and the stats are logged, and if
you're app is abusing wakelocks and CPU resource it will show up in
the "Battery Use" panel.

-- Mike

>> Unrelated to apps, the ability to say "please enter suspend as soon as
>> there's no more work (kernel or userspace) preventing it", in a
>> simple, non-racy way is useful.
>
> I just tend to agree with Kevin on questioning how different how
> different this actually is from runtime_pm. I guess I would need to dig
> through some documentation in order to understand but it seems really
> similar.
>
> --
> balbi
> --
> To unsubscribe from this list: send the line "unsubscribe linux-omap" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

James Bottomley <James.Bottomley@suse.de> writes:

> The technical reason for wanting suspend blockers (as has been stated
> more times than I can be bothered to go back and count) is that no-one
> can currently produce a working model for race free kernel to user work
> handoff

At least I've never heard this technial reason stated so succinctly.  It's
not in the changelogs or in the Documentation file included.

The way I undertand things, today's mainline kernel has a race-free
kernel-to-user work handoff already.

The possibility of races is introduced by the opportunistic suspend
feature itself (patch 1.)  The use of suspend blockers later in the
series is needed to avoid the potential races introduced by
opportunistic suspend.

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[James Bottomley]

On Mon, 2010-05-17 at 21:12 +0300, Felipe Balbi wrote:
> Hi,
> 
> On Mon, May 17, 2010 at 01:59:39PM -0400, James Bottomley wrote:
> > Have you actually tried this?  On my N1 with CM5.0.6 just running
> > powertop requires me to keep the USB system up (debugging cable) and
> > paths into the usb console ... all of this produces significant wakeup
> > distortion, mostly in the msm i2c subsystem.  But in all the noise it's
> > hard to find rogue applications.
> 
> Well, I use serial console, but in the worst case scenario I would make
> powertop save the output to a memory buffer a big one, and after
> finished flush to some mmc or anything like that.

Surely, depending on your UART FIFO depth, of course, a serial console
interrupts once every 16 characters or so ... how do you filter out that
storm of interrupts refreshing the powertop screen from the actual
application problems?

But anyway, the average user probably either doesn't have or doesn't
know how to get to a serial console on their phone ...

> > The technical reason for wanting suspend blockers (as has been stated
> > more times than I can be bothered to go back and count) is that no-one
> > can currently produce a working model for race free kernel to user work
> > handoff and, in the face of open app stores, rogue applications are a
> > significant problem.  The fact that suspend blockers enables easy
> > identification of power hogging apps is just a very useful side effect.
> 
> I still can't get over the fact that suspend_blockers are dealing with
> userland problems in kernel space. If we can't really trust apps, I'm
> sorry but companies like Google and Nokia (which I work for) will have
> to setup better application acceptance on their stores. The fact that we
> can get statistics of power usage from kernel space is actually really
> good and could be easily automated on an AppStore environment. If it's
> cause too many wakeups you report that to the developer of the app
> before accepting. The same feature could be shipped on the SDK, so
> developer has also early feedback about how good (or bad) his/her
> application really is to the system.
> 
> IMO we should be celebrating good apps, not dealing in kernel space with
> bad ones. And on top of all that, we would still need custom
> applications with suspend_blockers support built into them.

If you actually s/app/USB storage device/ (with a few other obvious text
changes) in most of the above two paragraphs, you've got a good
description of the problems we go through on an almost daily basis in
the kernel for USB storage ... and why we've grown a massive exception
table.

Just saying "devices should conform to specifications" is a wonderful
magic wand for wishing away all the problems bad devices cause and
bludgeoning manufacturers with the said spec wrapped around a large lead
brick is very cathartic but it doesn't change the fact that users blame
the kernel for not working with the bad devices ... and we gave up
trying to re-educate users on that score years ago.

James

Re: [PATCH 0/8] Suspend block api (version 6)

[Vitaly Wool]

On Mon, May 17, 2010 at 6:12 PM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:

>> and #2, the battery lifetime on the N770 and N800 (both of which I have)
>> is **appalling** **bad**.
>
> Appalling bad compared to what?
>
> What's probably more interesting in terms of rough comparisons is
> comparing similar devices with and without opportunistic suspend.  The
> Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
> OMAP3) and roughly the same kernel (2.6.2[89], although both are
> heavily patched from mainline.)
>
> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
>
> I don't know of any more objective comparison of the two, but as a
> user of both devices I can say that the active usage is basically the
> same (around a day) and the idle use is similar as well, even though
> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
> own usage suggests the n900 is a bit better in idle time, but I have
> not done any measuring or objective tests.  I'm guessing the
> difference is probably because the Droid does not use the deepest
> off-mode power states either in idle or suspend (IIRC) where the n900
> does.  I suspect that if both were using off-mode and had the same
> battery, these differences would go away.
>
> While this is not really a scientific comparison, it at least gives a
> rough idea.  If using opportunistic suspend was adding noticably
> better battery life, I think this would be a different discussion.

Exactly. The point is, opportunistic suspend doesn't in fact add any
value compared to dynamic PM + CPUIdle. It only produces some false
impression that one can handle power management right without using
dynamic PM. And this false impression is the cause for many really
ugly designs (like, for instance, 15 minutes touchscreen inactivity
delay before forcibly shutting down the wireless, as it's done in
stock Android framework).

Thanks,
   Vitaly

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 09:27:47PM +0200, Vitaly Wool wrote:

> Exactly. The point is, opportunistic suspend doesn't in fact add any
> value compared to dynamic PM + CPUIdle. It only produces some false
> impression that one can handle power management right without using
> dynamic PM. And this false impression is the cause for many really
> ugly designs (like, for instance, 15 minutes touchscreen inactivity
> delay before forcibly shutting down the wireless, as it's done in
> stock Android framework).

Run this (or equivalent code) on an N900 and on an Android. Measure the 
screen-off power draw on both.

int main() {
	int i;
	while (1)
		i++;
	return 0;
}

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Mon, May 17, 2010 at 03:24:27PM -0400, James Bottomley wrote:
> Surely, depending on your UART FIFO depth, of course, a serial console
> interrupts once every 16 characters or so ... how do you filter out that
> storm of interrupts refreshing the powertop screen from the actual
> application problems?
> 
> But anyway, the average user probably either doesn't have or doesn't
> know how to get to a serial console on their phone ...

like I said: use a big memory buffer and print to that. Only flush after
you're done profiling. Something like dmesg.

> If you actually s/app/USB storage device/ (with a few other obvious text
> changes) in most of the above two paragraphs, you've got a good
> description of the problems we go through on an almost daily basis in
> the kernel for USB storage ... and why we've grown a massive exception
> table.
> 
> Just saying "devices should conform to specifications" is a wonderful
> magic wand for wishing away all the problems bad devices cause and
> bludgeoning manufacturers with the said spec wrapped around a large lead
> brick is very cathartic but it doesn't change the fact that users blame
> the kernel for not working with the bad devices ... and we gave up
> trying to re-educate users on that score years ago.

that's a whole other story. Hardware issues are things which in 99.999%
of the cases we can't change. We have to work around them. Software
bugs, on the other hand, can be fixed much more easily. I'm sure you
agree with that, don't you ?

Trying to make a comparisson between hardware bug and software bug is
simply non-sense in this case.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

hi,

On Mon, May 17, 2010 at 10:38:40PM +0300, Felipe Balbi wrote:
> that's a whole other story. Hardware issues are things which in 99.999%
> of the cases we can't change. We have to work around them. Software
> bugs, on the other hand, can be fixed much more easily. I'm sure you
> agree with that, don't you ?
> 
> Trying to make a comparisson between hardware bug and software bug is
> simply non-sense in this case.

before you reply saying that most of the problems are firmware bugs, try
to file a bug to any of the usb storage manufacturers and wait for them
to fix. It's virtualy impossible, so let's consider it a problem that
has to be worked around.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[James Bottomley]

On Mon, 2010-05-17 at 22:39 +0300, Felipe Balbi wrote:
> hi,
> 
> On Mon, May 17, 2010 at 10:38:40PM +0300, Felipe Balbi wrote:
> > that's a whole other story. Hardware issues are things which in 99.999%
> > of the cases we can't change. We have to work around them. Software
> > bugs, on the other hand, can be fixed much more easily. I'm sure you
> > agree with that, don't you ?
> > 
> > Trying to make a comparisson between hardware bug and software bug is
> > simply non-sense in this case.
> 
> before you reply saying that most of the problems are firmware bugs, try
> to file a bug to any of the usb storage manufacturers and wait for them
> to fix. It's virtualy impossible, so let's consider it a problem that
> has to be worked around.

Right, because Firmware writers are from the rugged unresponsive uplands
of planet
ignore-user-complaints-and-eat-them-for-breakfast-if-they-file-bugs and
Software writers are from the emollient responsive groves of planet
harmony.  Obviously what would work for one wouldn't work for the other.

As a software writer, I fully buy into that world view.  The trouble is
that when I go to dinner with hardware people, they seem to be awfully
nice chaps ... almost exactly like me, in fact ...

James

Re: [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Mon, 2010-05-17 at 20:34 +0100, Matthew Garrett wrote:

> int main() {
> 	int i;
> 	while (1)
> 		i++;
> 	return 0;
> }
> 

We already know it's really rare on an embedded system you would get
this bad of an app. However, if you assume that you might have this 1 in
a million app, couldn't we just have a watch dog that identifies and
kills it. It would also let the user know what happened of course.
Killing the app seems pretty reasonable to me.

Daniel

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 12:50:40PM -0700, Daniel Walker wrote:
> On Mon, 2010-05-17 at 20:34 +0100, Matthew Garrett wrote:
> 
> > int main() {
> > 	int i;
> > 	while (1)
> > 		i++;
> > 	return 0;
> > }
> > 
> 
> We already know it's really rare on an embedded system you would get
> this bad of an app. However, if you assume that you might have this 1 in
> a million app, couldn't we just have a watch dog that identifies and
> kills it. It would also let the user know what happened of course.
> Killing the app seems pretty reasonable to me.

In a world where anyone can provide this kind of app for Android users, 
I'd submit that it's going to be significantly more common than that. 
I'd expect real world cases to be at least slightly more subtle, but...

But yeah. Fundamentally the real argument here isn't about suspend 
blockers. It's about whether it's valid to require that the Linux kernel 
provide mechanisms to enable power management in the face of poorly 
written applications. If you think that that argument is invalid then 
suspend blockers are an entirely unnecessary set of layering violations. 
If you think it's valid, suspend blockers are the only currently 
proposed solution that will work in a race-free manner.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Mon, May 17, 2010 at 12:50 PM, Daniel Walker <dwalker@codeaurora.org> wrote:
> On Mon, 2010-05-17 at 20:34 +0100, Matthew Garrett wrote:
>
>> int main() {
>>       int i;
>>       while (1)
>>               i++;
>>       return 0;
>> }
>>
>
> We already know it's really rare on an embedded system you would get
> this bad of an app. However, if you assume that you might have this 1 in
> a million app, couldn't we just have a watch dog that identifies and
> kills it. It would also let the user know what happened of course.
> Killing the app seems pretty reasonable to me.

However, it is not uncommon to run into apps that poll once (or
several times) a second for IO, or have other behavior that's not
*quite* as bad as constantly running but still ends up having a
significant impact on battery life.  Polling even on "reasonable"
intervals can make a huge difference in overall battery life while not
having a very visible impact on device performance when the device is
being used interactively.  opportunistic suspend helps a lot here.

We see this in the field with real apps on real devices -- there's a
lot of not so great code out there.

Sometimes those apps are apps that, despite being badly written, do
things users want.  Thus both reducing the impact of these bad apps
and educating users and developers about the apps being bad for their
battery life is worthwhile.

Brian
_______________________________________________
linux-pm mailing list
linux-pm@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/linux-pm

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mike Chan]

On Mon, May 17, 2010 at 12:27 PM, Vitaly Wool <vitalywool@gmail.com> wrote:
> On Mon, May 17, 2010 at 6:12 PM, Kevin Hilman
> <khilman@deeprootsystems.com> wrote:
>
>>> and #2, the battery lifetime on the N770 and N800 (both of which I have)
>>> is **appalling** **bad**.
>>
>> Appalling bad compared to what?
>>
>> What's probably more interesting in terms of rough comparisons is
>> comparing similar devices with and without opportunistic suspend.  The
>> Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
>> OMAP3) and roughly the same kernel (2.6.2[89], although both are
>> heavily patched from mainline.)
>>
>> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
>> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
>>
>> I don't know of any more objective comparison of the two, but as a
>> user of both devices I can say that the active usage is basically the
>> same (around a day) and the idle use is similar as well, even though
>> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
>> own usage suggests the n900 is a bit better in idle time, but I have
>> not done any measuring or objective tests.  I'm guessing the
>> difference is probably because the Droid does not use the deepest
>> off-mode power states either in idle or suspend (IIRC) where the n900
>> does.  I suspect that if both were using off-mode and had the same
>> battery, these differences would go away.
>>

Although both are OMAP3430 and run 2.6.29 you cannot compare the N900
and Droid's perceived user battery life to one another to evaluate
opportunistic suspend. There are many factors uncounted for such as:
network reception, screen brightness (and size), button back-light,
keyboard back-light, modem stack (CDMA vs UMTS). Also the difference
in uerspace.

>> While this is not really a scientific comparison, it at least gives a
>> rough idea.  If using opportunistic suspend was adding noticably
>> better battery life, I think this would be a different discussion.
>
> Exactly. The point is, opportunistic suspend doesn't in fact add any
> value compared to dynamic PM + CPUIdle. It only produces some false
> impression that one can handle power management right without using
> dynamic PM. And this false impression is the cause for many really
> ugly designs (like, for instance, 15 minutes touchscreen inactivity
> delay before forcibly shutting down the wireless, as it's done in
> stock Android framework).
>

Opportunistic suspend is an extension to the current suspend model,
not a replacement dynamic / run-time PM. If you can replace good old
suspend then this would be a different story.

As you mention, Droid uses opportunistic suspend + dynamic pm +
cpuidle + freq. So I decided to do some measurements on a Droid using
our 2.9.32 kernel
(http://android.git.kernel.org/?p=kernel/omap.git;a=summary). For a
little better apples to apples comparison.

Droid (idle system, airplane mode, screen off, 3 min interval):
measured average current
- with opportunity suspend: 3.19mA
- without opportunistic suspend: 3.5mA

Stock userspace build, all I did was replace the kernel. We are
hitting retention on idle as well as suspend for omap (instead of full
off-mode).

Also, your point about wifi, the 15 min timeout is in the framework
and is configurable in the code and via UI, nothing to do with kernel,
opportunistic suspend or run time suspend.


-- Mike

> Thanks,
>   Vitaly
> _______________________________________________
> linux-pm mailing list
> linux-pm@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/linux-pm
>

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Monday 17 May 2010, Kevin Hilman wrote:
> tytso@mit.edu writes:
...
> At least for OMAP, I don't consider "config it out" a viable option.
> 
> We would like one kernel (in particular one PM core) to be usable for
> a broad range of devices: maemo/meego, Android, webOS, Archos, and
> whatever else we haven't seen yet.

You don't need to config it out in the first place, so this argument doesn't
really fly.  Besides, I don't get how rejecting the opportunisting suspend
feature would help us achieve this goal.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Vitaly Wool]

On Mon, May 17, 2010 at 10:07 PM, Mike Chan <mike@android.com> wrote:
> On Mon, May 17, 2010 at 12:27 PM, Vitaly Wool <vitalywool@gmail.com> wrote:
>> On Mon, May 17, 2010 at 6:12 PM, Kevin Hilman
>> <khilman@deeprootsystems.com> wrote:
>>
>>>> and #2, the battery lifetime on the N770 and N800 (both of which I have)
>>>> is **appalling** **bad**.
>>>
>>> Appalling bad compared to what?
>>>
>>> What's probably more interesting in terms of rough comparisons is
>>> comparing similar devices with and without opportunistic suspend.  The
>>> Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
>>> OMAP3) and roughly the same kernel (2.6.2[89], although both are
>>> heavily patched from mainline.)
>>>
>>> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
>>> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
>>>
>>> I don't know of any more objective comparison of the two, but as a
>>> user of both devices I can say that the active usage is basically the
>>> same (around a day) and the idle use is similar as well, even though
>>> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
>>> own usage suggests the n900 is a bit better in idle time, but I have
>>> not done any measuring or objective tests.  I'm guessing the
>>> difference is probably because the Droid does not use the deepest
>>> off-mode power states either in idle or suspend (IIRC) where the n900
>>> does.  I suspect that if both were using off-mode and had the same
>>> battery, these differences would go away.
>>>
>
> Although both are OMAP3430 and run 2.6.29 you cannot compare the N900
> and Droid's perceived user battery life to one another to evaluate
> opportunistic suspend. There are many factors uncounted for such as:
> network reception, screen brightness (and size), button back-light,
> keyboard back-light, modem stack (CDMA vs UMTS). Also the difference
> in uerspace.
>
>>> While this is not really a scientific comparison, it at least gives a
>>> rough idea.  If using opportunistic suspend was adding noticably
>>> better battery life, I think this would be a different discussion.
>>
>> Exactly. The point is, opportunistic suspend doesn't in fact add any
>> value compared to dynamic PM + CPUIdle. It only produces some false
>> impression that one can handle power management right without using
>> dynamic PM. And this false impression is the cause for many really
>> ugly designs (like, for instance, 15 minutes touchscreen inactivity
>> delay before forcibly shutting down the wireless, as it's done in
>> stock Android framework).
>>
>
> Opportunistic suspend is an extension to the current suspend model,
> not a replacement dynamic / run-time PM. If you can replace good old
> suspend then this would be a different story.

Yes, but what does it extend? What aspect it makes the current suspend
model better in?

> As you mention, Droid uses opportunistic suspend + dynamic pm +
> cpuidle + freq. So I decided to do some measurements on a Droid using
> our 2.9.32 kernel
> (http://android.git.kernel.org/?p=kernel/omap.git;a=summary). For a
> little better apples to apples comparison.
>
> Droid (idle system, airplane mode, screen off, 3 min interval):
> measured average current
> - with opportunity suspend: 3.19mA
> - without opportunistic suspend: 3.5mA
>
> Stock userspace build, all I did was replace the kernel. We are
> hitting retention on idle as well as suspend for omap (instead of full
> off-mode).

That's implementation specific. If CPUIdle implemented CPU deep sleep,
I bet you would see different figures.

> Also, your point about wifi, the 15 min timeout is in the framework
> and is configurable in the code and via UI, nothing to do with kernel,
> opportunistic suspend or run time suspend.

You don't quite get it :) This should NOT at all be timeout driven.
This should be activity driven or constraint driven which perfectly
fits into the runtime PM paradigm but is in no way cleanly implemented
within the "pure" Android PM.

Thanks,
   Vitaly
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Monday 17 May 2010, Brian Swetland wrote:
> On Mon, May 17, 2010 at 11:39 AM, Felipe Balbi <me@felipebalbi.com> wrote:
...
> > but can anyone write an app that holds a suspend_blocker ?? If so, then
> > your goal is already broken, right ? I mean, if anyone can keep a
> > suspend_blocker held forever, you'll never ever sleep, right ? While
> > with runtime, if you keep the keypad open, only the keypad and the paths
> > directly related to it (probably the i2c controller and the power domain
> > where the i2c controller sits) will be kept alive, no ?
> 
> No, you'll never suspend, which is different from never going to the
> lowest CPU power state.  On shipping Android devices we aggressively
> completely power down the CPU in idle whenever we can (based on
> latency requirements generally).  We power off peripherals whenever
> they're not in use.
> 
> This is why I've stated previously that I don't think runtime PM and
> opportunistic suspend are competitive features.

Agreed.

> Everyone who cares about minimizing power should want runtime pm or at least
> similar functionality (our drivers have always powered down peripherals when
> not in use, even while the device is open, etc, prior to the existence
> of runtime PM).

Yes.

> If your environment is such that going to full suspend will not gain
> you anything, then don't use opportunistic suspend.

Exactly.

> We find that there are savings to be had with this model in Android which is
> why we use it.  If you are going to use opportunistic suspend,
> suspend_blockers provide useful functionality.

And as I said, I regard this as a legitimate approach to power management.

Thanks,
Rafael

Re: [PATCH 0/8] Suspend block api (version 6)

[Linus WALLEIJ]

[Matthew]

> Run this (or equivalent code) on an N900 and on an Android. Measure the
> screen-off power draw on both.
> 
> int main() {
> 	int i;
> 	while (1)
> 		i++;
> 	return 0;
> }

I understand the problem such that the scheduler cannot determine whether
a certain process should be allowed to schedule or not.

It strikes me that the intuitive solution is to group all un-trusted 
processes/threads/tasks that need to be shut off even if they are
scheduling into a control group and let the idle code ignore the
processes in this group.

Or is there something too naïve about this?

Android is already using control groups to give some processes an
extra share of CPU and some less IIRC.

Yours,
Linus Walleij

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Mike Chan]

On Mon, May 17, 2010 at 1:17 PM, Vitaly Wool <vitalywool@gmail.com> wrote:
> On Mon, May 17, 2010 at 10:07 PM, Mike Chan <mike@android.com> wrote:
>> On Mon, May 17, 2010 at 12:27 PM, Vitaly Wool <vitalywool@gmail.com> wrote:
>>> On Mon, May 17, 2010 at 6:12 PM, Kevin Hilman
>>> <khilman@deeprootsystems.com> wrote:
>>>
>>>>> and #2, the battery lifetime on the N770 and N800 (both of which I have)
>>>>> is **appalling** **bad**.
>>>>
>>>> Appalling bad compared to what?
>>>>
>>>> What's probably more interesting in terms of rough comparisons is
>>>> comparing similar devices with and without opportunistic suspend.  The
>>>> Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
>>>> OMAP3) and roughly the same kernel (2.6.2[89], although both are
>>>> heavily patched from mainline.)
>>>>
>>>> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
>>>> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
>>>>
>>>> I don't know of any more objective comparison of the two, but as a
>>>> user of both devices I can say that the active usage is basically the
>>>> same (around a day) and the idle use is similar as well, even though
>>>> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
>>>> own usage suggests the n900 is a bit better in idle time, but I have
>>>> not done any measuring or objective tests.  I'm guessing the
>>>> difference is probably because the Droid does not use the deepest
>>>> off-mode power states either in idle or suspend (IIRC) where the n900
>>>> does.  I suspect that if both were using off-mode and had the same
>>>> battery, these differences would go away.
>>>>
>>
>> Although both are OMAP3430 and run 2.6.29 you cannot compare the N900
>> and Droid's perceived user battery life to one another to evaluate
>> opportunistic suspend. There are many factors uncounted for such as:
>> network reception, screen brightness (and size), button back-light,
>> keyboard back-light, modem stack (CDMA vs UMTS). Also the difference
>> in uerspace.
>>
>>>> While this is not really a scientific comparison, it at least gives a
>>>> rough idea.  If using opportunistic suspend was adding noticably
>>>> better battery life, I think this would be a different discussion.
>>>


Okay, I measured with and without using the scientific method. With a
1390mAh battery, if you set your device in airplane mode you will get
435.7 hours of standby vs 397.14 hours of standby.

Is this data sufficient for a "different discussion" now?

>>> Exactly. The point is, opportunistic suspend doesn't in fact add any
>>> value compared to dynamic PM + CPUIdle. It only produces some false
>>> impression that one can handle power management right without using
>>> dynamic PM. And this false impression is the cause for many really
>>> ugly designs (like, for instance, 15 minutes touchscreen inactivity
>>> delay before forcibly shutting down the wireless, as it's done in
>>> stock Android framework).
>>>
>>
>> Opportunistic suspend is an extension to the current suspend model,
>> not a replacement dynamic / run-time PM. If you can replace good old
>> suspend then this would be a different story.
>
> Yes, but what does it extend? What aspect it makes the current suspend
> model better in?
>

Network configuration and cell reception is a big factor here. You can
easily get +4-8mA on the original numbers I gave below, so its large
enough to skew your "perceived" power usage.

>> As you mention, Droid uses opportunistic suspend + dynamic pm +
>> cpuidle + freq. So I decided to do some measurements on a Droid using
>> our 2.9.32 kernel
>> (http://android.git.kernel.org/?p=kernel/omap.git;a=summary). For a
>> little better apples to apples comparison.
>>
>> Droid (idle system, airplane mode, screen off, 3 min interval):
>> measured average current
>> - with opportunity suspend: 3.19mA
>> - without opportunistic suspend: 3.5mA
>>
>> Stock userspace build, all I did was replace the kernel. We are
>> hitting retention on idle as well as suspend for omap (instead of full
>> off-mode).
>
> That's implementation specific. If CPUIdle implemented CPU deep sleep,
> I bet you would see different figures.
>

The important part here is not the absolute value, but how they
compare relatively. If I added off-mode support then I the averages
would both drop, but they would not be the same (key point).

I simply wanted show with real numbers that there is a difference in
opportunistic suspend and without. Instead of purely hypothesizing
that there is no power benefit from using opportunistic suspend.


>> Also, your point about wifi, the 15 min timeout is in the framework
>> and is configurable in the code and via UI, nothing to do with kernel,
>> opportunistic suspend or run time suspend.
>
> You don't quite get it :) This should NOT at all be timeout driven.
> This should be activity driven or constraint driven which perfectly
> fits into the runtime PM paradigm but is in no way cleanly implemented
> within the "pure" Android PM.
>

I admit, our timeout approach is a bit hacky, but there is method for
the madness.

In order to properly determine when you should turn off wifi for
"perfect" power management, you need to know a few things. 1) When, in
the 'future' is the user going to turn on the screen. 2) How many
network packets will be sent  to the device while the screen is off.

Some of these factors are Android specific, in particular the device
always has a TCP connection open to google servers. So switching from
WIFI -> 3G for example causes us to generate extra network traffic as
we try to establish our SSL connection to google servers. Likewise
this cost is paid when moving from 3G -> wifi.

Its *really* hard to predict the future, so we use this timeout based
off of various heuristics, usability studies and power measurements.

So we attempt to minimize excessive network io for both power reasons
and network reasons (ie: cell carriers), as well as user experience
(latency to connect / disconnect to a wifi access point).

However we are not talking about kernel power management here, we are
talking about Android policy that might be dedicated by various OEM /
carrier requirements and user experience. I wouldn't necessarily go so
far and say its flat out "wrong" what we are doing, but I do feel that
it is Android specific enough that having our own policy for this is
valid.

If you feel strongly against this, I will be more than happy to dive
into details in an android specific thread off lkml.

-- Mike

> Thanks,
>   Vitaly
>

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 10:53:36PM +0200, Linus WALLEIJ wrote:
> I understand the problem such that the scheduler cannot determine whether
> a certain process should be allowed to schedule or not.
> 
> It strikes me that the intuitive solution is to group all un-trusted 
> processes/threads/tasks that need to be shut off even if they are
> scheduling into a control group and let the idle code ignore the
> processes in this group.
> 
> Or is there something too naïve about this?

It gets most of the way there (and I spent a while playing with it), but 
the problem is that not all wakeup events get proxied through the 
trusted userspace runtime. Network packets are the easiest example - a 
packet may wake the system, but if the process it's delivered to is 
frozen then obviously things get trickier. It's also racy, in that if 
you're currently awake and receive a packet that would otherwise have 
woken the system you may go to sleep between the packet being received 
and bubbling up through the kernel to get to userspace.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Linus WALLEIJ]

[Matthew]
> On Mon, May 17, 2010 at 10:53:36PM +0200, Linus WALLEIJ wrote:
> > I understand the problem such that the scheduler cannot determine whether
> > a certain process should be allowed to schedule or not.
> >
> > It strikes me that the intuitive solution is to group all un-trusted
> > processes/threads/tasks that need to be shut off even if they are
> > scheduling into a control group and let the idle code ignore the
> > processes in this group.
> >
> > Or is there something too naïve about this?
> 
> It gets most of the way there (and I spent a while playing with it), but
> the problem is that not all wakeup events get proxied through the
> trusted userspace runtime. Network packets are the easiest example - a
> packet may wake the system, but if the process it's delivered to is
> frozen then obviously things get trickier. It's also racy, in that if
> you're currently awake and receive a packet that would otherwise have
> woken the system you may go to sleep between the packet being received
> and bubbling up through the kernel to get to userspace.

Maybe I'm misunderstanding something basic here, but when you say frozen,
are you referring to freezer control groups
Documentation/cgroups/freezer-subsystem.txt?

I was more thinking about something new, that isn't freezing anything.
The only purpose would be to group the stuff that CPUidle can ignore,
and let CPUidle ignore it, so that the system can still be idled.

Yours,
Linus Walleij

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 11:16:52PM +0200, Linus WALLEIJ wrote:
> Maybe I'm misunderstanding something basic here, but when you say frozen,
> are you referring to freezer control groups
> Documentation/cgroups/freezer-subsystem.txt?
> 
> I was more thinking about something new, that isn't freezing anything.
> The only purpose would be to group the stuff that CPUidle can ignore,
> and let CPUidle ignore it, so that the system can still be idled.

So they'd be on the runqeue but wouldn't factor into cpuidle's 
calculations of when the next wakeup should be? Ok. I think that still 
leaves you with the same problem - you're not scheduling that task, so 
how do you know to execute it when a network packet is received? I think 
you also still have the race condition.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Matthew Garrett <mjg@redhat.com> writes:

> On Mon, May 17, 2010 at 09:27:47PM +0200, Vitaly Wool wrote:
>
>> Exactly. The point is, opportunistic suspend doesn't in fact add any
>> value compared to dynamic PM + CPUIdle. It only produces some false
>> impression that one can handle power management right without using
>> dynamic PM. And this false impression is the cause for many really
>> ugly designs (like, for instance, 15 minutes touchscreen inactivity
>> delay before forcibly shutting down the wireless, as it's done in
>> stock Android framework).
>
> Run this (or equivalent code) on an N900 and on an Android. Measure the 
> screen-off power draw on both.
>
> int main() {
> 	int i;
> 	while (1)
> 		i++;
> 	return 0;
> }

This kind of rogue app will also kill my CPU performance.  For rogue
CPU-hog apps, we use _tools_ to find and fix this kind of problem:
top, renice, kill, etc. that use features of the scheduler to find
and/or solve the problem.

For rogue apps that affect PM, what is being proposed is to bypass the
root causes (timers, scheduler, activity, etc.) and just freeze the
apps.

Rogue CPU-hog apps are generally not tolerated.  The problems are
found using tools and fixed (or removed.)  Why should we do any
different for rogue power-hog apps?

Maybe the tools aren't as good for finding power-sucking apps.
Then we should be focusing on the _tools_ instead of masking the
problem.

Kevin

Re: [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Mon, 2010-05-17 at 22:24 +0100, Matthew Garrett wrote:
> On Mon, May 17, 2010 at 11:16:52PM +0200, Linus WALLEIJ wrote:
> > Maybe I'm misunderstanding something basic here, but when you say frozen,
> > are you referring to freezer control groups
> > Documentation/cgroups/freezer-subsystem.txt?
> > 
> > I was more thinking about something new, that isn't freezing anything.
> > The only purpose would be to group the stuff that CPUidle can ignore,
> > and let CPUidle ignore it, so that the system can still be idled.
> 
> So they'd be on the runqeue but wouldn't factor into cpuidle's 
> calculations of when the next wakeup should be? Ok. I think that still 
> leaves you with the same problem - you're not scheduling that task, so 
> how do you know to execute it when a network packet is received? I think 
> you also still have the race condition.

Couldn't you special case the network packet situation ? Like the idle
loop could take into account that there are packets flowing through the
networking stack that may need to get handled.

Daniel

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 02:27:52PM -0700, Kevin Hilman wrote:
> Matthew Garrett <mjg@redhat.com> writes:
> > Run this (or equivalent code) on an N900 and on an Android. Measure the 
> > screen-off power draw on both.
> >
> > int main() {
> > 	int i;
> > 	while (1)
> > 		i++;
> > 	return 0;
> > }
> 
> This kind of rogue app will also kill my CPU performance.  For rogue
> CPU-hog apps, we use _tools_ to find and fix this kind of problem:
> top, renice, kill, etc. that use features of the scheduler to find
> and/or solve the problem.

If my phone is able to avoid losing almost all of its standby time 
without me having to care about whether my bouncing cow game was written 
by a complete fool or not, that means that my phone is *better* than one 
where I have to care. Would the world be better if said fool could be 
sent to reeducation camps before being allowed to write any more 
software? Probably, but sadly that doesn't seem to be something we can 
implement through code. I'd love it if we could assume that all software 
is going to be perfect, but in the absence of that ideal world there's a 
strong incentive to develop technology that does its best to cope with 
adverse situations.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 02:29:24PM -0700, Daniel Walker wrote:
> On Mon, 2010-05-17 at 22:24 +0100, Matthew Garrett wrote:
> > > I was more thinking about something new, that isn't freezing anything.
> > > The only purpose would be to group the stuff that CPUidle can ignore,
> > > and let CPUidle ignore it, so that the system can still be idled.
> > 
> > So they'd be on the runqeue but wouldn't factor into cpuidle's 
> > calculations of when the next wakeup should be? Ok. I think that still 
> > leaves you with the same problem - you're not scheduling that task, so 
> > how do you know to execute it when a network packet is received? I think 
> > you also still have the race condition.
> 
> Couldn't you special case the network packet situation ? Like the idle
> loop could take into account that there are packets flowing through the
> networking stack that may need to get handled.

And once you've done that for every wakeup source you have something 
that looks pretty much like suspend blockers.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Mon, May 17, 2010 at 2:27 PM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:
>
> This kind of rogue app will also kill my CPU performance.  For rogue
> CPU-hog apps, we use _tools_ to find and fix this kind of problem:
> top, renice, kill, etc. that use features of the scheduler to find
> and/or solve the problem.
>
> For rogue apps that affect PM, what is being proposed is to bypass the
> root causes (timers, scheduler, activity, etc.) and just freeze the
> apps.
>
> Rogue CPU-hog apps are generally not tolerated.  The problems are
> found using tools and fixed (or removed.)  Why should we do any
> different for rogue power-hog apps?
>
> Maybe the tools aren't as good for finding power-sucking apps.
> Then we should be focusing on the _tools_ instead of masking the
> problem.

Mass market consumer devices that allow arbitrary app installation are
a fun challenge because many (most?) users are not aware of the low
level details here or why one app is bad and another is good.  They
often don't even know which app of N they installed is causing their
problems.  We're providing this information as we can to help users
understand which apps are using cpu or preventing suspend, via the
battery usage UI.  We try to provide even more detailed information to
developers so they can test and adjust for poor cpu/power usage.

We are also doing our best to behave well even in the face of poorly
written apps.

I think you need to do both -- provide tools and education, but also
make the best of things.  If I can make a device with a poorly written
app make it over a day of standby instead of 4-8 hours of standby,
that has immediate benefit to the user, for example.

Brian
_______________________________________________
linux-pm mailing list
linux-pm@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/linux-pm

Re: [PATCH 0/8] Suspend block api (version 6)

[Daniel Walker]

On Mon, 2010-05-17 at 22:35 +0100, Matthew Garrett wrote:
> On Mon, May 17, 2010 at 02:29:24PM -0700, Daniel Walker wrote:
> > On Mon, 2010-05-17 at 22:24 +0100, Matthew Garrett wrote:
> > > > I was more thinking about something new, that isn't freezing anything.
> > > > The only purpose would be to group the stuff that CPUidle can ignore,
> > > > and let CPUidle ignore it, so that the system can still be idled.
> > > 
> > > So they'd be on the runqeue but wouldn't factor into cpuidle's 
> > > calculations of when the next wakeup should be? Ok. I think that still 
> > > leaves you with the same problem - you're not scheduling that task, so 
> > > how do you know to execute it when a network packet is received? I think 
> > > you also still have the race condition.
> > 
> > Couldn't you special case the network packet situation ? Like the idle
> > loop could take into account that there are packets flowing through the
> > networking stack that may need to get handled.
> 
> And once you've done that for every wakeup source you have something 
> that looks pretty much like suspend blockers.

Well, maybe (this is theoretical after all), but it's within the
confines of our current PM model .. You basically centralize all the
decision making into the kernel, and into idle. that seems to me more
powerful than suspend blockers since they interleave parts with
userspace.

Daniel

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Mike Chan <mike@android.com> writes:

> On Mon, May 17, 2010 at 12:27 PM, Vitaly Wool <vitalywool@gmail.com> wrote:
>> On Mon, May 17, 2010 at 6:12 PM, Kevin Hilman
>> <khilman@deeprootsystems.com> wrote:
>>
>>>> and #2, the battery lifetime on the N770 and N800 (both of which I have)
>>>> is **appalling** **bad**.
>>>
>>> Appalling bad compared to what?
>>>
>>> What's probably more interesting in terms of rough comparisons is
>>> comparing similar devices with and without opportunistic suspend.  The
>>> Nokia n900 (maemo) and the Moto Droid (android) use the same SoC (TI
>>> OMAP3) and roughly the same kernel (2.6.2[89], although both are
>>> heavily patched from mainline.)
>>>
>>> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
>>> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
>>>
>>> I don't know of any more objective comparison of the two, but as a
>>> user of both devices I can say that the active usage is basically the
>>> same (around a day) and the idle use is similar as well, even though
>>> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)  My
>>> own usage suggests the n900 is a bit better in idle time, but I have
>>> not done any measuring or objective tests.  I'm guessing the
>>> difference is probably because the Droid does not use the deepest
>>> off-mode power states either in idle or suspend (IIRC) where the n900
>>> does.  I suspect that if both were using off-mode and had the same
>>> battery, these differences would go away.
>>>
>
> Although both are OMAP3430 and run 2.6.29 you cannot compare the N900
> and Droid's perceived user battery life to one another to evaluate
> opportunistic suspend. There are many factors uncounted for such as:
> network reception, screen brightness (and size), button back-light,
> keyboard back-light, modem stack (CDMA vs UMTS). Also the difference
> in uerspace.

Agreed.  I was reluctant to even make the comparison for all those
reasons, but with the lack of real numbers, it was all I had to show a
very rough subjective guess, and at least show that with and without
opportunistic suspend, you're in the same ballpark.

>>> While this is not really a scientific comparison, it at least gives a
>>> rough idea.  If using opportunistic suspend was adding noticably
>>> better battery life, I think this would be a different discussion.
>>
>> Exactly. The point is, opportunistic suspend doesn't in fact add any
>> value compared to dynamic PM + CPUIdle. It only produces some false
>> impression that one can handle power management right without using
>> dynamic PM. And this false impression is the cause for many really
>> ugly designs (like, for instance, 15 minutes touchscreen inactivity
>> delay before forcibly shutting down the wireless, as it's done in
>> stock Android framework).
>>
>
> Opportunistic suspend is an extension to the current suspend model,
> not a replacement dynamic / run-time PM. If you can replace good old
> suspend then this would be a different story.
>
> As you mention, Droid uses opportunistic suspend + dynamic pm +
> cpuidle + freq. So I decided to do some measurements on a Droid using
> our 2.9.32 kernel
> (http://android.git.kernel.org/?p=kernel/omap.git;a=summary). For a
> little better apples to apples comparison.

Excellent.  Thanks for some real numbers.

(btw, sure would be nice to have the basic droid board files in mainline. ;)

> Droid (idle system, airplane mode, screen off, 3 min interval):
> measured average current
> - with opportunity suspend: 3.19mA
> - without opportunistic suspend: 3.5mA
>
> Stock userspace build, all I did was replace the kernel. We are
> hitting retention on idle as well as suspend for omap (instead of full
> off-mode).

[at risk of stating the obvious] Since both approaches hit the same
power states, if the system was truly idle, you'd expect the numbers
to be the same, right?  So what the difference shows is that the
system is not fully idle, IOW userspace and/or kernel wakeups are
cusing the difference.

It might also be instructive to see these numbers with a "noop"
userspace, like just booting into busybox init=/bin/sh (or the android
equivalent.)  That would show how much of that difference is due to
kernel idleness (or lack thereof.)

Even still, to me this all boils down to the lack of definition of the
problem and clear description of the solution

The fundamental problem is one of idleness.  What we want is the
system to be idle in order to hit the lowest power states.  We can't
get there because the system is not truly idle.

So, there are basically two solutions:

1) find and fix the problem spots so we can be idle more often
2) add a new definition of idle so we can be idle more often

Opporuntistic suspend takes the second approach where the new
definition of idle is "no suspend blockers held."

Of course, I clearly prefer the former, but it's also becoming clear
that since I'm the only one still whining about it, I must be too much
of an idealist to keep hoping for it.

Kevin



--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/8] Suspend block api (version 6)

[Kevin Hilman]

Matthew Garrett <mjg@redhat.com> writes:

> On Mon, May 17, 2010 at 02:27:52PM -0700, Kevin Hilman wrote:
>> Matthew Garrett <mjg@redhat.com> writes:
>> > Run this (or equivalent code) on an N900 and on an Android. Measure the 
>> > screen-off power draw on both.
>> >
>> > int main() {
>> > 	int i;
>> > 	while (1)
>> > 		i++;
>> > 	return 0;
>> > }
>> 
>> This kind of rogue app will also kill my CPU performance.  For rogue
>> CPU-hog apps, we use _tools_ to find and fix this kind of problem:
>> top, renice, kill, etc. that use features of the scheduler to find
>> and/or solve the problem.
>
> If my phone is able to avoid losing almost all of its standby time 
> without me having to care about whether my bouncing cow game was written 
> by a complete fool or not, that means that my phone is *better* than one 
> where I have to care. 

Someday I hope to see this bouncing cow app I keep hearing about.  I
must be pretty spectacular since can't help talking about it.  ;)

> Would the world be better if said fool could be 
> sent to reeducation camps before being allowed to write any more 
> software? Probably, but sadly that doesn't seem to be something we can 
> implement through code. I'd love it if we could assume that all software 
> is going to be perfect, but in the absence of that ideal world there's a 
> strong incentive to develop technology that does its best to cope with 
> adverse situations.

I completely agree.  

The only thing being debated is the implementation of the coping
mechanism.

In particular, why the coping mechanism for PM "badness" has to be a
large sledgehammer where other badness (CPU, IO, etc.) has been dealt
with using infrastructure like timers, scheduling, throttling, etc.

My shrink would say that the sledgehammer approach to badness might
work for the short term, but eventually that badness is going to seep
out in other, unpredictable places.

My shrink would also probably tell me that since I'm seem to be the
last one left who seems to care, it's probably time to throw in the
towel.

Kevin

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Brian Swetland]

On Mon, May 17, 2010 at 3:55 PM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:
>
> The fundamental problem is one of idleness.  What we want is the
> system to be idle in order to hit the lowest power states.  We can't
> get there because the system is not truly idle.
>
> So, there are basically two solutions:
>
> 1) find and fix the problem spots so we can be idle more often
> 2) add a new definition of idle so we can be idle more often
>
> Opporuntistic suspend takes the second approach where the new
> definition of idle is "no suspend blockers held."
>
> Of course, I clearly prefer the former, but it's also becoming clear
> that since I'm the only one still whining about it, I must be too much
> of an idealist to keep hoping for it.

I'd love to see the former work, but it is not something that I think
is going to be fixed rapidly.  It potentially involves many different
subsystems, and still requires some need for managing arbitrary
userspace agents which may have non-ideal behaviors (if we're going to
solve the problem for general purpose devices that can run arbitrary
user-installed software).  Incremental improvements are great though
(for example, that we can now be in lowest power idle for periods
greater than 2 seconds due to the change in .32).

Even when operating in opportunistic suspend, it is still advantageous
for idle to be as idle as possible and we should keep working toward
that goal.  If we get to the point where the power difference between
suspend-in-idle and opportunistic suspend is zero, then we no longer
need the latter.

I don't think anybody on the Google/Android side is arguing that we
*shouldn't* pursue dynamic pm and overall making idle more idle more
of the time.  We're just saying that here and now we are not at the
ideal and opportunistic suspend gains us real power savings and is
useful.

Brian

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hello Rafael,

On Sat, 15 May 2010, Rafael J. Wysocki wrote:

> And really, the only semi-technical argument against the opportunistic suspend
> feature I've seen so far in this thread is that it _may_ _be_ possible to
> achieve the same goal in a different way.  If I don't see anything more serious
> than that, I will take the patchset and push it to Linus (patches [1-6/8] from
> version 7 to be precise, as soon as the changelogs are improved as
> requested).

The Android opportunistic suspend series still has problems, even if 
opportunistic suspend mode is not enabled.  These problems can be fixed, 
but it's too late to fix them for this merge window.  There shouldn't be a 
rush to merge these patches, given the current amount of discussion.

Details:

A. The series still has problems, even if the opportunistic suspend
   mode is not enabled:

   1. Many suspend_block() calls will need to be added throughout the
      kernel and drivers[1], beyond just those in the current patch 
      series.  Android systems will not work without  these calls.  
      The primary stated goal of adding suspend-block calls is not to
      add a new feature, but to restore an approximation of the
      original, expected behavior of the kernel timer and scheduler
      subsystems: to keep running as long as there is work to be done.
      This approach -- requiring patching of the kernel to restore
      expected behavior -- is inferior to an approach that preserves
      the timer and scheduler subsystems' behavior and only patches
      areas where the kernel doesn't have enough information to make
      the right decision[2].

   2. suspend_block() calls will be difficult for driver and subsystem
      maintainers to test.  This is because the calls contain hidden
      dependencies to specific Android userspace setups and unrelated
      kernel code[3].

B. These problems can be fixed:

   1. Patch 1's opportunistic suspend governor can be converted into
      an "opportunistic system low power governor."  In other words,
      when the governor decides to put the system into a low-power
      state, the way to do it should be up to the architecture, system
      integrator, or user -- not only "queue_work(pm_wq,
      &suspend_work);"
   
   2. The opportunistic system low power governor can then be extended
      to handle multiple system low-power states ("S-states" in ACPI
      terms)[4], rather than just one low-power state; similar to the
      way that CPUIdle selects C-states.
   
   3. The system low power governor can then be entered before the
      last active CPU in the system enters WFI/HLT (i.e., immediately
      before CPUIdle calls state->enter() [5]).  At this point, the
      governor would respect timers and the scheduler, and most
      suspend-block calls can be removed from the kernel.

   4. The process-freezing behavior of the opportunistic suspend
      patches can be implemented by improving the Linux timer and
      scheduler subsystems.  If freezer cgroups won't do, then based
      on a quick glance last week, here might be some reasonable
      approaches for emulating the Android behavior:

      A. To prevent untrusted process timers from waking the system,
         the processes can be run in a cgroup that calls
         del_timer_sync() on all process timers upon receipt of some
         trigger.  When the system returns from the system low-power
         state and some trigger event happens, add_timer() can be
         called to add all the cgroup's timers back.

      B. To prevent untrusted processes from hogging CPUs, the
         processes can be run in a cgroup that puts all threads in
         TASK_RUNNING processes into TASK_INTERRUPTIBLE state and
         calls schedule() [6], upon receipt of some trigger.  This
         should allow processes to be awakened by an external system
         event, e.g., network packet received.  When the system
         returns from the system low-power state and some trigger
         event happens, those process threads can be placed back into
         TASK_RUNNING state and rescheduled.

      The trigger events could be as simple as writing '1' or '0' into
      some cgroup or sysfs control file.

   5. At this point there would likely be a few remaining
      suspend-block users that use suspend_block() as an ersatz PM
      constraint API.  These should be replaced by a declarative PM
      constraint API a la [7] or [8].  Architectures and system
      integrators must be allowed to control how these PM constraints
      are implemented, because different chipsets, boards, and
      use-cases may have different power management feature
      implementations.

   Fixing the above issues should remove any need to add a
   suspend-block API.  It should handle the process-freezing use case.
   Android can still enter full system suspend in their system idle
   governor, but it should no longer need to.  Since this approach
   will honor "trusted" timers and processes, it won't break code that
   relies on longstanding kernel scheduling assumptions.

C. There shouldn't be a rush to merge the existing Android series:

   1. The lack of suspend-block support should not prevent existing
      drivers from being upstreamed without suspend-block calls.
      Non-Android users will be using the drivers without
      suspend-blocks, so the drivers will need to work without
      suspend-blocks anyway.  Until a fixed version of the series is
      submitted, Android can just carry the suspend-block patches in
      their own tree: the suspend-block patch rebase load from one
      Linux kernel release to the next should be light.  Many of the
      code sites where suspend-block calls are currently present in
      Android's MSM kernel[9] are unlikely to change, so git should be
      able to merge those automatically.

   2. Once these patches are merged, it will be difficult to remove
      the suspend-block approach from the kernel[10].  It touches a lot of
      kernel code.  And the power management constraint that it
      specifies is so general ("block system suspend"), that, for the
      subset of suspend_block() use-cases that actually implement a
      driver PM constraint, it will be difficult to figure out what
      these constraints were originally intended to be.

   3. There is no imminent danger if these patches are not merged.
      These are not security patches, or patches that fix some
      existing code that could damage hardware.  There is no liability
      motivation to fast-track these patches, particularly given the
      current high level of discussion.

Some of the features that these opportunistic suspend patches implement 
are desirable, such as process freezing and a system low power governor 
independent from CPUIdle.  However, there are significant problems with 
the way that these opportunistic suspend patches are designed, since they 
break existing kernel semantics and then touch lots of other code to work 
around the breakage.

The right approach should be to implement these features in a way that 
does not require patching all over the kernel, does not break existing 
timer and scheduler behavior, and will not impact bottom-up power 
management approaches.


regards,

- Paul
     

1. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:27:56 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18658

2. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/plat-omap/include/plat/omap-pm.h;h=3ee41d7114929d771cadbb9f02191fd16c5b5abe;hb=ba2e1c5f25a99dec3873745031ad23ce3fd79bff

3. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:27:56 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18658

4. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu, 13
   May 2010 13:01:33 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18592

5. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=drivers/cpuidle/cpuidle.c;h=12fdd3987a36c2b09ab65b9d598f276e51fbf5b7;hb=ba2e1c5f25a99dec3873745031ad23ce3fd79bff#l88

6. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/signal.c;h=dbd7fe073c558dbc57080e7d0d9344b2cf47b2dd;hb=ba2e1c5f25a99dec3873745031ad23ce3fd79bff#l2696

7. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/arm/plat-omap/include/plat/omap-pm.h;h=3ee41d7114929d771cadbb9f02191fd16c5b5abe;hb=ba2e1c5f25a99dec3873745031ad23ce3fd79bff

8. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/pm_qos_params.c;h=3db49b9ca374ded32bf224f266e904a8006e61a8;hb=ba2e1c5f25a99dec3873745031ad23ce3fd79bff

9. Paul Walmsley E-mail to the linux-pm mailing list, dated Fri, 14
   May 2010 00:13:50 -0600:
   http://permalink.gmane.org/gmane.linux.power-management.general/18657

10. Paul Walmsley E-mail to the linux-pm mailing list, dated Wed, 12
    May 2010 21:35:30 -0600: http://lkml.org/lkml/2010/5/12/528

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Mon, May 17, 2010 at 07:39:44PM -0600, Paul Walmsley wrote:

>       A. To prevent untrusted process timers from waking the system,
>          the processes can be run in a cgroup that calls
>          del_timer_sync() on all process timers upon receipt of some
>          trigger.  When the system returns from the system low-power
>          state and some trigger event happens, add_timer() can be
>          called to add all the cgroup's timers back.

This doesn't work, for the reasons previously discussed.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1404 bytes --]

Hello Arve,

On Fri, 14 May 2010, Arve Hjønnevåg wrote:

> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
> >
> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
> >
> >> No, suspend blockers are mostly used to ensure wakeup events are not
> >> ignored, and to ensure tasks triggered by these wakeup events
> >> complete.
> >
> > Standard Linux systems don't need these,
> 
> If you don't want to lose wakeup events they do.  Standard Linux systems 
> support suspend, but since they usually don't have a lot of wakeup 
> events you don't run into a lot of problems.

Sorry, I don't follow.  What causes wakeup events to be lost?  Is it the 
current opportunistic suspend governor?  On OMAP Linux systems, as far as 
I know, we don't lose any wakeup events.

> > because the scheduler just keeps the system running as long as there 
> > is work to be done.
> 
> That is only true if you never use suspend.

If, instead of the current Android opportunistic suspend governor, the 
system entered suspend from pm_idle(), wouldn't that keep the system 
running as long as there is work to done?

As far as I can see, it's the current Android opportunistic suspend 
governor design in patch 1 that causes the system to enter suspend even 
when there is work to be done, since it will try to suspend even when the 
system is out of the idle loop.


- Paul

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH 0/8] Suspend block api (version 6)

[tytso]

On Mon, May 17, 2010 at 02:36:26PM -0700, Brian Swetland wrote:
> 
> Mass market consumer devices that allow arbitrary app installation are
> a fun challenge because many (most?) users are not aware of the low
> level details here or why one app is bad and another is good.  They
> often don't even know which app of N they installed is causing their
> problems.

... and this was definitely a problem with Maemo devices that I played
with (N770, N800, et. al.).  You'd install 3 or 4 applications, then
start running without the power adapter permanently attached, and hey
surprise!  The battery would be drained very quickly even though the
device was "idle".  Figuring out which application was at fault was
basically done via brute force, which was not fun....

					- Ted

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Mon, May 17, 2010 at 7:26 PM, Paul Walmsley <paul@pwsan.com> wrote:
> Hello Arve,
>
> On Fri, 14 May 2010, Arve Hjønnevåg wrote:
>
>> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> >
>> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
>> >
>> >> No, suspend blockers are mostly used to ensure wakeup events are not
>> >> ignored, and to ensure tasks triggered by these wakeup events
>> >> complete.
>> >
>> > Standard Linux systems don't need these,
>>
>> If you don't want to lose wakeup events they do.  Standard Linux systems
>> support suspend, but since they usually don't have a lot of wakeup
>> events you don't run into a lot of problems.
>
> Sorry, I don't follow.  What causes wakeup events to be lost?  Is it the
> current opportunistic suspend governor?  On OMAP Linux systems, as far as
> I know, we don't lose any wakeup events.
>
Have you used suspend?

>> > because the scheduler just keeps the system running as long as there
>> > is work to be done.
>>
>> That is only true if you never use suspend.
>
> If, instead of the current Android opportunistic suspend governor, the
> system entered suspend from pm_idle(), wouldn't that keep the system
> running as long as there is work to done?
>
How do you know if the work being done while suspending is work that
is needed to suspend or work that should abort suspend? When should
the system wake up?

> As far as I can see, it's the current Android opportunistic suspend
> governor design in patch 1 that causes the system to enter suspend even
> when there is work to be done, since it will try to suspend even when the
> system is out of the idle loop.
>

It does not matter how you enter suspend. Without opportunistic
suspend, once you tell the kernel that you want to suspend, you cannot
abort. If a wakeup event occurs at this point, it will probably not be
processed until the system wakes up for another wakeup event.

-- 
Arve Hjønnevåg

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Mon, May 17, 2010 at 09:49:35PM +0200, ext James Bottomley wrote:
>Right, because Firmware writers are from the rugged unresponsive uplands
>of planet
>ignore-user-complaints-and-eat-them-for-breakfast-if-they-file-bugs and
>Software writers are from the emollient responsive groves of planet
>harmony.  Obviously what would work for one wouldn't work for the other.
>
>As a software writer, I fully buy into that world view.  The trouble is
>that when I go to dinner with hardware people, they seem to be awfully
>nice chaps ... almost exactly like me, in fact ...

what does this add to suspend_blockers discussion ?

-- 
balbi

DefectiveByDesign.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Henrik Rydberg]

Arve Hjønnevåg wrote:
> It does not matter how you enter suspend. Without opportunistic
> suspend, once you tell the kernel that you want to suspend, you cannot
> abort. If a wakeup event occurs at this point, it will probably not be
> processed until the system wakes up for another wakeup event.
> 

Since it seems to be at the very core of the argument, how long time does it
take to suspend your system? Is it something that could be improved upon?

Henrik

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[James Bottomley]

On Tue, 2010-05-18 at 09:40 +0300, Felipe Balbi wrote:
> On Mon, May 17, 2010 at 09:49:35PM +0200, ext James Bottomley wrote:
> >Right, because Firmware writers are from the rugged unresponsive uplands
> >of planet
> >ignore-user-complaints-and-eat-them-for-breakfast-if-they-file-bugs and
> >Software writers are from the emollient responsive groves of planet
> >harmony.  Obviously what would work for one wouldn't work for the other.
> >
> >As a software writer, I fully buy into that world view.  The trouble is
> >that when I go to dinner with hardware people, they seem to be awfully
> >nice chaps ... almost exactly like me, in fact ...
> 
> what does this add to suspend_blockers discussion ?

Sorry I was evidently being too subtle.

The point is that if, as you acknowledge, that you can't train firmware
engineers to be responsive, there's no reason to think you can train
software engineers in the same quality ... they're very similar people.

The corollary is that real world systems have to operate in the face of
misbehaving hardware *and* software.

James

Re: [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Tuesday 18 May 2010, Henrik Rydberg wrote:
> Arve Hjønnevåg wrote:
> > It does not matter how you enter suspend. Without opportunistic
> > suspend, once you tell the kernel that you want to suspend, you cannot
> > abort. If a wakeup event occurs at this point, it will probably not be
> > processed until the system wakes up for another wakeup event.
> > 
> 
> Since it seems to be at the very core of the argument, how long time does it
> take to suspend your system? Is it something that could be improved upon?

It's not a matter of time.  It's a matter of lost keystrokes and such.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Tue, May 18, 2010 at 03:59:48PM +0200, ext James Bottomley wrote:
>On Tue, 2010-05-18 at 09:40 +0300, Felipe Balbi wrote:
>> On Mon, May 17, 2010 at 09:49:35PM +0200, ext James Bottomley wrote:
>> >Right, because Firmware writers are from the rugged unresponsive uplands
>> >of planet
>> >ignore-user-complaints-and-eat-them-for-breakfast-if-they-file-bugs and
>> >Software writers are from the emollient responsive groves of planet
>> >harmony.  Obviously what would work for one wouldn't work for the other.
>> >
>> >As a software writer, I fully buy into that world view.  The trouble is
>> >that when I go to dinner with hardware people, they seem to be awfully
>> >nice chaps ... almost exactly like me, in fact ...
>>
>> what does this add to suspend_blockers discussion ?
>
>Sorry I was evidently being too subtle.
>
>The point is that if, as you acknowledge, that you can't train firmware
>engineers to be responsive, there's no reason to think you can train
>software engineers in the same quality ... they're very similar people.

I wouldn't say it's up to the engineer himself, it's more related to how 
the company that person works for deal with such things.

>The corollary is that real world systems have to operate in the face of
>misbehaving hardware *and* software.

I still think the kernel shouldn't deal with broken applications and we 
shouldn't try to fix them in kernel space. We can, of course, try to 
find them and have all sorts of bells and whistles shouting 'process 
%s is preventing CPU from sleeping for %llu nanoseconds' or something 
like that.

-- 
balbi

DefectiveByDesign.org

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hi Matthew,

On Tue, 18 May 2010, Matthew Garrett wrote:

> On Mon, May 17, 2010 at 07:39:44PM -0600, Paul Walmsley wrote:
> 
> >       A. To prevent untrusted process timers from waking the system,
> >          the processes can be run in a cgroup that calls
> >          del_timer_sync() on all process timers upon receipt of some
> >          trigger.  When the system returns from the system low-power
> >          state and some trigger event happens, add_timer() can be
> >          called to add all the cgroup's timers back.
> 
> This doesn't work, for the reasons previously discussed.

Could you please elaborate, or provide a link to an E-mail in the archives 
that would explain why?

thanks,

- Paul

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Wed, May 19, 2010 at 10:17:01AM -0600, Paul Walmsley wrote:

> Could you please elaborate, or provide a link to an E-mail in the archives 
> that would explain why?

What would otherwise be a wakeup event may be delivered between you 
deciding to stop tasks and you actually stopping tasks. How do you know 
to restart tasks at that point?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Rafael J. Wysocki]

On Wednesday 19 May 2010, Felipe Balbi wrote:
> On Tue, May 18, 2010 at 03:59:48PM +0200, ext James Bottomley wrote:
> >On Tue, 2010-05-18 at 09:40 +0300, Felipe Balbi wrote:
> >> On Mon, May 17, 2010 at 09:49:35PM +0200, ext James Bottomley wrote:
> >> >Right, because Firmware writers are from the rugged unresponsive uplands
> >> >of planet
> >> >ignore-user-complaints-and-eat-them-for-breakfast-if-they-file-bugs and
> >> >Software writers are from the emollient responsive groves of planet
> >> >harmony.  Obviously what would work for one wouldn't work for the other.
> >> >
> >> >As a software writer, I fully buy into that world view.  The trouble is
> >> >that when I go to dinner with hardware people, they seem to be awfully
> >> >nice chaps ... almost exactly like me, in fact ...
> >>
> >> what does this add to suspend_blockers discussion ?
> >
> >Sorry I was evidently being too subtle.
> >
> >The point is that if, as you acknowledge, that you can't train firmware
> >engineers to be responsive, there's no reason to think you can train
> >software engineers in the same quality ... they're very similar people.
> 
> I wouldn't say it's up to the engineer himself, it's more related to how 
> the company that person works for deal with such things.
> 
> >The corollary is that real world systems have to operate in the face of
> >misbehaving hardware *and* software.
> 
> I still think the kernel shouldn't deal with broken applications and we 
> shouldn't try to fix them in kernel space. We can, of course, try to 
> find them and have all sorts of bells and whistles shouting 'process 
> %s is preventing CPU from sleeping for %llu nanoseconds' or something 
> like that.

Please note that this approach is not too practical for vendors who ship
systems like cell phones to the general public.

Rafael

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Wed, May 19, 2010 at 10:42:55PM +0200, ext Rafael J. Wysocki wrote:
>Please note that this approach is not too practical for vendors who ship
>systems like cell phones to the general public.

yeah, tell me about it :-p

during development on MeeGo devices we try to tackle down as much as 
possible the use_time offenders and start by filing bugs to those apps, 
instead of fixing their issues in kernel space.

if suspend_blockers could at least be transparent to applications, then 
it wouldn't be the best scenario but at least applications wouldn't have 
to be specially written to support that. And like I said, if anyone can 
hold a suspend_blocker forever the idea of "improving use_time" is easy 
to break, but then someone replied "anyone holding a suspend_blocker 
will show up in UI", and again I say you don't need suspend_blockers to 
have a fancy UI showing which processes are waking up the processor. 
Powertop already gathers that information, you just need to make a fancy 
UI around it.

-- 
balbi

DefectiveByDesign.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Florian Mickler]

On Wed, 19 May 2010 09:59:34 +0300
Felipe Balbi <felipe.balbi@nokia.com> wrote:


> >The corollary is that real world systems have to operate in the face of
> >misbehaving hardware *and* software.
> 
> I still think the kernel shouldn't deal with broken applications and we 
> shouldn't try to fix them in kernel space. We can, of course, try to 
> find them and have all sorts of bells and whistles shouting 'process 
> %s is preventing CPU from sleeping for %llu nanoseconds' or something 
> like that.
> 

But with that, you still shift the burden of exchanging that app with
an feature-equivalent non-broken version to the user. 
which is not user friendly and not necessary if you have a "smart"
enough kernel.

Cheers,
Flo 

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Thu, May 20, 2010 at 07:15:28AM +0200, Florian Mickler wrote:
> But with that, you still shift the burden of exchanging that app with
> an feature-equivalent non-broken version to the user. 
> which is not user friendly and not necessary if you have a "smart"
> enough kernel.

and _without that_, you shift the burden of having a working power
management completely into the kernel. Forcing the kernel to deal with
completely broken apps. What will happen is that apps developers won't
boder thinking about power consumption since the kernel is "smart"
enough to "fix" their mess.

To me that's much bigger burden to the kernel than the other option is
to apps.

-- 
balbi

Re: [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Thu, May 20, 2010 at 10:57:40AM +0200, ext Felipe Balbi wrote:
>boder thinking about power consumption since the kernel is "smart"
  ^I mean bother

-- 
balbi

DefectiveByDesign.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Florian Mickler]

On Thu, 20 May 2010 11:57:40 +0300
Felipe Balbi <me@felipebalbi.com> wrote:

> On Thu, May 20, 2010 at 07:15:28AM +0200, Florian Mickler wrote:
> > But with that, you still shift the burden of exchanging that app with
> > an feature-equivalent non-broken version to the user. 
> > which is not user friendly and not necessary if you have a "smart"
> > enough kernel.
> 
> and _without that_, you shift the burden of having a working power
> management completely into the kernel. Forcing the kernel to deal with
> completely broken apps. What will happen is that apps developers won't
> boder thinking about power consumption since the kernel is "smart"
> enough to "fix" their mess.
> 
> To me that's much bigger burden to the kernel than the other option is
> to apps.
> 

You said that already. For me this sounds like you want to take the
users hostage in order to get nice (poweraware) apps.

Robust system design can take crap and perform well. Users will most of
the time prefer a robust system over a nicely designed system. (Just
think of the ak-47)

I think we just have to agree to disagree here? 

Cheers,
Flo

p.s.: don't take me seriously, i'm just a user

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

Hi,

On Thu, May 20, 2010 at 12:05:19PM +0200, ext Florian Mickler wrote:
>You said that already. For me this sounds like you want to take the
>users hostage in order to get nice (poweraware) apps.

not the users, no. The app developers. They should know what bad 
applications can cause to a nicely done system.

>Robust system design can take crap and perform well. Users will most of
>the time prefer a robust system over a nicely designed system. (Just
>think of the ak-47)

(hope you're talking about the gun :-p)
put some bad bullets on ak-47 and see if it behaves well, a really 
crappy trigger will also make it fail. How robust can a system be with 
badly chosen components ?

>I think we just have to agree to disagree here?

I think so.

-- 
balbi

DefectiveByDesign.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Vladimir Pantelic]

Felipe Balbi wrote:
> Hi,
>
> On Wed, May 19, 2010 at 10:42:55PM +0200, ext Rafael J. Wysocki wrote:
>>Please note that this approach is not too practical for vendors who ship
>>systems like cell phones to the general public.
>
> yeah, tell me about it :-p
>
> during development on MeeGo devices we try to tackle down as much as
> possible the use_time offenders and start by filing bugs to those apps,
> instead of fixing their issues in kernel space.

And you will continue doing that once the Meego app store has 100k apps?

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Thu, May 20, 2010 at 01:27:25PM +0200, ext Vladimir Pantelic wrote:
>Felipe Balbi wrote:
>> Hi,
>>
>> On Wed, May 19, 2010 at 10:42:55PM +0200, ext Rafael J. Wysocki wrote:
>>>Please note that this approach is not too practical for vendors who ship
>>>systems like cell phones to the general public.
>>
>> yeah, tell me about it :-p
>>
>> during development on MeeGo devices we try to tackle down as much as
>> possible the use_time offenders and start by filing bugs to those apps,
>> instead of fixing their issues in kernel space.
>
>And you will continue doing that once the Meego app store has 100k apps?

I'm not here speaking for MeeGo. I'm presenting my own feelings and my 
own opinion regarding this issue. Don't bring the company into the game, 
please.

-- 
balbi

DefectiveByDesign.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[David Brownell]

> > during development on MeeGo devices we try to tackle
> down as much as
> > possible the use_time offenders and start by filing
> bugs to those apps,
> > instead of fixing their issues in kernel space.

Some apps do abuse kernel mechanisms, and whether the bug is in the
app or that kernel mechanism can be a judgement call.  I'd expect to
see some fixes be appropriately in-kernel; maybe not many though.

When reading about this suspend block stuff, does anyone else hear
eachoes of APM?  It's been ages since that was used, but ISTR it also
leveraged handshaking between kernel and userspace.  Are there lessons
to be applied from there to this discussion?

On first principles, I don't see anything wrong with acknowledging that
the kernel doesn't have a "whole system PM view" and thus its PM knowledge could usefully be augmented by information from userspace (applications), possibly on request.

Just what that broad PM view consists of gets kind of system-specific.  For OMAP hardware, with smart device-level power reduction active almost
all the time, it may look different from an ACPI laptop where the BIOS
is biased towards saving device power primarily in some suspend state(s) ... or some other hardware platform without much hardware or BIOS assist, where the main PM mechanisms involve software detection/instigation of hardware idleness (and potentially "off-ness").




> And you will continue doing that once the Meego app store
> has 100k apps?

I may have overlooked it, in one of the 100K messsages in my
mailbox about versions of suspend block/etc patches ...

But surely NOBODY is actually contending that broken aps NOT get fixed??

It's clear to me that tools are needed to identify power hogs; powertop can't be the extent of such tools.  (ISTR it doesn't monitor display power usage, for one thing; maybe newer versions do so.)  Once such hogs get identified they will need to get fixed.  Any other proposal seems broken to me...

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Felipe Balbi]

On Thu, May 20, 2010 at 10:40:17AM -0700, David Brownell wrote:
> Some apps do abuse kernel mechanisms, and whether the bug is in the
> app or that kernel mechanism can be a judgement call.  I'd expect to

hey come on, there's no judgement call for an app polling every second
to check battery status or some other status that doesn't change that
frequently.

> I may have overlooked it, in one of the 100K messsages in my mailbox
> about versions of suspend block/etc patches ...
> 
> But surely NOBODY is actually contending that broken aps NOT get
> fixed??
> 
> It's clear to me that tools are needed to identify power hogs;
> powertop can't be the extent of such tools.  (ISTR it doesn't monitor
> display power usage, for one thing; maybe newer versions do so.)  Once
> such hogs get identified they will need to get fixed.  Any other
> proposal seems broken to me...

that's my feeling too. I don't see any needs for suspend blockers in any
real system. I acknowledge we need tools probing power consumption to be
shipped to production device, that's a good idea, but forcing apps to
modify just to have that UI fill up some treeview, I think it's a bit
too much.

-- 
balbi

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[David Brownell]

> -0700, David Brownell wrote:
> > Some apps do abuse kernel mechanisms, and whether the
> bug is in the
> > app or that kernel mechanism can be a judgement
> call.  I'd expect to
> 
> hey come on, there's no judgement call for an app polling
> every second
> to check battery status or some other status that doesn't
> change that frequently.

Or something as broken as a non-terminating CPU loop ...


Of course not.  But if there's a kernel mechanism exposed to
userspace which really sucks down power ... either it's done
correctly and the problem is userspace abusing it, or it's done
wrong so that it's not possible to be used correctly.  There are
some things that just cost power, and that cost can't be escaped.

Re: [PATCH 0/8] Suspend block api (version 6)

[David Brownell]

> > > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
> > >
> > >> No, suspend blockers are mostly used to
> ensure wakeup events are not
> > >> ignored, and to ensure tasks triggered by
> these wakeup events
> > >> complete.
> > >
> > > Standard Linux systems don't need these,
> > 
> > If you don't want to lose wakeup events they do. 
> Standard Linux systems 
> > support suspend, but since they usually don't have a
> lot of wakeup 
> > events you don't run into a lot of problems.

Yes and no...  stick a bunch of USB hubs and devices
on the system, and you may have a lot of wake events.
Keyboard, mouse, network adapter, and so on.

 Way back when I made USB remote wakeup work, ISTR running
into a version of the issue you're raising.  Briefly, the
wake event could be queued in hardware, but there were also
various unavoidable races .... with ambiguity about just when
particular events should be recognized.  If the system entered
sleep before the wake event(s) triggered, or the relevant devices
suspended first, then things were clear; otherwise, not.  On
first blush, I'd expect every hardware wake mechanism would have
the same issues; the fact that USB wakes are mediated via khubd
is not a huge difference.

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2648 bytes --]

Hello Matthew,

On Wed, 19 May 2010, Matthew Garrett wrote:

> On Wed, May 19, 2010 at 10:17:01AM -0600, Paul Walmsley wrote:
> 
> > Could you please elaborate, or provide a link to an E-mail in the archives 
> > that would explain why?
> 
> What would otherwise be a wakeup event may be delivered between you 
> deciding to stop tasks and you actually stopping tasks. How do you know 
> to restart tasks at that point?

The original message you sent[1] quoted this part[2] of the proposal:

----
       A. To prevent untrusted process timers from waking the system,
          the processes can be run in a cgroup that calls
          del_timer_sync() on all process timers upon receipt of some
          trigger.  When the system returns from the system low-power
          state and some trigger event happens, add_timer() can be
          called to add all the cgroup's timers back.
----

That isn't intended to address the task restarting problem.  The task 
restarting problem is intended to be addressed by this part[3]:

----
      B. To prevent untrusted processes from hogging CPUs, the
         processes can be run in a cgroup that puts all threads in
         TASK_RUNNING processes into TASK_INTERRUPTIBLE state and
         calls schedule() [6], upon receipt of some trigger.  This
         should allow processes to be awakened by an external system
         event, e.g., network packet received.  When the system
         returns from the system low-power state and some trigger
         event happens, those process threads can be placed back into
         TASK_RUNNING state and rescheduled.
----

It sounds like Android doesn't even need to use full system suspend if 
some proposal like (A) and (B) are implemented[4].  In that case, all the 
code that deals with races in full system suspend (like suspend blockers) 
can be dropped.


- Paul

1. Matthew Garrett E-mail to the linux-pm mailing list, dated Tue May 18
   02:55:19 +0100
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025664.html

2. Paragraph B4A of Paul Walmsley E-mail to the linux-pm mailing list, 
   dated Mon May 17 19:39:44 CDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html

3. Paragraph B4B of Paul Walmsley E-mail to the linux-pm mailing list, 
   dated Mon May 17 19:39:44 CDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html

4. Arve Hjønnevåg E-mail to the linux-pm mailing list, dated Mon, May 17 
   20:06:33 PDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025668.html

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

Hi,

just wanted to clarify one point:

On Fri, 21 May 2010, Paul Walmsley wrote:

> In that case, all the code that deals with races in full system suspend 
> (like suspend blockers) can be dropped.

Not necessarily dropped entirely - just considered separately, apart from 
the contexts of "getting Android device drivers upstream" and "pausing 
processes/timers before entering a system low-power state".


- Paul

Re: [PATCH 0/8] Suspend block api (version 6)

[Matthew Garrett]

On Fri, May 21, 2010 at 10:51:58AM -0600, Paul Walmsley wrote:

>       B. To prevent untrusted processes from hogging CPUs, the
>          processes can be run in a cgroup that puts all threads in
>          TASK_RUNNING processes into TASK_INTERRUPTIBLE state and
>          calls schedule() [6], upon receipt of some trigger.  This
>          should allow processes to be awakened by an external system
>          event, e.g., network packet received.  When the system
>          returns from the system low-power state and some trigger
>          event happens, those process threads can be placed back into
>          TASK_RUNNING state and rescheduled.

I don't think that works. If I have an app that would generally complete 
some work and then call select(), you'll instead stop it in the middle 
of that work. If that app is the one that should process the wakeup 
event then it'll never be delivered because you've never got back to 
select().

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Pavel Machek]

Hi!

> > There are several general problems with the design of opportunistic
> > suspend and suspend-blocks.
> > 
> > 1. The opportunistic suspend code bypasses existing Linux kernel code,
> >    such as timers and the scheduler, that indicates when code
> >    needs to run, and when the system is idle.
> 
> Whoa!  That's not my understanding at all.
> 
> As I see it, opportunistic suspend doesn't bypass any code that isn't 
> already bypassed by the existing suspend code.  Users can do
> 
> 	echo mem >/sys/power/state
> 
> whenever they want, without regard to kernel timers and the scheduler 
> (other than the fact that the user's thread must be running in order to 
> carry out the write, of course).

Yep. And while I'm co-responsible for that interface, I would not
call it exactly nice. Yes, it does the job.

But imagine horrors atd/cron would have to do to work properly with
that interface... setting rtc wakeups etc.

So yes, mem > state already breaks promises, but lets not extend that.

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [linux-pm] [PATCH 0/8] Suspend block api (version 6)

[Pavel Machek]

On Thu 2010-05-13 21:08:14, Matthew Garrett wrote:
> On Thu, May 13, 2010 at 01:00:04PM -0700, Tony Lindgren wrote:
> 
> > The system stays running because there's something to do. The system
> > won't suspend until all the processors hit the kernel idle loop and
> > the next_timer_interrupt_critical() returns nothing.
> 
> At which point an application in a busy loop cripples you. I think we 

Application running busy loop already cripples you, with power going
from 15W to 50W in thinkpad case. Just avoid badly written apps.

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH 0/8] Suspend block api (version 6)

[Paul Walmsley]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 10357 bytes --]

Hello Arve,

I regret the delay in responding.

On Mon, 17 May 2010, Arve Hjønnevåg wrote:

> On Mon, May 17, 2010 at 7:26 PM, Paul Walmsley <paul@pwsan.com> wrote:
> > On Fri, 14 May 2010, Arve Hjønnevåg wrote:
> >> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
> >> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
> >> >
> >> >> No, suspend blockers are mostly used to ensure wakeup events are not
> >> >> ignored, and to ensure tasks triggered by these wakeup events
> >> >> complete.
> >> >
> >> > Standard Linux systems don't need these,
> >>
> >> If you don't want to lose wakeup events they do.  Standard Linux systems
> >> support suspend, but since they usually don't have a lot of wakeup
> >> events you don't run into a lot of problems.
> >
> > Sorry, I don't follow.  What causes wakeup events to be lost?  Is it the
> > current opportunistic suspend governor?  On OMAP Linux systems, as far as
> > I know, we don't lose any wakeup events.
> >
> Have you used suspend?

I see.  You are referring to wakeup event loss/delay caused by the full 
system suspend process (e.g., kernel/power/suspend.c:enter_state() 
onwards[1]), correct?  This message addresses this at the end of the mail.

> >> > because the scheduler just keeps the system running as long as there
> >> > is work to be done.
> >>
> >> That is only true if you never use suspend.
> >
> > If, instead of the current Android opportunistic suspend governor, the
> > system entered suspend from pm_idle(), wouldn't that keep the system
> > running as long as there is work to done?
> >
> How do you know if the work being done while suspending is work that
> is needed to suspend or work that should abort suspend?

Consider a suspending system in which all "untrusted" processes have been 
placed into TASK_INTERRUPTIBLE state and have had their timers 
disabled.[2] The only remaining work to do on the system is work that 
needs to be done as part of the suspend process, or work that is part of a 
"trusted" process or the kernel.  If the suspend needs to be aborted due 
to the arrival of an event, this can be handled as part of the standard 
suspend process (described below).

> When should the system wake up?

I suspect that I am misunderstanding your question, but the system should 
wake up the same way it does now: when a wakeup interrupt arrives (either 
from the next scheduled timer expiration, or from some external device).

> > As far as I can see, it's the current Android opportunistic suspend
> > governor design in patch 1 that causes the system to enter suspend even
> > when there is work to be done, since it will try to suspend even when the
> > system is out of the idle loop.
> 
> It does not matter how you enter suspend. Without opportunistic suspend, 
> once you tell the kernel that you want to suspend, you cannot abort.

Any driver should be able to abort suspend by returning an error from its 
struct device_driver/struct dev_pm_ops suspend function.  Consider this 
partial callgraph for full system suspend, from [3], [4]:

   kernel/power/suspend.c:   enter_state() ->
   kernel/power/suspend.c:    suspend_devices_and_enter() ->
drivers/base/power/main.c:     dpm_suspend_start() ->
drivers/base/power/main.c:      dpm_suspend() ->
drivers/base/power/main.c:       device_suspend() ->
drivers/base/power/main.c:        __device_suspend() ->
drivers/base/power/main.c:         pm_op() ->
drivers/base/power/main.c:          ops->suspend()

If ops->suspend() returns an error, it's ultimately passed back to 
suspend_devices_and_enter(), which aborts the suspend[5].

In this context, the main change that should be necessary is for 
driver/subsystem suspend functions to return an error when there are 
events pending.  For example, as an alternative to the Android series' 
evdev.c patch[6], the evdev.c suspend function could return an error such 
as -EBUSY if events are currently queued (example below).

Aborting suspend in the event of driver activity seems like the right 
thing to do in a system that attempts to enter full system suspend while 
idle.  But it is not the current expected Linux behavior, which also seems 
useful.  Any abort-suspend-if-busy behavior should be configurable.  One 
way would be to add a sysfs file for each driver/subsystem: perhaps 
something like 'abort_suspend_if_busy'.  The default value would be zero 
to preserve the current behavior.  Systems that wish to use opportunistic 
suspend could write a one to some or all of those files.

An untested, purely illustrative example, based on evdev.c, is at the end 
of this message.  It's not meant to be functional; it's just meant to 
demonstrate the basic idea in code form.

...

Like suspend-blockers, adding abort_suspend_if_busy support would require 
changes throughout many drivers and subsystems.  However, unlike 
suspend-blockers, the above abort_suspend_if_busy approach would not 
introduce any new APIs[7], nor would it change the default suspend 
behavior of the system.

As an aside, it seems that Android shouldn't currently need this if it's 
possible to pause "untrusted" processes and timers[8], since current 
shipping Android hardware can enter the same system low power state both 
with and without full system suspend[9].  But this could be useful for 
non-Android systems that still wish to use an idle-loop based, 
opportunistic full system suspend.


regards,

- Paul


1. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l258

2. Section B4 of Paul Walmsley E-mail to the linux-pm mailing list, 
   dated Mon May 17 19:39:44 CDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html

3. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l258

4. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=drivers/base/power/main.c;h=941fcb87e52a12765df2aaa2e2ab21267693af9f;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l1062

5. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l207

6. Arve Hjønnevåg E-mail to the linux-pm mailing list, dated Fri May 21 
   15:46:54 PDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025747.html

7. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu May 13 
   23:13:50 PDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025524.html

8. Paul Walmsley E-mail to the linux-pm mailing list, dated Mon May 17 
   18:39:44 PDT 2010:
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html  

9. Arve Hjønnevåg E-mail to the linux-pm mailing list, dated Mon, May 17 
   20:06:33 PDT 2010:   
   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025668.html



--------------------------------------------------------------------------------------

evdev.c abort_suspend_if_idle experiment:

---
 drivers/input/evdev.c |   64 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 63 insertions(+), 1 deletions(-)

diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c
index 2ee6c7a..23eaad1 100644
--- a/drivers/input/evdev.c
+++ b/drivers/input/evdev.c
@@ -20,8 +20,16 @@
 #include <linux/input.h>
 #include <linux/major.h>
 #include <linux/device.h>
+#include <linux/pm.h>
 #include "input-compat.h"
 
+/*
+ * abort_suspend_if_busy: set to 1 to prevent suspend as long as there
+ * is an event in the queue; set to 0 to allow suspend at any time.
+ * XXX Intended to be read from sysfs or the input subsystem.
+ */
+static int abort_suspend_if_busy;
+
 struct evdev {
 	int exist;
 	int open;
@@ -118,6 +126,36 @@ static int evdev_flush(struct file *file, fl_owner_t id)
 	return retval;
 }
 
+static int evdev_suspend(struct device *dev)
+{
+	struct evdev *evdev = container_of(dev, struct evdev, dev);
+	struct evdev_client *client;
+	int ret = 0;
+
+	if (!abort_suspend_if_busy)
+		return 0;
+
+	rcu_read_lock();
+
+	client = rcu_dereference(evdev->grab);
+	if (client) {
+		if (client->head != client->tail)
+			ret = -EBUSY;
+	} else {
+		list_for_each_entry_rcu(client, &evdev->client_list, node) {
+			if (client->head != client->tail) {
+				ret = -EBUSY;
+				break;
+			}
+		}
+	}
+
+	rcu_read_unlock();
+
+	return ret;
+}
+
+
 static void evdev_free(struct device *dev)
 {
 	struct evdev *evdev = container_of(dev, struct evdev, dev);
@@ -787,6 +825,21 @@ static void evdev_cleanup(struct evdev *evdev)
 	}
 }
 
+static const struct dev_pm_ops evdev_pm_ops = {
+	.suspend	= &evdev_suspend,
+};
+
+static char *evdev_devnode(struct device *dev, mode_t *mode)
+{
+	return kasprintf(GFP_KERNEL, "input/%s", dev_name(dev));
+}
+
+static struct class evdev_class = {
+	.name		= "evdev",
+	.devnode	= evdev_devnode,
+	.pm		= &evdev_pm_ops,
+};
+
 /*
  * Create new evdev device. Note that input core serializes calls
  * to connect and disconnect so we don't need to lock evdev_table here.
@@ -826,7 +879,7 @@ static int evdev_connect(struct input_handler *handler, struct input_dev *dev,
 	evdev->handle.private = evdev;
 
 	evdev->dev.devt = MKDEV(INPUT_MAJOR, EVDEV_MINOR_BASE + minor);
-	evdev->dev.class = &input_class;
+	evdev->dev.class = &evdev_class;
 	evdev->dev.parent = &dev->dev;
 	evdev->dev.release = evdev_free;
 	device_initialize(&evdev->dev);
@@ -883,12 +936,21 @@ static struct input_handler evdev_handler = {
 
 static int __init evdev_init(void)
 {
+	int err;
+
+	err = class_register(&input_class);
+	if (err) {
+		printk(KERN_ERR "evdev: unable to register evdev class\n");
+		return err;
+	}
+
 	return input_register_handler(&evdev_handler);
 }
 
 static void __exit evdev_exit(void)
 {
 	input_unregister_handler(&evdev_handler);
+	class_unregister(&evdev_class);
 }
 
 module_init(evdev_init);
-- 
1.7.1.GIT

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH 0/8] Suspend block api (version 6)

[Dmitry Torokhov]

On Fri, May 14, 2010 at 03:18:13PM -0700, Arve Hjønnevåg wrote:
> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
> >
> > Hello,
> >
> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
> >
> >> No, suspend blockers are mostly used to ensure wakeup events are not
> >> ignored, and to ensure tasks triggered by these wakeup events
> >> complete.
> >
> > Standard Linux systems don't need these,
> 
> If you don't want to lose wakeup events they do. Standard Linux
> systems support suspend, but since they usually don't have a lot of
> wakeup events you don't run into a lot of problems.
> 

What I do not quite understand is how exactly suspend blocks save us
from missing wakeup events. Consider case when all events have been
processed and system starts entering the sleep: we are in a half-sleep,
half awake state, with some devices already put to sleep and some just
now being processed. If user presses a key while physical input device
is being suspended, what will happen? Do you expect interrupt routine
abort suspend process? Initiate resume if suspend of the device has
completed?

Also, let's say device is suspended and is configured as a wakeup source
while the rest of the system is still awake... If I press a key I do not
think that it will necessarily abort suspend process. Or will it?

BTW, If you are concerned about events that already "left" physical
device but has not reached userspace yet - maybe instead of suspend
blockers we should make sure that all drivers throughout the chain
implement suspend/resume and refuse suspending if their queues are not
empty.  In input land that would mean extending suspend routine in
input_dev and adding one to evdev.

Thanks.

-- 
Dmitry

Re: [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Tue, 25 May 2010, Dmitry Torokhov wrote:

> What I do not quite understand is how exactly suspend blocks save us
> from missing wakeup events. Consider case when all events have been
> processed and system starts entering the sleep: we are in a half-sleep,
> half awake state, with some devices already put to sleep and some just
> now being processed. If user presses a key while physical input device
> is being suspended, what will happen? Do you expect interrupt routine
> abort suspend process? Initiate resume if suspend of the device has
> completed?

This depends on several factors, such as how the input device drivers
are written and how the hardware works, as well as the details of the 
timing.  Here are a few possibilities:

	The keypress is not handled but stays in the hardware as a
	wakeup event.  The system goes through its entire suspend
	procedure, and then the pending wakeup event immediately
	cases it to wake up.  Then the keypress gets handled
	normally.

	Or the keypress is treated as a normal IRQ and handled by
	the hardware.  The input subsystem puts the keystroke on
	the event queue (all done while in_interrupt), where it
	languishes because userspace is frozen and can't read it.
	The system goes to sleep and stays that way until something
	else wakes it up.  (This is a type-3 failure as described
	in my earlier email.)

	Or the keypress is handled normally and the input subsystem 
	interprets it as a runtime resume request, telling the PM core 
	to queue a resume call.  But the PM core's runtime workqueue 
	thread has already been frozen, and so the event remains on the 
	work queue until the system wakes up on its own.  (This is a
	type-2 failure.)

	Or we fix the PM core, so that when the resume call is added
	to the workqueue it realizes the suspend-in-progress needs
	to be aborted.  The system goes back to a fully-awake state
	and the keystroke is handled normally.

> Also, let's say device is suspended and is configured as a wakeup source
> while the rest of the system is still awake... If I press a key I do not
> think that it will necessarily abort suspend process. Or will it?

Since the keyboard is a wakeup device, it certainly should abort the
suspend.  Or at least it should wake the system up as soon as the
system finishes going to sleep.  At the moment, though, I'm not sure
that it does behave either way.

> BTW, If you are concerned about events that already "left" physical
> device but has not reached userspace yet - maybe instead of suspend
> blockers we should make sure that all drivers throughout the chain
> implement suspend/resume and refuse suspending if their queues are not
> empty.  In input land that would mean extending suspend routine in
> input_dev and adding one to evdev.

That's not the only problem.  We also have to consider events that have
reached userspace but not yet been fully processed.  The user thread
handling the event needs some way to prevent the system from suspending
until it is all done.

Alan Stern

Re: [PATCH 0/8] Suspend block api (version 6)

[Dmitry Torokhov]

On Tue, May 25, 2010 at 02:25:08PM -0400, Alan Stern wrote:
> On Tue, 25 May 2010, Dmitry Torokhov wrote:
> 
> > BTW, If you are concerned about events that already "left" physical
> > device but has not reached userspace yet - maybe instead of suspend
> > blockers we should make sure that all drivers throughout the chain
> > implement suspend/resume and refuse suspending if their queues are not
> > empty.  In input land that would mean extending suspend routine in
> > input_dev and adding one to evdev.
> 
> That's not the only problem.  We also have to consider events that have
> reached userspace but not yet been fully processed.  The user thread
> handling the event needs some way to prevent the system from suspending
> until it is all done.
> 

It is a problem if kernel initiated suspend transition on its own. I
believe that it is userspace responsibility to initiate sustend (and
make sure that needs of userspace, including processing of certain
events, are served beforehand).

-- 
Dmitry

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Tue, May 25, 2010 at 11:33 AM, Dmitry Torokhov
<dmitry.torokhov@gmail.com> wrote:
> On Tue, May 25, 2010 at 02:25:08PM -0400, Alan Stern wrote:
>> On Tue, 25 May 2010, Dmitry Torokhov wrote:
>>
>> > BTW, If you are concerned about events that already "left" physical
>> > device but has not reached userspace yet - maybe instead of suspend
>> > blockers we should make sure that all drivers throughout the chain
>> > implement suspend/resume and refuse suspending if their queues are not
>> > empty.  In input land that would mean extending suspend routine in
>> > input_dev and adding one to evdev.
>>
>> That's not the only problem.  We also have to consider events that have
>> reached userspace but not yet been fully processed.  The user thread
>> handling the event needs some way to prevent the system from suspending
>> until it is all done.
>>
>
> It is a problem if kernel initiated suspend transition on its own. I
> believe that it is userspace responsibility to initiate sustend (and
> make sure that needs of userspace, including processing of certain
> events, are served beforehand).

That only works if a single user-space thread initiates suspend and
consumes all wakeup events (or if the user-space code that initiates
suspend implements suspend blockers), and even then the kernel would
still need to block suspend in same places as it does if the kernel
initiates suspend. If a wakeup event happens right after user-space
initiates suspend, that suspend must be aborted. By only initiating
suspend from user-space, you force this to be a polling operation.

-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Dmitry Torokhov]

On Tue, May 25, 2010 at 03:05:41PM -0700, Arve Hjønnevåg wrote:
> On Tue, May 25, 2010 at 11:33 AM, Dmitry Torokhov
> <dmitry.torokhov@gmail.com> wrote:
> > On Tue, May 25, 2010 at 02:25:08PM -0400, Alan Stern wrote:
> >> On Tue, 25 May 2010, Dmitry Torokhov wrote:
> >>
> >> > BTW, If you are concerned about events that already "left" physical
> >> > device but has not reached userspace yet - maybe instead of suspend
> >> > blockers we should make sure that all drivers throughout the chain
> >> > implement suspend/resume and refuse suspending if their queues are not
> >> > empty.  In input land that would mean extending suspend routine in
> >> > input_dev and adding one to evdev.
> >>
> >> That's not the only problem.  We also have to consider events that have
> >> reached userspace but not yet been fully processed.  The user thread
> >> handling the event needs some way to prevent the system from suspending
> >> until it is all done.
> >>
> >
> > It is a problem if kernel initiated suspend transition on its own. I
> > believe that it is userspace responsibility to initiate sustend (and
> > make sure that needs of userspace, including processing of certain
> > events, are served beforehand).
> 
> That only works if a single user-space thread initiates suspend

Yes, there should be a single entity controlling suspend.

> and
> consumes all wakeup events (or if the user-space code that initiates
> suspend implements suspend blockers),

Yes.

> and even then the kernel would
> still need to block suspend in same places as it does if the kernel
> initiates suspend. If a wakeup event happens right after user-space
> initiates suspend, that suspend must be aborted.

This is true regardless of whether we use suspend blockers or not. If,
after initiating suspend, we detect wake up event that event shoudl be
acted upon and suspend should be ignored.

> By only initiating
> suspend from user-space, you force this to be a polling operation.
>

I am not sure what you mean by this.

-- 
Dmitry

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Tue, May 25, 2010 at 2:41 AM, Paul Walmsley <paul@pwsan.com> wrote:
> Hello Arve,
>
> I regret the delay in responding.
>
> On Mon, 17 May 2010, Arve Hjønnevåg wrote:
>
>> On Mon, May 17, 2010 at 7:26 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> > On Fri, 14 May 2010, Arve Hjønnevåg wrote:
>> >> On Fri, May 14, 2010 at 1:27 PM, Paul Walmsley <paul@pwsan.com> wrote:
>> >> > On Mon, 3 May 2010, Arve Hjønnevåg wrote:
>> >> >
>> >> >> No, suspend blockers are mostly used to ensure wakeup events are not
>> >> >> ignored, and to ensure tasks triggered by these wakeup events
>> >> >> complete.
>> >> >
>> >> > Standard Linux systems don't need these,
>> >>
>> >> If you don't want to lose wakeup events they do.  Standard Linux systems
>> >> support suspend, but since they usually don't have a lot of wakeup
>> >> events you don't run into a lot of problems.
>> >
>> > Sorry, I don't follow.  What causes wakeup events to be lost?  Is it the
>> > current opportunistic suspend governor?  On OMAP Linux systems, as far as
>> > I know, we don't lose any wakeup events.
>> >
>> Have you used suspend?
>
> I see.  You are referring to wakeup event loss/delay caused by the full
> system suspend process (e.g., kernel/power/suspend.c:enter_state()
> onwards[1]), correct?  This message addresses this at the end of the mail.
>
>> >> > because the scheduler just keeps the system running as long as there
>> >> > is work to be done.
>> >>
>> >> That is only true if you never use suspend.
>> >
>> > If, instead of the current Android opportunistic suspend governor, the
>> > system entered suspend from pm_idle(), wouldn't that keep the system
>> > running as long as there is work to done?
>> >
>> How do you know if the work being done while suspending is work that
>> is needed to suspend or work that should abort suspend?
>
> Consider a suspending system in which all "untrusted" processes have been
> placed into TASK_INTERRUPTIBLE state and have had their timers
> disabled.[2] The only remaining work to do on the system is work that
> needs to be done as part of the suspend process, or work that is part of a
> "trusted" process or the kernel.  If the suspend needs to be aborted due
> to the arrival of an event, this can be handled as part of the standard
> suspend process (described below).
>

That means modifying all kernel code that would use suspend blockers
to abort suspend by returning an error from suspend instead. How is
this better than using suspend blockers?

>> When should the system wake up?
>
> I suspect that I am misunderstanding your question, but the system should
> wake up the same way it does now: when a wakeup interrupt arrives (either
> from the next scheduled timer expiration, or from some external device).
>

That is not how the system wakes up from suspend now. The monotonic
clock stops, and timers are ignored.

>> > As far as I can see, it's the current Android opportunistic suspend
>> > governor design in patch 1 that causes the system to enter suspend even
>> > when there is work to be done, since it will try to suspend even when the
>> > system is out of the idle loop.
>>
>> It does not matter how you enter suspend. Without opportunistic suspend,
>> once you tell the kernel that you want to suspend, you cannot abort.
>
> Any driver should be able to abort suspend by returning an error from its
> struct device_driver/struct dev_pm_ops suspend function.  Consider this

Yes a driver can abort, but the user space code that initiated suspend cannot.

> partial callgraph for full system suspend, from [3], [4]:
>
>   kernel/power/suspend.c:   enter_state() ->
>   kernel/power/suspend.c:    suspend_devices_and_enter() ->
> drivers/base/power/main.c:     dpm_suspend_start() ->
> drivers/base/power/main.c:      dpm_suspend() ->
> drivers/base/power/main.c:       device_suspend() ->
> drivers/base/power/main.c:        __device_suspend() ->
> drivers/base/power/main.c:         pm_op() ->
> drivers/base/power/main.c:          ops->suspend()
>
> If ops->suspend() returns an error, it's ultimately passed back to
> suspend_devices_and_enter(), which aborts the suspend[5].
>
> In this context, the main change that should be necessary is for
> driver/subsystem suspend functions to return an error when there are
> events pending.  For example, as an alternative to the Android series'
> evdev.c patch[6], the evdev.c suspend function could return an error such
> as -EBUSY if events are currently queued (example below).
>
> Aborting suspend in the event of driver activity seems like the right
> thing to do in a system that attempts to enter full system suspend while
> idle.  But it is not the current expected Linux behavior, which also seems
> useful.  Any abort-suspend-if-busy behavior should be configurable.  One
> way would be to add a sysfs file for each driver/subsystem: perhaps
> something like 'abort_suspend_if_busy'.  The default value would be zero
> to preserve the current behavior.  Systems that wish to use opportunistic
> suspend could write a one to some or all of those files.
>
> An untested, purely illustrative example, based on evdev.c, is at the end
> of this message.  It's not meant to be functional; it's just meant to
> demonstrate the basic idea in code form.
>

Do you think this is simpler than using a suspend blocker?

> ...
>
> Like suspend-blockers, adding abort_suspend_if_busy support would require
> changes throughout many drivers and subsystems.  However, unlike
> suspend-blockers, the above abort_suspend_if_busy approach would not
> introduce any new APIs[7], nor would it change the default suspend
> behavior of the system.
>

Opportunistic suspend does not change the default behaviour either.
Your proposal requires that all the drivers that would block suspend
using a suspend blocker, now block suspend without the help of the
suspend blocker api.

> As an aside, it seems that Android shouldn't currently need this if it's
> possible to pause "untrusted" processes and timers[8], since current

Your proposal to "pause"processes by putting them in
TASK_INTERRUPTIBLE state and stop their timers does not work. It is
effectivly the same ase freezing them, since if you "paused" it while
it was busy, a wakeup event cannot unpause it.

> shipping Android hardware can enter the same system low power state both
> with and without full system suspend[9].  But this could be useful for
> non-Android systems that still wish to use an idle-loop based,
> opportunistic full system suspend.
>
>
> regards,
>
> - Paul
>
>
> 1. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l258
>
> 2. Section B4 of Paul Walmsley E-mail to the linux-pm mailing list,
>   dated Mon May 17 19:39:44 CDT 2010:
>   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html
>
> 3. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l258
>
> 4. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=drivers/base/power/main.c;h=941fcb87e52a12765df2aaa2e2ab21267693af9f;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l1062
>
> 5. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=kernel/power/suspend.c;h=56e7dbb8b996db295b4fc7cdf1b5f11a0409cb0c;hb=7e125f7b9cbfce4101191b8076d606c517a73066#l207
>
> 6. Arve Hjønnevåg E-mail to the linux-pm mailing list, dated Fri May 21
>   15:46:54 PDT 2010:
>   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025747.html
>
> 7. Paul Walmsley E-mail to the linux-pm mailing list, dated Thu May 13
>   23:13:50 PDT 2010:
>   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025524.html
>
> 8. Paul Walmsley E-mail to the linux-pm mailing list, dated Mon May 17
>   18:39:44 PDT 2010:
>   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025663.html
>
> 9. Arve Hjønnevåg E-mail to the linux-pm mailing list, dated Mon, May 17
>   20:06:33 PDT 2010:
>   https://lists.linux-foundation.org/pipermail/linux-pm/2010-May/025668.html
>
>
>
> --------------------------------------------------------------------------------------
>
> evdev.c abort_suspend_if_idle experiment:
>
> ---
>  drivers/input/evdev.c |   64 ++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 63 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c
> index 2ee6c7a..23eaad1 100644
> --- a/drivers/input/evdev.c
> +++ b/drivers/input/evdev.c
> @@ -20,8 +20,16 @@
>  #include <linux/input.h>
>  #include <linux/major.h>
>  #include <linux/device.h>
> +#include <linux/pm.h>
>  #include "input-compat.h"
>
> +/*
> + * abort_suspend_if_busy: set to 1 to prevent suspend as long as there
> + * is an event in the queue; set to 0 to allow suspend at any time.
> + * XXX Intended to be read from sysfs or the input subsystem.
> + */
> +static int abort_suspend_if_busy;
> +
>  struct evdev {
>        int exist;
>        int open;
> @@ -118,6 +126,36 @@ static int evdev_flush(struct file *file, fl_owner_t id)
>        return retval;
>  }
>
> +static int evdev_suspend(struct device *dev)
> +{
> +       struct evdev *evdev = container_of(dev, struct evdev, dev);
> +       struct evdev_client *client;
> +       int ret = 0;
> +
> +       if (!abort_suspend_if_busy)
> +               return 0;
> +
> +       rcu_read_lock();
> +
> +       client = rcu_dereference(evdev->grab);
> +       if (client) {
> +               if (client->head != client->tail)
> +                       ret = -EBUSY;
> +       } else {
> +               list_for_each_entry_rcu(client, &evdev->client_list, node) {
> +                       if (client->head != client->tail) {
> +                               ret = -EBUSY;
> +                               break;
> +                       }
> +               }
> +       }
> +
> +       rcu_read_unlock();
> +
> +       return ret;
> +}
> +
> +
>  static void evdev_free(struct device *dev)
>  {
>        struct evdev *evdev = container_of(dev, struct evdev, dev);
> @@ -787,6 +825,21 @@ static void evdev_cleanup(struct evdev *evdev)
>        }
>  }
>
> +static const struct dev_pm_ops evdev_pm_ops = {
> +       .suspend        = &evdev_suspend,
> +};
> +
> +static char *evdev_devnode(struct device *dev, mode_t *mode)
> +{
> +       return kasprintf(GFP_KERNEL, "input/%s", dev_name(dev));
> +}
> +
> +static struct class evdev_class = {
> +       .name           = "evdev",
> +       .devnode        = evdev_devnode,
> +       .pm             = &evdev_pm_ops,
> +};
> +
>  /*
>  * Create new evdev device. Note that input core serializes calls
>  * to connect and disconnect so we don't need to lock evdev_table here.
> @@ -826,7 +879,7 @@ static int evdev_connect(struct input_handler *handler, struct input_dev *dev,
>        evdev->handle.private = evdev;
>
>        evdev->dev.devt = MKDEV(INPUT_MAJOR, EVDEV_MINOR_BASE + minor);
> -       evdev->dev.class = &input_class;
> +       evdev->dev.class = &evdev_class;
>        evdev->dev.parent = &dev->dev;
>        evdev->dev.release = evdev_free;
>        device_initialize(&evdev->dev);
> @@ -883,12 +936,21 @@ static struct input_handler evdev_handler = {
>
>  static int __init evdev_init(void)
>  {
> +       int err;
> +
> +       err = class_register(&input_class);
> +       if (err) {
> +               printk(KERN_ERR "evdev: unable to register evdev class\n");
> +               return err;
> +       }
> +
>        return input_register_handler(&evdev_handler);
>  }
>
>  static void __exit evdev_exit(void)
>  {
>        input_unregister_handler(&evdev_handler);
> +       class_unregister(&evdev_class);
>  }
>
>  module_init(evdev_init);
> --
> 1.7.1.GIT



-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Linus WALLEIJ]

[Arve wrote]
> On Tue, May 25, 2010 at 2:41 AM, Paul Walmsley <paul@pwsan.com> wrote:
> > On Mon, 17 May 2010, Arve Hjønnevåg wrote:
> >>
> >> How do you know if the work being done while suspending is work that
> >> is needed to suspend or work that should abort suspend?
> >
> > Consider a suspending system in which all "untrusted" processes have been
> > placed into TASK_INTERRUPTIBLE state and have had their timers
> > disabled.[2] The only remaining work to do on the system is work that
> > needs to be done as part of the suspend process, or work that is part of
> > a
> > "trusted" process or the kernel.  If the suspend needs to be aborted due
> > to the arrival of an event, this can be handled as part of the standard
> > suspend process (described below).
> 
> That means modifying all kernel code that would use suspend blockers
> to abort suspend by returning an error from suspend instead. How is
> this better than using suspend blockers?

It will mean that the same API is used for blocking suspend in
regular suspend/resume, runtime PM runtime_suspend/runtime_resume
and now also suspend blockers, right? So only one code path for
drivers that want to support both, no?

That'll be a bit more elegant for drivers that want to support
both runtime PM and suspend blockers I believe.

Overall as a driver writer I'm quite confused at this point, if
I have a driver X, how is that code going to look if it is going
to support both runtime PM *and* suspend blockers? I don't say
it has to support both mechanisms in parallell, I'm even fine
with putting it in #ifdef:s, but how will the source code look?
A real-world example on a simple driver out of the Motorola Droid
git or so would help a lot.

It's quite relevant since e.g. all OMAP drivers will need this
modification to be used with Android-like systems, and at
ST-Ericsson we will need both runtime PM and suspend blockers for
our drivers as well. It will as a consequence affect the drivers
for all ARM reference boards since we use the drivers for ARM
PrimeCells. etc...

Now if this mechanism is going in, my main interest is that there
is some clearly cut way and design pattern for supporting both
runtime PM and suspend blocks.

Yours,
Linus Walleij

Re: [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Wed, 26 May 2010, Linus WALLEIJ wrote:

> Overall as a driver writer I'm quite confused at this point, if
> I have a driver X, how is that code going to look if it is going
> to support both runtime PM *and* suspend blockers? I don't say
> it has to support both mechanisms in parallell, I'm even fine
> with putting it in #ifdef:s, but how will the source code look?
> A real-world example on a simple driver out of the Motorola Droid
> git or so would help a lot.
> 
> It's quite relevant since e.g. all OMAP drivers will need this
> modification to be used with Android-like systems, and at
> ST-Ericsson we will need both runtime PM and suspend blockers for
> our drivers as well. It will as a consequence affect the drivers
> for all ARM reference boards since we use the drivers for ARM
> PrimeCells. etc...
> 
> Now if this mechanism is going in, my main interest is that there
> is some clearly cut way and design pattern for supporting both
> runtime PM and suspend blocks.

It's not difficult.  You just have to decide what situations should 
block opportunistic suspend.  For example, let's say that opportunistic 
suspend should be blocked when your event queue is non-empty.

Then in the routine that adds new events to the queue, while still
holding the spinlock that protects your queue, you enable your suspend
blocker.  Likewise, in the routine that takes events off the queue and
sends them to userspace, while holding the spinlock you test whether
the queue has become empty; if it has you disable your suspend blocker.

That's all.  No other changes are needed (except to create and destroy 
the suspend blocker, of course).  No other interaction with suspend, 
resume, or runtime PM.

Alan Stern

Re: [PATCH 0/8] Suspend block api (version 6)

[Linus WALLEIJ]

[Alan Stern]
> On Wed, 26 May 2010, Linus WALLEIJ wrote:
> (...)
> > Now if this mechanism is going in, my main interest is that there
> > is some clearly cut way and design pattern for supporting both
> > runtime PM and suspend blocks.
> 
> It's not difficult.  You just have to decide what situations should
> block opportunistic suspend.  For example, let's say that opportunistic
> suspend should be blocked when your event queue is non-empty.
> 
> Then in the routine that adds new events to the queue, while still
> holding the spinlock that protects your queue, you enable your suspend
> blocker.  Likewise, in the routine that takes events off the queue and
> sends them to userspace, while holding the spinlock you test whether
> the queue has become empty; if it has you disable your suspend blocker.
> 
> That's all.  No other changes are needed (except to create and destroy
> the suspend blocker, of course).  No other interaction with suspend,
> resume, or runtime PM.

OK I understand it will be rather OK to do this. For example I have
drivers/spi/amba-pl022.c which is quite clearly taking elements
off a queue and transmitting them.

Currently, as we do not have suspend blockers, we try to stop the
queue in suspend() and if that fails we return something negative
so that the suspend() is blocked by the negative return code.

Maybe the behaviour for an SPI bus should rather be to return
-EBUSY as long as there are events in the queue. I don't quite
know frankly.

(We haven't added runtime PM yet, it will likely be the same
thing.)

With suspend blockers we take a suspend blocker when we add
elements to the queue and release it when the queue is empty.

But with suspend blockers the need for suspend() to check if
the queue can be stopped or return -EBUSY if there are elements
in it is not necessary: suspend() simply won't be called
since a suspend blocker is taken. There is no way for
suspend() to stop a running queue like we do today if using
suspend blockers.

This means that driver writers only targeting configurations
where suspend blockers are used will probably not care about
handling suspend() calls by trying to stop the queue or
checking if there are things in it, because there will never
be anything there.

So while it is easy for me to have the driver work under both
suspend blockers and runtime PM or common suspend(), the
two configurations actually have different semantics of the
suspend() call: in the suspend blockers case you don't need
to check anything, just suspend, and in  the runtime PM
case you have to check that you can actually suspend.

Is this analysis correct?

If yes then OK, it's not totally elegant but if that is
where we have to go, I can live with it. There will likely
be people who implement for only one or the other semantic
behaviour, but we have worse things to cope with already.

Yours,
Linus Walleij

Re: [PATCH 0/8] Suspend block api (version 6)

[Florian Mickler]

On Thu, 27 May 2010 09:46:51 +0200
Linus WALLEIJ <linus.walleij@stericsson.com> wrote:


> If yes then OK, it's not totally elegant but if that is
> where we have to go, I can live with it. There will likely
> be people who implement for only one or the other semantic
> behaviour, but we have worse things to cope with already.

Alan Cox suggested, that this kind of explicit requirement definition
might be necessary for all drivers anyway in the long run.

That way, the semantic differences between those two cases would
vanish. 

Cheers,
Flo

Re: [PATCH 0/8] Suspend block api (version 6)

[Arve Hjønnevåg]

On Thu, May 27, 2010 at 12:46 AM, Linus WALLEIJ
<linus.walleij@stericsson.com> wrote:
> [Alan Stern]
>> On Wed, 26 May 2010, Linus WALLEIJ wrote:
>> (...)
>> > Now if this mechanism is going in, my main interest is that there
>> > is some clearly cut way and design pattern for supporting both
>> > runtime PM and suspend blocks.
>>
>> It's not difficult.  You just have to decide what situations should
>> block opportunistic suspend.  For example, let's say that opportunistic
>> suspend should be blocked when your event queue is non-empty.
>>
>> Then in the routine that adds new events to the queue, while still
>> holding the spinlock that protects your queue, you enable your suspend
>> blocker.  Likewise, in the routine that takes events off the queue and
>> sends them to userspace, while holding the spinlock you test whether
>> the queue has become empty; if it has you disable your suspend blocker.
>>
>> That's all.  No other changes are needed (except to create and destroy
>> the suspend blocker, of course).  No other interaction with suspend,
>> resume, or runtime PM.
>
> OK I understand it will be rather OK to do this. For example I have
> drivers/spi/amba-pl022.c which is quite clearly taking elements
> off a queue and transmitting them.
>
> Currently, as we do not have suspend blockers, we try to stop the
> queue in suspend() and if that fails we return something negative
> so that the suspend() is blocked by the negative return code.
>
> Maybe the behaviour for an SPI bus should rather be to return
> -EBUSY as long as there are events in the queue. I don't quite
> know frankly.
>
> (We haven't added runtime PM yet, it will likely be the same
> thing.)
>
> With suspend blockers we take a suspend blocker when we add
> elements to the queue and release it when the queue is empty.
>
> But with suspend blockers the need for suspend() to check if
> the queue can be stopped or return -EBUSY if there are elements
> in it is not necessary: suspend() simply won't be called
> since a suspend blocker is taken. There is no way for
> suspend() to stop a running queue like we do today if using
> suspend blockers.
>
> This means that driver writers only targeting configurations
> where suspend blockers are used will probably not care about
> handling suspend() calls by trying to stop the queue or
> checking if there are things in it, because there will never
> be anything there.
>
> So while it is easy for me to have the driver work under both
> suspend blockers and runtime PM or common suspend(), the
> two configurations actually have different semantics of the
> suspend() call: in the suspend blockers case you don't need
> to check anything, just suspend, and in  the runtime PM
> case you have to check that you can actually suspend.
>
> Is this analysis correct?
>

Mostly, yes. With the current suspend blocker implementation, if you
only start blocking suspend from a user-space thread, or a freezable
kernel thread, then your suspend hook will never be called while you
block suspend unless you use forced suspend. If you for instance start
blocking suspend in response to an interrupt on the other hand, there
is no guarantee that drivers you depend on have not already been
suspended or that they will not suspend after you block suspend, so
you may have to implement a suspend hook to manually abort suspend
even when using suspend blockers. You can add a check if your suspend
blocker is active within your critical section in your suspend hook.
Our alarm driver does this.

> If yes then OK, it's not totally elegant but if that is
> where we have to go, I can live with it. There will likely
> be people who implement for only one or the other semantic
> behaviour, but we have worse things to cope with already.
>

Yes, but you can support both forced suspend and opportunistic suspend
in the same driver. Supporting both is harder if the suspend blocker
api is not in the mainline kernel.

-- 
Arve Hjønnevåg

Re: [PATCH 0/8] Suspend block api (version 6)

[Alan Stern]

On Thu, 27 May 2010, Linus WALLEIJ wrote:

> OK I understand it will be rather OK to do this. For example I have
> drivers/spi/amba-pl022.c which is quite clearly taking elements
> off a queue and transmitting them.
> 
> Currently, as we do not have suspend blockers, we try to stop the
> queue in suspend() and if that fails we return something negative
> so that the suspend() is blocked by the negative return code.

How can it fail?  Even if the queue can't be stopped immediately, can't
the suspend routine block for a few milliseconds until the queue can be 
stopped?

> Maybe the behaviour for an SPI bus should rather be to return
> -EBUSY as long as there are events in the queue. I don't quite
> know frankly.

No.  It should stop the queue.

Think of it this way: Somebody has just closed the lid of their 
laptop and thrown the computer into a backpack.  You don't want the 
machine to fail to suspend merely because an SPI queue wasn't empty.

> (We haven't added runtime PM yet, it will likely be the same
> thing.)

Runtime PM is very different.  It is supposed to take effect only when 
the device is idle.  So failing if the queue is non-empty makes sense.

> With suspend blockers we take a suspend blocker when we add
> elements to the queue and release it when the queue is empty.
> 
> But with suspend blockers the need for suspend() to check if
> the queue can be stopped or return -EBUSY if there are elements
> in it is not necessary: suspend() simply won't be called
> since a suspend blocker is taken.

No, that's completely wrong.  The user can tell the computer to suspend
at any time, whether or not any suspend blockers are enabled.  Think of
the laptop-in-the-backpack case.

> There is no way for
> suspend() to stop a running queue like we do today if using
> suspend blockers.

I don't know what you mean by this, but it doesn't sound right.

> This means that driver writers only targeting configurations
> where suspend blockers are used will probably not care about
> handling suspend() calls by trying to stop the queue or
> checking if there are things in it, because there will never
> be anything there.

You also have to handle races.  What happens if your suspend routine is 
called, and at that moment an interrupt occurs, causing the driver to 
add an entry to the queue?

You're trying to over-simplify this, probably because you're not 
thinking about it in the right way.  To put it bluntly, suspend 
blockers should not affect with your suspend/resume routines at all.

> So while it is easy for me to have the driver work under both
> suspend blockers and runtime PM or common suspend(), the
> two configurations actually have different semantics of the
> suspend() call: in the suspend blockers case you don't need
> to check anything, just suspend, and in  the runtime PM
> case you have to check that you can actually suspend.

The suspend-blockers case is exactly the same as the normal suspend 
case.  The runtime PM case does require extra checking.

> Is this analysis correct?

No, it's mostly wrong.

Alan Stern

Re: [PATCH 0/8] Suspend block api (version 6)

[Linus WALLEIJ]

[Alan]
> On Thu, 27 May 2010, Linus WALLEIJ wrote:
> 
> > OK I understand it will be rather OK to do this. For example I have
> > drivers/spi/amba-pl022.c which is quite clearly taking elements
> > off a queue and transmitting them.
> >
> > Currently, as we do not have suspend blockers, we try to stop the
> > queue in suspend() and if that fails we return something negative
> > so that the suspend() is blocked by the negative return code.
> 
> How can it fail?  Even if the queue can't be stopped immediately, can't
> the suspend routine block for a few milliseconds until the queue can be
> stopped?
> 
> > Maybe the behaviour for an SPI bus should rather be to return
> > -EBUSY as long as there are events in the queue. I don't quite
> > know frankly.
> 
> No.  It should stop the queue.
> 
> Think of it this way: Somebody has just closed the lid of their
> laptop and thrown the computer into a backpack.  You don't want the
> machine to fail to suspend merely because an SPI queue wasn't empty.

Not quite! On the other end of that SPI link is our power
regulators. (I'm not making this up, they are really there.) The events
in the queue may be the register writes that actually put the system
to low-power sleep.

But the apropriate action may likely be to wait for this queue to become
empty before returning 0 from suspend().

Yours,
Linus Walleij

Re: [PATCH 0/8] Suspend block api (version 6)

[tytso]

On Mon, May 17, 2010 at 09:12:03AM -0700, Kevin Hilman wrote:
> 
> The n900 *never* suspends.  It only uses dynamic PM + CPUidle.
> The droid uses opportunistic suspend (as well as dynamic PM + CPUidle)
> 
> I don't know of any more objective comparison of the two, but as a
> user of both devices I can say that the active usage is basically the
> same (around a day) and the idle use is similar as well, even though
> the Droid has a slightly bigger battery (1400 mAh vs. 1320 mAh.)....

Just for a bit of light amusement, although hopefully we've killed the
meme that other platforms have absolutely no problems in this area
without using something like suspend blockers, I offer for your
consideration this thread from maemo-developers:

http://lists.maemo.org/pipermail/maemo-developers/2010-May/026490.html

Note how users conflate battery lifetime after downloading a random
application with the platform being "stable".  I also was amazed that
the thread degenerated into trying to detect processes that are taking
90% of the CPU.  It's not necessary for a process to be constantly
running before it starts chewing up your battery, and if people think
the "blame the victim" trick works (``It's the user's fault for
"approving" the application by installing it!''), I suspect that
the platform will be not be very successful...

							- Ted