From: "Rafael J. Wysocki" <rjw@sisk.pl>
To: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>
Cc: rdunlap@xenotime.net, len.brown@intel.com, pavel@ucw.cz,
a.p.zijlstra@chello.nl, linux-pm@vger.kernel.org,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] Documentation/power: Update docs about suspend and CPU hotplug
Date: Wed, 12 Oct 2011 00:02:49 +0200 [thread overview]
Message-ID: <201110120002.49278.rjw@sisk.pl> (raw)
In-Reply-To: <20111011201958.10065.78949.stgit@srivatsabhat.in.ibm.com>
On Tuesday, October 11, 2011, Srivatsa S. Bhat wrote:
> Update the documentation about the interaction between the suspend (S3) call
> path and the CPU hotplug infrastructure.
> This patch focusses only on the activities of the freezer, cpu hotplug and
> the notifications involved. It outlines how regular CPU hotplug differs from
> the way it is invoked during suspend and also tries to explain the locking
> involved.
>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> ---
>
> Documentation/power/00-INDEX | 2
> Documentation/power/suspend-and-cpuhotplug.txt | 113 ++++++++++++++++++++++++
> 2 files changed, 115 insertions(+), 0 deletions(-)
> create mode 100644 Documentation/power/suspend-and-cpuhotplug.txt
>
> diff --git a/Documentation/power/00-INDEX b/Documentation/power/00-INDEX
> index 45e9d4a..a4d682f 100644
> --- a/Documentation/power/00-INDEX
> +++ b/Documentation/power/00-INDEX
> @@ -26,6 +26,8 @@ s2ram.txt
> - How to get suspend to ram working (and debug it when it isn't)
> states.txt
> - System power management states
> +suspend-and-cpuhotplug.txt
> + - Explains the interaction between Suspend-to-RAM (S3) and CPU hotplug
> swsusp-and-swap-files.txt
> - Using swap files with software suspend (to disk)
> swsusp-dmcrypt.txt
> diff --git a/Documentation/power/suspend-and-cpuhotplug.txt b/Documentation/power/suspend-and-cpuhotplug.txt
> new file mode 100644
> index 0000000..d0ba411
> --- /dev/null
> +++ b/Documentation/power/suspend-and-cpuhotplug.txt
> @@ -0,0 +1,113 @@
> +Interaction of Suspend code (S3) with the CPU hotplug infrastructure
> + (C) 2011 Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>, GPL
> +
> +
> +I. How does the Suspend-to-RAM code interact with CPU hotplug infrastructure?
> +
> +Well, a picture speaks more than a thousand words... So ASCII art follows :-)
> +
> +[This depicts the current design in the kernel, and focusses only on the
> +interactions between suspend call paths involving the freezer and cpu hotplug
> +and also tries to explain the locking involved. It also outlines the
> +notifications involved.]
> +
> +On a high level, the suspend-resume cycle goes like this:
> +
> +|Freeze| -> |Disable nonboot| -> |Do suspend| -> |Enable nonboot| -> |Thaw |
> +|tasks | | cpus | | | | cpus | |tasks|
> +
> +
> +More details follow:
> +
> +Regular CPU hotplug Suspend call path
> +------------------- ---------------------------
> +
> +Write 0 (or 1) to Write 'mem' to
> +/sys/devices/system/cpu/cpu*/online /sys/power/state
> + sysfs file syfs file
> + | |
> + | v
> + | Acquire pm_mutex lock
> + | |
> + | v
> + | Send PM_SUSPEND_PREPARE notifications
> + | |
> + | v
> + | Freeze tasks
OK, so something appears to be missing here. Namely, the task writing to
/sys/devices/system/cpu/cpu*/online should be frozen at this point or
suspend should be aborted. I suppose neither of these happens and I wonder
why exactly.
Thanks,
Rafael
> + | |
> + | |
> + v v
> + cpu_down() disable_nonboot_cpus() /*start*/
> + | |
> + v v
> +Acquire cpu_add_remove_lock Acquire cpu_add_remove_lock
> + | |
> + v v
> +If cpu_hotplug_disabled is 1 Iterate over CURRENTLY online CPUs
> + return gracefully |
> + | |
> + | | ----
> + v v |
> + \ / |
> + -------- -------- |
> + \ / |
> + -------- -------- |L
> + \____/ |
> + | |
> + v |O
> + _cpu_down() |
> + [This takes cpuhotplug.lock |
> + before taking down the CPU |
> + and releases it when done] |O
> + While it is at it, notifications |
> + are sent when notable events occur, |
> + by running all registered callbacks. |
> + | |O
> + / \ |
> + / \ |
> + < > |
> + _______________________/ \_____________________ |P
> + | | |
> + v v |
> +Release cpu_add_remove_lock Note down these cpus in |
> +[That's it!, for frozen_cpus mask ----
> + regular CPU hotplug] |
> + v
> + Disable regular cpu hotplug
> + by setting cpu_hotplug_disabled=1
> + |
> + v
> + Release cpu_add_remove_lock
> + |
> + v
> + /* disable_nonboot_cpus() complete */
> + |
> + v
> + Do suspend
> +
> +
> +Resuming back is likewise, with the counterparts being (in the order of
> +execution during resume):
> +* enable_nonboot_cpus() which involves:
> + | Acquire cpu_add_remove_lock
> + | Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug
> + | Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop]
> + | Release cpu_add_remove_lock
> + v
> +
> +* thaw tasks
> +* send PM_POST_SUSPEND notifications
> +* Release pm_mutex lock.
> +
> +It is to be noted here that the pm_mutex lock is acquired at the very
> +beginning, when we are just starting out to suspend, and then released only
> +after the entire cycle is complete (i.e., suspend + resume).
> +
> +
> +Important files and functions/entry points:
> +------------------------------------------
> +
> +kernel/power/process.c : freeze_processes(), thaw_processes()
> +kernel/power/suspend.c : suspend_prepare(), suspend_enter(), suspend_finish()
> +kernel/cpu.c: cpu_[up|down](), _cpu_[up|down](), [disable|enable]_nonboot_cpus()
> +
>
>
>
next prev parent reply other threads:[~2011-10-11 22:00 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-10-11 20:19 [PATCH] Documentation/power: Update docs about suspend and CPU hotplug Srivatsa S. Bhat
2011-10-11 22:02 ` Rafael J. Wysocki [this message]
2011-10-12 4:17 ` Srivatsa S. Bhat
2011-10-12 19:19 ` Rafael J. Wysocki
2011-10-12 21:13 ` Srivatsa S. Bhat
2011-10-14 6:24 ` [PATCH v2] " Srivatsa S. Bhat
2011-10-14 15:44 ` Alan Stern
2011-10-14 18:18 ` Srivatsa S. Bhat
2011-10-17 13:10 ` [PATCH v3] " Srivatsa S. Bhat
2011-10-19 22:08 ` Rafael J. Wysocki
2011-10-15 22:42 ` [PATCH] " Rafael J. Wysocki
2011-10-16 0:14 ` Srivatsa S. Bhat
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201110120002.49278.rjw@sisk.pl \
--to=rjw@sisk.pl \
--cc=a.p.zijlstra@chello.nl \
--cc=len.brown@intel.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=pavel@ucw.cz \
--cc=rdunlap@xenotime.net \
--cc=srivatsa.bhat@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox