[PATCH v2] Documentation/power: Update docs about suspend and CPU hotplug

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

Update the documentation about the interaction between the suspend (S3) call
path and the CPU hotplug infrastructure.
This patch focusses only on the activities of the freezer, cpu hotplug and
the notifications involved. It outlines how regular CPU hotplug differs from
the way it is invoked during suspend and also tries to explain the locking
involved.

v2: Clarified the question, to emphasize that the document explains only
    the difference (and similarity) in the two code paths but not what happens
    when race conditions occur between them.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 Documentation/power/00-INDEX                   |    2 
 Documentation/power/suspend-and-cpuhotplug.txt |  118 ++++++++++++++++++++++++
 2 files changed, 120 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/power/suspend-and-cpuhotplug.txt

diff --git a/Documentation/power/00-INDEX b/Documentation/power/00-INDEX
index 45e9d4a..a4d682f 100644
--- a/Documentation/power/00-INDEX
+++ b/Documentation/power/00-INDEX
@@ -26,6 +26,8 @@ s2ram.txt
 	- How to get suspend to ram working (and debug it when it isn't)
 states.txt
 	- System power management states
+suspend-and-cpuhotplug.txt
+	- Explains the interaction between Suspend-to-RAM (S3) and CPU hotplug
 swsusp-and-swap-files.txt
 	- Using swap files with software suspend (to disk)
 swsusp-dmcrypt.txt
diff --git a/Documentation/power/suspend-and-cpuhotplug.txt b/Documentation/power/suspend-and-cpuhotplug.txt
new file mode 100644
index 0000000..be10902
--- /dev/null
+++ b/Documentation/power/suspend-and-cpuhotplug.txt
@@ -0,0 +1,118 @@
+Interaction of Suspend code (S3) with the CPU hotplug infrastructure
+   (C) 2011 Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>, GPL
+
+
+I. How does the regular CPU hotplug code differ from how the Suspend-to-RAM
+   infrastructure uses it internally? And where do the two of them share
+   common code?
+
+Well, a picture speaks more than a thousand words... So ASCII art follows :-)
+
+[This depicts the current design in the kernel, and focusses only on the
+interactions involving the freezer and cpu hotplug and also tries to explain
+the locking involved. It outlines the notifications involved as well.
+But please note that here, only the call paths are illustrated, with the aim
+of describing where they take different paths and where they share code.
+What happens when regular CPU hotplug and Suspend-to-RAM race with each other
+is not depicted here.]
+
+On a high level, the suspend-resume cycle goes like this:
+
+|Freeze| -> |Disable nonboot| -> |Do suspend| -> |Enable nonboot| -> |Thaw |
+|tasks |    |     cpus      |    |          |    |     cpus     |    |tasks|
+
+
+More details follow:
+
+Regular CPU hotplug                                   Suspend call path
+-------------------                              ---------------------------
+
+Write 0 (or 1) to                                     Write 'mem' to
+/sys/devices/system/cpu/cpu*/online                   /sys/power/state
+     sysfs file                                          syfs file
+         |                                                   |
+         |                                                   v
+         |                                         Acquire pm_mutex lock
+         |                                                   |
+         |                                                   v
+         |                                Send PM_SUSPEND_PREPARE notifications
+         |                                                   |
+         |                                                   v
+         |                                              Freeze tasks
+         |                                                   |
+         |                                                   |
+         v                                                   v
+     cpu_down()                                 disable_nonboot_cpus() /*start*/
+         |                                                   |
+         v                                                   v
+Acquire cpu_add_remove_lock                     Acquire cpu_add_remove_lock
+         |                                                   |
+         v                                                   v
+If cpu_hotplug_disabled is 1                 Iterate over CURRENTLY online CPUs
+   return gracefully                                         |
+         |                                                   |
+         |                                                   |           ----
+         v                                                   v               |
+           \                                               /                 |
+             --------                             --------                   |
+                      \                         /                            |
+                        --------       --------                              |L
+                                 \____/                                      |
+                                   |                                         |
+                                   v                                         |O
+                              _cpu_down()                                    |
+                     [This takes cpuhotplug.lock                             |
+                      before taking down the CPU                             |
+                      and releases it when done]                             |O
+                   While it is at it, notifications                          |
+                   are sent when notable events occur,                       |
+                   by running all registered callbacks.                      |
+                                   |                                         |O
+                                  / \                                        |
+                                 /   \                                       |
+                                <     >                                      |
+        _______________________/       \_____________________                |P
+       |                                                     |               |
+       v                                                     v               |
+Release cpu_add_remove_lock                      Note down these cpus in     |
+[That's it!, for                                     frozen_cpus mask    ----
+ regular CPU hotplug]                                        |
+                                                             v
+                                                  Disable regular cpu hotplug
+                                              by setting cpu_hotplug_disabled=1
+                                                             |
+                                                             v
+                                                Release cpu_add_remove_lock
+                                                             |
+                                                             v
+                                          /* disable_nonboot_cpus() complete */
+                                                             |
+                                                             v
+                                                        Do suspend
+
+
+Resuming back is likewise, with the counterparts being (in the order of
+execution during resume):
+* enable_nonboot_cpus() which involves:
+   |  Acquire cpu_add_remove_lock
+   |  Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug
+   |  Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop]
+   |  Release cpu_add_remove_lock
+   v
+
+* thaw tasks
+* send PM_POST_SUSPEND notifications
+* Release pm_mutex lock.
+
+It is to be noted here that the pm_mutex lock is acquired at the very
+beginning, when we are just starting out to suspend, and then released only
+after the entire cycle is complete (i.e., suspend + resume).
+
+
+Important files and functions/entry points:
+------------------------------------------
+
+kernel/power/process.c : freeze_processes(), thaw_processes()
+kernel/power/suspend.c : suspend_prepare(), suspend_enter(), suspend_finish()
+kernel/cpu.c: cpu_[up|down](), _cpu_[up|down](), [disable|enable]_nonboot_cpus()
+