[PATCH v4 0/2] PM: runtime: Overhaul kerneldoc, runtime_pm.rst docs

[PATCH v4 0/2] PM: runtime: Overhaul kerneldoc, runtime_pm.rst docs

[Brian Norris]

The runtime PM documentation could use some improvements and additions,
to help guide people less familiar with the main runtime PM concepts and
its internal implementation details.

v1 is here:
https://lore.kernel.org/all/20260904212000.4167880-1-briannorris@chromium.org/

v2 is here:
https://lore.kernel.org/all/20260923174711.1283986-1-briannorris@chromium.org/

v3 is here:
https://lore.kernel.org/all/20260930025044.104824-1-briannorris@chromium.org/

Most of the v1-v3 series has been applied already. For the remaining
work:

 - Clarifying core concepts in the Introduction. (Pretty much all new
   readers I encounter have a hard time with the difference between
   "enabled", "allowed", and "active".)

 - Adding Example driver patterns -- because the API is so large and
   complicated, it can help to try to walk people through standard
   practices, and what everything means in context.

Feel free to add suggestions! Within reason, I'm open to tackling more
here, as I think many people have many valid perspectives on exactly why
and how the docs do or don't serve people well today. Or I can tackle
less, if you think some of my choices are not improvements.

Some possible follow-ups I'm toying with:

 * Slimming down the API might be better than heavily documenting it. A
   smaller API is a more digestible API.

   My only concrete next step: drop __pm_runtime_put_autosuspend(). Its
   last user is nearly gone:
   https://lore.kernel.org/all/20260806-smmu-rpm-v4-1-8183d007331c@oss.qualcomm.com/

   I could also see deprecating one of
   pm_runtime_put_sync{,_suspend,_autosuspend}(). They all do slightly
   different things, but I'm not sure every difference is actually fully
   intentional (or at least, not necessary).

 * Tweaking some of the behavior on pm_runtime_barrier(). Today, it's
   very asymmetric, as it prefers resume. But I believe there may be
   value in making it flush (not just cancel) pending suspend too. That
   may be in a future proposal; for now, I just try to make its
   asymmetry more clear in the docs.

 * Sand down some more rough edges on return codes. For example, it's
   very difficult to get any useful meaning out of pm_runtime_put_sync()
   return codes. There's a high chance that anyone trying to treat
   return codes as errors is inviting bugs. (Is -EAGAIN a failure?)

   Of course, the answer there is not "document it better" -- we can
   make it easier to use.

 * Adjust the way devm_pm_runtime_enable() works, specifically for
   remove()/teardown. Currently, this is very hard to use correctly --
   some common driver patterns may assume that a device will tear down
   while RPM_SUSPENDED; but that's not actually guaranteed. Notably,
   this makes some of the "Examples" section fairly tricky/subtle.

Regards,
Brian

Changes in v4:
 * Significantly overhaul introduction, based on Rafael's
   feedback/rewrite
 * Drop most implementation details (struct fields) and API names from
   intro
 * Expand the "in use" and dependencies concepts in the introduction

Changes in v3:
 * Rewrite Examples to avoid devm_* at first, to highlight correct
   approaches
 * Include error-handling and remove() in any probe() Examples
 * Advise more caution when using devm_*, as ordering issues are subtle

Changes in v2:
 * Major rework on Introduction section, especially regarding "enabled"
   and "active"
 * Add appropriate teardown to "Probe with Hardware Powered Off"
   Example, as the remove() + power-off behavior is subtle here, and
   easy to get wrong
 * Drop changes that are already applied
 * Add a few new fix patches, noticed while reviewing the rest
 * Move Introduction patch near the end of the series, as it is a likely
   target for further discussion and modification.
 * Correct Ulf's email address
 * CC linux-doc

Brian Norris (2):
  PM: runtime: Expand introduction with core concepts and structure
  PM: runtime: Add Example Driver Patterns section

 Documentation/power/runtime_pm.rst | 589 ++++++++++++++++++++++++++++-
 1 file changed, 568 insertions(+), 21 deletions(-)

-- 
2.56.0.rc1.315.gc6ed9934b7-goog

[PATCH v4 1/2] PM: runtime: Expand introduction with core concepts and structure

[Brian Norris]

I commonly see people have difficulty learning how runtime PM works
because of the following key points [*]:

 1) there are several boolean concepts in runtime PM, with somewhat
    similar meanings:

     enabled / disabled
     active / suspended
     allowed / forbidden

 2) if these concepts are documented at all, they're scattered across
    the kerneldoc or Documentation/

 3) the runtime_pm.rst docs don't make any attempt to ease a reader into
    understanding the concepts, and instead jump straight into how it's
    implemented (queues, 'struct device' fields, helpers).

Let's try to remedy this a bit by discussing the core concepts and
highlights at the top of the introduction, and introduce a few
sub-headings, so it's easier to navigate different aspects of the
introduction.

While shuffling the intro around, I also see that the existing text
largely mirrors the layout of the following sections (2, 3, and 4), but
does so out of order. Reorder those, and point to section numbers.

[*] In addition to API complexity. I count 61 pm_*() helpers, 7 of which
    are variations of put() and 8 of which are variations of get().

Signed-off-by: Brian Norris <briannorris@chromium.org>
---

Changes in v4:
 * Significantly overhaul introduction, based on Rafael's
   feedback/rewrite
 * Drop most implementation details (struct fields) and API names from
   intro
 * Expand the "in use" and dependencies concepts in the introduction

Changes in v2:
Address review feedback around descriptions of "enabled" and "active". I
know not every point of discussion was settled, but I hope this updated
version resolves many of them and provides a better basis for further
improvement.
 * Avoid calling enabled and active "orthogonal"
 * Describe more of their inter-relationship
 * Prioritize talking about "enabled" first, since that's the first
   concept a reader should know about
 * Brief mentions of parent/child and supplier/consumer, and dependency
   handling
 * Other tweaks

 Documentation/power/runtime_pm.rst | 114 ++++++++++++++++++++++++-----
 1 file changed, 95 insertions(+), 19 deletions(-)

diff --git a/Documentation/power/runtime_pm.rst b/Documentation/power/runtime_pm.rst
index 68903d93a3e7..e4407b687f8b 100644
--- a/Documentation/power/runtime_pm.rst
+++ b/Documentation/power/runtime_pm.rst
@@ -13,31 +13,107 @@ Runtime Power Management Framework for I/O Devices
 1. Introduction
 ===============
 
-Support for runtime power management (runtime PM) of I/O devices is provided
-at the power management core (PM core) level by means of:
-
-* The power management workqueue pm_wq in which bus types and device drivers can
-  put their PM-related work items.  It is strongly recommended that pm_wq be
-  used for queuing all work items related to runtime PM, because this allows
-  them to be synchronized with system-wide power transitions (suspend to RAM,
-  hibernation and resume from system sleep states).  pm_wq is declared in
-  include/linux/pm_runtime.h and defined in kernel/power/main.c.
-
-* A number of runtime PM fields in the 'power' member of 'struct device' (which
-  is of the type 'struct dev_pm_info', defined in include/linux/pm.h) that can
-  be used for synchronizing runtime PM operations with one another.
+Runtime power management (or runtime PM, sometimes also referred to as RPM)
+allows individual I/O devices that are not in active use to be put into
+low-power states, in which they may not be operational or even accessible, and
+go back to the fully operational state as needed.  Power can be reduced this
+way without waiting for a transition into a system-wide sleep state.
+
+Core Concepts
+-------------
+
+Runtime PM operates around a few device-level concepts — whether a device is
+**active** or **suspended**; whether runtime PM is **enabled** or **disabled**;
+whether runtime PM is **allowed** or **forbidden**; and whether a device is
+**in use**.
+
+* **Active**: Runtime PM tracks whether or not devices are in active use, which
+  generally means that they are accessible and operational, and may be depended
+  on by something (for example, other devices or user space).  This is
+  represented by the **active** meta-state.  By contrast, the **suspended**
+  meta-state represents a promise that the given device will not be accessed.
+  This covers devices in low-power states, but it also may include devices that
+  are not fully operational.  There are also two transient meta-states,
+  **suspending** and **resuming**, representing transitions between **active**
+  and **suspended** often referred to as *runtime suspend* and *runtime
+  resume*, or just *suspend* and *resume*, respectively.  Every device handled
+  by runtime PM is in one of these four meta-states at any time, and its
+  meta-state is expected to reflect its actual physical configuration (that is,
+  for instance, if the device is not accessible, it must not be **active**).
+  The term "runtime PM status" used in what follows refers to the meta-states
+  described above.
+
+* **Enabled**: In order to track the runtime PM status and carry out
+  transitions between **active** and **suspended**, runtime PM needs to be
+  **enabled** for the given device.  If it is **disabled**, runtime PM
+  generally leaves the device alone.  This means in particular that the runtime
+  PM status of a device and its actual physical configuration may get out of
+  sync after disabling runtime PM for it.  Additionally, the runtime PM status
+  of a device may need to be explicitly adjusted to reflect its current
+  physical configuration before enabling runtime PM for it (e.g., during device
+  probe).  Once runtime PM is **enabled**, it begins managing PM status —
+  performing *runtime suspend* and *runtime resume* transitions based on its
+  understanding of whether a device is in use.
+
+  As a rule, all devices are initialized with runtime PM **disabled**, though
+  some bus types (such as PCI) manage some RPM initialization automatically,
+  and therefore probe devices in an **enabled** state.  For other devices,
+  drivers must enable runtime PM on their own to opt in.
+
+* **Allowed**: Runtime PM may be **forbidden** even though it has been enabled.
+  Doing so causes the given device to transition into the **active** meta-state
+  and prevents it from being suspended.  In other words, a device with
+  forbidden runtime PM remains **active** at least until runtime PM is
+  **allowed** for it again.  This mechanism can be managed by in-kernel APIs,
+  but more importantly, it is also available to user space via the
+  ``/sys/devices/.../power/control`` sysfs attribute of the given device.
+  Namely, writing ``on`` to that attribute forbids runtime PM, and writing
+  ``auto`` allows it.  User space may modify that attribute at any time, so the
+  ability to transition a device into the **suspended** meta-state via runtime
+  PM must not be relied on for correctness.
+
+* **In use**: Runtime PM determines whether a device should be transitioned to
+  an **active** state by its understanding of whether a device is in use.  The
+  first way a device is considered **in use** is by its own usage count — a
+  reference counter driven by a variety of get()/put() APIs.  Additionally, a
+  device may be considered **in use** by having active dependents — either
+  child devices, or device-linked consumers.  When a device is no longer in
+  use, runtime PM may choose to transition it to the **suspended** state.
+
+  Thus, when a device moves from **suspended** to **active**, it may also cause
+  its dependencies to resume, unless they are **active** already.  Conversely,
+  suspending a device may also cause its otherwise-unused dependencies to
+  suspend.
+
+* **Autosuspend**: Runtime PM also supports a feature called "autosuspend."
+  This is different than the ``control`` notion of "auto" (i.e., **allowed**).
+  Autosuspend is covered in `Section 9`_.
+
+Implementation Structure
+------------------------
+
+Support for runtime power management is provided at the power management core
+(PM core) level by means of:
 
 * Three device runtime PM callbacks in 'struct dev_pm_ops' (defined in
-  include/linux/pm.h).
+  include/linux/pm.h). See `Section 2`_.
+
+* A number of runtime PM fields in the 'power' member of 'struct device' that
+  can be used for synchronizing runtime PM operations with one another. These
+  are covered in `Section 3`_.
 
 * A set of helper functions defined in drivers/base/power/runtime.c that can be
   used for carrying out runtime PM operations in such a way that the
-  synchronization between them is taken care of by the PM core.  Bus types and
-  device drivers are encouraged to use these functions.
+  synchronization between them is taken care of by the PM core. Bus types and
+  device drivers are encouraged to use these functions. They are covered in
+  `Section 4`_.
 
-The runtime PM callbacks present in 'struct dev_pm_ops', the device runtime PM
-fields of 'struct dev_pm_info' and the core helper functions provided for
-runtime PM are described below.
+* The power management workqueue pm_wq in which bus types and device drivers can
+  put their PM-related work items. It is strongly recommended that pm_wq be
+  used for queuing all work items related to runtime PM, because this allows
+  them to be synchronized with system-wide power transitions (suspend to RAM,
+  hibernation and resume from system sleep states). pm_wq is declared in
+  include/linux/pm_runtime.h and defined in kernel/power/main.c.
 
 .. _Section 2:
 
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

[PATCH v4 2/2] PM: runtime: Add Example Driver Patterns section

[Brian Norris]

The runtime PM API surface is pretty large, but there are a few common
patterns that many drivers should follow. Add some illustrative
examples, to help guide the most common audience for runtime PM docs --
driver writers.

Signed-off-by: Brian Norris <briannorris@chromium.org>
---

Changes in v4:
 * "state" to "status", to be consistent with Introduction terminology

Changes in v3:
 * Rewrite Examples to avoid devm_* at first, to highlight correct
   approaches
 * Include error-handling and remove() in any probe() Examples
 * Advise more caution when using devm_*, as ordering issues are subtle

Changes in v2:
 * Add appropriate teardown to "Probe with Hardware Powered Off"
   Example, as the remove() + power-off behavior is subtle here, and
   easy to get wrong

 Documentation/power/runtime_pm.rst | 475 ++++++++++++++++++++++++++++-
 1 file changed, 473 insertions(+), 2 deletions(-)

diff --git a/Documentation/power/runtime_pm.rst b/Documentation/power/runtime_pm.rst
index e4407b687f8b..cb25e0bb4ca7 100644
--- a/Documentation/power/runtime_pm.rst
+++ b/Documentation/power/runtime_pm.rst
@@ -429,7 +429,8 @@ in ->probe(). Usually this means calling pm_runtime_disable(),
 pm_runtime_dont_use_autosuspend() etc. Alternatively, drivers can use
 devm_pm_runtime_enable() during probe, which automatically takes care of
 calling pm_runtime_disable() and pm_runtime_dont_use_autosuspend() upon driver
-detachment.
+detachment. See also the Examples in `Section 10`_ for some effective
+strategies for driver teardown.
 
 The user space can effectively disallow the driver of the device to power manage
 it at run time by changing the value of its /sys/devices/.../power/control
@@ -654,7 +655,9 @@ The implementation is well suited for asynchronous use in interrupt contexts.
 However such use inevitably involves races, because the PM core can't
 synchronize ->runtime_suspend() callbacks with the arrival of I/O requests.
 This synchronization must be handled by the driver, using its private lock.
-Here is a schematic pseudo-code example::
+Here is a schematic pseudo-code example:
+
+.. code-block:: c
 
 	foo_read_or_write(struct foo_priv *foo, void *data)
 	{
@@ -720,3 +723,471 @@ pm_runtime_autosuspend_expiration() from within the ->runtime_suspend()
 callback while holding its private lock.  If the function returns a nonzero
 value then the delay has not yet expired and the callback should return
 -EAGAIN.
+
+.. _Section 10:
+
+10. Example Driver Patterns
+===========================
+
+The runtime PM API is large and complex, but most device drivers follow a small
+set of canonical patterns when interacting with runtime PM. This section
+illustrates standard patterns for device probing, performing I/O, and handling
+interrupts.
+
+Probe and Initialization
+------------------------
+
+Basic Probe
+~~~~~~~~~~~
+
+A driver that powers on its hardware during probe and does not use autosuspend
+can initialize runtime PM using standard helpers:
+
+.. code-block:: c
+
+	static int foo_probe(struct platform_device *pdev)
+	{
+		struct device *dev = &pdev->dev;
+		struct foo_priv *priv;
+		int ret;
+
+		priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+		if (!priv)
+			return -ENOMEM;
+
+		platform_set_drvdata(pdev, priv);
+
+		/* Power on and initialize hardware registers... */
+
+		/*
+		 * The code above left hardware powered on and operational, so
+		 * tell the PM core that the device is active before enabling
+		 * runtime PM.
+		 */
+		pm_runtime_set_active(dev);
+		pm_runtime_enable(dev);
+
+		/*
+		 * Perform remaining driver setup (e.g. register interrupts, device nodes).
+		 * If any step fails after enabling runtime PM:
+		 */
+		ret = foo_register_device(priv);
+		if (ret)
+			goto err_pm_disable;
+
+		/*
+		 * Upon successful return from ->probe(), the driver core
+		 * automatically executes pm_request_idle(dev), allowing the
+		 * device to suspend asynchronously if its usage counter is zero.
+		 */
+		return 0;
+
+	err_pm_disable:
+		pm_runtime_disable(dev);
+		if (!pm_runtime_status_suspended(dev)) {
+			/*
+			 * Power back off, in case device did not suspend yet.
+			 * For example, leverage your .runtime_suspend()
+			 * implementation:
+			 */
+			foo_runtime_suspend(dev);
+			pm_runtime_set_suspended(dev);
+		}
+		return ret;
+	}
+
+	static void foo_remove(struct platform_device *pdev)
+	{
+		struct device *dev = &pdev->dev;
+		struct foo_priv *priv = platform_get_drvdata(pdev);
+
+		foo_unregister_device(priv);
+
+		pm_runtime_disable(dev);
+		/*
+		 * The runtime PM status is not guaranteed at this point. We
+		 * could be either suspended or active, so we need to handle
+		 * both.
+		 */
+		if (!pm_runtime_status_suspended(dev)) {
+			/* Power back off, in case device did not suspend yet. */
+			foo_runtime_suspend(dev);
+			pm_runtime_set_suspended(dev);
+		}
+	}
+
+Drivers using this pattern should ensure that hardware is powered off cleanly
+upon driver unbind or probe error. If the device was still active when detached
+(for example, if user space configured ``/sys/devices/.../power/control`` to
+``on``, or if the device has not yet idled), ``foo_remove()`` and
+``err_pm_disable`` disable runtime PM, check whether the device is not yet
+suspended using pm_runtime_status_suspended(), and manually power the device
+back off.
+
+Note that device-managed helpers such as devm_pm_runtime_enable() and
+devm_pm_runtime_set_active_enabled() exist, but they only call
+pm_runtime_disable() upon driver unbind; they do not take care of device
+quiesce and power-off. See the `Autosuspend and Device-Managed Probe`_ section
+below for how to encapsulate complete teardown in a custom devm action.
+
+Probe with Hardware Powered Off
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Many drivers prefer to keep hardware powered off or in low power until
+actually needed, avoiding duplicate power sequencing logic between ->probe()
+and ->runtime_resume(). Because the initial runtime PM status of a device is
+suspended by default, the driver can enable runtime PM directly and rely on
+pm_runtime_resume_and_get() to trigger the ->runtime_resume() callback when
+probe needs to access hardware:
+
+.. code-block:: c
+
+	static int foo_runtime_suspend(struct device *dev)
+	{
+		struct foo_priv *priv = dev_get_drvdata(dev);
+
+		clk_disable_unprepare(priv->clk);
+		regulator_disable(priv->supply);
+
+		return 0;
+	}
+
+	static int foo_runtime_resume(struct device *dev)
+	{
+		struct foo_priv *priv = dev_get_drvdata(dev);
+		int ret;
+
+		ret = regulator_enable(priv->supply);
+		if (ret)
+			return ret;
+
+		ret = clk_prepare_enable(priv->clk);
+		if (ret) {
+			regulator_disable(priv->supply);
+			return ret;
+		}
+
+		return 0;
+	}
+
+	static int foo_probe(struct platform_device *pdev)
+	{
+		struct device *dev = &pdev->dev;
+		struct foo_priv *priv;
+		int ret;
+
+		priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+		if (!priv)
+			return -ENOMEM;
+
+		platform_set_drvdata(pdev, priv);
+
+		/* Acquire regulators, clocks, GPIOs, and register map... */
+
+		/*
+		 * Hardware starts powered off. The default state is
+		 * RPM_SUSPENDED, so no need for:
+		 * pm_runtime_set_suspended(dev);
+		 */
+
+		pm_runtime_enable(dev);
+
+		/*
+		 * Power on the device via ->runtime_resume() to verify device
+		 * ID or perform initial hardware configuration.
+		 */
+		ret = pm_runtime_resume_and_get(dev);
+		if (ret < 0)
+			goto err_pm_disable;
+
+		ret = foo_verify_hardware_id(priv);
+		if (ret) {
+			pm_runtime_put_sync(dev);
+			goto err_pm_disable;
+		}
+
+		/*
+		 * Drop the usage counter, allowing ->runtime_suspend() to
+		 * power off the device until an I/O request arrives.
+		 */
+		pm_runtime_put(dev);
+
+		return 0;
+
+	err_pm_disable:
+		pm_runtime_disable(dev);
+		if (!pm_runtime_status_suspended(dev)) {
+			foo_runtime_suspend(dev);
+			pm_runtime_set_suspended(dev);
+		}
+		return ret;
+	}
+
+	static void foo_remove(struct platform_device *pdev)
+	{
+		struct device *dev = &pdev->dev;
+
+		pm_runtime_disable(dev);
+		if (!pm_runtime_status_suspended(dev)) {
+			foo_runtime_suspend(dev);
+			pm_runtime_set_suspended(dev);
+		}
+	}
+
+Note that this pattern requires ``CONFIG_PM``. When ``CONFIG_PM`` is
+disabled, pm_runtime_resume_and_get() returns 0 without calling
+->runtime_resume(), leaving hardware unpowered. Drivers using this pattern
+should typically depend on ``CONFIG_PM``.
+
+Similar to the Basic Probe example, drivers using this pattern also must
+exercise care when handling probe() errors or performing unbind/remove(), to
+ensure they power off and clean up resources that they manage with runtime
+suspend/resume. Typically, this may mean explicitly powering off if not already
+suspended.
+
+Autosuspend and Device-Managed Probe
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the driver uses autosuspend, it configures the autosuspend delay and enables
+autosuspend before enabling runtime PM.
+
+Additionally, drivers can use device-managed (``devm_*``) helpers to clean up
+resources on probe failure or driver unbind. This provides an alternative to
+writing complicated ``goto``-based error paths, with similar teardown code in
+remove(). Note that drivers must still exercise care to avoid assuming a
+particular runtime PM status in these cases; a device typically may be either
+**active** or **suspended**, and so ``devm``-based teardown code must be
+written accordingly.
+
+.. code-block:: c
+
+	/* Tear-down action, for error handling or driver unbind. */
+	static void foo_poweroff_action(void *data)
+	{
+		struct device *dev = data;
+
+		/* Runtime PM is already disabled by this point. */
+
+		if (!pm_runtime_status_suspended(dev)) {
+			/*
+			 * Power device back off, in case device was left active.
+			 * e.g., leverage .runtime_suspend() routine:
+			 */
+			foo_runtime_suspend(dev);
+
+			pm_runtime_set_suspended(dev);
+		}
+	}
+
+	static int foo_probe(struct platform_device *pdev)
+	{
+		struct device *dev = &pdev->dev;
+		struct foo_priv *priv;
+		int ret;
+
+		priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+		if (!priv)
+			return -ENOMEM;
+
+		platform_set_drvdata(pdev, priv);
+
+		/* Power on and initialize hardware registers... */
+
+		pm_runtime_set_active(dev);
+
+		/*
+		 * Guarantee hardware is powered off upon unbind or probe error.
+		 * Register this action before devm_pm_runtime_enable() to
+		 * ensure runtime PM status is frozen by the time we tear down
+		 * here.
+		 */
+		ret = devm_add_action_or_reset(dev, foo_poweroff_action, dev);
+		if (ret)
+			return ret;
+
+		/* Set autosuspend delay (e.g., 2000 ms) and enable autosuspend. */
+		pm_runtime_set_autosuspend_delay(dev, 2000);
+		pm_runtime_use_autosuspend(dev);
+
+		/*
+		 * Update last busy timestamp so the driver core's post-probe
+		 * pm_request_idle() respects the autosuspend delay.
+		 */
+		pm_runtime_mark_last_busy(dev);
+
+		/*
+		 * devm_pm_runtime_enable() ensures that pm_runtime_disable()
+		 * and pm_runtime_dont_use_autosuspend() are called upon driver
+		 * unbind.
+		 */
+		ret = devm_pm_runtime_enable(dev);
+		if (ret)
+			return ret;
+
+		return 0;
+	}
+
+Key points:
+
+* **Guaranteed Power-off on Detach**: Again similar to the Basic Probe example,
+  we must exercise care in case the device was still active during probe
+  failure or driver unbind. In this case, we encapsulate the teardown using
+  device-managed actions.
+* **LIFO Teardown Ordering**: Because devm releases resources in reverse
+  order of registration (LIFO), registering foo_poweroff_action() before
+  devm_pm_runtime_enable() ensures that runtime PM is disabled first, and the
+  device is suspended *before* devm disables any clocks or regulators
+  acquired earlier in ->probe().
+* **Omission of ->remove()**: If all other resources in the driver are managed
+  with ``devm`` helpers, this eliminates both manual ``goto``-based unwinding
+  in ->probe() and the need for an explicit ->remove() callback.
+
+Performing I/O Operations
+-------------------------
+
+Before accessing hardware registers or initiating I/O transfers, drivers must
+ensure the device is active by calling pm_runtime_resume_and_get() or similar.
+
+Basic I/O
+~~~~~~~~~
+
+For devices without autosuspend, work completion is signaled with
+pm_runtime_put(), which drops the usage counter and queues an asynchronous idle
+check once the counter reaches zero:
+
+.. code-block:: c
+
+	int foo_do_transfer(struct foo_priv *priv, void *buf, size_t count)
+	{
+		int ret;
+
+		ret = pm_runtime_resume_and_get(priv->dev);
+		if (ret < 0)
+			return ret;
+
+		/* Access hardware registers or perform data transfer... */
+		ret = foo_hardware_transfer(priv, buf, count);
+
+		/*
+		 * Drop usage counter and request asynchronous idle check (and
+		 * suspend, if possible).
+		 */
+		pm_runtime_put(priv->dev);
+
+		return ret;
+	}
+
+Autosuspend I/O
+~~~~~~~~~~~~~~~
+
+For devices using autosuspend, work completion is signaled with
+pm_runtime_put_autosuspend(), which drops the usage counter and defers
+suspension until the autosuspend delay expires:
+
+.. code-block:: c
+
+	int foo_do_transfer(struct foo_priv *priv, void *buf, size_t count)
+	{
+		int ret;
+
+		ret = pm_runtime_resume_and_get(priv->dev);
+		if (ret < 0)
+			return ret;
+
+		/* Access hardware registers or perform data transfer... */
+		ret = foo_hardware_transfer(priv, buf, count);
+
+		/*
+		 * Drop the usage counter and schedule an autosuspend once
+		 * the delay expires. Note that pm_runtime_put_autosuspend()
+		 * updates the last-access timestamp automatically.
+		 */
+		pm_runtime_put_autosuspend(priv->dev);
+
+		return ret;
+	}
+
+Synchronous Completion
+~~~~~~~~~~~~~~~~~~~~~~
+
+When immediate suspension is desired -- such as before unregistering a
+device or during shutdown -- synchronous put helpers can be used instead of
+their asynchronous counterparts. Which helper to use depends on whether
+autosuspend is configured:
+
+* For non-autosuspend devices, use pm_runtime_put_sync().
+* For devices that use autosuspend, use pm_runtime_put_sync_suspend(), which
+  ignores any configured autosuspend delay and forces immediate suspension.
+
+However, note several important caveats when relying on synchronous runtime
+PM helpers for power-down:
+
+* **Parents and Suppliers**: While the target device itself is suspended
+  synchronously, the PM core handles idle notifications for parents and
+  device link suppliers asynchronously. As a result, parent devices or
+  power domain suppliers are not guaranteed to be powered off when the
+  function returns.
+* **User Policy ("Forbidden")**: Runtime PM helpers respect system policy.
+  If user space has set ``/sys/devices/.../power/control`` to ``on``
+  (pm_runtime_forbid()), the PM core holds an extra reference on the
+  device, meaning dropping the driver's usage counter will not trigger a
+  suspend.
+
+Because of these constraints, synchronous put helpers may not be suitable
+when a driver functionally requires hardware to be powered off
+synchronously (for example, to perform a hardware reset or power cycle).
+Such requirements may necessitate other methods, such as disabling runtime
+PM with pm_runtime_disable() and explicitly executing the hardware
+power-down sequence.
+
+Interrupt Handling with Conditional Get
+---------------------------------------
+
+Interrupt handlers (especially in atomic or hardirq context) cannot typically
+invoke pm_runtime_resume_and_get(), because runtime-resume may sleep. Moreover,
+if an interrupt arrives while the device is suspended or transitioning to low
+power (e.g., on a shared interrupt line or spurious wakeups), attempting to
+read hardware registers could trigger a bus fault or system hang.
+
+To handle this safely, drivers can conditionally acquire a runtime PM reference
+using pm_runtime_get_if_in_use() or pm_runtime_get_if_active():
+
+.. code-block:: c
+
+	static irqreturn_t foo_irq_handler(int irq, void *dev_id)
+	{
+		struct foo_priv *priv = dev_id;
+		irqreturn_t ret = IRQ_NONE;
+
+		/*
+		 * Check if the device is active before reading hardware
+		 * registers. If the device is suspended, this interrupt
+		 * cannot belong to us (or was already serviced).
+		 *
+		 * Note that this also will drop interrupts while runtime PM is
+		 * disabled.
+		 */
+		if (pm_runtime_get_if_active(priv->dev) <= 0)
+			return IRQ_NONE;
+
+		/* Hardware is active and usage count is incremented */
+		if (foo_has_pending_irq(priv)) {
+			foo_service_irq(priv);
+			ret = IRQ_HANDLED;
+		}
+
+		/*
+		 * Release the reference acquired by pm_runtime_get_if_active().
+		 * For autosuspend devices, use pm_runtime_put_autosuspend();
+		 * for non-autosuspend devices, use pm_runtime_put().
+		 */
+		pm_runtime_put_autosuspend(priv->dev);
+
+		return ret;
+	}
+
+Both pm_runtime_get_if_in_use() and pm_runtime_get_if_active() are safe to use
+from an interrupt routine. One example where a device might be active but not
+"in use" is if autosuspend is used. A device will stay active for a while with
+no users. If interrupts should still be serviced for a device in this state,
+pm_runtime_get_if_active() should be used.
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

Re: [PATCH v4 1/2] PM: runtime: Expand introduction with core concepts and structure

[Rafael J. Wysocki (Intel)]

On Thu, Oct 1, 2026 at 5:51 PM Brian Norris <briannorris@chromium.org> wrote:
>
> I commonly see people have difficulty learning how runtime PM works
> because of the following key points [*]:
>
>  1) there are several boolean concepts in runtime PM, with somewhat
>     similar meanings:
>
>      enabled / disabled
>      active / suspended
>      allowed / forbidden
>
>  2) if these concepts are documented at all, they're scattered across
>     the kerneldoc or Documentation/
>
>  3) the runtime_pm.rst docs don't make any attempt to ease a reader into
>     understanding the concepts, and instead jump straight into how it's
>     implemented (queues, 'struct device' fields, helpers).
>
> Let's try to remedy this a bit by discussing the core concepts and
> highlights at the top of the introduction, and introduce a few
> sub-headings, so it's easier to navigate different aspects of the
> introduction.
>
> While shuffling the intro around, I also see that the existing text
> largely mirrors the layout of the following sections (2, 3, and 4), but
> does so out of order. Reorder those, and point to section numbers.
>
> [*] In addition to API complexity. I count 61 pm_*() helpers, 7 of which
>     are variations of put() and 8 of which are variations of get().
>
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> ---
>
> Changes in v4:
>  * Significantly overhaul introduction, based on Rafael's
>    feedback/rewrite
>  * Drop most implementation details (struct fields) and API names from
>    intro
>  * Expand the "in use" and dependencies concepts in the introduction
>
> Changes in v2:
> Address review feedback around descriptions of "enabled" and "active". I
> know not every point of discussion was settled, but I hope this updated
> version resolves many of them and provides a better basis for further
> improvement.
>  * Avoid calling enabled and active "orthogonal"
>  * Describe more of their inter-relationship
>  * Prioritize talking about "enabled" first, since that's the first
>    concept a reader should know about
>  * Brief mentions of parent/child and supplier/consumer, and dependency
>    handling
>  * Other tweaks
>
>  Documentation/power/runtime_pm.rst | 114 ++++++++++++++++++++++++-----
>  1 file changed, 95 insertions(+), 19 deletions(-)
>
> diff --git a/Documentation/power/runtime_pm.rst b/Documentation/power/runtime_pm.rst
> index 68903d93a3e7..e4407b687f8b 100644
> --- a/Documentation/power/runtime_pm.rst
> +++ b/Documentation/power/runtime_pm.rst
> @@ -13,31 +13,107 @@ Runtime Power Management Framework for I/O Devices
>  1. Introduction
>  ===============
>
> -Support for runtime power management (runtime PM) of I/O devices is provided
> -at the power management core (PM core) level by means of:
> -
> -* The power management workqueue pm_wq in which bus types and device drivers can
> -  put their PM-related work items.  It is strongly recommended that pm_wq be
> -  used for queuing all work items related to runtime PM, because this allows
> -  them to be synchronized with system-wide power transitions (suspend to RAM,
> -  hibernation and resume from system sleep states).  pm_wq is declared in
> -  include/linux/pm_runtime.h and defined in kernel/power/main.c.
> -
> -* A number of runtime PM fields in the 'power' member of 'struct device' (which
> -  is of the type 'struct dev_pm_info', defined in include/linux/pm.h) that can
> -  be used for synchronizing runtime PM operations with one another.
> +Runtime power management (or runtime PM, sometimes also referred to as RPM)
> +allows individual I/O devices that are not in active use to be put into
> +low-power states, in which they may not be operational or even accessible, and
> +go back to the fully operational state as needed.  Power can be reduced this
> +way without waiting for a transition into a system-wide sleep state.
> +
> +Core Concepts
> +-------------
> +
> +Runtime PM operates around a few device-level concepts — whether a device is
> +**active** or **suspended**; whether runtime PM is **enabled** or **disabled**;
> +whether runtime PM is **allowed** or **forbidden**; and whether a device is
> +**in use**.
> +
> +* **Active**: Runtime PM tracks whether or not devices are in active use, which
> +  generally means that they are accessible and operational, and may be depended
> +  on by something (for example, other devices or user space).  This is
> +  represented by the **active** meta-state.  By contrast, the **suspended**
> +  meta-state represents a promise that the given device will not be accessed.
> +  This covers devices in low-power states, but it also may include devices that
> +  are not fully operational.  There are also two transient meta-states,
> +  **suspending** and **resuming**, representing transitions between **active**
> +  and **suspended** often referred to as *runtime suspend* and *runtime
> +  resume*, or just *suspend* and *resume*, respectively.  Every device handled
> +  by runtime PM is in one of these four meta-states at any time, and its
> +  meta-state is expected to reflect its actual physical configuration (that is,
> +  for instance, if the device is not accessible, it must not be **active**).
> +  The term "runtime PM status" used in what follows refers to the meta-states
> +  described above.
> +
> +* **Enabled**: In order to track the runtime PM status and carry out
> +  transitions between **active** and **suspended**, runtime PM needs to be
> +  **enabled** for the given device.  If it is **disabled**, runtime PM
> +  generally leaves the device alone.  This means in particular that the runtime
> +  PM status of a device and its actual physical configuration may get out of
> +  sync after disabling runtime PM for it.  Additionally, the runtime PM status
> +  of a device may need to be explicitly adjusted to reflect its current
> +  physical configuration before enabling runtime PM for it (e.g., during device
> +  probe).  Once runtime PM is **enabled**, it begins managing PM status —
> +  performing *runtime suspend* and *runtime resume* transitions based on its
> +  understanding of whether a device is in use.
> +
> +  As a rule, all devices are initialized with runtime PM **disabled**, though
> +  some bus types (such as PCI) manage some RPM initialization automatically,
> +  and therefore probe devices in an **enabled** state.  For other devices,
> +  drivers must enable runtime PM on their own to opt in.
> +
> +* **Allowed**: Runtime PM may be **forbidden** even though it has been enabled.
> +  Doing so causes the given device to transition into the **active** meta-state
> +  and prevents it from being suspended.  In other words, a device with
> +  forbidden runtime PM remains **active** at least until runtime PM is
> +  **allowed** for it again.  This mechanism can be managed by in-kernel APIs,
> +  but more importantly, it is also available to user space via the
> +  ``/sys/devices/.../power/control`` sysfs attribute of the given device.
> +  Namely, writing ``on`` to that attribute forbids runtime PM, and writing
> +  ``auto`` allows it.  User space may modify that attribute at any time, so the
> +  ability to transition a device into the **suspended** meta-state via runtime
> +  PM must not be relied on for correctness.
> +
> +* **In use**: Runtime PM determines whether a device should be transitioned to
> +  an **active** state by its understanding of whether a device is in use.  The
> +  first way a device is considered **in use** is by its own usage count — a
> +  reference counter driven by a variety of get()/put() APIs.  Additionally, a
> +  device may be considered **in use** by having active dependents — either
> +  child devices, or device-linked consumers.  When a device is no longer in
> +  use, runtime PM may choose to transition it to the **suspended** state.
> +
> +  Thus, when a device moves from **suspended** to **active**, it may also cause
> +  its dependencies to resume, unless they are **active** already.  Conversely,
> +  suspending a device may also cause its otherwise-unused dependencies to
> +  suspend.
> +
> +* **Autosuspend**: Runtime PM also supports a feature called "autosuspend."
> +  This is different than the ``control`` notion of "auto" (i.e., **allowed**).
> +  Autosuspend is covered in `Section 9`_.
> +
> +Implementation Structure
> +------------------------
> +
> +Support for runtime power management is provided at the power management core
> +(PM core) level by means of:
>
>  * Three device runtime PM callbacks in 'struct dev_pm_ops' (defined in
> -  include/linux/pm.h).
> +  include/linux/pm.h). See `Section 2`_.
> +
> +* A number of runtime PM fields in the 'power' member of 'struct device' that
> +  can be used for synchronizing runtime PM operations with one another. These
> +  are covered in `Section 3`_.
>
>  * A set of helper functions defined in drivers/base/power/runtime.c that can be
>    used for carrying out runtime PM operations in such a way that the
> -  synchronization between them is taken care of by the PM core.  Bus types and
> -  device drivers are encouraged to use these functions.
> +  synchronization between them is taken care of by the PM core. Bus types and
> +  device drivers are encouraged to use these functions. They are covered in
> +  `Section 4`_.
>
> -The runtime PM callbacks present in 'struct dev_pm_ops', the device runtime PM
> -fields of 'struct dev_pm_info' and the core helper functions provided for
> -runtime PM are described below.
> +* The power management workqueue pm_wq in which bus types and device drivers can
> +  put their PM-related work items. It is strongly recommended that pm_wq be
> +  used for queuing all work items related to runtime PM, because this allows
> +  them to be synchronized with system-wide power transitions (suspend to RAM,
> +  hibernation and resume from system sleep states). pm_wq is declared in
> +  include/linux/pm_runtime.h and defined in kernel/power/main.c.
>
>  .. _Section 2:
>
> --

Applied as 7.4 material, thanks!

Re: [PATCH v4 2/2] PM: runtime: Add Example Driver Patterns section

[Rafael J. Wysocki (Intel)]

On Thu, Oct 1, 2026 at 5:51 PM Brian Norris <briannorris@chromium.org> wrote:
>
> The runtime PM API surface is pretty large, but there are a few common
> patterns that many drivers should follow. Add some illustrative
> examples, to help guide the most common audience for runtime PM docs --
> driver writers.
>
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> ---
>
> Changes in v4:
>  * "state" to "status", to be consistent with Introduction terminology
>
> Changes in v3:
>  * Rewrite Examples to avoid devm_* at first, to highlight correct
>    approaches
>  * Include error-handling and remove() in any probe() Examples
>  * Advise more caution when using devm_*, as ordering issues are subtle
>
> Changes in v2:
>  * Add appropriate teardown to "Probe with Hardware Powered Off"
>    Example, as the remove() + power-off behavior is subtle here, and
>    easy to get wrong
>
>  Documentation/power/runtime_pm.rst | 475 ++++++++++++++++++++++++++++-
>  1 file changed, 473 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/power/runtime_pm.rst b/Documentation/power/runtime_pm.rst
> index e4407b687f8b..cb25e0bb4ca7 100644
> --- a/Documentation/power/runtime_pm.rst
> +++ b/Documentation/power/runtime_pm.rst
> @@ -429,7 +429,8 @@ in ->probe(). Usually this means calling pm_runtime_disable(),
>  pm_runtime_dont_use_autosuspend() etc. Alternatively, drivers can use
>  devm_pm_runtime_enable() during probe, which automatically takes care of
>  calling pm_runtime_disable() and pm_runtime_dont_use_autosuspend() upon driver
> -detachment.
> +detachment. See also the Examples in `Section 10`_ for some effective
> +strategies for driver teardown.
>
>  The user space can effectively disallow the driver of the device to power manage
>  it at run time by changing the value of its /sys/devices/.../power/control
> @@ -654,7 +655,9 @@ The implementation is well suited for asynchronous use in interrupt contexts.
>  However such use inevitably involves races, because the PM core can't
>  synchronize ->runtime_suspend() callbacks with the arrival of I/O requests.
>  This synchronization must be handled by the driver, using its private lock.
> -Here is a schematic pseudo-code example::
> +Here is a schematic pseudo-code example:
> +
> +.. code-block:: c
>
>         foo_read_or_write(struct foo_priv *foo, void *data)
>         {
> @@ -720,3 +723,471 @@ pm_runtime_autosuspend_expiration() from within the ->runtime_suspend()
>  callback while holding its private lock.  If the function returns a nonzero
>  value then the delay has not yet expired and the callback should return
>  -EAGAIN.
> +
> +.. _Section 10:
> +
> +10. Example Driver Patterns
> +===========================
> +
> +The runtime PM API is large and complex, but most device drivers follow a small
> +set of canonical patterns when interacting with runtime PM. This section
> +illustrates standard patterns for device probing, performing I/O, and handling
> +interrupts.
> +
> +Probe and Initialization
> +------------------------
> +
> +Basic Probe
> +~~~~~~~~~~~
> +
> +A driver that powers on its hardware during probe and does not use autosuspend
> +can initialize runtime PM using standard helpers:
> +
> +.. code-block:: c
> +
> +       static int foo_probe(struct platform_device *pdev)
> +       {
> +               struct device *dev = &pdev->dev;
> +               struct foo_priv *priv;
> +               int ret;
> +
> +               priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
> +               if (!priv)
> +                       return -ENOMEM;
> +
> +               platform_set_drvdata(pdev, priv);
> +
> +               /* Power on and initialize hardware registers... */
> +
> +               /*
> +                * The code above left hardware powered on and operational, so
> +                * tell the PM core that the device is active before enabling
> +                * runtime PM.
> +                */
> +               pm_runtime_set_active(dev);
> +               pm_runtime_enable(dev);
> +
> +               /*
> +                * Perform remaining driver setup (e.g. register interrupts, device nodes).
> +                * If any step fails after enabling runtime PM:
> +                */
> +               ret = foo_register_device(priv);
> +               if (ret)
> +                       goto err_pm_disable;
> +
> +               /*
> +                * Upon successful return from ->probe(), the driver core
> +                * automatically executes pm_request_idle(dev), allowing the
> +                * device to suspend asynchronously if its usage counter is zero.
> +                */
> +               return 0;
> +
> +       err_pm_disable:
> +               pm_runtime_disable(dev);
> +               if (!pm_runtime_status_suspended(dev)) {
> +                       /*
> +                        * Power back off, in case device did not suspend yet.
> +                        * For example, leverage your .runtime_suspend()
> +                        * implementation:
> +                        */
> +                       foo_runtime_suspend(dev);

There are two reasons why this may not work, here and below.

First off, if a non-trivial PM domain or bus type etc is involved, its
.runtime_suspend() callback may need to be invoked in addition to
foo_runtime_suspend() to actually power off the device.  That's why
pm_runtime_force_suspend() looks for the callback to invoke.

Second, if the driver expects the device to be powered up and
operational at the beginning of probe, which this one does AFAICS, it
better should leave the device in that state on failing probe and
remove or its own expectations regarding the initial state will not be
met when it is re-probed.

> +                       pm_runtime_set_suspended(dev);
> +               }
> +               return ret;
> +       }
> +
> +       static void foo_remove(struct platform_device *pdev)
> +       {
> +               struct device *dev = &pdev->dev;
> +               struct foo_priv *priv = platform_get_drvdata(pdev);
> +
> +               foo_unregister_device(priv);
> +
> +               pm_runtime_disable(dev);
> +               /*
> +                * The runtime PM status is not guaranteed at this point. We
> +                * could be either suspended or active, so we need to handle
> +                * both.
> +                */
> +               if (!pm_runtime_status_suspended(dev)) {
> +                       /* Power back off, in case device did not suspend yet. */
> +                       foo_runtime_suspend(dev);
> +                       pm_runtime_set_suspended(dev);
> +               }
> +       }
> +

Re: [PATCH v4 2/2] PM: runtime: Add Example Driver Patterns section

[Brian Norris]

Hi Rafael,

On Wed, Oct 14, 2026 at 09:07:54PM +0200, Rafael J. Wysocki (Intel) wrote:
> On Thu, Oct 1, 2026 at 5:51 PM Brian Norris <briannorris@chromium.org> wrote:
> >
> > The runtime PM API surface is pretty large, but there are a few common
> > patterns that many drivers should follow. Add some illustrative
> > examples, to help guide the most common audience for runtime PM docs --
> > driver writers.
> >
> > Signed-off-by: Brian Norris <briannorris@chromium.org>
> > ---
> >
> > Changes in v4:
> >  * "state" to "status", to be consistent with Introduction terminology
> >
> > Changes in v3:
> >  * Rewrite Examples to avoid devm_* at first, to highlight correct
> >    approaches
> >  * Include error-handling and remove() in any probe() Examples
> >  * Advise more caution when using devm_*, as ordering issues are subtle
> >
> > Changes in v2:
> >  * Add appropriate teardown to "Probe with Hardware Powered Off"
> >    Example, as the remove() + power-off behavior is subtle here, and
> >    easy to get wrong
> >
> >  Documentation/power/runtime_pm.rst | 475 ++++++++++++++++++++++++++++-
> >  1 file changed, 473 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/power/runtime_pm.rst b/Documentation/power/runtime_pm.rst
> > index e4407b687f8b..cb25e0bb4ca7 100644
> > --- a/Documentation/power/runtime_pm.rst
> > +++ b/Documentation/power/runtime_pm.rst
> > @@ -429,7 +429,8 @@ in ->probe(). Usually this means calling pm_runtime_disable(),
> >  pm_runtime_dont_use_autosuspend() etc. Alternatively, drivers can use
> >  devm_pm_runtime_enable() during probe, which automatically takes care of
> >  calling pm_runtime_disable() and pm_runtime_dont_use_autosuspend() upon driver
> > -detachment.
> > +detachment. See also the Examples in `Section 10`_ for some effective
> > +strategies for driver teardown.
> >
> >  The user space can effectively disallow the driver of the device to power manage
> >  it at run time by changing the value of its /sys/devices/.../power/control
> > @@ -654,7 +655,9 @@ The implementation is well suited for asynchronous use in interrupt contexts.
> >  However such use inevitably involves races, because the PM core can't
> >  synchronize ->runtime_suspend() callbacks with the arrival of I/O requests.
> >  This synchronization must be handled by the driver, using its private lock.
> > -Here is a schematic pseudo-code example::
> > +Here is a schematic pseudo-code example:
> > +
> > +.. code-block:: c
> >
> >         foo_read_or_write(struct foo_priv *foo, void *data)
> >         {
> > @@ -720,3 +723,471 @@ pm_runtime_autosuspend_expiration() from within the ->runtime_suspend()
> >  callback while holding its private lock.  If the function returns a nonzero
> >  value then the delay has not yet expired and the callback should return
> >  -EAGAIN.
> > +
> > +.. _Section 10:
> > +
> > +10. Example Driver Patterns
> > +===========================
> > +
> > +The runtime PM API is large and complex, but most device drivers follow a small
> > +set of canonical patterns when interacting with runtime PM. This section
> > +illustrates standard patterns for device probing, performing I/O, and handling
> > +interrupts.
> > +
> > +Probe and Initialization
> > +------------------------
> > +
> > +Basic Probe
> > +~~~~~~~~~~~
> > +
> > +A driver that powers on its hardware during probe and does not use autosuspend
> > +can initialize runtime PM using standard helpers:
> > +
> > +.. code-block:: c
> > +
> > +       static int foo_probe(struct platform_device *pdev)
> > +       {
> > +               struct device *dev = &pdev->dev;
> > +               struct foo_priv *priv;
> > +               int ret;
> > +
> > +               priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
> > +               if (!priv)
> > +                       return -ENOMEM;
> > +
> > +               platform_set_drvdata(pdev, priv);
> > +
> > +               /* Power on and initialize hardware registers... */
> > +
> > +               /*
> > +                * The code above left hardware powered on and operational, so
> > +                * tell the PM core that the device is active before enabling
> > +                * runtime PM.
> > +                */
> > +               pm_runtime_set_active(dev);
> > +               pm_runtime_enable(dev);
> > +
> > +               /*
> > +                * Perform remaining driver setup (e.g. register interrupts, device nodes).
> > +                * If any step fails after enabling runtime PM:
> > +                */
> > +               ret = foo_register_device(priv);
> > +               if (ret)
> > +                       goto err_pm_disable;
> > +
> > +               /*
> > +                * Upon successful return from ->probe(), the driver core
> > +                * automatically executes pm_request_idle(dev), allowing the
> > +                * device to suspend asynchronously if its usage counter is zero.
> > +                */
> > +               return 0;
> > +
> > +       err_pm_disable:
> > +               pm_runtime_disable(dev);
> > +               if (!pm_runtime_status_suspended(dev)) {
> > +                       /*
> > +                        * Power back off, in case device did not suspend yet.
> > +                        * For example, leverage your .runtime_suspend()
> > +                        * implementation:
> > +                        */
> > +                       foo_runtime_suspend(dev);
> 
> There are two reasons why this may not work, here and below.
> 
> First off, if a non-trivial PM domain or bus type etc is involved, its
> .runtime_suspend() callback may need to be invoked in addition to
> foo_runtime_suspend() to actually power off the device.

Good point. While it's a platform_device (not a complex bus), I suppose
the PM domain may be relevant, at least for remove(). (probe() is
already handled automatically by the device core.)

> That's why
> pm_runtime_force_suspend() looks for the callback to invoke.

Would you be OK with pm_runtime_force_suspend() as an alternative here?
It seems like some drivers have started using that in remove(), which
doesn't quite match its docs, but otherwise seems OK to me. I was
considering that already here, and I even think it might be a good idea
to do automatically in devm_*, as I noted here:

  https://lore.kernel.org/all/arVWVA3lks7SopQH@google.com/

  (note: there's definitely at least one typo in there; should be
  !pm_runtime_status_suspended(); and it needs some other adjustments to
  actually verify the device was suspended at time of enable(); ... so
  consider it a concept, not a patch.)

But I kinda wanted a maintainer opinion before I went too far on that.

> Second, if the driver expects the device to be powered up and
> operational at the beginning of probe, which this one does AFAICS,

No, that's not the intention. See:

  "A driver that powers on its hardware during probe"

And the pseudocode/comment:

               /* Power on and initialize hardware registers... */

i.e., probe() is doing the power-on work, but it's elided.

Perhaps that wasn't obvious enough, so I can replace that with
more-convincing code-like structures
(regulator_get()/regulator_enable(), etc., for example) if you think
that would help.

Brian

> it
> better should leave the device in that state on failing probe and
> remove or its own expectations regarding the initial state will not be
> met when it is re-probed.
> 
> > +                       pm_runtime_set_suspended(dev);
> > +               }
> > +               return ret;
> > +       }
> > +
> > +       static void foo_remove(struct platform_device *pdev)
> > +       {
> > +               struct device *dev = &pdev->dev;
> > +               struct foo_priv *priv = platform_get_drvdata(pdev);
> > +
> > +               foo_unregister_device(priv);
> > +
> > +               pm_runtime_disable(dev);
> > +               /*
> > +                * The runtime PM status is not guaranteed at this point. We
> > +                * could be either suspended or active, so we need to handle
> > +                * both.
> > +                */
> > +               if (!pm_runtime_status_suspended(dev)) {
> > +                       /* Power back off, in case device did not suspend yet. */
> > +                       foo_runtime_suspend(dev);
> > +                       pm_runtime_set_suspended(dev);
> > +               }
> > +       }
> > +