* [PATCH 1/5] driver core: separate function to shutdown one device
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
@ 2026-03-19 14:11 ` David Jeffery
2026-03-19 14:11 ` [PATCH 2/5] driver core: do not always lock parent in shutdown David Jeffery
` (4 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: David Jeffery @ 2026-03-19 14:11 UTC (permalink / raw)
To: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
David Jeffery
Make a separate function for the part of device_shutdown() that does the
shutown for a single device. This is in preparation for making device
shutdown asynchronous.
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/base/core.c | 71 +++++++++++++++++++++++++--------------------
1 file changed, 39 insertions(+), 32 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 791f9e444df8..10586298e18b 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -4782,12 +4782,48 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
+static void shutdown_one_device(struct device *dev)
+{
+ struct device *parent = dev->parent;
+
+ /* hold lock to avoid race with probe/release */
+ if (parent)
+ device_lock(parent);
+ device_lock(dev);
+
+ /* Don't allow any more runtime suspends */
+ pm_runtime_get_noresume(dev);
+ pm_runtime_barrier(dev);
+
+ if (dev->class && dev->class->shutdown_pre) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown_pre\n");
+ dev->class->shutdown_pre(dev);
+ }
+ if (dev->bus && dev->bus->shutdown) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown\n");
+ dev->bus->shutdown(dev);
+ } else if (dev->driver && dev->driver->shutdown) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown\n");
+ dev->driver->shutdown(dev);
+ }
+
+ device_unlock(dev);
+ if (parent)
+ device_unlock(parent);
+
+ put_device(parent);
+ put_device(dev);
+}
+
/**
* device_shutdown - call ->shutdown() on each device to shutdown.
*/
void device_shutdown(void)
{
- struct device *dev, *parent;
+ struct device *dev;
wait_for_device_probe();
device_block_probing();
@@ -4809,7 +4845,7 @@ void device_shutdown(void)
* prevent it from being freed because parent's
* lock is to be held
*/
- parent = get_device(dev->parent);
+ get_device(dev->parent);
get_device(dev);
/*
* Make sure the device is off the kset list, in the
@@ -4818,36 +4854,7 @@ void device_shutdown(void)
list_del_init(&dev->kobj.entry);
spin_unlock(&devices_kset->list_lock);
- /* hold lock to avoid race with probe/release */
- if (parent)
- device_lock(parent);
- device_lock(dev);
-
- /* Don't allow any more runtime suspends */
- pm_runtime_get_noresume(dev);
- pm_runtime_barrier(dev);
-
- if (dev->class && dev->class->shutdown_pre) {
- if (initcall_debug)
- dev_info(dev, "shutdown_pre\n");
- dev->class->shutdown_pre(dev);
- }
- if (dev->bus && dev->bus->shutdown) {
- if (initcall_debug)
- dev_info(dev, "shutdown\n");
- dev->bus->shutdown(dev);
- } else if (dev->driver && dev->driver->shutdown) {
- if (initcall_debug)
- dev_info(dev, "shutdown\n");
- dev->driver->shutdown(dev);
- }
-
- device_unlock(dev);
- if (parent)
- device_unlock(parent);
-
- put_device(dev);
- put_device(parent);
+ shutdown_one_device(dev);
spin_lock(&devices_kset->list_lock);
}
--
2.53.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 2/5] driver core: do not always lock parent in shutdown
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
2026-03-19 14:11 ` [PATCH 1/5] driver core: separate function to shutdown one device David Jeffery
@ 2026-03-19 14:11 ` David Jeffery
2026-03-19 14:11 ` [PATCH 3/5] driver core: async device shutdown infrastructure David Jeffery
` (3 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: David Jeffery @ 2026-03-19 14:11 UTC (permalink / raw)
To: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
David Jeffery
Don't lock a parent device unless it is needed in device_shutdown. This
is in preparation for making device shutdown asynchronous, when it will
be needed to allow children of a common parent to shut down
simultaneously.
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/base/core.c | 23 ++++++++++++++---------
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 10586298e18b..2e9094f5c5aa 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -4782,13 +4782,8 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
-static void shutdown_one_device(struct device *dev)
+static void __shutdown_one_device(struct device *dev)
{
- struct device *parent = dev->parent;
-
- /* hold lock to avoid race with probe/release */
- if (parent)
- device_lock(parent);
device_lock(dev);
/* Don't allow any more runtime suspends */
@@ -4811,10 +4806,20 @@ static void shutdown_one_device(struct device *dev)
}
device_unlock(dev);
- if (parent)
- device_unlock(parent);
+}
- put_device(parent);
+static void shutdown_one_device(struct device *dev)
+{
+ /* hold lock to avoid race with probe/release */
+ if (dev->parent && dev->bus && dev->bus->need_parent_lock) {
+ device_lock(dev->parent);
+ __shutdown_one_device(dev);
+ device_unlock(dev->parent);
+ } else {
+ __shutdown_one_device(dev);
+ }
+
+ put_device(dev->parent);
put_device(dev);
}
--
2.53.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 3/5] driver core: async device shutdown infrastructure
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
2026-03-19 14:11 ` [PATCH 1/5] driver core: separate function to shutdown one device David Jeffery
2026-03-19 14:11 ` [PATCH 2/5] driver core: do not always lock parent in shutdown David Jeffery
@ 2026-03-19 14:11 ` David Jeffery
2026-03-23 9:43 ` Maurizio Lombardi
2026-03-19 14:11 ` [PATCH 4/5] PCI: enable async shutdown support David Jeffery
` (2 subsequent siblings)
5 siblings, 1 reply; 11+ messages in thread
From: David Jeffery @ 2026-03-19 14:11 UTC (permalink / raw)
To: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
David Jeffery
Patterned after async suspend, allow devices to mark themselves as wanting
to perform async shutdown. Devices using async shutdown wait only for their
dependencies to shutdown before executing their shutdown routine.
Sync shutdown devices are shut down one at a time and will only wait for an
async shutdown device if the async device is a dependency.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/base/base.h | 2 +
drivers/base/core.c | 104 ++++++++++++++++++++++++++++++++++++++++-
include/linux/device.h | 13 ++++++
3 files changed, 118 insertions(+), 1 deletion(-)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index 79d031d2d845..ea2a039e7907 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -113,6 +113,7 @@ struct driver_type {
* @device - pointer back to the struct device that this structure is
* associated with.
* @driver_type - The type of the bound Rust driver.
+ * @complete - completion for device shutdown ordering
* @dead - This device is currently either in the process of or has been
* removed from the system. Any asynchronous events scheduled for this
* device should exit without taking any action.
@@ -132,6 +133,7 @@ struct device_private {
#ifdef CONFIG_RUST
struct driver_type driver_type;
#endif
+ struct completion complete;
u8 dead:1;
};
#define to_device_private_parent(obj) \
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 2e9094f5c5aa..53568b820a13 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -9,6 +9,7 @@
*/
#include <linux/acpi.h>
+#include <linux/async.h>
#include <linux/blkdev.h>
#include <linux/cleanup.h>
#include <linux/cpufreq.h>
@@ -37,6 +38,10 @@
#include "physical_location.h"
#include "power/power.h"
+static bool async_shutdown = true;
+module_param(async_shutdown, bool, 0644);
+MODULE_PARM_DESC(async_shutdown, "Enable asynchronous device shutdown support");
+
/* Device links support. */
static LIST_HEAD(deferred_sync);
static unsigned int defer_sync_state_count = 1;
@@ -3538,6 +3543,7 @@ static int device_private_init(struct device *dev)
klist_init(&dev->p->klist_children, klist_children_get,
klist_children_put);
INIT_LIST_HEAD(&dev->p->deferred_probe);
+ init_completion(&dev->p->complete);
return 0;
}
@@ -4782,6 +4788,37 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
+static bool wants_async_shutdown(struct device *dev)
+{
+ return async_shutdown && dev->async_shutdown;
+}
+
+static int wait_for_device_shutdown(struct device *dev, void *data)
+{
+ bool async = *(bool *)data;
+
+ if (async || wants_async_shutdown(dev))
+ wait_for_completion(&dev->p->complete);
+
+ return 0;
+}
+
+static void wait_for_shutdown_dependencies(struct device *dev, bool async)
+{
+ struct device_link *link;
+ int idx;
+
+ device_for_each_child(dev, &async, wait_for_device_shutdown);
+
+ idx = device_links_read_lock();
+
+ dev_for_each_link_to_consumer(link, dev)
+ if (!device_link_flag_is_sync_state_only(link->flags))
+ wait_for_device_shutdown(link->consumer, &async);
+
+ device_links_read_unlock(idx);
+}
+
static void __shutdown_one_device(struct device *dev)
{
device_lock(dev);
@@ -4805,6 +4842,8 @@ static void __shutdown_one_device(struct device *dev)
dev->driver->shutdown(dev);
}
+ complete_all(&dev->p->complete);
+
device_unlock(dev);
}
@@ -4823,6 +4862,58 @@ static void shutdown_one_device(struct device *dev)
put_device(dev);
}
+static void async_shutdown_handler(void *data, async_cookie_t cookie)
+{
+ struct device *dev = data;
+
+ wait_for_shutdown_dependencies(dev, true);
+ shutdown_one_device(dev);
+}
+
+static bool shutdown_device_async(struct device *dev)
+{
+ if (async_schedule_dev_nocall(async_shutdown_handler, dev))
+ return true;
+ return false;
+}
+
+
+static void early_async_shutdown_devices(void)
+{
+ struct device *dev, *next, *needs_put = NULL;
+
+ if (!async_shutdown)
+ return;
+
+ spin_lock(&devices_kset->list_lock);
+
+ list_for_each_entry_safe_reverse(dev, next, &devices_kset->list,
+ kobj.entry) {
+ if (wants_async_shutdown(dev)) {
+ get_device(dev->parent);
+ get_device(dev);
+
+ if (shutdown_device_async(dev)) {
+ list_del_init(&dev->kobj.entry);
+ } else {
+ /*
+ * async failed, clean up extra references
+ * and run from the standard shutdown loop
+ */
+ needs_put = dev;
+ break;
+ }
+ }
+ }
+
+ spin_unlock(&devices_kset->list_lock);
+
+ if (needs_put) {
+ put_device(needs_put->parent);
+ put_device(needs_put);
+ }
+}
+
/**
* device_shutdown - call ->shutdown() on each device to shutdown.
*/
@@ -4835,6 +4926,12 @@ void device_shutdown(void)
cpufreq_suspend();
+ /*
+ * Start async device threads where possible to maximize potential
+ * parallelism and minimize false dependency on unrelated sync devices
+ */
+ early_async_shutdown_devices();
+
spin_lock(&devices_kset->list_lock);
/*
* Walk the devices list backward, shutting down each in turn.
@@ -4859,11 +4956,16 @@ void device_shutdown(void)
list_del_init(&dev->kobj.entry);
spin_unlock(&devices_kset->list_lock);
- shutdown_one_device(dev);
+ if (!wants_async_shutdown(dev) || !shutdown_device_async(dev)) {
+ wait_for_shutdown_dependencies(dev, false);
+ shutdown_one_device(dev);
+ }
spin_lock(&devices_kset->list_lock);
}
spin_unlock(&devices_kset->list_lock);
+
+ async_synchronize_full();
}
/*
diff --git a/include/linux/device.h b/include/linux/device.h
index 0be95294b6e6..da1db7d235c9 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -551,6 +551,8 @@ struct device_physical_location {
* @dma_skip_sync: DMA sync operations can be skipped for coherent buffers.
* @dma_iommu: Device is using default IOMMU implementation for DMA and
* doesn't rely on dma_ops structure.
+ * @async_shutdown: Device shutdown may be run asynchronously and in parallel
+ * to the shutdown of unrelated devices
*
* At the lowest level, every device in a Linux system is represented by an
* instance of struct device. The device structure contains the information
@@ -669,6 +671,7 @@ struct device {
#ifdef CONFIG_IOMMU_DMA
bool dma_iommu:1;
#endif
+ bool async_shutdown:1;
};
/**
@@ -824,6 +827,16 @@ static inline bool device_async_suspend_enabled(struct device *dev)
return !!dev->power.async_suspend;
}
+static inline bool device_enable_async_shutdown(struct device *dev)
+{
+ return dev->async_shutdown = true;
+}
+
+static inline bool device_async_shutdown_enabled(struct device *dev)
+{
+ return !!dev->async_shutdown;
+}
+
static inline bool device_pm_not_required(struct device *dev)
{
return dev->power.no_pm;
--
2.53.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 3/5] driver core: async device shutdown infrastructure
2026-03-19 14:11 ` [PATCH 3/5] driver core: async device shutdown infrastructure David Jeffery
@ 2026-03-23 9:43 ` Maurizio Lombardi
2026-03-23 14:07 ` David Jeffery
0 siblings, 1 reply; 11+ messages in thread
From: Maurizio Lombardi @ 2026-03-23 9:43 UTC (permalink / raw)
To: David Jeffery, linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas
On Thu Mar 19, 2026 at 3:11 PM CET, David Jeffery wrote:
> Patterned after async suspend, allow devices to mark themselves as wanting
> to perform async shutdown. Devices using async shutdown wait only for their
> dependencies to shutdown before executing their shutdown routine.
>
> Sync shutdown devices are shut down one at a time and will only wait for an
> async shutdown device if the async device is a dependency.
>
> Signed-off-by: David Jeffery <djeffery@redhat.com>
> Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
> Tested-by: Laurence Oberman <loberman@redhat.com>
> ---
> drivers/base/base.h | 2 +
> drivers/base/core.c | 104 ++++++++++++++++++++++++++++++++++++++++-
> include/linux/device.h | 13 ++++++
> 3 files changed, 118 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/base/base.h b/drivers/base/base.h
> index 79d031d2d845..ea2a039e7907 100644
> --- a/drivers/base/base.h
> +++ b/drivers/base/base.h
> @@ -113,6 +113,7 @@ struct driver_type {
> * @device - pointer back to the struct device that this structure is
> * associated with.
> * @driver_type - The type of the bound Rust driver.
> + * @complete - completion for device shutdown ordering
> * @dead - This device is currently either in the process of or has been
> * removed from the system. Any asynchronous events scheduled for this
> * device should exit without taking any action.
> @@ -132,6 +133,7 @@ struct device_private {
> #ifdef CONFIG_RUST
> struct driver_type driver_type;
> #endif
> + struct completion complete;
> u8 dead:1;
> };
> #define to_device_private_parent(obj) \
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 2e9094f5c5aa..53568b820a13 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -9,6 +9,7 @@
> */
>
> #include <linux/acpi.h>
> +#include <linux/async.h>
> #include <linux/blkdev.h>
> #include <linux/cleanup.h>
> #include <linux/cpufreq.h>
> @@ -37,6 +38,10 @@
> #include "physical_location.h"
> #include "power/power.h"
>
> +static bool async_shutdown = true;
> +module_param(async_shutdown, bool, 0644);
> +MODULE_PARM_DESC(async_shutdown, "Enable asynchronous device shutdown support");
> +
> /* Device links support. */
> static LIST_HEAD(deferred_sync);
> static unsigned int defer_sync_state_count = 1;
> @@ -3538,6 +3543,7 @@ static int device_private_init(struct device *dev)
> klist_init(&dev->p->klist_children, klist_children_get,
> klist_children_put);
> INIT_LIST_HEAD(&dev->p->deferred_probe);
> + init_completion(&dev->p->complete);
> return 0;
> }
>
> @@ -4782,6 +4788,37 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
> return error;
> }
>
> +static bool wants_async_shutdown(struct device *dev)
> +{
> + return async_shutdown && dev->async_shutdown;
> +}
> +
> +static int wait_for_device_shutdown(struct device *dev, void *data)
> +{
> + bool async = *(bool *)data;
> +
> + if (async || wants_async_shutdown(dev))
> + wait_for_completion(&dev->p->complete);
> +
> + return 0;
> +}
> +
> +static void wait_for_shutdown_dependencies(struct device *dev, bool async)
> +{
> + struct device_link *link;
> + int idx;
> +
> + device_for_each_child(dev, &async, wait_for_device_shutdown);
> +
> + idx = device_links_read_lock();
> +
> + dev_for_each_link_to_consumer(link, dev)
> + if (!device_link_flag_is_sync_state_only(link->flags))
> + wait_for_device_shutdown(link->consumer, &async);
> +
> + device_links_read_unlock(idx);
> +}
> +
> static void __shutdown_one_device(struct device *dev)
> {
> device_lock(dev);
> @@ -4805,6 +4842,8 @@ static void __shutdown_one_device(struct device *dev)
> dev->driver->shutdown(dev);
> }
>
> + complete_all(&dev->p->complete);
> +
> device_unlock(dev);
> }
>
> @@ -4823,6 +4862,58 @@ static void shutdown_one_device(struct device *dev)
> put_device(dev);
> }
>
> +static void async_shutdown_handler(void *data, async_cookie_t cookie)
> +{
> + struct device *dev = data;
> +
> + wait_for_shutdown_dependencies(dev, true);
> + shutdown_one_device(dev);
> +}
> +
> +static bool shutdown_device_async(struct device *dev)
> +{
> + if (async_schedule_dev_nocall(async_shutdown_handler, dev))
> + return true;
> + return false;
> +}
> +
> +
> +static void early_async_shutdown_devices(void)
> +{
> + struct device *dev, *next, *needs_put = NULL;
> +
> + if (!async_shutdown)
> + return;
> +
> + spin_lock(&devices_kset->list_lock);
> +
> + list_for_each_entry_safe_reverse(dev, next, &devices_kset->list,
> + kobj.entry) {
> + if (wants_async_shutdown(dev)) {
> + get_device(dev->parent);
> + get_device(dev);
> +
> + if (shutdown_device_async(dev)) {
> + list_del_init(&dev->kobj.entry);
> + } else {
> + /*
> + * async failed, clean up extra references
> + * and run from the standard shutdown loop
> + */
> + needs_put = dev;
> + break;
> + }
> + }
> + }
> +
> + spin_unlock(&devices_kset->list_lock);
> +
> + if (needs_put) {
> + put_device(needs_put->parent);
> + put_device(needs_put);
> + }
> +}
> +
> /**
> * device_shutdown - call ->shutdown() on each device to shutdown.
> */
> @@ -4835,6 +4926,12 @@ void device_shutdown(void)
>
> cpufreq_suspend();
>
> + /*
> + * Start async device threads where possible to maximize potential
> + * parallelism and minimize false dependency on unrelated sync devices
> + */
> + early_async_shutdown_devices();
> +
> spin_lock(&devices_kset->list_lock);
> /*
> * Walk the devices list backward, shutting down each in turn.
> @@ -4859,11 +4956,16 @@ void device_shutdown(void)
> list_del_init(&dev->kobj.entry);
> spin_unlock(&devices_kset->list_lock);
>
> - shutdown_one_device(dev);
> + if (!wants_async_shutdown(dev) || !shutdown_device_async(dev)) {
> + wait_for_shutdown_dependencies(dev, false);
> + shutdown_one_device(dev);
> + }
>
> spin_lock(&devices_kset->list_lock);
> }
> spin_unlock(&devices_kset->list_lock);
> +
> + async_synchronize_full();
> }
>
> /*
> diff --git a/include/linux/device.h b/include/linux/device.h
> index 0be95294b6e6..da1db7d235c9 100644
> --- a/include/linux/device.h
> +++ b/include/linux/device.h
> @@ -551,6 +551,8 @@ struct device_physical_location {
> * @dma_skip_sync: DMA sync operations can be skipped for coherent buffers.
> * @dma_iommu: Device is using default IOMMU implementation for DMA and
> * doesn't rely on dma_ops structure.
> + * @async_shutdown: Device shutdown may be run asynchronously and in parallel
> + * to the shutdown of unrelated devices
> *
> * At the lowest level, every device in a Linux system is represented by an
> * instance of struct device. The device structure contains the information
> @@ -669,6 +671,7 @@ struct device {
> #ifdef CONFIG_IOMMU_DMA
> bool dma_iommu:1;
> #endif
> + bool async_shutdown:1;
> };
>
> /**
> @@ -824,6 +827,16 @@ static inline bool device_async_suspend_enabled(struct device *dev)
> return !!dev->power.async_suspend;
> }
>
> +static inline bool device_enable_async_shutdown(struct device *dev)
> +{
> + return dev->async_shutdown = true;
> +}
Shouldn't this function just return void?
Maurizio
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH 3/5] driver core: async device shutdown infrastructure
2026-03-23 9:43 ` Maurizio Lombardi
@ 2026-03-23 14:07 ` David Jeffery
0 siblings, 0 replies; 11+ messages in thread
From: David Jeffery @ 2026-03-23 14:07 UTC (permalink / raw)
To: Maurizio Lombardi
Cc: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich,
Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas
On Mon, Mar 23, 2026 at 5:50 AM Maurizio Lombardi <mlombard@arkamax.eu> wrote:
>
> On Thu Mar 19, 2026 at 3:11 PM CET, David Jeffery wrote:
> > Patterned after async suspend, allow devices to mark themselves as wanting
> > to perform async shutdown. Devices using async shutdown wait only for their
> > dependencies to shutdown before executing their shutdown routine.
> >
> > Sync shutdown devices are shut down one at a time and will only wait for an
> > async shutdown device if the async device is a dependency.
> >
> > Signed-off-by: David Jeffery <djeffery@redhat.com>
> > Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
> > Tested-by: Laurence Oberman <loberman@redhat.com>
> > ---
> > drivers/base/base.h | 2 +
> > drivers/base/core.c | 104 ++++++++++++++++++++++++++++++++++++++++-
> > include/linux/device.h | 13 ++++++
> > 3 files changed, 118 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/base/base.h b/drivers/base/base.h
> > index 79d031d2d845..ea2a039e7907 100644
> > --- a/drivers/base/base.h
> > +++ b/drivers/base/base.h
> > @@ -113,6 +113,7 @@ struct driver_type {
> > * @device - pointer back to the struct device that this structure is
> > * associated with.
> > * @driver_type - The type of the bound Rust driver.
> > + * @complete - completion for device shutdown ordering
> > * @dead - This device is currently either in the process of or has been
> > * removed from the system. Any asynchronous events scheduled for this
> > * device should exit without taking any action.
> > @@ -132,6 +133,7 @@ struct device_private {
> > #ifdef CONFIG_RUST
> > struct driver_type driver_type;
> > #endif
> > + struct completion complete;
> > u8 dead:1;
> > };
> > #define to_device_private_parent(obj) \
> > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > index 2e9094f5c5aa..53568b820a13 100644
> > --- a/drivers/base/core.c
> > +++ b/drivers/base/core.c
> > @@ -9,6 +9,7 @@
> > */
> >
> > #include <linux/acpi.h>
> > +#include <linux/async.h>
> > #include <linux/blkdev.h>
> > #include <linux/cleanup.h>
> > #include <linux/cpufreq.h>
> > @@ -37,6 +38,10 @@
> > #include "physical_location.h"
> > #include "power/power.h"
> >
> > +static bool async_shutdown = true;
> > +module_param(async_shutdown, bool, 0644);
> > +MODULE_PARM_DESC(async_shutdown, "Enable asynchronous device shutdown support");
> > +
> > /* Device links support. */
> > static LIST_HEAD(deferred_sync);
> > static unsigned int defer_sync_state_count = 1;
> > @@ -3538,6 +3543,7 @@ static int device_private_init(struct device *dev)
> > klist_init(&dev->p->klist_children, klist_children_get,
> > klist_children_put);
> > INIT_LIST_HEAD(&dev->p->deferred_probe);
> > + init_completion(&dev->p->complete);
> > return 0;
> > }
> >
> > @@ -4782,6 +4788,37 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
> > return error;
> > }
> >
> > +static bool wants_async_shutdown(struct device *dev)
> > +{
> > + return async_shutdown && dev->async_shutdown;
> > +}
> > +
> > +static int wait_for_device_shutdown(struct device *dev, void *data)
> > +{
> > + bool async = *(bool *)data;
> > +
> > + if (async || wants_async_shutdown(dev))
> > + wait_for_completion(&dev->p->complete);
> > +
> > + return 0;
> > +}
> > +
> > +static void wait_for_shutdown_dependencies(struct device *dev, bool async)
> > +{
> > + struct device_link *link;
> > + int idx;
> > +
> > + device_for_each_child(dev, &async, wait_for_device_shutdown);
> > +
> > + idx = device_links_read_lock();
> > +
> > + dev_for_each_link_to_consumer(link, dev)
> > + if (!device_link_flag_is_sync_state_only(link->flags))
> > + wait_for_device_shutdown(link->consumer, &async);
> > +
> > + device_links_read_unlock(idx);
> > +}
> > +
> > static void __shutdown_one_device(struct device *dev)
> > {
> > device_lock(dev);
> > @@ -4805,6 +4842,8 @@ static void __shutdown_one_device(struct device *dev)
> > dev->driver->shutdown(dev);
> > }
> >
> > + complete_all(&dev->p->complete);
> > +
> > device_unlock(dev);
> > }
> >
> > @@ -4823,6 +4862,58 @@ static void shutdown_one_device(struct device *dev)
> > put_device(dev);
> > }
> >
> > +static void async_shutdown_handler(void *data, async_cookie_t cookie)
> > +{
> > + struct device *dev = data;
> > +
> > + wait_for_shutdown_dependencies(dev, true);
> > + shutdown_one_device(dev);
> > +}
> > +
> > +static bool shutdown_device_async(struct device *dev)
> > +{
> > + if (async_schedule_dev_nocall(async_shutdown_handler, dev))
> > + return true;
> > + return false;
> > +}
> > +
> > +
> > +static void early_async_shutdown_devices(void)
> > +{
> > + struct device *dev, *next, *needs_put = NULL;
> > +
> > + if (!async_shutdown)
> > + return;
> > +
> > + spin_lock(&devices_kset->list_lock);
> > +
> > + list_for_each_entry_safe_reverse(dev, next, &devices_kset->list,
> > + kobj.entry) {
> > + if (wants_async_shutdown(dev)) {
> > + get_device(dev->parent);
> > + get_device(dev);
> > +
> > + if (shutdown_device_async(dev)) {
> > + list_del_init(&dev->kobj.entry);
> > + } else {
> > + /*
> > + * async failed, clean up extra references
> > + * and run from the standard shutdown loop
> > + */
> > + needs_put = dev;
> > + break;
> > + }
> > + }
> > + }
> > +
> > + spin_unlock(&devices_kset->list_lock);
> > +
> > + if (needs_put) {
> > + put_device(needs_put->parent);
> > + put_device(needs_put);
> > + }
> > +}
> > +
> > /**
> > * device_shutdown - call ->shutdown() on each device to shutdown.
> > */
> > @@ -4835,6 +4926,12 @@ void device_shutdown(void)
> >
> > cpufreq_suspend();
> >
> > + /*
> > + * Start async device threads where possible to maximize potential
> > + * parallelism and minimize false dependency on unrelated sync devices
> > + */
> > + early_async_shutdown_devices();
> > +
> > spin_lock(&devices_kset->list_lock);
> > /*
> > * Walk the devices list backward, shutting down each in turn.
> > @@ -4859,11 +4956,16 @@ void device_shutdown(void)
> > list_del_init(&dev->kobj.entry);
> > spin_unlock(&devices_kset->list_lock);
> >
> > - shutdown_one_device(dev);
> > + if (!wants_async_shutdown(dev) || !shutdown_device_async(dev)) {
> > + wait_for_shutdown_dependencies(dev, false);
> > + shutdown_one_device(dev);
> > + }
> >
> > spin_lock(&devices_kset->list_lock);
> > }
> > spin_unlock(&devices_kset->list_lock);
> > +
> > + async_synchronize_full();
> > }
> >
> > /*
> > diff --git a/include/linux/device.h b/include/linux/device.h
> > index 0be95294b6e6..da1db7d235c9 100644
> > --- a/include/linux/device.h
> > +++ b/include/linux/device.h
> > @@ -551,6 +551,8 @@ struct device_physical_location {
> > * @dma_skip_sync: DMA sync operations can be skipped for coherent buffers.
> > * @dma_iommu: Device is using default IOMMU implementation for DMA and
> > * doesn't rely on dma_ops structure.
> > + * @async_shutdown: Device shutdown may be run asynchronously and in parallel
> > + * to the shutdown of unrelated devices
> > *
> > * At the lowest level, every device in a Linux system is represented by an
> > * instance of struct device. The device structure contains the information
> > @@ -669,6 +671,7 @@ struct device {
> > #ifdef CONFIG_IOMMU_DMA
> > bool dma_iommu:1;
> > #endif
> > + bool async_shutdown:1;
> > };
> >
> > /**
> > @@ -824,6 +827,16 @@ static inline bool device_async_suspend_enabled(struct device *dev)
> > return !!dev->power.async_suspend;
> > }
> >
> > +static inline bool device_enable_async_shutdown(struct device *dev)
> > +{
> > + return dev->async_shutdown = true;
> > +}
>
> Shouldn't this function just return void?
Yes, it should just be changed to a void.
David Jeffery
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 4/5] PCI: enable async shutdown support
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
` (2 preceding siblings ...)
2026-03-19 14:11 ` [PATCH 3/5] driver core: async device shutdown infrastructure David Jeffery
@ 2026-03-19 14:11 ` David Jeffery
2026-03-19 14:11 ` [PATCH 5/5] scsi: " David Jeffery
2026-03-19 17:13 ` [PATCH v12 0/5] shut down devices asynchronously Michael Kelley
5 siblings, 0 replies; 11+ messages in thread
From: David Jeffery @ 2026-03-19 14:11 UTC (permalink / raw)
To: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
David Jeffery
Like its async suspend support, allow PCI device shutdown to be performed
asynchronously to reduce shutdown time.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/pci/probe.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index bccc7a4bdd79..4d98bab2163d 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1040,6 +1040,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
bus->bridge = get_device(&bridge->dev);
device_enable_async_suspend(bus->bridge);
+ device_enable_async_shutdown(bus->bridge);
pci_set_bus_of_node(bus);
pci_set_bus_msi_domain(bus);
if (bridge->msi_domain && !dev_get_msi_domain(&bus->dev) &&
@@ -2749,6 +2750,7 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
pci_reassigndev_resource_alignment(dev);
pci_init_capabilities(dev);
+ device_enable_async_shutdown(&dev->dev);
/*
* Add the device to our list of discovered devices
--
2.53.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 5/5] scsi: enable async shutdown support
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
` (3 preceding siblings ...)
2026-03-19 14:11 ` [PATCH 4/5] PCI: enable async shutdown support David Jeffery
@ 2026-03-19 14:11 ` David Jeffery
2026-03-20 2:20 ` Martin K. Petersen
2026-03-19 17:13 ` [PATCH v12 0/5] shut down devices asynchronously Michael Kelley
5 siblings, 1 reply; 11+ messages in thread
From: David Jeffery @ 2026-03-19 14:11 UTC (permalink / raw)
To: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
David Jeffery
Like scsi's async suspend support, allow scsi devices to be shut down
asynchronously to reduce system shutdown time.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/scsi/hosts.c | 3 +++
drivers/scsi/scsi_scan.c | 1 +
drivers/scsi/scsi_sysfs.c | 4 ++++
3 files changed, 8 insertions(+)
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index e047747d4ecf..dfb38a82c0bf 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -273,6 +273,7 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,
pm_runtime_set_active(&shost->shost_gendev);
pm_runtime_enable(&shost->shost_gendev);
device_enable_async_suspend(&shost->shost_gendev);
+ device_enable_async_shutdown(&shost->shost_gendev);
error = device_add(&shost->shost_gendev);
if (error)
@@ -282,6 +283,7 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,
get_device(shost->shost_gendev.parent);
device_enable_async_suspend(&shost->shost_dev);
+ device_enable_async_shutdown(&shost->shost_dev);
get_device(&shost->shost_gendev);
error = device_add(&shost->shost_dev);
@@ -519,6 +521,7 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv
shost->shost_gendev.bus = &scsi_bus_type;
shost->shost_gendev.type = &scsi_host_type;
scsi_enable_async_suspend(&shost->shost_gendev);
+ device_enable_async_shutdown(&shost->shost_gendev);
device_initialize(&shost->shost_dev);
shost->shost_dev.parent = &shost->shost_gendev;
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 60c06fa4ec32..c42b33e8ea8a 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -518,6 +518,7 @@ static struct scsi_target *scsi_alloc_target(struct device *parent,
dev->bus = &scsi_bus_type;
dev->type = &scsi_target_type;
scsi_enable_async_suspend(dev);
+ device_enable_async_shutdown(dev);
starget->id = id;
starget->channel = channel;
starget->can_queue = 0;
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 6b8c5c05f294..c76ba17b206f 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1370,6 +1370,7 @@ static int scsi_target_add(struct scsi_target *starget)
pm_runtime_set_active(&starget->dev);
pm_runtime_enable(&starget->dev);
device_enable_async_suspend(&starget->dev);
+ device_enable_async_shutdown(&starget->dev);
return 0;
}
@@ -1396,6 +1397,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
transport_configure_device(&starget->dev);
device_enable_async_suspend(&sdev->sdev_gendev);
+ device_enable_async_shutdown(&sdev->sdev_gendev);
scsi_autopm_get_target(starget);
pm_runtime_set_active(&sdev->sdev_gendev);
if (!sdev->rpm_autosuspend)
@@ -1415,6 +1417,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
}
device_enable_async_suspend(&sdev->sdev_dev);
+ device_enable_async_shutdown(&sdev->sdev_dev);
error = device_add(&sdev->sdev_dev);
if (error) {
sdev_printk(KERN_INFO, sdev,
@@ -1670,6 +1673,7 @@ void scsi_sysfs_device_initialize(struct scsi_device *sdev)
sdev->sdev_gendev.bus = &scsi_bus_type;
sdev->sdev_gendev.type = &scsi_dev_type;
scsi_enable_async_suspend(&sdev->sdev_gendev);
+ device_enable_async_shutdown(&sdev->sdev_gendev);
dev_set_name(&sdev->sdev_gendev, "%d:%d:%d:%llu",
sdev->host->host_no, sdev->channel, sdev->id, sdev->lun);
sdev->sdev_gendev.groups = hostt->sdev_groups;
--
2.53.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 5/5] scsi: enable async shutdown support
2026-03-19 14:11 ` [PATCH 5/5] scsi: " David Jeffery
@ 2026-03-20 2:20 ` Martin K. Petersen
0 siblings, 0 replies; 11+ messages in thread
From: Martin K. Petersen @ 2026-03-20 2:20 UTC (permalink / raw)
To: David Jeffery
Cc: linux-kernel, driver-core, linux-pci, linux-scsi,
Greg Kroah-Hartman, Rafael J. Wysocki, Danilo Krummrich,
Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas
David,
> Like scsi's async suspend support, allow scsi devices to be shut down
> asynchronously to reduce system shutdown time.
Looks OK to me.
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
--
Martin K. Petersen
^ permalink raw reply [flat|nested] 11+ messages in thread
* RE: [PATCH v12 0/5] shut down devices asynchronously
2026-03-19 14:11 [PATCH v12 0/5] shut down devices asynchronously David Jeffery
` (4 preceding siblings ...)
2026-03-19 14:11 ` [PATCH 5/5] scsi: " David Jeffery
@ 2026-03-19 17:13 ` Michael Kelley
5 siblings, 0 replies; 11+ messages in thread
From: Michael Kelley @ 2026-03-19 17:13 UTC (permalink / raw)
To: David Jeffery, linux-kernel@vger.kernel.org,
driver-core@lists.linux.dev, linux-pci@vger.kernel.org,
linux-scsi@vger.kernel.org, Greg Kroah-Hartman, Rafael J. Wysocki,
Danilo Krummrich
Cc: Tarun Sahu, Pasha Tatashin, Michał Cłapiński,
Jordan Richards, Ewan Milne, John Meneghini, Lombardi, Maurizio,
Stuart Hayes, Laurence Oberman, Bart Van Assche, Bjorn Helgaas,
linux-hyperv@vger.kernel.org
From: David Jeffery <djeffery@redhat.com>
>
> This patchset allows the kernel to shutdown devices asynchronously and
> unrelated async devices to be shut down in parallel to each other.
>
> Only devices which explicitly enable it are shut down asynchronously. The
> default is for a device to be shut down from the synchronous shutdown loop.
>
> This can dramatically reduce system shutdown/reboot time on systems that
> have multiple devices that take many seconds to shut down (like certain
> NVMe drives). On one system tested, the shutdown time went from 11 minutes
> without this patch to 55 seconds with the patch. And on another system from
> 80 seconds to 11.
>
(Copying the linux-hyperv mailing list as FYI.)
Tested this patch set on two different x86/x64 VMs in the Azure public cloud.
Baseline kernel is linux-next20260312. The VMs are running on Hyper-V with
synthetic SCSI devices, PCI pass-thru NVME disks, and emulated PCI NVMe
disks, depending on the VM configuration.
First VM is an Azure L64s_v3, with 2 synthetic SCSI disks and 8 PCI NVMe
pass-thru disks. Time spent in device_shutdown() was reduced from
683 milliseconds to 150 milliseconds (averaged across 4 runs each).
Second VM is an Azure D32lds_v6, with 2 emulated NVMe disks and
4 NVMe pass-thru disks. Time sent in device_shutdown() was reduced from
1010 milliseconds to 610 milliseconds (averaged across 2 runs each).
In both cases, the results seem reasonable. None of these disks should
be particularly slow in shutting down, so the results are not as dramatic
are reported by David. But there is non-trivial improvement nonetheless.
Tested-by: Michael Kelley <mhklinux@outlook.com>
>
> Stuart Hayes (2):
> driver core: separate function to shutdown one device
> driver core: don't always lock parent in shutdown
>
> David Jeffery (5):
> driver core: async device shutdown infrastructure
> PCI: enable async shutdown support
> scsi: enable async shutdown support
>
> drivers/base/base.h | 2 +
> drivers/base/core.c | 176 +++++++++++++++++++++++++++++++-------
> drivers/pci/probe.c | 2 +
> drivers/scsi/hosts.c | 3 +
> drivers/scsi/scsi_scan.c | 1 +
> drivers/scsi/scsi_sysfs.c | 4 +
> include/linux/device.h | 13 +++
> 7 files changed, 170 insertions(+), 31 deletions(-)
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 11+ messages in thread