Re: [PATCH v8 3/4] driver core: shut down devices asynchronously

[Christophe JAILLET <christophe.jaillet@wanadoo.fr>]

Le 22/08/2024 à 22:28, Stuart Hayes a écrit :
> Add code to allow asynchronous shutdown of devices, ensuring that each
> device is shut down before its parents & suppliers.
> 
> Only devices with drivers that have async_shutdown_enable enabled will be
> shut down asynchronously.
> 
> This can dramatically reduce system shutdown/reboot time on systems that
> have multiple devices that take many seconds to shut down (like certain
> NVMe drives). On one system tested, the shutdown time went from 11 minutes
> without this patch to 55 seconds with the patch.
> 
> Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
> Signed-off-by: David Jeffery <djeffery@redhat.com>
> ---

...

> +/**
> + * shutdown_one_device_async
> + * @data: the pointer to the struct device to be shutdown
> + * @cookie: not used
> + *
> + * Shuts down one device, after waiting for shutdown_after to complete.
> + * shutdown_after should be set to the cookie of the last child or consumer
> + * of this device to be shutdown (if any), or to the cookie of the previous
> + * device to be shut down for devices that don't enable asynchronous shutdown.
> + */
> +static void shutdown_one_device_async(void *data, async_cookie_t cookie)
> +{
> +	struct device *dev = data;
> +
> +	async_synchronize_cookie_domain(dev->p->shutdown_after + 1, &sd_domain);
> +
> +	shutdown_one_device(dev);
> +}
> +
>   /**
>    * device_shutdown - call ->shutdown() on each device to shutdown.
>    */
>   void device_shutdown(void)
>   {
>   	struct device *dev, *parent;
> +	async_cookie_t cookie = 0;
> +	struct device_link *link;
> +	int idx;
>   
>   	wait_for_device_probe();
>   	device_block_probing();
> @@ -4852,11 +4878,37 @@ void device_shutdown(void)
>   		list_del_init(&dev->kobj.entry);
>   		spin_unlock(&devices_kset->list_lock);
>   
> -		shutdown_one_device(dev);
> +
> +		/*
> +		 * Set cookie for devices that will be shut down synchronously
> +		 */
> +		if (!dev->driver || !dev->driver->async_shutdown_enable)
> +			dev->p->shutdown_after = cookie;
> +
> +		get_device(dev);
> +		get_device(parent);
> +
> +		cookie = async_schedule_domain(shutdown_one_device_async,
> +					       dev, &sd_domain);
> +		/*
> +		 * Ensure parent & suppliers wait for this device to shut down
> +		 */
> +		if (parent) {
> +			parent->p->shutdown_after = cookie;
> +			put_device(parent);

Would it make sense to have this put_device() out of the if block?

IIUC, the behavior would be exactly the same, but it is more intuitive 
to have a put_device(parent) called for each get_device(parent) call.

Another way to keep symmetry is to have:
	if (parent)
		get_device(parent);
above.

> +		}
> +
> +		idx = device_links_read_lock();
> +		list_for_each_entry_rcu(link, &dev->links.suppliers, c_node,
> +				device_links_read_lock_held())
> +			link->supplier->p->shutdown_after = cookie;
> +		device_links_read_unlock(idx);
> +		put_device(dev);
>   
>   		spin_lock(&devices_kset->list_lock);
>   	}
>   	spin_unlock(&devices_kset->list_lock);
> +	async_synchronize_full_domain(&sd_domain);
>   }
>   
>   /*
> diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
> index 1fc8b68786de..2b6127faaa25 100644
> --- a/include/linux/device/driver.h
> +++ b/include/linux/device/driver.h
> @@ -56,6 +56,7 @@ enum probe_type {
>    * @mod_name:	Used for built-in modules.
>    * @suppress_bind_attrs: Disables bind/unbind via sysfs.
>    * @probe_type:	Type of the probe (synchronous or asynchronous) to use.
> + * @async_shutdown_enable: Enables devices to be shutdown asynchronously.
>    * @of_match_table: The open firmware table.
>    * @acpi_match_table: The ACPI match table.
>    * @probe:	Called to query the existence of a specific device,
> @@ -102,6 +103,7 @@ struct device_driver {
>   
>   	bool suppress_bind_attrs;	/* disables bind/unbind via sysfs */
>   	enum probe_type probe_type;
> +	bool async_shutdown_enable;

Maybe keep these 2 bools together to potentially avoid hole?

Just my 2c.

CJ

>   
>   	const struct of_device_id	*of_match_table;
>   	const struct acpi_device_id	*acpi_match_table;