Re: [PATCH v15 0/5] shut down devices asynchronously

[tarunsahu@google.com]

David Jeffery <djeffery@redhat.com> writes:

FWIW,
I have tested this series by opting it for NVMe devices and observed
significant performance benefit (from 9 sec to 2 sec).

Please feel free to use
Tested-by: Tarun Sahu <tarunsahu@google.com>

> This patchset allows the kernel to shutdown devices asynchronously and
> unrelated async devices to be shut down in parallel to each other.
>
> Only devices which explicitly enable it are shut down asynchronously. The
> default is for a device to be shut down from the synchronous shutdown loop.
>
> This can dramatically reduce system shutdown/reboot time on systems that
> have multiple devices that take many seconds to shut down (like certain
> NVMe drives). On one system tested, the shutdown time went from 11 minutes
> without this patch to 55 seconds with the patch. And on another system from
> 80 seconds to 11.
>
> And thank you to everyone who has spent some of their valuable time
> providing reviews, suggestions, criticisms, or tests on the various
> iterations of this patchset.
>
>
> Changes from V14:
>
> Remove unneeded use of '!!' with boolean type
>
> Changes from V13:
>
> Remove duplicate flagging of async shutdown on scsi hosts/targets/devices
>
> Changes from V12:
>
> Only acquire a parent reference if acquiring the parent's lock
> device_enable_async_shutdown should return void
> Minor comment and description cleanups
>
> Changes from V11:
>
>   * Swap the order of the first two patches
>   * Rework conditional parent locking so that lock and unlock no longer use
>     separate conditional checks
>   * Remove an used variable
>   * Comment and description text cleanups
>
> Changes from V10:
>
> Reworked to more closely match the design used for async suspend
>   * No longer uses async subsystem cookies for synchronization
>   * Minimized changes to struct device
>   * Enable async shutdown for pci and scsi devices which support async suspend
>
> Changes from V9:
>
> Address resource and timing issues when spawning a unique async thread
> for every device during shutdown:
>   * Make the asynchronous threads able to shut down multiple devices,
>     instead of spawning a unique thread for every device.
>   * Modify core kernel async code with a custom wake function so it
>     doesn't wake up a thread waiting to synchronize on a cookie until
>     the cookie has reached the desired value, instead of waking up
>     every waiting thread to check the cookie every time an async thread
>     ends.
>
> Changes from V8:
>
> Deal with shutdown hangs resulting when a parent/supplier device is
>   later in the devices_kset list than its children/consumers:
>   * Ignore sync_state_only devlinks for shutdown dependencies
>   * Ignore shutdown_after for devices that don't want async shutdown
>   * Add a sanity check to revert to sync shutdown for any device that
>     would otherwise wait for a child/consumer shutdown that hasn't
>     already been scheduled
>
> Changes from V7:
>
> Do not expose driver async_shutdown_enable in sysfs.
> Wrapped a long line.
>
> Changes from V6:
>
> Removed a sysfs attribute that allowed the async device shutdown to be
> "on" (with driver opt-out), "safe" (driver opt-in), or "off"... what was
> previously "safe" is now the only behavior, so drivers now only need to
> have the option to enable or disable async shutdown.
>
> Changes from V5:
>
> Separated into multiple patches to make review easier.
> Reworked some code to make it more readable
> Made devices wait for consumers to shut down, not just children
>   (suggested by David Jeffery)
>
> Changes from V4:
>
> Change code to use cookies for synchronization rather than async domains
> Allow async shutdown to be disabled via sysfs, and allow driver opt-in or
>   opt-out of async shutdown (when not disabled), with ability to control
>   driver opt-in/opt-out via sysfs
>   
> Changes from V3:
>
> Bug fix (used "parent" not "dev->parent" in device_shutdown)
>  
> Changes from V2:
>  
> Removed recursive functions to schedule children to be shutdown before
>   parents, since existing device_shutdown loop will already do this
>  
> Changes from V1:
>
> Rewritten using kernel async code (suggested by Lukas Wunner)
>
>
> Stuart Hayes (2):
>   driver core: separate function to shutdown one device
>   driver core: do not always lock parent in shutdown
>
> David Jeffery (3):
>   driver core: async device shutdown infrastructure
>   PCI: Enable async shutdown support
>   scsi: Enable async shutdown support
>
>  drivers/base/base.h       |   2 +
>  drivers/base/core.c       | 180 ++++++++++++++++++++++++++++++--------
>  drivers/pci/probe.c       |   2 +
>  drivers/scsi/hosts.c      |   2 +
>  drivers/scsi/scsi_sysfs.c |   3 +
>  include/linux/device.h    |  14 ++-
>  6 files changed, 166 insertions(+), 37 deletions(-)
>
> -- 
> 2.53.0