[RFC PATCH 0/6] async device shutdown support

[RFC PATCH 0/6] async device shutdown support

[David Jeffery]

This is another attempt to implement an acceptable implementation of async
device shutdown, inspired by a previous attempt by Tanjore Suresh. For
systems with many disks, async shutdown can greatly reduce shutdown times
from having slow operations run in parallel. The older patches were rejected,
with this new implementation attempting to fix my understanding of the flaws
in the older patches.

Using similar interfaces and building off the ideas of the older patches,
this patchset creates an async shutdown implementation which follows the
basic ordering of the shutdown list, ensuring the shutdown of any children
devices complete whether synchronous or asynchronous before performing
shutdown on a parent device.

In addition to an implementation for asynchronous pci nvme shutdown, this
patchset also adds support for async shutdown of sd devices for cache flush.
As an example of the effects of the patch, one system with a large amount of
disks went from over 30 seconds to shut down to less than 5.

The specific driver changes are the roughest part of this patchset. But the
acceptability of the core async functionality is critical. Any feedback on
flaws or improvements is appreciated.

David Jeffery (6):
  minimal async shutdown infrastructure
  Improve ability to perform async shutdown in parallel
  pci bus async shutdown support
  pci nvme async shutdown support
  scsi mid layer support for async command submit
  sd: async cache flush on shutdown

 drivers/base/base.h           |   1 +
 drivers/base/core.c           | 149 +++++++++++++++++++++++++++++++++-
 drivers/nvme/host/core.c      |  26 ++++--
 drivers/nvme/host/nvme.h      |   2 +
 drivers/nvme/host/pci.c       |  53 +++++++++++-
 drivers/pci/pci-driver.c      |  24 +++++-
 drivers/scsi/scsi_lib.c       | 138 ++++++++++++++++++++++++-------
 drivers/scsi/sd.c             |  66 +++++++++++++--
 drivers/scsi/sd.h             |   2 +
 include/linux/device/bus.h    |   8 +-
 include/linux/device/driver.h |   7 ++
 include/linux/pci.h           |   4 +
 include/scsi/scsi_device.h    |   8 ++
 13 files changed, 439 insertions(+), 49 deletions(-)

-- 
2.43.0

Re: [RFC PATCH 0/6] async device shutdown support

[Jeremy Allison]

> This is another attempt to implement an acceptable implementation of async
> device shutdown, inspired by a previous attempt by Tanjore Suresh. For
> systems with many disks, async shutdown can greatly reduce shutdown times
> from having slow operations run in parallel. The older patches were rejected,
> with this new implementation attempting to fix my understanding of the flaws
> in the older patches

Hi David,

It may have escaped your notice that I was shepherding a newer version
of Tanjore's original patchset through the nvme lists already. Please
look at version 5 here (I am working on version 6 currently).

https://lore.kernel.org/linux-nvme/20240129181953.1183084-1-jallison@ciq.com/

As your work is very similar (although has some of the same problems
that people already asked me to fix in earlier versions of the code)
maybe we can collaborate on getting a unified version of this work
reviewed.

Please take a look at the link above, and see if we can merge our efforts.

Thanks !

Jeremy Allison / CIQ.
Samba Team.

Re: [RFC PATCH 0/6] async device shutdown support

[Jeremy Allison]

> This is another attempt to implement an acceptable implementation of async
> device shutdown, inspired by a previous attempt by Tanjore Suresh. For
> systems with many disks, async shutdown can greatly reduce shutdown times
> from having slow operations run in parallel. The older patches were rejected,
> with this new implementation attempting to fix my understanding of the flaws
> in the older patches

Hi David,

It may have escaped your notice that I was shepherding a newer version
of Tanjore's original patchset through the nvme lists already. Please
look at version 5 here (I am working on version 6 currently).

https://lore.kernel.org/linux-nvme/20240129181953.1183084-1-jallison@ciq.com/

As your work is very similar (although has some of the same problems
that people already asked me to fix in earlier versions of the code)
maybe we can collaborate on getting a unified version of this work
reviewed.

Please take a look at the link above, and see if we can merge our efforts.

Thanks !

Jeremy Allison / CIQ.
Samba Team.

Re: [RFC PATCH 0/6] async device shutdown support

[Lukas Wunner]

[cc += Stuart, linux-pci; start of thread:
 https://lore.kernel.org/lkml/20240207184100.18066-1-djeffery@redhat.com/
]

On Wed, Feb 07, 2024 at 12:40:15PM -0800, Jeremy Allison wrote:
> On Wed, Feb 07, 2024 at 1:40:54PM -0500 David Jeffery wrote:
> > This is another attempt to implement an acceptable implementation of async
> > device shutdown, inspired by a previous attempt by Tanjore Suresh. For
> > systems with many disks, async shutdown can greatly reduce shutdown times
> > from having slow operations run in parallel.
> 
> It may have escaped your notice that I was shepherding a newer version
> of Tanjore's original patchset through the nvme lists already. Please
> look at version 5 here (I am working on version 6 currently).

Stuart Hayes has also been working on async shutdown and his approach
is much leaner than yours.  He needs testers.  Maybe you can give his
patch a spin and report back if it works for you, so that you guys can
collaborate instead of working on separate approaches?

Here's Stuart's v5:
https://lore.kernel.org/all/20231128210436.506510-1-stuart.w.hayes@gmail.com/

Thanks,

Lukas