Make NVME shutdown two-pass - Version 6

[Jeremy Allison <jallison@ciq.com>]

This is version 6 of a patchset originally written by
Tanjore Suresh <tansuresh@google.com> to make shutdown
of nvme devices two-pass.

Changes from Version 5:

1). Function nvme_request_shutdown() name changed to
nvme_ctrl_shutdown_start() as requested by Sagi Grimberg.
2). Git commit message reformatting to 75 columns
as requested by Bjorn Helgaas <bhelgaas@google.com>.

NB. An alternate patchset from David Jeffery <djeffery@redhat.com>
has been proposed that covers much of the same ground.

I'm happy to collaborate on merging these two patchsets
if that is requested by the Linux nvme community.

-------------------------------------------------------------
Currently the Linux nvme driver shutdown code steps
through each connected drive, sets the NVME_CC_SHN_NORMAL
(normal shutdown) flag and then polls the given drive
waiting for the response NVME_CSTS_SHST_CMPLT flag
(shutdown complete).

Each drive is taking around 13 seconds to respond to this.

The customer has 20+ drives on the box so this time adds
up on shutdown when the nvme driver is being shut down.

This patchset changes shutdown to proceed in parallel,
so the NVME_CC_SHN_NORMAL (normal shutdown) flag is
sent to all drives first, and then it polls waiting
for the NVME_CSTS_SHST_CMPLT flag (shutdown complete)
for all drives.

In the specific customer case it reduces the NVME
shutdown time from over 300 seconds to around 15
seconds.
-------------------------------------------------------------

Jeremy Allison
CIQ / Samba Team.