[PATCH 0/7] few nvme-rdma fixes for 4.18

[hch@lst.de (Christoph Hellwig)]

On Wed, Jun 20, 2018@11:53:01AM +0300, Sagi Grimberg wrote:
>
>>> 4 is needed to prevent a theoretical hang in controller removal, but
>>> never encountered.
>>
>> This one I'm indeed a little concerned about as I remember various
>> issues in this area in PCIe.  And RDMA is just different enough that
>> it doesn't look very similar.
>
> Actually, 4 follows PCIe one to one:
> --
>         /*
>          * The driver will not be starting up queues again if shutting down 
> so
>          * must flush all entered requests to their failed completion to 
> avoid
>          * deadlocking blk-mq hot-cpu notifier.
>          */
>         if (shutdown)
>                 nvme_start_queues(&dev->ctrl);
> --
> Nothing about this is PCIe specific, its about the controller going away
> with quiesced IO.

With the difference that PCIe does it at the end of nvme_dev_disable,
long after we've actually disabled or shut down the controller,
while RDMA does it before shutting down the controller.

> On a side note, If you look at nvme_reset_work and nvme_dev_disable, its
> not that different from what RDMA does, its just arranged slightly
> different. I'm still very much think that PCIe and RDMA and FC (and TCP)
> can have the same probe, reset, remove sequences. I guess it will just
> take some more time to get there.

Sharing more code would be great, especially for managing the block
layer state.