[PATCH v2 0/10] Centrelize common fabrics code to core drivers

[Max Gurtovoy <mgurtovoy@nvidia.com>]

Hi Christoph, Sagi, Keith and Co,
This series is intended to move common reconnection, device removal,
error recovery and reset logic to common fabrics and core drivers.
In this stage, it mainly deals with the RDMA and TCP drivers which had
the same logic for the above tasks. FC driver can start using this
framework in the future after some preparation work that should be done.

This will ease on code maintenance and reduce the code duplication
between the RDMA and TCP drivers.

Since there are some attributes that were moved to generic nvme_ctrl
structure, I've measured perf impact.

NVMe PCI Perf numbers using NVIDIA's NVMe SNAP Bluefield-2 device
(Fio 16 jobs, 128 iodepth, libaio):

IO size   rand READ (before/after)  rand WRITE (before/after)
-------- -------------------------  --------------------------
 512B          2189k/2197k                2031k/2092k
 1KB           2188k/2201k                2027k/2088k
 2KB           2166k/2181k                2023k/2078k
 4KB           2153k/2169k                2025k/2076k
 8KB           1381k/1381k                1378k/1375k
 16KB          669k/676k                  660k/664k
 32KB          320k/322k                  305k/305k
 64KB          157k/157k                  151k/151k

As we can see, there is no impact on the performance.

Changes from v1:
 - collect some "Reviewed-by" signatures (Sagi, Hannes, Chaitanya)
 - added reset flow to this series (thus added setup_ctrl callback)
 - added more common code to nvmf_init_ctrl
 - rename nvmf_teardown_ctrl to nvmf_uninit_ctrl (Sagi)
 - add bool arguments to use RDMA/TCP callbacks directly (Hannes)

Max Gurtovoy (10):
  nvme: add connect_work attribute to nvme ctrl
  nvme-fabrics: introduce nvmf_reconnect_or_remove API
  nvme: add err_work attribute to nvme ctrl
  nvme-fabrics: introduce nvmf_error_recovery API
  nvme/nvme-fabrics: introduce nvmf_error_recovery_work API
  nvme/nvme-fabrics: introduce nvmf_reconnect_ctrl_work API
  nvme-fabrics: add nvmf_init_ctrl/nvmf_uninit_ctrl API
  nvme-rdma: update WARN_ON condition during reset
  nvme/nvme-fabrics: move reset ctrl flow to common code
  nvme-fabrics: set common attributes during nvmf_init_ctrl

 drivers/nvme/host/fabrics.c | 121 +++++++++++++++++
 drivers/nvme/host/fabrics.h |   3 +
 drivers/nvme/host/fc.c      |  23 ++--
 drivers/nvme/host/nvme.h    |   8 ++
 drivers/nvme/host/rdma.c    | 252 +++++++++++-------------------------
 drivers/nvme/host/tcp.c     | 152 +++-------------------
 6 files changed, 241 insertions(+), 318 deletions(-)

-- 
2.18.1