[PATCH v5 00/16] TP8028 Rapid Path Failure Recovery

[Mohamed Khalfella <mkhalfella@purestorage.com>]

This patchset adds support for TP8028 Rapid Path Failure Recovery for
both the nvme target and initiator. Rapid Path Failure Recovery brings
Cross-Controller Reset (CCR) functionality to nvme. This allows an nvme
host to send an nvme command to a source nvme controller to reset the
impacted nvme controller, provided that both source and impacted
controllers are in the same nvme subsystem.

The main use of CCR is when one path to the nvme subsystem fails.
Inflight IOs on the impacted nvme controller need to be terminated first
before they can be retried on another path. Otherwise, data corruption
may happen. CCR provides a quick way to terminate these IOs on the
unreachable nvme controller, allowing recovery to move quickly and avoid
unnecessary delays. In case CCR is not possible, inflight requests are
held for a duration defined by TP4129 KATO Corrections and
Clarifications before they are allowed to be retried.


On the target side:

* New struct members have been added to support CCR. struct nvme_id_ctrl
  has been updated with CIU (Controller Instance Uniquifier), CIRN
  (Controller Instance Random Number), and CQT (Command Quiesce Time).
  The combination of CIU, CNTLID, and CIRN is used to identify the
  impacted controller in the CCR command.

* The CCR nvme command implemented on the target causes the impacted
  controller to fail and drop its connections to the host.

* The CCR log page contains the status of pending CCR requests. An entry
  is added to the log page after a CCR request is validated. Completed
  CCR requests are removed from the log page when the controller becomes
  ready or when requested in the Get Log Page command.

* An AEN is sent when a CCR completes to let the host know that it is
  safe to retry inflight requests.


On the host side:

* CIU, CIRN, and CQT have been added to struct nvme_ctrl. CIU and CIRN
  have been added to sysfs to make the values visible to the user.
  CIU and CIRN can be used to construct and manually send admin-passthru
  CCR commands.

* New controller states FENCING and FENCED have been added to make sure
  that inflight requests do not get canceled if they time out during the
  fencing process. FENCED exists so that the controller state machine
  does not have a transition from FENCING to RESETTING. Instead, FENCING
  -> FENCED -> RESETTING. This prevents a controller being fenced from
  getting reset. Only after fencing finishes is the impacted controller
  reset.

* Controller recovery in nvme_fence_ctrl() is invoked when a LIVE
  controller hits an error or when a request times out. CCR is attempted
  first to reset the impacted controller. If it fails, inflight requests
  are held until it is safe to retry them.

* Updated the nvme fabric transports nvme-tcp, nvme-rdma, and nvme-fc to
  use CCR recovery.


Ideally, all inflight requests should be held during controller recovery
and only retried after recovery is done. However, there are known
situations where that is not the case in this implementation. These gaps
will be addressed in future patches:

* A manual controller reset from sysfs will result in the controller
  going to the RESETTING state and all inflight requests being canceled
  immediately, and they may be retried on another path.

* A manual controller delete from sysfs will also result in all inflight
  requests being canceled immediately, and they may be retried on
  another path.

* In nvme-fc, the nvme controller will be deleted if the remote port
  disappears with no timeout specified. This results in immediate
  cancellation of requests that may be retried on another path.

* In nvme-rdma, if the HCA is removed, all nvme controllers will be
  deleted. This results in canceling inflight IOs, and they may be
  retried on another path.


Changes from v4:

- nvmet: Implement CCR nvme command
  - Validate transfer length at the start of
    nvmet_execute_cross_ctrl_reset()
  - Match the impacted controller on hostnqn in addition to cntlid in
    nvmet_ctrl_find_get_ccr()
  - Dropped a stray semicolon after the list_for_each_entry loop

- nvmet: Implement CCR logpage
  - Use kzalloc_obj() instead of kzalloc()
  - Rename the loop-local status to ccr_status to avoid shadowing

- nvme: Rapid Path Failure Recovery read controller identify fields
  - Store CCRL as a plain u8 ccrl instead of an atomic_t ccr_limit

- nvme: Implement cross-controller reset recovery
  - nvme_fence_ctrl() now returns the remaining wait time again instead
    of success/failure: 0 on CCR success or on fencing timeout, and the
    remaining jiffies when no source controller is available, so the
    caller can fall back to time-based recovery
  - On a per-path CCR failure, keep trying the remaining paths until the
    deadline instead of returning immediately
  - Rework the concurrency accounting to an up-counter ctrl->ccr_used
    checked against the ccrl limit instead of the ccr_limit countdown
    atomic. This was reworked because of a potential race condition:
    sctrl goes through nvme_init_identify(), which sets ctrl->ccr_limit,
    and then nvme_put_ctrl_ccr() increments it afterward.
  - Take ctrl->lock before checking state / accounting and use the
    nvme_ctrl_state() accessor

- nvme: Implement cross-controller reset completion
  - Use kmalloc_obj() instead of kmalloc()
  - Cap the number of walked log entries at NVMF_CCR_PER_PAGE

- nvme-tcp: Use CCR to recover controller that hits an error
- nvme-rdma: Use CCR to recover controller that hits an error
- nvme-fc: Use CCR to recover controller that hits an error
  - Adjust to the new nvme_fence_ctrl() return value; log "CCR failed,
    starting error recovery" instead of the errno

- nvme-fc: Do not cancel requests in io target before it is initialized
  - Dropped from the series; it was applied upstream.

- nvmet: Add support for CQT to nvme target
- nvme: Add support for CQT to nvme host
  - Re-added the CQT patches that were dropped in v4. CQT (Command
    Quiesce Time) is exposed as a target subsystem attribute defaulting
    to 0 and reported in Identify Controller; the host reads it, exposes
    it via sysfs, and uses it to bound time-based recovery (fenced_work)
    in nvme-tcp, nvme-rdma, and nvme-fc when CCR is not possible

v4: https://lore.kernel.org/all/20260328004518.1729186-1-mkhalfella@purestorage.com/

Mohamed Khalfella (16):
  nvmet: Rapid Path Failure Recovery set controller identify fields
  nvmet/debugfs: Export controller CIU and CIRN via debugfs
  nvmet: Implement CCR nvme command
  nvmet: Implement CCR logpage
  nvmet: Send an AEN on CCR completion
  nvme: Rapid Path Failure Recovery read controller identify fields
  nvme: Introduce FENCING and FENCED controller states
  nvme: Implement cross-controller reset recovery
  nvme: Implement cross-controller reset completion
  nvme-tcp: Use CCR to recover controller that hits an error
  nvme-rdma: Use CCR to recover controller that hits an error
  nvme-fc: Refactor IO error recovery
  nvme-fc: Use CCR to recover controller that hits an error
  nvme-fc: Hold inflight requests while in FENCING state
  nvmet: Add support for CQT to nvme target
  nvme: Add support for CQT to nvme host

 drivers/nvme/host/constants.c   |   1 +
 drivers/nvme/host/core.c        | 234 ++++++++++++++++++++++++++++++-
 drivers/nvme/host/fc.c          | 241 +++++++++++++++++++++++---------
 drivers/nvme/host/nvme.h        |  26 ++++
 drivers/nvme/host/rdma.c        |  63 ++++++++-
 drivers/nvme/host/sysfs.c       |  27 ++++
 drivers/nvme/host/tcp.c         |  63 ++++++++-
 drivers/nvme/target/admin-cmd.c | 126 +++++++++++++++++
 drivers/nvme/target/configfs.c  |  36 +++++
 drivers/nvme/target/core.c      | 115 ++++++++++++++-
 drivers/nvme/target/debugfs.c   |  21 +++
 drivers/nvme/target/nvmet.h     |  20 ++-
 include/linux/nvme.h            |  70 +++++++++-
 13 files changed, 968 insertions(+), 75 deletions(-)

-- 
2.54.0