[PATCH] nvme-pci: add trouble shooting steps for timeouts

[PATCH] nvme-pci: add trouble shooting steps for timeouts

[Keith Busch]

From: Keith Busch <kbusch@kernel.org>

Many users have encountered IO timeouts with a CSTS value of 0xffffffff,
which indicates a failure to read the register. While there are various
potential causes for this observation, faulty NVMe APST has been the
culprit quite frequently. Add the recommended troubleshooting steps in
the error output when this condition occurs.

Signed-off-by: Keith Busch <kbusch@kernel.org>
---
 drivers/nvme/host/pci.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 48f4f6eb877b..9d963f6cdbae 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1334,6 +1334,14 @@ static void nvme_warn_reset(struct nvme_dev *dev, u32 csts)
 		dev_warn(dev->ctrl.device,
 			 "controller is down; will reset: CSTS=0x%x, PCI_STATUS read failed (%d)\n",
 			 csts, result);
+
+	if (csts != ~0)
+		return;
+
+	dev_warn(dev->ctrl.device,
+		 "Does your device have a faulty power saving mode enabled?\n");
+	dev_warn(dev->ctrl.device,
+		 "Try \"nvme_core.default_ps_max_latency_us=0 pcie_aspm=off\" and report a bug\n");
 }
 
 static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
-- 
2.30.2

Re: [PATCH] nvme-pci: add trouble shooting steps for timeouts

[Chaitanya Kulkarni]

On 6/6/22 09:53, Keith Busch wrote:
> From: Keith Busch <kbusch@kernel.org>
> 
> Many users have encountered IO timeouts with a CSTS value of 0xffffffff,
> which indicates a failure to read the register. While there are various
> potential causes for this observation, faulty NVMe APST has been the
> culprit quite frequently. Add the recommended troubleshooting steps in
> the error output when this condition occurs.
> 
> Signed-off-by: Keith Busch <kbusch@kernel.org>
> ---

I've seen these reports with CSTS == 0xffffffff many times,
looks good, this should help...

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>

-ck

Re: [PATCH] nvme-pci: add trouble shooting steps for timeouts

[Christoph Hellwig]

Thanks,

applied to nvme-5.19.