Fwd: Enabling USB (auto)suspend for xHCI controllers incurs random device failures since kernel 4.15

[Mathias Nyman <mathias.nyman@linux.intel.com>]

On 11.05.2018 19:37, gert wrote:
> 
> Since the upgrade to Linux 4.15 (and also in 4.16), I'm experiencing an issue where all my USB devices just die seemingly without any cause. Both my laptop's internal (attached) keyboard as well as my external keyboard die.
> 
> Replugging the external keyboard unfortunately does not solve the problem. My touchpad, on the other hand, continues to work, though it may internally be connected via PS/2.
> 
> After this happens, I have only been able to solve it by rebooting.
> 
> In the logs, the following error can be found.
> 
> xhci_hcd 0000:3d:00.0: xHCI host controller not responding, assume dead
> 
> Previously, similar issues occurred to users that could be fixed by adding intel_iommu=false to the kernel parameters. This however seems to be a different problem, as it newly occurs in this specific kernel version and is not solved by the aforementioned solution.
> 
> This was also posted at the Archlinux forums [1], where we managed to pin down the issue to being related to xHCI and autosuspend (power management). I'm using powertop's --auto-tune and disabling the "good" setting for all xHCI controllers again makes the issue disappear. Linux 4.14 and lower also do not expose this issue.
> 
> Please also find attached the complete journalctl output of one boot from start to finish that exposed the issue, which may be helpful during debugging. [1]
> 

Hi

I got a report about a very similar issue, thread can be found at:
https://marc.info/?l=linux-usb&m=152335174903017&w=2

I didn't get any feedback about the suggested patch.
If you can test it, either by just by compiling my for-usb-linus branch:

git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git for-usb-linus

or alternatively applying the attached patch I would appreciate it.

If it doesn't help then we need to dig deeper into it, woith more detailed logs.

Thanks
Mathias

From d5fb6d354cd14d962d9f790f782ace0d06bf5002 Mon Sep 17 00:00:00 2001
From: Mathias Nyman <mathias.nyman@linux.intel.com>
Date: Fri, 4 May 2018 15:38:34 +0300
Subject: [PATCH] xhci: Fix perceived dead host due to runtime suspend race
 with event handler

Don't rely on event interrupt (EINT) bit alone to detect pending port
change in resume. If no change event is detected the host may be suspended
again, oterwise roothubs are resumed.

There is a lag in xHC setting EINT. If we don't notice the pending change
in resume, and the controller is runtime suspeded again, it causes the
event handler to assume host is dead as it will fail to read xHC registers
once PCI puts the controller to D3 state.

[  268.520969] xhci_hcd: xhci_resume: starting port polling.
[  268.520985] xhci_hcd: xhci_hub_status_data: stopping port polling.
[  268.521030] xhci_hcd: xhci_suspend: stopping port polling.
[  268.521040] xhci_hcd: // Setting command ring address to 0x349bd001
[  268.521139] xhci_hcd: Port Status Change Event for port 3
[  268.521149] xhci_hcd: resume root hub
[  268.521163] xhci_hcd: port resume event for port 3
[  268.521168] xhci_hcd: xHC is not running.
[  268.521174] xhci_hcd: handle_port_status: starting port polling.
[  268.596322] xhci_hcd: xhci_hc_died: xHCI host controller not responding, assume dead

The EINT lag is described in a additional note in xhci specs 4.19.2:

"Due to internal xHC scheduling and system delays, there will be a lag
between a change bit being set and the Port Status Change Event that it
generated being written to the Event Ring. If SW reads the PORTSC and
sees a change bit set, there is no guarantee that the corresponding Port
Status Change Event has already been written into the Event Ring."

Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com>
---
 drivers/usb/host/xhci.c | 37 ++++++++++++++++++++++++++++++++++---
 drivers/usb/host/xhci.h |  4 ++++
 2 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index 711da33..18e9fcf 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -844,6 +844,38 @@ static void xhci_disable_port_wake_on_bits(struct xhci_hcd *xhci)
 	spin_unlock_irqrestore(&xhci->lock, flags);
 }
 
+static bool xhci_pending_portevent(struct xhci_hcd *xhci)
+{
+	int	port_index;
+	u32	status;
+	u32	portsc;
+
+	status = readl(&xhci->op_regs->status);
+	if (status & STS_EINT)
+		return true;
+	/*
+	 * Checking STS_EINT is not enough as there is a lag between a change
+	 * bit being set and the Port Status Change Event that it generated
+	 * being written to the Event Ring. See note in xhci 1.1 section 4.19.2.
+	 */
+
+	port_index = xhci->num_usb2_ports;
+	while (port_index--) {
+		portsc = readl(xhci->usb2_ports[port_index]);
+		if (portsc & PORT_CHANGE_MASK ||
+		    (portsc & PORT_PLS_MASK) == XDEV_RESUME)
+			return true;
+	}
+	port_index = xhci->num_usb3_ports;
+	while (port_index--) {
+		portsc = readl(xhci->usb3_ports[port_index]);
+		if (portsc & PORT_CHANGE_MASK ||
+		    (portsc & PORT_PLS_MASK) == XDEV_RESUME)
+			return true;
+	}
+	return false;
+}
+
 /*
  * Stop HC (not bus-specific)
  *
@@ -945,7 +977,7 @@ EXPORT_SYMBOL_GPL(xhci_suspend);
  */
 int xhci_resume(struct xhci_hcd *xhci, bool hibernated)
 {
-	u32			command, temp = 0, status;
+	u32			command, temp = 0;
 	struct usb_hcd		*hcd = xhci_to_hcd(xhci);
 	struct usb_hcd		*secondary_hcd;
 	int			retval = 0;
@@ -1069,8 +1101,7 @@ int xhci_resume(struct xhci_hcd *xhci, bool hibernated)
  done:
 	if (retval == 0) {
 		/* Resume root hubs only when have pending events. */
-		status = readl(&xhci->op_regs->status);
-		if (status & STS_EINT) {
+		if (xhci_pending_portevent(xhci)) {
 			usb_hcd_resume_root_hub(xhci->shared_hcd);
 			usb_hcd_resume_root_hub(hcd);
 		}
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index 6dfc486..9751c13 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -382,6 +382,10 @@ struct xhci_op_regs {
 #define PORT_PLC	(1 << 22)
 /* port configure error change - port failed to configure its link partner */
 #define PORT_CEC	(1 << 23)
+#define PORT_CHANGE_MASK	(PORT_CSC | PORT_PEC | PORT_WRC | PORT_OCC | \
+				 PORT_RC | PORT_PLC | PORT_CEC)
+
+
 /* Cold Attach Status - xHC can set this bit to report device attached during
  * Sx state. Warm port reset should be perfomed to clear this bit and move port
  * to connected state.
-- 
2.7.4