[1/5] xhci: Fix perceived dead host due to runtime suspend race with event handler

[1/5] xhci: Fix perceived dead host due to runtime suspend race with event handler

[Mathias Nyman]

Don't rely on event interrupt (EINT) bit alone to detect pending port
change in resume. If no change event is detected the host may be suspended
again, oterwise roothubs are resumed.

There is a lag in xHC setting EINT. If we don't notice the pending change
in resume, and the controller is runtime suspeded again, it causes the
event handler to assume host is dead as it will fail to read xHC registers
once PCI puts the controller to D3 state.

[  268.520969] xhci_hcd: xhci_resume: starting port polling.
[  268.520985] xhci_hcd: xhci_hub_status_data: stopping port polling.
[  268.521030] xhci_hcd: xhci_suspend: stopping port polling.
[  268.521040] xhci_hcd: // Setting command ring address to 0x349bd001
[  268.521139] xhci_hcd: Port Status Change Event for port 3
[  268.521149] xhci_hcd: resume root hub
[  268.521163] xhci_hcd: port resume event for port 3
[  268.521168] xhci_hcd: xHC is not running.
[  268.521174] xhci_hcd: handle_port_status: starting port polling.
[  268.596322] xhci_hcd: xhci_hc_died: xHCI host controller not responding, assume dead

The EINT lag is described in a additional note in xhci specs 4.19.2:

"Due to internal xHC scheduling and system delays, there will be a lag
between a change bit being set and the Port Status Change Event that it
generated being written to the Event Ring. If SW reads the PORTSC and
sees a change bit set, there is no guarantee that the corresponding Port
Status Change Event has already been written into the Event Ring."

Cc: <stable@vger.kernel.org>
Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com>
---
 drivers/usb/host/xhci.c | 40 +++++++++++++++++++++++++++++++++++++---
 drivers/usb/host/xhci.h |  4 ++++
 2 files changed, 41 insertions(+), 3 deletions(-)

diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index 8c8da2d..f11ec61 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -908,6 +908,41 @@ static void xhci_disable_port_wake_on_bits(struct xhci_hcd *xhci)
 	spin_unlock_irqrestore(&xhci->lock, flags);
 }
 
+static bool xhci_pending_portevent(struct xhci_hcd *xhci)
+{
+	struct xhci_port	**ports;
+	int			port_index;
+	u32			status;
+	u32			portsc;
+
+	status = readl(&xhci->op_regs->status);
+	if (status & STS_EINT)
+		return true;
+	/*
+	 * Checking STS_EINT is not enough as there is a lag between a change
+	 * bit being set and the Port Status Change Event that it generated
+	 * being written to the Event Ring. See note in xhci 1.1 section 4.19.2.
+	 */
+
+	port_index = xhci->usb2_rhub.num_ports;
+	ports = xhci->usb2_rhub.ports;
+	while (port_index--) {
+		portsc = readl(ports[port_index]->addr);
+		if (portsc & PORT_CHANGE_MASK ||
+		    (portsc & PORT_PLS_MASK) == XDEV_RESUME)
+			return true;
+	}
+	port_index = xhci->usb3_rhub.num_ports;
+	ports = xhci->usb3_rhub.ports;
+	while (port_index--) {
+		portsc = readl(ports[port_index]->addr);
+		if (portsc & PORT_CHANGE_MASK ||
+		    (portsc & PORT_PLS_MASK) == XDEV_RESUME)
+			return true;
+	}
+	return false;
+}
+
 /*
  * Stop HC (not bus-specific)
  *
@@ -1009,7 +1044,7 @@ EXPORT_SYMBOL_GPL(xhci_suspend);
  */
 int xhci_resume(struct xhci_hcd *xhci, bool hibernated)
 {
-	u32			command, temp = 0, status;
+	u32			command, temp = 0;
 	struct usb_hcd		*hcd = xhci_to_hcd(xhci);
 	struct usb_hcd		*secondary_hcd;
 	int			retval = 0;
@@ -1134,8 +1169,7 @@ int xhci_resume(struct xhci_hcd *xhci, bool hibernated)
  done:
 	if (retval == 0) {
 		/* Resume root hubs only when have pending events. */
-		status = readl(&xhci->op_regs->status);
-		if (status & STS_EINT) {
+		if (xhci_pending_portevent(xhci)) {
 			usb_hcd_resume_root_hub(xhci->shared_hcd);
 			usb_hcd_resume_root_hub(hcd);
 		}
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index 939e2f86..841e89f 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -382,6 +382,10 @@ struct xhci_op_regs {
 #define PORT_PLC	(1 << 22)
 /* port configure error change - port failed to configure its link partner */
 #define PORT_CEC	(1 << 23)
+#define PORT_CHANGE_MASK	(PORT_CSC | PORT_PEC | PORT_WRC | PORT_OCC | \
+				 PORT_RC | PORT_PLC | PORT_CEC)
+
+
 /* Cold Attach Status - xHC can set this bit to report device attached during
  * Sx state. Warm port reset should be perfomed to clear this bit and move port
  * to connected state.

[1/5] xhci: Fix perceived dead host due to runtime suspend race with event handler

[Kai-Heng Feng]

Hi Mathias,

at 21:19, Mathias Nyman <mathias.nyman@linux.intel.com> wrote:

> Don't rely on event interrupt (EINT) bit alone to detect pending port
> change in resume. If no change event is detected the host may be suspended
> again, oterwise roothubs are resumed.
>
> There is a lag in xHC setting EINT. If we don't notice the pending change
> in resume, and the controller is runtime suspeded again, it causes the
> event handler to assume host is dead as it will fail to read xHC registers
> once PCI puts the controller to D3 state.
>
> [  268.520969] xhci_hcd: xhci_resume: starting port polling.
> [  268.520985] xhci_hcd: xhci_hub_status_data: stopping port polling.
> [  268.521030] xhci_hcd: xhci_suspend: stopping port polling.
> [  268.521040] xhci_hcd: // Setting command ring address to 0x349bd001
> [  268.521139] xhci_hcd: Port Status Change Event for port 3
> [  268.521149] xhci_hcd: resume root hub
> [  268.521163] xhci_hcd: port resume event for port 3
> [  268.521168] xhci_hcd: xHC is not running.
> [  268.521174] xhci_hcd: handle_port_status: starting port polling.
> [  268.596322] xhci_hcd: xhci_hc_died: xHCI host controller not  
> responding, assume dead
>
> The EINT lag is described in a additional note in xhci specs 4.19.2:
>
> "Due to internal xHC scheduling and system delays, there will be a lag
> between a change bit being set and the Port Status Change Event that it
> generated being written to the Event Ring. If SW reads the PORTSC and
> sees a change bit set, there is no guarantee that the corresponding Port
> Status Change Event has already been written into the Event Ring."
>
> Cc: <stable@vger.kernel.org>


I tried to backport this patch to v4.15, and "xhci: Create new structures  
to store xhci port information" series is a dependency for this patch.

The series brings substantial changes, so I am wondering if you have any  
plan to backport this patch to older kernels?

Kai-Heng


> Signed-off-by: Mathias Nyman <mathias.nyman@linux.intel.com>
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html