Regression: USB OTG port breaks after a few hours in host mode on iMX6

Regression: USB OTG port breaks after a few hours in host mode on iMX6

[Russell King - ARM Linux]

This bug is soo obscure, I'm not even sure how to start debugging it.
This never used to be a problem, but I'm not sure when it started as
USB (in general) is not something I use regularly.

In this setup, the USB OTG port is wired in host mode via pinmix
configuration of the USB OTG ID pin:

	MX6QDL_PAD_GPIO_1__USB_OTG_ID 0x13059

The problem characterised so far: after booting the kernel, USB seems
to work fine.  You can plug and unplug devices from both USB host and
USB OTG ports, and it's detected.

If you boot a kernel with nothing plugged in, leave it for maybe four
hours, then plug in a device to either port, the device will be seen
in the USB host port, but completely ignored in the USB OTG port.
Both USB OTG ports have power, confirmed last night when using a USB
microscope with built-in LED lighting: the LEDs are functional, the
device is otherwise completely ignored.

The above is basically what I know so far: I don't know when the port
fails, whether in the case of "boot, wait four hours, and it's dead"
whether it's dead from boot: when I've tried testing it immediately
after boot, it's worked.

As it takes around four hours to reproduce, and I have a massive patch
stack on top of the kernel for this platform, I'm unwilling to attempt
a git bisection to try and find when this occurred, but I know it's
been going on for a while now as I've noticed the same behaviour when
I transfer my logitech USB receiver to that port.

I had thought it was a power issue, but the USB camera last night
confirmed that it's a port driver issue.

Certainly earlier 3.x kernels did not have this behaviour.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Regression: USB OTG port breaks after a few hours in host mode on iMX6

[Peter Chen]

On Wed, Oct 07, 2015 at 09:22:22AM +0100, Russell King - ARM Linux wrote:
> This bug is soo obscure, I'm not even sure how to start debugging it.
> This never used to be a problem, but I'm not sure when it started as
> USB (in general) is not something I use regularly.
> 
> In this setup, the USB OTG port is wired in host mode via pinmix
> configuration of the USB OTG ID pin:
> 
> 	MX6QDL_PAD_GPIO_1__USB_OTG_ID 0x13059
> 

If the port has only host function, you can set dr_mode = "host" at
dts, it will configure this port for host only, and ignore id interrupt.

> The problem characterised so far: after booting the kernel, USB seems
> to work fine.  You can plug and unplug devices from both USB host and
> USB OTG ports, and it's detected.
> 
> If you boot a kernel with nothing plugged in, leave it for maybe four
> hours, then plug in a device to either port, the device will be seen
> in the USB host port, but completely ignored in the USB OTG port.
> Both USB OTG ports have power, confirmed last night when using a USB
> microscope with built-in LED lighting: the LEDs are functional, the
> device is otherwise completely ignored.

Does interrupt occur after plugging in? If not, it means wake-on-connect
does not wake up otg port. Else, the USB PHY may not be recovered well
after suspend.

> 
> The above is basically what I know so far: I don't know when the port
> fails, whether in the case of "boot, wait four hours, and it's dead"
> whether it's dead from boot: when I've tried testing it immediately
> after boot, it's worked.
> 
> As it takes around four hours to reproduce, and I have a massive patch
> stack on top of the kernel for this platform, I'm unwilling to attempt
> a git bisection to try and find when this occurred, but I know it's
> been going on for a while now as I've noticed the same behaviour when
> I transfer my logitech USB receiver to that port.
> 
> I had thought it was a power issue, but the USB camera last night
> confirmed that it's a port driver issue.

I am testing it with two boards to see if it can be reproduced.

> 
> Certainly earlier 3.x kernels did not have this behaviour.
> 

The biggest change is runtime pm has enabled at the controller side, it
will leave the phy in low power mode, and enable the usb wakeup if
nothing in the port. With below change at ci_hdrc_imx.c, the runtime pm
can be disabled at controller side.

 static const struct ci_hdrc_imx_platform_flag imx6q_usb_data = {
	 -       .flags = CI_HDRC_SUPPORTS_RUNTIME_PM |
	+      .flags = /*CI_HDRC_SUPPORTS_RUNTIME_PM | */
			CI_HDRC_TURN_VBUS_EARLY_ON |

-- 

Best Regards,
Peter Chen

Regression: USB OTG port breaks after a few hours in host mode on iMX6

[Peter Chen]

 
> 
> On Wed, Oct 07, 2015 at 09:22:22AM +0100, Russell King - ARM Linux wrote:
> > This bug is soo obscure, I'm not even sure how to start debugging it.
> > This never used to be a problem, but I'm not sure when it started as
> > USB (in general) is not something I use regularly.
> >
> > In this setup, the USB OTG port is wired in host mode via pinmix
> > configuration of the USB OTG ID pin:
> >
> > 	MX6QDL_PAD_GPIO_1__USB_OTG_ID 0x13059
> >
> 
> If the port has only host function, you can set dr_mode = "host" at dts, it will
> configure this port for host only, and ignore id interrupt.
> 
> > The problem characterised so far: after booting the kernel, USB seems
> > to work fine.  You can plug and unplug devices from both USB host and
> > USB OTG ports, and it's detected.
> >
> > If you boot a kernel with nothing plugged in, leave it for maybe four
> > hours, then plug in a device to either port, the device will be seen
> > in the USB host port, but completely ignored in the USB OTG port.
> > Both USB OTG ports have power, confirmed last night when using a USB
> > microscope with built-in LED lighting: the LEDs are functional, the
> > device is otherwise completely ignored.
> 
> Does interrupt occur after plugging in? If not, it means wake-on-connect does
> not wake up otg port. Else, the USB PHY may not be recovered well after
> suspend.
> 
> >
> > The above is basically what I know so far: I don't know when the port
> > fails, whether in the case of "boot, wait four hours, and it's dead"
> > whether it's dead from boot: when I've tried testing it immediately
> > after boot, it's worked.
> >
> > As it takes around four hours to reproduce, and I have a massive patch
> > stack on top of the kernel for this platform, I'm unwilling to attempt
> > a git bisection to try and find when this occurred, but I know it's
> > been going on for a while now as I've noticed the same behaviour when
> > I transfer my logitech USB receiver to that port.
> >
> > I had thought it was a power issue, but the USB camera last night
> > confirmed that it's a port driver issue.
> 
> I am testing it with two boards to see if it can be reproduced.
 
I can't reproduce it for 5 hours, and will change pinmux (do the same thing with your platform),
and do the overnight test.

Peter

Regression: USB OTG port breaks after a few hours in host mode on iMX6

[Russell King - ARM Linux]

On Thu, Oct 08, 2015 at 09:52:52AM +0000, Peter Chen wrote:
> I can't reproduce it for 5 hours, and will change pinmux (do the same
> thing with your platform), and do the overnight test.

There's definitely something weird going on.  Over night last night,
I left the Logitech universal receiver in the port, and this morning
it was indicating in /proc/interrupts:

283:         50          0          0          0       GPC  43 Edge      2184000.usb

I removed it, and now I have:

283:         50          0       1716          0       GPC  43 Edge      2184000.usb

which is increasing at a rate of 90 per minute.

Nothing in the kernel message log indicating why this may be.  It looks
like runtime PM doesn't work on this port:

/sys/bus/platform/devices/2184000.usb/power/runtime_active_time:109850496
/sys/bus/platform/devices/2184000.usb/power/runtime_status:active
/sys/bus/platform/devices/2184000.usb/power/runtime_suspended_time:0

whereas the other port (which has zero interrupts) it does:

/sys/bus/platform/devices/2184200.usb/power/runtime_active_time:16924
/sys/bus/platform/devices/2184200.usb/power/runtime_status:suspended
/sys/bus/platform/devices/2184200.usb/power/runtime_suspended_time:109861760

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Regression: USB OTG port breaks after a few hours in host mode on iMX6

[Peter Chen]

On Thu, Oct 08, 2015 at 05:16:06PM +0100, Russell King - ARM Linux wrote:
> On Thu, Oct 08, 2015 at 09:52:52AM +0000, Peter Chen wrote:
> > I can't reproduce it for 5 hours, and will change pinmux (do the same
> > thing with your platform), and do the overnight test.
> 

I still not reproduce it.

> There's definitely something weird going on.  Over night last night,
> I left the Logitech universal receiver in the port, and this morning
> it was indicating in /proc/interrupts:
> 
> 283:         50          0          0          0       GPC  43 Edge      2184000.usb
> 
> I removed it, and now I have:
> 
> 283:         50          0       1716          0       GPC  43 Edge      2184000.usb
> 
> which is increasing at a rate of 90 per minute.

It is strange your interrupt occurs at cpu2 at that time, and
so many interrupts during the removal. In order to avoid other
interrupts occurring during the removal, would you please change
your dts like below:

&usbotg {
	vbus-supply = <&reg_usb_otg_vbus>;
	dr_mode = "host";
	disable-over-current;
	status = "okay";
};

> 
> Nothing in the kernel message log indicating why this may be.  It looks
> like runtime PM doesn't work on this port:
> 
> /sys/bus/platform/devices/2184000.usb/power/runtime_active_time:109850496
> /sys/bus/platform/devices/2184000.usb/power/runtime_status:active
> /sys/bus/platform/devices/2184000.usb/power/runtime_suspended_time:0
> 

This may be correct, assume your usb receiver does not support runtime
pm, so the controller is active when it is on the port, when the
removal, the interrupt occurs so frequently, it has no chance to enter
suspend status.

> whereas the other port (which has zero interrupts) it does:
> 
> /sys/bus/platform/devices/2184200.usb/power/runtime_active_time:16924
> /sys/bus/platform/devices/2184200.usb/power/runtime_status:suspended
> /sys/bus/platform/devices/2184200.usb/power/runtime_suspended_time:109861760
> 
> -- 
> FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
> according to speedtest.net.

-- 

Best Regards,
Peter Chen