FSL/NXP P5020: USB problems with the latest Git kernels

FSL/NXP P5020: USB problems with the latest Git kernels

[Christian Zigotzky]

Hello,

There are USB problems with P5020 boards with the latest Git kernels.

Some users reported the following error messages:

skateman@X5000LNX:~$ grep -i -E 'usb.*error' /var/log/syslog
Aug 24 13:12:50 X5000LNX kernel: [ 6.552303] usb 1-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 6.649376] usb 2-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 6.794158] usb 1-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 6.883156] usb 2-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 7.139155] usb 1-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 7.226156] usb 2-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 7.363155] usb 1-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 7.451156] usb 2-1: device descriptor read=
/64, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 8.521156] usb 1-1: device not accepting a=
ddress 4, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 8.609156] usb 2-1: device not accepting a=
ddress 4, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 9.057155] usb 1-1: device not accepting a=
ddress 5, error -32
Aug 24 13:12:50 X5000LNX kernel: [ 9.145157] usb 2-1: device not accepting a=
ddress 5, error -32

I don=E2=80=99t have a P5020 board here. That means I can=E2=80=99t bisect a=
nd test. Therefore I need your help. The problematic Git commit must be some=
where between 22/08/18 1:20 PM UTC and 23/08/18 10:49 AM UTC.
The Git kernel from 22/08/18 1:20 PM UTC works with USB devices.
Could you please compile a Git kernel and test it on your P5020 board?

Thanks,
Christian=

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Christian Zigotzky]

Hello,

With the boot argument =E2=80=9Cmem=3D3500M=E2=80=9D the USB devices work. P=
lease compile the latest Git kernel and test it on your P5020 board. There i=
s a problem with the memory management since 22/08/18. I can=E2=80=99t test b=
ecause I don=E2=80=99t have a P5020 board here and our users can=E2=80=99t c=
ompile kernels.

Thanks,
Christian

Sent from my iPhone

> On 24. Aug 2018, at 20:11, Christian Zigotzky <chzigotzky@xenosoft.de> wro=
te:
>=20
> Hello,
>=20
> There are USB problems with P5020 boards with the latest Git kernels.
>=20
> Some users reported the following error messages:
>=20
> skateman@X5000LNX:~$ grep -i -E 'usb.*error' /var/log/syslog
> Aug 24 13:12:50 X5000LNX kernel: [ 6.552303] usb 1-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 6.649376] usb 2-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 6.794158] usb 1-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 6.883156] usb 2-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 7.139155] usb 1-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 7.226156] usb 2-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 7.363155] usb 1-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 7.451156] usb 2-1: device descriptor re=
ad/64, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 8.521156] usb 1-1: device not accepting=
 address 4, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 8.609156] usb 2-1: device not accepting=
 address 4, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 9.057155] usb 1-1: device not accepting=
 address 5, error -32
> Aug 24 13:12:50 X5000LNX kernel: [ 9.145157] usb 2-1: device not accepting=
 address 5, error -32
>=20
> I don=E2=80=99t have a P5020 board here. That means I can=E2=80=99t bisect=
 and test. Therefore I need your help. The problematic Git commit must be so=
mewhere between 22/08/18 1:20 PM UTC and 23/08/18 10:49 AM UTC.
> The Git kernel from 22/08/18 1:20 PM UTC works with USB devices.
> Could you please compile a Git kernel and test it on your P5020 board?
>=20
> Thanks,
> Christian

FSL/NXP P5020: USB problems with the latest Git kernels

[Christian Zigotzky]

Hello,

Our users tested the RC1 of kernel 4.19 on their P5020 boards today. Unfortu=
nately the USB bug still exists. With mem values bigger than 4096M, the USB m=
ouse and keyboard doesn=E2=80=99t work. With the bootarg mem=3D4096M, the US=
B devices work without any problems. Please compile the RC1 and test it on y=
our P5020 board. There is a problem with the memory management since 22/08/1=
8.=20

Thanks,
Christian

Sent from my iPhone

> On 25. Aug 2018, at 22:19, Christian Zigotzky <chzigotzky@xenosoft.de> wro=
te:
>=20
> Hello,
>=20
> With the boot argument =E2=80=9Cmem=3D3500M=E2=80=9D the USB devices work.=
 Please compile the latest Git kernel and test it on your P5020 board. There=
 is a problem with the memory management since 22/08/18. I can=E2=80=99t tes=
t because I don=E2=80=99t have a P5020 board here and our users can=E2=80=99=
t compile kernels.
>=20
> Thanks,
> Christian
>=20
> Sent from my iPhone
>=20
>> On 24. Aug 2018, at 20:11, Christian Zigotzky <chzigotzky@xenosoft.de> wr=
ote:
>>=20
>> Hello,
>>=20
>> There are USB problems with P5020 boards with the latest Git kernels.
>>=20
>> Some users reported the following error messages:
>>=20
>> skateman@X5000LNX:~$ grep -i -E 'usb.*error' /var/log/syslog
>> Aug 24 13:12:50 X5000LNX kernel: [ 6.552303] usb 1-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 6.649376] usb 2-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 6.794158] usb 1-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 6.883156] usb 2-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 7.139155] usb 1-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 7.226156] usb 2-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 7.363155] usb 1-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 7.451156] usb 2-1: device descriptor r=
ead/64, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 8.521156] usb 1-1: device not acceptin=
g address 4, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 8.609156] usb 2-1: device not acceptin=
g address 4, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 9.057155] usb 1-1: device not acceptin=
g address 5, error -32
>> Aug 24 13:12:50 X5000LNX kernel: [ 9.145157] usb 2-1: device not acceptin=
g address 5, error -32
>>=20
>> I don=E2=80=99t have a P5020 board here. That means I can=E2=80=99t bisec=
t and test. Therefore I need your help. The problematic Git commit must be s=
omewhere between 22/08/18 1:20 PM UTC and 23/08/18 10:49 AM UTC.
>> The Git kernel from 22/08/18 1:20 PM UTC works with USB devices.
>> Could you please compile a Git kernel and test it on your P5020 board?
>>=20
>> Thanks,
>> Christian

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Scott Wood]

On Mon, 2018-08-27 at 20:15 +0200, Christian Zigotzky wrote:
> Hello,
> 
> Our users tested the RC1 of kernel 4.19 on their P5020 boards today.
> Unfortunately the USB bug still exists. With mem values bigger than 4096M,
> the USB mouse and keyboard doesn’t work. With the bootarg mem=4096M, the USB
> devices work without any problems. Please compile the RC1 and test it on
> your P5020 board. There is a problem with the memory management since
> 22/08/18. 

I just tested 4.19-rc1 on a T4240 and got a similar problem with MMC.  MMC and
USB on these chips both have a 32-bit DMA limitation.  I'll look into it.

-Scott

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Darren Stevens]

Hello Scott

On 27/08/2018, Scott Wood wrote:
> On Mon, 2018-08-27 at 20:15 +0200, Christian Zigotzky wrote:
>> Hello,
>> 
>> Our users tested the RC1 of kernel 4.19 on their P5020 boards today.
>> Unfortunately the USB bug still exists. With mem values bigger than 4096M,
>> the USB mouse and keyboard doesn't work. With the bootarg mem=4096M, the USB
>> devices work without any problems. Please compile the RC1 and test it on
>> your P5020 board. There is a problem with the memory management since
>> 22/08/18. 
>
> I just tested 4.19-rc1 on a T4240 and got a similar problem with MMC.  MMC and
> USB on these chips both have a 32-bit DMA limitation.  I'll look into it.

I've run a bisect locally on my system and found the following bad commit:

# first bad commit: [1b1eeca7e4c19fa76d409d4c7b338dba21f2df45] init: allow initcall tables to be emitted using relative references

I can't see why this would fail though, and it only seems to affect fsl machines, and on mine it killed the onboard sata as well.

Hope this helps.

Regards
Darren

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Scott Wood]

On Tue, 2018-08-28 at 20:13 +0100, Darren Stevens wrote:
> Hello Scott
> 
> On 27/08/2018, Scott Wood wrote:
> > On Mon, 2018-08-27 at 20:15 +0200, Christian Zigotzky wrote:
> > > Hello,
> > > 
> > > Our users tested the RC1 of kernel 4.19 on their P5020 boards today.
> > > Unfortunately the USB bug still exists. With mem values bigger than
> > > 4096M,
> > > the USB mouse and keyboard doesn't work. With the bootarg mem=4096M, the
> > > USB
> > > devices work without any problems. Please compile the RC1 and test it on
> > > your P5020 board. There is a problem with the memory management since
> > > 22/08/18. 
> > 
> > I just tested 4.19-rc1 on a T4240 and got a similar problem with MMC.  MMC
> > and
> > USB on these chips both have a 32-bit DMA limitation.  I'll look into it.
> 
> I've run a bisect locally on my system and found the following bad commit:
> 
> # first bad commit: [1b1eeca7e4c19fa76d409d4c7b338dba21f2df45] init: allow
> initcall tables to be emitted using relative references
> 
> I can't see why this would fail though, and it only seems to affect fsl
> machines, and on mine it killed the onboard sata as well.
> 
> Hope this helps.


The patch is changing the order of initcalls.  swiotlb_setup_bus_notifier()
and corenet_gen_publish_devices() are both machine_arch_initcall(), and if the
latter happens before the former, then the bus notifier gets missed and
swiotlb doesn't get used.

-Scott

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Scott Wood]

On Wed, 2018-08-29 at 18:30 -0500, Scott Wood wrote:
> On Tue, 2018-08-28 at 20:13 +0100, Darren Stevens wrote:
> > Hello Scott
> > 
> > On 27/08/2018, Scott Wood wrote:
> > > On Mon, 2018-08-27 at 20:15 +0200, Christian Zigotzky wrote:
> > > > Hello,
> > > > 
> > > > Our users tested the RC1 of kernel 4.19 on their P5020 boards today.
> > > > Unfortunately the USB bug still exists. With mem values bigger than
> > > > 4096M,
> > > > the USB mouse and keyboard doesn't work. With the bootarg mem=4096M,
> > > > the
> > > > USB
> > > > devices work without any problems. Please compile the RC1 and test it
> > > > on
> > > > your P5020 board. There is a problem with the memory management since
> > > > 22/08/18. 
> > > 
> > > I just tested 4.19-rc1 on a T4240 and got a similar problem with
> > > MMC.  MMC
> > > and
> > > USB on these chips both have a 32-bit DMA limitation.  I'll look into
> > > it.
> > 
> > I've run a bisect locally on my system and found the following bad commit:
> > 
> > # first bad commit: [1b1eeca7e4c19fa76d409d4c7b338dba21f2df45] init: allow
> > initcall tables to be emitted using relative references
> > 
> > I can't see why this would fail though, and it only seems to affect fsl
> > machines, and on mine it killed the onboard sata as well.
> > 
> > Hope this helps.
> 
> 
> The patch is changing the order of initcalls.  swiotlb_setup_bus_notifier()
> and corenet_gen_publish_devices() are both machine_arch_initcall(), and if
> the
> latter happens before the former, then the bus notifier gets missed and
> swiotlb doesn't get used.

While it would be nice to fix this ordering dependency at some point, the
change was reverted on PPC by commit ff69279a44e9ba876466 ("powerpc: disable
support for relative ksymtab references").

-Scott

RE: FSL/NXP P5020: USB problems with the latest Git kernels

[Laurentiu Tudor]

SGkgU2NvdHQsDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0gDQo+IE9uIE1vbiwgMjAx
OC0wOC0yNyBhdCAyMDoxNSArMDIwMCwgQ2hyaXN0aWFuIFppZ290emt5IHdyb3RlOg0KPiA+IEhl
bGxvLA0KPiA+DQo+ID4gT3VyIHVzZXJzIHRlc3RlZCB0aGUgUkMxIG9mIGtlcm5lbCA0LjE5IG9u
IHRoZWlyIFA1MDIwIGJvYXJkcyB0b2RheS4NCj4gPiBVbmZvcnR1bmF0ZWx5IHRoZSBVU0IgYnVn
IHN0aWxsIGV4aXN0cy4gV2l0aCBtZW0gdmFsdWVzIGJpZ2dlciB0aGFuDQo+IDQwOTZNLA0KPiA+
IHRoZSBVU0IgbW91c2UgYW5kIGtleWJvYXJkIGRvZXNu4oCZdCB3b3JrLiBXaXRoIHRoZSBib290
YXJnIG1lbT00MDk2TSwgdGhlDQo+IFVTQg0KPiA+IGRldmljZXMgd29yayB3aXRob3V0IGFueSBw
cm9ibGVtcy4gUGxlYXNlIGNvbXBpbGUgdGhlIFJDMSBhbmQgdGVzdCBpdCBvbg0KPiA+IHlvdXIg
UDUwMjAgYm9hcmQuIFRoZXJlIGlzIGEgcHJvYmxlbSB3aXRoIHRoZSBtZW1vcnkgbWFuYWdlbWVu
dCBzaW5jZQ0KPiA+IDIyLzA4LzE4Lg0KPiANCj4gSSBqdXN0IHRlc3RlZCA0LjE5LXJjMSBvbiBh
IFQ0MjQwIGFuZCBnb3QgYSBzaW1pbGFyIHByb2JsZW0gd2l0aCBNTUMuICBNTUMNCj4gYW5kDQo+
IFVTQiBvbiB0aGVzZSBjaGlwcyBib3RoIGhhdmUgYSAzMi1iaXQgRE1BIGxpbWl0YXRpb24uICBJ
J2xsIGxvb2sgaW50byBpdC4NCg0KSSBlbmNvdW50ZXJlZCBzaW1pbGFyIGlzc3VlcyBvbiBMUzEw
NHhBIGNoaXBzLiBUaGlzIGlzIHRoZSB3b3JrYXJvdW5kIGZvciBNTUM6DQpodHRwczovL3BhdGNo
d29yay5rZXJuZWwub3JnL3BhdGNoLzEwNTA2NjI3Lw0KT24gdGhlIFVTQiBzaWRlIEkgZGlkbid0
IGZpbmQgYSBwcm9wZXIgcGxhY2UgaW4gdGhlIHVzYiBzdWJzeXN0ZW0gY29kZSBmb3IgYSB3b3Jr
YXJvdW5kIGJ1dCBpbnN0ZWFkDQpmb3VuZCBhbiB1bmRvY3VtZW50ZWQga2VybmVsIGFyZyAoKikg
dGhhdCBsaW1pdHMgdGhlIGRtYSBtYXNrIHRvIDMyIGJpdHM6DQp4aGNpLWhjZC5xdWlya3M9MHg4
MDAwMDAuDQoNCigqKSBodHRwczovL3BhdGNod29yay5rZXJuZWwub3JnL3BhdGNoLzEwNTA5MTU5
Lw0KDQotLS0NCkJlc3QgUmVnYXJkcywgTGF1cmVudGl1DQo=

Re: FSL/NXP P5020: USB problems with the latest Git kernels

[Scott Wood]

On Wed, 2018-09-05 at 14:08 +0000, Laurentiu Tudor wrote:
> Hi Scott,
> 
> > -----Original Message----- 
> > On Mon, 2018-08-27 at 20:15 +0200, Christian Zigotzky wrote:
> > > Hello,
> > > 
> > > Our users tested the RC1 of kernel 4.19 on their P5020 boards today.
> > > Unfortunately the USB bug still exists. With mem values bigger than
> > 
> > 4096M,
> > > the USB mouse and keyboard doesn’t work. With the bootarg mem=4096M, the
> > 
> > USB
> > > devices work without any problems. Please compile the RC1 and test it on
> > > your P5020 board. There is a problem with the memory management since
> > > 22/08/18.
> > 
> > I just tested 4.19-rc1 on a T4240 and got a similar problem with MMC.  MMC
> > and
> > USB on these chips both have a 32-bit DMA limitation.  I'll look into it.
> 
> I encountered similar issues on LS104xA chips. This is the workaround for
> MMC:
> https://patchwork.kernel.org/patch/10506627/

That's not a workaround; it's the proper way to handle DMA addressing limits.

> On the USB side I didn't find a proper place in the usb subsystem code for a
> workaround but instead
> found an undocumented kernel arg (*) that limits the dma mask to 32 bits:
> xhci-hcd.quirks=0x800000.
> 
> (*) https://patchwork.kernel.org/patch/10509159/

The mask should already be getting set to 32 bits.  The problem is a recent
patch that changed initcall order, which lead to swiotlb not being used.  We
need to rework the swiotlb notifier so it doesn't depend on initcall ordering.

-Scott