USB issue on Freescale's i.MX35 3-stack board

USB issue on Freescale's i.MX35 3-stack board

[Nguyen Dinh-R00091]

Hi,

On the Freescale i.MX35 3DS board, there is a total of 256MB of DDR2 and the configuration is 128MB on Chip-select 0 and 128MB on chip-select 1. So physically, the memory is not contiguous, so we boot up the system with these mem= parameters(mem=128M at 0x80000000 mem=128M at 0x90000000). The system seems to be operating fine and passes stress testing using memtester and oprofile, except for USB.

Running a USB copy stress test on USB mass storage between a USB HDD and an SD/MMC card causes the system to hang. The hang can occur at any time between 10 minutes to 2 hours. The hang is much harder to re-create if logs or delays are introduced.

A few data points from my debugging:
1) If I run the system at 128MB(either at physical 0x80000000 or 0x90000000) then the hang never occurs.
2) Running the same test on a Freescale i.MX53 with a similar memory configuration also does not exhibit the error.
3) This failure can be reproduced on the both the Freescale BSP and the kernel.org BSP for MX35-3DS system.
4) I'm suspecting a memory abort access when the USB HW is DMA-ing its qtd and buffers, but I can't seem to trap on this condition. Any logs that I introduce into the EHCI driver seems to make the hang go away.

If anyone has any idea on how I can further debug, or better yet a suggested fix, I would greatly appreciate it. Also I am available to work with anyone to fix this problem if anyone is interested.

Thanks and best regards,

Dinh

USB issue on Freescale's i.MX35 3-stack board

[Greg KH]

On Mon, May 16, 2011 at 02:15:36PM +0000, Nguyen Dinh-R00091 wrote:
> Hi,
> 
> On the Freescale i.MX35 3DS board, there is a total of 256MB of DDR2
> and the configuration is 128MB on Chip-select 0 and 128MB on
> chip-select 1. So physically, the memory is not contiguous, so we boot
> up the system with these mem= parameters(mem=128M at 0x80000000
> mem=128M at 0x90000000). The system seems to be operating fine and passes
> stress testing using memtester and oprofile, except for USB.
> 
> Running a USB copy stress test on USB mass storage between a USB HDD
> and an SD/MMC card causes the system to hang. The hang can occur at
> any time between 10 minutes to 2 hours. The hang is much harder to
> re-create if logs or delays are introduced.

What kernel version are you using, and what host controller drivers?

thanks,

greg k-h

USB issue on Freescale's i.MX35 3-stack board

[Nguyen Dinh-R00091]

Hi Greg,

>-----Original Message-----
>From: Greg KH [mailto:greg at kroah.com]
>Sent: Monday, May 16, 2011 3:32 PM
>To: Nguyen Dinh-R00091
>Cc: linux-usb at vger.kernel.org; linux-arm-kernel at lists.infradead.org; daniel at caiaq.de;
>grant.likely at secretlab.ca; u.kleine-koenig at pengutronix.de; dbrownell at users.sourceforge.net;
>s.hauer at pengutronix.de; Mahadevan Mahesh-R9AADQ
>Subject: Re: USB issue on Freescale's i.MX35 3-stack board
>
>On Mon, May 16, 2011 at 02:15:36PM +0000, Nguyen Dinh-R00091 wrote:
>> Hi,
>>
>> On the Freescale i.MX35 3DS board, there is a total of 256MB of DDR2
>> and the configuration is 128MB on Chip-select 0 and 128MB on
>> chip-select 1. So physically, the memory is not contiguous, so we boot
>> up the system with these mem= parameters(mem=128M at 0x80000000
>> mem=128M at 0x90000000). The system seems to be operating fine and passes
>> stress testing using memtester and oprofile, except for USB.
>>
>> Running a USB copy stress test on USB mass storage between a USB HDD
>> and an SD/MMC card causes the system to hang. The hang can occur at
>> any time between 10 minutes to 2 hours. The hang is much harder to
>> re-create if logs or delays are introduced.
>
>What kernel version are you using, and what host controller drivers?

It's the EHCI USB controller. I'm currently debugging on 2.6.35, but the bug is can also be seen on 2.6.38.

Dinh
>
>thanks,
>
>greg k-h

USB issue on Freescale's i.MX35 3-stack board

[Alan Stern]

On Mon, 16 May 2011, Nguyen Dinh-R00091 wrote:

> Hi,
> 
> On the Freescale i.MX35 3DS board, there is a total of 256MB of DDR2 and the configuration is 128MB on Chip-select 0 and 128MB on chip-select 1. So physically, the memory is not contiguous, so we boot up the system with these mem= parameters(mem=128M at 0x80000000 mem=128M at 0x90000000). The system seems to be operating fine and passes stress testing using memtester and oprofile, except for USB.
> 
> Running a USB copy stress test on USB mass storage between a USB HDD and an SD/MMC card causes the system to hang. The hang can occur at any time between 10 minutes to 2 hours. The hang is much harder to re-create if logs or delays are introduced.

Your test involves both the USB and MMC subsystems.  How can you tell
which one is responsible for the hang?

Have you tried testing these subsystems one at a time?

Alan Stern

USB issue on Freescale's i.MX35 3-stack board

[Nguyen Dinh-R00091]

Hi Alan,

>-----Original Message-----
>From: Alan Stern [mailto:stern at rowland.harvard.edu]
>Sent: Monday, May 16, 2011 3:49 PM
>To: Nguyen Dinh-R00091
>Cc: linux-usb at vger.kernel.org; linux-arm-kernel at lists.infradead.org; daniel at caiaq.de;
>grant.likely at secretlab.ca; u.kleine-koenig at pengutronix.de; s.hauer at pengutronix.de; Mahadevan Mahesh-
>R9AADQ
>Subject: Re: USB issue on Freescale's i.MX35 3-stack board
>
>On Mon, 16 May 2011, Nguyen Dinh-R00091 wrote:
>
>> Hi,
>>
>> On the Freescale i.MX35 3DS board, there is a total of 256MB of DDR2 and the configuration is 128MB
>on Chip-select 0 and 128MB on chip-select 1. So physically, the memory is not contiguous, so we boot
>up the system with these mem= parameters(mem=128M at 0x80000000 mem=128M at 0x90000000). The system seems to
>be operating fine and passes stress testing using memtester and oprofile, except for USB.
>>
>> Running a USB copy stress test on USB mass storage between a USB HDD and an SD/MMC card causes the
>system to hang. The hang can occur at any time between 10 minutes to 2 hours. The hang is much harder
>to re-create if logs or delays are introduced.
>
>Your test involves both the USB and MMC subsystems.  How can you tell
>which one is responsible for the hang?
>
>Have you tried testing these subsystems one at a time?

Yes, I've stress tested on USB alone and MMC alone. The issue doesn't seem to appear with a MMC test, but appears when I am moving files between 2 USB HDDs. I apologize for not adding that additional data point on my original email.

Dinh

>
>Alan Stern
>