[U-Boot] XHCI Issues

[U-Boot] XHCI Issues

[Aaron Williams]

Hi all,

I am bringing up XHCI support for our SOC and have run into several 
issues with U-Boot's XHCI code.

1. I need to use a wrapper I call xhci_readl/xhci_writel since all of 
our I/O is to 64-bit addresses rather than readl/writel which work for 
PCIe. While xhci_readl/xhci_writel is used in most places there are a 
few places this is missing.

2. A memory address used by a pointer is not a DMA address on MIPS. On 
MIPS a wrapper is needed for all read/write operations to descriptors to 
map the addresses. Generally U-Boot runs in KSEG0 where a pointer always 
has bit 31 set. In our case we run it in virtual memory with U-Boot 
loaded at the top of memory (which might be as high as 64GB+256MB).

3. xhci_alloc_virt_device is missing a conversion from big endian to 
little endian on line 401 of xhci-mem.c.

4. I keep getting hit with

BUG_ON(GET_COMP_CODE(le32_to_cpu(event->trans_event.transfer_len
         != COMP_STOP)));
on line 491 in xhci-ring.c. Commenting this out seems to work fine.

5. I also hit the BUG_ON on line 722 of xhci-ring.c since on MIPS a 
pointer is not a DMA address and the fact that we use 64-bit addresses 
where pointers are 32-bits.

I ran into similar issues with EHCI where again we have to map pointers 
to physical addresses and use our own wrappers for register accesses.

At some point I would love to merge our stuff back into the mainline 
U-Boot but it's going to be a huge job given that we probably have 
almost 250K lines of code involved for our MIPS SOCs.

-Aaron

-- 
Aaron Williams
Software Engineer
Cavium, Inc.
(408) 943-7198  (510) 789-8988 (cell)

[U-Boot] XHCI Issues

[Aaron Williams]

On 01/13/2014 06:20 PM, Aaron Williams wrote:
> Hi all,
>
> I am bringing up XHCI support for our SOC and have run into several 
> issues with U-Boot's XHCI code.
>
> 1. I need to use a wrapper I call xhci_readl/xhci_writel since all of 
> our I/O is to 64-bit addresses rather than readl/writel which work for 
> PCIe. While xhci_readl/xhci_writel is used in most places there are a 
> few places this is missing.
>
> 2. A memory address used by a pointer is not a DMA address on MIPS. On 
> MIPS a wrapper is needed for all read/write operations to descriptors 
> to map the addresses. Generally U-Boot runs in KSEG0 where a pointer 
> always has bit 31 set. In our case we run it in virtual memory with 
> U-Boot loaded at the top of memory (which might be as high as 
> 64GB+256MB).
>
> 3. xhci_alloc_virt_device is missing a conversion from big endian to 
> little endian on line 401 of xhci-mem.c.
>
> 4. I keep getting hit with
>
> BUG_ON(GET_COMP_CODE(le32_to_cpu(event->trans_event.transfer_len
>         != COMP_STOP)));
> on line 491 in xhci-ring.c. Commenting this out seems to work fine.
>
> 5. I also hit the BUG_ON on line 722 of xhci-ring.c since on MIPS a 
> pointer is not a DMA address and the fact that we use 64-bit addresses 
> where pointers are 32-bits.
>
> I ran into similar issues with EHCI where again we have to map 
> pointers to physical addresses and use our own wrappers for register 
> accesses.
>
> At some point I would love to merge our stuff back into the mainline 
> U-Boot but it's going to be a huge job given that we probably have 
> almost 250K lines of code involved for our MIPS SOCs.
>
> -Aaron
>
I have a follow-up:

I am seeing error messages when hub ports are reset.

In the output below I have a USB 3 thumb drive plugged into USB0 and a 
USB 2 thumb drive plugged into USB1. Each USB device has "2" ports, 
where port 0 is treated as a USB 3 port and port 1 is treated as a USB 
1/2 port. I think the "cannot reset port X!?" message is invalid since 
otherwise everything appears to be working.

I am also seeing rather slow transfers of around 1.9MiB/sec. Is this 
normal for XHCI? This is for both USB 2 and a high-speed USB 3 thumb 
drive when loading a 40MiB Linux kernel image.

-Aaron

-- 
Aaron Williams
Software Engineer
Cavium, Inc.
(408) 943-7198  (510) 789-8988 (cell)

[U-Boot] XHCI Issues

[Aaron Williams]

More info
On 01/13/2014 08:28 PM, Aaron Williams wrote:
> On 01/13/2014 06:20 PM, Aaron Williams wrote:
>> Hi all,
>>
>> I am bringing up XHCI support for our SOC and have run into several 
>> issues with U-Boot's XHCI code.
>>
>> 1. I need to use a wrapper I call xhci_readl/xhci_writel since all of 
>> our I/O is to 64-bit addresses rather than readl/writel which work 
>> for PCIe. While xhci_readl/xhci_writel is used in most places there 
>> are a few places this is missing.
>>
>> 2. A memory address used by a pointer is not a DMA address on MIPS. 
>> On MIPS a wrapper is needed for all read/write operations to 
>> descriptors to map the addresses. Generally U-Boot runs in KSEG0 
>> where a pointer always has bit 31 set. In our case we run it in 
>> virtual memory with U-Boot loaded at the top of memory (which might 
>> be as high as 64GB+256MB).
>>
>> 3. xhci_alloc_virt_device is missing a conversion from big endian to 
>> little endian on line 401 of xhci-mem.c.
>>
>> 4. I keep getting hit with
>>
>> BUG_ON(GET_COMP_CODE(le32_to_cpu(event->trans_event.transfer_len
>>         != COMP_STOP)));
>> on line 491 in xhci-ring.c. Commenting this out seems to work fine.
>>
>> 5. I also hit the BUG_ON on line 722 of xhci-ring.c since on MIPS a 
>> pointer is not a DMA address and the fact that we use 64-bit 
>> addresses where pointers are 32-bits.
>>
>> I ran into similar issues with EHCI where again we have to map 
>> pointers to physical addresses and use our own wrappers for register 
>> accesses.
>>
>> At some point I would love to merge our stuff back into the mainline 
>> U-Boot but it's going to be a huge job given that we probably have 
>> almost 250K lines of code involved for our MIPS SOCs.
>>
>> -Aaron
>>
> I have a follow-up:
>
> I am seeing error messages when hub ports are reset.
>
> In the output below I have a USB 3 thumb drive plugged into USB0 and a 
> USB 2 thumb drive plugged into USB1. Each USB device has "2" ports, 
> where port 0 is treated as a USB 3 port and port 1 is treated as a USB 
> 1/2 port. I think the "cannot reset port X!?" message is invalid since 
> otherwise everything appears to be working.
>
> I am also seeing rather slow transfers of around 1.9MiB/sec. Is this 
> normal for XHCI? This is for both USB 2 and a high-speed USB 3 thumb 
> drive when loading a 40MiB Linux kernel image.
>
> -Aaron
>
I have made some progress on the performance issue. The poor performance 
is due to the fact that usb_storage.c limits a transfer to 20 blocks 
except for EHCI where it limits it to 65535 blocks (see USB_MAX_XFER_BLK 
in usb_storage.c).

For XHCI if I set it to 65535 blocks I get failures where I se 
Unexpected XHCI event TRB. The type received is 33 but it expects 32. 
When I lowered this value to 4096 everything worked fine for a 
high-speed USB 3 device where I was getting over 90MiB/sec but it would 
fail with a USB 2 flash drive. Lowering this number to 1024 seems to 
work for both. I am guessing that something is timing out or something 
when the number of blocks is large.

Note that we have no issues with our XHCI implementation in Linux.

-Aaron

-- 
Aaron Williams
Software Engineer
Cavium, Inc.
(408) 943-7198  (510) 789-8988 (cell)