PCIe MSI support

PCIe MSI support

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Hi,

I was wondering whether anyone had gotten MSI to work with Tegra2? The code
in arch/arm/mach-tegra/pcie.c has some comments saying that it currently
isn't supported.

Looking through the Tegra2 TRM didn't show much related to MSI. In particular
there doesn't seem to be any mention of the AFI controller or any of it's
registers (except of how to enable/disable the AFI clock or reset the
controller).

Furthermore I wasn't able to get PCIe to work on Harmony with a mainline
kernel. Is something missing or am I just doing something wrong? I can make
it work with the Vibrante kernel, but that doesn't have MSI support either.
Since we're looking at designing an evaluation board with PCIe support I'm
very interested in seeing this supported in mainline. If there's anything I
can do to help, please let me know. Also if anyone can provide more insight
than the TRM, that would be great.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: PCIe MSI support

[Olof Johansson]

Hi,

On Thu, Nov 24, 2011 at 11:57 AM, Thierry Reding
<thierry.reding-RM9K5IK7kjKj5M59NBduVrNAH6kLmebB@public.gmane.org> wrote:
> Hi,
>
> I was wondering whether anyone had gotten MSI to work with Tegra2? The code
> in arch/arm/mach-tegra/pcie.c has some comments saying that it currently
> isn't supported.
>
> Looking through the Tegra2 TRM didn't show much related to MSI. In particular
> there doesn't seem to be any mention of the AFI controller or any of it's
> registers (except of how to enable/disable the AFI clock or reset the
> controller).
>
> Furthermore I wasn't able to get PCIe to work on Harmony with a mainline
> kernel. Is something missing or am I just doing something wrong?

I think there might be an issue with regulator setup for pci-e, I got
a vague bug report off-list from someone about it but I'm normally not
using devices with pci-e on them myself.

Note that trimslice does enable pci-e, so you can use that for comparison.

> I can make
> it work with the Vibrante kernel, but that doesn't have MSI support either.
> Since we're looking at designing an evaluation board with PCIe support I'm
> very interested in seeing this supported in mainline. If there's anything I
> can do to help, please let me know. Also if anyone can provide more insight
> than the TRM, that would be great.

I have no idea if MSI is supported or not. You are aware of the errata
list for PCI-e on Tegra though, right? You might want to consider the
implications of said problems before you decide to use PCI-e on a
product.

On more MSI details; I suggest you try to get access to internal
information through your Nvidia FAE -- I don't have any more
information on it than you do but if I wanted to find out that is the
route I would take myself.


Regards,

-Olof

Re: PCIe MSI support

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 2856 bytes --]

* Olof Johansson wrote:
> Hi,
> 
> On Thu, Nov 24, 2011 at 11:57 AM, Thierry Reding
> <thierry.reding-RM9K5IK7kjKj5M59NBduVrNAH6kLmebB@public.gmane.org> wrote:
> > Hi,
> >
> > I was wondering whether anyone had gotten MSI to work with Tegra2? The code
> > in arch/arm/mach-tegra/pcie.c has some comments saying that it currently
> > isn't supported.
> >
> > Looking through the Tegra2 TRM didn't show much related to MSI. In particular
> > there doesn't seem to be any mention of the AFI controller or any of it's
> > registers (except of how to enable/disable the AFI clock or reset the
> > controller).
> >
> > Furthermore I wasn't able to get PCIe to work on Harmony with a mainline
> > kernel. Is something missing or am I just doing something wrong?
> 
> I think there might be an issue with regulator setup for pci-e, I got
> a vague bug report off-list from someone about it but I'm normally not
> using devices with pci-e on them myself.

I've verified with a hardware engineer that the voltages indeed seem correct.
Even the PCIe reference clock can be measured at the mini-PCIe connectors. So
something must be going right.

> Note that trimslice does enable pci-e, so you can use that for comparison.

Trimslice doesn't seem to do much regarding PCIe initialization. It basically
just calls the tegra_pcie_init() and has the pinmuxing set up via the pinmux
table as opposed to Harmony.

I'm thinking about ordering a Trimslice to use for comparison. Has anyone
else verified that mainline has working PCIe for Trimslice? I think Mike
Rapoport is no longer a regular on this list. His last commits seem to
indicate that PCIe was working on Harmony as well at the time, so I'm a
little confused about what might have happened since then.

> > I can make
> > it work with the Vibrante kernel, but that doesn't have MSI support either.
> > Since we're looking at designing an evaluation board with PCIe support I'm
> > very interested in seeing this supported in mainline. If there's anything I
> > can do to help, please let me know. Also if anyone can provide more insight
> > than the TRM, that would be great.
> 
> I have no idea if MSI is supported or not. You are aware of the errata
> list for PCI-e on Tegra though, right? You might want to consider the
> implications of said problems before you decide to use PCI-e on a
> product.

Yes, I am aware of it. The problems shouldn't be show-stoppers, though.

> On more MSI details; I suggest you try to get access to internal
> information through your Nvidia FAE -- I don't have any more
> information on it than you do but if I wanted to find out that is the
> route I would take myself.

I did that and our FAE is now trying to make documentation available
regarding the AFI controller and MSI initialization.

Thanks,
Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: PCIe MSI support

[Olof Johansson]

Hi,

On Thu, Dec 1, 2011 at 11:43 PM, Thierry Reding
<thierry.reding-RM9K5IK7kjKj5M59NBduVrNAH6kLmebB@public.gmane.org> wrote:
> * Olof Johansson wrote:
>> Hi,
>>
>> On Thu, Nov 24, 2011 at 11:57 AM, Thierry Reding
>> <thierry.reding-RM9K5IK7kjKj5M59NBduVrNAH6kLmebB@public.gmane.org> wrote:
>> > Hi,
>> >
>> > I was wondering whether anyone had gotten MSI to work with Tegra2? The code
>> > in arch/arm/mach-tegra/pcie.c has some comments saying that it currently
>> > isn't supported.
>> >
>> > Looking through the Tegra2 TRM didn't show much related to MSI. In particular
>> > there doesn't seem to be any mention of the AFI controller or any of it's
>> > registers (except of how to enable/disable the AFI clock or reset the
>> > controller).
>> >
>> > Furthermore I wasn't able to get PCIe to work on Harmony with a mainline
>> > kernel. Is something missing or am I just doing something wrong?
>>
>> I think there might be an issue with regulator setup for pci-e, I got
>> a vague bug report off-list from someone about it but I'm normally not
>> using devices with pci-e on them myself.
>
> I've verified with a hardware engineer that the voltages indeed seem correct.
> Even the PCIe reference clock can be measured at the mini-PCIe connectors. So
> something must be going right.

Ok, that's interesting. I haven't had a chance to give my harmony a
spin with a pci-e card here lately, I'll try to give it a go as soon
as I can but it'll be a few more days.

>> Note that trimslice does enable pci-e, so you can use that for comparison.
>
> Trimslice doesn't seem to do much regarding PCIe initialization. It basically
> just calls the tegra_pcie_init() and has the pinmuxing set up via the pinmux
> table as opposed to Harmony.

Right.

> I'm thinking about ordering a Trimslice to use for comparison. Has anyone
> else verified that mainline has working PCIe for Trimslice? I think Mike
> Rapoport is no longer a regular on this list. His last commits seem to
> indicate that PCIe was working on Harmony as well at the time, so I'm a
> little confused about what might have happened since then.

PCI-e works on mainline on trimslice, I have verified that myself. It
has ethernet on pci-e, and both u-boot and the kernel can use it.



Regards,

Olof

Re: PCIe MSI support

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

* Olof Johansson wrote:
> Hi,
> 
> On Thu, Dec 1, 2011 at 11:43 PM, Thierry Reding
> <thierry.reding-RM9K5IK7kjKj5M59NBduVrNAH6kLmebB@public.gmane.org> wrote:
[...]
> > I've verified with a hardware engineer that the voltages indeed seem correct.
> > Even the PCIe reference clock can be measured at the mini-PCIe connectors. So
> > something must be going right.
> 
> Ok, that's interesting. I haven't had a chance to give my harmony a
> spin with a pci-e card here lately, I'll try to give it a go as soon
> as I can but it'll be a few more days.

That'd be great. Perhaps it works fine for you and I'm just doing something
wrong.

[...]
> > I'm thinking about ordering a Trimslice to use for comparison. Has anyone
> > else verified that mainline has working PCIe for Trimslice? I think Mike
> > Rapoport is no longer a regular on this list. His last commits seem to
> > indicate that PCIe was working on Harmony as well at the time, so I'm a
> > little confused about what might have happened since then.
> 
> PCI-e works on mainline on trimslice, I have verified that myself. It
> has ethernet on pci-e, and both u-boot and the kernel can use it.

I've ordered a Trim-Slice and should have it late next week or beginning of
the week after. I can also try an older version of mainline from around the
time that Mike mainlined the Harmony support. I assume that at the time it
must have worked. Perhaps the issue can be bisected.

Unfortunately I'm rather busy with other things as well, so I won't be able
to work on this fulltime.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]