i.MX6/PCIe and MSI interrupts

i.MX6/PCIe and MSI interrupts

[Jürgen Beisert]

Hi,

is anybody out there who has MSI interrupts successfully working on i.MX6? I 
tried with my i.MX6 and a Gbit network card but without success. In legacy 
interrupt mode it works.
I'm using various patches here from the list and the PCIe itself works. The 
designware driver is prepared for MSI interrupts and the Exynos driver makes 
use of it.
I added the same glue code to the i.MX6 driver but I can't get the INTD 
interrupt to fire when MSI arrives (while the INTA fires in legacy interrupt 
mode). Some idea what needs to be done in the i.MX6 to get MSI working? The 
Exynos driver additionally changes some bits in its chipset beside the PCIe 
unit to enable the MSI interrupt to its CPU. Are there some 'hidden' bits in 
the i.MX6 as well?

Regards,
Juergen

-- 
Pengutronix e.K. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| Juergen Beisert ? ? ? ? ? ? |
Linux Solutions for Science and Industry ? ? ?| http://www.pengutronix.de/  |

i.MX6/PCIe and MSI interrupts

[Marek Vasut]

On Friday, December 06, 2013 at 10:32:31 AM, J?rgen Beisert wrote:
> Hi,
> 
> is anybody out there who has MSI interrupts successfully working on i.MX6?
> I tried with my i.MX6 and a Gbit network card but without success. In
> legacy interrupt mode it works.

What precise model of card do you use ? Do you have a PCIe switch in the path or 
just direct RC-EP connection ?

> I'm using various patches here from the list and the PCIe itself works. The
> designware driver is prepared for MSI interrupts and the Exynos driver
> makes use of it.

OK

> I added the same glue code to the i.MX6 driver but I can't get the INTD
> interrupt to fire when MSI arrives (while the INTA fires in legacy
> interrupt mode). Some idea what needs to be done in the i.MX6 to get MSI
> working? The Exynos driver additionally changes some bits in its chipset
> beside the PCIe unit to enable the MSI interrupt to its CPU. Are there
> some 'hidden' bits in the i.MX6 as well?

Can you post your prototype patch for the PCIe MSI so we can see what you did 
there please?

Best regards,
Marek Vasut

i.MX6/PCIe and MSI interrupts

[Harro Haan]

On 6 December 2013 12:43, Marek Vasut <marex@denx.de> wrote:
> On Friday, December 06, 2013 at 10:32:31 AM, J?rgen Beisert wrote:
>> Hi,
>>
>> is anybody out there who has MSI interrupts successfully working on i.MX6?
>> I tried with my i.MX6 and a Gbit network card but without success. In
>> legacy interrupt mode it works.
>
> What precise model of card do you use ? Do you have a PCIe switch in the path or
> just direct RC-EP connection ?
>
>> I'm using various patches here from the list and the PCIe itself works. The
>> designware driver is prepared for MSI interrupts and the Exynos driver
>> makes use of it.
>
> OK
>
>> I added the same glue code to the i.MX6 driver but I can't get the INTD
>> interrupt to fire when MSI arrives (while the INTA fires in legacy
>> interrupt mode). Some idea what needs to be done in the i.MX6 to get MSI
>> working? The Exynos driver additionally changes some bits in its chipset
>> beside the PCIe unit to enable the MSI interrupt to its CPU. Are there
>> some 'hidden' bits in the i.MX6 as well?
>
> Can you post your prototype patch for the PCIe MSI so we can see what you did
> there please?

Attached are my patches of the first attempt to get MSI working for
the i.MX6. It looks like MSI is working but MSIX is not in combination
with e1000e and SabreSD. I did some successful iperf tests in MSI
mode. I did the following hack in
drivers/net/ethernet/intel/e1000e/param.c to disable MSIX:

-		if (adapter->flags & FLAG_HAS_MSIX) {
+		if (!(adapter->flags & FLAG_HAS_MSIX)/*TEMP*/) {

In MSIX mode, the imx6_pcie_msi_irq_handler occurs, but
e1000_intr_msix_rx, e1000_intr_msix_tx or e1000_intr_msix_other does
not get called.

My MSI tests are not successful when testing it with a Xilinx PCIe
development board, while this FPGA design works properly with FSL
imx_3.0.35 (or on x86).

Best regards,

Harro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-ARM-dts-imx6qdl.dtsi-add-msi-interrupt.patch
Type: text/x-patch
Size: 1056 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20131206/c0c0bb8e/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-PCI-imx6-add-support-for-MSI.patch
Type: text/x-patch
Size: 2085 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20131206/c0c0bb8e/attachment-0001.bin>

i.MX6/PCIe and MSI interrupts

[Jürgen Beisert]

Hi Harro,

On Friday 06 December 2013 13:39:09 Harro Haan wrote:
> [...]
> Attached are my patches of the first attempt to get MSI working for
> the i.MX6. It looks like MSI is working but MSIX is not in combination
> with e1000e and SabreSD. I did some successful iperf tests in MSI
> mode. I did the following hack in
> drivers/net/ethernet/intel/e1000e/param.c to disable MSIX:
>
> -		if (adapter->flags & FLAG_HAS_MSIX) {
> +		if (!(adapter->flags & FLAG_HAS_MSIX)/*TEMP*/) {

Arrrgh, after applying this change it starts to work immediately. So many 
hours...

> My MSI tests are not successful when testing it with a Xilinx PCIe
> development board, while this FPGA design works properly with FSL
> imx_3.0.35 (or on x86).

Same here. Network chip works with MSI, PCIe FPGA does not.

Regards and thanks for your help,
Juergen

-- 
Pengutronix e.K. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| Juergen Beisert ? ? ? ? ? ? |
Linux Solutions for Science and Industry ? ? ?| http://www.pengutronix.de/  |

回复：Re: i.MX6/PCIe and MSI interrupts

[Marek Vasut]

On Friday, December 06, 2013 at 11:19:05 PM, Richard Zhu wrote:
> Hi:
> I find a way make imx6 pcie msi/msix work without modify e1000e driver.
> But e1000e can?t work after suspend/resume.:(.
> The other ep devices (iwl wifi, xhci) can work well.
> I can post my modification next monday if you want.

Please do, thanks! :)

> Sent from Huawei Mobile
> 
> J?rgen Beisert <jbe@pengutronix.de>???
> 
> Hi Harro,
> 
> On Friday 06 December 2013 13:39:09 Harro Haan wrote:
> > [...]
> > Attached are my patches of the first attempt to get MSI working for
> > the i.MX6. It looks like MSI is working but MSIX is not in combination
> > with e1000e and SabreSD. I did some successful iperf tests in MSI
> > mode. I did the following hack in
> > drivers/net/ethernet/intel/e1000e/param.c to disable MSIX:
> > 
> > -             if (adapter->flags & FLAG_HAS_MSIX) {
> > +             if (!(adapter->flags & FLAG_HAS_MSIX)/*TEMP*/) {
> 
> Arrrgh, after applying this change it starts to work immediately. So many
> hours...
> 
> > My MSI tests are not successful when testing it with a Xilinx PCIe
> > development board, while this FPGA design works properly with FSL
> > imx_3.0.35 (or on x86).
> 
> Same here. Network chip works with MSI, PCIe FPGA does not.
> 
> Regards and thanks for your help,
> Juergen
> 
> --
> Pengutronix e.K.                              | Juergen Beisert            
> | Linux Solutions for Science and Industry      |
> http://www.pengutronix.de/  |

Best regards,
Marek Vasut

i.MX6/PCIe and MSI interrupts

[Marek Vasut]

On Friday, December 06, 2013 at 03:15:50 PM, J?rgen Beisert wrote:
> Hi Harro,
> 
> On Friday 06 December 2013 13:39:09 Harro Haan wrote:
> > [...]
> > Attached are my patches of the first attempt to get MSI working for
> > the i.MX6. It looks like MSI is working but MSIX is not in combination
> > with e1000e and SabreSD. I did some successful iperf tests in MSI
> > mode. I did the following hack in
> > drivers/net/ethernet/intel/e1000e/param.c to disable MSIX:
> > 
> > -		if (adapter->flags & FLAG_HAS_MSIX) {
> > +		if (!(adapter->flags & FLAG_HAS_MSIX)/*TEMP*/) {
> 
> Arrrgh, after applying this change it starts to work immediately. So many
> hours...
> 
> > My MSI tests are not successful when testing it with a Xilinx PCIe
> > development board, while this FPGA design works properly with FSL
> > imx_3.0.35 (or on x86).
> 
> Same here. Network chip works with MSI, PCIe FPGA does not.

Can you try patch from Message-ID: <20131210043409.GA2734@pratyush-vbox> ?

Best regards,
Marek Vasut

i.MX6/PCIe and MSI interrupts

[Harro Haan]

On 10 December 2013 15:12, Marek Vasut <marex@denx.de> wrote:
> On Friday, December 06, 2013 at 03:15:50 PM, J?rgen Beisert wrote:
>> Hi Harro,
>>
>> On Friday 06 December 2013 13:39:09 Harro Haan wrote:
>> > [...]
>> > Attached are my patches of the first attempt to get MSI working for
>> > the i.MX6. It looks like MSI is working but MSIX is not in combination
>> > with e1000e and SabreSD. I did some successful iperf tests in MSI
>> > mode. I did the following hack in
>> > drivers/net/ethernet/intel/e1000e/param.c to disable MSIX:
>> >
>> > -           if (adapter->flags & FLAG_HAS_MSIX) {
>> > +           if (!(adapter->flags & FLAG_HAS_MSIX)/*TEMP*/) {
>>
>> Arrrgh, after applying this change it starts to work immediately. So many
>> hours...
>>
>> > My MSI tests are not successful when testing it with a Xilinx PCIe
>> > development board, while this FPGA design works properly with FSL
>> > imx_3.0.35 (or on x86).
>>
>> Same here. Network chip works with MSI, PCIe FPGA does not.
>
> Can you try patch from Message-ID: <20131210043409.GA2734@pratyush-vbox> ?

I assume you mean the following patch:
[PATCH 08/12] pcie: designware: Fix IO transfers
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-December/218048.html

This patch did not solve the problem with the FPGA design. After
further debugging it turned out that I was missing MSI IRQs. Therefore
the following patch is supplied:
[PATCH] pci: designware: fix missing msi irqs
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-December/218629.html

Regards,

Harro