MVNETA irq with backport-3.8

MVNETA irq with backport-3.8

[Greg]

Thomas, Willy,

I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT 
file.

When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in 
mvneta_open, with the following error :
> mvneta d0070000.ethernet eth0: cannot request irq 25

The oddness is this is not the actual IRQ to be requested, and the DTS 
files explicitly specify the IRQ is 8 for this device.

I'm not familiar with the DT system, where should I start looking to 
find the source of the problem ?

Thanks!

MVNETA irq with backport-3.8

[Maxime Ripard]

Hi Greg,

Le 07/05/2013 16:14, Greg a ?crit :
> Thomas, Willy,

I'm not familiar with the Marvell code, so I won't be able to answer
your question, but the usual way of getting someone's attention is to
send him a mail directly, while keeping the list in Cc.

There's a high risk of your mail being overlooked or answered to very
late if you don't do so.

Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

MVNETA irq with backport-3.8

[Greg]

Le 07/05/2013 17:27, Maxime Ripard a ?crit :
> Hi Greg,
>
> Le 07/05/2013 16:14, Greg a ?crit :
>> Thomas, Willy,
> I'm not familiar with the Marvell code, so I won't be able to answer
> your question, but the usual way of getting someone's attention is to
> send him a mail directly, while keeping the list in Cc.
>
> There's a high risk of your mail being overlooked or answered to very
> late if you don't do so.
>
>
Thanks for the advice!

By the way, the problem might not be marvell-related since the irq 
number is given by :
> irq_of_parse_and_map(dn, 0);

I enabled debug output in drivers/of/irq.c and it reads :
> of_irq_map_one: dev=/soc/ethernet at d0070000, index=0
>  intspec=8 intlen=1
>  intsize=1 intlen=1
> of_irq_map_raw: par=/interrupt-controller at d0020000,intspec=[0x00000008 
> 0x00000003...],ointsize=1
> of_irq_map_raw: ipar=/interrupt-controller at d0020000, size=1
>  -> addrsize=1
>  -> got it !

I'll continue digging into this once the headache I got while reading 
the code is gone ;-)

MVNETA irq with backport-3.8

[Thomas Petazzoni]

Dear Greg,

On Tue, 07 May 2013 16:14:01 +0200, Greg wrote:

> I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT 
> file.
> 
> When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in 
> mvneta_open, with the following error :
> > mvneta d0070000.ethernet eth0: cannot request irq 25
> 
> The oddness is this is not the actual IRQ to be requested, and the DTS 
> files explicitly specify the IRQ is 8 for this device.
> 
> I'm not familiar with the DT system, where should I start looking to 
> find the source of the problem ?

This issue should be fixed by:

commit 7f23f62fc31c5c97947414c0937a72e08a947a41
Author: Gregory CLEMENT <gregory.clement@free-electrons.com>
Date:   Wed Mar 20 16:09:35 2013 +0100

    arm: mvebu: Use local interrupt only for the timer 0
    
    The commit 3a6f08a37 "arm: mvebu: Add support for local interrupt",
    managed the 28th first interrupts as local interrupt to match the
    hardware specification. Among these interrupts there are the Gigabits
    Ethernet ones used by the mvneta driver. Unfortunately the state of
    the percpu_irq API prevents the driver to use it.
    
    Indeed the interrupts have to be freed when the .stop() function is
    called. As the free_percpu_irq() function don't disable the interrupt
    line, we have to do it on each CPU before calling this. The function
    disable_percpu_irq() only disable the percpu on the current CPU and
    there is no function which allows to disable a percpu irq on a given
    CPU. Waiting for the extension of the percpu_irq API, this fix allows
    to use again the mvneta driver.
    
    Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
    Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
    Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
    Signed-off-by: Jason Cooper <jason@lakedaemon.net>

which is part of the backport-3.8 branch, so I'm a bit confused. Are
you sure you're using the latest backport-3.8 branch? If so, then I'll
do a test here to check this.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

MVNETA irq with backport-3.8

[Greg]

Le 07/05/2013 18:18, Thomas Petazzoni a ?crit :
> Dear Greg,
>
> On Tue, 07 May 2013 16:14:01 +0200, Greg wrote:
>
>> I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT
>> file.
>>
>> When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in
>> mvneta_open, with the following error :
>>> mvneta d0070000.ethernet eth0: cannot request irq 25
>> The oddness is this is not the actual IRQ to be requested, and the DTS
>> files explicitly specify the IRQ is 8 for this device.
>>
>> I'm not familiar with the DT system, where should I start looking to
>> find the source of the problem ?
> This issue should be fixed by:
>
> commit 7f23f62fc31c5c97947414c0937a72e08a947a41
> Author: Gregory CLEMENT <gregory.clement@free-electrons.com>
> Date:   Wed Mar 20 16:09:35 2013 +0100
>
>      arm: mvebu: Use local interrupt only for the timer 0
>      
>      The commit 3a6f08a37 "arm: mvebu: Add support for local interrupt",
>      managed the 28th first interrupts as local interrupt to match the
>      hardware specification. Among these interrupts there are the Gigabits
>      Ethernet ones used by the mvneta driver. Unfortunately the state of
>      the percpu_irq API prevents the driver to use it.
>      
>      Indeed the interrupts have to be freed when the .stop() function is
>      called. As the free_percpu_irq() function don't disable the interrupt
>      line, we have to do it on each CPU before calling this. The function
>      disable_percpu_irq() only disable the percpu on the current CPU and
>      there is no function which allows to disable a percpu irq on a given
>      CPU. Waiting for the extension of the percpu_irq API, this fix allows
>      to use again the mvneta driver.
>      
>      Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>      Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>      Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>      Signed-off-by: Jason Cooper <jason@lakedaemon.net>
>
> which is part of the backport-3.8 branch, so I'm a bit confused. Are
> you sure you're using the latest backport-3.8 branch? If so, then I'll
> do a test here to check this.
>
Thomas,

I could have done something wrong as I am not very familiar with git:
> root at axp-dev:/opt/linux-3.8-bp# git branch
> * backport-3.8
>   list
>   master
I cloned yesterday with :
> git clone -b backport-3.8 
> https://github.com/MISL-EBU-System-SW/mainline-public.git

is that ok ?

By the way I do have PCIe support enabled on the source tree I'm using.

Regards,

MVNETA irq with backport-3.8

[Greg]

Le 07/05/2013 18:18, Thomas Petazzoni a ?crit :
> Dear Greg,
>
> On Tue, 07 May 2013 16:14:01 +0200, Greg wrote:
>
>> I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT
>> file.
>>
>> When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in
>> mvneta_open, with the following error :
>>> mvneta d0070000.ethernet eth0: cannot request irq 25
>> The oddness is this is not the actual IRQ to be requested, and the DTS
>> files explicitly specify the IRQ is 8 for this device.
>>
>> I'm not familiar with the DT system, where should I start looking to
>> find the source of the problem ?
> This issue should be fixed by:
>
> commit 7f23f62fc31c5c97947414c0937a72e08a947a41
> Author: Gregory CLEMENT <gregory.clement@free-electrons.com>
> Date:   Wed Mar 20 16:09:35 2013 +0100
>
>      arm: mvebu: Use local interrupt only for the timer 0
>      
>      The commit 3a6f08a37 "arm: mvebu: Add support for local interrupt",
>      managed the 28th first interrupts as local interrupt to match the
>      hardware specification. Among these interrupts there are the Gigabits
>      Ethernet ones used by the mvneta driver. Unfortunately the state of
>      the percpu_irq API prevents the driver to use it.
>      
>      Indeed the interrupts have to be freed when the .stop() function is
>      called. As the free_percpu_irq() function don't disable the interrupt
>      line, we have to do it on each CPU before calling this. The function
>      disable_percpu_irq() only disable the percpu on the current CPU and
>      there is no function which allows to disable a percpu irq on a given
>      CPU. Waiting for the extension of the percpu_irq API, this fix allows
>      to use again the mvneta driver.
>      
>      Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>      Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>      Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>      Signed-off-by: Jason Cooper <jason@lakedaemon.net>
>
> which is part of the backport-3.8 branch, so I'm a bit confused. Are
> you sure you're using the latest backport-3.8 branch? If so, then I'll
> do a test here to check this.
>
By the way, the patch IS applied to the source tree I'm compiling.
I can see this in arch/arm/mach-mvebu/irq-armada-370-xp.c which is using 
ARMADA_370_XP_TIMER0_PER_CPU_IRQ instead of ARMADA_370_XP_MAX_PER_CPU_IRQS

Cheers,

MVNETA irq with backport-3.8

[Gregory CLEMENT]

Hi Greg,

On 05/07/2013 06:40 PM, Greg wrote:
> Le 07/05/2013 18:18, Thomas Petazzoni a ?crit :
>> Dear Greg,
>>
>> On Tue, 07 May 2013 16:14:01 +0200, Greg wrote:
>>
>>> I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT
>>> file.
>>>
>>> When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in
>>> mvneta_open, with the following error :
>>>> mvneta d0070000.ethernet eth0: cannot request irq 25
>>> The oddness is this is not the actual IRQ to be requested, and the DTS
>>> files explicitly specify the IRQ is 8 for this device.
>>>
>>> I'm not familiar with the DT system, where should I start looking to
>>> find the source of the problem ?
>> This issue should be fixed by:
>>
>> commit 7f23f62fc31c5c97947414c0937a72e08a947a41
>> Author: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> Date:   Wed Mar 20 16:09:35 2013 +0100
>>
>>      arm: mvebu: Use local interrupt only for the timer 0
>>      
>>      The commit 3a6f08a37 "arm: mvebu: Add support for local interrupt",
>>      managed the 28th first interrupts as local interrupt to match the
>>      hardware specification. Among these interrupts there are the Gigabits
>>      Ethernet ones used by the mvneta driver. Unfortunately the state of
>>      the percpu_irq API prevents the driver to use it.
>>      
>>      Indeed the interrupts have to be freed when the .stop() function is
>>      called. As the free_percpu_irq() function don't disable the interrupt
>>      line, we have to do it on each CPU before calling this. The function
>>      disable_percpu_irq() only disable the percpu on the current CPU and
>>      there is no function which allows to disable a percpu irq on a given
>>      CPU. Waiting for the extension of the percpu_irq API, this fix allows
>>      to use again the mvneta driver.
>>      
>>      Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>>      Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>>      Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>>      Signed-off-by: Jason Cooper <jason@lakedaemon.net>
>>
>> which is part of the backport-3.8 branch, so I'm a bit confused. Are
>> you sure you're using the latest backport-3.8 branch? If so, then I'll
>> do a test here to check this.
>>
> By the way, the patch IS applied to the source tree I'm compiling.
> I can see this in arch/arm/mach-mvebu/irq-armada-370-xp.c which is using 
> ARMADA_370_XP_TIMER0_PER_CPU_IRQ instead of ARMADA_370_XP_MAX_PER_CPU_IRQS

I built and tested backport-3.8 and indeed the Ethernet is broken.
It was cause by a recent batch of fixes that I added, now I have to
figure out why they have broken the ethernet whereas they were supposed
to make it work better!

Regards,

-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

MVNETA irq with backport-3.8

[Willy Tarreau]

Hi Gr?gory,

On Tue, May 07, 2013 at 11:27:48PM +0200, Gregory CLEMENT wrote:
> I built and tested backport-3.8 and indeed the Ethernet is broken.
> It was cause by a recent batch of fixes that I added, now I have to
> figure out why they have broken the ethernet whereas they were supposed
> to make it work better!

If that can help you, here is the list of (possibly relevant) patches
I have applied on top of 3.9 picked from your repository some time
ago (it's not up to date with the latest versions due to merge issues
inducing laziness on my side) :

$ git log --oneline --grep=free-electrons v3.9.. 
65d2b5c ARM: mvebu: Add Device Bus and CFI flash memory support to defconfig
26cabde ARM: mvebu: Add support for NOR flash device on Openblocks AX3 board
34b32e8 ARM: mvebu: Add support for NOR flash device on Armada XP-GP board
31504c0 ARM: mvebu: Add Device Bus support for Armada 370/XP SoC
8cc752e drivers: memory: Introduce Marvell EBU Device Bus driver
48c6ddc arm: mvebu: update defconfig with PCI and USB support
f4ed5ba arm: mvebu: PCIe Device Tree informations for Armada XP GP
b1ed2c5 arm: mvebu: PCIe Device Tree informations for Armada 370 DB
51e91a9 arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox
402f9ba arm: mvebu: PCIe Device Tree informations for Armada XP DB
4899409 arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4
fb6d69f arm: mvebu: add PCIe Device Tree informations for Armada XP
c523287 arm: mvebu: add PCIe Device Tree informations for Armada 370
998ee7a arm: mvebu: PCIe support is now available on mvebu
fc9b219 pci: PCIe driver for Marvell Armada 370/XP systems
4b765fb clk: mvebu: add more PCIe clocks for Armada XP
af3bcd9 clk: mvebu: create parent-child relation for PCIe clocks on Armada 370
6cee09e arm: pci: add a align_resource hook
80fa698 pci: infrastructure to add drivers in drivers/pci/host
91ca5a7 of/pci: Provide support for parsing PCI DT ranges property
e46b3e7 arm: plat-orion: remove addr-map code
d11befe arm: mach-mv78xx0: convert to use the mvebu-mbus driver
a508a65 arm: mach-orion5x: convert to use mvebu-mbus driver
e249639 arm: mach-dove: convert to use mvebu-mbus driver
70efe5e arm: mach-kirkwood: convert to use mvebu-mbus driver
37e82515 arm: mach-mvebu: convert to use mvebu-mbus driver
035f910 bus: introduce an Marvell EBU MBus driver
527d658 arm: mach-orion5x: use mv_mbus_dram_info() in PCI code
c1b44db arm: plat-orion: use mv_mbus_dram_info() in PCIe code
59463f4 arm: plat-orion: only build addr-map.c when needed

With those my AX3 works fine. If that can help, I can send them all to
you off-list, in case you notice a minor difference with something.

Regards,
Willy

MVNETA irq with backport-3.8

[Gregory CLEMENT]

Hi Willy,

On 05/07/2013 11:42 PM, Willy Tarreau wrote:
> Hi Gr?gory,
> 
> On Tue, May 07, 2013 at 11:27:48PM +0200, Gregory CLEMENT wrote:
>> I built and tested backport-3.8 and indeed the Ethernet is broken.
>> It was cause by a recent batch of fixes that I added, now I have to
>> figure out why they have broken the ethernet whereas they were supposed
>> to make it work better!
> 
> If that can help you, here is the list of (possibly relevant) patches
> I have applied on top of 3.9 picked from your repository some time
> ago (it's not up to date with the latest versions due to merge issues
> inducing laziness on my side) :
> 
> $ git log --oneline --grep=free-electrons v3.9.. 
> 65d2b5c ARM: mvebu: Add Device Bus and CFI flash memory support to defconfig
> 26cabde ARM: mvebu: Add support for NOR flash device on Openblocks AX3 board
> 34b32e8 ARM: mvebu: Add support for NOR flash device on Armada XP-GP board
> 31504c0 ARM: mvebu: Add Device Bus support for Armada 370/XP SoC
> 8cc752e drivers: memory: Introduce Marvell EBU Device Bus driver
> 48c6ddc arm: mvebu: update defconfig with PCI and USB support
> f4ed5ba arm: mvebu: PCIe Device Tree informations for Armada XP GP
> b1ed2c5 arm: mvebu: PCIe Device Tree informations for Armada 370 DB
> 51e91a9 arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox
> 402f9ba arm: mvebu: PCIe Device Tree informations for Armada XP DB
> 4899409 arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4
> fb6d69f arm: mvebu: add PCIe Device Tree informations for Armada XP
> c523287 arm: mvebu: add PCIe Device Tree informations for Armada 370
> 998ee7a arm: mvebu: PCIe support is now available on mvebu
> fc9b219 pci: PCIe driver for Marvell Armada 370/XP systems
> 4b765fb clk: mvebu: add more PCIe clocks for Armada XP
> af3bcd9 clk: mvebu: create parent-child relation for PCIe clocks on Armada 370
> 6cee09e arm: pci: add a align_resource hook
> 80fa698 pci: infrastructure to add drivers in drivers/pci/host
> 91ca5a7 of/pci: Provide support for parsing PCI DT ranges property
> e46b3e7 arm: plat-orion: remove addr-map code
> d11befe arm: mach-mv78xx0: convert to use the mvebu-mbus driver
> a508a65 arm: mach-orion5x: convert to use mvebu-mbus driver
> e249639 arm: mach-dove: convert to use mvebu-mbus driver
> 70efe5e arm: mach-kirkwood: convert to use mvebu-mbus driver
> 37e82515 arm: mach-mvebu: convert to use mvebu-mbus driver
> 035f910 bus: introduce an Marvell EBU MBus driver
> 527d658 arm: mach-orion5x: use mv_mbus_dram_info() in PCI code
> c1b44db arm: plat-orion: use mv_mbus_dram_info() in PCIe code
> 59463f4 arm: plat-orion: only build addr-map.c when needed
> 
> With those my AX3 works fine. If that can help, I can send them all to
> you off-list, in case you notice a minor difference with something.

Thanks for your help but I finally found the guilty commit, it was
"66d0539 net: mvneta: convert to percpu interrupt".

This commit was never submitted because finally I realized that it was
broken in SMP. Finally it was the commit "arm: mvebu: Use local
interrupt only for the timer 0" which was the correct solution.

With the first patch I had converted the mvneta driver to use percpu
IRQ, but as I explained in the log of the 2nd patch: "the interrupts
have to be freed when the .stop() function is called. As the
free_percpu_irq() function don't disable the interrupt line, we have
to do it on each CPU before calling this. The function
disable_percpu_irq() only disable the percpu on the current CPU and
there is no function which allows to disable a percpu irq on a given
CPU."

So Greg, the solution is just to revert the commit 66d0539. I am going
to update the backport-3.8 branch in a couple of minutes.

Regards,
-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

MVNETA irq with backport-3.8

[Willy Tarreau]

Hi Gr?gory,

On Tue, May 07, 2013 at 11:54:24PM +0200, Gregory CLEMENT wrote:
> Thanks for your help but I finally found the guilty commit, it was
> "66d0539 net: mvneta: convert to percpu interrupt".

Cool! Indeed I remember about this one that you proposed to me to
fix a regression I was experiencing on my mirabox after I picked
this one :

   f492fcf arm: mvebu: Add support for local interrupt

> This commit was never submitted because finally I realized that it was
> broken in SMP.

Yes I remember you explained me this.

> Finally it was the commit "arm: mvebu: Use local
> interrupt only for the timer 0" which was the correct solution.

Perfect.

(...)
> So Greg, the solution is just to revert the commit 66d0539. I am going
> to update the backport-3.8 branch in a couple of minutes.

Maybe it could be useful to think about backport-3.9 soon ? :-)

Best regards,
Willy

MVNETA irq with backport-3.8

[Greg]

Le 07/05/2013 23:54, Gregory CLEMENT a ?crit :
> Hi Willy,
>
> On 05/07/2013 11:42 PM, Willy Tarreau wrote:
>> Hi Gr?gory,
>>
>> On Tue, May 07, 2013 at 11:27:48PM +0200, Gregory CLEMENT wrote:
>>> I built and tested backport-3.8 and indeed the Ethernet is broken.
>>> It was cause by a recent batch of fixes that I added, now I have to
>>> figure out why they have broken the ethernet whereas they were supposed
>>> to make it work better!
>> If that can help you, here is the list of (possibly relevant) patches
>> I have applied on top of 3.9 picked from your repository some time
>> ago (it's not up to date with the latest versions due to merge issues
>> inducing laziness on my side) :
>>
>> $ git log --oneline --grep=free-electrons v3.9..
>> 65d2b5c ARM: mvebu: Add Device Bus and CFI flash memory support to defconfig
>> 26cabde ARM: mvebu: Add support for NOR flash device on Openblocks AX3 board
>> 34b32e8 ARM: mvebu: Add support for NOR flash device on Armada XP-GP board
>> 31504c0 ARM: mvebu: Add Device Bus support for Armada 370/XP SoC
>> 8cc752e drivers: memory: Introduce Marvell EBU Device Bus driver
>> 48c6ddc arm: mvebu: update defconfig with PCI and USB support
>> f4ed5ba arm: mvebu: PCIe Device Tree informations for Armada XP GP
>> b1ed2c5 arm: mvebu: PCIe Device Tree informations for Armada 370 DB
>> 51e91a9 arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox
>> 402f9ba arm: mvebu: PCIe Device Tree informations for Armada XP DB
>> 4899409 arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4
>> fb6d69f arm: mvebu: add PCIe Device Tree informations for Armada XP
>> c523287 arm: mvebu: add PCIe Device Tree informations for Armada 370
>> 998ee7a arm: mvebu: PCIe support is now available on mvebu
>> fc9b219 pci: PCIe driver for Marvell Armada 370/XP systems
>> 4b765fb clk: mvebu: add more PCIe clocks for Armada XP
>> af3bcd9 clk: mvebu: create parent-child relation for PCIe clocks on Armada 370
>> 6cee09e arm: pci: add a align_resource hook
>> 80fa698 pci: infrastructure to add drivers in drivers/pci/host
>> 91ca5a7 of/pci: Provide support for parsing PCI DT ranges property
>> e46b3e7 arm: plat-orion: remove addr-map code
>> d11befe arm: mach-mv78xx0: convert to use the mvebu-mbus driver
>> a508a65 arm: mach-orion5x: convert to use mvebu-mbus driver
>> e249639 arm: mach-dove: convert to use mvebu-mbus driver
>> 70efe5e arm: mach-kirkwood: convert to use mvebu-mbus driver
>> 37e82515 arm: mach-mvebu: convert to use mvebu-mbus driver
>> 035f910 bus: introduce an Marvell EBU MBus driver
>> 527d658 arm: mach-orion5x: use mv_mbus_dram_info() in PCI code
>> c1b44db arm: plat-orion: use mv_mbus_dram_info() in PCIe code
>> 59463f4 arm: plat-orion: only build addr-map.c when needed
>>
>> With those my AX3 works fine. If that can help, I can send them all to
>> you off-list, in case you notice a minor difference with something.
> Thanks for your help but I finally found the guilty commit, it was
> "66d0539 net: mvneta: convert to percpu interrupt".
>
> This commit was never submitted because finally I realized that it was
> broken in SMP. Finally it was the commit "arm: mvebu: Use local
> interrupt only for the timer 0" which was the correct solution.
>
> With the first patch I had converted the mvneta driver to use percpu
> IRQ, but as I explained in the log of the 2nd patch: "the interrupts
> have to be freed when the .stop() function is called. As the
> free_percpu_irq() function don't disable the interrupt line, we have
> to do it on each CPU before calling this. The function
> disable_percpu_irq() only disable the percpu on the current CPU and
> there is no function which allows to disable a percpu irq on a given
> CPU."
>
> So Greg, the solution is just to revert the commit 66d0539. I am going
> to update the backport-3.8 branch in a couple of minutes.
>
>
Gregory,

I pulled the change and got it working fine.
I also got the PCIe link established, I haven't tried to stress it yet, 
I will do so quickly.

I got a backtrace that might interrest you, a "deadlock" condition while 
polling network counters :

Deadlock detection:
> INFO: rcu_sched self-detected stall on CPU { 0}  (t=2100 jiffies 
> g=4229 c=4228 q=421)
> [<c0014e24>] (unwind_backtrace+0x0/0xfc) from [<c006c5c8>] 
> (rcu_check_callbacks+0x1d0/0x734)
> [<c006c5c8>] (rcu_check_callbacks+0x1d0/0x734) from [<c002ca24>] 
> (update_process_times+0x38/0x4c)
> [<c002ca24>] (update_process_times+0x38/0x4c) from [<c0058eb8>] 
> (tick_sched_timer+0x68/0x210)
> [<c0058eb8>] (tick_sched_timer+0x68/0x210) from [<c0040040>] 
> (__run_hrtimer.isra.17+0x74/0x134)
> [<c0040040>] (__run_hrtimer.isra.17+0x74/0x134) from [<c0040860>] 
> (hrtimer_interrupt+0xf8/0x2cc)
> [<c0040860>] (hrtimer_interrupt+0xf8/0x2cc) from [<c03569c0>] 
> (armada_370_xp_timer_interrupt+0x3c/0x50)
> [<c03569c0>] (armada_370_xp_timer_interrupt+0x3c/0x50) from 
> [<c0067420>] (handle_percpu_devid_irq+0x64/0x84)
> [<c0067420>] (handle_percpu_devid_irq+0x64/0x84) from [<c0063d08>] 
> (generic_handle_irq+0x24/0x30)
> [<c0063d08>] (generic_handle_irq+0x24/0x30) from [<c000eef4>] 
> (handle_IRQ+0x38/0x94)
> [<c000eef4>] (handle_IRQ+0x38/0x94) from [<c0008610>] 
> (armada_370_xp_handle_irq+0xa0/0xb4)
> [<c0008610>] (armada_370_xp_handle_irq+0xa0/0xb4) from [<c000dca0>] 
> (__irq_svc+0x40/0x50)
> Exception stack(0xee69bd20 to 0xee69bd68)
> bd20: 0094809d 00000001 f0ab5a55 0094809d eeba3000 ee69be10 00000610 
> c031a500
> bd40: ee69bf10 ef0f6380 4c000001 00000000 00000618 ee69bd68 c031a518 
> c031a558
> bd60: 00000013 ffffffff
Hung process
> [<c000dca0>] (__irq_svc+0x40/0x50) from [<c031a558>] 
> (mvneta_get_stats64+0x58/0xc0)
> [<c031a558>] (mvneta_get_stats64+0x58/0xc0) from [<c0379974>] 
> (dev_get_stats+0x38/0xb4)
> [<c0379974>] (dev_get_stats+0x38/0xb4) from [<c0379b10>] 
> (dev_seq_printf_stats+0x1c/0x114)
> [<c0379b10>] (dev_seq_printf_stats+0x1c/0x114) from [<c037d83c>] 
> (dev_seq_show+0x10/0x2c)
> [<c037d83c>] (dev_seq_show+0x10/0x2c) from [<c00bfe3c>] 
> (seq_read+0x330/0x4c8)
> [<c00bfe3c>] (seq_read+0x330/0x4c8) from [<c00ea2ac>] 
> (proc_reg_read+0x8c/0xd0)
> [<c00ea2ac>] (proc_reg_read+0x8c/0xd0) from [<c00a0554>] 
> (vfs_read+0xa0/0x134)
> [<c00a0554>] (vfs_read+0xa0/0x134) from [<c00a0628>] (sys_read+0x40/0x6c)
> [<c00a0628>] (sys_read+0x40/0x6c) from [<c000dfe0>] 
> (ret_fast_syscall+0x0/0x30)

I noticed this using "ethstatus" tool, you can see the counters are not 
incrementing smoothly under traffic.

Cheers,

MVNETA irq with backport-3.8

[Willy Tarreau]

Hi Greg,

On Tue, May 07, 2013 at 06:40:39PM +0200, Greg wrote:
> Le 07/05/2013 18:18, Thomas Petazzoni a ?crit :
> >Dear Greg,
> >
> >On Tue, 07 May 2013 16:14:01 +0200, Greg wrote:
> >
> >>I'm trying to boot off the the backport-3.-8 kernel with armada-xp-db DT
> >>file.
> >>
> >>When eth0 is brought up (I'm booting on NFS) the mvneta driver fails in
> >>mvneta_open, with the following error :
> >>>mvneta d0070000.ethernet eth0: cannot request irq 25
> >>The oddness is this is not the actual IRQ to be requested, and the DTS
> >>files explicitly specify the IRQ is 8 for this device.
> >>
> >>I'm not familiar with the DT system, where should I start looking to
> >>find the source of the problem ?
> >This issue should be fixed by:
> >
> >commit 7f23f62fc31c5c97947414c0937a72e08a947a41
> >Author: Gregory CLEMENT <gregory.clement@free-electrons.com>
> >Date:   Wed Mar 20 16:09:35 2013 +0100
> >
> >     arm: mvebu: Use local interrupt only for the timer 0
> >     
> >     The commit 3a6f08a37 "arm: mvebu: Add support for local interrupt",
> >     managed the 28th first interrupts as local interrupt to match the
> >     hardware specification. Among these interrupts there are the Gigabits
> >     Ethernet ones used by the mvneta driver. Unfortunately the state of
> >     the percpu_irq API prevents the driver to use it.
> >     
> >     Indeed the interrupts have to be freed when the .stop() function is
> >     called. As the free_percpu_irq() function don't disable the interrupt
> >     line, we have to do it on each CPU before calling this. The function
> >     disable_percpu_irq() only disable the percpu on the current CPU and
> >     there is no function which allows to disable a percpu irq on a given
> >     CPU. Waiting for the extension of the percpu_irq API, this fix allows
> >     to use again the mvneta driver.
> >     
> >     Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> >     Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> >     Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> >     Signed-off-by: Jason Cooper <jason@lakedaemon.net>
> >
> >which is part of the backport-3.8 branch, so I'm a bit confused. Are
> >you sure you're using the latest backport-3.8 branch? If so, then I'll
> >do a test here to check this.
> >
> By the way, the patch IS applied to the source tree I'm compiling.
> I can see this in arch/arm/mach-mvebu/irq-armada-370-xp.c which is using 
> ARMADA_370_XP_TIMER0_PER_CPU_IRQ instead of ARMADA_370_XP_MAX_PER_CPU_IRQS

On my OpenBlocks AX3, I'm not experiencing this issue. However it runs
a slightly different kernel, it's 3.9 in which I have merged most of
the patches you find in backport-3.8.

One thing I don't know yet is if the bootloader installed on this box
is DT compatible or not, since I always append the DT to the end of my
kernels :

  cat arch/arm/boot/zImage arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dtb > zImage.dtb

Do you know if yours is compatible ? Did you append a DTB file to your
kernel ?

BTW, we have more or less similar hardware. My CPU is a 78260 and I have
4 ports, like you, so I suspect we should have the same IRQs. The original
kernel says the board is a "DB-78460-BP rev 2.0", just in case it can help,
but I suspect that your board is probably based on a similar design.

I'm seeing something odd though : when booting with the Marvell-patched
kernel provided with the AX3, here are my IRQs :

           CPU0       CPU1       
  8:          1          1   axp_irq  mv_eth
 10:          0          0   axp_irq  mv_eth
 31:          0          0   axp_irq  mv64xxx_i2c
 32:      14193          0   axp_irq  mv64xxx_i2c
 41:        182          0   axp_irq  serial
 45:          0          0   axp_irq  ehci_hcd:usb1
 46:          0          0   axp_irq  ehci_hcd:usb2
 47:          0          0   axp_irq  ehci_hcd:usb3
 55:        317          1   axp_irq  sata_mv
107:          0          0   axp_irq  axp-temp

But when booting on 3.9+patches from the Free-Electrons' team,
I have different IRQs :

           CPU0       CPU1       
 16:        348        321  armada_370_xp_irq  armada_370_xp_per_cpu_tick
 17:        198          0  armada_370_xp_irq  serial
 24:          1          0  armada_370_xp_irq  sata_mv
 25:          0          0  armada_370_xp_irq  mvneta
 27:          0          0  armada_370_xp_irq  mv64xxx_i2c
 28:         85          0  armada_370_xp_irq  mv64xxx_i2c
 30:          0          0  armada_370_xp_irq  ehci_hcd:usb1
 31:          0          0  armada_370_xp_irq  ehci_hcd:usb2
110:          2          0  armada_370_xp_irq  d0060900.xor
111:          2          0  armada_370_xp_irq  d0060900.xor
112:          2          0  armada_370_xp_irq  d00f0900.xor
113:          2          0  armada_370_xp_irq  d00f0900.xor

And precisely your failing IRQ is 25 (as I'm having here) and you said
you saw IRQ 8 somewhere. So probably something got remapped or not
completely remapped (I don't know).

Are you interested in trying my kernel+initrd ? They're easy to boot
over tftp.

Willy