[U-Boot] Ethernet not found on Arria 5.

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Dear U-Boot support,

I'm migrating to new U-Boot version from 2013 and now have Ethernet not working both in U-Boot and in Linux (after booting).

I have custom board with Altera Arria 5 SocFpga onboard.
U-Boot version: 2016.03-rc1

In logs I can see:

 Net:   No ethernet found.

With more verbose:

 designware_eth_probe, iobase=ff702000, priv=1eb286a0
 ethernet at ff702000 PHY: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 not found
 designware_eth_probe, ret=-19
 No ethernet found.

Ethernet connection inside my board is following:
   [CPU] ----- [EMAC1] ----- [ FPGA ] ---- [ PHY(KSZ8081MNXIA) ]

I already tried to configure FPGA from Linux environment but it didn't solved the problem.
My U-Boot configuration was cloned from socfpga dev kit board with some modifications.
But Ethernet configuration I didn't touched yet.

So far I tried to debug it with no success. Also I played with env variables (ethact, ethaddr) and CONFIG_PHY_ADDR with no success as well.
Something tells me that I have incorrect EMAC configuration but I don't know how to tackle it.

Please help me identify the problem or at least give me some hints where to look to solve my issue.

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/04/2016 10:20 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Dear U-Boot support,
> 
> I'm migrating to new U-Boot version from 2013 and now have Ethernet
> not working both in U-Boot and in Linux (after booting).
> 
> I have custom board with Altera Arria 5 SocFpga onboard. U-Boot
> version: 2016.03-rc1
> 
> In logs I can see:
> 
> Net:   No ethernet found.
> 
> With more verbose:
> 
> designware_eth_probe, iobase=ff702000, priv=1eb286a0 
> ethernet at ff702000 PHY: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
> 19 20 21 22 23 24 25 26 27 28 29 30 31 not found 
> designware_eth_probe, ret=-19 No ethernet found.
> 
> Ethernet connection inside my board is following: [CPU] ----- [EMAC1]
> ----- [ FPGA ] ---- [ PHY(KSZ8081MNXIA) ]
> 
> I already tried to configure FPGA from Linux environment but it
> didn't solved the problem. My U-Boot configuration was cloned from
> socfpga dev kit board with some modifications. But Ethernet
> configuration I didn't touched yet.
> 
> So far I tried to debug it with no success. Also I played with env
> variables (ethact, ethaddr) and CONFIG_PHY_ADDR with no success as
> well. Something tells me that I have incorrect EMAC configuration but
> I don't know how to tackle it.
> 
> Please help me identify the problem or at least give me some hints
> where to look to solve my issue.

It seems like your PHY is not recongnised. Could there be some reset
line which is left asserted ?

Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi,

> It seems like your PHY is not recongnised. Could there be some reset
> line which is left asserted ?

I'm afraid I don't know how to check that.

But I have previous version of U-Boot (2013) where Ethernet is working.
Maybe I can check it there?
I already tried to go that path, but code is quite different there.

With U-Boot 2013 it is enough to config fpga and set env variables to have ping in both directions.

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/04/2016 01:53 PM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi,

Hi,

>> It seems like your PHY is not recongnised. Could there be some
>> reset line which is left asserted ?
> 
> I'm afraid I don't know how to check that.
> 
> But I have previous version of U-Boot (2013) where Ethernet is
> working. Maybe I can check it there? I already tried to go that path,
> but code is quite different there.
> 
> With U-Boot 2013 it is enough to config fpga and set env variables to
> have ping in both directions.

What do you mean by this ? Is your ethernet controller synthesised in
the FPGA ? The arriaV socdk u-boot uses the top-side ethernet port,
which is connected to the ethernet controller in the HPS.

-- 
Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi Marek,

> What do you mean by this ? Is your ethernet controller synthesised in
> the FPGA ? The arriaV socdk u-boot uses the top-side ethernet port,
> which is connected to the ethernet controller in the HPS.

I managed to get it working.
Right after configuring fpga from Linux I made a soft reset and PHY chip was successfully found.

Net: eth0: ethernet at ff702000

However there is still no ping in U-Boot.
After power reset I did:

># bridge disable
># fpga load 0 <addr> <size>
># bridge enable

># md 0xff706000 1
ff706000: 00000074          <-- this means fpga is in user mode

># setenv ethaddr ...
># setenv ipaddr ...
># setenv netmask ...
># setenv gatewayip ...

=> ping 192.168.1.126
Speed: 100, full duplex
Using ethernet at ff702000 device
ping failed; host 192.168.1.126 is not alive

With similar commands on previous U-Boot version I had ping.

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi,

> However there is still no ping in U-Boot.
> After power reset I did:
>
> ># bridge disable
> ># fpga load 0 <addr> <size>
> ># bridge enable
>
> ># md 0xff706000 1
> ff706000: 00000074          <-- this means fpga is in user mode
>
> ># setenv ethaddr ...
> ># setenv ipaddr ...
> ># setenv netmask ...
> ># setenv gatewayip ...
>
> => ping 192.168.1.126
> Speed: 100, full duplex
> Using ethernet at ff702000 device
> ping failed; host 192.168.1.126 is not alive
>
> With similar commands on previous U-Boot version I had ping.

Also using wireshark I found that board sends correct ARP packets to PC.
PC in it's turn send valid ARP response to the board.
But for some reason ARP reply is not handled by the board.
And board doesn't send ICMP packets to the PC.

I already checked ip addresses, they should be fine.
Also the same IP works fine for previous U-Boot version (2013).

What could be the reason for that?

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/09/2016 10:22 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi,
> 
>> However there is still no ping in U-Boot.
>> After power reset I did:
>>
>>> # bridge disable
>>> # fpga load 0 <addr> <size>
>>> # bridge enable
>>
>>> # md 0xff706000 1
>> ff706000: 00000074          <-- this means fpga is in user mode
>>
>>> # setenv ethaddr ...
>>> # setenv ipaddr ...
>>> # setenv netmask ...
>>> # setenv gatewayip ...
>>
>> => ping 192.168.1.126
>> Speed: 100, full duplex
>> Using ethernet at ff702000 device
>> ping failed; host 192.168.1.126 is not alive
>>
>> With similar commands on previous U-Boot version I had ping.
> 
> Also using wireshark I found that board sends correct ARP packets to PC.
> PC in it's turn send valid ARP response to the board.
> But for some reason ARP reply is not handled by the board.
> And board doesn't send ICMP packets to the PC.
> 
> I already checked ip addresses, they should be fine.
> Also the same IP works fine for previous U-Boot version (2013).
> 
> What could be the reason for that?

Perform usual test, disable cache (dcache off) .

And please CC the usual lineup, Chin and Dinh ;-)

-- 
Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi Marek,

> Perform usual test, disable cache (dcache off) .

I tried and result is still the same.

UPD: I did a little trick:
1. I started ping from the board side. That made the board listen to incoming packets (calling in infinite loop eth_rx() ).
2. Started ping from PC side.
3. In this case board receive ICMP packets from PC:

packet received
Receive from protocol 0x800
Got IP
len=60, v=45
Got ICMP ECHO REQUEST, return 74 bytes 

So, ICMP packets are handled by the board, but ARP packets not.

In my understanding it tells me that at least interface on the board side is alive.

I'm now doing some low-level debugging, however I think this is not the best idea. :)
Now dw_eth_recv (designware.c) always returns 0 length of the packet.

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Dinh Nguyen]

On 03/09/2016 08:00 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi Marek,
> 
>> Perform usual test, disable cache (dcache off) .
> 
> I tried and result is still the same.
> 
> UPD: I did a little trick:
> 1. I started ping from the board side. That made the board listen to incoming packets (calling in infinite loop eth_rx() ).
> 2. Started ping from PC side.
> 3. In this case board receive ICMP packets from PC:
> 
> packet received
> Receive from protocol 0x800
> Got IP
> len=60, v=45
> Got ICMP ECHO REQUEST, return 74 bytes 
> 
> So, ICMP packets are handled by the board, but ARP packets not.
> 
> In my understanding it tells me that at least interface on the board side is alive.
> 
> I'm now doing some low-level debugging, however I think this is not the best idea. :)
> Now dw_eth_recv (designware.c) always returns 0 length of the packet.
> 

I was able to tftp an kernel image using mainline U-Boot on my Arria5
board today. However, I wasn't able to dhcp, but I'm not sure if that's
the board or my network.


U-Boot 2016.03-rc3-00008-g08b2472 (Mar 09 2016 - 13:37:27 -0600)

CPU:   Altera SoCFPGA Platform
FPGA:  Altera Arria V, D5, version 0x0
BOOT:  SD/MMC External Transceiver (1.8V)
       Watchdog enabled
I2C:   ready
DRAM:  1 GiB
MMC:   dwmmc0 at ff704000: 0
In:    serial
Out:   serial
Err:   serial
Model: Altera SOCFPGA Arria V SoC Development Kit
Net:   eth0: ethernet at ff702000
Hit any key to stop autoboot:  0
=> tftp ${loadaddr} zImage
Speed: 1000, full duplex
Using ethernet at ff702000 device
TFTP from server 137.57.160.210; our IP address is 137.57.160.216
Filename 'zImage'.
Load address: 0x8000
Loading:
         #################################
         552.7 KiB/s
done
Bytes transferred = 3491984 (354890 hex)
=>

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/09/2016 10:40 PM, Dinh Nguyen wrote:
> On 03/09/2016 08:00 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
>> Hi Marek,
>>
>>> Perform usual test, disable cache (dcache off) .
>>
>> I tried and result is still the same.
>>
>> UPD: I did a little trick:
>> 1. I started ping from the board side. That made the board listen to incoming packets (calling in infinite loop eth_rx() ).
>> 2. Started ping from PC side.
>> 3. In this case board receive ICMP packets from PC:
>>
>> packet received
>> Receive from protocol 0x800
>> Got IP
>> len=60, v=45
>> Got ICMP ECHO REQUEST, return 74 bytes 
>>
>> So, ICMP packets are handled by the board, but ARP packets not.
>>
>> In my understanding it tells me that at least interface on the board side is alive.
>>
>> I'm now doing some low-level debugging, however I think this is not the best idea. :)
>> Now dw_eth_recv (designware.c) always returns 0 length of the packet.
>>
> 
> I was able to tftp an kernel image using mainline U-Boot on my Arria5
> board today. However, I wasn't able to dhcp, but I'm not sure if that's
> the board or my network.
> 
> 
> U-Boot 2016.03-rc3-00008-g08b2472 (Mar 09 2016 - 13:37:27 -0600)
> 
> CPU:   Altera SoCFPGA Platform
> FPGA:  Altera Arria V, D5, version 0x0
> BOOT:  SD/MMC External Transceiver (1.8V)
>        Watchdog enabled
> I2C:   ready
> DRAM:  1 GiB
> MMC:   dwmmc0 at ff704000: 0
> In:    serial
> Out:   serial
> Err:   serial
> Model: Altera SOCFPGA Arria V SoC Development Kit
> Net:   eth0: ethernet at ff702000
> Hit any key to stop autoboot:  0
> => tftp ${loadaddr} zImage
> Speed: 1000, full duplex
> Using ethernet at ff702000 device
> TFTP from server 137.57.160.210; our IP address is 137.57.160.216
> Filename 'zImage'.
> Load address: 0x8000
> Loading:
>          #################################
>          552.7 KiB/s
> done
> Bytes transferred = 3491984 (354890 hex)
> =>

Thanks for the test!

The speed looks weird, it should be in the 2-3MiB range.

Are you booting using mainline U-Boot SPL ? :-)

-- 
Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi Marek, Dinh,

> Are you booting using mainline U-Boot SPL ? :-)

No, we use SPL from U-Boot 2013.
I can quess what you will say now, but it somehow worked before (combination SPL + U-Boot from 2013).

Is there a way to capture fpga dumps?
I can then compare them to working case. 
I can assume that there will be lots of differences and I would have to check them manually, but it's better than nothing I think.

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/10/2016 09:58 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi Marek, Dinh,

Hi,

>> Are you booting using mainline U-Boot SPL ? :-)
> 
> No, we use SPL from U-Boot 2013.
> I can quess what you will say now, but it somehow worked before (combination SPL + U-Boot from 2013).

I will say exactly that, you are asking for support in mainline mailing
list, while you don't even use mainline. What sort of reaction do you
even expect ?

> Is there a way to capture fpga dumps?
> I can then compare them to working case. 
> I can assume that there will be lots of differences and I would have to check them manually, but it's better than nothing I think.

Why are you constantly hung on this FPGA part ? The ethernet is not
routed through the FPGA, it is connected directly to the HPS. Thus,
you don't have to care about the FPGA at all, you only care about the
configuration of the HPS.

> Best regards,
> Denis Bakhvalov
> 


-- 
Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi,

> Why are you constantly hung on this FPGA part ? The ethernet is not
> routed through the FPGA, it is connected directly to the HPS. Thus,
> you don't have to care about the FPGA at all, you only care about the
> configuration of the HPS.

Please excuse me for my small experience in this topic.
I'm just trying to find the way how to solve this issue. Maybe then take HPS dumps and compare them?

What I have for now:

OK case (U-Boot 2013):
ARP packets are sent from board to PC and back.
ICMP packets are sent from board to PC and back.
Ping is successful.

NOK case(U-Boot 2016):
ARP packets are sent from board to PC.
PC sends ARP reply but it is not recognized by the board.
Ping fails.
However when the board is "waiting" for ARP reply from the PC it can process ICMP packets from the PC and send reply to them.

I started thinking about: "What is that special in those ARP packets?"

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi,

I solved the Ethernet problem on our board.

The problem was in the register below:

Link: http://wl.altera.com/literature/hb/arria-v/hps.html#topic/sfo1410067853518.html
Registers used by the EMACs. All fields are reset by a cold or warm reset.
Module Instance	Base Address	Register Address
sysmgr	0xFFD08000	0xFFD08060

I found that difference while comparing the dumps between OK and NOK cases.

In new U-Boot (2016) the values of 
ctrl :: physel_0
ctrl :: physel_1
were always set to 
0x1	Select RGMII PHY interface

I changed this value to 
0x0	Select GMII/MII PHY interface

And it start working:

Using ethernet at ff702000 device
TFTP from server 192.168.1.126; our IP address is 192.168.1.130
Filename 'os.bin'.
Load address: 0x1b000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #########################################################
         856.4 KiB/s
done
Bytes transferred = 5949368 (5ac7b8 hex)

I knew about that requirement about MII interface between EMAC and PHY but it took me so long to find it.

But still I have this sort of question:
Why those two registers are always assigned to RGMII PHY interface (and default value is 0x2 	Select RMII PHY interface)?
In current code there is no way to change this value.

I changed it like this:

diff --git a/arch/arm/mach-socfpga/misc.c b/arch/arm/mach-socfpga/misc.c
old mode 100644
new mode 100755
index 9b43b92..295ed5a
--- a/arch/arm/mach-socfpga/misc.c
+++ b/arch/arm/mach-socfpga/misc.c
@@ -23,6 +23,8 @@

@@ -97,14 +99,19 @@ static void dwmac_deassert_reset(const unsigned int of_reset_id)
                     SYSMGR_EMACGRP_CTRL_PHYSEL_MASK << physhift);

        /* configure to PHY interface select choosed */
+#ifdef CONFIG_WORKAROUND
+       setbits_le32(&sysmgr_regs->emacgrp_ctrl,
+                    SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_GMII_MII << physhift);
+#else
        setbits_le32(&sysmgr_regs->emacgrp_ctrl,
                    SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_RGMII << physhift);
+#endif

        /* Release the EMAC controller from reset */
        socfpga_per_reset(reset, 0);
 }

Please evaluate my correction.
Maybe we can assign ctrl :: physel_0 and ctrl :: physel_1 based on some switch in config?

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/21/2016 09:16 AM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi,

Hi!

> I solved the Ethernet problem on our board.
> 
> The problem was in the register below:
> 
> Link: http://wl.altera.com/literature/hb/arria-v/hps.html#topic/sfo1410067853518.html
> Registers used by the EMACs. All fields are reset by a cold or warm reset.
> Module Instance	Base Address	Register Address
> sysmgr	0xFFD08000	0xFFD08060

Thanks for looking into it, try if the attached patch works for you.
Make sure you have correct phy-mode = "gmii"; DT node specified for
the GMAC you use, Arria V SoCDK surely uses phy-mode = "rgmii";

> I found that difference while comparing the dumps between OK and NOK cases.
> 
> In new U-Boot (2016) the values of 
> ctrl :: physel_0
> ctrl :: physel_1
> were always set to 
> 0x1	Select RGMII PHY interface
> 
> I changed this value to 
> 0x0	Select GMII/MII PHY interface

[...]

> But still I have this sort of question:
> Why those two registers are always assigned to RGMII PHY interface (and default value is 0x2 	Select RMII PHY interface)?
> In current code there is no way to change this value.

Most likely because noone ever had a board with PHY connected over
anything else but RGMII, so this went unnoticed.

> I changed it like this:
> 
> diff --git a/arch/arm/mach-socfpga/misc.c b/arch/arm/mach-socfpga/misc.c
> old mode 100644
> new mode 100755
> index 9b43b92..295ed5a
> --- a/arch/arm/mach-socfpga/misc.c
> +++ b/arch/arm/mach-socfpga/misc.c
> @@ -23,6 +23,8 @@
> 
> @@ -97,14 +99,19 @@ static void dwmac_deassert_reset(const unsigned int of_reset_id)
>                      SYSMGR_EMACGRP_CTRL_PHYSEL_MASK << physhift);
> 
>         /* configure to PHY interface select choosed */
> +#ifdef CONFIG_WORKAROUND
> +       setbits_le32(&sysmgr_regs->emacgrp_ctrl,
> +                    SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_GMII_MII << physhift);
> +#else
>         setbits_le32(&sysmgr_regs->emacgrp_ctrl,
>                     SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_RGMII << physhift);
> +#endif
> 
>         /* Release the EMAC controller from reset */
>         socfpga_per_reset(reset, 0);
>  }
> 
> Please evaluate my correction.
> Maybe we can assign ctrl :: physel_0 and ctrl :: physel_1 based on some switch in config?

We should parse the OF node phy-mode, which describes which mode your
PHY uses. If your DT is written correctly, then with the attached patch,
any PHY mode should work.

> Best regards,
> Denis Bakhvalov
> 


-- 
Best regards,
Marek Vasut
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-arm-socfpga-Handle-phy-mode-OF-property-for-GMACs.patch
Type: text/x-patch
Size: 2868 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160321/930e3d3f/attachment.bin>

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hi,

Working perfectly!
Thanks for helping me remove this nasty workaround.

> We should parse the OF node phy-mode, which describes which mode your
> PHY uses. 

Right!

Best regards,
Denis Bakhvalov

[U-Boot] Ethernet not found on Arria 5.

[Marek Vasut]

On 03/21/2016 01:34 PM, Bakhvalov, Denis (Nokia - PL/Wroclaw) wrote:
> Hi,
> 
> Working perfectly!
> Thanks for helping me remove this nasty workaround.

Thanks!

I sent the patch out and will pick it after standard review.
You can add a Tested-by tag on it if you want.

>> We should parse the OF node phy-mode, which describes which mode your
>> PHY uses. 
> 
> Right!
> 
> Best regards,
> Denis Bakhvalov
> 


-- 
Best regards,
Marek Vasut

[U-Boot] Ethernet not found on Arria 5.

[Dinh Nguyen]

On 03/09/2016 06:25 PM, Marek Vasut wrote:
> 
> Thanks for the test!
> 
> The speed looks weird, it should be in the 2-3MiB range.
> 
> Are you booting using mainline U-Boot SPL ? :-)
> 

Yes, I'm using mainline SPL. This Arria5 board is really old so I can't
really say that I have good working HW. But ethernet does seem to be
working for me. I look through all of the obvious areas(IOCSR, skew phy
settings, system manager for setting the PHY modes) and I can't see
anything wrong with the A5 ethernet support.

Dinh

[U-Boot] Ethernet not found on Arria 5.

[Bakhvalov, Denis]

Hello Dear U-Boot support,

Please comment on this also.

I have custom board with Altera Arria 5 SocFpga onboard.
U-Boot version: 2016.03-rc1

I had probems with configuring fpga from u-boot:

U-Boot > bridge disable
U-Boot > run config_fpga
FPGA: Could not configure
Command failed, result=-2

So, fpga did not reached configuration state in certain timeout (FPGAMGRREGS_MODE_CFGPHASE).
My workaround was based on U-Boot 2013 version were I had no such problem.

I fixed it like this:

diff --git a/drivers/fpga/socfpga.c b/drivers/fpga/socfpga.c
index 431e159..423ee23 100644
--- a/drivers/fpga/socfpga.c
+++ b/drivers/fpga/socfpga.c
@@ -269,7 +269,11 @@ int socfpga_load(Altera_desc *desc, const void *rbf_data, size_t rbf_size)
        /* Prior programming the FPGA, all bridges need to be shut off */

        /* Disable all signals from hps peripheral controller to fpga */
+#ifdef CONFIG_WORKAROUND
+       writel(0, &sysmgr_regs->fpgaintfgrp_module);
+#else
        writel(0, &sysmgr_regs->fpgaintfgrp_gbl);
+#endif

Please evaluate my workaround.
Maybe I had to make some additional step before configuring fpga?

Best regards,
Denis Bakhvalov