Re: imx8mq-evk: Outbound network packets lost

[Chris Spencer <spencercw@gmail.com>]

On Fri, 4 Jan 2019 at 16:52, Chris Spencer <spencercw@gmail.com> wrote:
> On Fri, 4 Jan 2019 at 15:33, Chris Spencer <spencercw@gmail.com> wrote:
> > On Thu, 3 Jan 2019 at 12:53, Chris Spencer <spencercw@gmail.com> wrote:
> > > Hi,
> > >
> > > I'm trying to get the i.MX8MQ-EVK board up and running with the
> > > recently added kernel support (I'm currently working off
> > > arm-soc/for-next) and am having some trouble with the networking. If I
> > > issue a ping from my test machine with tcpdump running on both sides I
> > > see this:
> > >
> > > On dev machine:
> > > 10:57:52.655882 ARP, Request who-has 10.60.9.15 tell linux-dev, length 28
> > > 10:57:53.677046 ARP, Request who-has 10.60.9.15 tell linux-dev, length 28
> > > 10:57:54.701024 ARP, Request who-has 10.60.9.15 tell linux-dev, length 28
> > > 10:57:55.725050 ARP, Request who-has 10.60.9.15 tell linux-dev, length 28
> > >
> > > On imx8:
> > > 00:05:16.514604 ARP, Request who-has 10.60.9.15 tell 10.60.9.101, length 46
> > > 00:05:16.514651 ARP, Reply 10.60.9.15 is-at 00:04:9f:05:a5:a5 (oui
> > > Freescale), length 28
> > > 00:05:17.578275 ARP, Request who-has 10.60.9.15 tell 10.60.9.101, length 46
> > > 00:05:17.578298 ARP, Reply 10.60.9.15 is-at 00:04:9f:05:a5:a5 (oui
> > > Freescale), length 28
> > > 00:05:18.644896 ARP, Request who-has 10.60.9.15 tell 10.60.9.101, length 46
> > > 00:05:18.644932 ARP, Reply 10.60.9.15 is-at 00:04:9f:05:a5:a5 (oui
> > > Freescale), length 28
> > > 00:05:19.711548 ARP, Request who-has 10.60.9.15 tell 10.60.9.101, length 46
> > > 00:05:19.711561 ARP, Reply 10.60.9.15 is-at 00:04:9f:05:a5:a5 (oui
> > > Freescale), length 28
> > >
> > > So the imx8 can receive packets, but anything it sends back seems to
> > > be lost. The machines are connected directly with a standard patch
> > > cable. I have tried with two different test machines and network
> > > cables with the same results. Firewalls are disabled on both sides.
> > > tcpdump shows 0 packets dropped by kernel on both sides.
> > >
> > > Kernel output related to the Ethernet controller:
> > > [    0.550204] libphy: Fixed MDIO Bus: probed
> > > [    0.555023] fec 30be0000.ethernet: 30be0000.ethernet supply phy not
> > > found, using dummy regulator
> > > [    0.564204] fec 30be0000.ethernet: Linked as a consumer to regulator.0
> > > [    0.575518] libphy: fec_enet_mii_bus: probed
> > > ...
> > > [    6.479277] fec 30be0000.ethernet eth0: Link is Up - 1Gbps/Full -
> > > flow control rx/tx
> > > [    6.487386] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > >
> > > Potentially useful command output:
> > > # ip addr
> > > ...
> > > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq qlen 1000
> > >     link/ether 00:04:9f:05:a5:a5 brd ff:ff:ff:ff:ff:ff
> > >     inet 10.60.9.15/24 brd 10.60.9.255 scope global eth0
> > >        valid_lft forever preferred_lft forever
> > >     inet6 fe80::204:9fff:fe05:a5a5/64 scope link
> > >        valid_lft forever preferred_lft forever
> > > # ifconfig eth0
> > > eth0      Link encap:Ethernet  HWaddr 00:04:9F:05:A5:A5
> > >           inet addr:10.60.9.15  Bcast:10.60.9.255  Mask:255.255.255.0
> > >           inet6 addr: fe80::204:9fff:fe05:a5a5/64 Scope:Link
> > >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> > >           RX packets:32 errors:0 dropped:0 overruns:0 frame:0
> > >           TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
> > >           collisions:0 txqueuelen:1000
> > >           RX bytes:2332 (2.2 KiB)  TX bytes:1080 (1.0 KiB)
> > >
> > > Is anyone able to provide some insight on what might be happening here?
> > >
> > > Thanks,
> > > Chris
> >
> > I've been digging into this further and it seems to be potentially
> > related to U-Boot. If I use the version of U-Boot that is shipped with
> > the device on the eMMC and use that to manually boot into my SD card
> > then the Ethernet controller seems to work fine.
> >
> > It looks like perhaps the physical layer is not being initialised
> > properly. If I execute the following commands on the bundled U-Boot
> > everything looks normal:
> >
> > u-boot=> mdio list
> > FEC0:
> > 0 - AR8031/AR8033 <--> ethernet@30be0000
> > u-boot=> mii info
> > PHY 0x00: OUI = 0x1374, Model = 0x07, Rev = 0x04, 1000baseX, FDX
> > u-boot=> mii read 0 2
> > 004D
> > u-boot=> mii read 0 3
> > D074
> >
> > But on my own build of U-Boot (current master
> > 7436f5e54d35bcad53befec90e2e67288071f74e), it seems every request to
> > the physical layer is returning zero, resulting in it using the
> > generic PHY driver instead of the AR8031:
> >
> > u-boot=> mdio list
> > FEC0:
> > 0 - Generic PHY <--> ethernet@30be0000
> > u-boot=> mii info
> > PHY 0x00: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x01: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x02: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x03: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x04: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x05: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x06: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x07: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x08: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x09: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0A: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0B: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0C: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0D: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0E: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x0F: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x10: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x11: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x12: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x13: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x14: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x15: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x16: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x17: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x18: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x19: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1A: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1B: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1C: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1D: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1E: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > PHY 0x1F: OUI = 0x0000, Model = 0x00, Rev = 0x00,  10baseT, HDX
> > u-boot=> mii read 0 2
> > 0000
> > u-boot=> mii read 0 3
> > 0000
> >
> > I have added some tracing to drivers/net/fec_mxc.c:fec_mdio_read() and
> > the read seems to be proceeding normally (i.e., it gets an interrupt
> > indicating that the read has completed), but the value always comes
> > back as zero.
> >
> > # Clear MII interrupt in ENET_EIR
> > writel(800000, 30be0004)
> > # Re-check ENET_EIR; as expected, the interrupt is not set
> > readl(30be0004) => 0
> > # Issue a read request to ENET_MMFR for phy 0 reg 2
> > writel(600a0000, 30be0040)
> > # Check ENET_EIR; the MII interrupt has been set, indicating the
> > transfer has completed
> > readl(30be0004) => 800000
> > # Clear the interrupt
> > writel(800000, 30be0004)
> > # Read the data from ENET_MMFR; the value is zero. I would expect to
> > see 004d in the lower 16 bits here
> > readl(30be0040) => 600a0000
> >
> > There's a decent chance I'm missing something obvious, but I'm not seeing it..
> >
> > Thanks,
> > Chris
>
> The plot thickens... I updated my Trusted Firmware-A BL31 [1] and that
> seems to have fixed the problem on the Linux side. I'm not sure what
> revision I was on before, but it would have been around 2-3 weeks out
> of date. It still doesn't work at all in U-Boot, although I'm not
> going to lose too much sleep if I can't get that to work.
>
> Chris
>
> [1] https://github.com/ARM-software/arm-trusted-firmware
> 9a207532f8216bf83fed0891fed9ed0bc72ca450

Any chance somebody from NXP can comment on this?

It seems updating the TF-A didn't actually do anything, it's just the
problem is erratic.

I have added some trace logging to the Linux 5.0-rc1 fec_main.c and
have found that the very first MDIO read often incorrectly returns
zero:

[    0.584706] fec_enet_mdio_read(mii_id = 0, regnum = 2) => 0
[    0.590566] fec_enet_mdio_read(mii_id = 0, regnum = 3) => d074

This results in the Generic PHY driver being used instead of the
Atheros 8031 and things don't work properly. Occasionally, the correct
value will be returned any everything works fine:

[    0.584674] fec_enet_mdio_read(mii_id = 0, regnum = 2) => 4d
[    0.590623] fec_enet_mdio_read(mii_id = 0, regnum = 3) => d074

I can work around the issue with this patch:

diff --git a/drivers/net/ethernet/freescale/fec_main.c
b/drivers/net/ethernet/freescale/fec_main.c
index ae0f88bce9aa..802f51308b2d 100644
--- a/drivers/net/ethernet/freescale/fec_main.c
+++ b/drivers/net/ethernet/freescale/fec_main.c
@@ -1789,6 +1789,7 @@ static int fec_enet_mdio_read(struct mii_bus
*bus, int mii_id, int regnum)
         goto out;
     }

+    udelay(100);
     ret = FEC_MMFR_DATA(readl(fep->hwp + FEC_MII_DATA));

 out:

It seems that the interrupt is being triggered before the register is
actually ready to be read. I tried a 10us delay but the value returned
from the very first read was random and incorrect.

Thanks,
Chris

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel