Marvell 88E6320 connected to i.MX8MN

Marvell 88E6320 connected to i.MX8MN

[Fabio Estevam]

Hi,

I am trying to make a Marvell 88E6320 switch to work on an imx8mn-based board
running kernel 6.0.5.

Ethernet is functional in U-Boot with Tim's series:
https://lore.kernel.org/all/20221004164918.2251714-1-tharvey@gateworks.com/

However, in the kernel I am not able to retrieve an IP via DHCP:

mv88e6085 30be0000.ethernet-1:00: switch 0x1150 detected: Marvell
88E6320, revision 2
fec 30be0000.ethernet eth0: registered PHC device 0
mv88e6085 30be0000.ethernet-1:00: switch 0x1150 detected: Marvell
88E6320, revision 2
mv88e6085 30be0000.ethernet-1:00: configuring for fixed/rgmii-id link mode
mv88e6085 30be0000.ethernet-1:00: Link is Up - 1Gbps/Full - flow control off
mv88e6085 30be0000.ethernet-1:00 lan3 (uninitialized): PHY
[!soc@0!bus@30800000!ethernet@30be0000!mdio!switch@0!mdio:03] driver
[Generic PHY] (irq=POLL)
mv88e6085 30be0000.ethernet-1:00 lan4 (uninitialized): PHY
[!soc@0!bus@30800000!ethernet@30be0000!mdio!switch@0!mdio:04] driver
[Generic PHY] (irq=POLL)
device eth0 entered promiscuous mode
DSA: tree 0 setup
...

~# udhcpc -i lan4
udhcpc: started, v1.31.1
[   25.174846] mv88e6085 30be0000.ethernet-1:00 lan4: configuring for
phy/gmii link mode
udhcpc: sending discover
[   27.242123] mv88e6085 30be0000.ethernet-1:00 lan4: Link is Up -
100Mbps/Full - flow control rx/tx
[   27.251064] IPv6: ADDRCONF(NETDEV_CHANGE): lan4: link becomes ready
udhcpc: sending discover
udhcpc: sending discover
udhcpc: sending discover
...

This is my devicetree:
https://pastebin.com/raw/TagQJK2a

The only way that I can get IP via DHCP to work in the kernel is if
I access the network inside U-Boot first and launch the kernel afterward.

It looks like U-Boot is doing some configuration that the kernel is missing.

Does anyone have any suggestions, please?

Thanks,

Fabio Estevam

Re: Marvell 88E6320 connected to i.MX8MN

[Andrew Lunn]

> This is my devicetree:
> https://pastebin.com/raw/TagQJK2a

You have rgmii-id on both the FEC and the CPU port. So in theory you
might be getting double the needed delays? The mv88e6xxx driver will
apply these delays on the CPU port, but i don't know if the FEC does.

The other thing i've done wrong in the past with FEC is get the pinmux
wrong, so the reference clock was not muxed. Check how the reference
clock should used, is it from the switch to the FEC, or the other way
around. If the FEC is providing it, is it ticking?

	Andrew

Re: Marvell 88E6320 connected to i.MX8MN

[Fabio Estevam]

Hi Andrew and Vladimir,

On Thu, Oct 27, 2022 at 9:33 AM Andrew Lunn <andrew@lunn.ch> wrote:

> You have rgmii-id on both the FEC and the CPU port. So in theory you
> might be getting double the needed delays? The mv88e6xxx driver will
> apply these delays on the CPU port, but i don't know if the FEC does.
>
> The other thing i've done wrong in the past with FEC is get the pinmux
> wrong, so the reference clock was not muxed. Check how the reference
> clock should used, is it from the switch to the FEC, or the other way
> around. If the FEC is providing it, is it ticking?

Thanks for your suggestions. It is working now.

mv88e6320_ops misses .port_set_rgmii_delay.

I will submit a fix upstream soon.

Thanks a lot!

Fabio Estevam

Re: Marvell 88E6320 connected to i.MX8MN

[Vladimir Oltean]

Hi Fabio,

On Wed, Oct 26, 2022 at 11:16:45PM -0300, Fabio Estevam wrote:
> Hi,
> 
> I am trying to make a Marvell 88E6320 switch to work on an imx8mn-based board
> running kernel 6.0.5.
> 
> Ethernet is functional in U-Boot with Tim's series:
> https://lore.kernel.org/all/20221004164918.2251714-1-tharvey@gateworks.com/
> 
> However, in the kernel I am not able to retrieve an IP via DHCP:
> 
> mv88e6085 30be0000.ethernet-1:00: switch 0x1150 detected: Marvell 88E6320, revision 2
> fec 30be0000.ethernet eth0: registered PHC device 0
> mv88e6085 30be0000.ethernet-1:00: switch 0x1150 detected: Marvell 88E6320, revision 2
> mv88e6085 30be0000.ethernet-1:00: configuring for fixed/rgmii-id link mode
> mv88e6085 30be0000.ethernet-1:00: Link is Up - 1Gbps/Full - flow control off
> mv88e6085 30be0000.ethernet-1:00 lan3 (uninitialized): PHY
> [!soc@0!bus@30800000!ethernet@30be0000!mdio!switch@0!mdio:03] driver [Generic PHY] (irq=POLL)
> mv88e6085 30be0000.ethernet-1:00 lan4 (uninitialized): PHY
> [!soc@0!bus@30800000!ethernet@30be0000!mdio!switch@0!mdio:04] driver [Generic PHY] (irq=POLL)

Looks like you are missing the Marvell PHY driver; the generic PHY
driver gets used. Can you enable CONFIG_MARVELL_PHY?

> device eth0 entered promiscuous mode
> DSA: tree 0 setup
> ...
> 
> ~# udhcpc -i lan4
> udhcpc: started, v1.31.1
> [   25.174846] mv88e6085 30be0000.ethernet-1:00 lan4: configuring for
> phy/gmii link mode
> udhcpc: sending discover
> [   27.242123] mv88e6085 30be0000.ethernet-1:00 lan4: Link is Up -
> 100Mbps/Full - flow control rx/tx
> [   27.251064] IPv6: ADDRCONF(NETDEV_CHANGE): lan4: link becomes ready
> udhcpc: sending discover
> udhcpc: sending discover
> udhcpc: sending discover
> ...
> 
> This is my devicetree:
> https://pastebin.com/raw/TagQJK2a
> 
> The only way that I can get IP via DHCP to work in the kernel is if
> I access the network inside U-Boot first and launch the kernel afterward.
> 
> It looks like U-Boot is doing some configuration that the kernel is missing.

Yeah, sounds like the Marvell PHY driver could be what's the difference.

> Does anyone have any suggestions, please?

If that doesn't work, the next step is to isolate things. Connect a
cable to the other switch port, create a bridge, and forward packets
between one station and the other. This doesn't involve the CPU port, so
you'll learn if the internal PHYs are the problem or the CPU port is.
Next step would be to collect ethtool -S lan0, ethtool -S eth0, and post
those.

Re: Marvell 88E6320 connected to i.MX8MN

[Fabio Estevam]

 Hi Vladimir,

On Thu, Oct 27, 2022 at 5:41 PM Vladimir Oltean <vladimir.oltean@nxp.com> wrote:

> Looks like you are missing the Marvell PHY driver; the generic PHY
> driver gets used. Can you enable CONFIG_MARVELL_PHY?

CONFIG_MARVELL_PHY is already selected.

However, there is no support for 88E6320 in the Marvell PHY driver.

Thanks



> > device eth0 entered promiscuous mode
> > DSA: tree 0 setup
> > ...
> >
> > ~# udhcpc -i lan4
> > udhcpc: started, v1.31.1
> > [   25.174846] mv88e6085 30be0000.ethernet-1:00 lan4: configuring for
> > phy/gmii link mode
> > udhcpc: sending discover
> > [   27.242123] mv88e6085 30be0000.ethernet-1:00 lan4: Link is Up -
> > 100Mbps/Full - flow control rx/tx
> > [   27.251064] IPv6: ADDRCONF(NETDEV_CHANGE): lan4: link becomes ready
> > udhcpc: sending discover
> > udhcpc: sending discover
> > udhcpc: sending discover
> > ...
> >
> > This is my devicetree:
> > https://pastebin.com/raw/TagQJK2a
> >
> > The only way that I can get IP via DHCP to work in the kernel is if
> > I access the network inside U-Boot first and launch the kernel afterward.
> >
> > It looks like U-Boot is doing some configuration that the kernel is missing.
>
> Yeah, sounds like the Marvell PHY driver could be what's the difference.
>
> > Does anyone have any suggestions, please?
>
> If that doesn't work, the next step is to isolate things. Connect a
> cable to the other switch port, create a bridge, and forward packets
> between one station and the other. This doesn't involve the CPU port, so
> you'll learn if the internal PHYs are the problem or the CPU port is.
> Next step would be to collect ethtool -S lan0, ethtool -S eth0, and post
> those.

Re: Marvell 88E6320 connected to i.MX8MN

[Andrew Lunn]

On Thu, Oct 27, 2022 at 11:24:40PM -0300, Fabio Estevam wrote:
>  Hi Vladimir,
> 
> On Thu, Oct 27, 2022 at 5:41 PM Vladimir Oltean <vladimir.oltean@nxp.com> wrote:
> 
> > Looks like you are missing the Marvell PHY driver; the generic PHY
> > driver gets used. Can you enable CONFIG_MARVELL_PHY?
> 
> CONFIG_MARVELL_PHY is already selected.
> 
> However, there is no support for 88E6320 in the Marvell PHY driver.

In theory, there should not be any support needed. The PHY registers 2
and 3 contain the PHY ID. If Marvell have reused an off the shelf PHY,
it should have an ID which the drivers knows.

However, a few Marvell Switches are broken by design, and registers 2
and 3 are empty. It is documented you should not use them to identify
the PHY. As usual for a datasheet, it gives no explanation why they
decided to break 802.3 Clause 22, but they have. So there is a
workaround for a couple of switches. See family_prod_id_table in
drivers/net/dsa/mv88e6xxx/chip.c

Under /sys/class/mdio_bus/ you should find all the PHYs and there is a
file phy_id. If it is 0, it has the above problem. Otherwise, you
might need to add the ID to drivers/net/phy/marvell.c.

You should be able to use mii-tool to dump the common PHY registers in
both your working and not working state and see if there is any
difference.

	Andrew