DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

[Benjamin Beckmeyer]

Hi all,

I'm currently working on a custom board with the imx6ull processor and the 6390 
switching chip. This is our hardware setup. 

------------     ---------         ---------    MAC     ------------
|   i.MX   | MAC |  PHY  |   PHY   |  PHY  |------------|  88E6390 |
|   6ULL   |-----|KSZ8081|---------|LAN8742|	MDIO	|P0        |
|          |     |ID 0x1 |         | ID0x0 |------------|          |
|          |     ---------         ---------            |          |
|          |	     |                                  |MULTI CHIP|
|          |	     |MDIO                              |ADDR MODE |
|          |	     |                                  |          |
|          |--------------------------------------------|   PHY ID |
|          |                    MDIO                    |     0x2  |
------------						------------

The switch is working properly so far, but I don't get any connection to linux.
Here is my device tree, I already did it with fixed-links, but from my 
understanding there can't be a fixed link because there are two dedicated phys.

---- snip
&fec1 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_enet1 &pinctrl_gpios>;
	reset-gpios = <&gpio5 9 GPIO_ACTIVE_LOW>;
	phy-supply = <&reg_3v3>;
	phy-mode = "rmii";
	phy-handle = <&fecphy1>;
	status = "okay";

	mdio0: mdio {
		#address-cells = <1>;
		#size-cells = <0>;

		fecphy1: fecphy@1 {
			reg = <0x1>;
		};
	};
};

&mdio0 {
	switch0: switch0@2 {
		compatible = "marvell,mv88e6190";
		reg = <2>;
		pinctrl-0 = <&pinctrl_gpios>;
		reset-gpios = <&gpio4 16 GPIO_ACTIVE_LOW>;

		dsa,member = <0 0>;

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				label = "cpu";
				ethernet = <&fec1>;
				phy-mode = "rmii";
				phy-handle = <&switch0phy0>;
			};

			port@1 {
				reg = <1>;
				label = "lan1";
				phy-handle = <&switch0phy1>;
			};

			port@2 {
				reg = <2>;
				label = "lan2";
				phy-handle = <&switch0phy2>;
			};

			port@3 {
				reg = <3>;
				label = "lan3";
				phy-handle = <&switch0phy3>;
			};

			port@4 {
				reg = <4>;
				label = "lan4";
				phy-handle = <&switch0phy4>;
			};

			port@5 {
				reg = <5>;
				label = "lan5";
				phy-handle = <&switch0phy5>;
			};

			port@6 {
				reg = <6>;
				label = "lan6";
				phy-handle = <&switch0phy6>;
			};

			port@7 {
				reg = <7>;
				label = "lan7";
				phy-handle = <&switch0phy7>;
			};

			port@8 {
				reg = <8>;
				label = "lan8";
				phy-handle = <&switch0phy8>;
			};
		};

		mdio {
			#address-cells = <1>;
			#size-cells = <0>;

			switch0phy1: switch0phy1@1 {
				reg = <0x1>;
			};
			switch0phy2: switch0phy2@2 {
				reg = <0x2>;
			};
			switch0phy3: switch0phy3@3 {
				reg = <0x3>;
			};
			switch0phy4: switch0phy4@4 {
				reg = <0x4>;
			};
			switch0phy5: switch0phy5@5 {
				reg = <0x5>;
			};
			switch0phy6: switch0phy6@6 {
				reg = <0x6>;
			};
			switch0phy7: switch0phy7@7 {
				reg = <0x7>;
			};
			switch0phy8: switch0phy8@8 {
				reg = <0x8>;
			};
		};

		mdio1 {
			compatible = "marvell,mv88e6xxx-mdio-external";
			#address-cells = <1>;
			#size-cells = <0>;

			switch0phy0: switch0phy0@0 {
				reg = <0x0>;
			};
		};
	};
};
---- snip

Do I miss something here? I know it is an odd layout, but we have a custom CPU 
capture with the KSZ PHY on it, so we thought it might work this way. 

The switch P0 is configured in PHY Mode because otherwise we don't have the 
RMII 50 MHz clock.

When we configure the devicetree without DSA, the switch is in forwarding mode.
When I ping the device from a directly connected PC, I see an ARP Request and 
the ARP Reply with tcpdump. We followed the signals back to the switch with an 
oscilloscope, so we assume that the connection is ok. 

But in boot up the following message is printed out:
[    1.551382] libphy: /soc/aips-bus@2100000/ethernet@2188000/mdio/switch0@2/mdio1: probed
[    1.562317] mdio_bus !soc!aips-bus@2100000!ethernet@2188000!mdio!switch0@2!mdio1: MDIO device at address 0 is missing.

So Linux or the DSA is not recognizing the LAN8742 PHY on PHY ID 0x0. Before 
we setup the P0 port to RMII PHY MODE (before it was set to RGMII, what is wrong)
it was recognized. But when I read the phy_id from /ssys/bus/mdio_bus it was 
only correct to the half. It should be 0x0007C131 but I got 0xffffC131 or 
sometimes 0x0007ffff.

When DSA is loaded our MII tool is not recognizing the switch on ID 2 (or at 
least with some odd values). The KSZ is recognized correctly with its Phy ID.
Does the DSA driver do something here?

I know it's a lot of information but maybe somebody can help me to get the DSA
working properly..

Thanks in advance.

Cheers,
Benjamin Beckmeyer

Re: DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

[Andrew Lunn]

On Wed, May 22, 2019 at 10:33:29AM +0200, Benjamin Beckmeyer wrote:
> Hi all,
> 
> I'm currently working on a custom board with the imx6ull processor and the 6390 
> switching chip. This is our hardware setup. 
> 
> ------------     ---------         ---------    MAC     ------------
> |   i.MX   | MAC |  PHY  |   PHY   |  PHY  |------------|  88E6390 |
> |   6ULL   |-----|KSZ8081|---------|LAN8742|	MDIO	|P0        |
> |          |     |ID 0x1 |         | ID0x0 |------------|          |
> |          |     ---------         ---------            |          |
> |          |	     |                                  |MULTI CHIP|
> |          |	     |MDIO                              |ADDR MODE |
> |          |	     |                                  |          |
> |          |--------------------------------------------|   PHY ID |
> |          |                    MDIO                    |     0x2  |
> ------------						------------

Hi Benjamin

KSZ8081 is a 10/100 PHY, i think.
LAN8742 is also a 10/100 PHY.

However, DSA will configure the CPU port MAC to its maximum speed. So
port 0 will be doing 1G. I don't know if specifying phy-mode = "rmii"
is enough. You should take a look at the port status and configuration
registers, see if the MAC is being forced to 1G, or 100M.

You could add a fixed-phy to port 0 with speed 100. That will at least
get the MAC part configured correctly.

Is the LAN8742 strapped so that on power on it will auto-neg? I've
seen a few board with this back-to-back PHY setup, and they just rely
on the PHYs doing the right thing on power up, no software involved.

   Andrew

Re: DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

[Benjamin Beckmeyer]

On 22.05.19 18:32, Andrew Lunn wrote:
> On Wed, May 22, 2019 at 10:33:29AM +0200, Benjamin Beckmeyer wrote:
>> Hi all,
>>
>> I'm currently working on a custom board with the imx6ull processor and the 6390 
>> switching chip. This is our hardware setup. 
>>
>> ------------     ---------         ---------    MAC     ------------
>> |   i.MX   | MAC |  PHY  |   PHY   |  PHY  |------------|  88E6390 |
>> |   6ULL   |-----|KSZ8081|---------|LAN8742|	MDIO	|P0        |
>> |          |     |ID 0x1 |         | ID0x0 |------------|          |
>> |          |     ---------         ---------            |          |
>> |          |	     |                                  |MULTI CHIP|
>> |          |	     |MDIO                              |ADDR MODE |
>> |          |	     |                                  |          |
>> |          |--------------------------------------------|   PHY ID |
>> |          |                    MDIO                    |     0x2  |
>> ------------						------------
> Hi Benjamin
>
> KSZ8081 is a 10/100 PHY, i think.
> LAN8742 is also a 10/100 PHY.
>
> However, DSA will configure the CPU port MAC to its maximum speed. So
> port 0 will be doing 1G. I don't know if specifying phy-mode = "rmii"
> is enough. You should take a look at the port status and configuration
> registers, see if the MAC is being forced to 1G, or 100M.
>
> You could add a fixed-phy to port 0 with speed 100. That will at least
> get the MAC part configured correctly.
>
> Is the LAN8742 strapped so that on power on it will auto-neg? I've
> seen a few board with this back-to-back PHY setup, and they just rely
> on the PHYs doing the right thing on power up, no software involved.
>
>    Andrew

Good morning Andrew,
thanks for your reply. You're right, both PHYs are 10/100.

I already added a fixed-link like this:

			port@0 {
				reg = <0>;
				label = "cpu";
				ethernet = <&fec1>;
				phy-mode = "rmii";
				phy-handle = <&switch0phy0>;
                                fixed-link {
                                        speed = <100>;
                                        full-duplex;
                                };
			};

I hope you mean that with fixed-phy? But this doesn't changed anything.

I would like to look at the port registers, but how can I read it when DSA 
is started? When I delete the whole DSA part from the devicetree I found the 
switch in multi chip addressing mode and can read the registers per indirect
reads from PHY ID 0x2. But when DSA is initialized our MII tool(which tool 
could I use for that? ethtool?) gives back some odd values. Of course, I 
would like to read it when DSA is running.

We will check the strapping again, because we have a strange behavior. On 
some boots the external MDIO found a PHY at PHY ID 0x0 and on some it found
nothing. 

Thanks,
Benjamin

Re: DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

[Andrew Lunn]

> thanks for your reply. You're right, both PHYs are 10/100.
> 
> I already added a fixed-link like this:
> 
> 			port@0 {
> 				reg = <0>;
> 				label = "cpu";
> 				ethernet = <&fec1>;
> 				phy-mode = "rmii";
> 				phy-handle = <&switch0phy0>;
>                                 fixed-link {
>                                         speed = <100>;
>                                         full-duplex;
>                                 };
> 			};
> 
> I hope you mean that with fixed-phy? But this doesn't changed anything.

You probably have multiple issues, and it is not going to work until
you have them all solved.

You can get access to the registers etc, using patches from:

https://github.com/vivien/linux.git dsa/debugfs

I've only seen the external MDIO bus on the 6390 used for C45 PHYs. So
there is a chance the driver code for C22 is broken?

      Andrew

Re: DSA setup IMX6ULL and Marvell 88E6390 with 2 Ethernet Phys - CPU Port is not working

[Benjamin Beckmeyer]

>> thanks for your reply. You're right, both PHYs are 10/100.
>>
>> I already added a fixed-link like this:
>>
>> 			port@0 {
>> 				reg = <0>;
>> 				label = "cpu";
>> 				ethernet = <&fec1>;
>> 				phy-mode = "rmii";
>> 				phy-handle = <&switch0phy0>;
>>                                 fixed-link {
>>                                         speed = <100>;
>>                                         full-duplex;
>>                                 };
>> 			};
>>
>> I hope you mean that with fixed-phy? But this doesn't changed anything.
> You probably have multiple issues, and it is not going to work until
> you have them all solved.
>
> You can get access to the registers etc, using patches from:
>
> https://github.com/vivien/linux.git dsa/debugfs
>
> I've only seen the external MDIO bus on the 6390 used for C45 PHYs. So
> there is a chance the driver code for C22 is broken?
>
>       Andrew
>
Hi Andrew,
after you helped me a lot with my other custom board this board is now
up and running too. 

I deleted the whole MDIO part from the device tree except the external MDIO
part (mdio1) and set up the bridge.

For both boards I haven't tested the fiber (serdes) channels but I will 
soon. When I'm running into problems with that, I will contact the mailing 
list again. 

Thanks,
Benjamin