dsa: using multi-gbps speeds on CPU port

dsa: using multi-gbps speeds on CPU port

[Maxime Chevallier]

Hello everyone,

I'm working on a setup where I have a 88e6390X DSA switch connected to
a CPU (an armada 8040) with 2500BaseX and RXAUI interfaces (we only use
one at a time).

I'm facing a limitation with the current way to represent that link,
where we use a fixed-link description in the CPU port, like this :

...
switch0: switch0@1 {
	...
	port@0 {
		reg = <0>;
		label = "cpu";
		ethernet = <&eth0>;
		phy-mode = "2500base-x";
		fixed-link {
			speed = <2500>;
			full-duplex;
		};
	};
};
...

In this scenario, the dsa core will try to create a PHY emulating the
fixed-link on the DSA port side. This can't work with link speeds above
1Gbps, since we don't have any emulation for these PHYs, which would be
using C45 MMDs.

We could add support to emulate these modes, but I think there were some
discussions about using phylink to support these higher speed fixed-link
modes, instead of using PHY emulation.

However using phylink in master DSA ports seems to be a bit tricky,
since master ports don't have a dedicated net_device, and instead
reference the CPU-side netdevice (if I understood correctly).

I'll be happy to help on that, but before prototyping anything, I wanted
to have your thougts on this, and see if you had any plans.

Thanks,

Maxime

Re: dsa: using multi-gbps speeds on CPU port

[Andrew Lunn]

On Wed, May 15, 2019 at 02:39:36PM +0200, Maxime Chevallier wrote:
> Hello everyone,
> 
> I'm working on a setup where I have a 88e6390X DSA switch connected to
> a CPU (an armada 8040) with 2500BaseX and RXAUI interfaces (we only use
> one at a time).

Hi Maxime

RXAUI should just work. By default, the CPU port is configured to its
maximum speed, so it should be doing 10Gbps. Slowing it down to
2500BaseX is however an issue.

> I'm facing a limitation with the current way to represent that link,
> where we use a fixed-link description in the CPU port, like this :
> 
> ...
> switch0: switch0@1 {
> 	...
> 	port@0 {
> 		reg = <0>;
> 		label = "cpu";
> 		ethernet = <&eth0>;
> 		phy-mode = "2500base-x";
> 		fixed-link {
> 			speed = <2500>;
> 			full-duplex;
> 		};
> 	};
> };
> ...
> 
> In this scenario, the dsa core will try to create a PHY emulating the
> fixed-link on the DSA port side. This can't work with link speeds above
> 1Gbps, since we don't have any emulation for these PHYs, which would be
> using C45 MMDs.
> 
> We could add support to emulate these modes, but I think there were some
> discussions about using phylink to support these higher speed fixed-link
> modes, instead of using PHY emulation.
> 
> However using phylink in master DSA ports seems to be a bit tricky,
> since master ports don't have a dedicated net_device, and instead
> reference the CPU-side netdevice (if I understood correctly).

I think you are getting your terminology wrong. 'master' is eth0 in
the example you gave above. CPU and DSA ports don't have netdev
structures, and so any PHY used with them is not corrected to a
netdev.

> I'll be happy to help on that, but before prototyping anything, I wanted
> to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we
already have all the code needed to configure a port, and an API to do
it, the adjust_link() callback. Things have moved on since then, and
we now have an additional API, .phylink_mac_config(). It might be
better to directly use that. If there is a max-speed property, create
a phylink_link_state structure, which has no reference to a netdev,
and pass it to .phylink_mac_config().

It is just an idea, but maybe you could investigate if that would
work.

On the master interface, the armada 8040, eth0, you still need
something. However, if you look at phylink_parse_fixedlink(), it puts
the speed etc into a phylink_link_state. It never instantiates a
fixed-phy. So i think that could be expanded to support higher speeds
without too much trouble. The interesting part is the IOCTL handler.

	Andrew

Re: dsa: using multi-gbps speeds on CPU port

[Maxime Chevallier]

Hi Andrew,

On Wed, 15 May 2019 15:27:01 +0200
Andrew Lunn <andrew@lunn.ch> wrote:

>I think you are getting your terminology wrong. 'master' is eth0 in
>the example you gave above. CPU and DSA ports don't have netdev
>structures, and so any PHY used with them is not corrected to a
>netdev.

Ah yes sorry, I'm still in the process of getting familiar with the
internals of DSA :/

>> I'll be happy to help on that, but before prototyping anything, I wanted
>> to have your thougts on this, and see if you had any plans.  
>
>There are two different issues here.
>
>1) Is using a fixed-link on a CPU or DSA port the right way to do this?
>2) Making fixed-link support > 1G.
>
>The reason i decided to use fixed-link on CPU and DSA ports is that we
>already have all the code needed to configure a port, and an API to do
>it, the adjust_link() callback. Things have moved on since then, and
>we now have an additional API, .phylink_mac_config(). It might be
>better to directly use that. If there is a max-speed property, create
>a phylink_link_state structure, which has no reference to a netdev,
>and pass it to .phylink_mac_config().
>
>It is just an idea, but maybe you could investigate if that would
>work.

Ok I see what you mean, this would allow us to get rid of the phydev
built from the fixed-link, and the .adjust_link call. I'll prototype
that, thanks for the hint.

>On the master interface, the armada 8040, eth0, you still need
>something. However, if you look at phylink_parse_fixedlink(), it puts
>the speed etc into a phylink_link_state. It never instantiates a
>fixed-phy. So i think that could be expanded to support higher speeds
>without too much trouble. The interesting part is the IOCTL handler.

Yes I'm not too worried about that part, unless I missed something this
shouldn't be too problematic.

Once again, thanks for your help,

Maxime

Re: dsa: using multi-gbps speeds on CPU port

[Florian Fainelli]

On 5/15/19 7:02 AM, Maxime Chevallier wrote:
> Hi Andrew,
> 
> On Wed, 15 May 2019 15:27:01 +0200
> Andrew Lunn <andrew@lunn.ch> wrote:
> 
>> I think you are getting your terminology wrong. 'master' is eth0 in
>> the example you gave above. CPU and DSA ports don't have netdev
>> structures, and so any PHY used with them is not corrected to a
>> netdev.
> 
> Ah yes sorry, I'm still in the process of getting familiar with the
> internals of DSA :/
> 
>>> I'll be happy to help on that, but before prototyping anything, I wanted
>>> to have your thougts on this, and see if you had any plans.  
>>
>> There are two different issues here.
>>
>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
>> 2) Making fixed-link support > 1G.
>>
>> The reason i decided to use fixed-link on CPU and DSA ports is that we
>> already have all the code needed to configure a port, and an API to do
>> it, the adjust_link() callback. Things have moved on since then, and
>> we now have an additional API, .phylink_mac_config(). It might be
>> better to directly use that. If there is a max-speed property, create
>> a phylink_link_state structure, which has no reference to a netdev,
>> and pass it to .phylink_mac_config().
>>
>> It is just an idea, but maybe you could investigate if that would
>> work.
> 
> Ok I see what you mean, this would allow us to get rid of the phydev
> built from the fixed-link, and the .adjust_link call. I'll prototype
> that, thanks for the hint.

Vladimir mentioned a few weeks ago that he is considering adding support
for PHYLIB and PHYLINK to run without a net_device instance, you two
should probably coordinate with each other and make sure both of your
requirements (which are likely the same) get addressed.
-- 
Florian

Re: dsa: using multi-gbps speeds on CPU port

[Russell King - ARM Linux admin]

On Wed, May 15, 2019 at 09:09:26AM -0700, Florian Fainelli wrote:
> Vladimir mentioned a few weeks ago that he is considering adding support
> for PHYLIB and PHYLINK to run without a net_device instance, you two
> should probably coordinate with each other and make sure both of your
> requirements (which are likely the same) get addressed.

I don't see how that's sane unless we just replace the "netdevice" in
there with an opaque "void *" and lose the typechecking.

That then means we'd need to eradicate all the messages therein, since
we can't use netdev_*() functions to print.

Then there's the patches I still have, that were rejected, and have had
no progress to get SFP working on 88x3310 - I'm just not bothering to
push them due to the rejection, and the lack of any ideas how to
approach this problem.  So we have the Macchiatobin which has now been
around for quite some time with SFP+ slots that are not particularly
functional with mainline kernels (but hey, I don't care, because they
work for me, because I have the patches that work!)

You all know where that is, I've tried pointing it out several times...

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: dsa: using multi-gbps speeds on CPU port

[Vladimir Oltean]

On Wed, 15 May 2019 at 19:19, Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Wed, May 15, 2019 at 09:09:26AM -0700, Florian Fainelli wrote:
> > Vladimir mentioned a few weeks ago that he is considering adding support
> > for PHYLIB and PHYLINK to run without a net_device instance, you two
> > should probably coordinate with each other and make sure both of your
> > requirements (which are likely the same) get addressed.
>
> I don't see how that's sane unless we just replace the "netdevice" in
> there with an opaque "void *" and lose the typechecking.
>
> That then means we'd need to eradicate all the messages therein, since
> we can't use netdev_*() functions to print.
>
> Then there's the patches I still have, that were rejected, and have had
> no progress to get SFP working on 88x3310 - I'm just not bothering to
> push them due to the rejection, and the lack of any ideas how to
> approach this problem.  So we have the Macchiatobin which has now been
> around for quite some time with SFP+ slots that are not particularly
> functional with mainline kernels (but hey, I don't care, because they
> work for me, because I have the patches that work!)
>
> You all know where that is, I've tried pointing it out several times...
>
> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up

I'm unfortunately not up to speed yet on what has been tried so far,
but I do plan to start prototyping the idea soon and I'll probably
find your patches then.
My basic idea is to interface a Raspberry Pi-like board to a dumb
"switch evaluation board" which has only the Ethernet ports and the
SPI/whatever control interface exposed. The DSA CPU/master port combo
in this case would go through a Cat5 cable, which is not going to pan
out very well currently because both the RPi-side PHY and the switch
board-side PHY need some massaging from their respective drivers. Both
PHYs are C22.
A rough idea of what I'm actually planning to do at:
https://www.spinics.net/lists/netdev/msg569087.html

-Vladimir

Re: dsa: using multi-gbps speeds on CPU port

[Andrew Lunn]

> My basic idea is to interface a Raspberry Pi-like board to a dumb
> "switch evaluation board" which has only the Ethernet ports and the
> SPI/whatever control interface exposed. The DSA CPU/master port combo
> in this case would go through a Cat5 cable, which is not going to pan
> out very well currently because both the RPi-side PHY and the switch
> board-side PHY need some massaging from their respective drivers. Both
> PHYs are C22.

Hi Vladimir

There are a number of boards like this, back to back PHYs. But they
all have the switch PHY strapped so they start on power on and
auto-negotiate. DSA then just works.

    Andrew

Re: dsa: using multi-gbps speeds on CPU port

[Maxime Chevallier]

Hi everyone,

On Wed, 15 May 2019 09:09:26 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

>On 5/15/19 7:02 AM, Maxime Chevallier wrote:
>> Hi Andrew,
>> 
>> On Wed, 15 May 2019 15:27:01 +0200
>> Andrew Lunn <andrew@lunn.ch> wrote:
>>   
>>> I think you are getting your terminology wrong. 'master' is eth0 in
>>> the example you gave above. CPU and DSA ports don't have netdev
>>> structures, and so any PHY used with them is not corrected to a
>>> netdev.  
>> 
>> Ah yes sorry, I'm still in the process of getting familiar with the
>> internals of DSA :/
>>   
>>>> I'll be happy to help on that, but before prototyping anything, I wanted
>>>> to have your thougts on this, and see if you had any plans.    
>>>
>>> There are two different issues here.
>>>
>>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
>>> 2) Making fixed-link support > 1G.
>>>
>>> The reason i decided to use fixed-link on CPU and DSA ports is that we
>>> already have all the code needed to configure a port, and an API to do
>>> it, the adjust_link() callback. Things have moved on since then, and
>>> we now have an additional API, .phylink_mac_config(). It might be
>>> better to directly use that. If there is a max-speed property, create
>>> a phylink_link_state structure, which has no reference to a netdev,
>>> and pass it to .phylink_mac_config().
>>>
>>> It is just an idea, but maybe you could investigate if that would
>>> work.  

I've quickly prototyped and tested this solution, and besides a few
tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks
like.

As Russell said, there wasn't anything needed on the master interface
side.

>
>Vladimir mentioned a few weeks ago that he is considering adding support
>for PHYLIB and PHYLINK to run without a net_device instance, you two
>should probably coordinate with each other and make sure both of your
>requirements (which are likely the same) get addressed.

That would help a lot solving this issue indeed, I'll be happy to help
on that, thanks for the tip !

Maxime


-- 
Maxime Chevallier, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com

RE: dsa: using multi-gbps speeds on CPU port

[Ioana Ciornei]

> Subject: Re: dsa: using multi-gbps speeds on CPU port
> 
> Hi everyone,
> 
> On Wed, 15 May 2019 09:09:26 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
> >On 5/15/19 7:02 AM, Maxime Chevallier wrote:
> >> Hi Andrew,
> >>
> >> On Wed, 15 May 2019 15:27:01 +0200
> >> Andrew Lunn <andrew@lunn.ch> wrote:
> >>
> >>> I think you are getting your terminology wrong. 'master' is eth0 in
> >>> the example you gave above. CPU and DSA ports don't have netdev
> >>> structures, and so any PHY used with them is not corrected to a
> >>> netdev.
> >>
> >> Ah yes sorry, I'm still in the process of getting familiar with the
> >> internals of DSA :/
> >>
> >>>> I'll be happy to help on that, but before prototyping anything, I wanted
> >>>> to have your thougts on this, and see if you had any plans.
> >>>
> >>> There are two different issues here.
> >>>
> >>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
> >>> 2) Making fixed-link support > 1G.
> >>>
> >>> The reason i decided to use fixed-link on CPU and DSA ports is that
> >>> we already have all the code needed to configure a port, and an API
> >>> to do it, the adjust_link() callback. Things have moved on since
> >>> then, and we now have an additional API, .phylink_mac_config(). It
> >>> might be better to directly use that. If there is a max-speed
> >>> property, create a phylink_link_state structure, which has no
> >>> reference to a netdev, and pass it to .phylink_mac_config().
> >>>
> >>> It is just an idea, but maybe you could investigate if that would
> >>> work.
> 
> I've quickly prototyped and tested this solution, and besides a few tweaks that
> are needed on the mv88e6xxx driver side, it works fine.
> 
> I'll post an RFC with this shortly, so that you can see what it looks like.
> 
> As Russell said, there wasn't anything needed on the master interface side.
> 
> >
> >Vladimir mentioned a few weeks ago that he is considering adding
> >support for PHYLIB and PHYLINK to run without a net_device instance,
> >you two should probably coordinate with each other and make sure both
> >of your requirements (which are likely the same) get addressed.
> 
> That would help a lot solving this issue indeed, I'll be happy to help on that,
> thanks for the tip !
> 
> Maxime
> 

Hi Maxime,

I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line. 

To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.

Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

-Ioana

Re: dsa: using multi-gbps speeds on CPU port

[Russell King - ARM Linux admin]

On Fri, May 17, 2019 at 05:37:00PM +0000, Ioana Ciornei wrote:
> > Subject: Re: dsa: using multi-gbps speeds on CPU port
> > 
> > Hi everyone,
> > 
> > On Wed, 15 May 2019 09:09:26 -0700
> > Florian Fainelli <f.fainelli@gmail.com> wrote:
> > 
> > >On 5/15/19 7:02 AM, Maxime Chevallier wrote:
> > >> Hi Andrew,
> > >>
> > >> On Wed, 15 May 2019 15:27:01 +0200
> > >> Andrew Lunn <andrew@lunn.ch> wrote:
> > >>
> > >>> I think you are getting your terminology wrong. 'master' is eth0 in
> > >>> the example you gave above. CPU and DSA ports don't have netdev
> > >>> structures, and so any PHY used with them is not corrected to a
> > >>> netdev.
> > >>
> > >> Ah yes sorry, I'm still in the process of getting familiar with the
> > >> internals of DSA :/
> > >>
> > >>>> I'll be happy to help on that, but before prototyping anything, I wanted
> > >>>> to have your thougts on this, and see if you had any plans.
> > >>>
> > >>> There are two different issues here.
> > >>>
> > >>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
> > >>> 2) Making fixed-link support > 1G.
> > >>>
> > >>> The reason i decided to use fixed-link on CPU and DSA ports is that
> > >>> we already have all the code needed to configure a port, and an API
> > >>> to do it, the adjust_link() callback. Things have moved on since
> > >>> then, and we now have an additional API, .phylink_mac_config(). It
> > >>> might be better to directly use that. If there is a max-speed
> > >>> property, create a phylink_link_state structure, which has no
> > >>> reference to a netdev, and pass it to .phylink_mac_config().
> > >>>
> > >>> It is just an idea, but maybe you could investigate if that would
> > >>> work.
> > 
> > I've quickly prototyped and tested this solution, and besides a few tweaks that
> > are needed on the mv88e6xxx driver side, it works fine.
> > 
> > I'll post an RFC with this shortly, so that you can see what it looks like.
> > 
> > As Russell said, there wasn't anything needed on the master interface side.
> > 
> > >
> > >Vladimir mentioned a few weeks ago that he is considering adding
> > >support for PHYLIB and PHYLINK to run without a net_device instance,
> > >you two should probably coordinate with each other and make sure both
> > >of your requirements (which are likely the same) get addressed.
> > 
> > That would help a lot solving this issue indeed, I'll be happy to help on that,
> > thanks for the tip !
> > 
> > Maxime
> > 
> 
> Hi Maxime,
> 
> I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line. 
> 
> To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.
> 
> Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

For SFP, I've already removed much of the netdev bits from that layer,
but I don't see any way to really get rid of it from phylink - we need
access to the netdev state there to know what the carrier state is for
the netdev (phylink tracks that state and manages the carrier state on
behalf of the MAC driver.)

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: dsa: using multi-gbps speeds on CPU port

[Florian Fainelli]

On 5/17/19 11:03 AM, Russell King - ARM Linux admin wrote:
> On Fri, May 17, 2019 at 05:37:00PM +0000, Ioana Ciornei wrote:
>>> Subject: Re: dsa: using multi-gbps speeds on CPU port
>>>
>>> Hi everyone,
>>>
>>> On Wed, 15 May 2019 09:09:26 -0700
>>> Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>
>>>> On 5/15/19 7:02 AM, Maxime Chevallier wrote:
>>>>> Hi Andrew,
>>>>>
>>>>> On Wed, 15 May 2019 15:27:01 +0200
>>>>> Andrew Lunn <andrew@lunn.ch> wrote:
>>>>>
>>>>>> I think you are getting your terminology wrong. 'master' is eth0 in
>>>>>> the example you gave above. CPU and DSA ports don't have netdev
>>>>>> structures, and so any PHY used with them is not corrected to a
>>>>>> netdev.
>>>>>
>>>>> Ah yes sorry, I'm still in the process of getting familiar with the
>>>>> internals of DSA :/
>>>>>
>>>>>>> I'll be happy to help on that, but before prototyping anything, I wanted
>>>>>>> to have your thougts on this, and see if you had any plans.
>>>>>>
>>>>>> There are two different issues here.
>>>>>>
>>>>>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
>>>>>> 2) Making fixed-link support > 1G.
>>>>>>
>>>>>> The reason i decided to use fixed-link on CPU and DSA ports is that
>>>>>> we already have all the code needed to configure a port, and an API
>>>>>> to do it, the adjust_link() callback. Things have moved on since
>>>>>> then, and we now have an additional API, .phylink_mac_config(). It
>>>>>> might be better to directly use that. If there is a max-speed
>>>>>> property, create a phylink_link_state structure, which has no
>>>>>> reference to a netdev, and pass it to .phylink_mac_config().
>>>>>>
>>>>>> It is just an idea, but maybe you could investigate if that would
>>>>>> work.
>>>
>>> I've quickly prototyped and tested this solution, and besides a few tweaks that
>>> are needed on the mv88e6xxx driver side, it works fine.
>>>
>>> I'll post an RFC with this shortly, so that you can see what it looks like.
>>>
>>> As Russell said, there wasn't anything needed on the master interface side.
>>>
>>>>
>>>> Vladimir mentioned a few weeks ago that he is considering adding
>>>> support for PHYLIB and PHYLINK to run without a net_device instance,
>>>> you two should probably coordinate with each other and make sure both
>>>> of your requirements (which are likely the same) get addressed.
>>>
>>> That would help a lot solving this issue indeed, I'll be happy to help on that,
>>> thanks for the tip !
>>>
>>> Maxime
>>>
>>
>> Hi Maxime,
>>
>> I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line. 
>>
>> To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.
>>
>> Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.
> 
> For SFP, I've already removed much of the netdev bits from that layer,
> but I don't see any way to really get rid of it from phylink - we need
> access to the netdev state there to know what the carrier state is for
> the netdev (phylink tracks that state and manages the carrier state on
> behalf of the MAC driver.)

We can make that a callback that is optional in case you want to use a
PHYLINK instance without a backing net_device. If you pass a valid
net_device pointer, then we default to netif_carrier_ok(), else the
caller of phylink_create() (which would have to be renamed, or exposed
with an additional argument, say phylink_create_cb()) needs to provide it.
-- 
Florian

Re: dsa: using multi-gbps speeds on CPU port

[Russell King - ARM Linux admin]

On Fri, May 17, 2019 at 11:10:10AM -0700, Florian Fainelli wrote:
> On 5/17/19 11:03 AM, Russell King - ARM Linux admin wrote:
> > On Fri, May 17, 2019 at 05:37:00PM +0000, Ioana Ciornei wrote:
> >>> Subject: Re: dsa: using multi-gbps speeds on CPU port
> >>>
> >>> Hi everyone,
> >>>
> >>> On Wed, 15 May 2019 09:09:26 -0700
> >>> Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>>
> >>>> On 5/15/19 7:02 AM, Maxime Chevallier wrote:
> >>>>> Hi Andrew,
> >>>>>
> >>>>> On Wed, 15 May 2019 15:27:01 +0200
> >>>>> Andrew Lunn <andrew@lunn.ch> wrote:
> >>>>>
> >>>>>> I think you are getting your terminology wrong. 'master' is eth0 in
> >>>>>> the example you gave above. CPU and DSA ports don't have netdev
> >>>>>> structures, and so any PHY used with them is not corrected to a
> >>>>>> netdev.
> >>>>>
> >>>>> Ah yes sorry, I'm still in the process of getting familiar with the
> >>>>> internals of DSA :/
> >>>>>
> >>>>>>> I'll be happy to help on that, but before prototyping anything, I wanted
> >>>>>>> to have your thougts on this, and see if you had any plans.
> >>>>>>
> >>>>>> There are two different issues here.
> >>>>>>
> >>>>>> 1) Is using a fixed-link on a CPU or DSA port the right way to do this?
> >>>>>> 2) Making fixed-link support > 1G.
> >>>>>>
> >>>>>> The reason i decided to use fixed-link on CPU and DSA ports is that
> >>>>>> we already have all the code needed to configure a port, and an API
> >>>>>> to do it, the adjust_link() callback. Things have moved on since
> >>>>>> then, and we now have an additional API, .phylink_mac_config(). It
> >>>>>> might be better to directly use that. If there is a max-speed
> >>>>>> property, create a phylink_link_state structure, which has no
> >>>>>> reference to a netdev, and pass it to .phylink_mac_config().
> >>>>>>
> >>>>>> It is just an idea, but maybe you could investigate if that would
> >>>>>> work.
> >>>
> >>> I've quickly prototyped and tested this solution, and besides a few tweaks that
> >>> are needed on the mv88e6xxx driver side, it works fine.
> >>>
> >>> I'll post an RFC with this shortly, so that you can see what it looks like.
> >>>
> >>> As Russell said, there wasn't anything needed on the master interface side.
> >>>
> >>>>
> >>>> Vladimir mentioned a few weeks ago that he is considering adding
> >>>> support for PHYLIB and PHYLINK to run without a net_device instance,
> >>>> you two should probably coordinate with each other and make sure both
> >>>> of your requirements (which are likely the same) get addressed.
> >>>
> >>> That would help a lot solving this issue indeed, I'll be happy to help on that,
> >>> thanks for the tip !
> >>>
> >>> Maxime
> >>>
> >>
> >> Hi Maxime,
> >>
> >> I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line. 
> >>
> >> To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.
> >>
> >> Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.
> > 
> > For SFP, I've already removed much of the netdev bits from that layer,
> > but I don't see any way to really get rid of it from phylink - we need
> > access to the netdev state there to know what the carrier state is for
> > the netdev (phylink tracks that state and manages the carrier state on
> > behalf of the MAC driver.)
> 
> We can make that a callback that is optional in case you want to use a
> PHYLINK instance without a backing net_device. If you pass a valid
> net_device pointer, then we default to netif_carrier_ok(), else the
> caller of phylink_create() (which would have to be renamed, or exposed
> with an additional argument, say phylink_create_cb()) needs to provide it.

You'll need:
- a callback to set the carrier state
- a callback to get the current carrier state

at the very least.

Then there's all the netdev printing functions that would have to be
converted to something (which really ought to keep staying which
Linux interface they're referring to).  Given that we are called
before the netdev has a name, and using the device struct isn't
sufficient, I don't see an easy solution to that.  We need it,
especially as we have boards with multiple ethernet devices using
this, so the information that phylink prints has to be meaningful.
Using the parent device is _painful_ so doesn't hack it - and we
have situations where we have multiple ethernet devices for one
parent device.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: dsa: using multi-gbps speeds on CPU port

[Russell King - ARM Linux admin]

On Wed, May 15, 2019 at 03:27:01PM +0200, Andrew Lunn wrote:
> On the master interface, the armada 8040, eth0, you still need
> something. However, if you look at phylink_parse_fixedlink(), it puts
> the speed etc into a phylink_link_state. It never instantiates a
> fixed-phy. So i think that could be expanded to support higher speeds
> without too much trouble. The interesting part is the IOCTL handler.

phylink already supports 2500 and 10G fixed-link (actually, it doesn't
care too much about the speed value, just passing it through), provided
phy_lookup_setting() and the MAC support the speed.

Since it doesn't bother with emulating a set of phy registers, which
then would be read by phylib and translated back to a speed and duplex
for the MAC to use, it's way more flexible when it comes to the old
emulated-phy fixed links code.

What it doesn't do is provide an emulation of C45 PHYs for mii-tool/
mii-diag - ethtool is the way forward, and it supports ethtool for
these speeds.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up