Representing cpu-port of a switch

Representing cpu-port of a switch

[sk.syed2]

Hi,
   Sorry if this has been discussed elsewhere. The relevant discussion
is here: https://www.spinics.net/lists/netdev/msg407313.html.
  But I have few questions in addition to those answered above. We
have simple 3-port switch, with two ports(connected via phys to
external(front panel)) and one cpu port connected to the cpu via dmas.
Our switch doesn't do any tagging protocol. It simply forwards a frame
to cpu based on fdb entry. Any frame can only be received/transmitted
only by this internal port.
Without tagging, we cant really use DSA, and hide the cpu/dsa port. So
if we expose this cpu port as a interface with fixed-phy
infrastructure does it create any problems? DSA documentation says one
cannot open a socket on cpu/dsa port and send/receive traffic. Is it
fairly common to use internal/cpu port as a network interface- i.e,
creating a socket and send/receive traffic?
One problem is how to report back when network errors(like if both
front panel ports are disconnected, the expectation is to bring this
cpu port down?).
We also need to offload all the switch configuration to switch-dev. So
the question is using switch-dev without DSA and representing a cpu
port as a normal network interface would be ok?

thanks
-syed

Re: Representing cpu-port of a switch

[Florian Fainelli]

On April 29, 2018 11:43:50 AM PDT, "sk.syed2" <sk.syed2@gmail.com> wrote:
>Hi,
>   Sorry if this has been discussed elsewhere. The relevant discussion
>is here: https://www.spinics.net/lists/netdev/msg407313.html.
>  But I have few questions in addition to those answered above. We
>have simple 3-port switch, with two ports(connected via phys to
>external(front panel)) and one cpu port connected to the cpu via dmas.

Sounds pretty standard so far.

>Our switch doesn't do any tagging protocol. It simply forwards a frame
>to cpu based on fdb entry. Any frame can only be received/transmitted
>only by this internal port.

Again, pretty standard. What you probably meant is that for host destined or initiated traffic you must use that internal port to egress/ingress frames towards the other external ports.


>Without tagging, we cant really use DSA, and hide the cpu/dsa port. So
>if we expose this cpu port as a interface with fixed-phy
>infrastructure does it create any problems?

Well the fact that you don't have a tagging protocol does not really mean you cannot use DSA. You could create your own tagging format which in your case could be as simple as keeping the prepended DMA packet descriptor and have the parsing of that descriptor be done in a DSA tagger driver. Not that I would necessarily recommend that though, see below.

 DSA documentation says one
>cannot open a socket on cpu/dsa port and send/receive traffic. Is it
>fairly common to use internal/cpu port as a network interface- i.e,
>creating a socket and send/receive traffic?

In the context of DSA, all external ports are represented by a network device. This means that the CPU/management port is only used to ingress/egress frames that include the tag which either the switch hardware inserts on its way to the host or conversely that the host must insert to have the switch do the appropriate switching operation. If you do not use tags and you still have a way to target specific external ports the same representation should happen and you do not want to expose the internal port because it will only be used to send/receive traffic from the external ports and it will not be used to send or receive traffic to itself (so to speak). Just like with DSA you should have the ability to create network devices that are per-port and you can use the HW provided information to deliver packets to the appropriate destination port, conversely send from the appropriate source port.


>One problem is how to report back when network errors(like if both
>front panel ports are disconnected, the expectation is to bring this
>cpu port down?).

The CPU port should be considered always UP and the external ports must have proper link notifications in place through PHYLIB/PHYLINK preferably. With link management in place the carrier state is properly managed and the network stack won't send traffic from a port that is not UP.

>We also need to offload all the switch configuration to switch-dev. So
>the question is using switch-dev without DSA and representing a cpu
>port as a normal network interface would be ok?

Using switchdev without DSA is absolutely okay, see rocker and mlxsw, but neither of those do represent their CPU/management port for the reasons outlined above that it is only used to convey traffic to/from other ports that have a proper network device representation.

>
>thanks
>-syed

Hello,
-- 
Florian

Re: Representing cpu-port of a switch

[sk.syed2]

> Again, pretty standard. What you probably meant is that for host destined or initiated traffic you must use that internal port to egress/ingress frames towards the other external ports.
>
Yes.

>
>>Without tagging, we cant really use DSA, and hide the cpu/dsa port. So
>>if we expose this cpu port as a interface with fixed-phy
>>infrastructure does it create any problems?
>
> Well the fact that you don't have a tagging protocol does not really mean you cannot use DSA. You could create your own tagging format which in your case could be as simple as keeping the prepended DMA packet descriptor and have the parsing of that descriptor be done in a DSA tagger driver.
This would need HW change wont it?

 Not that I would necessarily recommend that though, see below.
>

>  DSA documentation says one
>>cannot open a socket on cpu/dsa port and send/receive traffic. Is it
>>fairly common to use internal/cpu port as a network interface- i.e,
>>creating a socket and send/receive traffic?
>
> In the context of DSA, all external ports are represented by a network device. This means that the CPU/management port is only used to ingress/egress frames that include the tag which either the switch hardware inserts on its way to the host or conversely that the host must insert to have the switch do the appropriate switching operation. If you do not use tags and you still have a way to target specific external ports the same representation should happen and you do not want to expose the internal port because it will only be used to send/receive traffic from the external ports and it will not be used to send or receive traffic to itself (so to speak).
Just like with DSA you should have the ability to create network
devices that are per-port and you can use the HW provided information
to deliver packets to the appropriate destination port, conversely
send from the appropriate source port.
>
>
The switch HW doesn't have any provision to target specific external
port. I think I didnt make this clear. We would like to integrate a
switch along with an endpoint into a single HW. Meaning we would like
to use internal cpu port as any normal network port to send and
receive traffic. The external network ports will not be used to
send/receive normal traffic(except for control/mgmt frames). So, the
two external front panel ports tied to phys are lets say eth1, eth2.
The cpu port tied to DMAs is represented using fixed-link/always_on
interface say ep0(endpoint). Now applications can open a socket and
send/receive traffic from ep0. The switch does forwarding based on
programmed CAM. Do you see any problem with this approach?



>>One problem is how to report back when network errors(like if both
>>front panel ports are disconnected, the expectation is to bring this
>>cpu port down?).
>
> The CPU port should be considered always UP and the external ports must have proper link notifications in place through PHYLIB/PHYLINK preferably. With link management in place the carrier state is properly managed and the network stack won't send traffic from a port that is not UP.
>
>>We also need to offload all the switch configuration to switch-dev. So
>>the question is using switch-dev without DSA and representing a cpu
>>port as a normal network interface would be ok?
>
> Using switchdev without DSA is absolutely okay, see rocker and mlxsw, but neither of those do represent their CPU/management port for the reasons outlined above that it is only used to convey traffic to/from other ports that have a proper network device representation.
>
May be this is something elementary, but what do you mean by proper
network device representation that cpu port lacks?

Re: Representing cpu-port of a switch

[Florian Fainelli]

On 04/30/2018 06:06 AM, sk.syed2 wrote:
>> Again, pretty standard. What you probably meant is that for host destined or initiated traffic you must use that internal port to egress/ingress frames towards the other external ports.
>>
> Yes.
> 
>>
>>> Without tagging, we cant really use DSA, and hide the cpu/dsa port. So
>>> if we expose this cpu port as a interface with fixed-phy
>>> infrastructure does it create any problems?
>>
>> Well the fact that you don't have a tagging protocol does not really mean you cannot use DSA. You could create your own tagging format which in your case could be as simple as keeping the prepended DMA packet descriptor and have the parsing of that descriptor be done in a DSA tagger driver.
> This would need HW change wont it?
> 
>  Not that I would necessarily recommend that though, see below.
>>
> 
>>  DSA documentation says one
>>> cannot open a socket on cpu/dsa port and send/receive traffic. Is it
>>> fairly common to use internal/cpu port as a network interface- i.e,
>>> creating a socket and send/receive traffic?
>>
>> In the context of DSA, all external ports are represented by a network device. This means that the CPU/management port is only used to ingress/egress frames that include the tag which either the switch hardware inserts on its way to the host or conversely that the host must insert to have the switch do the appropriate switching operation. If you do not use tags and you still have a way to target specific external ports the same representation should happen and you do not want to expose the internal port because it will only be used to send/receive traffic from the external ports and it will not be used to send or receive traffic to itself (so to speak).
> Just like with DSA you should have the ability to create network
> devices that are per-port and you can use the HW provided information
> to deliver packets to the appropriate destination port, conversely
> send from the appropriate source port.
>>
>>
> The switch HW doesn't have any provision to target specific external
> port. I think I didnt make this clear. We would like to integrate a
> switch along with an endpoint into a single HW. Meaning we would like
> to use internal cpu port as any normal network port to send and
> receive traffic. The external network ports will not be used to
> send/receive normal traffic(except for control/mgmt frames). So, the
> two external front panel ports tied to phys are lets say eth1, eth2.

The only problem with that approach is that eth1 and eth2 are only
control interfaces, they cannot transfer any data, eth0 does that, see
below.

> The cpu port tied to DMAs is represented using fixed-link/always_on
> interface say ep0(endpoint). Now applications can open a socket and
> send/receive traffic from ep0. The switch does forwarding based on
> programmed CAM. Do you see any problem with this approach?

No, this is entirely similar to what we do in DSA with DSA_PROTO_TAG_NONE.

In premise, if you created a VLAN identifier for each of your front
panel port, you could emulate in SW what you do not have in HW which is
to target specific front panel ports.

> 
> 
> 
>>> One problem is how to report back when network errors(like if both
>>> front panel ports are disconnected, the expectation is to bring this
>>> cpu port down?).
>>
>> The CPU port should be considered always UP and the external ports must have proper link notifications in place through PHYLIB/PHYLINK preferably. With link management in place the carrier state is properly managed and the network stack won't send traffic from a port that is not UP.
>>
>>> We also need to offload all the switch configuration to switch-dev. So
>>> the question is using switch-dev without DSA and representing a cpu
>>> port as a normal network interface would be ok?
>>
>> Using switchdev without DSA is absolutely okay, see rocker and mlxsw, but neither of those do represent their CPU/management port for the reasons outlined above that it is only used to convey traffic to/from other ports that have a proper network device representation.
>>
> May be this is something elementary, but what do you mean by proper
> network device representation that cpu port lacks?

CPU ports in these cases are not created as a separate net_device
instance, they exist in the HW and at the SW level because we need to
use them to direct traffic to/from specific front-panel port using a
specific DMA capability (e.g: a special set of bits in a DMA
descriptor). In your case, you don't have that ability in your HW, so
you must actually create a CPU net_device for applications to be able to
send traffic.
-- 
Florian