Re: [patch net-next RFC 0/4] introduce infrastructure for support of switch chip datapath

[Thomas Graf <tgraf@suug.ch>]

On 04/02/14 at 09:47am, Florian Fainelli wrote:
> 2014-04-02 9:15 GMT-07:00 Scott Feldman <sfeldma@cumulusnetworks.com>:
> > On Apr 2, 2014, at 8:25 AM, John W. Linville <linville@tuxdriver.com> wrote:
> >> On Wed, Apr 02, 2014 at 10:32:49AM -0400, Andy Gospodarek wrote:
> >>> Maybe this all seems to matter-of-fact and the discussion has
> >>> evolved well beyond something this high-level, but there still seems
> >>> to be significant discussion about whether or not the ASIC should be
> >>> exported as a netdev and I'm just not seeing a compelling reason.
> >>> This was my attempt to explain why.  :)
> >>
> >> Andy and I discussed this off-line, so I am admittedly partial to
> >> the conclusions we shared as reflected above... :-)
> >>
> >> While I might be convinced that there should be _something_ to
> >> represent the switch chip for some purpose (e.g. topology mapping),
> >> I'm not at all convinced that thing should be a netdev.  I don't see
> >> where the switch chip by itself looks much like any other netdev at
> >> all, especially once you model the actual front-panel ports with
> >> their own netdevs.  I do know that having an extra "magic netdev"
> >> in the wireless space added a lot of confusion for no clear gain,
> >> leading to it later being abolished.
> >>
> >> Modeling at the switch level might make more sense from a flow
> >> management perspective?  But if those flows are managed using a netlink
> >> protocol, does it matter what sort of entity is listening and acting
> >> on those messages?  If a switch-specific interface is needed for that,
> >> we should build it rather than pretending it looks like a netdev.
> >> I also think that throwing the DSA switches in with flow-based and
> >> "Enterprise" switches may just be confusing things.
> >>
> >> I think that the opening bid should be a minimal hardware driver that
> >> models each front-panel port with a netdev and passes all traffic
> >> to/from the CPU.  Intelligence beyond that should be added on a
> >> 'can-do' basis, with individual drivers (or corresponding userland
> >> components) listening to existing netlink traffic and implementing
> >> support for existing protocols to the best of their abilities.
> >> Missing functionality in the netlink protocols or other functions
> >> (e.g. bonding, bridging, etc) can be evolved over time as we discover
> >> missing bits required for switch acceleration.
> >
> > I agree completely with your/Andy’s view.  It’s the switch port, not the switch, that needs to be modeled as a netdev.  The switch port is the abstraction that allows other existing virtual devices (bridges, bond, vxlans, etc) to cuddle against.  Is a switch port a special netdev in some way?  At a high level, not really.  I mean in sense it’s just eth48 on a super NIC.  OK, there may be some advantage to setting a IFF_SWITCH_PORT on the switch port netdev, so cuddling netdevs could get a hint that their data plane might be offloaded.
> >
> > I’ve been back-and-forth on the switch netdev.  Today I’m not for it.  But I’m still searching for a reason.  At one point I thought a switch netdev would be nice in a L3 router case where we needed a router IP address to do things like OSPF unnumbered interfaces, but even in that case, we can just put the router IP on lo.  Another reason would be to use the switch netdev as a place for switch-wide settings and status.  For example,
> > ethtool -S stats on switch netdev would show switch-wide stats like ACL drops or something like that.  Maybe a switch device is modeled as a new device class?  I guess it comes down to how much is duplicated between different vendors' switch driver implementations.
> 
> I think the idea behind exposing a switch net_device is to account for
> all special cases where there is not already an existing and
> well-defined model for switch-wide events/control/information that we
> might want to have. Why a net_device, because the switch ports will
> already be exposed as such, so mostly for consistency with the
> presented user-space interface. Whether that net_device exposes
> different child devices of different classes, e.g: MTD partitions to
> access firmware updates, SPI master/slave controller(s), MDIO
> controller(s), is yet to be defined I suppose.

Having a master net_device seemed logical to me at first just
like it always made sense to me to have software bridges be
represented by a net_device. I agree with a lot of the concerns
though.

I see the following uses for a master net_device:
 - represent slave/master relationship and provide IFF_UP control
 - expose non port specific statistics
 - flow configuration
 - tunnel configuration
 - allow creation of virtual ports that are not backed up with HW

I want to expand on the last point a bit. I specifically did not
mention IP configuration above which is what the bridge master is
used frequently. I absolutely like the OVS model where multiple
internal ports can be created which hook into the network stack
and can thus be assigned IPs. The model allows for separate internal
ports to be configured as different VLAN access ports for example.
They also provide multiple AF_PACKET rx handlers, etc.

 sw1p1 -+
 sw1p2 -+       +-sw1int0 (ip=30.0.0.1) -> netif_rx()
 sw1p3 -+- sw1 -+-sw1int1 (vlan=10, ip=10.0.0.1) -> netif_rx()
 sw1p4 -+       +-sw1vxlan0 (remote_ip=20.0.0.2)

If supported by the chip, flows can be setup automatically to
feed these virtual ports and setup encapsultion. Others will
require software fallback. Some will not support it at all.