Re: Distributed Switch Architecture(DSA)

[Lennert Buytenhek <buytenh@wantstofly.org>]

On Sat, Jun 19, 2010 at 04:22:18PM +0200, Joakim Tjernlund wrote:

> > > > > OK. With DSA, how does one configure VLANs, policing and
> > > > > parameters in the HW switch that don't map or exist in the
> > > > > linux bridge?
> > > >
> > > > The idea is to use existing kernel interface for this as much as
> > > > possible.  So e.g. if you do:
> > > >
> > > >    vconfig add lan1 123
> > > >    vconfig add lan2 123
> > > >    brctl addbr br123
> > > >    brctl addif br123 lan1.123
> > > >    brctl addif br123 lan2.123
> > > >
> > > > Then the DSA code (or some userspace netlink listener helper, or some
> > > > combination of both) should ideally also detect that VLAN 123 on
> > > > interfaces lan1 and lan2 are to be bridged together, and program the
> > > > switch chip accordingly.  I think all VLAN configurations that at least
> > > > the Marvell hardware supports can be expressed this way.
> > >
> > > Yes, but I image that this breaks down when you want to do something
> > > a bit more advanced. For example I don't think linux VLANs supports
> > > "shared VLAN learning"(SVL) and to configure a HW switch to do SVL
> > > one would first have to impl. that in Linux VLAN and then add the DSA
> > > code to get the config to the switch.
> >
> > Yes.  But that's really the best way to do it, in my humble opinion.
> 
> I will buy that for the moment. I can't see a better way either if
> you truly want to integrate a HW switch into linux. I just wish
> Linux VLANs had some support for SVL too

You know how to fix that. :)


> > If you don't go the host networking stack integration route, you end
> > up with something like the vendor drivers.  Which work fine for most
> > scenarios.. until you want to do something like talking TCP/IP using
> > the host TCP stack over some of the switch ports, at which point the
> > lack of host networking stack integration comes to bite you.
> 
> Just doing STP will bite you :)

Most people deal with this by running a userland STP daemon that uses
raw sockets to inject manually (i.e. in userspace) DSA-tagged packets
onto the eth0 (or whatever) interface.  This "works" (for some
definitions of 'works') for UDP apps such as a DHCP server as well --
this crappy approach unfortunately only really breaks down for TCP.


> > > Not sure how one would express whether VLAN tags should be stripped
> > > off or not when egressing the HW switch's physical port.
> >
> > If you transmit a packet onto 'lan', it will be sent to the switch chip
> > with an "untagged" DSA tag.  If you transmit a packet onto 'lan.123',
> > it will be sent to the switch chip with a "tagged" DSA tag.  See
> > net/dsa/tag_dsa.c for details.
> 
> Ah, now I get it, thanks.
> However, how does this work for LAN to LAN pkgs? LAN1 and LAN2 could be
> in the same VLAN but one is implicit(port) VLAN and the
> other is explicit.

If you tell the HW switch to forward these packets, they will never
appear at the CPU interface, so the DSA tagging/untagging doesn't enter
the picture.


> How do I config the HW switch to do that?

Tell the switch that the vlan is native on one of the ports but not on
the other.  It's been a while since I looked at the chip docs but there
are ways of doing this.


> > > Furthermore, suppose one have a big HW switch, 48 ports, and lots of
> > > VLANs in that HW switch one would have to create a lot of virtual I/Fs
> > > and VLANs in linux just to configure the HW switch. This wastes
> > > resources on the CPU.
> >
> > Where the 'resource waste' is on the order of a couple of tens or
> > hundreds of kilobytes of RAM.  If this is a problem for your host
> > CPU, I think you have bigger problems anyway.
> 
> That is not a very good argument, this is how bloat builds.

If you have a better way of getting all the features while spending
less resources, please step forward with your ideas.  The current design
is the best I could come up with, but I'm sure it's not optimal in its
current form.


> > > > To configure things like ingress/egress rate limiting and such in the
> > > > switch chip for which there is no Linux counterpart interface, I suppose
> > > > some sysfs interface or so might suffice.
> > >
> > > Yes, there are aspects of a HW switch that doesn't map into DSA currently.
> > > Perhaps one should add some framework to support this?
> >
> > Sounds good.
> 
> Any idea how such an framework should look like? What transport
> mechanism is suitable to talk to a user space daemon?

Have a look at netlink.