Using a bridge for DSA and non-DSA devices

Using a bridge for DSA and non-DSA devices

[Christian Eggers]

Usually a bridge does forwarding of traffic between different hardware
interfaces in software. For DSA, setting up a bridge configures the
hardware in a way that traffic is forwarded in hardware.

Is there any problem combining these two situations on a single bridge?
Currently I use a bridge for configuring a DSA switch with two DSA slave
interfaces. Can I add a non-DSA device (e.g. an USB Ethernet gadget)
to this bridge?


regards
Christian

Re: Using a bridge for DSA and non-DSA devices

[Vladimir Oltean]

Hi Christian,

On Mon, Dec 05, 2022 at 07:15:42PM +0100, Christian Eggers wrote:
> Usually a bridge does forwarding of traffic between different hardware
> interfaces in software. For DSA, setting up a bridge configures the
> hardware in a way that traffic is forwarded in hardware.
> 
> Is there any problem combining these two situations on a single bridge?
> Currently I use a bridge for configuring a DSA switch with two DSA slave
> interfaces. Can I add a non-DSA device (e.g. an USB Ethernet gadget)
> to this bridge?

In the model that the DSA core tries to impose, software bridging is
possible, as long as you understand the physical constraints (throughput
will be limited by the link speed of the CPU ports), and as long as the
switch doesn't use DSA_TAG_PROTO_NONE (a remnant of the past).

Unfortunately the results might depend on which switch driver you use
for this, since some driver cooperation is needed for smooth sailing,
and we don't see perfect uniformity. See the
ds->assisted_learning_on_cpu_port flag for some more details.

Did you already try to experiment with software bridging and faced any
issues?

Re: Using a bridge for DSA and non-DSA devices

[Christian Eggers]

Hi Vladimir,

On Monday, 5 December 2022, 20:08:05 CET, Vladimir Oltean wrote:
> Hi Christian,
> 
> In the model that the DSA core tries to impose, software bridging is
> possible, as long as you understand the physical constraints (throughput
> will be limited by the link speed of the CPU ports), and as long as the
> switch doesn't use DSA_TAG_PROTO_NONE (a remnant of the past).

my hope was that a "combined" mode would be possible where traffic
between the DSA slave ports is forwarded in hardware and only other
traffic requires CPU intervention. Our embedded device uses the
KSZ9563 3-port switch with two DSA slave ports. The intention is
that the customer can daisy chain multiple of our devices without
the need for extra Ethernet switches. For reasonable performance,
forwarding shall be done in hardware, especially as the external
port a 1 GBit Ethernet whilst the CPU port is only 100 MBit/s.

Beside the Ethernet interfaces I would like to add further connectivity
options (like USB gadget, WiFi, Bluetooth, ...). I consider these
interfaces as "secondary" in terms of performance and I think that
forwarding will not be an use case here. Currently, each secondary
interface has its own subnet (no bridging), but for every interface I add,
I have to "invent" new IP ranges which eventually collide with other
networks of the customers.

I already took the "lets use IP4LL" joker on one interface but I
learned today, that I cannot do this for further ones
(at least not without bridging). To make things even worse, we decided
to always configure IP4LL as a secondary address on the Ethernet
interfaces (NetworkManager just gained support for this). That's
why I estimate whether it makes sense to put all external interfaces
below a common bridge with only one IP address at all (or one pair
of DHCP+IP4LL).

> 
> Unfortunately the results might depend on which switch driver you use
> for this, since some driver cooperation is needed for smooth sailing,
> and we don't see perfect uniformity. See the
> ds->assisted_learning_on_cpu_port flag for some more details.

Thanks for the pointer, unfortunately this hasn't been implemented
yet for the KSZ switches.

> Did you already try to experiment with software bridging and faced any
> issues?

No, I didn't. But just to make it clear: Will the DSA framework
change to "pure software" switching as soon I add the first non-DSA
slave to an exisiting DSA bridge?

regards
Christian

Re: Using a bridge for DSA and non-DSA devices

[Vladimir Oltean]

On Mon, Dec 05, 2022 at 09:08:38PM +0100, Christian Eggers wrote:
> No, I didn't. But just to make it clear: Will the DSA framework
> change to "pure software" switching as soon I add the first non-DSA
> slave to an exisiting DSA bridge?

No, DSA doesn't change to pure software switching when foreign interfaces
are added to the bridge. The forwarding decision (hardware or software)
is taken per packet. From the perspective of the DSA switch, a station
behind a foreign bridge port is just behind the CPU port, hence the idea
of managing FDB entries towards that port. Packets forwarded by hardware
are simply not seen by software (if not flooded). Or if they're flooded,
software will set the skb->offload_fwd_mark flag, and this will tell the
bridge that the packet was already flooded to the ksz bridge ports, but
not to other (foreign) bridge ports. So the bridge will clone this skb
and send it to the USB adapter bridge port and what not.

But I'm not sure that you're asking the right question with that bridge,
since you don't seem to need Ethernet bridging with these other technologies.

I don't know what is the most civilized way to solve what you're trying
to achieve ("too many IP addresses when I just want to access the board
over one of the available point-to-point links"?!). Have you considered
putting a unique (routable) IP address on the loopback interface of
your board? If you put 1.2.3.4/32 on lo, Linux should respond to a ping
towards 1.2.3.4 no matter which interface the ICMP request came from.
Granted, the ICMP reply will have the IP address of the connected
interface as source.