Re: Is it ok for switch TCAMs to depend on the bridge state?

[Vladimir Oltean <vladimir.oltean@nxp.com>]

On Thu, Nov 11, 2021 at 03:46:35PM +0200, Ido Schimmel wrote:
> On Thu, Nov 11, 2021 at 11:52:55AM +0000, Vladimir Oltean wrote:
> > On Sun, Nov 07, 2021 at 01:50:36PM +0200, Ido Schimmel wrote:
> > > On Tue, Nov 02, 2021 at 11:03:53AM +0000, Vladimir Oltean wrote:
> > > > I've been reviewing a patch set which offloads to hardware some
> > > > tc-flower filters with some TSN-specific actions (ingress policing).
> > > > The keys of those offloaded tc-flower filters are not arbitrary, they
> > > > are the destination MAC address and VLAN ID of the frames, which is
> > > > relevant because these TSN policers are actually coupled with the
> > > > bridging service in hardware. So the premise of that patch set was that
> > > > the user would first need to add static FDB entries to the bridge with
> > > > the same key as the tc-flower key, before the tc-flower filters would be
> > > > accepted for offloading.
> > > 
> > > [...]
> > > 
> > > > I don't have a clear picture in my mind about what is wrong. An airplane
> > > > viewer might argue that the TCAM should be completely separate from the
> > > > bridging service, but I'm not completely sure that this can be achieved
> > > > in the aforementioned case with VLAN rewriting on ingress and on egress,
> > > > it would seem more natural for these features to operate on the
> > > > classified VLAN (which again, depends on VLAN awareness being turned on).
> > > > Alternatively, one might argue that the deletion of a bridge interface
> > > > should be vetoed, and so should the removal of a port from a bridge.
> > > > But that is quite complicated, and doesn't answer questions such as
> > > > "what should you do when you reboot".
> > > > Alternatively, one might say that letting the user remove TCAM
> > > > dependencies from the bridging service is fine, but the driver should
> > > > have a way to also unoffload the tc-flower keys as long as the
> > > > requirements are not satisfied. I think this is also difficult to
> > > > implement.
> > > 
> > > Regarding the question in the subject ("Is it ok for switch TCAMs to
> > > depend on the bridge state?"), I believe the answer is yes because there
> > > is no way to avoid it and effectively it is already happening.
> > > 
> > > To add to your examples and Jakub's, this is also how "ERSPAN" works in
> > > mlxsw. User space installs some flower filter with a mirror action
> > > towards a gretap netdev, but the HW does not do the forwarding towards
> > > the destination.
> > 
> > I don't understand this part. By "forwarding" you mean "mirroring" here,
> 
> Yes
> 
> > and the "destination" is the gretap interface which is offloaded?
> 
> No. See more below
> 
> > 
> > > Instead, it relies on the SW to tell it which headers
> > > (i.e., Eth, IP, GRE) to put on the mirrored packet and tell it from
> > > which port the packet should egress. When we have a bridge in the
> > > forwarding path, it means that the offload state of the filter is
> > > affected by FDB updates.
> > 
> > Here you're saying that the gretap interface whose local IP address is
> > the IP address of a bridge interface that is offloaded by mlxsw, and the
> > precise egress port is determined by the bridge's FDB? But since you
> > don't support bridging with foreign interfaces, why would the mirred
> > rule ever become unoffloaded?
> > 
> > I'm afraid that I don't understand this case very well.
> 
> In software, when you mirror to a gretap via act_mirred, the packet is
> cloned and transmitted through the gretap netdev. This netdev will then
> put a GRE header on the packet, specifying that the next protocol is
> Ethernet. It will then put an IP header on the packet with the
> configured source and destination IPs and route the packet towards its
> destination.
> 
> It is possible that routing will determine that the encapsulated packet
> should be transmitted via a bridge. In which case, the packet will also
> do an FDB lookup in the bridge before determining the egress port.
> 
> In hardware, we don't have a representation for the gretap device.
> Instead, the hardware is kept very simple and requires the driver to
> tell it:
> 
> a. Via which port to mirror the packet
> b. Which headers to encapsulate the packet with
> 
> So the "offload-ability" of the filter is conditioned on software being
> able to determine the correct path, which can change with time following
> FDB/routes/etc updates.

Understood now. So it depends upon a lot more things than just the
bridge state, also IP routes. I thought you were giving an example
related strictly to the bridge. Now it makes more sense. Thanks.