Re: net: dsa: mv88e6xxx architecture

[Luke Howard <lukeh@padl.com>]

Hi Vladimir,

> I think we need an AVB primer. What identifies an AVB stream? Should the
> kernel have a database of them? What actions need to be taken for AVB
> streams different than for best effort traffic? Is it about scheduling
> priority, or about resource reservations, or? Does the custom behavior
> pertain only to AVB streams or is it a more widely useful mechanism?

AVB is an umbrella term which encompasses 802.1AS (gPTP), 802.1Qav (CBS), 802.1Qat (SRP), and various application layer protocols. This patch series is concerned with CBS and, tangentially, SRP: the existing mv88e6xxx PTP support sufficies for 802.1AS, and the application layer protocols are of no concern to the kernel. Further, the existing TC abstractions for CBS also suffice, and the patch series simply implements those.

Your question: what is an AVB stream, and what (if any) state needs to be maintained in the kernel?

Stream reservations, between “talkers” (sources) and “listeners” (sinks) are coordinated by the Multiple Stream Reservation Protocol (MSRP). (Confusingly, _SRP_ itself is an umbrella term encompassing MVRP, MMRP and MSRP. The existing MVRP support in the kernel only implements part of the MRP state machine.)

MSRP coordinates talker advertisements (consisting of a 64-bit stream ID, a [typically multicast] DA, VID, bandwidth requirement, and PCP) with listener advertisements. It also coordinates “domain” advertisements which establish a mapping between AVB or SRP “classes”, and a (PCP, VID) tuple.

At a high level, the kernel does not need to know anything about SRP: it can, and IMO should (like the PTP state machine) be implemented completely in user space. Once SRP has established there is sufficient bandwidth and latency for a steam to flow, it adds the stream DA to the FDB or MDB, and configures the CBS policy appropriately on the egress port.

The catch is what to do with frames that share a priority with an AVB class but are not negotiated by SRP. These frames could crowd out frames from AVB streams. Marvell’s solution is a flag in the ATU which indicates that the DA was added by SRP. This is the one case where a new kernel interface (specifically, a flag passed down to {fdb,mdb}_add()), could be useful. In lieu of that, the patch currently uses a new Kconfig option to indicate that _all_ static FDB/MDB entries should have that flag set, noting that CBS is still usable without this option.

> What is the lifetime of an AVB packet in the Marvell DSA switches (in
> enhanced AVB mode)? Where do they go from these global queues? Still to
> the individual port TX queues, I assume? I find it a bit difficult to
> understand the concept of global queues.

A poor explanation on my part. Briefly: FPri is the switch’s internal frame priority derived from PCP or DSCP, and QPri is queue number.

* PCP/DSCP to FPri mappings are per port
* TX _queues_ are per-port
* The _mapping_ between FPri and QPri is per-switch (at least, pre-6390 family).
* Where the admission control described above is enabled, each SRP class must be assigned a designated FPri and QPri, and this configuration is per-switch

> I don't understand this. A mapping between which port's traffic classes
> and the global AVB classes? You're saying that mqprio TCs 2 and 1 of all
> ports are backed by the same queues?

The same queue _numbers_ but not the same queues. Configuring per-port FPri to QPri mappings is possible on the 6390 family, but not with SRP admission control. 

The current patch reference counts the MQPRIO policy so that it can be applied or removed from additional ports, but only replaced when one referent remains.

Cheers,
Luke