Re: [RFC] net: towards a generic PON framework

[Matheus Sampaio Queiroga <srherobrine20@gmail.com>]

>  > with tests on en7523 and en751221 with some users, I saw them
>  > attaching the OMCI network interface within a bridge, and in order
>  > not to expose this interface again to a user, I implemented my OMCI
>  > within the kernel
> 
> Was that OMCI interface exposed as ARPHRD_ETHER?
> 
> I ask because I do not think a packet-oriented OMCI net_device would
> need Ethernet semantics at all.
> 
> For example, it could use ARPHRD_NONE, have no synthetic L2 header,
> and use IFF_NOARP / IFF_POINTOPOINT.  In that case it should not be a
> valid device to enslave to an Ethernet bridge in the first place.
> 
> net_device itself is not synonymous with Ethernet.  CAN is another
> example of a packet-oriented protocol exposed through net_device with
> different link-layer semantics.
> 
> If the previous OMCI interface used ARPHRD_ETHER, then I wonder whether
> the ability to bridge it was a property of that implementation rather
> than a fundamental problem with representing OMCI as a net_device.
> 
> This would still leave the separate question of whether the G.988
> agent itself belongs in kernel or userspace.

I exposed it by poling in ioctl in the dirtiest way possible, by generic netlink and for sysfs. In all cases, there was an unnecessary overload on the CPU, making it very slow for other tasks
 
>  > with this I also had less overhead in the ethernet system, and faster
>  > responses to OLT due to the new implementation.
> 
> This is also interesting.
> 
> Do you have any measurements for the CPU overhead and OMCI response
> latency before and after moving the agent into the kernel?
> 
> It would help distinguish costs inherent to a userspace OMCI design
> from costs specific to the previous implementation.
> 
> I still think it may be useful to treat OMCC transport and OMCI agent
> placement as two separate design questions.
> 
> For example, a generic xPON layer might provide OMCC transport and
> GEM/T-CONT operations while allowing either:
> 
>    1. an in-kernel OMCI implementation, or
>    2. a userspace OMCI implementation through a packet interface.
> 
> That would avoid requiring every driver to make the same choice before
> we know whether that choice is actually hardware-independent.

with a similar port of the Airoha SDK I had CPU usage of around 2~15% depending on the number of MIB objects in memory, in addition to memory usage of around 122~353Mb in userspace
With OMCI in the kernel, the CPU overhead dropped to around 0.8~2.0% from what I was able to measure, and around 68~258Mb of memory usage with the same MIBs as the userspace

userspace depends on packages arriving at fe+qdma and then userspace, the omci in kernel arrives at fe+qdma and goes directly to omci without going through any other subsystem. with userspace we have responses in around 0.8~1ms, in kernel space we have responses in around 0.5~0.8ms, this for MIB queries already in memory.

for hardware queries we had responses of 1~4ms in userspace and 1ms in kernel space.

on the GEM/T-CONT after the packets are synchronized between the OLT and ONU comes the configuration of the datapath itself, on the Airoha x(gs)PON the pse/fe takes care of the GEM and T-CONT parts, what I do now is decode these values and inform the origin for the xPON subsystem, otherwise it is taken care of by hardware, besides that in the userspace I had to expose several things that are not good are exposed in userspace and much of the work ends up being in kernel space, so That's why I combined the entire system in kernel space.

>  > I am also aware of some Realtek SoCs having slightly different OMCI in
>  > which my subsystems need to be adjusted to see this data, something
>  > that fails in my current implementation
> 
> I think this is particularly useful evidence for keeping the generic
> boundary small.
> 
> If Realtek requires different OMCI handling, that seems to strengthen
> the case for separating common OMCC transport and PON hardware
> operations from the policy of where the G.988 implementation lives.

Much of what was written works with the Realtek SoC, it has the same data as the OMCI export, but instead of passing it as a skbuff package it is treated in a separate way from their systems, but contains the same data as the OMCI package.
When I said that my implementation needs to be an adapter, I'm talking about the issue of the origin of the data and the processing of the data and the delivery of the data to the hardware.
Today, a large part of the system still originates from Airoha SDK, but a lot of things don't match what was done in their SDK.

>  > The optical subsystem in which I implemented it already uses several
>  > subsystems already present in Linux, so temperatures, bias and optical
>  > power are already used within hwmon, for calibration I already use
>  > nvmem to obtain the data and start LDDLA.
> 
> This is very close to what I had in mind.
> 
> Using hwmon for telemetry and NVMEM for calibration seems much cleaner
> than exposing those through a PON-specific userspace ABI.
> 
> The remaining part I would like to understand better is the generic API
> between the PON MAC and the fixed optical frontend, especially for:
> 
>    - LOS;
>    - TX enable;
>    - burst enable / rearm;
>    - protocol and rate selection;
>    - wavelength selection where applicable.
> 
> I think this is also where Benjamin's earlier BOSA/SFP discussion is
> directly relevant, and I would like to hear what the PHY/phylink/SFP
> maintainers think before suggesting a concrete API.

Benjamin idea was to expose the LDDLA subsystem within what we currently have in "sff,sfp"/"sff,sff", but to do so we have to adapt and expose several functions and emulated data for subsystems already present. but the problem that fell was Semtech's LDDLA, since it works for several vendors,
So I implemented a more generic subsystem where it exposes several functions that the LDDLA supports, so it can provide data to the MAC xPON systems such as frequency/type that the LDDLA supports (GPON, EPON, XSPON, XEPON, etc.), BOSA status, hwmon, tx enable, rearm.

From what I've seen in LDDLA Semtech and Airoha, what controls the frequency selector is the PHY and the loaded calibration, and it's not something that the optical subsystem should deal with for now in my view, this should only happen when the PHY doesn't control this part, something that I find difficult for any vendor to implement.

>  > a good part of the subsystem depends on values coming from the
>  > userspace, that's why we wrote a generic netlink and sysfs for
>  > userspace communication
> 
> That also makes sense.
> 
> For ONU identity in particular, I still think it is worth separating
> hardware description from provisioning policy.
> 
> Some values may come from NVMEM, some may be derived from other device
> identity, and some may be supplied by userspace.  I am therefore not
> sure that values such as an OMCI vendor ID should become part of the DT
> ABI unless they really describe immutable hardware properties.
> 
> I will spend some time reading through the implementation you posted.
> 
> At this point I think we have two useful implementation models:
> 
>    - kernel PON MAC + userspace G.988;
>    - kernel PON MAC + in-kernel G.988.
> 
> That is a good basis for identifying which parts are genuinely generic
> and which parts should remain implementation choices.

values can be exchanged by the userspace controller for user-supplied values, SN, MIBs, etc...

Normally these are values predefined by each device vendor, in this example of dts it was from a TP-Link device, where all or most of them use the initials "TPLG" plus part of the device's mac address for the GPON SN, there is little data that must be fixed, the rest must come from userspace.

Then comes the issue of "profiles" where each OLT has its behavior slightly different from the ITU standard, something that could be a matter of some MIB data or even the status of the response, which is why I have some "profiles" ready in the OMCI implementation and a "Fake" status option