Re: [PATCH v14 00/15] phy: Add support for Lynx 10G SerDes

[Sean Anderson <sean.anderson@seco.com>]

On 8/21/23 15:58, Vladimir Oltean wrote:
> On Mon, Aug 21, 2023 at 02:46:53PM -0400, Sean Anderson wrote:
>> After further review, it seems the reason 28g can get away without this
>> is because there's a one-to-one mapping between protocol controllers and
>> lanes. Unfortunately, that regularity is not present for 10g.
>> 
>> --Sean
> 
> There are some things I saw in your phy-fsl-lynx-10g.c driver and device
> tree bindings that I don't understand (the concept of lane groups)

Each lane group corresponds to a phy device (struct phy). For protocols
like PCIe or XAUI which can use multiple lanes, this lets the driver
coordinate configuring all lanes at once in the correct order.

> and
> I'm not sure if they're related with what you're saying here, so if you
> could elaborate a bit more (ideally with an example) on the one-to-one
> mapping and the specific problems it causes, it would be great.

For e.g. the LS2088A, SerDes1 lanes H-A use SG1-8 and XFI1-8. SerDes2
lanes A-H use SG9-16 and XFI9-16. Each lane has its own controller, and
the mapping is 1-to-1. In the PCCRs, each controller uses the same
value, and is mapped in a regular way. So you can go directly from the
lane number to the right value to mask in the PCCR, with a very simple
translation scheme.

For e.g. the LS1046A, SerDes1 lane D uses XFI.9 (aka XFIA) and lane C
uses XFI.10 (aka XFIB). This is the opposite of how SerDes1 lanes A-D
use SGMII.9, .10, .5, and .6 (aka SGMIIA-D).

For e.g. the T4240, SerDes1 lanes A-H use sg1.5, .6, .10, .9, .1, .2,
.3, .4 (aka SGMII E, F, H, G, A, B, C, D).

For e.g. the B4860, SerDes lanes A uses sgmii1 or sgmii5 and B uses
sgmii2 or sgmii6. The MAC selected is determined based on the value
programmed into PCCR2.

While I appreciate that your hardware engineers did a better job for
28g, many 10g serdes arbitrarily map lanes to protocol controllers.
I think the mapping is too irregular to tame, and it is better to say
"if you want this configuration, program this value".

> I may be off with my understanding of the regularity you are talking about,
> but the LX2160 (and Lynx 28G block) also has multi-lane protocols like 40G,
> 100G, assuming that's what you are talking about. I haven't started yet
> working on those for the mtip_backplane driver, but I'm not currently
> seeing a problem with the architecture where a phy_device represents a
> single lane that's part of a multi-lane port, and not an entire group.

Resetting one lane in a group will reset the rest, which could confuse
the driver. Additionally, treating the lanes as one phy lets us set the
reset direction and first lane bits correctly.

> In my imagination, there are 2 cases:
> - all 4 lanes are managed by the single dpaa2-mac consumer (which has 4
>   phandles, and iterates over them with a "for" loop)
> - each of the 4 lanes is managed by the respective backplane AN/LT core,
>   and thus, there's one phandle to each lane

By doing the grouping in the driver, we also simplify the consumer
implementation. The MAC can always use a single phy, without worrying
about the actual number of lanes. This matches the hardware, since the
MAC is going to talk XGMII (or whatever) to the protocol controller
anyway.

I think it will be a lot easier to add multi-lane support with this
method because it gives the driver more information about what's going
on. The driver can control the whole configuration/reset process and the
timing.

> I sketched some dt-bindings for the second case here, so I guess it must
> be the first scenario that's somehow problematic?
> https://cas5-0-urlprotect.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fpatchwork.kernel.org%2fproject%2fnetdevbpf%2fpatch%2f20230817150644.3605105%2d9%2dvladimir.oltean%40nxp.com%2f&umid=9e644233-009e-4197-a266-5d9a85eb1148&auth=d807158c60b7d2502abde8a2fc01f40662980862-cc1d5330d84af8fa40745b165a44849db50f8a67

Yes, no issues with the second case.

--Sean