Re: [PATCH 0/3] support setting lanes

[Ajit Khaparde <ajit.khaparde@broadcom.com>]

[-- Attachment #1: Type: text/plain, Size: 6441 bytes --]

On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
>
> On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote:
> >
> >
> >
> > On 2024/3/21 16:28, Thomas Monjalon wrote:
> > > 21/03/2024 03:02, huangdengdui:
> > >>
> > >> On 2024/3/20 20:31, Ferruh Yigit wrote:
> > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote:
> > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >>>>>
> > >>>>> 12/03/2024 08:52, Dengdui Huang:
> > >>>>>> Some speeds can be achieved with different number of lanes. For example,
> > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps.
> > >>>>>> When use different lanes, the port cannot be up.
> > >>>>>
> > >>>>> I'm not sure what you are referring to.
> > >>>>> I suppose it is not PCI lanes.
> > >>>>> Please could you link to an explanation of how a port is split in lanes?
> > >>>>> Which hardware does this?
> > >>>>>
> > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports.
> > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed
> > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed
> > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed
> > >>>>
> > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides
> > >>>> the matching combination speed x lanes that works. In future if a new speed
> > >>>> is implemented with more than 8 lanes, there wouldn't be a need
> > >>>> to touch this speed command. Using separate lane command would
> > >>>> be a better alternative to support already shipped products and only new
> > >>>> drivers would consider this lanes configuration, if applicable.
> > >>>>
> > >>>
> > >>> As far as I understand, lane is related to the physical layer of the
> > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW
> > >>> block and each set called as a 'lane' and multiple lanes work together
> > >>> to achieve desired speed. (please correct me if this is wrong).
> > >>>
> > >>> Why not just configuring the speed is not enough? Why user needs to know
> > >>> the detail and configuration of the lanes?
> > >>> Will it work if driver/device configure the "speed x lane" internally
> > >>> for the requested speed?
> > >>>
> > >>> Is there a benefit to force specific lane count for a specific speed
> > >>> (like power optimization, just a wild guess)?
> > >>>
> > >>>
> > >>> And +1 for auto-negotiation if possible.
> > >>
> > >> As you said above,，multiple lanes work together to achieve desired speed.
> > >> For example, the following solutions can be used to implement 100G:
> > >> 1、Combines four 25G lanes
> > >> 2、Combines two 50G lanes
> > >> 3、A single 100G lane
> > >>
> > >> It is assumed that two ports are interconnected and the two ports support
> > >> the foregoing three solutions. But, we just configured the speed to 100G and
> > >> one port uses four 25G lanes by default and the other port uses two 50G lanes
> > >> by default, the port cannot be up. In this case, we need to configure the
> > >> two ports to use the same solutions (for example, uses two 50G lanes)
> > >> so that the ports can be up.
> > >
> > > Why this config is not OK? How do we know?
> > > Really I have a very bad feeling about this feature.
> > >
> > >
> > Sorry, I don't quite understand your question.
> > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes?
> >
> > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1]
> > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard.
>
> Typically, low-level drivers like FW configure this.
>
> For example, If FW configures, 100G port as 100GBASE-SR2 then two
> ethdev(port 0 and port1) will show up.
> Now, assume if we expose this API and Can end user configure port 1 as
> 25G lines if so,
> a) What happens to port0 and it states?
There should be no impact to port0.

> b) Will port2, port3 will show up after issuing this API(As end user
> configured 25Gx4 for 100G)? Will application needs to hotplug to get
> use ports.
No. The port count does not change. Nor does the number of PCI
functions seen by the host. Unless designed otherwise.

Changing the lane count does not change anything in physical terms.
What changes is the modulation or the signaling scheme.
The number of lanes which can be supported is determined by
the PHY itself and the cables used and needs to be negotiated appropriately
with the remote partner - which is just like using forced Ethernet Speed
instead of auto-negotiated speeds.

I tried to search for some links, but it has not been easy.
So let me try to put something down. Sorry for the long mail.

In the above example, if the 100G port is configured for 2 lanes,
the SerDes in the PHY is configured such that each lane communicates
at 50G with the remote partner.
If the port is configured for 4 lanes, the internal SerDes is programmed
to use a lower 25G signaling scheme.

Each lane provides a parallel data transmission path and are
typically implemented using multiple pairs of copper wires or optical fibers.

When the lane configuration of the Ethernet port is changed, the port speed
does not change. But the PHY FW uses a suitable modulation scheme to
achieve the total aggregate port speed.

The choice of modulation schemes include NRZ and PAM4.

Typically 100G uses four lanes at 25 Gbps each and happens to be NRZ.
For higher Ethernet speeds, the shift has been to use PAM4 signaling
which allows twice the transmission rates compared to NRZ signaling.

So the same 100G port can be configured to send signals to the remote
partner using two lanes at 50G + 50G, or just 1 lane operating at 100G.

It gets interesting as the speeds increase.
200G can be achieved with 4 lanes at 50G or 2 lanes at 100G.

PAM4 offers improved bandwidth efficiency by encoding two bits per symbol
compared to NRZ's one bit per symbol. The Ethernet standards committee
I think has already completed work on 200G speed per lane.

BTW -
PAM4 stands for 4-Level Pulse Amplitude Modulation.
NRZ stands for Non-Return-to-Zero.

Thanks
Ajit

>
> > https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9844436

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]