[net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Alexander Duyck]

The fbnic driver up till now had avoided actually reporting link as the
phylink setup only supported up to 40G configurations. This changeset is
meant to start addressing that by adding support for 50G and 100G interface
types as well as the 200GBASE-CR4 media type which we can run them over.

With that basic support added fbnic can then set those types based on the
EEPROM configuration provided by the firmware and then report those speeds
out using the information provided via the phylink call for getting the
link ksettings. This provides the basic MAC support and enables supporting
the speeds as well as configuring flow control.

After this I plan to add support for a PHY that will represent the SerDes
PHY being used to manage the link as we need a way to indicate link
training into phylink to prevent link flaps on the PCS while the SerDes is
in training, and then after that I will look at rolling support for our
PCS/PMA into the XPCS driver.

---

Alexander Duyck (6):
      net: phy: Add interface types for 50G and 100G
      fbnic: Do not consider mailbox "initialized" until we have verified fw version
      fbnic: Replace 'link_mode' with 'aui'
      fbnic: Set correct supported modes and speeds based on FW setting
      fbnic: Add support for reporting link config
      fbnic: Add support for setting/getting pause configuration


 .../net/ethernet/meta/fbnic/fbnic_ethtool.c   |   5 +
 drivers/net/ethernet/meta/fbnic/fbnic_fw.c    |  23 +++-
 drivers/net/ethernet/meta/fbnic/fbnic_fw.h    |   8 +-
 drivers/net/ethernet/meta/fbnic/fbnic_mac.c   |  89 +++++--------
 drivers/net/ethernet/meta/fbnic/fbnic_mac.h   |  21 +--
 .../net/ethernet/meta/fbnic/fbnic_netdev.c    |   2 -
 .../net/ethernet/meta/fbnic/fbnic_netdev.h    |  11 +-
 .../net/ethernet/meta/fbnic/fbnic_phylink.c   | 126 +++++++++++++++---
 drivers/net/phy/phy-core.c                    |   3 +
 drivers/net/phy/phy_caps.c                    |   9 ++
 drivers/net/phy/phylink.c                     |  13 ++
 drivers/net/phy/sfp-bus.c                     |  22 +++
 include/linux/phy.h                           |  12 ++
 include/linux/sfp.h                           |   1 +
 14 files changed, 257 insertions(+), 88 deletions(-)

--

[net-next PATCH 1/6] net: phy: Add interface types for 50G and 100G

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

Add support for 802.3cd based interface types 50GBASE-R and 100GBASE-P.
This choice in naming is based on section 135 of the 802.3-2022 IEEE
Standard.

In addition it is adding support for what I am referring to as LAUI
which is based on annex 135C of the IEEE Standard, and shares many
similarities with the 25/50G consortium. The main difference between the
two is that IEEE spec refers to LAUI as the AUI before the RS(544/514) FEC,
whereas the 25/50G use this lane and frequency combination after going
through RS(528/514), Base-R or no FEC at all.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/phy/phy-core.c |    3 +++
 drivers/net/phy/phy_caps.c |    9 +++++++++
 drivers/net/phy/phylink.c  |   13 +++++++++++++
 drivers/net/phy/sfp-bus.c  |   22 ++++++++++++++++++++++
 include/linux/phy.h        |   12 ++++++++++++
 include/linux/sfp.h        |    1 +
 6 files changed, 60 insertions(+)

diff --git a/drivers/net/phy/phy-core.c b/drivers/net/phy/phy-core.c
index e177037f9110..a8c1b60c46a4 100644
--- a/drivers/net/phy/phy-core.c
+++ b/drivers/net/phy/phy-core.c
@@ -142,6 +142,9 @@ int phy_interface_num_ports(phy_interface_t interface)
 	case PHY_INTERFACE_MODE_RXAUI:
 	case PHY_INTERFACE_MODE_XAUI:
 	case PHY_INTERFACE_MODE_1000BASEKX:
+	case PHY_INTERFACE_MODE_50GBASER:
+	case PHY_INTERFACE_MODE_LAUI:
+	case PHY_INTERFACE_MODE_100GBASEP:
 		return 1;
 	case PHY_INTERFACE_MODE_QSGMII:
 	case PHY_INTERFACE_MODE_QUSGMII:
diff --git a/drivers/net/phy/phy_caps.c b/drivers/net/phy/phy_caps.c
index 703321689726..063e4a11614c 100644
--- a/drivers/net/phy/phy_caps.c
+++ b/drivers/net/phy/phy_caps.c
@@ -345,6 +345,15 @@ unsigned long phy_caps_from_interface(phy_interface_t interface)
 		link_caps |= BIT(LINK_CAPA_40000FD);
 		break;
 
+	case PHY_INTERFACE_MODE_50GBASER:
+	case PHY_INTERFACE_MODE_LAUI:
+		link_caps |= BIT(LINK_CAPA_50000FD);
+		break;
+
+	case PHY_INTERFACE_MODE_100GBASEP:
+		link_caps |= BIT(LINK_CAPA_100000FD);
+		break;
+
 	case PHY_INTERFACE_MODE_INTERNAL:
 		link_caps |= LINK_CAPA_ALL;
 		break;
diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index 0faa3d97e06b..67218d278ce6 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -127,6 +127,9 @@ do {									\
 #endif
 
 static const phy_interface_t phylink_sfp_interface_preference[] = {
+	PHY_INTERFACE_MODE_100GBASEP,
+	PHY_INTERFACE_MODE_50GBASER,
+	PHY_INTERFACE_MODE_LAUI,
 	PHY_INTERFACE_MODE_25GBASER,
 	PHY_INTERFACE_MODE_USXGMII,
 	PHY_INTERFACE_MODE_10GBASER,
@@ -274,6 +277,13 @@ static int phylink_interface_max_speed(phy_interface_t interface)
 	case PHY_INTERFACE_MODE_XLGMII:
 		return SPEED_40000;
 
+	case PHY_INTERFACE_MODE_50GBASER:
+	case PHY_INTERFACE_MODE_LAUI:
+		return SPEED_50000;
+
+	case PHY_INTERFACE_MODE_100GBASEP:
+		return SPEED_100000;
+
 	case PHY_INTERFACE_MODE_INTERNAL:
 	case PHY_INTERFACE_MODE_NA:
 	case PHY_INTERFACE_MODE_MAX:
@@ -798,6 +808,9 @@ static int phylink_parse_mode(struct phylink *pl,
 		case PHY_INTERFACE_MODE_10GKR:
 		case PHY_INTERFACE_MODE_10GBASER:
 		case PHY_INTERFACE_MODE_XLGMII:
+		case PHY_INTERFACE_MODE_50GBASER:
+		case PHY_INTERFACE_MODE_LAUI:
+		case PHY_INTERFACE_MODE_100GBASEP:
 			caps = ~(MAC_SYM_PAUSE | MAC_ASYM_PAUSE);
 			caps = phylink_get_capabilities(pl->link_config.interface, caps,
 							RATE_MATCH_NONE);
diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c
index f13c00b5b449..949bf95db8df 100644
--- a/drivers/net/phy/sfp-bus.c
+++ b/drivers/net/phy/sfp-bus.c
@@ -275,6 +275,8 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id,
 		break;
 	case SFF8024_ECC_100GBASE_CR4:
 		phylink_set(modes, 100000baseCR4_Full);
+		phylink_set(modes, 50000baseCR2_Full);
+		__set_bit(PHY_INTERFACE_MODE_LAUI, interfaces);
 		fallthrough;
 	case SFF8024_ECC_25GBASE_CR_S:
 	case SFF8024_ECC_25GBASE_CR_N:
@@ -294,6 +296,12 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id,
 		phylink_set(modes, 2500baseT_Full);
 		__set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces);
 		break;
+	case SFF8024_ECC_200GBASE_CR4:
+		phylink_set(modes, 100000baseCR2_Full);
+		__set_bit(PHY_INTERFACE_MODE_100GBASEP, interfaces);
+		phylink_set(modes, 50000baseCR_Full);
+		__set_bit(PHY_INTERFACE_MODE_50GBASER, interfaces);
+		break;
 	default:
 		dev_warn(bus->sfp_dev,
 			 "Unknown/unsupported extended compliance code: 0x%02x\n",
@@ -357,6 +365,20 @@ EXPORT_SYMBOL_GPL(sfp_parse_support);
 phy_interface_t sfp_select_interface(struct sfp_bus *bus,
 				     const unsigned long *link_modes)
 {
+	if (phylink_test(link_modes, 100000baseCR2_Full) ||
+	    phylink_test(link_modes, 100000baseKR2_Full) ||
+	    phylink_test(link_modes, 100000baseSR2_Full))
+		return PHY_INTERFACE_MODE_100GBASEP;
+
+	if (phylink_test(link_modes, 50000baseCR_Full) ||
+	    phylink_test(link_modes, 50000baseKR_Full) ||
+	    phylink_test(link_modes, 50000baseSR_Full))
+		return PHY_INTERFACE_MODE_50GBASER;
+
+	if (phylink_test(link_modes, 50000baseCR2_Full) ||
+	    phylink_test(link_modes, 50000baseKR2_Full))
+		return PHY_INTERFACE_MODE_LAUI;
+
 	if (phylink_test(link_modes, 25000baseCR_Full) ||
 	    phylink_test(link_modes, 25000baseKR_Full) ||
 	    phylink_test(link_modes, 25000baseSR_Full))
diff --git a/include/linux/phy.h b/include/linux/phy.h
index e194dad1623d..5095f89b01c6 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -103,6 +103,9 @@ extern const int phy_basic_ports_array[3];
  * @PHY_INTERFACE_MODE_QUSGMII: Quad Universal SGMII
  * @PHY_INTERFACE_MODE_1000BASEKX: 1000Base-KX - with Clause 73 AN
  * @PHY_INTERFACE_MODE_10G_QXGMII: 10G-QXGMII - 4 ports over 10G USXGMII
+ * @PHY_INTERFACE_MODE_50GBASER: 50GBase-R - with Clause 134 FEC
+ * @PHY_INTERFACE_MODE_LAUI: 50 Gigabit Attachment Unit Interface
+ * @PHY_INTERFACE_MODE_100GBASEP: 100GBase-P - with Clause 134 FEC
  * @PHY_INTERFACE_MODE_MAX: Book keeping
  *
  * Describes the interface between the MAC and PHY.
@@ -144,6 +147,9 @@ typedef enum {
 	PHY_INTERFACE_MODE_QUSGMII,
 	PHY_INTERFACE_MODE_1000BASEKX,
 	PHY_INTERFACE_MODE_10G_QXGMII,
+	PHY_INTERFACE_MODE_50GBASER,
+	PHY_INTERFACE_MODE_LAUI,
+	PHY_INTERFACE_MODE_100GBASEP,
 	PHY_INTERFACE_MODE_MAX,
 } phy_interface_t;
 
@@ -260,6 +266,12 @@ static inline const char *phy_modes(phy_interface_t interface)
 		return "qusgmii";
 	case PHY_INTERFACE_MODE_10G_QXGMII:
 		return "10g-qxgmii";
+	case PHY_INTERFACE_MODE_50GBASER:
+		return "50gbase-r";
+	case PHY_INTERFACE_MODE_LAUI:
+		return "laui";
+	case PHY_INTERFACE_MODE_100GBASEP:
+		return "100gbase-p";
 	default:
 		return "unknown";
 	}
diff --git a/include/linux/sfp.h b/include/linux/sfp.h
index 60c65cea74f6..c2034a344e49 100644
--- a/include/linux/sfp.h
+++ b/include/linux/sfp.h
@@ -334,6 +334,7 @@ enum {
 	SFF8024_ECC_10GBASE_T_SR	= 0x1c,
 	SFF8024_ECC_5GBASE_T		= 0x1d,
 	SFF8024_ECC_2_5GBASE_T		= 0x1e,
+	SFF8024_ECC_200GBASE_CR4	= 0x40,
 };
 
 /* SFP EEPROM registers */

[net-next PATCH 2/6] fbnic: Do not consider mailbox "initialized" until we have verified fw version

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

In order for us to make use of the information in the PHY we need to verify
that the FW capabilities message has been processed. To do so we should
poll until the FW version in the capabilities message is at least at the
minimum level needed to verify that the FW can support the basic PHY config
messages.

As such this change adds logic so that we will poll the mailbox until such
time as the FW version can be populated with an acceptable value. If it
doesn't reach a sufficicient value the mailbox will be considered to have
timed out.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_fw.c |   23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
index e2368075ab8c..44876500961a 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
@@ -95,6 +95,9 @@ void fbnic_mbx_init(struct fbnic_dev *fbd)
 	/* Initialize lock to protect Tx ring */
 	spin_lock_init(&fbd->fw_tx_lock);
 
+	/* Reset FW Capabilities */
+	memset(&fbd->fw_cap, 0, sizeof(fbd->fw_cap));
+
 	/* Reinitialize mailbox memory */
 	for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++)
 		memset(&fbd->mbx[i], 0, sizeof(struct fbnic_fw_mbx));
@@ -1120,6 +1123,7 @@ void fbnic_mbx_poll(struct fbnic_dev *fbd)
 
 int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd)
 {
+	struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX];
 	unsigned long timeout = jiffies + 10 * HZ + 1;
 	int err, i;
 
@@ -1152,8 +1156,23 @@ int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd)
 	if (err)
 		goto clean_mbx;
 
-	/* Use "1" to indicate we entered the state waiting for a response */
-	fbd->fw_cap.running.mgmt.version = 1;
+	/* Poll until we get a current management firmware version, use "1"
+	 * to indicate we entered the polling state waiting for a response
+	 */
+	for (fbd->fw_cap.running.mgmt.version = 1;
+	     fbd->fw_cap.running.mgmt.version < MIN_FW_VERSION_CODE;) {
+		if (!tx_mbx->ready)
+			err = -ENODEV;
+		if (err)
+			goto clean_mbx;
+
+		msleep(20);
+		fbnic_mbx_poll(fbd);
+
+		/* set err, but wait till mgmt.version check to report it */
+		if (!time_is_after_jiffies(timeout))
+			err = -ETIMEDOUT;
+	}
 
 	return 0;
 clean_mbx:

[net-next PATCH 3/6] fbnic: Replace 'link_mode' with 'aui'

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

The way we were using "link_mode" really was more to describe the
interface between the attachment unit interface(s) we were using on the
device. Specifically the AUI is describing the modulation and the number of
lanes we are using. So we can simplify this by replacing link_mode with
aui.

In addition this change makes it so that the enum we use for the FW values
represents actual link modes that will be normally advertised by a link
partner. The general idea is to look at using this to populate
lp_advertising in the future so that we don't have to force the value and
can instead default to autoneg allowing the user to change it should they
want to force the link down or are doing some sort of manufacturing test
with a loopback plug.

Lastly we make the transition from fw_settings to aui/fec a one time thing
during phylink_init. The general idea is when we start phylink we should no
longer update the setting based on the FW and instead only allow the user
to provide the settings.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_fw.h      |    8 +-
 drivers/net/ethernet/meta/fbnic/fbnic_mac.c     |   89 +++++++++--------------
 drivers/net/ethernet/meta/fbnic/fbnic_mac.h     |   20 +++--
 drivers/net/ethernet/meta/fbnic/fbnic_netdev.c  |    2 -
 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h  |    3 -
 drivers/net/ethernet/meta/fbnic/fbnic_phylink.c |   17 ++--
 6 files changed, 58 insertions(+), 81 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
index 08bc4b918de7..08bf87c5ddf6 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
@@ -155,10 +155,10 @@ enum {
 };
 
 enum {
-	FBNIC_FW_LINK_SPEED_25R1		= 1,
-	FBNIC_FW_LINK_SPEED_50R2		= 2,
-	FBNIC_FW_LINK_SPEED_50R1		= 3,
-	FBNIC_FW_LINK_SPEED_100R2		= 4,
+	FBNIC_FW_LINK_MODE_25CR			= 1,
+	FBNIC_FW_LINK_MODE_50CR2		= 2,
+	FBNIC_FW_LINK_MODE_50CR			= 3,
+	FBNIC_FW_LINK_MODE_100CR2		= 4,
 };
 
 enum {
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
index 10e108c1fcd0..19159885b28e 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
@@ -468,15 +468,15 @@ static bool fbnic_mac_get_pcs_link_status(struct fbnic_dev *fbd)
 		return false;
 
 	/* Define the expected lane mask for the status bits we need to check */
-	switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) {
-	case FBNIC_LINK_100R2:
+	switch (fbn->aui) {
+	case FBNIC_AUI_100GAUI2:
 		lane_mask = 0xf;
 		break;
-	case FBNIC_LINK_50R1:
+	case FBNIC_AUI_50GAUI1:
 		lane_mask = 3;
 		break;
-	case FBNIC_LINK_50R2:
-		switch (fbn->fec & FBNIC_FEC_MODE_MASK) {
+	case FBNIC_AUI_LAUI2:
+		switch (fbn->fec) {
 		case FBNIC_FEC_OFF:
 			lane_mask = 0x63;
 			break;
@@ -488,7 +488,7 @@ static bool fbnic_mac_get_pcs_link_status(struct fbnic_dev *fbd)
 			break;
 		}
 		break;
-	case FBNIC_LINK_25R1:
+	case FBNIC_AUI_25GAUI:
 		lane_mask = 1;
 		break;
 	}
@@ -540,53 +540,39 @@ static bool fbnic_pcs_get_link_asic(struct fbnic_dev *fbd)
 	return link;
 }
 
-static void fbnic_pcs_get_fw_settings(struct fbnic_dev *fbd)
+static void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec)
 {
-	struct fbnic_net *fbn = netdev_priv(fbd->netdev);
-	u8 link_mode = fbn->link_mode;
-	u8 fec = fbn->fec;
-
-	/* Update FEC first to reflect FW current mode */
-	if (fbn->fec & FBNIC_FEC_AUTO) {
-		switch (fbd->fw_cap.link_fec) {
-		case FBNIC_FW_LINK_FEC_NONE:
-			fec = FBNIC_FEC_OFF;
-			break;
-		case FBNIC_FW_LINK_FEC_RS:
-			fec = FBNIC_FEC_RS;
-			break;
-		case FBNIC_FW_LINK_FEC_BASER:
-			fec = FBNIC_FEC_BASER;
-			break;
-		default:
-			return;
-		}
-
-		fbn->fec = fec;
+	/* Retrieve default speed from FW */
+	switch (fbd->fw_cap.link_speed) {
+	case FBNIC_FW_LINK_MODE_25CR:
+		*aui = FBNIC_AUI_25GAUI;
+		break;
+	case FBNIC_FW_LINK_MODE_50CR2:
+	default:
+		*aui = FBNIC_AUI_LAUI2;
+		break;
+	case FBNIC_FW_LINK_MODE_50CR:
+		*aui = FBNIC_AUI_50GAUI1;
+		*fec = FBNIC_FEC_RS;
+		return;
+	case FBNIC_FW_LINK_MODE_100CR2:
+		*aui = FBNIC_AUI_100GAUI2;
+		*fec = FBNIC_FEC_RS;
+		return;
 	}
 
-	/* Do nothing if AUTO mode is not engaged */
-	if (fbn->link_mode & FBNIC_LINK_AUTO) {
-		switch (fbd->fw_cap.link_speed) {
-		case FBNIC_FW_LINK_SPEED_25R1:
-			link_mode = FBNIC_LINK_25R1;
-			break;
-		case FBNIC_FW_LINK_SPEED_50R2:
-			link_mode = FBNIC_LINK_50R2;
-			break;
-		case FBNIC_FW_LINK_SPEED_50R1:
-			link_mode = FBNIC_LINK_50R1;
-			fec = FBNIC_FEC_RS;
-			break;
-		case FBNIC_FW_LINK_SPEED_100R2:
-			link_mode = FBNIC_LINK_100R2;
-			fec = FBNIC_FEC_RS;
-			break;
-		default:
-			return;
-		}
-
-		fbn->link_mode = link_mode;
+	/* Update FEC first to reflect FW current mode */
+	switch (fbd->fw_cap.link_fec) {
+	case FBNIC_FW_LINK_FEC_NONE:
+		*fec = FBNIC_FEC_OFF;
+		break;
+	case FBNIC_FW_LINK_FEC_RS:
+	default:
+		*fec = FBNIC_FEC_RS;
+		break;
+	case FBNIC_FW_LINK_FEC_BASER:
+		*fec = FBNIC_FEC_BASER;
+		break;
 	}
 }
 
@@ -596,9 +582,6 @@ static int fbnic_pcs_enable_asic(struct fbnic_dev *fbd)
 	wr32(fbd, FBNIC_SIG_PCS_INTR_MASK, ~0);
 	wr32(fbd, FBNIC_SIG_PCS_INTR_STS, ~0);
 
-	/* Pull in settings from FW */
-	fbnic_pcs_get_fw_settings(fbd);
-
 	return 0;
 }
 
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
index 05a591653e09..f228b12144be 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
@@ -30,22 +30,22 @@ enum {
 
 #define FBNIC_FEC_MODE_MASK	(FBNIC_FEC_AUTO - 1)
 
-/* Treat the link modes as a set of modulation/lanes bitmask:
+/* Treat the AUI modes as a modulation/lanes bitmask:
  * Bit 0: Lane Count, 0 = R1, 1 = R2
  * Bit 1: Modulation, 0 = NRZ, 1 = PAM4
- * Bit 2: Retrieve link mode from FW
+ * Bit 2: Unknown Modulation/Lane Configuration
  */
 enum {
-	FBNIC_LINK_25R1		= 0,
-	FBNIC_LINK_50R2		= 1,
-	FBNIC_LINK_50R1		= 2,
-	FBNIC_LINK_100R2	= 3,
-	FBNIC_LINK_AUTO		= 4,
+	FBNIC_AUI_25GAUI	= 0,	/* 25.7812GBd	25.78125 * 1 */
+	FBNIC_AUI_LAUI2		= 1,	/* 51.5625GBd	25.78128 * 2 */
+	FBNIC_AUI_50GAUI1	= 2,	/* 53.125GBd	53.125   * 1 */
+	FBNIC_AUI_100GAUI2	= 3,	/* 106.25GBd	53.125   * 2 */
+	FBNIC_AUI_UNKNOWN	= 4,
 };
 
-#define FBNIC_LINK_MODE_R2	(FBNIC_LINK_50R2)
-#define FBNIC_LINK_MODE_PAM4	(FBNIC_LINK_50R1)
-#define FBNIC_LINK_MODE_MASK	(FBNIC_LINK_AUTO - 1)
+#define FBNIC_AUI_MODE_R2	(FBNIC_AUI_LAUI2)
+#define FBNIC_AUI_MODE_PAM4	(FBNIC_AUI_50GAUI1)
+#define FBNIC_AUI_MODE_MASK	(FBNIC_AUI_UNKNOWN - 1)
 
 enum fbnic_sensor_id {
 	FBNIC_SENSOR_TEMP,		/* Temp in millidegrees Centigrade */
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c
index aa812c63d5af..7bd7812d9c06 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c
@@ -736,8 +736,6 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd)
 	 */
 	netdev->ethtool->wol_enabled = true;
 
-	fbn->fec = FBNIC_FEC_AUTO | FBNIC_FEC_RS;
-	fbn->link_mode = FBNIC_LINK_AUTO | FBNIC_LINK_50R2;
 	netif_carrier_off(netdev);
 
 	netif_tx_stop_all_queues(netdev);
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
index 561837e80ec8..c30c060b72e0 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
@@ -42,9 +42,8 @@ struct fbnic_net {
 	struct phylink_config phylink_config;
 	struct phylink_pcs phylink_pcs;
 
-	/* TBD: Remove these when phylink supports FEC and lane config */
+	u8 aui;
 	u8 fec;
-	u8 link_mode;
 
 	/* Cached top bits of the HW time counter for 40b -> 64b conversion */
 	u32 time_high;
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
index 860b02b22c15..edd8738c981a 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
@@ -21,23 +21,20 @@ fbnic_phylink_pcs_get_state(struct phylink_pcs *pcs, unsigned int neg_mode,
 	struct fbnic_net *fbn = fbnic_pcs_to_net(pcs);
 	struct fbnic_dev *fbd = fbn->fbd;
 
-	/* For now we use hard-coded defaults and FW config to determine
-	 * the current values. In future patches we will add support for
-	 * reconfiguring these values and changing link settings.
-	 */
-	switch (fbd->fw_cap.link_speed) {
-	case FBNIC_FW_LINK_SPEED_25R1:
+	switch (fbn->aui) {
+	case FBNIC_AUI_25GAUI:
 		state->speed = SPEED_25000;
 		break;
-	case FBNIC_FW_LINK_SPEED_50R2:
+	case FBNIC_AUI_LAUI2:
+	case FBNIC_AUI_50GAUI1:
 		state->speed = SPEED_50000;
 		break;
-	case FBNIC_FW_LINK_SPEED_100R2:
+	case FBNIC_AUI_100GAUI2:
 		state->speed = SPEED_100000;
 		break;
 	default:
-		state->speed = SPEED_UNKNOWN;
-		break;
+		state->link = 0;
+		return;
 	}
 
 	state->duplex = DUPLEX_FULL;

Re: [net-next PATCH 3/6] fbnic: Replace 'link_mode' with 'aui'

[Jakub Kicinski]

On Tue, 10 Jun 2025 07:51:28 -0700 Alexander Duyck wrote:
> -static void fbnic_pcs_get_fw_settings(struct fbnic_dev *fbd)
> +static void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec)

We get a transient "unused function" warning on this patch.
Looks like the next patch adds a declaration in the header,
let's do it here to avoid the warning?
-- 
pw-bot: cr

Re: [net-next PATCH 3/6] fbnic: Replace 'link_mode' with 'aui'

[Alexander Duyck]

On Wed, Jun 11, 2025 at 2:00 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 10 Jun 2025 07:51:28 -0700 Alexander Duyck wrote:
> > -static void fbnic_pcs_get_fw_settings(struct fbnic_dev *fbd)
> > +static void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec)
>
> We get a transient "unused function" warning on this patch.
> Looks like the next patch adds a declaration in the header,
> let's do it here to avoid the warning?
> --
> pw-bot: cr

Actually it is a bigger bug as we aren't pulling the values from the
FW as it looks like I accidentally pulled things forward a bit. I am
running some tests now with the fix. It is pretty small as I just have
to not pull out the use of the function in the MAC code until the next
patch.

[net-next PATCH 4/6] fbnic: Set correct supported modes and speeds based on FW setting

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

The fbnic driver was using the XLGMII link mode to enable phylink, however
that mode wasn't the correct one to use as the NIC doesn't actually use
XLGMII, it is using a combinations of 25G, 50G, and 100G interface modes
and configuring those via pins exposed on the PCS, MAC, and PHY interfaces.
To more accurately reflect that we should drop the uxe of XGMII and XLGMII
and instead use the correct interface types.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_mac.c     |    2 +
 drivers/net/ethernet/meta/fbnic/fbnic_mac.h     |    1 +
 drivers/net/ethernet/meta/fbnic/fbnic_phylink.c |   32 +++++++++++++++++++----
 3 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
index 19159885b28e..32be1cf849b8 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
@@ -540,7 +540,7 @@ static bool fbnic_pcs_get_link_asic(struct fbnic_dev *fbd)
 	return link;
 }
 
-static void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec)
+void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec)
 {
 	/* Retrieve default speed from FW */
 	switch (fbd->fw_cap.link_speed) {
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
index f228b12144be..f0a238ece3f5 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h
@@ -93,4 +93,5 @@ struct fbnic_mac {
 };
 
 int fbnic_mac_init(struct fbnic_dev *fbd);
+void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd, u8 *aui, u8 *fec);
 #endif /* _FBNIC_MAC_H_ */
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
index edd8738c981a..a693a9f4d5fd 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
@@ -8,6 +8,22 @@
 #include "fbnic_mac.h"
 #include "fbnic_netdev.h"
 
+static phy_interface_t fbnic_phylink_select_interface(u8 aui)
+{
+	switch (aui) {
+	case FBNIC_AUI_100GAUI2:
+		return PHY_INTERFACE_MODE_100GBASEP;
+	case FBNIC_AUI_50GAUI1:
+		return PHY_INTERFACE_MODE_50GBASER;
+	case FBNIC_AUI_LAUI2:
+		return PHY_INTERFACE_MODE_LAUI;
+	case FBNIC_AUI_25GAUI:
+		return PHY_INTERFACE_MODE_25GBASER;
+	}
+
+	return PHY_INTERFACE_MODE_NA;
+}
+
 static struct fbnic_net *
 fbnic_pcs_to_net(struct phylink_pcs *pcs)
 {
@@ -128,6 +144,7 @@ static const struct phylink_mac_ops fbnic_phylink_mac_ops = {
 int fbnic_phylink_init(struct net_device *netdev)
 {
 	struct fbnic_net *fbn = netdev_priv(netdev);
+	struct fbnic_dev *fbd = fbn->fbd;
 	struct phylink *phylink;
 
 	fbn->phylink_pcs.ops = &fbnic_phylink_pcs_ops;
@@ -135,18 +152,23 @@ int fbnic_phylink_init(struct net_device *netdev)
 	fbn->phylink_config.dev = &netdev->dev;
 	fbn->phylink_config.type = PHYLINK_NETDEV;
 	fbn->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_ASYM_PAUSE |
-					       MAC_10000FD | MAC_25000FD |
-					       MAC_40000FD | MAC_50000FD |
+					       MAC_25000FD | MAC_50000FD |
 					       MAC_100000FD;
 	fbn->phylink_config.default_an_inband = true;
 
-	__set_bit(PHY_INTERFACE_MODE_XGMII,
+	__set_bit(PHY_INTERFACE_MODE_100GBASEP,
 		  fbn->phylink_config.supported_interfaces);
-	__set_bit(PHY_INTERFACE_MODE_XLGMII,
+	__set_bit(PHY_INTERFACE_MODE_50GBASER,
 		  fbn->phylink_config.supported_interfaces);
+	__set_bit(PHY_INTERFACE_MODE_LAUI,
+		  fbn->phylink_config.supported_interfaces);
+	__set_bit(PHY_INTERFACE_MODE_25GBASER,
+		  fbn->phylink_config.supported_interfaces);
+
+	fbnic_mac_get_fw_settings(fbd, &fbn->aui, &fbn->fec);
 
 	phylink = phylink_create(&fbn->phylink_config, NULL,
-				 PHY_INTERFACE_MODE_XLGMII,
+				 fbnic_phylink_select_interface(fbn->aui),
 				 &fbnic_phylink_mac_ops);
 	if (IS_ERR(phylink))
 		return PTR_ERR(phylink);

[net-next PATCH 5/6] fbnic: Add support for reporting link config

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

This change adds some basic support for reporting the current link config
to the user via ethtool. Currently the main components reported are the
carrier status, link speed, and FEC.

For now we are handling the FEC directly as phylink doesn't have support
for it. The plan is to work on incorporating FEC support into phylink and
eventually adding the ability for us to set the FEC configuration through
phylink itself.

In addition as we don't yet have SFP or PHY support the listed modes
supported are including ones not supported by the media we are attached to.
That will hopefully be addressed once we can get the QSFP modules
supported.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c |    3 +
 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h  |    4 ++
 drivers/net/ethernet/meta/fbnic/fbnic_phylink.c |   61 +++++++++++++++++++++++
 3 files changed, 68 insertions(+)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
index 5c7556c8c4c5..1b70e63e7ada 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
@@ -1620,6 +1620,7 @@ static const struct ethtool_ops fbnic_ethtool_ops = {
 	.get_drvinfo		= fbnic_get_drvinfo,
 	.get_regs_len		= fbnic_get_regs_len,
 	.get_regs		= fbnic_get_regs,
+	.get_link		= ethtool_op_get_link,
 	.get_coalesce		= fbnic_get_coalesce,
 	.set_coalesce		= fbnic_set_coalesce,
 	.get_ringparam		= fbnic_get_ringparam,
@@ -1640,6 +1641,8 @@ static const struct ethtool_ops fbnic_ethtool_ops = {
 	.set_channels		= fbnic_set_channels,
 	.get_ts_info		= fbnic_get_ts_info,
 	.get_ts_stats		= fbnic_get_ts_stats,
+	.get_link_ksettings	= fbnic_phylink_ethtool_ksettings_get,
+	.get_fecparam		= fbnic_phylink_get_fecparam,
 	.get_eth_mac_stats	= fbnic_get_eth_mac_stats,
 };
 
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
index c30c060b72e0..943a52c77ed3 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
@@ -92,5 +92,9 @@ void fbnic_time_stop(struct fbnic_net *fbn);
 void __fbnic_set_rx_mode(struct net_device *netdev);
 void fbnic_clear_rx_mode(struct net_device *netdev);
 
+int fbnic_phylink_ethtool_ksettings_get(struct net_device *netdev,
+					struct ethtool_link_ksettings *cmd);
+int fbnic_phylink_get_fecparam(struct net_device *netdev,
+			       struct ethtool_fecparam *fecparam);
 int fbnic_phylink_init(struct net_device *netdev);
 #endif /* _FBNIC_NETDEV_H_ */
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
index a693a9f4d5fd..be6e8db328b3 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
@@ -24,6 +24,67 @@ static phy_interface_t fbnic_phylink_select_interface(u8 aui)
 	return PHY_INTERFACE_MODE_NA;
 }
 
+static void
+fbnic_phylink_set_supported_fec_modes(unsigned long *supported)
+{
+	/* The NIC can support up to 8 possible combinations.
+	 * Either 50G-CR, or 100G-CR2
+	 *   This is with RS FEC mode only
+	 * Either 25G-CR, or 50G-CR2
+	 *   This is with No FEC, RS, or Base-R
+	 */
+	if (phylink_test(supported, 100000baseCR2_Full) ||
+	    phylink_test(supported, 50000baseCR_Full))
+		phylink_set(supported, FEC_RS);
+	if (phylink_test(supported, 50000baseCR2_Full) ||
+	    phylink_test(supported, 25000baseCR_Full)) {
+		phylink_set(supported, FEC_BASER);
+		phylink_set(supported, FEC_NONE);
+		phylink_set(supported, FEC_RS);
+	}
+}
+
+int fbnic_phylink_ethtool_ksettings_get(struct net_device *netdev,
+					struct ethtool_link_ksettings *cmd)
+{
+	struct fbnic_net *fbn = netdev_priv(netdev);
+	int err;
+
+	err = phylink_ethtool_ksettings_get(fbn->phylink, cmd);
+	if (!err) {
+		unsigned long *supp = cmd->link_modes.supported;
+
+		cmd->base.port = PORT_DA;
+		cmd->lanes = (fbn->aui & FBNIC_AUI_MODE_R2) ? 2 : 1;
+
+		fbnic_phylink_set_supported_fec_modes(supp);
+	}
+
+	return err;
+}
+
+int fbnic_phylink_get_fecparam(struct net_device *netdev,
+			       struct ethtool_fecparam *fecparam)
+{
+	struct fbnic_net *fbn = netdev_priv(netdev);
+
+	if (fbn->fec & FBNIC_FEC_RS) {
+		fecparam->active_fec = ETHTOOL_FEC_RS;
+		fecparam->fec = ETHTOOL_FEC_RS;
+	} else if (fbn->fec & FBNIC_FEC_BASER) {
+		fecparam->active_fec = ETHTOOL_FEC_BASER;
+		fecparam->fec = ETHTOOL_FEC_BASER;
+	} else {
+		fecparam->active_fec = ETHTOOL_FEC_OFF;
+		fecparam->fec = ETHTOOL_FEC_OFF;
+	}
+
+	if (fbn->fec & FBNIC_FEC_AUTO || (fbn->aui & FBNIC_AUI_MODE_PAM4))
+		fecparam->fec |= ETHTOOL_FEC_AUTO;
+
+	return 0;
+}
+
 static struct fbnic_net *
 fbnic_pcs_to_net(struct phylink_pcs *pcs)
 {

[net-next PATCH 6/6] fbnic: Add support for setting/getting pause configuration

[Alexander Duyck]

From: Alexander Duyck <alexanderduyck@fb.com>

Phylink already handles most of the pieces necessary for configuring
autonegotation. With that being the case we can add support for
getting/setting pause now by just passing through the ethtool call to the
phylink code.

Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c |    2 ++
 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h  |    4 ++++
 drivers/net/ethernet/meta/fbnic/fbnic_phylink.c |   16 ++++++++++++++++
 3 files changed, 22 insertions(+)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
index 1b70e63e7ada..12e6f2483419 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
@@ -1625,6 +1625,8 @@ static const struct ethtool_ops fbnic_ethtool_ops = {
 	.set_coalesce		= fbnic_set_coalesce,
 	.get_ringparam		= fbnic_get_ringparam,
 	.set_ringparam		= fbnic_set_ringparam,
+	.get_pauseparam		= fbnic_phylink_get_pauseparam,
+	.set_pauseparam		= fbnic_phylink_set_pauseparam,
 	.get_strings		= fbnic_get_strings,
 	.get_ethtool_stats	= fbnic_get_ethtool_stats,
 	.get_sset_count		= fbnic_get_sset_count,
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
index 943a52c77ed3..a3dc85d3838b 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h
@@ -92,6 +92,10 @@ void fbnic_time_stop(struct fbnic_net *fbn);
 void __fbnic_set_rx_mode(struct net_device *netdev);
 void fbnic_clear_rx_mode(struct net_device *netdev);
 
+void fbnic_phylink_get_pauseparam(struct net_device *netdev,
+				  struct ethtool_pauseparam *pause);
+int fbnic_phylink_set_pauseparam(struct net_device *netdev,
+				 struct ethtool_pauseparam *pause);
 int fbnic_phylink_ethtool_ksettings_get(struct net_device *netdev,
 					struct ethtool_link_ksettings *cmd);
 int fbnic_phylink_get_fecparam(struct net_device *netdev,
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
index be6e8db328b3..9da6d5e2ea40 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_phylink.c
@@ -24,6 +24,22 @@ static phy_interface_t fbnic_phylink_select_interface(u8 aui)
 	return PHY_INTERFACE_MODE_NA;
 }
 
+void fbnic_phylink_get_pauseparam(struct net_device *netdev,
+				  struct ethtool_pauseparam *pause)
+{
+	struct fbnic_net *fbn = netdev_priv(netdev);
+
+	phylink_ethtool_get_pauseparam(fbn->phylink, pause);
+}
+
+int fbnic_phylink_set_pauseparam(struct net_device *netdev,
+				 struct ethtool_pauseparam *pause)
+{
+	struct fbnic_net *fbn = netdev_priv(netdev);
+
+	return phylink_ethtool_set_pauseparam(fbn->phylink, pause);
+}
+
 static void
 fbnic_phylink_set_supported_fec_modes(unsigned long *supported)
 {

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Leon Romanovsky]

On Tue, Jun 10, 2025 at 07:51:08AM -0700, Alexander Duyck wrote:
> The fbnic driver up till now had avoided actually reporting link as the
> phylink setup only supported up to 40G configurations. This changeset is
> meant to start addressing that by adding support for 50G and 100G interface
> types as well as the 200GBASE-CR4 media type which we can run them over.
> 
> With that basic support added fbnic can then set those types based on the
> EEPROM configuration provided by the firmware and then report those speeds
> out using the information provided via the phylink call for getting the
> link ksettings. This provides the basic MAC support and enables supporting
> the speeds as well as configuring flow control.
> 
> After this I plan to add support for a PHY that will represent the SerDes
> PHY being used to manage the link as we need a way to indicate link
> training into phylink to prevent link flaps on the PCS while the SerDes is
> in training, and then after that I will look at rolling support for our
> PCS/PMA into the XPCS driver.

<...>

> Alexander Duyck (6):
>       net: phy: Add interface types for 50G and 100G

<...>

>  drivers/net/phy/phy-core.c                    |   3 +
>  drivers/net/phy/phy_caps.c                    |   9 ++
>  drivers/net/phy/phylink.c                     |  13 ++
>  drivers/net/phy/sfp-bus.c                     |  22 +++
>  include/linux/phy.h                           |  12 ++
>  include/linux/sfp.h                           |   1 +
>  14 files changed, 257 insertions(+), 88 deletions(-)

when the fbnic was proposed for merge, the overall agreement was that
this driver is ok as long as no-core changes will be required for this
driver to work and now, year later, such changes are proposed here.

https://lore.kernel.org/netdev/ZhZC1kKMCKRvgIhd@nanopsycho/

Thanks

> 
> --
> 
> 

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Andrew Lunn]

On Thu, Jun 12, 2025 at 12:42:34PM +0300, Leon Romanovsky wrote:
> On Tue, Jun 10, 2025 at 07:51:08AM -0700, Alexander Duyck wrote:
> > The fbnic driver up till now had avoided actually reporting link as the
> > phylink setup only supported up to 40G configurations. This changeset is
> > meant to start addressing that by adding support for 50G and 100G interface
> > types as well as the 200GBASE-CR4 media type which we can run them over.
> > 
> > With that basic support added fbnic can then set those types based on the
> > EEPROM configuration provided by the firmware and then report those speeds
> > out using the information provided via the phylink call for getting the
> > link ksettings. This provides the basic MAC support and enables supporting
> > the speeds as well as configuring flow control.
> > 
> > After this I plan to add support for a PHY that will represent the SerDes
> > PHY being used to manage the link as we need a way to indicate link
> > training into phylink to prevent link flaps on the PCS while the SerDes is
> > in training, and then after that I will look at rolling support for our
> > PCS/PMA into the XPCS driver.
> 
> <...>
> 
> > Alexander Duyck (6):
> >       net: phy: Add interface types for 50G and 100G
> 
> <...>
> 
> >  drivers/net/phy/phy-core.c                    |   3 +
> >  drivers/net/phy/phy_caps.c                    |   9 ++
> >  drivers/net/phy/phylink.c                     |  13 ++
> >  drivers/net/phy/sfp-bus.c                     |  22 +++
> >  include/linux/phy.h                           |  12 ++
> >  include/linux/sfp.h                           |   1 +
> >  14 files changed, 257 insertions(+), 88 deletions(-)
> 
> when the fbnic was proposed for merge, the overall agreement was that
> this driver is ok as long as no-core changes will be required for this
> driver to work and now, year later, such changes are proposed here.

I would say these are natural extensions to support additional speeds
in the 'core'. We always said fbnic would be pushing the edges of the
linux core support for SFP, because all other vendors in this space
reinvent the wheel and hide it away in firmware. fbnic is different
and Linux is actually driving the hardware.

There are no algorithmic changes here, no maintenance burden, it is
following IEEE, and it will be useful for other devices in the
future. So as one of the Maintainers of this code, i'm happy to accept
this, with the usual proviso it compiles without warnings, checkpatch
clean, Russell is also happy with it, etc.

	Andrew

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Leon Romanovsky]

On Thu, Jun 12, 2025 at 02:41:33PM +0200, Andrew Lunn wrote:
> On Thu, Jun 12, 2025 at 12:42:34PM +0300, Leon Romanovsky wrote:
> > On Tue, Jun 10, 2025 at 07:51:08AM -0700, Alexander Duyck wrote:
> > > The fbnic driver up till now had avoided actually reporting link as the
> > > phylink setup only supported up to 40G configurations. This changeset is
> > > meant to start addressing that by adding support for 50G and 100G interface
> > > types as well as the 200GBASE-CR4 media type which we can run them over.
> > > 
> > > With that basic support added fbnic can then set those types based on the
> > > EEPROM configuration provided by the firmware and then report those speeds
> > > out using the information provided via the phylink call for getting the
> > > link ksettings. This provides the basic MAC support and enables supporting
> > > the speeds as well as configuring flow control.
> > > 
> > > After this I plan to add support for a PHY that will represent the SerDes
> > > PHY being used to manage the link as we need a way to indicate link
> > > training into phylink to prevent link flaps on the PCS while the SerDes is
> > > in training, and then after that I will look at rolling support for our
> > > PCS/PMA into the XPCS driver.
> > 
> > <...>
> > 
> > > Alexander Duyck (6):
> > >       net: phy: Add interface types for 50G and 100G
> > 
> > <...>
> > 
> > >  drivers/net/phy/phy-core.c                    |   3 +
> > >  drivers/net/phy/phy_caps.c                    |   9 ++
> > >  drivers/net/phy/phylink.c                     |  13 ++
> > >  drivers/net/phy/sfp-bus.c                     |  22 +++
> > >  include/linux/phy.h                           |  12 ++
> > >  include/linux/sfp.h                           |   1 +
> > >  14 files changed, 257 insertions(+), 88 deletions(-)
> > 
> > when the fbnic was proposed for merge, the overall agreement was that
> > this driver is ok as long as no-core changes will be required for this
> > driver to work and now, year later, such changes are proposed here.
> 
> I would say these are natural extensions to support additional speeds
> in the 'core'. We always said fbnic would be pushing the edges of the
> linux core support for SFP, because all other vendors in this space
> reinvent the wheel and hide it away in firmware. fbnic is different
> and Linux is actually driving the hardware.

How exactly they can hide speed declarations in the FW and still support it?

In addition, it is unclear what the last sentence means. FBNIC has FW like
any other device. The main difference is that Meta doesn't need to support
gazillion customers and can use same FW across their fleet without need
to configure it, (According to open source information).
https://docs.kernel.org/networking/device_drivers/ethernet/meta/fbnic.html

> 
> There are no algorithmic changes here, no maintenance burden, it is
> following IEEE, and it will be useful for other devices in the
> future. So as one of the Maintainers of this code, i'm happy to accept
> this, with the usual proviso it compiles without warnings, checkpatch
> clean, Russell is also happy with it, etc.

I didn't expect anything different here. This is exactly how agreed
boundaries are pushed - salami slicing.

Thanks

> 
> 	Andrew
> 

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Andrew Lunn]

> > > when the fbnic was proposed for merge, the overall agreement was that
> > > this driver is ok as long as no-core changes will be required for this
> > > driver to work and now, year later, such changes are proposed here.
> > 
> > I would say these are natural extensions to support additional speeds
> > in the 'core'. We always said fbnic would be pushing the edges of the
> > linux core support for SFP, because all other vendors in this space
> > reinvent the wheel and hide it away in firmware. fbnic is different
> > and Linux is actually driving the hardware.

> How exactly they can hide speed declarations in the FW and still support it?
 
You obviously did not spend time to look at the code and understand
what it is doing. This is used to map the EEPROM contents of the SFP
to how the PCS etc should be configured. So far, this has only been
used for speeds up to 10Gbps. This code is mostly used by SoCs, and at
the moment most SoCs inbuilt interfaces top out at 10G. fbnic is
pushing this core code to higher speeds.

You can easily hide speed declarations in firmware and still support
it because we are not talking about the ethtool API here. This is a
lower level. A FW driven device will have its own code for parsing the
SFP EEPROM and configuring the PCS etc, without needing anything from
Linux.

> In addition, it is unclear what the last sentence means. FBNIC has FW like
> any other device.

From what i have seen, it has a small amount of firmware. However,
Linux is actually controlling most of the hardware. The PCS is
controlled by linux, the SFP will be controlled by linux, the GPIOs
and I2C bus for the SFP will be controlled by Linux, all very typical
for SoC class devices, but untypical for data centre class devices. It
is also not the only one, the wangxun txgbe is doing something
similar, but at slower speeds. There is potential here for cross
pollination between these two devices/drivers.

	Andrew

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Leon Romanovsky]

On Thu, Jun 12, 2025 at 08:29:07PM +0200, Andrew Lunn wrote:
> > > > when the fbnic was proposed for merge, the overall agreement was that
> > > > this driver is ok as long as no-core changes will be required for this
> > > > driver to work and now, year later, such changes are proposed here.
> > > 
> > > I would say these are natural extensions to support additional speeds
> > > in the 'core'. We always said fbnic would be pushing the edges of the
> > > linux core support for SFP, because all other vendors in this space
> > > reinvent the wheel and hide it away in firmware. fbnic is different
> > > and Linux is actually driving the hardware.
> 
> > How exactly they can hide speed declarations in the FW and still support it?
>  
> You obviously did not spend time to look at the code and understand
> what it is doing. This is used to map the EEPROM contents of the SFP
> to how the PCS etc should be configured. So far, this has only been
> used for speeds up to 10Gbps. This code is mostly used by SoCs, and at
> the moment most SoCs inbuilt interfaces top out at 10G. fbnic is
> pushing this core code to higher speeds.
> 
> You can easily hide speed declarations in firmware and still support
> it because we are not talking about the ethtool API here. This is a
> lower level. A FW driven device will have its own code for parsing the
> SFP EEPROM and configuring the PCS etc, without needing anything from
> Linux.

Excellent, like you said, no one needs this code except fbnic, which is
exactly as was agreed - no core in/out API changes special for fbnic.

> 
> > In addition, it is unclear what the last sentence means. FBNIC has FW like
> > any other device.
> 
> From what i have seen, it has a small amount of firmware. 

Initial claim was no-FW, now we are talking about "small amount".
There is no fbnic devices in the market, FW is not open-source to
justify the last claim.

> However, Linux is actually controlling most of the hardware.

Even this claim contradicts their own press releases:
https://www.marvell.com/company/newsroom/marvell-delivers-custom-ethernet-network-interface-controller-solution-at-ocp.html

"Complete firmware control with access to all hardware internals enabling the ability
to deliver innovative customized capabilities and reduce mean time to resolve potential
issues."

Thanks

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Russell King (Oracle)]

On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> Excellent, like you said, no one needs this code except fbnic, which is
> exactly as was agreed - no core in/out API changes special for fbnic.

Rather than getting all religious about this, I'd prefer to ask a
different question.

Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
and would anyone else use them? That's the real question here, and
*not* whomever is submitting the patches or who is the first user.

So, let's look into this. According to the proposed code and comments,
PHY_INTERFACE_MODE_50GBASER is a single lane for 50G with clause 134
FEC.

LAUI seems to also be a single lane 50G, no mention about FEC (so one
assumes it has none) and the comment states it's an attachment unit
interface. It doesn't mention anything else about it.

Lastly, we have PHY_INTERFACE_MDOE_100GBASEP, which is again a single
lane running at 100G with clause 134 FEC.

I assume these are *all* IEEE 802.3 defined protocols, sadly my 2018
version of 802.3 doesn't cover them. If they are, then someday, it is
probable that someone will want these definitions.

Now, looking at the SFP code:
- We already have SFF8024_ECC_100GBASE_CR4 which is value 0x0b.
  SFF-8024 says that this is "100GBASE-CR4, 25GBASE-CR, CA-25G-L,
  50GBASE-CR2 with RS (Clause 91) FEC".

  We have a linkmode for 100GBASE-CR4 which we already use, and the
  code adds the LAUI interface. 

  Well, "50GBASE-CR2" is 50GBASE-R over two lanes over a copper cable.
  So, this doesn't fit as LAUI is as per the definition above
  extracted from the code.

- Adding SFF8024_ECC_200GBASE_CR4 which has value 0x40. SFF-8024
  says this is "50GBASE-CR, 100GBASE-CR2, 200GBASE-CR4". No other
  information, e.g. whether FEC is supported or not.

  We do have ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, which is added.
  This is added with PHY_INTERFACE_MODE_50GBASER

  Similar for ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, but with
  PHY_INTERFACE_MDOE_100GBASEP. BASE-P doesn't sound like it's
  compatible with BASE-R, but I have no information on this.

  Finally, we have ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT which
  has not been added.

So, it looks to me like some of these additions could be useful one
day, but I'm not convinced that their use with SFPs is correct.

Now, the next question is whether we have anyone else who could
possibly use this.

Well, we have the LX2160A SoC in mainline, used on SolidRun boards
that are available. These support 25GBASE-R, what could be called
50GBASE-R2 (CAUI-2), and what could be called 100GBASE-R4 (CAUI-4).

This is currently as far as my analysis has gone, and I'm starting
to fall asleep, so it's time to stop trying to comment further on
this right now. Some of what I've written may not be entirely
accurate either. I'm unlikely to have time to provide any further
comment until after the weekend.

However, I think a summary would be... the additions could be useful
but currently seem to me wrongly used.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Alexander Duyck]

On Fri, Jun 13, 2025 at 3:44 PM Russell King (Oracle)
<linux@armlinux.org.uk> wrote:
>
> On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > Excellent, like you said, no one needs this code except fbnic, which is
> > exactly as was agreed - no core in/out API changes special for fbnic.
>
> Rather than getting all religious about this, I'd prefer to ask a
> different question.
>
> Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> and would anyone else use them? That's the real question here, and
> *not* whomever is submitting the patches or who is the first user.
>
> So, let's look into this. According to the proposed code and comments,
> PHY_INTERFACE_MODE_50GBASER is a single lane for 50G with clause 134
> FEC.
>
> LAUI seems to also be a single lane 50G, no mention about FEC (so one
> assumes it has none) and the comment states it's an attachment unit
> interface. It doesn't mention anything else about it.
>
> Lastly, we have PHY_INTERFACE_MDOE_100GBASEP, which is again a single
> lane running at 100G with clause 134 FEC.
>
> I assume these are *all* IEEE 802.3 defined protocols, sadly my 2018
> version of 802.3 doesn't cover them. If they are, then someday, it is
> probable that someone will want these definitions.

Yes, they are all 802.3 protocols. The definition for these all start
with clause 131-140 in the IEEE spec.

> Now, looking at the SFP code:
> - We already have SFF8024_ECC_100GBASE_CR4 which is value 0x0b.
>   SFF-8024 says that this is "100GBASE-CR4, 25GBASE-CR, CA-25G-L,
>   50GBASE-CR2 with RS (Clause 91) FEC".
>
>   We have a linkmode for 100GBASE-CR4 which we already use, and the
>   code adds the LAUI interface.

The LAUI interface is based on the definition in clause 135 of the
IEEE spec. Basically the spec calls it out as LAUI-2 which is used for
cases where the FEC is either not present or implemented past the PCS
PMA.

>   Well, "50GBASE-CR2" is 50GBASE-R over two lanes over a copper cable.
>   So, this doesn't fit as LAUI is as per the definition above
>   extracted from the code.

In the 50G spec LAUI is a 2 lane setup that is running over NRZ, the
PAM4 variant is 50GAUI and can run over 2 lanes or 1.

> - Adding SFF8024_ECC_200GBASE_CR4 which has value 0x40. SFF-8024
>   says this is "50GBASE-CR, 100GBASE-CR2, 200GBASE-CR4". No other
>   information, e.g. whether FEC is supported or not.

Yeah, it makes it about as clear as mud. Especially since they use "R"
in the naming, but then in the IEEE spec they explain that the
100GbaseP is essentially the R form for 2 lanes or less but they
rename it with "P" to indicate PAM4 instead of NRZ.

>   We do have ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, which is added.
>   This is added with PHY_INTERFACE_MODE_50GBASER
>
>   Similar for ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, but with
>   PHY_INTERFACE_MDOE_100GBASEP. BASE-P doesn't sound like it's
>   compatible with BASE-R, but I have no information on this.
>
>   Finally, we have ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT which
>   has not been added.

I was only adding what I could test. Basically in our case we have the
CR4 cable that is running 2 links at CR2 based on the division of the
cable.

> So, it looks to me like some of these additions could be useful one
> day, but I'm not convinced that their use with SFPs is correct.

Arguably this isn't SPF, this QSFP which is divisible. For QSFP and
QSFP+ they didn't do as good a job of explaining it as they did with
QSFP-DD where the CMIS provides an explanation on how to advertise the
ability to split up the connection into mulitple links.

> Now, the next question is whether we have anyone else who could
> possibly use this.
>
> Well, we have the LX2160A SoC in mainline, used on SolidRun boards
> that are available. These support 25GBASE-R, what could be called
> 50GBASE-R2 (CAUI-2), and what could be called 100GBASE-R4 (CAUI-4).
>
> This is currently as far as my analysis has gone, and I'm starting
> to fall asleep, so it's time to stop trying to comment further on
> this right now. Some of what I've written may not be entirely
> accurate either. I'm unlikely to have time to provide any further
> comment until after the weekend.
>
> However, I think a summary would be... the additions could be useful
> but currently seem to me wrongly used.

If needed I can probably drop the 200G QSFP support for now until we
can get around to actually adding QSFP support instead of just 200G
support. I am pretty sure only the 50G could be supported by a SFP as
it would be a single lane setup, I don't know if SFP can support
multiple lanes anyway. I was mostly just following the pattern that
had enabled 100G for a SFP even though I am pretty sure that can only
be supported by a QSFP with the 25G being the breakout version of that
link.

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Russell King (Oracle)]

On Sat, Jun 14, 2025 at 09:26:00AM -0700, Alexander Duyck wrote:
> On Fri, Jun 13, 2025 at 3:44 PM Russell King (Oracle)
> <linux@armlinux.org.uk> wrote:
> >
> > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > Excellent, like you said, no one needs this code except fbnic, which is
> > > exactly as was agreed - no core in/out API changes special for fbnic.
> >
> > Rather than getting all religious about this, I'd prefer to ask a
> > different question.
> >
> > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > and would anyone else use them? That's the real question here, and
> > *not* whomever is submitting the patches or who is the first user.
> >
> > So, let's look into this. According to the proposed code and comments,
> > PHY_INTERFACE_MODE_50GBASER is a single lane for 50G with clause 134
> > FEC.
> >
> > LAUI seems to also be a single lane 50G, no mention about FEC (so one
> > assumes it has none) and the comment states it's an attachment unit
> > interface. It doesn't mention anything else about it.
> >
> > Lastly, we have PHY_INTERFACE_MDOE_100GBASEP, which is again a single
> > lane running at 100G with clause 134 FEC.
> >
> > I assume these are *all* IEEE 802.3 defined protocols, sadly my 2018
> > version of 802.3 doesn't cover them. If they are, then someday, it is
> > probable that someone will want these definitions.
> 
> Yes, they are all 802.3 protocols. The definition for these all start
> with clause 131-140 in the IEEE spec.

Good.

> > Now, looking at the SFP code:
> > - We already have SFF8024_ECC_100GBASE_CR4 which is value 0x0b.
> >   SFF-8024 says that this is "100GBASE-CR4, 25GBASE-CR, CA-25G-L,
> >   50GBASE-CR2 with RS (Clause 91) FEC".
> >
> >   We have a linkmode for 100GBASE-CR4 which we already use, and the
> >   code adds the LAUI interface.
> 
> The LAUI interface is based on the definition in clause 135 of the
> IEEE spec. Basically the spec calls it out as LAUI-2 which is used for
> cases where the FEC is either not present or implemented past the PCS
> PMA.

Please name things as per the 802.3 spec, although "based on" suggests
it isn't strictly to the 802.3 spec... I don't have a recent enough spec
to be able to check yet.

> >   Well, "50GBASE-CR2" is 50GBASE-R over two lanes over a copper cable.
> >   So, this doesn't fit as LAUI is as per the definition above
> >   extracted from the code.
> 
> In the 50G spec LAUI is a 2 lane setup that is running over NRZ, the
> PAM4 variant is 50GAUI and can run over 2 lanes or 1.

I guess what you're saying here is that 802.3 LAUI-2 is NRZ, whereas
your LAUI is PAM4 ? Please nail this down properly, is your LAUI
specific to your implementation, or is it defined by 802.3?

> > - Adding SFF8024_ECC_200GBASE_CR4 which has value 0x40. SFF-8024
> >   says this is "50GBASE-CR, 100GBASE-CR2, 200GBASE-CR4". No other
> >   information, e.g. whether FEC is supported or not.
> 
> Yeah, it makes it about as clear as mud. Especially since they use "R"
> in the naming, but then in the IEEE spec they explain that the
> 100GbaseP is essentially the R form for 2 lanes or less but they
> rename it with "P" to indicate PAM4 instead of NRZ.

So, 100GBASE-R is four lanes, 100GBASE-P is two or one lane using PAM4,
right?

> >   We do have ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, which is added.
> >   This is added with PHY_INTERFACE_MODE_50GBASER
> >
> >   Similar for ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, but with
> >   PHY_INTERFACE_MDOE_100GBASEP. BASE-P doesn't sound like it's
> >   compatible with BASE-R, but I have no information on this.
> >
> >   Finally, we have ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT which
> >   has not been added.
> 
> I was only adding what I could test. Basically in our case we have the
> CR4 cable that is running 2 links at CR2 based on the division of the
> cable.

Much of these translations are based on documentation rather than
testing, especially for speeds >10G. Adding the faster speeds helps
avoid a stream of patches as people test other modules, unless the
spec is actually wrong.

> > So, it looks to me like some of these additions could be useful one
> > day, but I'm not convinced that their use with SFPs is correct.
> 
> Arguably this isn't SPF, this QSFP which is divisible. For QSFP and
> QSFP+ they didn't do as good a job of explaining it as they did with
> QSFP-DD where the CMIS provides an explanation on how to advertise the
> ability to split up the connection into mulitple links.



> 
> > Now, the next question is whether we have anyone else who could
> > possibly use this.
> >
> > Well, we have the LX2160A SoC in mainline, used on SolidRun boards
> > that are available. These support 25GBASE-R, what could be called
> > 50GBASE-R2 (CAUI-2), and what could be called 100GBASE-R4 (CAUI-4).
> >
> > This is currently as far as my analysis has gone, and I'm starting
> > to fall asleep, so it's time to stop trying to comment further on
> > this right now. Some of what I've written may not be entirely
> > accurate either. I'm unlikely to have time to provide any further
> > comment until after the weekend.
> >
> > However, I think a summary would be... the additions could be useful
> > but currently seem to me wrongly used.
> 
> If needed I can probably drop the 200G QSFP support for now until we
> can get around to actually adding QSFP support instead of just 200G
> support. I am pretty sure only the 50G could be supported by a SFP as
> it would be a single lane setup, I don't know if SFP can support
> multiple lanes anyway.

Electrically, a SFP cage only has a fixed number of pins, and only has
sufficient to support one lane. As such, I'm not sure it makes sense to
add the >1 lane protocols to phylink_sfp_interfaces. I think if we want
to do that, we need to include the number of lanes that the cage has at
the use sites.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Alexander Duyck]

On Mon, Jun 16, 2025 at 2:49 AM Russell King (Oracle)
<linux@armlinux.org.uk> wrote:
>
> On Sat, Jun 14, 2025 at 09:26:00AM -0700, Alexander Duyck wrote:
> > On Fri, Jun 13, 2025 at 3:44 PM Russell King (Oracle)
> > <linux@armlinux.org.uk> wrote:
> > >
> > > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > > Excellent, like you said, no one needs this code except fbnic, which is
> > > > exactly as was agreed - no core in/out API changes special for fbnic.
> > >
> > > Rather than getting all religious about this, I'd prefer to ask a
> > > different question.
> > >
> > > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > > and would anyone else use them? That's the real question here, and
> > > *not* whomever is submitting the patches or who is the first user.
> > >
> > > So, let's look into this. According to the proposed code and comments,
> > > PHY_INTERFACE_MODE_50GBASER is a single lane for 50G with clause 134
> > > FEC.
> > >
> > > LAUI seems to also be a single lane 50G, no mention about FEC (so one
> > > assumes it has none) and the comment states it's an attachment unit
> > > interface. It doesn't mention anything else about it.
> > >
> > > Lastly, we have PHY_INTERFACE_MDOE_100GBASEP, which is again a single
> > > lane running at 100G with clause 134 FEC.
> > >
> > > I assume these are *all* IEEE 802.3 defined protocols, sadly my 2018
> > > version of 802.3 doesn't cover them. If they are, then someday, it is
> > > probable that someone will want these definitions.
> >
> > Yes, they are all 802.3 protocols. The definition for these all start
> > with clause 131-140 in the IEEE spec.
>
> Good.
>
> > > Now, looking at the SFP code:
> > > - We already have SFF8024_ECC_100GBASE_CR4 which is value 0x0b.
> > >   SFF-8024 says that this is "100GBASE-CR4, 25GBASE-CR, CA-25G-L,
> > >   50GBASE-CR2 with RS (Clause 91) FEC".
> > >
> > >   We have a linkmode for 100GBASE-CR4 which we already use, and the
> > >   code adds the LAUI interface.
> >
> > The LAUI interface is based on the definition in clause 135 of the
> > IEEE spec. Basically the spec calls it out as LAUI-2 which is used for
> > cases where the FEC is either not present or implemented past the PCS
> > PMA.
>
> Please name things as per the 802.3 spec, although "based on" suggests
> it isn't strictly to the 802.3 spec... I don't have a recent enough spec
> to be able to check yet.

So just to confirm, would prefer I include the 2 then? The datasheet I
was working with referred to the mode as RXLAUI and then required
clock modification to take it from 20.625Gbd to 25.78125Gbd per lane.
Since there was the clock modification aspect I thought it better to
just call it as LAUI since this is already 2 steps away from XLAUI as
we reduced the lanes and bumped up the clock.

> > >   Well, "50GBASE-CR2" is 50GBASE-R over two lanes over a copper cable.
> > >   So, this doesn't fit as LAUI is as per the definition above
> > >   extracted from the code.
> >
> > In the 50G spec LAUI is a 2 lane setup that is running over NRZ, the
> > PAM4 variant is 50GAUI and can run over 2 lanes or 1.
>
> I guess what you're saying here is that 802.3 LAUI-2 is NRZ, whereas
> your LAUI is PAM4 ? Please nail this down properly, is your LAUI
> specific to your implementation, or is it defined by 802.3?

No. The LAUI is NRZ running at 25.78125Gbd without FEC over 2 lanes.
Where things go out-of-spec would be when we add the FEC for the
25/50G consortium and then push that out over the wire. At that point
we would no longer be within the IEEE spec as the expectation is for
clause 134 FEC for all the 50G links. So the main non-spec piece would
be the use of something other than RS(544/514) which isn't used by
anybody for 50G links that I can tell as the main use seems to be
RS(528/514) to match up to 25GbaseR/100GBaseR4 and the 25/50G
consortium implementation.

50GAUI is running at 26.5625Gbd w/ RS(544,514) FEC, aka clause 134,
and either runs 2 lanes of NRZ, or 1 lane of PAM4. As per the spec,
50GAUI it is a fair bit more flexible too. It allows for running it
back and forth through PMA MUXes to either increase or decrease the
lane count.

I don't think our setup is anything proprietary. I am suspecting I
will need to carry the code into the XPCS driver as well as I think
they are supporting this mode with just XLGMII currently, and I am
pretty sure we are using the same IP. Where I think we might differ is
that we provide an option to change the interface mode via registers
that control the signals to the PCS. So we can shift the PCS up and
down depending on if we want 25R1, 50R2, 50R1, or 100R2.

> > > - Adding SFF8024_ECC_200GBASE_CR4 which has value 0x40. SFF-8024
> > >   says this is "50GBASE-CR, 100GBASE-CR2, 200GBASE-CR4". No other
> > >   information, e.g. whether FEC is supported or not.
> >
> > Yeah, it makes it about as clear as mud. Especially since they use "R"
> > in the naming, but then in the IEEE spec they explain that the
> > 100GbaseP is essentially the R form for 2 lanes or less but they
> > rename it with "P" to indicate PAM4 instead of NRZ.
>
> So, 100GBASE-R is four lanes, 100GBASE-P is two or one lane using PAM4,
> right?

Yes.

> > >   We do have ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, which is added.
> > >   This is added with PHY_INTERFACE_MODE_50GBASER
> > >
> > >   Similar for ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, but with
> > >   PHY_INTERFACE_MDOE_100GBASEP. BASE-P doesn't sound like it's
> > >   compatible with BASE-R, but I have no information on this.
> > >
> > >   Finally, we have ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT which
> > >   has not been added.
> >
> > I was only adding what I could test. Basically in our case we have the
> > CR4 cable that is running 2 links at CR2 based on the division of the
> > cable.
>
> Much of these translations are based on documentation rather than
> testing, especially for speeds >10G. Adding the faster speeds helps
> avoid a stream of patches as people test other modules, unless the
> spec is actually wrong.

Well I have been able to test the above, and from what I can tell it
is correct based on the spec but assumes multi-lane QSFP support. I
think naming is going to be the biggest challenge with all this as
there are 2 specs floating around for this section based on the 25/50G
Consortium and IEEE 802.3 so I am going to try to stick to the 802.3
for the naming, but I worry that some of the lines might get blurred
like the LAUI setup I did which from what I can tell is somewhat a nod
to the setups where this can run without FEC being enabled, or
something other than the Clause 134 FEC.

> > > So, it looks to me like some of these additions could be useful one
> > > day, but I'm not convinced that their use with SFPs is correct.
> >
> > Arguably this isn't SPF, this QSFP which is divisible. For QSFP and
> > QSFP+ they didn't do as good a job of explaining it as they did with
> > QSFP-DD where the CMIS provides an explanation on how to advertise the
> > ability to split up the connection into mulitple links.
>
>
>
> >
> > > Now, the next question is whether we have anyone else who could
> > > possibly use this.
> > >
> > > Well, we have the LX2160A SoC in mainline, used on SolidRun boards
> > > that are available. These support 25GBASE-R, what could be called
> > > 50GBASE-R2 (CAUI-2), and what could be called 100GBASE-R4 (CAUI-4).
> > >
> > > This is currently as far as my analysis has gone, and I'm starting
> > > to fall asleep, so it's time to stop trying to comment further on
> > > this right now. Some of what I've written may not be entirely
> > > accurate either. I'm unlikely to have time to provide any further
> > > comment until after the weekend.
> > >
> > > However, I think a summary would be... the additions could be useful
> > > but currently seem to me wrongly used.
> >
> > If needed I can probably drop the 200G QSFP support for now until we
> > can get around to actually adding QSFP support instead of just 200G
> > support. I am pretty sure only the 50G could be supported by a SFP as
> > it would be a single lane setup, I don't know if SFP can support
> > multiple lanes anyway.
>
> Electrically, a SFP cage only has a fixed number of pins, and only has
> sufficient to support one lane. As such, I'm not sure it makes sense to
> add the >1 lane protocols to phylink_sfp_interfaces. I think if we want
> to do that, we need to include the number of lanes that the cage has at
> the use sites.

Okay. What I can do for now is clean up the code so that we only allow
the one lane protocols in the SFP setup. I just wasn't sure if that
was the case as 100G was already supported and if I recall correctly
that required 4 lanes to support so I had just been following the
pattern from there.

I can re-introduce the multi-lane setups for QSFP when we look at
getting QSFP support enabled as that is what really introduces the
concept of partitioning the link anyway.

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Leon Romanovsky]

On Fri, Jun 13, 2025 at 11:43:58PM +0100, Russell King (Oracle) wrote:
> On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > Excellent, like you said, no one needs this code except fbnic, which is
> > exactly as was agreed - no core in/out API changes special for fbnic.
> 
> Rather than getting all religious about this, I'd prefer to ask a
> different question.
> 
> Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> and would anyone else use them? That's the real question here, and
> *not* whomever is submitting the patches or who is the first user.

Right now, the answer is no. There are no available devices in the
market which implement it.

Thanks

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Russell King (Oracle)]

On Mon, Jun 16, 2025 at 01:33:27PM +0300, Leon Romanovsky wrote:
> On Fri, Jun 13, 2025 at 11:43:58PM +0100, Russell King (Oracle) wrote:
> > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > Excellent, like you said, no one needs this code except fbnic, which is
> > > exactly as was agreed - no core in/out API changes special for fbnic.
> > 
> > Rather than getting all religious about this, I'd prefer to ask a
> > different question.
> > 
> > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > and would anyone else use them? That's the real question here, and
> > *not* whomever is submitting the patches or who is the first user.
> 
> Right now, the answer is no. There are no available devices in the
> market which implement it.

That's strictly your own opinion, I doubt it's based on facts.

LX2160A supports:
	50GBASE-R/CAUI-2(C2C/C2M) and
	100GBASE-R/CAUI-4(C2C/C2M)
modes. These are two and four lanes respectively. These date from
before clauses were added to IEEE 802.3 for these speeds, and are based
on the 25/50G Specification Rev 1.6.

https://ethernettechnologyconsortium.org/wp-content/uploads/2020/03/25G-50G-Specification-FINAL.pdf

This is 2.0 but the revision list indicates no changes since rev 1.6.

This specification states that 50GBASE-R is based upon 802.3 clause 82
operating at an overall rate of 50Gb/s, each lane operates at 25.78125GHz
with MLD4 bonding. The other references that contain 50GBASE-R are
50GBASE-R2, which indicates that it is two lanes. FEC is optional, and
may be clause 74 or 91. This is what the LX2160A implements, but I
believe without FEC - two lanes operating at this speed. The LX2160A
was initially designed as a TOR SoC.

We now need to compare this to what is being proposed, and what is now
present in 802.3 as referenced by the new modes.

Now, the definition of PHY_INTERFACE_MODE_50GBASER from Meta says that
it's "50GBase-R - with Clause 134 FEC". Let's first dive into clause 134.
This clause states that it is RS-FEC based on clause 91 but with
modifications to support operation for 50G. So this doesn't tie up with
LX2160A.

Next, let's look at PHY_INTERFACE_MODE_LAUI from Meta, and the
clarification is that it's LAUI-2 in IEEE 802.3. Clause 133 defines the
PCS. Clause 135 defines the PMA for 50GBASE-R and refers to LAUI-2 in
annex 135B. From annex 135B, LAUI-2 uses NRZ signalling over two lanes
at 25.78125GBd. This matches what's implemented by the LX2160A.

So there we have it - we have an embedded SoC that implements at least
one of the definitions that Meta have in this patch.

This disagrees with your assertion "There are no available devices in
the market which implement it."

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Leon Romanovsky]

On Mon, Jun 16, 2025 at 12:35:25PM +0100, Russell King (Oracle) wrote:
> On Mon, Jun 16, 2025 at 01:33:27PM +0300, Leon Romanovsky wrote:
> > On Fri, Jun 13, 2025 at 11:43:58PM +0100, Russell King (Oracle) wrote:
> > > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > > Excellent, like you said, no one needs this code except fbnic, which is
> > > > exactly as was agreed - no core in/out API changes special for fbnic.
> > > 
> > > Rather than getting all religious about this, I'd prefer to ask a
> > > different question.
> > > 
> > > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > > and would anyone else use them? That's the real question here, and
> > > *not* whomever is submitting the patches or who is the first user.
> > 
> > Right now, the answer is no. There are no available devices in the
> > market which implement it.
> 
> That's strictly your own opinion, I doubt it's based on facts.

I based it on wrong assumption, that device on the market in 2025 needs
to be part of upstream Linux kernel too. I was wrong, sorry about that.

Thanks

Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

[Alexander Duyck]

On Mon, Jun 16, 2025 at 3:33 AM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Fri, Jun 13, 2025 at 11:43:58PM +0100, Russell King (Oracle) wrote:
> > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > Excellent, like you said, no one needs this code except fbnic, which is
> > > exactly as was agreed - no core in/out API changes special for fbnic.
> >
> > Rather than getting all religious about this, I'd prefer to ask a
> > different question.
> >
> > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > and would anyone else use them? That's the real question here, and
> > *not* whomever is submitting the patches or who is the first user.
>
> Right now, the answer is no. There are no available devices in the
> market which implement it.
>
> Thanks

That's not really true. From what I can tell the XPCS driver is lying
about what it supports and is advertising an XLGMII interface as being
capable of doing 25, 50, and 100G speeds. Just take a look at the
xpcs_xlgmii feature set
(https://elixir.bootlin.com/linux/v6.15.1/source/drivers/net/pcs/pcs-xpcs.c#L40).
Somehow it is saying that XLGMII supports all those speeds, but only
the 40G ones really apply. The rest are a combination of most of the
modes I am calling out here.

One of the things on my list to fix is the XPCS driver as I am pretty
sure the DW IP we are using is the same IP. It is just that nobody
ever went through and enabled the correct interface modes as that
hardware is somewhat hacky in that the PMA configuration essentially
determines what the actual setup is and I suspect we may be the only
part that is allowing dynamic reconfiguration of the external pins to
allow it to switch between 10, 25, 40, 50, and 100G speeds and FEC
configurations.