* [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs
@ 2026-08-31 2:13 illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
` (9 more replies)
0 siblings, 10 replies; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch series represents the first phase. We plan to integrate it in
two phases: the first phase covers mailbox and chip configuration,
while the second phase involves net dev configuration.
Together, they will provide basic PF-based Ethernet port transmission and
reception capabilities.
After that, we will consider other features, such as ethtool support,
flow management, adminq messaging, VF support, debugfs support, etc.
To ensure compatibility, our architecture is divided into the following
layers:
1. Dev Layer (Device Layer)
The top-level business logic layer where all operations are
device-centric. Every operation is performed relative to the device
context. The intergration of base functions encompasses:
management(ctrl only for leonis pf0), network(net_dev,this time not
contained),common.
2. Dispatch Layer
The distribution from services to specific data operations is mainly
divided into two types: direct pass-through and handling by the
management PF. It shields the upper layer from the differences in
specific underlying locations.
It describes the processing locations and paths of the services.
3. Resource Layer
Handles tasks dispatched from Dispatch Layer. These tasks fall into two
categories:
3.1 Hardware control
The Resource Layer further invokes the HW Layer when hardware access is
needed, as only the HW Layer has OS-level privileges.
3.2 Software resource management
Operations like packet statistics collection that don't require hardware
access.
4. HW Layer (Hardware Layer)
Serves the Resource Layer by interacting with different hardware
chipsets.Writes to hardware registers to drive the hardware based on
Resource Layer directives.
5. Channel Layer
Handle communication between PF0(has ctrl func) and other PF,and provide
basic interaction channels.
6. Common Layer
Provides fundamental services
changes v25->v26
Link to v25:https://lore.kernel.org/netdev/20260819001117.46785-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v24->v25
Link to v24:https://lore.kernel.org/netdev/20260807044422.2356-1-illusion.wang@nebula-matrix.com/
1.AI review issues
2.Issues found by Jakub
changes v23->v24
Link to v23:https://lore.kernel.org/netdev/20260731094242.2655-1-illusion.wang@nebula-matrix.com/
1.AI review issues
2.Issues found by Jakub
changes v22->v23
Link to v22:https://lore.kernel.org/netdev/20260723040110.91410-1-illusion.wang@nebula-matrix.com/
AI review issues
chages v21->v22
Link to v21:https://lore.kernel.org/netdev/20260708064742.35391-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v20->v21
Link to v20:https://lore.kernel.org/netdev/20260630010718.4346-1-illusion.wang@nebula-matrix.com/
I have dropped the old Patch 2 titled "add our driver architecture"
Add new Patch 2 "add core driver architecture and HW layer initialization".
Split original patch8 into three separate patches as suggested
changes v19->v20
Link to v19:https://lore.kernel.org/netdev/20260617044702.2439-1-illusion.wang@nebula-matrix.com/
Starting from V20, I have dropped the old Patch 3 titled "P4
configuration invoked during chip initialization". This functionality
will be reimplemented in next phase using the request_firmware() API
with an external firmware blob.
changes v18->v19
Link to v18:https://lore.kernel.org/netdev/20260611044916.2383-1-illusion.wang@nebula-matrix.com/
changes v17->v18
Link to v17:https://lore.kernel.org/netdev/20260601093149.25905-1-illusion.wang@nebula-matrix.com/
changes v16->v17
Link to v16:https://lore.kernel.org/netdev/20260526035453.2359-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v15->v16
Link to v15:https://lore.kernel.org/netdev/20260520032950.4874-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v14->v15
Link to v14:https://lore.kernel.org/netdev/20260513011649.4404-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v13->v14
Link to v13:https://lore.kernel.org/netdev/20260428114910.2616-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v12->v13
Link to v12:https://lore.kernel.org/netdev/20260415033608.2438-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v11->v12
Link to v11:https://lore.kernel.org/netdev/20260408093739.56001-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v10->v11
Link to v10:https://lore.kernel.org/netdev/20260401022318.28550-1-illusion.wang@nebula-matrix.com/
1.Issues found by Mohsin
2.AI review issues
changes v9->v10
Link to v9:https://lore.kernel.org/netdev/20260325040048.2313-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
changes v8->v9
Link to v8:https://lore.kernel.org/netdev/20260317034533.5600-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
Changes v7→v8
Link to v7:https://lore.kernel.org/netdev/20260310120959.22015-1-illusion.wang@nebula-matrix.com/
1.Issues found by Paolo
Changes v6->v7
Link to v6:https://lore.kernel.org/netdev/20260306033451.5196-1-illusion.wang@nebula-matrix.com/
1.Issue found by Jakub
2.AI review issue
Changes v5->v6
Link to V5:https://lore.kernel.org/netdev/20260226073840.3222-1-illusion.wang@nebula-matrix.com/
1.put all standard linux includes files the .c file which needs it & others
--Andrew
2.AI review issue
Changes v4->v5
Link to V4:https://lore.kernel.org/netdev/20260206021608.85381-1-illusion.wang@nebula-matrix.com/
1.change nbl_core to nbl & change ** pointers to *pointers & others
--Andrew
2.AI review issue
Changes v3->v4
Link to v3: https://lore.kernel.org/netdev/20260123011804.31263-1-illusion.wang@nebula-matrix.com
1.cut down to part of a mini driver(mailbox and chip init)
--Jakub Kicinski Simon Horman(some sort of staged approached)
2.modify issues found by ai.
3. Reverse Christmas tree/nbl_err/devm_kfree/remove some macros/
void type to real type/others
--Andrew Lunn
4.change deprecated pci_enable_msix_range to pci_alloc_irq_vectors
5.delete service layer
6.the style of kconfig---Randy Dunlap
7.add to Documentation/networking/device_drivers/ethernet/index.rst
--Simon Horman
Changes v2 →v3
Link to v2: https://lore.kernel.org/netdev/20260109100146.63569-1-illusion.wang@nebula-matrix.com/
1.cut down to a mini driver:
delete vf support
use promisc mode to cut down flow management
drop patch15 in v2
delete adminq msg
delete abnormal handling
delete some unimportant interfaces
2.modify issues found by ai review
Changes v1->v2
Link to v1: https://lore.kernel.org/netdev/20251223035113.31122-1-illusion.wang@nebula-matrix.com/
1.Format Issues and Compilation Issues
- Paolo Abeni
2.add sysfs patch and drop coexisting patch
- Andrew Lunn
3.delete some unimportant ndo operations
4.add machine generated headers patch
5.Modify the issues found in patch1-2 and apply the same fixes to other
patches
6.modify issues found by nipa
illusion wang (10):
net/nebula-matrix: add minimum nbl build framework
net/nebula-matrix: add core driver architecture and HW layer
initialization
net/nebula-matrix: add channel layer
net/nebula-matrix: add common resource implementation
net/nebula-matrix: add intr resource implementation
net/nebula-matrix: add chip-wide hardware init/deinit implementation
net/nebula-matrix: dispatch: add control-level routing core
infrastructure
net/nebula-matrix: dispatch: implement channel RPC framework and
serialize hardware ops
net/nebula-matrix: add common/ctrl dev init/remove operation
net/nebula-matrix: add common dev start/stop operation
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +
MAINTAINERS | 10 +
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 31 +
drivers/net/ethernet/nebula-matrix/Makefile | 6 +
.../net/ethernet/nebula-matrix/nbl/Makefile | 15 +
.../nbl/nbl_channel/nbl_channel.c | 1220 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 170 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 231 ++++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 32 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 45 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 504 +++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 55 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 650 +++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 25 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 23 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 1093 +++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 352 +++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 346 +++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 12 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 77 ++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 402 ++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 150 ++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 109 ++
.../nbl/nbl_include/nbl_def_channel.h | 182 +++
.../nbl/nbl_include/nbl_def_common.h | 69 +
.../nbl/nbl_include/nbl_def_dev.h | 16 +
.../nbl/nbl_include/nbl_def_dispatch.h | 56 +
.../nbl/nbl_include/nbl_def_hw.h | 68 +
.../nbl/nbl_include/nbl_def_resource.h | 38 +
.../nbl/nbl_include/nbl_include.h | 56 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 243 ++++
36 files changed, 6350 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
--
2.47.3
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,01/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
` (8 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch adds the minimum build infrastructure:
1. Add Kconfig, Makefile and MAINTAINERS entries;
2. Add corresponding driver documentation;
3. Add PCI driver skeleton with empty stubs for nbl driver.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +++++
MAINTAINERS | 10 ++
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 31 ++++++
drivers/net/ethernet/nebula-matrix/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 13 +++
.../nbl/nbl_include/nbl_include.h | 14 +++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 100 ++++++++++++++++++
11 files changed, 211 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
index d9980c84487a..403ae9c2b06a 100644
--- a/Documentation/networking/device_drivers/ethernet/index.rst
+++ b/Documentation/networking/device_drivers/ethernet/index.rst
@@ -48,6 +48,7 @@ Contents:
meta/fbnic
microsoft/netvsc
mucse/rnpgbe
+ nebula-matrix/nbl
netronome/nfp
pensando/ionic
pensando/ionic_rdma
diff --git a/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
new file mode 100644
index 000000000000..ff38302968c5
--- /dev/null
+++ b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
@@ -0,0 +1,28 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================================================================
+Linux Base Driver for Nebula-matrix m18110-NIC/m18000-NIC family
+================================================================
+
+Overview:
+=========
+The m18110-NIC/m18000-NIC is a series of network interface cards for the Data
+Center Area.
+
+The driver supports link-speed 100GbE/25GE/10GE.
+
+m18110-NIC/m18000-NIC devices support MSI-X interrupt vector for each Tx/Rx
+queue and interrupt moderation.
+
+m18110-NIC/m18000-NIC devices support also various offload features such as
+checksum offload, Receive-Side Scaling(RSS).
+
+Support
+=======
+
+For more information about m18110-NIC/m18000-NIC, please visit the following URL:
+https://www.nebula-matrix.com/snic_s1000_en
+
+If an issue is identified with the released source code on the supported kernel
+with a supported adapter, email the specific information related to the issue to
+open@nebula-matrix.com.
diff --git a/MAINTAINERS b/MAINTAINERS
index 460cb7268845..ab55ba0da0a2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18654,6 +18654,16 @@ F: Documentation/devicetree/bindings/hwmon/nuvoton,nct7363.yaml
F: Documentation/hwmon/nct7363.rst
F: drivers/hwmon/nct7363.c
+NEBULA-MATRIX ETHERNET DRIVER (nebula-matrix)
+M: Illusion Wang <illusion.wang@nebula-matrix.com>
+M: Dimon Zhao <dimon.zhao@nebula-matrix.com>
+M: Alvin Wang <alvin.wang@nebula-matrix.com>
+M: Sam Chen <sam.chen@nebula-matrix.com>
+L: netdev@vger.kernel.org
+S: Maintained
+F: Documentation/networking/device_drivers/ethernet/nebula-matrix/
+F: drivers/net/ethernet/nebula-matrix/
+
NETCONSOLE
M: Breno Leitao <leitao@debian.org>
S: Maintained
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 8581ccba1505..c2b0161d0bec 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -130,6 +130,7 @@ config FEALNX
source "drivers/net/ethernet/ni/Kconfig"
source "drivers/net/ethernet/natsemi/Kconfig"
+source "drivers/net/ethernet/nebula-matrix/Kconfig"
source "drivers/net/ethernet/netronome/Kconfig"
source "drivers/net/ethernet/8390/Kconfig"
source "drivers/net/ethernet/nvidia/Kconfig"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index 2b1153d35b52..a306dae23bcb 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -67,6 +67,7 @@ obj-$(CONFIG_NET_VENDOR_MUCSE) += mucse/
obj-$(CONFIG_NET_VENDOR_MYRI) += myricom/
obj-$(CONFIG_FEALNX) += fealnx.o
obj-$(CONFIG_NET_VENDOR_NATSEMI) += natsemi/
+obj-$(CONFIG_NET_VENDOR_NEBULA_MATRIX) += nebula-matrix/
obj-$(CONFIG_NET_VENDOR_NETRONOME) += netronome/
obj-$(CONFIG_NET_VENDOR_NI) += ni/
obj-$(CONFIG_NET_VENDOR_NVIDIA) += nvidia/
diff --git a/drivers/net/ethernet/nebula-matrix/Kconfig b/drivers/net/ethernet/nebula-matrix/Kconfig
new file mode 100644
index 000000000000..f16e9663eaed
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Kconfig
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Nebula-matrix network device configuration
+#
+
+config NET_VENDOR_NEBULA_MATRIX
+ bool "Nebula-matrix devices"
+ default y
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Nebula-matrix cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_NEBULA_MATRIX
+
+config NBL
+ tristate "Nebula-matrix Ethernet Controller m18110/m18000 support"
+ depends on PCI && (64BIT || COMPILE_TEST) && !CPU_BIG_ENDIAN
+ help
+ This driver supports Nebula-matrix Ethernet Controller m18110/m18000
+ Family of devices.
+
+ More specific information on configuring the driver is in
+ <file:Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst>.
+
+ To compile this driver as a module, choose M here. The module
+ will be called nbl.
+
+endif # NET_VENDOR_NEBULA_MATRIX
diff --git a/drivers/net/ethernet/nebula-matrix/Makefile b/drivers/net/ethernet/nebula-matrix/Makefile
new file mode 100644
index 000000000000..42cdf2db8f0c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Nebula-matrix network device drivers.
+#
+
+obj-$(CONFIG_NBL) += nbl/
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
new file mode 100644
index 000000000000..6c14d1071c0c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Nebula Matrix Limited.
+
+obj-$(CONFIG_NBL) := nbl.o
+
+nbl-objs += nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
new file mode 100644
index 000000000000..a3d63c698aea
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CORE_H_
+#define _NBL_CORE_H_
+
+enum {
+ NBL_CAP_HAS_NET_BIT,
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
new file mode 100644
index 000000000000..16b10bcf1d36
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INCLUDE_H_
+#define _NBL_INCLUDE_H_
+
+#include <linux/types.h>
+
+/* ------ Basic definitions ------- */
+#define NBL_DRIVER_NAME "nbl"
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
new file mode 100644
index 000000000000..199626159a4c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/module.h>
+#include <linux/bits.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_core.h"
+
+static int nbl_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ return -ENODEV;
+}
+
+static void nbl_remove(struct pci_dev *pdev)
+{
+}
+
+/*
+ * PCI Device IDs for Leonis/NBL Network Controllers
+ *
+ * Vendor ID: 0x1F0F
+ * SNIC v3r1 product Device IDs range: 0x3403-0x3412
+ */
+#define NBL_VENDOR_ID 0x1F0F
+
+#define NBL_DEVICE_ID_M18110 0x3403
+#define NBL_DEVICE_ID_M18110_LX 0x3404
+#define NBL_DEVICE_ID_M18110_BASE_T 0x3405
+#define NBL_DEVICE_ID_M18110_LX_BASE_T 0x3406
+#define NBL_DEVICE_ID_M18110_OCP 0x3407
+#define NBL_DEVICE_ID_M18110_LX_OCP 0x3408
+#define NBL_DEVICE_ID_M18110_BASE_T_OCP 0x3409
+#define NBL_DEVICE_ID_M18110_LX_BASE_T_OCP 0x340a
+#define NBL_DEVICE_ID_M18000 0x340b
+#define NBL_DEVICE_ID_M18000_LX 0x340c
+#define NBL_DEVICE_ID_M18000_BASE_T 0x340d
+#define NBL_DEVICE_ID_M18000_LX_BASE_T 0x340e
+#define NBL_DEVICE_ID_M18000_OCP 0x340f
+#define NBL_DEVICE_ID_M18000_LX_OCP 0x3410
+#define NBL_DEVICE_ID_M18000_BASE_T_OCP 0x3411
+#define NBL_DEVICE_ID_M18000_LX_BASE_T_OCP 0x3412
+
+/* All below IDs belong to Leonis ASIC family, different form-factor variants,
+ * share the same hardware initialization flow without differentiated ops.
+ */
+static const struct pci_device_id nbl_id_table[] = {
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ /* required as sentinel */
+ { }
+};
+MODULE_DEVICE_TABLE(pci, nbl_id_table);
+
+static struct pci_driver nbl_driver = {
+ .name = NBL_DRIVER_NAME,
+ .id_table = nbl_id_table,
+ .probe = nbl_probe,
+ .remove = nbl_remove,
+};
+
+module_pci_driver(nbl_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Nebula Matrix Network Driver");
+MODULE_AUTHOR("Illusion Wang <illusion.wang@nebula-matrix.com>");
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,02/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
` (7 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add the fundamental driver architecture framework and Leonis hardware
layer initialization for NBL NIC family.
- nbl_adapter/nbl_core/nbl_common_info core device context
- PCI probe/remove entry and basic device capability parsing
- Leonis hardware BAR resource request and ioremap logic
This patch establishes the lowest HW layer and core infrastructure,
preparing for subsequent device implementations.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 3 +-
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 18 +++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 152 ++++++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 ++
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 32 ++++
.../nbl/nbl_include/nbl_def_common.h | 32 ++++
.../nbl/nbl_include/nbl_def_hw.h | 17 ++
.../nbl/nbl_include/nbl_include.h | 9 ++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 94 ++++++++++-
9 files changed, 369 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 6c14d1071c0c..cc060cf8bf75 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,4 +3,5 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_main.o
+nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index a3d63c698aea..1cd6587a8fcb 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -6,8 +6,26 @@
#ifndef _NBL_CORE_H_
#define _NBL_CORE_H_
+#include <linux/pci.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_common.h"
+
enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_core {
+ struct nbl_hw_mgt *hw_mgt;
+};
+
+struct nbl_adapter {
+ struct pci_dev *pdev;
+ struct nbl_core core;
+ struct nbl_common_info common;
+};
+
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param);
+void nbl_core_remove(struct nbl_adapter *adapter);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
new file mode 100644
index 000000000000..a67d1a674466
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/bitfield.h>
+#include "nbl_hw_leonis.h"
+
+/* Structure starts here, adding an op should not modify anything below */
+static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
+{
+ struct device *dev = common->dev;
+ struct nbl_hw_mgt *hw_mgt;
+
+ hw_mgt = devm_kzalloc(dev, sizeof(*hw_mgt), GFP_KERNEL);
+ if (!hw_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ hw_mgt->common = common;
+
+ return hw_mgt;
+}
+
+static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
+ const char *name)
+{
+ int bar;
+ int ret;
+
+ for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
+ if (!(mask & BIT(bar)))
+ continue;
+ ret = pcim_request_region(pdev, bar, name);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter)
+{
+ resource_size_t expect_sz = NBL_MEM_BAR_TOTAL_SIZE;
+ struct nbl_common_info *common = &adapter->common;
+ struct pci_dev *pdev = common->pdev;
+ struct nbl_hw_mgt *hw_mgt = NULL;
+ resource_size_t bar_len;
+ u32 bar_mask;
+ int ret;
+
+ hw_mgt = nbl_hw_setup_hw_mgt(common);
+ if (IS_ERR(hw_mgt)) {
+ ret = PTR_ERR(hw_mgt);
+ goto setup_mgt_fail;
+ }
+ bar_mask = BIT(NBL_MEMORY_BAR) | BIT(NBL_MAILBOX_BAR);
+ ret = nbl_pcim_request_selected_bars(pdev, bar_mask, NBL_DRIVER_NAME);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Request memory bar failed, err = %d\n",
+ ret);
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MEMORY_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MEMORY_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MEMORY BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (common->has_ctrl) {
+ /*
+ * Hardware layout: MEMORY BAR total size is 64M.
+ * The tail NBL_RDMA_NOTIFY_LEN bytes of the 64M BAR are
+ * reserved exclusively for RDMA notify hardware.
+ * Ethernet driver must avoid mapping this reserved tail
+ * to prevent x86 PAT aliasing conflict between eth net
+ * mapping and RDMA driver WC mapping. Mapping starts
+ * at BAR offset 0.
+ *
+ * Skip trailing NBL_RDMA_NOTIFY_LEN bytes at BAR tail.
+ * Round size down to page boundary to avoid ioremap
+ * rounding up and accidentally including RDMA reserved
+ * region when PAGE_SIZE > 8KiB.
+ */
+ if (bar_len < NBL_MEM_BAR_TOTAL_SIZE) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pr smaller than expected %pr\n",
+ &bar_len, &expect_sz);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->hw_size = PAGE_ALIGN_DOWN(NBL_MEM_BAR_TOTAL_SIZE -
+ NBL_RDMA_NOTIFY_LEN);
+ hw_mgt->hw_addr =
+ pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ } else {
+ if (bar_len < NBL_REG_NET_ONLY_LEN) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pr too small for net only reg space\n",
+ &bar_len);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->hw_size = NBL_REG_NET_ONLY_LEN;
+ hw_mgt->hw_addr = pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ }
+ if (!hw_mgt->hw_addr) {
+ dev_err(&pdev->dev, "MEMORY BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MAILBOX_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MAILBOX_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MAILBOX BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (bar_len == 0) {
+ dev_err(&pdev->dev, "MAILBOX BAR length is zero\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->mailbox_bar_hw_addr = pcim_iomap(pdev, NBL_MAILBOX_BAR,
+ bar_len);
+ if (!hw_mgt->mailbox_bar_hw_addr) {
+ dev_err(&pdev->dev, "MAILBOX BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->mailbox_bar_size = bar_len;
+
+ adapter->core.hw_mgt = hw_mgt;
+
+ return 0;
+
+setup_mgt_fail:
+ return ret;
+}
+
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter)
+{
+ /* All BAR mappings & PCI regions are managed by pcim/devres,
+ * no manual iounmap / release required
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
new file mode 100644
index 000000000000..a3397f5be353
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_LEONIS_H_
+#define _NBL_HW_LEONIS_H_
+
+#include <linux/types.h>
+
+#include "../../nbl_include/nbl_include.h"
+#include "../nbl_hw_reg.h"
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
new file mode 100644
index 000000000000..5e9823e01d39
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_REG_H_
+#define _NBL_HW_REG_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_MEMORY_BAR 0
+#define NBL_MAILBOX_BAR 2
+#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
+#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+/*
+ * PCI MEMORY BAR total size: 64MiB.
+ */
+#define NBL_MEM_BAR_TOTAL_SIZE (64ULL << 20)
+
+struct nbl_hw_mgt {
+ struct nbl_common_info *common;
+ u8 __iomem *hw_addr;
+ u8 __iomem *mailbox_bar_hw_addr;
+ resource_size_t hw_size;
+ resource_size_t mailbox_bar_size;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
new file mode 100644
index 000000000000..da30244fe75d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_COMMON_H_
+#define _NBL_DEF_COMMON_H_
+
+#include <linux/types.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include "nbl_include.h"
+
+struct nbl_common_info {
+ struct pci_dev *pdev;
+ struct device *dev;
+ u32 msg_enable;
+ u16 vsi_id;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 eth_num;
+
+ u8 function;
+ u8 devid;
+ u8 bus;
+ u8 hw_bus;
+
+ u8 has_ctrl;
+ u8 has_net;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
new file mode 100644
index 000000000000..ecbf440e4366
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_HW_H_
+#define _NBL_DEF_HW_H_
+
+#include <linux/types.h>
+
+struct nbl_hw_mgt;
+struct nbl_adapter;
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter);
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 16b10bcf1d36..14e7b19f9a4c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,5 +10,14 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+struct nbl_func_caps {
+ u32 has_ctrl:1;
+ u32 has_net:1;
+ u32 rsv:30;
+};
+
+struct nbl_init_param {
+ struct nbl_func_caps caps;
+};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 199626159a4c..f2552bc73293 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,16 +8,108 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param)
+{
+ struct nbl_common_info *common;
+ struct nbl_adapter *adapter;
+ int ret;
+
+ adapter = devm_kzalloc(&pdev->dev, sizeof(*adapter), GFP_KERNEL);
+ if (!adapter)
+ return ERR_PTR(-ENOMEM);
+
+ adapter->pdev = pdev;
+ common = &adapter->common;
+
+ common->pdev = pdev;
+ common->dev = &pdev->dev;
+ common->has_ctrl = param->caps.has_ctrl;
+ common->has_net = param->caps.has_net;
+ common->function = PCI_FUNC(pdev->devfn);
+ common->devid = PCI_SLOT(pdev->devfn);
+ common->bus = pdev->bus->number;
+
+ ret = nbl_hw_init_leonis(adapter);
+ if (ret)
+ goto hw_init_fail;
+
+ return adapter;
+hw_init_fail:
+ return ERR_PTR(ret);
+}
+
+void nbl_core_remove(struct nbl_adapter *adapter)
+{
+ nbl_hw_remove_leonis(adapter);
+}
+
+static void nbl_get_func_param(struct pci_dev *pdev, kernel_ulong_t driver_data,
+ struct nbl_init_param *param)
+{
+ param->caps.has_net = !!(driver_data & BIT(NBL_CAP_HAS_NET_BIT));
+
+ /*
+ * Hardware fixed rule: physical PF0 is the only management PF with
+ * global ctrl capability. All PFs share identical PCI device ID, so
+ * distinguish control PF via physical function ID.
+ *
+ * Hardware & firmware design FORBID passing any PF through to virtual
+ * machines, there is no scenario where a non-management PF appears
+ * as Func 0 inside guest. Thus using PCI_FUNC(pdev->devfn) to identify
+ * control PF is safe on our platform.
+ */
+ if ((PCI_FUNC(pdev->devfn) == 0) && !pdev->is_virtfn)
+ param->caps.has_ctrl = 1;
+}
+
static int nbl_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
- return -ENODEV;
+ struct nbl_init_param param = { { 0 } };
+ struct device *dev = &pdev->dev;
+ struct nbl_adapter *adapter;
+ int err;
+
+ err = pcim_enable_device(pdev);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to enable PCI dev, err=%d\n", err);
+ return err;
+ }
+
+ nbl_get_func_param(pdev, id->driver_data, ¶m);
+ /* never return fail when DMA_BIT_MASK(64) */
+ dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+
+ pci_set_master(pdev);
+
+ adapter = nbl_core_init(pdev, ¶m);
+ if (IS_ERR(adapter)) {
+ dev_err(dev, "Nbl adapter init fail: %pe\n", adapter);
+ err = PTR_ERR(adapter);
+ goto adapter_init_err;
+ }
+ pci_set_drvdata(pdev, adapter);
+ return 0;
+adapter_init_err:
+ pci_clear_master(pdev);
+ return err;
}
static void nbl_remove(struct pci_dev *pdev)
{
+ struct nbl_adapter *adapter = pci_get_drvdata(pdev);
+
+ if (!adapter)
+ return;
+ pci_set_drvdata(pdev, NULL);
+ nbl_core_remove(adapter);
+
+ pci_clear_master(pdev);
}
/*
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 03/10] net/nebula-matrix: add channel layer
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,03/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
` (6 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add generic channel management layer for Nebula Matrix network adapter,
which serves as the core inter-PF communication component, providing
standardized message transmission, queue management and hardware
abstraction capabilities.
The new channel layer implements mailbox-based inter-PF(PF0 <-> other PFs)
communication framework with the following core features:
1. Dynamic Message Handling Framework
- Implement hash-table-based message handler management with O(1) lookup by
message type, supporting dynamic message handler registration
- Support two transmission modes: fire-and-forget and synchronous send with
ACK waiting
- Implement dedicated ACK message processing path to match synchronous
request-response semantics
- Add TX slot concurrency control, return -EAGAIN when all outstanding
slots are occupied under high load
- Dual data transmission format: small embedded payload in TX descriptor,
large payload via external DMA buffer
- Support full cleanup of message handlers during driver teardown
2. Mailbox Queue Management
- Initialize TX/RX descriptor rings and data buffers via coherent DMA
allocation and devm managed memory
- Complete queue lifecycle management: hardware queue init/config/stop/
teardown
- Support dual RX processing modes: interrupt-driven and mailbox polling
path, offload heavy RX descriptor cleanup work to dedicated workqueue
to reduce interrupt latency
- Implement inflight TX traffic draining and pending ACK request abortion
during queue teardown
- Queue resource lifecycle design: DMA buffers are allocated once in probe
phase and released automatically by devm on driver remove, dynamic
runtime queue reinit is not supported
- Maintain accurate RX ring empty/full state differentiation via reserved
hole entry mechanism, avoiding ring overflow and repeated reception
3. Hardware Abstraction Layer(HW OPS)
- Abstract hardware-specific mailbox operations into independent hw_ops
layer, decouple channel logic from hardware implementation
- Provide hardware queue configuration, tail pointer doorbell update, PF
mailbox routing table configuration interfaces
- Add register lock protection for hardware register read/write to
ensure concurrent access safety
4. Common Utility & Infrastructure
- Implement generic thread-safe hash table management for message handler
storage
- Create dedicated device-bound workqueue for RX cleanup task
scheduling
- Supplement core data structures, state management and bitmask state
control interfaces
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 4 +-
.../nbl/nbl_channel/nbl_channel.c | 1220 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 170 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 209 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 32 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 7 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 179 +++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 56 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 34 +
.../nbl/nbl_include/nbl_def_channel.h | 126 ++
.../nbl/nbl_include/nbl_def_common.h | 18 +
.../nbl/nbl_include/nbl_def_hw.h | 33 +
.../nbl/nbl_include/nbl_include.h | 3 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 7 +
14 files changed, 2097 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index cc060cf8bf75..04e1aa1fb4bd 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,5 +3,7 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+nbl-objs += nbl_common/nbl_common.o \
+ nbl_channel/nbl_channel.o \
+ nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
new file mode 100644
index 000000000000..c29ff76a75ae
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
@@ -0,0 +1,1220 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/bitfield.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/dma-mapping.h>
+#include <linux/atomic.h>
+#include <linux/wait.h>
+#include "nbl_channel.h"
+
+static int nbl_chan_add_msg_handler(struct nbl_channel_mgt *chan_mgt,
+ u16 msg_type, nbl_chan_resp func,
+ void *priv)
+{
+ struct nbl_chan_msg_node_data handler = { 0 };
+ int ret;
+
+ handler.func = func;
+ handler.priv = priv;
+ ret = nbl_common_alloc_hash_node(chan_mgt->handle_hash_tbl, &msg_type,
+ &handler, NULL);
+
+ return ret;
+}
+
+static int nbl_chan_init_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_hash_tbl_key tbl_key = { 0 };
+
+ tbl_key.dev = common->dev;
+ tbl_key.key_size = sizeof(u16);
+ tbl_key.data_size = sizeof(struct nbl_chan_msg_node_data);
+ tbl_key.bucket_size = NBL_CHAN_HANDLER_TBL_BUCKET_SIZE;
+
+ chan_mgt->handle_hash_tbl = nbl_common_init_hash_table(&tbl_key);
+ if (!chan_mgt->handle_hash_tbl)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void nbl_chan_remove_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ if (!chan_mgt->handle_hash_tbl)
+ return;
+ nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl);
+ chan_mgt->handle_hash_tbl = NULL;
+}
+
+static void nbl_chan_init_queue_param(struct nbl_chan_info *chan_info,
+ u16 num_txq_entries, u16 num_rxq_entries,
+ u16 txq_buf_size, u16 rxq_buf_size)
+{
+ chan_info->num_txq_entries = num_txq_entries;
+ chan_info->num_rxq_entries = num_rxq_entries;
+ chan_info->txq_buf_size = txq_buf_size;
+ chan_info->rxq_buf_size = rxq_buf_size;
+ atomic_set(&chan_info->inflight_tx_cnt, 0);
+ WRITE_ONCE(chan_info->shutdn, false);
+ WRITE_ONCE(chan_info->active, false);
+ WRITE_ONCE(chan_info->wait_head_index, 0);
+ memset(chan_info->state, 0, sizeof(chan_info->state));
+ init_waitqueue_head(&chan_info->inflight_wait);
+}
+
+static int nbl_chan_init_tx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_txq_entries * sizeof(struct nbl_chan_tx_desc);
+ u16 i;
+
+ txq->desc.tx_desc =
+ dmam_alloc_coherent(dev, size, &txq->dma, GFP_KERNEL);
+ if (!txq->desc.tx_desc)
+ return -ENOMEM;
+
+ chan_info->wait = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*chan_info->wait), GFP_KERNEL);
+ if (!chan_info->wait)
+ return -ENOMEM;
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ init_waitqueue_head(&chan_info->wait[i].wait_queue);
+ WRITE_ONCE(chan_info->wait[i].status, NBL_MBX_STATUS_IDLE);
+ WRITE_ONCE(chan_info->wait[i].acked, 0);
+ WRITE_ONCE(chan_info->wait[i].ack_data, NULL);
+ WRITE_ONCE(chan_info->wait[i].ack_data_len, 0);
+ WRITE_ONCE(chan_info->wait[i].ack_err, 0);
+ WRITE_ONCE(chan_info->wait[i].msg_type, 0);
+ WRITE_ONCE(chan_info->wait[i].msg_index, 0);
+ WRITE_ONCE(chan_info->wait[i].dstid, 0);
+ }
+
+ txq->buf = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*txq->buf), GFP_KERNEL);
+ if (!txq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_rx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_rxq_entries * sizeof(struct nbl_chan_rx_desc);
+
+ rxq->desc.rx_desc =
+ dmam_alloc_coherent(dev, size, &rxq->dma, GFP_KERNEL);
+ if (!rxq->desc.rx_desc) {
+ dev_err_ratelimited(dev,
+ "Allocate DMA for chan rx descriptor ring failed\n");
+ return -ENOMEM;
+ }
+
+ rxq->buf = devm_kcalloc(dev, chan_info->num_rxq_entries,
+ sizeof(*rxq->buf), GFP_KERNEL);
+ if (!rxq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_init_tx_queue(common, chan_info);
+ if (err)
+ return err;
+
+ err = nbl_chan_init_rx_queue(common, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_config_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info, bool tx)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ struct nbl_chan_ring *ring;
+ dma_addr_t addr;
+ int size_bwid;
+
+ if (tx)
+ ring = &chan_info->txq;
+ else
+ ring = &chan_info->rxq;
+ addr = ring->dma;
+ if (tx) {
+ size_bwid = ilog2(chan_info->num_txq_entries);
+ hw_ops->config_mailbox_txq(p, addr, size_bwid);
+ } else {
+ size_bwid = ilog2(chan_info->num_rxq_entries);
+ hw_ops->config_mailbox_rxq(p, addr, size_bwid);
+ }
+}
+
+static int nbl_chan_alloc_all_tx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ buf = &txq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->txq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan tx queue failed\n");
+ return -ENOMEM;
+ }
+ }
+
+ txq->next_to_clean = 0;
+ txq->next_to_use = 0;
+ txq->tail_ptr = 0;
+
+ return 0;
+}
+
+static void nbl_chan_cfg_qinfo_map_table(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ u32 pf_mask = 0;
+ u8 func_id;
+
+ /*
+ * k_pf_mask rule: bit N == 0 means PF#N enabled, bit N == 1 masked out.
+ * Program mailbox QINFO entry for each hardware-active PF func_id.
+ *
+ * Note: This loop iterates over raw hardware PF func_id.
+ * Upper resource initialization nbl_res_init_pf_num() enforces
+ * product constraints: only 1/2/4 contiguous PFs(PF0 / PF0~1 / PF0~3)
+ * are allowed. Non-contiguous or unsupported PF count will be rejected
+ * before reaching this function.
+ */
+ hw_ops->get_host_pf_mask(p, &pf_mask);
+ for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
+ if (!(pf_mask & (1 << func_id)))
+ hw_ops->cfg_mailbox_qinfo(p, func_id, bus,
+ devid, func_id);
+ }
+}
+
+static int nbl_chan_alloc_all_rx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_rx_desc *desc;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_rxq_entries; i++) {
+ buf = &rxq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->rxq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan rx queue failed\n");
+ goto err;
+ }
+ }
+
+ desc = rxq->desc.rx_desc;
+ /*
+ * Initially leave one RX descriptor unused so that
+ * next_to_clean and next_to_use can distinguish an empty
+ * ring from a full ring.
+ *
+ * The unused slot is replenished as RX descriptors are
+ * consumed and recycled.
+ */
+ for (i = 0; i < chan_info->num_rxq_entries - 1; i++) {
+ buf = &rxq->buf[i];
+ desc[i].buf_addr = cpu_to_le64(buf->pa);
+ desc[i].buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+ desc[i].flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+ }
+
+ rxq->next_to_clean = 0;
+ rxq->next_to_use = chan_info->num_rxq_entries - 1;
+ rxq->tail_ptr = chan_info->num_rxq_entries - 1;
+
+ return 0;
+err:
+ return -ENOMEM;
+}
+
+static int nbl_chan_alloc_all_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_alloc_all_tx_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_rx_bufs(chan_mgt, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_stop_queue(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+
+ hw_ops->stop_mailbox_rxq(chan_mgt->hw_ops_tbl->priv);
+ hw_ops->stop_mailbox_txq(chan_mgt->hw_ops_tbl->priv);
+}
+
+static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_chan_waitqueue_head *wait_head;
+ struct work_struct *task;
+ int ret = 0;
+ u16 i;
+
+ if (!READ_ONCE(chan_info->active)) {
+ dev_warn(chan_mgt->common->dev, "channel not active, skip duplicate teardown\n");
+ return 0;
+ }
+ /*
+ * Step1:
+ * block new sender
+ */
+ mutex_lock(&chan_info->state_lock);
+ WRITE_ONCE(chan_info->shutdn, true);
+ task = READ_ONCE(chan_info->clean_task);
+ WRITE_ONCE(chan_info->clean_task, NULL);
+ mutex_unlock(&chan_info->state_lock);
+ /*
+ * Step2:
+ * abort pending ACK waiters
+ */
+
+ mutex_lock(&chan_info->pending_lock);
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait_head = &chan_info->wait[i];
+ /* Only wake threads that are actually waiting */
+ if (READ_ONCE(wait_head->status) == NBL_MBX_STATUS_WAITING) {
+ /* Update all status fields first */
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->ack_err, (s32)-EIO);
+ /* Ensure status visible before acked flag */
+ smp_wmb();
+ WRITE_ONCE(wait_head->acked, 1);
+ wake_up(&wait_head->wait_queue);
+ }
+ }
+ mutex_unlock(&chan_info->pending_lock);
+ /*
+ * Step3:
+ * wait all sender exit
+ *
+ * Drain strategy mirrors mlx5 command interface teardown:
+ * set shutdown flag first, abort all pending waiters, then
+ * block until inflight_tx_cnt reaches zero.
+ *
+ * A timeout here is treated as an exceptional condition rather
+ * than a fatal error, following the same rationale as mlx5:
+ * - shutdn is already set, so every sender path observes it
+ * at its next checkpoint and exits;
+ * - each sender has its own bounded timeout (3s ACK wait,
+ * 12ms max TX ring poll), all far shorter than this 5s
+ * drain window, so stalling beyond 5s should never happen
+ * unless a sender is blocked inside an unrecoverable MMIO
+ * access (hardwed hardware);
+ * - in that hardware-dead case proceeding with teardown cannot
+ * make the situation worse, and avoids hanging rmmod forever.
+ */
+ ret = wait_event_timeout(chan_info->inflight_wait,
+ atomic_read(&chan_info->inflight_tx_cnt) == 0,
+ msecs_to_jiffies(5000));
+
+ if (!ret) {
+ dev_warn(chan_mgt->common->dev,
+ "teardown: inflight tx drain timeout\n");
+ ret = -ETIMEDOUT;
+ }
+
+ /* After all TX drained, stop hardware queue */
+ nbl_chan_stop_queue(chan_mgt);
+
+ /* All send paths drained, safely cancel cleanup work */
+ if (task)
+ cancel_work_sync(task);
+ WRITE_ONCE(chan_info->active, false);
+ return ret;
+}
+
+static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ int err;
+
+ if (READ_ONCE(chan_info->active)) {
+ dev_warn(common->dev, "channel already active, reject duplicate setup\n");
+ return -EBUSY;
+ }
+ nbl_chan_init_queue_param(chan_info, NBL_CHAN_QUEUE_LEN,
+ NBL_CHAN_QUEUE_LEN, NBL_CHAN_BUF_LEN,
+ NBL_CHAN_BUF_LEN);
+ err = nbl_chan_init_queue(common, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
+ nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+ WRITE_ONCE(chan_info->active, true);
+ return 0;
+}
+
+static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_tx_param *param)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct nbl_chan_tx_desc *tx_desc =
+ NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_use);
+ struct nbl_chan_buf *tx_buf =
+ NBL_CHAN_TX_RING_TO_BUF(txq, txq->next_to_use);
+
+ if (param->arg_len > NBL_CHAN_BUF_LEN - sizeof(*tx_desc))
+ return -EINVAL;
+
+ tx_desc->dstid = cpu_to_le16(param->dstid);
+ tx_desc->msg_type = cpu_to_le16(param->msg_type);
+ tx_desc->msgid = cpu_to_le16(param->msgid);
+
+ /*
+ * srcid field is filled by mailbox hardware after peer receives this
+ * packet, driver producer never writes srcid; reused descriptor slots
+ * will contain stale srcid value temporarily until hardware overwrites
+ * it.
+ */
+ if (param->arg_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ if (param->arg)
+ memcpy(tx_buf->va, param->arg, param->arg_len);
+ tx_desc->buf_addr = cpu_to_le64(tx_buf->pa);
+ tx_desc->buf_len = cpu_to_le16(param->arg_len);
+ tx_desc->data_len = 0;
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ } else {
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ memset(&tx_desc->buf_addr, 0, sizeof(tx_desc->buf_addr));
+ if (param->arg && param->arg_len > 0)
+ memcpy(tx_desc->data, param->arg, param->arg_len);
+ tx_desc->buf_len = 0;
+ tx_desc->data_len = cpu_to_le16(param->arg_len);
+ }
+ /* Ensure descriptor data visible to device before AVAIL flag */
+ dma_wmb();
+ tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
+
+ txq->next_to_use =
+ NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
+ txq->tail_ptr++;
+
+ return 0;
+}
+
+static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ int max_retries = NBL_CHAN_TX_WAIT_TIMES;
+ struct nbl_chan_tx_desc *tx_desc;
+ int retry_count = 0;
+ u16 msg_type;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ txq->tail_ptr, NBL_MB_TX_QID);
+
+ tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_clean);
+ /*
+ * Poll for HW to mark descriptor as USED.
+ * Mailbox is a low-speed control channel for management commands.
+ * We avoid enabling dedicated per-TX interrupt for single control
+ * message to reduce interrupt overhead, so use bounded polling
+ * with small delay instead.
+ */
+ while (retry_count < max_retries) {
+ if (READ_ONCE(chan_info->shutdn))
+ return -ESHUTDOWN;
+
+ /* Order descriptor read after hardware DMA completion */
+ dma_rmb();
+ if (le16_to_cpu(READ_ONCE(tx_desc->flags)) &
+ BIT(NBL_CHAN_TX_DESC_USED)) {
+ break;
+ }
+
+ retry_count++;
+ if (retry_count == max_retries) {
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_err_ratelimited(dev, "chan send msg type: %d timeout\n",
+ msg_type);
+ txq->next_to_clean = txq->next_to_use;
+ return -ETIMEDOUT;
+ }
+ usleep_range(NBL_CHAN_TX_WAIT_US, NBL_CHAN_TX_WAIT_US_MAX);
+ }
+
+ txq->next_to_clean = txq->next_to_use;
+
+ return 0;
+}
+
+static void nbl_chan_recv_ack_msg(void *priv, u16 srcid, u16 msgid, void *data,
+ u32 data_len)
+{
+ struct nbl_channel_mgt *chan_mgt = (struct nbl_channel_mgt *)priv;
+ struct nbl_chan_waitqueue_head *wait_head = NULL;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ u16 w_dstid, w_msgtype, w_msgidx;
+ u32 *payload = data;
+ u16 ack_msgtype = 0;
+ u16 ack_msgid = 0;
+ u32 ack_datalen;
+ void *ack_data;
+ u32 copy_len;
+ int w_status;
+
+ if (READ_ONCE(chan_info->shutdn))
+ return;
+ if (data_len > NBL_CHAN_BUF_LEN ||
+ data_len < NBL_CHAN_ACK_HEAD_LEN * sizeof(u32)) {
+ dev_err_ratelimited(dev, "Invalid ACK data_len: %u\n",
+ data_len);
+ return;
+ }
+ ack_datalen = data_len - NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ ack_msgtype = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_TYPE_POS));
+ ack_msgid = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_ID_POS));
+ if (FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid) >=
+ chan_info->num_txq_entries) {
+ dev_err_ratelimited(dev, "chan recv msg id: %u err\n",
+ ack_msgid);
+ return;
+ }
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid)];
+
+ mutex_lock(&chan_info->pending_lock);
+
+ /* Cache repeated READ_ONCE values */
+ w_dstid = READ_ONCE(wait_head->dstid);
+ w_status = READ_ONCE(wait_head->status);
+ w_msgtype = READ_ONCE(wait_head->msg_type);
+ w_msgidx = READ_ONCE(wait_head->msg_index);
+
+ if (srcid != w_dstid) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev, "ACK srcid=%u != dstid=%u, rejecting\n",
+ srcid, w_dstid);
+ return;
+ }
+ if (w_status != NBL_MBX_STATUS_WAITING) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack invalid status, wait msgtype:%u idx:%u status:%d ack msgtype:%u msgid:%u datalen:%u\n",
+ w_msgtype, w_msgidx, w_status,
+ ack_msgtype, ack_msgid, ack_datalen);
+ return;
+ }
+
+ if (w_msgtype != ack_msgtype) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack msgtype mismatch, wait msgtype:%u idx:%u ack msgtype:%u msgid:%u\n",
+ w_msgtype, w_msgidx, ack_msgtype,
+ ack_msgid);
+ return;
+ }
+ if (FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, ack_msgid) != w_msgidx) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Stale ACK: expected index=%u, got msgid=%u\n",
+ w_msgidx, ack_msgid);
+ return;
+ }
+
+ WRITE_ONCE(wait_head->ack_err,
+ (s32)le32_to_cpu(*(__le32 *)&payload[NBL_CHAN_ACK_RET_POS]));
+
+ copy_len = min_t(u32, READ_ONCE(wait_head->ack_data_len), ack_datalen);
+ if (READ_ONCE(wait_head->ack_err) >= 0 && copy_len > 0) {
+ ack_data = READ_ONCE(wait_head->ack_data);
+ if (!ack_data) {
+ dev_err_ratelimited(dev, "ACK payload dropped: ack_data is NULL\n");
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ goto ack_done;
+ }
+ memcpy((char *)ack_data,
+ payload + NBL_CHAN_ACK_HEAD_LEN, copy_len);
+ WRITE_ONCE(wait_head->ack_data_len, (u16)copy_len);
+ } else {
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ }
+ack_done:
+ /* Guarantee payload data finished before acked flag visible */
+ smp_wmb();
+ WRITE_ONCE(wait_head->acked, 1);
+ mutex_unlock(&chan_info->pending_lock);
+ wake_up(&wait_head->wait_queue);
+}
+
+static void nbl_chan_recv_msg(struct nbl_channel_mgt *chan_mgt, void *data)
+{
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_msg_node_data *msg_handler;
+ u16 msg_type, payload_len, srcid, msgid;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct nbl_chan_tx_desc *tx_desc;
+ void *payload;
+ size_t avail_space;
+ u16 data_len_fw;
+
+ if (READ_ONCE(chan_info->shutdn))
+ return;
+
+ tx_desc = data;
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_dbg(dev, "recv msg_type: %d\n", msg_type);
+
+ srcid = le16_to_cpu(READ_ONCE(tx_desc->srcid));
+ msgid = le16_to_cpu(READ_ONCE(tx_desc->msgid));
+
+ if (msg_type >= NBL_CHAN_MSG_MAILBOX_MAX)
+ return;
+
+ data_len_fw = le16_to_cpu(READ_ONCE(tx_desc->data_len));
+ if (data_len_fw) {
+ payload_len = data_len_fw;
+
+ if (payload_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ dev_err_ratelimited(dev,
+ "data_len=%u exceeds embedded buffer size=%u\n",
+ payload_len,
+ NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN);
+ return;
+ }
+ /* Small pkt: payload stored inside descriptor data[] array */
+ payload = tx_desc->data;
+ } else {
+ payload_len = le16_to_cpu(READ_ONCE(tx_desc->buf_len));
+
+ avail_space = NBL_CHAN_BUF_LEN - sizeof(*tx_desc);
+ if (payload_len > avail_space) {
+ dev_err_ratelimited(dev,
+ "buf_len=%u exceeds external buffer size=%zu\n",
+ payload_len, avail_space);
+ return;
+ }
+ /* Large pkt: payload follows immediately after tx_desc */
+ payload = tx_desc + 1;
+ }
+
+ msg_handler = nbl_common_get_hash_node(chan_mgt->handle_hash_tbl,
+ &msg_type);
+ if (!msg_handler || !msg_handler->func) {
+ dev_err_ratelimited(dev,
+ "No handler for msg_type: %u (srcid=%u, msgid=%u)\n",
+ msg_type, srcid, msgid);
+ return;
+ }
+
+ msg_handler->func(msg_handler->priv, srcid, msgid, payload,
+ payload_len);
+}
+
+static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_ring *rxq)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ u16 next_to_use;
+
+ next_to_use = rxq->next_to_use;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_use);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_use);
+
+ /*
+ * Recycle the RX descriptor at next_to_use. The initial
+ * unused slot is intentionally recycled after the first
+ * RX descriptor is consumed, allowing the ring to become
+ * fully populated while next_to_clean tracks the consumer.
+ */
+ rx_desc->buf_addr = cpu_to_le64(rx_buf->pa);
+ rx_desc->buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+
+ /*
+ * DMA Write Memory Barrier:
+ * Ensures all previous DMA-mapped writes (buffer address/length)
+ * are completed before the descriptor flags are updated.
+ * This prevents hardware from seeing a partially updated descriptor
+ * where flags are set but buffer info isn't ready yet.
+ */
+ dma_wmb();
+
+ rx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+
+ rxq->next_to_use++;
+ if (rxq->next_to_use == chan_info->num_rxq_entries)
+ rxq->next_to_use = 0;
+ rxq->tail_ptr++;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+}
+
+static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ u32 budget = NBL_CHAN_RX_CLEAN_BUDGET;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ struct work_struct *task;
+ bool more_work = false;
+ u16 next_to_clean;
+ u16 flags;
+
+ next_to_clean = rxq->next_to_clean;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ while (le16_to_cpu(READ_ONCE(rx_desc->flags)) &
+ BIT(NBL_CHAN_RX_DESC_USED)) {
+ flags = le16_to_cpu(READ_ONCE(rx_desc->flags));
+
+ if (READ_ONCE(chan_info->shutdn))
+ break;
+ if (!(flags & BIT(NBL_CHAN_RX_DESC_WRITE)))
+ dev_dbg(dev,
+ "mailbox rx flag 0x%x missing NBL_CHAN_RX_DESC_WRITE\n",
+ flags);
+
+ /* Make sure hardware written descriptor visible to CPU */
+ dma_rmb();
+ nbl_chan_recv_msg(chan_mgt, rx_buf->va);
+ nbl_chan_advance_rx_ring(chan_mgt, chan_info, rxq);
+ next_to_clean++;
+ if (next_to_clean == chan_info->num_rxq_entries)
+ next_to_clean = 0;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ if (--budget == 0) {
+ more_work = true;
+ break;
+ }
+ cond_resched();
+ }
+ rxq->next_to_clean = next_to_clean;
+
+ mutex_lock(&chan_info->state_lock);
+ /* Prevent queue_work after teardown clears clean_task */
+ if (READ_ONCE(chan_info->shutdn)) {
+ mutex_unlock(&chan_info->state_lock);
+ return;
+ }
+ if (common->wq && more_work) {
+ task = READ_ONCE(chan_info->clean_task);
+ if (task)
+ queue_work(common->wq, task);
+ }
+ mutex_unlock(&chan_info->state_lock);
+}
+
+static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ nbl_chan_clean_queue(chan_mgt, chan_info);
+}
+
+static int nbl_chan_get_msg_id(struct nbl_chan_info *chan_info,
+ u16 *msgid)
+{
+ int search_loc = READ_ONCE(chan_info->wait_head_index), i;
+ struct nbl_chan_waitqueue_head *wait = NULL;
+ int status;
+
+ lockdep_assert_held(&chan_info->pending_lock);
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait = &chan_info->wait[search_loc];
+ status = READ_ONCE(wait->status);
+ if (status == NBL_MBX_STATUS_IDLE ||
+ status == NBL_MBX_STATUS_TIMEOUT) {
+ WRITE_ONCE(wait->msg_index,
+ NBL_NEXT_ID(wait->msg_index,
+ NBL_CHAN_MSG_INDEX_MAX));
+
+ *msgid = FIELD_PREP(NBL_CHAN_MSGID_INDEX_MASK,
+ wait->msg_index) |
+ FIELD_PREP(NBL_CHAN_MSGID_LOC_MASK,
+ search_loc);
+
+ /* Advance starting search position for next caller */
+ WRITE_ONCE(chan_info->wait_head_index, search_loc);
+ return 0;
+ }
+
+ search_loc = NBL_NEXT_ID(search_loc,
+ chan_info->num_txq_entries - 1);
+ }
+
+ /*
+ * All tx slots are occupied. May happen under high transmit load
+ * or delayed remote ACK responses. Caller should retry later.
+ */
+ return -EAGAIN;
+}
+
+static void nbl_chan_reset_wait_head(struct nbl_chan_info *chan_info,
+ struct nbl_chan_waitqueue_head *wait_head)
+{
+ lockdep_assert_held(&chan_info->pending_lock);
+
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ WRITE_ONCE(wait_head->ack_err, 0);
+ WRITE_ONCE(wait_head->msg_type, 0);
+ WRITE_ONCE(wait_head->dstid, 0);
+}
+
+static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_waitqueue_head *wait_head = NULL;
+ struct nbl_chan_tx_param tx_param = { 0 };
+ int i = NBL_CHAN_TX_WAIT_ACK_TIMES;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct device *dev = common->dev;
+ struct work_struct *task;
+ u16 msgid = 0;
+ int ret;
+
+ if (chan_send->resp_len > NBL_CHAN_BUF_LEN) {
+ dev_err_ratelimited(dev, "resp_len %zu exceeds max %d\n",
+ chan_send->resp_len, NBL_CHAN_BUF_LEN);
+ return -EINVAL;
+ }
+
+ mutex_lock(&chan_info->state_lock);
+ if (READ_ONCE(chan_info->shutdn)) {
+ mutex_unlock(&chan_info->state_lock);
+ return -ESHUTDOWN;
+ }
+ atomic_inc(&chan_info->inflight_tx_cnt);
+ mutex_unlock(&chan_info->state_lock);
+
+ tx_param.msg_type = chan_send->msg_type;
+ tx_param.arg = chan_send->arg;
+ tx_param.arg_len = chan_send->arg_len;
+ tx_param.dstid = chan_send->dstid;
+ tx_param.msgid = msgid;
+ if (chan_send->ack) {
+ mutex_lock(&chan_info->pending_lock);
+
+ ret = nbl_chan_get_msg_id(chan_info, &msgid);
+ if (ret) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Channel tx wait head full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type,
+ chan_send->dstid);
+ goto out_clean_inflight;
+ }
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK,
+ msgid)];
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, chan_send->resp);
+ WRITE_ONCE(wait_head->ack_data_len, chan_send->resp_len);
+ WRITE_ONCE(wait_head->msg_type, chan_send->msg_type);
+ WRITE_ONCE(wait_head->msg_index,
+ FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, msgid));
+ WRITE_ONCE(wait_head->dstid, chan_send->dstid);
+
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_WAITING);
+ mutex_unlock(&chan_info->pending_lock);
+
+ tx_param.msgid = msgid;
+ }
+
+ mutex_lock(&chan_info->txq_lock);
+ ret = nbl_chan_update_txqueue(chan_mgt, chan_info, &tx_param);
+ if (ret) {
+ mutex_unlock(&chan_info->txq_lock);
+ dev_err_ratelimited(dev,
+ "Channel tx queue full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type, chan_send->dstid);
+ if (wait_head)
+ goto out_clear_wait_slot;
+ goto out_clean_inflight;
+ }
+
+ ret = nbl_chan_kick_tx_ring(chan_mgt, chan_info);
+ mutex_unlock(&chan_info->txq_lock);
+ if (ret) {
+ if (wait_head)
+ goto out_clear_wait_slot;
+ goto out_clean_inflight;
+ }
+
+ if (!chan_send->ack) {
+ ret = 0;
+ goto out_clean_inflight;
+ }
+
+ if (test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) {
+ while (!READ_ONCE(wait_head->acked)) {
+ /*
+ * avoids long task blocking when interrupt mode is
+ * disabled mid-wait. Cannot guarantee subsequent ACK
+ * delivery after interrupt mask off, only prevents
+ * infinite blocking. Spurious timeout is possible.
+ */
+ ret = wait_event_timeout(wait_head->wait_queue,
+ READ_ONCE(wait_head->acked) ||
+ READ_ONCE(chan_info->shutdn) ||
+ !test_bit(NBL_CHAN_IRQ_RDY,
+ chan_info->state),
+ NBL_CHAN_ACK_WAIT_TIME);
+
+ if (READ_ONCE(chan_info->shutdn)) {
+ ret = -ESHUTDOWN;
+ goto out_clear_wait_slot;
+ }
+ if (!test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) {
+ ret = -EIO;
+ goto out_clear_wait_slot;
+ }
+ if (ret == 0) {
+ mutex_lock(&chan_info->pending_lock);
+ if (READ_ONCE(wait_head->status) ==
+ NBL_MBX_STATUS_WAITING) {
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ /*
+ * Ensure all status/ack slot
+ * updates are visible before subsequent
+ * readers observe acked == 0
+ */
+ smp_wmb();
+ }
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Channel waiting ack failed, message type: %d, msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ goto out_clear_wait_slot;
+ }
+
+ if (READ_ONCE(wait_head->acked))
+ break;
+ }
+ if (READ_ONCE(wait_head->acked)) {
+ /*
+ * Load ordering: observe acked flag before
+ * reading ACK payload metadata.
+ */
+ smp_rmb();
+ chan_send->ack_len = READ_ONCE(wait_head->ack_data_len);
+ ret = READ_ONCE(wait_head->ack_err);
+ }
+ } else {
+ /* Polling path for synchronous ACK */
+ while (i--) {
+ if (READ_ONCE(chan_info->shutdn)) {
+ ret = -ESHUTDOWN;
+ goto out_clear_wait_slot;
+ }
+
+ mutex_lock(&chan_info->state_lock);
+ task = READ_ONCE(chan_info->clean_task);
+ if (common->wq && task &&
+ !READ_ONCE(chan_info->shutdn) &&
+ !work_pending(task))
+ queue_work(common->wq, task);
+ mutex_unlock(&chan_info->state_lock);
+ if (READ_ONCE(wait_head->acked)) {
+ /*
+ * Guarantee load order: observe acked
+ * flag before reading ack payload metadata.
+ */
+ smp_rmb();
+ chan_send->ack_len =
+ READ_ONCE(wait_head->ack_data_len);
+ ret = READ_ONCE(wait_head->ack_err);
+ goto out_clear_wait_slot;
+ }
+
+ usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
+ NBL_CHAN_TX_WAIT_ACK_US_MAX);
+ cond_resched();
+ }
+
+ dev_err_ratelimited(dev,
+ "Channel polling ack failed, message type: %d msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ }
+
+out_clear_wait_slot:
+ mutex_lock(&chan_info->pending_lock);
+ nbl_chan_reset_wait_head(chan_info, wait_head);
+ mutex_unlock(&chan_info->pending_lock);
+
+out_clean_inflight:
+ mutex_lock(&chan_info->state_lock);
+ if (atomic_dec_and_test(&chan_info->inflight_tx_cnt))
+ wake_up(&chan_info->inflight_wait);
+ mutex_unlock(&chan_info->state_lock);
+ return ret;
+}
+
+static int nbl_chan_send_ack(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack)
+{
+ size_t head_len = NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ size_t data_len = chan_ack->data_len;
+ struct nbl_chan_send_info chan_send;
+ __le32 *tmp;
+ size_t len;
+ int ret;
+
+ if (data_len >
+ NBL_CHAN_BUF_LEN - sizeof(struct nbl_chan_tx_desc) - head_len)
+ return -EINVAL;
+
+ len = head_len + data_len;
+ tmp = kzalloc(len, GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ *(__le16 *)&tmp[NBL_CHAN_MSG_TYPE_POS] =
+ cpu_to_le16(chan_ack->msg_type);
+ *(__le16 *)&tmp[NBL_CHAN_MSG_ID_POS] = cpu_to_le16(chan_ack->msgid);
+ tmp[NBL_CHAN_ACK_RET_POS] = cpu_to_le32(chan_ack->err);
+ if (chan_ack->data && chan_ack->data_len)
+ memcpy(&tmp[NBL_CHAN_ACK_HEAD_LEN], chan_ack->data,
+ chan_ack->data_len);
+
+ nbl_chan_fill_send_info(&chan_send, chan_ack->dstid, NBL_CHAN_MSG_ACK,
+ tmp, len, NULL, 0, 0);
+ ret = nbl_chan_send_msg(chan_mgt, &chan_send);
+ kfree(tmp);
+
+ return ret;
+}
+
+static int nbl_chan_register_msg(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback)
+{
+ return nbl_chan_add_msg_handler(chan_mgt, msg_type, func, callback);
+}
+
+static bool nbl_chan_check_queue_exist(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ return chan_info ? true : false;
+}
+
+static void nbl_chan_register_chan_task(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ mutex_lock(&chan_info->state_lock);
+ if (!READ_ONCE(chan_info->shutdn))
+ WRITE_ONCE(chan_info->clean_task, task);
+ mutex_unlock(&chan_info->state_lock);
+}
+
+static void nbl_chan_set_queue_state(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ if (set)
+ set_bit(state, chan_info->state);
+ else
+ clear_bit(state, chan_info->state);
+}
+
+static struct nbl_channel_ops chan_ops = {
+ .send_msg = nbl_chan_send_msg,
+ .send_ack = nbl_chan_send_ack,
+ .register_msg = nbl_chan_register_msg,
+ .unregister_all_msg = nbl_chan_remove_msg_handler,
+ .cfg_chan_qinfo_map_table = nbl_chan_cfg_qinfo_map_table,
+ .check_queue_exist = nbl_chan_check_queue_exist,
+ .setup_queue = nbl_chan_setup_queue,
+ .teardown_queue = nbl_chan_teardown_queue,
+ .clean_queue_subtask = nbl_chan_clean_queue_subtask,
+ .register_chan_task = nbl_chan_register_chan_task,
+ .set_queue_state = nbl_chan_set_queue_state,
+};
+
+static struct nbl_channel_mgt *
+nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adapter->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adapter->common;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ struct nbl_chan_info *mailbox;
+ int ret;
+
+ chan_mgt = devm_kzalloc(dev, sizeof(*chan_mgt), GFP_KERNEL);
+ if (!chan_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ chan_mgt->common = common;
+ chan_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ mailbox = devm_kzalloc(dev, sizeof(*mailbox), GFP_KERNEL);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+ mailbox->chan_type = NBL_CHAN_TYPE_MAILBOX;
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX] = mailbox;
+
+ ret = nbl_chan_init_msg_handler(chan_mgt);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->txq_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->state_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->pending_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ return chan_mgt;
+}
+
+static struct nbl_channel_ops_tbl *
+nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ int ret;
+
+ chan_ops_tbl = devm_kzalloc(dev, sizeof(*chan_ops_tbl), GFP_KERNEL);
+ if (!chan_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!chan_ops.send_msg || !chan_ops.send_ack ||
+ !chan_ops.register_msg || !chan_ops.unregister_all_msg ||
+ !chan_ops.cfg_chan_qinfo_map_table ||
+ !chan_ops.check_queue_exist || !chan_ops.setup_queue ||
+ !chan_ops.teardown_queue || !chan_ops.clean_queue_subtask ||
+ !chan_ops.register_chan_task || !chan_ops.set_queue_state)
+ return ERR_PTR(-EINVAL);
+
+ chan_ops_tbl->ops = &chan_ops;
+ chan_ops_tbl->priv = chan_mgt;
+
+ ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
+ nbl_chan_recv_ack_msg, chan_mgt);
+ if (ret)
+ return ERR_PTR(ret);
+
+ return chan_ops_tbl;
+}
+
+int nbl_chan_init_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ int ret;
+
+ chan_mgt = nbl_chan_setup_chan_mgt(adap);
+ if (IS_ERR(chan_mgt)) {
+ ret = PTR_ERR(chan_mgt);
+ goto exit;
+ }
+
+ chan_ops_tbl = nbl_chan_setup_ops(dev, chan_mgt);
+ if (IS_ERR(chan_ops_tbl)) {
+ ret = PTR_ERR(chan_ops_tbl);
+ goto cleanup_mgt;
+ }
+
+ adap->intf.channel_ops_tbl = chan_ops_tbl;
+ adap->core.chan_mgt = chan_mgt;
+ ret = nbl_common_create_wq(&adap->common);
+ if (ret)
+ goto cleanup_mgt;
+ return 0;
+
+cleanup_mgt:
+ nbl_chan_remove_msg_handler(chan_mgt);
+exit:
+ return ret;
+}
+
+void nbl_chan_remove_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_mgt *chan_mgt = adap->core.chan_mgt;
+
+ if (!chan_mgt)
+ return;
+ nbl_common_destroy_wq(&adap->common);
+ /*
+ * All channel queues shall be torn down earlier in remove path
+ * to drain inflight tx workers and stop hardware before destroying
+ * message handler hash table.
+ */
+ nbl_chan_remove_msg_handler(chan_mgt);
+ adap->core.chan_mgt = NULL;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
new file mode 100644
index 000000000000..e6d96768a5b5
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
@@ -0,0 +1,170 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHANNEL_H_
+#define _NBL_CHANNEL_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_CHAN_TX_RING_TO_DESC(tx_ring, i) \
+ (&((((tx_ring)->desc.tx_desc))[i]))
+#define NBL_CHAN_RX_RING_TO_DESC(rx_ring, i) \
+ (&((((rx_ring)->desc.rx_desc))[i]))
+#define NBL_CHAN_TX_RING_TO_BUF(tx_ring, i) (&(((tx_ring)->buf)[i]))
+#define NBL_CHAN_RX_RING_TO_BUF(rx_ring, i) (&(((rx_ring)->buf)[i]))
+
+#define NBL_CHAN_TX_WAIT_US 100
+#define NBL_CHAN_TX_WAIT_US_MAX 120
+#define NBL_CHAN_TX_WAIT_TIMES 100
+#define NBL_CHAN_TX_WAIT_ACK_US_MIN 1000
+#define NBL_CHAN_TX_WAIT_ACK_US_MAX 1200
+#define NBL_CHAN_TX_WAIT_ACK_TIMES 5000
+#define NBL_CHAN_QUEUE_LEN 256
+#define NBL_CHAN_BUF_LEN 4096
+#define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
+
+#define NBL_CHAN_TX_DESC_AVAIL 0
+#define NBL_CHAN_TX_DESC_USED 1
+#define NBL_CHAN_RX_DESC_WRITE 1
+#define NBL_CHAN_RX_DESC_AVAIL 3
+#define NBL_CHAN_RX_DESC_USED 4
+
+#define NBL_CHAN_ACK_HEAD_LEN 3
+#define NBL_CHAN_ACK_RET_POS 2
+#define NBL_CHAN_MSG_ID_POS 1
+#define NBL_CHAN_MSG_TYPE_POS 0
+
+#define NBL_CHAN_ACK_WAIT_TIME (3 * HZ)
+#define NBL_CHAN_RX_CLEAN_BUDGET 64
+#define NBL_CHAN_HANDLER_TBL_BUCKET_SIZE 512
+
+enum {
+ NBL_MB_RX_QID = 0,
+ NBL_MB_TX_QID = 1,
+};
+
+enum {
+ NBL_MBX_STATUS_IDLE = 0,
+ NBL_MBX_STATUS_WAITING,
+ NBL_MBX_STATUS_TIMEOUT,
+};
+
+struct nbl_chan_tx_param {
+ enum nbl_chan_msg_type msg_type;
+ void *arg;
+ size_t arg_len;
+ u16 dstid;
+ u16 msgid;
+};
+
+struct nbl_chan_buf {
+ void *va;
+ dma_addr_t pa;
+ size_t size;
+};
+
+struct nbl_chan_tx_desc {
+ __le16 flags;
+ __le16 srcid;
+ __le16 dstid;
+ __le16 data_len;
+ __le16 buf_len;
+ __le64 buf_addr;
+ __le16 msg_type;
+ u8 data[NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN];
+ __le16 msgid;
+ u8 rsv[26];
+} __packed;
+
+struct nbl_chan_rx_desc {
+ __le16 flags;
+ __le32 buf_len;
+ __le16 buf_id;
+ __le64 buf_addr;
+} __packed;
+
+union nbl_chan_desc_ptr {
+ struct nbl_chan_tx_desc *tx_desc;
+ struct nbl_chan_rx_desc *rx_desc;
+};
+
+struct nbl_chan_ring {
+ union nbl_chan_desc_ptr desc;
+ struct nbl_chan_buf *buf;
+ u16 next_to_use;
+ u16 tail_ptr; /* hardware does modulo ring size internally */
+ u16 next_to_clean;
+ dma_addr_t dma;
+};
+
+#define NBL_CHAN_MSG_INDEX_MAX 63
+
+#define NBL_CHAN_MSGID_INDEX_MASK GENMASK(5, 0)
+#define NBL_CHAN_MSGID_LOC_MASK GENMASK(13, 6)
+
+static inline void nbl_chan_update_tail_ptr(struct nbl_hw_ops *hw_ops,
+ void *hw_priv, u32 tail_ptr, u8 qid)
+{
+ hw_ops->update_mailbox_queue_tail_ptr(hw_priv, tail_ptr, qid);
+}
+
+struct nbl_chan_waitqueue_head {
+ struct wait_queue_head wait_queue;
+ char *ack_data;
+ int acked;
+ s32 ack_err;
+ u16 ack_data_len;
+ u16 msg_type;
+ int status;
+ u8 msg_index;
+ u16 dstid;
+};
+
+struct nbl_chan_info {
+ wait_queue_head_t inflight_wait;
+ struct nbl_chan_ring txq;
+ struct nbl_chan_ring rxq;
+ struct nbl_chan_waitqueue_head *wait;
+ /*
+ *Protects access to the TX queue (txq) and related metadata.
+ *This mutex ensures exclusive access when updating the TX queue
+ */
+ struct mutex txq_lock;
+ /* Guards channel state bitmap, active and shutdn flags */
+ struct mutex state_lock;
+ /* Guards pending requests and pending work list operations */
+ struct mutex pending_lock;
+ struct work_struct *clean_task;
+ u16 wait_head_index;
+ u16 num_txq_entries;
+ u16 num_rxq_entries;
+ u16 txq_buf_size;
+ u16 rxq_buf_size;
+ DECLARE_BITMAP(state, NBL_CHAN_STATE_NBITS);
+ u8 chan_type;
+ atomic_t inflight_tx_cnt;
+ bool shutdn;
+ bool active;
+};
+
+struct nbl_chan_msg_node_data {
+ nbl_chan_resp func;
+ void *priv;
+};
+
+struct nbl_channel_mgt {
+ struct nbl_common_info *common;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_chan_info *chan_info[NBL_CHAN_TYPE_MAX];
+ struct nbl_hash_tbl_mgt *handle_hash_tbl;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
new file mode 100644
index 000000000000..38abf41d9bb0
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/jhash.h>
+#include "nbl_common.h"
+
+void nbl_common_destroy_wq(struct nbl_common_info *common)
+{
+ if (!common || !common->wq)
+ return;
+
+ destroy_workqueue(common->wq);
+ common->wq = NULL;
+}
+
+int nbl_common_create_wq(struct nbl_common_info *common)
+{
+ char wq_name[32];
+
+ snprintf(wq_name, sizeof(wq_name), "nbl_wq_%s", pci_name(common->pdev));
+ common->wq = alloc_workqueue(wq_name, WQ_UNBOUND, 0);
+ if (!common->wq) {
+ dev_err(common->dev, "Failed to alloc workqueue %s\n", wq_name);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
+{
+ u32 hash;
+
+ if (bucket_size == 0 || bucket_size == 1)
+ return 0;
+
+ hash = jhash(key, key_size, 0);
+
+ /* Use bitmask if bucket_size is a power of 2 */
+ if ((bucket_size & (bucket_size - 1)) == 0)
+ return hash & (bucket_size - 1);
+ return hash % bucket_size;
+}
+
+/**
+ * nbl_common_init_hash_table - initialize per-device hash table
+ * @key: hash table creation parameters
+ *
+ * Return: allocated tbl mgt pointer, NULL on failure.
+ */
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key)
+{
+ struct nbl_hash_tbl_mgt *tbl_mgt;
+ u32 bucket_size;
+ u32 i;
+
+ tbl_mgt = devm_kzalloc(key->dev, sizeof(*tbl_mgt), GFP_KERNEL);
+ if (!tbl_mgt)
+ return NULL;
+
+ bucket_size = key->bucket_size;
+ tbl_mgt->hash = devm_kcalloc(key->dev, bucket_size,
+ sizeof(struct hlist_head), GFP_KERNEL);
+ if (!tbl_mgt->hash)
+ return NULL;
+
+ tbl_mgt->bucket_locks = devm_kcalloc(key->dev, bucket_size,
+ sizeof(spinlock_t), GFP_KERNEL);
+ if (!tbl_mgt->bucket_locks)
+ return NULL;
+
+ for (i = 0; i < bucket_size; i++) {
+ INIT_HLIST_HEAD(&tbl_mgt->hash[i]);
+ spin_lock_init(&tbl_mgt->bucket_locks[i]);
+ }
+
+ memcpy(&tbl_mgt->tbl_key, key, sizeof(tbl_mgt->tbl_key));
+ tbl_mgt->node_num = 0;
+
+ return tbl_mgt;
+}
+
+/**
+ * nbl_common_alloc_hash_node - insert handler node into hash table
+ * @tbl_mgt: hash table manager
+ * @key: match key (msg_type)
+ * @data: handler callback info
+ * @out_data: optional pointer to return allocated data ptr
+ *
+ * Caller context: process context for dynamic registration, init path safe.
+ * Protected by per-bucket spin_lock_bh to avoid race with concurrent lookup.
+ *
+ * Return: 0 on success, -ENOMEM on allocation failure.
+ */
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data)
+{
+ struct nbl_hash_entry_node *hash_node;
+ u16 data_size;
+ u16 node_size;
+ u32 hash_val;
+ u16 key_size;
+
+ node_size = sizeof(*hash_node);
+ hash_node = kzalloc(node_size, GFP_KERNEL);
+ if (!hash_node)
+ return -ENOMEM;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_node->key = kzalloc(key_size, GFP_KERNEL);
+ if (!hash_node->key)
+ goto alloc_key_failed;
+
+ data_size = tbl_mgt->tbl_key.data_size;
+ hash_node->data = kzalloc(data_size, GFP_KERNEL);
+ if (!hash_node->data)
+ goto alloc_data_failed;
+
+ memcpy(hash_node->key, key, key_size);
+ memcpy(hash_node->data, data, data_size);
+
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[hash_val]);
+ hlist_add_head(&hash_node->node, tbl_mgt->hash + hash_val);
+ tbl_mgt->node_num++;
+ spin_unlock_bh(&tbl_mgt->bucket_locks[hash_val]);
+
+ if (out_data)
+ *out_data = hash_node->data;
+
+ return 0;
+
+alloc_data_failed:
+ kfree(hash_node->key);
+alloc_key_failed:
+ kfree(hash_node);
+ return -ENOMEM;
+}
+
+/**
+ * nbl_common_get_hash_node - lookup handler from hash table
+ * @tbl_mgt: hash table manager
+ * @key: lookup key
+ *
+ * All accessors use spin_lock_bh so that process-context holders
+ * disable softirq and cannot deadlock against a concurrent softirq
+ * caller (e.g. NAPI RX path). Safe in both process and softirq
+ * context.
+ *
+ * Return: attached handler data if found, NULL otherwise.
+ */
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_head *head;
+ void *data = NULL;
+ u32 hash_val;
+ u16 key_size;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+ head = tbl_mgt->hash + hash_val;
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[hash_val]);
+ hlist_for_each_entry(hash_node, head, node) {
+ if (!memcmp(hash_node->key, key, key_size)) {
+ data = hash_node->data;
+ break;
+ }
+ }
+ spin_unlock_bh(&tbl_mgt->bucket_locks[hash_val]);
+
+ return data;
+}
+
+/*
+ * Free all hash nodes in the table.
+ */
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_node *safe_node;
+ struct hlist_head *head;
+ u32 i;
+
+ if (!tbl_mgt)
+ return;
+
+ for (i = 0; i < tbl_mgt->tbl_key.bucket_size; i++) {
+ head = tbl_mgt->hash + i;
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[i]);
+ hlist_for_each_entry_safe(hash_node, safe_node, head, node) {
+ hlist_del(&hash_node->node);
+ tbl_mgt->node_num--;
+ kfree(hash_node->key);
+ kfree(hash_node->data);
+ kfree(hash_node);
+ }
+ spin_unlock_bh(&tbl_mgt->bucket_locks[i]);
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
new file mode 100644
index 000000000000..159421e53902
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_COMMON_H_
+#define _NBL_COMMON_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_common.h"
+
+struct nbl_hash_tbl_mgt {
+ struct nbl_hash_tbl_key tbl_key;
+ struct hlist_head *hash;
+ /**
+ * bucket_locks: per-bucket spinlock array
+ * Each hash bucket corresponds to an independent spinlock.
+ * Protects concurrent hash list modification
+ */
+ spinlock_t *bucket_locks;
+ u16 node_num;
+};
+
+struct nbl_hash_entry_node {
+ struct hlist_node node;
+ void *key;
+ void *data;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 1cd6587a8fcb..f998a2b44e5c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -14,13 +14,20 @@ enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_interface {
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_channel_ops_tbl *channel_ops_tbl;
+};
+
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_channel_mgt *chan_mgt;
};
struct nbl_adapter {
struct pci_dev *pdev;
struct nbl_core core;
+ struct nbl_interface intf;
struct nbl_common_info common;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index a67d1a674466..24b15418e601 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,156 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+ if (reg >= (u64)hw_mgt->mailbox_bar_size ||
+ reg + len > (u64)hw_mgt->mailbox_bar_size) {
+ dev_err_once(hw_mgt->common->dev,
+ "mbx write out of range: reg=0x%llx len=%u bar_size=%pa\n",
+ reg, len, &hw_mgt->mailbox_bar_size);
+ return;
+ }
+ for (i = 0; i < len / 4; i++)
+ nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
+}
+
+static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+
+ spin_lock(&hw_mgt->reg_lock);
+
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ spin_lock(&hw_mgt->reg_lock);
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx)
+{
+ /* local_qid 0 and 1 denote rx and tx queue respectively */
+ u32 local_qid = txrx;
+ u32 value = ((u32)tail_ptr << 16) | local_qid;
+
+ /* wmb for doorbell */
+ wmb();
+ nbl_mbx_wr32(hw_mgt, NBL_MAILBOX_NOTIFY_ADDR, value);
+}
+
+static void nbl_hw_config_mailbox_rxq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_config_mailbox_txq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_rxq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_txq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
+{
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_K_PF_MASK_REG, pf_mask,
+ sizeof(*pf_mask));
+}
+
+static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function)
+{
+ u32 data = 0;
+
+ data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
+ &data,
+ sizeof(data));
+}
+
+static struct nbl_hw_ops hw_ops = {
+ .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
+ .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
+ .config_mailbox_txq = nbl_hw_config_mailbox_txq,
+ .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
+ .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
+ .get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+
+};
+
/* Structure starts here, adding an op should not modify anything below */
static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
{
@@ -25,6 +175,27 @@ static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
return hw_mgt;
}
+static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
+ struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct device *dev;
+
+ dev = common->dev;
+ hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
+ if (!hw_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ !hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
+ !hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
+ !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ return ERR_PTR(-EINVAL);
+ hw_ops_tbl->ops = &hw_ops;
+ hw_ops_tbl->priv = hw_mgt;
+
+ return hw_ops_tbl;
+}
+
static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
const char *name)
{
@@ -45,6 +216,7 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
{
resource_size_t expect_sz = NBL_MEM_BAR_TOTAL_SIZE;
struct nbl_common_info *common = &adapter->common;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = NULL;
struct pci_dev *pdev = common->pdev;
struct nbl_hw_mgt *hw_mgt = NULL;
resource_size_t bar_len;
@@ -135,7 +307,14 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
goto setup_mgt_fail;
}
hw_mgt->mailbox_bar_size = bar_len;
+ spin_lock_init(&hw_mgt->reg_lock);
+ hw_ops_tbl = nbl_hw_setup_ops(common, hw_mgt);
+ if (IS_ERR(hw_ops_tbl)) {
+ ret = PTR_ERR(hw_ops_tbl);
+ goto setup_mgt_fail;
+ }
+ adapter->intf.hw_ops_tbl = hw_ops_tbl;
adapter->core.hw_mgt = hw_mgt;
return 0;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index a3397f5be353..99ee126db9dc 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,4 +11,60 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+/* ---------- REG BASE ADDR ---------- */
+/* Interface modules base addr */
+#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
+#define NBL_INTF_HOST_PADPT_BASE 0x00f4c000
+#define NBL_INTF_HOST_MAILBOX_BASE 0x00fb0000
+#define NBL_INTF_HOST_PCIE_BASE 0X01504000
+/* DP modules base addr */
+#define NBL_DP_USTORE_BASE 0x00104000
+#define NBL_DP_UQM_BASE 0x00114000
+#define NBL_DP_UPED_BASE 0x0015c000
+#define NBL_DP_UVN_BASE 0x00244000
+#define NBL_DP_DSCH_BASE 0x00404000
+#define NBL_DP_SHAPING_BASE 0x00504000
+#define NBL_DP_DVN_BASE 0x00514000
+#define NBL_DP_DSTORE_BASE 0x00704000
+#define NBL_DP_DQM_BASE 0x00714000
+#define NBL_DP_DPED_BASE 0x0075c000
+#define NBL_DP_DDMUX_BASE 0x00984000
+/* -------- MAILBOX BAR2 ----- */
+#define NBL_MAILBOX_NOTIFY_ADDR 0x00000000
+#define NBL_MAILBOX_BAR_REG 0x00000000
+#define NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR 0x10
+#define NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR 0x20
+#define NBL_MAILBOX_QINFO_CFG_DBG_TABLE_ADDR 0x30
+
+/* -------- MAILBOX -------- */
+
+/* mailbox BAR qinfo_cfg_table */
+#define MAILBOX_QINFO_CFG_TABLE_DWLEN 4
+/* data[2] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK GENMASK(3, 0)
+/* data[3] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK BIT(0)
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK BIT(1)
+#define NBL_MAILBOX_QINFO_CFG_DIF_ERR_MASK BIT(2)
+#define NBL_MAILBOX_QINFO_CFG_PTR_ERR_MASK BIT(3)
+struct nbl_mailbox_qinfo_cfg_table {
+ u32 data[MAILBOX_QINFO_CFG_TABLE_DWLEN];
+};
+
+/* -------- MAILBOX BAR0 ----- */
+/* mailbox qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id) \
+ (NBL_INTF_HOST_MAILBOX_BASE + 0x00001000 + (func_id) * sizeof(u32))
+
+/* MAILBOX qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_MAILBOX_QINFO_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_MAILBOX_QINFO_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK GENMASK(28, 16)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK BIT(29)
+
+/* -------- HOST_PCIE -------- */
+#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
+#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
index 5e9823e01d39..35604bdff2ae 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
+#include "../nbl_include/nbl_def_channel.h"
#include "../nbl_include/nbl_def_hw.h"
#include "../nbl_include/nbl_def_common.h"
#include "../nbl_core.h"
@@ -16,6 +17,7 @@
#define NBL_MAILBOX_BAR 2
#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+#define NBL_HW_DUMMY_REG 0x1300904
/*
* PCI MEMORY BAR total size: 64MiB.
*/
@@ -27,6 +29,38 @@ struct nbl_hw_mgt {
u8 __iomem *mailbox_bar_hw_addr;
resource_size_t hw_size;
resource_size_t mailbox_bar_size;
+ spinlock_t reg_lock; /* Protect reg access */
};
+static inline u32 rd32(u8 __iomem *addr, u64 reg)
+{
+ return readl(addr + reg);
+}
+
+static inline void wr32(u8 __iomem *addr, u64 reg, u32 value)
+{
+ writel(value, addr + reg);
+}
+
+static inline void nbl_hw_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ /* Used for emu, make sure that we won't write too frequently */
+ wr32(hw_mgt->hw_addr, reg, value);
+}
+
+static inline u32 nbl_hw_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return rd32(hw_mgt->hw_addr, reg);
+}
+
+static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
+static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return readl(hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
new file mode 100644
index 000000000000..4fc987c0e56d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_CHANNEL_H_
+#define _NBL_DEF_CHANNEL_H_
+
+#include <linux/types.h>
+
+struct nbl_channel_mgt;
+struct nbl_adapter;
+
+typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+
+/*
+ * Mailbox wire opcodes, stable wire ABI shared between driver and firmware.
+ * Each opcode has a fixed assigned number to preserve compatibility.
+ * ABI compatibility rules:
+ * 1. New opcodes shall only be appended before NBL_CHAN_MSG_MAILBOX_MAX;
+ * 2. Reordering, inserting or deleting existing enumerators breaks driver-
+ * firmware interoperability and must be avoided;
+ * 3. Modifications to existing opcodes require synchronized firmware ABI
+ * updates.
+ *
+ * Only opcodes currently used by in-tree driver logic are defined here.
+ * Unimplemented feature opcodes (KTLS, IPsec, vDPA, mirror etc.) will be
+ * added incrementally together with their corresponding driver
+ * implementation patches.
+ */
+enum nbl_chan_msg_type {
+ NBL_CHAN_MSG_ACK = 0,
+ /* mailbox msg end */
+ NBL_CHAN_MSG_MAILBOX_MAX,
+};
+
+enum nbl_chan_state {
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_STATE_NBITS
+};
+
+struct nbl_chan_send_info {
+ void *arg;
+ size_t arg_len;
+ void *resp;
+ size_t resp_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 ack;
+ u16 ack_len;
+};
+
+struct nbl_chan_ack_info {
+ void *data;
+ int err;
+ u32 data_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 msgid;
+};
+
+enum nbl_channel_type {
+ NBL_CHAN_TYPE_MAILBOX,
+ NBL_CHAN_TYPE_MAX
+};
+
+static inline void
+nbl_chan_fill_send_info(struct nbl_chan_send_info *info,
+ u16 dst_id, u16 msg_type,
+ void *argument, u32 arg_length,
+ void *response, u32 resp_length,
+ bool need_ack)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->arg = argument;
+ info->arg_len = arg_length;
+ info->resp = response;
+ info->resp_len = resp_length;
+ info->ack = need_ack;
+}
+
+static inline void
+nbl_chan_fill_ack_info(struct nbl_chan_ack_info *info,
+ u16 dst_id, u16 msg_type, u16 msg_id,
+ int err_code, void *ack_data, u32 data_length)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->msgid = msg_id;
+ info->err = err_code;
+ info->data = ack_data;
+ info->data_len = data_length;
+}
+
+struct nbl_channel_ops {
+ int (*send_msg)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send);
+ int (*send_ack)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack);
+ int (*register_msg)(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback_priv);
+ void (*unregister_all_msg)(struct nbl_channel_mgt *chan_mgt);
+ void (*cfg_chan_qinfo_map_table)(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid);
+ bool (*check_queue_exist)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ int (*setup_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ int (*teardown_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ void (*clean_queue_subtask)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ void (*register_chan_task)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task);
+ void (*set_queue_state)(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set);
+};
+
+struct nbl_channel_ops_tbl {
+ struct nbl_channel_ops *ops;
+ struct nbl_channel_mgt *priv;
+};
+
+int nbl_chan_init_common(struct nbl_adapter *adapter);
+void nbl_chan_remove_common(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index da30244fe75d..4916c384611b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -12,6 +12,7 @@
#include "nbl_include.h"
struct nbl_common_info {
+ struct workqueue_struct *wq;
struct pci_dev *pdev;
struct device *dev;
u32 msg_enable;
@@ -29,4 +30,21 @@ struct nbl_common_info {
u8 has_net;
};
+struct nbl_hash_tbl_key {
+ struct device *dev;
+ u16 key_size;
+ u16 data_size; /* no include key or node member */
+ u16 bucket_size;
+ u16 resv;
+};
+
+void nbl_common_destroy_wq(struct nbl_common_info *common);
+int nbl_common_create_wq(struct nbl_common_info *common);
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data);
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index ecbf440e4366..587ac0c58c24 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -10,6 +10,39 @@
struct nbl_hw_mgt;
struct nbl_adapter;
+struct nbl_hw_ops {
+ void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx);
+ void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*config_mailbox_txq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*stop_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt);
+ void (*stop_mailbox_txq)(struct nbl_hw_mgt *hw_mgt);
+ /**
+ * get_host_pf_mask - Fetch host PF mask from firmware k_pf_mask reg
+ * @priv: hw ops private context
+ * @pf_mask: output pointer for PF mask value
+ *
+ * Mask: bit N == 0 → PF#N enabled; bit N == 1 → PF#N masked out.
+ * PF0 cannot be masked, bit0 is reserved and has no hardware effect.
+ * All-zero mask indicates all supported PFs are present and enabled.
+ * This mask value is NOT an indicator of firmware readiness.
+ *
+ * Product limitation: firmware shall only produce configurations with
+ * 1 PF(PF0), 2 PFs(PF0+PF1), or 4 PFs(PF0~PF3). Sparse PF layout
+ * or other PF counts are unsupported by driver resource management.
+ */
+ void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+
+ void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function);
+};
+
+struct nbl_hw_ops_tbl {
+ struct nbl_hw_ops *ops;
+ struct nbl_hw_mgt *priv;
+};
int nbl_hw_init_leonis(struct nbl_adapter *adapter);
void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 14e7b19f9a4c..f2d802397d98 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,6 +10,9 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+#define NBL_MAX_PF 8
+#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index f2552bc73293..b7c80ea54c8d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -38,13 +39,19 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
if (ret)
goto hw_init_fail;
+ ret = nbl_chan_init_common(adapter);
+ if (ret)
+ goto chan_init_fail;
return adapter;
+chan_init_fail:
+ nbl_hw_remove_leonis(adapter);
hw_init_fail:
return ERR_PTR(ret);
}
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 04/10] net/nebula-matrix: add common resource implementation
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (2 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,04/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
` (5 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
The Resource layer processes entries and data of chip modules to implement
entry management operations.
This patch provides the common part of resource layer, including conversion
helpers between vsi_id, func_id, eth_id, and pf_id. These mappings are used
by upper layers and the resource layer itself.
nbl_res_start() initializes VSI/Eth/PF data structures only for control
devices (`common->has_ctrl == true`). Framework dispatch layer ensures
resource mapping APIs such as nbl_res_func_id_to_vsi_id() are only invoked
on control devices.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 2 +
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 22 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 70 +++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 +
.../nbl_hw_leonis/nbl_resource_leonis.c | 334 ++++++++++++++++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 10 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 120 +++++++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 73 ++++
.../nbl/nbl_include/nbl_def_channel.h | 10 +
.../nbl/nbl_include/nbl_def_common.h | 19 +
.../nbl/nbl_include/nbl_def_hw.h | 5 +
.../nbl/nbl_include/nbl_def_resource.h | 29 ++
.../nbl/nbl_include/nbl_include.h | 6 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
15 files changed, 724 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 04e1aa1fb4bd..3dab9519a277 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -6,4 +6,6 @@ obj-$(CONFIG_NBL) := nbl.o
nbl-objs += nbl_common/nbl_common.o \
nbl_channel/nbl_channel.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
+ nbl_hw/nbl_resource.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
index 38abf41d9bb0..597c11ed7fd8 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -30,6 +30,28 @@ int nbl_common_create_wq(struct nbl_common_info *common)
return 0;
}
+/**
+ * nbl_common_func_id_to_rel_pf_id - convert absolute PF id to relative PF id
+ * @common: common device info
+ * @pf_id: absolute PF identifier
+ * @rel_pf_id: output relative pf id
+ *
+ * Leonis uses fixed mgt_pf = 0. Support future non-zero management PF.
+ *
+ * Return: 0 on success, -EINVAL on invalid arguments.
+ */
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id)
+{
+ if (!rel_pf_id)
+ return -EINVAL;
+
+ if (pf_id < common->mgt_pf)
+ return -EINVAL;
+ *rel_pf_id = pf_id - common->mgt_pf;
+ return 0;
+}
+
static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
{
u32 hash;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index f998a2b44e5c..dd24ebec0171 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -16,11 +16,13 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_resource_ops_tbl *resource_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_resource_mgt *res_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 24b15418e601..3637c00cdfce 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,24 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_read_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+ if (reg >= (u64)hw_mgt->mailbox_bar_size ||
+ reg + len > (u64)hw_mgt->mailbox_bar_size) {
+ dev_err_once(hw_mgt->common->dev,
+ "mbx read out of range: reg=0x%llx len=%u bar_size=%pa\n",
+ reg, len, &hw_mgt->mailbox_bar_size);
+ return;
+ }
+ for (i = 0; i < len / 4; i++)
+ data[i] = nbl_mbx_rd32(hw_mgt, reg + i * sizeof(u32));
+}
+
static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
const u32 *data, u32 len)
{
@@ -58,6 +76,21 @@ static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
spin_unlock(&hw_mgt->reg_lock);
}
+/*
+ * Registers reset to zero after cold boot / FLR / bus reset. Firmware
+ * programs valid values before driver probe, so zero is only seen on
+ * hardware fault or register read failure. Initialize data=0 to guard
+ * against nbl_hw_read_mbx_regs() early-return on bounds-check failure.
+ */
+static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
+{
+ u32 data = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW6_OFFSET, &data,
+ sizeof(data));
+ *eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -136,6 +169,15 @@ static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
sizeof(*pf_mask));
}
+static void nbl_hw_get_real_bus(struct nbl_hw_mgt *hw_mgt, u8 *bus)
+{
+ u32 data = 0;
+
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_TL_CFG_BUSDEV, &data,
+ sizeof(data));
+ *bus = FIELD_GET(NBL_PCIE_BUS_MASK, data);
+}
+
static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function)
{
@@ -149,6 +191,26 @@ static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
sizeof(data));
}
+/*
+ * Registers reset to zero after cold boot / FLR / bus reset. Firmware
+ * programs valid values before driver probe, so zero is only seen on
+ * hardware fault or register read failure. Initialize data=0 to guard
+ * against nbl_hw_read_mbx_regs() early-return on bounds-check failure.
+ */
+static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board_info)
+{
+ u32 data = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW3_OFFSET, &data,
+ sizeof(data));
+ board_info->eth_num = FIELD_GET(NBL_FW_BOARD_DW3_PORT_NUM_MASK, data);
+ board_info->eth_speed =
+ FIELD_GET(NBL_FW_BOARD_DW3_PORT_SPEED_MASK, data);
+ board_info->p4_version =
+ FIELD_GET(NBL_FW_BOARD_DW3_P4_VERSION_MASK, data);
+}
+
static struct nbl_hw_ops hw_ops = {
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
@@ -156,8 +218,12 @@ static struct nbl_hw_ops hw_ops = {
.stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
.stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
.get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .get_real_bus = nbl_hw_get_real_bus,
+
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .get_fw_eth_map = nbl_hw_get_fw_eth_map,
+ .get_board_info = nbl_hw_get_board_info,
};
/* Structure starts here, adding an op should not modify anything below */
@@ -188,7 +254,9 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
if (!hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
- !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ !hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
+ !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
hw_ops_tbl->priv = hw_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 99ee126db9dc..88772a11124a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -67,4 +67,18 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+#define NBL_FW_BOARD_CONFIG 0x200
+#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
+#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
+
+#define NBL_FW_BOARD_DW3_PORT_TYPE_MASK BIT(0)
+#define NBL_FW_BOARD_DW3_PORT_NUM_MASK GENMASK(7, 1)
+#define NBL_FW_BOARD_DW3_PORT_SPEED_MASK GENMASK(9, 8)
+#define NBL_FW_BOARD_DW3_GPIO_TYPE_MASK GENMASK(12, 10)
+#define NBL_FW_BOARD_DW3_P4_VERSION_MASK GENMASK(13, 13)
+
+#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
+#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
new file mode 100644
index 000000000000..a719cc8fbf8a
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -0,0 +1,334 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include "nbl_resource_leonis.h"
+
+static struct nbl_resource_ops res_ops = {
+ .get_vsi_id = nbl_res_func_id_to_vsi_id,
+ .get_eth_id = nbl_res_get_eth_id,
+};
+
+static struct nbl_resource_mgt *
+nbl_res_setup_res_mgt(struct nbl_common_info *common)
+{
+ struct nbl_resource_info *resource_info;
+ struct nbl_resource_mgt *res_mgt;
+ struct device *dev = common->dev;
+
+ res_mgt = devm_kzalloc(dev, sizeof(*res_mgt), GFP_KERNEL);
+ if (!res_mgt)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->common = common;
+
+ resource_info =
+ devm_kzalloc(dev, sizeof(*resource_info), GFP_KERNEL);
+ if (!resource_info)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->resource_info = resource_info;
+
+ return res_mgt;
+}
+
+static struct nbl_resource_ops_tbl *
+nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+
+ res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
+ if (!res_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!res_ops.get_vsi_id || !res_ops.get_eth_id)
+ return ERR_PTR(-EINVAL);
+ res_ops_tbl->ops = &res_ops;
+ res_ops_tbl->priv = res_mgt;
+
+ return res_ops_tbl;
+}
+
+static int nbl_res_ctrl_dev_setup_eth_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_eth_info *eth_info;
+ u32 eth_bitmap = 0, eth_id;
+ u32 eth_num = 0;
+ u32 fw_port_num;
+ int i;
+
+ eth_info = devm_kzalloc(dev, sizeof(*eth_info), GFP_KERNEL);
+ if (!eth_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->eth_info = eth_info;
+
+ fw_port_num = res_mgt->resource_info->board_info.eth_num;
+
+ hw_ops->get_fw_eth_map(res_mgt->hw_ops_tbl->priv, ð_bitmap);
+ if (eth_bitmap & ~((1 << NBL_MAX_ETHERNET) - 1)) {
+ dev_err(dev, "FW reported invalid eth_bitmap 0x%x\n",
+ eth_bitmap);
+ return -EINVAL;
+ }
+ if (fw_port_num != hweight32(eth_bitmap)) {
+ dev_err(dev, "FW inconsistency: port_num=%u, bitmap=0x%x\n",
+ fw_port_num, eth_bitmap);
+ return -EINVAL;
+ }
+ /*
+ * Firmware is ready before probe. Valid port counts are 1/2/4;
+ * 0 (invalid config), 3 (unsupported topology), and >4 (exceeds
+ * hardware max) are all rejected with -EINVAL.
+ */
+ if (fw_port_num == 0 || fw_port_num == 3 ||
+ fw_port_num > NBL_MAX_ETHERNET) {
+ dev_err(dev, "FW reports %u Ethernet ports, unsupported (valid: 1/2/4)\n",
+ fw_port_num);
+ return -EINVAL;
+ }
+ eth_info->eth_num = fw_port_num;
+ /* Intentional design constraint: each PF maps to exactly one
+ * Ethernet port. This couples PF identity to port identity
+ * and is required by nbl_res_get_eth_id() which indexes
+ * eth_info->eth_id[] by relative PF id.
+ */
+ if (res_mgt->resource_info->max_pf != eth_info->eth_num) {
+ dev_err(dev, "Invalid PF-to-port topology: max_pf=%u, eth_num=%u\n",
+ res_mgt->resource_info->max_pf, eth_info->eth_num);
+ return -EINVAL;
+ }
+
+ /*
+ * Original comment said dual-port board eth_id fixed to 0,2;
+ * Code accepts any contiguous valid bitmap bits (0/1 or 0/2 etc).
+ * Firmware only needs to report correct count of active ports,
+ * no hard-coded fixed bit positions required.
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if ((1 << i) & eth_bitmap) {
+ set_bit(i, eth_info->eth_bitmap);
+ eth_info->eth_id[eth_num] = i;
+ eth_info->logic_eth_id[i] = eth_num;
+ eth_num++;
+ }
+ }
+
+ for (i = 0; i < res_mgt->resource_info->max_pf; i++) {
+ eth_id = eth_info->eth_id[i];
+ eth_info->pf_bitmap[eth_id] |= BIT(i);
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_sriov_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ struct device *dev = common->dev;
+ u8 hw_bus = 0;
+ u16 function;
+ u16 func_id;
+
+ hw_ops->get_real_bus(p, &hw_bus);
+ if (common->function + res_mgt->resource_info->max_pf > NBL_MAX_PF) {
+ dev_err(dev, "PF count exceeds available function space\n");
+ return -EINVAL;
+ }
+ sriov_info = devm_kcalloc(dev, res_mgt->resource_info->max_pf,
+ sizeof(*sriov_info), GFP_KERNEL);
+ if (!sriov_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->sriov_info = sriov_info;
+ /*
+ * common->hw_bus supplies bus number for channel mailbox QINFO mapping.
+ * Execution order guarantee: this assignment runs before
+ * cfg_chan_qinfo_map_table() in nbl_dev_start(), only executed
+ * on control PF path.
+ */
+ common->hw_bus = hw_bus;
+
+ for (func_id = 0; func_id < res_mgt->resource_info->max_pf; func_id++) {
+ sriov_info = res_mgt->resource_info->sriov_info + func_id;
+ function = common->function + func_id;
+ sriov_info->bdf = PCI_DEVID(common->hw_bus,
+ PCI_DEVFN(common->devid, function));
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_vsi_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ struct nbl_vsi_info *vsi_info;
+ int i;
+
+ vsi_info = devm_kzalloc(dev, sizeof(*vsi_info), GFP_KERNEL);
+ if (!vsi_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->vsi_info = vsi_info;
+ /*
+ * case 1 one port(1pf)
+ * pf0 (NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0
+ * case 2 two port(2pf)
+ * pf0,pf1(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,512
+ * case 3 four port(4pf)
+ * pf0,pf1,pf2,pf3(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,256,512,768
+ */
+
+ vsi_info->num = eth_info->eth_num;
+ /*
+ * eth_num can be 1/2/4:
+ * - 2/4 ports use dedicated gap constants;
+ * - 1 port falls back to NBL_DEFAULT_VSI_ID_GAP (1024).
+ * All three values produce valid base_id offsets.
+ */
+ for (i = 0; i < vsi_info->num; i++) {
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].base_id =
+ i * nbl_vsi_id_gap(vsi_info->num);
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].num = 1;
+ }
+
+ return 0;
+}
+
+static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ u32 exp_contiguous_mask = 0;
+ u32 pf_mask = 0;
+ u32 pf_num = 0;
+ int i;
+
+ hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
+
+ /*
+ * k_pf_mask register rule:
+ * bit N == 0 -> PF#N enabled; bit N == 1 -> PF#N masked out.
+ * Hardware constraint: bit0 is reserved, PF0 cannot be masked.
+ * All-zero pf_mask means all PF0~PF7 are enabled.
+ *
+ * Product firmware constraint: only 3 valid configurations supported:
+ * 1 PF (PF0 only): pf_num = 1, mask = 0xfe
+ * 2 PFs (PF0,PF1): pf_num = 2, mask = 0xfc
+ * 4 PFs (PF0~PF3): pf_num = 4, mask = 0xf0
+ * No other PF count or sparse/non-contiguous PF layout is allowed.
+ */
+ for (i = 0; i < NBL_MAX_PF; i++) {
+ if (!(pf_mask & (1 << i)))
+ pf_num++;
+ }
+
+ /*
+ * Sanity check: enabled PFs must be contiguous starting from PF0.
+ * Current resource framework uses relative PF id, sparse PF layout
+ * will cause mismatch between resource layer and hardware func_id.
+ */
+ for (i = 0; i < pf_num; i++)
+ exp_contiguous_mask |= BIT(i);
+ if ((pf_mask & exp_contiguous_mask) != 0) {
+ dev_err(res_mgt->common->dev,
+ "pf_mask 0x%08x: non-contiguous enabled PF, unsupported\n",
+ pf_mask);
+ return -EINVAL;
+ }
+
+ /* Only allow product-specified PF count: 1 / 2 / 4 */
+ if (pf_num != 1 && pf_num != 2 && pf_num != 4) {
+ dev_err(res_mgt->common->dev,
+ "Invalid pf_num=%u (mask=0x%08x), only 1/2/4 PFs supported\n",
+ pf_num, pf_mask);
+ return -EINVAL;
+ }
+
+ res_mgt->resource_info->max_pf = pf_num;
+
+ return 0;
+}
+
+static void nbl_res_init_board_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+
+ hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
+ &res_mgt->resource_info->board_info);
+}
+
+static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ int ret = 0;
+
+ if (common->has_ctrl) {
+ nbl_res_init_board_info(res_mgt);
+
+ ret = nbl_res_init_pf_num(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_sriov_info_init(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_setup_eth_info(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int nbl_res_init_leonis(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl = adap->intf.channel_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adap->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adap->common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_resource_mgt *res_mgt;
+ int ret;
+
+ res_mgt = nbl_res_setup_res_mgt(common);
+ if (IS_ERR(res_mgt)) {
+ ret = PTR_ERR(res_mgt);
+ return ret;
+ }
+ res_mgt->chan_ops_tbl = chan_ops_tbl;
+ res_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ ret = nbl_res_start(res_mgt);
+ if (ret)
+ return ret;
+
+ res_ops_tbl = nbl_res_setup_ops(dev, res_mgt);
+ if (IS_ERR(res_ops_tbl)) {
+ ret = PTR_ERR(res_ops_tbl);
+ return ret;
+ }
+ adap->intf.resource_ops_tbl = res_ops_tbl;
+ adap->core.res_mgt = res_mgt;
+
+ return 0;
+}
+
+void nbl_res_remove_leonis(struct nbl_adapter *adap)
+{
+ /*
+ * No resource release here because all memory uses devm managed
+ * allocation
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
new file mode 100644
index 000000000000..b9355262c00d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_LEONIS_H_
+#define _NBL_RESOURCE_LEONIS_H_
+
+#include "../nbl_resource.h"
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
new file mode 100644
index 000000000000..6fa0e0d550f4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -0,0 +1,120 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/pci.h>
+#include "nbl_resource.h"
+
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ enum nbl_vsi_serv_type dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !vsi_id) {
+ dev_dbg(dev, "No control plane or null vsi output ptr\n");
+ return -EINVAL;
+ }
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= vsi_info->num) {
+ dev_err(dev, "PF %d (diff=%u) exceeds vsi_info->num (%u)\n",
+ pfid, rel_pf_id, vsi_info->num);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_pf_dev_vsi_type_to_hw_vsi_type(res_mgt, type, &dst_type);
+ if (ret) {
+ dev_err(dev, "Invalid vsi type %u func_id %u\n", type, func_id);
+ return ret;
+ }
+ *vsi_id = vsi_info->serv_info[rel_pf_id][dst_type].base_id;
+ return 0;
+}
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int j = NBL_VSI_SERV_PF_DATA_TYPE;
+ int pf_id, i;
+
+ if (!common->has_ctrl) {
+ dev_dbg(dev, "No control plane available\n");
+ return -EINVAL;
+ }
+ for (i = 0; i < vsi_info->num; i++) {
+ if (vsi_id >= vsi_info->serv_info[i][j].base_id &&
+ (vsi_id < vsi_info->serv_info[i][j].base_id +
+ vsi_info->serv_info[i][j].num)) {
+ pf_id = i + common->mgt_pf;
+ if (pf_id >= NBL_MAX_PF) {
+ dev_err(dev, "PF ID overflow\n");
+ return -ERANGE;
+ }
+ return pf_id;
+ }
+ }
+
+ dev_dbg(dev, "VSI ID %u not found\n", vsi_id);
+ return -ENOENT;
+}
+
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ int rel_pf_id;
+ int abs_pf_id;
+
+ if (!common->has_ctrl || !eth_num || !eth_id || !logic_eth_id)
+ return -EINVAL;
+ abs_pf_id = nbl_res_vsi_id_to_pf_id(res_mgt, vsi_id);
+ if (abs_pf_id < 0) {
+ dev_err(dev, "Failed to get PF ID from VSI ID %u\n", vsi_id);
+ return -EINVAL;
+ }
+ if (abs_pf_id != pfid) {
+ dev_err(dev, "func_id %u does not match pf derived from vsi_id %u\n",
+ pfid, vsi_id);
+ return -EINVAL;
+ }
+ rel_pf_id = abs_pf_id - common->mgt_pf;
+
+ if (rel_pf_id >= eth_info->eth_num) {
+ dev_err(dev, "rel_pf_id %d out of range [0, %u)\n",
+ rel_pf_id, eth_info->eth_num);
+ return -ERANGE;
+ }
+
+ *eth_num = eth_info->eth_num;
+ *eth_id = eth_info->eth_id[rel_pf_id];
+ *logic_eth_id = rel_pf_id;
+ return 0;
+}
+
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type)
+{
+ switch (src_type) {
+ case NBL_VSI_DATA:
+ *dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ return 0;
+ default:
+ dev_err_once(res_mgt->common->dev,
+ "Unsupported vsi src_type %u\n", src_type);
+ return -EINVAL;
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
new file mode 100644
index 000000000000..226903be080f
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_H_
+#define _NBL_RESOURCE_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_resource_mgt;
+
+/* --------- INFO ---------- */
+struct nbl_sriov_info {
+ unsigned int bdf;
+};
+
+struct nbl_eth_info {
+ DECLARE_BITMAP(eth_bitmap, NBL_MAX_ETHERNET);
+ u8 pf_bitmap[NBL_MAX_ETHERNET];
+ u8 eth_num;
+ u8 resv[3];
+ u8 eth_id[NBL_MAX_ETHERNET];
+ u8 logic_eth_id[NBL_MAX_ETHERNET];
+};
+
+enum nbl_vsi_serv_type {
+ NBL_VSI_SERV_PF_DATA_TYPE,
+ NBL_VSI_SERV_MAX_TYPE,
+};
+
+struct nbl_vsi_serv_info {
+ u16 base_id;
+ u16 num;
+};
+
+struct nbl_vsi_info {
+ u16 num;
+ struct nbl_vsi_serv_info serv_info[NBL_MAX_ETHERNET]
+ [NBL_VSI_SERV_MAX_TYPE];
+};
+
+struct nbl_resource_info {
+ struct nbl_sriov_info *sriov_info;
+ struct nbl_eth_info *eth_info;
+ struct nbl_vsi_info *vsi_info;
+ u8 max_pf;
+ struct nbl_board_port_info board_info;
+};
+
+struct nbl_resource_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_info *resource_info;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+};
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 4fc987c0e56d..61dd97c779ef 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -39,6 +39,16 @@ enum nbl_chan_state {
NBL_CHAN_STATE_NBITS
};
+struct nbl_board_port_info {
+ u8 eth_num;
+ u8 eth_speed;
+ u8 p4_version;
+ u8 rsv[5];
+};
+
+static_assert(sizeof(struct nbl_board_port_info) == 8,
+ "nbl_board_port_info size must be 8 bytes");
+
struct nbl_chan_send_info {
void *arg;
size_t arg_len;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index 4916c384611b..b3bdecb23b47 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -11,6 +11,22 @@
#include <linux/device.h>
#include "nbl_include.h"
+#define NBL_TWO_ETHERNET_PORT 2
+#define NBL_FOUR_ETHERNET_PORT 4
+#define NBL_DEFAULT_VSI_ID_GAP 1024
+#define NBL_TWO_ETHERNET_VSI_ID_GAP 512
+#define NBL_FOUR_ETHERNET_VSI_ID_GAP 256
+
+static inline u32 nbl_vsi_id_gap(u32 m)
+{
+ if (m == NBL_FOUR_ETHERNET_PORT)
+ return NBL_FOUR_ETHERNET_VSI_ID_GAP;
+ else if (m == NBL_TWO_ETHERNET_PORT)
+ return NBL_TWO_ETHERNET_VSI_ID_GAP;
+
+ return NBL_DEFAULT_VSI_ID_GAP;
+}
+
struct nbl_common_info {
struct workqueue_struct *wq;
struct pci_dev *pdev;
@@ -25,6 +41,7 @@ struct nbl_common_info {
u8 devid;
u8 bus;
u8 hw_bus;
+ u16 mgt_pf;
u8 has_ctrl;
u8 has_net;
@@ -40,6 +57,8 @@ struct nbl_hash_tbl_key {
void nbl_common_destroy_wq(struct nbl_common_info *common);
int nbl_common_create_wq(struct nbl_common_info *common);
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id);
struct nbl_hash_tbl_mgt *
nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index 587ac0c58c24..ee53f9e10a8e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
+struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
@@ -34,9 +35,13 @@ struct nbl_hw_ops {
* or other PF counts are unsupported by driver resource management.
*/
void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+ void (*get_real_bus)(struct nbl_hw_mgt *hw_mgt, u8 *bus);
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
+ void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board);
};
struct nbl_hw_ops_tbl {
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
new file mode 100644
index 000000000000..7136b282fb80
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_RESOURCE_H_
+#define _NBL_DEF_RESOURCE_H_
+
+#include <linux/types.h>
+
+struct nbl_resource_mgt;
+struct nbl_adapter;
+
+struct nbl_resource_ops {
+ int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id);
+};
+
+struct nbl_resource_ops_tbl {
+ struct nbl_resource_ops *ops;
+ struct nbl_resource_mgt *priv;
+};
+
+int nbl_res_init_leonis(struct nbl_adapter *adapter);
+void nbl_res_remove_leonis(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index f2d802397d98..59e44feab44f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,12 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_ETHERNET 4
+
+enum {
+ NBL_VSI_DATA = 0,
+};
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index b7c80ea54c8d..1aafed2d46d7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -10,6 +10,7 @@
#include "nbl_include/nbl_include.h"
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -27,6 +28,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
adapter->pdev = pdev;
common = &adapter->common;
+ common->mgt_pf = 0;
common->pdev = pdev;
common->dev = &pdev->dev;
common->has_ctrl = param->caps.has_ctrl;
@@ -42,7 +44,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_chan_init_common(adapter);
if (ret)
goto chan_init_fail;
+
+ ret = nbl_res_init_leonis(adapter);
+ if (ret)
+ goto res_init_fail;
return adapter;
+res_init_fail:
+ nbl_chan_remove_common(adapter);
chan_init_fail:
nbl_hw_remove_leonis(adapter);
hw_init_fail:
@@ -51,6 +59,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 05/10] net/nebula-matrix: add intr resource implementation
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (3 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,05/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
` (4 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Introduce nbl_interrupt module to manage driver-wide global MSI-X vector
index space, and extend hw_ops with hardware callbacks to program
vendor-specific internal MSI-X mapping registers, populate MSI-X metadata,
and bind interrupt vectors to PF mailbox channels.
Core interfaces:
1. cfg_msix_map
Allocates global MSI-X indices from independent network/other interrupt
bitmaps (intr_net_bmap, intr_other_bmap). All coherent DMA buffers for
new configuration are allocated upfront; old hardware state is torn down
only after new allocation succeeds to avoid interrupt loss. Writes
MSI-X table DMA address and PF BDF into NBL_PCOMPLETER_FUNCTION_MSIX_MAP.
Physical PCI MSI-X vector allocation lives in device layer via
nbl_dev_init_interrupt_scheme().
2. destroy_msix_map
Recycle global MSI-X vectors, clear hardware MSI-X mappings, release
coherent DMA memory and interrupt descriptor array. A bounded sleep is
inserted prior to DMA deallocation to prevent IOMMU faults from lingering
hardware DMA access after MMIO disables the mapping. Two-stage teardown
avoids torn hardware read states by retaining valid DMA addresses before
clearing hardware valid bits.
3. set_mailbox_irq
Toggle mailbox MSI-X routing for a specific PF by updating
NBL_MAILBOX_QINFO_MAP_REG_ARR with the assigned global vector ID. All
register RMW operations are serialized by per-device reg_lock to prevent
concurrent hardware register corruption.
4. cfg_msix_info
Program PADPT_HOST_MSIX_INFO & PCOMPLETER_HOST_MSIX_FID_TABLE to record
each vector's PF BDF and MSI-X enable mask attributes, with strict
programming/teardown order to avoid inconsistent hardware state.
This module solely manages the chip's internal MSI-X routing table and
software vector allocator. It never calls kernel PCI MSI-X allocation
APIs (pci_alloc_irq_vectors() etc.); physical PCI MSI-X vector
allocation is handled separately by the device layer.
The interrupt manager is instantiated via nbl_intr_mgt_start() during
resource initialization and attached to the resource management context.
Locking note: exclusive serialization of cfg_msix_map / destroy_msix_map /
set_mailbox_irq is guaranteed by the dispatch-layer mutex introduced in
a subsequent RPC framework patch. Callers must hold the corresponding lock.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 152 ++++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 42 ++
.../nbl_hw_leonis/nbl_resource_leonis.c | 11 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 11 +
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 402 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 30 ++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 36 ++
.../nbl/nbl_include/nbl_def_hw.h | 10 +
.../nbl/nbl_include/nbl_def_resource.h | 6 +
.../nbl/nbl_include/nbl_include.h | 1 +
13 files changed, 721 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 3dab9519a277..5aec8e44f5d7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -8,4 +8,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
+ nbl_hw/nbl_interrupt.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 3637c00cdfce..fd6bc992fa0f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -46,6 +46,30 @@ static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
}
+static void nbl_hw_rd_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+}
+
+static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+}
+
static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
u32 len)
{
@@ -91,6 +115,124 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+/*
+ * nbl_hw_set_mailbox_irq - read-modify-write NBL_MAILBOX_QINFO_MAP_REG_ARR
+ *
+ * The full RMW sequence is wrapped by reg_lock, so concurrent register
+ * access from different CPUs is already serialized safely.
+ * nbl_hw_cfg_mailbox_qinfo() overwrites the entire register during init,
+ * which unconditionally clears MSIX_IDX and MSIX_IDX_VALID bits, disabling
+ * mailbox MSIX interrupt routing for this PF.
+ */
+static void nbl_hw_set_mailbox_irq(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id)
+{
+ u32 data = 0;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ data &= ~(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK |
+ NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK);
+ if (en_msix)
+ data |= FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK,
+ global_vec_id) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK,
+ 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+ nbl_flush_writes(hw_mgt);
+}
+
+static void nbl_hw_cfg_msix_map(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function)
+{
+ struct nbl_function_msix_map function_msix_map;
+
+ memset(&function_msix_map, 0, sizeof(function_msix_map));
+ if (valid) {
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ /* use ctrl dev's bdf, because the dma memory was
+ * allocated by it
+ */
+ function_msix_map.data[2] =
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK,
+ function) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_BUS_MASK, bus) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_VALID_MASK, 1);
+ } else {
+ /*
+ * reg_lock prevents concurrent CPU writes to the same
+ * function's MSIX entry, but cannot synchronize hardware DMA
+ * reads. Upper layer uses two-stage destruction + sync sleep
+ * to avoid torn hardware read of partial MSIX entry.
+ * Keep valid live dma address here, only clear VALID flag.
+ */
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ function_msix_map.data[2] = 0;
+ }
+
+ nbl_hw_wr_regs_lock(hw_mgt,
+ NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(func_id),
+ function_msix_map.data, sizeof(function_msix_map));
+}
+
+static void nbl_hw_cfg_msix_info(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function, bool msix_mask_en)
+{
+ u32 host_msix_fid = 0;
+ struct nbl_host_msix_info msix_info;
+
+ memset(&msix_info, 0, sizeof(msix_info));
+ if (valid) {
+ host_msix_fid =
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK,
+ func_id) |
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK,
+ 1);
+
+ msix_info.data[1] =
+ FIELD_PREP(NBL_HOST_MSIX_INFO_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_BUS_MASK, bus) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_VALID_MASK, 1);
+
+ if (msix_mask_en)
+ msix_info.data[1] |=
+ FIELD_PREP(NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK, 1);
+ }
+ spin_lock(&hw_mgt->reg_lock);
+ /*
+ * Programming order rule:
+ * Enable: PADPT_HOST_MSIX_INFO -> PCOMPLETER_HOST_MSIX_FID_TABLE
+ * Teardown: reverse order, clear FID VLD first to avoid inconsistent
+ * state
+ */
+ if (valid) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ } else {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -212,6 +354,10 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .cfg_msix_map = nbl_hw_cfg_msix_map,
+ .cfg_msix_info = nbl_hw_cfg_msix_info,
+ .flush_write = nbl_flush_writes,
+
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
.config_mailbox_txq = nbl_hw_config_mailbox_txq,
@@ -221,6 +367,7 @@ static struct nbl_hw_ops hw_ops = {
.get_real_bus = nbl_hw_get_real_bus,
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .set_mailbox_irq = nbl_hw_set_mailbox_irq,
.get_fw_eth_map = nbl_hw_get_fw_eth_map,
.get_board_info = nbl_hw_get_board_info,
@@ -251,11 +398,12 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
if (!hw_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ if (!hw_ops.cfg_msix_map || !hw_ops.cfg_msix_info ||
+ !hw_ops.flush_write || !hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
!hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
- !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.cfg_mailbox_qinfo || !hw_ops.set_mailbox_irq ||
!hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 88772a11124a..fed2fb16bff8 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -68,6 +68,48 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+
+/* -------- HOST_PADPT -------- */
+/* host_padpt host_msix_info */
+#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
+ (vector_id) * sizeof(struct nbl_host_msix_info))
+
+#define NBL_HOST_MSIX_INFO_DWLEN 2
+/* data[0] */
+#define NBL_HOST_MSIX_INFO_INTRL_PNUM_MASK GENMASK(15, 0)
+#define NBL_HOST_MSIX_INFO_INTRL_RATE_MASK GENMASK(31, 16)
+/* data[1] */
+#define NBL_HOST_MSIX_INFO_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_HOST_MSIX_INFO_DEVID_MASK GENMASK(7, 3)
+#define NBL_HOST_MSIX_INFO_BUS_MASK GENMASK(15, 8)
+#define NBL_HOST_MSIX_INFO_VALID_MASK BIT(16)
+#define NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK BIT(17)
+struct nbl_host_msix_info {
+ u32 data[NBL_HOST_MSIX_INFO_DWLEN];
+};
+
+/* -------- HOST_PCOMPLETER -------- */
+/* pcompleter_host pcompleter_host_virtio_qid_map_table */
+#define NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x00004000 + \
+ (i) * sizeof(struct nbl_function_msix_map))
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x0003a000 + (i) * sizeof(u32))
+
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK GENMASK(9, 0)
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK BIT(10)
+
+#define NBL_FUNC_MSIX_MAP_DWLEN 4
+/* data[2] */
+#define NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_FUNCTION_MSIX_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_FUNCTION_MSIX_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_FUNCTION_MSIX_MAP_VALID_MASK BIT(16)
+struct nbl_function_msix_map {
+ u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
+};
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index a719cc8fbf8a..4b6a5bc8715a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -10,6 +10,9 @@
static struct nbl_resource_ops res_ops = {
.get_vsi_id = nbl_res_func_id_to_vsi_id,
.get_eth_id = nbl_res_get_eth_id,
+ .cfg_msix_map = nbl_res_intr_cfg_msix_map,
+ .destroy_msix_map = nbl_res_intr_destroy_msix_map,
+ .set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
};
static struct nbl_resource_mgt *
@@ -41,7 +44,9 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
if (!res_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!res_ops.get_vsi_id || !res_ops.get_eth_id)
+ if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
+ !res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
+ !res_ops.set_mailbox_irq)
return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
@@ -287,6 +292,10 @@ static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
if (ret)
return ret;
+
+ ret = nbl_intr_mgt_start(res_mgt);
+ if (ret)
+ return ret;
}
return 0;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index b9355262c00d..6eb4dc9e695a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -7,4 +7,5 @@
#define _NBL_RESOURCE_LEONIS_H_
#include "../nbl_resource.h"
+#include "../nbl_interrupt.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
index 35604bdff2ae..6b4e513e6c48 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -58,6 +58,17 @@ static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
}
+/*
+ * Only call this when has_ctrl=true, which maps enough space
+ * (bar_len - 8192) to cover NBL_HW_DUMMY_REG (0x1300904).
+ * The flow/design guarantees this is only called in the
+ * has_ctrl path.
+ */
+static inline void nbl_flush_writes(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_rd32(hw_mgt, NBL_HW_DUMMY_REG);
+}
+
static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
{
return readl(hw_mgt->mailbox_bar_hw_addr + reg);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
new file mode 100644
index 000000000000..52872264e906
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
@@ -0,0 +1,402 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/bitfield.h>
+#include "nbl_interrupt.h"
+
+#define NBL_MSIX_DMA_SYNC_MIN_US 100
+#define NBL_MSIX_DMA_SYNC_MAX_US 120
+
+static void nbl_intr_release_bitmap(struct nbl_resource_mgt *res_mgt,
+ u16 *vec_buf, u16 cnt)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ u16 bit;
+ u16 i;
+
+ if (!vec_buf || cnt == 0)
+ return;
+
+ for (i = 0; i < cnt; i++) {
+ u16 intr_index = vec_buf[i];
+
+ if (intr_index >= NBL_NET_INTR_BASE) {
+ bit = intr_index - NBL_NET_INTR_BASE;
+ if (bit < NBL_MAX_NET_INTERRUPT)
+ clear_bit(bit, intr_mgt->intr_net_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid net intr index %u\n",
+ intr_index);
+ } else {
+ if (intr_index < NBL_MAX_OTHER_INTERRUPT)
+ clear_bit(intr_index,
+ intr_mgt->intr_other_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid other intr index %u\n",
+ intr_index);
+ }
+ }
+}
+
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_msix_map_table *msix_map_table;
+ u16 *interrupts;
+ u16 intr_num, i;
+ dma_addr_t live_dma;
+
+ if (!res_mgt->common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+
+ intr_num = intr_mgt->func_intr_res[func_id].num_interrupts;
+ interrupts = intr_mgt->func_intr_res[func_id].interrupts;
+ msix_map_table = &intr_mgt->func_intr_res[func_id].msix_map_table;
+
+ if (!interrupts || !msix_map_table->base_addr) {
+ dev_dbg(dev, "No interrupt resources for func %u\n", func_id);
+ return 0;
+ }
+
+ live_dma = msix_map_table->dma;
+
+ /* Step 1: mask each MSIX vector in hardware first */
+ for (i = 0; i < intr_num; i++) {
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, false, interrupts[i],
+ 0, 0, 0, false);
+ }
+
+ nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
+
+ /*
+ * Stage 1 tear down: retain valid DMA address, ONLY clear VALID bit
+ * avoid hardware torn read (VALID=1 & dma_addr=0)
+ */
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ false, live_dma, 0, 0, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Hardware provides no idle status register for MSIX map DMA engine.
+ * Use bounded sleep to mitigate race between posted MMIO disable writes
+ * and ongoing in-flight table read DMA access.
+ * After sleep, hardware no longer performs DMA access to MSIX table.
+ */
+ usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
+
+ /*
+ * Stage 2: hardware has quiesced MSIX table DMA access,
+ * fully zero MSIX map entry safely now
+ */
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ false, 0, 0, 0, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Now safe to release old MSIX DMA memory, prevents devres accumulation
+ * leak Since hardware DMA has quiesced after sleep, no IOMMU fault risk
+ * anymore.
+ */
+ if (msix_map_table->base_addr) {
+ dmam_free_coherent(dev, msix_map_table->size,
+ msix_map_table->base_addr,
+ msix_map_table->dma);
+ }
+
+ /* Release runtime allocated interrupt vector buffer */
+ kfree(intr_mgt->func_intr_res[func_id].interrupts);
+ intr_mgt->func_intr_res[func_id].interrupts = NULL;
+ intr_mgt->func_intr_res[func_id].num_interrupts = 0;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = 0;
+
+ /* Clear stale MSIX table pointers for safety */
+ msix_map_table->base_addr = NULL;
+ msix_map_table->dma = 0;
+ msix_map_table->size = 0;
+
+ return 0;
+}
+
+/**
+ * nbl_res_intr_cfg_msix_map - allocate & program MSI-X mapping table
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @num_net_msix: required net data interrupt vectors
+ * @num_others_msix: required control interrupt vectors
+ * @net_msix_mask_en: enable mask for net interrupt entries
+ *
+ * Allocate interrupt vectors and coherent DMA table in advance;
+ * only destroy old configuration once all allocations succeed.
+ *
+ * Note: There exists a transient window after tearing down old MSI-X hardware
+ * state before programming new mapping. Atomic table swap is unsupported on
+ * current silicon, this gap is accepted as hardware limitation.
+ *
+ * Caller note: this function has no internal locking. Serialization
+ * must be guaranteed at upper dispatch layer.
+ *
+ * Old MSIX table memory will be explicitly freed inside destroy_msix_map()
+ * after waiting for hardware DMA quiesce, so repeated reconfiguration will not
+ * accumulate devres-managed DMA memory.
+ * Return: 0 on success, negative errno on failure
+ */
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_msix_map_table *tmp_msix_tbl = NULL;
+ struct nbl_msix_map_table *official_tbl;
+ struct nbl_msix_map *msix_map_entries;
+ struct device *dev = common->dev;
+ u16 requested, intr_index;
+ u8 bus, devid, function;
+ bool msix_mask_en = false;
+ u16 *tmp_interrupts = NULL;
+ u16 global_vec;
+ int ret = 0;
+ u16 i;
+
+ if (!common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+ if (num_net_msix == 0 && num_others_msix == 0) {
+ dev_err(dev, "MSI-X vector count cannot both be zero\n");
+ return -EINVAL;
+ }
+ if (num_net_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES ||
+ num_others_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "MSI-X count out of limit: net=%u, others=%u\n",
+ num_net_msix, num_others_msix);
+ return -EINVAL;
+ }
+
+ if (check_add_overflow(num_net_msix, num_others_msix, &requested) ||
+ requested > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "Total MSI-X vectors %u exceeds maximum %u\n",
+ requested, NBL_MSIX_MAP_TABLE_MAX_ENTRIES);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_func_id_to_bdf(res_mgt, func_id, &bus, &devid, &function);
+ if (ret)
+ return ret;
+
+ /*
+ * Phase1: Pre-allocate ALL new resources first.
+ * Do NOT destroy old configuration before all allocations succeed.
+ */
+ tmp_msix_tbl = kzalloc_obj(*tmp_msix_tbl);
+ if (!tmp_msix_tbl) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ tmp_msix_tbl->size =
+ sizeof(struct nbl_msix_map) * NBL_MSIX_MAP_TABLE_MAX_ENTRIES;
+ /*
+ * Hardware requires fixed stride table layout; allocate full size
+ * even when only partial entries are used. Memory managed by devm.
+ */
+ tmp_msix_tbl->base_addr = dmam_alloc_coherent(dev, tmp_msix_tbl->size,
+ &tmp_msix_tbl->dma,
+ GFP_KERNEL);
+ if (!tmp_msix_tbl->base_addr) {
+ dev_err(dev, "Failed to allocate DMA memory for MSIX table\n");
+ ret = -ENOMEM;
+ goto free_tmp_tbl;
+ }
+
+ tmp_interrupts = kcalloc(requested, sizeof(tmp_interrupts[0]),
+ GFP_KERNEL);
+ if (!tmp_interrupts) {
+ ret = -ENOMEM;
+ goto free_tmp_tbl;
+ }
+
+ /* Allocate net interrupt vectors */
+ for (i = 0; i < num_net_msix; i++) {
+ intr_index = find_first_zero_bit(intr_mgt->intr_net_bmap,
+ NBL_MAX_NET_INTERRUPT);
+ if (intr_index == NBL_MAX_NET_INTERRUPT) {
+ dev_err(dev, "No free net interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs;
+ }
+ tmp_interrupts[i] = intr_index + NBL_NET_INTR_BASE;
+ set_bit(intr_index, intr_mgt->intr_net_bmap);
+ }
+
+ /* Allocate other interrupt vectors */
+ for (; i < requested; i++) {
+ intr_index =
+ find_first_zero_bit(intr_mgt->intr_other_bmap,
+ NBL_MAX_OTHER_INTERRUPT);
+ if (intr_index == NBL_MAX_OTHER_INTERRUPT) {
+ dev_err(dev, "No free control interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs;
+ }
+ tmp_interrupts[i] = intr_index;
+ set_bit(intr_index, intr_mgt->intr_other_bmap);
+ }
+
+ /*
+ * Phase2: All new resource allocation succeeded.
+ * Now tear down old MSIX hardware configuration.
+ */
+ ret = nbl_res_intr_destroy_msix_map(res_mgt, func_id);
+ if (ret)
+ goto release_vecs;
+
+ /* Swap temporary resources into official entry */
+ official_tbl = &intr_mgt->func_intr_res[func_id].msix_map_table;
+ official_tbl->base_addr = tmp_msix_tbl->base_addr;
+ official_tbl->dma = tmp_msix_tbl->dma;
+ official_tbl->size = tmp_msix_tbl->size;
+ kfree(tmp_msix_tbl);
+ tmp_msix_tbl = NULL;
+
+ intr_mgt->func_intr_res[func_id].interrupts = tmp_interrupts;
+ intr_mgt->func_intr_res[func_id].num_interrupts = requested;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = num_net_msix;
+ tmp_interrupts = NULL;
+
+ /* Fill MSIX map table and program hardware */
+ msix_map_entries = official_tbl->base_addr;
+ for (i = 0; i < requested; i++) {
+ global_vec = intr_mgt->func_intr_res[func_id].interrupts[i];
+ msix_map_entries[i].data =
+ cpu_to_le16(FIELD_PREP(NBL_MSIX_MAP_VALID_MASK, 1) |
+ FIELD_PREP(NBL_MSIX_MAP_INDEX_MASK,
+ global_vec));
+
+ msix_mask_en = (i < num_net_msix && net_msix_mask_en) ? true :
+ false;
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, true, global_vec,
+ bus, devid, function,
+ msix_mask_en);
+ }
+
+ /* Flush CPU writes to coherent memory before hardware DMA access */
+ dma_wmb();
+
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ true, official_tbl->dma, common->hw_bus,
+ common->devid, common->function);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+out:
+ return ret;
+
+release_vecs:
+ nbl_intr_release_bitmap(res_mgt, tmp_interrupts, i);
+free_tmp_tbl:
+ /* Release DMA buffer allocated by dmam_alloc_coherent first */
+ if (tmp_msix_tbl && tmp_msix_tbl->base_addr) {
+ dmam_free_coherent(dev, tmp_msix_tbl->size,
+ tmp_msix_tbl->base_addr,
+ tmp_msix_tbl->dma);
+ }
+ kfree(tmp_msix_tbl);
+ kfree(tmp_interrupts);
+ tmp_msix_tbl = NULL;
+ tmp_interrupts = NULL;
+ goto out;
+}
+
+/**
+ * nbl_res_intr_set_mailbox_irq - bind mailbox IRQ to specified vector
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @vector_id: index inside local interrupt array
+ * @en_msix: enable/disable mailbox interrupt
+ *
+ * Caller note: this function has no internal locking. Serialization
+ * must be guaranteed at upper dispatch layer.
+ *
+ * Return: 0 on success, negative errno on parameter check or hw failure
+ */
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ u16 global_vec_id;
+
+ if (!common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "func_id %u out of range\n", func_id);
+ return -EINVAL;
+ }
+ if (!intr_mgt->func_intr_res[func_id].interrupts) {
+ dev_err(dev, "func %u MSIX map not configured\n", func_id);
+ return -ENODEV;
+ }
+ if (vector_id >= intr_mgt->func_intr_res[func_id].num_interrupts) {
+ dev_err(dev, "vector_id %u out of range (max %u)\n",
+ vector_id,
+ intr_mgt->func_intr_res[func_id].num_interrupts - 1);
+ return -EINVAL;
+ }
+
+ global_vec_id = intr_mgt->func_intr_res[func_id].interrupts[vector_id];
+ hw_ops->set_mailbox_irq(res_mgt->hw_ops_tbl->priv, func_id,
+ en_msix, global_vec_id);
+
+ return 0;
+}
+
+static struct nbl_interrupt_mgt *nbl_intr_setup_mgt(struct device *dev)
+{
+ struct nbl_interrupt_mgt *intr_mgt;
+
+ intr_mgt = devm_kzalloc(dev, sizeof(*intr_mgt), GFP_KERNEL);
+ if (!intr_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ bitmap_zero(intr_mgt->intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ bitmap_zero(intr_mgt->intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+
+ return intr_mgt;
+}
+
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_interrupt_mgt *intr_mgt;
+ int ret;
+
+ intr_mgt = nbl_intr_setup_mgt(dev);
+ if (IS_ERR(intr_mgt)) {
+ ret = PTR_ERR(intr_mgt);
+ return ret;
+ }
+ res_mgt->intr_mgt = intr_mgt;
+ return 0;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
new file mode 100644
index 000000000000..9f66f5e19c98
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INTERRUPT_H_
+#define _NBL_INTERRUPT_H_
+
+#include "nbl_resource.h"
+
+#define NBL_MSIX_MAP_TABLE_MAX_ENTRIES 1024
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id);
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en);
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
index 6fa0e0d550f4..48d727718f0b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -68,6 +68,36 @@ int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
return -ENOENT;
}
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ int pfid = func_id;
+ u8 pf_bus, devfn;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !bus || !dev || !function)
+ return -EINVAL;
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= res_mgt->resource_info->max_pf) {
+ dev_err(common->dev, "PF ID %u exceeds maximum supported PF count %u\n",
+ pfid, res_mgt->resource_info->max_pf);
+ return -ERANGE;
+ }
+ sriov_info = res_mgt->resource_info->sriov_info + rel_pf_id;
+ pf_bus = PCI_BUS_NUM(sriov_info->bdf);
+ devfn = sriov_info->bdf & 0xff;
+ *bus = pf_bus;
+ *dev = PCI_SLOT(devfn);
+ *function = PCI_FUNC(devfn);
+
+ return 0;
+}
+
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
{
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
index 226903be080f..e2d0f04fdfb7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -17,6 +17,38 @@
struct nbl_resource_mgt;
+/* --------- INTERRUPT ---------- */
+#define NBL_MAX_OTHER_INTERRUPT 1024
+#define NBL_MAX_NET_INTERRUPT 4096
+#define NBL_NET_INTR_BASE NBL_MAX_OTHER_INTERRUPT
+
+#define NBL_MSIX_MAP_VALID_MASK BIT(0)
+#define NBL_MSIX_MAP_INDEX_MASK GENMASK(13, 1)
+#define NBL_MSIX_MAP_RSV_MASK GENMASK(15, 14)
+
+struct nbl_msix_map {
+ __le16 data;
+};
+
+struct nbl_msix_map_table {
+ struct nbl_msix_map *base_addr;
+ dma_addr_t dma;
+ size_t size;
+};
+
+struct nbl_func_interrupt_resource_mng {
+ u16 num_interrupts;
+ u16 num_net_interrupts;
+ u16 *interrupts;
+ struct nbl_msix_map_table msix_map_table;
+};
+
+struct nbl_interrupt_mgt {
+ DECLARE_BITMAP(intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ DECLARE_BITMAP(intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+ struct nbl_func_interrupt_resource_mng func_intr_res[NBL_MAX_FUNC];
+};
+
/* --------- INFO ---------- */
struct nbl_sriov_info {
unsigned int bdf;
@@ -60,13 +92,17 @@ struct nbl_resource_mgt {
struct nbl_resource_info *resource_info;
struct nbl_channel_ops_tbl *chan_ops_tbl;
struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_interrupt_mgt *intr_mgt;
};
int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function);
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt);
int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
u16 src_type,
enum nbl_vsi_serv_type *dst_type);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index ee53f9e10a8e..fd86eef4a0d3 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -12,6 +12,14 @@ struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function);
+ void (*cfg_msix_info)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function,
+ bool net_msix_mask_en);
+ void (*flush_write)(struct nbl_hw_mgt *hw_mgt);
void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx);
void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
@@ -39,6 +47,8 @@ struct nbl_hw_ops {
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*set_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id);
void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
struct nbl_board_port_info *board);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index 7136b282fb80..e718ea41a816 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,12 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id);
+ int (*set_mailbox_irq)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vector_id, bool en_msix);
int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 59e44feab44f..2c959832c32f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,7 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
enum {
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (4 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,06/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
` (3 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add Leonis chip-wide hardware initialization and deinitialization
logic to setup the full datapath pipeline, including packet engines,
scheduler, buffer stores, queue manager, port traffic shaping,
and flow control, plus the driver_status hardware flag consumed
by firmware.
This patch introduces hw_ops.init_module/deinit_module and corresponding
resource_ops entry points. The actual invocation path for these callbacks
lands in a subsequent control device initialization patch; only the ops
registration is added here.
DP sub-init modules (called from nbl_dp_init()):
- dped, uped: Data/User Packet Engine Driver
- dsch: Scheduler core (qid limit init, PSHA enable inside shaping init)
- ustore, dstore: Buffer Store modules
- dvn, uvn: Virtual Descriptor Network modules
- uqm: Queue Manager
- nbl_shaping_init(): Per-port traffic shaping + DSCH PSHA config
Chip init sequence (nbl_hw_init_module()):
1. nbl_dp_init() — All DP sub-modules listed above
2. nbl_intf_init() — Host adapter padpt flow control
- nbl_host_padpt_init() — Host padpt flow control registers
3. nbl_hw_set_driver_status() + nbl_flush_writes()
Introduce firmware quirk interface at mailbox offset NBL_LEONIS_QUIRKS_OFFSET.
Read value ~0u indicates no active quirks. Currently defined quirk:
NBL_QUIRK_UVN_PREFETCH_ALIGN (BIT(1)), BIT(0) is reserved for future use.
Chip deinit design relies on firmware cleanup: only driver_status flag
is cleared and flushed; FW releases all chip hardware state automatically.
The caller guarantees init_module/deinit_module only runs on control PF,
so no extra has_ctrl check is placed inside these entry points.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 23 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 548 +++++++++++++++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 226 ++++++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 5 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nbl/nbl_include/nbl_def_hw.h | 3 +
.../nbl/nbl_include/nbl_def_resource.h | 3 +
.../nbl/nbl_include/nbl_include.h | 21 +
10 files changed, 841 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 5aec8e44f5d7..be314b909d66 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -9,4 +9,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
+ nbl_hw/nbl_chip.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
new file mode 100644
index 000000000000..e1ba7b7e36ca
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include "nbl_chip.h"
+
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+
+ hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
+}
+
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
+{
+ u8 eth_speed = res_mgt->resource_info->board_info.eth_speed;
+ u8 eth_num = res_mgt->resource_info->board_info.eth_num;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
+
+ return hw_ops->init_module(p, eth_speed, eth_num);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
new file mode 100644
index 000000000000..d14093ba916c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHIP_H_
+#define _NBL_CHIP_H_
+
+#include "nbl_resource.h"
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt);
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index fd6bc992fa0f..7f2626db731a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -115,6 +115,548 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+static u32 nbl_hw_get_quirks(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quirks = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_LEONIS_QUIRKS_OFFSET, &quirks,
+ sizeof(u32));
+
+ if (quirks == ~0u)
+ return 0;
+
+ return quirks;
+}
+
+static void nbl_configure_dped_checksum(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 data = 0;
+
+ /* DPED dped_l4_ck_cmd_40 for sctp */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ data |= FIELD_PREP(NBL_DPED_L4_CK_CMD_40_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_dped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_VLAN_OFFSET, 0xC);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_0, 0x8);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_1, 0x4);
+ spin_unlock(&hw_mgt->reg_lock);
+ /* dped checksum offload */
+ nbl_configure_dped_checksum(hw_mgt);
+}
+
+static void nbl_uped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 hw_edit = 0;
+
+ /* V4 TCP: l3_len = 0 */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit &= ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+
+ /* V6 TCP: l3_len = 1 */
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit = (hw_edit & ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK) |
+ FIELD_PREP(NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
+{
+ struct nbl_shaping_dvn_dport_u dvn_dport = { 0 };
+ struct nbl_shaping_dport_u dport = { 0 };
+ u32 rate, half_rate;
+ u32 depth;
+ u64 low_val, high_val;
+
+ switch (speed) {
+ case NBL_FW_PORT_SPEED_100G:
+ rate = 100000;
+ break;
+ case NBL_FW_PORT_SPEED_50G:
+ rate = 50000;
+ break;
+ case NBL_FW_PORT_SPEED_25G:
+ rate = 25000;
+ break;
+ case NBL_FW_PORT_SPEED_10G:
+ rate = 10000;
+ break;
+ default:
+ dev_err(hw_mgt->common->dev,
+ "Unsupported port speed %u for eth%u\n", speed, eth_id);
+ return -EINVAL;
+ }
+
+ half_rate = rate / 2;
+ depth = max_t(u32, rate * 2, NBL_LR_LEONIS_NET_BUCKET_DEPTH);
+
+ /* 1. clear valid first
+ * dport and dvn_dport are zero-initialised above, so VALID=0 already
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 2. write config words (valid=0, safe) */
+ low_val = FIELD_PREP(DPORT_CIR_MASK, rate) |
+ FIELD_PREP(DPORT_PIR_MASK, rate) |
+ FIELD_PREP(DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(DPORT_PBS_MASK, depth);
+ /* Fixed split, independent of host endian */
+ dport.data[0] = lower_32_bits(low_val);
+ dport.data[1] = upper_32_bits(low_val);
+ dport.data[2] = lower_32_bits(high_val);
+ dport.data[3] = upper_32_bits(high_val);
+
+ low_val = FIELD_PREP(DPORT_CIR_MASK, half_rate) |
+ FIELD_PREP(DPORT_PIR_MASK, rate) |
+ FIELD_PREP(DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(DPORT_PBS_MASK, depth);
+ dvn_dport.data[0] = lower_32_bits(low_val);
+ dvn_dport.data[1] = upper_32_bits(low_val);
+ dvn_dport.data[2] = lower_32_bits(high_val);
+ dvn_dport.data[3] = upper_32_bits(high_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 3. commit: set valid last */
+ low_val = FIELD_PREP(DPORT_VALID_MASK, 1);
+ dport.data[0] |= lower_32_bits(low_val);
+
+ low_val = FIELD_PREP(DPORT_VALID_MASK, 1);
+ dvn_dport.data[0] |= lower_32_bits(low_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static int nbl_shaping_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+#define NBL_SHAPING_FLUSH_INTERVAL 128
+ struct nbl_shaping_net_u net_shaping = { 0 };
+ u32 eth_bitmap = 0;
+ u32 psha_en = 0;
+ int ret;
+ int i;
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ ret = nbl_shaping_eth_init(hw_mgt, i, speed);
+ if (ret)
+ return ret;
+ }
+ psha_en = eth_bitmap & GENMASK(3, 0);
+ psha_en = FIELD_PREP(NBL_DSCH_PSHA_EN_MASK, psha_en);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DSCH_PSHA_EN_ADDR, &psha_en,
+ sizeof(psha_en));
+
+ for (i = 0; i < NBL_MAX_FUNC; i++) {
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_SHAPING_NET_REG(i),
+ net_shaping.data,
+ sizeof(net_shaping));
+ if ((i + 1) % NBL_SHAPING_FLUSH_INTERVAL == 0)
+ nbl_flush_writes(hw_mgt);
+ }
+ nbl_flush_writes(hw_mgt);
+ return 0;
+}
+
+static void nbl_dsch_qid_max_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quanta = 0;
+
+ quanta = FIELD_PREP(NBL_DSCH_VN_QUANTA_H_QUA_MASK, NBL_HOST_QUANTA) |
+ FIELD_PREP(NBL_DSCH_VN_QUANTA_E_QUA_MASK, NBL_ECPU_QUANTA);
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSCH_VN_QUANTA_ADDR, &quanta,
+ sizeof(quanta));
+ nbl_hw_wr32(hw_mgt, NBL_DSCH_HOST_QID_MAX, NBL_MAX_QUEUE_ID);
+
+ nbl_hw_wr32(hw_mgt, NBL_DVN_ECPU_QUEUE_NUM, 0);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_ECPU_QUEUE_NUM, 0);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th = 0;
+ u32 pkt_len = 0;
+ int i;
+
+ if (eth_num != 1 && eth_num != 2 && eth_num != 4)
+ return -EINVAL;
+ /* Read current packet length config
+ *(to preserve other fields while updating 'min')
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+ /* min arp packet length 42 (14 + 28) */
+ pkt_len &= ~NBL_USTORE_PKT_LEN_MIN_MASK;
+ pkt_len |= FIELD_PREP(NBL_USTORE_PKT_LEN_MIN_MASK, 42);
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_EN_MASK, 1);
+ if (eth_num == 1)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_SINGLE_ETH_DROP_TH);
+ else if (eth_num == 2)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_DUAL_ETH_DROP_TH);
+ else
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_QUAD_ETH_DROP_TH);
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PORT_DROP_TH_REG_ARR(i),
+ &drop_th, sizeof(drop_th));
+ }
+
+ /* Clear port drop/truncate counters by reading them
+ * (hardware has read-to-clear behavior for these registers)
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_DROP_PKT(i));
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_TRUN_PKT(i));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static void nbl_dstore_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th = 0;
+ u32 fc_th = 0;
+ u32 bp_th = 0;
+ int i;
+
+ for (i = 0; i < NBL_DSTORE_PORT_DROP_TH_DEPTH; i++) {
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ drop_th &= ~NBL_DSTORE_PORT_DROP_EN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ bp_th |= FIELD_PREP(NBL_DSTORE_DISC_BP_TH_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ spin_unlock(&hw_mgt->reg_lock);
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ fc_th &= ~(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK |
+ NBL_DSTORE_D_DPORT_FC_XON_TH_MASK);
+ if (speed == NBL_FW_PORT_SPEED_100G) {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH_100G) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH_100G);
+ } else {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH);
+ }
+
+ fc_th |= FIELD_PREP(NBL_DSTORE_D_DPORT_FC_FC_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+}
+
+static void nbl_dvn_descreq_num_cfg(struct nbl_hw_mgt *hw_mgt, u8 descreq_num)
+{
+ u8 split_ring_num = (descreq_num >> 3) & 0x1;
+ u8 ring_num = descreq_num & 0x7;
+ u32 num_cfg = 0;
+
+ num_cfg = FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK,
+ split_ring_num) |
+ FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK,
+ ring_num);
+
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, &num_cfg,
+ sizeof(num_cfg));
+}
+
+static void nbl_dvn_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 timeout = 0;
+ u32 ro_flag = 0;
+
+ timeout = FIELD_PREP(NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK,
+ DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESC_WR_MERGE_TIMEOUT, &timeout,
+ sizeof(timeout));
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ if (pcie_relaxed_ordering_enabled(hw_mgt->common->pdev)) {
+ ro_flag |=
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1);
+ } else {
+ ro_flag &=
+ ~(FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ spin_unlock(&hw_mgt->reg_lock);
+ if (speed == NBL_FW_PORT_SPEED_100G)
+ nbl_dvn_descreq_num_cfg(hw_mgt,
+ DEFAULT_DVN_100G_DESCREQ_NUMCFG);
+ else
+ nbl_dvn_descreq_num_cfg(hw_mgt, DEFAULT_DVN_DESCREQ_NUMCFG);
+}
+
+static void nbl_uvn_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u16 wr_timeout = NBL_UVN_DESC_WR_TIMEOUT_VAL;
+ u32 timeout = NBL_UVN_DESC_RD_WAIT_TICKS;
+ u32 desc_wr_timeout = 0;
+ u32 prefetch_init = 0;
+ bool ro_enabled;
+ u32 flag = 0;
+ u32 mask = 0;
+ u32 quirks;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_DESC_RD_WAIT, timeout);
+ desc_wr_timeout =
+ FIELD_PREP(NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK, wr_timeout);
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT, &desc_wr_timeout,
+ sizeof(desc_wr_timeout));
+ ro_enabled = pcie_relaxed_ordering_enabled(hw_mgt->common->pdev);
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+ if (ro_enabled) {
+ flag |= FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1);
+ flag &= ~FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1);
+ } else {
+ flag &= ~(FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+ mask |= FIELD_PREP(NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK, 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+
+ spin_unlock(&hw_mgt->reg_lock);
+ quirks = nbl_hw_get_quirks(hw_mgt);
+ /*
+ * sel=0: use configured num; sel=1: use internal calc (max 32)
+ * Default is sel=1, unless NBL_QUIRKS_UVN_PREFETCH_ALIGN is set,
+ * in which case override to sel=0.
+ */
+ prefetch_init =
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK,
+ NBL_UVN_DESC_PREFETCH_NUM) |
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK,
+ (quirks & NBL_QUIRK_UVN_PREFETCH_ALIGN) ? 0 : 1);
+
+ return nbl_hw_wr_regs_lock(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,
+ &prefetch_init, sizeof(prefetch_init));
+}
+
+static void nbl_uqm_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 que_type = 0;
+ u32 cnt = 0;
+ int i;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_FWD_DROP_CNT, &cnt, sizeof(cnt));
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_SLICE_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_LEN_ADD_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_HEAD_PNTR_ADD_CNT, &cnt,
+ sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_WEIGHT_ADD_CNT, &cnt, sizeof(cnt));
+
+ for (i = 0; i < NBL_UQM_PORT_DROP_DEPTH; i++) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_CNT + (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_SLICE_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ }
+
+ for (i = 0; i < NBL_UQM_DPORT_DROP_DEPTH; i++)
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_DPORT_DROP_CNT + (sizeof(cnt) * i), &cnt,
+ sizeof(cnt));
+ /* bit 0: bp mode , bit1: drop mode, resv bit1-31 */
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_QUE_TYPE, &que_type, sizeof(que_type));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_dp_init(struct nbl_hw_mgt *hw_mgt, u8 speed, u8 eth_num)
+{
+ int ret;
+
+ nbl_dped_init(hw_mgt);
+ nbl_uped_init(hw_mgt);
+ ret = nbl_shaping_init(hw_mgt, speed);
+ if (ret)
+ return ret;
+ nbl_dsch_qid_max_init(hw_mgt);
+ ret = nbl_ustore_init(hw_mgt, eth_num);
+ if (ret)
+ return ret;
+ nbl_dstore_init(hw_mgt, speed);
+ nbl_dvn_init(hw_mgt, speed);
+ nbl_uvn_init(hw_mgt);
+ nbl_uqm_init(hw_mgt);
+ return 0;
+}
+
+static void nbl_host_padpt_init(struct nbl_hw_mgt *hw_mgt)
+{
+ /* padpt flow control register */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP,
+ NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PD_DN,
+ NBL_HOST_PADPT_CFG_FC_PD_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PH_DN,
+ NBL_HOST_PADPT_CFG_FC_PH_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN,
+ NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_intf_init(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_host_padpt_init(hw_mgt);
+}
+
+static void nbl_hw_set_driver_status(struct nbl_hw_mgt *hw_mgt, bool active)
+{
+ u32 status;
+
+ spin_lock(&hw_mgt->reg_lock);
+ status = nbl_hw_rd32(hw_mgt, NBL_DRIVER_STATUS_REG);
+
+ status &= ~BIT(NBL_DRIVER_STATUS_BIT);
+ status |= FIELD_PREP(BIT(NBL_DRIVER_STATUS_BIT), active);
+
+ nbl_hw_wr32(hw_mgt, NBL_DRIVER_STATUS_REG, status);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+/*
+ * This design is intentional. Setting driver status to false is the
+ * official teardown mechanism: it notifies firmware to perform full
+ * cleanup of all per-PF hardware state, including qinfo registers.
+ * An inverse helper would duplicate work that the firmware already
+ * does, and would add error-path complexity for no benefit. We keep
+ * the deinit path minimal and rely on firmware cleanup for correctness,
+ * including abnormal hardware reset scenarios.
+ */
+static void nbl_hw_deinit_module(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_set_driver_status(hw_mgt, false);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+}
+
+/*
+ * Full chip hardware initialization is handled by firmware.
+ * This function only configures driver-level table entries and registers.
+ */
+static int nbl_hw_init_module(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
+ u8 eth_num)
+{
+ int ret;
+
+ ret = nbl_dp_init(hw_mgt, eth_speed, eth_num);
+ if (ret)
+ return ret;
+ nbl_intf_init(hw_mgt);
+ nbl_hw_set_driver_status(hw_mgt, true);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+
+ return 0;
+}
+
/*
* nbl_hw_set_mailbox_irq - read-modify-write NBL_MAILBOX_QINFO_MAP_REG_ARR
*
@@ -354,6 +896,9 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .init_module = nbl_hw_init_module,
+ .deinit_module = nbl_hw_deinit_module,
+
.cfg_msix_map = nbl_hw_cfg_msix_map,
.cfg_msix_info = nbl_hw_cfg_msix_info,
.flush_write = nbl_flush_writes,
@@ -404,7 +949,8 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
!hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
!hw_ops.cfg_mailbox_qinfo || !hw_ops.set_mailbox_irq ||
- !hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
+ !hw_ops.get_fw_eth_map || !hw_ops.get_board_info ||
+ !hw_ops.init_module || !hw_ops.deinit_module)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
hw_ops_tbl->priv = hw_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index fed2fb16bff8..95d8a12bdcb5 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,6 +11,9 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+#define NBL_DRIVER_STATUS_REG 0x1300444
+#define NBL_DRIVER_STATUS_BIT 16
+
/* ---------- REG BASE ADDR ---------- */
/* Interface modules base addr */
#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
@@ -70,6 +73,17 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
/* -------- HOST_PADPT -------- */
+#define NBL_HOST_PADPT_HOST_CFG_FC_PD_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000160)
+#define NBL_HOST_PADPT_HOST_CFG_FC_PH_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000164)
+#define NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN \
+ (NBL_INTF_HOST_PADPT_BASE + 0x0000016C)
+#define NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00000170)
+
+#define NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL 0x10400
+#define NBL_HOST_PADPT_CFG_FC_PD_DN_VAL 0x10080
+#define NBL_HOST_PADPT_CFG_FC_PH_DN_VAL 0x10010
+#define NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL 0x10010
/* host_padpt host_msix_info */
#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
(NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
@@ -110,6 +124,216 @@ struct nbl_function_msix_map {
u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
};
+/* ---------- DPED ---------- */
+#define NBL_DPED_VLAN_OFFSET (NBL_DP_DPED_BASE + 0x000003F4)
+#define NBL_DPED_DSCP_OFFSET_0 (NBL_DP_DPED_BASE + 0x000003F8)
+#define NBL_DPED_DSCP_OFFSET_1 (NBL_DP_DPED_BASE + 0x000003FC)
+/* DPED hw_edt_prof/ UPED hw_edt_prof */
+
+#define NBL_DPED_L4_CK_CMD_40_ADDR 0x75c338
+#define NBL_DPED_L4_CK_CMD_40_DEPTH 1
+#define NBL_DPED_L4_CK_CMD_40_WIDTH 32
+#define NBL_DPED_L4_CK_CMD_40_DWLEN 1
+
+#define NBL_DPED_L4_CK_CMD_40_VALUE_MASK GENMASK(7, 0)
+#define NBL_DPED_L4_CK_CMD_40_LEN_IN_OFT_MASK GENMASK(14, 8)
+#define NBL_DPED_L4_CK_CMD_40_LEN_PHID_MASK GENMASK(16, 15)
+#define NBL_DPED_L4_CK_CMD_40_LEN_VLD_MASK BIT(17)
+#define NBL_DPED_L4_CK_CMD_40_DATA_VLD_MASK GENMASK(18, 18)
+#define NBL_DPED_L4_CK_CMD_40_IN_OFT_MASK GENMASK(25, 19)
+#define NBL_DPED_L4_CK_CMD_40_PHID_MASK GENMASK(27, 26)
+#define NBL_DPED_L4_CK_CMD_40_FLAG_MASK BIT(28)
+#define NBL_DPED_L4_CK_CMD_40_MODE_MASK BIT(29)
+#define NBL_DPED_L4_CK_CMD_40_RSV_MASK BIT(30)
+#define NBL_DPED_L4_CK_CMD_40_EN_MASK BIT(31)
+
+/* ---------- UPED ---------- */
+/* UPED uped_hw_edt_prof */
+#define NBL_UPED_HW_EDT_PROF_TABLE(i) \
+ (NBL_DP_UPED_BASE + 0x00001000 + (i) * sizeof(u32))
+#define NBL_UPED_V4_TCP_IDX 5
+#define NBL_UPED_V6_TCP_IDX 6
+#define NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK GENMASK(3, 2)
+
+/* ---------- DSCH ---------- */
+#define NBL_DSCH_PSHA_EN_MASK GENMASK(3, 0)
+/* DSCH dsch maxqid */
+#define NBL_DSCH_HOST_QID_MAX (NBL_DP_DSCH_BASE + 0x00000118)
+#define NBL_DSCH_VN_QUANTA_ADDR (NBL_DP_DSCH_BASE + 0x00000134)
+
+#define NBL_MAX_QUEUE_ID 0x7ff
+#define NBL_HOST_QUANTA 0x8000
+#define NBL_ECPU_QUANTA 0x1000
+
+#define NBL_DSCH_VN_QUANTA_H_QUA_MASK GENMASK(15, 0)
+#define NBL_DSCH_VN_QUANTA_E_QUA_MASK GENMASK(31, 16)
+
+/* ---------- DVN ---------- */
+/* DVN dvn_queue_table */
+#define NBL_DVN_ECPU_QUEUE_NUM (NBL_DP_DVN_BASE + 0x0000041C)
+#define NBL_DVN_DESCREQ_NUM_CFG (NBL_DP_DVN_BASE + 0x00000430)
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT (NBL_DP_DVN_BASE + 0x00000480)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG (NBL_DP_DVN_BASE + 0x0000045C)
+
+#define DEFAULT_DVN_DESCREQ_NUMCFG 0x03
+#define DEFAULT_DVN_100G_DESCREQ_NUMCFG 0x07
+
+#define DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX 0x3FF
+
+/* spilit ring descreq_num 0:8,1:16 */
+#define NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK BIT(0)
+/* packet ring descreq_num
+ * 0:8,1:12,2:16;3:20,4:24,5:26;6:32,7:32
+ */
+#define NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK GENMASK(6, 4)
+
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK GENMASK(9, 0)
+
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK BIT(0)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK BIT(1)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK BIT(2)
+
+/* ---------- UVN ---------- */
+/* UVN uvn_queue_table */
+
+#define NBL_UVN_DESC_RD_WAIT (NBL_DP_UVN_BASE + 0x0000020C)
+#define NBL_UVN_QUEUE_ERR_MASK (NBL_DP_UVN_BASE + 0x00000224)
+#define NBL_UVN_ECPU_QUEUE_NUM (NBL_DP_UVN_BASE + 0x0000023C)
+#define NBL_UVN_DESC_WR_TIMEOUT (NBL_DP_UVN_BASE + 0x00000214)
+#define NBL_UVN_DIF_REQ_RO_FLAG (NBL_DP_UVN_BASE + 0x00000250)
+#define NBL_UVN_DESC_PREFETCH_INIT (NBL_DP_UVN_BASE + 0x00000204)
+#define NBL_UVN_DESC_PREFETCH_NUM 4
+
+#define NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK BIT(0)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK BIT(1)
+#define NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK BIT(2)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK BIT(3)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK GENMASK(14, 0)
+#define NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK BIT(15)
+
+#define NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK BIT(5)
+
+#define NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK GENMASK(7, 0)
+#define NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK BIT(16)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_VAL 0x12c
+/* 200us = 200000ns / 1.67ns per tick = 119760 ticks */
+#define NBL_UVN_DESC_RD_WAIT_TICKS 119760
+
+/* -------- USTORE -------- */
+#define NBL_USTORE_PKT_LEN_ADDR (NBL_DP_USTORE_BASE + 0x00000108)
+#define NBL_USTORE_PORT_DROP_TH_REG_ARR(port_id) \
+ (NBL_DP_USTORE_BASE + 0x00000150 + (port_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_DROP_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002500 + (eth_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_TRUN_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002540 + (eth_id) * sizeof(u32))
+
+#define NBL_USTORE_SINGLE_ETH_DROP_TH 0xC80
+#define NBL_USTORE_DUAL_ETH_DROP_TH 0x640
+#define NBL_USTORE_QUAD_ETH_DROP_TH 0x320
+
+/* USTORE pkt_len */
+#define NBL_USTORE_PKT_LEN_MIN_MASK GENMASK(6, 0)
+
+/* USTORE port_drop_th */
+#define NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK GENMASK(11, 0)
+#define NBL_USTORE_PORT_DROP_TH_EN_MASK BIT(31)
+
+/* UQM*/
+#define NBL_UQM_QUE_TYPE (NBL_DP_UQM_BASE + 0x0000013c)
+#define NBL_UQM_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009C0)
+#define NBL_UQM_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009C4)
+#define NBL_UQM_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x000009C8)
+#define NBL_UQM_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x000009CC)
+#define NBL_UQM_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x000009D0)
+#define NBL_UQM_PORT_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009D4)
+#define NBL_UQM_PORT_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009F4)
+#define NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A14)
+#define NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A34)
+#define NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A54)
+#define NBL_UQM_FWD_DROP_CNT (NBL_DP_UQM_BASE + 0x00000A80)
+#define NBL_UQM_DPORT_DROP_CNT (NBL_DP_UQM_BASE + 0x00000B74)
+
+#define NBL_UQM_PORT_DROP_DEPTH 6
+#define NBL_UQM_DPORT_DROP_DEPTH 16
+
+/* --------- SHAPING --------- */
+
+/* cir 1, bandwidth 1kB/s in protol environment */
+/* cir 1, bandwidth 1Mb/s */
+#define NBL_LR_LEONIS_NET_BUCKET_DEPTH 9600
+#define NBL_SHAPING_DPORT_ADDR 0x504700
+#define NBL_SHAPING_DPORT_DWLEN 4
+#define NBL_SHAPING_DPORT_REG(r) \
+ (NBL_SHAPING_DPORT_ADDR + (NBL_SHAPING_DPORT_DWLEN * 4) * (r))
+#define NBL_SHAPING_DVN_DPORT_ADDR 0x504750
+#define NBL_SHAPING_DVN_DPORT_DWLEN 4
+#define NBL_SHAPING_DVN_DPORT_REG(r) \
+ (NBL_SHAPING_DVN_DPORT_ADDR + (NBL_SHAPING_DVN_DPORT_DWLEN * 4) * (r))
+#define NBL_DSCH_PSHA_EN_ADDR 0x404314
+#define NBL_SHAPING_NET_ADDR 0x505800
+#define NBL_SHAPING_NET_DWLEN 4
+#define NBL_SHAPING_NET_REG(r) \
+ (NBL_SHAPING_NET_ADDR + (NBL_SHAPING_NET_DWLEN * 4) * (r))
+
+#define DPORT_VALID_MASK (0x1ULL << 0)
+#define DPORT_DEPTH_MASK (0x7FFFFULL << 1) // [19:1]
+#define DPORT_CIR_MASK (0x7FFFFULL << 20) // [38:20]
+#define DPORT_PIR_MASK (0x7FFFFULL << 39) // [57:39]
+#define DPORT_CBS_MASK_LOW (0x3FULL << 58) // [63:58]
+#define DPORT_CBS_MASK_HIGH (0x7FFFULL << (0)) // [78:64] -> high[14:0]
+#define DPORT_PBS_MASK (0x1FFFFFULL << (79 - 64)) // [99:79] -> high[35:15]
+
+/* SHAPING shaping_net */
+struct nbl_shaping_net_u {
+ u32 data[NBL_SHAPING_NET_DWLEN];
+};
+
+struct nbl_shaping_dport_u {
+ u32 data[NBL_SHAPING_DPORT_DWLEN];
+};
+
+struct nbl_shaping_dvn_dport_u {
+ u32 data[NBL_SHAPING_DVN_DPORT_DWLEN];
+};
+
+/* -------- DSTORE -------- */
+#define NBL_DSTORE_D_DPORT_FC_TH_ADDR 0x704600
+#define NBL_DSTORE_D_DPORT_FC_TH_DEPTH 5
+#define NBL_DSTORE_D_DPORT_FC_TH_WIDTH 32
+#define NBL_DSTORE_D_DPORT_FC_TH_DWLEN 1
+
+#define NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK GENMASK(10, 0)
+#define NBL_DSTORE_D_DPORT_FC_XON_TH_MASK GENMASK(26, 16)
+#define NBL_DSTORE_D_DPORT_FC_FC_EN_MASK BIT(31)
+
+#define NBL_DSTORE_D_DPORT_FC_TH_REG(r) \
+ (NBL_DSTORE_D_DPORT_FC_TH_ADDR + \
+ (NBL_DSTORE_D_DPORT_FC_TH_DWLEN * 4) * (r))
+#define NBL_DSTORE_PORT_DROP_TH_ADDR 0x704150
+#define NBL_DSTORE_PORT_DROP_TH_DEPTH 6
+#define NBL_DSTORE_PORT_DROP_TH_WIDTH 32
+#define NBL_DSTORE_PORT_DROP_TH_DWLEN 1
+
+#define NBL_DSTORE_PORT_DROP_DISC_TH_MASK GENMASK(9, 0)
+#define NBL_DSTORE_PORT_DROP_EN_MASK BIT(31)
+
+#define NBL_DSTORE_DROP_XOFF_TH 0xC8
+#define NBL_DSTORE_DROP_XON_TH 0x64
+
+#define NBL_DSTORE_DROP_XOFF_TH_100G 0x1F4
+#define NBL_DSTORE_DROP_XON_TH_100G 0x12C
+
+#define NBL_DSTORE_DISC_BP_TH (NBL_DP_DSTORE_BASE + 0x00000630)
+
+#define NBL_DSTORE_DISC_BP_TH_EN_MASK BIT(31)
+
+#define NBL_DSTORE_PORT_DROP_TH_REG(r) \
+ (NBL_DSTORE_PORT_DROP_TH_ADDR + \
+ (NBL_DSTORE_PORT_DROP_TH_DWLEN * 4) * (r))
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
@@ -123,4 +347,6 @@ struct nbl_function_msix_map {
#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+#define NBL_LEONIS_QUIRKS_OFFSET 0x00000140
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index 4b6a5bc8715a..5080c25dc554 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -13,6 +13,8 @@ static struct nbl_resource_ops res_ops = {
.cfg_msix_map = nbl_res_intr_cfg_msix_map,
.destroy_msix_map = nbl_res_intr_destroy_msix_map,
.set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
+ .init_module = nbl_res_chip_init_module,
+ .deinit_module = nbl_res_chip_deinit_module,
};
static struct nbl_resource_mgt *
@@ -46,7 +48,8 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
return ERR_PTR(-ENOMEM);
if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
!res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
- !res_ops.set_mailbox_irq)
+ !res_ops.set_mailbox_irq || !res_ops.init_module ||
+ !res_ops.deinit_module)
return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index 6eb4dc9e695a..f1cb0f23240b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -8,4 +8,5 @@
#include "../nbl_resource.h"
#include "../nbl_interrupt.h"
+#include "../nbl_chip.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index fd86eef4a0d3..e1623b5702d6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -12,6 +12,9 @@ struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ int (*init_module)(struct nbl_hw_mgt *hw_mgt, u8 eth_speed, u8 eth_num);
+ void (*deinit_module)(struct nbl_hw_mgt *hw_mgt);
+
void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
bool valid, dma_addr_t dma_addr, u8 bus,
u8 devid, u8 function);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index e718ea41a816..8dc64e806c1e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,9 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*init_module)(struct nbl_resource_mgt *res_mgt);
+ void (*deinit_module)(struct nbl_resource_mgt *res_mgt);
+
int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 num_net_msix, u16 num_others_msix,
bool net_msix_mask_en);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 2c959832c32f..ebf85702cef6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -30,4 +30,25 @@ struct nbl_init_param {
struct nbl_func_caps caps;
};
+/*
+ * Firmware ABI defines port speed enum fixed, value 0 represents 10G, cannot
+ * reassign 0 to INVALID for compatibility
+ */
+enum nbl_fw_port_speed {
+ NBL_FW_PORT_SPEED_10G,
+ NBL_FW_PORT_SPEED_25G,
+ NBL_FW_PORT_SPEED_50G,
+ NBL_FW_PORT_SPEED_100G,
+};
+
+/*
+ * Firmware quirk word @ NBL_LEONIS_QUIRKS_OFFSET (0x140)
+ * Sentinel value: ~0U (0xFFFFFFFF) = firmware reports no active quirks
+ * BIT(0): NBL_QUIRKS_NO_TOE – ABI defined, driver implementation pending
+ * BIT(1): NBL_QUIRK_UVN_PREFETCH_ALIGN – control UVN descriptor prefetch
+ * selection
+ */
+#define NBL_QUIRKS_NO_TOE BIT(0)
+#define NBL_QUIRK_UVN_PREFETCH_ALIGN BIT(1)
+
#endif
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (5 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
` (2 subsequent siblings)
9 siblings, 0 replies; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Implement core dispatch layer infrastructure for control-plane routing:
Allocate dispatch management structure and dispatch ops table
Provide init_module/deinit_module wrapper callbacks for chip resource ops
Introduce ctrl_lvl bitmask tracking; enable MGT level only for Control PF
Regular PF NET level routing will be added in a subsequent patch.
Kerneldoc on nbl_dispatch_ops documents that init_module/deinit_module
must only be invoked on Control PF behind has_ctrl guard.
This skeleton establishes dispatch management flow. It adds forward
definitions of channel wire ABI structures and response enums required
for later channel RPC framework, without implementing message handling.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 117 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 23 ++++
.../nbl/nbl_include/nbl_def_channel.h | 40 ++++++
.../nbl/nbl_include/nbl_def_dispatch.h | 40 ++++++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 8 ++
7 files changed, 231 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index be314b909d66..b7eebd89b4d1 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -10,4 +10,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
+ nbl_core/nbl_dispatch.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index dd24ebec0171..4d8cea8d8ab3 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -17,12 +17,14 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
struct nbl_resource_ops_tbl *resource_ops_tbl;
+ struct nbl_dispatch_ops_tbl *dispatch_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
+ struct nbl_dispatch_mgt *disp_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
new file mode 100644
index 000000000000..3da5f8351fa4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dispatch.h"
+
+static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->deinit_module)
+ res_ops->deinit_module(p);
+}
+
+static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->init_module)
+ return res_ops->init_module(p);
+ return -EOPNOTSUPP;
+}
+
+static void nbl_disp_set_ctrl_bit(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
+{
+ set_bit(lvl, disp_mgt->ctrl_lvl);
+}
+
+static void nbl_disp_refresh_ctrl_ops(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
+
+ if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
+ disp_ops->init_module = nbl_disp_init_module;
+ disp_ops->deinit_module = nbl_disp_deinit_module;
+ }
+}
+
+static struct nbl_dispatch_mgt *
+nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dispatch_mgt *disp_mgt;
+ struct device *dev = common->dev;
+
+ disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
+ if (!disp_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ disp_mgt->common = common;
+ return disp_mgt;
+}
+
+static struct nbl_dispatch_ops_tbl *
+nbl_disp_setup_ops(struct device *dev, struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_dispatch_ops *disp_ops;
+
+ disp_ops_tbl = devm_kzalloc(dev, sizeof(*disp_ops_tbl), GFP_KERNEL);
+ if (!disp_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops = devm_kzalloc(dev, sizeof(*disp_ops), GFP_KERNEL);
+ if (!disp_ops)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops_tbl->ops = disp_ops;
+ disp_ops_tbl->priv = disp_mgt;
+
+ return disp_ops_tbl;
+}
+
+int nbl_disp_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_resource_ops_tbl *res_ops_tbl =
+ adapter->intf.resource_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_dispatch_mgt *disp_mgt;
+ int ret;
+
+ disp_mgt = nbl_disp_setup_disp_mgt(common);
+ if (IS_ERR(disp_mgt)) {
+ ret = PTR_ERR(disp_mgt);
+ return ret;
+ }
+
+ disp_ops_tbl = nbl_disp_setup_ops(dev, disp_mgt);
+ if (IS_ERR(disp_ops_tbl)) {
+ ret = PTR_ERR(disp_ops_tbl);
+ return ret;
+ }
+
+ disp_mgt->res_ops_tbl = res_ops_tbl;
+ disp_mgt->chan_ops_tbl = chan_ops_tbl;
+ disp_mgt->disp_ops_tbl = disp_ops_tbl;
+ adapter->core.disp_mgt = disp_mgt;
+ adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+
+ if (common->has_ctrl)
+ nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+
+ nbl_disp_refresh_ctrl_ops(disp_mgt);
+ return 0;
+}
+
+void nbl_disp_remove(struct nbl_adapter *adapter)
+{
+ /* All dispatch objects allocated via devm */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
new file mode 100644
index 000000000000..a7e5802344b4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DISPATCH_H_
+#define _NBL_DISPATCH_H_
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_dispatch_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 61dd97c779ef..42d2dc1ebede 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -13,6 +13,12 @@ struct nbl_adapter;
typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+enum {
+ NBL_CHAN_RESP_OK = 0,
+ NBL_CHAN_RESP_ERR = 1,
+ NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+};
+
/*
* Mailbox wire opcodes, stable wire ABI shared between driver and firmware.
* Each opcode has a fixed assigned number to preserve compatibility.
@@ -39,6 +45,32 @@ enum nbl_chan_state {
NBL_CHAN_STATE_NBITS
};
+struct nbl_chan_param_cfg_msix_map {
+ __le16 num_net_msix;
+ __le16 num_others_msix;
+ __le16 msix_mask_en;
+ __le16 rsvd;
+};
+
+struct nbl_chan_param_set_mailbox_irq {
+ __le16 vector_id;
+ u8 en_msix;
+ u8 rsvd;
+};
+
+struct nbl_chan_param_get_vsi_id {
+ __le16 vsi_id;
+ __le16 type;
+};
+
+struct nbl_chan_param_get_eth_id {
+ __le16 vsi_id;
+ u8 eth_num;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 rsvd[3];
+};
+
struct nbl_board_port_info {
u8 eth_num;
u8 eth_speed;
@@ -46,6 +78,14 @@ struct nbl_board_port_info {
u8 rsv[5];
};
+static_assert(sizeof(struct nbl_chan_param_cfg_msix_map) == 8,
+ "nbl_chan_param_cfg_msix_map size must be 8 bytes");
+static_assert(sizeof(struct nbl_chan_param_set_mailbox_irq) == 4,
+ "nbl_chan_param_set_mailbox_irq size must be 4 bytes");
+static_assert(sizeof(struct nbl_chan_param_get_vsi_id) == 4,
+ "nbl_chan_param_get_vsi_id size must be 4 bytes");
+static_assert(sizeof(struct nbl_chan_param_get_eth_id) == 8,
+ "nbl_chan_param_get_eth_id size must be 8 bytes");
static_assert(sizeof(struct nbl_board_port_info) == 8,
"nbl_board_port_info size must be 8 bytes");
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
new file mode 100644
index 000000000000..d5e7da0fef6e
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DISPATCH_H_
+#define _NBL_DEF_DISPATCH_H_
+
+#include <linux/types.h>
+
+struct nbl_dispatch_mgt;
+struct nbl_adapter;
+enum {
+ NBL_DISP_CTRL_LVL_MGT,
+ NBL_DISP_CTRL_LVL_NET,
+ NBL_DISP_CTRL_LVL_MAX,
+};
+
+/**
+ * struct nbl_dispatch_ops - dispatch control plane operation callbacks
+ * @init_module: dispatch layer initialization, ONLY valid on Control PF,
+ * caller must check has_ctrl guard
+ * @deinit_module: dispatch layer cleanup, ONLY valid on Control PF,
+ * caller must check has_ctrl guard
+ * Warning: All ops except init_module/deinit_module can be safely called
+ * on PF/VF; init/deinit hooks are control-PF exclusive to prevent NULL ptr.
+ */
+struct nbl_dispatch_ops {
+ int (*init_module)(struct nbl_dispatch_mgt *disp_mgt);
+ void (*deinit_module)(struct nbl_dispatch_mgt *disp_mgt);
+};
+
+struct nbl_dispatch_ops_tbl {
+ struct nbl_dispatch_ops *ops;
+ struct nbl_dispatch_mgt *priv;
+};
+
+int nbl_disp_init(struct nbl_adapter *adapter);
+void nbl_disp_remove(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 1aafed2d46d7..5d5c0bbf418c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -11,6 +11,7 @@
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
+#include "nbl_include/nbl_def_dispatch.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -48,7 +49,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_res_init_leonis(adapter);
if (ret)
goto res_init_fail;
+
+ ret = nbl_disp_init(adapter);
+ if (ret)
+ goto disp_init_fail;
return adapter;
+disp_init_fail:
+ nbl_res_remove_leonis(adapter);
res_init_fail:
nbl_chan_remove_common(adapter);
chan_init_fail:
@@ -59,6 +66,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (6 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
@ 2026-08-31 2:13 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,08/10] " netdev-bot+sashiko
2026-08-31 2:14 ` [PATCH v26 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-08-31 2:14 ` [PATCH v26 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:13 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add bidirectional mailbox RPC for cross-PF resource coordination,
implement request/response handlers for these operations:
configure_msix_map, destroy_msix_map, set_mailbox_irq, get_vsi_id,
get_eth_id.
Dispatch operations are resolved dynamically based on PF control
capability:
- Control PF invokes local hardware operations directly.
- Non-control PF forwards requests via mailbox RPC to the manager PF.
Introduce per-dispatch mutex ops_mutex_lock to serialize mutable
hardware operations including MSI-X map and mailbox IRQ setup.
This eliminates race windows between local control paths and
asynchronous remote mailbox RPC response handlers.
Read-only get_vsi_id() / get_eth_id() only access static init-time
metadata without concurrent modifications, so they require no
locking.
Add helper to register channel response callbacks; extend wire
protocol with new message types and NBL_CHAN_RESP_PERM_DENY error
code. Translate channel wire status codes to standard Linux errnos
to unify error semantics for upper dispatch consumers.
Improve message payload validation for forward compatibility:
reject requests with insufficient payload length, enforce bounds
checks before parsing incoming RPC parameters. Fix unimplemented
operation check ordering to avoid potential NULL pointer
dereferences.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 535 +++++++++++++++++-
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 2 +
.../nbl/nbl_include/nbl_def_channel.h | 6 +
.../nbl/nbl_include/nbl_def_dispatch.h | 16 +
.../nbl/nbl_include/nbl_include.h | 2 +
5 files changed, 560 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
index 3da5f8351fa4..239d8317a9c5 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -3,9 +3,170 @@
* Copyright (c) 2026 Nebula Matrix Limited.
*/
#include <linux/device.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include "nbl_dispatch.h"
+static int nbl_disp_chan_get_vsi_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.type = cpu_to_le16(type);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_VSI_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ *vsi_id = le16_to_cpu(result.vsi_id);
+ return 0;
+}
+
+static void nbl_disp_chan_get_vsi_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ u16 vsi_id = 0;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->get_vsi_id) {
+ ret = res_ops->get_vsi_id(p, src_id, le16_to_cpu(param.type),
+ &vsi_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ result.vsi_id = cpu_to_le16(vsi_id);
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_VSI_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_VSI_ID);
+}
+
+static int nbl_disp_chan_get_eth_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.vsi_id = cpu_to_le16(vsi_id);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_ETH_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ *eth_num = result.eth_num;
+ *eth_id = result.eth_id;
+ *logic_eth_id = result.logic_eth_id;
+
+ return 0;
+}
+
+static void nbl_disp_chan_get_eth_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->get_eth_id) {
+ ret = res_ops->get_eth_id(p, src_id, le16_to_cpu(param.vsi_id),
+ &result.eth_num, &result.eth_id,
+ &result.logic_eth_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_ETH_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_ETH_ID);
+}
+
static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
{
struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
@@ -25,6 +186,346 @@ static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
return -EOPNOTSUPP;
}
+static int nbl_disp_cfg_msix_map(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->cfg_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, common->mgt_pf, num_net_msix,
+ num_others_msix, net_msix_mask_en);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int
+nbl_disp_chan_cfg_msix_map_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.num_net_msix = cpu_to_le16(num_net_msix);
+ param.num_others_msix = cpu_to_le16(num_others_msix);
+ param.msix_mask_en = cpu_to_le16(!!net_msix_mask_en);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ ¶m, sizeof(param),
+ NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_cfg_msix_map_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->cfg_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, src_id,
+ le16_to_cpu(param.num_net_msix),
+ le16_to_cpu(param.num_others_msix),
+ !!le16_to_cpu(param.msix_mask_en));
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP);
+}
+
+static int nbl_disp_chan_destroy_msix_map_req(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ NULL, 0, NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_destroy_msix_map_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (res_ops->destroy_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, src_id);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_DESTROY_MSIX_MAP);
+}
+
+static int nbl_disp_chan_set_mailbox_irq_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.vector_id = cpu_to_le16(vector_id);
+ param.en_msix = !!en_msix;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ ¶m, sizeof(param), NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_set_mailbox_irq_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ bool en_msix;
+ u16 vector_id;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+ vector_id = le16_to_cpu(param.vector_id);
+ en_msix = !!param.en_msix;
+
+ if (res_ops->set_mailbox_irq) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, src_id, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_MAILBOX_SET_IRQ);
+}
+
+static int nbl_disp_destroy_msix_map(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->destroy_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, common->mgt_pf);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_set_mailbox_irq(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->set_mailbox_irq)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, common->mgt_pf, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_get_vsi_id(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_vsi_id)
+ return res_ops->get_vsi_id(p, common->mgt_pf, type, vsi_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_get_eth_id(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_eth_id)
+ return res_ops->get_eth_id(p, common->mgt_pf, vsi_id,
+ eth_num, eth_id, logic_eth_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_setup_msg(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *p = disp_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+ int _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ nbl_disp_chan_cfg_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ nbl_disp_chan_destroy_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ nbl_disp_chan_set_mailbox_irq_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_VSI_ID,
+ nbl_disp_chan_get_vsi_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_ETH_ID,
+ nbl_disp_chan_get_eth_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ if (ret)
+ chan_ops->unregister_all_msg(p);
+ return ret;
+}
+
static void nbl_disp_set_ctrl_bit(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
{
set_bit(lvl, disp_mgt->ctrl_lvl);
@@ -37,6 +538,18 @@ static void nbl_disp_refresh_ctrl_ops(struct nbl_dispatch_mgt *disp_mgt)
if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
disp_ops->init_module = nbl_disp_init_module;
disp_ops->deinit_module = nbl_disp_deinit_module;
+ disp_ops->cfg_msix_map = nbl_disp_cfg_msix_map;
+ disp_ops->destroy_msix_map = nbl_disp_destroy_msix_map;
+ disp_ops->set_mailbox_irq = nbl_disp_set_mailbox_irq;
+ disp_ops->get_vsi_id = nbl_disp_get_vsi_id;
+ disp_ops->get_eth_id = nbl_disp_get_eth_id;
+ } else {
+ disp_ops->cfg_msix_map =
+ nbl_disp_chan_cfg_msix_map_req;
+ disp_ops->destroy_msix_map = nbl_disp_chan_destroy_msix_map_req;
+ disp_ops->set_mailbox_irq = nbl_disp_chan_set_mailbox_irq_req;
+ disp_ops->get_vsi_id = nbl_disp_chan_get_vsi_id_req;
+ disp_ops->get_eth_id = nbl_disp_chan_get_eth_id_req;
}
}
@@ -45,12 +558,16 @@ nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
{
struct nbl_dispatch_mgt *disp_mgt;
struct device *dev = common->dev;
+ int err;
disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
if (!disp_mgt)
return ERR_PTR(-ENOMEM);
disp_mgt->common = common;
+ err = devm_mutex_init(common->dev, &disp_mgt->ops_mutex_lock);
+ if (err)
+ return ERR_PTR(err);
return disp_mgt;
}
@@ -104,14 +621,30 @@ int nbl_disp_init(struct nbl_adapter *adapter)
adapter->core.disp_mgt = disp_mgt;
adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+ ret = nbl_disp_setup_msg(disp_mgt);
+ if (ret)
+ return ret;
+
if (common->has_ctrl)
nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+ /*
+ * For non-control PF with network capability, enable net control
+ * level.
+ * All dispatch ops declared with NBL_DISP_CTRL_LVL_MGT fall back
+ * to remote mailbox msg_req handlers when MGT bit is not set.
+ */
+ if (common->has_net)
+ nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_NET);
nbl_disp_refresh_ctrl_ops(disp_mgt);
return 0;
}
void nbl_disp_remove(struct nbl_adapter *adapter)
{
- /* All dispatch objects allocated via devm */
+ /*
+ * All dispatch objects allocated via devm
+ * All message handlers will be cleaned up inside channel layer
+ * nbl_chan_remove_common() at final device tear-down
+ */
}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
index a7e5802344b4..8549048f76e9 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -18,6 +18,8 @@ struct nbl_dispatch_mgt {
struct nbl_channel_ops_tbl *chan_ops_tbl;
struct nbl_dispatch_ops_tbl *disp_ops_tbl;
DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+ /* use for the caller not in interrupt */
+ struct mutex ops_mutex_lock;
};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 42d2dc1ebede..e5bc79e13fac 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -17,6 +17,7 @@ enum {
NBL_CHAN_RESP_OK = 0,
NBL_CHAN_RESP_ERR = 1,
NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+ NBL_CHAN_RESP_PERM_DENY = 3,
};
/*
@@ -36,6 +37,11 @@ enum {
*/
enum nbl_chan_msg_type {
NBL_CHAN_MSG_ACK = 0,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP = 17,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP = 18,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ = 19,
+ NBL_CHAN_MSG_GET_VSI_ID = 21,
+ NBL_CHAN_MSG_GET_ETH_ID = 67,
/* mailbox msg end */
NBL_CHAN_MSG_MAILBOX_MAX,
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
index d5e7da0fef6e..61083a750da4 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
@@ -22,12 +22,28 @@ enum {
* caller must check has_ctrl guard
* @deinit_module: dispatch layer cleanup, ONLY valid on Control PF,
* caller must check has_ctrl guard
+ * @cfg_msix_map: configure function msix mapping table
+ * @destroy_msix_map: tear down msix mapping resource
+ * @set_mailbox_irq: bind mailbox interrupt to specified msix vector
+ * @get_vsi_id: resolve VSI ID by type
+ * @get_eth_id: resolve eth port info from VSI ID
+ *
* Warning: All ops except init_module/deinit_module can be safely called
* on PF/VF; init/deinit hooks are control-PF exclusive to prevent NULL ptr.
*/
struct nbl_dispatch_ops {
int (*init_module)(struct nbl_dispatch_mgt *disp_mgt);
void (*deinit_module)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*cfg_msix_map)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*set_mailbox_irq)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix);
+ int (*get_vsi_id)(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
};
struct nbl_dispatch_ops_tbl {
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index ebf85702cef6..a53138530c54 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -15,6 +15,8 @@
#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
+/* Product firmware only supports 1/2/4 contiguous PFs (ID 0~3) */
+#define NBL_MAX_PF_SRC_ID 3
enum {
NBL_VSI_DATA = 0,
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (7 preceding siblings ...)
2026-08-31 2:13 ` [PATCH v26 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
@ 2026-08-31 2:14 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,09/10] " netdev-bot+sashiko
2026-08-31 2:14 ` [PATCH v26 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:14 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
net/nebula-matrix: add common/ctrl dev init/remove operation
Add nbl_dev core infrastructure. Introduce nbl_dev_setup_common_dev()
and nbl_dev_setup_ctrl_dev() initialization helpers with paired
teardown routines, and hook them into nbl_dev_init() and nbl_dev_remove().
Chip core hardware initialization is handled by firmware during power-on.
The driver configures functional table entries after hardware becomes ready,
so calling nbl_dev_setup_common_dev() before ctrl dev setup is safe.
Enforce the deinit sequence: remove ctrl dev first to notify firmware
to clean all per-PF hardware state including qinfo registers. Afterwards
tear down common device queue resources. Firmware cleanup ensures valid
register access during common dev deinit and prevents PCIe master abort.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 1 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 236 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 55 ++++
.../nbl/nbl_include/nbl_def_dev.h | 14 ++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
6 files changed, 316 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index b7eebd89b4d1..71fbe3ee7e62 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -11,4 +11,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
nbl_core/nbl_dispatch.o \
+ nbl_core/nbl_dev.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 4d8cea8d8ab3..c3c4dd685bf6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -25,6 +25,7 @@ struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
struct nbl_dispatch_mgt *disp_mgt;
+ struct nbl_dev_mgt *dev_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
new file mode 100644
index 000000000000..4fc52cadf60f
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dev.h"
+
+static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
+}
+
+/* ---------- Channel config ---------- */
+static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ if (!chan_ops->check_queue_exist(priv, chan_type))
+ return;
+
+ chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
+}
+
+static int nbl_dev_setup_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->setup_queue(priv, chan_type);
+
+ return ret;
+}
+
+static int nbl_dev_remove_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->teardown_queue(priv, chan_type);
+
+ return ret;
+}
+
+static void nbl_dev_register_chan_task(struct nbl_dev_mgt *dev_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ if (chan_ops->check_queue_exist(dev_mgt->chan_ops_tbl->priv, chan_type))
+ chan_ops->register_chan_task(dev_mgt->chan_ops_tbl->priv,
+ chan_type, task);
+}
+
+/* ---------- Tasks config ---------- */
+static void nbl_dev_clean_mailbox_task(struct work_struct *work)
+{
+ struct nbl_dev_common *common_dev =
+ container_of(work, struct nbl_dev_common, clean_mbx_task);
+ struct nbl_dev_mgt *dev_mgt = common_dev->dev_mgt;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ chan_ops->clean_queue_subtask(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_TYPE_MAILBOX);
+}
+
+/* ---------- Dev init process ---------- */
+static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dispatch_mgt *priv = dev_mgt->disp_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+ struct nbl_dev_common *common_dev;
+ int ret;
+
+ common_dev = devm_kzalloc(&adapter->pdev->dev, sizeof(*common_dev),
+ GFP_KERNEL);
+ if (!common_dev)
+ return -ENOMEM;
+ common_dev->dev_mgt = dev_mgt;
+
+ ret = nbl_dev_setup_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
+ &common_dev->clean_mbx_task);
+ /*
+ * Even if has_ctrl=false (no dedicated control PF channel), we fetch
+ * VSI/ETH info via regular mailbox message instead of dedicated
+ * control command.
+ */
+ ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
+ if (ret)
+ goto err_cleanup;
+ ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
+ &common->eth_id, &common->logic_eth_id);
+ if (ret)
+ goto err_cleanup;
+
+ dev_mgt->common_dev = common_dev;
+ nbl_dev_init_msix_cnt(dev_mgt);
+
+ return 0;
+err_cleanup:
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+ return ret;
+}
+
+static void nbl_dev_remove_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+
+ if (!common_dev)
+ return;
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+}
+
+static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ int ret;
+
+ ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
+ if (ret)
+ return ret;
+
+ nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+
+ return 0;
+}
+
+/*
+ * This is intentional. The qinfo registers are managed by the chip
+ * firmware, not by the driver. Setting driver status to false is the
+ * designed teardown mechanism — it notifies the firmware, which then
+ * performs its own cleanup of all per-PF state including the qinfo
+ * registers.
+ * An inverse helper would duplicate work that the firmware already
+ * does, and would add error-path complexity for no benefit. We keep
+ * the deinit path minimal and rely on the firmware cleanup for
+ * correctness, including in abnormal reset scenarios.
+ */
+static void nbl_dev_remove_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ disp_ops->deinit_module(dev_mgt->disp_ops_tbl->priv);
+}
+
+static struct nbl_dev_mgt *nbl_dev_setup_dev_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dev_mgt *dev_mgt;
+
+ dev_mgt = devm_kzalloc(common->dev, sizeof(*dev_mgt), GFP_KERNEL);
+ if (!dev_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ dev_mgt->common = common;
+ return dev_mgt;
+}
+
+int nbl_dev_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl =
+ adapter->intf.dispatch_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct nbl_dev_mgt *dev_mgt;
+ int ret;
+
+ dev_mgt = nbl_dev_setup_dev_mgt(common);
+ if (IS_ERR(dev_mgt)) {
+ ret = PTR_ERR(dev_mgt);
+ return ret;
+ }
+
+ dev_mgt->disp_ops_tbl = disp_ops_tbl;
+ dev_mgt->chan_ops_tbl = chan_ops_tbl;
+ adapter->core.dev_mgt = dev_mgt;
+
+ /*
+ * Chip hardware initialization is completed by firmware at power-up.
+ * Only driver functional table/register config follows here, safe to
+ * access hardware registers before ctrl dev setup.
+ */
+ ret = nbl_dev_setup_common_dev(adapter);
+ if (ret)
+ goto setup_err;
+
+ if (common->has_ctrl) {
+ ret = nbl_dev_setup_ctrl_dev(adapter);
+ if (ret)
+ goto setup_ctrl_dev_fail;
+ }
+
+ return 0;
+setup_ctrl_dev_fail:
+ nbl_dev_remove_common_dev(adapter);
+setup_err:
+ return ret;
+}
+
+/*
+ * Teardown order: ctrl dev first, then common dev.
+ * nbl_dev_remove_ctrl_dev() notifies firmware to clean all per-PF state
+ * (including qinfo registers), so subsequent common dev queue cleanup
+ * will not trigger PCIe master abort or invalid register access.
+ */
+void nbl_dev_remove(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+
+ if (common->has_ctrl)
+ nbl_dev_remove_ctrl_dev(adapter);
+ nbl_dev_remove_common_dev(adapter);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
new file mode 100644
index 000000000000..24e890fd8987
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEV_H_
+#define _NBL_DEV_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_dev.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_STRING_NAME_LEN 32
+
+enum nbl_msix_serv_type {
+ NBL_MSIX_NET_TYPE,
+ NBL_MSIX_MAILBOX_TYPE,
+ NBL_MSIX_TYPE_MAX
+};
+
+struct nbl_msix_serv_info {
+ char irq_name[NBL_STRING_NAME_LEN];
+ u16 num;
+ u16 base_vector_id;
+ /* true: hw report msix, hw need to mask actively */
+ bool hw_self_mask_en;
+};
+
+struct nbl_msix_info {
+ struct nbl_msix_serv_info serv_info[NBL_MSIX_TYPE_MAX];
+};
+
+struct nbl_dev_common {
+ struct nbl_dev_mgt *dev_mgt;
+ struct nbl_msix_info msix_info;
+ char mailbox_name[NBL_STRING_NAME_LEN];
+ /* for ctrl-dev/net-dev mailbox recv msg */
+ struct work_struct clean_mbx_task;
+};
+
+struct nbl_dev_mgt {
+ struct nbl_common_info *common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dev_common *common_dev;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
new file mode 100644
index 000000000000..51cf04e4c552
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DEV_H_
+#define _NBL_DEF_DEV_H_
+
+struct nbl_adapter;
+
+int nbl_dev_init(struct nbl_adapter *adapter);
+void nbl_dev_remove(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 5d5c0bbf418c..9896c1b49be2 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -12,6 +12,7 @@
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_dispatch.h"
+#include "nbl_include/nbl_def_dev.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -53,7 +54,14 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_disp_init(adapter);
if (ret)
goto disp_init_fail;
+
+ ret = nbl_dev_init(adapter);
+ if (ret)
+ goto dev_init_fail;
return adapter;
+
+dev_init_fail:
+ nbl_disp_remove(adapter);
disp_init_fail:
nbl_res_remove_leonis(adapter);
res_init_fail:
@@ -66,6 +74,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_dev_remove(adapter);
nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v26 net-next 10/10] net/nebula-matrix: add common dev start/stop operation
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (8 preceding siblings ...)
2026-08-31 2:14 ` [PATCH v26 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-08-31 2:14 ` illusion.wang
2026-09-03 17:18 ` [v26,net-next,10/10] " netdev-bot+sashiko
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-08-31 2:14 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Implement nbl_dev_start() and nbl_dev_stop() to manage MSI-X mapping,
mailbox interrupt initialization and deinitialization.
nbl_dev_start() performs device startup steps:
- Configure hardware MSI-X mapping table
- Allocate required MSI-X IRQ vectors via pci_alloc_irq_vectors()
- Request mailbox interrupt and attach interrupt handler
- Enable hardware mailbox interrupt and mark channel interrupt ready
nbl_dev_stop() tears down resources with strict ordering to avoid stale
message ACK handling:
- Switch channel software state to polling mode before masking hardware
interrupt
- Release mailbox IRQ and free MSI-X vector resources
- Destroy hardware MSI-X mapping table
Add thin wrappers nbl_core_start() / nbl_core_stop() and hook them into
PCI probe and remove paths to control the device lifecycle.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 268 ++++++++++++++++++
.../nbl/nbl_include/nbl_def_dev.h | 2 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 18 ++
4 files changed, 290 insertions(+)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index c3c4dd685bf6..56872c8ca9a7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -39,5 +39,7 @@ struct nbl_adapter {
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param);
void nbl_core_remove(struct nbl_adapter *adapter);
+int nbl_core_start(struct nbl_adapter *adapter);
+void nbl_core_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
index 4fc52cadf60f..26b785c9a05f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -6,6 +6,17 @@
#include <linux/pci.h>
#include "nbl_dev.h"
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt);
+
+/* ---------- Interrupt config ---------- */
+static irqreturn_t nbl_dev_clean_mailbox(int __always_unused irq, void *data)
+{
+ struct nbl_dev_mgt *dev_mgt = (struct nbl_dev_mgt *)data;
+
+ nbl_dev_clean_mailbox_schedule(dev_mgt);
+ return IRQ_HANDLED;
+}
+
static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
{
struct nbl_dev_common *dev_common = dev_mgt->common_dev;
@@ -14,6 +25,189 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
}
+static int nbl_dev_request_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+ int err;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num < 0) {
+ dev_err(common->dev, "Failed to get mailbox IRQ vector: %d\n",
+ irq_num);
+ return irq_num;
+ }
+
+ snprintf(dev_common->mailbox_name, sizeof(dev_common->mailbox_name),
+ "nbl_mailbox@pci:%s", pci_name(common->pdev));
+ err = request_irq(irq_num, nbl_dev_clean_mailbox, 0,
+ dev_common->mailbox_name, dev_mgt);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static void nbl_dev_free_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num >= 0)
+ free_irq(irq_num, dev_mgt);
+}
+
+static int nbl_dev_enable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_TYPE_MAILBOX, true);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, true);
+}
+
+static int nbl_dev_disable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ /*
+ * Disable sequence invariant: update software state first, then mask
+ * hardware interrupt. Must not reverse the order.
+ *
+ * If hardware interrupt is masked before clearing INTERRUPT_READY,
+ * the hardware may still transmit outstanding ACK packets for in-flight
+ * messages. Subsequent switch to polling mode discards pending ACK
+ * processing, triggering "Channel waiting ack failed" and "Skip ack
+ * with invalid status" errors.
+ *
+ * By entering polling mode first, any late hardware interrupts are
+ * ignored without pending ACK expectations, then hardware interrupt
+ * can be safely disabled.
+ *
+ * This helper is invoked in two paths:
+ * 1. Error unwind path of nbl_dev_start(): followed immediately by
+ * nbl_dev_free_mailbox_irq() and full channel teardown. No new mailbox
+ * interrupts can fire afterwards, and subsequent cancel_work_sync()
+ * drains pending cleanup work before resources are released.
+ * 2. Normal device stop path nbl_dev_stop(): free_irq() blocks until
+ * any in-flight hardirq handler completes and prevents new interrupts.
+ * cancel_work_sync() then waits for any already running mailbox cleanup
+ * work to finish, or cancels queued but unstarted work items before
+ * final channel destruction. No stuck descriptors linger in either
+ * scenario.
+ */
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_TYPE_MAILBOX, false);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, false);
+}
+
+static int nbl_dev_cfg_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ bool mask_en = msix_info->serv_info[NBL_MSIX_NET_TYPE].hw_self_mask_en;
+ u16 msix_net_num = msix_info->serv_info[NBL_MSIX_NET_TYPE].num;
+ u16 msix_not_net_num = 0;
+ int err, i;
+
+ msix_info->serv_info[NBL_MSIX_NET_TYPE].base_vector_id = 0;
+ for (i = NBL_MSIX_NET_TYPE + 1; i < NBL_MSIX_TYPE_MAX; i++)
+ msix_info->serv_info[i].base_vector_id =
+ msix_info->serv_info[i - 1].base_vector_id +
+ msix_info->serv_info[i - 1].num;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++) {
+ if (i == NBL_MSIX_NET_TYPE)
+ continue;
+ msix_not_net_num += msix_info->serv_info[i].num;
+ }
+
+ err = disp_ops->cfg_msix_map(dev_mgt->disp_ops_tbl->priv,
+ msix_net_num, msix_not_net_num,
+ mask_en);
+
+ return err;
+}
+
+static int nbl_dev_destroy_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ return disp_ops->destroy_msix_map(dev_mgt->disp_ops_tbl->priv);
+}
+
+static int nbl_dev_init_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ int needed = 0;
+ int err;
+ int i;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++)
+ needed += msix_info->serv_info[i].num;
+
+ err = pci_alloc_irq_vectors(common->pdev, needed, needed,
+ PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
+ if (err < 0) {
+ dev_err(common->dev,
+ "pci_alloc_irq_vectors failed, err = %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static void nbl_dev_clear_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_common_info *common = dev_mgt->common;
+
+ pci_free_irq_vectors(common->pdev);
+}
+
/* ---------- Channel config ---------- */
static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
{
@@ -73,6 +267,14 @@ static void nbl_dev_clean_mailbox_task(struct work_struct *work)
NBL_CHAN_TYPE_MAILBOX);
}
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ queue_work(common->wq, &common_dev->clean_mbx_task);
+}
+
/* ---------- Dev init process ---------- */
static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
{
@@ -234,3 +436,69 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
nbl_dev_remove_ctrl_dev(adapter);
nbl_dev_remove_common_dev(adapter);
}
+
+/* ---------- Dev start process ---------- */
+int nbl_dev_start(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int cleanup_ret;
+ int ret;
+
+ ret = nbl_dev_cfg_msix_map(dev_mgt);
+ if (ret)
+ goto cfg_msix_map_err;
+
+ ret = nbl_dev_init_interrupt_scheme(dev_mgt);
+ if (ret)
+ goto init_interrupt_scheme_err;
+ ret = nbl_dev_request_mailbox_irq(dev_mgt);
+ if (ret)
+ goto mailbox_request_irq_err;
+ ret = nbl_dev_enable_mailbox_irq(dev_mgt);
+ if (ret)
+ goto enable_mailbox_irq_err;
+
+ return 0;
+
+enable_mailbox_irq_err:
+ cleanup_ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to disable mailbox IRQ: %d\n", cleanup_ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+ cancel_work_sync(&common_dev->clean_mbx_task);
+mailbox_request_irq_err:
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+init_interrupt_scheme_err:
+cfg_msix_map_err:
+ cleanup_ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to destroy MSI-X map: %d\n", cleanup_ret);
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ return ret;
+}
+
+void nbl_dev_stop(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int ret;
+
+ ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to disable mailbox IRQ: %d\n", ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+ ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to destroy MSI-X map: %d\n", ret);
+ /*
+ * destroy_msix_map() sends ack-requested messages which may requeue
+ * clean_mbx_task via polling send path. Drain work after the operation.
+ */
+ cancel_work_sync(&common_dev->clean_mbx_task);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
index 51cf04e4c552..a66c633a0e7a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -10,5 +10,7 @@ struct nbl_adapter;
int nbl_dev_init(struct nbl_adapter *adapter);
void nbl_dev_remove(struct nbl_adapter *adapter);
+int nbl_dev_start(struct nbl_adapter *adapter);
+void nbl_dev_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 9896c1b49be2..02cb35d32012 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -16,6 +16,16 @@
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+int nbl_core_start(struct nbl_adapter *adapter)
+{
+ return nbl_dev_start(adapter);
+}
+
+void nbl_core_stop(struct nbl_adapter *adapter)
+{
+ nbl_dev_stop(adapter);
+}
+
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param)
{
@@ -127,7 +137,13 @@ static int nbl_probe(struct pci_dev *pdev,
goto adapter_init_err;
}
pci_set_drvdata(pdev, adapter);
+ err = nbl_core_start(adapter);
+ if (err)
+ goto core_start_err;
+
return 0;
+core_start_err:
+ nbl_core_remove(adapter);
adapter_init_err:
pci_clear_master(pdev);
return err;
@@ -140,6 +156,8 @@ static void nbl_remove(struct pci_dev *pdev)
if (!adapter)
return;
pci_set_drvdata(pdev, NULL);
+
+ nbl_core_stop(adapter);
nbl_core_remove(adapter);
pci_clear_master(pdev);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [v26,net-next,01/10] net/nebula-matrix: add minimum nbl build framework
2026-08-31 2:13 ` [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 0 · Low: 1
- [Low] The documentation file added by this patch,
Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst…
--
net/nebula-matrix: add minimum nbl build framework
This adds the Kconfig/Makefile/MAINTAINERS entries for a new
drivers/net/ethernet/nebula-matrix/ vendor directory, a driver
documentation file, and a PCI driver skeleton (nbl_main.c) whose probe
and remove callbacks are stubs.
> diff --git a/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
> new file mode 100644
> index 000000000000..ff38302968c5
> --- /dev/null
> +++ b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
> @@ -0,0 +1,28 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +================================================================
> +Linux Base Driver for Nebula-matrix m18110-NIC/m18000-NIC family
> +================================================================
> +
> +Overview:
> +=========
> +The m18110-NIC/m18000-NIC is a series of network interface cards for the Data
> +Center Area.
> +
> +The driver supports link-speed 100GbE/25GE/10GE.
> +
> +m18110-NIC/m18000-NIC devices support MSI-X interrupt vector for each Tx/Rx
> +queue and interrupt moderation.
> +
> +m18110-NIC/m18000-NIC devices support also various offload features such as
> +checksum offload, Receive-Side Scaling(RSS).
[Severity: Low]
Do these three statements describe what the driver actually does, either
at this commit or at the end of the series?
Grepping the vendor directory at the last patch of the series
("net/nebula-matrix: add common dev start/stop operation",
9b90629960395de83faaf5146e29ffc3cb854c5b) finds nothing for any of the
netdev or offload plumbing these sentences describe:
git grep -E 'register_netdev|netdev_ops|alloc_etherdev|ethtool_ops|NETIF_F_' \
drivers/net/ethernet/nebula-matrix/
(no matches)
The directory at that point contains only nbl_main.c plus nbl_channel/,
nbl_common/, nbl_core/, nbl_hw/ and nbl_include/, so the code binds the
PCI function, maps the BARs and brings up the PF mailbox, but never
registers a net_device, never programs per-queue Tx/Rx MSI-X vectors, and
has no RSS, checksum offload, interrupt moderation or link-speed
reporting.
Would it make sense to describe only what is implemented, and add the
link speed / per-queue MSI-X / moderation / RSS / checksum text in the
patches that introduce those features?
> +
> +Support
> +=======
> +
> +For more information about m18110-NIC/m18000-NIC, please visit the following URL:
> +https://www.nebula-matrix.com/snic_s1000_en
Is this the right product page? The URL names snic_s1000, while
nbl_main.c documents the matched IDs as "SNIC v3r1 product Device IDs
range: 0x3403-0x3412" for m18110/m18000.
> +
> +If an issue is identified with the released source code on the supported kernel
> +with a supported adapter, email the specific information related to the issue to
> +open@nebula-matrix.com.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/Kconfig b/drivers/net/ethernet/nebula-matrix/Kconfig
> new file mode 100644
> index 000000000000..f16e9663eaed
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/Kconfig
[ ... ]
> +config NBL
> + tristate "Nebula-matrix Ethernet Controller m18110/m18000 support"
> + depends on PCI && (64BIT || COMPILE_TEST) && !CPU_BIG_ENDIAN
> + help
> + This driver supports Nebula-matrix Ethernet Controller m18110/m18000
> + Family of devices.
> +
> + More specific information on configuring the driver is in
> + <file:Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst>.
The help text points users at the .rst above as the description of the
option's capabilities, so the same over-claim reaches anyone selecting
CONFIG_NBL: the prompt says "Ethernet Controller m18110/m18000 support",
but no network interface is created by any commit in this series.
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,02/10] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-08-31 2:13 ` [PATCH v26 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 1 · Low: 0
- [Medium] In `nbl_hw_init_leonis()` (drivers/net/ethernet/nebula-
matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c) two error paths print…
--
net/nebula-matrix: add core driver architecture and HW layer initialization
This adds the nbl_adapter/nbl_core/nbl_common_info device context, the PCI
probe/remove entry points with basic capability parsing, and the Leonis
hardware layer that requests the BARs and ioremaps them. One question
below about the BAR size error paths.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> new file mode 100644
> index 000000000000..a67d1a674466
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -0,0 +1,152 @@
[ ... ]
> +int nbl_hw_init_leonis(struct nbl_adapter *adapter)
> +{
> + resource_size_t expect_sz = NBL_MEM_BAR_TOTAL_SIZE;
> + struct nbl_common_info *common = &adapter->common;
> + struct pci_dev *pdev = common->pdev;
> + struct nbl_hw_mgt *hw_mgt = NULL;
> + resource_size_t bar_len;
[ ... ]
> + if (bar_len < NBL_MEM_BAR_TOTAL_SIZE) {
> + dev_err(&pdev->dev,
> + "MEMORY BAR len %pr smaller than expected %pr\n",
> + &bar_len, &expect_sz);
[Severity: Medium]
Should these two be %pa rather than %pr?
Both bar_len and expect_sz are plain resource_size_t locals on the stack,
but %pr is the struct resource specifier. vsprintf hands it to
resource_string(), which dereferences the argument as a struct resource:
lib/vsprintf.c:resource_string() {
...
if (res->flags & IORESOURCE_IO) {
...
p = hex_range(p, pend, res->start, res->end, *specp);
...
}
struct resource places flags at offset 24 on 64-bit, so does this read
roughly 32 bytes past the end of each 8-byte local and print adjacent
kernel stack contents (both the decoded resource type and the second hex
value) into dmesg?
Nothing warns at build time because every %p variant just consumes a
void *. Documentation/core-api/printk-formats.rst lists %pa[p] for
phys_addr_t / resource_size_t, so %pa &bar_len (or a cast to
unsigned long long with %llu) would seem to be what is wanted here.
The path is reached from nbl_probe() -> nbl_core_init() ->
nbl_hw_init_leonis() whenever common->has_ctrl is set (function 0, not a
VF) and pci_resource_len(pdev, NBL_MEMORY_BAR) is below 64MiB.
> + ret = -EINVAL;
> + goto setup_mgt_fail;
> + }
[ ... ]
> + } else {
> + if (bar_len < NBL_REG_NET_ONLY_LEN) {
> + dev_err(&pdev->dev,
> + "MEMORY BAR len %pr too small for net only reg space\n",
> + &bar_len);
[Severity: Medium]
Same %pr versus %pa question on this second site in nbl_hw_init_leonis(),
which fires on a non-ctrl function when the MEMORY BAR is reported smaller
than 8KiB.
This one still appears unchanged at the end of the series, so no later
patch in the set seems to address it.
> + ret = -EINVAL;
> + goto setup_mgt_fail;
> + }
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,03/10] net/nebula-matrix: add channel layer
2026-08-31 2:13 ` [PATCH v26 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 22 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 2 · Low: 20
- [Medium] nbl_chan_get_msg_id() treats a wait slot in
NBL_MBX_STATUS_TIMEOUT as free, but the timed-out owner has not…
- [Medium] nbl_chan_kick_tx_ring() rings the TX doorbell and polls for
NBL_CHAN_TX_DESC_USED; on timeout it executes 'txq->next_to_clean =…
- [Low] Commit message describes an operational mailbox framework (queue
lifecycle, dual RX modes, DMA buffers "allocated once in probe phase")…
- [Low] nbl_chan_setup_queue() re-runs all
dmam_alloc_coherent()/devm_kcalloc() allocations (~2 MB per cycle)…
- [Low] The data paths (nbl_chan_send_msg(), nbl_chan_clean_queue()) gate
only on chan_info->shutdn and never on chan_info->active, so a send or…
- [Low] The commit message claims the patch adds "register lock
protection for hardware register read/write to ensure concurrent…
- [Low] The comment in nbl_chan_cfg_qinfo_map_table() justifies iterating
raw hardware PF func_ids by claiming nbl_res_init_pf_num() already…
- [Low] nbl_hw_rd_regs_lock()/nbl_hw_wr_regs_lock() perform
readl()/writel() at BAR0 offsets 0x1505004 (k_pf_mask) and 0xfb1000+…
- [Low] nbl_chan_cfg_qinfo_map_table() consumes the raw k_pf_mask value
without honouring the driver's own documented contract (bit0 reserved…
- [Low] The kernel-doc block for struct nbl_hw_ops::get_host_pf_mask
documents '@priv: hw ops private context', but the documented member's…
- [Low] The drain-strategy comment in nbl_chan_teardown_queue() justifies
the 5 s inflight-drain timeout by asserting that every sender has 'its…
- [Low] nbl_chan_get_msg_id() stores the slot it just claimed back into
chan_info->wait_head_index under a comment saying 'Advance starting…
- [Low] nbl_hw_wr32() in nbl_hw_reg.h carries the comment 'Used for emu,
make sure that we won't write too frequently' although the body is a…
- [Low] nbl_chan_teardown_queue() returns the raw wait_event_timeout()
result, so on success it returns a positive jiffies count (up to…
- [Low] The device-removal path never tears down an active mailbox queue
before releasing channel resources: nbl_chan_remove_common() destroys…
- [Low] nbl_common_get_hash_node() returns hash_node->data after dropping
the per-bucket spin_lock_bh with no reference count or RCU, and…
- [Low] chan_info->active/shutdn, the state bitmap, inflight_wait and
inflight_tx_cnt are documented as guarded by state_lock, but…
- [Low] nbl_chan_set_queue_state() clears NBL_CHAN_IRQ_RDY, which is part
of the wait condition used by the interrupt-mode ACK waiter in…
- [Low] In nbl_chan_update_txqueue() the external-buffer path copies the
payload only 'if (param->arg)' but unconditionally programs…
- [Low] Every channel op taking a u8 chan_type indexes
chan_mgt->chan_info[chan_type] (array size NBL_CHAN_TYPE_MAX == 1)…
- [Low] In nbl_chan_kick_tx_ring()'s completion-poll loop, dma_rmb() is
issued before the load of tx_desc->flags with the comment 'Order…
- [Low] nbl_hw_init_leonis() accepts any non-zero mailbox BAR length,
while nbl_hw_write_mbx_regs() silently drops out-of-range register…
--
net/nebula-matrix: add channel layer
This adds the mailbox based inter-PF channel layer for the nbl driver,
including the message handler hash table, TX/RX descriptor ring
management with coherent DMA buffers, and a hardware abstraction layer
for the mailbox queue registers. A few questions below about the
channel state machine, the TX completion poll and some comments that
do not appear to match the code.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> new file mode 100644
> index 000000000000..c29ff76a75ae
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -0,0 +1,1220 @@
[ ... ]
> +static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
[ ... ]
> + /*
> + * Step3:
> + * wait all sender exit
> + *
> + * Drain strategy mirrors mlx5 command interface teardown:
> + * set shutdown flag first, abort all pending waiters, then
> + * block until inflight_tx_cnt reaches zero.
> + *
> + * A timeout here is treated as an exceptional condition rather
> + * than a fatal error, following the same rationale as mlx5:
> + * - shutdn is already set, so every sender path observes it
> + * at its next checkpoint and exits;
> + * - each sender has its own bounded timeout (3s ACK wait,
> + * 12ms max TX ring poll), all far shorter than this 5s
> + * drain window, so stalling beyond 5s should never happen
[Severity: Low]
Is the "3s ACK wait" bound accurate for both wait modes? The 3 * HZ
bound (NBL_CHAN_ACK_WAIT_TIME) applies only to the interrupt branch of
nbl_chan_send_msg(). The polling branch loops
NBL_CHAN_TX_WAIT_ACK_TIMES times with usleep_range():
int i = NBL_CHAN_TX_WAIT_ACK_TIMES;
...
while (i--) {
...
usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
NBL_CHAN_TX_WAIT_ACK_US_MAX);
With NBL_CHAN_TX_WAIT_ACK_TIMES = 5000 and 1000-1200 us per iteration
that is roughly 5-6 s, which is longer than the 5 s drain window the
comment says every sender bound is "far shorter" than. Should the
comment be corrected, or the constants adjusted?
> + * - in that hardware-dead case proceeding with teardown cannot
> + * make the situation worse, and avoids hanging rmmod forever.
> + */
> + ret = wait_event_timeout(chan_info->inflight_wait,
> + atomic_read(&chan_info->inflight_tx_cnt) == 0,
> + msecs_to_jiffies(5000));
> +
> + if (!ret) {
> + dev_warn(chan_mgt->common->dev,
> + "teardown: inflight tx drain timeout\n");
> + ret = -ETIMEDOUT;
> + }
[Severity: Low]
On the success path this returns the raw wait_event_timeout() value,
i.e. the remaining jiffies (up to ~5000), through
int (*teardown_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
while the early "channel not active" path of the same function returns
0. Would it be better to normalise the success case to 0 so a caller
doing "if (ret) goto err;" does not treat a fully drained queue as a
failure?
> +
> + /* After all TX drained, stop hardware queue */
> + nbl_chan_stop_queue(chan_mgt);
> +
> + /* All send paths drained, safely cancel cleanup work */
> + if (task)
> + cancel_work_sync(task);
> + WRITE_ONCE(chan_info->active, false);
> + return ret;
> +}
> +
> +static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
> +{
[ ... ]
> + nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
> + nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
> + nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
> + rxq->tail_ptr, NBL_MB_RX_QID);
> + WRITE_ONCE(chan_info->active, true);
> + return 0;
> +}
[Severity: Low]
Can this mark the channel active when the hardware queues were never
programmed? nbl_chan_config_queue() calls config_mailbox_txq/rxq,
which are void and end up in nbl_hw_write_mbx_regs(); that helper drops
out-of-range writes after a dev_err_once() and returns void, so the
failure cannot reach here.
nbl_hw_init_leonis() only rejects a mailbox BAR of length zero, while
the queue configuration tables are written at
NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR (0x10) and
NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR (0x20) plus 16 bytes each. On a
BAR2 smaller than 0x30 the writes are silently skipped, the RX doorbell
is still rung and active is set to true.
Would it make sense to validate the mailbox BAR against the highest
register offset the driver uses at probe time, and/or let the config
ops return an error?
> +
> +static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
[ ... ]
> +static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
[ ... ]
> + while (retry_count < max_retries) {
> + if (READ_ONCE(chan_info->shutdn))
> + return -ESHUTDOWN;
> +
> + /* Order descriptor read after hardware DMA completion */
> + dma_rmb();
> + if (le16_to_cpu(READ_ONCE(tx_desc->flags)) &
> + BIT(NBL_CHAN_TX_DESC_USED)) {
> + break;
> + }
[Severity: Low]
Does the dma_rmb() here provide the ordering the comment describes? It
is issued before the load of tx_desc->flags, so it cannot order later
reads of device-written descriptor fields against observing USED. The
RX side in nbl_chan_clean_queue() uses the other order:
while (le16_to_cpu(READ_ONCE(rx_desc->flags)) &
BIT(NBL_CHAN_RX_DESC_USED)) {
...
/* Make sure hardware written descriptor visible to CPU */
dma_rmb();
nbl_chan_recv_msg(chan_mgt, rx_buf->va);
Should the TX poll follow the same pattern, loading flags first and
placing dma_rmb() between the USED test and any other descriptor field?
> +
> + retry_count++;
> + if (retry_count == max_retries) {
> + msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
> + dev_err_ratelimited(dev, "chan send msg type: %d timeout\n",
> + msg_type);
> + txq->next_to_clean = txq->next_to_use;
> + return -ETIMEDOUT;
> + }
> + usleep_range(NBL_CHAN_TX_WAIT_US, NBL_CHAN_TX_WAIT_US_MAX);
> + }
[Severity: Medium]
On the timeout path the descriptor is still marked AVAIL, i.e. owned by
the device, but next_to_clean is force-synced to next_to_use and the
mailbox TX queue is left enabled with no reset. The -ESHUTDOWN early
return leaves next_to_clean behind next_to_use with a device-owned
descriptor outstanding.
nbl_chan_update_txqueue() then republishes slots with no producer /
consumer fullness or ownership test:
struct nbl_chan_tx_desc *tx_desc =
NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_use);
struct nbl_chan_buf *tx_buf =
NBL_CHAN_TX_RING_TO_BUF(txq, txq->next_to_use);
...
if (param->arg)
memcpy(tx_buf->va, param->arg, param->arg_len);
...
tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
After NBL_CHAN_QUEUE_LEN further sends the ring wraps onto that slot.
Can this rewrite a descriptor and its coherent tx_buf while the device
is still fetching or DMAing them? Would quiescing the queue with
stop_mailbox_txq() (or rolling tail_ptr back) on the timeout path, plus
a fullness check in nbl_chan_update_txqueue(), be needed here?
> +
> + txq->next_to_clean = txq->next_to_use;
> +
> + return 0;
> +}
[ ... ]
> +static int nbl_chan_get_msg_id(struct nbl_chan_info *chan_info,
> + u16 *msgid)
> +{
> + int search_loc = READ_ONCE(chan_info->wait_head_index), i;
> + struct nbl_chan_waitqueue_head *wait = NULL;
> + int status;
> +
> + lockdep_assert_held(&chan_info->pending_lock);
> + for (i = 0; i < chan_info->num_txq_entries; i++) {
> + wait = &chan_info->wait[search_loc];
> + status = READ_ONCE(wait->status);
> + if (status == NBL_MBX_STATUS_IDLE ||
> + status == NBL_MBX_STATUS_TIMEOUT) {
[Severity: Medium]
Is a slot in NBL_MBX_STATUS_TIMEOUT really free at this point? The
timed-out owner sets that status under pending_lock and then drops the
lock before it releases the slot:
nbl_chan_send_msg()
mutex_lock(&chan_info->pending_lock);
if (READ_ONCE(wait_head->status) == NBL_MBX_STATUS_WAITING) {
WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
...
}
mutex_unlock(&chan_info->pending_lock);
dev_err_ratelimited(...);
ret = -ETIMEDOUT;
goto out_clear_wait_slot;
out_clear_wait_slot:
mutex_lock(&chan_info->pending_lock);
nbl_chan_reset_wait_head(chan_info, wait_head);
If a second sender claims the same slot in that window, the reset then
clears the new owner's state, since nbl_chan_reset_wait_head() has no
ownership check:
WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
WRITE_ONCE(wait_head->ack_data, NULL);
WRITE_ONCE(wait_head->ack_data_len, 0);
The second sender can then be woken on the shared wait_queue and
consume another request's ack_data_len / ack_err:
if (READ_ONCE(wait_head->acked)) {
smp_rmb();
chan_send->ack_len = READ_ONCE(wait_head->ack_data_len);
ret = READ_ONCE(wait_head->ack_err);
Can that return success with a non-zero ack_len while chan_send->resp
was never written, so the caller reads uninitialised memory as a
mailbox reply? Would releasing the slot to IDLE in the same critical
section that sets TIMEOUT, or adding a generation/ownership check in
nbl_chan_reset_wait_head(), close this?
> + WRITE_ONCE(wait->msg_index,
> + NBL_NEXT_ID(wait->msg_index,
> + NBL_CHAN_MSG_INDEX_MAX));
> +
> + *msgid = FIELD_PREP(NBL_CHAN_MSGID_INDEX_MASK,
> + wait->msg_index) |
> + FIELD_PREP(NBL_CHAN_MSGID_LOC_MASK,
> + search_loc);
> +
> + /* Advance starting search position for next caller */
> + WRITE_ONCE(chan_info->wait_head_index, search_loc);
[Severity: Low]
The comment says the search position is advanced, but the slot that was
just claimed is stored back, so the next caller starts at the same
index. Was NBL_NEXT_ID(search_loc, chan_info->num_txq_entries - 1)
intended?
As written, a single sender keeps reusing wait[0], so the 6-bit
generation field NBL_CHAN_MSGID_INDEX_MASK wraps every 64 requests
instead of the msgid space being spread over the 256 slots. That is
the field nbl_chan_recv_ack_msg() uses to reject stale ACKs:
if (FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, ack_msgid) != w_msgidx) {
> + return 0;
> + }
[ ... ]
> +static void nbl_chan_set_queue_state(struct nbl_channel_mgt *chan_mgt,
> + enum nbl_chan_state state, u8 chan_type,
> + u8 set)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + if (set)
> + set_bit(state, chan_info->state);
> + else
> + clear_bit(state, chan_info->state);
> +}
[Severity: Low]
Clearing NBL_CHAN_IRQ_RDY makes the wait condition in
nbl_chan_send_msg() true:
ret = wait_event_timeout(wait_head->wait_queue,
READ_ONCE(wait_head->acked) ||
READ_ONCE(chan_info->shutdn) ||
!test_bit(NBL_CHAN_IRQ_RDY,
chan_info->state),
NBL_CHAN_ACK_WAIT_TIME);
but no wake_up() of the per-slot wait queues follows the clear_bit().
Should the waiters be woken here? Otherwise a sleeping sender only
notices the mode change when the 3 s timeout expires and then reports
-ETIMEDOUT rather than the -EIO the !test_bit() branch is meant to
return.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> new file mode 100644
> index 000000000000..e6d96768a5b5
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> @@ -0,0 +1,170 @@
[ ... ]
> +#define NBL_CHAN_TX_WAIT_ACK_US_MIN 1000
> +#define NBL_CHAN_TX_WAIT_ACK_US_MAX 1200
> +#define NBL_CHAN_TX_WAIT_ACK_TIMES 5000
[ ... ]
> +#define NBL_CHAN_MSG_INDEX_MAX 63
> +
> +#define NBL_CHAN_MSGID_INDEX_MASK GENMASK(5, 0)
> +#define NBL_CHAN_MSGID_LOC_MASK GENMASK(13, 6)
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index a67d1a674466..24b15418e601 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -10,6 +10,156 @@
> #include <linux/bitfield.h>
> #include "nbl_hw_leonis.h"
>
> +static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
> + const u32 *data, u32 len)
> +{
> + u32 i;
> +
> + if (len % 4)
> + return;
> + if (reg >= (u64)hw_mgt->mailbox_bar_size ||
> + reg + len > (u64)hw_mgt->mailbox_bar_size) {
> + dev_err_once(hw_mgt->common->dev,
> + "mbx write out of range: reg=0x%llx len=%u bar_size=%pa\n",
> + reg, len, &hw_mgt->mailbox_bar_size);
> + return;
> + }
> + for (i = 0; i < len / 4; i++)
> + nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
> +}
> +
> +static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
> + u32 len)
> +{
> + u32 size = len / 4;
> + u32 i;
> +
> + if (len % 4)
> + return;
> +
> + spin_lock(&hw_mgt->reg_lock);
> +
> + for (i = 0; i < size; i++)
> + data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
> + spin_unlock(&hw_mgt->reg_lock);
> +}
[Severity: Low]
The changelog says:
- Add register lock protection for hardware register read/write to
ensure concurrent access safety
Does that cover the mailbox path? Only nbl_hw_rd_regs_lock() and
nbl_hw_wr_regs_lock() take hw_mgt->reg_lock. nbl_hw_write_mbx_regs()
above takes no lock, and it is the only writer used by
config_mailbox_rxq/txq and stop_mailbox_rxq/txq for the four dword
QINFO configuration sequence, which looks like it wants to be one
atomic programming step. nbl_hw_update_mailbox_queue_tail_ptr() is
also unlocked. Should the changelog be narrowed, or should the mailbox
BAR helpers take the same lock?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
> index 5e9823e01d39..35604bdff2ae 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
[ ... ]
> +static inline void nbl_hw_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
> +{
> + /* Used for emu, make sure that we won't write too frequently */
> + wr32(hw_mgt->hw_addr, reg, value);
> +}
[Severity: Low]
The comment promises rate limiting, but the body is a plain writel()
with no delay or throttling. Should the comment be dropped?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index ecbf440e4366..587ac0c58c24 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> @@ -10,6 +10,39 @@
[ ... ]
> + /**
> + * get_host_pf_mask - Fetch host PF mask from firmware k_pf_mask reg
> + * @priv: hw ops private context
> + * @pf_mask: output pointer for PF mask value
[Severity: Low]
The documented member has no priv parameter:
void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
kernel-doc will report an excess parameter priv and a missing hw_mgt.
Should the tag be renamed to @hw_mgt?
[ ... ]
> + void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,04/10] net/nebula-matrix: add common resource implementation
2026-08-31 2:13 ` [PATCH v26 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 6 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 2 · Low: 4
- [Medium] New, undisclosed fatal probe conditions derived purely from
firmware/register content.
- [Medium] The kernel-doc contract of nbl_hw_ops::get_host_pf_mask in
nbl_include/nbl_def_hw.h states 'PF0 cannot be masked, bit0 is…
- [Low] The commit message and a new code comment describe consumers and
an execution-order guarantee that do not exist at this commit, and the…
- [Low] nbl_include/nbl_def_channel.h newly uses static_assert() but is
not self-contained: its only include is <linux/types.h> (which pulls…
- [Low] The kernel-doc block for nbl_hw_ops::get_host_pf_mask documents
'@priv: hw ops private context', but the declared first parameter is…
- [Low] Comment/implementation mismatch in
nbl_res_ctrl_dev_setup_eth_info(): the comment states 'Code accepts…
--
net/nebula-matrix: add common resource implementation
This patch adds the common resource layer for the nbl driver, with helpers
converting between vsi_id, func_id, eth_id and pf_id, plus new hw ops to
read the board info, the FW eth bitmap and the real bus number.
nbl_res_init_leonis() is now called from nbl_core_init() and initializes
the VSI/Eth/PF data only for control devices.
A few questions below about the new firmware topology checks, the ops
documentation and some of the new comments.
> nbl_res_start() initializes VSI/Eth/PF data structures only for control
> devices (`common->has_ctrl == true`). Framework dispatch layer ensures
> resource mapping APIs such as nbl_res_func_id_to_vsi_id() are only invoked
> on control devices.
[Severity: Low]
Which dispatch layer is meant here? At this commit nbl_core_init() returns
right after nbl_res_init_leonis():
ret = nbl_res_init_leonis(adapter);
if (ret)
goto res_init_fail;
return adapter;
adap->intf.resource_ops_tbl is published but has no consumer yet, and
nbl_res_vsi_id_to_pf_id() and nbl_res_pf_dev_vsi_type_to_hw_vsi_type() are
non-static with no users outside nbl_resource.c.
Could the commit message describe the guarantee that exists at this point in
the series, i.e. the has_ctrl checks inside the helpers themselves?
The same concern applies to the ordering comment in
nbl_res_ctrl_dev_sriov_info_init(), see below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> new file mode 100644
> index 000000000000..a719cc8fbf8a
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> @@ -0,0 +1,334 @@
[ ... ]
> + eth_info->eth_num = fw_port_num;
> + /* Intentional design constraint: each PF maps to exactly one
> + * Ethernet port. This couples PF identity to port identity
> + * and is required by nbl_res_get_eth_id() which indexes
> + * eth_info->eth_id[] by relative PF id.
> + */
> + if (res_mgt->resource_info->max_pf != eth_info->eth_num) {
> + dev_err(dev, "Invalid PF-to-port topology: max_pf=%u, eth_num=%u\n",
> + res_mgt->resource_info->max_pf, eth_info->eth_num);
> + return -EINVAL;
> + }
[Severity: Medium]
Is the equality of these two values actually guaranteed by the hardware?
max_pf comes from the k_pf_mask register via nbl_res_init_pf_num(), while
eth_num comes from NBL_FW_BOARD_DW3 port_num via nbl_hw_get_board_info().
The two quantities are reported independently by firmware, and the only
statement that they must match is this comment.
The consequence is a probe failure: nbl_probe() -> nbl_core_init() ->
nbl_res_init_leonis() -> nbl_res_start() -> nbl_res_ctrl_dev_setup_eth_info()
returns -EINVAL, and nbl_core_init() turns that into ERR_PTR(ret), so the
control PF does not bind at all.
Would it make sense to document this firmware contract in nbl_def_hw.h next
to get_host_pf_mask/get_board_info, and to mention in the changelog that this
patch introduces new fatal probe conditions?
> +
> + /*
> + * Original comment said dual-port board eth_id fixed to 0,2;
> + * Code accepts any contiguous valid bitmap bits (0/1 or 0/2 etc).
> + * Firmware only needs to report correct count of active ports,
> + * no hard-coded fixed bit positions required.
> + */
> + for (i = 0; i < NBL_MAX_ETHERNET; i++) {
> + if ((1 << i) & eth_bitmap) {
> + set_bit(i, eth_info->eth_bitmap);
> + eth_info->eth_id[eth_num] = i;
> + eth_info->logic_eth_id[i] = eth_num;
> + eth_num++;
> + }
> + }
[Severity: Low]
This isn't a bug, but the word "contiguous" in the comment does not match the
loop, and the example given (0/2) is not contiguous either.
The loop imposes no contiguity rule: any subset of bits 0..3 whose population
count equals board_info.eth_num (and equals max_pf) is accepted, so
eth_bitmap = 0b1010 is mapped to PF0/PF1 as well.
Should the word be dropped, or should the bitmap actually be checked for
contiguity?
[ ... ]
> + res_mgt->resource_info->sriov_info = sriov_info;
> + /*
> + * common->hw_bus supplies bus number for channel mailbox QINFO mapping.
> + * Execution order guarantee: this assignment runs before
> + * cfg_chan_qinfo_map_table() in nbl_dev_start(), only executed
> + * on control PF path.
> + */
> + common->hw_bus = hw_bus;
[Severity: Low]
Where is nbl_dev_start()? It does not exist at this commit, and later in the
series cfg_chan_qinfo_map_table() is reached from nbl_dev_setup_ctrl_dev() via
nbl_dev_setup_chan_qinfo():
nbl_dev_setup_ctrl_dev()
nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX)
cfg_chan_qinfo_map_table()
so the function named in the ordering comment is wrong even at the end of the
series. Could the comment name the actual caller?
[ ... ]
> +static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + u32 exp_contiguous_mask = 0;
> + u32 pf_mask = 0;
> + u32 pf_num = 0;
> + int i;
> +
> + hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
> +
> + /*
> + * k_pf_mask register rule:
> + * bit N == 0 -> PF#N enabled; bit N == 1 -> PF#N masked out.
> + * Hardware constraint: bit0 is reserved, PF0 cannot be masked.
> + * All-zero pf_mask means all PF0~PF7 are enabled.
> + *
> + * Product firmware constraint: only 3 valid configurations supported:
> + * 1 PF (PF0 only): pf_num = 1, mask = 0xfe
> + * 2 PFs (PF0,PF1): pf_num = 2, mask = 0xfc
> + * 4 PFs (PF0~PF3): pf_num = 4, mask = 0xf0
> + * No other PF count or sparse/non-contiguous PF layout is allowed.
> + */
> + for (i = 0; i < NBL_MAX_PF; i++) {
> + if (!(pf_mask & (1 << i)))
> + pf_num++;
> + }
> +
> + /*
> + * Sanity check: enabled PFs must be contiguous starting from PF0.
> + * Current resource framework uses relative PF id, sparse PF layout
> + * will cause mismatch between resource layer and hardware func_id.
> + */
> + for (i = 0; i < pf_num; i++)
> + exp_contiguous_mask |= BIT(i);
> + if ((pf_mask & exp_contiguous_mask) != 0) {
> + dev_err(res_mgt->common->dev,
> + "pf_mask 0x%08x: non-contiguous enabled PF, unsupported\n",
> + pf_mask);
> + return -EINVAL;
> + }
[Severity: Medium]
Does this treatment of bit0 agree with the ops documentation?
nbl_def_hw.h says of the same register:
* PF0 cannot be masked, bit0 is reserved and has no hardware effect.
* All-zero mask indicates all supported PFs are present and enabled.
but here bit0 is counted as a normal PF enable bit, and the contiguity test
then requires it to be clear: exp_contiguous_mask always contains BIT(0) for
any accepted pf_num >= 1, so a pf_mask with bit0 set fails.
Two examples: mask 0xff (PF0 only, with the documented-as-reserved bit0 also
set) gives pf_num == 0, and mask 0xfd gives pf_num == 1 with bit0 set and
trips the contiguity check. Both end in -EINVAL and an aborted probe.
Should nbl_res_init_pf_num() ignore bit0, or should the header comment be
corrected to say bit0 is a real enable bit?
> +
> + /* Only allow product-specified PF count: 1 / 2 / 4 */
> + if (pf_num != 1 && pf_num != 2 && pf_num != 4) {
> + dev_err(res_mgt->common->dev,
> + "Invalid pf_num=%u (mask=0x%08x), only 1/2/4 PFs supported\n",
> + pf_num, pf_mask);
> + return -EINVAL;
> + }
[Severity: Medium]
What happens on a board where the k_pf_mask register still reads its reset
value of zero? nbl_def_hw.h describes an all-zero mask as "all supported PFs
are present and enabled", which yields pf_num == 8 here and is then rejected
with -EINVAL, failing probe.
Is the all-zero case expected to be rejected, and if so should the header
comment say so?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> index 4fc987c0e56d..61dd97c779ef 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> @@ -39,6 +39,16 @@ enum nbl_chan_state {
> NBL_CHAN_STATE_NBITS
> };
>
> +struct nbl_board_port_info {
> + u8 eth_num;
> + u8 eth_speed;
> + u8 p4_version;
> + u8 rsv[5];
> +};
> +
> +static_assert(sizeof(struct nbl_board_port_info) == 8,
> + "nbl_board_port_info size must be 8 bytes");
> +
[Severity: Low]
Should this header include linux/build_bug.h? static_assert is defined there,
while the only include in nbl_def_channel.h is linux/types.h.
It builds today because every current translation unit pulls in linux/device.h
or linux/pci.h before the nbl headers, but nbl_hw_reg.h already includes
nbl_def_channel.h before linux/pci.h arrives via nbl_core.h, so a future .c or
.h that includes nbl_def_channel.h first would fail with an implicit
declaration of static_assert.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index 587ac0c58c24..ee53f9e10a8e 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> @@ -34,9 +35,13 @@ struct nbl_hw_ops {
> * or other PF counts are unsupported by driver resource management.
> */
> void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
> + void (*get_real_bus)(struct nbl_hw_mgt *hw_mgt, u8 *bus);
[Severity: Low]
This isn't a bug, but the kernel-doc block just above get_host_pf_mask
documents "@priv: hw ops private context" while the declared first parameter
is struct nbl_hw_mgt *hw_mgt, so scripts/kernel-doc -Wall reports an
undescribed parameter hw_mgt plus an excess @priv description.
Since this patch extends the same ops structure and its new resource-layer
consumer relies on that comment block for the pf_mask contract, could the
parameter name be fixed here?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> index b7c80ea54c8d..1aafed2d46d7 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> @@ -42,7 +44,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
> ret = nbl_chan_init_common(adapter);
> if (ret)
> goto chan_init_fail;
> +
> + ret = nbl_res_init_leonis(adapter);
> + if (ret)
> + goto res_init_fail;
> return adapter;
[Severity: Medium]
This is the point where the new firmware topology validations become probe
failures: any -EINVAL from nbl_res_init_pf_num() or
nbl_res_ctrl_dev_setup_eth_info() propagates here and nbl_core_init() returns
ERR_PTR(ret), so nbl_probe() reports "Nbl adapter init fail" and the device
does not bind.
Could the changelog mention that this patch alone can turn a previously
successful probe into -EINVAL depending on register content?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,05/10] net/nebula-matrix: add intr resource implementation
2026-08-31 2:13 ` [PATCH v26 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 8 potential
issue(s) to consider:
Critical: 0 · High: 2 · Medium: 1 · Low: 5
- [High] Teardown asymmetry in the interrupt manager:
nbl_intr_mgt_start() has no counterpart, and nothing ever walks…
- [High] nbl_res_func_id_to_bdf explicitly treats all func_ids as PF IDs,
causing MSI-X configuration to fail for all VFs.
- [Medium] nbl_res_intr_destroy_msix_map() releases the coherent MSI-X
map table with dmam_free_coherent() after only a fixed…
- [Low] The kernel-doc of nbl_res_intr_set_mailbox_irq() states "Return:
0 on success, negative errno on parameter check or hw failure", but…
- [Low] (Retained for policy reasons; my verification indicates this is
NOT a defect.) Concern: nbl_res_intr_destroy_msix_map() recycles the…
- [Low] (Retained for policy reasons; my verification indicates this is
NOT a defect.) Concern: the commit message credits…
- [Low] (Retained for policy reasons; my verification indicates this is
resolved later in the series.) Concern: nbl_res_intr_cfg_msix_map()…
- [Low] (Retained for policy reasons; my verification indicates this is
resolved later in the series.) Concern: per-function interrupt…
--
net/nebula-matrix: add intr resource implementation
This adds the nbl_interrupt module, which owns the driver-wide global
MSI-X index space (intr_net_bmap / intr_other_bmap) and programs the
chip-internal MSI-X mapping registers. It exposes cfg_msix_map,
destroy_msix_map and set_mailbox_irq through the resource ops, and
instantiates the manager from nbl_res_start() via nbl_intr_mgt_start().
A few questions below, mostly around teardown.
Physical PCI MSI-X vector allocation lives in device layer via
nbl_dev_init_interrupt_scheme().
[Severity: Low]
This isn't a bug, but nbl_dev_init_interrupt_scheme() does not exist in
the tree at this commit, and neither does any caller of cfg_msix_map,
destroy_msix_map or set_mailbox_irq. Both arrive later in the series
("dispatch: implement channel RPC framework and serialize hardware ops"
and "add common dev start/stop operation").
Could the message say the symbol is added later in the series, so a
reader bisecting to this commit does not go looking for it?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> new file mode 100644
> index 000000000000..52872264e906
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> @@ -0,0 +1,402 @@
[ ... ]
> +int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
> + u16 func_id)
> +{
[ ... ]
> + /* Step 1: mask each MSIX vector in hardware first */
> + for (i = 0; i < intr_num; i++) {
> + hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
> + func_id, false, interrupts[i],
> + 0, 0, 0, false);
> + }
> +
> + nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
[Severity: Low]
This isn't a bug in the final series, but the resource layer here looks
self-inconsistent. nbl_res_intr_destroy_msix_map() returns the global
vector ids to intr_net_bmap / intr_other_bmap and clears
PADPT_HOST_MSIX_INFO, PCOMPLETER_HOST_MSIX_FID_TABLE and
FUNCTION_MSIX_MAP, but it never clears the MSIX_IDX / MSIX_IDX_VALID
fields in NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id) that
nbl_hw_set_mailbox_irq() programmed.
Afterwards the repair path is closed too, because
nbl_res_intr_set_mailbox_irq() rejects the disable case once the array
is gone:
if (!intr_mgt->func_intr_res[func_id].interrupts) {
dev_err(dev, "func %u MSIX map not configured\n", func_id);
return -ENODEV;
}
even though nbl_hw_set_mailbox_irq() ignores global_vec_id when
en_msix is false.
I checked the end of the series and nothing reaches that state:
nbl_dev_stop() calls nbl_dev_disable_mailbox_irq() before
nbl_dev_destroy_msix_map(), nbl_dev_start() configures the map before
enabling the mailbox irq, and cfg_msix_map() has a single caller. So
the ordering happens to be correct everywhere.
The ordering requirement is only prose though, and the RPC handler
nbl_disp_chan_destroy_msix_map_resp() accepts a destroy request from a
peer without requiring a prior set_mailbox_irq(false). Would it be
worth having destroy_msix_map() clear the mailbox route it owns, and
letting the disable path succeed when no map is configured?
> +
> + /*
> + * Stage 1 tear down: retain valid DMA address, ONLY clear VALID bit
> + * avoid hardware torn read (VALID=1 & dma_addr=0)
> + */
> + hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
> + false, live_dma, 0, 0, 0);
> + hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
> +
> + /*
> + * Hardware provides no idle status register for MSIX map DMA engine.
> + * Use bounded sleep to mitigate race between posted MMIO disable writes
> + * and ongoing in-flight table read DMA access.
> + * After sleep, hardware no longer performs DMA access to MSIX table.
> + */
> + usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
> +
> + /*
> + * Stage 2: hardware has quiesced MSIX table DMA access,
> + * fully zero MSIX map entry safely now
> + */
> + hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
> + false, 0, 0, 0, 0);
> + hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
> +
> + /*
> + * Now safe to release old MSIX DMA memory, prevents devres accumulation
> + * leak Since hardware DMA has quiesced after sleep, no IOMMU fault risk
> + * anymore.
> + */
> + if (msix_map_table->base_addr) {
> + dmam_free_coherent(dev, msix_map_table->size,
> + msix_map_table->base_addr,
> + msix_map_table->dma);
> + }
[Severity: Medium]
Can the 100-120 us sleep be relied on here? The comment states "After
sleep, hardware no longer performs DMA access to MSIX table", but as
the changelog itself says there is no idle status register for this
engine, so there is no device-observable handshake backing that claim.
The sequence is:
cfg_msix_map(valid=false, live_dma) /* posted MMIO */
flush_write()
usleep_range(100, 120)
dmam_free_coherent(...) /* IOVA torn down */
A table fetch already issued before the VALID clear was observed can
complete after dmam_free_coherent() has unmapped the buffer. PCIe
bounds read completion only by the completion timeout range (tens of
microseconds up to tens of milliseconds), not by 100 us, and a switch,
congested link or an IOMMU in the path can push past it.
The outcome would be exactly what this code is trying to avoid: an
IOMMU translation fault, or a fetch of memory already handed to another
owner followed by an MSI-X raised with a garbage global index.
Is there any status bit, or a read-back of a register the engine
updates, that could be polled instead? Failing that, would retaining
the buffer (freeing it only at detach, or after a much longer bound)
be safer than a fixed sleep?
> +
> + /* Release runtime allocated interrupt vector buffer */
> + kfree(intr_mgt->func_intr_res[func_id].interrupts);
> + intr_mgt->func_intr_res[func_id].interrupts = NULL;
> + intr_mgt->func_intr_res[func_id].num_interrupts = 0;
> + intr_mgt->func_intr_res[func_id].num_net_interrupts = 0;
[ ... ]
> +int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
> + u16 func_id, u16 num_net_msix,
> + u16 num_others_msix,
> + bool net_msix_mask_en)
> +{
[ ... ]
> + /* Allocate net interrupt vectors */
> + for (i = 0; i < num_net_msix; i++) {
> + intr_index = find_first_zero_bit(intr_mgt->intr_net_bmap,
> + NBL_MAX_NET_INTERRUPT);
> + if (intr_index == NBL_MAX_NET_INTERRUPT) {
> + dev_err(dev, "No free net interrupt vectors left\n");
> + ret = -EAGAIN;
> + goto release_vecs;
> + }
> + tmp_interrupts[i] = intr_index + NBL_NET_INTR_BASE;
> + set_bit(intr_index, intr_mgt->intr_net_bmap);
> + }
[Severity: Low]
The find_first_zero_bit() plus separate set_bit() on the driver-wide
intr_net_bmap / intr_other_bmap is not atomic, so at this commit two
concurrent calls for different func_ids could hand the same global
index to two functions.
I checked the rest of the series and this is covered: the later
dispatch patch wraps every entry point (nbl_disp_cfg_msix_map(),
nbl_disp_destroy_msix_map(), nbl_disp_set_mailbox_irq() and the three
matching nbl_disp_chan_*_resp() handlers) in
disp_mgt->ops_mutex_lock, and there is one disp_mgt per adapter, so
the allocations end up serialized.
Since the requirement lives only in the comment, would a
lockdep_assert_held() on that mutex, or a double-underscore name for
these helpers, make the contract checkable?
> +
> + /* Allocate other interrupt vectors */
> + for (; i < requested; i++) {
> + intr_index =
> + find_first_zero_bit(intr_mgt->intr_other_bmap,
> + NBL_MAX_OTHER_INTERRUPT);
[ ... ]
> +/**
> + * nbl_res_intr_set_mailbox_irq - bind mailbox IRQ to specified vector
> + * @res_mgt: resource management instance
> + * @func_id: target function identifier
> + * @vector_id: index inside local interrupt array
> + * @en_msix: enable/disable mailbox interrupt
> + *
> + * Caller note: this function has no internal locking. Serialization
> + * must be guaranteed at upper dispatch layer.
> + *
> + * Return: 0 on success, negative errno on parameter check or hw failure
> + */
[Severity: Low]
Can this function ever report a hardware failure? The only hardware
call it makes is hw_ops->set_mailbox_irq(), which is declared void in
struct nbl_hw_ops (nbl_def_hw.h), and its only implementation
nbl_hw_set_mailbox_irq() returns nothing:
void (*set_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
bool en_msix, u16 global_vec_id);
So every non-zero return comes from the three parameter and state
checks. Should the "or hw failure" wording be dropped, or should the
hw op return a status?
> +int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
> + u16 func_id, u16 vector_id,
> + bool en_msix)
> +{
[ ... ]
> + if (!intr_mgt->func_intr_res[func_id].interrupts) {
> + dev_err(dev, "func %u MSIX map not configured\n", func_id);
> + return -ENODEV;
> + }
> + if (vector_id >= intr_mgt->func_intr_res[func_id].num_interrupts) {
> + dev_err(dev, "vector_id %u out of range (max %u)\n",
> + vector_id,
> + intr_mgt->func_intr_res[func_id].num_interrupts - 1);
> + return -EINVAL;
> + }
> +
> + global_vec_id = intr_mgt->func_intr_res[func_id].interrupts[vector_id];
[Severity: Low]
Same observation for the per-function state. At this commit
func_intr_res[].interrupts is validated and then dereferenced here
while nbl_res_intr_destroy_msix_map() may kfree() it, and two
concurrent destroys would snapshot the same interrupts and
msix_map_table->base_addr pointers and free them twice.
The later dispatch patch closes both by taking the same per-adapter
ops_mutex_lock around all six entry points into these three functions,
and nbl_intr_mgt_start() is the only other toucher and runs
single-threaded during probe, so no in-tree path hits either case.
The resource layer still relies on prose only ("Serialization must be
guaranteed at upper dispatch layer") with nothing asserting it.
[ ... ]
> +static struct nbl_interrupt_mgt *nbl_intr_setup_mgt(struct device *dev)
> +{
> + struct nbl_interrupt_mgt *intr_mgt;
> +
> + intr_mgt = devm_kzalloc(dev, sizeof(*intr_mgt), GFP_KERNEL);
> + if (!intr_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + bitmap_zero(intr_mgt->intr_net_bmap, NBL_MAX_NET_INTERRUPT);
> + bitmap_zero(intr_mgt->intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
> +
> + return intr_mgt;
> +}
> +
> +int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt)
> +{
> + struct device *dev = res_mgt->common->dev;
> + struct nbl_interrupt_mgt *intr_mgt;
> + int ret;
> +
> + intr_mgt = nbl_intr_setup_mgt(dev);
> + if (IS_ERR(intr_mgt)) {
> + ret = PTR_ERR(intr_mgt);
> + return ret;
> + }
> + res_mgt->intr_mgt = intr_mgt;
> + return 0;
> +}
[Severity: High]
This helper treats func_id as a PF id throughout, but the callers
hand it the whole function id space. nbl_res_intr_cfg_msix_map(),
nbl_res_intr_destroy_msix_map() and nbl_res_intr_set_mailbox_irq() all
bound func_id by NBL_MAX_FUNC, which this patch defines as 520, and
intr_mgt->func_intr_res[] is sized the same way, so the interrupt
manager is clearly meant to track VFs as well as the 8 PFs.
For any func_id past the PF range nbl_common_func_id_to_rel_pf_id()
subtracts common->mgt_pf and the resulting rel_pf_id then trips this
check against resource_info->max_pf, so nbl_res_func_id_to_bdf()
returns -ERANGE and nbl_res_intr_cfg_msix_map() aborts before it
allocates anything. A VF whose map is programmed on its behalf by the
ctrl PF (the CONFIGURE_MSIX_MAP RPC handler added later in the series
passes the peer's src_id straight through as func_id) can therefore
never get an MSI-X mapping, and the failure is silent apart from a
dev_err that reports the value as "PF ID".
The lookup itself has no VF support either: sriov_info[rel_pf_id].bdf
only records the PF bdf, and there is nothing here that applies the
SR-IOV VF offset/stride to derive a VF's bus/dev/func.
Is VF handling intended to land in a later patch? If so, would it be
clearer to reject the out-of-PF-range func_id in
nbl_res_intr_cfg_msix_map() with an explicit "VF not supported yet"
path, rather than having it surface as -ERANGE from a BDF lookup that
reports it as a bad PF id? Or should nbl_res_func_id_to_bdf() learn
to map a VF func_id to its owning PF and compute the VF bdf from the
capability offset and stride?
[Severity: High]
Does nbl_intr_mgt_start() need a counterpart? Nothing in the series
walks intr_mgt->func_intr_res[] at detach, and nbl_res_remove_leonis()
is still an empty function whose comment says:
/*
* No resource release here because all memory uses devm managed
* allocation
*/
That comment no longer holds after this patch, since the per-function
vector array is kcalloc()'d in nbl_res_intr_cfg_msix_map() and only
freed in nbl_res_intr_destroy_msix_map().
The part I am unsure about is the remote PFs. On a 2 or 4 PF card,
PF1..PF3 configure their maps over the mailbox RPC, so
nbl_disp_chan_cfg_msix_map_resp() -> nbl_res_intr_cfg_msix_map(res_mgt,
src_id) allocates both the kcalloc()'d interrupts array and a
dmam_alloc_coherent() MSI-X table on the ctrl PF's device, recorded in
the ctrl PF's func_intr_res[1..3].
Later in the series nbl_dev_stop() only destroys its own function's
map:
ret = nbl_dev_destroy_msix_map(dev_mgt);
so when the ctrl PF detaches first (plain rmmod nbl detaches in bind
order, and unbinding PF0 alone through sysfs is a normal admin action)
the path is:
nbl_core_stop() /* destroys only func_id == common->mgt_pf */
nbl_core_remove()
nbl_res_remove_leonis() /* empty */
devres release
Two things follow from that. Does this leak
func_intr_res[1..3].interrupts, since kfree() on those arrays only ever
runs from nbl_res_intr_destroy_msix_map()?
And can devres release the remote PFs' coherent MSI-X tables while
NBL_PCOMPLETER_FUNCTION_MSIX_MAP(func), NBL_PADPT_HOST_MSIX_INFO and
NBL_PCOMPLETER_HOST_MSIX_FID_TABLE for those functions still hold
VALID=1 and the live bus address? That skips the two-stage clear plus
quiesce sleep that destroy_msix_map() documents as required before
freeing the table. The remaining PFs cannot repair it later either,
because their nbl_dev_stop() RPCs go to a mailbox that is already gone.
Would an nbl_intr_mgt_stop() in nbl_res_remove_leonis(), iterating
func_intr_res[] and calling nbl_res_intr_destroy_msix_map() for each
configured func_id, cover this?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> index a719cc8fbf8a..4b6a5bc8715a 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
[ ... ]
> @@ -287,6 +292,10 @@ static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
> ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
> if (ret)
> return ret;
> +
> + ret = nbl_intr_mgt_start(res_mgt);
> + if (ret)
> + return ret;
> }
>
> return 0;
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index ee53f9e10a8e..fd86eef4a0d3 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
[ ... ]
> @@ -39,6 +47,8 @@ struct nbl_hw_ops {
>
> void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> u8 bus, u8 devid, u8 function);
> + void (*set_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool en_msix, u16 global_vec_id);
> void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
> void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
> struct nbl_board_port_info *board);
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation
2026-08-31 2:13 ` [PATCH v26 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 9 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 2 · Low: 7
- [Medium] nbl_dp_init() programs DPED, UPED, shaping and DSCH into
hardware before nbl_ustore_init() validates the firmware-reported…
- [Medium] Within nbl_uvn_init() the driver read-modify-writes
NBL_UVN_DIF_REQ_RO_FLAG and NBL_UVN_QUEUE_ERR_MASK to preserve…
- [Low] hw_ops->init_module / hw_ops->deinit_module (and the resource
wrappers nbl_res_chip_init_module / nbl_res_chip_deinit_module)…
- [Low] New firmware quirk ABI is described inconsistently.
- [Low] nbl_hw_deinit_module() clears the driver_status bit and issues a
dummy-register read, with no acknowledgement or completion polling for…
- [Low] New register definitions mix conventions and pollute the global
macro namespace: NBL_SHAPING_DPORT_ADDR 0x504700 /…
- [Low] nbl_uvn_init() is declared `static void` yet its last statement
is `return nbl_hw_wr_regs_lock(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,…
- [Low] The shaping register block documents two incompatible units for a
CIR value of 1: nbl_hw_leonis.h:264 says "cir 1, bandwidth 1kB/s in…
- [Low] The only documentation of the UQM que_type register added by this
patch contradicts itself: `/* bit 0: bp mode , bit1: drop mode, resv…
--
net/nebula-matrix: add chip-wide hardware init/deinit implementation
This adds the Leonis chip-wide init/deinit path, programming the DP
sub-modules (dped, uped, dsch, ustore, dstore, dvn, uvn, uqm), per-port
shaping and host padpt flow control, plus the driver_status flag consumed by
firmware. It also registers hw_ops.init_module/deinit_module and the matching
resource_ops entry points, and introduces a firmware quirk word read from the
mailbox. A few questions below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
> new file mode 100644
> index 000000000000..e1ba7b7e36ca
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2026 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include "nbl_chip.h"
> +
> +void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> +
> + hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
> +}
> +
> +int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
> +{
> + u8 eth_speed = res_mgt->resource_info->board_info.eth_speed;
> + u8 eth_num = res_mgt->resource_info->board_info.eth_num;
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
> +
> + return hw_ops->init_module(p, eth_speed, eth_num);
> +}
[Severity: Low]
This isn't a bug, but would it be worth adding the has_ctrl check here for
consistency with the sibling resource ops?
Every other resource op added so far starts with:
if (!common->has_ctrl)
return -EINVAL;
(see nbl_res_intr_cfg_msix_map() and friends in nbl_interrupt.c and the
helpers in nbl_resource.c), while nbl_res_chip_init_module() and
nbl_res_chip_deinit_module() go straight into chip-global register
programming.
The registers touched by that path are well outside the mapping a non-control
function gets, for example NBL_DRIVER_STATUS_REG 0x1300444 and the dummy
register used by nbl_flush_writes() at 0x1300904, while nbl_hw_init_leonis()
only ioremaps NBL_REG_NET_ONLY_LEN when has_ctrl is not set, and
nbl_hw_wr32()/nbl_hw_rd32() do not bound-check against hw_size.
I checked and this is not reachable: nbl_dev_init() only calls
nbl_dev_setup_ctrl_dev() under if (common->has_ctrl), and
disp_ops->init_module/deinit_module are only assigned for
NBL_DISP_CTRL_LVL_MGT, which is only set when common->has_ctrl. So this is
purely a defense-in-depth remark, matching what the commit message already
says about the caller guarantee.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index fd6bc992fa0f..7f2626db731a 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -115,6 +115,548 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
> *eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
> }
>
> +static u32 nbl_hw_get_quirks(struct nbl_hw_mgt *hw_mgt)
> +{
> + u32 quirks = 0;
> +
> + nbl_hw_read_mbx_regs(hw_mgt, NBL_LEONIS_QUIRKS_OFFSET, &quirks,
> + sizeof(u32));
> +
> + if (quirks == ~0u)
> + return 0;
> +
> + return quirks;
> +}
[Severity: Low]
The quirk interface is described in three places and the three descriptions
do not agree.
The commit message says:
NBL_QUIRK_UVN_PREFETCH_ALIGN (BIT(1)), BIT(0) is reserved for future use.
but nbl_include.h defines BIT(0) as a named quirk:
#define NBL_QUIRKS_NO_TOE BIT(0)
#define NBL_QUIRK_UVN_PREFETCH_ALIGN BIT(1)
with the comment "ABI defined, driver implementation pending".
NBL_QUIRKS_NO_TOE has no user anywhere in the driver at the end of the
series. Should the commit message say BIT(0) is defined but unused, or
should the define be dropped until it is used?
The two prefixes also differ (NBL_QUIRKS_NO_TOE versus
NBL_QUIRK_UVN_PREFETCH_ALIGN), and the explanatory comment in nbl_uvn_init()
names a macro that does not exist in the tree:
* Default is sel=1, unless NBL_QUIRKS_UVN_PREFETCH_ALIGN is set,
while the code a few lines below uses NBL_QUIRK_UVN_PREFETCH_ALIGN.
One more question on the sentinel above: only ~0u is treated as "no quirks",
so on firmware predating this ABI, any other stale non-zero content at
mailbox offset 0x140 is taken as live quirk bits and can flip
NBL_UVN_DESC_PREFETCH_INIT_SEL. Is there a version or magic word that could
gate the read instead? I cannot inspect firmware images, so this part is
speculative.
[ ... ]
> +static int nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
> +{
> + struct nbl_shaping_dvn_dport_u dvn_dport = { 0 };
> + struct nbl_shaping_dport_u dport = { 0 };
> + u32 rate, half_rate;
> + u32 depth;
> + u64 low_val, high_val;
> +
> + switch (speed) {
> + case NBL_FW_PORT_SPEED_100G:
> + rate = 100000;
> + break;
[ ... ]
> + /* 2. write config words (valid=0, safe) */
> + low_val = FIELD_PREP(DPORT_CIR_MASK, rate) |
> + FIELD_PREP(DPORT_PIR_MASK, rate) |
[Severity: Low]
This isn't a bug, but which unit do these fields actually use? The header
documents two incompatible ones right above the shaping defines:
/* cir 1, bandwidth 1kB/s in protol environment */
/* cir 1, bandwidth 1Mb/s */
The code here writes 10000/25000/50000/100000 for 10/25/50/100 Gbit/s, which
matches only the 1 Mb/s reading; under the 1 kB/s reading a 100G port would
be shaped to roughly 0.8 Gbit/s. Could the stale comment be deleted or
qualified?
[ ... ]
> +static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
> +{
> + u32 eth_bitmap = 0;
> + u32 drop_th = 0;
> + u32 pkt_len = 0;
> + int i;
> +
> + if (eth_num != 1 && eth_num != 2 && eth_num != 4)
> + return -EINVAL;
[Severity: Medium]
Is it intended that eth_num is validated only here, after several sub-modules
have already been committed to the chip?
nbl_dp_init() programs DPED, UPED, shaping and DSCH before reaching this
check:
nbl_dped_init(hw_mgt);
nbl_uped_init(hw_mgt);
ret = nbl_shaping_init(hw_mgt, speed);
if (ret)
return ret;
nbl_dsch_qid_max_init(hw_mgt);
ret = nbl_ustore_init(hw_mgt, eth_num);
if (ret)
return ret;
eth_num comes straight from firmware. nbl_hw_get_board_info() does:
board_info->eth_num = FIELD_GET(NBL_FW_BOARD_DW3_PORT_NUM_MASK, data);
and nbl_res_init_board_info() does not validate it, so a board reporting 0, 3
or 8 (or a mailbox read that returns early on its own bounds check, leaving
eth_num 0) fails here with the shaping DPORT/DVN_DPORT VALID bits already set,
DSCH quanta and QID_MAX applied, DPED L4 checksum enabled and the UPED
profiles edited.
There is also no unwind, and nbl_hw_init_module() returns the error before
nbl_hw_set_driver_status(hw_mgt, true), so the firmware cleanup the commit
message relies on ("FW releases all chip hardware state automatically") is
never armed for a failed init, and nbl_hw_deinit_module() only clears a bit
that was never set.
Could eth_num be validated before the first register write?
[ ... ]
> +static void nbl_dvn_descreq_num_cfg(struct nbl_hw_mgt *hw_mgt, u8 descreq_num)
> +{
> + u8 split_ring_num = (descreq_num >> 3) & 0x1;
> + u8 ring_num = descreq_num & 0x7;
> + u32 num_cfg = 0;
> +
> + num_cfg = FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK,
> + split_ring_num) |
> + FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK,
> + ring_num);
> +
> + nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, &num_cfg,
> + sizeof(num_cfg));
> +}
[ ... ]
> +static void nbl_uvn_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + u16 wr_timeout = NBL_UVN_DESC_WR_TIMEOUT_VAL;
> + u32 timeout = NBL_UVN_DESC_RD_WAIT_TICKS;
> + u32 desc_wr_timeout = 0;
> + u32 prefetch_init = 0;
> + bool ro_enabled;
> + u32 flag = 0;
> + u32 mask = 0;
> + u32 quirks;
> +
> + spin_lock(&hw_mgt->reg_lock);
> + nbl_hw_wr32(hw_mgt, NBL_UVN_DESC_RD_WAIT, timeout);
> + desc_wr_timeout =
> + FIELD_PREP(NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK, wr_timeout);
> + nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT, &desc_wr_timeout,
> + sizeof(desc_wr_timeout));
[Severity: Medium]
Does this write clear NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK? The patch itself
declares that bit as a live field of the same register:
#define NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK GENMASK(14, 0)
#define NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK BIT(15)
but desc_wr_timeout starts at 0 and only NUM is filled in, so BIT(15) is
written back as 0 whatever firmware programmed. A few lines below, this same
function read-modify-writes NBL_UVN_DIF_REQ_RO_FLAG and
NBL_UVN_QUEUE_ERR_MASK precisely to preserve the other bits, so the
inconsistency is internal to the function.
The same pattern applies to two other registers in this patch:
- NBL_UVN_DESC_PREFETCH_INIT is written from scratch with only NUM[7:0] and
SEL[16]
- NBL_DVN_DESCREQ_NUM_CFG in nbl_dvn_descreq_num_cfg() above is written
with only BIT(0) and GENMASK(6, 4)
Since deinit relies entirely on firmware cleanup, whatever gets zeroed here
stays zeroed for the life of the chip. Should these be read-modify-writes
too? I could not find the reset defaults of those bits documented in the
tree, so this is based on the field definitions the patch adds.
> + ro_enabled = pcie_relaxed_ordering_enabled(hw_mgt->common->pdev);
[ ... ]
> + spin_unlock(&hw_mgt->reg_lock);
> + quirks = nbl_hw_get_quirks(hw_mgt);
> + /*
> + * sel=0: use configured num; sel=1: use internal calc (max 32)
> + * Default is sel=1, unless NBL_QUIRKS_UVN_PREFETCH_ALIGN is set,
> + * in which case override to sel=0.
> + */
> + prefetch_init =
> + FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK,
> + NBL_UVN_DESC_PREFETCH_NUM) |
> + FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK,
> + (quirks & NBL_QUIRK_UVN_PREFETCH_ALIGN) ? 0 : 1);
> +
> + return nbl_hw_wr_regs_lock(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,
> + &prefetch_init, sizeof(prefetch_init));
> +}
[Severity: Low]
Was this return intended? nbl_uvn_init() is declared static void and
nbl_hw_wr_regs_lock() also returns void, so this is a return with an
expression in a void function, which ISO C forbids and -Wpedantic
diagnoses. Every other sub-init helper in this patch calls the write helper
as a plain statement, and nbl_dp_init() calls nbl_uvn_init() as a statement
too, so nothing is propagated.
While on that topic, only nbl_shaping_init() and nbl_ustore_init() can report
failure to the int-returning nbl_hw_init_module(); the rest are void. Is the
mixed failure model deliberate?
[ ... ]
> + for (i = 0; i < NBL_UQM_DPORT_DROP_DEPTH; i++)
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_DPORT_DROP_CNT + (sizeof(cnt) * i), &cnt,
> + sizeof(cnt));
> + /* bit 0: bp mode , bit1: drop mode, resv bit1-31 */
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_QUE_TYPE, &que_type, sizeof(que_type));
[Severity: Low]
This comment gives bit1 two meanings: "drop mode" and part of the reserved
range. Should the reserved range read bit2-31?
[ ... ]
> +/*
> + * This design is intentional. Setting driver status to false is the
> + * official teardown mechanism: it notifies firmware to perform full
> + * cleanup of all per-PF hardware state, including qinfo registers.
> + * An inverse helper would duplicate work that the firmware already
> + * does, and would add error-path complexity for no benefit. We keep
> + * the deinit path minimal and rely on firmware cleanup for correctness,
> + * including abnormal hardware reset scenarios.
> + */
> +static void nbl_hw_deinit_module(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_hw_set_driver_status(hw_mgt, false);
> + /* ensure registers written */
> + nbl_flush_writes(hw_mgt);
> +}
[Severity: Low]
This isn't a bug, but is there any way for the driver to observe that the
firmware cleanup completed? nbl_flush_writes() only reads the dummy register
to push the posted write out, it is not an acknowledgement, and everything
init armed stays set on return: shaper VALID bits, DSCH PSHA, USTORE
port-drop, DSTORE flow control, DVN/UVN RO and error-mask bits, DPED L4
checksum.
I could not construct a failure path from this in the current series. The
only DMA engine set up so far is the mailbox channel, and
nbl_chan_teardown_queue() calls nbl_chan_stop_queue(), which stops the
queues in hardware:
hw_ops->stop_mailbox_rxq(chan_mgt->hw_ops_tbl->priv);
hw_ops->stop_mailbox_txq(chan_mgt->hw_ops_tbl->priv);
before the buffers are released, and nbl_dev_remove() runs the ctrl-dev
teardown before the common-dev teardown. So this is only a remark: a defined
"cleanup complete" indication with a timeout would make the firmware contract
verifiable from the driver side.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> index fed2fb16bff8..95d8a12bdcb5 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> @@ -110,6 +124,216 @@ struct nbl_function_msix_map {
> u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
> };
>
[ ... ]
> +#define NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK GENMASK(14, 0)
> +#define NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK BIT(15)
[ ... ]
> +/* --------- SHAPING --------- */
> +
> +/* cir 1, bandwidth 1kB/s in protol environment */
> +/* cir 1, bandwidth 1Mb/s */
> +#define NBL_LR_LEONIS_NET_BUCKET_DEPTH 9600
> +#define NBL_SHAPING_DPORT_ADDR 0x504700
> +#define NBL_SHAPING_DPORT_DWLEN 4
> +#define NBL_SHAPING_DPORT_REG(r) \
> + (NBL_SHAPING_DPORT_ADDR + (NBL_SHAPING_DPORT_DWLEN * 4) * (r))
> +#define NBL_SHAPING_DVN_DPORT_ADDR 0x504750
[ ... ]
> +#define DPORT_VALID_MASK (0x1ULL << 0)
> +#define DPORT_DEPTH_MASK (0x7FFFFULL << 1) // [19:1]
> +#define DPORT_CIR_MASK (0x7FFFFULL << 20) // [38:20]
> +#define DPORT_PIR_MASK (0x7FFFFULL << 39) // [57:39]
> +#define DPORT_CBS_MASK_LOW (0x3FULL << 58) // [63:58]
> +#define DPORT_CBS_MASK_HIGH (0x7FFFULL << (0)) // [78:64] -> high[14:0]
> +#define DPORT_PBS_MASK (0x1FFFFFULL << (79 - 64)) // [99:79] -> high[35:15]
[Severity: Low]
This isn't a bug, but the new register defines mix several conventions.
The C99 // comments above are a checkpatch error, and the masks are
hand-rolled shifts where the rest of this header uses GENMASK()/
GENMASK_ULL(). The DPORT_* and DEFAULT_DVN_* macros also lack the NBL_
prefix used by everything else, so they land in the global macro namespace.
Several of the new addresses are spelled out numerically even though the
matching base already exists in this header:
NBL_SHAPING_DPORT_ADDR 0x504700
NBL_SHAPING_DVN_DPORT_ADDR 0x504750
NBL_SHAPING_NET_ADDR 0x505800
all sit inside NBL_DP_SHAPING_BASE (0x00504000), which is now left with no
user at all in the tree. Same for NBL_DSCH_PSHA_EN_ADDR 0x404314
(NBL_DP_DSCH_BASE + 0x314), NBL_DSTORE_D_DPORT_FC_TH_ADDR 0x704600 and
NBL_DSTORE_PORT_DROP_TH_ADDR 0x704150 (NBL_DP_DSTORE_BASE + 0x600 / + 0x150),
and NBL_DPED_L4_CK_CMD_40_ADDR 0x75c338 (NBL_DP_DPED_BASE + 0x338).
Finally, a number of the added defines have no user at the end of the series,
for example the NBL_DPED_L4_CK_CMD_40_* field masks other than EN,
NBL_DSTORE_PORT_DROP_DISC_TH_MASK, NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK and the
_DEPTH/_WIDTH/_DWLEN triplets. Could the unused ones be dropped until they
are needed?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops
2026-08-31 2:13 ` [PATCH v26 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 8 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 4 · Low: 4
- [Medium] All five new *_req() helpers translate the return value of
chan_ops->send_msg() with a switch that only enumerates the four wire…
- [Medium] nbl_disp_setup_msg()'s rollback calls
chan_ops->unregister_all_msg(), which is…
- [Medium] nbl_disp_chan_get_vsi_id_req() and
nbl_disp_chan_get_eth_id_req() consume the ACK payload without…
- [Medium] The newly exposed destroy_msix_map op (local
nbl_disp_destroy_msix_map(), remote…
- [Low] The commit message describes work the diff does not contain.
- [Low] nbl_disp_init() sets NBL_DISP_CTRL_LVL_NET whenever
common->has_net, with a comment claiming this enables the 'net control…
- [Low] The kernel-doc block that this patch extends states 'Warning: All
ops except init_module/deinit_module can be safely called on PF/VF',…
- [Low] The newly exposed NBL_CHAN_MSG_CONFIGURE_MSIX_MAP handler
forwards peer-supplied vector counts straight to the global allocator…
--
net/nebula-matrix: dispatch: implement channel RPC framework and serialize
hardware ops
This adds mailbox RPC request/response handlers for configure_msix_map,
destroy_msix_map, set_mailbox_irq, get_vsi_id and get_eth_id, resolving the
dispatch ops to either local hardware calls (control PF) or remote requests
to the manager PF. It also adds a per-dispatch mutex to serialize the
mutable hardware ops, plus new wire message types and a permission-denied
status code.
A few questions below about the commit message and about the new handlers.
The commit message says:
Improve message payload validation for forward compatibility:
reject requests with insufficient payload length, enforce bounds
checks before parsing incoming RPC parameters. Fix unimplemented
operation check ordering to avoid potential NULL pointer
dereferences.
and:
Add helper to register channel response callbacks; extend wire
protocol with new message types and NBL_CHAN_RESP_PERM_DENY error
code.
[Severity: Low]
Which pre-existing code do the words "Improve", "Fix ... ordering" and
"Add helper" refer to?
At the parent commit the only helpers in nbl_dispatch.c are
nbl_disp_init_module() and nbl_disp_deinit_module(), and both already had
the guard:
if (res_ops->init_module)
return res_ops->init_module(p);
return -EOPNOTSUPP;
Neither is touched by this patch, so no check ordering appears to change.
Likewise, every handler containing a data_len < sizeof(param) test is
introduced by this patch, so there is no earlier validation being improved.
register_msg() and unregister_all_msg() also already exist in
struct nbl_channel_ops, and nbl_chan_setup_ops() already registers
NBL_CHAN_MSG_ACK; what this patch adds is a dispatch-layer setup function,
nbl_disp_setup_msg(), not a channel helper.
Could the wording be changed to describe the new code as new, so readers do
not look for a Fixes: tag and a previously broken commit?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> index 3da5f8351fa4..239d8317a9c5 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> @@ -3,9 +3,170 @@
> * Copyright (c) 2026 Nebula Matrix Limited.
> */
> #include <linux/device.h>
> +#include <linux/mutex.h>
> #include <linux/pci.h>
> #include "nbl_dispatch.h"
>
> +static int nbl_disp_chan_get_vsi_id_req(struct nbl_dispatch_mgt *disp_mgt,
> + u16 type, u16 *vsi_id)
> +{
> + struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
> + struct nbl_common_info *common = disp_mgt->common;
> + struct nbl_chan_param_get_vsi_id result = { 0 };
> + struct nbl_chan_param_get_vsi_id param = { 0 };
> + struct nbl_chan_send_info chan_send = {0};
> + int ret;
> +
> + param.type = cpu_to_le16(type);
> +
> + nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
> + NBL_CHAN_MSG_GET_VSI_ID,
> + ¶m, sizeof(param), &result,
> + sizeof(result), 1);
> + ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
> + switch (ret) {
> + case NBL_CHAN_RESP_OK:
> + break;
> + case NBL_CHAN_RESP_UNIMPLEMENTED:
> + return -EOPNOTSUPP;
> + case NBL_CHAN_RESP_ERR:
> + return -EREMOTEIO;
> + case NBL_CHAN_RESP_PERM_DENY:
> + return -EPERM;
> + default:
> + return -EREMOTEIO;
> + }
[Severity: Medium]
Does this switch turn every local transport error into -EREMOTEIO?
nbl_chan_send_msg() only returns a wire status code when an ACK actually
arrives. Otherwise it returns a negative Linux errno:
nbl_channel/nbl_channel.c:nbl_chan_send_msg() {
...
if (READ_ONCE(chan_info->shutdn)) { ... return -ESHUTDOWN; }
...
ret = -ETIMEDOUT;
...
ret = READ_ONCE(wait_head->ack_err);
}
plus -EINVAL for oversized arg_len/resp_len, -EAGAIN when all TX wait slots
are busy, and -EIO when the mailbox IRQ is disabled mid-wait.
All of those land in the default arm, so a caller cannot tell a retryable
or teardown condition from a genuine remote hardware failure, and
nbl_dev_stop() ends up logging a remote I/O error for what was really
-ESHUTDOWN or -ETIMEDOUT. The same switch is repeated in
nbl_disp_chan_get_eth_id_req(), nbl_disp_chan_cfg_msix_map_req(),
nbl_disp_chan_destroy_msix_map_req() and
nbl_disp_chan_set_mailbox_irq_req().
Would an early "if (ret < 0) return ret;" before the switch preserve the
error semantics the changelog aims for?
> + *vsi_id = le16_to_cpu(result.vsi_id);
> + return 0;
> +}
[ ... ]
> + ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
> + switch (ret) {
> + case NBL_CHAN_RESP_OK:
> + break;
> + case NBL_CHAN_RESP_UNIMPLEMENTED:
> + return -EOPNOTSUPP;
> + case NBL_CHAN_RESP_ERR:
> + return -EREMOTEIO;
> + case NBL_CHAN_RESP_PERM_DENY:
> + return -EPERM;
> + default:
> + return -EREMOTEIO;
> + }
> + *eth_num = result.eth_num;
> + *eth_id = result.eth_id;
> + *logic_eth_id = result.logic_eth_id;
> +
> + return 0;
> +}
[Severity: Medium]
Can a short ACK make these two helpers report success with zeroed
identifiers?
nbl_chan_recv_ack_msg() takes the status straight from the wire and only
clamps the payload copy, reporting the effective length back through
chan_send->ack_len:
nbl_channel/nbl_channel.c:nbl_chan_recv_ack_msg() {
...
copy_len = min_t(u32, READ_ONCE(wait_head->ack_data_len), ack_datalen);
if (READ_ONCE(wait_head->ack_err) >= 0 && copy_len > 0) {
...
WRITE_ONCE(wait_head->ack_data_len, (u16)copy_len);
}
}
nbl_disp_chan_get_vsi_id_req() and nbl_disp_chan_get_eth_id_req() never look
at chan_send.ack_len, so a peer answering status OK with a truncated or
empty payload leaves the "= { 0 }" initialiser values in place and the
caller gets vsi_id 0, eth_id 0, logic_eth_id 0, eth_num 0 and a 0 return.
Per nbl_res_ctrl_dev_vsi_info_init() ("pf0,pf1,pf2,pf3 vsi is 0,256,512,768")
vsi_id 0 and eth_id 0 are valid identifiers belonging to PF0, so a
non-control PF would silently adopt another function's VSI/port identity.
There is also no range check on the returned values, unlike the local
producer:
nbl_hw/nbl_resource.c:nbl_res_get_eth_id() {
...
if (rel_pf_id >= eth_info->eth_num) {
dev_err(dev, "rel_pf_id %d out of range [0, %u)\n", ...);
return -ERANGE;
}
}
Should these helpers check chan_send.ack_len against sizeof(result) and
bound eth_num/eth_id/logic_eth_id against NBL_MAX_ETHERNET before handing
the values to the caller?
> @@ -25,6 +186,346 @@ static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
> return -EOPNOTSUPP;
> }
>
[ ... ]
> + if (res_ops->cfg_msix_map) {
> + mutex_lock(&disp_mgt->ops_mutex_lock);
> + ret = res_ops->cfg_msix_map(p, src_id,
> + le16_to_cpu(param.num_net_msix),
> + le16_to_cpu(param.num_others_msix),
> + !!le16_to_cpu(param.msix_mask_en));
> + mutex_unlock(&disp_mgt->ops_mutex_lock);
> + if (ret)
> + err = NBL_CHAN_RESP_ERR;
> + } else {
> + err = NBL_CHAN_RESP_UNIMPLEMENTED;
> + }
[Severity: Low]
Should there be a per-function limit on the requested vector counts here?
After the coarse src_id check the peer-supplied counts go straight to the
global allocator, which only enforces device-wide maxima:
nbl_hw/nbl_interrupt.c:nbl_res_intr_cfg_msix_map() {
...
if (num_net_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES ||
num_others_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) { ... }
}
The accepted maximum for num_others_msix equals the whole shared pool
(NBL_MAX_OTHER_INTERRUPT is 1024), so a single request for 1024 drains
intr_other_bmap and every later allocation for any function, including the
control PF's own mailbox and misc vectors, then fails with -EAGAIN.
Each request also does a full 1024-entry coherent DMA table allocation and
the usleep_range(100, 120) quiesce inside nbl_res_intr_destroy_msix_map()
while ops_mutex_lock is held, which stalls the control PF's mailbox work
item.
> + if (src_id > NBL_MAX_PF_SRC_ID) {
> + err = NBL_CHAN_RESP_PERM_DENY;
> + goto ack_out;
> + }
> + if (res_ops->destroy_msix_map) {
> + mutex_lock(&disp_mgt->ops_mutex_lock);
> + ret = res_ops->destroy_msix_map(p, src_id);
> + mutex_unlock(&disp_mgt->ops_mutex_lock);
> + if (ret)
> + err = NBL_CHAN_RESP_ERR;
> + } else {
> + err = NBL_CHAN_RESP_UNIMPLEMENTED;
> + }
[Severity: Medium]
Does making destroy_msix_map remotely invokable leave the requester's
mailbox interrupt routing armed on a recycled vector?
nbl_res_intr_destroy_msix_map() masks the per-vector entries, returns the
global vector numbers to the bitmaps and frees the per-function array:
nbl_hw/nbl_interrupt.c:nbl_res_intr_destroy_msix_map() {
...
nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
...
kfree(intr_mgt->func_intr_res[func_id].interrupts);
intr_mgt->func_intr_res[func_id].interrupts = NULL;
}
but MSIX_IDX/MSIX_IDX_VALID in NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
written only by nbl_hw_set_mailbox_irq(), is left pointing at the vector
that was just released. It cannot be cleared afterwards either:
nbl_hw/nbl_interrupt.c:nbl_res_intr_set_mailbox_irq() {
if (!intr_mgt->func_intr_res[func_id].interrupts) {
dev_err(dev, "func %u MSIX map not configured\n", func_id);
return -ENODEV;
}
}
A later nbl_res_intr_cfg_msix_map() for another function re-validates the
same global vector with the new owner's BDF, so a mailbox event for the old
function would raise an interrupt owned by a different PF.
nbl_res_intr_cfg_msix_map() also calls the same teardown internally, so a
peer sending CONFIGURE_MSIX_MAP twice, or DESTROY_MSIX_MAP while its
mailbox IRQ is still armed, reaches this from the wire.
Would it be better to clear the mailbox QINFO mapping as part of
destroy_msix_map, or at least document the required ordering against
set_mailbox_irq in nbl_def_dispatch.h?
> +static int nbl_disp_setup_msg(struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *p = disp_mgt->chan_ops_tbl->priv;
> + int ret = 0;
> + int _ret;
> +
> + _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
> + nbl_disp_chan_cfg_msix_map_resp,
> + disp_mgt);
> + if (_ret < 0 && !ret)
> + ret = _ret;
[ ... ]
> + if (ret)
> + chan_ops->unregister_all_msg(p);
> + return ret;
> +}
[Severity: Medium]
Does this rollback destroy state owned by the channel layer?
unregister_all_msg is nbl_chan_remove_msg_handler():
nbl_channel/nbl_channel.c:nbl_chan_remove_msg_handler() {
if (!chan_mgt->handle_hash_tbl)
return;
nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl);
chan_mgt->handle_hash_tbl = NULL;
}
That frees every handler node, including the channel's own ACK handler
registered by nbl_chan_setup_ops():
ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
nbl_chan_recv_ack_msg, chan_mgt);
and then NULLs the shared table pointer with no re-init path, since
nbl_chan_init_msg_handler() is only called from nbl_chan_setup_chan_mgt().
Neither accessor tolerates a NULL table:
nbl_common/nbl_common.c:nbl_common_get_hash_node() {
...
key_size = tbl_mgt->tbl_key.key_size;
}
Registration is also unpaired at this layer: nbl_disp_remove() is empty and
delegates handler teardown to nbl_chan_remove_common(). As far as I can
tell no crash is reachable today, because the only failure mode of
nbl_chan_register_msg() is -ENOMEM during nbl_disp_init(), and at that point
no mailbox queue, IRQ or clean_mbx_task exists yet, so nothing looks the
table up during the unwind. Would it be more robust to unregister only the
five message types this function registered, and to do it from
nbl_disp_remove()?
> @@ -104,14 +621,30 @@ int nbl_disp_init(struct nbl_adapter *adapter)
> adapter->core.disp_mgt = disp_mgt;
> adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
>
> + ret = nbl_disp_setup_msg(disp_mgt);
> + if (ret)
> + return ret;
> +
> if (common->has_ctrl)
> nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
>
> + /*
> + * For non-control PF with network capability, enable net control
> + * level.
> + * All dispatch ops declared with NBL_DISP_CTRL_LVL_MGT fall back
> + * to remote mailbox msg_req handlers when MGT bit is not set.
> + */
> + if (common->has_net)
> + nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_NET);
[Severity: Low]
Is anything reading NBL_DISP_CTRL_LVL_NET?
Grepping the nbl driver at this commit, the symbol appears only in its enum
definition in nbl_def_dispatch.h and in this new set_bit(). The single
consumer of ctrl_lvl only tests the MGT bit:
nbl_core/nbl_dispatch.c:nbl_disp_refresh_ctrl_ops() {
if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
...
}
so the NET bit is write-only state.
The comment also says "For non-control PF with network capability", but the
condition is just common->has_net, and every entry in nbl_id_table sets
BIT(NBL_CAP_HAS_NET_BIT), so the control PF takes this branch too. Should
the condition and the comment be reconciled, or the bit dropped until a
reader exists?
> nbl_disp_refresh_ctrl_ops(disp_mgt);
> return 0;
> }
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> index d5e7da0fef6e..61083a750da4 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> @@ -22,12 +22,28 @@ enum {
> * caller must check has_ctrl guard
> * @deinit_module: dispatch layer cleanup, ONLY valid on Control PF,
> * caller must check has_ctrl guard
> + * @cfg_msix_map: configure function msix mapping table
> + * @destroy_msix_map: tear down msix mapping resource
> + * @set_mailbox_irq: bind mailbox interrupt to specified msix vector
> + * @get_vsi_id: resolve VSI ID by type
> + * @get_eth_id: resolve eth port info from VSI ID
> + *
> * Warning: All ops except init_module/deinit_module can be safely called
> * on PF/VF; init/deinit hooks are control-PF exclusive to prevent NULL ptr.
> */
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index ebf85702cef6..a53138530c54 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -15,6 +15,8 @@
>
> #define NBL_MAX_FUNC 520
> #define NBL_MAX_ETHERNET 4
> +/* Product firmware only supports 1/2/4 contiguous PFs (ID 0~3) */
> +#define NBL_MAX_PF_SRC_ID 3
[Severity: Low]
Does this constant contradict the documented contract just above?
The kernel-doc extended by this patch says the five new ops "can be safely
called on PF/VF", but on a non-control function they resolve to the *_req()
helpers, and every new responder starts with:
if (src_id > NBL_MAX_PF_SRC_ID) {
err = NBL_CHAN_RESP_PERM_DENY;
goto ack_out;
}
src_id is the mailbox descriptor srcid in a function-id space of
NBL_MAX_FUNC (520), so any VF requester is denied by construction and its
*_req() helper returns -EPERM.
The hardcoded 3 also disagrees with the runtime model used elsewhere:
nbl_common_func_id_to_rel_pf_id() computes pf_id - common->mgt_pf and is
documented to "Support future non-zero management PF", while
nbl_res_func_id_to_bdf() and nbl_res_get_eth_id() validate against
resource_info->max_pf. With mgt_pf != 0 the legitimate absolute PF ids
exceed 3 and would be denied as well.
No VF can bind this driver today, since nbl_id_table lists only PF device
ids and mgt_pf is hardcoded to 0 in nbl_core_init(), so this is a
documentation and future-proofing question rather than an active failure.
Would validating src_id against mgt_pf plus max_pf be preferable to the new
constant, or should the "PF/VF" wording be adjusted?
>
> enum {
> NBL_VSI_DATA = 0,
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,09/10] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-08-31 2:14 ` [PATCH v26 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 5 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 3 · Low: 2
- [Medium] nbl_dev_setup_common_dev() issues two synchronous ACK-waiting
mailbox requests (disp_ops->get_vsi_id(), disp_ops->get_eth_id())…
- [Medium] nbl_dev_setup_common_dev() enables the PF mailbox queue
(nbl_chan_setup_queue(): config_mailbox_txq/rxq with QUEUE_EN=1,…
- [Medium] nbl_dev_remove() relies on nbl_dev_remove_ctrl_dev() ->
disp_ops->deinit_module() -> nbl_hw_deinit_module(), which only does a…
- [Low] The justification comment block added above
nbl_dev_remove_ctrl_dev() in nbl_core/nbl_dev.c is factually wrong and…
- [Low] The in-tree comment documenting the ordering contract for
common->hw_bus in nbl_res_ctrl_dev_sriov_info_init()…
--
net/nebula-matrix: add common/ctrl dev init/remove operation
This adds the nbl_dev core layer with nbl_dev_setup_common_dev() and
nbl_dev_setup_ctrl_dev() plus their teardown counterparts, and wires them
into nbl_dev_init()/nbl_dev_remove() from nbl_core_init()/nbl_core_remove().
A few questions below about the probe/remove ordering and about some of the
new comments and the commit message.
> Enforce the deinit sequence: remove ctrl dev first to notify firmware
> to clean all per-PF hardware state including qinfo registers. Afterwards
> tear down common device queue resources. Firmware cleanup ensures valid
> register access during common dev deinit and prevents PCIe master abort.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> new file mode 100644
> index 000000000000..4fc52cadf60f
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> @@ -0,0 +1,236 @@
[ ... ]
> +/* ---------- Channel config ---------- */
> +static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + struct nbl_common_info *common = dev_mgt->common;
> +
> + if (!chan_ops->check_queue_exist(priv, chan_type))
> + return;
> +
> + chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
> +}
[Severity: Low]
This is the only in-tree caller of cfg_chan_qinfo_map_table(), and it is
reached as nbl_dev_init() -> nbl_dev_setup_ctrl_dev() ->
nbl_dev_setup_chan_qinfo(). The existing comment that documents the
ordering contract for common->hw_bus names a different call site:
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c:nbl_res_ctrl_dev_sriov_info_init() {
/*
* common->hw_bus supplies bus number for channel mailbox QINFO mapping.
* Execution order guarantee: this assignment runs before
* cfg_chan_qinfo_map_table() in nbl_dev_start(), only executed
* on control PF path.
*/
common->hw_bus = hw_bus;
}
There is no nbl_dev_start() at this commit, and the nbl_dev_start() added
later in the series calls nbl_dev_cfg_msix_map(),
nbl_dev_init_interrupt_scheme(), nbl_dev_request_mailbox_irq() and
nbl_dev_enable_mailbox_irq() only, so it never programs the qinfo map. The
ordering itself does hold today, since nbl_res_init_leonis() runs before
nbl_dev_init() in nbl_core_init(). Could that comment be updated to point
at nbl_dev_setup_ctrl_dev() instead?
[ ... ]
> +/* ---------- Dev init process ---------- */
> +static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
> +{
[ ... ]
> + ret = nbl_dev_setup_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> + if (ret)
> + return ret;
> +
> + INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
> + nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
> + &common_dev->clean_mbx_task);
> + /*
> + * Even if has_ctrl=false (no dedicated control PF channel), we fetch
> + * VSI/ETH info via regular mailbox message instead of dedicated
> + * control command.
> + */
> + ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
> + if (ret)
> + goto err_cleanup;
> + ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
> + &common->eth_id, &common->logic_eth_id);
> + if (ret)
> + goto err_cleanup;
[Severity: Medium]
Can these two calls complete at this point in the series?
On a non-control PF, get_vsi_id()/get_eth_id() resolve to
nbl_disp_chan_get_vsi_id_req()/nbl_disp_chan_get_eth_id_req(), i.e. real
mailbox round trips to mgt_pf that wait for an ACK.
At this commit nothing in the tree requests a mailbox interrupt and
NBL_CHAN_IRQ_RDY is never set, so in nbl_chan_send_msg() the polling branch
only drains the sender's own RX ring:
nbl_channel/nbl_channel.c:nbl_chan_send_msg() {
if (test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) { ... }
else { /* Polling path for synchronous ACK */ }
}
The control PF has nothing driving nbl_chan_clean_queue() on its side, so
the responder never runs and both requests appear to time out, failing
probe on every non-control PF.
The later patch in this series ("net/nebula-matrix: add common dev
start/stop operation") adds nbl_dev_request_mailbox_irq() and
nbl_dev_enable_mailbox_irq(), and nbl_probe() then calls nbl_core_start()
-> nbl_dev_start(), which is what makes the control PF drain its RX ring
and answer. So the ACK path does exist at the end of the series and this
looks limited to the intermediate tree state. Would it be possible to
order the series so probe works at each step, or is the intermediate
bisect state intentional here?
Also, nbl_dev_init_msix_cnt() records
msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1 in this patch while no
vector is allocated or requested yet, which fits the same observation.
[ ... ]
> +static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> + int ret;
> +
> + ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
> + if (ret)
> + return ret;
> +
> + nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> +
> + return 0;
> +}
> +
> +/*
> + * This is intentional. The qinfo registers are managed by the chip
> + * firmware, not by the driver. Setting driver status to false is the
> + * designed teardown mechanism — it notifies the firmware, which then
> + * performs its own cleanup of all per-PF state including the qinfo
> + * registers.
> + * An inverse helper would duplicate work that the firmware already
> + * does, and would add error-path complexity for no benefit. We keep
> + * the deinit path minimal and rely on the firmware cleanup for
> + * correctness, including in abnormal reset scenarios.
> + */
> +static void nbl_dev_remove_ctrl_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> +
> + disp_ops->deinit_module(dev_mgt->disp_ops_tbl->priv);
> +}
[Severity: Low]
Is the claim that "The qinfo registers are managed by the chip firmware,
not by the driver" accurate? The path added just above this comment
programs them from the driver:
nbl_dev_setup_chan_qinfo()
chan_ops->cfg_chan_qinfo_map_table()
nbl_chan_cfg_qinfo_map_table()
hw_ops->cfg_mailbox_qinfo()
nbl_hw_cfg_mailbox_qinfo()
nbl_channel/nbl_channel.c:nbl_chan_cfg_qinfo_map_table() {
for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
if (!(pf_mask & (1 << func_id)))
hw_ops->cfg_mailbox_qinfo(p, func_id, bus, devid, func_id);
}
}
Only the cleanup is delegated to firmware, so would something like "the
qinfo registers are cleaned up by firmware on driver-status change" be
closer to what happens?
Two more notes on the same block: it is a near-verbatim copy of the comment
already sitting above nbl_hw_deinit_module() in
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c, which is where the described register
write actually lives, and the opening "This is intentional." reads as a
reply to review feedback rather than as documentation. Could it be dropped
here and kept only at the hw layer?
Relatedly, the commit message says:
> Introduce nbl_dev_setup_common_dev()
> and nbl_dev_setup_ctrl_dev() initialization helpers with paired
> teardown routines
nbl_dev_setup_ctrl_dev() does init_module() plus qinfo map programming while
nbl_dev_remove_ctrl_dev() does deinit_module() only, so the two are not
symmetric.
[ ... ]
> +int nbl_dev_init(struct nbl_adapter *adapter)
> +{
[ ... ]
> + /*
> + * Chip hardware initialization is completed by firmware at power-up.
> + * Only driver functional table/register config follows here, safe to
> + * access hardware registers before ctrl dev setup.
> + */
> + ret = nbl_dev_setup_common_dev(adapter);
> + if (ret)
> + goto setup_err;
> +
> + if (common->has_ctrl) {
> + ret = nbl_dev_setup_ctrl_dev(adapter);
> + if (ret)
> + goto setup_ctrl_dev_fail;
> + }
[Severity: Medium]
What guarantees the mailbox routing table entry for a non-control PF is
programmed before that PF starts using its mailbox?
nbl_dev_setup_common_dev() enables the PF mailbox queue
(nbl_chan_setup_queue() -> config_mailbox_txq/rxq with QUEUE_EN=1, RST=0,
RX tail written) and then immediately sends mailbox messages, but
NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id) is written only afterwards and only
on the control PF:
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c:nbl_hw_cfg_mailbox_qinfo() {
data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
}
has_ctrl is set only for PCI function 0 in nbl_get_func_param(), so PF1..PF3
never program that table themselves. Their probe path depends on PF0 having
already done it, and nothing in nbl_dev_init() checks or enforces that.
Cases where the entry may not be programmed yet look reachable: PF0 unbound
via sysfs, PF0 bound to a different driver, or concurrent probing when async
probing is enabled. If the entry still holds zero, the mailbox ring BDF is
also zero. Best case that means the two ACK timeouts above and a probe
failure with no retry; is a write towards a zero BDF also possible on this
chip, and if so what stops it?
> +/*
> + * Teardown order: ctrl dev first, then common dev.
> + * nbl_dev_remove_ctrl_dev() notifies firmware to clean all per-PF state
> + * (including qinfo registers), so subsequent common dev queue cleanup
> + * will not trigger PCIe master abort or invalid register access.
> + */
> +void nbl_dev_remove(struct nbl_adapter *adapter)
> +{
> + struct nbl_common_info *common = &adapter->common;
> +
> + if (common->has_ctrl)
> + nbl_dev_remove_ctrl_dev(adapter);
> + nbl_dev_remove_common_dev(adapter);
> +}
[Severity: Medium]
How is the firmware cleanup serialized against the queue teardown that
follows it? The notification is a bare register write plus a flush:
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c:nbl_hw_deinit_module() {
nbl_hw_set_driver_status(hw_mgt, false);
/* ensure registers written */
nbl_flush_writes(hw_mgt);
}
There is no completion poll, status read-back or delay before
nbl_dev_remove_common_dev() -> nbl_chan_teardown_queue() writes the queue
reset:
nbl_hw_stop_mailbox_rxq() / nbl_hw_stop_mailbox_txq() {
cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
cfg_tbl.data, sizeof(cfg_tbl));
}
and the QUEUE_RST/QUEUE_EN state is never read back, so a firmware write
landing after the driver's reset write would leave the queue enabled in
hardware while the driver believes it is stopped, with no way to notice.
Could a driver-status handshake or a queue-reset read-back be added, or the
claim in the commit message be dropped:
> Firmware cleanup ensures valid
> register access during common dev deinit and prevents PCIe master abort.
For what it is worth, the DMA-after-free variant of this does not seem to
apply: nbl_remove() calls pci_clear_master(pdev) before returning, and the
dmam_alloc_coherent() rings are released by devres only after .remove()
returns, so bus mastering is already off when the ring memory goes away.
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [v26,net-next,10/10] net/nebula-matrix: add common dev start/stop operation
2026-08-31 2:14 ` [PATCH v26 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
@ 2026-09-03 17:18 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-03 17:18 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider:
Critical: 0 · High: 1 · Medium: 1 · Low: 1
- [High] This patch adds the first live callers of
cfg_msix_map()/destroy_msix_map(), so the control PF (function 0) now…
- [Medium] In nbl_dev_stop() (nbl_core/nbl_dev.c:489-495) and in the
identical unwind inside nbl_dev_start() (nbl_dev.c:465-472), a failure…
- [Low] Claim (assessed FALSE POSITIVE): nbl_dev_disable_mailbox_irq()
clears NBL_CHAN_IRQ_RDY (nbl_core/nbl_dev.c:137-139) without waking…
--
net/nebula-matrix: add common dev start/stop operation
This adds nbl_dev_start() and nbl_dev_stop() to configure the hardware
MSI-X map, allocate MSI-X vectors, request the mailbox interrupt and tear
all of that down again. Thin nbl_core_start() / nbl_core_stop() wrappers
are hooked into the PCI probe and remove paths.
A few questions on the new start/stop paths below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> index 4fc52cadf60f..26b785c9a05f 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
[ ... ]
> @@ -14,6 +25,189 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
> msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
> }
>
[ ... ]
> +static int nbl_dev_disable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
> +{
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_dev_common *dev_common = dev_mgt->common_dev;
> + struct nbl_msix_info *msix_info = &dev_common->msix_info;
> + u16 local_vec_id;
> +
> + if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
> + return 0;
> +
> + local_vec_id =
> + msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
[ ... comment block snipped ... ]
> + chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
> + NBL_CHAN_IRQ_RDY,
> + NBL_CHAN_TYPE_MAILBOX, false);
> +
> + return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
> + local_vec_id, false);
> +}
[Severity: Low]
This isn't a bug in the tree as it stands, but clearing NBL_CHAN_IRQ_RDY
here does not wake the per-message ACK wait queues that test that same
bit.
nbl_chan_send_msg() sleeps on it:
ret = wait_event_timeout(wait_head->wait_queue,
READ_ONCE(wait_head->acked) ||
READ_ONCE(chan_info->shutdn) ||
!test_bit(NBL_CHAN_IRQ_RDY,
chan_info->state),
NBL_CHAN_ACK_WAIT_TIME);
At this point in the series every ack-requesting sender is one of the
nbl_disp_chan_*_req() helpers, all reached only from
nbl_dev_setup_common_dev(), nbl_dev_start() and nbl_dev_stop(), which the
driver core serializes against this path, and nbl_chan_send_ack() sends
with ack = 0 and never sleeps. So no waiter can be stranded today.
Once netdev or ethtool paths start sending mailbox messages, would this
also need the WRITE_ONCE(wait_head->acked, 1) plus wake_up() sequence that
nbl_chan_teardown_queue() already does?
[ ... ]
> @@ -234,3 +436,69 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
> nbl_dev_remove_ctrl_dev(adapter);
> nbl_dev_remove_common_dev(adapter);
> }
> +
> +/* ---------- Dev start process ---------- */
> +int nbl_dev_start(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int cleanup_ret;
> + int ret;
> +
> + ret = nbl_dev_cfg_msix_map(dev_mgt);
> + if (ret)
> + goto cfg_msix_map_err;
[Severity: High]
This is the first live caller of cfg_msix_map() / destroy_msix_map(), so
the control PF now allocates per-function MSI-X state on behalf of the
other PFs. Is there anything on the control PF side that ever releases
that state?
On the control PF, nbl_res_intr_cfg_msix_map() allocates:
tmp_msix_tbl->base_addr = dmam_alloc_coherent(dev, tmp_msix_tbl->size,
&tmp_msix_tbl->dma,
GFP_KERNEL);
...
tmp_interrupts = kcalloc(requested, sizeof(tmp_interrupts[0]), GFP_KERNEL);
stores both in intr_mgt->func_intr_res[func_id] and programs the hardware
map with VALID=1. The only release path is
nbl_res_intr_destroy_msix_map(), which for a peer function runs only when
that peer PF executes its own nbl_dev_stop() and its DESTROY_MSIX_MAP
mailbox request succeeds.
On the control PF side, nbl_intr_mgt_start() has no stop counterpart,
nbl_res_remove_leonis() is a stub:
void nbl_res_remove_leonis(struct nbl_adapter *adap)
{
/*
* No resource release here because all memory uses devm managed
* allocation
*/
}
and nbl_dev_remove_ctrl_dev() only calls disp_ops->deinit_module().
Does unbinding or hot-removing the control PF while another PF is still
bound leak the kcalloc()'d func_intr_res[].interrupts arrays? intr_mgt
itself is devm_kzalloc()'d, so the pointers to those arrays look
unreachable once devres frees it, and the comment above claims everything
is devm managed.
In the same scenario, can the peer function be left with
NBL_PCOMPLETER_FUNCTION_MSIX_MAP still VALID=1 pointing at the coherent
table that devres just reclaimed together with the control PF, so a later
interrupt from that function makes the device DMA-read freed memory? The
peer's own DESTROY_MSIX_MAP request has no manager left at that point, and
nbl_dev_stop() only logs the failure.
> +
> + ret = nbl_dev_init_interrupt_scheme(dev_mgt);
> + if (ret)
> + goto init_interrupt_scheme_err;
> + ret = nbl_dev_request_mailbox_irq(dev_mgt);
> + if (ret)
> + goto mailbox_request_irq_err;
> + ret = nbl_dev_enable_mailbox_irq(dev_mgt);
> + if (ret)
> + goto enable_mailbox_irq_err;
> +
> + return 0;
> +
> +enable_mailbox_irq_err:
> + cleanup_ret = nbl_dev_disable_mailbox_irq(dev_mgt);
> + if (cleanup_ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to disable mailbox IRQ: %d\n", cleanup_ret);
> + nbl_dev_free_mailbox_irq(dev_mgt);
> + cancel_work_sync(&common_dev->clean_mbx_task);
> +mailbox_request_irq_err:
> + nbl_dev_clear_interrupt_scheme(dev_mgt);
> +init_interrupt_scheme_err:
> +cfg_msix_map_err:
> + cleanup_ret = nbl_dev_destroy_msix_map(dev_mgt);
> + if (cleanup_ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to destroy MSI-X map: %d\n", cleanup_ret);
> + cancel_work_sync(&common_dev->clean_mbx_task);
> + return ret;
> +}
> +
> +void nbl_dev_stop(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int ret;
> +
> + ret = nbl_dev_disable_mailbox_irq(dev_mgt);
> + if (ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to disable mailbox IRQ: %d\n", ret);
> + nbl_dev_free_mailbox_irq(dev_mgt);
> + nbl_dev_clear_interrupt_scheme(dev_mgt);
> + ret = nbl_dev_destroy_msix_map(dev_mgt);
> + if (ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to destroy MSI-X map: %d\n", ret);
[Severity: Medium]
Here a failure of nbl_dev_disable_mailbox_irq() is only logged and
teardown carries on. On a non-control PF that helper is a mailbox RPC,
nbl_disp_chan_set_mailbox_irq_req(), which can return -EREMOTEIO,
-ETIMEDOUT or -ESHUTDOWN when the manager PF is unresponsive or already
unbound.
It is also the only operation that clears MSIX_IDX and MSIX_IDX_VALID in
NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id):
nbl_hw_leonis.c:nbl_hw_set_mailbox_irq() {
data &= ~(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK |
NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK);
}
So if that RPC fails, the mailbox queue stays armed to emit an MSI-X
message while nbl_dev_free_mailbox_irq() and then pci_free_irq_vectors()
run. Can that leave the device sending a vector with no handler behind
it?
The remaining masking path is reached over the same mailbox channel:
nbl_interrupt.c:nbl_res_intr_destroy_msix_map() {
for (i = 0; i < intr_num; i++) {
hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
func_id, false, interrupts[i],
0, 0, 0, false);
}
}
and its failure is likewise only logged, so when the channel is wedged
nothing quiesces the interrupt source. Should stop fail loudly or use a
local register write instead of the RPC for the masking step?
Separately, even on the success path nbl_dev_clear_interrupt_scheme()
runs before nbl_dev_destroy_msix_map(), i.e. the kernel-side vectors are
released while the device-side mapping is still valid. Was that ordering
intended, given that only the (fallible) disable makes it safe?
The identical unwind in nbl_dev_start() under enable_mailbox_irq_err
ignores both failures in the same way, so probe can fail with the MSI-X
map and mailbox routing still armed on the device.
> + /*
> + * destroy_msix_map() sends ack-requested messages which may requeue
> + * clean_mbx_task via polling send path. Drain work after the operation.
> + */
> + cancel_work_sync(&common_dev->clean_mbx_task);
> +}
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260831021408.2325-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2026-09-03 17:18 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-31 2:13 [PATCH v26 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-09-03 17:18 ` [v26,net-next,01/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
2026-09-03 17:18 ` [v26,net-next,02/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
2026-09-03 17:18 ` [v26,net-next,03/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
2026-09-03 17:18 ` [v26,net-next,04/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
2026-09-03 17:18 ` [v26,net-next,05/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
2026-09-03 17:18 ` [v26,net-next,06/10] " netdev-bot+sashiko
2026-08-31 2:13 ` [PATCH v26 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
2026-08-31 2:13 ` [PATCH v26 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
2026-09-03 17:18 ` [v26,net-next,08/10] " netdev-bot+sashiko
2026-08-31 2:14 ` [PATCH v26 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-09-03 17:18 ` [v26,net-next,09/10] " netdev-bot+sashiko
2026-08-31 2:14 ` [PATCH v26 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
2026-09-03 17:18 ` [v26,net-next,10/10] " netdev-bot+sashiko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox