* [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs
@ 2026-09-07 12:38 illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
` (9 more replies)
0 siblings, 10 replies; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch series represents the first phase. We plan to integrate it in
two phases: the first phase covers mailbox and chip configuration,
while the second phase involves net dev configuration.
Together, they will provide basic PF-based Ethernet port transmission and
reception capabilities.
After that, we will consider other features, such as ethtool support,
flow management, adminq messaging, VF support, debugfs support, etc.
To ensure compatibility, our architecture is divided into the following
layers:
1. Dev Layer (Device Layer)
The top-level business logic layer where all operations are
device-centric. Every operation is performed relative to the device
context. The intergration of base functions encompasses:
management(ctrl only for leonis pf0), network(net_dev,this time not
contained),common.
2. Dispatch Layer
The distribution from services to specific data operations is mainly
divided into two types: direct pass-through and handling by the
management PF. It shields the upper layer from the differences in
specific underlying locations.
It describes the processing locations and paths of the services.
3. Resource Layer
Handles tasks dispatched from Dispatch Layer. These tasks fall into two
categories:
3.1 Hardware control
The Resource Layer further invokes the HW Layer when hardware access is
needed, as only the HW Layer has OS-level privileges.
3.2 Software resource management
Operations like packet statistics collection that don't require hardware
access.
4. HW Layer (Hardware Layer)
Serves the Resource Layer by interacting with different hardware
chipsets.Writes to hardware registers to drive the hardware based on
Resource Layer directives.
5. Channel Layer
Handle communication between PF0(has ctrl func) and other PF,and provide
basic interaction channels.
6. Common Layer
Provides fundamental services
changes v26->v27
Link to v26:https://lore.kernel.org/netdev/20260831021408.2325-1-illusion.wang@nebula-matrix.com/
1.AI review issues
changes v25->v26
Link to v25:https://lore.kernel.org/netdev/20260819001117.46785-1-illusion.wang@nebula-matrix.com/
1.AI review issues
changes v24->v25
Link to v24:https://lore.kernel.org/netdev/20260807044422.2356-1-illusion.wang@nebula-matrix.com/
1.AI review issues
2.Issues found by Jakub
changes v23->v24
Link to v23:https://lore.kernel.org/netdev/20260731094242.2655-1-illusion.wang@nebula-matrix.com/
1.AI review issues
2.Issues found by Jakub
changes v22->v23
Link to v22:https://lore.kernel.org/netdev/20260723040110.91410-1-illusion.wang@nebula-matrix.com/
AI review issues
chages v21->v22
Link to v21:https://lore.kernel.org/netdev/20260708064742.35391-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v20->v21
Link to v20:https://lore.kernel.org/netdev/20260630010718.4346-1-illusion.wang@nebula-matrix.com/
I have dropped the old Patch 2 titled "add our driver architecture"
Add new Patch 2 "add core driver architecture and HW layer initialization".
Split original patch8 into three separate patches as suggested
changes v19->v20
Link to v19:https://lore.kernel.org/netdev/20260617044702.2439-1-illusion.wang@nebula-matrix.com/
Starting from V20, I have dropped the old Patch 3 titled "P4
configuration invoked during chip initialization". This functionality
will be reimplemented in next phase using the request_firmware() API
with an external firmware blob.
changes v18->v19
Link to v18:https://lore.kernel.org/netdev/20260611044916.2383-1-illusion.wang@nebula-matrix.com/
changes v17->v18
Link to v17:https://lore.kernel.org/netdev/20260601093149.25905-1-illusion.wang@nebula-matrix.com/
changes v16->v17
Link to v16:https://lore.kernel.org/netdev/20260526035453.2359-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v15->v16
Link to v15:https://lore.kernel.org/netdev/20260520032950.4874-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v14->v15
Link to v14:https://lore.kernel.org/netdev/20260513011649.4404-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v13->v14
Link to v13:https://lore.kernel.org/netdev/20260428114910.2616-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v12->v13
Link to v12:https://lore.kernel.org/netdev/20260415033608.2438-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v11->v12
Link to v11:https://lore.kernel.org/netdev/20260408093739.56001-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v10->v11
Link to v10:https://lore.kernel.org/netdev/20260401022318.28550-1-illusion.wang@nebula-matrix.com/
1.Issues found by Mohsin
2.AI review issues
changes v9->v10
Link to v9:https://lore.kernel.org/netdev/20260325040048.2313-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
changes v8->v9
Link to v8:https://lore.kernel.org/netdev/20260317034533.5600-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
Changes v7→v8
Link to v7:https://lore.kernel.org/netdev/20260310120959.22015-1-illusion.wang@nebula-matrix.com/
1.Issues found by Paolo
Changes v6->v7
Link to v6:https://lore.kernel.org/netdev/20260306033451.5196-1-illusion.wang@nebula-matrix.com/
1.Issue found by Jakub
2.AI review issue
Changes v5->v6
Link to V5:https://lore.kernel.org/netdev/20260226073840.3222-1-illusion.wang@nebula-matrix.com/
1.put all standard linux includes files the .c file which needs it & others
--Andrew
2.AI review issue
Changes v4->v5
Link to V4:https://lore.kernel.org/netdev/20260206021608.85381-1-illusion.wang@nebula-matrix.com/
1.change nbl_core to nbl & change ** pointers to *pointers & others
--Andrew
2.AI review issue
Changes v3->v4
Link to v3: https://lore.kernel.org/netdev/20260123011804.31263-1-illusion.wang@nebula-matrix.com
1.cut down to part of a mini driver(mailbox and chip init)
--Jakub Kicinski Simon Horman(some sort of staged approached)
2.modify issues found by ai.
3. Reverse Christmas tree/nbl_err/devm_kfree/remove some macros/
void type to real type/others
--Andrew Lunn
4.change deprecated pci_enable_msix_range to pci_alloc_irq_vectors
5.delete service layer
6.the style of kconfig---Randy Dunlap
7.add to Documentation/networking/device_drivers/ethernet/index.rst
--Simon Horman
Changes v2 →v3
Link to v2: https://lore.kernel.org/netdev/20260109100146.63569-1-illusion.wang@nebula-matrix.com/
1.cut down to a mini driver:
delete vf support
use promisc mode to cut down flow management
drop patch15 in v2
delete adminq msg
delete abnormal handling
delete some unimportant interfaces
2.modify issues found by ai review
Changes v1->v2
Link to v1: https://lore.kernel.org/netdev/20251223035113.31122-1-illusion.wang@nebula-matrix.com/
1.Format Issues and Compilation Issues
- Paolo Abeni
2.add sysfs patch and drop coexisting patch
- Andrew Lunn
3.delete some unimportant ndo operations
4.add machine generated headers patch
5.Modify the issues found in patch1-2 and apply the same fixes to other
patches
6.modify issues found by nipa
illusion wang (10):
net/nebula-matrix: add minimum nbl build framework
net/nebula-matrix: add core driver architecture and HW layer
initialization
net/nebula-matrix: add channel layer
net/nebula-matrix: add common resource implementation
net/nebula-matrix: add intr resource implementation
net/nebula-matrix: add chip-wide hardware init/deinit implementation
net/nebula-matrix: dispatch: add control-level routing core
infrastructure
net/nebula-matrix: dispatch: implement channel RPC framework and
serialize hardware ops
net/nebula-matrix: add common/ctrl dev init/remove operation
net/nebula-matrix: add common dev start/stop operation
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +
MAINTAINERS | 10 +
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 32 +
drivers/net/ethernet/nebula-matrix/Makefile | 6 +
.../net/ethernet/nebula-matrix/nbl/Makefile | 15 +
.../nbl/nbl_channel/nbl_channel.c | 1314 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 170 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 231 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 32 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 45 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 529 +++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 55 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 667 +++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 25 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 32 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 1132 ++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 338 +++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 363 +++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 12 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 76 +
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 544 +++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 152 ++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 110 ++
.../nbl/nbl_include/nbl_def_channel.h | 183 +++
.../nbl/nbl_include/nbl_def_common.h | 69 +
.../nbl/nbl_include/nbl_def_dev.h | 16 +
.../nbl/nbl_include/nbl_def_dispatch.h | 56 +
.../nbl/nbl_include/nbl_def_hw.h | 82 +
.../nbl/nbl_include/nbl_def_resource.h | 38 +
.../nbl/nbl_include/nbl_include.h | 54 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 243 +++
36 files changed, 6695 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
--
2.47.3
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
` (8 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch adds the minimum build infrastructure:
1. Add Kconfig, Makefile and MAINTAINERS entries;
2. Add corresponding driver documentation;
3. Add PCI driver skeleton with empty stubs for nbl driver.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +++++
MAINTAINERS | 10 ++
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 32 ++++++
drivers/net/ethernet/nebula-matrix/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 13 +++
.../nbl/nbl_include/nbl_include.h | 14 +++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 100 ++++++++++++++++++
11 files changed, 212 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
index d9980c84487a..403ae9c2b06a 100644
--- a/Documentation/networking/device_drivers/ethernet/index.rst
+++ b/Documentation/networking/device_drivers/ethernet/index.rst
@@ -48,6 +48,7 @@ Contents:
meta/fbnic
microsoft/netvsc
mucse/rnpgbe
+ nebula-matrix/nbl
netronome/nfp
pensando/ionic
pensando/ionic_rdma
diff --git a/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
new file mode 100644
index 000000000000..fa6e32654439
--- /dev/null
+++ b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
@@ -0,0 +1,28 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================================================================
+Linux Base Driver for Nebula-matrix m18110-NIC/m18000-NIC family
+================================================================
+
+Overview:
+=========
+The m18110-NIC/m18000-NIC (marketed as SNIC s1000) is a series
+of network interface cards for the Data Center Area.
+
+This driver provides the core infrastructure for m18110/m18000-NIC
+devices, including:
+
+- PCI device enumeration and resource (BAR) management
+- Firmware command interface via PF mailbox
+- Channel-based communication between driver and firmware
+- Device initialization and teardown
+
+Support
+=======
+
+For more information about m18110-NIC/m18000-NIC, please visit the following URL:
+https://www.nebula-matrix.com/snic_s1000_en
+
+If an issue is identified with the released source code on the supported kernel
+with a supported adapter, email the specific information related to the issue to
+open@nebula-matrix.com.
diff --git a/MAINTAINERS b/MAINTAINERS
index 460cb7268845..ab55ba0da0a2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18654,6 +18654,16 @@ F: Documentation/devicetree/bindings/hwmon/nuvoton,nct7363.yaml
F: Documentation/hwmon/nct7363.rst
F: drivers/hwmon/nct7363.c
+NEBULA-MATRIX ETHERNET DRIVER (nebula-matrix)
+M: Illusion Wang <illusion.wang@nebula-matrix.com>
+M: Dimon Zhao <dimon.zhao@nebula-matrix.com>
+M: Alvin Wang <alvin.wang@nebula-matrix.com>
+M: Sam Chen <sam.chen@nebula-matrix.com>
+L: netdev@vger.kernel.org
+S: Maintained
+F: Documentation/networking/device_drivers/ethernet/nebula-matrix/
+F: drivers/net/ethernet/nebula-matrix/
+
NETCONSOLE
M: Breno Leitao <leitao@debian.org>
S: Maintained
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 8581ccba1505..c2b0161d0bec 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -130,6 +130,7 @@ config FEALNX
source "drivers/net/ethernet/ni/Kconfig"
source "drivers/net/ethernet/natsemi/Kconfig"
+source "drivers/net/ethernet/nebula-matrix/Kconfig"
source "drivers/net/ethernet/netronome/Kconfig"
source "drivers/net/ethernet/8390/Kconfig"
source "drivers/net/ethernet/nvidia/Kconfig"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index 2b1153d35b52..a306dae23bcb 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -67,6 +67,7 @@ obj-$(CONFIG_NET_VENDOR_MUCSE) += mucse/
obj-$(CONFIG_NET_VENDOR_MYRI) += myricom/
obj-$(CONFIG_FEALNX) += fealnx.o
obj-$(CONFIG_NET_VENDOR_NATSEMI) += natsemi/
+obj-$(CONFIG_NET_VENDOR_NEBULA_MATRIX) += nebula-matrix/
obj-$(CONFIG_NET_VENDOR_NETRONOME) += netronome/
obj-$(CONFIG_NET_VENDOR_NI) += ni/
obj-$(CONFIG_NET_VENDOR_NVIDIA) += nvidia/
diff --git a/drivers/net/ethernet/nebula-matrix/Kconfig b/drivers/net/ethernet/nebula-matrix/Kconfig
new file mode 100644
index 000000000000..dd0e91d14131
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Kconfig
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Nebula-matrix network device configuration
+#
+
+config NET_VENDOR_NEBULA_MATRIX
+ bool "Nebula-matrix devices"
+ default y
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Nebula-matrix cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_NEBULA_MATRIX
+
+config NBL
+ tristate "Nebula-matrix Ethernet Controller m18110/m18000 support"
+ depends on PCI && (64BIT || COMPILE_TEST) && !CPU_BIG_ENDIAN
+ help
+ This driver provides the core infrastructure for Nebula-matrix
+ Ethernet Controller m18110/m18000 Family of devices, including
+ PCI enumeration, firmware mailbox and channel communication.
+
+ More specific information on configuring the driver is in
+ <file:Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst>.
+
+ To compile this driver as a module, choose M here. The module
+ will be called nbl.
+
+endif # NET_VENDOR_NEBULA_MATRIX
diff --git a/drivers/net/ethernet/nebula-matrix/Makefile b/drivers/net/ethernet/nebula-matrix/Makefile
new file mode 100644
index 000000000000..42cdf2db8f0c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Nebula-matrix network device drivers.
+#
+
+obj-$(CONFIG_NBL) += nbl/
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
new file mode 100644
index 000000000000..6c14d1071c0c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Nebula Matrix Limited.
+
+obj-$(CONFIG_NBL) := nbl.o
+
+nbl-objs += nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
new file mode 100644
index 000000000000..a3d63c698aea
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CORE_H_
+#define _NBL_CORE_H_
+
+enum {
+ NBL_CAP_HAS_NET_BIT,
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
new file mode 100644
index 000000000000..16b10bcf1d36
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INCLUDE_H_
+#define _NBL_INCLUDE_H_
+
+#include <linux/types.h>
+
+/* ------ Basic definitions ------- */
+#define NBL_DRIVER_NAME "nbl"
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
new file mode 100644
index 000000000000..199626159a4c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/module.h>
+#include <linux/bits.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_core.h"
+
+static int nbl_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ return -ENODEV;
+}
+
+static void nbl_remove(struct pci_dev *pdev)
+{
+}
+
+/*
+ * PCI Device IDs for Leonis/NBL Network Controllers
+ *
+ * Vendor ID: 0x1F0F
+ * SNIC v3r1 product Device IDs range: 0x3403-0x3412
+ */
+#define NBL_VENDOR_ID 0x1F0F
+
+#define NBL_DEVICE_ID_M18110 0x3403
+#define NBL_DEVICE_ID_M18110_LX 0x3404
+#define NBL_DEVICE_ID_M18110_BASE_T 0x3405
+#define NBL_DEVICE_ID_M18110_LX_BASE_T 0x3406
+#define NBL_DEVICE_ID_M18110_OCP 0x3407
+#define NBL_DEVICE_ID_M18110_LX_OCP 0x3408
+#define NBL_DEVICE_ID_M18110_BASE_T_OCP 0x3409
+#define NBL_DEVICE_ID_M18110_LX_BASE_T_OCP 0x340a
+#define NBL_DEVICE_ID_M18000 0x340b
+#define NBL_DEVICE_ID_M18000_LX 0x340c
+#define NBL_DEVICE_ID_M18000_BASE_T 0x340d
+#define NBL_DEVICE_ID_M18000_LX_BASE_T 0x340e
+#define NBL_DEVICE_ID_M18000_OCP 0x340f
+#define NBL_DEVICE_ID_M18000_LX_OCP 0x3410
+#define NBL_DEVICE_ID_M18000_BASE_T_OCP 0x3411
+#define NBL_DEVICE_ID_M18000_LX_BASE_T_OCP 0x3412
+
+/* All below IDs belong to Leonis ASIC family, different form-factor variants,
+ * share the same hardware initialization flow without differentiated ops.
+ */
+static const struct pci_device_id nbl_id_table[] = {
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ /* required as sentinel */
+ { }
+};
+MODULE_DEVICE_TABLE(pci, nbl_id_table);
+
+static struct pci_driver nbl_driver = {
+ .name = NBL_DRIVER_NAME,
+ .id_table = nbl_id_table,
+ .probe = nbl_probe,
+ .remove = nbl_remove,
+};
+
+module_pci_driver(nbl_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Nebula Matrix Network Driver");
+MODULE_AUTHOR("Illusion Wang <illusion.wang@nebula-matrix.com>");
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
` (7 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add the fundamental driver architecture framework and Leonis hardware
layer initialization for NBL NIC family.
- nbl_adapter/nbl_core/nbl_common_info core device context
- PCI probe/remove entry and basic device capability parsing
- Leonis hardware BAR resource request and ioremap logic
This patch establishes the lowest HW layer and core infrastructure,
preparing for subsequent device implementations.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 3 +-
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 18 ++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 154 ++++++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 15 ++
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 32 ++++
.../nbl/nbl_include/nbl_def_common.h | 32 ++++
.../nbl/nbl_include/nbl_def_hw.h | 17 ++
.../nbl/nbl_include/nbl_include.h | 9 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 94 ++++++++++-
9 files changed, 372 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 6c14d1071c0c..cc060cf8bf75 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,4 +3,5 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_main.o
+nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index a3d63c698aea..1cd6587a8fcb 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -6,8 +6,26 @@
#ifndef _NBL_CORE_H_
#define _NBL_CORE_H_
+#include <linux/pci.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_common.h"
+
enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_core {
+ struct nbl_hw_mgt *hw_mgt;
+};
+
+struct nbl_adapter {
+ struct pci_dev *pdev;
+ struct nbl_core core;
+ struct nbl_common_info common;
+};
+
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param);
+void nbl_core_remove(struct nbl_adapter *adapter);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
new file mode 100644
index 000000000000..0ba0b7f643c6
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/bitfield.h>
+#include "nbl_hw_leonis.h"
+
+/* Structure starts here, adding an op should not modify anything below */
+static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
+{
+ struct device *dev = common->dev;
+ struct nbl_hw_mgt *hw_mgt;
+
+ hw_mgt = devm_kzalloc(dev, sizeof(*hw_mgt), GFP_KERNEL);
+ if (!hw_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ hw_mgt->common = common;
+
+ return hw_mgt;
+}
+
+static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
+ const char *name)
+{
+ int bar;
+ int ret;
+
+ for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
+ if (!(mask & BIT(bar)))
+ continue;
+ ret = pcim_request_region(pdev, bar, name);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter)
+{
+ resource_size_t expect_sz = NBL_MEM_BAR_TOTAL_SIZE;
+ struct nbl_common_info *common = &adapter->common;
+ struct pci_dev *pdev = common->pdev;
+ struct nbl_hw_mgt *hw_mgt = NULL;
+ resource_size_t bar_len;
+ u32 bar_mask;
+ int ret;
+
+ hw_mgt = nbl_hw_setup_hw_mgt(common);
+ if (IS_ERR(hw_mgt)) {
+ ret = PTR_ERR(hw_mgt);
+ goto setup_mgt_fail;
+ }
+ bar_mask = BIT(NBL_MEMORY_BAR) | BIT(NBL_MAILBOX_BAR);
+ ret = nbl_pcim_request_selected_bars(pdev, bar_mask, NBL_DRIVER_NAME);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Request memory bar failed, err = %d\n",
+ ret);
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MEMORY_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MEMORY_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MEMORY BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (common->has_ctrl) {
+ /*
+ * Hardware layout: MEMORY BAR total size is 64M.
+ * The tail NBL_RDMA_NOTIFY_LEN bytes of the 64M BAR are
+ * reserved exclusively for RDMA notify hardware.
+ * Ethernet driver must avoid mapping this reserved tail
+ * to prevent x86 PAT aliasing conflict between eth net
+ * mapping and RDMA driver WC mapping. Mapping starts
+ * at BAR offset 0.
+ *
+ * Skip trailing NBL_RDMA_NOTIFY_LEN bytes at BAR tail.
+ * Round size down to page boundary to avoid ioremap
+ * rounding up and accidentally including RDMA reserved
+ * region when PAGE_SIZE > 8KiB.
+ */
+ if (bar_len < NBL_MEM_BAR_TOTAL_SIZE) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pa smaller than expected %pa\n",
+ &bar_len, &expect_sz);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->hw_size = PAGE_ALIGN_DOWN(NBL_MEM_BAR_TOTAL_SIZE -
+ NBL_RDMA_NOTIFY_LEN);
+ hw_mgt->hw_addr =
+ pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ } else {
+ if (bar_len < NBL_REG_NET_ONLY_LEN) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pa too small for net only reg space\n",
+ &bar_len);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->hw_size = NBL_REG_NET_ONLY_LEN;
+ hw_mgt->hw_addr = pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ }
+ if (!hw_mgt->hw_addr) {
+ dev_err(&pdev->dev, "MEMORY BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MAILBOX_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MAILBOX_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MAILBOX BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (bar_len < NBL_BAR2_MAX_LEN) {
+ dev_err(&pdev->dev, "MAILBOX BAR length %pa too small\n",
+ &bar_len);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->mailbox_bar_hw_addr = pcim_iomap(pdev, NBL_MAILBOX_BAR,
+ bar_len);
+ if (!hw_mgt->mailbox_bar_hw_addr) {
+ dev_err(&pdev->dev, "MAILBOX BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+
+ hw_mgt->mailbox_bar_size = bar_len;
+
+ adapter->core.hw_mgt = hw_mgt;
+
+ return 0;
+
+setup_mgt_fail:
+ return ret;
+}
+
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter)
+{
+ /* All BAR mappings & PCI regions are managed by pcim/devres,
+ * no manual iounmap / release required
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
new file mode 100644
index 000000000000..1f9e509dc631
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_LEONIS_H_
+#define _NBL_HW_LEONIS_H_
+
+#include <linux/types.h>
+
+#include "../../nbl_include/nbl_include.h"
+#include "../nbl_hw_reg.h"
+
+#define NBL_BAR2_MAX_LEN 0x300
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
new file mode 100644
index 000000000000..5e9823e01d39
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_REG_H_
+#define _NBL_HW_REG_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_MEMORY_BAR 0
+#define NBL_MAILBOX_BAR 2
+#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
+#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+/*
+ * PCI MEMORY BAR total size: 64MiB.
+ */
+#define NBL_MEM_BAR_TOTAL_SIZE (64ULL << 20)
+
+struct nbl_hw_mgt {
+ struct nbl_common_info *common;
+ u8 __iomem *hw_addr;
+ u8 __iomem *mailbox_bar_hw_addr;
+ resource_size_t hw_size;
+ resource_size_t mailbox_bar_size;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
new file mode 100644
index 000000000000..da30244fe75d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_COMMON_H_
+#define _NBL_DEF_COMMON_H_
+
+#include <linux/types.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include "nbl_include.h"
+
+struct nbl_common_info {
+ struct pci_dev *pdev;
+ struct device *dev;
+ u32 msg_enable;
+ u16 vsi_id;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 eth_num;
+
+ u8 function;
+ u8 devid;
+ u8 bus;
+ u8 hw_bus;
+
+ u8 has_ctrl;
+ u8 has_net;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
new file mode 100644
index 000000000000..ecbf440e4366
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_HW_H_
+#define _NBL_DEF_HW_H_
+
+#include <linux/types.h>
+
+struct nbl_hw_mgt;
+struct nbl_adapter;
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter);
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 16b10bcf1d36..14e7b19f9a4c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,5 +10,14 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+struct nbl_func_caps {
+ u32 has_ctrl:1;
+ u32 has_net:1;
+ u32 rsv:30;
+};
+
+struct nbl_init_param {
+ struct nbl_func_caps caps;
+};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 199626159a4c..f2552bc73293 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,16 +8,108 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param)
+{
+ struct nbl_common_info *common;
+ struct nbl_adapter *adapter;
+ int ret;
+
+ adapter = devm_kzalloc(&pdev->dev, sizeof(*adapter), GFP_KERNEL);
+ if (!adapter)
+ return ERR_PTR(-ENOMEM);
+
+ adapter->pdev = pdev;
+ common = &adapter->common;
+
+ common->pdev = pdev;
+ common->dev = &pdev->dev;
+ common->has_ctrl = param->caps.has_ctrl;
+ common->has_net = param->caps.has_net;
+ common->function = PCI_FUNC(pdev->devfn);
+ common->devid = PCI_SLOT(pdev->devfn);
+ common->bus = pdev->bus->number;
+
+ ret = nbl_hw_init_leonis(adapter);
+ if (ret)
+ goto hw_init_fail;
+
+ return adapter;
+hw_init_fail:
+ return ERR_PTR(ret);
+}
+
+void nbl_core_remove(struct nbl_adapter *adapter)
+{
+ nbl_hw_remove_leonis(adapter);
+}
+
+static void nbl_get_func_param(struct pci_dev *pdev, kernel_ulong_t driver_data,
+ struct nbl_init_param *param)
+{
+ param->caps.has_net = !!(driver_data & BIT(NBL_CAP_HAS_NET_BIT));
+
+ /*
+ * Hardware fixed rule: physical PF0 is the only management PF with
+ * global ctrl capability. All PFs share identical PCI device ID, so
+ * distinguish control PF via physical function ID.
+ *
+ * Hardware & firmware design FORBID passing any PF through to virtual
+ * machines, there is no scenario where a non-management PF appears
+ * as Func 0 inside guest. Thus using PCI_FUNC(pdev->devfn) to identify
+ * control PF is safe on our platform.
+ */
+ if ((PCI_FUNC(pdev->devfn) == 0) && !pdev->is_virtfn)
+ param->caps.has_ctrl = 1;
+}
+
static int nbl_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
- return -ENODEV;
+ struct nbl_init_param param = { { 0 } };
+ struct device *dev = &pdev->dev;
+ struct nbl_adapter *adapter;
+ int err;
+
+ err = pcim_enable_device(pdev);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to enable PCI dev, err=%d\n", err);
+ return err;
+ }
+
+ nbl_get_func_param(pdev, id->driver_data, ¶m);
+ /* never return fail when DMA_BIT_MASK(64) */
+ dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+
+ pci_set_master(pdev);
+
+ adapter = nbl_core_init(pdev, ¶m);
+ if (IS_ERR(adapter)) {
+ dev_err(dev, "Nbl adapter init fail: %pe\n", adapter);
+ err = PTR_ERR(adapter);
+ goto adapter_init_err;
+ }
+ pci_set_drvdata(pdev, adapter);
+ return 0;
+adapter_init_err:
+ pci_clear_master(pdev);
+ return err;
}
static void nbl_remove(struct pci_dev *pdev)
{
+ struct nbl_adapter *adapter = pci_get_drvdata(pdev);
+
+ if (!adapter)
+ return;
+ pci_set_drvdata(pdev, NULL);
+ nbl_core_remove(adapter);
+
+ pci_clear_master(pdev);
}
/*
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
` (6 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add generic channel management layer for Nebula Matrix network adapter,
which serves as the core inter-PF communication component, providing
standardized message transmission, queue management and hardware
abstraction capabilities.
The new channel layer implements mailbox-based inter-PF(PF0 <-> other PFs)
communication framework with the following core features:
1. Dynamic Message Handling Framework
- Implement hash-table-based message handler management with O(1) lookup by
message type, supporting dynamic message handler registration
- Support two transmission modes: fire-and-forget and synchronous send with
ACK waiting
- Implement dedicated ACK message processing path to match synchronous
request-response semantics
- Add TX slot concurrency control, return -EAGAIN when all outstanding
slots are occupied under high load
- Dual data transmission format: small embedded payload in TX descriptor,
large payload via external DMA buffer
- Support full cleanup of message handlers during driver teardown
2. Mailbox Queue Management
- Initialize TX/RX descriptor rings and data buffers via coherent DMA
allocation and devm managed memory
- Complete queue lifecycle management: hardware queue init/config/stop/
teardown
- Support dual RX processing modes: interrupt-driven and mailbox polling
path, offload heavy RX descriptor cleanup work to dedicated workqueue
to reduce interrupt latency
- Implement inflight TX traffic draining and pending ACK request abortion
during queue teardown
- Queue resource lifecycle design: DMA buffers are allocated once in probe
phase and released automatically by devm on driver remove, dynamic
runtime queue reinit is not supported
- Maintain accurate RX ring empty/full state differentiation via reserved
hole entry mechanism, avoiding ring overflow and repeated reception
3. Hardware Abstraction Layer(HW OPS)
- Abstract hardware-specific mailbox operations into independent hw_ops
layer, decouple channel logic from hardware implementation
- Provide hardware queue configuration, tail pointer doorbell update, PF
mailbox routing table configuration interfaces
- Add per-hw_mgt reg_lock to protect read-modify-write sequences on the
general-purpose register BAR (hw_addr, BAR0), such as QINFO map table
entries and k_pf_mask reads
- Mailbox BAR2 register access is intentionally left unlocked: single
dword mailbox writes are atomic, and the multi-dword QINFO config
sequence (config_mailbox_txq/rxq, stop_mailbox_txq/rxq) only runs
during queue setup/teardown when chan_info->active is false, with no
concurrent producers. TX tail_ptr doorbell updates are serialized by
the per-queue txq_lock. Adding a global lock to the mailbox fast path
was measured to cause CPU soft lockup under re-probe stress.
4. Common Utility & Infrastructure
- Implement generic thread-safe hash table management for message handler
storage
- Create dedicated device-bound workqueue for RX cleanup task
scheduling
- Supplement core data structures, state management and bitmask state
control interfaces
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 4 +-
.../nbl/nbl_channel/nbl_channel.c | 1314 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 170 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 209 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 32 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 7 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 179 +++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 56 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 33 +
.../nbl/nbl_include/nbl_def_channel.h | 126 ++
.../nbl/nbl_include/nbl_def_common.h | 18 +
.../nbl/nbl_include/nbl_def_hw.h | 36 +
.../nbl/nbl_include/nbl_include.h | 3 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 7 +
14 files changed, 2193 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index cc060cf8bf75..04e1aa1fb4bd 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,5 +3,7 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+nbl-objs += nbl_common/nbl_common.o \
+ nbl_channel/nbl_channel.o \
+ nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
new file mode 100644
index 000000000000..879e7bab14e9
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
@@ -0,0 +1,1314 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/bitfield.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/dma-mapping.h>
+#include <linux/atomic.h>
+#include <linux/wait.h>
+#include "nbl_channel.h"
+
+static int nbl_chan_add_msg_handler(struct nbl_channel_mgt *chan_mgt,
+ u16 msg_type, nbl_chan_resp func,
+ void *priv)
+{
+ struct nbl_chan_msg_node_data handler = { 0 };
+ int ret;
+
+ handler.func = func;
+ handler.priv = priv;
+ ret = nbl_common_alloc_hash_node(chan_mgt->handle_hash_tbl, &msg_type,
+ &handler, NULL);
+
+ return ret;
+}
+
+static int nbl_chan_init_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_hash_tbl_key tbl_key = { 0 };
+
+ tbl_key.dev = common->dev;
+ tbl_key.key_size = sizeof(u16);
+ tbl_key.data_size = sizeof(struct nbl_chan_msg_node_data);
+ tbl_key.bucket_size = NBL_CHAN_HANDLER_TBL_BUCKET_SIZE;
+
+ chan_mgt->handle_hash_tbl = nbl_common_init_hash_table(&tbl_key);
+ if (!chan_mgt->handle_hash_tbl)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void nbl_chan_remove_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ if (!chan_mgt->handle_hash_tbl)
+ return;
+ nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl);
+ chan_mgt->handle_hash_tbl = NULL;
+}
+
+static void nbl_chan_init_queue_param(struct nbl_chan_info *chan_info,
+ u16 num_txq_entries, u16 num_rxq_entries,
+ u16 txq_buf_size, u16 rxq_buf_size)
+{
+ chan_info->num_txq_entries = num_txq_entries;
+ chan_info->num_rxq_entries = num_rxq_entries;
+ chan_info->txq_buf_size = txq_buf_size;
+ chan_info->rxq_buf_size = rxq_buf_size;
+ atomic_set(&chan_info->inflight_tx_cnt, 0);
+ WRITE_ONCE(chan_info->shutdn, false);
+ WRITE_ONCE(chan_info->active, false);
+ WRITE_ONCE(chan_info->wait_head_index, 0);
+ memset(chan_info->state, 0, sizeof(chan_info->state));
+ init_waitqueue_head(&chan_info->inflight_wait);
+}
+
+static int nbl_chan_init_tx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_txq_entries * sizeof(struct nbl_chan_tx_desc);
+ u16 i;
+
+ txq->desc.tx_desc =
+ dmam_alloc_coherent(dev, size, &txq->dma, GFP_KERNEL);
+ if (!txq->desc.tx_desc)
+ return -ENOMEM;
+
+ chan_info->wait = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*chan_info->wait), GFP_KERNEL);
+ if (!chan_info->wait)
+ return -ENOMEM;
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ init_waitqueue_head(&chan_info->wait[i].wait_queue);
+ WRITE_ONCE(chan_info->wait[i].status, NBL_MBX_STATUS_IDLE);
+ WRITE_ONCE(chan_info->wait[i].acked, 0);
+ WRITE_ONCE(chan_info->wait[i].ack_data, NULL);
+ WRITE_ONCE(chan_info->wait[i].ack_data_len, 0);
+ WRITE_ONCE(chan_info->wait[i].ack_err, 0);
+ WRITE_ONCE(chan_info->wait[i].msg_type, 0);
+ WRITE_ONCE(chan_info->wait[i].msg_index, 0);
+ WRITE_ONCE(chan_info->wait[i].dstid, 0);
+ }
+
+ txq->buf = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*txq->buf), GFP_KERNEL);
+ if (!txq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_rx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_rxq_entries * sizeof(struct nbl_chan_rx_desc);
+
+ rxq->desc.rx_desc =
+ dmam_alloc_coherent(dev, size, &rxq->dma, GFP_KERNEL);
+ if (!rxq->desc.rx_desc) {
+ dev_err_ratelimited(dev,
+ "Allocate DMA for chan rx descriptor ring failed\n");
+ return -ENOMEM;
+ }
+
+ rxq->buf = devm_kcalloc(dev, chan_info->num_rxq_entries,
+ sizeof(*rxq->buf), GFP_KERNEL);
+ if (!rxq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_init_tx_queue(common, chan_info);
+ if (err)
+ return err;
+
+ err = nbl_chan_init_rx_queue(common, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_config_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info, bool tx)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ struct nbl_chan_ring *ring;
+ dma_addr_t addr;
+ int size_bwid;
+
+ if (tx)
+ ring = &chan_info->txq;
+ else
+ ring = &chan_info->rxq;
+ addr = ring->dma;
+ if (tx) {
+ size_bwid = ilog2(chan_info->num_txq_entries);
+ hw_ops->config_mailbox_txq(p, addr, size_bwid);
+ } else {
+ size_bwid = ilog2(chan_info->num_rxq_entries);
+ hw_ops->config_mailbox_rxq(p, addr, size_bwid);
+ }
+}
+
+static int nbl_chan_alloc_all_tx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ buf = &txq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->txq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan tx queue failed\n");
+ return -ENOMEM;
+ }
+ }
+
+ txq->next_to_clean = 0;
+ txq->next_to_use = 0;
+ txq->tail_ptr = 0;
+
+ return 0;
+}
+
+static void nbl_chan_cfg_qinfo_map_table(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ u32 pf_mask = 0;
+ u8 func_id;
+
+ /*
+ * k_pf_mask rule: bit N == 0 means PF#N enabled, bit N == 1 masked out.
+ * Program mailbox QINFO entry for each hardware-active PF func_id.
+ *
+ * Note: This loop iterates over raw hardware PF func_id.
+ * Upper resource initialization nbl_res_init_pf_num() enforces
+ * product constraints: only 1/2/4 contiguous PFs(PF0 / PF0~1 / PF0~3)
+ * are allowed. Non-contiguous or unsupported PF count will be rejected
+ * before reaching this function.
+ */
+ hw_ops->get_host_pf_mask(p, &pf_mask);
+ for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
+ if (!(pf_mask & (1 << func_id)))
+ hw_ops->cfg_mailbox_qinfo(p, func_id, bus,
+ devid, func_id);
+ }
+}
+
+static int nbl_chan_alloc_all_rx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_rx_desc *desc;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_rxq_entries; i++) {
+ buf = &rxq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->rxq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan rx queue failed\n");
+ goto err;
+ }
+ }
+
+ desc = rxq->desc.rx_desc;
+ /*
+ * Initially leave one RX descriptor unused so that
+ * next_to_clean and next_to_use can distinguish an empty
+ * ring from a full ring.
+ *
+ * The unused slot is replenished as RX descriptors are
+ * consumed and recycled.
+ */
+ for (i = 0; i < chan_info->num_rxq_entries - 1; i++) {
+ buf = &rxq->buf[i];
+ desc[i].buf_addr = cpu_to_le64(buf->pa);
+ desc[i].buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+ desc[i].flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+ }
+
+ rxq->next_to_clean = 0;
+ rxq->next_to_use = chan_info->num_rxq_entries - 1;
+ rxq->tail_ptr = chan_info->num_rxq_entries - 1;
+
+ return 0;
+err:
+ return -ENOMEM;
+}
+
+static int nbl_chan_alloc_all_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_alloc_all_tx_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_rx_bufs(chan_mgt, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_stop_queue(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+
+ hw_ops->stop_mailbox_rxq(chan_mgt->hw_ops_tbl->priv);
+ hw_ops->stop_mailbox_txq(chan_mgt->hw_ops_tbl->priv);
+}
+
+static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_chan_waitqueue_head *wait_head;
+ struct work_struct *task;
+ int ret = 0;
+ u16 i;
+
+ if (!READ_ONCE(chan_info->active)) {
+ dev_warn(chan_mgt->common->dev, "channel not active, skip duplicate teardown\n");
+ return 0;
+ }
+ /*
+ * Step1:
+ * block new sender
+ */
+ mutex_lock(&chan_info->state_lock);
+ WRITE_ONCE(chan_info->shutdn, true);
+ task = READ_ONCE(chan_info->clean_task);
+ WRITE_ONCE(chan_info->clean_task, NULL);
+ mutex_unlock(&chan_info->state_lock);
+ /*
+ * Step2:
+ * abort pending ACK waiters
+ */
+
+ mutex_lock(&chan_info->pending_lock);
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait_head = &chan_info->wait[i];
+ /* Only wake threads that are actually waiting */
+ if (READ_ONCE(wait_head->status) == NBL_MBX_STATUS_WAITING) {
+ /* Update all status fields first */
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->ack_err, (s32)-EIO);
+ /* Ensure status visible before acked flag */
+ smp_wmb();
+ WRITE_ONCE(wait_head->acked, 1);
+ wake_up(&wait_head->wait_queue);
+ }
+ }
+ mutex_unlock(&chan_info->pending_lock);
+ /*
+ * Step3:
+ * wait all sender exit
+ *
+ * Drain strategy mirrors mlx5 command interface teardown:
+ * set shutdown flag first, abort all pending waiters, then
+ * block until inflight_tx_cnt reaches zero.
+ *
+ * A timeout here is treated as an exceptional condition rather
+ * than a fatal error, following the same rationale as mlx5:
+ * - shutdn is already set, so every sender path observes it
+ * at its next checkpoint and exits;
+ * - interrupt-driven senders have a 3s ACK wait
+ * (NBL_CHAN_ACK_WAIT_TIME); polling senders re-check shutdn
+ * on every 1-1.2ms iteration, so both exit promptly after
+ * shutdown is signaled rather than running to full timeout;
+ * - timeout is treated as an exceptional condition; callers
+ * must not access queue resources while an inflight sender
+ * may still be active;
+ * - in that hardware-dead case proceeding with teardown cannot
+ * make the situation worse, and avoids hanging rmmod forever.
+ */
+ ret = wait_event_timeout(chan_info->inflight_wait,
+ atomic_read(&chan_info->inflight_tx_cnt) == 0,
+ msecs_to_jiffies(5000));
+
+ if (!ret) {
+ dev_warn(chan_mgt->common->dev,
+ "teardown: inflight tx drain timeout\n");
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ }
+
+ /* After all TX drained, stop hardware queue */
+ nbl_chan_stop_queue(chan_mgt);
+
+ /* All send paths drained, safely cancel cleanup work */
+ if (task)
+ cancel_work_sync(task);
+ WRITE_ONCE(chan_info->active, false);
+ return ret;
+}
+
+static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ int err;
+
+ if (READ_ONCE(chan_info->active)) {
+ dev_warn(common->dev, "channel already active, reject duplicate setup\n");
+ return -EBUSY;
+ }
+ nbl_chan_init_queue_param(chan_info, NBL_CHAN_QUEUE_LEN,
+ NBL_CHAN_QUEUE_LEN, NBL_CHAN_BUF_LEN,
+ NBL_CHAN_BUF_LEN);
+ err = nbl_chan_init_queue(common, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
+ nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+ WRITE_ONCE(chan_info->active, true);
+ return 0;
+}
+
+static bool nbl_chan_txq_full(struct nbl_chan_ring *txq,
+ u16 num_entries)
+{
+ return NBL_NEXT_ID(txq->next_to_use, num_entries - 1) ==
+ txq->next_to_clean;
+}
+
+static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_tx_param *param)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct nbl_chan_tx_desc *tx_desc;
+ struct nbl_chan_buf *tx_buf;
+
+ if (nbl_chan_txq_full(txq, chan_info->num_txq_entries))
+ return -EBUSY;
+ if (param->arg_len > NBL_CHAN_BUF_LEN - sizeof(*tx_desc))
+ return -EINVAL;
+ tx_desc =
+ NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_use);
+ tx_buf =
+ NBL_CHAN_TX_RING_TO_BUF(txq, txq->next_to_use);
+ tx_desc->dstid = cpu_to_le16(param->dstid);
+ tx_desc->msg_type = cpu_to_le16(param->msg_type);
+ tx_desc->msgid = cpu_to_le16(param->msgid);
+
+ /*
+ * srcid field is filled by mailbox hardware after peer receives this
+ * packet, driver producer never writes srcid; reused descriptor slots
+ * will contain stale srcid value temporarily until hardware overwrites
+ * it.
+ */
+ if (param->arg_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ if (param->arg)
+ memcpy(tx_buf->va, param->arg, param->arg_len);
+ tx_desc->buf_addr = cpu_to_le64(tx_buf->pa);
+ tx_desc->buf_len = cpu_to_le16(param->arg_len);
+ tx_desc->data_len = 0;
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ } else {
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ memset(&tx_desc->buf_addr, 0, sizeof(tx_desc->buf_addr));
+ if (param->arg && param->arg_len > 0)
+ memcpy(tx_desc->data, param->arg, param->arg_len);
+ tx_desc->buf_len = 0;
+ tx_desc->data_len = cpu_to_le16(param->arg_len);
+ }
+ /* Ensure descriptor data visible to device before AVAIL flag */
+ dma_wmb();
+ tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
+
+ txq->next_to_use =
+ NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
+ txq->tail_ptr++;
+
+ return 0;
+}
+
+/*
+ * Quiesce the TX mailbox queue and reclaim all outstanding
+ * descriptors. Called from the timeout path of nbl_chan_kick_tx_ring()
+ * with txq_lock held.
+ *
+ * The device failed to fetch/complete the current descriptor within the
+ * polling window. We assert QUEUE_RST to stop further DMA fetches,
+ * reclaim every descriptor between next_to_clean and next_to_use,
+ * reset the software tail_ptr counter to match the hardware reset state,
+ * and re-enable the queue so subsequent sends can proceed.
+ */
+static void nbl_chan_quiesce_and_reclaim_tx(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *hw_priv = chan_mgt->hw_ops_tbl->priv;
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct nbl_chan_tx_desc *tx_desc;
+
+ /* Assert QUEUE_RST to stop hardware fetching new descriptors */
+ hw_ops->stop_mailbox_txq(hw_priv);
+ /* Ensure reset write is posted before reclaiming descriptors */
+ wmb();
+
+ /*
+ * Reclaim all outstanding descriptors between next_to_clean and
+ * next_to_use. Under txq_lock there is at most one in-flight
+ * descriptor, but iterate the full range for robustness.
+ */
+ while (txq->next_to_clean != txq->next_to_use) {
+ tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq,
+ txq->next_to_clean);
+ WRITE_ONCE(tx_desc->flags, 0);
+ txq->next_to_clean =
+ NBL_NEXT_ID(txq->next_to_clean,
+ chan_info->num_txq_entries - 1);
+ }
+
+ /*
+ * Hardware tail_ptr counter is cleared by QUEUE_RST. Reset
+ * software counter to match so the next doorbell update does
+ * not produce a false 16-bit wrap delta.
+ */
+ txq->tail_ptr = 0;
+
+ /* Re-enable queue with current ring base and size */
+ nbl_chan_config_queue(chan_mgt, chan_info, true);
+}
+
+static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ int max_retries = NBL_CHAN_TX_WAIT_TIMES;
+ struct nbl_chan_tx_desc *tx_desc;
+ int retry_count = 0;
+ u16 msg_type;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ txq->tail_ptr, NBL_MB_TX_QID);
+
+ tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_clean);
+ /*
+ * Poll for HW to mark descriptor as USED.
+ * Mailbox is a low-speed control channel for management commands.
+ * We avoid enabling dedicated per-TX interrupt for single control
+ * message to reduce interrupt overhead, so use bounded polling
+ * with small delay instead.
+ */
+ while (retry_count < max_retries) {
+ if (READ_ONCE(chan_info->shutdn))
+ return -ESHUTDOWN;
+
+ if (le16_to_cpu(READ_ONCE(tx_desc->flags)) &
+ BIT(NBL_CHAN_TX_DESC_USED)) {
+ /*
+ * Order reads of other device-written descriptor
+ * fields after observing USED. Matches the RX side
+ * pattern in nbl_chan_clean_queue().
+ */
+ dma_rmb();
+ break;
+ }
+
+ retry_count++;
+ if (retry_count == max_retries) {
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_err_ratelimited(dev, "chan send msg type: %d timeout\n",
+ msg_type);
+ /*
+ * Device failed to complete this descriptor.
+ * Quiesce the queue, reclaim the timed-out
+ * descriptor, and re-enable so future sends can
+ * proceed instead of stalling the ring full.
+ */
+ nbl_chan_quiesce_and_reclaim_tx(chan_mgt,
+ chan_info);
+ return -ETIMEDOUT;
+ }
+ usleep_range(NBL_CHAN_TX_WAIT_US, NBL_CHAN_TX_WAIT_US_MAX);
+ }
+
+ txq->next_to_clean = txq->next_to_use;
+
+ return 0;
+}
+
+static void nbl_chan_recv_ack_msg(void *priv, u16 srcid, u16 msgid, void *data,
+ u32 data_len)
+{
+ struct nbl_channel_mgt *chan_mgt = (struct nbl_channel_mgt *)priv;
+ struct nbl_chan_waitqueue_head *wait_head = NULL;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ u16 w_dstid, w_msgtype, w_msgidx;
+ u32 *payload = data;
+ u16 ack_msgtype = 0;
+ u16 ack_msgid = 0;
+ u32 ack_datalen;
+ void *ack_data;
+ u32 copy_len;
+ int w_status;
+
+ if (READ_ONCE(chan_info->shutdn))
+ return;
+ if (data_len > NBL_CHAN_BUF_LEN ||
+ data_len < NBL_CHAN_ACK_HEAD_LEN * sizeof(u32)) {
+ dev_err_ratelimited(dev, "Invalid ACK data_len: %u\n",
+ data_len);
+ return;
+ }
+ ack_datalen = data_len - NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ ack_msgtype = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_TYPE_POS));
+ ack_msgid = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_ID_POS));
+ if (FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid) >=
+ chan_info->num_txq_entries) {
+ dev_err_ratelimited(dev, "chan recv msg id: %u err\n",
+ ack_msgid);
+ return;
+ }
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid)];
+
+ mutex_lock(&chan_info->pending_lock);
+
+ /* Cache repeated READ_ONCE values */
+ w_dstid = READ_ONCE(wait_head->dstid);
+ w_status = READ_ONCE(wait_head->status);
+ w_msgtype = READ_ONCE(wait_head->msg_type);
+ w_msgidx = READ_ONCE(wait_head->msg_index);
+
+ if (srcid != w_dstid) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev, "ACK srcid=%u != dstid=%u, rejecting\n",
+ srcid, w_dstid);
+ return;
+ }
+ if (w_status != NBL_MBX_STATUS_WAITING) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack invalid status, wait msgtype:%u idx:%u status:%d ack msgtype:%u msgid:%u datalen:%u\n",
+ w_msgtype, w_msgidx, w_status,
+ ack_msgtype, ack_msgid, ack_datalen);
+ return;
+ }
+
+ if (w_msgtype != ack_msgtype) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack msgtype mismatch, wait msgtype:%u idx:%u ack msgtype:%u msgid:%u\n",
+ w_msgtype, w_msgidx, ack_msgtype,
+ ack_msgid);
+ return;
+ }
+ if (FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, ack_msgid) != w_msgidx) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Stale ACK: expected index=%u, got msgid=%u\n",
+ w_msgidx, ack_msgid);
+ return;
+ }
+
+ WRITE_ONCE(wait_head->ack_err,
+ (s32)le32_to_cpu(*(__le32 *)&payload[NBL_CHAN_ACK_RET_POS]));
+
+ copy_len = min_t(u32, READ_ONCE(wait_head->ack_data_len), ack_datalen);
+ if (READ_ONCE(wait_head->ack_err) >= 0 && copy_len > 0) {
+ ack_data = READ_ONCE(wait_head->ack_data);
+ if (!ack_data) {
+ dev_err_ratelimited(dev, "ACK payload dropped: ack_data is NULL\n");
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ goto ack_done;
+ }
+ memcpy((char *)ack_data,
+ payload + NBL_CHAN_ACK_HEAD_LEN, copy_len);
+ WRITE_ONCE(wait_head->ack_data_len, (u16)copy_len);
+ } else {
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ }
+ack_done:
+ /* Guarantee payload data finished before acked flag visible */
+ smp_wmb();
+ WRITE_ONCE(wait_head->acked, 1);
+ mutex_unlock(&chan_info->pending_lock);
+ wake_up(&wait_head->wait_queue);
+}
+
+static void nbl_chan_recv_msg(struct nbl_channel_mgt *chan_mgt, void *data)
+{
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_msg_node_data *msg_handler;
+ u16 msg_type, payload_len, srcid, msgid;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct nbl_chan_tx_desc *tx_desc;
+ void *payload;
+ size_t avail_space;
+ u16 data_len_fw;
+
+ if (READ_ONCE(chan_info->shutdn))
+ return;
+
+ tx_desc = data;
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_dbg(dev, "recv msg_type: %d\n", msg_type);
+
+ srcid = le16_to_cpu(READ_ONCE(tx_desc->srcid));
+ msgid = le16_to_cpu(READ_ONCE(tx_desc->msgid));
+
+ if (msg_type >= NBL_CHAN_MSG_MAILBOX_MAX)
+ return;
+
+ data_len_fw = le16_to_cpu(READ_ONCE(tx_desc->data_len));
+ if (data_len_fw) {
+ payload_len = data_len_fw;
+
+ if (payload_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ dev_err_ratelimited(dev,
+ "data_len=%u exceeds embedded buffer size=%u\n",
+ payload_len,
+ NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN);
+ return;
+ }
+ /* Small pkt: payload stored inside descriptor data[] array */
+ payload = tx_desc->data;
+ } else {
+ payload_len = le16_to_cpu(READ_ONCE(tx_desc->buf_len));
+
+ avail_space = NBL_CHAN_BUF_LEN - sizeof(*tx_desc);
+ if (payload_len > avail_space) {
+ dev_err_ratelimited(dev,
+ "buf_len=%u exceeds external buffer size=%zu\n",
+ payload_len, avail_space);
+ return;
+ }
+ /* Large pkt: payload follows immediately after tx_desc */
+ payload = tx_desc + 1;
+ }
+
+ msg_handler = nbl_common_get_hash_node(chan_mgt->handle_hash_tbl,
+ &msg_type);
+ if (!msg_handler || !msg_handler->func) {
+ dev_err_ratelimited(dev,
+ "No handler for msg_type: %u (srcid=%u, msgid=%u)\n",
+ msg_type, srcid, msgid);
+ return;
+ }
+
+ msg_handler->func(msg_handler->priv, srcid, msgid, payload,
+ payload_len);
+}
+
+static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_ring *rxq)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ u16 next_to_use;
+
+ next_to_use = rxq->next_to_use;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_use);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_use);
+
+ /*
+ * Recycle the RX descriptor at next_to_use. The initial
+ * unused slot is intentionally recycled after the first
+ * RX descriptor is consumed, allowing the ring to become
+ * fully populated while next_to_clean tracks the consumer.
+ */
+ rx_desc->buf_addr = cpu_to_le64(rx_buf->pa);
+ rx_desc->buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+
+ /*
+ * DMA Write Memory Barrier:
+ * Ensures all previous DMA-mapped writes (buffer address/length)
+ * are completed before the descriptor flags are updated.
+ * This prevents hardware from seeing a partially updated descriptor
+ * where flags are set but buffer info isn't ready yet.
+ */
+ dma_wmb();
+
+ rx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+
+ rxq->next_to_use++;
+ if (rxq->next_to_use == chan_info->num_rxq_entries)
+ rxq->next_to_use = 0;
+ rxq->tail_ptr++;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+}
+
+static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ u32 budget = NBL_CHAN_RX_CLEAN_BUDGET;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ struct work_struct *task;
+ bool more_work = false;
+ u16 next_to_clean;
+ u16 flags;
+
+ next_to_clean = rxq->next_to_clean;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ while (le16_to_cpu(READ_ONCE(rx_desc->flags)) &
+ BIT(NBL_CHAN_RX_DESC_USED)) {
+ flags = le16_to_cpu(READ_ONCE(rx_desc->flags));
+
+ if (READ_ONCE(chan_info->shutdn))
+ break;
+ if (!(flags & BIT(NBL_CHAN_RX_DESC_WRITE)))
+ dev_dbg(dev,
+ "mailbox rx flag 0x%x missing NBL_CHAN_RX_DESC_WRITE\n",
+ flags);
+
+ /* Make sure hardware written descriptor visible to CPU */
+ dma_rmb();
+ nbl_chan_recv_msg(chan_mgt, rx_buf->va);
+ nbl_chan_advance_rx_ring(chan_mgt, chan_info, rxq);
+ next_to_clean++;
+ if (next_to_clean == chan_info->num_rxq_entries)
+ next_to_clean = 0;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ if (--budget == 0) {
+ more_work = true;
+ break;
+ }
+ cond_resched();
+ }
+ rxq->next_to_clean = next_to_clean;
+
+ mutex_lock(&chan_info->state_lock);
+ /* Prevent queue_work after teardown clears clean_task */
+ if (READ_ONCE(chan_info->shutdn)) {
+ mutex_unlock(&chan_info->state_lock);
+ return;
+ }
+ if (common->wq && more_work) {
+ task = READ_ONCE(chan_info->clean_task);
+ if (task)
+ queue_work(common->wq, task);
+ }
+ mutex_unlock(&chan_info->state_lock);
+}
+
+static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ nbl_chan_clean_queue(chan_mgt, chan_info);
+}
+
+static int nbl_chan_get_msg_id(struct nbl_chan_info *chan_info,
+ u16 *msgid)
+{
+ int search_loc = READ_ONCE(chan_info->wait_head_index), i;
+ struct nbl_chan_waitqueue_head *wait = NULL;
+ int status;
+ int next;
+
+ lockdep_assert_held(&chan_info->pending_lock);
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait = &chan_info->wait[search_loc];
+ status = READ_ONCE(wait->status);
+ if (status == NBL_MBX_STATUS_IDLE ||
+ status == NBL_MBX_STATUS_TIMEOUT) {
+ WRITE_ONCE(wait->msg_index,
+ NBL_NEXT_ID(wait->msg_index,
+ NBL_CHAN_MSG_INDEX_MAX));
+
+ *msgid = FIELD_PREP(NBL_CHAN_MSGID_INDEX_MASK,
+ wait->msg_index) |
+ FIELD_PREP(NBL_CHAN_MSGID_LOC_MASK,
+ search_loc);
+
+ /* Advance starting search position for next caller */
+ next = NBL_NEXT_ID(search_loc,
+ chan_info->num_txq_entries - 1);
+ WRITE_ONCE(chan_info->wait_head_index, next);
+ return 0;
+ }
+
+ search_loc = NBL_NEXT_ID(search_loc,
+ chan_info->num_txq_entries - 1);
+ }
+
+ /*
+ * All tx slots are occupied. May happen under high transmit load
+ * or delayed remote ACK responses. Caller should retry later.
+ */
+ return -EAGAIN;
+}
+
+static void nbl_chan_reset_wait_head(struct nbl_chan_info *chan_info,
+ struct nbl_chan_waitqueue_head *wait_head)
+{
+ lockdep_assert_held(&chan_info->pending_lock);
+
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ WRITE_ONCE(wait_head->ack_err, 0);
+ WRITE_ONCE(wait_head->msg_type, 0);
+ WRITE_ONCE(wait_head->dstid, 0);
+}
+
+static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_waitqueue_head *wait_head = NULL;
+ struct nbl_chan_tx_param tx_param = { 0 };
+ int i = NBL_CHAN_TX_WAIT_ACK_TIMES;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct device *dev = common->dev;
+ struct work_struct *task;
+ u16 msgid = 0;
+ int ret;
+
+ if (chan_send->resp_len > NBL_CHAN_BUF_LEN) {
+ dev_err_ratelimited(dev, "resp_len %zu exceeds max %d\n",
+ chan_send->resp_len, NBL_CHAN_BUF_LEN);
+ return -EINVAL;
+ }
+
+ mutex_lock(&chan_info->state_lock);
+ if (READ_ONCE(chan_info->shutdn)) {
+ mutex_unlock(&chan_info->state_lock);
+ return -ESHUTDOWN;
+ }
+ atomic_inc(&chan_info->inflight_tx_cnt);
+ mutex_unlock(&chan_info->state_lock);
+
+ tx_param.msg_type = chan_send->msg_type;
+ tx_param.arg = chan_send->arg;
+ tx_param.arg_len = chan_send->arg_len;
+ tx_param.dstid = chan_send->dstid;
+ tx_param.msgid = msgid;
+ if (chan_send->ack) {
+ mutex_lock(&chan_info->pending_lock);
+
+ ret = nbl_chan_get_msg_id(chan_info, &msgid);
+ if (ret) {
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Channel tx wait head full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type,
+ chan_send->dstid);
+ goto out_clean_inflight;
+ }
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK,
+ msgid)];
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, chan_send->resp);
+ WRITE_ONCE(wait_head->ack_data_len, chan_send->resp_len);
+ WRITE_ONCE(wait_head->msg_type, chan_send->msg_type);
+ WRITE_ONCE(wait_head->msg_index,
+ FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, msgid));
+ WRITE_ONCE(wait_head->dstid, chan_send->dstid);
+
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_WAITING);
+ mutex_unlock(&chan_info->pending_lock);
+
+ tx_param.msgid = msgid;
+ }
+
+ mutex_lock(&chan_info->txq_lock);
+ ret = nbl_chan_update_txqueue(chan_mgt, chan_info, &tx_param);
+ if (ret) {
+ mutex_unlock(&chan_info->txq_lock);
+ dev_err_ratelimited(dev,
+ "Channel tx queue full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type, chan_send->dstid);
+ if (wait_head)
+ goto out_clear_wait_slot;
+ goto out_clean_inflight;
+ }
+
+ ret = nbl_chan_kick_tx_ring(chan_mgt, chan_info);
+ mutex_unlock(&chan_info->txq_lock);
+ if (ret) {
+ if (wait_head)
+ goto out_clear_wait_slot;
+ goto out_clean_inflight;
+ }
+
+ if (!chan_send->ack) {
+ ret = 0;
+ goto out_clean_inflight;
+ }
+
+ if (test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) {
+ while (!READ_ONCE(wait_head->acked)) {
+ /*
+ * avoids long task blocking when interrupt mode is
+ * disabled mid-wait. Cannot guarantee subsequent ACK
+ * delivery after interrupt mask off, only prevents
+ * infinite blocking. Spurious timeout is possible.
+ */
+ ret = wait_event_timeout(wait_head->wait_queue,
+ READ_ONCE(wait_head->acked) ||
+ READ_ONCE(chan_info->shutdn) ||
+ !test_bit(NBL_CHAN_IRQ_RDY,
+ chan_info->state),
+ NBL_CHAN_ACK_WAIT_TIME);
+
+ if (READ_ONCE(chan_info->shutdn)) {
+ ret = -ESHUTDOWN;
+ goto out_clear_wait_slot;
+ }
+ if (!test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) {
+ ret = -EIO;
+ goto out_clear_wait_slot;
+ }
+ if (ret == 0) {
+ mutex_lock(&chan_info->pending_lock);
+ if (READ_ONCE(wait_head->status) ==
+ NBL_MBX_STATUS_WAITING) {
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ /*
+ * Ensure all status/ack slot
+ * updates are visible before subsequent
+ * readers observe acked == 0
+ */
+ smp_wmb();
+ }
+ mutex_unlock(&chan_info->pending_lock);
+ dev_err_ratelimited(dev,
+ "Channel waiting ack failed, message type: %d, msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ /*
+ * TIMEOUT slots can be reused by another
+ * sender. The current sender no longer
+ * owns the slot after transitioning it to
+ * TIMEOUT.
+ */
+ goto out_clean_inflight;
+ }
+
+ if (READ_ONCE(wait_head->acked))
+ break;
+ }
+ if (READ_ONCE(wait_head->acked)) {
+ /*
+ * Load ordering: observe acked flag before
+ * reading ACK payload metadata.
+ */
+ smp_rmb();
+ chan_send->ack_len = READ_ONCE(wait_head->ack_data_len);
+ ret = READ_ONCE(wait_head->ack_err);
+ }
+ } else {
+ /* Polling path for synchronous ACK */
+ while (i--) {
+ if (READ_ONCE(chan_info->shutdn)) {
+ ret = -ESHUTDOWN;
+ goto out_clear_wait_slot;
+ }
+
+ mutex_lock(&chan_info->state_lock);
+ task = READ_ONCE(chan_info->clean_task);
+ if (common->wq && task &&
+ !READ_ONCE(chan_info->shutdn) &&
+ !work_pending(task))
+ queue_work(common->wq, task);
+ mutex_unlock(&chan_info->state_lock);
+ if (READ_ONCE(wait_head->acked)) {
+ /*
+ * Guarantee load order: observe acked
+ * flag before reading ack payload metadata.
+ */
+ smp_rmb();
+ chan_send->ack_len =
+ READ_ONCE(wait_head->ack_data_len);
+ ret = READ_ONCE(wait_head->ack_err);
+ goto out_clear_wait_slot;
+ }
+
+ usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
+ NBL_CHAN_TX_WAIT_ACK_US_MAX);
+ cond_resched();
+ }
+
+ dev_err_ratelimited(dev,
+ "Channel polling ack failed, message type: %d msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ }
+
+out_clear_wait_slot:
+ mutex_lock(&chan_info->pending_lock);
+ nbl_chan_reset_wait_head(chan_info, wait_head);
+ mutex_unlock(&chan_info->pending_lock);
+
+out_clean_inflight:
+ mutex_lock(&chan_info->state_lock);
+ if (atomic_dec_and_test(&chan_info->inflight_tx_cnt))
+ wake_up(&chan_info->inflight_wait);
+ mutex_unlock(&chan_info->state_lock);
+ return ret;
+}
+
+static int nbl_chan_send_ack(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack)
+{
+ size_t head_len = NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ size_t data_len = chan_ack->data_len;
+ struct nbl_chan_send_info chan_send;
+ __le32 *tmp;
+ size_t len;
+ int ret;
+
+ if (data_len >
+ NBL_CHAN_BUF_LEN - sizeof(struct nbl_chan_tx_desc) - head_len)
+ return -EINVAL;
+
+ len = head_len + data_len;
+ tmp = kzalloc(len, GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ *(__le16 *)&tmp[NBL_CHAN_MSG_TYPE_POS] =
+ cpu_to_le16(chan_ack->msg_type);
+ *(__le16 *)&tmp[NBL_CHAN_MSG_ID_POS] = cpu_to_le16(chan_ack->msgid);
+ tmp[NBL_CHAN_ACK_RET_POS] = cpu_to_le32(chan_ack->err);
+ if (chan_ack->data && chan_ack->data_len)
+ memcpy(&tmp[NBL_CHAN_ACK_HEAD_LEN], chan_ack->data,
+ chan_ack->data_len);
+
+ nbl_chan_fill_send_info(&chan_send, chan_ack->dstid, NBL_CHAN_MSG_ACK,
+ tmp, len, NULL, 0, 0);
+ ret = nbl_chan_send_msg(chan_mgt, &chan_send);
+ kfree(tmp);
+
+ return ret;
+}
+
+static int nbl_chan_register_msg(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback)
+{
+ return nbl_chan_add_msg_handler(chan_mgt, msg_type, func, callback);
+}
+
+static bool nbl_chan_check_queue_exist(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ return chan_info ? true : false;
+}
+
+static void nbl_chan_register_chan_task(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ mutex_lock(&chan_info->state_lock);
+ if (!READ_ONCE(chan_info->shutdn))
+ WRITE_ONCE(chan_info->clean_task, task);
+ mutex_unlock(&chan_info->state_lock);
+}
+
+static void nbl_chan_set_queue_state(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ int i;
+
+ if (set)
+ set_bit(state, chan_info->state);
+ else
+ clear_bit(state, chan_info->state);
+ /*
+ * When clearing IRQ_RDY, wake all per-slot wait queues so
+ * sleeping senders observe the condition immediately and
+ * return -EIO instead of waiting out the 3s timeout and
+ * reporting -ETIMEDOUT.
+ */
+ if (!set && state == NBL_CHAN_IRQ_RDY) {
+ for (i = 0; i < chan_info->num_txq_entries; i++)
+ wake_up_all(&chan_info->wait[i].wait_queue);
+ }
+}
+
+static struct nbl_channel_ops chan_ops = {
+ .send_msg = nbl_chan_send_msg,
+ .send_ack = nbl_chan_send_ack,
+ .register_msg = nbl_chan_register_msg,
+ .unregister_all_msg = nbl_chan_remove_msg_handler,
+ .cfg_chan_qinfo_map_table = nbl_chan_cfg_qinfo_map_table,
+ .check_queue_exist = nbl_chan_check_queue_exist,
+ .setup_queue = nbl_chan_setup_queue,
+ .teardown_queue = nbl_chan_teardown_queue,
+ .clean_queue_subtask = nbl_chan_clean_queue_subtask,
+ .register_chan_task = nbl_chan_register_chan_task,
+ .set_queue_state = nbl_chan_set_queue_state,
+};
+
+static struct nbl_channel_mgt *
+nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adapter->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adapter->common;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ struct nbl_chan_info *mailbox;
+ int ret;
+
+ chan_mgt = devm_kzalloc(dev, sizeof(*chan_mgt), GFP_KERNEL);
+ if (!chan_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ chan_mgt->common = common;
+ chan_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ mailbox = devm_kzalloc(dev, sizeof(*mailbox), GFP_KERNEL);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+ mailbox->chan_type = NBL_CHAN_TYPE_MAILBOX;
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX] = mailbox;
+
+ ret = nbl_chan_init_msg_handler(chan_mgt);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->txq_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->state_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ ret = devm_mutex_init(common->dev, &mailbox->pending_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ return chan_mgt;
+}
+
+static struct nbl_channel_ops_tbl *
+nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ int ret;
+
+ chan_ops_tbl = devm_kzalloc(dev, sizeof(*chan_ops_tbl), GFP_KERNEL);
+ if (!chan_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!chan_ops.send_msg || !chan_ops.send_ack ||
+ !chan_ops.register_msg || !chan_ops.unregister_all_msg ||
+ !chan_ops.cfg_chan_qinfo_map_table ||
+ !chan_ops.check_queue_exist || !chan_ops.setup_queue ||
+ !chan_ops.teardown_queue || !chan_ops.clean_queue_subtask ||
+ !chan_ops.register_chan_task || !chan_ops.set_queue_state)
+ return ERR_PTR(-EINVAL);
+
+ chan_ops_tbl->ops = &chan_ops;
+ chan_ops_tbl->priv = chan_mgt;
+
+ ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
+ nbl_chan_recv_ack_msg, chan_mgt);
+ if (ret)
+ return ERR_PTR(ret);
+
+ return chan_ops_tbl;
+}
+
+int nbl_chan_init_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ int ret;
+
+ chan_mgt = nbl_chan_setup_chan_mgt(adap);
+ if (IS_ERR(chan_mgt)) {
+ ret = PTR_ERR(chan_mgt);
+ goto exit;
+ }
+
+ chan_ops_tbl = nbl_chan_setup_ops(dev, chan_mgt);
+ if (IS_ERR(chan_ops_tbl)) {
+ ret = PTR_ERR(chan_ops_tbl);
+ goto cleanup_mgt;
+ }
+
+ adap->intf.channel_ops_tbl = chan_ops_tbl;
+ adap->core.chan_mgt = chan_mgt;
+ ret = nbl_common_create_wq(&adap->common);
+ if (ret)
+ goto cleanup_mgt;
+ return 0;
+
+cleanup_mgt:
+ nbl_chan_remove_msg_handler(chan_mgt);
+exit:
+ return ret;
+}
+
+void nbl_chan_remove_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_mgt *chan_mgt = adap->core.chan_mgt;
+
+ if (!chan_mgt)
+ return;
+ nbl_common_destroy_wq(&adap->common);
+ /*
+ * All channel queues shall be torn down earlier in remove path
+ * to drain inflight tx workers and stop hardware before destroying
+ * message handler hash table.
+ */
+ nbl_chan_remove_msg_handler(chan_mgt);
+ adap->core.chan_mgt = NULL;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
new file mode 100644
index 000000000000..e6d96768a5b5
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
@@ -0,0 +1,170 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHANNEL_H_
+#define _NBL_CHANNEL_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_CHAN_TX_RING_TO_DESC(tx_ring, i) \
+ (&((((tx_ring)->desc.tx_desc))[i]))
+#define NBL_CHAN_RX_RING_TO_DESC(rx_ring, i) \
+ (&((((rx_ring)->desc.rx_desc))[i]))
+#define NBL_CHAN_TX_RING_TO_BUF(tx_ring, i) (&(((tx_ring)->buf)[i]))
+#define NBL_CHAN_RX_RING_TO_BUF(rx_ring, i) (&(((rx_ring)->buf)[i]))
+
+#define NBL_CHAN_TX_WAIT_US 100
+#define NBL_CHAN_TX_WAIT_US_MAX 120
+#define NBL_CHAN_TX_WAIT_TIMES 100
+#define NBL_CHAN_TX_WAIT_ACK_US_MIN 1000
+#define NBL_CHAN_TX_WAIT_ACK_US_MAX 1200
+#define NBL_CHAN_TX_WAIT_ACK_TIMES 5000
+#define NBL_CHAN_QUEUE_LEN 256
+#define NBL_CHAN_BUF_LEN 4096
+#define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
+
+#define NBL_CHAN_TX_DESC_AVAIL 0
+#define NBL_CHAN_TX_DESC_USED 1
+#define NBL_CHAN_RX_DESC_WRITE 1
+#define NBL_CHAN_RX_DESC_AVAIL 3
+#define NBL_CHAN_RX_DESC_USED 4
+
+#define NBL_CHAN_ACK_HEAD_LEN 3
+#define NBL_CHAN_ACK_RET_POS 2
+#define NBL_CHAN_MSG_ID_POS 1
+#define NBL_CHAN_MSG_TYPE_POS 0
+
+#define NBL_CHAN_ACK_WAIT_TIME (3 * HZ)
+#define NBL_CHAN_RX_CLEAN_BUDGET 64
+#define NBL_CHAN_HANDLER_TBL_BUCKET_SIZE 512
+
+enum {
+ NBL_MB_RX_QID = 0,
+ NBL_MB_TX_QID = 1,
+};
+
+enum {
+ NBL_MBX_STATUS_IDLE = 0,
+ NBL_MBX_STATUS_WAITING,
+ NBL_MBX_STATUS_TIMEOUT,
+};
+
+struct nbl_chan_tx_param {
+ enum nbl_chan_msg_type msg_type;
+ void *arg;
+ size_t arg_len;
+ u16 dstid;
+ u16 msgid;
+};
+
+struct nbl_chan_buf {
+ void *va;
+ dma_addr_t pa;
+ size_t size;
+};
+
+struct nbl_chan_tx_desc {
+ __le16 flags;
+ __le16 srcid;
+ __le16 dstid;
+ __le16 data_len;
+ __le16 buf_len;
+ __le64 buf_addr;
+ __le16 msg_type;
+ u8 data[NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN];
+ __le16 msgid;
+ u8 rsv[26];
+} __packed;
+
+struct nbl_chan_rx_desc {
+ __le16 flags;
+ __le32 buf_len;
+ __le16 buf_id;
+ __le64 buf_addr;
+} __packed;
+
+union nbl_chan_desc_ptr {
+ struct nbl_chan_tx_desc *tx_desc;
+ struct nbl_chan_rx_desc *rx_desc;
+};
+
+struct nbl_chan_ring {
+ union nbl_chan_desc_ptr desc;
+ struct nbl_chan_buf *buf;
+ u16 next_to_use;
+ u16 tail_ptr; /* hardware does modulo ring size internally */
+ u16 next_to_clean;
+ dma_addr_t dma;
+};
+
+#define NBL_CHAN_MSG_INDEX_MAX 63
+
+#define NBL_CHAN_MSGID_INDEX_MASK GENMASK(5, 0)
+#define NBL_CHAN_MSGID_LOC_MASK GENMASK(13, 6)
+
+static inline void nbl_chan_update_tail_ptr(struct nbl_hw_ops *hw_ops,
+ void *hw_priv, u32 tail_ptr, u8 qid)
+{
+ hw_ops->update_mailbox_queue_tail_ptr(hw_priv, tail_ptr, qid);
+}
+
+struct nbl_chan_waitqueue_head {
+ struct wait_queue_head wait_queue;
+ char *ack_data;
+ int acked;
+ s32 ack_err;
+ u16 ack_data_len;
+ u16 msg_type;
+ int status;
+ u8 msg_index;
+ u16 dstid;
+};
+
+struct nbl_chan_info {
+ wait_queue_head_t inflight_wait;
+ struct nbl_chan_ring txq;
+ struct nbl_chan_ring rxq;
+ struct nbl_chan_waitqueue_head *wait;
+ /*
+ *Protects access to the TX queue (txq) and related metadata.
+ *This mutex ensures exclusive access when updating the TX queue
+ */
+ struct mutex txq_lock;
+ /* Guards channel state bitmap, active and shutdn flags */
+ struct mutex state_lock;
+ /* Guards pending requests and pending work list operations */
+ struct mutex pending_lock;
+ struct work_struct *clean_task;
+ u16 wait_head_index;
+ u16 num_txq_entries;
+ u16 num_rxq_entries;
+ u16 txq_buf_size;
+ u16 rxq_buf_size;
+ DECLARE_BITMAP(state, NBL_CHAN_STATE_NBITS);
+ u8 chan_type;
+ atomic_t inflight_tx_cnt;
+ bool shutdn;
+ bool active;
+};
+
+struct nbl_chan_msg_node_data {
+ nbl_chan_resp func;
+ void *priv;
+};
+
+struct nbl_channel_mgt {
+ struct nbl_common_info *common;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_chan_info *chan_info[NBL_CHAN_TYPE_MAX];
+ struct nbl_hash_tbl_mgt *handle_hash_tbl;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
new file mode 100644
index 000000000000..38abf41d9bb0
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/jhash.h>
+#include "nbl_common.h"
+
+void nbl_common_destroy_wq(struct nbl_common_info *common)
+{
+ if (!common || !common->wq)
+ return;
+
+ destroy_workqueue(common->wq);
+ common->wq = NULL;
+}
+
+int nbl_common_create_wq(struct nbl_common_info *common)
+{
+ char wq_name[32];
+
+ snprintf(wq_name, sizeof(wq_name), "nbl_wq_%s", pci_name(common->pdev));
+ common->wq = alloc_workqueue(wq_name, WQ_UNBOUND, 0);
+ if (!common->wq) {
+ dev_err(common->dev, "Failed to alloc workqueue %s\n", wq_name);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
+{
+ u32 hash;
+
+ if (bucket_size == 0 || bucket_size == 1)
+ return 0;
+
+ hash = jhash(key, key_size, 0);
+
+ /* Use bitmask if bucket_size is a power of 2 */
+ if ((bucket_size & (bucket_size - 1)) == 0)
+ return hash & (bucket_size - 1);
+ return hash % bucket_size;
+}
+
+/**
+ * nbl_common_init_hash_table - initialize per-device hash table
+ * @key: hash table creation parameters
+ *
+ * Return: allocated tbl mgt pointer, NULL on failure.
+ */
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key)
+{
+ struct nbl_hash_tbl_mgt *tbl_mgt;
+ u32 bucket_size;
+ u32 i;
+
+ tbl_mgt = devm_kzalloc(key->dev, sizeof(*tbl_mgt), GFP_KERNEL);
+ if (!tbl_mgt)
+ return NULL;
+
+ bucket_size = key->bucket_size;
+ tbl_mgt->hash = devm_kcalloc(key->dev, bucket_size,
+ sizeof(struct hlist_head), GFP_KERNEL);
+ if (!tbl_mgt->hash)
+ return NULL;
+
+ tbl_mgt->bucket_locks = devm_kcalloc(key->dev, bucket_size,
+ sizeof(spinlock_t), GFP_KERNEL);
+ if (!tbl_mgt->bucket_locks)
+ return NULL;
+
+ for (i = 0; i < bucket_size; i++) {
+ INIT_HLIST_HEAD(&tbl_mgt->hash[i]);
+ spin_lock_init(&tbl_mgt->bucket_locks[i]);
+ }
+
+ memcpy(&tbl_mgt->tbl_key, key, sizeof(tbl_mgt->tbl_key));
+ tbl_mgt->node_num = 0;
+
+ return tbl_mgt;
+}
+
+/**
+ * nbl_common_alloc_hash_node - insert handler node into hash table
+ * @tbl_mgt: hash table manager
+ * @key: match key (msg_type)
+ * @data: handler callback info
+ * @out_data: optional pointer to return allocated data ptr
+ *
+ * Caller context: process context for dynamic registration, init path safe.
+ * Protected by per-bucket spin_lock_bh to avoid race with concurrent lookup.
+ *
+ * Return: 0 on success, -ENOMEM on allocation failure.
+ */
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data)
+{
+ struct nbl_hash_entry_node *hash_node;
+ u16 data_size;
+ u16 node_size;
+ u32 hash_val;
+ u16 key_size;
+
+ node_size = sizeof(*hash_node);
+ hash_node = kzalloc(node_size, GFP_KERNEL);
+ if (!hash_node)
+ return -ENOMEM;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_node->key = kzalloc(key_size, GFP_KERNEL);
+ if (!hash_node->key)
+ goto alloc_key_failed;
+
+ data_size = tbl_mgt->tbl_key.data_size;
+ hash_node->data = kzalloc(data_size, GFP_KERNEL);
+ if (!hash_node->data)
+ goto alloc_data_failed;
+
+ memcpy(hash_node->key, key, key_size);
+ memcpy(hash_node->data, data, data_size);
+
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[hash_val]);
+ hlist_add_head(&hash_node->node, tbl_mgt->hash + hash_val);
+ tbl_mgt->node_num++;
+ spin_unlock_bh(&tbl_mgt->bucket_locks[hash_val]);
+
+ if (out_data)
+ *out_data = hash_node->data;
+
+ return 0;
+
+alloc_data_failed:
+ kfree(hash_node->key);
+alloc_key_failed:
+ kfree(hash_node);
+ return -ENOMEM;
+}
+
+/**
+ * nbl_common_get_hash_node - lookup handler from hash table
+ * @tbl_mgt: hash table manager
+ * @key: lookup key
+ *
+ * All accessors use spin_lock_bh so that process-context holders
+ * disable softirq and cannot deadlock against a concurrent softirq
+ * caller (e.g. NAPI RX path). Safe in both process and softirq
+ * context.
+ *
+ * Return: attached handler data if found, NULL otherwise.
+ */
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_head *head;
+ void *data = NULL;
+ u32 hash_val;
+ u16 key_size;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+ head = tbl_mgt->hash + hash_val;
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[hash_val]);
+ hlist_for_each_entry(hash_node, head, node) {
+ if (!memcmp(hash_node->key, key, key_size)) {
+ data = hash_node->data;
+ break;
+ }
+ }
+ spin_unlock_bh(&tbl_mgt->bucket_locks[hash_val]);
+
+ return data;
+}
+
+/*
+ * Free all hash nodes in the table.
+ */
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_node *safe_node;
+ struct hlist_head *head;
+ u32 i;
+
+ if (!tbl_mgt)
+ return;
+
+ for (i = 0; i < tbl_mgt->tbl_key.bucket_size; i++) {
+ head = tbl_mgt->hash + i;
+
+ spin_lock_bh(&tbl_mgt->bucket_locks[i]);
+ hlist_for_each_entry_safe(hash_node, safe_node, head, node) {
+ hlist_del(&hash_node->node);
+ tbl_mgt->node_num--;
+ kfree(hash_node->key);
+ kfree(hash_node->data);
+ kfree(hash_node);
+ }
+ spin_unlock_bh(&tbl_mgt->bucket_locks[i]);
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
new file mode 100644
index 000000000000..159421e53902
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_COMMON_H_
+#define _NBL_COMMON_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_common.h"
+
+struct nbl_hash_tbl_mgt {
+ struct nbl_hash_tbl_key tbl_key;
+ struct hlist_head *hash;
+ /**
+ * bucket_locks: per-bucket spinlock array
+ * Each hash bucket corresponds to an independent spinlock.
+ * Protects concurrent hash list modification
+ */
+ spinlock_t *bucket_locks;
+ u16 node_num;
+};
+
+struct nbl_hash_entry_node {
+ struct hlist_node node;
+ void *key;
+ void *data;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 1cd6587a8fcb..f998a2b44e5c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -14,13 +14,20 @@ enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_interface {
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_channel_ops_tbl *channel_ops_tbl;
+};
+
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_channel_mgt *chan_mgt;
};
struct nbl_adapter {
struct pci_dev *pdev;
struct nbl_core core;
+ struct nbl_interface intf;
struct nbl_common_info common;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 0ba0b7f643c6..379dbb9dc003 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,156 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+ if (reg >= (u64)hw_mgt->mailbox_bar_size ||
+ reg + len > (u64)hw_mgt->mailbox_bar_size) {
+ dev_err_once(hw_mgt->common->dev,
+ "mbx write out of range: reg=0x%llx len=%u bar_size=%pa\n",
+ reg, len, &hw_mgt->mailbox_bar_size);
+ return;
+ }
+ for (i = 0; i < len / 4; i++)
+ nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
+}
+
+static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+
+ spin_lock(&hw_mgt->reg_lock);
+
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ spin_lock(&hw_mgt->reg_lock);
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx)
+{
+ /* local_qid 0 and 1 denote rx and tx queue respectively */
+ u32 local_qid = txrx;
+ u32 value = ((u32)tail_ptr << 16) | local_qid;
+
+ /* wmb for doorbell */
+ wmb();
+ nbl_mbx_wr32(hw_mgt, NBL_MAILBOX_NOTIFY_ADDR, value);
+}
+
+static void nbl_hw_config_mailbox_rxq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_config_mailbox_txq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_rxq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_txq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
+{
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_K_PF_MASK_REG, pf_mask,
+ sizeof(*pf_mask));
+}
+
+static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function)
+{
+ u32 data = 0;
+
+ data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
+ &data,
+ sizeof(data));
+}
+
+static struct nbl_hw_ops hw_ops = {
+ .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
+ .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
+ .config_mailbox_txq = nbl_hw_config_mailbox_txq,
+ .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
+ .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
+ .get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+
+};
+
/* Structure starts here, adding an op should not modify anything below */
static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
{
@@ -25,6 +175,27 @@ static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
return hw_mgt;
}
+static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
+ struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct device *dev;
+
+ dev = common->dev;
+ hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
+ if (!hw_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ !hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
+ !hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
+ !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ return ERR_PTR(-EINVAL);
+ hw_ops_tbl->ops = &hw_ops;
+ hw_ops_tbl->priv = hw_mgt;
+
+ return hw_ops_tbl;
+}
+
static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
const char *name)
{
@@ -45,6 +216,7 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
{
resource_size_t expect_sz = NBL_MEM_BAR_TOTAL_SIZE;
struct nbl_common_info *common = &adapter->common;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = NULL;
struct pci_dev *pdev = common->pdev;
struct nbl_hw_mgt *hw_mgt = NULL;
resource_size_t bar_len;
@@ -137,7 +309,14 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
}
hw_mgt->mailbox_bar_size = bar_len;
+ spin_lock_init(&hw_mgt->reg_lock);
+ hw_ops_tbl = nbl_hw_setup_ops(common, hw_mgt);
+ if (IS_ERR(hw_ops_tbl)) {
+ ret = PTR_ERR(hw_ops_tbl);
+ goto setup_mgt_fail;
+ }
+ adapter->intf.hw_ops_tbl = hw_ops_tbl;
adapter->core.hw_mgt = hw_mgt;
return 0;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 1f9e509dc631..71691e893790 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,5 +11,61 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+/* ---------- REG BASE ADDR ---------- */
+/* Interface modules base addr */
+#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
+#define NBL_INTF_HOST_PADPT_BASE 0x00f4c000
+#define NBL_INTF_HOST_MAILBOX_BASE 0x00fb0000
+#define NBL_INTF_HOST_PCIE_BASE 0X01504000
+/* DP modules base addr */
+#define NBL_DP_USTORE_BASE 0x00104000
+#define NBL_DP_UQM_BASE 0x00114000
+#define NBL_DP_UPED_BASE 0x0015c000
+#define NBL_DP_UVN_BASE 0x00244000
+#define NBL_DP_DSCH_BASE 0x00404000
+#define NBL_DP_SHAPING_BASE 0x00504000
+#define NBL_DP_DVN_BASE 0x00514000
+#define NBL_DP_DSTORE_BASE 0x00704000
+#define NBL_DP_DQM_BASE 0x00714000
+#define NBL_DP_DPED_BASE 0x0075c000
+#define NBL_DP_DDMUX_BASE 0x00984000
+/* -------- MAILBOX BAR2 ----- */
+#define NBL_MAILBOX_NOTIFY_ADDR 0x00000000
+#define NBL_MAILBOX_BAR_REG 0x00000000
+#define NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR 0x10
+#define NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR 0x20
+#define NBL_MAILBOX_QINFO_CFG_DBG_TABLE_ADDR 0x30
+
+/* -------- MAILBOX -------- */
+
+/* mailbox BAR qinfo_cfg_table */
+#define MAILBOX_QINFO_CFG_TABLE_DWLEN 4
+/* data[2] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK GENMASK(3, 0)
+/* data[3] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK BIT(0)
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK BIT(1)
+#define NBL_MAILBOX_QINFO_CFG_DIF_ERR_MASK BIT(2)
+#define NBL_MAILBOX_QINFO_CFG_PTR_ERR_MASK BIT(3)
+struct nbl_mailbox_qinfo_cfg_table {
+ u32 data[MAILBOX_QINFO_CFG_TABLE_DWLEN];
+};
+
+/* -------- MAILBOX BAR0 ----- */
+/* mailbox qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id) \
+ (NBL_INTF_HOST_MAILBOX_BASE + 0x00001000 + (func_id) * sizeof(u32))
+
+/* MAILBOX qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_MAILBOX_QINFO_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_MAILBOX_QINFO_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK GENMASK(28, 16)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK BIT(29)
+
+/* -------- HOST_PCIE -------- */
+#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
+#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+
#define NBL_BAR2_MAX_LEN 0x300
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
index 5e9823e01d39..fdf7b3d96087 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
+#include "../nbl_include/nbl_def_channel.h"
#include "../nbl_include/nbl_def_hw.h"
#include "../nbl_include/nbl_def_common.h"
#include "../nbl_core.h"
@@ -16,6 +17,7 @@
#define NBL_MAILBOX_BAR 2
#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+#define NBL_HW_DUMMY_REG 0x1300904
/*
* PCI MEMORY BAR total size: 64MiB.
*/
@@ -27,6 +29,37 @@ struct nbl_hw_mgt {
u8 __iomem *mailbox_bar_hw_addr;
resource_size_t hw_size;
resource_size_t mailbox_bar_size;
+ spinlock_t reg_lock; /* Protect reg access */
};
+static inline u32 rd32(u8 __iomem *addr, u64 reg)
+{
+ return readl(addr + reg);
+}
+
+static inline void wr32(u8 __iomem *addr, u64 reg, u32 value)
+{
+ writel(value, addr + reg);
+}
+
+static inline void nbl_hw_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ wr32(hw_mgt->hw_addr, reg, value);
+}
+
+static inline u32 nbl_hw_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return rd32(hw_mgt->hw_addr, reg);
+}
+
+static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
+static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return readl(hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
new file mode 100644
index 000000000000..4fc987c0e56d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_CHANNEL_H_
+#define _NBL_DEF_CHANNEL_H_
+
+#include <linux/types.h>
+
+struct nbl_channel_mgt;
+struct nbl_adapter;
+
+typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+
+/*
+ * Mailbox wire opcodes, stable wire ABI shared between driver and firmware.
+ * Each opcode has a fixed assigned number to preserve compatibility.
+ * ABI compatibility rules:
+ * 1. New opcodes shall only be appended before NBL_CHAN_MSG_MAILBOX_MAX;
+ * 2. Reordering, inserting or deleting existing enumerators breaks driver-
+ * firmware interoperability and must be avoided;
+ * 3. Modifications to existing opcodes require synchronized firmware ABI
+ * updates.
+ *
+ * Only opcodes currently used by in-tree driver logic are defined here.
+ * Unimplemented feature opcodes (KTLS, IPsec, vDPA, mirror etc.) will be
+ * added incrementally together with their corresponding driver
+ * implementation patches.
+ */
+enum nbl_chan_msg_type {
+ NBL_CHAN_MSG_ACK = 0,
+ /* mailbox msg end */
+ NBL_CHAN_MSG_MAILBOX_MAX,
+};
+
+enum nbl_chan_state {
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_STATE_NBITS
+};
+
+struct nbl_chan_send_info {
+ void *arg;
+ size_t arg_len;
+ void *resp;
+ size_t resp_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 ack;
+ u16 ack_len;
+};
+
+struct nbl_chan_ack_info {
+ void *data;
+ int err;
+ u32 data_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 msgid;
+};
+
+enum nbl_channel_type {
+ NBL_CHAN_TYPE_MAILBOX,
+ NBL_CHAN_TYPE_MAX
+};
+
+static inline void
+nbl_chan_fill_send_info(struct nbl_chan_send_info *info,
+ u16 dst_id, u16 msg_type,
+ void *argument, u32 arg_length,
+ void *response, u32 resp_length,
+ bool need_ack)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->arg = argument;
+ info->arg_len = arg_length;
+ info->resp = response;
+ info->resp_len = resp_length;
+ info->ack = need_ack;
+}
+
+static inline void
+nbl_chan_fill_ack_info(struct nbl_chan_ack_info *info,
+ u16 dst_id, u16 msg_type, u16 msg_id,
+ int err_code, void *ack_data, u32 data_length)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->msgid = msg_id;
+ info->err = err_code;
+ info->data = ack_data;
+ info->data_len = data_length;
+}
+
+struct nbl_channel_ops {
+ int (*send_msg)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send);
+ int (*send_ack)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack);
+ int (*register_msg)(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback_priv);
+ void (*unregister_all_msg)(struct nbl_channel_mgt *chan_mgt);
+ void (*cfg_chan_qinfo_map_table)(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid);
+ bool (*check_queue_exist)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ int (*setup_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ int (*teardown_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ void (*clean_queue_subtask)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ void (*register_chan_task)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task);
+ void (*set_queue_state)(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set);
+};
+
+struct nbl_channel_ops_tbl {
+ struct nbl_channel_ops *ops;
+ struct nbl_channel_mgt *priv;
+};
+
+int nbl_chan_init_common(struct nbl_adapter *adapter);
+void nbl_chan_remove_common(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index da30244fe75d..4916c384611b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -12,6 +12,7 @@
#include "nbl_include.h"
struct nbl_common_info {
+ struct workqueue_struct *wq;
struct pci_dev *pdev;
struct device *dev;
u32 msg_enable;
@@ -29,4 +30,21 @@ struct nbl_common_info {
u8 has_net;
};
+struct nbl_hash_tbl_key {
+ struct device *dev;
+ u16 key_size;
+ u16 data_size; /* no include key or node member */
+ u16 bucket_size;
+ u16 resv;
+};
+
+void nbl_common_destroy_wq(struct nbl_common_info *common);
+int nbl_common_create_wq(struct nbl_common_info *common);
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data);
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index ecbf440e4366..be63e53d41c6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -10,6 +10,42 @@
struct nbl_hw_mgt;
struct nbl_adapter;
+struct nbl_hw_ops {
+ void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx);
+ void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*config_mailbox_txq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*stop_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt);
+ void (*stop_mailbox_txq)(struct nbl_hw_mgt *hw_mgt);
+ /**
+ * get_host_pf_mask - Fetch host PF mask from firmware k_pf_mask reg
+ * @hw_mgt: hardware management context
+ * @pf_mask: output pointer for PF mask value
+ *
+ * k_pf_mask register rule:
+ * bit N == 0 -> PF#N enabled; bit N == 1 -> PF#N masked out.
+ * bit0 is PF0's mask bit (not reserved); PF0 can be masked but
+ * the driver requires at least PF0 enabled.
+ * Only 1/2/4 PFs are supported:
+ * 1 PF (PF0): mask = 0xfe
+ * 2 PFs (PF0,PF1): mask = 0xfc
+ * 4 PFs (PF0~PF3): mask = 0xf0
+ * All-zero mask (0x00) means all 8 PFs enabled, which is
+ * unsupported by the driver and rejected with -EINVAL.
+ *
+ */
+ void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+
+ void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function);
+};
+
+struct nbl_hw_ops_tbl {
+ struct nbl_hw_ops *ops;
+ struct nbl_hw_mgt *priv;
+};
int nbl_hw_init_leonis(struct nbl_adapter *adapter);
void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 14e7b19f9a4c..f2d802397d98 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,6 +10,9 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+#define NBL_MAX_PF 8
+#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index f2552bc73293..b7c80ea54c8d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -38,13 +39,19 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
if (ret)
goto hw_init_fail;
+ ret = nbl_chan_init_common(adapter);
+ if (ret)
+ goto chan_init_fail;
return adapter;
+chan_init_fail:
+ nbl_hw_remove_leonis(adapter);
hw_init_fail:
return ERR_PTR(ret);
}
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (2 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
` (5 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
The Resource layer owns chip-module resource entries and identity
mappings, providing lookup and conversion helpers between vsi_id,
func_id, eth_id and pf_id for upper dispatch and device layers.
This patch adds the common (chip-independent) resource layer:
- nbl_res_init_pf_num(): reads firmware k_pf_mask and validates the
enabled PF topology. Only 1/2/4 contiguous PFs starting from PF0
are accepted; non-contiguous layouts, unsupported counts (0, 3,
5-8) and the all-zero mask (all 8 PFs enabled) are rejected with
-EINVAL, causing probe to fail early.
- nbl_res_ctrl_dev_sriov_info_init(): derives each PF's BDF from the
hardware bus number (read via get_real_bus) and the function
offset, stored in sriov_info[] for MSI-X map programming.
- nbl_res_ctrl_dev_setup_eth_info(): validates that firmware port
count (get_board_info()->eth_num) matches max_pf and the eth
bitmap reported by get_fw_eth_map(), then builds the per-PF
eth_id / logic_eth_id lookup tables.
- nbl_res_ctrl_dev_vsi_info_init(): assigns per-PF VSI base IDs
(gap 1024/512/256 for 1/2/4 ports respectively).
- Conversion helpers nbl_res_func_id_to_vsi_id(),
nbl_res_vsi_id_to_pf_id() and nbl_res_get_eth_id() with full
range and has_ctrl guards.
All initialization above runs only on the control PF
(common->has_ctrl == true). The dispatch layer serializes all
resource ops through ops_mutex_lock and routes non-control PF
requests to the control PF via mailbox RPC, so these helpers are
never invoked directly on a non-control device.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 2 +
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 22 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 70 +++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 +
.../nbl_hw_leonis/nbl_resource_leonis.c | 335 ++++++++++++++++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 10 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 120 +++++++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 72 ++++
.../nbl/nbl_include/nbl_def_channel.h | 10 +
.../nbl/nbl_include/nbl_def_common.h | 21 +-
.../nbl/nbl_include/nbl_def_hw.h | 16 +
.../nbl/nbl_include/nbl_def_resource.h | 29 ++
.../nbl/nbl_include/nbl_include.h | 6 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
15 files changed, 736 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 04e1aa1fb4bd..3dab9519a277 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -6,4 +6,6 @@ obj-$(CONFIG_NBL) := nbl.o
nbl-objs += nbl_common/nbl_common.o \
nbl_channel/nbl_channel.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
+ nbl_hw/nbl_resource.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
index 38abf41d9bb0..597c11ed7fd8 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -30,6 +30,28 @@ int nbl_common_create_wq(struct nbl_common_info *common)
return 0;
}
+/**
+ * nbl_common_func_id_to_rel_pf_id - convert absolute PF id to relative PF id
+ * @common: common device info
+ * @pf_id: absolute PF identifier
+ * @rel_pf_id: output relative pf id
+ *
+ * Leonis uses fixed mgt_pf = 0. Support future non-zero management PF.
+ *
+ * Return: 0 on success, -EINVAL on invalid arguments.
+ */
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id)
+{
+ if (!rel_pf_id)
+ return -EINVAL;
+
+ if (pf_id < common->mgt_pf)
+ return -EINVAL;
+ *rel_pf_id = pf_id - common->mgt_pf;
+ return 0;
+}
+
static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
{
u32 hash;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index f998a2b44e5c..dd24ebec0171 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -16,11 +16,13 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_resource_ops_tbl *resource_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_resource_mgt *res_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 379dbb9dc003..b4aba4faa555 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,24 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_read_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+ if (reg >= (u64)hw_mgt->mailbox_bar_size ||
+ reg + len > (u64)hw_mgt->mailbox_bar_size) {
+ dev_err_once(hw_mgt->common->dev,
+ "mbx read out of range: reg=0x%llx len=%u bar_size=%pa\n",
+ reg, len, &hw_mgt->mailbox_bar_size);
+ return;
+ }
+ for (i = 0; i < len / 4; i++)
+ data[i] = nbl_mbx_rd32(hw_mgt, reg + i * sizeof(u32));
+}
+
static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
const u32 *data, u32 len)
{
@@ -58,6 +76,21 @@ static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
spin_unlock(&hw_mgt->reg_lock);
}
+/*
+ * Registers reset to zero after cold boot / FLR / bus reset. Firmware
+ * programs valid values before driver probe, so zero is only seen on
+ * hardware fault or register read failure. Initialize data=0 to guard
+ * against nbl_hw_read_mbx_regs() early-return on bounds-check failure.
+ */
+static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
+{
+ u32 data = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW6_OFFSET, &data,
+ sizeof(data));
+ *eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -136,6 +169,15 @@ static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
sizeof(*pf_mask));
}
+static void nbl_hw_get_real_bus(struct nbl_hw_mgt *hw_mgt, u8 *bus)
+{
+ u32 data = 0;
+
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_TL_CFG_BUSDEV, &data,
+ sizeof(data));
+ *bus = FIELD_GET(NBL_PCIE_BUS_MASK, data);
+}
+
static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function)
{
@@ -149,6 +191,26 @@ static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
sizeof(data));
}
+/*
+ * Registers reset to zero after cold boot / FLR / bus reset. Firmware
+ * programs valid values before driver probe, so zero is only seen on
+ * hardware fault or register read failure. Initialize data=0 to guard
+ * against nbl_hw_read_mbx_regs() early-return on bounds-check failure.
+ */
+static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board_info)
+{
+ u32 data = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW3_OFFSET, &data,
+ sizeof(data));
+ board_info->eth_num = FIELD_GET(NBL_FW_BOARD_DW3_PORT_NUM_MASK, data);
+ board_info->eth_speed =
+ FIELD_GET(NBL_FW_BOARD_DW3_PORT_SPEED_MASK, data);
+ board_info->p4_version =
+ FIELD_GET(NBL_FW_BOARD_DW3_P4_VERSION_MASK, data);
+}
+
static struct nbl_hw_ops hw_ops = {
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
@@ -156,8 +218,12 @@ static struct nbl_hw_ops hw_ops = {
.stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
.stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
.get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .get_real_bus = nbl_hw_get_real_bus,
+
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .get_fw_eth_map = nbl_hw_get_fw_eth_map,
+ .get_board_info = nbl_hw_get_board_info,
};
/* Structure starts here, adding an op should not modify anything below */
@@ -188,7 +254,9 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
if (!hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
- !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ !hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
+ !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
hw_ops_tbl->priv = hw_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 71691e893790..86d42a0a5687 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -67,5 +67,19 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+#define NBL_FW_BOARD_CONFIG 0x200
+#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
+#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
+
+#define NBL_FW_BOARD_DW3_PORT_TYPE_MASK BIT(0)
+#define NBL_FW_BOARD_DW3_PORT_NUM_MASK GENMASK(7, 1)
+#define NBL_FW_BOARD_DW3_PORT_SPEED_MASK GENMASK(9, 8)
+#define NBL_FW_BOARD_DW3_GPIO_TYPE_MASK GENMASK(12, 10)
+#define NBL_FW_BOARD_DW3_P4_VERSION_MASK GENMASK(13, 13)
+
+#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
+#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+
#define NBL_BAR2_MAX_LEN 0x300
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
new file mode 100644
index 000000000000..46180522295a
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -0,0 +1,335 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include "nbl_resource_leonis.h"
+
+static struct nbl_resource_ops res_ops = {
+ .get_vsi_id = nbl_res_func_id_to_vsi_id,
+ .get_eth_id = nbl_res_get_eth_id,
+};
+
+static struct nbl_resource_mgt *
+nbl_res_setup_res_mgt(struct nbl_common_info *common)
+{
+ struct nbl_resource_info *resource_info;
+ struct nbl_resource_mgt *res_mgt;
+ struct device *dev = common->dev;
+
+ res_mgt = devm_kzalloc(dev, sizeof(*res_mgt), GFP_KERNEL);
+ if (!res_mgt)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->common = common;
+
+ resource_info =
+ devm_kzalloc(dev, sizeof(*resource_info), GFP_KERNEL);
+ if (!resource_info)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->resource_info = resource_info;
+
+ return res_mgt;
+}
+
+static struct nbl_resource_ops_tbl *
+nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+
+ res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
+ if (!res_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!res_ops.get_vsi_id || !res_ops.get_eth_id)
+ return ERR_PTR(-EINVAL);
+ res_ops_tbl->ops = &res_ops;
+ res_ops_tbl->priv = res_mgt;
+
+ return res_ops_tbl;
+}
+
+static int nbl_res_ctrl_dev_setup_eth_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_eth_info *eth_info;
+ u32 eth_bitmap = 0, eth_id;
+ u32 eth_num = 0;
+ u32 fw_port_num;
+ int i;
+
+ eth_info = devm_kzalloc(dev, sizeof(*eth_info), GFP_KERNEL);
+ if (!eth_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->eth_info = eth_info;
+
+ fw_port_num = res_mgt->resource_info->board_info.eth_num;
+
+ hw_ops->get_fw_eth_map(res_mgt->hw_ops_tbl->priv, ð_bitmap);
+ if (eth_bitmap & ~((1 << NBL_MAX_ETHERNET) - 1)) {
+ dev_err(dev, "FW reported invalid eth_bitmap 0x%x\n",
+ eth_bitmap);
+ return -EINVAL;
+ }
+ if (fw_port_num != hweight32(eth_bitmap)) {
+ dev_err(dev, "FW inconsistency: port_num=%u, bitmap=0x%x\n",
+ fw_port_num, eth_bitmap);
+ return -EINVAL;
+ }
+ /*
+ * Firmware is ready before probe. Valid port counts are 1/2/4;
+ * 0 (invalid config), 3 (unsupported topology), and >4 (exceeds
+ * hardware max) are all rejected with -EINVAL.
+ */
+ if (fw_port_num == 0 || fw_port_num == 3 ||
+ fw_port_num > NBL_MAX_ETHERNET) {
+ dev_err(dev, "FW reports %u Ethernet ports, unsupported (valid: 1/2/4)\n",
+ fw_port_num);
+ return -EINVAL;
+ }
+ eth_info->eth_num = fw_port_num;
+ /* Intentional design constraint: each PF maps to exactly one
+ * Ethernet port. This couples PF identity to port identity
+ * and is required by nbl_res_get_eth_id() which indexes
+ * eth_info->eth_id[] by relative PF id.
+ */
+ if (res_mgt->common->max_pf != eth_info->eth_num) {
+ dev_err(dev, "Invalid PF-to-port topology: max_pf=%u, eth_num=%u\n",
+ res_mgt->common->max_pf, eth_info->eth_num);
+ return -EINVAL;
+ }
+
+ /*
+ * Any subset of valid bitmap bits is accepted (e.g. 0/1, 0/2,
+ * 1/3, etc.). Firmware only needs to report the correct count
+ * of active ports; no hard-coded fixed bit positions required.
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if ((1 << i) & eth_bitmap) {
+ set_bit(i, eth_info->eth_bitmap);
+ eth_info->eth_id[eth_num] = i;
+ eth_info->logic_eth_id[i] = eth_num;
+ eth_num++;
+ }
+ }
+
+ for (i = 0; i < res_mgt->common->max_pf; i++) {
+ eth_id = eth_info->eth_id[i];
+ eth_info->pf_bitmap[eth_id] |= BIT(i);
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_sriov_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ struct device *dev = common->dev;
+ u8 hw_bus = 0;
+ u16 function;
+ u16 func_id;
+
+ hw_ops->get_real_bus(p, &hw_bus);
+ if (common->function + common->max_pf > NBL_MAX_PF) {
+ dev_err(dev, "PF count exceeds available function space\n");
+ return -EINVAL;
+ }
+ sriov_info = devm_kcalloc(dev, common->max_pf,
+ sizeof(*sriov_info), GFP_KERNEL);
+ if (!sriov_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->sriov_info = sriov_info;
+ /*
+ * common->hw_bus supplies bus number for channel mailbox QINFO mapping.
+ * Execution order guarantee: this assignment runs before
+ * cfg_chan_qinfo_map_table(), only executed
+ * on control PF path.
+ */
+ common->hw_bus = hw_bus;
+
+ for (func_id = 0; func_id < common->max_pf; func_id++) {
+ sriov_info = res_mgt->resource_info->sriov_info + func_id;
+ function = common->function + func_id;
+ sriov_info->bdf = PCI_DEVID(common->hw_bus,
+ PCI_DEVFN(common->devid, function));
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_vsi_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ struct nbl_vsi_info *vsi_info;
+ int i;
+
+ vsi_info = devm_kzalloc(dev, sizeof(*vsi_info), GFP_KERNEL);
+ if (!vsi_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->vsi_info = vsi_info;
+ /*
+ * case 1 one port(1pf)
+ * pf0 (NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0
+ * case 2 two port(2pf)
+ * pf0,pf1(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,512
+ * case 3 four port(4pf)
+ * pf0,pf1,pf2,pf3(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,256,512,768
+ */
+
+ vsi_info->num = eth_info->eth_num;
+ /*
+ * eth_num can be 1/2/4:
+ * - 2/4 ports use dedicated gap constants;
+ * - 1 port falls back to NBL_DEFAULT_VSI_ID_GAP (1024).
+ * All three values produce valid base_id offsets.
+ */
+ for (i = 0; i < vsi_info->num; i++) {
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].base_id =
+ i * nbl_vsi_id_gap(vsi_info->num);
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].num = 1;
+ }
+
+ return 0;
+}
+
+static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ u32 exp_contiguous_mask = 0;
+ u32 pf_mask = 0;
+ u32 pf_num = 0;
+ int i;
+
+ hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
+
+ /*
+ * k_pf_mask register rule:
+ * bit N == 0 -> PF#N enabled; bit N == 1 -> PF#N masked out.
+ * Hardware constraint: bit0 is PF0's mask bit; driver requires
+ * PF0 enabled as management PF, so bit0 must be clear.
+ * All-zero pf_mask means all PF0~PF7 are enabled, which is unsupported
+ * by the driver
+ *
+ * Product firmware constraint: only 3 valid configurations supported:
+ * 1 PF (PF0 only): pf_num = 1, mask = 0xfe
+ * 2 PFs (PF0,PF1): pf_num = 2, mask = 0xfc
+ * 4 PFs (PF0~PF3): pf_num = 4, mask = 0xf0
+ * No other PF count or sparse/non-contiguous PF layout is allowed.
+ */
+ for (i = 0; i < NBL_MAX_PF; i++) {
+ if (!(pf_mask & (1 << i)))
+ pf_num++;
+ }
+
+ /*
+ * Sanity check: enabled PFs must be contiguous starting from PF0.
+ * Current resource framework uses relative PF id, sparse PF layout
+ * will cause mismatch between resource layer and hardware func_id.
+ */
+ for (i = 0; i < pf_num; i++)
+ exp_contiguous_mask |= BIT(i);
+ if ((pf_mask & exp_contiguous_mask) != 0) {
+ dev_err(res_mgt->common->dev,
+ "pf_mask 0x%08x: non-contiguous enabled PF, unsupported\n",
+ pf_mask);
+ return -EINVAL;
+ }
+
+ /* Only allow product-specified PF count: 1 / 2 / 4 */
+ if (pf_num != 1 && pf_num != 2 && pf_num != 4) {
+ dev_err(res_mgt->common->dev,
+ "Invalid pf_num=%u (mask=0x%08x), only 1/2/4 PFs supported\n",
+ pf_num, pf_mask);
+ return -EINVAL;
+ }
+
+ res_mgt->common->max_pf = pf_num;
+
+ return 0;
+}
+
+static void nbl_res_init_board_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+
+ hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
+ &res_mgt->resource_info->board_info);
+}
+
+static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ int ret = 0;
+
+ if (common->has_ctrl) {
+ nbl_res_init_board_info(res_mgt);
+
+ ret = nbl_res_init_pf_num(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_sriov_info_init(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_setup_eth_info(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int nbl_res_init_leonis(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl = adap->intf.channel_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adap->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adap->common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_resource_mgt *res_mgt;
+ int ret;
+
+ res_mgt = nbl_res_setup_res_mgt(common);
+ if (IS_ERR(res_mgt)) {
+ ret = PTR_ERR(res_mgt);
+ return ret;
+ }
+ res_mgt->chan_ops_tbl = chan_ops_tbl;
+ res_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ ret = nbl_res_start(res_mgt);
+ if (ret)
+ return ret;
+
+ res_ops_tbl = nbl_res_setup_ops(dev, res_mgt);
+ if (IS_ERR(res_ops_tbl)) {
+ ret = PTR_ERR(res_ops_tbl);
+ return ret;
+ }
+ adap->intf.resource_ops_tbl = res_ops_tbl;
+ adap->core.res_mgt = res_mgt;
+
+ return 0;
+}
+
+void nbl_res_remove_leonis(struct nbl_adapter *adap)
+{
+ /*
+ * No resource release here because all memory uses devm managed
+ * allocation
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
new file mode 100644
index 000000000000..b9355262c00d
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_LEONIS_H_
+#define _NBL_RESOURCE_LEONIS_H_
+
+#include "../nbl_resource.h"
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
new file mode 100644
index 000000000000..6fa0e0d550f4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -0,0 +1,120 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/pci.h>
+#include "nbl_resource.h"
+
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ enum nbl_vsi_serv_type dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !vsi_id) {
+ dev_dbg(dev, "No control plane or null vsi output ptr\n");
+ return -EINVAL;
+ }
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= vsi_info->num) {
+ dev_err(dev, "PF %d (diff=%u) exceeds vsi_info->num (%u)\n",
+ pfid, rel_pf_id, vsi_info->num);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_pf_dev_vsi_type_to_hw_vsi_type(res_mgt, type, &dst_type);
+ if (ret) {
+ dev_err(dev, "Invalid vsi type %u func_id %u\n", type, func_id);
+ return ret;
+ }
+ *vsi_id = vsi_info->serv_info[rel_pf_id][dst_type].base_id;
+ return 0;
+}
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int j = NBL_VSI_SERV_PF_DATA_TYPE;
+ int pf_id, i;
+
+ if (!common->has_ctrl) {
+ dev_dbg(dev, "No control plane available\n");
+ return -EINVAL;
+ }
+ for (i = 0; i < vsi_info->num; i++) {
+ if (vsi_id >= vsi_info->serv_info[i][j].base_id &&
+ (vsi_id < vsi_info->serv_info[i][j].base_id +
+ vsi_info->serv_info[i][j].num)) {
+ pf_id = i + common->mgt_pf;
+ if (pf_id >= NBL_MAX_PF) {
+ dev_err(dev, "PF ID overflow\n");
+ return -ERANGE;
+ }
+ return pf_id;
+ }
+ }
+
+ dev_dbg(dev, "VSI ID %u not found\n", vsi_id);
+ return -ENOENT;
+}
+
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ int rel_pf_id;
+ int abs_pf_id;
+
+ if (!common->has_ctrl || !eth_num || !eth_id || !logic_eth_id)
+ return -EINVAL;
+ abs_pf_id = nbl_res_vsi_id_to_pf_id(res_mgt, vsi_id);
+ if (abs_pf_id < 0) {
+ dev_err(dev, "Failed to get PF ID from VSI ID %u\n", vsi_id);
+ return -EINVAL;
+ }
+ if (abs_pf_id != pfid) {
+ dev_err(dev, "func_id %u does not match pf derived from vsi_id %u\n",
+ pfid, vsi_id);
+ return -EINVAL;
+ }
+ rel_pf_id = abs_pf_id - common->mgt_pf;
+
+ if (rel_pf_id >= eth_info->eth_num) {
+ dev_err(dev, "rel_pf_id %d out of range [0, %u)\n",
+ rel_pf_id, eth_info->eth_num);
+ return -ERANGE;
+ }
+
+ *eth_num = eth_info->eth_num;
+ *eth_id = eth_info->eth_id[rel_pf_id];
+ *logic_eth_id = rel_pf_id;
+ return 0;
+}
+
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type)
+{
+ switch (src_type) {
+ case NBL_VSI_DATA:
+ *dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ return 0;
+ default:
+ dev_err_once(res_mgt->common->dev,
+ "Unsupported vsi src_type %u\n", src_type);
+ return -EINVAL;
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
new file mode 100644
index 000000000000..a3bc7b3aecde
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_H_
+#define _NBL_RESOURCE_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_resource_mgt;
+
+/* --------- INFO ---------- */
+struct nbl_sriov_info {
+ unsigned int bdf;
+};
+
+struct nbl_eth_info {
+ DECLARE_BITMAP(eth_bitmap, NBL_MAX_ETHERNET);
+ u8 pf_bitmap[NBL_MAX_ETHERNET];
+ u8 eth_num;
+ u8 resv[3];
+ u8 eth_id[NBL_MAX_ETHERNET];
+ u8 logic_eth_id[NBL_MAX_ETHERNET];
+};
+
+enum nbl_vsi_serv_type {
+ NBL_VSI_SERV_PF_DATA_TYPE,
+ NBL_VSI_SERV_MAX_TYPE,
+};
+
+struct nbl_vsi_serv_info {
+ u16 base_id;
+ u16 num;
+};
+
+struct nbl_vsi_info {
+ u16 num;
+ struct nbl_vsi_serv_info serv_info[NBL_MAX_ETHERNET]
+ [NBL_VSI_SERV_MAX_TYPE];
+};
+
+struct nbl_resource_info {
+ struct nbl_sriov_info *sriov_info;
+ struct nbl_eth_info *eth_info;
+ struct nbl_vsi_info *vsi_info;
+ struct nbl_board_port_info board_info;
+};
+
+struct nbl_resource_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_info *resource_info;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+};
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 4fc987c0e56d..61dd97c779ef 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -39,6 +39,16 @@ enum nbl_chan_state {
NBL_CHAN_STATE_NBITS
};
+struct nbl_board_port_info {
+ u8 eth_num;
+ u8 eth_speed;
+ u8 p4_version;
+ u8 rsv[5];
+};
+
+static_assert(sizeof(struct nbl_board_port_info) == 8,
+ "nbl_board_port_info size must be 8 bytes");
+
struct nbl_chan_send_info {
void *arg;
size_t arg_len;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index 4916c384611b..3c2cb34192df 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -11,11 +11,26 @@
#include <linux/device.h>
#include "nbl_include.h"
+#define NBL_TWO_ETHERNET_PORT 2
+#define NBL_FOUR_ETHERNET_PORT 4
+#define NBL_DEFAULT_VSI_ID_GAP 1024
+#define NBL_TWO_ETHERNET_VSI_ID_GAP 512
+#define NBL_FOUR_ETHERNET_VSI_ID_GAP 256
+
+static inline u32 nbl_vsi_id_gap(u32 m)
+{
+ if (m == NBL_FOUR_ETHERNET_PORT)
+ return NBL_FOUR_ETHERNET_VSI_ID_GAP;
+ else if (m == NBL_TWO_ETHERNET_PORT)
+ return NBL_TWO_ETHERNET_VSI_ID_GAP;
+
+ return NBL_DEFAULT_VSI_ID_GAP;
+}
+
struct nbl_common_info {
struct workqueue_struct *wq;
struct pci_dev *pdev;
struct device *dev;
- u32 msg_enable;
u16 vsi_id;
u8 eth_id;
u8 logic_eth_id;
@@ -25,9 +40,11 @@ struct nbl_common_info {
u8 devid;
u8 bus;
u8 hw_bus;
+ u16 mgt_pf;
u8 has_ctrl;
u8 has_net;
+ u8 max_pf;
};
struct nbl_hash_tbl_key {
@@ -40,6 +57,8 @@ struct nbl_hash_tbl_key {
void nbl_common_destroy_wq(struct nbl_common_info *common);
int nbl_common_create_wq(struct nbl_common_info *common);
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id);
struct nbl_hash_tbl_mgt *
nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index be63e53d41c6..e05248c66afb 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
+struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
@@ -35,11 +36,26 @@ struct nbl_hw_ops {
* All-zero mask (0x00) means all 8 PFs enabled, which is
* unsupported by the driver and rejected with -EINVAL.
*
+ * Firmware contract: number of unmasked PFs MUST equal
+ * get_board_info()->eth_num.
*/
void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+ void (*get_real_bus)(struct nbl_hw_mgt *hw_mgt, u8 *bus);
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
+ /**
+ * get_board_info - Fetch board info from firmware
+ * @hw_mgt: hardware management context
+ * @board_info: output pointer for board info structure
+ *
+ * Firmware contract: board_info.eth_num MUST equal the number of
+ * unmasked PFs from get_host_pf_mask(). See get_host_pf_mask for
+ * details.
+ */
+ void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board);
};
struct nbl_hw_ops_tbl {
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
new file mode 100644
index 000000000000..7136b282fb80
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_RESOURCE_H_
+#define _NBL_DEF_RESOURCE_H_
+
+#include <linux/types.h>
+
+struct nbl_resource_mgt;
+struct nbl_adapter;
+
+struct nbl_resource_ops {
+ int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id);
+};
+
+struct nbl_resource_ops_tbl {
+ struct nbl_resource_ops *ops;
+ struct nbl_resource_mgt *priv;
+};
+
+int nbl_res_init_leonis(struct nbl_adapter *adapter);
+void nbl_res_remove_leonis(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index f2d802397d98..59e44feab44f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,12 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_ETHERNET 4
+
+enum {
+ NBL_VSI_DATA = 0,
+};
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index b7c80ea54c8d..1aafed2d46d7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -10,6 +10,7 @@
#include "nbl_include/nbl_include.h"
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -27,6 +28,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
adapter->pdev = pdev;
common = &adapter->common;
+ common->mgt_pf = 0;
common->pdev = pdev;
common->dev = &pdev->dev;
common->has_ctrl = param->caps.has_ctrl;
@@ -42,7 +44,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_chan_init_common(adapter);
if (ret)
goto chan_init_fail;
+
+ ret = nbl_res_init_leonis(adapter);
+ if (ret)
+ goto res_init_fail;
return adapter;
+res_init_fail:
+ nbl_chan_remove_common(adapter);
chan_init_fail:
nbl_hw_remove_leonis(adapter);
hw_init_fail:
@@ -51,6 +59,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 05/10] net/nebula-matrix: add intr resource implementation
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (3 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
` (4 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Introduce nbl_interrupt module to manage the driver-wide global
MSI-X vector index space (intr_net_bmap / intr_other_bmap) and
program the chip-internal MSI-X mapping registers.
Core interfaces:
1. cfg_msix_map
Allocates global MSI-X indices from independent net/other interrupt
bitmaps. All coherent DMA buffers for the new configuration are
allocated upfront; old hardware state is torn down only after all
allocations succeed to avoid interrupt loss. Writes the MSI-X
table DMA address and control-PF BDF into
NBL_PCOMPLETER_FUNCTION_MSIX_MAP.
Physical PCI MSI-X vector allocation (pci_alloc_irq_vectors etc.)
is handled separately by the device layer; the corresponding
nbl_dev_init_interrupt_scheme() entry point is added in a later
patch in this series.
2. destroy_msix_map
Recycles global MSI-X vectors, clears hardware MSI-X mappings,
releases coherent DMA memory and the interrupt descriptor array.
Step 0 disables mailbox IRQ routing before teardown; Step 1 masks
each vector. A two-stage hardware teardown retains the live DMA
address while clearing VALID, sleeps 1 ms to allow in-flight table
fetch DMA to quiesce (best-effort; no idle status register exists),
then zeroes the entry before freeing memory.
3. set_mailbox_irq
Toggles mailbox MSI-X routing for a specific PF by updating
NBL_MAILBOX_QINFO_MAP_REG_ARR. The disable path does not require
a configured MSI-X map, so destroy_msix_map can always clear the
route before releasing vectors.
4. cfg_msix_info
Programs PADPT_HOST_MSIX_INFO and PCOMPLETER_HOST_MSIX_FID_TABLE
with strict enable/teardown ordering to avoid inconsistent hardware
state.
The interrupt manager owns a self-contained mutex (intr_mgt->lock)
that protects the global vector bitmaps and per-function state.
All public entry points (cfg/destroy/set_irq) take this lock
internally; callers need not hold any upper-layer lock.
nbl_intr_mgt_stop() iterates all 520 function IDs and destroys any
leftover MSI-X map (including maps for remote PFs configured via
mailbox RPC), followed by a final global quiesce sleep. It is
called from nbl_res_remove_leonis() before devres releases the
coherent tables.
VF func_ids are not supported: nbl_res_func_id_to_bdf() returns
-EOPNOTSUPP for IDs beyond the PF range.
The manager is instantiated via nbl_intr_mgt_start() during
resource initialization and attached to the resource management
context.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 153 ++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 42 ++
.../nbl_hw_leonis/nbl_resource_leonis.c | 31 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 11 +
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 544 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 32 ++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 38 ++
.../nbl/nbl_include/nbl_def_hw.h | 10 +
.../nbl/nbl_include/nbl_def_resource.h | 6 +
.../nbl/nbl_include/nbl_include.h | 1 +
13 files changed, 885 insertions(+), 6 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 3dab9519a277..5aec8e44f5d7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -8,4 +8,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
+ nbl_hw/nbl_interrupt.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index b4aba4faa555..4c2e57761023 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -46,6 +46,30 @@ static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
}
+static void nbl_hw_rd_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+}
+
+static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+}
+
static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
u32 len)
{
@@ -91,6 +115,124 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+/*
+ * nbl_hw_set_mailbox_irq - read-modify-write NBL_MAILBOX_QINFO_MAP_REG_ARR
+ *
+ * The full RMW sequence is wrapped by reg_lock, so concurrent register
+ * access from different CPUs is already serialized safely.
+ * nbl_hw_cfg_mailbox_qinfo() overwrites the entire register during init,
+ * which unconditionally clears MSIX_IDX and MSIX_IDX_VALID bits, disabling
+ * mailbox MSIX interrupt routing for this PF.
+ */
+static void nbl_hw_set_mailbox_irq(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id)
+{
+ u32 data = 0;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ data &= ~(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK |
+ NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK);
+ if (en_msix)
+ data |= FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK,
+ global_vec_id) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK,
+ 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+ nbl_flush_writes(hw_mgt);
+}
+
+static void nbl_hw_cfg_msix_map(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function)
+{
+ struct nbl_function_msix_map function_msix_map;
+
+ memset(&function_msix_map, 0, sizeof(function_msix_map));
+ if (valid) {
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ /* use ctrl dev's bdf, because the dma memory was
+ * allocated by it
+ */
+ function_msix_map.data[2] =
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK,
+ function) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_BUS_MASK, bus) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_VALID_MASK, 1);
+ } else {
+ /*
+ * reg_lock prevents concurrent CPU writes to the same
+ * function's MSIX entry, but cannot synchronize hardware DMA
+ * reads. Upper layer uses two-stage destruction + sync sleep
+ * to avoid torn hardware read of partial MSIX entry.
+ * Keep valid live dma address here, only clear VALID flag.
+ */
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ function_msix_map.data[2] = 0;
+ }
+
+ nbl_hw_wr_regs_lock(hw_mgt,
+ NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(func_id),
+ function_msix_map.data, sizeof(function_msix_map));
+}
+
+static void nbl_hw_cfg_msix_info(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function, bool msix_mask_en)
+{
+ u32 host_msix_fid = 0;
+ struct nbl_host_msix_info msix_info;
+
+ memset(&msix_info, 0, sizeof(msix_info));
+ if (valid) {
+ host_msix_fid =
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK,
+ func_id) |
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK,
+ 1);
+
+ msix_info.data[1] =
+ FIELD_PREP(NBL_HOST_MSIX_INFO_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_BUS_MASK, bus) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_VALID_MASK, 1);
+
+ if (msix_mask_en)
+ msix_info.data[1] |=
+ FIELD_PREP(NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK, 1);
+ }
+ spin_lock(&hw_mgt->reg_lock);
+ /*
+ * Programming order rule:
+ * Enable: PADPT_HOST_MSIX_INFO -> PCOMPLETER_HOST_MSIX_FID_TABLE
+ * Teardown: reverse order, clear FID VLD first to avoid inconsistent
+ * state
+ */
+ if (valid) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ } else {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -212,6 +354,10 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .cfg_msix_map = nbl_hw_cfg_msix_map,
+ .cfg_msix_info = nbl_hw_cfg_msix_info,
+ .flush_write = nbl_flush_writes,
+
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
.config_mailbox_txq = nbl_hw_config_mailbox_txq,
@@ -221,6 +367,7 @@ static struct nbl_hw_ops hw_ops = {
.get_real_bus = nbl_hw_get_real_bus,
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .set_mailbox_irq = nbl_hw_set_mailbox_irq,
.get_fw_eth_map = nbl_hw_get_fw_eth_map,
.get_board_info = nbl_hw_get_board_info,
@@ -251,11 +398,12 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
if (!hw_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ if (!hw_ops.cfg_msix_map || !hw_ops.cfg_msix_info ||
+ !hw_ops.flush_write || !hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
!hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
- !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.cfg_mailbox_qinfo || !hw_ops.set_mailbox_irq ||
!hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
@@ -375,7 +523,6 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
ret = -EIO;
goto setup_mgt_fail;
}
-
hw_mgt->mailbox_bar_size = bar_len;
spin_lock_init(&hw_mgt->reg_lock);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 86d42a0a5687..5cde9f6496c2 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -68,6 +68,48 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+
+/* -------- HOST_PADPT -------- */
+/* host_padpt host_msix_info */
+#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
+ (vector_id) * sizeof(struct nbl_host_msix_info))
+
+#define NBL_HOST_MSIX_INFO_DWLEN 2
+/* data[0] */
+#define NBL_HOST_MSIX_INFO_INTRL_PNUM_MASK GENMASK(15, 0)
+#define NBL_HOST_MSIX_INFO_INTRL_RATE_MASK GENMASK(31, 16)
+/* data[1] */
+#define NBL_HOST_MSIX_INFO_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_HOST_MSIX_INFO_DEVID_MASK GENMASK(7, 3)
+#define NBL_HOST_MSIX_INFO_BUS_MASK GENMASK(15, 8)
+#define NBL_HOST_MSIX_INFO_VALID_MASK BIT(16)
+#define NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK BIT(17)
+struct nbl_host_msix_info {
+ u32 data[NBL_HOST_MSIX_INFO_DWLEN];
+};
+
+/* -------- HOST_PCOMPLETER -------- */
+/* pcompleter_host pcompleter_host_virtio_qid_map_table */
+#define NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x00004000 + \
+ (i) * sizeof(struct nbl_function_msix_map))
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x0003a000 + (i) * sizeof(u32))
+
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK GENMASK(9, 0)
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK BIT(10)
+
+#define NBL_FUNC_MSIX_MAP_DWLEN 4
+/* data[2] */
+#define NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_FUNCTION_MSIX_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_FUNCTION_MSIX_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_FUNCTION_MSIX_MAP_VALID_MASK BIT(16)
+struct nbl_function_msix_map {
+ u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
+};
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index 46180522295a..c1f10f1f6b77 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -10,6 +10,9 @@
static struct nbl_resource_ops res_ops = {
.get_vsi_id = nbl_res_func_id_to_vsi_id,
.get_eth_id = nbl_res_get_eth_id,
+ .cfg_msix_map = nbl_res_intr_cfg_msix_map,
+ .destroy_msix_map = nbl_res_intr_destroy_msix_map,
+ .set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
};
static struct nbl_resource_mgt *
@@ -41,7 +44,9 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
if (!res_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!res_ops.get_vsi_id || !res_ops.get_eth_id)
+ if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
+ !res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
+ !res_ops.set_mailbox_irq)
return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
@@ -288,6 +293,10 @@ static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
if (ret)
return ret;
+
+ ret = nbl_intr_mgt_start(res_mgt);
+ if (ret)
+ return ret;
}
return 0;
@@ -328,8 +337,24 @@ int nbl_res_init_leonis(struct nbl_adapter *adap)
void nbl_res_remove_leonis(struct nbl_adapter *adap)
{
+ struct nbl_resource_mgt *res_mgt = adap->core.res_mgt;
+ struct nbl_common_info *common = &adap->common;
+
+ if (!res_mgt)
+ return;
+
+ /*
+ * Tear down all MSI-X maps before devres releases the coherent
+ * tables. This is critical on the control PF, which may hold
+ * maps for remote PFs that are still bound.
+ */
+ if (common->has_ctrl && res_mgt->intr_mgt)
+ nbl_intr_mgt_stop(res_mgt);
+
/*
- * No resource release here because all memory uses devm managed
- * allocation
+ * Note: the per-function interrupts arrays (kcalloc) are freed
+ * by nbl_intr_mgt_stop() above. The coherent MSI-X tables
+ * (dmam_alloc_coherent) and intr_mgt itself (devm_kzalloc) are
+ * released by devres after this function returns.
*/
}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index b9355262c00d..6eb4dc9e695a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -7,4 +7,5 @@
#define _NBL_RESOURCE_LEONIS_H_
#include "../nbl_resource.h"
+#include "../nbl_interrupt.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
index fdf7b3d96087..c93086f3bfef 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -57,6 +57,17 @@ static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
}
+/*
+ * Only call this when has_ctrl=true, which maps enough space
+ * (bar_len - 8192) to cover NBL_HW_DUMMY_REG (0x1300904).
+ * The flow/design guarantees this is only called in the
+ * has_ctrl path.
+ */
+static inline void nbl_flush_writes(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_rd32(hw_mgt, NBL_HW_DUMMY_REG);
+}
+
static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
{
return readl(hw_mgt->mailbox_bar_hw_addr + reg);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
new file mode 100644
index 000000000000..fd3b71a05c23
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
@@ -0,0 +1,544 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/bitfield.h>
+#include "nbl_interrupt.h"
+
+#define NBL_MSIX_DMA_SYNC_MIN_US 1000
+#define NBL_MSIX_DMA_SYNC_MAX_US 1200
+
+/*
+ * Release global vector IDs back to intr_net_bmap / intr_other_bmap.
+ * Caller must hold intr_mgt->lock.
+ */
+static void nbl_intr_release_bitmap(struct nbl_resource_mgt *res_mgt,
+ u16 *vec_buf, u16 cnt)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ u16 bit;
+ u16 i;
+
+ lockdep_assert_held(&intr_mgt->lock);
+
+ if (!vec_buf || cnt == 0)
+ return;
+
+ for (i = 0; i < cnt; i++) {
+ u16 intr_index = vec_buf[i];
+
+ if (intr_index >= NBL_NET_INTR_BASE) {
+ bit = intr_index - NBL_NET_INTR_BASE;
+ if (bit < NBL_MAX_NET_INTERRUPT)
+ clear_bit(bit, intr_mgt->intr_net_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid net intr index %u\n",
+ intr_index);
+ } else {
+ if (intr_index < NBL_MAX_OTHER_INTERRUPT)
+ clear_bit(intr_index,
+ intr_mgt->intr_other_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid other intr index %u\n",
+ intr_index);
+ }
+ }
+}
+
+/*
+ * Internal (unlocked) mailbox IRQ bind. Caller must hold
+ * intr_mgt->lock. The disable path does not require a configured
+ * MSI-X map because the hardware op ignores global_vec_id when
+ * en_msix=false.
+ */
+static int __nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ u16 global_vec_id;
+
+ lockdep_assert_held(&intr_mgt->lock);
+
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "func_id %u out of range\n", func_id);
+ return -EINVAL;
+ }
+
+ if (!en_msix) {
+ hw_ops->set_mailbox_irq(res_mgt->hw_ops_tbl->priv,
+ func_id, false, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+ return 0;
+ }
+
+ if (!intr_mgt->func_intr_res[func_id].interrupts) {
+ dev_err(dev, "func %u MSIX map not configured\n", func_id);
+ return -ENODEV;
+ }
+ if (vector_id >= intr_mgt->func_intr_res[func_id].num_interrupts) {
+ dev_err(dev, "vector_id %u out of range (max %u)\n",
+ vector_id,
+ intr_mgt->func_intr_res[func_id].num_interrupts - 1);
+ return -EINVAL;
+ }
+
+ global_vec_id = intr_mgt->func_intr_res[func_id].interrupts[vector_id];
+ hw_ops->set_mailbox_irq(res_mgt->hw_ops_tbl->priv, func_id,
+ en_msix, global_vec_id);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ return 0;
+}
+
+/*
+ * Internal (unlocked) MSI-X map teardown. Caller must hold
+ * intr_mgt->lock. This exists because cfg_msix_map() and
+ * nbl_intr_mgt_stop() need to destroy a map while already holding
+ * the lock; the public nbl_res_intr_destroy_msix_map() wraps this
+ * with mutex_lock/unlock.
+ */
+static int __nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_msix_map_table *msix_map_table;
+ u16 *interrupts;
+ u16 intr_num, i;
+
+ lockdep_assert_held(&intr_mgt->lock);
+
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+
+ interrupts = intr_mgt->func_intr_res[func_id].interrupts;
+ if (!interrupts)
+ return 0; /* already destroyed or never configured */
+
+ intr_num = intr_mgt->func_intr_res[func_id].num_interrupts;
+ msix_map_table = &intr_mgt->func_intr_res[func_id].msix_map_table;
+
+ /* Step 0: disable mailbox IRQ routing before tearing down map */
+ __nbl_res_intr_set_mailbox_irq(res_mgt, func_id, 0, false);
+
+ /* Step 1: mask each MSIX vector in hardware first */
+ for (i = 0; i < intr_num; i++) {
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, false, interrupts[i],
+ 0, 0, 0, false);
+ }
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Stage 1 tear down: retain valid DMA address, ONLY clear
+ * VALID bit to avoid hardware torn read (VALID=1 & dma_addr=0).
+ */
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ false, msix_map_table->dma, 0, 0, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Hardware provides no idle status register for the MSIX map
+ * DMA engine. Use a bounded sleep to mitigate the race between
+ * posted MMIO disable writes and an ongoing in-flight table
+ * read DMA.
+ *
+ * This is best-effort, not a guarantee: a table fetch already
+ * issued before the VALID clear was observed can complete after
+ * this sleep. On the normal teardown path the mailbox channel
+ * is stopped before this function runs, so no new interrupts
+ * can trigger table fetches. On the residual cleanup path in
+ * nbl_intr_mgt_stop(), a longer global quiesce is applied
+ * after all functions are torn down.
+ */
+ usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
+
+ /* safe to release global vector IDs, pcompler no longer reads table */
+ nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
+
+ /*
+ * Stage 2: hardware has quiesced MSIX table DMA access, fully
+ * zero the MSIX map entry safely now.
+ */
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ false, 0, 0, 0, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Now safe to release the MSIX DMA coherent memory. Hardware
+ * DMA has quiesced after the sleep above, so no IOMMU fault
+ * risk remains.
+ */
+ if (msix_map_table->base_addr) {
+ dmam_free_coherent(dev, msix_map_table->size,
+ msix_map_table->base_addr,
+ msix_map_table->dma);
+ msix_map_table->base_addr = NULL;
+ msix_map_table->dma = 0;
+ msix_map_table->size = 0;
+ }
+
+ /* Release runtime-allocated interrupt vector buffer */
+ kfree(interrupts);
+ intr_mgt->func_intr_res[func_id].interrupts = NULL;
+ intr_mgt->func_intr_res[func_id].num_interrupts = 0;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = 0;
+ return 0;
+}
+
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ int ret;
+
+ if (!intr_mgt)
+ return -EINVAL;
+
+ mutex_lock(&intr_mgt->lock);
+ ret = __nbl_res_intr_destroy_msix_map(res_mgt, func_id);
+ mutex_unlock(&intr_mgt->lock);
+ return ret;
+}
+
+/**
+ * nbl_res_intr_cfg_msix_map - allocate & program MSI-X mapping table
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @num_net_msix: required net data interrupt vectors
+ * @num_others_msix: required control interrupt vectors
+ * @net_msix_mask_en: enable mask for net interrupt entries
+ *
+ * Allocate interrupt vectors and coherent DMA table in advance;
+ * only destroy old configuration once all allocations succeed.
+ *
+ * Note: There exists a transient window after tearing down old MSI-X
+ * hardware state before programming new mapping. Atomic table swap is
+ * unsupported on current silicon, this gap is accepted as hardware
+ * limitation.
+ *
+ * Serialization: this function takes intr_mgt->lock internally to
+ * protect the global vector bitmaps and per-function state against
+ * concurrent callers.
+ *
+ * Old MSIX table memory is explicitly freed inside the locked
+ * destroy path after a bounded DMA quiesce sleep (best-effort;
+ * hardware provides no idle status register), so repeated
+ * reconfiguration does not accumulate devres-managed DMA memory.
+ *
+ * Return: 0 on success, negative errno on failure
+ */
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_msix_map_table *tmp_msix_tbl = NULL;
+ struct nbl_msix_map_table *official_tbl;
+ struct nbl_msix_map *msix_map_entries;
+ struct device *dev = common->dev;
+ u16 requested, intr_index;
+ u8 bus, devid, function;
+ bool entry_masked = false;
+ u16 *tmp_interrupts = NULL;
+ u16 global_vec;
+ int ret = 0;
+ u16 i;
+
+ if (!intr_mgt)
+ return -EINVAL;
+ if (!common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+
+ if (num_net_msix == 0 && num_others_msix == 0) {
+ dev_err(dev, "MSI-X vector count cannot both be zero\n");
+ return -EINVAL;
+ }
+
+ if (num_net_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES ||
+ num_others_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "MSI-X count out of limit: net=%u, others=%u\n",
+ num_net_msix, num_others_msix);
+ return -EINVAL;
+ }
+
+ if (check_add_overflow(num_net_msix, num_others_msix, &requested) ||
+ requested > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "Total MSI-X vectors %u exceeds maximum %u\n",
+ requested, NBL_MSIX_MAP_TABLE_MAX_ENTRIES);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_func_id_to_bdf(res_mgt, func_id, &bus, &devid, &function);
+ if (ret) {
+ if (ret == -EOPNOTSUPP)
+ dev_err(dev,
+ "MSI-X mapping for VF func_id=%u is not supported\n",
+ func_id);
+ return ret;
+ }
+
+ mutex_lock(&intr_mgt->lock);
+
+ /*
+ * Phase1: Pre-allocate ALL new resources first.
+ * Do NOT destroy old configuration before all allocations succeed.
+ */
+ tmp_msix_tbl = kzalloc_obj(*tmp_msix_tbl);
+ if (!tmp_msix_tbl) {
+ ret = -ENOMEM;
+ goto out_unlock;
+ }
+
+ tmp_msix_tbl->size =
+ sizeof(struct nbl_msix_map) * NBL_MSIX_MAP_TABLE_MAX_ENTRIES;
+ /*
+ * Hardware requires fixed stride table layout; allocate full size
+ * even when only partial entries are used. Memory managed by devm.
+ */
+ tmp_msix_tbl->base_addr = dmam_alloc_coherent(dev, tmp_msix_tbl->size,
+ &tmp_msix_tbl->dma,
+ GFP_KERNEL);
+ if (!tmp_msix_tbl->base_addr) {
+ dev_err(dev, "Failed to allocate DMA memory for MSIX table\n");
+ ret = -ENOMEM;
+ goto free_tmp_tbl_unlock;
+ }
+
+ tmp_interrupts = kcalloc(requested, sizeof(tmp_interrupts[0]),
+ GFP_KERNEL);
+ if (!tmp_interrupts) {
+ ret = -ENOMEM;
+ goto free_tmp_tbl_unlock;
+ }
+
+ /* Allocate net interrupt vectors */
+ for (i = 0; i < num_net_msix; i++) {
+ intr_index = find_first_zero_bit(intr_mgt->intr_net_bmap,
+ NBL_MAX_NET_INTERRUPT);
+ if (intr_index == NBL_MAX_NET_INTERRUPT) {
+ dev_err(dev, "No free net interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs_unlock;
+ }
+ tmp_interrupts[i] = intr_index + NBL_NET_INTR_BASE;
+ set_bit(intr_index, intr_mgt->intr_net_bmap);
+ }
+
+ /* Allocate other interrupt vectors */
+ for (; i < requested; i++) {
+ intr_index =
+ find_first_zero_bit(intr_mgt->intr_other_bmap,
+ NBL_MAX_OTHER_INTERRUPT);
+ if (intr_index == NBL_MAX_OTHER_INTERRUPT) {
+ dev_err(dev, "No free control interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs_unlock;
+ }
+ tmp_interrupts[i] = intr_index;
+ set_bit(intr_index, intr_mgt->intr_other_bmap);
+ }
+
+ /*
+ * Phase2: All new resource allocation succeeded.
+ * Now tear down old MSIX hardware configuration.
+ * Call the unlocked internal version since we hold the lock.
+ */
+ ret = __nbl_res_intr_destroy_msix_map(res_mgt, func_id);
+ if (ret)
+ goto release_vecs_unlock;
+
+ /* Swap temporary resources into official entry */
+ official_tbl = &intr_mgt->func_intr_res[func_id].msix_map_table;
+ official_tbl->base_addr = tmp_msix_tbl->base_addr;
+ official_tbl->dma = tmp_msix_tbl->dma;
+ official_tbl->size = tmp_msix_tbl->size;
+ kfree(tmp_msix_tbl);
+ tmp_msix_tbl = NULL;
+
+ intr_mgt->func_intr_res[func_id].interrupts = tmp_interrupts;
+ intr_mgt->func_intr_res[func_id].num_interrupts = requested;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = num_net_msix;
+ tmp_interrupts = NULL;
+
+ /*
+ * NOTE: After this point tmp_interrupts is NULL and the official
+ * entry owns the vectors. If a future revision adds fallible
+ * operations below (e.g. cfg_msix_map returning an error), the
+ * rollback must release vectors through
+ * intr_mgt->func_intr_res[func_id].interrupts, not tmp_interrupts.
+ */
+
+ /* Fill MSIX map table and program hardware */
+ msix_map_entries = official_tbl->base_addr;
+ for (i = 0; i < requested; i++) {
+ global_vec = intr_mgt->func_intr_res[func_id].interrupts[i];
+ msix_map_entries[i].data =
+ cpu_to_le16(FIELD_PREP(NBL_MSIX_MAP_VALID_MASK, 1) |
+ FIELD_PREP(NBL_MSIX_MAP_INDEX_MASK,
+ global_vec));
+
+ entry_masked = (i < num_net_msix && net_msix_mask_en);
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, true, global_vec,
+ bus, devid, function,
+ entry_masked);
+ }
+
+ /* Flush CPU writes to coherent memory before hardware DMA access */
+ dma_wmb();
+ /*
+ * cfg_msix_map uses the control PF's own BDF (common->hw_bus etc.),
+ * not the target function's BDF. This BDF tags the pcompler DMA
+ * read of the MSI-X map table as originating from the control PF.
+ * The target function's BDF (bus/devid/function from
+ * nbl_res_func_id_to_bdf) is used only in cfg_msix_info for the
+ * host_msix_ctrl table entry BDF filtering.
+ */
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ true, official_tbl->dma, common->hw_bus,
+ common->devid, common->function);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ mutex_unlock(&intr_mgt->lock);
+ return 0;
+
+release_vecs_unlock:
+ nbl_intr_release_bitmap(res_mgt, tmp_interrupts, i);
+free_tmp_tbl_unlock:
+ /* Release DMA buffer allocated by dmam_alloc_coherent first */
+ if (tmp_msix_tbl && tmp_msix_tbl->base_addr) {
+ dmam_free_coherent(dev, tmp_msix_tbl->size,
+ tmp_msix_tbl->base_addr,
+ tmp_msix_tbl->dma);
+ }
+ kfree(tmp_msix_tbl);
+ kfree(tmp_interrupts);
+out_unlock:
+ mutex_unlock(&intr_mgt->lock);
+ return ret;
+}
+
+/**
+ * nbl_res_intr_set_mailbox_irq - bind mailbox IRQ to specified vector
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @vector_id: index inside local interrupt array
+ * @en_msix: enable/disable mailbox interrupt
+ *
+ * Serialization: takes intr_mgt->lock internally.
+ *
+ * Return: 0 on success, negative errno on parameter or state check
+ * failure. The hardware op is void and cannot report failure.
+ */
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_common_info *common = res_mgt->common;
+ int ret;
+
+ if (!intr_mgt)
+ return -EINVAL;
+ if (!common->has_ctrl)
+ return -EINVAL;
+
+ mutex_lock(&intr_mgt->lock);
+ ret = __nbl_res_intr_set_mailbox_irq(res_mgt, func_id,
+ vector_id, en_msix);
+ mutex_unlock(&intr_mgt->lock);
+ return ret;
+}
+
+static struct nbl_interrupt_mgt *nbl_intr_setup_mgt(struct device *dev)
+{
+ struct nbl_interrupt_mgt *intr_mgt;
+ int err;
+
+ intr_mgt = devm_kzalloc(dev, sizeof(*intr_mgt), GFP_KERNEL);
+ if (!intr_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ err = devm_mutex_init(dev, &intr_mgt->lock);
+ if (err)
+ return ERR_PTR(err);
+ bitmap_zero(intr_mgt->intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ bitmap_zero(intr_mgt->intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+
+ return intr_mgt;
+}
+
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_interrupt_mgt *intr_mgt;
+ int ret;
+
+ intr_mgt = nbl_intr_setup_mgt(dev);
+ if (IS_ERR(intr_mgt)) {
+ ret = PTR_ERR(intr_mgt);
+ return ret;
+ }
+ res_mgt->intr_mgt = intr_mgt;
+ return 0;
+}
+
+void nbl_intr_mgt_stop(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ u16 func_id;
+ int ret;
+
+ if (!intr_mgt)
+ return;
+
+ mutex_lock(&intr_mgt->lock);
+ for (func_id = 0; func_id < NBL_MAX_FUNC; func_id++) {
+ if (intr_mgt->func_intr_res[func_id].interrupts) {
+ dev_info(res_mgt->common->dev,
+ "intr_mgt_stop: destroying leftover map for func %u\n",
+ func_id);
+ ret = __nbl_res_intr_destroy_msix_map(res_mgt,
+ func_id);
+ if (ret)
+ dev_warn(res_mgt->common->dev,
+ "intr_mgt_stop: destroy map for func %u failed: %d\n",
+ func_id, ret);
+ }
+ }
+ mutex_unlock(&intr_mgt->lock);
+
+ /*
+ * Global quiesce after all functions are torn down. Each
+ * destroy has an internal 1ms sleep between Stage 1 (clear
+ * VALID) and Stage 2 (zero dma_addr), but Stage 2 itself has
+ * no trailing sleep. This final wait covers the last
+ * function's Stage 2 and any straggler DMA from
+ * concurrently-torndown functions.
+ */
+ usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
+
+ res_mgt->intr_mgt = NULL;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
new file mode 100644
index 000000000000..9f66f5e19c98
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INTERRUPT_H_
+#define _NBL_INTERRUPT_H_
+
+#include "nbl_resource.h"
+
+#define NBL_MSIX_MAP_TABLE_MAX_ENTRIES 1024
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id);
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en);
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
index 6fa0e0d550f4..411790adfb39 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -68,6 +68,38 @@ int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
return -ENOENT;
}
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ int pfid = func_id;
+ u8 pf_bus, devfn;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !bus || !dev || !function)
+ return -EINVAL;
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= common->max_pf) {
+ dev_err(common->dev,
+ "func_id=%u rel_pf_id=%u exceeds max_pf=%u, VF BDF unsupported\n",
+ pfid, rel_pf_id,
+ common->max_pf);
+ return -EOPNOTSUPP;
+ }
+ sriov_info = res_mgt->resource_info->sriov_info + rel_pf_id;
+ pf_bus = PCI_BUS_NUM(sriov_info->bdf);
+ devfn = sriov_info->bdf & 0xff;
+ *bus = pf_bus;
+ *dev = PCI_SLOT(devfn);
+ *function = PCI_FUNC(devfn);
+
+ return 0;
+}
+
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
{
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
index a3bc7b3aecde..5f0dfc74c068 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -17,6 +17,39 @@
struct nbl_resource_mgt;
+/* --------- INTERRUPT ---------- */
+#define NBL_MAX_OTHER_INTERRUPT 1024
+#define NBL_MAX_NET_INTERRUPT 4096
+#define NBL_NET_INTR_BASE NBL_MAX_OTHER_INTERRUPT
+
+#define NBL_MSIX_MAP_VALID_MASK BIT(0)
+#define NBL_MSIX_MAP_INDEX_MASK GENMASK(13, 1)
+#define NBL_MSIX_MAP_RSV_MASK GENMASK(15, 14)
+
+struct nbl_msix_map {
+ __le16 data;
+};
+
+struct nbl_msix_map_table {
+ struct nbl_msix_map *base_addr;
+ dma_addr_t dma;
+ size_t size;
+};
+
+struct nbl_func_interrupt_resource_mng {
+ u16 num_interrupts;
+ u16 num_net_interrupts;
+ u16 *interrupts;
+ struct nbl_msix_map_table msix_map_table;
+};
+
+struct nbl_interrupt_mgt {
+ struct mutex lock; /* Protects bitmap + func_intr_res[] */
+ DECLARE_BITMAP(intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ DECLARE_BITMAP(intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+ struct nbl_func_interrupt_resource_mng func_intr_res[NBL_MAX_FUNC];
+};
+
/* --------- INFO ---------- */
struct nbl_sriov_info {
unsigned int bdf;
@@ -59,14 +92,19 @@ struct nbl_resource_mgt {
struct nbl_resource_info *resource_info;
struct nbl_channel_ops_tbl *chan_ops_tbl;
struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_interrupt_mgt *intr_mgt;
};
int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function);
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt);
int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
u16 src_type,
enum nbl_vsi_serv_type *dst_type);
+void nbl_intr_mgt_stop(struct nbl_resource_mgt *res_mgt);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index e05248c66afb..f83e6ea9d58f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -12,6 +12,14 @@ struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function);
+ void (*cfg_msix_info)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function,
+ bool net_msix_mask_en);
+ void (*flush_write)(struct nbl_hw_mgt *hw_mgt);
void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx);
void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
@@ -44,6 +52,8 @@ struct nbl_hw_ops {
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*set_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id);
void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
/**
* get_board_info - Fetch board info from firmware
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index 7136b282fb80..e718ea41a816 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,12 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id);
+ int (*set_mailbox_irq)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vector_id, bool en_msix);
int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 59e44feab44f..2c959832c32f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,7 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
enum {
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (4 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
` (3 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add Leonis chip-wide hardware initialization and deinitialization
logic to setup the full datapath pipeline, including packet engines,
scheduler, buffer stores, queue manager, port traffic shaping,
and flow control, plus the driver_status hardware flag consumed
by firmware.
This patch introduces hw_ops.init_module/deinit_module and
corresponding resource_ops entry points. The actual invocation
path for these callbacks lands in a subsequent control device
initialization patch; only the ops registration is added here.
DP sub-init modules (called from nbl_dp_init()):
- dped, uped: Data/User Packet Engine Driver
- dsch: Scheduler core (qid limit, VN quanta)
- ustore, dstore: Buffer Store modules
- dvn, uvn: Virtual Descriptor Network modules
- uqm: Queue Manager
- nbl_shaping_init(): Per-port traffic shaping + DSCH PSHA config
Chip init sequence (nbl_hw_init_module()):
1. nbl_dp_init() — All DP sub-modules listed above
2. nbl_intf_init() — Host adapter padpt flow control
3. nbl_hw_set_driver_status() + nbl_flush_writes()
Both nbl_res_chip_init_module() and nbl_res_chip_deinit_module()
carry an explicit has_ctrl guard for defense-in-depth, matching the
sibling resource ops, even though the dispatch layer only assigns
these ops to NBL_DISP_CTRL_LVL_MGT.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 32 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 586 +++++++++++++++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 211 +++++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 5 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nbl/nbl_include/nbl_def_hw.h | 3 +
.../nbl/nbl_include/nbl_def_resource.h | 3 +
.../nbl/nbl_include/nbl_include.h | 21 +
10 files changed, 873 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 5aec8e44f5d7..be314b909d66 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -9,4 +9,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
+ nbl_hw/nbl_chip.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
new file mode 100644
index 000000000000..419eb6392ada
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include "nbl_chip.h"
+
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+
+ if (!common->has_ctrl)
+ return;
+ hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
+}
+
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_hw_ops *hw_ops;
+ u8 eth_speed, eth_num;
+ struct nbl_hw_mgt *p;
+
+ if (!common->has_ctrl)
+ return -EINVAL;
+ eth_speed = res_mgt->resource_info->board_info.eth_speed;
+ eth_num = res_mgt->resource_info->board_info.eth_num;
+ hw_ops = res_mgt->hw_ops_tbl->ops;
+ p = res_mgt->hw_ops_tbl->priv;
+ return hw_ops->init_module(p, eth_speed, eth_num);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
new file mode 100644
index 000000000000..d14093ba916c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHIP_H_
+#define _NBL_CHIP_H_
+
+#include "nbl_resource.h"
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt);
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 4c2e57761023..9f1054067e14 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -115,6 +115,585 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+static u32 nbl_hw_get_quirks(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quirks = 0;
+
+ /*
+ * Read quirk bits from mailbox register.
+ * All supported firmware implement the quirk ABI,
+ * firmware always populates NBL_LEONIS_QUIRKS_OFFSET.
+ * Value ~0U indicates no active quirks.
+ */
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_LEONIS_QUIRKS_OFFSET, &quirks,
+ sizeof(u32));
+
+ if (quirks == ~0u)
+ return 0;
+
+ return quirks;
+}
+
+static void nbl_configure_dped_checksum(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 data = 0;
+
+ /* DPED dped_l4_ck_cmd_40 for sctp */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ data |= FIELD_PREP(NBL_DPED_L4_CK_CMD_40_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_dped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_VLAN_OFFSET, 0xC);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_0, 0x8);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_1, 0x4);
+ spin_unlock(&hw_mgt->reg_lock);
+ /* dped checksum offload */
+ nbl_configure_dped_checksum(hw_mgt);
+}
+
+static void nbl_uped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 hw_edit = 0;
+
+ /* V4 TCP: l3_len = 0 */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit &= ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+
+ /* V6 TCP: l3_len = 1 */
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit = (hw_edit & ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK) |
+ FIELD_PREP(NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
+{
+ struct nbl_shaping_dvn_dport_u dvn_dport = { 0 };
+ struct nbl_shaping_dport_u dport = { 0 };
+ u32 rate, half_rate;
+ u32 depth;
+ u64 low_val, high_val;
+
+ switch (speed) {
+ case NBL_FW_PORT_SPEED_100G:
+ rate = 100000;
+ break;
+ case NBL_FW_PORT_SPEED_50G:
+ rate = 50000;
+ break;
+ case NBL_FW_PORT_SPEED_25G:
+ rate = 25000;
+ break;
+ case NBL_FW_PORT_SPEED_10G:
+ rate = 10000;
+ break;
+ default:
+ dev_err(hw_mgt->common->dev,
+ "Unsupported port speed %u for eth%u\n", speed, eth_id);
+ return -EINVAL;
+ }
+
+ half_rate = rate / 2;
+ depth = max_t(u32, rate * 2, NBL_LR_LEONIS_NET_BUCKET_DEPTH);
+
+ /* 1. clear valid first
+ * dport and dvn_dport are zero-initialised above, so VALID=0 already
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 2. write config words (valid=0, safe) */
+ low_val = FIELD_PREP(NBL_DPORT_CIR_MASK, rate) |
+ FIELD_PREP(NBL_DPORT_PIR_MASK, rate) |
+ FIELD_PREP(NBL_DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(NBL_DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(NBL_DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(NBL_DPORT_PBS_MASK, depth);
+ /* Fixed split, independent of host endian */
+ dport.data[0] = lower_32_bits(low_val);
+ dport.data[1] = upper_32_bits(low_val);
+ dport.data[2] = lower_32_bits(high_val);
+ dport.data[3] = upper_32_bits(high_val);
+
+ low_val = FIELD_PREP(NBL_DPORT_CIR_MASK, half_rate) |
+ FIELD_PREP(NBL_DPORT_PIR_MASK, rate) |
+ FIELD_PREP(NBL_DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(NBL_DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(NBL_DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(NBL_DPORT_PBS_MASK, depth);
+ dvn_dport.data[0] = lower_32_bits(low_val);
+ dvn_dport.data[1] = upper_32_bits(low_val);
+ dvn_dport.data[2] = lower_32_bits(high_val);
+ dvn_dport.data[3] = upper_32_bits(high_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 3. commit: set valid last */
+ low_val = FIELD_PREP(NBL_DPORT_VALID_MASK, 1);
+ dport.data[0] |= lower_32_bits(low_val);
+
+ low_val = FIELD_PREP(NBL_DPORT_VALID_MASK, 1);
+ dvn_dport.data[0] |= lower_32_bits(low_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static int nbl_shaping_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+#define NBL_SHAPING_FLUSH_INTERVAL 128
+ struct nbl_shaping_net_u net_shaping = { 0 };
+ u32 eth_bitmap = 0;
+ u32 psha_en = 0;
+ int ret;
+ int i;
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ ret = nbl_shaping_eth_init(hw_mgt, i, speed);
+ if (ret)
+ return ret;
+ }
+ psha_en = eth_bitmap & GENMASK(3, 0);
+ psha_en = FIELD_PREP(NBL_DSCH_PSHA_EN_MASK, psha_en);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DSCH_PSHA_EN_ADDR, &psha_en,
+ sizeof(psha_en));
+
+ for (i = 0; i < NBL_MAX_FUNC; i++) {
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_SHAPING_NET_REG(i),
+ net_shaping.data,
+ sizeof(net_shaping));
+ if ((i + 1) % NBL_SHAPING_FLUSH_INTERVAL == 0)
+ nbl_flush_writes(hw_mgt);
+ }
+ nbl_flush_writes(hw_mgt);
+ return 0;
+}
+
+static void nbl_dsch_qid_max_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quanta = 0;
+
+ quanta = FIELD_PREP(NBL_DSCH_VN_QUANTA_H_QUA_MASK, NBL_HOST_QUANTA) |
+ FIELD_PREP(NBL_DSCH_VN_QUANTA_E_QUA_MASK, NBL_ECPU_QUANTA);
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSCH_VN_QUANTA_ADDR, &quanta,
+ sizeof(quanta));
+ nbl_hw_wr32(hw_mgt, NBL_DSCH_HOST_QID_MAX, NBL_MAX_QUEUE_ID);
+
+ nbl_hw_wr32(hw_mgt, NBL_DVN_ECPU_QUEUE_NUM, 0);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_ECPU_QUEUE_NUM, 0);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th = 0;
+ u32 pkt_len = 0;
+ int i;
+
+ /*
+ * eth_num is validated in the resource layer:
+ * nbl_res_init_pf_num() requires 1/2/4 PFs, and
+ * nbl_res_ctrl_dev_setup_eth_info() requires max_pf == eth_num.
+ * This is a defensive check only; if it triggers, the resource
+ * layer validation was bypassed, which is a bug.
+ */
+ if (WARN_ON(eth_num != 1 && eth_num != 2 && eth_num != 4))
+ return -EINVAL;
+ /* Read current packet length config
+ *(to preserve other fields while updating 'min')
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+ /* min arp packet length 42 (14 + 28) */
+ pkt_len &= ~NBL_USTORE_PKT_LEN_MIN_MASK;
+ pkt_len |= FIELD_PREP(NBL_USTORE_PKT_LEN_MIN_MASK, 42);
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_EN_MASK, 1);
+ if (eth_num == 1)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_SINGLE_ETH_DROP_TH);
+ else if (eth_num == 2)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_DUAL_ETH_DROP_TH);
+ else
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_QUAD_ETH_DROP_TH);
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PORT_DROP_TH_REG_ARR(i),
+ &drop_th, sizeof(drop_th));
+ }
+
+ /* Clear port drop/truncate counters by reading them
+ * (hardware has read-to-clear behavior for these registers)
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_DROP_PKT(i));
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_TRUN_PKT(i));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static void nbl_dstore_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th = 0;
+ u32 fc_th = 0;
+ u32 bp_th = 0;
+ int i;
+
+ for (i = 0; i < NBL_DSTORE_PORT_DROP_TH_DEPTH; i++) {
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ drop_th &= ~NBL_DSTORE_PORT_DROP_EN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ bp_th |= FIELD_PREP(NBL_DSTORE_DISC_BP_TH_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ spin_unlock(&hw_mgt->reg_lock);
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ fc_th &= ~(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK |
+ NBL_DSTORE_D_DPORT_FC_XON_TH_MASK);
+ if (speed == NBL_FW_PORT_SPEED_100G) {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH_100G) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH_100G);
+ } else {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH);
+ }
+
+ fc_th |= FIELD_PREP(NBL_DSTORE_D_DPORT_FC_FC_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+}
+
+static void nbl_dvn_descreq_num_cfg(struct nbl_hw_mgt *hw_mgt, u8 descreq_num)
+{
+ u8 split_ring_num = (descreq_num >> 3) & 0x1;
+ u8 ring_num = descreq_num & 0x7;
+ u32 num_cfg;
+ u32 reg_val;
+
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, ®_val,
+ sizeof(reg_val));
+
+ num_cfg = FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK,
+ split_ring_num) |
+ FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK,
+ ring_num);
+ reg_val &= ~(NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK |
+ NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK);
+ reg_val |= num_cfg;
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, ®_val,
+ sizeof(reg_val));
+}
+
+static void nbl_dvn_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 timeout = 0;
+ u32 ro_flag = 0;
+
+ timeout = FIELD_PREP(NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK,
+ DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESC_WR_MERGE_TIMEOUT, &timeout,
+ sizeof(timeout));
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ if (pcie_relaxed_ordering_enabled(hw_mgt->common->pdev)) {
+ ro_flag |=
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1);
+ } else {
+ ro_flag &=
+ ~(FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ spin_unlock(&hw_mgt->reg_lock);
+ if (speed == NBL_FW_PORT_SPEED_100G)
+ nbl_dvn_descreq_num_cfg(hw_mgt,
+ DEFAULT_DVN_100G_DESCREQ_NUMCFG);
+ else
+ nbl_dvn_descreq_num_cfg(hw_mgt, DEFAULT_DVN_DESCREQ_NUMCFG);
+}
+
+static void nbl_uvn_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u16 wr_timeout = NBL_UVN_DESC_WR_TIMEOUT_VAL;
+ u32 timeout = NBL_UVN_DESC_RD_WAIT_TICKS;
+ u32 prefetch_init = 0;
+ bool ro_enabled;
+ u32 flag = 0;
+ u32 mask = 0;
+ u32 quirks;
+ u32 reg_val;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_DESC_RD_WAIT, timeout);
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT,
+ ®_val, sizeof(reg_val));
+ reg_val &= ~NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK;
+ reg_val |= FIELD_PREP(NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK, wr_timeout);
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT, ®_val,
+ sizeof(reg_val));
+ ro_enabled = pcie_relaxed_ordering_enabled(hw_mgt->common->pdev);
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+ if (ro_enabled) {
+ flag |= FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1);
+ flag &= ~FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1);
+ } else {
+ flag &= ~(FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+ mask |= FIELD_PREP(NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK, 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+
+ spin_unlock(&hw_mgt->reg_lock);
+ quirks = nbl_hw_get_quirks(hw_mgt);
+ /*
+ * sel=0: use configured num; sel=1: use internal calc (max 32)
+ * Default is sel=1, unless NBL_QUIRK_UVN_PREFETCH_ALIGN is set,
+ * in which case override to sel=0.
+ */
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,
+ ®_val, sizeof(reg_val));
+ prefetch_init =
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK,
+ NBL_UVN_DESC_PREFETCH_NUM) |
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK,
+ (quirks & NBL_QUIRK_UVN_PREFETCH_ALIGN) ? 0 : 1);
+ reg_val &= ~(NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK |
+ NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK);
+ reg_val |= prefetch_init;
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,
+ ®_val, sizeof(reg_val));
+}
+
+static void nbl_uqm_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 que_type = 0;
+ u32 cnt = 0;
+ int i;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_FWD_DROP_CNT, &cnt, sizeof(cnt));
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_SLICE_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_LEN_ADD_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_HEAD_PNTR_ADD_CNT, &cnt,
+ sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_WEIGHT_ADD_CNT, &cnt, sizeof(cnt));
+
+ for (i = 0; i < NBL_UQM_PORT_DROP_DEPTH; i++) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_CNT + (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_SLICE_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ }
+
+ for (i = 0; i < NBL_UQM_DPORT_DROP_DEPTH; i++)
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_DPORT_DROP_CNT + (sizeof(cnt) * i), &cnt,
+ sizeof(cnt));
+ /* bit0: 0=bp mode, 1=drop mode, resv bit1-31 */
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_QUE_TYPE, &que_type, sizeof(que_type));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_dp_init(struct nbl_hw_mgt *hw_mgt, u8 speed, u8 eth_num)
+{
+ int ret;
+
+ nbl_dped_init(hw_mgt);
+ nbl_uped_init(hw_mgt);
+ ret = nbl_shaping_init(hw_mgt, speed);
+ if (ret)
+ return ret;
+ nbl_dsch_qid_max_init(hw_mgt);
+ ret = nbl_ustore_init(hw_mgt, eth_num);
+ if (ret)
+ return ret;
+ nbl_dstore_init(hw_mgt, speed);
+ nbl_dvn_init(hw_mgt, speed);
+ nbl_uvn_init(hw_mgt);
+ nbl_uqm_init(hw_mgt);
+ return 0;
+}
+
+static void nbl_host_padpt_init(struct nbl_hw_mgt *hw_mgt)
+{
+ /* padpt flow control register */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP,
+ NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PD_DN,
+ NBL_HOST_PADPT_CFG_FC_PD_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PH_DN,
+ NBL_HOST_PADPT_CFG_FC_PH_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN,
+ NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_intf_init(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_host_padpt_init(hw_mgt);
+}
+
+static void nbl_hw_set_driver_status(struct nbl_hw_mgt *hw_mgt, bool active)
+{
+ u32 status;
+
+ spin_lock(&hw_mgt->reg_lock);
+ status = nbl_hw_rd32(hw_mgt, NBL_DRIVER_STATUS_REG);
+
+ status &= ~BIT(NBL_DRIVER_STATUS_BIT);
+ status |= FIELD_PREP(BIT(NBL_DRIVER_STATUS_BIT), active);
+
+ nbl_hw_wr32(hw_mgt, NBL_DRIVER_STATUS_REG, status);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+/*
+ * This design is intentional. Setting driver status to false is the
+ * official teardown mechanism: it notifies firmware to perform full
+ * cleanup of all per-PF hardware state, including qinfo registers.
+ * An inverse helper would duplicate work that the firmware already
+ * does, and would add error-path complexity for no benefit. We keep
+ * the deinit path minimal and rely on firmware cleanup for correctness,
+ * including abnormal hardware reset scenarios.
+ */
+static void nbl_hw_deinit_module(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_set_driver_status(hw_mgt, false);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+ /*
+ * Firmware cleanup is asynchronous: there is no cleanup-complete
+ * status register in the current hardware revision. The posted
+ * write flush above only ensures driver_status reaches the chip;
+ * firmware may still be performing per-PF state cleanup when this
+ * function returns.
+ *
+ * Caller must ensure no new DMA is initiated after this point.
+ * The mailbox channel is stopped by nbl_chan_teardown_queue()
+ * before this function is called, so no in-flight mailbox DMA
+ * remains.
+ */
+}
+
+/*
+ * Full chip hardware initialization is handled by firmware.
+ * This function only configures driver-level table entries and registers.
+ */
+static int nbl_hw_init_module(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
+ u8 eth_num)
+{
+ int ret;
+
+ ret = nbl_dp_init(hw_mgt, eth_speed, eth_num);
+ if (ret)
+ return ret;
+ nbl_intf_init(hw_mgt);
+ nbl_hw_set_driver_status(hw_mgt, true);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+
+ return 0;
+}
+
/*
* nbl_hw_set_mailbox_irq - read-modify-write NBL_MAILBOX_QINFO_MAP_REG_ARR
*
@@ -354,6 +933,9 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .init_module = nbl_hw_init_module,
+ .deinit_module = nbl_hw_deinit_module,
+
.cfg_msix_map = nbl_hw_cfg_msix_map,
.cfg_msix_info = nbl_hw_cfg_msix_info,
.flush_write = nbl_flush_writes,
@@ -404,7 +986,8 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
!hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
!hw_ops.cfg_mailbox_qinfo || !hw_ops.set_mailbox_irq ||
- !hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
+ !hw_ops.get_fw_eth_map || !hw_ops.get_board_info ||
+ !hw_ops.init_module || !hw_ops.deinit_module)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
hw_ops_tbl->priv = hw_mgt;
@@ -523,6 +1106,7 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
ret = -EIO;
goto setup_mgt_fail;
}
+
hw_mgt->mailbox_bar_size = bar_len;
spin_lock_init(&hw_mgt->reg_lock);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 5cde9f6496c2..742ffb530961 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,6 +11,9 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+#define NBL_DRIVER_STATUS_REG 0x1300444
+#define NBL_DRIVER_STATUS_BIT 16
+
/* ---------- REG BASE ADDR ---------- */
/* Interface modules base addr */
#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
@@ -70,6 +73,17 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
/* -------- HOST_PADPT -------- */
+#define NBL_HOST_PADPT_HOST_CFG_FC_PD_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000160)
+#define NBL_HOST_PADPT_HOST_CFG_FC_PH_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000164)
+#define NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN \
+ (NBL_INTF_HOST_PADPT_BASE + 0x0000016C)
+#define NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00000170)
+
+#define NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL 0x10400
+#define NBL_HOST_PADPT_CFG_FC_PD_DN_VAL 0x10080
+#define NBL_HOST_PADPT_CFG_FC_PH_DN_VAL 0x10010
+#define NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL 0x10010
/* host_padpt host_msix_info */
#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
(NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
@@ -110,6 +124,202 @@ struct nbl_function_msix_map {
u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
};
+/* ---------- DPED ---------- */
+#define NBL_DPED_VLAN_OFFSET (NBL_DP_DPED_BASE + 0x000003F4)
+#define NBL_DPED_DSCP_OFFSET_0 (NBL_DP_DPED_BASE + 0x000003F8)
+#define NBL_DPED_DSCP_OFFSET_1 (NBL_DP_DPED_BASE + 0x000003FC)
+/* DPED hw_edt_prof/ UPED hw_edt_prof */
+
+#define NBL_DPED_L4_CK_CMD_40_ADDR (NBL_DP_DPED_BASE + 0x00000338)
+
+#define NBL_DPED_L4_CK_CMD_40_EN_MASK BIT(31)
+
+/* ---------- UPED ---------- */
+/* UPED uped_hw_edt_prof */
+#define NBL_UPED_HW_EDT_PROF_TABLE(i) \
+ (NBL_DP_UPED_BASE + 0x00001000 + (i) * sizeof(u32))
+#define NBL_UPED_V4_TCP_IDX 5
+#define NBL_UPED_V6_TCP_IDX 6
+#define NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK GENMASK(3, 2)
+
+/* ---------- DSCH ---------- */
+#define NBL_DSCH_PSHA_EN_MASK GENMASK(3, 0)
+/* DSCH dsch maxqid */
+#define NBL_DSCH_HOST_QID_MAX (NBL_DP_DSCH_BASE + 0x00000118)
+#define NBL_DSCH_VN_QUANTA_ADDR (NBL_DP_DSCH_BASE + 0x00000134)
+
+#define NBL_MAX_QUEUE_ID 0x7ff
+#define NBL_HOST_QUANTA 0x8000
+#define NBL_ECPU_QUANTA 0x1000
+
+#define NBL_DSCH_VN_QUANTA_H_QUA_MASK GENMASK(15, 0)
+#define NBL_DSCH_VN_QUANTA_E_QUA_MASK GENMASK(31, 16)
+
+/* ---------- DVN ---------- */
+/* DVN dvn_queue_table */
+#define NBL_DVN_ECPU_QUEUE_NUM (NBL_DP_DVN_BASE + 0x0000041C)
+#define NBL_DVN_DESCREQ_NUM_CFG (NBL_DP_DVN_BASE + 0x00000430)
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT (NBL_DP_DVN_BASE + 0x00000480)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG (NBL_DP_DVN_BASE + 0x0000045C)
+
+#define DEFAULT_DVN_DESCREQ_NUMCFG 0x03
+#define DEFAULT_DVN_100G_DESCREQ_NUMCFG 0x07
+
+#define DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX 0x3FF
+
+/* spilit ring descreq_num 0:8,1:16 */
+#define NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK BIT(0)
+/* packet ring descreq_num
+ * 0:8,1:12,2:16;3:20,4:24,5:26;6:32,7:32
+ */
+#define NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK GENMASK(6, 4)
+
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK GENMASK(9, 0)
+
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK BIT(0)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK BIT(1)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK BIT(2)
+
+/* ---------- UVN ---------- */
+/* UVN uvn_queue_table */
+
+#define NBL_UVN_DESC_RD_WAIT (NBL_DP_UVN_BASE + 0x0000020C)
+#define NBL_UVN_QUEUE_ERR_MASK (NBL_DP_UVN_BASE + 0x00000224)
+#define NBL_UVN_ECPU_QUEUE_NUM (NBL_DP_UVN_BASE + 0x0000023C)
+#define NBL_UVN_DESC_WR_TIMEOUT (NBL_DP_UVN_BASE + 0x00000214)
+#define NBL_UVN_DIF_REQ_RO_FLAG (NBL_DP_UVN_BASE + 0x00000250)
+#define NBL_UVN_DESC_PREFETCH_INIT (NBL_DP_UVN_BASE + 0x00000204)
+#define NBL_UVN_DESC_PREFETCH_NUM 4
+
+#define NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK BIT(0)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK BIT(1)
+#define NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK BIT(2)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK BIT(3)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK GENMASK(14, 0)
+
+#define NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK BIT(5)
+
+#define NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK GENMASK(7, 0)
+#define NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK BIT(16)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_VAL 0x12c
+/* 200us = 200000ns / 1.67ns per tick = 119760 ticks */
+#define NBL_UVN_DESC_RD_WAIT_TICKS 119760
+
+/* -------- USTORE -------- */
+#define NBL_USTORE_PKT_LEN_ADDR (NBL_DP_USTORE_BASE + 0x00000108)
+#define NBL_USTORE_PORT_DROP_TH_REG_ARR(port_id) \
+ (NBL_DP_USTORE_BASE + 0x00000150 + (port_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_DROP_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002500 + (eth_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_TRUN_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002540 + (eth_id) * sizeof(u32))
+
+#define NBL_USTORE_SINGLE_ETH_DROP_TH 0xC80
+#define NBL_USTORE_DUAL_ETH_DROP_TH 0x640
+#define NBL_USTORE_QUAD_ETH_DROP_TH 0x320
+
+/* USTORE pkt_len */
+#define NBL_USTORE_PKT_LEN_MIN_MASK GENMASK(6, 0)
+
+/* USTORE port_drop_th */
+#define NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK GENMASK(11, 0)
+#define NBL_USTORE_PORT_DROP_TH_EN_MASK BIT(31)
+
+/* UQM*/
+#define NBL_UQM_QUE_TYPE (NBL_DP_UQM_BASE + 0x0000013c)
+#define NBL_UQM_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009C0)
+#define NBL_UQM_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009C4)
+#define NBL_UQM_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x000009C8)
+#define NBL_UQM_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x000009CC)
+#define NBL_UQM_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x000009D0)
+#define NBL_UQM_PORT_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009D4)
+#define NBL_UQM_PORT_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009F4)
+#define NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A14)
+#define NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A34)
+#define NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A54)
+#define NBL_UQM_FWD_DROP_CNT (NBL_DP_UQM_BASE + 0x00000A80)
+#define NBL_UQM_DPORT_DROP_CNT (NBL_DP_UQM_BASE + 0x00000B74)
+
+#define NBL_UQM_PORT_DROP_DEPTH 6
+#define NBL_UQM_DPORT_DROP_DEPTH 16
+
+/* --------- SHAPING --------- */
+
+/* Shaping rate unit: 1 = 1 Mbit/s.
+ * e.g. 100000 = 100 Gbit/s, 25000 = 25 Gbit/s.
+ */
+#define NBL_LR_LEONIS_NET_BUCKET_DEPTH 9600
+#define NBL_SHAPING_DPORT_ADDR (NBL_DP_SHAPING_BASE + 0x700)
+#define NBL_SHAPING_DPORT_DWLEN 4
+#define NBL_SHAPING_DPORT_REG(r) \
+ (NBL_SHAPING_DPORT_ADDR + (NBL_SHAPING_DPORT_DWLEN * 4) * (r))
+#define NBL_SHAPING_DVN_DPORT_ADDR (NBL_DP_SHAPING_BASE + 0x750)
+#define NBL_SHAPING_DVN_DPORT_DWLEN 4
+#define NBL_SHAPING_DVN_DPORT_REG(r) \
+ (NBL_SHAPING_DVN_DPORT_ADDR + (NBL_SHAPING_DVN_DPORT_DWLEN * 4) * (r))
+#define NBL_DSCH_PSHA_EN_ADDR (NBL_DP_DSCH_BASE + 0x00000314)
+#define NBL_SHAPING_NET_ADDR (NBL_DP_SHAPING_BASE + 0x1800)
+#define NBL_SHAPING_NET_DWLEN 4
+#define NBL_SHAPING_NET_REG(r) \
+ (NBL_SHAPING_NET_ADDR + (NBL_SHAPING_NET_DWLEN * 4) * (r))
+
+#define NBL_DPORT_VALID_MASK GENMASK_ULL(0, 0)
+#define NBL_DPORT_DEPTH_MASK GENMASK_ULL(19, 1)
+#define NBL_DPORT_CIR_MASK GENMASK_ULL(38, 20)
+#define NBL_DPORT_PIR_MASK GENMASK_ULL(57, 39)
+#define NBL_DPORT_CBS_MASK_LOW GENMASK_ULL(63, 58)
+#define NBL_DPORT_CBS_MASK_HIGH GENMASK_ULL(14, 0)
+#define NBL_DPORT_PBS_MASK GENMASK_ULL(35, 15)
+
+/* SHAPING shaping_net */
+struct nbl_shaping_net_u {
+ u32 data[NBL_SHAPING_NET_DWLEN];
+};
+
+struct nbl_shaping_dport_u {
+ u32 data[NBL_SHAPING_DPORT_DWLEN];
+};
+
+struct nbl_shaping_dvn_dport_u {
+ u32 data[NBL_SHAPING_DVN_DPORT_DWLEN];
+};
+
+/* -------- DSTORE -------- */
+#define NBL_DSTORE_D_DPORT_FC_TH_ADDR (NBL_DP_DSTORE_BASE + 0x00000600)
+#define NBL_DSTORE_D_DPORT_FC_TH_DEPTH 5
+#define NBL_DSTORE_D_DPORT_FC_TH_WIDTH 32
+#define NBL_DSTORE_D_DPORT_FC_TH_DWLEN 1
+
+#define NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK GENMASK(10, 0)
+#define NBL_DSTORE_D_DPORT_FC_XON_TH_MASK GENMASK(26, 16)
+#define NBL_DSTORE_D_DPORT_FC_FC_EN_MASK BIT(31)
+
+#define NBL_DSTORE_D_DPORT_FC_TH_REG(r) \
+ (NBL_DSTORE_D_DPORT_FC_TH_ADDR + \
+ (NBL_DSTORE_D_DPORT_FC_TH_DWLEN * 4) * (r))
+#define NBL_DSTORE_PORT_DROP_TH_ADDR (NBL_DP_DSTORE_BASE + 0x00000150)
+#define NBL_DSTORE_PORT_DROP_TH_DEPTH 6
+#define NBL_DSTORE_PORT_DROP_TH_WIDTH 32
+#define NBL_DSTORE_PORT_DROP_TH_DWLEN 1
+
+#define NBL_DSTORE_PORT_DROP_EN_MASK BIT(31)
+
+#define NBL_DSTORE_DROP_XOFF_TH 0xC8
+#define NBL_DSTORE_DROP_XON_TH 0x64
+
+#define NBL_DSTORE_DROP_XOFF_TH_100G 0x1F4
+#define NBL_DSTORE_DROP_XON_TH_100G 0x12C
+
+#define NBL_DSTORE_DISC_BP_TH (NBL_DP_DSTORE_BASE + 0x00000630)
+
+#define NBL_DSTORE_DISC_BP_TH_EN_MASK BIT(31)
+
+#define NBL_DSTORE_PORT_DROP_TH_REG(r) \
+ (NBL_DSTORE_PORT_DROP_TH_ADDR + \
+ (NBL_DSTORE_PORT_DROP_TH_DWLEN * 4) * (r))
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
@@ -123,5 +333,6 @@ struct nbl_function_msix_map {
#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+#define NBL_LEONIS_QUIRKS_OFFSET 0x00000140
#define NBL_BAR2_MAX_LEN 0x300
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index c1f10f1f6b77..aad9c5f881dc 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -13,6 +13,8 @@ static struct nbl_resource_ops res_ops = {
.cfg_msix_map = nbl_res_intr_cfg_msix_map,
.destroy_msix_map = nbl_res_intr_destroy_msix_map,
.set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
+ .init_module = nbl_res_chip_init_module,
+ .deinit_module = nbl_res_chip_deinit_module,
};
static struct nbl_resource_mgt *
@@ -46,7 +48,8 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
return ERR_PTR(-ENOMEM);
if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
!res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
- !res_ops.set_mailbox_irq)
+ !res_ops.set_mailbox_irq || !res_ops.init_module ||
+ !res_ops.deinit_module)
return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index 6eb4dc9e695a..f1cb0f23240b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -8,4 +8,5 @@
#include "../nbl_resource.h"
#include "../nbl_interrupt.h"
+#include "../nbl_chip.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index f83e6ea9d58f..f598697972f3 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -12,6 +12,9 @@ struct nbl_board_port_info;
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ int (*init_module)(struct nbl_hw_mgt *hw_mgt, u8 eth_speed, u8 eth_num);
+ void (*deinit_module)(struct nbl_hw_mgt *hw_mgt);
+
void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
bool valid, dma_addr_t dma_addr, u8 bus,
u8 devid, u8 function);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index e718ea41a816..8dc64e806c1e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,9 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*init_module)(struct nbl_resource_mgt *res_mgt);
+ void (*deinit_module)(struct nbl_resource_mgt *res_mgt);
+
int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 num_net_msix, u16 num_others_msix,
bool net_msix_mask_en);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 2c959832c32f..f3c807836d24 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -30,4 +30,25 @@ struct nbl_init_param {
struct nbl_func_caps caps;
};
+/*
+ * Firmware ABI defines port speed enum fixed, value 0 represents 10G, cannot
+ * reassign 0 to INVALID for compatibility
+ */
+enum nbl_fw_port_speed {
+ NBL_FW_PORT_SPEED_10G,
+ NBL_FW_PORT_SPEED_25G,
+ NBL_FW_PORT_SPEED_50G,
+ NBL_FW_PORT_SPEED_100G,
+};
+
+/*
+ * Firmware quirk word @ NBL_LEONIS_QUIRKS_OFFSET (0x140)
+ * Sentinel value: ~0U (0xFFFFFFFF) = firmware reports no active quirks
+ * BIT(0): NBL_QUIRK_NO_TOE – ABI defined, driver implementation pending
+ * BIT(1): NBL_QUIRK_UVN_PREFETCH_ALIGN – control UVN descriptor prefetch
+ * selection
+ */
+#define NBL_QUIRK_NO_TOE BIT(0)
+#define NBL_QUIRK_UVN_PREFETCH_ALIGN BIT(1)
+
#endif
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (5 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
` (2 subsequent siblings)
9 siblings, 0 replies; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Implement core dispatch layer infrastructure for control-plane routing:
Allocate dispatch management structure and dispatch ops table
Provide init_module/deinit_module wrapper callbacks for chip resource ops
Introduce ctrl_lvl bitmask tracking; enable MGT level only for Control PF
Regular PF NET level routing will be added in a subsequent patch.
Kerneldoc on nbl_dispatch_ops documents that init_module/deinit_module
must only be invoked on Control PF behind has_ctrl guard.
This skeleton establishes dispatch management flow. It adds forward
definitions of channel wire ABI structures and response enums required
for later channel RPC framework, without implementing message handling.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 117 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 23 ++++
.../nbl/nbl_include/nbl_def_channel.h | 41 ++++++
.../nbl/nbl_include/nbl_def_dispatch.h | 39 ++++++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 8 ++
7 files changed, 231 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index be314b909d66..b7eebd89b4d1 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -10,4 +10,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
+ nbl_core/nbl_dispatch.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index dd24ebec0171..4d8cea8d8ab3 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -17,12 +17,14 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
struct nbl_resource_ops_tbl *resource_ops_tbl;
+ struct nbl_dispatch_ops_tbl *dispatch_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
+ struct nbl_dispatch_mgt *disp_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
new file mode 100644
index 000000000000..3da5f8351fa4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dispatch.h"
+
+static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->deinit_module)
+ res_ops->deinit_module(p);
+}
+
+static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->init_module)
+ return res_ops->init_module(p);
+ return -EOPNOTSUPP;
+}
+
+static void nbl_disp_set_ctrl_bit(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
+{
+ set_bit(lvl, disp_mgt->ctrl_lvl);
+}
+
+static void nbl_disp_refresh_ctrl_ops(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
+
+ if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
+ disp_ops->init_module = nbl_disp_init_module;
+ disp_ops->deinit_module = nbl_disp_deinit_module;
+ }
+}
+
+static struct nbl_dispatch_mgt *
+nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dispatch_mgt *disp_mgt;
+ struct device *dev = common->dev;
+
+ disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
+ if (!disp_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ disp_mgt->common = common;
+ return disp_mgt;
+}
+
+static struct nbl_dispatch_ops_tbl *
+nbl_disp_setup_ops(struct device *dev, struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_dispatch_ops *disp_ops;
+
+ disp_ops_tbl = devm_kzalloc(dev, sizeof(*disp_ops_tbl), GFP_KERNEL);
+ if (!disp_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops = devm_kzalloc(dev, sizeof(*disp_ops), GFP_KERNEL);
+ if (!disp_ops)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops_tbl->ops = disp_ops;
+ disp_ops_tbl->priv = disp_mgt;
+
+ return disp_ops_tbl;
+}
+
+int nbl_disp_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_resource_ops_tbl *res_ops_tbl =
+ adapter->intf.resource_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_dispatch_mgt *disp_mgt;
+ int ret;
+
+ disp_mgt = nbl_disp_setup_disp_mgt(common);
+ if (IS_ERR(disp_mgt)) {
+ ret = PTR_ERR(disp_mgt);
+ return ret;
+ }
+
+ disp_ops_tbl = nbl_disp_setup_ops(dev, disp_mgt);
+ if (IS_ERR(disp_ops_tbl)) {
+ ret = PTR_ERR(disp_ops_tbl);
+ return ret;
+ }
+
+ disp_mgt->res_ops_tbl = res_ops_tbl;
+ disp_mgt->chan_ops_tbl = chan_ops_tbl;
+ disp_mgt->disp_ops_tbl = disp_ops_tbl;
+ adapter->core.disp_mgt = disp_mgt;
+ adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+
+ if (common->has_ctrl)
+ nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+
+ nbl_disp_refresh_ctrl_ops(disp_mgt);
+ return 0;
+}
+
+void nbl_disp_remove(struct nbl_adapter *adapter)
+{
+ /* All dispatch objects allocated via devm */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
new file mode 100644
index 000000000000..a7e5802344b4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DISPATCH_H_
+#define _NBL_DISPATCH_H_
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_dispatch_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 61dd97c779ef..58a9fa97fbf3 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -7,12 +7,19 @@
#define _NBL_DEF_CHANNEL_H_
#include <linux/types.h>
+#include <linux/build_bug.h>
struct nbl_channel_mgt;
struct nbl_adapter;
typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+enum {
+ NBL_CHAN_RESP_OK = 0,
+ NBL_CHAN_RESP_ERR = 1,
+ NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+};
+
/*
* Mailbox wire opcodes, stable wire ABI shared between driver and firmware.
* Each opcode has a fixed assigned number to preserve compatibility.
@@ -39,6 +46,32 @@ enum nbl_chan_state {
NBL_CHAN_STATE_NBITS
};
+struct nbl_chan_param_cfg_msix_map {
+ __le16 num_net_msix;
+ __le16 num_others_msix;
+ __le16 msix_mask_en;
+ __le16 rsvd;
+};
+
+struct nbl_chan_param_set_mailbox_irq {
+ __le16 vector_id;
+ u8 en_msix;
+ u8 rsvd;
+};
+
+struct nbl_chan_param_get_vsi_id {
+ __le16 vsi_id;
+ __le16 type;
+};
+
+struct nbl_chan_param_get_eth_id {
+ __le16 vsi_id;
+ u8 eth_num;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 rsvd[3];
+};
+
struct nbl_board_port_info {
u8 eth_num;
u8 eth_speed;
@@ -46,6 +79,14 @@ struct nbl_board_port_info {
u8 rsv[5];
};
+static_assert(sizeof(struct nbl_chan_param_cfg_msix_map) == 8,
+ "nbl_chan_param_cfg_msix_map size must be 8 bytes");
+static_assert(sizeof(struct nbl_chan_param_set_mailbox_irq) == 4,
+ "nbl_chan_param_set_mailbox_irq size must be 4 bytes");
+static_assert(sizeof(struct nbl_chan_param_get_vsi_id) == 4,
+ "nbl_chan_param_get_vsi_id size must be 4 bytes");
+static_assert(sizeof(struct nbl_chan_param_get_eth_id) == 8,
+ "nbl_chan_param_get_eth_id size must be 8 bytes");
static_assert(sizeof(struct nbl_board_port_info) == 8,
"nbl_board_port_info size must be 8 bytes");
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
new file mode 100644
index 000000000000..b0daabb05d39
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DISPATCH_H_
+#define _NBL_DEF_DISPATCH_H_
+
+#include <linux/types.h>
+
+struct nbl_dispatch_mgt;
+struct nbl_adapter;
+enum {
+ NBL_DISP_CTRL_LVL_MGT,
+ NBL_DISP_CTRL_LVL_MAX,
+};
+
+/**
+ * struct nbl_dispatch_ops - dispatch control plane operation callbacks
+ * @init_module: dispatch layer initialization, ONLY valid on Control PF,
+ * caller must check has_ctrl guard
+ * @deinit_module: dispatch layer cleanup, ONLY valid on Control PF,
+ * caller must check has_ctrl guard
+ * Warning: All ops except init_module/deinit_module can be safely called
+ * on PF/VF; init/deinit hooks are control-PF exclusive to prevent NULL ptr.
+ */
+struct nbl_dispatch_ops {
+ int (*init_module)(struct nbl_dispatch_mgt *disp_mgt);
+ void (*deinit_module)(struct nbl_dispatch_mgt *disp_mgt);
+};
+
+struct nbl_dispatch_ops_tbl {
+ struct nbl_dispatch_ops *ops;
+ struct nbl_dispatch_mgt *priv;
+};
+
+int nbl_disp_init(struct nbl_adapter *adapter);
+void nbl_disp_remove(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 1aafed2d46d7..5d5c0bbf418c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -11,6 +11,7 @@
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
+#include "nbl_include/nbl_def_dispatch.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -48,7 +49,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_res_init_leonis(adapter);
if (ret)
goto res_init_fail;
+
+ ret = nbl_disp_init(adapter);
+ if (ret)
+ goto disp_init_fail;
return adapter;
+disp_init_fail:
+ nbl_res_remove_leonis(adapter);
res_init_fail:
nbl_chan_remove_common(adapter);
chan_init_fail:
@@ -59,6 +66,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (6 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add bidirectional mailbox RPC for cross-PF resource coordination,
implementing request/response handlers for five operations:
configure_msix_map, destroy_msix_map, set_mailbox_irq, get_vsi_id,
get_eth_id.
Dispatch operations are resolved dynamically based on PF control
capability via nbl_disp_refresh_ctrl_ops():
- Control PF (NBL_DISP_CTRL_LVL_MGT): invokes local hardware ops
directly.
- Non-control PF with net capability (NBL_DISP_CTRL_LVL_NET):
forwards requests via mailbox RPC to the manager PF.
- Neither level set: all dispatch ops are NULL.
Introduce per-dispatch mutex ops_mutex_lock to serialize mutable
hardware operations (cfg_msix_map, destroy_msix_map, set_mailbox_irq).
This eliminates race windows between local control paths and
asynchronous remote mailbox RPC response handlers. Read-only
get_vsi_id() / get_eth_id() only access static init-time metadata
without concurrent modifications, so they require no locking.
nbl_disp_setup_msg() registers five dispatch-layer wire message
response handlers with the channel layer:
NBL_CHAN_MSG_CONFIGURE_MSIX_MAP, NBL_CHAN_MSG_DESTROY_MSIX_MAP,
NBL_CHAN_MSG_MAILBOX_SET_IRQ, NBL_CHAN_MSG_GET_VSI_ID,
NBL_CHAN_MSG_GET_ETH_ID.
Extend wire protocol with new message types and
NBL_CHAN_RESP_PERM_DENY error code.
Every responder validates:
- src_id against common->max_pf at runtime (rejects VF and
out-of-range function IDs with PERM_DENY);
- incoming payload length against the expected parameter struct size
before parsing;
- unimplemented operation via NULL function pointer check before
dispatch.
Request-side helpers translate channel wire status codes to standard
Linux errnos: OK->0, UNIMPLEMENTED->-EOPNOTSUPP, ERR->-EREMOTEIO,
PERM_DENY->-EPERM. Local transport errors (-ETIMEDOUT, -ESHUTDOWN,
-EAGAIN, -EIO, -EINVAL) are passed through unchanged via an early
"if (ret < 0) return ret" before the status switch.
get_vsi_id_req() and get_eth_id_req() verify chan_send.ack_len
matches sizeof(result) before dereferencing the response, rejecting
truncated ACKs with -EREMOTEIO to prevent adoption of zeroed
identifiers from a short reply.
destroy_msix_map responder clears the mailbox QINFO MSIX routing
(MSIX_IDX / MSIX_IDX_VALID) as Step 0 before tearing down the
MSI-X map, ensuring a recycled global vector cannot trigger an
interrupt owned by a different function.
cfg_msix_map pre-allocates all coherent DMA buffers and global
vector indices before destroying the old configuration, then swaps
the new resources into the per-function entry to avoid interrupt
loss during reconfiguration.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 552 +++++++++++++++++-
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 2 +
.../nbl/nbl_include/nbl_def_channel.h | 6 +
.../nbl/nbl_include/nbl_def_dispatch.h | 17 +
4 files changed, 576 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
index 3da5f8351fa4..5be2d0887ad4 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -3,9 +3,186 @@
* Copyright (c) 2026 Nebula Matrix Limited.
*/
#include <linux/device.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include "nbl_dispatch.h"
+static int nbl_disp_chan_get_vsi_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.type = cpu_to_le16(type);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_VSI_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ if (ret < 0)
+ return ret;
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ if (chan_send.ack_len != sizeof(result)) {
+ dev_err(disp_mgt->common->dev,
+ "get_vsi_id: short ACK, ack_len=%u expected %zu\n",
+ chan_send.ack_len, sizeof(result));
+ return -EREMOTEIO;
+ }
+ *vsi_id = le16_to_cpu(result.vsi_id);
+ return 0;
+}
+
+static void nbl_disp_chan_get_vsi_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ u16 vsi_id = 0;
+ int ret;
+
+ if (src_id >= disp_mgt->common->max_pf) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->get_vsi_id) {
+ ret = res_ops->get_vsi_id(p, src_id, le16_to_cpu(param.type),
+ &vsi_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ result.vsi_id = cpu_to_le16(vsi_id);
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_VSI_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_VSI_ID);
+}
+
+static int nbl_disp_chan_get_eth_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.vsi_id = cpu_to_le16(vsi_id);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_ETH_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ if (ret < 0)
+ return ret;
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ if (chan_send.ack_len != sizeof(result)) {
+ dev_err(disp_mgt->common->dev,
+ "get_eth_id: short ACK, ack_len=%u expected %zu\n",
+ chan_send.ack_len, sizeof(result));
+ return -EREMOTEIO;
+ }
+ *eth_num = result.eth_num;
+ *eth_id = result.eth_id;
+ *logic_eth_id = result.logic_eth_id;
+
+ return 0;
+}
+
+static void nbl_disp_chan_get_eth_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id >= disp_mgt->common->max_pf) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->get_eth_id) {
+ ret = res_ops->get_eth_id(p, src_id, le16_to_cpu(param.vsi_id),
+ &result.eth_num, &result.eth_id,
+ &result.logic_eth_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_ETH_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_ETH_ID);
+}
+
static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
{
struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
@@ -25,6 +202,352 @@ static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
return -EOPNOTSUPP;
}
+static int nbl_disp_cfg_msix_map(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->cfg_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, common->mgt_pf, num_net_msix,
+ num_others_msix, net_msix_mask_en);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int
+nbl_disp_chan_cfg_msix_map_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.num_net_msix = cpu_to_le16(num_net_msix);
+ param.num_others_msix = cpu_to_le16(num_others_msix);
+ param.msix_mask_en = cpu_to_le16(!!net_msix_mask_en);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ ¶m, sizeof(param),
+ NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ if (ret < 0)
+ return ret;
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_cfg_msix_map_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id >= disp_mgt->common->max_pf) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+
+ if (res_ops->cfg_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, src_id,
+ le16_to_cpu(param.num_net_msix),
+ le16_to_cpu(param.num_others_msix),
+ !!le16_to_cpu(param.msix_mask_en));
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP);
+}
+
+static int nbl_disp_chan_destroy_msix_map_req(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ NULL, 0, NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ if (ret < 0)
+ return ret;
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_destroy_msix_map_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id >= disp_mgt->common->max_pf) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (res_ops->destroy_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, src_id);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_DESTROY_MSIX_MAP);
+}
+
+static int nbl_disp_chan_set_mailbox_irq_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send = {0};
+ int ret;
+
+ param.vector_id = cpu_to_le16(vector_id);
+ param.en_msix = !!en_msix;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ ¶m, sizeof(param), NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ if (ret < 0)
+ return ret;
+ switch (ret) {
+ case NBL_CHAN_RESP_OK:
+ break;
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ default:
+ return -EREMOTEIO;
+ }
+ return 0;
+}
+
+static void nbl_disp_chan_set_mailbox_irq_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ bool en_msix;
+ u16 vector_id;
+ int ret;
+
+ if (src_id >= disp_mgt->common->max_pf) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len < sizeof(param)) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ memcpy(¶m, data, sizeof(param));
+ vector_id = le16_to_cpu(param.vector_id);
+ en_msix = !!param.en_msix;
+
+ if (res_ops->set_mailbox_irq) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, src_id, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_MAILBOX_SET_IRQ);
+}
+
+static int nbl_disp_destroy_msix_map(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->destroy_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, common->mgt_pf);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_set_mailbox_irq(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->set_mailbox_irq)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, common->mgt_pf, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_get_vsi_id(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_vsi_id)
+ return res_ops->get_vsi_id(p, common->mgt_pf, type, vsi_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_get_eth_id(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_eth_id)
+ return res_ops->get_eth_id(p, common->mgt_pf, vsi_id,
+ eth_num, eth_id, logic_eth_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_setup_msg(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *p = disp_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+ int _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ nbl_disp_chan_cfg_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ nbl_disp_chan_destroy_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ nbl_disp_chan_set_mailbox_irq_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_VSI_ID,
+ nbl_disp_chan_get_vsi_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_ETH_ID,
+ nbl_disp_chan_get_eth_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ if (ret)
+ chan_ops->unregister_all_msg(p);
+ return ret;
+}
+
static void nbl_disp_set_ctrl_bit(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
{
set_bit(lvl, disp_mgt->ctrl_lvl);
@@ -34,9 +557,22 @@ static void nbl_disp_refresh_ctrl_ops(struct nbl_dispatch_mgt *disp_mgt)
{
struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
+ memset(disp_ops, 0, sizeof(*disp_ops));
if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
disp_ops->init_module = nbl_disp_init_module;
disp_ops->deinit_module = nbl_disp_deinit_module;
+ disp_ops->cfg_msix_map = nbl_disp_cfg_msix_map;
+ disp_ops->destroy_msix_map = nbl_disp_destroy_msix_map;
+ disp_ops->set_mailbox_irq = nbl_disp_set_mailbox_irq;
+ disp_ops->get_vsi_id = nbl_disp_get_vsi_id;
+ disp_ops->get_eth_id = nbl_disp_get_eth_id;
+ } else if (test_bit(NBL_DISP_CTRL_LVL_NET, disp_mgt->ctrl_lvl)) {
+ disp_ops->cfg_msix_map =
+ nbl_disp_chan_cfg_msix_map_req;
+ disp_ops->destroy_msix_map = nbl_disp_chan_destroy_msix_map_req;
+ disp_ops->set_mailbox_irq = nbl_disp_chan_set_mailbox_irq_req;
+ disp_ops->get_vsi_id = nbl_disp_chan_get_vsi_id_req;
+ disp_ops->get_eth_id = nbl_disp_chan_get_eth_id_req;
}
}
@@ -45,12 +581,16 @@ nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
{
struct nbl_dispatch_mgt *disp_mgt;
struct device *dev = common->dev;
+ int err;
disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
if (!disp_mgt)
return ERR_PTR(-ENOMEM);
disp_mgt->common = common;
+ err = devm_mutex_init(common->dev, &disp_mgt->ops_mutex_lock);
+ if (err)
+ return ERR_PTR(err);
return disp_mgt;
}
@@ -104,14 +644,24 @@ int nbl_disp_init(struct nbl_adapter *adapter)
adapter->core.disp_mgt = disp_mgt;
adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+ ret = nbl_disp_setup_msg(disp_mgt);
+ if (ret)
+ return ret;
+
if (common->has_ctrl)
nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+ if (common->has_net)
+ nbl_disp_set_ctrl_bit(disp_mgt, NBL_DISP_CTRL_LVL_NET);
nbl_disp_refresh_ctrl_ops(disp_mgt);
return 0;
}
void nbl_disp_remove(struct nbl_adapter *adapter)
{
- /* All dispatch objects allocated via devm */
+ /*
+ * All dispatch objects allocated via devm
+ * All message handlers will be cleaned up inside channel layer
+ * nbl_chan_remove_common() at final device tear-down
+ */
}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
index a7e5802344b4..8549048f76e9 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -18,6 +18,8 @@ struct nbl_dispatch_mgt {
struct nbl_channel_ops_tbl *chan_ops_tbl;
struct nbl_dispatch_ops_tbl *disp_ops_tbl;
DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+ /* use for the caller not in interrupt */
+ struct mutex ops_mutex_lock;
};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 58a9fa97fbf3..6f8fc14a51f0 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -18,6 +18,7 @@ enum {
NBL_CHAN_RESP_OK = 0,
NBL_CHAN_RESP_ERR = 1,
NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+ NBL_CHAN_RESP_PERM_DENY = 3,
};
/*
@@ -37,6 +38,11 @@ enum {
*/
enum nbl_chan_msg_type {
NBL_CHAN_MSG_ACK = 0,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP = 17,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP = 18,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ = 19,
+ NBL_CHAN_MSG_GET_VSI_ID = 21,
+ NBL_CHAN_MSG_GET_ETH_ID = 67,
/* mailbox msg end */
NBL_CHAN_MSG_MAILBOX_MAX,
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
index b0daabb05d39..61083a750da4 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
@@ -12,6 +12,7 @@ struct nbl_dispatch_mgt;
struct nbl_adapter;
enum {
NBL_DISP_CTRL_LVL_MGT,
+ NBL_DISP_CTRL_LVL_NET,
NBL_DISP_CTRL_LVL_MAX,
};
@@ -21,12 +22,28 @@ enum {
* caller must check has_ctrl guard
* @deinit_module: dispatch layer cleanup, ONLY valid on Control PF,
* caller must check has_ctrl guard
+ * @cfg_msix_map: configure function msix mapping table
+ * @destroy_msix_map: tear down msix mapping resource
+ * @set_mailbox_irq: bind mailbox interrupt to specified msix vector
+ * @get_vsi_id: resolve VSI ID by type
+ * @get_eth_id: resolve eth port info from VSI ID
+ *
* Warning: All ops except init_module/deinit_module can be safely called
* on PF/VF; init/deinit hooks are control-PF exclusive to prevent NULL ptr.
*/
struct nbl_dispatch_ops {
int (*init_module)(struct nbl_dispatch_mgt *disp_mgt);
void (*deinit_module)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*cfg_msix_map)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*set_mailbox_irq)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix);
+ int (*get_vsi_id)(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
};
struct nbl_dispatch_ops_tbl {
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (7 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add nbl_dev core infrastructure. Introduce
nbl_dev_setup_common_dev() and nbl_dev_setup_ctrl_dev() initialization
helpers with their teardown counterparts, and wire them into
nbl_dev_init() / nbl_dev_remove() from nbl_core_init() /
nbl_core_remove().
Common device setup (nbl_dev_setup_common_dev):
- Allocate per-device state (nbl_dev_common)
- Set up mailbox channel queue and register cleanup work
- Initialize MSI-X service counts (mailbox = 1 vector; actual
allocation deferred to nbl_dev_start() in the next patch)
- On the control PF only: fetch VSI/ETH identity via local
dispatch ops. Non-control PFs defer this to nbl_dev_start(),
when the mailbox channel is fully operational and the control
PF's responder is running; this keeps every intermediate commit
bisect-bootable on multi-PF systems.
Control device setup (nbl_dev_setup_ctrl_dev, control PF only):
- disp_ops->init_module(): chip-wide hardware init (datapath
pipeline, driver_status flag)
- Program mailbox QINFO routing map (bus/devid for each PF)
Chip core hardware initialization is handled by firmware at
power-on. The driver only configures functional table entries
after hardware becomes ready, so common_dev setup before ctrl_dev
setup is safe.
Deinit order: ctrl dev first, then common dev.
nbl_dev_remove_ctrl_dev() calls deinit_module(), which sets
driver_status=false to notify firmware to asynchronously clean
all per-PF hardware state including qinfo registers. The qinfo
map is not explicitly cleared by the driver; firmware handles it
on driver_status change. No completion handshake exists in the
current hardware revision; the caller must ensure no new DMA is
initiated after deinit (the mailbox channel is stopped first).
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 1 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 233 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 55 +++++
.../nbl/nbl_include/nbl_def_dev.h | 14 ++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
6 files changed, 313 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index b7eebd89b4d1..71fbe3ee7e62 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -11,4 +11,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
nbl_core/nbl_dispatch.o \
+ nbl_core/nbl_dev.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 4d8cea8d8ab3..c3c4dd685bf6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -25,6 +25,7 @@ struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
struct nbl_dispatch_mgt *disp_mgt;
+ struct nbl_dev_mgt *dev_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
new file mode 100644
index 000000000000..7813ff31a3cf
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dev.h"
+
+static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+
+ /* mailbox vector allocated in nbl_dev_start() via
+ * nbl_dev_request_mailbox_irq()
+ */
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
+}
+
+/* ---------- Channel config ---------- */
+static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ if (!chan_ops->check_queue_exist(priv, chan_type))
+ return;
+
+ chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
+}
+
+static int nbl_dev_setup_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->setup_queue(priv, chan_type);
+
+ return ret;
+}
+
+static int nbl_dev_remove_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->teardown_queue(priv, chan_type);
+
+ return ret;
+}
+
+static void nbl_dev_register_chan_task(struct nbl_dev_mgt *dev_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ if (chan_ops->check_queue_exist(dev_mgt->chan_ops_tbl->priv, chan_type))
+ chan_ops->register_chan_task(dev_mgt->chan_ops_tbl->priv,
+ chan_type, task);
+}
+
+/* ---------- Tasks config ---------- */
+static void nbl_dev_clean_mailbox_task(struct work_struct *work)
+{
+ struct nbl_dev_common *common_dev =
+ container_of(work, struct nbl_dev_common, clean_mbx_task);
+ struct nbl_dev_mgt *dev_mgt = common_dev->dev_mgt;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ chan_ops->clean_queue_subtask(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_TYPE_MAILBOX);
+}
+
+/* ---------- Dev init process ---------- */
+static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dispatch_mgt *priv = dev_mgt->disp_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+ struct nbl_dev_common *common_dev;
+ int ret;
+
+ common_dev = devm_kzalloc(&adapter->pdev->dev, sizeof(*common_dev),
+ GFP_KERNEL);
+ if (!common_dev)
+ return -ENOMEM;
+ common_dev->dev_mgt = dev_mgt;
+
+ ret = nbl_dev_setup_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
+ &common_dev->clean_mbx_task);
+ if (common->has_ctrl) {
+ ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
+ if (ret)
+ goto err_cleanup;
+ ret = disp_ops->get_eth_id(priv, common->vsi_id,
+ &common->eth_num,
+ &common->eth_id,
+ &common->logic_eth_id);
+ if (ret)
+ goto err_cleanup;
+ }
+
+ dev_mgt->common_dev = common_dev;
+ nbl_dev_init_msix_cnt(dev_mgt);
+
+ return 0;
+err_cleanup:
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+ return ret;
+}
+
+static void nbl_dev_remove_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+
+ if (!common_dev)
+ return;
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+}
+
+static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ int ret;
+
+ ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
+ if (ret)
+ return ret;
+
+ nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+
+ return 0;
+}
+
+/*
+ * Tear down control device: deinit_module sets driver_status=false
+ * to notify firmware to clean all per-PF hardware state (including
+ * qinfo registers). The qinfo map programmed in setup_ctrl_dev is
+ * not explicitly cleared; firmware handles it on driver_status change.
+ */
+static void nbl_dev_remove_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ disp_ops->deinit_module(dev_mgt->disp_ops_tbl->priv);
+}
+
+static struct nbl_dev_mgt *nbl_dev_setup_dev_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dev_mgt *dev_mgt;
+
+ dev_mgt = devm_kzalloc(common->dev, sizeof(*dev_mgt), GFP_KERNEL);
+ if (!dev_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ dev_mgt->common = common;
+ return dev_mgt;
+}
+
+int nbl_dev_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl =
+ adapter->intf.dispatch_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct nbl_dev_mgt *dev_mgt;
+ int ret;
+
+ dev_mgt = nbl_dev_setup_dev_mgt(common);
+ if (IS_ERR(dev_mgt)) {
+ ret = PTR_ERR(dev_mgt);
+ return ret;
+ }
+
+ dev_mgt->disp_ops_tbl = disp_ops_tbl;
+ dev_mgt->chan_ops_tbl = chan_ops_tbl;
+ adapter->core.dev_mgt = dev_mgt;
+
+ /*
+ * Chip hardware initialization is completed by firmware at power-up.
+ * Only driver functional table/register config follows here, safe to
+ * access hardware registers before ctrl dev setup.
+ */
+ ret = nbl_dev_setup_common_dev(adapter);
+ if (ret)
+ goto setup_err;
+
+ if (common->has_ctrl) {
+ ret = nbl_dev_setup_ctrl_dev(adapter);
+ if (ret)
+ goto setup_ctrl_dev_fail;
+ }
+
+ return 0;
+setup_ctrl_dev_fail:
+ nbl_dev_remove_common_dev(adapter);
+setup_err:
+ return ret;
+}
+
+/*
+ * Teardown order: ctrl dev first, then common dev.
+ * nbl_dev_remove_ctrl_dev() notifies firmware to clean all per-PF state
+ * (including qinfo registers), so subsequent common dev queue cleanup
+ * will not trigger PCIe master abort or invalid register access.
+ */
+void nbl_dev_remove(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+
+ if (common->has_ctrl)
+ nbl_dev_remove_ctrl_dev(adapter);
+ nbl_dev_remove_common_dev(adapter);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
new file mode 100644
index 000000000000..24e890fd8987
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEV_H_
+#define _NBL_DEV_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_dev.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_STRING_NAME_LEN 32
+
+enum nbl_msix_serv_type {
+ NBL_MSIX_NET_TYPE,
+ NBL_MSIX_MAILBOX_TYPE,
+ NBL_MSIX_TYPE_MAX
+};
+
+struct nbl_msix_serv_info {
+ char irq_name[NBL_STRING_NAME_LEN];
+ u16 num;
+ u16 base_vector_id;
+ /* true: hw report msix, hw need to mask actively */
+ bool hw_self_mask_en;
+};
+
+struct nbl_msix_info {
+ struct nbl_msix_serv_info serv_info[NBL_MSIX_TYPE_MAX];
+};
+
+struct nbl_dev_common {
+ struct nbl_dev_mgt *dev_mgt;
+ struct nbl_msix_info msix_info;
+ char mailbox_name[NBL_STRING_NAME_LEN];
+ /* for ctrl-dev/net-dev mailbox recv msg */
+ struct work_struct clean_mbx_task;
+};
+
+struct nbl_dev_mgt {
+ struct nbl_common_info *common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dev_common *common_dev;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
new file mode 100644
index 000000000000..51cf04e4c552
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2026 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DEV_H_
+#define _NBL_DEF_DEV_H_
+
+struct nbl_adapter;
+
+int nbl_dev_init(struct nbl_adapter *adapter);
+void nbl_dev_remove(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 5d5c0bbf418c..9896c1b49be2 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -12,6 +12,7 @@
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_dispatch.h"
+#include "nbl_include/nbl_def_dev.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -53,7 +54,14 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_disp_init(adapter);
if (ret)
goto disp_init_fail;
+
+ ret = nbl_dev_init(adapter);
+ if (ret)
+ goto dev_init_fail;
return adapter;
+
+dev_init_fail:
+ nbl_disp_remove(adapter);
disp_init_fail:
nbl_res_remove_leonis(adapter);
res_init_fail:
@@ -66,6 +74,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_dev_remove(adapter);
nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
` (8 preceding siblings ...)
2026-09-07 12:38 ` [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-09-07 12:38 ` illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
9 siblings, 1 reply; 20+ messages in thread
From: illusion.wang @ 2026-09-07 12:38 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Implement nbl_dev_start() and nbl_dev_stop() to manage MSI-X
mapping, mailbox interrupt initialization and deinitialization.
nbl_dev_start() performs device startup in strict order:
1. Configure hardware MSI-X mapping table (cfg_msix_map)
2. Allocate MSI-X IRQ vectors via pci_alloc_irq_vectors()
3. Request mailbox interrupt and attach handler
(nbl_dev_clean_mailbox -> schedule cleanup work)
4. Enable hardware mailbox interrupt and mark channel
interrupt ready (NBL_CHAN_IRQ_RDY)
On non-control PFs, VSI/ETH identity is fetched via mailbox RPC
in nbl_dev_setup_common_dev() during probe, before start runs.
The control PF's responder is reachable once its own probe has
completed and the mailbox queue is operational.
nbl_dev_stop() tears down resources in the reverse order, with
the device-side mapping destroyed before kernel-side vectors are
released:
1. Switch channel to polling mode (clear NBL_CHAN_IRQ_RDY)
before masking hardware mailbox interrupt. This ordering
avoids discarding in-flight ACKs that would trigger
"Channel waiting ack failed" errors.
2. Release mailbox IRQ via free_irq()
3. Destroy hardware MSI-X mapping table (masks all device
vectors and clears the pcompleter map entry)
4. Free kernel-side MSI-X vectors via pci_free_irq_vectors()
5. Drain mailbox cleanup work via cancel_work_sync()
The start error-unwind path uses the same reverse ordering
(err_disable_irq -> err_destroy_map -> err_cancel_work), so a
failure at any stage leaves the device and kernel in a consistent
state. Rollback failures are logged; pci_clear_master() in
nbl_remove() provides the final safety net if a teardown RPC to
the manager PF cannot complete.
Add thin wrappers nbl_core_start() / nbl_core_stop() and hook
them into PCI probe and remove paths to control the device
lifecycle.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 318 +++++++++++++++++-
.../nbl/nbl_include/nbl_def_dev.h | 2 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 18 +
4 files changed, 329 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index c3c4dd685bf6..56872c8ca9a7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -39,5 +39,7 @@ struct nbl_adapter {
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param);
void nbl_core_remove(struct nbl_adapter *adapter);
+int nbl_core_start(struct nbl_adapter *adapter);
+void nbl_core_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
index 7813ff31a3cf..af003a240e7b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -6,6 +6,17 @@
#include <linux/pci.h>
#include "nbl_dev.h"
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt);
+
+/* ---------- Interrupt config ---------- */
+static irqreturn_t nbl_dev_clean_mailbox(int __always_unused irq, void *data)
+{
+ struct nbl_dev_mgt *dev_mgt = (struct nbl_dev_mgt *)data;
+
+ nbl_dev_clean_mailbox_schedule(dev_mgt);
+ return IRQ_HANDLED;
+}
+
static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
{
struct nbl_dev_common *dev_common = dev_mgt->common_dev;
@@ -17,6 +28,189 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
}
+static int nbl_dev_request_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+ int err;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num < 0) {
+ dev_err(common->dev, "Failed to get mailbox IRQ vector: %d\n",
+ irq_num);
+ return irq_num;
+ }
+
+ snprintf(dev_common->mailbox_name, sizeof(dev_common->mailbox_name),
+ "nbl_mailbox@pci:%s", pci_name(common->pdev));
+ err = request_irq(irq_num, nbl_dev_clean_mailbox, 0,
+ dev_common->mailbox_name, dev_mgt);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static void nbl_dev_free_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num >= 0)
+ free_irq(irq_num, dev_mgt);
+}
+
+static int nbl_dev_enable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_TYPE_MAILBOX, true);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, true);
+}
+
+static int nbl_dev_disable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ /*
+ * Disable sequence invariant: update software state first, then mask
+ * hardware interrupt. Must not reverse the order.
+ *
+ * If hardware interrupt is masked before clearing INTERRUPT_READY,
+ * the hardware may still transmit outstanding ACK packets for in-flight
+ * messages. Subsequent switch to polling mode discards pending ACK
+ * processing, triggering "Channel waiting ack failed" and "Skip ack
+ * with invalid status" errors.
+ *
+ * By entering polling mode first, any late hardware interrupts are
+ * ignored without pending ACK expectations, then hardware interrupt
+ * can be safely disabled.
+ *
+ * This helper is invoked in two paths:
+ * 1. Error unwind path of nbl_dev_start(): followed immediately by
+ * nbl_dev_free_mailbox_irq() and full channel teardown. No new mailbox
+ * interrupts can fire afterwards, and subsequent cancel_work_sync()
+ * drains pending cleanup work before resources are released.
+ * 2. Normal device stop path nbl_dev_stop(): free_irq() blocks until
+ * any in-flight hardirq handler completes and prevents new interrupts.
+ * cancel_work_sync() then waits for any already running mailbox cleanup
+ * work to finish, or cancels queued but unstarted work items before
+ * final channel destruction. No stuck descriptors linger in either
+ * scenario.
+ */
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_IRQ_RDY,
+ NBL_CHAN_TYPE_MAILBOX, false);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, false);
+}
+
+static int nbl_dev_cfg_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ bool mask_en = msix_info->serv_info[NBL_MSIX_NET_TYPE].hw_self_mask_en;
+ u16 msix_net_num = msix_info->serv_info[NBL_MSIX_NET_TYPE].num;
+ u16 msix_not_net_num = 0;
+ int err, i;
+
+ msix_info->serv_info[NBL_MSIX_NET_TYPE].base_vector_id = 0;
+ for (i = NBL_MSIX_NET_TYPE + 1; i < NBL_MSIX_TYPE_MAX; i++)
+ msix_info->serv_info[i].base_vector_id =
+ msix_info->serv_info[i - 1].base_vector_id +
+ msix_info->serv_info[i - 1].num;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++) {
+ if (i == NBL_MSIX_NET_TYPE)
+ continue;
+ msix_not_net_num += msix_info->serv_info[i].num;
+ }
+
+ err = disp_ops->cfg_msix_map(dev_mgt->disp_ops_tbl->priv,
+ msix_net_num, msix_not_net_num,
+ mask_en);
+
+ return err;
+}
+
+static int nbl_dev_destroy_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ return disp_ops->destroy_msix_map(dev_mgt->disp_ops_tbl->priv);
+}
+
+static int nbl_dev_init_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ int needed = 0;
+ int err;
+ int i;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++)
+ needed += msix_info->serv_info[i].num;
+
+ err = pci_alloc_irq_vectors(common->pdev, needed, needed,
+ PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
+ if (err < 0) {
+ dev_err(common->dev,
+ "pci_alloc_irq_vectors failed, err = %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static void nbl_dev_clear_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_common_info *common = dev_mgt->common;
+
+ pci_free_irq_vectors(common->pdev);
+}
+
/* ---------- Channel config ---------- */
static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
{
@@ -76,6 +270,14 @@ static void nbl_dev_clean_mailbox_task(struct work_struct *work)
NBL_CHAN_TYPE_MAILBOX);
}
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ queue_work(common->wq, &common_dev->clean_mbx_task);
+}
+
/* ---------- Dev init process ---------- */
static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
{
@@ -99,17 +301,18 @@ static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
&common_dev->clean_mbx_task);
- if (common->has_ctrl) {
- ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
- if (ret)
- goto err_cleanup;
- ret = disp_ops->get_eth_id(priv, common->vsi_id,
- &common->eth_num,
- &common->eth_id,
- &common->logic_eth_id);
- if (ret)
- goto err_cleanup;
- }
+ /*
+ * Even if has_ctrl=false (no dedicated control PF channel), we fetch
+ * VSI/ETH info via regular mailbox message instead of dedicated
+ * control command.
+ */
+ ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
+ if (ret)
+ goto err_cleanup;
+ ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
+ &common->eth_id, &common->logic_eth_id);
+ if (ret)
+ goto err_cleanup;
dev_mgt->common_dev = common_dev;
nbl_dev_init_msix_cnt(dev_mgt);
@@ -231,3 +434,96 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
nbl_dev_remove_ctrl_dev(adapter);
nbl_dev_remove_common_dev(adapter);
}
+
+/* ---------- Dev start process ---------- */
+int nbl_dev_start(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int cleanup_ret;
+ int ret;
+
+ ret = nbl_dev_cfg_msix_map(dev_mgt);
+ if (ret)
+ goto err_cancel_work;
+
+ ret = nbl_dev_init_interrupt_scheme(dev_mgt);
+ if (ret)
+ goto err_destroy_map;
+
+ ret = nbl_dev_request_mailbox_irq(dev_mgt);
+ if (ret)
+ goto err_destroy_map;
+
+ ret = nbl_dev_enable_mailbox_irq(dev_mgt);
+ if (ret)
+ goto err_disable_irq;
+
+ return 0;
+
+err_disable_irq:
+ cleanup_ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "rollback: disable mailbox IRQ failed: %d\n",
+ cleanup_ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+err_destroy_map:
+ /*
+ * Destroy device-side MSI-X map BEFORE releasing kernel-side
+ * vectors. This masks all hardware vectors and clears the
+ * pcompleter map entry, so no MSI-X message can fire after
+ * pci_free_irq_vectors() releases the vectors.
+ *
+ * For non-control PFs this is a polling-mode mailbox RPC
+ * (IRQ_RDY already cleared by disable above, or never set).
+ * If the RPC fails the device may remain armed; pci_clear_master()
+ * in nbl_remove() is the final safety net.
+ */
+ cleanup_ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "rollback: destroy MSI-X map failed: %d\n",
+ cleanup_ret);
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+err_cancel_work:
+ /*
+ * destroy_msix_map() polling send may requeue clean_mbx_task.
+ * Drain before returning on all rollback paths.
+ */
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ return ret;
+}
+
+void nbl_dev_stop(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int ret;
+
+ ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to disable mailbox IRQ: %d\n", ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+
+ /*
+ * Destroy hardware MSI-X map BEFORE releasing kernel-side
+ * vectors. Masks all device vectors and clears pcompleter
+ * map entry first; pci_free_irq_vectors() then runs with no
+ * possibility of a stale MSI-X message arriving.
+ */
+ ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to destroy MSI-X map: %d\n", ret);
+
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+
+ /*
+ * destroy_msix_map() sends ack-requested messages which may
+ * requeue clean_mbx_task via polling send path. Drain work
+ * after the operation.
+ */
+ cancel_work_sync(&common_dev->clean_mbx_task);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
index 51cf04e4c552..a66c633a0e7a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -10,5 +10,7 @@ struct nbl_adapter;
int nbl_dev_init(struct nbl_adapter *adapter);
void nbl_dev_remove(struct nbl_adapter *adapter);
+int nbl_dev_start(struct nbl_adapter *adapter);
+void nbl_dev_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 9896c1b49be2..02cb35d32012 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -16,6 +16,16 @@
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+int nbl_core_start(struct nbl_adapter *adapter)
+{
+ return nbl_dev_start(adapter);
+}
+
+void nbl_core_stop(struct nbl_adapter *adapter)
+{
+ nbl_dev_stop(adapter);
+}
+
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param)
{
@@ -127,7 +137,13 @@ static int nbl_probe(struct pci_dev *pdev,
goto adapter_init_err;
}
pci_set_drvdata(pdev, adapter);
+ err = nbl_core_start(adapter);
+ if (err)
+ goto core_start_err;
+
return 0;
+core_start_err:
+ nbl_core_remove(adapter);
adapter_init_err:
pci_clear_master(pdev);
return err;
@@ -140,6 +156,8 @@ static void nbl_remove(struct pci_dev *pdev)
if (!adapter)
return;
pci_set_drvdata(pdev, NULL);
+
+ nbl_core_stop(adapter);
nbl_core_remove(adapter);
pci_clear_master(pdev);
--
2.47.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework
2026-09-07 12:38 ` [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 0 · Low: 1
- [Low] Documentation/Kconfig prose added by this patch describes
functionality that does not exist at this commit:…
--
net/nebula-matrix: add minimum nbl build framework
This patch adds the Kconfig, Makefile and MAINTAINERS entries for a new
nebula-matrix ethernet vendor directory, a documentation file for the nbl
driver, and a PCI driver skeleton whose probe and remove callbacks are
stubs.
> diff --git a/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
> new file mode 100644
> index 0000000000000..5540291cf3766
> --- /dev/null
> +++ b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
> @@ -0,0 +1,28 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +================================================================
> +Linux Base Driver for Nebula-matrix m18110-NIC/m18000-NIC family
> +================================================================
> +
> +Overview:
> +=========
> +The m18110-NIC/m18000-NIC (marketed as SNIC s1000) is a series
> +of network interface cards for the Data Center Area.
> +
> +This driver provides the core infrastructure for m18110/m18000-NIC
> +devices, including:
> +
> +- PCI device enumeration and resource (BAR) management
> +- Firmware command interface via PF mailbox
> +- Channel-based communication between driver and firmware
> +- Device initialization and teardown
[Severity: Low]
This isn't a bug, but does the documentation added here describe code that
exists at this commit? The only C file added is nbl_main.c, where
nbl_probe() returns -ENODEV and nbl_remove() is empty, so there is no BAR
management, no PF mailbox command interface, no channel communication and no
device init/teardown yet.
The same wording appears in the CONFIG_NBL help text below.
All four items are implemented by the end of the series (nbl_probe() ends up
calling nbl_core_init(), which chains nbl_hw_init_leonis(),
nbl_chan_init_common(), nbl_res_init_leonis(), nbl_disp_init() and
nbl_dev_init()), and the .rst text is not touched again, so this is only a
skew within the series.
Would it be preferable to word the first patch's doc and help text to match
the stub state, or to add this prose together with the code it describes?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/Kconfig b/drivers/net/ethernet/nebula-matrix/Kconfig
> new file mode 100644
> index 0000000000000..dd0e91d14131f
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/Kconfig
> @@ -0,0 +1,32 @@
[ ... ]
> +config NBL
> + tristate "Nebula-matrix Ethernet Controller m18110/m18000 support"
> + depends on PCI && (64BIT || COMPILE_TEST) && !CPU_BIG_ENDIAN
> + help
> + This driver provides the core infrastructure for Nebula-matrix
> + Ethernet Controller m18110/m18000 Family of devices, including
> + PCI enumeration, firmware mailbox and channel communication.
This is the menuconfig text mentioned above; at this commit selecting
CONFIG_NBL=m builds a module that only registers a pci_driver with an
always-failing probe.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> new file mode 100644
> index 0000000000000..199626159a4ce
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> @@ -0,0 +1,100 @@
[ ... ]
> +static int nbl_probe(struct pci_dev *pdev,
> + const struct pci_device_id *id)
> +{
> + return -ENODEV;
> +}
> +
> +static void nbl_remove(struct pci_dev *pdev)
> +{
> +}
[ ... ]
> +static const struct pci_device_id nbl_id_table[] = {
> + { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110),
> + .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
[ ... ]
> + /* required as sentinel */
> + { }
> +};
> +MODULE_DEVICE_TABLE(pci, nbl_id_table);
Since MODULE_DEVICE_TABLE() publishes all 16 IDs in the 0x1F0F:0x3403-0x3412
range, udev will autoload nbl on matching hardware at this commit and the
device stays unbound because of the -ENODEV above. Is that the intended
intermediate state for a bisect point?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-09-07 12:38 ` [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 0 · Low: 2
- [Low] Series-structure remark (validated as NOT a defect): nbl_probe()
changes from an unconditional -ENODEV to a success path that enables…
- [Low] Dead code introduced by this patch: (a) nbl_hw_leonis.c includes
<linux/spinlock.h> and <linux/bitfield.h> although the file at this…
--
net/nebula-matrix: add core driver architecture and HW layer initialization
This adds the nbl_adapter/nbl_core/nbl_common_info device context, turns
the previously stubbed nbl_probe() into a working PCI probe path, and adds
the Leonis hardware layer that requests BAR0/BAR2 and ioremaps them via
pcim helpers.
A couple of small questions below about unused code added by this patch and
about the point in the series where the device starts being claimed.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index 6c14d1071c0c2..cc060cf8bf759 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -3,4 +3,5 @@
>
> obj-$(CONFIG_NBL) := nbl.o
>
> -nbl-objs += nbl_main.o
> +nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
> + nbl_main.o
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> new file mode 100644
> index 0000000000000..0ba0b7f643c64
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -0,0 +1,154 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2026 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include <linux/bits.h>
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +#include <linux/bitfield.h>
> +#include "nbl_hw_leonis.h"
[Severity: Low]
This isn't a bug, but are linux/spinlock.h and linux/bitfield.h needed here
yet? Grepping nbl_hw_leonis.c at this commit for spin_lock or FIELD_ returns
no matches, so both look like leftovers from splitting a larger file (the
spinlock and FIELD_*() users appear only in later patches). Could they be
added in the patch that first uses them?
[ ... ]
> + hw_mgt->hw_size = PAGE_ALIGN_DOWN(NBL_MEM_BAR_TOTAL_SIZE -
> + NBL_RDMA_NOTIFY_LEN);
> + hw_mgt->hw_addr =
> + pcim_iomap(pdev, NBL_MEMORY_BAR,
> + hw_mgt->hw_size);
[Severity: Low]
Is hw_mgt->hw_size read anywhere? It is stored in both branches but never
used to bound a register access, here or later in the series, where only
mailbox_bar_size ends up being checked. Passing the size straight to
pcim_iomap() and dropping the field would work too.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> new file mode 100644
> index 0000000000000..da30244fe75d6
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> @@ -0,0 +1,32 @@
[ ... ]
> +struct nbl_common_info {
> + struct pci_dev *pdev;
> + struct device *dev;
> + u32 msg_enable;
> + u16 vsi_id;
[Severity: Low]
What uses msg_enable? At this commit the only occurrence in the whole nbl
directory is this declaration, and by the end of the series struct
nbl_common_info no longer has the member at all, so it is added and then
silently removed within the same series. Should it just be dropped here?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> index 199626159a4ce..f2552bc732937 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
[ ... ]
> static int nbl_probe(struct pci_dev *pdev,
> const struct pci_device_id *id)
> {
> - return -ENODEV;
> + struct nbl_init_param param = { { 0 } };
> + struct device *dev = &pdev->dev;
> + struct nbl_adapter *adapter;
> + int err;
> +
> + err = pcim_enable_device(pdev);
> + if (err) {
> + dev_err(&pdev->dev, "Failed to enable PCI dev, err=%d\n", err);
> + return err;
> + }
> +
> + nbl_get_func_param(pdev, id->driver_data, ¶m);
> + /* never return fail when DMA_BIT_MASK(64) */
> + dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
> +
> + pci_set_master(pdev);
> +
> + adapter = nbl_core_init(pdev, ¶m);
> + if (IS_ERR(adapter)) {
> + dev_err(dev, "Nbl adapter init fail: %pe\n", adapter);
> + err = PTR_ERR(adapter);
> + goto adapter_init_err;
> + }
> + pci_set_drvdata(pdev, adapter);
> + return 0;
> +adapter_init_err:
> + pci_clear_master(pdev);
> + return err;
> }
[Severity: Low]
This isn't a bug, but from this commit on nbl_probe() returns success after
pcim_enable_device(), dma_set_mask_and_coherent(), pci_set_master() and the
BAR mappings, while no network interface is created.
Grepping the nbl directory for register_netdev or alloc_etherdev at the end
of the series finds no matches either, so the module claims every ID in
nbl_id_table[] and exposes no netdev at any point in the series.
The commit message does describe this as "preparing for subsequent device
implementations", so the only question is placement: would it be preferable
to keep probe returning -ENODEV until the netdev registration lands, so that
bisecting between these patches does not leave the hardware bound to a
driver that offers nothing?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer
2026-09-07 12:38 ` [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 12 potential
issue(s) to consider:
Critical: 0 · High: 1 · Medium: 4 · Low: 7
- [High] nbl_chan_quiesce_and_reclaim_tx() (TX timeout recovery) resets
the software doorbell counter txq->tail_ptr to 0 to match the…
- [Medium] The commit message justifies leaving the mailbox BAR2 register
path unlocked by asserting that the multi-dword QINFO config sequences…
- [Medium] nbl_chan_recv_ack_msg() completes a request slot (copies the
response, sets wait_head->acked = 1) but never transitions…
- [Medium] nbl_chan_quiesce_and_reclaim_tx() uses wmb() with the comment
"Ensure reset write is posted before reclaiming descriptors" and then…
- [Medium] nbl_hw_cfg_mailbox_qinfo() builds the QINFO map dword from
scratch (FUNCTION/DEVID/BUS only) and writes it, unconditionally…
- [Low] Concern (verified FALSE POSITIVE): nbl_chan_send_msg() never
checks chan_info->active, so a send issued before…
- [Low] Concern (verified FALSE POSITIVE): nbl_chan_teardown_queue()
frees nothing but clears chan_info->active, so a subsequent…
- [Low] Concern (verified FALSE POSITIVE): the commit message advertises
working capabilities (dual RX modes, complete queue lifecycle, PF…
- [Low] Concern (verified FALSE POSITIVE): nbl_common_get_hash_node()
returns hash_node->data after dropping the per-bucket spinlock while…
- [Low] Concern (verified FALSE POSITIVE): setup_queue/teardown_queue
check and modify chan_info->active, shutdn, the state bitmap,…
- [Low] Concern (verified FALSE POSITIVE): nbl_chan_update_txqueue()
accepts arg_len > 0 with arg == NULL; in the external-buffer branch it…
- [Low] Concern (verified FALSE POSITIVE): nbl_chan_remove_common()
destroys the workqueue and frees the handler table but never tears…
--
net/nebula-matrix: add channel layer
This adds the mailbox-based inter-PF channel layer for the Nebula Matrix
adapter, including the TX/RX descriptor ring management, a hash table of
message handlers, and the hardware ops that program the mailbox QINFO
tables and the tail pointer doorbell.
A few questions about the TX timeout recovery path, the ACK completion
handshake, and the QINFO map table write.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> new file mode 100644
> index 0000000000000..879e7bab14e94
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
[ ... ]
> +static int nbl_chan_alloc_all_tx_bufs(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
[ ... ]
> + txq->next_to_clean = 0;
> + txq->next_to_use = 0;
> + txq->tail_ptr = 0;
> +
> + return 0;
> +}
[ ... ]
> +static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info,
> + struct nbl_chan_tx_param *param)
> +{
[ ... ]
> + /* Ensure descriptor data visible to device before AVAIL flag */
> + dma_wmb();
> + tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
> +
> + txq->next_to_use =
> + NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
> + txq->tail_ptr++;
> +
> + return 0;
> +}
> +
> +/*
> + * Quiesce the TX mailbox queue and reclaim all outstanding
> + * descriptors. Called from the timeout path of nbl_chan_kick_tx_ring()
> + * with txq_lock held.
> + *
> + * The device failed to fetch/complete the current descriptor within the
> + * polling window. We assert QUEUE_RST to stop further DMA fetches,
> + * reclaim every descriptor between next_to_clean and next_to_use,
> + * reset the software tail_ptr counter to match the hardware reset state,
> + * and re-enable the queue so subsequent sends can proceed.
> + */
> +static void nbl_chan_quiesce_and_reclaim_tx(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_hw_mgt *hw_priv = chan_mgt->hw_ops_tbl->priv;
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct nbl_chan_tx_desc *tx_desc;
> +
> + /* Assert QUEUE_RST to stop hardware fetching new descriptors */
> + hw_ops->stop_mailbox_txq(hw_priv);
> + /* Ensure reset write is posted before reclaiming descriptors */
> + wmb();
[Severity: Medium]
Does this wmb() actually guarantee the posted QUEUE_RST write has reached
the device before the descriptor flags below are rewritten?
writel() already contains the MMIO ordering barriers (__io_bw() and
__io_aw() in include/asm-generic/io.h), so the extra wmb() adds nothing
with respect to the device. nbl_hw_stop_mailbox_txq() only performs
posted BAR2 writes with no read-back, and nbl_chan_config_queue() below
de-asserts RST and sets EN in the immediately following writes:
cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
The QINFO_CFG layout added in this patch exposes RST/EN/DIF_ERR/PTR_ERR
but no reset-done bit, and nothing reads the register back.
Would a read from the same BAR, or a device status poll, be needed before
mutating descriptors the device may still own and before re-enabling the
queue?
> +
> + /*
> + * Reclaim all outstanding descriptors between next_to_clean and
> + * next_to_use. Under txq_lock there is at most one in-flight
> + * descriptor, but iterate the full range for robustness.
> + */
> + while (txq->next_to_clean != txq->next_to_use) {
> + tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq,
> + txq->next_to_clean);
> + WRITE_ONCE(tx_desc->flags, 0);
> + txq->next_to_clean =
> + NBL_NEXT_ID(txq->next_to_clean,
> + chan_info->num_txq_entries - 1);
> + }
> +
> + /*
> + * Hardware tail_ptr counter is cleared by QUEUE_RST. Reset
> + * software counter to match so the next doorbell update does
> + * not produce a false 16-bit wrap delta.
> + */
> + txq->tail_ptr = 0;
[Severity: High]
tail_ptr is reset to 0 here, but next_to_use and next_to_clean keep their
pre-timeout values. Can this leave the ring index and the doorbell
counter permanently out of step?
Everywhere else the two advance together:
nbl_chan_alloc_all_tx_bufs():
txq->next_to_clean = 0;
txq->next_to_use = 0;
txq->tail_ptr = 0;
nbl_chan_update_txqueue():
txq->next_to_use =
NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
txq->tail_ptr++;
After a timeout on slot X the reclaim loop above leaves
next_to_use == next_to_clean == X + 1 while tail_ptr == 0.
The next send then writes descriptor X + 1 and rings the doorbell with
tail_ptr == 1. If QUEUE_RST cleared the hardware fetch pointer as the
comment states, the device looks at slot 0, whose flags the reclaim loop
just zeroed, while nbl_chan_kick_tx_ring() polls descriptor X + 1 for
NBL_CHAN_TX_DESC_USED, times out again and re-enters this same recovery.
Would that make every subsequent nbl_chan_send_msg() return -ETIMEDOUT?
And if QUEUE_RST does not clear the hardware counter, does writing
tail_ptr = 0 produce exactly the bogus 16-bit delta the comment is trying
to avoid?
Should next_to_use and next_to_clean be reset to 0 here as well?
> +
> + /* Re-enable queue with current ring base and size */
> + nbl_chan_config_queue(chan_mgt, chan_info, true);
> +}
[Severity: Medium]
The commit message states:
Mailbox BAR2 register access is intentionally left unlocked: single
dword mailbox writes are atomic, and the multi-dword QINFO config
sequence (config_mailbox_txq/rxq, stop_mailbox_txq/rxq) only runs
during queue setup/teardown when chan_info->active is false, with no
concurrent producers.
Does this recovery path match that description? It issues
stop_mailbox_txq() and config_mailbox_txq() at runtime from the send
path, and chan_info->active is only cleared at the very end of
nbl_chan_teardown_queue(), so active is true throughout.
Related to that, nbl_chan_teardown_queue() calls nbl_chan_stop_queue()
without taking txq_lock, and it does so even when the drain times out:
if (!ret) {
dev_warn(chan_mgt->common->dev,
"teardown: inflight tx drain timeout\n");
ret = -ETIMEDOUT;
}
...
/* After all TX drained, stop hardware queue */
nbl_chan_stop_queue(chan_mgt);
Can teardown's 4-dword QINFO sequence then interleave with a sender still
executing the stop + config sequence above, leaving QUEUE_EN=1 on a ring
the driver considers torn down?
> +
> +static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct device *dev = chan_mgt->common->dev;
> + int max_retries = NBL_CHAN_TX_WAIT_TIMES;
> + struct nbl_chan_tx_desc *tx_desc;
> + int retry_count = 0;
> + u16 msg_type;
> +
> + nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
> + txq->tail_ptr, NBL_MB_TX_QID);
> +
> + tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_clean);
[ ... ]
> + retry_count++;
> + if (retry_count == max_retries) {
> + msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
> + dev_err_ratelimited(dev, "chan send msg type: %d timeout\n",
> + msg_type);
> + /*
> + * Device failed to complete this descriptor.
> + * Quiesce the queue, reclaim the timed-out
> + * descriptor, and re-enable so future sends can
> + * proceed instead of stalling the ring full.
> + */
> + nbl_chan_quiesce_and_reclaim_tx(chan_mgt,
> + chan_info);
> + return -ETIMEDOUT;
> + }
> + usleep_range(NBL_CHAN_TX_WAIT_US, NBL_CHAN_TX_WAIT_US_MAX);
> + }
> +
> + txq->next_to_clean = txq->next_to_use;
> +
> + return 0;
> +}
[ ... ]
> +static void nbl_chan_recv_ack_msg(void *priv, u16 srcid, u16 msgid, void *data,
> + u32 data_len)
> +{
[ ... ]
> + if (w_status != NBL_MBX_STATUS_WAITING) {
> + mutex_unlock(&chan_info->pending_lock);
> + dev_err_ratelimited(dev,
> + "Skip ack invalid status, wait msgtype:%u idx:%u status:%d ack msgtype:%u msgid:%u datalen:%u\n",
> + w_msgtype, w_msgidx, w_status,
> + ack_msgtype, ack_msgid, ack_datalen);
> + return;
> + }
[ ... ]
> +ack_done:
> + /* Guarantee payload data finished before acked flag visible */
> + smp_wmb();
> + WRITE_ONCE(wait_head->acked, 1);
> + mutex_unlock(&chan_info->pending_lock);
> + wake_up(&wait_head->wait_queue);
> +}
[Severity: Medium]
This completes the slot by setting acked = 1 but leaves
wait_head->status at NBL_MBX_STATUS_WAITING. Can that make the timeout
paths in nbl_chan_send_msg() discard a delivered ACK?
The interrupt path tests only status:
if (ret == 0) {
mutex_lock(&chan_info->pending_lock);
if (READ_ONCE(wait_head->status) ==
NBL_MBX_STATUS_WAITING) {
WRITE_ONCE(wait_head->status,
NBL_MBX_STATUS_TIMEOUT);
WRITE_ONCE(wait_head->acked, 0);
So if the RX cleanup work runs the code above in the window between
wait_event_timeout() evaluating its condition for the last time and the
sender taking pending_lock, the sender sees WAITING, clears acked and
returns -ETIMEDOUT for a request that was answered.
The polling path has a similar window: it checks acked, then sleeps
NBL_CHAN_TX_WAIT_ACK_US_MIN..MAX, and after the last iteration falls
straight through to ret = -ETIMEDOUT with no further locked check, so an
ACK arriving during that final ~1 ms sleep is dropped.
Since status also stays WAITING, would a duplicate ACK for the same
msgid still pass all the validation above and memcpy into the requester's
response buffer a second time?
Should the ACK handler move the slot to a completed state (or should the
timeout transitions re-check acked under pending_lock)?
> +
> +static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_send_info *chan_send)
> +{
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index 0ba0b7f643c64..379dbb9dc0034 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -10,6 +10,156 @@
[ ... ]
> +static void nbl_hw_stop_mailbox_txq(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
> +
> + memset(&cfg_tbl, 0, sizeof(cfg_tbl));
> + cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
> + cfg_tbl.data, sizeof(cfg_tbl));
> +}
> +
> +static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
> +{
> + nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_K_PF_MASK_REG, pf_mask,
> + sizeof(*pf_mask));
> +}
> +
> +static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + u8 bus, u8 devid, u8 function)
> +{
> + u32 data = 0;
> +
> + data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
> + FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
> + FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
> + nbl_hw_wr_regs_lock(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
> + &data,
> + sizeof(data));
> +}
[Severity: Medium]
This builds the whole dword from FUNCTION/DEVID/BUS and writes it, but
the MSI-X fields share the same dword:
#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK GENMASK(28, 16)
#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK BIT(29)
Does this zero the mailbox MSI-X routing of the entries it rewrites?
nbl_chan_cfg_qinfo_map_table() programs an entry for every unmasked PF,
not just the local function:
for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
if (!(pf_mask & (1 << func_id)))
hw_ops->cfg_mailbox_qinfo(p, func_id, bus,
devid, func_id);
}
so re-probing one PF would clear MSIX_IDX and MSIX_IDX_VALID for its
already-bound siblings. Those siblings keep NBL_CHAN_IRQ_RDY set, so
each synchronous send would wait out NBL_CHAN_ACK_WAIT_TIME and fail with
-ETIMEDOUT until they are re-bound.
The commit message also says:
Add per-hw_mgt reg_lock to protect read-modify-write sequences on the
general-purpose register BAR (hw_addr, BAR0), such as QINFO map table
entries and k_pf_mask reads
Where is the read-modify-write on the QINFO map entry? This helper only
writes. Should it read the entry first and preserve the MSI-X fields?
> +
> +static struct nbl_hw_ops hw_ops = {
> + .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
> + .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
> + .config_mailbox_txq = nbl_hw_config_mailbox_txq,
> + .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
> + .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
> + .get_host_pf_mask = nbl_hw_get_host_pf_mask,
> + .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
> +
> +};
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> index 1f9e509dc6312..71691e893790e 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> @@ -11,5 +11,61 @@
[ ... ]
> +/* MAILBOX qinfo_map_table */
> +#define NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK GENMASK(2, 0)
> +#define NBL_MAILBOX_QINFO_MAP_DEVID_MASK GENMASK(7, 3)
> +#define NBL_MAILBOX_QINFO_MAP_BUS_MASK GENMASK(15, 8)
> +#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK GENMASK(28, 16)
> +#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK BIT(29)
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation
2026-09-07 12:38 ` [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 0 · Low: 2
- [Low] DISCARDED (baseline decision false): claim that the commit
message's statement 'The dispatch layer serializes all resource ops…
- [Low] Kernel-doc/declaration mismatch for the new hw op: in
drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h the…
--
net/nebula-matrix: add common resource implementation
This adds the chip-independent resource layer for the nbl driver, which
owns the PF/eth/vsi identity tables and the lookup helpers between
vsi_id, func_id, eth_id and pf_id. It also adds two new hardware ops,
get_fw_eth_map() and get_board_info(), plus get_real_bus(), used during
control-PF initialization.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index be63e53d41c61..e05248c66afbb 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
[ ... ]
> @@ -35,11 +36,26 @@ struct nbl_hw_ops {
> * All-zero mask (0x00) means all 8 PFs enabled, which is
> * unsupported by the driver and rejected with -EINVAL.
> *
> + * Firmware contract: number of unmasked PFs MUST equal
> + * get_board_info()->eth_num.
> */
> void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
> + void (*get_real_bus)(struct nbl_hw_mgt *hw_mgt, u8 *bus);
>
> void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> u8 bus, u8 devid, u8 function);
> + void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
> + /**
> + * get_board_info - Fetch board info from firmware
> + * @hw_mgt: hardware management context
> + * @board_info: output pointer for board info structure
> + *
> + * Firmware contract: board_info.eth_num MUST equal the number of
> + * unmasked PFs from get_host_pf_mask(). See get_host_pf_mask for
> + * details.
> + */
> + void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
> + struct nbl_board_port_info *board);
[Severity: Low]
This isn't a bug, but the kernel-doc block for get_board_info in
nbl_def_hw.h documents @board_info while the callback is declared with the
parameter named board:
* @board_info: output pointer for board info structure
...
void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
struct nbl_board_port_info *board);
Would make W=1 warn here with "Excess function parameter 'board_info'
description" and "Function parameter or struct member 'board' not
described"?
The implementation in nbl_hw_leonis.c uses the documented name:
static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
struct nbl_board_port_info *board_info)
so renaming the declared parameter to board_info would keep the three
consistent. The mismatch is still present at the end of the series.
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 05/10] net/nebula-matrix: add intr resource implementation
2026-09-07 12:38 ` [PATCH v27 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 4 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 1 · Low: 3
- [Medium] Coherent MSI-X map table is released based on a timed guess,
not on proven device quiescence.
- [Low] Two in-tree comments state mutually exclusive ownership models
for the coherent MSI-X tables.
- [Low] The 'Step 1' comment in __nbl_res_intr_destroy_msix_map() says
'mask each MSIX vector in hardware first', but the call it annotates…
- [Low] Reported concern: nbl_intr_mgt_stop() publishes the 'dead' state
with a plain store 'res_mgt->intr_mgt = NULL' outside intr_mgt->lock,…
--
net/nebula-matrix: add intr resource implementation
This adds the nbl_interrupt module which owns the driver-wide global MSI-X
index space and programs the chip-internal MSI-X map/info registers. It
adds cfg_msix_map, destroy_msix_map, set_mailbox_irq and cfg_msix_info,
plus nbl_intr_mgt_start()/nbl_intr_mgt_stop() wired into the leonis
resource layer.
A few questions about the teardown path and about some of the comments
below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index b4aba4faa555a..4c2e577610230 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
[ ... ]
> @@ -91,6 +115,124 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
[ ... ]
> +static void nbl_hw_cfg_msix_info(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool valid, u16 interrupt_id, u8 bus,
> + u8 devid, u8 function, bool msix_mask_en)
> +{
> + u32 host_msix_fid = 0;
> + struct nbl_host_msix_info msix_info;
> +
> + memset(&msix_info, 0, sizeof(msix_info));
> + if (valid) {
> + host_msix_fid =
> + FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK,
> + func_id) |
> + FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK,
> + 1);
> +
> + msix_info.data[1] =
> + FIELD_PREP(NBL_HOST_MSIX_INFO_FUNCTION_MASK, function) |
> + FIELD_PREP(NBL_HOST_MSIX_INFO_DEVID_MASK, devid) |
> + FIELD_PREP(NBL_HOST_MSIX_INFO_BUS_MASK, bus) |
> + FIELD_PREP(NBL_HOST_MSIX_INFO_VALID_MASK, 1);
> +
> + if (msix_mask_en)
> + msix_info.data[1] |=
> + FIELD_PREP(NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK, 1);
> + }
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> index 46180522295a3..c1f10f1f6b775 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
[ ... ]
> @@ -328,8 +337,24 @@ int nbl_res_init_leonis(struct nbl_adapter *adap)
>
> void nbl_res_remove_leonis(struct nbl_adapter *adap)
> {
> + struct nbl_resource_mgt *res_mgt = adap->core.res_mgt;
> + struct nbl_common_info *common = &adap->common;
> +
> + if (!res_mgt)
> + return;
> +
> + /*
> + * Tear down all MSI-X maps before devres releases the coherent
> + * tables. This is critical on the control PF, which may hold
> + * maps for remote PFs that are still bound.
> + */
> + if (common->has_ctrl && res_mgt->intr_mgt)
> + nbl_intr_mgt_stop(res_mgt);
> +
> /*
> - * No resource release here because all memory uses devm managed
> - * allocation
> + * Note: the per-function interrupts arrays (kcalloc) are freed
> + * by nbl_intr_mgt_stop() above. The coherent MSI-X tables
> + * (dmam_alloc_coherent) and intr_mgt itself (devm_kzalloc) are
> + * released by devres after this function returns.
> */
> }
[Severity: Low]
Is this comment still accurate?
nbl_intr_mgt_stop() loops over every function id and calls
__nbl_res_intr_destroy_msix_map(), which itself calls dmam_free_coherent()
on msix_map_table->base_addr and then sets it to NULL. So by the time
nbl_res_remove_leonis() returns, there should be no devres-managed
coherent MSI-X table left for devres to release.
The kernel-doc on nbl_res_intr_cfg_msix_map() states the opposite
ownership model:
* Old MSIX table memory is explicitly freed inside the locked
* destroy path after a bounded DMA quiesce sleep (best-effort;
* hardware provides no idle status register), so repeated
* reconfiguration does not accumulate devres-managed DMA memory.
The commit message carries the same wording as the comment here, "It is
called from nbl_res_remove_leonis() before devres releases the coherent
tables."
Could these be reconciled so a later change does not drop the explicit
dmam_free_coherent() on the assumption devres still owns the tables?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> new file mode 100644
> index 0000000000000..fd3b71a05c236
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> @@ -0,0 +1,544 @@
[ ... ]
> + /* Step 0: disable mailbox IRQ routing before tearing down map */
> + __nbl_res_intr_set_mailbox_irq(res_mgt, func_id, 0, false);
> +
> + /* Step 1: mask each MSIX vector in hardware first */
> + for (i = 0; i < intr_num; i++) {
> + hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
> + func_id, false, interrupts[i],
> + 0, 0, 0, false);
> + }
> + hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
[Severity: Low]
Does this loop actually mask anything?
The call passes valid=false and msix_mask_en=false, and
nbl_hw_cfg_msix_info() starts with:
memset(&msix_info, 0, sizeof(msix_info));
if (valid) {
...
}
so for valid=false both host_msix_fid and msix_info.data[] stay zero and
the writes to NBL_PCOMPLETER_HOST_MSIX_FID_TABLE() and
NBL_PADPT_HOST_MSIX_INFO_REG_ARR() drive
NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK (BIT(17)) and
NBL_HOST_MSIX_INFO_VALID_MASK (BIT(16)) to 0 and clear the FID VLD bit.
That invalidates the entry and turns mask-enable off rather than masking
the vector. Would "invalidate each MSIX info entry" describe the step
more accurately? The commit message has the same wording, "Step 1 masks
each vector."
> + /*
> + * Stage 1 tear down: retain valid DMA address, ONLY clear
> + * VALID bit to avoid hardware torn read (VALID=1 & dma_addr=0).
> + */
> + hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
> + false, msix_map_table->dma, 0, 0, 0);
> + hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
> +
> + /*
> + * Hardware provides no idle status register for the MSIX map
> + * DMA engine. Use a bounded sleep to mitigate the race between
> + * posted MMIO disable writes and an ongoing in-flight table
> + * read DMA.
> + *
> + * This is best-effort, not a guarantee: a table fetch already
> + * issued before the VALID clear was observed can complete after
> + * this sleep. On the normal teardown path the mailbox channel
> + * is stopped before this function runs, so no new interrupts
> + * can trigger table fetches. On the residual cleanup path in
> + * nbl_intr_mgt_stop(), a longer global quiesce is applied
> + * after all functions are torn down.
> + */
> + usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
> +
> + /* safe to release global vector IDs, pcompler no longer reads table */
> + nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
> +
> + /*
> + * Stage 2: hardware has quiesced MSIX table DMA access, fully
> + * zero the MSIX map entry safely now.
> + */
> + hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
> + false, 0, 0, 0, 0);
> + hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
> +
> + /*
> + * Now safe to release the MSIX DMA coherent memory. Hardware
> + * DMA has quiesced after the sleep above, so no IOMMU fault
> + * risk remains.
> + */
> + if (msix_map_table->base_addr) {
> + dmam_free_coherent(dev, msix_map_table->size,
> + msix_map_table->base_addr,
> + msix_map_table->dma);
> + msix_map_table->base_addr = NULL;
> + msix_map_table->dma = 0;
> + msix_map_table->size = 0;
> + }
[Severity: Medium]
Can the coherent table be freed here while the pcompleter still has a
table fetch outstanding?
The two comments in __nbl_res_intr_destroy_msix_map() disagree with each
other. The first says:
* This is best-effort, not a guarantee: a table fetch already
* issued before the VALID clear was observed can complete after
* this sleep.
and a few lines later:
* Hardware DMA has quiesced after the sleep above, so no IOMMU
* fault risk remains.
followed by dmam_free_coherent() on the buffer the device DMA-reads.
If a fetch does land after the free, with an IOMMU the read hits an
unmapped IOVA and produces a DMAR/SMMU fault; without one it reads
recycled memory whose bytes are interpreted as an MSI-X map entry
(NBL_MSIX_MAP_VALID_MASK plus the 13-bit NBL_MSIX_MAP_INDEX_MASK), which
could steer an interrupt to a global vector now owned by a different
function.
The table size does not depend on the request:
tmp_msix_tbl->size =
sizeof(struct nbl_msix_map) * NBL_MSIX_MAP_TABLE_MAX_ENTRIES;
Since the allocation is a fixed 2 KiB regardless of num_net_msix /
num_others_msix, would it be simpler to allocate the table once per
function and rewrite the entries in place, so the free/realloc cycle, the
timed sleeps and the admitted race all go away?
The commit message describes the same mechanism, "sleeps 1 ms to allow
in-flight table fetch DMA to quiesce (best-effort; no idle status register
exists), then zeroes the entry before freeing memory."
[ ... ]
> +void nbl_intr_mgt_stop(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
> + u16 func_id;
> + int ret;
> +
> + if (!intr_mgt)
> + return;
> +
> + mutex_lock(&intr_mgt->lock);
> + for (func_id = 0; func_id < NBL_MAX_FUNC; func_id++) {
> + if (intr_mgt->func_intr_res[func_id].interrupts) {
> + dev_info(res_mgt->common->dev,
> + "intr_mgt_stop: destroying leftover map for func %u\n",
> + func_id);
> + ret = __nbl_res_intr_destroy_msix_map(res_mgt,
> + func_id);
> + if (ret)
> + dev_warn(res_mgt->common->dev,
> + "intr_mgt_stop: destroy map for func %u failed: %d\n",
> + func_id, ret);
> + }
> + }
> + mutex_unlock(&intr_mgt->lock);
> +
> + /*
> + * Global quiesce after all functions are torn down. Each
> + * destroy has an internal 1ms sleep between Stage 1 (clear
> + * VALID) and Stage 2 (zero dma_addr), but Stage 2 itself has
> + * no trailing sleep. This final wait covers the last
> + * function's Stage 2 and any straggler DMA from
> + * concurrently-torndown functions.
> + */
> + usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
[Severity: Medium]
Does this final usleep_range() still cover anything?
The loop above has already called __nbl_res_intr_destroy_msix_map() for
every func_id, and each of those calls ends with dmam_free_coherent() on
that function's table. By the time this sleep runs there is no coherent
table left that a straggler DMA could safely land in, so the wait appears
to be after the window the comment says it protects.
Should this wait happen before the per-function dmam_free_coherent()
instead, or is it simply not needed?
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation
2026-09-07 12:38 ` [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 1 · Low: 0
- [Medium] No hardware unwind path exists for the chip-wide state armed
by nbl_hw_init_module() (nbl_hw_leonis.c:681), neither on init failure…
--
net/nebula-matrix: add chip-wide hardware init/deinit implementation
This adds the Leonis chip-wide init/deinit implementation, programming the
datapath pipeline (packet engines, scheduler, buffer stores, queue manager,
port shaping and flow control) plus the driver_status flag read by firmware.
The new hw_ops.init_module/deinit_module and the matching resource_ops entry
points are registered here, with the invocation path landing in a later
patch.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index 4c2e577610230..9f1054067e14c 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -115,6 +115,585 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
[ ... ]
> +static int nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
> +{
> + struct nbl_shaping_dvn_dport_u dvn_dport = { 0 };
> + struct nbl_shaping_dport_u dport = { 0 };
> + u32 rate, half_rate;
> + u32 depth;
> + u64 low_val, high_val;
> +
> + switch (speed) {
> + case NBL_FW_PORT_SPEED_100G:
> + rate = 100000;
> + break;
[ ... ]
> + default:
> + dev_err(hw_mgt->common->dev,
> + "Unsupported port speed %u for eth%u\n", speed, eth_id);
> + return -EINVAL;
> + }
[ ... ]
> + /* 3. commit: set valid last */
> + low_val = FIELD_PREP(NBL_DPORT_VALID_MASK, 1);
> + dport.data[0] |= lower_32_bits(low_val);
> +
> + low_val = FIELD_PREP(NBL_DPORT_VALID_MASK, 1);
> + dvn_dport.data[0] |= lower_32_bits(low_val);
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
> + sizeof(dport));
> + nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
> + dvn_dport.data, sizeof(dvn_dport));
> + spin_unlock(&hw_mgt->reg_lock);
> + return 0;
> +}
> +
> +static int nbl_shaping_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
> +{
> +#define NBL_SHAPING_FLUSH_INTERVAL 128
> + struct nbl_shaping_net_u net_shaping = { 0 };
> + u32 eth_bitmap = 0;
> + u32 psha_en = 0;
> + int ret;
> + int i;
> +
> + nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
> + for (i = 0; i < NBL_MAX_ETHERNET; i++) {
> + if (!(eth_bitmap & BIT(i)))
> + continue;
> + ret = nbl_shaping_eth_init(hw_mgt, i, speed);
> + if (ret)
> + return ret;
> + }
> + psha_en = eth_bitmap & GENMASK(3, 0);
> + psha_en = FIELD_PREP(NBL_DSCH_PSHA_EN_MASK, psha_en);
> + nbl_hw_wr_regs_lock(hw_mgt, NBL_DSCH_PSHA_EN_ADDR, &psha_en,
> + sizeof(psha_en));
[ ... ]
> +static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
> +{
[ ... ]
> + if (WARN_ON(eth_num != 1 && eth_num != 2 && eth_num != 4))
> + return -EINVAL;
[ ... ]
> + fc_th |= FIELD_PREP(NBL_DSTORE_D_DPORT_FC_FC_EN_MASK, 1);
> + nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
> + sizeof(fc_th));
> + spin_unlock(&hw_mgt->reg_lock);
> + }
> +}
[ ... ]
> +static int nbl_dp_init(struct nbl_hw_mgt *hw_mgt, u8 speed, u8 eth_num)
> +{
> + int ret;
> +
> + nbl_dped_init(hw_mgt);
> + nbl_uped_init(hw_mgt);
> + ret = nbl_shaping_init(hw_mgt, speed);
> + if (ret)
> + return ret;
> + nbl_dsch_qid_max_init(hw_mgt);
> + ret = nbl_ustore_init(hw_mgt, eth_num);
> + if (ret)
> + return ret;
> + nbl_dstore_init(hw_mgt, speed);
> + nbl_dvn_init(hw_mgt, speed);
> + nbl_uvn_init(hw_mgt);
> + nbl_uqm_init(hw_mgt);
> + return 0;
> +}
[ ... ]
> +/*
> + * This design is intentional. Setting driver status to false is the
> + * official teardown mechanism: it notifies firmware to perform full
> + * cleanup of all per-PF hardware state, including qinfo registers.
> + * An inverse helper would duplicate work that the firmware already
> + * does, and would add error-path complexity for no benefit. We keep
> + * the deinit path minimal and rely on firmware cleanup for correctness,
> + * including abnormal hardware reset scenarios.
> + */
> +static void nbl_hw_deinit_module(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_hw_set_driver_status(hw_mgt, false);
> + /* ensure registers written */
> + nbl_flush_writes(hw_mgt);
[ ... ]
> +static int nbl_hw_init_module(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
> + u8 eth_num)
> +{
> + int ret;
> +
> + ret = nbl_dp_init(hw_mgt, eth_speed, eth_num);
> + if (ret)
> + return ret;
> + nbl_intf_init(hw_mgt);
> + nbl_hw_set_driver_status(hw_mgt, true);
[Severity: Medium]
nbl_hw_init_module() can return an error after a good part of the chip has
already been programmed, and nothing puts those registers back. There are
two failure points inside nbl_dp_init():
nbl_shaping_init() -> nbl_shaping_eth_init() returns -EINVAL from the
switch default for any speed outside 10/25/50/100G. By then the ports
earlier in the eth_bitmap loop have already been committed with VALID=1,
and NBL_DSCH_PSHA_EN plus the NBL_SHAPING_NET_REG() clearing loop are
skipped.
nbl_ustore_init() returns -EINVAL from
WARN_ON(eth_num != 1 && eth_num != 2 && eth_num != 4) after DPED, UPED,
shaping and DSCH were already programmed.
The speed comes from firmware, read by nbl_hw_get_board_info() out of
NBL_FW_BOARD_DW3, so a value outside enum nbl_fw_port_speed reaches
nbl_shaping_eth_init() without any earlier check.
In both cases the error is propagated before
nbl_hw_set_driver_status(hw_mgt, true), so firmware is never told a driver
attached and the firmware-side cleanup that deinit relies on is not
triggered either.
Should nbl_dp_init() unwind the sub-modules it has already programmed, or
should eth_speed and eth_num be validated before the first register write?
On the teardown side, the comment above nbl_hw_deinit_module() says
setting driver status to false makes firmware "perform full cleanup of all
per-PF hardware state, including qinfo registers", but the state this
patch programs is chip-wide rather than per-PF:
shaping DPORT/DVN_DPORT entries with VALID=1
NBL_DSCH_PSHA_EN
USTORE per-port drop threshold EN
DSTORE DISC_BP_TH EN and per-port FC_EN
DVN/UVN relaxed-ordering flags
UVN prefetch init and queue err mask
UQM que_type
Most of those are programmed read-modify-write, for example:
nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
bp_th |= FIELD_PREP(NBL_DSTORE_DISC_BP_TH_EN_MASK, 1);
nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
so after an unbind or a failed control PF probe, do these enables stay
armed and get carried into the next attach, or into another consumer such
as vfio-pci? Also, the relaxed-ordering bits are derived from the PCIe
DEVCTL value observed at init time only.
Could the commit message or the comment state which of these chip-wide
registers firmware actually reverts, so the claim can be checked against
the register list above?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops
2026-09-07 12:38 ` [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 7 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 1 · Low: 6
- [Medium] All five new request helpers do 'ret =
chan_ops->send_msg(...); if (ret < 0) return ret;' before the…
- [Low] Commit message attributes pre-existing resource-layer behaviour
to this patch.
- [Low] The commit message justifies the new per-dispatch ops_mutex_lock
with "This eliminates race windows between local control paths and…
- [Low] Over-broad error-path cleanup: on any partial registration
failure, nbl_disp_setup_msg() calls chan_ops->unregister_all_msg(p),…
- [Low] Claimed namespace inconsistency: the five new responders validate
the requester with 'src_id >= disp_mgt->common->max_pf' and pass…
- [Low] Claimed teardown race / use-after-teardown: the five responders
registered by nbl_disp_setup_msg() are never unregistered…
- [Low] Claimed self-defeating RPC: destroy_msix_map (and cfg_msix_map,
which internally destroys first) tear down the *requesting* function's…
--
net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops
This adds mailbox RPC request/response handlers for five dispatch
operations (configure_msix_map, destroy_msix_map, set_mailbox_irq,
get_vsi_id, get_eth_id), resolves the dispatch ops table based on the PF
control level, and introduces a per-dispatch mutex around the mutable
hardware ops.
A few questions on the changelog and on the new code below.
Two paragraphs of the changelog describe behaviour that lives in
nbl_hw/nbl_interrupt.c, which this patch does not touch:
> destroy_msix_map responder clears the mailbox QINFO MSIX routing
> (MSIX_IDX / MSIX_IDX_VALID) as Step 0 before tearing down the
> MSI-X map, ensuring a recycled global vector cannot trigger an
> interrupt owned by a different function.
>
> cfg_msix_map pre-allocates all coherent DMA buffers and global
> vector indices before destroying the old configuration, then swaps
> the new resources into the per-function entry to avoid interrupt
> loss during reconfiguration.
[Severity: Low]
Are these two paragraphs describing this patch?
nbl_disp_chan_destroy_msix_map_resp() only validates src_id and calls
res_ops->destroy_msix_map(), and nbl_disp_chan_cfg_msix_map_resp() only
forwards to res_ops->cfg_msix_map().
The "Step 0" comment and the pre-allocate/swap sequence are in
__nbl_res_intr_destroy_msix_map() and nbl_res_intr_cfg_msix_map() in
nbl_hw/nbl_interrupt.c:
/* Step 0: disable mailbox IRQ routing before tearing down map */
__nbl_res_intr_set_mailbox_irq(res_mgt, func_id, 0, false);
That file is unchanged here, and the earlier series commit that added the
intr resource implementation already documents both behaviours. Could
these two paragraphs be dropped or reworded to describe only the
forwarding the responders do?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> index 3da5f8351fa42..5be2d0887ad41 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> @@ -3,9 +3,186 @@
> * Copyright (c) 2026 Nebula Matrix Limited.
> */
> #include <linux/device.h>
> +#include <linux/mutex.h>
> #include <linux/pci.h>
> #include "nbl_dispatch.h"
>
> +static int nbl_disp_chan_get_vsi_id_req(struct nbl_dispatch_mgt *disp_mgt,
> + u16 type, u16 *vsi_id)
> +{
> + struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
> + struct nbl_common_info *common = disp_mgt->common;
> + struct nbl_chan_param_get_vsi_id result = { 0 };
> + struct nbl_chan_param_get_vsi_id param = { 0 };
> + struct nbl_chan_send_info chan_send = {0};
> + int ret;
> +
> + param.type = cpu_to_le16(type);
> +
> + nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
> + NBL_CHAN_MSG_GET_VSI_ID,
> + ¶m, sizeof(param), &result,
> + sizeof(result), 1);
> + ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
> + if (ret < 0)
> + return ret;
> + switch (ret) {
[Severity: Medium]
The changelog says the early return here only passes through local
transport errors:
> Local transport errors (-ETIMEDOUT, -ESHUTDOWN,
> -EAGAIN, -EIO, -EINVAL) are passed through unchanged via an early
> "if (ret < 0) return ret" before the status switch.
Can this early return also pass through a value taken straight off the
wire? On a successful ACK, nbl_chan_send_msg() returns
READ_ONCE(wait_head->ack_err), and nbl_chan_recv_ack_msg() fills that
field without any validation:
WRITE_ONCE(wait_head->ack_err,
(s32)le32_to_cpu(*(__le32 *)&payload[NBL_CHAN_ACK_RET_POS]));
So any negative 32-bit value the peer puts in the ACK is returned to the
caller as if it were a locally generated errno, bypassing the
NBL_CHAN_RESP_* translation switch that follows. The same pattern is in
nbl_disp_chan_get_eth_id_req(), nbl_disp_chan_cfg_msix_map_req(),
nbl_disp_chan_destroy_msix_map_req() and
nbl_disp_chan_set_mailbox_irq_req().
For get_vsi_id()/get_eth_id() the value reaches ERR_PTR():
nbl_dev_setup_common_dev() -> nbl_dev_init() -> nbl_core_init()
return ERR_PTR(ret);
nbl_probe() then only recognises the range [-4095,-1]:
adapter = nbl_core_init(pdev, ¶m);
if (IS_ERR(adapter)) { ... }
err = nbl_core_start(adapter);
With a magnitude larger than MAX_ERRNO, IS_ERR() is false and
nbl_core_start() runs on a bogus pointer. Would it make sense to clamp
or reject out-of-range ack_err values in nbl_chan_recv_ack_msg(), or to
sanitize the return in these helpers rather than passing it through?
> + case NBL_CHAN_RESP_OK:
> + break;
> + case NBL_CHAN_RESP_UNIMPLEMENTED:
> + return -EOPNOTSUPP;
> + case NBL_CHAN_RESP_ERR:
> + return -EREMOTEIO;
> + case NBL_CHAN_RESP_PERM_DENY:
> + return -EPERM;
> + default:
> + return -EREMOTEIO;
> + }
[ ... ]
> +static void nbl_disp_chan_get_vsi_id_resp(void *priv, u16 src_id, u16 msg_id,
> + void *data, u32 data_len)
> +{
[ ... ]
> + if (src_id >= disp_mgt->common->max_pf) {
> + err = NBL_CHAN_RESP_PERM_DENY;
> + goto ack_out;
> + }
[Severity: Low]
This isn't a bug today, but the five responders check src_id against
[0, max_pf) and then hand src_id to res_ops->* as a func_id, while the
resource layer defines the PF func_id space relative to common->mgt_pf:
nbl_common_func_id_to_rel_pf_id() {
if (pf_id < common->mgt_pf)
return -EINVAL;
*rel_pf_id = pf_id - common->mgt_pf;
...
}
Since nbl_core_init() sets common->mgt_pf = 0 unconditionally and nothing
else writes it, the two ranges are identical, so there is no divergence
at the moment. Would using nbl_common_func_id_to_rel_pf_id() in the
responders keep the two layers in agreement if the "future non-zero
management PF" mentioned in the kernel-doc is ever implemented?
> + if (data_len < sizeof(param)) {
> + err = NBL_CHAN_RESP_ERR;
> + goto ack_out;
> + }
> + memcpy(¶m, data, sizeof(param));
[ ... ]
> @@ -25,6 +202,352 @@ static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
> return -EOPNOTSUPP;
> }
>
> +static int nbl_disp_cfg_msix_map(struct nbl_dispatch_mgt *disp_mgt,
> + u16 num_net_msix, u16 num_others_msix,
> + bool net_msix_mask_en)
> +{
> + struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
> + struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
> + struct nbl_common_info *common = disp_mgt->common;
> + int ret;
> +
> + if (!res_ops->cfg_msix_map)
> + return -EOPNOTSUPP;
> + mutex_lock(&disp_mgt->ops_mutex_lock);
> + ret = res_ops->cfg_msix_map(p, common->mgt_pf, num_net_msix,
> + num_others_msix, net_msix_mask_en);
> + mutex_unlock(&disp_mgt->ops_mutex_lock);
> + return ret;
> +}
[Severity: Low]
The changelog says of ops_mutex_lock:
> This eliminates race windows between local control paths and
> asynchronous remote mailbox RPC response handlers.
Is there a race window left to eliminate? The three wrapped ops already
serialize themselves in nbl_interrupt.c:
nbl_res_intr_destroy_msix_map() {
mutex_lock(&intr_mgt->lock);
ret = __nbl_res_intr_destroy_msix_map(res_mgt, func_id);
mutex_unlock(&intr_mgt->lock);
}
nbl_res_intr_set_mailbox_irq() does the same, and
nbl_res_intr_cfg_msix_map() holds intr_mgt->lock across its
allocate/destroy/swap sequence. The earlier series commit states "All
public entry points (cfg/destroy/set_irq) take this lock internally;
callers need not hold any upper-layer lock."
So this nests a second mutex around calls that are already mutually
exclusive (in a consistent order, so no deadlock). Could the rationale
be reworded, or the outer mutex dropped, if the intent is only defensive
serialization?
[ ... ]
> +static void nbl_disp_chan_destroy_msix_map_resp(void *priv, u16 src_id,
> + u16 msg_id, void *data,
> + u32 data_len)
> +{
[ ... ]
> + if (res_ops->destroy_msix_map) {
> + mutex_lock(&disp_mgt->ops_mutex_lock);
> + ret = res_ops->destroy_msix_map(p, src_id);
> + mutex_unlock(&disp_mgt->ops_mutex_lock);
> + if (ret)
> + err = NBL_CHAN_RESP_ERR;
> + } else {
> + err = NBL_CHAN_RESP_UNIMPLEMENTED;
> + }
> +ack_out:
> + nbl_chan_fill_ack_info(&chan_ack, src_id,
> + NBL_CHAN_MSG_DESTROY_MSIX_MAP, msg_id,
> + err, NULL, 0);
> + ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
[Severity: Low]
Does the ACK here still reach the requester? This tears down the
requesting function's map, and __nbl_res_intr_destroy_msix_map() disables
that function's mailbox MSI-X route first:
/* Step 0: disable mailbox IRQ routing before tearing down map */
__nbl_res_intr_set_mailbox_irq(res_mgt, func_id, 0, false);
The ACK is then sent over the mailbox whose interrupt was just disarmed,
and the NBL_CHAN_IRQ_RDY branch of nbl_chan_send_msg() only waits on
wait_event_timeout(). The same applies to cfg_msix_map, which destroys
before reconfiguring.
Looking at the callers added later in the series, this appears to be
avoided: nbl_dev_disable_mailbox_irq() clears NBL_CHAN_IRQ_RDY before
issuing the RPC, nbl_dev_stop() and the err_destroy_map path of
nbl_dev_start() only call destroy_msix_map() after that, and
nbl_dev_cfg_msix_map() runs before nbl_dev_enable_mailbox_irq() ever sets
IRQ_RDY, so send_msg() takes the polling branch and consumes the ACK
without an interrupt. Is that ordering an invariant worth documenting
next to these responders, since it is what keeps them from timing out on
an operation that succeeded?
[ ... ]
> +static int nbl_disp_setup_msg(struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *p = disp_mgt->chan_ops_tbl->priv;
> + int ret = 0;
> + int _ret;
> +
> + _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
> + nbl_disp_chan_cfg_msix_map_resp,
> + disp_mgt);
> + if (_ret < 0 && !ret)
> + ret = _ret;
[ ... ]
> + if (ret)
> + chan_ops->unregister_all_msg(p);
> + return ret;
> +}
[Severity: Low]
Does this rollback destroy state the dispatch layer does not own?
unregister_all_msg is nbl_chan_remove_msg_handler():
nbl_channel.c:nbl_chan_remove_msg_handler() {
if (!chan_mgt->handle_hash_tbl)
return;
nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl);
chan_mgt->handle_hash_tbl = NULL;
}
That frees the whole per-adapter handler table, including the channel
layer's own NBL_CHAN_MSG_ACK handler registered in nbl_chan_setup_ops():
ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
nbl_chan_recv_ack_msg, chan_mgt);
and leaves chan_mgt (alive until nbl_chan_remove_common()) with
handle_hash_tbl == NULL, which nbl_chan_recv_msg() would pass into
nbl_common_get_hash_node(), whose first statement is:
key_size = tbl_mgt->tbl_key.key_size;
with no NULL check. The failure itself is reachable, since
nbl_chan_add_msg_handler() -> nbl_common_alloc_hash_node() can return
-ENOMEM. Today nothing can crash, because on failure nbl_disp_init()
returns the error and probe is aborted before the mailbox RX ring is ever
set up, so no frame is received and no ACK is awaited.
There is no per-msg-type unregister in struct nbl_channel_ops, so a
correct partial rollback cannot be expressed with the current interface.
Would adding an unregister for a single msg_type, or simply leaving the
already-registered handlers in place on failure, be preferable to tearing
down the shared table?
[ ... ]
> @@ -104,14 +644,24 @@ int nbl_disp_init(struct nbl_adapter *adapter)
> adapter->core.disp_mgt = disp_mgt;
> adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
>
> + ret = nbl_disp_setup_msg(disp_mgt);
> + if (ret)
> + return ret;
> +
[ ... ]
> void nbl_disp_remove(struct nbl_adapter *adapter)
> {
> - /* All dispatch objects allocated via devm */
> + /*
> + * All dispatch objects allocated via devm
> + * All message handlers will be cleaned up inside channel layer
> + * nbl_chan_remove_common() at final device tear-down
> + */
> }
[Severity: Low]
Since the five responders registered by nbl_disp_setup_msg() are never
unregistered, can one of them run after the resource layer has been
stopped? In nbl_core_remove() the order is nbl_disp_remove() ->
nbl_res_remove_leonis() (nbl_intr_mgt_stop() frees the per-function
interrupt arrays and clears res_mgt->intr_mgt) -> nbl_chan_remove_common().
Reading the later commits in the series, this looks closed:
nbl_dev_remove() runs first in nbl_core_remove(), and
nbl_dev_remove_common_dev() does
cancel_work_sync(&common_dev->clean_mbx_task);
nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
which sets chan_info->shutdn, and nbl_chan_recv_msg() bails out on
if (READ_ONCE(chan_info->shutdn))
return;
so no responder can execute once the mailbox queue is torn down. Is that
dependency worth spelling out in the comment, i.e. that the safety of the
no-op relies on the dev layer quiescing the channel before
nbl_res_remove_leonis() runs, rather than on nbl_chan_remove_common()?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-09-07 12:38 ` [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 4 potential
issue(s) to consider:
Critical: 0 · High: 1 · Medium: 1 · Low: 2
- [High] nbl_dev_remove() (drivers/net/ethernet/nebula-
matrix/nbl/nbl_core/nbl_dev.c) calls nbl_dev_remove_ctrl_dev() ->…
- [Medium] nbl_dev_remove_ctrl_dev() -> deinit_module() clears
driver_status, which per nbl_hw_deinit_module()'s own documentation…
- [Low] If disp_ops->init_module() fails, nbl_dev_setup_ctrl_dev()
returns the error and nbl_dev_init() unwinds only the common dev (goto…
- [Low] The int return of nbl_dev_remove_chan_queue() - which propagates
nbl_chan_teardown_queue()'s -ETIMEDOUT, documented as 'callers must…
--
net/nebula-matrix: add common/ctrl dev init/remove operation
This adds the nbl_dev core layer with nbl_dev_setup_common_dev() and
nbl_dev_setup_ctrl_dev() plus their teardown counterparts, and wires
nbl_dev_init() / nbl_dev_remove() into nbl_core_init() / nbl_core_remove().
A few questions about the teardown ordering and the error paths below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> new file mode 100644
> index 0000000000000..7813ff31a3cfc
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> @@ -0,0 +1,233 @@
[ ... ]
> +/* ---------- Channel config ---------- */
> +static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + struct nbl_common_info *common = dev_mgt->common;
> +
> + if (!chan_ops->check_queue_exist(priv, chan_type))
> + return;
> +
> + chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
> +}
[ ... ]
> +static int nbl_dev_remove_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + int ret = 0;
> +
> + if (chan_ops->check_queue_exist(priv, chan_type))
> + ret = chan_ops->teardown_queue(priv, chan_type);
> +
> + return ret;
> +}
[ ... ]
> +static void nbl_dev_remove_common_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> +
> + if (!common_dev)
> + return;
> + cancel_work_sync(&common_dev->clean_mbx_task);
> + nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
[Severity: Low]
nbl_dev_remove_chan_queue() returns int, and it propagates the -ETIMEDOUT
that nbl_chan_teardown_queue() reports when the inflight drain does not
complete:
ret = wait_event_timeout(chan_info->inflight_wait,
atomic_read(&chan_info->inflight_tx_cnt) == 0,
msecs_to_jiffies(5000));
if (!ret) { ... ret = -ETIMEDOUT; }
Both new callers, here and the err_cleanup path in
nbl_dev_setup_common_dev(), discard that status, and nbl_dev_remove() is
void, so nbl_core_remove() goes on to destroy the workqueue and the message
handler table while devres later frees the dmam_alloc_coherent() mailbox
rings.
As far as I can tell this is not reachable in the current tree: every
nbl_chan_send_msg() caller in the series runs either in the probe/remove
thread itself or inside clean_mbx_task, and cancel_work_sync() runs just
above, so inflight_tx_cnt is always 0 here today. Is the intent to keep the
status unchecked once asynchronous senders (netdev/ethtool paths) are added,
or should the timeout at least be logged?
> + nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
> +}
> +
> +static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> + int ret;
> +
> + ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
> + if (ret)
> + return ret;
[Severity: Low]
When init_module() fails there is no hardware rollback here, and
nbl_dev_init() only unwinds the common dev (goto setup_ctrl_dev_fail ->
nbl_dev_remove_common_dev()), so deinit_module() is never called.
nbl_hw_init_module() can fail partway through:
ret = nbl_dp_init(hw_mgt, eth_speed, eth_num);
if (ret)
return ret;
nbl_intf_init(hw_mgt);
nbl_hw_set_driver_status(hw_mgt, true);
and nbl_dp_init() programs nbl_dped_init()/nbl_uped_init() before the first
call that can return an error (nbl_shaping_init(), and later
nbl_ustore_init()).
Since nbl_hw_set_driver_status(hw_mgt, true) is only reached at the end,
the driver_status true->false transition that the patch relies on for
cleanup can never fire for this partially programmed chip-wide datapath
state. Should this path either roll back explicitly or call deinit_module()?
> +
> + nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> +
> + return 0;
> +}
> +
> +/*
> + * Tear down control device: deinit_module sets driver_status=false
> + * to notify firmware to clean all per-PF hardware state (including
> + * qinfo registers). The qinfo map programmed in setup_ctrl_dev is
> + * not explicitly cleared; firmware handles it on driver_status change.
> + */
> +static void nbl_dev_remove_ctrl_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> +
> + disp_ops->deinit_module(dev_mgt->disp_ops_tbl->priv);
> +}
[Severity: Medium]
What happens to sibling PFs that are still bound when only the control PF
is unbound?
nbl_dev_setup_chan_qinfo() -> nbl_chan_cfg_qinfo_map_table() programs the
mailbox routing for every hardware-active func_id, i.e. chip-global state
shared with the other PF driver instances.
Per the comment above, deinit_module() then makes firmware clean all per-PF
hardware state including those qinfo registers, but only this adapter's
mailbox queue is stopped afterwards by nbl_dev_remove_common_dev().
The resource layer already acknowledges that remote PFs can still be bound
at this point:
/*
* Tear down all MSI-X maps before devres releases the coherent
* tables. This is critical on the control PF, which may hold
* maps for remote PFs that are still bound.
*/
There is no refcount or cross-instance exclusion here, so after
"echo <control PF BDF> > /sys/bus/pci/drivers/nbl/unbind" the sibling
instances keep their mailbox rings armed and their software state marked
active while their routing entries are gone; their next dispatch RPC
(get_vsi_id, cfg/destroy_msix_map, set_mailbox_irq) can then only fail by
ACK timeout. Is that intentional, or should control-PF teardown be
serialized against still-bound siblings?
[ ... ]
> +int nbl_dev_init(struct nbl_adapter *adapter)
> +{
[ ... ]
> + ret = nbl_dev_setup_common_dev(adapter);
> + if (ret)
> + goto setup_err;
> +
> + if (common->has_ctrl) {
> + ret = nbl_dev_setup_ctrl_dev(adapter);
> + if (ret)
> + goto setup_ctrl_dev_fail;
> + }
> +
> + return 0;
> +setup_ctrl_dev_fail:
> + nbl_dev_remove_common_dev(adapter);
> +setup_err:
> + return ret;
> +}
> +
> +/*
> + * Teardown order: ctrl dev first, then common dev.
> + * nbl_dev_remove_ctrl_dev() notifies firmware to clean all per-PF state
> + * (including qinfo registers), so subsequent common dev queue cleanup
> + * will not trigger PCIe master abort or invalid register access.
> + */
> +void nbl_dev_remove(struct nbl_adapter *adapter)
> +{
> + struct nbl_common_info *common = &adapter->common;
> +
> + if (common->has_ctrl)
> + nbl_dev_remove_ctrl_dev(adapter);
> + nbl_dev_remove_common_dev(adapter);
> +}
[Severity: High]
Is this ordering the inverse of what the hardware layer asks for?
nbl_hw_deinit_module() documents the precondition:
/* Caller must ensure no new DMA is initiated after this point.
* The mailbox channel is stopped by nbl_chan_teardown_queue()
* before this function is called, so no in-flight mailbox DMA
* remains. */
nbl_hw_set_driver_status(hw_mgt, false);
But nbl_chan_teardown_queue() is the only place the mailbox hardware queues
are actually stopped and senders drained:
WRITE_ONCE(chan_info->shutdn, true);
...
/* After all TX drained, stop hardware queue */
nbl_chan_stop_queue(chan_mgt);
and it runs after deinit_module() in this path:
nbl_dev_remove()
nbl_dev_remove_ctrl_dev()
disp_ops->deinit_module() /* driver_status = false */
nbl_dev_remove_common_dev()
nbl_dev_remove_chan_queue()
nbl_chan_teardown_queue() /* shutdn, drain, stop queue */
At the end of the series the remove path is nbl_remove() -> nbl_core_stop()
-> nbl_core_remove() -> nbl_dev_remove(), and nbl_dev_stop() only frees the
IRQ and the MSI-X map, so the mailbox rings are still enabled with posted RX
descriptors when driver_status is cleared. Can firmware then wipe the
routing/queue state while the chip can still DMA into those rings, for
example for a message another PF sends to the control PF?
The commit message also states the opposite of the code:
No completion handshake exists in the
current hardware revision; the caller must ensure no new DMA is
initiated after deinit (the mailbox channel is stopped first).
The mailbox channel is stopped last here, not first. Should the common dev
teardown run before deinit_module(), or should nbl_dev_remove_ctrl_dev()
stop the channel itself before calling deinit_module()?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation
2026-09-07 12:38 ` [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
@ 2026-09-11 3:41 ` netdev-bot+sashiko
0 siblings, 0 replies; 20+ messages in thread
From: netdev-bot+sashiko @ 2026-09-11 3:41 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
kuba, horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, linux-kernel
Thank you for your contribution! Sashiko AI review found 8 potential
issue(s) to consider:
Critical: 0 · High: 0 · Medium: 5 · Low: 3
- [Medium] nbl_dev_setup_common_dev() (drivers/net/ethernet/nebula-
matrix/nbl/nbl_core/nbl_dev.c:304-314) drops the `if…
- [Medium] nbl_dev_stop() (nbl_dev.c:510-521) and the nbl_dev_start()
err_destroy_map unwind (nbl_dev.c:472-488) only log a failed…
- [Medium] nbl_dev_enable_mailbox_irq() (nbl_dev.c:80-99) publishes the
software interrupt-mode flag before the hardware interrupt exists: it…
- [Medium] In nbl_dev_start() (nbl_dev.c:446-448) a failure of
nbl_dev_cfg_msix_map() jumps to err_cancel_work, deliberately skipping…
- [Medium] The device is enabled with pcim_enable_device() in
nbl_probe(), which makes MSI/MSI-X vector release devres-managed:…
- [Low] The err_destroy_map comment added to nbl_dev_start()
(nbl_dev.c:480-481) states "If the RPC fails the device may remain…
- [Low] The comment in nbl_dev_init_msix_cnt() (nbl_dev.c:22-24) states
that the mailbox vector is "allocated in nbl_dev_start() via…
- [Low] Several newly added call sites in nbl_dev.c have continuation
lines that do not line up with the open parenthesis (checkpatch CHECK:…
--
net/nebula-matrix: add common dev start/stop operation
This adds nbl_dev_start() and nbl_dev_stop() to configure the hardware
MSI-X map, allocate MSI-X vectors, request the mailbox interrupt and tear
all of that down again. Thin nbl_core_start() / nbl_core_stop() wrappers
are hooked into nbl_probe() and nbl_remove(). The VSI/ETH identity fetch
in nbl_dev_setup_common_dev() is also changed to run unconditionally.
A few questions below on the interrupt enable ordering, the rollback paths
and some of the new comments.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> index 7813ff31a3cfc..af003a240e7ba 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
[ ... ]
> @@ -17,6 +28,189 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
> msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
> }
>
[ ... ]
> +static int nbl_dev_enable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
> +{
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_dev_common *dev_common = dev_mgt->common_dev;
> + struct nbl_msix_info *msix_info = &dev_common->msix_info;
> + u16 local_vec_id;
> +
> + if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
> + return 0;
> +
> + local_vec_id =
> + msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
> + chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
> + NBL_CHAN_IRQ_RDY,
> + NBL_CHAN_TYPE_MAILBOX, true);
> +
> + return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
> + local_vec_id, true);
> +}
[Severity: Medium]
Is this ordering the reverse of the invariant documented a few lines below
in nbl_dev_disable_mailbox_irq()? NBL_CHAN_IRQ_RDY is published here
before the hardware interrupt is armed, and on a non-control PF
set_mailbox_irq() is itself an ack-requesting mailbox RPC
(nbl_disp_chan_set_mailbox_irq_req()).
With IRQ_RDY already set, nbl_chan_send_msg() takes the sleeping branch:
if (test_bit(NBL_CHAN_IRQ_RDY, chan_info->state)) {
while (!READ_ONCE(wait_head->acked)) {
ret = wait_event_timeout(wait_head->wait_queue, ...);
That branch has no queue_work(clean_task) fallback, unlike the polling
else-branch, so the ring is only drained by the very mailbox MSI-X this RPC
is supposed to arm. nbl_hw_cfg_mailbox_qinfo() clears MSIX_IDX and
MSIX_IDX_VALID at init and only nbl_hw_set_mailbox_irq() sets them again.
If the responder's nbl_disp_chan_set_mailbox_irq_resp() takes an error
branch, it still sends the ack without arming the requester's routing:
if (src_id >= disp_mgt->common->max_pf) { err = NBL_CHAN_RESP_PERM_DENY; goto ack_out; }
...
ack_out:
nbl_chan_fill_ack_info(&chan_ack, src_id, NBL_CHAN_MSG_MAILBOX_SET_IRQ, msg_id, err, NULL, 0);
Can that wakeup then be lost, so the sender burns the full
NBL_CHAN_ACK_WAIT_TIME and reports -ETIMEDOUT instead of the remote error,
with the late ack later hitting a slot already marked
NBL_MBX_STATUS_TIMEOUT? That looks like exactly the "Channel waiting ack
failed" / "Skip ack with invalid status" class of message the disable-path
comment says it wants to avoid.
Would arming the hardware while still in polling mode and setting
NBL_CHAN_IRQ_RDY afterwards be the mirror of the disable path?
[Severity: Low]
This isn't a bug, but the continuation line here is indented past the open
parenthesis, which checkpatch flags as "Alignment should match open
parenthesis". The same applies to the set_mailbox_irq() call in
nbl_dev_disable_mailbox_irq(), the cfg_msix_map() call in
nbl_dev_cfg_msix_map(), and the reworked get_eth_id() call in
nbl_dev_setup_common_dev().
[ ... ]
> +static int nbl_dev_init_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
> +{
> + struct nbl_dev_common *dev_common = dev_mgt->common_dev;
> + struct nbl_msix_info *msix_info = &dev_common->msix_info;
> + struct nbl_common_info *common = dev_mgt->common;
> + int needed = 0;
> + int err;
> + int i;
> +
> + for (i = 0; i < NBL_MSIX_TYPE_MAX; i++)
> + needed += msix_info->serv_info[i].num;
> +
> + err = pci_alloc_irq_vectors(common->pdev, needed, needed,
> + PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
[Severity: Low]
With this function added, is the existing comment in nbl_dev_init_msix_cnt()
still accurate?
/* mailbox vector allocated in nbl_dev_start() via
* nbl_dev_request_mailbox_irq()
*/
The vectors are allocated here by pci_alloc_irq_vectors(), while
nbl_dev_request_mailbox_irq() only resolves an already allocated vector via
pci_irq_vector() and attaches a handler with request_irq(). The matching
release is likewise in nbl_dev_clear_interrupt_scheme(), not in
nbl_dev_free_mailbox_irq(). The commit message itself lists allocation as
step 2 and the IRQ request as step 3, so should that comment be updated to
point at nbl_dev_init_interrupt_scheme()?
> + if (err < 0) {
> + dev_err(common->dev,
> + "pci_alloc_irq_vectors failed, err = %d\n", err);
> + return err;
> + }
> +
> + return 0;
> +}
> +
> +static void nbl_dev_clear_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
> +{
> + struct nbl_common_info *common = dev_mgt->common;
> +
> + pci_free_irq_vectors(common->pdev);
> +}
[Severity: Medium]
nbl_probe() enables the device with pcim_enable_device(), which makes MSI-X
release devres-managed: pci_alloc_irq_vectors() -> __pci_enable_msix_range()
-> pci_setup_msi_context() -> pcim_setup_msi_release() registers
pcim_msi_release(), and that action calls pci_free_irq_vectors() itself on
detach.
Is the explicit call here safe in that case? The kernel-doc of
pci_free_irq_vectors() in drivers/pci/msi/api.c says:
* WARNING: Do not call this function if the device has been enabled
* with pcim_enable_device(). In that case, IRQ vectors are automatically
* managed via pcim_msi_release() and calling pci_free_irq_vectors() can
* lead to double-free issues.
Today the second call appears to be absorbed by the !dev->msix_enabled early
return in pci_disable_msix(), but should the driver either use unmanaged
pci_enable_device() with explicit cleanup, or leave the release to the
managed lifecycle?
[ ... ]
> @@ -99,17 +301,18 @@ static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
> INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
> nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
> &common_dev->clean_mbx_task);
> - if (common->has_ctrl) {
> - ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
> - if (ret)
> - goto err_cleanup;
> - ret = disp_ops->get_eth_id(priv, common->vsi_id,
> - &common->eth_num,
> - &common->eth_id,
> - &common->logic_eth_id);
> - if (ret)
> - goto err_cleanup;
> - }
> + /*
> + * Even if has_ctrl=false (no dedicated control PF channel), we fetch
> + * VSI/ETH info via regular mailbox message instead of dedicated
> + * control command.
> + */
> + ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
> + if (ret)
> + goto err_cleanup;
> + ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
> + &common->eth_id, &common->logic_eth_id);
> + if (ret)
> + goto err_cleanup;
[Severity: Medium]
Dropping the has_ctrl guard makes every non-control PF issue two
synchronous mailbox RPCs from inside its own probe():
nbl_probe() -> nbl_core_init() -> nbl_dev_init() ->
nbl_dev_setup_common_dev() -> nbl_disp_chan_get_vsi_id_req() ->
nbl_chan_send_msg(dstid = common->mgt_pf = 0)
and the transport error propagates straight out:
ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
if (ret < 0)
return ret;
What happens if function 0 is not bound yet? The mailbox QINFO entries used
to route a non-control PF's traffic are programmed only by the control PF:
nbl_chan_cfg_qinfo_map_table() {
hw_ops->get_host_pf_mask(p, &pf_mask);
for (func_id = 0; func_id < NBL_MAX_PF; func_id++)
if (!(pf_mask & (1 << func_id)))
hw_ops->cfg_mailbox_qinfo(p, func_id, bus, devid, func_id);
}
and that is only reached from nbl_dev_setup_ctrl_dev(). So before PF0's
probe has run, does the non-control PF get -ETIMEDOUT here and fail probe
permanently, with no -EPROBE_DEFER and no retry?
This is reachable without special hardware, for example unbinding PF0 and
then binding another PF, PF0 probe failing for an unrelated reason, or
async_probe making PF1 race ahead of PF0.
The commit message says:
The control PF's responder is reachable once its own probe has
completed and the mailbox queue is operational.
Should the identity fetch be deferred to a later point such as
nbl_dev_start() or ndo_open(), or should probe return -EPROBE_DEFER when
the manager PF has not answered? This cross-PF probe dependency also looks
unrelated to the patch subject.
>
> dev_mgt->common_dev = common_dev;
> nbl_dev_init_msix_cnt(dev_mgt);
> @@ -231,3 +434,96 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
> nbl_dev_remove_ctrl_dev(adapter);
> nbl_dev_remove_common_dev(adapter);
> }
> +
> +/* ---------- Dev start process ---------- */
> +int nbl_dev_start(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int cleanup_ret;
> + int ret;
> +
> + ret = nbl_dev_cfg_msix_map(dev_mgt);
> + if (ret)
> + goto err_cancel_work;
[Severity: Medium]
Does skipping nbl_dev_destroy_msix_map() on this branch leak resources on
the manager PF?
On a non-control PF, cfg_msix_map is an ack-requesting RPC
(nbl_disp_chan_cfg_msix_map_req()), and `if (ret < 0) return ret;` cannot
tell "remote never configured" from "remote configured but the ack was lost
or timed out". On the responder side the commit point has no rollback:
nbl_res_intr_cfg_msix_map() {
intr_mgt->func_intr_res[func_id].interrupts = tmp_interrupts;
...
hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
true, official_tbl->dma, common->hw_bus,
common->devid, common->function);
}
so the coherent MSI-X map table, the interrupts[] array, the global vector
IDs taken from intr_net_bmap/intr_other_bmap and an armed FUNCTION_MSIX_MAP
entry all stay committed for the requesting function.
The requester never sends NBL_CHAN_MSG_DESTROY_MSIX_MAP afterwards, since
nbl_probe() only runs nbl_core_remove() when nbl_core_start() fails:
core_start_err:
nbl_core_remove(adapter);
Calling nbl_dev_destroy_msix_map() on this path looks harmless, as
__nbl_res_intr_destroy_msix_map() returns 0 immediately when interrupts is
NULL. Would jumping to err_destroy_map here be more consistent with the
commit message claim that "a failure at any stage leaves the device and
kernel in a consistent state"?
> +
> + ret = nbl_dev_init_interrupt_scheme(dev_mgt);
> + if (ret)
> + goto err_destroy_map;
> +
> + ret = nbl_dev_request_mailbox_irq(dev_mgt);
> + if (ret)
> + goto err_destroy_map;
> +
> + ret = nbl_dev_enable_mailbox_irq(dev_mgt);
> + if (ret)
> + goto err_disable_irq;
> +
> + return 0;
> +
> +err_disable_irq:
> + cleanup_ret = nbl_dev_disable_mailbox_irq(dev_mgt);
> + if (cleanup_ret)
> + dev_err(dev_mgt->common->dev,
> + "rollback: disable mailbox IRQ failed: %d\n",
> + cleanup_ret);
> + nbl_dev_free_mailbox_irq(dev_mgt);
> +err_destroy_map:
> + /*
> + * Destroy device-side MSI-X map BEFORE releasing kernel-side
> + * vectors. This masks all hardware vectors and clears the
> + * pcompleter map entry, so no MSI-X message can fire after
> + * pci_free_irq_vectors() releases the vectors.
> + *
> + * For non-control PFs this is a polling-mode mailbox RPC
> + * (IRQ_RDY already cleared by disable above, or never set).
> + * If the RPC fails the device may remain armed; pci_clear_master()
> + * in nbl_remove() is the final safety net.
> + */
[Severity: Low]
Is nbl_remove() actually reached on this path? A nbl_dev_start() failure
propagates through nbl_core_start() into nbl_probe(), which does
`goto core_start_err` and falls through to the adapter_init_err label:
core_start_err:
nbl_core_remove(adapter);
adapter_init_err:
pci_clear_master(pdev);
return err;
The PCI core does not call .remove after a failed probe, so the safety net
described here appears to live in nbl_probe()'s error label rather than in
nbl_remove(). Could the comment name the label that really runs?
> + cleanup_ret = nbl_dev_destroy_msix_map(dev_mgt);
> + if (cleanup_ret)
> + dev_err(dev_mgt->common->dev,
> + "rollback: destroy MSI-X map failed: %d\n",
> + cleanup_ret);
> + nbl_dev_clear_interrupt_scheme(dev_mgt);
> +err_cancel_work:
> + /*
> + * destroy_msix_map() polling send may requeue clean_mbx_task.
> + * Drain before returning on all rollback paths.
> + */
> + cancel_work_sync(&common_dev->clean_mbx_task);
> + return ret;
> +}
> +
> +void nbl_dev_stop(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int ret;
> +
> + ret = nbl_dev_disable_mailbox_irq(dev_mgt);
> + if (ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to disable mailbox IRQ: %d\n", ret);
> + nbl_dev_free_mailbox_irq(dev_mgt);
> +
> + /*
> + * Destroy hardware MSI-X map BEFORE releasing kernel-side
> + * vectors. Masks all device vectors and clears pcompleter
> + * map entry first; pci_free_irq_vectors() then runs with no
> + * possibility of a stale MSI-X message arriving.
> + */
> + ret = nbl_dev_destroy_msix_map(dev_mgt);
> + if (ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to destroy MSI-X map: %d\n", ret);
> +
> + nbl_dev_clear_interrupt_scheme(dev_mgt);
[Severity: Medium]
Does the code above match the comment when nbl_dev_destroy_msix_map()
fails? The failure is only logged and pci_free_irq_vectors() runs anyway,
so the device-side pcompleter map entry stays valid and the vectors stay
armed while the host side releases them. The same applies to the
err_destroy_map path in nbl_dev_start().
On a non-control PF that RPC can return -ETIMEDOUT, -EREMOTEIO or -EPERM
whenever the manager PF cannot answer (PF0 unbound first, lost ack,
responder error).
The fallback named in both the comment and the changelog also looks like it
runs too late. In nbl_remove():
nbl_core_stop(adapter);
nbl_core_remove(adapter);
pci_clear_master(pdev);
pci_clear_master() only executes after the vectors are already freed. And
nbl_res_intr_cfg_msix_map() tags the map-table DMA with the control PF's
BDF:
/*
* cfg_msix_map uses the control PF's own BDF (common->hw_bus etc.),
* not the target function's BDF. ...
*/
hw_ops->cfg_msix_map(..., true, official_tbl->dma, common->hw_bus, common->devid, common->function);
so can clearing bus mastering on the departing non-control PF quiesce that
DMA at all? Should the teardown either retry or keep the vectors until the
device-side map is confirmed destroyed, and should the comment and the
changelog sentence "pci_clear_master() in nbl_remove() provides the final
safety net if a teardown RPC to the manager PF cannot complete" be adjusted?
> +
> + /*
> + * destroy_msix_map() sends ack-requested messages which may
> + * requeue clean_mbx_task via polling send path. Drain work
> + * after the operation.
> + */
> + cancel_work_sync(&common_dev->clean_mbx_task);
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> index 9896c1b49be2b..02cb35d32012f 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
[ ... ]
> @@ -127,7 +137,13 @@ static int nbl_probe(struct pci_dev *pdev,
> goto adapter_init_err;
> }
> pci_set_drvdata(pdev, adapter);
> + err = nbl_core_start(adapter);
> + if (err)
> + goto core_start_err;
> +
> return 0;
> +core_start_err:
> + nbl_core_remove(adapter);
> adapter_init_err:
> pci_clear_master(pdev);
> return err;
[ ... ]
Thanks for considering these.
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260907123848.30256-1-illusion.wang%40nebula-matrix.com
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2026-09-11 3:41 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-07 12:38 [PATCH v27 net-next 00/10] nbl driver for Nebulamatrix NICs illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 01/10] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 02/10] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 03/10] net/nebula-matrix: add channel layer illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 04/10] net/nebula-matrix: add common resource implementation illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 05/10] net/nebula-matrix: add intr " illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 06/10] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 07/10] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
2026-09-07 12:38 ` [PATCH v27 net-next 08/10] net/nebula-matrix: dispatch: implement channel RPC framework and serialize hardware ops illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 09/10] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
2026-09-07 12:38 ` [PATCH v27 net-next 10/10] net/nebula-matrix: add common dev start/stop operation illusion.wang
2026-09-11 3:41 ` netdev-bot+sashiko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox