* [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs
@ 2026-08-07 4:44 illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 01/11] net/nebula-matrix: add minimum nbl build framework illusion.wang
` (10 more replies)
0 siblings, 11 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch series represents the first phase. We plan to integrate it in
two phases: the first phase covers mailbox and chip configuration,
while the second phase involves net dev configuration.
Together, they will provide basic PF-based Ethernet port transmission and
reception capabilities.
After that, we will consider other features, such as ethtool support,
flow management, adminq messaging, VF support, debugfs support, etc.
To ensure compatibility, our architecture is divided into the following
layers:
1. Dev Layer (Device Layer)
The top-level business logic layer where all operations are
device-centric. Every operation is performed relative to the device
context. The intergration of base functions encompasses:
management(ctrl only for leonis pf0), network(net_dev,this time not
contained),common.
2. Dispatch Layer
The distribution from services to specific data operations is mainly
divided into two types: direct pass-through and handling by the
management PF. It shields the upper layer from the differences in
specific underlying locations.
It describes the processing locations and paths of the services.
3. Resource Layer
Handles tasks dispatched from Dispatch Layer. These tasks fall into two
categories:
3.1 Hardware control
The Resource Layer further invokes the HW Layer when hardware access is
needed, as only the HW Layer has OS-level privileges.
3.2 Software resource management
Operations like packet statistics collection that don't require hardware
access.
4. HW Layer (Hardware Layer)
Serves the Resource Layer by interacting with different hardware
chipsets.Writes to hardware registers to drive the hardware based on
Resource Layer directives.
5. Channel Layer
Handle communication between PF0(has ctrl func) and other PF,and provide
basic interaction channels.
6. Common Layer
Provides fundamental services
changes v23->v24
Link to v23:https://lore.kernel.org/netdev/20260731094242.2655-1-illusion.wang@nebula-matrix.com/
1.AI review issues
2.Issues found by Jakub
changes v22->v23
Link to v22:https://lore.kernel.org/netdev/20260723040110.91410-1-illusion.wang@nebula-matrix.com/
AI review issues
chages v21->v22
Link to v21:https://lore.kernel.org/netdev/20260708064742.35391-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v20->v21
Link to v20:https://lore.kernel.org/netdev/20260630010718.4346-1-illusion.wang@nebula-matrix.com/
I have dropped the old Patch 2 titled "add our driver architecture"
Add new Patch 2 "add core driver architecture and HW layer initialization".
Split original patch8 into three separate patches as suggested
changes v19->v20
Link to v19:https://lore.kernel.org/netdev/20260617044702.2439-1-illusion.wang@nebula-matrix.com/
Starting from V20, I have dropped the old Patch 3 titled "P4
configuration invoked during chip initialization". This functionality
will be reimplemented in next phase using the request_firmware() API
with an external firmware blob.
changes v18->v19
Link to v18:https://lore.kernel.org/netdev/20260611044916.2383-1-illusion.wang@nebula-matrix.com/
changes v17->v18
Link to v17:https://lore.kernel.org/netdev/20260601093149.25905-1-illusion.wang@nebula-matrix.com/
changes v16->v17
Link to v16:https://lore.kernel.org/netdev/20260526035453.2359-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v15->v16
Link to v15:https://lore.kernel.org/netdev/20260520032950.4874-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v14->v15
Link to v14:https://lore.kernel.org/netdev/20260513011649.4404-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v13->v14
Link to v13:https://lore.kernel.org/netdev/20260428114910.2616-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v12->v13
Link to v12:https://lore.kernel.org/netdev/20260415033608.2438-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v11->v12
Link to v11:https://lore.kernel.org/netdev/20260408093739.56001-1-illusion.wang@nebula-matrix.com/
AI review issues
changes v10->v11
Link to v10:https://lore.kernel.org/netdev/20260401022318.28550-1-illusion.wang@nebula-matrix.com/
1.Issues found by Mohsin
2.AI review issues
changes v9->v10
Link to v9:https://lore.kernel.org/netdev/20260325040048.2313-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
changes v8->v9
Link to v8:https://lore.kernel.org/netdev/20260317034533.5600-1-illusion.wang@nebula-matrix.com/
1.Issues found by Jakub
2.AI review issue
Changes v7→v8
Link to v7:https://lore.kernel.org/netdev/20260310120959.22015-1-illusion.wang@nebula-matrix.com/
1.Issues found by Paolo
Changes v6->v7
Link to v6:https://lore.kernel.org/netdev/20260306033451.5196-1-illusion.wang@nebula-matrix.com/
1.Issue found by Jakub
2.AI review issue
Changes v5->v6
Link to V5:https://lore.kernel.org/netdev/20260226073840.3222-1-illusion.wang@nebula-matrix.com/
1.put all standard linux includes files the .c file which needs it & others
--Andrew
2.AI review issue
Changes v4->v5
Link to V4:https://lore.kernel.org/netdev/20260206021608.85381-1-illusion.wang@nebula-matrix.com/
1.change nbl_core to nbl & change ** pointers to *pointers & others
--Andrew
2.AI review issue
Changes v3->v4
Link to v3: https://lore.kernel.org/netdev/20260123011804.31263-1-illusion.wang@nebula-matrix.com
1.cut down to part of a mini driver(mailbox and chip init)
--Jakub Kicinski Simon Horman(some sort of staged approached)
2.modify issues found by ai.
3. Reverse Christmas tree/nbl_err/devm_kfree/remove some macros/
void type to real type/others
--Andrew Lunn
4.change deprecated pci_enable_msix_range to pci_alloc_irq_vectors
5.delete service layer
6.the style of kconfig---Randy Dunlap
7.add to Documentation/networking/device_drivers/ethernet/index.rst
--Simon Horman
Changes v2 →v3
Link to v2: https://lore.kernel.org/netdev/20260109100146.63569-1-illusion.wang@nebula-matrix.com/
1.cut down to a mini driver:
delete vf support
use promisc mode to cut down flow management
drop patch15 in v2
delete adminq msg
delete abnormal handling
delete some unimportant interfaces
2.modify issues found by ai review
Changes v1->v2
Link to v1: https://lore.kernel.org/netdev/20251223035113.31122-1-illusion.wang@nebula-matrix.com/
1.Format Issues and Compilation Issues
- Paolo Abeni
2.add sysfs patch and drop coexisting patch
- Andrew Lunn
3.delete some unimportant ndo operations
4.add machine generated headers patch
5.Modify the issues found in patch1-2 and apply the same fixes to other
patches
6.modify issues found by nipa
illusion wang (11):
net/nebula-matrix: add minimum nbl build framework
net/nebula-matrix: add core driver architecture and HW layer
initialization
net/nebula-matrix: add channel wire opcode enum definitions
net/nebula-matrix: add channel layer
net/nebula-matrix: add common resource implementation
net/nebula-matrix: add intr resource implementation
net/nebula-matrix: add chip-wide hardware init/deinit implementation
net/nebula-matrix: dispatch: add control-level routing core
infrastructure
net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops
mutex
net/nebula-matrix: add common/ctrl dev init/remove operation
net/nebula-matrix: add common dev start/stop operation
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +
MAINTAINERS | 10 +
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 31 +
drivers/net/ethernet/nebula-matrix/Makefile | 6 +
.../net/ethernet/nebula-matrix/nbl/Makefile | 15 +
.../nbl/nbl_channel/nbl_channel.c | 1142 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 173 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 206 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 29 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 45 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 499 +++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 57 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 633 +++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 26 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 23 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 1086 ++++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 355 +++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 321 +++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 12 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 78 ++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 405 ++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 150 +++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 109 ++
.../nbl/nbl_include/nbl_def_channel.h | 379 ++++++
.../nbl/nbl_include/nbl_def_common.h | 69 +
.../nbl/nbl_include/nbl_def_dev.h | 16 +
.../nbl/nbl_include/nbl_def_dispatch.h | 42 +
.../nbl/nbl_include/nbl_def_hw.h | 53 +
.../nbl/nbl_include/nbl_def_resource.h | 38 +
.../nbl/nbl_include/nbl_include.h | 49 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 247 ++++
36 files changed, 6368 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
--
2.47.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 01/11] net/nebula-matrix: add minimum nbl build framework
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
` (9 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
This patch adds the minimum build infrastructure:
1. Add Kconfig, Makefile and MAINTAINERS entries;
2. Add corresponding driver documentation;
3. Add PCI driver skeleton with empty stubs for nbl driver.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../device_drivers/ethernet/index.rst | 1 +
.../ethernet/nebula-matrix/nbl.rst | 28 +++++
MAINTAINERS | 10 ++
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/nebula-matrix/Kconfig | 31 ++++++
drivers/net/ethernet/nebula-matrix/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/Makefile | 6 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 13 +++
.../nbl/nbl_include/nbl_include.h | 14 +++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 100 ++++++++++++++++++
11 files changed, 211 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
create mode 100644 drivers/net/ethernet/nebula-matrix/Kconfig
create mode 100644 drivers/net/ethernet/nebula-matrix/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/Makefile
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
index 786a23c84b90..4d74b954b0ba 100644
--- a/Documentation/networking/device_drivers/ethernet/index.rst
+++ b/Documentation/networking/device_drivers/ethernet/index.rst
@@ -47,6 +47,7 @@ Contents:
meta/fbnic
microsoft/netvsc
mucse/rnpgbe
+ nebula-matrix/nbl
netronome/nfp
pensando/ionic
pensando/ionic_rdma
diff --git a/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
new file mode 100644
index 000000000000..ff38302968c5
--- /dev/null
+++ b/Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst
@@ -0,0 +1,28 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================================================================
+Linux Base Driver for Nebula-matrix m18110-NIC/m18000-NIC family
+================================================================
+
+Overview:
+=========
+The m18110-NIC/m18000-NIC is a series of network interface cards for the Data
+Center Area.
+
+The driver supports link-speed 100GbE/25GE/10GE.
+
+m18110-NIC/m18000-NIC devices support MSI-X interrupt vector for each Tx/Rx
+queue and interrupt moderation.
+
+m18110-NIC/m18000-NIC devices support also various offload features such as
+checksum offload, Receive-Side Scaling(RSS).
+
+Support
+=======
+
+For more information about m18110-NIC/m18000-NIC, please visit the following URL:
+https://www.nebula-matrix.com/snic_s1000_en
+
+If an issue is identified with the released source code on the supported kernel
+with a supported adapter, email the specific information related to the issue to
+open@nebula-matrix.com.
diff --git a/MAINTAINERS b/MAINTAINERS
index 00e7b26e0a23..ab0910a6836d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18597,6 +18597,16 @@ F: Documentation/devicetree/bindings/hwmon/nuvoton,nct7363.yaml
F: Documentation/hwmon/nct7363.rst
F: drivers/hwmon/nct7363.c
+NEBULA-MATRIX ETHERNET DRIVER (nebula-matrix)
+M: Illusion Wang <illusion.wang@nebula-matrix.com>
+M: Dimon Zhao <dimon.zhao@nebula-matrix.com>
+M: Alvin Wang <alvin.wang@nebula-matrix.com>
+M: Sam Chen <sam.chen@nebula-matrix.com>
+L: netdev@vger.kernel.org
+S: Maintained
+F: Documentation/networking/device_drivers/ethernet/nebula-matrix/
+F: drivers/net/ethernet/nebula-matrix/
+
NETCONSOLE
M: Breno Leitao <leitao@debian.org>
S: Maintained
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 78c79ad7bba5..5c21d02c4e11 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -130,6 +130,7 @@ config FEALNX
source "drivers/net/ethernet/ni/Kconfig"
source "drivers/net/ethernet/natsemi/Kconfig"
+source "drivers/net/ethernet/nebula-matrix/Kconfig"
source "drivers/net/ethernet/netronome/Kconfig"
source "drivers/net/ethernet/8390/Kconfig"
source "drivers/net/ethernet/nvidia/Kconfig"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index bba55d9af387..accce62a79a6 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -67,6 +67,7 @@ obj-$(CONFIG_NET_VENDOR_MUCSE) += mucse/
obj-$(CONFIG_NET_VENDOR_MYRI) += myricom/
obj-$(CONFIG_FEALNX) += fealnx.o
obj-$(CONFIG_NET_VENDOR_NATSEMI) += natsemi/
+obj-$(CONFIG_NET_VENDOR_NEBULA_MATRIX) += nebula-matrix/
obj-$(CONFIG_NET_VENDOR_NETRONOME) += netronome/
obj-$(CONFIG_NET_VENDOR_NI) += ni/
obj-$(CONFIG_NET_VENDOR_NVIDIA) += nvidia/
diff --git a/drivers/net/ethernet/nebula-matrix/Kconfig b/drivers/net/ethernet/nebula-matrix/Kconfig
new file mode 100644
index 000000000000..f16e9663eaed
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Kconfig
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Nebula-matrix network device configuration
+#
+
+config NET_VENDOR_NEBULA_MATRIX
+ bool "Nebula-matrix devices"
+ default y
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Nebula-matrix cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_NEBULA_MATRIX
+
+config NBL
+ tristate "Nebula-matrix Ethernet Controller m18110/m18000 support"
+ depends on PCI && (64BIT || COMPILE_TEST) && !CPU_BIG_ENDIAN
+ help
+ This driver supports Nebula-matrix Ethernet Controller m18110/m18000
+ Family of devices.
+
+ More specific information on configuring the driver is in
+ <file:Documentation/networking/device_drivers/ethernet/nebula-matrix/nbl.rst>.
+
+ To compile this driver as a module, choose M here. The module
+ will be called nbl.
+
+endif # NET_VENDOR_NEBULA_MATRIX
diff --git a/drivers/net/ethernet/nebula-matrix/Makefile b/drivers/net/ethernet/nebula-matrix/Makefile
new file mode 100644
index 000000000000..42cdf2db8f0c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Nebula-matrix network device drivers.
+#
+
+obj-$(CONFIG_NBL) += nbl/
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
new file mode 100644
index 000000000000..b90fba239401
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Nebula Matrix Limited.
+
+obj-$(CONFIG_NBL) := nbl.o
+
+nbl-objs += nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
new file mode 100644
index 000000000000..59f5b92944cc
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CORE_H_
+#define _NBL_CORE_H_
+
+enum {
+ NBL_CAP_HAS_NET_BIT,
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
new file mode 100644
index 000000000000..cd99f96e1568
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INCLUDE_H_
+#define _NBL_INCLUDE_H_
+
+#include <linux/types.h>
+
+/* ------ Basic definitions ------- */
+#define NBL_DRIVER_NAME "nbl"
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
new file mode 100644
index 000000000000..7c969c854bdc
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/module.h>
+#include <linux/bits.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_core.h"
+
+static int nbl_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ return -ENODEV;
+}
+
+static void nbl_remove(struct pci_dev *pdev)
+{
+}
+
+/*
+ * PCI Device IDs for Leonis/NBL Network Controllers
+ *
+ * Vendor ID: 0x1F0F
+ * SNIC v3r1 product Device IDs range: 0x3403-0x3412
+ */
+#define NBL_VENDOR_ID 0x1F0F
+
+#define NBL_DEVICE_ID_M18110 0x3403
+#define NBL_DEVICE_ID_M18110_LX 0x3404
+#define NBL_DEVICE_ID_M18110_BASE_T 0x3405
+#define NBL_DEVICE_ID_M18110_LX_BASE_T 0x3406
+#define NBL_DEVICE_ID_M18110_OCP 0x3407
+#define NBL_DEVICE_ID_M18110_LX_OCP 0x3408
+#define NBL_DEVICE_ID_M18110_BASE_T_OCP 0x3409
+#define NBL_DEVICE_ID_M18110_LX_BASE_T_OCP 0x340a
+#define NBL_DEVICE_ID_M18000 0x340b
+#define NBL_DEVICE_ID_M18000_LX 0x340c
+#define NBL_DEVICE_ID_M18000_BASE_T 0x340d
+#define NBL_DEVICE_ID_M18000_LX_BASE_T 0x340e
+#define NBL_DEVICE_ID_M18000_OCP 0x340f
+#define NBL_DEVICE_ID_M18000_LX_OCP 0x3410
+#define NBL_DEVICE_ID_M18000_BASE_T_OCP 0x3411
+#define NBL_DEVICE_ID_M18000_LX_BASE_T_OCP 0x3412
+
+/* All below IDs belong to Leonis ASIC family, different form-factor variants,
+ * share the same hardware initialization flow without differentiated ops.
+ */
+static const struct pci_device_id nbl_id_table[] = {
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18110_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ { PCI_DEVICE(NBL_VENDOR_ID, NBL_DEVICE_ID_M18000_LX_BASE_T_OCP),
+ .driver_data = BIT(NBL_CAP_HAS_NET_BIT) },
+ /* required as sentinel */
+ { }
+};
+MODULE_DEVICE_TABLE(pci, nbl_id_table);
+
+static struct pci_driver nbl_driver = {
+ .name = NBL_DRIVER_NAME,
+ .id_table = nbl_id_table,
+ .probe = nbl_probe,
+ .remove = nbl_remove,
+};
+
+module_pci_driver(nbl_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Nebula Matrix Network Driver");
+MODULE_AUTHOR("Illusion Wang <illusion.wang@nebula-matrix.com>");
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 01/11] net/nebula-matrix: add minimum nbl build framework illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-14 19:33 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 03/11] net/nebula-matrix: add channel wire opcode enum definitions illusion.wang
` (8 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add the fundamental driver architecture framework and Leonis hardware
layer initialization for NBL NIC family.
- nbl_adapter/nbl_core/nbl_common_info core device context
- PCI probe/remove entry and basic device capability parsing
- 64/32-bit DMA mask fallback and PCI master setup
- Leonis hardware BAR resource request and ioremap logic
This patch establishes the lowest HW layer and core infrastructure,
preparing for subsequent device implementations.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 3 +-
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 18 +++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 152 ++++++++++++++++++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 ++
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 33 ++++
.../nbl/nbl_include/nbl_def_common.h | 32 ++++
.../nbl/nbl_include/nbl_def_hw.h | 17 ++
.../nbl/nbl_include/nbl_include.h | 9 ++
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 97 ++++++++++-
9 files changed, 373 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index b90fba239401..caa863d3a582 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,4 +3,5 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_main.o
+nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 59f5b92944cc..e2e33fbb848a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -6,8 +6,26 @@
#ifndef _NBL_CORE_H_
#define _NBL_CORE_H_
+#include <linux/pci.h>
+#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_common.h"
+
enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_core {
+ struct nbl_hw_mgt *hw_mgt;
+};
+
+struct nbl_adapter {
+ struct pci_dev *pdev;
+ struct nbl_core core;
+ struct nbl_common_info common;
+};
+
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param);
+void nbl_core_remove(struct nbl_adapter *adapter);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
new file mode 100644
index 000000000000..d66d7ca8a2bb
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/bitfield.h>
+#include "nbl_hw_leonis.h"
+
+/* Structure starts here, adding an op should not modify anything below */
+static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
+{
+ struct device *dev = common->dev;
+ struct nbl_hw_mgt *hw_mgt;
+
+ hw_mgt = devm_kzalloc(dev, sizeof(*hw_mgt), GFP_KERNEL);
+ if (!hw_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ hw_mgt->common = common;
+
+ return hw_mgt;
+}
+
+static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
+ const char *name)
+{
+ int bar;
+ int ret;
+
+ for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
+ if (!(mask & BIT(bar)))
+ continue;
+ ret = pcim_request_region(pdev, bar, name);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct pci_dev *pdev = common->pdev;
+ struct nbl_hw_mgt *hw_mgt = NULL;
+ resource_size_t bar_len;
+ u32 bar_mask;
+ int ret;
+
+ hw_mgt = nbl_hw_setup_hw_mgt(common);
+ if (IS_ERR(hw_mgt)) {
+ ret = PTR_ERR(hw_mgt);
+ goto setup_mgt_fail;
+ }
+ bar_mask = BIT(NBL_MEMORY_BAR) | BIT(NBL_MAILBOX_BAR);
+ ret = nbl_pcim_request_selected_bars(pdev, bar_mask, NBL_DRIVER_NAME);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Request memory bar failed, err = %d\n",
+ ret);
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MEMORY_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MEMORY_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MEMORY BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (common->has_ctrl) {
+ /*
+ * Hardware layout: MEMORY BAR total size is 64M.
+ * The front NBL_HW_REG_SPACE_SIZE (32M) holds module
+ * registers and table entries. The trailing NBL_RDMA_NOTIFY_LEN
+ * bytes within this 32M region are reserved exclusively for
+ * RDMA notification hardware logic.
+ * Ethernet driver must avoid mapping this reserved range to
+ * prevent x86 PAT attribute aliasing conflict between eth UC
+ * mapping and RDMA driver WC mapping.
+ * Mapping range starts at offset 0, length equals
+ * NBL_HW_REG_SPACE_SIZE - NBL_RDMA_NOTIFY_LEN.
+ */
+ if (bar_len < NBL_HW_REG_SPACE_SIZE) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pa too small for ctrl reg space\n",
+ &bar_len);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ /*
+ * Spec: Front 32MiB of 64M MEM BAR contains register space.
+ * Skip trailing NBL_RDMA_NOTIFY_LEN bytes inside 32M boundary
+ * never map that RDMA exclusive partition to avoid PAT conflict
+ */
+ hw_mgt->hw_size = NBL_HW_REG_SPACE_SIZE - NBL_RDMA_NOTIFY_LEN;
+ hw_mgt->hw_addr =
+ pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ } else {
+ if (bar_len < NBL_REG_NET_ONLY_LEN) {
+ dev_err(&pdev->dev,
+ "MEMORY BAR len %pa too small for net only reg space\n",
+ &bar_len);
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->hw_size = NBL_REG_NET_ONLY_LEN;
+ hw_mgt->hw_addr = pcim_iomap(pdev, NBL_MEMORY_BAR,
+ hw_mgt->hw_size);
+ }
+ if (!hw_mgt->hw_addr) {
+ dev_err(&pdev->dev, "MEMORY BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+
+ bar_len = pci_resource_len(pdev, NBL_MAILBOX_BAR);
+ if (!(pci_resource_flags(pdev, NBL_MAILBOX_BAR) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "MAILBOX BAR is not memory resource\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ if (bar_len == 0) {
+ dev_err(&pdev->dev, "MAILBOX BAR length is zero\n");
+ ret = -EINVAL;
+ goto setup_mgt_fail;
+ }
+ hw_mgt->mailbox_bar_hw_addr = pcim_iomap(pdev, NBL_MAILBOX_BAR,
+ bar_len);
+ if (!hw_mgt->mailbox_bar_hw_addr) {
+ dev_err(&pdev->dev, "MAILBOX BAR pcim_iomap failed\n");
+ ret = -EIO;
+ goto setup_mgt_fail;
+ }
+
+ adapter->core.hw_mgt = hw_mgt;
+
+ return 0;
+
+setup_mgt_fail:
+ return ret;
+}
+
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter)
+{
+ /* All BAR mappings & PCI regions are managed by pcim/devres,
+ * no manual iounmap / release required
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
new file mode 100644
index 000000000000..77c67b67ba31
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_LEONIS_H_
+#define _NBL_HW_LEONIS_H_
+
+#include <linux/types.h>
+
+#include "../../nbl_include/nbl_include.h"
+#include "../nbl_hw_reg.h"
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
new file mode 100644
index 000000000000..1a199357e85f
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_HW_REG_H_
+#define _NBL_HW_REG_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_MEMORY_BAR 0
+#define NBL_MAILBOX_BAR 2
+#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
+#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+/*
+ * PCI MEMORY BAR total size: 64MiB.
+ * First 32MiB: module registers + table entry address range.
+ * Region after 32MiB is unused for ethernet driver.
+ */
+#define NBL_HW_REG_SPACE_SIZE (32ULL << 20)
+
+struct nbl_hw_mgt {
+ struct nbl_common_info *common;
+ u8 __iomem *hw_addr;
+ u8 __iomem *mailbox_bar_hw_addr;
+ resource_size_t hw_size;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
new file mode 100644
index 000000000000..5b7ec68d620b
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_COMMON_H_
+#define _NBL_DEF_COMMON_H_
+
+#include <linux/types.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include "nbl_include.h"
+
+struct nbl_common_info {
+ struct pci_dev *pdev;
+ struct device *dev;
+ u32 msg_enable;
+ u16 vsi_id;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 eth_num;
+
+ u8 function;
+ u8 devid;
+ u8 bus;
+ u8 hw_bus;
+
+ u8 has_ctrl;
+ u8 has_net;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
new file mode 100644
index 000000000000..517cfa5c755e
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_HW_H_
+#define _NBL_DEF_HW_H_
+
+#include <linux/types.h>
+
+struct nbl_hw_mgt;
+struct nbl_adapter;
+
+int nbl_hw_init_leonis(struct nbl_adapter *adapter);
+void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index cd99f96e1568..70f4f4b4c49c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,5 +10,14 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+struct nbl_func_caps {
+ u32 has_ctrl:1;
+ u32 has_net:1;
+ u32 rsv:30;
+};
+
+struct nbl_init_param {
+ struct nbl_func_caps caps;
+};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 7c969c854bdc..bd9f682b8323 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,16 +8,111 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
+ struct nbl_init_param *param)
+{
+ struct nbl_common_info *common;
+ struct nbl_adapter *adapter;
+ int ret;
+
+ adapter = devm_kzalloc(&pdev->dev, sizeof(*adapter), GFP_KERNEL);
+ if (!adapter)
+ return ERR_PTR(-ENOMEM);
+
+ adapter->pdev = pdev;
+ common = &adapter->common;
+
+ common->pdev = pdev;
+ common->dev = &pdev->dev;
+ common->has_ctrl = param->caps.has_ctrl;
+ common->has_net = param->caps.has_net;
+ common->function = PCI_FUNC(pdev->devfn);
+ common->devid = PCI_SLOT(pdev->devfn);
+ common->bus = pdev->bus->number;
+
+ ret = nbl_hw_init_leonis(adapter);
+ if (ret)
+ goto hw_init_fail;
+
+ return adapter;
+hw_init_fail:
+ return ERR_PTR(ret);
+}
+
+void nbl_core_remove(struct nbl_adapter *adapter)
+{
+ nbl_hw_remove_leonis(adapter);
+}
+
+static void nbl_get_func_param(struct pci_dev *pdev, kernel_ulong_t driver_data,
+ struct nbl_init_param *param)
+{
+ param->caps.has_net = !!(driver_data & BIT(NBL_CAP_HAS_NET_BIT));
+
+ /*
+ * Only PF0 possesses ctrl capability; all PFs share identical PCI
+ * device ID, so we handle this via special function ID judgement.
+ */
+ if ((PCI_FUNC(pdev->devfn) == 0) && !pdev->is_virtfn)
+ param->caps.has_ctrl = 1;
+}
+
static int nbl_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
- return -ENODEV;
+ struct nbl_init_param param = { { 0 } };
+ struct device *dev = &pdev->dev;
+ struct nbl_adapter *adapter;
+ int err;
+
+ err = pcim_enable_device(pdev);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to enable PCI dev, err=%d\n", err);
+ return err;
+ }
+
+ nbl_get_func_param(pdev, id->driver_data, ¶m);
+
+ err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_dbg(dev, "Configure DMA 64 bit mask failed, err = %d\n",
+ err);
+ err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(dev,
+ "Configure DMA 32 bit mask failed, err = %d\n",
+ err);
+ goto configure_dma_err;
+ }
+ }
+ pci_set_master(pdev);
+
+ adapter = nbl_core_init(pdev, ¶m);
+ if (IS_ERR(adapter)) {
+ dev_err(dev, "Nbl adapter init fail: %pe\n", adapter);
+ err = PTR_ERR(adapter);
+ goto adapter_init_err;
+ }
+ pci_set_drvdata(pdev, adapter);
+ return 0;
+adapter_init_err:
+ pci_clear_master(pdev);
+configure_dma_err:
+ return err;
}
static void nbl_remove(struct pci_dev *pdev)
{
+ struct nbl_adapter *adapter = pci_get_drvdata(pdev);
+
+ if (adapter)
+ nbl_core_remove(adapter);
+
+ pci_clear_master(pdev);
}
/*
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 03/11] net/nebula-matrix: add channel wire opcode enum definitions
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 01/11] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer illusion.wang
` (7 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add enum nbl_chan_msg_type into nbl_def_channel.h, defining all
PF/VF/firmware channel wire opcodes used for inter-component mailbox
communication.
Each enumerator carries an explicit fixed numeric value, acting as stable
wire-format message ID shared between driver and firmware.
ABI constraint: New opcodes must only be appended before
NBL_CHAN_MSG_MAILBOX_MAX.
Reordering, inserting or deleting existing entries will break
cross-version driver-firmware interoperability. Any modification to
existing opcode assignments requires coordinated firmware ABI change.
This commit introduces the opcode ID enumeration. Message payload
structures and complete channel layer logic will be added in subsequent
separate patches.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../nbl/nbl_include/nbl_def_channel.h | 242 ++++++++++++++++++
1 file changed, 242 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
new file mode 100644
index 000000000000..be374667c338
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_CHANNEL_H_
+#define _NBL_DEF_CHANNEL_H_
+
+/*
+ * Mailbox wire opcodes
+ * Every opcode is assigned explicit fixed numeric value, stable wire ABI
+ * shared between driver and firmware.
+ * Firmware/driver wire compatibility rule:
+ * New entries must only be appended before NBL_CHAN_MSG_MAILBOX_MAX.
+ * Reordering / inserting / deleting existing items will break cross-version
+ * interoperability. Any change to existing enumerators requires matching
+ * firmware ABI adjustment.
+ */
+enum nbl_chan_msg_type {
+ NBL_CHAN_MSG_ACK = 0,
+ NBL_CHAN_MSG_ADD_MACVLAN = 1,
+ NBL_CHAN_MSG_DEL_MACVLAN = 2,
+ NBL_CHAN_MSG_ADD_MULTI_RULE = 3,
+ NBL_CHAN_MSG_DEL_MULTI_RULE = 4,
+ NBL_CHAN_MSG_SETUP_MULTI_GROUP = 5,
+ NBL_CHAN_MSG_REMOVE_MULTI_GROUP = 6,
+ NBL_CHAN_MSG_REGISTER_NET = 7,
+ NBL_CHAN_MSG_UNREGISTER_NET = 8,
+ NBL_CHAN_MSG_ALLOC_TXRX_QUEUES = 9,
+ NBL_CHAN_MSG_FREE_TXRX_QUEUES = 10,
+ NBL_CHAN_MSG_SETUP_QUEUE = 11,
+ NBL_CHAN_MSG_REMOVE_ALL_QUEUES = 12,
+ NBL_CHAN_MSG_CFG_DSCH = 13,
+ NBL_CHAN_MSG_SETUP_CQS = 14,
+ NBL_CHAN_MSG_REMOVE_CQS = 15,
+ NBL_CHAN_MSG_CFG_QDISC_MQPRIO = 16,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP = 17,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP = 18,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ = 19,
+ NBL_CHAN_MSG_GET_GLOBAL_VECTOR = 20,
+ NBL_CHAN_MSG_GET_VSI_ID = 21,
+ NBL_CHAN_MSG_SET_PROMISC_MODE = 22,
+ NBL_CHAN_MSG_GET_FIRMWARE_VERSION = 23,
+ NBL_CHAN_MSG_GET_QUEUE_ERR_STATS = 24,
+ NBL_CHAN_MSG_GET_COALESCE = 25,
+ NBL_CHAN_MSG_SET_COALESCE = 26,
+ NBL_CHAN_MSG_SET_SPOOF_CHECK_ADDR = 27,
+ NBL_CHAN_MSG_SET_VF_SPOOF_CHECK = 28,
+ NBL_CHAN_MSG_GET_RXFH_INDIR_SIZE = 29,
+ NBL_CHAN_MSG_GET_RXFH_INDIR = 30,
+ NBL_CHAN_MSG_GET_RXFH_RSS_KEY = 31,
+ NBL_CHAN_MSG_GET_RXFH_RSS_ALG_SEL = 32,
+ NBL_CHAN_MSG_GET_HW_CAPS = 33,
+ NBL_CHAN_MSG_GET_HW_STATE = 34,
+ NBL_CHAN_MSG_REGISTER_RDMA = 35,
+ NBL_CHAN_MSG_UNREGISTER_RDMA = 36,
+ NBL_CHAN_MSG_GET_REAL_HW_ADDR = 37,
+ NBL_CHAN_MSG_GET_REAL_BDF = 38,
+ NBL_CHAN_MSG_GRC_PROCESS = 39,
+ NBL_CHAN_MSG_SET_SFP_STATE = 40,
+ NBL_CHAN_MSG_SET_ETH_LOOPBACK = 41,
+ NBL_CHAN_MSG_CHECK_ACTIVE_VF = 42,
+ NBL_CHAN_MSG_GET_PRODUCT_FLEX_CAP = 43,
+ NBL_CHAN_MSG_ALLOC_KTLS_TX_INDEX = 44,
+ NBL_CHAN_MSG_FREE_KTLS_TX_INDEX = 45,
+ NBL_CHAN_MSG_CFG_KTLS_TX_KEYMAT = 46,
+ NBL_CHAN_MSG_ALLOC_KTLS_RX_INDEX = 47,
+ NBL_CHAN_MSG_FREE_KTLS_RX_INDEX = 48,
+ NBL_CHAN_MSG_CFG_KTLS_RX_KEYMAT = 49,
+ NBL_CHAN_MSG_CFG_KTLS_RX_RECORD = 50,
+ NBL_CHAN_MSG_ADD_KTLS_RX_FLOW = 51,
+ NBL_CHAN_MSG_DEL_KTLS_RX_FLOW = 52,
+ NBL_CHAN_MSG_ALLOC_IPSEC_TX_INDEX = 53,
+ NBL_CHAN_MSG_FREE_IPSEC_TX_INDEX = 54,
+ NBL_CHAN_MSG_ALLOC_IPSEC_RX_INDEX = 55,
+ NBL_CHAN_MSG_FREE_IPSEC_RX_INDEX = 56,
+ NBL_CHAN_MSG_CFG_IPSEC_TX_SAD = 57,
+ NBL_CHAN_MSG_CFG_IPSEC_RX_SAD = 58,
+ NBL_CHAN_MSG_ADD_IPSEC_TX_FLOW = 59,
+ NBL_CHAN_MSG_DEL_IPSEC_TX_FLOW = 60,
+ NBL_CHAN_MSG_ADD_IPSEC_RX_FLOW = 61,
+ NBL_CHAN_MSG_DEL_IPSEC_RX_FLOW = 62,
+ NBL_CHAN_MSG_NOTIFY_IPSEC_HARD_EXPIRE = 63,
+ NBL_CHAN_MSG_GET_MBX_IRQ_NUM = 64,
+ NBL_CHAN_MSG_CLEAR_FLOW = 65,
+ NBL_CHAN_MSG_CLEAR_QUEUE = 66,
+ NBL_CHAN_MSG_GET_ETH_ID = 67,
+ NBL_CHAN_MSG_SET_OFFLOAD_STATUS = 68,
+ NBL_CHAN_MSG_INIT_OFLD = 69,
+ NBL_CHAN_MSG_INIT_CMDQ = 70,
+ NBL_CHAN_MSG_DESTROY_CMDQ = 71,
+ NBL_CHAN_MSG_RESET_CMDQ = 72,
+ NBL_CHAN_MSG_INIT_FLOW = 73,
+ NBL_CHAN_MSG_DEINIT_FLOW = 74,
+ NBL_CHAN_MSG_OFFLOAD_FLOW_RULE = 75,
+ NBL_CHAN_MSG_GET_ACL_SWITCH = 76,
+ NBL_CHAN_MSG_GET_VSI_GLOBAL_QUEUE_ID = 77,
+ NBL_CHAN_MSG_INIT_REP = 78,
+ NBL_CHAN_MSG_GET_LINE_RATE_INFO = 79,
+ NBL_CHAN_MSG_REGISTER_NET_REP = 80,
+ NBL_CHAN_MSG_UNREGISTER_NET_REP = 81,
+ NBL_CHAN_MSG_REGISTER_ETH_REP = 82,
+ NBL_CHAN_MSG_UNREGISTER_ETH_REP = 83,
+ NBL_CHAN_MSG_REGISTER_UPCALL_PORT = 84,
+ NBL_CHAN_MSG_UNREGISTER_UPCALL_PORT = 85,
+ NBL_CHAN_MSG_GET_PORT_STATE = 86,
+ NBL_CHAN_MSG_SET_PORT_ADVERTISING = 87,
+ NBL_CHAN_MSG_GET_MODULE_INFO = 88,
+ NBL_CHAN_MSG_GET_MODULE_EEPROM = 89,
+ NBL_CHAN_MSG_GET_LINK_STATE = 90,
+ NBL_CHAN_MSG_NOTIFY_LINK_STATE = 91,
+ NBL_CHAN_MSG_GET_QUEUE_CXT = 92,
+ NBL_CHAN_MSG_CFG_LOG = 93,
+ NBL_CHAN_MSG_INIT_VDPAQ = 94,
+ NBL_CHAN_MSG_DESTROY_VDPAQ = 95,
+ NBL_CHAN_MSG_GET_UPCALL_PORT = 96,
+ NBL_CHAN_MSG_NOTIFY_ETH_REP_LINK_STATE = 97,
+ NBL_CHAN_MSG_SET_ETH_MAC_ADDR = 98,
+ NBL_CHAN_MSG_GET_FUNCTION_ID = 99,
+ NBL_CHAN_MSG_GET_CHIP_TEMPERATURE = 100,
+ NBL_CHAN_MSG_DISABLE_HW_FLOW = 101,
+ NBL_CHAN_MSG_ENABLE_HW_FLOW = 102,
+ NBL_CHAN_MSG_SET_UPCALL_RULE = 103,
+ NBL_CHAN_MSG_UNSET_UPCALL_RULE = 104,
+ NBL_CHAN_MSG_GET_REG_DUMP = 105,
+ NBL_CHAN_MSG_GET_REG_DUMP_LEN = 106,
+ NBL_CHAN_MSG_CFG_LAG_HASH_ALGORITHM = 107,
+ NBL_CHAN_MSG_CFG_LAG_MEMBER_FWD = 108,
+ NBL_CHAN_MSG_CFG_LAG_MEMBER_LIST = 109,
+ NBL_CHAN_MSG_CFG_LAG_MEMBER_UP_ATTR = 110,
+ NBL_CHAN_MSG_ADD_LAG_FLOW = 111,
+ NBL_CHAN_MSG_DEL_LAG_FLOW = 112,
+ NBL_CHAN_MSG_SWITCHDEV_INIT_CMDQ = 113,
+ NBL_CHAN_MSG_SWITCHDEV_DEINIT_CMDQ = 114,
+ NBL_CHAN_MSG_SET_TC_FLOW_INFO = 115,
+ NBL_CHAN_MSG_UNSET_TC_FLOW_INFO = 116,
+ NBL_CHAN_MSG_INIT_ACL = 117,
+ NBL_CHAN_MSG_UNINIT_ACL = 118,
+ NBL_CHAN_MSG_CFG_LAG_MCC = 119,
+ NBL_CHAN_MSG_REGISTER_VSI2Q = 120,
+ NBL_CHAN_MSG_SETUP_Q2VSI = 121,
+ NBL_CHAN_MSG_REMOVE_Q2VSI = 122,
+ NBL_CHAN_MSG_SETUP_RSS = 123,
+ NBL_CHAN_MSG_REMOVE_RSS = 124,
+ NBL_CHAN_MSG_GET_REP_QUEUE_INFO = 125,
+ NBL_CHAN_MSG_CTRL_PORT_LED = 126,
+ NBL_CHAN_MSG_NWAY_RESET = 127,
+ NBL_CHAN_MSG_SET_INTL_SUPPRESS_LEVEL = 128,
+ NBL_CHAN_MSG_GET_ETH_STATS = 129,
+ NBL_CHAN_MSG_GET_MODULE_TEMPERATURE = 130,
+ NBL_CHAN_MSG_GET_BOARD_INFO = 131,
+ NBL_CHAN_MSG_GET_P4_USED = 132,
+ NBL_CHAN_MSG_GET_VF_BASE_VSI_ID = 133,
+ NBL_CHAN_MSG_ADD_LLDP_FLOW = 134,
+ NBL_CHAN_MSG_DEL_LLDP_FLOW = 135,
+ NBL_CHAN_MSG_CFG_ETH_BOND_INFO = 136,
+ NBL_CHAN_MSG_CFG_DUPPKT_MCC = 137,
+ NBL_CHAN_MSG_ADD_ND_UPCALL_FLOW = 138,
+ NBL_CHAN_MSG_DEL_ND_UPCALL_FLOW = 139,
+ NBL_CHAN_MSG_GET_BOARD_ID = 140,
+ NBL_CHAN_MSG_SET_SHAPING_DPORT_VLD = 141,
+ NBL_CHAN_MSG_SET_DPORT_FC_TH_VLD = 142,
+ NBL_CHAN_MSG_REGISTER_RDMA_BOND = 143,
+ NBL_CHAN_MSG_UNREGISTER_RDMA_BOND = 144,
+ NBL_CHAN_MSG_RESTORE_NETDEV_QUEUE = 145,
+ NBL_CHAN_MSG_RESTART_NETDEV_QUEUE = 146,
+ NBL_CHAN_MSG_RESTORE_HW_QUEUE = 147,
+ NBL_CHAN_MSG_KEEP_ALIVE = 148,
+ NBL_CHAN_MSG_GET_BASE_MAC_ADDR = 149,
+ NBL_CHAN_MSG_CFG_BOND_SHAPING = 150,
+ NBL_CHAN_MSG_CFG_BGID_BACK_PRESSURE = 151,
+ NBL_CHAN_MSG_ALLOC_KT_BLOCK = 152,
+ NBL_CHAN_MSG_FREE_KT_BLOCK = 153,
+ NBL_CHAN_MSG_GET_USER_QUEUE_INFO = 154,
+ NBL_CHAN_MSG_GET_ETH_BOND_INFO = 155,
+ NBL_CHAN_MSG_CLEAR_ACCEL_FLOW = 156,
+ NBL_CHAN_MSG_SET_BRIDGE_MODE = 157,
+ NBL_CHAN_MSG_GET_VF_FUNCTION_ID = 158,
+ NBL_CHAN_MSG_NOTIFY_LINK_FORCED = 159,
+ NBL_CHAN_MSG_SET_PMD_DEBUG = 160,
+ NBL_CHAN_MSG_REGISTER_FUNC_MAC = 161,
+ NBL_CHAN_MSG_SET_TX_RATE = 162,
+ NBL_CHAN_MSG_REGISTER_FUNC_LINK_FORCED = 163,
+ NBL_CHAN_MSG_GET_LINK_FORCED = 164,
+ NBL_CHAN_MSG_REGISTER_FUNC_VLAN = 165,
+ NBL_CHAN_MSG_GET_FD_FLOW = 166,
+ NBL_CHAN_MSG_GET_FD_FLOW_CNT = 167,
+ NBL_CHAN_MSG_GET_FD_FLOW_ALL = 168,
+ NBL_CHAN_MSG_GET_FD_FLOW_MAX = 169,
+ NBL_CHAN_MSG_REPLACE_FD_FLOW = 170,
+ NBL_CHAN_MSG_REMOVE_FD_FLOW = 171,
+ NBL_CHAN_MSG_CFG_FD_FLOW_STATE = 172,
+ NBL_CHAN_MSG_REGISTER_FUNC_RATE = 173,
+ NBL_CHAN_MSG_NOTIFY_VLAN = 174,
+ NBL_CHAN_MSG_GET_XDP_QUEUE_INFO = 175,
+ NBL_CHAN_MSG_STOP_ABNORMAL_SW_QUEUE = 176,
+ NBL_CHAN_MSG_STOP_ABNORMAL_HW_QUEUE = 177,
+ NBL_CHAN_MSG_NOTIFY_RESET_EVENT = 178,
+ NBL_CHAN_MSG_ACK_RESET_EVENT = 179,
+ NBL_CHAN_MSG_GET_VF_VSI_ID = 180,
+ NBL_CHAN_MSG_CONFIGURE_QOS = 181,
+ NBL_CHAN_MSG_GET_PFC_BUFFER_SIZE = 182,
+ NBL_CHAN_MSG_SET_PFC_BUFFER_SIZE = 183,
+ NBL_CHAN_MSG_GET_VF_STATS = 184,
+ NBL_CHAN_MSG_REGISTER_FUNC_TRUST = 185,
+ NBL_CHAN_MSG_NOTIFY_TRUST = 186,
+ NBL_CHAN_MSG_CHECK_VF_IS_ACTIVE = 187,
+ NBL_CHAN_MSG_GET_ETH_ABNORMAL_STATS = 188,
+ NBL_CHAN_MSG_GET_ETH_CTRL_STATS = 189,
+ NBL_CHAN_MSG_GET_PAUSE_STATS = 190,
+ NBL_CHAN_MSG_GET_ETH_MAC_STATS = 191,
+ NBL_CHAN_MSG_GET_FEC_STATS = 192,
+ NBL_CHAN_MSG_CFG_MULTI_MCAST_RULE = 193,
+ NBL_CHAN_MSG_GET_LINK_DOWN_COUNT = 194,
+ NBL_CHAN_MSG_GET_LINK_STATUS_OPCODE = 195,
+ NBL_CHAN_MSG_GET_RMON_STATS = 196,
+ NBL_CHAN_MSG_REGISTER_PF_NAME = 197,
+ NBL_CHAN_MSG_GET_PF_NAME = 198,
+ NBL_CHAN_MSG_CONFIGURE_RDMA_BW = 199,
+ NBL_CHAN_MSG_SET_RATE_LIMIT = 200,
+ NBL_CHAN_MSG_SET_TC_WGT = 201,
+ NBL_CHAN_MSG_REMOVE_QUEUE = 202,
+ NBL_CHAN_MSG_GET_MIRROR_TABLE_ID = 203,
+ NBL_CHAN_MSG_CONFIGURE_MIRROR = 204,
+ NBL_CHAN_MSG_CONFIGURE_MIRROR_TABLE = 205,
+ NBL_CHAN_MSG_CLEAR_MIRROR_CFG = 206,
+ NBL_CHAN_MSG_MIRROR_OUTPUTPORT_NOTIFY = 207,
+ NBL_CHAN_MSG_CHECK_FLOWTABLE_SPEC = 208,
+ NBL_CHAN_MSG_CHECK_VF_IS_VDPA = 209,
+ NBL_CHAN_MSG_GET_VDPA_VF_STATS = 210,
+ NBL_CHAN_MSG_SET_RX_RATE = 211,
+ NBL_CHAN_MSG_GET_UVN_PKT_DROP_STATS = 212,
+ NBL_CHAN_MSG_GET_USTORE_PKT_DROP_STATS = 213,
+ NBL_CHAN_MSG_GET_USTORE_TOTAL_PKT_DROP_STATS = 214,
+ NBL_CHAN_MSG_SET_WOL = 215,
+ NBL_CHAN_MSG_INIT_VF_MSIX_MAP = 216,
+ NBL_CHAN_MSG_GET_ST_NAME = 217,
+ /* mailbox msg end */
+ NBL_CHAN_MSG_MAILBOX_MAX,
+};
+
+#endif
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (2 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 03/11] net/nebula-matrix: add channel wire opcode enum definitions illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-14 19:38 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 05/11] net/nebula-matrix: add common resource implementation illusion.wang
` (6 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
A channel management layer provides a structured approach to handle
communication between different components and drivers. Here's a summary
of its key functionalities:
1. Message Handling Framework
Message Registration: Functions (nbl_chan_register_msg) allow dynamic
registration of message handlers for specific message types, enabling
extensible communication protocols.
Message Sending/Acknowledgment: Core functions (nbl_chan_send_msg,
nbl_chan_send_ack) handle message transmission. It supports two modes:
fire-and-forget transmission, and synchronous send with ACK waiting.
Received ACK messages are processed via nbl_chan_recv_ack_msg.
TX slot allocation logic is implemented to limit concurrent outstanding
messages; returns -EAGAIN when all waiting slots are occupied.
Hash-Based Handler Lookup: A hash table (`handle_hash_tbl`) stores
message handlers for efficient O(1) lookup by message type.
The entire table is removed via `nbl_chan_remove_msg_handler` during
driver teardown; per-message-type unregister is not implemented
in this version.
Tx descriptor supports two data formats: small embedded payload inside
descriptor, and large payload via external DMA buffer.
2. Channel Types and Queue Management
Mailbox Channel: implemented for inter-PF communication (PF0 <-> other PFs).
Queue Initialization: Functions (nbl_chan_init_queue,
nbl_chan_init_tx_queue) allocate resources:
- TX/RX descriptors: dmam_alloc_coherent()
- TX/RX buffer metadata arrays (txq->buf, rxq->buf): devm_kcalloc()
Queue Teardown: nbl_chan_teardown_queue() stops hardware queues, cancels
pending work items (clean_task), and waits for in-flight send threads
to drain. It does NOT free DMA memory, which is released automatically
via devm on driver remove.
IMPORTANT - Resource Lifecycle Design:
DMA memory allocated with dmam_alloc_coherent() is intentionally NOT
freed in nbl_chan_teardown_queue(). The queues are allocated once
during driver probe and freed only during driver remove. Dynamic
per-PF queue teardown and re-creation during runtime is not supported.
Queue Configuration: Hardware-specific queue parameters (e.g., buffer
sizes, entry counts) are set via nbl_chan_config_queue, with hardware
interactions delegated to hw_ops.
RX completion relies on work_struct clean_task for descriptor processing.
3. Hardware Abstraction Layer (HW Ops)
Hardware-Specific Operations: The nbl_hw_ops structure abstracts
hardware interactions: queue configuration (config_mailbox_txq/rxq),
tail pointer doorbell updates (update_mailbox_queue_tail_ptr),
and PF mailbox routing table configuration.
NOTE: This initial implementation provides interrupt-driven receive path,
where actual RX message processing is scheduled into workqueue after
interrupt notification. Polling-based mailbox receive is not supported
in this patch and will be added in subsequent series.
And the optimization to requeue clean_task inside nbl_chan_clean_queue()
when processing budget is exhausted is deferred to follow-up patches.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 4 +-
.../nbl/nbl_channel/nbl_channel.c | 1068 +++++++++++++++++
.../nbl/nbl_channel/nbl_channel.h | 170 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 166 +++
.../nebula-matrix/nbl/nbl_common/nbl_common.h | 29 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 7 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 174 +++
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 56 +
.../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 45 +
.../nbl/nbl_include/nbl_def_channel.h | 104 ++
.../nbl/nbl_include/nbl_def_common.h | 15 +
.../nbl/nbl_include/nbl_def_hw.h | 19 +
.../nbl/nbl_include/nbl_include.h | 3 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 7 +
14 files changed, 1866 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index caa863d3a582..6dc1539cee1f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -3,5 +3,7 @@
obj-$(CONFIG_NBL) := nbl.o
-nbl-objs += nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+nbl-objs += nbl_common/nbl_common.o \
+ nbl_channel/nbl_channel.o \
+ nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
new file mode 100644
index 000000000000..3eb795199217
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
@@ -0,0 +1,1068 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/bitfield.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include <linux/dma-mapping.h>
+#include <linux/atomic.h>
+#include <linux/wait.h>
+#include "nbl_channel.h"
+
+static int nbl_chan_add_msg_handler(struct nbl_channel_mgt *chan_mgt,
+ u16 msg_type, nbl_chan_resp func,
+ void *priv)
+{
+ struct nbl_chan_msg_node_data handler = { 0 };
+ int ret;
+
+ handler.func = func;
+ handler.priv = priv;
+ ret = nbl_common_alloc_hash_node(chan_mgt->handle_hash_tbl, &msg_type,
+ &handler, NULL);
+
+ return ret;
+}
+
+static int nbl_chan_init_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_hash_tbl_key tbl_key = { 0 };
+
+ tbl_key.dev = common->dev;
+ tbl_key.key_size = sizeof(u16);
+ tbl_key.data_size = sizeof(struct nbl_chan_msg_node_data);
+ tbl_key.bucket_size = NBL_CHAN_HANDLER_TBL_BUCKET_SIZE;
+
+ chan_mgt->handle_hash_tbl = nbl_common_init_hash_table(&tbl_key);
+ if (!chan_mgt->handle_hash_tbl)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void nbl_chan_remove_msg_handler(struct nbl_channel_mgt *chan_mgt)
+{
+ if (!chan_mgt->handle_hash_tbl)
+ return;
+ nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl);
+ chan_mgt->handle_hash_tbl = NULL;
+}
+
+static void nbl_chan_init_queue_param(struct nbl_chan_info *chan_info,
+ u16 num_txq_entries, u16 num_rxq_entries,
+ u16 txq_buf_size, u16 rxq_buf_size)
+{
+ chan_info->num_txq_entries = num_txq_entries;
+ chan_info->num_rxq_entries = num_rxq_entries;
+ chan_info->txq_buf_size = txq_buf_size;
+ chan_info->rxq_buf_size = rxq_buf_size;
+ atomic_set(&chan_info->inflight_tx_cnt, 0);
+ WRITE_ONCE(chan_info->shutdown, false);
+}
+
+static int nbl_chan_init_tx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_txq_entries * sizeof(struct nbl_chan_tx_desc);
+ u16 i;
+
+ txq->desc.tx_desc =
+ dmam_alloc_coherent(dev, size, &txq->dma, GFP_KERNEL);
+ if (!txq->desc.tx_desc)
+ return -ENOMEM;
+
+ chan_info->wait = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*chan_info->wait), GFP_KERNEL);
+ if (!chan_info->wait)
+ return -ENOMEM;
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ init_waitqueue_head(&chan_info->wait[i].wait_queue);
+ chan_info->wait[i].status = NBL_MBX_STATUS_IDLE;
+ spin_lock_init(&chan_info->wait[i].status_lock);
+ }
+
+ txq->buf = devm_kcalloc(dev, chan_info->num_txq_entries,
+ sizeof(*txq->buf), GFP_KERNEL);
+ if (!txq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_rx_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = common->dev;
+ size_t size =
+ chan_info->num_rxq_entries * sizeof(struct nbl_chan_rx_desc);
+
+ rxq->desc.rx_desc =
+ dmam_alloc_coherent(dev, size, &rxq->dma, GFP_KERNEL);
+ if (!rxq->desc.rx_desc) {
+ dev_err_ratelimited(dev,
+ "Allocate DMA for chan rx descriptor ring failed\n");
+ return -ENOMEM;
+ }
+
+ rxq->buf = devm_kcalloc(dev, chan_info->num_rxq_entries,
+ sizeof(*rxq->buf), GFP_KERNEL);
+ if (!rxq->buf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int nbl_chan_init_queue(struct nbl_common_info *common,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_init_tx_queue(common, chan_info);
+ if (err)
+ return err;
+
+ err = nbl_chan_init_rx_queue(common, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_config_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info, bool tx)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ struct nbl_chan_ring *ring;
+ dma_addr_t addr;
+ int size_bwid;
+
+ if (tx)
+ ring = &chan_info->txq;
+ else
+ ring = &chan_info->rxq;
+ addr = ring->dma;
+ if (tx) {
+ size_bwid = ilog2(chan_info->num_txq_entries);
+ hw_ops->config_mailbox_txq(p, addr, size_bwid);
+ } else {
+ size_bwid = ilog2(chan_info->num_rxq_entries);
+ hw_ops->config_mailbox_rxq(p, addr, size_bwid);
+ }
+}
+
+static int nbl_chan_alloc_all_tx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ buf = &txq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->txq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan tx queue failed\n");
+ return -ENOMEM;
+ }
+ }
+
+ txq->next_to_clean = 0;
+ txq->next_to_use = 0;
+ txq->tail_ptr = 0;
+
+ return 0;
+}
+
+static void nbl_chan_cfg_qinfo_map_table(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
+ u32 pf_mask = 0;
+ u8 func_id;
+
+ hw_ops->get_host_pf_mask(p, &pf_mask);
+ for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
+ if (!(pf_mask & (1 << func_id)))
+ hw_ops->cfg_mailbox_qinfo(p, func_id, bus,
+ devid, func_id);
+ }
+}
+
+static int nbl_chan_alloc_all_rx_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_rx_desc *desc;
+ struct nbl_chan_buf *buf;
+ u16 i;
+
+ for (i = 0; i < chan_info->num_rxq_entries; i++) {
+ buf = &rxq->buf[i];
+ buf->va = dmam_alloc_coherent(dev, chan_info->rxq_buf_size,
+ &buf->pa, GFP_KERNEL);
+ if (!buf->va) {
+ dev_err_ratelimited(dev,
+ "Allocate buffer for chan rx queue failed\n");
+ goto err;
+ }
+ }
+
+ desc = rxq->desc.rx_desc;
+ /*
+ * RX ring adopts empty-ring convention: reserve one unused entry
+ * to distinguish queue FULL vs queue EMPTY.
+ * We initialize entries [0 ... num_rxq_entries - 2],
+ * leave the last entry uninitialized as the hole marker.
+ */
+ for (i = 0; i < chan_info->num_rxq_entries - 1; i++) {
+ buf = &rxq->buf[i];
+ desc[i].buf_addr = cpu_to_le64(buf->pa);
+ desc[i].buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+ desc[i].flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+ }
+
+ rxq->next_to_clean = 0;
+ /* Reserve last slot as hole to differentiate full/empty status */
+ rxq->next_to_use = chan_info->num_rxq_entries - 1;
+ rxq->tail_ptr = chan_info->num_rxq_entries - 1;
+
+ return 0;
+err:
+ return -ENOMEM;
+}
+
+static int nbl_chan_alloc_all_bufs(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ int err;
+
+ err = nbl_chan_alloc_all_tx_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_rx_bufs(chan_mgt, chan_info);
+
+ return err;
+}
+
+static void nbl_chan_stop_queue(struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+
+ hw_ops->stop_mailbox_rxq(chan_mgt->hw_ops_tbl->priv);
+ hw_ops->stop_mailbox_txq(chan_mgt->hw_ops_tbl->priv);
+}
+
+static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_chan_waitqueue_head *wait_head;
+ unsigned long wait_start, wait_timeout;
+ int ret = 0;
+ u16 i;
+
+ /* Step 1: Mark shutdown flag, reject all new send requests */
+ WRITE_ONCE(chan_info->shutdown, true);
+ /*
+ * Full memory barrier: Ensure the write to 'shutdown' becomes visible
+ * to all other CPUs BEFORE we start waiting on inflight_tx_cnt.
+ * Peer send_msg threads check shutdown flag prior to taking new work;
+ * this barrier prevents reordering that would let send paths observe
+ * inflight_tx_cnt == 0 but still accept new messages.
+ */
+ smp_mb();
+
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait_head = &chan_info->wait[i];
+ spin_lock_irq(&wait_head->status_lock);
+ /* Only wake threads that are actually waiting */
+ if (READ_ONCE(wait_head->status) == NBL_MBX_STATUS_WAITING) {
+ /* Mark as timeout so waking threads know to abort */
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->acked, 1);
+ WRITE_ONCE(wait_head->ack_err, (s32)-EIO);
+ wake_up(&wait_head->wait_queue);
+ }
+ spin_unlock_irq(&wait_head->status_lock);
+ }
+ /* Stop hardware queues */
+ nbl_chan_stop_queue(chan_mgt);
+
+ /*
+ * Step 2: Wait all in-flight send_msg threads exit FIRST.
+ * Guarantee no further queue_work() can be triggered during cleanup.
+ * Add timeout to avoid permanent blocking.
+ */
+ wait_start = jiffies;
+ wait_timeout = msecs_to_jiffies(5000);
+ while (atomic_read(&chan_info->inflight_tx_cnt) != 0) {
+ if (time_after(jiffies, wait_start + wait_timeout)) {
+ dev_warn(chan_mgt->common->dev,
+ "teardown: wait inflight_tx_cnt timeout, force continue\n");
+ ret = -ETIMEDOUT;
+ break;
+ }
+ cpu_relax();
+ usleep_range(1000, 2000);
+ }
+
+ /* Step3: All send paths drained, safely cancel cleanup work */
+ if (chan_info->clean_task) {
+ cancel_work_sync(chan_info->clean_task);
+ chan_info->clean_task = NULL;
+ }
+ return ret;
+}
+
+static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ int err;
+
+ if (READ_ONCE(chan_info->init_done))
+ return 0;
+ err = devm_mutex_init(common->dev, &chan_info->txq_lock);
+ if (err)
+ return err;
+ nbl_chan_init_queue_param(chan_info, NBL_CHAN_QUEUE_LEN,
+ NBL_CHAN_QUEUE_LEN, NBL_CHAN_BUF_LEN,
+ NBL_CHAN_BUF_LEN);
+ err = nbl_chan_init_queue(common, chan_info);
+ if (err)
+ return err;
+ err = nbl_chan_alloc_all_bufs(chan_mgt, chan_info);
+ if (err)
+ return err;
+ nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
+ nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+ WRITE_ONCE(chan_info->init_done, true);
+ return 0;
+}
+
+static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_tx_param *param)
+{
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct nbl_chan_tx_desc *tx_desc =
+ NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_use);
+ struct nbl_chan_buf *tx_buf =
+ NBL_CHAN_TX_RING_TO_BUF(txq, txq->next_to_use);
+
+ if (param->arg_len > NBL_CHAN_BUF_LEN - sizeof(*tx_desc))
+ return -EINVAL;
+
+ tx_desc->dstid = cpu_to_le16(param->dstid);
+ tx_desc->msg_type = cpu_to_le16(param->msg_type);
+ tx_desc->msgid = cpu_to_le16(param->msgid);
+
+ if (param->arg_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ memcpy(tx_buf->va, param->arg, param->arg_len);
+ tx_desc->buf_addr = cpu_to_le64(tx_buf->pa);
+ tx_desc->buf_len = cpu_to_le16(param->arg_len);
+ tx_desc->data_len = 0;
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ } else {
+ memset(tx_desc->data, 0, sizeof(tx_desc->data));
+ memset(&tx_desc->buf_addr, 0, sizeof(tx_desc->buf_addr));
+ memcpy(tx_desc->data, param->arg, param->arg_len);
+ tx_desc->buf_len = 0;
+ tx_desc->data_len = cpu_to_le16(param->arg_len);
+ }
+ dma_wmb();
+ tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
+
+ txq->next_to_use =
+ NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
+ txq->tail_ptr++;
+
+ return 0;
+}
+
+static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_ring *txq = &chan_info->txq;
+ struct device *dev = chan_mgt->common->dev;
+ int max_retries = NBL_CHAN_TX_WAIT_TIMES;
+ struct nbl_chan_tx_desc *tx_desc;
+ int retry_count = 0;
+ u16 msg_type;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ txq->tail_ptr, NBL_MB_TX_QID);
+
+ tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_clean);
+ while (retry_count < max_retries) {
+ if (READ_ONCE(chan_info->shutdown))
+ return -ESHUTDOWN;
+
+ /* Ensure HW written descriptor visible to CPU */
+ smp_rmb();
+ if (le16_to_cpu(READ_ONCE(tx_desc->flags)) &
+ BIT(NBL_CHAN_TX_DESC_USED)) {
+ break;
+ }
+
+ retry_count++;
+ if (retry_count == max_retries) {
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_err_ratelimited(dev, "chan send msg type: %d timeout\n",
+ msg_type);
+ txq->next_to_clean = txq->next_to_use;
+ return -ETIMEDOUT;
+ }
+ usleep_range(NBL_CHAN_TX_WAIT_US, NBL_CHAN_TX_WAIT_US_MAX);
+ }
+
+ txq->next_to_clean = txq->next_to_use;
+
+ return 0;
+}
+
+static void nbl_chan_recv_ack_msg(void *priv, u16 srcid, u16 msgid, void *data,
+ u32 data_len)
+{
+ struct nbl_channel_mgt *chan_mgt = (struct nbl_channel_mgt *)priv;
+ struct nbl_chan_waitqueue_head *wait_head = NULL;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ u32 ack_datalen, ack_msgtype = 0;
+ u32 *payload = data;
+ u16 ack_msgid = 0;
+ u32 copy_len;
+
+ if (READ_ONCE(chan_info->shutdown))
+ return;
+ if (data_len > NBL_CHAN_BUF_LEN ||
+ data_len < NBL_CHAN_ACK_HEAD_LEN * sizeof(u32)) {
+ dev_err_ratelimited(dev, "Invalid ACK data_len: %u\n",
+ data_len);
+ return;
+ }
+ ack_datalen = data_len - NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ ack_msgtype = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_TYPE_POS));
+ ack_msgid = le16_to_cpu(*(__le16 *)(payload + NBL_CHAN_MSG_ID_POS));
+ if (FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid) >=
+ chan_info->num_txq_entries) {
+ dev_err_ratelimited(dev, "chan recv msg id: %d err\n",
+ ack_msgid);
+ return;
+ }
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, ack_msgid)];
+ spin_lock_irq(&wait_head->status_lock);
+ if (srcid != READ_ONCE(wait_head->dstid)) {
+ /* Do not modify the status; the slot remains WAITING,
+ * and the sender will time out normally
+ */
+ spin_unlock_irq(&wait_head->status_lock);
+ dev_err_ratelimited(dev, "ACK srcid=%u != dstid=%u, rejecting\n",
+ srcid, READ_ONCE(wait_head->dstid));
+ return;
+ }
+ if (READ_ONCE(wait_head->status) != NBL_MBX_STATUS_WAITING) {
+ spin_unlock_irq(&wait_head->status_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack with invalid status, wait msgtype:%u msg_index:%u status:%d ack_data_len:%d, ack msgtype:%u msgid:%u datalen:%d\n",
+ READ_ONCE(wait_head->msg_type),
+ READ_ONCE(wait_head->msg_index),
+ READ_ONCE(wait_head->status),
+ READ_ONCE(wait_head->ack_data_len),
+ ack_msgtype, ack_msgid, ack_datalen);
+ return;
+ }
+
+ if (READ_ONCE(wait_head->msg_type) != ack_msgtype) {
+ spin_unlock_irq(&wait_head->status_lock);
+ dev_err_ratelimited(dev,
+ "Skip ack msg type mismatch, wait msgtype:%u msg_index:%u status:%d ack_data_len:%d, ack msgtype:%u msgid:%u datalen:%d\n",
+ READ_ONCE(wait_head->msg_type),
+ READ_ONCE(wait_head->msg_index),
+ READ_ONCE(wait_head->status),
+ READ_ONCE(wait_head->ack_data_len),
+ ack_msgtype, ack_msgid, ack_datalen);
+ return;
+ }
+ if (FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, ack_msgid) !=
+ READ_ONCE(wait_head->msg_index)) {
+ spin_unlock_irq(&wait_head->status_lock);
+ dev_err_ratelimited(dev,
+ "Stale ACK: expected index=%u, got msgid %u\n",
+ READ_ONCE(wait_head->msg_index), ack_msgid);
+ return;
+ }
+
+ wait_head->ack_err =
+ (s32)le32_to_cpu(*(__le32 *)(payload + NBL_CHAN_ACK_RET_POS));
+
+ copy_len = min_t(u32, READ_ONCE(wait_head->ack_data_len), ack_datalen);
+ if (wait_head->ack_err >= 0 && copy_len > 0) {
+ if (!READ_ONCE(wait_head->ack_data)) {
+ dev_err_ratelimited(dev, "ACK payload dropped: ack_data is NULL\n");
+ wait_head->ack_data_len = 0;
+ goto ack_done;
+ }
+ memcpy((char *)wait_head->ack_data,
+ payload + NBL_CHAN_ACK_HEAD_LEN, copy_len);
+ wait_head->ack_data_len = (u16)copy_len;
+ } else {
+ wait_head->ack_data_len = 0;
+ }
+ack_done:
+ WRITE_ONCE(wait_head->acked, 1);
+ spin_unlock_irq(&wait_head->status_lock);
+ if (READ_ONCE(wait_head->acked))
+ wake_up(&wait_head->wait_queue);
+}
+
+static void nbl_chan_recv_msg(struct nbl_channel_mgt *chan_mgt, void *data)
+{
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_msg_node_data *msg_handler;
+ u16 msg_type, payload_len, srcid, msgid;
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct nbl_chan_tx_desc *tx_desc;
+ void *payload;
+ size_t size;
+
+ if (READ_ONCE(chan_info->shutdown))
+ return;
+ tx_desc = data;
+ msg_type = le16_to_cpu(READ_ONCE(tx_desc->msg_type));
+ dev_dbg(dev, "recv msg_type: %d\n", msg_type);
+
+ srcid = le16_to_cpu(READ_ONCE(tx_desc->srcid));
+ msgid = le16_to_cpu(READ_ONCE(tx_desc->msgid));
+ /* Only check if the value exceeds the maximum, relying on the hash
+ * table to filter invalid message IDs.
+ * The gap values are reserved for future protocol extensions.
+ */
+ if (msg_type >= NBL_CHAN_MSG_MAILBOX_MAX)
+ return;
+
+ if (le16_to_cpu(READ_ONCE(tx_desc->data_len))) {
+ payload_len = le16_to_cpu(READ_ONCE(tx_desc->data_len));
+ if (payload_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
+ dev_err_ratelimited(dev,
+ "data_len=%u exceeds embedded buffer size=%u\n",
+ payload_len,
+ NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN);
+ return;
+ }
+ payload = tx_desc->data;
+ } else {
+ payload_len = le16_to_cpu(READ_ONCE(tx_desc->buf_len));
+ size = NBL_CHAN_BUF_LEN - sizeof(*tx_desc);
+ if (payload_len > size) {
+ dev_err_ratelimited(dev,
+ "buf_len=%u exceeds external buffer size=%zu\n",
+ payload_len,
+ size);
+ return;
+ }
+ payload = tx_desc + 1;
+ }
+
+ msg_handler =
+ nbl_common_get_hash_node(chan_mgt->handle_hash_tbl, &msg_type);
+ if (!msg_handler || !msg_handler->func) {
+ dev_err_ratelimited(dev,
+ "No handler for msg_type: %u (srcid=%u, msgid=%u)\n",
+ msg_type, srcid, msgid);
+ return;
+ }
+ msg_handler->func(msg_handler->priv, srcid, msgid, payload,
+ payload_len);
+}
+
+static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info,
+ struct nbl_chan_ring *rxq)
+{
+ struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ u16 next_to_use;
+
+ next_to_use = rxq->next_to_use;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_use);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_use);
+
+ rx_desc->buf_addr = cpu_to_le64(rx_buf->pa);
+ rx_desc->buf_len = cpu_to_le32(chan_info->rxq_buf_size);
+
+ /*
+ * DMA Write Memory Barrier:
+ * Ensures all previous DMA-mapped writes (buffer address/length)
+ * are completed before the descriptor flags are updated.
+ * This prevents hardware from seeing a partially updated descriptor
+ * where flags are set but buffer info isn't ready yet.
+ */
+ dma_wmb();
+
+ rx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_RX_DESC_AVAIL));
+
+ rxq->next_to_use++;
+ if (rxq->next_to_use == chan_info->num_rxq_entries)
+ rxq->next_to_use = 0;
+ rxq->tail_ptr++;
+
+ nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
+ rxq->tail_ptr, NBL_MB_RX_QID);
+}
+
+static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_info *chan_info)
+{
+ struct nbl_chan_ring *rxq = &chan_info->rxq;
+ struct device *dev = chan_mgt->common->dev;
+ struct nbl_chan_rx_desc *rx_desc;
+ struct nbl_chan_buf *rx_buf;
+ u16 next_to_clean;
+ u32 budget = 64;
+ u16 flags;
+
+ next_to_clean = rxq->next_to_clean;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ while (le16_to_cpu(READ_ONCE(rx_desc->flags)) &
+ BIT(NBL_CHAN_RX_DESC_USED)) {
+ flags = le16_to_cpu(READ_ONCE(rx_desc->flags));
+
+ if (READ_ONCE(chan_info->shutdown))
+ break;
+ if (!(flags & BIT(NBL_CHAN_RX_DESC_WRITE)))
+ dev_dbg(dev,
+ "mailbox rx flag 0x%x missing NBL_CHAN_RX_DESC_WRITE\n",
+ flags);
+
+ dma_rmb();
+ nbl_chan_recv_msg(chan_mgt, rx_buf->va);
+ nbl_chan_advance_rx_ring(chan_mgt, chan_info, rxq);
+ next_to_clean++;
+ if (next_to_clean == chan_info->num_rxq_entries)
+ next_to_clean = 0;
+ rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
+ rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
+ if (--budget == 0)
+ break;
+ cond_resched();
+ }
+ rxq->next_to_clean = next_to_clean;
+}
+
+static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ nbl_chan_clean_queue(chan_mgt, chan_info);
+}
+
+static int nbl_chan_get_msg_id(struct nbl_chan_info *chan_info,
+ u16 *msgid)
+{
+ int search_loc = chan_info->wait_head_index, i;
+ struct nbl_chan_waitqueue_head *wait = NULL;
+ int status;
+
+ for (i = 0; i < chan_info->num_txq_entries; i++) {
+ wait = &chan_info->wait[search_loc];
+
+ spin_lock_irq(&wait->status_lock);
+ status = READ_ONCE(wait->status);
+ if (status == NBL_MBX_STATUS_IDLE ||
+ status == NBL_MBX_STATUS_TIMEOUT) {
+ WRITE_ONCE(wait->msg_index,
+ NBL_NEXT_ID(wait->msg_index,
+ NBL_CHAN_MSG_INDEX_MAX));
+
+ *msgid = FIELD_PREP(NBL_CHAN_MSGID_INDEX_MASK,
+ wait->msg_index) |
+ FIELD_PREP(NBL_CHAN_MSGID_LOC_MASK,
+ search_loc);
+ spin_unlock_irq(&wait->status_lock);
+
+ /* Advance starting search position for next caller */
+ WRITE_ONCE(chan_info->wait_head_index, search_loc);
+ return 0;
+ }
+ spin_unlock_irq(&wait->status_lock);
+
+ search_loc = NBL_NEXT_ID(search_loc,
+ chan_info->num_txq_entries - 1);
+ }
+
+ /*
+ * All tx slots are occupied. May happen under high transmit load
+ * or delayed remote ACK responses. Caller should retry later.
+ */
+ return -EAGAIN;
+}
+
+static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send)
+{
+ struct nbl_common_info *common = chan_mgt->common;
+ struct nbl_chan_waitqueue_head *wait_head;
+ struct nbl_chan_tx_param tx_param = { 0 };
+ struct nbl_chan_info *chan_info =
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
+ struct device *dev = common->dev;
+ bool inflight_inc = false;
+ u16 msgid = 0;
+ int ret;
+ u16 tmp;
+
+ if (chan_send->resp_len > NBL_CHAN_BUF_LEN) {
+ dev_err_ratelimited(dev, "resp_len %zu exceeds max %d\n",
+ chan_send->resp_len, NBL_CHAN_BUF_LEN);
+ return -EINVAL;
+ }
+
+ atomic_inc(&chan_info->inflight_tx_cnt);
+ inflight_inc = true;
+ /*
+ * Ensure reading chan_info->shutdown happens
+ * strictly after inflight_tx_cnt increment above on this CPU.
+ * Prevent compiler/cpu reordering: read shutdown before incrementing
+ * inflight counter, which would cause missing shutdown check and leak
+ * tx inflight count.
+ */
+ smp_rmb();
+ if (READ_ONCE(chan_info->shutdown)) {
+ atomic_dec(&chan_info->inflight_tx_cnt);
+ return -ESHUTDOWN;
+ }
+
+ mutex_lock(&chan_info->txq_lock);
+ if (test_bit(NBL_CHAN_ABNORMAL, chan_info->state)) {
+ ret = -EIO;
+ goto unlock_out;
+ }
+ ret = nbl_chan_get_msg_id(chan_info, &msgid);
+ if (ret) {
+ dev_err_ratelimited(dev,
+ "Channel tx wait head full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type, chan_send->dstid);
+ goto unlock_out;
+ }
+
+ tx_param.msg_type = chan_send->msg_type;
+ tx_param.arg = chan_send->arg;
+ tx_param.arg_len = chan_send->arg_len;
+ tx_param.dstid = chan_send->dstid;
+ tx_param.msgid = msgid;
+
+ ret = nbl_chan_update_txqueue(chan_mgt, chan_info, &tx_param);
+ if (ret) {
+ dev_err_ratelimited(dev,
+ "Channel tx queue full, send msgtype:%u to dstid:%u failed\n",
+ chan_send->msg_type, chan_send->dstid);
+ goto unlock_out;
+ }
+
+ wait_head =
+ &chan_info->wait[FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, msgid)];
+ spin_lock_irq(&wait_head->status_lock);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, chan_send->resp);
+ WRITE_ONCE(wait_head->ack_data_len, chan_send->resp_len);
+ WRITE_ONCE(wait_head->msg_type, chan_send->msg_type);
+ WRITE_ONCE(wait_head->msg_index,
+ FIELD_GET(NBL_CHAN_MSGID_INDEX_MASK, msgid));
+ WRITE_ONCE(wait_head->dstid, chan_send->dstid);
+
+ WRITE_ONCE(wait_head->status, chan_send->ack ?
+ NBL_MBX_STATUS_WAITING : NBL_MBX_STATUS_IDLE);
+ spin_unlock_irq(&wait_head->status_lock);
+
+ ret = nbl_chan_kick_tx_ring(chan_mgt, chan_info);
+ if (ret) {
+ spin_lock_irq(&wait_head->status_lock);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ spin_unlock_irq(&wait_head->status_lock);
+ goto unlock_out;
+ }
+ mutex_unlock(&chan_info->txq_lock);
+ if (!chan_send->ack) {
+ tmp = FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, msgid);
+ wait_head = &chan_info->wait[tmp];
+ spin_lock_irq(&wait_head->status_lock);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
+ spin_unlock_irq(&wait_head->status_lock);
+ atomic_dec(&chan_info->inflight_tx_cnt);
+ return 0;
+ }
+
+ /* choose wait strategy based on interrupt state */
+ if (test_bit(NBL_CHAN_INTERRUPT_READY, chan_info->state)) {
+ spin_lock_irq(&wait_head->status_lock);
+ while (!READ_ONCE(wait_head->acked)) {
+ spin_unlock_irq(&wait_head->status_lock);
+ ret = wait_event_timeout(wait_head->wait_queue,
+ READ_ONCE(wait_head->acked) ||
+ READ_ONCE(chan_info->shutdown),
+ NBL_CHAN_ACK_WAIT_TIME);
+ spin_lock_irq(&wait_head->status_lock);
+
+ if (READ_ONCE(chan_info->shutdown)) {
+ ret = -ESHUTDOWN;
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_IDLE);
+ spin_unlock_irq(&wait_head->status_lock);
+ goto inflight_dec_out;
+ }
+ if (ret == 0) {
+ if (READ_ONCE(wait_head->status) ==
+ NBL_MBX_STATUS_WAITING) {
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_TIMEOUT);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ }
+ spin_unlock_irq(&wait_head->status_lock);
+ dev_err_ratelimited(dev,
+ "Channel waiting ack failed, message type: %d, msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ goto inflight_dec_out;
+ }
+
+ if (READ_ONCE(wait_head->acked))
+ break;
+ }
+
+ chan_send->ack_len = wait_head->ack_data_len;
+ ret = wait_head->ack_err;
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ spin_unlock_irq(&wait_head->status_lock);
+ goto inflight_dec_out;
+ }
+
+inflight_dec_out:
+ atomic_dec(&chan_info->inflight_tx_cnt);
+ return ret;
+
+unlock_out:
+ mutex_unlock(&chan_info->txq_lock);
+ if (inflight_inc)
+ atomic_dec(&chan_info->inflight_tx_cnt);
+ return ret;
+}
+
+static int nbl_chan_send_ack(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack)
+{
+ size_t head_len = NBL_CHAN_ACK_HEAD_LEN * sizeof(u32);
+ size_t data_len = chan_ack->data_len;
+ struct nbl_chan_send_info chan_send;
+ __le32 *tmp;
+ size_t len;
+ int ret;
+
+ if (data_len >
+ NBL_CHAN_BUF_LEN - sizeof(struct nbl_chan_tx_desc) - head_len)
+ return -EINVAL;
+
+ len = head_len + data_len;
+ tmp = kzalloc(len, GFP_ATOMIC);
+ if (!tmp)
+ return -ENOMEM;
+
+ *(__le16 *)&tmp[NBL_CHAN_MSG_TYPE_POS] =
+ cpu_to_le16(chan_ack->msg_type);
+ *(__le16 *)&tmp[NBL_CHAN_MSG_ID_POS] = cpu_to_le16(chan_ack->msgid);
+ tmp[NBL_CHAN_ACK_RET_POS] = cpu_to_le32(chan_ack->err);
+ if (chan_ack->data && chan_ack->data_len)
+ memcpy(&tmp[NBL_CHAN_ACK_HEAD_LEN], chan_ack->data,
+ chan_ack->data_len);
+
+ nbl_chan_fill_send_info(&chan_send, chan_ack->dstid, NBL_CHAN_MSG_ACK,
+ tmp, len, NULL, 0, 0);
+ ret = nbl_chan_send_msg(chan_mgt, &chan_send);
+ kfree(tmp);
+
+ return ret;
+}
+
+static int nbl_chan_register_msg(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback)
+{
+ return nbl_chan_add_msg_handler(chan_mgt, msg_type, func, callback);
+}
+
+static bool nbl_chan_check_queue_exist(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ return chan_info ? true : false;
+}
+
+static void nbl_chan_register_chan_task(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ chan_info->clean_task = task;
+}
+
+static void nbl_chan_set_queue_state(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set)
+{
+ struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
+
+ if (set)
+ set_bit(state, chan_info->state);
+ else
+ clear_bit(state, chan_info->state);
+}
+
+static struct nbl_channel_ops chan_ops = {
+ .send_msg = nbl_chan_send_msg,
+ .send_ack = nbl_chan_send_ack,
+ .register_msg = nbl_chan_register_msg,
+ .unregister_all_msg = nbl_chan_remove_msg_handler,
+ .cfg_chan_qinfo_map_table = nbl_chan_cfg_qinfo_map_table,
+ .check_queue_exist = nbl_chan_check_queue_exist,
+ .setup_queue = nbl_chan_setup_queue,
+ .teardown_queue = nbl_chan_teardown_queue,
+ .clean_queue_subtask = nbl_chan_clean_queue_subtask,
+ .register_chan_task = nbl_chan_register_chan_task,
+ .set_queue_state = nbl_chan_set_queue_state,
+};
+
+static struct nbl_channel_mgt *
+nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adapter->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adapter->common;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ struct nbl_chan_info *mailbox;
+ int ret;
+
+ chan_mgt = devm_kzalloc(dev, sizeof(*chan_mgt), GFP_KERNEL);
+ if (!chan_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ chan_mgt->common = common;
+ chan_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ mailbox = devm_kzalloc(dev, sizeof(*mailbox), GFP_KERNEL);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+ mailbox->chan_type = NBL_CHAN_TYPE_MAILBOX;
+ chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX] = mailbox;
+
+ ret = nbl_chan_init_msg_handler(chan_mgt);
+ if (ret)
+ return ERR_PTR(-ENOMEM);
+
+ return chan_mgt;
+}
+
+static struct nbl_channel_ops_tbl *
+nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ int ret;
+
+ chan_ops_tbl = devm_kzalloc(dev, sizeof(*chan_ops_tbl), GFP_KERNEL);
+ if (!chan_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!chan_ops.send_msg || !chan_ops.send_ack ||
+ !chan_ops.register_msg || !chan_ops.unregister_all_msg ||
+ !chan_ops.cfg_chan_qinfo_map_table ||
+ !chan_ops.check_queue_exist || !chan_ops.setup_queue ||
+ !chan_ops.teardown_queue || !chan_ops.clean_queue_subtask ||
+ !chan_ops.register_chan_task || !chan_ops.set_queue_state)
+ return ERR_PTR(-EINVAL);
+
+ chan_ops_tbl->ops = &chan_ops;
+ chan_ops_tbl->priv = chan_mgt;
+
+ ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
+ nbl_chan_recv_ack_msg, chan_mgt);
+ if (ret)
+ return ERR_PTR(-ENOMEM);
+
+ return chan_ops_tbl;
+}
+
+int nbl_chan_init_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_channel_mgt *chan_mgt;
+ int ret;
+
+ chan_mgt = nbl_chan_setup_chan_mgt(adap);
+ if (IS_ERR(chan_mgt)) {
+ ret = PTR_ERR(chan_mgt);
+ goto exit;
+ }
+
+ chan_ops_tbl = nbl_chan_setup_ops(dev, chan_mgt);
+ if (IS_ERR(chan_ops_tbl)) {
+ ret = PTR_ERR(chan_ops_tbl);
+ goto cleanup_mgt;
+ }
+ adap->intf.channel_ops_tbl = chan_ops_tbl;
+ adap->core.chan_mgt = chan_mgt;
+ return 0;
+
+cleanup_mgt:
+ nbl_chan_remove_msg_handler(chan_mgt);
+exit:
+ return ret;
+}
+
+void nbl_chan_remove_common(struct nbl_adapter *adap)
+{
+ struct nbl_channel_mgt *chan_mgt = adap->core.chan_mgt;
+
+ if (!chan_mgt)
+ return;
+
+ /*
+ * All channel queues shall be torn down earlier in remove path
+ * to drain inflight tx workers and stop hardware before destroying
+ * message handler hash table.
+ */
+ nbl_chan_remove_msg_handler(chan_mgt);
+ adap->core.chan_mgt = NULL;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
new file mode 100644
index 000000000000..92db39e1b05e
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
@@ -0,0 +1,170 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHANNEL_H_
+#define _NBL_CHANNEL_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_CHAN_TX_RING_TO_DESC(tx_ring, i) \
+ (&((((tx_ring)->desc.tx_desc))[i]))
+#define NBL_CHAN_RX_RING_TO_DESC(rx_ring, i) \
+ (&((((rx_ring)->desc.rx_desc))[i]))
+#define NBL_CHAN_TX_RING_TO_BUF(tx_ring, i) (&(((tx_ring)->buf)[i]))
+#define NBL_CHAN_RX_RING_TO_BUF(rx_ring, i) (&(((rx_ring)->buf)[i]))
+
+#define NBL_CHAN_TX_WAIT_US 100
+#define NBL_CHAN_TX_WAIT_US_MAX 120
+#define NBL_CHAN_TX_WAIT_TIMES 100
+#define NBL_CHAN_QUEUE_LEN 256
+#define NBL_CHAN_BUF_LEN 4096
+#define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
+
+#define NBL_CHAN_TX_DESC_AVAIL 0
+#define NBL_CHAN_TX_DESC_USED 1
+#define NBL_CHAN_RX_DESC_WRITE 1
+#define NBL_CHAN_RX_DESC_AVAIL 3
+#define NBL_CHAN_RX_DESC_USED 4
+
+#define NBL_CHAN_ACK_HEAD_LEN 3
+#define NBL_CHAN_ACK_RET_POS 2
+#define NBL_CHAN_MSG_ID_POS 1
+#define NBL_CHAN_MSG_TYPE_POS 0
+
+#define NBL_CHAN_ACK_WAIT_TIME (3 * HZ)
+
+#define NBL_CHAN_HANDLER_TBL_BUCKET_SIZE 512
+
+enum {
+ NBL_MB_RX_QID = 0,
+ NBL_MB_TX_QID = 1,
+};
+
+enum {
+ NBL_MBX_STATUS_IDLE = 0,
+ NBL_MBX_STATUS_WAITING,
+ NBL_MBX_STATUS_TIMEOUT,
+};
+
+struct nbl_chan_tx_param {
+ enum nbl_chan_msg_type msg_type;
+ void *arg;
+ size_t arg_len;
+ u16 dstid;
+ u16 msgid;
+};
+
+struct nbl_chan_buf {
+ void *va;
+ dma_addr_t pa;
+ size_t size;
+};
+
+struct nbl_chan_tx_desc {
+ __le16 flags;
+ __le16 srcid;
+ __le16 dstid;
+ __le16 data_len;
+ __le16 buf_len;
+ __le64 buf_addr;
+ __le16 msg_type;
+ u8 data[16];
+ __le16 msgid;
+ u8 rsv[26];
+} __packed;
+
+struct nbl_chan_rx_desc {
+ __le16 flags;
+ __le32 buf_len;
+ __le16 buf_id;
+ __le64 buf_addr;
+} __packed;
+
+union nbl_chan_desc_ptr {
+ struct nbl_chan_tx_desc *tx_desc;
+ struct nbl_chan_rx_desc *rx_desc;
+};
+
+struct nbl_chan_ring {
+ union nbl_chan_desc_ptr desc;
+ struct nbl_chan_buf *buf;
+ u16 next_to_use;
+ u16 tail_ptr; /* hardware does modulo ring size internally */
+ u16 next_to_clean;
+ dma_addr_t dma;
+};
+
+#define NBL_CHAN_MSG_INDEX_MAX 63
+
+#define NBL_CHAN_MSGID_INDEX_MASK GENMASK(5, 0)
+#define NBL_CHAN_MSGID_LOC_MASK GENMASK(13, 6)
+
+static inline void nbl_chan_update_tail_ptr(struct nbl_hw_ops *hw_ops,
+ void *hw_priv, u32 tail_ptr, u8 qid)
+{
+ hw_ops->update_mailbox_queue_tail_ptr(hw_priv, tail_ptr, qid);
+}
+
+struct nbl_chan_waitqueue_head {
+ struct wait_queue_head wait_queue;
+ char *ack_data;
+ int acked;
+ s32 ack_err;
+ u16 ack_data_len;
+ u16 msg_type;
+ /*
+ * Spinlock protecting all fields.
+ * Must be held when reading/writing: status, acked, ack_err,
+ * ack_data_len, etc.
+ * The lock ensures atomic updates of these fields and
+ * proper memory ordering with smp_wmb()/smp_rmb().
+ */
+ spinlock_t status_lock;
+ int status;
+ u8 msg_index;
+ u16 dstid;
+};
+
+struct nbl_chan_info {
+ struct nbl_chan_ring txq;
+ struct nbl_chan_ring rxq;
+ struct nbl_chan_waitqueue_head *wait;
+ /*
+ *Protects access to the TX queue (txq) and related metadata.
+ *This mutex ensures exclusive access when updating the TX queue
+ */
+ struct mutex txq_lock;
+ struct work_struct *clean_task;
+ u16 wait_head_index;
+ u16 num_txq_entries;
+ u16 num_rxq_entries;
+ u16 txq_buf_size;
+ u16 rxq_buf_size;
+ DECLARE_BITMAP(state, NBL_CHAN_STATE_NBITS);
+ u8 chan_type;
+ atomic_t inflight_tx_cnt;
+ bool shutdown;
+ bool init_done;
+};
+
+struct nbl_chan_msg_node_data {
+ nbl_chan_resp func;
+ void *priv;
+};
+
+struct nbl_channel_mgt {
+ struct nbl_common_info *common;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_chan_info *chan_info[NBL_CHAN_TYPE_MAX];
+ struct nbl_hash_tbl_mgt *handle_hash_tbl;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
new file mode 100644
index 000000000000..fc9b489ca02c
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -0,0 +1,166 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/jhash.h>
+#include "nbl_common.h"
+
+static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
+{
+ u32 hash;
+
+ if (bucket_size == 0 || bucket_size == NBL_HASH_TBL_LIST_BUCKET_SIZE)
+ return 0;
+
+ hash = jhash(key, key_size, 0);
+
+ /* Use bitmask if bucket_size is a power of 2 */
+ if ((bucket_size & (bucket_size - 1)) == 0)
+ return hash & (bucket_size - 1);
+ else
+ return hash % bucket_size;
+}
+
+/*
+ * alloc a hash table
+ * the table support multi thread
+ */
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key)
+{
+ struct nbl_hash_tbl_mgt *tbl_mgt;
+ int bucket_size;
+ int i;
+
+ tbl_mgt = devm_kzalloc(key->dev, sizeof(*tbl_mgt), GFP_KERNEL);
+ if (!tbl_mgt)
+ return NULL;
+
+ bucket_size = key->bucket_size;
+ tbl_mgt->hash = devm_kcalloc(key->dev, bucket_size,
+ sizeof(struct hlist_head), GFP_KERNEL);
+ if (!tbl_mgt->hash)
+ goto alloc_hash_failed;
+
+ for (i = 0; i < bucket_size; i++)
+ INIT_HLIST_HEAD(tbl_mgt->hash + i);
+
+ memcpy(&tbl_mgt->tbl_key, key, sizeof(struct nbl_hash_tbl_key));
+
+ return tbl_mgt;
+
+alloc_hash_failed:
+ return NULL;
+}
+
+/*
+ * Allocate a hash node and add to the hash table.
+ *
+ * Note: The hash table is protected by the caller's mutex (txq_lock),
+ * not lock-free. The "single context" comment refers to: this function
+ * is only called during init from nbl_chan_init_msg_handler, so there
+ * is no concurrent mutation during init.
+ * After init, the table is read-only (no unregister API), so no lock is needed
+ * for lookups either.
+ *
+ * The tbl_mgt and bucket array are devm-allocated, so they are automatically
+ * freed on device detach. Only the hash nodes themselves need explicit cleanup.
+ */
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data)
+{
+ struct nbl_hash_entry_node *hash_node;
+ u16 data_size;
+ u16 node_size;
+ u32 hash_val;
+ u16 key_size;
+
+ node_size = sizeof(*hash_node);
+ hash_node = kzalloc(node_size, GFP_KERNEL);
+ if (!hash_node)
+ return -ENOMEM;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_node->key = kzalloc(key_size, GFP_KERNEL);
+ if (!hash_node->key)
+ goto alloc_key_failed;
+
+ data_size = tbl_mgt->tbl_key.data_size;
+ hash_node->data = kzalloc(data_size, GFP_KERNEL);
+ if (!hash_node->data)
+ goto alloc_data_failed;
+
+ memcpy(hash_node->key, key, key_size);
+ memcpy(hash_node->data, data, data_size);
+
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+
+ hlist_add_head(&hash_node->node, tbl_mgt->hash + hash_val);
+ tbl_mgt->node_num++;
+ if (out_data)
+ *out_data = hash_node->data;
+
+ return 0;
+
+alloc_data_failed:
+ kfree(hash_node->key);
+alloc_key_failed:
+ kfree(hash_node);
+ return -ENOMEM;
+}
+
+/*
+ * get a hash node, return the data if node exist
+ */
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_head *head;
+ void *data = NULL;
+ u32 hash_val;
+ u16 key_size;
+
+ key_size = tbl_mgt->tbl_key.key_size;
+ hash_val = nbl_common_calc_hash_key(key, key_size,
+ tbl_mgt->tbl_key.bucket_size);
+ head = tbl_mgt->hash + hash_val;
+
+ hlist_for_each_entry(hash_node, head, node)
+ if (!memcmp(hash_node->key, key, key_size)) {
+ data = hash_node->data;
+ break;
+ }
+
+ return data;
+}
+
+static void nbl_common_detach_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt,
+ struct nbl_hash_entry_node *hash_node)
+{
+ hlist_del(&hash_node->node);
+ kfree(hash_node->key);
+ kfree(hash_node->data);
+ kfree(hash_node);
+ tbl_mgt->node_num--;
+}
+
+/*
+ * Free all hash nodes in the table.
+ */
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt)
+{
+ struct nbl_hash_entry_node *hash_node;
+ struct hlist_node *safe_node;
+ struct hlist_head *head;
+ u32 i;
+
+ for (i = 0; i < tbl_mgt->tbl_key.bucket_size; i++) {
+ head = tbl_mgt->hash + i;
+ hlist_for_each_entry_safe(hash_node, safe_node, head, node) {
+ nbl_common_detach_hash_node(tbl_mgt, hash_node);
+ }
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
new file mode 100644
index 000000000000..ad8fb7ec9f45
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_COMMON_H_
+#define _NBL_COMMON_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_common.h"
+
+/* list only need one bucket size */
+#define NBL_HASH_TBL_LIST_BUCKET_SIZE 1
+
+struct nbl_hash_tbl_mgt {
+ struct nbl_hash_tbl_key tbl_key;
+ struct hlist_head *hash;
+ u16 node_num;
+};
+
+struct nbl_hash_entry_node {
+ struct hlist_node node;
+ void *key;
+ void *data;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index e2e33fbb848a..66861d684220 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -14,13 +14,20 @@ enum {
NBL_CAP_HAS_NET_BIT,
};
+struct nbl_interface {
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_channel_ops_tbl *channel_ops_tbl;
+};
+
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_channel_mgt *chan_mgt;
};
struct nbl_adapter {
struct pci_dev *pdev;
struct nbl_core core;
+ struct nbl_interface intf;
struct nbl_common_info common;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index d66d7ca8a2bb..7faac0718038 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,150 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+
+ for (i = 0; i < len / 4; i++)
+ nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
+}
+
+static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+
+ spin_lock(&hw_mgt->reg_lock);
+
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
+ const u32 *data, u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ spin_lock(&hw_mgt->reg_lock);
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx)
+{
+ /* local_qid 0 and 1 denote rx and tx queue respectively */
+ u32 local_qid = txrx;
+ u32 value = ((u32)tail_ptr << 16) | local_qid;
+
+ /* wmb for doorbell */
+ wmb();
+ nbl_mbx_wr32(hw_mgt, NBL_MAILBOX_NOTIFY_ADDR, value);
+}
+
+static void nbl_hw_config_mailbox_rxq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_config_mailbox_txq(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+
+ cfg_tbl.data[0] = lower_32_bits(dma_addr);
+ cfg_tbl.data[1] = upper_32_bits(dma_addr);
+ cfg_tbl.data[2] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK,
+ size_bwid);
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 0) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_rxq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_stop_mailbox_txq(struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_mailbox_qinfo_cfg_table cfg_tbl;
+
+ memset(&cfg_tbl, 0, sizeof(cfg_tbl));
+ cfg_tbl.data[3] = FIELD_PREP(NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK, 1);
+ nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
+ cfg_tbl.data, sizeof(cfg_tbl));
+}
+
+static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
+{
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_K_PF_MASK_REG, pf_mask,
+ sizeof(*pf_mask));
+}
+
+static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function)
+{
+ u32 data = 0;
+
+ data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
+ &data,
+ sizeof(data));
+}
+
+static struct nbl_hw_ops hw_ops = {
+ .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
+ .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
+ .config_mailbox_txq = nbl_hw_config_mailbox_txq,
+ .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
+ .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
+ .get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+
+};
+
/* Structure starts here, adding an op should not modify anything below */
static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
{
@@ -25,6 +169,27 @@ static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
return hw_mgt;
}
+static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
+ struct nbl_hw_mgt *hw_mgt)
+{
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct device *dev;
+
+ dev = common->dev;
+ hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
+ if (!hw_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+ if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ !hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
+ !hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
+ !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ return ERR_PTR(-EINVAL);
+ hw_ops_tbl->ops = &hw_ops;
+ hw_ops_tbl->priv = hw_mgt;
+
+ return hw_ops_tbl;
+}
+
static int nbl_pcim_request_selected_bars(struct pci_dev *pdev, u32 mask,
const char *name)
{
@@ -45,6 +210,7 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
{
struct nbl_common_info *common = &adapter->common;
struct pci_dev *pdev = common->pdev;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = NULL;
struct nbl_hw_mgt *hw_mgt = NULL;
resource_size_t bar_len;
u32 bar_mask;
@@ -136,6 +302,14 @@ int nbl_hw_init_leonis(struct nbl_adapter *adapter)
goto setup_mgt_fail;
}
+ spin_lock_init(&hw_mgt->reg_lock);
+
+ hw_ops_tbl = nbl_hw_setup_ops(common, hw_mgt);
+ if (IS_ERR(hw_ops_tbl)) {
+ ret = PTR_ERR(hw_ops_tbl);
+ goto setup_mgt_fail;
+ }
+ adapter->intf.hw_ops_tbl = hw_ops_tbl;
adapter->core.hw_mgt = hw_mgt;
return 0;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 77c67b67ba31..1d2dd10e6239 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,4 +11,60 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+/* ---------- REG BASE ADDR ---------- */
+/* Interface modules base addr */
+#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
+#define NBL_INTF_HOST_PADPT_BASE 0x00f4c000
+#define NBL_INTF_HOST_MAILBOX_BASE 0x00fb0000
+#define NBL_INTF_HOST_PCIE_BASE 0X01504000
+/* DP modules base addr */
+#define NBL_DP_USTORE_BASE 0x00104000
+#define NBL_DP_UQM_BASE 0x00114000
+#define NBL_DP_UPED_BASE 0x0015c000
+#define NBL_DP_UVN_BASE 0x00244000
+#define NBL_DP_DSCH_BASE 0x00404000
+#define NBL_DP_SHAPING_BASE 0x00504000
+#define NBL_DP_DVN_BASE 0x00514000
+#define NBL_DP_DSTORE_BASE 0x00704000
+#define NBL_DP_DQM_BASE 0x00714000
+#define NBL_DP_DPED_BASE 0x0075c000
+#define NBL_DP_DDMUX_BASE 0x00984000
+/* -------- MAILBOX BAR2 ----- */
+#define NBL_MAILBOX_NOTIFY_ADDR 0x00000000
+#define NBL_MAILBOX_BAR_REG 0x00000000
+#define NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR 0x10
+#define NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR 0x20
+#define NBL_MAILBOX_QINFO_CFG_DBG_TABLE_ADDR 0x30
+
+/* -------- MAILBOX -------- */
+
+/* mailbox BAR qinfo_cfg_table */
+#define MAILBOX_QINFO_CFG_TABLE_DWLEN 4
+/* data[2] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_SIZE_BWID_MASK GENMASK(3, 0)
+/* data[3] */
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK BIT(0)
+#define NBL_MAILBOX_QINFO_CFG_QUEUE_EN_MASK BIT(1)
+#define NBL_MAILBOX_QINFO_CFG_DIF_ERR_MASK BIT(2)
+#define NBL_MAILBOX_QINFO_CFG_PTR_ERR_MASK BIT(3)
+struct nbl_mailbox_qinfo_cfg_table {
+ u32 data[MAILBOX_QINFO_CFG_TABLE_DWLEN];
+};
+
+/* -------- MAILBOX BAR0 ----- */
+/* mailbox qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id) \
+ (NBL_INTF_HOST_MAILBOX_BASE + 0x00001000 + (func_id) * sizeof(u32))
+
+/* MAILBOX qinfo_map_table */
+#define NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_MAILBOX_QINFO_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_MAILBOX_QINFO_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK GENMASK(28, 16)
+#define NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK BIT(29)
+
+/* -------- HOST_PCIE -------- */
+#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
+#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
index 1a199357e85f..0026cee101aa 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
+#include "../nbl_include/nbl_def_channel.h"
#include "../nbl_include/nbl_def_hw.h"
#include "../nbl_include/nbl_def_common.h"
#include "../nbl_core.h"
@@ -16,6 +17,7 @@
#define NBL_MAILBOX_BAR 2
#define NBL_RDMA_NOTIFY_LEN (8ULL << 10)
#define NBL_REG_NET_ONLY_LEN (8ULL << 10)
+#define NBL_HW_DUMMY_REG 0x1300904
/*
* PCI MEMORY BAR total size: 64MiB.
* First 32MiB: module registers + table entry address range.
@@ -28,6 +30,49 @@ struct nbl_hw_mgt {
u8 __iomem *hw_addr;
u8 __iomem *mailbox_bar_hw_addr;
resource_size_t hw_size;
+ spinlock_t reg_lock; /* Protect reg access */
};
+static inline u32 rd32(u8 __iomem *addr, u64 reg)
+{
+ return readl(addr + reg);
+}
+
+static inline void wr32(u8 __iomem *addr, u64 reg, u32 value)
+{
+ writel(value, addr + reg);
+}
+
+static inline void nbl_hw_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ /* Used for emu, make sure that we won't write too frequently */
+ wr32(hw_mgt->hw_addr, reg, value);
+}
+
+static inline u32 nbl_hw_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return rd32(hw_mgt->hw_addr, reg);
+}
+
+static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
+{
+ writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
+/*
+ * Only call this when has_ctrl=true, which maps enough space
+ * (bar_len - 8192) to cover NBL_HW_DUMMY_REG (0x1300904).
+ * The flow/design guarantees this is only called in the
+ * has_ctrl path.
+ */
+static inline void nbl_flush_writes(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_rd32(hw_mgt, NBL_HW_DUMMY_REG);
+}
+
+static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
+{
+ return readl(hw_mgt->mailbox_bar_hw_addr + reg);
+}
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index be374667c338..b5a7b069d834 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -6,6 +6,13 @@
#ifndef _NBL_DEF_CHANNEL_H_
#define _NBL_DEF_CHANNEL_H_
+#include <linux/types.h>
+
+struct nbl_channel_mgt;
+struct nbl_adapter;
+
+typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+
/*
* Mailbox wire opcodes
* Every opcode is assigned explicit fixed numeric value, stable wire ABI
@@ -239,4 +246,101 @@ enum nbl_chan_msg_type {
NBL_CHAN_MSG_MAILBOX_MAX,
};
+enum nbl_chan_state {
+ NBL_CHAN_INTERRUPT_READY,
+ NBL_CHAN_ABNORMAL,
+ NBL_CHAN_STATE_NBITS
+};
+
+struct nbl_board_port_info {
+ u8 eth_num;
+ u8 eth_speed;
+ u8 p4_version;
+ u8 rsv[5];
+};
+
+struct nbl_chan_send_info {
+ void *arg;
+ size_t arg_len;
+ void *resp;
+ size_t resp_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 ack;
+ u16 ack_len;
+};
+
+struct nbl_chan_ack_info {
+ void *data;
+ int err;
+ u32 data_len;
+ u16 dstid;
+ u16 msg_type;
+ u16 msgid;
+};
+
+enum nbl_channel_type {
+ NBL_CHAN_TYPE_MAILBOX,
+ NBL_CHAN_TYPE_MAX
+};
+
+static inline void
+nbl_chan_fill_send_info(struct nbl_chan_send_info *info,
+ u16 dst_id, u16 msg_type,
+ void *argument, u32 arg_length,
+ void *response, u32 resp_length,
+ bool need_ack)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->arg = argument;
+ info->arg_len = arg_length;
+ info->resp = response;
+ info->resp_len = resp_length;
+ info->ack = need_ack;
+}
+
+static inline void
+nbl_chan_fill_ack_info(struct nbl_chan_ack_info *info,
+ u16 dst_id, u16 msg_type, u16 msg_id,
+ int err_code, void *ack_data, u32 data_length)
+{
+ info->dstid = dst_id;
+ info->msg_type = msg_type;
+ info->msgid = msg_id;
+ info->err = err_code;
+ info->data = ack_data;
+ info->data_len = data_length;
+}
+
+struct nbl_channel_ops {
+ int (*send_msg)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_send_info *chan_send);
+ int (*send_ack)(struct nbl_channel_mgt *chan_mgt,
+ struct nbl_chan_ack_info *chan_ack);
+ int (*register_msg)(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
+ nbl_chan_resp func, void *callback_priv);
+ void (*unregister_all_msg)(struct nbl_channel_mgt *chan_mgt);
+ void (*cfg_chan_qinfo_map_table)(struct nbl_channel_mgt *chan_mgt,
+ u8 bus, u8 devid);
+ bool (*check_queue_exist)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ int (*setup_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ int (*teardown_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
+ void (*clean_queue_subtask)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type);
+ void (*register_chan_task)(struct nbl_channel_mgt *chan_mgt,
+ u8 chan_type, struct work_struct *task);
+ void (*set_queue_state)(struct nbl_channel_mgt *chan_mgt,
+ enum nbl_chan_state state, u8 chan_type,
+ u8 set);
+};
+
+struct nbl_channel_ops_tbl {
+ struct nbl_channel_ops *ops;
+ struct nbl_channel_mgt *priv;
+};
+
+int nbl_chan_init_common(struct nbl_adapter *adapter);
+void nbl_chan_remove_common(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index 5b7ec68d620b..4f651a227539 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -29,4 +29,19 @@ struct nbl_common_info {
u8 has_net;
};
+struct nbl_hash_tbl_key {
+ struct device *dev;
+ u16 key_size;
+ u16 data_size; /* no include key or node member */
+ u16 bucket_size;
+ u16 resv;
+};
+
+struct nbl_hash_tbl_mgt *
+nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
+void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
+int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
+ void *data, void **out_data);
+void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key);
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index 517cfa5c755e..9cba73fe24ac 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -10,6 +10,25 @@
struct nbl_hw_mgt;
struct nbl_adapter;
+struct nbl_hw_ops {
+ void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
+ u16 tail_ptr, u8 txrx);
+ void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*config_mailbox_txq)(struct nbl_hw_mgt *hw_mgt,
+ dma_addr_t dma_addr, int size_bwid);
+ void (*stop_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt);
+ void (*stop_mailbox_txq)(struct nbl_hw_mgt *hw_mgt);
+ void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+
+ void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ u8 bus, u8 devid, u8 function);
+};
+
+struct nbl_hw_ops_tbl {
+ struct nbl_hw_ops *ops;
+ struct nbl_hw_mgt *priv;
+};
int nbl_hw_init_leonis(struct nbl_adapter *adapter);
void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 70f4f4b4c49c..4dc0615ecc82 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -10,6 +10,9 @@
/* ------ Basic definitions ------- */
#define NBL_DRIVER_NAME "nbl"
+#define NBL_MAX_PF 8
+#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index bd9f682b8323..2d30c0eed00f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/bits.h>
#include "nbl_include/nbl_include.h"
+#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -38,13 +39,19 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
if (ret)
goto hw_init_fail;
+ ret = nbl_chan_init_common(adapter);
+ if (ret)
+ goto chan_init_fail;
return adapter;
+chan_init_fail:
+ nbl_hw_remove_leonis(adapter);
hw_init_fail:
return ERR_PTR(ret);
}
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 05/11] net/nebula-matrix: add common resource implementation
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (3 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 06/11] net/nebula-matrix: add intr " illusion.wang
` (5 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
The Resource layer processes the entries/data of various modules within
the processing chip to accomplish specific entry management operations,
this describes the module business capabilities of the chip and the data
it manages.
The resource layer comprises the following sub-modules: common,
interrupt, and vsi(txrx,queue not contained this time)
This patch provides the common part, including the conversion
relationships among vsi_id, func_id, eth_id, and pf_id. These
relationships may be utilized in the upper layer or the resource layer.
Key Assumptions:
- nbl_res_start() initializes VSI/Eth/PF data structures **only for
control devices** (`common->has_ctrl == true`).
- APIs like nbl_res_func_id_to_vsi_id() **are guaranteed to be called
only on control devices** by the framework's dispatch layer.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 2 +
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 19 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 62 +++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 +
.../nbl_hw_leonis/nbl_resource_leonis.c | 311 ++++++++++++++++++
.../nbl_hw_leonis/nbl_resource_leonis.h | 10 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 120 +++++++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 74 +++++
.../nbl/nbl_include/nbl_def_common.h | 19 ++
.../nbl/nbl_include/nbl_def_hw.h | 4 +
.../nbl/nbl_include/nbl_def_resource.h | 29 ++
.../nbl/nbl_include/nbl_include.h | 6 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
14 files changed, 680 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 6dc1539cee1f..831b6cb51c37 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -6,4 +6,6 @@ obj-$(CONFIG_NBL) := nbl.o
nbl-objs += nbl_common/nbl_common.o \
nbl_channel/nbl_channel.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
+ nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
+ nbl_hw/nbl_resource.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
index fc9b489ca02c..149681dd216d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -7,6 +7,25 @@
#include <linux/jhash.h>
#include "nbl_common.h"
+/*
+ * nbl_common_func_id_to_rel_pf_id - convert absolute PF id to relative PF id
+ *
+ * Leonis currently uses fixed mgt_pf = 0. This helper maintains unified
+ * conversion logic for future platforms supporting non-zero management PF.
+ * Return: 0 on success, -EINVAL if pf_id < common->mgt_pf or rel_pf_id is NULL.
+ */
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id)
+{
+ if (!rel_pf_id)
+ return -EINVAL;
+
+ if (pf_id < common->mgt_pf)
+ return -EINVAL;
+ *rel_pf_id = pf_id - common->mgt_pf;
+ return 0;
+}
+
static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
{
u32 hash;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 66861d684220..6d7f000dde85 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -16,11 +16,13 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_resource_ops_tbl *resource_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
+ struct nbl_resource_mgt *res_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 7faac0718038..ddfb4b0aa653 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -10,6 +10,18 @@
#include <linux/bitfield.h>
#include "nbl_hw_leonis.h"
+static void nbl_hw_read_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 i;
+
+ if (len % 4)
+ return;
+
+ for (i = 0; i < len / 4; i++)
+ data[i] = nbl_mbx_rd32(hw_mgt, reg + i * sizeof(u32));
+}
+
static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
const u32 *data, u32 len)
{
@@ -52,6 +64,20 @@ static void nbl_hw_wr_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg,
spin_unlock(&hw_mgt->reg_lock);
}
+/*
+ * Per hardware spec, registers reset to zero after cold boot, FLR and bus
+ * reset.Zero value indicates firmware not initialized; upper layer retries
+ * setup. No stale valid configuration can persist.
+ */
+static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
+{
+ u32 data;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW6_OFFSET, &data,
+ sizeof(data));
+ *eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -130,6 +156,15 @@ static void nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask)
sizeof(*pf_mask));
}
+static void nbl_hw_get_real_bus(struct nbl_hw_mgt *hw_mgt, u8 *bus)
+{
+ u32 data;
+
+ nbl_hw_rd_regs_lock(hw_mgt, NBL_PCIE_HOST_TL_CFG_BUSDEV, &data,
+ sizeof(data));
+ *bus = FIELD_GET(NBL_PCIE_BUS_MASK, data);
+}
+
static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function)
{
@@ -143,6 +178,25 @@ static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
sizeof(data));
}
+/*
+ * Per hardware spec, registers reset to zero after cold boot, FLR and bus
+ * reset.Zero value indicates firmware not initialized; upper layer retries
+ * setup. No stale valid configuration can persist.
+ */
+static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board_info)
+{
+ u32 data = 0;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW3_OFFSET, &data,
+ sizeof(data));
+ board_info->eth_num = FIELD_GET(NBL_FW_BOARD_DW3_PORT_NUM_MASK, data);
+ board_info->eth_speed =
+ FIELD_GET(NBL_FW_BOARD_DW3_PORT_SPEED_MASK, data);
+ board_info->p4_version =
+ FIELD_GET(NBL_FW_BOARD_DW3_P4_VERSION_MASK, data);
+}
+
static struct nbl_hw_ops hw_ops = {
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
@@ -150,8 +204,12 @@ static struct nbl_hw_ops hw_ops = {
.stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
.stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
.get_host_pf_mask = nbl_hw_get_host_pf_mask,
+ .get_real_bus = nbl_hw_get_real_bus,
+
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .get_fw_eth_map = nbl_hw_get_fw_eth_map,
+ .get_board_info = nbl_hw_get_board_info,
};
/* Structure starts here, adding an op should not modify anything below */
@@ -182,7 +240,9 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
if (!hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
- !hw_ops.get_host_pf_mask || !hw_ops.cfg_mailbox_qinfo)
+ !hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
+ !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
hw_ops_tbl->priv = hw_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 1d2dd10e6239..1dee07cb7156 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -67,4 +67,18 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_K_PF_MASK_REG (NBL_INTF_HOST_PCIE_BASE + 0x00001004)
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
+#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+#define NBL_FW_BOARD_CONFIG 0x200
+#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
+#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
+
+#define NBL_FW_BOARD_DW3_PORT_TYPE_MASK BIT(0)
+#define NBL_FW_BOARD_DW3_PORT_NUM_MASK GENMASK(7, 1)
+#define NBL_FW_BOARD_DW3_PORT_SPEED_MASK GENMASK(9, 8)
+#define NBL_FW_BOARD_DW3_GPIO_TYPE_MASK GENMASK(12, 10)
+#define NBL_FW_BOARD_DW3_P4_VERSION_MASK GENMASK(13, 13)
+
+#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
+#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
new file mode 100644
index 000000000000..029c1178f393
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include <linux/bits.h>
+#include "nbl_resource_leonis.h"
+
+static struct nbl_resource_ops res_ops = {
+ .get_vsi_id = nbl_res_func_id_to_vsi_id,
+ .get_eth_id = nbl_res_get_eth_id,
+};
+
+static struct nbl_resource_mgt *
+nbl_res_setup_res_mgt(struct nbl_common_info *common)
+{
+ struct nbl_resource_info *resource_info;
+ struct nbl_resource_mgt *res_mgt;
+ struct device *dev = common->dev;
+
+ res_mgt = devm_kzalloc(dev, sizeof(*res_mgt), GFP_KERNEL);
+ if (!res_mgt)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->common = common;
+
+ resource_info =
+ devm_kzalloc(dev, sizeof(*resource_info), GFP_KERNEL);
+ if (!resource_info)
+ return ERR_PTR(-ENOMEM);
+ res_mgt->resource_info = resource_info;
+
+ return res_mgt;
+}
+
+static struct nbl_resource_ops_tbl *
+nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+
+ res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
+ if (!res_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+
+ res_ops_tbl->ops = &res_ops;
+ res_ops_tbl->priv = res_mgt;
+
+ return res_ops_tbl;
+}
+
+static int nbl_res_ctrl_dev_setup_eth_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_eth_info *eth_info;
+ u32 eth_bitmap = 0, eth_id;
+ u32 eth_num = 0;
+ u32 fw_port_num;
+ int i;
+
+ eth_info = devm_kzalloc(dev, sizeof(*eth_info), GFP_KERNEL);
+ if (!eth_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->eth_info = eth_info;
+
+ fw_port_num = res_mgt->resource_info->board_info.eth_num;
+
+ hw_ops->get_fw_eth_map(res_mgt->hw_ops_tbl->priv, ð_bitmap);
+ if (eth_bitmap & ~((1 << NBL_MAX_ETHERNET) - 1)) {
+ dev_err(dev, "FW reported invalid eth_bitmap 0x%x\n",
+ eth_bitmap);
+ return -EINVAL;
+ }
+ if (fw_port_num != hweight32(eth_bitmap)) {
+ dev_err(dev, "FW inconsistency: port_num=%u, bitmap=0x%x\n",
+ fw_port_num, eth_bitmap);
+ return -EINVAL;
+ }
+ /* Reject zero ports early to avoid delayed topology mismatch error */
+ if (fw_port_num == 0) {
+ dev_warn(dev, "FW reports zero ethernet ports, defer probe waiting FW ready\n");
+ return -EPROBE_DEFER;
+ }
+ if (fw_port_num > NBL_MAX_ETHERNET || fw_port_num == 3) {
+ dev_err(dev, "FW reports %u Ethernet ports, not supported\n",
+ fw_port_num);
+ return -EINVAL;
+ }
+ eth_info->eth_num = fw_port_num;
+ /* Intentional design constraint: each PF maps to exactly one
+ * Ethernet port. This couples PF identity to port identity
+ * and is required by nbl_res_get_eth_id() which indexes
+ * eth_info->eth_id[] by relative PF id.
+ */
+ if (res_mgt->resource_info->max_pf != eth_info->eth_num) {
+ dev_err(dev, "Invalid PF-to-port topology: max_pf=%u, eth_num=%u\n",
+ res_mgt->resource_info->max_pf, eth_info->eth_num);
+ return -EINVAL;
+ }
+
+ /*
+ * Original comment said dual-port board eth_id fixed to 0,2;
+ * Code accepts any contiguous valid bitmap bits (0/1 or 0/2 etc).
+ * Firmware only needs to report correct count of active ports,
+ * no hard-coded fixed bit positions required.
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if ((1 << i) & eth_bitmap) {
+ set_bit(i, eth_info->eth_bitmap);
+ eth_info->eth_id[eth_num] = i;
+ eth_info->logic_eth_id[i] = eth_num;
+ eth_num++;
+ }
+ }
+
+ for (i = 0; i < res_mgt->resource_info->max_pf; i++) {
+ eth_id = eth_info->eth_id[i];
+ eth_info->pf_bitmap[eth_id] |= BIT(i);
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_sriov_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ struct device *dev = common->dev;
+ u8 hw_bus = 0;
+ u16 function;
+ u16 func_id;
+
+ hw_ops->get_real_bus(p, &hw_bus);
+ if (common->function + res_mgt->resource_info->max_pf > NBL_MAX_PF) {
+ dev_err(dev, "PF count exceeds available function space\n");
+ return -EINVAL;
+ }
+ sriov_info = devm_kcalloc(dev, res_mgt->resource_info->max_pf,
+ sizeof(*sriov_info), GFP_KERNEL);
+ if (!sriov_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->sriov_info = sriov_info;
+ /*
+ * common->hw_bus supplies bus number for channel mailbox QINFO mapping.
+ * Execution order guarantee: this assignment runs before
+ * cfg_chan_qinfo_map_table() in nbl_dev_start(), only executed
+ * on control PF path.
+ */
+ common->hw_bus = hw_bus;
+
+ for (func_id = 0; func_id < res_mgt->resource_info->max_pf; func_id++) {
+ sriov_info = res_mgt->resource_info->sriov_info + func_id;
+ function = common->function + func_id;
+ sriov_info->bdf = PCI_DEVID(common->hw_bus,
+ PCI_DEVFN(common->devid, function));
+ }
+
+ return 0;
+}
+
+static int nbl_res_ctrl_dev_vsi_info_init(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ struct nbl_vsi_info *vsi_info;
+ int i;
+
+ vsi_info = devm_kzalloc(dev, sizeof(*vsi_info), GFP_KERNEL);
+ if (!vsi_info)
+ return -ENOMEM;
+
+ res_mgt->resource_info->vsi_info = vsi_info;
+ /*
+ * case 1 one port(1pf)
+ * pf0 (NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0
+ * case 2 two port(2pf)
+ * pf0,pf1(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,512
+ * case 3 four port(4pf)
+ * pf0,pf1,pf2,pf3(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,256,512,768
+ */
+
+ vsi_info->num = eth_info->eth_num;
+ /*
+ * eth_num can be 1/2/4:
+ * - 2/4 ports use dedicated gap constants;
+ * - 1 port falls back to NBL_DEFAULT_VSI_ID_GAP (1024).
+ * All three values produce valid base_id offsets.
+ */
+ for (i = 0; i < vsi_info->num; i++) {
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].base_id =
+ i * nbl_vsi_id_gap(vsi_info->num);
+ vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].num = 1;
+ }
+
+ return 0;
+}
+
+static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ u32 pf_mask = 0;
+ u32 pf_num = 0;
+ int i;
+
+ if (hw_ops->get_host_pf_mask)
+ hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
+ if (pf_mask == 0) {
+ dev_warn(res_mgt->common->dev, "FW not ready, defer probe\n");
+ return -EPROBE_DEFER;
+ }
+ /*
+ * Hardware guarantees pf_mask has contiguous cleared bits
+ * starting from bit 0 (e.g., 0b11111100, not 0b01010101).
+ * This allows us to stop at the first set bit.
+ */
+ for (i = 0; i < NBL_MAX_PF; i++) {
+ if (!(pf_mask & (1 << i)))
+ pf_num++;
+ }
+ if (pf_num == 0 || pf_num > NBL_MAX_ETHERNET || pf_num == 3) {
+ dev_err(res_mgt->common->dev, "Invalid pf_num=%u, reject resource init\n",
+ pf_num);
+ return -EINVAL;
+ }
+ res_mgt->resource_info->max_pf = pf_num;
+
+ return 0;
+}
+
+static void nbl_res_init_board_info(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+
+ if (hw_ops->get_board_info)
+ hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
+ &res_mgt->resource_info->board_info);
+}
+
+static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ int ret = 0;
+
+ if (common->has_ctrl) {
+ nbl_res_init_board_info(res_mgt);
+
+ ret = nbl_res_init_pf_num(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_sriov_info_init(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_setup_eth_info(res_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int nbl_res_init_leonis(struct nbl_adapter *adap)
+{
+ struct nbl_channel_ops_tbl *chan_ops_tbl = adap->intf.channel_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl = adap->intf.hw_ops_tbl;
+ struct nbl_common_info *common = &adap->common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct device *dev = &adap->pdev->dev;
+ struct nbl_resource_mgt *res_mgt;
+ int ret;
+
+ res_mgt = nbl_res_setup_res_mgt(common);
+ if (IS_ERR(res_mgt)) {
+ ret = PTR_ERR(res_mgt);
+ return ret;
+ }
+ res_mgt->chan_ops_tbl = chan_ops_tbl;
+ res_mgt->hw_ops_tbl = hw_ops_tbl;
+
+ ret = nbl_res_start(res_mgt);
+ if (ret)
+ return ret;
+
+ res_ops_tbl = nbl_res_setup_ops(dev, res_mgt);
+ if (IS_ERR(res_ops_tbl)) {
+ ret = PTR_ERR(res_ops_tbl);
+ return ret;
+ }
+ adap->intf.resource_ops_tbl = res_ops_tbl;
+ adap->core.res_mgt = res_mgt;
+
+ return 0;
+}
+
+void nbl_res_remove_leonis(struct nbl_adapter *adap)
+{
+ /*
+ * No resource release here because all memory uses devm managed
+ * allocation
+ */
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
new file mode 100644
index 000000000000..4e61a5c141e5
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_LEONIS_H_
+#define _NBL_RESOURCE_LEONIS_H_
+
+#include "../nbl_resource.h"
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
new file mode 100644
index 000000000000..8dc3bc101eb6
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -0,0 +1,120 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#include <linux/pci.h>
+#include "nbl_resource.h"
+
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ enum nbl_vsi_serv_type dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !vsi_id) {
+ dev_dbg(dev, "No control plane or null vsi output ptr\n");
+ return -EINVAL;
+ }
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= vsi_info->num) {
+ dev_err(dev, "PF %d (diff=%u) exceeds vsi_info->num (%u)\n",
+ pfid, rel_pf_id, vsi_info->num);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_pf_dev_vsi_type_to_hw_vsi_type(res_mgt, type, &dst_type);
+ if (ret) {
+ dev_err(dev, "Invalid vsi type %u func_id %u\n", type, func_id);
+ return ret;
+ }
+ *vsi_id = vsi_info->serv_info[rel_pf_id][dst_type].base_id;
+ return 0;
+}
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
+{
+ struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int j = NBL_VSI_SERV_PF_DATA_TYPE;
+ int pf_id, i;
+
+ if (!common->has_ctrl) {
+ dev_dbg(dev, "No control plane available\n");
+ return -EINVAL;
+ }
+ for (i = 0; i < vsi_info->num; i++) {
+ if (vsi_id >= vsi_info->serv_info[i][j].base_id &&
+ (vsi_id < vsi_info->serv_info[i][j].base_id +
+ vsi_info->serv_info[i][j].num)) {
+ pf_id = i + common->mgt_pf;
+ if (pf_id >= NBL_MAX_PF) {
+ dev_err(dev, "PF ID overflow\n");
+ return -ERANGE;
+ }
+ return pf_id;
+ }
+ }
+
+ dev_dbg(dev, "VSI ID %u not found\n", vsi_id);
+ return -ENOENT;
+}
+
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = res_mgt->common->dev;
+ int pfid = func_id;
+ int rel_pf_id;
+ int abs_pf_id;
+
+ if (!common->has_ctrl || !eth_num || !eth_id || !logic_eth_id)
+ return -EINVAL;
+ abs_pf_id = nbl_res_vsi_id_to_pf_id(res_mgt, vsi_id);
+ if (abs_pf_id < 0) {
+ dev_err(dev, "Failed to get PF ID from VSI ID %u\n", vsi_id);
+ return -EINVAL;
+ }
+ if (abs_pf_id != pfid) {
+ dev_err(dev, "func_id %u does not match pf derived from vsi_id %u\n",
+ pfid, vsi_id);
+ return -EINVAL;
+ }
+ rel_pf_id = abs_pf_id - common->mgt_pf;
+
+ if (rel_pf_id >= eth_info->eth_num) {
+ dev_err(dev, "rel_pf_id %d out of range [0, %u)\n",
+ rel_pf_id, eth_info->eth_num);
+ return -ERANGE;
+ }
+
+ *eth_num = eth_info->eth_num;
+ *eth_id = eth_info->eth_id[rel_pf_id];
+ *logic_eth_id = rel_pf_id;
+ return 0;
+}
+
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type)
+{
+ switch (src_type) {
+ case NBL_VSI_DATA:
+ *dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
+ return 0;
+ default:
+ dev_err_once(res_mgt->common->dev,
+ "Unsupported vsi src_type %u\n", src_type);
+ return -EINVAL;
+ }
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
new file mode 100644
index 000000000000..8650310bf9b3
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_RESOURCE_H_
+#define _NBL_RESOURCE_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_resource_mgt;
+
+/* --------- INFO ---------- */
+struct nbl_sriov_info {
+ unsigned int bdf;
+};
+
+struct nbl_eth_info {
+ DECLARE_BITMAP(eth_bitmap, NBL_MAX_ETHERNET);
+ u8 pf_bitmap[NBL_MAX_ETHERNET];
+ u8 eth_num;
+ u8 resv[3];
+ u8 eth_id[NBL_MAX_PF];
+ u8 logic_eth_id[NBL_MAX_PF];
+};
+
+enum nbl_vsi_serv_type {
+ NBL_VSI_SERV_PF_DATA_TYPE,
+ NBL_VSI_SERV_MAX_TYPE,
+};
+
+struct nbl_vsi_serv_info {
+ u16 base_id;
+ u16 num;
+};
+
+struct nbl_vsi_info {
+ u16 num;
+ struct nbl_vsi_serv_info serv_info[NBL_MAX_ETHERNET]
+ [NBL_VSI_SERV_MAX_TYPE];
+};
+
+struct nbl_resource_info {
+ struct nbl_sriov_info *sriov_info;
+ struct nbl_eth_info *eth_info;
+ struct nbl_vsi_info *vsi_info;
+ u8 max_pf;
+ struct nbl_board_port_info board_info;
+};
+
+struct nbl_resource_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_info *resource_info;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_hw_ops_tbl *hw_ops_tbl;
+ struct nbl_interrupt_mgt *intr_mgt;
+};
+
+int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
+int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
+ u16 src_type,
+ enum nbl_vsi_serv_type *dst_type);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index 4f651a227539..af7e7225a879 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -11,6 +11,22 @@
#include <linux/device.h>
#include "nbl_include.h"
+#define NBL_TWO_ETHERNET_PORT 2
+#define NBL_FOUR_ETHERNET_PORT 4
+#define NBL_DEFAULT_VSI_ID_GAP 1024
+#define NBL_TWO_ETHERNET_VSI_ID_GAP 512
+#define NBL_FOUR_ETHERNET_VSI_ID_GAP 256
+
+static inline u32 nbl_vsi_id_gap(u32 m)
+{
+ if (m == NBL_FOUR_ETHERNET_PORT)
+ return NBL_FOUR_ETHERNET_VSI_ID_GAP;
+ else if (m == NBL_TWO_ETHERNET_PORT)
+ return NBL_TWO_ETHERNET_VSI_ID_GAP;
+
+ return NBL_DEFAULT_VSI_ID_GAP;
+}
+
struct nbl_common_info {
struct pci_dev *pdev;
struct device *dev;
@@ -24,6 +40,7 @@ struct nbl_common_info {
u8 devid;
u8 bus;
u8 hw_bus;
+ u16 mgt_pf;
u8 has_ctrl;
u8 has_net;
@@ -37,6 +54,8 @@ struct nbl_hash_tbl_key {
u16 resv;
};
+int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
+ u32 *rel_pf_id);
struct nbl_hash_tbl_mgt *
nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index 9cba73fe24ac..74ed7be2ab7d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -20,9 +20,13 @@ struct nbl_hw_ops {
void (*stop_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt);
void (*stop_mailbox_txq)(struct nbl_hw_mgt *hw_mgt);
void (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt, u32 *pf_mask);
+ void (*get_real_bus)(struct nbl_hw_mgt *hw_mgt, u8 *bus);
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
+ void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
+ struct nbl_board_port_info *board);
};
struct nbl_hw_ops_tbl {
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
new file mode 100644
index 000000000000..cf034ab589fd
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_RESOURCE_H_
+#define _NBL_DEF_RESOURCE_H_
+
+#include <linux/types.h>
+
+struct nbl_resource_mgt;
+struct nbl_adapter;
+
+struct nbl_resource_ops {
+ int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 type, u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id);
+};
+
+struct nbl_resource_ops_tbl {
+ struct nbl_resource_ops *ops;
+ struct nbl_resource_mgt *priv;
+};
+
+int nbl_res_init_leonis(struct nbl_adapter *adapter);
+void nbl_res_remove_leonis(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 4dc0615ecc82..b10d827f412c 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,12 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_ETHERNET 4
+
+enum {
+ NBL_VSI_DATA = 0,
+};
+
struct nbl_func_caps {
u32 has_ctrl:1;
u32 has_net:1;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 2d30c0eed00f..3e3de6dabb10 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -10,6 +10,7 @@
#include "nbl_include/nbl_include.h"
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
+#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -27,6 +28,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
adapter->pdev = pdev;
common = &adapter->common;
+ common->mgt_pf = 0;
common->pdev = pdev;
common->dev = &pdev->dev;
common->has_ctrl = param->caps.has_ctrl;
@@ -42,7 +44,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_chan_init_common(adapter);
if (ret)
goto chan_init_fail;
+
+ ret = nbl_res_init_leonis(adapter);
+ if (ret)
+ goto res_init_fail;
return adapter;
+res_init_fail:
+ nbl_chan_remove_common(adapter);
chan_init_fail:
nbl_hw_remove_leonis(adapter);
hw_init_fail:
@@ -51,6 +59,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 06/11] net/nebula-matrix: add intr resource implementation
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (4 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 05/11] net/nebula-matrix: add common resource implementation illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 07/11] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
` (4 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add nbl_interrupt module to manage driver global MSI-X vector indices,
and extend hw_ops with callbacks to program vendor-specific internal
MSI-X mapping registers, configure MSI-X metadata, and bind interrupts
to PF mailbox channels.
Key interfaces:
- configure_msix_map:
Allocates driver global MSI-X indices via separate net/other interrupt
bitmaps (interrupt_net_bitmap / interrupt_others_bitmap).
Preallocates all required coherent DMA resources first; only tears down
old hardware configuration after all new allocations succeed.
Programs NBL_PCOMPLETER_FUNCTION_MSIX_MAP with the MSI-X table DMA address
and function BDF info. PCI MSI-X vector allocation itself is performed
via nbl_dev_alloc_msix_intr() at upper layer.
- destroy_msix_map:
Releases allocated global vector indices, clears hardware MSI-X mappings,
frees DMA buffer and interrupt array. A delay is inserted before releasing
DMA memory to mitigate potential IOMMU faults caused by in-flight hardware
DMA access after MMIO disable.
- set_mailbox_irq:
Enables or disables MSI-X routing for a given PF mailbox by updating
NBL_MAILBOX_QINFO_MAP_REG_ARR with the target global vector id.
- configure_msix_info:
Programs PADPT_HOST_MSIX_INFO and PCOMPLETER_HOST_MSIX_FID_TABLE
to record function BDF and MSI-X mask enable attribute for each global
interrupt vector.
This module manages chip-internal MSI-X routing table and global vector
index space. It does NOT invoke kernel PCI MSI-X allocation helpers
(pci_alloc_irq_vectors, pci_msix_alloc_irq_at, irq_domain); physical
PCI MSI-X vector allocation belongs to separate device layer.
Interrupt manager instance is created during resource startup
(nbl_intr_mgt_start) and attached to resource management context.
Note: Mutual exclusion for configure_msix_map, destroy_msix_map, and
set_mailbox_irq is enforced by dispatch layer. These routines assume
caller holds required locks.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 149 ++++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 42 ++
.../nbl_hw_leonis/nbl_resource_leonis.c | 21 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 405 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 21 +
.../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 30 ++
.../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 35 ++
.../nbl/nbl_include/nbl_def_hw.h | 10 +
.../nbl/nbl_include/nbl_def_resource.h | 6 +
.../nbl/nbl_include/nbl_include.h | 1 +
12 files changed, 713 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 831b6cb51c37..bcedf278bab2 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -8,4 +8,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
+ nbl_hw/nbl_interrupt.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index ddfb4b0aa653..5e23b942b257 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -34,6 +34,30 @@ static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
}
+static void nbl_hw_rd_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
+}
+
+static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
+ u32 len)
+{
+ u32 size = len / 4;
+ u32 i;
+
+ if (len % 4)
+ return;
+ for (i = 0; i < size; i++)
+ wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
+}
+
static void nbl_hw_rd_regs_lock(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
u32 len)
{
@@ -78,6 +102,121 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+/*
+ * nbl_hw_set_mailbox_irq - read-modify-write of NBL_MAILBOX_QINFO_MAP_REG_ARR
+ *
+ * Note: This RMW is currently safe because the two callers are strictly
+ * sequential: nbl_hw_cfg_mailbox_qinfo() runs at init, nbl_hw_set_mailbox_irq()
+ * runs at start. Future reset/hot-add/VF-reinit paths must preserve this
+ * ordering; do not introduce concurrent access to this register without
+ * making the RMW atomic (e.g., hold reg_lock across read+write, or use
+ * cached state instead of read).
+ */
+static void nbl_hw_set_mailbox_irq(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id)
+{
+ u32 data = 0;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ data &= ~(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK |
+ NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK);
+ if (en_msix)
+ data |= FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_MASK,
+ global_vec_id) |
+ FIELD_PREP(NBL_MAILBOX_QINFO_MAP_MSIX_IDX_VALID_MASK,
+ 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id), &data,
+ sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+ nbl_flush_writes(hw_mgt);
+}
+
+static void nbl_hw_cfg_msix_map(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function)
+{
+ struct nbl_function_msix_map function_msix_map;
+
+ memset(&function_msix_map, 0, sizeof(function_msix_map));
+ if (valid) {
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ /* use ctrl dev's bdf, because the dma memory was
+ * allocated by it
+ */
+ function_msix_map.data[2] =
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK,
+ function) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_BUS_MASK, bus) |
+ FIELD_PREP(NBL_FUNCTION_MSIX_MAP_VALID_MASK, 1);
+ } else {
+ /* Write full MSIX entry atomically under reg_lock,
+ * hardware cannot read partial updated entry
+ */
+ function_msix_map.data[0] = lower_32_bits(dma_addr);
+ function_msix_map.data[1] = upper_32_bits(dma_addr);
+ function_msix_map.data[2] = 0;
+ }
+
+ nbl_hw_wr_regs_lock(hw_mgt,
+ NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(func_id),
+ function_msix_map.data, sizeof(function_msix_map));
+}
+
+static void nbl_hw_cfg_msix_info(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function, bool msix_mask_en)
+{
+ u32 host_msix_fid = 0;
+ struct nbl_host_msix_info msix_info;
+
+ memset(&msix_info, 0, sizeof(msix_info));
+ if (valid) {
+ host_msix_fid =
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK,
+ func_id) |
+ FIELD_PREP(NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK,
+ 1);
+
+ msix_info.data[1] =
+ FIELD_PREP(NBL_HOST_MSIX_INFO_FUNCTION_MASK, function) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_DEVID_MASK, devid) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_BUS_MASK, bus) |
+ FIELD_PREP(NBL_HOST_MSIX_INFO_VALID_MASK, 1);
+
+ if (msix_mask_en)
+ msix_info.data[1] |=
+ FIELD_PREP(NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK, 1);
+ }
+ spin_lock(&hw_mgt->reg_lock);
+ /*
+ * Programming order rule:
+ * Enable: PADPT_HOST_MSIX_INFO -> PCOMPLETER_HOST_MSIX_FID_TABLE
+ * Teardown: reverse order, clear FID VLD first to avoid inconsistent
+ * state
+ */
+ if (valid) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ } else {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
+ &host_msix_fid, sizeof(host_msix_fid));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
+ msix_info.data, sizeof(msix_info));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx)
{
@@ -198,6 +337,10 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .cfg_msix_map = nbl_hw_cfg_msix_map,
+ .cfg_msix_info = nbl_hw_cfg_msix_info,
+ .flush_write = nbl_flush_writes,
+
.update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
.config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
.config_mailbox_txq = nbl_hw_config_mailbox_txq,
@@ -207,6 +350,7 @@ static struct nbl_hw_ops hw_ops = {
.get_real_bus = nbl_hw_get_real_bus,
.cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
+ .set_mailbox_irq = nbl_hw_set_mailbox_irq,
.get_fw_eth_map = nbl_hw_get_fw_eth_map,
.get_board_info = nbl_hw_get_board_info,
@@ -237,11 +381,12 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
if (!hw_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!hw_ops.update_mailbox_queue_tail_ptr ||
+ if (!hw_ops.cfg_msix_map || !hw_ops.cfg_msix_info ||
+ !hw_ops.flush_write || !hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
!hw_ops.get_host_pf_mask || !hw_ops.get_real_bus ||
- !hw_ops.cfg_mailbox_qinfo ||
+ !hw_ops.cfg_mailbox_qinfo || !hw_ops.set_mailbox_irq ||
!hw_ops.get_fw_eth_map || !hw_ops.get_board_info)
return ERR_PTR(-EINVAL);
hw_ops_tbl->ops = &hw_ops;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 1dee07cb7156..1d6d614be7d6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -68,6 +68,48 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_HOST_TL_CFG_BUSDEV (NBL_INTF_HOST_PCIE_BASE + 0x11040)
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
+
+/* -------- HOST_PADPT -------- */
+/* host_padpt host_msix_info */
+#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
+ (vector_id) * sizeof(struct nbl_host_msix_info))
+
+#define NBL_HOST_MSIX_INFO_DWLEN 2
+/* data[0] */
+#define NBL_HOST_MSIX_INFO_INTRL_PNUM_MASK GENMASK(15, 0)
+#define NBL_HOST_MSIX_INFO_INTRL_RATE_MASK GENMASK(31, 16)
+/* data[1] */
+#define NBL_HOST_MSIX_INFO_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_HOST_MSIX_INFO_DEVID_MASK GENMASK(7, 3)
+#define NBL_HOST_MSIX_INFO_BUS_MASK GENMASK(15, 8)
+#define NBL_HOST_MSIX_INFO_VALID_MASK BIT(16)
+#define NBL_HOST_MSIX_INFO_MSIX_MASK_EN_MASK BIT(17)
+struct nbl_host_msix_info {
+ u32 data[NBL_HOST_MSIX_INFO_DWLEN];
+};
+
+/* -------- HOST_PCOMPLETER -------- */
+/* pcompleter_host pcompleter_host_virtio_qid_map_table */
+#define NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x00004000 + \
+ (i) * sizeof(struct nbl_function_msix_map))
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(i) \
+ (NBL_INTF_HOST_PCOMPLETER_BASE + 0x0003a000 + (i) * sizeof(u32))
+
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_FID_MASK GENMASK(9, 0)
+#define NBL_PCOMPLETER_HOST_MSIX_FID_TABLE_VLD_MASK BIT(10)
+
+#define NBL_FUNC_MSIX_MAP_DWLEN 4
+/* data[2] */
+#define NBL_FUNCTION_MSIX_MAP_FUNCTION_MASK GENMASK(2, 0)
+#define NBL_FUNCTION_MSIX_MAP_DEVID_MASK GENMASK(7, 3)
+#define NBL_FUNCTION_MSIX_MAP_BUS_MASK GENMASK(15, 8)
+#define NBL_FUNCTION_MSIX_MAP_VALID_MASK BIT(16)
+struct nbl_function_msix_map {
+ u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
+};
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index 029c1178f393..e192bd18380d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -10,6 +10,9 @@
static struct nbl_resource_ops res_ops = {
.get_vsi_id = nbl_res_func_id_to_vsi_id,
.get_eth_id = nbl_res_get_eth_id,
+ .cfg_msix_map = nbl_res_intr_cfg_msix_map,
+ .destroy_msix_map = nbl_res_intr_destroy_msix_map,
+ .set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
};
static struct nbl_resource_mgt *
@@ -41,7 +44,10 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
if (!res_ops_tbl)
return ERR_PTR(-ENOMEM);
-
+ if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
+ !res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
+ !res_ops.set_mailbox_irq)
+ return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
@@ -65,7 +71,6 @@ static int nbl_res_ctrl_dev_setup_eth_info(struct nbl_resource_mgt *res_mgt)
res_mgt->resource_info->eth_info = eth_info;
fw_port_num = res_mgt->resource_info->board_info.eth_num;
-
hw_ops->get_fw_eth_map(res_mgt->hw_ops_tbl->priv, ð_bitmap);
if (eth_bitmap & ~((1 << NBL_MAX_ETHERNET) - 1)) {
dev_err(dev, "FW reported invalid eth_bitmap 0x%x\n",
@@ -207,8 +212,7 @@ static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
u32 pf_num = 0;
int i;
- if (hw_ops->get_host_pf_mask)
- hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
+ hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv, &pf_mask);
if (pf_mask == 0) {
dev_warn(res_mgt->common->dev, "FW not ready, defer probe\n");
return -EPROBE_DEFER;
@@ -236,9 +240,8 @@ static void nbl_res_init_board_info(struct nbl_resource_mgt *res_mgt)
{
struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
- if (hw_ops->get_board_info)
- hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
- &res_mgt->resource_info->board_info);
+ hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
+ &res_mgt->resource_info->board_info);
}
static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
@@ -264,6 +267,10 @@ static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
if (ret)
return ret;
+
+ ret = nbl_intr_mgt_start(res_mgt);
+ if (ret)
+ return ret;
}
return 0;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index 4e61a5c141e5..1da2abcaf00f 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -7,4 +7,5 @@
#define _NBL_RESOURCE_LEONIS_H_
#include "../nbl_resource.h"
+#include "../nbl_interrupt.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
new file mode 100644
index 000000000000..34410d0d9022
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
@@ -0,0 +1,405 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/bitfield.h>
+#include "nbl_interrupt.h"
+
+#define NBL_MSIX_DMA_SYNC_MIN_US 100
+#define NBL_MSIX_DMA_SYNC_MAX_US 120
+
+static void nbl_intr_release_bitmap(struct nbl_resource_mgt *res_mgt,
+ u16 *vec_buf, u16 cnt)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ u16 bit;
+ u16 i;
+
+ if (!vec_buf || cnt == 0)
+ return;
+
+ for (i = 0; i < cnt; i++) {
+ u16 intr_index = vec_buf[i];
+
+ if (intr_index >= NBL_NET_INTR_BASE) {
+ bit = intr_index - NBL_NET_INTR_BASE;
+ if (bit < NBL_MAX_NET_INTERRUPT)
+ clear_bit(bit, intr_mgt->intr_net_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid net intr index %u\n",
+ intr_index);
+ } else {
+ if (intr_index < NBL_MAX_OTHER_INTERRUPT)
+ clear_bit(intr_index,
+ intr_mgt->intr_other_bmap);
+ else
+ dev_warn(res_mgt->common->dev,
+ "invalid other intr index %u\n",
+ intr_index);
+ }
+ }
+}
+
+/**
+ * nbl_res_intr_destroy_msix_map - tear down MSI-X mapping for specified PF
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ *
+ * Clear interrupt bitmaps and disable MSI-X via hardware callbacks.
+ * DMA coherent table buffer uses devm allocation and remains valid
+ * until device removal to avoid IOMMU faults caused by in-flight hardware DMA.
+ *
+ * Caller note: this function has no internal locking. Serialization
+ * must be guaranteed at upper dispatch layer.
+ * Caller must ensure all concurrent resource operations are excluded.
+ *
+ * Caller guarantee: hardware cannot issue outstanding DMA to these regions,
+ * all relevant IRQs have been released or not yet configured.
+ *
+ * Return: 0 on success, negative errno on failure
+ */
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_msix_map_table *msix_map_table;
+ u16 *interrupts;
+ u16 intr_num, i;
+
+ if (!res_mgt->common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+
+ intr_num = intr_mgt->func_intr_res[func_id].num_interrupts;
+ interrupts = intr_mgt->func_intr_res[func_id].interrupts;
+ msix_map_table = &intr_mgt->func_intr_res[func_id].msix_map_table;
+
+ if (!interrupts || !msix_map_table->base_addr) {
+ dev_dbg(dev, "No interrupt resources for func %u\n", func_id);
+ return 0;
+ }
+
+ for (i = 0; i < intr_num; i++) {
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, false, interrupts[i],
+ 0, 0, 0, false);
+ }
+
+ nbl_intr_release_bitmap(res_mgt, interrupts, intr_num);
+
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ false, 0, 0, 0, 0);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+ /*
+ * Hardware provides no idle status register for MSIX map DMA engine.
+ * Use bounded sleep to mitigate race between posted MMIO disable writes
+ * and ongoing in-flight table read DMA access.
+ * After sleep, hardware no longer performs DMA access to MSIX table,
+ * safe to free DMA memory now.
+ */
+ usleep_range(NBL_MSIX_DMA_SYNC_MIN_US, NBL_MSIX_DMA_SYNC_MAX_US);
+
+ /*
+ * Now safe to release old MSIX DMA memory, prevents devres accumulation
+ * leak Since hardware DMA has quiesced after sleep, no IOMMU fault risk
+ * anymore.
+ */
+ if (msix_map_table->base_addr) {
+ dmam_free_coherent(dev, msix_map_table->size,
+ msix_map_table->base_addr,
+ msix_map_table->dma);
+ }
+
+ /* Release runtime allocated interrupt vector buffer */
+ kfree(intr_mgt->func_intr_res[func_id].interrupts);
+ intr_mgt->func_intr_res[func_id].interrupts = NULL;
+ intr_mgt->func_intr_res[func_id].num_interrupts = 0;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = 0;
+
+ /* Clear stale MSIX table pointers for safety */
+ msix_map_table->base_addr = NULL;
+ msix_map_table->dma = 0;
+ msix_map_table->size = 0;
+
+ return 0;
+}
+
+/**
+ * nbl_res_intr_cfg_msix_map - allocate & program MSI-X mapping table
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @num_net_msix: required net data interrupt vectors
+ * @num_others_msix: required control interrupt vectors
+ * @net_msix_mask_en: enable mask for net interrupt entries
+ *
+ * Allocate interrupt vectors and coherent DMA table in advance;
+ * only destroy old configuration once all allocations succeed.
+ *
+ * Note: There exists a transient window after tearing down old MSI-X hardware
+ * state before programming new mapping. Atomic table swap is unsupported on
+ * current silicon, this gap is accepted as hardware limitation.
+ *
+ * Caller note: this function has no internal locking. Serialization
+ * must be guaranteed at upper dispatch layer.
+ *
+ * Old MSIX table memory will be explicitly freed inside destroy_msix_map()
+ * after waiting for hardware DMA quiesce, so repeated reconfiguration will not
+ * accumulate devres-managed DMA memory.
+ * Return: 0 on success, negative errno on failure
+ */
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_msix_map_table *tmp_msix_tbl = NULL;
+ struct nbl_msix_map_table *official_tbl;
+ struct nbl_msix_map *msix_map_entries;
+ struct device *dev = common->dev;
+ u16 requested, intr_index;
+ u8 bus, devid, function;
+ bool msix_mask_en = false;
+ u16 *tmp_interrupts = NULL;
+ u16 global_vec;
+ int ret = 0;
+ u16 i;
+
+ if (!common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "Invalid func_id %u\n", func_id);
+ return -EINVAL;
+ }
+ if (num_net_msix == 0 && num_others_msix == 0) {
+ dev_err(dev, "MSI-X vector count cannot both be zero\n");
+ return -EINVAL;
+ }
+ if (num_net_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES ||
+ num_others_msix > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "MSI-X count out of limit: net=%u, others=%u\n",
+ num_net_msix, num_others_msix);
+ return -EINVAL;
+ }
+
+ if (check_add_overflow(num_net_msix, num_others_msix, &requested) ||
+ requested > NBL_MSIX_MAP_TABLE_MAX_ENTRIES) {
+ dev_err(dev, "Total MSI-X vectors %u exceeds maximum %u\n",
+ requested, NBL_MSIX_MAP_TABLE_MAX_ENTRIES);
+ return -EINVAL;
+ }
+
+ ret = nbl_res_func_id_to_bdf(res_mgt, func_id, &bus, &devid, &function);
+ if (ret)
+ return ret;
+
+ /*
+ * Phase1: Pre-allocate ALL new resources first.
+ * Do NOT destroy old configuration before all allocations succeed.
+ */
+ tmp_msix_tbl = kzalloc_obj(*tmp_msix_tbl);
+ if (!tmp_msix_tbl) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ tmp_msix_tbl->size =
+ sizeof(struct nbl_msix_map) * NBL_MSIX_MAP_TABLE_MAX_ENTRIES;
+ /*
+ * Hardware requires fixed stride table layout; allocate full size
+ * even when only partial entries are used. Memory managed by devm.
+ */
+ tmp_msix_tbl->base_addr = dmam_alloc_coherent(dev, tmp_msix_tbl->size,
+ &tmp_msix_tbl->dma,
+ GFP_KERNEL);
+ if (!tmp_msix_tbl->base_addr) {
+ dev_err(dev, "Failed to allocate DMA memory for MSIX table\n");
+ ret = -ENOMEM;
+ goto free_tmp_tbl;
+ }
+
+ tmp_interrupts = kcalloc(requested, sizeof(tmp_interrupts[0]),
+ GFP_KERNEL);
+ if (!tmp_interrupts) {
+ ret = -ENOMEM;
+ goto free_tmp_tbl;
+ }
+
+ /* Allocate net interrupt vectors */
+ for (i = 0; i < num_net_msix; i++) {
+ intr_index = find_first_zero_bit(intr_mgt->intr_net_bmap,
+ NBL_MAX_NET_INTERRUPT);
+ if (intr_index == NBL_MAX_NET_INTERRUPT) {
+ dev_err(dev, "No free net interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs;
+ }
+ tmp_interrupts[i] = intr_index + NBL_NET_INTR_BASE;
+ set_bit(intr_index, intr_mgt->intr_net_bmap);
+ }
+
+ /* Allocate other interrupt vectors */
+ for (; i < requested; i++) {
+ intr_index =
+ find_first_zero_bit(intr_mgt->intr_other_bmap,
+ NBL_MAX_OTHER_INTERRUPT);
+ if (intr_index == NBL_MAX_OTHER_INTERRUPT) {
+ dev_err(dev, "No free control interrupt vectors left\n");
+ ret = -EAGAIN;
+ goto release_vecs;
+ }
+ tmp_interrupts[i] = intr_index;
+ set_bit(intr_index, intr_mgt->intr_other_bmap);
+ }
+
+ /*
+ * Phase2: All new resource allocation succeeded.
+ * Now tear down old MSIX hardware configuration.
+ */
+ ret = nbl_res_intr_destroy_msix_map(res_mgt, func_id);
+ if (ret)
+ goto release_vecs;
+
+ /* Swap temporary resources into official entry */
+ official_tbl = &intr_mgt->func_intr_res[func_id].msix_map_table;
+ official_tbl->base_addr = tmp_msix_tbl->base_addr;
+ official_tbl->dma = tmp_msix_tbl->dma;
+ official_tbl->size = tmp_msix_tbl->size;
+ kfree(tmp_msix_tbl);
+ tmp_msix_tbl = NULL;
+
+ intr_mgt->func_intr_res[func_id].interrupts = tmp_interrupts;
+ intr_mgt->func_intr_res[func_id].num_interrupts = requested;
+ intr_mgt->func_intr_res[func_id].num_net_interrupts = num_net_msix;
+ tmp_interrupts = NULL;
+
+ /* Fill MSIX map table and program hardware */
+ msix_map_entries = official_tbl->base_addr;
+ for (i = 0; i < requested; i++) {
+ global_vec = intr_mgt->func_intr_res[func_id].interrupts[i];
+ msix_map_entries[i].data =
+ cpu_to_le16(FIELD_PREP(NBL_MSIX_MAP_VALID_MASK, 1) |
+ FIELD_PREP(NBL_MSIX_MAP_INDEX_MASK,
+ global_vec));
+
+ msix_mask_en = (i < num_net_msix && net_msix_mask_en) ? true :
+ false;
+ hw_ops->cfg_msix_info(res_mgt->hw_ops_tbl->priv,
+ func_id, true, global_vec,
+ bus, devid, function,
+ msix_mask_en);
+ }
+
+ /* Flush CPU writes to coherent memory before hardware DMA access */
+ dma_wmb();
+
+ hw_ops->cfg_msix_map(res_mgt->hw_ops_tbl->priv, func_id,
+ true, official_tbl->dma, common->hw_bus,
+ common->devid, common->function);
+ hw_ops->flush_write(res_mgt->hw_ops_tbl->priv);
+
+out:
+ return ret;
+
+release_vecs:
+ nbl_intr_release_bitmap(res_mgt, tmp_interrupts, i);
+free_tmp_tbl:
+ /* Release DMA buffer allocated by dmam_alloc_coherent first */
+ if (tmp_msix_tbl && tmp_msix_tbl->base_addr) {
+ dmam_free_coherent(dev, tmp_msix_tbl->size,
+ tmp_msix_tbl->base_addr,
+ tmp_msix_tbl->dma);
+ }
+ kfree(tmp_msix_tbl);
+ kfree(tmp_interrupts);
+ tmp_msix_tbl = NULL;
+ tmp_interrupts = NULL;
+ goto out;
+}
+
+/**
+ * nbl_res_intr_set_mailbox_irq - bind mailbox IRQ to specified vector
+ * @res_mgt: resource management instance
+ * @func_id: target function identifier
+ * @vector_id: index inside local interrupt array
+ * @en_msix: enable/disable mailbox interrupt
+ *
+ * Caller note: this function has no internal locking. Serialization
+ * must be guaranteed at upper dispatch layer.
+ *
+ * Return: 0 on success, negative errno on parameter check or hw failure
+ */
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix)
+{
+ struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_common_info *common = res_mgt->common;
+ struct device *dev = common->dev;
+ u16 global_vec_id;
+
+ if (!common->has_ctrl)
+ return -EINVAL;
+ if (func_id >= NBL_MAX_FUNC) {
+ dev_err(dev, "func_id %u out of range\n", func_id);
+ return -EINVAL;
+ }
+ if (!intr_mgt->func_intr_res[func_id].interrupts) {
+ dev_err(dev, "func %u MSIX map not configured\n", func_id);
+ return -ENODEV;
+ }
+ if (vector_id >= intr_mgt->func_intr_res[func_id].num_interrupts) {
+ dev_err(dev, "vector_id %u out of range (max %u)\n",
+ vector_id,
+ intr_mgt->func_intr_res[func_id].num_interrupts - 1);
+ return -EINVAL;
+ }
+
+ global_vec_id = intr_mgt->func_intr_res[func_id].interrupts[vector_id];
+ hw_ops->set_mailbox_irq(res_mgt->hw_ops_tbl->priv, func_id,
+ en_msix, global_vec_id);
+
+ return 0;
+}
+
+static struct nbl_interrupt_mgt *nbl_intr_setup_mgt(struct device *dev)
+{
+ struct nbl_interrupt_mgt *intr_mgt;
+
+ intr_mgt = devm_kzalloc(dev, sizeof(*intr_mgt), GFP_KERNEL);
+ if (!intr_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ bitmap_zero(intr_mgt->intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ bitmap_zero(intr_mgt->intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+
+ return intr_mgt;
+}
+
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt)
+{
+ struct device *dev = res_mgt->common->dev;
+ struct nbl_interrupt_mgt *intr_mgt;
+ int ret;
+
+ intr_mgt = nbl_intr_setup_mgt(dev);
+ if (IS_ERR(intr_mgt)) {
+ ret = PTR_ERR(intr_mgt);
+ return ret;
+ }
+ res_mgt->intr_mgt = intr_mgt;
+ return 0;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
new file mode 100644
index 000000000000..a4b2de5b2351
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_INTERRUPT_H_
+#define _NBL_INTERRUPT_H_
+
+#include "nbl_resource.h"
+
+#define NBL_MSIX_MAP_TABLE_MAX_ENTRIES 1024
+int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id);
+int nbl_res_intr_cfg_msix_map(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 num_net_msix,
+ u16 num_others_msix,
+ bool net_msix_mask_en);
+int nbl_res_intr_set_mailbox_irq(struct nbl_resource_mgt *res_mgt,
+ u16 func_id, u16 vector_id,
+ bool en_msix);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
index 8dc3bc101eb6..a6e2c7eeed0e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
@@ -68,6 +68,36 @@ int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
return -ENOENT;
}
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function)
+{
+ struct nbl_common_info *common = res_mgt->common;
+ struct nbl_sriov_info *sriov_info;
+ int pfid = func_id;
+ u8 pf_bus, devfn;
+ u32 rel_pf_id;
+ int ret;
+
+ if (!common->has_ctrl || !bus || !dev || !function)
+ return -EINVAL;
+ ret = nbl_common_func_id_to_rel_pf_id(common, pfid, &rel_pf_id);
+ if (ret)
+ return ret;
+ if (rel_pf_id >= res_mgt->resource_info->max_pf) {
+ dev_err(common->dev, "PF ID %u exceeds maximum supported PF count %u\n",
+ pfid, res_mgt->resource_info->max_pf);
+ return -ERANGE;
+ }
+ sriov_info = res_mgt->resource_info->sriov_info + rel_pf_id;
+ pf_bus = PCI_BUS_NUM(sriov_info->bdf);
+ devfn = sriov_info->bdf & 0xff;
+ *bus = pf_bus;
+ *dev = PCI_SLOT(devfn);
+ *function = PCI_FUNC(devfn);
+
+ return 0;
+}
+
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
{
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
index 8650310bf9b3..7f7624e9215d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
@@ -17,6 +17,38 @@
struct nbl_resource_mgt;
+/* --------- INTERRUPT ---------- */
+#define NBL_MAX_OTHER_INTERRUPT 1024
+#define NBL_MAX_NET_INTERRUPT 4096
+#define NBL_NET_INTR_BASE NBL_MAX_OTHER_INTERRUPT
+
+#define NBL_MSIX_MAP_VALID_MASK BIT(0)
+#define NBL_MSIX_MAP_INDEX_MASK GENMASK(13, 1)
+#define NBL_MSIX_MAP_RSV_MASK GENMASK(15, 14)
+
+struct nbl_msix_map {
+ __le16 data;
+};
+
+struct nbl_msix_map_table {
+ struct nbl_msix_map *base_addr;
+ dma_addr_t dma;
+ size_t size;
+};
+
+struct nbl_func_interrupt_resource_mng {
+ u16 num_interrupts;
+ u16 num_net_interrupts;
+ u16 *interrupts;
+ struct nbl_msix_map_table msix_map_table;
+};
+
+struct nbl_interrupt_mgt {
+ DECLARE_BITMAP(intr_net_bmap, NBL_MAX_NET_INTERRUPT);
+ DECLARE_BITMAP(intr_other_bmap, NBL_MAX_OTHER_INTERRUPT);
+ struct nbl_func_interrupt_resource_mng func_intr_res[NBL_MAX_FUNC];
+};
+
/* --------- INFO ---------- */
struct nbl_sriov_info {
unsigned int bdf;
@@ -66,8 +98,11 @@ struct nbl_resource_mgt {
int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
int nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
+int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u8 *bus, u8 *dev, u8 *function);
int nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 vsi_id, u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt);
int nbl_res_pf_dev_vsi_type_to_hw_vsi_type(struct nbl_resource_mgt *res_mgt,
u16 src_type,
enum nbl_vsi_serv_type *dst_type);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index 74ed7be2ab7d..d5d5985db5a9 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -11,6 +11,14 @@
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, dma_addr_t dma_addr, u8 bus,
+ u8 devid, u8 function);
+ void (*cfg_msix_info)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool valid, u16 interrupt_id, u8 bus,
+ u8 devid, u8 function,
+ bool net_msix_mask_en);
+ void (*flush_write)(struct nbl_hw_mgt *hw_mgt);
void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
u16 tail_ptr, u8 txrx);
void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
@@ -24,6 +32,8 @@ struct nbl_hw_ops {
void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
u8 bus, u8 devid, u8 function);
+ void (*set_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
+ bool en_msix, u16 global_vec_id);
void (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt, u32 *eth_map);
void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
struct nbl_board_port_info *board);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index cf034ab589fd..2317381b1e09 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,12 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id);
+ int (*set_mailbox_irq)(struct nbl_resource_mgt *res_mgt, u16 func_id,
+ u16 vector_id, bool en_msix);
int (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 type, u16 *vsi_id);
int (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index b10d827f412c..e0f51bb93a0b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -13,6 +13,7 @@
#define NBL_MAX_PF 8
#define NBL_NEXT_ID(id, max) (((id) + 1) % ((max) + 1))
+#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
enum {
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 07/11] net/nebula-matrix: add chip-wide hardware init/deinit implementation
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (5 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 06/11] net/nebula-matrix: add intr " illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
` (3 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add Leonis chip-wide hardware initialization and deinitialization
logic to setup the full datapath pipeline, including packet engines,
scheduler, buffer stores, queue manager, port traffic shaping,
and flow control, and the driver_status hardware flag consumed
by firmware.
HW layer overview:
The HW layer code is highly chip-specific and may benefit from
additional review since it cannot be cross-checked against other
implementations.
DP sub-init modules (called from nbl_dp_init()):
- dped, uped: Data/User Packet Engine Driver
- dsch: Scheduler core (qid limit init, PSHA enable inside shaping init)
- ustore, dstore: Buffer Store modules
- dvn, uvn: Virtual Descriptor Network modules
- uqm: Queue Manager
- nbl_shaping_init(): Per-port traffic shaping + DSCH PSHA config
Chip init sequence (nbl_hw_init_chip_module()):
1. nbl_dp_init() — All DP sub-modules listed above
2. nbl_intf_init() — Host adapter padpt flow control
- nbl_host_padpt_init() — Host padpt flow control registers
3. nbl_hw_set_driver_status() + nbl_flush_writes()
Chip deinit logic relies on firmware cleanup: only driver_status flag
is cleared and flushed; FW releases partial hardware state automatically.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 26 +
.../nebula-matrix/nbl/nbl_hw/nbl_chip.h | 12 +
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 557 +++++++++++++++++-
.../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 229 +++++++
.../nbl_hw_leonis/nbl_resource_leonis.c | 5 +-
.../nbl_hw_leonis/nbl_resource_leonis.h | 1 +
.../nbl/nbl_include/nbl_def_hw.h | 3 +
.../nbl/nbl_include/nbl_def_resource.h | 3 +
.../nbl/nbl_include/nbl_include.h | 13 +
10 files changed, 848 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index bcedf278bab2..ef5b6ada70e5 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -9,4 +9,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
+ nbl_hw/nbl_chip.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
new file mode 100644
index 000000000000..bd175e200408
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
@@ -0,0 +1,26 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include "nbl_chip.h"
+
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
+{
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+
+ if (hw_ops->deinit_module)
+ hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
+}
+
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
+{
+ u8 eth_speed = res_mgt->resource_info->board_info.eth_speed;
+ u8 eth_num = res_mgt->resource_info->board_info.eth_num;
+ struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
+ struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
+
+ if (hw_ops->init_module)
+ return hw_ops->init_module(p, eth_speed, eth_num);
+ return -EOPNOTSUPP;
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
new file mode 100644
index 000000000000..d8ec8ea63d04
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_CHIP_H_
+#define _NBL_CHIP_H_
+
+#include "nbl_resource.h"
+int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt);
+void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
index 5e23b942b257..f36d400e6db0 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
@@ -102,6 +102,557 @@ static void nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt, u32 *eth_map)
*eth_map = FIELD_GET(NBL_FW_BOARD_DW6_ETH_BITMAP_MASK, data);
}
+static u32 nbl_hw_get_quirks(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quirks;
+
+ nbl_hw_read_mbx_regs(hw_mgt, NBL_LEONIS_QUIRKS_OFFSET, &quirks,
+ sizeof(u32));
+
+ if (quirks == ~0u)
+ return 0;
+
+ return quirks;
+}
+
+static void nbl_configure_dped_checksum(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 data;
+
+ /* DPED dped_l4_ck_cmd_40 for sctp */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ data |= FIELD_PREP(NBL_DPED_L4_CK_CMD_40_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, &data, sizeof(data));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_dped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_VLAN_OFFSET, 0xC);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_0, 0x8);
+ nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_1, 0x4);
+ spin_unlock(&hw_mgt->reg_lock);
+ /* dped checksum offload */
+ nbl_configure_dped_checksum(hw_mgt);
+}
+
+static void nbl_uped_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 hw_edit;
+
+ /* V4 TCP: l3_len = 0 */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit &= ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V4_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+
+ /* V6 TCP: l3_len = 1 */
+ nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ hw_edit = (hw_edit & ~NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK) |
+ FIELD_PREP(NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_UPED_V6_TCP_IDX),
+ &hw_edit, sizeof(hw_edit));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
+{
+ struct nbl_shaping_dvn_dport_u dvn_dport = { 0 };
+ struct nbl_shaping_dport_u dport = { 0 };
+ u32 rate, half_rate;
+ u32 depth;
+ u64 low_val, high_val;
+
+ switch (speed) {
+ case NBL_FW_PORT_SPEED_100G:
+ rate = 100000;
+ break;
+ case NBL_FW_PORT_SPEED_50G:
+ rate = 50000;
+ break;
+ case NBL_FW_PORT_SPEED_25G:
+ rate = 25000;
+ break;
+ case NBL_FW_PORT_SPEED_10G:
+ rate = 10000;
+ break;
+ default:
+ dev_err(hw_mgt->common->dev,
+ "Unsupported port speed %u for eth%u\n", speed, eth_id);
+ return -EINVAL;
+ }
+
+ half_rate = rate / 2;
+ depth = max_t(u32, rate * 2, NBL_LR_LEONIS_NET_BUCKET_DEPTH);
+
+ /* 1. clear valid first
+ * dport and dvn_dport are zero-initialised above, so VALID=0 already
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 2. write config words (valid=0, safe) */
+ low_val = FIELD_PREP(DPORT_CIR_MASK, rate) |
+ FIELD_PREP(DPORT_PIR_MASK, rate) |
+ FIELD_PREP(DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(DPORT_PBS_MASK, depth);
+ /* Fixed split, independent of host endian */
+ dport.data[0] = lower_32_bits(low_val);
+ dport.data[1] = upper_32_bits(low_val);
+ dport.data[2] = lower_32_bits(high_val);
+ dport.data[3] = upper_32_bits(high_val);
+
+ low_val = FIELD_PREP(DPORT_CIR_MASK, half_rate) |
+ FIELD_PREP(DPORT_PIR_MASK, rate) |
+ FIELD_PREP(DPORT_DEPTH_MASK, depth) |
+ FIELD_PREP(DPORT_CBS_MASK_LOW, depth & 0x3F);
+ high_val = FIELD_PREP(DPORT_CBS_MASK_HIGH, depth >> 6) |
+ FIELD_PREP(DPORT_PBS_MASK, depth);
+ dvn_dport.data[0] = lower_32_bits(low_val);
+ dvn_dport.data[1] = upper_32_bits(low_val);
+ dvn_dport.data[2] = lower_32_bits(high_val);
+ dvn_dport.data[3] = upper_32_bits(high_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+
+ /* 3. commit: set valid last */
+ low_val = FIELD_PREP(DPORT_VALID_MASK, 1);
+ dport.data[0] |= lower_32_bits(low_val);
+
+ low_val = FIELD_PREP(DPORT_VALID_MASK, 1);
+ dvn_dport.data[0] |= lower_32_bits(low_val);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), dport.data,
+ sizeof(dport));
+ nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
+ dvn_dport.data, sizeof(dvn_dport));
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static int nbl_shaping_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+#define NBL_SHAPING_FLUSH_INTERVAL 128
+ struct nbl_shaping_net_u net_shaping = { 0 };
+ u32 eth_bitmap = 0;
+ u32 psha_en = 0;
+ int ret;
+ int i;
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ ret = nbl_shaping_eth_init(hw_mgt, i, speed);
+ if (ret)
+ return ret;
+ }
+ psha_en = FIELD_PREP(NBL_DSCH_PSHA_EN_MASK, 0xF);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DSCH_PSHA_EN_ADDR, &psha_en,
+ sizeof(psha_en));
+
+ for (i = 0; i < NBL_MAX_FUNC; i++) {
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_SHAPING_NET_REG(i),
+ net_shaping.data,
+ sizeof(net_shaping));
+ if ((i + 1) % NBL_SHAPING_FLUSH_INTERVAL == 0)
+ nbl_flush_writes(hw_mgt);
+ }
+ nbl_flush_writes(hw_mgt);
+ return 0;
+}
+
+static void nbl_dsch_qid_max_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 quanta = 0;
+
+ quanta = FIELD_PREP(NBL_DSCH_VN_QUANTA_H_QUA_MASK, NBL_HOST_QUANTA) |
+ FIELD_PREP(NBL_DSCH_VN_QUANTA_E_QUA_MASK, NBL_ECPU_QUANTA);
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSCH_VN_QUANTA_ADDR, &quanta,
+ sizeof(quanta));
+ nbl_hw_wr32(hw_mgt, NBL_DSCH_HOST_QID_MAX, NBL_MAX_QUEUE_ID);
+
+ nbl_hw_wr32(hw_mgt, NBL_DVN_ECPU_QUEUE_NUM, 0);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_ECPU_QUEUE_NUM, 0);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th = 0;
+ u32 pkt_len;
+ int i;
+
+ if (eth_num != 1 && eth_num != 2 && eth_num != 4)
+ return -EINVAL;
+ /* Read current packet length config
+ *(to preserve other fields while updating 'min')
+ */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+ /* min arp packet length 42 (14 + 28) */
+ pkt_len &= ~NBL_USTORE_PKT_LEN_MIN_MASK;
+ pkt_len |= FIELD_PREP(NBL_USTORE_PKT_LEN_MIN_MASK, 42);
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, &pkt_len,
+ sizeof(pkt_len));
+
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_EN_MASK, 1);
+ if (eth_num == 1)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_SINGLE_ETH_DROP_TH);
+ else if (eth_num == 2)
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_DUAL_ETH_DROP_TH);
+ else
+ drop_th |= FIELD_PREP(NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK,
+ NBL_USTORE_QUAD_ETH_DROP_TH);
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PORT_DROP_TH_REG_ARR(i),
+ &drop_th, sizeof(drop_th));
+ }
+
+ /* Clear port drop/truncate counters by reading them
+ * (hardware has read-to-clear behavior for these registers)
+ */
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_DROP_PKT(i));
+ nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_TRUN_PKT(i));
+ }
+ spin_unlock(&hw_mgt->reg_lock);
+ return 0;
+}
+
+static void nbl_dstore_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 eth_bitmap = 0;
+ u32 drop_th;
+ u32 fc_th;
+ u32 bp_th;
+ int i;
+
+ for (i = 0; i < NBL_DSTORE_PORT_DROP_TH_DEPTH; i++) {
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ drop_th &= ~NBL_DSTORE_PORT_DROP_EN_MASK;
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i), &drop_th,
+ sizeof(drop_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ bp_th |= FIELD_PREP(NBL_DSTORE_DISC_BP_TH_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, &bp_th, sizeof(bp_th));
+ spin_unlock(&hw_mgt->reg_lock);
+
+ nbl_hw_get_fw_eth_map(hw_mgt, ð_bitmap);
+ for (i = 0; i < NBL_MAX_ETHERNET; i++) {
+ if (!(eth_bitmap & BIT(i)))
+ continue;
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ fc_th &= ~(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK |
+ NBL_DSTORE_D_DPORT_FC_XON_TH_MASK);
+ if (speed == NBL_FW_PORT_SPEED_100G) {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH_100G) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH_100G);
+ } else {
+ fc_th |=
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK,
+ NBL_DSTORE_DROP_XOFF_TH) |
+ FIELD_PREP(NBL_DSTORE_D_DPORT_FC_XON_TH_MASK,
+ NBL_DSTORE_DROP_XON_TH);
+ }
+
+ fc_th |= FIELD_PREP(NBL_DSTORE_D_DPORT_FC_FC_EN_MASK, 1);
+ nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i), &fc_th,
+ sizeof(fc_th));
+ spin_unlock(&hw_mgt->reg_lock);
+ }
+}
+
+static void nbl_dvn_descreq_num_cfg(struct nbl_hw_mgt *hw_mgt, u8 descreq_num)
+{
+ u8 split_ring_num = (descreq_num >> 3) & 0x1;
+ u8 ring_num = descreq_num & 0x7;
+ u32 num_cfg = 0;
+
+ num_cfg = FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK,
+ split_ring_num) |
+ FIELD_PREP(NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK,
+ ring_num);
+
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, &num_cfg,
+ sizeof(num_cfg));
+}
+
+static void nbl_dvn_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
+{
+ u32 timeout = 0;
+ u32 ro_flag = 0;
+
+ timeout = FIELD_PREP(NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK,
+ DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX);
+ nbl_hw_wr_regs_lock(hw_mgt, NBL_DVN_DESC_WR_MERGE_TIMEOUT, &timeout,
+ sizeof(timeout));
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_rd_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ if (pcie_relaxed_ordering_enabled(hw_mgt->common->pdev)) {
+ ro_flag |=
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1);
+ } else {
+ ro_flag &=
+ ~(FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK,
+ 1) |
+ FIELD_PREP(NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK,
+ 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, &ro_flag,
+ sizeof(ro_flag));
+ spin_unlock(&hw_mgt->reg_lock);
+ if (speed == NBL_FW_PORT_SPEED_100G)
+ nbl_dvn_descreq_num_cfg(hw_mgt,
+ DEFAULT_DVN_100G_DESCREQ_NUMCFG);
+ else
+ nbl_dvn_descreq_num_cfg(hw_mgt, DEFAULT_DVN_DESCREQ_NUMCFG);
+}
+
+static void nbl_uvn_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u16 wr_timeout = NBL_UVN_DESC_WR_TIMEOUT_VAL;
+ u32 timeout = NBL_UVN_DESC_RD_WAIT_TICKS;
+ u32 desc_wr_timeout = 0;
+ u32 prefetch_init = 0;
+ bool ro_enabled;
+ u32 flag = 0;
+ u32 mask = 0;
+ u32 quirks;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_UVN_DESC_RD_WAIT, timeout);
+ desc_wr_timeout =
+ FIELD_PREP(NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK, wr_timeout);
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT, &desc_wr_timeout,
+ sizeof(desc_wr_timeout));
+ ro_enabled = pcie_relaxed_ordering_enabled(hw_mgt->common->pdev);
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+ if (ro_enabled) {
+ flag |= FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1);
+ flag &= ~FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1);
+ } else {
+ flag &= ~(FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK, 1) |
+ FIELD_PREP(NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK, 1));
+ }
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, &flag, sizeof(flag));
+
+ nbl_hw_rd_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+ mask |= FIELD_PREP(NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK, 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, &mask, sizeof(mask));
+
+ quirks = nbl_hw_get_quirks(hw_mgt);
+ /*
+ * sel=0: use configured num; sel=1: use internal calc (max 32)
+ * Default is sel=1, unless NBL_QUIRKS_UVN_PREFETCH_ALIGN is set,
+ * in which case override to sel=0.
+ */
+ prefetch_init =
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK,
+ NBL_UVN_DESC_PREFETCH_NUM) |
+ FIELD_PREP(NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK,
+ (quirks & NBL_QUIRK_UVN_PREFETCH_ALIGN) ? 0 :
+ 1);
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT, &prefetch_init,
+ sizeof(prefetch_init));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_uqm_init(struct nbl_hw_mgt *hw_mgt)
+{
+ u32 que_type = 0;
+ u32 cnt = 0;
+ int i;
+
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_FWD_DROP_CNT, &cnt, sizeof(cnt));
+
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_SLICE_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_LEN_ADD_CNT, &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_HEAD_PNTR_ADD_CNT, &cnt,
+ sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_WEIGHT_ADD_CNT, &cnt, sizeof(cnt));
+
+ for (i = 0; i < NBL_UQM_PORT_DROP_DEPTH; i++) {
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_CNT + (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_SLICE_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT +
+ (sizeof(cnt) * i),
+ &cnt, sizeof(cnt));
+ }
+
+ for (i = 0; i < NBL_UQM_DPORT_DROP_DEPTH; i++)
+ nbl_hw_wr_regs(hw_mgt,
+ NBL_UQM_DPORT_DROP_CNT + (sizeof(cnt) * i), &cnt,
+ sizeof(cnt));
+ /* bit 0: bp mode , bit1: drop mode, resv bit1-31 */
+ nbl_hw_wr_regs(hw_mgt, NBL_UQM_QUE_TYPE, &que_type, sizeof(que_type));
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static int nbl_dp_init(struct nbl_hw_mgt *hw_mgt, u8 speed, u8 eth_num)
+{
+ int ret;
+
+ nbl_dped_init(hw_mgt);
+ nbl_uped_init(hw_mgt);
+ ret = nbl_shaping_init(hw_mgt, speed);
+ if (ret)
+ return ret;
+ nbl_dsch_qid_max_init(hw_mgt);
+ ret = nbl_ustore_init(hw_mgt, eth_num);
+ if (ret)
+ return ret;
+ nbl_dstore_init(hw_mgt, speed);
+ nbl_dvn_init(hw_mgt, speed);
+ nbl_uvn_init(hw_mgt);
+ nbl_uqm_init(hw_mgt);
+ return 0;
+}
+
+static void nbl_host_padpt_init(struct nbl_hw_mgt *hw_mgt)
+{
+ /* padpt flow control register */
+ spin_lock(&hw_mgt->reg_lock);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP,
+ NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PD_DN,
+ NBL_HOST_PADPT_CFG_FC_PD_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PH_DN,
+ NBL_HOST_PADPT_CFG_FC_PH_DN_VAL);
+ nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN,
+ NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+static void nbl_intf_init(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_host_padpt_init(hw_mgt);
+}
+
+static void nbl_hw_set_driver_status(struct nbl_hw_mgt *hw_mgt, bool active)
+{
+ u32 status;
+
+ spin_lock(&hw_mgt->reg_lock);
+ status = nbl_hw_rd32(hw_mgt, NBL_DRIVER_STATUS_REG);
+
+ status &= ~BIT(NBL_DRIVER_STATUS_BIT);
+ status |= FIELD_PREP(BIT(NBL_DRIVER_STATUS_BIT), active);
+
+ nbl_hw_wr32(hw_mgt, NBL_DRIVER_STATUS_REG, status);
+ spin_unlock(&hw_mgt->reg_lock);
+}
+
+/*
+ * This design is intentional. Setting driver status to false is the
+ * official teardown mechanism: it notifies firmware to perform full
+ * cleanup of all per-PF hardware state, including qinfo registers.
+ * An inverse helper would duplicate work that the firmware already
+ * does, and would add error-path complexity for no benefit. We keep
+ * the deinit path minimal and rely on firmware cleanup for correctness,
+ * including abnormal hardware reset scenarios.
+ */
+static void nbl_hw_deinit_module(struct nbl_hw_mgt *hw_mgt)
+{
+ nbl_hw_set_driver_status(hw_mgt, false);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+}
+
+/*
+ * Full chip hardware initialization is handled by firmware.
+ * This function only configures driver-level table entries and registers.
+ */
+static int nbl_hw_init_module(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
+ u8 eth_num)
+{
+ int ret;
+
+ ret = nbl_dp_init(hw_mgt, eth_speed, eth_num);
+ if (ret)
+ goto notify_fw;
+ nbl_intf_init(hw_mgt);
+ nbl_hw_set_driver_status(hw_mgt, true);
+ /* ensure registers written */
+ nbl_flush_writes(hw_mgt);
+
+ return 0;
+notify_fw:
+ /*
+ * On failure path we set driver_status=false directly.
+ * The FW checks this flag and cleans up partial state,
+ * so no explicit rollback is needed.
+ */
+ nbl_hw_set_driver_status(hw_mgt, false);
+ nbl_flush_writes(hw_mgt);
+ return ret;
+}
+
/*
* nbl_hw_set_mailbox_irq - read-modify-write of NBL_MAILBOX_QINFO_MAP_REG_ARR
*
@@ -337,6 +888,9 @@ static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
}
static struct nbl_hw_ops hw_ops = {
+ .init_module = nbl_hw_init_module,
+ .deinit_module = nbl_hw_deinit_module,
+
.cfg_msix_map = nbl_hw_cfg_msix_map,
.cfg_msix_info = nbl_hw_cfg_msix_info,
.flush_write = nbl_flush_writes,
@@ -381,7 +935,8 @@ static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
hw_ops_tbl = devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
if (!hw_ops_tbl)
return ERR_PTR(-ENOMEM);
- if (!hw_ops.cfg_msix_map || !hw_ops.cfg_msix_info ||
+ if (!hw_ops.init_module || !hw_ops.deinit_module ||
+ !hw_ops.cfg_msix_map || !hw_ops.cfg_msix_info ||
!hw_ops.flush_write || !hw_ops.update_mailbox_queue_tail_ptr ||
!hw_ops.config_mailbox_rxq || !hw_ops.config_mailbox_txq ||
!hw_ops.stop_mailbox_rxq || !hw_ops.stop_mailbox_txq ||
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
index 1d6d614be7d6..e096f2169180 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
@@ -11,6 +11,9 @@
#include "../../nbl_include/nbl_include.h"
#include "../nbl_hw_reg.h"
+#define NBL_DRIVER_STATUS_REG 0x1300444
+#define NBL_DRIVER_STATUS_BIT 16
+
/* ---------- REG BASE ADDR ---------- */
/* Interface modules base addr */
#define NBL_INTF_HOST_PCOMPLETER_BASE 0x00f08000
@@ -70,6 +73,17 @@ struct nbl_mailbox_qinfo_cfg_table {
#define NBL_PCIE_BUS_MASK GENMASK(12, 5)
/* -------- HOST_PADPT -------- */
+#define NBL_HOST_PADPT_HOST_CFG_FC_PD_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000160)
+#define NBL_HOST_PADPT_HOST_CFG_FC_PH_DN (NBL_INTF_HOST_PADPT_BASE + 0x00000164)
+#define NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN \
+ (NBL_INTF_HOST_PADPT_BASE + 0x0000016C)
+#define NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP \
+ (NBL_INTF_HOST_PADPT_BASE + 0x00000170)
+
+#define NBL_HOST_PADPT_CFG_FC_CPLH_UP_VAL 0x10400
+#define NBL_HOST_PADPT_CFG_FC_PD_DN_VAL 0x10080
+#define NBL_HOST_PADPT_CFG_FC_PH_DN_VAL 0x10010
+#define NBL_HOST_PADPT_CFG_FC_NPH_DN_VAL 0x10010
/* host_padpt host_msix_info */
#define NBL_PADPT_HOST_MSIX_INFO_REG_ARR(vector_id) \
(NBL_INTF_HOST_PADPT_BASE + 0x00010000 + \
@@ -110,6 +124,219 @@ struct nbl_function_msix_map {
u32 data[NBL_FUNC_MSIX_MAP_DWLEN];
};
+/* ---------- DPED ---------- */
+#define NBL_DPED_VLAN_OFFSET (NBL_DP_DPED_BASE + 0x000003F4)
+#define NBL_DPED_DSCP_OFFSET_0 (NBL_DP_DPED_BASE + 0x000003F8)
+#define NBL_DPED_DSCP_OFFSET_1 (NBL_DP_DPED_BASE + 0x000003FC)
+/* DPED hw_edt_prof/ UPED hw_edt_prof */
+
+#define NBL_DPED_L4_CK_CMD_40_ADDR 0x75c338
+#define NBL_DPED_L4_CK_CMD_40_DEPTH 1
+#define NBL_DPED_L4_CK_CMD_40_WIDTH 32
+#define NBL_DPED_L4_CK_CMD_40_DWLEN 1
+
+#define NBL_DPED_L4_CK_CMD_40_VALUE_MASK GENMASK(7, 0)
+#define NBL_DPED_L4_CK_CMD_40_LEN_IN_OFT_MASK GENMASK(14, 8)
+#define NBL_DPED_L4_CK_CMD_40_LEN_PHID_MASK GENMASK(16, 15)
+#define NBL_DPED_L4_CK_CMD_40_LEN_VLD_MASK BIT(17)
+#define NBL_DPED_L4_CK_CMD_40_DATA_VLD_MASK GENMASK(18, 18)
+#define NBL_DPED_L4_CK_CMD_40_IN_OFT_MASK GENMASK(25, 19)
+#define NBL_DPED_L4_CK_CMD_40_PHID_MASK GENMASK(27, 26)
+#define NBL_DPED_L4_CK_CMD_40_FLAG_MASK BIT(28)
+#define NBL_DPED_L4_CK_CMD_40_MODE_MASK BIT(29)
+#define NBL_DPED_L4_CK_CMD_40_RSV_MASK BIT(30)
+#define NBL_DPED_L4_CK_CMD_40_EN_MASK BIT(31)
+
+/* ---------- UPED ---------- */
+/* UPED uped_hw_edt_prof */
+#define NBL_UPED_HW_EDT_PROF_TABLE(i) \
+ (NBL_DP_UPED_BASE + 0x00001000 + (i) * sizeof(u32))
+#define NBL_UPED_V4_TCP_IDX 5
+#define NBL_UPED_V6_TCP_IDX 6
+#define NBL_PED_HW_EDIT_PROFILE_L3_LEN_MASK GENMASK(3, 2)
+
+/* ---------- DSCH ---------- */
+#define NBL_DSCH_PSHA_EN_MASK GENMASK(3, 0)
+/* DSCH dsch maxqid */
+#define NBL_DSCH_HOST_QID_MAX (NBL_DP_DSCH_BASE + 0x00000118)
+#define NBL_DSCH_VN_QUANTA_ADDR (NBL_DP_DSCH_BASE + 0x00000134)
+
+#define NBL_MAX_QUEUE_ID 0x7ff
+#define NBL_HOST_QUANTA 0x8000
+#define NBL_ECPU_QUANTA 0x1000
+
+#define NBL_DSCH_VN_QUANTA_H_QUA_MASK GENMASK(15, 0)
+#define NBL_DSCH_VN_QUANTA_E_QUA_MASK GENMASK(31, 16)
+
+/* ---------- DVN ---------- */
+/* DVN dvn_queue_table */
+#define NBL_DVN_ECPU_QUEUE_NUM (NBL_DP_DVN_BASE + 0x0000041C)
+#define NBL_DVN_DESCREQ_NUM_CFG (NBL_DP_DVN_BASE + 0x00000430)
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT (NBL_DP_DVN_BASE + 0x00000480)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG (NBL_DP_DVN_BASE + 0x0000045C)
+
+#define DEFAULT_DVN_DESCREQ_NUMCFG 0x03
+#define DEFAULT_DVN_100G_DESCREQ_NUMCFG 0x07
+
+#define DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX 0x3FF
+
+/* spilit ring descreq_num 0:8,1:16 */
+#define NBL_DVN_DESCREQ_NUM_CFG_AVRING_DESREQ_NUM_CFG_MASK BIT(0)
+/* packet ring descreq_num
+ * 0:8,1:12,2:16;3:20,4:24,5:26;6:32,7:32
+ */
+#define NBL_DVN_DESCREQ_NUM_CFG_PACKED_L1_NUM_MASK GENMASK(6, 4)
+
+#define NBL_DVN_DESC_WR_MERGE_TIMEOUT_CFG_CYCLE_MASK GENMASK(9, 0)
+
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DESC_RO_EN_MASK BIT(0)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_DATA_RO_EN_MASK BIT(1)
+#define NBL_DVN_DIF_REQ_RD_RO_FLAG_AVRING_RO_EN_MASK BIT(2)
+
+/* ---------- UVN ---------- */
+/* UVN uvn_queue_table */
+
+#define NBL_UVN_DESC_RD_WAIT (NBL_DP_UVN_BASE + 0x0000020C)
+#define NBL_UVN_QUEUE_ERR_MASK (NBL_DP_UVN_BASE + 0x00000224)
+#define NBL_UVN_ECPU_QUEUE_NUM (NBL_DP_UVN_BASE + 0x0000023C)
+#define NBL_UVN_DESC_WR_TIMEOUT (NBL_DP_UVN_BASE + 0x00000214)
+#define NBL_UVN_DIF_REQ_RO_FLAG (NBL_DP_UVN_BASE + 0x00000250)
+#define NBL_UVN_DESC_PREFETCH_INIT (NBL_DP_UVN_BASE + 0x00000204)
+#define NBL_UVN_DESC_PREFETCH_NUM 4
+
+#define NBL_UVN_DIF_REQ_RO_FLAG_AVAIL_RD_MASK BIT(0)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_RD_MASK BIT(1)
+#define NBL_UVN_DIF_REQ_RO_FLAG_PKT_WR_MASK BIT(2)
+#define NBL_UVN_DIF_REQ_RO_FLAG_DESC_WR_MASK BIT(3)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_NUM_MASK GENMASK(14, 0)
+#define NBL_UVN_DESC_WR_TIMEOUT_MASK_MASK BIT(15)
+
+#define NBL_UVN_QUEUE_ERR_MASK_DIF_ERR_MASK BIT(5)
+
+#define NBL_UVN_DESC_PREFETCH_INIT_NUM_MASK GENMASK(7, 0)
+#define NBL_UVN_DESC_PREFETCH_INIT_SEL_MASK BIT(16)
+
+#define NBL_UVN_DESC_WR_TIMEOUT_VAL 0x12c
+/* 200us = 200000ns / 1.67ns per tick = 119760 ticks */
+#define NBL_UVN_DESC_RD_WAIT_TICKS 119760
+
+/* -------- USTORE -------- */
+#define NBL_USTORE_PKT_LEN_ADDR (NBL_DP_USTORE_BASE + 0x00000108)
+#define NBL_USTORE_PORT_DROP_TH_REG_ARR(port_id) \
+ (NBL_DP_USTORE_BASE + 0x00000150 + (port_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_DROP_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002500 + (eth_id) * sizeof(u32))
+#define NBL_USTORE_BUF_PORT_TRUN_PKT(eth_id) \
+ (NBL_DP_USTORE_BASE + 0x00002540 + (eth_id) * sizeof(u32))
+
+#define NBL_USTORE_SINGLE_ETH_DROP_TH 0xC80
+#define NBL_USTORE_DUAL_ETH_DROP_TH 0x640
+#define NBL_USTORE_QUAD_ETH_DROP_TH 0x320
+
+/* USTORE pkt_len */
+#define NBL_USTORE_PKT_LEN_MIN_MASK GENMASK(6, 0)
+
+/* USTORE port_drop_th */
+#define NBL_USTORE_PORT_DROP_TH_DISC_TH_MASK GENMASK(11, 0)
+#define NBL_USTORE_PORT_DROP_TH_EN_MASK BIT(31)
+
+/* UQM*/
+#define NBL_UQM_QUE_TYPE (NBL_DP_UQM_BASE + 0x0000013c)
+#define NBL_UQM_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009C0)
+#define NBL_UQM_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009C4)
+#define NBL_UQM_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x000009C8)
+#define NBL_UQM_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x000009CC)
+#define NBL_UQM_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x000009D0)
+#define NBL_UQM_PORT_DROP_PKT_CNT (NBL_DP_UQM_BASE + 0x000009D4)
+#define NBL_UQM_PORT_DROP_PKT_SLICE_CNT (NBL_DP_UQM_BASE + 0x000009F4)
+#define NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A14)
+#define NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A34)
+#define NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT (NBL_DP_UQM_BASE + 0x00000A54)
+#define NBL_UQM_FWD_DROP_CNT (NBL_DP_UQM_BASE + 0x00000A80)
+#define NBL_UQM_DPORT_DROP_CNT (NBL_DP_UQM_BASE + 0x00000B74)
+
+#define NBL_UQM_PORT_DROP_DEPTH 6
+#define NBL_UQM_DPORT_DROP_DEPTH 16
+
+/* --------- SHAPING --------- */
+#define NBL_SHAPING_NET(i) \
+ (NBL_DP_SHAPING_BASE + 0x00001800 + \
+ (i) * sizeof(struct nbl_shaping_net))
+
+/* cir 1, bandwidth 1kB/s in protol environment */
+/* cir 1, bandwidth 1Mb/s */
+#define NBL_LR_LEONIS_NET_BUCKET_DEPTH 9600
+#define NBL_SHAPING_DPORT_ADDR 0x504700
+#define NBL_SHAPING_DPORT_DWLEN 4
+#define NBL_SHAPING_DPORT_REG(r) \
+ (NBL_SHAPING_DPORT_ADDR + (NBL_SHAPING_DPORT_DWLEN * 4) * (r))
+#define NBL_SHAPING_DVN_DPORT_ADDR 0x504750
+#define NBL_SHAPING_DVN_DPORT_DWLEN 4
+#define NBL_SHAPING_DVN_DPORT_REG(r) \
+ (NBL_SHAPING_DVN_DPORT_ADDR + (NBL_SHAPING_DVN_DPORT_DWLEN * 4) * (r))
+#define NBL_DSCH_PSHA_EN_ADDR 0x404314
+#define NBL_SHAPING_NET_ADDR 0x505800
+#define NBL_SHAPING_NET_DWLEN 4
+#define NBL_SHAPING_NET_REG(r) \
+ (NBL_SHAPING_NET_ADDR + (NBL_SHAPING_NET_DWLEN * 4) * (r))
+
+#define DPORT_VALID_MASK (0x1ULL << 0)
+#define DPORT_DEPTH_MASK (0x7FFFFULL << 1) // [19:1]
+#define DPORT_CIR_MASK (0x7FFFFULL << 20) // [38:20]
+#define DPORT_PIR_MASK (0x7FFFFULL << 39) // [57:39]
+#define DPORT_CBS_MASK_LOW (0x3FULL << 58) // [63:58]
+#define DPORT_CBS_MASK_HIGH (0x7FFFULL << (0)) // [78:64] -> high[14:0]
+#define DPORT_PBS_MASK (0x1FFFFFULL << (79 - 64)) // [99:79] -> high[35:15]
+
+/* SHAPING shaping_net */
+struct nbl_shaping_net_u {
+ u32 data[NBL_SHAPING_NET_DWLEN];
+};
+
+struct nbl_shaping_dport_u {
+ u32 data[NBL_SHAPING_DPORT_DWLEN];
+};
+
+struct nbl_shaping_dvn_dport_u {
+ u32 data[NBL_SHAPING_DVN_DPORT_DWLEN];
+};
+
+/* -------- DSTORE -------- */
+#define NBL_DSTORE_D_DPORT_FC_TH_ADDR 0x704600
+#define NBL_DSTORE_D_DPORT_FC_TH_DEPTH 5
+#define NBL_DSTORE_D_DPORT_FC_TH_WIDTH 32
+#define NBL_DSTORE_D_DPORT_FC_TH_DWLEN 1
+
+#define NBL_DSTORE_D_DPORT_FC_XOFF_TH_MASK GENMASK(10, 0)
+#define NBL_DSTORE_D_DPORT_FC_XON_TH_MASK GENMASK(26, 16)
+#define NBL_DSTORE_D_DPORT_FC_FC_EN_MASK BIT(31)
+
+#define NBL_DSTORE_D_DPORT_FC_TH_REG(r) \
+ (NBL_DSTORE_D_DPORT_FC_TH_ADDR + \
+ (NBL_DSTORE_D_DPORT_FC_TH_DWLEN * 4) * (r))
+#define NBL_DSTORE_PORT_DROP_TH_ADDR 0x704150
+#define NBL_DSTORE_PORT_DROP_TH_DEPTH 6
+#define NBL_DSTORE_PORT_DROP_TH_WIDTH 32
+#define NBL_DSTORE_PORT_DROP_TH_DWLEN 1
+
+#define NBL_DSTORE_PORT_DROP_DISC_TH_MASK GENMASK(9, 0)
+#define NBL_DSTORE_PORT_DROP_EN_MASK BIT(31)
+
+#define NBL_DSTORE_DROP_XOFF_TH 0xC8
+#define NBL_DSTORE_DROP_XON_TH 0x64
+
+#define NBL_DSTORE_DROP_XOFF_TH_100G 0x1F4
+#define NBL_DSTORE_DROP_XON_TH_100G 0x12C
+
+#define NBL_DSTORE_DISC_BP_TH (NBL_DP_DSTORE_BASE + 0x00000630)
+
+#define NBL_DSTORE_DISC_BP_TH_EN_MASK BIT(31)
+
+#define NBL_DSTORE_PORT_DROP_TH_REG(r) \
+ (NBL_DSTORE_PORT_DROP_TH_ADDR + \
+ (NBL_DSTORE_PORT_DROP_TH_DWLEN * 4) * (r))
+
#define NBL_FW_BOARD_CONFIG 0x200
#define NBL_FW_BOARD_DW3_OFFSET (NBL_FW_BOARD_CONFIG + 12)
#define NBL_FW_BOARD_DW6_OFFSET (NBL_FW_BOARD_CONFIG + 24)
@@ -123,4 +350,6 @@ struct nbl_function_msix_map {
#define NBL_FW_BOARD_DW6_LANE_BITMAP_MASK GENMASK(7, 0)
#define NBL_FW_BOARD_DW6_ETH_BITMAP_MASK GENMASK(15, 8)
+#define NBL_LEONIS_QUIRKS_OFFSET 0x00000140
+
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
index e192bd18380d..dadd74f054ee 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
@@ -13,6 +13,8 @@ static struct nbl_resource_ops res_ops = {
.cfg_msix_map = nbl_res_intr_cfg_msix_map,
.destroy_msix_map = nbl_res_intr_destroy_msix_map,
.set_mailbox_irq = nbl_res_intr_set_mailbox_irq,
+ .init_module = nbl_res_chip_init_module,
+ .deinit_module = nbl_res_chip_deinit_module,
};
static struct nbl_resource_mgt *
@@ -46,7 +48,8 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
return ERR_PTR(-ENOMEM);
if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
!res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
- !res_ops.set_mailbox_irq)
+ !res_ops.set_mailbox_irq || !res_ops.init_module ||
+ !res_ops.deinit_module)
return ERR_PTR(-EINVAL);
res_ops_tbl->ops = &res_ops;
res_ops_tbl->priv = res_mgt;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
index 1da2abcaf00f..f9a53b5d4627 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
@@ -8,4 +8,5 @@
#include "../nbl_resource.h"
#include "../nbl_interrupt.h"
+#include "../nbl_chip.h"
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
index d5d5985db5a9..31647d3abcbe 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
@@ -11,6 +11,9 @@
struct nbl_hw_mgt;
struct nbl_adapter;
struct nbl_hw_ops {
+ int (*init_module)(struct nbl_hw_mgt *hw_mgt, u8 eth_speed, u8 eth_num);
+ void (*deinit_module)(struct nbl_hw_mgt *hw_mgt);
+
void (*cfg_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
bool valid, dma_addr_t dma_addr, u8 bus,
u8 devid, u8 function);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
index 2317381b1e09..58be5262ec84 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
@@ -12,6 +12,9 @@ struct nbl_resource_mgt;
struct nbl_adapter;
struct nbl_resource_ops {
+ int (*init_module)(struct nbl_resource_mgt *res_mgt);
+ void (*deinit_module)(struct nbl_resource_mgt *res_mgt);
+
int (*cfg_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
u16 num_net_msix, u16 num_others_msix,
bool net_msix_mask_en);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index e0f51bb93a0b..3f35982cb02a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -30,4 +30,17 @@ struct nbl_init_param {
struct nbl_func_caps caps;
};
+/*
+ * Firmware ABI defines port speed enum fixed, value 0 represents 10G, cannot
+ * reassign 0 to INVALID for compatibility
+ */
+enum nbl_fw_port_speed {
+ NBL_FW_PORT_SPEED_10G,
+ NBL_FW_PORT_SPEED_25G,
+ NBL_FW_PORT_SPEED_50G,
+ NBL_FW_PORT_SPEED_100G,
+};
+
+#define NBL_QUIRK_UVN_PREFETCH_ALIGN BIT(1)
+
#endif
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (6 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 07/11] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-14 19:45 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 09/11] net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops mutex illusion.wang
` (2 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add base dispatch layer infrastructure for control-level routing:
1. Dispatch management & ops table structures allocation
2. X-macro op table template for uniform dispatch entry registration
3. Control PF / regular PF routing logic via ctrl_lvl bitmask
4. Local chip init/deinit dispatch wrappers (no channel dependency)
Document constraint: init_module/deinit_module only valid
on Control PF, caller must guard with has_ctrl to avoid NULL deref.
This patch only provides core routing skeleton, no channel message
handling or resource locking logic.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nbl/nbl_channel/nbl_channel.c | 4 +
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 112 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 24 ++++
.../nbl/nbl_include/nbl_def_channel.h | 32 +++++
.../nbl/nbl_include/nbl_def_dispatch.h | 42 +++++++
.../nbl/nbl_include/nbl_include.h | 2 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 8 ++
9 files changed, 227 insertions(+)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index ef5b6ada70e5..56464f576cbe 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -10,4 +10,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_resource.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
+ nbl_core/nbl_dispatch.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
index 3eb795199217..9e0f9b6c4c3a 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
@@ -1030,6 +1030,10 @@ int nbl_chan_init_common(struct nbl_adapter *adap)
struct nbl_channel_mgt *chan_mgt;
int ret;
+ BUILD_BUG_ON(sizeof(struct nbl_chan_param_cfg_msix_map) != 8);
+ BUILD_BUG_ON(sizeof(struct nbl_chan_param_set_mailbox_irq) != 4);
+ BUILD_BUG_ON(sizeof(struct nbl_chan_param_get_vsi_id) != 4);
+ BUILD_BUG_ON(sizeof(struct nbl_chan_param_get_eth_id) != 8);
chan_mgt = nbl_chan_setup_chan_mgt(adap);
if (IS_ERR(chan_mgt)) {
ret = PTR_ERR(chan_mgt);
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 6d7f000dde85..566d67130975 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -17,12 +17,14 @@ enum {
struct nbl_interface {
struct nbl_hw_ops_tbl *hw_ops_tbl;
struct nbl_resource_ops_tbl *resource_ops_tbl;
+ struct nbl_dispatch_ops_tbl *dispatch_ops_tbl;
struct nbl_channel_ops_tbl *channel_ops_tbl;
};
struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
+ struct nbl_dispatch_mgt *disp_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
new file mode 100644
index 000000000000..9358651cbf7e
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dispatch.h"
+
+static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->deinit_module)
+ res_ops->deinit_module(p);
+}
+
+static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+
+ if (res_ops->init_module)
+ return res_ops->init_module(p);
+ return -EOPNOTSUPP;
+}
+
+static void nbl_disp_setup_ctrl_lvl(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
+{
+ struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
+
+ set_bit(lvl, disp_mgt->ctrl_lvl);
+
+ if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
+ disp_ops->init_module = nbl_disp_init_module;
+ disp_ops->deinit_module = nbl_disp_deinit_module;
+ }
+}
+
+static struct nbl_dispatch_mgt *
+nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dispatch_mgt *disp_mgt;
+ struct device *dev = common->dev;
+
+ disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
+ if (!disp_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ disp_mgt->common = common;
+ return disp_mgt;
+}
+
+static struct nbl_dispatch_ops_tbl *
+nbl_disp_setup_ops(struct device *dev, struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_dispatch_ops *disp_ops;
+
+ disp_ops_tbl = devm_kzalloc(dev, sizeof(*disp_ops_tbl), GFP_KERNEL);
+ if (!disp_ops_tbl)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops = devm_kzalloc(dev, sizeof(*disp_ops), GFP_KERNEL);
+ if (!disp_ops)
+ return ERR_PTR(-ENOMEM);
+
+ disp_ops_tbl->ops = disp_ops;
+ disp_ops_tbl->priv = disp_mgt;
+
+ return disp_ops_tbl;
+}
+
+int nbl_disp_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_resource_ops_tbl *res_ops_tbl =
+ adapter->intf.resource_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct device *dev = &adapter->pdev->dev;
+ struct nbl_dispatch_mgt *disp_mgt;
+ int ret;
+
+ disp_mgt = nbl_disp_setup_disp_mgt(common);
+ if (IS_ERR(disp_mgt)) {
+ ret = PTR_ERR(disp_mgt);
+ return ret;
+ }
+
+ disp_ops_tbl = nbl_disp_setup_ops(dev, disp_mgt);
+ if (IS_ERR(disp_ops_tbl)) {
+ ret = PTR_ERR(disp_ops_tbl);
+ return ret;
+ }
+
+ disp_mgt->res_ops_tbl = res_ops_tbl;
+ disp_mgt->chan_ops_tbl = chan_ops_tbl;
+ disp_mgt->disp_ops_tbl = disp_ops_tbl;
+ adapter->core.disp_mgt = disp_mgt;
+ adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+
+ if (common->has_ctrl)
+ nbl_disp_setup_ctrl_lvl(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+
+ return 0;
+}
+
+void nbl_disp_remove(struct nbl_adapter *adapter)
+{
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
new file mode 100644
index 000000000000..f06b90075af4
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DISPATCH_H_
+#define _NBL_DISPATCH_H_
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+struct nbl_dispatch_mgt {
+ struct nbl_common_info *common;
+ struct nbl_resource_ops_tbl *res_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index b5a7b069d834..23bdd453ee73 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -13,6 +13,12 @@ struct nbl_adapter;
typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
+enum {
+ NBL_CHAN_RESP_OK = 0,
+ NBL_CHAN_RESP_ERR = 1,
+ NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+};
+
/*
* Mailbox wire opcodes
* Every opcode is assigned explicit fixed numeric value, stable wire ABI
@@ -252,6 +258,32 @@ enum nbl_chan_state {
NBL_CHAN_STATE_NBITS
};
+struct nbl_chan_param_cfg_msix_map {
+ __le16 num_net_msix;
+ __le16 num_others_msix;
+ __le16 msix_mask_en;
+ __le16 rsvd;
+};
+
+struct nbl_chan_param_set_mailbox_irq {
+ __le16 vector_id;
+ u8 en_msix;
+ u8 rsvd;
+};
+
+struct nbl_chan_param_get_vsi_id {
+ __le16 vsi_id;
+ __le16 type;
+};
+
+struct nbl_chan_param_get_eth_id {
+ __le16 vsi_id;
+ u8 eth_num;
+ u8 eth_id;
+ u8 logic_eth_id;
+ u8 rsvd[3];
+};
+
struct nbl_board_port_info {
u8 eth_num;
u8 eth_speed;
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
new file mode 100644
index 000000000000..b26425e0112b
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DISPATCH_H_
+#define _NBL_DEF_DISPATCH_H_
+
+#include <linux/types.h>
+
+struct nbl_dispatch_mgt;
+struct nbl_adapter;
+enum {
+ NBL_DISP_CTRL_LVL_NEVER = 0,
+ NBL_DISP_CTRL_LVL_MGT,
+ NBL_DISP_CTRL_LVL_NET,
+ NBL_DISP_CTRL_LVL_MAX,
+};
+
+struct nbl_dispatch_ops {
+ int (*init_module)(struct nbl_dispatch_mgt *disp_mgt);
+ void (*deinit_module)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*cfg_msix_map)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en);
+ int (*destroy_msix_map)(struct nbl_dispatch_mgt *disp_mgt);
+ int (*set_mailbox_irq)(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix);
+ int (*get_vsi_id)(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id);
+ int (*get_eth_id)(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id);
+};
+
+struct nbl_dispatch_ops_tbl {
+ struct nbl_dispatch_ops *ops;
+ struct nbl_dispatch_mgt *priv;
+};
+
+int nbl_disp_init(struct nbl_adapter *adapter);
+void nbl_disp_remove(struct nbl_adapter *adapter);
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 3f35982cb02a..5f33a5de908d 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -15,6 +15,8 @@
#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
+/* Used for macros to pass checkpatch */
+#define NBL_NAME(x) x
enum {
NBL_VSI_DATA = 0,
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 3e3de6dabb10..6bd74c933c78 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -11,6 +11,7 @@
#include "nbl_include/nbl_def_channel.h"
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
+#include "nbl_include/nbl_def_dispatch.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -48,7 +49,13 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_res_init_leonis(adapter);
if (ret)
goto res_init_fail;
+
+ ret = nbl_disp_init(adapter);
+ if (ret)
+ goto disp_init_fail;
return adapter;
+disp_init_fail:
+ nbl_res_remove_leonis(adapter);
res_init_fail:
nbl_chan_remove_common(adapter);
chan_init_fail:
@@ -59,6 +66,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
nbl_hw_remove_leonis(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 09/11] net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops mutex
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (7 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation illusion.wang
10 siblings, 0 replies; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Implement bidirectional channel request/response RPC handlers for 5 dispatch
resource operations: get_vsi_id, get_eth_id, configure_msix_map,
destroy_msix_map, set_mailbox_irq.
- Extend X-macro NBL_DISP_OPS_TBL to pair local dispatch entry points
with cross-PF mailbox request/response callbacks.
- Add registration helper nbl_disp_setup_msg() to auto-register channel
response handlers for ops with valid msg_type.
- Introduce NBL_CHAN_RESP_UNIMPLEMENTED dedicated error code for missing
resource ops implementation on remote PF.
- Add dev_err logging when sending RPC ACK back to remote PF fails.
The existing init_module/deinit_module dispatch entries use msg_type = -1,
so they do not register any mailbox channel handlers and are excluded from
RPC logic.
- Separate local channel transmission errors and remote RPC wire response codes:
The channel layer returns ACK header err via nbl_chan_send_msg() when wait_ack=1.
Translate wire protocol defined remote codes NBL_CHAN_RESP_* to standard kernel
errno at dispatch request wrapper to avoid semantic confusion between wire
handshake values and system error numbers exposed to upper layers.
- Collapse all fine-grained resource operation sub-errors into generic
NBL_CHAN_RESP_ERR on wire to keep message payload layout unchanged and
maintain backward compatibility with legacy firmware.
- Fix unimplemented ops check order in response handler to prevent NULL pointer
invocation of resource ops.
Add input sanitization for mailbox response handlers:
The driver maintains cross-version interoperability; older peers may send
truncated payloads. Existing min_t() + zero-init local param logic absorbs
partial messages and zero-fills missing fields. To mitigate risk:
1. Allow truncated payloads (0 < data_len < sizeof(param)) for backward
compatibility, continue zero-filling missing fields.
2. Reject data_len == 0 messages entirely, avoid invoking resource ops
with all-zero initialized parameters.
nbl_disp_chan_destroy_msix_map_resp carries no input payload and skips
payload length validation.
Add ops_mutex_lock to serialize concurrent hardware-modifying dispatch
resource operations, preventing race conditions between PF local calls
and remote mailbox message handlers that manipulate MSI-X mapping and
mailbox IRQ state.
1. Introduce disp_mgt->ops_mutex_lock, initialized via devm_mutex_init
at disp_mgt allocation time; symmetrically destroyed automatically
by devres on device detach, eliminating double mutex_destroy risk.
2. Add NBL_OPS_CALL_LOCK_RET macro to wrap hardware-modifying ops with
exclusive lock protection for unified locking semantics.
3. Wrap configure_msix_map / destroy_msix_map / set_mailbox_irq with
ops_mutex_lock; these ops mutate shared MSI-X and IRQ hardware state
and can race between local PF control paths and cross-PF mailbox RPCs.
The read-only get_vsi_id / get_eth_id routines only consume static
init-time metadata with no concurrent writers, so they require no locking.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 521 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 2 +
.../nbl/nbl_include/nbl_def_channel.h | 1 +
.../nbl/nbl_include/nbl_include.h | 1 +
4 files changed, 525 insertions(+)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
index 9358651cbf7e..3fef901c8e25 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
@@ -3,9 +3,162 @@
* Copyright (c) 2025 Nebula Matrix Limited.
*/
#include <linux/device.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include "nbl_dispatch.h"
+static int nbl_disp_chan_get_vsi_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 type, u16 *vsi_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_send_info chan_send;
+ int ret;
+
+ param.type = cpu_to_le16(type);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_VSI_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_OK:
+ break;
+ default:
+ return ret;
+ }
+ *vsi_id = le16_to_cpu(result.vsi_id);
+ return 0;
+}
+
+static void nbl_disp_chan_get_vsi_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_get_vsi_id result = { 0 };
+ struct nbl_chan_param_get_vsi_id param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ u16 vsi_id = 0;
+ int copy_len;
+ int ret;
+
+ if (data_len == 0) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ copy_len = min_t(size_t, data_len, sizeof(param));
+ memcpy(¶m, data, copy_len);
+
+ if (res_ops->get_vsi_id) {
+ ret = res_ops->get_vsi_id(p, src_id, le16_to_cpu(param.type),
+ &vsi_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ result.vsi_id = cpu_to_le16(vsi_id);
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_VSI_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_VSI_ID);
+}
+
+static int nbl_disp_chan_get_eth_id_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vsi_id, u8 *eth_num, u8 *eth_id,
+ u8 *logic_eth_id)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct nbl_chan_send_info chan_send;
+ int ret;
+
+ param.vsi_id = cpu_to_le16(vsi_id);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_GET_ETH_ID,
+ ¶m, sizeof(param), &result,
+ sizeof(result), 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_OK:
+ break;
+ default:
+ return ret;
+ }
+ *eth_num = result.eth_num;
+ *eth_id = result.eth_id;
+ *logic_eth_id = result.logic_eth_id;
+
+ return 0;
+}
+
+static void nbl_disp_chan_get_eth_id_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_get_eth_id result = { 0 };
+ struct nbl_chan_param_get_eth_id param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int copy_len;
+ int ret;
+
+ if (data_len == 0) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ copy_len = min_t(size_t, data_len, sizeof(param));
+ memcpy(¶m, data, copy_len);
+
+ if (res_ops->get_eth_id) {
+ ret = res_ops->get_eth_id(p, src_id, le16_to_cpu(param.vsi_id),
+ &result.eth_num, &result.eth_id,
+ &result.logic_eth_id);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_GET_ETH_ID, msg_id, err,
+ &result, sizeof(result));
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_GET_ETH_ID);
+}
+
static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
{
struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
@@ -25,6 +178,341 @@ static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
return -EOPNOTSUPP;
}
+static int nbl_disp_cfg_msix_map(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->cfg_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, common->mgt_pf, num_net_msix,
+ num_others_msix, net_msix_mask_en);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int
+nbl_disp_chan_cfg_msix_map_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 num_net_msix, u16 num_others_msix,
+ bool net_msix_mask_en)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_send_info chan_send;
+ int ret;
+
+ param.num_net_msix = cpu_to_le16(num_net_msix);
+ param.num_others_msix = cpu_to_le16(num_others_msix);
+ param.msix_mask_en = cpu_to_le16(!!net_msix_mask_en);
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ ¶m, sizeof(param),
+ NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_OK:
+ return ret;
+ default:
+ return ret;
+ }
+}
+
+static void nbl_disp_chan_cfg_msix_map_resp(void *priv, u16 src_id, u16 msg_id,
+ void *data, u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_param_cfg_msix_map param = { 0 };
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int copy_len;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (data_len == 0) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ copy_len = min_t(size_t, data_len, sizeof(param));
+ memcpy(¶m, data, copy_len);
+
+ if (res_ops->cfg_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->cfg_msix_map(p, src_id,
+ le16_to_cpu(param.num_net_msix),
+ le16_to_cpu(param.num_others_msix),
+ !!le16_to_cpu(param.msix_mask_en));
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_CONFIGURE_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP);
+}
+
+static int nbl_disp_chan_destroy_msix_map_req(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send;
+ int ret;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ NULL, 0, NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_PERM_DENY:
+ return -EPERM;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_OK:
+ return ret;
+ default:
+ return ret;
+ }
+}
+
+static void nbl_disp_chan_destroy_msix_map_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ int ret;
+
+ if (src_id > NBL_MAX_PF_SRC_ID) {
+ err = NBL_CHAN_RESP_PERM_DENY;
+ goto ack_out;
+ }
+ if (res_ops->destroy_msix_map) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, src_id);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_DESTROY_MSIX_MAP, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_DESTROY_MSIX_MAP);
+}
+
+static int nbl_disp_chan_set_mailbox_irq_req(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct nbl_common_info *common = disp_mgt->common;
+ struct nbl_chan_send_info chan_send;
+ int ret;
+
+ param.vector_id = cpu_to_le16(vector_id);
+ param.en_msix = !!en_msix;
+
+ nbl_chan_fill_send_info(&chan_send, common->mgt_pf,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ ¶m, sizeof(param), NULL, 0, 1);
+ ret = chan_ops->send_msg(disp_mgt->chan_ops_tbl->priv, &chan_send);
+ switch (ret) {
+ case NBL_CHAN_RESP_UNIMPLEMENTED:
+ return -EOPNOTSUPP;
+ case NBL_CHAN_RESP_ERR:
+ return -EREMOTEIO;
+ case NBL_CHAN_RESP_OK:
+ return ret;
+ default:
+ return ret;
+ }
+}
+
+static void nbl_disp_chan_set_mailbox_irq_resp(void *priv, u16 src_id,
+ u16 msg_id, void *data,
+ u32 data_len)
+{
+ struct nbl_dispatch_mgt *disp_mgt = (struct nbl_dispatch_mgt *)priv;
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_chan_param_set_mailbox_irq param = { 0 };
+ struct device *dev = disp_mgt->common->dev;
+ struct nbl_chan_ack_info chan_ack;
+ int err = NBL_CHAN_RESP_OK;
+ bool en_msix;
+ u16 vector_id;
+ int copy_len;
+ int ret;
+
+ if (data_len == 0) {
+ err = NBL_CHAN_RESP_ERR;
+ goto ack_out;
+ }
+ copy_len = min_t(size_t, data_len, sizeof(param));
+ memcpy(¶m, data, copy_len);
+ vector_id = le16_to_cpu(param.vector_id);
+ en_msix = !!param.en_msix;
+
+ if (res_ops->set_mailbox_irq) {
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, src_id, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ if (ret)
+ err = NBL_CHAN_RESP_ERR;
+ } else {
+ err = NBL_CHAN_RESP_UNIMPLEMENTED;
+ }
+
+ack_out:
+ nbl_chan_fill_ack_info(&chan_ack, src_id,
+ NBL_CHAN_MSG_MAILBOX_SET_IRQ, msg_id,
+ err, NULL, 0);
+ ret = chan_ops->send_ack(disp_mgt->chan_ops_tbl->priv, &chan_ack);
+ if (ret)
+ dev_err(dev,
+ "channel send ack failed with ret: %d, msg_type: %d\n",
+ ret, NBL_CHAN_MSG_MAILBOX_SET_IRQ);
+}
+
+static int nbl_disp_destroy_msix_map(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->destroy_msix_map)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->destroy_msix_map(p, common->mgt_pf);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_set_mailbox_irq(struct nbl_dispatch_mgt *disp_mgt,
+ u16 vector_id, bool en_msix)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+ int ret;
+
+ if (!res_ops->set_mailbox_irq)
+ return -EOPNOTSUPP;
+ mutex_lock(&disp_mgt->ops_mutex_lock);
+ ret = res_ops->set_mailbox_irq(p, common->mgt_pf, vector_id, en_msix);
+ mutex_unlock(&disp_mgt->ops_mutex_lock);
+ return ret;
+}
+
+static int nbl_disp_get_vsi_id(struct nbl_dispatch_mgt *disp_mgt, u16 type,
+ u16 *vsi_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_vsi_id)
+ return res_ops->get_vsi_id(p, common->mgt_pf, type, vsi_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_get_eth_id(struct nbl_dispatch_mgt *disp_mgt, u16 vsi_id,
+ u8 *eth_num, u8 *eth_id, u8 *logic_eth_id)
+{
+ struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
+ struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
+ struct nbl_common_info *common = disp_mgt->common;
+
+ if (res_ops->get_eth_id)
+ return res_ops->get_eth_id(p, common->mgt_pf, vsi_id,
+ eth_num, eth_id, logic_eth_id);
+ return -EOPNOTSUPP;
+}
+
+static int nbl_disp_setup_msg(struct nbl_dispatch_mgt *disp_mgt)
+{
+ struct nbl_channel_ops *chan_ops = disp_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *p = disp_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+ int _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
+ nbl_disp_chan_cfg_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_DESTROY_MSIX_MAP,
+ nbl_disp_chan_destroy_msix_map_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_MAILBOX_SET_IRQ,
+ nbl_disp_chan_set_mailbox_irq_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_VSI_ID,
+ nbl_disp_chan_get_vsi_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ _ret = chan_ops->register_msg(p, NBL_CHAN_MSG_GET_ETH_ID,
+ nbl_disp_chan_get_eth_id_resp,
+ disp_mgt);
+ if (_ret < 0 && !ret)
+ ret = _ret;
+
+ if (ret)
+ chan_ops->unregister_all_msg(p);
+ return ret;
+}
+
static void nbl_disp_setup_ctrl_lvl(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
{
struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
@@ -34,6 +522,18 @@ static void nbl_disp_setup_ctrl_lvl(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
disp_ops->init_module = nbl_disp_init_module;
disp_ops->deinit_module = nbl_disp_deinit_module;
+ disp_ops->cfg_msix_map = nbl_disp_cfg_msix_map;
+ disp_ops->destroy_msix_map = nbl_disp_destroy_msix_map;
+ disp_ops->set_mailbox_irq = nbl_disp_set_mailbox_irq;
+ disp_ops->get_vsi_id = nbl_disp_get_vsi_id;
+ disp_ops->get_eth_id = nbl_disp_get_eth_id;
+ } else {
+ disp_ops->cfg_msix_map =
+ nbl_disp_chan_cfg_msix_map_req;
+ disp_ops->destroy_msix_map = nbl_disp_chan_destroy_msix_map_req;
+ disp_ops->set_mailbox_irq = nbl_disp_chan_set_mailbox_irq_req;
+ disp_ops->get_vsi_id = nbl_disp_chan_get_vsi_id_req;
+ disp_ops->get_eth_id = nbl_disp_chan_get_eth_id_req;
}
}
@@ -42,12 +542,16 @@ nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
{
struct nbl_dispatch_mgt *disp_mgt;
struct device *dev = common->dev;
+ int err;
disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
if (!disp_mgt)
return ERR_PTR(-ENOMEM);
disp_mgt->common = common;
+ err = devm_mutex_init(common->dev, &disp_mgt->ops_mutex_lock);
+ if (err)
+ return ERR_PTR(err);
return disp_mgt;
}
@@ -101,12 +605,29 @@ int nbl_disp_init(struct nbl_adapter *adapter)
adapter->core.disp_mgt = disp_mgt;
adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
+ ret = nbl_disp_setup_msg(disp_mgt);
+ if (ret)
+ return ret;
+
if (common->has_ctrl)
nbl_disp_setup_ctrl_lvl(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
+ /*
+ * For non-control PF with network capability, enable net control
+ * level.
+ * All dispatch ops declared with NBL_DISP_CTRL_LVL_MGT fall back
+ * to remote mailbox msg_req handlers when MGT bit is not set.
+ */
+ if (common->has_net)
+ nbl_disp_setup_ctrl_lvl(disp_mgt, NBL_DISP_CTRL_LVL_NET);
+
return 0;
}
void nbl_disp_remove(struct nbl_adapter *adapter)
{
+ /*
+ * All message handlers will be cleaned up inside channel layer
+ * nbl_chan_remove_common() at final device tear-down
+ */
}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
index f06b90075af4..644094c9dd8e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
@@ -19,6 +19,8 @@ struct nbl_dispatch_mgt {
struct nbl_channel_ops_tbl *chan_ops_tbl;
struct nbl_dispatch_ops_tbl *disp_ops_tbl;
DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
+ /* use for the caller not in interrupt */
+ struct mutex ops_mutex_lock;
};
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
index 23bdd453ee73..4db33a6e0151 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
@@ -17,6 +17,7 @@ enum {
NBL_CHAN_RESP_OK = 0,
NBL_CHAN_RESP_ERR = 1,
NBL_CHAN_RESP_UNIMPLEMENTED = 2,
+ NBL_CHAN_RESP_PERM_DENY = 3,
};
/*
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
index 5f33a5de908d..aa61051c8034 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
@@ -15,6 +15,7 @@
#define NBL_MAX_FUNC 520
#define NBL_MAX_ETHERNET 4
+#define NBL_MAX_PF_SRC_ID 3
/* Used for macros to pass checkpatch */
#define NBL_NAME(x) x
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (8 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 09/11] net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops mutex illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-14 19:48 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation illusion.wang
10 siblings, 1 reply; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add nbl_dev_setup_common_dev() and nbl_dev_setup_ctrl_dev() initialization
helpers with paired teardown routines, and hook them into nbl_dev_init /
nbl_dev_remove.
Mailbox message processing is implemented via work_struct. This task
registration framework is reused for interrupt-driven receive now, and
can be shared with polling receive mode introduced in follow-up patches.
Extend channel send logic with polling ACK fallback path. During early
device initialization before interrupts are enabled, synchronous mailbox
requests rely on polling to wait for firmware ACK, avoiding silent
success and uninitialized variable reads when NBL_CHAN_INTERRUPT_READY
is not yet set.
Chip core hardware initialization is fully handled by firmware during
power-on; the driver only configures functional table entries and
registers after hardware is ready. Thus invoking
nbl_dev_setup_common_dev() before ctrl dev setup does not trigger
hardware faults, all register accesses are safe.
Teardown order explanation:
1. Remove ctrl dev first: notify firmware to clean all per-PF hardware
state including qinfo registers via driver status flag.
2. Then tear down common device queue resources.
Firmware cleanup ensures qinfo registers are valid during subsequent
common dev deinit writes, avoiding PCIe master abort or hardware access
panics.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
.../nbl/nbl_channel/nbl_channel.c | 72 +++++-
.../nbl/nbl_channel/nbl_channel.h | 3 +
.../nebula-matrix/nbl/nbl_common/nbl_common.c | 21 ++
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 1 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 240 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_core/nbl_dev.h | 57 +++++
.../nbl/nbl_include/nbl_def_common.h | 3 +
.../nbl/nbl_include/nbl_def_dev.h | 14 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 9 +
10 files changed, 420 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
index 56464f576cbe..0febda209865 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
+++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
@@ -11,4 +11,5 @@ nbl-objs += nbl_common/nbl_common.o \
nbl_hw/nbl_interrupt.o \
nbl_hw/nbl_chip.o \
nbl_core/nbl_dispatch.o \
+ nbl_core/nbl_dev.o \
nbl_main.o
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
index 9e0f9b6c4c3a..3b7f3876a075 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
@@ -636,10 +636,13 @@ static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
struct nbl_chan_info *chan_info)
{
+ struct nbl_common_info *common = chan_mgt->common;
struct nbl_chan_ring *rxq = &chan_info->rxq;
struct device *dev = chan_mgt->common->dev;
struct nbl_chan_rx_desc *rx_desc;
struct nbl_chan_buf *rx_buf;
+ struct work_struct *task;
+ bool more_work = false;
u16 next_to_clean;
u32 budget = 64;
u16 flags;
@@ -666,11 +669,22 @@ static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
next_to_clean = 0;
rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
- if (--budget == 0)
+ if (--budget == 0) {
+ more_work = true;
break;
+ }
cond_resched();
}
rxq->next_to_clean = next_to_clean;
+
+ /* Prevent queue_work after teardown clears clean_task */
+ if (READ_ONCE(chan_info->shutdown))
+ return;
+ if (common->wq && more_work) {
+ task = READ_ONCE(chan_info->clean_task);
+ if (task)
+ queue_work(common->wq, task);
+ }
}
static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
@@ -728,10 +742,12 @@ static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
struct nbl_common_info *common = chan_mgt->common;
struct nbl_chan_waitqueue_head *wait_head;
struct nbl_chan_tx_param tx_param = { 0 };
+ int i = NBL_CHAN_TX_WAIT_ACK_TIMES;
struct nbl_chan_info *chan_info =
chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
struct device *dev = common->dev;
bool inflight_inc = false;
+ struct work_struct *task;
u16 msgid = 0;
int ret;
u16 tmp;
@@ -869,6 +885,60 @@ static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
WRITE_ONCE(wait_head->ack_data_len, 0);
spin_unlock_irq(&wait_head->status_lock);
goto inflight_dec_out;
+ } else {
+ /* Polling wait path for synchronous ACK */
+ while (i--) {
+ if (READ_ONCE(chan_info->shutdown)) {
+ ret = -ESHUTDOWN;
+ /*
+ * Shutdown happens during polling wait,
+ * reclaim slot
+ */
+ spin_lock_irq(&wait_head->status_lock);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_TIMEOUT);
+ spin_unlock_irq(&wait_head->status_lock);
+ goto inflight_dec_out;
+ }
+
+ task = READ_ONCE(chan_info->clean_task);
+ if (common->wq && task &&
+ !READ_ONCE(chan_info->shutdown))
+ queue_work(common->wq, task);
+
+ spin_lock_irq(&wait_head->status_lock);
+ if (READ_ONCE(wait_head->acked)) {
+ chan_send->ack_len = wait_head->ack_data_len;
+ ret = wait_head->ack_err;
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->status,
+ NBL_MBX_STATUS_IDLE);
+ spin_unlock_irq(&wait_head->status_lock);
+ goto inflight_dec_out;
+ }
+ spin_unlock_irq(&wait_head->status_lock);
+
+ usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
+ NBL_CHAN_TX_WAIT_ACK_US_MAX);
+ cond_resched();
+ }
+
+ /* Polling loop exhausted, reclaim slot */
+ spin_lock_irq(&wait_head->status_lock);
+ WRITE_ONCE(wait_head->acked, 0);
+ WRITE_ONCE(wait_head->ack_data, NULL);
+ WRITE_ONCE(wait_head->ack_data_len, 0);
+ WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
+ spin_unlock_irq(&wait_head->status_lock);
+
+ dev_err_ratelimited(dev,
+ "Channel polling ack failed, message type: %d msg id: %u\n",
+ chan_send->msg_type, msgid);
+ ret = -ETIMEDOUT;
+ goto inflight_dec_out;
}
inflight_dec_out:
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
index 92db39e1b05e..d59e660be9c7 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
@@ -24,6 +24,9 @@
#define NBL_CHAN_TX_WAIT_US 100
#define NBL_CHAN_TX_WAIT_US_MAX 120
#define NBL_CHAN_TX_WAIT_TIMES 100
+#define NBL_CHAN_TX_WAIT_ACK_US_MIN 1000
+#define NBL_CHAN_TX_WAIT_ACK_US_MAX 1200
+#define NBL_CHAN_TX_WAIT_ACK_TIMES 5000
#define NBL_CHAN_QUEUE_LEN 256
#define NBL_CHAN_BUF_LEN 4096
#define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
index 149681dd216d..f10ef78ee7f6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
@@ -7,6 +7,27 @@
#include <linux/jhash.h>
#include "nbl_common.h"
+void nbl_common_destroy_wq(struct nbl_common_info *common)
+{
+ destroy_workqueue(common->wq);
+}
+
+int nbl_common_create_wq(struct nbl_common_info *common)
+{
+ char wq_name[32];
+
+ snprintf(wq_name, sizeof(wq_name), "nbl_wq_%s", pci_name(common->pdev));
+ common->wq = alloc_workqueue("%s", WQ_UNBOUND, 0, wq_name);
+ if (!common->wq) {
+ pr_err("Failed to create workqueue nbl_wq\n");
+ goto alloc_wq_failed;
+ }
+
+ return 0;
+alloc_wq_failed:
+ return -ENOMEM;
+}
+
/*
* nbl_common_func_id_to_rel_pf_id - convert absolute PF id to relative PF id
*
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 566d67130975..9519aacbf0ea 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -25,6 +25,7 @@ struct nbl_core {
struct nbl_hw_mgt *hw_mgt;
struct nbl_resource_mgt *res_mgt;
struct nbl_dispatch_mgt *disp_mgt;
+ struct nbl_dev_mgt *dev_mgt;
struct nbl_channel_mgt *chan_mgt;
};
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
new file mode 100644
index 000000000000..bdb4d6a45fbc
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -0,0 +1,240 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+#include <linux/device.h>
+#include <linux/pci.h>
+#include "nbl_dev.h"
+
+static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
+}
+
+/* ---------- Channel config ---------- */
+static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ if (!chan_ops->check_queue_exist(priv, chan_type))
+ return;
+
+ chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
+}
+
+static int nbl_dev_setup_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->setup_queue(priv, chan_type);
+
+ return ret;
+}
+
+static int nbl_dev_remove_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
+ int ret = 0;
+
+ if (chan_ops->check_queue_exist(priv, chan_type))
+ ret = chan_ops->teardown_queue(priv, chan_type);
+
+ return ret;
+}
+
+static void nbl_dev_register_chan_task(struct nbl_dev_mgt *dev_mgt,
+ u8 chan_type, struct work_struct *task)
+{
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ if (chan_ops->check_queue_exist(dev_mgt->chan_ops_tbl->priv, chan_type))
+ chan_ops->register_chan_task(dev_mgt->chan_ops_tbl->priv,
+ chan_type, task);
+}
+
+/* ---------- Tasks config ---------- */
+static void nbl_dev_clean_mailbox_task(struct work_struct *work)
+{
+ struct nbl_dev_common *common_dev =
+ container_of(work, struct nbl_dev_common, clean_mbx_task);
+ struct nbl_dev_mgt *dev_mgt = common_dev->dev_mgt;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+
+ chan_ops->clean_queue_subtask(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_TYPE_MAILBOX);
+}
+
+/* ---------- Dev init process ---------- */
+static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dispatch_mgt *priv = dev_mgt->disp_ops_tbl->priv;
+ struct nbl_common_info *common = dev_mgt->common;
+ struct nbl_dev_common *common_dev;
+ int ret;
+
+ common_dev = devm_kzalloc(&adapter->pdev->dev, sizeof(*common_dev),
+ GFP_KERNEL);
+ if (!common_dev)
+ return -ENOMEM;
+ common_dev->dev_mgt = dev_mgt;
+
+ ret = nbl_dev_setup_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
+ &common_dev->clean_mbx_task);
+ /*
+ * Even if has_ctrl=false (no dedicated control PF channel), we fetch
+ * VSI/ETH info via regular mailbox message instead of
+ * dedicated control command.
+ */
+ ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
+ if (ret)
+ goto err_cleanup;
+ ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
+ &common->eth_id, &common->logic_eth_id);
+ if (ret)
+ goto err_cleanup;
+
+ dev_mgt->common_dev = common_dev;
+ nbl_dev_init_msix_cnt(dev_mgt);
+
+ return 0;
+err_cleanup:
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ return ret;
+}
+
+static void nbl_dev_remove_common_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+
+ if (!common_dev)
+ return;
+ nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
+}
+
+static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ int ret;
+
+ ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
+ if (ret)
+ return ret;
+
+ nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
+
+ return 0;
+}
+
+/*
+ * This is intentional. The qinfo registers are managed by the chip
+ * firmware, not by the driver. Setting driver status to false is the
+ * designed teardown mechanism — it notifies the firmware, which then
+ * performs its own cleanup of all per-PF state including the qinfo
+ * registers.
+ * An inverse helper would duplicate work that the firmware already
+ * does, and would add error-path complexity for no benefit. We keep
+ * the deinit path minimal and rely on the firmware cleanup for
+ * correctness, including in abnormal reset scenarios.
+ */
+static void nbl_dev_remove_ctrl_dev(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ disp_ops->deinit_module(dev_mgt->disp_ops_tbl->priv);
+}
+
+static struct nbl_dev_mgt *nbl_dev_setup_dev_mgt(struct nbl_common_info *common)
+{
+ struct nbl_dev_mgt *dev_mgt;
+
+ dev_mgt = devm_kzalloc(common->dev, sizeof(*dev_mgt), GFP_KERNEL);
+ if (!dev_mgt)
+ return ERR_PTR(-ENOMEM);
+
+ dev_mgt->common = common;
+ return dev_mgt;
+}
+
+int nbl_dev_init(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl =
+ adapter->intf.dispatch_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl =
+ adapter->intf.channel_ops_tbl;
+ struct nbl_dev_mgt *dev_mgt;
+ int ret;
+
+ dev_mgt = nbl_dev_setup_dev_mgt(common);
+ if (IS_ERR(dev_mgt)) {
+ ret = PTR_ERR(dev_mgt);
+ return ret;
+ }
+
+ dev_mgt->disp_ops_tbl = disp_ops_tbl;
+ dev_mgt->chan_ops_tbl = chan_ops_tbl;
+ adapter->core.dev_mgt = dev_mgt;
+ ret = nbl_common_create_wq(common);
+ if (ret)
+ return ret;
+ /*
+ * Chip hardware initialization is completed by firmware at power-up.
+ * Only driver functional table/register config follows here, safe to
+ * access hardware registers before ctrl dev setup.
+ */
+ ret = nbl_dev_setup_common_dev(adapter);
+ if (ret)
+ goto setup_err;
+
+ if (common->has_ctrl) {
+ ret = nbl_dev_setup_ctrl_dev(adapter);
+ if (ret)
+ goto setup_ctrl_dev_fail;
+ }
+
+ return 0;
+setup_ctrl_dev_fail:
+ nbl_dev_remove_common_dev(adapter);
+setup_err:
+ nbl_common_destroy_wq(common);
+ return ret;
+}
+
+/*
+ * Teardown order: ctrl dev first, then common dev.
+ * nbl_dev_remove_ctrl_dev() notifies firmware to clean all per-PF state
+ * (including qinfo registers), so subsequent common dev queue cleanup
+ * will not trigger PCIe master abort or invalid register access.
+ */
+void nbl_dev_remove(struct nbl_adapter *adapter)
+{
+ struct nbl_common_info *common = &adapter->common;
+
+ if (common->has_ctrl)
+ nbl_dev_remove_ctrl_dev(adapter);
+ nbl_dev_remove_common_dev(adapter);
+ nbl_common_destroy_wq(common);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
new file mode 100644
index 000000000000..7666ab9a73f3
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEV_H_
+#define _NBL_DEV_H_
+
+#include <linux/types.h>
+
+#include "../nbl_include/nbl_include.h"
+#include "../nbl_include/nbl_def_channel.h"
+#include "../nbl_include/nbl_def_hw.h"
+#include "../nbl_include/nbl_def_resource.h"
+#include "../nbl_include/nbl_def_dispatch.h"
+#include "../nbl_include/nbl_def_dev.h"
+#include "../nbl_include/nbl_def_common.h"
+#include "../nbl_core.h"
+
+#define NBL_STRING_NAME_LEN 32
+
+enum nbl_msix_serv_type {
+ /* virtio_dev has a config vector_id, and the vector_id need is 0 */
+ NBL_MSIX_VIRTIO_TYPE = 0,
+ NBL_MSIX_NET_TYPE,
+ NBL_MSIX_MAILBOX_TYPE,
+ NBL_MSIX_TYPE_MAX
+};
+
+struct nbl_msix_serv_info {
+ char irq_name[NBL_STRING_NAME_LEN];
+ u16 num;
+ u16 base_vector_id;
+ /* true: hw report msix, hw need to mask actively */
+ bool hw_self_mask_en;
+};
+
+struct nbl_msix_info {
+ struct nbl_msix_serv_info serv_info[NBL_MSIX_TYPE_MAX];
+};
+
+struct nbl_dev_common {
+ struct nbl_dev_mgt *dev_mgt;
+ struct nbl_msix_info msix_info;
+ char mailbox_name[NBL_STRING_NAME_LEN];
+ /* for ctrl-dev/net-dev mailbox recv msg */
+ struct work_struct clean_mbx_task;
+};
+
+struct nbl_dev_mgt {
+ struct nbl_common_info *common;
+ struct nbl_dispatch_ops_tbl *disp_ops_tbl;
+ struct nbl_channel_ops_tbl *chan_ops_tbl;
+ struct nbl_dev_common *common_dev;
+};
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
index af7e7225a879..75882bfe4c93 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
@@ -28,6 +28,7 @@ static inline u32 nbl_vsi_id_gap(u32 m)
}
struct nbl_common_info {
+ struct workqueue_struct *wq;
struct pci_dev *pdev;
struct device *dev;
u32 msg_enable;
@@ -54,6 +55,8 @@ struct nbl_hash_tbl_key {
u16 resv;
};
+void nbl_common_destroy_wq(struct nbl_common_info *common);
+int nbl_common_create_wq(struct nbl_common_info *common);
int nbl_common_func_id_to_rel_pf_id(struct nbl_common_info *common, u32 pf_id,
u32 *rel_pf_id);
struct nbl_hash_tbl_mgt *
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
new file mode 100644
index 000000000000..b422a4edf0a9
--- /dev/null
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Nebula Matrix Limited.
+ */
+
+#ifndef _NBL_DEF_DEV_H_
+#define _NBL_DEF_DEV_H_
+
+struct nbl_adapter;
+
+int nbl_dev_init(struct nbl_adapter *adapter);
+void nbl_dev_remove(struct nbl_adapter *adapter);
+
+#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 6bd74c933c78..8a968a06b5b6 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -12,6 +12,7 @@
#include "nbl_include/nbl_def_hw.h"
#include "nbl_include/nbl_def_resource.h"
#include "nbl_include/nbl_def_dispatch.h"
+#include "nbl_include/nbl_def_dev.h"
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
@@ -53,7 +54,14 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
ret = nbl_disp_init(adapter);
if (ret)
goto disp_init_fail;
+
+ ret = nbl_dev_init(adapter);
+ if (ret)
+ goto dev_init_fail;
return adapter;
+
+dev_init_fail:
+ nbl_disp_remove(adapter);
disp_init_fail:
nbl_res_remove_leonis(adapter);
res_init_fail:
@@ -66,6 +74,7 @@ struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
void nbl_core_remove(struct nbl_adapter *adapter)
{
+ nbl_dev_remove(adapter);
nbl_disp_remove(adapter);
nbl_res_remove_leonis(adapter);
nbl_chan_remove_common(adapter);
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
` (9 preceding siblings ...)
2026-08-07 4:44 ` [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-08-07 4:44 ` illusion.wang
2026-08-14 19:48 ` Jakub Kicinski
10 siblings, 1 reply; 17+ messages in thread
From: illusion.wang @ 2026-08-07 4:44 UTC (permalink / raw)
To: dimon.zhao, illusion.wang, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, horms, linux-doc, pabeni,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, open list
From: illusion wang <illusion.wang@nebula-matrix.com>
Add device start/stop helper functions to manage MSI-X vector mapping,
mailbox interrupt resource initialization and deinitialization.
Create unbound global workqueue for mailbox task scheduling; this shared
work infrastructure supports both current interrupt-driven receive and
future polling receive path.
This patch implements complete startup and shutdown flow for
common device resources:
1. Add nbl_dev_start() to finish device startup procedure:
- Configure hardware MSI-X mapping table for different interrupt types
- Allocate required MSI-X irq vectors via pci_alloc_irq_vectors
- Request mailbox IRQ; interrupt handler schedules mailbox processing work
- Enable hardware mailbox interrupt and mark channel interrupt ready
2. Add nbl_dev_stop() to tear down device resources safely in strict order:
- Update software channel state first to switch to polling, then mask
hardware interrupt to avoid stale ACK handling
- Free mailbox IRQ handler and release MSI-X vector resources
- Destroy hardware MSI-X mapping table
Extend channel TX waiting logic with polling fallback and shutdown detection.
When interrupt path is disabled, send routines rely on active polling to
wait for message ACK, preventing deadlock during device stop.
Hook nbl_core_start() / nbl_core_stop() into PCI probe and remove paths.
Signed-off-by: illusion wang <illusion.wang@nebula-matrix.com>
---
.../net/ethernet/nebula-matrix/nbl/nbl_core.h | 2 +
.../nebula-matrix/nbl/nbl_core/nbl_dev.c | 259 ++++++++++++++++++
.../nebula-matrix/nbl/nbl_hw/nbl_chip.c | 7 +-
.../nbl/nbl_include/nbl_def_dev.h | 2 +
.../net/ethernet/nebula-matrix/nbl/nbl_main.c | 21 +-
5 files changed, 285 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
index 9519aacbf0ea..a48bd9a0ef1e 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
@@ -39,5 +39,7 @@ struct nbl_adapter {
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param);
void nbl_core_remove(struct nbl_adapter *adapter);
+int nbl_core_start(struct nbl_adapter *adapter);
+void nbl_core_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
index bdb4d6a45fbc..3a8b7ee2e475 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
@@ -6,6 +6,17 @@
#include <linux/pci.h>
#include "nbl_dev.h"
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt);
+
+/* ---------- Interrupt config ---------- */
+static irqreturn_t nbl_dev_clean_mailbox(int __always_unused irq, void *data)
+{
+ struct nbl_dev_mgt *dev_mgt = (struct nbl_dev_mgt *)data;
+
+ nbl_dev_clean_mailbox_schedule(dev_mgt);
+ return IRQ_HANDLED;
+}
+
static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
{
struct nbl_dev_common *dev_common = dev_mgt->common_dev;
@@ -14,6 +25,187 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
}
+static int nbl_dev_request_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+ int err;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num < 0) {
+ dev_err(common->dev, "Failed to get mailbox IRQ vector: %d\n",
+ irq_num);
+ return irq_num;
+ }
+
+ snprintf(dev_common->mailbox_name, sizeof(dev_common->mailbox_name),
+ "nbl_mailbox@pci:%s", pci_name(common->pdev));
+ err = request_irq(irq_num, nbl_dev_clean_mailbox, 0,
+ dev_common->mailbox_name, dev_mgt);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static void nbl_dev_free_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ u16 local_vec_id;
+ int irq_num;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ irq_num = pci_irq_vector(common->pdev, local_vec_id);
+ if (irq_num >= 0)
+ free_irq(irq_num, dev_mgt);
+}
+
+static int nbl_dev_enable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_INTERRUPT_READY,
+ NBL_CHAN_TYPE_MAILBOX, true);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, true);
+}
+
+static int nbl_dev_disable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ u16 local_vec_id;
+
+ if (!msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num)
+ return 0;
+
+ local_vec_id =
+ msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].base_vector_id;
+ /*
+ * Disable sequence invariant: update software state first, then mask
+ * hardware interrupt. Must not reverse the order.
+ *
+ * If hardware interrupt is masked before clearing INTERRUPT_READY,
+ * the hardware may still transmit outstanding ACK packets for in-flight
+ * messages. Subsequent switch to polling mode discards pending ACK
+ * processing, triggering "Channel waiting ack failed" and "Skip ack
+ * with invalid status" errors.
+ *
+ * By entering polling mode first, any late hardware interrupts are
+ * ignored without pending ACK expectations, then hardware interrupt
+ * can be safely disabled.
+ *
+ * This helper is invoked in two paths:
+ * 1. Error unwind path of nbl_dev_start(): immediately followed by
+ * nbl_dev_free_mailbox_irq(), channel resources
+ * are fully torn down afterwards, no stale descriptors remain.
+ * 2. Normal device stop path nbl_dev_stop(): free_irq() synchronously
+ * waits for any in-flight threaded irq bottom-half handler to
+ * finish execution, all pending mailbox cleanup work completes
+ * before channel teardown in late remove stage. No stuck
+ * descriptors will linger in either scenario.
+ */
+ chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
+ NBL_CHAN_INTERRUPT_READY,
+ NBL_CHAN_TYPE_MAILBOX, false);
+
+ return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
+ local_vec_id, false);
+}
+
+static int nbl_dev_cfg_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ bool mask_en = msix_info->serv_info[NBL_MSIX_NET_TYPE].hw_self_mask_en;
+ u16 msix_net_num = msix_info->serv_info[NBL_MSIX_NET_TYPE].num;
+ u16 msix_not_net_num = 0;
+ int err, i;
+
+ msix_info->serv_info[NBL_MSIX_VIRTIO_TYPE].base_vector_id = 0;
+ for (i = NBL_MSIX_NET_TYPE; i < NBL_MSIX_TYPE_MAX; i++)
+ msix_info->serv_info[i].base_vector_id =
+ msix_info->serv_info[i - 1].base_vector_id +
+ msix_info->serv_info[i - 1].num;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++) {
+ if (i == NBL_MSIX_NET_TYPE)
+ continue;
+ msix_not_net_num += msix_info->serv_info[i].num;
+ }
+
+ err = disp_ops->cfg_msix_map(dev_mgt->disp_ops_tbl->priv,
+ msix_net_num, msix_not_net_num,
+ mask_en);
+
+ return err;
+}
+
+static int nbl_dev_destroy_msix_map(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
+
+ return disp_ops->destroy_msix_map(dev_mgt->disp_ops_tbl->priv);
+}
+
+static int nbl_dev_init_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *dev_common = dev_mgt->common_dev;
+ struct nbl_msix_info *msix_info = &dev_common->msix_info;
+ struct nbl_common_info *common = dev_mgt->common;
+ int needed = 0;
+ int err;
+ int i;
+
+ for (i = 0; i < NBL_MSIX_TYPE_MAX; i++)
+ needed += msix_info->serv_info[i].num;
+
+ err = pci_alloc_irq_vectors(common->pdev, needed, needed,
+ PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
+ if (err < 0) {
+ dev_err(common->dev,
+ "pci_alloc_irq_vectors failed, err = %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static void nbl_dev_clear_interrupt_scheme(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_common_info *common = dev_mgt->common;
+
+ pci_free_irq_vectors(common->pdev);
+}
+
/* ---------- Channel config ---------- */
static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
{
@@ -73,6 +265,14 @@ static void nbl_dev_clean_mailbox_task(struct work_struct *work)
NBL_CHAN_TYPE_MAILBOX);
}
+static void nbl_dev_clean_mailbox_schedule(struct nbl_dev_mgt *dev_mgt)
+{
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ struct nbl_common_info *common = dev_mgt->common;
+
+ queue_work(common->wq, &common_dev->clean_mbx_task);
+}
+
/* ---------- Dev init process ---------- */
static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
{
@@ -238,3 +438,62 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
nbl_dev_remove_common_dev(adapter);
nbl_common_destroy_wq(common);
}
+
+/* ---------- Dev start process ---------- */
+int nbl_dev_start(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int cleanup_ret;
+ int ret;
+
+ ret = nbl_dev_cfg_msix_map(dev_mgt);
+ if (ret)
+ return ret;
+
+ ret = nbl_dev_init_interrupt_scheme(dev_mgt);
+ if (ret)
+ goto init_interrupt_scheme_err;
+ ret = nbl_dev_request_mailbox_irq(dev_mgt);
+ if (ret)
+ goto mailbox_request_irq_err;
+ ret = nbl_dev_enable_mailbox_irq(dev_mgt);
+ if (ret)
+ goto enable_mailbox_irq_err;
+
+ return 0;
+enable_mailbox_irq_err:
+ cleanup_ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to disable mailbox IRQ: %d\n", cleanup_ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+ cancel_work_sync(&common_dev->clean_mbx_task);
+mailbox_request_irq_err:
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+init_interrupt_scheme_err:
+ cleanup_ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (cleanup_ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to destroy MSI-X map: %d\n", cleanup_ret);
+ return ret;
+}
+
+void nbl_dev_stop(struct nbl_adapter *adapter)
+{
+ struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
+ struct nbl_dev_common *common_dev = dev_mgt->common_dev;
+ int ret;
+
+ ret = nbl_dev_disable_mailbox_irq(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to disable mailbox IRQ: %d\n", ret);
+ nbl_dev_free_mailbox_irq(dev_mgt);
+ cancel_work_sync(&common_dev->clean_mbx_task);
+ nbl_dev_clear_interrupt_scheme(dev_mgt);
+ ret = nbl_dev_destroy_msix_map(dev_mgt);
+ if (ret)
+ dev_err(dev_mgt->common->dev,
+ "Failed to destroy MSI-X map: %d\n", ret);
+}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
index bd175e200408..c1e6180a7010 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
@@ -9,8 +9,7 @@ void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
{
struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
- if (hw_ops->deinit_module)
- hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
+ hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
}
int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
@@ -20,7 +19,5 @@ int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
- if (hw_ops->init_module)
- return hw_ops->init_module(p, eth_speed, eth_num);
- return -EOPNOTSUPP;
+ return hw_ops->init_module(p, eth_speed, eth_num);
}
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
index b422a4edf0a9..32e6cce38d39 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
@@ -10,5 +10,7 @@ struct nbl_adapter;
int nbl_dev_init(struct nbl_adapter *adapter);
void nbl_dev_remove(struct nbl_adapter *adapter);
+int nbl_dev_start(struct nbl_adapter *adapter);
+void nbl_dev_stop(struct nbl_adapter *adapter);
#endif
diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
index 8a968a06b5b6..c12a53da6a9b 100644
--- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
+++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
@@ -16,6 +16,16 @@
#include "nbl_include/nbl_def_common.h"
#include "nbl_core.h"
+int nbl_core_start(struct nbl_adapter *adapter)
+{
+ return nbl_dev_start(adapter);
+}
+
+void nbl_core_stop(struct nbl_adapter *adapter)
+{
+ nbl_dev_stop(adapter);
+}
+
struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
struct nbl_init_param *param)
{
@@ -131,7 +141,14 @@ static int nbl_probe(struct pci_dev *pdev,
goto adapter_init_err;
}
pci_set_drvdata(pdev, adapter);
+ err = nbl_core_start(adapter);
+ if (err)
+ goto core_start_err;
+
return 0;
+core_start_err:
+ pci_set_drvdata(pdev, NULL);
+ nbl_core_remove(adapter);
adapter_init_err:
pci_clear_master(pdev);
configure_dma_err:
@@ -142,8 +159,10 @@ static void nbl_remove(struct pci_dev *pdev)
{
struct nbl_adapter *adapter = pci_get_drvdata(pdev);
- if (adapter)
+ if (adapter) {
+ nbl_core_stop(adapter);
nbl_core_remove(adapter);
+ }
pci_clear_master(pdev);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization
2026-08-07 4:44 ` [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
@ 2026-08-14 19:33 ` Jakub Kicinski
0 siblings, 0 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-08-14 19:33 UTC (permalink / raw)
To: illusion.wang
Cc: dimon.zhao, alvin.wang, sam.chen, netdev, andrew+netdev, corbet,
horms, linux-doc, pabeni, vadim.fedorenko, lukas.bulwahn,
edumazet, enelsonmoore, skhan, hkallweit1, open list
On Fri, 7 Aug 2026 12:44:06 +0800 illusion.wang wrote:
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
Documentation says:
dma_set_mask_and_coherent() never return fail when DMA_BIT_MASK(64)
Please, don't error check this :/
> + if (err) {
> + dev_dbg(dev, "Configure DMA 64 bit mask failed, err = %d\n",
> + err);
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> + if (err) {
> + dev_err(dev,
> + "Configure DMA 32 bit mask failed, err = %d\n",
> + err);
> + goto configure_dma_err;
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer
2026-08-07 4:44 ` [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer illusion.wang
@ 2026-08-14 19:38 ` Jakub Kicinski
0 siblings, 0 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-08-14 19:38 UTC (permalink / raw)
To: illusion.wang
Cc: Jakub Kicinski, dimon.zhao, alvin.wang, sam.chen, netdev,
andrew+netdev, corbet, horms, linux-doc, pabeni, vadim.fedorenko,
lukas.bulwahn, edumazet, enelsonmoore, skhan, hkallweit1,
linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
net/nebula-matrix: add channel layer
This adds a mailbox based channel layer for the nbl driver, with message
registration through a hash table, synchronous send-with-ack and
fire-and-forget send, and TX/RX ring setup and teardown backed by devm
and dmam allocations. A leonis hardware operations table is added for
mailbox queue configuration and doorbell updates.
A few questions on the locking, barriers and teardown ordering below.
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> new file mode 100644
> index 000000000000..3eb795199217
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -0,0 +1,1068 @@
[ ... ]
> +static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> + struct nbl_chan_waitqueue_head *wait_head;
> + unsigned long wait_start, wait_timeout;
> + int ret = 0;
> + u16 i;
> +
> + /* Step 1: Mark shutdown flag, reject all new send requests */
> + WRITE_ONCE(chan_info->shutdown, true);
[ ... ]
> + /* Stop hardware queues */
> + nbl_chan_stop_queue(chan_mgt);
> +
> + /*
> + * Step 2: Wait all in-flight send_msg threads exit FIRST.
> + * Guarantee no further queue_work() can be triggered during cleanup.
> + * Add timeout to avoid permanent blocking.
> + */
> + wait_start = jiffies;
> + wait_timeout = msecs_to_jiffies(5000);
> + while (atomic_read(&chan_info->inflight_tx_cnt) != 0) {
[Severity: Low]
The comment says "Step 2: Wait all in-flight send_msg threads exit FIRST",
but nbl_chan_stop_queue() runs before the inflight_tx_cnt drain loop. Is
that ordering intended?
nbl_chan_stop_queue() -> nbl_hw_stop_mailbox_txq()/rxq() asserts
NBL_MAILBOX_QINFO_CFG_QUEUE_RST_MASK and clears QUEUE_EN, so a sender that
already passed the early shutdown check can still publish a descriptor and
ring the doorbell into a queue that is held in reset:
nbl_chan_send_msg()
nbl_chan_update_txqueue() /* no shutdown check at all */
tx_desc->flags = cpu_to_le16(BIT(NBL_CHAN_TX_DESC_AVAIL));
txq->tail_ptr++;
nbl_chan_kick_tx_ring()
nbl_chan_update_tail_ptr(...) /* doorbell, before the shutdown re-check */
Would draining inflight_tx_cnt before nbl_chan_stop_queue() avoid the driver
tail pointer diverging from the device consumer pointer?
> + if (time_after(jiffies, wait_start + wait_timeout)) {
> + dev_warn(chan_mgt->common->dev,
> + "teardown: wait inflight_tx_cnt timeout, force continue\n");
> + ret = -ETIMEDOUT;
> + break;
> + }
> + cpu_relax();
> + usleep_range(1000, 2000);
> + }
> +
> + /* Step3: All send paths drained, safely cancel cleanup work */
> + if (chan_info->clean_task) {
> + cancel_work_sync(chan_info->clean_task);
> + chan_info->clean_task = NULL;
> + }
> + return ret;
> +}
> +
> +static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_common_info *common = chan_mgt->common;
> + struct nbl_chan_ring *rxq = &chan_info->rxq;
> + int err;
> +
> + if (READ_ONCE(chan_info->init_done))
> + return 0;
[Severity: Low]
Are .setup_queue and .teardown_queue meant to be a symmetric pair?
nbl_chan_teardown_queue() leaves chan_info->init_done true and
chan_info->shutdown true, and does not reset txq/rxq next_to_use,
next_to_clean or tail_ptr. With the init_done early return above, a
setup_queue() after a teardown_queue() returns 0 without re-running
nbl_chan_config_queue() (the only code that deasserts QUEUE_RST and sets
QUEUE_EN), without re-ringing the RX doorbell, and without clearing
shutdown, since only nbl_chan_init_queue_param() does that and it sits
behind the guard:
if (READ_ONCE(chan_info->init_done))
return 0;
...
nbl_chan_init_queue_param(chan_info, ...); /* clears shutdown */
Today both ops are only reached from probe and remove with a freshly
devm_kzalloc'ed chan_info, so nothing hits this. Should teardown clear
init_done and the ring indices so a later stop/restart cycle (device reset,
PF down/up) does not silently succeed on a channel whose queues are still in
reset?
[ ... ]
> +static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info,
> + struct nbl_chan_tx_param *param)
> +{
[ ... ]
> + tx_desc->dstid = cpu_to_le16(param->dstid);
> + tx_desc->msg_type = cpu_to_le16(param->msg_type);
> + tx_desc->msgid = cpu_to_le16(param->msgid);
[Severity: Low]
Who writes tx_desc->srcid? This is the only producer of the descriptor and
it sets dstid, msg_type, msgid, the data/buf fields and flags, but never
assigns or clears srcid, while both receive paths consume it:
nbl_chan_recv_msg()
srcid = le16_to_cpu(READ_ONCE(tx_desc->srcid));
...
msg_handler->func(msg_handler->priv, srcid, msgid, payload, payload_len);
nbl_chan_recv_ack_msg()
if (srcid != READ_ONCE(wait_head->dstid)) {
... "ACK srcid=%u != dstid=%u, rejecting"
If the mailbox engine populates srcid in the copy delivered to the peer,
could a comment say so? Otherwise the field is whatever the reused ring
slot held.
[ ... ]
> +static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_send_info *chan_send)
> +{
[ ... ]
> + atomic_inc(&chan_info->inflight_tx_cnt);
> + inflight_inc = true;
> + /*
> + * Ensure reading chan_info->shutdown happens
> + * strictly after inflight_tx_cnt increment above on this CPU.
> + * Prevent compiler/cpu reordering: read shutdown before incrementing
> + * inflight counter, which would cause missing shutdown check and leak
> + * tx inflight count.
> + */
> + smp_rmb();
> + if (READ_ONCE(chan_info->shutdown)) {
> + atomic_dec(&chan_info->inflight_tx_cnt);
> + return -ESHUTDOWN;
> + }
[Severity: Medium]
Does smp_rmb() give the ordering the comment claims here? This is a store
followed by a load, and smp_rmb() only orders load-load pairs; the
non-value-returning atomic_inc() provides no ordering of its own.
The peer half in nbl_chan_teardown_queue() uses a full barrier and so
depends on the sender mirroring it:
CPU0 nbl_chan_send_msg() CPU1 nbl_chan_teardown_queue()
atomic_inc(&inflight_tx_cnt); WRITE_ONCE(shutdown, true);
smp_rmb(); smp_mb();
load shutdown -> false atomic_read(&inflight_tx_cnt) -> 0
Can the shutdown load be satisfied before the increment is visible, so that
teardown concludes the drain, calls nbl_chan_stop_queue() and
cancel_work_sync() and returns while CPU0 goes on to take txq_lock, write a
descriptor and ring the doorbell? Since teardown_queue() is the last thing
guarding the rings before devres releases the dmam_alloc_coherent()
descriptor rings and the devm_kcalloc() wait array, would smp_mb() (or
atomic_fetch_inc()) be the right primitive here?
> +
> + mutex_lock(&chan_info->txq_lock);
> + if (test_bit(NBL_CHAN_ABNORMAL, chan_info->state)) {
> + ret = -EIO;
> + goto unlock_out;
> + }
[Severity: Low]
Is anything expected to set NBL_CHAN_ABNORMAL? This test_bit() is its only
user in the tree; the only writer would be nbl_chan_set_queue_state(), and
its callers pass NBL_CHAN_INTERRUPT_READY, so this branch cannot be taken.
Related, the patch defines NBL_MAILBOX_QINFO_CFG_DIF_ERR_MASK and
NBL_MAILBOX_QINFO_CFG_PTR_ERR_MASK in nbl_hw_leonis.h, but the qinfo cfg
table is only ever written (nbl_hw_write_mbx_regs has no read-back
counterpart), so those status bits are never sampled. With no reader, a
queue the hardware has flagged as errored is never detected, and
nbl_chan_kick_tx_ring() just logs and advances next_to_clean:
dev_err_ratelimited(dev, "chan send msg type: %d timeout\n", msg_type);
txq->next_to_clean = txq->next_to_use;
return -ETIMEDOUT;
Is the error-bit sampling and queue reconfigure planned for a follow-up?
[ ... ]
> + ret = nbl_chan_kick_tx_ring(chan_mgt, chan_info);
> + if (ret) {
[ ... ]
> + goto unlock_out;
> + }
> + mutex_unlock(&chan_info->txq_lock);
> + if (!chan_send->ack) {
> + tmp = FIELD_GET(NBL_CHAN_MSGID_LOC_MASK, msgid);
> + wait_head = &chan_info->wait[tmp];
> + spin_lock_irq(&wait_head->status_lock);
> + WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
> + spin_unlock_irq(&wait_head->status_lock);
> + atomic_dec(&chan_info->inflight_tx_cnt);
> + return 0;
> + }
[Severity: Medium]
Can this write to a slot the caller no longer owns? txq_lock is what
serializes TX slot ownership, and it is dropped just above. The slot was
already set to IDLE inside the earlier status_lock block:
WRITE_ONCE(wait_head->status, chan_send->ack ?
NBL_MBX_STATUS_WAITING : NBL_MBX_STATUS_IDLE);
so this second write looks redundant. In the gap, another sender can be
handed the same slot, and nbl_chan_get_msg_id() starts its search exactly
there because wait_head_index was just pointed at it:
int search_loc = chan_info->wait_head_index, i;
...
if (status == NBL_MBX_STATUS_IDLE ||
status == NBL_MBX_STATUS_TIMEOUT) {
...
WRITE_ONCE(chan_info->wait_head_index, search_loc);
Sender B then arms the slot WAITING and blocks, sender A forces it back to
IDLE, and the ACK is dropped:
nbl_chan_recv_ack_msg()
if (READ_ONCE(wait_head->status) != NBL_MBX_STATUS_WAITING) {
spin_unlock_irq(&wait_head->status_lock);
dev_err_ratelimited(dev, "Skip ack with invalid status, ...");
B then fails with -ETIMEDOUT after NBL_CHAN_ACK_WAIT_TIME. The interrupt
mode timeout path only cleans up when it still sees NBL_MBX_STATUS_WAITING,
so it can instead clobber a third sender's freshly armed slot, setting it to
TIMEOUT and NULLing its ack_data while that sender waits. Note
nbl_chan_send_ack() is exactly a chan_send->ack == 0 sender, so both
participants are the workqueue ACK sender and a synchronous RPC. Can this
write simply be dropped?
[ ... ]
> +static int nbl_chan_send_ack(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_ack_info *chan_ack)
> +{
[ ... ]
> + len = head_len + data_len;
> + tmp = kzalloc(len, GFP_ATOMIC);
> + if (!tmp)
> + return -ENOMEM;
[Severity: Low]
Is GFP_ATOMIC needed here? This path cannot be atomic: nbl_chan_send_msg()
below takes mutex_lock(&chan_info->txq_lock) and nbl_chan_kick_tx_ring()
calls usleep_range(). The only caller chain is the clean_task work item
(the mailbox IRQ handler just schedules work), so GFP_KERNEL would match the
actual context, and the GFP_ATOMIC here documents a contract the code cannot
honour.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> new file mode 100644
> index 000000000000..92db39e1b05e
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> @@ -0,0 +1,170 @@
[ ... ]
> + /*
> + * Spinlock protecting all fields.
> + * Must be held when reading/writing: status, acked, ack_err,
> + * ack_data_len, etc.
> + * The lock ensures atomic updates of these fields and
> + * proper memory ordering with smp_wmb()/smp_rmb().
> + */
> + spinlock_t status_lock;
[Severity: Low]
Does this comment match the code? Two readers of acked run with
status_lock dropped:
nbl_chan_send_msg()
ret = wait_event_timeout(wait_head->wait_queue,
READ_ONCE(wait_head->acked) || ...
nbl_chan_recv_ack_msg()
spin_unlock_irq(&wait_head->status_lock);
if (READ_ONCE(wait_head->acked))
wake_up(&wait_head->wait_queue);
There is also no smp_wmb() anywhere in the driver, so the documented barrier
pairing does not exist. Could the comment be narrowed to the fields the
lock actually serializes?
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> new file mode 100644
> index 000000000000..fc9b489ca02c
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> @@ -0,0 +1,166 @@
[ ... ]
> +/*
> + * Allocate a hash node and add to the hash table.
> + *
> + * Note: The hash table is protected by the caller's mutex (txq_lock),
> + * not lock-free. The "single context" comment refers to: this function
> + * is only called during init from nbl_chan_init_msg_handler, so there
> + * is no concurrent mutation during init.
> + * After init, the table is read-only (no unregister API), so no lock is needed
> + * for lookups either.
[Severity: Low]
Both claims in this note look inaccurate.
nbl_chan_init_msg_handler() only creates the table via
nbl_common_init_hash_table(); the caller of this function is
nbl_chan_add_msg_handler() reached through the exported op:
static int nbl_chan_register_msg(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
nbl_chan_resp func, void *callback)
{
return nbl_chan_add_msg_handler(chan_mgt, msg_type, func, callback);
}
and chan_info->txq_lock is taken only by nbl_chan_send_msg(), never on the
registration path. Insertion is a plain hlist_add_head() plus a non-atomic
node_num++, while nbl_common_get_hash_node() traverses with the non-RCU
hlist_for_each_entry() from the clean_task work item.
Today all registrations complete before the RX queue is programmed, so
nothing races. Could the note instead state that requirement, since the
current wording points at a mutex that is not involved?
[ ... ]
Cross-instance finding from sashiko-gemini (b71ef5b11f4458a07533fdf4c44ce80b1dc5c6eaa0b8f201e2c68859763e947e):
[Severity: Medium]
The code implements the `clean_task` requeue optimization, directly contradicting the commit message.
Cross-instance finding from sashiko-gemini (f61ba5020ca4df734e90e4522b80858df46b2e1c1ebb7645f42be89a01ba9a4a):
[Severity: Critical]
Wait slot leak and stack corruption when interrupts are not ready in nbl_chan_send_msg.
--
pw-bot: cr
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure
2026-08-07 4:44 ` [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
@ 2026-08-14 19:45 ` Jakub Kicinski
0 siblings, 0 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-08-14 19:45 UTC (permalink / raw)
To: illusion.wang
Cc: Jakub Kicinski, dimon.zhao, alvin.wang, sam.chen, netdev,
andrew+netdev, corbet, horms, linux-doc, pabeni, vadim.fedorenko,
lukas.bulwahn, edumazet, enelsonmoore, skhan, hkallweit1,
linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
net/nebula-matrix: dispatch: add control-level routing core infrastructure
This adds the nbl dispatch layer: a dispatch mgt struct, an ops table
allocated with devm_kzalloc(), a ctrl_lvl bitmask, and local wrappers for
the resource layer init_module/deinit_module calls. nbl_disp_init() is
wired into nbl_core_init() and nbl_disp_remove() into nbl_core_remove().
A few questions about the changelog and the new code below.
> Add base dispatch layer infrastructure for control-level routing:
> 1. Dispatch management & ops table structures allocation
> 2. X-macro op table template for uniform dispatch entry registration
> 3. Control PF / regular PF routing logic via ctrl_lvl bitmask
> 4. Local chip init/deinit dispatch wrappers (no channel dependency)
>
> Document constraint: init_module/deinit_module only valid
> on Control PF, caller must guard with has_ctrl to avoid NULL deref.
[Severity: Medium]
Do bullets 2 and 3 and the "Document constraint" paragraph match what the
patch contains?
For bullet 2, there is no X-macro anywhere in the patch. The only related
artifact is the identity macro added to nbl_include.h:
/* Used for macros to pass checkpatch */
#define NBL_NAME(x) x
and the two ops that do get registered are registered by plain pointer
assignment in nbl_disp_setup_ctrl_lvl():
disp_ops->init_module = nbl_disp_init_module;
disp_ops->deinit_module = nbl_disp_deinit_module;
A grep for NBL_NAME( under drivers/net/ethernet/nebula-matrix/ returns a
single hit, the definition itself, both at this commit and at the end of
the series, so no X-macro registration table is introduced later either.
For bullet 3, only the control PF direction is installed here:
if (common->has_ctrl)
nbl_disp_setup_ctrl_lvl(disp_mgt, NBL_DISP_CTRL_LVL_MGT);
NBL_DISP_CTRL_LVL_NET matches only its own enum declaration at this
commit, and the non-ctrl route appears later in "net/nebula-matrix:
dispatch: add channel RPC framework & shared hw ops mutex". Could the
bullet be deferred to that patch?
For the "Document constraint" paragraph, no comment describing the
has_ctrl requirement appears in nbl_dispatch.c, nbl_dispatch.h or
nbl_def_dispatch.h. Would it make sense to put that constraint next to
the init_module/deinit_module declarations in struct nbl_dispatch_ops so a
future caller can see it?
> This patch only provides core routing skeleton, no channel message
> handling or resource locking logic.
[Severity: Low]
This statement seems to conflict with the nbl_channel.c and
nbl_def_channel.h hunks below: the patch adds a new on-wire response code
enum (NBL_CHAN_RESP_OK/ERR/UNIMPLEMENTED), four mailbox payload layouts,
and four BUILD_BUG_ON() assertions inside nbl_chan_init_common(). Could
the changelog mention that this patch establishes a peer/firmware facing
wire contract?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> index 3eb795199217..9e0f9b6c4c3a 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -1030,6 +1030,10 @@ int nbl_chan_init_common(struct nbl_adapter *adap)
> struct nbl_channel_mgt *chan_mgt;
> int ret;
>
> + BUILD_BUG_ON(sizeof(struct nbl_chan_param_cfg_msix_map) != 8);
> + BUILD_BUG_ON(sizeof(struct nbl_chan_param_set_mailbox_irq) != 4);
> + BUILD_BUG_ON(sizeof(struct nbl_chan_param_get_vsi_id) != 4);
> + BUILD_BUG_ON(sizeof(struct nbl_chan_param_get_eth_id) != 8);
[Severity: Low]
The four sizes are numerically correct (2+2+2+2, 2+1+1, 2+2 and
2+1+1+1+3), so there is no functional problem here.
These are dispatch layer message structs though, and the assertions are
placed in the channel layer's runtime init function. Would static_assert()
next to each struct definition in nbl_def_channel.h keep the wire format
check with the format it describes?
> chan_mgt = nbl_chan_setup_chan_mgt(adap);
> if (IS_ERR(chan_mgt)) {
> ret = PTR_ERR(chan_mgt);
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> new file mode 100644
> index 000000000000..9358651cbf7e
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> @@ -0,0 +1,112 @@
[ ... ]
> +static void nbl_disp_deinit_module(struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
> + struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
> +
> + if (res_ops->deinit_module)
> + res_ops->deinit_module(p);
> +}
> +
> +static int nbl_disp_init_module(struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_resource_ops *res_ops = disp_mgt->res_ops_tbl->ops;
> + struct nbl_resource_mgt *p = disp_mgt->res_ops_tbl->priv;
> +
> + if (res_ops->init_module)
> + return res_ops->init_module(p);
> + return -EOPNOTSUPP;
> +}
[Severity: Low]
Can these NULL tests ever be false, and can -EOPNOTSUPP ever be returned?
nbl_res_setup_ops() already refuses to publish an incomplete table:
if (!res_ops.get_vsi_id || !res_ops.get_eth_id ||
!res_ops.cfg_msix_map || !res_ops.destroy_msix_map ||
!res_ops.set_mailbox_irq || !res_ops.init_module ||
!res_ops.deinit_module)
return ERR_PTR(-EINVAL);
and nbl_core_init() -> nbl_res_init_leonis() propagates that failure, so
nbl_disp_init() only runs with both pointers set.
That makes the guards and the -EOPNOTSUPP path unreachable while
advertising a return value a caller may try to handle. This is still the
case at the end of the series. Would it be cleaner to keep the validation
in one layer only?
> +
> +static void nbl_disp_setup_ctrl_lvl(struct nbl_dispatch_mgt *disp_mgt, u32 lvl)
> +{
> + struct nbl_dispatch_ops *disp_ops = disp_mgt->disp_ops_tbl->ops;
> +
> + set_bit(lvl, disp_mgt->ctrl_lvl);
> +
> + if (test_bit(NBL_DISP_CTRL_LVL_MGT, disp_mgt->ctrl_lvl)) {
> + disp_ops->init_module = nbl_disp_init_module;
> + disp_ops->deinit_module = nbl_disp_deinit_module;
> + }
> +}
[ ... ]
> +static struct nbl_dispatch_ops_tbl *
> +nbl_disp_setup_ops(struct device *dev, struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl;
> + struct nbl_dispatch_ops *disp_ops;
> +
> + disp_ops_tbl = devm_kzalloc(dev, sizeof(*disp_ops_tbl), GFP_KERNEL);
> + if (!disp_ops_tbl)
> + return ERR_PTR(-ENOMEM);
> +
> + disp_ops = devm_kzalloc(dev, sizeof(*disp_ops), GFP_KERNEL);
> + if (!disp_ops)
> + return ERR_PTR(-ENOMEM);
> +
> + disp_ops_tbl->ops = disp_ops;
> + disp_ops_tbl->priv = disp_mgt;
[Severity: Low]
This publishes a zeroed struct nbl_dispatch_ops through
adapter->intf.dispatch_ops_tbl, and at this commit only init_module and
deinit_module are ever filled, and only when common->has_ctrl is set. The
other five declared ops (cfg_msix_map, destroy_msix_map, set_mailbox_irq,
get_vsi_id, get_eth_id) stay NULL for every PF.
Unlike the sibling constructors nbl_res_setup_ops() and nbl_hw_setup_ops(),
which return ERR_PTR(-EINVAL) when any op is NULL, the dispatch layer never
validates the table it hands out.
To be clear about reachability: I could not find a consumer that would hit
a NULL op. By the end of the series nbl_disp_setup_ctrl_lvl() fills the
remaining five ops in both the MGT branch and the new else branch, and the
only call sites of init_module/deinit_module, nbl_dev_setup_ctrl_dev() and
nbl_dev_remove_ctrl_dev(), are reached only under if (common->has_ctrl) in
nbl_dev_init()/nbl_dev_remove().
Would adding the same completeness check the resource and hw layers use,
and documenting the has_ctrl-only validity of init_module/deinit_module in
the header, be worth doing here?
> +
> + return disp_ops_tbl;
> +}
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> index b5a7b069d834..23bdd453ee73 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> @@ -13,6 +13,12 @@ struct nbl_adapter;
>
> typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
>
> +enum {
> + NBL_CHAN_RESP_OK = 0,
> + NBL_CHAN_RESP_ERR = 1,
> + NBL_CHAN_RESP_UNIMPLEMENTED = 2,
> +};
> +
[Severity: Low]
These response codes and the four nbl_chan_param_* layouts below have no
producer or consumer at this commit; they gain users in the following patch
"net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops
mutex", which is fine for a staged series. The point is only that the
changelog says this patch has no channel message handling while it defines
the channel response ABI.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> new file mode 100644
> index 000000000000..b26425e0112b
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> @@ -0,0 +1,42 @@
[ ... ]
> +enum {
> + NBL_DISP_CTRL_LVL_NEVER = 0,
> + NBL_DISP_CTRL_LVL_MGT,
> + NBL_DISP_CTRL_LVL_NET,
> + NBL_DISP_CTRL_LVL_MAX,
> +};
[Severity: Medium]
NBL_DISP_CTRL_LVL_NET is declared here but never set or tested at this
commit, which is the enumerator the changelog's "Control PF / regular PF
routing logic via ctrl_lvl bitmask" would rely on.
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index 3f35982cb02a..5f33a5de908d 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -15,6 +15,8 @@
>
> #define NBL_MAX_FUNC 520
> #define NBL_MAX_ETHERNET 4
> +/* Used for macros to pass checkpatch */
> +#define NBL_NAME(x) x
[Severity: Low]
This identity macro has no users. A grep for NBL_NAME( under
drivers/net/ethernet/nebula-matrix/ returns only this definition, both at
this commit and at the end of the series, and the X-macro table given as
its justification is never added.
Also, what exactly does checkpatch object to? The comment says the macro
exists so later constructs pass checkpatch rather than to address the
underlying complaint. Could this define be dropped?
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation
2026-08-07 4:44 ` [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
@ 2026-08-14 19:48 ` Jakub Kicinski
0 siblings, 0 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-08-14 19:48 UTC (permalink / raw)
To: illusion.wang
Cc: Jakub Kicinski, dimon.zhao, alvin.wang, sam.chen, netdev,
andrew+netdev, corbet, horms, linux-doc, pabeni, vadim.fedorenko,
lukas.bulwahn, edumazet, enelsonmoore, skhan, hkallweit1,
linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
net/nebula-matrix: add common/ctrl dev init/remove operation
This adds nbl_dev.c with nbl_dev_setup_common_dev() and
nbl_dev_setup_ctrl_dev() plus their teardown paths, hooks them into
nbl_core_init()/nbl_core_remove(), adds a per-adapter workqueue helper, and
extends nbl_chan_send_msg() with a polling ACK path used when
NBL_CHAN_INTERRUPT_READY is not set.
A few questions on the changelog and on the new code below.
[Severity: Low]
Is this paragraph describing the state of the tree at this commit?
> Mailbox message processing is implemented via work_struct. This task
> registration framework is reused for interrupt-driven receive now, and
> can be shared with polling receive mode introduced in follow-up patches.
At this commit NBL_CHAN_INTERRUPT_READY is only ever tested, in
nbl_chan_send_msg(); nothing sets it, and there is no request_irq() or
pci_alloc_irq_vectors() anywhere under drivers/net/ethernet/nebula-matrix/.
The bit is first set (and the mailbox IRQ first requested) by the later
"net/nebula-matrix: add common dev start/stop operation" patch.
So the relationship looks inverted: polling is what is live now, and the
wait_event_timeout() branch of nbl_chan_send_msg() is unreachable here.
Every synchronous request issued by this patch (get_vsi_id/get_eth_id from
nbl_dev_setup_common_dev()) goes through the new polling loop, which can
run NBL_CHAN_TX_WAIT_ACK_TIMES (5000) iterations of roughly 1 ms, i.e. up
to about 5 seconds per message during probe. Would it be worth saying that
explicitly?
Also on this sentence:
> requests rely on polling to wait for firmware ACK, avoiding silent
> success and uninitialized variable reads when NBL_CHAN_INTERRUPT_READY
> is not yet set.
Before this patch, ret still held 0 from nbl_chan_kick_tx_ring() when the
if/else fell through, so the old behaviour was returning success with an
unpopulated response buffer rather than reading an uninitialized variable.
Could the wording be adjusted?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> index 9e0f9b6c4c3a..3b7f3876a075 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -636,10 +636,13 @@ static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
> static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
> struct nbl_chan_info *chan_info)
> {
> + struct nbl_common_info *common = chan_mgt->common;
> struct nbl_chan_ring *rxq = &chan_info->rxq;
> struct device *dev = chan_mgt->common->dev;
> struct nbl_chan_rx_desc *rx_desc;
> struct nbl_chan_buf *rx_buf;
> + struct work_struct *task;
> + bool more_work = false;
> u16 next_to_clean;
> u32 budget = 64;
> u16 flags;
> @@ -666,11 +669,22 @@ static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
> next_to_clean = 0;
> rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
> rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
> - if (--budget == 0)
> + if (--budget == 0) {
> + more_work = true;
> break;
> + }
> cond_resched();
> }
> rxq->next_to_clean = next_to_clean;
> +
[Severity: Low]
Should the changelog mention that nbl_chan_clean_queue() becomes a
self-rearming worker here? The changelog only says "Mailbox message
processing is implemented via work_struct" and describes the polling ACK
fallback, but this hunk also changes the semantics of an existing function
in nbl_channel.c: it now re-queues its own work item on budget exhaustion
and returns early on shutdown.
Small related point: more_work is set purely on budget exhaustion, so a
ring holding exactly 64 used descriptors re-arms the work once with nothing
left to clean. Is that intended?
> + /* Prevent queue_work after teardown clears clean_task */
> + if (READ_ONCE(chan_info->shutdown))
> + return;
[Severity: Low]
Does this comment hold? The guard is two independent unordered loads
followed by queue_work(), with nothing serializing them against
nbl_chan_teardown_queue().
Looking at nbl_chan_teardown_queue():
while (atomic_read(&chan_info->inflight_tx_cnt) != 0) {
if (time_after(jiffies, wait_start + wait_timeout)) {
dev_warn(chan_mgt->common->dev,
"teardown: wait inflight_tx_cnt timeout, force continue\n");
ret = -ETIMEDOUT;
break;
}
...
}
/* Step3: All send paths drained, safely cancel cleanup work */
if (chan_info->clean_task) {
cancel_work_sync(chan_info->clean_task);
chan_info->clean_task = NULL;
}
The drain is bounded at 5 s and teardown continues regardless, so a context
that has already observed shutdown == false and a non-NULL clean_task can
still call queue_work() after teardown finished. The flag check narrows the
window but does not close it, and the same pattern is repeated in the new
polling loop of nbl_chan_send_msg(). Could the comment be reworded, or the
publish/clear of clean_task be made to actually interlock with the readers?
> + if (common->wq && more_work) {
> + task = READ_ONCE(chan_info->clean_task);
> + if (task)
> + queue_work(common->wq, task);
> + }
> }
>
> static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
[ ... ]
> @@ -869,6 +885,60 @@ static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
> WRITE_ONCE(wait_head->ack_data_len, 0);
> spin_unlock_irq(&wait_head->status_lock);
> goto inflight_dec_out;
> + } else {
> + /* Polling wait path for synchronous ACK */
> + while (i--) {
> + if (READ_ONCE(chan_info->shutdown)) {
> + ret = -ESHUTDOWN;
> + /*
> + * Shutdown happens during polling wait,
> + * reclaim slot
> + */
> + spin_lock_irq(&wait_head->status_lock);
> + WRITE_ONCE(wait_head->acked, 0);
> + WRITE_ONCE(wait_head->ack_data, NULL);
> + WRITE_ONCE(wait_head->ack_data_len, 0);
> + WRITE_ONCE(wait_head->status,
> + NBL_MBX_STATUS_TIMEOUT);
> + spin_unlock_irq(&wait_head->status_lock);
> + goto inflight_dec_out;
> + }
> +
> + task = READ_ONCE(chan_info->clean_task);
> + if (common->wq && task &&
> + !READ_ONCE(chan_info->shutdown))
> + queue_work(common->wq, task);
> +
> + spin_lock_irq(&wait_head->status_lock);
> + if (READ_ONCE(wait_head->acked)) {
> + chan_send->ack_len = wait_head->ack_data_len;
> + ret = wait_head->ack_err;
> + WRITE_ONCE(wait_head->acked, 0);
> + WRITE_ONCE(wait_head->status,
> + NBL_MBX_STATUS_IDLE);
> + spin_unlock_irq(&wait_head->status_lock);
> + goto inflight_dec_out;
> + }
[Severity: Low]
Is it intentional that this success path leaves wait_head->ack_data and
wait_head->ack_data_len set while publishing the slot as
NBL_MBX_STATUS_IDLE?
Every other exit of nbl_chan_send_msg() clears both, for example the
interrupt-wait success path just above:
WRITE_ONCE(wait_head->acked, 0);
WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_IDLE);
WRITE_ONCE(wait_head->ack_data, NULL);
WRITE_ONCE(wait_head->ack_data_len, 0);
and so do the new polling shutdown and polling timeout blocks. Here the
slot is released as IDLE with ack_data still pointing at the caller's stack
response buffer (for instance the local result struct in
nbl_disp_chan_get_vsi_id_req()) and a non-zero ack_data_len.
Today nothing dereferences it, because nbl_chan_recv_ack_msg() only writes
through ack_data when status == NBL_MBX_STATUS_WAITING, and the send path
re-assigns both fields before re-arming a slot as WAITING. The visible
effect is a stale length in the "Skip ack with invalid status" diagnostic.
Should this path clear them anyway to keep the slot-release contract
uniform?
> + spin_unlock_irq(&wait_head->status_lock);
> +
> + usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
> + NBL_CHAN_TX_WAIT_ACK_US_MAX);
> + cond_resched();
> + }
> +
> + /* Polling loop exhausted, reclaim slot */
> + spin_lock_irq(&wait_head->status_lock);
> + WRITE_ONCE(wait_head->acked, 0);
> + WRITE_ONCE(wait_head->ack_data, NULL);
> + WRITE_ONCE(wait_head->ack_data_len, 0);
> + WRITE_ONCE(wait_head->status, NBL_MBX_STATUS_TIMEOUT);
> + spin_unlock_irq(&wait_head->status_lock);
[Severity: Low]
Can this block throw away an ACK that arrived during the last sleep?
The final iteration tests acked before its usleep_range(1000, 1200), and
this cleanup then clears acked and forces NBL_MBX_STATUS_TIMEOUT without
re-testing it. An ACK delivered in that last window is processed by the
clean work item, which reaches nbl_chan_recv_ack_msg():
ack_done:
WRITE_ONCE(wait_head->acked, 1);
spin_unlock_irq(&wait_head->status_lock);
That only sets acked/ack_err/ack_data_len and leaves status at
NBL_MBX_STATUS_WAITING, so the cleanup here cannot distinguish "never
acked" from "just acked". The caller gets -ETIMEDOUT even though the
response was already copied into its buffer, and in
nbl_dev_setup_common_dev() that aborts nbl_dev_init() and the whole probe.
The interrupt path re-tests the status before overwriting it; should this
one do the same?
> +
> + dev_err_ratelimited(dev,
> + "Channel polling ack failed, message type: %d msg id: %u\n",
> + chan_send->msg_type, msgid);
> + ret = -ETIMEDOUT;
> + goto inflight_dec_out;
> }
>
> inflight_dec_out:
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> index 92db39e1b05e..d59e660be9c7 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> @@ -24,6 +24,9 @@
> #define NBL_CHAN_TX_WAIT_US 100
> #define NBL_CHAN_TX_WAIT_US_MAX 120
> #define NBL_CHAN_TX_WAIT_TIMES 100
> +#define NBL_CHAN_TX_WAIT_ACK_US_MIN 1000
> +#define NBL_CHAN_TX_WAIT_ACK_US_MAX 1200
> +#define NBL_CHAN_TX_WAIT_ACK_TIMES 5000
> #define NBL_CHAN_QUEUE_LEN 256
> #define NBL_CHAN_BUF_LEN 4096
> #define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> index 149681dd216d..f10ef78ee7f6 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> @@ -7,6 +7,27 @@
> #include <linux/jhash.h>
> #include "nbl_common.h"
>
> +void nbl_common_destroy_wq(struct nbl_common_info *common)
> +{
> + destroy_workqueue(common->wq);
> +}
[Severity: Medium]
Should this set common->wq = NULL after destroy_workqueue()?
Both new queue_work() sites use the pointer itself as the liveness test:
nbl_chan_clean_queue():
if (common->wq && more_work) {
task = READ_ONCE(chan_info->clean_task);
if (task)
queue_work(common->wq, task);
}
nbl_chan_send_msg() polling loop:
task = READ_ONCE(chan_info->clean_task);
if (common->wq && task && !READ_ONCE(chan_info->shutdown))
queue_work(common->wq, task);
Can this end up as a use-after-free of the workqueue? nbl_dev_remove()
runs nbl_dev_remove_common_dev() (which reaches nbl_chan_teardown_queue(),
whose inflight drain is explicitly best-effort: after 5 s it prints
"teardown: wait inflight_tx_cnt timeout, force continue", sets -ETIMEDOUT
and proceeds to cancel_work_sync() plus clean_task = NULL), then calls
nbl_common_destroy_wq(common) unconditionally, ignoring that result.
Meanwhile the polling loop itself can run 5000 iterations of 1.0-1.2 ms,
which is longer than the 5 s drain bound. A sender preempted between its
shutdown check and queue_work(common->wq, task) would then queue onto a
freed workqueue and re-arm a work item after cancel_work_sync(). Would
clearing common->wq under the same ordering that clears clean_task, plus a
NULL guard here, close that?
> +
> +int nbl_common_create_wq(struct nbl_common_info *common)
> +{
> + char wq_name[32];
> +
> + snprintf(wq_name, sizeof(wq_name), "nbl_wq_%s", pci_name(common->pdev));
> + common->wq = alloc_workqueue("%s", WQ_UNBOUND, 0, wq_name);
> + if (!common->wq) {
> + pr_err("Failed to create workqueue nbl_wq\n");
> + goto alloc_wq_failed;
> + }
> +
> + return 0;
> +alloc_wq_failed:
> + return -ENOMEM;
> +}
> +
[ ... ]
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> new file mode 100644
> index 000000000000..bdb4d6a45fbc
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
[ ... ]
> +/* ---------- Channel config ---------- */
> +static void nbl_dev_setup_chan_qinfo(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + struct nbl_common_info *common = dev_mgt->common;
> +
> + if (!chan_ops->check_queue_exist(priv, chan_type))
> + return;
> +
> + chan_ops->cfg_chan_qinfo_map_table(priv, common->hw_bus, common->devid);
> +}
[Severity: Low]
This isn't a bug introduced here, but this patch adds the only caller of
cfg_chan_qinfo_map_table(), which makes an existing comment checkable and
wrong. In nbl_res_ctrl_dev_sriov_info_init() the hw_bus assignment carries:
/*
* common->hw_bus supplies bus number for channel mailbox QINFO mapping.
* Execution order guarantee: this assignment runs before
* cfg_chan_qinfo_map_table() in nbl_dev_start(), only executed
* on control PF path.
*/
There is no nbl_dev_start() at this commit, and even after
"net/nebula-matrix: add common dev start/stop operation" adds one, it does
not call cfg_chan_qinfo_map_table(); the call lives in
nbl_dev_setup_chan_qinfo() from nbl_dev_setup_ctrl_dev(). The ordering
claim itself holds, only the named consumer is wrong. Could the comment be
updated to name nbl_dev_setup_ctrl_dev()?
[Severity: Medium]
Can this write clobber the mailbox MSI-X fields of other PFs?
nbl_chan_cfg_qinfo_map_table() loops over every host PF:
hw_ops->get_host_pf_mask(p, &pf_mask);
for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
if (!(pf_mask & (1 << func_id)))
hw_ops->cfg_mailbox_qinfo(p, func_id, bus,
devid, func_id);
and nbl_hw_cfg_mailbox_qinfo() is a full-word write built from zero:
u32 data = 0;
data = FIELD_PREP(NBL_MAILBOX_QINFO_MAP_FUNCTION_MASK, function) |
FIELD_PREP(NBL_MAILBOX_QINFO_MAP_DEVID_MASK, devid) |
FIELD_PREP(NBL_MAILBOX_QINFO_MAP_BUS_MASK, bus);
nbl_hw_wr_regs_lock(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
&data, sizeof(data));
so NBL_MAILBOX_QINFO_MAP_MSIX_IDX and MSIX_IDX_VALID are zeroed. Those are
exactly the fields nbl_hw_set_mailbox_irq() programs in the same register,
and that function already warns about this:
/*
* Note: This RMW is currently safe because the two callers are strictly
* sequential: nbl_hw_cfg_mailbox_qinfo() runs at init, nbl_hw_set_mailbox_irq()
* runs at start. Future reset/hot-add/VF-reinit paths must preserve this
* ordering; ...
*/
This patch puts the clobbering write on the ctrl-PF probe path, and it
rewrites entries belonging to other functions. If PF0 is unbound and
rebound while other PF instances stay bound with their mailbox MSI-X
already programmed on their behalf by the ctrl PF, do those functions lose
MSIX_IDX_VALID in hardware while their drivers still have
NBL_CHAN_INTERRUPT_READY set and keep waiting on interrupts? Would making
this write an RMW under reg_lock, or limiting it to func_ids owned by this
PF, be preferable?
> +
> +static int nbl_dev_setup_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + int ret = 0;
> +
> + if (chan_ops->check_queue_exist(priv, chan_type))
> + ret = chan_ops->setup_queue(priv, chan_type);
> +
> + return ret;
> +}
> +
> +static int nbl_dev_remove_chan_queue(struct nbl_dev_mgt *dev_mgt, u8 chan_type)
> +{
> + struct nbl_channel_ops *chan_ops = dev_mgt->chan_ops_tbl->ops;
> + struct nbl_channel_mgt *priv = dev_mgt->chan_ops_tbl->priv;
> + int ret = 0;
> +
> + if (chan_ops->check_queue_exist(priv, chan_type))
> + ret = chan_ops->teardown_queue(priv, chan_type);
> +
> + return ret;
> +}
[ ... ]
> +/* ---------- Dev init process ---------- */
> +static int nbl_dev_setup_common_dev(struct nbl_adapter *adapter)
> +{
[ ... ]
> + ret = nbl_dev_setup_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> + if (ret)
> + return ret;
> +
> + INIT_WORK(&common_dev->clean_mbx_task, nbl_dev_clean_mailbox_task);
> + nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX,
> + &common_dev->clean_mbx_task);
[Severity: Low]
Should the publication of clean_task here be annotated? The store side is a
plain assignment in nbl_chan_register_chan_task():
chan_info->clean_task = task;
and the clearing side in nbl_chan_teardown_queue() is likewise a plain
chan_info->clean_task = NULL, while all consumers added by this patch read
it with READ_ONCE() and immediately queue_work() the result. The
asymmetry is an unmarked-write/marked-read race that KCSAN would flag,
and in principle a cross-CPU reader could observe the pointer before the
INIT_WORK() stores to work->func. I could not find such a remote reader at
this commit or at the end of the series, so this looks latent rather than
live. Would WRITE_ONCE() or smp_store_release() here match the readers?
> + /*
> + * Even if has_ctrl=false (no dedicated control PF channel), we fetch
> + * VSI/ETH info via regular mailbox message instead of
> + * dedicated control command.
> + */
> + ret = disp_ops->get_vsi_id(priv, NBL_VSI_DATA, &common->vsi_id);
> + if (ret)
> + goto err_cleanup;
> + ret = disp_ops->get_eth_id(priv, common->vsi_id, &common->eth_num,
> + &common->eth_id, &common->logic_eth_id);
> + if (ret)
> + goto err_cleanup;
[ ... ]
> +static void nbl_dev_remove_common_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> +
> + if (!common_dev)
> + return;
> + nbl_dev_remove_chan_queue(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> + cancel_work_sync(&common_dev->clean_mbx_task);
> + nbl_dev_register_chan_task(dev_mgt, NBL_CHAN_TYPE_MAILBOX, NULL);
> +}
[Severity: Low]
nbl_chan_teardown_queue() and nbl_chan_setup_queue() came from the earlier
"net/nebula-matrix: add channel layer" patch, so this isn't introduced
here, but this patch adds the first callers, so it becomes reachable.
Teardown puts both mailbox queues into hardware reset via
stop_mailbox_rxq()/stop_mailbox_txq() but leaves chan_info->init_done
true, chan_info->shutdown true, and all ring indices (txq->next_to_use,
txq->tail_ptr, rxq->next_to_clean/next_to_use/tail_ptr) at stale values.
Since nbl_chan_setup_queue() starts with:
if (READ_ONCE(chan_info->init_done))
return 0;
would a later re-init silently report success without re-programming the
queue base/size/QUEUE_EN registers, re-arming RX descriptors, or clearing
shutdown? Today the only setup caller is nbl_dev_setup_common_dev() and
chan_info is freshly devm_kzalloc()'d per probe, so nothing re-inits within
one adapter lifetime, but the reset/recovery paths the changelog mentions
would hit it. Should teardown clear init_done/shutdown and reset the ring
indices?
Related ordering question in the same function: nbl_chan_stop_queue() runs
before the inflight-sender drain, so a sender that already passed its
shutdown check can still publish an AVAIL descriptor, bump txq->tail_ptr
and ring the doorbell of an already-reset queue. At the end of the series
the mailbox worker sends ACKs via nbl_chan_send_ack() concurrently with
removal, so this is reachable; the effect looks like a dropped ACK plus a
tail counter permanently ahead of the zeroed hardware tail. Would draining
before stopping the queues be safer?
> +
> +static int nbl_dev_setup_ctrl_dev(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dispatch_ops *disp_ops = dev_mgt->disp_ops_tbl->ops;
> + int ret;
> +
> + ret = disp_ops->init_module(dev_mgt->disp_ops_tbl->priv);
> + if (ret)
> + return ret;
> +
> + nbl_dev_setup_chan_qinfo(dev_mgt, NBL_CHAN_TYPE_MAILBOX);
> +
> + return 0;
> +}
[ ... ]
> +int nbl_dev_init(struct nbl_adapter *adapter)
> +{
[ ... ]
> + /*
> + * Chip hardware initialization is completed by firmware at power-up.
> + * Only driver functional table/register config follows here, safe to
> + * access hardware registers before ctrl dev setup.
> + */
> + ret = nbl_dev_setup_common_dev(adapter);
> + if (ret)
> + goto setup_err;
> +
> + if (common->has_ctrl) {
> + ret = nbl_dev_setup_ctrl_dev(adapter);
> + if (ret)
> + goto setup_ctrl_dev_fail;
> + }
[Severity: Medium]
Does the firmware really cover everything this ordering depends on?
nbl_dev_setup_common_dev() reaches nbl_chan_setup_queue(), which enables
mailbox DMA before nbl_dev_setup_ctrl_dev() has run:
nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
nbl_chan_update_tail_ptr(hw_ops, chan_mgt->hw_ops_tbl->priv,
rxq->tail_ptr, NBL_MB_RX_QID);
Only afterwards does nbl_dev_setup_ctrl_dev() call init_module(), which
goes to nbl_hw_init_module() -> nbl_intf_init() -> nbl_host_padpt_init()
writing NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP and the other flow-control
credit registers, plus nbl_hw_set_driver_status(true); and then
nbl_dev_setup_chan_qinfo() programs the per-function routing map
NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id). Those are driver writes, not
firmware ones, so the changelog statement
> Chip core hardware initialization is fully handled by firmware during
> power-on; the driver only configures functional table entries and
> registers after hardware is ready.
does not seem to cover them. Per the driver's own comment in
nbl_hw_get_board_info(), these registers read zero after cold boot, FLR or
bus reset, so between queue enable and the qinfo write the mailbox block is
armed with an all-zero (00:00.0) routing entry.
For a PF with has_ctrl == false, nbl_dev_setup_ctrl_dev() is skipped
entirely, yet nbl_dev_setup_common_dev() deliberately issues real mailbox
transactions (get_vsi_id/get_eth_id resolve to the request variants). Does
that leave such a PF depending on a separate ctrl PF instance having
already programmed its qinfo entry, with nothing in the code enforcing or
checking it? If the ctrl PF is unbound or assigned to vfio-pci, the new
polling fallback turns this into a silent multi-second stall before
-ETIMEDOUT. Would it be safer to program the credits, driver status and
qinfo map before enabling the mailbox queues?
> +
> + return 0;
> +setup_ctrl_dev_fail:
> + nbl_dev_remove_common_dev(adapter);
> +setup_err:
> + nbl_common_destroy_wq(common);
> + return ret;
> +}
[ ... ]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation
2026-08-07 4:44 ` [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation illusion.wang
@ 2026-08-14 19:48 ` Jakub Kicinski
0 siblings, 0 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-08-14 19:48 UTC (permalink / raw)
To: illusion.wang
Cc: Jakub Kicinski, dimon.zhao, alvin.wang, sam.chen, netdev,
andrew+netdev, corbet, horms, linux-doc, pabeni, vadim.fedorenko,
lukas.bulwahn, edumazet, enelsonmoore, skhan, hkallweit1,
linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
net/nebula-matrix: add common dev start/stop operation
Adds nbl_dev_start()/nbl_dev_stop() to configure the hardware MSI-X map,
allocate MSI-X vectors, request/free the mailbox IRQ and enable/disable the
mailbox interrupt, and hooks nbl_core_start()/nbl_core_stop() into the PCI
probe and remove paths. Also drops the NULL checks around the chip
init_module/deinit_module ops.
A few questions below.
> net/nebula-matrix: add common dev start/stop operation
>
> Add device start/stop helper functions to manage MSI-X vector mapping,
> mailbox interrupt resource initialization and deinitialization.
> Create unbound global workqueue for mailbox task scheduling; this shared
> work infrastructure supports both current interrupt-driven receive and
> future polling receive path.
[ ... ]
> Extend channel TX waiting logic with polling fallback and shutdown detection.
> When interrupt path is disabled, send routines rely on active polling to
> wait for message ACK, preventing deadlock during device stop.
[Severity: Low]
Do these two paragraphs describe changes that are in this patch?
Diffing this commit against its parent for nbl_common/ and nbl_channel/
gives an empty diff. The workqueue is created by the pre-existing
nbl_common_create_wq():
common->wq = alloc_workqueue("%s", WQ_UNBOUND, 0, wq_name);
which is per-PCI-device rather than global, and the polling/shutdown-aware
ACK wait already exists in nbl_chan_send_msg() from the preceding
"add common/ctrl dev init/remove operation" commit.
This patch only consumes both, via queue_work() in
nbl_dev_clean_mailbox_schedule() and chan_ops->set_queue_state(). Could
the changelog be adjusted so bisection and backporting point at the commit
that actually adds them?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> index bdb4d6a45fbc..3a8b7ee2e475 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
[ ... ]
> @@ -14,6 +25,187 @@ static void nbl_dev_init_msix_cnt(struct nbl_dev_mgt *dev_mgt)
> msix_info->serv_info[NBL_MSIX_MAILBOX_TYPE].num = 1;
> }
>
> +static int nbl_dev_request_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
> +{
[ ... ]
> + snprintf(dev_common->mailbox_name, sizeof(dev_common->mailbox_name),
> + "nbl_mailbox@pci:%s", pci_name(common->pdev));
> + err = request_irq(irq_num, nbl_dev_clean_mailbox, 0,
> + dev_common->mailbox_name, dev_mgt);
> + if (err)
> + return err;
> +
> + return 0;
> +}
[ ... ]
> +static int nbl_dev_disable_mailbox_irq(struct nbl_dev_mgt *dev_mgt)
> +{
[ ... ]
> + * This helper is invoked in two paths:
> + * 1. Error unwind path of nbl_dev_start(): immediately followed by
> + * nbl_dev_free_mailbox_irq(), channel resources
> + * are fully torn down afterwards, no stale descriptors remain.
> + * 2. Normal device stop path nbl_dev_stop(): free_irq() synchronously
> + * waits for any in-flight threaded irq bottom-half handler to
> + * finish execution, all pending mailbox cleanup work completes
> + * before channel teardown in late remove stage. No stuck
> + * descriptors will linger in either scenario.
> + */
[Severity: Low]
Is there actually a threaded IRQ bottom half here for free_irq() to wait
on?
nbl_dev_request_mailbox_irq() uses request_irq(), which passes
thread_fn == NULL to request_threaded_irq(), so nbl_dev_clean_mailbox()
runs entirely in hardirq context.
The real bottom half is a workqueue item:
static irqreturn_t nbl_dev_clean_mailbox(int irq, void *data)
{
nbl_dev_clean_mailbox_schedule(dev_mgt); /* queue_work(common->wq, ...) */
return IRQ_HANDLED;
}
free_irq() has no knowledge of that work item. And nbl_dev_stop()
follows free_irq() with cancel_work_sync(&common_dev->clean_mbx_task),
which dequeues a queued-but-not-yet-started item so it never runs, rather
than letting "all pending mailbox cleanup work complete".
Could the comment be reworded to match what the code does?
> + chan_ops->set_queue_state(dev_mgt->chan_ops_tbl->priv,
> + NBL_CHAN_INTERRUPT_READY,
> + NBL_CHAN_TYPE_MAILBOX, false);
> +
> + return disp_ops->set_mailbox_irq(dev_mgt->disp_ops_tbl->priv,
> + local_vec_id, false);
> +}
[Severity: Low]
This isn't a bug today, but the wait strategy in nbl_chan_send_msg() is
picked once from an unlocked test_bit():
if (test_bit(NBL_CHAN_INTERRUPT_READY, chan_info->state)) {
... wait_event_timeout(wait_head->wait_queue,
READ_ONCE(wait_head->acked) || READ_ONCE(chan_info->shutdown),
NBL_CHAN_ACK_WAIT_TIME);
A sender that already sampled the bit as set and committed to that branch
would not be released by nbl_dev_stop(), since the stop path never sets
chan_info->shutdown (only nbl_chan_teardown_queue() does, and it runs
later) and never wakes the wait queues, so it would burn the full ack
timeout and report a spurious failure.
Enumerating the ack=1 senders, all of them are the dispatch request
wrappers called only from nbl_dev_init()/nbl_dev_start()/nbl_dev_stop(),
i.e. probe/remove for that device, which the driver core serialises, and
the mailbox RX path only sends acks with ack=0. So no second waiter can
exist for a given chan_info at the moment. Is it worth re-checking the
bit inside the wait, or setting shutdown/waking the waiters here, for
future concurrent users?
[ ... ]
> @@ -238,3 +438,62 @@ void nbl_dev_remove(struct nbl_adapter *adapter)
[ ... ]
> +/* ---------- Dev start process ---------- */
> +int nbl_dev_start(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int cleanup_ret;
> + int ret;
> +
> + ret = nbl_dev_cfg_msix_map(dev_mgt);
> + if (ret)
> + return ret;
[Severity: Medium]
Should this return jump to init_interrupt_scheme_err instead, so that
nbl_dev_destroy_msix_map() runs?
Every other step in the sequence has an unwind label, and this one can
fail after the hardware side already committed. On a non-control PF
(has_ctrl == 0, i.e. any function other than PF0), cfg_msix_map is
nbl_disp_chan_cfg_msix_map_req(), a mailbox RPC that returns whatever
nbl_chan_send_msg() returns, including -ETIMEDOUT when the ack is lost or
late, while the control PF responder already ran res_ops->cfg_msix_map()
successfully.
On the control PF that call reaches the point of no return:
/* Phase2: All new resource allocation succeeded. */
ret = nbl_res_intr_destroy_msix_map(res_mgt, func_id);
...
intr_mgt->func_intr_res[func_id].interrupts = tmp_interrupts;
after which the kcalloc'ed interrupts[] array, the dmam_alloc_coherent
MSI-X map table and the bits set in intr_net_bmap/intr_other_bmap are only
released by nbl_res_intr_destroy_msix_map(), reachable only through
destroy_msix_map.
Combined with the probe error path below clearing drvdata, nbl_dev_stop()
never runs afterwards, so destroy_msix_map is never issued. Does this
leak the control PF's interrupts[] array, the MSI-X map DMA table and the
vector bitmap bits?
> +
> + ret = nbl_dev_init_interrupt_scheme(dev_mgt);
> + if (ret)
> + goto init_interrupt_scheme_err;
[ ... ]
> +void nbl_dev_stop(struct nbl_adapter *adapter)
> +{
> + struct nbl_dev_mgt *dev_mgt = adapter->core.dev_mgt;
> + struct nbl_dev_common *common_dev = dev_mgt->common_dev;
> + int ret;
> +
> + ret = nbl_dev_disable_mailbox_irq(dev_mgt);
> + if (ret)
> + dev_err(dev_mgt->common->dev,
> + "Failed to disable mailbox IRQ: %d\n", ret);
> + nbl_dev_free_mailbox_irq(dev_mgt);
> + cancel_work_sync(&common_dev->clean_mbx_task);
> + nbl_dev_clear_interrupt_scheme(dev_mgt);
> + ret = nbl_dev_destroy_msix_map(dev_mgt);
[Severity: Low]
Does this cancel_work_sync() achieve anything, given what the next call
does?
nbl_dev_disable_mailbox_irq() has just cleared NBL_CHAN_INTERRUPT_READY,
so on a non-control PF nbl_dev_destroy_msix_map() ->
nbl_disp_chan_destroy_msix_map_req() -> send_msg(..., ack=1) takes the
polling branch of nbl_chan_send_msg(), which re-arms the same work item on
each poll iteration:
task = READ_ONCE(chan_info->clean_task);
if (common->wq && task && !READ_ONCE(chan_info->shutdown))
queue_work(common->wq, task);
clean_task is &common_dev->clean_mbx_task, registered by
nbl_dev_register_chan_task(), so clean_mbx_task is pending or running
again by the time nbl_dev_stop() returns.
It is harmless today because nbl_dev_remove_chan_queue() ->
nbl_chan_teardown_queue() later drains inflight senders, cancel_work_sync()s
clean_task and NULLs it before nbl_common_destroy_wq(). Would moving the
cancel after destroy_msix_map(), or dropping it, be clearer?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
> index bd175e200408..c1e6180a7010 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_chip.c
> @@ -9,8 +9,7 @@ void nbl_res_chip_deinit_module(struct nbl_resource_mgt *res_mgt)
> {
> struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
>
> - if (hw_ops->deinit_module)
> - hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
> + hw_ops->deinit_module(res_mgt->hw_ops_tbl->priv);
> }
>
> int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
> @@ -20,7 +19,5 @@ int nbl_res_chip_init_module(struct nbl_resource_mgt *res_mgt)
> struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
>
> - if (hw_ops->init_module)
> - return hw_ops->init_module(p, eth_speed, eth_num);
> - return -EOPNOTSUPP;
> + return hw_ops->init_module(p, eth_speed, eth_num);
> }
[Severity: Low]
Is this hunk related to dev start/stop? Nothing in the changelog mentions
removing the NULL guards or the -EOPNOTSUPP fallback.
The removal looks safe today, since the single hw_ops instance statically
initialises both callbacks and nbl_hw_setup_ops() rejects a table with
NULL init_module/deinit_module, but none of that is visible from the
changelog. Could this go into its own patch, or at least be mentioned?
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> index 8a968a06b5b6..c12a53da6a9b 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> @@ -131,7 +141,14 @@ static int nbl_probe(struct pci_dev *pdev,
> goto adapter_init_err;
> }
> pci_set_drvdata(pdev, adapter);
> + err = nbl_core_start(adapter);
> + if (err)
> + goto core_start_err;
> +
> return 0;
> +core_start_err:
> + pci_set_drvdata(pdev, NULL);
> + nbl_core_remove(adapter);
> adapter_init_err:
> pci_clear_master(pdev);
> configure_dma_err:
This is the path referenced above: clearing drvdata here means nbl_remove()
sees a NULL adapter, so nbl_core_stop() and therefore
nbl_dev_destroy_msix_map() can never run for a probe that failed inside
nbl_dev_start().
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2026-08-14 19:48 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-07 4:44 [PATCH v24 net-next 00/11] nbl driver for Nebulamatrix NICs illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 01/11] net/nebula-matrix: add minimum nbl build framework illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 02/11] net/nebula-matrix: add core driver architecture and HW layer initialization illusion.wang
2026-08-14 19:33 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 03/11] net/nebula-matrix: add channel wire opcode enum definitions illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 04/11] net/nebula-matrix: add channel layer illusion.wang
2026-08-14 19:38 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 05/11] net/nebula-matrix: add common resource implementation illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 06/11] net/nebula-matrix: add intr " illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 07/11] net/nebula-matrix: add chip-wide hardware init/deinit implementation illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 08/11] net/nebula-matrix: dispatch: add control-level routing core infrastructure illusion.wang
2026-08-14 19:45 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 09/11] net/nebula-matrix: dispatch: add channel RPC framework & shared hw ops mutex illusion.wang
2026-08-07 4:44 ` [PATCH v24 net-next 10/11] net/nebula-matrix: add common/ctrl dev init/remove operation illusion.wang
2026-08-14 19:48 ` Jakub Kicinski
2026-08-07 4:44 ` [PATCH v24 net-next 11/11] net/nebula-matrix: add common dev start/stop operation illusion.wang
2026-08-14 19:48 ` Jakub Kicinski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox