* [PATCH v1 01/15] MAINTAINERS: add the Yunsilicon XSC drivers
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
@ 2026-11-04 15:56 ` Xin Tian
2026-11-04 15:56 ` [PATCH v1 02/15] net/xsc: add PCI core driver skeleton Xin Tian
` (13 subsequent siblings)
14 siblings, 0 replies; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:56 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add a MAINTAINERS entry for the Yunsilicon XSC drivers. The hardware
presents both an ethernet function and an RDMA function, so the entry
covers two subtrees, drivers/net/ethernet/yunsilicon/xsc and
drivers/infiniband/hw/xsc, and lists both netdev and linux-rdma as
list addresses. The file patterns are declared ahead of the code so
that maintainership is established before the first driver patch
lands.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
MAINTAINERS | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index c24144478..34902a130 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -30069,6 +30069,15 @@ S: Maintained
F: Documentation/input/devices/yealink.rst
F: drivers/input/misc/yealink.*
+YUNSILICON XSC DRIVERS
+M: Honggang Wei <weihg@yunsilicon.com>
+M: Xin Tian <tianx@yunsilicon.com>
+L: netdev@vger.kernel.org
+L: linux-rdma@vger.kernel.org
+S: Maintained
+F: drivers/infiniband/hw/xsc
+F: drivers/net/ethernet/yunsilicon/xsc
+
ZD1211RW WIRELESS DRIVER
L: linux-wireless@vger.kernel.org
S: Orphan
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v1 02/15] net/xsc: add PCI core driver skeleton
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
2026-11-04 15:56 ` [PATCH v1 01/15] MAINTAINERS: add the Yunsilicon XSC drivers Xin Tian
@ 2026-11-04 15:56 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:56 ` [PATCH v1 03/15] net/xsc: add the register HAL Xin Tian
` (12 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:56 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the PCI core driver for Yunsilicon XSC controllers. The driver
binds to the PCI functions and provides the shared infrastructure
that the ethernet and RDMA function drivers build on: a central
xsc_core_device per PCI function, firmware capability parsing, DMA
buffer abstractions, and the QP/CQ/EQ resource tables.
Introduce the common headers describing the firmware interface:
the HSI wire formats (hca_cap, command mailboxes, QP/CQ contexts,
WQE/CQE layouts), the command interface structures, and the device
resource tables. The capabilities are read from firmware at init
and drive most later configuration, including the 64-bit feature
flag bits and the MPT/MTT geometry.
The probe path keeps to PCI lifecycle work: map BAR 0, set a 64-bit
DMA mask with a 2G segment size limit, save the PCI state, and
allocate the core device. Support PF device IDs for vendor 0x1f67;
SR-IOV and VF handling are not part of this series.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/net/ethernet/Kconfig | 1 +
drivers/net/ethernet/Makefile | 1 +
drivers/net/ethernet/yunsilicon/Kconfig | 26 +
drivers/net/ethernet/yunsilicon/Makefile | 8 +
.../ethernet/yunsilicon/xsc/common/xsc_cmd.h | 1025 +++++++++++++++++
.../yunsilicon/xsc/common/xsc_cmd_api.h | 25 +
.../ethernet/yunsilicon/xsc/common/xsc_cmdq.h | 191 +++
.../ethernet/yunsilicon/xsc/common/xsc_core.h | 662 +++++++++++
.../yunsilicon/xsc/common/xsc_device.h | 94 ++
.../ethernet/yunsilicon/xsc/common/xsc_hsi.h | 123 ++
.../ethernet/yunsilicon/xsc/common/xsc_reg.h | 28 +
.../net/ethernet/yunsilicon/xsc/pci/Kconfig | 15 +
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 9 +
.../net/ethernet/yunsilicon/xsc/pci/main.c | 255 ++++
14 files changed, 2463 insertions(+)
create mode 100644 drivers/net/ethernet/yunsilicon/Kconfig
create mode 100644 drivers/net/ethernet/yunsilicon/Makefile
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/main.c
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 8581ccba1..3f357b773 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -70,6 +70,7 @@ source "drivers/net/ethernet/i825xx/Kconfig"
source "drivers/net/ethernet/ibm/Kconfig"
source "drivers/net/ethernet/intel/Kconfig"
source "drivers/net/ethernet/xscale/Kconfig"
+source "drivers/net/ethernet/yunsilicon/Kconfig"
config JME
tristate "JMicron(R) PCI-Express Gigabit Ethernet support"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index 2b1153d35..4ad89b36b 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -50,6 +50,7 @@ obj-$(CONFIG_NET_VENDOR_INTEL) += intel/
obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/
obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/
obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/
+obj-$(CONFIG_NET_VENDOR_YUNSILICON) += yunsilicon/
obj-$(CONFIG_JME) += jme.o
obj-$(CONFIG_KORINA) += korina.o
obj-$(CONFIG_LANTIQ_ETOP) += lantiq_etop.o
diff --git a/drivers/net/ethernet/yunsilicon/Kconfig b/drivers/net/ethernet/yunsilicon/Kconfig
new file mode 100644
index 000000000..ffdfdf4bf
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/Kconfig
@@ -0,0 +1,26 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+#
+# Yunsilicon driver configuration
+#
+
+config NET_VENDOR_YUNSILICON
+ bool "Yunsilicon devices"
+ default y
+ depends on PCI
+ depends on 64BIT || COMPILE_TEST
+ help
+ If you have a network (Ethernet) device belonging to this class,
+ say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Yunsilicon cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_YUNSILICON
+
+source "drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig"
+
+endif # NET_VENDOR_YUNSILICON
diff --git a/drivers/net/ethernet/yunsilicon/Makefile b/drivers/net/ethernet/yunsilicon/Makefile
new file mode 100644
index 000000000..b2924ccaf
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+#
+# Makefile for the Yunsilicon device drivers.
+#
+
+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc/pci/
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
new file mode 100644
index 000000000..70dbf9782
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
@@ -0,0 +1,1025 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CMD_H
+#define __XSC_CMD_H
+
+#include <linux/if_ether.h>
+
+#define XSC_CMDQ_VERSION 0x32
+
+#define XSC_BOARD_SN_LEN 32
+#define XSC_DSCP_PCP_UNSET 255
+#define XSC_MAX_NUM_PCIE_INTF 2
+#define XSC_MAX_PF_NUM_PER_PCIE 8
+
+/* hca_cap mbox version */
+#define XSC_CMD_QUERY_HCA_CAP_V1 1
+#define XSC_CMD_QUERY_HCA_CAP_V2 2
+#define XSC_CMD_QUERY_HCA_CAP_V3 3
+#define XSC_CMD_QUERY_HCA_CAP_V4 4
+#define XSC_CMD_QUERY_HCA_CAP_V5 5
+#define XSC_CMD_QUERY_HCA_CAP_V6 6
+#define XSC_CMD_QUERY_HCA_CAP_V7 7
+#define XSC_CMD_QUERY_HCA_CAP_V8 8
+#define XSC_CMD_QUERY_HCA_CAP_V9 9
+
+#define XSC_CMD_SET_CQ_CONTEXT_V1 1
+
+enum {
+ XSC_CMDQ_GET_HW_STATS_RDMA_V0 = 0,
+ XSC_CMDQ_GET_HW_STATS_RDMA_V1 = 1,
+
+ XSC_CMDQ_GET_HW_STATS_RDMA_UNKNOW,
+};
+
+enum {
+ XSC_CMD_STAT_OK = 0x0,
+ XSC_CMD_STAT_INT_ERR = 0x1,
+ XSC_CMD_STAT_BAD_OP_ERR = 0x2,
+ XSC_CMD_STAT_BAD_PARAM_ERR = 0x3,
+ XSC_CMD_STAT_BAD_SYS_STATE_ERR = 0x4,
+ XSC_CMD_STAT_BAD_RES_ERR = 0x5,
+ XSC_CMD_STAT_RES_BUSY = 0x6,
+ XSC_CMD_STAT_LIM_ERR = 0x8,
+ XSC_CMD_STAT_BAD_RES_STATE_ERR = 0x9,
+ XSC_CMD_STAT_IX_ERR = 0xa,
+ XSC_CMD_STAT_NO_RES_ERR = 0xf,
+ XSC_CMD_STAT_BAD_QP_STATE_ERR = 0x10,
+ XSC_CMD_STAT_BAD_PKT_ERR = 0x30,
+ XSC_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
+ XSC_CMD_STAT_BAD_INP_LEN_ERR = 0x50,
+ XSC_CMD_STAT_BAD_OUTP_LEN_ERR = 0x51,
+};
+
+/* status aliases used by the RDMA driver (same protocol values) */
+#define XSC_CMD_STATUS_OK XSC_CMD_STAT_OK
+#define XSC_CMD_STATUS_FAIL XSC_CMD_STAT_INT_ERR
+#define XSC_CMD_STATUS_NOT_SUPPORTED XSC_CMD_STAT_BAD_OP_ERR
+#define XSC_CMD_STATUS_BAD_PARAM XSC_CMD_STAT_BAD_PARAM_ERR
+#define XSC_CMD_STATUS_INVAL_RES XSC_CMD_STAT_BAD_RES_ERR
+#define XSC_CMD_STATUS_BUSY XSC_CMD_STAT_RES_BUSY
+#define XSC_CMD_STATUS_INVAL_DATA XSC_CMD_STAT_LIM_ERR
+#define XSC_CMD_STATUS_NOT_FOUND XSC_CMD_STAT_IX_ERR
+#define XSC_CMD_STATUS_NO_RES XSC_CMD_STAT_NO_RES_ERR
+#define XSC_CMD_STATUS_BAD_INBUF XSC_CMD_STAT_BAD_INP_LEN_ERR
+#define XSC_CMD_STATUS_BAD_OUTBUF XSC_CMD_STAT_BAD_OUTP_LEN_ERR
+
+/* extended status codes */
+#define XSC_CMD_STATUS_INVAL_FUNC 0x41
+#define XSC_CMD_STATUS_NO_MPT_RES 0x42
+#define XSC_CMD_STATUS_NO_MTT_RES 0x43
+#define XSC_CMD_STATUS_NO_EQN_RES 0x44
+#define XSC_CMD_STATUS_NO_EQ_PA_RES 0x45
+#define XSC_CMD_STATUS_NO_CQN_RES 0x46
+#define XSC_CMD_STATUS_NO_CQ_PA_RES 0x47
+#define XSC_CMD_STATUS_NO_QPN_RES 0x48
+#define XSC_CMD_STATUS_NO_QP_PA_RES 0x49
+#define XSC_CMD_STATUS_NO_PDN_RES 0x4a
+#define XSC_CMD_STATUS_QP_FLUSH_BUSY 0x4b
+#define XSC_CMD_STATUS_QP_FLUSH_PENDING 0x4c
+#define XSC_CMD_STATUS_INVAL_OPCODE 0x52
+
+enum {
+ XSC_CMD_OP_QUERY_HCA_CAP = 0x100,
+ XSC_CMD_OP_QUERY_CMDQ_VERSION = 0x10a,
+ XSC_CMD_OP_FUNCTION_RESET = 0x10c,
+ XSC_CMD_OP_DUMMY = 0x10d,
+ XSC_CMD_OP_QUERY_GUID = 0x113,
+ XSC_CMD_OP_ACTIVATE_HW_CONFIG = 0x114,
+
+ XSC_CMD_OP_QUERY_HW_STATS = 0x1200,
+ XSC_CMD_OP_QUERY_HW_STATS_RDMA = 0x1203,
+ XSC_CMD_OP_QUERY_HW_GLOBAL_STATS = 0x1210,
+
+ XSC_CMD_OP_CREATE_MKEY = 0x200,
+ XSC_CMD_OP_DESTROY_MKEY = 0x202,
+ XSC_CMD_OP_REG_MR = 0x204,
+ XSC_CMD_OP_DEREG_MR = 0x205,
+ XSC_CMD_OP_SET_MPT = 0x206,
+ XSC_CMD_OP_SET_MTT = 0x207,
+
+ XSC_CMD_OP_CREATE_EQ = 0x301,
+ XSC_CMD_OP_DESTROY_EQ = 0x302,
+
+ XSC_CMD_OP_CREATE_CQ = 0x400,
+ XSC_CMD_OP_DESTROY_CQ = 0x401,
+ XSC_CMD_OP_SET_CQ_CONTEXT = 0x406,
+ XSC_CMD_OP_SET_CQ_BUF_PA = 0x407,
+ XSC_CMD_OP_CREATE_CQ_EX = 0x408,
+
+ XSC_CMD_OP_CREATE_QP = 0x500,
+ XSC_CMD_OP_DESTROY_QP = 0x501,
+ XSC_CMD_OP_RST2INIT_QP = 0x502,
+ XSC_CMD_OP_INIT2RTR_QP = 0x503,
+ XSC_CMD_OP_RTR2RTS_QP = 0x504,
+ XSC_CMD_OP_RTS2RTS_QP = 0x505,
+ XSC_CMD_OP_SQERR2RTS_QP = 0x506,
+ XSC_CMD_OP_2ERR_QP = 0x507,
+ XSC_CMD_OP_RTS2SQD_QP = 0x508,
+ XSC_CMD_OP_SQD2RTS_QP = 0x509,
+ XSC_CMD_OP_2RST_QP = 0x50a,
+ XSC_CMD_OP_QUERY_QP = 0x50b,
+ XSC_CMD_OP_INIT2INIT_QP = 0x50e,
+ XSC_CMD_OP_SQD2SQD_QP = 0x511,
+ XSC_CMD_OP_CREATE_MULTI_QP = 0x515,
+
+ XSC_CMD_OP_ALLOC_PD = 0x800,
+ XSC_CMD_OP_DEALLOC_PD = 0x801,
+
+ XSC_CMD_OP_MODIFY_RAW_QP = 0x81f,
+
+ XSC_CMD_OP_ENABLE_NIC_HCA = 0x810,
+ XSC_CMD_OP_DISABLE_NIC_HCA = 0x811,
+
+ XSC_CMD_OP_QUERY_VPORT_STATE = 0x822,
+ XSC_CMD_OP_QUERY_LINK_INFO = 0x832,
+ XSC_CMD_OP_MODIFY_LINK_INFO = 0x834,
+ XSC_CMD_OP_QUERY_EVENT_TYPE = 0x831,
+
+ XSC_CMD_OP_ENABLE_MSIX = 0x850,
+
+ XSC_CMD_OP_SET_MTU = 0x1100,
+ XSC_CMD_OP_QUERY_ETH_MAC = 0X1101,
+
+ XSC_CMD_OP_SET_PORT_ADMIN_STATUS = 0x1801,
+
+ XSC_CMD_OP_MAX
+};
+
+enum xsc_dma_direct {
+ XSC_DMA_DIR_TO_MAC,
+ XSC_DMA_DIR_READ,
+ XSC_DMA_DIR_WRITE,
+ XSC_DMA_DIR_LOOPBACK,
+ XSC_DMA_DIR_MAX
+};
+
+/* hw feature bitmap, 32bit */
+enum xsc_hw_feature_flag {
+ XSC_HW_RDMA_SUPPORT = BIT(0),
+ XSC_HW_PFC_PRIO_STATISTIC_SUPPORT = BIT(1),
+ XSC_HW_PFC_STALL_STATS_SUPPORT = BIT(3),
+ XSC_HW_RDMA_CM_SUPPORT = BIT(5),
+ XSC_HW_MTU_8192_SUPPORT = BIT(11),
+ XSC_HW_RDMA_ATOMIC_SUPPORT = BIT(17),
+ XSC_HW_MULTIDOORBELL_SUPPORT = BIT(18),
+ XSC_HW_CQE64_SUPPORT = BIT(21),
+
+ XSC_HW_LAST_FEATURE = BIT(31)
+};
+
+struct xsc_inbox_hdr {
+ __be16 opcode;
+ u8 rsvd[4];
+ __be16 ver; /* cmd version */
+};
+
+struct xsc_outbox_hdr {
+ u8 status;
+ u8 rsvd[5];
+ __be16 ver;
+};
+
+/*CQ mbox*/
+struct xsc_cq_context {
+ __be16 eqn; /* event queue number */
+ __be16 pa_num; /* physical address count in the ctx */
+ __be16 glb_func_id;
+ u8 log_cq_sz;
+ u8 cq_type;
+};
+
+struct xsc_cq_context_ex {
+ struct xsc_cq_context ctx;
+ u8 page_shift;
+ u8 overrun_ignore;
+ u8 rsvd[2];
+ __be32 pa_num_ex;
+};
+
+struct xsc_create_cq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context ctx;
+ __be64 pas[]; /* physical address list */
+};
+
+struct xsc_create_cq_ex_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context_ex ctx_ex;
+ __be64 pas[];
+};
+
+struct xsc_create_cq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 cqn; /* completion queue number */
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_cq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 cqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_cq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_set_cq_context_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context_ex ctx_ex;
+};
+
+struct xsc_set_cq_context_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 cqn;
+ __be32 cq_pa_list_base;
+};
+
+struct xsc_set_cq_buf_pa_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 pa_list_start;
+ __be32 pa_num;
+ __be64 pas[];
+};
+
+struct xsc_set_cq_buf_pa_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MKey / MPT / MTT mbox */
+struct xsc_create_mkey_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[4];
+};
+
+struct xsc_create_mkey_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 mkey;
+};
+
+struct xsc_destroy_mkey_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 mkey;
+};
+
+struct xsc_destroy_mkey_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd;
+};
+
+struct xsc_mpt_item {
+ __be32 pdn;
+ __be32 pa_num;
+ __be32 len;
+ __be32 mkey;
+ __be32 len_h;
+ u8 is_gpu;
+ u8 acc;
+ u8 page_mode;
+ u8 map_en;
+ __be64 va_base;
+};
+
+struct xsc_set_mpt_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_mpt_item mpt_item;
+};
+
+struct xsc_set_mpt_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 mtt_base;
+ u8 rsvd[4];
+};
+
+struct xsc_mtt_setting {
+ __be32 mtt_base;
+ __be32 pa_num;
+ __be64 pas[];
+};
+
+struct xsc_set_mtt_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_mtt_setting mtt_setting;
+};
+
+struct xsc_set_mtt_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/*QP mbox*/
+enum {
+ XSC_MODULE_SPEED_UNKNOWN,
+ XSC_MODULE_SPEED_10G,
+ XSC_MODULE_SPEED_25G,
+ XSC_MODULE_SPEED_40G_R4,
+ XSC_MODULE_SPEED_50G_R,
+ XSC_MODULE_SPEED_50G_R2,
+ XSC_MODULE_SPEED_100G_R2,
+ XSC_MODULE_SPEED_100G_R4,
+ XSC_MODULE_SPEED_200G_R4,
+ XSC_MODULE_SPEED_200G_R8,
+ XSC_MODULE_SPEED_400G_R8,
+ XSC_MODULE_SPEED_400G_R4,
+};
+
+struct __packed xsc_create_qp_request {
+ __be16 input_qpn;
+ __be16 pa_num;
+ u8 qp_type;
+ u8 log_sq_sz;
+ u8 log_rq_sz;
+ u8 dma_direct;
+ __be32 pdn; /* protect domain number */
+ __be16 cqn_send;
+ __be16 cqn_recv;
+ __be16 glb_funcid;
+ /*rsvd, the old logic_port */
+ u8 page_shift;
+ u8 rsvd[5];
+ __be64 pas[];
+};
+
+struct xsc_create_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_create_qp_request req;
+};
+
+struct xsc_create_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 qpn; /* queue pair number */
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MR mbox */
+struct xsc_register_mr_request {
+ __be32 pdn;
+ __be32 pa_num;
+ __be64 len;
+ __be32 mkey;
+ u8 is_gpu;
+ u8 acc;
+ u8 page_mode;
+ u8 map_en;
+ __be64 va_base;
+ __be64 pas[];
+};
+
+struct xsc_register_mr_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_register_mr_request req;
+};
+
+struct xsc_unregister_mr_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 mkey;
+ u8 rsvd[4];
+};
+
+struct xsc_unregister_mr_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+enum qp_access_flag {
+ QP_ACCESS_REMOTE_READ = (1 << 0),
+ QP_ACCESS_REMOTE_WRITE = (1 << 1),
+ QP_ACCESS_REMOTE_ATOMIC = (1 << 2),
+ QP_ACCESS_MODIFY_QP_TYPE = (1 << 3),
+};
+
+#define XSC_QP_CONTEXT_V1 1
+#define XSC_QP_CONTEXT_V2 2
+
+struct xsc_qp_context {
+ __be32 remote_qpn;
+ __be32 cqn_send;
+ __be32 cqn_recv;
+ __be32 next_send_psn;
+ __be32 next_recv_psn;
+ __be32 pdn;
+ __be16 src_udp_port;
+ __be16 path_id;
+ u8 mtu_mode;
+ u8 lag_sel;
+ u8 lag_sel_en;
+ u8 retry_cnt;
+ u8 rnr_retry;
+ u8 dscp;
+ u8 state;
+ u8 hop_limit;
+ u8 dmac[6];
+ u8 smac[6];
+ __be32 dip[4];
+ __be32 sip[4];
+ __be16 ip_type;
+ __be16 grp_id;
+ u8 vlan_valid;
+ u8 dci_cfi_prio_sl;
+ __be16 vlan_id;
+ u8 qp_out_port;
+ u8 pcie_no;
+ __be16 lag_id;
+ __be16 func_id;
+ u8 max_sra;
+ u8 max_rra;
+ u8 no_need_wait;
+ u8 swith_qp_type;
+ u8 rsvd0[2];
+ __be32 qp_access_flags;
+};
+
+struct xsc_modify_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ struct xsc_qp_context ctx;
+ u8 no_need_wait;
+};
+
+struct xsc_modify_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_query_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ u8 rsvd[4];
+};
+
+struct xsc_query_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_qp_context ctx;
+};
+
+struct xsc_create_multiqp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 qp_num;
+ u8 qp_type;
+ u8 rsvd;
+ __be32 req_len;
+ u8 data[];
+};
+
+struct xsc_create_multiqp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 qpn_base;
+};
+
+/* PD mbox */
+struct xsc_alloc_pd_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_alloc_pd_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 pdn;
+ u8 rsvd[4];
+};
+
+struct xsc_dealloc_pd_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 pdn;
+ u8 rsvd[4];
+};
+
+struct xsc_dealloc_pd_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MSIX TABLE mbox */
+struct xsc_msix_table_info_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 index;
+ u8 rsvd[6];
+};
+
+struct xsc_msix_table_info_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 addr_lo;
+ __be32 addr_hi;
+ __be32 data;
+};
+
+/*EQ mbox*/
+struct xsc_eq_context {
+ __be16 vecidx;
+ __be16 pa_num;
+ u8 log_eq_sz;
+ u8 rsvd1;
+ __be16 glb_func_id;
+ u8 is_async_eq;
+ u8 rsvd;
+};
+
+struct xsc_create_eq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_eq_context ctx;
+ __be64 pas[];
+};
+
+struct xsc_create_eq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 eqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_eq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 eqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_eq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_cmdq_ver_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_cmdq_ver_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be16 cmdq_ver;
+ u8 rsvd[6];
+};
+
+struct xsc_fw_version {
+ u8 fw_version_major;
+ u8 fw_version_minor;
+ __be16 fw_version_patch;
+ __be32 fw_version_tweak;
+ u8 fw_version_extra_flag;
+ u8 rsvd[7];
+};
+
+struct __packed xsc_hca_cap {
+ u8 rsvd1[12];
+ u8 send_seg_num;
+ u8 send_wqe_shift;
+ u8 recv_seg_num;
+ u8 recv_wqe_shift;
+ u8 log_max_srq_sz;
+ u8 log_max_qp_sz;
+ u8 log_max_mtt;
+ u8 log_max_qp;
+ u8 log_max_strq_sz;
+ u8 log_max_srqs;
+ u8 rsvd4[2];
+ u8 log_max_tso;
+ u8 log_max_cq_sz;
+ u8 rsvd6;
+ u8 log_max_cq;
+ u8 log_max_eq_sz;
+ u8 log_max_mkey;
+ u8 log_max_msix;
+ u8 log_max_eq;
+ u8 max_indirection;
+ u8 log_max_mrw_sz;
+ u8 log_max_bsf_list_sz;
+ u8 log_max_klm_list_sz;
+ u8 rsvd_8_0;
+ u8 log_max_ra_req_dc;
+ u8 rsvd_8_1;
+ u8 log_max_ra_res_dc;
+ u8 rsvd9;
+ u8 log_max_ra_req_qp;
+ u8 log_max_qp_depth;
+ u8 log_max_ra_res_qp;
+ __be16 max_vfs;
+ __be16 raweth_qp_id_end;
+ __be16 raw_tpe_qp_num;
+ __be16 max_qp_count;
+ __be16 raweth_qp_id_base;
+ u8 rsvd13;
+ u8 local_ca_ack_delay;
+ u8 max_num_eqs;
+ u8 num_ports;
+ u8 log_max_msg;
+ u8 mac_port;
+ __be16 raweth_rss_qp_id_base;
+ __be16 stat_rate_support;
+ u8 rsvd16[8];
+ __be64 flags;
+ u8 rsvd17;
+ u8 uar_sz;
+ u8 rsvd18;
+ u8 log_pg_sz;
+ __be16 bf_log_bf_reg_size;
+ __be16 msix_base;
+ __be16 msix_num;
+ __be16 max_desc_sz_sq;
+ u8 rsvd20[2];
+ __be16 max_desc_sz_rq;
+ u8 rsvd21[2];
+ __be16 max_desc_sz_sq_dc;
+ u8 rsvd22[4];
+ __be16 max_qp_mcg;
+ u8 rsvd23;
+ u8 log_max_mcg;
+ u8 rsvd24;
+ u8 log_max_pd;
+ u8 rsvd25;
+ u8 log_max_xrcd;
+ u8 rsvd26[40];
+ __be32 uar_page_sz;
+ u8 rsvd27[8];
+ __be32 hw_feature_flag; /* enum xsc_hw_feature_flag */
+ __be16 pf0_vf_funcid_base;
+ __be16 pf0_vf_funcid_top;
+ __be16 pf1_vf_funcid_base;
+ __be16 pf1_vf_funcid_top;
+ __be16 pcie0_pf_funcid_base;
+ __be16 pcie0_pf_funcid_top;
+ __be16 pcie1_pf_funcid_base;
+ __be16 pcie1_pf_funcid_top;
+ u8 log_msx_atomic_size_qp;
+ u8 pcie_host;
+ u8 rsvd28;
+ u8 log_msx_atomic_size_dc;
+ u8 board_sn[XSC_BOARD_SN_LEN];
+ u8 max_tc;
+ u8 mac_bit;
+ __be16 funcid_to_logic_port;
+ u8 rsvd29[6];
+ u8 nif_port_num;
+ u8 reg_mr_via_cmdq;
+ __be32 hca_core_clock;
+ __be32 max_rwq_indirection_tables; /* rss_caps */
+ __be32 max_rwq_indirection_table_size; /* rss_caps */
+ __be32 chip_ver_h;
+ __be32 chip_ver_m;
+ __be32 chip_ver_l;
+ __be32 hotfix_num;
+ __be32 feature_flag;
+ __be32 rx_pkt_len_max;
+ __be32 glb_func_id;
+ __be64 tx_db;
+ __be64 rx_db;
+ __be64 complete_db;
+ __be64 complete_reg;
+ __be64 event_db;
+ __be32 qp_rate_limit_min;
+ __be32 qp_rate_limit_max;
+ struct xsc_fw_version fw_ver;
+ u8 lag_logic_port_ofst;
+ u8 rsvd32[7];
+ /* V1 */
+ __be64 max_mr_size;
+ __be16 max_cmd_in_len;
+ __be16 max_cmd_out_len;
+ /* V2 */
+ __be32 max_qp;
+ __be32 max_cq;
+ __be32 max_pd;
+ __be32 max_mtt;
+ /* V3 */
+ __be32 mpt_tbl_addr;
+ __be32 mpt_tbl_depth;
+ __be32 mpt_tbl_width;
+ __be32 mtt_inst_base_addr;
+ __be32 mtt_inst_stride;
+ __be32 mtt_inst_num_log;
+ __be32 mtt_inst_depth;
+ /* V4 */
+ __be16 vf_funcid_base[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];
+ __be16 vf_funcid_top[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];
+ __be16 pf_funcid_base[XSC_MAX_NUM_PCIE_INTF];
+ __be16 pf_funcid_top[XSC_MAX_NUM_PCIE_INTF];
+ u8 pcie_no;
+ u8 pf_id;
+ __be16 vf_id;
+ u8 pcie_host_num;
+ u8 pf_num_per_pcie;
+ /* V6 */
+ __be16 ft_support;
+ /* V7 */
+ __be16 total_dynamic_vf_msix;
+ __be16 min_dynamic_vf_msix;
+ __be16 max_dynamic_vf_msix;
+ /* V8 */
+ u8 max_sgl_rd;
+ u8 rsvd30;
+ /* V9 */
+ __be32 mdb_num;
+ u8 rsvd31[4];
+ __be64 mdb_base;
+};
+
+struct xsc_cmd_query_hca_cap_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 cpu_num;
+ u8 rsvd[6];
+};
+
+struct xsc_cmd_query_hca_cap_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[8];
+ struct xsc_hca_cap hca_cap;
+};
+
+struct xsc_query_vport_state_out {
+ struct xsc_outbox_hdr hdr;
+ u8 admin_state:4;
+ u8 state:4;
+};
+
+struct xsc_query_vport_state_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 data0;
+#define XSC_QUERY_VPORT_OTHER_VPORT BIT(0)
+#define XSC_QUERY_VPORT_VPORT_NUM_MASK GENMASK(16, 1)
+};
+
+enum {
+ XSC_CMD_EVENT_RESP_CHANGE_LINK = BIT(0),
+ XSC_CMD_EVENT_RESP_TEMP_WARN = BIT(1),
+ XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION = BIT(2),
+};
+
+struct xsc_event_resp {
+ u8 resp_cmd_type;
+};
+
+struct xsc_event_query_type_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[2];
+};
+
+struct xsc_event_query_type_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_event_resp ctx;
+};
+
+struct xsc_modify_raw_qp_request {
+ __be16 qpn;
+ __be16 lag_id;
+ __be16 func_id;
+ u8 dma_direct;
+ u8 prio;
+ u8 qp_out_port;
+ u8 rsvd[7];
+};
+
+struct xsc_modify_raw_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 pcie_no;
+ u8 rsvd[7];
+ struct xsc_modify_raw_qp_request req;
+};
+
+struct xsc_modify_raw_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_set_mtu_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 mtu;
+ __be16 rx_buf_sz_min;
+ u8 mac_port;
+ u8 rsvd;
+};
+
+struct xsc_set_mtu_mbox_out {
+ struct xsc_outbox_hdr hdr;
+};
+
+struct xsc_query_eth_mac_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 index;
+};
+
+struct xsc_query_eth_mac_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 mac[ETH_ALEN];
+};
+
+enum {
+ XSC_TBM_CAP_HASH_PPH = 0,
+ XSC_TBM_CAP_RSS,
+ XSC_TBM_CAP_PP_BYPASS,
+ XSC_TBM_CAP_PCT_DROP_CONFIG,
+};
+
+struct xsc_nic_attr {
+ __be16 caps;
+ __be16 caps_mask;
+ u8 mac_addr[ETH_ALEN];
+};
+
+struct xsc_rss_attr {
+ u8 rss_en;
+ u8 hfunc;
+ __be16 rqn_base;
+ __be16 rqn_num;
+ u8 rsvd[2];
+ __be32 hash_tmpl;
+};
+
+struct xsc_cmd_enable_nic_hca_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_nic_attr nic;
+ struct xsc_rss_attr rss;
+};
+
+struct xsc_cmd_enable_nic_hca_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[2];
+};
+
+struct xsc_nic_dis_attr {
+ __be16 caps;
+};
+
+struct xsc_cmd_disable_nic_hca_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_nic_dis_attr nic;
+};
+
+struct xsc_cmd_disable_nic_hca_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[4];
+};
+
+struct xsc_function_reset_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 glb_func_id;
+ u8 rsvd[6];
+};
+
+struct xsc_function_reset_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_guid_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_guid_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be64 guid;
+};
+
+struct xsc_cmd_activate_hw_config_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_activate_hw_config_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_event_linkinfo {
+ u8 linkstatus; /*0:down, 1:up*/
+ u8 port;
+ u8 duplex;
+ u8 autoneg;
+ __be32 linkspeed;
+ __be64 supported;
+ __be64 advertising;
+ __be64 supported_fec; /* reserved, not support currently */
+ __be64 advertised_fec; /* reserved, not support currently */
+ __be64 supported_speed[2];
+ __be64 advertising_speed[2];
+};
+
+struct xsc_event_query_linkinfo_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_event_linkinfo ctx;
+};
+
+struct xsc_event_set_port_admin_status_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 status;
+};
+
+struct xsc_event_set_port_admin_status_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 status;
+};
+
+/* HW stats mbox */
+struct xsc_hw_stats_rdma_pf {
+ /*by mac port*/
+__be64 rdma_tx_pkts;
+__be64 rdma_tx_bytes;
+__be64 rdma_rx_pkts;
+__be64 rdma_rx_bytes;
+__be64 np_cnp_sent;
+__be64 rp_cnp_handled;
+__be64 np_ecn_marked_roce_packets;
+__be64 rp_cnp_ignored;
+__be64 read_rsp_out_of_seq;
+__be64 implied_nak_seq_err;
+ /*by function*/
+__be64 out_of_sequence;
+__be64 packet_seq_err;
+__be64 out_of_buffer;
+__be64 rnr_nak_retry_err;
+__be64 local_ack_timeout_err;
+__be64 rx_read_requests;
+__be64 rx_write_requests;
+__be64 duplicate_requests;
+__be64 rdma_tx_pkts_func;
+__be64 rdma_tx_payload_bytes;
+__be64 rdma_rx_pkts_func;
+__be64 rdma_rx_payload_bytes;
+ /*global*/
+__be64 rdma_loopback_pkts;
+__be64 rdma_loopback_bytes;
+ /*for diamond*/
+__be64 out_of_sequence_sr;
+__be64 packet_seq_err_sr;
+__be64 rdma_ndp_rx_pkts;
+__be64 rdma_ndp_rx_trimmed_pkts;
+__be64 rdma_ndp_trimmed_pkts_sr;
+__be64 rx_icrc_encapsulated;
+};
+
+struct xsc_hw_stats_rdma_vf {
+ /*by function*/
+__be64 rdma_tx_pkts_func;
+__be64 rdma_tx_payload_bytes;
+__be64 rdma_rx_pkts_func;
+__be64 rdma_rx_payload_bytes;
+
+__be64 out_of_sequence;
+__be64 packet_seq_err;
+__be64 out_of_buffer;
+__be64 rnr_nak_retry_err;
+__be64 local_ack_timeout_err;
+__be64 rx_read_requests;
+__be64 rx_write_requests;
+__be64 duplicate_requests;
+};
+
+struct xsc_hw_stats_rdma {
+ u8 is_pf;
+ u8 rsv;
+ __be16 ver;
+ union {
+ struct xsc_hw_stats_rdma_pf pf_stats;
+ struct xsc_hw_stats_rdma_vf vf_stats;
+ } stats;
+};
+
+#define XSC_CMD_QUERY_HW_STATS_V1 1
+struct xsc_hw_stats_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 mac_port;
+ u8 is_lag;
+ u8 lag_member_num;
+ u8 member_port[];
+};
+
+struct xsc_hw_stats_rdma_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_hw_stats_rdma hw_stats;
+};
+
+struct xsc_hw_global_stats_rdma {
+ /*by global*/
+__be64 rdma_loopback_pkts;
+__be64 rdma_loopback_bytes;
+__be64 rx_icrc_encapsulated;
+__be64 req_cqe_error;
+__be64 resp_cqe_error;
+__be64 cqe_msg_code_error;
+};
+
+struct xsc_hw_global_stats_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsv[4];
+};
+
+struct xsc_hw_global_stats_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_hw_global_stats_rdma hw_stats;
+};
+
+#endif /* __XSC_CMD_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h
new file mode 100644
index 000000000..2ea02dca2
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CMD_API_H
+#define __XSC_CMD_API_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+
+int xsc_cmd_init(struct xsc_core_device *xdev);
+void xsc_cmd_cleanup(struct xsc_core_device *xdev);
+void xsc_cmd_use_events(struct xsc_core_device *xdev);
+void xsc_cmd_use_polling(struct xsc_core_device *xdev);
+int xsc_cmd_err_handler(struct xsc_core_device *xdev);
+void xsc_cmd_resp_handler(struct xsc_core_device *xdev);
+int xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size);
+int xsc_cmd_version_check(struct xsc_core_device *xdev);
+
+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
new file mode 100644
index 000000000..c163bd179
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
@@ -0,0 +1,191 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CMDQ_H
+#define __XSC_CMDQ_H
+
+#include <linux/if_ether.h>
+#include <linux/semaphore.h>
+
+#include "common/xsc_cmd.h"
+
+#define XSC_CMD_MAX_RETRY_CNT 3
+
+enum {
+ XSC_CMD_TIMEOUT_MSEC = 10 * 1000,
+ XSC_CMD_WQ_MAX_NAME = 32,
+};
+
+enum {
+ XSC_CMD_DATA, /* print command payload only */
+ XSC_CMD_TIME, /* print command execution time */
+};
+
+enum {
+ XSC_MAX_COMMANDS = 32,
+ XSC_CMD_DATA_BLOCK_SIZE = 512,
+ XSC_PCI_CMD_XPORT = 7,
+};
+
+struct xsc_cmd_prot_block {
+ u8 data[XSC_CMD_DATA_BLOCK_SIZE];
+ u8 rsvd0[48];
+ __be64 next;
+ __be32 block_num;
+ /* init to 0, dma user should change this val to 1 */
+ u8 owner_status;
+ u8 token;
+ u8 ctrl_sig;
+ u8 sig;
+};
+
+struct cache_ent {
+ /* protect block chain allocations */
+ spinlock_t lock;
+ struct list_head head;
+};
+
+struct cmd_msg_cache {
+ struct cache_ent large;
+ struct cache_ent med;
+
+};
+
+#define XSC_CMD_FIRST_SIZE 8
+struct xsc_cmd_first {
+ __be32 data[XSC_CMD_FIRST_SIZE];
+};
+
+struct xsc_cmd_mailbox {
+ void *buf;
+ dma_addr_t dma;
+ struct xsc_cmd_mailbox *next;
+};
+
+struct xsc_cmd_msg {
+ struct list_head list;
+ struct cache_ent *cache;
+ u32 len;
+ struct xsc_cmd_first first;
+ struct xsc_cmd_mailbox *next;
+};
+
+#define XSC_RSP_FIRST_SIZE 14
+struct xsc_rsp_first {
+ /* can be larger, xsc_rsp_layout */
+ __be32 data[XSC_RSP_FIRST_SIZE];
+};
+
+struct xsc_rsp_msg {
+ struct list_head list;
+ struct cache_ent *cache;
+ u32 len;
+ struct xsc_rsp_first first;
+ struct xsc_cmd_mailbox *next;
+};
+
+typedef void (*xsc_cmd_cbk_t)(int status, void *context);
+
+/* hw will use this for some records(e.g. vf_id) */
+struct cmdq_rsv {
+ u16 vf_id;
+ u8 rsv[2];
+};
+
+/* related with hw, won't change */
+
+struct xsc_cmd_layout {
+ struct cmdq_rsv rsv0;
+ __be32 inlen;
+ __be64 in_ptr;
+ __be32 in[XSC_CMD_FIRST_SIZE];
+ __be64 out_ptr;
+ __be32 outlen;
+ u8 token;
+ u8 sig;
+ u8 idx;
+ u8 type_owner;
+};
+
+struct xsc_rsp_layout {
+ struct cmdq_rsv rsv0;
+ __be32 out[XSC_RSP_FIRST_SIZE];
+ u8 token;
+ u8 sig;
+ u8 idx;
+ u8 type_owner;
+};
+#define XSC_CMD_TYPE_MASK GENMASK(6, 0)
+#define XSC_CMD_OWNER_BIT BIT(7)
+
+struct xsc_cmd_work_ent {
+ struct xsc_cmd_msg *in;
+ struct xsc_rsp_msg *out;
+ int idx;
+ struct completion done;
+ struct xsc_cmd *cmd;
+ struct work_struct work;
+ struct xsc_cmd_layout *lay;
+ struct xsc_rsp_layout *rsp_lay;
+ int ret;
+ u8 status;
+ u8 token;
+};
+
+enum xsc_cmd_status {
+ XSC_CMD_STATUS_NORMAL,
+ XSC_CMD_STATUS_TIMEDOUT,
+};
+
+enum xsc_cmd_mode {
+ XSC_CMD_MODE_POLLING,
+ XSC_CMD_MODE_EVENTS,
+};
+
+struct xsc_cmd {
+ void *cmd_buf;
+ void *cq_buf;
+ dma_addr_t dma;
+ dma_addr_t cq_dma;
+ u16 cmd_pid;
+ u16 cq_cid;
+ u8 owner_bit;
+ u8 retry_cnt;
+ u8 cmdif_rev;
+ u8 log_sz;
+ u8 log_stride;
+ int max_reg_cmds;
+ int events;
+ u32 __iomem *vector;
+
+ /* protect command queue allocations */
+ spinlock_t alloc_lock;
+ /* protect token allocations */
+ spinlock_t token_lock;
+ /* protect cmdq req pid doorbell */
+ spinlock_t doorbell_lock;
+ u8 token;
+ unsigned long cmd_entry_mask;
+ char wq_name[XSC_CMD_WQ_MAX_NAME];
+ struct workqueue_struct *wq;
+ struct task_struct *cq_task;
+ /* The semaphore limits the number of working commands
+ * to max_reg_cmds. Each exec_cmd call does a down
+ * to acquire the semaphore before execution and an up
+ * after completion, ensuring no more than max_reg_cmds
+ * commands run at the same time.
+ */
+ struct semaphore sem;
+ enum xsc_cmd_mode mode;
+ struct xsc_cmd_work_ent *ent_arr[XSC_MAX_COMMANDS];
+ struct dma_pool *pool;
+ struct cmd_msg_cache cache;
+ int checksum_disabled;
+ unsigned int irqn;
+ u8 ownerbit_learned;
+ u8 cmd_status;
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
new file mode 100644
index 000000000..9775412d2
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
@@ -0,0 +1,662 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CORE_H
+#define __XSC_CORE_H
+
+#include <linux/pci.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/bitfield.h>
+#include <linux/if_vlan.h>
+#include <linux/radix-tree.h>
+#include <linux/reboot.h>
+
+#include "common/xsc_reg.h"
+#include "common/xsc_cmdq.h"
+
+#define XSC_PCI_VENDOR_ID 0x1f67
+
+#define XSC_MC_PF_DEV_ID 0x1011
+
+#define XSC_MF_HOST_PF_DEV_ID 0x1051
+#define XSC_MF_SOC_PF_DEV_ID 0x1053
+
+#define XSC_MS_PF_DEV_ID 0x1111
+
+#define XSC_MV_HOST_PF_DEV_ID 0x1151
+#define XSC_MV_SOC_PF_DEV_ID 0x1153
+
+#define XSC_PAGE_SHIFT_4K 12
+#define XSC_PAGE_SIZE_4K (_AC(1, UL) << XSC_PAGE_SHIFT_4K)
+
+#define XSC_ORIGIN_PF_BAR_SIZE (256 * 1024 * 1024)
+
+enum xsc_coredev_type {
+ XSC_COREDEV_PF,
+ XSC_COREDEV_SF
+};
+
+enum {
+ XSC_MAX_EQ_NAME = 20
+};
+
+enum {
+ XSC_MAX_IRQ_NAME = 32
+};
+
+enum {
+ XSC_MAX_FW_PORTS = 1,
+};
+
+enum {
+ XSC_BF_REGS_PER_PAGE = 4,
+ XSC_MAX_UAR_PAGES = 1 << 8,
+ XSC_MAX_UUARS = XSC_MAX_UAR_PAGES * XSC_BF_REGS_PER_PAGE,
+};
+
+/* alloc */
+struct xsc_buf_list {
+ void *buf;
+ dma_addr_t map;
+};
+
+struct xsc_buf {
+ struct xsc_buf_list direct;
+ struct xsc_buf_list *page_list;
+ unsigned long nbufs;
+ unsigned long npages;
+ unsigned long page_shift;
+ unsigned long size;
+};
+
+struct xsc_hw_reg {
+ u32 tx_db;
+ u32 rx_db;
+ u32 complete_db;
+ u32 complete_reg;
+ u32 event_db;
+ u32 req_pid_addr;
+ u32 req_cid_addr;
+ u32 rsp_pid_addr;
+ u32 rsp_cid_addr;
+ u32 req_buf_h_addr;
+ u32 req_buf_l_addr;
+ u32 rsp_buf_h_addr;
+ u32 rsp_buf_l_addr;
+ u32 msix_vec_addr;
+ u32 interrupt_stat_addr;
+ u32 mpt_tbl_addr;
+ u32 mpt_tbl_depth;
+ u32 mpt_tbl_width;
+ u32 mtt_inst_base_addr;
+ u32 mtt_inst_stride;
+ u32 mtt_inst_num_log;
+ u32 mtt_inst_depth;
+};
+
+struct xsc_frag_buf {
+ struct xsc_buf_list *frags;
+ unsigned long npages;
+ unsigned long size;
+ u8 page_shift;
+};
+
+struct xsc_frag_buf_ctrl {
+ struct xsc_buf_list *frags;
+ u32 sz_m1;
+ u16 frag_sz_m1;
+ u16 strides_offset;
+ u8 log_sz;
+ u8 log_stride;
+ u8 log_frag_strides;
+};
+
+/* qp */
+struct xsc_send_wqe_ctrl_seg {
+ __le32 data0;
+ __le32 msg_len;
+ __le32 data2;
+ __le32 data3;
+};
+#define XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK GENMASK(7, 0)
+#define XSC_WQE_CTRL_SEG_DATA0_WITH_IMM BIT(8)
+#define XSC_WQE_CTRL_SEG_DATA0_CSUM_EN_MASK GENMASK(10, 9)
+#define XSC_WQE_CTRL_SEG_DATA0_DS_DATA_NUM_MASK GENMASK(15, 11)
+#define XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK GENMASK(31, 16)
+#define XSC_WQE_CTRL_SEG_DATA2_HAS_PPH BIT(0)
+#define XSC_WQE_CTRL_SEG_DATA2_SO_TYPE BIT(1)
+#define XSC_WQE_CTRL_SEG_DATA2_SO_DATA_SIZE_MASK GENMASK(15, 2)
+#define XSC_WQE_CTRL_SEG_DATA2_SO_HDR_LEN_MASK GENMASK(31, 24)
+#define XSC_WQE_CTRL_SEG_DATA3_SE BIT(0)
+#define XSC_WQE_CTRL_SEG_DATA3_CE BIT(1)
+
+struct xsc_wqe_data_seg {
+ union {
+ struct {
+ __le32 data0;
+ __le32 mkey;
+ __le64 va;
+ };
+ struct {
+ u8 in_line_data[16];
+ };
+ };
+};
+#define XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK GENMASK(31, 1)
+#define XSC_INVALID_LKEY 0x100
+
+struct xsc_core_qp {
+ void (*event)(struct xsc_core_qp *qp, int type);
+ u32 qpn;
+ atomic_t refcount;
+ struct completion free;
+ int pid;
+ u16 qp_type;
+ u16 eth_queue_type;
+ u16 qp_type_internal;
+ u16 grp_id;
+ u8 mac_id;
+ u8 err_occurred;
+ u32 qp_protocol_mode;
+ u32 qp_access_flags;
+};
+
+struct xsc_qp_table {
+ spinlock_t lock; /* protect radix tree */
+ struct radix_tree_root tree;
+};
+
+/* cq */
+#define XSC_MAX_NAME_LEN 32
+#define XSC_MAX_IBDEV_NAME_LEN 64
+
+struct xsc_priv {
+ char name[XSC_MAX_NAME_LEN];
+ char ibdev_name[XSC_MAX_IBDEV_NAME_LEN];
+ int cm_pcp;
+ int cm_dscp;
+ int force_pcp;
+ int force_dscp;
+};
+
+enum xsc_dev_event {
+ XSC_DEV_EVENT_SYS_ERROR,
+ XSC_DEV_EVENT_PORT_UP,
+ XSC_DEV_EVENT_PORT_DOWN,
+ XSC_DEV_EVENT_PORT_INITIALIZED,
+ XSC_DEV_EVENT_LID_CHANGE,
+ XSC_DEV_EVENT_PKEY_CHANGE,
+ XSC_DEV_EVENT_GUID_CHANGE,
+ XSC_DEV_EVENT_CLIENT_REREG,
+};
+
+enum {
+ RDMA_PROTO_ROCEV2,
+ RDMA_PROTO_VEROCE,
+};
+
+#define VEROCE_UDP_DPORT 4794
+#define MAX_MR_CACHE_ENTRIES 5000
+
+enum xsc_event {
+ XSC_EVENT_TYPE_COMP = 0x0,
+ /* mad */
+ XSC_EVENT_TYPE_COMM_EST = 0x02,
+ XSC_EVENT_TYPE_CQ_ERROR = 0x04,
+ XSC_EVENT_TYPE_WQ_CATAS_ERROR = 0x05,
+ XSC_EVENT_TYPE_INTERNAL_ERROR = 0x08,
+ /* IBV_EVENT_QP_REQ_ERR */
+ XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR = 0x10,
+ /* IBV_EVENT_QP_ACCESS_ERR */
+ XSC_EVENT_TYPE_WQ_ACCESS_ERROR = 0x11,
+};
+
+struct xsc_cqe {
+ __le32 data0;
+ __le32 imm_data;
+ __le32 msg_len;
+ __le32 vni;
+ __le64 data2;
+ u8 placeholder2;
+ u8 rsv3;
+ __le16 rsv[1];
+ __le32 data1;
+};
+#define XSC_CQE_DATA0_MSG_OPCD_MASK GENMASK(7, 0)
+#define XSC_CQE_DATA0_ERR_CODE_MASK GENMASK(6, 0)
+#define XSC_CQE_DATA0_IS_ERR BIT(7)
+#define XSC_CQE_DATA0_QP_ID_MASK GENMASK(22, 8)
+#define XSC_CQE_DATA0_SE BIT(24)
+#define XSC_CQE_DATA0_HAS_PPH BIT(25)
+#define XSC_CQE_DATA0_TYPE BIT(26)
+#define XSC_CQE_DATA0_WITH_IMM BIT(27)
+#define XSC_CQE_DATA0_CSUM_ERR_MASK GENMASK(31, 28)
+#define XSC_CQE_DATA2_TS_MASK GENMASK_ULL(47, 0)
+#define XSC_CQE_DATA2_WQE_ID_MASK GENMASK_ULL(63, 48)
+#define XSC_CQE_DATA1_OWNER BIT(31)
+
+struct xsc_cqe64 {
+ struct xsc_cqe cqe;
+ u8 padding[32];
+};
+
+struct xsc_core_cq {
+ u32 cqn;
+ u32 cqe_sz;
+ struct xsc_core_device *xdev;
+ struct xsc_core_device *dev;
+ atomic_t refcount;
+ struct completion free;
+ unsigned int vector;
+ unsigned int irqn;
+ void (*comp)(struct xsc_core_cq *cq);
+ void (*event)(struct xsc_core_cq *cq,
+ enum xsc_event);
+ u32 cons_index;
+ unsigned int arm_sn;
+ int pid;
+ u32 reg_next_cid;
+ u32 reg_done_pid;
+ struct xsc_eq *eq;
+};
+
+struct xsc_cq_table {
+ spinlock_t lock; /* protect radix tree */
+ struct radix_tree_root tree;
+};
+
+/* mr */
+struct xsc_core_mr {
+ u64 iova;
+ u64 size;
+ u32 key;
+ u32 pd;
+ u32 access;
+};
+
+/* eq */
+struct xsc_eq {
+ struct xsc_core_device *dev;
+ struct xsc_cq_table cq_table;
+ u32 cons_index;
+ struct xsc_buf buf;
+ unsigned int irqn;
+ u32 eqn;
+ u32 nent;
+ cpumask_var_t mask;
+ char name[XSC_MAX_EQ_NAME];
+ struct list_head list;
+ u16 index;
+};
+
+struct xsc_eq_table {
+ void __iomem *update_ci;
+ void __iomem *update_arm_ci;
+ struct list_head comp_eqs_list;
+ struct xsc_eq pages_eq;
+ struct xsc_eq async_eq;
+ struct xsc_eq cmd_eq;
+ int num_comp_vectors;
+ int eq_vec_comp_base;
+ /* protect EQs list */
+ spinlock_t lock;
+};
+
+/* irq */
+struct xsc_irq_info {
+ cpumask_var_t mask;
+ char name[XSC_MAX_IRQ_NAME];
+};
+
+/* adev */
+#define XSC_PCI_DRV_NAME "xsc_pci"
+#define XSC_ETH_ADEV_NAME "eth"
+#define XSC_IB_ADEV_NAME "ib"
+
+enum xsc_adev_idx {
+ XSC_ADEV_IDX_ETH,
+ XSC_ADEV_IDX_IB,
+ XSC_ADEV_IDX_MAX,
+};
+
+struct xsc_adev {
+ struct auxiliary_device adev;
+ struct xsc_core_device *xdev;
+
+ int idx;
+};
+
+struct xsc_board_info {
+ u32 board_id;
+ char board_sn[XSC_BOARD_SN_LEN];
+ __be64 guid;
+ u8 guid_valid;
+ u8 hw_config_activated;
+ struct xsc_msg_opcode *msg_opcode;
+};
+
+struct xsc_caps {
+ u8 log_max_eq;
+ u8 log_max_cq;
+ u8 log_max_qp;
+ u8 log_max_mkey;
+ u8 log_max_pd;
+ u8 log_max_srq;
+ u8 log_max_msix;
+ u32 max_cq;
+ u32 max_qp;
+ u32 max_pd;
+ u32 max_cqes;
+ u32 max_wqes;
+ u32 max_sq_desc_sz;
+ u32 max_rq_desc_sz;
+ u64 flags;
+ u16 stat_rate_support;
+ u32 log_max_msg;
+ u32 num_ports;
+ u32 max_ra_res_qp;
+ u32 max_ra_req_qp;
+ u32 max_srq_wqes;
+ u32 bf_reg_size;
+ u32 bf_regs_per_page;
+ u32 reserved_lkey;
+ u8 local_ca_ack_delay;
+ u8 log_max_mcg;
+ u16 max_qp_mcg;
+ u32 min_page_sz;
+ u32 send_ds_num;
+ u32 send_wqe_shift;
+ u32 recv_ds_num;
+ u32 recv_wqe_shift;
+ u32 rx_pkt_len_max;
+
+ u32 func_caps;
+
+ u16 max_vfs;
+ u8 log_max_qp_depth;
+ u8 log_max_current_uc_list;
+ u8 log_max_current_mc_list;
+ u16 log_max_vlan_list;
+ u8 fdb_multi_path_to_table;
+ u8 log_esw_max_sched_depth;
+
+ u8 max_num_sf_partitions;
+ u8 log_max_esw_sf;
+ u16 sf_base_id;
+
+ u32 qos_caps;
+ u32 nic_caps;
+ u16 msix_base;
+ u16 msix_num;
+ u8 log_max_mtt;
+ u8 log_max_tso;
+ u32 hca_core_clock;
+ u32 max_rwq_indirection_tables; /* rss_caps */
+ u32 max_rwq_indirection_table_size; /* rss_caps */
+ u16 raweth_qp_id_end;
+ u32 qp_rate_limit_min;
+ u32 qp_rate_limit_max;
+ u32 hw_feature_flag;
+ u16 pf0_vf_funcid_base;
+ u16 pf0_vf_funcid_top;
+ u16 pf1_vf_funcid_base;
+ u16 pf1_vf_funcid_top;
+ u16 pcie0_pf_funcid_base;
+ u16 pcie0_pf_funcid_top;
+ u16 pcie1_pf_funcid_base;
+ u16 pcie1_pf_funcid_top;
+ u8 nif_port_num;
+ u8 pcie_host;
+ u8 mac_bit;
+ u16 funcid_to_logic_port;
+ u8 lag_logic_port_ofst;
+
+ /* RDMA / MR resources (populated from hca_cap V1~V9) */
+ u64 max_mr_size;
+ u16 max_cmd_in_len;
+ u16 max_cmd_out_len;
+ u32 max_mtt;
+ u32 mpt_tbl_addr;
+ u32 mpt_tbl_depth;
+ u32 mpt_tbl_width;
+ u32 mtt_inst_base_addr;
+ u32 mtt_inst_stride;
+ u32 mtt_inst_num_log;
+ u32 mtt_inst_depth;
+};
+#define XSC_CAP_PORT_TYPE BIT(1)
+#define XSC_CAP_EMBEDDED_CPU BIT(2)
+#define XSC_CAP_ESWITCH_MANAGER BIT(3)
+#define XSC_CAP_ECPF_VPORT_EXISTS BIT(4)
+#define XSC_CAP_VPORT_GROUP_MANAGER BIT(5)
+#define XSC_CAP_SF BIT(6)
+#define XSC_CAP_WQE_INLINE_MODE_MASK GENMASK(9, 7)
+#define XSC_CAP_RAWETH_QP_ID_BASE_MASK GENMASK(24, 10)
+#define XSC_CAP_MAX_TC_MASK GENMASK(7, 0)
+#define XSC_CAP_ETS BIT(8)
+#define XSC_CAP_DCBX BIT(9)
+#define XSC_CAP_DSCP BIT(10)
+#define XSC_CAP_SBCAM_REG BIT(11)
+#define XSC_CAP_QOS BIT(12)
+#define XSC_CAP_PORT_BUF BIT(13)
+#define XSC_CAP_RAW_TPE_QP_NUM_MASK GENMASK(31, 16)
+#define XSC_CAP_MAX_NUM_EQS_MASK GENMASK(7, 0)
+#define XSC_CAP_MAC_PORT_MASK GENMASK(15, 8)
+#define XSC_CAP_RAWETH_RSS_QP_ID_BASE_MASK GENMASK(31, 16)
+
+struct xsc_srq_cache_wr;
+
+/* xsc_core */
+struct xsc_dev_resource {
+ struct xsc_qp_table qp_table;
+ struct xsc_cq_table cq_table;
+ struct xsc_eq_table eq_table;
+ struct xsc_irq_info *irq_info;
+ /* protect buffer allocation according to numa node */
+ struct mutex alloc_mutex;
+ /* protect mkey key allocation */
+ spinlock_t mkey_lock;
+ u8 mkey_key;
+
+ /* srq number allocation, srqn is a software index only */
+ spinlock_t srq_lock;
+ unsigned long *srq_tbl;
+ struct xsc_srq_cache_wr *srq_cache_wr;
+ u32 max_srq_num;
+};
+
+struct xsc_core_device {
+ struct pci_dev *pdev;
+ struct device *device;
+ int adev_id;
+ struct xsc_adev **xsc_adev_list;
+ void *eth_priv;
+ struct xsc_dev_resource *dev_res;
+ int numa_node;
+
+ /* async event plumbing, owned by the PCI layer */
+ struct work_struct event_work;
+ struct workqueue_struct *event_wq;
+ void (*link_event_handler)(void *xdev);
+
+ void __iomem *bar;
+ int bar_num;
+ enum xsc_coredev_type coredev_type;
+ struct xsc_hw_reg hal_regs;
+
+ u8 mac_port;
+ u8 pcie_no;
+ u8 pf_id;
+ u16 glb_func_id;
+ u16 gsi_qpn;
+ atomic_t num_qps;
+
+ u16 msix_vec_base;
+
+ void *netdev;
+
+ u32 rdma_proto_mode;
+ u16 cm_udp_dst_port;
+ u16 veroce_udp_dst_port;
+ u8 read_flush;
+ u8 bql_thresh;
+
+ struct xsc_priv priv;
+
+ struct xsc_cmd cmd;
+ u16 cmdq_ver;
+
+ struct xsc_caps caps;
+ struct xsc_board_info *board_info;
+
+ u32 chip_ver_h;
+ u32 chip_ver_m;
+ u32 chip_ver_l;
+ u32 hotfix_num;
+ u32 feature_flag;
+
+ u8 fw_version_major;
+ u8 fw_version_minor;
+ u16 fw_version_patch;
+ u32 fw_version_tweak;
+ u8 fw_version_extra_flag;
+ cpumask_var_t xps_cpumask;
+};
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,
+ struct xsc_create_qp_mbox_in *in,
+ int insize, u32 *p_qpn);
+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,
+ u32 qpn, u16 status);
+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn);
+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,
+ struct xsc_create_multiqp_mbox_in *in,
+ int insize,
+ u32 *p_qpn_base);
+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,
+ struct xsc_modify_raw_qp_mbox_in *in);
+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq,
+ struct xsc_create_cq_mbox_in *in,
+ int insize);
+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq);
+
+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i);
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn);
+
+/*
+ * Link event callback registration. The events are dispatched from the PCI
+ * layer's own workqueue; unregister detaches the callback and waits for one
+ * already in flight, so it must precede freeing what the handler touches.
+ */
+void xsc_irq_event_register(struct xsc_core_device *xdev,
+ void (*handler)(void *xdev));
+void xsc_irq_event_unregister(struct xsc_core_device *xdev);
+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,
+ __be64 *pas, unsigned int npages);
+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,
+ unsigned long size,
+ struct xsc_frag_buf *buf,
+ int node);
+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,
+ struct xsc_frag_buf *buf);
+
+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport);
+
+static inline void *xsc_buf_offset(struct xsc_buf *buf, unsigned long offset)
+{
+ if (likely(buf->nbufs == 1))
+ return buf->direct.buf + offset;
+ else
+ return buf->page_list[offset >> PAGE_SHIFT].buf +
+ (offset & (PAGE_SIZE - 1));
+}
+
+static inline u8 get_cqe_opcode(struct xsc_cqe *cqe)
+{
+ return FIELD_GET(XSC_CQE_DATA0_MSG_OPCD_MASK, le32_to_cpu(cqe->data0));
+}
+
+static inline bool xsc_core_is_pf(const struct xsc_core_device *dev)
+{
+ return dev->coredev_type == XSC_COREDEV_PF;
+}
+
+static inline bool is_support_hw_pf_stats(struct xsc_core_device *dev)
+{
+ return xsc_core_is_pf(dev);
+}
+
+static inline bool is_support_cqe64(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_CQE64_SUPPORT;
+}
+
+static inline bool is_support_rdma_cm(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_CM_SUPPORT;
+}
+
+static inline bool is_support_rdma_atomic(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_ATOMIC_SUPPORT;
+}
+
+static inline bool is_support_rdma(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_SUPPORT;
+}
+
+static inline bool is_support_8k_mtu(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_MTU_8192_SUPPORT;
+}
+
+static inline bool is_pf_bar_compressed(struct xsc_core_device *dev)
+{
+ return pci_resource_len(dev->pdev, 0) != XSC_ORIGIN_PF_BAR_SIZE;
+}
+
+static inline bool is_support_multi_doorbell(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_MULTIDOORBELL_SUPPORT;
+}
+
+void xsc_remove_rdma_driver(void);
+
+void xsc_init_hal(struct xsc_core_device *xdev);
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr,
+ u32 key, u32 length);
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt,
+ u32 max_qp_depth);
+void xsc_set_wqe_id(struct xsc_core_device *xdev, void *cseg, u32 wqe_id);
+int xsc_get_cqe_wqe_id(struct xsc_core_device *xdev, void *cqe);
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);
+int xsc_get_link_speed(struct xsc_core_device *xdev);
+void xsc_set_mtt_info(struct xsc_core_device *xdev);
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h
new file mode 100644
index 000000000..302534839
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_DEVICE_H
+#define __XSC_DEVICE_H
+
+#include <linux/types.h>
+
+/* QP type */
+enum {
+ XSC_QUEUE_TYPE_RDMA_RC = 0,
+ XSC_QUEUE_TYPE_RDMA_MAD = 1,
+ XSC_QUEUE_TYPE_RAW = 2,
+ XSC_QUEUE_TYPE_VIRTIO_NET = 3,
+ XSC_QUEUE_TYPE_VIRTIO_BLK = 4,
+ XSC_QUEUE_TYPE_RAW_TPE = 5,
+ XSC_QUEUE_TYPE_RAW_TSO = 6,
+ XSC_QUEUE_TYPE_RAW_TX = 7,
+ XSC_QUEUE_TYPE_SNIFFER = 8,
+ XSC_QUEUE_TYPE_WORC = 9,
+ XSC_QUEUE_TYPE_INVALID = 0xFF,
+};
+
+enum {
+ XSC_PERM_LOCAL_READ = 1 << 0,
+ XSC_PERM_LOCAL_WRITE = 1 << 1,
+ XSC_PERM_REMOTE_READ = 1 << 2,
+ XSC_PERM_REMOTE_WRITE = 1 << 3,
+ XSC_PERM_ATOMIC = 1 << 4,
+ XSC_PERM_UMR_EN = 1 << 7,
+};
+
+enum {
+ XSC_DEV_CAP_FLAG_RC = 1LL << 0,
+ XSC_DEV_CAP_FLAG_UC = 1LL << 1,
+ XSC_DEV_CAP_FLAG_UD = 1LL << 2,
+ XSC_DEV_CAP_FLAG_XRC = 1LL << 3,
+ XSC_DEV_CAP_FLAG_SRQ = 1LL << 6,
+ XSC_DEV_CAP_FLAG_BAD_PKEY_CNTR = 1LL << 8,
+ XSC_DEV_CAP_FLAG_BAD_QKEY_CNTR = 1LL << 9,
+ XSC_DEV_CAP_FLAG_APM = 1LL << 17,
+ XSC_DEV_CAP_FLAG_ATOMIC = 1LL << 18,
+ XSC_DEV_CAP_FLAG_ON_DMND_PG = 1LL << 24,
+ XSC_DEV_CAP_FLAG_RESIZE_SRQ = 1LL << 32,
+ XSC_DEV_CAP_FLAG_REMOTE_FENCE = 1LL << 38,
+ XSC_DEV_CAP_FLAG_TLP_HINTS = 1LL << 39,
+ XSC_DEV_CAP_FLAG_SIG_HAND_OVER = 1LL << 40,
+ XSC_DEV_CAP_FLAG_DCT = 1LL << 41,
+ XSC_DEV_CAP_FLAG_CMDIF_CSUM = 1LL << 46,
+};
+
+enum xsc_traffic_types {
+ XSC_TT_IPV4,
+ XSC_TT_IPV4_TCP,
+ XSC_TT_IPV4_UDP,
+ XSC_TT_IPV6,
+ XSC_TT_IPV6_TCP,
+ XSC_TT_IPV6_UDP,
+ XSC_TT_IPV4_IPSEC_AH,
+ XSC_TT_IPV6_IPSEC_AH,
+ XSC_TT_IPV4_IPSEC_ESP,
+ XSC_TT_IPV6_IPSEC_ESP,
+ XSC_TT_ANY,
+ XSC_NUM_TT,
+};
+
+#define XSC_NUM_INDIR_TIRS XSC_NUM_TT
+
+enum {
+ XSC_L3_PROT_TYPE_IPV4 = BIT(0),
+ XSC_L3_PROT_TYPE_IPV6 = BIT(1),
+};
+
+enum {
+ XSC_L4_PROT_TYPE_TCP = BIT(0),
+ XSC_L4_PROT_TYPE_UDP = BIT(1),
+};
+
+struct xsc_tirc_config {
+ u8 l3_prot_type;
+ u8 l4_prot_type;
+ u32 rx_hash_fields;
+};
+
+enum {
+ XSC_HASH_FUNC_XOR = 0,
+ XSC_HASH_FUNC_TOP = 1,
+ XSC_HASH_FUNC_TOP_SYM = 2,
+ XSC_HASH_FUNC_RSV = 3,
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h
new file mode 100644
index 000000000..aa2049f89
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_HSI_H
+#define XSC_HSI_H
+
+#include <asm/byteorder.h>
+
+#include <linux/types.h>
+#include <linux/bitops.h>
+
+#define WR_LE_32(x, val) (x = __cpu_to_le32(val))
+#define WR_LE_64(x, val) (x = __cpu_to_le64(val))
+
+#define WR_BE_32(x, val) (x = __cpu_to_be32(val))
+
+#define RD_LE_32(x) __le32_to_cpu(x)
+#define RD_BE_32(x) __be32_to_cpu(x)
+
+struct xsc_msg_opcode {
+ u32 send;
+ u32 send_with_imm;
+ u32 rdma_write;
+ u32 rdma_write_with_imm;
+ u32 rdma_read;
+ u32 atomic_cmp_and_swp;
+ u32 atomic_fetch_and_add;
+ u32 masked_atomic_cmp_and_swp;
+ u32 masked_atomic_fetch_and_add;
+ u32 mad;
+};
+
+/* sw cqe opcode */
+enum {
+ XSC_OPCODE_RDMA_REQ_SEND = 0,
+ XSC_OPCODE_RDMA_REQ_SEND_IMMDT = 1,
+ XSC_OPCODE_RDMA_RSP_RECV = 2,
+ XSC_OPCODE_RDMA_RSP_RECV_IMMDT = 3,
+ XSC_OPCODE_RDMA_REQ_WRITE = 4,
+ XSC_OPCODE_RDMA_REQ_WRITE_IMMDT = 5,
+ XSC_OPCODE_RDMA_RSP_WRITE_IMMDT = 6,
+ XSC_OPCODE_RDMA_REQ_READ = 7,
+ XSC_OPCODE_RDMA_REQ_ERROR = 8,
+ XSC_OPCODE_RDMA_RSP_ERROR = 9,
+ XSC_OPCODE_RDMA_CQE_ERROR = 10,
+ XSC_OPCODE_RDMA_MAD_REQ_SEND = 11,
+ XSC_OPCODE_RDMA_MAD_RSP_RECV = 12,
+ XSC_OPCODE_RDMA_CQE_RAW_SNF = 13,
+};
+
+enum {
+ XSC_SEND_SEG_MAX = 32,
+ XSC_BASE_WQE_SHIFT = 4,
+ XSC_SEND_SEG_NUM = 4,
+ XSC_SEND_WQE_SHIFT = 6,
+ XSC_CTRL_SEG_NUM = 1,
+ XSC_RADDR_SEG_NUM = 1,
+};
+
+enum {
+ XSC_RECV_SEG_MAX = 4,
+ XSC_RECV_SEG_NUM = 1,
+ XSC_RECV_WQE_SHIFT = 4,
+};
+
+enum {
+ XSC_INLINE_SIZE_MAX = 15,
+};
+
+#define XSC_MPT_MAP_EN 0
+
+enum {
+ XSC_REQ = 0,
+ XSC_RSP = 1,
+};
+
+enum {
+ XSC_WITHOUT_IMMDT = 0,
+ XSC_WITH_IMMDT = 1,
+};
+
+/* CQE error codes */
+enum {
+ XSC_ERR_CODE_NAK_RETRY = 0x40,
+ XSC_ERR_CODE_NAK_OPCODE = 0x41,
+ XSC_ERR_CODE_NAK_MR = 0x42,
+ XSC_ERR_CODE_NAK_OPERATION = 0x43,
+ XSC_ERR_CODE_NAK_RNR = 0x44,
+ XSC_ERR_CODE_LOCAL_MR = 0x45,
+ XSC_ERR_CODE_LOCAL_LEN = 0x46,
+ XSC_ERR_CODE_LOCAL_OPCODE = 0x47,
+ XSC_ERR_CODE_CQ_OVER_FLOW = 0x48,
+ XSC_ERR_CODE_STRG_ACC_GEN_CQE = 0x4c,
+ XSC_ERR_CODE_CQE_ACC = 0x4d,
+ XSC_ERR_CODE_FLUSH = 0x4e,
+ XSC_ERR_CODE_MALF_WQE_HOST = 0x50,
+ XSC_ERR_CODE_MALF_WQE_INFO = 0x51,
+ XSC_ERR_CODE_MR_NON_NAK = 0x52,
+ XSC_ERR_CODE_OPCODE_GEN_CQE = 0x61,
+ XSC_ERR_CODE_MANY_READ = 0x62,
+ XSC_ERR_CODE_LEN_GEN_CQE = 0x63,
+ XSC_ERR_CODE_MR = 0x65,
+ XSC_ERR_CODE_MR_GEN_CQE = 0x66,
+ XSC_ERR_CODE_OPERATION = 0x67,
+ XSC_ERR_CODE_MALF_WQE_INFO_GEN_NAK = 0x68,
+};
+
+enum xsc_qp_state {
+ XSC_QP_STATE_RST = 0,
+ XSC_QP_STATE_INIT = 1,
+ XSC_QP_STATE_RTR = 2,
+ XSC_QP_STATE_RTS = 3,
+ XSC_QP_STATE_SQER = 4,
+ XSC_QP_STATE_SQD = 5,
+ XSC_QP_STATE_ERR = 6,
+ XSC_QP_STATE_SQ_DRAINING = 7,
+ XSC_QP_STATE_SUSPENDED = 9,
+ XSC_QP_NUM_STATE
+};
+
+#endif /* XSC_HSI_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h
new file mode 100644
index 000000000..52747bb29
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_REG_H
+#define XSC_REG_H
+#define CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0x0
+#define CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0x4
+#define CMDQM_HOST_REQ_PID_MEM_ADDR 0x8
+#define CMDQM_HOST_REQ_CID_MEM_ADDR 0xc
+#define CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0x10
+#define CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0x14
+#define CMDQM_HOST_RSP_PID_MEM_ADDR 0x18
+#define CMDQM_HOST_RSP_CID_MEM_ADDR 0x1c
+#define CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0x20
+#define CMDQM_VECTOR_ID_MEM_ADDR 0x24
+
+#define DB_CQ_FUNC_MEM_ADDR 0x8c4
+#define DB_EQ_FUNC_MEM_ADDR 0x8c8
+#define DB_CQ_CID_DIRECT_MEM_ADDR 0x8cc
+#define TX_DB_FUNC_MEM_ADDR 0x8d0
+#define RX_DB_FUNC_MEM_ADDR 0x8d4
+
+#define XSC_MDB_NUM 64
+#define QPM_TX_MDB_BASE_REG_ADDR 0xd00
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig
new file mode 100644
index 000000000..1cc3f7b49
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Yunsilicon PCI configuration
+#
+
+config YUNSILICON_XSC_PCI
+ tristate "Yunsilicon XSC PCI driver"
+ depends on PCI
+ help
+ This driver is common for Yunsilicon XSC
+ ethernet and RDMA drivers.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_pci.
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
new file mode 100644
index 000000000..c163aa950
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
+
+xsc_pci-y := main.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
new file mode 100644
index 000000000..af654fc04
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+
+static const struct pci_device_id xsc_pci_id_table[] = {
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MC_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_HOST_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_SOC_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MS_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_HOST_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_SOC_PF_DEV_ID) },
+ { 0 }
+};
+
+static int xsc_set_dma_caps(struct pci_dev *pdev)
+{
+ int err;
+
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (!err)
+ dma_set_max_seg_size(&pdev->dev, SZ_2G);
+
+ return err;
+}
+
+static int xsc_pci_init(struct xsc_core_device *xdev,
+ const struct pci_device_id *id)
+{
+ struct pci_dev *pdev = xdev->pdev;
+ void __iomem *bar_base;
+ int bar_num = 0;
+ int err;
+
+ xdev->numa_node = dev_to_node(&pdev->dev);
+
+ err = pci_enable_device(pdev);
+ if (err) {
+ pci_err(pdev, "failed to enable PCI device: err=%d\n", err);
+ goto err_out;
+ }
+
+ err = pci_request_region(pdev, bar_num, KBUILD_MODNAME);
+ if (err) {
+ pci_err(pdev, "failed to request %s pci_region=%d: err=%d\n",
+ KBUILD_MODNAME, bar_num, err);
+ goto err_disable;
+ }
+
+ pci_set_master(pdev);
+
+ err = xsc_set_dma_caps(pdev);
+ if (err) {
+ pci_err(pdev, "failed to set DMA capabilities mask: err=%d\n",
+ err);
+ goto err_clr_master;
+ }
+
+ bar_base = pci_ioremap_bar(pdev, bar_num);
+ if (!bar_base) {
+ pci_err(pdev, "failed to ioremap %s bar%d\n", KBUILD_MODNAME,
+ bar_num);
+ err = -ENOMEM;
+ goto err_clr_master;
+ }
+
+ err = pci_save_state(pdev);
+ if (err) {
+ pci_err(pdev, "pci_save_state failed: err=%d\n", err);
+ goto err_io_unmap;
+ }
+
+ xdev->bar_num = bar_num;
+ xdev->bar = bar_base;
+
+ return 0;
+
+err_io_unmap:
+ pci_iounmap(pdev, bar_base);
+err_clr_master:
+ pci_clear_master(pdev);
+ pci_release_region(pdev, bar_num);
+err_disable:
+ pci_disable_device(pdev);
+err_out:
+ return err;
+}
+
+static void xsc_pci_fini(struct xsc_core_device *xdev)
+{
+ struct pci_dev *pdev = xdev->pdev;
+
+ pci_iounmap(pdev, xdev->bar);
+ pci_clear_master(pdev);
+ pci_release_region(pdev, xdev->bar_num);
+ pci_disable_device(pdev);
+}
+
+static int xsc_dev_res_init(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res;
+
+ dev_res = kvzalloc_obj(*dev_res, GFP_KERNEL);
+ if (!dev_res)
+ return -ENOMEM;
+
+ xdev->dev_res = dev_res;
+ mutex_init(&dev_res->alloc_mutex);
+ spin_lock_init(&dev_res->mkey_lock);
+
+ return 0;
+}
+
+static void xsc_dev_res_cleanup(struct xsc_core_device *xdev)
+{
+ kfree(xdev->dev_res);
+}
+
+static int xsc_core_dev_init(struct xsc_core_device *xdev)
+{
+ int err;
+
+ err = xsc_dev_res_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc dev res init failed %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static void xsc_core_dev_cleanup(struct xsc_core_device *xdev)
+{
+ xsc_dev_res_cleanup(xdev);
+}
+
+static int xsc_hw_setup(struct xsc_core_device *xdev)
+{
+ return 0;
+}
+
+static void xsc_hw_cleanup(struct xsc_core_device *xdev)
+{
+}
+
+static int xsc_load(struct xsc_core_device *xdev)
+{
+ int err = 0;
+
+ err = xsc_hw_setup(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_hw_setup failed %d\n", err);
+ goto err_out;
+ }
+
+ return 0;
+err_out:
+ return err;
+}
+
+static void xsc_unload(struct xsc_core_device *xdev)
+{
+ xsc_hw_cleanup(xdev);
+}
+
+static int xsc_pci_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *id)
+{
+ struct xsc_core_device *xdev;
+ int err;
+
+ xdev = kzalloc_obj(*xdev, GFP_KERNEL);
+ if (!xdev)
+ return -ENOMEM;
+
+ xdev->pdev = pci_dev;
+ xdev->device = &pci_dev->dev;
+ xdev->coredev_type = XSC_COREDEV_PF;
+
+ pci_set_drvdata(pci_dev, xdev);
+ err = xsc_pci_init(xdev, id);
+ if (err) {
+ pci_err(pci_dev, "xsc_pci_init failed %d\n", err);
+ goto err_unset_pci_drvdata;
+ }
+
+ err = xsc_core_dev_init(xdev);
+ if (err) {
+ pci_err(pci_dev, "xsc_core_dev_init failed %d\n", err);
+ goto err_pci_fini;
+ }
+
+ err = xsc_load(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_load failed %d\n", err);
+ goto err_core_dev_cleanup;
+ }
+
+ return 0;
+err_core_dev_cleanup:
+ xsc_core_dev_cleanup(xdev);
+err_pci_fini:
+ xsc_pci_fini(xdev);
+err_unset_pci_drvdata:
+ pci_set_drvdata(pci_dev, NULL);
+ kfree(xdev);
+
+ return err;
+}
+
+static void xsc_pci_remove(struct pci_dev *pci_dev)
+{
+ struct xsc_core_device *xdev = pci_get_drvdata(pci_dev);
+
+ xsc_unload(xdev);
+ xsc_core_dev_cleanup(xdev);
+ xsc_pci_fini(xdev);
+ pci_set_drvdata(pci_dev, NULL);
+ kfree(xdev);
+}
+
+static struct pci_driver xsc_pci_driver = {
+ .name = "xsc-pci",
+ .id_table = xsc_pci_id_table,
+ .probe = xsc_pci_probe,
+ .remove = xsc_pci_remove,
+};
+
+static int __init xsc_init(void)
+{
+ int err;
+
+ err = pci_register_driver(&xsc_pci_driver);
+ if (err) {
+ pr_err("failed to register pci driver\n");
+ return err;
+ }
+
+ return 0;
+}
+
+static void __exit xsc_fini(void)
+{
+ pci_unregister_driver(&xsc_pci_driver);
+}
+
+module_init(xsc_init);
+module_exit(xsc_fini);
+
+MODULE_DEVICE_TABLE(pci, xsc_pci_id_table);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Yunsilicon XSC PCI driver");
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 02/15] net/xsc: add PCI core driver skeleton
2026-11-04 15:56 ` [PATCH v1 02/15] net/xsc: add PCI core driver skeleton Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] net/xsc: endian-dependent bitfields in struct xsc_query_vport_state_out
--
commit b11d75eddf1256b25caffaed37629ef47a9f6953
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add PCI core driver skeleton
This commit adds the PCI core driver skeleton and common firmware interface
headers for Yunsilicon XSC controllers. It implements basic PCI device
probing, BAR mapping, DMA configuration, and core device resource allocation.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
> new file mode 100644
> index 0000000000000..70dbf9782f318
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
[ ... ]
> +struct xsc_query_vport_state_out {
> + struct xsc_outbox_hdr hdr;
> + u8 admin_state:4;
> + u8 state:4;
> +};
> +
> +struct xsc_query_vport_state_in {
> + struct xsc_inbox_hdr hdr;
> + __be32 data0;
> +#define XSC_QUERY_VPORT_OTHER_VPORT BIT(0)
> +#define XSC_QUERY_VPORT_VPORT_NUM_MASK GENMASK(16, 1)
> +};
[Severity: Medium]
In struct xsc_query_vport_state_out, admin_state and state are defined as
unguarded 4-bit C bitfields within a byte populated directly from the
firmware DMA response mailbox via xsc_core_query_vport_state() ->
xsc_cmd_exec() -> xsc_copy_from_rsp_msg().
On big-endian architectures, the compiler reverses the bit order of these
two 4-bit fields relative to little-endian architectures. When
xsc_core_query_vport_state() reads out.state, won't it read the admin_state
nibble instead of the operational vport state nibble on big-endian hosts?
Should struct xsc_query_vport_state_out use a plain u8 field with GENMASK()
and FIELD_GET(), similar to how struct xsc_query_vport_state_in defines
XSC_QUERY_VPORT_OTHER_VPORT and XSC_QUERY_VPORT_VPORT_NUM_MASK?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=2
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 03/15] net/xsc: add the register HAL
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
2026-11-04 15:56 ` [PATCH v1 01/15] MAINTAINERS: add the Yunsilicon XSC drivers Xin Tian
2026-11-04 15:56 ` [PATCH v1 02/15] net/xsc: add PCI core driver skeleton Xin Tian
@ 2026-11-04 15:56 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 04/15] net/xsc: add the command queue Xin Tian
` (11 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:56 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the hardware abstraction layer that translates logical register
accesses into BAR-relative MMIO. The XSC BAR layout differs between
full-BAR (256 MB) parts and parts with a compressed register window,
so the map is selected at init from the BAR size and stored per
device in xdev->hal_regs. One implementation, for the "Andes" chip
generation, is provided.
The layer implements the doorbell encodings for TX, RX, CQ
arm/notify and EQ update, each preceded by a write barrier to order
descriptor writes before the MMIO store. TX can use a bank of 64
multi-doorbell registers when the firmware reports support. It also
provides the command-queue register plumbing (req/rsp PIDs, ring
base addresses, error status) including the cmdq interrupt handler
that re-rings the request doorbell on read/write errors, the mkey
and MR page-mode helpers, and the message-opcode tables the RDMA
send path later uses to map IB opcodes to firmware message opcodes.
The implementation is static behind xsc_* wrappers taking a struct
xsc_core_device; a few helpers are exported for the ethernet and
RDMA modules.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
.../yunsilicon/xsc/pci/hal/andes_impl.c | 629 ++++++++++++++++++
.../yunsilicon/xsc/pci/hal/andes_reg.h | 43 ++
.../ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h | 60 ++
.../net/ethernet/yunsilicon/xsc/pci/main.c | 3 +
5 files changed, 736 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index c163aa950..81b65232e 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o
+xsc_pci-y := main.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
new file mode 100644
index 000000000..69f0daae5
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
@@ -0,0 +1,629 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include "andes_reg.h"
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+#include "common/xsc_reg.h"
+#include "xsc_hal.h"
+
+/* doorbell words */
+#define XSC_ANDES_TX_DB_DATA0_NEXT_PID_MASK GENMASK(15, 0)
+#define XSC_ANDES_TX_DB_DATA0_QP_ID_MASK GENMASK(30, 16)
+#define XSC_ANDES_RX_DB_DATA0_NEXT_PID_MASK GENMASK(12, 0)
+#define XSC_ANDES_RX_DB_DATA0_QP_ID_MASK GENMASK(27, 13)
+#define XSC_ANDES_CQ_DB_DATA0_NEXT_CID_MASK GENMASK(15, 0)
+#define XSC_ANDES_CQ_DB_DATA0_CQ_ID_MASK GENMASK(30, 16)
+#define XSC_ANDES_CQ_DB_DATA0_ARM BIT(31)
+#define XSC_ANDES_EQ_DB_DATA0_NEXT_CID_MASK GENMASK(10, 0)
+#define XSC_ANDES_EQ_DB_DATA0_EQ_ID_MASK GENMASK(21, 11)
+#define XSC_ANDES_EQ_DB_DATA0_ARM BIT(22)
+
+/* command queue error status bits */
+#define XSC_ANDES_CMDQ_ERR_HW_READ_REQ_ERR BIT(0)
+#define XSC_ANDES_CMDQ_ERR_HW_WRITE_REQ_ERR BIT(1)
+#define XSC_ANDES_CMDQ_ERR_REQ_PID_ERR BIT(2)
+#define XSC_ANDES_CMDQ_ERR_RSP_CID_ERR BIT(3)
+
+/* message opcode */
+enum {
+ XSC_MSG_OPCODE_SEND = 0,
+ XSC_MSG_OPCODE_RDMA_WRITE = 1,
+ XSC_MSG_OPCODE_RDMA_READ = 2,
+ XSC_MSG_OPCODE_MAD = 3,
+ XSC_MSG_OPCODE_RDMA_ACK = 4,
+ XSC_MSG_OPCODE_RDMA_ACK_READ = 5,
+ XSC_MSG_OPCODE_RDMA_CNP = 6,
+ XSC_MSG_OPCODE_RAW = 7,
+ XSC_MSG_OPCODE_VIRTIO_NET = 8,
+ XSC_MSG_OPCODE_VIRTIO_BLK = 9,
+ XSC_MSG_OPCODE_RAW_TPE = 10,
+ XSC_MSG_OPCODE_INIT_QP_REQ = 11,
+ XSC_MSG_OPCODE_INIT_QP_RSP = 12,
+ XSC_MSG_OPCODE_INIT_PATH_REQ = 13,
+ XSC_MSG_OPCODE_INIT_PATH_RSP = 14,
+
+ XSC_MSG_OPCODE_UNSUPPORT = -1,
+};
+
+#define REG_ADDR(bp, offset) ((bp) + (offset))
+
+static u32 xsc_read32(void __iomem *bar, u32 off)
+{
+ return readl(REG_ADDR(bar, off));
+}
+
+static void xsc_write32(void __iomem *bar, u32 off, u32 val)
+{
+ writel(val, REG_ADDR(bar, off));
+}
+
+static void andes_ring_tx_doorbell(struct xsc_core_device *xdev,
+ struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 sqn, u32 next_pid,
+ bool mdb)
+{
+ u32 db = FIELD_PREP(XSC_ANDES_TX_DB_DATA0_NEXT_PID_MASK, next_pid) |
+ FIELD_PREP(XSC_ANDES_TX_DB_DATA0_QP_ID_MASK, sqn);
+ u32 db_reg = mdb ? QPM_TX_MDB_BASE_REG_ADDR + (sqn & (XSC_MDB_NUM - 1)) * sizeof(u32) :
+ regs->tx_db;
+
+ /* Make sure that descriptors are written before
+ * updating doorbell record and ringing the doorbell
+ */
+ wmb();
+
+ writel(db, REG_ADDR(bar, db_reg));
+}
+
+static void andes_ring_rx_doorbell(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 rqn, u32 next_pid)
+{
+ u32 db = FIELD_PREP(XSC_ANDES_RX_DB_DATA0_NEXT_PID_MASK, next_pid) |
+ FIELD_PREP(XSC_ANDES_RX_DB_DATA0_QP_ID_MASK, rqn);
+
+ /* Make sure that descriptors are written before
+ * updating doorbell record and ringing the doorbell
+ */
+ wmb();
+
+ writel(db, REG_ADDR(bar, regs->rx_db));
+}
+
+static void andes_update_cq_db(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 cqn, u32 next_cid,
+ u8 solicited)
+{
+ u32 db = FIELD_PREP(XSC_ANDES_CQ_DB_DATA0_NEXT_CID_MASK, next_cid) |
+ FIELD_PREP(XSC_ANDES_CQ_DB_DATA0_CQ_ID_MASK, cqn) |
+ FIELD_PREP(XSC_ANDES_CQ_DB_DATA0_ARM, !!solicited);
+
+ /* Make sure that the doorbell record in host memory is
+ * written before ringing the doorbell via PCI MMIO.
+ */
+ wmb();
+ writel(db, REG_ADDR(bar, regs->complete_db));
+}
+
+static void andes_set_cq_ci(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 cqn, u32 next_cid)
+{
+ u32 db = FIELD_PREP(XSC_ANDES_CQ_DB_DATA0_NEXT_CID_MASK, next_cid) |
+ FIELD_PREP(XSC_ANDES_CQ_DB_DATA0_CQ_ID_MASK, cqn);
+
+ /* make sure val write to memory done */
+ wmb();
+ writel(db, REG_ADDR(bar, regs->complete_reg));
+}
+
+static void andes_set_eq_ci(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 eqn, u32 next_cid,
+ u8 arm)
+{
+ u32 db = FIELD_PREP(XSC_ANDES_EQ_DB_DATA0_NEXT_CID_MASK, next_cid) |
+ FIELD_PREP(XSC_ANDES_EQ_DB_DATA0_EQ_ID_MASK, eqn) |
+ FIELD_PREP(XSC_ANDES_EQ_DB_DATA0_ARM, !!arm);
+
+ /* make sure val write to memory done */
+ wmb();
+ writel(db, REG_ADDR(bar, regs->event_db));
+}
+
+static u8 andes_get_mr_page_mode(u8 page_shift)
+{
+enum {
+ PAGE_SHIFT_4K = 12,
+ XSC_PAGE_SHIFT_64K = 16,
+ XSC_PAGE_SHIFT_2M = 21,
+ XSC_PAGE_SHIFT_1G = 30,
+};
+
+enum {
+ XSC_PAGE_MODE_4K = 0,
+ XSC_PAGE_MODE_64K = 1,
+ XSC_PAGE_MODE_2M = 2,
+ XSC_PAGE_MODE_1G = 3,
+};
+
+ return (page_shift == XSC_PAGE_SHIFT_4K ? XSC_PAGE_MODE_4K :
+ (page_shift == XSC_PAGE_SHIFT_64K ? XSC_PAGE_MODE_64K :
+ (page_shift == XSC_PAGE_SHIFT_2M ? XSC_PAGE_MODE_2M : XSC_PAGE_MODE_1G)));
+}
+
+static inline u32 andes_mkey_to_idx(u32 mkey)
+{
+ return mkey >> 17;
+}
+
+static inline u32 andes_idx_to_mkey(u32 mkey_idx)
+{
+ return mkey_idx << 17;
+}
+
+static const u32 xsc_msg_opcode[][2][2] = {
+ [XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND,
+ [XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND_IMMDT,
+ [XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV,
+ [XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_WRITE_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_READ,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_MAD][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_REQ_SEND,
+ [XSC_MSG_OPCODE_MAD][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_MAD][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_RSP_RECV,
+ [XSC_MSG_OPCODE_MAD][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+};
+
+struct andes_cqe {
+ __le32 data0;
+ __le32 imm_data;
+ __le32 msg_len;
+ __le32 vni;
+ __le64 data2;
+ __le16 rsv[2];
+ __le32 data1;
+};
+
+static int andes_get_cqe_wqe_id(void *cqe)
+{
+ return FIELD_GET(XSC_CQE_DATA2_WQE_ID_MASK,
+ le64_to_cpu(((struct andes_cqe *)cqe)->data2));
+}
+
+static u8 andes_get_cqe_opcode(void *cqe)
+{
+ u32 val = le32_to_cpu(((struct andes_cqe *)cqe)->data0);
+ u8 msg_opcode = FIELD_GET(XSC_CQE_DATA0_MSG_OPCD_MASK, val);
+
+ if (val & XSC_CQE_DATA0_IS_ERR)
+ return (val & XSC_CQE_DATA0_TYPE) ? XSC_OPCODE_RDMA_RSP_ERROR :
+ XSC_OPCODE_RDMA_REQ_ERROR;
+ if (msg_opcode > XSC_MSG_OPCODE_MAD)
+ return XSC_OPCODE_RDMA_CQE_ERROR;
+ return xsc_msg_opcode[msg_opcode][(val & XSC_CQE_DATA0_TYPE) != 0]
+ [(val & XSC_CQE_DATA0_WITH_IMM) != 0];
+}
+
+struct andes_data_seg {
+ __le32 data0;
+ __le32 key;
+ __le64 addr;
+};
+
+static void andes_set_data_seg(void *data_seg, u32 length, u32 key, u64 addr)
+{
+ struct andes_data_seg *seg = data_seg;
+
+ seg->data0 = cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK,
+ length));
+ seg->key = cpu_to_le32(key);
+ seg->addr = cpu_to_le64(addr);
+}
+
+struct andes_send_wqe_ctrl_seg {
+ __le32 data0;
+ __le32 msg_len;
+ __le32 data2;
+ __le32 data3;
+};
+
+static void andes_set_wqe_id(void *cseg, u32 wqe_id)
+{
+ struct xsc_send_wqe_ctrl_seg *seg = cseg;
+ u32 val = le32_to_cpu(seg->data0);
+
+ seg->data0 = cpu_to_le32((val & ~XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK,
+ (u16)wqe_id));
+}
+
+static struct xsc_msg_opcode xsc_ib_opcode = {
+ .send = XSC_MSG_OPCODE_SEND,
+ .send_with_imm = XSC_MSG_OPCODE_SEND,
+ .rdma_write = XSC_MSG_OPCODE_RDMA_WRITE,
+ .rdma_write_with_imm = XSC_MSG_OPCODE_RDMA_WRITE,
+ .rdma_read = XSC_MSG_OPCODE_RDMA_READ,
+ .atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,
+ .atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,
+ .masked_atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,
+ .masked_atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,
+ .mad = XSC_MSG_OPCODE_MAD,
+};
+
+static struct xsc_msg_opcode *andes_get_msg_opcode(void)
+{
+ return &xsc_ib_opcode;
+}
+
+static struct xsc_hw_reg andes_pf_regs = {
+ .req_pid_addr = HIF_CMDQM_HOST_REQ_PID_MEM_ADDR - 0xa0000000,
+ .req_cid_addr = HIF_CMDQM_HOST_REQ_CID_MEM_ADDR - 0xa0000000,
+ .rsp_pid_addr = HIF_CMDQM_HOST_RSP_PID_MEM_ADDR - 0xa0000000,
+ .rsp_cid_addr = HIF_CMDQM_HOST_RSP_CID_MEM_ADDR - 0xa0000000,
+ .req_buf_h_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,
+ .req_buf_l_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,
+ .rsp_buf_h_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,
+ .rsp_buf_l_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,
+ .msix_vec_addr = HIF_CMDQM_VECTOR_ID_MEM_ADDR - 0xa0000000,
+ .interrupt_stat_addr = HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,
+ .mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,
+ .mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,
+ .mtt_inst_stride = 0,
+ .mtt_inst_num_log = 0,
+ .mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,
+};
+
+/* BAR-relative window shared by bar-compressed PF and VF */
+static struct xsc_hw_reg andes_compressed_regs = {
+ .tx_db = TX_DB_FUNC_MEM_ADDR,
+ .rx_db = RX_DB_FUNC_MEM_ADDR,
+ .complete_db = DB_CQ_FUNC_MEM_ADDR,
+ .complete_reg = DB_CQ_CID_DIRECT_MEM_ADDR,
+ .event_db = DB_EQ_FUNC_MEM_ADDR,
+ .req_pid_addr = CMDQM_HOST_REQ_PID_MEM_ADDR,
+ .req_cid_addr = CMDQM_HOST_REQ_CID_MEM_ADDR,
+ .rsp_pid_addr = CMDQM_HOST_RSP_PID_MEM_ADDR,
+ .rsp_cid_addr = CMDQM_HOST_RSP_CID_MEM_ADDR,
+ .req_buf_h_addr = CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR,
+ .req_buf_l_addr = CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR,
+ .rsp_buf_h_addr = CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR,
+ .rsp_buf_l_addr = CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR,
+ .msix_vec_addr = CMDQM_VECTOR_ID_MEM_ADDR,
+ .interrupt_stat_addr = CMDQM_HOST_VF_ERR_STS_MEM_ADDR,
+ .mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,
+ .mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,
+ .mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,
+ .mtt_inst_stride = 0,
+ .mtt_inst_num_log = 0,
+ .mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,
+};
+
+void xsc_init_hal(struct xsc_core_device *xdev)
+{
+ struct xsc_hw_reg *regs;
+
+ if (is_pf_bar_compressed(xdev))
+ regs = &andes_compressed_regs;
+ else
+ regs = &andes_pf_regs;
+
+ /* every device gets its own copy, xsc_set_mtt_info() and
+ * xsc_set_pf_db_addr() fill it with device-specific values
+ */
+ memcpy(&xdev->hal_regs, regs, sizeof(xdev->hal_regs));
+}
+
+void xsc_set_mtt_info(struct xsc_core_device *xdev)
+{
+ struct xsc_hw_reg *regs = &xdev->hal_regs;
+
+ regs->mpt_tbl_addr = xdev->caps.mpt_tbl_addr - 0xa0000000;
+ regs->mpt_tbl_depth = xdev->caps.mpt_tbl_depth;
+ regs->mpt_tbl_width = xdev->caps.mpt_tbl_width;
+ regs->mtt_inst_base_addr = xdev->caps.mtt_inst_base_addr - 0xa0000000;
+ regs->mtt_inst_stride = xdev->caps.mtt_inst_stride;
+ regs->mtt_inst_num_log = xdev->caps.mtt_inst_num_log;
+ regs->mtt_inst_depth = xdev->caps.mtt_inst_depth;
+}
+
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db)
+{
+ struct xsc_hw_reg *regs = &xdev->hal_regs;
+
+ if (is_pf_bar_compressed(xdev))
+ return;
+
+ if (xsc_core_is_pf(xdev)) {
+ regs->tx_db = tx_db - 0xa0000000;
+ regs->rx_db = rx_db - 0xa0000000;
+ regs->complete_db = cq_db - 0xa0000000;
+ regs->complete_reg = cq_reg - 0xa0000000;
+ regs->event_db = eq_db - 0xa0000000;
+ }
+}
+
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db)
+{
+ struct xsc_hw_reg *regs = &xdev->hal_regs;
+
+ if (!xsc_core_is_pf(xdev) || is_pf_bar_compressed(xdev)) {
+ if (tx_db)
+ *tx_db = regs->tx_db;
+ if (rx_db)
+ *rx_db = regs->rx_db;
+ if (cq_db)
+ *cq_db = regs->complete_db;
+ if (cq_reg)
+ *cq_reg = regs->complete_reg;
+ if (eq_db)
+ *eq_db = regs->event_db;
+ } else {
+ if (tx_db)
+ *tx_db = regs->tx_db + 0xa0000000;
+ if (rx_db)
+ *rx_db = regs->rx_db + 0xa0000000;
+ if (cq_db)
+ *cq_db = regs->complete_db + 0xa0000000;
+ if (cq_reg)
+ *cq_reg = regs->complete_reg + 0xa0000000;
+ if (eq_db)
+ *eq_db = regs->event_db + 0xa0000000;
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_get_db_addr);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid)
+{
+ bool mdb = is_support_multi_doorbell(xdev);
+
+ andes_ring_tx_doorbell(xdev, &xdev->hal_regs, xdev->bar, sqn,
+ next_pid, mdb);
+}
+EXPORT_SYMBOL_GPL(xsc_update_tx_db);
+
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid)
+{
+ andes_ring_rx_doorbell(&xdev->hal_regs, xdev->bar, rqn, next_pid);
+}
+EXPORT_SYMBOL_GPL(xsc_update_rx_db);
+
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited)
+{
+ andes_update_cq_db(&xdev->hal_regs, xdev->bar, cqn, next_cid, solicited);
+}
+EXPORT_SYMBOL_GPL(xsc_arm_cq);
+
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid)
+{
+ andes_set_cq_ci(&xdev->hal_regs, xdev->bar, cqn, next_cid);
+}
+EXPORT_SYMBOL_GPL(xsc_update_cq_ci);
+
+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm)
+{
+ andes_set_eq_ci(&xdev->hal_regs, xdev->bar, eqn, next_cid, arm);
+}
+
+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs.req_pid_addr, req_pid);
+}
+
+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs.rsp_cid_addr, rsp_cid);
+}
+
+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev)
+{
+ return xsc_read32(xdev->bar, xdev->hal_regs.req_pid_addr);
+}
+
+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev)
+{
+ return xsc_read32(xdev->bar, xdev->hal_regs.req_cid_addr);
+}
+
+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev)
+{
+ return xsc_read32(xdev->bar, xdev->hal_regs.rsp_pid_addr);
+}
+
+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev)
+{
+ return xsc_read32(xdev->bar, xdev->hal_regs.rsp_cid_addr);
+}
+
+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs.req_buf_h_addr, haddr);
+ xsc_write32(xdev->bar, xdev->hal_regs.req_buf_l_addr, laddr);
+}
+
+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs.rsp_buf_h_addr, haddr);
+ xsc_write32(xdev->bar, xdev->hal_regs.rsp_buf_l_addr, laddr);
+}
+
+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs.msix_vec_addr, vector);
+}
+
+void xsc_check_cmdq_status(struct xsc_core_device *xdev)
+{
+ u32 status;
+
+ status = xsc_read32(xdev->bar, xdev->hal_regs.interrupt_stat_addr);
+ if (status)
+ xsc_write32(xdev->bar, xdev->hal_regs.interrupt_stat_addr, status);
+}
+
+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev)
+{
+ int err = 0;
+ int retry = 0;
+ u32 stat;
+ u32 raw;
+
+ raw = xsc_read32(xdev->bar, xdev->hal_regs.interrupt_stat_addr);
+ stat = raw;
+ while (stat != 0) {
+ err++;
+ if (stat & XSC_ANDES_CMDQ_ERR_HW_READ_REQ_ERR) {
+ retry = 1;
+ stat &= ~XSC_ANDES_CMDQ_ERR_HW_READ_REQ_ERR;
+ pci_err(xdev->pdev, "hw report read req from host failed!\n");
+ } else if (stat & XSC_ANDES_CMDQ_ERR_HW_WRITE_REQ_ERR) {
+ retry = 1;
+ stat &= ~XSC_ANDES_CMDQ_ERR_HW_WRITE_REQ_ERR;
+ pci_err(xdev->pdev, "hw report write req to fw failed!\n");
+ } else if (stat & XSC_ANDES_CMDQ_ERR_REQ_PID_ERR) {
+ stat &= ~XSC_ANDES_CMDQ_ERR_REQ_PID_ERR;
+ pci_err(xdev->pdev, "hw report unexpected req pid!\n");
+ } else if (stat & XSC_ANDES_CMDQ_ERR_RSP_CID_ERR) {
+ stat &= ~XSC_ANDES_CMDQ_ERR_RSP_CID_ERR;
+ pci_err(xdev->pdev, "hw report unexpected rsp cid!\n");
+ } else {
+ stat = 0;
+ pci_err(xdev->pdev, "ignore unknown interrupt!\n");
+ }
+ }
+
+ if (retry) {
+ unsigned long flags;
+
+ if (xdev->cmd.retry_cnt >= XSC_CMD_MAX_RETRY_CNT) {
+ pci_warn(xdev->pdev, "err handler have retried for %d times, stop retry\n",
+ xdev->cmd.retry_cnt);
+ goto clear_err_int;
+ }
+ spin_lock_irqsave(&xdev->cmd.doorbell_lock, flags);
+ xsc_write32(xdev->bar, xdev->hal_regs.req_pid_addr,
+ xdev->cmd.cmd_pid);
+ xdev->cmd.retry_cnt++;
+ spin_unlock_irqrestore(&xdev->cmd.doorbell_lock, flags);
+ }
+
+clear_err_int:
+ if (err)
+ xsc_write32(xdev->bar, xdev->hal_regs.interrupt_stat_addr, raw);
+
+ return err;
+}
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift)
+{
+ return andes_get_mr_page_mode(page_shift);
+}
+EXPORT_SYMBOL_GPL(xsc_get_mr_page_mode);
+
+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey)
+{
+ return andes_mkey_to_idx(mkey);
+}
+
+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx)
+{
+ return andes_idx_to_mkey(mkey_idx);
+}
+
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe)
+{
+ return andes_get_cqe_opcode(cqe);
+}
+EXPORT_SYMBOL_GPL(xsc_get_cqe_opcode);
+
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver)
+{
+ u64 mask;
+
+ if (xsc_core_is_pf(xdev))
+ mask = 0xFFFFFF;
+ else
+ mask = 0xfff;
+
+ return mask;
+}
+EXPORT_SYMBOL_GPL(xsc_get_rdma_stat_mask);
+
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length)
+{
+ andes_set_data_seg(data_seg, length, key, addr);
+}
+EXPORT_SYMBOL_GPL(xsc_set_data_seg);
+
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev)
+{
+ return 32 * 1024;
+}
+EXPORT_SYMBOL_GPL(xsc_get_max_qp_depth);
+
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth)
+{
+ if (*wqe_cnt > max_qp_depth)
+ *wqe_cnt = max_qp_depth;
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(xsc_check_max_qp_depth);
+
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev)
+{
+ return xdev->caps.recv_ds_num;
+}
+EXPORT_SYMBOL_GPL(xsc_get_recv_ds_num);
+
+void *xsc_get_msg_opcode(struct xsc_core_device *xdev)
+{
+ return andes_get_msg_opcode();
+}
+
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev)
+{
+ u64 max_mpt_mr_size = xdev->caps.max_mr_size;
+ u64 max_mtt_mr_size = (u64)xdev->caps.max_mtt * XSC_PAGE_SIZE_4K;
+
+ return min_t(u64, max_mpt_mr_size, max_mtt_mr_size);
+}
+EXPORT_SYMBOL_GPL(xsc_get_max_mr_size);
+
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base)
+{
+ if (is_support_multi_doorbell(xdev)) {
+ *num = XSC_MDB_NUM;
+ *base = QPM_TX_MDB_BASE_REG_ADDR;
+ } else {
+ *num = 0;
+ *base = 0;
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_get_multidb_info);
+
+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id)
+{
+ andes_set_wqe_id(cseg, wqe_id);
+}
+EXPORT_SYMBOL_GPL(xsc_set_wqe_id);
+
+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe)
+{
+ return andes_get_cqe_wqe_id(cqe);
+}
+EXPORT_SYMBOL_GPL(xsc_get_cqe_wqe_id);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h
new file mode 100644
index 000000000..cd25b64a5
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+/* generated time:
+ * Thu Feb 29 15:33:50 CST 2024
+ */
+
+#ifndef ANDES_REG_H
+#define ANDES_REG_H
+
+//hif_irq_csr_defines.h
+
+//hif_cpm_csr_defines.h
+
+//mmc_csr_defines.h
+#define MMC_MPT_TBL_MEM_DEPTH 32768
+#define MMC_MTT_TBL_MEM_DEPTH 262144
+#define MMC_MPT_TBL_MEM_WIDTH 256
+#define MMC_MPT_TBL_MEM_ADDR 0xa4100000
+#define MMC_MTT_TBL_MEM_ADDR 0xa4200000
+
+//clsf_dma_csr_defines.h
+
+//hif_tbl_csr_defines.h
+
+//hif_cmdqm_csr_defines.h
+#define HIF_CMDQM_HOST_REQ_PID_MEM_ADDR 0xa1026000
+#define HIF_CMDQM_HOST_REQ_CID_MEM_ADDR 0xa1028000
+#define HIF_CMDQM_HOST_RSP_PID_MEM_ADDR 0xa102e000
+#define HIF_CMDQM_HOST_RSP_CID_MEM_ADDR 0xa1030000
+#define HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0xa1022000
+#define HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0xa1024000
+#define HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0xa102a000
+#define HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0xa102c000
+#define HIF_CMDQM_VECTOR_ID_MEM_ADDR 0xa1034000
+#define HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0xa1032000
+
+//PSV use
+//hif_irq_csr_defines.h
+
+#endif /* XSC_HW_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
new file mode 100644
index 000000000..f6292a91b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_HAL_H
+#define XSC_HAL_H
+
+struct xsc_core_device;
+
+/* HAL entry points exported to the PCI/RDMA/ETH layers */
+void xsc_init_hal(struct xsc_core_device *xdev);
+void xsc_set_mtt_info(struct xsc_core_device *xdev);
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);
+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm);
+
+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid);
+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid);
+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev);
+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);
+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);
+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector);
+void xsc_check_cmdq_status(struct xsc_core_device *xdev);
+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev);
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);
+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey);
+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx);
+
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length);
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth);
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);
+void *xsc_get_msg_opcode(struct xsc_core_device *xdev);
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);
+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id);
+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe);
+
+enum {
+ CQ_STAT_FIRED,
+ CQ_STAT_KEEP,
+ CQ_STAT_ARM_NEXT,
+ CQ_STAT_ARM_SOLICITED,
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
index af654fc04..ab428d20f 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -4,6 +4,7 @@
*/
#include "common/xsc_core.h"
+#include "hal/xsc_hal.h"
static const struct pci_device_id xsc_pci_id_table[] = {
{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MC_PF_DEV_ID) },
@@ -75,6 +76,8 @@ static int xsc_pci_init(struct xsc_core_device *xdev,
xdev->bar_num = bar_num;
xdev->bar = bar_base;
+ xsc_init_hal(xdev);
+
return 0;
err_io_unmap:
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 03/15] net/xsc: add the register HAL
2026-11-04 15:56 ` [PATCH v1 03/15] net/xsc: add the register HAL Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] net/xsc: duplicate declarations, dead definitions, and missing types.h in HAL
--
commit 2bedc4243c5f325b253b8d3867197c1b8085594e
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add the register HAL
This commit adds the hardware abstraction layer for translating logical
register accesses into BAR-relative MMIO for the Andes chip generation. It
implements doorbell encodings, command-queue register accessors, and WQE/CQE
and MR helper functions.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
> new file mode 100644
> index 0000000000000..69f0daae5abe6
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
> @@ -0,0 +1,629 @@
[ ... ]
> +static void andes_ring_tx_doorbell(struct xsc_core_device *xdev,
> + struct xsc_hw_reg *regs,
> + void __iomem *bar, u32 sqn, u32 next_pid,
> + bool mdb)
> +{
> + u32 db = FIELD_PREP(XSC_ANDES_TX_DB_DATA0_NEXT_PID_MASK, next_pid) |
> + FIELD_PREP(XSC_ANDES_TX_DB_DATA0_QP_ID_MASK, sqn);
> + u32 db_reg = mdb ? QPM_TX_MDB_BASE_REG_ADDR + (sqn & (XSC_MDB_NUM - 1)) * sizeof(u32) :
> + regs->tx_db;
> +
> + /* Make sure that descriptors are written before
> + * updating doorbell record and ringing the doorbell
> + */
> + wmb();
> +
> + writel(db, REG_ADDR(bar, db_reg));
> +}
[ ... ]
> +static u8 andes_get_mr_page_mode(u8 page_shift)
> +{
> +enum {
> + PAGE_SHIFT_4K = 12,
> + XSC_PAGE_SHIFT_64K = 16,
> + XSC_PAGE_SHIFT_2M = 21,
> + XSC_PAGE_SHIFT_1G = 30,
> +};
> +
> +enum {
> + XSC_PAGE_MODE_4K = 0,
> + XSC_PAGE_MODE_64K = 1,
> + XSC_PAGE_MODE_2M = 2,
> + XSC_PAGE_MODE_1G = 3,
> +};
> +
> + return (page_shift == XSC_PAGE_SHIFT_4K ? XSC_PAGE_MODE_4K :
> + (page_shift == XSC_PAGE_SHIFT_64K ? XSC_PAGE_MODE_64K :
> + (page_shift == XSC_PAGE_SHIFT_2M ? XSC_PAGE_MODE_2M : XSC_PAGE_MODE_1G)));
> +}
[ ... ]
> +struct andes_send_wqe_ctrl_seg {
> + __le32 data0;
> + __le32 msg_len;
> + __le32 data2;
> + __le32 data3;
> +};
> +
> +static void andes_set_wqe_id(void *cseg, u32 wqe_id)
> +{
> + struct xsc_send_wqe_ctrl_seg *seg = cseg;
> + u32 val = le32_to_cpu(seg->data0);
> +
> + seg->data0 = cpu_to_le32((val & ~XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK) |
> + FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK,
> + (u16)wqe_id));
> +}
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
> new file mode 100644
> index 0000000000000..f6292a91ba2fb
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
> @@ -0,0 +1,60 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
> + * All rights reserved.
> + */
> +
> +#ifndef XSC_HAL_H
> +#define XSC_HAL_H
> +
> +struct xsc_core_device;
> +
> +/* HAL entry points exported to the PCI/RDMA/ETH layers */
> +void xsc_init_hal(struct xsc_core_device *xdev);
> +void xsc_set_mtt_info(struct xsc_core_device *xdev);
> +void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
> + u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);
> +void xsc_get_db_addr(struct xsc_core_device *xdev,
> + u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);
> +
> +void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);
> +void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);
> +void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);
> +void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);
[Severity: Low]
This isn't a bug, but there are a few unused definitions and duplicated
declarations across the new HAL files:
In drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h, the prototypes use
u64, u32, u16, u8, and bool without including linux/types.h, and 19 of the
function declarations (such as xsc_init_hal(), xsc_set_mtt_info(),
xsc_set_pf_db_addr(), xsc_get_db_addr(), and the doorbell and WQE/CQE
helpers) duplicate prototypes that are already declared in
drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h:
drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h:
void xsc_init_hal(struct xsc_core_device *xdev);
u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);
u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);
...
In drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c:
- In andes_ring_tx_doorbell(), the xdev parameter is unused.
- In andes_get_mr_page_mode(), the local unindented enum defines
PAGE_SHIFT_4K and XSC_PAGE_SHIFT_1G, neither of which is used because the
return expression compares against XSC_PAGE_SHIFT_4K from
common/xsc_core.h and defaults to XSC_PAGE_MODE_1G.
- struct andes_send_wqe_ctrl_seg is defined right above andes_set_wqe_id()
but is never referenced because andes_set_wqe_id() casts cseg to
struct xsc_send_wqe_ctrl_seg from common/xsc_core.h; similarly,
struct andes_cqe and struct andes_data_seg duplicate struct xsc_cqe and
struct xsc_wqe_data_seg in common/xsc_core.h.
Could these unused definitions be dropped and the header declarations
consolidated with common/xsc_core.h (and linux/types.h included in
xsc_hal.h)?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=3
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 04/15] net/xsc: add the command queue
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (2 preceding siblings ...)
2026-11-04 15:56 ` [PATCH v1 03/15] net/xsc: add the register HAL Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 05/15] net/xsc: add event queues, interrupts and the Xin Tian
` (10 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the command queue used to talk to firmware. Requests and
responses travel through a pair of DMA-coherent PAGE_SIZE rings of
32 64-byte descriptors, with a dma_pool of 512-byte protected blocks
backing mailbox payloads too large to fit inline. A request fills
the inline descriptor fields, chains overflow mailboxes, then rings
the request doorbell with the advanced producer index.
Completion can run either from a polling kthread or from the
command-CQ interrupt; both match the response against the in-use
slot bitmap and complete the waiter. Command slots are allocated
from a bitmap under a spinlock and the doorbell is rung under a
separate lock so that concurrent submitters do not corrupt the
producer index. A ten second timeout moves the queue into a
timed-out state after which submitters fail fast rather than hang.
At init the current producer and consumer indices are read back from
hardware, so the queue resumes correctly after a kexec or FLR.
On top of the transport, add the initial firmware handshake:
validate the command interface version, query the HCA capabilities
into struct xsc_caps, query and activate the GUID, reset the
function, and cache the per-board GUID and message-opcode table in a
small table keyed by board serial number so the work is shared by
all PCI functions of the same physical card.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../ethernet/yunsilicon/xsc/common/xsc_cmdq.h | 1 +
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/pci/cmdq.c | 1244 +++++++++++++++++
drivers/net/ethernet/yunsilicon/xsc/pci/hw.c | 316 +++++
drivers/net/ethernet/yunsilicon/xsc/pci/hw.h | 18 +
.../net/ethernet/yunsilicon/xsc/pci/main.c | 55 +
6 files changed, 1635 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/hw.h
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
index c163bd179..256d6a2ce 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
@@ -25,6 +25,7 @@ enum {
enum {
XSC_MAX_COMMANDS = 32,
+ XSC_CMDQ_INDEX_MASK = XSC_MAX_COMMANDS - 1,
XSC_CMD_DATA_BLOCK_SIZE = 512,
XSC_PCI_CMD_XPORT = 7,
};
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index 81b65232e..656cdffa1 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
new file mode 100644
index 000000000..c001b3122
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
@@ -0,0 +1,1244 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ * Copyright (c) 2013-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/random.h>
+#include <linux/kthread.h>
+#include <linux/io-mapping.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_core.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ CMD_IF_REV = 3,
+};
+
+enum {
+ NUM_LONG_LISTS = 2,
+ NUM_MED_LISTS = 64,
+ LONG_LIST_SIZE = (2ULL * 1024 * 1024 * 1024 / PAGE_SIZE) * 8 + 16 +
+ XSC_CMD_DATA_BLOCK_SIZE,
+ MED_LIST_SIZE = 16 + XSC_CMD_DATA_BLOCK_SIZE,
+};
+
+enum {
+ XSC_CMD_DELIVERY_STAT_OK = 0x0,
+ XSC_CMD_DELIVERY_STAT_SIGNAT_ERR = 0x1,
+ XSC_CMD_DELIVERY_STAT_TOK_ERR = 0x2,
+ XSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR = 0x3,
+ XSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR = 0x4,
+ XSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR = 0x5,
+ XSC_CMD_DELIVERY_STAT_FW_ERR = 0x6,
+ XSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR = 0x7,
+ XSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR = 0x8,
+ XSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR = 0x9,
+ XSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR = 0x10,
+};
+
+static struct xsc_cmd_work_ent *xsc_alloc_cmd(struct xsc_cmd *cmd,
+ struct xsc_cmd_msg *in,
+ struct xsc_rsp_msg *out)
+{
+ struct xsc_cmd_work_ent *ent;
+
+ ent = kzalloc_obj(*ent, GFP_KERNEL);
+ if (!ent)
+ return ERR_PTR(-ENOMEM);
+
+ ent->in = in;
+ ent->out = out;
+ ent->cmd = cmd;
+
+ return ent;
+}
+
+static u8 xsc_alloc_token(struct xsc_cmd *cmd)
+{
+ u8 token;
+
+ spin_lock(&cmd->token_lock);
+ token = cmd->token++ % 255 + 1;
+ spin_unlock(&cmd->token_lock);
+
+ return token;
+}
+
+static int xsc_alloc_ent(struct xsc_cmd *cmd)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&cmd->alloc_lock, flags);
+ ret = find_first_bit(&cmd->cmd_entry_mask, cmd->max_reg_cmds);
+ if (ret < cmd->max_reg_cmds)
+ clear_bit(ret, &cmd->cmd_entry_mask);
+ spin_unlock_irqrestore(&cmd->alloc_lock, flags);
+
+ return ret < cmd->max_reg_cmds ? ret : -ENOSPC;
+}
+
+static void xsc_free_ent(struct xsc_cmd *cmd, int idx)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&cmd->alloc_lock, flags);
+ set_bit(idx, &cmd->cmd_entry_mask);
+ spin_unlock_irqrestore(&cmd->alloc_lock, flags);
+}
+
+static struct xsc_cmd_layout *xsc_get_inst(struct xsc_cmd *cmd, int idx)
+{
+ return cmd->cmd_buf + (idx << cmd->log_stride);
+}
+
+static struct xsc_rsp_layout *xsc_get_cq_inst(struct xsc_cmd *cmd, int idx)
+{
+ return cmd->cq_buf + (idx << cmd->log_stride);
+}
+
+static u8 xsc_rsp_owner_bit(struct xsc_rsp_layout *rsp)
+{
+ return !!(rsp->type_owner & XSC_CMD_OWNER_BIT);
+}
+
+static u8 xsc_xor8_buf(void *buf, int len)
+{
+ u8 *ptr = buf;
+ u8 sum = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ sum ^= ptr[i];
+
+ return sum;
+}
+
+static int xsc_verify_block_sig(struct xsc_cmd_prot_block *block)
+{
+ /* rsvd0 was set to 0xFF by fw */
+ if (xsc_xor8_buf(block->rsvd0,
+ sizeof(*block) - sizeof(block->data) - 1) != 0xff)
+ return -EINVAL;
+
+ if (xsc_xor8_buf(block, sizeof(*block)) != 0xff)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void xsc_calc_block_sig(struct xsc_cmd_prot_block *block, u8 token)
+{
+ block->token = token;
+ block->ctrl_sig = ~xsc_xor8_buf(block->rsvd0,
+ sizeof(*block) - sizeof(block->data) - 2);
+ block->sig = ~xsc_xor8_buf(block, sizeof(*block) - 1);
+}
+
+static void xsc_calc_chain_sig(struct xsc_cmd_mailbox *head, u8 token)
+{
+ struct xsc_cmd_mailbox *next = head;
+
+ while (next) {
+ xsc_calc_block_sig(next->buf, token);
+ next = next->next;
+ }
+}
+
+static void xsc_set_signature(struct xsc_cmd_work_ent *ent)
+{
+ ent->lay->sig = ~xsc_xor8_buf(ent->lay, sizeof(*ent->lay));
+ xsc_calc_chain_sig(ent->in->next, ent->token);
+ xsc_calc_chain_sig(ent->out->next, ent->token);
+}
+
+static int xsc_verify_signature(struct xsc_cmd_work_ent *ent)
+{
+ struct xsc_cmd_mailbox *next = ent->out->next;
+ int err;
+ u8 sig;
+
+ sig = xsc_xor8_buf(ent->rsp_lay, sizeof(*ent->rsp_lay));
+ if (sig != 0xff)
+ return -EINVAL;
+
+ while (next) {
+ err = xsc_verify_block_sig(next->buf);
+ if (err)
+ return err;
+
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static const char *xsc_command_str(int command)
+{
+ switch (command) {
+ case XSC_CMD_OP_QUERY_HCA_CAP:
+ return "QUERY_HCA_CAP";
+
+ case XSC_CMD_OP_QUERY_CMDQ_VERSION:
+ return "QUERY_CMDQ_VERSION";
+
+ case XSC_CMD_OP_FUNCTION_RESET:
+ return "FUNCTION_RESET";
+
+ case XSC_CMD_OP_DUMMY:
+ return "DUMMY_CMD";
+
+ case XSC_CMD_OP_CREATE_EQ:
+ return "CREATE_EQ";
+
+ case XSC_CMD_OP_DESTROY_EQ:
+ return "DESTROY_EQ";
+
+ case XSC_CMD_OP_CREATE_CQ:
+ return "CREATE_CQ";
+
+ case XSC_CMD_OP_DESTROY_CQ:
+ return "DESTROY_CQ";
+
+ case XSC_CMD_OP_CREATE_QP:
+ return "CREATE_QP";
+
+ case XSC_CMD_OP_DESTROY_QP:
+ return "DESTROY_QP";
+
+ case XSC_CMD_OP_RST2INIT_QP:
+ return "RST2INIT_QP";
+
+ case XSC_CMD_OP_INIT2RTR_QP:
+ return "INIT2RTR_QP";
+
+ case XSC_CMD_OP_RTR2RTS_QP:
+ return "RTR2RTS_QP";
+
+ case XSC_CMD_OP_RTS2RTS_QP:
+ return "RTS2RTS_QP";
+
+ case XSC_CMD_OP_SQERR2RTS_QP:
+ return "SQERR2RTS_QP";
+
+ case XSC_CMD_OP_2ERR_QP:
+ return "2ERR_QP";
+
+ case XSC_CMD_OP_RTS2SQD_QP:
+ return "RTS2SQD_QP";
+
+ case XSC_CMD_OP_SQD2RTS_QP:
+ return "SQD2RTS_QP";
+
+ case XSC_CMD_OP_2RST_QP:
+ return "2RST_QP";
+
+ case XSC_CMD_OP_INIT2INIT_QP:
+ return "INIT2INIT_QP";
+
+ case XSC_CMD_OP_MODIFY_RAW_QP:
+ return "MODIFY_RAW_QP";
+
+ case XSC_CMD_OP_ENABLE_NIC_HCA:
+ return "ENABLE_NIC_HCA";
+
+ case XSC_CMD_OP_DISABLE_NIC_HCA:
+ return "DISABLE_NIC_HCA";
+
+ case XSC_CMD_OP_QUERY_VPORT_STATE:
+ return "QUERY_VPORT_STATE";
+
+ case XSC_CMD_OP_QUERY_EVENT_TYPE:
+ return "QUERY_EVENT_TYPE";
+
+ case XSC_CMD_OP_ENABLE_MSIX:
+ return "ENABLE_MSIX";
+
+ case XSC_CMD_OP_SET_MTU:
+ return "SET_MTU";
+
+ case XSC_CMD_OP_QUERY_ETH_MAC:
+ return "QUERY_ETH_MAC";
+
+ default: return "unknown command opcode";
+ }
+}
+
+static void cmd_work_handler(struct work_struct *work)
+{
+ struct xsc_cmd_work_ent *ent;
+ struct xsc_core_device *xdev;
+ struct xsc_cmd_layout *lay;
+ struct semaphore *sem;
+ struct xsc_cmd *cmd;
+ unsigned long flags;
+
+ ent = container_of(work, struct xsc_cmd_work_ent, work);
+ cmd = ent->cmd;
+ xdev = container_of(cmd, struct xsc_core_device, cmd);
+ sem = &cmd->sem;
+ down(sem);
+ ent->idx = xsc_alloc_ent(cmd);
+ if (ent->idx < 0) {
+ pci_err(xdev->pdev, "failed to allocate command entry\n");
+ ent->ret = -ENOMEM;
+ complete(&ent->done);
+ up(sem);
+ return;
+ }
+
+ ent->token = xsc_alloc_token(cmd);
+ cmd->ent_arr[ent->idx] = ent;
+
+ spin_lock_irqsave(&cmd->doorbell_lock, flags);
+ lay = xsc_get_inst(cmd, cmd->cmd_pid);
+ ent->lay = lay;
+ memset(lay, 0, sizeof(*lay));
+ memcpy(lay->in, ent->in->first.data, sizeof(lay->in));
+ if (ent->in->next)
+ lay->in_ptr = cpu_to_be64(ent->in->next->dma);
+ lay->inlen = cpu_to_be32(ent->in->len);
+ if (ent->out->next)
+ lay->out_ptr = cpu_to_be64(ent->out->next->dma);
+ lay->outlen = cpu_to_be32(ent->out->len);
+ lay->type_owner = FIELD_PREP(XSC_CMD_TYPE_MASK, XSC_PCI_CMD_XPORT);
+ lay->token = ent->token;
+ lay->idx = ent->idx;
+ if (!cmd->checksum_disabled)
+ xsc_set_signature(ent);
+ else
+ lay->sig = 0xff;
+
+ /* ring doorbell after the descriptor is valid */
+ wmb();
+
+ cmd->cmd_pid = (cmd->cmd_pid + 1) % (1 << cmd->log_sz);
+ xsc_update_cmdq_req_pid(xdev, cmd->cmd_pid);
+ spin_unlock_irqrestore(&cmd->doorbell_lock, flags);
+}
+
+static const char *xsc_deliv_status_to_str(u8 status)
+{
+ switch (status) {
+ case XSC_CMD_DELIVERY_STAT_OK:
+ return "no errors";
+ case XSC_CMD_DELIVERY_STAT_SIGNAT_ERR:
+ return "signature error";
+ case XSC_CMD_DELIVERY_STAT_TOK_ERR:
+ return "token error";
+ case XSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR:
+ return "bad block number";
+ case XSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR:
+ return "output pointer not aligned to block size";
+ case XSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR:
+ return "input pointer not aligned to block size";
+ case XSC_CMD_DELIVERY_STAT_FW_ERR:
+ return "firmware internal error";
+ case XSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR:
+ return "command input length error";
+ case XSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR:
+ return "command output length error";
+ case XSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR:
+ return "reserved fields not cleared";
+ case XSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR:
+ return "bad command descriptor type";
+ default:
+ return "unknown status code";
+ }
+}
+
+static u16 xsc_msg_to_opcode(struct xsc_cmd_msg *in)
+{
+ struct xsc_inbox_hdr *hdr = (struct xsc_inbox_hdr *)(in->first.data);
+
+ return be16_to_cpu(hdr->opcode);
+}
+
+static int xsc_wait_func(struct xsc_core_device *xdev,
+ struct xsc_cmd_work_ent *ent)
+{
+ unsigned long timeout = msecs_to_jiffies(XSC_CMD_TIMEOUT_MSEC);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int err;
+
+ if (!wait_for_completion_timeout(&ent->done, timeout))
+ err = -ETIMEDOUT;
+ else
+ err = ent->ret;
+
+ if (err == -ETIMEDOUT) {
+ cmd->cmd_status = XSC_CMD_STATUS_TIMEDOUT;
+ pci_err(xdev->pdev, "wait for %s(0x%x) response timeout!\n",
+ xsc_command_str(xsc_msg_to_opcode(ent->in)),
+ xsc_msg_to_opcode(ent->in));
+ } else if (err) {
+ pci_err(xdev->pdev, "err %d, delivery status %s(%d)\n", err,
+ xsc_deliv_status_to_str(ent->status), ent->status);
+ }
+
+ return err;
+}
+
+/* Notes:
+ * 1. Callback functions may not sleep
+ * 2. page queue commands do not support asynchrous completion
+ */
+static int xsc_cmd_invoke(struct xsc_core_device *xdev, struct xsc_cmd_msg *in,
+ struct xsc_rsp_msg *out, u8 *status)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_work_ent *ent;
+ int err = 0;
+
+ ent = xsc_alloc_cmd(cmd, in, out);
+ if (IS_ERR(ent))
+ return PTR_ERR(ent);
+
+ init_completion(&ent->done);
+ INIT_WORK(&ent->work, cmd_work_handler);
+ if (!queue_work(cmd->wq, &ent->work)) {
+ pci_err(xdev->pdev, "failed to queue work\n");
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ err = xsc_wait_func(xdev, ent);
+ if (err == -ETIMEDOUT)
+ /* leave the entry for the completion handler; the driver
+ * is marked timed out and rejects all further commands
+ */
+ return err;
+
+ *status = ent->status;
+ kfree(ent);
+
+ return err;
+
+err_free:
+ kfree(ent);
+ return err;
+}
+
+static int xsc_copy_to_cmd_msg(struct xsc_cmd_msg *to, void *from,
+ unsigned int size)
+{
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_mailbox *next;
+ unsigned int copy;
+
+ if (!to || !from)
+ return -ENOMEM;
+
+ copy = min(size, sizeof(to->first.data));
+ memcpy(to->first.data, from, copy);
+ size -= copy;
+ from += copy;
+
+ next = to->next;
+ while (size) {
+ if (!next) {
+ WARN_ONCE(1, "Mail box not enough\n");
+ return -ENOMEM;
+ }
+
+ copy = min(size, XSC_CMD_DATA_BLOCK_SIZE);
+ block = next->buf;
+ memcpy(block->data, from, copy);
+ block->owner_status = 0;
+ from += copy;
+ size -= copy;
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static int xsc_copy_from_rsp_msg(void *to, struct xsc_rsp_msg *from,
+ unsigned int size)
+{
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_mailbox *next;
+ unsigned int copy;
+
+ if (!to || !from)
+ return -ENOMEM;
+
+ copy = min(size, sizeof(from->first.data));
+ memcpy(to, from->first.data, copy);
+ size -= copy;
+ to += copy;
+
+ next = from->next;
+ while (size) {
+ if (!next) {
+ WARN_ONCE(1, "Mail too short\n");
+ return -ENOMEM;
+ }
+
+ copy = min(size, XSC_CMD_DATA_BLOCK_SIZE);
+ block = next->buf;
+ if (!block->owner_status)
+ return -EIO;
+
+ memcpy(to, block->data, copy);
+ to += copy;
+ size -= copy;
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static struct xsc_cmd_mailbox *xsc_alloc_cmd_box(struct xsc_core_device *xdev,
+ gfp_t flags)
+{
+ struct xsc_cmd_mailbox *mailbox;
+
+ mailbox = kmalloc_obj(*mailbox, flags);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+
+ mailbox->buf = dma_pool_zalloc(xdev->cmd.pool, flags,
+ &mailbox->dma);
+ if (!mailbox->buf) {
+ kfree(mailbox);
+ return ERR_PTR(-ENOMEM);
+ }
+ mailbox->next = NULL;
+
+ return mailbox;
+}
+
+static void xsc_free_cmd_box(struct xsc_core_device *xdev,
+ struct xsc_cmd_mailbox *mailbox)
+{
+ dma_pool_free(xdev->cmd.pool, mailbox->buf, mailbox->dma);
+
+ kfree(mailbox);
+}
+
+static struct xsc_cmd_msg *xsc_alloc_cmd_msg(struct xsc_core_device *xdev,
+ gfp_t flags, unsigned int size)
+{
+ struct xsc_cmd_mailbox *tmp, *head = NULL;
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_msg *msg;
+ int blen;
+ int err;
+ int n;
+ int i;
+
+ msg = kzalloc_obj(*msg, GFP_KERNEL);
+ if (!msg)
+ return ERR_PTR(-ENOMEM);
+
+ blen = size - min(sizeof(msg->first.data), size);
+ n = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;
+
+ for (i = 0; i < n; i++) {
+ tmp = xsc_alloc_cmd_box(xdev, flags);
+ if (IS_ERR(tmp)) {
+ pci_err(xdev->pdev, "failed allocating block\n");
+ err = PTR_ERR(tmp);
+ goto err_free;
+ }
+
+ block = tmp->buf;
+ tmp->next = head;
+ block->next = cpu_to_be64(tmp->next ? tmp->next->dma : 0);
+ block->block_num = cpu_to_be32(n - i - 1);
+ head = tmp;
+ }
+ msg->next = head;
+ msg->len = size;
+ return msg;
+
+err_free:
+ while (head) {
+ tmp = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = tmp;
+ }
+ kfree(msg);
+
+ return ERR_PTR(err);
+}
+
+static void xsc_free_cmd_msg(struct xsc_core_device *xdev,
+ struct xsc_cmd_msg *msg)
+{
+ struct xsc_cmd_mailbox *head = msg->next;
+ struct xsc_cmd_mailbox *next;
+
+ while (head) {
+ next = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = next;
+ }
+ kfree(msg);
+}
+
+static struct xsc_rsp_msg *xsc_alloc_rsp_msg(struct xsc_core_device *xdev,
+ gfp_t flags, unsigned int size)
+{
+ struct xsc_cmd_mailbox *tmp, *head = NULL;
+ struct xsc_cmd_prot_block *block;
+ struct xsc_rsp_msg *msg;
+ int blen;
+ int err;
+ int n;
+ int i;
+
+ msg = kzalloc_obj(*msg, GFP_KERNEL);
+ if (!msg)
+ return ERR_PTR(-ENOMEM);
+
+ blen = size - min_t(int, sizeof(msg->first.data), size);
+ n = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;
+
+ for (i = 0; i < n; i++) {
+ tmp = xsc_alloc_cmd_box(xdev, flags);
+ if (IS_ERR(tmp)) {
+ pci_err(xdev->pdev, "failed allocating block\n");
+ err = PTR_ERR(tmp);
+ goto err_free;
+ }
+
+ block = tmp->buf;
+ tmp->next = head;
+ block->next = cpu_to_be64(tmp->next ? tmp->next->dma : 0);
+ block->block_num = cpu_to_be32(n - i - 1);
+ head = tmp;
+ }
+ msg->next = head;
+ msg->len = size;
+ return msg;
+
+err_free:
+ while (head) {
+ tmp = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = tmp;
+ }
+ kfree(msg);
+
+ return ERR_PTR(err);
+}
+
+static void xsc_free_rsp_msg(struct xsc_core_device *xdev,
+ struct xsc_rsp_msg *msg)
+{
+ struct xsc_cmd_mailbox *head = msg->next;
+ struct xsc_cmd_mailbox *next;
+
+ while (head) {
+ next = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = next;
+ }
+ kfree(msg);
+}
+
+static void xsc_set_wqname(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+
+ snprintf(cmd->wq_name, sizeof(cmd->wq_name), "xsc_cmd_%s",
+ dev_name(&xdev->pdev->dev));
+}
+
+void xsc_cmd_use_events(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int i;
+
+ /* flush first: cmd_work_handler() takes a semaphore permit, so
+ * flushing after all permits are held would deadlock against a
+ * just-queued work item
+ */
+ flush_workqueue(cmd->wq);
+
+ /* keep the cmdq interrupt out while the polling kthread is still
+ * draining the queue
+ */
+ disable_irq(cmd->irqn);
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ down(&cmd->sem);
+
+ cmd->mode = XSC_CMD_MODE_EVENTS;
+
+ while (cmd->cmd_pid != cmd->cq_cid)
+ msleep(20);
+ kthread_stop(cmd->cq_task);
+ cmd->cq_task = NULL;
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ up(&cmd->sem);
+
+ enable_irq(cmd->irqn);
+}
+
+static int xsc_cmd_cq_polling(void *data);
+void xsc_cmd_use_polling(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int i;
+
+ flush_workqueue(cmd->wq);
+
+ disable_irq(cmd->irqn);
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ down(&cmd->sem);
+
+ cmd->mode = XSC_CMD_MODE_POLLING;
+ cmd->cq_task = kthread_create(xsc_cmd_cq_polling,
+ (void *)xdev,
+ "xsc_cmd_cq_polling");
+ if (!IS_ERR(cmd->cq_task))
+ wake_up_process(cmd->cq_task);
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ up(&cmd->sem);
+
+ enable_irq(cmd->irqn);
+}
+
+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr)
+{
+ switch (hdr->status) {
+ case XSC_CMD_STATUS_OK:
+ return 0;
+ case XSC_CMD_STATUS_FAIL:
+ return -EIO;
+ case XSC_CMD_STATUS_NOT_SUPPORTED:
+ /* sole -EOPNOTSUPP source; CREATE_CQ_EX compat keys on it */
+ return -EOPNOTSUPP;
+ case XSC_CMD_STATUS_BAD_PARAM:
+ return -EINVAL;
+ case XSC_CMD_STATUS_INVAL_RES:
+ case XSC_CMD_STATUS_INVAL_DATA:
+ case XSC_CMD_STATUS_NO_RES:
+ case XSC_CMD_STATUS_NO_MPT_RES:
+ case XSC_CMD_STATUS_NO_MTT_RES:
+ case XSC_CMD_STATUS_NO_EQN_RES:
+ case XSC_CMD_STATUS_NO_EQ_PA_RES:
+ case XSC_CMD_STATUS_NO_CQN_RES:
+ case XSC_CMD_STATUS_NO_CQ_PA_RES:
+ case XSC_CMD_STATUS_NO_QPN_RES:
+ case XSC_CMD_STATUS_NO_QP_PA_RES:
+ case XSC_CMD_STATUS_NO_PDN_RES:
+ case XSC_CMD_STATUS_BAD_INBUF:
+ case XSC_CMD_STATUS_BAD_OUTBUF:
+ return -EIO;
+ case XSC_CMD_STATUS_BUSY:
+ case XSC_CMD_STATUS_QP_FLUSH_BUSY:
+ return -EBUSY;
+ case XSC_CMD_STATUS_QP_FLUSH_PENDING:
+ return -EAGAIN;
+ case XSC_CMD_STATUS_NOT_FOUND:
+ case XSC_CMD_STATUS_INVAL_FUNC:
+ return -ENODEV;
+ default:
+ return -EIO;
+ }
+}
+
+static struct xsc_cmd_msg *xsc_alloc_msg(struct xsc_core_device *xdev,
+ unsigned int in_size)
+{
+ struct xsc_cmd_msg *msg = ERR_PTR(-ENOMEM);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct cache_ent *ent = NULL;
+
+ if (in_size > MED_LIST_SIZE && in_size <= LONG_LIST_SIZE)
+ ent = &cmd->cache.large;
+ else if (in_size > 16 && in_size <= MED_LIST_SIZE)
+ ent = &cmd->cache.med;
+
+ if (ent) {
+ spin_lock(&ent->lock);
+ if (!list_empty(&ent->head)) {
+ msg = list_entry(ent->head.next, typeof(*msg), list);
+ /* For cached lists, we must explicitly state what is
+ * the real size
+ */
+ msg->len = in_size;
+ list_del(&msg->list);
+ }
+ spin_unlock(&ent->lock);
+ }
+
+ if (IS_ERR(msg))
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, in_size);
+
+ return msg;
+}
+
+static void xsc_free_msg(struct xsc_core_device *xdev, struct xsc_cmd_msg *msg)
+{
+ if (msg->cache) {
+ spin_lock(&msg->cache->lock);
+ list_add_tail(&msg->list, &msg->cache->head);
+ spin_unlock(&msg->cache->lock);
+ } else {
+ xsc_free_cmd_msg(xdev, msg);
+ }
+}
+
+static int _xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_msg *outb;
+ struct xsc_cmd_msg *inb;
+ u8 status = 0;
+ int err;
+
+ if (cmd->cmd_status == XSC_CMD_STATUS_TIMEDOUT)
+ return -ETIMEDOUT;
+
+ inb = xsc_alloc_msg(xdev, in_size);
+ if (IS_ERR(inb)) {
+ err = PTR_ERR(inb);
+ return err;
+ }
+
+ err = xsc_copy_to_cmd_msg(inb, in, in_size);
+ if (err) {
+ pci_err(xdev->pdev, "copy to cmd_msg err %d\n", err);
+ goto err_free_msg;
+ }
+
+ outb = xsc_alloc_rsp_msg(xdev, GFP_KERNEL, out_size);
+ if (IS_ERR(outb)) {
+ err = PTR_ERR(outb);
+ goto err_free_msg;
+ }
+
+ err = xsc_cmd_invoke(xdev, inb, outb, &status);
+ if (err == -ETIMEDOUT)
+ /* the entry stays registered for the completion handler
+ * and keeps its mailboxes
+ */
+ return err;
+
+ if (err)
+ goto err_free_rsp;
+
+ if (status) {
+ pci_err(xdev->pdev, "opcode:%#x, err %d, status %d\n",
+ xsc_msg_to_opcode(inb), err, status);
+ xsc_copy_from_rsp_msg(out, outb, out_size);
+ err = -EREMOTEIO;
+ goto err_free_rsp;
+ }
+
+ err = xsc_copy_from_rsp_msg(out, outb, out_size);
+
+err_free_rsp:
+ xsc_free_rsp_msg(xdev, outb);
+
+err_free_msg:
+ xsc_free_msg(xdev, inb);
+ return err;
+}
+
+int xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size)
+{
+ int err;
+
+ err = _xsc_cmd_exec(xdev, in, in_size, out, out_size);
+ if (err == -EREMOTEIO)
+ err = xsc_cmd_status_to_err((struct xsc_outbox_hdr *)out);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_cmd_exec);
+
+static void xsc_destroy_msg_cache(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_msg *msg;
+ struct xsc_cmd_msg *n;
+
+ list_for_each_entry_safe(msg, n, &cmd->cache.large.head, list) {
+ list_del(&msg->list);
+ xsc_free_cmd_msg(xdev, msg);
+ }
+
+ list_for_each_entry_safe(msg, n, &cmd->cache.med.head, list) {
+ list_del(&msg->list);
+ xsc_free_cmd_msg(xdev, msg);
+ }
+}
+
+static int xsc_create_msg_cache(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_msg *msg;
+ int err;
+ int i;
+
+ spin_lock_init(&cmd->cache.large.lock);
+ INIT_LIST_HEAD(&cmd->cache.large.head);
+ spin_lock_init(&cmd->cache.med.lock);
+ INIT_LIST_HEAD(&cmd->cache.med.head);
+
+ for (i = 0; i < NUM_LONG_LISTS; i++) {
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, LONG_LIST_SIZE);
+ if (IS_ERR(msg)) {
+ err = PTR_ERR(msg);
+ goto err_destroy;
+ }
+ msg->cache = &cmd->cache.large;
+ list_add_tail(&msg->list, &cmd->cache.large.head);
+ }
+
+ for (i = 0; i < NUM_MED_LISTS; i++) {
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, MED_LIST_SIZE);
+ if (IS_ERR(msg)) {
+ err = PTR_ERR(msg);
+ goto err_destroy;
+ }
+ msg->cache = &cmd->cache.med;
+ list_add_tail(&msg->list, &cmd->cache.med.head);
+ }
+
+ return 0;
+
+err_destroy:
+ xsc_destroy_msg_cache(xdev);
+ return err;
+}
+
+static void xsc_cmd_comp_handler(struct xsc_core_device *xdev, u8 idx,
+ struct xsc_rsp_layout *rsp)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_work_ent *ent;
+
+ if (idx >= cmd->max_reg_cmds || (cmd->cmd_entry_mask & BIT(idx))) {
+ pci_err(xdev->pdev, "idx[%d] exceed max cmds, or has no relative request.\n",
+ idx);
+ return;
+ }
+ ent = cmd->ent_arr[idx];
+ ent->rsp_lay = rsp;
+
+ memcpy(ent->out->first.data,
+ ent->rsp_lay->out,
+ sizeof(ent->rsp_lay->out));
+ if (!cmd->checksum_disabled)
+ ent->ret = xsc_verify_signature(ent);
+ else
+ ent->ret = 0;
+ ent->status = ((struct xsc_outbox_hdr *)ent->out->first.data)->status;
+ cmd->retry_cnt = 0;
+
+ xsc_free_ent(cmd, ent->idx);
+ complete(&ent->done);
+ up(&cmd->sem);
+}
+
+static int xsc_cmd_cq_polling(void *data)
+{
+ struct xsc_core_device *xdev = data;
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_layout *rsp;
+ u32 cq_pid;
+
+ while (!kthread_should_stop()) {
+ if (need_resched())
+ schedule();
+ cq_pid = xsc_get_cmdq_rsp_pid(xdev);
+ if (cmd->cq_cid == cq_pid) {
+ usleep_range(1000, 2000);
+ continue;
+ }
+
+ rsp = xsc_get_cq_inst(cmd, cmd->cq_cid);
+ if (!cmd->ownerbit_learned) {
+ cmd->ownerbit_learned = 1;
+ cmd->owner_bit = xsc_rsp_owner_bit(rsp);
+ }
+ if (cmd->owner_bit != xsc_rsp_owner_bit(rsp)) {
+ dev_err_ratelimited(&xdev->pdev->dev,
+ "hw update cq doorbell but buf not ready %u %u\n",
+ cmd->cq_cid, cq_pid);
+ msleep(20);
+ continue;
+ }
+
+ /* the hardware updates the payload after flipping the
+ * owner bit; order the payload reads behind the check
+ */
+ dma_rmb();
+
+ xsc_cmd_comp_handler(xdev, rsp->idx, rsp);
+
+ cmd->cq_cid = (cmd->cq_cid + 1) % (1 << cmd->log_sz);
+
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+ if (cmd->cq_cid == 0)
+ cmd->owner_bit = !cmd->owner_bit;
+ }
+
+ return 0;
+}
+
+int xsc_cmd_err_handler(struct xsc_core_device *xdev)
+{
+ return xsc_handle_cmdq_interrupt(xdev);
+}
+
+void xsc_cmd_resp_handler(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_layout *rsp;
+ const int budget = 32;
+ int count = 0;
+ u32 cq_pid;
+
+ while (count < budget) {
+ cq_pid = xsc_get_cmdq_rsp_pid(xdev);
+ if (cq_pid == cmd->cq_cid)
+ return;
+
+ rsp = xsc_get_cq_inst(cmd, cmd->cq_cid);
+ if (!cmd->ownerbit_learned) {
+ cmd->ownerbit_learned = 1;
+ cmd->owner_bit = xsc_rsp_owner_bit(rsp);
+ }
+ if (cmd->owner_bit != xsc_rsp_owner_bit(rsp)) {
+ pci_err(xdev->pdev, "hw update cq doorbell but buf not ready %u %u\n",
+ cmd->cq_cid, cq_pid);
+ return;
+ }
+
+ dma_rmb();
+
+ xsc_cmd_comp_handler(xdev, rsp->idx, rsp);
+
+ cmd->cq_cid = (cmd->cq_cid + 1) % (1 << cmd->log_sz);
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+ if (cmd->cq_cid == 0)
+ cmd->owner_bit = !cmd->owner_bit;
+
+ count++;
+ }
+}
+
+static void xsc_cmd_handle_rsp_before_reload
+(struct xsc_cmd *cmd, struct xsc_core_device *xdev)
+{
+ u32 rsp_pid, rsp_cid;
+
+ rsp_pid = xsc_get_cmdq_rsp_pid(xdev) & XSC_CMDQ_INDEX_MASK;
+ rsp_cid = xsc_get_cmdq_rsp_cid(xdev) & XSC_CMDQ_INDEX_MASK;
+ if (rsp_pid == rsp_cid)
+ return;
+
+ cmd->cq_cid = rsp_pid;
+
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+}
+
+int xsc_cmd_init(struct xsc_core_device *xdev)
+{
+ unsigned int size = sizeof(struct xsc_cmd_prot_block);
+ int align = roundup_pow_of_two(size);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ u32 cmd_h, cmd_l;
+ int err;
+
+ cmd->pool = dma_pool_create("xsc_cmd",
+ &xdev->pdev->dev,
+ size, align, 0);
+ if (!cmd->pool)
+ return -ENOMEM;
+
+ cmd->cmd_buf = dma_alloc_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ &cmd->dma, GFP_KERNEL);
+ if (!cmd->cmd_buf) {
+ err = -ENOMEM;
+ goto err_free_pool;
+ }
+ cmd->cq_buf = dma_alloc_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ &cmd->cq_dma, GFP_KERNEL);
+ if (!cmd->cq_buf) {
+ err = -ENOMEM;
+ goto err_free_cmd;
+ }
+
+ memset(cmd->cmd_buf, 0, PAGE_SIZE);
+ memset(cmd->cq_buf, 0, PAGE_SIZE);
+
+ cmd->cmd_pid = xsc_get_cmdq_req_pid(xdev) & XSC_CMDQ_INDEX_MASK;
+ cmd->cq_cid = xsc_get_cmdq_rsp_cid(xdev) & XSC_CMDQ_INDEX_MASK;
+ cmd->ownerbit_learned = 0;
+
+ xsc_cmd_handle_rsp_before_reload(cmd, xdev);
+
+#define ELEMENT_SIZE_LOG 6 /* 64B */
+#define Q_DEPTH_LOG 5 /* 32 */
+
+ cmd->log_sz = Q_DEPTH_LOG;
+ cmd->log_stride = ELEMENT_SIZE_LOG;
+
+ if (1 << cmd->log_sz > XSC_MAX_COMMANDS) {
+ dev_err(&xdev->pdev->dev, "firmware reports too many outstanding commands %d\n",
+ 1 << cmd->log_sz);
+ err = -EINVAL;
+ goto err_free_cq;
+ }
+
+ if (cmd->log_sz + cmd->log_stride > PAGE_SHIFT) {
+ dev_err(&xdev->pdev->dev, "command queue size overflow\n");
+ err = -EINVAL;
+ goto err_free_cq;
+ }
+
+ cmd->checksum_disabled = 1;
+ cmd->max_reg_cmds = 1 << cmd->log_sz;
+ bitmap_fill(&cmd->cmd_entry_mask, cmd->max_reg_cmds);
+
+ spin_lock_init(&cmd->alloc_lock);
+ spin_lock_init(&cmd->token_lock);
+ spin_lock_init(&cmd->doorbell_lock);
+
+ sema_init(&cmd->sem, cmd->max_reg_cmds);
+
+ cmd_h = upper_32_bits(cmd->dma);
+ cmd_l = lower_32_bits(cmd->dma);
+ if (cmd_l & 0xfff) {
+ dev_err(&xdev->pdev->dev, "invalid command queue address\n");
+ err = -ENOMEM;
+ goto err_free_cq;
+ }
+
+ xsc_set_cmdq_req_buf_addr(xdev, cmd_h, cmd_l);
+
+ cmd_h = upper_32_bits(cmd->cq_dma);
+ cmd_l = lower_32_bits(cmd->cq_dma);
+ if (cmd_l & 0xfff) {
+ dev_err(&xdev->pdev->dev, "invalid command queue address\n");
+ err = -ENOMEM;
+ goto err_free_cq;
+ }
+ xsc_set_cmdq_rsp_buf_addr(xdev, cmd_h, cmd_l);
+
+ /* Make sure firmware sees the complete address before we proceed */
+ wmb();
+
+ cmd->mode = XSC_CMD_MODE_POLLING;
+ cmd->cmd_status = XSC_CMD_STATUS_NORMAL;
+
+ err = xsc_create_msg_cache(xdev);
+ if (err) {
+ dev_err(&xdev->pdev->dev, "failed to create command cache\n");
+ goto err_free_cq;
+ }
+
+ xsc_set_wqname(xdev);
+ cmd->wq = create_singlethread_workqueue(cmd->wq_name);
+ if (!cmd->wq) {
+ dev_err(&xdev->pdev->dev, "failed to create command workqueue\n");
+ err = -ENOMEM;
+ goto err_destroy_cache;
+ }
+
+ cmd->cq_task = kthread_create(xsc_cmd_cq_polling,
+ (void *)xdev,
+ "xsc_cmd_cq_polling");
+ if (IS_ERR(cmd->cq_task)) {
+ dev_err(&xdev->pdev->dev, "failed to create cq task\n");
+ err = PTR_ERR(cmd->cq_task);
+ goto err_destroy_wq;
+ }
+ wake_up_process(cmd->cq_task);
+
+ /* clear abnormal state to avoid the impact of previous error */
+ xsc_check_cmdq_status(xdev);
+
+ return 0;
+
+err_destroy_wq:
+ destroy_workqueue(cmd->wq);
+
+err_destroy_cache:
+ xsc_destroy_msg_cache(xdev);
+
+err_free_cq:
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cq_buf,
+ cmd->cq_dma);
+
+err_free_cmd:
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cmd_buf,
+ cmd->dma);
+
+err_free_pool:
+ dma_pool_destroy(cmd->pool);
+
+ return err;
+}
+
+void xsc_cmd_cleanup(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+
+ destroy_workqueue(cmd->wq);
+ if (!IS_ERR_OR_NULL(cmd->cq_task))
+ kthread_stop(cmd->cq_task);
+ xsc_destroy_msg_cache(xdev);
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cq_buf,
+ cmd->cq_dma);
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cmd_buf,
+ cmd->dma);
+ dma_pool_destroy(cmd->pool);
+}
+
+int xsc_cmd_version_check(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_query_cmdq_ver_mbox_out *out;
+ struct xsc_cmd_query_cmdq_ver_mbox_in in;
+
+ int err;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_CMDQ_VERSION);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err)
+ goto err_free;
+
+ if (be16_to_cpu(out->cmdq_ver) != XSC_CMDQ_VERSION) {
+ pci_err(xdev->pdev, "cmdq version check failed, expecting %d, actual %d\n",
+ XSC_CMDQ_VERSION, be16_to_cpu(out->cmdq_ver));
+ err = -EINVAL;
+ goto err_free;
+ }
+ xdev->cmdq_ver = XSC_CMDQ_VERSION;
+
+err_free:
+ kfree(out);
+ return err;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
new file mode 100644
index 000000000..0358a40cc
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
@@ -0,0 +1,316 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/vmalloc.h>
+
+#include "common/xsc_cmd_api.h"
+#include "hw.h"
+#include "hal/xsc_hal.h"
+
+#define MAX_BOARD_NUM 32
+
+static struct xsc_board_info *board_info[MAX_BOARD_NUM];
+static DEFINE_MUTEX(board_info_lock);
+
+static struct xsc_board_info *xsc_get_board_info(char *board_sn)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++) {
+ if (!board_info[i])
+ continue;
+ if (!strncmp(board_info[i]->board_sn, board_sn,
+ XSC_BOARD_SN_LEN))
+ return board_info[i];
+ }
+
+ return NULL;
+}
+
+static struct xsc_board_info *xsc_alloc_board_info(void)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++) {
+ if (!board_info[i])
+ break;
+ }
+ if (i == MAX_BOARD_NUM)
+ return NULL;
+ board_info[i] = vzalloc(sizeof(*board_info[i]));
+ if (!board_info[i])
+ return NULL;
+ board_info[i]->board_id = i;
+ return board_info[i];
+}
+
+void xsc_free_board_info(void)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++)
+ vfree(board_info[i]);
+}
+
+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,
+ struct xsc_caps *caps)
+{
+ struct xsc_cmd_query_hca_cap_mbox_out *out;
+ struct xsc_cmd_query_hca_cap_mbox_in in;
+ struct xsc_board_info *board_info;
+ int err;
+ u16 t16;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HCA_CAP);
+ in.cpu_num = cpu_to_be16(num_online_cpus());
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err)
+ goto err_free;
+
+ xdev->glb_func_id = be32_to_cpu(out->hca_cap.glb_func_id);
+ caps->pcie0_pf_funcid_base =
+ be16_to_cpu(out->hca_cap.pcie0_pf_funcid_base);
+ caps->pcie0_pf_funcid_top =
+ be16_to_cpu(out->hca_cap.pcie0_pf_funcid_top);
+ caps->pcie1_pf_funcid_base =
+ be16_to_cpu(out->hca_cap.pcie1_pf_funcid_base);
+ caps->pcie1_pf_funcid_top =
+ be16_to_cpu(out->hca_cap.pcie1_pf_funcid_top);
+ caps->funcid_to_logic_port =
+ be16_to_cpu(out->hca_cap.funcid_to_logic_port);
+
+ caps->pcie_host = out->hca_cap.pcie_host;
+ caps->nif_port_num = out->hca_cap.nif_port_num;
+ caps->hw_feature_flag = be32_to_cpu(out->hca_cap.hw_feature_flag);
+
+ caps->func_caps = (caps->func_caps & ~XSC_CAP_RAWETH_QP_ID_BASE_MASK) |
+ FIELD_PREP(XSC_CAP_RAWETH_QP_ID_BASE_MASK,
+ be16_to_cpu(out->hca_cap.raweth_qp_id_base));
+ caps->raweth_qp_id_end = be16_to_cpu(out->hca_cap.raweth_qp_id_end);
+ caps->nic_caps = (caps->nic_caps & ~XSC_CAP_RAWETH_RSS_QP_ID_BASE_MASK) |
+ FIELD_PREP(XSC_CAP_RAWETH_RSS_QP_ID_BASE_MASK,
+ be16_to_cpu(out->hca_cap.raweth_rss_qp_id_base));
+ caps->qos_caps = (caps->qos_caps & ~XSC_CAP_RAW_TPE_QP_NUM_MASK) |
+ FIELD_PREP(XSC_CAP_RAW_TPE_QP_NUM_MASK,
+ be16_to_cpu(out->hca_cap.raw_tpe_qp_num));
+ caps->max_cqes = 1 << out->hca_cap.log_max_cq_sz;
+ caps->max_wqes = 1 << out->hca_cap.log_max_qp_sz;
+ caps->max_sq_desc_sz = be16_to_cpu(out->hca_cap.max_desc_sz_sq);
+ caps->max_rq_desc_sz = be16_to_cpu(out->hca_cap.max_desc_sz_rq);
+ caps->flags = be64_to_cpu(out->hca_cap.flags);
+ caps->stat_rate_support = be16_to_cpu(out->hca_cap.stat_rate_support);
+ caps->log_max_msg = out->hca_cap.log_max_msg & 0x1f;
+ caps->num_ports = out->hca_cap.num_ports & 0xf;
+ caps->log_max_cq = out->hca_cap.log_max_cq & 0x1f;
+ caps->log_max_eq = out->hca_cap.log_max_eq & 0xf;
+ caps->log_max_msix = out->hca_cap.log_max_msix & 0xf;
+ caps->nic_caps = (caps->nic_caps & ~XSC_CAP_MAC_PORT_MASK) |
+ FIELD_PREP(XSC_CAP_MAC_PORT_MASK,
+ out->hca_cap.mac_port & 0xff);
+ xdev->mac_port = FIELD_GET(XSC_CAP_MAC_PORT_MASK, caps->nic_caps);
+ if (caps->num_ports > XSC_MAX_FW_PORTS) {
+ pci_err(xdev->pdev, "device has %d ports while the driver supports max %d ports\n",
+ caps->num_ports, XSC_MAX_FW_PORTS);
+ err = -EINVAL;
+ goto err_free;
+ }
+ caps->send_ds_num = out->hca_cap.send_seg_num;
+ caps->send_wqe_shift = out->hca_cap.send_wqe_shift;
+ caps->recv_ds_num = out->hca_cap.recv_seg_num;
+ caps->recv_wqe_shift = out->hca_cap.recv_wqe_shift;
+
+ caps->func_caps &= ~XSC_CAP_EMBEDDED_CPU;
+ caps->func_caps &= ~XSC_CAP_ECPF_VPORT_EXISTS;
+ caps->log_max_current_uc_list = 0;
+ caps->log_max_current_mc_list = 0;
+ caps->log_max_vlan_list = 8;
+ caps->log_max_qp = out->hca_cap.log_max_qp & 0x1f;
+ caps->log_max_mkey = out->hca_cap.log_max_mkey & 0x3f;
+ caps->log_max_pd = out->hca_cap.log_max_pd & 0x1f;
+ caps->log_max_srq = out->hca_cap.log_max_srqs & 0x1f;
+ caps->local_ca_ack_delay = out->hca_cap.local_ca_ack_delay & 0x1f;
+ caps->log_max_mcg = out->hca_cap.log_max_mcg;
+ caps->log_max_mtt = out->hca_cap.log_max_mtt;
+ caps->log_max_tso = out->hca_cap.log_max_tso;
+ caps->hca_core_clock = be32_to_cpu(out->hca_cap.hca_core_clock);
+ caps->max_rwq_indirection_tables =
+ be32_to_cpu(out->hca_cap.max_rwq_indirection_tables);
+ caps->max_rwq_indirection_table_size =
+ be32_to_cpu(out->hca_cap.max_rwq_indirection_table_size);
+ caps->max_qp_mcg = be16_to_cpu(out->hca_cap.max_qp_mcg);
+ caps->max_ra_res_qp = 1 << (out->hca_cap.log_max_ra_res_qp & 0x3f);
+ caps->max_ra_req_qp = 1 << (out->hca_cap.log_max_ra_req_qp & 0x3f);
+ caps->max_srq_wqes = 1 << out->hca_cap.log_max_srq_sz;
+ caps->rx_pkt_len_max = be32_to_cpu(out->hca_cap.rx_pkt_len_max);
+ caps->max_vfs = be16_to_cpu(out->hca_cap.max_vfs);
+ caps->qp_rate_limit_min = be32_to_cpu(out->hca_cap.qp_rate_limit_min);
+ caps->qp_rate_limit_max = be32_to_cpu(out->hca_cap.qp_rate_limit_max);
+
+ caps->msix_base = be16_to_cpu(out->hca_cap.msix_base);
+ caps->msix_num = be16_to_cpu(out->hca_cap.msix_num);
+
+ t16 = be16_to_cpu(out->hca_cap.bf_log_bf_reg_size);
+ if (t16 & 0x8000) {
+ caps->bf_reg_size = 1 << (t16 & 0x1f);
+ caps->bf_regs_per_page = XSC_BF_REGS_PER_PAGE;
+ } else {
+ caps->bf_reg_size = 0;
+ caps->bf_regs_per_page = 0;
+ }
+ caps->min_page_sz = ~(u32)((1 << PAGE_SHIFT) - 1);
+
+ caps->qos_caps |= XSC_CAP_DCBX;
+ caps->qos_caps |= XSC_CAP_QOS;
+ caps->qos_caps |= XSC_CAP_ETS;
+ caps->qos_caps |= XSC_CAP_DSCP;
+ caps->qos_caps = (caps->qos_caps & ~XSC_CAP_MAX_TC_MASK) |
+ FIELD_PREP(XSC_CAP_MAX_TC_MASK, out->hca_cap.max_tc);
+ caps->log_max_qp_depth = out->hca_cap.log_max_qp_depth & 0xff;
+ caps->mac_bit = out->hca_cap.mac_bit;
+ caps->lag_logic_port_ofst = out->hca_cap.lag_logic_port_ofst;
+
+ caps->max_qp = be32_to_cpu(out->hca_cap.max_qp);
+ caps->max_cq = be32_to_cpu(out->hca_cap.max_cq);
+ caps->max_pd = be32_to_cpu(out->hca_cap.max_pd);
+
+ if (be16_to_cpu(out->hdr.ver) >= XSC_CMD_QUERY_HCA_CAP_V2)
+ caps->max_mtt = be32_to_cpu(out->hca_cap.max_mtt);
+ else
+ caps->max_mtt = 1 << (out->hca_cap.log_max_mtt & 0x3f);
+
+ if (be16_to_cpu(out->hdr.ver) >= XSC_CMD_QUERY_HCA_CAP_V1) {
+ caps->max_mr_size = be64_to_cpu(out->hca_cap.max_mr_size);
+ caps->max_cmd_in_len = be16_to_cpu(out->hca_cap.max_cmd_in_len);
+ caps->max_cmd_out_len = be16_to_cpu(out->hca_cap.max_cmd_out_len);
+ } else {
+ caps->max_mr_size = (1ull << 32) - 1;
+ caps->max_cmd_in_len = 10240;
+ caps->max_cmd_out_len = 2048;
+ }
+
+ if (be16_to_cpu(out->hdr.ver) >= XSC_CMD_QUERY_HCA_CAP_V3) {
+ caps->mpt_tbl_addr = be32_to_cpu(out->hca_cap.mpt_tbl_addr);
+ caps->mpt_tbl_depth = be32_to_cpu(out->hca_cap.mpt_tbl_depth);
+ caps->mpt_tbl_width = be32_to_cpu(out->hca_cap.mpt_tbl_width);
+ caps->mtt_inst_base_addr = be32_to_cpu(out->hca_cap.mtt_inst_base_addr);
+ caps->mtt_inst_stride = be32_to_cpu(out->hca_cap.mtt_inst_stride);
+ caps->mtt_inst_num_log = be32_to_cpu(out->hca_cap.mtt_inst_num_log);
+ caps->mtt_inst_depth = be32_to_cpu(out->hca_cap.mtt_inst_depth);
+ xsc_set_mtt_info(xdev);
+ }
+
+ xdev->pcie_no = out->hca_cap.pcie_no;
+ xdev->pf_id = out->hca_cap.pf_id;
+
+ xdev->chip_ver_h = be32_to_cpu(out->hca_cap.chip_ver_h);
+ xdev->chip_ver_m = be32_to_cpu(out->hca_cap.chip_ver_m);
+ xdev->chip_ver_l = be32_to_cpu(out->hca_cap.chip_ver_l);
+ xdev->hotfix_num = be32_to_cpu(out->hca_cap.hotfix_num);
+ xdev->feature_flag = be32_to_cpu(out->hca_cap.feature_flag);
+
+ mutex_lock(&board_info_lock);
+ board_info = xsc_get_board_info(out->hca_cap.board_sn);
+ if (!board_info) {
+ board_info = xsc_alloc_board_info();
+ if (!board_info) {
+ err = -ENOMEM;
+ mutex_unlock(&board_info_lock);
+ goto err_free;
+ }
+
+ memcpy(board_info->board_sn,
+ out->hca_cap.board_sn,
+ sizeof(out->hca_cap.board_sn));
+ }
+ mutex_unlock(&board_info_lock);
+
+ xdev->board_info = board_info;
+
+ xsc_set_pf_db_addr(xdev, be64_to_cpu(out->hca_cap.tx_db),
+ be64_to_cpu(out->hca_cap.rx_db),
+ be64_to_cpu(out->hca_cap.complete_db),
+ be64_to_cpu(out->hca_cap.complete_reg),
+ be64_to_cpu(out->hca_cap.event_db));
+
+ xdev->fw_version_major = out->hca_cap.fw_ver.fw_version_major;
+ xdev->fw_version_minor = out->hca_cap.fw_ver.fw_version_minor;
+ xdev->fw_version_patch =
+ be16_to_cpu(out->hca_cap.fw_ver.fw_version_patch);
+ xdev->fw_version_tweak =
+ be32_to_cpu(out->hca_cap.fw_ver.fw_version_tweak);
+ xdev->fw_version_extra_flag = out->hca_cap.fw_ver.fw_version_extra_flag;
+
+ kfree(out);
+ return 0;
+err_free:
+ kfree(out);
+ return err;
+}
+
+static int xsc_cmd_query_guid(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_query_guid_mbox_out out;
+ struct xsc_cmd_query_guid_mbox_in in = {};
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_GUID);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ xdev->board_info->guid = out.guid;
+ xdev->board_info->guid_valid = 1;
+ return 0;
+}
+
+int xsc_query_guid(struct xsc_core_device *xdev)
+{
+ if (xdev->board_info->guid_valid)
+ return 0;
+
+ return xsc_cmd_query_guid(xdev);
+}
+
+static int xsc_cmd_activate_hw_config(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_activate_hw_config_mbox_out out;
+ struct xsc_cmd_activate_hw_config_mbox_in in = {};
+ int err = 0;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ACTIVATE_HW_CONFIG);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ xdev->board_info->hw_config_activated = 1;
+ return 0;
+}
+
+int xsc_activate_hw_config(struct xsc_core_device *xdev)
+{
+ if (xdev->board_info->hw_config_activated)
+ return 0;
+
+ return xsc_cmd_activate_hw_config(xdev);
+}
+
+int xsc_reset_function_resource(struct xsc_core_device *xdev)
+{
+ struct xsc_function_reset_mbox_out out;
+ struct xsc_function_reset_mbox_in in;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_FUNCTION_RESET);
+ in.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h
new file mode 100644
index 000000000..5843a72c8
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_HW_H
+#define __XSC_HW_H
+
+#include "common/xsc_core.h"
+
+void xsc_free_board_info(void);
+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,
+ struct xsc_caps *caps);
+int xsc_query_guid(struct xsc_core_device *xdev);
+int xsc_activate_hw_config(struct xsc_core_device *xdev);
+int xsc_reset_function_resource(struct xsc_core_device *xdev);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
index ab428d20f..53dd8fdf1 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -4,6 +4,8 @@
*/
#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "hw.h"
#include "hal/xsc_hal.h"
static const struct pci_device_id xsc_pci_id_table[] = {
@@ -141,11 +143,63 @@ static void xsc_core_dev_cleanup(struct xsc_core_device *xdev)
static int xsc_hw_setup(struct xsc_core_device *xdev)
{
+ int err;
+
+ err = xsc_cmd_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed initializing command interface, aborting\n");
+ goto err_out;
+ }
+
+ err = xsc_cmd_version_check(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to check cmdq version\n");
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_cmd_query_hca_cap(xdev, &xdev->caps);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to query hca, err=%d\n", err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_query_guid(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to query guid, err=%d\n", err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_activate_hw_config(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to activate hw config, err=%d\n",
+ err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_reset_function_resource(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to reset function resource\n");
+ goto err_cmd_cleanup;
+ }
+
+ /* Required by the rdma client, which is probed right after this. */
+ xdev->board_info->msg_opcode = xsc_get_msg_opcode(xdev);
+ if (!xdev->board_info->msg_opcode) {
+ pci_err(xdev->pdev, "failed to get msg opcode table\n");
+ err = -ENODEV;
+ goto err_cmd_cleanup;
+ }
+
return 0;
+err_cmd_cleanup:
+ xsc_cmd_cleanup(xdev);
+err_out:
+ return err;
}
static void xsc_hw_cleanup(struct xsc_core_device *xdev)
{
+ xsc_cmd_cleanup(xdev);
}
static int xsc_load(struct xsc_core_device *xdev)
@@ -247,6 +301,7 @@ static int __init xsc_init(void)
static void __exit xsc_fini(void)
{
pci_unregister_driver(&xsc_pci_driver);
+ xsc_free_board_info();
}
module_init(xsc_init);
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 04/15] net/xsc: add the command queue
2026-11-04 15:57 ` [PATCH v1 04/15] net/xsc: add the command queue Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [High] net/xsc: leaked command resources and wq hang on timeout in _xsc_cmd_exec()
- [High] net/xsc: IRQ deadlock and unchecked kthread_create() in xsc_cmd_use_polling()
- [High] net/xsc: infinite loop and premature up() in xsc_cmd_use_events()
- [Medium] net/xsc: wrong 16-byte inline threshold in xsc_alloc_msg() corrupts commands
- [Medium] net/xsc: broken CQ owner-bit learning in xsc_cmd_cq_polling()
- [Medium] net/xsc: unsynchronized shared board_info access in xsc_hw_setup()
- [Medium] net/xsc: unguarded V2 capability fields and 32-bit shifts in xsc_cmd_query_hca_cap()
--
commit bb1aecb9bb4cc115b4d4581a9902a3b7bb2ae064
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add the command queue
This commit adds the firmware command queue implementation, mailbox memory
management, and initial hardware capability and board configuration queries
for the Yunsilicon XSC PCI driver.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
> new file mode 100644
> index 0000000000000..c001b312224d6
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
> @@ -0,0 +1,1244 @@
[ ... ]
> +enum {
> + NUM_LONG_LISTS = 2,
> + NUM_MED_LISTS = 64,
> + LONG_LIST_SIZE = (2ULL * 1024 * 1024 * 1024 / PAGE_SIZE) * 8 + 16 +
> + XSC_CMD_DATA_BLOCK_SIZE,
> + MED_LIST_SIZE = 16 + XSC_CMD_DATA_BLOCK_SIZE,
> +};
[ ... ]
> +static int xsc_cmd_invoke(struct xsc_core_device *xdev, struct xsc_cmd_msg *in,
> + struct xsc_rsp_msg *out, u8 *status)
> +{
> + struct xsc_cmd *cmd = &xdev->cmd;
> + struct xsc_cmd_work_ent *ent;
> + int err = 0;
> +
> + ent = xsc_alloc_cmd(cmd, in, out);
> + if (IS_ERR(ent))
> + return PTR_ERR(ent);
> +
> + init_completion(&ent->done);
> + INIT_WORK(&ent->work, cmd_work_handler);
> + if (!queue_work(cmd->wq, &ent->work)) {
> + pci_err(xdev->pdev, "failed to queue work\n");
> + err = -ENOMEM;
> + goto err_free;
> + }
> +
> + err = xsc_wait_func(xdev, ent);
> + if (err == -ETIMEDOUT)
> + /* leave the entry for the completion handler; the driver
> + * is marked timed out and rejects all further commands
> + */
> + return err;
[ ... ]
> +void xsc_cmd_use_events(struct xsc_core_device *xdev)
> +{
> + struct xsc_cmd *cmd = &xdev->cmd;
> + int i;
> +
> + /* flush first: cmd_work_handler() takes a semaphore permit, so
> + * flushing after all permits are held would deadlock against a
> + * just-queued work item
> + */
> + flush_workqueue(cmd->wq);
> +
> + /* keep the cmdq interrupt out while the polling kthread is still
> + * draining the queue
> + */
> + disable_irq(cmd->irqn);
> +
> + for (i = 0; i < cmd->max_reg_cmds; i++)
> + down(&cmd->sem);
> +
> + cmd->mode = XSC_CMD_MODE_EVENTS;
> +
> + while (cmd->cmd_pid != cmd->cq_cid)
> + msleep(20);
> + kthread_stop(cmd->cq_task);
> + cmd->cq_task = NULL;
> +
> + for (i = 0; i < cmd->max_reg_cmds; i++)
> + up(&cmd->sem);
> +
> + enable_irq(cmd->irqn);
> +}
[Severity: High]
Can the while (cmd->cmd_pid != cmd->cq_cid) loop in xsc_cmd_use_events()
spin forever if the request producer index and response consumer index start
at different offsets?
In xsc_cmd_init(), cmd->cmd_pid and cmd->cq_cid are initialized from separate
hardware registers:
cmd->cmd_pid = xsc_get_cmdq_req_pid(xdev) & XSC_CMDQ_INDEX_MASK;
cmd->cq_cid = xsc_get_cmdq_rsp_cid(xdev) & XSC_CMDQ_INDEX_MASK;
If the hardware request and response indices do not have the exact same
modulo-32 value after a kexec, FLR, or prior command error, cmd->cmd_pid and
cmd->cq_cid maintain a constant non-zero offset even when all commands have
completed, so cmd->cmd_pid != cmd->cq_cid will never become false.
In addition, in xsc_cmd_cq_polling() and xsc_cmd_resp_handler(),
xsc_cmd_comp_handler() calls up(&cmd->sem) before the caller advances
cmd->cq_cid, updates hardware via xsc_update_cmdq_rsp_cid(), and flips
cmd->owner_bit. Doesn't that allow xsc_cmd_use_events() or
xsc_cmd_use_polling() to acquire the final semaphore permit before the
completion handler finishes updating the CQ consumer state?
> +static int xsc_cmd_cq_polling(void *data);
> +void xsc_cmd_use_polling(struct xsc_core_device *xdev)
> +{
> + struct xsc_cmd *cmd = &xdev->cmd;
> + int i;
> +
> + flush_workqueue(cmd->wq);
> +
> + disable_irq(cmd->irqn);
> +
> + for (i = 0; i < cmd->max_reg_cmds; i++)
> + down(&cmd->sem);
> +
> + cmd->mode = XSC_CMD_MODE_POLLING;
> + cmd->cq_task = kthread_create(xsc_cmd_cq_polling,
> + (void *)xdev,
> + "xsc_cmd_cq_polling");
> + if (!IS_ERR(cmd->cq_task))
> + wake_up_process(cmd->cq_task);
> +
> + for (i = 0; i < cmd->max_reg_cmds; i++)
> + up(&cmd->sem);
> +
> + enable_irq(cmd->irqn);
> +}
[Severity: High]
Won't xsc_cmd_use_polling() deadlock on cmd->sem if a command is still in
flight when disable_irq(cmd->irqn) is called?
In XSC_CMD_MODE_EVENTS, cmd->cq_task is NULL and completions are driven by
the interrupt handler on cmd->irqn calling xsc_cmd_resp_handler() ->
xsc_cmd_comp_handler() -> up(&cmd->sem). Because cmd_work_handler() returns
as soon as it rings the doorbell while the in-flight command continues
holding its permit on cmd->sem, calling disable_irq(cmd->irqn) before
acquiring all cmd->max_reg_cmds permits masks the completion interrupt while
no polling thread is running, so up(&cmd->sem) cannot be called.
Also, why does xsc_cmd_use_polling() call enable_irq(cmd->irqn) at the end
after starting cmd->cq_task? With the interrupt enabled in polling mode,
xsc_cmd_resp_handler() and xsc_cmd_cq_polling() can race on cmd->cq_cid and
cmd->owner_bit.
Finally, if kthread_create() fails in xsc_cmd_use_polling(), cmd->cq_task
stores an ERR_PTR while cmd->mode is set to XSC_CMD_MODE_POLLING, and
xsc_cmd_use_events() later calls kthread_stop(cmd->cq_task) without checking
IS_ERR_OR_NULL(cmd->cq_task).
[ ... ]
> +static struct xsc_cmd_msg *xsc_alloc_msg(struct xsc_core_device *xdev,
> + unsigned int in_size)
> +{
> + struct xsc_cmd_msg *msg = ERR_PTR(-ENOMEM);
> + struct xsc_cmd *cmd = &xdev->cmd;
> + struct cache_ent *ent = NULL;
> +
> + if (in_size > MED_LIST_SIZE && in_size <= LONG_LIST_SIZE)
> + ent = &cmd->cache.large;
> + else if (in_size > 16 && in_size <= MED_LIST_SIZE)
> + ent = &cmd->cache.med;
> +
> + if (ent) {
> + spin_lock(&ent->lock);
> + if (!list_empty(&ent->head)) {
> + msg = list_entry(ent->head.next, typeof(*msg), list);
> + /* For cached lists, we must explicitly state what is
> + * the real size
> + */
> + msg->len = in_size;
> + list_del(&msg->list);
> + }
> + spin_unlock(&ent->lock);
> + }
> +
> + if (IS_ERR(msg))
> + msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, in_size);
> +
> + return msg;
> +}
[Severity: Medium]
Should the inline threshold in xsc_alloc_msg(), MED_LIST_SIZE, and
LONG_LIST_SIZE be sizeof(msg->first.data) (32 bytes) instead of 16?
In struct xsc_cmd_first, data is 32 bytes (__be32 data[8]), and both
xsc_alloc_cmd_msg() and xsc_copy_to_cmd_msg() copy up to
sizeof(to->first.data) (32 bytes) inline before using overflow mailboxes in
to->next.
Because xsc_alloc_msg() checks in_size > 16, a command with 16 < in_size <= 32
(such as XSC_CMD_OP_ENABLE_NIC_HCA or XSC_CMD_OP_MODIFY_RAW_QP) is served from
cmd->cache.med, which has a pre-allocated overflow mailbox in msg->next.
xsc_copy_to_cmd_msg() consumes all in_size bytes into to->first.data (leaving
the remaining 32 - in_size inline bytes with stale cached data) and never
initializes to->next->buf, yet cmd_work_handler() still copies all 32 bytes
of ent->in->first.data into lay->in and sets lay->in_ptr to ent->in->next->dma.
[ ... ]
> +static int _xsc_cmd_exec(struct xsc_core_device *xdev,
> + void *in, unsigned int in_size,
> + void *out, unsigned int out_size)
> +{
> + struct xsc_cmd *cmd = &xdev->cmd;
> + struct xsc_rsp_msg *outb;
> + struct xsc_cmd_msg *inb;
> + u8 status = 0;
> + int err;
> +
> + if (cmd->cmd_status == XSC_CMD_STATUS_TIMEDOUT)
> + return -ETIMEDOUT;
> +
> + inb = xsc_alloc_msg(xdev, in_size);
> + if (IS_ERR(inb)) {
> + err = PTR_ERR(inb);
> + return err;
> + }
> +
> + err = xsc_copy_to_cmd_msg(inb, in, in_size);
> + if (err) {
> + pci_err(xdev->pdev, "copy to cmd_msg err %d\n", err);
> + goto err_free_msg;
> + }
> +
> + outb = xsc_alloc_rsp_msg(xdev, GFP_KERNEL, out_size);
> + if (IS_ERR(outb)) {
> + err = PTR_ERR(outb);
> + goto err_free_msg;
> + }
> +
> + err = xsc_cmd_invoke(xdev, inb, outb, &status);
> + if (err == -ETIMEDOUT)
> + /* the entry stays registered for the completion handler
> + * and keeps its mailboxes
> + */
> + return err;
[ ... ]
> +static void xsc_cmd_comp_handler(struct xsc_core_device *xdev, u8 idx,
> + struct xsc_rsp_layout *rsp)
> +{
[ ... ]
> + xsc_free_ent(cmd, ent->idx);
> + complete(&ent->done);
> + up(&cmd->sem);
> +}
> +
> +static int xsc_cmd_cq_polling(void *data)
> +{
> + struct xsc_core_device *xdev = data;
> + struct xsc_cmd *cmd = &xdev->cmd;
> + struct xsc_rsp_layout *rsp;
> + u32 cq_pid;
> +
> + while (!kthread_should_stop()) {
> + if (need_resched())
> + schedule();
> + cq_pid = xsc_get_cmdq_rsp_pid(xdev);
> + if (cmd->cq_cid == cq_pid) {
> + usleep_range(1000, 2000);
> + continue;
> + }
> +
> + rsp = xsc_get_cq_inst(cmd, cmd->cq_cid);
> + if (!cmd->ownerbit_learned) {
> + cmd->ownerbit_learned = 1;
> + cmd->owner_bit = xsc_rsp_owner_bit(rsp);
> + }
> + if (cmd->owner_bit != xsc_rsp_owner_bit(rsp)) {
> + dev_err_ratelimited(&xdev->pdev->dev,
> + "hw update cq doorbell but buf not ready %u %u\n",
> + cmd->cq_cid, cq_pid);
> + msleep(20);
> + continue;
> + }
[Severity: Medium]
Does learning cmd->owner_bit from xsc_rsp_owner_bit(rsp) right before
checking cmd->owner_bit != xsc_rsp_owner_bit(rsp) bypass the readiness check
on the first response in xsc_cmd_cq_polling() and xsc_cmd_resp_handler()?
In xsc_cmd_init(), cmd->cq_buf is zeroed with memset(cmd->cq_buf, 0,
PAGE_SIZE) and cmd->ownerbit_learned is set to 0. When the first completion
is polled, if the MMIO read of rsp_pid sees the updated producer index before
the device DMA write of the response descriptor reaches cmd->cq_buf,
xsc_rsp_owner_bit(rsp) reads 0 from the zeroed buffer, sets cmd->owner_bit = 0,
and immediately passes the cmd->owner_bit != xsc_rsp_owner_bit(rsp) check with
an all-zero descriptor while latching the wrong owner-bit polarity for subsequent
slots.
[ ... ]
> +void xsc_cmd_cleanup(struct xsc_core_device *xdev)
> +{
> + struct xsc_cmd *cmd = &xdev->cmd;
> +
> + destroy_workqueue(cmd->wq);
> + if (!IS_ERR_OR_NULL(cmd->cq_task))
> + kthread_stop(cmd->cq_task);
> + xsc_destroy_msg_cache(xdev);
> + dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cq_buf,
> + cmd->cq_dma);
> + dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE, cmd->cmd_buf,
> + cmd->dma);
> + dma_pool_destroy(cmd->pool);
> +}
[Severity: High]
When a command times out in xsc_cmd_invoke() and _xsc_cmd_exec(), where are
ent, inb, and outb freed?
On -ETIMEDOUT, xsc_cmd_invoke() leaves ent allocated and _xsc_cmd_exec()
returns early without freeing outb or returning/freeing inb. However, neither
xsc_cmd_comp_handler() (if a late completion arrives) nor xsc_cmd_cleanup()
(during error unwind in xsc_hw_setup() or device teardown) ever frees ent,
inb, or outb. Because inb (when detached from cmd->cache or allocated on
demand) and outb (when out_size > sizeof(outb->first.data), such as in
xsc_cmd_query_hca_cap()) hold blocks allocated from cmd->pool,
dma_pool_destroy(cmd->pool) in xsc_cmd_cleanup() will fail to destroy the
pool due to still-allocated DMA blocks and leak coherent DMA memory.
Furthermore, if cmd->max_reg_cmds commands time out while another work item
is queued on cmd->wq, cmd_work_handler() for that queued item will block
indefinitely in down(&cmd->sem), causing destroy_workqueue(cmd->wq) in
xsc_cmd_cleanup() to hang.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
> new file mode 100644
> index 0000000000000..0358a40cca9b8
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
> @@ -0,0 +1,316 @@
[ ... ]
> +int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,
> + struct xsc_caps *caps)
> +{
[ ... ]
> + caps->max_qp_mcg = be16_to_cpu(out->hca_cap.max_qp_mcg);
> + caps->max_ra_res_qp = 1 << (out->hca_cap.log_max_ra_res_qp & 0x3f);
> + caps->max_ra_req_qp = 1 << (out->hca_cap.log_max_ra_req_qp & 0x3f);
> + caps->max_srq_wqes = 1 << out->hca_cap.log_max_srq_sz;
[ ... ]
> + caps->max_qp = be32_to_cpu(out->hca_cap.max_qp);
> + caps->max_cq = be32_to_cpu(out->hca_cap.max_cq);
> + caps->max_pd = be32_to_cpu(out->hca_cap.max_pd);
> +
> + if (be16_to_cpu(out->hdr.ver) >= XSC_CMD_QUERY_HCA_CAP_V2)
> + caps->max_mtt = be32_to_cpu(out->hca_cap.max_mtt);
> + else
> + caps->max_mtt = 1 << (out->hca_cap.log_max_mtt & 0x3f);
[Severity: Medium]
In struct xsc_hca_cap, max_qp, max_cq, and max_pd are defined alongside
max_mtt in the V2 section of the mailbox. When be16_to_cpu(out->hdr.ver) is
earlier than XSC_CMD_QUERY_HCA_CAP_V2, won't reading out->hca_cap.max_qp,
out->hca_cap.max_cq, and out->hca_cap.max_pd unconditionally leave
caps->max_qp, caps->max_cq, and caps->max_pd as 0 instead of falling back to
1U << caps->log_max_qp, 1U << caps->log_max_cq, and 1U << caps->log_max_pd?
Also, in expressions like 1 << (out->hca_cap.log_max_mtt & 0x3f),
1 << (out->hca_cap.log_max_ra_res_qp & 0x3f), and
1 << out->hca_cap.log_max_srq_sz, can shifting the signed 32-bit integer
literal 1 by a 6-bit or 8-bit field overflow a 32-bit signed int?
[ ... ]
> +static int xsc_cmd_query_guid(struct xsc_core_device *xdev)
> +{
> + struct xsc_cmd_query_guid_mbox_out out;
> + struct xsc_cmd_query_guid_mbox_in in = {};
> + int err;
> +
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_GUID);
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> + if (err)
> + return err;
> +
> + xdev->board_info->guid = out.guid;
> + xdev->board_info->guid_valid = 1;
> + return 0;
> +}
> +
> +int xsc_query_guid(struct xsc_core_device *xdev)
> +{
> + if (xdev->board_info->guid_valid)
> + return 0;
> +
> + return xsc_cmd_query_guid(xdev);
> +}
> +
> +static int xsc_cmd_activate_hw_config(struct xsc_core_device *xdev)
> +{
> + struct xsc_cmd_activate_hw_config_mbox_out out;
> + struct xsc_cmd_activate_hw_config_mbox_in in = {};
> + int err = 0;
> +
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ACTIVATE_HW_CONFIG);
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> + if (err)
> + return err;
> +
> + xdev->board_info->hw_config_activated = 1;
> + return 0;
> +}
> +
> +int xsc_activate_hw_config(struct xsc_core_device *xdev)
> +{
> + if (xdev->board_info->hw_config_activated)
> + return 0;
> +
> + return xsc_cmd_activate_hw_config(xdev);
> +}
[Severity: Medium]
Since xdev->board_info is shared across all PCI functions of the same board,
can concurrent probes of multiple functions on the same card race in
xsc_query_guid(), xsc_activate_hw_config(), and xsc_hw_setup()?
In xsc_cmd_query_hca_cap(), board_info_lock protects the lookup and
allocation of xdev->board_info, but xsc_query_guid() and
xsc_activate_hw_config() check and update guid_valid, guid, and
hw_config_activated (and xsc_hw_setup() updates xdev->board_info->msg_opcode)
without holding board_info_lock or using memory barriers. Two PCI functions
probing in parallel can both see guid_valid == 0 or hw_config_activated == 0
and issue duplicate XSC_CMD_OP_QUERY_GUID and XSC_CMD_OP_ACTIVATE_HW_CONFIG
commands or observe guid_valid == 1 before the store to guid is visible.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=4
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 05/15] net/xsc: add event queues, interrupts and the
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (3 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 04/15] net/xsc: add the command queue Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 06/15] net/xsc: add the CQ and QP resource pools Xin Tian
` (9 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
auxiliary bus
Add the event queue mechanism and split the core device into
function drivers over the auxiliary bus.
EQs are DMA-coherent rings whose entries carry an event type, a
sub-type and a 15-bit queue id. The async EQ is bound to vector 0,
the command interfaces to vectors 1 and 2, and completion EQs are
spread over the remaining MSI-X vectors, allocated from 4 up to the
firmware-reported maximum with the firmware-visible vector computed
from a base offset. Events that need process context are queued to
a singlethread workqueue, which queries the event type from firmware
and dispatches link-change notifications to registered handlers and
temperature warnings. Per-completion-vector IRQ affinity hints use
cpumask_local_spread.
Add the DMA allocation helpers used throughout the driver: a
coherent buffer abstraction that falls back from one large
allocation to page-chained fragments with vmap, and NUMA-local
fragment allocation driven by temporarily retargeting the device's
NUMA node.
Finally, register one auxiliary device per function, named
xsc_pci.eth and xsc_pci.ib, so the ethernet and RDMA drivers can
bind to their function independently of the PCI core.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/pci/adev.c | 142 +++++
.../net/ethernet/yunsilicon/xsc/pci/adev.h | 17 +
.../net/ethernet/yunsilicon/xsc/pci/alloc.c | 237 +++++++++
.../net/ethernet/yunsilicon/xsc/pci/alloc.h | 17 +
drivers/net/ethernet/yunsilicon/xsc/pci/cq.h | 17 +
drivers/net/ethernet/yunsilicon/xsc/pci/eq.c | 266 ++++++++++
drivers/net/ethernet/yunsilicon/xsc/pci/eq.h | 39 ++
.../net/ethernet/yunsilicon/xsc/pci/main.c | 25 +
.../net/ethernet/yunsilicon/xsc/pci/pci_irq.c | 494 ++++++++++++++++++
.../net/ethernet/yunsilicon/xsc/pci/pci_irq.h | 18 +
drivers/net/ethernet/yunsilicon/xsc/pci/qp.h | 28 +
12 files changed, 1301 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/adev.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/cq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/eq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index 656cdffa1..e264990f9 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o alloc.o eq.o pci_irq.o adev.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
new file mode 100644
index 000000000..c8e8f1a9f
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/auxiliary_bus.h>
+#include <linux/idr.h>
+
+#include "adev.h"
+
+static DEFINE_IDA(xsc_adev_ida);
+
+static const char * const xsc_adev_name[] = {
+ [XSC_ADEV_IDX_ETH] = XSC_ETH_ADEV_NAME,
+ [XSC_ADEV_IDX_IB] = XSC_IB_ADEV_NAME,
+};
+
+static void xsc_release_adev(struct device *dev)
+{
+ struct xsc_adev *xsc_adev =
+ container_of(dev, struct xsc_adev, adev.dev);
+
+ kfree(xsc_adev);
+}
+
+int xsc_adev_add(struct xsc_core_device *xdev, int idx)
+{
+ struct auxiliary_device *adev;
+ struct xsc_adev *xsc_adev;
+ int ret;
+
+ if (!xdev->xsc_adev_list)
+ return -ENODEV;
+
+ if (xdev->xsc_adev_list[idx])
+ return 0;
+
+ xsc_adev = kzalloc_obj(*xsc_adev, GFP_KERNEL);
+ if (!xsc_adev)
+ return -ENOMEM;
+
+ adev = &xsc_adev->adev;
+ adev->name = xsc_adev_name[idx];
+ adev->id = xdev->adev_id;
+ adev->dev.parent = &xdev->pdev->dev;
+ adev->dev.release = xsc_release_adev;
+ xsc_adev->xdev = xdev;
+ xsc_adev->idx = idx;
+
+ ret = auxiliary_device_init(adev);
+ if (ret) {
+ kfree(xsc_adev);
+ return ret;
+ }
+
+ ret = auxiliary_device_add(adev);
+ if (ret) {
+ auxiliary_device_uninit(adev);
+ return ret;
+ }
+
+ xdev->xsc_adev_list[idx] = xsc_adev;
+
+ return 0;
+}
+
+void xsc_adev_del(struct xsc_core_device *xdev, int idx)
+{
+ struct xsc_adev *xsc_adev;
+ struct auxiliary_device *adev;
+
+ if (!xdev->xsc_adev_list)
+ return;
+
+ xsc_adev = xdev->xsc_adev_list[idx];
+ if (!xsc_adev)
+ return;
+
+ /* Clear the slot before teardown: the driver's remove callback runs
+ * synchronously inside auxiliary_device_delete() and may re-enter
+ * xsc_adev_add() for this index.
+ */
+ xdev->xsc_adev_list[idx] = NULL;
+
+ adev = &xsc_adev->adev;
+ auxiliary_device_delete(adev);
+ auxiliary_device_uninit(adev);
+}
+
+int xsc_adev_init(struct xsc_core_device *xdev)
+{
+ struct xsc_adev **xsc_adev_list;
+ int adev_id;
+ int ret;
+
+ xsc_adev_list = kcalloc(ARRAY_SIZE(xsc_adev_name),
+ sizeof(struct xsc_adev *),
+ GFP_KERNEL);
+ if (!xsc_adev_list)
+ return -ENOMEM;
+ xdev->xsc_adev_list = xsc_adev_list;
+
+ adev_id = ida_alloc(&xsc_adev_ida, GFP_KERNEL);
+ if (adev_id < 0) {
+ ret = adev_id;
+ goto err_free_adev_list;
+ }
+ xdev->adev_id = adev_id;
+
+ ret = xsc_adev_add(xdev, XSC_ADEV_IDX_ETH);
+ if (ret)
+ goto err_dalloc_adev_id;
+
+ ret = xsc_adev_add(xdev, XSC_ADEV_IDX_IB);
+ if (ret)
+ goto err_unreg_eth_adev;
+
+ return 0;
+
+err_unreg_eth_adev:
+ xsc_adev_del(xdev, XSC_ADEV_IDX_ETH);
+err_dalloc_adev_id:
+ ida_free(&xsc_adev_ida, xdev->adev_id);
+err_free_adev_list:
+ kfree(xsc_adev_list);
+ xdev->xsc_adev_list = NULL;
+
+ return ret;
+}
+
+void xsc_adev_uninit(struct xsc_core_device *xdev)
+{
+ if (!xdev->xsc_adev_list)
+ return;
+
+ xsc_adev_del(xdev, XSC_ADEV_IDX_IB);
+ xsc_adev_del(xdev, XSC_ADEV_IDX_ETH);
+ ida_free(&xsc_adev_ida, xdev->adev_id);
+ kfree(xdev->xsc_adev_list);
+ xdev->xsc_adev_list = NULL;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h
new file mode 100644
index 000000000..fb5789ec4
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ADEV_H
+#define __XSC_ADEV_H
+
+#include "common/xsc_core.h"
+
+int xsc_adev_init(struct xsc_core_device *xdev);
+void xsc_adev_uninit(struct xsc_core_device *xdev);
+
+int xsc_adev_add(struct xsc_core_device *xdev, int idx);
+void xsc_adev_del(struct xsc_core_device *xdev, int idx);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c
new file mode 100644
index 000000000..b9d7f8a14
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c
@@ -0,0 +1,237 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/export.h>
+#include <linux/bitmap.h>
+#include <linux/dma-mapping.h>
+#include <linux/vmalloc.h>
+
+#include "alloc.h"
+
+/* Handling for queue buffers -- we allocate a bunch of memory and
+ * register it in a memory region at HCA virtual address 0. If the
+ * requested size is > max_direct, we split the allocation into
+ * multiple pages, so we don't require too much contiguous memory.
+ */
+int xsc_buf_alloc(struct xsc_core_device *xdev,
+ unsigned long size,
+ unsigned long max_direct,
+ struct xsc_buf *buf)
+{
+ dma_addr_t t;
+
+ buf->size = size;
+ if (size <= max_direct) {
+ buf->nbufs = 1;
+ buf->npages = 1;
+ buf->page_shift = get_order(size) + PAGE_SHIFT;
+ buf->direct.buf = dma_alloc_coherent(&xdev->pdev->dev,
+ size,
+ &t,
+ GFP_KERNEL);
+ if (!buf->direct.buf)
+ return -ENOMEM;
+
+ buf->direct.map = t;
+
+ while (t & GENMASK(buf->page_shift - 1, 0)) {
+ --buf->page_shift;
+ buf->npages *= 2;
+ }
+ } else {
+ struct page **pages;
+ int i;
+
+ buf->direct.buf = NULL;
+ buf->nbufs = DIV_ROUND_UP(size, PAGE_SIZE);
+ buf->npages = buf->nbufs;
+ buf->page_shift = PAGE_SHIFT;
+ buf->page_list = kcalloc(buf->nbufs, sizeof(*buf->page_list),
+ GFP_KERNEL);
+ if (!buf->page_list)
+ return -ENOMEM;
+
+ for (i = 0; i < buf->nbufs; i++) {
+ buf->page_list[i].buf =
+ dma_alloc_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ &t, GFP_KERNEL);
+ if (!buf->page_list[i].buf)
+ goto err_free;
+
+ buf->page_list[i].map = t;
+ }
+
+ pages = kmalloc_array(buf->nbufs, sizeof(*pages), GFP_KERNEL);
+ if (!pages)
+ goto err_free;
+ for (i = 0; i < buf->nbufs; i++) {
+ void *addr = buf->page_list[i].buf;
+
+ if (is_vmalloc_addr(addr))
+ pages[i] = vmalloc_to_page(addr);
+ else
+ pages[i] = virt_to_page(addr);
+ }
+ buf->direct.buf = vmap(pages, buf->nbufs, VM_MAP, PAGE_KERNEL);
+ kfree(pages);
+ if (!buf->direct.buf)
+ goto err_free;
+ }
+
+ return 0;
+
+err_free:
+ xsc_buf_free(xdev, buf);
+
+ return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(xsc_buf_alloc);
+
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf)
+{
+ int i;
+
+ if (buf->nbufs == 1) {
+ dma_free_coherent(&xdev->pdev->dev, buf->size, buf->direct.buf,
+ buf->direct.map);
+ } else {
+ if (buf->direct.buf)
+ vunmap(buf->direct.buf);
+
+ for (i = 0; i < buf->nbufs; i++)
+ if (buf->page_list[i].buf)
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ buf->page_list[i].buf,
+ buf->page_list[i].map);
+ kfree(buf->page_list);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_buf_free);
+
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages)
+{
+ unsigned int shift = PAGE_SHIFT - XSC_PAGE_SHIFT_4K;
+ unsigned int mask = (1 << shift) - 1;
+ u64 addr;
+ int i;
+
+ for (i = 0; i < npages; i++) {
+ if (buf->nbufs == 1)
+ addr = buf->direct.map + (i << XSC_PAGE_SHIFT_4K);
+ else
+ addr = buf->page_list[i >> shift].map +
+ ((i & mask) << XSC_PAGE_SHIFT_4K);
+
+ pas[i] = cpu_to_be64(addr);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_fill_page_array);
+
+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,
+ __be64 *pas,
+ unsigned int npages)
+{
+ int i;
+ dma_addr_t addr;
+ unsigned int shift = PAGE_SHIFT - XSC_PAGE_SHIFT_4K;
+ unsigned int mask = (1 << shift) - 1;
+
+ for (i = 0; i < npages; i++) {
+ addr = buf->frags[i >> shift].map +
+ ((i & mask) << XSC_PAGE_SHIFT_4K);
+ pas[i] = cpu_to_be64(addr);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_core_fill_page_frag_array);
+
+static void *xsc_dma_zalloc_coherent_node(struct xsc_core_device *xdev,
+ size_t size, dma_addr_t *dma_handle,
+ int node)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct device *device = &xdev->pdev->dev;
+ int original_node;
+ void *cpu_handle;
+
+ /* WA for kernels that don't use numa_mem_id in alloc_pages_node */
+ if (node == NUMA_NO_NODE)
+ node = numa_mem_id();
+
+ mutex_lock(&dev_res->alloc_mutex);
+ original_node = dev_to_node(device);
+ set_dev_node(device, node);
+ cpu_handle = dma_alloc_coherent(device, size, dma_handle,
+ GFP_KERNEL);
+ set_dev_node(device, original_node);
+ mutex_unlock(&dev_res->alloc_mutex);
+ return cpu_handle;
+}
+
+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,
+ unsigned long size,
+ struct xsc_frag_buf *buf,
+ int node)
+{
+ int i;
+
+ buf->size = size;
+ buf->npages = DIV_ROUND_UP(size, PAGE_SIZE);
+ buf->page_shift = PAGE_SHIFT;
+ buf->frags = kcalloc(buf->npages, sizeof(struct xsc_buf_list),
+ GFP_KERNEL);
+ if (!buf->frags)
+ goto err_out;
+
+ for (i = 0; i < buf->npages; i++) {
+ unsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);
+ struct xsc_buf_list *frag = &buf->frags[i];
+
+ frag->buf = xsc_dma_zalloc_coherent_node(xdev, frag_sz,
+ &frag->map, node);
+ if (!frag->buf)
+ goto err_free_buf;
+ if (frag->map & ((1 << buf->page_shift) - 1)) {
+ dma_free_coherent(&xdev->pdev->dev, frag_sz,
+ buf->frags[i].buf, buf->frags[i].map);
+ pci_err(xdev->pdev, "unexpected map alignment: %pad, page_shift=%d\n",
+ &frag->map, buf->page_shift);
+ goto err_free_buf;
+ }
+ size -= frag_sz;
+ }
+
+ return 0;
+
+err_free_buf:
+ while (i--)
+ dma_free_coherent(&xdev->pdev->dev,
+ PAGE_SIZE,
+ buf->frags[i].buf,
+ buf->frags[i].map);
+ kfree(buf->frags);
+err_out:
+ return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_alloc_node);
+
+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,
+ struct xsc_frag_buf *buf)
+{
+ unsigned long size = buf->size;
+ int i;
+
+ for (i = 0; i < buf->npages; i++) {
+ unsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);
+
+ dma_free_coherent(&xdev->pdev->dev, frag_sz, buf->frags[i].buf,
+ buf->frags[i].map);
+ size -= frag_sz;
+ }
+ kfree(buf->frags);
+}
+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_free);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h
new file mode 100644
index 000000000..aef39e632
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ALLOC_H
+#define __XSC_ALLOC_H
+
+#include "common/xsc_core.h"
+
+int xsc_buf_alloc(struct xsc_core_device *xdev,
+ unsigned long size,
+ unsigned long max_direct,
+ struct xsc_buf *buf);
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h
new file mode 100644
index 000000000..afe6cbb0d
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CQ_H
+#define __XSC_CQ_H
+
+#include "common/xsc_core.h"
+
+void xsc_init_cq_table(struct xsc_core_device *xdev);
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+#endif /* __XSC_CQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
new file mode 100644
index 000000000..63e5bb29f
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/mm.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_core.h"
+#include "qp.h"
+#include "alloc.h"
+#include "eq.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ XSC_EQE_SIZE = sizeof(struct xsc_eqe),
+};
+
+enum {
+ XSC_NUM_SPARE_EQE = 0x80,
+ XSC_NUM_ASYNC_EQE = 0x100,
+};
+
+static int xsc_cmd_destroy_eq(struct xsc_core_device *xdev, u32 eqn)
+{
+ struct xsc_destroy_eq_mbox_out out = {0};
+ struct xsc_destroy_eq_mbox_in in = {0};
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_EQ);
+ in.eqn = cpu_to_be32(eqn);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+static struct xsc_eqe *get_eqe(struct xsc_eq *eq, u32 entry)
+{
+ return xsc_buf_offset(&eq->buf, entry * XSC_EQE_SIZE);
+}
+
+static struct xsc_eqe *next_eqe_sw(struct xsc_eq *eq)
+{
+ struct xsc_eqe *eqe = get_eqe(eq, eq->cons_index & (eq->nent - 1));
+
+ return (!!(eqe->owner_data & XSC_EQE_OWNER) ^
+ !!(eq->cons_index & eq->nent)) ? NULL : eqe;
+}
+
+static void eq_update_ci(struct xsc_eq *eq, int arm)
+{
+ xsc_update_eq_ci(eq->dev, eq->eqn, eq->cons_index, arm);
+}
+
+static void xsc_cq_completion(struct xsc_core_device *xdev, u32 cqn)
+{
+}
+
+static void xsc_eq_cq_event(struct xsc_core_device *xdev,
+ u32 cqn, int event_type)
+{
+}
+
+static int xsc_eq_int(struct xsc_core_device *xdev, struct xsc_eq *eq)
+{
+ u32 cqn, queue_id;
+ struct xsc_eqe *eqe;
+ int eqes_found = 0;
+ int set_ci = 0;
+
+ while ((eqe = next_eqe_sw(eq))) {
+ /* Make sure we read EQ entry contents after we've
+ * checked the ownership bit.
+ */
+ rmb();
+ switch (eqe->type) {
+ case XSC_EVENT_TYPE_COMP:
+ case XSC_EVENT_TYPE_INTERNAL_ERROR:
+ /* eqe is changing */
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ cqn = queue_id;
+ xsc_cq_completion(xdev, cqn);
+ break;
+
+ case XSC_EVENT_TYPE_CQ_ERROR:
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ cqn = queue_id;
+ xsc_eq_cq_event(xdev, cqn, eqe->type);
+ break;
+ default:
+ break;
+ }
+
+ ++eq->cons_index;
+ eqes_found = 1;
+ ++set_ci;
+
+ /* The HCA will think the queue has overflowed if we
+ * don't tell it we've been processing events. We
+ * create our EQs with XSC_NUM_SPARE_EQE extra
+ * entries, so we must update our consumer index at
+ * least that often.
+ */
+ if (unlikely(set_ci >= XSC_NUM_SPARE_EQE)) {
+ eq_update_ci(eq, 0);
+ set_ci = 0;
+ }
+ }
+
+ eq_update_ci(eq, 1);
+
+ return eqes_found;
+}
+
+static irqreturn_t xsc_msix_handler(int irq, void *eq_ptr)
+{
+ struct xsc_core_device *xdev;
+ struct xsc_eq *eq = eq_ptr;
+
+ xdev = eq->dev;
+ xsc_eq_int(xdev, eq);
+
+ /* MSI-X vectors always belong to us */
+ return IRQ_HANDLED;
+}
+
+static void init_eq_buf(struct xsc_eq *eq)
+{
+ struct xsc_eqe *eqe;
+ int i;
+
+ for (i = 0; i < eq->nent; i++) {
+ eqe = get_eqe(eq, i);
+ eqe->owner_data |= XSC_EQE_OWNER;
+ }
+}
+
+int xsc_create_map_eq(struct xsc_core_device *xdev,
+ struct xsc_eq *eq, u16 vecidx,
+ u32 nent, const char *name)
+{
+ u16 msix_vec_offset = xdev->msix_vec_base + vecidx;
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct xsc_create_eq_mbox_out out;
+ struct xsc_create_eq_mbox_in *in;
+ unsigned int hw_npages;
+ u32 inlen;
+ int err;
+
+ eq->nent = roundup_pow_of_two(roundup(nent, XSC_NUM_SPARE_EQE));
+ err = xsc_buf_alloc(xdev, eq->nent * XSC_EQE_SIZE, PAGE_SIZE, &eq->buf);
+ if (err)
+ return err;
+
+ init_eq_buf(eq);
+
+ hw_npages = DIV_ROUND_UP(eq->nent * XSC_EQE_SIZE, XSC_PAGE_SIZE_4K);
+ inlen = sizeof(*in) + sizeof(in->pas[0]) * hw_npages;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_free_eq_buf;
+ }
+ memset(&out, 0, sizeof(out));
+
+ xsc_fill_page_array(&eq->buf, in->pas, hw_npages);
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_EQ);
+ in->ctx.log_eq_sz = ilog2(eq->nent);
+ in->ctx.vecidx = cpu_to_be16(msix_vec_offset);
+ in->ctx.pa_num = cpu_to_be16(hw_npages);
+ in->ctx.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+ in->ctx.is_async_eq = (vecidx == XSC_EQ_VEC_ASYNC ? 1 : 0);
+
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err)
+ goto err_free_in;
+
+ snprintf(dev_res->irq_info[vecidx].name, XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ name, pci_name(xdev->pdev));
+
+ eq->eqn = be32_to_cpu(out.eqn);
+ eq->irqn = pci_irq_vector(xdev->pdev, vecidx);
+ eq->dev = xdev;
+ eq->index = vecidx;
+
+ err = request_irq(eq->irqn, xsc_msix_handler, 0,
+ dev_res->irq_info[vecidx].name, eq);
+ if (err)
+ goto err_destroy_eq;
+
+ /* EQs are created in ARMED state
+ */
+ eq_update_ci(eq, 1);
+ kvfree(in);
+ return 0;
+
+err_destroy_eq:
+ xsc_cmd_destroy_eq(xdev, eq->eqn);
+
+err_free_in:
+ kvfree(in);
+
+err_free_eq_buf:
+ xsc_buf_free(xdev, &eq->buf);
+ return err;
+}
+
+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq)
+{
+ int err;
+
+ free_irq(eq->irqn, eq);
+ err = xsc_cmd_destroy_eq(xdev, eq->eqn);
+ if (err)
+ pci_err(xdev->pdev, "failed to destroy a previously created eq: eqn %d\n",
+ eq->eqn);
+ xsc_buf_free(xdev, &eq->buf);
+
+ return err;
+}
+
+void xsc_eq_init(struct xsc_core_device *xdev)
+{
+ spin_lock_init(&xdev->dev_res->eq_table.lock);
+}
+
+int xsc_start_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int err;
+
+ err = xsc_create_map_eq(xdev, &table->async_eq, XSC_EQ_VEC_ASYNC,
+ XSC_NUM_ASYNC_EQE, "xsc_async_eq");
+ if (err)
+ pci_err(xdev->pdev, "failed to create async EQ %d\n", err);
+
+ return err;
+}
+
+void xsc_stop_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+
+ xsc_destroy_unmap_eq(xdev, &table->async_eq);
+}
+
+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq_ret = NULL;
+ struct xsc_eq *eq, *n;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ if (eq->index == i) {
+ eq_ret = eq;
+ break;
+ }
+ }
+ spin_unlock(&table->lock);
+
+ return eq_ret;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eq_get);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h
new file mode 100644
index 000000000..3e7869e12
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __EQ_H
+#define __EQ_H
+
+#include "common/xsc_core.h"
+
+enum {
+ XSC_EQ_VEC_ASYNC = 0,
+ XSC_VEC_CMD = 1,
+ XSC_VEC_CMD_EVENT = 2,
+ XSC_DMA_READ_DONE_VEC = 3,
+ XSC_EQ_VEC_COMP_BASE,
+};
+
+struct xsc_eqe {
+ u8 type;
+ u8 sub_type;
+ __le16 queue_id_data;
+#define XSC_EQE_QUEUE_ID_MASK GENMASK(14, 0)
+
+ u8 err_code;
+ u8 rsvd[2];
+ u8 owner_data;
+#define XSC_EQE_OWNER BIT(7)
+};
+
+int xsc_create_map_eq(struct xsc_core_device *xdev,
+ struct xsc_eq *eq, u16 vecidx,
+ u32 nent, const char *name);
+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq);
+void xsc_eq_init(struct xsc_core_device *xdev);
+int xsc_start_eqs(struct xsc_core_device *xdev);
+void xsc_stop_eqs(struct xsc_core_device *xdev);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
index 53dd8fdf1..9548cb953 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -6,6 +6,11 @@
#include "common/xsc_core.h"
#include "common/xsc_cmd_api.h"
#include "hw.h"
+#include "qp.h"
+#include "cq.h"
+#include "eq.h"
+#include "pci_irq.h"
+#include "adev.h"
#include "hal/xsc_hal.h"
static const struct pci_device_id xsc_pci_id_table[] = {
@@ -190,6 +195,8 @@ static int xsc_hw_setup(struct xsc_core_device *xdev)
goto err_cmd_cleanup;
}
+ xsc_eq_init(xdev);
+
return 0;
err_cmd_cleanup:
xsc_cmd_cleanup(xdev);
@@ -212,13 +219,31 @@ static int xsc_load(struct xsc_core_device *xdev)
goto err_out;
}
+ err = xsc_irq_eq_create(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_irq_eq_create failed %d\n", err);
+ goto err_hw_cleanup;
+ }
+
+ err = xsc_adev_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_adev_init failed %d\n", err);
+ goto err_irq_eq_destroy;
+ }
+
return 0;
+err_irq_eq_destroy:
+ xsc_irq_eq_destroy(xdev);
+err_hw_cleanup:
+ xsc_hw_cleanup(xdev);
err_out:
return err;
}
static void xsc_unload(struct xsc_core_device *xdev)
{
+ xsc_adev_uninit(xdev);
+ xsc_irq_eq_destroy(xdev);
xsc_hw_cleanup(xdev);
}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
new file mode 100644
index 000000000..d83f13f38
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
@@ -0,0 +1,494 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+#include <linux/interrupt.h>
+#include <linux/notifier.h>
+#include <linux/module.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_core.h"
+#include "eq.h"
+#include "pci_irq.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ XSC_COMP_EQ_SIZE = 1024,
+};
+
+static void xsc_free_irq(struct xsc_core_device *xdev, unsigned int vector)
+{
+ unsigned int irqn = 0;
+
+ irqn = pci_irq_vector(xdev->pdev, vector);
+ disable_irq(irqn);
+ free_irq(irqn, xdev);
+}
+
+static int set_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq = xsc_core_eq_get(xdev, i);
+ int vecidx = table->eq_vec_comp_base + i;
+ unsigned int irqn;
+
+ irqn = pci_irq_vector(xdev->pdev, vecidx);
+ if (!zalloc_cpumask_var(&eq->mask, GFP_KERNEL)) {
+ pci_err(xdev->pdev, "zalloc_cpumask_var rx cpumask failed");
+ return -ENOMEM;
+ }
+
+ cpumask_set_cpu(cpumask_local_spread(i, xdev->numa_node), eq->mask);
+ if (irq_set_affinity_hint(irqn, eq->mask))
+ pci_warn(xdev->pdev, "failed to set affinity hint for irq %u\n",
+ irqn);
+
+ return 0;
+}
+
+static void clear_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq = xsc_core_eq_get(xdev, i);
+ int vecidx = table->eq_vec_comp_base + i;
+ int irqn;
+
+ irqn = pci_irq_vector(xdev->pdev, vecidx);
+ irq_set_affinity_hint(irqn, NULL);
+ free_cpumask_var(eq->mask);
+}
+
+static int set_comp_irq_affinity_hints(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int nvec = table->num_comp_vectors;
+ int err;
+ int i;
+
+ if (!zalloc_cpumask_var(&xdev->xps_cpumask, GFP_KERNEL))
+ return -ENOMEM;
+
+ for (i = 0; i < nvec; i++) {
+ err = set_comp_irq_affinity_hint(xdev, i);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void clear_comp_irq_affinity_hints(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int nvec = table->num_comp_vectors;
+ int i;
+
+ for (i = 0; i < nvec; i++)
+ clear_comp_irq_affinity_hint(xdev, i);
+ free_cpumask_var(xdev->xps_cpumask);
+}
+
+static int xsc_alloc_irq_vectors(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct xsc_eq_table *table = &dev_res->eq_table;
+ int nvec = xdev->caps.msix_num;
+ int nvec_base;
+ int err;
+
+ nvec_base = XSC_EQ_VEC_COMP_BASE;
+ if (nvec <= nvec_base) {
+ pci_err(xdev->pdev, "failed to alloc irq vector(%d)\n", nvec);
+ return -ENOMEM;
+ }
+
+ dev_res->irq_info = kcalloc(nvec, sizeof(*dev_res->irq_info),
+ GFP_KERNEL);
+ if (!dev_res->irq_info)
+ return -ENOMEM;
+
+ nvec = pci_alloc_irq_vectors(xdev->pdev, nvec_base + 1, nvec,
+ PCI_IRQ_MSIX);
+ if (nvec < 0) {
+ err = nvec;
+ goto err_free_irq_info;
+ }
+
+ table->eq_vec_comp_base = nvec_base;
+ table->num_comp_vectors = nvec - nvec_base;
+ /*
+ * caps.msix_base is the start of this function's slice in the
+ * firmware's own vector numbering space. EQ and cmdq setup
+ * report msix_vec_base + vecidx to the firmware, while the host
+ * IRQ is requested on MSI-X table entry vecidx; the firmware
+ * translates back to the host table entry.
+ */
+ xdev->msix_vec_base = xdev->caps.msix_base;
+
+ return 0;
+
+err_free_irq_info:
+ kfree(dev_res->irq_info);
+ return err;
+}
+
+static void xsc_free_irq_vectors(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ pci_free_irq_vectors(xdev->pdev);
+ kfree(dev_res->irq_info);
+}
+
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq, *n;
+ int err = -ENOENT;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ if (eq->index == vector) {
+ *eqn = eq->eqn;
+ *irqn = eq->irqn;
+ err = 0;
+ break;
+ }
+ }
+ spin_unlock(&table->lock);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_vector2eqn);
+
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn)
+{
+ u32 cqn = 0;
+ int err;
+
+ err = xsc_core_vector2eqn(xdev, vector, &cqn, irqn);
+ if (!err)
+ *eqn = (int)cqn;
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_vector2eqn);
+
+static void free_comp_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq, *n;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ list_del(&eq->list);
+ spin_unlock(&table->lock);
+ if (xsc_destroy_unmap_eq(xdev, eq))
+ pci_err(xdev->pdev, "failed to destroy EQ 0x%x\n",
+ eq->eqn);
+ kfree(eq);
+ spin_lock(&table->lock);
+ }
+ spin_unlock(&table->lock);
+}
+
+static int alloc_comp_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ char name[XSC_MAX_IRQ_NAME];
+ struct xsc_eq *eq;
+ int ncomp_vec;
+ u32 nent;
+ int err;
+ int i;
+
+ INIT_LIST_HEAD(&table->comp_eqs_list);
+ ncomp_vec = table->num_comp_vectors;
+ nent = XSC_COMP_EQ_SIZE;
+
+ for (i = 0; i < ncomp_vec; i++) {
+ eq = kzalloc_obj(*eq, GFP_KERNEL);
+ if (!eq) {
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ snprintf(name, XSC_MAX_IRQ_NAME, "xsc_comp%d", i);
+ err = xsc_create_map_eq(xdev, eq,
+ i + table->eq_vec_comp_base,
+ nent, name);
+ if (err) {
+ kfree(eq);
+ goto err_free;
+ }
+
+ eq->index = i;
+ spin_lock(&table->lock);
+ list_add_tail(&eq->list, &table->comp_eqs_list);
+ spin_unlock(&table->lock);
+ }
+
+ return 0;
+
+err_free:
+ free_comp_eqs(xdev);
+ return err;
+}
+
+static irqreturn_t xsc_cmd_handler(int irq, void *arg)
+{
+ struct xsc_core_device *xdev = (struct xsc_core_device *)arg;
+ int err;
+
+ /* outside events mode the polling kthread owns the command cq */
+ if (xdev->cmd.mode != XSC_CMD_MODE_EVENTS)
+ return IRQ_HANDLED;
+
+ disable_irq_nosync(xdev->cmd.irqn);
+ err = xsc_cmd_err_handler(xdev);
+ if (!err)
+ xsc_cmd_resp_handler(xdev);
+ enable_irq(xdev->cmd.irqn);
+
+ return IRQ_HANDLED;
+}
+
+static int xsc_request_irq_for_cmdq(struct xsc_core_device *xdev, u8 vecidx)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ xsc_set_cmdq_msix_vector(xdev, xdev->msix_vec_base + vecidx);
+
+ snprintf(dev_res->irq_info[vecidx].name, XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ "xsc_cmd", pci_name(xdev->pdev));
+ xdev->cmd.irqn = pci_irq_vector(xdev->pdev, vecidx);
+ return request_irq(xdev->cmd.irqn, xsc_cmd_handler, 0,
+ dev_res->irq_info[vecidx].name, xdev);
+}
+
+static void xsc_free_irq_for_cmdq(struct xsc_core_device *xdev)
+{
+ xsc_free_irq(xdev, XSC_VEC_CMD);
+}
+
+/*
+ * Handle an asynchronous event. This runs on the PCI layer's own workqueue,
+ * so it outlives any auxiliary driver: the driver only ever registers a
+ * callback here, and the memory this touches belongs to xsc_core_device
+ * rather than to the eth or RDMA private area.
+ */
+static void xsc_event_work(struct work_struct *work)
+{
+ struct xsc_core_device *xdev = container_of(work, struct xsc_core_device,
+ event_work);
+ struct xsc_event_query_type_mbox_out out = {};
+ struct xsc_event_query_type_mbox_in in = {};
+ u8 event;
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_EVENT_TYPE);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to query event type, err=%d, status=%d\n",
+ err, out.hdr.status);
+ return;
+ }
+
+ event = out.ctx.resp_cmd_type;
+ while (event) {
+ if (event & XSC_CMD_EVENT_RESP_CHANGE_LINK) {
+ void (*handler)(void *xdev);
+
+ handler = READ_ONCE(xdev->link_event_handler);
+ if (handler)
+ handler(xdev);
+ event &= ~XSC_CMD_EVENT_RESP_CHANGE_LINK;
+ } else if (event & XSC_CMD_EVENT_RESP_TEMP_WARN) {
+ pci_warn(xdev->pdev, "[Minor]nic chip temperature high warning\n");
+ event &= ~XSC_CMD_EVENT_RESP_TEMP_WARN;
+ } else if (event & XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION) {
+ pci_warn(xdev->pdev, "[Critical]nic chip was over-temperature\n");
+ event &= ~XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION;
+ } else {
+ pci_dbg(xdev->pdev, "unknown event cmdtype=0x%x\n",
+ out.ctx.resp_cmd_type);
+ event = 0;
+ }
+ }
+}
+
+static irqreturn_t xsc_event_handler(int irq, void *arg)
+{
+ struct xsc_core_device *xdev = (struct xsc_core_device *)arg;
+
+ queue_work(xdev->event_wq, &xdev->event_work);
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * Register the link event callback.
+ *
+ * The events are dispatched from the PCI layer's own workqueue, whose
+ * contents are not the client's business (it also carries the temperature
+ * warnings, which are logged here), so the client must not touch
+ * xdev->event_wq itself.
+ */
+void xsc_irq_event_register(struct xsc_core_device *xdev,
+ void (*handler)(void *xdev))
+{
+ WRITE_ONCE(xdev->link_event_handler, handler);
+}
+EXPORT_SYMBOL_GPL(xsc_irq_event_register);
+
+/*
+ * Detach the callback and wait for an event that is already in flight to
+ * finish with it, in the same way free_irq() both unregisters and waits.
+ * Must be called before the caller frees whatever its handler dereferences.
+ * Safe to call when no handler is registered.
+ */
+void xsc_irq_event_unregister(struct xsc_core_device *xdev)
+{
+ WRITE_ONCE(xdev->link_event_handler, NULL);
+
+ /* Only exists once MSI-X, which requests the event IRQ, is up. */
+ if (xdev->event_wq)
+ flush_workqueue(xdev->event_wq);
+}
+EXPORT_SYMBOL_GPL(xsc_irq_event_unregister);
+
+static int xsc_request_irq_for_event(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ snprintf(dev_res->irq_info[XSC_VEC_CMD_EVENT].name,
+ XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ "xsc_eth_event", pci_name(xdev->pdev));
+ return request_irq(pci_irq_vector(xdev->pdev, XSC_VEC_CMD_EVENT),
+ xsc_event_handler, 0,
+ dev_res->irq_info[XSC_VEC_CMD_EVENT].name, xdev);
+}
+
+static void xsc_free_irq_for_event(struct xsc_core_device *xdev)
+{
+ xsc_free_irq(xdev, XSC_VEC_CMD_EVENT);
+}
+
+static int xsc_cmd_enable_msix(struct xsc_core_device *xdev)
+{
+ struct xsc_msix_table_info_mbox_out out;
+ struct xsc_msix_table_info_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_MSIX);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "xsc_cmd_exec enable msix failed %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+int xsc_irq_eq_create(struct xsc_core_device *xdev)
+{
+ int err;
+
+ err = xsc_alloc_irq_vectors(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "enable msix failed, err=%d\n", err);
+ goto err_out;
+ }
+
+ err = xsc_start_eqs(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to start EQs, err=%d\n", err);
+ goto err_free_irq_vectors;
+ }
+
+ err = alloc_comp_eqs(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to alloc comp EQs, err=%d\n", err);
+ goto err_stop_eqs;
+ }
+
+ err = xsc_request_irq_for_cmdq(xdev, XSC_VEC_CMD);
+ if (err) {
+ pci_err(xdev->pdev, "failed to request irq for cmdq, err=%d\n",
+ err);
+ goto err_free_comp_eqs;
+ }
+
+ xdev->event_wq = create_singlethread_workqueue("xsc_event");
+ if (!xdev->event_wq) {
+ pci_err(xdev->pdev, "failed to create event workqueue\n");
+ err = -ENOMEM;
+ goto err_free_irq_cmdq;
+ }
+ INIT_WORK(&xdev->event_work, xsc_event_work);
+
+ err = xsc_request_irq_for_event(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to request irq for event, err=%d\n",
+ err);
+ goto err_destroy_event_wq;
+ }
+
+ err = set_comp_irq_affinity_hints(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to alloc affinity hint cpumask, err=%d\n",
+ err);
+ goto err_clear_hints;
+ }
+
+ xsc_cmd_use_events(xdev);
+ err = xsc_cmd_enable_msix(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_cmd_enable_msix failed %d.\n", err);
+ xsc_cmd_use_polling(xdev);
+ goto err_clear_hints;
+ }
+ return 0;
+
+err_clear_hints:
+ clear_comp_irq_affinity_hints(xdev);
+ xsc_free_irq_for_event(xdev);
+err_destroy_event_wq:
+ destroy_workqueue(xdev->event_wq);
+err_free_irq_cmdq:
+ xsc_free_irq_for_cmdq(xdev);
+err_free_comp_eqs:
+ free_comp_eqs(xdev);
+err_stop_eqs:
+ xsc_stop_eqs(xdev);
+err_free_irq_vectors:
+ xsc_free_irq_vectors(xdev);
+err_out:
+ return err;
+}
+
+void xsc_irq_eq_destroy(struct xsc_core_device *xdev)
+{
+ xsc_stop_eqs(xdev);
+ clear_comp_irq_affinity_hints(xdev);
+ free_comp_eqs(xdev);
+
+ /* xsc_free_irq() disables and synchronises the event IRQ, so no new
+ * work can be queued once this returns.
+ */
+ xsc_free_irq_for_event(xdev);
+ destroy_workqueue(xdev->event_wq);
+ xdev->event_wq = NULL;
+ xsc_free_irq_for_cmdq(xdev);
+ xsc_free_irq_vectors(xdev);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h
new file mode 100644
index 000000000..750251d1b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PCI_IRQ_H
+#define __XSC_PCI_IRQ_H
+
+#include "common/xsc_core.h"
+
+int xsc_irq_eq_create(struct xsc_core_device *xdev);
+void xsc_irq_eq_destroy(struct xsc_core_device *xdev);
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn);
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
new file mode 100644
index 000000000..08af2008a
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_QP_H
+#define __XSC_QP_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+
+void xsc_init_qp_table(struct xsc_core_device *xdev);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+
+#endif /* __XSC_QP_H */
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 05/15] net/xsc: add event queues, interrupts and the
2026-11-04 15:57 ` [PATCH v1 05/15] net/xsc: add event queues, interrupts and the Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] net/xsc: deadlock in xsc_cmd_use_polling() if xsc_cmd_enable_msix() times out
- [Low] net/xsc: missing select AUXILIARY_BUS in Kconfig for xsc_adev_add()
--
commit bbb498d789473438951b00cc57ce44b8544a92ad
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add event queues, interrupts and the
This commit adds event queue support, MSI-X interrupt allocation and event
dispatch, DMA buffer allocation helpers, and auxiliary bus device registration
for the ethernet and RDMA function drivers.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
> new file mode 100644
> index 0000000000000..c8e8f1a9ff5e3
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
> @@ -0,0 +1,142 @@
[ ... ]
> +int xsc_adev_add(struct xsc_core_device *xdev, int idx)
> +{
[ ... ]
> + ret = auxiliary_device_init(adev);
> + if (ret) {
> + kfree(xsc_adev);
> + return ret;
> + }
> +
> + ret = auxiliary_device_add(adev);
[Severity: Low]
In drivers/net/ethernet/yunsilicon/xsc/pci/Makefile, adev.o is unconditionally
linked into xsc_pci.o when CONFIG_YUNSILICON_XSC_PCI is enabled, and
xsc_adev_add() calls auxiliary_device_init() and auxiliary_device_add().
In drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig, YUNSILICON_XSC_PCI only
specifies:
config YUNSILICON_XSC_PCI
tristate "Yunsilicon XSC PCI driver"
depends on PCI
Since AUXILIARY_BUS is a hidden symbol without fallback stubs when disabled,
will building with CONFIG_YUNSILICON_XSC_PCI enabled and CONFIG_AUXILIARY_BUS
unselected fail at link time with undefined references to
auxiliary_device_init() and __auxiliary_device_add()? Should
YUNSILICON_XSC_PCI select AUXILIARY_BUS in Kconfig?
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
> new file mode 100644
> index 0000000000000..d83f13f38e9b8
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
> @@ -0,0 +1,494 @@
[ ... ]
> +int xsc_irq_eq_create(struct xsc_core_device *xdev)
> +{
[ ... ]
> + xsc_cmd_use_events(xdev);
> + err = xsc_cmd_enable_msix(xdev);
> + if (err) {
> + pci_err(xdev->pdev, "xsc_cmd_enable_msix failed %d.\n", err);
> + xsc_cmd_use_polling(xdev);
> + goto err_clear_hints;
> + }
[Severity: High]
Can this sequence deadlock in xsc_cmd_use_polling() if xsc_cmd_enable_msix()
times out?
In xsc_irq_eq_create(), xsc_cmd_use_events() stops the command CQ polling
kthread (cmd->cq_task) and switches cmd->mode to XSC_CMD_MODE_EVENTS before
xsc_cmd_enable_msix() issues XSC_CMD_OP_ENABLE_MSIX to the firmware.
When xsc_cmd_enable_msix() calls xsc_cmd_exec() -> _xsc_cmd_exec() ->
xsc_cmd_invoke(), cmd_work_handler() acquires one permit of cmd->sem via
down(&cmd->sem). Because polling is already stopped and MSI-X is not yet
enabled in firmware (or if MSI-X delivery fails), no completion interrupt may
arrive on cmd->irqn. When xsc_wait_func() times out, xsc_cmd_invoke() leaves
the cmd->sem permit held for the completion handler:
drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c:xsc_cmd_invoke() {
...
err = xsc_wait_func(xdev, ent);
if (err == -ETIMEDOUT)
/* leave the entry for the completion handler; the driver
* is marked timed out and rejects all further commands
*/
return err;
...
}
The error path in xsc_irq_eq_create() then calls xsc_cmd_use_polling(), which
disables cmd->irqn and waits to acquire all cmd->max_reg_cmds permits of
cmd->sem before recreating cmd->cq_task:
drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c:xsc_cmd_use_polling() {
...
flush_workqueue(cmd->wq);
disable_irq(cmd->irqn);
for (i = 0; i < cmd->max_reg_cmds; i++)
down(&cmd->sem);
...
}
With one permit still held by the timed-out command and both the IRQ disabled
and the polling kthread stopped, wouldn't down(&cmd->sem) block forever? Should
xsc_cmd_enable_msix() be called while the command interface is still in
polling mode before switching to events with xsc_cmd_use_events()?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=5
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 06/15] net/xsc: add the CQ and QP resource pools
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (4 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 05/15] net/xsc: add event queues, interrupts and the Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 07/15] net/xsc: add ethernet netdev registration Xin Tian
` (8 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the device-wide registries for completion queues and queue pairs,
and wire completion events from the EQs into them.
CQs and QPs live in radix trees keyed by CQN and QPN and are looked
up from the EQ dispatch under RCU with a reference count taken on
hit; destroy removes the entry, synchronizes RCU and waits for
in-flight users before issuing the firmware destroy command, so a
completion arriving concurrently with teardown cannot touch freed
memory. QP destroy issues two RST transitions first to drain
pending work.
Provide two CQ creation paths: a legacy one used by the ethernet
function, and an extended one used by RDMA that falls back to the
legacy command when firmware rejects it and splits large contexts
into a context command followed by chunked buffer-address commands
when the context exceeds the maximum command payload. New CQs are
armed after creation.
Add the QP creation, destruction and state transition helpers,
including raw-TSO queues and multi-QP RSS receive queues needed by
the ethernet function, and route asynchronous QP events to the
event callback stored in the core QP.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../ethernet/yunsilicon/xsc/common/xsc_core.h | 1 -
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
drivers/net/ethernet/yunsilicon/xsc/pci/cq.c | 319 ++++++++++++++++
drivers/net/ethernet/yunsilicon/xsc/pci/eq.c | 54 ++-
.../net/ethernet/yunsilicon/xsc/pci/main.c | 2 +
drivers/net/ethernet/yunsilicon/xsc/pci/qp.c | 355 ++++++++++++++++++
drivers/net/ethernet/yunsilicon/xsc/pci/qp.h | 2 +-
7 files changed, 731 insertions(+), 4 deletions(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
index 9775412d2..bc41e88cc 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
@@ -255,7 +255,6 @@ struct xsc_core_cq {
void (*event)(struct xsc_core_cq *cq,
enum xsc_event);
u32 cons_index;
- unsigned int arm_sn;
int pid;
u32 reg_next_cid;
u32 reg_done_pid;
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index e264990f9..5c947da2a 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o alloc.o eq.o pci_irq.o adev.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o alloc.o eq.o pci_irq.o adev.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
new file mode 100644
index 000000000..e3e13ef53
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
@@ -0,0 +1,319 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "cq.h"
+
+void xsc_init_cq_table(struct xsc_core_device *xdev)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+
+ spin_lock_init(&table->lock);
+ INIT_RADIX_TREE(&table->tree, GFP_ATOMIC);
+}
+
+static int xsc_create_cq(struct xsc_core_device *xdev, u32 *p_cqn,
+ struct xsc_create_cq_mbox_in *in, int insize)
+{
+ struct xsc_create_cq_mbox_out out;
+ int ret;
+
+ memset(&out, 0, sizeof(out));
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);
+ ret = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to create cq, err=%d out.status=%u\n",
+ ret, out.hdr.status);
+ return ret;
+ }
+
+ *p_cqn = be32_to_cpu(out.cqn) & 0xffffff;
+ return 0;
+}
+
+static int xsc_destroy_cq(struct xsc_core_device *xdev, u32 cqn)
+{
+ struct xsc_destroy_cq_mbox_out out;
+ struct xsc_destroy_cq_mbox_in in;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
+ in.cqn = cpu_to_be32(cqn);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq,
+ struct xsc_create_cq_mbox_in *in,
+ int insize)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ u32 cqn;
+ int ret;
+ int err;
+
+ ret = xsc_create_cq(xdev, &cqn, in, insize);
+ if (ret) {
+ pci_err(xdev->pdev, "xsc_create_cq failed\n");
+ return -ENOEXEC;
+ }
+ xcq->cqn = cqn;
+ xcq->cons_index = 0;
+ atomic_set(&xcq->refcount, 1);
+ init_completion(&xcq->free);
+
+ spin_lock_irq(&table->lock);
+ ret = radix_tree_insert(&table->tree, xcq->cqn, xcq);
+ spin_unlock_irq(&table->lock);
+ if (ret)
+ goto err_insert_cq;
+ return 0;
+err_insert_cq:
+ err = xsc_destroy_cq(xdev, cqn);
+ if (err)
+ pci_err(xdev->pdev, "failed to destroy cqn=%d, err=%d\n",
+ xcq->cqn, err);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_cq);
+
+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *tmp;
+ int err;
+
+ spin_lock_irq(&table->lock);
+ tmp = radix_tree_delete(&table->tree, xcq->cqn);
+ spin_unlock_irq(&table->lock);
+ if (!tmp) {
+ err = -ENOENT;
+ goto err_delete_cq;
+ }
+
+ if (tmp != xcq) {
+ err = -EINVAL;
+ goto err_delete_cq;
+ }
+
+ /* the completion reader looks the cq up under rcu_read_lock and
+ * takes its reference afterwards; wait for it before dropping
+ * the tree reference so it never touches a freed cq
+ */
+ synchronize_rcu();
+
+ err = xsc_destroy_cq(xdev, xcq->cqn);
+ if (err)
+ goto err_destroy_cq;
+
+ if (atomic_dec_and_test(&xcq->refcount))
+ complete(&xcq->free);
+ wait_for_completion(&xcq->free);
+ return 0;
+
+err_destroy_cq:
+ pci_err(xdev->pdev, "failed to destroy cqn=%d, err=%d\n",
+ xcq->cqn, err);
+ return err;
+err_delete_cq:
+ pci_err(xdev->pdev, "cqn=%d not found in tree, err=%d\n",
+ xcq->cqn, err);
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_cq);
+
+static int xsc_set_cq_context(struct xsc_core_device *xdev,
+ struct xsc_cq_context_ex *ctx_ex,
+ u32 *pa_list_base, u32 *cqn)
+{
+ struct xsc_set_cq_context_mbox_in in;
+ struct xsc_set_cq_context_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memcpy(&in.ctx_ex, ctx_ex, sizeof(*ctx_ex));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_CONTEXT);
+ in.hdr.ver = cpu_to_be16(XSC_CMD_SET_CQ_CONTEXT_V1);
+
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set cq context\n");
+ return -EIO;
+ }
+
+ *pa_list_base = be32_to_cpu(out.cq_pa_list_base);
+ *cqn = be32_to_cpu(out.cqn);
+ return 0;
+}
+
+static int xsc_set_cq_buf_pa(struct xsc_core_device *xdev,
+ struct xsc_create_cq_ex_mbox_in *req,
+ u32 pa_list_base, u32 cqn)
+{
+ struct xsc_set_cq_buf_pa_mbox_in *in;
+ struct xsc_set_cq_buf_pa_mbox_out out;
+ u32 pa_num_total = be32_to_cpu(req->ctx_ex.pa_num_ex);
+ u32 pa_num_max;
+ u32 pa_num_left = pa_num_total;
+ u32 pa_num, copy_len;
+ int in_len, err;
+
+ if (xdev->caps.max_cmd_in_len < sizeof(*in) + sizeof(__be64)) {
+ pci_err(xdev->pdev, "max_cmd_in_len %u too small for CQ PA list\n",
+ xdev->caps.max_cmd_in_len);
+ return -EINVAL;
+ }
+
+ pa_num_max = (xdev->caps.max_cmd_in_len - sizeof(*in)) / sizeof(__be64);
+
+ while (pa_num_left) {
+ pa_num = min(pa_num_max, pa_num_left);
+ copy_len = pa_num * sizeof(__be64);
+ in_len = sizeof(*in) + copy_len;
+
+ in = kvzalloc(in_len, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_BUF_PA);
+ in->pa_list_start = cpu_to_be32(pa_list_base);
+ in->pa_num = cpu_to_be32(pa_num);
+ memcpy(in->pas, &req->pas[pa_num_total - pa_num_left], copy_len);
+ pa_num_left -= pa_num;
+ pa_list_base += pa_num;
+ memset(&out, 0, sizeof(out));
+
+ err = xsc_cmd_exec(xdev, in, in_len, &out, sizeof(out));
+ kvfree(in);
+
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set cq buf pa, cqn=%d\n", cqn);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int xsc_create_cq_compat(struct xsc_core_device *xdev,
+ struct xsc_create_cq_ex_mbox_in *in,
+ struct xsc_create_cq_mbox_out *out)
+{
+ struct xsc_create_cq_mbox_in *_in;
+ int _inlen = sizeof(*_in) +
+ be16_to_cpu(in->ctx_ex.ctx.pa_num) * sizeof(__be64);
+ int err;
+
+ _in = kvzalloc(_inlen, GFP_KERNEL);
+ if (!_in)
+ return -ENOMEM;
+
+ _in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);
+ memcpy(&_in->ctx, &in->ctx_ex.ctx, sizeof(_in->ctx));
+ memcpy(&_in->pas, &in->pas, _inlen - sizeof(*_in));
+ memset(out, 0, sizeof(*out));
+
+ err = xsc_cmd_exec(xdev, _in, _inlen, out, sizeof(*out));
+ kvfree(_in);
+
+ return err;
+}
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_create_cq_mbox_out out;
+ u32 pa_list_base = 0;
+ u32 cqn = 0;
+ int err;
+
+ if (inlen < xdev->caps.max_cmd_in_len) {
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ_EX);
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err == -EOPNOTSUPP) {
+ err = xsc_create_cq_compat(xdev, in, &out);
+ if (err)
+ return err;
+ } else if (err) {
+ return err;
+ }
+
+ cqn = be32_to_cpu(out.cqn);
+ } else {
+ err = xsc_set_cq_context(xdev, &in->ctx_ex, &pa_list_base, &cqn);
+ if (err)
+ return err;
+
+ err = xsc_set_cq_buf_pa(xdev, in, pa_list_base, cqn);
+ if (err)
+ goto err_cmd;
+ }
+
+ cq->cqn = cqn;
+ cq->cons_index = 0;
+ cq->xdev = xdev;
+ cq->dev = xdev;
+ atomic_set(&cq->refcount, 1);
+ init_completion(&cq->free);
+
+ spin_lock_irq(&table->lock);
+ err = radix_tree_insert(&table->tree, cq->cqn, cq);
+ spin_unlock_irq(&table->lock);
+ if (err)
+ goto err_cmd;
+
+ xsc_arm_cq(xdev, cq->cqn, 0, 0);
+
+ cq->pid = current->pid;
+ return 0;
+
+err_cmd:
+ xsc_destroy_cq(xdev, cqn);
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_cq);
+
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_destroy_cq_mbox_in in;
+ struct xsc_destroy_cq_mbox_out out;
+ struct xsc_core_cq *tmp;
+ int err;
+
+ spin_lock_irq(&table->lock);
+ tmp = radix_tree_delete(&table->tree, cq->cqn);
+ spin_unlock_irq(&table->lock);
+ if (!tmp || tmp != cq)
+ pci_warn(xdev->pdev, "cq 0x%x not found in tree\n", cq->cqn);
+
+ /* wait for lockless completion readers as in
+ * xsc_core_eth_destroy_cq()
+ */
+ synchronize_rcu();
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
+ in.cqn = cpu_to_be32(cq->cqn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
+ wait_for_completion(&cq->free);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_cq);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
index 63e5bb29f..7c6e46741 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
@@ -53,16 +53,60 @@ static void eq_update_ci(struct xsc_eq *eq, int arm)
static void xsc_cq_completion(struct xsc_core_device *xdev, u32 cqn)
{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *cq;
+
+ rcu_read_lock();
+ cq = radix_tree_lookup(&table->tree, cqn);
+ if (likely(cq))
+ atomic_inc(&cq->refcount);
+ rcu_read_unlock();
+
+ if (!cq) {
+ pci_err(xdev->pdev, "Completion event for bogus CQ, cqn=%d\n",
+ cqn);
+ return;
+ }
+
+ if (!cq->comp)
+ pci_err(xdev->pdev, "cq->comp is NULL\n");
+ else
+ cq->comp(cq);
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
}
static void xsc_eq_cq_event(struct xsc_core_device *xdev,
u32 cqn, int event_type)
{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *cq;
+
+ spin_lock(&table->lock);
+ cq = radix_tree_lookup(&table->tree, cqn);
+ if (likely(cq))
+ atomic_inc(&cq->refcount);
+ spin_unlock(&table->lock);
+
+ if (unlikely(!cq)) {
+ pci_err(xdev->pdev, "Async event for bogus CQ, cqn=%d\n",
+ cqn);
+ return;
+ }
+
+ if (cq->event)
+ cq->event(cq, event_type);
+ else
+ pci_err(xdev->pdev, "cq->event is NULL, cqn=%d\n", cq->cqn);
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
}
static int xsc_eq_int(struct xsc_core_device *xdev, struct xsc_eq *eq)
{
- u32 cqn, queue_id;
+ u32 cqn, qpn, queue_id;
struct xsc_eqe *eqe;
int eqes_found = 0;
int set_ci = 0;
@@ -88,6 +132,14 @@ static int xsc_eq_int(struct xsc_core_device *xdev, struct xsc_eq *eq)
cqn = queue_id;
xsc_eq_cq_event(xdev, cqn, eqe->type);
break;
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ qpn = queue_id;
+ xsc_qp_event(xdev, qpn, eqe->type);
+ break;
default:
break;
}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
index 9548cb953..cb2f37395 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -195,6 +195,8 @@ static int xsc_hw_setup(struct xsc_core_device *xdev)
goto err_cmd_cleanup;
}
+ xsc_init_cq_table(xdev);
+ xsc_init_qp_table(xdev);
xsc_eq_init(xdev);
return 0;
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
new file mode 100644
index 000000000..bbcdb18e1
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
@@ -0,0 +1,355 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/gfp.h>
+#include <linux/time.h>
+#include <linux/export.h>
+#include <linux/kthread.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "qp.h"
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ int err;
+
+ /* initialize before the insert: once in the tree, the qp can be
+ * found and referenced by the async event handler
+ */
+ atomic_set(&qp->refcount, 1);
+ init_completion(&qp->free);
+ qp->pid = current->pid;
+
+ spin_lock_irq(&table->lock);
+ err = radix_tree_insert(&table->tree, qp->qpn, qp);
+ spin_unlock_irq(&table->lock);
+ if (err)
+ return err;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_resource_common);
+
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ struct xsc_core_qp *tmp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&table->lock, flags);
+ tmp = radix_tree_delete(&table->tree, qp->qpn);
+ spin_unlock_irqrestore(&table->lock, flags);
+ if (!tmp) {
+ pci_warn(xdev->pdev, "qp 0x%x not found in tree\n", qp->qpn);
+ return;
+ }
+
+ if (atomic_dec_and_test(&qp->refcount))
+ complete(&qp->free);
+ wait_for_completion(&qp->free);
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_resource_common);
+
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ struct xsc_core_qp *qp;
+
+ spin_lock(&table->lock);
+
+ qp = radix_tree_lookup(&table->tree, qpn);
+ if (qp)
+ atomic_inc(&qp->refcount);
+
+ spin_unlock(&table->lock);
+
+ if (!qp) {
+ pci_err(xdev->pdev, "Async event for bogus QP 0x%x\n", qpn);
+ return;
+ }
+
+ if (qp->event)
+ qp->event(qp, event_type);
+ else
+ pci_err(xdev->pdev, "qp->event is NULL, qpn=0x%x\n", qp->qpn);
+
+ if (atomic_dec_and_test(&qp->refcount))
+ complete(&qp->free);
+}
+
+void xsc_init_qp_table(struct xsc_core_device *xdev)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+
+ spin_lock_init(&table->lock);
+ INIT_RADIX_TREE(&table->tree, GFP_ATOMIC);
+}
+
+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,
+ struct xsc_create_qp_mbox_in *in,
+ int insize, u32 *p_qpn)
+{
+ struct xsc_create_qp_mbox_out out = {};
+ int ret;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);
+ ret = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to create sq, err=%d out.status=%u\n",
+ ret, out.hdr.status);
+ return ret;
+ }
+
+ *p_qpn = be32_to_cpu(out.qpn) & 0xffffff;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_qp);
+
+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,
+ u32 qpn, u16 status)
+{
+ struct xsc_modify_qp_mbox_out out = {};
+ struct xsc_modify_qp_mbox_in in = {};
+ int ret = 0;
+
+ in.hdr.opcode = cpu_to_be16(status);
+ in.qpn = cpu_to_be32(qpn);
+ in.no_need_wait = 1;
+
+ ret = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to modify qp %u status=%u, err=%d out.status %u\n",
+ qpn, status, ret, out.hdr.status);
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_qp_status);
+
+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn)
+{
+ struct xsc_destroy_qp_mbox_out out;
+ struct xsc_destroy_qp_mbox_in in;
+ int err;
+
+ err = xsc_core_eth_modify_qp_status(xdev, qpn, XSC_CMD_OP_2RST_QP);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set sq%d status=rst, err=%d\n",
+ qpn, err);
+ return err;
+ }
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ in.qpn = cpu_to_be32(qpn);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to destroy sq%d, err=%d out.status=%u\n",
+ qpn, err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_qp);
+
+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,
+ struct xsc_modify_raw_qp_mbox_in *in)
+{
+ struct xsc_modify_raw_qp_mbox_out out = {};
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_MODIFY_RAW_QP);
+
+ err = xsc_cmd_exec(xdev, in, sizeof(struct xsc_modify_raw_qp_mbox_in),
+ &out, sizeof(struct xsc_modify_raw_qp_mbox_out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to modify sq, err=%d out.status=%u\n",
+ err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_raw_qp);
+
+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,
+ struct xsc_create_multiqp_mbox_in *in,
+ int insize,
+ u32 *p_qpn_base)
+{
+ struct xsc_create_multiqp_mbox_out out = {};
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MULTI_QP);
+ err = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev,
+ "failed to create rss rq, qp_num=%d, type=%d, err=%d out.status=%u\n",
+ be16_to_cpu(in->qp_num), in->qp_type, err, out.hdr.status);
+ return err;
+ }
+
+ *p_qpn_base = be32_to_cpu(out.qpn_base) & 0xffffff;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_rss_qp_rqs);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen)
+{
+ struct xsc_create_qp_mbox_out out;
+ struct xsc_destroy_qp_mbox_in din;
+ struct xsc_destroy_qp_mbox_out dout;
+ int err;
+
+ memset(&out, 0, sizeof(out));
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);
+
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err)
+ return err;
+
+ qp->qpn = be32_to_cpu(out.qpn) & 0xffffff;
+
+ err = xsc_core_create_resource_common(xdev, qp);
+ if (err)
+ goto err_cmd;
+
+ atomic_inc(&xdev->num_qps);
+ return 0;
+
+err_cmd:
+ memset(&din, 0, sizeof(din));
+ memset(&dout, 0, sizeof(dout));
+ din.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ din.qpn = cpu_to_be32(qp->qpn);
+ xsc_cmd_exec(xdev, &din, sizeof(din), &dout, sizeof(dout));
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_qp);
+
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_destroy_qp_mbox_in in;
+ struct xsc_destroy_qp_mbox_out out;
+ int err;
+
+ xsc_core_destroy_resource_common(xdev, qp);
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ in.qpn = cpu_to_be32(qp->qpn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ atomic_dec(&xdev->num_qps);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_qp);
+
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in,
+ struct xsc_core_qp *qp)
+{
+ static const u16 optab[XSC_QP_NUM_STATE][XSC_QP_NUM_STATE] = {
+ [XSC_QP_STATE_RST] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_INIT] = XSC_CMD_OP_RST2INIT_QP,
+ },
+ [XSC_QP_STATE_INIT] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_INIT] = XSC_CMD_OP_INIT2INIT_QP,
+ [XSC_QP_STATE_RTR] = XSC_CMD_OP_INIT2RTR_QP,
+ },
+ [XSC_QP_STATE_RTR] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_RTR2RTS_QP,
+ },
+ [XSC_QP_STATE_RTS] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_RTS2RTS_QP,
+ [XSC_QP_STATE_SQD] = XSC_CMD_OP_RTS2SQD_QP,
+ },
+ [XSC_QP_STATE_SQD] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_SQD2RTS_QP,
+ [XSC_QP_STATE_SQD] = XSC_CMD_OP_SQD2SQD_QP,
+ },
+ [XSC_QP_STATE_SQER] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_SQERR2RTS_QP,
+ },
+ [XSC_QP_STATE_ERR] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ }
+ };
+
+ struct xsc_modify_qp_mbox_out out;
+ u16 op;
+
+ if (cur_state >= XSC_QP_NUM_STATE || new_state >= XSC_QP_NUM_STATE ||
+ !optab[cur_state][new_state])
+ return -EINVAL;
+
+ memset(&out, 0, sizeof(out));
+ op = optab[cur_state][new_state];
+
+ if (new_state == XSC_QP_STATE_RTR) {
+ in->ctx.pcie_no = xdev->pcie_no;
+ in->ctx.func_id = cpu_to_be16(xdev->glb_func_id);
+ }
+
+ in->hdr.opcode = cpu_to_be16(op);
+ in->qpn = cpu_to_be32(qp->qpn);
+ in->no_need_wait = 1;
+
+ if (xsc_cmd_exec(xdev, in, sizeof(*in), &out, sizeof(out)))
+ return -ENOEXEC;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_qp_modify);
+
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen)
+{
+ struct xsc_query_qp_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(out, 0, outlen);
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_QP);
+ in.qpn = cpu_to_be32(qp->qpn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, outlen);
+ if (err)
+ return err;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_qp_query);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
index 08af2008a..487e8b0c9 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
@@ -20,7 +20,7 @@ int xsc_core_destroy_qp(struct xsc_core_device *xdev,
struct xsc_core_qp *qp);
int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
enum xsc_qp_state new_state,
- struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_modify_qp_mbox_in *in,
struct xsc_core_qp *qp);
int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
struct xsc_query_qp_mbox_out *out, int outlen);
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 06/15] net/xsc: add the CQ and QP resource pools
2026-11-04 15:57 ` [PATCH v1 06/15] net/xsc: add the CQ and QP resource pools Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] net/xsc: premature DESTROY_CQ and skipped refcount wait in CQ teardown
- [Medium] net/xsc: missing abort on invalid radix tree delete in xsc_core_destroy_cq()
- [Medium] net/xsc: inverted CQ/QP destroy ordering and inaccurate commit description
- [Low] net/xsc: duplicated CQ teardown in xsc_core_destroy_cq() and changelog mismatch
--
commit bc5206144b3c6edd9114d2a7f0311c3a018b5c6a
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add the CQ and QP resource pools
This commit adds the device-wide radix-tree registries and lifecycle helpers
for completion queues and queue pairs, and wires completion and asynchronous
events from EQs into their respective callbacks.
> net/xsc: add the CQ and QP resource pools
>
> Add the device-wide registries for completion queues and queue pairs,
> and wire completion events from the EQs into them.
>
> CQs and QPs live in radix trees keyed by CQN and QPN and are looked
> up from the EQ dispatch under RCU with a reference count taken on
> hit; destroy removes the entry, synchronizes RCU and waits for
> in-flight users before issuing the firmware destroy command, so a
> completion arriving concurrently with teardown cannot touch freed
> memory. QP destroy issues two RST transitions first to drain
> pending work.
>
> Provide two CQ creation paths: a legacy one used by the ethernet
> function, and an extended one used by RDMA that falls back to the
> legacy command when firmware rejects it and splits large contexts
> into a context command followed by chunked buffer-address commands
> when the context exceeds the maximum command payload. New CQs are
> armed after creation.
>
> Add the QP creation, destruction and state transition helpers,
> including raw-TSO queues and multi-QP RSS receive queues needed by
> the ethernet function, and route asynchronous QP events to the
> event callback stored in the core QP.
>
> Signed-off-by: Xin Tian <tianx@yunsilicon.com>
[Severity: Medium]
Does the commit message match what the patch actually implements in a few
places?
First, the changelog states that both CQs and QPs are looked up under RCU
and synchronized with RCU on destroy, whereas xsc_qp_event() and
xsc_eq_cq_event() perform radix_tree_lookup() under spin_lock(&table->lock),
and xsc_core_destroy_resource_common() does not call synchronize_rcu().
Second, the changelog states that destroy waits for in-flight users before
issuing the firmware destroy command, but both xsc_core_eth_destroy_cq() and
xsc_core_destroy_cq() issue XSC_CMD_OP_DESTROY_CQ before decrementing
refcount and calling wait_for_completion().
Third, the changelog states that QP destroy issues two RST transitions
first, which appears to misread the opcode name XSC_CMD_OP_2RST_QP
(transition to RST): xsc_core_eth_destroy_qp() issues a single
XSC_CMD_OP_2RST_QP command, and xsc_core_destroy_qp() does not issue a RST
transition at all.
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
> new file mode 100644
> index 0000000000000..e3e13ef531988
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
> @@ -0,0 +1,319 @@
[ ... ]
> +static int xsc_destroy_cq(struct xsc_core_device *xdev, u32 cqn)
> +{
> + struct xsc_destroy_cq_mbox_out out;
> + struct xsc_destroy_cq_mbox_in in;
> +
> + memset(&in, 0, sizeof(in));
> + memset(&out, 0, sizeof(out));
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
> + in.cqn = cpu_to_be32(cqn);
> +
> + return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> +}
[ ... ]
> +int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,
> + struct xsc_core_cq *xcq)
> +{
> + struct xsc_cq_table *table = &xdev->dev_res->cq_table;
> + struct xsc_core_cq *tmp;
> + int err;
> +
> + spin_lock_irq(&table->lock);
> + tmp = radix_tree_delete(&table->tree, xcq->cqn);
> + spin_unlock_irq(&table->lock);
> + if (!tmp) {
> + err = -ENOENT;
> + goto err_delete_cq;
> + }
> +
> + if (tmp != xcq) {
> + err = -EINVAL;
> + goto err_delete_cq;
> + }
> +
> + /* the completion reader looks the cq up under rcu_read_lock and
> + * takes its reference afterwards; wait for it before dropping
> + * the tree reference so it never touches a freed cq
> + */
> + synchronize_rcu();
> +
> + err = xsc_destroy_cq(xdev, xcq->cqn);
> + if (err)
> + goto err_destroy_cq;
> +
> + if (atomic_dec_and_test(&xcq->refcount))
> + complete(&xcq->free);
> + wait_for_completion(&xcq->free);
> + return 0;
[Severity: High]
Should atomic_dec_and_test(&xcq->refcount) and
wait_for_completion(&xcq->free) happen before xsc_destroy_cq() here (and
likewise before xsc_cmd_exec() in xsc_core_destroy_cq())?
In xsc_cq_completion(), rcu_read_lock() only covers radix_tree_lookup() and
atomic_inc(&cq->refcount):
drivers/net/ethernet/yunsilicon/xsc/pci/eq.c:xsc_cq_completion() {
rcu_read_lock();
cq = radix_tree_lookup(&table->tree, cqn);
if (likely(cq))
atomic_inc(&cq->refcount);
rcu_read_unlock();
...
if (!cq->comp)
pci_err(xdev->pdev, "cq->comp is NULL\n");
else
cq->comp(cq);
if (atomic_dec_and_test(&cq->refcount))
complete(&cq->free);
}
Similarly, xsc_eq_cq_event() drops table->lock before calling cq->event().
Because cq->comp() and cq->event() run outside rcu_read_lock() and
table->lock, synchronize_rcu() only guarantees that no new reader can take a
reference; it does not wait for an in-flight callback that already took a
reference to finish. Issuing XSC_CMD_OP_DESTROY_CQ before waiting on
xcq->free destroys the hardware CQ (allowing the firmware to reassign the
CQN) while cq->comp() or cq->event() is still executing on another CPU.
Furthermore, if xsc_destroy_cq() fails and jumps to err_destroy_cq, xcq has
already been deleted from table->tree, but the function returns without
waiting for in-flight callbacks on xcq->free. If the caller frees xcq after
destroy returns, an in-flight xsc_cq_completion() or xsc_eq_cq_event() will
access freed memory; if the caller retries xsc_core_eth_destroy_cq(),
radix_tree_delete() will return NULL and fail with -ENOENT without
destroying the firmware CQ.
[ ... ]
> +int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq)
> +{
> + struct xsc_cq_table *table = &xdev->dev_res->cq_table;
> + struct xsc_destroy_cq_mbox_in in;
> + struct xsc_destroy_cq_mbox_out out;
> + struct xsc_core_cq *tmp;
> + int err;
> +
> + spin_lock_irq(&table->lock);
> + tmp = radix_tree_delete(&table->tree, cq->cqn);
> + spin_unlock_irq(&table->lock);
> + if (!tmp || tmp != cq)
> + pci_warn(xdev->pdev, "cq 0x%x not found in tree\n", cq->cqn);
[Severity: Medium]
If !tmp or tmp != cq here, shouldn't xsc_core_destroy_cq() abort and return
an error instead of continuing after pci_warn()?
If the entry was not found (for instance, if a prior destroy call deleted it
from table->tree before failing in xsc_cmd_exec(), or on a duplicate
destroy), continuing will execute XSC_CMD_OP_DESTROY_CQ and then call
atomic_dec_and_test(&cq->refcount) followed by
wait_for_completion(&cq->free). If cq->refcount was already decremented, it
will underflow to -1 and hang indefinitely in wait_for_completion().
Also, in both xsc_core_eth_destroy_cq() and xsc_core_destroy_cq(), calling
radix_tree_delete(&table->tree, cq->cqn) before verifying that the stored
entry matches cq will remove an unrelated active CQ (tmp) from table->tree
when tmp != cq. Would it be safer to check radix_tree_lookup() under
table->lock before deleting?
> +
> + /* wait for lockless completion readers as in
> + * xsc_core_eth_destroy_cq()
> + */
> + synchronize_rcu();
> +
> + memset(&in, 0, sizeof(in));
> + memset(&out, 0, sizeof(out));
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
> + in.cqn = cpu_to_be32(cq->cqn);
> +
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> + if (err)
> + return err;
> +
> + if (atomic_dec_and_test(&cq->refcount))
> + complete(&cq->free);
> + wait_for_completion(&cq->free);
> +
> + return 0;
> +}
[Severity: Low]
This isn't a bug, but xsc_core_destroy_cq() open-codes xsc_destroy_cq() and
duplicates the radix-tree removal, synchronize_rcu(), and completion wait
sequence from xsc_core_eth_destroy_cq(). Could xsc_core_destroy_cq() and
xsc_core_eth_destroy_cq() share this teardown logic?
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
> new file mode 100644
> index 0000000000000..bbcdb18e1b84f
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
> @@ -0,0 +1,355 @@
[ ... ]
> +int xsc_core_destroy_qp(struct xsc_core_device *xdev,
> + struct xsc_core_qp *qp)
> +{
> + struct xsc_destroy_qp_mbox_in in;
> + struct xsc_destroy_qp_mbox_out out;
> + int err;
> +
> + xsc_core_destroy_resource_common(xdev, qp);
> +
> + memset(&in, 0, sizeof(in));
> + memset(&out, 0, sizeof(out));
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
> + in.qpn = cpu_to_be32(qp->qpn);
> +
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> + if (err)
> + return err;
> +
> + atomic_dec(&xdev->num_qps);
> + return 0;
> +}
[Severity: Medium]
If xsc_cmd_exec() fails in xsc_core_destroy_qp(), xsc_core_destroy_resource_common()
has already removed qp from qp_table.tree and consumed its initial refcount,
while xdev->num_qps remains incremented. How should a caller recover if
XSC_CMD_OP_DESTROY_QP fails after the QP has been removed from the resource
table?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=6
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 07/15] net/xsc: add ethernet netdev registration
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (5 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 06/15] net/xsc: add the CQ and QP resource pools Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 08/15] net/xsc: add ethernet TX path Xin Tian
` (7 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the ethernet function driver, which binds to the xsc_pci.eth
auxiliary device and registers a netdev on top of the core device.
Probe sizes the channel count from the number of completion vectors
(capped at 256), allocates a netdev with a TX and RX queue per
channel, retrieves the permanent MAC address from firmware with a
random fallback, and programs the MTU through the firmware command
interface before registering the device. Each channel pairs an
ethernet queue pair with a NAPI instance pinned to a CPU; the queue
wrappers for CQ, SQ, RQ and the cyclic work queue are introduced
here.
The open and stop paths set the carrier state from the firmware
vport state, install default XPS CPU masks, and toggle the port
administrative state through firmware. This commit stops short of
the data path: start_xmit drops packets, the receive path is not
implemented, and the netdev advertises only NETIF_F_HIGHDMA.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/net/ethernet/yunsilicon/Kconfig | 2 +-
drivers/net/ethernet/yunsilicon/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/net/Kconfig | 17 +
.../net/ethernet/yunsilicon/xsc/net/Makefile | 9 +
.../net/ethernet/yunsilicon/xsc/net/main.c | 480 ++++++++++++++++++
.../net/ethernet/yunsilicon/xsc/net/xsc_eth.h | 45 ++
.../yunsilicon/xsc/net/xsc_eth_common.h | 123 +++++
.../yunsilicon/xsc/net/xsc_eth_stats.h | 34 ++
.../yunsilicon/xsc/net/xsc_eth_txrx.h | 91 ++++
.../ethernet/yunsilicon/xsc/net/xsc_eth_wq.c | 80 +++
.../ethernet/yunsilicon/xsc/net/xsc_eth_wq.h | 185 +++++++
.../net/ethernet/yunsilicon/xsc/net/xsc_pp.h | 38 ++
.../net/ethernet/yunsilicon/xsc/net/xsc_pph.h | 42 ++
.../ethernet/yunsilicon/xsc/net/xsc_queue.h | 183 +++++++
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/pci/vport.c | 34 ++
16 files changed, 1364 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/Makefile
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/main.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
diff --git a/drivers/net/ethernet/yunsilicon/Kconfig b/drivers/net/ethernet/yunsilicon/Kconfig
index ffdfdf4bf..46af1fdd0 100644
--- a/drivers/net/ethernet/yunsilicon/Kconfig
+++ b/drivers/net/ethernet/yunsilicon/Kconfig
@@ -1,7 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
# All rights reserved.
-#
# Yunsilicon driver configuration
#
@@ -21,6 +20,7 @@ config NET_VENDOR_YUNSILICON
if NET_VENDOR_YUNSILICON
+source "drivers/net/ethernet/yunsilicon/xsc/net/Kconfig"
source "drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig"
endif # NET_VENDOR_YUNSILICON
diff --git a/drivers/net/ethernet/yunsilicon/Makefile b/drivers/net/ethernet/yunsilicon/Makefile
index b2924ccaf..a89a5ce24 100644
--- a/drivers/net/ethernet/yunsilicon/Makefile
+++ b/drivers/net/ethernet/yunsilicon/Makefile
@@ -1,8 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
# All rights reserved.
-#
# Makefile for the Yunsilicon device drivers.
#
+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc/net/
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc/pci/
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
new file mode 100644
index 000000000..7428b2629
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Yunsilicon driver configuration
+#
+
+config YUNSILICON_XSC_ETH
+ tristate "Yunsilicon XSC ethernet driver"
+ depends on YUNSILICON_XSC_PCI
+ depends on NET
+ select PAGE_POOL
+ help
+ This driver provides ethernet support for
+ Yunsilicon XSC devices.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_eth.
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
new file mode 100644
index 000000000..0e6249cec
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc_eth.o
+
+xsc_eth-y := main.o xsc_eth_wq.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
new file mode 100644
index 000000000..21a50085f
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
@@ -0,0 +1,480 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/irq.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_device.h"
+#include "xsc_pp.h"
+#include "xsc_eth_common.h"
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+
+static const struct xsc_tirc_config tirc_default_config[XSC_NUM_INDIR_TIRS] = {
+ [XSC_TT_IPV4] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = 0,
+ .rx_hash_fields = XSC_HASH_IP,
+ },
+ [XSC_TT_IPV4_TCP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = XSC_L4_PROT_TYPE_TCP,
+ .rx_hash_fields = XSC_HASH_IP_PORTS,
+ },
+ [XSC_TT_IPV4_UDP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = XSC_L4_PROT_TYPE_UDP,
+ .rx_hash_fields = XSC_HASH_IP_PORTS,
+ },
+ [XSC_TT_IPV6] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = 0,
+ .rx_hash_fields = XSC_HASH_IP6,
+ },
+ [XSC_TT_IPV6_TCP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = XSC_L4_PROT_TYPE_TCP,
+ .rx_hash_fields = XSC_HASH_IP6_PORTS,
+ },
+ [XSC_TT_IPV6_UDP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = XSC_L4_PROT_TYPE_UDP,
+ .rx_hash_fields = XSC_HASH_IP6_PORTS,
+ },
+};
+
+static int xsc_get_max_num_channels(struct xsc_core_device *xdev)
+{
+ return min_t(int, xdev->dev_res->eq_table.num_comp_vectors,
+ XSC_ETH_MAX_NUM_CHANNELS);
+}
+
+static void xsc_build_rss_param(struct xsc_rss_params *rss_param)
+{
+ enum xsc_traffic_types tt;
+
+ for (tt = 0; tt < XSC_NUM_INDIR_TIRS; tt++) {
+ rss_param->rx_hash_fields[tt] =
+ tirc_default_config[tt].rx_hash_fields;
+ }
+ rss_param->rss_hash_tmpl = XSC_HASH_IP_PORTS | XSC_HASH_IP6_PORTS;
+}
+
+static void xsc_eth_build_nic_params(struct xsc_adapter *adapter, u32 ch_num)
+{
+ struct xsc_eth_params *params = &adapter->nic_param;
+
+ params->mtu = SW_DEFAULT_MTU;
+
+ params->max_num_ch = ch_num;
+ params->num_channels = ch_num;
+
+ xsc_build_rss_param(&adapter->rss_param);
+}
+
+static int xsc_eth_netdev_init(struct xsc_adapter *adapter)
+{
+ unsigned int node, nch;
+
+ nch = adapter->nic_param.max_num_ch;
+ node = dev_to_node(adapter->dev);
+ adapter->txq2sq = kcalloc_node(nch,
+ sizeof(*adapter->txq2sq),
+ GFP_KERNEL, node);
+ if (!adapter->txq2sq)
+ return -ENOMEM;
+
+ netif_carrier_off(adapter->netdev);
+
+ return 0;
+}
+
+static bool xsc_eth_get_link_status(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ u16 vport = 0;
+ bool link_up;
+
+ link_up = xsc_core_query_vport_state(xdev, vport);
+
+ return link_up ? true : false;
+}
+
+static int xsc_eth_change_link_status(struct xsc_adapter *adapter)
+{
+ bool link_up;
+
+ link_up = xsc_eth_get_link_status(adapter);
+
+ if (link_up && !netif_carrier_ok(adapter->netdev)) {
+ netdev_info(adapter->netdev, "Link up\n");
+ netif_carrier_on(adapter->netdev);
+ } else if (!link_up && netif_carrier_ok(adapter->netdev)) {
+ netdev_info(adapter->netdev, "Link down\n");
+ netif_carrier_off(adapter->netdev);
+ }
+
+ return 0;
+}
+
+/*
+ * Registered with the PCI layer, which runs it on its own workqueue for
+ * every XSC_CMD_EVENT_RESP_CHANGE_LINK event.
+ */
+static void xsc_eth_link_event_handler(void *arg)
+{
+ struct xsc_core_device *xdev = arg;
+ struct xsc_adapter *adapter = xdev->eth_priv;
+ int err;
+
+ if (!adapter)
+ return;
+
+ err = xsc_eth_change_link_status(adapter);
+ if (err)
+ netdev_err(adapter->netdev, "failed to change linkstatus, err=%d\n",
+ err);
+}
+
+static void xsc_set_default_xps_cpumasks(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct xsc_channel *c;
+ unsigned int ix;
+
+ for (ix = 0; ix < adapter->channels.num_chl; ix++) {
+ c = &adapter->channels.c[ix];
+
+ cpumask_clear(xdev->xps_cpumask);
+ cpumask_set_cpu(c->cpu, xdev->xps_cpumask);
+ netif_set_xps_queue(adapter->netdev, xdev->xps_cpumask, ix);
+ }
+}
+
+static int xsc_set_port_admin_status(struct xsc_adapter *adapter,
+ enum xsc_port_status status)
+{
+ struct xsc_event_set_port_admin_status_mbox_out out;
+ struct xsc_event_set_port_admin_status_mbox_in in = {};
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_PORT_ADMIN_STATUS);
+ in.status = cpu_to_be16(status);
+
+ return xsc_cmd_exec(adapter->xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+static int xsc_eth_open(struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ int ret;
+
+ if (adapter->opened) {
+ netdev_err(adapter->netdev, "ndo_open on an already opened device\n");
+ return 0;
+ }
+
+ if (xsc_eth_get_link_status(adapter)) {
+ netdev_info(netdev, "Link up\n");
+ netif_carrier_on(adapter->netdev);
+ } else {
+ netdev_info(netdev, "Link down\n");
+ }
+
+ adapter->opened = true;
+
+ xsc_set_default_xps_cpumasks(adapter);
+
+ ret = xsc_set_port_admin_status(adapter, XSC_PORT_UP);
+ if (ret)
+ netdev_err(netdev, "failed to set port admin status up, err=%d\n",
+ ret);
+
+ return 0;
+}
+
+static int xsc_eth_close(struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ int ret;
+
+ if (!adapter->opened)
+ return 0;
+ adapter->opened = false;
+
+ netif_carrier_off(adapter->netdev);
+
+ ret = xsc_set_port_admin_status(adapter, XSC_PORT_DOWN);
+ if (ret)
+ netdev_err(adapter->netdev, "failed to set port admin status down, err=%d\n",
+ ret);
+
+ return ret;
+}
+
+static int xsc_eth_set_hw_mtu(struct xsc_core_device *xdev,
+ u16 mtu, u16 rx_buf_sz)
+{
+ struct xsc_set_mtu_mbox_out out;
+ struct xsc_set_mtu_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(struct xsc_set_mtu_mbox_in));
+ memset(&out, 0, sizeof(struct xsc_set_mtu_mbox_out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTU);
+ in.mtu = cpu_to_be16(mtu);
+ in.rx_buf_sz_min = cpu_to_be16(rx_buf_sz);
+ in.mac_port = xdev->mac_port;
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(struct xsc_set_mtu_mbox_in), &out,
+ sizeof(struct xsc_set_mtu_mbox_out));
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "failed to set hw_mtu=%u rx_buf_sz=%u, err=%d, status=%d\n",
+ mtu, rx_buf_sz, err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+
+/* Real xmit arrives with the TX path patch; keep TX live until then. */
+static netdev_tx_t xsc_eth_xmit_drop(struct sk_buff *skb,
+ struct net_device *netdev)
+{
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+}
+
+static const struct net_device_ops xsc_netdev_ops = {
+ .ndo_open = xsc_eth_open,
+ .ndo_stop = xsc_eth_close,
+ .ndo_start_xmit = xsc_eth_xmit_drop,
+};
+
+static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ /* Set up network device as normal. */
+ netdev->netdev_ops = &xsc_netdev_ops;
+
+ netdev->min_mtu = SW_DEFAULT_MTU;
+ netdev->max_mtu = SW_DEFAULT_MTU;
+ /*mtu - macheaderlen - ipheaderlen should be aligned in 8B*/
+ netdev->mtu = SW_DEFAULT_MTU;
+
+ netdev->features = NETIF_F_HIGHDMA;
+}
+
+static int xsc_eth_nic_init(struct xsc_adapter *adapter, u32 ch_num)
+{
+ int err;
+
+ xsc_eth_build_nic_params(adapter, ch_num);
+
+ err = xsc_eth_netdev_init(adapter);
+ if (err)
+ return err;
+
+ xsc_eth_build_nic_netdev(adapter);
+
+ return 0;
+}
+
+static void xsc_eth_nic_cleanup(struct xsc_adapter *adapter)
+{
+ kfree(adapter->txq2sq);
+}
+
+static int xsc_eth_get_mac(struct xsc_core_device *xdev, char *mac)
+{
+ struct xsc_query_eth_mac_mbox_out *out;
+ struct xsc_query_eth_mac_mbox_in in;
+ int err;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_ETH_MAC);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err || out->hdr.status) {
+ netdev_warn(xsc_dev_to_netdev(xdev),
+ "get mac failed! err=%d, out.status=%u\n",
+ err, out->hdr.status);
+ err = -ENOEXEC;
+ goto exit;
+ }
+
+ memcpy(mac, out->mac, ETH_ALEN);
+ netdev_dbg(xsc_dev_to_netdev(xdev),
+ "get mac %02x:%02x:%02x:%02x:%02x:%02x\n",
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+
+exit:
+ kfree(out);
+ return err;
+}
+
+static void xsc_eth_l2_addr_init(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ char mac[6] = {0};
+ int ret = 0;
+
+ ret = xsc_eth_get_mac(adapter->xdev, mac);
+ if (ret) {
+ netdev_warn(netdev, "get mac failed %d, generate random mac...",
+ ret);
+ eth_random_addr(mac);
+ netdev->addr_assign_type = NET_ADDR_RANDOM;
+ } else {
+ netdev->addr_assign_type = NET_ADDR_PERM;
+ }
+ eth_hw_addr_set(netdev, mac);
+
+ if (netdev->addr_assign_type == NET_ADDR_PERM &&
+ !is_valid_ether_addr(netdev->perm_addr))
+ memcpy(netdev->perm_addr, netdev->dev_addr, netdev->addr_len);
+}
+
+/*
+ * Program the MAC address and the MTU into the hardware. Called once from
+ * probe(), before the netdev is registered: the netdev layer marks the
+ * device present itself in register_netdevice(), so there is nothing to
+ * attach here and no need to take the rtnl lock.
+ */
+static int xsc_eth_nic_enable(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+
+ xsc_eth_l2_addr_init(adapter);
+
+ return xsc_eth_set_hw_mtu(xdev, XSC_SW2HW_MTU(adapter->nic_param.mtu),
+ XSC_SW2HW_RX_PKT_LEN(adapter->nic_param.mtu));
+}
+
+static int xsc_eth_probe(struct auxiliary_device *adev,
+ const struct auxiliary_device_id *adev_id)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ int num_chl;
+ int err;
+
+ num_chl = xsc_get_max_num_channels(xdev);
+
+ netdev = alloc_etherdev_mqs(sizeof(struct xsc_adapter),
+ num_chl, num_chl);
+ if (!netdev) {
+ pr_err("alloc_etherdev_mqs failed, txq=%d, rxq=%d\n",
+ num_chl, num_chl);
+ return -ENOMEM;
+ }
+
+ netdev->dev.parent = &xdev->pdev->dev;
+ adapter = netdev_priv(netdev);
+ adapter->netdev = netdev;
+ adapter->pdev = xdev->pdev;
+ adapter->dev = &adapter->pdev->dev;
+ adapter->xdev = xdev;
+ xdev->eth_priv = adapter;
+
+ err = xsc_eth_nic_init(adapter, num_chl);
+ if (err) {
+ netdev_err(netdev, "xsc_eth_nic_init failed, err=%d\n", err);
+ goto err_free_netdev;
+ }
+
+ err = xsc_eth_nic_enable(adapter);
+ if (err) {
+ netdev_err(netdev, "xsc_eth_nic_enable failed, err=%d\n", err);
+ goto err_nic_cleanup;
+ }
+
+ adapter->stats = kvzalloc_obj(*adapter->stats, GFP_KERNEL);
+ if (!adapter->stats) {
+ err = -ENOMEM;
+ goto err_nic_cleanup;
+ }
+
+ err = register_netdev(netdev);
+ if (err) {
+ netdev_err(netdev, "register_netdev failed, err=%d\n", err);
+ goto err_free_stats;
+ }
+
+ xdev->netdev = netdev;
+
+ xsc_irq_event_register(xdev, xsc_eth_link_event_handler);
+
+ return 0;
+
+err_free_stats:
+ kvfree(adapter->stats);
+err_nic_cleanup:
+ xsc_eth_nic_cleanup(adapter);
+err_free_netdev:
+ xdev->eth_priv = NULL;
+ free_netdev(netdev);
+
+ return err;
+}
+
+static void xsc_eth_remove(struct auxiliary_device *adev)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_adapter *adapter;
+
+ if (!xdev)
+ return;
+
+ adapter = xdev->eth_priv;
+ if (!adapter)
+ return;
+
+ /* Stop link events reaching this adapter and wait out one already in
+ * flight, which would otherwise touch the memory freed below.
+ */
+ xsc_irq_event_unregister(xdev);
+
+ /* Runs ndo_stop(), so the queues and the HCA are shut down first. */
+ unregister_netdev(adapter->netdev);
+
+ /* Must precede free_netdev(): the adapter lives in the netdev priv. */
+ xsc_eth_nic_cleanup(adapter);
+ kvfree(adapter->stats);
+ free_netdev(adapter->netdev);
+
+ xdev->netdev = NULL;
+ xdev->eth_priv = NULL;
+}
+
+static const struct auxiliary_device_id xsc_eth_id_table[] = {
+ { .name = XSC_PCI_DRV_NAME "." XSC_ETH_ADEV_NAME },
+ {},
+};
+MODULE_DEVICE_TABLE(auxiliary, xsc_eth_id_table);
+
+static struct auxiliary_driver xsc_eth_driver = {
+ .name = "eth",
+ .probe = xsc_eth_probe,
+ .remove = xsc_eth_remove,
+ .id_table = xsc_eth_id_table,
+};
+module_auxiliary_driver(xsc_eth_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Yunsilicon XSC ethernet driver");
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
new file mode 100644
index 000000000..4d9f92e09
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ETH_H
+#define __XSC_ETH_H
+
+#include <linux/udp.h>
+
+#include "common/xsc_device.h"
+#include "xsc_eth_common.h"
+#include "xsc_eth_stats.h"
+
+#define XSC_INVALID_LKEY 0x100
+
+struct xsc_rss_params {
+ u32 rx_hash_fields[XSC_NUM_INDIR_TIRS];
+ u32 rss_hash_tmpl;
+};
+
+struct xsc_adapter {
+ struct net_device *netdev;
+ struct pci_dev *pdev;
+ struct device *dev;
+ struct xsc_core_device *xdev;
+
+ struct xsc_eth_params nic_param;
+ struct xsc_rss_params rss_param;
+
+ struct xsc_eth_channels channels;
+ struct xsc_sq **txq2sq;
+
+ /* true between ndo_open and ndo_stop */
+ bool opened;
+
+ struct xsc_stats *stats;
+};
+
+static inline struct net_device *xsc_dev_to_netdev(struct xsc_core_device *xdev)
+{
+ return ((struct xsc_adapter *)xdev->eth_priv)->netdev;
+}
+
+#endif /* __XSC_ETH_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h
new file mode 100644
index 000000000..262a888a5
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ETH_COMMON_H
+#define __XSC_ETH_COMMON_H
+
+#include "xsc_queue.h"
+#include "xsc_pph.h"
+
+#define SW_DEFAULT_MTU ETH_DATA_LEN
+
+#define XSC_ETH_HW_MTU_SEND 9800
+#define XSC_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN)
+#define XSC_SW2HW_MTU(mtu) ((mtu) + XSC_ETH_HARD_MTU)
+#define XSC_SW2HW_HLEN (ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN + \
+ XSC_PPH_HEAD_LEN)
+#define XSC_SW2HW_FRAG_SIZE(mtu) ((mtu) + XSC_SW2HW_HLEN)
+#define XSC_ETH_RX_MAX_HEAD_ROOM 256
+#define XSC_SW2HW_RX_PKT_LEN(mtu) \
+ ((mtu) + ETH_HLEN + XSC_ETH_RX_MAX_HEAD_ROOM)
+
+#define XSC_RX_MAX_HEAD (256)
+
+#define XSC_QPN_SQN_STUB 1025
+
+#define XSC_ETH_MAX_NUM_CHANNELS 256
+
+#define XSC_SKB_FRAG_SZ(len) (SKB_DATA_ALIGN(len) + \
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))
+
+#define XSC_RQCQ_ELE_SZ 32
+#define XSC_SQCQ_ELE_SZ 32
+
+#define XSC_LOG_RQCQ_SZ 0xb
+#define XSC_LOG_SQCQ_SZ 0xa
+
+#define XSC_RQCQ_ELE_NUM BIT(XSC_LOG_RQCQ_SZ)
+#define XSC_SQCQ_ELE_NUM BIT(XSC_LOG_SQCQ_SZ)
+
+enum xsc_port_status {
+ XSC_PORT_DOWN = 0,
+ XSC_PORT_UP = 1,
+};
+
+enum xsc_queue_type {
+ XSC_QUEUE_TYPE_RQCQ,
+ XSC_QUEUE_TYPE_SQCQ,
+ XSC_QUEUE_TYPE_RQ,
+ XSC_QUEUE_TYPE_SQ,
+};
+
+struct xsc_queue_attr {
+ u32 ele_num;
+ u32 ele_size;
+ u8 ele_log_size;
+ u8 q_log_size;
+};
+
+struct xsc_eth_rx_wqe_cyc {
+ DECLARE_FLEX_ARRAY(struct xsc_wqe_data_seg, data);
+};
+
+struct xsc_cq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr cq_attr;
+};
+
+struct xsc_rq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr rq_attr;
+ struct xsc_rq_frags_info frags_info;
+};
+
+struct xsc_sq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr sq_attr;
+};
+
+struct xsc_channel_param {
+ struct xsc_cq_param rqcq_param;
+ struct xsc_cq_param sqcq_param;
+ struct xsc_rq_param rq_param;
+ struct xsc_sq_param sq_param;
+};
+
+struct xsc_eth_qp {
+ struct xsc_rq rq;
+ struct xsc_sq sq;
+};
+
+struct xsc_eth_params {
+ u16 num_channels;
+ u16 max_num_ch;
+ u32 mtu;
+ u32 sq_size;
+ u32 rq_size;
+ u32 rq_frags_size;
+};
+
+struct xsc_channel {
+ /* data path */
+ struct xsc_eth_qp qp;
+ struct napi_struct napi;
+ int chl_idx;
+
+ /*relationship*/
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ int cpu;
+
+ /* data path - accessed per napi poll */
+ const struct cpumask *aff_mask;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_eth_channels {
+ struct xsc_channel *c;
+ unsigned int num_chl;
+ u32 rqn_base;
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
new file mode 100644
index 000000000..a391afe97
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_EN_STATS_H
+#define __XSC_EN_STATS_H
+
+#include "xsc_eth_common.h"
+
+struct xsc_rq_stats {
+ u64 packets;
+ u64 bytes;
+};
+
+struct xsc_sq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 dropped;
+};
+
+struct xsc_channel_stats {
+ struct xsc_sq_stats sq;
+ struct xsc_rq_stats rq;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_stats {
+ struct xsc_channel_stats channel_stats[XSC_ETH_MAX_NUM_CHANNELS];
+};
+
+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,
+ struct rtnl_link_stats64 *s);
+
+#endif /* XSC_EN_STATS_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
new file mode 100644
index 000000000..dece589ab
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_RXTX_H
+#define __XSC_RXTX_H
+
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+
+#include "xsc_eth.h"
+
+enum {
+ XSC_ETH_WQE_NONE_CSUM,
+ XSC_ETH_WQE_INNER_CSUM,
+ XSC_ETH_WQE_OUTER_CSUM,
+ XSC_ETH_WQE_INNER_AND_OUTER_CSUM,
+};
+
+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq);
+void xsc_cq_notify_hw(struct xsc_cq *cq);
+int xsc_eth_napi_poll(struct napi_struct *napi, int budget);
+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq);
+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,
+ struct xsc_rq *rq, struct xsc_cqe *cqe);
+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix);
+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph);
+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph);
+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget);
+
+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev);
+
+static inline void xsc_tx_dma_unmap(struct device *dev, struct xsc_sq_dma *dma)
+{
+ switch (dma->type) {
+ case XSC_DMA_MAP_SINGLE:
+ dma_unmap_single(dev, dma->addr, dma->size, DMA_TO_DEVICE);
+ break;
+ case XSC_DMA_MAP_PAGE:
+ dma_unmap_page(dev, dma->addr, dma->size, DMA_TO_DEVICE);
+ break;
+ default:
+ break;
+ }
+}
+
+static inline struct xsc_sq_dma *xsc_dma_get(struct xsc_sq *sq, u32 i)
+{
+ return &sq->db.dma_fifo[i & sq->dma_fifo_mask];
+}
+
+static inline bool xsc_wqc_has_room_for(struct xsc_wq_cyc *wq,
+ u16 cc, u16 pc, u16 n)
+{
+ return (xsc_wq_cyc_ctr2ix(wq, cc - pc) >= n) || (cc == pc);
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_cqe_buff(struct xsc_cqwq *wq, u32 ix)
+{
+ struct xsc_cqe *cqe = xsc_frag_buf_get_wqe(&wq->fbc, ix);
+
+ return cqe;
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_cqe(struct xsc_cqwq *wq)
+{
+ u32 ci = xsc_cqwq_get_ci(wq);
+ u8 cqe_ownership_bit;
+ struct xsc_cqe *cqe;
+ u8 sw_ownership_val;
+
+ cqe = xsc_cqwq_get_cqe_buff(wq, ci);
+
+ cqe_ownership_bit = !!(le32_to_cpu(cqe->data1) & XSC_CQE_DATA1_OWNER);
+ sw_ownership_val = xsc_cqwq_get_wrap_cnt(wq) & 1;
+
+ if (cqe_ownership_bit != sw_ownership_val)
+ return NULL;
+
+ /* ensure cqe content is read after cqe ownership bit */
+ dma_rmb();
+
+ return cqe;
+}
+
+#endif /* XSC_RXTX_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c
new file mode 100644
index 000000000..b69a4e373
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.
+ */
+
+#include "xsc_eth_wq.h"
+#include "xsc_eth.h"
+
+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq)
+{
+ return (u32)wq->fbc.sz_m1 + 1;
+}
+
+static u32 wq_get_byte_sz(u8 log_sz, u8 log_stride)
+{
+ return ((u32)1 << log_sz) << log_stride;
+}
+
+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_cqwq *wq,
+ struct xsc_wq_ctrl *wq_ctrl)
+{
+ u8 log_wq_stride = ele_log_size;
+ u8 log_wq_sz = q_log_size;
+ int err;
+
+ err = xsc_core_frag_buf_alloc_node(xdev,
+ wq_get_byte_sz(log_wq_sz,
+ log_wq_stride),
+ &wq_ctrl->buf,
+ param->buf_numa_node);
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "xsc_core_frag_buf_alloc_node failed, %d\n", err);
+ return err;
+ }
+
+ xsc_init_fbc(wq_ctrl->buf.frags, log_wq_stride, log_wq_sz, &wq->fbc);
+ wq_ctrl->xdev = xdev;
+ return 0;
+}
+
+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_wq_cyc *wq,
+ struct xsc_wq_ctrl *wq_ctrl)
+{
+ struct xsc_frag_buf_ctrl *fbc = &wq->fbc;
+ u8 log_wq_stride = ele_log_size;
+ u8 log_wq_sz = q_log_size;
+ int err;
+
+ err = xsc_core_frag_buf_alloc_node(xdev,
+ wq_get_byte_sz(log_wq_sz,
+ log_wq_stride),
+ &wq_ctrl->buf,
+ param->buf_numa_node);
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "xsc_core_frag_buf_alloc_node failed, %d\n", err);
+ return err;
+ }
+
+ xsc_init_fbc(wq_ctrl->buf.frags, log_wq_stride, log_wq_sz, fbc);
+ wq->sz = xsc_wq_cyc_get_size(wq);
+ wq_ctrl->xdev = xdev;
+ return 0;
+}
+
+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl)
+{
+ xsc_core_frag_buf_free(wq_ctrl->xdev, &wq_ctrl->buf);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h
new file mode 100644
index 000000000..01cee9fd1
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h
@@ -0,0 +1,185 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.
+ */
+
+#ifndef __XSC_WQ_H
+#define __XSC_WQ_H
+
+#include "common/xsc_core.h"
+
+struct xsc_wq_param {
+ int buf_numa_node;
+ int db_numa_node;
+};
+
+struct xsc_wq_ctrl {
+ struct xsc_core_device *xdev;
+ struct xsc_frag_buf buf;
+};
+
+struct xsc_wq_cyc {
+ struct xsc_frag_buf_ctrl fbc;
+ u16 sz;
+ u16 wqe_ctr;
+ u16 cur_sz;
+};
+
+struct xsc_cqwq {
+ struct xsc_frag_buf_ctrl fbc;
+ __be32 *db;
+ u32 cc; /* consumer counter */
+};
+
+enum xsc_res_type {
+ XSC_RES_RQ,
+ XSC_RES_SQ,
+};
+
+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq);
+
+/*api for eth driver*/
+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_cqwq *wq,
+ struct xsc_wq_ctrl *wq_ctrl);
+
+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_wq_cyc *wq,
+ struct xsc_wq_ctrl *wq_ctrl);
+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl);
+
+static inline void xsc_init_fbc_offset(struct xsc_buf_list *frags,
+ u8 log_stride, u8 log_sz,
+ u16 strides_offset,
+ struct xsc_frag_buf_ctrl *fbc)
+{
+ fbc->frags = frags;
+ fbc->log_stride = log_stride;
+ fbc->log_sz = log_sz;
+ fbc->sz_m1 = (1 << fbc->log_sz) - 1;
+ fbc->log_frag_strides = PAGE_SHIFT - fbc->log_stride;
+ fbc->frag_sz_m1 = (1 << fbc->log_frag_strides) - 1;
+ fbc->strides_offset = strides_offset;
+}
+
+static inline void xsc_init_fbc(struct xsc_buf_list *frags,
+ u8 log_stride, u8 log_sz,
+ struct xsc_frag_buf_ctrl *fbc)
+{
+ xsc_init_fbc_offset(frags, log_stride, log_sz, 0, fbc);
+}
+
+static inline void *xsc_frag_buf_get_wqe(struct xsc_frag_buf_ctrl *fbc,
+ u32 ix)
+{
+ unsigned int frag;
+
+ ix += fbc->strides_offset;
+ frag = ix >> fbc->log_frag_strides;
+
+ return fbc->frags[frag].buf +
+ ((fbc->frag_sz_m1 & ix) << fbc->log_stride);
+}
+
+static inline u32
+xsc_frag_buf_get_idx_last_contig_stride(struct xsc_frag_buf_ctrl *fbc, u32 ix)
+{
+ u32 last_frag_stride_idx = (ix + fbc->strides_offset) | fbc->frag_sz_m1;
+
+ return min_t(u32, last_frag_stride_idx - fbc->strides_offset,
+ fbc->sz_m1);
+}
+
+static inline int xsc_wq_cyc_missing(struct xsc_wq_cyc *wq)
+{
+ return wq->sz - wq->cur_sz;
+}
+
+static inline int xsc_wq_cyc_is_empty(struct xsc_wq_cyc *wq)
+{
+ return !wq->cur_sz;
+}
+
+static inline void xsc_wq_cyc_push(struct xsc_wq_cyc *wq)
+{
+ wq->wqe_ctr++;
+ wq->cur_sz++;
+}
+
+static inline void xsc_wq_cyc_push_n(struct xsc_wq_cyc *wq, u8 n)
+{
+ wq->wqe_ctr += n;
+ wq->cur_sz += n;
+}
+
+static inline void xsc_wq_cyc_pop(struct xsc_wq_cyc *wq)
+{
+ wq->cur_sz--;
+}
+
+static inline u16 xsc_wq_cyc_ctr2ix(struct xsc_wq_cyc *wq, u16 ctr)
+{
+ return ctr & wq->fbc.sz_m1;
+}
+
+static inline u16 xsc_wq_cyc_get_head(struct xsc_wq_cyc *wq)
+{
+ return xsc_wq_cyc_ctr2ix(wq, wq->wqe_ctr);
+}
+
+static inline u16 xsc_wq_cyc_get_tail(struct xsc_wq_cyc *wq)
+{
+ return xsc_wq_cyc_ctr2ix(wq, wq->wqe_ctr - wq->cur_sz);
+}
+
+static inline void *xsc_wq_cyc_get_wqe(struct xsc_wq_cyc *wq, u16 ix)
+{
+ return xsc_frag_buf_get_wqe(&wq->fbc, ix);
+}
+
+static inline u32 xsc_cqwq_ctr2ix(struct xsc_cqwq *wq, u32 ctr)
+{
+ return ctr & wq->fbc.sz_m1;
+}
+
+static inline u32 xsc_cqwq_get_ci(struct xsc_cqwq *wq)
+{
+ return xsc_cqwq_ctr2ix(wq, wq->cc);
+}
+
+static inline u32 xsc_cqwq_get_ctr_wrap_cnt(struct xsc_cqwq *wq, u32 ctr)
+{
+ return ctr >> wq->fbc.log_sz;
+}
+
+static inline u32 xsc_cqwq_get_wrap_cnt(struct xsc_cqwq *wq)
+{
+ return xsc_cqwq_get_ctr_wrap_cnt(wq, wq->cc);
+}
+
+static inline void xsc_cqwq_pop(struct xsc_cqwq *wq)
+{
+ wq->cc++;
+}
+
+static inline u32 xsc_cqwq_get_size(struct xsc_cqwq *wq)
+{
+ return wq->fbc.sz_m1 + 1;
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_wqe(struct xsc_cqwq *wq, u32 ix)
+{
+ struct xsc_cqe *cqe = xsc_frag_buf_get_wqe(&wq->fbc, ix);
+
+ return cqe;
+}
+
+#endif /* __XSC_WQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h
new file mode 100644
index 000000000..cc9eec866
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PP_H
+#define __XSC_PP_H
+
+enum {
+ XSC_HASH_FIELD_SEL_SRC_IP = BIT(0),
+ XSC_HASH_FIELD_SEL_PROTO = BIT(1),
+ XSC_HASH_FIELD_SEL_DST_IP = BIT(2),
+ XSC_HASH_FIELD_SEL_SPORT = BIT(3),
+ XSC_HASH_FIELD_SEL_DPORT = BIT(4),
+ XSC_HASH_FIELD_SEL_SRC_IPV6 = BIT(5),
+ XSC_HASH_FIELD_SEL_DST_IPV6 = BIT(6),
+ XSC_HASH_FIELD_SEL_SPORT_V6 = BIT(7),
+ XSC_HASH_FIELD_SEL_DPORT_V6 = BIT(8),
+};
+
+#define XSC_HASH_IP (XSC_HASH_FIELD_SEL_SRC_IP |\
+ XSC_HASH_FIELD_SEL_DST_IP |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP_PORTS (XSC_HASH_FIELD_SEL_SRC_IP |\
+ XSC_HASH_FIELD_SEL_DST_IP |\
+ XSC_HASH_FIELD_SEL_SPORT |\
+ XSC_HASH_FIELD_SEL_DPORT |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP6 (XSC_HASH_FIELD_SEL_SRC_IPV6 |\
+ XSC_HASH_FIELD_SEL_DST_IPV6 |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP6_PORTS (XSC_HASH_FIELD_SEL_SRC_IPV6 |\
+ XSC_HASH_FIELD_SEL_DST_IPV6 |\
+ XSC_HASH_FIELD_SEL_SPORT_V6 |\
+ XSC_HASH_FIELD_SEL_DPORT_V6 |\
+ XSC_HASH_FIELD_SEL_PROTO)
+
+#endif /* __XSC_PP_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h
new file mode 100644
index 000000000..cc544df2c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PPH_H
+#define __XSC_PPH_H
+
+#define XSC_PPH_HEAD_LEN 64
+
+#define OUTER_L3_BIT BIT(3)
+#define OUTER_L4_BIT BIT(2)
+#define INNER_L3_BIT BIT(1)
+#define INNER_L4_BIT BIT(0)
+#define OUTER_BIT (OUTER_L3_BIT | OUTER_L4_BIT)
+#define INNER_BIT (INNER_L3_BIT | INNER_L4_BIT)
+#define OUTER_AND_INNER (OUTER_BIT | INNER_BIT)
+
+#define PACKET_UNKNOWN BIT(4)
+
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET (6UL)
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK (0XF00)
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET (8)
+
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET (20UL)
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK (0XFF)
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET (0)
+
+#define XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(PPH_BASE_ADDR) \
+ ((*(u16 *)((u8 *)(PPH_BASE_ADDR) + \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET) & \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK) >> \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET)
+
+#define XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(PPH_BASE_ADDR) \
+ ((*(u8 *)((u8 *)(PPH_BASE_ADDR) + \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET) & \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK) >> \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET)
+
+#endif /* __XSC_PPH_H */
+
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h
new file mode 100644
index 000000000..78007db79
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h
@@ -0,0 +1,183 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#ifndef __XSC_QUEUE_H
+#define __XSC_QUEUE_H
+
+#include <net/page_pool/types.h>
+#include <net/page_pool/helpers.h>
+
+#include "common/xsc_core.h"
+#include "xsc_eth_wq.h"
+
+enum {
+ XSC_SEND_WQE_DS = 16,
+ XSC_SEND_WQE_BB = 64,
+};
+
+enum {
+ XSC_RECV_WQE_DS = 16,
+ XSC_RECV_WQE_BB = 16,
+};
+
+enum {
+ XSC_ETH_RQ_STATE_ENABLED,
+};
+
+#define XSC_SEND_WQEBB_CTRL_NUM_DS 1
+
+/* each ds holds one fragment in skb */
+#define XSC_MAX_RX_FRAGS 4
+#define XSC_RX_FRAG_SZ_ORDER 0
+#define XSC_RX_FRAG_SZ (PAGE_SIZE << XSC_RX_FRAG_SZ_ORDER)
+#define DEFAULT_FRAG_SIZE (2048)
+
+enum {
+ XSC_ETH_SQ_STATE_ENABLED,
+};
+
+struct xsc_cq {
+ /* data path - accessed per cqe */
+ struct xsc_cqwq wq;
+
+ /* data path - accessed per napi poll */
+ u16 event_ctr;
+ struct napi_struct *napi;
+ struct xsc_core_cq xcq;
+ struct xsc_channel *channel;
+
+ /* control */
+ struct xsc_core_device *xdev;
+ struct xsc_wq_ctrl wq_ctrl;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_wqe_frag_info {
+ struct xsc_dma_info *di;
+ u32 offset;
+ u8 last_in_page;
+};
+
+struct xsc_dma_info {
+ struct page *page;
+ dma_addr_t addr;
+};
+
+struct xsc_rq_frag_info {
+ int frag_size;
+ int frag_stride;
+};
+
+struct xsc_rq_frags_info {
+ struct xsc_rq_frag_info arr[XSC_MAX_RX_FRAGS];
+ u8 num_frags;
+ u8 log_num_frags;
+ u8 wqe_bulk;
+ u8 wqe_bulk_min;
+ u8 frags_max_num;
+};
+
+struct xsc_rq;
+typedef void (*xsc_fp_handle_rx_cqe)(struct xsc_cqwq *cqwq, struct xsc_rq *rq,
+ struct xsc_cqe *cqe);
+typedef bool (*xsc_fp_post_rx_wqes)(struct xsc_rq *rq);
+typedef void (*xsc_fp_dealloc_wqe)(struct xsc_rq *rq, u16 ix);
+typedef struct sk_buff * (*xsc_fp_skb_from_cqe)(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt,
+ u8 has_pph);
+
+struct xsc_rq {
+ struct xsc_core_qp cqp;
+ struct {
+ struct xsc_wq_cyc wq;
+ struct xsc_wqe_frag_info *frags;
+ struct xsc_dma_info *di;
+ struct xsc_rq_frags_info info;
+ xsc_fp_skb_from_cqe skb_from_cqe;
+ } wqe;
+
+ struct {
+ u8 map_dir; /* dma map direction */
+ } buff;
+
+ struct page_pool *page_pool;
+ struct xsc_wq_ctrl wq_ctrl;
+ struct xsc_cq cq;
+ u32 rqn;
+ int ix;
+
+ unsigned long state;
+ struct xsc_rq_stats *stats;
+
+ u32 hw_mtu;
+ u32 frags_sz;
+
+ xsc_fp_handle_rx_cqe handle_rx_cqe;
+ xsc_fp_post_rx_wqes post_wqes;
+ xsc_fp_dealloc_wqe dealloc_wqe;
+} ____cacheline_aligned_in_smp;
+
+enum xsc_dma_map_type {
+ XSC_DMA_MAP_SINGLE,
+ XSC_DMA_MAP_PAGE
+};
+
+struct xsc_sq_dma {
+ dma_addr_t addr;
+ u32 size;
+ enum xsc_dma_map_type type;
+};
+
+struct xsc_tx_wqe_info {
+ struct sk_buff *skb;
+ u32 num_bytes;
+ u8 num_wqebbs;
+ u8 num_dma;
+};
+
+struct xsc_tx_wqe {
+ struct xsc_send_wqe_ctrl_seg ctrl;
+ struct xsc_wqe_data_seg data[];
+};
+
+struct xsc_sq {
+ struct xsc_core_qp cqp;
+ /* dirtied @completion */
+ u16 cc;
+ u32 dma_fifo_cc;
+
+ /* dirtied @xmit */
+ u16 pc ____cacheline_aligned_in_smp;
+ u32 dma_fifo_pc;
+
+ struct xsc_cq cq;
+
+ /* read only */
+ struct xsc_wq_cyc wq;
+ u32 dma_fifo_mask;
+ struct xsc_sq_stats *stats;
+ struct {
+ struct xsc_sq_dma *dma_fifo;
+ struct xsc_tx_wqe_info *wqe_info;
+ } db;
+ void __iomem *uar_map;
+ struct netdev_queue *txq;
+ u32 sqn;
+ u16 stop_room;
+
+ __be32 mkey_be;
+ unsigned long state;
+ unsigned int hw_mtu;
+
+ /* control path */
+ struct xsc_wq_ctrl wq_ctrl;
+ struct xsc_channel *channel;
+ int ch_ix;
+ int txq_ix;
+} ____cacheline_aligned_in_smp;
+
+#endif /* __XSC_QUEUE_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index 5c947da2a..b01fca3ac 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o alloc.o eq.o pci_irq.o adev.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
new file mode 100644
index 000000000..5118ff36b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+
+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport)
+{
+ struct xsc_query_vport_state_out out;
+ struct xsc_query_vport_state_in in;
+ u32 val = 0;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_VPORT_STATE);
+ val |= FIELD_PREP(XSC_QUERY_VPORT_VPORT_NUM_MASK, vport);
+ if (vport)
+ val |= XSC_QUERY_VPORT_OTHER_VPORT;
+ in.data0 = cpu_to_be32(val);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to query vport state, err=%d, status=%d\n",
+ err, out.hdr.status);
+ return err;
+ }
+
+ return out.state;
+}
+EXPORT_SYMBOL_GPL(xsc_core_query_vport_state);
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 07/15] net/xsc: add ethernet netdev registration
2026-11-04 15:57 ` [PATCH v1 07/15] net/xsc: add ethernet netdev registration Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Medium] net/xsc: xsc_eth_open() ignores xsc_set_port_admin_status() failure
- [Medium] net/xsc: unsynchronized carrier updates in xsc_eth_change_link_status()
- [Medium] net/xsc: missing MAC validation and signed char in xsc_eth_l2_addr_init()
- [Low] net/xsc: missing select AUXILIARY_BUS in YUNSILICON_XSC_ETH Kconfig
--
commit 9545c37957efb4f015ca5b167db9263b28b79635
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add ethernet netdev registration
This commit adds the xsc ethernet auxiliary driver to allocate and register a
net_device on top of the core device, initialize MAC address and hardware MTU
settings, and implement open, stop, and link status event handling along with
queue wrapper definitions.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
> new file mode 100644
> index 0000000000000..7428b26295b66
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
> +# All rights reserved.
> +# Yunsilicon driver configuration
> +#
> +
> +config YUNSILICON_XSC_ETH
> + tristate "Yunsilicon XSC ethernet driver"
> + depends on YUNSILICON_XSC_PCI
> + depends on NET
> + select PAGE_POOL
[Severity: Low]
Since drivers/net/ethernet/yunsilicon/xsc/net/main.c registers an
auxiliary_driver via module_auxiliary_driver(), should YUNSILICON_XSC_ETH
(or YUNSILICON_XSC_PCI) also select AUXILIARY_BUS so the build does not fail
with undefined references to __auxiliary_driver_register() and
auxiliary_driver_unregister() when no other enabled driver selects
CONFIG_AUXILIARY_BUS?
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> new file mode 100644
> index 0000000000000..21a50085fa879
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
[ ... ]
> +static int xsc_eth_change_link_status(struct xsc_adapter *adapter)
> +{
> + bool link_up;
> +
> + link_up = xsc_eth_get_link_status(adapter);
> +
> + if (link_up && !netif_carrier_ok(adapter->netdev)) {
> + netdev_info(adapter->netdev, "Link up\n");
> + netif_carrier_on(adapter->netdev);
> + } else if (!link_up && netif_carrier_ok(adapter->netdev)) {
> + netdev_info(adapter->netdev, "Link down\n");
> + netif_carrier_off(adapter->netdev);
> + }
> +
> + return 0;
> +}
> +
> +/*
> + * Registered with the PCI layer, which runs it on its own workqueue for
> + * every XSC_CMD_EVENT_RESP_CHANGE_LINK event.
> + */
> +static void xsc_eth_link_event_handler(void *arg)
> +{
> + struct xsc_core_device *xdev = arg;
> + struct xsc_adapter *adapter = xdev->eth_priv;
> + int err;
> +
> + if (!adapter)
> + return;
> +
> + err = xsc_eth_change_link_status(adapter);
> + if (err)
> + netdev_err(adapter->netdev, "failed to change linkstatus, err=%d\n",
> + err);
> +}
[Severity: Medium]
How are carrier state updates in xsc_eth_link_event_handler() ->
xsc_eth_change_link_status() synchronized with xsc_eth_open() and
xsc_eth_close()?
xsc_eth_probe() registers xsc_eth_link_event_handler() via
xsc_irq_event_register() while adapter->opened is false, and xsc_eth_close()
leaves it registered after clearing adapter->opened and calling
netif_carrier_off(). When an XSC_CMD_EVENT_RESP_CHANGE_LINK event runs on
xdev->event_wq, xsc_eth_change_link_status() neither checks adapter->opened
nor serializes with xsc_eth_open() or xsc_eth_close():
1. If xsc_eth_change_link_status() runs while the device is closed (or racing
with xsc_eth_close() before XSC_PORT_DOWN completes in firmware) and sees
link_up == true, it calls netif_carrier_on() on a closed interface. If the
link later goes down before or during xsc_eth_open(), the else branch in
xsc_eth_open() logs "Link down" without calling netif_carrier_off(), leaving
netif_carrier_ok() stuck true while the link is down.
2. If xsc_eth_open() sees xsc_eth_get_link_status() == true and a concurrent
link-down event runs xsc_eth_change_link_status() before xsc_eth_open() calls
netif_carrier_on(), xsc_eth_change_link_status() sees link_up == false while
netif_carrier_ok() is still false and does nothing, after which
xsc_eth_open() calls netif_carrier_on() on a down link.
3. In xsc_eth_probe(), register_netdev() is called before setting
xdev->netdev = netdev and before calling xsc_irq_event_register(), so an
immediate open after registration can race with initialization of those
fields.
[ ... ]
> +static int xsc_eth_open(struct net_device *netdev)
> +{
> + struct xsc_adapter *adapter = netdev_priv(netdev);
> + int ret;
> +
> + if (adapter->opened) {
> + netdev_err(adapter->netdev, "ndo_open on an already opened device\n");
> + return 0;
> + }
> +
> + if (xsc_eth_get_link_status(adapter)) {
> + netdev_info(netdev, "Link up\n");
> + netif_carrier_on(adapter->netdev);
> + } else {
> + netdev_info(netdev, "Link down\n");
> + }
> +
> + adapter->opened = true;
> +
> + xsc_set_default_xps_cpumasks(adapter);
> +
> + ret = xsc_set_port_admin_status(adapter, XSC_PORT_UP);
> + if (ret)
> + netdev_err(netdev, "failed to set port admin status up, err=%d\n",
> + ret);
> +
> + return 0;
> +}
[Severity: Medium]
If xsc_set_port_admin_status(adapter, XSC_PORT_UP) fails in xsc_eth_open(),
should xsc_eth_open() roll back adapter->opened and carrier state and return
ret instead of returning 0?
Returning 0 on failure leaves the netdev marked IFF_UP with adapter->opened
set to true and potentially netif_carrier_on() active even though bringing
the port administrative state up in firmware failed. Also, querying
xsc_eth_get_link_status(adapter) before calling
xsc_set_port_admin_status(adapter, XSC_PORT_UP) reads the vport link state
while the port is still administratively down.
[ ... ]
> +static int xsc_eth_get_mac(struct xsc_core_device *xdev, char *mac)
> +{
> + struct xsc_query_eth_mac_mbox_out *out;
> + struct xsc_query_eth_mac_mbox_in in;
> + int err;
> +
> + out = kzalloc_obj(*out, GFP_KERNEL);
> + if (!out)
> + return -ENOMEM;
> +
> + memset(&in, 0, sizeof(in));
> + in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_ETH_MAC);
> +
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
> + if (err || out->hdr.status) {
> + netdev_warn(xsc_dev_to_netdev(xdev),
> + "get mac failed! err=%d, out.status=%u\n",
> + err, out->hdr.status);
> + err = -ENOEXEC;
> + goto exit;
> + }
> +
> + memcpy(mac, out->mac, ETH_ALEN);
> + netdev_dbg(xsc_dev_to_netdev(xdev),
> + "get mac %02x:%02x:%02x:%02x:%02x:%02x\n",
> + mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
> +
> +exit:
> + kfree(out);
> + return err;
> +}
> +
> +static void xsc_eth_l2_addr_init(struct xsc_adapter *adapter)
> +{
> + struct net_device *netdev = adapter->netdev;
> + char mac[6] = {0};
> + int ret = 0;
> +
> + ret = xsc_eth_get_mac(adapter->xdev, mac);
> + if (ret) {
> + netdev_warn(netdev, "get mac failed %d, generate random mac...",
> + ret);
> + eth_random_addr(mac);
> + netdev->addr_assign_type = NET_ADDR_RANDOM;
> + } else {
> + netdev->addr_assign_type = NET_ADDR_PERM;
> + }
> + eth_hw_addr_set(netdev, mac);
> +
> + if (netdev->addr_assign_type == NET_ADDR_PERM &&
> + !is_valid_ether_addr(netdev->perm_addr))
> + memcpy(netdev->perm_addr, netdev->dev_addr, netdev->addr_len);
> +}
[Severity: Medium]
In xsc_eth_l2_addr_init() and xsc_eth_get_mac(), what happens if firmware
returns an all-zero or multicast MAC address without a command error?
xsc_eth_l2_addr_init() checks !is_valid_ether_addr(netdev->perm_addr) after
eth_hw_addr_set(), which is always true on a freshly allocated net_device
because netdev->perm_addr is still zeroed, rather than checking
is_valid_ether_addr(mac) before accepting the firmware MAC address. If
firmware returns an invalid MAC address, it will be assigned to dev_addr and
perm_addr instead of falling back to eth_hw_addr_random(netdev).
Also, mac is declared as char rather than u8, which causes pointer-sign
mismatches with eth_random_addr() and eth_hw_addr_set(), and sign-extends
bytes >= 0x80 when promoted to int for %02x in netdev_dbg() on architectures
where char is signed.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=7
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 08/15] net/xsc: add ethernet TX path
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (6 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 07/15] net/xsc: add ethernet netdev registration Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 09/15] net/xsc: add ethernet RX path and statistics Xin Tian
` (6 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement the ethernet transmit path. Opening the interface now
brings up the full software side: per-channel CQs and a raw-TSO
queue pair per channel, one NAPI instance polling the completion
queues, and the queue-to-sq mapping used by the stack. Queues are
NUMA-bound to their channel CPU, NAPI is enabled before the queue
pairs are transitioned to RTS, and the doorbell is programmed only
after a write barrier.
ndo_start_xmit builds a WQE from a control segment plus one data
segment per mapped fragment. Checksum offload mode is chosen from
skb->ip_summed, and TSO carries the MSS and inner header size in the
data segment fields. Over-length packets that would need more than
one WQE are linearized and retried. Each fragment's DMA mapping is
recorded in a completion-ordered FIFO so the completion path unmaps
in the order it was issued. The doorbell is skipped when
netdev_xmit_more() indicates more work is coming, stop_room keeps
one WQE slot free so the queue is stopped before it can overflow,
and byte queue limits track completion progress.
The NAPI poll loop drains the TX completion queue, unmaps DMA,
consumes SKBs with napi_consume_skb, updates the consumer index and
re-arms the CQ. Advertise SG, IP and IPv6 checksum offload, and
TSO.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../ethernet/yunsilicon/xsc/common/xsc_core.h | 3 +-
.../net/ethernet/yunsilicon/xsc/net/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/net/main.c | 936 +++++++++++++++++-
.../ethernet/yunsilicon/xsc/net/xsc_eth_tx.c | 345 +++++++
.../yunsilicon/xsc/net/xsc_eth_txrx.c | 143 +++
.../yunsilicon/xsc/net/xsc_eth_txrx.h | 1 +
.../net/ethernet/yunsilicon/xsc/pci/vport.c | 6 +-
7 files changed, 1415 insertions(+), 21 deletions(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
index bc41e88cc..ea536389e 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
@@ -568,7 +568,8 @@ int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,
void xsc_core_frag_buf_free(struct xsc_core_device *xdev,
struct xsc_frag_buf *buf);
-u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport);
+int xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport,
+ u8 *state);
static inline void *xsc_buf_offset(struct xsc_buf *buf, unsigned long offset)
{
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
index 0e6249cec..ffb819884 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc_eth.o
-xsc_eth-y := main.o xsc_eth_wq.o
+xsc_eth-y := main.o xsc_eth_wq.o xsc_eth_txrx.o xsc_eth_tx.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
index 21a50085f..a02f95145 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
@@ -95,22 +95,825 @@ static int xsc_eth_netdev_init(struct xsc_adapter *adapter)
return 0;
}
-static bool xsc_eth_get_link_status(struct xsc_adapter *adapter)
+static void xsc_eth_build_queue_param(struct xsc_adapter *adapter,
+ struct xsc_queue_attr *attr, u8 type)
{
struct xsc_core_device *xdev = adapter->xdev;
- u16 vport = 0;
- bool link_up;
- link_up = xsc_core_query_vport_state(xdev, vport);
+ if (adapter->nic_param.sq_size == 0)
+ adapter->nic_param.sq_size = min_t(u32,
+ BIT(xdev->caps.log_max_qp_depth),
+ XSC_SQCQ_ELE_NUM);
+ if (adapter->nic_param.rq_size == 0)
+ adapter->nic_param.rq_size = BIT(xdev->caps.log_max_qp_depth);
+
+ if (type == XSC_QUEUE_TYPE_RQCQ) {
+ attr->ele_num = min_t(int, XSC_RQCQ_ELE_NUM,
+ xdev->caps.max_cqes);
+ attr->ele_size = XSC_RQCQ_ELE_SZ;
+ attr->ele_log_size = order_base_2(XSC_RQCQ_ELE_SZ);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_SQCQ) {
+ attr->ele_num = min_t(int, XSC_SQCQ_ELE_NUM,
+ xdev->caps.max_cqes);
+ attr->ele_size = XSC_SQCQ_ELE_SZ;
+ attr->ele_log_size = order_base_2(XSC_SQCQ_ELE_SZ);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_RQ) {
+ attr->ele_num = adapter->nic_param.rq_size;
+ attr->ele_size = xdev->caps.recv_ds_num * XSC_RECV_WQE_DS;
+ attr->ele_log_size = order_base_2(attr->ele_size);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_SQ) {
+ attr->ele_num = adapter->nic_param.sq_size;
+ attr->ele_size = xdev->caps.send_ds_num * XSC_SEND_WQE_DS;
+ attr->ele_log_size = order_base_2(attr->ele_size);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ }
+}
+
+static u32 xsc_rx_get_linear_frag_sz(u32 mtu)
+{
+ u32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);
+
+ return XSC_SKB_FRAG_SZ(byte_count);
+}
+
+static bool xsc_rx_is_linear_skb(u32 mtu)
+{
+ u32 linear_frag_sz = xsc_rx_get_linear_frag_sz(mtu);
+
+ return linear_frag_sz <= PAGE_SIZE;
+}
+
+static u32 xsc_get_rq_frag_info(struct xsc_rq_frags_info *frags_info, u32 mtu)
+{
+ u32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);
+ int frag_stride;
+ int i = 0;
+
+ if (xsc_rx_is_linear_skb(mtu)) {
+ frag_stride = xsc_rx_get_linear_frag_sz(mtu);
+ frag_stride = roundup_pow_of_two(frag_stride);
+
+ frags_info->arr[0].frag_size = byte_count;
+ frags_info->arr[0].frag_stride = frag_stride;
+ frags_info->num_frags = 1;
+ frags_info->wqe_bulk = PAGE_SIZE / frag_stride;
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ goto out;
+ }
+
+ if (byte_count <= DEFAULT_FRAG_SIZE) {
+ frags_info->arr[0].frag_size = DEFAULT_FRAG_SIZE;
+ frags_info->arr[0].frag_stride = DEFAULT_FRAG_SIZE;
+ frags_info->num_frags = 1;
+ } else if (byte_count <= XSC_PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ } else if (byte_count <= (XSC_PAGE_SIZE_4K + DEFAULT_FRAG_SIZE)) {
+ if (PAGE_SIZE < 2 * XSC_PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = XSC_PAGE_SIZE_4K;
+ frags_info->arr[1].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[1].frag_stride = XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 2;
+ } else {
+ frags_info->arr[0].frag_size = 2 * XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 2 * XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ } else if (byte_count <= 2 * XSC_PAGE_SIZE_4K) {
+ if (PAGE_SIZE < 2 * XSC_PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = XSC_PAGE_SIZE_4K;
+ frags_info->arr[1].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[1].frag_stride = XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 2;
+ } else {
+ frags_info->arr[0].frag_size = 2 * XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 2 * XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ } else {
+ if (PAGE_SIZE < 4 * XSC_PAGE_SIZE_4K) {
+ frags_info->num_frags =
+ roundup(byte_count,
+ XSC_PAGE_SIZE_4K) / XSC_PAGE_SIZE_4K;
+ for (i = 0; i < frags_info->num_frags; i++) {
+ frags_info->arr[i].frag_size = XSC_PAGE_SIZE_4K;
+ frags_info->arr[i].frag_stride = XSC_PAGE_SIZE_4K;
+ }
+ } else {
+ frags_info->arr[0].frag_size = 4 * XSC_PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 4 * XSC_PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ }
+
+ if (PAGE_SIZE <= XSC_PAGE_SIZE_4K) {
+ frags_info->wqe_bulk_min = 4;
+ frags_info->wqe_bulk = max_t(u8, frags_info->wqe_bulk_min, 8);
+ } else if (PAGE_SIZE <= 2 * XSC_PAGE_SIZE_4K) {
+ frags_info->wqe_bulk = 2;
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ } else {
+ frags_info->wqe_bulk =
+ PAGE_SIZE /
+ (frags_info->num_frags * frags_info->arr[0].frag_size);
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ }
+
+out:
+ frags_info->log_num_frags = order_base_2(frags_info->num_frags);
+
+ return frags_info->num_frags * frags_info->arr[0].frag_size;
+}
+
+static void xsc_build_rq_frags_info(struct xsc_queue_attr *attr,
+ struct xsc_rq_frags_info *frags_info,
+ struct xsc_eth_params *params)
+{
+ params->rq_frags_size = xsc_get_rq_frag_info(frags_info, params->mtu);
+ frags_info->frags_max_num = attr->ele_size / XSC_RECV_WQE_DS;
+}
+
+static void xsc_eth_build_channel_param(struct xsc_adapter *adapter,
+ struct xsc_channel_param *chl_param)
+{
+ xsc_eth_build_queue_param(adapter, &chl_param->rqcq_param.cq_attr,
+ XSC_QUEUE_TYPE_RQCQ);
+ chl_param->rqcq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->sqcq_param.cq_attr,
+ XSC_QUEUE_TYPE_SQCQ);
+ chl_param->sqcq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->sq_param.sq_attr,
+ XSC_QUEUE_TYPE_SQ);
+ chl_param->sq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->rq_param.rq_attr,
+ XSC_QUEUE_TYPE_RQ);
+ chl_param->rq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_build_rq_frags_info(&chl_param->rq_param.rq_attr,
+ &chl_param->rq_param.frags_info,
+ &adapter->nic_param);
+}
- return link_up ? true : false;
+static void xsc_eth_cq_error_event(struct xsc_core_cq *xcq,
+ enum xsc_event event)
+{
+ struct xsc_cq *xsc_cq = container_of(xcq, struct xsc_cq, xcq);
+ struct xsc_core_device *xdev = xsc_cq->xdev;
+
+ if (event != XSC_EVENT_TYPE_CQ_ERROR) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "Unexpected event type %d on CQ %06x\n",
+ event, xcq->cqn);
+ return;
+ }
+
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "Eth catch CQ ERROR: %x, cqn: %d\n", event, xcq->cqn);
+}
+
+static void xsc_eth_completion_event(struct xsc_core_cq *xcq)
+{
+ struct xsc_cq *cq = container_of(xcq, struct xsc_cq, xcq);
+ struct xsc_core_device *xdev = cq->xdev;
+ struct xsc_rq *rq = NULL;
+
+ if (unlikely(!cq->channel)) {
+ netdev_err(xsc_dev_to_netdev(xdev), "cq%d->channel is null\n",
+ xcq->cqn);
+ return;
+ }
+
+ rq = &cq->channel->qp.rq;
+
+ if (!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state))
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "ch%d_cq%d, napi_flag=0x%lx\n",
+ cq->channel->chl_idx, xcq->cqn,
+ cq->napi->state);
+
+ napi_schedule(cq->napi);
+ cq->event_ctr++;
+}
+
+static int xsc_eth_alloc_cq(struct xsc_channel *c, struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ u8 ele_log_size = pcq_param->cq_attr.ele_log_size;
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ u8 q_log_size = pcq_param->cq_attr.q_log_size;
+ struct xsc_core_cq *core_cq = &pcq->xcq;
+ int ret;
+ u32 i;
+
+ pcq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+ pcq_param->wq.buf_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_cqwq_create(xdev, &pcq_param->wq,
+ q_log_size, ele_log_size, &pcq->wq,
+ &pcq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ core_cq->cqe_sz = pcq_param->cq_attr.ele_num;
+ core_cq->comp = xsc_eth_completion_event;
+ core_cq->event = xsc_eth_cq_error_event;
+ core_cq->vector = c->chl_idx;
+
+ for (i = 0; i < xsc_cqwq_get_size(&pcq->wq); i++) {
+ struct xsc_cqe *cqe = xsc_cqwq_get_wqe(&pcq->wq, i);
+
+ cqe->data1 |= cpu_to_le32(XSC_CQE_DATA1_OWNER);
+ }
+ pcq->xdev = xdev;
+
+ return ret;
+}
+
+static int xsc_eth_set_cq(struct xsc_channel *c,
+ struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ struct xsc_create_cq_mbox_in *in;
+ unsigned int hw_npages;
+ unsigned int irqn;
+ int inlen;
+ u32 eqn;
+ int ret;
+
+ hw_npages = DIV_ROUND_UP(pcq->wq_ctrl.buf.size, XSC_PAGE_SIZE_4K);
+ /*mbox size + pas size*/
+ inlen = sizeof(struct xsc_create_cq_mbox_in) +
+ sizeof(__be64) * hw_npages;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ /*construct param of in struct*/
+ ret = xsc_core_vector2eqn(xdev, c->chl_idx, &eqn, &irqn);
+ if (ret)
+ goto err_free;
+
+ in->ctx.eqn = cpu_to_be16(eqn);
+ in->ctx.log_cq_sz = pcq_param->cq_attr.q_log_size;
+ in->ctx.pa_num = cpu_to_be16(hw_npages);
+ in->ctx.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&pcq->wq_ctrl.buf,
+ &in->pas[0],
+ hw_npages);
+
+ ret = xsc_core_eth_create_cq(c->adapter->xdev, &pcq->xcq, in, inlen);
+ if (ret)
+ goto err_free;
+ pcq->xcq.irqn = irqn;
+ pcq->xcq.eq = xsc_core_eq_get(xdev, pcq->xcq.vector);
+ kvfree(in);
+ return 0;
+
+err_free:
+ kvfree(in);
+ return ret;
+}
+
+static void xsc_eth_free_cq(struct xsc_cq *cq)
+{
+ xsc_eth_wq_destroy(&cq->wq_ctrl);
+}
+
+static int xsc_eth_open_cq(struct xsc_channel *c,
+ struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ int ret;
+
+ ret = xsc_eth_alloc_cq(c, pcq, pcq_param);
+ if (ret)
+ return ret;
+
+ ret = xsc_eth_set_cq(c, pcq, pcq_param);
+ if (ret)
+ goto err_free_cq;
+
+ xsc_cq_notify_hw_rearm(pcq);
+
+ pcq->napi = &c->napi;
+ pcq->channel = c;
+
+ return 0;
+
+err_free_cq:
+ xsc_eth_free_cq(pcq);
+ return ret;
+}
+
+static void xsc_eth_close_cq(struct xsc_channel *c, struct xsc_cq *pcq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+
+ if (xsc_core_eth_destroy_cq(xdev, &pcq->xcq))
+ netdev_err(xsc_dev_to_netdev(xdev), "failed to close ch%d cq%d\n",
+ c->chl_idx, pcq->xcq.cqn);
+
+ xsc_eth_free_cq(pcq);
+}
+
+static void xsc_free_qp_sq_db(struct xsc_sq *sq)
+{
+ kvfree(sq->db.wqe_info);
+ kvfree(sq->db.dma_fifo);
+ sq->db.wqe_info = NULL;
+ sq->db.dma_fifo = NULL;
+}
+
+static void xsc_free_qp_sq(struct xsc_sq *sq)
+{
+ xsc_free_qp_sq_db(sq);
+ xsc_eth_wq_destroy(&sq->wq_ctrl);
+}
+
+static int xsc_eth_alloc_qp_sq_db(struct xsc_sq *sq, int numa)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ int wq_sz;
+ int df_sz;
+
+ wq_sz = xsc_wq_cyc_get_size(&sq->wq);
+ df_sz = wq_sz * xdev->caps.send_ds_num;
+ sq->db.dma_fifo = kvzalloc_node(array_size(df_sz,
+ sizeof(*sq->db.dma_fifo)),
+ GFP_KERNEL,
+ numa);
+ sq->db.wqe_info = kvzalloc_node(array_size(wq_sz,
+ sizeof(*sq->db.wqe_info)),
+ GFP_KERNEL,
+ numa);
+
+ if (!sq->db.dma_fifo || !sq->db.wqe_info) {
+ xsc_free_qp_sq_db(sq);
+ return -ENOMEM;
+ }
+
+ sq->dma_fifo_mask = df_sz - 1;
+
+ return 0;
+}
+
+static void xsc_eth_qp_event(struct xsc_core_qp *qp, int type)
+{
+ struct xsc_core_device *xdev;
+ struct xsc_rq *rq;
+ struct xsc_sq *sq;
+
+ if (qp->eth_queue_type == XSC_RES_RQ) {
+ rq = container_of(qp, struct xsc_rq, cqp);
+ xdev = rq->cq.xdev;
+ } else if (qp->eth_queue_type == XSC_RES_SQ) {
+ sq = container_of(qp, struct xsc_sq, cqp);
+ xdev = sq->cq.xdev;
+ } else {
+ pr_err("%s:Unknown eth qp type %d\n", __func__, type);
+ return;
+ }
+
+ switch (type) {
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "%s:Async event %x on QP %d\n",
+ __func__, type, qp->qpn);
+ break;
+ default:
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "%s: Unexpected event type %d on QP %d\n",
+ __func__, type, qp->qpn);
+ return;
+ }
+}
+
+static int xsc_eth_open_qp_sq(struct xsc_channel *c,
+ struct xsc_sq *psq,
+ struct xsc_sq_param *psq_param)
+{
+ u8 ele_log_size = psq_param->sq_attr.ele_log_size;
+ u8 q_log_size = psq_param->sq_attr.q_log_size;
+ struct xsc_modify_raw_qp_mbox_in *modify_in;
+ struct xsc_channel_stats *channel_stats;
+ struct xsc_create_qp_mbox_in *in;
+ struct xsc_core_device *xdev;
+ struct xsc_adapter *adapter;
+ struct xsc_stats *stats;
+ unsigned int hw_npages;
+ int inlen;
+ int ret;
+
+ adapter = c->adapter;
+ xdev = adapter->xdev;
+ stats = adapter->stats;
+ channel_stats = &stats->channel_stats[c->chl_idx];
+ psq->stats = &channel_stats->sq;
+ psq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_wq_cyc_create(xdev, &psq_param->wq,
+ q_log_size, ele_log_size, &psq->wq,
+ &psq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ hw_npages = DIV_ROUND_UP(psq->wq_ctrl.buf.size, XSC_PAGE_SIZE_4K);
+ inlen = sizeof(struct xsc_create_qp_mbox_in) +
+ sizeof(__be64) * hw_npages;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ ret = -ENOMEM;
+ goto err_sq_wq_destroy;
+ }
+ in->req.input_qpn = cpu_to_be16(XSC_QPN_SQN_STUB); /*no use for eth*/
+ in->req.qp_type = XSC_QUEUE_TYPE_RAW_TSO; /*default sq is tso qp*/
+ in->req.log_sq_sz = ilog2(xdev->caps.send_ds_num) + q_log_size;
+ in->req.pa_num = cpu_to_be16(hw_npages);
+ in->req.cqn_send = cpu_to_be16(psq->cq.xcq.cqn);
+ in->req.cqn_recv = in->req.cqn_send;
+ in->req.glb_funcid = cpu_to_be16(xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&psq->wq_ctrl.buf,
+ &in->req.pas[0], hw_npages);
+
+ ret = xsc_core_eth_create_qp(xdev, in, inlen, &psq->sqn);
+ if (ret)
+ goto err_sq_in_destroy;
+
+ psq->cqp.qpn = psq->sqn;
+ psq->cqp.event = xsc_eth_qp_event;
+ psq->cqp.eth_queue_type = XSC_RES_SQ;
+
+ ret = xsc_core_create_resource_common(xdev, &psq->cqp);
+ if (ret) {
+ netdev_err(adapter->netdev, "%s:error qp:%d errno:%d\n",
+ __func__, psq->sqn, ret);
+ goto err_sq_destroy;
+ }
+
+ psq->channel = c;
+ psq->ch_ix = c->chl_idx;
+ psq->txq_ix = psq->ch_ix;
+
+ /*need to querify from hardware*/
+ psq->hw_mtu = XSC_ETH_HW_MTU_SEND;
+ psq->stop_room = 1;
+
+ ret = xsc_eth_alloc_qp_sq_db(psq, psq_param->wq.db_numa_node);
+ if (ret)
+ goto err_sq_common_destroy;
+
+ inlen = sizeof(struct xsc_modify_raw_qp_mbox_in);
+ modify_in = kvzalloc(inlen, GFP_KERNEL);
+ if (!modify_in) {
+ ret = -ENOMEM;
+ goto err_sq_common_destroy;
+ }
+
+ modify_in->req.qp_out_port = xdev->pf_id;
+ modify_in->pcie_no = xdev->pcie_no;
+ modify_in->req.qpn = cpu_to_be16((u16)(psq->sqn));
+ modify_in->req.func_id = cpu_to_be16(xdev->glb_func_id);
+ modify_in->req.dma_direct = XSC_DMA_DIR_TO_MAC;
+ modify_in->req.prio = 0; /* single traffic class */
+ ret = xsc_core_eth_modify_raw_qp(xdev, modify_in);
+ if (ret)
+ goto err_sq_modify_in_destroy;
+
+ kvfree(modify_in);
+ kvfree(in);
+
+ return 0;
+
+err_sq_modify_in_destroy:
+ kvfree(modify_in);
+
+err_sq_common_destroy:
+ xsc_free_qp_sq_db(psq);
+
+ xsc_core_destroy_resource_common(xdev, &psq->cqp);
+
+err_sq_destroy:
+ xsc_core_eth_destroy_qp(xdev, psq->cqp.qpn);
+
+err_sq_in_destroy:
+ kvfree(in);
+
+err_sq_wq_destroy:
+ xsc_eth_wq_destroy(&psq->wq_ctrl);
+ return ret;
+}
+
+static void xsc_eth_close_qp_sq(struct xsc_channel *c, struct xsc_sq *psq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+
+ xsc_core_destroy_resource_common(xdev, &psq->cqp);
+
+ if (xsc_core_eth_destroy_qp(xdev, psq->cqp.qpn))
+ netdev_err(c->adapter->netdev, "failed to close ch%d sq%d\n",
+ c->chl_idx, psq->cqp.qpn);
+
+ xsc_free_tx_wqe(c->adapter->dev, psq);
+ xsc_free_qp_sq(psq);
+}
+
+static int xsc_eth_open_channel(struct xsc_adapter *adapter,
+ int idx,
+ struct xsc_channel *c,
+ struct xsc_channel_param *chl_param)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ const struct cpumask *aff;
+ unsigned int irqn;
+ u32 eqn;
+ int ret;
+
+ c->adapter = adapter;
+ c->netdev = adapter->netdev;
+ c->chl_idx = idx;
+
+ ret = xsc_core_vector2eqn(xdev, c->chl_idx, &eqn, &irqn);
+ if (ret)
+ goto err_out;
+ aff = irq_get_affinity_mask(irqn);
+ c->aff_mask = aff;
+ c->cpu = cpumask_first(aff);
+
+ ret = xsc_eth_open_cq(c, &c->qp.rq.cq, &chl_param->rqcq_param);
+ if (ret)
+ goto err_out;
+
+ ret = xsc_eth_open_cq(c, &c->qp.sq.cq, &chl_param->sqcq_param);
+ if (ret)
+ goto err_close_rq_cq;
+
+ ret = xsc_eth_open_qp_sq(c, &c->qp.sq, &chl_param->sq_param);
+ if (ret)
+ goto err_close_sq_cq;
+
+ netif_napi_add(netdev, &c->napi, xsc_eth_napi_poll);
+
+ netdev_dbg(adapter->netdev, "open channel%d ok\n", idx);
+ return 0;
+
+err_close_sq_cq:
+ xsc_eth_close_cq(c, &c->qp.sq.cq);
+err_close_rq_cq:
+ xsc_eth_close_cq(c, &c->qp.rq.cq);
+err_out:
+ netdev_err(adapter->netdev, "failed to open channel: ch%d, err=%d\n",
+ idx, ret);
+ return ret;
+}
+
+static int xsc_eth_modify_qps_channel(struct xsc_adapter *adapter,
+ struct xsc_channel *c)
+{
+ return xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.sq.sqn,
+ XSC_CMD_OP_RTR2RTS_QP);
+}
+
+static int xsc_eth_modify_qps(struct xsc_adapter *adapter,
+ struct xsc_eth_channels *chls)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < chls->num_chl; i++) {
+ struct xsc_channel *c = &chls->c[i];
+
+ ret = xsc_eth_modify_qps_channel(adapter, c);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static void xsc_eth_close_channel(struct xsc_channel *c, bool free_rq)
+{
+ xsc_eth_close_qp_sq(c, &c->qp.sq);
+ xsc_eth_close_cq(c, &c->qp.sq.cq);
+
+ netif_napi_del(&c->napi);
+}
+
+static int xsc_eth_open_channels(struct xsc_adapter *adapter)
+{
+ struct xsc_eth_channels *chls = &adapter->channels;
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct xsc_channel_param *chl_param;
+ int ret = 0;
+ int i;
+
+ chls->num_chl = adapter->nic_param.num_channels;
+ chls->c = kcalloc_node(chls->num_chl, sizeof(struct xsc_channel),
+ GFP_KERNEL, xdev->numa_node);
+ if (!chls->c) {
+ ret = -ENOMEM;
+ goto err_out;
+ }
+
+ chl_param = kvzalloc_obj(*chl_param, GFP_KERNEL);
+ if (!chl_param) {
+ ret = -ENOMEM;
+ goto err_free_ch;
+ }
+
+ xsc_eth_build_channel_param(adapter, chl_param);
+
+ for (i = 0; i < chls->num_chl; i++) {
+ ret = xsc_eth_open_channel(adapter, i, &chls->c[i], chl_param);
+ if (ret)
+ goto err_close_channel;
+ }
+
+ for (i = 0; i < chls->num_chl; i++)
+ napi_enable(&chls->c[i].napi);
+
+ /* flush cache to memory before interrupt and napi_poll running */
+ smp_wmb();
+
+ ret = xsc_eth_modify_qps(adapter, chls);
+ if (ret) {
+ for (i = 0; i < chls->num_chl; i++)
+ napi_disable(&chls->c[i].napi);
+ goto err_close_channel;
+ }
+
+ kvfree(chl_param);
+ return 0;
+
+err_close_channel:
+ for (--i; i >= 0; i--)
+ xsc_eth_close_channel(&chls->c[i], false);
+
+ kvfree(chl_param);
+err_free_ch:
+ kfree(chls->c);
+err_out:
+ chls->num_chl = 0;
+ netdev_err(adapter->netdev, "failed to open %d channels, err=%d\n",
+ chls->num_chl, ret);
+ return ret;
+}
+
+static void xsc_eth_close_channels(struct xsc_adapter *adapter)
+{
+ struct xsc_channel *c;
+ int i;
+
+ for (i = 0; i < adapter->channels.num_chl; i++) {
+ c = &adapter->channels.c[i];
+
+ xsc_eth_close_channel(c, true);
+ }
+
+ kfree(adapter->channels.c);
+ adapter->channels.num_chl = 0;
+}
+
+static void xsc_eth_build_tx2sq_maps(struct xsc_adapter *adapter)
+{
+ struct xsc_channel *c;
+ struct xsc_sq *psq;
+ int i;
+
+ for (i = 0; i < adapter->channels.num_chl; i++) {
+ c = &adapter->channels.c[i];
+ psq = &c->qp.sq;
+ adapter->txq2sq[psq->txq_ix] = psq;
+ }
+}
+
+static void xsc_eth_activate_txqsq(struct xsc_channel *c)
+{
+ struct xsc_sq *psq = &c->qp.sq;
+
+ psq->txq = netdev_get_tx_queue(psq->channel->netdev, psq->txq_ix);
+ set_bit(XSC_ETH_SQ_STATE_ENABLED, &psq->state);
+ netdev_tx_reset_queue(psq->txq);
+ netif_tx_start_queue(psq->txq);
+}
+
+static void xsc_eth_deactivate_txqsq(struct xsc_channel *c)
+{
+ clear_bit(XSC_ETH_SQ_STATE_ENABLED, &c->qp.sq.state);
+}
+
+static void xsc_eth_activate_channel(struct xsc_channel *c)
+{
+ xsc_eth_activate_txqsq(c);
+}
+
+static void xsc_eth_deactivate_channel(struct xsc_channel *c)
+{
+ xsc_eth_deactivate_txqsq(c);
+}
+
+static void xsc_eth_activate_channels(struct xsc_eth_channels *chs)
+{
+ int i;
+
+ for (i = 0; i < chs->num_chl; i++)
+ xsc_eth_activate_channel(&chs->c[i]);
+}
+
+static void xsc_eth_deactivate_channels(struct xsc_eth_channels *chs)
+{
+ int i;
+
+ for (i = 0; i < chs->num_chl; i++)
+ xsc_eth_deactivate_channel(&chs->c[i]);
+
+ /* Sync with all NAPIs to wait until they stop using queues. */
+ synchronize_net();
+
+ for (i = 0; i < chs->num_chl; i++)
+ /* last doorbell out */
+ napi_disable(&chs->c[i].napi);
+}
+
+static void xsc_eth_activate_priv_channels(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ netif_set_real_num_tx_queues(netdev, adapter->channels.num_chl);
+ netif_set_real_num_rx_queues(netdev, adapter->channels.num_chl);
+
+ xsc_eth_build_tx2sq_maps(adapter);
+ xsc_eth_activate_channels(&adapter->channels);
+ netif_tx_start_all_queues(adapter->netdev);
+}
+
+static void xsc_eth_deactivate_priv_channels(struct xsc_adapter *adapter)
+{
+ netif_tx_disable(adapter->netdev);
+ xsc_eth_deactivate_channels(&adapter->channels);
+}
+
+static int xsc_eth_sw_init(struct xsc_adapter *adapter)
+{
+ int ret;
+
+ ret = xsc_eth_open_channels(adapter);
+ if (ret)
+ return ret;
+
+ xsc_eth_activate_priv_channels(adapter);
+
+ return 0;
+}
+
+static void xsc_eth_sw_deinit(struct xsc_adapter *adapter)
+{
+ xsc_eth_deactivate_priv_channels(adapter);
+
+ return xsc_eth_close_channels(adapter);
+}
+
+static int xsc_eth_get_link_status(struct xsc_adapter *adapter, bool *link_up)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ u8 state;
+ int err;
+
+ err = xsc_core_query_vport_state(xdev, 0, &state);
+ if (err)
+ return err;
+
+ *link_up = state;
+ return 0;
}
static int xsc_eth_change_link_status(struct xsc_adapter *adapter)
{
bool link_up;
+ int err;
- link_up = xsc_eth_get_link_status(adapter);
+ err = xsc_eth_get_link_status(adapter, &link_up);
+ if (err) {
+ netdev_err(adapter->netdev, "failed to query link status, err=%d\n",
+ err);
+ return err;
+ }
if (link_up && !netif_carrier_ok(adapter->netdev)) {
netdev_info(adapter->netdev, "Link up\n");
@@ -142,6 +945,84 @@ static void xsc_eth_link_event_handler(void *arg)
err);
}
+static bool xsc_get_pct_drop_config(struct xsc_core_device *xdev)
+{
+ return (xdev->pdev->device == XSC_MC_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MF_SOC_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MS_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MV_SOC_PF_DEV_ID);
+}
+
+static int xsc_eth_enable_nic_hca(struct xsc_adapter *adapter)
+{
+ struct xsc_cmd_enable_nic_hca_mbox_out out = {};
+ struct xsc_cmd_enable_nic_hca_mbox_in in = {};
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ u16 caps_mask = 0;
+ u16 caps = 0;
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_NIC_HCA);
+
+ in.rss.rss_en = 1;
+ in.rss.rqn_base = cpu_to_be16(adapter->channels.rqn_base -
+ FIELD_GET(XSC_CAP_RAWETH_RSS_QP_ID_BASE_MASK,
+ xdev->caps.nic_caps));
+ in.rss.rqn_num = cpu_to_be16(adapter->channels.num_chl);
+ /* hfunc is a constant: the driver never negotiates anything else */
+ in.rss.hfunc = XSC_HASH_FUNC_TOP;
+ in.rss.hash_tmpl = cpu_to_be32(adapter->rss_param.rss_hash_tmpl);
+ caps_mask |= BIT(XSC_TBM_CAP_RSS);
+
+ if (netdev->features & NETIF_F_RXCSUM)
+ caps |= BIT(XSC_TBM_CAP_HASH_PPH);
+ caps_mask |= BIT(XSC_TBM_CAP_HASH_PPH);
+
+ if (xsc_get_pct_drop_config(xdev))
+ caps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+ caps_mask |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+
+ memcpy(in.nic.mac_addr, netdev->dev_addr, ETH_ALEN);
+
+ in.nic.caps = cpu_to_be16(caps);
+ in.nic.caps_mask = cpu_to_be16(caps_mask);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ netdev_err(netdev, "failed!! err=%d, status=%d\n",
+ err, out.hdr.status);
+ return -ENOEXEC;
+ }
+
+ return 0;
+}
+
+static int xsc_eth_disable_nic_hca(struct xsc_adapter *adapter)
+{
+ struct xsc_cmd_disable_nic_hca_mbox_out out = {};
+ struct xsc_cmd_disable_nic_hca_mbox_in in = {};
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ u16 caps = 0;
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DISABLE_NIC_HCA);
+
+ if (xsc_get_pct_drop_config(xdev))
+ caps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+
+ in.nic.caps = cpu_to_be16(caps);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ netdev_err(netdev, "failed!! err=%d, status=%d\n",
+ err, out.hdr.status);
+ return -ENOEXEC;
+ }
+
+ return 0;
+}
+
static void xsc_set_default_xps_cpumasks(struct xsc_adapter *adapter)
{
struct xsc_core_device *xdev = adapter->xdev;
@@ -172,14 +1053,26 @@ static int xsc_set_port_admin_status(struct xsc_adapter *adapter,
static int xsc_eth_open(struct net_device *netdev)
{
struct xsc_adapter *adapter = netdev_priv(netdev);
+ bool link_up;
int ret;
if (adapter->opened) {
netdev_err(adapter->netdev, "ndo_open on an already opened device\n");
return 0;
}
+ ret = xsc_eth_sw_init(adapter);
+ if (ret)
+ return ret;
- if (xsc_eth_get_link_status(adapter)) {
+ ret = xsc_eth_enable_nic_hca(adapter);
+ if (ret)
+ goto err_sw_deinit;
+
+ ret = xsc_eth_get_link_status(adapter, &link_up);
+ if (ret)
+ netdev_warn(netdev, "failed to query link status, err=%d\n", ret);
+
+ if (!ret && link_up) {
netdev_info(netdev, "Link up\n");
netif_carrier_on(adapter->netdev);
} else {
@@ -196,6 +1089,10 @@ static int xsc_eth_open(struct net_device *netdev)
ret);
return 0;
+
+err_sw_deinit:
+ xsc_eth_sw_deinit(adapter);
+ return ret;
}
static int xsc_eth_close(struct net_device *netdev)
@@ -209,6 +1106,13 @@ static int xsc_eth_close(struct net_device *netdev)
netif_carrier_off(adapter->netdev);
+ xsc_eth_sw_deinit(adapter);
+
+ ret = xsc_eth_disable_nic_hca(adapter);
+ if (ret)
+ netdev_err(adapter->netdev, "failed to disable nic hca, err=%d\n",
+ ret);
+
ret = xsc_set_port_admin_status(adapter, XSC_PORT_DOWN);
if (ret)
netdev_err(adapter->netdev, "failed to set port admin status down, err=%d\n",
@@ -244,18 +1148,10 @@ static int xsc_eth_set_hw_mtu(struct xsc_core_device *xdev,
return 0;
}
-/* Real xmit arrives with the TX path patch; keep TX live until then. */
-static netdev_tx_t xsc_eth_xmit_drop(struct sk_buff *skb,
- struct net_device *netdev)
-{
- dev_kfree_skb_any(skb);
- return NETDEV_TX_OK;
-}
-
static const struct net_device_ops xsc_netdev_ops = {
.ndo_open = xsc_eth_open,
.ndo_stop = xsc_eth_close,
- .ndo_start_xmit = xsc_eth_xmit_drop,
+ .ndo_start_xmit = xsc_eth_xmit_start,
};
static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
@@ -270,7 +1166,13 @@ static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
/*mtu - macheaderlen - ipheaderlen should be aligned in 8B*/
netdev->mtu = SW_DEFAULT_MTU;
- netdev->features = NETIF_F_HIGHDMA;
+ netdev->vlan_features |= NETIF_F_SG |
+ NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
+ NETIF_F_TSO | NETIF_F_TSO6;
+
+ netdev->hw_features = netdev->vlan_features;
+
+ netdev->features = netdev->hw_features | NETIF_F_HIGHDMA;
}
static int xsc_eth_nic_init(struct xsc_adapter *adapter, u32 ch_num)
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
new file mode 100644
index 000000000..376b2cd0d
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
@@ -0,0 +1,345 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/tcp.h>
+
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+
+#define XSC_OPCODE_RAW 7
+
+static void xsc_dma_push(struct xsc_sq *sq, dma_addr_t addr, u32 size,
+ enum xsc_dma_map_type map_type)
+{
+ struct xsc_sq_dma *dma = xsc_dma_get(sq, sq->dma_fifo_pc++);
+
+ dma->addr = addr;
+ dma->size = size;
+ dma->type = map_type;
+}
+
+static void xsc_dma_unmap_wqe(struct xsc_sq *sq, u8 num_dma)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct device *dev = adapter->dev;
+ int i;
+
+ for (i = 0; i < num_dma; i++) {
+ struct xsc_sq_dma *last_pushed_dma;
+
+ last_pushed_dma = xsc_dma_get(sq, --sq->dma_fifo_pc);
+ xsc_tx_dma_unmap(dev, last_pushed_dma);
+ }
+}
+
+static void *xsc_sq_fetch_wqe(struct xsc_sq *sq, size_t size, u16 *pi)
+{
+ struct xsc_wq_cyc *wq = &sq->wq;
+ void *wqe;
+
+ /*caution, sp->pc is default to be zero*/
+ *pi = xsc_wq_cyc_ctr2ix(wq, sq->pc);
+ wqe = xsc_wq_cyc_get_wqe(wq, *pi);
+ memset(wqe, 0, size);
+
+ return wqe;
+}
+
+static u16 xsc_tx_get_gso_ihs(struct xsc_sq *sq, struct sk_buff *skb)
+{
+ u16 ihs;
+
+ if (skb->encapsulation) {
+ ihs = skb_inner_transport_offset(skb) + inner_tcp_hdrlen(skb);
+ } else {
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4)
+ ihs = skb_transport_offset(skb) + sizeof(struct udphdr);
+ else
+ ihs = skb_transport_offset(skb) + tcp_hdrlen(skb);
+ }
+
+ return ihs;
+}
+
+static void xsc_txwqe_build_cseg_csum(struct xsc_sq *sq,
+ struct sk_buff *skb,
+ struct xsc_send_wqe_ctrl_seg *cseg)
+{
+ u32 val = le32_to_cpu(cseg->data0);
+
+ if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) {
+ if (skb->encapsulation)
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_CSUM_EN_MASK,
+ XSC_ETH_WQE_INNER_AND_OUTER_CSUM);
+ else
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_CSUM_EN_MASK,
+ XSC_ETH_WQE_OUTER_CSUM);
+ } else {
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_CSUM_EN_MASK,
+ XSC_ETH_WQE_NONE_CSUM);
+ }
+ cseg->data0 = cpu_to_le32(val);
+}
+
+static void xsc_txwqe_build_csegs(struct xsc_sq *sq, struct sk_buff *skb,
+ u16 mss, u16 ihs,
+ u8 opcode, u16 ds_cnt, u32 msglen,
+ struct xsc_send_wqe_ctrl_seg *cseg)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ int send_wqe_ds_num_log;
+ u32 val = 0;
+
+ send_wqe_ds_num_log = ilog2(xdev->caps.send_ds_num);
+ xsc_txwqe_build_cseg_csum(sq, skb, cseg);
+
+ if (mss != 0) {
+ val |= XSC_WQE_CTRL_SEG_DATA2_HAS_PPH |
+ XSC_WQE_CTRL_SEG_DATA2_SO_TYPE |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DATA2_SO_HDR_LEN_MASK, ihs) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DATA2_SO_DATA_SIZE_MASK, mss);
+ cseg->data2 = cpu_to_le32(val);
+ }
+
+ val = le32_to_cpu(cseg->data0);
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK, opcode) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK,
+ sq->pc << send_wqe_ds_num_log) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_DS_DATA_NUM_MASK,
+ ds_cnt - XSC_SEND_WQEBB_CTRL_NUM_DS);
+ cseg->data0 = cpu_to_le32(val);
+ cseg->msg_len = cpu_to_le32(msglen);
+ cseg->data3 = cpu_to_le32(XSC_WQE_CTRL_SEG_DATA3_CE);
+}
+
+static int xsc_txwqe_build_dsegs(struct xsc_sq *sq, struct sk_buff *skb,
+ u16 headlen,
+ struct xsc_wqe_data_seg *dseg)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct device *dev = adapter->dev;
+ dma_addr_t dma_addr = 0;
+ u8 num_dma = 0;
+ int i;
+
+ if (headlen) {
+ dma_addr = dma_map_single(dev,
+ skb->data,
+ headlen,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma_addr)))
+ goto err_dma_unmap_wqe;
+
+ dseg->va = cpu_to_le64(dma_addr);
+ dseg->mkey = cpu_to_le32(be32_to_cpu(sq->mkey_be));
+ dseg->data0 |=
+ cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK,
+ headlen));
+
+ xsc_dma_push(sq, dma_addr, headlen, XSC_DMA_MAP_SINGLE);
+ num_dma++;
+ dseg++;
+ }
+
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
+ int fsz = skb_frag_size(frag);
+
+ dma_addr = skb_frag_dma_map(dev, frag, 0, fsz, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma_addr)))
+ goto err_dma_unmap_wqe;
+
+ dseg->va = cpu_to_le64(dma_addr);
+ dseg->mkey = cpu_to_le32(be32_to_cpu(sq->mkey_be));
+ dseg->data0 |=
+ cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK,
+ fsz));
+
+ xsc_dma_push(sq, dma_addr, fsz, XSC_DMA_MAP_PAGE);
+ num_dma++;
+ dseg++;
+ }
+
+ return num_dma;
+
+err_dma_unmap_wqe:
+ xsc_dma_unmap_wqe(sq, num_dma);
+ return -ENOMEM;
+}
+
+static void xsc_sq_notify_hw(struct xsc_wq_cyc *wq, u16 pc,
+ struct xsc_sq *sq)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct xsc_core_device *xdev = adapter->xdev;
+ int send_ds_num_log = ilog2(xdev->caps.send_ds_num);
+
+ /* reverse wqe index to ds index */
+ xsc_update_tx_db(xdev, sq->sqn, pc << send_ds_num_log);
+}
+
+static void xsc_txwqe_complete(struct xsc_sq *sq, struct sk_buff *skb,
+ u8 opcode, u16 ds_cnt,
+ u8 num_wqebbs, u32 num_bytes, u8 num_dma,
+ struct xsc_tx_wqe_info *wi)
+{
+ struct xsc_wq_cyc *wq = &sq->wq;
+
+ wi->num_bytes = num_bytes;
+ wi->num_dma = num_dma;
+ wi->num_wqebbs = num_wqebbs;
+ wi->skb = skb;
+
+ netdev_tx_sent_queue(sq->txq, num_bytes);
+
+ sq->pc += wi->num_wqebbs;
+
+ if (unlikely(!xsc_wqc_has_room_for(wq, sq->cc, sq->pc, sq->stop_room)))
+ netif_tx_stop_queue(sq->txq);
+
+ if (!netdev_xmit_more() || netif_xmit_stopped(sq->txq))
+ xsc_sq_notify_hw(wq, sq->pc, sq);
+}
+
+static uint32_t xsc_eth_xmit_frame(struct sk_buff *skb,
+ struct xsc_sq *sq,
+ struct xsc_tx_wqe *wqe,
+ u16 pi)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ struct xsc_sq_stats *stats = sq->stats;
+ struct xsc_send_wqe_ctrl_seg *cseg;
+ struct xsc_wqe_data_seg *dseg;
+ struct xsc_tx_wqe_info *wi;
+ u16 mss, ihs, headlen;
+ u32 num_bytes, num_pkts;
+ u8 num_wqebbs;
+ int num_dma;
+ u16 ds_cnt;
+ u8 opcode;
+
+retry_send:
+ /* Calc ihs and ds cnt, no writes to wqe yet; ctrl-ds is reduced by
+ * ds_data_num.
+ */
+ ds_cnt = XSC_SEND_WQEBB_CTRL_NUM_DS;
+
+ if (skb_is_gso(skb)) {
+ opcode = XSC_OPCODE_RAW;
+ mss = skb_shinfo(skb)->gso_size;
+ ihs = xsc_tx_get_gso_ihs(sq, skb);
+ num_bytes = skb->len +
+ (skb_shinfo(skb)->gso_segs - 1) * ihs;
+ num_pkts = skb_shinfo(skb)->gso_segs;
+ } else {
+ opcode = XSC_OPCODE_RAW;
+ mss = 0;
+ ihs = 0;
+ num_bytes = skb->len;
+ num_pkts = 1;
+ }
+
+ /*linear data in skb*/
+ headlen = skb->len - skb->data_len;
+ ds_cnt += !!headlen;
+ ds_cnt += skb_shinfo(skb)->nr_frags;
+
+ /* Check packet size. */
+ if (unlikely(mss == 0 && skb->len > sq->hw_mtu))
+ goto err_drop;
+
+ num_wqebbs = DIV_ROUND_UP(ds_cnt, xdev->caps.send_ds_num);
+ /*if ds_cnt exceed one wqe, drop it*/
+ if (num_wqebbs != 1) {
+ if (skb_linearize(skb))
+ goto err_drop;
+ goto retry_send;
+ }
+
+ /* fill wqe */
+ wi = (struct xsc_tx_wqe_info *)&sq->db.wqe_info[pi];
+ cseg = &wqe->ctrl;
+ dseg = &wqe->data[0];
+
+ if (unlikely(skb->len == 0))
+ goto err_drop;
+
+ xsc_txwqe_build_csegs(sq, skb, mss, ihs,
+ opcode, ds_cnt, skb->len, cseg);
+
+ /*inline header is also use dma to transport*/
+ num_dma = xsc_txwqe_build_dsegs(sq, skb, headlen, dseg);
+ if (unlikely(num_dma < 0))
+ goto err_drop;
+
+ xsc_txwqe_complete(sq, skb, opcode, ds_cnt, num_wqebbs, num_bytes,
+ num_dma, wi);
+ stats->bytes += num_bytes;
+ stats->packets += num_pkts;
+
+ return NETDEV_TX_OK;
+
+err_drop:
+ stats->dropped++;
+ dev_kfree_skb_any(skb);
+
+ return NETDEV_TX_OK;
+}
+
+void xsc_free_tx_wqe(struct device *dev, struct xsc_sq *sq)
+{
+ struct xsc_tx_wqe_info *wi;
+ struct sk_buff *skb;
+ u16 ci, npkts = 0;
+ u32 nbytes = 0;
+ int i;
+
+ while (sq->cc != sq->pc) {
+ ci = xsc_wq_cyc_ctr2ix(&sq->wq, sq->cc);
+ wi = &sq->db.wqe_info[ci];
+ skb = wi->skb;
+
+ if (!skb) { /* nop */
+ sq->cc++;
+ continue;
+ }
+
+ for (i = 0; i < wi->num_dma; i++) {
+ struct xsc_sq_dma *dma =
+ xsc_dma_get(sq, sq->dma_fifo_cc++);
+
+ xsc_tx_dma_unmap(dev, dma);
+ }
+
+ dev_kfree_skb_any(skb);
+ npkts++;
+ nbytes += wi->num_bytes;
+ sq->cc += wi->num_wqebbs;
+ }
+
+ if (sq->txq)
+ netdev_tx_completed_queue(sq->txq, npkts, nbytes);
+}
+
+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ struct xsc_tx_wqe *wqe;
+ struct xsc_sq *sq;
+ u32 ds_num;
+ u16 pi;
+
+ if (!adapter->opened)
+ return NETDEV_TX_BUSY;
+
+ sq = adapter->txq2sq[skb_get_queue_mapping(skb)];
+ if (unlikely(!sq))
+ return NETDEV_TX_BUSY;
+
+ ds_num = adapter->xdev->caps.send_ds_num;
+ wqe = xsc_sq_fetch_wqe(sq, ds_num * XSC_SEND_WQE_DS, &pi);
+
+ return xsc_eth_xmit_frame(skb, sq, wqe, pi);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
new file mode 100644
index 000000000..262421740
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
@@ -0,0 +1,143 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "xsc_eth_common.h"
+#include "xsc_eth_txrx.h"
+
+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq)
+{
+ xsc_arm_cq(cq->xdev, cq->xcq.cqn, cq->wq.cc, 0);
+}
+
+void xsc_cq_notify_hw(struct xsc_cq *cq)
+{
+ xsc_update_cq_ci(cq->xdev, cq->xcq.cqn, cq->wq.cc);
+}
+
+static bool xsc_channel_no_affinity_change(struct xsc_channel *c)
+{
+ int current_cpu = smp_processor_id();
+
+ return cpumask_test_cpu(current_cpu, c->aff_mask);
+}
+
+static void xsc_dump_error_sqcqe(struct xsc_sq *sq,
+ struct xsc_cqe *cqe)
+{
+ struct net_device *netdev = sq->channel->netdev;
+ u32 ci = xsc_cqwq_get_ci(&sq->cq.wq);
+
+ net_err_ratelimited("Err cqe on dev %s cqn=0x%x ci=0x%x sqn=0x%x err_code=0x%x qpid=0x%lx\n",
+ netdev->name, sq->cq.xcq.cqn, ci,
+ sq->sqn, get_cqe_opcode(cqe),
+ FIELD_GET(XSC_CQE_DATA0_QP_ID_MASK,
+ le32_to_cpu(cqe->data0)));
+}
+
+static bool xsc_poll_tx_cq(struct xsc_cq *cq, int napi_budget)
+{
+ struct xsc_adapter *adapter;
+ struct xsc_cqe *cqe;
+ struct device *dev;
+ struct xsc_sq *sq;
+ u32 dma_fifo_cc;
+ u32 nbytes = 0;
+ u16 npkts = 0;
+ int i = 0;
+ u16 sqcc;
+
+ sq = container_of(cq, struct xsc_sq, cq);
+ if (!test_bit(XSC_ETH_SQ_STATE_ENABLED, &sq->state))
+ return false;
+
+ adapter = sq->channel->adapter;
+ dev = adapter->dev;
+
+ cqe = xsc_cqwq_get_cqe(&cq->wq);
+ if (!cqe)
+ goto out;
+
+ if (unlikely(get_cqe_opcode(cqe) & BIT(7)))
+ xsc_dump_error_sqcqe(sq, cqe);
+
+ sqcc = sq->cc;
+
+ /* avoid dirtying sq cache line every cqe */
+ dma_fifo_cc = sq->dma_fifo_cc;
+ i = 0;
+ do {
+ struct xsc_tx_wqe_info *wi;
+ struct sk_buff *skb;
+ int j;
+ u16 ci;
+
+ xsc_cqwq_pop(&cq->wq);
+
+ ci = xsc_wq_cyc_ctr2ix(&sq->wq, sqcc);
+ wi = &sq->db.wqe_info[ci];
+ skb = wi->skb;
+
+ if (unlikely(!skb))
+ continue;
+
+ for (j = 0; j < wi->num_dma; j++) {
+ struct xsc_sq_dma *dma = xsc_dma_get(sq, dma_fifo_cc++);
+
+ xsc_tx_dma_unmap(dev, dma);
+ }
+
+ npkts++;
+ nbytes += wi->num_bytes;
+ sqcc += wi->num_wqebbs;
+ napi_consume_skb(skb, 0);
+
+ } while ((++i < napi_budget) && (cqe = xsc_cqwq_get_cqe(&cq->wq)));
+
+ xsc_cq_notify_hw(cq);
+
+ /* ensure cq space is freed before enabling more cqes */
+ wmb();
+
+ sq->dma_fifo_cc = dma_fifo_cc;
+ sq->cc = sqcc;
+
+ netdev_tx_completed_queue(sq->txq, npkts, nbytes);
+
+ if (netif_tx_queue_stopped(sq->txq) &&
+ xsc_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, sq->stop_room)) {
+ netif_tx_wake_queue(sq->txq);
+ }
+
+out:
+ return (i == napi_budget);
+}
+
+int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct xsc_channel *c = container_of(napi, struct xsc_channel, napi);
+ struct xsc_eth_params *params = &c->adapter->nic_param;
+ bool busy = false;
+ int tx_budget;
+
+ rcu_read_lock();
+
+ tx_budget = params->sq_size >> 2;
+ busy |= xsc_poll_tx_cq(&c->qp.sq.cq, tx_budget);
+
+ if (busy) {
+ if (likely(xsc_channel_no_affinity_change(c))) {
+ rcu_read_unlock();
+ return budget;
+ }
+ }
+
+ if (unlikely(!napi_complete_done(napi, 0)))
+ goto err_out;
+
+ xsc_cq_notify_hw_rearm(&c->qp.sq.cq);
+err_out:
+ rcu_read_unlock();
+ return 0;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
index dece589ab..bb477fd4a 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
@@ -33,6 +33,7 @@ struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,
u32 cqe_bcnt, u8 has_pph);
int xsc_poll_rx_cq(struct xsc_cq *cq, int budget);
+void xsc_free_tx_wqe(struct device *dev, struct xsc_sq *sq);
netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev);
static inline void xsc_tx_dma_unmap(struct device *dev, struct xsc_sq_dma *dma)
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
index 5118ff36b..9e40ec001 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
@@ -6,7 +6,8 @@
#include "common/xsc_core.h"
#include "common/xsc_cmd_api.h"
-u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport)
+int xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport,
+ u8 *state)
{
struct xsc_query_vport_state_out out;
struct xsc_query_vport_state_in in;
@@ -29,6 +30,7 @@ u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport)
return err;
}
- return out.state;
+ *state = out.state;
+ return 0;
}
EXPORT_SYMBOL_GPL(xsc_core_query_vport_state);
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 08/15] net/xsc: add ethernet TX path
2026-11-04 15:57 ` [PATCH v1 08/15] net/xsc: add ethernet TX path Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [High] net/xsc: NETDEV_TX_BUSY on active queues in xsc_eth_xmit_start()
- [High] net/xsc: race waking stopped TX queues in xsc_eth_deactivate_priv_channels()
- [High] net/xsc: CQs armed before NAPI and queue state init in xsc_eth_open_channels()
- [Medium] net/xsc: missed error CQE checks in xsc_poll_tx_cq() poll loop
- [Medium] net/xsc: missing TX doorbell on err_drop path in xsc_eth_xmit_frame()
- [Medium] net/xsc: fix CQE error check, sqcc update, and budget in xsc_poll_tx_cq()
- [Low] net/xsc: chls->num_chl zeroed before error log in xsc_eth_open_channels()
--
commit 251dc7b87bb643704bb618df57c60cfb11aec1a1
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add ethernet TX path
This commit implements the Ethernet transmit path for the xsc driver,
including per-channel CQ and raw-TSO SQ setup, NAPI completion polling,
WQE construction with DMA mapping and BQL tracking, and offload feature
advertisement.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> index 21a50085fa879..a02f95145d018 100644
> --- a/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> @@ -95,22 +95,825 @@ static int xsc_eth_netdev_init(struct xsc_adapter *adapter)
[ ... ]
> +static void xsc_eth_completion_event(struct xsc_core_cq *xcq)
> +{
> + struct xsc_cq *cq = container_of(xcq, struct xsc_cq, xcq);
> + struct xsc_core_device *xdev = cq->xdev;
> + struct xsc_rq *rq = NULL;
> +
> + if (unlikely(!cq->channel)) {
> + netdev_err(xsc_dev_to_netdev(xdev), "cq%d->channel is null\n",
> + xcq->cqn);
> + return;
> + }
> +
> + rq = &cq->channel->qp.rq;
> +
> + if (!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state))
> + netdev_err(xsc_dev_to_netdev(xdev),
> + "ch%d_cq%d, napi_flag=0x%lx\n",
> + cq->channel->chl_idx, xcq->cqn,
> + cq->napi->state);
> +
> + napi_schedule(cq->napi);
> + cq->event_ctr++;
> +}
[ ... ]
> +static int xsc_eth_open_cq(struct xsc_channel *c,
> + struct xsc_cq *pcq,
> + struct xsc_cq_param *pcq_param)
> +{
> + int ret;
> +
> + ret = xsc_eth_alloc_cq(c, pcq, pcq_param);
> + if (ret)
> + return ret;
> +
> + ret = xsc_eth_set_cq(c, pcq, pcq_param);
> + if (ret)
> + goto err_free_cq;
> +
> + xsc_cq_notify_hw_rearm(pcq);
> +
> + pcq->napi = &c->napi;
> + pcq->channel = c;
> +
> + return 0;
> +
> +err_free_cq:
> + xsc_eth_free_cq(pcq);
> + return ret;
> +}
[ ... ]
> +static int xsc_eth_open_channel(struct xsc_adapter *adapter,
> + int idx,
> + struct xsc_channel *c,
> + struct xsc_channel_param *chl_param)
> +{
> + struct xsc_core_device *xdev = adapter->xdev;
> + struct net_device *netdev = adapter->netdev;
> + const struct cpumask *aff;
> + unsigned int irqn;
> + u32 eqn;
> + int ret;
> +
> + c->adapter = adapter;
> + c->netdev = adapter->netdev;
> + c->chl_idx = idx;
> +
> + ret = xsc_core_vector2eqn(xdev, c->chl_idx, &eqn, &irqn);
> + if (ret)
> + goto err_out;
> + aff = irq_get_affinity_mask(irqn);
> + c->aff_mask = aff;
> + c->cpu = cpumask_first(aff);
> +
> + ret = xsc_eth_open_cq(c, &c->qp.rq.cq, &chl_param->rqcq_param);
> + if (ret)
> + goto err_out;
> +
> + ret = xsc_eth_open_cq(c, &c->qp.sq.cq, &chl_param->sqcq_param);
> + if (ret)
> + goto err_close_rq_cq;
> +
> + ret = xsc_eth_open_qp_sq(c, &c->qp.sq, &chl_param->sq_param);
> + if (ret)
> + goto err_close_sq_cq;
> +
> + netif_napi_add(netdev, &c->napi, xsc_eth_napi_poll);
[Severity: High]
Could this ordering between CQ creation, NAPI initialization, and queue
state enablement cause a NULL pointer dereference or unratelimited log
spam from hardirq context?
In xsc_eth_open_cq(), xsc_eth_set_cq() calls xsc_core_eth_create_cq(),
which inserts &pcq->xcq into xdev->dev_res->cq_table.tree while MSI-X EQs
are active, and then calls xsc_cq_notify_hw_rearm(pcq) before setting
pcq->napi and pcq->channel.
Furthermore, in xsc_eth_open_channel(), both CQs (rq.cq and sq.cq) are
opened and armed before netif_napi_add(netdev, &c->napi, xsc_eth_napi_poll)
initializes c->napi. Because c is zero-allocated with kcalloc_node(),
c->napi.state has NAPIF_STATE_SCHED cleared and c->napi.poll is NULL prior
to netif_napi_add(). If xsc_eq_int() dispatches XSC_EVENT_TYPE_COMP or
XSC_EVENT_TYPE_INTERNAL_ERROR for the CQ after pcq->channel is assigned
and before netif_napi_add() runs, xsc_eth_completion_event() will call
napi_schedule(cq->napi) on an uninitialized napi_struct.
In addition, xsc_eth_completion_event() checks
!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state) and logs an unratelimited
netdev_err() from hardirq context. In this commit XSC_ETH_RQ_STATE_ENABLED
is never set anywhere, so every TX completion interrupt triggers
netdev_err(). Even once RX sets XSC_ETH_RQ_STATE_ENABLED in
xsc_eth_activate_priv_channels(), xsc_eth_open_channels() transitions QPs
to RTS before xsc_eth_activate_priv_channels() sets the enabled state bits
(and xsc_eth_deactivate_channels() clears them before tearing down the QPs
and CQs), during which time xsc_poll_tx_cq() returns early when
XSC_ETH_SQ_STATE_ENABLED is clear while xsc_eth_napi_poll() re-arms the CQ
without draining it.
Should netif_napi_add() and pcq->napi / pcq->channel initialization happen
before registering and arming the CQs, and should the queue state bits be
set before arming CQs and transitioning QPs to RTS?
> +
> + netdev_dbg(adapter->netdev, "open channel%d ok\n", idx);
> + return 0;
[ ... ]
> +err_close_channel:
> + for (--i; i >= 0; i--)
> + xsc_eth_close_channel(&chls->c[i], false);
> +
> + kvfree(chl_param);
> +err_free_ch:
> + kfree(chls->c);
> +err_out:
> + chls->num_chl = 0;
> + netdev_err(adapter->netdev, "failed to open %d channels, err=%d\n",
> + chls->num_chl, ret);
> + return ret;
> +}
[Severity: Low]
In xsc_eth_open_channels(), chls->num_chl is set to 0 right before the
netdev_err() call at err_out. Won't this always log "failed to open 0
channels" on failure instead of the number of channels that failed to
open?
[ ... ]
> +static void xsc_eth_deactivate_channels(struct xsc_eth_channels *chs)
> +{
> + int i;
> +
> + for (i = 0; i < chs->num_chl; i++)
> + xsc_eth_deactivate_channel(&chs->c[i]);
> +
> + /* Sync with all NAPIs to wait until they stop using queues. */
> + synchronize_net();
> +
> + for (i = 0; i < chs->num_chl; i++)
> + /* last doorbell out */
> + napi_disable(&chs->c[i].napi);
> +}
> +
> +static void xsc_eth_activate_priv_channels(struct xsc_adapter *adapter)
> +{
> + struct net_device *netdev = adapter->netdev;
> +
> + netif_set_real_num_tx_queues(netdev, adapter->channels.num_chl);
> + netif_set_real_num_rx_queues(netdev, adapter->channels.num_chl);
> +
> + xsc_eth_build_tx2sq_maps(adapter);
> + xsc_eth_activate_channels(&adapter->channels);
> + netif_tx_start_all_queues(adapter->netdev);
> +}
> +
> +static void xsc_eth_deactivate_priv_channels(struct xsc_adapter *adapter)
> +{
> + netif_tx_disable(adapter->netdev);
> + xsc_eth_deactivate_channels(&adapter->channels);
> +}
[Severity: High]
Can a concurrent NAPI poll re-wake a TX queue during teardown in
xsc_eth_deactivate_priv_channels()?
xsc_eth_deactivate_priv_channels() calls netif_tx_disable(adapter->netdev)
before xsc_eth_deactivate_channels() clears XSC_ETH_SQ_STATE_ENABLED and
disables NAPI. If xsc_poll_tx_cq() is running concurrently on another CPU,
it can still see XSC_ETH_SQ_STATE_ENABLED set, observe
netif_tx_queue_stopped(sq->txq) is true (just set by netif_tx_disable()),
and call netif_tx_wake_queue(sq->txq):
drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c:xsc_poll_tx_cq() {
...
if (netif_tx_queue_stopped(sq->txq) &&
xsc_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, sq->stop_room)) {
netif_tx_wake_queue(sq->txq);
}
}
Wouldn't that clear __QUEUE_STATE_DRV_XOFF and leave the TX queue awake
while xsc_eth_sw_deinit() proceeds to free TX WQEs and close the channels?
Should xsc_eth_deactivate_channels() run before netif_tx_disable()?
[ ... ]
> @@ -172,14 +1053,26 @@ static int xsc_set_port_admin_status(struct xsc_adapter *adapter,
> static int xsc_eth_open(struct net_device *netdev)
> {
> struct xsc_adapter *adapter = netdev_priv(netdev);
> + bool link_up;
> int ret;
>
> if (adapter->opened) {
> netdev_err(adapter->netdev, "ndo_open on an already opened device\n");
> return 0;
> }
> + ret = xsc_eth_sw_init(adapter);
> + if (ret)
> + return ret;
>
> - if (xsc_eth_get_link_status(adapter)) {
> + ret = xsc_eth_enable_nic_hca(adapter);
> + if (ret)
> + goto err_sw_deinit;
> +
> + ret = xsc_eth_get_link_status(adapter, &link_up);
> + if (ret)
> + netdev_warn(netdev, "failed to query link status, err=%d\n", ret);
> +
> + if (!ret && link_up) {
> netdev_info(netdev, "Link up\n");
> netif_carrier_on(adapter->netdev);
> } else {
> @@ -196,6 +1089,10 @@ static int xsc_eth_open(struct net_device *netdev)
> ret);
>
> return 0;
> +
> +err_sw_deinit:
> + xsc_eth_sw_deinit(adapter);
> + return ret;
> }
>
> static int xsc_eth_close(struct net_device *netdev)
> @@ -209,6 +1106,13 @@ static int xsc_eth_close(struct net_device *netdev)
>
> netif_carrier_off(adapter->netdev);
>
> + xsc_eth_sw_deinit(adapter);
[Severity: High]
Can xsc_eth_xmit_start() return NETDEV_TX_BUSY on an active, unstopped TX
queue during xsc_eth_open() and xsc_eth_close()?
In xsc_eth_open(), xsc_eth_sw_init() calls xsc_eth_activate_priv_channels(),
which calls netif_set_real_num_tx_queues() and
netif_set_real_num_rx_queues() (without checking their return values) and
starts all TX queues via netif_tx_start_queue() and
netif_tx_start_all_queues() while adapter->opened is still false.
xsc_eth_open() then sleeps in xsc_eth_enable_nic_hca() and
xsc_eth_get_link_status() and calls netif_carrier_on() before setting
adapter->opened = true (and xsc_eth_link_event_handler() can also turn the
carrier on asynchronously).
Similarly, in xsc_eth_close(), adapter->opened = false is set before
netif_carrier_off() and before xsc_eth_sw_deinit() calls
netif_tx_disable().
During both windows, TX queues are active and unstopped while
adapter->opened is false. Any packet reaching xsc_eth_xmit_start() will
hit:
drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c:xsc_eth_xmit_start() {
if (!adapter->opened)
return NETDEV_TX_BUSY;
}
without stopping the TX queue, causing sch_direct_xmit() to requeue the skb
and reschedule NET_TX_SOFTIRQ in a tight retry loop (and leaving any
earlier skbs in a netdev_xmit_more() batch without a doorbell ring).
Should adapter->opened be set before starting the TX queues (or TX queues
started only after HCA enablement and adapter->opened = true), and in
xsc_eth_close() should the TX queues be disabled before clearing
adapter->opened?
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
> new file mode 100644
> index 0000000000000..376b2cd0d8f29
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
[ ... ]
> +static void xsc_txwqe_complete(struct xsc_sq *sq, struct sk_buff *skb,
> + u8 opcode, u16 ds_cnt,
> + u8 num_wqebbs, u32 num_bytes, u8 num_dma,
> + struct xsc_tx_wqe_info *wi)
> +{
> + struct xsc_wq_cyc *wq = &sq->wq;
> +
> + wi->num_bytes = num_bytes;
> + wi->num_dma = num_dma;
> + wi->num_wqebbs = num_wqebbs;
> + wi->skb = skb;
> +
> + netdev_tx_sent_queue(sq->txq, num_bytes);
> +
> + sq->pc += wi->num_wqebbs;
> +
> + if (unlikely(!xsc_wqc_has_room_for(wq, sq->cc, sq->pc, sq->stop_room)))
> + netif_tx_stop_queue(sq->txq);
> +
> + if (!netdev_xmit_more() || netif_xmit_stopped(sq->txq))
> + xsc_sq_notify_hw(wq, sq->pc, sq);
> +}
> +
> +static uint32_t xsc_eth_xmit_frame(struct sk_buff *skb,
> + struct xsc_sq *sq,
> + struct xsc_tx_wqe *wqe,
> + u16 pi)
> +{
[ ... ]
> + xsc_txwqe_complete(sq, skb, opcode, ds_cnt, num_wqebbs, num_bytes,
> + num_dma, wi);
> + stats->bytes += num_bytes;
> + stats->packets += num_pkts;
> +
> + return NETDEV_TX_OK;
> +
> +err_drop:
> + stats->dropped++;
> + dev_kfree_skb_any(skb);
> +
> + return NETDEV_TX_OK;
> +}
[Severity: Medium]
What happens to previously queued WQEs in a netdev_xmit_more() batch if a
subsequent packet in that batch takes the err_drop path in
xsc_eth_xmit_frame()?
When earlier packets in a batch are queued with netdev_xmit_more() true and
the queue is not stopped, xsc_txwqe_complete() advances sq->pc and skips
xsc_sq_notify_hw(). If a later packet (such as the final packet in the
batch where netdev_xmit_more() is false) jumps to err_drop (for example on
skb->len > sq->hw_mtu, skb_linearize() failure, skb->len == 0, or DMA
mapping failure in xsc_txwqe_build_dsegs()), err_drop frees the skb and
returns without ringing the doorbell for the earlier WQEs. Should err_drop
call xsc_sq_notify_hw() when !netdev_xmit_more() or when uncommitted WQEs
are pending?
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
> new file mode 100644
> index 0000000000000..262421740e974
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
[ ... ]
> +static bool xsc_poll_tx_cq(struct xsc_cq *cq, int napi_budget)
> +{
> + struct xsc_adapter *adapter;
> + struct xsc_cqe *cqe;
> + struct device *dev;
> + struct xsc_sq *sq;
> + u32 dma_fifo_cc;
> + u32 nbytes = 0;
> + u16 npkts = 0;
> + int i = 0;
> + u16 sqcc;
> +
> + sq = container_of(cq, struct xsc_sq, cq);
> + if (!test_bit(XSC_ETH_SQ_STATE_ENABLED, &sq->state))
> + return false;
> +
> + adapter = sq->channel->adapter;
> + dev = adapter->dev;
> +
> + cqe = xsc_cqwq_get_cqe(&cq->wq);
> + if (!cqe)
> + goto out;
> +
> + if (unlikely(get_cqe_opcode(cqe) & BIT(7)))
> + xsc_dump_error_sqcqe(sq, cqe);
> +
> + sqcc = sq->cc;
> +
> + /* avoid dirtying sq cache line every cqe */
> + dma_fifo_cc = sq->dma_fifo_cc;
> + i = 0;
> + do {
> + struct xsc_tx_wqe_info *wi;
> + struct sk_buff *skb;
> + int j;
> + u16 ci;
> +
> + xsc_cqwq_pop(&cq->wq);
> +
> + ci = xsc_wq_cyc_ctr2ix(&sq->wq, sqcc);
> + wi = &sq->db.wqe_info[ci];
> + skb = wi->skb;
> +
> + if (unlikely(!skb))
> + continue;
> +
> + for (j = 0; j < wi->num_dma; j++) {
> + struct xsc_sq_dma *dma = xsc_dma_get(sq, dma_fifo_cc++);
> +
> + xsc_tx_dma_unmap(dev, dma);
> + }
> +
> + npkts++;
> + nbytes += wi->num_bytes;
> + sqcc += wi->num_wqebbs;
> + napi_consume_skb(skb, 0);
> +
> + } while ((++i < napi_budget) && (cqe = xsc_cqwq_get_cqe(&cq->wq)));
[Severity: Medium]
There are a few issues in the xsc_poll_tx_cq() loop:
1. get_cqe_opcode(cqe) & BIT(7) is only checked once before the do-while
loop rather than inside the loop before xsc_cqwq_pop(&cq->wq). Won't any
error CQEs fetched after the first CQE in a NAPI poll batch be silently
ignored instead of logged via xsc_dump_error_sqcqe()?
2. If unlikely(!skb) is taken after xsc_cqwq_pop(&cq->wq), the loop
executes continue without advancing sqcc (unlike xsc_free_tx_wqe(), which
increments sq->cc++), and wi->skb is never cleared when a WQE completes.
Could that desynchronize sqcc from the CQ if a nop WQE is encountered?
3. napi_consume_skb(skb, 0) is called with a hardcoded budget of 0, which
causes napi_consume_skb() to always take the !budget non-NAPI fallback
(dev_consume_skb_any()) instead of using the NAPI per-CPU skb cache. Should
the NAPI poll budget be passed through to napi_consume_skb() here?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=8
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 09/15] net/xsc: add ethernet RX path and statistics
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (7 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 08/15] net/xsc: add ethernet TX path Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 10/15] RDMA/xsc: add RDMA device support Xin Tian
` (5 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement the ethernet receive path on top of page pools. Each RQ
gets a page pool with DMA mapping and device sync; receive pages are
partitioned into fixed-size fragments laid out at successive
offsets, and a page is recycled only when its last fragment is
consumed, so the pool holds a single reference per page. WQEs are
replenished in batches and the RX doorbell is rung after the batch.
Frames are built either linearly, with build_skb on the received
page and the packet-parse header length reserved as headroom, or
non-linearly for MTUs too large for the fragment size, by copying
the header into an skb and attaching the remaining fragments as
page frags. The completion handler validates the CQE, drops
truncated and oversized frames, reports checksum status including
tunnel inner and outer checksums, extracts the RSS hash with its
layer classification, records the receive queue and hands the frame
to GRO.
Also implement the RSS receive side: a set of receive queue pairs
created through a single multi-QP firmware command that returns a
contiguous QP number base, wired to the RSS indirection table.
Add ndo_get_stats64 folding per-channel TX and RX software counters
into the rtnl statistics, and advertise RX checksum and RX hash
offload.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
.../net/ethernet/yunsilicon/xsc/net/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/net/main.c | 364 +++++++++++-
.../net/ethernet/yunsilicon/xsc/net/xsc_eth.h | 2 -
.../ethernet/yunsilicon/xsc/net/xsc_eth_rx.c | 556 ++++++++++++++++++
.../yunsilicon/xsc/net/xsc_eth_stats.c | 40 ++
.../yunsilicon/xsc/net/xsc_eth_stats.h | 1 +
.../yunsilicon/xsc/net/xsc_eth_txrx.c | 17 +-
7 files changed, 975 insertions(+), 7 deletions(-)
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
index ffb819884..43d049e42 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc_eth.o
-xsc_eth-y := main.o xsc_eth_wq.o xsc_eth_txrx.o xsc_eth_tx.o
+xsc_eth-y := main.o xsc_eth_wq.o xsc_eth_txrx.o xsc_eth_tx.o xsc_eth_rx.o xsc_eth_stats.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
index a02f95145..e56e0eec3 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
@@ -686,6 +686,14 @@ static int xsc_eth_open_channel(struct xsc_adapter *adapter,
static int xsc_eth_modify_qps_channel(struct xsc_adapter *adapter,
struct xsc_channel *c)
{
+ int ret;
+
+ c->qp.rq.post_wqes(&c->qp.rq);
+ ret = xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.rq.rqn,
+ XSC_CMD_OP_RTR2RTS_QP);
+ if (ret)
+ return ret;
+
return xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.sq.sqn,
XSC_CMD_OP_RTR2RTS_QP);
}
@@ -707,8 +715,329 @@ static int xsc_eth_modify_qps(struct xsc_adapter *adapter,
return 0;
}
+static void xsc_eth_init_frags_partition(struct xsc_rq *rq)
+{
+ struct xsc_wqe_frag_info next_frag = {};
+ struct xsc_wqe_frag_info *prev;
+ int i;
+
+ next_frag.di = &rq->wqe.di[0];
+ next_frag.offset = 0;
+ prev = NULL;
+
+ for (i = 0; i < xsc_wq_cyc_get_size(&rq->wqe.wq); i++) {
+ struct xsc_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+ struct xsc_wqe_frag_info *frag =
+ &rq->wqe.frags[i << rq->wqe.info.log_num_frags];
+ int f;
+
+ for (f = 0; f < rq->wqe.info.num_frags; f++, frag++) {
+ if (next_frag.offset + frag_info[f].frag_stride >
+ XSC_RX_FRAG_SZ) {
+ next_frag.di++;
+ next_frag.offset = 0;
+ if (prev)
+ prev->last_in_page = 1;
+ }
+ *frag = next_frag;
+
+ /* prepare next */
+ next_frag.offset += frag_info[f].frag_stride;
+ prev = frag;
+ }
+ }
+
+ if (prev)
+ prev->last_in_page = 1;
+}
+
+static int xsc_eth_init_di_list(struct xsc_rq *rq, u32 wq_sz, int cpu)
+{
+ int len = wq_sz << rq->wqe.info.log_num_frags;
+
+ rq->wqe.di = kvzalloc_node(array_size(len, sizeof(*rq->wqe.di)),
+ GFP_KERNEL, cpu_to_node(cpu));
+ if (!rq->wqe.di)
+ return -ENOMEM;
+
+ xsc_eth_init_frags_partition(rq);
+
+ return 0;
+}
+
+static void xsc_eth_free_di_list(struct xsc_rq *rq)
+{
+ kvfree(rq->wqe.di);
+}
+
+static int xsc_eth_alloc_rq(struct xsc_channel *c,
+ struct xsc_rq *prq,
+ struct xsc_rq_param *prq_param)
+{
+ u8 ele_log_size = prq_param->rq_attr.ele_log_size;
+ u8 q_log_size = prq_param->rq_attr.q_log_size;
+ struct page_pool_params pagepool_params = {0};
+ struct xsc_adapter *adapter = c->adapter;
+ struct xsc_channel_stats *channel_stats;
+ u32 pool_size = 1 << q_log_size;
+ struct xsc_stats *stats;
+ int ret = 0;
+ u32 wq_sz;
+ int i, f;
+
+ stats = c->adapter->stats;
+ channel_stats = &stats->channel_stats[c->chl_idx];
+ prq->stats = &channel_stats->rq;
+ prq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_wq_cyc_create(c->adapter->xdev, &prq_param->wq,
+ q_log_size, ele_log_size, &prq->wqe.wq,
+ &prq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ wq_sz = xsc_wq_cyc_get_size(&prq->wqe.wq);
+
+ prq->wqe.info = prq_param->frags_info;
+ prq->wqe.frags =
+ kvzalloc_node(array_size((wq_sz << prq->wqe.info.log_num_frags),
+ sizeof(*prq->wqe.frags)),
+ GFP_KERNEL,
+ cpu_to_node(c->cpu));
+ if (!prq->wqe.frags) {
+ ret = -ENOMEM;
+ goto err_destroy_wq;
+ }
+
+ ret = xsc_eth_init_di_list(prq, wq_sz, c->cpu);
+ if (ret)
+ goto err_free_frags;
+
+ prq->buff.map_dir = DMA_FROM_DEVICE;
+
+ /* Create a page_pool and register it with rxq */
+ pool_size = wq_sz << prq->wqe.info.log_num_frags;
+ pagepool_params.order = XSC_RX_FRAG_SZ_ORDER;
+ pagepool_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV;
+ pagepool_params.pool_size = pool_size;
+ pagepool_params.nid = cpu_to_node(c->cpu);
+ pagepool_params.dev = c->adapter->dev;
+ pagepool_params.dma_dir = prq->buff.map_dir;
+ pagepool_params.max_len = PAGE_SIZE;
+
+ prq->page_pool = page_pool_create(&pagepool_params);
+ if (IS_ERR(prq->page_pool)) {
+ ret = PTR_ERR(prq->page_pool);
+ prq->page_pool = NULL;
+ goto err_free_di_list;
+ }
+
+ if (c->chl_idx == 0)
+ netdev_dbg(adapter->netdev,
+ "page pool: size=%d, cpu=%d, pool_numa=%d, mtu=%d, wqe_numa=%d\n",
+ pool_size, c->cpu, pagepool_params.nid,
+ adapter->nic_param.mtu,
+ prq_param->wq.buf_numa_node);
+
+ for (i = 0; i < wq_sz; i++) {
+ struct xsc_eth_rx_wqe_cyc *wqe =
+ xsc_wq_cyc_get_wqe(&prq->wqe.wq, i);
+
+ for (f = 0; f < prq->wqe.info.num_frags; f++) {
+ u32 frag_size = prq->wqe.info.arr[f].frag_size;
+ u32 val = 0;
+
+ val |= FIELD_PREP(XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK,
+ frag_size);
+ wqe->data[f].data0 = cpu_to_le32(val);
+ wqe->data[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);
+ }
+
+ for (; f < prq->wqe.info.frags_max_num; f++) {
+ wqe->data[f].data0 = 0;
+ wqe->data[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);
+ wqe->data[f].va = 0;
+ }
+ }
+
+ prq->post_wqes = xsc_eth_post_rx_wqes;
+ prq->handle_rx_cqe = xsc_eth_handle_rx_cqe;
+ prq->dealloc_wqe = xsc_eth_dealloc_rx_wqe;
+ prq->wqe.skb_from_cqe = xsc_rx_is_linear_skb(adapter->nic_param.mtu) ?
+ xsc_skb_from_cqe_linear :
+ xsc_skb_from_cqe_nonlinear;
+ prq->ix = c->chl_idx;
+ prq->frags_sz = adapter->nic_param.rq_frags_size;
+
+ return 0;
+
+err_free_di_list:
+ xsc_eth_free_di_list(prq);
+err_free_frags:
+ kvfree(prq->wqe.frags);
+err_destroy_wq:
+ xsc_eth_wq_destroy(&prq->wq_ctrl);
+ return ret;
+}
+
+static void xsc_free_qp_rq(struct xsc_rq *rq)
+{
+ kvfree(rq->wqe.frags);
+ kvfree(rq->wqe.di);
+
+ if (rq->page_pool)
+ page_pool_destroy(rq->page_pool);
+
+ xsc_eth_wq_destroy(&rq->wq_ctrl);
+}
+
+static int xsc_eth_open_rss_qp_rqs(struct xsc_adapter *adapter,
+ struct xsc_rq_param *prq_param,
+ struct xsc_eth_channels *chls,
+ unsigned int num_chl)
+{
+ u8 q_log_size = prq_param->rq_attr.q_log_size;
+ struct xsc_create_multiqp_mbox_in *in;
+ struct xsc_create_qp_request *req;
+ unsigned int hw_npages;
+ struct xsc_channel *c;
+ int ret = 0;
+ struct xsc_rq *prq;
+ int paslen = 0;
+ int entry_len;
+ u32 rqn_base;
+ int i;
+ int inlen;
+
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ ret = xsc_eth_alloc_rq(c, prq, prq_param);
+ if (ret)
+ goto err_alloc_rqs;
+
+ hw_npages = DIV_ROUND_UP(prq->wq_ctrl.buf.size, XSC_PAGE_SIZE_4K);
+ /*support different npages number smoothly*/
+ entry_len = sizeof(struct xsc_create_qp_request) +
+ sizeof(__be64) * hw_npages;
+
+ paslen += entry_len;
+ }
+
+ inlen = sizeof(struct xsc_create_multiqp_mbox_in) + paslen;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ ret = -ENOMEM;
+ goto err_create_rss_rqs;
+ }
+
+ in->qp_num = cpu_to_be16(num_chl);
+ in->qp_type = XSC_QUEUE_TYPE_RAW;
+ in->req_len = cpu_to_be32(inlen);
+
+ req = (struct xsc_create_qp_request *)&in->data[0];
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ hw_npages = DIV_ROUND_UP(prq->wq_ctrl.buf.size, XSC_PAGE_SIZE_4K);
+ /* no use for eth */
+ req->input_qpn = cpu_to_be16(0);
+ req->qp_type = XSC_QUEUE_TYPE_RAW;
+ req->log_rq_sz = ilog2(adapter->xdev->caps.recv_ds_num)
+ + q_log_size;
+ req->pa_num = cpu_to_be16(hw_npages);
+ req->cqn_recv = cpu_to_be16(prq->cq.xcq.cqn);
+ req->cqn_send = req->cqn_recv;
+ req->glb_funcid = cpu_to_be16(adapter->xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&prq->wq_ctrl.buf,
+ &req->pas[0], hw_npages);
+ entry_len = sizeof(struct xsc_create_qp_request) +
+ sizeof(__be64) * hw_npages;
+ req = (struct xsc_create_qp_request *)
+ ((u8 *)req + entry_len);
+ }
+
+ ret = xsc_core_eth_create_rss_qp_rqs(adapter->xdev, in, inlen,
+ &rqn_base);
+ kvfree(in);
+ if (ret)
+ goto err_create_rss_rqs;
+
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ prq->rqn = rqn_base + i;
+ prq->cqp.qpn = prq->rqn;
+ prq->cqp.event = xsc_eth_qp_event;
+ prq->cqp.eth_queue_type = XSC_RES_RQ;
+ ret = xsc_core_create_resource_common(adapter->xdev,
+ &prq->cqp);
+ if (ret) {
+ netdev_err(adapter->netdev,
+ "create resource common error qp:%d errno:%d\n",
+ prq->rqn, ret);
+ goto err_destroy_rss_rqs;
+ }
+ }
+
+ adapter->channels.rqn_base = rqn_base;
+ return 0;
+
+err_destroy_rss_rqs:
+ while (--i >= 0)
+ xsc_core_destroy_resource_common(adapter->xdev,
+ &chls->c[i].qp.rq.cqp);
+
+ for (i = 0; i < num_chl; i++)
+ xsc_core_eth_destroy_qp(adapter->xdev, rqn_base + i);
+
+err_create_rss_rqs:
+ i = num_chl;
+err_alloc_rqs:
+ for (--i; i >= 0; i--) {
+ c = &chls->c[i];
+ xsc_free_qp_rq(&c->qp.rq);
+ }
+ return ret;
+}
+
+static void xsc_eth_free_rx_wqe(struct xsc_rq *rq)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u16 wqe_ix;
+
+ while (!xsc_wq_cyc_is_empty(wq)) {
+ wqe_ix = xsc_wq_cyc_get_tail(wq);
+ rq->dealloc_wqe(rq, wqe_ix);
+ xsc_wq_cyc_pop(wq);
+ }
+}
+
+static void xsc_eth_close_qp_rq(struct xsc_channel *c, struct xsc_rq *prq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+
+ xsc_core_destroy_resource_common(xdev, &prq->cqp);
+
+ if (xsc_core_eth_destroy_qp(xdev, prq->cqp.qpn))
+ netdev_err(c->adapter->netdev, "failed to close ch%d rq%d\n",
+ c->chl_idx, prq->cqp.qpn);
+
+ xsc_eth_free_rx_wqe(prq);
+ xsc_free_qp_rq(prq);
+}
+
static void xsc_eth_close_channel(struct xsc_channel *c, bool free_rq)
{
+ if (free_rq)
+ xsc_eth_close_qp_rq(c, &c->qp.rq);
+ xsc_eth_close_cq(c, &c->qp.rq.cq);
+ memset(&c->qp.rq, 0, sizeof(struct xsc_rq));
+
xsc_eth_close_qp_sq(c, &c->qp.sq);
xsc_eth_close_cq(c, &c->qp.sq.cq);
@@ -720,6 +1049,7 @@ static int xsc_eth_open_channels(struct xsc_adapter *adapter)
struct xsc_eth_channels *chls = &adapter->channels;
struct xsc_core_device *xdev = adapter->xdev;
struct xsc_channel_param *chl_param;
+ bool free_rq = false;
int ret = 0;
int i;
@@ -745,6 +1075,14 @@ static int xsc_eth_open_channels(struct xsc_adapter *adapter)
goto err_close_channel;
}
+ ret = xsc_eth_open_rss_qp_rqs(adapter,
+ &chl_param->rq_param,
+ chls,
+ chls->num_chl);
+ if (ret)
+ goto err_close_channel;
+ free_rq = true;
+
for (i = 0; i < chls->num_chl; i++)
napi_enable(&chls->c[i].napi);
@@ -763,7 +1101,7 @@ static int xsc_eth_open_channels(struct xsc_adapter *adapter)
err_close_channel:
for (--i; i >= 0; i--)
- xsc_eth_close_channel(&chls->c[i], false);
+ xsc_eth_close_channel(&chls->c[i], free_rq);
kvfree(chl_param);
err_free_ch:
@@ -818,13 +1156,25 @@ static void xsc_eth_deactivate_txqsq(struct xsc_channel *c)
clear_bit(XSC_ETH_SQ_STATE_ENABLED, &c->qp.sq.state);
}
+static void xsc_activate_rq(struct xsc_channel *c)
+{
+ set_bit(XSC_ETH_RQ_STATE_ENABLED, &c->qp.rq.state);
+}
+
+static void xsc_deactivate_rq(struct xsc_channel *c)
+{
+ clear_bit(XSC_ETH_RQ_STATE_ENABLED, &c->qp.rq.state);
+}
+
static void xsc_eth_activate_channel(struct xsc_channel *c)
{
xsc_eth_activate_txqsq(c);
+ xsc_activate_rq(c);
}
static void xsc_eth_deactivate_channel(struct xsc_channel *c)
{
+ xsc_deactivate_rq(c);
xsc_eth_deactivate_txqsq(c);
}
@@ -1121,6 +1471,14 @@ static int xsc_eth_close(struct net_device *netdev)
return ret;
}
+static void xsc_eth_get_stats(struct net_device *netdev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+
+ xsc_eth_fold_sw_stats64(adapter, stats);
+}
+
static int xsc_eth_set_hw_mtu(struct xsc_core_device *xdev,
u16 mtu, u16 rx_buf_sz)
{
@@ -1152,6 +1510,7 @@ static const struct net_device_ops xsc_netdev_ops = {
.ndo_open = xsc_eth_open,
.ndo_stop = xsc_eth_close,
.ndo_start_xmit = xsc_eth_xmit_start,
+ .ndo_get_stats64 = xsc_eth_get_stats,
};
static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
@@ -1168,7 +1527,8 @@ static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
netdev->vlan_features |= NETIF_F_SG |
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_TSO | NETIF_F_TSO6;
+ NETIF_F_TSO | NETIF_F_TSO6 |
+ NETIF_F_RXCSUM | NETIF_F_RXHASH;
netdev->hw_features = netdev->vlan_features;
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
index 4d9f92e09..92f055395 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
@@ -12,8 +12,6 @@
#include "xsc_eth_common.h"
#include "xsc_eth_stats.h"
-#define XSC_INVALID_LKEY 0x100
-
struct xsc_rss_params {
u32 rx_hash_fields[XSC_NUM_INDIR_TIRS];
u32 rss_hash_tmpl;
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
new file mode 100644
index 000000000..00ac7eacc
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
@@ -0,0 +1,556 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#include <linux/net_tstamp.h>
+#include <linux/device.h>
+
+#include "xsc_pp.h"
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+#include "xsc_eth_common.h"
+#include "xsc_pph.h"
+
+static void xsc_rq_notify_hw(struct xsc_rq *rq)
+{
+ struct xsc_core_device *xdev = rq->cq.xdev;
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u64 rqwqe_id;
+
+ rqwqe_id = wq->wqe_ctr << (ilog2(xdev->caps.recv_ds_num));
+ /* reverse wqe index to ds index */
+ xsc_update_rx_db(xdev, rq->rqn, rqwqe_id);
+}
+
+static void xsc_skb_set_hash(struct xsc_adapter *adapter,
+ struct xsc_cqe *cqe,
+ struct sk_buff *skb)
+{
+ struct xsc_rss_params *rss = &adapter->rss_param;
+ bool l3_hash = false;
+ bool l4_hash = false;
+ u32 hash_field;
+ int ht = 0;
+
+ if (adapter->netdev->features & NETIF_F_RXHASH) {
+ if (skb->protocol == htons(ETH_P_IP)) {
+ hash_field = rss->rx_hash_fields[XSC_TT_IPV4_TCP];
+ if (hash_field & XSC_HASH_FIELD_SEL_SRC_IP ||
+ hash_field & XSC_HASH_FIELD_SEL_DST_IP)
+ l3_hash = true;
+
+ if (hash_field & XSC_HASH_FIELD_SEL_SPORT ||
+ hash_field & XSC_HASH_FIELD_SEL_DPORT)
+ l4_hash = true;
+ } else if (skb->protocol == htons(ETH_P_IPV6)) {
+ hash_field = rss->rx_hash_fields[XSC_TT_IPV6_TCP];
+ if (hash_field & XSC_HASH_FIELD_SEL_SRC_IPV6 ||
+ hash_field & XSC_HASH_FIELD_SEL_DST_IPV6)
+ l3_hash = true;
+
+ if (hash_field & XSC_HASH_FIELD_SEL_SPORT_V6 ||
+ hash_field & XSC_HASH_FIELD_SEL_DPORT_V6)
+ l4_hash = true;
+ }
+
+ if (l3_hash && l4_hash)
+ ht = PKT_HASH_TYPE_L4;
+ else if (l3_hash)
+ ht = PKT_HASH_TYPE_L3;
+ if (ht)
+ skb_set_hash(skb, le32_to_cpu(cqe->vni), ht);
+ }
+}
+
+static void xsc_handle_csum(struct xsc_cqe *cqe, struct xsc_rq *rq,
+ struct sk_buff *skb, struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_dma_info *dma_info;
+ struct net_device *netdev;
+ void *hw_pph;
+ struct xsc_channel *c;
+ int offset_from;
+
+ c = rq->cq.channel;
+ netdev = c->adapter->netdev;
+ dma_info = wi->di;
+ offset_from = wi->offset;
+ hw_pph = page_address(dma_info->page) + offset_from;
+
+ if (unlikely((netdev->features & NETIF_F_RXCSUM) == 0))
+ goto csum_none;
+
+ if (!(le32_to_cpu(cqe->data0) & XSC_CQE_DATA0_HAS_PPH))
+ goto csum_none;
+
+ dma_sync_single_for_cpu(rq->cq.xdev->device,
+ dma_info->addr + offset_from,
+ XSC_PPH_HEAD_LEN, DMA_FROM_DEVICE);
+
+ if (unlikely(XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(hw_pph) & PACKET_UNKNOWN))
+ goto csum_none;
+
+ if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_DATA0_CSUM_ERR_MASK, le32_to_cpu(cqe->data0)) &
+ OUTER_AND_INNER))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb->csum_level = 1;
+ skb->encapsulation = 1;
+ } else if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_DATA0_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ OUTER_BIT) &&
+ (FIELD_GET(XSC_CQE_DATA0_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ INNER_BIT))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb->csum_level = 0;
+ skb->encapsulation = 1;
+ } else if (!XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_DATA0_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ OUTER_BIT))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ }
+
+ goto out;
+
+csum_none:
+ skb->csum = 0;
+ skb->ip_summed = CHECKSUM_NONE;
+out:
+ return;
+}
+
+static void xsc_build_rx_skb(struct xsc_cqe *cqe,
+ u32 cqe_bcnt,
+ struct xsc_rq *rq,
+ struct sk_buff *skb,
+ struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ struct xsc_channel *c;
+
+ c = rq->cq.channel;
+ adapter = c->adapter;
+ netdev = c->netdev;
+
+ skb->mac_len = ETH_HLEN;
+
+ skb_record_rx_queue(skb, rq->ix);
+ xsc_handle_csum(cqe, rq, skb, wi);
+
+ skb->protocol = eth_type_trans(skb, netdev);
+ xsc_skb_set_hash(adapter, cqe, skb);
+}
+
+static void xsc_complete_rx_cqe(struct xsc_rq *rq,
+ struct xsc_cqe *cqe,
+ u32 cqe_bcnt,
+ struct sk_buff *skb,
+ struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_rq_stats *stats = rq->stats;
+
+ stats->packets++;
+ stats->bytes += cqe_bcnt;
+ xsc_build_rx_skb(cqe, cqe_bcnt, rq, skb, wi);
+}
+
+static void xsc_add_skb_frag(struct xsc_rq *rq,
+ struct sk_buff *skb,
+ struct xsc_dma_info *di,
+ u32 frag_offset, u32 len,
+ unsigned int truesize)
+{
+ struct xsc_channel *c = rq->cq.channel;
+ struct device *dev = c->adapter->dev;
+
+ dma_sync_single_for_cpu(dev, di->addr + frag_offset,
+ len, DMA_FROM_DEVICE);
+ page_ref_inc(di->page);
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+ di->page, frag_offset, len, truesize);
+}
+
+static void xsc_copy_skb_header(struct device *dev,
+ struct sk_buff *skb,
+ struct xsc_dma_info *dma_info,
+ int offset_from, u32 headlen)
+{
+ void *from = page_address(dma_info->page) + offset_from;
+ /* Aligning len to sizeof(long) optimizes memcpy performance */
+ unsigned int len = ALIGN(headlen, sizeof(long));
+
+ dma_sync_single_for_cpu(dev, dma_info->addr + offset_from, len,
+ DMA_FROM_DEVICE);
+ skb_copy_to_linear_data(skb, from, len);
+}
+
+static struct sk_buff *xsc_build_linear_skb(struct xsc_rq *rq, void *va,
+ u32 frag_size, u16 headroom,
+ u32 cqe_bcnt)
+{
+ struct sk_buff *skb = build_skb(va, frag_size);
+
+ if (unlikely(!skb))
+ return NULL;
+
+ skb_reserve(skb, headroom);
+ skb_put(skb, cqe_bcnt);
+
+ return skb;
+}
+
+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph)
+{
+ int pph_len = has_pph ? XSC_PPH_HEAD_LEN : 0;
+ struct xsc_dma_info *di = wi->di;
+ struct sk_buff *skb;
+ void *va, *data;
+ u32 frag_size;
+
+ va = page_address(di->page) + wi->offset;
+ data = va + pph_len;
+ frag_size = XSC_SKB_FRAG_SZ(cqe_bcnt);
+
+ dma_sync_single_range_for_cpu(rq->cq.xdev->device, di->addr, wi->offset,
+ frag_size, DMA_FROM_DEVICE);
+ net_prefetchw(va); /* xdp_frame data area */
+ net_prefetch(data);
+
+ skb = xsc_build_linear_skb(rq, va, frag_size, pph_len,
+ (cqe_bcnt - pph_len));
+ if (unlikely(!skb))
+ return NULL;
+
+ /* queue up for recycling/reuse */
+ page_ref_inc(di->page);
+
+ return skb;
+}
+
+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph)
+{
+ struct xsc_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+ u16 headlen = min_t(u32, XSC_RX_MAX_HEAD, cqe_bcnt);
+ struct xsc_wqe_frag_info *head_wi = wi;
+ u16 head_offset = head_wi->offset;
+ u16 byte_cnt = cqe_bcnt - headlen;
+ u16 frag_consumed_bytes = 0;
+ u16 frag_headlen = headlen;
+ struct net_device *netdev;
+ struct xsc_channel *c;
+ struct sk_buff *skb;
+ struct device *dev;
+ u8 fragcnt = 0;
+
+ c = rq->cq.channel;
+ dev = c->adapter->dev;
+ netdev = c->adapter->netdev;
+
+ skb = napi_alloc_skb(rq->cq.napi, ALIGN(XSC_RX_MAX_HEAD, sizeof(long)));
+ if (unlikely(!skb))
+ return NULL;
+
+ net_prefetchw(skb->data);
+
+ if (likely(has_pph)) {
+ headlen = min_t(u32, XSC_RX_MAX_HEAD,
+ (cqe_bcnt - XSC_PPH_HEAD_LEN));
+ frag_headlen = headlen + XSC_PPH_HEAD_LEN;
+ byte_cnt = cqe_bcnt - headlen - XSC_PPH_HEAD_LEN;
+ head_offset += XSC_PPH_HEAD_LEN;
+ }
+
+ while (byte_cnt) {
+ /*figure out whether the first fragment can be a page ?*/
+ frag_consumed_bytes =
+ min_t(u16, frag_info->frag_size - frag_headlen,
+ byte_cnt);
+
+ xsc_add_skb_frag(rq, skb, wi->di, wi->offset + frag_headlen,
+ frag_consumed_bytes, frag_info->frag_stride);
+ byte_cnt -= frag_consumed_bytes;
+
+ /*to protect extend wqe read, drop exceed bytes*/
+ frag_headlen = 0;
+ fragcnt++;
+ if (fragcnt == rq->wqe.info.num_frags) {
+ if (byte_cnt) {
+ netdev_warn(netdev,
+ "large packet reach the maximum rev-wqe num.\n");
+ netdev_warn(netdev,
+ "%u bytes dropped: frag_num=%d, headlen=%d, cqe_cnt=%d, frag0_bytes=%d, frag_size=%d\n",
+ byte_cnt, fragcnt,
+ headlen, cqe_bcnt,
+ frag_consumed_bytes,
+ frag_info->frag_size);
+ }
+ break;
+ }
+
+ frag_info++;
+ wi++;
+ }
+
+ /* copy header */
+ xsc_copy_skb_header(dev, skb, head_wi->di, head_offset, headlen);
+
+ /* skb linear part was allocated with headlen and aligned to long */
+ skb->tail += headlen;
+ skb->len += headlen;
+
+ return skb;
+}
+
+static void xsc_put_rx_frag(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *frag)
+{
+ if (frag->last_in_page)
+ page_pool_recycle_direct(rq->page_pool, frag->di->page);
+}
+
+static struct xsc_wqe_frag_info *get_frag(struct xsc_rq *rq, u16 ix)
+{
+ return &rq->wqe.frags[ix << rq->wqe.info.log_num_frags];
+}
+
+static void xsc_free_rx_wqe(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi)
+{
+ int i;
+
+ for (i = 0; i < rq->wqe.info.num_frags; i++, wi++)
+ xsc_put_rx_frag(rq, wi);
+}
+
+static void xsc_dump_error_rqcqe(struct xsc_rq *rq,
+ struct xsc_cqe *cqe)
+{
+ struct net_device *netdev;
+ struct xsc_channel *c;
+ u32 ci;
+
+ c = rq->cq.channel;
+ netdev = c->adapter->netdev;
+ ci = xsc_cqwq_get_ci(&rq->cq.wq);
+
+ net_err_ratelimited("Error cqe on dev=%s, cqn=%d, ci=%d, rqn=%d, qpn=%ld, error_code=0x%x\n",
+ netdev->name, rq->cq.xcq.cqn, ci,
+ rq->rqn,
+ FIELD_GET(XSC_CQE_DATA0_QP_ID_MASK,
+ le32_to_cpu(cqe->data0)),
+ get_cqe_opcode(cqe));
+}
+
+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,
+ struct xsc_rq *rq, struct xsc_cqe *cqe)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u8 cqe_opcode = get_cqe_opcode(cqe);
+ struct xsc_wqe_frag_info *wi;
+ struct sk_buff *skb;
+ u32 cqe_bcnt;
+ u16 ci;
+
+ ci = xsc_wq_cyc_ctr2ix(wq, cqwq->cc);
+ wi = get_frag(rq, ci);
+ if (unlikely(cqe_opcode & BIT(7))) {
+ xsc_dump_error_rqcqe(rq, cqe);
+ goto free_wqe;
+ }
+
+ cqe_bcnt = le32_to_cpu(cqe->msg_len);
+ if ((le32_to_cpu(cqe->data0) & XSC_CQE_DATA0_HAS_PPH) &&
+ cqe_bcnt <= XSC_PPH_HEAD_LEN)
+ goto free_wqe;
+
+ if (unlikely(cqe_bcnt < ETH_HLEN + ((le32_to_cpu(cqe->data0) &
+ XSC_CQE_DATA0_HAS_PPH) ?
+ XSC_PPH_HEAD_LEN : 0))) {
+ rq->stats->dropped++;
+ goto free_wqe;
+ }
+
+ if (unlikely(cqe_bcnt > rq->frags_sz))
+ goto free_wqe;
+
+ cqe_bcnt = min_t(u32, cqe_bcnt, rq->frags_sz);
+ skb = rq->wqe.skb_from_cqe(rq, wi, cqe_bcnt,
+ !!(le32_to_cpu(cqe->data0) &
+ XSC_CQE_DATA0_HAS_PPH));
+ if (!skb)
+ goto free_wqe;
+
+ xsc_complete_rx_cqe(rq, cqe,
+ (le32_to_cpu(cqe->data0) & XSC_CQE_DATA0_HAS_PPH) ?
+ cqe_bcnt - XSC_PPH_HEAD_LEN : cqe_bcnt,
+ skb, wi);
+
+ napi_gro_receive(rq->cq.napi, skb);
+
+free_wqe:
+ xsc_free_rx_wqe(rq, wi);
+ xsc_wq_cyc_pop(wq);
+}
+
+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget)
+{
+ struct xsc_rq *rq = container_of(cq, struct xsc_rq, cq);
+ struct xsc_cqwq *cqwq = &cq->wq;
+ struct xsc_cqe *cqe;
+ int work_done = 0;
+
+ if (!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state))
+ return 0;
+
+ while ((work_done < budget) && (cqe = xsc_cqwq_get_cqe(cqwq))) {
+ rq->handle_rx_cqe(cqwq, rq, cqe);
+ ++work_done;
+
+ xsc_cqwq_pop(cqwq);
+ }
+
+ if (!work_done)
+ goto out;
+
+ xsc_cq_notify_hw(cq);
+ /* ensure cq space is freed before enabling more cqes */
+ wmb();
+
+out:
+ return work_done;
+}
+
+static int xsc_page_alloc_pool(struct xsc_rq *rq,
+ struct xsc_dma_info *dma_info)
+{
+ dma_info->page = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!dma_info->page))
+ return -ENOMEM;
+ dma_info->addr = page_pool_get_dma_addr(dma_info->page);
+
+ return 0;
+}
+
+static int xsc_get_rx_frag(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *frag)
+{
+ int err = 0;
+
+ if (!frag->offset)
+ /* On first frag (offset == 0), replenish page (dma_info
+ * actually). Other frags that point to the same dma_info
+ * (with a different offset) should just use the new one
+ * without replenishing again by themselves.
+ */
+ err = xsc_page_alloc_pool(rq, frag->di);
+
+ return err;
+}
+
+static int xsc_alloc_rx_wqe(struct xsc_rq *rq,
+ struct xsc_eth_rx_wqe_cyc *wqe,
+ u16 ix)
+{
+ struct xsc_wqe_frag_info *frag = get_frag(rq, ix);
+ u64 addr;
+ int err;
+ int i;
+
+ for (i = 0; i < rq->wqe.info.num_frags; i++, frag++) {
+ err = xsc_get_rx_frag(rq, frag);
+ if (unlikely(err))
+ goto err_free_frags;
+
+ addr = frag->di->addr + frag->offset;
+ wqe->data[i].va = cpu_to_le64(addr);
+ }
+
+ return 0;
+
+err_free_frags:
+ while (--i >= 0)
+ xsc_put_rx_frag(rq, --frag);
+
+ return err;
+}
+
+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix)
+{
+ struct xsc_wqe_frag_info *wi = get_frag(rq, ix);
+
+ xsc_free_rx_wqe(rq, wi);
+}
+
+static int xsc_alloc_rx_wqes(struct xsc_rq *rq, u16 ix, u8 wqe_bulk)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ struct xsc_eth_rx_wqe_cyc *wqe;
+ int err;
+ int idx;
+ int i;
+
+ for (i = 0; i < wqe_bulk; i++) {
+ idx = xsc_wq_cyc_ctr2ix(wq, (ix + i));
+ wqe = xsc_wq_cyc_get_wqe(wq, idx);
+
+ err = xsc_alloc_rx_wqe(rq, wqe, idx);
+ if (unlikely(err))
+ goto err_free_wqes;
+ }
+
+ return 0;
+
+err_free_wqes:
+ while (--i >= 0)
+ xsc_eth_dealloc_rx_wqe(rq, xsc_wq_cyc_ctr2ix(wq, ix + i));
+
+ return err;
+}
+
+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u8 wqe_bulk, wqe_bulk_min;
+ int err = 0;
+ int alloc;
+ u16 head;
+
+ wqe_bulk = rq->wqe.info.wqe_bulk;
+ wqe_bulk_min = rq->wqe.info.wqe_bulk_min;
+ if (xsc_wq_cyc_missing(wq) < wqe_bulk)
+ return false;
+
+ do {
+ head = xsc_wq_cyc_get_head(wq);
+
+ alloc = min_t(int, wqe_bulk, xsc_wq_cyc_missing(wq));
+ if (alloc < wqe_bulk && alloc >= wqe_bulk_min)
+ alloc = alloc & 0xfffffffe;
+
+ if (alloc > 0) {
+ err = xsc_alloc_rx_wqes(rq, head, alloc);
+ if (unlikely(err))
+ break;
+
+ xsc_wq_cyc_push_n(wq, alloc);
+ }
+ } while (xsc_wq_cyc_missing(wq) >= wqe_bulk_min);
+
+ dma_wmb();
+
+ /* ensure wqes are visible to device before updating doorbell record */
+ xsc_rq_notify_hw(rq);
+
+ return !!err;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c
new file mode 100644
index 000000000..adec41675
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "xsc_eth_stats.h"
+#include "xsc_eth.h"
+
+static int xsc_get_netdev_max_channels(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ return min_t(unsigned int, netdev->num_rx_queues,
+ netdev->num_tx_queues);
+}
+
+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,
+ struct rtnl_link_stats64 *s)
+{
+ int i;
+
+ for (i = 0; i < xsc_get_netdev_max_channels(adapter); i++) {
+ struct xsc_channel_stats *channel_stats;
+ struct xsc_rq_stats *rq_stats;
+ struct xsc_sq_stats *sq_stats;
+
+ channel_stats = &adapter->stats->channel_stats[i];
+ rq_stats = &channel_stats->rq;
+ sq_stats = &channel_stats->sq;
+
+ s->rx_packets += rq_stats->packets;
+ s->rx_bytes += rq_stats->bytes;
+ s->rx_length_errors += rq_stats->dropped;
+
+ s->tx_packets += sq_stats->packets;
+ s->tx_bytes += sq_stats->bytes;
+ s->tx_dropped += sq_stats->dropped;
+ }
+}
+
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
index a391afe97..0f11ef9ae 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
@@ -11,6 +11,7 @@
struct xsc_rq_stats {
u64 packets;
u64 bytes;
+ u64 dropped;
};
struct xsc_sq_stats {
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
index 262421740..42e630547 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
@@ -118,7 +118,9 @@ int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
{
struct xsc_channel *c = container_of(napi, struct xsc_channel, napi);
struct xsc_eth_params *params = &c->adapter->nic_param;
+ struct xsc_rq *rq = &c->qp.rq;
bool busy = false;
+ int work_done = 0;
int tx_budget;
rcu_read_lock();
@@ -126,18 +128,29 @@ int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
tx_budget = params->sq_size >> 2;
busy |= xsc_poll_tx_cq(&c->qp.sq.cq, tx_budget);
+ /* budget=0 means: don't poll rx rings */
+ if (likely(budget)) {
+ work_done = xsc_poll_rx_cq(&rq->cq, budget);
+ busy |= work_done == budget;
+ }
+
+ busy |= rq->post_wqes(rq);
+
if (busy) {
if (likely(xsc_channel_no_affinity_change(c))) {
rcu_read_unlock();
return budget;
}
+ if (budget && work_done == budget)
+ work_done--;
}
- if (unlikely(!napi_complete_done(napi, 0)))
+ if (unlikely(!napi_complete_done(napi, work_done)))
goto err_out;
xsc_cq_notify_hw_rearm(&c->qp.sq.cq);
+ xsc_cq_notify_hw_rearm(&rq->cq);
err_out:
rcu_read_unlock();
- return 0;
+ return work_done;
}
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 09/15] net/xsc: add ethernet RX path and statistics
2026-11-04 15:57 ` [PATCH v1 09/15] net/xsc: add ethernet RX path and statistics Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] net/xsc: missing RQ state check in xsc_eth_post_rx_wqes() and NAPI order bug
- [Medium] net/xsc: unchecked initial RQ refill and doorbell write on alloc failure
- [Medium] net/xsc: xsc_skb_set_hash() classifies all IPv4/IPv6 packets as L4 hash
- [Medium] net/xsc: missing drop stats and dead clamp in xsc_eth_handle_rx_cqe()
- [Medium] net/xsc: host-endian u16 read in XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE()
- [Low] net/xsc: xsc_eth_open_channels() zeroes chls->num_chl before logging it
--
commit 1ef80bdcef5b1d9741776856d8624e7ce0755662
Author: Xin Tian <tianx@yunsilicon.com>
net/xsc: add ethernet RX path and statistics
This commit implements the ethernet receive path using page pools, linear and
non-linear SKB construction, CQE checksum and RSS hash handling, and RSS RQ
creation. It also wires up ndo_get_stats64 and advertises RX checksum and RX
hash offloads.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> index a02f95145d018..e56e0eec30515 100644
> --- a/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
> @@ -686,6 +686,14 @@ static int xsc_eth_open_channel(struct xsc_adapter *adapter,
> static int xsc_eth_modify_qps_channel(struct xsc_adapter *adapter,
> struct xsc_channel *c)
> {
> + int ret;
> +
> + c->qp.rq.post_wqes(&c->qp.rq);
> + ret = xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.rq.rqn,
> + XSC_CMD_OP_RTR2RTS_QP);
> + if (ret)
> + return ret;
> +
> return xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.sq.sqn,
> XSC_CMD_OP_RTR2RTS_QP);
> }
[ ... ]
> @@ -745,6 +1075,14 @@ static int xsc_eth_open_channels(struct xsc_adapter *adapter)
> goto err_close_channel;
> }
>
> + ret = xsc_eth_open_rss_qp_rqs(adapter,
> + &chl_param->rq_param,
> + chls,
> + chls->num_chl);
> + if (ret)
> + goto err_close_channel;
> + free_rq = true;
> +
> for (i = 0; i < chls->num_chl; i++)
> napi_enable(&chls->c[i].napi);
>
> @@ -763,7 +1101,7 @@ static int xsc_eth_open_channels(struct xsc_adapter *adapter)
>
> err_close_channel:
> for (--i; i >= 0; i--)
> - xsc_eth_close_channel(&chls->c[i], false);
> + xsc_eth_close_channel(&chls->c[i], free_rq);
>
> kvfree(chl_param);
> err_free_ch:
[Severity: Low]
When xsc_eth_open_channels() fails (for example if xsc_eth_open_rss_qp_rqs()
or xsc_eth_modify_qps() returns an error), it jumps through err_close_channel
to err_out:
err_out:
chls->num_chl = 0;
netdev_err(adapter->netdev, "failed to open %d channels, err=%d\n",
chls->num_chl, ret);
Since chls->num_chl is cleared to 0 right before netdev_err(), won't this
always log "failed to open 0 channels" instead of the number of channels that
failed to open?
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
> new file mode 100644
> index 0000000000000..00ac7eacc0e99
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
> @@ -0,0 +1,556 @@
[ ... ]
> +static void xsc_skb_set_hash(struct xsc_adapter *adapter,
> + struct xsc_cqe *cqe,
> + struct sk_buff *skb)
> +{
> + struct xsc_rss_params *rss = &adapter->rss_param;
> + bool l3_hash = false;
> + bool l4_hash = false;
> + u32 hash_field;
> + int ht = 0;
> +
> + if (adapter->netdev->features & NETIF_F_RXHASH) {
> + if (skb->protocol == htons(ETH_P_IP)) {
> + hash_field = rss->rx_hash_fields[XSC_TT_IPV4_TCP];
> + if (hash_field & XSC_HASH_FIELD_SEL_SRC_IP ||
> + hash_field & XSC_HASH_FIELD_SEL_DST_IP)
> + l3_hash = true;
> +
> + if (hash_field & XSC_HASH_FIELD_SEL_SPORT ||
> + hash_field & XSC_HASH_FIELD_SEL_DPORT)
> + l4_hash = true;
> + } else if (skb->protocol == htons(ETH_P_IPV6)) {
> + hash_field = rss->rx_hash_fields[XSC_TT_IPV6_TCP];
> + if (hash_field & XSC_HASH_FIELD_SEL_SRC_IPV6 ||
> + hash_field & XSC_HASH_FIELD_SEL_DST_IPV6)
> + l3_hash = true;
> +
> + if (hash_field & XSC_HASH_FIELD_SEL_SPORT_V6 ||
> + hash_field & XSC_HASH_FIELD_SEL_DPORT_V6)
> + l4_hash = true;
> + }
> +
> + if (l3_hash && l4_hash)
> + ht = PKT_HASH_TYPE_L4;
> + else if (l3_hash)
> + ht = PKT_HASH_TYPE_L3;
> + if (ht)
> + skb_set_hash(skb, le32_to_cpu(cqe->vni), ht);
> + }
> +}
[Severity: Medium]
In xsc_skb_set_hash(), any packet with skb->protocol equal to htons(ETH_P_IP)
looks up rss->rx_hash_fields[XSC_TT_IPV4_TCP], and any packet with
htons(ETH_P_IPV6) looks up rss->rx_hash_fields[XSC_TT_IPV6_TCP].
In xsc_build_rss_param(), those two entries are initialized to
XSC_HASH_IP_PORTS and XSC_HASH_IP6_PORTS, which always have both the L3 IP
address bits and L4 port bits set.
Because the actual L4 protocol and fragmentation status of the received packet
are not checked here, won't every IPv4 and IPv6 packet (including ICMP, other
non-TCP/UDP protocols, and fragmented IP packets) be unconditionally reported
to the network stack with PKT_HASH_TYPE_L4?
> +
> +static void xsc_handle_csum(struct xsc_cqe *cqe, struct xsc_rq *rq,
> + struct sk_buff *skb, struct xsc_wqe_frag_info *wi)
> +{
> + struct xsc_dma_info *dma_info;
> + struct net_device *netdev;
> + void *hw_pph;
> + struct xsc_channel *c;
> + int offset_from;
> +
> + c = rq->cq.channel;
> + netdev = c->adapter->netdev;
> + dma_info = wi->di;
> + offset_from = wi->offset;
> + hw_pph = page_address(dma_info->page) + offset_from;
> +
> + if (unlikely((netdev->features & NETIF_F_RXCSUM) == 0))
> + goto csum_none;
> +
> + if (!(le32_to_cpu(cqe->data0) & XSC_CQE_DATA0_HAS_PPH))
> + goto csum_none;
> +
> + dma_sync_single_for_cpu(rq->cq.xdev->device,
> + dma_info->addr + offset_from,
> + XSC_PPH_HEAD_LEN, DMA_FROM_DEVICE);
> +
> + if (unlikely(XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(hw_pph) & PACKET_UNKNOWN))
> + goto csum_none;
> +
> + if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
> + (!(FIELD_GET(XSC_CQE_DATA0_CSUM_ERR_MASK, le32_to_cpu(cqe->data0)) &
> + OUTER_AND_INNER))) {
[Severity: Medium]
In xsc_handle_csum(), XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) is used to
check tunnel encapsulation on the DMA-synced packet parse header. Looking at
drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h:
#define XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(PPH_BASE_ADDR) \
((*(u16 *)((u8 *)(PPH_BASE_ADDR) + \
EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET) & \
EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK) >> \
EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET)
Since this dereferences a u16 from the DMA buffer in host byte order without
endianness conversion and masks with 0x0f00 >> 8, won't it read the wrong
byte of hw_pph on big-endian architectures and misclassify tunnel checksums?
[ ... ]
> +void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,
> + struct xsc_rq *rq, struct xsc_cqe *cqe)
> +{
> + struct xsc_wq_cyc *wq = &rq->wqe.wq;
> + u8 cqe_opcode = get_cqe_opcode(cqe);
> + struct xsc_wqe_frag_info *wi;
> + struct sk_buff *skb;
> + u32 cqe_bcnt;
> + u16 ci;
> +
> + ci = xsc_wq_cyc_ctr2ix(wq, cqwq->cc);
> + wi = get_frag(rq, ci);
> + if (unlikely(cqe_opcode & BIT(7))) {
> + xsc_dump_error_rqcqe(rq, cqe);
> + goto free_wqe;
> + }
> +
> + cqe_bcnt = le32_to_cpu(cqe->msg_len);
> + if ((le32_to_cpu(cqe->data0) & XSC_CQE_DATA0_HAS_PPH) &&
> + cqe_bcnt <= XSC_PPH_HEAD_LEN)
> + goto free_wqe;
> +
> + if (unlikely(cqe_bcnt < ETH_HLEN + ((le32_to_cpu(cqe->data0) &
> + XSC_CQE_DATA0_HAS_PPH) ?
> + XSC_PPH_HEAD_LEN : 0))) {
> + rq->stats->dropped++;
> + goto free_wqe;
> + }
> +
> + if (unlikely(cqe_bcnt > rq->frags_sz))
> + goto free_wqe;
> +
> + cqe_bcnt = min_t(u32, cqe_bcnt, rq->frags_sz);
[Severity: Medium]
In xsc_eth_handle_rx_cqe(), when XSC_CQE_DATA0_HAS_PPH is set and cqe_bcnt is
less than or equal to XSC_PPH_HEAD_LEN, or when cqe_bcnt > rq->frags_sz, the
frame jumps to free_wqe without incrementing rq->stats->dropped (which
xsc_eth_fold_sw_stats64() folds into rx_length_errors).
In addition, the second check (cqe_bcnt < ETH_HLEN + XSC_PPH_HEAD_LEN) already
covers cqe_bcnt <= XSC_PPH_HEAD_LEN when XSC_CQE_DATA0_HAS_PPH is set, and the
min_t(u32, cqe_bcnt, rq->frags_sz) clamp right after the cqe_bcnt > rq->frags_sz
branch is unreachable.
Should the first check and the redundant min_t() be removed, and should the
cqe_bcnt > rq->frags_sz path also increment rq->stats->dropped?
[ ... ]
> +bool xsc_eth_post_rx_wqes(struct xsc_rq *rq)
> +{
> + struct xsc_wq_cyc *wq = &rq->wqe.wq;
> + u8 wqe_bulk, wqe_bulk_min;
> + int err = 0;
> + int alloc;
> + u16 head;
> +
> + wqe_bulk = rq->wqe.info.wqe_bulk;
> + wqe_bulk_min = rq->wqe.info.wqe_bulk_min;
> + if (xsc_wq_cyc_missing(wq) < wqe_bulk)
> + return false;
> +
> + do {
> + head = xsc_wq_cyc_get_head(wq);
> +
> + alloc = min_t(int, wqe_bulk, xsc_wq_cyc_missing(wq));
> + if (alloc < wqe_bulk && alloc >= wqe_bulk_min)
> + alloc = alloc & 0xfffffffe;
> +
> + if (alloc > 0) {
> + err = xsc_alloc_rx_wqes(rq, head, alloc);
> + if (unlikely(err))
> + break;
> +
> + xsc_wq_cyc_push_n(wq, alloc);
> + }
> + } while (xsc_wq_cyc_missing(wq) >= wqe_bulk_min);
> +
> + dma_wmb();
> +
> + /* ensure wqes are visible to device before updating doorbell record */
> + xsc_rq_notify_hw(rq);
> +
> + return !!err;
> +}
[Severity: Medium]
If xsc_alloc_rx_wqes() fails on the first iteration of the loop in
xsc_eth_post_rx_wqes(), no WQEs are pushed to wq, but execution still falls
through to xsc_rq_notify_hw(rq) and writes the unchanged wq->wqe_ctr to the
hardware RX doorbell.
In addition, during ndo_open, xsc_eth_modify_qps_channel() calls
c->qp.rq.post_wqes(&c->qp.rq) without checking its return value or whether the
RQ ring is still empty before transitioning the RQ to RTS. If initial page
allocation fails, won't the RQ enter RTS with zero posted WQEs and never
receive completions to trigger a refill?
[ ... ]
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
> index 262421740e974..42e6305478913 100644
> --- a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
> +++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
> @@ -118,7 +118,9 @@ int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
> {
> struct xsc_channel *c = container_of(napi, struct xsc_channel, napi);
> struct xsc_eth_params *params = &c->adapter->nic_param;
> + struct xsc_rq *rq = &c->qp.rq;
> bool busy = false;
> + int work_done = 0;
> int tx_budget;
>
> rcu_read_lock();
> @@ -126,18 +128,29 @@ int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
> tx_budget = params->sq_size >> 2;
> busy |= xsc_poll_tx_cq(&c->qp.sq.cq, tx_budget);
>
> + /* budget=0 means: don't poll rx rings */
> + if (likely(budget)) {
> + work_done = xsc_poll_rx_cq(&rq->cq, budget);
> + busy |= work_done == budget;
> + }
> +
> + busy |= rq->post_wqes(rq);
[Severity: High]
Can xsc_eth_post_rx_wqes() race with NAPI or run while the RQ is disabled?
Looking at the channel open sequence in xsc_eth_sw_init():
xsc_eth_open_channels()
for (i = 0; i < chls->num_chl; i++)
napi_enable(&chls->c[i].napi);
smp_wmb();
xsc_eth_modify_qps()
xsc_eth_modify_qps_channel()
c->qp.rq.post_wqes(&c->qp.rq);
xsc_core_eth_modify_qp_status(..., XSC_CMD_OP_RTR2RTS_QP);
xsc_eth_activate_priv_channels()
xsc_eth_activate_channel()
xsc_activate_rq() -> set_bit(XSC_ETH_RQ_STATE_ENABLED, &c->qp.rq.state)
Unlike xsc_poll_rx_cq(), xsc_eth_post_rx_wqes() does not check
test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state), and xsc_eth_napi_poll() calls
rq->post_wqes(rq) unconditionally (even when budget == 0 or when
XSC_ETH_RQ_STATE_ENABLED is not set).
This appears to allow several issues:
1. Because napi_enable() is called before xsc_eth_modify_qps_channel() calls
c->qp.rq.post_wqes(&c->qp.rq) in process context without BH disabled, a
concurrent NAPI poll (via busy-poll, netpoll, or a CQ interrupt) can run
rq->post_wqes(rq) at the same time on the same RQ, corrupting wq->wqe_ctr,
wq->cur_sz, frag->di->page, and the non-thread-safe page_pool alloc cache.
2. While xsc_eth_modify_qps() is iterating over channels across sleeping
firmware commands, earlier channels have already transitioned to RTS while
XSC_ETH_RQ_STATE_ENABLED is still 0, so incoming packets can trigger
xsc_eth_completion_event() while xsc_poll_rx_cq() refuses to drain the CQ.
3. During teardown in xsc_eth_deactivate_channels(), xsc_deactivate_rq()
clears XSC_ETH_RQ_STATE_ENABLED before napi_disable(), but in-flight NAPI
polls will still call rq->post_wqes(rq), allocating pages and ringing the
RX doorbell on the deactivated RQ (and if page allocation fails, returning
busy = true so xsc_eth_napi_poll() never completes NAPI).
Should xsc_eth_post_rx_wqes() check test_bit(XSC_ETH_RQ_STATE_ENABLED,
&rq->state), and should XSC_ETH_RQ_STATE_ENABLED and initial WQE posting happen
before napi_enable() and the RTR2RTS transition?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=9
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 10/15] RDMA/xsc: add RDMA device support
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (8 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 09/15] net/xsc: add ethernet RX path and statistics Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 11/15] RDMA/xsc: add RDMA PD/MR Xin Tian
` (4 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the RDMA function driver, which binds to the xsc_pci.ib
auxiliary device and registers an ib_device backed by the shared
core device.
The ib_device is registered as an IB CA named xscale_%d with the
RoCE port attributes derived from the shared ethernet netdev: port
state and MTU come from the netdev, the link speed is queried
through a new core helper in the PCI driver, the node GUID is
derived from the netdev MAC, and probe returns -EPROBE_DEFER until
the netdev appears. query_device fills the attribute set from
firmware capabilities and, for userspace queries, builds a response
gated on offsetofend() so older userspace libraries keep working.
Expose the uverbs ABI for userspace in include/uapi/rdma/xsc-abi.h:
ucontext allocation returns the doorbell and register window
addresses plus multidb and CQE-size information, and mmap selects
among the TX/RX doorbell, CQ arm and multi-doorbell register pages,
placed with rdma_user_mmap_io so the core pins their lifetime to the
ucontext. RDMA_DRIVER_XSC is appended to the rdma_driver_id enum.
The IOMMU state is classified at probe time, before
ib_register_device(), because the MAD agent can post QP1 traffic the
moment registration returns; the classification decides how the MAD
data path translates DMA addresses back to CPU addresses later.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
drivers/infiniband/hw/xsc/Kconfig | 17 +
drivers/infiniband/hw/xsc/Makefile | 9 +
drivers/infiniband/hw/xsc/main.c | 713 ++++++++++++++++++
drivers/infiniband/hw/xsc/xsc_ib.h | 524 +++++++++++++
.../net/ethernet/yunsilicon/xsc/common/cq.h | 26 +
.../ethernet/yunsilicon/xsc/common/driver.h | 94 +++
.../net/ethernet/yunsilicon/xsc/common/qp.h | 104 +++
.../net/ethernet/yunsilicon/xsc/common/srq.h | 54 ++
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
.../net/ethernet/yunsilicon/xsc/pci/link.c | 34 +
.../net/ethernet/yunsilicon/xsc/pci/pci_irq.c | 5 +-
include/uapi/rdma/ib_user_ioctl_verbs.h | 1 +
include/uapi/rdma/xsc-abi.h | 295 ++++++++
15 files changed, 1876 insertions(+), 4 deletions(-)
create mode 100644 drivers/infiniband/hw/xsc/Kconfig
create mode 100644 drivers/infiniband/hw/xsc/Makefile
create mode 100644 drivers/infiniband/hw/xsc/main.c
create mode 100644 drivers/infiniband/hw/xsc/xsc_ib.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/cq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/driver.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/qp.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/common/srq.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/link.c
create mode 100644 include/uapi/rdma/xsc-abi.h
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 086195758..b3cc506db 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -100,6 +100,7 @@ source "drivers/infiniband/hw/ocrdma/Kconfig"
source "drivers/infiniband/hw/qedr/Kconfig"
source "drivers/infiniband/hw/usnic/Kconfig"
source "drivers/infiniband/hw/vmw_pvrdma/Kconfig"
+source "drivers/infiniband/hw/xsc/Kconfig"
source "drivers/infiniband/sw/rdmavt/Kconfig"
endif # !UML
source "drivers/infiniband/sw/rxe/Kconfig"
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
index c42b22ac3..1bcd2d50a 100644
--- a/drivers/infiniband/hw/Makefile
+++ b/drivers/infiniband/hw/Makefile
@@ -16,3 +16,4 @@ obj-$(CONFIG_INFINIBAND_BNXT_RE) += bnxt_re/
obj-$(CONFIG_INFINIBAND_BNG_RE) += bng_re/
obj-$(CONFIG_INFINIBAND_ERDMA) += erdma/
obj-$(CONFIG_INFINIBAND_IONIC) += ionic/
+obj-$(CONFIG_INFINIBAND_XSC) += xsc/
diff --git a/drivers/infiniband/hw/xsc/Kconfig b/drivers/infiniband/hw/xsc/Kconfig
new file mode 100644
index 000000000..5cf46df2d
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+config INFINIBAND_XSC
+ tristate "Yunsilicon XSC RDMA driver"
+ depends on INFINIBAND && PCI && INET
+ depends on YUNSILICON_XSC_PCI && YUNSILICON_XSC_ETH
+ help
+ This driver provides RDMA support for Yunsilicon XSC devices.
+ It uses the auxiliary device published by the XSC PCI core and,
+ for now, the netdev registered by the XSC ethernet driver to
+ derive the port state and GID. Because of that netdev
+ dependency it also depends on YUNSILICON_XSC_ETH.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_ib.
diff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile
new file mode 100644
index 000000000..6c4ec1ca0
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o
+
+xsc_ib-y := main.o
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
new file mode 100644
index 000000000..322fad38b
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -0,0 +1,713 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/init.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/namei.h>
+#include <linux/kobject.h>
+#include <linux/kernfs.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/io-mapping.h>
+#include <linux/sched.h>
+#include <linux/netdevice.h>
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+#include "common/xsc_cmd.h"
+#include "common/driver.h"
+
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_addr.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_umem.h>
+
+#include <rdma/xsc-abi.h>
+#include "xsc_ib.h"
+
+#define DRIVER_NAME "xsc_ib"
+
+#define XSC_IB_GID_TBL_LEN 256
+#define XSC_IB_MAX_PKEYS 0x80
+
+MODULE_DESCRIPTION("Yunsilicon HCA IB driver");
+MODULE_LICENSE("GPL");
+
+static int xsc_ib_query_device(struct ib_device *ibdev,
+ struct ib_device_attr *props,
+ struct ib_udata *udata)
+{
+ size_t udata_outlen = (udata) ? udata->outlen : 0;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_query_device_resp resp = {};
+ int err = -ENOMEM;
+ size_t resp_len;
+ int max_rq_sg;
+ u64 max_tso;
+ u64 flags;
+
+ err = ib_is_udata_in_empty(udata);
+ if (err)
+ return err;
+
+ resp_len = sizeof(resp.comp_mask) + sizeof(resp.response_length);
+ if (udata_outlen && udata_outlen < resp_len)
+ return -EINVAL;
+
+ resp.response_length = resp_len;
+
+ props->fw_ver = (u64)dev->xdev->fw_version_major |
+ ((u64)dev->xdev->fw_version_minor << 8) |
+ ((u64)dev->xdev->fw_version_patch << 16) |
+ ((u64)dev->xdev->fw_version_tweak << 32);
+
+ props->device_cap_flags = IB_DEVICE_CHANGE_PHY_PORT |
+ IB_DEVICE_PORT_ACTIVE_EVENT |
+ IB_DEVICE_SYS_IMAGE_GUID |
+ IB_DEVICE_RC_RNR_NAK_GEN;
+ props->kernel_cap_flags = IBK_BLOCK_MULTICAST_LOOPBACK;
+ props->kernel_cap_flags |= IBK_LOCAL_DMA_LKEY;
+
+ flags = dev->xdev->caps.flags;
+ if (flags & XSC_DEV_CAP_FLAG_BAD_PKEY_CNTR)
+ props->device_cap_flags |= IB_DEVICE_BAD_PKEY_CNTR;
+ if (flags & XSC_DEV_CAP_FLAG_BAD_QKEY_CNTR)
+ props->device_cap_flags |= IB_DEVICE_BAD_QKEY_CNTR;
+ if (flags & XSC_DEV_CAP_FLAG_APM)
+ props->device_cap_flags |= IB_DEVICE_AUTO_PATH_MIG;
+ props->page_size_cap = dev->xdev->caps.min_page_sz;
+ props->max_mr_size = xsc_get_max_mr_size(dev->xdev);
+ props->max_qp = dev->xdev->caps.max_qp;
+ props->max_qp_wr = xsc_get_max_qp_depth(dev->xdev);
+ max_rq_sg = dev->xdev->caps.max_rq_desc_sz / sizeof(struct xsc_wqe_data_seg);
+
+ props->max_send_sge = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM -
+ XSC_RADDR_SEG_NUM;
+ props->max_recv_sge = dev->xdev->caps.recv_ds_num;
+ props->max_sge_rd = 1;
+ props->max_cq = dev->xdev->caps.max_cq;
+ props->max_cqe = dev->xdev->caps.max_cqes;
+ props->max_mr = (1 << dev->xdev->caps.log_max_mkey) - 1;
+ props->max_pd = dev->xdev->caps.max_pd;
+ props->max_qp_rd_atom = dev->xdev->caps.max_ra_res_qp;
+ props->max_qp_init_rd_atom = dev->xdev->caps.max_ra_req_qp;
+ props->max_res_rd_atom = props->max_qp_rd_atom * props->max_qp;
+ props->max_srq =
+ dev->xdev->caps.log_max_srq ? (1 << dev->xdev->caps.log_max_srq) : 0;
+ props->max_srq_wr = dev->xdev->caps.max_srq_wqes - 1;
+ props->max_srq_sge = dev->xdev->caps.log_max_srq ? (max_rq_sg - 1) : 0;
+ props->local_ca_ack_delay = dev->xdev->caps.local_ca_ack_delay;
+ props->atomic_cap = is_support_rdma_atomic(dev->xdev) ?
+ IB_ATOMIC_HCA : IB_ATOMIC_NONE;
+ props->masked_atomic_cap = IB_ATOMIC_HCA;
+ props->max_mcast_grp =
+ dev->xdev->caps.log_max_mcg ? (1 << dev->xdev->caps.log_max_mcg) : 0;
+ props->max_mcast_qp_attach = dev->xdev->caps.max_qp_mcg;
+ props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
+ props->max_mcast_grp;
+
+ props->sys_image_guid = dev->xdev->board_info->guid;
+ props->vendor_id = dev->xdev->pdev->vendor;
+ props->vendor_part_id = dev->xdev->pdev->device;
+ props->hw_ver = ((dev->xdev->chip_ver_l & 0xffff) << 16) |
+ (dev->xdev->hotfix_num & 0xffff);
+ props->max_pkeys = XSC_IB_MAX_PKEYS;
+ props->max_wq_type_rq = dev->xdev->caps.max_qp;
+ props->hca_core_clock = dev->xdev->caps.hca_core_clock * 1000;
+ props->rss_caps.max_rwq_indirection_tables =
+ dev->xdev->caps.max_rwq_indirection_tables;
+ props->rss_caps.max_rwq_indirection_table_size =
+ dev->xdev->caps.max_rwq_indirection_table_size;
+ props->rss_caps.supported_qpts = 1 << IB_QPT_RAW_PACKET;
+
+ if (offsetofend(typeof(resp), tso_caps) <= udata_outlen) {
+ max_tso = dev->xdev->caps.log_max_tso ? (1 << dev->xdev->caps.log_max_tso) : 0;
+ if (max_tso) {
+ resp.tso_caps.max_tso = max_tso;
+ resp.tso_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
+ resp.response_length += sizeof(resp.tso_caps);
+ }
+ }
+
+ if (offsetofend(typeof(resp), rss_caps) <= udata_outlen) {
+ resp.rss_caps.rx_hash_function = XSC_RX_HASH_FUNC_TOEPLITZ;
+ resp.rss_caps.rx_hash_fields_mask =
+ XSC_RX_HASH_SRC_IPV4 |
+ XSC_RX_HASH_DST_IPV4 |
+ XSC_RX_HASH_SRC_IPV6 |
+ XSC_RX_HASH_DST_IPV6 |
+ XSC_RX_HASH_SRC_PORT_TCP |
+ XSC_RX_HASH_DST_PORT_TCP |
+ XSC_RX_HASH_SRC_PORT_UDP |
+ XSC_RX_HASH_DST_PORT_UDP |
+ XSC_RX_HASH_INNER;
+ resp.response_length += sizeof(resp.rss_caps);
+ }
+
+ if (offsetofend(typeof(resp), packet_pacing_caps) <= udata_outlen) {
+ resp.packet_pacing_caps.qp_rate_limit_max =
+ dev->xdev->caps.qp_rate_limit_max;
+ resp.packet_pacing_caps.qp_rate_limit_min =
+ dev->xdev->caps.qp_rate_limit_min;
+ resp.packet_pacing_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
+
+ resp.response_length += sizeof(resp.packet_pacing_caps);
+ }
+
+ if (udata_outlen) {
+ err = ib_copy_to_udata(udata, &resp, resp.response_length);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "copy response info to udata fail,err=%d\n", err);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static void xsc_calc_link_info(struct xsc_core_device *xdev,
+ struct ib_port_attr *props)
+{
+ switch (xsc_get_link_speed(xdev)) {
+ case XSC_MODULE_SPEED_10G:
+ props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
+ props->active_width = 1;
+ break;
+ case XSC_MODULE_SPEED_25G:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 1;
+ break;
+ case XSC_MODULE_SPEED_40G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
+ props->active_width = 2;
+ break;
+ case XSC_MODULE_SPEED_50G_R:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 1;
+ break;
+ case XSC_MODULE_SPEED_50G_R2:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 1;
+ break;
+ case XSC_MODULE_SPEED_100G_R2:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 2;
+ break;
+ case XSC_MODULE_SPEED_100G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 2;
+ break;
+ case XSC_MODULE_SPEED_200G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 2;
+ break;
+ case XSC_MODULE_SPEED_200G_R8:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 4;
+ break;
+ case XSC_MODULE_SPEED_400G_R8:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 4;
+ break;
+ case XSC_MODULE_SPEED_400G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_100GB;
+ props->active_width = 2;
+ break;
+ default:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 1;
+ break;
+ }
+}
+
+static enum rdma_link_layer xsc_ib_port_link_layer(struct ib_device *ibdev, u32 port)
+{
+ return IB_LINK_LAYER_ETHERNET;
+}
+
+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,
+ struct ib_port_attr *props)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_core_device *xdev = dev->xdev;
+ struct net_device *ndev;
+
+ if (port < 1 || port > xdev->caps.num_ports) {
+ ibdev_warn(&dev->ib_dev, "invalid port number %d\n", port);
+ return -EINVAL;
+ }
+
+ ndev = ib_device_get_netdev(ibdev, port);
+ if (!ndev)
+ return -ENODEV;
+
+ memset(props, 0, sizeof(*props));
+
+ props->state = ib_get_curr_port_state(ndev);
+
+ props->active_mtu = iboe_get_mtu(ndev->mtu);
+ props->gid_tbl_len = XSC_IB_GID_TBL_LEN;
+ props->port_cap_flags = IB_PORT_CM_SUP | IB_PORT_OTHER_LOCAL_CHANGES_SUP;
+ props->max_msg_sz = SZ_1G;
+ props->bad_pkey_cntr = 0;
+ props->qkey_viol_cntr = 0;
+ props->pkey_tbl_len = 1;
+ props->lid = 0;
+ props->sm_lid = 0;
+ props->lmc = 0;
+ props->max_vl_num = 0;
+ props->sm_sl = 0;
+ props->subnet_timeout = 0;
+ props->init_type_reply = 0;
+ if (!is_support_rdma(xdev)) {
+ props->active_width = 1;
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ } else {
+ xsc_calc_link_info(xdev, props);
+ }
+
+ props->phys_state = netif_carrier_ok(ndev) ? XSC_RDMA_PHY_STATE_LINK_UP :
+ XSC_RDMA_PHY_STATE_DISABLED;
+ dev_put(ndev);
+ return 0;
+}
+
+static int xsc_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index,
+ u16 *pkey)
+{
+ *pkey = 0xffff;
+ return 0;
+}
+
+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx,
+ struct ib_udata *udata)
+{
+ struct ib_device *ibdev = uctx->device;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_alloc_ucontext_req req = {};
+ struct xsc_ib_alloc_ucontext_resp resp;
+ int err;
+
+ if (!dev->ib_active)
+ return -EAGAIN;
+
+ err = ib_copy_from_udata(&req, udata, min(udata->inlen, sizeof(req)));
+ if (err)
+ return err;
+
+ memset(&resp, 0, sizeof(resp));
+ resp.qp_tab_size = dev->xdev->caps.max_qp;
+ resp.cache_line_size = L1_CACHE_BYTES;
+ resp.max_sq_desc_sz = dev->xdev->caps.max_sq_desc_sz;
+ resp.max_rq_desc_sz = dev->xdev->caps.max_rq_desc_sz;
+ resp.max_send_wqebb = dev->xdev->caps.max_wqes;
+ resp.max_recv_wr = dev->xdev->caps.max_wqes;
+ xsc_get_db_addr(dev->xdev, &resp.qpm_tx_db, &resp.qpm_rx_db,
+ &resp.cqm_armdb, &resp.cqm_next_cid_reg, NULL);
+ resp.send_ds_num = dev->xdev->caps.send_ds_num;
+ resp.recv_ds_num = dev->xdev->caps.recv_ds_num;
+ resp.cmds_supp_uhw |= XSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE;
+ resp.device_id = dev->xdev->pdev->device;
+
+ if (req.abi_ver >= ALLOC_UCONTEXT_VER_V1) {
+ xsc_get_multidb_info(dev->xdev, &resp.multidb_num, &resp.tx_multidb_base);
+ resp.rdma_proto_mode = dev->xdev->rdma_proto_mode;
+ }
+
+ if (is_support_cqe64(dev->xdev))
+ resp.hw_feature_flag |= XSC_HW_FEATURE_FLAG_SUPPORT_CQE64;
+
+ resp.num_ports = dev->xdev->caps.num_ports;
+
+ return ib_copy_to_udata(udata, &resp, min_t(int, udata->outlen, sizeof(resp)));
+}
+
+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext)
+{
+}
+
+static int xsc_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibcontext->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
+ resource_size_t reg_base;
+ resource_size_t reg_size = PAGE_SIZE;
+ resource_size_t bar_end;
+ bool db_arm = false;
+ u64 addr;
+ u64 tx_db = 0;
+ u64 rx_db = 0;
+ u64 cq_db = 0;
+ u64 cq_reg = 0;
+
+ xsc_get_db_addr(xdev, &tx_db, &rx_db, &cq_db, &cq_reg, NULL);
+ dev_dbg(&xdev->pdev->dev, "offset:0x%lx\n", offset);
+
+ if (offset == (tx_db & PAGE_MASK)) {
+ addr = tx_db;
+ db_arm = true;
+ } else if (offset == (rx_db & PAGE_MASK)) {
+ addr = rx_db;
+ db_arm = true;
+ } else if (offset == (cq_reg & PAGE_MASK)) {
+ addr = cq_reg;
+ db_arm = true;
+ } else if (offset == (cq_db & PAGE_MASK)) {
+ addr = cq_db;
+ db_arm = true;
+ } else if (offset == (QPM_TX_MDB_BASE_REG_ADDR & PAGE_MASK)) {
+ addr = QPM_TX_MDB_BASE_REG_ADDR;
+ } else {
+ return -EINVAL;
+ }
+
+ /*
+ * On an uncompressed PF the doorbell addresses are hardware
+ * addresses (BAR offset + 0xa0000000 window base); the mdb values
+ * are BAR offsets already.
+ */
+ if (db_arm && xsc_core_is_pf(xdev) && !is_pf_bar_compressed(xdev))
+ addr -= 0xA0000000;
+
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (addr & PAGE_MASK);
+ bar_end = pci_resource_start(xdev->pdev, xdev->bar_num) +
+ pci_resource_len(xdev->pdev, xdev->bar_num);
+ dev_dbg(&xdev->pdev->dev, "regbase:0x%llx", reg_base);
+
+ if (vma->vm_end - vma->vm_start != reg_size)
+ return -EINVAL;
+
+ if (reg_base + reg_size > bar_end)
+ return -EINVAL;
+
+ return rdma_user_mmap_io(ibcontext, vma, reg_base >> PAGE_SHIFT,
+ reg_size, pgprot_noncached(vma->vm_page_prot),
+ NULL);
+}
+
+static int xsc_port_immutable(struct ib_device *ibdev, u32 port_num,
+ struct ib_port_immutable *immutable)
+{
+ struct ib_port_attr attr;
+ int err;
+
+ err = ib_query_port(ibdev, port_num, &attr);
+ if (err)
+ return err;
+
+ immutable->pkey_tbl_len = attr.pkey_tbl_len;
+ immutable->gid_tbl_len = attr.gid_tbl_len;
+ immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE |
+ RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+ immutable->max_mad_size = IB_MGMT_MAD_SIZE * 2;
+
+ return 0;
+}
+
+static void xsc_get_guid(const u8 *dev_addr, u8 *guid)
+{
+ u8 mac[ETH_ALEN];
+
+ /* MAC-48 to EUI-64 mapping */
+ memcpy(mac, dev_addr, ETH_ALEN);
+ guid[0] = mac[0] ^ 2;
+ guid[1] = mac[1];
+ guid[2] = mac[2];
+ guid[3] = 0xff;
+ guid[4] = 0xfe;
+ guid[5] = mac[3];
+ guid[6] = mac[4];
+ guid[7] = mac[5];
+}
+
+static int init_node_data(struct xsc_ib_dev *dev)
+{
+ struct net_device *ndev = (struct net_device *)dev->xdev->netdev;
+
+ if (!ndev)
+ return -EPROBE_DEFER;
+
+ strscpy(dev->ib_dev.node_desc, "xsc_node_desc",
+ sizeof(dev->ib_dev.node_desc));
+ xsc_get_guid(ndev->dev_addr, (u8 *)&dev->ib_dev.node_guid);
+
+ return 0;
+}
+
+static void xsc_ib_get_dev_fw_str(struct ib_device *ibdev, char *str)
+{
+ struct xsc_core_device *dev = to_xdev(ibdev)->xdev;
+ u8 ver_major = dev->fw_version_major;
+ u8 ver_minor = dev->fw_version_minor;
+ u16 ver_patch = dev->fw_version_patch;
+ u32 ver_tweak = dev->fw_version_tweak;
+
+ if (ver_tweak == 0) {
+ snprintf(str, IB_FW_VERSION_NAME_MAX, "v%u.%u.%u",
+ ver_major, ver_minor, ver_patch);
+ } else {
+ snprintf(str, IB_FW_VERSION_NAME_MAX, "v%u.%u.%u+%u",
+ ver_major, ver_minor, ver_patch, ver_tweak);
+ }
+}
+
+static void xsc_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
+{
+}
+
+static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
+{
+ dev->ib_dev.ops.owner = THIS_MODULE;
+ dev->ib_dev.ops.uverbs_abi_ver = XSC_IB_UVERBS_ABI_VERSION;
+ dev->ib_dev.ops.driver_id = RDMA_DRIVER_XSC;
+ dev->ib_dev.ops.disassociate_ucontext = xsc_ib_disassociate_ucontext;
+ dev->ib_dev.ops.query_device = xsc_ib_query_device;
+ dev->ib_dev.ops.query_port = xsc_ib_query_port;
+ dev->ib_dev.ops.query_pkey = xsc_ib_query_pkey;
+
+ dev->ib_dev.ops.alloc_ucontext = xsc_ib_alloc_ucontext;
+ dev->ib_dev.ops.dealloc_ucontext = xsc_ib_dealloc_ucontext;
+ dev->ib_dev.ops.mmap = xsc_ib_mmap;
+
+
+ dev->ib_dev.ops.get_link_layer = xsc_ib_port_link_layer;
+
+
+ dev->ib_dev.ops.get_port_immutable = xsc_port_immutable;
+
+ dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
+
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ucontext, xsc_ib_ucontext, ibucontext);
+}
+
+/*
+ * Classify how DMA addresses relate to physical addresses, so that the MAD
+ * path knows whether it can recover a kernel virtual address from one. See
+ * the XSC_IB_IOMMU_MAP_* comment in xsc_ib.h for what each state means and
+ * which are usable.
+ *
+ * A device whose IOMMU group carries any domain other than a DMA-API one is
+ * reported as UNKNOWN: such a domain is either not usable with the DMA API
+ * at all or translates addresses the driver cannot resolve, and neither is
+ * something the MAD path can work with.
+ *
+ * Runs before ib_register_device(), so it takes the DMA-capable device
+ * directly: ib_device.dma_device is only assigned inside registration, and
+ * the MAD client may post traffic as soon as registration returns.
+ */
+static void init_iommu_state(struct xsc_ib_dev *dev, struct device *dma_dev)
+{
+ struct iommu_domain *domain;
+
+ dev->iommu_state = XSC_IB_IOMMU_MAP_DISABLE;
+ domain = iommu_get_domain_for_dev(dma_dev);
+ if (domain) {
+ if (domain->type & __IOMMU_DOMAIN_DMA_API)
+ dev->iommu_state = XSC_IB_IOMMU_MAP_NORMAL;
+ else
+ dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
+ } else {
+ /*
+ * No domain, so the DMA address should be the physical
+ * address. Confirm with a streaming map of linear-map
+ * memory: kmalloc memory is always in the direct map, so
+ * virt_to_phys() on it is unconditionally valid, and this
+ * is the same mapping operation the MAD path performs on
+ * its buffers. If the address is translated anyway
+ * (software bounce buffering, or a hypervisor doing the
+ * translation), the driver cannot recover the virtual
+ * address.
+ */
+ dma_addr_t dma_addr;
+ void *tmp;
+
+ tmp = kmalloc(PAGE_SIZE, GFP_KERNEL);
+ if (!tmp) {
+ /* Cannot even allocate; assume the worst. */
+ dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
+ } else {
+ dma_addr = dma_map_single(dma_dev, tmp,
+ PAGE_SIZE, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dma_dev, dma_addr)) {
+ dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
+ } else {
+ if (virt_to_phys(tmp) != dma_addr)
+ dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
+ dma_unmap_single(dma_dev, dma_addr,
+ PAGE_SIZE, DMA_BIDIRECTIONAL);
+ }
+ kfree(tmp);
+ }
+ }
+
+ ibdev_dbg(&dev->ib_dev, "iommu dma map state=%d\n", dev->iommu_state);
+}
+
+static int init_one(struct xsc_core_device *xdev,
+ struct xsc_ib_dev **ibdev)
+{
+ struct xsc_ib_dev *dev;
+ int err;
+
+ dev = (struct xsc_ib_dev *)ib_alloc_device(xsc_ib_dev, ib_dev);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->xdev = xdev;
+ strscpy(dev->ib_dev.name, "xscale_%d", IB_DEVICE_NAME_MAX);
+
+ memcpy(dev->xdev->priv.ibdev_name, dev->ib_dev.name, IB_DEVICE_NAME_MAX);
+
+ dev->num_comp_vectors = xdev->dev_res->eq_table.num_comp_vectors;
+
+ dev->ib_dev.node_type = RDMA_NODE_IB_CA;
+ dev->ib_dev.local_dma_lkey = 0xFF;
+ dev->num_ports = xdev->caps.num_ports;
+ dev->ib_dev.phys_port_cnt = dev->num_ports;
+ dev->ib_dev.num_comp_vectors = dev->num_comp_vectors;
+ dev->ib_dev.dev.parent = &xdev->pdev->dev;
+ err = ib_device_set_netdev(&dev->ib_dev, xdev->netdev, 1);
+ if (err)
+ goto err_free;
+
+ xsc_ib_dev_setting(dev);
+ dev->xdev->priv.cm_dscp = XSC_DSCP_PCP_UNSET;
+ dev->xdev->priv.cm_pcp = XSC_DSCP_PCP_UNSET;
+ dev->xdev->priv.force_pcp = XSC_DSCP_PCP_UNSET;
+ dev->xdev->priv.force_dscp = XSC_DSCP_PCP_UNSET;
+ dev->xdev->rdma_proto_mode = RDMA_PROTO_ROCEV2;
+ dev->xdev->cm_udp_dst_port = ROCE_V2_UDP_DPORT;
+ dev->xdev->veroce_udp_dst_port = VEROCE_UDP_DPORT;
+
+ dev->ib_dev.uverbs_cmd_mask =
+ (1ull << IB_USER_VERBS_CMD_GET_CONTEXT) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_PORT) |
+ (1ull << IB_USER_VERBS_CMD_ALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_DEALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_AH) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_AH) |
+ (1ull << IB_USER_VERBS_CMD_REG_MR) |
+ (1ull << IB_USER_VERBS_CMD_REREG_MR) |
+ (1ull << IB_USER_VERBS_CMD_DEREG_MR) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_CQ) |
+ (1ull << IB_USER_VERBS_CMD_RESIZE_CQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_QP) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_QP) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
+ (1ull << IB_USER_VERBS_CMD_ATTACH_MCAST) |
+ (1ull << IB_USER_VERBS_CMD_DETACH_MCAST) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_OPEN_QP);
+
+ err = init_node_data(dev);
+ if (err)
+ goto err_free;
+
+ init_iommu_state(dev, xdev->device);
+ err = ib_register_device(&dev->ib_dev, dev->ib_dev.name,
+ dev->xdev->device);
+ if (err)
+ goto err_free;
+
+ rdma_roce_rescan_device(&dev->ib_dev);
+ dev->ib_active = true;
+ *ibdev = dev;
+
+
+ return 0;
+
+err_free:
+ ib_dealloc_device((struct ib_device *)dev);
+
+ return err;
+}
+
+static void remove_one(struct xsc_core_device *xdev, void *intf_ctx)
+{
+ struct xsc_ib_dev *dev = (struct xsc_ib_dev *)intf_ctx;
+
+ ib_unregister_device(&dev->ib_dev);
+ ib_dealloc_device(&dev->ib_dev);
+}
+
+static int xsc_ib_probe(struct auxiliary_device *adev,
+ const struct auxiliary_device_id *adev_id)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_ib_dev *ibdev = NULL;
+ int ret;
+
+ ret = init_one(xdev, &ibdev);
+ if (ret) {
+ dev_err(&adev->dev, "failed to add RDMA device, ret = %d\n",
+ ret);
+ return ret;
+ }
+
+ auxiliary_set_drvdata(adev, ibdev);
+
+ return 0;
+}
+
+static void xsc_ib_remove(struct auxiliary_device *adev)
+{
+ struct xsc_ib_dev *ibdev = auxiliary_get_drvdata(adev);
+
+ if (!ibdev)
+ return;
+
+ remove_one(ibdev->xdev, ibdev);
+}
+
+static const struct auxiliary_device_id xsc_ib_id_table[] = {
+ { .name = XSC_PCI_DRV_NAME "." XSC_IB_ADEV_NAME },
+ {},
+};
+MODULE_DEVICE_TABLE(auxiliary, xsc_ib_id_table);
+
+static struct auxiliary_driver xsc_ib_driver = {
+ .name = XSC_IB_ADEV_NAME,
+ .probe = xsc_ib_probe,
+ .remove = xsc_ib_remove,
+ .id_table = xsc_ib_id_table,
+};
+
+void xsc_remove_rdma_driver(void)
+{
+ auxiliary_driver_unregister(&xsc_ib_driver);
+}
+
+static int __init xsc_ib_init(void)
+{
+ int ret;
+
+ ret = auxiliary_driver_register(&xsc_ib_driver);
+ if (ret) {
+ pr_err("failed to register auxiliary driver\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void __exit xsc_ib_cleanup(void)
+{
+ xsc_remove_rdma_driver();
+}
+
+module_init(xsc_ib_init);
+module_exit(xsc_ib_cleanup);
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
new file mode 100644
index 000000000..b91d4aa37
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -0,0 +1,524 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_IB_H
+#define XSC_IB_H
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_pack.h>
+#include <rdma/ib_mad.h>
+#include "common/xsc_core.h"
+#include "common/driver.h"
+#include "common/cq.h"
+#include "common/srq.h"
+#include "common/qp.h"
+#include <linux/types.h>
+#include <linux/iommu.h>
+#include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
+#include <crypto/hash.h>
+#include <rdma/uverbs_ioctl.h>
+
+#define XSC_PAGE_SHIFT_4K 12
+#define XSC_PAGE_SHIFT_64K 16
+#define XSC_PAGE_SHIFT_2M 21
+#define XSC_PAGE_SHIFT_1G 30
+#define XSC_PAGE_SZ_4K BIT(XSC_PAGE_SHIFT_4K)
+#define XSC_PAGE_SZ_64K BIT(XSC_PAGE_SHIFT_64K)
+#define XSC_PAGE_SZ_2M BIT(XSC_PAGE_SHIFT_2M)
+#define XSC_PAGE_SZ_1G BIT(XSC_PAGE_SHIFT_1G)
+#define XSC_MR_PAGE_CAP_MASK (XSC_PAGE_SZ_4K | \
+ XSC_PAGE_SZ_64K | \
+ XSC_PAGE_SZ_2M | \
+ XSC_PAGE_SZ_1G)
+
+
+struct xsc_ib_ucontext {
+ struct ib_ucontext ibucontext;
+};
+
+struct xsc_ib_pd {
+ struct ib_pd ibpd;
+ u32 pdn;
+ u32 pa_lkey;
+};
+
+struct wr_list {
+ u16 opcode;
+ u16 next;
+};
+
+struct xsc_ib_wq {
+ u64 *wrid;
+ u32 *wr_data;
+ struct wr_list *w_list;
+ unsigned long *wqe_head;
+
+ /* serialize post to the work queue
+ */
+ spinlock_t lock;
+ int wqe_cnt;
+ int ds_cnt;
+ int max_post;
+ int max_gs;
+ int offset;
+ int wqe_shift;
+ unsigned int head;
+ unsigned int tail;
+ u32 cur_post;
+ u32 last_poll;
+ void *qend;
+ void *hdr_buf;
+ u32 hdr_size;
+ dma_addr_t hdr_dma;
+ int mad_queue_depth;
+ int mad_index;
+ u32 *wr_opcode;
+ u32 *need_flush;
+ atomic_t flush_wqe_cnt;
+ int *prev_ce_wqe;
+ int last_comp_ce_wqe;
+ int last_post_ce_wqe;
+};
+
+enum {
+ XSC_QP_USER,
+ XSC_QP_KERNEL,
+};
+
+struct xsc_ib_qp {
+ struct ib_qp ibqp;
+ struct xsc_core_qp xqp;
+ struct xsc_buf buf;
+
+ struct xsc_ib_wq rq;
+
+ u32 doorbell_qpn;
+ u8 sq_signal_bits;
+ struct xsc_ib_wq sq;
+
+ struct ib_umem *umem;
+ int buf_size;
+
+ /* serialize qp state modifications
+ */
+ struct mutex mutex;
+ u32 flags;
+ u8 port;
+ u8 alt_port;
+ u8 atomic_rd_en;
+ u8 resp_depth;
+ u8 state;
+ int max_inline_data;
+ int has_rq;
+
+ int create_type;
+ u32 pa_lkey;
+ /* For QP1 */
+ struct ib_ud_header qp1_hdr;
+ u32 send_psn;
+ struct xsc_qp_context ctx;
+ struct ib_cq *send_cq;
+ struct ib_cq *recv_cq;
+ /* For qp resources */
+ spinlock_t lock;
+
+ struct list_head cq_recv_list;
+ struct list_head cq_send_list;
+ bool post_send_invalid;
+ bool post_recv_invalid;
+ struct ib_recv_wr *srq_head_wr;
+ struct ib_recv_wr *srq_tail_wr;
+ bool has_trig_cq_comp_handle;
+};
+
+struct xsc_ib_cq_buf {
+ struct xsc_buf buf;
+ struct ib_umem *umem;
+ int cqe_size;
+};
+
+enum xsc_ib_qp_flags {
+ XSC_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 0,
+ XSC_IB_QP_SIGNATURE_HANDLING = 1 << 1,
+};
+
+struct xsc_ib_cq {
+ struct ib_cq ibcq;
+ struct xsc_core_cq xcq;
+ struct xsc_ib_cq_buf buf;
+
+ /* serialize access to the CQ
+ */
+ spinlock_t lock;
+
+ /* protect resize cq
+ */
+ struct mutex resize_mutex;
+ struct xsc_ib_cq_resize *resize_buf;
+ struct ib_umem *resize_umem;
+ int cqe_size;
+ struct list_head send_qp_list;
+ struct list_head recv_qp_list;
+ struct list_head err_state_qp_list;
+ u32 create_flags;
+ int invalid;
+};
+
+enum xsc_srq_wr_per_qp {
+ XSC_SRQ_MIN_WR_NUM = 32,
+ XSC_SRQ_MAX_WR_NUM = 128,
+};
+
+enum {
+ XSC_SRQ_USER,
+ XSC_SRQ_KERNEL,
+};
+
+struct xsc_ib_srq {
+ struct ib_srq ibsrq;
+ struct xsc_core_srq xsrq;
+ u64 *wrid;
+ /* protect SRQ hanlding
+ */
+ spinlock_t lock;
+ int head;
+ int tail;
+ struct ib_umem *umem;
+ /* serialize arming a SRQ
+ */
+ struct mutex mutex;
+ int qp_num;
+ int create_type;
+};
+
+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp);
+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id);
+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr);
+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init);
+
+struct xsc_ib_mr {
+ struct ib_mr ibmr;
+ struct xsc_core_mr xmr;
+ struct ib_umem *umem;
+ int npages;
+};
+
+struct xsc_ib_dev {
+ struct ib_device ib_dev;
+ struct xsc_core_device *xdev;
+ int num_ports;
+ int num_comp_vectors;
+ u8 ib_active;
+ int iommu_state;
+};
+
+struct xsc_err_state_qp_node {
+ struct list_head entry;
+ u32 qp_id;
+ bool is_sq;
+};
+
+struct xsc_ib_ah {
+ struct ib_ah ibah;
+ struct xsc_av av;
+};
+
+/*
+ * Conversion helpers.
+ *
+ * The first group goes from a uverbs object to the private structure that
+ * embeds it; the second goes from the core object to the private structure
+ * that embeds that one. Both directions exist for cq, qp and srq, so the
+ * names spell out which object is being converted.
+ */
+static inline struct xsc_ib_dev *to_xdev(struct ib_device *ibdev)
+{
+ return container_of(ibdev, struct xsc_ib_dev, ib_dev);
+}
+
+static inline struct xsc_ib_pd *to_xpd(struct ib_pd *ibpd)
+{
+ return container_of(ibpd, struct xsc_ib_pd, ibpd);
+}
+
+static inline struct xsc_ib_mr *to_xmr(struct ib_mr *ibmr)
+{
+ return container_of(ibmr, struct xsc_ib_mr, ibmr);
+}
+
+static inline struct xsc_ib_ah *to_xah(struct ib_ah *ibah)
+{
+ return container_of(ibah, struct xsc_ib_ah, ibah);
+}
+
+static inline struct xsc_ib_cq *to_xcq(struct ib_cq *ibcq)
+{
+ return container_of(ibcq, struct xsc_ib_cq, ibcq);
+}
+
+static inline struct xsc_ib_qp *to_xqp(struct ib_qp *ibqp)
+{
+ return container_of(ibqp, struct xsc_ib_qp, ibqp);
+}
+
+static inline struct xsc_ib_srq *to_xsrq(struct ib_srq *ibsrq)
+{
+ return container_of(ibsrq, struct xsc_ib_srq, ibsrq);
+}
+
+static inline struct xsc_ib_cq *core_to_xcq(struct xsc_core_cq *xcq)
+{
+ return container_of(xcq, struct xsc_ib_cq, xcq);
+}
+
+static inline struct xsc_ib_qp *core_to_xqp(struct xsc_core_qp *xqp)
+{
+ return container_of(xqp, struct xsc_ib_qp, xqp);
+}
+
+static inline struct xsc_ib_srq *core_to_xsrq(struct xsc_core_srq *xsrq)
+{
+ return container_of(xsrq, struct xsc_ib_srq, xsrq);
+}
+
+int xsc_ib_create_ah(struct ib_ah *ibah,
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata);
+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags);
+int xsc_ib_create_cq(struct ib_cq *ibcq,
+ const struct ib_cq_init_attr *attr,
+ struct uverbs_attr_bundle *attrs);
+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata);
+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata);
+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);
+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata);
+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata);
+
+int xsc_ib_create_srq(struct ib_srq *ib_srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata);
+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata);
+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr);
+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr);
+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata);
+
+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,
+ struct ib_port_attr *props);
+
+int xsc_ib_create_qp(struct ib_qp *ibqp,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata);
+
+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
+
+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);
+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ enum xsc_qp_state cur_state, enum xsc_qp_state state);
+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata);
+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
+ struct ib_qp_init_attr *qp_init_attr);
+
+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
+ const struct ib_send_wr **bad_wr);
+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr);
+
+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n);
+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc);
+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
+ u64 virt_addr, int access_flags,
+ struct ib_dmah *dmah,
+ struct ib_udata *udata);
+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset);
+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr, int *count, int *shift,
+ int *ncont);
+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, __be64 *pas, int npages, bool need_to_devide);
+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
+ u64 length, u64 virt_addr,
+ int fd, int access_flags,
+ struct ib_dmah *dmah,
+ struct uverbs_attr_bundle *attrs);
+
+void xsc_ib_drain_rq(struct ib_qp *qp);
+void xsc_ib_drain_sq(struct ib_qp *qp);
+
+static inline u8 convert_access(int acc)
+{
+ return (acc & IB_ACCESS_REMOTE_ATOMIC ? XSC_PERM_ATOMIC : 0) |
+ (acc & IB_ACCESS_REMOTE_WRITE ? XSC_PERM_REMOTE_WRITE : 0) |
+ (acc & IB_ACCESS_REMOTE_READ ? XSC_PERM_REMOTE_READ : 0) |
+ (acc & IB_ACCESS_LOCAL_WRITE ? XSC_PERM_LOCAL_WRITE : 0) |
+ XSC_PERM_LOCAL_READ;
+}
+
+#define IB_MTU_8192 6
+
+static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)
+{
+ mtu = mtu - (IB_GRH_BYTES + IB_UDP_BYTES + IB_BTH_BYTES +
+ IB_EXT_XRC_BYTES + IB_EXT_ATOMICETH_BYTES +
+ IB_ICRC_BYTES);
+ if (mtu >= 8192)
+ return IB_MTU_8192;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_4096))
+ return IB_MTU_4096;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_1024))
+ return IB_MTU_1024;
+ else
+ return 0;
+}
+
+#define XSC_IB_IOMMU_MAP_DISABLE 0
+#define XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN 1
+#define XSC_IB_IOMMU_MAP_NORMAL 2
+
+static inline int xsc_ib_iommu_dma_map(struct ib_device *ibdev)
+{
+ return to_xdev(ibdev)->iommu_state;
+}
+
+static inline void *xsc_ib_iova_to_virt(struct ib_device *ibdev, dma_addr_t iova)
+{
+ phys_addr_t phyaddr;
+ struct iommu_domain *domain;
+
+ domain = iommu_get_domain_for_dev(ibdev->dma_device);
+ if (likely(domain)) {
+ phyaddr = iommu_iova_to_phys(domain, iova);
+ phyaddr |= iova & (PAGE_SIZE - 1);
+ } else {
+ phyaddr = dma_to_phys(ibdev->dma_device, iova);
+ }
+
+ return phys_to_virt(phyaddr);
+}
+
+struct ib_mad_list_head {
+ struct list_head list;
+ struct ib_cqe cqe;
+ struct ib_mad_queue *mad_queue;
+};
+
+#define IB_MAD_SEND_REQ_MAX_SG 2
+struct ib_mad_send_wr_private {
+ struct ib_mad_list_head mad_list;
+ struct list_head agent_list;
+ struct ib_mad_agent_private *mad_agent_priv;
+ struct ib_mad_send_buf send_buf;
+ u64 header_mapping;
+ u64 payload_mapping;
+ struct ib_ud_wr send_wr;
+ struct ib_sge sg_list[IB_MAD_SEND_REQ_MAX_SG];
+ __be64 tid;
+ unsigned long timeout;
+ int max_retries;
+ int retries_left;
+ int retry;
+ int refcount;
+ enum ib_wc_status status;
+
+ /* RMPP control */
+ struct list_head rmpp_list;
+ struct ib_rmpp_segment *last_ack_seg;
+ struct ib_rmpp_segment *cur_seg;
+ int last_ack;
+ int seg_num;
+ int newwin;
+ int pad;
+};
+
+struct ib_mad_private_header {
+ struct ib_mad_list_head mad_list;
+ struct ib_mad_recv_wc recv_wc;
+ struct ib_wc wc;
+ u64 mapping;
+} __packed;
+
+struct ib_mad_private {
+ struct ib_mad_private_header header;
+ size_t mad_size;
+ struct ib_grh grh;
+ DECLARE_FLEX_ARRAY(u8, mad);
+} __packed;
+
+static inline void *xsc_ib_send_mad_sg_virt_addr(struct ib_device *ibdev,
+ const struct ib_send_wr *wr,
+ int sg)
+{
+ struct ib_mad_send_wr_private *mad_send_wr;
+ struct ib_mad_list_head *mad_list;
+ int iommu_state = xsc_ib_iommu_dma_map(ibdev);
+
+ /* direct dma mapping */
+ if (!iommu_state)
+ return phys_to_virt(dma_to_phys(ibdev->dma_device, wr->sg_list[sg].addr));
+
+ if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
+ return xsc_ib_iova_to_virt(ibdev, wr->sg_list[sg].addr);
+
+ mad_list = container_of(wr->wr_cqe, struct ib_mad_list_head, cqe);
+ mad_send_wr = container_of(mad_list, struct ib_mad_send_wr_private,
+ mad_list);
+
+ /* sg_list[0] */
+ if (sg == 0)
+ return mad_send_wr->send_buf.mad;
+
+ /* sg_list[1] */
+ if (mad_send_wr->send_buf.seg_count)
+ return ib_get_rmpp_segment(&mad_send_wr->send_buf,
+ mad_send_wr->seg_num);
+ return mad_send_wr->send_buf.mad + mad_send_wr->send_buf.hdr_len;
+}
+
+static inline void *xsc_ib_recv_mad_sg_virt_addr(struct ib_device *ibdev,
+ struct ib_wc *wc,
+ u64 sg_addr)
+{
+ struct ib_mad_private_header *mad_priv_hdr;
+ struct ib_mad_private *recv;
+ struct ib_mad_list_head *mad_list;
+ int iommu_state = xsc_ib_iommu_dma_map(ibdev);
+
+ /* direct dma mapping */
+ if (!iommu_state)
+ return phys_to_virt(dma_to_phys(ibdev->dma_device, sg_addr));
+
+ if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
+ return xsc_ib_iova_to_virt(ibdev, sg_addr);
+
+ mad_list = container_of(wc->wr_cqe, struct ib_mad_list_head, cqe);
+ mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, mad_list);
+ recv = container_of(mad_priv_hdr, struct ib_mad_private, header);
+ return &recv->grh;
+}
+
+struct xsc_ib_burst_info {
+ __u32 max_burst_sz;
+ __u16 typical_pkt_sz;
+ __u16 reserved;
+};
+
+struct xsc_ib_modify_qp {
+ __u32 reserved;
+ struct xsc_ib_burst_info burst_info;
+ __u32 profile;
+};
+
+#endif /* XSC_IB_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/cq.h b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h
new file mode 100644
index 000000000..ec2c00435
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_CORE_CQ_H
+#define XSC_CORE_CQ_H
+
+#include "common/xsc_core.h"
+
+enum {
+ CQE_SIZE_64 = 0,
+ CQE_SIZE_128 = 1,
+};
+
+enum {
+ XSC_CQ_DB_REQ_NOT_SOL = 1,
+ XSC_CQ_DB_REQ_NOT = 0,
+};
+
+void xsc_init_cq_table(struct xsc_core_device *xdev);
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+#endif /* XSC_CORE_CQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
new file mode 100644
index 000000000..362099a7c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_DRIVER_H
+#define XSC_DRIVER_H
+
+#include <linux/kernel.h>
+#include <linux/completion.h>
+#include <linux/pci.h>
+#include <linux/spinlock_types.h>
+#include <linux/semaphore.h>
+#include <linux/vmalloc.h>
+#include <linux/radix-tree.h>
+#include <rdma/ib_verbs.h>
+
+#include "common/xsc_device.h"
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+#include "common/srq.h"
+
+struct rxe_bth {
+ u8 opcode;
+ u8 flags;
+ __be16 pkey;
+ __be32 qpn;
+ __be32 apsn;
+};
+
+struct rxe_deth {
+ __be32 qkey;
+ __be32 src_qp;
+};
+
+static inline void *xsc_vzalloc(unsigned long size)
+{
+ void *rtn;
+
+ rtn = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);
+ if (!rtn)
+ rtn = vzalloc(size);
+ return rtn;
+}
+
+static inline void xsc_vfree(const void *addr)
+{
+ if (addr && is_vmalloc_addr(addr))
+ vfree(addr);
+ else
+ kfree(addr);
+}
+
+int xsc_buf_alloc(struct xsc_core_device *xdev, unsigned long size,
+ unsigned long max_direct, struct xsc_buf *buf);
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen);
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn);
+
+#endif /* XSC_DRIVER_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/qp.h b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h
new file mode 100644
index 000000000..af23e6a31
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h
@@ -0,0 +1,104 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_QP_H
+#define XSC_QP_H
+
+#include <linux/radix-tree.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+
+enum {
+ XSC_QP_PM_MIGRATED = 0x3,
+ XSC_QP_PM_ARMED = 0x0,
+ XSC_QP_PM_REARM = 0x1
+};
+
+enum {
+ XSC_WQE_CTRL_CQ_UPDATE = 2 << 2,
+ XSC_WQE_CTRL_SOLICITED = 1 << 1,
+};
+
+struct xsc_av {
+ union {
+ struct {
+ __be32 qkey;
+ __be32 reserved;
+ } qkey;
+ __be64 dc_key;
+ } key;
+ __be32 dqp_dct;
+ u8 stat_rate_sl;
+ u8 fl_mlid;
+ union {
+ __be16 rlid;
+ __be16 udp_sport;
+ };
+ u8 reserved0[4];
+ u8 rmac[6];
+ u8 tclass;
+ u8 hop_limit;
+ __be32 grh_gid_fl;
+ u8 rgid[16];
+};
+
+struct xsc_qp_path {
+ u8 fl;
+ u8 rsvd3;
+ u8 free_ar;
+ u8 pkey_index;
+ u8 rsvd0;
+ u8 grh_mlid;
+ __be16 rlid;
+ u8 ackto_lt;
+ u8 mgid_index;
+ u8 static_rate;
+ u8 hop_limit;
+ __be32 tclass_flowlabel;
+ u8 rgid[16];
+ u8 rsvd1[4];
+ u8 sl;
+ u8 port;
+ u8 rsvd2[6];
+ u8 dmac[6];
+ u8 smac[6];
+ __be16 af_type;
+ __be32 sip[4];
+ __be32 dip[4];
+ __be16 sport;
+ u8 ecn_dscp;
+ u8 vlan_valid;
+ __be16 vlan_id;
+ u8 dci_cfi_prio_sl; /* not left moved yet. */
+};
+
+static inline struct xsc_core_qp *__xsc_qp_lookup(struct xsc_core_device *xdev, u32 qpn)
+{
+ return radix_tree_lookup(&xdev->dev_res->qp_table.tree, qpn);
+}
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_init_qp_table(struct xsc_core_device *xdev);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+
+#endif /* XSC_QP_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/srq.h b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h
new file mode 100644
index 000000000..0a2e660e1
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_SRQ_H
+#define XSC_SRQ_H
+
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#include "common/xsc_core.h"
+
+struct ib_qp;
+struct ib_recv_wr;
+
+#define XSC_MAX_SRQ_NUM 8192
+#define XSC_MAX_SRQ_LOG_SIZE 13
+
+struct xsc_srq_qp_node {
+ struct list_head entry;
+ u32 qp_id;
+ struct ib_qp *ibqp;
+};
+
+struct xsc_srq_cache_wr {
+ int wr_num;
+ int max_wr_num;
+ struct ib_recv_wr *head_wr;
+ struct ib_recv_wr *tail_wr;
+};
+
+struct xsc_core_srq {
+ u32 srqn;
+ int max;
+ size_t max_gs;
+ size_t max_avail_gather;
+ int wqe_shift;
+ void (*event)(struct xsc_core_srq *srq, enum xsc_event e);
+
+ /* protect ctrl list */
+ spinlock_t lock;
+ struct list_head ctrl_list;
+ u16 uid;
+
+ struct list_head srq_qp_list;
+ struct xsc_srq_cache_wr *cache_wr;
+};
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+
+#endif /* XSC_SRQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index b01fca3ac..87481db3f 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o link.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/link.c b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
new file mode 100644
index 000000000..272ddc3a2
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+
+int xsc_get_link_speed(struct xsc_core_device *xdev)
+{
+ struct xsc_event_query_linkinfo_mbox_out out;
+ struct xsc_event_linkinfo linkinfo;
+ struct xsc_inbox_hdr in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_LINK_INFO);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to get link info, err=%d\n", err);
+ return XSC_MODULE_SPEED_25G;
+ }
+
+ linkinfo = out.ctx;
+ linkinfo.linkspeed = be32_to_cpu(linkinfo.linkspeed);
+
+ return linkinfo.linkspeed;
+}
+EXPORT_SYMBOL_GPL(xsc_get_link_speed);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
index d83f13f38..8837ef2db 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
@@ -294,9 +294,8 @@ static void xsc_event_work(struct work_struct *work)
in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_EVENT_TYPE);
err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
- if (err || out.hdr.status) {
- pci_err(xdev->pdev, "failed to query event type, err=%d, status=%d\n",
- err, out.hdr.status);
+ if (err) {
+ pci_err(xdev->pdev, "failed to query event type, err=%d\n", err);
return;
}
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7b..53fc99057 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -257,6 +257,7 @@ enum rdma_driver_id {
RDMA_DRIVER_ERDMA,
RDMA_DRIVER_MANA,
RDMA_DRIVER_IONIC,
+ RDMA_DRIVER_XSC,
};
enum ib_uverbs_gid_type {
diff --git a/include/uapi/rdma/xsc-abi.h b/include/uapi/rdma/xsc-abi.h
new file mode 100644
index 000000000..7e19b23a3
--- /dev/null
+++ b/include/uapi/rdma/xsc-abi.h
@@ -0,0 +1,295 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ABI_H__
+#define __XSC_ABI_H__
+
+#include <linux/types.h>
+#include <linux/if_ether.h> /* For ETH_ALEN. */
+
+/* Increment this value if any changes that break userspace ABI
+ * compatibility are made.
+ */
+#define XSC_IB_UVERBS_ABI_VERSION 1
+
+/* Make sure that all structs defined in this file remain laid out so
+ * that they pack the same way on 32-bit and 64-bit architectures (to
+ * avoid incompatibility between 32-bit userspace and 64-bit kernels).
+ * In particular do not use pointer types -- pass pointers in __u64
+ * instead.
+ */
+
+enum {
+ XSC_QP_FLAG_SIGNATURE = 1 << 0,
+ XSC_QP_FLAG_SCATTER_CQE = 1 << 1,
+ XSC_QP_FLAG_TUNNEL_OFFLOADS = 1 << 2,
+ XSC_QP_FLAG_BFREG_INDEX = 1 << 3,
+ XSC_QP_FLAG_TYPE_DCT = 1 << 4,
+ XSC_QP_FLAG_TYPE_DCI = 1 << 5,
+ XSC_QP_FLAG_TIR_ALLOW_SELF_LB_UC = 1 << 6,
+ XSC_QP_FLAG_TIR_ALLOW_SELF_LB_MC = 1 << 7,
+ XSC_QP_FLAG_ALLOW_SCATTER_CQE = 1 << 8,
+ XSC_QP_FLAG_RAWPACKET_TSO = 1 << 9,
+ XSC_QP_FLAG_RAWPACKET_TX = 1 << 10,
+ XSC_QP_FLAG_RAWPACKET_SNIFFER = 1 << 11,
+ XSC_QP_FLAG_WORC = 1 << 12,
+};
+
+#define ALLOC_UCONTEXT_VER_V1 1
+#define ALLOC_UCONTEXT_VER_V2 2
+
+#define CURRENT_UCONTEXT_ABI_VER ALLOC_UCONTEXT_VER_V2
+
+struct xsc_ib_alloc_ucontext_req {
+ __u32 abi_ver;
+ __u32 rsvd1;
+};
+
+enum xsc_user_cmds_supp_uhw {
+ XSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE = 1 << 0,
+ XSC_USER_CMDS_SUPP_UHW_CREATE_AH = 1 << 1,
+};
+
+enum {
+ XSC_HW_FEATURE_FLAG_SUPPORT_CQE64 = 1 << 0,
+};
+
+struct xsc_ib_alloc_ucontext_resp {
+ __u32 qp_tab_size;
+ __u32 cache_line_size;
+ __u16 max_sq_desc_sz;
+ __u16 max_rq_desc_sz;
+ __u32 max_send_wqebb;
+ __u32 max_recv_wr;
+ __u16 num_ports;
+ __u16 device_id;
+ __u64 qpm_tx_db;
+ __u64 qpm_rx_db;
+ __u64 cqm_next_cid_reg;
+ __u64 cqm_armdb;
+ __u32 send_ds_num;
+ __u32 recv_ds_num;
+ __u32 cmds_supp_uhw;
+ __u32 pad;
+ __u32 multidb_num;
+ __u32 tx_multidb_base;
+ __u32 rdma_proto_mode;
+ __u32 hw_feature_flag;
+ __u32 reserved[32];
+};
+
+struct xsc_ib_create_qp {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 sq_wqe_count;
+ __u32 rq_wqe_count;
+ __u32 rq_wqe_shift;
+ __u32 flags;
+
+ int dmabuf_fd;
+ __aligned_u64 dmabuf_sz;
+};
+
+struct xsc_ib_create_qp_resp {
+ __u32 uuar_index;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_srq {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 flags;
+ __u32 reserved0;
+ __u32 uidx;
+ __u32 reserved1;
+};
+
+struct xsc_ib_create_srq_resp {
+ __u32 srqn;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_cq {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 cqe_size;
+
+ int dmabuf_fd;
+ __aligned_u64 dmabuf_sz;
+};
+
+struct xsc_ib_create_cq_resp {
+ __u32 cqn;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_ah_resp {
+ __u32 response_length;
+ __u8 dmac[ETH_ALEN];
+ __u8 reserved[6];
+};
+
+struct xsc_ib_alloc_pd_resp {
+ __u32 pdn;
+};
+
+struct xsc_ib_tso_caps {
+ __u32 max_tso; /* Maximum tso payload size in bytes */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_UD
+ */
+ __u32 supported_qpts;
+};
+
+/* RX Hash function flags */
+enum xsc_rx_hash_function_flags {
+ XSC_RX_HASH_FUNC_TOEPLITZ = 1 << 0,
+};
+
+enum xsc_rdma_link_speed {
+ XSC_RDMA_LINK_SPEED_2_5GB = 1 << 0,
+ XSC_RDMA_LINK_SPEED_5GB = 1 << 1,
+ XSC_RDMA_LINK_SPEED_10GB = 1 << 3,
+ XSC_RDMA_LINK_SPEED_14GB = 1 << 4,
+ XSC_RDMA_LINK_SPEED_25GB = 1 << 5,
+ XSC_RDMA_LINK_SPEED_50GB = 1 << 6,
+ XSC_RDMA_LINK_SPEED_100GB = 1 << 7,
+};
+
+enum xsc_rdma_phys_state {
+ XSC_RDMA_PHY_STATE_SLEEP = 1,
+ XSC_RDMA_PHY_STATE_POLLING,
+ XSC_RDMA_PHY_STATE_DISABLED,
+ XSC_RDMA_PHY_STATE_PORT_CONFIGURATION_TRAINNING,
+ XSC_RDMA_PHY_STATE_LINK_UP,
+ XSC_RDMA_PHY_STATE_LINK_ERROR_RECOVERY,
+ XSC_RDMA_PHY_STATE_PHY_TEST,
+};
+
+/*
+ * RX Hash flags, these flags allows to set which incoming packet's field should
+ * participates in RX Hash. Each flag represent certain packet's field,
+ * when the flag is set the field that is represented by the flag will
+ * participate in RX Hash calculation.
+ * Note: *IPV4 and *IPV6 flags can't be enabled together on the same QP
+ * and *TCP and *UDP flags can't be enabled together on the same QP.
+ */
+enum xsc_rx_hash_fields {
+ XSC_RX_HASH_SRC_IPV4 = 1 << 0,
+ XSC_RX_HASH_DST_IPV4 = 1 << 1,
+ XSC_RX_HASH_SRC_IPV6 = 1 << 2,
+ XSC_RX_HASH_DST_IPV6 = 1 << 3,
+ XSC_RX_HASH_SRC_PORT_TCP = 1 << 4,
+ XSC_RX_HASH_DST_PORT_TCP = 1 << 5,
+ XSC_RX_HASH_SRC_PORT_UDP = 1 << 6,
+ XSC_RX_HASH_DST_PORT_UDP = 1 << 7,
+ XSC_RX_HASH_IPSEC_SPI = 1 << 8,
+ /* Save bits for future fields */
+ XSC_RX_HASH_INNER = (1UL << 31),
+};
+
+struct xsc_ib_rss_caps {
+ __aligned_u64 rx_hash_fields_mask; /* enum xsc_rx_hash_fields */
+ __u8 rx_hash_function; /* enum xsc_rx_hash_function_flags */
+ __u8 reserved[7];
+};
+
+enum xsc_ib_cqe_comp_res_format {
+ XSC_IB_CQE_RES_FORMAT_HASH = 1 << 0,
+ XSC_IB_CQE_RES_FORMAT_CSUM = 1 << 1,
+ XSC_IB_CQE_RES_FORMAT_CSUM_STRIDX = 1 << 2,
+};
+
+struct xsc_ib_cqe_comp_caps {
+ __u32 max_num;
+ __u32 supported_format; /* enum xsc_ib_cqe_comp_res_format */
+};
+
+enum xsc_ib_packet_pacing_cap_flags {
+ XSC_IB_PP_SUPPORT_BURST = 1 << 0,
+};
+
+struct xsc_packet_pacing_caps {
+ __u32 qp_rate_limit_min;
+ __u32 qp_rate_limit_max; /* In kpbs */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+ __u8 cap_flags; /* enum xsc_ib_packet_pacing_cap_flags */
+ __u8 reserved[3];
+};
+
+enum xsc_ib_mpw_caps {
+ MPW_RESERVED = 1 << 0,
+ XSC_IB_ALLOW_MPW = 1 << 1,
+ XSC_IB_SUPPORT_EMPW = 1 << 2,
+};
+
+enum xsc_ib_sw_parsing_offloads {
+ XSC_IB_SW_PARSING = 1 << 0,
+ XSC_IB_SW_PARSING_CSUM = 1 << 1,
+ XSC_IB_SW_PARSING_LSO = 1 << 2,
+};
+
+struct xsc_ib_sw_parsing_caps {
+ __u32 sw_parsing_offloads; /* enum xsc_ib_sw_parsing_offloads */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+};
+
+struct xsc_ib_striding_rq_caps {
+ __u32 min_single_stride_log_num_of_bytes;
+ __u32 max_single_stride_log_num_of_bytes;
+ __u32 min_single_wqe_log_num_of_strides;
+ __u32 max_single_wqe_log_num_of_strides;
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+ __u32 reserved;
+};
+
+enum xsc_ib_query_dev_resp_flags {
+ /* Support 128B CQE compression */
+ XSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_COMP = 1 << 0,
+ XSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_PAD = 1 << 1,
+};
+
+enum xsc_ib_tunnel_offloads {
+ XSC_IB_TUNNELED_OFFLOADS_VXLAN = 1 << 0,
+ XSC_IB_TUNNELED_OFFLOADS_GRE = 1 << 1,
+ XSC_IB_TUNNELED_OFFLOADS_GENEVE = 1 << 2,
+ XSC_IB_TUNNELED_OFFLOADS_MPLS_GRE = 1 << 3,
+ XSC_IB_TUNNELED_OFFLOADS_MPLS_UDP = 1 << 4,
+};
+
+struct xsc_ib_query_device_resp {
+ __u32 comp_mask;
+ __u32 response_length;
+ struct xsc_ib_tso_caps tso_caps;
+ struct xsc_ib_rss_caps rss_caps;
+ struct xsc_ib_cqe_comp_caps cqe_comp_caps;
+ struct xsc_packet_pacing_caps packet_pacing_caps;
+ __u32 xsc_ib_support_multi_pkt_send_wqes;
+ __u32 flags; /* Use enum xsc_ib_query_dev_resp_flags */
+ struct xsc_ib_sw_parsing_caps sw_parsing_caps;
+ struct xsc_ib_striding_rq_caps striding_rq_caps;
+ __u32 tunnel_offloads_caps; /* enum xsc_ib_tunnel_offloads */
+ __u32 reserved;
+};
+
+#endif /* __XSC_ABI_H__ */
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 10/15] RDMA/xsc: add RDMA device support
2026-11-04 15:57 ` [PATCH v1 10/15] RDMA/xsc: add RDMA device support Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 10 potential issue(s) to consider:
- [Critical] RDMA/xsc: xsc_ib_mmap() exposes privileged BAR0 command queue registers
- [High] RDMA/xsc: unsynchronized netdev attachment and missing NETDEV_UNREGISTER
- [High] RDMA/xsc: xsc_ib_query_device() miscalculates resp.response_length
- [High] RDMA/xsc: IOVA page offset corruption and identity domain misclassification
- [Medium] RDMA/xsc: xsc_ib_query_port() leaves max_mtu as 0 and misencodes link info
- [Medium] RDMA/xsc: init_one() sets ib_active after ib_register_device() and copies unexpanded name
- [Medium] RDMA/xsc: wrong masked_atomic_cap and 32-bit shifts in xsc_ib_query_device()
- [Medium] RDMA/xsc: implicit padding, raw int types, and unvalidated udata in UAPI
- [Low] RDMA/xsc: missing select AUXILIARY_BUS in Kconfig
- [Low] RDMA/xsc: endianness warning in xsc_get_link_speed() and helper duplication
--
commit 78a9c302585dd86ed70444386c757ac8476b121c
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add RDMA device support
This commit introduces the xsc_ib auxiliary RDMA driver, registering an
ib_device backed by the shared xsc core device and netdev, and exposes the
userspace ABI in include/uapi/rdma/xsc-abi.h.
> diff --git a/drivers/infiniband/hw/xsc/Kconfig b/drivers/infiniband/hw/xsc/Kconfig
> new file mode 100644
> index 0000000000000..5cf46df2d4f12
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/Kconfig
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
> +# All rights reserved.
> +
> +config INFINIBAND_XSC
> + tristate "Yunsilicon XSC RDMA driver"
> + depends on INFINIBAND && PCI && INET
> + depends on YUNSILICON_XSC_PCI && YUNSILICON_XSC_ETH
[Severity: Low]
Since xsc_ib_init() and xsc_ib_cleanup() call auxiliary_driver_register()
and auxiliary_driver_unregister(), and neither YUNSILICON_XSC_PCI nor
YUNSILICON_XSC_ETH selects AUXILIARY_BUS, can building with
CONFIG_INFINIBAND_XSC fail at link/modpost time with undefined references
to __auxiliary_driver_register and auxiliary_driver_unregister when no
other enabled driver selects CONFIG_AUXILIARY_BUS? Should INFINIBAND_XSC
(or YUNSILICON_XSC_PCI) select AUXILIARY_BUS?
> diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
> new file mode 100644
> index 0000000000000..322fad38b5df2
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/main.c
[ ... ]
> +static int xsc_ib_query_device(struct ib_device *ibdev,
> + struct ib_device_attr *props,
> + struct ib_udata *udata)
> +{
[ ... ]
> + props->max_cq = dev->xdev->caps.max_cq;
> + props->max_cqe = dev->xdev->caps.max_cqes;
> + props->max_mr = (1 << dev->xdev->caps.log_max_mkey) - 1;
> + props->max_pd = dev->xdev->caps.max_pd;
> + props->max_qp_rd_atom = dev->xdev->caps.max_ra_res_qp;
> + props->max_qp_init_rd_atom = dev->xdev->caps.max_ra_req_qp;
> + props->max_res_rd_atom = props->max_qp_rd_atom * props->max_qp;
> + props->max_srq =
> + dev->xdev->caps.log_max_srq ? (1 << dev->xdev->caps.log_max_srq) : 0;
> + props->max_srq_wr = dev->xdev->caps.max_srq_wqes - 1;
> + props->max_srq_sge = dev->xdev->caps.log_max_srq ? (max_rq_sg - 1) : 0;
> + props->local_ca_ack_delay = dev->xdev->caps.local_ca_ack_delay;
> + props->atomic_cap = is_support_rdma_atomic(dev->xdev) ?
> + IB_ATOMIC_HCA : IB_ATOMIC_NONE;
> + props->masked_atomic_cap = IB_ATOMIC_HCA;
[Severity: Medium]
In xsc_ib_query_device(), props->atomic_cap is gated on
is_support_rdma_atomic(dev->xdev), but props->masked_atomic_cap is
unconditionally set to IB_ATOMIC_HCA. Does this advertise masked atomic
support on hardware where RDMA atomics are not supported?
Also, in the shifts for props->max_mr = (1 << dev->xdev->caps.log_max_mkey) - 1
and max_tso = dev->xdev->caps.log_max_tso ? (1 << dev->xdev->caps.log_max_tso) : 0
below, can the 32-bit signed literal 1 overflow if log_max_mkey or
log_max_tso is >= 31, instead of using BIT_ULL()?
[ ... ]
> + if (offsetofend(typeof(resp), tso_caps) <= udata_outlen) {
> + max_tso = dev->xdev->caps.log_max_tso ? (1 << dev->xdev->caps.log_max_tso) : 0;
> + if (max_tso) {
> + resp.tso_caps.max_tso = max_tso;
> + resp.tso_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
> + resp.response_length += sizeof(resp.tso_caps);
> + }
> + }
> +
> + if (offsetofend(typeof(resp), rss_caps) <= udata_outlen) {
> + resp.rss_caps.rx_hash_function = XSC_RX_HASH_FUNC_TOEPLITZ;
> + resp.rss_caps.rx_hash_fields_mask =
> + XSC_RX_HASH_SRC_IPV4 |
> + XSC_RX_HASH_DST_IPV4 |
> + XSC_RX_HASH_SRC_IPV6 |
> + XSC_RX_HASH_DST_IPV6 |
> + XSC_RX_HASH_SRC_PORT_TCP |
> + XSC_RX_HASH_DST_PORT_TCP |
> + XSC_RX_HASH_SRC_PORT_UDP |
> + XSC_RX_HASH_DST_PORT_UDP |
> + XSC_RX_HASH_INNER;
> + resp.response_length += sizeof(resp.rss_caps);
> + }
> +
> + if (offsetofend(typeof(resp), packet_pacing_caps) <= udata_outlen) {
> + resp.packet_pacing_caps.qp_rate_limit_max =
> + dev->xdev->caps.qp_rate_limit_max;
> + resp.packet_pacing_caps.qp_rate_limit_min =
> + dev->xdev->caps.qp_rate_limit_min;
> + resp.packet_pacing_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
> +
> + resp.response_length += sizeof(resp.packet_pacing_caps);
> + }
> +
> + if (udata_outlen) {
> + err = ib_copy_to_udata(udata, &resp, resp.response_length);
[Severity: High]
Does xsc_ib_query_device() miscalculate resp.response_length and truncate
the response copied to userspace?
In struct xsc_ib_query_device_resp:
include/uapi/rdma/xsc-abi.h:
struct xsc_ib_query_device_resp {
__u32 comp_mask;
__u32 response_length;
struct xsc_ib_tso_caps tso_caps;
struct xsc_ib_rss_caps rss_caps;
struct xsc_ib_cqe_comp_caps cqe_comp_caps;
struct xsc_packet_pacing_caps packet_pacing_caps;
...
};
First, when dev->xdev->caps.log_max_tso is 0, sizeof(resp.tso_caps) is not
added to resp.response_length, even though rss_caps and packet_pacing_caps
sit at fixed offsets after tso_caps.
Second, cqe_comp_caps sits between rss_caps and packet_pacing_caps, but
sizeof(resp.cqe_comp_caps) is never added to resp.response_length.
When udata_outlen is large enough to include packet_pacing_caps (56 bytes),
resp.response_length only reaches 48 (or 40 when max_tso is 0), so
ib_copy_to_udata(udata, &resp, resp.response_length) fails to copy the full
packet_pacing_caps (and truncates rss_caps when max_tso is 0). Should each
block set resp.response_length = offsetofend(typeof(resp), <field>) instead
of incrementing it by sizeof()?
[ ... ]
> +static void xsc_calc_link_info(struct xsc_core_device *xdev,
> + struct ib_port_attr *props)
> +{
> + switch (xsc_get_link_speed(xdev)) {
> + case XSC_MODULE_SPEED_10G:
> + props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
> + props->active_width = 1;
> + break;
> + case XSC_MODULE_SPEED_25G:
> + props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
> + props->active_width = 1;
> + break;
> + case XSC_MODULE_SPEED_40G_R4:
> + props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
> + props->active_width = 2;
> + break;
> + case XSC_MODULE_SPEED_50G_R:
> + props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
> + props->active_width = 1;
> + break;
> + case XSC_MODULE_SPEED_50G_R2:
> + props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
> + props->active_width = 1;
> + break;
> + case XSC_MODULE_SPEED_100G_R2:
> + props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
> + props->active_width = 2;
> + break;
[ ... ]
> +int xsc_ib_query_port(struct ib_device *ibdev, u32 port,
> + struct ib_port_attr *props)
> +{
> + struct xsc_ib_dev *dev = to_xdev(ibdev);
> + struct xsc_core_device *xdev = dev->xdev;
> + struct net_device *ndev;
> +
> + if (port < 1 || port > xdev->caps.num_ports) {
> + ibdev_warn(&dev->ib_dev, "invalid port number %d\n", port);
> + return -EINVAL;
> + }
> +
> + ndev = ib_device_get_netdev(ibdev, port);
> + if (!ndev)
> + return -ENODEV;
> +
> + memset(props, 0, sizeof(*props));
> +
> + props->state = ib_get_curr_port_state(ndev);
> +
> + props->active_mtu = iboe_get_mtu(ndev->mtu);
> + props->gid_tbl_len = XSC_IB_GID_TBL_LEN;
[Severity: Medium]
In xsc_ib_query_port(), props is zeroed with memset() and props->active_mtu
is populated, but props->max_mtu is never set. Doesn't leaving
props->max_mtu as 0 cause ib_mtu_enum_to_int(props->max_mtu) to return -1
and break callers and ULPs that validate path_mtu <= max_mtu?
Additionally, in xsc_calc_link_info() and xsc_ib_query_port(), why are
driver-private UAPI enums (XSC_RDMA_LINK_SPEED_*, XSC_RDMA_PHY_STATE_*) and
raw width numbers used instead of the standard enum ib_port_speed, enum
ib_port_width, and enum ib_port_phys_state constants? In particular, for
XSC_MODULE_SPEED_50G_R2 (2x25G) it sets speed 50G with width 1 (1x50G), and
for XSC_MODULE_SPEED_100G_R2 (2x50G) it sets speed 25G with width 2
(IB_WIDTH_4X, i.e., 4x25G instead of IB_WIDTH_2X = 16 with 50G).
[ ... ]
> +int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx,
> + struct ib_udata *udata)
> +{
> + struct ib_device *ibdev = uctx->device;
> + struct xsc_ib_dev *dev = to_xdev(ibdev);
> + struct xsc_ib_alloc_ucontext_req req = {};
> + struct xsc_ib_alloc_ucontext_resp resp;
> + int err;
> +
> + if (!dev->ib_active)
> + return -EAGAIN;
> +
> + err = ib_copy_from_udata(&req, udata, min(udata->inlen, sizeof(req)));
> + if (err)
> + return err;
[ ... ]
> +static int xsc_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
> +{
> + struct xsc_ib_dev *dev = to_xdev(ibcontext->device);
> + struct xsc_core_device *xdev = dev->xdev;
> + unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
> + resource_size_t reg_base;
> + resource_size_t reg_size = PAGE_SIZE;
> + resource_size_t bar_end;
> + bool db_arm = false;
> + u64 addr;
> + u64 tx_db = 0;
> + u64 rx_db = 0;
> + u64 cq_db = 0;
> + u64 cq_reg = 0;
> +
> + xsc_get_db_addr(xdev, &tx_db, &rx_db, &cq_db, &cq_reg, NULL);
> + dev_dbg(&xdev->pdev->dev, "offset:0x%lx\n", offset);
> +
> + if (offset == (tx_db & PAGE_MASK)) {
> + addr = tx_db;
> + db_arm = true;
> + } else if (offset == (rx_db & PAGE_MASK)) {
> + addr = rx_db;
> + db_arm = true;
> + } else if (offset == (cq_reg & PAGE_MASK)) {
> + addr = cq_reg;
> + db_arm = true;
> + } else if (offset == (cq_db & PAGE_MASK)) {
> + addr = cq_db;
> + db_arm = true;
> + } else if (offset == (QPM_TX_MDB_BASE_REG_ADDR & PAGE_MASK)) {
> + addr = QPM_TX_MDB_BASE_REG_ADDR;
> + } else {
> + return -EINVAL;
> + }
> +
> + /*
> + * On an uncompressed PF the doorbell addresses are hardware
> + * addresses (BAR offset + 0xa0000000 window base); the mdb values
> + * are BAR offsets already.
> + */
> + if (db_arm && xsc_core_is_pf(xdev) && !is_pf_bar_compressed(xdev))
> + addr -= 0xA0000000;
> +
> + reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (addr & PAGE_MASK);
> + bar_end = pci_resource_start(xdev->pdev, xdev->bar_num) +
> + pci_resource_len(xdev->pdev, xdev->bar_num);
> + dev_dbg(&xdev->pdev->dev, "regbase:0x%llx", reg_base);
> +
> + if (vma->vm_end - vma->vm_start != reg_size)
> + return -EINVAL;
> +
> + if (reg_base + reg_size > bar_end)
> + return -EINVAL;
> +
> + return rdma_user_mmap_io(ibcontext, vma, reg_base >> PAGE_SHIFT,
> + reg_size, pgprot_noncached(vma->vm_page_prot),
> + NULL);
> +}
[Severity: Critical]
Does xsc_ib_mmap() expose the privileged BAR0 firmware command queue
registers to unprivileged userspace?
In drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h, the firmware
command queue registers and EQ doorbell sit in the first 4KB page of the
BAR alongside the compressed doorbells and QPM_TX_MDB_BASE_REG_ADDR:
#define CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0x0
#define CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0x4
#define CMDQM_HOST_REQ_PID_MEM_ADDR 0x8
#define CMDQM_HOST_REQ_CID_MEM_ADDR 0xc
#define CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0x10
#define CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0x14
#define CMDQM_HOST_RSP_PID_MEM_ADDR 0x18
#define CMDQM_HOST_RSP_CID_MEM_ADDR 0x1c
#define CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0x20
#define CMDQM_VECTOR_ID_MEM_ADDR 0x24
...
#define DB_CQ_FUNC_MEM_ADDR 0x8c4
#define DB_EQ_FUNC_MEM_ADDR 0x8c8
#define DB_CQ_CID_DIRECT_MEM_ADDR 0x8cc
#define TX_DB_FUNC_MEM_ADDR 0x8d0
#define RX_DB_FUNC_MEM_ADDR 0x8d4
...
#define QPM_TX_MDB_BASE_REG_ADDR 0xd00
Because xsc_ib_mmap() accepts offset == (QPM_TX_MDB_BASE_REG_ADDR & PAGE_MASK)
(which is 0x0 on all functions, without even checking
is_support_multi_doorbell(xdev)), and on compressed-BAR PFs and VFs tx_db,
rx_db, cq_db, and cq_reg are also at 0x8c4..0x8d4, any unprivileged user
with access to /dev/infiniband/uverbsX can mmap BAR offset 0 writable via
rdma_user_mmap_io() and overwrite the command queue DMA ring base physical
addresses and producer/consumer indices.
Also, in xsc_ib_mmap(), computing offset = vma->vm_pgoff << PAGE_SHIFT
truncates the upper PAGE_SHIFT bits of vma->vm_pgoff.
[ ... ]
> +static void init_iommu_state(struct xsc_ib_dev *dev, struct device *dma_dev)
> +{
> + struct iommu_domain *domain;
> +
> + dev->iommu_state = XSC_IB_IOMMU_MAP_DISABLE;
> + domain = iommu_get_domain_for_dev(dma_dev);
> + if (domain) {
> + if (domain->type & __IOMMU_DOMAIN_DMA_API)
> + dev->iommu_state = XSC_IB_IOMMU_MAP_NORMAL;
> + else
> + dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
> + } else {
[Severity: High]
In init_iommu_state(), when the system runs with IOMMU passthrough enabled
(such as iommu.passthrough=1 or iommu=pt), iommu_get_domain_for_dev()
returns an identity domain whose type is IOMMU_DOMAIN_IDENTITY
(__IOMMU_DOMAIN_PT), which does not have __IOMMU_DOMAIN_DMA_API set. Doesn't
this misclassify identity-mapped devices as XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN
instead of treating them as direct-mapped?
[ ... ]
> +static int init_one(struct xsc_core_device *xdev,
> + struct xsc_ib_dev **ibdev)
> +{
> + struct xsc_ib_dev *dev;
> + int err;
> +
> + dev = (struct xsc_ib_dev *)ib_alloc_device(xsc_ib_dev, ib_dev);
> + if (!dev)
> + return -ENOMEM;
> +
> + dev->xdev = xdev;
> + strscpy(dev->ib_dev.name, "xscale_%d", IB_DEVICE_NAME_MAX);
> +
> + memcpy(dev->xdev->priv.ibdev_name, dev->ib_dev.name, IB_DEVICE_NAME_MAX);
> +
> + dev->num_comp_vectors = xdev->dev_res->eq_table.num_comp_vectors;
> +
> + dev->ib_dev.node_type = RDMA_NODE_IB_CA;
> + dev->ib_dev.local_dma_lkey = 0xFF;
> + dev->num_ports = xdev->caps.num_ports;
> + dev->ib_dev.phys_port_cnt = dev->num_ports;
> + dev->ib_dev.num_comp_vectors = dev->num_comp_vectors;
> + dev->ib_dev.dev.parent = &xdev->pdev->dev;
> + err = ib_device_set_netdev(&dev->ib_dev, xdev->netdev, 1);
> + if (err)
> + goto err_free;
[Severity: High]
How is the lifetime of xdev->netdev synchronized between the ethernet driver
and xsc_ib?
First, in init_one(), ib_device_set_netdev(&dev->ib_dev, xdev->netdev, 1)
is called before init_node_data(dev) checks whether dev->xdev->netdev is
NULL to return -EPROBE_DEFER. If xdev->netdev is NULL when
ib_device_set_netdev() runs and xsc_eth_probe() assigns xdev->netdev before
init_node_data() runs, ib_device_set_netdev() binds NULL while
init_node_data() succeeds, causing ib_register_device() ->
xsc_port_immutable() -> ib_query_port() -> xsc_ib_query_port() to fail probe
with -ENODEV instead of deferring.
Second, in xsc_eth_remove():
drivers/net/ethernet/yunsilicon/xsc/net/main.c:xsc_eth_remove() {
...
unregister_netdev(adapter->netdev);
...
free_netdev(adapter->netdev);
xdev->netdev = NULL;
...
}
Because ib_device_set_netdev() takes a netdev_hold() reference on
xdev->netdev and xsc_ib never registers a netdevice notifier to clear the
netdev on NETDEV_UNREGISTER, won't unbinding xsc_pci.eth or unloading
xsc_eth while xsc_ib is loaded hang indefinitely in unregister_netdev()
waiting for xsc_ib's reference to be released? And if xsc_ib_probe() runs
concurrently with xsc_eth_remove(), can init_one() and init_node_data()
access an unregistering or already-freed net_device before xdev->netdev is
set to NULL?
> +
> + xsc_ib_dev_setting(dev);
[ ... ]
> + err = init_node_data(dev);
> + if (err)
> + goto err_free;
> +
> + init_iommu_state(dev, xdev->device);
> + err = ib_register_device(&dev->ib_dev, dev->ib_dev.name,
> + dev->xdev->device);
> + if (err)
> + goto err_free;
> +
> + rdma_roce_rescan_device(&dev->ib_dev);
> + dev->ib_active = true;
> + *ibdev = dev;
[Severity: Medium]
In init_one(), dev->ib_active = true is set only after ib_register_device()
and a redundant rdma_roce_rescan_device() have already completed. Since
ib_register_device() exposes the uverbs device to userspace and sends device
addition events before returning, can a userspace process open the device
immediately and hit:
drivers/infiniband/hw/xsc/main.c:xsc_ib_alloc_ucontext() {
if (!dev->ib_active)
return -EAGAIN;
}
Also, earlier in init_one(), memcpy(dev->xdev->priv.ibdev_name,
dev->ib_dev.name, IB_DEVICE_NAME_MAX) runs before ib_register_device()
expands the format string, leaving the literal "xscale_%d" in
dev->xdev->priv.ibdev_name instead of the assigned device name.
> diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
> new file mode 100644
> index 0000000000000..b91d4aa371bcf
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/xsc_ib.h
[ ... ]
> +#define IB_MTU_8192 6
> +
> +static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)
> +{
> + mtu = mtu - (IB_GRH_BYTES + IB_UDP_BYTES + IB_BTH_BYTES +
> + IB_EXT_XRC_BYTES + IB_EXT_ATOMICETH_BYTES +
> + IB_ICRC_BYTES);
> + if (mtu >= 8192)
> + return IB_MTU_8192;
> + else if (mtu >= ib_mtu_enum_to_int(IB_MTU_4096))
> + return IB_MTU_4096;
> + else if (mtu >= ib_mtu_enum_to_int(IB_MTU_1024))
> + return IB_MTU_1024;
> + else
> + return 0;
> +}
[Severity: Medium]
In xsc_net_to_ib_mtu(), mtu is an unsigned int, so if mtu is less than the
96-byte header overhead, won't the subtraction underflow to a huge unsigned
value and return IB_MTU_8192 (which is also outside enum ib_mtu)?
> +
> +#define XSC_IB_IOMMU_MAP_DISABLE 0
> +#define XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN 1
> +#define XSC_IB_IOMMU_MAP_NORMAL 2
> +
> +static inline int xsc_ib_iommu_dma_map(struct ib_device *ibdev)
> +{
> + return to_xdev(ibdev)->iommu_state;
> +}
> +
> +static inline void *xsc_ib_iova_to_virt(struct ib_device *ibdev, dma_addr_t iova)
> +{
> + phys_addr_t phyaddr;
> + struct iommu_domain *domain;
> +
> + domain = iommu_get_domain_for_dev(ibdev->dma_device);
> + if (likely(domain)) {
> + phyaddr = iommu_iova_to_phys(domain, iova);
> + phyaddr |= iova & (PAGE_SIZE - 1);
> + } else {
> + phyaddr = dma_to_phys(ibdev->dma_device, iova);
> + }
> +
> + return phys_to_virt(phyaddr);
> +}
[Severity: High]
In xsc_ib_iova_to_virt(), iommu_iova_to_phys(domain, iova) already preserves
the sub-granule offset in the returned physical address. On architectures
where the CPU PAGE_SIZE is larger than the IOMMU page granule (for example,
64KB CPU pages with a 4KB IOMMU granule), IOVA bits between the IOMMU page
shift and CPU PAGE_SHIFT do not match the physical page offset. Doesn't
ORing iova & (PAGE_SIZE - 1) into phyaddr corrupt those physical address
bits before calling phys_to_virt()? Also, what happens if
iommu_iova_to_phys() returns 0 on an unmapped IOVA?
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
> new file mode 100644
> index 0000000000000..362099a7c0918
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
[ ... ]
> +static inline void *xsc_vzalloc(unsigned long size)
> +{
> + void *rtn;
> +
> + rtn = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);
> + if (!rtn)
> + rtn = vzalloc(size);
> + return rtn;
> +}
> +
> +static inline void xsc_vfree(const void *addr)
> +{
> + if (addr && is_vmalloc_addr(addr))
> + vfree(addr);
> + else
> + kfree(addr);
> +}
[Severity: Low]
This isn't a bug, but xsc_vzalloc() and xsc_vfree() duplicate the standard
kvzalloc() and kvfree() helpers. Similarly, in drivers/infiniband/hw/xsc/main.c,
xsc_get_guid() duplicates addrconf_addr_eui48(), xsc_ib_dev_setting() sets
dev->ib_dev.ops fields individually instead of using a static const struct
ib_device_ops with ib_set_device_ops(), xsc_ib_init()/xsc_ib_cleanup() can
use module_auxiliary_driver(), and dev_dbg() in xsc_ib_mmap() prints
resource_size_t reg_base with 0x%llx instead of %pa.
> diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/link.c b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
> new file mode 100644
> index 0000000000000..272ddc3a29b10
> --- /dev/null
> +++ b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
> @@ -0,0 +1,34 @@
[ ... ]
> +int xsc_get_link_speed(struct xsc_core_device *xdev)
> +{
> + struct xsc_event_query_linkinfo_mbox_out out;
> + struct xsc_event_linkinfo linkinfo;
> + struct xsc_inbox_hdr in;
> + int err;
> +
> + memset(&in, 0, sizeof(in));
> + memset(&out, 0, sizeof(out));
> + in.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_LINK_INFO);
> +
> + err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
> + if (err) {
> + pci_err(xdev->pdev, "failed to get link info, err=%d\n", err);
> + return XSC_MODULE_SPEED_25G;
> + }
> +
> + linkinfo = out.ctx;
> + linkinfo.linkspeed = be32_to_cpu(linkinfo.linkspeed);
> +
> + return linkinfo.linkspeed;
> +}
[Severity: Low]
In struct xsc_event_linkinfo, linkspeed is declared as __be32. Doesn't
assigning be32_to_cpu(linkinfo.linkspeed) back to linkinfo.linkspeed trigger
a sparse endianness warning instead of returning
be32_to_cpu(out.ctx.linkspeed) directly?
> diff --git a/include/uapi/rdma/xsc-abi.h b/include/uapi/rdma/xsc-abi.h
> new file mode 100644
> index 0000000000000..7e19b23a3842b
> --- /dev/null
> +++ b/include/uapi/rdma/xsc-abi.h
[ ... ]
> +struct xsc_ib_alloc_ucontext_req {
> + __u32 abi_ver;
> + __u32 rsvd1;
> +};
[ ... ]
> +struct xsc_ib_create_qp {
> + __u64 buf_addr;
> + __u64 db_addr;
> + __u32 sq_wqe_count;
> + __u32 rq_wqe_count;
> + __u32 rq_wqe_shift;
> + __u32 flags;
> +
> + int dmabuf_fd;
> + __aligned_u64 dmabuf_sz;
> +};
[ ... ]
> +struct xsc_ib_create_cq {
> + __u64 buf_addr;
> + __u64 db_addr;
> + __u32 cqe_size;
> +
> + int dmabuf_fd;
> + __aligned_u64 dmabuf_sz;
> +};
[ ... ]
> +struct xsc_ib_alloc_pd_resp {
> + __u32 pdn;
> +};
[Severity: Medium]
In include/uapi/rdma/xsc-abi.h, struct xsc_ib_create_qp has an unnamed
4-byte compiler padding hole between int dmabuf_fd and __aligned_u64
dmabuf_sz, both struct xsc_ib_create_qp and struct xsc_ib_create_cq use raw
int instead of __s32 for dmabuf_fd and __u64 instead of __aligned_u64 for
the 64-bit fields, and struct xsc_ib_alloc_pd_resp is not padded to an
8-byte boundary with a trailing __u32 reserved field.
Also, in xsc_ib_alloc_ucontext(), should req.rsvd1 and any trailing bytes
beyond sizeof(req) in udata be checked for zero (e.g., via
ib_is_udata_cleared()) so future ABI extensions can rely on them?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=10
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 11/15] RDMA/xsc: add RDMA PD/MR
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (9 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 10/15] RDMA/xsc: add RDMA device support Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 12/15] RDMA/xsc: add RDMA CQ/SRQ/AH Xin Tian
` (3 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement the protection domain and memory region verbs.
alloc_pd allocates a PD through firmware and returns its number to
userspace. Before ib_register_device(), register the implicit local
DMA MR with mkey 0xFF covering the whole device: the MAD agent posts
QP1 traffic the moment registration returns and assumes lkey 0xFF is
valid, so it must exist by then.
User MRs are registered by building a firmware mailbox with the DMA
address list from the umem. The page size is selected with
ib_umem_find_best_pgsz() among the 4K/64K/2M/1G sizes the device
supports, the page list is bounded by the firmware MTT capacity, and
the variable-length mailbox size is checked for overflow before
allocation. dmabuf-backed MRs use ib_umem_dmabuf_get_pinned() and
share the same registration path. get_dma_mr registers a whole-
device implicitly mapped MR for kernel callers. Deregistration
releases the MPT/MTT and destroys the mkey.
On the core side, add the PD and MR firmware commands: mkey values
are allocated under a lock as a key byte combined with a hardware
index, and registration programs the MPT then writes the page
address list in chunks bounded by the maximum command payload,
rolling back the registration if a chunk fails.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/hw/xsc/Makefile | 2 +-
drivers/infiniband/hw/xsc/main.c | 45 +++
drivers/infiniband/hw/xsc/mem.c | 160 +++++++++
drivers/infiniband/hw/xsc/mr.c | 335 ++++++++++++++++++
drivers/infiniband/hw/xsc/xsc_ib.h | 2 +-
.../ethernet/yunsilicon/xsc/common/driver.h | 1 +
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 2 +-
drivers/net/ethernet/yunsilicon/xsc/pci/cq.c | 8 +-
drivers/net/ethernet/yunsilicon/xsc/pci/mr.c | 202 +++++++++++
drivers/net/ethernet/yunsilicon/xsc/pci/mr.h | 18 +
drivers/net/ethernet/yunsilicon/xsc/pci/pd.c | 49 +++
drivers/net/ethernet/yunsilicon/xsc/pci/pd.h | 14 +
12 files changed, 831 insertions(+), 7 deletions(-)
create mode 100644 drivers/infiniband/hw/xsc/mem.c
create mode 100644 drivers/infiniband/hw/xsc/mr.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/mr.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/mr.h
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/pd.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/pd.h
diff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile
index 6c4ec1ca0..b535a6653 100644
--- a/drivers/infiniband/hw/xsc/Makefile
+++ b/drivers/infiniband/hw/xsc/Makefile
@@ -6,4 +6,4 @@ ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o
-xsc_ib-y := main.o
+xsc_ib-y := main.o mem.o mr.o
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
index 322fad38b..a212a5888 100644
--- a/drivers/infiniband/hw/xsc/main.c
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -391,6 +391,40 @@ static int xsc_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma
NULL);
}
+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
+{
+ struct ib_device *ibdev = ibpd->device;
+ struct xsc_ib_alloc_pd_resp resp;
+ struct xsc_ib_pd *pd;
+ int err;
+
+ pd = to_xpd(ibpd);
+
+ err = xsc_core_alloc_pd(to_xdev(ibdev)->xdev, &pd->pdn);
+ if (err)
+ return err;
+
+ if (udata) {
+ resp.pdn = pd->pdn;
+ if (ib_copy_to_udata(udata, &resp, sizeof(resp))) {
+ xsc_core_dealloc_pd(to_xdev(ibdev)->xdev, pd->pdn);
+ return -EFAULT;
+ }
+ } else {
+ pd->pa_lkey = 0;
+ }
+
+ return 0;
+}
+
+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_pd *xpd = to_xpd(pd);
+
+ return xsc_core_dealloc_pd(dev->xdev, xpd->pdn);
+}
+
static int xsc_port_immutable(struct ib_device *ibdev, u32 port_num,
struct ib_port_immutable *immutable)
{
@@ -475,15 +509,22 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.dealloc_ucontext = xsc_ib_dealloc_ucontext;
dev->ib_dev.ops.mmap = xsc_ib_mmap;
+ dev->ib_dev.ops.alloc_pd = xsc_ib_alloc_pd;
+ dev->ib_dev.ops.dealloc_pd = xsc_ib_dealloc_pd;
dev->ib_dev.ops.get_link_layer = xsc_ib_port_link_layer;
+ dev->ib_dev.ops.get_dma_mr = xsc_ib_get_dma_mr;
+ dev->ib_dev.ops.reg_user_mr = xsc_ib_reg_user_mr; /* optional */
+ dev->ib_dev.ops.reg_user_mr_dmabuf = xsc_ib_reg_user_mr_dmabuf;
+ dev->ib_dev.ops.dereg_mr = xsc_ib_dereg_mr;
dev->ib_dev.ops.get_port_immutable = xsc_port_immutable;
dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ucontext, xsc_ib_ucontext, ibucontext);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_pd, xsc_ib_pd, ibpd);
}
/*
@@ -616,6 +657,10 @@ static int init_one(struct xsc_core_device *xdev,
if (err)
goto err_free;
+ err = xsc_reg_local_dma_mr(xdev);
+ if (err)
+ goto err_free;
+
init_iommu_state(dev, xdev->device);
err = ib_register_device(&dev->ib_dev, dev->ib_dev.name,
dev->xdev->device);
diff --git a/drivers/infiniband/hw/xsc/mem.c b/drivers/infiniband/hw/xsc/mem.c
new file mode 100644
index 000000000..cb1376648
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/mem.c
@@ -0,0 +1,160 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+
+/**
+ * __xsc_ib_cont_pages - scan a umem for compound pages
+ * @umem: umem object to scan
+ * @addr: ib virtual address requested by the user
+ * @max_page_shift: upper limit for the detected page shift, 0 for none
+ * @count: number of PAGE_SIZE pages covered by umem
+ * @shift: page shift for the compound pages found in the region
+ * @ncont: number of compound pages
+ */
+static void __xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
+ unsigned long max_page_shift,
+ int *count, int *shift, int *ncont)
+{
+ unsigned long tmp;
+ unsigned long m;
+ u64 base = ~0, p = 0;
+ u64 len, pfn;
+ int i = 0;
+ struct scatterlist *sg;
+ int entry;
+ /* TODO: need peer mem support */
+ unsigned long page_shift = PAGE_SHIFT;
+
+ addr = addr >> page_shift;
+ tmp = (unsigned long)addr;
+ m = find_first_bit(&tmp, BITS_PER_LONG);
+ if (max_page_shift)
+ m = min_t(unsigned long, max_page_shift - page_shift, m);
+ for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
+ len = sg_dma_len(sg) >> page_shift;
+ pfn = sg_dma_address(sg) >> page_shift;
+ if (base + p != pfn) {
+ /* If either the offset or the new
+ * base are unaligned update m
+ */
+ tmp = (unsigned long)(pfn | p);
+ if (!IS_ALIGNED(tmp, 1UL << m))
+ m = find_first_bit(&tmp, BITS_PER_LONG);
+
+ base = pfn;
+ p = 0;
+ }
+
+ p += len;
+ i += len;
+ }
+
+ if (i) {
+ m = min_t(unsigned long, ilog2(roundup_pow_of_two(i)), m);
+ *ncont = DIV_ROUND_UP(i, (1 << m));
+ } else {
+ m = 0;
+ *ncont = 0;
+ }
+ *shift = page_shift + m;
+ *count = i;
+}
+
+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
+ int *count, int *shift, int *ncont)
+{
+ /* no limit for page_shift */
+ __xsc_ib_cont_pages(umem, addr, 0, count, shift, ncont);
+}
+
+static void __xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, size_t offset, size_t num_pages,
+ __be64 *pas, int access_flags, bool need_to_devide)
+{
+ unsigned long umem_page_shift = PAGE_SHIFT;
+ int shift = page_shift - umem_page_shift;
+ int mask = (1 << shift) - 1;
+ int i = 0;
+ int k, idx;
+ u64 cur = 0;
+ u64 base;
+ int len;
+ struct scatterlist *sg;
+ int entry;
+
+ for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
+ if (need_to_devide)
+ len = sg_dma_len(sg) >> XSC_PAGE_SHIFT_4K;
+ else
+ len = sg_dma_len(sg) >> umem_page_shift;
+ base = sg_dma_address(sg);
+
+ /* Skip elements below offset */
+ if (i + len < offset << shift) {
+ i += len;
+ continue;
+ }
+
+ /* Skip pages below offset */
+ if (i < offset << shift) {
+ k = (offset << shift) - i;
+ i = offset << shift;
+ } else {
+ k = 0;
+ }
+
+ for (; k < len; k++) {
+ if (!(i & mask)) {
+ if (need_to_devide)
+ cur = base + (k << XSC_PAGE_SHIFT_4K);
+ else
+ cur = base + (k << umem_page_shift);
+ cur |= access_flags;
+ idx = (i >> shift) - offset;
+
+ pas[idx] = cpu_to_be64(cur);
+ ibdev_dbg(&dev->ib_dev, "pas[%d] 0x%llx\n",
+ i >> shift, be64_to_cpu(pas[idx]));
+ }
+ i++;
+
+ /* Stop after num_pages reached */
+ if (i >> shift >= offset + num_pages)
+ return;
+ }
+ }
+}
+
+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, __be64 *pas, int npages, bool need_to_devide)
+{
+ return __xsc_ib_populate_pas(dev, umem, page_shift, 0,
+ npages, pas, 0, need_to_devide);
+}
+
+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset)
+{
+ u64 page_size;
+ u64 page_mask;
+ u64 off_size;
+ u64 off_mask;
+ u64 buf_off;
+
+ page_size = 1ULL << page_shift;
+ page_mask = page_size - 1;
+ buf_off = addr & page_mask;
+ off_size = page_size >> 6;
+ off_mask = off_size - 1;
+
+ if (buf_off & off_mask)
+ return -EINVAL;
+
+ *offset = buf_off >> ilog2(off_size);
+ return 0;
+}
diff --git a/drivers/infiniband/hw/xsc/mr.c b/drivers/infiniband/hw/xsc/mr.c
new file mode 100644
index 000000000..b5dc088fc
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/mr.c
@@ -0,0 +1,335 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kref.h>
+#include <linux/random.h>
+#include <linux/debugfs.h>
+#include <linux/export.h>
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
+#include <rdma/ib_umem.h>
+#include <rdma/iter.h>
+#include "common/xsc_cmd.h"
+#include <linux/dma-direct.h>
+#include "xsc_ib.h"
+
+enum {
+ DEF_CACHE_SIZE = 10,
+};
+
+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ struct xsc_register_mr_mbox_in *in;
+ struct xsc_register_mr_request *req;
+ struct xsc_ib_mr *mr;
+ int err;
+
+ mr = kzalloc_obj(*mr, GFP_KERNEL);
+ if (!mr)
+ return ERR_PTR(-ENOMEM);
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ req = &in->req;
+ req->pdn = cpu_to_be32(to_xpd(pd)->pdn);
+ req->acc = convert_access(acc);
+ req->va_base = 0;
+ req->map_en = !(XSC_MPT_MAP_EN);
+
+ err = xsc_core_create_mkey(xdev, &mr->xmr);
+ if (err)
+ goto err_in;
+ req->mkey = cpu_to_be32(mr->xmr.key);
+ err = xsc_core_register_mr(xdev, &mr->xmr, in, sizeof(*in));
+ if (err)
+ goto err_reg_mr;
+ kfree(in);
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->umem = NULL;
+
+ return &mr->ibmr;
+err_reg_mr:
+ xsc_core_destroy_mkey(xdev, &mr->xmr);
+err_in:
+ kfree(in);
+
+err_free:
+ kfree(mr);
+
+ return ERR_PTR(err);
+}
+
+static struct xsc_ib_mr *reg_create(struct ib_pd *pd, u64 virt_addr,
+ u64 length, struct ib_umem *umem,
+ size_t npages, int page_shift,
+ int access_flags)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_register_mr_mbox_in *in;
+ struct xsc_ib_mr *mr;
+ struct ib_block_iter biter;
+ __be64 *pas;
+ size_t pas_size;
+ size_t inlen;
+ int err;
+
+ mr = kzalloc_obj(*mr, GFP_KERNEL);
+ if (!mr) {
+ err = -ENOMEM;
+ goto err_0;
+ }
+
+ if (npages > dev->xdev->caps.max_mtt) {
+ ibdev_err(&dev->ib_dev, "npages(%zu) exceeded.\n", npages);
+ err = -EINVAL;
+ goto err_1;
+ }
+
+ if (check_mul_overflow(npages, sizeof(*in->req.pas), &pas_size) ||
+ check_add_overflow(pas_size, sizeof(*in), &inlen) ||
+ inlen > INT_MAX) {
+ err = -EINVAL;
+ goto err_1;
+ }
+ in = xsc_vzalloc(inlen);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_1;
+ }
+ err = xsc_core_create_mkey(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "create mkey failed\n");
+ goto err_2;
+ }
+
+ pas = in->req.pas;
+ rdma_umem_for_each_dma_block(umem, &biter, 1UL << page_shift) {
+ *pas = cpu_to_be64(rdma_block_iter_dma_address(&biter));
+ pas++;
+ }
+
+ in->req.acc = convert_access(access_flags);
+ in->req.pa_num = cpu_to_be32((u32)npages);
+ in->req.pdn = cpu_to_be32(to_xpd(pd)->pdn);
+ in->req.va_base = cpu_to_be64(virt_addr);
+ in->req.map_en = XSC_MPT_MAP_EN;
+ in->req.len = cpu_to_be64(length);
+ in->req.page_mode = xsc_get_mr_page_mode(dev->xdev, page_shift);
+ ibdev_dbg(&dev->ib_dev, "read_flush hwconfig %s\n",
+ dev->xdev->read_flush ? "enable" : "disable");
+ in->req.is_gpu = 0;
+ in->req.mkey = cpu_to_be32(mr->xmr.key);
+ err = xsc_core_register_mr(dev->xdev, &mr->xmr, in, inlen);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "register mr failed, err = %d\n", err);
+ goto err_reg_mr;
+ }
+ mr->umem = umem;
+ xsc_vfree(in);
+
+ ibdev_dbg(&dev->ib_dev, "mkey = 0x%x\n", mr->xmr.key);
+
+ return mr;
+err_reg_mr:
+ xsc_core_destroy_mkey(dev->xdev, &mr->xmr);
+err_2:
+ xsc_vfree(in);
+err_1:
+ kfree(mr);
+err_0:
+ return ERR_PTR(err);
+}
+
+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
+ u64 virt_addr, int access_flags,
+ struct ib_dmah *dmah,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_mr *mr = NULL;
+ struct ib_umem *umem;
+ unsigned long pgsz;
+ int page_shift;
+ size_t npages;
+ int err;
+
+ if (dmah)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (length > dev->xdev->caps.max_mr_size) {
+ ibdev_err(&dev->ib_dev, "reg user mr length(%llu) exceeded.\n", length);
+ return ERR_PTR(-EINVAL);
+ }
+
+ ibdev_dbg(&dev->ib_dev, "start 0x%llx, virt_addr 0x%llx, length 0x%llx\n",
+ start, virt_addr, length);
+
+ umem = ib_umem_get_va(&dev->ib_dev, start, length, access_flags);
+ if (IS_ERR(umem)) {
+ ibdev_dbg(&dev->ib_dev, "umem get failed\n");
+ return (void *)umem;
+ }
+
+ pgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);
+ if (!pgsz) {
+ ibdev_warn(&dev->ib_dev, "find best page size failed\n");
+ err = -EINVAL;
+ goto error;
+ }
+
+ page_shift = order_base_2(pgsz);
+ npages = ib_umem_num_dma_blocks(umem, pgsz);
+
+ ibdev_dbg(&dev->ib_dev, "npages %zu, page_shift %d\n", npages, page_shift);
+
+ mr = reg_create(pd, virt_addr, length, umem, npages, page_shift,
+ access_flags);
+ if (IS_ERR(mr)) {
+ err = PTR_ERR(mr);
+ goto error;
+ }
+
+ ibdev_dbg(&dev->ib_dev, "mkey 0x%x\n", mr->xmr.key);
+
+ mr->umem = umem;
+ mr->npages = npages;
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->ibmr.length = length;
+ return &mr->ibmr;
+
+error:
+ ib_umem_release(umem);
+ return ERR_PTR(err);
+}
+
+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
+ u64 length, u64 virt_addr,
+ int fd, int access_flags,
+ struct ib_dmah *dmah,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct ib_umem_dmabuf *umem_dmabuf;
+ struct ib_umem *umem;
+ struct xsc_ib_mr *mr;
+ unsigned long pgsz;
+ int page_shift;
+ size_t npages;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM))
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (dmah)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (length > dev->xdev->caps.max_mr_size) {
+ ibdev_err(&dev->ib_dev, "reg user dmabuf mr length(%llu) exceeded.\n",
+ length);
+ return ERR_PTR(-EINVAL);
+ }
+
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev,
+ offset, length,
+ fd, access_flags);
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_dbg(&dev->ib_dev, "umem_dmabuf get failed (%ld)\n",
+ PTR_ERR(umem_dmabuf));
+ return ERR_CAST(umem_dmabuf);
+ }
+ umem = &umem_dmabuf->umem;
+
+ pgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);
+ if (!pgsz) {
+ ibdev_warn(&dev->ib_dev, "find best page size failed\n");
+ err = -EINVAL;
+ goto err_release;
+ }
+
+ page_shift = order_base_2(pgsz);
+ npages = ib_umem_num_dma_blocks(umem, pgsz);
+
+ mr = reg_create(pd, virt_addr, length, umem, npages, page_shift,
+ access_flags);
+ if (IS_ERR(mr)) {
+ err = PTR_ERR(mr);
+ goto err_release;
+ }
+
+ mr->umem = umem;
+ mr->npages = npages;
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->ibmr.length = length;
+
+ return &mr->ibmr;
+err_release:
+ ib_umem_release(umem);
+ return ERR_PTR(err);
+}
+
+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibmr->device);
+ struct xsc_ib_mr *mr = to_xmr(ibmr);
+ struct ib_umem *umem = mr->umem;
+ int err;
+
+ ibdev_dbg(&dev->ib_dev, "dereg mkey = 0x%x\n", mr->xmr.key);
+
+ if (mr->npages) {
+ err = xsc_core_dereg_mr(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "failed to dereg mr 0x%x (%d)\n",
+ mr->xmr.key, err);
+ return err;
+ }
+ }
+ err = xsc_core_destroy_mkey(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "failed to destroy mkey 0x%x (%d)\n",
+ mr->xmr.key, err);
+ return err;
+ }
+
+ if (umem)
+ ib_umem_release(umem);
+
+ kfree(mr);
+
+ return 0;
+}
+
+int xsc_reg_local_dma_mr(struct xsc_core_device *dev)
+{
+ struct xsc_register_mr_mbox_in in;
+ int err = 0;
+
+ in.req.pdn = 0;
+ in.req.pa_num = 0;
+ in.req.len = 0;
+ in.req.mkey = cpu_to_be32(0xFF);
+ in.req.acc = XSC_PERM_LOCAL_WRITE | XSC_PERM_LOCAL_READ;
+ in.req.page_mode = xsc_get_mr_page_mode(dev, XSC_PAGE_SHIFT_4K);
+ in.req.is_gpu = 0;
+ in.req.map_en = !(XSC_MPT_MAP_EN);
+ in.req.va_base = 0;
+
+ err = xsc_core_register_mr(dev, NULL, &in, sizeof(in));
+ if (err)
+ dev_err(&dev->pdev->dev,
+ "failed to register the local dma mkey 0xFF (%d)\n", err);
+
+ return err;
+}
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
index b91d4aa37..eb4d9e630 100644
--- a/drivers/infiniband/hw/xsc/xsc_ib.h
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -207,7 +207,7 @@ struct xsc_ib_mr {
struct ib_mr ibmr;
struct xsc_core_mr xmr;
struct ib_umem *umem;
- int npages;
+ u32 npages;
};
struct xsc_ib_dev {
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
index 362099a7c..0c91574d6 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
@@ -63,6 +63,7 @@ int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
struct xsc_register_mr_mbox_in *in, int inlen);
int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_reg_local_dma_mr(struct xsc_core_device *xdev);
int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);
int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index 87481db3f..065d52ebf 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,4 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o link.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o pd.o mr.o link.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
index e3e13ef53..a964236bd 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
@@ -143,9 +143,9 @@ static int xsc_set_cq_context(struct xsc_core_device *xdev,
memset(&out, 0, sizeof(out));
err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
- if (err || out.hdr.status) {
+ if (err) {
pci_err(xdev->pdev, "failed to set cq context\n");
- return -EIO;
+ return err;
}
*pa_list_base = be32_to_cpu(out.cq_pa_list_base);
@@ -193,9 +193,9 @@ static int xsc_set_cq_buf_pa(struct xsc_core_device *xdev,
err = xsc_cmd_exec(xdev, in, in_len, &out, sizeof(out));
kvfree(in);
- if (err || out.hdr.status) {
+ if (err) {
pci_err(xdev->pdev, "failed to set cq buf pa, cqn=%d\n", cqn);
- return -EINVAL;
+ return err;
}
}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c
new file mode 100644
index 000000000..b07e8743f
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c
@@ -0,0 +1,202 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "hal/xsc_hal.h"
+#include "mr.h"
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ struct xsc_create_mkey_mbox_in in;
+ struct xsc_create_mkey_mbox_out out;
+ u8 key;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ spin_lock(&xdev->dev_res->mkey_lock);
+ key = 0x80 + xdev->dev_res->mkey_key++;
+ spin_unlock(&xdev->dev_res->mkey_lock);
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MKEY);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to create mkey, err = %d\n", err);
+ return err;
+ }
+
+ mr->key = xsc_idx_to_mkey(xdev, be32_to_cpu(out.mkey) & 0xffffff) | key;
+ pci_dbg(xdev->pdev, "out 0x%x, key 0x%x, mkey 0x%x\n",
+ be32_to_cpu(out.mkey), key, mr->key);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_mkey);
+
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ struct xsc_destroy_mkey_mbox_in in;
+ struct xsc_destroy_mkey_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_MKEY);
+ in.mkey = cpu_to_be32(mr->key);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_mkey);
+
+static int xsc_set_mpt_via_cmdq(struct xsc_core_device *xdev,
+ struct xsc_register_mr_mbox_in *in_cmd,
+ u32 *mtt_base)
+{
+ struct xsc_register_mr_request *req = &in_cmd->req;
+ struct xsc_set_mpt_mbox_in *in;
+ struct xsc_set_mpt_mbox_out out;
+ u64 mem_size;
+ int err;
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ mem_size = be64_to_cpu(req->len);
+ in->mpt_item.pdn = req->pdn;
+ in->mpt_item.pa_num = req->pa_num;
+ in->mpt_item.len = cpu_to_be32((u32)mem_size);
+ in->mpt_item.len_h = cpu_to_be32((u32)(mem_size >> 32));
+ in->mpt_item.mkey = req->mkey;
+ in->mpt_item.is_gpu = req->is_gpu;
+ in->mpt_item.acc = req->acc;
+ in->mpt_item.page_mode = req->page_mode;
+ in->mpt_item.map_en = req->map_en;
+ in->mpt_item.va_base = req->va_base;
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MPT);
+
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, in, sizeof(*in), &out, sizeof(out));
+ kfree(in);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mpt, err = %d\n", err);
+ return err;
+ }
+
+ *mtt_base = be32_to_cpu(out.mtt_base);
+ return 0;
+}
+
+#define PA_NUM_PER_CMD 1024
+
+static int xsc_set_mtt_via_cmdq(struct xsc_core_device *xdev,
+ struct xsc_register_mr_mbox_in *in_cmd,
+ u32 mtt_base)
+{
+ struct xsc_register_mr_request *req = &in_cmd->req;
+ struct xsc_set_mtt_mbox_in *seg_in;
+ struct xsc_set_mtt_mbox_out seg_out;
+ int tot_pg_num = be32_to_cpu(req->pa_num);
+ int tot_seg_num, seg_pa_num, seg_idx;
+ int pa_idx_base = 0;
+ int in_len, err, i;
+
+ tot_seg_num = (tot_pg_num & (PA_NUM_PER_CMD - 1)) ?
+ ((tot_pg_num >> 10) + 1) : (tot_pg_num >> 10);
+
+ for (seg_idx = 0; seg_idx < tot_seg_num; seg_idx++) {
+ seg_pa_num = (seg_idx != tot_seg_num - 1) ? PA_NUM_PER_CMD :
+ (tot_pg_num - ((tot_seg_num - 1) << 10));
+ in_len = (seg_pa_num << 3) + sizeof(*seg_in);
+ seg_in = kzalloc(in_len, GFP_KERNEL);
+ if (!seg_in)
+ return -ENOMEM;
+
+ seg_in->mtt_setting.mtt_base = cpu_to_be32(mtt_base);
+ seg_in->mtt_setting.pa_num = cpu_to_be32(seg_pa_num);
+ for (i = 0; i < seg_pa_num; i++)
+ seg_in->mtt_setting.pas[i] = req->pas[pa_idx_base + i];
+ seg_in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTT);
+
+ memset(&seg_out, 0, sizeof(seg_out));
+ pci_dbg(xdev->pdev,
+ "set mtt seg %d, pa_num %d, pa_idx_base %d, tot_seg %d\n",
+ seg_idx, seg_pa_num, pa_idx_base, tot_seg_num);
+ err = xsc_cmd_exec(xdev, seg_in, in_len, &seg_out, sizeof(seg_out));
+ kfree(seg_in);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mtt seg %d, err = %d\n",
+ seg_idx, err);
+ return err;
+ }
+
+ pa_idx_base += seg_pa_num;
+ mtt_base += seg_pa_num;
+ }
+
+ return 0;
+}
+
+static int xsc_dereg_mr_via_cmdq(struct xsc_core_device *xdev, u32 mkey)
+{
+ struct xsc_unregister_mr_mbox_in in;
+ struct xsc_unregister_mr_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEREG_MR);
+ in.mkey = cpu_to_be32(xsc_mkey_to_idx(xdev, mkey));
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to dereg mr, err = %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen)
+{
+ u32 mtt_base;
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_REG_MR);
+
+ err = xsc_set_mpt_via_cmdq(xdev, in, &mtt_base);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mpt via cmdq\n");
+ return err;
+ }
+
+ err = xsc_set_mtt_via_cmdq(xdev, in, mtt_base);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mtt via cmdq\n");
+ xsc_dereg_mr_via_cmdq(xdev, be32_to_cpu(in->req.mkey));
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_register_mr);
+
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ return xsc_dereg_mr_via_cmdq(xdev, mr->key);
+}
+EXPORT_SYMBOL_GPL(xsc_core_dereg_mr);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h
new file mode 100644
index 000000000..edafecda6
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_MR_H
+#define __XSC_MR_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen);
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+
+#endif /* __XSC_MR_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c
new file mode 100644
index 000000000..e8e55bc4c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "pd.h"
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn)
+{
+ struct xsc_alloc_pd_mbox_in in;
+ struct xsc_alloc_pd_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ALLOC_PD);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ *pdn = be32_to_cpu(out.pdn) & 0xffffff;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_alloc_pd);
+
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn)
+{
+ struct xsc_dealloc_pd_mbox_in in;
+ struct xsc_dealloc_pd_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEALLOC_PD);
+ in.pdn = cpu_to_be32(pdn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_dealloc_pd);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h
new file mode 100644
index 000000000..d9a99baea
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PD_H
+#define __XSC_PD_H
+
+#include "common/xsc_core.h"
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);
+
+#endif /* __XSC_PD_H */
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 11/15] RDMA/xsc: add RDMA PD/MR
2026-11-04 15:57 ` [PATCH v1 11/15] RDMA/xsc: add RDMA PD/MR Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] RDMA/xsc: shift out of bounds in __xsc_ib_cont_pages() when addr is 0
- [High] RDMA/xsc: negative shift and 32-bit overflow in __xsc_ib_populate_pas()
- [Medium] RDMA/xsc: skipped MPT deregistration for DMA MRs and repeat dereg in xsc_ib_dereg_mr()
- [Medium] RDMA/xsc: missing mr->ibmr.iova and page_size init in user MR registration
--
commit cab7841014d8407bdc13cf06eb834f8c3aa28962
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add RDMA PD/MR
This commit implements the protection domain and memory region verbs for the
xsc RDMA driver, including user MR, dmabuf MR, and DMA MR registration as well
as the corresponding firmware commands.
> diff --git a/drivers/infiniband/hw/xsc/mem.c b/drivers/infiniband/hw/xsc/mem.c
> new file mode 100644
> index 0000000000000..cb13766480f1c
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/mem.c
> @@ -0,0 +1,160 @@
[ ... ]
> +static void __xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
> + unsigned long max_page_shift,
> + int *count, int *shift, int *ncont)
> +{
> + unsigned long tmp;
> + unsigned long m;
> + u64 base = ~0, p = 0;
> + u64 len, pfn;
> + int i = 0;
> + struct scatterlist *sg;
> + int entry;
> + /* TODO: need peer mem support */
> + unsigned long page_shift = PAGE_SHIFT;
> +
> + addr = addr >> page_shift;
> + tmp = (unsigned long)addr;
> + m = find_first_bit(&tmp, BITS_PER_LONG);
> + if (max_page_shift)
> + m = min_t(unsigned long, max_page_shift - page_shift, m);
> + for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
> + len = sg_dma_len(sg) >> page_shift;
> + pfn = sg_dma_address(sg) >> page_shift;
> + if (base + p != pfn) {
> + /* If either the offset or the new
> + * base are unaligned update m
> + */
> + tmp = (unsigned long)(pfn | p);
> + if (!IS_ALIGNED(tmp, 1UL << m))
> + m = find_first_bit(&tmp, BITS_PER_LONG);
> +
> + base = pfn;
> + p = 0;
> + }
> +
> + p += len;
> + i += len;
> + }
> +
> + if (i) {
> + m = min_t(unsigned long, ilog2(roundup_pow_of_two(i)), m);
> + *ncont = DIV_ROUND_UP(i, (1 << m));
> + } else {
> + m = 0;
> + *ncont = 0;
> + }
> + *shift = page_shift + m;
> + *count = i;
> +}
> +
> +void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
> + int *count, int *shift, int *ncont)
> +{
> + /* no limit for page_shift */
> + __xsc_ib_cont_pages(umem, addr, 0, count, shift, ncont);
> +}
[Severity: High]
Can __xsc_ib_cont_pages() trigger a shift-out-of-bounds undefined behavior and
miscalculate the compound page size when addr >> page_shift is 0?
xsc_ib_cont_pages() calls __xsc_ib_cont_pages() with max_page_shift == 0. When
addr >> page_shift is 0 (or on 32-bit builds when the lower 32 bits of
addr >> page_shift are 0), find_first_bit(&tmp, BITS_PER_LONG) returns
BITS_PER_LONG and m is not clamped because max_page_shift is 0.
Inside the scatterlist loop, IS_ALIGNED(tmp, 1UL << m) then evaluates
1UL << BITS_PER_LONG, which is undefined behavior in C. On architectures that
mask the shift count to BITS_PER_LONG - 1, 1UL << BITS_PER_LONG evaluates to
1UL, making IS_ALIGNED(tmp, 1UL) unconditionally true so m is never reduced to
match unaligned or discontiguous scatterlist segments. The same happens if
pfn | p is 0 on the first segment.
Also, in DIV_ROUND_UP(i, (1 << m)), 1 << m uses a 32-bit signed int literal,
which overflows if m >= 31. Should m be clamped to a valid shift range and use
1UL << m?
> +
> +static void __xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
> + int page_shift, size_t offset, size_t num_pages,
> + __be64 *pas, int access_flags, bool need_to_devide)
> +{
> + unsigned long umem_page_shift = PAGE_SHIFT;
> + int shift = page_shift - umem_page_shift;
> + int mask = (1 << shift) - 1;
> + int i = 0;
> + int k, idx;
> + u64 cur = 0;
> + u64 base;
> + int len;
> + struct scatterlist *sg;
> + int entry;
> +
> + for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
> + if (need_to_devide)
> + len = sg_dma_len(sg) >> XSC_PAGE_SHIFT_4K;
> + else
> + len = sg_dma_len(sg) >> umem_page_shift;
> + base = sg_dma_address(sg);
> +
> + /* Skip elements below offset */
> + if (i + len < offset << shift) {
> + i += len;
> + continue;
> + }
> +
> + /* Skip pages below offset */
> + if (i < offset << shift) {
> + k = (offset << shift) - i;
> + i = offset << shift;
> + } else {
> + k = 0;
> + }
> +
> + for (; k < len; k++) {
> + if (!(i & mask)) {
> + if (need_to_devide)
> + cur = base + (k << XSC_PAGE_SHIFT_4K);
> + else
> + cur = base + (k << umem_page_shift);
[Severity: High]
Does __xsc_ib_populate_pas() produce a negative shift or misindex pas[] when
need_to_devide is true, and can k << XSC_PAGE_SHIFT_4K overflow 32-bit signed
arithmetic?
At the start of __xsc_ib_populate_pas(), umem_page_shift is hardcoded to
PAGE_SHIFT to compute shift = page_shift - umem_page_shift and
mask = (1 << shift) - 1. However, when need_to_devide is true, the loop
divides scatterlist segments into 4K units (XSC_PAGE_SHIFT_4K). On kernels
where PAGE_SHIFT > 12 (such as 64K-page arm64 kernels), passing
page_shift == XSC_PAGE_SHIFT_4K makes shift negative, causing negative shift
undefined behavior in (1 << shift), (offset << shift), and (i >> shift). And
if page_shift > PAGE_SHIFT while need_to_devide is true, i counts 4K pages
while shift is relative to PAGE_SHIFT, causing pas[] entries to be skipped.
Additionally, k is declared as a 32-bit signed int, so k << XSC_PAGE_SHIFT_4K
and k << umem_page_shift are evaluated as 32-bit signed integers before being
added to u64 base. Because the PCI layer configures a 2 GiB max DMA segment
size (SZ_2G), a 2 GiB segment has len == 524288 (0x80000) 4K pages; when k
reaches 0x80000, k << 12 overflows 32-bit signed int to -2147483648 and
sign-extends to 0xffffffff80000000ULL when added to base.
> diff --git a/drivers/infiniband/hw/xsc/mr.c b/drivers/infiniband/hw/xsc/mr.c
> new file mode 100644
> index 0000000000000..b5dc088fcbbde
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/mr.c
> @@ -0,0 +1,335 @@
[ ... ]
> +struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc)
> +{
> + struct xsc_ib_dev *dev = to_xdev(pd->device);
> + struct xsc_core_device *xdev = dev->xdev;
> + struct xsc_register_mr_mbox_in *in;
> + struct xsc_register_mr_request *req;
> + struct xsc_ib_mr *mr;
> + int err;
> +
> + mr = kzalloc_obj(*mr, GFP_KERNEL);
> + if (!mr)
> + return ERR_PTR(-ENOMEM);
> +
> + in = kzalloc_obj(*in, GFP_KERNEL);
> + if (!in) {
> + err = -ENOMEM;
> + goto err_free;
> + }
> +
> + req = &in->req;
> + req->pdn = cpu_to_be32(to_xpd(pd)->pdn);
> + req->acc = convert_access(acc);
> + req->va_base = 0;
> + req->map_en = !(XSC_MPT_MAP_EN);
> +
> + err = xsc_core_create_mkey(xdev, &mr->xmr);
> + if (err)
> + goto err_in;
> + req->mkey = cpu_to_be32(mr->xmr.key);
> + err = xsc_core_register_mr(xdev, &mr->xmr, in, sizeof(*in));
> + if (err)
> + goto err_reg_mr;
> + kfree(in);
> + mr->ibmr.lkey = mr->xmr.key;
> + mr->ibmr.rkey = mr->xmr.key;
> + mr->umem = NULL;
> +
> + return &mr->ibmr;
[ ... ]
> +struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
> + u64 virt_addr, int access_flags,
> + struct ib_dmah *dmah,
> + struct ib_udata *udata)
> +{
[ ... ]
> + mr->umem = umem;
> + mr->npages = npages;
> + mr->ibmr.lkey = mr->xmr.key;
> + mr->ibmr.rkey = mr->xmr.key;
> + mr->ibmr.length = length;
> + return &mr->ibmr;
> +
> +error:
> + ib_umem_release(umem);
> + return ERR_PTR(err);
> +}
> +
> +struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
> + u64 length, u64 virt_addr,
> + int fd, int access_flags,
> + struct ib_dmah *dmah,
> + struct uverbs_attr_bundle *attrs)
> +{
[ ... ]
> + mr->umem = umem;
> + mr->npages = npages;
> + mr->ibmr.lkey = mr->xmr.key;
> + mr->ibmr.rkey = mr->xmr.key;
> + mr->ibmr.length = length;
> +
> + return &mr->ibmr;
> +err_release:
> + ib_umem_release(umem);
> + return ERR_PTR(err);
> +}
[Severity: Medium]
Should xsc_ib_reg_user_mr() and xsc_ib_reg_user_mr_dmabuf() also initialize
mr->ibmr.iova and mr->ibmr.page_size?
Unlike the legacy write path in ib_uverbs_reg_mr(), the ioctl paths
UVERBS_METHOD_REG_MR and UVERBS_METHOD_REG_DMABUF_MR in uverbs_std_types_mr.c
do not set mr->iova, and UVERBS_METHOD_QUERY_MR copies mr->iova back to
userspace via UVERBS_ATTR_QUERY_MR_RESP_IOVA. Leaving mr->ibmr.iova and
mr->ibmr.page_size unset causes MR queries to report iova == 0 instead of
virt_addr and leaves page_size as 0.
Also, since CONFIG_INFINIBAND_XSC does not depend on 64BIT, passing u64
start/offset and u64 length directly to ib_umem_get_va() and
ib_umem_dmabuf_get_pinned() truncates 64-bit values to 32-bit unsigned long
and size_t on 32-bit builds while programming the untruncated 64-bit length
into the MPT.
> +
> +int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
> +{
> + struct xsc_ib_dev *dev = to_xdev(ibmr->device);
> + struct xsc_ib_mr *mr = to_xmr(ibmr);
> + struct ib_umem *umem = mr->umem;
> + int err;
> +
> + ibdev_dbg(&dev->ib_dev, "dereg mkey = 0x%x\n", mr->xmr.key);
> +
> + if (mr->npages) {
> + err = xsc_core_dereg_mr(dev->xdev, &mr->xmr);
> + if (err) {
> + ibdev_warn(&dev->ib_dev, "failed to dereg mr 0x%x (%d)\n",
> + mr->xmr.key, err);
> + return err;
> + }
> + }
> + err = xsc_core_destroy_mkey(dev->xdev, &mr->xmr);
> + if (err) {
> + ibdev_warn(&dev->ib_dev, "failed to destroy mkey 0x%x (%d)\n",
> + mr->xmr.key, err);
> + return err;
> + }
[Severity: Medium]
Why is xsc_core_dereg_mr() gated on mr->npages in xsc_ib_dereg_mr()?
In xsc_ib_get_dma_mr(), the driver calls xsc_core_register_mr(), which
programs the hardware MPT entry via XSC_CMD_OP_SET_MPT, while mr->npages
remains 0. When a DMA MR is later deregistered in xsc_ib_dereg_mr(), the
if (mr->npages) check skips xsc_core_dereg_mr() and only calls
xsc_core_destroy_mkey(), leaving the MPT entry programmed in firmware.
Conversely, for user MRs where mr->npages > 0, if xsc_core_dereg_mr() succeeds
but xsc_core_destroy_mkey() fails, xsc_ib_dereg_mr() returns an error without
clearing mr->npages, so a subsequent retry will invoke xsc_core_dereg_mr() a
second time on the already-deregistered MR.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=11
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 12/15] RDMA/xsc: add RDMA CQ/SRQ/AH
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (10 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 11/15] RDMA/xsc: add RDMA PD/MR Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 13/15] RDMA/xsc: add QP verbs Xin Tian
` (2 subsequent siblings)
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement the completion queue, shared receive queue and address
handle verbs.
CQs get a ring of completion entries, in the 32- or 64-byte form
depending on the firmware capability, backed by a coherent DMA
buffer for kernel CQs or by a user umem (including dmabuf) for user
CQs. The completion and error callbacks are installed before the
CQ is inserted into the device tree, since insertion makes it
visible to the EQ event path. Polling translates hardware opcodes
to ib_wc opcodes, detects lost completions by checking that each
signaled WQE chains from the previous one and synthesizing an error
completion when it does not, converts QP1 receive completions back
to the GRH form the MAD layer expects, and generates flush
completions for QPs that entered the error state. CQ clean compacts
the ring when a QP is destroyed or reset.
SRQs are implemented by caching posted receive WRs in the core
device and feeding them to the attached QPs only as their
outstanding WR count falls below a low-water mark, so a single
posted list can serve many attached QPs. The QP list is ordered by
outstanding count on each post, unposted WRs are returned to the
cache when a QP is destroyed, and the receive queue number is
allocated from a bitmap in the core device rather than through
firmware.
AHs are software-only address vectors: the AV is filled from the
GRH and, for RoCE, the remote MAC, with the UDP source port derived
from the flow label (generated and written back when absent).
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/hw/xsc/Makefile | 2 +-
drivers/infiniband/hw/xsc/ah.c | 144 +++
drivers/infiniband/hw/xsc/cq.c | 940 ++++++++++++++++++
drivers/infiniband/hw/xsc/main.c | 15 +
drivers/infiniband/hw/xsc/qp.c | 257 +++++
drivers/infiniband/hw/xsc/srq.c | 462 +++++++++
drivers/infiniband/hw/xsc/xsc_ib.h | 20 +-
.../net/ethernet/yunsilicon/xsc/pci/Makefile | 3 +-
.../net/ethernet/yunsilicon/xsc/pci/main.c | 12 +-
drivers/net/ethernet/yunsilicon/xsc/pci/srq.c | 92 ++
drivers/net/ethernet/yunsilicon/xsc/pci/srq.h | 16 +
11 files changed, 1959 insertions(+), 4 deletions(-)
create mode 100644 drivers/infiniband/hw/xsc/ah.c
create mode 100644 drivers/infiniband/hw/xsc/cq.c
create mode 100644 drivers/infiniband/hw/xsc/qp.c
create mode 100644 drivers/infiniband/hw/xsc/srq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/srq.c
create mode 100644 drivers/net/ethernet/yunsilicon/xsc/pci/srq.h
diff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile
index b535a6653..d5ad0e573 100644
--- a/drivers/infiniband/hw/xsc/Makefile
+++ b/drivers/infiniband/hw/xsc/Makefile
@@ -6,4 +6,4 @@ ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o
-xsc_ib-y := main.o mem.o mr.o
+xsc_ib-y := main.o cq.o qp.o mem.o mr.o srq.o ah.o
diff --git a/drivers/infiniband/hw/xsc/ah.c b/drivers/infiniband/hw/xsc/ah.c
new file mode 100644
index 000000000..4d916dbe7
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/ah.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+
+static u32 xsc_calc_roce_udp_flow_label(void)
+{
+ /* Generate a non-zero 20 bit flow label. */
+ return get_random_u32_below(IB_GRH_FLOWLABEL_MASK) + 1;
+}
+
+static u16 xsc_ah_get_udp_sport(const struct xsc_ib_dev *dev,
+ struct rdma_ah_attr *ah_attr)
+{
+ enum ib_gid_type gid_type = ah_attr->grh.sgid_attr->gid_type;
+ u16 sport = 0;
+ u32 fl = 0;
+
+ if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP &&
+ (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) &&
+ (ah_attr->grh.flow_label & IB_GRH_FLOWLABEL_MASK)) {
+ fl = ah_attr->grh.flow_label;
+ } else {
+ /*generate a 20bit flow_label and output to user layer*/
+ fl = xsc_calc_roce_udp_flow_label();
+ ah_attr->grh.flow_label = fl;
+ }
+
+ sport = rdma_flow_label_to_udp_sport(fl);
+ ibdev_dbg(&dev->ib_dev, "fl=0x%x,sport=0x%x\n", fl, sport);
+ return sport;
+}
+
+static struct ib_ah *create_ib_ah(struct xsc_ib_dev *dev,
+ struct xsc_ib_ah *ah,
+ struct rdma_ah_attr *ah_attr)
+{
+ enum ib_gid_type gid_type;
+
+ if (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) {
+ const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
+
+ memcpy(ah->av.rgid, &grh->dgid, 16);
+ ah->av.grh_gid_fl = cpu_to_be32(grh->flow_label |
+ (1 << 30) |
+ grh->sgid_index << 20);
+ ah->av.hop_limit = grh->hop_limit;
+ ah->av.tclass = grh->traffic_class;
+ }
+
+ ah->av.stat_rate_sl = (rdma_ah_get_static_rate(ah_attr) << 4);
+
+ if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ gid_type = ah_attr->grh.sgid_attr->gid_type;
+
+ memcpy(ah->av.rmac, ah_attr->roce.dmac,
+ sizeof(ah_attr->roce.dmac));
+
+ ah->av.udp_sport = xsc_ah_get_udp_sport(dev, ah_attr);
+ ah->av.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) & 0x7) << 1;
+ if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP)
+#define XSC_ECN_ENABLED BIT(1)
+ ah->av.tclass |= XSC_ECN_ENABLED;
+ } else {
+ ah->av.rlid = cpu_to_be16(rdma_ah_get_dlid(ah_attr));
+ ah->av.fl_mlid = rdma_ah_get_path_bits(ah_attr) & 0x7f;
+ ah->av.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) & 0xf);
+ }
+
+ return &ah->ibah;
+}
+
+int xsc_ib_create_ah(struct ib_ah *ibah,
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_ah *ah = to_xah(ibah);
+ struct xsc_ib_dev *dev = to_xdev(ibah->device);
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
+ enum rdma_ah_attr_type ah_type = ah_attr->type;
+
+ if (ah_type == RDMA_AH_ATTR_TYPE_ROCE &&
+ !(rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH))
+ return -EINVAL;
+
+ if (ah_type == RDMA_AH_ATTR_TYPE_ROCE && udata) {
+ int err;
+ struct xsc_ib_create_ah_resp resp = {};
+ u32 min_resp_len = offsetof(typeof(resp), dmac) +
+ sizeof(resp.dmac);
+
+ if (udata->outlen < min_resp_len)
+ return -EINVAL;
+
+ resp.response_length = min_resp_len;
+ memcpy(resp.dmac, ah_attr->roce.dmac, ETH_ALEN);
+ err = ib_copy_to_udata(udata, &resp, resp.response_length);
+ if (err)
+ return err;
+ }
+
+ create_ib_ah(dev, ah, ah_attr); /* never fails */
+ return 0;
+}
+
+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr)
+{
+ struct xsc_ib_ah *ah = to_xah(ibah);
+ u32 tmp;
+
+ memset(ah_attr, 0, sizeof(*ah_attr));
+ ah_attr->type = ibah->type;
+
+ tmp = be32_to_cpu(ah->av.grh_gid_fl);
+ if (tmp & (1 << 30)) {
+ rdma_ah_set_grh(ah_attr, NULL, tmp & IB_GRH_FLOWLABEL_MASK,
+ (tmp >> 20) & 0xff, ah->av.hop_limit,
+ ah->av.tclass);
+ rdma_ah_set_dgid_raw(ah_attr, ah->av.rgid);
+ }
+
+ rdma_ah_set_static_rate(ah_attr, ah->av.stat_rate_sl >> 4);
+ if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ rdma_ah_set_sl(ah_attr, (ah->av.stat_rate_sl >> 1) & 0x7);
+ } else {
+ rdma_ah_set_dlid(ah_attr, be16_to_cpu(ah->av.rlid));
+ rdma_ah_set_path_bits(ah_attr, ah->av.fl_mlid & 0x7f);
+ rdma_ah_set_sl(ah_attr, ah->av.stat_rate_sl & 0xf);
+ }
+
+ return 0;
+}
+
+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags)
+{
+ return 0;
+}
diff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c
new file mode 100644
index 000000000..36d122f8d
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/cq.c
@@ -0,0 +1,940 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kref.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+#include "common/xsc_hsi.h"
+#include <linux/dma-direct.h>
+
+#ifndef IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN
+#define IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN BIT(1)
+#endif
+
+enum {
+ CQ_OK = 0,
+ CQ_EMPTY = -1,
+ CQ_POLL_ERR = -2
+};
+
+enum {
+ XSC_CQE_APP_TAG_MATCHING = 1,
+};
+
+enum {
+ XSC_CQE_APP_OP_TM_CONSUMED = 0x1,
+ XSC_CQE_APP_OP_TM_EXPECTED = 0x2,
+ XSC_CQE_APP_OP_TM_UNEXPECTED = 0x3,
+ XSC_CQE_APP_OP_TM_NO_TAG = 0x4,
+ XSC_CQE_APP_OP_TM_APPEND = 0x5,
+ XSC_CQE_APP_OP_TM_REMOVE = 0x6,
+ XSC_CQE_APP_OP_TM_NOOP = 0x7,
+ XSC_CQE_APP_OP_TM_CONSUMED_SW_RDNV = 0x9,
+ XSC_CQE_APP_OP_TM_CONSUMED_MSG = 0xA,
+ XSC_CQE_APP_OP_TM_CONSUMED_MSG_SW_RDNV = 0xB,
+ XSC_CQE_APP_OP_TM_MSG_COMPLETION_CANCELED = 0xC,
+};
+
+static const u32 xsc_cqe_opcode[] = {
+ [XSC_OPCODE_RDMA_REQ_SEND] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_REQ_SEND_IMMDT] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_RSP_RECV] = IB_WC_RECV,
+ [XSC_OPCODE_RDMA_RSP_RECV_IMMDT] = IB_WC_RECV,
+ [XSC_OPCODE_RDMA_REQ_WRITE] = IB_WC_RDMA_WRITE,
+ [XSC_OPCODE_RDMA_REQ_WRITE_IMMDT] = IB_WC_RDMA_WRITE,
+ [XSC_OPCODE_RDMA_RSP_WRITE_IMMDT] = IB_WC_RECV_RDMA_WITH_IMM,
+ [XSC_OPCODE_RDMA_REQ_READ] = IB_WC_RDMA_READ,
+ [XSC_OPCODE_RDMA_MAD_REQ_SEND] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_MAD_RSP_RECV] = IB_WC_RECV,
+};
+
+static void xsc_ib_cq_comp(struct xsc_core_cq *cq)
+{
+ struct ib_cq *ibcq = &core_to_xcq(cq)->ibcq;
+
+ ibcq->comp_handler(ibcq, ibcq->cq_context);
+}
+
+static void xsc_ib_cq_event(struct xsc_core_cq *xcq, enum xsc_event type)
+{
+ struct xsc_ib_cq *cq = container_of(xcq, struct xsc_ib_cq, xcq);
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ struct ib_cq *ibcq = &cq->ibcq;
+ struct ib_event event;
+
+ if (type != XSC_EVENT_TYPE_CQ_ERROR) {
+ ibdev_err(&dev->ib_dev, "Unexpected event type %d on CQ %06x\n",
+ type, xcq->cqn);
+ return;
+ }
+
+ if (ibcq->event_handler) {
+ event.device = &dev->ib_dev;
+ event.event = IB_EVENT_CQ_ERR;
+ event.element.cq = ibcq;
+ ibcq->event_handler(&event, ibcq->cq_context);
+ pr_info("xsc_ib: xsc async event 0x%x, ib event %d\n",
+ XSC_EVENT_TYPE_CQ_ERROR, IB_EVENT_CQ_ERR);
+ }
+}
+
+static void *get_cqe_from_buf(struct xsc_ib_cq_buf *buf, int n, int size)
+{
+ return xsc_buf_offset(&buf->buf, n * size);
+}
+
+static void *get_cqe(struct xsc_ib_cq *cq, int n)
+{
+ return get_cqe_from_buf(&cq->buf, n, cq->xcq.cqe_sz);
+}
+
+static void *get_sw_cqe(struct xsc_ib_cq *cq, int n)
+{
+ struct xsc_cqe *cqe;
+
+ cqe = (struct xsc_cqe *)get_cqe(cq, n & (cq->ibcq.cqe - 1));
+
+ return (!!(le32_to_cpu(cqe->data1) & XSC_CQE_DATA1_OWNER) ^
+ !!(n & cq->ibcq.cqe)) ? NULL : cqe;
+}
+
+static inline void handle_good_req(struct ib_wc *wc,
+ struct xsc_cqe *cqe,
+ u8 opcode)
+{
+ wc->opcode = xsc_cqe_opcode[opcode];
+ if (opcode == XSC_OPCODE_RDMA_REQ_READ)
+ wc->byte_len = RD_LE_32(cqe->msg_len);
+ wc->status = IB_WC_SUCCESS;
+}
+
+static void handle_responder(struct ib_wc *wc, struct xsc_cqe *cqe,
+ struct xsc_ib_qp *qp, u8 opcode)
+{
+ struct xsc_ib_wq *wq = &qp->rq;
+ u16 idx;
+
+ wc->byte_len = RD_LE_32(cqe->msg_len);
+ wc->opcode = xsc_cqe_opcode[opcode];
+ wc->status = IB_WC_SUCCESS;
+
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ wc->wr_id = wq->wrid[idx];
+ atomic_dec(&wq->flush_wqe_cnt);
+ ++wq->tail;
+}
+
+static void *get_wqe(struct xsc_ib_qp *qp, int offset)
+{
+ return xsc_buf_offset(&qp->buf, offset);
+}
+
+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->rq.offset + (n << qp->rq.wqe_shift));
+}
+
+static void *get_seg_wqe(void *first, int n)
+{
+ return first + (n << XSC_BASE_WQE_SHIFT);
+}
+
+static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
+ struct xsc_ib_qp **cur_qp,
+ struct ib_wc *wc,
+ struct xsc_cqe *cqe,
+ u8 opcode)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ void *recv;
+ struct xsc_wqe_data_seg *data_seg;
+ struct iphdr *ip4h = NULL;
+ struct ipv6hdr *ip6h;
+ struct udphdr *udph;
+ struct ib_unpacked_eth *eth;
+ struct ib_unpacked_vlan *vlan;
+ struct ib_grh *grh;
+ struct ib_mad *mad;
+ struct rxe_bth *bth;
+ struct rxe_deth *deth;
+ unsigned int pading_sz = 0;
+ struct xsc_ib_wq *wq;
+ int idx;
+ u16 eth_type;
+ void *l3_start;
+
+ wq = &(*cur_qp)->rq;
+ idx = wq->tail & (wq->wqe_cnt - 1);
+
+ handle_responder(wc, cqe, *cur_qp, opcode);
+
+ data_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);
+ recv = xsc_ib_recv_mad_sg_virt_addr(&dev->ib_dev, wc, data_seg->va);
+
+ eth = (struct ib_unpacked_eth *)recv;
+ grh = (struct ib_grh *)recv;
+ if (eth->type == htons(ETH_P_8021Q)) {
+ vlan = (struct ib_unpacked_vlan *)(eth + 1);
+ eth_type = ntohs(vlan->type);
+ l3_start = vlan + 1;
+
+ wc->vlan_id = ntohs(vlan->tag) & 0x0fff;
+ wc->sl = (ntohs(vlan->tag) >> 13) & 0x7;
+ wc->wc_flags |= IB_WC_WITH_VLAN;
+ } else {
+ eth_type = ntohs(eth->type);
+ l3_start = eth + 1;
+ }
+
+ if (eth_type == ETH_P_IP) {
+ ip4h = (struct iphdr *)l3_start;
+ udph = (struct udphdr *)(ip4h + 1);
+ } else {
+ ip6h = (struct ipv6hdr *)l3_start;
+ udph = (struct udphdr *)(ip6h + 1);
+ }
+ bth = (struct rxe_bth *)(udph + 1);
+ deth = (struct rxe_deth *)(bth + 1);
+ mad = (struct ib_mad *)(deth + 1);
+
+ if (eth_type == ETH_P_IP) {
+ pading_sz = sizeof(*grh) - sizeof(*ip4h);
+ memmove((u8 *)(grh + 1) - sizeof(*ip4h), ip4h, sizeof(*ip4h));
+ memset(grh, 0, pading_sz);
+ } else {
+ memmove(grh, ip6h, sizeof(*ip6h));
+ }
+ memmove(grh + 1, mad, sizeof(*mad));
+
+ wc->wc_flags |= IB_WC_GRH;
+
+ ibdev_dbg(&dev->ib_dev, "recv cqe idx:%u, len:%u\n", wq->tail, wc->byte_len);
+ ibdev_dbg(&dev->ib_dev, "qp[%d] recv MAD packet, msg_len=%d\n",
+ (*cur_qp)->xqp.qpn, wc->byte_len);
+ wc->status = IB_WC_SUCCESS;
+}
+
+static void xsc_generate_lost_cqe(struct xsc_ib_dev *dev, struct xsc_ib_wq *wq,
+ int qpn, int idx, struct ib_wc *wc)
+{
+ int lost_cqe_idx = wq->prev_ce_wqe[idx];
+
+ ibdev_err(&dev->ib_dev, "detected lost cqe, cur wqe idx of qp%d is %d\n", qpn, idx);
+
+ while (wq->prev_ce_wqe[lost_cqe_idx] != wq->last_comp_ce_wqe)
+ lost_cqe_idx = wq->prev_ce_wqe[lost_cqe_idx];
+ ibdev_err(&dev->ib_dev, "generate lost cqe idx of qp%d is %d\n", qpn, lost_cqe_idx);
+
+ wq->last_comp_ce_wqe = lost_cqe_idx;
+ wc->wr_id = wq->wrid[lost_cqe_idx];
+ wq->tail = wq->wqe_head[lost_cqe_idx] + 1;
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[lost_cqe_idx] = 0;
+ wc->status = IB_WC_GENERAL_ERR;
+}
+
+static int xsc_poll_one(struct xsc_ib_cq *cq,
+ struct xsc_ib_qp **cur_qp,
+ struct ib_wc *wc, bool *trigger)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ struct xsc_core_qp *xqp;
+ struct xsc_ib_wq *wq;
+ u8 opcode;
+ u32 qpn;
+ int idx;
+ int ret;
+ struct xsc_cqe *cqe;
+ u32 *p = NULL;
+ struct xsc_ib_srq *xsrq;
+
+ *trigger = false;
+ cqe = get_sw_cqe(cq, cq->xcq.cons_index);
+ if (!cqe)
+ return -EAGAIN;
+
+ ++cq->xcq.cons_index;
+
+ /* Make sure we read CQ entry contents after we've checked the
+ * ownership bit.
+ */
+ rmb();
+
+ p = (u32 *)cqe;
+
+ qpn = FIELD_GET(XSC_CQE_DATA0_QP_ID_MASK, le32_to_cpu(cqe->data0));
+ if (!*cur_qp || (qpn != (*cur_qp)->ibqp.qp_num)) {
+ /* We do not have to take the QP table lock here,
+ * because CQs will be locked while QPs are removed
+ * from the table.
+ */
+ xqp = __xsc_qp_lookup(dev->xdev, qpn);
+ if (unlikely(!xqp)) {
+ ibdev_warn(&dev->ib_dev, "CQE@CQ %d for unknown QPN %d\n",
+ cq->xcq.cqn, qpn);
+ return -EINVAL;
+ }
+
+ *cur_qp = core_to_xqp(xqp);
+ }
+
+ memset(wc, 0, sizeof(*wc));
+ wc->qp = &(*cur_qp)->ibqp;
+ opcode = xsc_get_cqe_opcode(dev->xdev, cqe);
+ switch (opcode) {
+ case XSC_OPCODE_RDMA_REQ_SEND_IMMDT:
+ case XSC_OPCODE_RDMA_REQ_WRITE_IMMDT:
+ wc->wc_flags |= IB_WC_WITH_IMM;
+ fallthrough;
+ case XSC_OPCODE_RDMA_REQ_SEND:
+ case XSC_OPCODE_RDMA_REQ_WRITE:
+ case XSC_OPCODE_RDMA_REQ_READ:
+ case XSC_OPCODE_RDMA_MAD_REQ_SEND:
+ wq = &(*cur_qp)->sq;
+ idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
+ idx &= (wq->wqe_cnt - 1);
+ if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
+ cq->invalid = 1;
+ break;
+ }
+ wq->last_comp_ce_wqe = idx;
+ handle_good_req(wc, cqe, opcode);
+ wc->wr_id = wq->wrid[idx];
+ wq->tail = wq->wqe_head[idx] + 1;
+ if (opcode != XSC_OPCODE_RDMA_MAD_REQ_SEND)
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[idx] = 0;
+ ibdev_dbg(&dev->ib_dev, "wqeid:%u, wq tail:%u qpn:%u\n", idx, wq->tail, qpn);
+ wc->status = IB_WC_SUCCESS;
+ break;
+ case XSC_OPCODE_RDMA_RSP_RECV_IMMDT:
+ case XSC_OPCODE_RDMA_RSP_WRITE_IMMDT:
+ wc->wc_flags |= IB_WC_WITH_IMM;
+ WR_BE_32(wc->ex.imm_data, RD_LE_32(cqe->imm_data));
+ fallthrough;
+ case XSC_OPCODE_RDMA_RSP_RECV:
+ wq = &(*cur_qp)->rq;
+ handle_responder(wc, cqe, *cur_qp, opcode);
+ if ((*cur_qp)->ibqp.srq) {
+ xsrq = to_xsrq((*cur_qp)->ibqp.srq);
+ xsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc->wr_id);
+ }
+ ibdev_dbg(&dev->ib_dev, "recv cqe idx:%u, len:%u, qpn:%u\n",
+ wq->tail, wc->byte_len, qpn);
+ wc->status = IB_WC_SUCCESS;
+ break;
+
+ case XSC_OPCODE_RDMA_MAD_RSP_RECV:
+ ibdev_dbg(&dev->ib_dev, "recv MAD, qpn:%u\n", qpn);
+ xsc_handle_rdma_mad_resp_recv(cq, cur_qp, wc, cqe, opcode);
+ break;
+
+ case XSC_OPCODE_RDMA_REQ_ERROR:
+ wq = &(*cur_qp)->sq;
+ idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
+ idx &= (wq->wqe_cnt - 1);
+
+ if (!wq->need_flush[idx]) {
+ unsigned int i;
+ unsigned int tmp;
+ unsigned int prev = -1;
+
+ for (i = wq->tail; i != wq->head; i++) {
+ tmp = i & (wq->wqe_cnt - 1);
+ if (tmp == idx)
+ break;
+ if (wq->need_flush[tmp])
+ prev = tmp;
+ }
+
+ if (prev != -1) {
+ if (wq->prev_ce_wqe[prev] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, prev, wc);
+ cq->invalid = 1;
+ break;
+ }
+ }
+ } else {
+ if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
+ cq->invalid = 1;
+ break;
+ }
+ }
+
+ wc->wr_id = wq->wrid[idx];
+ wq->tail = wq->wqe_head[idx] + 1;
+ if (wq->need_flush[idx])
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[idx] = 0;
+ ibdev_err(&dev->ib_dev, "req error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ ret = xsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)->state,
+ XSC_QP_STATE_ERR, trigger);
+ if (ret)
+ ibdev_err(&dev->ib_dev,
+ "flush registration failed for qpn %d on cqn %d, err %d\n",
+ qpn, cq->xcq.cqn, ret);
+ break;
+ case XSC_OPCODE_RDMA_RSP_ERROR:
+ wq = &(*cur_qp)->rq;
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ wc->wr_id = wq->wrid[idx];
+ wq->tail++;
+ atomic_dec(&wq->flush_wqe_cnt);
+ if ((*cur_qp)->ibqp.srq) {
+ xsrq = to_xsrq((*cur_qp)->ibqp.srq);
+ xsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc->wr_id);
+ }
+ ibdev_err(&dev->ib_dev, "rsp error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ ret = xsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)->state,
+ XSC_QP_STATE_ERR, trigger);
+ if (ret)
+ ibdev_err(&dev->ib_dev,
+ "flush registration failed for qpn %d on cqn %d, err %d\n",
+ qpn, cq->xcq.cqn, ret);
+ break;
+
+ default:
+ ibdev_err(&dev->ib_dev, "completion error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ wc->wr_id = 0;
+ break;
+ }
+
+ return 0;
+}
+
+static inline void gen_flush_err_cqe(struct xsc_err_state_qp_node *err_node,
+ struct ib_qp *ibqp, struct xsc_ib_wq *wq, u32 idx,
+ struct ib_wc *wc)
+{
+ memset(wc, 0, sizeof(*wc));
+ if (err_node->is_sq) {
+ switch (wq->wr_opcode[idx]) {
+ case IB_WR_SEND:
+ case IB_WR_SEND_WITH_IMM:
+ wc->opcode = IB_WC_SEND;
+ break;
+ case IB_WR_RDMA_WRITE:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ wc->opcode = IB_WC_RDMA_WRITE;
+ break;
+ case IB_WR_RDMA_READ:
+ wc->opcode = IB_WC_RDMA_READ;
+ }
+ } else {
+ wc->opcode = IB_WC_RECV;
+ }
+
+ wc->qp = ibqp;
+ wc->status = IB_WC_WR_FLUSH_ERR;
+ wc->vendor_err = XSC_ERR_CODE_FLUSH;
+ wc->wr_id = wq->wrid[idx];
+ wq->tail++;
+ atomic_dec(&wq->flush_wqe_cnt);
+ if (err_node->is_sq)
+ wq->need_flush[idx] = 0;
+}
+
+static inline int xsc_generate_flush_err_cqe(struct ib_cq *ibcq,
+ int ne, int *npolled, struct ib_wc *wc)
+{
+ u32 qp_id = 0;
+ int flush_wqe_cnt = 0;
+ int sw_npolled = 0;
+ u32 idx = 0;
+ struct xsc_err_state_qp_node *err_qp_node;
+ struct xsc_core_qp *xqp;
+ struct xsc_ib_cq *cq = to_xcq(ibcq);
+ struct xsc_ib_wq *wq;
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ unsigned long flags;
+
+ spin_lock_irqsave(&cq->err_state_qp_lock, flags);
+ list_for_each_entry(err_qp_node, &cq->err_state_qp_list, entry) {
+ if (!err_qp_node)
+ break;
+
+ sw_npolled = 0;
+ qp_id = err_qp_node->qp_id;
+ xqp = __xsc_qp_lookup(dev->xdev, qp_id);
+ if (unlikely(!xqp)) {
+ ibdev_warn(&dev->ib_dev, "CQE@CQ %d for unknown QPN %d\n",
+ cq->xcq.cqn, qp_id);
+ continue;
+ }
+ wq = err_qp_node->is_sq ? &(core_to_xqp(xqp)->sq) : &(core_to_xqp(xqp)->rq);
+ flush_wqe_cnt = atomic_read(&wq->flush_wqe_cnt);
+ ibdev_dbg(&dev->ib_dev, "is_sq %d, flush_wq_cnt %d, ne %d, npolled %d, qp_id %d, cqn %d\n",
+ err_qp_node->is_sq, flush_wqe_cnt, ne, *npolled, qp_id, cq->xcq.cqn);
+
+ while (wq->tail != wq->head) {
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ if (err_qp_node->is_sq && !wq->need_flush[idx]) {
+ wq->tail++;
+ continue;
+ } else {
+ gen_flush_err_cqe(err_qp_node, &(core_to_xqp(xqp)->ibqp), wq,
+ idx, wc + *npolled + sw_npolled);
+ ++sw_npolled;
+ }
+
+ if (sw_npolled == ne - *npolled)
+ break;
+ }
+ *npolled += sw_npolled;
+ if (*npolled == ne)
+ break;
+ }
+ spin_unlock_irqrestore(&cq->err_state_qp_lock, flags);
+
+ return 0;
+}
+
+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
+{
+ struct xsc_ib_cq *cq = to_xcq(ibcq);
+ struct xsc_core_cq *xcq = &cq->xcq;
+ struct xsc_ib_qp *cur_qp = NULL;
+ bool trigger = false;
+ int npolled = 0;
+ int err = 0;
+ unsigned long flags;
+ u32 next_cid;
+
+ spin_lock_irqsave(&cq->lock, flags);
+ next_cid = xcq->cons_index;
+
+ for (npolled = 0; npolled < num_entries; npolled++) {
+ if (cq->invalid) {
+ err = -EINVAL;
+ break;
+ }
+ err = xsc_poll_one(cq, &cur_qp, wc + npolled, &trigger);
+ if (err)
+ break;
+ if (trigger) {
+ npolled++;
+ break;
+ }
+ }
+
+ if (err) {
+ if (npolled < num_entries && !(list_empty(&cq->err_state_qp_list)))
+ xsc_generate_flush_err_cqe(ibcq, num_entries, &npolled, wc);
+ }
+
+ if (next_cid != xcq->cons_index)
+ xsc_update_cq_ci(xcq->dev, xcq->cqn, xcq->cons_index);
+
+ spin_unlock_irqrestore(&cq->lock, flags);
+
+ if (trigger)
+ xsc_ib_trigger_cq_comp_handler(cur_qp->ibqp.send_cq,
+ cur_qp->ibqp.recv_cq);
+
+ return npolled;
+}
+
+static int xsc_cqe_is_empty(struct xsc_ib_cq *cq)
+{
+ struct xsc_cqe *cqe = get_sw_cqe(cq, cq->xcq.cons_index);
+
+ if (cq->invalid)
+ return 1;
+
+ if (!cqe)
+ return 1;
+
+ return 0;
+}
+
+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
+{
+ struct xsc_ib_cq *xcq = to_xcq(ibcq);
+ struct xsc_core_cq *cq = &xcq->xcq;
+ int ret = 0;
+ unsigned long irq_flags;
+ u8 solicited = 0;
+
+ spin_lock_irqsave(&xcq->lock, irq_flags);
+ if (flags & IB_CQ_NEXT_COMP)
+ solicited = 0;
+ else if (flags & IB_CQ_SOLICITED)
+ solicited = 1;/* arm next:0 arm solicited:1 */
+
+ if ((flags & IB_CQ_REPORT_MISSED_EVENTS) && (!xsc_cqe_is_empty(xcq))) {
+ ret = 1;
+ goto out;
+ }
+
+ xsc_arm_cq(cq->dev, cq->cqn, cq->cons_index, solicited);
+out:
+ spin_unlock_irqrestore(&xcq->lock, irq_flags);
+ return ret;
+}
+
+static int alloc_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf,
+ int nent, int cqe_size)
+{
+ int err;
+
+ err = xsc_buf_alloc(dev->xdev, nent * cqe_size,
+ PAGE_SIZE, &buf->buf);
+ if (err)
+ return err;
+
+ buf->cqe_size = cqe_size;
+
+ return 0;
+}
+
+static void free_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf)
+{
+ xsc_buf_free(dev->xdev, &buf->buf);
+}
+
+static struct ib_umem *cq_umem_get(struct xsc_ib_dev *dev,
+ struct ib_udata *udata, struct ib_ucontext *context,
+ int entries, struct xsc_ib_create_cq *ucmd)
+{
+ struct ib_umem *umem;
+
+ umem = ib_umem_get_va(&dev->ib_dev, ucmd->buf_addr,
+ entries * ucmd->cqe_size,
+ IB_ACCESS_LOCAL_WRITE);
+ return umem;
+}
+
+static int create_cq_user(struct xsc_ib_dev *dev, struct ib_udata *udata,
+ struct ib_ucontext *context, struct xsc_ib_cq *cq,
+ int entries, struct xsc_create_cq_ex_mbox_in **cqb,
+ int *cqe_size, int *index, int *inlen)
+{
+ struct xsc_ib_create_cq ucmd;
+ int page_shift;
+ int npages;
+ int ncont;
+ int err;
+ int log_cq_sz;
+ int hw_npages;
+ struct ib_umem_dmabuf *umem_dmabuf;
+
+ if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
+ return -EFAULT;
+
+ if (ucmd.cqe_size != sizeof(struct xsc_cqe))
+ return -EINVAL;
+
+ *cqe_size = ucmd.cqe_size;
+
+ if (ucmd.dmabuf_fd != -1) {
+ size_t cq_ring_sz = (size_t)entries * ucmd.cqe_size;
+
+ if (ucmd.dmabuf_sz < cq_ring_sz)
+ return -EINVAL;
+
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd.dmabuf_sz,
+ ucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
+
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_warn(&dev->ib_dev, "umem_dmabuf get failed by fd %d (%ld)\n",
+ ucmd.dmabuf_fd, PTR_ERR(umem_dmabuf));
+ return -ENOMEM;
+ }
+
+ cq->buf.umem = &umem_dmabuf->umem;
+ ibdev_dbg(&dev->ib_dev, "get umem_dmabuf success by fd %d, umem_dmabuf_addr %p\n",
+ ucmd.dmabuf_fd, umem_dmabuf);
+ } else {
+ cq->buf.umem = cq_umem_get(dev, udata, context, entries, &ucmd);
+ }
+
+ if (IS_ERR(cq->buf.umem)) {
+ err = PTR_ERR(cq->buf.umem);
+ return err;
+ }
+
+ xsc_ib_cont_pages(cq->buf.umem, ucmd.buf_addr, &npages, &page_shift,
+ &ncont);
+ if (ncont != npages) {
+ ibdev_dbg(&dev->ib_dev, "bad page_shift:%d, ncont:%d\n", page_shift, ncont);
+ /* amber doesn't support compound pages */
+ page_shift = PAGE_SHIFT;
+ ncont = npages;
+ ibdev_dbg(&dev->ib_dev, "overwrite to page_shift:%d, ncont:%d\n",
+ page_shift, ncont);
+ }
+ log_cq_sz = ilog2(entries);
+ hw_npages = DIV_ROUND_UP((1 << log_cq_sz) * (*cqe_size), XSC_PAGE_SIZE_4K);
+ ibdev_dbg(&dev->ib_dev, "addr 0x%llx, entries %d, npages %d, hw_npages %d\n",
+ ucmd.buf_addr, entries, npages, hw_npages);
+
+ *inlen = sizeof(**cqb) + sizeof(*(*cqb)->pas) * hw_npages;
+ *cqb = xsc_vzalloc(*inlen);
+ if (!*cqb) {
+ err = -ENOMEM;
+ goto err_umem;
+ }
+ xsc_ib_populate_pas(dev, cq->buf.umem, page_shift, (*cqb)->pas, hw_npages, true);
+ (*cqb)->ctx_ex.ctx.pa_num = cpu_to_be16(hw_npages);
+ (*cqb)->ctx_ex.pa_num_ex = cpu_to_be32(hw_npages);
+
+ return 0;
+
+err_umem:
+ ib_umem_release(cq->buf.umem);
+ return err;
+}
+
+static void destroy_cq_user(struct xsc_ib_cq *cq, struct ib_udata *udata)
+{
+ ib_umem_release(cq->buf.umem);
+}
+
+static int create_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq,
+ int entries, int cqe_size,
+ struct xsc_create_cq_ex_mbox_in **cqb,
+ int *index, int *inlen)
+{
+ int err;
+ int i = 0;
+ struct xsc_cqe *cqe = NULL;
+ int hw_npages;
+
+ cq->xcq.cqe_sz = cqe_size;
+
+ err = alloc_cq_buf(dev, &cq->buf, entries, cqe_size);
+ if (err)
+ return err;
+
+ for (i = 0; i < entries; i++) {
+ cqe = (struct xsc_cqe *)get_cqe(cq, i);
+ cqe->data1 |= cpu_to_le32(XSC_CQE_DATA1_OWNER);
+ }
+
+ hw_npages = DIV_ROUND_UP(entries * cqe_size, XSC_PAGE_SIZE_4K);
+ *inlen = sizeof(**cqb) + sizeof(*(*cqb)->pas) * hw_npages;
+ *cqb = xsc_vzalloc(*inlen);
+ if (!*cqb) {
+ err = -ENOMEM;
+ goto err_buf;
+ }
+ xsc_fill_page_array(&cq->buf.buf, (*cqb)->pas, hw_npages);
+ (*cqb)->ctx_ex.ctx.pa_num = cpu_to_be16(hw_npages);
+ (*cqb)->ctx_ex.pa_num_ex = cpu_to_be32(hw_npages);
+
+ return 0;
+
+err_buf:
+ free_cq_buf(dev, &cq->buf);
+ return err;
+}
+
+static void destroy_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq)
+{
+ free_cq_buf(dev, &cq->buf);
+}
+
+int xsc_ib_create_cq(struct ib_cq *ibcq,
+ const struct ib_cq_init_attr *attr,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_device *ibdev = ibcq->device;
+ struct ib_udata *udata = attrs ? &attrs->driver_udata : NULL;
+ int entries = attr->cqe;
+ int vector = attr->comp_vector;
+ struct xsc_create_cq_ex_mbox_in *cqb = NULL;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_cq *cq;
+ int index;
+ int inlen = 0;
+ int cqe_size;
+ int irqn;
+ int err;
+ unsigned int eqn;
+
+ entries = roundup_pow_of_two(entries);
+
+ ibdev_dbg(&dev->ib_dev, "entries:%d, vector:%d, max_cqes:%d\n", entries, vector,
+ dev->xdev->caps.max_cqes);
+
+ if (entries > dev->xdev->caps.max_cqes)
+ entries = dev->xdev->caps.max_cqes;
+ cq = to_xcq(ibcq);
+ cq->ibcq.cqe = entries;
+ mutex_init(&cq->resize_mutex);
+ spin_lock_init(&cq->lock);
+ cq->resize_buf = NULL;
+ cq->resize_umem = NULL;
+ cq->create_flags = attr->flags;
+ cq->invalid = 0;
+
+ if (udata) {
+ err = create_cq_user(dev, udata, NULL, cq, entries,
+ &cqb, &cqe_size, &index, &inlen);
+ if (err)
+ goto err_create;
+ } else {
+ cqe_size = is_support_cqe64(dev->xdev) ?
+ sizeof(struct xsc_cqe64) : sizeof(struct xsc_cqe);
+ err = create_cq_kernel(dev, cq, entries, cqe_size, &cqb, &index, &inlen);
+ if (err)
+ goto err_create;
+ }
+
+ cq->cqe_size = cqe_size;
+ cqb->ctx_ex.ctx.log_cq_sz = ilog2(entries);
+ cqb->ctx_ex.ctx.glb_func_id = cpu_to_be16(dev->xdev->glb_func_id);
+
+ err = xsc_vector2eqn(dev->xdev, vector, &eqn, &irqn);
+ if (err)
+ goto err_cqb;
+
+ cqb->ctx_ex.ctx.eqn = eqn;
+ cqb->ctx_ex.ctx.eqn = cpu_to_be16(cqb->ctx_ex.ctx.eqn);
+ cqb->ctx_ex.page_shift = XSC_PAGE_SHIFT_4K;
+ if (cq->create_flags & IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN)
+ cqb->ctx_ex.overrun_ignore = 1;
+
+ /* the cq becomes visible to the event handlers once
+ * xsc_core_create_cq() inserts it into the tree, so the
+ * callbacks must be in place before that
+ */
+ cq->xcq.comp = xsc_ib_cq_comp;
+ cq->xcq.event = xsc_ib_cq_event;
+
+ err = xsc_core_create_cq(dev->xdev, &cq->xcq, cqb, inlen);
+ if (err)
+ goto err_cqb;
+
+ ibdev_dbg(&dev->ib_dev, "succeeded to create cqn %d, vector=%d, cq_sz=%d, eqn=%d\n",
+ cq->xcq.cqn, vector, entries, eqn);
+ cq->xcq.irqn = irqn;
+
+ if (udata) {
+ if (ib_copy_to_udata(udata, &cq->xcq.cqn, sizeof(__u32))) {
+ err = -EFAULT;
+ goto err_cmd;
+ }
+ }
+
+ INIT_LIST_HEAD(&cq->send_qp_list);
+ INIT_LIST_HEAD(&cq->recv_qp_list);
+ INIT_LIST_HEAD(&cq->err_state_qp_list);
+ spin_lock_init(&cq->err_state_qp_lock);
+ xsc_vfree(cqb);
+
+ return 0;
+
+err_cmd:
+ xsc_core_destroy_cq(dev->xdev, &cq->xcq);
+
+err_cqb:
+ xsc_vfree(cqb);
+ if (udata)
+ destroy_cq_user(cq, udata);
+ else
+ destroy_cq_kernel(dev, cq);
+
+err_create:
+
+ return err;
+}
+
+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->device);
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_err_state_qp_node *tmp = NULL, *err_qp_node = NULL;
+ unsigned long flags;
+ int err;
+
+ err = xsc_core_destroy_cq(dev->xdev, &xcq->xcq);
+ if (err)
+ return err;
+
+ spin_lock_irqsave(&xcq->err_state_qp_lock, flags);
+ list_for_each_entry_safe(err_qp_node, tmp, &xcq->err_state_qp_list, entry) {
+ list_del(&err_qp_node->entry);
+ kfree(err_qp_node);
+ }
+ spin_unlock_irqrestore(&xcq->err_state_qp_lock, flags);
+
+ if (udata)
+ destroy_cq_user(xcq, udata);
+ else
+ destroy_cq_kernel(dev, xcq);
+
+ return 0;
+}
+
+static int is_equal_rsn(struct xsc_cqe *cqe, u32 rsn)
+{
+ u32 qpn = FIELD_GET(XSC_CQE_DATA0_QP_ID_MASK, le32_to_cpu(cqe->data0));
+ return rsn == qpn;
+}
+
+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 rsn)
+{
+ struct xsc_cqe *cqe, *dest;
+ u32 prod_index;
+ int nfreed = 0;
+ u8 owner_bit;
+
+ if (!cq)
+ return;
+
+ /* First we need to find the current producer index, so we
+ * know where to start cleaning from. It doesn't matter if HW
+ * adds new entries after this loop -- the QP we're worried
+ * about is already in RESET, so the new entries won't come
+ * from our QP and therefore don't need to be checked.
+ */
+ for (prod_index = cq->xcq.cons_index; get_sw_cqe(cq, prod_index); prod_index++)
+ if (prod_index == cq->xcq.cons_index + cq->ibcq.cqe)
+ break;
+
+ /* Now sweep backwards through the CQ, removing CQ entries
+ * that match our QP by copying older entries on top of them.
+ */
+ while ((int)(--prod_index) - (int)cq->xcq.cons_index >= 0) {
+ cqe = (struct xsc_cqe *)get_cqe(cq, prod_index & (cq->ibcq.cqe - 1));
+ if (is_equal_rsn(cqe, rsn)) {
+ ++nfreed;
+ } else if (nfreed) {
+ dest = (struct xsc_cqe *)get_cqe(cq, (prod_index + nfreed) &
+ (cq->ibcq.cqe - 1));
+ owner_bit = !!(le32_to_cpu(dest->data1) & XSC_CQE_DATA1_OWNER);
+ memcpy(dest, cqe, cq->xcq.cqe_sz);
+ dest->data1 = cpu_to_le32((le32_to_cpu(dest->data1) &
+ ~XSC_CQE_DATA1_OWNER) |
+ (owner_bit ? XSC_CQE_DATA1_OWNER : 0));
+ }
+ }
+
+ if (nfreed) {
+ cq->xcq.cons_index += nfreed;
+ xsc_update_cq_ci(cq->xcq.dev, cq->xcq.cqn, cq->xcq.cons_index);
+ }
+}
+
+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn)
+{
+ if (!cq)
+ return;
+
+ spin_lock_irq(&cq->lock);
+ __xsc_ib_cq_clean(cq, qpn);
+ spin_unlock_irq(&cq->lock);
+}
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
index a212a5888..c3f104615 100644
--- a/drivers/infiniband/hw/xsc/main.c
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -511,9 +511,16 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.alloc_pd = xsc_ib_alloc_pd;
dev->ib_dev.ops.dealloc_pd = xsc_ib_dealloc_pd;
+ dev->ib_dev.ops.create_ah = xsc_ib_create_ah;
+ dev->ib_dev.ops.query_ah = xsc_ib_query_ah;
+ dev->ib_dev.ops.destroy_ah = xsc_ib_destroy_ah;
dev->ib_dev.ops.get_link_layer = xsc_ib_port_link_layer;
+ dev->ib_dev.ops.create_cq = xsc_ib_create_cq;
+ dev->ib_dev.ops.destroy_cq = xsc_ib_destroy_cq;
+ dev->ib_dev.ops.poll_cq = xsc_ib_poll_cq;
+ dev->ib_dev.ops.req_notify_cq = xsc_ib_arm_cq;
dev->ib_dev.ops.get_dma_mr = xsc_ib_get_dma_mr;
dev->ib_dev.ops.reg_user_mr = xsc_ib_reg_user_mr; /* optional */
dev->ib_dev.ops.reg_user_mr_dmabuf = xsc_ib_reg_user_mr_dmabuf;
@@ -522,9 +529,17 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.get_port_immutable = xsc_port_immutable;
dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
+ dev->ib_dev.ops.create_srq = xsc_ib_create_srq;
+ dev->ib_dev.ops.destroy_srq = xsc_ib_destroy_srq;
+ dev->ib_dev.ops.query_srq = xsc_ib_query_srq;
+ dev->ib_dev.ops.modify_srq = xsc_ib_modify_srq;
+ dev->ib_dev.ops.post_srq_recv = xsc_ib_post_srq_recv;
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ucontext, xsc_ib_ucontext, ibucontext);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ah, xsc_ib_ah, ibah);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_cq, xsc_ib_cq, ibcq);
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_pd, xsc_ib_pd, ibpd);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_srq, xsc_ib_srq, ibsrq);
}
/*
diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
new file mode 100644
index 000000000..a41d880cf
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/qp.c
@@ -0,0 +1,257 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+#include "common/xsc_hsi.h"
+#include <linux/crc32.h>
+#include <linux/dma-direct.h>
+#include <rdma/ib_addr.h>
+
+#ifndef ETH_P_IBOE
+#define ETH_P_IBOE 0x8915
+#endif
+
+#define MAD_QUEUE_DEPTH 128
+
+enum {
+ XSC_IB_CACHE_LINE_SIZE = 64,
+};
+
+#define UDP_SPORT_MASK_EN 0x40000000
+#define UDP_SPORT_MASK_EN_OFFSET 30
+#define UDP_SPORT_MASK 0xffff
+#define UDP_SPORT_OFFSET 0
+
+static int is_qp1(enum ib_qp_type qp_type)
+{
+ return qp_type == IB_QPT_GSI;
+}
+
+static void *get_wqe(struct xsc_ib_qp *qp, int offset)
+{
+ return xsc_buf_offset(&qp->buf, offset);
+}
+
+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->rq.offset + (n << qp->rq.wqe_shift));
+}
+
+static void *get_seg_wqe(void *first, int n)
+{
+ return first + (n << XSC_BASE_WQE_SHIFT);
+}
+
+#define MAX_QP1_SQ_HDR_SIZE_V2 512
+#define MAX_QP1_SQ_HDR_SIZE 86
+ /* Ethernet header = 14 */
+ /* ib_grh = 40 (provided by MAD) */
+ /* ib_bth + ib_deth = 20 */
+ /* MAD = 256 (provided by MAD) */
+ /* iCRC = 4 */
+#define MAX_QP1_RQ_HDR_SIZE_V2 512
+
+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ enum xsc_qp_state cur_state, enum xsc_qp_state state,
+ bool *triggered)
+{
+ struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
+ struct xsc_ib_cq *send_xcq, *recv_xcq;
+ unsigned long flags;
+
+ *triggered = false;
+ ibdev_dbg(&dev->ib_dev, "modify qp: qpid %d, cur_qp_state %d, qp_state %d\n",
+ qp->xqp.qpn, cur_state, state);
+ if (cur_state == XSC_QP_STATE_ERR && state != XSC_QP_STATE_ERR) {
+ if (!qp->ibqp.srq && qp->recv_cq) {
+ recv_xcq = to_xcq(qp->recv_cq);
+ spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
+ list_for_each_entry_safe(err_rq_node, tmp,
+ &recv_xcq->err_state_qp_list, entry) {
+ if (err_rq_node->qp_id == qp->xqp.qpn && !err_rq_node->is_sq) {
+ list_del(&err_rq_node->entry);
+ kfree(err_rq_node);
+ }
+ }
+ spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
+ }
+
+ if (qp->send_cq) {
+ send_xcq = to_xcq(qp->send_cq);
+ spin_lock_irqsave(&send_xcq->err_state_qp_lock, flags);
+ list_for_each_entry_safe(err_sq_node, tmp,
+ &send_xcq->err_state_qp_list, entry) {
+ if (err_sq_node->qp_id == qp->xqp.qpn && err_sq_node->is_sq) {
+ list_del(&err_sq_node->entry);
+ kfree(err_sq_node);
+ }
+ }
+ spin_unlock_irqrestore(&send_xcq->err_state_qp_lock, flags);
+ }
+ return 0;
+ }
+
+ if (cur_state != XSC_QP_STATE_ERR && state == XSC_QP_STATE_ERR &&
+ !qp->has_trig_cq_comp_handle) {
+ if (!qp->ibqp.srq && qp->recv_cq) {
+ err_rq_node = kzalloc_obj(*err_rq_node, GFP_ATOMIC);
+ if (!err_rq_node)
+ return -ENOMEM;
+ err_rq_node->qp_id = qp->xqp.qpn;
+ err_rq_node->is_sq = false;
+ recv_xcq = to_xcq(qp->recv_cq);
+ spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
+ list_add_tail(&err_rq_node->entry, &recv_xcq->err_state_qp_list);
+ spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
+ }
+
+ if (qp->send_cq) {
+ err_sq_node = kzalloc_obj(*err_sq_node, GFP_ATOMIC);
+ if (!err_sq_node) {
+ if (err_rq_node) {
+ spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
+ list_del(&err_rq_node->entry);
+ spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
+ kfree(err_rq_node);
+ }
+ return -ENOMEM;
+ }
+ err_sq_node->qp_id = qp->xqp.qpn;
+ err_sq_node->is_sq = true;
+ send_xcq = to_xcq(qp->send_cq);
+ spin_lock_irqsave(&send_xcq->err_state_qp_lock, flags);
+ list_add_tail(&err_sq_node->entry, &send_xcq->err_state_qp_list);
+ spin_unlock_irqrestore(&send_xcq->err_state_qp_lock, flags);
+ }
+
+ qp->has_trig_cq_comp_handle = true;
+ *triggered = true;
+ }
+ return 0;
+}
+
+enum xsc_ib_qp_ex_attr_mask {
+ XSC_IB_QP_FLUSH = 1 << 21,
+};
+
+static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq, struct xsc_ib_cq *cq)
+{
+ unsigned int cur;
+
+ cur = wq->head - wq->tail;
+ if (likely(cur + nreq < wq->max_post))
+ return 0;
+
+ spin_lock(&cq->lock);
+ cur = wq->head - wq->tail;
+ spin_unlock(&cq->lock);
+
+ return cur + nreq >= wq->max_post;
+}
+
+#define BTH_QPN_MASK (0x00ffffff)
+#define BTH_PSN_MASK (0x00ffffff)
+
+/* Compute a partial ICRC for all the IB transport headers. */
+
+/* Routine for sending QP1 packets for RoCE V1 an V2 */
+/* TODO: sq hdr buf should be created dynamically for mult entry */
+
+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr)
+{
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ struct xsc_wqe_data_seg *recv_head;
+ struct xsc_wqe_data_seg *data_seg;
+ unsigned long flags = 0;
+ int err = 0;
+ u32 next_pid = 0;
+ int nreq = 0;
+ u32 idx;
+ int i;
+
+ spin_lock_irqsave(&qp->rq.lock, flags);
+
+ idx = qp->rq.head & (qp->rq.wqe_cnt - 1);
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (unlikely(xsc_wq_overflow(&qp->rq, nreq, to_xcq(qp->ibqp.recv_cq)))) {
+ ibdev_err(&dev->ib_dev, "recv work queue overflow\n");
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > qp->rq.max_gs)) {
+ ibdev_err(&dev->ib_dev, "max gs exceeded %d (max = %d)\n",
+ wr->num_sge, qp->rq.max_gs);
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(qp->post_recv_invalid)) {
+ ibdev_dbg(&dev->ib_dev, "qp %d has been draind, post recv invalid...\n",
+ qp->xqp.qpn);
+ err = -EIO;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ recv_head = get_recv_wqe(qp, idx);
+
+ /*
+ * Fill the whole DS list: zero-length SGEs and the tail
+ * beyond num_sge must be explicitly invalidated, since the
+ * ring reuses WQEs and stale segments would be consumed by
+ * the hardware.
+ */
+ for (i = 0; i < qp->rq.max_gs; ++i) {
+ data_seg = get_seg_wqe(recv_head, i);
+ if (i < wr->num_sge && likely(wr->sg_list[i].length)) {
+ data_seg->data0 =
+ cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_DATA0_SEG_LEN_MASK,
+ wr->sg_list[i].length));
+ WR_LE_64(data_seg->va, wr->sg_list[i].addr);
+ WR_LE_32(data_seg->mkey, wr->sg_list[i].lkey);
+ } else {
+ data_seg->data0 = 0;
+ WR_LE_64(data_seg->va, 0);
+ WR_LE_32(data_seg->mkey, XSC_INVALID_LKEY);
+ }
+ }
+
+ qp->rq.wrid[idx] = wr->wr_id;
+
+ idx = (idx + 1) & (qp->rq.wqe_cnt - 1);
+ atomic_inc(&qp->rq.flush_wqe_cnt);
+ }
+
+out:
+ if (likely(nreq)) {
+ qp->rq.head += nreq;
+ next_pid = qp->rq.head << (qp->rq.wqe_shift - XSC_BASE_WQE_SHIFT);
+
+ xsc_update_rx_db(xdev, qp->doorbell_qpn, next_pid);
+ }
+
+ spin_unlock_irqrestore(&qp->rq.lock, flags);
+
+ return err;
+}
+
+struct xsc_ib_drain_cqe {
+ struct ib_cqe cqe;
+ struct completion done;
+ int qpn;
+};
+
+
diff --git a/drivers/infiniband/hw/xsc/srq.c b/drivers/infiniband/hw/xsc/srq.c
new file mode 100644
index 000000000..b791fc97f
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/srq.c
@@ -0,0 +1,462 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/slab.h>
+#include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
+#include "xsc_ib.h"
+#include "common/srq.h"
+#include <rdma/xsc-abi.h>
+#include <linux/list_sort.h>
+
+static int xsc_init_cache_wr(struct ib_srq_init_attr *attr, struct xsc_ib_srq *srq)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ if (srqn >= XSC_MAX_SRQ_NUM)
+ return -EINVAL;
+
+ memset(&cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));
+ cache_wr[srqn].max_wr_num = attr->attr.max_wr;
+ return 0;
+}
+
+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ if (!qp->srq_head_wr)
+ return;
+
+ if (qp->srq_tail_wr)
+ qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
+ cache_wr[srqn].head_wr = qp->srq_head_wr;
+ qp->srq_head_wr = NULL;
+ qp->srq_tail_wr = NULL;
+}
+
+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id)
+{
+ struct ib_recv_wr *tmp_wr, *prev_wr = NULL;
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ unsigned long flags;
+
+ spin_lock_irqsave(&srq->lock, flags);
+ tmp_wr = qp->srq_head_wr;
+ for (; tmp_wr; tmp_wr = tmp_wr->next) {
+ if (tmp_wr->wr_id == wr_id) {
+ if (prev_wr)
+ prev_wr->next = tmp_wr->next;
+ else
+ qp->srq_head_wr = tmp_wr->next;
+ if (qp->srq_tail_wr == tmp_wr)
+ qp->srq_tail_wr = prev_wr;
+
+ kfree(tmp_wr->sg_list);
+ kfree(tmp_wr);
+ cache_wr[srqn].wr_num--;
+ break;
+ }
+ prev_wr = tmp_wr;
+ }
+ spin_unlock_irqrestore(&srq->lock, flags);
+}
+
+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr)
+{
+ struct ib_recv_wr *prev_wr = NULL;
+
+ for (; wr; wr = wr->next) {
+ kfree(prev_wr);
+ kfree(wr->sg_list);
+ prev_wr = wr;
+ }
+ kfree(prev_wr);
+}
+
+static void xsc_destroy_cache_wr(struct xsc_ib_srq *srq)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ xsc_free_all_cache_wrs(cache_wr[srqn].head_wr);
+ memset(&cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));
+}
+
+static void xsc_ib_srq_event(struct xsc_core_srq *srq, enum xsc_event type)
+{
+ struct ib_event event;
+ struct ib_srq *ibsrq = &core_to_xsrq(srq)->ibsrq;
+
+ if (ibsrq->event_handler) {
+ event.device = ibsrq->device;
+ event.element.srq = ibsrq;
+ switch (type) {
+ default:
+ pr_warn("xsc_ib: Unexpected event type %d on SRQ %06x\n",
+ type, srq->srqn);
+ return;
+ }
+
+ ibsrq->event_handler(&event, ibsrq->srq_context);
+ }
+}
+
+static int create_srq_user(struct ib_pd *pd, struct xsc_ib_srq *srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata, int buf_size)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_create_srq ucmd = {};
+ size_t ucmdlen;
+
+ ucmdlen = min(udata->inlen, sizeof(ucmd));
+
+ if (ib_copy_from_udata(&ucmd, udata, ucmdlen)) {
+ ibdev_dbg(&dev->ib_dev, "failed copy udata\n");
+ return -EFAULT;
+ }
+
+ if (ucmd.reserved0 || ucmd.reserved1)
+ return -EINVAL;
+
+ if (udata->inlen > sizeof(ucmd) &&
+ !ib_is_udata_cleared(udata, sizeof(ucmd),
+ udata->inlen - sizeof(ucmd)))
+ return -EINVAL;
+
+ srq->create_type = XSC_SRQ_USER;
+
+ return 0;
+}
+
+static int create_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq,
+ struct ib_srq_init_attr *init_attr, int buf_size)
+{
+ srq->create_type = XSC_SRQ_KERNEL;
+ return 0;
+}
+
+static void destroy_srq_user(struct xsc_ib_srq *srq, struct ib_udata *udata)
+{
+}
+
+static void destroy_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq)
+{
+}
+
+int xsc_ib_create_srq(struct ib_srq *ib_srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev;
+ struct xsc_ib_srq *srq;
+ int err;
+ u32 max_srq_wqes = 1 << XSC_MAX_SRQ_LOG_SIZE;
+ struct ib_pd *pd = ib_srq->pd;
+
+ if (pd) {
+ dev = to_xdev(pd->device);
+ } else {
+ pr_warn("%s: no PD for SRQ\n", __func__);
+ err = -EINVAL;
+ goto err_ret;
+ }
+
+ if (init_attr->srq_type != IB_SRQT_BASIC) {
+ err = -EOPNOTSUPP;
+ goto err_ret;
+ }
+
+ /* Sanity check SRQ size before proceeding */
+ if (init_attr->attr.max_wr >= max_srq_wqes) {
+ ibdev_dbg(&dev->ib_dev, "max_wr %d, cap %d\n",
+ init_attr->attr.max_wr,
+ max_srq_wqes);
+ err = -EINVAL;
+ goto err_ret;
+ }
+
+ srq = to_xsrq(ib_srq);
+
+ mutex_init(&srq->mutex);
+ spin_lock_init(&srq->lock);
+ srq->xsrq.max = roundup_pow_of_two(init_attr->attr.max_wr + 1);
+ srq->xsrq.max_gs = init_attr->attr.max_sge;
+
+ if (udata)
+ err = create_srq_user(pd, srq, NULL, udata, 0);
+ else
+ err = create_srq_kernel(dev, srq, init_attr, 0);
+
+ if (err) {
+ ibdev_dbg(&dev->ib_dev, "create srq rsc failed, ret %d", err);
+ goto err_create_rsc;
+ }
+
+ err = xsc_core_create_srq(dev->xdev, &srq->xsrq);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create srq failed, err=%d\n", err);
+ goto err_create;
+ }
+ ibdev_dbg(&dev->ib_dev, "create SRQ with srqn 0x%x, max_wr %d\n",
+ srq->xsrq.srqn, init_attr->attr.max_wr);
+
+ srq->xsrq.event = xsc_ib_srq_event;
+ srq->ibsrq.ext.xrc.srq_num = srq->xsrq.srqn;
+
+ if (udata) {
+ struct xsc_ib_create_srq_resp resp = {
+ .srqn = srq->xsrq.srqn,
+ };
+
+ if (ib_copy_to_udata(udata, &resp,
+ min(udata->outlen, sizeof(resp)))) {
+ ibdev_dbg(&dev->ib_dev, "copy to user failed\n");
+ err = -EFAULT;
+ goto err_rsp;
+ }
+ } else {
+ err = xsc_init_cache_wr(init_attr, srq);
+ if (err)
+ goto err_rsp;
+ INIT_LIST_HEAD(&srq->xsrq.srq_qp_list);
+ }
+
+ init_attr->attr.max_wr = srq->xsrq.max - 1;
+
+ return 0;
+err_rsp:
+ xsc_core_destroy_srq(dev->xdev, &srq->xsrq);
+err_create:
+ if (udata)
+ destroy_srq_user(srq, udata);
+ else
+ destroy_srq_kernel(dev, srq);
+err_create_rsc:
+err_ret:
+ return err;
+}
+
+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata)
+{
+ /* We don't support resizing SRQs yet */
+ if (attr_mask & IB_SRQ_MAX_WR)
+ return -EINVAL;
+
+ if (attr_mask & IB_SRQ_LIMIT)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr)
+{
+ struct xsc_ib_srq *srq = to_xsrq(ibsrq);
+
+ srq_attr->srq_limit = 0;
+ srq_attr->max_wr = srq->xsrq.max - 1;
+ srq_attr->max_sge = srq->xsrq.max_gs;
+
+ return 0;
+}
+
+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata)
+{
+ struct xsc_ib_srq *xsrq = to_xsrq(srq);
+ struct xsc_ib_dev *dev = to_xdev(srq->device);
+
+ if (xsrq->create_type == XSC_SRQ_USER) {
+ destroy_srq_user(xsrq, NULL);
+ } else {
+ xsc_destroy_cache_wr(xsrq);
+ destroy_srq_kernel(dev, xsrq);
+ }
+ xsc_core_destroy_srq(dev->xdev, &xsrq->xsrq);
+
+ ibdev_dbg(&dev->ib_dev, "destroy srq %d successfully.\n", xsrq->xsrq.srqn);
+ return 0;
+}
+
+static int cmp_flush_wqe_cnt(void *priv,
+ const struct list_head *a,
+ const struct list_head *b)
+{
+ const struct xsc_srq_qp_node *na =
+ list_entry(a, struct xsc_srq_qp_node, entry);
+ const struct xsc_srq_qp_node *nb =
+ list_entry(b, struct xsc_srq_qp_node, entry);
+
+ int fa = atomic_read(&(to_xqp(na->ibqp)->rq.flush_wqe_cnt));
+ int fb = atomic_read(&(to_xqp(nb->ibqp)->rq.flush_wqe_cnt));
+
+ return fa - fb;
+}
+
+static int xsc_cache_srq_wr(struct xsc_ib_srq *srq, const struct ib_recv_wr *wr, int *nreq)
+{
+ int i, n;
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ struct ib_recv_wr *head_wr = NULL, *new_wr = NULL, *prev_wr = NULL;
+ const struct ib_recv_wr *tmp;
+
+ for (n = 0, tmp = wr; tmp; tmp = tmp->next, n++)
+ ;
+
+ if (cache_wr[srqn].wr_num + n > cache_wr[srqn].max_wr_num)
+ return -1;
+
+ for (n = 0; wr; wr = wr->next, n++) {
+ new_wr = kcalloc(1, sizeof(*new_wr), GFP_ATOMIC);
+ if (!new_wr)
+ goto err;
+ new_wr->sg_list = kcalloc(wr->num_sge, sizeof(*new_wr->sg_list),
+ GFP_ATOMIC);
+ if (!new_wr->sg_list) {
+ kfree(new_wr);
+ goto err;
+ }
+ new_wr->wr_id = wr->wr_id;
+ for (i = 0; i < wr->num_sge; i++)
+ memcpy(&new_wr->sg_list[i], &wr->sg_list[i],
+ sizeof(struct ib_sge));
+ new_wr->num_sge = wr->num_sge;
+ if (prev_wr)
+ prev_wr->next = new_wr;
+ else
+ head_wr = new_wr;
+ prev_wr = new_wr;
+ };
+
+ if (!cache_wr[srqn].head_wr)
+ cache_wr[srqn].head_wr = head_wr;
+ else
+ cache_wr[srqn].tail_wr->next = head_wr;
+
+ cache_wr[srqn].tail_wr = new_wr;
+ cache_wr[srqn].wr_num += n;
+ *nreq = n;
+
+ return 0;
+
+err:
+ xsc_free_all_cache_wrs(head_wr);
+ return -ENOMEM;
+}
+
+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ int err = 0;
+ int i;
+ const struct ib_recv_wr *tmp_wr;
+ const struct ib_recv_wr *bad_wr;
+ struct ib_recv_wr *wr_head, *next_wr;
+ struct ib_recv_wr *prev_tail;
+
+ if (!cache_wr[srqn].head_wr)
+ return;
+
+ wr_head = cache_wr[srqn].head_wr;
+ next_wr = cache_wr[srqn].head_wr;
+ prev_tail = qp->srq_tail_wr;
+
+ if (init || !qp->srq_tail_wr)
+ qp->srq_head_wr = wr_head;
+ else
+ qp->srq_tail_wr->next = wr_head;
+
+ qp->srq_tail_wr = wr_head;
+ for (i = 0; next_wr && i < (wr_num - 1); i++) {
+ next_wr = next_wr->next;
+ if (next_wr)
+ qp->srq_tail_wr = next_wr;
+ }
+
+ cache_wr[srqn].head_wr = qp->srq_tail_wr->next;
+ qp->srq_tail_wr->next = NULL;
+ tmp_wr = wr_head;
+ err = xsc_ib_post_recv(&qp->ibqp, tmp_wr, &bad_wr);
+ if (err) {
+ qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
+ qp->srq_tail_wr = NULL;
+ for (; wr_head && wr_head != bad_wr; wr_head = wr_head->next)
+ qp->srq_tail_wr = wr_head;
+
+ if (!qp->srq_tail_wr) {
+ cache_wr[srqn].head_wr = wr_head;
+ if (prev_tail && !init) {
+ prev_tail->next = NULL;
+ qp->srq_tail_wr = prev_tail;
+ } else {
+ qp->srq_head_wr = NULL;
+ }
+ } else {
+ cache_wr[srq->xsrq.srqn].head_wr = qp->srq_tail_wr->next;
+ qp->srq_tail_wr->next = NULL;
+ }
+ }
+}
+
+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr)
+{
+ struct xsc_ib_srq *srq = to_xsrq(ibsrq);
+ struct xsc_ib_dev *dev = to_xdev(ibsrq->device);
+ struct xsc_srq_qp_node *qp_node;
+ struct xsc_ib_qp *qp;
+ unsigned long flags;
+ int err = 0;
+ int flush_wqe_cnt = 0;
+ int nreq = 0;
+ int wr_num = 0;
+
+ spin_lock_irqsave(&srq->lock, flags);
+
+ err = xsc_cache_srq_wr(srq, wr, &nreq);
+ if (err == -ENOMEM) {
+ ibdev_err(&dev->ib_dev, "failed to cache SRQ WRs, err=%d\n", err);
+ *bad_wr = wr;
+ goto out;
+ }
+ if (err) {
+ ibdev_err(&dev->ib_dev, "No more space to cache SRQ WRs...");
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (list_empty(&srq->xsrq.srq_qp_list))
+ goto out;
+
+ list_sort(NULL, &srq->xsrq.srq_qp_list, cmp_flush_wqe_cnt);
+
+ list_for_each_entry(qp_node, &srq->xsrq.srq_qp_list, entry) {
+ if (!qp_node || nreq <= 0)
+ break;
+
+ qp = to_xqp(qp_node->ibqp);
+ flush_wqe_cnt = atomic_read(&qp->rq.flush_wqe_cnt);
+ if (flush_wqe_cnt >= XSC_SRQ_MIN_WR_NUM)
+ continue;
+ wr_num = min_t(int, nreq, XSC_SRQ_MIN_WR_NUM - flush_wqe_cnt);
+ nreq -= wr_num;
+ ibdev_dbg(&dev->ib_dev, "srq_qp_list qp node qp %d, flush cnt %d, nreq %d, wr_num %d\n",
+ qp_node->qp_id, flush_wqe_cnt, nreq, wr_num);
+ xsc_post_cache_wrs(srq, qp, wr_num, false);
+ }
+
+out:
+ spin_unlock_irqrestore(&srq->lock, flags);
+
+ return err;
+}
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
index eb4d9e630..d7b0ff176 100644
--- a/drivers/infiniband/hw/xsc/xsc_ib.h
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -167,6 +167,7 @@ struct xsc_ib_cq {
struct list_head send_qp_list;
struct list_head recv_qp_list;
struct list_head err_state_qp_list;
+ spinlock_t err_state_qp_lock;
u32 create_flags;
int invalid;
};
@@ -324,8 +325,25 @@ void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);
+
+/*
+ * Wake the CQ completion handlers of a QP that has just moved to ERR. Must
+ * be called with no cq->lock held: the handler may poll or rearm the CQ,
+ * which takes that lock again.
+ */
+static inline void xsc_ib_trigger_cq_comp_handler(struct ib_cq *send_cq,
+ struct ib_cq *recv_cq)
+{
+ if (send_cq)
+ send_cq->comp_handler(send_cq, send_cq->cq_context);
+
+ if (recv_cq && send_cq != recv_cq)
+ recv_cq->comp_handler(recv_cq, recv_cq->cq_context);
+}
+
int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
- enum xsc_qp_state cur_state, enum xsc_qp_state state);
+ enum xsc_qp_state cur_state, enum xsc_qp_state state,
+ bool *triggered);
int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata);
int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
index 065d52ebf..fe0f9d175 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -6,4 +6,5 @@ ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
-xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o pd.o mr.o link.o alloc.o eq.o pci_irq.o adev.o vport.o hal/andes_impl.o
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o pd.o mr.o srq.o link.o alloc.o eq.o pci_irq.o adev.o vport.o \
+ hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
index cb2f37395..27a38c052 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -8,6 +8,7 @@
#include "hw.h"
#include "qp.h"
#include "cq.h"
+#include "srq.h"
#include "eq.h"
#include "pci_irq.h"
#include "adev.h"
@@ -120,12 +121,21 @@ static int xsc_dev_res_init(struct xsc_core_device *xdev)
mutex_init(&dev_res->alloc_mutex);
spin_lock_init(&dev_res->mkey_lock);
+ if (xsc_create_srq_res(xdev))
+ goto err_free;
+
return 0;
+
+err_free:
+ kvfree(dev_res);
+ xdev->dev_res = NULL;
+ return -ENOMEM;
}
static void xsc_dev_res_cleanup(struct xsc_core_device *xdev)
{
- kfree(xdev->dev_res);
+ xsc_destroy_srq_res(xdev);
+ kvfree(xdev->dev_res);
}
static int xsc_core_dev_init(struct xsc_core_device *xdev)
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c
new file mode 100644
index 000000000..d855be680
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include "common/xsc_core.h"
+#include "common/srq.h"
+#include "srq.h"
+
+int xsc_create_srq_res(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ u32 max_srq_num = XSC_MAX_SRQ_NUM;
+
+ dev_res->srq_tbl = bitmap_zalloc(max_srq_num, GFP_KERNEL);
+ if (!dev_res->srq_tbl)
+ return -ENOMEM;
+
+ dev_res->srq_cache_wr = vzalloc(max_srq_num *
+ sizeof(*dev_res->srq_cache_wr));
+ if (!dev_res->srq_cache_wr) {
+ bitmap_free(dev_res->srq_tbl);
+ dev_res->srq_tbl = NULL;
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&dev_res->srq_lock);
+ dev_res->max_srq_num = max_srq_num;
+
+ return 0;
+}
+
+void xsc_destroy_srq_res(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ vfree(dev_res->srq_cache_wr);
+ dev_res->srq_cache_wr = NULL;
+ bitmap_free(dev_res->srq_tbl);
+ dev_res->srq_tbl = NULL;
+ dev_res->max_srq_num = 0;
+}
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ u32 srqn;
+
+ spin_lock(&dev_res->srq_lock);
+ srqn = find_first_zero_bit(dev_res->srq_tbl, dev_res->max_srq_num);
+ if (srqn == dev_res->max_srq_num) {
+ spin_unlock(&dev_res->srq_lock);
+ pci_err(xdev->pdev, "no free srq number\n");
+ return -ENOMEM;
+ }
+ set_bit(srqn, dev_res->srq_tbl);
+ spin_unlock(&dev_res->srq_lock);
+
+ srq->srqn = srqn;
+ /* array base: callers index it by srqn */
+ srq->cache_wr = dev_res->srq_cache_wr;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_srq);
+
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ if (srq->srqn >= dev_res->max_srq_num) {
+ pci_err(xdev->pdev, "invalid srqn %u\n", srq->srqn);
+ return -EINVAL;
+ }
+
+ spin_lock(&dev_res->srq_lock);
+ clear_bit(srq->srqn, dev_res->srq_tbl);
+ spin_unlock(&dev_res->srq_lock);
+
+ /*
+ * The caller has already released the cached recv WRs in
+ * cache_wr[srqn] before this call, so the slot is clean by the
+ * time the srqn becomes free for reuse.
+ */
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_srq);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h
new file mode 100644
index 000000000..a70faf9e7
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_SRQ_H
+#define __XSC_SRQ_H
+
+#include "common/xsc_core.h"
+#include "common/srq.h"
+
+/* return 0 on success, negative errno otherwise */
+int xsc_create_srq_res(struct xsc_core_device *xdev);
+void xsc_destroy_srq_res(struct xsc_core_device *xdev);
+
+#endif /* __XSC_SRQ_H */
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 12/15] RDMA/xsc: add RDMA CQ/SRQ/AH
2026-11-04 15:57 ` [PATCH v1 12/15] RDMA/xsc: add RDMA CQ/SRQ/AH Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 13 potential issue(s) to consider:
- [Critical] RDMA/xsc: use-after-free via stale tail_wr in SRQ WR cache management
- [High] RDMA/xsc: validate attr->cqe, udata, and 64B CQE size in xsc_ib_create_cq()
- [High] RDMA/xsc: fix lost-CQE handling and permanent CQ invalidation in xsc_poll_one()
- [High] RDMA/xsc: fix races and unlocked cur_qp use in QP error flush handling
- [High] RDMA/xsc: missing create_user_ah callback in xsc_ib_dev_setting()
- [High] RDMA/xsc: broken error CQE flushing and state tracking in cq.c and qp.c
- [High] RDMA/xsc: missing SRQ WR replenishment, sge validation, and max_wr sync
- [Medium] RDMA/xsc: fix flow_label order, udp_sport endianness, and tclass in create_ib_ah()
- [Medium] RDMA/xsc: fix data_seg->va endianness and DMA sync in xsc_handle_rdma_mad_resp_recv()
- [Medium] RDMA/xsc: initialize CQ lists/locks before publishing CQ and guard comp_handler
- [Medium] RDMA/xsc: lost generated flow_label and endian mismatch in create_ib_ah()
- [Medium] RDMA/xsc: shift-out-of-bounds in __xsc_ib_cont_pages() from create_cq_user()
- [Low] RDMA/xsc: fix xsc_vector2eqn() pointer-sign and endianness in xsc_ib_create_cq()
--
commit 26a9accc1990ad9da5c91694c9f25f02f7b052ac
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add RDMA CQ/SRQ/AH
This commit adds completion queue, shared receive queue, and address handle
verb implementations for the Yunsilicon xsc RDMA driver.
> diff --git a/drivers/infiniband/hw/xsc/ah.c b/drivers/infiniband/hw/xsc/ah.c
> new file mode 100644
> index 0000000000000..4d916dbe7dd25
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/ah.c
> @@ -0,0 +1,144 @@
[ ... ]
> +static u16 xsc_ah_get_udp_sport(const struct xsc_ib_dev *dev,
> + struct rdma_ah_attr *ah_attr)
> +{
> + enum ib_gid_type gid_type = ah_attr->grh.sgid_attr->gid_type;
> + u16 sport = 0;
> + u32 fl = 0;
> +
> + if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP &&
> + (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) &&
> + (ah_attr->grh.flow_label & IB_GRH_FLOWLABEL_MASK)) {
> + fl = ah_attr->grh.flow_label;
> + } else {
> + /*generate a 20bit flow_label and output to user layer*/
> + fl = xsc_calc_roce_udp_flow_label();
> + ah_attr->grh.flow_label = fl;
> + }
> +
> + sport = rdma_flow_label_to_udp_sport(fl);
> + ibdev_dbg(&dev->ib_dev, "fl=0x%x,sport=0x%x\n", fl, sport);
> + return sport;
> +}
> +
> +static struct ib_ah *create_ib_ah(struct xsc_ib_dev *dev,
> + struct xsc_ib_ah *ah,
> + struct rdma_ah_attr *ah_attr)
> +{
> + enum ib_gid_type gid_type;
> +
> + if (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) {
> + const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
> +
> + memcpy(ah->av.rgid, &grh->dgid, 16);
> + ah->av.grh_gid_fl = cpu_to_be32(grh->flow_label |
> + (1 << 30) |
> + grh->sgid_index << 20);
> + ah->av.hop_limit = grh->hop_limit;
> + ah->av.tclass = grh->traffic_class;
> + }
> +
> + ah->av.stat_rate_sl = (rdma_ah_get_static_rate(ah_attr) << 4);
> +
> + if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
> + gid_type = ah_attr->grh.sgid_attr->gid_type;
> +
> + memcpy(ah->av.rmac, ah_attr->roce.dmac,
> + sizeof(ah_attr->roce.dmac));
> +
> + ah->av.udp_sport = xsc_ah_get_udp_sport(dev, ah_attr);
> + ah->av.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) & 0x7) << 1;
> + if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP)
> +#define XSC_ECN_ENABLED BIT(1)
> + ah->av.tclass |= XSC_ECN_ENABLED;
[Severity: Medium]
Does create_ib_ah() lose the generated flow label and store udp_sport with
the wrong byte order?
When IB_AH_GRH is set, create_ib_ah() packs grh->flow_label into
ah->av.grh_gid_fl (without masking with IB_GRH_FLOWLABEL_MASK) before
calling xsc_ah_get_udp_sport(dev, ah_attr). When ah_attr->grh.flow_label is
0, xsc_ah_get_udp_sport() generates a random 20-bit flow label and writes
it to ah_attr->grh.flow_label, but ah->av.grh_gid_fl has already been
populated with 0 and is never updated, so xsc_ib_query_ah() later reports a
flow label of 0.
In addition, struct xsc_av declares udp_sport as __be16:
drivers/net/ethernet/yunsilicon/xsc/common/qp.h:
union {
__be16 rlid;
__be16 udp_sport;
};
However, xsc_ah_get_udp_sport() returns a host-endian u16 from
rdma_flow_label_to_udp_sport(fl), and create_ib_ah() assigns it directly to
ah->av.udp_sport without cpu_to_be16().
Finally, create_ib_ah() sets XSC_ECN_ENABLED in ah->av.tclass for RoCEv2,
and xsc_ib_query_ah() passes ah->av.tclass directly to rdma_ah_set_grh()
without masking out XSC_ECN_ENABLED, reporting a modified traffic_class to
callers. Should xsc_ah_get_udp_sport() run before packing ah->av.grh_gid_fl,
convert udp_sport with cpu_to_be16(), and avoid leaking XSC_ECN_ENABLED in
xsc_ib_query_ah()?
> diff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c
> new file mode 100644
> index 0000000000000..36d122f8d46fb
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/cq.c
> @@ -0,0 +1,940 @@
[ ... ]
> +static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
> + struct xsc_ib_qp **cur_qp,
> + struct ib_wc *wc,
> + struct xsc_cqe *cqe,
> + u8 opcode)
> +{
> + struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
> + void *recv;
> + struct xsc_wqe_data_seg *data_seg;
> + struct iphdr *ip4h = NULL;
> + struct ipv6hdr *ip6h;
> + struct udphdr *udph;
> + struct ib_unpacked_eth *eth;
> + struct ib_unpacked_vlan *vlan;
> + struct ib_grh *grh;
> + struct ib_mad *mad;
> + struct rxe_bth *bth;
> + struct rxe_deth *deth;
> + unsigned int pading_sz = 0;
> + struct xsc_ib_wq *wq;
> + int idx;
> + u16 eth_type;
> + void *l3_start;
> +
> + wq = &(*cur_qp)->rq;
> + idx = wq->tail & (wq->wqe_cnt - 1);
> +
> + handle_responder(wc, cqe, *cur_qp, opcode);
> +
> + data_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);
> + recv = xsc_ib_recv_mad_sg_virt_addr(&dev->ib_dev, wc, data_seg->va);
> +
> + eth = (struct ib_unpacked_eth *)recv;
> + grh = (struct ib_grh *)recv;
> + if (eth->type == htons(ETH_P_8021Q)) {
> + vlan = (struct ib_unpacked_vlan *)(eth + 1);
> + eth_type = ntohs(vlan->type);
> + l3_start = vlan + 1;
> +
> + wc->vlan_id = ntohs(vlan->tag) & 0x0fff;
> + wc->sl = (ntohs(vlan->tag) >> 13) & 0x7;
> + wc->wc_flags |= IB_WC_WITH_VLAN;
> + } else {
> + eth_type = ntohs(eth->type);
> + l3_start = eth + 1;
> + }
> +
> + if (eth_type == ETH_P_IP) {
> + ip4h = (struct iphdr *)l3_start;
> + udph = (struct udphdr *)(ip4h + 1);
> + } else {
> + ip6h = (struct ipv6hdr *)l3_start;
> + udph = (struct udphdr *)(ip6h + 1);
> + }
> + bth = (struct rxe_bth *)(udph + 1);
> + deth = (struct rxe_deth *)(bth + 1);
> + mad = (struct ib_mad *)(deth + 1);
> +
> + if (eth_type == ETH_P_IP) {
> + pading_sz = sizeof(*grh) - sizeof(*ip4h);
> + memmove((u8 *)(grh + 1) - sizeof(*ip4h), ip4h, sizeof(*ip4h));
> + memset(grh, 0, pading_sz);
> + } else {
> + memmove(grh, ip6h, sizeof(*ip6h));
> + }
> + memmove(grh + 1, mad, sizeof(*mad));
[Severity: Medium]
In xsc_handle_rdma_mad_resp_recv(), data_seg->va is declared as __le64 in
struct xsc_wqe_data_seg and written via WR_LE_64(data_seg->va,
wr->sg_list[i].addr) in xsc_ib_post_recv(), but it is passed directly to
xsc_ib_recv_mad_sg_virt_addr() (and in commit c77226a0e2a3 to
xsc_ib_mad_sg_virt_addr()) without le64_to_cpu(). On big-endian systems,
won't this translate a byte-swapped DMA address and corrupt memory in
memmove() and memset()?
Also, the IB MAD layer maps the receive buffer with
ib_dma_map_single(..., DMA_FROM_DEVICE) when posting the receive WR and
only unmaps it in ib_mad_recv_done() after poll_cq returns. Is DMA
synchronization via dma_sync_single_for_cpu() and
dma_sync_single_for_device() needed around the in-place header rewrite here?
[ ... ]
> +static int xsc_poll_one(struct xsc_ib_cq *cq,
> + struct xsc_ib_qp **cur_qp,
> + struct ib_wc *wc, bool *trigger)
> +{
> + struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
> + struct xsc_core_qp *xqp;
> + struct xsc_ib_wq *wq;
> + u8 opcode;
> + u32 qpn;
> + int idx;
> + int ret;
> + struct xsc_cqe *cqe;
> + u32 *p = NULL;
> + struct xsc_ib_srq *xsrq;
> +
> + *trigger = false;
> + cqe = get_sw_cqe(cq, cq->xcq.cons_index);
> + if (!cqe)
> + return -EAGAIN;
> +
> + ++cq->xcq.cons_index;
[ ... ]
> + switch (opcode) {
> + case XSC_OPCODE_RDMA_REQ_SEND_IMMDT:
> + case XSC_OPCODE_RDMA_REQ_WRITE_IMMDT:
> + wc->wc_flags |= IB_WC_WITH_IMM;
> + fallthrough;
> + case XSC_OPCODE_RDMA_REQ_SEND:
> + case XSC_OPCODE_RDMA_REQ_WRITE:
> + case XSC_OPCODE_RDMA_REQ_READ:
> + case XSC_OPCODE_RDMA_MAD_REQ_SEND:
> + wq = &(*cur_qp)->sq;
> + idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
> + idx &= (wq->wqe_cnt - 1);
> + if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
> + xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
> + cq->invalid = 1;
> + break;
> + }
[ ... ]
> + case XSC_OPCODE_RDMA_REQ_ERROR:
> + wq = &(*cur_qp)->sq;
> + idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
> + idx &= (wq->wqe_cnt - 1);
> +
> + if (!wq->need_flush[idx]) {
> + unsigned int i;
> + unsigned int tmp;
> + unsigned int prev = -1;
> +
> + for (i = wq->tail; i != wq->head; i++) {
> + tmp = i & (wq->wqe_cnt - 1);
> + if (tmp == idx)
> + break;
> + if (wq->need_flush[tmp])
> + prev = tmp;
> + }
> +
> + if (prev != -1) {
> + if (wq->prev_ce_wqe[prev] != wq->last_comp_ce_wqe) {
> + xsc_generate_lost_cqe(dev, wq, qpn, prev, wc);
> + cq->invalid = 1;
> + break;
> + }
> + }
> + } else {
> + if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
> + xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
> + cq->invalid = 1;
> + break;
> + }
> + }
[Severity: High]
How is a CQ supposed to recover after xsc_poll_one() detects a lost CQE?
At the start of xsc_poll_one(), ++cq->xcq.cons_index consumes the hardware
CQE for WQE idx. When wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe,
xsc_generate_lost_cqe() synthesizes an IB_WC_GENERAL_ERR completion for the
earlier lost_cqe_idx, dropping the already-consumed hardware CQE for idx.
Those paths also set cq->invalid = 1, which is never cleared anywhere in
the driver, and they do not call xsc_ib_err_state_qp() to place the QP onto
cq->err_state_qp_list. On every subsequent call to xsc_ib_poll_cq(),
cq->invalid causes the poll loop to immediately break with err = -EINVAL,
permanently blocking all completions on this CQ (even for other QPs sharing
the CQ) and leaving the QP's remaining WQEs unflushed.
Furthermore, in the XSC_OPCODE_RDMA_REQ_ERROR branch when
!wq->need_flush[idx] (an unsignaled WQE completed with error), the loop
finds the preceding uncompleted signaled WQE prev (where
wq->need_flush[prev] is still set). Because prev itself is an uncompleted
signaled WQE prior to idx, if only prev was lost then
wq->prev_ce_wqe[prev] == wq->last_comp_ce_wqe; checking
wq->prev_ce_wqe[prev] != wq->last_comp_ce_wqe misses the case where prev
itself is the lost WQE (and xsc_generate_lost_cqe() starts at
wq->prev_ce_wqe[prev] instead of prev).
[ ... ]
> +static inline int xsc_generate_flush_err_cqe(struct ib_cq *ibcq,
> + int ne, int *npolled, struct ib_wc *wc)
> +{
> + u32 qp_id = 0;
> + int flush_wqe_cnt = 0;
> + int sw_npolled = 0;
> + u32 idx = 0;
> + struct xsc_err_state_qp_node *err_qp_node;
> + struct xsc_core_qp *xqp;
> + struct xsc_ib_cq *cq = to_xcq(ibcq);
> + struct xsc_ib_wq *wq;
> + struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&cq->err_state_qp_lock, flags);
> + list_for_each_entry(err_qp_node, &cq->err_state_qp_list, entry) {
> + if (!err_qp_node)
> + break;
> +
> + sw_npolled = 0;
> + qp_id = err_qp_node->qp_id;
> + xqp = __xsc_qp_lookup(dev->xdev, qp_id);
> + if (unlikely(!xqp)) {
> + ibdev_warn(&dev->ib_dev, "CQE@CQ %d for unknown QPN %d\n",
> + cq->xcq.cqn, qp_id);
> + continue;
> + }
> + wq = err_qp_node->is_sq ? &(core_to_xqp(xqp)->sq) : &(core_to_xqp(xqp)->rq);
> + flush_wqe_cnt = atomic_read(&wq->flush_wqe_cnt);
> + ibdev_dbg(&dev->ib_dev, "is_sq %d, flush_wq_cnt %d, ne %d, npolled %d, qp_id %d, cqn %d\n",
> + err_qp_node->is_sq, flush_wqe_cnt, ne, *npolled, qp_id, cq->xcq.cqn);
> +
> + while (wq->tail != wq->head) {
> + idx = wq->tail & (wq->wqe_cnt - 1);
> + if (err_qp_node->is_sq && !wq->need_flush[idx]) {
> + wq->tail++;
> + continue;
> + } else {
> + gen_flush_err_cqe(err_qp_node, &(core_to_xqp(xqp)->ibqp), wq,
> + idx, wc + *npolled + sw_npolled);
> + ++sw_npolled;
> + }
> +
> + if (sw_npolled == ne - *npolled)
> + break;
> + }
> + *npolled += sw_npolled;
> + if (*npolled == ne)
> + break;
> + }
> + spin_unlock_irqrestore(&cq->err_state_qp_lock, flags);
> +
> + return 0;
> +}
> +
> +int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
> +{
> + struct xsc_ib_cq *cq = to_xcq(ibcq);
> + struct xsc_core_cq *xcq = &cq->xcq;
> + struct xsc_ib_qp *cur_qp = NULL;
> + bool trigger = false;
> + int npolled = 0;
> + int err = 0;
> + unsigned long flags;
> + u32 next_cid;
> +
> + spin_lock_irqsave(&cq->lock, flags);
> + next_cid = xcq->cons_index;
> +
> + for (npolled = 0; npolled < num_entries; npolled++) {
> + if (cq->invalid) {
> + err = -EINVAL;
> + break;
> + }
> + err = xsc_poll_one(cq, &cur_qp, wc + npolled, &trigger);
> + if (err)
> + break;
> + if (trigger) {
> + npolled++;
> + break;
> + }
> + }
> +
> + if (err) {
> + if (npolled < num_entries && !(list_empty(&cq->err_state_qp_list)))
> + xsc_generate_flush_err_cqe(ibcq, num_entries, &npolled, wc);
> + }
> +
> + if (next_cid != xcq->cons_index)
> + xsc_update_cq_ci(xcq->dev, xcq->cqn, xcq->cons_index);
> +
> + spin_unlock_irqrestore(&cq->lock, flags);
> +
> + if (trigger)
> + xsc_ib_trigger_cq_comp_handler(cur_qp->ibqp.send_cq,
> + cur_qp->ibqp.recv_cq);
> +
> + return npolled;
> +}
[Severity: High]
Can xsc_ib_poll_cq() dereference cur_qp after it has been freed, or race on
the WQ state during error flushing?
1. In xsc_poll_one(), cur_qp is looked up via __xsc_qp_lookup() without
taking a reference count, relying on cq->lock to serialize against QP
destruction. However, xsc_ib_poll_cq() drops cq->lock before dereferencing
cur_qp->ibqp.send_cq and cur_qp->ibqp.recv_cq in the if (trigger) branch,
so a concurrent xsc_ib_destroy_qp() can free cur_qp right after cq->lock is
released.
2. In xsc_ib_poll_cq(), list_empty(&cq->err_state_qp_list) is checked
without holding cq->err_state_qp_lock, and xsc_generate_flush_err_cqe()
reads and updates wq->head, wq->tail, wq->wrid[idx], wq->wr_opcode[idx], and
wq->need_flush[idx] without holding wq->lock (qp->rq.lock or qp->sq.lock),
racing with concurrent xsc_ib_post_recv() or xsc_ib_post_send().
> +static int xsc_cqe_is_empty(struct xsc_ib_cq *cq)
> +{
> + struct xsc_cqe *cqe = get_sw_cqe(cq, cq->xcq.cons_index);
> +
> + if (cq->invalid)
> + return 1;
> +
> + if (!cqe)
> + return 1;
> +
> + return 0;
> +}
[Severity: High]
Won't breaking out of xsc_ib_poll_cq() on trigger while xsc_cqe_is_empty()
ignores cq->err_state_qp_list stall software flush completions on
IB_POLL_SOFTIRQ CQs?
When xsc_poll_one() processes XSC_OPCODE_RDMA_REQ_ERROR or
XSC_OPCODE_RDMA_RSP_ERROR, it calls xsc_ib_err_state_qp(), which adds the
QP to cq->err_state_qp_list and sets *trigger = true, while xsc_poll_one()
returns 0. In xsc_ib_poll_cq(), if (trigger) increments npolled and breaks
out of the loop with err == 0, skipping xsc_generate_flush_err_cqe() (which
only runs when err != 0) and returning a partial count npolled < num_entries.
For an IB_POLL_SOFTIRQ CQ polled from ib_poll_handler() -> __ib_process_cq():
1. Returning npolled < IB_POLL_BATCH stops __ib_process_cq() immediately.
2. xsc_ib_trigger_cq_comp_handler() calls ib_cq_completion_softirq() ->
irq_poll_sched(&cq->iop) before irq_poll_complete(&cq->iop) has run,
which is a no-op because IRQ_POLL_F_SCHED is still set.
3. ib_poll_handler() then calls irq_poll_complete(&cq->iop) and
ib_req_notify_cq(cq, IB_POLL_FLAGS) -> xsc_ib_arm_cq().
4. Because xsc_cqe_is_empty() only checks get_sw_cqe() and ignores pending
entries in cq->err_state_qp_list, xsc_ib_arm_cq() returns 0 and cq->iop
is never rescheduled, stranding all remaining WRs in
cq->err_state_qp_list and hanging callers such as ib_drain_qp().
[ ... ]
> +static int create_cq_user(struct xsc_ib_dev *dev, struct ib_udata *udata,
> + struct ib_ucontext *context, struct xsc_ib_cq *cq,
> + int entries, struct xsc_create_cq_ex_mbox_in **cqb,
> + int *cqe_size, int *index, int *inlen)
> +{
> + struct xsc_ib_create_cq ucmd;
> + int page_shift;
> + int npages;
> + int ncont;
> + int err;
> + int log_cq_sz;
> + int hw_npages;
> + struct ib_umem_dmabuf *umem_dmabuf;
> +
> + if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
> + return -EFAULT;
> +
> + if (ucmd.cqe_size != sizeof(struct xsc_cqe))
> + return -EINVAL;
> +
> + *cqe_size = ucmd.cqe_size;
> +
> + if (ucmd.dmabuf_fd != -1) {
> + size_t cq_ring_sz = (size_t)entries * ucmd.cqe_size;
> +
> + if (ucmd.dmabuf_sz < cq_ring_sz)
> + return -EINVAL;
> +
> + umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd.dmabuf_sz,
> + ucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
> +
> + if (IS_ERR(umem_dmabuf)) {
> + ibdev_warn(&dev->ib_dev, "umem_dmabuf get failed by fd %d (%ld)\n",
> + ucmd.dmabuf_fd, PTR_ERR(umem_dmabuf));
> + return -ENOMEM;
> + }
> +
> + cq->buf.umem = &umem_dmabuf->umem;
> + ibdev_dbg(&dev->ib_dev, "get umem_dmabuf success by fd %d, umem_dmabuf_addr %p\n",
> + ucmd.dmabuf_fd, umem_dmabuf);
> + } else {
> + cq->buf.umem = cq_umem_get(dev, udata, context, entries, &ucmd);
> + }
> +
> + if (IS_ERR(cq->buf.umem)) {
> + err = PTR_ERR(cq->buf.umem);
> + return err;
> + }
> +
> + xsc_ib_cont_pages(cq->buf.umem, ucmd.buf_addr, &npages, &page_shift,
> + &ncont);
[Severity: Medium]
Can calling xsc_ib_cont_pages() with ucmd.buf_addr trigger a 64-bit shift
out of bounds in __xsc_ib_cont_pages()?
When creating a dmabuf-backed user CQ (or any user CQ where ucmd.buf_addr <
PAGE_SIZE), ucmd.buf_addr is 0. xsc_ib_cont_pages() calls
__xsc_ib_cont_pages() with max_page_shift = 0:
drivers/infiniband/hw/xsc/mem.c:__xsc_ib_cont_pages() {
addr = addr >> page_shift;
tmp = (unsigned long)addr;
m = find_first_bit(&tmp, BITS_PER_LONG);
if (max_page_shift)
m = min_t(unsigned long, max_page_shift - page_shift, m);
for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
len = sg_dma_len(sg) >> page_shift;
pfn = sg_dma_address(sg) >> page_shift;
if (base + p != pfn) {
tmp = (unsigned long)(pfn | p);
if (!IS_ALIGNED(tmp, 1UL << m))
m = find_first_bit(&tmp, BITS_PER_LONG);
}
When addr >> page_shift is 0, find_first_bit(&tmp, BITS_PER_LONG) returns
BITS_PER_LONG (64 on 64-bit platforms). Because max_page_shift is 0, m is
not clamped, and on the first scatterlist entry (where base is ~0ULL),
!IS_ALIGNED(tmp, 1UL << m) evaluates 1UL << 64.
[ ... ]
> +int xsc_ib_create_cq(struct ib_cq *ibcq,
> + const struct ib_cq_init_attr *attr,
> + struct uverbs_attr_bundle *attrs)
> +{
> + struct ib_device *ibdev = ibcq->device;
> + struct ib_udata *udata = attrs ? &attrs->driver_udata : NULL;
> + int entries = attr->cqe;
> + int vector = attr->comp_vector;
> + struct xsc_create_cq_ex_mbox_in *cqb = NULL;
> + struct xsc_ib_dev *dev = to_xdev(ibdev);
> + struct xsc_ib_cq *cq;
> + int index;
> + int inlen = 0;
> + int cqe_size;
> + int irqn;
> + int err;
> + unsigned int eqn;
> +
> + entries = roundup_pow_of_two(entries);
> +
> + ibdev_dbg(&dev->ib_dev, "entries:%d, vector:%d, max_cqes:%d\n", entries, vector,
> + dev->xdev->caps.max_cqes);
> +
> + if (entries > dev->xdev->caps.max_cqes)
> + entries = dev->xdev->caps.max_cqes;
> + cq = to_xcq(ibcq);
> + cq->ibcq.cqe = entries;
> + mutex_init(&cq->resize_mutex);
> + spin_lock_init(&cq->lock);
> + cq->resize_buf = NULL;
> + cq->resize_umem = NULL;
> + cq->create_flags = attr->flags;
> + cq->invalid = 0;
> +
> + if (udata) {
> + err = create_cq_user(dev, udata, NULL, cq, entries,
> + &cqb, &cqe_size, &index, &inlen);
> + if (err)
> + goto err_create;
> + } else {
> + cqe_size = is_support_cqe64(dev->xdev) ?
> + sizeof(struct xsc_cqe64) : sizeof(struct xsc_cqe);
> + err = create_cq_kernel(dev, cq, entries, cqe_size, &cqb, &index, &inlen);
> + if (err)
> + goto err_create;
> + }
[Severity: High]
Does xsc_ib_create_cq() properly validate attr->cqe, udata lengths, and
64-byte CQE sizes for user CQs?
1. attr->cqe (an unsigned int from userspace) is assigned to signed int
entries and passed to roundup_pow_of_two(entries) before checking
dev->xdev->caps.max_cqes. If attr->cqe is in [0x40000001, 0x7fffffff],
roundup_pow_of_two(entries) returns 0x80000000UL, overflowing signed int
entries to INT_MIN; if attr->cqe >= 0x80000000, entries is negative and
sign-extends in roundup_pow_of_two(), causing a 1UL << 64 shift UB. Both
bypass the entries > dev->xdev->caps.max_cqes check. Even when entries >
dev->xdev->caps.max_cqes without overflowing, silently clamping entries to
dev->xdev->caps.max_cqes instead of returning -EINVAL creates a hardware CQ
smaller than the ring buffer allocated by userspace.
2. On hardware where is_support_cqe64(dev->xdev) is true,
xsc_ib_alloc_ucontext() advertises XSC_HW_FEATURE_FLAG_SUPPORT_CQE64 to
userspace, and xsc_ib_create_cq() uses sizeof(struct xsc_cqe64) (64 bytes)
for kernel CQs. However, create_cq_user() checks:
if (ucmd.cqe_size != sizeof(struct xsc_cqe))
return -EINVAL;
which rejects 64-byte CQEs from userspace (or maps only half the required
pages if userspace passes 32 on 64-byte CQE hardware). In addition,
create_cq_user() never initializes cq->xcq.cqe_sz, unlike
create_cq_kernel().
3. Neither create_cq_user() nor xsc_ib_create_cq() checks udata->inlen or
udata->outlen before calling ib_copy_from_udata() and ib_copy_to_udata(),
and unsupported bits in attr->flags are not rejected.
> + cq->cqe_size = cqe_size;
> + cqb->ctx_ex.ctx.log_cq_sz = ilog2(entries);
> + cqb->ctx_ex.ctx.glb_func_id = cpu_to_be16(dev->xdev->glb_func_id);
> +
> + err = xsc_vector2eqn(dev->xdev, vector, &eqn, &irqn);
> + if (err)
> + goto err_cqb;
> +
> + cqb->ctx_ex.ctx.eqn = eqn;
> + cqb->ctx_ex.ctx.eqn = cpu_to_be16(cqb->ctx_ex.ctx.eqn);
[Severity: Low]
In drivers/net/ethernet/yunsilicon/xsc/common/driver.h, xsc_vector2eqn() is
declared as:
int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
unsigned int *irqn);
In xsc_ib_create_cq(), eqn is declared as unsigned int and irqn as int,
which triggers -Wpointer-sign warnings on both arguments when calling
xsc_vector2eqn(dev->xdev, vector, &eqn, &irqn).
Also, cqb->ctx_ex.ctx.eqn is __be16; assigning host-endian eqn to it before
calling cpu_to_be16(cqb->ctx_ex.ctx.eqn) triggers sparse endianness
warnings. Should this be cqb->ctx_ex.ctx.eqn = cpu_to_be16(eqn) with eqn
declared as int and irqn as unsigned int?
> + cqb->ctx_ex.page_shift = XSC_PAGE_SHIFT_4K;
> + if (cq->create_flags & IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN)
> + cqb->ctx_ex.overrun_ignore = 1;
> +
> + /* the cq becomes visible to the event handlers once
> + * xsc_core_create_cq() inserts it into the tree, so the
> + * callbacks must be in place before that
> + */
> + cq->xcq.comp = xsc_ib_cq_comp;
> + cq->xcq.event = xsc_ib_cq_event;
> +
> + err = xsc_core_create_cq(dev->xdev, &cq->xcq, cqb, inlen);
> + if (err)
> + goto err_cqb;
> +
> + ibdev_dbg(&dev->ib_dev, "succeeded to create cqn %d, vector=%d, cq_sz=%d, eqn=%d\n",
> + cq->xcq.cqn, vector, entries, eqn);
> + cq->xcq.irqn = irqn;
> +
> + if (udata) {
> + if (ib_copy_to_udata(udata, &cq->xcq.cqn, sizeof(__u32))) {
> + err = -EFAULT;
> + goto err_cmd;
> + }
> + }
> +
> + INIT_LIST_HEAD(&cq->send_qp_list);
> + INIT_LIST_HEAD(&cq->recv_qp_list);
> + INIT_LIST_HEAD(&cq->err_state_qp_list);
> + spin_lock_init(&cq->err_state_qp_lock);
[Severity: Medium]
Should send_qp_list, recv_qp_list, err_state_qp_list, and err_state_qp_lock
be initialized before calling xsc_core_create_cq(), and should
xsc_ib_cq_comp() guard against a NULL comp_handler?
xsc_core_create_cq() inserts &cq->xcq into xdev->dev_res->cq_table.tree and
immediately arms the CQ in hardware via xsc_arm_cq(xdev, cq->cqn, 0, 0). At
that point, the CQ is visible to EQ interrupts (and for kernel CQs created
via __ib_alloc_cq(), ibcq->comp_handler is not set until after
ops.create_cq() returns, while xsc_ib_cq_comp() calls ibcq->comp_handler
unconditionally). Initializing the lists and err_state_qp_lock after
xsc_core_create_cq() and after the ib_copy_to_udata() error path leaves
them uninitialized while the CQ is already live.
> diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
> index a212a5888b214..c3f1046158c4b 100644
> --- a/drivers/infiniband/hw/xsc/main.c
> +++ b/drivers/infiniband/hw/xsc/main.c
> @@ -511,9 +511,16 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
>
> dev->ib_dev.ops.alloc_pd = xsc_ib_alloc_pd;
> dev->ib_dev.ops.dealloc_pd = xsc_ib_dealloc_pd;
> + dev->ib_dev.ops.create_ah = xsc_ib_create_ah;
> + dev->ib_dev.ops.query_ah = xsc_ib_query_ah;
> + dev->ib_dev.ops.destroy_ah = xsc_ib_destroy_ah;
[Severity: High]
Should xsc_ib_dev_setting() also wire dev->ib_dev.ops.create_user_ah =
xsc_ib_create_ah?
In drivers/infiniband/core/uverbs_cmd.c, UVERBS_OBJECT_AH requires
UAPI_DEF_OBJ_NEEDS_FN(create_user_ah), and _rdma_create_ah() in
drivers/infiniband/core/verbs.c calls device->ops.create_user_ah(ah,
&init_attr, udata) when udata is non-NULL, while calling
device->ops.create_ah(ah, &init_attr, NULL) only for kernel callers. Without
create_user_ah set, userspace cannot create AHs via uverbs, and the udata
response handling in xsc_ib_create_ah() is unreachable.
> diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
> new file mode 100644
> index 0000000000000..a41d880cf03d9
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/qp.c
> @@ -0,0 +1,257 @@
[ ... ]
> +static int is_qp1(enum ib_qp_type qp_type)
> +{
> + return qp_type == IB_QPT_GSI;
> +}
[Severity: Low]
At this commit, is_qp1() has no callers in drivers/infiniband/hw/xsc/qp.c
(callers are added in the next commit in the series, f535ae7002b2
("RDMA/xsc: add QP verbs")), which triggers a -Wunused-function warning
during bisection when building at this commit. Could is_qp1() be moved to
the commit that introduces its callers?
[ ... ]
> +int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
> + enum xsc_qp_state cur_state, enum xsc_qp_state state,
> + bool *triggered)
> +{
> + struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
> + struct xsc_ib_cq *send_xcq, *recv_xcq;
> + unsigned long flags;
> +
> + *triggered = false;
> + ibdev_dbg(&dev->ib_dev, "modify qp: qpid %d, cur_qp_state %d, qp_state %d\n",
> + qp->xqp.qpn, cur_state, state);
> + if (cur_state == XSC_QP_STATE_ERR && state != XSC_QP_STATE_ERR) {
> + if (!qp->ibqp.srq && qp->recv_cq) {
> + recv_xcq = to_xcq(qp->recv_cq);
> + spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
> + list_for_each_entry_safe(err_rq_node, tmp,
> + &recv_xcq->err_state_qp_list, entry) {
> + if (err_rq_node->qp_id == qp->xqp.qpn && !err_rq_node->is_sq) {
> + list_del(&err_rq_node->entry);
> + kfree(err_rq_node);
> + }
> + }
> + spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
> + }
> +
> + if (qp->send_cq) {
> + send_xcq = to_xcq(qp->send_cq);
> + spin_lock_irqsave(&send_xcq->err_state_qp_lock, flags);
> + list_for_each_entry_safe(err_sq_node, tmp,
> + &send_xcq->err_state_qp_list, entry) {
> + if (err_sq_node->qp_id == qp->xqp.qpn && err_sq_node->is_sq) {
> + list_del(&err_sq_node->entry);
> + kfree(err_sq_node);
> + }
> + }
> + spin_unlock_irqrestore(&send_xcq->err_state_qp_lock, flags);
> + }
> + return 0;
> + }
> +
> + if (cur_state != XSC_QP_STATE_ERR && state == XSC_QP_STATE_ERR &&
> + !qp->has_trig_cq_comp_handle) {
> + if (!qp->ibqp.srq && qp->recv_cq) {
> + err_rq_node = kzalloc_obj(*err_rq_node, GFP_ATOMIC);
> + if (!err_rq_node)
> + return -ENOMEM;
> + err_rq_node->qp_id = qp->xqp.qpn;
> + err_rq_node->is_sq = false;
> + recv_xcq = to_xcq(qp->recv_cq);
> + spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
> + list_add_tail(&err_rq_node->entry, &recv_xcq->err_state_qp_list);
> + spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
> + }
> +
> + if (qp->send_cq) {
> + err_sq_node = kzalloc_obj(*err_sq_node, GFP_ATOMIC);
> + if (!err_sq_node) {
> + if (err_rq_node) {
> + spin_lock_irqsave(&recv_xcq->err_state_qp_lock, flags);
> + list_del(&err_rq_node->entry);
> + spin_unlock_irqrestore(&recv_xcq->err_state_qp_lock, flags);
> + kfree(err_rq_node);
> + }
> + return -ENOMEM;
> + }
> + err_sq_node->qp_id = qp->xqp.qpn;
> + err_sq_node->is_sq = true;
> + send_xcq = to_xcq(qp->send_cq);
> + spin_lock_irqsave(&send_xcq->err_state_qp_lock, flags);
> + list_add_tail(&err_sq_node->entry, &send_xcq->err_state_qp_list);
> + spin_unlock_irqrestore(&send_xcq->err_state_qp_lock, flags);
> + }
> +
> + qp->has_trig_cq_comp_handle = true;
> + *triggered = true;
> + }
> + return 0;
> +}
[Severity: High]
Can xsc_ib_err_state_qp() leave stale or duplicate xsc_err_state_qp_node
entries on cq->err_state_qp_list?
1. When xsc_poll_one() calls xsc_ib_err_state_qp(dev, *cur_qp,
(*cur_qp)->state, XSC_QP_STATE_ERR, trigger), (*cur_qp)->state is an enum
ib_qp_state (not converted to enum xsc_qp_state), and (*cur_qp)->state is
not updated to reflect the error state. When the QP is later modified out
of error (e.g. to RESET), cur_state == XSC_QP_STATE_ERR is false, so the
err_rq_node and err_sq_node entries are never removed from
err_state_qp_list. Moreover, the removal branch does not reset
qp->has_trig_cq_comp_handle = false.
2. qp->has_trig_cq_comp_handle is tested and set without holding a per-QP
lock. If a QP has distinct send_cq and recv_cq that are polled concurrently
on two CPUs when both queues encounter an error CQE, both CPUs can see
!qp->has_trig_cq_comp_handle and insert duplicate err_rq_node and
err_sq_node entries onto the CQs' err_state_qp_list.
> diff --git a/drivers/infiniband/hw/xsc/srq.c b/drivers/infiniband/hw/xsc/srq.c
> new file mode 100644
> index 0000000000000..b791fc97f6861
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/srq.c
> @@ -0,0 +1,462 @@
[ ... ]
> +void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp)
> +{
> + int srqn = srq->xsrq.srqn;
> + struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
> +
> + if (!qp->srq_head_wr)
> + return;
> +
> + if (qp->srq_tail_wr)
> + qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
> + cache_wr[srqn].head_wr = qp->srq_head_wr;
> + qp->srq_head_wr = NULL;
> + qp->srq_tail_wr = NULL;
> +}
> +
> +void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id)
> +{
> + struct ib_recv_wr *tmp_wr, *prev_wr = NULL;
> + int srqn = srq->xsrq.srqn;
> + struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&srq->lock, flags);
> + tmp_wr = qp->srq_head_wr;
> + for (; tmp_wr; tmp_wr = tmp_wr->next) {
> + if (tmp_wr->wr_id == wr_id) {
> + if (prev_wr)
> + prev_wr->next = tmp_wr->next;
> + else
> + qp->srq_head_wr = tmp_wr->next;
> + if (qp->srq_tail_wr == tmp_wr)
> + qp->srq_tail_wr = prev_wr;
> +
> + kfree(tmp_wr->sg_list);
> + kfree(tmp_wr);
> + cache_wr[srqn].wr_num--;
> + break;
> + }
> + prev_wr = tmp_wr;
> + }
> + spin_unlock_irqrestore(&srq->lock, flags);
> +}
[ ... ]
> +int xsc_ib_create_srq(struct ib_srq *ib_srq,
> + struct ib_srq_init_attr *init_attr,
> + struct ib_udata *udata)
> +{
[ ... ]
> + if (udata) {
> + struct xsc_ib_create_srq_resp resp = {
> + .srqn = srq->xsrq.srqn,
> + };
> +
> + if (ib_copy_to_udata(udata, &resp,
> + min(udata->outlen, sizeof(resp)))) {
> + ibdev_dbg(&dev->ib_dev, "copy to user failed\n");
> + err = -EFAULT;
> + goto err_rsp;
> + }
> + } else {
> + err = xsc_init_cache_wr(init_attr, srq);
> + if (err)
> + goto err_rsp;
> + INIT_LIST_HEAD(&srq->xsrq.srq_qp_list);
> + }
> +
> + init_attr->attr.max_wr = srq->xsrq.max - 1;
[ ... ]
> +static int xsc_cache_srq_wr(struct xsc_ib_srq *srq, const struct ib_recv_wr *wr, int *nreq)
> +{
> + int i, n;
> + int srqn = srq->xsrq.srqn;
> + struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
> + struct ib_recv_wr *head_wr = NULL, *new_wr = NULL, *prev_wr = NULL;
> + const struct ib_recv_wr *tmp;
> +
> + for (n = 0, tmp = wr; tmp; tmp = tmp->next, n++)
> + ;
> +
> + if (cache_wr[srqn].wr_num + n > cache_wr[srqn].max_wr_num)
> + return -1;
> +
> + for (n = 0; wr; wr = wr->next, n++) {
> + new_wr = kcalloc(1, sizeof(*new_wr), GFP_ATOMIC);
> + if (!new_wr)
> + goto err;
> + new_wr->sg_list = kcalloc(wr->num_sge, sizeof(*new_wr->sg_list),
> + GFP_ATOMIC);
> + if (!new_wr->sg_list) {
> + kfree(new_wr);
> + goto err;
> + }
> + new_wr->wr_id = wr->wr_id;
> + for (i = 0; i < wr->num_sge; i++)
> + memcpy(&new_wr->sg_list[i], &wr->sg_list[i],
> + sizeof(struct ib_sge));
> + new_wr->num_sge = wr->num_sge;
> + if (prev_wr)
> + prev_wr->next = new_wr;
> + else
> + head_wr = new_wr;
> + prev_wr = new_wr;
> + };
> +
> + if (!cache_wr[srqn].head_wr)
> + cache_wr[srqn].head_wr = head_wr;
> + else
> + cache_wr[srqn].tail_wr->next = head_wr;
> +
> + cache_wr[srqn].tail_wr = new_wr;
> + cache_wr[srqn].wr_num += n;
> + *nreq = n;
> +
> + return 0;
> +
> +err:
> + xsc_free_all_cache_wrs(head_wr);
> + return -ENOMEM;
> +}
> +
> +void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init)
> +{
> + int srqn = srq->xsrq.srqn;
> + struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
> + int err = 0;
> + int i;
> + const struct ib_recv_wr *tmp_wr;
> + const struct ib_recv_wr *bad_wr;
> + struct ib_recv_wr *wr_head, *next_wr;
> + struct ib_recv_wr *prev_tail;
> +
> + if (!cache_wr[srqn].head_wr)
> + return;
> +
> + wr_head = cache_wr[srqn].head_wr;
> + next_wr = cache_wr[srqn].head_wr;
> + prev_tail = qp->srq_tail_wr;
> +
> + if (init || !qp->srq_tail_wr)
> + qp->srq_head_wr = wr_head;
> + else
> + qp->srq_tail_wr->next = wr_head;
> +
> + qp->srq_tail_wr = wr_head;
> + for (i = 0; next_wr && i < (wr_num - 1); i++) {
> + next_wr = next_wr->next;
> + if (next_wr)
> + qp->srq_tail_wr = next_wr;
> + }
> +
> + cache_wr[srqn].head_wr = qp->srq_tail_wr->next;
> + qp->srq_tail_wr->next = NULL;
> + tmp_wr = wr_head;
> + err = xsc_ib_post_recv(&qp->ibqp, tmp_wr, &bad_wr);
> + if (err) {
> + qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
> + qp->srq_tail_wr = NULL;
> + for (; wr_head && wr_head != bad_wr; wr_head = wr_head->next)
> + qp->srq_tail_wr = wr_head;
> +
> + if (!qp->srq_tail_wr) {
> + cache_wr[srqn].head_wr = wr_head;
> + if (prev_tail && !init) {
> + prev_tail->next = NULL;
> + qp->srq_tail_wr = prev_tail;
> + } else {
> + qp->srq_head_wr = NULL;
> + }
> + } else {
> + cache_wr[srq->xsrq.srqn].head_wr = qp->srq_tail_wr->next;
> + qp->srq_tail_wr->next = NULL;
> + }
> + }
> +}
[Severity: Critical]
Can xsc_cache_srq_wr() cause a use-after-free via a stale
cache_wr[srqn].tail_wr pointer?
1. When xsc_post_cache_wrs() moves all cached WRs from
cache_wr[srqn].head_wr onto an attached QP, cache_wr[srqn].head_wr becomes
NULL, but cache_wr[srqn].tail_wr is not cleared and remains pointing to the
last WR now owned by the QP.
2. When that WR completes on the QP, xsc_free_one_cache_srq_wr() frees it
with kfree(tmp_wr), leaving cache_wr[srqn].tail_wr as a dangling pointer.
3. If xsc_restore_cache_srq_wrs() is then called while
cache_wr[srqn].head_wr is NULL (or if xsc_ib_post_recv() fails inside
xsc_post_cache_wrs() when cache_wr[srqn].head_wr was emptied),
cache_wr[srqn].head_wr becomes non-NULL again without updating
cache_wr[srqn].tail_wr.
4. On the next call to xsc_ib_post_srq_recv() -> xsc_cache_srq_wr(),
cache_wr[srqn].head_wr is non-NULL, so it executes:
cache_wr[srqn].tail_wr->next = head_wr;
which writes to the freed tmp_wr slab object (or corrupts an active QP WR
chain) and leaks head_wr.
Also, if xsc_ib_post_srq_recv() is called with wr == NULL while
cache_wr[srqn].head_wr is non-NULL, xsc_cache_srq_wr() sets
cache_wr[srqn].tail_wr = new_wr (NULL), causing a NULL pointer dereference
on the subsequent xsc_cache_srq_wr() call.
> +int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
> + const struct ib_recv_wr **bad_wr)
> +{
> + struct xsc_ib_srq *srq = to_xsrq(ibsrq);
> + struct xsc_ib_dev *dev = to_xdev(ibsrq->device);
> + struct xsc_srq_qp_node *qp_node;
> + struct xsc_ib_qp *qp;
> + unsigned long flags;
> + int err = 0;
> + int flush_wqe_cnt = 0;
> + int nreq = 0;
> + int wr_num = 0;
> +
> + spin_lock_irqsave(&srq->lock, flags);
> +
> + err = xsc_cache_srq_wr(srq, wr, &nreq);
> + if (err == -ENOMEM) {
> + ibdev_err(&dev->ib_dev, "failed to cache SRQ WRs, err=%d\n", err);
> + *bad_wr = wr;
> + goto out;
> + }
> + if (err) {
> + ibdev_err(&dev->ib_dev, "No more space to cache SRQ WRs...");
> + err = -EINVAL;
> + *bad_wr = wr;
> + goto out;
> + }
> +
> + if (list_empty(&srq->xsrq.srq_qp_list))
> + goto out;
> +
> + list_sort(NULL, &srq->xsrq.srq_qp_list, cmp_flush_wqe_cnt);
> +
> + list_for_each_entry(qp_node, &srq->xsrq.srq_qp_list, entry) {
> + if (!qp_node || nreq <= 0)
> + break;
> +
> + qp = to_xqp(qp_node->ibqp);
> + flush_wqe_cnt = atomic_read(&qp->rq.flush_wqe_cnt);
> + if (flush_wqe_cnt >= XSC_SRQ_MIN_WR_NUM)
> + continue;
> + wr_num = min_t(int, nreq, XSC_SRQ_MIN_WR_NUM - flush_wqe_cnt);
> + nreq -= wr_num;
> + ibdev_dbg(&dev->ib_dev, "srq_qp_list qp node qp %d, flush cnt %d, nreq %d, wr_num %d\n",
> + qp_node->qp_id, flush_wqe_cnt, nreq, wr_num);
> + xsc_post_cache_wrs(srq, qp, wr_num, false);
> + }
[Severity: High]
How are attached QPs replenished once their initial 32 WRs
(XSC_SRQ_MIN_WR_NUM) complete?
1. When > 32 WRs are posted to an SRQ, xsc_ib_post_srq_recv() posts at most
XSC_SRQ_MIN_WR_NUM (32) WRs to each attached QP and leaves the rest in
cache_wr[srqn].head_wr. When those WRs complete on a QP in xsc_poll_one()
-> xsc_free_one_cache_srq_wr(), the completed WR is freed and
cache_wr[srqn].wr_num is decremented, but xsc_post_cache_wrs() is never
called to replenish the QP's RQ from cache_wr[srqn].head_wr. Moreover,
xsc_ib_post_srq_recv() bounds wr_num by nreq (the count from the current
call) rather than the total cached WRs in cache_wr[srqn].head_wr, leaving
previously cached WRs stranded.
2. In xsc_ib_create_srq(), xsc_init_cache_wr(init_attr, srq) is called
before init_attr->attr.max_wr = srq->xsrq.max - 1, so
cache_wr[srqn].max_wr_num stores the unrounded max_wr while
xsc_ib_create_srq() and xsc_ib_query_srq() report srq->xsrq.max - 1. Posting
up to the reported max_wr will then fail with -EINVAL in xsc_cache_srq_wr().
3. Neither xsc_ib_create_srq() nor xsc_cache_srq_wr() validates max_sge or
wr->num_sge against srq->xsrq.max_gs (or < 0). If a WR with num_sge >
qp->rq.max_gs is cached in xsc_cache_srq_wr(), xsc_ib_post_recv() inside
xsc_post_cache_wrs() rejects it with -EINVAL, and xsc_post_cache_wrs()
places bad_wr back at the head of cache_wr[srqn].head_wr while
xsc_ib_post_srq_recv() returns 0, permanently blocking subsequent SRQ posts
from reaching attached QPs.
> diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
> index eb4d9e6309744..d7b0ff1768383 100644
> --- a/drivers/infiniband/hw/xsc/xsc_ib.h
> +++ b/drivers/infiniband/hw/xsc/xsc_ib.h
> @@ -324,8 +325,25 @@ void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
> void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
>
> int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);
> +
> +/*
> + * Wake the CQ completion handlers of a QP that has just moved to ERR. Must
> + * be called with no cq->lock held: the handler may poll or rearm the CQ,
> + * which takes that lock again.
> + */
> +static inline void xsc_ib_trigger_cq_comp_handler(struct ib_cq *send_cq,
> + struct ib_cq *recv_cq)
> +{
> + if (send_cq)
> + send_cq->comp_handler(send_cq, send_cq->cq_context);
> +
> + if (recv_cq && send_cq != recv_cq)
> + recv_cq->comp_handler(recv_cq, recv_cq->cq_context);
> +}
[Severity: Medium]
What happens in xsc_ib_trigger_cq_comp_handler() when a kernel CQ uses
IB_POLL_DIRECT or has a NULL comp_handler?
For CQs allocated with IB_POLL_DIRECT, cq->comp_handler is set to
ib_cq_completion_direct() in drivers/infiniband/core/cq.c, which triggers
WARN_ONCE(1, "got unsolicited completion for CQ %p\n", cq). Should
xsc_ib_trigger_cq_comp_handler() check that comp_handler is non-NULL and
that poll_ctx != IB_POLL_DIRECT before invoking comp_handler?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=12
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 13/15] RDMA/xsc: add QP verbs
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (11 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 12/15] RDMA/xsc: add RDMA CQ/SRQ/AH Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 14/15] RDMA/xsc: add the RDMA send path Xin Tian
2026-11-04 15:57 ` [PATCH v1 15/15] RDMA/xsc: add RDMA counters and sysfs Xin Tian
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement queue pair creation, modification, query and destruction.
Kernel QPs get per-slot bookkeeping arrays mapping WQE slots to
work requests, tracking signaled completions and the chain of
previously completed WQEs that the poll loop uses for lost-completion
detection; GSI QPs additionally get a coherent DMA ring holding
pre-built RoCEv2 headers for QP1 traffic. Supported types are RC,
the MAD QP0/QP1 pair and raw packet queues; queue sizing derives
from the firmware capabilities, with the send queue inline data
limited by the data segment count.
The QP event callback is installed before the QP is registered in
the core device for the same reason as with CQs. After creation
the QP is linked onto the send and receive CQ lists under both CQ
locks, ordered by CQN with lockdep nesting to avoid ABBA deadlock
when the two CQs differ. Destruction moves the QP to RESET, unlinks
it from the CQ lists, sweeps stale completions from both CQs and
then destroys the firmware object.
modify_qp validates the transition with ib_modify_qp_is_ok() and
builds the firmware context: PSNs, retries, RNR handling and access
flags, plus the path record for AV modifications. The RoCE path
resolves source and destination GID/IP/MAC/VLAN through the core
helpers and derives the UDP source port from the flow label. QPs
with an SRQ receive their first cached WRs when moved from RESET to
INIT, and RESET transitions reinitialize the work queues and clean
both CQs.
User QP creation returns no driver data, so the core's empty-udata
response is used and the corresponding struct is dropped from the
ABI header.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/hw/xsc/main.c | 6 +
drivers/infiniband/hw/xsc/qp.c | 1400 +++++++++++++++++++++++++++-
drivers/infiniband/hw/xsc/xsc_ib.h | 8 +-
include/uapi/rdma/xsc-abi.h | 5 -
4 files changed, 1398 insertions(+), 21 deletions(-)
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
index c3f104615..138ca1a80 100644
--- a/drivers/infiniband/hw/xsc/main.c
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -517,6 +517,11 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.get_link_layer = xsc_ib_port_link_layer;
+ dev->ib_dev.ops.create_qp = xsc_ib_create_qp;
+ dev->ib_dev.ops.modify_qp = xsc_ib_modify_qp;
+ dev->ib_dev.ops.query_qp = xsc_ib_query_qp;
+ dev->ib_dev.ops.destroy_qp = xsc_ib_destroy_qp;
+ dev->ib_dev.ops.post_recv = xsc_ib_post_recv;
dev->ib_dev.ops.create_cq = xsc_ib_create_cq;
dev->ib_dev.ops.destroy_cq = xsc_ib_destroy_cq;
dev->ib_dev.ops.poll_cq = xsc_ib_poll_cq;
@@ -539,6 +544,7 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ah, xsc_ib_ah, ibah);
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_cq, xsc_ib_cq, ibcq);
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_pd, xsc_ib_pd, ibpd);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_qp, xsc_ib_qp, ibqp);
dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_srq, xsc_ib_srq, ibsrq);
}
diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
index a41d880cf..0dfb10294 100644
--- a/drivers/infiniband/hw/xsc/qp.c
+++ b/drivers/infiniband/hw/xsc/qp.c
@@ -6,6 +6,7 @@
#include <linux/module.h>
#include <rdma/ib_umem.h>
+#include <rdma/ib_cache.h>
#include "xsc_ib.h"
#include <rdma/xsc-abi.h>
#include "common/xsc_hsi.h"
@@ -28,11 +29,21 @@ enum {
#define UDP_SPORT_MASK 0xffff
#define UDP_SPORT_OFFSET 0
+static int is_qp0(enum ib_qp_type qp_type)
+{
+ return qp_type == IB_QPT_SMI;
+}
+
static int is_qp1(enum ib_qp_type qp_type)
{
return qp_type == IB_QPT_GSI;
}
+static int is_sqp(enum ib_qp_type qp_type)
+{
+ return is_qp0(qp_type) || is_qp1(qp_type);
+}
+
static void *get_wqe(struct xsc_ib_qp *qp, int offset)
{
return xsc_buf_offset(&qp->buf, offset);
@@ -48,6 +59,300 @@ static void *get_seg_wqe(void *first, int n)
return first + (n << XSC_BASE_WQE_SHIFT);
}
+static void xsc_ib_qp_event(struct xsc_core_qp *qp, int type)
+{
+ struct ib_qp *ibqp = &core_to_xqp(qp)->ibqp;
+ struct ib_event event;
+
+ if (ibqp->event_handler) {
+ event.device = ibqp->device;
+ event.element.qp = ibqp;
+ switch (type) {
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ event.event = IB_EVENT_QP_FATAL;
+ break;
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ event.event = IB_EVENT_QP_REQ_ERR;
+ break;
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ event.event = IB_EVENT_QP_ACCESS_ERR;
+ break;
+ default:
+ pr_warn("xsc_ib: Unexpected event type %d on QP %06x\n", type, qp->qpn);
+ return;
+ }
+
+ ibqp->event_handler(&event, ibqp->qp_context);
+ pr_info("xsc_ib: xsc async event 0x%x, ib event %d\n", type, event.event);
+ }
+}
+
+static int set_rq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *init_attr,
+ int has_rq, struct xsc_ib_qp *qp, struct xsc_ib_create_qp *ucmd)
+{
+ u32 wqe_cnt = init_attr->cap.max_recv_wr ?
+ roundup_pow_of_two(init_attr->cap.max_recv_wr) : 0;
+
+ if (init_attr->srq)
+ wqe_cnt = to_xsrq(init_attr->srq)->xsrq.max;
+
+ if (wqe_cnt > dev->xdev->caps.max_wqes) {
+ ibdev_err(&dev->ib_dev, "max_recv_wr:%d exceed max rq depth\n",
+ init_attr->cap.max_recv_wr);
+ return -EINVAL;
+ }
+
+ if (!has_rq) {
+ qp->rq.max_gs = 0;
+ qp->rq.wqe_cnt = 0;
+ qp->rq.wqe_shift = 0;
+ } else if (!ucmd) {
+ qp->rq.wqe_cnt = wqe_cnt;
+ if (qp->xqp.qp_type == IB_QPT_GSI)
+ qp->rq.wqe_shift = ilog2(xsc_get_recv_ds_num(dev->xdev))
+ + XSC_RECV_WQE_SHIFT;
+ else
+ qp->rq.wqe_shift = dev->xdev->caps.recv_wqe_shift;
+ qp->rq.max_gs = dev->xdev->caps.recv_ds_num;
+ qp->rq.max_post = qp->rq.wqe_cnt;
+ }
+
+ return 0;
+}
+
+static int calc_sq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *attr,
+ struct xsc_ib_qp *qp)
+{
+ int wqe_size;
+ size_t wq_size;
+
+ if (!attr->cap.max_send_wr) {
+ ibdev_err(&dev->ib_dev, "invalid max_send_wr:%d\n", attr->cap.max_send_wr);
+ return -1;
+ }
+
+ wqe_size = 1 << dev->xdev->caps.send_wqe_shift;
+ qp->max_inline_data = (dev->xdev->caps.send_ds_num - 2) * sizeof(struct xsc_wqe_data_seg);
+ attr->cap.max_inline_data = qp->max_inline_data;
+
+ attr->cap.max_send_wr = min_t(u32, attr->cap.max_send_wr, dev->xdev->caps.max_wqes);
+ qp->sq.wqe_cnt = roundup_pow_of_two(attr->cap.max_send_wr);
+
+ qp->sq.ds_cnt = qp->sq.wqe_cnt << (dev->xdev->caps.send_wqe_shift - XSC_BASE_WQE_SHIFT);
+ wq_size = (size_t)qp->sq.wqe_cnt * wqe_size;
+ if (wq_size > INT_MAX) {
+ ibdev_err(&dev->ib_dev, "SQ size overflow\n");
+ return -EINVAL;
+ }
+ qp->sq.wqe_shift = ilog2(wqe_size);
+ qp->sq.max_gs = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM - XSC_RADDR_SEG_NUM;
+ qp->sq.max_post = qp->sq.wqe_cnt;
+
+ return wq_size;
+}
+
+static int qp_has_rq(struct ib_qp_init_attr *attr)
+{
+ if (!attr->srq && !attr->cap.max_recv_wr)
+ return 0;
+
+ return 1;
+}
+
+static enum xsc_qp_state to_xsc_state(enum ib_qp_state state)
+{
+ switch (state) {
+ case IB_QPS_RESET: return XSC_QP_STATE_RST;
+ case IB_QPS_INIT: return XSC_QP_STATE_INIT;
+ case IB_QPS_RTR: return XSC_QP_STATE_RTR;
+ case IB_QPS_RTS: return XSC_QP_STATE_RTS;
+ case IB_QPS_SQD: return XSC_QP_STATE_SQD;
+ case IB_QPS_SQE: return XSC_QP_STATE_SQER;
+ case IB_QPS_ERR: return XSC_QP_STATE_ERR;
+ default: return -1;
+ }
+}
+
+static char *qp_state_to_str(enum ib_qp_state state)
+{
+ switch (state) {
+ case IB_QPS_RESET: return "RST";
+ case IB_QPS_INIT: return "INIT";
+ case IB_QPS_RTR: return "RTR";
+ case IB_QPS_RTS: return "RTS";
+ case IB_QPS_SQD: return "SQD";
+ case IB_QPS_SQE: return "SQE";
+ case IB_QPS_ERR: return "ERR";
+ default: return "UNKNOWN";
+ }
+}
+
+static struct ib_umem *qp_umem_get(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ struct ib_udata *udata, struct ib_pd *pd,
+ struct xsc_ib_create_qp *ucmd)
+{
+ struct ib_umem *umem;
+
+ umem = ib_umem_get_va(&dev->ib_dev, ucmd->buf_addr, qp->buf_size,
+ IB_ACCESS_LOCAL_WRITE);
+ return umem;
+}
+
+static int create_user_qp(struct xsc_ib_dev *dev, struct ib_pd *pd,
+ struct xsc_ib_qp *qp, struct ib_udata *udata,
+ struct xsc_ib_create_qp *ucmd,
+ struct xsc_create_qp_mbox_in **in, int *inlen)
+{
+ int page_shift;
+ int npages;
+ u32 offset;
+ int ncont;
+ int err;
+ int hw_npages;
+ u32 sq_shift, rq_shift;
+ u32 sq_wqe_cnt, rq_wqe_cnt;
+ struct ib_umem_dmabuf *umem_dmabuf;
+
+ ibdev_dbg(&dev->ib_dev, "buf_addr:0x%lx db_addr:0x%lx sq cnt:%u, rq cnt:%u, rq shift:%u\n",
+ (uintptr_t)ucmd->buf_addr, (uintptr_t)ucmd->db_addr,
+ ucmd->sq_wqe_count, ucmd->rq_wqe_count, ucmd->rq_wqe_shift);
+
+ sq_shift = dev->xdev->caps.send_wqe_shift;
+ rq_shift = ucmd->rq_wqe_shift;
+ if (sq_shift < XSC_BASE_WQE_SHIFT || sq_shift > 31 ||
+ (ucmd->rq_wqe_count &&
+ (rq_shift < XSC_BASE_WQE_SHIFT || rq_shift > 31))) {
+ ibdev_err(&dev->ib_dev, "invalid user queue parameters\n");
+ return -EINVAL;
+ }
+
+ if (ucmd->sq_wqe_count) {
+ if (!is_power_of_2(ucmd->sq_wqe_count) ||
+ ucmd->sq_wqe_count < (1u << (sq_shift - XSC_BASE_WQE_SHIFT))) {
+ ibdev_err(&dev->ib_dev, "invalid user sq parameters\n");
+ return -EINVAL;
+ }
+ sq_wqe_cnt = ucmd->sq_wqe_count >> (sq_shift - XSC_BASE_WQE_SHIFT);
+ } else {
+ sq_wqe_cnt = 0;
+ }
+
+ if (ucmd->rq_wqe_count) {
+ if (!is_power_of_2(ucmd->rq_wqe_count) ||
+ ucmd->rq_wqe_count < (1u << (rq_shift - XSC_BASE_WQE_SHIFT))) {
+ ibdev_err(&dev->ib_dev, "invalid user rq parameters\n");
+ return -EINVAL;
+ }
+ rq_wqe_cnt = ucmd->rq_wqe_count >> (rq_shift - XSC_BASE_WQE_SHIFT);
+ } else {
+ rq_wqe_cnt = 0;
+ }
+
+ if (sq_wqe_cnt > dev->xdev->caps.max_wqes ||
+ rq_wqe_cnt > dev->xdev->caps.max_wqes) {
+ ibdev_err(&dev->ib_dev, "user queue depth exceeds max %u\n",
+ dev->xdev->caps.max_wqes);
+ return -EINVAL;
+ }
+
+ qp->sq.ds_cnt = ucmd->sq_wqe_count;
+ qp->sq.wqe_cnt = sq_wqe_cnt;
+ qp->sq.wqe_shift = sq_wqe_cnt ? sq_shift : 0;
+ qp->sq.max_gs = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM - XSC_RADDR_SEG_NUM;
+ qp->rq.ds_cnt = ucmd->rq_wqe_count;
+ qp->rq.wqe_cnt = rq_wqe_cnt;
+ qp->rq.wqe_shift = rq_wqe_cnt ? rq_shift : 0;
+ if (rq_wqe_cnt) {
+ qp->rq.max_gs = 1;
+ qp->rq.max_post = rq_wqe_cnt;
+ }
+
+ qp->rq.offset = 0;
+ qp->sq.offset = (u32)qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ qp->buf_size = (u64)qp->sq.offset +
+ ((u64)qp->sq.wqe_cnt << qp->sq.wqe_shift);
+ if (qp->buf_size > (u64)U16_MAX * XSC_PAGE_SIZE_4K) {
+ ibdev_err(&dev->ib_dev, "QP buffer size too large\n");
+ return -EINVAL;
+ }
+ hw_npages = DIV_ROUND_UP(qp->buf_size, XSC_PAGE_SIZE_4K);
+
+ if (udata->inlen >= offsetofend(typeof(*ucmd), dmabuf_fd) &&
+ ucmd->dmabuf_fd != -1) {
+ if (ucmd->dmabuf_sz < qp->buf_size) {
+ ibdev_err(&dev->ib_dev, "dmabuf_sz %llu < qp buf_size %llu\n",
+ ucmd->dmabuf_sz, qp->buf_size);
+ return -EINVAL;
+ }
+
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd->dmabuf_sz,
+ ucmd->dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
+
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_warn(&dev->ib_dev, "qp umem_dmabuf get failed (%ld)\n",
+ PTR_ERR(umem_dmabuf));
+ return -ENOMEM;
+ }
+
+ qp->umem = &umem_dmabuf->umem;
+ } else {
+ qp->umem = qp_umem_get(dev, qp, udata, pd, ucmd);
+ }
+
+ if (IS_ERR(qp->umem)) {
+ ibdev_err(&dev->ib_dev, "umem_get failed\n");
+ err = PTR_ERR(qp->umem);
+ goto err_uuar;
+ }
+
+ xsc_ib_cont_pages(qp->umem, ucmd->buf_addr, &npages, &page_shift,
+ &ncont);
+ if (ncont != npages) {
+ /* TODO: peer memory support failed */
+ page_shift = PAGE_SHIFT;
+ ncont = npages;
+ }
+
+ err = xsc_ib_get_buf_offset(ucmd->buf_addr, page_shift, &offset);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "bad offset:%d\n", offset);
+ goto err_umem;
+ }
+ ibdev_dbg(&dev->ib_dev, "npage:%d, page_shift:%d, ncont:%d, offset:%u, hw_npages %d\n",
+ npages, page_shift, ncont, offset, hw_npages);
+
+ *inlen = sizeof(**in) + (size_t)hw_npages * sizeof((*in)->req.pas[0]);
+ *in = xsc_vzalloc(*inlen);
+ if (!*in) {
+ err = -ENOMEM;
+ goto err_umem;
+ }
+ xsc_ib_populate_pas(dev, qp->umem, page_shift, (*in)->req.pas, hw_npages, true);
+ (*in)->req.pa_num = cpu_to_be16(hw_npages);
+
+ err = ib_respond_empty_udata(udata);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to respond to udata, err=%d\n", err);
+ goto err_umem;
+ }
+ qp->create_type = XSC_QP_USER;
+
+ return 0;
+
+err_umem:
+ xsc_vfree(*in);
+ *in = NULL;
+ ib_umem_release(qp->umem);
+
+err_uuar:
+ return err;
+}
+
+static void destroy_qp_user(struct ib_pd *__maybe_unused pd, struct xsc_ib_qp *qp)
+{
+ ib_umem_release(qp->umem);
+}
+
#define MAX_QP1_SQ_HDR_SIZE_V2 512
#define MAX_QP1_SQ_HDR_SIZE 86
/* Ethernet header = 14 */
@@ -57,6 +362,724 @@ static void *get_seg_wqe(void *first, int n)
/* iCRC = 4 */
#define MAX_QP1_RQ_HDR_SIZE_V2 512
+static int create_kernel_qp(struct xsc_ib_dev *dev,
+ struct ib_qp_init_attr *init_attr,
+ struct xsc_ib_qp *qp,
+ struct xsc_create_qp_mbox_in **in, int *inlen)
+{
+ int err;
+ int sq_size;
+ int hw_npages;
+
+ sq_size = calc_sq_size(dev, init_attr, qp);
+ if (sq_size < 0) {
+ err = -EINVAL;
+ ibdev_err(&dev->ib_dev, "err %d\n", err);
+ return err;
+ }
+
+ if (qp->rq.wqe_cnt)
+ qp->rq.ds_cnt = (u32)qp->rq.wqe_cnt <<
+ (qp->rq.wqe_shift - XSC_BASE_WQE_SHIFT);
+ else
+ qp->rq.ds_cnt = 0;
+ qp->rq.offset = 0;
+ qp->sq.offset = (u32)qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ qp->buf_size = (u64)qp->sq.offset + sq_size;
+ qp->send_psn = 0;
+
+ err = xsc_buf_alloc(dev->xdev, qp->buf_size, PAGE_SIZE, &qp->buf);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to alloc qp buffer,err=%d\n", err);
+ return err;
+ }
+
+ qp->sq.qend = qp->buf.direct.buf + qp->sq.offset + sq_size;
+ hw_npages = DIV_ROUND_UP(qp->buf_size, XSC_PAGE_SIZE_4K);
+ if (hw_npages > U16_MAX) {
+ ibdev_err(&dev->ib_dev, "QP page count exceeded\n");
+ err = -EINVAL;
+ goto err_buf;
+ }
+
+ *inlen = sizeof(**in) + (size_t)hw_npages * sizeof((*in)->req.pas[0]);
+ *in = xsc_vzalloc(*inlen);
+ if (!*in) {
+ err = -ENOMEM;
+ goto err_buf;
+ }
+
+ xsc_fill_page_array(&qp->buf, (*in)->req.pas, hw_npages);
+ (*in)->req.pa_num = cpu_to_be16(hw_npages);
+
+ qp->sq.wrid = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wrid), GFP_KERNEL);
+ qp->sq.wr_data = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wr_data), GFP_KERNEL);
+ qp->rq.wrid = kcalloc(qp->rq.wqe_cnt, sizeof(*qp->rq.wrid), GFP_KERNEL);
+ qp->sq.w_list = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.w_list), GFP_KERNEL);
+ qp->sq.wqe_head = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wqe_head), GFP_KERNEL);
+ qp->sq.wr_opcode = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wr_opcode), GFP_KERNEL);
+ qp->sq.need_flush = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.need_flush), GFP_KERNEL);
+ qp->sq.prev_ce_wqe = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.prev_ce_wqe), GFP_KERNEL);
+ qp->sq.last_comp_ce_wqe = -1;
+ qp->sq.last_post_ce_wqe = -1;
+
+ if (!qp->sq.wrid || !qp->sq.wr_data || !qp->rq.wrid || !qp->sq.prev_ce_wqe ||
+ !qp->sq.w_list || !qp->sq.wqe_head || !qp->sq.wr_opcode || !qp->sq.need_flush) {
+ err = -ENOMEM;
+ goto err_wrid;
+ }
+ qp->create_type = XSC_QP_KERNEL;
+
+ if (init_attr->qp_type == IB_QPT_GSI) {
+ qp->sq.mad_index = 0;
+ qp->sq.mad_queue_depth = MAD_QUEUE_DEPTH;
+ qp->sq.hdr_size = MAX_QP1_SQ_HDR_SIZE_V2 * MAD_QUEUE_DEPTH;
+ qp->sq.hdr_buf = dma_alloc_coherent(dev->ib_dev.dma_device,
+ qp->sq.hdr_size,
+ &qp->sq.hdr_dma,
+ GFP_KERNEL);
+ if (!qp->sq.hdr_buf) {
+ err = -ENOMEM;
+ ibdev_err(&dev->ib_dev, "Failed to create sq_hdr_buf");
+ goto err_wrid;
+ }
+ }
+
+ return 0;
+
+err_wrid:
+ xsc_vfree(*in);
+ *in = NULL;
+ kfree(qp->sq.prev_ce_wqe);
+ kfree(qp->sq.need_flush);
+ kfree(qp->sq.wr_opcode);
+ kfree(qp->sq.wqe_head);
+ kfree(qp->sq.w_list);
+ kfree(qp->sq.wrid);
+ kfree(qp->sq.wr_data);
+ kfree(qp->rq.wrid);
+
+err_buf:
+ xsc_buf_free(dev->xdev, &qp->buf);
+ return err;
+}
+
+static void destroy_qp_kernel(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)
+{
+ if (qp->sq.hdr_buf)
+ dma_free_coherent(dev->ib_dev.dma_device, qp->sq.hdr_size,
+ qp->sq.hdr_buf, qp->sq.hdr_dma);
+ kfree(qp->sq.prev_ce_wqe);
+ kfree(qp->sq.need_flush);
+ kfree(qp->sq.wr_opcode);
+ kfree(qp->sq.wqe_head);
+ kfree(qp->sq.w_list);
+ kfree(qp->sq.wrid);
+ kfree(qp->sq.wr_data);
+ kfree(qp->rq.wrid);
+ xsc_buf_free(dev->xdev, &qp->buf);
+}
+
+static u8 ib_to_xsc_qp_type(enum ib_qp_type qp_type, __u32 flags)
+{
+ if (qp_type == IB_QPT_RC) {
+ if (flags & XSC_QP_FLAG_WORC)
+ return XSC_QUEUE_TYPE_WORC;
+ else
+ return XSC_QUEUE_TYPE_RDMA_RC;
+ } else if ((qp_type == IB_QPT_GSI) || (qp_type == IB_QPT_SMI)) {
+ return XSC_QUEUE_TYPE_RDMA_MAD;
+ } else if (qp_type == IB_QPT_RAW_PACKET) {
+ if (flags & XSC_QP_FLAG_RAWPACKET_TSO)
+ return XSC_QUEUE_TYPE_RAW_TSO;
+ else if (flags & XSC_QP_FLAG_RAWPACKET_TX)
+ return XSC_QUEUE_TYPE_RAW_TX;
+ else if (flags & XSC_QP_FLAG_RAWPACKET_SNIFFER)
+ return XSC_QUEUE_TYPE_SNIFFER;
+ else
+ return XSC_QUEUE_TYPE_RAW;
+ } else {
+ return XSC_QUEUE_TYPE_INVALID;
+ }
+}
+
+static void get_cqs(enum ib_qp_type qp_type,
+ struct ib_cq *ib_send_cq, struct ib_cq *ib_recv_cq,
+ struct xsc_ib_cq **send_cq, struct xsc_ib_cq **recv_cq)
+{
+ switch (qp_type) {
+ case IB_QPT_SMI:
+ case IB_QPT_GSI:
+ case IB_QPT_RC:
+ case IB_QPT_UC:
+ case IB_QPT_UD:
+ case IB_QPT_RAW_PACKET:
+ *send_cq = ib_send_cq ? to_xcq(ib_send_cq) : NULL;
+ *recv_cq = ib_recv_cq ? to_xcq(ib_recv_cq) : NULL;
+ break;
+ default:
+ *send_cq = NULL;
+ *recv_cq = NULL;
+ break;
+ }
+}
+
+static void xsc_ib_lock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)
+ __acquires(&send_cq->lock) __acquires(&recv_cq->lock)
+{
+ if (send_cq) {
+ if (recv_cq) {
+ if (send_cq->xcq.cqn < recv_cq->xcq.cqn) {
+ spin_lock_irq(&send_cq->lock);
+ spin_lock_nested(&recv_cq->lock,
+ SINGLE_DEPTH_NESTING);
+ } else if (send_cq->xcq.cqn == recv_cq->xcq.cqn) {
+ spin_lock_irq(&send_cq->lock);
+ __acquire(&recv_cq->lock);
+ } else {
+ spin_lock_irq(&recv_cq->lock);
+ spin_lock_nested(&send_cq->lock,
+ SINGLE_DEPTH_NESTING);
+ }
+ } else {
+ spin_lock_irq(&send_cq->lock);
+ }
+ } else if (recv_cq) {
+ spin_lock_irq(&recv_cq->lock);
+ }
+}
+
+static void xsc_ib_unlock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)
+ __releases(&send_cq->lock) __releases(&recv_cq->lock)
+{
+ if (send_cq) {
+ if (recv_cq) {
+ if (send_cq->xcq.cqn < recv_cq->xcq.cqn) {
+ spin_unlock(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else if (send_cq->xcq.cqn == recv_cq->xcq.cqn) {
+ __release(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else {
+ spin_unlock(&send_cq->lock);
+ spin_unlock_irq(&recv_cq->lock);
+ }
+ } else {
+ spin_unlock_irq(&send_cq->lock);
+ }
+ } else if (recv_cq) {
+ spin_unlock_irq(&recv_cq->lock);
+ }
+}
+
+static int create_qp_common(struct xsc_ib_dev *dev, struct ib_pd *pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata, struct xsc_ib_qp *qp)
+{
+ struct xsc_create_qp_mbox_in *in = NULL;
+ struct xsc_ib_create_qp ucmd = {};
+ struct xsc_ib_cq *send_cq;
+ struct xsc_ib_cq *recv_cq;
+ struct xsc_srq_qp_node *qp_node = NULL;
+ int inlen = sizeof(*in);
+ int err;
+ char buf[256];
+ char *ptr = buf;
+ int ret = 0;
+
+ mutex_init(&qp->mutex);
+ spin_lock_init(&qp->sq.lock);
+ spin_lock_init(&qp->rq.lock);
+ spin_lock_init(&qp->lock);
+
+ if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
+ qp->sq_signal_bits = XSC_WQE_CTRL_CQ_UPDATE;
+
+ if (pd->uobject) {
+ if (ib_copy_from_udata(&ucmd, udata,
+ min(udata->inlen, sizeof(ucmd)))) {
+ ibdev_err(&dev->ib_dev, "failed to copy from udata\n");
+ return -EFAULT;
+ }
+ }
+
+ qp->has_rq = qp_has_rq(init_attr);
+
+ err = set_rq_size(dev, init_attr, qp->has_rq,
+ qp, pd->uobject ? &ucmd : NULL);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to set rq size %d\n", err);
+ return err;
+ }
+
+ if (pd->uobject) {
+ err = create_user_qp(dev, pd, qp, udata, &ucmd, &in, &inlen);
+ if (err)
+ ibdev_err(&dev->ib_dev, "failed to create user qp, err = %d\n", err);
+ } else {
+ err = create_kernel_qp(dev, init_attr, qp, &in, &inlen);
+ if (err)
+ ibdev_err(&dev->ib_dev, "failed to create kernel qp, err = %d\n", err);
+ else
+ qp->pa_lkey = to_xpd(pd)->pa_lkey;
+ }
+
+ if (err)
+ return err;
+
+ if (is_sqp(init_attr->qp_type))
+ qp->port = init_attr->port_num;
+
+ if (is_qp1(init_attr->qp_type))
+ in->req.input_qpn = cpu_to_be16(1);
+
+ in->req.pdn = cpu_to_be32(to_xpd(pd)->pdn);
+
+ if (qp->rq.ds_cnt)
+ in->req.log_rq_sz = ilog2(qp->rq.ds_cnt);
+
+ if (qp->sq.ds_cnt)
+ in->req.log_sq_sz = ilog2(qp->sq.ds_cnt);
+ else
+ in->req.log_sq_sz = ilog2(0x80);
+
+ if (init_attr->send_cq) {
+ qp->send_cq = init_attr->send_cq;
+ in->req.cqn_send = to_xcq(init_attr->send_cq)->xcq.cqn;
+ in->req.cqn_send = cpu_to_be16(in->req.cqn_send);
+ }
+
+ if (init_attr->recv_cq) {
+ qp->recv_cq = init_attr->recv_cq;
+ in->req.cqn_recv = to_xcq(init_attr->recv_cq)->xcq.cqn;
+ in->req.cqn_recv = cpu_to_be16(in->req.cqn_recv);
+ }
+
+ in->req.qp_type = ib_to_xsc_qp_type(init_attr->qp_type, pd->uobject ?
+ ucmd.flags : 0);
+
+ if (in->req.qp_type == XSC_QUEUE_TYPE_INVALID) {
+ ibdev_err(&dev->ib_dev, "invalid qp type:%d\n", init_attr->qp_type);
+ goto err_create;
+ }
+ in->req.glb_funcid = cpu_to_be16(dev->xdev->glb_func_id);
+ in->req.page_shift = PAGE_SHIFT;
+
+ qp->xqp.qp_type_internal = in->req.qp_type;
+
+ if (!pd->uobject && init_attr->srq) {
+ qp_node = kvzalloc_obj(*qp_node, GFP_KERNEL);
+ if (!qp_node) {
+ err = -ENOMEM;
+ goto err_create;
+ }
+ }
+
+ /* the qp becomes visible to the event handler once
+ * xsc_core_create_qp() inserts it into the tree, so the
+ * callback must be in place before that
+ */
+ qp->xqp.event = xsc_ib_qp_event;
+
+ err = xsc_core_create_qp(dev->xdev, &qp->xqp, in, inlen);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create qp failed, err=%d\n", err);
+ goto err_create;
+ }
+
+ qp->doorbell_qpn = qp->xqp.qpn;
+
+ qp->xqp.qp_type = init_attr->qp_type;
+ ret += snprintf(ptr + ret, 256 - ret, "pdn=%d,", to_xpd(pd)->pdn);
+ ret += snprintf(ptr + ret, 256 - ret, "log_rq_sz=%d,", in->req.log_rq_sz);
+ ret += snprintf(ptr + ret, 256 - ret, "log_sq_sz=%d,", in->req.log_sq_sz);
+ ret += snprintf(ptr + ret, 256 - ret, "scqn=%d,", to_xcq(init_attr->send_cq)->xcq.cqn);
+ ret += snprintf(ptr + ret, 256 - ret, "rcqn=%d", to_xcq(init_attr->recv_cq)->xcq.cqn);
+
+ get_cqs(init_attr->qp_type, init_attr->send_cq, init_attr->recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ if (send_cq)
+ list_add_tail(&qp->cq_send_list, &send_cq->send_qp_list);
+ if (recv_cq)
+ list_add_tail(&qp->cq_recv_list, &recv_cq->recv_qp_list);
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ if (qp_node) {
+ struct xsc_ib_srq *xsrq = to_xsrq(init_attr->srq);
+ unsigned long irq_flags;
+
+ qp_node->ibqp = &qp->ibqp;
+ qp_node->qp_id = qp->xqp.qpn;
+ ibdev_dbg(&dev->ib_dev, "qpn:%u add into srq_qp_list\n", qp_node->qp_id);
+ spin_lock_irqsave(&xsrq->lock, irq_flags);
+ list_add_tail(&qp_node->entry, &xsrq->xsrq.srq_qp_list);
+ spin_unlock_irqrestore(&xsrq->lock, irq_flags);
+ }
+
+ qp->xqp.qp_protocol_mode = dev->xdev->rdma_proto_mode;
+
+ ibdev_dbg(&dev->ib_dev, "succeeded to create qp:%d, %s\n", qp->xqp.qpn, buf);
+
+ xsc_vfree(in);
+
+ return 0;
+
+err_create:
+ kvfree(qp_node);
+ if (qp->create_type == XSC_QP_USER)
+ destroy_qp_user(pd, qp);
+ else if (qp->create_type == XSC_QP_KERNEL)
+ destroy_qp_kernel(dev, qp);
+
+ xsc_vfree(in);
+ return err;
+}
+
+static struct xsc_ib_pd *get_pd(struct xsc_ib_qp *qp)
+{
+ return to_xpd(qp->ibqp.pd);
+}
+
+static void destroy_qp_common(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)
+{
+ struct xsc_ib_cq *send_cq = NULL;
+ struct xsc_ib_cq *recv_cq = NULL;
+ struct xsc_modify_qp_mbox_in in = {};
+ int err;
+
+ if (qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW ||
+ qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TSO ||
+ qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TX ||
+ qp->state != IB_QPS_RESET)
+ if (xsc_core_qp_modify(dev->xdev, to_xsc_state(qp->state),
+ XSC_QP_STATE_RST, &in, &qp->xqp))
+ ibdev_warn(&dev->ib_dev, "modify QP %06x to RESET failed\n", qp->xqp.qpn);
+
+ get_cqs(qp->xqp.qp_type, qp->ibqp.send_cq, qp->ibqp.recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ if (send_cq)
+ list_del(&qp->cq_send_list);
+
+ if (recv_cq)
+ list_del(&qp->cq_recv_list);
+
+ if (qp->create_type == XSC_QP_KERNEL) {
+ __xsc_ib_cq_clean(recv_cq, qp->xqp.qpn);
+ if (send_cq != recv_cq)
+ __xsc_ib_cq_clean(send_cq, qp->xqp.qpn);
+ }
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ err = xsc_core_destroy_qp(dev->xdev, &qp->xqp);
+ if (err)
+ ibdev_warn(&dev->ib_dev, "failed to destroy QP 0x%x\n", qp->xqp.qpn);
+
+ if (qp->create_type == XSC_QP_KERNEL)
+ destroy_qp_kernel(dev, qp);
+ else if (qp->create_type == XSC_QP_USER)
+ destroy_qp_user(&get_pd(qp)->ibpd, qp);
+}
+
+static const char *ib_qp_type_str(enum ib_qp_type type)
+{
+ switch (type) {
+ case IB_QPT_SMI:
+ return "IB_QPT_SMI";
+ case IB_QPT_GSI:
+ return "IB_QPT_GSI";
+ case IB_QPT_RC:
+ return "IB_QPT_RC";
+ case IB_QPT_UC:
+ return "IB_QPT_UC";
+ case IB_QPT_UD:
+ return "IB_QPT_UD";
+ case IB_QPT_RAW_IPV6:
+ return "IB_QPT_RAW_IPV6";
+ case IB_QPT_RAW_ETHERTYPE:
+ return "IB_QPT_RAW_ETHERTYPE";
+ case IB_QPT_RAW_PACKET:
+ return "IB_QPT_RAW_PACKET";
+ case IB_QPT_MAX:
+ default:
+ return "Invalid QP type";
+ }
+}
+
+int xsc_ib_create_qp(struct ib_qp *ibqp,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev;
+ struct xsc_ib_qp *qp;
+ struct ib_pd *pd = ibqp->pd;
+ int err;
+
+ if (pd) {
+ dev = to_xdev(pd->device);
+ } else {
+ pr_warn("%s: no PD for transport %s\n", __func__,
+ ib_qp_type_str(init_attr->qp_type));
+ return -EINVAL;
+ }
+
+ if (init_attr->qp_type != IB_QPT_RAW_PACKET) {
+ if (!is_support_rdma(dev->xdev) ||
+ (is_qp1(init_attr->qp_type) && !is_support_rdma_cm(dev->xdev))) {
+ return -EPROTONOSUPPORT;
+ }
+ }
+
+ qp = to_xqp(ibqp);
+
+ qp->xqp.qp_type = init_attr->qp_type;
+
+ switch (init_attr->qp_type) {
+ case IB_QPT_RC:
+ case IB_QPT_SMI:
+ case IB_QPT_GSI:
+ case IB_QPT_RAW_PACKET:
+ err = create_qp_common(dev, pd, init_attr, udata, qp);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create_qp_common failed\n");
+ return err;
+ }
+
+ if (is_qp0(init_attr->qp_type)) {
+ qp->ibqp.qp_num = 0;
+ } else if (is_qp1(init_attr->qp_type)) {
+ qp->ibqp.qp_num = 1;
+ dev->xdev->gsi_qpn = qp->xqp.qpn;
+ } else {
+ qp->ibqp.qp_num = qp->xqp.qpn;
+ }
+
+ break;
+
+ case IB_QPT_RAW_IPV6:
+ case IB_QPT_RAW_ETHERTYPE:
+ case IB_QPT_MAX:
+ default:
+ ibdev_err(&dev->ib_dev, "unsupported qp type %d\n",
+ init_attr->qp_type);
+ /* Don't support raw QPs */
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
+ struct xsc_srq_qp_node *qp_node = NULL, *tmp_qp_node = NULL;
+ unsigned long flags;
+ struct xsc_ib_srq *xsrq;
+
+ if (qp->qp_type == IB_QPT_RC) {
+ if (xqp->recv_cq) {
+ spin_lock_irq(&to_xcq(xqp->recv_cq)->err_state_qp_lock);
+ list_for_each_entry_safe(err_rq_node, tmp,
+ &to_xcq(xqp->recv_cq)->err_state_qp_list, entry) {
+ if (err_rq_node->qp_id == xqp->xqp.qpn && !err_rq_node->is_sq) {
+ list_del(&err_rq_node->entry);
+ kfree(err_rq_node);
+ }
+ }
+ spin_unlock_irq(&to_xcq(xqp->recv_cq)->err_state_qp_lock);
+ }
+
+ if (xqp->send_cq) {
+ spin_lock_irq(&to_xcq(xqp->send_cq)->err_state_qp_lock);
+ list_for_each_entry_safe(err_sq_node, tmp,
+ &to_xcq(xqp->send_cq)->err_state_qp_list, entry) {
+ if (err_sq_node->qp_id == xqp->xqp.qpn && err_sq_node->is_sq) {
+ list_del(&err_sq_node->entry);
+ kfree(err_sq_node);
+ }
+ }
+ spin_unlock_irq(&to_xcq(xqp->send_cq)->err_state_qp_lock);
+ }
+ }
+
+ if (xqp->create_type == XSC_QP_KERNEL && qp->srq) {
+ if (!to_xsrq(qp->srq) || list_empty(&to_xsrq(qp->srq)->xsrq.srq_qp_list))
+ ibdev_warn(&dev->ib_dev, "srq_qp_list is empty!\n");
+
+ xsrq = to_xsrq(qp->srq);
+ spin_lock_irqsave(&xsrq->lock, flags);
+ xsc_restore_cache_srq_wrs(xsrq, xqp);
+ list_for_each_entry_safe(qp_node, tmp_qp_node,
+ &to_xsrq(qp->srq)->xsrq.srq_qp_list, entry) {
+ if (qp_node->qp_id == xqp->xqp.qpn) {
+ to_xsrq(qp->srq)->qp_num--;
+ ibdev_dbg(&dev->ib_dev, "srq_qp_list del node qpn:%u qp_num:%d\n",
+ qp_node->qp_id, to_xsrq(qp->srq)->qp_num);
+ list_del_init(&qp_node->entry);
+ kvfree(qp_node);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&xsrq->lock, flags);
+ }
+
+ destroy_qp_common(dev, xqp);
+
+ return 0;
+}
+
+static inline u16 xsc_calc_udp_sport(u32 lqpn, u32 rqpn)
+{
+ unsigned char *p;
+ u8 ports[2];
+ u16 sport;
+ u64 tqpn;
+
+ tqpn = ((u64)(lqpn & 0xffffff)) * ((u64)(rqpn & 0xffffff));
+ p = (unsigned char *)&tqpn;
+ ports[0] = p[0] ^ p[2] ^ p[4];
+ ports[1] = p[1] ^ p[3] ^ p[5];
+ sport = *((u16 *)ports) | 0xC000;
+
+ return sport;
+}
+
+static inline void xsc_path_set_udp_sport(struct xsc_qp_path *path,
+ const struct rdma_ah_attr *ah,
+ u32 lqpn, u32 rqpn)
+{
+ if ((ah->grh.flow_label & UDP_SPORT_MASK) != 0) {
+ if ((ah->grh.flow_label & UDP_SPORT_MASK_EN) == 0)
+ path->sport = cpu_to_be16(rdma_flow_label_to_udp_sport(ah->grh.flow_label));
+ else
+ path->sport = cpu_to_be16((ah->grh.flow_label & UDP_SPORT_MASK) >>
+ UDP_SPORT_OFFSET);
+ } else {
+ path->sport = cpu_to_be16(xsc_calc_udp_sport(lqpn, rqpn));
+ }
+}
+
+static int xsc_set_path(struct xsc_ib_dev *dev, const struct rdma_ah_attr *ah,
+ struct xsc_qp_path *path, u8 port, int attr_mask,
+ u32 path_flags, const struct ib_qp_attr *attr, struct xsc_ib_qp *qp)
+{
+ struct ib_global_route *grh = rdma_ah_retrieve_grh((struct rdma_ah_attr *)ah);
+ union ib_gid *dgid = &grh->dgid;
+ const struct ib_gid_attr *sgid_attr = grh->sgid_attr;
+ union ib_gid *sgid = &((struct ib_gid_attr *)sgid_attr)->gid;
+ union {
+ struct sockaddr _sockaddr;
+ struct sockaddr_in _sockaddr_in;
+ struct sockaddr_in6 _sockaddr_in6;
+ } sgid_addr, dgid_addr;
+ int force_pcp, force_dscp;
+ char buf[256] = {0};
+ char *ptr = buf;
+ struct net_device *ndev;
+ u16 vlan_id = 0xffff;
+ int err;
+ int ret = 0;
+
+ if (ah->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ if (!(rdma_ah_get_ah_flags(ah) & IB_AH_GRH))
+ return -EINVAL;
+
+ if (qp->ibqp.qp_type == IB_QPT_RC ||
+ qp->ibqp.qp_type == IB_QPT_UC) {
+ if (((ah->grh.flow_label & UDP_SPORT_MASK_EN) != 0) &&
+ qp->xqp.qp_protocol_mode == RDMA_PROTO_VEROCE) {
+ ibdev_err(&dev->ib_dev,
+ "qp type is RDMA_PROTO_VEROCE, using flow_label to set the UDP sport %d is not allowed.",
+ (ah->grh.flow_label & UDP_SPORT_MASK) >>
+ UDP_SPORT_OFFSET);
+ return -EINVAL;
+ }
+ xsc_path_set_udp_sport(path, ah, qp->ibqp.qp_num, attr->dest_qp_num);
+ }
+
+ if (sgid_attr->gid_type != IB_GID_TYPE_ROCE_UDP_ENCAP) {
+ ibdev_err(&dev->ib_dev, "gid type not ROCEv2\n");
+ return -EINVAL;
+ }
+
+ force_dscp = dev->xdev->priv.force_dscp;
+ if (force_dscp == XSC_DSCP_PCP_UNSET)
+ path->ecn_dscp = (grh->traffic_class >> 2) & 0x3f;
+ else
+ path->ecn_dscp = force_dscp;
+ path->hop_limit = grh->hop_limit;
+
+ rdma_gid2ip((struct sockaddr *)&sgid_addr, sgid);
+ rdma_gid2ip((struct sockaddr *)&dgid_addr, dgid);
+
+ if (sgid_addr._sockaddr.sa_family == AF_INET &&
+ dgid_addr._sockaddr.sa_family == AF_INET) {
+ memcpy(path->sip, &sgid_addr._sockaddr_in.sin_addr.s_addr,
+ sizeof(struct in_addr));
+ memcpy(path->dip, &dgid_addr._sockaddr_in.sin_addr.s_addr,
+ sizeof(struct in_addr));
+ path->af_type = AF_INET;
+ ret += snprintf(ptr + ret, 256 - ret, "sip=%#x,",
+ be32_to_cpu(path->sip[0]));
+ ret += snprintf(ptr + ret, 256 - ret, "dip=%#x,",
+ be32_to_cpu(path->dip[0]));
+ } else if (sgid_addr._sockaddr.sa_family == AF_INET6 &&
+ dgid_addr._sockaddr.sa_family == AF_INET6) {
+ memcpy(path->sip, &sgid_addr._sockaddr_in6.sin6_addr.s6_addr,
+ sizeof(path->sip));
+ memcpy(path->dip, &dgid_addr._sockaddr_in6.sin6_addr.s6_addr,
+ sizeof(path->dip));
+ path->af_type = AF_INET6;
+ ret += snprintf(ptr + ret, 256 - ret, "sip=%08x%08x%08x%08x,",
+ be32_to_cpu(path->sip[0]), be32_to_cpu(path->sip[1]),
+ be32_to_cpu(path->sip[2]), be32_to_cpu(path->sip[3]));
+ ret += snprintf(ptr + ret, 256 - ret, "dip=%08x%08x%08x%08x,",
+ be32_to_cpu(path->dip[0]), be32_to_cpu(path->dip[1]),
+ be32_to_cpu(path->dip[2]), be32_to_cpu(path->dip[3]));
+ } else {
+ return -EINVAL;
+ }
+
+ ndev = ib_device_get_netdev(&dev->ib_dev, port);
+ if (!ndev)
+ return -ENODEV;
+ ether_addr_copy(path->smac, ndev->dev_addr);
+ dev_put(ndev);
+
+ memcpy(path->dmac, ah->roce.dmac, sizeof(ah->roce.dmac));
+ ret += snprintf(ptr + ret, 256 - ret, "smac=%02x%02x%02x%02x%02x%02x,",
+ path->smac[0], path->smac[1], path->smac[2],
+ path->smac[3], path->smac[4], path->smac[5]);
+ ret += snprintf(ptr + ret, 256 - ret, "dmac=%02x%02x%02x%02x%02x%02x",
+ path->dmac[0], path->dmac[1], path->dmac[2],
+ path->dmac[3], path->dmac[4], path->dmac[5]);
+ if (is_zero_ether_addr(path->dmac))
+ ibdev_err(&dev->ib_dev, "path dmac is zero!\n");
+ else
+ ibdev_dbg(&dev->ib_dev, "ib path info:%s\n", buf);
+
+ err = rdma_read_gid_l2_fields(sgid_attr, &vlan_id, NULL);
+ if (err)
+ return err;
+
+ if (vlan_id != 0xffff) {
+ path->vlan_valid = 1;
+ path->vlan_id = cpu_to_be16(vlan_id);
+
+ force_pcp = dev->xdev->priv.force_pcp;
+ if (force_pcp == XSC_DSCP_PCP_UNSET)
+ path->dci_cfi_prio_sl = (ah->sl & 0x7);
+ else
+ path->dci_cfi_prio_sl = force_pcp;
+ } else {
+ path->vlan_valid = 0;
+ }
+ }
+ ibdev_dbg(&dev->ib_dev, "path dscp %d pcp %d\n", path->ecn_dscp, path->dci_cfi_prio_sl);
+ return 0;
+}
+
int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
enum xsc_qp_state cur_state, enum xsc_qp_state state,
bool *triggered)
@@ -136,21 +1159,280 @@ int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
return 0;
}
-enum xsc_ib_qp_ex_attr_mask {
- XSC_IB_QP_FLUSH = 1 << 21,
-};
+static inline void xsc_set_qp_access_flag(struct xsc_modify_qp_mbox_in *in,
+ struct xsc_qp_context *context, int access_flags)
+{
+ in->hdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V1);
+ context->qp_access_flags = 0;
+
+ if (access_flags & IB_ACCESS_REMOTE_READ)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_READ;
+ if (access_flags & IB_ACCESS_REMOTE_WRITE)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_WRITE;
+ if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_ATOMIC;
-static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq, struct xsc_ib_cq *cq)
+ context->qp_access_flags = cpu_to_be32(context->qp_access_flags);
+}
+
+static int __xsc_ib_modify_qp(struct ib_qp *ibqp,
+ const struct ib_qp_attr *attr, int attr_mask,
+ enum ib_qp_state cur_state, enum ib_qp_state new_state,
+ const struct xsc_ib_modify_qp *ucmd,
+ struct ib_udata *udata)
{
- unsigned int cur;
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_cq *send_cq, *recv_cq;
+ struct xsc_qp_context *context;
+ struct xsc_modify_qp_mbox_in *in;
+ struct xsc_qp_path path = {};
+ int err;
+ char buf[256] = {0};
+ char *ptr = buf;
+ int ret = 0;
+ struct xsc_core_device *xdev = dev->xdev;
- cur = wq->head - wq->tail;
- if (likely(cur + nreq < wq->max_post))
- return 0;
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ context = &qp->ctx;
+
+ if (attr_mask & IB_QP_PATH_MTU) {
+ if (attr->path_mtu != IB_MTU_1024 &&
+ attr->path_mtu != IB_MTU_4096 &&
+ attr->path_mtu != IB_MTU_8192) {
+ ibdev_warn(&dev->ib_dev, "invalid mtu %d\n", attr->path_mtu);
+ }
+
+ if (attr->path_mtu < IB_MTU_4096)
+ context->mtu_mode = 0;
+ else if (attr->path_mtu < IB_MTU_8192)
+ context->mtu_mode = 1;
+ else
+ context->mtu_mode = 2;
+ ret = snprintf(ptr, 256, "path_mtu=%d,", attr->path_mtu);
+ }
+
+ if (attr_mask & IB_QP_DEST_QPN) {
+ context->remote_qpn = cpu_to_be32(attr->dest_qp_num);
+ ret += snprintf(ptr + ret, 256 - ret, "dest_qp_num=%d,", attr->dest_qp_num);
+ }
+
+ if (attr_mask & IB_QP_AV) {
+ err = xsc_set_path(dev, &attr->ah_attr, &path,
+ attr_mask & IB_QP_PORT ? attr->port_num : qp->port,
+ attr_mask, 0, attr, qp);
+ if (err)
+ goto out;
+
+ context->src_udp_port = path.sport;
+ context->dscp = path.ecn_dscp;
+ context->hop_limit = path.hop_limit;
+ context->ip_type = (path.af_type == AF_INET ? 0 : 1);
+ context->ip_type = cpu_to_be16(context->ip_type);
+ memcpy(context->dip, path.dip, sizeof(context->dip));
+ memcpy(context->sip, path.sip, sizeof(context->sip));
+ memcpy(context->dmac, path.dmac, sizeof(path.dmac));
+ memcpy(context->smac, path.smac, sizeof(path.smac));
+
+ context->vlan_valid = path.vlan_valid;
+ context->dci_cfi_prio_sl = path.dci_cfi_prio_sl;
+ context->vlan_id = path.vlan_id;
+ }
+
+ if (attr_mask & IB_QP_RNR_RETRY) {
+ context->rnr_retry = attr->rnr_retry;
+ ret += snprintf(ptr + ret, 256 - ret, "rnr_retry=%d,", attr->rnr_retry);
+ }
+
+ if (attr_mask & IB_QP_RETRY_CNT) {
+ context->retry_cnt = attr->retry_cnt;
+ ret += snprintf(ptr + ret, 256 - ret, "retry_cnt=%d,", attr->retry_cnt);
+ }
+
+ if (attr_mask & IB_QP_MAX_QP_RD_ATOMIC && attr->max_rd_atomic)
+ context->max_sra = min_t(u32, attr->max_rd_atomic, dev->xdev->caps.max_ra_req_qp);
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC && attr->max_dest_rd_atomic)
+ context->max_rra = attr->max_dest_rd_atomic;
+
+ if (attr_mask & IB_QP_SQ_PSN) {
+ context->next_send_psn = cpu_to_be32(attr->sq_psn);
+ ret += snprintf(ptr + ret, 256 - ret, "sq_psn=%#x,", attr->sq_psn);
+ }
+
+ if (attr_mask & IB_QP_RQ_PSN) {
+ context->next_recv_psn = cpu_to_be32(attr->rq_psn);
+ ret += snprintf(ptr + ret, 256 - ret, "rq_psn=%#x,", attr->rq_psn);
+ }
+
+ if (attr_mask & IB_QP_ACCESS_FLAGS) {
+ xsc_set_qp_access_flag(in, context, attr->qp_access_flags);
+ ret += snprintf(ptr + ret, 256 - ret, "qp_access_flags=%#x,\n",
+ attr->qp_access_flags);
+ }
+
+ if (udata && udata->inlen && attr_mask & IB_QP_STATE && new_state == IB_QPS_RTR) {
+ if (!ucmd->profile &&
+ xdev->rdma_proto_mode == RDMA_PROTO_VEROCE) {
+ context->swith_qp_type = 1;
+ context->qp_access_flags |= cpu_to_be32(QP_ACCESS_MODIFY_QP_TYPE);
+ qp->xqp.qp_protocol_mode =
+ ucmd->profile ? RDMA_PROTO_VEROCE : RDMA_PROTO_ROCEV2;
+ in->hdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V2);
+ ibdev_dbg(&dev->ib_dev,
+ "local is VEROCE,remote is RoCEv2, switch qp_type from VEROCE to RoCEv2!\n");
+ }
+ }
+
+ if (attr_mask & IB_QP_STATE && new_state == IB_QPS_RESET)
+ qp->xqp.qp_protocol_mode = xdev->rdma_proto_mode;
+
+ memcpy(&in->ctx, context, sizeof(*context));
+ err = xsc_core_qp_modify(xdev, to_xsc_state(cur_state),
+ to_xsc_state(new_state), in, &qp->xqp);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to modify qp[%d] from %s to %s\n",
+ qp->xqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state));
+ goto out;
+ }
+
+ qp->state = new_state;
+ ibdev_dbg(&dev->ib_dev, "succeeded to modify qp[%d] from %s to %s with attr_mask=%#x, %s\n",
+ qp->xqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state),
+ attr_mask, buf);
+
+ if (qp->state == IB_QPS_INIT) {
+ qp->post_recv_invalid = false;
+ qp->post_send_invalid = false;
+ }
+
+ if (attr_mask & IB_QP_ACCESS_FLAGS)
+ qp->atomic_rd_en = attr->qp_access_flags;
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
+ qp->resp_depth = attr->max_dest_rd_atomic;
+ if (attr_mask & IB_QP_PORT)
+ qp->port = attr->port_num;
+ if (attr_mask & IB_QP_ALT_PATH)
+ qp->alt_port = attr->alt_port_num;
+
+ /*
+ * If we moved a kernel QP to RESET, clean up all old CQ
+ * entries and reinitialize the QP.
+ */
+ if (new_state == IB_QPS_RESET && !ibqp->uobject) {
+ get_cqs(qp->xqp.qp_type, qp->ibqp.send_cq, qp->ibqp.recv_cq, &send_cq, &recv_cq);
+ xsc_ib_cq_clean(recv_cq, qp->xqp.qpn);
+ if (send_cq != recv_cq)
+ xsc_ib_cq_clean(send_cq, qp->xqp.qpn);
+
+ qp->rq.head = 0;
+ qp->rq.tail = 0;
+ qp->sq.head = 0;
+ qp->sq.tail = 0;
+ qp->sq.cur_post = 0;
+ qp->sq.last_poll = 0;
+ qp->sq.last_comp_ce_wqe = -1;
+ qp->sq.last_post_ce_wqe = -1;
+ qp->has_trig_cq_comp_handle = false;
+ }
+
+out:
+ kfree(in);
+ return err;
+}
+
+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ enum ib_qp_state cur_state, new_state;
+ int err = -EINVAL;
+ int wr_num;
+ struct xsc_ib_srq *srq;
+ unsigned long flags;
+ struct xsc_ib_cq *send_cq;
+ struct xsc_ib_cq *recv_cq;
+ struct xsc_ib_modify_qp ucmd = {};
+
+ if (udata && udata->inlen) {
+ if (udata->inlen < offsetofend(typeof(ucmd), profile))
+ return -EINVAL;
+
+ if (ib_copy_from_udata(&ucmd, udata,
+ min(udata->inlen, sizeof(ucmd))))
+ return -EFAULT;
+ }
+
+ mutex_lock(&qp->mutex);
+
+ cur_state = attr_mask & IB_QP_CUR_STATE ? attr->cur_qp_state : qp->state;
+ new_state = attr_mask & IB_QP_STATE ? attr->qp_state : cur_state;
+
+ if ((attr_mask & IB_QP_PORT) &&
+ (attr->port_num == 0 || attr->port_num > dev->xdev->caps.num_ports)) {
+ ibdev_err(&dev->ib_dev, "error port num\n");
+ goto out;
+ }
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC &&
+ attr->max_dest_rd_atomic > dev->xdev->caps.max_ra_res_qp) {
+ ibdev_err(&dev->ib_dev, "dest rd atomic:%u exeeded", attr->max_dest_rd_atomic);
+ goto out;
+ }
+
+ if (cur_state == new_state && cur_state == IB_QPS_RESET) {
+ err = 0;
+ goto out;
+ }
+
+ if (!ib_modify_qp_is_ok(cur_state, new_state, ibqp->qp_type, attr_mask)) {
+ ibdev_err(&dev->ib_dev, "ib_modify_qp not ok, type=%d mask=0x%x state from%d to%d\n",
+ ibqp->qp_type, attr_mask, cur_state, new_state);
+ goto out;
+ }
+
+ err = __xsc_ib_modify_qp(ibqp, attr, attr_mask, cur_state, new_state, &ucmd, udata);
+
+ if (!err && (attr_mask & IB_QP_STATE) && !udata) {
+ bool trigger;
+
+ get_cqs(qp->xqp.qp_type, qp->send_cq, qp->recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ err = xsc_ib_err_state_qp(dev, qp, to_xsc_state(cur_state),
+ to_xsc_state(new_state), &trigger);
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ if (!err && trigger)
+ xsc_ib_trigger_cq_comp_handler(qp->send_cq, qp->recv_cq);
+ }
+
+ if (!err && !udata && (attr_mask & IB_QP_STATE) && ibqp->srq &&
+ cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
+ srq = to_xsrq(ibqp->srq);
+ spin_lock_irqsave(&srq->lock, flags);
+ srq->qp_num++;
+ wr_num = XSC_SRQ_MIN_WR_NUM;
+ ibdev_dbg(&dev->ib_dev,
+ "SRQ %d: qp %d to INIT, post %d srq wrs, qp_num %d\n",
+ srq->xsrq.srqn, qp->xqp.qpn, wr_num, srq->qp_num);
+ xsc_post_cache_wrs(srq, qp, wr_num, true);
+ spin_unlock_irqrestore(&srq->lock, flags);
+ }
+
+out:
+ mutex_unlock(&qp->mutex);
+ return err;
+}
+
+static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq)
+{
+ unsigned int cur;
- spin_lock(&cq->lock);
cur = wq->head - wq->tail;
- spin_unlock(&cq->lock);
return cur + nreq >= wq->max_post;
}
@@ -183,7 +1465,7 @@ int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
idx = qp->rq.head & (qp->rq.wqe_cnt - 1);
for (nreq = 0; wr; ++nreq, wr = wr->next) {
- if (unlikely(xsc_wq_overflow(&qp->rq, nreq, to_xcq(qp->ibqp.recv_cq)))) {
+ if (unlikely(xsc_wq_overflow(&qp->rq, nreq))) {
ibdev_err(&dev->ib_dev, "recv work queue overflow\n");
err = -ENOMEM;
*bad_wr = wr;
@@ -248,10 +1530,104 @@ int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
return err;
}
+static inline enum ib_qp_state to_ib_qp_state(enum xsc_qp_state xsc_state)
+{
+ switch (xsc_state) {
+ case XSC_QP_STATE_RST: return IB_QPS_RESET;
+ case XSC_QP_STATE_INIT: return IB_QPS_INIT;
+ case XSC_QP_STATE_RTR: return IB_QPS_RTR;
+ case XSC_QP_STATE_RTS: return IB_QPS_RTS;
+ case XSC_QP_STATE_SQ_DRAINING:
+ case XSC_QP_STATE_SQD: return IB_QPS_SQD;
+ case XSC_QP_STATE_SQER: return IB_QPS_SQE;
+ case XSC_QP_STATE_ERR: return IB_QPS_ERR;
+ default: return -1;
+ }
+}
+
+static inline enum ib_mig_state to_ib_mig_state(int xsc_mig_state)
+{
+ switch (xsc_mig_state) {
+ case XSC_QP_PM_ARMED: return IB_MIG_ARMED;
+ case XSC_QP_PM_REARM: return IB_MIG_REARM;
+ case XSC_QP_PM_MIGRATED: return IB_MIG_MIGRATED;
+ default: return -1;
+ }
+}
+
+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_query_qp_mbox_out *outb;
+ struct xsc_qp_context *context;
+ int err = 0;
+
+ mutex_lock(&qp->mutex);
+
+ outb = kzalloc_obj(*outb, GFP_KERNEL);
+ if (!outb) {
+ err = -ENOMEM;
+ goto out;
+ }
+ context = &outb->ctx;
+ err = xsc_core_qp_query(dev->xdev, &qp->xqp, outb, sizeof(*outb));
+ if (err)
+ goto out_free;
+
+ if (qp->xqp.err_occurred) {
+ qp->state = IB_QPS_ERR;
+ qp->xqp.err_occurred = 0;
+ }
+ qp_attr->qp_state = qp->state;
+ qp_attr->path_mtu = context->mtu_mode ? IB_MTU_4096 : IB_MTU_1024;
+ qp_attr->rq_psn = be32_to_cpu(context->next_recv_psn) & 0xffffff;
+ qp_attr->sq_psn = be32_to_cpu(context->next_send_psn) & 0xffffff;
+ qp_attr->dest_qp_num = be32_to_cpu(context->remote_qpn) & 0xffffff;
+ qp_attr->sq_draining = 0;
+ qp_attr->retry_cnt = context->retry_cnt;
+ qp_attr->rnr_retry = context->rnr_retry;
+ qp_attr->cur_qp_state = qp_attr->qp_state;
+ qp_attr->cap.max_recv_wr = qp->rq.wqe_cnt;
+ qp_attr->cap.max_recv_sge = qp->rq.max_gs;
+ qp_attr->max_rd_atomic = context->max_sra;
+ qp_attr->max_dest_rd_atomic = context->max_rra;
+ qp_attr->port_num = 1;
+
+ if (!ibqp->uobject) {
+ qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
+ qp_attr->cap.max_send_sge = qp->sq.max_gs;
+ } else {
+ qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
+ qp_attr->cap.max_send_sge = qp->sq.max_gs;
+ }
+
+ /* We don't support inline sends for kernel QPs (yet), and we
+ * don't know what userspace's value should be.
+ */
+ qp_attr->cap.max_inline_data = 0;
+
+ qp_init_attr->cap = qp_attr->cap;
+
+ qp_init_attr->create_flags = 0;
+ if (qp->flags & XSC_IB_QP_BLOCK_MULTICAST_LOOPBACK)
+ qp_init_attr->create_flags |= IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK;
+
+ qp_init_attr->sq_sig_type = qp->sq_signal_bits & XSC_WQE_CTRL_CQ_UPDATE ?
+ IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR;
+
+out_free:
+ kfree(outb);
+
+out:
+ mutex_unlock(&qp->mutex);
+ return err;
+}
+
struct xsc_ib_drain_cqe {
struct ib_cqe cqe;
struct completion done;
int qpn;
};
-
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
index d7b0ff176..126fcf789 100644
--- a/drivers/infiniband/hw/xsc/xsc_ib.h
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -63,9 +63,9 @@ struct xsc_ib_wq {
/* serialize post to the work queue
*/
spinlock_t lock;
- int wqe_cnt;
- int ds_cnt;
- int max_post;
+ u32 wqe_cnt;
+ u32 ds_cnt;
+ u32 max_post;
int max_gs;
int offset;
int wqe_shift;
@@ -104,7 +104,7 @@ struct xsc_ib_qp {
struct xsc_ib_wq sq;
struct ib_umem *umem;
- int buf_size;
+ u64 buf_size;
/* serialize qp state modifications
*/
diff --git a/include/uapi/rdma/xsc-abi.h b/include/uapi/rdma/xsc-abi.h
index 7e19b23a3..f75a24f2a 100644
--- a/include/uapi/rdma/xsc-abi.h
+++ b/include/uapi/rdma/xsc-abi.h
@@ -93,11 +93,6 @@ struct xsc_ib_create_qp {
__aligned_u64 dmabuf_sz;
};
-struct xsc_ib_create_qp_resp {
- __u32 uuar_index;
- __u32 reserved;
-};
-
struct xsc_ib_create_srq {
__u64 buf_addr;
__u64 db_addr;
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 13/15] RDMA/xsc: add QP verbs
2026-11-04 15:57 ` [PATCH v1 13/15] RDMA/xsc: add QP verbs Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 12 potential issue(s) to consider:
- [Critical] RDMA/xsc: heap out-of-bounds write in create_user_qp() when hw_npages is 0
- [High] RDMA/xsc: NULL CQ dereference in create_qp_common() debug formatting
- [High] RDMA/xsc: 32-bit overflow in create_user_qp() sq.offset calculation
- [High] RDMA/xsc: incomplete and racy err_state_qp_list cleanup in xsc_ib_destroy_qp()
- [High] RDMA/xsc: leaked error nodes and unlocked WQ reset in xsc_ib_modify_qp()
- [High] RDMA/xsc: SRQ WR leak and qp_num corruption across QP state transitions
- [High] RDMA/xsc: uninitialized kernel memory leak to user in xsc_ib_query_qp()
- [Medium] RDMA/xsc: insufficient udata->inlen checks in create_user_qp()
- [Medium] RDMA/xsc: uninitialized stack variable offset logged in create_user_qp()
- [Medium] RDMA/xsc: integer truncation in set_rq_size() before max_wqes check
- [Medium] RDMA/xsc: IB_MTU_8192 misreported as IB_MTU_4096 in xsc_ib_query_qp()
- [Medium] RDMA/xsc: premature ctx mutation and wrong page_shift in QP verbs
--
commit f535ae7002b2233def13ec88aba5f48f2448d9b9
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add QP verbs
This commit implements queue pair creation, modification, query, and
destruction callbacks for the xsc IB driver, supporting RC, MAD QP0/QP1, and
raw packet queue pairs for both kernel and user-space callers.
> diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
> index a41d880cf03d9..0dfb102949070 100644
> --- a/drivers/infiniband/hw/xsc/qp.c
> +++ b/drivers/infiniband/hw/xsc/qp.c
[ ... ]
> +static int set_rq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *init_attr,
> + int has_rq, struct xsc_ib_qp *qp, struct xsc_ib_create_qp *ucmd)
> +{
> + u32 wqe_cnt = init_attr->cap.max_recv_wr ?
> + roundup_pow_of_two(init_attr->cap.max_recv_wr) : 0;
> +
> + if (init_attr->srq)
> + wqe_cnt = to_xsrq(init_attr->srq)->xsrq.max;
> +
> + if (wqe_cnt > dev->xdev->caps.max_wqes) {
> + ibdev_err(&dev->ib_dev, "max_recv_wr:%d exceed max rq depth\n",
> + init_attr->cap.max_recv_wr);
> + return -EINVAL;
> + }
[ ... ]
> +static int calc_sq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *attr,
> + struct xsc_ib_qp *qp)
> +{
> + int wqe_size;
> + size_t wq_size;
> +
> + if (!attr->cap.max_send_wr) {
> + ibdev_err(&dev->ib_dev, "invalid max_send_wr:%d\n", attr->cap.max_send_wr);
> + return -1;
> + }
> +
> + wqe_size = 1 << dev->xdev->caps.send_wqe_shift;
> + qp->max_inline_data = (dev->xdev->caps.send_ds_num - 2) * sizeof(struct xsc_wqe_data_seg);
> + attr->cap.max_inline_data = qp->max_inline_data;
> +
> + attr->cap.max_send_wr = min_t(u32, attr->cap.max_send_wr, dev->xdev->caps.max_wqes);
> + qp->sq.wqe_cnt = roundup_pow_of_two(attr->cap.max_send_wr);
[Severity: Medium]
In set_rq_size(), if init_attr->cap.max_recv_wr is greater than 0x80000000U
and init_attr->srq is NULL, roundup_pow_of_two(init_attr->cap.max_recv_wr)
returns 1UL << 32 on 64-bit platforms (or invokes shift undefined behavior on
32-bit platforms) and truncates to 0 when assigned to u32 wqe_cnt.
When wqe_cnt truncates to 0, the wqe_cnt > dev->xdev->caps.max_wqes check is
bypassed and, for kernel QPs, qp->rq.wqe_cnt and qp->rq.max_post remain 0
even though qp->has_rq is 1.
Also, in calc_sq_size(), attr->cap.max_send_wr is silently clamped to
dev->xdev->caps.max_wqes with min_t() instead of returning -EINVAL when the
requested depth exceeds device limits.
Should set_rq_size() and calc_sq_size() validate init_attr->cap.max_recv_wr
and attr->cap.max_send_wr against dev->xdev->caps.max_wqes before calling
roundup_pow_of_two()?
[ ... ]
> +static int create_user_qp(struct xsc_ib_dev *dev, struct ib_pd *pd,
> + struct xsc_ib_qp *qp, struct ib_udata *udata,
> + struct xsc_ib_create_qp *ucmd,
> + struct xsc_create_qp_mbox_in **in, int *inlen)
> +{
> + int page_shift;
> + int npages;
> + u32 offset;
> + int ncont;
> + int err;
> + int hw_npages;
> + u32 sq_shift, rq_shift;
> + u32 sq_wqe_cnt, rq_wqe_cnt;
> + struct ib_umem_dmabuf *umem_dmabuf;
> +
> + ibdev_dbg(&dev->ib_dev, "buf_addr:0x%lx db_addr:0x%lx sq cnt:%u, rq cnt:%u, rq shift:%u\n",
> + (uintptr_t)ucmd->buf_addr, (uintptr_t)ucmd->db_addr,
> + ucmd->sq_wqe_count, ucmd->rq_wqe_count, ucmd->rq_wqe_shift);
> +
> + sq_shift = dev->xdev->caps.send_wqe_shift;
> + rq_shift = ucmd->rq_wqe_shift;
> + if (sq_shift < XSC_BASE_WQE_SHIFT || sq_shift > 31 ||
> + (ucmd->rq_wqe_count &&
> + (rq_shift < XSC_BASE_WQE_SHIFT || rq_shift > 31))) {
> + ibdev_err(&dev->ib_dev, "invalid user queue parameters\n");
> + return -EINVAL;
> + }
> +
> + if (ucmd->sq_wqe_count) {
> + if (!is_power_of_2(ucmd->sq_wqe_count) ||
> + ucmd->sq_wqe_count < (1u << (sq_shift - XSC_BASE_WQE_SHIFT))) {
> + ibdev_err(&dev->ib_dev, "invalid user sq parameters\n");
> + return -EINVAL;
> + }
> + sq_wqe_cnt = ucmd->sq_wqe_count >> (sq_shift - XSC_BASE_WQE_SHIFT);
> + } else {
> + sq_wqe_cnt = 0;
> + }
> +
> + if (ucmd->rq_wqe_count) {
> + if (!is_power_of_2(ucmd->rq_wqe_count) ||
> + ucmd->rq_wqe_count < (1u << (rq_shift - XSC_BASE_WQE_SHIFT))) {
> + ibdev_err(&dev->ib_dev, "invalid user rq parameters\n");
> + return -EINVAL;
> + }
> + rq_wqe_cnt = ucmd->rq_wqe_count >> (rq_shift - XSC_BASE_WQE_SHIFT);
> + } else {
> + rq_wqe_cnt = 0;
> + }
> +
> + if (sq_wqe_cnt > dev->xdev->caps.max_wqes ||
> + rq_wqe_cnt > dev->xdev->caps.max_wqes) {
> + ibdev_err(&dev->ib_dev, "user queue depth exceeds max %u\n",
> + dev->xdev->caps.max_wqes);
> + return -EINVAL;
> + }
> +
> + qp->sq.ds_cnt = ucmd->sq_wqe_count;
> + qp->sq.wqe_cnt = sq_wqe_cnt;
> + qp->sq.wqe_shift = sq_wqe_cnt ? sq_shift : 0;
> + qp->sq.max_gs = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM - XSC_RADDR_SEG_NUM;
> + qp->rq.ds_cnt = ucmd->rq_wqe_count;
> + qp->rq.wqe_cnt = rq_wqe_cnt;
> + qp->rq.wqe_shift = rq_wqe_cnt ? rq_shift : 0;
> + if (rq_wqe_cnt) {
> + qp->rq.max_gs = 1;
> + qp->rq.max_post = rq_wqe_cnt;
> + }
> +
> + qp->rq.offset = 0;
> + qp->sq.offset = (u32)qp->rq.wqe_cnt << qp->rq.wqe_shift;
> + qp->buf_size = (u64)qp->sq.offset +
> + ((u64)qp->sq.wqe_cnt << qp->sq.wqe_shift);
> + if (qp->buf_size > (u64)U16_MAX * XSC_PAGE_SIZE_4K) {
> + ibdev_err(&dev->ib_dev, "QP buffer size too large\n");
> + return -EINVAL;
> + }
[Severity: High]
Can the 32-bit shift in qp->sq.offset = (u32)qp->rq.wqe_cnt <<
qp->rq.wqe_shift overflow and bypass the qp->buf_size check in
create_user_qp()?
Because rq_shift is permitted up to 31 and struct xsc_ib_wq defines offset as
a signed 32-bit int, a user supplying ucmd->rq_wqe_shift = 31 and
ucmd->rq_wqe_count = 1U << 28 gets rq_wqe_cnt = 2 (which passes the max_wqes
check), while (u32)qp->rq.wqe_cnt << qp->rq.wqe_shift wraps in 32-bit
arithmetic to 0.
When qp->sq.offset wraps to 0, qp->buf_size only accounts for the SQ size and
passes the U16_MAX * XSC_PAGE_SIZE_4K check, so the driver pins and populates
PAS entries only for the SQ while create_qp_common() programs
in->req.log_rq_sz = ilog2(qp->rq.ds_cnt) (a 4 GiB RQ) into the hardware.
Similarly, if ucmd->sq_wqe_count is 0, create_user_qp() allocates 0 bytes for
the SQ in qp->buf_size, whereas create_qp_common() still sets
in->req.log_sq_sz = ilog2(0x80) in the firmware request.
> + hw_npages = DIV_ROUND_UP(qp->buf_size, XSC_PAGE_SIZE_4K);
> +
> + if (udata->inlen >= offsetofend(typeof(*ucmd), dmabuf_fd) &&
> + ucmd->dmabuf_fd != -1) {
> + if (ucmd->dmabuf_sz < qp->buf_size) {
[Severity: Medium]
In create_user_qp(), does the dmabuf check verify enough of udata->inlen
before reading ucmd->dmabuf_sz?
Checking udata->inlen >= offsetofend(typeof(*ucmd), dmabuf_fd) only ensures
the buffer covers dmabuf_fd (36 bytes), not the following dmabuf_sz field (up
to 48 bytes). Also, in create_qp_common(), ib_copy_from_udata() copies
min(udata->inlen, sizeof(ucmd)) without checking that udata->inlen is at
least offsetofend(struct xsc_ib_create_qp, flags) for the mandatory base
fields.
Should create_qp_common() reject udata->inlen smaller than
offsetofend(typeof(ucmd), flags) and create_user_qp() check
offsetofend(typeof(*ucmd), dmabuf_sz)?
> + ibdev_err(&dev->ib_dev, "dmabuf_sz %llu < qp buf_size %llu\n",
> + ucmd->dmabuf_sz, qp->buf_size);
> + return -EINVAL;
> + }
> +
> + umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd->dmabuf_sz,
> + ucmd->dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
[ ... ]
> + err = xsc_ib_get_buf_offset(ucmd->buf_addr, page_shift, &offset);
> + if (err) {
> + ibdev_err(&dev->ib_dev, "bad offset:%d\n", offset);
> + goto err_umem;
> + }
[Severity: Medium]
In create_user_qp(), offset is declared on the stack without initialization.
Looking at xsc_ib_get_buf_offset() in drivers/infiniband/hw/xsc/mem.c:
if (buf_off & off_mask)
return -EINVAL;
*offset = buf_off >> ilog2(off_size);
return 0;
When ucmd->buf_addr fails the alignment check, xsc_ib_get_buf_offset()
returns -EINVAL without initializing *offset, so ibdev_err() prints
uninitialized kernel stack memory. Also, when offset is non-zero, hw_npages =
DIV_ROUND_UP(qp->buf_size, XSC_PAGE_SIZE_4K) does not account for the
sub-page offset if the buffer crosses a 4K page boundary, and offset is never
passed to firmware.
> + ibdev_dbg(&dev->ib_dev, "npage:%d, page_shift:%d, ncont:%d, offset:%u, hw_npages %d\n",
> + npages, page_shift, ncont, offset, hw_npages);
> +
> + *inlen = sizeof(**in) + (size_t)hw_npages * sizeof((*in)->req.pas[0]);
> + *in = xsc_vzalloc(*inlen);
> + if (!*in) {
> + err = -ENOMEM;
> + goto err_umem;
> + }
> + xsc_ib_populate_pas(dev, qp->umem, page_shift, (*in)->req.pas, hw_npages, true);
[Severity: Critical]
Can create_user_qp() trigger an out-of-bounds heap write into (*in)->req.pas
when hw_npages is 0?
If userspace passes ucmd->sq_wqe_count == 0 and ucmd->rq_wqe_count == 0 (or
triggers the 32-bit overflow in qp->sq.offset with sq_wqe_count == 0),
qp->buf_size is 0 and hw_npages is 0. If userspace also provides a valid
ucmd->dmabuf_fd != -1 with ucmd->dmabuf_sz >= PAGE_SIZE, the check
ucmd->dmabuf_sz < qp->buf_size is false and ib_umem_dmabuf_get_pinned()
returns a non-empty umem.
create_user_qp() then allocates *in with *inlen = sizeof(**in) (zero elements
in req.pas[]) and calls xsc_ib_populate_pas() -> __xsc_ib_populate_pas() with
num_pages = 0:
drivers/infiniband/hw/xsc/mem.c:__xsc_ib_populate_pas()
pas[idx] = cpu_to_be64(cur);
...
i++;
/* Stop after num_pages reached */
if (i >> shift >= offset + num_pages)
return;
Because __xsc_ib_populate_pas() writes pas[0] before checking whether
num_pages has been reached, this overwrites 8 bytes past the end of *in.
Should create_user_qp() reject qp->buf_size == 0 (or both sq_wqe_count == 0
and rq_wqe_count == 0)?
[ ... ]
> +static int create_qp_common(struct xsc_ib_dev *dev, struct ib_pd *pd,
> + struct ib_qp_init_attr *init_attr,
> + struct ib_udata *udata, struct xsc_ib_qp *qp)
> +{
[ ... ]
> + if (init_attr->send_cq) {
> + qp->send_cq = init_attr->send_cq;
> + in->req.cqn_send = to_xcq(init_attr->send_cq)->xcq.cqn;
> + in->req.cqn_send = cpu_to_be16(in->req.cqn_send);
> + }
> +
> + if (init_attr->recv_cq) {
> + qp->recv_cq = init_attr->recv_cq;
> + in->req.cqn_recv = to_xcq(init_attr->recv_cq)->xcq.cqn;
> + in->req.cqn_recv = cpu_to_be16(in->req.cqn_recv);
> + }
[ ... ]
> + err = xsc_core_create_qp(dev->xdev, &qp->xqp, in, inlen);
> + if (err) {
> + ibdev_err(&dev->ib_dev, "create qp failed, err=%d\n", err);
> + goto err_create;
> + }
> +
> + qp->doorbell_qpn = qp->xqp.qpn;
> +
> + qp->xqp.qp_type = init_attr->qp_type;
> + ret += snprintf(ptr + ret, 256 - ret, "pdn=%d,", to_xpd(pd)->pdn);
> + ret += snprintf(ptr + ret, 256 - ret, "log_rq_sz=%d,", in->req.log_rq_sz);
> + ret += snprintf(ptr + ret, 256 - ret, "log_sq_sz=%d,", in->req.log_sq_sz);
> + ret += snprintf(ptr + ret, 256 - ret, "scqn=%d,", to_xcq(init_attr->send_cq)->xcq.cqn);
> + ret += snprintf(ptr + ret, 256 - ret, "rcqn=%d", to_xcq(init_attr->recv_cq)->xcq.cqn);
[Severity: High]
Earlier in create_qp_common(), init_attr->send_cq and init_attr->recv_cq are
checked for NULL before dereferencing, and get_cqs() also guards both
pointers.
However, these two snprintf() calls unconditionally dereference
to_xcq(init_attr->send_cq) and to_xcq(init_attr->recv_cq). Since ibcq is at
offset 0 of struct xsc_ib_cq, won't creating a QP with a NULL send_cq or
recv_cq cause a NULL pointer dereference here right after xsc_core_create_qp()
has already created the QP in firmware?
[ ... ]
> +int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata)
> +{
> + struct xsc_ib_dev *dev = to_xdev(qp->device);
> + struct xsc_ib_qp *xqp = to_xqp(qp);
> + struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
> + struct xsc_srq_qp_node *qp_node = NULL, *tmp_qp_node = NULL;
> + unsigned long flags;
> + struct xsc_ib_srq *xsrq;
> +
> + if (qp->qp_type == IB_QPT_RC) {
> + if (xqp->recv_cq) {
> + spin_lock_irq(&to_xcq(xqp->recv_cq)->err_state_qp_lock);
> + list_for_each_entry_safe(err_rq_node, tmp,
> + &to_xcq(xqp->recv_cq)->err_state_qp_list, entry) {
> + if (err_rq_node->qp_id == xqp->xqp.qpn && !err_rq_node->is_sq) {
> + list_del(&err_rq_node->entry);
> + kfree(err_rq_node);
> + }
> + }
> + spin_unlock_irq(&to_xcq(xqp->recv_cq)->err_state_qp_lock);
> + }
> +
> + if (xqp->send_cq) {
> + spin_lock_irq(&to_xcq(xqp->send_cq)->err_state_qp_lock);
> + list_for_each_entry_safe(err_sq_node, tmp,
> + &to_xcq(xqp->send_cq)->err_state_qp_list, entry) {
> + if (err_sq_node->qp_id == xqp->xqp.qpn && err_sq_node->is_sq) {
> + list_del(&err_sq_node->entry);
> + kfree(err_sq_node);
> + }
> + }
> + spin_unlock_irq(&to_xcq(xqp->send_cq)->err_state_qp_lock);
> + }
> + }
[Severity: High]
Can xsc_ib_destroy_qp() leak err_state_qp_node entries or race with concurrent
CQ polling when cleaning err_state_qp_list?
First, xsc_ib_modify_qp() and xsc_poll_one() call xsc_ib_err_state_qp() for
any kernel QP transitioning to XSC_QP_STATE_ERR regardless of qp_type, but
xsc_ib_destroy_qp() only purges err_state_qp_list when qp->qp_type ==
IB_QPT_RC. If a non-RC kernel QP (such as IB_QPT_GSI, IB_QPT_SMI, or
IB_QPT_RAW_PACKET) enters IB_QPS_ERR and is destroyed, its err_rq_node and
err_sq_node entries remain on the CQs' err_state_qp_list.
Second, even for IB_QPT_RC QPs, xsc_ib_destroy_qp() purges err_state_qp_list
before destroy_qp_common() moves the hardware QP to XSC_QP_STATE_RST and
cleans the CQs under xsc_ib_lock_cqs(). While xsc_core_qp_modify() is
sleeping in destroy_qp_common(), a concurrent xsc_ib_poll_cq() on another CPU
can poll an error CQE for this QP and call xsc_ib_err_state_qp(), inserting
new nodes onto err_state_qp_list after this cleanup loop has already run, and
xsc_generate_flush_err_cqe() walks err_state_qp_list without holding cq->lock
while destroy_qp_kernel() frees the QP work queue buffers.
Should the err_state_qp_list removal happen for all kernel QP types inside
destroy_qp_common() while holding the CQ locks after the QP has been moved to
RESET?
[ ... ]
> +static inline void xsc_set_qp_access_flag(struct xsc_modify_qp_mbox_in *in,
> + struct xsc_qp_context *context, int access_flags)
> +{
> + in->hdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V1);
> + context->qp_access_flags = 0;
[ ... ]
> +static int __xsc_ib_modify_qp(struct ib_qp *ibqp,
> + const struct ib_qp_attr *attr, int attr_mask,
> + enum ib_qp_state cur_state, enum ib_qp_state new_state,
> + const struct xsc_ib_modify_qp *ucmd,
> + struct ib_udata *udata)
> +{
> + struct xsc_ib_dev *dev = to_xdev(ibqp->device);
> + struct xsc_ib_qp *qp = to_xqp(ibqp);
> + struct xsc_ib_cq *send_cq, *recv_cq;
> + struct xsc_qp_context *context;
> + struct xsc_modify_qp_mbox_in *in;
> + struct xsc_qp_path path = {};
> + int err;
> + char buf[256] = {0};
> + char *ptr = buf;
> + int ret = 0;
> + struct xsc_core_device *xdev = dev->xdev;
> +
> + in = kzalloc_obj(*in, GFP_KERNEL);
> + if (!in)
> + return -ENOMEM;
> +
> + context = &qp->ctx;
[Severity: Medium]
In __xsc_ib_modify_qp(), context points directly to &qp->ctx and mutates the
cached context in place before xsc_set_path() or xsc_core_qp_modify()
succeeds, and transitioning to IB_QPS_RESET never clears stale fields in
qp->ctx. In addition, xsc_set_qp_access_flag() sets in->hdr.ver =
cpu_to_be16(XSC_QP_CONTEXT_V1) only when IB_QP_ACCESS_FLAGS is present in
attr_mask, so on subsequent transitions where IB_QP_ACCESS_FLAGS is omitted,
in->hdr.ver remains 0 even though the cached context->qp_access_flags is
copied into in->ctx.
Also, in create_qp_common(), in->req.page_shift is set to PAGE_SHIFT even
though create_user_qp() and create_kernel_qp() always compute hw_npages
(in->req.pa_num) and populate in->req.pas[] in 4 KB units (XSC_PAGE_SIZE_4K /
XSC_PAGE_SHIFT_4K). On architectures with 64 KB pages (PAGE_SHIFT == 16),
won't passing PAGE_SHIFT to firmware mismatch the 4 KB PAS entries?
[ ... ]
> + /*
> + * If we moved a kernel QP to RESET, clean up all old CQ
> + * entries and reinitialize the QP.
> + */
> + if (new_state == IB_QPS_RESET && !ibqp->uobject) {
> + get_cqs(qp->xqp.qp_type, qp->ibqp.send_cq, qp->ibqp.recv_cq, &send_cq, &recv_cq);
> + xsc_ib_cq_clean(recv_cq, qp->xqp.qpn);
> + if (send_cq != recv_cq)
> + xsc_ib_cq_clean(send_cq, qp->xqp.qpn);
> +
> + qp->rq.head = 0;
> + qp->rq.tail = 0;
> + qp->sq.head = 0;
> + qp->sq.tail = 0;
> + qp->sq.cur_post = 0;
> + qp->sq.last_poll = 0;
> + qp->sq.last_comp_ce_wqe = -1;
> + qp->sq.last_post_ce_wqe = -1;
> + qp->has_trig_cq_comp_handle = false;
> + }
[ ... ]
> + err = __xsc_ib_modify_qp(ibqp, attr, attr_mask, cur_state, new_state, &ucmd, udata);
> +
> + if (!err && (attr_mask & IB_QP_STATE) && !udata) {
> + bool trigger;
> +
> + get_cqs(qp->xqp.qp_type, qp->send_cq, qp->recv_cq, &send_cq, &recv_cq);
> + xsc_ib_lock_cqs(send_cq, recv_cq);
> + err = xsc_ib_err_state_qp(dev, qp, to_xsc_state(cur_state),
> + to_xsc_state(new_state), &trigger);
> + xsc_ib_unlock_cqs(send_cq, recv_cq);
> +
> + if (!err && trigger)
> + xsc_ib_trigger_cq_comp_handler(qp->send_cq, qp->recv_cq);
> + }
[Severity: High]
When a kernel QP enters error state via a hardware error CQE in
xsc_poll_one(), xsc_ib_err_state_qp() queues err_rq_node and err_sq_node onto
the CQs' err_state_qp_list and sets qp->has_trig_cq_comp_handle = true, but
qp->state remains at its previous software state (such as IB_QPS_RTS).
When xsc_ib_modify_qp() later resets the QP to IB_QPS_RESET, cur_state is
taken from qp->state (not IB_QPS_ERR), so the removal branch in
xsc_ib_err_state_qp() (cur_state == XSC_QP_STATE_ERR && state !=
XSC_QP_STATE_ERR) is not taken. Because __xsc_ib_modify_qp() also resets
qp->has_trig_cq_comp_handle = false, won't the existing err_rq_node and
err_sq_node be leaked on err_state_qp_list and duplicated if the QP enters
error state again?
Furthermore, in __xsc_ib_modify_qp(), qp->rq and qp->sq indices are zeroed
without holding cq->lock, qp->rq.lock, or qp->sq.lock before
xsc_ib_err_state_qp() removes the QP from err_state_qp_list, and
qp->rq.flush_wqe_cnt, qp->sq.flush_wqe_cnt, and qp->sq.need_flush[] are not
cleared. Can a concurrent xsc_ib_poll_cq() running
xsc_generate_flush_err_cqe() race with this reset?
> +
> + if (!err && !udata && (attr_mask & IB_QP_STATE) && ibqp->srq &&
> + cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
> + srq = to_xsrq(ibqp->srq);
> + spin_lock_irqsave(&srq->lock, flags);
> + srq->qp_num++;
> + wr_num = XSC_SRQ_MIN_WR_NUM;
> + ibdev_dbg(&dev->ib_dev,
> + "SRQ %d: qp %d to INIT, post %d srq wrs, qp_num %d\n",
> + srq->xsrq.srqn, qp->xqp.qpn, wr_num, srq->qp_num);
> + xsc_post_cache_wrs(srq, qp, wr_num, true);
> + spin_unlock_irqrestore(&srq->lock, flags);
> + }
[Severity: High]
Can kernel QPs attached to an SRQ leak cached WRs and corrupt srq->qp_num
across QP creation, reset, and destruction?
1. In create_qp_common(), a kernel QP with init_attr->srq is added to
xsrq->xsrq.srq_qp_list while still in IB_QPS_RESET and without
incrementing xsrq->qp_num. If xsc_ib_post_srq_recv() runs before the QP
transitions to IB_QPS_INIT, it sees the QP on srq_qp_list with
flush_wqe_cnt == 0 and calls xsc_post_cache_wrs(srq, qp, wr_num, false),
posting WQEs and ringing the RX doorbell on a RESET QP. When
xsc_ib_modify_qp() then transitions the QP from IB_QPS_RESET to
IB_QPS_INIT, xsc_post_cache_wrs(srq, qp, wr_num, true) overwrites
qp->srq_head_wr because init is true, leaking the previously attached
ib_recv_wr and sg_list allocations.
2. When a kernel QP with an SRQ is modified back to IB_QPS_RESET in
__xsc_ib_modify_qp(), qp->rq.head and qp->rq.tail are zeroed without
calling xsc_restore_cache_srq_wrs(), resetting qp->rq.flush_wqe_cnt, or
decrementing srq->qp_num. Transitioning from IB_QPS_RESET to IB_QPS_INIT
again increments srq->qp_num a second time and overwrites qp->srq_head_wr.
Conversely, if a QP is destroyed in IB_QPS_RESET before ever reaching
IB_QPS_INIT, xsc_ib_destroy_qp() decrements srq->qp_num even though it was
never incremented.
3. In xsc_ib_destroy_qp(), list_empty(&to_xsrq(qp->srq)->xsrq.srq_qp_list) is
checked outside xsrq->lock (which races with list_sort() in
xsc_ib_post_srq_recv()), and xsc_restore_cache_srq_wrs() is called before
destroy_qp_common() transitions the QP to RESET.
[ ... ]
> +int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
> + struct ib_qp_init_attr *qp_init_attr)
> +{
> + struct xsc_ib_dev *dev = to_xdev(ibqp->device);
> + struct xsc_ib_qp *qp = to_xqp(ibqp);
> + struct xsc_query_qp_mbox_out *outb;
> + struct xsc_qp_context *context;
> + int err = 0;
> +
> + mutex_lock(&qp->mutex);
[Severity: High]
Does xsc_ib_query_qp() leak uninitialized kernel memory to userspace?
In drivers/infiniband/core/uverbs_cmd.c, ib_uverbs_query_qp() allocates attr
and init_attr with kmalloc_obj() (not kzalloc), and ib_query_qp() only zeroes
the two sgid_attr pointers before calling the driver's query_qp callback.
Because xsc_ib_query_qp() does not memset() qp_attr or qp_init_attr before
populating a subset of fields, uninitialized heap bytes in fields such as
path_mig_state, qkey, qp_access_flags, pkey_index, min_rnr_timer, timeout,
alt_port_num, alt_timeout, ah_attr (including 32 bytes of GRH/dgid data when
the uninitialized ah_flags byte has IB_AH_GRH set), and qp_init_attr are
copied back to userspace in struct ib_uverbs_query_qp_resp.
> +
> + outb = kzalloc_obj(*outb, GFP_KERNEL);
> + if (!outb) {
> + err = -ENOMEM;
> + goto out;
> + }
> + context = &outb->ctx;
> + err = xsc_core_qp_query(dev->xdev, &qp->xqp, outb, sizeof(*outb));
> + if (err)
> + goto out_free;
> +
> + if (qp->xqp.err_occurred) {
> + qp->state = IB_QPS_ERR;
> + qp->xqp.err_occurred = 0;
> + }
> + qp_attr->qp_state = qp->state;
> + qp_attr->path_mtu = context->mtu_mode ? IB_MTU_4096 : IB_MTU_1024;
> + qp_attr->rq_psn = be32_to_cpu(context->next_recv_psn) & 0xffffff;
> + qp_attr->sq_psn = be32_to_cpu(context->next_send_psn) & 0xffffff;
> + qp_attr->dest_qp_num = be32_to_cpu(context->remote_qpn) & 0xffffff;
> + qp_attr->sq_draining = 0;
> + qp_attr->retry_cnt = context->retry_cnt;
> + qp_attr->rnr_retry = context->rnr_retry;
> + qp_attr->cur_qp_state = qp_attr->qp_state;
> + qp_attr->cap.max_recv_wr = qp->rq.wqe_cnt;
> + qp_attr->cap.max_recv_sge = qp->rq.max_gs;
> + qp_attr->max_rd_atomic = context->max_sra;
> + qp_attr->max_dest_rd_atomic = context->max_rra;
> + qp_attr->port_num = 1;
> +
> + if (!ibqp->uobject) {
> + qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
> + qp_attr->cap.max_send_sge = qp->sq.max_gs;
> + } else {
> + qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
> + qp_attr->cap.max_send_sge = qp->sq.max_gs;
> + }
[Severity: Medium]
In __xsc_ib_modify_qp(), context->mtu_mode is set to 0 for IB_MTU_1024, 1 for
IB_MTU_4096, and 2 for IB_MTU_8192, but xsc_ib_query_qp() decodes
context->mtu_mode ? IB_MTU_4096 : IB_MTU_1024, reporting IB_MTU_4096 when
mtu_mode is 2. Should xsc_ib_query_qp() map mtu_mode == 2 to IB_MTU_8192?
Also, both branches of if (!ibqp->uobject) in xsc_ib_query_qp() are
identical, and the static helpers to_ib_qp_state() and to_ib_mig_state()
defined just above xsc_ib_query_qp() are unused.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=13
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 14/15] RDMA/xsc: add the RDMA send path
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (12 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 13/15] RDMA/xsc: add QP verbs Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
2026-11-04 15:57 ` [PATCH v1 15/15] RDMA/xsc: add RDMA counters and sysfs Xin Tian
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Implement the work request post and send path.
post_send validates the opcode, work queue headroom, SGE count (RDMA
reads are limited to a single SGE) and the drain gate, then writes
the WQE under the send queue lock. RC queues build send, immediate,
write and remote-read WQEs and record the slot metadata the poll
loop needs; unsupported opcodes such as local invalidate and the
register-MR work requests are rejected. Since the hardware rejects
zero-length RDMA reads, such a request is issued as a write and the
poll loop restores the read opcode on completion.
For QP1 and UD traffic the path packs a complete RoCEv2 header
(ethernet, IPv4 or IPv6, UDP, BTH, DETH) into a per-QP DMA header
slot, computes the ICRC over masked pseudo-headers and payload, and
posts a single SGE referencing the header. The MAD payload is
copied into the header buffer through a DMA-to-CPU translation
helper, which fails when the IOMMU state makes CPU access to DMA
addresses impossible; the same helper failure makes QP1 receive
handling report a local access error.
Add the drain helpers the core uses on destruction: move the QP to
error, post tagged drain work requests and wait for their
completions, polling directly for direct-poll CQs and using the
completion callback otherwise.
The shared BTH and DETH header structs are renamed from the rxe
prefix to an xsc prefix to avoid colliding with the soft-RoCE module
on the include path.
The message-opcode map and the software CRC32 table are now built at
probe time since the send path is their first user.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/hw/xsc/cq.c | 36 +-
drivers/infiniband/hw/xsc/main.c | 22 +-
drivers/infiniband/hw/xsc/qp.c | 732 +++++++++++++++++-
drivers/infiniband/hw/xsc/xsc_ib.h | 144 ++--
.../ethernet/yunsilicon/xsc/common/driver.h | 4 +-
5 files changed, 815 insertions(+), 123 deletions(-)
diff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c
index 36d122f8d..a5ab8d00d 100644
--- a/drivers/infiniband/hw/xsc/cq.c
+++ b/drivers/infiniband/hw/xsc/cq.c
@@ -143,11 +143,11 @@ static void *get_seg_wqe(void *first, int n)
return first + (n << XSC_BASE_WQE_SHIFT);
}
-static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
- struct xsc_ib_qp **cur_qp,
- struct ib_wc *wc,
- struct xsc_cqe *cqe,
- u8 opcode)
+static int xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
+ struct xsc_ib_qp **cur_qp,
+ struct ib_wc *wc,
+ struct xsc_cqe *cqe,
+ u8 opcode)
{
struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
void *recv;
@@ -159,8 +159,8 @@ static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
struct ib_unpacked_vlan *vlan;
struct ib_grh *grh;
struct ib_mad *mad;
- struct rxe_bth *bth;
- struct rxe_deth *deth;
+ struct xsc_bth *bth;
+ struct xsc_deth *deth;
unsigned int pading_sz = 0;
struct xsc_ib_wq *wq;
int idx;
@@ -173,7 +173,13 @@ static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
handle_responder(wc, cqe, *cur_qp, opcode);
data_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);
- recv = xsc_ib_recv_mad_sg_virt_addr(&dev->ib_dev, wc, data_seg->va);
+ recv = xsc_ib_mad_sg_virt_addr(&dev->ib_dev, data_seg->va);
+ if (unlikely(!recv)) {
+ ibdev_err(&dev->ib_dev,
+ "cannot map the MAD receive buffer to a virtual address\n");
+ wc->status = IB_WC_LOC_ACCESS_ERR;
+ return -EOPNOTSUPP;
+ }
eth = (struct ib_unpacked_eth *)recv;
grh = (struct ib_grh *)recv;
@@ -197,8 +203,8 @@ static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
ip6h = (struct ipv6hdr *)l3_start;
udph = (struct udphdr *)(ip6h + 1);
}
- bth = (struct rxe_bth *)(udph + 1);
- deth = (struct rxe_deth *)(bth + 1);
+ bth = (struct xsc_bth *)(udph + 1);
+ deth = (struct xsc_deth *)(bth + 1);
mad = (struct ib_mad *)(deth + 1);
if (eth_type == ETH_P_IP) {
@@ -216,6 +222,8 @@ static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
ibdev_dbg(&dev->ib_dev, "qp[%d] recv MAD packet, msg_len=%d\n",
(*cur_qp)->xqp.qpn, wc->byte_len);
wc->status = IB_WC_SUCCESS;
+
+ return 0;
}
static void xsc_generate_lost_cqe(struct xsc_ib_dev *dev, struct xsc_ib_wq *wq,
@@ -303,6 +311,10 @@ static int xsc_poll_one(struct xsc_ib_cq *cq,
break;
}
wq->last_comp_ce_wqe = idx;
+ if (wq->zero_length_read[idx]) {
+ opcode = XSC_OPCODE_RDMA_REQ_READ;
+ wq->zero_length_read[idx] = 0;
+ }
handle_good_req(wc, cqe, opcode);
wc->wr_id = wq->wrid[idx];
wq->tail = wq->wqe_head[idx] + 1;
@@ -331,7 +343,8 @@ static int xsc_poll_one(struct xsc_ib_cq *cq,
case XSC_OPCODE_RDMA_MAD_RSP_RECV:
ibdev_dbg(&dev->ib_dev, "recv MAD, qpn:%u\n", qpn);
- xsc_handle_rdma_mad_resp_recv(cq, cur_qp, wc, cqe, opcode);
+ if (xsc_handle_rdma_mad_resp_recv(cq, cur_qp, wc, cqe, opcode))
+ wc->status = IB_WC_LOC_ACCESS_ERR;
break;
case XSC_OPCODE_RDMA_REQ_ERROR:
@@ -372,6 +385,7 @@ static int xsc_poll_one(struct xsc_ib_cq *cq,
if (wq->need_flush[idx])
atomic_dec(&wq->flush_wqe_cnt);
wq->need_flush[idx] = 0;
+ wq->zero_length_read[idx] = 0;
ibdev_err(&dev->ib_dev, "req error\n%08x %08x %08x %08x %08x %08x\n",
p[0], p[1], p[2], p[3], p[5], p[6]);
wc->status = IB_WC_GENERAL_ERR;
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
index 138ca1a80..057468798 100644
--- a/drivers/infiniband/hw/xsc/main.c
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -474,6 +474,22 @@ static int init_node_data(struct xsc_ib_dev *dev)
return 0;
}
+static void crc_table_init(struct xsc_ib_dev *dev)
+{
+ u32 c, i, j;
+
+ for (i = 0; i < 256; i++) {
+ c = i;
+ for (j = 0; j < 8; j++) {
+ if (c & 1)
+ c = 0xedb88320L ^ (c >> 1);
+ else
+ c = c >> 1;
+ }
+ dev->crc_32_table[i] = c;
+ }
+}
+
static void xsc_ib_get_dev_fw_str(struct ib_device *ibdev, char *str)
{
struct xsc_core_device *dev = to_xdev(ibdev)->xdev;
@@ -521,6 +537,7 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.modify_qp = xsc_ib_modify_qp;
dev->ib_dev.ops.query_qp = xsc_ib_query_qp;
dev->ib_dev.ops.destroy_qp = xsc_ib_destroy_qp;
+ dev->ib_dev.ops.post_send = xsc_ib_post_send;
dev->ib_dev.ops.post_recv = xsc_ib_post_recv;
dev->ib_dev.ops.create_cq = xsc_ib_create_cq;
dev->ib_dev.ops.destroy_cq = xsc_ib_destroy_cq;
@@ -533,6 +550,8 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.get_port_immutable = xsc_port_immutable;
+ dev->ib_dev.ops.drain_sq = xsc_ib_drain_sq;
+ dev->ib_dev.ops.drain_rq = xsc_ib_drain_rq;
dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
dev->ib_dev.ops.create_srq = xsc_ib_create_srq;
dev->ib_dev.ops.destroy_srq = xsc_ib_destroy_srq;
@@ -678,6 +697,8 @@ static int init_one(struct xsc_core_device *xdev,
if (err)
goto err_free;
+ crc_table_init(dev);
+ build_opcode_map(dev);
err = xsc_reg_local_dma_mr(xdev);
if (err)
goto err_free;
@@ -688,7 +709,6 @@ static int init_one(struct xsc_core_device *xdev,
if (err)
goto err_free;
- rdma_roce_rescan_device(&dev->ib_dev);
dev->ib_active = true;
*ibdev = dev;
diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
index 0dfb10294..46ca3dd95 100644
--- a/drivers/infiniband/hw/xsc/qp.c
+++ b/drivers/infiniband/hw/xsc/qp.c
@@ -11,13 +11,10 @@
#include <rdma/xsc-abi.h>
#include "common/xsc_hsi.h"
#include <linux/crc32.h>
+#include <linux/unaligned.h>
#include <linux/dma-direct.h>
#include <rdma/ib_addr.h>
-#ifndef ETH_P_IBOE
-#define ETH_P_IBOE 0x8915
-#endif
-
#define MAD_QUEUE_DEPTH 128
enum {
@@ -29,6 +26,23 @@ enum {
#define UDP_SPORT_MASK 0xffff
#define UDP_SPORT_OFFSET 0
+void build_opcode_map(struct xsc_ib_dev *dev)
+{
+ struct xsc_msg_opcode *xsc_ib_opcode = dev->xdev->board_info->msg_opcode;
+
+ dev->opcode_map[IB_WR_SEND] = xsc_ib_opcode->send;
+ dev->opcode_map[IB_WR_SEND_WITH_IMM] = xsc_ib_opcode->send_with_imm;
+ dev->opcode_map[IB_WR_RDMA_WRITE] = xsc_ib_opcode->rdma_write;
+ dev->opcode_map[IB_WR_RDMA_WRITE_WITH_IMM] = xsc_ib_opcode->rdma_write_with_imm;
+ dev->opcode_map[IB_WR_RDMA_READ] = xsc_ib_opcode->rdma_read;
+ dev->opcode_map[IB_WR_ATOMIC_CMP_AND_SWP] = xsc_ib_opcode->atomic_cmp_and_swp;
+ dev->opcode_map[IB_WR_ATOMIC_FETCH_AND_ADD] = xsc_ib_opcode->atomic_fetch_and_add;
+ dev->opcode_map[IB_WR_MASKED_ATOMIC_CMP_AND_SWP] =
+ xsc_ib_opcode->masked_atomic_cmp_and_swp;
+ dev->opcode_map[IB_WR_MASKED_ATOMIC_FETCH_AND_ADD] =
+ xsc_ib_opcode->masked_atomic_fetch_and_add;
+}
+
static int is_qp0(enum ib_qp_type qp_type)
{
return qp_type == IB_QPT_SMI;
@@ -59,6 +73,83 @@ static void *get_seg_wqe(void *first, int n)
return first + (n << XSC_BASE_WQE_SHIFT);
}
+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->sq.offset + (n << qp->sq.wqe_shift));
+}
+
+static inline void set_remote_addr_seg(struct xsc_core_device *xdev,
+ struct xsc_wqe_data_seg *remote_seg,
+ u32 msg_len, u64 remote_addr, u32 rkey)
+{
+ xsc_set_data_seg(xdev, remote_seg, remote_addr, rkey, msg_len);
+}
+
+static void set_local_data_seg(struct xsc_core_device *xdev,
+ struct xsc_wqe_data_seg *data_seg, struct ib_sge *sg)
+{
+ xsc_set_data_seg(xdev, data_seg, sg->addr, sg->lkey, sg->length);
+}
+
+static int set_data_inl_seg(struct xsc_ib_qp *qp, const struct ib_send_wr *wr, void *ctrl,
+ unsigned int seg_index, unsigned int *num_segs)
+{
+ unsigned int ds_len = sizeof(struct xsc_wqe_data_seg);
+ unsigned int segs_per_wqe;
+ unsigned int offset = 0;
+ size_t total_len = 0;
+ struct xsc_wqe_data_seg *data_seg;
+ void *addr;
+ int len;
+ int i;
+
+ segs_per_wqe = XSC_SEND_SEG_NUM << (qp->sq.wqe_shift - XSC_SEND_WQE_SHIFT);
+
+ for (i = 0; i < wr->num_sge; ++i)
+ total_len += wr->sg_list[i].length;
+
+ /* max_inline_data is the byte budget of the whole WQE, not of a SGE */
+ if (total_len > qp->max_inline_data)
+ return -ENOMEM;
+
+ /* bound the payload segments by the WQE slot before any memcpy lands */
+ if (seg_index + DIV_ROUND_UP(total_len, ds_len) > segs_per_wqe)
+ return -ENOMEM;
+
+ for (i = 0; i < wr->num_sge; ++i) {
+ addr = (void *)(unsigned long)wr->sg_list[i].addr;
+ len = wr->sg_list[i].length;
+
+ while (len) {
+ unsigned int copy_len;
+
+ if (!offset) {
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ seg_index++;
+ }
+ copy_len = min_t(unsigned int, len, ds_len - offset);
+ memcpy((char *)data_seg + offset, addr, copy_len);
+ offset = (offset + copy_len) % ds_len;
+ addr += copy_len;
+ len -= copy_len;
+ }
+ }
+
+ *num_segs = seg_index;
+ return 0;
+}
+
+static __be32 send_ieth(const struct ib_send_wr *wr)
+{
+ switch (wr->opcode) {
+ case IB_WR_SEND_WITH_IMM:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ return wr->ex.imm_data;
+ default:
+ return 0;
+ }
+}
+
static void xsc_ib_qp_event(struct xsc_core_qp *qp, int type)
{
struct ib_qp *ibqp = &core_to_xqp(qp)->ibqp;
@@ -308,7 +399,6 @@ static int create_user_qp(struct xsc_ib_dev *dev, struct ib_pd *pd,
xsc_ib_cont_pages(qp->umem, ucmd->buf_addr, &npages, &page_shift,
&ncont);
if (ncont != npages) {
- /* TODO: peer memory support failed */
page_shift = PAGE_SHIFT;
ncont = npages;
}
@@ -419,12 +509,15 @@ static int create_kernel_qp(struct xsc_ib_dev *dev,
qp->sq.wqe_head = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wqe_head), GFP_KERNEL);
qp->sq.wr_opcode = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wr_opcode), GFP_KERNEL);
qp->sq.need_flush = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.need_flush), GFP_KERNEL);
+ qp->sq.zero_length_read = kcalloc(qp->sq.wqe_cnt,
+ sizeof(*qp->sq.zero_length_read), GFP_KERNEL);
qp->sq.prev_ce_wqe = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.prev_ce_wqe), GFP_KERNEL);
qp->sq.last_comp_ce_wqe = -1;
qp->sq.last_post_ce_wqe = -1;
if (!qp->sq.wrid || !qp->sq.wr_data || !qp->rq.wrid || !qp->sq.prev_ce_wqe ||
- !qp->sq.w_list || !qp->sq.wqe_head || !qp->sq.wr_opcode || !qp->sq.need_flush) {
+ !qp->sq.w_list || !qp->sq.wqe_head || !qp->sq.wr_opcode || !qp->sq.need_flush ||
+ !qp->sq.zero_length_read) {
err = -ENOMEM;
goto err_wrid;
}
@@ -450,6 +543,7 @@ static int create_kernel_qp(struct xsc_ib_dev *dev,
err_wrid:
xsc_vfree(*in);
*in = NULL;
+ kfree(qp->sq.zero_length_read);
kfree(qp->sq.prev_ce_wqe);
kfree(qp->sq.need_flush);
kfree(qp->sq.wr_opcode);
@@ -471,6 +565,7 @@ static void destroy_qp_kernel(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)
qp->sq.hdr_buf, qp->sq.hdr_dma);
kfree(qp->sq.prev_ce_wqe);
kfree(qp->sq.need_flush);
+ kfree(qp->sq.zero_length_read);
kfree(qp->sq.wr_opcode);
kfree(qp->sq.wqe_head);
kfree(qp->sq.w_list);
@@ -1437,13 +1532,538 @@ static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq)
return cur + nreq >= wq->max_post;
}
+static inline void xsc_post_send_db(struct xsc_ib_qp *qp,
+ struct xsc_core_device *xdev,
+ int nreq)
+{
+ u32 next_pid;
+
+ if (unlikely(!nreq))
+ return;
+
+ qp->sq.head += nreq;
+
+ next_pid = qp->sq.head << (qp->sq.wqe_shift - XSC_BASE_WQE_SHIFT);
+ xsc_update_tx_db(xdev, qp->doorbell_qpn, next_pid);
+}
+
+static inline u32 xsc_crc32(struct xsc_ib_dev *dev, u32 crc, u8 *buf, size_t len)
+{
+ u32 i;
+
+ for (i = 0; i < len; i++)
+ crc = dev->crc_32_table[(crc ^ buf[i]) & 0xff] ^ (crc >> 8);
+
+ return crc;
+}
+
#define BTH_QPN_MASK (0x00ffffff)
#define BTH_PSN_MASK (0x00ffffff)
/* Compute a partial ICRC for all the IB transport headers. */
+static int xsc_icrc_hdr(struct xsc_ib_dev *dev, void *pkt, u32 size, u32 *icrc)
+{
+ struct iphdr *ip4h = NULL;
+ struct ipv6hdr *ip6h = NULL;
+ struct udphdr *udph;
+ struct ib_unpacked_eth *eth;
+ struct xsc_bth *bth;
+ struct ib_unpacked_deth *deth;
+ struct ib_unpacked_vlan *vlan;
+ int crc;
+ int crc_field_len;
+ __be16 l3_type;
+ u8 *l3_start;
+
+ int hdr_size;
+
+ /* pseudo header buffer size is calculate using ipv6 header size since
+ * it is bigger than ipv4
+ */
+ u8 pshdr[sizeof(struct udphdr) +
+ sizeof(struct ipv6hdr) +
+ sizeof(*bth) + sizeof(*deth)] __aligned(4);
+
+ eth = pkt;
+
+ if (eth->type == htons(ETH_P_8021Q)) {
+ vlan = (struct ib_unpacked_vlan *)(eth + 1);
+ l3_type = vlan->type;
+ l3_start = (u8 *)(vlan + 1);
+ size -= 4;
+ } else {
+ l3_type = eth->type;
+ l3_start = (u8 *)(eth + 1);
+ }
+
+ hdr_size = sizeof(struct udphdr) +
+ (l3_type == htons(ETH_P_IP) ?
+ sizeof(struct iphdr) : sizeof(struct ipv6hdr));
-/* Routine for sending QP1 packets for RoCE V1 an V2 */
-/* TODO: sq hdr buf should be created dynamically for mult entry */
+ crc_field_len = hdr_size + sizeof(*bth) + sizeof(*deth);
+
+ if (crc_field_len != size) {
+ ibdev_err(&dev->ib_dev, "Unmatched hdr: expect %d actual %d\n",
+ crc_field_len, size);
+ return -EINVAL;
+ }
+
+ ip4h = (struct iphdr *)(l3_start);
+ ip6h = (struct ipv6hdr *)(l3_start);
+ udph = (struct udphdr *)(ip4h + 1);
+ bth = (struct xsc_bth *)(udph + 1);
+
+ memcpy(pshdr, l3_start, crc_field_len);
+
+ /* This seed is the result of computing a CRC with a seed of
+ * 0xfffffff and 8 bytes of 0xff representing a masked LRH.
+ */
+ crc = 0xdebb20e3;
+
+ if (l3_type == htons(ETH_P_IP)) { /* IPv4 */
+ memcpy(pshdr, ip4h, hdr_size);
+ ip4h = (struct iphdr *)pshdr;
+ udph = (struct udphdr *)(ip4h + 1);
+
+ ip4h->ttl = 0xff;
+ ip4h->check = CSUM_MANGLED_0;
+ ip4h->tos = 0xff;
+ } else { /* IPv6 */
+ memcpy(pshdr, ip6h, hdr_size);
+ ip6h = (struct ipv6hdr *)pshdr;
+ udph = (struct udphdr *)(ip6h + 1);
+
+ memset(ip6h->flow_lbl, 0xff, sizeof(ip6h->flow_lbl));
+ ip6h->priority = 0xf;
+ ip6h->hop_limit = 0xff;
+ }
+ udph->check = CSUM_MANGLED_0;
+
+ bth = (struct xsc_bth *)(udph + 1);
+ /* exclude bth.resv8a */
+ bth->qpn |= cpu_to_be32(~BTH_QPN_MASK);
+
+ *icrc = xsc_crc32(dev, crc, pshdr, crc_field_len);
+
+ return 0;
+}
+
+/* Routine for sending QP1 packets for RoCE V2 */
+static int build_qp1_send_v2(struct xsc_ib_dev *dev,
+ struct xsc_ib_qp *qp,
+ const struct ib_send_wr *wr,
+ struct ib_sge *sge,
+ int payload_size, u32 *crc)
+{
+ struct xsc_ib_ah *ah = container_of(ud_wr((struct ib_send_wr *)wr)->ah, struct xsc_ib_ah,
+ ibah);
+ const struct ib_gid_attr *sgid_attr = ah->ibah.sgid_attr;
+ u16 ether_type;
+ bool is_eth = false;
+ bool is_vlan = false;
+ u8 ip_version = 0;
+ u16 vlan_id = 0xFFFF;
+ int rc = 0;
+ int cm_pcp = 0;
+ void *hdr_buf;
+ struct net_device *ndev;
+
+ memset(&qp->qp1_hdr, 0, sizeof(qp->qp1_hdr));
+
+ if (!qp->sq.hdr_buf) {
+ ibdev_err(&dev->ib_dev, "QP1 buffer is empty!");
+ return -ENOMEM;
+ }
+ hdr_buf = (u8 *)qp->sq.hdr_buf + MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index;
+
+ if (!sgid_attr) {
+ ibdev_err(&dev->ib_dev, "sgid_addr is null\n");
+ return -ENXIO;
+ }
+
+ if (sgid_attr->gid_type != IB_GID_TYPE_ROCE_UDP_ENCAP) {
+ ibdev_err(&dev->ib_dev, "RoCEv1 GID type not supported\n");
+ return -EINVAL;
+ }
+
+ rc = rdma_read_gid_l2_fields(sgid_attr, &vlan_id, NULL);
+ if (rc)
+ return rc;
+
+ if (ipv6_addr_v4mapped((struct in6_addr *)&sgid_attr->gid)) {
+ ip_version = 4;
+ ether_type = ETH_P_IP;
+ } else {
+ ip_version = 6;
+ ether_type = ETH_P_IPV6;
+ }
+
+ is_eth = true;
+ is_vlan = (vlan_id && (vlan_id < 0x1000)) ? true : false;
+
+ ib_ud_header_init(payload_size, !is_eth, is_eth, is_vlan, false,
+ ip_version, true, 0, &qp->qp1_hdr);
+
+ /* ETH */
+ ether_addr_copy(qp->qp1_hdr.eth.dmac_h, ah->av.rmac);
+ ndev = ib_device_get_netdev(&dev->ib_dev, qp->port);
+ if (!ndev) {
+ ibdev_err(&dev->ib_dev, "no netdev for port %u\n", qp->port);
+ return -ENODEV;
+ }
+ ether_addr_copy(qp->qp1_hdr.eth.smac_h, ndev->dev_addr);
+ dev_put(ndev);
+
+ /* For vlan, check the sgid for vlan existence */
+ if (!is_vlan) {
+ qp->qp1_hdr.eth.type = cpu_to_be16(ether_type);
+ } else {
+ if (dev->xdev->priv.cm_pcp != XSC_DSCP_PCP_UNSET)
+ cm_pcp = dev->xdev->priv.cm_pcp << 13;
+ else
+ cm_pcp = ((ah->av.stat_rate_sl >> 1) & 0x7) << 13;
+ qp->qp1_hdr.vlan.type = cpu_to_be16(ether_type);
+ qp->qp1_hdr.vlan.tag = cpu_to_be16(vlan_id | cm_pcp);
+ }
+
+#define ECN_CAPABLE_TRANSPORT 0x2
+ if (ip_version == 6) {
+ memcpy(qp->qp1_hdr.grh.source_gid.raw, sgid_attr->gid.raw,
+ sizeof(sgid_attr->gid));
+ memcpy(qp->qp1_hdr.grh.destination_gid.raw, ah->av.rgid,
+ sizeof(ah->av.rgid));
+ qp->qp1_hdr.grh.hop_limit = ah->av.hop_limit;
+
+ if (dev->xdev->priv.cm_dscp != XSC_DSCP_PCP_UNSET)
+ qp->qp1_hdr.grh.traffic_class =
+ (dev->xdev->priv.cm_dscp << 2) | ECN_CAPABLE_TRANSPORT;
+ else
+ qp->qp1_hdr.grh.traffic_class = ECN_CAPABLE_TRANSPORT;
+ }
+
+ if (ip_version == 4) {
+ if (dev->xdev->priv.cm_dscp != XSC_DSCP_PCP_UNSET)
+ qp->qp1_hdr.ip4.tos =
+ (dev->xdev->priv.cm_dscp << 2) | ECN_CAPABLE_TRANSPORT;
+ else
+ qp->qp1_hdr.ip4.tos = ECN_CAPABLE_TRANSPORT;
+ qp->qp1_hdr.ip4.id = 0;
+ qp->qp1_hdr.ip4.frag_off = htons(IP_DF);
+ qp->qp1_hdr.ip4.ttl = ah->av.hop_limit;
+
+ memcpy(&qp->qp1_hdr.ip4.saddr, sgid_attr->gid.raw + 12, 4);
+ memcpy(&qp->qp1_hdr.ip4.daddr, ah->av.rgid + 12, 4);
+ qp->qp1_hdr.ip4.check = ib_ud_ip4_csum(&qp->qp1_hdr);
+ }
+
+ qp->qp1_hdr.udp.dport = htons(dev->xdev->cm_udp_dst_port);
+ qp->qp1_hdr.udp.sport = htons(ah->av.udp_sport);
+ qp->qp1_hdr.udp.csum = 0;
+ ibdev_dbg(&dev->ib_dev, "CM packet used udp_sport=%d\n", ah->av.udp_sport);
+
+ /* BTH */
+ if (wr->opcode == IB_WR_SEND_WITH_IMM) {
+ qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
+ qp->qp1_hdr.immediate_present = 1;
+ } else {
+ qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY;
+ }
+ if (wr->send_flags & IB_SEND_SOLICITED)
+ qp->qp1_hdr.bth.solicited_event = 1;
+ /* pad_count */
+ qp->qp1_hdr.bth.pad_count = (4 - payload_size) & 3;
+
+ /* P_key for QP1 is for all members */
+ qp->qp1_hdr.bth.pkey = cpu_to_be16(0xFFFF);
+ qp->qp1_hdr.bth.destination_qpn = IB_QP1;
+ qp->qp1_hdr.bth.ack_req = 0;
+ qp->send_psn++;
+ qp->send_psn &= BTH_PSN_MASK;
+ qp->qp1_hdr.bth.psn = cpu_to_be32(qp->send_psn);
+ /* DETH */
+ /* Use the privileged Q_Key for QP1 */
+ qp->qp1_hdr.deth.qkey = cpu_to_be32(IB_QP1_QKEY);
+ qp->qp1_hdr.deth.source_qpn = IB_QP1;
+
+ /* Pack the QP1 to the transmit buffer */
+ sge->addr = (dma_addr_t)(qp->sq.hdr_dma + MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index);
+ sge->lkey = 0xFFFFFFFF;
+ sge->length = MAX_QP1_SQ_HDR_SIZE;
+
+ ib_ud_header_pack(&qp->qp1_hdr, hdr_buf);
+ /*
+ * Max Header buf size for IPV6 RoCE V2 is 86,
+ * which is same as the QP1 SQ header buffer.
+ * Header buf size for IPV4 RoCE V2 can be 66.
+ * ETH(14) + VLAN(4)+ IP(20) + UDP (8) + BTH(20).
+ * Subtract 20 bytes from QP1 SQ header buf size
+ */
+ if (ip_version == 4)
+ sge->length -= 20;
+
+ /* Subtract 4 bytes for non vlan packets */
+ if (!is_vlan)
+ sge->length -= 4;
+
+ rc = xsc_icrc_hdr(dev, hdr_buf, sge->length - sizeof(struct ib_unpacked_eth), crc);
+ if (rc) {
+ ibdev_err(&dev->ib_dev, "CRC error: hdr size %ld\n",
+ sge->length - sizeof(struct ib_unpacked_eth));
+ }
+ return rc;
+}
+
+static void zero_send_ds(struct xsc_ib_qp *qp, int idx)
+{
+ void *seg;
+ int i;
+ int ds_num;
+ u64 *p;
+
+ ds_num = XSC_SEND_SEG_NUM << (qp->sq.wqe_shift - XSC_SEND_WQE_SHIFT);
+ seg = (void *)xsc_get_send_wqe(qp, idx);
+ for (i = 1; i < ds_num; i++) {
+ p = get_seg_wqe(seg, i);
+ p[0] = 0;
+ p[1] = 0;
+ }
+}
+
+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
+ const struct ib_send_wr **bad_wr)
+{
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ void *seg;
+ struct xsc_send_wqe_ctrl_seg *ctrl;
+ struct xsc_wqe_data_seg *data_seg;
+ u32 crc;
+ int nreq = 0;
+ int err = 0;
+ int i;
+ unsigned int idx;
+ unsigned long irqflag = 0;
+ struct ib_sge sg;
+ u8 *cur_p = NULL;
+ u8 *mad_send_base = NULL;
+ void *vaddr;
+ struct xsc_msg_opcode *xsc_ib_opcode = dev->xdev->board_info->msg_opcode;
+
+ spin_lock_irqsave(&qp->sq.lock, irqflag);
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ unsigned int seg_index = 1;
+ unsigned int msg_len = 0;
+ u32 ds_data_num = 0;
+ u32 ctrl_val = 0;
+ bool ce;
+ struct ib_sge *sgl = &wr->sg_list[0];
+ int sg_n = wr->num_sge;
+
+ if (unlikely(wr->opcode == IB_WR_REG_MR ||
+ wr->opcode == IB_WR_LOCAL_INV ||
+ wr->opcode == IB_WR_SEND_WITH_INV)) {
+ ibdev_err(&dev->ib_dev, "opcode %d not supported\n",
+ wr->opcode);
+ err = -EOPNOTSUPP;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->opcode < 0 || wr->opcode >= ARRAY_SIZE(dev->opcode_map))) {
+ ibdev_err(&dev->ib_dev, "bad opcode %d\n", wr->opcode);
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(xsc_wq_overflow(&qp->sq, nreq))) {
+ ibdev_err(&dev->ib_dev, "send work queue overflow\n");
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > qp->sq.max_gs)) {
+ ibdev_err(&dev->ib_dev, "max gs exceeded %d (max = %d)\n",
+ wr->num_sge, qp->sq.max_gs);
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->opcode == IB_WR_RDMA_READ && wr->num_sge > 1)) {
+ ibdev_err(&dev->ib_dev, "rdma read, max gs exceeded %d (max = 1)\n",
+ wr->num_sge);
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(qp->post_send_invalid)) {
+ ibdev_dbg(&dev->ib_dev, "qp %d has been draind, post send invalid...\n",
+ qp->xqp.qpn);
+ err = -EIO;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ idx = qp->sq.cur_post & (qp->sq.wqe_cnt - 1);
+ zero_send_ds(qp, idx);
+ seg = xsc_get_send_wqe(qp, idx);
+ ctrl = seg;
+ xsc_set_wqe_id(dev->xdev, ctrl, qp->sq.cur_post <<
+ (qp->sq.wqe_shift - XSC_BASE_WQE_SHIFT));
+ ctrl->data0 &= cpu_to_le32(XSC_WQE_CTRL_SEG_DATA0_WQE_ID_MASK);
+ ce = (wr->send_flags & IB_SEND_SIGNALED) ||
+ (qp->sq_signal_bits & XSC_WQE_CTRL_CQ_UPDATE);
+ for (i = 0; i < wr->num_sge; ++i) {
+ if (likely(wr->sg_list[i].length))
+ msg_len += wr->sg_list[i].length;
+ }
+ ctrl->msg_len = msg_len;
+
+ switch (ibqp->qp_type) {
+ case IB_QPT_RC:
+ ds_data_num = wr->num_sge;
+ switch (wr->opcode) {
+ case IB_WR_SEND:
+ break;
+ case IB_WR_SEND_WITH_IMM:
+ ctrl_val |= XSC_WQE_CTRL_SEG_DATA0_WITH_IMM;
+ WR_LE_32(ctrl->data2, RD_BE_32(send_ieth(wr)));
+ break;
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ ctrl_val |= XSC_WQE_CTRL_SEG_DATA0_WITH_IMM;
+ WR_LE_32(ctrl->data2, RD_BE_32(send_ieth(wr)));
+ fallthrough;
+ case IB_WR_RDMA_READ:
+ case IB_WR_RDMA_WRITE:
+ if (msg_len == 0)
+ break;
+ ds_data_num++;
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ set_remote_addr_seg(dev->xdev, data_seg,
+ msg_len,
+ rdma_wr(wr)->remote_addr,
+ rdma_wr(wr)->rkey);
+ seg_index++;
+ break;
+ default:
+ ibdev_err(&dev->ib_dev, "unsupported send opcode %u\n",
+ wr->opcode);
+ err = -EPERM;
+ *bad_wr = wr;
+ goto out;
+ }
+ if (unlikely(wr->opcode == IB_WR_RDMA_READ && msg_len == 0)) {
+ ibdev_dbg(&dev->ib_dev, "post zero-length read wr\n");
+ /* HW rejects zero-length reads; poll backfills the
+ * READ opcode from qp->sq.zero_length_read[]
+ */
+ ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
+ dev->opcode_map[IB_WR_RDMA_WRITE]);
+ qp->sq.zero_length_read[idx] = 1;
+ } else {
+ ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
+ dev->opcode_map[wr->opcode]);
+ }
+ break;
+ case IB_QPT_UD:
+ case IB_QPT_GSI:
+ ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
+ xsc_ib_opcode->mad);
+ ds_data_num++;
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ mad_send_base = (u8 *)qp->sq.hdr_buf +
+ MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index;
+
+ err = build_qp1_send_v2(dev, qp, wr, &sg, msg_len, &crc);
+ if (err) {
+ *bad_wr = wr;
+ goto out;
+ }
+
+ cur_p = mad_send_base + sg.length;
+ for (i = 0; i < wr->num_sge; ++i) {
+ if (likely(wr->sg_list[i].length)) {
+ vaddr = xsc_ib_mad_sg_virt_addr(&dev->ib_dev,
+ wr->sg_list[i].addr);
+ if (unlikely(!vaddr)) {
+ ibdev_err(&dev->ib_dev,
+ "cannot map MAD sg entry %d to a virtual address\n",
+ i);
+ err = -EOPNOTSUPP;
+ *bad_wr = wr;
+ goto out;
+ }
+ memcpy(cur_p, vaddr, wr->sg_list[i].length);
+ }
+ cur_p += wr->sg_list[i].length;
+ }
+ crc = xsc_crc32(dev, crc, mad_send_base + sg.length, ctrl->msg_len);
+ ctrl->msg_len += sg.length;
+ seg_index++;
+
+ put_unaligned(~crc, (u32 *)&mad_send_base[ctrl->msg_len]);
+ ctrl->msg_len += sizeof(crc);
+ sg.length = ctrl->msg_len;
+ set_local_data_seg(dev->xdev, data_seg, &sg);
+ ibdev_dbg(&dev->ib_dev, "qp[%d] send MAD packet, msg_len:%d\n",
+ qp->xqp.qpn, ctrl->msg_len);
+ qp->sq.mad_index = (qp->sq.mad_index + 1) % MAD_QUEUE_DEPTH;
+
+ sg_n = 0;
+ break;
+ default:
+ ibdev_err(&dev->ib_dev, "qp type:%u NOT supported\n", ibqp->qp_type);
+ err = -EPERM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_DS_DATA_NUM_MASK, ds_data_num);
+ ctrl->data0 |= cpu_to_le32(ctrl_val);
+ ctrl->data3 = cpu_to_le32((wr->send_flags & IB_SEND_SOLICITED ?
+ XSC_WQE_CTRL_SEG_DATA3_SE : 0) |
+ (ce ? XSC_WQE_CTRL_SEG_DATA3_CE : 0));
+
+ if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
+ unsigned int segs_used;
+
+ err = set_data_inl_seg(qp, wr, ctrl, seg_index, &segs_used);
+ if (unlikely(err)) {
+ *bad_wr = wr;
+ ibdev_err(&dev->ib_dev, "inline layout failed, err %d\n", err);
+ goto out;
+ }
+ ds_data_num = segs_used - 1;
+ } else {
+ for (i = 0; i < sg_n; ++i, ++seg_index) {
+ if (likely(sgl[i].length)) {
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ set_local_data_seg(dev->xdev, data_seg, &sgl[i]);
+ }
+ }
+ }
+ qp->sq.wrid[idx] = wr->wr_id;
+ qp->sq.wqe_head[idx] = qp->sq.head + nreq;
+ qp->sq.cur_post += 1;
+ if (ce) {
+ atomic_inc(&qp->sq.flush_wqe_cnt);
+ qp->sq.need_flush[idx] = 1;
+ qp->sq.prev_ce_wqe[idx] = qp->sq.last_post_ce_wqe;
+ qp->sq.last_post_ce_wqe = idx;
+ }
+ qp->sq.wr_opcode[idx] = wr->opcode;
+ }
+out:
+ ibdev_dbg(&dev->ib_dev, "nreq:%d\n", nreq);
+ xsc_post_send_db(qp, dev->xdev, nreq);
+ spin_unlock_irqrestore(&qp->sq.lock, irqflag);
+
+ return err;
+}
int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
const struct ib_recv_wr **bad_wr)
@@ -1628,6 +2248,100 @@ int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_
struct xsc_ib_drain_cqe {
struct ib_cqe cqe;
struct completion done;
- int qpn;
};
+static void xsc_ib_drain_rq_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct xsc_ib_drain_cqe *cqe = container_of(wc->wr_cqe,
+ struct xsc_ib_drain_cqe, cqe);
+
+ complete(&cqe->done);
+}
+
+static void xsc_ib_drain_sq_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct xsc_ib_drain_cqe *cqe = container_of(wc->wr_cqe,
+ struct xsc_ib_drain_cqe, cqe);
+
+ complete(&cqe->done);
+}
+
+void xsc_ib_drain_rq(struct ib_qp *qp)
+{
+ struct ib_cq *cq = qp->recv_cq;
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+ struct xsc_ib_drain_cqe rdrain;
+ const struct ib_recv_wr rwr = {
+ .next = NULL,
+ {.wr_cqe = &rdrain.cqe,},
+ };
+ const struct ib_recv_wr *bad_rwr;
+ int ret;
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+
+ ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
+ return;
+ }
+
+ rdrain.cqe.done = xsc_ib_drain_rq_done;
+ init_completion(&rdrain.done);
+
+ ret = xsc_ib_post_recv(qp, &rwr, &bad_rwr);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
+ return;
+ }
+ xqp->post_recv_invalid = true;
+
+ if (cq->poll_ctx == IB_POLL_DIRECT)
+ while (wait_for_completion_timeout(&rdrain.done, HZ / 10) <= 0)
+ ib_process_cq_direct(cq, -1);
+ else
+ xcq->xcq.comp(&xcq->xcq);
+ wait_for_completion(&rdrain.done);
+}
+
+void xsc_ib_drain_sq(struct ib_qp *qp)
+{
+ struct ib_cq *cq = qp->send_cq;
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+ struct xsc_ib_drain_cqe sdrain;
+ const struct ib_send_wr *bad_swr;
+ const struct ib_send_wr swr = {
+ .next = NULL,
+ { .wr_cqe = &sdrain.cqe, },
+ .opcode = IB_WR_RDMA_WRITE,
+ .send_flags = IB_SEND_SIGNALED,
+ };
+ int ret;
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+
+ ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
+ return;
+ }
+
+ sdrain.cqe.done = xsc_ib_drain_sq_done;
+ init_completion(&sdrain.done);
+
+ ret = xsc_ib_post_send(qp, &swr, &bad_swr);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
+ return;
+ }
+ xqp->post_send_invalid = true;
+
+ if (cq->poll_ctx == IB_POLL_DIRECT)
+ while (wait_for_completion_timeout(&sdrain.done, HZ / 10) <= 0)
+ ib_process_cq_direct(cq, -1);
+ else
+ xcq->xcq.comp(&xcq->xcq);
+ wait_for_completion(&sdrain.done);
+}
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
index 126fcf789..fe111b2c6 100644
--- a/drivers/infiniband/hw/xsc/xsc_ib.h
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -10,7 +10,6 @@
#include <linux/kernel.h>
#include <linux/sched.h>
#include <rdma/ib_verbs.h>
-#include <rdma/ib_smi.h>
#include <rdma/ib_pack.h>
#include <rdma/ib_mad.h>
#include "common/xsc_core.h"
@@ -22,7 +21,6 @@
#include <linux/iommu.h>
#include <linux/dma-direct.h>
#include <linux/dma-mapping.h>
-#include <crypto/hash.h>
#include <rdma/uverbs_ioctl.h>
#define XSC_PAGE_SHIFT_4K 12
@@ -82,6 +80,7 @@ struct xsc_ib_wq {
u32 *wr_opcode;
u32 *need_flush;
atomic_t flush_wqe_cnt;
+ u8 *zero_length_read;
int *prev_ce_wqe;
int last_comp_ce_wqe;
int last_post_ce_wqe;
@@ -211,13 +210,17 @@ struct xsc_ib_mr {
u32 npages;
};
+#define MAX_OPCODE IB_WR_MASKED_ATOMIC_FETCH_AND_ADD
+
struct xsc_ib_dev {
struct ib_device ib_dev;
struct xsc_core_device *xdev;
int num_ports;
int num_comp_vectors;
u8 ib_active;
+ u32 crc_32_table[256];
int iommu_state;
+ u32 opcode_map[MAX_OPCODE + 1];
};
struct xsc_err_state_qp_node {
@@ -289,6 +292,8 @@ static inline struct xsc_ib_srq *core_to_xsrq(struct xsc_core_srq *xsrq)
return container_of(xsrq, struct xsc_ib_srq, xsrq);
}
+void build_opcode_map(struct xsc_ib_dev *dev);
+
int xsc_ib_create_ah(struct ib_ah *ibah,
struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
@@ -402,6 +407,23 @@ static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)
return 0;
}
+/*
+ * MMIO doorbells and MAD payloads are accessed by the CPU, so the driver
+ * needs the kernel virtual address behind a DMA address. There is no such
+ * conversion API, so the address space is classified once at probe time and
+ * the matching trick is used:
+ *
+ * NORMAL - an IOMMU DMA domain is attached; iova_to_phys() is available.
+ * DISABLE - no IOMMU translation, DMA address == physical address.
+ * UNKNOWN - address translation exists but is not owned by the IOMMU
+ * layer (forced software bounce buffering, or a hypervisor that
+ * translates DMA addresses), so neither trick works.
+ *
+ * UNKNOWN is not supported: MAD and doorbell traffic cannot be addressed by
+ * the CPU there, and QP1 (SMI/GSI) creation is refused as a result, which
+ * disables SA/CM on such a system. Systems with an IOMMU or without any
+ * translation are unaffected.
+ */
#define XSC_IB_IOMMU_MAP_DISABLE 0
#define XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN 1
#define XSC_IB_IOMMU_MAP_NORMAL 2
@@ -417,114 +439,36 @@ static inline void *xsc_ib_iova_to_virt(struct ib_device *ibdev, dma_addr_t iova
struct iommu_domain *domain;
domain = iommu_get_domain_for_dev(ibdev->dma_device);
- if (likely(domain)) {
- phyaddr = iommu_iova_to_phys(domain, iova);
- phyaddr |= iova & (PAGE_SIZE - 1);
- } else {
- phyaddr = dma_to_phys(ibdev->dma_device, iova);
- }
+ if (unlikely(!domain))
+ return NULL;
+
+ phyaddr = iommu_iova_to_phys(domain, iova);
+ if (unlikely(!phyaddr))
+ return NULL;
+ phyaddr |= iova & (PAGE_SIZE - 1);
return phys_to_virt(phyaddr);
}
-struct ib_mad_list_head {
- struct list_head list;
- struct ib_cqe cqe;
- struct ib_mad_queue *mad_queue;
-};
-
-#define IB_MAD_SEND_REQ_MAX_SG 2
-struct ib_mad_send_wr_private {
- struct ib_mad_list_head mad_list;
- struct list_head agent_list;
- struct ib_mad_agent_private *mad_agent_priv;
- struct ib_mad_send_buf send_buf;
- u64 header_mapping;
- u64 payload_mapping;
- struct ib_ud_wr send_wr;
- struct ib_sge sg_list[IB_MAD_SEND_REQ_MAX_SG];
- __be64 tid;
- unsigned long timeout;
- int max_retries;
- int retries_left;
- int retry;
- int refcount;
- enum ib_wc_status status;
-
- /* RMPP control */
- struct list_head rmpp_list;
- struct ib_rmpp_segment *last_ack_seg;
- struct ib_rmpp_segment *cur_seg;
- int last_ack;
- int seg_num;
- int newwin;
- int pad;
-};
-
-struct ib_mad_private_header {
- struct ib_mad_list_head mad_list;
- struct ib_mad_recv_wc recv_wc;
- struct ib_wc wc;
- u64 mapping;
-} __packed;
-
-struct ib_mad_private {
- struct ib_mad_private_header header;
- size_t mad_size;
- struct ib_grh grh;
- DECLARE_FLEX_ARRAY(u8, mad);
-} __packed;
-
-static inline void *xsc_ib_send_mad_sg_virt_addr(struct ib_device *ibdev,
- const struct ib_send_wr *wr,
- int sg)
+/*
+ * Translate the DMA address of a MAD scatter/gather entry to a kernel
+ * virtual address so that the payload can be copied to or from the WQE.
+ * Returns NULL when the address cannot be translated, which happens only
+ * for XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN (see above) or when the IOMMU has no
+ * mapping for the address.
+ */
+static inline void *xsc_ib_mad_sg_virt_addr(struct ib_device *ibdev,
+ dma_addr_t dma_addr)
{
- struct ib_mad_send_wr_private *mad_send_wr;
- struct ib_mad_list_head *mad_list;
int iommu_state = xsc_ib_iommu_dma_map(ibdev);
- /* direct dma mapping */
- if (!iommu_state)
- return phys_to_virt(dma_to_phys(ibdev->dma_device, wr->sg_list[sg].addr));
-
if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
- return xsc_ib_iova_to_virt(ibdev, wr->sg_list[sg].addr);
-
- mad_list = container_of(wr->wr_cqe, struct ib_mad_list_head, cqe);
- mad_send_wr = container_of(mad_list, struct ib_mad_send_wr_private,
- mad_list);
-
- /* sg_list[0] */
- if (sg == 0)
- return mad_send_wr->send_buf.mad;
-
- /* sg_list[1] */
- if (mad_send_wr->send_buf.seg_count)
- return ib_get_rmpp_segment(&mad_send_wr->send_buf,
- mad_send_wr->seg_num);
- return mad_send_wr->send_buf.mad + mad_send_wr->send_buf.hdr_len;
-}
+ return xsc_ib_iova_to_virt(ibdev, dma_addr);
-static inline void *xsc_ib_recv_mad_sg_virt_addr(struct ib_device *ibdev,
- struct ib_wc *wc,
- u64 sg_addr)
-{
- struct ib_mad_private_header *mad_priv_hdr;
- struct ib_mad_private *recv;
- struct ib_mad_list_head *mad_list;
- int iommu_state = xsc_ib_iommu_dma_map(ibdev);
-
- /* direct dma mapping */
- if (!iommu_state)
- return phys_to_virt(dma_to_phys(ibdev->dma_device, sg_addr));
-
- if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
- return xsc_ib_iova_to_virt(ibdev, sg_addr);
+ if (iommu_state == XSC_IB_IOMMU_MAP_DISABLE)
+ return phys_to_virt(dma_to_phys(ibdev->dma_device, dma_addr));
- mad_list = container_of(wc->wr_cqe, struct ib_mad_list_head, cqe);
- mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, mad_list);
- recv = container_of(mad_priv_hdr, struct ib_mad_private, header);
- return &recv->grh;
+ return NULL;
}
struct xsc_ib_burst_info {
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
index 0c91574d6..d58d75fb3 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
@@ -22,7 +22,7 @@
#include "common/xsc_hsi.h"
#include "common/srq.h"
-struct rxe_bth {
+struct xsc_bth {
u8 opcode;
u8 flags;
__be16 pkey;
@@ -30,7 +30,7 @@ struct rxe_bth {
__be32 apsn;
};
-struct rxe_deth {
+struct xsc_deth {
__be32 qkey;
__be32 src_qp;
};
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 14/15] RDMA/xsc: add the RDMA send path
2026-11-04 15:57 ` [PATCH v1 14/15] RDMA/xsc: add the RDMA send path Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 13 potential issue(s) to consider:
- [Critical] RDMA/xsc: stack OOB read in build_qp1_send_v2() from xsc_ib_drain_sq()
- [Critical] RDMA/xsc: hdr_buf DMA buffer overflow on MAD/UD send in xsc_ib_post_send()
- [High] RDMA/xsc: broken inline ds_data_num and zero-length SGEs in xsc_ib_post_send()
- [High] RDMA/xsc: completion deadlock and lost wakeup in xsc_ib_drain_sq() and rq
- [High] RDMA/xsc: stale zero_length_read state corrupts completions in xsc_poll_one()
- [High] RDMA/xsc: missing endian conversions on ctrl->msg_len and crc in xsc_ib_post_send()
- [High] RDMA/xsc: missing le64_to_cpu() on data_seg->va in xsc_handle_rdma_mad_resp_recv()
- [High] RDMA/xsc: unsynchronized WQ access in xsc_generate_flush_err_cqe() and post paths
- [Medium] RDMA/xsc: broken IB_WR_SEND_WITH_IMM header packing in build_qp1_send_v2()
- [Low] RDMA/xsc: duplicate crc32_le() implementation and inaccurate IOMMU comment
- [Low] RDMA/xsc: unbalanced flush_wqe_cnt for MAD sends in xsc_ib_post_send()
- [Low] RDMA/xsc: wrong format specifier %ld for size_t in build_qp1_send_v2()
- [Low] RDMA/xsc: comment in xsc_ib.h falsely claims QP1 creation is refused on UNKNOWN IOMMU
--
commit c77226a0e2a3291230f2f8d59b1b33bb5ef256ca
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add the RDMA send path
This commit implements the RDMA send path for the xsc driver, including
xsc_ib_post_send() for RC and QP1/UD traffic, zero-length RDMA read
handling, and the xsc_ib_drain_sq() and xsc_ib_drain_rq() helpers.
> diff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c
> index 36d122f8d46fb..a5ab8d00d9a6b 100644
> --- a/drivers/infiniband/hw/xsc/cq.c
> +++ b/drivers/infiniband/hw/xsc/cq.c
[ ... ]
> @@ -173,7 +173,13 @@ static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
> handle_responder(wc, cqe, *cur_qp, opcode);
>
> data_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);
> - recv = xsc_ib_recv_mad_sg_virt_addr(&dev->ib_dev, wc, data_seg->va);
> + recv = xsc_ib_mad_sg_virt_addr(&dev->ib_dev, data_seg->va);
[Severity: High]
In xsc_handle_rdma_mad_resp_recv(), data_seg->va is a __le64 field populated
in xsc_ib_post_recv() via WR_LE_64(data_seg->va, wr->sg_list[i].addr).
Should data_seg->va be converted with le64_to_cpu() before passing it as a
dma_addr_t to xsc_ib_mad_sg_virt_addr() so that the DMA address is not
byte-swapped on big-endian architectures?
> diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
> index 138ca1a80083b..0574687982c24 100644
> --- a/drivers/infiniband/hw/xsc/main.c
> +++ b/drivers/infiniband/hw/xsc/main.c
> @@ -474,6 +474,22 @@ static int init_node_data(struct xsc_ib_dev *dev)
> return 0;
> }
>
> +static void crc_table_init(struct xsc_ib_dev *dev)
> +{
> + u32 c, i, j;
> +
> + for (i = 0; i < 256; i++) {
> + c = i;
> + for (j = 0; j < 8; j++) {
> + if (c & 1)
> + c = 0xedb88320L ^ (c >> 1);
> + else
> + c = c >> 1;
> + }
> + dev->crc_32_table[i] = c;
> + }
> +}
[Severity: Low]
This isn't a bug, but drivers/infiniband/hw/xsc/qp.c already includes
linux/crc32.h, which provides crc32_le() using the same 0xedb88320 polynomial.
Could xsc_icrc_hdr() and xsc_ib_post_send() use crc32_le() directly instead of
maintaining dev->crc_32_table[], crc_table_init(), and xsc_crc32()?
> diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
> index 0dfb102949070..46ca3dd95df35 100644
> --- a/drivers/infiniband/hw/xsc/qp.c
> +++ b/drivers/infiniband/hw/xsc/qp.c
[ ... ]
> @@ -1437,13 +1532,538 @@ static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq)
> return cur + nreq >= wq->max_post;
> }
>
> +static inline void xsc_post_send_db(struct xsc_ib_qp *qp,
> + struct xsc_core_device *xdev,
> + int nreq)
> +{
> + u32 next_pid;
> +
> + if (unlikely(!nreq))
> + return;
> +
> + qp->sq.head += nreq;
> +
> + next_pid = qp->sq.head << (qp->sq.wqe_shift - XSC_BASE_WQE_SHIFT);
> + xsc_update_tx_db(xdev, qp->doorbell_qpn, next_pid);
> +}
[Severity: High]
Can xsc_post_send_db() and xsc_ib_post_recv() race with
xsc_generate_flush_err_cqe() during QP drain or error transitions?
xsc_ib_post_send() holds only qp->sq.lock while populating qp->sq.wrid[idx],
qp->sq.wqe_head[idx], qp->sq.need_flush[idx], and qp->sq.wr_opcode[idx], and
then xsc_post_send_db() advances qp->sq.head before the barrier in
xsc_update_tx_db(). Similarly, xsc_ib_post_recv() holds only qp->rq.lock.
Meanwhile, in drivers/infiniband/hw/xsc/cq.c, xsc_ib_poll_cq() calls
xsc_generate_flush_err_cqe() while holding only cq->lock and
cq->err_state_qp_lock:
drivers/infiniband/hw/xsc/cq.c:xsc_generate_flush_err_cqe() {
...
while (wq->tail != wq->head) {
idx = wq->tail & (wq->wqe_cnt - 1);
if (err_qp_node->is_sq && !wq->need_flush[idx]) {
wq->tail++;
continue;
} else {
gen_flush_err_cqe(err_qp_node, &(core_to_xqp(xqp)->ibqp), wq,
idx, wc + *npolled + sw_npolled);
++sw_npolled;
}
...
}
When xsc_ib_drain_sq() or xsc_ib_drain_rq() moves the QP to IB_QPS_ERR first
and then posts the drain WR, could xsc_generate_flush_err_cqe() on another CPU
observe the updated wq->head alongside a stale wq->need_flush[idx] or
wq->wrid[idx], skipping the drain WR or invoking a stale wr_cqe callback?
[ ... ]
> +/* Routine for sending QP1 packets for RoCE V2 */
> +static int build_qp1_send_v2(struct xsc_ib_dev *dev,
> + struct xsc_ib_qp *qp,
> + const struct ib_send_wr *wr,
> + struct ib_sge *sge,
> + int payload_size, u32 *crc)
> +{
> + struct xsc_ib_ah *ah = container_of(ud_wr((struct ib_send_wr *)wr)->ah, struct xsc_ib_ah,
> + ibah);
> + const struct ib_gid_attr *sgid_attr = ah->ibah.sgid_attr;
[Severity: Critical]
Does draining a GSI or UD send queue via xsc_ib_drain_sq() read past the end
of the stack-allocated work request in build_qp1_send_v2()?
In xsc_ib_drain_sq(), swr is declared on the stack as a base struct ib_send_wr
with .opcode = IB_WR_RDMA_WRITE:
xsc_ib_drain_sq()
-> xsc_ib_post_send(qp, &swr, &bad_swr)
-> case IB_QPT_UD / IB_QPT_GSI:
-> build_qp1_send_v2(dev, qp, wr, &sg, msg_len, &crc)
Because xsc_ib_post_send() does not check for drain WRs or validate wr->opcode
in the IB_QPT_UD and IB_QPT_GSI cases, build_qp1_send_v2() casts &swr via
ud_wr(wr)->ah, reading past the end of struct ib_send_wr on the stack and
dereferencing ah->ibah.sgid_attr.
Also, for IB_QPT_SMI or IB_QPT_RAW_PACKET QPs, doesn't xsc_ib_post_send() hit
the default branch and fail xsc_ib_drain_sq() with -EPERM?
[ ... ]
> + is_eth = true;
> + is_vlan = (vlan_id && (vlan_id < 0x1000)) ? true : false;
> +
> + ib_ud_header_init(payload_size, !is_eth, is_eth, is_vlan, false,
> + ip_version, true, 0, &qp->qp1_hdr);
[ ... ]
> + /* BTH */
> + if (wr->opcode == IB_WR_SEND_WITH_IMM) {
> + qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
> + qp->qp1_hdr.immediate_present = 1;
> + } else {
> + qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY;
> + }
[Severity: Medium]
When wr->opcode == IB_WR_SEND_WITH_IMM in build_qp1_send_v2(),
ib_ud_header_init() is called with immediate_present = 0, and
qp->qp1_hdr.immediate_present is later set to 1 without setting
qp->qp1_hdr.immediate_data or increasing sge->length by 4 bytes.
Doesn't ib_ud_header_pack() then write 4 uninitialized bytes of immediate_data
at the end of the header that are excluded from the IP/UDP lengths and
sge->length, and then overwritten when xsc_ib_post_send() copies the payload
to mad_send_base + sg.length?
[ ... ]
> + rc = xsc_icrc_hdr(dev, hdr_buf, sge->length - sizeof(struct ib_unpacked_eth), crc);
> + if (rc) {
> + ibdev_err(&dev->ib_dev, "CRC error: hdr size %ld\n",
> + sge->length - sizeof(struct ib_unpacked_eth));
> + }
> + return rc;
> +}
[Severity: Low]
In build_qp1_send_v2(), sge->length - sizeof(struct ib_unpacked_eth) has type
size_t. Should the format specifier be %zu instead of %ld to avoid -Wformat
warnings on 32-bit COMPILE_TEST builds?
[ ... ]
> + for (i = 0; i < wr->num_sge; ++i) {
> + if (likely(wr->sg_list[i].length))
> + msg_len += wr->sg_list[i].length;
> + }
> + ctrl->msg_len = msg_len;
[Severity: High]
In struct xsc_send_wqe_ctrl_seg, ctrl->msg_len is declared as __le32.
In xsc_ib_post_send(), msg_len is assigned to ctrl->msg_len without
cpu_to_le32(), and in the IB_QPT_UD / IB_QPT_GSI branch below:
crc = xsc_crc32(dev, crc, mad_send_base + sg.length, ctrl->msg_len);
ctrl->msg_len += sg.length;
seg_index++;
put_unaligned(~crc, (u32 *)&mad_send_base[ctrl->msg_len]);
ctrl->msg_len += sizeof(crc);
sg.length = ctrl->msg_len;
ctrl->msg_len is used directly in host-endian arithmetic and array indexing,
and ~crc is written in host byte order with put_unaligned() instead of
put_unaligned_le32(). Won't this corrupt the WQE length, buffer offset, and
ICRC on big-endian systems?
[ ... ]
> + if (unlikely(wr->opcode == IB_WR_RDMA_READ && msg_len == 0)) {
> + ibdev_dbg(&dev->ib_dev, "post zero-length read wr\n");
> + /* HW rejects zero-length reads; poll backfills the
> + * READ opcode from qp->sq.zero_length_read[]
> + */
> + ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
> + dev->opcode_map[IB_WR_RDMA_WRITE]);
> + qp->sq.zero_length_read[idx] = 1;
> + } else {
> + ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
> + dev->opcode_map[wr->opcode]);
> + }
> + break;
[Severity: High]
Can qp->sq.zero_length_read[idx] remain set across WQE slot reuse and corrupt
a later completion in xsc_poll_one()?
In xsc_ib_post_send(), qp->sq.zero_length_read[idx] is set to 1 for any
zero-length IB_WR_RDMA_READ, and the else branch does not clear it to 0. In
xsc_poll_one(), wq->zero_length_read[idx] is only cleared when a normal or
error hardware CQE is polled for that idx.
If a zero-length RDMA read is posted unsignaled (!ce), or if it is completed
via xsc_generate_lost_cqe() or software flush in gen_flush_err_cqe(), or if
set_data_inl_seg() fails later in xsc_ib_post_send(), zero_length_read[idx]
stays 1 and will overwrite the completion opcode of a subsequent signaled SEND
or WRITE that reuses slot idx.
> + case IB_QPT_UD:
> + case IB_QPT_GSI:
> + ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_MSG_OPCODE_MASK,
> + xsc_ib_opcode->mad);
> + ds_data_num++;
> + data_seg = get_seg_wqe(ctrl, seg_index);
> + mad_send_base = (u8 *)qp->sq.hdr_buf +
> + MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index;
> +
> + err = build_qp1_send_v2(dev, qp, wr, &sg, msg_len, &crc);
> + if (err) {
> + *bad_wr = wr;
> + goto out;
> + }
> +
> + cur_p = mad_send_base + sg.length;
> + for (i = 0; i < wr->num_sge; ++i) {
> + if (likely(wr->sg_list[i].length)) {
> + vaddr = xsc_ib_mad_sg_virt_addr(&dev->ib_dev,
> + wr->sg_list[i].addr);
> + if (unlikely(!vaddr)) {
> + ibdev_err(&dev->ib_dev,
> + "cannot map MAD sg entry %d to a virtual address\n",
> + i);
> + err = -EOPNOTSUPP;
> + *bad_wr = wr;
> + goto out;
> + }
> + memcpy(cur_p, vaddr, wr->sg_list[i].length);
> + }
> + cur_p += wr->sg_list[i].length;
> + }
> + crc = xsc_crc32(dev, crc, mad_send_base + sg.length, ctrl->msg_len);
> + ctrl->msg_len += sg.length;
> + seg_index++;
> +
> + put_unaligned(~crc, (u32 *)&mad_send_base[ctrl->msg_len]);
[Severity: Critical]
Can this memcpy() and put_unaligned() overflow the 512-byte slot in
qp->sq.hdr_buf?
In create_kernel_qp(), qp->sq.hdr_buf is allocated with MAD_QUEUE_DEPTH slots
of MAX_QP1_SQ_HDR_SIZE_V2 (512) bytes each, while xsc_port_immutable() sets
immutable->max_mad_size = IB_MGMT_MAD_SIZE * 2 (512 bytes).
In xsc_ib_post_send(), build_qp1_send_v2() writes up to 86 bytes of RoCEv2
headers into mad_send_base, and then xsc_ib_post_send() copies msg_len bytes
of SGE payload plus 4 bytes of ICRC immediately after sg.length without
checking that sg.length + msg_len + sizeof(crc) fits within
MAX_QP1_SQ_HDR_SIZE_V2. A 512-byte MAD (or any UD payload over 422 bytes) will
write up to 602 bytes into the 512-byte slot and past the end of
qp->sq.hdr_buf on the last slot.
[ ... ]
> + ctrl_val |= FIELD_PREP(XSC_WQE_CTRL_SEG_DATA0_DS_DATA_NUM_MASK, ds_data_num);
> + ctrl->data0 |= cpu_to_le32(ctrl_val);
> + ctrl->data3 = cpu_to_le32((wr->send_flags & IB_SEND_SOLICITED ?
> + XSC_WQE_CTRL_SEG_DATA3_SE : 0) |
> + (ce ? XSC_WQE_CTRL_SEG_DATA3_CE : 0));
> +
> + if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
> + unsigned int segs_used;
> +
> + err = set_data_inl_seg(qp, wr, ctrl, seg_index, &segs_used);
> + if (unlikely(err)) {
> + *bad_wr = wr;
> + ibdev_err(&dev->ib_dev, "inline layout failed, err %d\n", err);
> + goto out;
> + }
> + ds_data_num = segs_used - 1;
> + } else {
> + for (i = 0; i < sg_n; ++i, ++seg_index) {
> + if (likely(sgl[i].length)) {
> + data_seg = get_seg_wqe(ctrl, seg_index);
> + set_local_data_seg(dev->xdev, data_seg, &sgl[i]);
> + }
> + }
> + }
[Severity: High]
There appear to be three issues with how ds_data_num and the data segments are
populated here in xsc_ib_post_send():
1. ctrl->data0 is written with ds_data_num before the IB_SEND_INLINE block
runs, so ds_data_num = segs_used - 1 after set_data_inl_seg() is never
written to ctrl->data0.
2. For IB_QPT_UD and IB_QPT_GSI, sg_n is set to 0 after copying the payload
into qp->sq.hdr_buf, but the inline check tests wr->num_sge instead of
sg_n, which would cause set_data_inl_seg() to append extra inline segments
on UD/GSI QPs.
3. In the non-inline else loop, seg_index is incremented on every iteration
even when sgl[i].length == 0 and set_local_data_seg() is skipped, while
ds_data_num still includes zero-length SGEs, leaving zeroed segment holes
in the WQE.
> + qp->sq.wrid[idx] = wr->wr_id;
> + qp->sq.wqe_head[idx] = qp->sq.head + nreq;
> + qp->sq.cur_post += 1;
> + if (ce) {
> + atomic_inc(&qp->sq.flush_wqe_cnt);
> + qp->sq.need_flush[idx] = 1;
> + qp->sq.prev_ce_wqe[idx] = qp->sq.last_post_ce_wqe;
> + qp->sq.last_post_ce_wqe = idx;
> + }
> + qp->sq.wr_opcode[idx] = wr->opcode;
[Severity: Low]
When ce is true in xsc_ib_post_send(), qp->sq.flush_wqe_cnt is incremented for
all QP types including IB_QPT_GSI, but in drivers/infiniband/hw/xsc/cq.c,
xsc_poll_one() skips decrementing it when a MAD send completes:
drivers/infiniband/hw/xsc/cq.c:xsc_poll_one() {
...
if (opcode != XSC_OPCODE_RDMA_MAD_REQ_SEND)
atomic_dec(&wq->flush_wqe_cnt);
...
}
Doesn't this cause qp->sq.flush_wqe_cnt to grow on every signaled MAD send?
[ ... ]
> +void xsc_ib_drain_rq(struct ib_qp *qp)
> +{
> + struct ib_cq *cq = qp->recv_cq;
> + struct xsc_ib_cq *xcq = to_xcq(cq);
> + struct xsc_ib_qp *xqp = to_xqp(qp);
> + struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
> + struct xsc_ib_drain_cqe rdrain;
> + const struct ib_recv_wr rwr = {
> + .next = NULL,
> + {.wr_cqe = &rdrain.cqe,},
> + };
> + const struct ib_recv_wr *bad_rwr;
> + int ret;
> + struct xsc_ib_dev *dev = to_xdev(qp->device);
> +
> + ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
> + if (ret) {
> + ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
> + return;
> + }
> +
> + rdrain.cqe.done = xsc_ib_drain_rq_done;
> + init_completion(&rdrain.done);
> +
> + ret = xsc_ib_post_recv(qp, &rwr, &bad_rwr);
> + if (ret) {
> + ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
> + return;
> + }
> + xqp->post_recv_invalid = true;
> +
> + if (cq->poll_ctx == IB_POLL_DIRECT)
> + while (wait_for_completion_timeout(&rdrain.done, HZ / 10) <= 0)
> + ib_process_cq_direct(cq, -1);
> + else
> + xcq->xcq.comp(&xcq->xcq);
> + wait_for_completion(&rdrain.done);
> +}
> +
> +void xsc_ib_drain_sq(struct ib_qp *qp)
> +{
> + struct ib_cq *cq = qp->send_cq;
> + struct xsc_ib_cq *xcq = to_xcq(cq);
> + struct xsc_ib_qp *xqp = to_xqp(qp);
> + struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
> + struct xsc_ib_drain_cqe sdrain;
> + const struct ib_send_wr *bad_swr;
> + const struct ib_send_wr swr = {
> + .next = NULL,
> + { .wr_cqe = &sdrain.cqe, },
> + .opcode = IB_WR_RDMA_WRITE,
> + .send_flags = IB_SEND_SIGNALED,
> + };
> + int ret;
> + struct xsc_ib_dev *dev = to_xdev(qp->device);
> +
> + ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
> + if (ret) {
> + ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
> + return;
> + }
> +
> + sdrain.cqe.done = xsc_ib_drain_sq_done;
> + init_completion(&sdrain.done);
> +
> + ret = xsc_ib_post_send(qp, &swr, &bad_swr);
> + if (ret) {
> + ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
> + return;
> + }
> + xqp->post_send_invalid = true;
> +
> + if (cq->poll_ctx == IB_POLL_DIRECT)
> + while (wait_for_completion_timeout(&sdrain.done, HZ / 10) <= 0)
> + ib_process_cq_direct(cq, -1);
> + else
> + xcq->xcq.comp(&xcq->xcq);
> + wait_for_completion(&sdrain.done);
> +}
[Severity: High]
Can xsc_ib_drain_rq() and xsc_ib_drain_sq() hang when waiting for the drain
completion?
1. When cq->poll_ctx == IB_POLL_DIRECT, wait_for_completion_timeout() in the
while loop condition consumes rdrain.done / sdrain.done when it returns a
positive value, and then the unconditional wait_for_completion() after the
if/else block waits a second time on the already-consumed completion.
Additionally, ib_modify_qp() transitions the QP to IB_QPS_ERR, which calls
xsc_ib_trigger_cq_comp_handler() and triggers the WARN_ONCE() in
ib_cq_completion_direct() for IB_POLL_DIRECT CQs.
2. For IB_POLL_SOFTIRQ CQs, ib_modify_qp() schedules cq->iop before the drain
WR is posted. If ib_poll_handler() finishes polling before the drain WR is
posted, and xcq->xcq.comp(&xcq->xcq) runs before ib_poll_handler() calls
irq_poll_complete(), irq_poll_sched() is a no-op while IRQ_POLL_F_SCHED is
still set. When ib_poll_handler() then calls ib_req_notify_cq() ->
xsc_ib_arm_cq(), xsc_cqe_is_empty() only checks hardware CQEs via
get_sw_cqe() and ignores software flush entries on cq->err_state_qp_list,
so cq->iop is never rescheduled and wait_for_completion() hangs.
3. If the SQ or RQ is already full when drain is called, xsc_wq_overflow() in
xsc_ib_post_send() / xsc_ib_post_recv() will reject the drain WR with
-ENOMEM and return early without draining the queue.
> diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
> index 126fcf7892a12..fe111b2c61370 100644
> --- a/drivers/infiniband/hw/xsc/xsc_ib.h
> +++ b/drivers/infiniband/hw/xsc/xsc_ib.h
[ ... ]
> @@ -402,6 +407,23 @@ static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)
> return 0;
> }
>
> +/*
> + * MMIO doorbells and MAD payloads are accessed by the CPU, so the driver
> + * needs the kernel virtual address behind a DMA address. There is no such
> + * conversion API, so the address space is classified once at probe time and
> + * the matching trick is used:
> + *
> + * NORMAL - an IOMMU DMA domain is attached; iova_to_phys() is available.
> + * DISABLE - no IOMMU translation, DMA address == physical address.
> + * UNKNOWN - address translation exists but is not owned by the IOMMU
> + * layer (forced software bounce buffering, or a hypervisor that
> + * translates DMA addresses), so neither trick works.
> + *
> + * UNKNOWN is not supported: MAD and doorbell traffic cannot be addressed by
> + * the CPU there, and QP1 (SMI/GSI) creation is refused as a result, which
> + * disables SA/CM on such a system. Systems with an IOMMU or without any
> + * translation are unaffected.
> + */
[Severity: Low]
This comment states that QP1 (SMI/GSI) creation is refused when the IOMMU
state is UNKNOWN, but xsc_ib_create_qp() in drivers/infiniband/hw/xsc/qp.c
does not check dev->iommu_state and instead allows QP1 creation to succeed,
failing later in xsc_ib_post_send() and xsc_handle_rdma_mad_resp_recv().
Should either xsc_ib_create_qp() reject IB_QPT_GSI when dev->iommu_state ==
XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN, or this comment be updated to match the code?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=14
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v1 15/15] RDMA/xsc: add RDMA counters and sysfs
2026-11-04 15:57 [PATCH v1 00/15] Yunsilicon XSC ethernet and RDMA drivers Xin Tian
` (13 preceding siblings ...)
2026-11-04 15:57 ` [PATCH v1 14/15] RDMA/xsc: add the RDMA send path Xin Tian
@ 2026-11-04 15:57 ` Xin Tian
2026-11-05 16:09 ` sashiko-bot
14 siblings, 1 reply; 30+ messages in thread
From: Xin Tian @ 2026-11-04 15:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jason Gunthorpe, Leon Romanovsky, Junxian Huang
Cc: weihg, linux-kernel, netdev, linux-rdma
Add the rdma hw_stats implementation and two driver sysfs
attributes.
get_hw_stats reads the per-function RDMA statistics from firmware
and converts the big-endian fields into the registered counter set
using offsets recorded in the descriptor table. The set covers
TX/RX packet and byte counts, RoCE congestion control counters
(ECN marks, CNP sent and handled, CNP ignores) and per-function
error counters such as out-of-buffer, RNR NAK retries and local ACK
timeouts. Six card-global counters (loopback traffic, encapsulated
ICRC, completion and message code errors) come from a separate
firmware query and are only registered on PF 0, since the global
statistics are valid only there.
Expose hca_type, reporting the PCI device ID, and hw_rev, composed
from the chip version and hotfix number, as read-only attributes on
the ib_device class device, attached through the standard
device_group so the driver core manages their lifetime.
Signed-off-by: Xin Tian <tianx@yunsilicon.com>
---
drivers/infiniband/hw/xsc/Makefile | 3 +-
drivers/infiniband/hw/xsc/counters.c | 134 ++++++++++++++++++
drivers/infiniband/hw/xsc/main.c | 4 +-
drivers/infiniband/hw/xsc/xsc_ib.h | 3 +
drivers/infiniband/hw/xsc/xsc_ib_sysfs.c | 50 +++++++
.../ethernet/yunsilicon/xsc/common/xsc_core.h | 1 +
6 files changed, 193 insertions(+), 2 deletions(-)
create mode 100644 drivers/infiniband/hw/xsc/counters.c
create mode 100644 drivers/infiniband/hw/xsc/xsc_ib_sysfs.c
diff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile
index d5ad0e573..48a6d2b0a 100644
--- a/drivers/infiniband/hw/xsc/Makefile
+++ b/drivers/infiniband/hw/xsc/Makefile
@@ -6,4 +6,5 @@ ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o
-xsc_ib-y := main.o cq.o qp.o mem.o mr.o srq.o ah.o
+xsc_ib-y := main.o cq.o qp.o mem.o mr.o ah.o counters.o \
+ xsc_ib_sysfs.o srq.o
diff --git a/drivers/infiniband/hw/xsc/counters.c b/drivers/infiniband/hw/xsc/counters.c
new file mode 100644
index 000000000..c29a5bf80
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/counters.c
@@ -0,0 +1,134 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+#include "xsc_ib.h"
+
+/*
+ * Per-function/per-mac-port counters reported by
+ * XSC_CMD_OP_QUERY_HW_STATS_RDMA. .priv carries the byte offset of the
+ * counter inside the firmware statistics payload.
+ */
+#define XSC_HW_STAT(type, fld) \
+ { .name = #fld, .priv = (const void *)(uintptr_t)offsetof(type, fld) }
+
+/*
+ * Card-global counters from XSC_CMD_OP_QUERY_HW_GLOBAL_STATS. They are
+ * only valid on PF 0, so the name is prefixed to make that visible to
+ * whoever reads the counters.
+ */
+#define XSC_HW_GLOBAL_STAT(type, fld) \
+ { .name = "global_" #fld, \
+ .priv = (const void *)(uintptr_t)offsetof(type, fld) }
+
+#define XSC_NUM_PF_STATS 22
+
+static const struct rdma_stat_desc xsc_rdma_pf_stats_descs[] = {
+ /* by mac port */
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_bytes),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_bytes),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, np_cnp_sent),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_handled),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, np_ecn_marked_roce_packets),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_ignored),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, read_rsp_out_of_seq),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, implied_nak_seq_err),
+ /* by function */
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, out_of_sequence),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, packet_seq_err),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, out_of_buffer),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rnr_nak_retry_err),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, local_ack_timeout_err),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rx_read_requests),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rx_write_requests),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, duplicate_requests),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts_func),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_payload_bytes),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts_func),
+ XSC_HW_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_payload_bytes),
+ /*
+ * The PF payload also carries rdma_loopback_[pkts,bytes], but they
+ * are the same registers as the card-global ones. Expose them once,
+ * from the global query.
+ */
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_pkts),
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_bytes),
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, rx_icrc_encapsulated),
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, req_cqe_error),
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, resp_cqe_error),
+ XSC_HW_GLOBAL_STAT(struct xsc_hw_global_stats_rdma, cqe_msg_code_error),
+};
+
+static int xsc_ib_query_hw_stats_rdma(struct xsc_core_device *xdev,
+ struct xsc_hw_stats_rdma *stats_rdma)
+{
+ struct xsc_hw_stats_mbox_in in = {};
+ struct xsc_hw_stats_rdma_mbox_out out = {};
+ int ret;
+
+ in.mac_port = xdev->mac_port;
+ in.hdr.ver = cpu_to_be16(XSC_CMDQ_GET_HW_STATS_RDMA_V1);
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_STATS_RDMA);
+
+ ret = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (ret || out.hdr.status)
+ return -EIO;
+
+ memcpy(stats_rdma, &out.hw_stats, sizeof(*stats_rdma));
+ return 0;
+}
+
+struct rdma_hw_stats *xsc_ib_alloc_hw_device_stats(struct ib_device *ib_dev)
+{
+ struct xsc_core_device *xdev = to_xdev(ib_dev)->xdev;
+ int num;
+
+ num = xdev->pf_id == 0 ? ARRAY_SIZE(xsc_rdma_pf_stats_descs)
+ : XSC_NUM_PF_STATS;
+
+ return rdma_alloc_hw_stats_struct(xsc_rdma_pf_stats_descs, num,
+ RDMA_HW_STATS_DEFAULT_LIFESPAN);
+}
+
+int xsc_ib_get_hw_stats(struct ib_device *ib_dev, struct rdma_hw_stats *stats,
+ u32 port_num, int index)
+{
+ struct xsc_core_device *xdev = to_xdev(ib_dev)->xdev;
+ struct xsc_hw_stats_rdma stats_rdma;
+ u8 *hw_stats;
+ int ret;
+ int i;
+
+ ret = xsc_ib_query_hw_stats_rdma(xdev, &stats_rdma);
+ if (ret)
+ return ret;
+
+ hw_stats = (u8 *)&stats_rdma.stats.pf_stats;
+
+ for (i = 0; i < stats->num_counters && i < XSC_NUM_PF_STATS; i++)
+ stats->value[i] =
+ be64_to_cpu(*(u64 *)(hw_stats + (uintptr_t)stats->descs[i].priv));
+
+ if (stats->num_counters > XSC_NUM_PF_STATS) {
+ struct xsc_hw_global_stats_mbox_in gin = {};
+ struct xsc_hw_global_stats_mbox_out gout = {};
+
+ gin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_GLOBAL_STATS);
+ ret = xsc_cmd_exec(xdev, &gin, sizeof(gin), &gout, sizeof(gout));
+ if (ret || gout.hdr.status)
+ return -EIO;
+
+ hw_stats = (u8 *)&gout.hw_stats;
+ for (i = XSC_NUM_PF_STATS; i < stats->num_counters; i++)
+ stats->value[i] = be64_to_cpu(
+ *(u64 *)(hw_stats + (uintptr_t)stats->descs[i].priv));
+ }
+
+ return stats->num_counters;
+}
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
index 057468798..0690b3e2f 100644
--- a/drivers/infiniband/hw/xsc/main.c
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -552,7 +552,10 @@ static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
dev->ib_dev.ops.drain_sq = xsc_ib_drain_sq;
dev->ib_dev.ops.drain_rq = xsc_ib_drain_rq;
+ dev->ib_dev.ops.device_group = xsc_ib_attr_group();
dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
+ dev->ib_dev.ops.alloc_hw_device_stats = xsc_ib_alloc_hw_device_stats;
+ dev->ib_dev.ops.get_hw_stats = xsc_ib_get_hw_stats;
dev->ib_dev.ops.create_srq = xsc_ib_create_srq;
dev->ib_dev.ops.destroy_srq = xsc_ib_destroy_srq;
dev->ib_dev.ops.query_srq = xsc_ib_query_srq;
@@ -712,7 +715,6 @@ static int init_one(struct xsc_core_device *xdev,
dev->ib_active = true;
*ibdev = dev;
-
return 0;
err_free:
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
index fe111b2c6..30cafad88 100644
--- a/drivers/infiniband/hw/xsc/xsc_ib.h
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -306,6 +306,9 @@ int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata);
int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata);
void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);
+struct rdma_hw_stats *xsc_ib_alloc_hw_device_stats(struct ib_device *ib_dev);
+int xsc_ib_get_hw_stats(struct ib_device *ib_dev, struct rdma_hw_stats *stats,
+ u32 port_num, int index);
int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata);
int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata);
diff --git a/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c
new file mode 100644
index 000000000..c8d199a0e
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/sysfs.h>
+#include <linux/types.h>
+#include <linux/time.h>
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+#include "common/driver.h"
+#include "common/xsc_cmd.h"
+#include "xsc_ib.h"
+
+static ssize_t hca_type_show(struct device *device, struct device_attribute *attr, char *buf)
+{
+ struct xsc_ib_dev *dev = rdma_device_to_drv_device(device, struct xsc_ib_dev, ib_dev);
+
+ return sysfs_emit(buf, "%x\n", dev->xdev->pdev->device);
+}
+
+static DEVICE_ATTR_RO(hca_type);
+
+static ssize_t hw_rev_show(struct device *device, struct device_attribute *attr, char *buf)
+{
+ struct xsc_ib_dev *dev = rdma_device_to_drv_device(device, struct xsc_ib_dev, ib_dev);
+ u32 hw_ver;
+
+ hw_ver = ((dev->xdev->chip_ver_l & 0xffff) << 16) |
+ (dev->xdev->hotfix_num & 0xffff);
+ return sysfs_emit(buf, "0x%x\n", hw_ver);
+}
+
+static DEVICE_ATTR_RO(hw_rev);
+
+static struct attribute *xsc_class_attributes[] = {
+ &dev_attr_hca_type.attr,
+ &dev_attr_hw_rev.attr,
+ NULL,
+};
+
+static const struct attribute_group xsc_attr_group = {
+ .attrs = xsc_class_attributes,
+};
+
+const struct attribute_group *xsc_ib_attr_group(void)
+{
+ return &xsc_attr_group;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
index ea536389e..dfffde442 100644
--- a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
@@ -631,6 +631,7 @@ static inline bool is_support_multi_doorbell(struct xsc_core_device *dev)
}
void xsc_remove_rdma_driver(void);
+const struct attribute_group *xsc_ib_attr_group(void);
void xsc_init_hal(struct xsc_core_device *xdev);
--
2.18.4
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v1 15/15] RDMA/xsc: add RDMA counters and sysfs
2026-11-04 15:57 ` [PATCH v1 15/15] RDMA/xsc: add RDMA counters and sysfs Xin Tian
@ 2026-11-05 16:09 ` sashiko-bot
0 siblings, 0 replies; 30+ messages in thread
From: sashiko-bot @ 2026-11-05 16:09 UTC (permalink / raw)
To: Xin Tian; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] RDMA/xsc: cast counter pointers to __be64 * in xsc_ib_get_hw_stats()
--
commit 9a3d01fb1064f09ed97cbe2bb05c71e3ed77198c
Author: Xin Tian <tianx@yunsilicon.com>
RDMA/xsc: add RDMA counters and sysfs
This commit adds the RDMA hw_stats callbacks to query per-function and
card-global statistics from firmware, and exposes the hca_type and hw_rev
sysfs attributes on the ib_device class device.
> diff --git a/drivers/infiniband/hw/xsc/counters.c b/drivers/infiniband/hw/xsc/counters.c
> new file mode 100644
> index 0000000000000..c29a5bf8007f1
> --- /dev/null
> +++ b/drivers/infiniband/hw/xsc/counters.c
> @@ -0,0 +1,134 @@
[ ... ]
> +int xsc_ib_get_hw_stats(struct ib_device *ib_dev, struct rdma_hw_stats *stats,
> + u32 port_num, int index)
> +{
> + struct xsc_core_device *xdev = to_xdev(ib_dev)->xdev;
> + struct xsc_hw_stats_rdma stats_rdma;
> + u8 *hw_stats;
> + int ret;
> + int i;
> +
> + ret = xsc_ib_query_hw_stats_rdma(xdev, &stats_rdma);
> + if (ret)
> + return ret;
> +
> + hw_stats = (u8 *)&stats_rdma.stats.pf_stats;
> +
> + for (i = 0; i < stats->num_counters && i < XSC_NUM_PF_STATS; i++)
> + stats->value[i] =
> + be64_to_cpu(*(u64 *)(hw_stats + (uintptr_t)stats->descs[i].priv));
> +
> + if (stats->num_counters > XSC_NUM_PF_STATS) {
> + struct xsc_hw_global_stats_mbox_in gin = {};
> + struct xsc_hw_global_stats_mbox_out gout = {};
> +
> + gin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_GLOBAL_STATS);
> + ret = xsc_cmd_exec(xdev, &gin, sizeof(gin), &gout, sizeof(gout));
> + if (ret || gout.hdr.status)
> + return -EIO;
> +
> + hw_stats = (u8 *)&gout.hw_stats;
> + for (i = XSC_NUM_PF_STATS; i < stats->num_counters; i++)
> + stats->value[i] = be64_to_cpu(
> + *(u64 *)(hw_stats + (uintptr_t)stats->descs[i].priv));
> + }
[Severity: Low]
In xsc_ib_get_hw_stats(), both loops cast the counter field address to
u64 * before passing the dereferenced value to be64_to_cpu().
In drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h, the counter
fields in struct xsc_hw_stats_rdma_pf and struct xsc_hw_global_stats_rdma
are declared as __be64. Passing a host-typed u64 expression to
be64_to_cpu() triggers Sparse endianness warnings (cast to restricted
__be64) when building with C=1.
Should these casts in xsc_ib_get_hw_stats() use (__be64 *) instead of
(u64 *)?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261104155650.37053-1-tianx@yunsilicon.com?part=15
^ permalink raw reply [flat|nested] 30+ messages in thread