* [RFC 01/18] net/hinic3: add intro doc for hinic3
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-05-20 15:30 ` Stephen Hemminger
2025-04-18 7:02 ` [RFC 10/18] net/hinic3: add context and work queue support Feifei Wang
` (9 subsequent siblings)
10 siblings, 1 reply; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Feifei Wang, Yi Chen, Xin Wang
From: Feifei Wang <wangfeifei40@huawei.com>
This patch adds some basic files to describe the hinic3 driver.
Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
Signed-off-by: Yi Chen <chenyi221@huawei.com>
Reviewed-by: Xin Wang <wangxin679@h-partners.com>
---
.mailmap | 4 +-
MAINTAINERS | 6 +++
doc/guides/nics/hinic3.rst | 52 ++++++++++++++++++++++++++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_25_07.rst | 32 +---------------
5 files changed, 64 insertions(+), 31 deletions(-)
create mode 100644 doc/guides/nics/hinic3.rst
diff --git a/.mailmap b/.mailmap
index d8439b79ce..8c1341e783 100644
--- a/.mailmap
+++ b/.mailmap
@@ -429,7 +429,7 @@ Fang TongHao <fangtonghao@sangfor.com.cn>
Fan Zhang <fanzhang.oss@gmail.com> <roy.fan.zhang@intel.com>
Farah Smith <farah.smith@broadcom.com>
Fei Chen <chenwei.0515@bytedance.com>
-Feifei Wang <feifei.wang2@arm.com> <feifei.wang@arm.com>
+Feifei Wang <wangfeifei40@huawei.com> <feifei.wang1218@gmail.com> <feifei.wang2@arm.com> <feifei.wang@arm.com> <wff_light@vip.163.com>
Fei Qin <fei.qin@corigine.com>
Fengjiang Liu <liufengjiang.0426@bytedance.com>
Fengnan Chang <changfengnan@bytedance.com>
@@ -1718,6 +1718,7 @@ Xingguang He <xingguang.he@intel.com>
Xingyou Chen <niatlantice@gmail.com>
Xing Wang <xing_wang@realsil.com.cn>
Xinying Yu <xinying.yu@corigine.com>
+Xin Wang <wangxin679@h-partners.com>
Xin Long <longxin.xl@alibaba-inc.com>
Xi Zhang <xix.zhang@intel.com>
Xuan Ding <xuan.ding@intel.com>
@@ -1750,6 +1751,7 @@ Yelena Krivosheev <yelena@marvell.com>
Yerden Zhumabekov <e_zhumabekov@sts.kz> <yerden.zhumabekov@sts.kz>
Yevgeny Kliteynik <kliteyn@nvidia.com>
Yicai Lu <luyicai@huawei.com>
+Yi Chen <chenyi221@huawei.com>
Yiding Zhou <yidingx.zhou@intel.com>
Yi Li <liyi1@chinatelecom.cn>
Yi Liu <yi.liu@nxp.com>
diff --git a/MAINTAINERS b/MAINTAINERS
index 167cc74a15..f96a27210d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -773,6 +773,12 @@ F: drivers/net/hinic/
F: doc/guides/nics/hinic.rst
F: doc/guides/nics/features/hinic.ini
+Huawei hinic3
+M: Feifei Wang <wangfeifei40@huawei.com>
+F: drivers/net/hinic3/
+F: doc/guides/nics/hinic3.rst
+F: doc/guides/nics/features/hinic3.ini
+
Intel Network Common Code
M: Bruce Richardson <bruce.richardson@intel.com>
T: git://dpdk.org/next/dpdk-next-net-intel
diff --git a/doc/guides/nics/hinic3.rst b/doc/guides/nics/hinic3.rst
new file mode 100644
index 0000000000..c7080c8c1d
--- /dev/null
+++ b/doc/guides/nics/hinic3.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2025 Huawei Technologies Co., Ltd
+
+
+HINIC Poll Mode Driver
+======================
+
+The hinic3 PMD (**librte_net_hinic3**) provides poll mode driver support
+for 25Gbps/100Gbps/200Gbps Huawei SPx series Network Adapters.
+
+Features
+--------
+
+- Multi arch support: x86_64, ARMv8.
+- Multiple queues for TX and RX
+- Receiver Side Scaling (RSS)
+- flow filtering
+- Checksum offload
+- TSO offload
+- Promiscuous mode
+- Port hardware statistics
+- Link state information
+- Link flow control
+- Scattered and gather for TX and RX
+- Allmulticast mode
+- MTU update
+- Multicast MAC filter
+- Flow API
+- Set Link down or up
+- VLAN filter and VLAN offload
+- SR-IOV - Partially supported at this point, VFIO only
+- FW version
+- LRO
+
+Prerequisites
+-------------
+
+- Learning about Huawei Hi1823 Series Intelligent NICs using
+ `<https://www.hikunpeng.com/compute/component/nic>`_.
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+Limitations or Known issues
+---------------------------
+X86-32, Windows, and BSD are not supported yet.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 10a2eca3b0..5ae4021ccb 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
fm10k
gve
hinic
+ hinic3
hns3
i40e
ice
diff --git a/doc/guides/rel_notes/release_25_07.rst b/doc/guides/rel_notes/release_25_07.rst
index 093b85d206..1d65cf7829 100644
--- a/doc/guides/rel_notes/release_25_07.rst
+++ b/doc/guides/rel_notes/release_25_07.rst
@@ -24,37 +24,9 @@ DPDK Release 25.07
New Features
------------
-.. This section should contain new features added in this release.
- Sample format:
-
- * **Add a title in the past tense with a full stop.**
-
- Add a short 1-2 sentence description in the past tense.
- The description should be enough to allow someone scanning
- the release notes to understand the new feature.
-
- If the feature adds a lot of sub-features you can use a bullet list
- like this:
-
- * Added feature foo to do something.
- * Enhanced feature bar to do something else.
-
- Refer to the previous release notes for examples.
-
- Suggested order in release notes items:
- * Core libs (EAL, mempool, ring, mbuf, buses)
- * Device abstraction libs and PMDs (ordered alphabetically by vendor name)
- - ethdev (lib, PMDs)
- - cryptodev (lib, PMDs)
- - eventdev (lib, PMDs)
- - etc
- * Other libs
- * Apps, Examples, Tools (if significant)
-
- This section is a comment. Do not overwrite or remove it.
- Also, make sure to start the actual text at the margin.
- =======================================================
+* **Added Huawei hinic3 net driver [EXPERIMENTAL].**
+ * Added network driver for the Huawei SPx series Network Adapters.
Removed Items
-------------
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [RFC 01/18] net/hinic3: add intro doc for hinic3
2025-04-18 7:02 ` [RFC 01/18] net/hinic3: add intro doc for hinic3 Feifei Wang
@ 2025-05-20 15:30 ` Stephen Hemminger
0 siblings, 0 replies; 15+ messages in thread
From: Stephen Hemminger @ 2025-05-20 15:30 UTC (permalink / raw)
To: Feifei Wang; +Cc: dev, Feifei Wang, Yi Chen, Xin Wang
On Fri, 18 Apr 2025 15:02:39 +0800
Feifei Wang <wff_light@vip.163.com> wrote:
> diff --git a/doc/guides/rel_notes/release_25_07.rst b/doc/guides/rel_notes/release_25_07.rst
> index 093b85d206..1d65cf7829 100644
> --- a/doc/guides/rel_notes/release_25_07.rst
> +++ b/doc/guides/rel_notes/release_25_07.rst
> @@ -24,37 +24,9 @@ DPDK Release 25.07
> New Features
> ------------
>
> -.. This section should contain new features added in this release.
> - Sample format:
> -
> - * **Add a title in the past tense with a full stop.**
> -
> - Add a short 1-2 sentence description in the past tense.
> - The description should be enough to allow someone scanning
> - the release notes to understand the new feature.
> -
> - If the feature adds a lot of sub-features you can use a bullet list
> - like this:
> -
> - * Added feature foo to do something.
> - * Enhanced feature bar to do something else.
> -
> - Refer to the previous release notes for examples.
> -
> - Suggested order in release notes items:
> - * Core libs (EAL, mempool, ring, mbuf, buses)
> - * Device abstraction libs and PMDs (ordered alphabetically by vendor name)
> - - ethdev (lib, PMDs)
> - - cryptodev (lib, PMDs)
> - - eventdev (lib, PMDs)
> - - etc
> - * Other libs
> - * Apps, Examples, Tools (if significant)
> -
> - This section is a comment. Do not overwrite or remove it.
> - Also, make sure to start the actual text at the margin.
> - =======================================================
> +* **Added Huawei hinic3 net driver [EXPERIMENTAL].**
>
> + * Added network driver for the Huawei SPx series Network Adapters.
Leave the comment section in the release notes alone, it gets handled
as part of the release process.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [RFC 10/18] net/hinic3: add context and work queue support
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
2025-04-18 7:02 ` [RFC 01/18] net/hinic3: add intro doc for hinic3 Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 11/18] net/hinic3: add a mailbox communication module Feifei Wang
` (8 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Xin Wang, Feifei Wang, Yi Chen
From: Xin Wang <wangxin679@h-partners.com>
Work queue is used for cmdq and tx/rx buff description.
Nic business needs to configure cmdq context and txq/rxq
context. This patch adds data structures and function codes
for work queue and context.
Signed-off-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
Reviewed-by: Yi Chen <chenyi221@huawei.com>
---
drivers/net/hinic3/base/hinic3_wq.c | 148 ++++++++++++++++++++++++++++
drivers/net/hinic3/base/hinic3_wq.h | 109 ++++++++++++++++++++
2 files changed, 257 insertions(+)
create mode 100644 drivers/net/hinic3/base/hinic3_wq.c
create mode 100644 drivers/net/hinic3/base/hinic3_wq.h
diff --git a/drivers/net/hinic3/base/hinic3_wq.c b/drivers/net/hinic3/base/hinic3_wq.c
new file mode 100644
index 0000000000..9bccb10c9a
--- /dev/null
+++ b/drivers/net/hinic3/base/hinic3_wq.c
@@ -0,0 +1,148 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+#include <rte_bus_pci.h>
+#include <rte_errno.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_pci.h>
+
+#include "hinic3_compat.h"
+#include "hinic3_hwdev.h"
+#include "hinic3_wq.h"
+
+static void
+free_wq_pages(struct hinic3_wq *wq)
+{
+ hinic3_memzone_free(wq->wq_mz);
+
+ wq->queue_buf_paddr = 0;
+ wq->queue_buf_vaddr = 0;
+}
+
+static int
+alloc_wq_pages(struct hinic3_hwdev *hwdev, struct hinic3_wq *wq, int qid)
+{
+ const struct rte_memzone *wq_mz;
+
+ wq_mz = hinic3_dma_zone_reserve(hwdev->eth_dev, "hinic3_wq_mz",
+ (uint16_t)qid, wq->wq_buf_size,
+ RTE_PGSIZE_256K, SOCKET_ID_ANY);
+ if (!wq_mz) {
+ PMD_DRV_LOG(ERR, "Allocate wq[%d] rq_mz failed", qid);
+ return -ENOMEM;
+ }
+
+ memset(wq_mz->addr, 0, wq->wq_buf_size);
+ wq->wq_mz = wq_mz;
+ wq->queue_buf_paddr = wq_mz->iova;
+ wq->queue_buf_vaddr = (u64)(u64 *)wq_mz->addr;
+
+ return 0;
+}
+
+void
+hinic3_put_wqe(struct hinic3_wq *wq, int num_wqebbs)
+{
+ wq->cons_idx += num_wqebbs;
+ rte_atomic_fetch_add_explicit(&wq->delta, num_wqebbs,
+ rte_memory_order_seq_cst);
+}
+
+void *
+hinic3_read_wqe(struct hinic3_wq *wq, int num_wqebbs, u16 *cons_idx)
+{
+ u16 curr_cons_idx;
+
+ if ((rte_atomic_load_explicit(&wq->delta, rte_memory_order_seq_cst) +
+ num_wqebbs) > wq->q_depth)
+ return NULL;
+
+ curr_cons_idx = (u16)(wq->cons_idx);
+
+ curr_cons_idx = MASKED_WQE_IDX(wq, curr_cons_idx);
+
+ *cons_idx = curr_cons_idx;
+
+ return WQ_WQE_ADDR(wq, (u32)(*cons_idx));
+}
+
+int
+hinic3_cmdq_alloc(struct hinic3_wq *wq, void *dev, int cmdq_blocks,
+ u32 wq_buf_size, u32 wqebb_shift, u16 q_depth)
+{
+ struct hinic3_hwdev *hwdev = (struct hinic3_hwdev *)dev;
+ int i, j;
+ int err;
+
+ /* Validate q_depth is power of 2 & wqebb_size is not 0. */
+ for (i = 0; i < cmdq_blocks; i++) {
+ wq[i].wqebb_size = 1U << wqebb_shift;
+ wq[i].wqebb_shift = wqebb_shift;
+ wq[i].wq_buf_size = wq_buf_size;
+ wq[i].q_depth = q_depth;
+
+ err = alloc_wq_pages(hwdev, &wq[i], i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to alloc CMDQ blocks");
+ goto cmdq_block_err;
+ }
+
+ wq[i].cons_idx = 0;
+ wq[i].prod_idx = 0;
+ rte_atomic_store_explicit(&wq[i].delta, q_depth,
+ rte_memory_order_seq_cst);
+
+ wq[i].mask = q_depth - 1;
+ }
+
+ return 0;
+
+cmdq_block_err:
+ for (j = 0; j < i; j++)
+ free_wq_pages(&wq[j]);
+
+ return err;
+}
+
+void
+hinic3_cmdq_free(struct hinic3_wq *wq, int cmdq_blocks)
+{
+ int i;
+
+ for (i = 0; i < cmdq_blocks; i++)
+ free_wq_pages(&wq[i]);
+}
+
+void
+hinic3_wq_wqe_pg_clear(struct hinic3_wq *wq)
+{
+ wq->cons_idx = 0;
+ wq->prod_idx = 0;
+
+ memset((void *)wq->queue_buf_vaddr, 0, wq->wq_buf_size);
+}
+
+void *
+hinic3_get_wqe(struct hinic3_wq *wq, int num_wqebbs, u16 *prod_idx)
+{
+ u16 curr_prod_idx;
+
+ rte_atomic_fetch_sub_explicit(&wq->delta, num_wqebbs,
+ rte_memory_order_seq_cst);
+ curr_prod_idx = (u16)(wq->prod_idx);
+ wq->prod_idx += num_wqebbs;
+ *prod_idx = MASKED_WQE_IDX(wq, curr_prod_idx);
+
+ return WQ_WQE_ADDR(wq, (u32)(*prod_idx));
+}
+
+void
+hinic3_set_sge(struct hinic3_sge *sge, uint64_t addr, u32 len)
+{
+ sge->hi_addr = upper_32_bits(addr);
+ sge->lo_addr = lower_32_bits(addr);
+ sge->len = len;
+}
diff --git a/drivers/net/hinic3/base/hinic3_wq.h b/drivers/net/hinic3/base/hinic3_wq.h
new file mode 100644
index 0000000000..84d54c2aeb
--- /dev/null
+++ b/drivers/net/hinic3/base/hinic3_wq.h
@@ -0,0 +1,109 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_WQ_H_
+#define _HINIC3_WQ_H_
+
+/* Use 0-level CLA, page size must be: SQ 16B(wqe) * 64k(max_q_depth). */
+#define HINIC3_DEFAULT_WQ_PAGE_SIZE 0x100000
+#define HINIC3_HW_WQ_PAGE_SIZE 0x1000
+
+#define MASKED_WQE_IDX(wq, idx) ((idx) & (wq)->mask)
+
+#define WQ_WQE_ADDR(wq, idx) \
+ ({ \
+ typeof(wq) __wq = (wq); \
+ (void *)((u64)(__wq->queue_buf_vaddr) + ((idx) << __wq->wqebb_shift)); \
+ })
+
+struct hinic3_sge {
+ u32 hi_addr;
+ u32 lo_addr;
+ u32 len;
+};
+
+struct hinic3_wq {
+ /* The addresses are 64 bit in the HW. */
+ u64 queue_buf_vaddr;
+
+ u16 q_depth;
+ u16 mask;
+ RTE_ATOMIC(int32_t)delta;
+
+ u32 cons_idx;
+ u32 prod_idx;
+
+ u64 queue_buf_paddr;
+
+ u32 wqebb_size;
+ u32 wqebb_shift;
+
+ u32 wq_buf_size;
+
+ const struct rte_memzone *wq_mz;
+
+ u32 rsvd[5];
+};
+
+void hinic3_put_wqe(struct hinic3_wq *wq, int num_wqebbs);
+
+/**
+ * Read a WQE and update CI.
+ *
+ * @param[in] wq
+ * The work queue structure.
+ * @param[in] num_wqebbs
+ * The number of work queue elements to read.
+ * @param[out] cons_idx
+ * The updated consumer index.
+ *
+ * @return
+ * The address of WQE, or NULL if not enough elements are available.
+ */
+void *hinic3_read_wqe(struct hinic3_wq *wq, int num_wqebbs, u16 *cons_idx);
+
+/**
+ * Allocate command queue blocks and initialize related parameters.
+ *
+ * @param[in] wq
+ * The cmdq->wq structure.
+ * @param[in] dev
+ * The device context for the hardware.
+ * @param[in] cmdq_blocks
+ * The number of command queue blocks to allocate.
+ * @param[in] wq_buf_size
+ * The size of each work queue buffer.
+ * @param[in] wqebb_shift
+ * The shift value for determining the work queue element size.
+ * @param[in] q_depth
+ * The depth of each command queue.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int hinic3_cmdq_alloc(struct hinic3_wq *wq, void *dev, int cmdq_blocks,
+ u32 wq_buf_size, u32 wqebb_shift, u16 q_depth);
+
+void hinic3_cmdq_free(struct hinic3_wq *wq, int cmdq_blocks);
+
+void hinic3_wq_wqe_pg_clear(struct hinic3_wq *wq);
+
+/**
+ * Get WQE and update PI.
+ *
+ * @param[in] wq
+ * The cmdq->wq structure.
+ * @param[in] num_wqebbs
+ * The number of work queue elements to allocate.
+ * @param[out] prod_idx
+ * The updated producer index, masked according to the queue size.
+ *
+ * @return
+ * The address of the work queue element.
+ */
+void *hinic3_get_wqe(struct hinic3_wq *wq, int num_wqebbs, u16 *prod_idx);
+
+void hinic3_set_sge(struct hinic3_sge *sge, uint64_t addr, u32 len);
+
+#endif /* _HINIC3_WQ_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 11/18] net/hinic3: add a mailbox communication module
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
2025-04-18 7:02 ` [RFC 01/18] net/hinic3: add intro doc for hinic3 Feifei Wang
2025-04-18 7:02 ` [RFC 10/18] net/hinic3: add context and work queue support Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 12/18] net/hinic3: add device initailization Feifei Wang
` (7 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Yi Chen, Xin Wang, Feifei Wang
From: Yi Chen <chenyi221@huawei.com>
This patch adds support for mailbox of hinic3 PMD driver,
mailbox is used for communication between PF/VF driver and MPU.
This patch provides mailbox-related data structures and functional
code.
Signed-off-by: Yi Chen <chenyi221@huawei.com>
Reviewed-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
---
drivers/net/hinic3/base/hinic3_mbox.c | 1392 +++++++++++++++++++++++++
drivers/net/hinic3/base/hinic3_mbox.h | 199 ++++
2 files changed, 1591 insertions(+)
create mode 100644 drivers/net/hinic3/base/hinic3_mbox.c
create mode 100644 drivers/net/hinic3/base/hinic3_mbox.h
diff --git a/drivers/net/hinic3/base/hinic3_mbox.c b/drivers/net/hinic3/base/hinic3_mbox.c
new file mode 100644
index 0000000000..78dfee2b1c
--- /dev/null
+++ b/drivers/net/hinic3/base/hinic3_mbox.c
@@ -0,0 +1,1392 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include "hinic3_compat.h"
+#include "hinic3_csr.h"
+#include "hinic3_eqs.h"
+#include "hinic3_hw_cfg.h"
+#include "hinic3_hwdev.h"
+#include "hinic3_hwif.h"
+#include "hinic3_mbox.h"
+#include "hinic3_mgmt.h"
+#include "hinic3_nic_event.h"
+
+#define HINIC3_MBOX_INT_DST_FUNC_SHIFT 0
+#define HINIC3_MBOX_INT_DST_AEQN_SHIFT 10
+#define HINIC3_MBOX_INT_SRC_RESP_AEQN_SHIFT 12
+#define HINIC3_MBOX_INT_STAT_DMA_SHIFT 14
+/* The size of data to be send (unit of 4 bytes). */
+#define HINIC3_MBOX_INT_TX_SIZE_SHIFT 20
+/* SO_RO(strong order, relax order). */
+#define HINIC3_MBOX_INT_STAT_DMA_SO_RO_SHIFT 25
+#define HINIC3_MBOX_INT_WB_EN_SHIFT 28
+
+#define HINIC3_MBOX_INT_DST_AEQN_MASK 0x3
+#define HINIC3_MBOX_INT_SRC_RESP_AEQN_MASK 0x3
+#define HINIC3_MBOX_INT_STAT_DMA_MASK 0x3F
+#define HINIC3_MBOX_INT_TX_SIZE_MASK 0x1F
+#define HINIC3_MBOX_INT_STAT_DMA_SO_RO_MASK 0x3
+#define HINIC3_MBOX_INT_WB_EN_MASK 0x1
+
+#define HINIC3_MBOX_INT_SET(val, field) \
+ (((val) & HINIC3_MBOX_INT_##field##_MASK) \
+ << HINIC3_MBOX_INT_##field##_SHIFT)
+
+enum hinic3_mbox_tx_status {
+ TX_NOT_DONE = 1,
+};
+
+#define HINIC3_MBOX_CTRL_TRIGGER_AEQE_SHIFT 0
+
+/*
+ * Specifies the issue request for the message data.
+ * 0 - Tx request is done;
+ * 1 - Tx request is in process.
+ */
+#define HINIC3_MBOX_CTRL_TX_STATUS_SHIFT 1
+#define HINIC3_MBOX_CTRL_DST_FUNC_SHIFT 16
+
+#define HINIC3_MBOX_CTRL_TRIGGER_AEQE_MASK 0x1
+#define HINIC3_MBOX_CTRL_TX_STATUS_MASK 0x1
+#define HINIC3_MBOX_CTRL_DST_FUNC_MASK 0x1FFF
+
+#define HINIC3_MBOX_CTRL_SET(val, field) \
+ (((val) & HINIC3_MBOX_CTRL_##field##_MASK) \
+ << HINIC3_MBOX_CTRL_##field##_SHIFT)
+
+#define MBOX_SEGLEN_MASK \
+ HINIC3_MSG_HEADER_SET(HINIC3_MSG_HEADER_SEG_LEN_MASK, SEG_LEN)
+
+#define MBOX_MSG_POLLING_TIMEOUT 500000 /* Unit is 10us. */
+#define HINIC3_MBOX_COMP_TIME 40000U
+
+#define MBOX_MAX_BUF_SZ 2048UL
+#define MBOX_HEADER_SZ 8
+#define HINIC3_MBOX_DATA_SIZE (MBOX_MAX_BUF_SZ - MBOX_HEADER_SZ)
+
+#define MBOX_TLP_HEADER_SZ 16
+
+/* Mbox size is 64B, 8B for mbox_header, 8B reserved. */
+#define MBOX_SEG_LEN 48
+#define MBOX_SEG_LEN_ALIGN 4
+#define MBOX_WB_STATUS_LEN 16UL
+
+/* Mbox write back status is 16B, only first 4B is used. */
+#define MBOX_WB_STATUS_ERRCODE_MASK 0xFFFF
+#define MBOX_WB_STATUS_MASK 0xFF
+#define MBOX_WB_ERROR_CODE_MASK 0xFF00
+#define MBOX_WB_STATUS_FINISHED_SUCCESS 0xFF
+#define MBOX_WB_STATUS_FINISHED_WITH_ERR 0xFE
+#define MBOX_WB_STATUS_NOT_FINISHED 0x00
+
+/* Determine the write back status. */
+#define MBOX_STATUS_FINISHED(wb) \
+ (((wb) & MBOX_WB_STATUS_MASK) != MBOX_WB_STATUS_NOT_FINISHED)
+#define MBOX_STATUS_SUCCESS(wb) \
+ (((wb) & MBOX_WB_STATUS_MASK) == MBOX_WB_STATUS_FINISHED_SUCCESS)
+#define MBOX_STATUS_ERRCODE(wb) ((wb) & MBOX_WB_ERROR_CODE_MASK)
+
+/* Indicate the value related to the sequence ID. */
+#define SEQ_ID_START_VAL 0
+#define SEQ_ID_MAX_VAL 42
+
+#define DST_AEQ_IDX_DEFAULT_VAL 0
+#define SRC_AEQ_IDX_DEFAULT_VAL 0
+#define NO_DMA_ATTRIBUTE_VAL 0
+
+#define MBOX_MSG_NO_DATA_LEN 1
+
+/* Obtain the specified content of the mailbox. */
+#define MBOX_BODY_FROM_HDR(header) ((u8 *)(header) + MBOX_HEADER_SZ)
+#define MBOX_AREA(hwif) \
+ ((hwif)->cfg_regs_base + HINIC3_FUNC_CSR_MAILBOX_DATA_OFF)
+
+#define IS_PF_OR_PPF_SRC(src_func_idx) ((src_func_idx) < HINIC3_MAX_PF_FUNCS)
+
+#define MBOX_RESPONSE_ERROR 0x1
+#define MBOX_MSG_ID_MASK 0xF
+#define MBOX_MSG_ID(func_to_func) ((func_to_func)->send_msg_id)
+#define MBOX_MSG_ID_INC(func_to_func) \
+ ({ \
+ typeof(func_to_func) __func = (func_to_func); \
+ MBOX_MSG_ID(__func) = (MBOX_MSG_ID(__func) + 1) & \
+ MBOX_MSG_ID_MASK; \
+ })
+
+/* Max message counter waits to process for one function. */
+#define HINIC3_MAX_MSG_CNT_TO_PROCESS 10
+
+enum mbox_ordering_type {
+ STRONG_ORDER,
+};
+
+enum mbox_write_back_type {
+ WRITE_BACK = 1,
+};
+
+enum mbox_aeq_trig_type {
+ NOT_TRIGGER,
+ TRIGGER,
+};
+
+static int send_mbox_to_func(struct hinic3_mbox *func_to_func,
+ enum hinic3_mod_type mod, u16 cmd, void *msg,
+ u16 msg_len, u16 dst_func,
+ enum hinic3_msg_direction_type direction,
+ enum hinic3_msg_ack_type ack_type,
+ struct mbox_msg_info *msg_info);
+static int send_tlp_mbox_to_func(struct hinic3_mbox *func_to_func,
+ enum hinic3_mod_type mod, u16 cmd, void *msg,
+ u16 msg_len, u16 dst_func,
+ enum hinic3_msg_direction_type direction,
+ enum hinic3_msg_ack_type ack_type,
+ struct mbox_msg_info *msg_info);
+
+static int
+recv_vf_mbox_handler(struct hinic3_mbox *func_to_func,
+ struct hinic3_recv_mbox *recv_mbox, void *buf_out,
+ u16 *out_size, __rte_unused void *param)
+{
+ int err = 0;
+
+ /*
+ * Invoke the corresponding processing function according to the type of
+ * the received mailbox.
+ */
+ switch (recv_mbox->mod) {
+ case HINIC3_MOD_COMM:
+ err = vf_handle_pf_comm_mbox(func_to_func->hwdev, func_to_func,
+ recv_mbox->cmd, recv_mbox->mbox,
+ recv_mbox->mbox_len, buf_out,
+ out_size);
+ break;
+ case HINIC3_MOD_CFGM:
+ err = cfg_mbx_vf_proc_msg(func_to_func->hwdev,
+ func_to_func->hwdev->cfg_mgmt,
+ recv_mbox->cmd, recv_mbox->mbox, recv_mbox->mbox_len,
+ buf_out, out_size);
+ break;
+ case HINIC3_MOD_L2NIC:
+ err = hinic3_vf_event_handler(func_to_func->hwdev,
+ func_to_func->hwdev->cfg_mgmt,
+ recv_mbox->cmd, recv_mbox->mbox, recv_mbox->mbox_len,
+ buf_out, out_size);
+ break;
+ case HINIC3_MOD_HILINK:
+ err = hinic3_vf_mag_event_handler(func_to_func->hwdev,
+ func_to_func->hwdev->cfg_mgmt,
+ recv_mbox->cmd, recv_mbox->mbox, recv_mbox->mbox_len,
+ buf_out, out_size);
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "No handler, mod: %d", recv_mbox->mod);
+ err = HINIC3_MBOX_VF_CMD_ERROR;
+ break;
+ }
+
+ return err;
+}
+
+/**
+ * Respond to the accept packet, construct a response message, and send it.
+ *
+ * @param[in] func_to_func
+ * Context for inter-function communication.
+ * @param[in] recv_mbox
+ * Pointer to the received inter-function mailbox structure.
+ * @param[in] err
+ * Error Code.
+ * @param[in] out_size
+ * Output Size.
+ * @param[in] src_func_idx
+ * Index of the source function.
+ */
+static void
+response_for_recv_func_mbox(struct hinic3_mbox *func_to_func,
+ struct hinic3_recv_mbox *recv_mbox, int err,
+ u16 out_size, u16 src_func_idx)
+{
+ struct mbox_msg_info msg_info = {0};
+
+ if (recv_mbox->ack_type == HINIC3_MSG_ACK) {
+ msg_info.msg_id = recv_mbox->msg_info.msg_id;
+ if (err)
+ msg_info.status = HINIC3_MBOX_PF_SEND_ERR;
+
+ /* Select the sending function based on the packet type. */
+ if (IS_TLP_MBX(src_func_idx))
+ send_tlp_mbox_to_func(func_to_func, recv_mbox->mod,
+ recv_mbox->cmd,
+ recv_mbox->buf_out, out_size,
+ src_func_idx, HINIC3_MSG_RESPONSE,
+ HINIC3_MSG_NO_ACK, &msg_info);
+ else
+ send_mbox_to_func(func_to_func, recv_mbox->mod,
+ recv_mbox->cmd, recv_mbox->buf_out,
+ out_size, src_func_idx,
+ HINIC3_MSG_RESPONSE,
+ HINIC3_MSG_NO_ACK, &msg_info);
+ }
+}
+
+static bool
+check_func_mbox_ack_first(u8 mod)
+{
+ return mod == HINIC3_MOD_HILINK;
+}
+
+static void
+recv_func_mbox_handler(struct hinic3_mbox *func_to_func,
+ struct hinic3_recv_mbox *recv_mbox, u16 src_func_idx,
+ void *param)
+{
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+ void *buf_out = recv_mbox->buf_out;
+ bool ack_first = false;
+ u16 out_size = MBOX_MAX_BUF_SZ;
+ int err = 0;
+ /* Check whether the response is the first ACK message. */
+ ack_first = check_func_mbox_ack_first(recv_mbox->mod);
+ if (ack_first && recv_mbox->ack_type == HINIC3_MSG_ACK) {
+ response_for_recv_func_mbox(func_to_func, recv_mbox, err,
+ out_size, src_func_idx);
+ }
+
+ /* Processe mailbox information in the VF. */
+ if (HINIC3_IS_VF(hwdev)) {
+ err = recv_vf_mbox_handler(func_to_func, recv_mbox, buf_out,
+ &out_size, param);
+ } else {
+ err = -EINVAL;
+ PMD_DRV_LOG(ERR,
+ "PMD doesn't support non-VF handle mailbox message");
+ }
+
+ if (!out_size || err)
+ out_size = MBOX_MSG_NO_DATA_LEN;
+
+ if (!ack_first && recv_mbox->ack_type == HINIC3_MSG_ACK) {
+ response_for_recv_func_mbox(func_to_func, recv_mbox, err,
+ out_size, src_func_idx);
+ }
+}
+
+/**
+ * Processe mailbox responses from functions.
+ *
+ * @param[in] func_to_func
+ * Mailbox for inter-function communication.
+ * @param[in] recv_mbox
+ * Received mailbox message.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+resp_mbox_handler(struct hinic3_mbox *func_to_func,
+ struct hinic3_recv_mbox *recv_mbox)
+{
+ int ret;
+ rte_spinlock_lock(&func_to_func->mbox_lock);
+ if (recv_mbox->msg_info.msg_id == func_to_func->send_msg_id &&
+ func_to_func->event_flag == EVENT_START) {
+ func_to_func->event_flag = EVENT_SUCCESS;
+ ret = 0;
+ } else {
+ PMD_DRV_LOG(ERR,
+ "Mbox response timeout, current send msg id(0x%x), "
+ "recv msg id(0x%x), status(0x%x)",
+ func_to_func->send_msg_id,
+ recv_mbox->msg_info.msg_id,
+ recv_mbox->msg_info.status);
+ ret = HINIC3_MSG_HANDLER_RES;
+ }
+ rte_spinlock_unlock(&func_to_func->mbox_lock);
+ return ret;
+}
+
+/**
+ * Check whether the received mailbox message segment is valid.
+ *
+ * @param[out] recv_mbox
+ * Received mailbox message.
+ * @param[in] mbox_header
+ * Mailbox header.
+ * @return
+ * The value true indicates valid, and the value false indicates invalid.
+ */
+static bool
+check_mbox_segment(struct hinic3_recv_mbox *recv_mbox, u64 mbox_header)
+{
+ u8 seq_id, seg_len, msg_id, mod;
+ u16 src_func_idx, cmd;
+
+ /* Get info from the mailbox header. */
+ seq_id = HINIC3_MSG_HEADER_GET(mbox_header, SEQID);
+ seg_len = HINIC3_MSG_HEADER_GET(mbox_header, SEG_LEN);
+ src_func_idx = HINIC3_MSG_HEADER_GET(mbox_header, SRC_GLB_FUNC_IDX);
+ msg_id = HINIC3_MSG_HEADER_GET(mbox_header, MSG_ID);
+ mod = HINIC3_MSG_HEADER_GET(mbox_header, MODULE);
+ cmd = HINIC3_MSG_HEADER_GET(mbox_header, CMD);
+
+ if (seq_id > SEQ_ID_MAX_VAL || seg_len > MBOX_SEG_LEN)
+ goto seg_err;
+
+ /* New message segment, which saves its information to recv_mbox. */
+ if (seq_id == 0) {
+ recv_mbox->seq_id = seq_id;
+ recv_mbox->msg_info.msg_id = msg_id;
+ recv_mbox->mod = mod;
+ recv_mbox->cmd = cmd;
+ } else {
+ if ((seq_id != recv_mbox->seq_id + 1) ||
+ msg_id != recv_mbox->msg_info.msg_id ||
+ mod != recv_mbox->mod || cmd != recv_mbox->cmd)
+ goto seg_err;
+
+ recv_mbox->seq_id = seq_id;
+ }
+
+ return true;
+
+seg_err:
+ PMD_DRV_LOG(ERR,
+ "Mailbox segment check failed, src func id: 0x%x, "
+ "front seg info: seq id: 0x%x, msg id: 0x%x, mod: 0x%x, "
+ "cmd: 0x%x",
+ src_func_idx, recv_mbox->seq_id, recv_mbox->msg_info.msg_id,
+ recv_mbox->mod, recv_mbox->cmd);
+ PMD_DRV_LOG(ERR,
+ "Current seg info: seg len: 0x%x, seq id: 0x%x, "
+ "msg id: 0x%x, mod: 0x%x, cmd: 0x%x",
+ seg_len, seq_id, msg_id, mod, cmd);
+
+ return false;
+}
+
+static int
+recv_mbox_handler(struct hinic3_mbox *func_to_func, void *header,
+ struct hinic3_recv_mbox *recv_mbox, void *param)
+{
+ u64 mbox_header = *((u64 *)header);
+ void *mbox_body = MBOX_BODY_FROM_HDR(header);
+ u16 src_func_idx;
+ int pos;
+ u8 seq_id;
+ /* Obtain information from the mailbox header. */
+ seq_id = HINIC3_MSG_HEADER_GET(mbox_header, SEQID);
+ src_func_idx = HINIC3_MSG_HEADER_GET(mbox_header, SRC_GLB_FUNC_IDX);
+
+ if (!check_mbox_segment(recv_mbox, mbox_header)) {
+ recv_mbox->seq_id = SEQ_ID_MAX_VAL;
+ return HINIC3_MSG_HANDLER_RES;
+ }
+
+ pos = seq_id * MBOX_SEG_LEN;
+ memcpy((void *)((u8 *)recv_mbox->mbox + pos), (void *)mbox_body,
+ (size_t)HINIC3_MSG_HEADER_GET(mbox_header, SEG_LEN));
+
+ if (!HINIC3_MSG_HEADER_GET(mbox_header, LAST))
+ return HINIC3_MSG_HANDLER_RES;
+ /* Setting the information about the recv mailbox. */
+ recv_mbox->cmd = HINIC3_MSG_HEADER_GET(mbox_header, CMD);
+ recv_mbox->mod = HINIC3_MSG_HEADER_GET(mbox_header, MODULE);
+ recv_mbox->mbox_len = HINIC3_MSG_HEADER_GET(mbox_header, MSG_LEN);
+ recv_mbox->ack_type = HINIC3_MSG_HEADER_GET(mbox_header, NO_ACK);
+ recv_mbox->msg_info.msg_id = HINIC3_MSG_HEADER_GET(mbox_header, MSG_ID);
+ recv_mbox->msg_info.status = HINIC3_MSG_HEADER_GET(mbox_header, STATUS);
+ recv_mbox->seq_id = SEQ_ID_MAX_VAL;
+
+ /*
+ * If the received message is a response message, call the mbox response
+ * processing function.
+ */
+ if (HINIC3_MSG_HEADER_GET(mbox_header, DIRECTION) ==
+ HINIC3_MSG_RESPONSE) {
+ return resp_mbox_handler(func_to_func, recv_mbox);
+ }
+
+ recv_func_mbox_handler(func_to_func, recv_mbox, src_func_idx, param);
+ return HINIC3_MSG_HANDLER_RES;
+}
+
+static inline int
+hinic3_mbox_get_index(int func)
+{
+ return (func == HINIC3_MGMT_SRC_ID) ? HINIC3_MBOX_MPU_INDEX
+ : HINIC3_MBOX_PF_INDEX;
+}
+
+int
+hinic3_mbox_func_aeqe_handler(void *handle, u8 *header, __rte_unused u8 size,
+ void *param)
+{
+ struct hinic3_mbox *func_to_func = NULL;
+ struct hinic3_recv_mbox *recv_mbox = NULL;
+ u64 mbox_header = *((u64 *)header);
+ u64 src, dir;
+ /* Obtain the mailbox for communication between functions. */
+ func_to_func = ((struct hinic3_hwdev *)handle)->func_to_func;
+
+ dir = HINIC3_MSG_HEADER_GET(mbox_header, DIRECTION);
+ src = HINIC3_MSG_HEADER_GET(mbox_header, SRC_GLB_FUNC_IDX);
+
+ src = hinic3_mbox_get_index((int)src);
+ recv_mbox = (dir == HINIC3_MSG_DIRECT_SEND)
+ ? &func_to_func->mbox_send[src]
+ : &func_to_func->mbox_resp[src];
+ /* Processing Received Mailbox info. */
+ return recv_mbox_handler(func_to_func, (u64 *)header, recv_mbox, param);
+}
+
+static void
+clear_mbox_status(struct hinic3_send_mbox *mbox)
+{
+ *mbox->wb_status = 0;
+
+ /* Clear mailbox write back status. */
+ rte_wmb();
+}
+
+static void
+mbox_copy_header(struct hinic3_send_mbox *mbox, u64 *header)
+{
+ u32 *data = (u32 *)header;
+ u32 i, idx_max = MBOX_HEADER_SZ / sizeof(u32);
+
+ for (i = 0; i < idx_max; i++) {
+ rte_write32(cpu_to_be32(*(data + i)),
+ mbox->data + i * sizeof(u32));
+ }
+}
+
+#define MBOX_DMA_MSG_INIT_XOR_VAL 0x5a5a5a5a
+static u32
+mbox_dma_msg_xor(u32 *data, u16 msg_len)
+{
+ u32 xor = MBOX_DMA_MSG_INIT_XOR_VAL;
+ u16 dw_len = msg_len / sizeof(u32);
+ u16 i;
+
+ for (i = 0; i < dw_len; i++)
+ xor ^= data[i];
+
+ return xor;
+}
+
+static void
+mbox_copy_send_data_addr(struct hinic3_send_mbox *mbox, u16 seg_len)
+{
+ u32 addr_h, addr_l, xor;
+
+ xor = mbox_dma_msg_xor(mbox->sbuff_vaddr, seg_len);
+ addr_h = upper_32_bits(mbox->sbuff_paddr);
+ addr_l = lower_32_bits(mbox->sbuff_paddr);
+
+ rte_write32(cpu_to_be32(xor), mbox->data + MBOX_HEADER_SZ);
+ rte_write32(cpu_to_be32(addr_h),
+ mbox->data + MBOX_HEADER_SZ + sizeof(u32));
+ rte_write32(cpu_to_be32(addr_l),
+ mbox->data + MBOX_HEADER_SZ + 0x2 * sizeof(u32));
+ rte_write32(cpu_to_be32((u32)seg_len),
+ mbox->data + MBOX_HEADER_SZ + 0x3 * sizeof(u32));
+ /* Reserved field. */
+ rte_write32(0, mbox->data + MBOX_HEADER_SZ + 0x4 * sizeof(u32));
+ rte_write32(0, mbox->data + MBOX_HEADER_SZ + 0x5 * sizeof(u32));
+}
+
+static void
+mbox_copy_send_data(struct hinic3_send_mbox *mbox, void *seg, u16 seg_len)
+{
+ u32 *data = seg;
+ u32 data_len, chk_sz = sizeof(u32);
+ u32 i, idx_max;
+ u8 mbox_max_buf[MBOX_SEG_LEN] = {0};
+
+ /* The mbox message should be aligned in 4 bytes. */
+ if (seg_len % chk_sz) {
+ rte_memcpy(mbox_max_buf, seg, seg_len);
+ data = (u32 *)mbox_max_buf;
+ }
+
+ data_len = seg_len;
+ idx_max = RTE_ALIGN(data_len, chk_sz) / chk_sz;
+
+ for (i = 0; i < idx_max; i++) {
+ rte_write32(cpu_to_be32(*(data + i)),
+ mbox->data + MBOX_HEADER_SZ + i * sizeof(u32));
+ }
+}
+
+static void
+write_mbox_msg_attr(struct hinic3_mbox *func_to_func, u16 dst_func,
+ u16 dst_aeqn, u16 seg_len)
+{
+ u32 mbox_int, mbox_ctrl;
+
+ /* If VF, function ids must self-learning by HW(PPF=1 PF=0). */
+ if (HINIC3_IS_VF(func_to_func->hwdev) &&
+ dst_func != HINIC3_MGMT_SRC_ID) {
+ if (dst_func == HINIC3_HWIF_PPF_IDX(func_to_func->hwdev->hwif))
+ dst_func = 1;
+ else
+ dst_func = 0;
+ }
+ /* Set the interrupt attribute of the mailbox. */
+ mbox_int = HINIC3_MBOX_INT_SET(dst_aeqn, DST_AEQN) |
+ HINIC3_MBOX_INT_SET(0, SRC_RESP_AEQN) |
+ HINIC3_MBOX_INT_SET(NO_DMA_ATTRIBUTE_VAL, STAT_DMA) |
+ HINIC3_MBOX_INT_SET(RTE_ALIGN(seg_len + MBOX_HEADER_SZ,
+ MBOX_SEG_LEN_ALIGN) >>
+ 2,
+ TX_SIZE) |
+ HINIC3_MBOX_INT_SET(STRONG_ORDER, STAT_DMA_SO_RO) |
+ HINIC3_MBOX_INT_SET(WRITE_BACK, WB_EN);
+
+ /* The interrupt attribute is written to the interrupt register. */
+ hinic3_hwif_write_reg(func_to_func->hwdev->hwif,
+ HINIC3_FUNC_CSR_MAILBOX_INT_OFFSET_OFF, mbox_int);
+
+ rte_wmb(); /**< Writing the mbox intr attributes */
+
+ /* Set the control attributes of the mailbox and write to register. */
+ mbox_ctrl = HINIC3_MBOX_CTRL_SET(TX_NOT_DONE, TX_STATUS);
+ mbox_ctrl |= HINIC3_MBOX_CTRL_SET(NOT_TRIGGER, TRIGGER_AEQE);
+ mbox_ctrl |= HINIC3_MBOX_CTRL_SET(dst_func, DST_FUNC);
+ hinic3_hwif_write_reg(func_to_func->hwdev->hwif,
+ HINIC3_FUNC_CSR_MAILBOX_CONTROL_OFF, mbox_ctrl);
+}
+
+/**
+ * Read the value of the mailbox register of the hardware device.
+ *
+ * @param[in] hwdev
+ * Pointer to hardware device structure.
+ */
+static void
+dump_mbox_reg(struct hinic3_hwdev *hwdev)
+{
+ u32 val;
+ /* Read the value of the MBOX control register. */
+ val = hinic3_hwif_read_reg(hwdev->hwif,
+ HINIC3_FUNC_CSR_MAILBOX_CONTROL_OFF);
+ PMD_DRV_LOG(ERR, "Mailbox control reg: 0x%x", val);
+ /* Read the value of the MBOX interrupt offset register. */
+ val = hinic3_hwif_read_reg(hwdev->hwif,
+ HINIC3_FUNC_CSR_MAILBOX_INT_OFFSET_OFF);
+ PMD_DRV_LOG(ERR, "Mailbox interrupt offset: 0x%x", val);
+}
+
+static u16
+get_mbox_status(struct hinic3_send_mbox *mbox)
+{
+ /* Write back is 16B, but only use first 4B. */
+ u64 wb_val = be64_to_cpu(*mbox->wb_status);
+
+ rte_rmb(); /**< Verify reading before check. */
+
+ return (u16)(wb_val & MBOX_WB_STATUS_ERRCODE_MASK);
+}
+
+/**
+ * Sending Mailbox Message Segment.
+ *
+ * @param[in] func_to_func
+ * Mailbox for inter-function communication.
+ * @param[in] header
+ * Mailbox header.
+ * @param[in] dst_func
+ * Indicate destination func.
+ * @param[in] seg
+ * Segment data to be sent.
+ * @param[in] seg_len
+ * Length of the segment to be sent.
+ * @param[in] msg_info
+ * Indicate the message information.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+send_mbox_seg(struct hinic3_mbox *func_to_func, u64 header, u16 dst_func,
+ void *seg, u16 seg_len, __rte_unused void *msg_info)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+ u8 num_aeqs = hwdev->hwif->attr.num_aeqs;
+ u16 dst_aeqn, wb_status = 0, errcode;
+ u16 seq_dir = HINIC3_MSG_HEADER_GET(header, DIRECTION);
+ u32 cnt = 0;
+
+ /* Mbox to mgmt cpu, hardware doesn't care dst aeq id. */
+ if (num_aeqs >= 2)
+ dst_aeqn = (seq_dir == HINIC3_MSG_DIRECT_SEND)
+ ? HINIC3_ASYNC_MSG_AEQ
+ : HINIC3_MBOX_RSP_MSG_AEQ;
+ else
+ dst_aeqn = 0;
+
+ clear_mbox_status(send_mbox);
+ mbox_copy_header(send_mbox, &header);
+ mbox_copy_send_data(send_mbox, seg, seg_len);
+
+ /* Set mailbox msg seg len. */
+ write_mbox_msg_attr(func_to_func, dst_func, dst_aeqn, seg_len);
+ rte_wmb(); /**< Writing the mbox msg attributes. */
+
+ /* Wait until the status of the mailbox changes to Complete. */
+ while (cnt < MBOX_MSG_POLLING_TIMEOUT) {
+ wb_status = get_mbox_status(send_mbox);
+ if (MBOX_STATUS_FINISHED(wb_status))
+ break;
+
+ rte_delay_us(10);
+ cnt++;
+ }
+
+ if (cnt == MBOX_MSG_POLLING_TIMEOUT) {
+ PMD_DRV_LOG(ERR,
+ "Send mailbox segment timeout, wb status: 0x%x",
+ wb_status);
+ dump_mbox_reg(hwdev);
+ return -ETIMEDOUT;
+ }
+
+ if (!MBOX_STATUS_SUCCESS(wb_status)) {
+ PMD_DRV_LOG(ERR,
+ "Send mailbox segment to function %d error, wb "
+ "status: 0x%x",
+ dst_func, wb_status);
+ errcode = MBOX_STATUS_ERRCODE(wb_status);
+ return errcode ? errcode : -EFAULT;
+ }
+
+ return 0;
+}
+
+static int
+send_tlp_mbox_seg(struct hinic3_mbox *func_to_func, u64 header, u16 dst_func,
+ void *seg, u16 seg_len, __rte_unused void *msg_info)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+ u8 num_aeqs = hwdev->hwif->attr.num_aeqs;
+ u16 dst_aeqn, errcode, wb_status = 0;
+ u16 seq_dir = HINIC3_MSG_HEADER_GET(header, DIRECTION);
+ u32 cnt = 0;
+
+ /* Mbox to mgmt cpu, hardware doesn't care dst aeq id. */
+ if (num_aeqs >= 2)
+ dst_aeqn = (seq_dir == HINIC3_MSG_DIRECT_SEND)
+ ? HINIC3_ASYNC_MSG_AEQ
+ : HINIC3_MBOX_RSP_MSG_AEQ;
+ else
+ dst_aeqn = 0;
+
+ clear_mbox_status(send_mbox);
+ mbox_copy_header(send_mbox, &header);
+
+ /* Copy data to DMA buffer. */
+ memcpy((void *)send_mbox->sbuff_vaddr, (void *)seg, (size_t)seg_len);
+
+ /*
+ * Copy data address to mailbox ctrl CSR(Control and Status Register).
+ */
+ mbox_copy_send_data_addr(send_mbox, seg_len);
+
+ /* Set mailbox msg header size. */
+ write_mbox_msg_attr(func_to_func, dst_func, dst_aeqn,
+ MBOX_TLP_HEADER_SZ);
+
+ rte_wmb(); /**< Writing the mbox msg attributes. */
+
+ /* Wait until the status of the mailbox changes to Complete. */
+ while (cnt < MBOX_MSG_POLLING_TIMEOUT) {
+ wb_status = get_mbox_status(send_mbox);
+ if (MBOX_STATUS_FINISHED(wb_status))
+ break;
+
+ rte_delay_us(10);
+ cnt++;
+ }
+
+ if (cnt == MBOX_MSG_POLLING_TIMEOUT) {
+ PMD_DRV_LOG(ERR,
+ "Send mailbox segment timeout, wb status: 0x%x",
+ wb_status);
+ dump_mbox_reg(hwdev);
+ return -ETIMEDOUT;
+ }
+
+ if (!MBOX_STATUS_SUCCESS(wb_status)) {
+ PMD_DRV_LOG(ERR,
+ "Send mailbox segment to function %d error, wb "
+ "status: 0x%x",
+ dst_func, wb_status);
+ errcode = MBOX_STATUS_ERRCODE(wb_status);
+ return errcode ? errcode : -EFAULT;
+ }
+
+ return 0;
+}
+
+static int
+send_mbox_to_func(struct hinic3_mbox *func_to_func, enum hinic3_mod_type mod,
+ u16 cmd, void *msg, u16 msg_len, u16 dst_func,
+ enum hinic3_msg_direction_type direction,
+ enum hinic3_msg_ack_type ack_type,
+ struct mbox_msg_info *msg_info)
+{
+ int err = 0;
+ u32 seq_id = 0;
+ u16 seg_len = MBOX_SEG_LEN;
+ u16 rsp_aeq_id, left = msg_len;
+ u8 *msg_seg = (u8 *)msg;
+ u64 header = 0;
+
+ rsp_aeq_id = HINIC3_MBOX_RSP_MSG_AEQ;
+
+ err = hinic3_mutex_lock(&func_to_func->msg_send_mutex);
+ if (err)
+ return err;
+
+ /* Set the header message. */
+ header = HINIC3_MSG_HEADER_SET(msg_len, MSG_LEN) |
+ HINIC3_MSG_HEADER_SET(mod, MODULE) |
+ HINIC3_MSG_HEADER_SET(seg_len, SEG_LEN) |
+ HINIC3_MSG_HEADER_SET(ack_type, NO_ACK) |
+ HINIC3_MSG_HEADER_SET(HINIC3_DATA_INLINE, DATA_TYPE) |
+ HINIC3_MSG_HEADER_SET(SEQ_ID_START_VAL, SEQID) |
+ HINIC3_MSG_HEADER_SET(NOT_LAST_SEGMENT, LAST) |
+ HINIC3_MSG_HEADER_SET(direction, DIRECTION) |
+ HINIC3_MSG_HEADER_SET(cmd, CMD) |
+ /* The VF's offset to it's associated PF. */
+ HINIC3_MSG_HEADER_SET(msg_info->msg_id, MSG_ID) |
+ HINIC3_MSG_HEADER_SET(rsp_aeq_id, AEQ_ID) |
+ HINIC3_MSG_HEADER_SET(HINIC3_MSG_FROM_MBOX, SOURCE) |
+ HINIC3_MSG_HEADER_SET(!!msg_info->status, STATUS);
+ /* Loop until all messages are sent. */
+ while (!(HINIC3_MSG_HEADER_GET(header, LAST))) {
+ if (left <= MBOX_SEG_LEN) {
+ header &= ~MBOX_SEGLEN_MASK;
+ header |= HINIC3_MSG_HEADER_SET(left, SEG_LEN);
+ header |= HINIC3_MSG_HEADER_SET(LAST_SEGMENT, LAST);
+
+ seg_len = left;
+ }
+
+ err = send_mbox_seg(func_to_func, header, dst_func, msg_seg,
+ seg_len, msg_info);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Send mbox seg failed, seq_id: 0x%" PRIx64,
+ HINIC3_MSG_HEADER_GET(header, SEQID));
+
+ goto send_err;
+ }
+
+ left -= MBOX_SEG_LEN;
+ msg_seg += MBOX_SEG_LEN;
+
+ seq_id++;
+ header &= ~(HINIC3_MSG_HEADER_SET(HINIC3_MSG_HEADER_SEQID_MASK,
+ SEQID));
+ header |= HINIC3_MSG_HEADER_SET(seq_id, SEQID);
+ }
+
+send_err:
+ (void)hinic3_mutex_unlock(&func_to_func->msg_send_mutex);
+
+ return err;
+}
+
+static int
+send_tlp_mbox_to_func(struct hinic3_mbox *func_to_func,
+ enum hinic3_mod_type mod, u16 cmd, void *msg, u16 msg_len,
+ u16 dst_func, enum hinic3_msg_direction_type direction,
+ enum hinic3_msg_ack_type ack_type,
+ struct mbox_msg_info *msg_info)
+{
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+ u8 *msg_seg = (u8 *)msg;
+ int err = 0;
+ u16 rsp_aeq_id;
+ u64 header = 0;
+
+ rsp_aeq_id = HINIC3_MBOX_RSP_MSG_AEQ;
+
+ err = hinic3_mutex_lock(&func_to_func->msg_send_mutex);
+ if (err)
+ return err;
+
+ /* Set the header message. */
+ header = HINIC3_MSG_HEADER_SET(MBOX_TLP_HEADER_SZ, MSG_LEN) |
+ HINIC3_MSG_HEADER_SET(MBOX_TLP_HEADER_SZ, SEG_LEN) |
+ HINIC3_MSG_HEADER_SET(mod, MODULE) |
+ HINIC3_MSG_HEADER_SET(LAST_SEGMENT, LAST) |
+ HINIC3_MSG_HEADER_SET(ack_type, NO_ACK) |
+ HINIC3_MSG_HEADER_SET(HINIC3_DATA_DMA, DATA_TYPE) |
+ HINIC3_MSG_HEADER_SET(SEQ_ID_START_VAL, SEQID) |
+ HINIC3_MSG_HEADER_SET(direction, DIRECTION) |
+ HINIC3_MSG_HEADER_SET(cmd, CMD) |
+ /* The VF's offset to it's associated PF. */
+ HINIC3_MSG_HEADER_SET(msg_info->msg_id, MSG_ID) |
+ HINIC3_MSG_HEADER_SET(rsp_aeq_id, AEQ_ID) |
+ HINIC3_MSG_HEADER_SET(HINIC3_MSG_FROM_MBOX, SOURCE) |
+ HINIC3_MSG_HEADER_SET(!!msg_info->status, STATUS) |
+ HINIC3_MSG_HEADER_SET(hinic3_global_func_id(hwdev),
+ SRC_GLB_FUNC_IDX);
+
+ /* Send a message. */
+ err = send_tlp_mbox_seg(func_to_func, header, dst_func, msg_seg,
+ msg_len, msg_info);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Send mbox seg failed, seq_id: 0x%" PRIx64,
+ HINIC3_MSG_HEADER_GET(header, SEQID));
+ }
+
+ (void)hinic3_mutex_unlock(&func_to_func->msg_send_mutex);
+
+ return err;
+}
+
+/**
+ * Set mailbox F2F(Function to Function) event status.
+ *
+ * @param[out] func_to_func
+ * Context for inter-function communication.
+ * @param[in] event_flag
+ * Event status enumerated value.
+ */
+static void
+set_mbox_to_func_event(struct hinic3_mbox *func_to_func,
+ enum mbox_event_state event_flag)
+{
+ rte_spinlock_lock(&func_to_func->mbox_lock);
+ func_to_func->event_flag = event_flag;
+ rte_spinlock_unlock(&func_to_func->mbox_lock);
+}
+
+/**
+ * Send data from one function to another and receive responses.
+ *
+ * @param[in] func_to_func
+ * Context for inter-function communication.
+ * @param[in] mod
+ * Command queue module type.
+ * @param[in] cmd
+ * Indicate the command to be executed.
+ the command to be executed.
+ * @param[in] dst_func
+ * Indicate destination func.
+ * @param[in] buf_in
+ * Pointer to the input buffer.
+ * @param[in] in_size
+ * Input buffer size.
+ * @param[out] buf_out
+ * Pointer to the output buffer.
+ * @param[out] out_size
+ * Output buffer size.
+ * @param[in] timeout
+ * Timeout interval for waiting for a response.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_mbox_to_func(struct hinic3_mbox *func_to_func, enum hinic3_mod_type mod,
+ u16 cmd, u16 dst_func, void *buf_in, u16 in_size,
+ void *buf_out, u16 *out_size, u32 timeout)
+{
+ /* Use mbox_resp to hole data which responsed from other function. */
+ struct hinic3_recv_mbox *mbox_for_resp = NULL;
+ struct mbox_msg_info msg_info = {0};
+ struct hinic3_eq *aeq = NULL;
+ u16 mbox_rsp_idx;
+ u32 time;
+ int err;
+
+ mbox_rsp_idx = (u16)hinic3_mbox_get_index(dst_func);
+ mbox_for_resp = &func_to_func->mbox_resp[mbox_rsp_idx];
+
+ err = hinic3_mutex_lock(&func_to_func->mbox_send_mutex);
+ if (err)
+ return err;
+
+ /* Set message ID and start event. */
+ msg_info.msg_id = MBOX_MSG_ID_INC(func_to_func);
+ set_mbox_to_func_event(func_to_func, EVENT_START);
+
+ /* Select a function to send messages based on the dst_func type. */
+ if (IS_TLP_MBX(dst_func))
+ err = send_tlp_mbox_to_func(func_to_func,
+ mod, cmd, buf_in, in_size, dst_func,
+ HINIC3_MSG_DIRECT_SEND, HINIC3_MSG_ACK, &msg_info);
+ else
+ err = send_mbox_to_func(func_to_func, mod, cmd, buf_in, in_size,
+ dst_func, HINIC3_MSG_DIRECT_SEND,
+ HINIC3_MSG_ACK, &msg_info);
+
+ if (err) {
+ PMD_DRV_LOG(ERR, "Send mailbox failed, msg_id: %d",
+ msg_info.msg_id);
+ set_mbox_to_func_event(func_to_func, EVENT_FAIL);
+ goto send_err;
+ }
+
+ /* Wait for the response message. */
+ time = msecs_to_jiffies(timeout ? timeout : HINIC3_MBOX_COMP_TIME);
+ aeq = &func_to_func->hwdev->aeqs->aeq[HINIC3_MBOX_RSP_MSG_AEQ];
+ err = hinic3_aeq_poll_msg(aeq, time, NULL);
+ if (err) {
+ set_mbox_to_func_event(func_to_func, EVENT_TIMEOUT);
+ PMD_DRV_LOG(ERR, "Send mailbox message time out");
+ err = -ETIMEDOUT;
+ goto send_err;
+ }
+
+ /* Check whether mod and command of the rsp message match the sent message. */
+ if (mod != mbox_for_resp->mod || cmd != mbox_for_resp->cmd) {
+ PMD_DRV_LOG(ERR,
+ "Invalid response mbox message, mod: 0x%x, cmd: "
+ "0x%x, expect mod: 0x%x, cmd: 0x%x",
+ mbox_for_resp->mod, mbox_for_resp->cmd, mod, cmd);
+ err = -EFAULT;
+ goto send_err;
+ }
+
+ /* Check the response status. */
+ if (mbox_for_resp->msg_info.status) {
+ err = mbox_for_resp->msg_info.status;
+ goto send_err;
+ }
+
+ /* Check whether the length of the response message is valid. */
+ if (buf_out && out_size) {
+ if (*out_size < mbox_for_resp->mbox_len) {
+ PMD_DRV_LOG(ERR,
+ "Invalid response mbox message length: %d for "
+ "mod: %d cmd: %d, should less than: %d",
+ mbox_for_resp->mbox_len, mod, cmd, *out_size);
+ err = -EFAULT;
+ goto send_err;
+ }
+
+ if (mbox_for_resp->mbox_len)
+ memcpy(buf_out, mbox_for_resp->mbox,
+ (size_t)(mbox_for_resp->mbox_len));
+
+ *out_size = mbox_for_resp->mbox_len;
+ }
+
+send_err:
+ (void)hinic3_mutex_unlock(&func_to_func->mbox_send_mutex);
+
+ return err;
+}
+
+static int
+mbox_func_params_valid(__rte_unused struct hinic3_mbox *func_to_func,
+ void *buf_in, u16 in_size)
+{
+ if (!buf_in || !in_size)
+ return -EINVAL;
+
+ if (in_size > HINIC3_MBOX_DATA_SIZE) {
+ PMD_DRV_LOG(ERR, "Mbox msg len(%d) exceed limit(%" PRIu64 ")",
+ in_size, HINIC3_MBOX_DATA_SIZE);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+hinic3_mbox_to_func_no_ack(struct hinic3_hwdev *hwdev, u16 func_idx,
+ enum hinic3_mod_type mod, u16 cmd, void *buf_in,
+ u16 in_size)
+{
+ struct hinic3_mbox *func_to_func = hwdev->func_to_func;
+ struct mbox_msg_info msg_info = {0};
+ int err;
+
+ err = mbox_func_params_valid(hwdev->func_to_func, buf_in, in_size);
+ if (err)
+ return err;
+
+ err = hinic3_mutex_lock(&func_to_func->mbox_send_mutex);
+ if (err)
+ return err;
+
+ if (IS_TLP_MBX(func_idx))
+ err = send_tlp_mbox_to_func(func_to_func,
+ mod, cmd, buf_in, in_size, func_idx,
+ HINIC3_MSG_DIRECT_SEND, HINIC3_MSG_NO_ACK, &msg_info);
+ else
+ err = send_mbox_to_func(func_to_func, mod, cmd, buf_in, in_size,
+ func_idx, HINIC3_MSG_DIRECT_SEND,
+ HINIC3_MSG_NO_ACK, &msg_info);
+ if (err)
+ PMD_DRV_LOG(ERR, "Send mailbox no ack failed");
+
+ (void)hinic3_mutex_unlock(&func_to_func->mbox_send_mutex);
+
+ return err;
+}
+
+int
+hinic3_send_mbox_to_mgmt(struct hinic3_hwdev *hwdev, enum hinic3_mod_type mod,
+ u16 cmd, void *buf_in, u16 in_size, void *buf_out,
+ u16 *out_size, u32 timeout)
+{
+ struct hinic3_mbox *func_to_func = hwdev->func_to_func;
+ int err;
+ /* Verify the validity of the input parameters. */
+ err = mbox_func_params_valid(func_to_func, buf_in, in_size);
+ if (err)
+ return err;
+
+ return hinic3_mbox_to_func(func_to_func, mod, cmd, HINIC3_MGMT_SRC_ID,
+ buf_in, in_size, buf_out, out_size, timeout);
+}
+
+void
+hinic3_response_mbox_to_mgmt(struct hinic3_hwdev *hwdev,
+ enum hinic3_mod_type mod, u16 cmd, void *buf_in,
+ u16 in_size, u16 msg_id)
+{
+ struct mbox_msg_info msg_info;
+ u16 dst_func;
+
+ msg_info.msg_id = (u8)msg_id;
+ msg_info.status = 0;
+ dst_func = HINIC3_MGMT_SRC_ID;
+
+ if (IS_TLP_MBX(dst_func))
+ send_tlp_mbox_to_func(hwdev->func_to_func, mod, cmd, buf_in,
+ in_size, HINIC3_MGMT_SRC_ID,
+ HINIC3_MSG_RESPONSE, HINIC3_MSG_NO_ACK,
+ &msg_info);
+ else
+ send_mbox_to_func(hwdev->func_to_func, mod, cmd, buf_in,
+ in_size, HINIC3_MGMT_SRC_ID,
+ HINIC3_MSG_RESPONSE, HINIC3_MSG_NO_ACK,
+ &msg_info);
+}
+
+int
+hinic3_send_mbox_to_mgmt_no_ack(struct hinic3_hwdev *hwdev,
+ enum hinic3_mod_type mod, u16 cmd, void *buf_in,
+ u16 in_size)
+{
+ struct hinic3_mbox *func_to_func = hwdev->func_to_func;
+ int err;
+
+ err = mbox_func_params_valid(func_to_func, buf_in, in_size);
+ if (err)
+ return err;
+
+ return hinic3_mbox_to_func_no_ack(hwdev, HINIC3_MGMT_SRC_ID, mod, cmd,
+ buf_in, in_size);
+}
+
+int
+hinic3_mbox_to_pf(struct hinic3_hwdev *hwdev, enum hinic3_mod_type mod, u16 cmd,
+ void *buf_in, u16 in_size, void *buf_out, u16 *out_size,
+ u32 timeout)
+{
+ int err;
+
+ if (!hwdev)
+ return -EINVAL;
+
+ /* Check the validity of parameters. */
+ err = mbox_func_params_valid(hwdev->func_to_func, buf_in, in_size);
+ if (err)
+ return err;
+
+ if (!HINIC3_IS_VF(hwdev)) {
+ PMD_DRV_LOG(ERR, "Params error, func_type: %d",
+ hinic3_func_type(hwdev));
+ return -EINVAL;
+ }
+
+ /* Sending Email to PF. */
+ return hinic3_mbox_to_func(hwdev->func_to_func, mod, cmd,
+ hinic3_pf_id_of_vf(hwdev), buf_in, in_size,
+ buf_out, out_size, timeout);
+}
+
+int
+hinic3_mbox_to_vf(struct hinic3_hwdev *hwdev, enum hinic3_mod_type mod,
+ u16 vf_id, u16 cmd, void *buf_in, u16 in_size, void *buf_out,
+ u16 *out_size, u32 timeout)
+{
+ struct hinic3_mbox *func_to_func = NULL;
+ u16 dst_func_idx;
+ int err = 0;
+
+ if (!hwdev)
+ return -EINVAL;
+
+ func_to_func = hwdev->func_to_func;
+ err = mbox_func_params_valid(func_to_func, buf_in, in_size);
+ if (err)
+ return err;
+
+ if (HINIC3_IS_VF(hwdev)) {
+ PMD_DRV_LOG(ERR, "Params error, func_type: %d",
+ hinic3_func_type(hwdev));
+ return -EINVAL;
+ }
+
+ if (!vf_id) {
+ PMD_DRV_LOG(ERR, "VF id: %d error!", vf_id);
+ return -EINVAL;
+ }
+
+ /*
+ * The sum of vf_offset_to_pf + vf_id is the VF's global function id of
+ * VF in this pf.
+ */
+ dst_func_idx = hinic3_glb_pf_vf_offset(hwdev) + vf_id;
+
+ return hinic3_mbox_to_func(func_to_func, mod, cmd, dst_func_idx, buf_in,
+ in_size, buf_out, out_size, timeout);
+}
+
+static int
+init_mbox_info(struct hinic3_recv_mbox *mbox_info, int mbox_max_buf_sz)
+{
+ int err;
+
+ mbox_info->seq_id = SEQ_ID_MAX_VAL;
+
+ mbox_info->mbox =
+ rte_zmalloc("mbox", (size_t)mbox_max_buf_sz, 1); /*lint !e571*/
+ if (!mbox_info->mbox)
+ return -ENOMEM;
+
+ mbox_info->buf_out = rte_zmalloc("mbox_buf_out",
+ (size_t)mbox_max_buf_sz, 1); /*lint !e571*/
+ if (!mbox_info->buf_out) {
+ err = -ENOMEM;
+ goto alloc_buf_out_err;
+ }
+
+ return 0;
+
+alloc_buf_out_err:
+ rte_free(mbox_info->mbox);
+
+ return err;
+}
+
+static void
+clean_mbox_info(struct hinic3_recv_mbox *mbox_info)
+{
+ rte_free(mbox_info->buf_out);
+ rte_free(mbox_info->mbox);
+}
+
+static int
+alloc_mbox_info(struct hinic3_recv_mbox *mbox_info, int mbox_max_buf_sz)
+{
+ u16 func_idx, i;
+ int err;
+
+ for (func_idx = 0; func_idx < HINIC3_MAX_FUNCTIONS + 1; func_idx++) {
+ err = init_mbox_info(&mbox_info[func_idx], mbox_max_buf_sz);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init mbox info failed");
+ goto init_mbox_info_err;
+ }
+ }
+
+ return 0;
+
+init_mbox_info_err:
+ for (i = 0; i < func_idx; i++)
+ clean_mbox_info(&mbox_info[i]);
+
+ return err;
+}
+
+static void
+free_mbox_info(struct hinic3_recv_mbox *mbox_info)
+{
+ u16 func_idx;
+
+ for (func_idx = 0; func_idx < HINIC3_MAX_FUNCTIONS + 1; func_idx++)
+ clean_mbox_info(&mbox_info[func_idx]);
+}
+
+static void
+prepare_send_mbox(struct hinic3_mbox *func_to_func)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+
+ send_mbox->data = MBOX_AREA(func_to_func->hwdev->hwif);
+}
+
+/**
+ * Allocate memory for the write-back state of the mailbox and write to
+ * register.
+ *
+ * @param[in] func_to_func
+ * Context for inter-function communication.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+alloc_mbox_wb_status(struct hinic3_mbox *func_to_func)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+ u32 addr_h, addr_l;
+
+ /* Reserved DMA area. */
+ send_mbox->wb_mz = hinic3_dma_zone_reserve(hwdev->eth_dev,
+ "wb_mz", 0, MBOX_WB_STATUS_LEN,
+ RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+ if (!send_mbox->wb_mz)
+ return -ENOMEM;
+
+ send_mbox->wb_vaddr = send_mbox->wb_mz->addr;
+ send_mbox->wb_paddr = send_mbox->wb_mz->iova;
+ send_mbox->wb_status = send_mbox->wb_vaddr;
+
+ addr_h = upper_32_bits(send_mbox->wb_paddr);
+ addr_l = lower_32_bits(send_mbox->wb_paddr);
+
+ /* Write info to the register. */
+ hinic3_hwif_write_reg(hwdev->hwif, HINIC3_FUNC_CSR_MAILBOX_RESULT_H_OFF,
+ addr_h);
+ hinic3_hwif_write_reg(hwdev->hwif, HINIC3_FUNC_CSR_MAILBOX_RESULT_L_OFF,
+ addr_l);
+
+ return 0;
+}
+
+static void
+free_mbox_wb_status(struct hinic3_mbox *func_to_func)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+
+ hinic3_hwif_write_reg(hwdev->hwif, HINIC3_FUNC_CSR_MAILBOX_RESULT_H_OFF,
+ 0);
+ hinic3_hwif_write_reg(hwdev->hwif, HINIC3_FUNC_CSR_MAILBOX_RESULT_L_OFF,
+ 0);
+
+ hinic3_memzone_free(send_mbox->wb_mz);
+}
+
+static int
+alloc_mbox_tlp_buffer(struct hinic3_mbox *func_to_func)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+ struct hinic3_hwdev *hwdev = func_to_func->hwdev;
+
+ send_mbox->sbuff_mz = hinic3_dma_zone_reserve(hwdev->eth_dev,
+ "sbuff_mz", 0, MBOX_MAX_BUF_SZ, MBOX_MAX_BUF_SZ,
+ SOCKET_ID_ANY);
+ if (!send_mbox->sbuff_mz)
+ return -ENOMEM;
+
+ send_mbox->sbuff_vaddr = send_mbox->sbuff_mz->addr;
+ send_mbox->sbuff_paddr = send_mbox->sbuff_mz->iova;
+
+ return 0;
+}
+
+static void
+free_mbox_tlp_buffer(struct hinic3_mbox *func_to_func)
+{
+ struct hinic3_send_mbox *send_mbox = &func_to_func->send_mbox;
+
+ hinic3_memzone_free(send_mbox->sbuff_mz);
+}
+
+/**
+ * Initialize function to function communication.
+ *
+ * @param[in] hwdev
+ * Pointer to hardware device structure.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int
+hinic3_func_to_func_init(struct hinic3_hwdev *hwdev)
+{
+ struct hinic3_mbox *func_to_func;
+ int err;
+
+ func_to_func = rte_zmalloc("func_to_func", sizeof(*func_to_func), 1);
+ if (!func_to_func)
+ return -ENOMEM;
+
+ hwdev->func_to_func = func_to_func;
+ func_to_func->hwdev = hwdev;
+ (void)hinic3_mutex_init(&func_to_func->mbox_send_mutex, NULL);
+ (void)hinic3_mutex_init(&func_to_func->msg_send_mutex, NULL);
+ rte_spinlock_init(&func_to_func->mbox_lock);
+
+ /* Alloc the memory required by the mailbox. */
+ err = alloc_mbox_info(func_to_func->mbox_send, MBOX_MAX_BUF_SZ);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc mem for mbox_active failed");
+ goto alloc_mbox_for_send_err;
+ }
+
+ err = alloc_mbox_info(func_to_func->mbox_resp, MBOX_MAX_BUF_SZ);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc mem for mbox_passive failed");
+ goto alloc_mbox_for_resp_err;
+ }
+
+ err = alloc_mbox_tlp_buffer(func_to_func);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc mbox send buffer failed");
+ goto alloc_tlp_buffer_err;
+ }
+
+ err = alloc_mbox_wb_status(func_to_func);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc mbox write back status failed");
+ goto alloc_wb_status_err;
+ }
+
+ prepare_send_mbox(func_to_func);
+
+ return 0;
+
+alloc_wb_status_err:
+ free_mbox_tlp_buffer(func_to_func);
+
+alloc_tlp_buffer_err:
+ free_mbox_info(func_to_func->mbox_resp);
+
+alloc_mbox_for_resp_err:
+ free_mbox_info(func_to_func->mbox_send);
+
+alloc_mbox_for_send_err:
+ (void)hinic3_mutex_destroy(&func_to_func->msg_send_mutex);
+ (void)hinic3_mutex_destroy(&func_to_func->mbox_send_mutex);
+ rte_free(func_to_func);
+
+ return err;
+}
+
+void
+hinic3_func_to_func_free(struct hinic3_hwdev *hwdev)
+{
+ struct hinic3_mbox *func_to_func = hwdev->func_to_func;
+
+ free_mbox_wb_status(func_to_func);
+ free_mbox_tlp_buffer(func_to_func);
+ free_mbox_info(func_to_func->mbox_resp);
+ free_mbox_info(func_to_func->mbox_send);
+ (void)hinic3_mutex_destroy(&func_to_func->mbox_send_mutex);
+ (void)hinic3_mutex_destroy(&func_to_func->msg_send_mutex);
+
+ rte_free(func_to_func);
+}
diff --git a/drivers/net/hinic3/base/hinic3_mbox.h b/drivers/net/hinic3/base/hinic3_mbox.h
new file mode 100644
index 0000000000..eaf315952f
--- /dev/null
+++ b/drivers/net/hinic3/base/hinic3_mbox.h
@@ -0,0 +1,199 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_MBOX_H_
+#define _HINIC3_MBOX_H_
+
+#include "hinic3_mgmt.h"
+
+#define HINIC3_MBOX_PF_SEND_ERR 0x1
+#define HINIC3_MBOX_PF_BUSY_ACTIVE_FW 0x2
+#define HINIC3_MBOX_VF_CMD_ERROR 0x3
+
+#define HINIC3_MGMT_SRC_ID 0x1FFF
+
+#define HINIC3_MAX_PF_FUNCS 32
+
+/* Message header define. */
+#define HINIC3_MSG_HEADER_SRC_GLB_FUNC_IDX_SHIFT 0
+#define HINIC3_MSG_HEADER_STATUS_SHIFT 13
+#define HINIC3_MSG_HEADER_SOURCE_SHIFT 15
+#define HINIC3_MSG_HEADER_AEQ_ID_SHIFT 16
+#define HINIC3_MSG_HEADER_MSG_ID_SHIFT 18
+#define HINIC3_MSG_HEADER_CMD_SHIFT 22
+
+#define HINIC3_MSG_HEADER_MSG_LEN_SHIFT 32
+#define HINIC3_MSG_HEADER_MODULE_SHIFT 43
+#define HINIC3_MSG_HEADER_SEG_LEN_SHIFT 48
+#define HINIC3_MSG_HEADER_NO_ACK_SHIFT 54
+#define HINIC3_MSG_HEADER_DATA_TYPE_SHIFT 55
+#define HINIC3_MSG_HEADER_SEQID_SHIFT 56
+#define HINIC3_MSG_HEADER_LAST_SHIFT 62
+#define HINIC3_MSG_HEADER_DIRECTION_SHIFT 63
+
+#define HINIC3_MSG_HEADER_CMD_MASK 0x3FF
+#define HINIC3_MSG_HEADER_MSG_ID_MASK 0xF
+#define HINIC3_MSG_HEADER_AEQ_ID_MASK 0x3
+#define HINIC3_MSG_HEADER_SOURCE_MASK 0x1
+#define HINIC3_MSG_HEADER_STATUS_MASK 0x1
+#define HINIC3_MSG_HEADER_SRC_GLB_FUNC_IDX_MASK 0x1FFF
+
+#define HINIC3_MSG_HEADER_MSG_LEN_MASK 0x7FF
+#define HINIC3_MSG_HEADER_MODULE_MASK 0x1F
+#define HINIC3_MSG_HEADER_SEG_LEN_MASK 0x3F
+#define HINIC3_MSG_HEADER_NO_ACK_MASK 0x1
+#define HINIC3_MSG_HEADER_DATA_TYPE_MASK 0x1
+#define HINIC3_MSG_HEADER_SEQID_MASK 0x3F
+#define HINIC3_MSG_HEADER_LAST_MASK 0x1
+#define HINIC3_MSG_HEADER_DIRECTION_MASK 0x1
+
+#define HINIC3_MSG_HEADER_GET(val, field) \
+ (((val) >> HINIC3_MSG_HEADER_##field##_SHIFT) & \
+ HINIC3_MSG_HEADER_##field##_MASK)
+#define HINIC3_MSG_HEADER_SET(val, field) \
+ ((u64)(((u64)(val)) & HINIC3_MSG_HEADER_##field##_MASK) \
+ << HINIC3_MSG_HEADER_##field##_SHIFT)
+
+#define IS_TLP_MBX(dst_func) ((dst_func) == HINIC3_MGMT_SRC_ID)
+
+enum hinic3_msg_direction_type {
+ HINIC3_MSG_DIRECT_SEND = 0,
+ HINIC3_MSG_RESPONSE = 1
+};
+
+enum hinic3_msg_segment_type { NOT_LAST_SEGMENT = 0, LAST_SEGMENT = 1 };
+
+enum hinic3_msg_ack_type { HINIC3_MSG_ACK, HINIC3_MSG_NO_ACK };
+
+enum hinic3_data_type { HINIC3_DATA_INLINE = 0, HINIC3_DATA_DMA = 1 };
+
+enum hinic3_msg_src_type { HINIC3_MSG_FROM_MGMT = 0, HINIC3_MSG_FROM_MBOX = 1 };
+
+enum hinic3_msg_aeq_type {
+ HINIC3_ASYNC_MSG_AEQ = 0,
+ /* Indicate dst_func or mgmt cpu which aeq to response mbox message. */
+ HINIC3_MBOX_RSP_MSG_AEQ = 1,
+ /* Indicate mgmt cpu which aeq to response api cmd message. */
+ HINIC3_MGMT_RSP_MSG_AEQ = 2
+};
+
+enum hinic3_mbox_seg_errcode {
+ MBOX_ERRCODE_NO_ERRORS = 0,
+ /* VF sends the mailbox data to the wrong destination functions. */
+ MBOX_ERRCODE_VF_TO_WRONG_FUNC = 0x100,
+ /* PPF sends the mailbox data to the wrong destination functions. */
+ MBOX_ERRCODE_PPF_TO_WRONG_FUNC = 0x200,
+ /* PF sends the mailbox data to the wrong destination functions. */
+ MBOX_ERRCODE_PF_TO_WRONG_FUNC = 0x300,
+ /* The mailbox data size is set to all zero. */
+ MBOX_ERRCODE_ZERO_DATA_SIZE = 0x400,
+ /* The sender func attribute has not been learned by CPI hardware. */
+ MBOX_ERRCODE_UNKNOWN_SRC_FUNC = 0x500,
+ /* The receiver func attr has not been learned by CPI hardware. */
+ MBOX_ERRCODE_UNKNOWN_DES_FUNC = 0x600
+};
+
+enum hinic3_mbox_func_index {
+ HINIC3_MBOX_MPU_INDEX = 0,
+ HINIC3_MBOX_PF_INDEX = 1,
+ HINIC3_MAX_FUNCTIONS = 2,
+};
+
+struct mbox_msg_info {
+ u8 msg_id;
+ u8 status; /**< Can only use 3 bit. */
+};
+
+struct hinic3_recv_mbox {
+ void *mbox;
+ u16 cmd;
+ enum hinic3_mod_type mod;
+ u16 mbox_len;
+ void *buf_out;
+ enum hinic3_msg_ack_type ack_type;
+ struct mbox_msg_info msg_info;
+ u8 seq_id;
+ RTE_ATOMIC(int32_t)msg_cnt;
+};
+
+struct hinic3_send_mbox {
+ u8 *data;
+ u64 *wb_status; /**< Write back status. */
+
+ const struct rte_memzone *wb_mz;
+ void *wb_vaddr; /**< Write back virtual address. */
+ rte_iova_t wb_paddr; /**< Write back physical address. */
+
+ const struct rte_memzone *sbuff_mz;
+ void *sbuff_vaddr;
+ rte_iova_t sbuff_paddr;
+};
+
+enum mbox_event_state {
+ EVENT_START = 0,
+ EVENT_FAIL,
+ EVENT_SUCCESS,
+ EVENT_TIMEOUT,
+ EVENT_END
+};
+
+/* Execution status of the callback function. */
+enum hinic3_mbox_cb_state {
+ HINIC3_VF_MBOX_CB_REG = 0,
+ HINIC3_VF_MBOX_CB_RUNNING,
+ HINIC3_PF_MBOX_CB_REG,
+ HINIC3_PF_MBOX_CB_RUNNING,
+ HINIC3_PPF_MBOX_CB_REG,
+ HINIC3_PPF_MBOX_CB_RUNNING,
+ HINIC3_PPF_TO_PF_MBOX_CB_REG,
+ HINIC3_PPF_TO_PF_MBOX_CB_RUNNIG
+};
+
+struct hinic3_mbox {
+ struct hinic3_hwdev *hwdev;
+
+ pthread_mutex_t mbox_send_mutex;
+ pthread_mutex_t msg_send_mutex;
+
+ struct hinic3_send_mbox send_mbox;
+
+ /* Last element for mgmt. */
+ struct hinic3_recv_mbox mbox_resp[HINIC3_MAX_FUNCTIONS + 1];
+ struct hinic3_recv_mbox mbox_send[HINIC3_MAX_FUNCTIONS + 1];
+
+ u8 send_msg_id;
+ enum mbox_event_state event_flag;
+ /* Lock for mbox event flag. */
+ rte_spinlock_t mbox_lock;
+};
+
+int hinic3_mbox_func_aeqe_handler(void *handle, u8 *header,
+ __rte_unused u8 size, void *param);
+
+int hinic3_func_to_func_init(struct hinic3_hwdev *hwdev);
+
+void hinic3_func_to_func_free(struct hinic3_hwdev *hwdev);
+
+int hinic3_send_mbox_to_mgmt(struct hinic3_hwdev *hwdev,
+ enum hinic3_mod_type mod, u16 cmd, void *buf_in,
+ u16 in_size, void *buf_out, u16 *out_size,
+ u32 timeout);
+
+void hinic3_response_mbox_to_mgmt(struct hinic3_hwdev *hwdev,
+ enum hinic3_mod_type mod, u16 cmd,
+ void *buf_in, u16 in_size, u16 msg_id);
+
+int hinic3_send_mbox_to_mgmt_no_ack(struct hinic3_hwdev *hwdev,
+ enum hinic3_mod_type mod, u16 cmd,
+ void *buf_in, u16 in_size);
+
+int hinic3_mbox_to_pf(struct hinic3_hwdev *hwdev, enum hinic3_mod_type mod,
+ u16 cmd, void *buf_in, u16 in_size, void *buf_out,
+ u16 *out_size, u32 timeout);
+
+int hinic3_mbox_to_vf(struct hinic3_hwdev *hwdev, enum hinic3_mod_type mod,
+ u16 vf_id, u16 cmd, void *buf_in, u16 in_size,
+ void *buf_out, u16 *out_size, u32 timeout);
+
+#endif /**< _HINIC3_MBOX_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 12/18] net/hinic3: add device initailization
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (2 preceding siblings ...)
2025-04-18 7:02 ` [RFC 11/18] net/hinic3: add a mailbox communication module Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 13/18] net/hinic3: add dev ops Feifei Wang
` (6 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Xin Wang, Feifei Wang, Yi Chen
From: Xin Wang <wangxin679@h-partners.com>
This patch contains data structures and function codes
related to device initialization.
Signed-off-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
Reviewed-by: Yi Chen <chenyi221@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 514 +++++++++++++++++++++++++++++
drivers/net/hinic3/hinic3_ethdev.h | 119 +++++++
2 files changed, 633 insertions(+)
create mode 100644 drivers/net/hinic3/hinic3_ethdev.c
create mode 100644 drivers/net/hinic3/hinic3_ethdev.h
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
new file mode 100644
index 0000000000..c4b2f5ffe4
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -0,0 +1,514 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include <rte_pci.h>
+#include <rte_bus_pci.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_errno.h>
+#include <rte_ether.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_csr.h"
+#include "base/hinic3_wq.h"
+#include "base/hinic3_eqs.h"
+#include "base/hinic3_cmdq.h"
+#include "base/hinic3_hwdev.h"
+#include "base/hinic3_hwif.h"
+#include "base/hinic3_hw_cfg.h"
+#include "base/hinic3_hw_comm.h"
+#include "base/hinic3_nic_cfg.h"
+#include "base/hinic3_nic_event.h"
+#include "hinic3_ethdev.h"
+
+/**
+ * Interrupt handler triggered by NIC for handling specific event.
+ *
+ * @param[in] param
+ * The address of parameter (struct rte_eth_dev *) regsitered before.
+ */
+static void
+hinic3_dev_interrupt_handler(void *param)
+{
+ struct rte_eth_dev *dev = param;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (!hinic3_get_bit(HINIC3_DEV_INTR_EN, &nic_dev->dev_status)) {
+ PMD_DRV_LOG(WARNING,
+ "Intr is disabled, ignore intr event, "
+ "dev_name: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return;
+ }
+
+ /* Aeq0 msg handler. */
+ hinic3_dev_handle_aeq_event(nic_dev->hwdev, param);
+}
+
+static void
+hinic3_deinit_sw_rxtxqs(struct hinic3_nic_dev *nic_dev)
+{
+ rte_free(nic_dev->txqs);
+ nic_dev->txqs = NULL;
+
+ rte_free(nic_dev->rxqs);
+ nic_dev->rxqs = NULL;
+}
+
+/**
+ * Init mac_vlan table in hardwares.
+ *
+ * @param[in] eth_dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_init_mac_table(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev =
+ HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ u8 addr_bytes[RTE_ETHER_ADDR_LEN];
+ u16 func_id = 0;
+ int err = 0;
+
+ err = hinic3_get_default_mac(nic_dev->hwdev, addr_bytes,
+ RTE_ETHER_ADDR_LEN);
+ if (err)
+ return err;
+
+ rte_ether_addr_copy((struct rte_ether_addr *)addr_bytes,
+ ð_dev->data->mac_addrs[0]);
+ if (rte_is_zero_ether_addr(ð_dev->data->mac_addrs[0]))
+ rte_eth_random_addr(eth_dev->data->mac_addrs[0].addr_bytes);
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+ err = hinic3_set_mac(nic_dev->hwdev,
+ eth_dev->data->mac_addrs[0].addr_bytes, 0,
+ func_id);
+ if (err && err != HINIC3_PF_SET_VF_ALREADY)
+ return err;
+
+ rte_ether_addr_copy(ð_dev->data->mac_addrs[0],
+ &nic_dev->default_addr);
+
+ return 0;
+}
+
+/**
+ * Deinit mac_vlan table in hardware.
+ *
+ * @param[in] eth_dev
+ * Pointer to ethernet device structure.
+ */
+static void
+hinic3_deinit_mac_addr(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev =
+ HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ u16 func_id = 0;
+ int err;
+ int i;
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+
+ for (i = 0; i < HINIC3_MAX_UC_MAC_ADDRS; i++) {
+ if (rte_is_zero_ether_addr(ð_dev->data->mac_addrs[i]))
+ continue;
+
+ err = hinic3_del_mac(nic_dev->hwdev,
+ eth_dev->data->mac_addrs[i].addr_bytes, 0,
+ func_id);
+ if (err && err != HINIC3_PF_SET_VF_ALREADY)
+ PMD_DRV_LOG(ERR,
+ "Delete mac table failed, dev_name: %s",
+ eth_dev->data->name);
+
+ memset(ð_dev->data->mac_addrs[i], 0,
+ sizeof(struct rte_ether_addr));
+ }
+
+ /* Delete multicast mac addrs. */
+ hinic3_delete_mc_addr_list(nic_dev);
+}
+
+/**
+ * Check the valid CoS bitmap to determine the available CoS IDs and set
+ * the default CoS ID to the highest valid one.
+ *
+ * @param[in] hwdev
+ * Pointer to hardware device structure.
+ * @param[out] cos_id
+ * Pointer to store the default CoS ID.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_pf_get_default_cos(struct hinic3_hwdev *hwdev, u8 *cos_id)
+{
+ u8 default_cos = 0;
+ u8 valid_cos_bitmap;
+ u8 i;
+
+ valid_cos_bitmap = hwdev->cfg_mgmt->svc_cap.cos_valid_bitmap;
+ if (!valid_cos_bitmap) {
+ PMD_DRV_LOG(ERR, "PF has none cos to support");
+ return -EFAULT;
+ }
+
+ for (i = 0; i < HINIC3_COS_NUM_MAX; i++) {
+ if (valid_cos_bitmap & BIT(i))
+ /* Find max cos id as default cos. */
+ default_cos = i;
+ }
+
+ *cos_id = default_cos;
+
+ return 0;
+}
+
+static int
+hinic3_init_default_cos(struct hinic3_nic_dev *nic_dev)
+{
+ u8 cos_id = 0;
+ int err;
+
+ if (!HINIC3_IS_VF(nic_dev->hwdev)) {
+ err = hinic3_pf_get_default_cos(nic_dev->hwdev, &cos_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Get PF default cos failed, err: %d",
+ err);
+ return err;
+ }
+ } else {
+ err = hinic3_vf_get_default_cos(nic_dev->hwdev, &cos_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Get VF default cos failed, err: %d",
+ err);
+ return err;
+ }
+ }
+
+ nic_dev->default_cos = cos_id;
+ PMD_DRV_LOG(INFO, "Default cos %d", nic_dev->default_cos);
+ return 0;
+}
+
+/**
+ * Initialize Class of Service (CoS). For PF devices, it also sync the link
+ * status with the physical port.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_set_default_hw_feature(struct hinic3_nic_dev *nic_dev)
+{
+ int err;
+
+ err = hinic3_init_default_cos(nic_dev);
+ if (err)
+ return err;
+
+ if (hinic3_func_type(nic_dev->hwdev) == TYPE_VF)
+ return 0;
+
+ err = hinic3_set_link_status_follow(nic_dev->hwdev,
+ HINIC3_LINK_FOLLOW_PORT);
+ if (err == HINIC3_MGMT_CMD_UNSUPPORTED)
+ PMD_DRV_LOG(WARNING, "Don't support to set link status follow "
+ "phy port status");
+ else if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * Initialize the network function, including hardware configuration, memory
+ * allocation for data structures, MAC address setup, and interrupt enabling.
+ * It also registers interrupt callbacks and sets default hardware features.
+ * If any step fails, appropriate cleanup is performed.
+ *
+ * @param[out] eth_dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_func_init(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_tcam_info *tcam_info = NULL;
+ struct hinic3_nic_dev *nic_dev = NULL;
+ struct rte_pci_device *pci_dev = NULL;
+ int err;
+
+ pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+
+ /* EAL is secondary and eth_dev is already created. */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ PMD_DRV_LOG(INFO, "Initialize %s in secondary process",
+ eth_dev->data->name);
+
+ return 0;
+ }
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ memset(nic_dev, 0, sizeof(*nic_dev));
+ (void)snprintf(nic_dev->dev_name, sizeof(nic_dev->dev_name),
+ "dbdf-%.4x:%.2x:%.2x.%x", pci_dev->addr.domain,
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function);
+
+ /* Alloc mac_addrs. */
+ eth_dev->data->mac_addrs = rte_zmalloc("hinic3_mac",
+ HINIC3_MAX_UC_MAC_ADDRS * sizeof(struct rte_ether_addr), 0);
+ if (!eth_dev->data->mac_addrs) {
+ PMD_DRV_LOG(ERR,
+ "Allocate %zx bytes to store MAC addresses "
+ "failed, dev_name: %s",
+ HINIC3_MAX_UC_MAC_ADDRS *
+ sizeof(struct rte_ether_addr),
+ eth_dev->data->name);
+ err = -ENOMEM;
+ goto alloc_eth_addr_fail;
+ }
+
+ nic_dev->mc_list = rte_zmalloc("hinic3_mc",
+ HINIC3_MAX_MC_MAC_ADDRS * sizeof(struct rte_ether_addr), 0);
+ if (!nic_dev->mc_list) {
+ PMD_DRV_LOG(ERR,
+ "Allocate %zx bytes to store multicast "
+ "addresses failed, dev_name: %s",
+ HINIC3_MAX_MC_MAC_ADDRS *
+ sizeof(struct rte_ether_addr),
+ eth_dev->data->name);
+ err = -ENOMEM;
+ goto alloc_mc_list_fail;
+ }
+
+ /* Create hardware device. */
+ nic_dev->hwdev = rte_zmalloc("hinic3_hwdev", sizeof(*nic_dev->hwdev),
+ RTE_CACHE_LINE_SIZE);
+ if (!nic_dev->hwdev) {
+ PMD_DRV_LOG(ERR, "Allocate hwdev memory failed, dev_name: %s",
+ eth_dev->data->name);
+ err = -ENOMEM;
+ goto alloc_hwdev_mem_fail;
+ }
+ nic_dev->hwdev->pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+ nic_dev->hwdev->dev_handle = nic_dev;
+ nic_dev->hwdev->eth_dev = eth_dev;
+ nic_dev->hwdev->port_id = eth_dev->data->port_id;
+
+ err = hinic3_init_hwdev(nic_dev->hwdev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init chip hwdev failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_hwdev_fail;
+ }
+
+ nic_dev->max_sqs = hinic3_func_max_sqs(nic_dev->hwdev);
+ nic_dev->max_rqs = hinic3_func_max_rqs(nic_dev->hwdev);
+
+ err = hinic3_init_nic_hwdev(nic_dev->hwdev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init nic hwdev failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_nic_hwdev_fail;
+ }
+
+ err = hinic3_get_feature_from_hw(nic_dev->hwdev, &nic_dev->feature_cap,
+ 1);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Get nic feature from hardware failed, dev_name: %s",
+ eth_dev->data->name);
+ goto get_cap_fail;
+ }
+
+ err = hinic3_init_sw_rxtxqs(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init sw rxqs or txqs failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_sw_rxtxqs_fail;
+ }
+
+ err = hinic3_init_mac_table(eth_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init mac table failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_mac_table_fail;
+ }
+
+ /* Set hardware feature to default status. */
+ err = hinic3_set_default_hw_feature(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set hw default features failed, dev_name: %s",
+ eth_dev->data->name);
+ goto set_default_feature_fail;
+ }
+
+ /* Register callback func to eal lib. */
+ err = rte_intr_callback_register(PCI_DEV_TO_INTR_HANDLE(pci_dev),
+ hinic3_dev_interrupt_handler,
+ (void *)eth_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Register intr callback failed, dev_name: %s",
+ eth_dev->data->name);
+ goto reg_intr_cb_fail;
+ }
+
+ /* Enable uio/vfio intr/eventfd mapping. */
+ err = rte_intr_enable(PCI_DEV_TO_INTR_HANDLE(pci_dev));
+ if (err) {
+ PMD_DRV_LOG(ERR, "Enable rte interrupt failed, dev_name: %s",
+ eth_dev->data->name);
+ goto enable_intr_fail;
+ }
+ tcam_info = &nic_dev->tcam;
+ memset(tcam_info, 0, sizeof(struct hinic3_tcam_info));
+ TAILQ_INIT(&tcam_info->tcam_list);
+ TAILQ_INIT(&tcam_info->tcam_dynamic_info.tcam_dynamic_list);
+ TAILQ_INIT(&nic_dev->filter_ethertype_list);
+ TAILQ_INIT(&nic_dev->filter_fdir_rule_list);
+
+ hinic3_mutex_init(&nic_dev->rx_mode_mutex, NULL);
+
+ hinic3_set_bit(HINIC3_DEV_INTR_EN, &nic_dev->dev_status);
+
+ hinic3_set_bit(HINIC3_DEV_INIT, &nic_dev->dev_status);
+ PMD_DRV_LOG(INFO, "Initialize %s in primary succeed",
+ eth_dev->data->name);
+
+ /**
+ * Queue xstats filled automatically by ethdev layer.
+ */
+ eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ return 0;
+
+enable_intr_fail:
+ (void)rte_intr_callback_unregister(PCI_DEV_TO_INTR_HANDLE(pci_dev),
+ hinic3_dev_interrupt_handler,
+ (void *)eth_dev);
+
+reg_intr_cb_fail:
+set_default_feature_fail:
+ hinic3_deinit_mac_addr(eth_dev);
+
+init_mac_table_fail:
+ hinic3_deinit_sw_rxtxqs(nic_dev);
+
+init_sw_rxtxqs_fail:
+ hinic3_free_nic_hwdev(nic_dev->hwdev);
+
+get_cap_fail:
+init_nic_hwdev_fail:
+ hinic3_free_hwdev(nic_dev->hwdev);
+ eth_dev->dev_ops = NULL;
+ eth_dev->rx_queue_count = NULL;
+ eth_dev->rx_descriptor_status = NULL;
+ eth_dev->tx_descriptor_status = NULL;
+
+init_hwdev_fail:
+ rte_free(nic_dev->hwdev);
+ nic_dev->hwdev = NULL;
+
+alloc_hwdev_mem_fail:
+ rte_free(nic_dev->mc_list);
+ nic_dev->mc_list = NULL;
+
+alloc_mc_list_fail:
+ rte_free(eth_dev->data->mac_addrs);
+ eth_dev->data->mac_addrs = NULL;
+
+alloc_eth_addr_fail:
+ PMD_DRV_LOG(ERR, "Initialize %s in primary failed",
+ eth_dev->data->name);
+ return err;
+}
+
+static int
+hinic3_dev_init(struct rte_eth_dev *eth_dev)
+{
+ struct rte_pci_device *pci_dev;
+
+ pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+
+ PMD_DRV_LOG(INFO, "Initializing %.4x:%.2x:%.2x.%x in %s process",
+ pci_dev->addr.domain, pci_dev->addr.bus,
+ pci_dev->addr.devid, pci_dev->addr.function,
+ (rte_eal_process_type() == RTE_PROC_PRIMARY) ? "primary"
+ : "secondary");
+
+ PMD_DRV_LOG(INFO, "Network Interface pmd driver version: %s",
+ HINIC3_PMD_DRV_VERSION);
+
+ return hinic3_func_init(eth_dev);
+}
+
+static int
+hinic3_dev_uninit(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ hinic3_clear_bit(HINIC3_DEV_INIT, &nic_dev->dev_status);
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ return hinic3_dev_close(dev);
+}
+
+static const struct rte_pci_id pci_id_hinic3_map[] = {
+#ifdef CONFIG_SP_VID_DID
+ {RTE_PCI_DEVICE(PCI_VENDOR_ID_SPNIC, HINIC3_DEV_ID_STANDARD)},
+ {RTE_PCI_DEVICE(PCI_VENDOR_ID_SPNIC, HINIC3_DEV_ID_VF)},
+#else
+ {RTE_PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, HINIC3_DEV_ID_STANDARD)},
+ {RTE_PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, HINIC3_DEV_ID_VF)},
+#endif
+
+ {.vendor_id = 0},
+};
+
+static int
+hinic3_pci_probe(__rte_unused struct rte_pci_driver *pci_drv,
+ struct rte_pci_device *pci_dev)
+{
+ return rte_eth_dev_pci_generic_probe(pci_dev,
+ sizeof(struct hinic3_nic_dev), hinic3_dev_init);
+}
+
+static int
+hinic3_pci_remove(struct rte_pci_device *pci_dev)
+{
+ return rte_eth_dev_pci_generic_remove(pci_dev, hinic3_dev_uninit);
+}
+
+static struct rte_pci_driver rte_hinic3_pmd = {
+ .id_table = pci_id_hinic3_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC,
+ .probe = hinic3_pci_probe,
+ .remove = hinic3_pci_remove,
+};
+
+RTE_PMD_REGISTER_PCI(net_hinic3, rte_hinic3_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_hinic3, pci_id_hinic3_map);
+
+RTE_INIT(hinic3_init_log)
+{
+ hinic3_logtype = rte_log_register("pmd.net.hinic3");
+ if (hinic3_logtype >= 0)
+ rte_log_set_level(hinic3_logtype, RTE_LOG_INFO);
+}
diff --git a/drivers/net/hinic3/hinic3_ethdev.h b/drivers/net/hinic3/hinic3_ethdev.h
new file mode 100644
index 0000000000..a69cf972e7
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_ethdev.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_ETHDEV_H_
+#define _HINIC3_ETHDEV_H_
+
+#include <rte_ethdev.h>
+#include <rte_ethdev_core.h>
+
+#define HINIC3_PMD_DRV_VERSION "B106"
+
+#define PCI_DEV_TO_INTR_HANDLE(pci_dev) ((pci_dev)->intr_handle)
+
+#define HINIC3_PKT_RX_L4_CKSUM_BAD RTE_MBUF_F_RX_L4_CKSUM_BAD
+#define HINIC3_PKT_RX_IP_CKSUM_BAD RTE_MBUF_F_RX_IP_CKSUM_BAD
+#define HINIC3_PKT_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN
+#define HINIC3_PKT_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_L4_CKSUM_GOOD
+#define HINIC3_PKT_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD
+#define HINIC3_PKT_TX_TCP_SEG RTE_MBUF_F_TX_TCP_SEG
+#define HINIC3_PKT_TX_UDP_CKSUM RTE_MBUF_F_TX_UDP_CKSUM
+#define HINIC3_PKT_TX_TCP_CKSUM RTE_MBUF_F_TX_TCP_CKSUM
+#define HINIC3_PKT_TX_IP_CKSUM RTE_MBUF_F_TX_IP_CKSUM
+#define HINIC3_PKT_TX_VLAN_PKT RTE_MBUF_F_TX_VLAN
+#define HINIC3_PKT_TX_L4_MASK RTE_MBUF_F_TX_L4_MASK
+#define HINIC3_PKT_TX_SCTP_CKSUM RTE_MBUF_F_TX_SCTP_CKSUM
+#define HINIC3_PKT_TX_IPV6 RTE_MBUF_F_TX_IPV6
+#define HINIC3_PKT_TX_IPV4 RTE_MBUF_F_TX_IPV4
+#define HINIC3_PKT_RX_VLAN RTE_MBUF_F_RX_VLAN
+#define HINIC3_PKT_RX_VLAN_STRIPPED RTE_MBUF_F_RX_VLAN_STRIPPED
+#define HINIC3_PKT_RX_RSS_HASH RTE_MBUF_F_RX_RSS_HASH
+#define HINIC3_PKT_TX_TUNNEL_MASK RTE_MBUF_F_TX_TUNNEL_MASK
+#define HINIC3_PKT_TX_TUNNEL_VXLAN RTE_MBUF_F_TX_TUNNEL_VXLAN
+#define HINIC3_PKT_TX_OUTER_IP_CKSUM RTE_MBUF_F_TX_OUTER_IP_CKSUM
+#define HINIC3_PKT_TX_OUTER_IPV6 RTE_MBUF_F_TX_OUTER_IPV6
+#define HINIC3_PKT_RX_LRO RTE_MBUF_F_RX_LRO
+#define HINIC3_PKT_TX_L4_NO_CKSUM RTE_MBUF_F_TX_L4_NO_CKSUM
+
+#define HINCI3_CPY_MEMPOOL_NAME "cpy_mempool"
+/* Mbuf pool for copy invalid mbuf segs. */
+#define HINIC3_COPY_MEMPOOL_DEPTH 1024
+#define HINIC3_COPY_MEMPOOL_CACHE 128
+#define HINIC3_COPY_MBUF_SIZE 4096
+
+#define HINIC3_DEV_NAME_LEN 32
+#define DEV_STOP_DELAY_MS 100
+#define DEV_START_DELAY_MS 100
+
+#define HINIC3_UINT32_BIT_SIZE (CHAR_BIT * sizeof(uint32_t))
+#define HINIC3_VFTA_SIZE (4096 / HINIC3_UINT32_BIT_SIZE)
+#define HINIC3_MAX_QUEUE_NUM 64
+
+#define HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev) \
+ ((struct hinic3_nic_dev *)(dev)->data->dev_private)
+
+enum hinic3_dev_status {
+ HINIC3_DEV_INIT,
+ HINIC3_DEV_CLOSE,
+ HINIC3_DEV_START,
+ HINIC3_DEV_INTR_EN
+};
+
+enum hinic3_tx_cvlan_type {
+ HINIC3_TX_TPID0,
+};
+
+enum nic_feature_cap {
+ NIC_F_CSUM = BIT(0),
+ NIC_F_SCTP_CRC = BIT(1),
+ NIC_F_TSO = BIT(2),
+ NIC_F_LRO = BIT(3),
+ NIC_F_UFO = BIT(4),
+ NIC_F_RSS = BIT(5),
+ NIC_F_RX_VLAN_FILTER = BIT(6),
+ NIC_F_RX_VLAN_STRIP = BIT(7),
+ NIC_F_TX_VLAN_INSERT = BIT(8),
+ NIC_F_VXLAN_OFFLOAD = BIT(9),
+ NIC_F_IPSEC_OFFLOAD = BIT(10),
+ NIC_F_FDIR = BIT(11),
+ NIC_F_PROMISC = BIT(12),
+ NIC_F_ALLMULTI = BIT(13),
+};
+
+#define DEFAULT_DRV_FEATURE 0x3FFF
+
+struct hinic3_nic_dev {
+ struct hinic3_hwdev *hwdev; /**< Hardware device. */
+ struct hinic3_txq **txqs;
+ struct hinic3_rxq **rxqs;
+ struct rte_mempool *cpy_mpool;
+
+ u16 num_sqs;
+ u16 num_rqs;
+ u16 max_sqs;
+ u16 max_rqs;
+
+ u16 rx_buff_len;
+ u16 mtu_size;
+
+ u32 rx_mode;
+ u8 rx_queue_list[HINIC3_MAX_QUEUE_NUM];
+ rte_spinlock_t queue_list_lock;
+
+ pthread_mutex_t rx_mode_mutex;
+
+ u32 default_cos;
+ u32 rx_csum_en;
+
+ unsigned long dev_status;
+
+ struct rte_ether_addr default_addr;
+ struct rte_ether_addr *mc_list;
+
+ char dev_name[HINIC3_DEV_NAME_LEN];
+ u64 feature_cap;
+ u32 vfta[HINIC3_VFTA_SIZE]; /**< VLAN bitmap. */
+};
+
+#endif /* _HINIC3_ETHDEV_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 13/18] net/hinic3: add dev ops
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (3 preceding siblings ...)
2025-04-18 7:02 ` [RFC 12/18] net/hinic3: add device initailization Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 14/18] net/hinic3: add Rx/Tx functions Feifei Wang
` (5 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Feifei Wang, Xin Wang, Yi Chen
From: Feifei Wang <wangfeifei40@huawei.com>
Add ops related function codes.
Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
Signed-off-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Yi Chen <chenyi221@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 2918 +++++++++++++++++++++++++++-
drivers/net/hinic3/hinic3_nic_io.c | 827 ++++++++
drivers/net/hinic3/hinic3_nic_io.h | 169 ++
drivers/net/hinic3/hinic3_rx.c | 811 ++++++++
drivers/net/hinic3/hinic3_rx.h | 356 ++++
drivers/net/hinic3/hinic3_tx.c | 274 +++
drivers/net/hinic3/hinic3_tx.h | 314 +++
7 files changed, 5652 insertions(+), 17 deletions(-)
create mode 100644 drivers/net/hinic3/hinic3_nic_io.c
create mode 100644 drivers/net/hinic3/hinic3_nic_io.h
create mode 100644 drivers/net/hinic3/hinic3_rx.c
create mode 100644 drivers/net/hinic3/hinic3_rx.h
create mode 100644 drivers/net/hinic3/hinic3_tx.c
create mode 100644 drivers/net/hinic3/hinic3_tx.h
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index c4b2f5ffe4..de380dddbb 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -21,42 +21,2917 @@
#include "base/hinic3_hw_comm.h"
#include "base/hinic3_nic_cfg.h"
#include "base/hinic3_nic_event.h"
+#include "hinic3_pmd_nic_io.h"
+#include "hinic3_pmd_tx.h"
+#include "hinic3_pmd_rx.h"
#include "hinic3_ethdev.h"
+#define HINIC3_MIN_RX_BUF_SIZE 1024
+
+#define HINIC3_DEFAULT_BURST_SIZE 32
+#define HINIC3_DEFAULT_NB_QUEUES 1
+#define HINIC3_DEFAULT_RING_SIZE 1024
+#define HINIC3_MAX_LRO_SIZE 65536
+
+#define HINIC3_DEFAULT_RX_FREE_THRESH 32
+#define HINIC3_DEFAULT_TX_FREE_THRESH 32
+
+#define HINIC3_RX_WAIT_CYCLE_THRESH 500
+
+/**
+ * Get the 32-bit VFTA bit mask for the lower 5 bits of the VLAN ID.
+ *
+ * Vlan_id is a 12 bit number. The VFTA array is actually a 4096 bit array,
+ * 128 of 32bit elements. 2^5 = 32. The val of lower 5 bits specifies the bit
+ * in the 32bit element. The higher 7 bit val specifies VFTA array index.
+ */
+#define HINIC3_VFTA_BIT(vlan_id) (1 << ((vlan_id) & 0x1F))
+/**
+ * Get the VFTA index from the upper 7 bits of the VLAN ID.
+ */
+#define HINIC3_VFTA_IDX(vlan_id) ((vlan_id) >> 5)
+
+#define HINIC3_LRO_DEFAULT_TIME_LIMIT 16
+#define HINIC3_LRO_UNIT_WQE_SIZE 1024 /**< Bytes. */
+
+#define HINIC3_MAX_RX_PKT_LEN(rxmod) ((rxmod).mtu)
+int hinic3_logtype; /**< Driver-specific log messages type. */
+
+/**
+ * The different receive modes for the NIC.
+ *
+ * The receive modes are represented as bit flags that control how the
+ * NIC handles various types of network traffic.
+ */
+enum hinic3_rx_mod {
+ /* Enable unicast receive mode. */
+ HINIC3_RX_MODE_UC = 1 << 0,
+ /* Enable multicast receive mode. */
+ HINIC3_RX_MODE_MC = 1 << 1,
+ /* Enable broadcast receive mode. */
+ HINIC3_RX_MODE_BC = 1 << 2,
+ /* Enable receive mode for all multicast addresses. */
+ HINIC3_RX_MODE_MC_ALL = 1 << 3,
+ /* Enable promiscuous mode, receiving all packets. */
+ HINIC3_RX_MODE_PROMISC = 1 << 4,
+};
+
+#define HINIC3_DEFAULT_RX_MODE \
+ (HINIC3_RX_MODE_UC | HINIC3_RX_MODE_MC | HINIC3_RX_MODE_BC)
+
+struct hinic3_xstats_name_off {
+ char name[RTE_ETH_XSTATS_NAME_SIZE];
+ u32 offset;
+};
+
+#define HINIC3_FUNC_STAT(_stat_item) \
+ { \
+ .name = #_stat_item, \
+ .offset = offsetof(struct hinic3_vport_stats, _stat_item), \
+ }
+
+static const struct hinic3_xstats_name_off hinic3_vport_stats_strings[] = {
+ HINIC3_FUNC_STAT(tx_unicast_pkts_vport),
+ HINIC3_FUNC_STAT(tx_unicast_bytes_vport),
+ HINIC3_FUNC_STAT(tx_multicast_pkts_vport),
+ HINIC3_FUNC_STAT(tx_multicast_bytes_vport),
+ HINIC3_FUNC_STAT(tx_broadcast_pkts_vport),
+ HINIC3_FUNC_STAT(tx_broadcast_bytes_vport),
+
+ HINIC3_FUNC_STAT(rx_unicast_pkts_vport),
+ HINIC3_FUNC_STAT(rx_unicast_bytes_vport),
+ HINIC3_FUNC_STAT(rx_multicast_pkts_vport),
+ HINIC3_FUNC_STAT(rx_multicast_bytes_vport),
+ HINIC3_FUNC_STAT(rx_broadcast_pkts_vport),
+ HINIC3_FUNC_STAT(rx_broadcast_bytes_vport),
+
+ HINIC3_FUNC_STAT(tx_discard_vport),
+ HINIC3_FUNC_STAT(rx_discard_vport),
+ HINIC3_FUNC_STAT(tx_err_vport),
+ HINIC3_FUNC_STAT(rx_err_vport),
+};
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+#define HINIC3_VPORT_XSTATS_NUM ARRAY_SIZE(hinic3_vport_stats_strings)
+
+#define HINIC3_PORT_STAT(_stat_item) \
+ { \
+ .name = #_stat_item, \
+ .offset = offsetof(struct mag_phy_port_stats, _stat_item), \
+ }
+
+static const struct hinic3_xstats_name_off hinic3_phyport_stats_strings[] = {
+ HINIC3_PORT_STAT(mac_tx_fragment_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_undersize_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_undermin_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_64_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_65_127_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_128_255_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_256_511_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_512_1023_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_1024_1518_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_1519_2047_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_2048_4095_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_4096_8191_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_8192_9216_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_9217_12287_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_12288_16383_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_1519_max_bad_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_1519_max_good_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_oversize_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_jabber_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_bad_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_bad_oct_num),
+ HINIC3_PORT_STAT(mac_tx_good_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_good_oct_num),
+ HINIC3_PORT_STAT(mac_tx_total_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_total_oct_num),
+ HINIC3_PORT_STAT(mac_tx_uni_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_multi_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_broad_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pause_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri0_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri1_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri2_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri3_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri4_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri5_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri6_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_pfc_pri7_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_control_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_err_all_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_from_app_good_pkt_num),
+ HINIC3_PORT_STAT(mac_tx_from_app_bad_pkt_num),
+
+ HINIC3_PORT_STAT(mac_rx_fragment_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_undersize_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_undermin_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_64_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_65_127_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_128_255_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_256_511_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_512_1023_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_1024_1518_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_1519_2047_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_2048_4095_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_4096_8191_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_8192_9216_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_9217_12287_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_12288_16383_oct_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_1519_max_bad_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_1519_max_good_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_oversize_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_jabber_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_bad_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_bad_oct_num),
+ HINIC3_PORT_STAT(mac_rx_good_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_good_oct_num),
+ HINIC3_PORT_STAT(mac_rx_total_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_total_oct_num),
+ HINIC3_PORT_STAT(mac_rx_uni_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_multi_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_broad_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pause_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri0_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri1_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri2_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri3_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri4_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri5_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri6_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_pfc_pri7_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_control_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_sym_err_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_fcs_err_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_send_app_good_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_send_app_bad_pkt_num),
+ HINIC3_PORT_STAT(mac_rx_unfilter_pkt_num),
+};
+
+#define HINIC3_PHYPORT_XSTATS_NUM ARRAY_SIZE(hinic3_phyport_stats_strings)
+
+#define HINIC3_RXQ_STAT(_stat_item) \
+ { \
+ .name = #_stat_item, \
+ .offset = offsetof(struct hinic3_rxq_stats, _stat_item), \
+ }
+
+/**
+ * The name and offset field of RXQ statistic items.
+ *
+ * The inclusion of additional statistics depends on the compilation flags:
+ * - `HINIC3_XSTAT_RXBUF_INFO` enables buffer-related stats.
+ * - `HINIC3_XSTAT_PROF_RX` enables performance timing stats.
+ * - `HINIC3_XSTAT_MBUF_USE` enables memory buffer usage stats.
+ */
+static const struct hinic3_xstats_name_off hinic3_rxq_stats_strings[] = {
+ HINIC3_RXQ_STAT(rx_nombuf),
+ HINIC3_RXQ_STAT(burst_pkts),
+ HINIC3_RXQ_STAT(errors),
+ HINIC3_RXQ_STAT(csum_errors),
+ HINIC3_RXQ_STAT(other_errors),
+ HINIC3_RXQ_STAT(empty),
+
+#ifdef HINIC3_XSTAT_RXBUF_INFO
+ HINIC3_RXQ_STAT(rx_mbuf),
+ HINIC3_RXQ_STAT(rx_avail),
+ HINIC3_RXQ_STAT(rx_hole),
+#endif
+
+#ifdef HINIC3_XSTAT_PROF_RX
+ HINIC3_RXQ_STAT(app_tsc),
+ HINIC3_RXQ_STAT(pmd_tsc),
+#endif
+
+#ifdef HINIC3_XSTAT_MBUF_USE
+ HINIC3_RXQ_STAT(rx_alloc_mbuf_bytes),
+ HINIC3_RXQ_STAT(rx_free_mbuf_bytes),
+ HINIC3_RXQ_STAT(rx_left_mbuf_bytes),
+#endif
+};
+
+#define HINIC3_RXQ_XSTATS_NUM ARRAY_SIZE(hinic3_rxq_stats_strings)
+
+#define HINIC3_TXQ_STAT(_stat_item) \
+ { \
+ .name = #_stat_item, \
+ .offset = offsetof(struct hinic3_txq_stats, _stat_item), \
+ }
+
+/**
+ * The name and offset field of TXQ statistic items.
+ *
+ * The inclusion of additional statistics depends on the compilation flags:
+ * - `HINIC3_XSTAT_PROF_TX` enables performance timing stats.
+ * - `HINIC3_XSTAT_MBUF_USE` enables memory buffer usage stats.
+ */
+static const struct hinic3_xstats_name_off hinic3_txq_stats_strings[] = {
+ HINIC3_TXQ_STAT(tx_busy),
+ HINIC3_TXQ_STAT(offload_errors),
+ HINIC3_TXQ_STAT(burst_pkts),
+ HINIC3_TXQ_STAT(sge_len0),
+ HINIC3_TXQ_STAT(mbuf_null),
+
+#ifdef HINIC3_XSTAT_PROF_TX
+ HINIC3_TXQ_STAT(app_tsc),
+ HINIC3_TXQ_STAT(pmd_tsc),
+#endif
+
+#ifdef HINIC3_XSTAT_MBUF_USE
+ HINIC3_TXQ_STAT(tx_left_mbuf_bytes),
+#endif
+};
+
+#define HINIC3_TXQ_XSTATS_NUM ARRAY_SIZE(hinic3_txq_stats_strings)
+
+static int
+hinic3_xstats_calc_num(struct hinic3_nic_dev *nic_dev)
+{
+ if (HINIC3_IS_VF(nic_dev->hwdev)) {
+ return (HINIC3_VPORT_XSTATS_NUM +
+ HINIC3_RXQ_XSTATS_NUM * nic_dev->num_rqs +
+ HINIC3_TXQ_XSTATS_NUM * nic_dev->num_sqs);
+ } else {
+ return (HINIC3_VPORT_XSTATS_NUM + HINIC3_PHYPORT_XSTATS_NUM +
+ HINIC3_RXQ_XSTATS_NUM * nic_dev->num_rqs +
+ HINIC3_TXQ_XSTATS_NUM * nic_dev->num_sqs);
+ }
+}
+
+#define HINIC3_MAX_QUEUE_DEPTH 16384
+#define HINIC3_MIN_QUEUE_DEPTH 128
+#define HINIC3_TXD_ALIGN 1
+#define HINIC3_RXD_ALIGN 1
+
+static const struct rte_eth_desc_lim hinic3_rx_desc_lim = {
+ .nb_max = HINIC3_MAX_QUEUE_DEPTH,
+ .nb_min = HINIC3_MIN_QUEUE_DEPTH,
+ .nb_align = HINIC3_RXD_ALIGN,
+};
+
+static const struct rte_eth_desc_lim hinic3_tx_desc_lim = {
+ .nb_max = HINIC3_MAX_QUEUE_DEPTH,
+ .nb_min = HINIC3_MIN_QUEUE_DEPTH,
+ .nb_align = HINIC3_TXD_ALIGN,
+};
+
+static void hinic3_deinit_mac_addr(struct rte_eth_dev *eth_dev);
+
+static int hinic3_copy_mempool_init(struct hinic3_nic_dev *nic_dev);
+
+static void hinic3_copy_mempool_uninit(struct hinic3_nic_dev *nic_dev);
+
+/**
+ * Interrupt handler triggered by NIC for handling specific event.
+ *
+ * @param[in] param
+ * The address of parameter (struct rte_eth_dev *) regsitered before.
+ */
+static void
+hinic3_dev_interrupt_handler(void *param)
+{
+ struct rte_eth_dev *dev = param;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (!hinic3_get_bit(HINIC3_DEV_INTR_EN, &nic_dev->dev_status)) {
+ PMD_DRV_LOG(WARNING,
+ "Intr is disabled, ignore intr event, "
+ "dev_name: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return;
+ }
+
+ /* Aeq0 msg handler. */
+ hinic3_dev_handle_aeq_event(nic_dev->hwdev, param);
+}
+
+/**
+ * Do the config for TX/Rx queues, include queue number, mtu size and RSS.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_configure(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ nic_dev->num_sqs = dev->data->nb_tx_queues;
+ nic_dev->num_rqs = dev->data->nb_rx_queues;
+
+ if (nic_dev->num_sqs > nic_dev->max_sqs ||
+ nic_dev->num_rqs > nic_dev->max_rqs) {
+ PMD_DRV_LOG(ERR,
+ "num_sqs: %d or num_rqs: %d larger than "
+ "max_sqs: %d or max_rqs: %d",
+ nic_dev->num_sqs, nic_dev->num_rqs,
+ nic_dev->max_sqs, nic_dev->max_rqs);
+ return -EINVAL;
+ }
+
+ /* The range of mtu is 384~9600. */
+
+ if (HINIC3_MAX_RX_PKT_LEN(dev->data->dev_conf.rxmode) <
+ HINIC3_MIN_FRAME_SIZE ||
+ HINIC3_MAX_RX_PKT_LEN(dev->data->dev_conf.rxmode) >
+ HINIC3_MAX_JUMBO_FRAME_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Max rx pkt len out of range, max_rx_pkt_len: %d, "
+ "expect between %d and %d",
+ HINIC3_MAX_RX_PKT_LEN(dev->data->dev_conf.rxmode),
+ HINIC3_MIN_FRAME_SIZE, HINIC3_MAX_JUMBO_FRAME_SIZE);
+ return -EINVAL;
+ }
+ nic_dev->mtu_size =
+ (u16)HINIC3_PKTLEN_TO_MTU(HINIC3_MAX_RX_PKT_LEN(dev->data->dev_conf.rxmode));
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ /* Clear fdir filter. */
+ hinic3_free_fdir_filter(dev);
+
+ return 0;
+}
+
+/**
+ * Get information about the device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] info
+ * Info structure for ethernet device.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ info->max_rx_queues = nic_dev->max_rqs;
+ info->max_tx_queues = nic_dev->max_sqs;
+ info->min_rx_bufsize = HINIC3_MIN_RX_BUF_SIZE;
+ info->max_rx_pktlen = HINIC3_MAX_JUMBO_FRAME_SIZE;
+ info->max_mac_addrs = HINIC3_MAX_UC_MAC_ADDRS;
+ info->min_mtu = HINIC3_MIN_MTU_SIZE;
+ info->max_mtu = HINIC3_MAX_MTU_SIZE;
+ info->max_lro_pkt_size = HINIC3_MAX_LRO_SIZE;
+
+ info->rx_queue_offload_capa = 0;
+ info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | RTE_ETH_RX_OFFLOAD_VLAN_FILTER |
+ RTE_ETH_RX_OFFLOAD_SCATTER | RTE_ETH_RX_OFFLOAD_TCP_LRO |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ info->tx_queue_offload_capa = 0;
+ info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM | RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
+ info->hash_key_size = HINIC3_RSS_KEY_SIZE;
+ info->reta_size = HINIC3_RSS_INDIR_SIZE;
+ info->flow_type_rss_offloads = HINIC3_RSS_OFFLOAD_ALL;
+
+ info->rx_desc_lim = hinic3_rx_desc_lim;
+ info->tx_desc_lim = hinic3_tx_desc_lim;
+
+ /* Driver-preferred rx/tx parameters. */
+ info->default_rxportconf.burst_size = HINIC3_DEFAULT_BURST_SIZE;
+ info->default_txportconf.burst_size = HINIC3_DEFAULT_BURST_SIZE;
+ info->default_rxportconf.nb_queues = HINIC3_DEFAULT_NB_QUEUES;
+ info->default_txportconf.nb_queues = HINIC3_DEFAULT_NB_QUEUES;
+ info->default_rxportconf.ring_size = HINIC3_DEFAULT_RING_SIZE;
+ info->default_txportconf.ring_size = HINIC3_DEFAULT_RING_SIZE;
+
+ return 0;
+}
+
+static int
+hinic3_fw_version_get(struct rte_eth_dev *dev, char *fw_version, size_t fw_size)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ char mgmt_ver[MGMT_VERSION_MAX_LEN] = {0};
+ int err;
+
+ err = hinic3_get_mgmt_version(nic_dev->hwdev, mgmt_ver,
+ HINIC3_MGMT_VERSION_MAX_LEN);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Get fw version failed");
+ return -EIO;
+ }
+
+ if (fw_size < strlen((char *)mgmt_ver) + 1)
+ return (strlen((char *)mgmt_ver) + 1);
+
+ (void)snprintf(fw_version, fw_size, "%s", mgmt_ver);
+
+ return 0;
+}
+
+/**
+ * Set ethernet device link state up.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_set_link_up(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int err;
+
+ /*
+ * Vport enable will set function valid in mpu.
+ * So dev start status need to be checked before vport enable.
+ */
+ if (hinic3_get_bit(HINIC3_DEV_START, &nic_dev->dev_status)) {
+ err = hinic3_set_vport_enable(nic_dev->hwdev, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Enable vport failed, dev_name: %s",
+ nic_dev->dev_name);
+ return err;
+ }
+ }
+
+ /* Link status follow phy port status, mpu will open pma. */
+ err = hinic3_set_port_enable(nic_dev->hwdev, true);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Set MAC link up failed, dev_name: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * Set ethernet device link state down.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_set_link_down(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int err;
+
+ err = hinic3_set_vport_enable(nic_dev->hwdev, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Disable vport failed, dev_name: %s",
+ nic_dev->dev_name);
+ return err;
+ }
+
+ /* Link status follow phy port status, mpu will close pma. */
+ err = hinic3_set_port_enable(nic_dev->hwdev, false);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Set MAC link down failed, dev_name: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * Get device physical link information.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] wait_to_complete
+ * Wait for request completion.
+ *
+ * @return
+ * 0 : Link status changed
+ * -1 : Link status not changed.
+ */
+static int
+hinic3_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+#define CHECK_INTERVAL 10 /**< 10ms. */
+#define MAX_REPEAT_TIME 100 /**< 1s (100 * 10ms) in total. */
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_eth_link link;
+ u8 link_state;
+ unsigned int rep_cnt = MAX_REPEAT_TIME;
+ int ret;
+
+ memset(&link, 0, sizeof(link));
+ do {
+ /* Get link status information from hardware. */
+ ret = hinic3_get_link_state(nic_dev->hwdev, &link_state);
+ if (ret) {
+ link.link_status = RTE_ETH_LINK_DOWN;
+ link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ link.link_duplex = RTE_ETH_LINK_HALF_DUPLEX;
+ link.link_autoneg = RTE_ETH_LINK_FIXED;
+ goto out;
+ }
+
+ get_port_info(nic_dev->hwdev, link_state, &link);
+
+ if (!wait_to_complete || link.link_status)
+ break;
+
+ rte_delay_ms(CHECK_INTERVAL);
+ } while (rep_cnt--);
+
+out:
+ return rte_eth_linkstatus_set(dev, &link);
+}
+
+/**
+ * Reset all RX queues (RXQs).
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ */
+static void
+hinic3_reset_rx_queue(struct rte_eth_dev *dev)
+{
+ struct hinic3_rxq *rxq = NULL;
+ struct hinic3_nic_dev *nic_dev;
+ int q_id = 0;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ for (q_id = 0; q_id < nic_dev->num_rqs; q_id++) {
+ rxq = nic_dev->rxqs[q_id];
+
+ rxq->cons_idx = 0;
+ rxq->prod_idx = 0;
+ rxq->delta = rxq->q_depth;
+ rxq->next_to_update = 0;
+ }
+}
+
+/**
+ * Reset all TX queues (TXQs).
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ */
+static void
+hinic3_reset_tx_queue(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev;
+ struct hinic3_txq *txq = NULL;
+ int q_id = 0;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ for (q_id = 0; q_id < nic_dev->num_sqs; q_id++) {
+ txq = nic_dev->txqs[q_id];
+
+ txq->cons_idx = 0;
+ txq->prod_idx = 0;
+ txq->owner = 1;
+
+ /* Clear hardware ci. */
+ *txq->ci_vaddr_base = 0;
+ }
+}
+
+/**
+ * Create the receive queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] qid
+ * Receive queue index.
+ * @param[in] nb_desc
+ * Number of descriptors for receive queue.
+ * @param[in] socket_id
+ * Socket index on which memory must be allocated.
+ * @param[in] rx_conf
+ * Thresholds parameters (unused_).
+ * @param[in] mp
+ * Memory pool for buffer allocations.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qid, uint16_t nb_desc,
+ unsigned int socket_id,
+ __rte_unused const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct hinic3_nic_dev *nic_dev;
+ struct hinic3_rxq *rxq = NULL;
+ const struct rte_memzone *rq_mz = NULL;
+ const struct rte_memzone *cqe_mz = NULL;
+ const struct rte_memzone *pi_mz = NULL;
+ u16 rq_depth, rx_free_thresh;
+ u32 queue_buf_size;
+ void *db_addr = NULL;
+ int wqe_count;
+ u32 buf_size;
+ int err;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ /* Queue depth must be power of 2, otherwise will be aligned up. */
+ rq_depth = (nb_desc & (nb_desc - 1))
+ ? ((u16)(1U << (ilog2(nb_desc) + 1)))
+ : nb_desc;
+
+ /*
+ * Validate number of receive descriptors.
+ * It must not exceed hardware maximum and minimum.
+ */
+ if (rq_depth > HINIC3_MAX_QUEUE_DEPTH ||
+ rq_depth < HINIC3_MIN_QUEUE_DEPTH) {
+ PMD_DRV_LOG(ERR,
+ "RX queue depth is out of range from %d to %d,"
+ "(nb_desc: %d, q_depth: %d, port: %d queue: %d)",
+ HINIC3_MIN_QUEUE_DEPTH, HINIC3_MAX_QUEUE_DEPTH,
+ (int)nb_desc, (int)rq_depth,
+ (int)dev->data->port_id, (int)qid);
+ return -EINVAL;
+ }
+
+ /*
+ * The RX descriptor ring will be cleaned after rxq->rx_free_thresh
+ * descriptors are used or if the number of descriptors required
+ * to transmit a packet is greater than the number of free RX
+ * descriptors.
+ * The following constraints must be satisfied:
+ * - rx_free_thresh must be greater than 0.
+ * - rx_free_thresh must be less than the size of the ring minus 1.
+ * When set to zero use default values.
+ */
+ rx_free_thresh = (u16)((rx_conf->rx_free_thresh)
+ ? rx_conf->rx_free_thresh
+ : HINIC3_DEFAULT_RX_FREE_THRESH);
+ if (rx_free_thresh >= (rq_depth - 1)) {
+ PMD_DRV_LOG(ERR,
+ "rx_free_thresh must be less than the number "
+ "of RX descriptors minus 1, rx_free_thresh: %u "
+ "port: %d queue: %d)",
+ (unsigned int)rx_free_thresh,
+ (int)dev->data->port_id, (int)qid);
+
+ return -EINVAL;
+ }
+
+ rxq = rte_zmalloc_socket("hinic3_rq", sizeof(struct hinic3_rxq),
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!rxq) {
+ PMD_DRV_LOG(ERR, "Allocate rxq[%d] failed, dev_name: %s", qid,
+ dev->data->name);
+
+ return -ENOMEM;
+ }
+
+ /* Init rq parameters. */
+ rxq->nic_dev = nic_dev;
+ nic_dev->rxqs[qid] = rxq;
+ rxq->mb_pool = mp;
+ rxq->q_id = qid;
+ rxq->next_to_update = 0;
+ rxq->q_depth = rq_depth;
+ rxq->q_mask = rq_depth - 1;
+ rxq->delta = rq_depth;
+ rxq->cons_idx = 0;
+ rxq->prod_idx = 0;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->rxinfo_align_end = rxq->q_depth - rxq->rx_free_thresh;
+ rxq->port_id = dev->data->port_id;
+ rxq->wait_time_cycle = HINIC3_RX_WAIT_CYCLE_THRESH;
+
+ /* If buf_len used for function table, need to translated. */
+ u16 rx_buf_size =
+ rte_pktmbuf_data_room_size(rxq->mb_pool) - RTE_PKTMBUF_HEADROOM;
+ err = hinic3_convert_rx_buf_size(rx_buf_size, &buf_size);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Adjust buf size failed, dev_name: %s",
+ dev->data->name);
+ goto adjust_bufsize_fail;
+ }
+
+ if (buf_size >= HINIC3_RX_BUF_SIZE_4K &&
+ buf_size < HINIC3_RX_BUF_SIZE_16K)
+ rxq->wqe_type = HINIC3_EXTEND_RQ_WQE;
+ else
+ rxq->wqe_type = HINIC3_NORMAL_RQ_WQE;
+
+ rxq->wqebb_shift = HINIC3_RQ_WQEBB_SHIFT + rxq->wqe_type;
+ rxq->wqebb_size = (u16)BIT(rxq->wqebb_shift);
+
+ rxq->buf_len = (u16)buf_size;
+ rxq->rx_buff_shift = ilog2(rxq->buf_len);
+
+ pi_mz = hinic3_dma_zone_reserve(dev, "hinic3_rq_pi", qid, RTE_PGSIZE_4K,
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!pi_mz) {
+ PMD_DRV_LOG(ERR, "Allocate rxq[%d] pi_mz failed, dev_name: %s",
+ qid, dev->data->name);
+ err = -ENOMEM;
+ goto alloc_pi_mz_fail;
+ }
+ rxq->pi_mz = pi_mz;
+ rxq->pi_dma_addr = pi_mz->iova;
+ rxq->pi_virt_addr = pi_mz->addr;
+
+ err = hinic3_alloc_db_addr(nic_dev->hwdev, &db_addr, HINIC3_DB_TYPE_RQ);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc rq doorbell addr failed");
+ goto alloc_db_err_fail;
+ }
+ rxq->db_addr = db_addr;
+
+ queue_buf_size = BIT(rxq->wqebb_shift) * rq_depth;
+ rq_mz = hinic3_dma_zone_reserve(dev, "hinic3_rq_mz", qid,
+ queue_buf_size, RTE_PGSIZE_256K,
+ (int)socket_id);
+ if (!rq_mz) {
+ PMD_DRV_LOG(ERR, "Allocate rxq[%d] rq_mz failed, dev_name: %s",
+ qid, dev->data->name);
+ err = -ENOMEM;
+ goto alloc_rq_mz_fail;
+ }
+
+ memset(rq_mz->addr, 0, queue_buf_size);
+ rxq->rq_mz = rq_mz;
+ rxq->queue_buf_paddr = rq_mz->iova;
+ rxq->queue_buf_vaddr = rq_mz->addr;
+
+ rxq->rx_info = rte_zmalloc_socket("rx_info",
+ rq_depth * sizeof(*rxq->rx_info),
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!rxq->rx_info) {
+ PMD_DRV_LOG(ERR, "Allocate rx_info failed, dev_name: %s",
+ dev->data->name);
+ err = -ENOMEM;
+ goto alloc_rx_info_fail;
+ }
+
+ cqe_mz = hinic3_dma_zone_reserve(dev, "hinic3_cqe_mz", qid,
+ rq_depth * sizeof(*rxq->rx_cqe),
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!cqe_mz) {
+ PMD_DRV_LOG(ERR, "Allocate cqe mem zone failed, dev_name: %s",
+ dev->data->name);
+ err = -ENOMEM;
+ goto alloc_cqe_mz_fail;
+ }
+ memset(cqe_mz->addr, 0, rq_depth * sizeof(*rxq->rx_cqe));
+ rxq->cqe_mz = cqe_mz;
+ rxq->cqe_start_paddr = cqe_mz->iova;
+ rxq->cqe_start_vaddr = cqe_mz->addr;
+ rxq->rx_cqe = (struct hinic3_rq_cqe *)rxq->cqe_start_vaddr;
+
+ wqe_count = hinic3_rx_fill_wqe(rxq);
+ if (wqe_count != rq_depth) {
+ PMD_DRV_LOG(ERR,
+ "Fill rx wqe failed, wqe_count: %d, dev_name: %s",
+ wqe_count, dev->data->name);
+ err = -ENOMEM;
+ goto fill_rx_wqe_fail;
+ }
+ /* Record rxq pointer in rte_eth rx_queues. */
+ dev->data->rx_queues[qid] = rxq;
+
+ return 0;
+
+fill_rx_wqe_fail:
+ hinic3_memzone_free(rxq->cqe_mz);
+alloc_cqe_mz_fail:
+ rte_free(rxq->rx_info);
+
+alloc_rx_info_fail:
+ hinic3_memzone_free(rxq->rq_mz);
+
+alloc_rq_mz_fail:
+alloc_db_err_fail:
+ hinic3_memzone_free(rxq->pi_mz);
+
+alloc_pi_mz_fail:
+adjust_bufsize_fail:
+ rte_free(rxq);
+ nic_dev->rxqs[qid] = NULL;
+
+ return err;
+}
+
+/**
+ * Create the transmit queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] queue_idx
+ * Transmit queue index.
+ * @param[in] nb_desc
+ * Number of descriptors for transmit queue.
+ * @param[in] socket_id
+ * Socket index on which memory must be allocated.
+ * @param[in] tx_conf
+ * Tx queue configuration parameters (unused_).
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qid, uint16_t nb_desc,
+ unsigned int socket_id,
+ __rte_unused const struct rte_eth_txconf *tx_conf)
+{
+ struct hinic3_nic_dev *nic_dev;
+ struct hinic3_hwdev *hwdev;
+ struct hinic3_txq *txq = NULL;
+ const struct rte_memzone *sq_mz = NULL;
+ const struct rte_memzone *ci_mz = NULL;
+ void *db_addr = NULL;
+ u16 sq_depth, tx_free_thresh;
+ u32 queue_buf_size;
+ int err;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ hwdev = nic_dev->hwdev;
+
+ /* Queue depth must be power of 2, otherwise will be aligned up. */
+ sq_depth = (nb_desc & (nb_desc - 1))
+ ? ((u16)(1U << (ilog2(nb_desc) + 1)))
+ : nb_desc;
+
+ /*
+ * Validate number of transmit descriptors.
+ * It must not exceed hardware maximum and minimum.
+ */
+ if (sq_depth > HINIC3_MAX_QUEUE_DEPTH ||
+ sq_depth < HINIC3_MIN_QUEUE_DEPTH) {
+ PMD_DRV_LOG(ERR,
+ "TX queue depth is out of range from %d to %d,"
+ "(nb_desc: %d, q_depth: %d, port: %d queue: %d)",
+ HINIC3_MIN_QUEUE_DEPTH, HINIC3_MAX_QUEUE_DEPTH,
+ (int)nb_desc, (int)sq_depth,
+ (int)dev->data->port_id, (int)qid);
+ return -EINVAL;
+ }
+
+ /*
+ * The TX descriptor ring will be cleaned after txq->tx_free_thresh
+ * descriptors are used or if the number of descriptors required
+ * to transmit a packet is greater than the number of free TX
+ * descriptors.
+ * The following constraints must be satisfied:
+ * - tx_free_thresh must be greater than 0.
+ * - tx_free_thresh must be less than the size of the ring minus 1.
+ * When set to zero use default values.
+ */
+ tx_free_thresh = (u16)((tx_conf->tx_free_thresh)
+ ? tx_conf->tx_free_thresh
+ : HINIC3_DEFAULT_TX_FREE_THRESH);
+ if (tx_free_thresh >= (sq_depth - 1)) {
+ PMD_DRV_LOG(ERR,
+ "tx_free_thresh must be less than the number of tx "
+ "descriptors minus 1, tx_free_thresh: %u port: %d "
+ "queue: %d",
+ (unsigned int)tx_free_thresh,
+ (int)dev->data->port_id, (int)qid);
+ return -EINVAL;
+ }
+
+ txq = rte_zmalloc_socket("hinic3_tx_queue", sizeof(struct hinic3_txq),
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!txq) {
+ PMD_DRV_LOG(ERR, "Allocate txq[%d] failed, dev_name: %s", qid,
+ dev->data->name);
+ return -ENOMEM;
+ }
+ nic_dev->txqs[qid] = txq;
+ txq->nic_dev = nic_dev;
+ txq->q_id = qid;
+ txq->q_depth = sq_depth;
+ txq->q_mask = sq_depth - 1;
+ txq->cons_idx = 0;
+ txq->prod_idx = 0;
+ txq->wqebb_shift = HINIC3_SQ_WQEBB_SHIFT;
+ txq->wqebb_size = (u16)BIT(txq->wqebb_shift);
+ txq->tx_free_thresh = tx_free_thresh;
+ txq->owner = 1;
+ txq->cos = nic_dev->default_cos;
+
+ ci_mz = hinic3_dma_zone_reserve(dev, "hinic3_sq_ci", qid,
+ HINIC3_CI_Q_ADDR_SIZE,
+ HINIC3_CI_Q_ADDR_SIZE, (int)socket_id);
+ if (!ci_mz) {
+ PMD_DRV_LOG(ERR, "Allocate txq[%d] ci_mz failed, dev_name: %s",
+ qid, dev->data->name);
+ err = -ENOMEM;
+ goto alloc_ci_mz_fail;
+ }
+ txq->ci_mz = ci_mz;
+ txq->ci_dma_base = ci_mz->iova;
+ txq->ci_vaddr_base = (volatile u16 *)ci_mz->addr;
+
+ queue_buf_size = BIT(txq->wqebb_shift) * sq_depth;
+ sq_mz = hinic3_dma_zone_reserve(dev, "hinic3_sq_mz", qid,
+ queue_buf_size, RTE_PGSIZE_256K,
+ (int)socket_id);
+ if (!sq_mz) {
+ PMD_DRV_LOG(ERR, "Allocate txq[%d] sq_mz failed, dev_name: %s",
+ qid, dev->data->name);
+ err = -ENOMEM;
+ goto alloc_sq_mz_fail;
+ }
+ memset(sq_mz->addr, 0, queue_buf_size);
+ txq->sq_mz = sq_mz;
+ txq->queue_buf_paddr = sq_mz->iova;
+ txq->queue_buf_vaddr = sq_mz->addr;
+ txq->sq_head_addr = (u64)txq->queue_buf_vaddr;
+ txq->sq_bot_sge_addr = txq->sq_head_addr + queue_buf_size;
+
+ err = hinic3_alloc_db_addr(hwdev, &db_addr, HINIC3_DB_TYPE_SQ);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc sq doorbell addr failed");
+ goto alloc_db_err_fail;
+ }
+ txq->db_addr = db_addr;
+
+ txq->tx_info = rte_zmalloc_socket("tx_info",
+ sq_depth * sizeof(*txq->tx_info),
+ RTE_CACHE_LINE_SIZE, (int)socket_id);
+ if (!txq->tx_info) {
+ PMD_DRV_LOG(ERR, "Allocate tx_info failed, dev_name: %s",
+ dev->data->name);
+ err = -ENOMEM;
+ goto alloc_tx_info_fail;
+ }
+
+ /* Record txq pointer in rte_eth tx_queues. */
+ dev->data->tx_queues[qid] = txq;
+
+ return 0;
+
+alloc_tx_info_fail:
+alloc_db_err_fail:
+ hinic3_memzone_free(txq->sq_mz);
+
+alloc_sq_mz_fail:
+ hinic3_memzone_free(txq->ci_mz);
+
+alloc_ci_mz_fail:
+ rte_free(txq);
+ return err;
+}
+
+static void
+hinic3_rx_queue_release(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ if (dev == NULL || dev->data == NULL || dev->data->rx_queues == NULL) {
+ PMD_DRV_LOG(WARNING, "rx queue is null when release");
+ return;
+ }
+ if (queue_id >= dev->data->nb_rx_queues) {
+ PMD_DRV_LOG(WARNING, "eth_dev: %s, rx queue id: %u is illegal",
+ dev->data->name, queue_id);
+ return;
+ }
+ struct hinic3_rxq *rxq = dev->data->rx_queues[queue_id];
+ struct hinic3_nic_dev *nic_dev = NULL;
+
+ if (!rxq) {
+ PMD_DRV_LOG(WARNING, "Rxq is null when release");
+ return;
+ }
+
+ nic_dev = rxq->nic_dev;
+
+ hinic3_free_rxq_mbufs(rxq);
+
+ hinic3_memzone_free(rxq->cqe_mz);
+
+ rte_free(rxq->rx_info);
+ rxq->rx_info = NULL;
+
+ hinic3_memzone_free(rxq->rq_mz);
+
+ hinic3_memzone_free(rxq->pi_mz);
+
+ nic_dev->rxqs[rxq->q_id] = NULL;
+ rte_free(rxq);
+ dev->data->rx_queues[queue_id] = NULL;
+}
+
+static void
+hinic3_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ if (dev == NULL || dev->data == NULL || dev->data->tx_queues == NULL) {
+ PMD_DRV_LOG(WARNING, "tx queue is null when release");
+ return;
+ }
+ if (queue_id >= dev->data->nb_tx_queues) {
+ PMD_DRV_LOG(WARNING, "eth_dev: %s, tx queue id: %u is illegal",
+ dev->data->name, queue_id);
+ return;
+ }
+ struct hinic3_txq *txq = dev->data->tx_queues[queue_id];
+ struct hinic3_nic_dev *nic_dev = NULL;
+
+ if (!txq) {
+ PMD_DRV_LOG(WARNING, "Txq is null when release");
+ return;
+ }
+ PMD_DRV_LOG(INFO, "%s txq_idx:%d queue release.",
+ txq->nic_dev->dev_name, txq->q_id);
+ nic_dev = txq->nic_dev;
+
+ hinic3_free_txq_mbufs(txq);
+
+ rte_free(txq->tx_info);
+ txq->tx_info = NULL;
+
+ hinic3_memzone_free(txq->sq_mz);
+
+ hinic3_memzone_free(txq->ci_mz);
+
+ nic_dev->txqs[txq->q_id] = NULL;
+ rte_free(txq);
+ dev->data->tx_queues[queue_id] = NULL;
+}
+
+/**
+ * Start RXQ and enables flow director (fdir) filter for RXQ.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] rq_id
+ * RX queue ID to be started.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_rx_queue_start(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t rq_id)
+{
+ struct hinic3_rxq *rxq = NULL;
+ int rc;
+
+ if (rq_id < dev->data->nb_rx_queues) {
+ rxq = dev->data->rx_queues[rq_id];
+
+ rc = hinic3_start_rq(dev, rxq);
+ if (rc) {
+ PMD_DRV_LOG(ERR,
+ "Start rx queue failed, eth_dev:%s, "
+ "queue_idx:%d",
+ dev->data->name, rq_id);
+ return rc;
+ }
+
+ dev->data->rx_queue_state[rq_id] = RTE_ETH_QUEUE_STATE_STARTED;
+ }
+ rc = hinic3_enable_rxq_fdir_filter(dev, (u32)rq_id, (u32)true);
+ if (rc) {
+ PMD_DRV_LOG(ERR, "Failed to enable rq : %d fdir filter.",
+ rq_id);
+ return rc;
+ }
+ return 0;
+}
+
+/**
+ * Stop RXQ and disable flow director (fdir) filter for RXQ.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] rq_id
+ * RX queue ID to be stopped.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_rx_queue_stop(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t rq_id)
+{
+ struct hinic3_rxq *rxq = NULL;
+ int rc;
+
+ if (rq_id < dev->data->nb_rx_queues) {
+ rxq = dev->data->rx_queues[rq_id];
+
+ rc = hinic3_stop_rq(dev, rxq);
+ if (rc) {
+ PMD_DRV_LOG(ERR,
+ "Stop rx queue failed, eth_dev:%s, "
+ "queue_idx:%d",
+ dev->data->name, rq_id);
+ return rc;
+ }
+
+ dev->data->rx_queue_state[rq_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+ }
+ rc = hinic3_enable_rxq_fdir_filter(dev, (u32)rq_id, (u32)false);
+ if (rc) {
+ PMD_DRV_LOG(ERR, "Failed to disable rq : %d fdir filter.",
+ rq_id);
+ return rc;
+ }
+
+ return 0;
+}
+
+static int
+hinic3_dev_tx_queue_start(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t sq_id)
+{
+ struct hinic3_txq *txq = NULL;
+
+ PMD_DRV_LOG(INFO, "Start tx queue, eth_dev:%s, queue_idx:%d",
+ dev->data->name, sq_id);
+
+ txq = dev->data->tx_queues[sq_id];
+ HINIC3_SET_TXQ_STARTED(txq);
+ dev->data->tx_queue_state[sq_id] = RTE_ETH_QUEUE_STATE_STARTED;
+ return 0;
+}
+
+static int
+hinic3_dev_tx_queue_stop(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t sq_id)
+{
+ struct hinic3_txq *txq = NULL;
+ int rc;
+
+ if (sq_id < dev->data->nb_tx_queues) {
+ txq = dev->data->tx_queues[sq_id];
+ rc = hinic3_stop_sq(txq);
+ if (rc) {
+ PMD_DRV_LOG(ERR,
+ "Stop tx queue failed, eth_dev:%s, "
+ "queue_idx:%d",
+ dev->data->name, sq_id);
+ return rc;
+ }
+
+ HINIC3_SET_TXQ_STOPPED(txq);
+ dev->data->tx_queue_state[sq_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+ }
+
+ return 0;
+}
+
+int
+hinic3_dev_rx_queue_intr_enable(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ struct rte_intr_handle *intr_handle = PCI_DEV_TO_INTR_HANDLE(pci_dev);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u16 msix_intr;
+
+ if (!rte_intr_dp_is_en(intr_handle) || !intr_handle->intr_vec)
+ return 0;
+
+ if (queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ msix_intr = (u16)intr_handle->intr_vec[queue_id];
+ hinic3_set_msix_auto_mask_state(nic_dev->hwdev, msix_intr,
+ HINIC3_SET_MSIX_AUTO_MASK);
+ hinic3_set_msix_state(nic_dev->hwdev, msix_intr, HINIC3_MSIX_ENABLE);
+
+ return 0;
+}
+
+int
+hinic3_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ struct rte_intr_handle *intr_handle = PCI_DEV_TO_INTR_HANDLE(pci_dev);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u16 msix_intr;
+
+ if (!rte_intr_dp_is_en(intr_handle) || !intr_handle->intr_vec)
+ return 0;
+
+ if (queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ msix_intr = (u16)intr_handle->intr_vec[queue_id];
+ hinic3_set_msix_auto_mask_state(nic_dev->hwdev, msix_intr,
+ HINIC3_CLR_MSIX_AUTO_MASK);
+ hinic3_set_msix_state(nic_dev->hwdev, msix_intr, HINIC3_MSIX_DISABLE);
+ hinic3_misx_intr_clear_resend_bit(nic_dev->hwdev, msix_intr,
+ MSIX_RESEND_TIMER_CLEAR);
+
+ return 0;
+}
+
+static uint32_t
+hinic3_dev_rx_queue_count(__rte_unused void *rx_queue)
+{
+ return 0;
+}
+
+static int
+hinic3_dev_rx_descriptor_status(__rte_unused void *rx_queue,
+ __rte_unused uint16_t offset)
+{
+ return 0;
+}
+
+static int
+hinic3_dev_tx_descriptor_status(__rte_unused void *tx_queue,
+ __rte_unused uint16_t offset)
+{
+ return 0;
+}
+
+static int
+hinic3_set_lro(struct hinic3_nic_dev *nic_dev, struct rte_eth_conf *dev_conf)
+{
+ bool lro_en;
+ int max_lro_size, lro_max_pkt_len;
+ int err;
+
+ /* Config lro. */
+ lro_en = dev_conf->rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO ? true
+ : false;
+ max_lro_size = (int)(dev_conf->rxmode.max_lro_pkt_size);
+ /* `max_lro_size` is divisible by `HINIC3_LRO_UNIT_WQE_SIZE`. */
+ lro_max_pkt_len = max_lro_size / HINIC3_LRO_UNIT_WQE_SIZE
+ ? max_lro_size / HINIC3_LRO_UNIT_WQE_SIZE
+ : 1;
+
+ PMD_DRV_LOG(INFO,
+ "max_lro_size: %d, rx_buff_len: %d, lro_max_pkt_len: %d",
+ max_lro_size, nic_dev->rx_buff_len, lro_max_pkt_len);
+ PMD_DRV_LOG(INFO, "max_rx_pkt_len: %d",
+ HINIC3_MAX_RX_PKT_LEN(dev_conf->rxmode));
+ err = hinic3_set_rx_lro_state(nic_dev->hwdev, lro_en,
+ HINIC3_LRO_DEFAULT_TIME_LIMIT,
+ lro_max_pkt_len);
+ if (err)
+ PMD_DRV_LOG(ERR, "Set lro state failed, err: %d", err);
+ return err;
+}
+
+static int
+hinic3_set_vlan(struct rte_eth_dev *dev, struct rte_eth_conf *dev_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ bool vlan_filter, vlan_strip;
+ int err;
+
+ /* Config vlan filter. */
+ vlan_filter = dev_conf->rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER;
+
+ err = hinic3_set_vlan_fliter(nic_dev->hwdev, vlan_filter);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Config vlan filter failed, device: %s, port_id: "
+ "%d, err: %d",
+ nic_dev->dev_name, dev->data->port_id, err);
+ return err;
+ }
+
+ /* Config vlan stripping. */
+ vlan_strip = dev_conf->rxmode.offloads & RTE_ETH_RX_OFFLOAD_VLAN_STRIP;
+
+ err = hinic3_set_rx_vlan_offload(nic_dev->hwdev, vlan_strip);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Config vlan strip failed, device: %s, port_id: "
+ "%d, err: %d",
+ nic_dev->dev_name, dev->data->port_id, err);
+ }
+
+ return err;
+}
+
+/**
+ * Configure RX mode, checksum offload, LRO, RSS, VLAN and initialize the RXQ
+ * list.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_set_rxtx_configure(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_eth_conf *dev_conf = &dev->data->dev_conf;
+ struct rte_eth_rss_conf *rss_conf = NULL;
+ int err;
+
+ /* Config rx mode. */
+ err = hinic3_set_rx_mode(nic_dev->hwdev, HINIC3_DEFAULT_RX_MODE);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set rx_mode: 0x%x failed",
+ HINIC3_DEFAULT_RX_MODE);
+ return err;
+ }
+ nic_dev->rx_mode = HINIC3_DEFAULT_RX_MODE;
+
+ /* Config rx checksum offload. */
+ if (dev_conf->rxmode.offloads & RTE_ETH_RX_OFFLOAD_CHECKSUM)
+ nic_dev->rx_csum_en = HINIC3_DEFAULT_RX_CSUM_OFFLOAD;
+
+ err = hinic3_set_lro(nic_dev, dev_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set lro failed");
+ return err;
+ }
+ /* Config RSS. */
+ if ((dev_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) &&
+ nic_dev->num_rqs > 1) {
+ rss_conf = &dev_conf->rx_adv_conf.rss_conf;
+ err = hinic3_update_rss_config(dev, rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set rss config failed, err: %d", err);
+ return err;
+ }
+ }
+
+ err = hinic3_set_vlan(dev, dev_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set vlan failed, err: %d", err);
+ return err;
+ }
+
+ hinic3_init_rx_queue_list(nic_dev);
+
+ return 0;
+}
+
+/**
+ * Disable RX mode and RSS, and free associated resources.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ */
+static void
+hinic3_remove_rxtx_configure(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u8 prio_tc[HINIC3_DCB_UP_MAX] = {0};
+
+ hinic3_set_rx_mode(nic_dev->hwdev, 0);
+
+ if (nic_dev->rss_state == HINIC3_RSS_ENABLE) {
+ hinic3_rss_cfg(nic_dev->hwdev, HINIC3_RSS_DISABLE, 0, prio_tc);
+ hinic3_rss_template_free(nic_dev->hwdev);
+ nic_dev->rss_state = HINIC3_RSS_DISABLE;
+ }
+}
+
+static bool
+hinic3_find_vlan_filter(struct hinic3_nic_dev *nic_dev, uint16_t vlan_id)
+{
+ u32 vid_idx, vid_bit;
+
+ vid_idx = HINIC3_VFTA_IDX(vlan_id);
+ vid_bit = HINIC3_VFTA_BIT(vlan_id);
+
+ return (nic_dev->vfta[vid_idx] & vid_bit) ? true : false;
+}
+
+static void
+hinic3_store_vlan_filter(struct hinic3_nic_dev *nic_dev, u16 vlan_id, bool on)
+{
+ u32 vid_idx, vid_bit;
+
+ vid_idx = HINIC3_VFTA_IDX(vlan_id);
+ vid_bit = HINIC3_VFTA_BIT(vlan_id);
+
+ if (on)
+ nic_dev->vfta[vid_idx] |= vid_bit;
+ else
+ nic_dev->vfta[vid_idx] &= ~vid_bit;
+}
+
+static void
+hinic3_remove_all_vlanid(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int vlan_id;
+ u16 func_id;
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+
+ for (vlan_id = 1; vlan_id < RTE_ETHER_MAX_VLAN_ID; vlan_id++) {
+ if (hinic3_find_vlan_filter(nic_dev, vlan_id)) {
+ hinic3_del_vlan(nic_dev->hwdev, vlan_id, func_id);
+ hinic3_store_vlan_filter(nic_dev, vlan_id, false);
+ }
+ }
+}
+
+static void
+hinic3_disable_interrupt(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+
+ if (!hinic3_get_bit(HINIC3_DEV_INIT, &nic_dev->dev_status))
+ return;
+
+ /* Disable rte interrupt. */
+ rte_intr_disable(PCI_DEV_TO_INTR_HANDLE(pci_dev));
+ rte_intr_callback_unregister(PCI_DEV_TO_INTR_HANDLE(pci_dev),
+ hinic3_dev_interrupt_handler, (void *)dev);
+}
+
+static void
+hinic3_enable_interrupt(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+
+ if (!hinic3_get_bit(HINIC3_DEV_INIT, &nic_dev->dev_status))
+ return;
+
+ /* Enable rte interrupt. */
+ rte_intr_enable(PCI_DEV_TO_INTR_HANDLE(pci_dev));
+ rte_intr_callback_register(PCI_DEV_TO_INTR_HANDLE(pci_dev),
+ hinic3_dev_interrupt_handler, (void *)dev);
+}
+
+#define HINIC3_RX_VEC_START RTE_INTR_VEC_RXTX_OFFSET
+
+/** Dp interrupt msix attribute. */
+#define HINIC3_TXRX_MSIX_PENDING_LIMIT 2
+#define HINIC3_TXRX_MSIX_COALESC_TIMER 2
+#define HINIC3_TXRX_MSIX_RESEND_TIMER_CFG 7
+
+static int
+hinic3_init_rxq_msix_attr(void *hwdev, u16 msix_index)
+{
+ struct interrupt_info info = {0};
+ int err;
+
+ info.lli_set = 0;
+ info.interrupt_coalesc_set = 1;
+ info.pending_limt = HINIC3_TXRX_MSIX_PENDING_LIMIT;
+ info.coalesc_timer_cfg = HINIC3_TXRX_MSIX_COALESC_TIMER;
+ info.resend_timer_cfg = HINIC3_TXRX_MSIX_RESEND_TIMER_CFG;
+
+ info.msix_index = msix_index;
+ err = hinic3_set_interrupt_cfg(hwdev, info);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set msix attr failed, msix_index %d",
+ msix_index);
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
+static void
+hinic3_deinit_rxq_intr(struct rte_eth_dev *dev)
+{
+ struct rte_intr_handle *intr_handle = dev->intr_handle;
+
+ rte_intr_efd_disable(intr_handle);
+ if (intr_handle->intr_vec) {
+ rte_free(intr_handle->intr_vec);
+ intr_handle->intr_vec = NULL;
+ }
+}
+
+/**
+ * Initialize RX queue interrupts by enabling MSI-X, allocate interrupt vectors,
+ * and configure interrupt attributes for each RX queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, negative error code on failure.
+ * - -ENOTSUP if MSI-X interrupts are not supported.
+ * - Error code if enabling event file descriptors fails.
+ * - -ENOMEM if allocating interrupt vectors fails.
+ */
+static int
+hinic3_init_rxq_intr(struct rte_eth_dev *dev)
+{
+ struct rte_intr_handle *intr_handle = NULL;
+ struct hinic3_nic_dev *nic_dev = NULL;
+ struct hinic3_rxq *rxq = NULL;
+ u32 nb_rx_queues, i;
+ int err;
+
+ intr_handle = dev->intr_handle;
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ if (!dev->data->dev_conf.intr_conf.rxq)
+ return 0;
+
+ if (!rte_intr_cap_multiple(intr_handle)) {
+ PMD_DRV_LOG(ERR, "Rx queue interrupts require MSI-X interrupts"
+ " (vfio-pci driver)");
+ return -ENOTSUP;
+ }
+
+ nb_rx_queues = dev->data->nb_rx_queues;
+ err = rte_intr_efd_enable(intr_handle, nb_rx_queues);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to enable event fds for Rx queue interrupts");
+ return err;
+ }
+
+ intr_handle->intr_vec =
+ rte_zmalloc("hinic_intr_vec", nb_rx_queues * sizeof(int), 0);
+ if (intr_handle->intr_vec == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate intr_vec");
+ rte_intr_efd_disable(intr_handle);
+ return -ENOMEM;
+ }
+ intr_handle->vec_list_size = nb_rx_queues;
+ for (i = 0; i < nb_rx_queues; i++)
+ intr_handle->intr_vec[i] = (int)(i + HINIC3_RX_VEC_START);
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ rxq->dp_intr_en = 1;
+ rxq->msix_entry_idx = (u16)intr_handle->intr_vec[i];
+
+ err = hinic3_init_rxq_msix_attr(nic_dev->hwdev,
+ rxq->msix_entry_idx);
+ if (err) {
+ hinic3_deinit_rxq_intr(dev);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static int
+hinic3_init_sw_rxtxqs(struct hinic3_nic_dev *nic_dev)
+{
+ u32 txq_size;
+ u32 rxq_size;
+
+ /* Allocate software txq array. */
+ txq_size = nic_dev->max_sqs * sizeof(*nic_dev->txqs);
+ nic_dev->txqs =
+ rte_zmalloc("hinic3_txqs", txq_size, RTE_CACHE_LINE_SIZE);
+ if (!nic_dev->txqs) {
+ PMD_DRV_LOG(ERR, "Allocate txqs failed");
+ return -ENOMEM;
+ }
+
+ /* Allocate software rxq array. */
+ rxq_size = nic_dev->max_rqs * sizeof(*nic_dev->rxqs);
+ nic_dev->rxqs =
+ rte_zmalloc("hinic3_rxqs", rxq_size, RTE_CACHE_LINE_SIZE);
+ if (!nic_dev->rxqs) {
+ /* Free txqs. */
+ rte_free(nic_dev->txqs);
+ nic_dev->txqs = NULL;
+
+ PMD_DRV_LOG(ERR, "Allocate rxqs failed");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void
+hinic3_deinit_sw_rxtxqs(struct hinic3_nic_dev *nic_dev)
+{
+ rte_free(nic_dev->txqs);
+ nic_dev->txqs = NULL;
+
+ rte_free(nic_dev->rxqs);
+ nic_dev->rxqs = NULL;
+}
+
+static void
+hinic3_disable_queue_intr(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_intr_handle *intr_handle = dev->intr_handle;
+ int msix_intr;
+ int i;
+
+ if (intr_handle->intr_vec == NULL)
+ return;
+
+ for (i = 0; i < nic_dev->num_rqs; i++) {
+ msix_intr = intr_handle->intr_vec[i];
+ hinic3_set_msix_state(nic_dev->hwdev, (u16)msix_intr,
+ HINIC3_MSIX_DISABLE);
+ hinic3_misx_intr_clear_resend_bit(nic_dev->hwdev,
+ (u16)msix_intr,
+ MSIX_RESEND_TIMER_CLEAR);
+ }
+}
+
+/**
+ * Start the device.
+ *
+ * Initialize function table, TXQ and TXQ context, configure RX offload, and
+ * enable vport and port to prepare receiving packets.
+ *
+ * @param[in] eth_dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_start(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+ u64 nic_features;
+ struct hinic3_rxq *rxq = NULL;
+ int i;
+ int err;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ err = hinic3_copy_mempool_init(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Create copy mempool failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_mpool_fail;
+ }
+ hinic3_update_msix_info(nic_dev->hwdev->hwif);
+ hinic3_disable_interrupt(eth_dev);
+ err = hinic3_init_rxq_intr(eth_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init rxq intr fail, eth_dev:%s",
+ eth_dev->data->name);
+ goto init_rxq_intr_fail;
+ }
+
+ hinic3_get_func_rx_buf_size(nic_dev);
+ err = hinic3_init_function_table(nic_dev->hwdev, nic_dev->rx_buff_len);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init function table failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_func_tbl_fail;
+ }
+
+ nic_features = hinic3_get_driver_feature(nic_dev);
+ /*
+ * You can update the features supported by the driver according to the
+ * scenario here.
+ */
+ nic_features &= DEFAULT_DRV_FEATURE;
+ hinic3_update_driver_feature(nic_dev, nic_features);
+
+ err = hinic3_set_feature_to_hw(nic_dev->hwdev, &nic_dev->feature_cap,
+ 1);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to set nic features to hardware, err %d",
+ err);
+ goto get_feature_err;
+ }
+
+ /* Reset rx and tx queue. */
+ hinic3_reset_rx_queue(eth_dev);
+ hinic3_reset_tx_queue(eth_dev);
+
+ /* Init txq and rxq context. */
+ err = hinic3_init_qp_ctxts(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init qp context failed, dev_name: %s",
+ eth_dev->data->name);
+ goto init_qp_fail;
+ }
+
+ /* Set default mtu. */
+ err = hinic3_set_port_mtu(nic_dev->hwdev, nic_dev->mtu_size);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set mtu_size[%d] failed, dev_name: %s",
+ nic_dev->mtu_size, eth_dev->data->name);
+ goto set_mtu_fail;
+ }
+ eth_dev->data->mtu = nic_dev->mtu_size;
+
+ /* Set rx configuration: rss/checksum/rxmode/lro. */
+ err = hinic3_set_rxtx_configure(eth_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set rx config failed, dev_name: %s",
+ eth_dev->data->name);
+ goto set_rxtx_config_fail;
+ }
+
+ /* Enable dev interrupt. */
+ hinic3_enable_interrupt(eth_dev);
+ err = hinic3_start_all_rqs(eth_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set rx config failed, dev_name: %s",
+ eth_dev->data->name);
+ goto start_rqs_fail;
+ }
+
+ hinic3_start_all_sqs(eth_dev);
+
+ /* Open virtual port and ready to start packet receiving. */
+ err = hinic3_set_vport_enable(nic_dev->hwdev, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Enable vport failed, dev_name: %s",
+ eth_dev->data->name);
+ goto en_vport_fail;
+ }
+
+ /* Open physical port and start packet receiving. */
+ err = hinic3_set_port_enable(nic_dev->hwdev, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Enable physical port failed, dev_name: %s",
+ eth_dev->data->name);
+ goto en_port_fail;
+ }
+
+ /* Update eth_dev link status. */
+ if (eth_dev->data->dev_conf.intr_conf.lsc != 0)
+ (void)hinic3_link_update(eth_dev, 0);
+
+ hinic3_set_bit(HINIC3_DEV_START, &nic_dev->dev_status);
+
+ return 0;
+
+en_port_fail:
+ (void)hinic3_set_vport_enable(nic_dev->hwdev, false);
+
+en_vport_fail:
+ /* Flush tx && rx chip resources in case of setting vport fake fail. */
+ (void)hinic3_flush_qps_res(nic_dev->hwdev);
+ rte_delay_ms(DEV_START_DELAY_MS);
+ for (i = 0; i < nic_dev->num_rqs; i++) {
+ rxq = nic_dev->rxqs[i];
+ hinic3_remove_rq_from_rx_queue_list(nic_dev, rxq->q_id);
+ hinic3_free_rxq_mbufs(rxq);
+ hinic3_dev_rx_queue_intr_disable(eth_dev, rxq->q_id);
+ eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+ eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+ }
+start_rqs_fail:
+ hinic3_remove_rxtx_configure(eth_dev);
+
+set_rxtx_config_fail:
+set_mtu_fail:
+ hinic3_free_qp_ctxts(nic_dev->hwdev);
+
+init_qp_fail:
+get_feature_err:
+init_func_tbl_fail:
+ hinic3_deinit_rxq_intr(eth_dev);
+init_rxq_intr_fail:
+ hinic3_copy_mempool_uninit(nic_dev);
+init_mpool_fail:
+ return err;
+}
+
+/**
+ * Look up or creates a memory pool for storing packet buffers used in copy
+ * operations.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * `-ENOMEM`: Memory pool creation fails.
+ */
+static int
+hinic3_copy_mempool_init(struct hinic3_nic_dev *nic_dev)
+{
+ nic_dev->cpy_mpool = rte_mempool_lookup(HINCI3_CPY_MEMPOOL_NAME);
+ if (nic_dev->cpy_mpool == NULL) {
+ nic_dev->cpy_mpool = rte_pktmbuf_pool_create(HINCI3_CPY_MEMPOOL_NAME,
+ HINIC3_COPY_MEMPOOL_DEPTH, HINIC3_COPY_MEMPOOL_CACHE,
+ 0, HINIC3_COPY_MBUF_SIZE, (int)rte_socket_id());
+ if (nic_dev->cpy_mpool == NULL) {
+ PMD_DRV_LOG(ERR,
+ "Create copy mempool failed, errno: %d, "
+ "dev_name: %s",
+ rte_errno, HINCI3_CPY_MEMPOOL_NAME);
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Clear the reference to the copy memory pool without freeing it.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ */
+static void
+hinic3_copy_mempool_uninit(struct hinic3_nic_dev *nic_dev)
+{
+ nic_dev->cpy_mpool = NULL;
+}
+
+/**
+ * Stop the device.
+ *
+ * Stop phy port and vport, flush pending io request, clean context configure
+ * and free io resourece.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ */
+static int
+hinic3_dev_stop(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev;
+ struct rte_eth_link link;
+ int err;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ if (!hinic3_test_and_clear_bit(HINIC3_DEV_START,
+ &nic_dev->dev_status)) {
+ PMD_DRV_LOG(INFO, "Device %s already stopped",
+ nic_dev->dev_name);
+ return 0;
+ }
+
+ /* Stop phy port and vport. */
+ err = hinic3_set_port_enable(nic_dev->hwdev, false);
+ if (err)
+ PMD_DRV_LOG(WARNING,
+ "Disable phy port failed, error: %d, "
+ "dev_name: %s, port_id: %d",
+ err, dev->data->name, dev->data->port_id);
+
+ err = hinic3_set_vport_enable(nic_dev->hwdev, false);
+ if (err)
+ PMD_DRV_LOG(WARNING,
+ "Disable vport failed, error: %d, "
+ "dev_name: %s, port_id: %d",
+ err, dev->data->name, dev->data->port_id);
+
+ /* Clear recorded link status. */
+ memset(&link, 0, sizeof(link));
+ (void)rte_eth_linkstatus_set(dev, &link);
+
+ /* Disable dp interrupt. */
+ hinic3_disable_queue_intr(dev);
+ hinic3_deinit_rxq_intr(dev);
+
+ /* Flush pending io request. */
+ hinic3_flush_txqs(nic_dev);
+
+ /* After set vport disable 100ms, no packets will be send to host. */
+ rte_delay_ms(DEV_STOP_DELAY_MS);
+
+ hinic3_flush_qps_res(nic_dev->hwdev);
+
+ /* Clean RSS table and rx_mode. */
+ hinic3_remove_rxtx_configure(dev);
+
+ /* Clean root context. */
+ hinic3_free_qp_ctxts(nic_dev->hwdev);
+
+ /* Free all tx and rx mbufs. */
+ hinic3_free_all_txq_mbufs(nic_dev);
+ hinic3_free_all_rxq_mbufs(nic_dev);
+
+ /* Free mempool. */
+ hinic3_copy_mempool_uninit(nic_dev);
+ return 0;
+}
+
+static void
+hinic3_dev_release(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev =
+ HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+ int qid;
+
+ /* Release io resource. */
+ for (qid = 0; qid < nic_dev->num_sqs; qid++)
+ hinic3_tx_queue_release(eth_dev, qid);
+
+ for (qid = 0; qid < nic_dev->num_rqs; qid++)
+ hinic3_rx_queue_release(eth_dev, qid);
+
+ hinic3_deinit_sw_rxtxqs(nic_dev);
+
+ hinic3_deinit_mac_addr(eth_dev);
+ rte_free(nic_dev->mc_list);
+
+ hinic3_remove_all_vlanid(eth_dev);
+
+ hinic3_clear_bit(HINIC3_DEV_INTR_EN, &nic_dev->dev_status);
+ hinic3_set_msix_state(nic_dev->hwdev, 0, HINIC3_MSIX_DISABLE);
+ rte_intr_disable(PCI_DEV_TO_INTR_HANDLE(pci_dev));
+ (void)rte_intr_callback_unregister(PCI_DEV_TO_INTR_HANDLE(pci_dev),
+ hinic3_dev_interrupt_handler,
+ (void *)eth_dev);
+
+ /* Destroy rx mode mutex. */
+ hinic3_mutex_destroy(&nic_dev->rx_mode_mutex);
+
+ hinic3_free_nic_hwdev(nic_dev->hwdev);
+ hinic3_free_hwdev(nic_dev->hwdev);
+
+ eth_dev->rx_pkt_burst = NULL;
+ eth_dev->tx_pkt_burst = NULL;
+ eth_dev->dev_ops = NULL;
+ eth_dev->rx_queue_count = NULL;
+ eth_dev->rx_descriptor_status = NULL;
+ eth_dev->tx_descriptor_status = NULL;
+
+ rte_free(nic_dev->hwdev);
+ nic_dev->hwdev = NULL;
+}
+
+/**
+ * Close the device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_close(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev =
+ HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+ int ret;
+
+ if (hinic3_test_and_set_bit(HINIC3_DEV_CLOSE, &nic_dev->dev_status)) {
+ PMD_DRV_LOG(WARNING, "Device %s already closed",
+ nic_dev->dev_name);
+ return 0;
+ }
+
+ ret = hinic3_dev_stop(eth_dev);
+
+ hinic3_dev_release(eth_dev);
+ return ret;
+}
+
+static int
+hinic3_dev_reset(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+#define MIN_RX_BUFFER_SIZE 256
+#define MIN_RX_BUFFER_SIZE_SMALL_MODE 1518
+
+static int
+hinic3_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int err = 0;
+
+ PMD_DRV_LOG(INFO, "Set port mtu, port_id: %d, mtu: %d, max_pkt_len: %d",
+ dev->data->port_id, mtu, HINIC3_MTU_TO_PKTLEN(mtu));
+
+ if (mtu < HINIC3_MIN_MTU_SIZE || mtu > HINIC3_MAX_MTU_SIZE) {
+ PMD_DRV_LOG(ERR, "Invalid mtu: %d, must between %d and %d", mtu,
+ HINIC3_MIN_MTU_SIZE, HINIC3_MAX_MTU_SIZE);
+ return -EINVAL;
+ }
+
+ err = hinic3_set_port_mtu(nic_dev->hwdev, mtu);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set port mtu failed, err: %d", err);
+ return err;
+ }
+
+ /* Update max frame size. */
+ HINIC3_MAX_RX_PKT_LEN(dev->data->dev_conf.rxmode) =
+ HINIC3_MTU_TO_PKTLEN(mtu);
+ nic_dev->mtu_size = mtu;
+ return err;
+}
+
+/**
+ * Add or delete vlan id.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] vlan_id
+ * Vlan id is used to filter vlan packets.
+ * @param[in] enable
+ * Disable or enable vlan filter function.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_vlan_filter_set(struct rte_eth_dev *dev, uint16_t vlan_id, int enable)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int err = 0;
+ u16 func_id;
+
+ if (vlan_id >= RTE_ETHER_MAX_VLAN_ID)
+ return -EINVAL;
+
+ if (vlan_id == 0)
+ return 0;
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+
+ if (enable) {
+ /* If vlanid is already set, just return. */
+ if (hinic3_find_vlan_filter(nic_dev, vlan_id)) {
+ PMD_DRV_LOG(INFO, "Vlan %u has been added, device: %s",
+ vlan_id, nic_dev->dev_name);
+ return 0;
+ }
+
+ err = hinic3_add_vlan(nic_dev->hwdev, vlan_id, func_id);
+ } else {
+ /* If vlanid can't be found, just return. */
+ if (!hinic3_find_vlan_filter(nic_dev, vlan_id)) {
+ PMD_DRV_LOG(INFO,
+ "Vlan %u is not in the vlan filter list, "
+ "device: %s",
+ vlan_id, nic_dev->dev_name);
+ return 0;
+ }
+
+ err = hinic3_del_vlan(nic_dev->hwdev, vlan_id, func_id);
+ }
+
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "%s vlan failed, func_id: %d, vlan_id: %d, err: %d",
+ enable ? "Add" : "Remove", func_id, vlan_id, err);
+ return err;
+ }
+
+ hinic3_store_vlan_filter(nic_dev, vlan_id, enable);
+
+ PMD_DRV_LOG(INFO, "%s vlan %u succeed, device: %s",
+ enable ? "Add" : "Remove", vlan_id, nic_dev->dev_name);
+
+ return 0;
+}
+
+/**
+ * Enable or disable vlan offload.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] mask
+ * Definitions used for VLAN setting, vlan filter of vlan strip.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_vlan_offload_set(struct rte_eth_dev *dev, int mask)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ bool on;
+ int err;
+
+ /* Enable or disable VLAN filter. */
+ if (mask & RTE_ETH_VLAN_FILTER_MASK) {
+ on = (rxmode->offloads & RTE_ETH_RX_OFFLOAD_VLAN_FILTER)
+ ? true
+ : false;
+ err = hinic3_set_vlan_fliter(nic_dev->hwdev, on);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "%s vlan filter failed, device: %s, "
+ "port_id: %d, err: %d",
+ on ? "Enable" : "Disable",
+ nic_dev->dev_name, dev->data->port_id, err);
+ return err;
+ }
+
+ PMD_DRV_LOG(INFO,
+ "%s vlan filter succeed, device: %s, port_id: %d",
+ on ? "Enable" : "Disable", nic_dev->dev_name,
+ dev->data->port_id);
+ }
+
+ /* Enable or disable VLAN stripping. */
+ if (mask & RTE_ETH_VLAN_STRIP_MASK) {
+ on = (rxmode->offloads & RTE_ETH_RX_OFFLOAD_VLAN_STRIP) ? true
+ : false;
+ err = hinic3_set_rx_vlan_offload(nic_dev->hwdev, on);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "%s vlan strip failed, device: %s, "
+ "port_id: %d, err: %d",
+ on ? "Enable" : "Disable",
+ nic_dev->dev_name, dev->data->port_id, err);
+ return err;
+ }
+
+ PMD_DRV_LOG(INFO,
+ "%s vlan strip succeed, device: %s, port_id: %d",
+ on ? "Enable" : "Disable", nic_dev->dev_name,
+ dev->data->port_id);
+ }
+ return 0;
+}
+
+/**
+ * Enable allmulticast mode.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_allmulticast_enable(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 rx_mode;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->rx_mode_mutex);
+ if (err)
+ return err;
+
+ rx_mode = nic_dev->rx_mode | HINIC3_RX_MODE_MC_ALL;
+
+ err = hinic3_set_rx_mode(nic_dev->hwdev, rx_mode);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+ PMD_DRV_LOG(ERR, "Enable allmulticast failed, error: %d", err);
+ return err;
+ }
+
+ nic_dev->rx_mode = rx_mode;
+
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+
+ PMD_DRV_LOG(INFO,
+ "Enable allmulticast succeed, nic_dev: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return 0;
+}
+
+/**
+ * Disable allmulticast mode.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_allmulticast_disable(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 rx_mode;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->rx_mode_mutex);
+ if (err)
+ return err;
+
+ rx_mode = nic_dev->rx_mode & (~HINIC3_RX_MODE_MC_ALL);
+
+ err = hinic3_set_rx_mode(nic_dev->hwdev, rx_mode);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+ PMD_DRV_LOG(ERR, "Disable allmulticast failed, error: %d", err);
+ return err;
+ }
+
+ nic_dev->rx_mode = rx_mode;
+
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+
+ PMD_DRV_LOG(INFO,
+ "Disable allmulticast succeed, nic_dev: %s, port_id: %d",
+ nic_dev->dev_name, dev->data->port_id);
+ return 0;
+}
+
+/**
+ * Get device generic statistics.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] stats
+ * Stats structure output buffer.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_vport_stats vport_stats;
+ struct hinic3_rxq *rxq = NULL;
+ struct hinic3_txq *txq = NULL;
+ int i, err, q_num;
+ u64 rx_discards_pmd = 0;
+
+ err = hinic3_get_vport_stats(nic_dev->hwdev, &vport_stats);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Get vport stats from fw failed, nic_dev: %s",
+ nic_dev->dev_name);
+ return err;
+ }
+
+ dev->data->rx_mbuf_alloc_failed = 0;
+
+ /* Rx queue stats. */
+ q_num = (nic_dev->num_rqs < RTE_ETHDEV_QUEUE_STAT_CNTRS)
+ ? nic_dev->num_rqs
+ : RTE_ETHDEV_QUEUE_STAT_CNTRS;
+ for (i = 0; i < q_num; i++) {
+ rxq = nic_dev->rxqs[i];
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_left_mbuf_bytes =
+ rxq->rxq_stats.rx_alloc_mbuf_bytes -
+ rxq->rxq_stats.rx_free_mbuf_bytes;
+#endif
+ rxq->rxq_stats.errors = rxq->rxq_stats.csum_errors +
+ rxq->rxq_stats.other_errors;
+
+ stats->q_ipackets[i] = rxq->rxq_stats.packets;
+ stats->q_ibytes[i] = rxq->rxq_stats.bytes;
+ stats->q_errors[i] = rxq->rxq_stats.errors;
+
+ stats->ierrors += rxq->rxq_stats.errors;
+ rx_discards_pmd += rxq->rxq_stats.dropped;
+ dev->data->rx_mbuf_alloc_failed += rxq->rxq_stats.rx_nombuf;
+ }
+
+ /* Tx queue stats. */
+ q_num = (nic_dev->num_sqs < RTE_ETHDEV_QUEUE_STAT_CNTRS)
+ ? nic_dev->num_sqs
+ : RTE_ETHDEV_QUEUE_STAT_CNTRS;
+ for (i = 0; i < q_num; i++) {
+ txq = nic_dev->txqs[i];
+ stats->q_opackets[i] = txq->txq_stats.packets;
+ stats->q_obytes[i] = txq->txq_stats.bytes;
+ stats->oerrors += (txq->txq_stats.tx_busy +
+ txq->txq_stats.offload_errors);
+ }
+
+ /* Vport stats. */
+ stats->oerrors += vport_stats.tx_discard_vport;
+
+ stats->imissed = vport_stats.rx_discard_vport + rx_discards_pmd;
+
+ stats->ipackets =
+ (vport_stats.rx_unicast_pkts_vport +
+ vport_stats.rx_multicast_pkts_vport +
+ vport_stats.rx_broadcast_pkts_vport - rx_discards_pmd);
+
+ stats->opackets = (vport_stats.tx_unicast_pkts_vport +
+ vport_stats.tx_multicast_pkts_vport +
+ vport_stats.tx_broadcast_pkts_vport);
+
+ stats->ibytes = (vport_stats.rx_unicast_bytes_vport +
+ vport_stats.rx_multicast_bytes_vport +
+ vport_stats.rx_broadcast_bytes_vport);
+
+ stats->obytes = (vport_stats.tx_unicast_bytes_vport +
+ vport_stats.tx_multicast_bytes_vport +
+ vport_stats.tx_broadcast_bytes_vport);
+ return 0;
+}
+
/**
- * Interrupt handler triggered by NIC for handling specific event.
+ * Clear device generic statistics.
*
- * @param[in] param
- * The address of parameter (struct rte_eth_dev *) regsitered before.
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_stats_reset(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_rxq *rxq = NULL;
+ struct hinic3_txq *txq = NULL;
+ int qid;
+ int err;
+
+ err = hinic3_clear_vport_stats(nic_dev->hwdev);
+ if (err)
+ return err;
+
+ for (qid = 0; qid < nic_dev->num_rqs; qid++) {
+ rxq = nic_dev->rxqs[qid];
+ memset(&rxq->rxq_stats, 0, sizeof(struct hinic3_rxq_stats));
+ }
+
+ for (qid = 0; qid < nic_dev->num_sqs; qid++) {
+ txq = nic_dev->txqs[qid];
+ memset(&txq->txq_stats, 0, sizeof(struct hinic3_txq_stats));
+ }
+
+ return 0;
+}
+
+/**
+ * Get device extended statistics.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] xstats
+ * Pointer to rte extended stats table.
+ * @param[in] n
+ * The size of the stats table.
+ *
+ * @return
+ * positive: Number of extended stats on success and stats is filled.
+ * negative: Failure.
+ */
+static int
+hinic3_dev_xstats_get(struct rte_eth_dev *dev, struct rte_eth_xstat *xstats,
+ unsigned int n)
+{
+ struct hinic3_nic_dev *nic_dev;
+ struct mag_phy_port_stats port_stats;
+ struct hinic3_vport_stats vport_stats;
+ struct hinic3_rxq *rxq = NULL;
+ struct hinic3_rxq_stats rxq_stats;
+ struct hinic3_txq *txq = NULL;
+ struct hinic3_txq_stats txq_stats;
+ u16 qid;
+ u32 i;
+ int err, count;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ count = hinic3_xstats_calc_num(nic_dev);
+ if ((int)n < count)
+ return count;
+
+ count = 0;
+
+ /* Get stats from rxq stats structure. */
+ for (qid = 0; qid < nic_dev->num_rqs; qid++) {
+ rxq = nic_dev->rxqs[qid];
+
+#ifdef HINIC3_XSTAT_RXBUF_INFO
+ hinic3_get_stats(rxq);
+#endif
+
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_left_mbuf_bytes =
+ rxq->rxq_stats.rx_alloc_mbuf_bytes -
+ rxq->rxq_stats.rx_free_mbuf_bytes;
+#endif
+ rxq->rxq_stats.errors = rxq->rxq_stats.csum_errors +
+ rxq->rxq_stats.other_errors;
+
+ memcpy((void *)&rxq_stats, (void *)&rxq->rxq_stats,
+ sizeof(rxq->rxq_stats));
+
+ for (i = 0; i < HINIC3_RXQ_XSTATS_NUM; i++) {
+ xstats[count].value = *(uint64_t *)(((char *)&rxq_stats) +
+ hinic3_rxq_stats_strings[i].offset);
+ xstats[count].id = count;
+ count++;
+ }
+ }
+
+ /* Get stats from txq stats structure. */
+ for (qid = 0; qid < nic_dev->num_sqs; qid++) {
+ txq = nic_dev->txqs[qid];
+ memcpy((void *)&txq_stats, (void *)&txq->txq_stats,
+ sizeof(txq->txq_stats));
+
+ for (i = 0; i < HINIC3_TXQ_XSTATS_NUM; i++) {
+ xstats[count].value = *(uint64_t *)(((char *)&txq_stats) +
+ hinic3_txq_stats_strings[i].offset);
+ xstats[count].id = count;
+ count++;
+ }
+ }
+
+ /* Get stats from vport stats structure. */
+ err = hinic3_get_vport_stats(nic_dev->hwdev, &vport_stats);
+ if (err)
+ return err;
+
+ for (i = 0; i < HINIC3_VPORT_XSTATS_NUM; i++) {
+ xstats[count].value =
+ *(uint64_t *)(((char *)&vport_stats) +
+ hinic3_vport_stats_strings[i].offset);
+ xstats[count].id = count;
+ count++;
+ }
+
+ if (HINIC3_IS_VF(nic_dev->hwdev))
+ return count;
+
+ /* Get stats from phy port stats structure. */
+ err = hinic3_get_phy_port_stats(nic_dev->hwdev, &port_stats);
+ if (err)
+ return err;
+
+ for (i = 0; i < HINIC3_PHYPORT_XSTATS_NUM; i++) {
+ xstats[count].value =
+ *(uint64_t *)(((char *)&port_stats) +
+ hinic3_phyport_stats_strings[i].offset);
+ xstats[count].id = count;
+ count++;
+ }
+
+ return count;
+}
+
+/**
+ * Clear device extended statistics.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_xstats_reset(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int err;
+
+ err = hinic3_dev_stats_reset(dev);
+ if (err)
+ return err;
+
+ if (hinic3_func_type(nic_dev->hwdev) != TYPE_VF) {
+ err = hinic3_clear_phy_port_stats(nic_dev->hwdev);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * Retrieve names of extended device statistics.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] xstats_names
+ * Buffer to insert names into.
+ *
+ * @return
+ * Number of xstats names.
+ */
+static int
+hinic3_dev_xstats_get_names(struct rte_eth_dev *dev,
+ struct rte_eth_xstat_name *xstats_names,
+ __rte_unused unsigned int limit)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int count = 0;
+ u16 i, q_num;
+
+ if (xstats_names == NULL)
+ return hinic3_xstats_calc_num(nic_dev);
+
+ /* Get pmd rxq stats name. */
+ for (q_num = 0; q_num < nic_dev->num_rqs; q_num++) {
+ for (i = 0; i < HINIC3_RXQ_XSTATS_NUM; i++) {
+ snprintf(xstats_names[count].name,
+ sizeof(xstats_names[count].name),
+ "rxq%d_%s_pmd", q_num,
+ hinic3_rxq_stats_strings[i].name);
+ count++;
+ }
+ }
+
+ /* Get pmd txq stats name. */
+ for (q_num = 0; q_num < nic_dev->num_sqs; q_num++) {
+ for (i = 0; i < HINIC3_TXQ_XSTATS_NUM; i++) {
+ snprintf(xstats_names[count].name,
+ sizeof(xstats_names[count].name),
+ "txq%d_%s_pmd", q_num,
+ hinic3_txq_stats_strings[i].name);
+ count++;
+ }
+ }
+
+ /* Get vport stats name. */
+ for (i = 0; i < HINIC3_VPORT_XSTATS_NUM; i++) {
+ snprintf(xstats_names[count].name,
+ sizeof(xstats_names[count].name), "%s",
+ hinic3_vport_stats_strings[i].name);
+ count++;
+ }
+
+ if (HINIC3_IS_VF(nic_dev->hwdev))
+ return count;
+
+ /* Get phy port stats name. */
+ for (i = 0; i < HINIC3_PHYPORT_XSTATS_NUM; i++) {
+ snprintf(xstats_names[count].name,
+ sizeof(xstats_names[count].name), "%s",
+ hinic3_phyport_stats_strings[i].name);
+ count++;
+ }
+
+ return count;
+}
+
+/**
+ * Function used to get supported ptypes of an Ethernet device.
+ *
+ * @param[in] dev
+ * ethdev handle of port.
+ * @param[out] no_of_elements
+ * number of ptypes elements. Must be initialized to 0.
+ *
+ * @return
+ * Success, array of ptypes elements and valid no_of_elements > 0.
+ * Failures, NULL.
*/
+static const uint32_t *
+hinic3_dev_supported_ptypes_get(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused size_t *no_of_elements)
+{
+ return 0;
+}
+
static void
-hinic3_dev_interrupt_handler(void *param)
+hinic3_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_rxq_info *rxq_info)
+{
+ struct hinic3_rxq *rxq = dev->data->rx_queues[queue_id];
+
+ rxq_info->mp = rxq->mb_pool;
+ rxq_info->nb_desc = rxq->q_depth;
+}
+
+static void
+hinic3_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_txq_info *txq_qinfo)
+{
+ struct hinic3_txq *txq = dev->data->tx_queues[queue_id];
+
+ txq_qinfo->nb_desc = txq->q_depth;
+}
+
+/**
+ * Update MAC address.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] addr
+ * Pointer to MAC address.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_set_mac_addr(struct rte_eth_dev *dev, struct rte_ether_addr *addr)
{
- struct rte_eth_dev *dev = param;
struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ char mac_addr[RTE_ETHER_ADDR_FMT_SIZE];
+ u16 func_id;
+ int err;
- if (!hinic3_get_bit(HINIC3_DEV_INTR_EN, &nic_dev->dev_status)) {
- PMD_DRV_LOG(WARNING,
- "Intr is disabled, ignore intr event, "
- "dev_name: %s, port_id: %d",
- nic_dev->dev_name, dev->data->port_id);
+ if (!rte_is_valid_assigned_ether_addr(addr)) {
+ rte_ether_format_addr(mac_addr, RTE_ETHER_ADDR_FMT_SIZE, addr);
+ PMD_DRV_LOG(ERR, "Set invalid MAC address %s", mac_addr);
+ return -EINVAL;
+ }
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+ err = hinic3_update_mac(nic_dev->hwdev,
+ nic_dev->default_addr.addr_bytes,
+ addr->addr_bytes, 0, func_id);
+ if (err)
+ return err;
+
+ rte_ether_addr_copy(addr, &nic_dev->default_addr);
+ rte_ether_format_addr(mac_addr, RTE_ETHER_ADDR_FMT_SIZE,
+ &nic_dev->default_addr);
+
+ PMD_DRV_LOG(INFO, "Set new MAC address %s", mac_addr);
+ return 0;
+}
+
+/**
+ * Remove a MAC address.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] index
+ * MAC address index.
+ */
+static void
+hinic3_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u16 func_id;
+ int err;
+
+ if (index >= HINIC3_MAX_UC_MAC_ADDRS) {
+ PMD_DRV_LOG(INFO, "Remove MAC index(%u) is out of range",
+ index);
return;
}
- /* Aeq0 msg handler. */
- hinic3_dev_handle_aeq_event(nic_dev->hwdev, param);
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+ err = hinic3_del_mac(nic_dev->hwdev,
+ dev->data->mac_addrs[index].addr_bytes, 0,
+ func_id);
+ if (err)
+ PMD_DRV_LOG(ERR, "Remove MAC index(%u) failed", index);
+}
+
+/**
+ * Add a MAC address.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] mac_addr
+ * MAC address to register.
+ * @param[in] index
+ * MAC address index.
+ * @param[in] vmdq
+ * VMDq pool index to associate address with (unused_).
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr,
+ uint32_t index, __rte_unused uint32_t vmdq)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ unsigned int i;
+ u16 func_id;
+ int err;
+
+ if (!rte_is_valid_assigned_ether_addr(mac_addr)) {
+ PMD_DRV_LOG(ERR, "Add invalid MAC address");
+ return -EINVAL;
+ }
+
+ if (index >= HINIC3_MAX_UC_MAC_ADDRS) {
+ PMD_DRV_LOG(ERR, "Add MAC index(%u) is out of range", index);
+ return -EINVAL;
+ }
+
+ /* Make sure this address doesn't already be configured. */
+ for (i = 0; i < HINIC3_MAX_UC_MAC_ADDRS; i++) {
+ if (rte_is_same_ether_addr(mac_addr,
+ &dev->data->mac_addrs[i])) {
+ PMD_DRV_LOG(ERR, "MAC address is already configured");
+ return -EADDRINUSE;
+ }
+ }
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+ err = hinic3_set_mac(nic_dev->hwdev, mac_addr->addr_bytes, 0, func_id);
+ if (err)
+ return err;
+
+ return 0;
}
+/**
+ * Delete all multicast MAC addresses from the NIC device.
+ *
+ * This function iterates over the list of multicast MAC addresses and removes
+ * each address from the NIC device by calling `hinic3_del_mac`. After each
+ * deletion, the address is reset to zero.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ */
static void
-hinic3_deinit_sw_rxtxqs(struct hinic3_nic_dev *nic_dev)
+hinic3_delete_mc_addr_list(struct hinic3_nic_dev *nic_dev)
{
- rte_free(nic_dev->txqs);
- nic_dev->txqs = NULL;
+ u16 func_id;
+ u32 i;
- rte_free(nic_dev->rxqs);
- nic_dev->rxqs = NULL;
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+
+ for (i = 0; i < HINIC3_MAX_MC_MAC_ADDRS; i++) {
+ if (rte_is_zero_ether_addr(&nic_dev->mc_list[i]))
+ break;
+
+ hinic3_del_mac(nic_dev->hwdev, nic_dev->mc_list[i].addr_bytes,
+ 0, func_id);
+ memset(&nic_dev->mc_list[i], 0, sizeof(struct rte_ether_addr));
+ }
+}
+
+/**
+ * Set multicast MAC address.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] mc_addr_set
+ * Pointer to multicast MAC address.
+ * @param[in] nb_mc_addr
+ * The number of multicast MAC address to set.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_set_mc_addr_list(struct rte_eth_dev *dev,
+ struct rte_ether_addr *mc_addr_set, uint32_t nb_mc_addr)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ char mac_addr[RTE_ETHER_ADDR_FMT_SIZE];
+ u16 func_id;
+ int err;
+ u32 i;
+
+ func_id = hinic3_global_func_id(nic_dev->hwdev);
+
+ /* Delete old multi_cast addrs firstly. */
+ hinic3_delete_mc_addr_list(nic_dev);
+
+ if (nb_mc_addr > HINIC3_MAX_MC_MAC_ADDRS)
+ return -EINVAL;
+
+ for (i = 0; i < nb_mc_addr; i++) {
+ if (!rte_is_multicast_ether_addr(&mc_addr_set[i])) {
+ rte_ether_format_addr(mac_addr, RTE_ETHER_ADDR_FMT_SIZE,
+ &mc_addr_set[i]);
+ PMD_DRV_LOG(ERR,
+ "Set mc MAC addr failed, addr(%s) invalid",
+ mac_addr);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < nb_mc_addr; i++) {
+ err = hinic3_set_mac(nic_dev->hwdev, mc_addr_set[i].addr_bytes,
+ 0, func_id);
+ if (err) {
+ hinic3_delete_mc_addr_list(nic_dev);
+ return err;
+ }
+
+ rte_ether_addr_copy(&mc_addr_set[i], &nic_dev->mc_list[i]);
+ }
+
+ return 0;
+}
+
+static int
+hinic3_get_reg(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused struct rte_dev_reg_info *regs)
+{
+ return 0;
}
+static const struct eth_dev_ops hinic3_pmd_ops = {
+ .dev_configure = hinic3_dev_configure,
+ .dev_infos_get = hinic3_dev_infos_get,
+ .fw_version_get = hinic3_fw_version_get,
+ .dev_set_link_up = hinic3_dev_set_link_up,
+ .dev_set_link_down = hinic3_dev_set_link_down,
+ .link_update = hinic3_link_update,
+ .rx_queue_setup = hinic3_rx_queue_setup,
+ .tx_queue_setup = hinic3_tx_queue_setup,
+ .rx_queue_release = hinic3_rx_queue_release,
+ .tx_queue_release = hinic3_tx_queue_release,
+ .rx_queue_start = hinic3_dev_rx_queue_start,
+ .rx_queue_stop = hinic3_dev_rx_queue_stop,
+ .tx_queue_start = hinic3_dev_tx_queue_start,
+ .tx_queue_stop = hinic3_dev_tx_queue_stop,
+ .rx_queue_intr_enable = hinic3_dev_rx_queue_intr_enable,
+ .rx_queue_intr_disable = hinic3_dev_rx_queue_intr_disable,
+ .dev_start = hinic3_dev_start,
+ .dev_stop = hinic3_dev_stop,
+ .dev_close = hinic3_dev_close,
+ .dev_reset = hinic3_dev_reset,
+ .mtu_set = hinic3_dev_set_mtu,
+ .vlan_filter_set = hinic3_vlan_filter_set,
+ .vlan_offload_set = hinic3_vlan_offload_set,
+ .allmulticast_enable = hinic3_dev_allmulticast_enable,
+ .allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .stats_get = hinic3_dev_stats_get,
+ .stats_reset = hinic3_dev_stats_reset,
+ .xstats_get = hinic3_dev_xstats_get,
+ .xstats_reset = hinic3_dev_xstats_reset,
+ .xstats_get_names = hinic3_dev_xstats_get_names,
+ .dev_supported_ptypes_get = hinic3_dev_supported_ptypes_get,
+ .rxq_info_get = hinic3_rxq_info_get,
+ .txq_info_get = hinic3_txq_info_get,
+ .mac_addr_set = hinic3_set_mac_addr,
+ .mac_addr_remove = hinic3_mac_addr_remove,
+ .mac_addr_add = hinic3_mac_addr_add,
+ .set_mc_addr_list = hinic3_set_mc_addr_list,
+ .get_reg = hinic3_get_reg,
+};
+
+static const struct eth_dev_ops hinic3_pmd_vf_ops = {
+ .dev_configure = hinic3_dev_configure,
+ .dev_infos_get = hinic3_dev_infos_get,
+ .fw_version_get = hinic3_fw_version_get,
+ .rx_queue_setup = hinic3_rx_queue_setup,
+ .tx_queue_setup = hinic3_tx_queue_setup,
+ .rx_queue_intr_enable = hinic3_dev_rx_queue_intr_enable,
+ .rx_queue_intr_disable = hinic3_dev_rx_queue_intr_disable,
+
+ .rx_queue_start = hinic3_dev_rx_queue_start,
+ .rx_queue_stop = hinic3_dev_rx_queue_stop,
+ .tx_queue_start = hinic3_dev_tx_queue_start,
+ .tx_queue_stop = hinic3_dev_tx_queue_stop,
+
+ .dev_start = hinic3_dev_start,
+ .link_update = hinic3_link_update,
+ .rx_queue_release = hinic3_rx_queue_release,
+ .tx_queue_release = hinic3_tx_queue_release,
+ .dev_stop = hinic3_dev_stop,
+ .dev_close = hinic3_dev_close,
+ .mtu_set = hinic3_dev_set_mtu,
+ .vlan_filter_set = hinic3_vlan_filter_set,
+ .vlan_offload_set = hinic3_vlan_offload_set,
+ .allmulticast_enable = hinic3_dev_allmulticast_enable,
+ .allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .stats_get = hinic3_dev_stats_get,
+ .stats_reset = hinic3_dev_stats_reset,
+ .xstats_get = hinic3_dev_xstats_get,
+ .xstats_reset = hinic3_dev_xstats_reset,
+ .xstats_get_names = hinic3_dev_xstats_get_names,
+ .rxq_info_get = hinic3_rxq_info_get,
+ .txq_info_get = hinic3_txq_info_get,
+ .mac_addr_set = hinic3_set_mac_addr,
+ .mac_addr_remove = hinic3_mac_addr_remove,
+ .mac_addr_add = hinic3_mac_addr_add,
+ .set_mc_addr_list = hinic3_set_mc_addr_list,
+};
+
/**
* Init mac_vlan table in hardwares.
*
@@ -319,6 +3194,15 @@ hinic3_func_init(struct rte_eth_dev *eth_dev)
nic_dev->max_sqs = hinic3_func_max_sqs(nic_dev->hwdev);
nic_dev->max_rqs = hinic3_func_max_rqs(nic_dev->hwdev);
+ if (HINIC3_FUNC_TYPE(nic_dev->hwdev) == TYPE_VF)
+ eth_dev->dev_ops = &hinic3_pmd_vf_ops;
+ else
+ eth_dev->dev_ops = &hinic3_pmd_ops;
+
+ eth_dev->rx_queue_count = hinic3_dev_rx_queue_count;
+ eth_dev->rx_descriptor_status = hinic3_dev_rx_descriptor_status;
+ eth_dev->tx_descriptor_status = hinic3_dev_tx_descriptor_status;
+
err = hinic3_init_nic_hwdev(nic_dev->hwdev);
if (err) {
PMD_DRV_LOG(ERR, "Init nic hwdev failed, dev_name: %s",
diff --git a/drivers/net/hinic3/hinic3_nic_io.c b/drivers/net/hinic3/hinic3_nic_io.c
new file mode 100644
index 0000000000..aba5a641bc
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_nic_io.c
@@ -0,0 +1,827 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_config.h>
+#include <rte_errno.h>
+#include <rte_ether.h>
+#include <rte_io.h>
+#include <rte_malloc.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_pci.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_cmd.h"
+#include "base/hinic3_cmdq.h"
+#include "base/hinic3_hw_comm.h"
+#include "base/hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_nic_io.h"
+#include "hinic3_rx.h"
+#include "hinic3_tx.h"
+
+#define HINIC3_DEAULT_TX_CI_PENDING_LIMIT 3
+#define HINIC3_DEAULT_TX_CI_COALESCING_TIME 16
+#define HINIC3_DEAULT_DROP_THD_ON 0xFFFF
+#define HINIC3_DEAULT_DROP_THD_OFF 0
+
+#define WQ_PREFETCH_MAX 6
+#define WQ_PREFETCH_MIN 1
+#define WQ_PREFETCH_THRESHOLD 256
+
+#define HINIC3_Q_CTXT_MAX \
+ ((u16)(((HINIC3_CMDQ_BUF_SIZE - 8) - RTE_PKTMBUF_HEADROOM) / 64))
+
+enum hinic3_qp_ctxt_type {
+ HINIC3_QP_CTXT_TYPE_SQ,
+ HINIC3_QP_CTXT_TYPE_RQ,
+};
+
+struct hinic3_qp_ctxt_header {
+ u16 num_queues;
+ u16 queue_type;
+ u16 start_qid;
+ u16 rsvd;
+};
+
+struct hinic3_sq_ctxt {
+ u32 ci_pi;
+ u32 drop_mode_sp; /**< Packet drop mode and special flags. */
+ u32 wq_pfn_hi_owner; /**< High PFN and ownership flag. */
+ u32 wq_pfn_lo; /**< Low bits of work queue PFN. */
+
+ u32 rsvd0; /**< Reserved field 0. */
+ u32 pkt_drop_thd; /**< Packet drop threshold. */
+ u32 global_sq_id;
+ u32 vlan_ceq_attr; /**< VLAN and CEQ attributes. */
+
+ u32 pref_cache; /**< Cache prefetch settings for the queue. */
+ u32 pref_ci_owner; /**< Prefetch settings for CI and ownership. */
+ u32 pref_wq_pfn_hi_ci; /**< Prefetch settings for high PFN and CI. */
+ u32 pref_wq_pfn_lo; /**< Prefetch settings for low PFN. */
+
+ u32 rsvd8; /**< Reserved field 8. */
+ u32 rsvd9; /**< Reserved field 9. */
+ u32 wq_block_pfn_hi; /**< High bits of work queue block PFN. */
+ u32 wq_block_pfn_lo; /**< Low bits of work queue block PFN. */
+};
+
+struct hinic3_rq_ctxt {
+ u32 ci_pi;
+ u32 ceq_attr; /**< Completion event queue attributes. */
+ u32 wq_pfn_hi_type_owner; /**< High PFN, WQE type and ownership flag. */
+ u32 wq_pfn_lo; /**< Low bits of work queue PFN. */
+
+ u32 rsvd[3]; /**< Reserved field. */
+ u32 cqe_sge_len; /**< CQE scatter/gather element length. */
+
+ u32 pref_cache; /**< Cache prefetch settings for the queue. */
+ u32 pref_ci_owner; /**< Prefetch settings for CI and ownership. */
+ u32 pref_wq_pfn_hi_ci; /**< Prefetch settings for high PFN and CI. */
+ u32 pref_wq_pfn_lo; /**< Prefetch settings for low PFN. */
+
+ u32 pi_paddr_hi; /**< High 32-bits of PI DMA address. */
+ u32 pi_paddr_lo; /**< Low 32-bits of PI DMA address. */
+ u32 wq_block_pfn_hi; /**< High bits of work queue block PFN. */
+ u32 wq_block_pfn_lo; /**< Low bits of work queue block PFN. */
+};
+
+struct hinic3_sq_ctxt_block {
+ struct hinic3_qp_ctxt_header cmdq_hdr;
+ struct hinic3_sq_ctxt sq_ctxt[HINIC3_Q_CTXT_MAX];
+};
+
+struct hinic3_rq_ctxt_block {
+ struct hinic3_qp_ctxt_header cmdq_hdr;
+ struct hinic3_rq_ctxt rq_ctxt[HINIC3_Q_CTXT_MAX];
+};
+
+struct hinic3_clean_queue_ctxt {
+ struct hinic3_qp_ctxt_header cmdq_hdr;
+ u32 rsvd;
+};
+
+#define SQ_CTXT_SIZE(num_sqs) \
+ ((u16)(sizeof(struct hinic3_qp_ctxt_header) + \
+ (num_sqs) * sizeof(struct hinic3_sq_ctxt)))
+
+#define RQ_CTXT_SIZE(num_rqs) \
+ ((u16)(sizeof(struct hinic3_qp_ctxt_header) + \
+ (num_rqs) * sizeof(struct hinic3_rq_ctxt)))
+
+#define CI_IDX_HIGH_SHIFH 12
+
+#define CI_HIGN_IDX(val) ((val) >> CI_IDX_HIGH_SHIFH)
+
+#define SQ_CTXT_PI_IDX_SHIFT 0
+#define SQ_CTXT_CI_IDX_SHIFT 16
+
+#define SQ_CTXT_PI_IDX_MASK 0xFFFFU
+#define SQ_CTXT_CI_IDX_MASK 0xFFFFU
+
+#define SQ_CTXT_CI_PI_SET(val, member) \
+ (((val) & SQ_CTXT_##member##_MASK) << SQ_CTXT_##member##_SHIFT)
+
+#define SQ_CTXT_MODE_SP_FLAG_SHIFT 0
+#define SQ_CTXT_MODE_PKT_DROP_SHIFT 1
+
+#define SQ_CTXT_MODE_SP_FLAG_MASK 0x1U
+#define SQ_CTXT_MODE_PKT_DROP_MASK 0x1U
+
+#define SQ_CTXT_MODE_SET(val, member) \
+ (((val) & SQ_CTXT_MODE_##member##_MASK) \
+ << SQ_CTXT_MODE_##member##_SHIFT)
+
+#define SQ_CTXT_WQ_PAGE_HI_PFN_SHIFT 0
+#define SQ_CTXT_WQ_PAGE_OWNER_SHIFT 23
+
+#define SQ_CTXT_WQ_PAGE_HI_PFN_MASK 0xFFFFFU
+#define SQ_CTXT_WQ_PAGE_OWNER_MASK 0x1U
+
+#define SQ_CTXT_WQ_PAGE_SET(val, member) \
+ (((val) & SQ_CTXT_WQ_PAGE_##member##_MASK) \
+ << SQ_CTXT_WQ_PAGE_##member##_SHIFT)
+
+#define SQ_CTXT_PKT_DROP_THD_ON_SHIFT 0
+#define SQ_CTXT_PKT_DROP_THD_OFF_SHIFT 16
+
+#define SQ_CTXT_PKT_DROP_THD_ON_MASK 0xFFFFU
+#define SQ_CTXT_PKT_DROP_THD_OFF_MASK 0xFFFFU
+
+#define SQ_CTXT_PKT_DROP_THD_SET(val, member) \
+ (((val) & SQ_CTXT_PKT_DROP_##member##_MASK) \
+ << SQ_CTXT_PKT_DROP_##member##_SHIFT)
+
+#define SQ_CTXT_GLOBAL_SQ_ID_SHIFT 0
+
+#define SQ_CTXT_GLOBAL_SQ_ID_MASK 0x1FFFU
+
+#define SQ_CTXT_GLOBAL_QUEUE_ID_SET(val, member) \
+ (((val) & SQ_CTXT_##member##_MASK) << SQ_CTXT_##member##_SHIFT)
+
+#define SQ_CTXT_VLAN_TAG_SHIFT 0
+#define SQ_CTXT_VLAN_TYPE_SEL_SHIFT 16
+#define SQ_CTXT_VLAN_INSERT_MODE_SHIFT 19
+#define SQ_CTXT_VLAN_CEQ_EN_SHIFT 23
+
+#define SQ_CTXT_VLAN_TAG_MASK 0xFFFFU
+#define SQ_CTXT_VLAN_TYPE_SEL_MASK 0x7U
+#define SQ_CTXT_VLAN_INSERT_MODE_MASK 0x3U
+#define SQ_CTXT_VLAN_CEQ_EN_MASK 0x1U
+
+#define SQ_CTXT_VLAN_CEQ_SET(val, member) \
+ (((val) & SQ_CTXT_VLAN_##member##_MASK) \
+ << SQ_CTXT_VLAN_##member##_SHIFT)
+
+#define SQ_CTXT_PREF_CACHE_THRESHOLD_SHIFT 0
+#define SQ_CTXT_PREF_CACHE_MAX_SHIFT 14
+#define SQ_CTXT_PREF_CACHE_MIN_SHIFT 25
+
+#define SQ_CTXT_PREF_CACHE_THRESHOLD_MASK 0x3FFFU
+#define SQ_CTXT_PREF_CACHE_MAX_MASK 0x7FFU
+#define SQ_CTXT_PREF_CACHE_MIN_MASK 0x7FU
+
+#define SQ_CTXT_PREF_CI_HI_SHIFT 0
+#define SQ_CTXT_PREF_OWNER_SHIFT 4
+
+#define SQ_CTXT_PREF_CI_HI_MASK 0xFU
+#define SQ_CTXT_PREF_OWNER_MASK 0x1U
+
+#define SQ_CTXT_PREF_WQ_PFN_HI_SHIFT 0
+#define SQ_CTXT_PREF_CI_LOW_SHIFT 20
+
+#define SQ_CTXT_PREF_WQ_PFN_HI_MASK 0xFFFFFU
+#define SQ_CTXT_PREF_CI_LOW_MASK 0xFFFU
+
+#define SQ_CTXT_PREF_SET(val, member) \
+ (((val) & SQ_CTXT_PREF_##member##_MASK) \
+ << SQ_CTXT_PREF_##member##_SHIFT)
+
+#define SQ_CTXT_WQ_BLOCK_PFN_HI_SHIFT 0
+
+#define SQ_CTXT_WQ_BLOCK_PFN_HI_MASK 0x7FFFFFU
+
+#define SQ_CTXT_WQ_BLOCK_SET(val, member) \
+ (((val) & SQ_CTXT_WQ_BLOCK_##member##_MASK) \
+ << SQ_CTXT_WQ_BLOCK_##member##_SHIFT)
+
+#define RQ_CTXT_PI_IDX_SHIFT 0
+#define RQ_CTXT_CI_IDX_SHIFT 16
+
+#define RQ_CTXT_PI_IDX_MASK 0xFFFFU
+#define RQ_CTXT_CI_IDX_MASK 0xFFFFU
+
+#define RQ_CTXT_CI_PI_SET(val, member) \
+ (((val) & RQ_CTXT_##member##_MASK) << RQ_CTXT_##member##_SHIFT)
+
+#define RQ_CTXT_CEQ_ATTR_INTR_SHIFT 21
+#define RQ_CTXT_CEQ_ATTR_INTR_ARM_SHIFT 30
+#define RQ_CTXT_CEQ_ATTR_EN_SHIFT 31
+
+#define RQ_CTXT_CEQ_ATTR_INTR_MASK 0x3FFU
+#define RQ_CTXT_CEQ_ATTR_INTR_ARM_MASK 0x1U
+#define RQ_CTXT_CEQ_ATTR_EN_MASK 0x1U
+
+#define RQ_CTXT_CEQ_ATTR_SET(val, member) \
+ (((val) & RQ_CTXT_CEQ_ATTR_##member##_MASK) \
+ << RQ_CTXT_CEQ_ATTR_##member##_SHIFT)
+
+#define RQ_CTXT_WQ_PAGE_HI_PFN_SHIFT 0
+#define RQ_CTXT_WQ_PAGE_WQE_TYPE_SHIFT 28
+#define RQ_CTXT_WQ_PAGE_OWNER_SHIFT 31
+
+#define RQ_CTXT_WQ_PAGE_HI_PFN_MASK 0xFFFFFU
+#define RQ_CTXT_WQ_PAGE_WQE_TYPE_MASK 0x3U
+#define RQ_CTXT_WQ_PAGE_OWNER_MASK 0x1U
+
+#define RQ_CTXT_WQ_PAGE_SET(val, member) \
+ (((val) & RQ_CTXT_WQ_PAGE_##member##_MASK) \
+ << RQ_CTXT_WQ_PAGE_##member##_SHIFT)
+
+#define RQ_CTXT_CQE_LEN_SHIFT 28
+
+#define RQ_CTXT_CQE_LEN_MASK 0x3U
+
+#define RQ_CTXT_CQE_LEN_SET(val, member) \
+ (((val) & RQ_CTXT_##member##_MASK) << RQ_CTXT_##member##_SHIFT)
+
+#define RQ_CTXT_PREF_CACHE_THRESHOLD_SHIFT 0
+#define RQ_CTXT_PREF_CACHE_MAX_SHIFT 14
+#define RQ_CTXT_PREF_CACHE_MIN_SHIFT 25
+
+#define RQ_CTXT_PREF_CACHE_THRESHOLD_MASK 0x3FFFU
+#define RQ_CTXT_PREF_CACHE_MAX_MASK 0x7FFU
+#define RQ_CTXT_PREF_CACHE_MIN_MASK 0x7FU
+
+#define RQ_CTXT_PREF_CI_HI_SHIFT 0
+#define RQ_CTXT_PREF_OWNER_SHIFT 4
+
+#define RQ_CTXT_PREF_CI_HI_MASK 0xFU
+#define RQ_CTXT_PREF_OWNER_MASK 0x1U
+
+#define RQ_CTXT_PREF_WQ_PFN_HI_SHIFT 0
+#define RQ_CTXT_PREF_CI_LOW_SHIFT 20
+
+#define RQ_CTXT_PREF_WQ_PFN_HI_MASK 0xFFFFFU
+#define RQ_CTXT_PREF_CI_LOW_MASK 0xFFFU
+
+#define RQ_CTXT_PREF_SET(val, member) \
+ (((val) & RQ_CTXT_PREF_##member##_MASK) \
+ << RQ_CTXT_PREF_##member##_SHIFT)
+
+#define RQ_CTXT_WQ_BLOCK_PFN_HI_SHIFT 0
+
+#define RQ_CTXT_WQ_BLOCK_PFN_HI_MASK 0x7FFFFFU
+
+#define RQ_CTXT_WQ_BLOCK_SET(val, member) \
+ (((val) & RQ_CTXT_WQ_BLOCK_##member##_MASK) \
+ << RQ_CTXT_WQ_BLOCK_##member##_SHIFT)
+
+#define SIZE_16BYTES(size) (RTE_ALIGN((size), 16) >> 4)
+
+#define WQ_PAGE_PFN_SHIFT 12
+#define WQ_BLOCK_PFN_SHIFT 9
+
+#define WQ_PAGE_PFN(page_addr) ((page_addr) >> WQ_PAGE_PFN_SHIFT)
+#define WQ_BLOCK_PFN(page_addr) ((page_addr) >> WQ_BLOCK_PFN_SHIFT)
+
+/**
+ * Prepare the command queue header and converted it to big-endian format.
+ *
+ * @param[out] qp_ctxt_hdr
+ * Pointer to command queue context header structure to be initialized.
+ * @param[in] ctxt_type
+ * Type of context (SQ/RQ) to be set in header.
+ * @param[in] num_queues
+ * Number of queues.
+ * @param[in] q_id
+ * Starting queue ID for this context.
+ */
+static void
+hinic3_qp_prepare_cmdq_header(struct hinic3_qp_ctxt_header *qp_ctxt_hdr,
+ enum hinic3_qp_ctxt_type ctxt_type,
+ u16 num_queues, u16 q_id)
+{
+ qp_ctxt_hdr->queue_type = ctxt_type;
+ qp_ctxt_hdr->num_queues = num_queues;
+ qp_ctxt_hdr->start_qid = q_id;
+ qp_ctxt_hdr->rsvd = 0;
+
+ rte_mb();
+
+ hinic3_cpu_to_be32(qp_ctxt_hdr, sizeof(*qp_ctxt_hdr));
+}
+
+/**
+ * Initialize context structure for specified TXQ by configuring various queue
+ * parameters (e.g., ci, pi, work queue page addresses).
+ *
+ * @param[in] sq
+ * Pointer to TXQ structure.
+ * @param[in] sq_id
+ * ID of TXQ being configured.
+ * @param[out] sq_ctxt
+ * Pointer to structure that will hold TXQ context.
+ */
+static void
+hinic3_sq_prepare_ctxt(struct hinic3_txq *sq, u16 sq_id,
+ struct hinic3_sq_ctxt *sq_ctxt)
+{
+ u64 wq_page_addr, wq_page_pfn, wq_block_pfn;
+ u32 wq_page_pfn_hi, wq_page_pfn_lo, wq_block_pfn_hi, wq_block_pfn_lo;
+ u16 pi_start, ci_start;
+
+ ci_start = sq->cons_idx & sq->q_mask;
+ pi_start = sq->prod_idx & sq->q_mask;
+
+ /* Read the first page from hardware table. */
+ wq_page_addr = sq->queue_buf_paddr;
+
+ wq_page_pfn = WQ_PAGE_PFN(wq_page_addr);
+ wq_page_pfn_hi = upper_32_bits(wq_page_pfn);
+ wq_page_pfn_lo = lower_32_bits(wq_page_pfn);
+
+ /* Use 0-level CLA. */
+ wq_block_pfn = WQ_BLOCK_PFN(wq_page_addr);
+ wq_block_pfn_hi = upper_32_bits(wq_block_pfn);
+ wq_block_pfn_lo = lower_32_bits(wq_block_pfn);
+
+ sq_ctxt->ci_pi = SQ_CTXT_CI_PI_SET(ci_start, CI_IDX) |
+ SQ_CTXT_CI_PI_SET(pi_start, PI_IDX);
+
+ sq_ctxt->drop_mode_sp = SQ_CTXT_MODE_SET(0, SP_FLAG) |
+ SQ_CTXT_MODE_SET(0, PKT_DROP);
+
+ sq_ctxt->wq_pfn_hi_owner = SQ_CTXT_WQ_PAGE_SET(wq_page_pfn_hi, HI_PFN) |
+ SQ_CTXT_WQ_PAGE_SET(1, OWNER);
+
+ sq_ctxt->wq_pfn_lo = wq_page_pfn_lo;
+
+ sq_ctxt->pkt_drop_thd =
+ SQ_CTXT_PKT_DROP_THD_SET(HINIC3_DEAULT_DROP_THD_ON, THD_ON) |
+ SQ_CTXT_PKT_DROP_THD_SET(HINIC3_DEAULT_DROP_THD_OFF, THD_OFF);
+
+ sq_ctxt->global_sq_id =
+ SQ_CTXT_GLOBAL_QUEUE_ID_SET(sq_id, GLOBAL_SQ_ID);
+
+ /* Insert c-vlan in default. */
+ sq_ctxt->vlan_ceq_attr = SQ_CTXT_VLAN_CEQ_SET(0, CEQ_EN) |
+ SQ_CTXT_VLAN_CEQ_SET(1, INSERT_MODE);
+
+ sq_ctxt->rsvd0 = 0;
+
+ sq_ctxt->pref_cache =
+ SQ_CTXT_PREF_SET(WQ_PREFETCH_MIN, CACHE_MIN) |
+ SQ_CTXT_PREF_SET(WQ_PREFETCH_MAX, CACHE_MAX) |
+ SQ_CTXT_PREF_SET(WQ_PREFETCH_THRESHOLD, CACHE_THRESHOLD);
+
+ sq_ctxt->pref_ci_owner =
+ SQ_CTXT_PREF_SET(CI_HIGN_IDX(ci_start), CI_HI) |
+ SQ_CTXT_PREF_SET(1, OWNER);
+
+ sq_ctxt->pref_wq_pfn_hi_ci =
+ SQ_CTXT_PREF_SET(ci_start, CI_LOW) |
+ SQ_CTXT_PREF_SET(wq_page_pfn_hi, WQ_PFN_HI);
+
+ sq_ctxt->pref_wq_pfn_lo = wq_page_pfn_lo;
+
+ sq_ctxt->wq_block_pfn_hi =
+ SQ_CTXT_WQ_BLOCK_SET(wq_block_pfn_hi, PFN_HI);
+
+ sq_ctxt->wq_block_pfn_lo = wq_block_pfn_lo;
+
+ rte_mb();
+
+ hinic3_cpu_to_be32(sq_ctxt, sizeof(*sq_ctxt));
+}
+
+/**
+ * Initialize context structure for specified RXQ by configuring various queue
+ * parameters (e.g., ci, pi, work queue page addresses).
+ *
+ * @param[in] rq
+ * Pointer to RXQ structure.
+ * @param[out] rq_ctxt
+ * Pointer to structure that will hold RXQ context.
+ */
+static void
+hinic3_rq_prepare_ctxt(struct hinic3_rxq *rq, struct hinic3_rq_ctxt *rq_ctxt)
+{
+ u64 wq_page_addr, wq_page_pfn, wq_block_pfn;
+ u32 wq_page_pfn_hi, wq_page_pfn_lo, wq_block_pfn_hi, wq_block_pfn_lo;
+ u16 pi_start, ci_start;
+ u16 wqe_type = rq->wqebb_shift - HINIC3_RQ_WQEBB_SHIFT;
+ u8 intr_disable;
+
+ /* RQ depth is in unit of 8 Bytes. */
+ ci_start = (u16)((rq->cons_idx & rq->q_mask) << wqe_type);
+ pi_start = (u16)((rq->prod_idx & rq->q_mask) << wqe_type);
+
+ /* Read the first page from hardware table. */
+ wq_page_addr = rq->queue_buf_paddr;
+
+ wq_page_pfn = WQ_PAGE_PFN(wq_page_addr);
+ wq_page_pfn_hi = upper_32_bits(wq_page_pfn);
+ wq_page_pfn_lo = lower_32_bits(wq_page_pfn);
+
+ /* Use 0-level CLA. */
+ wq_block_pfn = WQ_BLOCK_PFN(wq_page_addr);
+ wq_block_pfn_hi = upper_32_bits(wq_block_pfn);
+ wq_block_pfn_lo = lower_32_bits(wq_block_pfn);
+
+ rq_ctxt->ci_pi = RQ_CTXT_CI_PI_SET(ci_start, CI_IDX) |
+ RQ_CTXT_CI_PI_SET(pi_start, PI_IDX);
+
+ /* RQ doesn't need ceq, msix_entry_idx set 1, but mask not enable. */
+ intr_disable = rq->dp_intr_en ? 0 : 1;
+ rq_ctxt->ceq_attr = RQ_CTXT_CEQ_ATTR_SET(intr_disable, EN) |
+ RQ_CTXT_CEQ_ATTR_SET(0, INTR_ARM) |
+ RQ_CTXT_CEQ_ATTR_SET(rq->msix_entry_idx, INTR);
+
+ /* Use 32Byte WQE with SGE for CQE in default. */
+ rq_ctxt->wq_pfn_hi_type_owner =
+ RQ_CTXT_WQ_PAGE_SET(wq_page_pfn_hi, HI_PFN) |
+ RQ_CTXT_WQ_PAGE_SET(1, OWNER);
+
+ switch (wqe_type) {
+ case HINIC3_EXTEND_RQ_WQE:
+ /* Use 32Byte WQE with SGE for CQE. */
+ rq_ctxt->wq_pfn_hi_type_owner |=
+ RQ_CTXT_WQ_PAGE_SET(0, WQE_TYPE);
+ break;
+ case HINIC3_NORMAL_RQ_WQE:
+ /* Use 16Byte WQE with 32Bytes SGE for CQE. */
+ rq_ctxt->wq_pfn_hi_type_owner |=
+ RQ_CTXT_WQ_PAGE_SET(2, WQE_TYPE);
+ rq_ctxt->cqe_sge_len = RQ_CTXT_CQE_LEN_SET(1, CQE_LEN);
+ break;
+ default:
+ PMD_DRV_LOG(INFO, "Invalid rq wqe type: %u", wqe_type);
+ }
+
+ rq_ctxt->wq_pfn_lo = wq_page_pfn_lo;
+
+ rq_ctxt->pref_cache =
+ RQ_CTXT_PREF_SET(WQ_PREFETCH_MIN, CACHE_MIN) |
+ RQ_CTXT_PREF_SET(WQ_PREFETCH_MAX, CACHE_MAX) |
+ RQ_CTXT_PREF_SET(WQ_PREFETCH_THRESHOLD, CACHE_THRESHOLD);
+
+ rq_ctxt->pref_ci_owner =
+ RQ_CTXT_PREF_SET(CI_HIGN_IDX(ci_start), CI_HI) |
+ RQ_CTXT_PREF_SET(1, OWNER);
+
+ rq_ctxt->pref_wq_pfn_hi_ci =
+ RQ_CTXT_PREF_SET(wq_page_pfn_hi, WQ_PFN_HI) |
+ RQ_CTXT_PREF_SET(ci_start, CI_LOW);
+
+ rq_ctxt->pref_wq_pfn_lo = wq_page_pfn_lo;
+
+ rq_ctxt->pi_paddr_hi = upper_32_bits(rq->pi_dma_addr);
+ rq_ctxt->pi_paddr_lo = lower_32_bits(rq->pi_dma_addr);
+
+ rq_ctxt->wq_block_pfn_hi =
+ RQ_CTXT_WQ_BLOCK_SET(wq_block_pfn_hi, PFN_HI);
+
+ rq_ctxt->wq_block_pfn_lo = wq_block_pfn_lo;
+ rte_mb();
+
+ hinic3_cpu_to_be32(rq_ctxt, sizeof(*rq_ctxt));
+}
+
+/**
+ * Allocate a command buffer, prepare context for each SQ queue by setting
+ * various parameters, send context data to hardware. It processes SQ queues in
+ * batches, with each batch not exceeding `HINIC3_Q_CTXT_MAX` SQ contexts.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ *
+ * @return
+ * 0 on success, a negative error code on failure.
+ * - -ENOMEM if the memory allocation for the command buffer fails.
+ * - -EFAULT if the hardware returns an error while processing the context data.
+ */
+static int
+init_sq_ctxts(struct hinic3_nic_dev *nic_dev)
+{
+ struct hinic3_sq_ctxt_block *sq_ctxt_block = NULL;
+ struct hinic3_sq_ctxt *sq_ctxt = NULL;
+ struct hinic3_cmd_buf *cmd_buf = NULL;
+ struct hinic3_txq *sq = NULL;
+ u64 out_param = 0;
+ u16 q_id, curr_id, max_ctxts, i;
+ int err = 0;
+
+ cmd_buf = hinic3_alloc_cmd_buf(nic_dev->hwdev);
+ if (!cmd_buf) {
+ PMD_DRV_LOG(ERR, "Allocate cmd buf for sq ctx failed");
+ return -ENOMEM;
+ }
+
+ q_id = 0;
+ while (q_id < nic_dev->num_sqs) {
+ sq_ctxt_block = cmd_buf->buf;
+ sq_ctxt = sq_ctxt_block->sq_ctxt;
+
+ max_ctxts = (nic_dev->num_sqs - q_id) > HINIC3_Q_CTXT_MAX
+ ? HINIC3_Q_CTXT_MAX
+ : (nic_dev->num_sqs - q_id);
+
+ hinic3_qp_prepare_cmdq_header(&sq_ctxt_block->cmdq_hdr,
+ HINIC3_QP_CTXT_TYPE_SQ, max_ctxts,
+ q_id);
+
+ for (i = 0; i < max_ctxts; i++) {
+ curr_id = q_id + i;
+ sq = nic_dev->txqs[curr_id];
+ hinic3_sq_prepare_ctxt(sq, curr_id, &sq_ctxt[i]);
+ }
+
+ cmd_buf->size = SQ_CTXT_SIZE(max_ctxts);
+ rte_mb();
+ err = hinic3_cmdq_direct_resp(nic_dev->hwdev, HINIC3_MOD_L2NIC,
+ HINIC3_UCODE_CMD_MODIFY_QUEUE_CTX,
+ cmd_buf, &out_param, 0);
+ if (err || out_param != 0) {
+ PMD_DRV_LOG(ERR,
+ "Set SQ ctxts failed, "
+ "err: %d, out_param: %" PRIu64,
+ err, out_param);
+
+ err = -EFAULT;
+ break;
+ }
+
+ q_id += max_ctxts;
+ }
+
+ hinic3_free_cmd_buf(cmd_buf);
+ return err;
+}
+
+/**
+ * Initialize context for all RQ in device.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ *
+ * @return
+ * 0 on success, a negative error code on failure.
+ * - -ENOMEM if the memory allocation for the command buffer fails.
+ * - -EFAULT if the hardware returns an error while processing the context data.
+ */
+static int
+init_rq_ctxts(struct hinic3_nic_dev *nic_dev)
+{
+ struct hinic3_rq_ctxt_block *rq_ctxt_block = NULL;
+ struct hinic3_rq_ctxt *rq_ctxt = NULL;
+ struct hinic3_cmd_buf *cmd_buf = NULL;
+ struct hinic3_rxq *rq = NULL;
+ u64 out_param = 0;
+ u16 q_id, curr_id, max_ctxts, i;
+ int err = 0;
+
+ cmd_buf = hinic3_alloc_cmd_buf(nic_dev->hwdev);
+ if (!cmd_buf) {
+ PMD_DRV_LOG(ERR, "Allocate cmd buf for rq ctx failed");
+ return -ENOMEM;
+ }
+
+ q_id = 0;
+ while (q_id < nic_dev->num_rqs) {
+ rq_ctxt_block = cmd_buf->buf;
+ rq_ctxt = rq_ctxt_block->rq_ctxt;
+
+ max_ctxts = (nic_dev->num_rqs - q_id) > HINIC3_Q_CTXT_MAX
+ ? HINIC3_Q_CTXT_MAX
+ : (nic_dev->num_rqs - q_id);
+
+ hinic3_qp_prepare_cmdq_header(&rq_ctxt_block->cmdq_hdr,
+ HINIC3_QP_CTXT_TYPE_RQ, max_ctxts,
+ q_id);
+
+ for (i = 0; i < max_ctxts; i++) {
+ curr_id = q_id + i;
+ rq = nic_dev->rxqs[curr_id];
+ hinic3_rq_prepare_ctxt(rq, &rq_ctxt[i]);
+ }
+
+ cmd_buf->size = RQ_CTXT_SIZE(max_ctxts);
+ rte_mb();
+ err = hinic3_cmdq_direct_resp(nic_dev->hwdev, HINIC3_MOD_L2NIC,
+ HINIC3_UCODE_CMD_MODIFY_QUEUE_CTX,
+ cmd_buf, &out_param, 0);
+ if (err || out_param != 0) {
+ PMD_DRV_LOG(ERR,
+ "Set RQ ctxts failed, "
+ "err: %d, out_param: %" PRIu64,
+ err, out_param);
+ err = -EFAULT;
+ break;
+ }
+
+ q_id += max_ctxts;
+ }
+
+ hinic3_free_cmd_buf(cmd_buf);
+ return err;
+}
+
+/**
+ * Allocate memory for command buffer, construct related command request, send a
+ * command to hardware to clean up queue offload context.
+ *
+ * @param[in] nic_dev
+ * Pointer to NIC device structure.
+ * @param[in] ctxt_type
+ * The type of queue context to clean.
+ * The queue context type that determines which queue type to clean up.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+clean_queue_offload_ctxt(struct hinic3_nic_dev *nic_dev,
+ enum hinic3_qp_ctxt_type ctxt_type)
+{
+ struct hinic3_clean_queue_ctxt *ctxt_block = NULL;
+ struct hinic3_cmd_buf *cmd_buf;
+ u64 out_param = 0;
+ int err;
+
+ cmd_buf = hinic3_alloc_cmd_buf(nic_dev->hwdev);
+ if (!cmd_buf) {
+ PMD_DRV_LOG(ERR, "Allocate cmd buf for LRO/TSO space failed");
+ return -ENOMEM;
+ }
+
+ /* Construct related command request. */
+ ctxt_block = cmd_buf->buf;
+ /* Assumed max_rqs must be equal to max_sqs. */
+ ctxt_block->cmdq_hdr.num_queues = nic_dev->max_sqs;
+ ctxt_block->cmdq_hdr.queue_type = ctxt_type;
+ ctxt_block->cmdq_hdr.start_qid = 0;
+ /*
+ * Add a memory barrier to ensure that instructions are not out of order
+ * due to compilation optimization.
+ */
+ rte_mb();
+
+ hinic3_cpu_to_be32(ctxt_block, sizeof(*ctxt_block));
+
+ cmd_buf->size = sizeof(*ctxt_block);
+
+ /* Send a command to hardware to clean up queue offload context. */
+ err = hinic3_cmdq_direct_resp(nic_dev->hwdev, HINIC3_MOD_L2NIC,
+ HINIC3_UCODE_CMD_CLEAN_QUEUE_CONTEXT,
+ cmd_buf, &out_param, 0);
+ if ((err) || (out_param)) {
+ PMD_DRV_LOG(ERR,
+ "Clean queue offload ctxts failed, "
+ "err: %d, out_param: %" PRIu64,
+ err, out_param);
+ err = -EFAULT;
+ }
+
+ hinic3_free_cmd_buf(cmd_buf);
+ return err;
+}
+
+static int
+clean_qp_offload_ctxt(struct hinic3_nic_dev *nic_dev)
+{
+ /* Clean LRO/TSO context space. */
+ return (clean_queue_offload_ctxt(nic_dev, HINIC3_QP_CTXT_TYPE_SQ) ||
+ clean_queue_offload_ctxt(nic_dev, HINIC3_QP_CTXT_TYPE_RQ));
+}
+
+void
+hinic3_get_func_rx_buf_size(void *dev)
+{
+ struct hinic3_nic_dev *nic_dev = (struct hinic3_nic_dev *)dev;
+ struct hinic3_rxq *rxq = NULL;
+ u16 q_id;
+ u16 buf_size = 0;
+
+ for (q_id = 0; q_id < nic_dev->num_rqs; q_id++) {
+ rxq = nic_dev->rxqs[q_id];
+
+ if (rxq == NULL)
+ continue;
+
+ if (q_id == 0)
+ buf_size = rxq->buf_len;
+
+ buf_size = buf_size > rxq->buf_len ? rxq->buf_len : buf_size;
+ }
+
+ nic_dev->rx_buff_len = buf_size;
+}
+
+int
+hinic3_init_qp_ctxts(void *dev)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+ struct hinic3_hwdev *hwdev = NULL;
+ struct hinic3_sq_attr sq_attr;
+ u32 rq_depth = 0;
+ u32 sq_depth = 0;
+ u16 q_id;
+ int err;
+
+ if (!dev)
+ return -EINVAL;
+
+ nic_dev = (struct hinic3_nic_dev *)dev;
+ hwdev = nic_dev->hwdev;
+
+ err = init_sq_ctxts(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init SQ ctxts failed");
+ return err;
+ }
+
+ err = init_rq_ctxts(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init RQ ctxts failed");
+ return err;
+ }
+
+ err = clean_qp_offload_ctxt(nic_dev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Clean qp offload ctxts failed");
+ return err;
+ }
+
+ if (nic_dev->num_rqs != 0)
+ rq_depth = ((u32)nic_dev->rxqs[0]->q_depth)
+ << nic_dev->rxqs[0]->wqe_type;
+
+ if (nic_dev->num_sqs != 0)
+ sq_depth = nic_dev->txqs[0]->q_depth;
+
+ err = hinic3_set_root_ctxt(hwdev, rq_depth, sq_depth,
+ nic_dev->rx_buff_len);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set root context failed");
+ return err;
+ }
+
+ /* Configure CI tables for each SQ. */
+ for (q_id = 0; q_id < nic_dev->num_sqs; q_id++) {
+ sq_attr.ci_dma_base = nic_dev->txqs[q_id]->ci_dma_base >> 0x2;
+ sq_attr.pending_limit = HINIC3_DEAULT_TX_CI_PENDING_LIMIT;
+ sq_attr.coalescing_time = HINIC3_DEAULT_TX_CI_COALESCING_TIME;
+ sq_attr.intr_en = 0;
+ sq_attr.intr_idx = 0; /**< Tx doesn't need interrupt. */
+ sq_attr.l2nic_sqn = q_id;
+ sq_attr.dma_attr_off = 0;
+ err = hinic3_set_ci_table(hwdev, &sq_attr);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set ci table failed");
+ goto set_cons_idx_table_err;
+ }
+ }
+
+ return 0;
+
+set_cons_idx_table_err:
+ hinic3_clean_root_ctxt(hwdev);
+ return err;
+}
+
+void
+hinic3_free_qp_ctxts(void *hwdev)
+{
+ if (!hwdev)
+ return;
+
+ hinic3_clean_root_ctxt(hwdev);
+}
+
+void
+hinic3_update_driver_feature(void *dev, u64 s_feature)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+
+ if (!dev)
+ return;
+
+ nic_dev = (struct hinic3_nic_dev *)dev;
+ nic_dev->feature_cap = s_feature;
+
+ PMD_DRV_LOG(INFO, "Update nic feature to 0x%" PRIx64,
+ nic_dev->feature_cap);
+}
+
+u64
+hinic3_get_driver_feature(void *dev)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+
+ nic_dev = (struct hinic3_nic_dev *)dev;
+
+ return nic_dev->feature_cap;
+}
diff --git a/drivers/net/hinic3/hinic3_nic_io.h b/drivers/net/hinic3/hinic3_nic_io.h
new file mode 100644
index 0000000000..39ffb3c8fd
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_nic_io.h
@@ -0,0 +1,169 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_NIC_IO_H_
+#define _HINIC3_NIC_IO_H_
+
+#define HINIC3_SQ_WQEBB_SHIFT 4
+#define HINIC3_RQ_WQEBB_SHIFT 3
+
+#define HINIC3_SQ_WQEBB_SIZE BIT(HINIC3_SQ_WQEBB_SHIFT)
+#define HINIC3_CQE_SIZE_SHIFT 4
+
+/* Ci addr should RTE_CACHE_SIZE(64B) alignment for performance. */
+#define HINIC3_CI_Q_ADDR_SIZE 64
+
+#define CI_TABLE_SIZE(num_qps, pg_sz) \
+ (RTE_ALIGN((num_qps) * HINIC3_CI_Q_ADDR_SIZE, pg_sz))
+
+#define HINIC3_CI_VADDR(base_addr, q_id) \
+ ((u8 *)(base_addr) + (q_id) * HINIC3_CI_Q_ADDR_SIZE)
+
+#define HINIC3_CI_PADDR(base_paddr, q_id) \
+ ((base_paddr) + (q_id) * HINIC3_CI_Q_ADDR_SIZE)
+
+enum hinic3_rq_wqe_type {
+ HINIC3_COMPACT_RQ_WQE,
+ HINIC3_NORMAL_RQ_WQE,
+ HINIC3_EXTEND_RQ_WQE
+};
+
+enum hinic3_queue_type {
+ HINIC3_SQ,
+ HINIC3_RQ,
+ HINIC3_MAX_QUEUE_TYPE,
+};
+
+/* Doorbell info. */
+struct hinic3_db {
+ u32 db_info;
+ u32 pi_hi;
+};
+
+#define DB_INFO_QID_SHIFT 0
+#define DB_INFO_NON_FILTER_SHIFT 22
+#define DB_INFO_CFLAG_SHIFT 23
+#define DB_INFO_COS_SHIFT 24
+#define DB_INFO_TYPE_SHIFT 27
+
+#define DB_INFO_QID_MASK 0x1FFFU
+#define DB_INFO_NON_FILTER_MASK 0x1U
+#define DB_INFO_CFLAG_MASK 0x1U
+#define DB_INFO_COS_MASK 0x7U
+#define DB_INFO_TYPE_MASK 0x1FU
+#define DB_INFO_SET(val, member) \
+ (((u32)(val) & DB_INFO_##member##_MASK) << DB_INFO_##member##_SHIFT)
+
+#define DB_PI_LOW_MASK 0xFFU
+#define DB_PI_HIGH_MASK 0xFFU
+#define DB_PI_LOW(pi) ((pi) & DB_PI_LOW_MASK)
+#define DB_PI_HI_SHIFT 8
+#define DB_PI_HIGH(pi) (((pi) >> DB_PI_HI_SHIFT) & DB_PI_HIGH_MASK)
+#define DB_INFO_UPPER_32(val) (((u64)(val)) << 32)
+
+#define DB_ADDR(db_addr, pi) ((u64 *)(db_addr) + DB_PI_LOW(pi))
+#define SRC_TYPE 1
+
+/* Cflag data path. */
+#define SQ_CFLAG_DP 0
+#define RQ_CFLAG_DP 1
+
+#define MASKED_QUEUE_IDX(queue, idx) ((idx) & (queue)->q_mask)
+
+#define NIC_WQE_ADDR(queue, idx) \
+ ({ \
+ typeof(queue) __queue = (queue); \
+ (void *)((u64)(__queue->queue_buf_vaddr) + \
+ ((idx) << __queue->wqebb_shift)); \
+ })
+
+/**
+ * Write send queue doorbell.
+ *
+ * @param[in] db_addr
+ * Doorbell address.
+ * @param[in] q_id
+ * Send queue id.
+ * @param[in] cos
+ * Send queue cos.
+ * @param[in] cflag
+ * Cflag data path.
+ * @param[in] pi
+ * Send queue pi.
+ */
+static inline void
+hinic3_write_db(void *db_addr, u16 q_id, int cos, u8 cflag, u16 pi)
+{
+ u64 db;
+
+ /* Hardware will do endianness coverting. */
+ db = DB_PI_HIGH(pi);
+ db = DB_INFO_UPPER_32(db) | DB_INFO_SET(SRC_TYPE, TYPE) |
+ DB_INFO_SET(cflag, CFLAG) | DB_INFO_SET(cos, COS) |
+ DB_INFO_SET(q_id, QID);
+
+ rte_wmb(); /**< Write all before the doorbell. */
+
+ rte_write64(*((u64 *)&db), DB_ADDR(db_addr, pi));
+}
+
+/**
+ * Get minimum RX buffer size for device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ */
+void hinic3_get_func_rx_buf_size(void *dev);
+
+/**
+ * Initialize qps contexts, set SQ ci attributes, arm all SQ.
+ *
+ * Function will perform following steps:
+ * - Initialize SQ contexts.
+ * - Initialize RQ contexts.
+ * - Clean QP offload contexts of SQ and RQ.
+ * - Set root context for device.
+ * - Configure CI tables for each SQ.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int hinic3_init_qp_ctxts(void *dev);
+
+/**
+ * Free queue pair context.
+ *
+ * @param[in] hwdev
+ * Pointer to hardware device structure.
+ */
+void hinic3_free_qp_ctxts(void *hwdev);
+
+/**
+ * Update driver feature capabilities.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] s_feature
+ * s_feature driver supported.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+void hinic3_update_driver_feature(void *dev, u64 s_feature);
+
+/**
+ * Get driver feature capabilities.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * Feature capabilities of driver.
+ */
+u64 hinic3_get_driver_feature(void *dev);
+
+#endif /* _HINIC3_NIC_IO_H_ */
diff --git a/drivers/net/hinic3/hinic3_rx.c b/drivers/net/hinic3/hinic3_rx.c
new file mode 100644
index 0000000000..a1dc960236
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_rx.c
@@ -0,0 +1,811 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+#include <rte_ether.h>
+#include <rte_mbuf.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_pmd_hwif.h"
+#include "base/hinic3_pmd_hwdev.h"
+#include "base/hinic3_pmd_wq.h"
+#include "base/hinic3_pmd_nic_cfg.h"
+#include "hinic3_pmd_nic_io.h"
+#include "hinic3_pmd_ethdev.h"
+#include "hinic3_pmd_tx.h"
+#include "hinic3_pmd_rx.h"
+
+/**
+ * Get wqe from receive queue.
+ *
+ * @param[in] rxq
+ * Receive queue.
+ * @param[out] rq_wqe
+ * Receive queue wqe.
+ * @param[out] pi
+ * Current pi.
+ */
+static inline void
+hinic3_get_rq_wqe(struct hinic3_rxq *rxq, struct hinic3_rq_wqe **rq_wqe,
+ u16 *pi)
+{
+ *pi = MASKED_QUEUE_IDX(rxq, rxq->prod_idx);
+
+ /* Get only one rxq wqe. */
+ rxq->prod_idx++;
+ rxq->delta--;
+
+ *rq_wqe = NIC_WQE_ADDR(rxq, *pi);
+}
+
+/**
+ * Put wqe into receive queue.
+ *
+ * @param[in] rxq
+ * Receive queue.
+ * @param[in] wqe_cnt
+ * Wqebb counters.
+ */
+static inline void
+hinic3_put_rq_wqe(struct hinic3_rxq *rxq, u16 wqe_cnt)
+{
+ rxq->delta += wqe_cnt;
+ rxq->prod_idx -= wqe_cnt;
+}
+
+/**
+ * Get receive queue local pi.
+ *
+ * @param[in] rxq
+ * Receive queue.
+ * @return
+ * Receive queue local pi.
+ */
+static inline u16
+hinic3_get_rq_local_pi(struct hinic3_rxq *rxq)
+{
+ return MASKED_QUEUE_IDX(rxq, rxq->prod_idx);
+}
+
+/**
+ * Update receive queue hardware pi.
+ *
+ * @param[in] rxq
+ * Receive queue
+ * @param[in] pi
+ * Receive queue pi to update
+ */
+static inline void
+hinic3_update_rq_hw_pi(struct hinic3_rxq *rxq, u16 pi)
+{
+ *rxq->pi_virt_addr =
+ (u16)cpu_to_be16((pi & rxq->q_mask) << rxq->wqe_type);
+}
+
+u16
+hinic3_rx_fill_wqe(struct hinic3_rxq *rxq)
+{
+ struct hinic3_rq_wqe *rq_wqe = NULL;
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+ rte_iova_t cqe_dma;
+ u16 pi = 0;
+ u16 i;
+
+ cqe_dma = rxq->cqe_start_paddr;
+ for (i = 0; i < rxq->q_depth; i++) {
+ hinic3_get_rq_wqe(rxq, &rq_wqe, &pi);
+ if (!rq_wqe) {
+ PMD_DRV_LOG(ERR,
+ "Get rq wqe failed, rxq id: %d, wqe id: %d",
+ rxq->q_id, i);
+ break;
+ }
+
+ if (rxq->wqe_type == HINIC3_EXTEND_RQ_WQE) {
+ /* Unit of cqe length is 16B. */
+ hinic3_set_sge(&rq_wqe->extend_wqe.cqe_sect.sge,
+ cqe_dma,
+ HINIC3_CQE_LEN >> HINIC3_CQE_SIZE_SHIFT);
+ /* Use fixed len. */
+ rq_wqe->extend_wqe.buf_desc.sge.len =
+ nic_dev->rx_buff_len;
+ } else {
+ rq_wqe->normal_wqe.cqe_hi_addr = upper_32_bits(cqe_dma);
+ rq_wqe->normal_wqe.cqe_lo_addr = lower_32_bits(cqe_dma);
+ }
+
+ cqe_dma += sizeof(struct hinic3_rq_cqe);
+
+ hinic3_hw_be32_len(rq_wqe, rxq->wqebb_size);
+ }
+
+ hinic3_put_rq_wqe(rxq, i);
+
+ return i;
+}
+
+static struct rte_mbuf *
+hinic3_rx_alloc_mbuf(struct hinic3_rxq *rxq, rte_iova_t *dma_addr)
+{
+ struct rte_mbuf *mbuf = NULL;
+
+ if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, &mbuf, 1) != 0))
+ return NULL;
+
+ *dma_addr = rte_mbuf_data_iova_default(mbuf);
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_alloc_mbuf_bytes++;
+#endif
+ return mbuf;
+}
+
+#ifdef HINIC3_XSTAT_RXBUF_INFO
+static void
+hinic3_rxq_buffer_done_count(struct hinic3_rxq *rxq)
+{
+ u16 sw_ci, avail_pkts = 0, hit_done = 0, cqe_hole = 0;
+ u32 status;
+ volatile struct hinic3_rq_cqe *rx_cqe;
+
+ for (sw_ci = 0; sw_ci < rxq->q_depth; sw_ci++) {
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+
+ /* Check current ci is done. */
+ status = rx_cqe->status;
+ if (!HINIC3_GET_RX_DONE(status)) {
+ if (hit_done) {
+ cqe_hole++;
+ hit_done = 0;
+ }
+ continue;
+ }
+
+ avail_pkts++;
+ hit_done = 1;
+ }
+
+ rxq->rxq_stats.rx_avail = avail_pkts;
+ rxq->rxq_stats.rx_hole = cqe_hole;
+}
+
+void
+hinic3_get_stats(struct hinic3_rxq *rxq)
+{
+ rxq->rxq_stats.rx_mbuf = rxq->q_depth - hinic3_get_rq_free_wqebb(rxq);
+
+ hinic3_rxq_buffer_done_count(rxq);
+}
+#endif
+
+u16
+hinic3_rx_fill_buffers(struct hinic3_rxq *rxq)
+{
+ struct hinic3_rq_wqe *rq_wqe = NULL;
+ struct hinic3_rx_info *rx_info = NULL;
+ struct rte_mbuf *mb = NULL;
+ rte_iova_t dma_addr;
+ u16 i, free_wqebbs;
+
+ free_wqebbs = rxq->delta - 1;
+ for (i = 0; i < free_wqebbs; i++) {
+ rx_info = &rxq->rx_info[rxq->next_to_update];
+
+ mb = hinic3_rx_alloc_mbuf(rxq, &dma_addr);
+ if (!mb) {
+ PMD_DRV_LOG(ERR, "Alloc mbuf failed");
+ break;
+ }
+
+ rx_info->mbuf = mb;
+
+ rq_wqe = NIC_WQE_ADDR(rxq, rxq->next_to_update);
+
+ /* Fill buffer address only. */
+ if (rxq->wqe_type == HINIC3_EXTEND_RQ_WQE) {
+ rq_wqe->extend_wqe.buf_desc.sge.hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ rq_wqe->extend_wqe.buf_desc.sge.lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ } else {
+ rq_wqe->normal_wqe.buf_hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ rq_wqe->normal_wqe.buf_lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ }
+
+ rxq->next_to_update = (rxq->next_to_update + 1) & rxq->q_mask;
+ }
+
+ if (likely(i > 0)) {
+#ifndef HINIC3_RQ_DB
+ hinic3_write_db(rxq->db_addr, rxq->q_id, 0, RQ_CFLAG_DP,
+ (u16)(rxq->next_to_update << rxq->wqe_type));
+ /* Init rxq contxet used, need to optimization. */
+ rxq->prod_idx = rxq->next_to_update;
+#else
+ rte_wmb();
+ rxq->prod_idx = rxq->next_to_update;
+ hinic3_update_rq_hw_pi(rxq, rxq->next_to_update);
+#endif
+ rxq->delta -= i;
+ } else {
+ PMD_DRV_LOG(ERR, "Alloc rx buffers failed, rxq_id: %d",
+ rxq->q_id);
+ }
+
+ return i;
+}
+
+void
+hinic3_free_rxq_mbufs(struct hinic3_rxq *rxq)
+{
+ struct hinic3_rx_info *rx_info = NULL;
+ int free_wqebbs = hinic3_get_rq_free_wqebb(rxq) + 1;
+ volatile struct hinic3_rq_cqe *rx_cqe = NULL;
+ u16 ci;
+
+ while (free_wqebbs++ < rxq->q_depth) {
+ ci = hinic3_get_rq_local_ci(rxq);
+
+ rx_cqe = &rxq->rx_cqe[ci];
+
+ /* Clear done bit. */
+ rx_cqe->status = 0;
+
+ rx_info = &rxq->rx_info[ci];
+ rte_pktmbuf_free(rx_info->mbuf);
+ rx_info->mbuf = NULL;
+
+ hinic3_update_rq_local_ci(rxq, 1);
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_free_mbuf_bytes++;
+#endif
+ }
+}
+
+void
+hinic3_free_all_rxq_mbufs(struct hinic3_nic_dev *nic_dev)
+{
+ u16 qid;
+
+ for (qid = 0; qid < nic_dev->num_rqs; qid++)
+ hinic3_free_rxq_mbufs(nic_dev->rxqs[qid]);
+}
+
+static u32
+hinic3_rx_alloc_mbuf_bulk(struct hinic3_rxq *rxq, struct rte_mbuf **mbufs,
+ u32 exp_mbuf_cnt)
+{
+ u32 avail_cnt;
+ int err;
+
+ err = rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, exp_mbuf_cnt);
+ if (likely(err == 0)) {
+ avail_cnt = exp_mbuf_cnt;
+ } else {
+ avail_cnt = 0;
+ rxq->rxq_stats.rx_nombuf += exp_mbuf_cnt;
+ }
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_alloc_mbuf_bytes += avail_cnt;
+#endif
+ return avail_cnt;
+}
+
+static int
+hinic3_rearm_rxq_mbuf(struct hinic3_rxq *rxq)
+{
+ struct hinic3_rq_wqe *rq_wqe = NULL;
+ struct rte_mbuf **rearm_mbufs;
+ u32 i, free_wqebbs, rearm_wqebbs, exp_wqebbs;
+ rte_iova_t dma_addr;
+ u16 pi;
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+
+ /* Check free wqebb cnt fo rearm. */
+ free_wqebbs = hinic3_get_rq_free_wqebb(rxq);
+ if (unlikely(free_wqebbs < rxq->rx_free_thresh))
+ return -ENOMEM;
+
+ /* Get rearm mbuf array. */
+ pi = hinic3_get_rq_local_pi(rxq);
+ rearm_mbufs = (struct rte_mbuf **)(&rxq->rx_info[pi]);
+
+ /* Check rxq free wqebbs turn around. */
+ exp_wqebbs = rxq->q_depth - pi;
+ if (free_wqebbs < exp_wqebbs)
+ exp_wqebbs = free_wqebbs;
+
+ /* Alloc mbuf in bulk. */
+ rearm_wqebbs = hinic3_rx_alloc_mbuf_bulk(rxq, rearm_mbufs, exp_wqebbs);
+ if (unlikely(rearm_wqebbs == 0))
+ return -ENOMEM;
+
+ /* Rearm rxq mbuf. */
+ rq_wqe = NIC_WQE_ADDR(rxq, pi);
+ for (i = 0; i < rearm_wqebbs; i++) {
+ dma_addr = rte_mbuf_data_iova_default(rearm_mbufs[i]);
+
+ /* Fill buffer address only. */
+ if (rxq->wqe_type == HINIC3_EXTEND_RQ_WQE) {
+ rq_wqe->extend_wqe.buf_desc.sge.hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ rq_wqe->extend_wqe.buf_desc.sge.lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ rq_wqe->extend_wqe.buf_desc.sge.len =
+ nic_dev->rx_buff_len;
+ } else {
+ rq_wqe->normal_wqe.buf_hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ rq_wqe->normal_wqe.buf_lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ }
+
+ rq_wqe =
+ (struct hinic3_rq_wqe *)((u64)rq_wqe + rxq->wqebb_size);
+ }
+ rxq->prod_idx += rearm_wqebbs;
+ rxq->delta -= rearm_wqebbs;
+
+#ifndef HINIC3_RQ_DB
+ hinic3_write_db(rxq->db_addr, rxq->q_id, 0, RQ_CFLAG_DP,
+ ((pi + rearm_wqebbs) & rxq->q_mask) << rxq->wqe_type);
+#else
+ /* Update rxq hw_pi. */
+ rte_wmb();
+ hinic3_update_rq_hw_pi(rxq, pi + rearm_wqebbs);
+#endif
+ return 0;
+}
+
+static int
+hinic3_init_rss_key(struct hinic3_nic_dev *nic_dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ u8 default_rss_key[HINIC3_RSS_KEY_SIZE] = {
+ 0x6d, 0x5a, 0x56, 0xda, 0x25, 0x5b, 0x0e, 0xc2,
+ 0x41, 0x67, 0x25, 0x3d, 0x43, 0xa3, 0x8f, 0xb0,
+ 0xd0, 0xca, 0x2b, 0xcb, 0xae, 0x7b, 0x30, 0xb4,
+ 0x77, 0xcb, 0x2d, 0xa3, 0x80, 0x30, 0xf2, 0x0c,
+ 0x6a, 0x42, 0xb7, 0x3b, 0xbe, 0xac, 0x01, 0xfa};
+ u8 hashkey[HINIC3_RSS_KEY_SIZE] = {0};
+ int err;
+
+ if (rss_conf->rss_key == NULL ||
+ rss_conf->rss_key_len > HINIC3_RSS_KEY_SIZE)
+ memcpy(hashkey, default_rss_key, HINIC3_RSS_KEY_SIZE);
+ else
+ memcpy(hashkey, rss_conf->rss_key, rss_conf->rss_key_len);
+
+ err = hinic3_rss_set_hash_key(nic_dev->hwdev, hashkey,
+ HINIC3_RSS_KEY_SIZE);
+ if (err)
+ return err;
+
+ memcpy(nic_dev->rss_key, hashkey, HINIC3_RSS_KEY_SIZE);
+ return 0;
+}
+
+void
+hinic3_add_rq_to_rx_queue_list(struct hinic3_nic_dev *nic_dev, u16 queue_id)
+{
+ u8 rss_queue_count = nic_dev->num_rss;
+
+ RTE_ASSERT(rss_queue_count <= (RTE_DIM(nic_dev->rx_queue_list) - 1));
+
+ nic_dev->rx_queue_list[rss_queue_count] = (u8)queue_id;
+ nic_dev->num_rss++;
+}
+
+void
+hinic3_init_rx_queue_list(struct hinic3_nic_dev *nic_dev)
+{
+ nic_dev->num_rss = 0;
+}
+
+static void
+hinic3_fill_indir_tbl(struct hinic3_nic_dev *nic_dev, u32 *indir_tbl)
+{
+ u8 rss_queue_count = nic_dev->num_rss;
+ int i = 0;
+ int j;
+
+ if (rss_queue_count == 0) {
+ /* Delete q_id from indir tbl. */
+ for (i = 0; i < HINIC3_RSS_INDIR_SIZE; i++)
+ /* Invalid value in indir tbl. */
+ indir_tbl[i] = 0xFFFF;
+ } else {
+ while (i < HINIC3_RSS_INDIR_SIZE)
+ for (j = 0; (j < rss_queue_count) &&
+ (i < HINIC3_RSS_INDIR_SIZE); j++)
+ indir_tbl[i++] = nic_dev->rx_queue_list[j];
+ }
+}
+
+int
+hinic3_refill_indir_rqid(struct hinic3_rxq *rxq)
+{
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+ u32 *indir_tbl;
+ int err;
+
+ indir_tbl = rte_zmalloc(NULL, HINIC3_RSS_INDIR_SIZE * sizeof(u32), 0);
+ if (!indir_tbl) {
+ PMD_DRV_LOG(ERR,
+ "Alloc indir_tbl mem failed, "
+ "eth_dev:%s, queue_idx:%d",
+ nic_dev->dev_name, rxq->q_id);
+ return -ENOMEM;
+ }
+
+ /* Build indir tbl according to the number of rss queue. */
+ hinic3_fill_indir_tbl(nic_dev, indir_tbl);
+
+ err = hinic3_rss_set_indir_tbl(nic_dev->hwdev, indir_tbl,
+ HINIC3_RSS_INDIR_SIZE);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Set indrect table failed, eth_dev:%s, queue_idx:%d",
+ nic_dev->dev_name, rxq->q_id);
+ goto out;
+ }
+
+out:
+ rte_free(indir_tbl);
+ return err;
+}
+
+static int
+hinic3_init_rss_type(struct hinic3_nic_dev *nic_dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct hinic3_rss_type rss_type = {0};
+ u64 rss_hf = rss_conf->rss_hf;
+ int err;
+
+ rss_type.ipv4 = (rss_hf & (RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4)) ? 1 : 0;
+ rss_type.tcp_ipv4 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP) ? 1 : 0;
+ rss_type.ipv6 = (rss_hf & (RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6)) ? 1 : 0;
+ rss_type.ipv6_ext = (rss_hf & RTE_ETH_RSS_IPV6_EX) ? 1 : 0;
+ rss_type.tcp_ipv6 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP) ? 1 : 0;
+ rss_type.tcp_ipv6_ext = (rss_hf & RTE_ETH_RSS_IPV6_TCP_EX) ? 1 : 0;
+ rss_type.udp_ipv4 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP) ? 1 : 0;
+ rss_type.udp_ipv6 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP) ? 1 : 0;
+
+ err = hinic3_set_rss_type(nic_dev->hwdev, rss_type);
+ return err;
+}
+
+int
+hinic3_update_rss_config(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u8 prio_tc[HINIC3_DCB_UP_MAX] = {0};
+ u8 num_tc = 0;
+ int err;
+
+ if (rss_conf->rss_hf == 0) {
+ rss_conf->rss_hf = HINIC3_RSS_OFFLOAD_ALL;
+ } else if ((rss_conf->rss_hf & HINIC3_RSS_OFFLOAD_ALL) == 0) {
+ PMD_DRV_LOG(ERR, "Does't support rss hash type: %" PRIu64,
+ rss_conf->rss_hf);
+ return -EINVAL;
+ }
+
+ err = hinic3_rss_template_alloc(nic_dev->hwdev);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Alloc rss template failed, err: %d", err);
+ return err;
+ }
+
+ err = hinic3_init_rss_key(nic_dev, rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init rss hash key failed, err: %d", err);
+ goto init_rss_fail;
+ }
+
+ err = hinic3_init_rss_type(nic_dev, rss_conf);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init rss hash type failed, err: %d", err);
+ goto init_rss_fail;
+ }
+
+ err = hinic3_rss_set_hash_engine(nic_dev->hwdev,
+ HINIC3_RSS_HASH_ENGINE_TYPE_TOEP);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Init rss hash function failed, err: %d", err);
+ goto init_rss_fail;
+ }
+
+ err = hinic3_rss_cfg(nic_dev->hwdev, HINIC3_RSS_ENABLE, num_tc,
+ prio_tc);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Enable rss failed, err: %d", err);
+ goto init_rss_fail;
+ }
+
+ nic_dev->rss_state = HINIC3_RSS_ENABLE;
+ return 0;
+
+init_rss_fail:
+ if (hinic3_rss_template_free(nic_dev->hwdev))
+ PMD_DRV_LOG(WARNING, "Free rss template failed");
+
+ return err;
+}
+
+/**
+ * Search given queue array to find possition of given id.
+ * Return queue pos or queue_count if not found.
+ */
+static u8
+hinic3_find_queue_pos_by_rq_id(u8 *queues, u8 queues_count, u8 queue_id)
+{
+ u8 pos;
+
+ for (pos = 0; pos < queues_count; pos++) {
+ if (queue_id == queues[pos])
+ break;
+ }
+
+ return pos;
+}
+
+void
+hinic3_remove_rq_from_rx_queue_list(struct hinic3_nic_dev *nic_dev,
+ u16 queue_id)
+{
+ u8 queue_pos;
+ u8 rss_queue_count = nic_dev->num_rss;
+
+ queue_pos = hinic3_find_queue_pos_by_rq_id(nic_dev->rx_queue_list,
+ rss_queue_count,
+ (u8)queue_id);
+ /*
+ * If queue was not at the end of the list,
+ * shift started queues up queue array list.
+ */
+ if (queue_pos < rss_queue_count) {
+ rss_queue_count--;
+ memmove(nic_dev->rx_queue_list + queue_pos,
+ nic_dev->rx_queue_list + queue_pos + 1,
+ (rss_queue_count - queue_pos) *
+ sizeof(nic_dev->rx_queue_list[0]));
+ }
+
+ RTE_ASSERT(rss_queue_count < RTE_DIM(nic_dev->rx_queue_list));
+ nic_dev->num_rss = rss_queue_count;
+}
+
+static void
+hinic3_rx_queue_release_mbufs(struct hinic3_rxq *rxq)
+{
+ u16 sw_ci, ci_mask, free_wqebbs;
+ u16 rx_buf_len;
+ u32 status, vlan_len, pkt_len;
+ u32 pkt_left_len = 0;
+ u32 nr_released = 0;
+ struct hinic3_rx_info *rx_info;
+ volatile struct hinic3_rq_cqe *rx_cqe;
+
+ sw_ci = hinic3_get_rq_local_ci(rxq);
+ rx_info = &rxq->rx_info[sw_ci];
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+ free_wqebbs = hinic3_get_rq_free_wqebb(rxq) + 1;
+ status = rx_cqe->status;
+ ci_mask = rxq->q_mask;
+
+ while (free_wqebbs < rxq->q_depth) {
+ rx_buf_len = rxq->buf_len;
+ if (pkt_left_len != 0) {
+ /* Flush continues jumbo rqe. */
+ pkt_left_len = (pkt_left_len <= rx_buf_len)
+ ? 0
+ : (pkt_left_len - rx_buf_len);
+ } else if (HINIC3_GET_RX_FLUSH(status)) {
+ /* Flush one released rqe. */
+ pkt_left_len = 0;
+ } else if (HINIC3_GET_RX_DONE(status)) {
+ /* Flush single packet or first jumbo rqe. */
+ vlan_len = hinic3_hw_cpu32(rx_cqe->vlan_len);
+ pkt_len = HINIC3_GET_RX_PKT_LEN(vlan_len);
+ pkt_left_len = (pkt_len <= rx_buf_len)
+ ? 0
+ : (pkt_len - rx_buf_len);
+ } else {
+ break;
+ }
+ rte_pktmbuf_free(rx_info->mbuf);
+
+ rx_info->mbuf = NULL;
+ rx_cqe->status = 0;
+ nr_released++;
+ free_wqebbs++;
+
+ /* Update ci to next cqe. */
+ sw_ci++;
+ sw_ci &= ci_mask;
+ rx_info = &rxq->rx_info[sw_ci];
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+ status = rx_cqe->status;
+ }
+
+ hinic3_update_rq_local_ci(rxq, (u16)nr_released);
+}
+
+int
+hinic3_poll_rq_empty(struct hinic3_rxq *rxq)
+{
+ unsigned long timeout;
+ int free_wqebb;
+ int err = -EFAULT;
+
+ timeout = msecs_to_jiffies(HINIC3_FLUSH_QUEUE_TIMEOUT) + jiffies;
+ do {
+ free_wqebb = hinic3_get_rq_free_wqebb(rxq) + 1;
+ if (free_wqebb == rxq->q_depth) {
+ err = 0;
+ break;
+ }
+ hinic3_rx_queue_release_mbufs(rxq);
+ rte_delay_us(1);
+ } while (time_before(jiffies, timeout));
+
+ return err;
+}
+
+void
+hinic3_dump_cqe_status(struct hinic3_rxq *rxq, u32 *cqe_done_cnt,
+ u32 *cqe_hole_cnt, u32 *head_ci, u32 *head_done)
+{
+ u16 sw_ci;
+ u16 avail_pkts = 0;
+ u16 hit_done = 0;
+ u16 cqe_hole = 0;
+ u32 status;
+ volatile struct hinic3_rq_cqe *rx_cqe;
+
+ sw_ci = hinic3_get_rq_local_ci(rxq);
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+ status = rx_cqe->status;
+ *head_done = HINIC3_GET_RX_DONE(status);
+ *head_ci = sw_ci;
+
+ for (sw_ci = 0; sw_ci < rxq->q_depth; sw_ci++) {
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+
+ /* Check current ci is done. */
+ status = rx_cqe->status;
+ if (!HINIC3_GET_RX_DONE(status) ||
+ !HINIC3_GET_RX_FLUSH(status)) {
+ if (hit_done) {
+ cqe_hole++;
+ hit_done = 0;
+ }
+
+ continue;
+ }
+
+ avail_pkts++;
+ hit_done = 1;
+ }
+
+ *cqe_done_cnt = avail_pkts;
+ *cqe_hole_cnt = cqe_hole;
+}
+
+int
+hinic3_stop_rq(struct rte_eth_dev *eth_dev, struct hinic3_rxq *rxq)
+{
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+ u32 cqe_done_cnt = 0;
+ u32 cqe_hole_cnt = 0;
+ u32 head_ci, head_done;
+ int err;
+
+ /* Disable rxq intr. */
+ hinic3_dev_rx_queue_intr_disable(eth_dev, rxq->q_id);
+
+ /* Lock dev queue switch. */
+ rte_spinlock_lock(&nic_dev->queue_list_lock);
+
+ if (nic_dev->num_rss == 1) {
+ err = hinic3_set_vport_enable(nic_dev->hwdev, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "%s Disable vport failed, rc:%d",
+ nic_dev->dev_name, err);
+ }
+ }
+ hinic3_remove_rq_from_rx_queue_list(nic_dev, rxq->q_id);
+
+ /*
+ * If RSS is enable, remove q_id from rss indir table.
+ * If RSS is disable, no mbuf in rq, pakcet will be dropped.
+ */
+ if (nic_dev->rss_state == HINIC3_RSS_ENABLE) {
+ err = hinic3_refill_indir_rqid(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Clear rq in indirect table failed, "
+ "eth_dev:%s, queue_idx:%d",
+ nic_dev->dev_name, rxq->q_id);
+ hinic3_add_rq_to_rx_queue_list(nic_dev, rxq->q_id);
+ goto set_indir_failed;
+ }
+ }
+
+ /* Unlock dev queue list switch. */
+ rte_spinlock_unlock(&nic_dev->queue_list_lock);
+
+ /* Send flush rxq cmd to device. */
+ err = hinic3_set_rq_flush(nic_dev->hwdev, rxq->q_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Flush rq failed, eth_dev:%s, queue_idx:%d",
+ nic_dev->dev_name, rxq->q_id);
+ goto rq_flush_failed;
+ }
+
+ err = hinic3_poll_rq_empty(rxq);
+ if (err) {
+ hinic3_dump_cqe_status(rxq, &cqe_done_cnt, &cqe_hole_cnt,
+ &head_ci, &head_done);
+ PMD_DRV_LOG(ERR,
+ "Poll rq empty timeout, eth_dev:%s, queue_idx:%d, "
+ "mbuf_left:%d, "
+ "cqe_done:%d, cqe_hole:%d, cqe[%d].done=%d",
+ nic_dev->dev_name, rxq->q_id,
+ rxq->q_depth - hinic3_get_rq_free_wqebb(rxq),
+ cqe_done_cnt, cqe_hole_cnt, head_ci, head_done);
+ goto poll_rq_failed;
+ }
+
+ return 0;
+
+poll_rq_failed:
+rq_flush_failed:
+ rte_spinlock_lock(&nic_dev->queue_list_lock);
+set_indir_failed:
+ hinic3_add_rq_to_rx_queue_list(nic_dev, rxq->q_id);
+ if (nic_dev->rss_state == HINIC3_RSS_ENABLE)
+ (void)hinic3_refill_indir_rqid(rxq);
+ rte_spinlock_unlock(&nic_dev->queue_list_lock);
+ hinic3_dev_rx_queue_intr_enable(eth_dev, rxq->q_id);
+ return err;
+}
+
+int
+hinic3_start_rq(struct rte_eth_dev *eth_dev, struct hinic3_rxq *rxq)
+{
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+ int err = 0;
+
+ /* Lock dev queue switch. */
+ rte_spinlock_lock(&nic_dev->queue_list_lock);
+ hinic3_add_rq_to_rx_queue_list(nic_dev, rxq->q_id);
+
+ if (nic_dev->rss_state == HINIC3_RSS_ENABLE) {
+ err = hinic3_refill_indir_rqid(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Refill rq to indrect table failed, "
+ "eth_dev:%s, queue_idx:%d err:%d",
+ nic_dev->dev_name, rxq->q_id, err);
+ hinic3_remove_rq_from_rx_queue_list(nic_dev, rxq->q_id);
+ }
+ }
+ hinic3_rearm_rxq_mbuf(rxq);
+ if (rxq->nic_dev->num_rss == 1) {
+ err = hinic3_set_vport_enable(nic_dev->hwdev, true);
+ if (err)
+ PMD_DRV_LOG(ERR, "%s enable vport failed, err:%d",
+ nic_dev->dev_name, err);
+ }
+
+ /* Unlock dev queue list switch. */
+ rte_spinlock_unlock(&nic_dev->queue_list_lock);
+
+ hinic3_dev_rx_queue_intr_enable(eth_dev, rxq->q_id);
+
+ return err;
+}
diff --git a/drivers/net/hinic3/hinic3_rx.h b/drivers/net/hinic3/hinic3_rx.h
new file mode 100644
index 0000000000..56386b2511
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_rx.h
@@ -0,0 +1,356 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_RX_H_
+#define _HINIC3_RX_H_
+
+#include "hinic3_wq.h"
+#include "hinic3_nic_io.h"
+
+#define RQ_CQE_OFFOLAD_TYPE_PKT_TYPE_SHIFT 0
+#define RQ_CQE_OFFOLAD_TYPE_PKT_UMBCAST_SHIFT 19
+#define RQ_CQE_OFFOLAD_TYPE_VLAN_EN_SHIFT 21
+#define RQ_CQE_OFFOLAD_TYPE_RSS_TYPE_SHIFT 24
+
+#define RQ_CQE_OFFOLAD_TYPE_PKT_TYPE_MASK 0xFFFU
+#define RQ_CQE_OFFOLAD_TYPE_PKT_UMBCAST_MASK 0x3U
+#define RQ_CQE_OFFOLAD_TYPE_VLAN_EN_MASK 0x1U
+#define RQ_CQE_OFFOLAD_TYPE_RSS_TYPE_MASK 0xFFU
+
+#define DPI_EXT_ACTION_FILED (1ULL << 32)
+
+#define RQ_CQE_OFFOLAD_TYPE_GET(val, member) \
+ (((val) >> RQ_CQE_OFFOLAD_TYPE_##member##_SHIFT) & \
+ RQ_CQE_OFFOLAD_TYPE_##member##_MASK)
+
+#define HINIC3_GET_RX_PKT_TYPE(offload_type) \
+ RQ_CQE_OFFOLAD_TYPE_GET(offload_type, PKT_TYPE)
+
+#define HINIC3_GET_RX_PKT_UMBCAST(offload_type) \
+ RQ_CQE_OFFOLAD_TYPE_GET(offload_type, PKT_UMBCAST)
+
+#define HINIC3_GET_RX_VLAN_OFFLOAD_EN(offload_type) \
+ RQ_CQE_OFFOLAD_TYPE_GET(offload_type, VLAN_EN)
+
+#define HINIC3_GET_RSS_TYPES(offload_type) \
+ RQ_CQE_OFFOLAD_TYPE_GET(offload_type, RSS_TYPE)
+
+#define RQ_CQE_SGE_VLAN_SHIFT 0
+#define RQ_CQE_SGE_LEN_SHIFT 16
+
+#define RQ_CQE_SGE_VLAN_MASK 0xFFFFU
+#define RQ_CQE_SGE_LEN_MASK 0xFFFFU
+
+#define RQ_CQE_SGE_GET(val, member) \
+ (((val) >> RQ_CQE_SGE_##member##_SHIFT) & RQ_CQE_SGE_##member##_MASK)
+
+#define HINIC3_GET_RX_VLAN_TAG(vlan_len) RQ_CQE_SGE_GET(vlan_len, VLAN)
+
+#define HINIC3_GET_RX_PKT_LEN(vlan_len) RQ_CQE_SGE_GET(vlan_len, LEN)
+
+#define RQ_CQE_STATUS_CSUM_ERR_SHIFT 0
+#define RQ_CQE_STATUS_NUM_LRO_SHIFT 16
+#define RQ_CQE_STATUS_LRO_PUSH_SHIFT 25
+#define RQ_CQE_STATUS_LRO_ENTER_SHIFT 26
+#define RQ_CQE_STATUS_LRO_INTR_SHIFT 27
+
+#define RQ_CQE_STATUS_BP_EN_SHIFT 30
+#define RQ_CQE_STATUS_RXDONE_SHIFT 31
+#define RQ_CQE_STATUS_DECRY_PKT_SHIFT 29
+#define RQ_CQE_STATUS_FLUSH_SHIFT 28
+
+#define RQ_CQE_STATUS_CSUM_ERR_MASK 0xFFFFU
+#define RQ_CQE_STATUS_NUM_LRO_MASK 0xFFU
+#define RQ_CQE_STATUS_LRO_PUSH_MASK 0X1U
+#define RQ_CQE_STATUS_LRO_ENTER_MASK 0X1U
+#define RQ_CQE_STATUS_LRO_INTR_MASK 0X1U
+#define RQ_CQE_STATUS_BP_EN_MASK 0X1U
+#define RQ_CQE_STATUS_RXDONE_MASK 0x1U
+#define RQ_CQE_STATUS_FLUSH_MASK 0x1U
+#define RQ_CQE_STATUS_DECRY_PKT_MASK 0x1U
+
+#define RQ_CQE_STATUS_GET(val, member) \
+ (((val) >> RQ_CQE_STATUS_##member##_SHIFT) & \
+ RQ_CQE_STATUS_##member##_MASK)
+
+#define HINIC3_GET_RX_CSUM_ERR(status) RQ_CQE_STATUS_GET(status, CSUM_ERR)
+
+#define HINIC3_GET_RX_DONE(status) RQ_CQE_STATUS_GET(status, RXDONE)
+
+#define HINIC3_GET_RX_FLUSH(status) RQ_CQE_STATUS_GET(status, FLUSH)
+
+#define HINIC3_GET_RX_BP_EN(status) RQ_CQE_STATUS_GET(status, BP_EN)
+
+#define HINIC3_GET_RX_NUM_LRO(status) RQ_CQE_STATUS_GET(status, NUM_LRO)
+
+#define HINIC3_RX_IS_DECRY_PKT(status) RQ_CQE_STATUS_GET(status, DECRY_PKT)
+
+#define RQ_CQE_SUPER_CQE_EN_SHIFT 0
+#define RQ_CQE_PKT_NUM_SHIFT 1
+#define RQ_CQE_PKT_LAST_LEN_SHIFT 6
+#define RQ_CQE_PKT_FIRST_LEN_SHIFT 19
+
+#define RQ_CQE_SUPER_CQE_EN_MASK 0x1
+#define RQ_CQE_PKT_NUM_MASK 0x1FU
+#define RQ_CQE_PKT_FIRST_LEN_MASK 0x1FFFU
+#define RQ_CQE_PKT_LAST_LEN_MASK 0x1FFFU
+
+#define RQ_CQE_PKT_NUM_GET(val, member) \
+ (((val) >> RQ_CQE_PKT_##member##_SHIFT) & RQ_CQE_PKT_##member##_MASK)
+#define HINIC3_GET_RQ_CQE_PKT_NUM(pkt_info) RQ_CQE_PKT_NUM_GET(pkt_info, NUM)
+
+#define RQ_CQE_SUPER_CQE_EN_GET(val, member) \
+ (((val) >> RQ_CQE_##member##_SHIFT) & RQ_CQE_##member##_MASK)
+
+#define HINIC3_GET_SUPER_CQE_EN(pkt_info) \
+ RQ_CQE_SUPER_CQE_EN_GET(pkt_info, SUPER_CQE_EN)
+
+#define RQ_CQE_PKT_LEN_GET(val, member) \
+ (((val) >> RQ_CQE_PKT_##member##_SHIFT) & RQ_CQE_PKT_##member##_MASK)
+
+#define RQ_CQE_DECRY_INFO_DECRY_STATUS_SHIFT 8
+#define RQ_CQE_DECRY_INFO_ESP_NEXT_HEAD_SHIFT 0
+
+#define RQ_CQE_DECRY_INFO_DECRY_STATUS_MASK 0xFFU
+#define RQ_CQE_DECRY_INFO_ESP_NEXT_HEAD_MASK 0xFFU
+
+#define RQ_CQE_DECRY_INFO_GET(val, member) \
+ (((val) >> RQ_CQE_DECRY_INFO_##member##_SHIFT) & \
+ RQ_CQE_DECRY_INFO_##member##_MASK)
+
+#define HINIC3_GET_DECRYPT_STATUS(decry_info) \
+ RQ_CQE_DECRY_INFO_GET(decry_info, DECRY_STATUS)
+
+#define HINIC3_GET_ESP_NEXT_HEAD(decry_info) \
+ RQ_CQE_DECRY_INFO_GET(decry_info, ESP_NEXT_HEAD)
+
+/* Rx cqe checksum err */
+#define HINIC3_RX_CSUM_IP_CSUM_ERR BIT(0)
+#define HINIC3_RX_CSUM_TCP_CSUM_ERR BIT(1)
+#define HINIC3_RX_CSUM_UDP_CSUM_ERR BIT(2)
+#define HINIC3_RX_CSUM_IGMP_CSUM_ERR BIT(3)
+#define HINIC3_RX_CSUM_ICMP_V4_CSUM_ERR BIT(4)
+#define HINIC3_RX_CSUM_ICMP_V6_CSUM_ERR BIT(5)
+#define HINIC3_RX_CSUM_SCTP_CRC_ERR BIT(6)
+#define HINIC3_RX_CSUM_HW_CHECK_NONE BIT(7)
+#define HINIC3_RX_CSUM_IPSU_OTHER_ERR BIT(8)
+
+#define HINIC3_DEFAULT_RX_CSUM_OFFLOAD 0xFFF
+#define HINIC3_CQE_LEN 32
+
+#define HINIC3_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER | \
+ RTE_ETH_RSS_IPV6_EX | \
+ RTE_ETH_RSS_IPV6_TCP_EX | \
+ RTE_ETH_RSS_IPV6_UDP_EX)
+
+struct hinic3_rxq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 errors;
+ u64 csum_errors;
+ u64 other_errors;
+ u64 unlock_bp;
+ u64 dropped;
+
+ u64 rx_nombuf;
+ u64 rx_discards;
+ u64 burst_pkts;
+ u64 empty;
+ u64 tsc;
+#ifdef HINIC3_XSTAT_MBUF_USE
+ u64 rx_alloc_mbuf_bytes;
+ u64 rx_free_mbuf_bytes;
+ u64 rx_left_mbuf_bytes;
+#endif
+
+#ifdef HINIC3_XSTAT_RXBUF_INFO
+ u64 rx_mbuf;
+ u64 rx_avail;
+ u64 rx_hole;
+#endif
+
+#ifdef HINIC3_XSTAT_PROF_RX
+ u64 app_tsc;
+ u64 pmd_tsc;
+#endif
+};
+
+struct __rte_cache_aligned hinic3_rq_cqe {
+ u32 status;
+ u32 vlan_len;
+
+ u32 offload_type;
+ u32 hash_val;
+ u32 mark_id_0;
+ u32 mark_id_1;
+ u32 mark_id_2;
+ u32 pkt_info;
+};
+
+/**
+ * Attention: please do not add any member in hinic3_rx_info
+ * because rxq bulk rearm mode will write mbuf in rx_info.
+ */
+struct hinic3_rx_info {
+ struct rte_mbuf *mbuf;
+};
+
+struct hinic3_sge_sect {
+ struct hinic3_sge sge;
+ u32 rsvd;
+};
+
+struct hinic3_rq_extend_wqe {
+ struct hinic3_sge_sect buf_desc;
+ struct hinic3_sge_sect cqe_sect;
+};
+
+struct hinic3_rq_normal_wqe {
+ u32 buf_hi_addr;
+ u32 buf_lo_addr;
+ u32 cqe_hi_addr;
+ u32 cqe_lo_addr;
+};
+
+struct hinic3_rq_wqe {
+ union {
+ struct hinic3_rq_normal_wqe normal_wqe;
+ struct hinic3_rq_extend_wqe extend_wqe;
+ };
+};
+
+struct __rte_cache_aligned hinic3_rxq {
+ struct hinic3_nic_dev *nic_dev;
+
+ u16 q_id;
+ u16 q_depth;
+ u16 q_mask;
+ u16 buf_len;
+
+ u32 rx_buff_shift;
+
+ u16 rx_free_thresh;
+ u16 rxinfo_align_end;
+ u16 wqebb_shift;
+ u16 wqebb_size;
+
+ u16 wqe_type;
+ u16 cons_idx;
+ u16 prod_idx;
+ u16 delta;
+
+ u16 next_to_update;
+ u16 port_id;
+
+ const struct rte_memzone *rq_mz;
+ void *queue_buf_vaddr; /**< rxq dma info */
+ rte_iova_t queue_buf_paddr;
+
+ const struct rte_memzone *pi_mz;
+ u16 *pi_virt_addr;
+ void *db_addr;
+ rte_iova_t pi_dma_addr;
+
+ struct hinic3_rx_info *rx_info;
+ struct hinic3_rq_cqe *rx_cqe;
+ struct rte_mempool *mb_pool;
+
+ const struct rte_memzone *cqe_mz;
+ rte_iova_t cqe_start_paddr;
+ void *cqe_start_vaddr;
+ u8 dp_intr_en;
+ u16 msix_entry_idx;
+
+ unsigned long status;
+ u64 wait_time_cycle;
+
+ struct hinic3_rxq_stats rxq_stats;
+#ifdef HINIC3_XSTAT_PROF_RX
+ uint64_t prof_rx_end_tsc; /**< Performance profiling. */
+#endif
+};
+
+u16 hinic3_rx_fill_wqe(struct hinic3_rxq *rxq);
+
+u16 hinic3_rx_fill_buffers(struct hinic3_rxq *rxq);
+
+void hinic3_free_rxq_mbufs(struct hinic3_rxq *rxq);
+
+void hinic3_free_all_rxq_mbufs(struct hinic3_nic_dev *nic_dev);
+
+int hinic3_update_rss_config(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf);
+
+int hinic3_poll_rq_empty(struct hinic3_rxq *rxq);
+
+void hinic3_dump_cqe_status(struct hinic3_rxq *rxq, u32 *cqe_done_cnt,
+ u32 *cqe_hole_cnt, u32 *head_ci, u32 *head_done);
+
+int hinic3_stop_rq(struct rte_eth_dev *eth_dev, struct hinic3_rxq *rxq);
+
+int hinic3_start_rq(struct rte_eth_dev *eth_dev, struct hinic3_rxq *rxq);
+
+u16 hinic3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
+
+void hinic3_add_rq_to_rx_queue_list(struct hinic3_nic_dev *nic_dev,
+ u16 queue_id);
+
+int hinic3_refill_indir_rqid(struct hinic3_rxq *rxq);
+
+void hinic3_init_rx_queue_list(struct hinic3_nic_dev *nic_dev);
+
+void hinic3_remove_rq_from_rx_queue_list(struct hinic3_nic_dev *nic_dev,
+ u16 queue_id);
+int hinic3_start_all_rqs(struct rte_eth_dev *eth_dev);
+
+#ifdef HINIC3_XSTAT_RXBUF_INFO
+void hinic3_get_stats(struct hinic3_rxq *rxq);
+#endif
+
+/**
+ * Get receive queue local ci.
+ *
+ * @param[in] rxq
+ * Pointer to receive queue structure.
+ * @return
+ * Receive queue local ci.
+ */
+static inline u16
+hinic3_get_rq_local_ci(struct hinic3_rxq *rxq)
+{
+ return MASKED_QUEUE_IDX(rxq, rxq->cons_idx);
+}
+
+static inline u16
+hinic3_get_rq_free_wqebb(struct hinic3_rxq *rxq)
+{
+ return rxq->delta - 1;
+}
+
+/**
+ * Update receive queue local ci.
+ *
+ * @param[in] rxq
+ * Pointer to receive queue structure.
+ * @param[out] wqe_cnt
+ * Wqebb counters.
+ */
+static inline void
+hinic3_update_rq_local_ci(struct hinic3_rxq *rxq, u16 wqe_cnt)
+{
+ rxq->cons_idx += wqe_cnt;
+ rxq->delta += wqe_cnt;
+}
+
+#endif /* _HINIC3_RX_H_ */
diff --git a/drivers/net/hinic3/hinic3_tx.c b/drivers/net/hinic3/hinic3_tx.c
new file mode 100644
index 0000000000..6f8c42e0c3
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_tx.c
@@ -0,0 +1,274 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include <rte_ether.h>
+#include <rte_io.h>
+#include <rte_mbuf.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_nic_cfg.h"
+#include "base/hinic3_hwdev.h"
+#include "hinic3_nic_io.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_tx.h"
+
+#define HINIC3_TX_TASK_WRAPPED 1
+#define HINIC3_TX_BD_DESC_WRAPPED 2
+
+#define TX_MSS_DEFAULT 0x3E00
+#define TX_MSS_MIN 0x50
+
+#define HINIC3_MAX_TX_FREE_BULK 64
+
+#define MAX_PAYLOAD_OFFSET 221
+
+#define HINIC3_TX_OUTER_CHECKSUM_FLAG_SET 1
+#define HINIC3_TX_OUTER_CHECKSUM_FLAG_NO_SET 0
+
+#define HINIC3_TX_OFFLOAD_MASK \
+ (HINIC3_TX_CKSUM_OFFLOAD_MASK | HINIC3_PKT_TX_VLAN_PKT)
+
+#define HINIC3_TX_CKSUM_OFFLOAD_MASK \
+ (HINIC3_PKT_TX_IP_CKSUM | HINIC3_PKT_TX_TCP_CKSUM | \
+ HINIC3_PKT_TX_UDP_CKSUM | HINIC3_PKT_TX_SCTP_CKSUM | \
+ HINIC3_PKT_TX_OUTER_IP_CKSUM | HINIC3_PKT_TX_TCP_SEG)
+
+static inline u16
+hinic3_get_sq_free_wqebbs(struct hinic3_txq *sq)
+{
+ return ((sq->q_depth -
+ (((sq->prod_idx - sq->cons_idx) + sq->q_depth) & sq->q_mask)) -
+ 1);
+}
+
+static inline void
+hinic3_update_sq_local_ci(struct hinic3_txq *sq, u16 wqe_cnt)
+{
+ sq->cons_idx += wqe_cnt;
+}
+
+static inline u16
+hinic3_get_sq_local_ci(struct hinic3_txq *sq)
+{
+ return MASKED_QUEUE_IDX(sq, sq->cons_idx);
+}
+
+static inline u16
+hinic3_get_sq_hw_ci(struct hinic3_txq *sq)
+{
+ return MASKED_QUEUE_IDX(sq, hinic3_hw_cpu16(*sq->ci_vaddr_base));
+}
+
+int
+hinic3_start_all_sqs(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+ struct hinic3_txq *txq = NULL;
+ int i;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+
+ for (i = 0; i < nic_dev->num_sqs; i++) {
+ txq = eth_dev->data->tx_queues[i];
+ HINIC3_SET_TXQ_STARTED(txq);
+ eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return 0;
+}
+
+static inline void
+hinic3_free_cpy_mbuf(struct hinic3_nic_dev *nic_dev __rte_unused,
+ struct rte_mbuf *cpy_skb)
+{
+ rte_pktmbuf_free(cpy_skb);
+}
+
+/**
+ * Cleans up buffers (mbuf) in the send queue (txq) and returns these buffers to
+ * their memory pool.
+ *
+ * @param[in] txq
+ * Point to send queue.
+ * @param[in] free_cnt
+ * Number of mbufs to be released.
+ * @return
+ * Number of released mbufs.
+ */
+static int
+hinic3_xmit_mbuf_cleanup(struct hinic3_txq *txq, u32 free_cnt)
+{
+ struct hinic3_tx_info *tx_info = NULL;
+ struct rte_mbuf *mbuf = NULL;
+ struct rte_mbuf *mbuf_temp = NULL;
+ struct rte_mbuf *mbuf_free[HINIC3_MAX_TX_FREE_BULK];
+
+ int nb_free = 0;
+ int wqebb_cnt = 0;
+ u16 hw_ci, sw_ci, sq_mask;
+ u32 i;
+
+ hw_ci = hinic3_get_sq_hw_ci(txq);
+ sw_ci = hinic3_get_sq_local_ci(txq);
+ sq_mask = txq->q_mask;
+
+ for (i = 0; i < free_cnt; ++i) {
+ tx_info = &txq->tx_info[sw_ci];
+ if (hw_ci == sw_ci ||
+ (((hw_ci - sw_ci) & sq_mask) < tx_info->wqebb_cnt))
+ break;
+ /*
+ * The cpy_mbuf is usually used in the arge-sized package
+ * scenario.
+ */
+ if (unlikely(tx_info->cpy_mbuf != NULL)) {
+ hinic3_free_cpy_mbuf(txq->nic_dev, tx_info->cpy_mbuf);
+ tx_info->cpy_mbuf = NULL;
+ }
+ sw_ci = (sw_ci + tx_info->wqebb_cnt) & sq_mask;
+
+ wqebb_cnt += tx_info->wqebb_cnt;
+ mbuf = tx_info->mbuf;
+
+ if (likely(mbuf->nb_segs == 1)) {
+ mbuf_temp = rte_pktmbuf_prefree_seg(mbuf);
+ tx_info->mbuf = NULL;
+ if (unlikely(mbuf_temp == NULL))
+ continue;
+
+ mbuf_free[nb_free++] = mbuf_temp;
+ /*
+ * If the pools of different mbufs are different,
+ * release the mbufs of the same pool.
+ */
+ if (unlikely(mbuf_temp->pool != mbuf_free[0]->pool ||
+ nb_free >= HINIC3_MAX_TX_FREE_BULK)) {
+ rte_mempool_put_bulk(mbuf_free[0]->pool,
+ (void **)mbuf_free,
+ (nb_free - 1));
+ nb_free = 0;
+ mbuf_free[nb_free++] = mbuf_temp;
+ }
+ } else {
+ rte_pktmbuf_free(mbuf);
+ tx_info->mbuf = NULL;
+ }
+ }
+
+ if (nb_free > 0)
+ rte_mempool_put_bulk(mbuf_free[0]->pool, (void **)mbuf_free,
+ nb_free);
+
+ hinic3_update_sq_local_ci(txq, wqebb_cnt);
+
+ return i;
+}
+
+static inline void
+hinic3_tx_free_mbuf_force(struct hinic3_txq *txq __rte_unused,
+ struct rte_mbuf *mbuf)
+{
+ rte_pktmbuf_free(mbuf);
+}
+
+/**
+ * Release the mbuf and update the consumer index for sending queue.
+ *
+ * @param[in] txq
+ * Point to send queue.
+ */
+void
+hinic3_free_txq_mbufs(struct hinic3_txq *txq)
+{
+ struct hinic3_tx_info *tx_info = NULL;
+ u16 free_wqebbs;
+ u16 ci;
+
+ free_wqebbs = hinic3_get_sq_free_wqebbs(txq) + 1;
+
+ while (free_wqebbs < txq->q_depth) {
+ ci = hinic3_get_sq_local_ci(txq);
+
+ tx_info = &txq->tx_info[ci];
+ if (unlikely(tx_info->cpy_mbuf != NULL)) {
+ hinic3_free_cpy_mbuf(txq->nic_dev, tx_info->cpy_mbuf);
+ tx_info->cpy_mbuf = NULL;
+ }
+ hinic3_tx_free_mbuf_force(txq, tx_info->mbuf);
+ hinic3_update_sq_local_ci(txq, (u16)(tx_info->wqebb_cnt));
+
+ free_wqebbs = (u16)(free_wqebbs + tx_info->wqebb_cnt);
+ tx_info->mbuf = NULL;
+ }
+}
+
+void
+hinic3_free_all_txq_mbufs(struct hinic3_nic_dev *nic_dev)
+{
+ u16 qid;
+ for (qid = 0; qid < nic_dev->num_sqs; qid++)
+ hinic3_free_txq_mbufs(nic_dev->txqs[qid]);
+}
+
+int
+hinic3_tx_done_cleanup(void *txq, u32 free_cnt)
+{
+ struct hinic3_txq *tx_queue = txq;
+ u32 try_free_cnt = !free_cnt ? tx_queue->q_depth : free_cnt;
+
+ return hinic3_xmit_mbuf_cleanup(tx_queue, try_free_cnt);
+}
+
+int
+hinic3_stop_sq(struct hinic3_txq *txq)
+{
+ struct hinic3_nic_dev *nic_dev = txq->nic_dev;
+ unsigned long timeout;
+ int err = -EFAULT;
+ int free_wqebbs;
+
+ timeout = msecs_to_jiffies(HINIC3_FLUSH_QUEUE_TIMEOUT) + jiffies;
+ do {
+ hinic3_tx_done_cleanup(txq, 0);
+ free_wqebbs = hinic3_get_sq_free_wqebbs(txq) + 1;
+ if (free_wqebbs == txq->q_depth) {
+ err = 0;
+ break;
+ }
+
+ rte_delay_us(1);
+ } while (time_before(jiffies, timeout));
+
+ if (err)
+ PMD_DRV_LOG(WARNING,
+ "%s Wait sq empty timeout, queue_idx: %u, "
+ "sw_ci: %u, hw_ci: %u, sw_pi: %u, free_wqebbs: %u, "
+ "q_depth:%u",
+ nic_dev->dev_name, txq->q_id,
+ hinic3_get_sq_local_ci(txq),
+ hinic3_get_sq_hw_ci(txq),
+ MASKED_QUEUE_IDX(txq, txq->prod_idx), free_wqebbs,
+ txq->q_depth);
+
+ return err;
+}
+
+/**
+ * Stop all sending queues (SQs).
+ *
+ * @param[in] txq
+ * Point to send queue.
+ */
+void
+hinic3_flush_txqs(struct hinic3_nic_dev *nic_dev)
+{
+ u16 qid;
+ int err;
+
+ for (qid = 0; qid < nic_dev->num_sqs; qid++) {
+ err = hinic3_stop_sq(nic_dev->txqs[qid]);
+ if (err)
+ PMD_DRV_LOG(ERR, "Stop sq%d failed", qid);
+ }
+}
diff --git a/drivers/net/hinic3/hinic3_tx.h b/drivers/net/hinic3/hinic3_tx.h
new file mode 100644
index 0000000000..f4c61ea1b1
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_tx.h
@@ -0,0 +1,314 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_TX_H_
+#define _HINIC3_TX_H_
+
+#define MAX_SINGLE_SGE_SIZE 65536
+#define HINIC3_NONTSO_PKT_MAX_SGE 38 /**< non-tso max sge 38. */
+#define HINIC3_NONTSO_SEG_NUM_VALID(num) ((num) <= HINIC3_NONTSO_PKT_MAX_SGE)
+
+#define HINIC3_TSO_PKT_MAX_SGE 127 /**< tso max sge 127. */
+#define HINIC3_TSO_SEG_NUM_INVALID(num) ((num) > HINIC3_TSO_PKT_MAX_SGE)
+
+/* Tx offload info. */
+struct hinic3_tx_offload_info {
+ u8 outer_l2_len;
+ u8 outer_l3_type;
+ u16 outer_l3_len;
+
+ u8 inner_l2_len;
+ u8 inner_l3_type;
+ u16 inner_l3_len;
+
+ u8 tunnel_length;
+ u8 tunnel_type;
+ u8 inner_l4_type;
+ u8 inner_l4_len;
+
+ u16 payload_offset;
+ u8 inner_l4_tcp_udp;
+ u8 rsvd0; /**< Reserved field. */
+};
+
+/* Tx wqe ctx. */
+struct hinic3_wqe_info {
+ u8 around; /**< Indicates whether the WQE is bypassed. */
+ u8 cpy_mbuf_cnt;
+ u16 sge_cnt;
+
+ u8 offload;
+ u8 rsvd0; /**< Reserved field 0. */
+ u16 payload_offset;
+
+ u8 wrapped;
+ u8 owner;
+ u16 pi;
+
+ u16 wqebb_cnt;
+ u16 rsvd1; /**< Reserved field 1. */
+
+ u32 queue_info;
+};
+
+/* Descriptor for the send queue of wqe. */
+struct hinic3_sq_wqe_desc {
+ u32 ctrl_len;
+ u32 queue_info;
+ u32 hi_addr;
+ u32 lo_addr;
+};
+
+/* Describes the send queue task. */
+struct hinic3_sq_task {
+ u32 pkt_info0;
+ u32 ip_identify;
+ u32 pkt_info2;
+ u32 vlan_offload;
+};
+
+/* Descriptor that describes the transmit queue buffer. */
+struct hinic3_sq_bufdesc {
+ u32 len; /**< 31-bits Length, L2NIC only use length[17:0]. */
+ u32 rsvd; /**< Reserved field. */
+ u32 hi_addr; /**< Upper address. */
+ u32 lo_addr; /**< Lower address. */
+};
+
+/* Compact work queue entry that describes the send queue (SQ). */
+struct hinic3_sq_compact_wqe {
+ struct hinic3_sq_wqe_desc wqe_desc;
+};
+
+/* Extend work queue entry that describes the send queue (SQ). */
+struct hinic3_sq_extend_wqe {
+ struct hinic3_sq_wqe_desc wqe_desc;
+ struct hinic3_sq_task task;
+ struct hinic3_sq_bufdesc buf_desc[];
+};
+
+struct hinic3_sq_wqe {
+ union {
+ struct hinic3_sq_compact_wqe compact_wqe;
+ struct hinic3_sq_extend_wqe extend_wqe;
+ };
+};
+
+struct hinic3_sq_wqe_combo {
+ struct hinic3_sq_wqe_desc *hdr;
+ struct hinic3_sq_task *task;
+ struct hinic3_sq_bufdesc *bds_head;
+ u32 wqe_type;
+ u32 task_type;
+};
+
+enum sq_wqe_data_format {
+ SQ_NORMAL_WQE = 0,
+};
+
+/* Indicates the type of a WQE. */
+enum sq_wqe_ec_type {
+ SQ_WQE_COMPACT_TYPE = 0,
+ SQ_WQE_EXTENDED_TYPE = 1,
+};
+
+#define COMPACT_WQE_MAX_CTRL_LEN 0x3FFF
+
+/* Indicates the type of tasks with different lengths. */
+enum sq_wqe_tasksect_len_type {
+ SQ_WQE_TASKSECT_46BITS = 0,
+ SQ_WQE_TASKSECT_16BYTES = 1,
+};
+
+/** Setting and obtaining queue information */
+#define SQ_CTRL_BD0_LEN_SHIFT 0
+#define SQ_CTRL_RSVD_SHIFT 18
+#define SQ_CTRL_BUFDESC_NUM_SHIFT 19
+#define SQ_CTRL_TASKSECT_LEN_SHIFT 27
+#define SQ_CTRL_DATA_FORMAT_SHIFT 28
+#define SQ_CTRL_DIRECT_SHIFT 29
+#define SQ_CTRL_EXTENDED_SHIFT 30
+#define SQ_CTRL_OWNER_SHIFT 31
+
+#define SQ_CTRL_BD0_LEN_MASK 0x3FFFFU
+#define SQ_CTRL_RSVD_MASK 0x1U
+#define SQ_CTRL_BUFDESC_NUM_MASK 0xFFU
+#define SQ_CTRL_TASKSECT_LEN_MASK 0x1U
+#define SQ_CTRL_DATA_FORMAT_MASK 0x1U
+#define SQ_CTRL_DIRECT_MASK 0x1U
+#define SQ_CTRL_EXTENDED_MASK 0x1U
+#define SQ_CTRL_OWNER_MASK 0x1U
+
+#define SQ_CTRL_SET(val, member) \
+ (((u32)(val) & SQ_CTRL_##member##_MASK) << SQ_CTRL_##member##_SHIFT)
+#define SQ_CTRL_GET(val, member) \
+ (((val) >> SQ_CTRL_##member##_SHIFT) & SQ_CTRL_##member##_MASK)
+#define SQ_CTRL_CLEAR(val, member) \
+ ((val) & (~(SQ_CTRL_##member##_MASK << SQ_CTRL_##member##_SHIFT)))
+
+#define SQ_CTRL_QUEUE_INFO_PKT_TYPE_SHIFT 0
+#define SQ_CTRL_QUEUE_INFO_PLDOFF_SHIFT 2
+#define SQ_CTRL_QUEUE_INFO_UFO_SHIFT 10
+#define SQ_CTRL_QUEUE_INFO_TSO_SHIFT 11
+#define SQ_CTRL_QUEUE_INFO_TCPUDP_CS_SHIFT 12
+#define SQ_CTRL_QUEUE_INFO_MSS_SHIFT 13
+#define SQ_CTRL_QUEUE_INFO_SCTP_SHIFT 27
+#define SQ_CTRL_QUEUE_INFO_UC_SHIFT 28
+#define SQ_CTRL_QUEUE_INFO_PRI_SHIFT 29
+
+#define SQ_CTRL_QUEUE_INFO_PKT_TYPE_MASK 0x3U
+#define SQ_CTRL_QUEUE_INFO_PLDOFF_MASK 0xFFU
+#define SQ_CTRL_QUEUE_INFO_UFO_MASK 0x1U
+#define SQ_CTRL_QUEUE_INFO_TSO_MASK 0x1U
+#define SQ_CTRL_QUEUE_INFO_TCPUDP_CS_MASK 0x1U
+#define SQ_CTRL_QUEUE_INFO_MSS_MASK 0x3FFFU
+#define SQ_CTRL_QUEUE_INFO_SCTP_MASK 0x1U
+#define SQ_CTRL_QUEUE_INFO_UC_MASK 0x1U
+#define SQ_CTRL_QUEUE_INFO_PRI_MASK 0x7U
+
+#define SQ_CTRL_QUEUE_INFO_SET(val, member) \
+ (((u32)(val) & SQ_CTRL_QUEUE_INFO_##member##_MASK) \
+ << SQ_CTRL_QUEUE_INFO_##member##_SHIFT)
+#define SQ_CTRL_QUEUE_INFO_GET(val, member) \
+ (((val) >> SQ_CTRL_QUEUE_INFO_##member##_SHIFT) & \
+ SQ_CTRL_QUEUE_INFO_##member##_MASK)
+#define SQ_CTRL_QUEUE_INFO_CLEAR(val, member) \
+ ((val) & (~(SQ_CTRL_QUEUE_INFO_##member##_MASK \
+ << SQ_CTRL_QUEUE_INFO_##member##_SHIFT)))
+
+/* Setting and obtaining task information */
+#define SQ_TASK_INFO0_TUNNEL_FLAG_SHIFT 19
+#define SQ_TASK_INFO0_ESP_NEXT_PROTO_SHIFT 22
+#define SQ_TASK_INFO0_INNER_L4_EN_SHIFT 24
+#define SQ_TASK_INFO0_INNER_L3_EN_SHIFT 25
+#define SQ_TASK_INFO0_INNER_L4_PSEUDO_SHIFT 26
+#define SQ_TASK_INFO0_OUT_L4_EN_SHIFT 27
+#define SQ_TASK_INFO0_OUT_L3_EN_SHIFT 28
+#define SQ_TASK_INFO0_OUT_L4_PSEUDO_SHIFT 29
+#define SQ_TASK_INFO0_ESP_OFFLOAD_SHIFT 30
+#define SQ_TASK_INFO0_IPSEC_PROTO_SHIFT 31
+
+#define SQ_TASK_INFO0_TUNNEL_FLAG_MASK 0x1U
+#define SQ_TASK_INFO0_ESP_NEXT_PROTO_MASK 0x3U
+#define SQ_TASK_INFO0_INNER_L4_EN_MASK 0x1U
+#define SQ_TASK_INFO0_INNER_L3_EN_MASK 0x1U
+#define SQ_TASK_INFO0_INNER_L4_PSEUDO_MASK 0x1U
+#define SQ_TASK_INFO0_OUT_L4_EN_MASK 0x1U
+#define SQ_TASK_INFO0_OUT_L3_EN_MASK 0x1U
+#define SQ_TASK_INFO0_OUT_L4_PSEUDO_MASK 0x1U
+#define SQ_TASK_INFO0_ESP_OFFLOAD_MASK 0x1U
+#define SQ_TASK_INFO0_IPSEC_PROTO_MASK 0x1U
+
+#define SQ_TASK_INFO0_SET(val, member) \
+ (((u32)(val) & SQ_TASK_INFO0_##member##_MASK) \
+ << SQ_TASK_INFO0_##member##_SHIFT)
+#define SQ_TASK_INFO0_GET(val, member) \
+ (((val) >> SQ_TASK_INFO0_##member##_SHIFT) & \
+ SQ_TASK_INFO0_##member##_MASK)
+
+#define SQ_TASK_INFO1_SET(val, member) \
+ (((val) & SQ_TASK_INFO1_##member##_MASK) \
+ << SQ_TASK_INFO1_##member##_SHIFT)
+#define SQ_TASK_INFO1_GET(val, member) \
+ (((val) >> SQ_TASK_INFO1_##member##_SHIFT) & \
+ SQ_TASK_INFO1_##member##_MASK)
+
+#define SQ_TASK_INFO3_VLAN_TAG_SHIFT 0
+#define SQ_TASK_INFO3_VLAN_TYPE_SHIFT 16
+#define SQ_TASK_INFO3_VLAN_TAG_VALID_SHIFT 19
+
+#define SQ_TASK_INFO3_VLAN_TAG_MASK 0xFFFFU
+#define SQ_TASK_INFO3_VLAN_TYPE_MASK 0x7U
+#define SQ_TASK_INFO3_VLAN_TAG_VALID_MASK 0x1U
+
+#define SQ_TASK_INFO3_SET(val, member) \
+ (((val) & SQ_TASK_INFO3_##member##_MASK) \
+ << SQ_TASK_INFO3_##member##_SHIFT)
+#define SQ_TASK_INFO3_GET(val, member) \
+ (((val) >> SQ_TASK_INFO3_##member##_SHIFT) & \
+ SQ_TASK_INFO3_##member##_MASK)
+
+/* Defines the TX queue status. */
+enum hinic3_txq_status {
+ HINIC3_TXQ_STATUS_START = 0,
+ HINIC3_TXQ_STATUS_STOP,
+};
+
+/* Setting and obtaining status information. */
+#define HINIC3_TXQ_IS_STARTED(txq) ((txq)->status == HINIC3_TXQ_STATUS_START)
+#define HINIC3_TXQ_IS_STOPPED(txq) ((txq)->status == HINIC3_TXQ_STATUS_STOP)
+#define HINIC3_SET_TXQ_STARTED(txq) ((txq)->status = HINIC3_TXQ_STATUS_START)
+#define HINIC3_SET_TXQ_STOPPED(txq) ((txq)->status = HINIC3_TXQ_STATUS_STOP)
+
+#define HINIC3_FLUSH_QUEUE_TIMEOUT 3000
+
+/* Txq info. */
+struct hinic3_txq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 tx_busy;
+ u64 offload_errors;
+ u64 burst_pkts;
+ u64 sge_len0;
+ u64 mbuf_null;
+ u64 cpy_pkts;
+ u64 sge_len_too_large;
+
+#ifdef HINIC3_XSTAT_PROF_TX
+ u64 app_tsc;
+ u64 pmd_tsc;
+#endif
+
+#ifdef HINIC3_XSTAT_MBUF_USE
+ u64 tx_left_mbuf_bytes;
+#endif
+};
+
+/* Structure for storing the information sent. */
+struct hinic3_tx_info {
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *cpy_mbuf;
+ int wqebb_cnt;
+};
+
+/* Indicates the sending queue of information. */
+struct __rte_cache_aligned hinic3_txq {
+ struct hinic3_nic_dev *nic_dev;
+ u16 q_id;
+ u16 q_depth;
+ u16 q_mask;
+ u16 wqebb_size;
+ u16 wqebb_shift;
+ u16 cons_idx;
+ u16 prod_idx;
+ u16 status;
+
+ u16 tx_free_thresh;
+ u16 owner;
+ void *db_addr;
+ struct hinic3_tx_info *tx_info;
+
+ const struct rte_memzone *sq_mz;
+ void *queue_buf_vaddr;
+ rte_iova_t queue_buf_paddr;
+
+ const struct rte_memzone *ci_mz;
+ volatile u16 *ci_vaddr_base;
+ rte_iova_t ci_dma_base;
+ u64 sq_head_addr;
+ u64 sq_bot_sge_addr;
+ u32 cos;
+ struct hinic3_txq_stats txq_stats;
+#ifdef HINIC3_XSTAT_PROF_TX
+ uint64_t prof_tx_end_tsc;
+#endif
+};
+
+void hinic3_flush_txqs(struct hinic3_nic_dev *nic_dev);
+void hinic3_free_txq_mbufs(struct hinic3_txq *txq);
+void hinic3_free_all_txq_mbufs(struct hinic3_nic_dev *nic_dev);
+int hinic3_stop_sq(struct hinic3_txq *txq);
+int hinic3_start_all_sqs(struct rte_eth_dev *eth_dev);
+int hinic3_tx_done_cleanup(void *txq, uint32_t free_cnt);
+#endif /**< _HINIC3_TX_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 14/18] net/hinic3: add Rx/Tx functions
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (4 preceding siblings ...)
2025-04-18 7:02 ` [RFC 13/18] net/hinic3: add dev ops Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 15/18] net/hinic3: add MML and EEPROM access feature Feifei Wang
` (4 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Feifei Wang, Yi Chen, Xin Wang
From: Feifei Wang <wangfeifei40@huawei.com>
This patch add package sending and receiving function codes.
Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
Signed-off-by: Yi Chen <chenyi221@huawei.com>
Reviewed-by: Xin Wang <wangxin679@h-partners.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 9 +-
drivers/net/hinic3/hinic3_rx.c | 301 +++++++++++-
drivers/net/hinic3/hinic3_tx.c | 754 +++++++++++++++++++++++++++++
drivers/net/hinic3/hinic3_tx.h | 1 +
4 files changed, 1054 insertions(+), 11 deletions(-)
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index de380dddbb..7cd101e5c3 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -21,9 +21,9 @@
#include "base/hinic3_hw_comm.h"
#include "base/hinic3_nic_cfg.h"
#include "base/hinic3_nic_event.h"
-#include "hinic3_pmd_nic_io.h"
-#include "hinic3_pmd_tx.h"
-#include "hinic3_pmd_rx.h"
+#include "hinic3_nic_io.h"
+#include "hinic3_tx.h"
+#include "hinic3_rx.h"
#include "hinic3_ethdev.h"
#define HINIC3_MIN_RX_BUF_SIZE 1024
@@ -3337,6 +3337,9 @@ hinic3_dev_init(struct rte_eth_dev *eth_dev)
PMD_DRV_LOG(INFO, "Network Interface pmd driver version: %s",
HINIC3_PMD_DRV_VERSION);
+ eth_dev->rx_pkt_burst = hinic3_recv_pkts;
+ eth_dev->tx_pkt_burst = hinic3_xmit_pkts;
+
return hinic3_func_init(eth_dev);
}
diff --git a/drivers/net/hinic3/hinic3_rx.c b/drivers/net/hinic3/hinic3_rx.c
index a1dc960236..318d9aadc3 100644
--- a/drivers/net/hinic3/hinic3_rx.c
+++ b/drivers/net/hinic3/hinic3_rx.c
@@ -5,14 +5,14 @@
#include <rte_mbuf.h>
#include "base/hinic3_compat.h"
-#include "base/hinic3_pmd_hwif.h"
-#include "base/hinic3_pmd_hwdev.h"
-#include "base/hinic3_pmd_wq.h"
-#include "base/hinic3_pmd_nic_cfg.h"
-#include "hinic3_pmd_nic_io.h"
-#include "hinic3_pmd_ethdev.h"
-#include "hinic3_pmd_tx.h"
-#include "hinic3_pmd_rx.h"
+#include "base/hinic3_hwif.h"
+#include "base/hinic3_hwdev.h"
+#include "base/hinic3_wq.h"
+#include "base/hinic3_nic_cfg.h"
+#include "hinic3_nic_io.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_tx.h"
+#include "hinic3_rx.h"
/**
* Get wqe from receive queue.
@@ -809,3 +809,288 @@ hinic3_start_rq(struct rte_eth_dev *eth_dev, struct hinic3_rxq *rxq)
return err;
}
+
+
+static inline u64
+hinic3_rx_vlan(u32 offload_type, u32 vlan_len, u16 *vlan_tci)
+{
+ uint16_t vlan_tag;
+
+ vlan_tag = HINIC3_GET_RX_VLAN_TAG(vlan_len);
+ if (!HINIC3_GET_RX_VLAN_OFFLOAD_EN(offload_type) || vlan_tag == 0) {
+ *vlan_tci = 0;
+ return 0;
+ }
+
+ *vlan_tci = vlan_tag;
+
+ return HINIC3_PKT_RX_VLAN | HINIC3_PKT_RX_VLAN_STRIPPED;
+}
+
+static inline u64
+hinic3_rx_csum(uint32_t status, struct hinic3_rxq *rxq)
+{
+ struct hinic3_nic_dev *nic_dev = rxq->nic_dev;
+ u32 csum_err;
+ u64 flags;
+
+ if (unlikely(!(nic_dev->rx_csum_en & HINIC3_DEFAULT_RX_CSUM_OFFLOAD)))
+ return HINIC3_PKT_RX_IP_CKSUM_UNKNOWN;
+
+ csum_err = HINIC3_GET_RX_CSUM_ERR(status);
+ if (likely(csum_err == 0))
+ return (HINIC3_PKT_RX_IP_CKSUM_GOOD |
+ HINIC3_PKT_RX_L4_CKSUM_GOOD);
+
+ /*
+ * If bypass bit is set, all other err status indications should be
+ * ignored.
+ */
+ if (unlikely(csum_err & HINIC3_RX_CSUM_HW_CHECK_NONE))
+ return HINIC3_PKT_RX_IP_CKSUM_UNKNOWN;
+
+ flags = 0;
+
+ /* IP checksum error. */
+ if (csum_err & HINIC3_RX_CSUM_IP_CSUM_ERR) {
+ flags |= HINIC3_PKT_RX_IP_CKSUM_BAD;
+ rxq->rxq_stats.csum_errors++;
+ }
+
+ /* L4 checksum error. */
+ if ((csum_err & HINIC3_RX_CSUM_TCP_CSUM_ERR) ||
+ (csum_err & HINIC3_RX_CSUM_UDP_CSUM_ERR) ||
+ (csum_err & HINIC3_RX_CSUM_SCTP_CRC_ERR)) {
+ flags |= HINIC3_PKT_RX_L4_CKSUM_BAD;
+ rxq->rxq_stats.csum_errors++;
+ }
+
+ if (unlikely(csum_err == HINIC3_RX_CSUM_IPSU_OTHER_ERR))
+ rxq->rxq_stats.other_errors++;
+
+ return flags;
+}
+
+static inline u64
+hinic3_rx_rss_hash(u32 offload_type, u32 rss_hash_value, u32 *rss_hash)
+{
+ u32 rss_type;
+
+ rss_type = HINIC3_GET_RSS_TYPES(offload_type);
+ if (likely(rss_type != 0)) {
+ *rss_hash = rss_hash_value;
+ return HINIC3_PKT_RX_RSS_HASH;
+ }
+
+ return 0;
+}
+
+static void
+hinic3_recv_jumbo_pkt(struct hinic3_rxq *rxq, struct rte_mbuf *head_mbuf,
+ u32 remain_pkt_len)
+{
+ struct rte_mbuf *cur_mbuf = NULL;
+ struct rte_mbuf *rxm = NULL;
+ struct hinic3_rx_info *rx_info = NULL;
+ u16 sw_ci, rx_buf_len = rxq->buf_len;
+ u32 pkt_len;
+
+ while (remain_pkt_len > 0) {
+ sw_ci = hinic3_get_rq_local_ci(rxq);
+ rx_info = &rxq->rx_info[sw_ci];
+
+ hinic3_update_rq_local_ci(rxq, 1);
+
+ pkt_len = remain_pkt_len > rx_buf_len ? rx_buf_len
+ : remain_pkt_len;
+ remain_pkt_len -= pkt_len;
+
+ cur_mbuf = rx_info->mbuf;
+ cur_mbuf->data_len = (u16)pkt_len;
+ cur_mbuf->next = NULL;
+
+ head_mbuf->pkt_len += cur_mbuf->data_len;
+ head_mbuf->nb_segs++;
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_free_mbuf_bytes++;
+#endif
+ if (!rxm)
+ head_mbuf->next = cur_mbuf;
+ else
+ rxm->next = cur_mbuf;
+
+ rxm = cur_mbuf;
+ }
+}
+
+int
+hinic3_start_all_rqs(struct rte_eth_dev *eth_dev)
+{
+ struct hinic3_nic_dev *nic_dev = NULL;
+ struct hinic3_rxq *rxq = NULL;
+ int err = 0;
+ int i;
+
+ nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(eth_dev);
+
+ for (i = 0; i < nic_dev->num_rqs; i++) {
+ rxq = eth_dev->data->rx_queues[i];
+ hinic3_add_rq_to_rx_queue_list(nic_dev, rxq->q_id);
+ err = hinic3_rearm_rxq_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Fail to alloc mbuf for Rx queue %d, "
+ "qid = %u, need_mbuf: %d",
+ i, rxq->q_id, rxq->q_depth);
+ goto out;
+ }
+ hinic3_dev_rx_queue_intr_enable(eth_dev, rxq->q_id);
+ eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ if (nic_dev->rss_state == HINIC3_RSS_ENABLE) {
+ err = hinic3_refill_indir_rqid(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Refill rq to indrect table failed, "
+ "eth_dev:%s, queue_idx:%d, err:%d",
+ rxq->nic_dev->dev_name, rxq->q_id, err);
+ goto out;
+ }
+ }
+
+ return 0;
+out:
+ for (i = 0; i < nic_dev->num_rqs; i++) {
+ rxq = eth_dev->data->rx_queues[i];
+ hinic3_remove_rq_from_rx_queue_list(nic_dev, rxq->q_id);
+ hinic3_free_rxq_mbufs(rxq);
+ hinic3_dev_rx_queue_intr_disable(eth_dev, rxq->q_id);
+ eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+ }
+ return err;
+}
+
+#define HINIC3_RX_EMPTY_THRESHOLD 3
+u16
+hinic3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ struct hinic3_rxq *rxq = rx_queue;
+ struct hinic3_rx_info *rx_info = NULL;
+ volatile struct hinic3_rq_cqe *rx_cqe = NULL;
+ struct rte_mbuf *rxm = NULL;
+ u16 sw_ci, rx_buf_len, wqebb_cnt = 0, pkts = 0;
+ u32 status, pkt_len, vlan_len, offload_type, lro_num;
+ u64 rx_bytes = 0;
+ u32 hash_value;
+
+#ifdef HINIC3_XSTAT_PROF_RX
+ uint64_t t1 = rte_get_tsc_cycles();
+ uint64_t t2;
+#endif
+ if (((rte_get_timer_cycles() - rxq->rxq_stats.tsc) < rxq->wait_time_cycle) &&
+ rxq->rxq_stats.empty >= HINIC3_RX_EMPTY_THRESHOLD)
+ goto out;
+
+ sw_ci = hinic3_get_rq_local_ci(rxq);
+ rx_buf_len = rxq->buf_len;
+
+ while (pkts < nb_pkts) {
+ rx_cqe = &rxq->rx_cqe[sw_ci];
+ status = hinic3_hw_cpu32((u32)(rte_atomic_load_explicit(&rx_cqe->status,
+ rte_memory_order_acquire)));
+ if (!HINIC3_GET_RX_DONE(status)) {
+ rxq->rxq_stats.empty++;
+ break;
+ }
+
+ vlan_len = hinic3_hw_cpu32(rx_cqe->vlan_len);
+
+ pkt_len = HINIC3_GET_RX_PKT_LEN(vlan_len);
+
+ rx_info = &rxq->rx_info[sw_ci];
+ rxm = rx_info->mbuf;
+
+ /* 1. Next ci point and prefetch. */
+ sw_ci++;
+ sw_ci &= rxq->q_mask;
+
+ /* 2. Prefetch next mbuf first 64B. */
+ rte_prefetch0(rxq->rx_info[sw_ci].mbuf);
+
+ /* 3. Jumbo frame process. */
+ if (likely(pkt_len <= (u32)rx_buf_len)) {
+ rxm->data_len = (u16)pkt_len;
+ rxm->pkt_len = pkt_len;
+ wqebb_cnt++;
+ } else {
+ rxm->data_len = rx_buf_len;
+ rxm->pkt_len = rx_buf_len;
+
+ /*
+ * If receive jumbo, updating ci will be done by
+ * hinic3_recv_jumbo_pkt function.
+ */
+ hinic3_update_rq_local_ci(rxq, wqebb_cnt + 1);
+ wqebb_cnt = 0;
+ hinic3_recv_jumbo_pkt(rxq, rxm, pkt_len - rx_buf_len);
+ sw_ci = hinic3_get_rq_local_ci(rxq);
+ }
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->port = rxq->port_id;
+
+ /* 4. Rx checksum offload. */
+ rxm->ol_flags |= hinic3_rx_csum(status, rxq);
+
+ /* 5. Vlan offload. */
+ offload_type = hinic3_hw_cpu32(rx_cqe->offload_type);
+
+ rxm->ol_flags |=
+ hinic3_rx_vlan(offload_type, vlan_len, &rxm->vlan_tci);
+
+ /* 6. RSS. */
+ hash_value = hinic3_hw_cpu32(rx_cqe->hash_val);
+ rxm->ol_flags |= hinic3_rx_rss_hash(offload_type, hash_value,
+ &rxm->hash.rss);
+ /* 8. LRO. */
+ lro_num = HINIC3_GET_RX_NUM_LRO(status);
+ if (unlikely(lro_num != 0)) {
+ rxm->ol_flags |= HINIC3_PKT_RX_LRO;
+ rxm->tso_segsz = pkt_len / lro_num;
+ }
+
+ rx_cqe->status = 0;
+
+ rx_bytes += pkt_len;
+ rx_pkts[pkts++] = rxm;
+ }
+
+ if (pkts) {
+ /* 9. Update local ci. */
+ hinic3_update_rq_local_ci(rxq, wqebb_cnt);
+
+ /* Update packet stats. */
+ rxq->rxq_stats.packets += pkts;
+ rxq->rxq_stats.bytes += rx_bytes;
+ rxq->rxq_stats.empty = 0;
+#ifdef HINIC3_XSTAT_MBUF_USE
+ rxq->rxq_stats.rx_free_mbuf_bytes += pkts;
+#endif
+ }
+ rxq->rxq_stats.burst_pkts = pkts;
+ rxq->rxq_stats.tsc = rte_get_timer_cycles();
+out:
+ /* 10. Rearm mbuf to rxq. */
+ hinic3_rearm_rxq_mbuf(rxq);
+
+#ifdef HINIC3_XSTAT_PROF_RX
+ /* Do profiling stats. */
+ t2 = rte_get_tsc_cycles();
+ rxq->rxq_stats.app_tsc = t1 - rxq->prof_rx_end_tsc;
+ rxq->prof_rx_end_tsc = t2;
+ rxq->rxq_stats.pmd_tsc = t2 - t1;
+#endif
+
+ return pkts;
+}
diff --git a/drivers/net/hinic3/hinic3_tx.c b/drivers/net/hinic3/hinic3_tx.c
index 6f8c42e0c3..c2157ab4b9 100644
--- a/drivers/net/hinic3/hinic3_tx.c
+++ b/drivers/net/hinic3/hinic3_tx.c
@@ -60,6 +60,98 @@ hinic3_get_sq_hw_ci(struct hinic3_txq *sq)
return MASKED_QUEUE_IDX(sq, hinic3_hw_cpu16(*sq->ci_vaddr_base));
}
+static void *
+hinic3_get_sq_wqe(struct hinic3_txq *sq, struct hinic3_wqe_info *wqe_info)
+{
+ u16 cur_pi = MASKED_QUEUE_IDX(sq, sq->prod_idx);
+ u32 end_pi;
+
+ end_pi = cur_pi + wqe_info->wqebb_cnt;
+ sq->prod_idx += wqe_info->wqebb_cnt;
+
+ wqe_info->owner = (u8)(sq->owner);
+ wqe_info->pi = cur_pi;
+ wqe_info->wrapped = 0;
+
+ if (unlikely(end_pi >= sq->q_depth)) {
+ sq->owner = !sq->owner;
+
+ if (likely(end_pi > sq->q_depth))
+ wqe_info->wrapped = (u8)(sq->q_depth - cur_pi);
+ }
+
+ return NIC_WQE_ADDR(sq, cur_pi);
+}
+
+static inline void
+hinic3_put_sq_wqe(struct hinic3_txq *sq, struct hinic3_wqe_info *wqe_info)
+{
+ if (wqe_info->owner != sq->owner)
+ sq->owner = wqe_info->owner;
+
+ sq->prod_idx -= wqe_info->wqebb_cnt;
+}
+
+/**
+ * Sets the WQE combination information in the transmit queue (SQ).
+ *
+ * @param[in] txq
+ * Point to send queue.
+ * @param[out] wqe_combo
+ * Point to wqe_combo of send queue(SQ).
+ * @param[in] wqe
+ * Point to wqe of send queue(SQ).
+ * @param[in] wqe_info
+ * Point to wqe_info of send queue(SQ).
+ */
+static void
+hinic3_set_wqe_combo(struct hinic3_txq *txq,
+ struct hinic3_sq_wqe_combo *wqe_combo,
+ struct hinic3_sq_wqe *wqe,
+ struct hinic3_wqe_info *wqe_info)
+{
+ wqe_combo->hdr = &wqe->compact_wqe.wqe_desc;
+
+ if (wqe_info->offload) {
+ if (wqe_info->wrapped == HINIC3_TX_TASK_WRAPPED) {
+ wqe_combo->task = (struct hinic3_sq_task *)
+ (void *)txq->sq_head_addr;
+ wqe_combo->bds_head = (struct hinic3_sq_bufdesc *)
+ (void *)(txq->sq_head_addr + txq->wqebb_size);
+ } else if (wqe_info->wrapped == HINIC3_TX_BD_DESC_WRAPPED) {
+ wqe_combo->task = &wqe->extend_wqe.task;
+ wqe_combo->bds_head = (struct hinic3_sq_bufdesc *)
+ (void *)(txq->sq_head_addr);
+ } else {
+ wqe_combo->task = &wqe->extend_wqe.task;
+ wqe_combo->bds_head = wqe->extend_wqe.buf_desc;
+ }
+
+ wqe_combo->wqe_type = SQ_WQE_EXTENDED_TYPE;
+ wqe_combo->task_type = SQ_WQE_TASKSECT_16BYTES;
+
+ return;
+ }
+
+ if (wqe_info->wrapped == HINIC3_TX_TASK_WRAPPED) {
+ wqe_combo->bds_head = (struct hinic3_sq_bufdesc *)
+ (void *)(txq->sq_head_addr);
+ } else {
+ wqe_combo->bds_head =
+ (struct hinic3_sq_bufdesc *)(&wqe->extend_wqe.task);
+ }
+
+ if (wqe_info->wqebb_cnt > 1) {
+ wqe_combo->wqe_type = SQ_WQE_EXTENDED_TYPE;
+ wqe_combo->task_type = SQ_WQE_TASKSECT_46BITS;
+
+ /* This section used as vlan insert, needs to clear. */
+ wqe_combo->bds_head->rsvd = 0;
+ } else {
+ wqe_combo->wqe_type = SQ_WQE_COMPACT_TYPE;
+ }
+}
+
int
hinic3_start_all_sqs(struct rte_eth_dev *eth_dev)
{
@@ -220,6 +312,668 @@ hinic3_tx_done_cleanup(void *txq, u32 free_cnt)
return hinic3_xmit_mbuf_cleanup(tx_queue, try_free_cnt);
}
+/**
+ * Prepare the data packet to be sent and calculate the internal L3 offset.
+ *
+ * @param[in] mbuf
+ * Point to the mbuf to be processed.
+ * @param[out] inner_l3_offset
+ * Inner(IP Layer) L3 layer offset.
+ * @return
+ * 0 as success, -EINVAL as failure.
+ */
+static int
+hinic3_tx_offload_pkt_prepare(struct rte_mbuf *mbuf, u16 *inner_l3_offset)
+{
+ uint64_t ol_flags = mbuf->ol_flags;
+
+ /* Only support vxlan offload. */
+ if ((ol_flags & HINIC3_PKT_TX_TUNNEL_MASK) &&
+ (!(ol_flags & HINIC3_PKT_TX_TUNNEL_VXLAN)))
+ return -EINVAL;
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ if (rte_validate_tx_offload(mbuf) != 0)
+ return -EINVAL;
+#endif
+ /* Support tunnel. */
+ if ((ol_flags & HINIC3_PKT_TX_TUNNEL_MASK)) {
+ if ((ol_flags & HINIC3_PKT_TX_OUTER_IP_CKSUM) ||
+ (ol_flags & HINIC3_PKT_TX_OUTER_IPV6) ||
+ (ol_flags & HINIC3_PKT_TX_TCP_SEG)) {
+ /*
+ * For this senmatic, l2_len of mbuf means
+ * len(out_udp + vxlan + in_eth).
+ */
+ *inner_l3_offset = mbuf->l2_len + mbuf->outer_l2_len +
+ mbuf->outer_l3_len;
+ } else {
+ /*
+ * For this senmatic, l2_len of mbuf means
+ * len(out_eth + out_ip + out_udp + vxlan + in_eth).
+ */
+ *inner_l3_offset = mbuf->l2_len;
+ }
+ } else {
+ /* For non-tunnel type pkts. */
+ *inner_l3_offset = mbuf->l2_len;
+ }
+
+ return 0;
+}
+
+static inline void
+hinic3_set_vlan_tx_offload(struct hinic3_sq_task *task, u16 vlan_tag,
+ u8 vlan_type)
+{
+ task->vlan_offload = SQ_TASK_INFO3_SET(vlan_tag, VLAN_TAG) |
+ SQ_TASK_INFO3_SET(vlan_type, VLAN_TYPE) |
+ SQ_TASK_INFO3_SET(1U, VLAN_TAG_VALID);
+}
+
+/**
+ * Set the corresponding offload information based on ol_flags of the mbuf.
+ *
+ * @param[in] mbuf
+ * Point to the mbuf for which offload needs to be set in the sending queue.
+ * @param[out] task
+ * Point to task of send queue(SQ).
+ * @param[out] wqe_info
+ * Point to wqe_info of send queue(SQ).
+ * @return
+ * 0 as success, -EINVAL as failure.
+ */
+static int
+hinic3_set_tx_offload(struct rte_mbuf *mbuf, struct hinic3_sq_task *task,
+ struct hinic3_wqe_info *wqe_info)
+{
+ uint64_t ol_flags = mbuf->ol_flags;
+ u16 pld_offset = 0;
+ u32 queue_info = 0;
+ u16 vlan_tag;
+
+ task->pkt_info0 = 0;
+ task->ip_identify = 0;
+ task->pkt_info2 = 0;
+ task->vlan_offload = 0;
+
+ /* Vlan offload. */
+ if (unlikely(ol_flags & HINIC3_PKT_TX_VLAN_PKT)) {
+ vlan_tag = mbuf->vlan_tci;
+ hinic3_set_vlan_tx_offload(task, vlan_tag, HINIC3_TX_TPID0);
+ task->vlan_offload = hinic3_hw_be32(task->vlan_offload);
+ }
+ /* Cksum offload. */
+ if (!(ol_flags & HINIC3_TX_CKSUM_OFFLOAD_MASK))
+ return 0;
+
+ /* Tso offload. */
+ if (ol_flags & HINIC3_PKT_TX_TCP_SEG) {
+ pld_offset = wqe_info->payload_offset;
+ if ((pld_offset >> 1) > MAX_PAYLOAD_OFFSET)
+ return -EINVAL;
+
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, INNER_L4_EN);
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, INNER_L3_EN);
+
+ queue_info |= SQ_CTRL_QUEUE_INFO_SET(1U, TSO);
+ queue_info |= SQ_CTRL_QUEUE_INFO_SET(pld_offset >> 1, PLDOFF);
+
+ /* Set MSS value. */
+ queue_info = SQ_CTRL_QUEUE_INFO_CLEAR(queue_info, MSS);
+ queue_info |= SQ_CTRL_QUEUE_INFO_SET(mbuf->tso_segsz, MSS);
+ } else {
+ if (ol_flags & HINIC3_PKT_TX_IP_CKSUM)
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, INNER_L3_EN);
+
+ switch (ol_flags & HINIC3_PKT_TX_L4_MASK) {
+ case HINIC3_PKT_TX_TCP_CKSUM:
+ case HINIC3_PKT_TX_UDP_CKSUM:
+ case HINIC3_PKT_TX_SCTP_CKSUM:
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, INNER_L4_EN);
+ break;
+
+ case HINIC3_PKT_TX_L4_NO_CKSUM:
+ break;
+
+ default:
+ PMD_DRV_LOG(INFO, "not support pkt type");
+ return -EINVAL;
+ }
+ }
+
+ /* For vxlan, also can support PKT_TX_TUNNEL_GRE, etc. */
+ switch (ol_flags & HINIC3_PKT_TX_TUNNEL_MASK) {
+ case HINIC3_PKT_TX_TUNNEL_VXLAN:
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, TUNNEL_FLAG);
+ break;
+
+ case 0:
+ break;
+
+ default:
+ /* For non UDP/GRE tunneling, drop the tunnel packet. */
+ PMD_DRV_LOG(INFO, "not support tunnel pkt type");
+ return -EINVAL;
+ }
+
+ if (ol_flags & HINIC3_PKT_TX_OUTER_IP_CKSUM)
+ task->pkt_info0 |= SQ_TASK_INFO0_SET(1U, OUT_L3_EN);
+
+ task->pkt_info0 = hinic3_hw_be32(task->pkt_info0);
+ task->pkt_info2 = hinic3_hw_be32(task->pkt_info2);
+ wqe_info->queue_info = queue_info;
+
+ return 0;
+}
+
+/**
+ * Check whether the number of segments in the mbuf is valid.
+ *
+ * @param[in] mbuf
+ * Point to the mbuf to be verified.
+ * @param[in] wqe_info
+ * Point to wqe_info of send queue(SQ).
+ * @return
+ * true as valid, false as invalid.
+ */
+static bool
+hinic3_is_tso_sge_valid(struct rte_mbuf *mbuf, struct hinic3_wqe_info *wqe_info)
+{
+ u32 total_len, limit_len, checked_len, left_len, adjust_mss;
+ u32 i, max_sges, left_sges, first_len;
+ struct rte_mbuf *mbuf_head, *mbuf_first;
+ struct rte_mbuf *mbuf_pre = mbuf;
+
+ left_sges = mbuf->nb_segs;
+ mbuf_head = mbuf;
+ mbuf_first = mbuf;
+
+ /* Tso sge number validation. */
+ if (unlikely(left_sges >= HINIC3_NONTSO_PKT_MAX_SGE)) {
+ checked_len = 0;
+ total_len = 0;
+ first_len = 0;
+ adjust_mss = mbuf->tso_segsz >= TX_MSS_MIN ? mbuf->tso_segsz
+ : TX_MSS_MIN;
+ max_sges = HINIC3_NONTSO_PKT_MAX_SGE - 1;
+ limit_len = adjust_mss + wqe_info->payload_offset;
+
+ for (i = 0; (i < max_sges) && (total_len < limit_len); i++) {
+ total_len += mbuf->data_len;
+ mbuf_pre = mbuf;
+ mbuf = mbuf->next;
+ }
+
+ /* Each continues 38 mbufs segmust do one check. */
+ while (left_sges >= HINIC3_NONTSO_PKT_MAX_SGE) {
+ if (total_len >= limit_len) {
+ /* Update the limit len. */
+ limit_len = adjust_mss;
+ /* Update checked len. */
+ checked_len += first_len;
+ /* Record the first len. */
+ first_len = mbuf_first->data_len;
+ /* First mbuf move to the next. */
+ mbuf_first = mbuf_first->next;
+ /* Update total len. */
+ total_len -= first_len;
+ left_sges--;
+ i--;
+ for (;
+ (i < max_sges) && (total_len < limit_len);
+ i++) {
+ total_len += mbuf->data_len;
+ mbuf_pre = mbuf;
+ mbuf = mbuf->next;
+ }
+ } else {
+ /* Try to copy if not valid. */
+ checked_len += (total_len - mbuf_pre->data_len);
+
+ left_len = mbuf_head->pkt_len - checked_len;
+ if (left_len > HINIC3_COPY_MBUF_SIZE)
+ return false;
+ wqe_info->sge_cnt = (u16)(mbuf_head->nb_segs +
+ i - left_sges);
+ wqe_info->cpy_mbuf_cnt = 1;
+
+ return true;
+ }
+ } /**< End of while. */
+ }
+
+ wqe_info->sge_cnt = mbuf_head->nb_segs;
+
+ return true;
+}
+
+/**
+ * Checks and processes transport offload information for data packets.
+ *
+ * @param[in] mbuf
+ * Point to the mbuf to send.
+ * @param[in] wqe_info
+ * Point to wqe_info of send queue(SQ).
+ * @return
+ * 0 as success, -EINVAL as failure.
+ */
+static int
+hinic3_get_tx_offload(struct rte_mbuf *mbuf, struct hinic3_wqe_info *wqe_info)
+{
+ uint64_t ol_flags = mbuf->ol_flags;
+ u16 i, total_len, inner_l3_offset = 0;
+ int err;
+ struct rte_mbuf *mbuf_pkt = NULL;
+
+ wqe_info->sge_cnt = mbuf->nb_segs;
+ /* Check if the packet set available offload flags. */
+ if (!(ol_flags & HINIC3_TX_OFFLOAD_MASK)) {
+ wqe_info->offload = 0;
+ return 0;
+ }
+
+ wqe_info->offload = 1;
+ err = hinic3_tx_offload_pkt_prepare(mbuf, &inner_l3_offset);
+ if (err)
+ return err;
+
+ /* Non tso mbuf only check sge num. */
+ if (likely(!(mbuf->ol_flags & HINIC3_PKT_TX_TCP_SEG))) {
+ if (unlikely(mbuf->pkt_len > MAX_SINGLE_SGE_SIZE))
+ /* Non tso packet len must less than 64KB. */
+ return -EINVAL;
+
+ if (likely(HINIC3_NONTSO_SEG_NUM_VALID(mbuf->nb_segs)))
+ /* Valid non-tso mbuf. */
+ return 0;
+
+ /*
+ * The number of non-tso packet fragments must be less than 38,
+ * and mbuf segs greater than 38 must be copied to other
+ * buffers.
+ */
+ total_len = 0;
+ mbuf_pkt = mbuf;
+ for (i = 0; i < (HINIC3_NONTSO_PKT_MAX_SGE - 1); i++) {
+ total_len += mbuf_pkt->data_len;
+ mbuf_pkt = mbuf_pkt->next;
+ }
+
+ /* Default support copy total 4k mbuf segs. */
+ if ((u32)(total_len + (u16)HINIC3_COPY_MBUF_SIZE) <
+ mbuf->pkt_len)
+ return -EINVAL;
+
+ wqe_info->sge_cnt = HINIC3_NONTSO_PKT_MAX_SGE;
+ wqe_info->cpy_mbuf_cnt = 1;
+
+ return 0;
+ }
+
+ /* Tso mbuf. */
+ wqe_info->payload_offset =
+ inner_l3_offset + mbuf->l3_len + mbuf->l4_len;
+
+ /* Too many mbuf segs. */
+ if (unlikely(HINIC3_TSO_SEG_NUM_INVALID(mbuf->nb_segs)))
+ return -EINVAL;
+
+ /* Check whether can cover all tso mbuf segs or not. */
+ if (unlikely(!hinic3_is_tso_sge_valid(mbuf, wqe_info)))
+ return -EINVAL;
+
+ return 0;
+}
+
+static inline void
+hinic3_set_buf_desc(struct hinic3_sq_bufdesc *buf_descs, rte_iova_t addr,
+ u32 len)
+{
+ buf_descs->hi_addr = hinic3_hw_be32(upper_32_bits(addr));
+ buf_descs->lo_addr = hinic3_hw_be32(lower_32_bits(addr));
+ buf_descs->len = hinic3_hw_be32(len);
+}
+
+static inline struct rte_mbuf *
+hinic3_alloc_cpy_mbuf(struct hinic3_nic_dev *nic_dev)
+{
+ return rte_pktmbuf_alloc(nic_dev->cpy_mpool);
+}
+
+/**
+ * Copy packets in the send queue(SQ).
+ *
+ * @param[in] nic_dev
+ * Point to nic device.
+ * @param[in] mbuf
+ * Point to the source mbuf.
+ * @param[in] seg_cnt
+ * Number of mbuf segments to be copied.
+ * @result
+ * The address of the copied mbuf.
+ */
+static void *
+hinic3_copy_tx_mbuf(struct hinic3_nic_dev *nic_dev, struct rte_mbuf *mbuf,
+ u16 sge_cnt)
+{
+ struct rte_mbuf *dst_mbuf;
+ u32 offset = 0;
+ u16 i;
+
+ if (unlikely(!nic_dev->cpy_mpool))
+ return NULL;
+
+ dst_mbuf = hinic3_alloc_cpy_mbuf(nic_dev);
+ if (unlikely(!dst_mbuf))
+ return NULL;
+
+ dst_mbuf->data_off = 0;
+ dst_mbuf->data_len = 0;
+ for (i = 0; i < sge_cnt; i++) {
+ rte_memcpy((u8 *)dst_mbuf->buf_addr + offset,
+ (u8 *)mbuf->buf_addr + mbuf->data_off,
+ mbuf->data_len);
+ dst_mbuf->data_len += mbuf->data_len;
+ offset += mbuf->data_len;
+ mbuf = mbuf->next;
+ }
+ dst_mbuf->pkt_len = dst_mbuf->data_len;
+
+ return dst_mbuf;
+}
+
+/**
+ * Map the TX mbuf to the DMA address space and set related information for
+ * subsequent DMA transmission.
+ *
+ * @param[in] txq
+ * Point to send queue.
+ * @param[in] mbuf
+ * Point to the tx mbuf.
+ * @param[out] wqe_combo
+ * Point to send queue wqe_combo.
+ * @param[in] wqe_info
+ * Point to wqe_info of send queue(SQ).
+ * @result
+ * 0 as success, -EINVAL as failure.
+ */
+static int
+hinic3_mbuf_dma_map_sge(struct hinic3_txq *txq, struct rte_mbuf *mbuf,
+ struct hinic3_sq_wqe_combo *wqe_combo,
+ struct hinic3_wqe_info *wqe_info)
+{
+ struct hinic3_sq_wqe_desc *wqe_desc = wqe_combo->hdr;
+ struct hinic3_sq_bufdesc *buf_desc = wqe_combo->bds_head;
+
+ uint16_t nb_segs = wqe_info->sge_cnt - wqe_info->cpy_mbuf_cnt;
+ uint16_t real_segs = mbuf->nb_segs;
+ rte_iova_t dma_addr;
+ u32 i;
+
+ for (i = 0; i < nb_segs; i++) {
+ if (unlikely(mbuf == NULL)) {
+ txq->txq_stats.mbuf_null++;
+ return -EINVAL;
+ }
+
+ if (unlikely(mbuf->data_len == 0)) {
+ txq->txq_stats.sge_len0++;
+ return -EINVAL;
+ }
+
+ dma_addr = rte_mbuf_data_iova(mbuf);
+ if (i == 0) {
+ if (wqe_combo->wqe_type == SQ_WQE_COMPACT_TYPE &&
+ mbuf->data_len > COMPACT_WQE_MAX_CTRL_LEN) {
+ txq->txq_stats.sge_len_too_large++;
+ return -EINVAL;
+ }
+
+ wqe_desc->hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ wqe_desc->lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ wqe_desc->ctrl_len = mbuf->data_len;
+ } else {
+ /*
+ * Parts of wqe is in sq bottom while parts
+ * of wqe is in sq head.
+ */
+ if (unlikely(wqe_info->wrapped &&
+ (u64)buf_desc == txq->sq_bot_sge_addr))
+ buf_desc = (struct hinic3_sq_bufdesc *)
+ (void *)txq->sq_head_addr;
+
+ hinic3_set_buf_desc(buf_desc, dma_addr, mbuf->data_len);
+ buf_desc++;
+ }
+ mbuf = mbuf->next;
+ }
+
+ /* For now: support over 38 sge, copy the last 2 mbuf. */
+ if (unlikely(wqe_info->cpy_mbuf_cnt != 0)) {
+ /*
+ * Copy invalid mbuf segs to a valid buffer, lost performance.
+ */
+ txq->txq_stats.cpy_pkts += 1;
+ mbuf = hinic3_copy_tx_mbuf(txq->nic_dev, mbuf,
+ real_segs - nb_segs);
+ if (unlikely(!mbuf))
+ return -EINVAL;
+
+ txq->tx_info[wqe_info->pi].cpy_mbuf = mbuf;
+
+ /* Deal with the last mbuf. */
+ dma_addr = rte_mbuf_data_iova(mbuf);
+ if (unlikely(mbuf->data_len == 0)) {
+ txq->txq_stats.sge_len0++;
+ return -EINVAL;
+ }
+ /*
+ * Parts of wqe is in sq bottom while parts
+ * of wqe is in sq head.
+ */
+ if (i == 0) {
+ wqe_desc->hi_addr =
+ hinic3_hw_be32(upper_32_bits(dma_addr));
+ wqe_desc->lo_addr =
+ hinic3_hw_be32(lower_32_bits(dma_addr));
+ wqe_desc->ctrl_len = mbuf->data_len;
+ } else {
+ if (unlikely(wqe_info->wrapped &&
+ ((u64)buf_desc == txq->sq_bot_sge_addr)))
+ buf_desc = (struct hinic3_sq_bufdesc *)
+ txq->sq_head_addr;
+
+ hinic3_set_buf_desc(buf_desc, dma_addr, mbuf->data_len);
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Sets and configures fields in the transmit queue control descriptor based on
+ * the WQE type.
+ *
+ * @param[out] wqe_combo
+ * Point to wqe_combo of send queue.
+ * @param[in] wqe_info
+ * Point to wqe_info of send queue.
+ */
+static void
+hinic3_prepare_sq_ctrl(struct hinic3_sq_wqe_combo *wqe_combo,
+ struct hinic3_wqe_info *wqe_info)
+{
+ struct hinic3_sq_wqe_desc *wqe_desc = wqe_combo->hdr;
+
+ if (wqe_combo->wqe_type == SQ_WQE_COMPACT_TYPE) {
+ wqe_desc->ctrl_len |=
+ SQ_CTRL_SET(SQ_NORMAL_WQE, DATA_FORMAT) |
+ SQ_CTRL_SET(wqe_combo->wqe_type, EXTENDED) |
+ SQ_CTRL_SET(wqe_info->owner, OWNER);
+ wqe_desc->ctrl_len = hinic3_hw_be32(wqe_desc->ctrl_len);
+
+ /* Compact wqe queue_info will transfer to ucode. */
+ wqe_desc->queue_info = 0;
+
+ return;
+ }
+
+ wqe_desc->ctrl_len |= SQ_CTRL_SET(wqe_info->sge_cnt, BUFDESC_NUM) |
+ SQ_CTRL_SET(wqe_combo->task_type, TASKSECT_LEN) |
+ SQ_CTRL_SET(SQ_NORMAL_WQE, DATA_FORMAT) |
+ SQ_CTRL_SET(wqe_combo->wqe_type, EXTENDED) |
+ SQ_CTRL_SET(wqe_info->owner, OWNER);
+
+ wqe_desc->ctrl_len = hinic3_hw_be32(wqe_desc->ctrl_len);
+
+ wqe_desc->queue_info = wqe_info->queue_info;
+ wqe_desc->queue_info |= SQ_CTRL_QUEUE_INFO_SET(1U, UC);
+
+ if (!SQ_CTRL_QUEUE_INFO_GET(wqe_desc->queue_info, MSS)) {
+ wqe_desc->queue_info |=
+ SQ_CTRL_QUEUE_INFO_SET(TX_MSS_DEFAULT, MSS);
+ } else if (SQ_CTRL_QUEUE_INFO_GET(wqe_desc->queue_info, MSS) <
+ TX_MSS_MIN) {
+ /* Mss should not less than 80. */
+ wqe_desc->queue_info =
+ SQ_CTRL_QUEUE_INFO_CLEAR(wqe_desc->queue_info, MSS);
+ wqe_desc->queue_info |= SQ_CTRL_QUEUE_INFO_SET(TX_MSS_MIN, MSS);
+ }
+
+ wqe_desc->queue_info = hinic3_hw_be32(wqe_desc->queue_info);
+}
+
+/**
+ * It is responsible for sending data packets.
+ *
+ * @param[in] tx_queue
+ * Point to send queue.
+ * @param[in] tx_pkts
+ * Pointer to the array of data packets to be sent.
+ * @param[in] nb_pkts
+ * Number of sent packets.
+ * @return
+ * Number of actually sent packets.
+ */
+u16
+hinic3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct hinic3_txq *txq = tx_queue;
+ struct hinic3_tx_info *tx_info = NULL;
+ struct rte_mbuf *mbuf_pkt = NULL;
+ struct hinic3_sq_wqe_combo wqe_combo = {0};
+ struct hinic3_sq_wqe *sq_wqe = NULL;
+ struct hinic3_wqe_info wqe_info = {0};
+
+ u32 offload_err, free_cnt;
+ u64 tx_bytes = 0;
+ u16 free_wqebb_cnt, nb_tx;
+ int err;
+
+#ifdef HINIC3_XSTAT_PROF_TX
+ uint64_t t1, t2;
+ t1 = rte_get_tsc_cycles();
+#endif
+
+ if (unlikely(!HINIC3_TXQ_IS_STARTED(txq)))
+ return 0;
+
+ free_cnt = txq->tx_free_thresh;
+ /* Reclaim tx mbuf before xmit new packets. */
+ if (hinic3_get_sq_free_wqebbs(txq) < txq->tx_free_thresh)
+ hinic3_xmit_mbuf_cleanup(txq, free_cnt);
+
+ /* Tx loop routine. */
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ mbuf_pkt = *tx_pkts++;
+ if (unlikely(hinic3_get_tx_offload(mbuf_pkt, &wqe_info))) {
+ txq->txq_stats.offload_errors++;
+ break;
+ }
+
+ if (!wqe_info.offload)
+ wqe_info.wqebb_cnt = wqe_info.sge_cnt;
+ else
+ /* Use extended sq wqe with normal TS. */
+ wqe_info.wqebb_cnt = wqe_info.sge_cnt + 1;
+
+ free_wqebb_cnt = hinic3_get_sq_free_wqebbs(txq);
+ if (unlikely(wqe_info.wqebb_cnt > free_wqebb_cnt)) {
+ /* Reclaim again. */
+ hinic3_xmit_mbuf_cleanup(txq, free_cnt);
+ free_wqebb_cnt = hinic3_get_sq_free_wqebbs(txq);
+ if (unlikely(wqe_info.wqebb_cnt > free_wqebb_cnt)) {
+ txq->txq_stats.tx_busy += (nb_pkts - nb_tx);
+ break;
+ }
+ }
+
+ /* Get sq wqe address from wqe_page. */
+ sq_wqe = hinic3_get_sq_wqe(txq, &wqe_info);
+ if (unlikely(!sq_wqe)) {
+ txq->txq_stats.tx_busy++;
+ break;
+ }
+
+ /* Task or bd section maybe warpped for one wqe. */
+ hinic3_set_wqe_combo(txq, &wqe_combo, sq_wqe, &wqe_info);
+
+ wqe_info.queue_info = 0;
+ /* Fill tx packet offload into qsf and task field. */
+ if (wqe_info.offload) {
+ offload_err = hinic3_set_tx_offload(mbuf_pkt,
+ wqe_combo.task,
+ &wqe_info);
+ if (unlikely(offload_err)) {
+ hinic3_put_sq_wqe(txq, &wqe_info);
+ txq->txq_stats.offload_errors++;
+ break;
+ }
+ }
+
+ /* Fill sq_wqe buf_desc and bd_desc. */
+ err = hinic3_mbuf_dma_map_sge(txq, mbuf_pkt, &wqe_combo,
+ &wqe_info);
+ if (err) {
+ hinic3_put_sq_wqe(txq, &wqe_info);
+ txq->txq_stats.offload_errors++;
+ break;
+ }
+
+ /* Record tx info. */
+ tx_info = &txq->tx_info[wqe_info.pi];
+ tx_info->mbuf = mbuf_pkt;
+ tx_info->wqebb_cnt = wqe_info.wqebb_cnt;
+
+ hinic3_prepare_sq_ctrl(&wqe_combo, &wqe_info);
+
+ tx_bytes += mbuf_pkt->pkt_len;
+ }
+
+ /* Update txq stats. */
+ if (nb_tx) {
+ hinic3_write_db(txq->db_addr, txq->q_id, (int)(txq->cos),
+ SQ_CFLAG_DP,
+ MASKED_QUEUE_IDX(txq, txq->prod_idx));
+ txq->txq_stats.packets += nb_tx;
+ txq->txq_stats.bytes += tx_bytes;
+ }
+ txq->txq_stats.burst_pkts = nb_tx;
+
+#ifdef HINIC3_XSTAT_PROF_TX
+ t2 = rte_get_tsc_cycles();
+ txq->txq_stats.app_tsc = t1 - txq->prof_tx_end_tsc;
+ txq->prof_tx_end_tsc = t2;
+ txq->txq_stats.pmd_tsc = t2 - t1;
+ txq->txq_stats.burst_pkts = nb_tx;
+#endif
+
+ return nb_tx;
+}
+
int
hinic3_stop_sq(struct hinic3_txq *txq)
{
diff --git a/drivers/net/hinic3/hinic3_tx.h b/drivers/net/hinic3/hinic3_tx.h
index f4c61ea1b1..6026b3fabc 100644
--- a/drivers/net/hinic3/hinic3_tx.h
+++ b/drivers/net/hinic3/hinic3_tx.h
@@ -308,6 +308,7 @@ struct __rte_cache_aligned hinic3_txq {
void hinic3_flush_txqs(struct hinic3_nic_dev *nic_dev);
void hinic3_free_txq_mbufs(struct hinic3_txq *txq);
void hinic3_free_all_txq_mbufs(struct hinic3_nic_dev *nic_dev);
+u16 hinic3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
int hinic3_stop_sq(struct hinic3_txq *txq);
int hinic3_start_all_sqs(struct rte_eth_dev *eth_dev);
int hinic3_tx_done_cleanup(void *txq, uint32_t free_cnt);
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 15/18] net/hinic3: add MML and EEPROM access feature
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (5 preceding siblings ...)
2025-04-18 7:02 ` [RFC 14/18] net/hinic3: add Rx/Tx functions Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 16/18] net/hinic3: add RSS promiscuous ops Feifei Wang
` (3 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Xin Wang, Feifei Wang, Yi Chen
From: Xin Wang <wangxin679@h-partners.com>
Add man-machine language support and implements the get eeprom method.
Signed-off-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
Reviewed-by: Yi Chen <chenyi221@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 13 +
drivers/net/hinic3/mml/hinic3_dbg.c | 171 +++++
drivers/net/hinic3/mml/hinic3_dbg.h | 160 +++++
drivers/net/hinic3/mml/hinic3_mml_cmd.c | 375 +++++++++++
drivers/net/hinic3/mml/hinic3_mml_cmd.h | 131 ++++
drivers/net/hinic3/mml/hinic3_mml_ioctl.c | 215 +++++++
drivers/net/hinic3/mml/hinic3_mml_lib.c | 136 ++++
drivers/net/hinic3/mml/hinic3_mml_lib.h | 275 ++++++++
drivers/net/hinic3/mml/hinic3_mml_main.c | 167 +++++
drivers/net/hinic3/mml/hinic3_mml_queue.c | 749 ++++++++++++++++++++++
drivers/net/hinic3/mml/hinic3_mml_queue.h | 256 ++++++++
drivers/net/hinic3/mml/meson.build | 62 ++
12 files changed, 2710 insertions(+)
create mode 100644 drivers/net/hinic3/mml/hinic3_dbg.c
create mode 100644 drivers/net/hinic3/mml/hinic3_dbg.h
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_cmd.c
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_cmd.h
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_ioctl.c
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_lib.c
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_lib.h
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_main.c
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_queue.c
create mode 100644 drivers/net/hinic3/mml/hinic3_mml_queue.h
create mode 100644 drivers/net/hinic3/mml/meson.build
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index 7cd101e5c3..9c5decb867 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -21,6 +21,7 @@
#include "base/hinic3_hw_comm.h"
#include "base/hinic3_nic_cfg.h"
#include "base/hinic3_nic_event.h"
+#include "mml/hinic3_mml_lib.h"
#include "hinic3_nic_io.h"
#include "hinic3_tx.h"
#include "hinic3_rx.h"
@@ -2276,6 +2277,16 @@ hinic3_dev_allmulticast_disable(struct rte_eth_dev *dev)
return 0;
}
+static int
+hinic3_get_eeprom(__rte_unused struct rte_eth_dev *dev,
+ struct rte_dev_eeprom_info *info)
+{
+#define MAX_BUF_OUT_LEN 2048
+
+ return hinic3_pmd_mml_lib(info->data, info->offset, info->data,
+ &info->length, MAX_BUF_OUT_LEN);
+}
+
/**
* Get device generic statistics.
*
@@ -2879,6 +2890,7 @@ static const struct eth_dev_ops hinic3_pmd_ops = {
.vlan_offload_set = hinic3_vlan_offload_set,
.allmulticast_enable = hinic3_dev_allmulticast_enable,
.allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .get_eeprom = hinic3_get_eeprom,
.stats_get = hinic3_dev_stats_get,
.stats_reset = hinic3_dev_stats_reset,
.xstats_get = hinic3_dev_xstats_get,
@@ -2919,6 +2931,7 @@ static const struct eth_dev_ops hinic3_pmd_vf_ops = {
.vlan_offload_set = hinic3_vlan_offload_set,
.allmulticast_enable = hinic3_dev_allmulticast_enable,
.allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .get_eeprom = hinic3_get_eeprom,
.stats_get = hinic3_dev_stats_get,
.stats_reset = hinic3_dev_stats_reset,
.xstats_get = hinic3_dev_xstats_get,
diff --git a/drivers/net/hinic3/mml/hinic3_dbg.c b/drivers/net/hinic3/mml/hinic3_dbg.c
new file mode 100644
index 0000000000..7525b68dee
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_dbg.c
@@ -0,0 +1,171 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#include "hinic3_compat.h"
+#include "hinic3_hwif.h"
+#include "hinic3_hwdev.h"
+#include "hinic3_wq.h"
+#include "hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_rx.h"
+#include "hinic3_tx.h"
+#include "hinic3_nic_io.h"
+#include "hinic3_dbg.h"
+
+#define DB_IDX(db, db_base) \
+ ((u32)(((ulong)(db) - (ulong)(db_base)) / HINIC3_DB_PAGE_SIZE))
+
+int
+hinic3_dbg_get_rq_info(void *hwdev, uint16_t q_id,
+ struct hinic3_dbg_rq_info *rq_info, u16 *msg_size)
+{
+ struct hinic3_hwdev *dev = (struct hinic3_hwdev *)hwdev;
+ struct hinic3_nic_dev *nic_dev =
+ (struct hinic3_nic_dev *)dev->dev_handle;
+ struct hinic3_rxq *rxq = NULL;
+
+ if (q_id >= nic_dev->num_rqs) {
+ PMD_DRV_LOG(ERR, "Invalid rx queue id, q_id: %d, num_rqs: %d",
+ q_id, nic_dev->num_rqs);
+ return -EINVAL;
+ }
+
+ rq_info->q_id = q_id;
+ rxq = nic_dev->rxqs[q_id];
+
+ rq_info->hw_pi = (u16)cpu_to_be16(*rxq->pi_virt_addr);
+ rq_info->ci = rxq->cons_idx & rxq->q_mask;
+ rq_info->sw_pi = rxq->prod_idx & rxq->q_mask;
+ rq_info->wqebb_size = HINIC3_SQ_WQEBB_SIZE;
+ rq_info->q_depth = rxq->q_depth;
+ rq_info->buf_len = rxq->buf_len;
+ rq_info->ci_wqe_page_addr = rxq->queue_buf_vaddr;
+ rq_info->ci_cla_tbl_addr = NULL;
+ rq_info->msix_idx = 0;
+ rq_info->msix_vector = 0;
+
+ *msg_size = sizeof(*rq_info);
+
+ return 0;
+}
+
+int
+hinic3_dbg_get_rx_cqe_info(void *hwdev, uint16_t q_id, uint16_t idx,
+ void *buf_out, uint16_t *out_size)
+{
+ struct hinic3_hwdev *dev = (struct hinic3_hwdev *)hwdev;
+ struct hinic3_nic_dev *nic_dev =
+ (struct hinic3_nic_dev *)dev->dev_handle;
+
+ if (q_id >= nic_dev->num_rqs || idx >= nic_dev->rxqs[q_id]->q_depth)
+ return -EFAULT;
+
+ (void)memcpy(buf_out, (void *)&nic_dev->rxqs[q_id]->rx_cqe[idx],
+ sizeof(struct hinic3_rq_cqe));
+ *out_size = sizeof(struct hinic3_rq_cqe);
+
+ return 0;
+}
+
+int
+hinic3_dbg_get_sq_info(void *dev, u16 q_id, struct hinic3_dbg_sq_info *sq_info,
+ u16 *msg_size)
+{
+ struct hinic3_hwdev *hwdev = (struct hinic3_hwdev *)dev;
+ struct hinic3_nic_dev *nic_dev =
+ (struct hinic3_nic_dev *)hwdev->dev_handle;
+ struct hinic3_txq *txq = NULL;
+
+ if (q_id >= nic_dev->num_sqs) {
+ PMD_DRV_LOG(ERR,
+ "Inputting tx queue id is larger than actual tx "
+ "queue number, qid: %d, num_sqs: %d",
+ q_id, nic_dev->num_sqs);
+ return -EINVAL;
+ }
+
+ sq_info->q_id = q_id;
+ txq = nic_dev->txqs[q_id];
+
+ sq_info->pi = txq->prod_idx & txq->q_mask;
+ sq_info->ci = txq->cons_idx & txq->q_mask;
+ sq_info->fi = (*(u16 *)txq->ci_vaddr_base) & txq->q_mask;
+ sq_info->q_depth = txq->q_depth;
+ sq_info->weqbb_size = HINIC3_SQ_WQEBB_SIZE;
+ sq_info->ci_addr =
+ (volatile u16 *)HINIC3_CI_VADDR(txq->ci_vaddr_base, q_id);
+ sq_info->cla_addr = txq->queue_buf_paddr;
+ sq_info->db_addr.phy_addr = (u64 *)txq->db_addr;
+ sq_info->pg_idx = DB_IDX(txq->db_addr, hwdev->hwif->db_base);
+
+ *msg_size = sizeof(*sq_info);
+
+ return 0;
+}
+
+int
+hinic3_dbg_get_sq_wqe_info(void *dev, u16 q_id, u16 idx, u16 wqebb_cnt, u8 *wqe,
+ u16 *wqe_size)
+{
+ struct hinic3_hwdev *hwdev = (struct hinic3_hwdev *)dev;
+ struct hinic3_nic_dev *nic_dev =
+ (struct hinic3_nic_dev *)hwdev->dev_handle;
+ struct hinic3_txq *txq = NULL;
+ void *src_wqe = NULL;
+ u32 offset;
+
+ if (q_id >= nic_dev->num_sqs) {
+ PMD_DRV_LOG(ERR,
+ "Inputting tx queue id is larger than actual tx "
+ "queue number, qid: %d, num_sqs: %d",
+ q_id, nic_dev->num_sqs);
+ return -EINVAL;
+ }
+
+ txq = nic_dev->txqs[q_id];
+ if (idx + wqebb_cnt > txq->q_depth)
+ return -EFAULT;
+
+ src_wqe = (void *)txq->queue_buf_vaddr;
+ offset = (u32)idx << txq->wqebb_shift;
+
+ (void)memcpy((void *)wqe, (void *)((u8 *)src_wqe + offset),
+ (size_t)((u32)wqebb_cnt << txq->wqebb_shift));
+
+ *wqe_size = (u16)((u32)wqebb_cnt << txq->wqebb_shift);
+ return 0;
+}
+
+int
+hinic3_dbg_get_rq_wqe_info(void *dev, u16 q_id, u16 idx, u16 wqebb_cnt, u8 *wqe,
+ u16 *wqe_size)
+{
+ struct hinic3_hwdev *hwdev = (struct hinic3_hwdev *)dev;
+ struct hinic3_nic_dev *nic_dev =
+ (struct hinic3_nic_dev *)hwdev->dev_handle;
+ struct hinic3_rxq *rxq = NULL;
+ void *src_wqe = NULL;
+ u32 offset;
+
+ if (q_id >= nic_dev->num_rqs) {
+ PMD_DRV_LOG(ERR,
+ "Inputting rx queue id is larger than actual rx "
+ "queue number, qid: %d, num_rqs: %d",
+ q_id, nic_dev->num_rqs);
+ return -EINVAL;
+ }
+
+ rxq = nic_dev->rxqs[q_id];
+ if (idx + wqebb_cnt > rxq->q_depth)
+ return -EFAULT;
+
+ src_wqe = (void *)rxq->queue_buf_vaddr;
+ offset = (u32)idx << rxq->wqebb_shift;
+
+ (void)memcpy((void *)wqe, (void *)((u8 *)src_wqe + offset),
+ (size_t)((u32)wqebb_cnt << rxq->wqebb_shift));
+
+ *wqe_size = (u16)((u32)wqebb_cnt << rxq->wqebb_shift);
+ return 0;
+}
diff --git a/drivers/net/hinic3/mml/hinic3_dbg.h b/drivers/net/hinic3/mml/hinic3_dbg.h
new file mode 100644
index 0000000000..bac96c84a0
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_dbg.h
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#ifndef _HINIC3_MML_DBG_H
+#define _HINIC3_MML_DBG_H
+
+/* nic_tool */
+struct hinic3_tx_hw_page {
+ u64 *phy_addr;
+ u64 *map_addr;
+};
+
+/* nic_tool */
+struct hinic3_dbg_sq_info {
+ u16 q_id;
+ u16 pi;
+ u16 ci; /**< sw_ci */
+ u16 fi; /**< hw_ci */
+
+ u32 q_depth;
+ u16 weqbb_size;
+
+ volatile u16 *ci_addr;
+ u64 cla_addr;
+
+ struct hinic3_tx_hw_page db_addr;
+ u32 pg_idx;
+};
+
+/* nic_tool */
+struct hinic3_dbg_rq_info {
+ u16 q_id;
+ u16 hw_pi;
+ u16 ci; /**< sw_ci */
+ u16 sw_pi;
+ u16 wqebb_size;
+ u16 q_depth;
+ u16 buf_len;
+
+ void *ci_wqe_page_addr;
+ void *ci_cla_tbl_addr;
+ u16 msix_idx;
+ u32 msix_vector;
+};
+
+void *hinic3_dbg_get_sq_wq_handle(void *hwdev, u16 q_id);
+
+void *hinic3_dbg_get_rq_wq_handle(void *hwdev, u16 q_id);
+
+void *hinic3_dbg_get_sq_ci_addr(void *hwdev, u16 q_id);
+
+u16 hinic3_dbg_get_global_qpn(void *hwdev);
+
+/**
+ * Get details of specified RX queue and store in `rq_info`.
+ *
+ * @param[in] hwdev
+ * Pointer to the hardware device.
+ * @param[in] q_id
+ * RX queue ID.
+ * @param[out] rq_info
+ * Structure to store RX queue information.
+ * @param[out] msg_size
+ * Size of the message.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int hinic3_dbg_get_rq_info(void *hwdev, uint16_t q_id,
+ struct hinic3_dbg_rq_info *rq_info, u16 *msg_size);
+
+/**
+ * Get the RX CQE at the specified index from the given RX queue.
+ *
+ * @param[in] hwdev
+ * Pointer to hardware device structure.
+ * @param[in] q_id
+ * RX queue ID.
+ * @param[in] idx
+ * Index of the CQE.
+ * @param[out] buf_out
+ * Buffer to store the CQE.
+ * @param[out] out_size
+ * Size of the CQE.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int hinic3_dbg_get_rx_cqe_info(void *hwdev, uint16_t q_id, uint16_t idx,
+ void *buf_out, uint16_t *out_size);
+
+/**
+ * Get SQ information for debugging.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] q_id
+ * ID of SQ to retrieve information for.
+ * @param[out] sq_info
+ * Pointer to the structure where the SQ information will be stored.
+ * @param[out] msg_size
+ * The size (in bytes) of the `sq_info` structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -EINVAL if the queue ID is invalid.
+ */
+int hinic3_dbg_get_sq_info(void *hwdev, u16 q_id,
+ struct hinic3_dbg_sq_info *sq_info, u16 *msg_size);
+
+/**
+ * Get WQE information from a send queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] q_id
+ * The ID of the send queue from which to retrieve WQE information.
+ * @param[in] idx
+ * The index of the first WQE to retrieve.
+ * @param[in] wqebb_cnt
+ * The number of WQEBBs to retrieve.
+ * @param[out] wqe
+ * Pointer to the buffer where the WQE data will be stored.
+ * @param[out] wqe_size
+ * The size (in bytes) of the retrieved WQE data.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -EINVAL if queue ID invalid.
+ * - -EFAULT if index invalid.
+ */
+int hinic3_dbg_get_sq_wqe_info(void *hwdev, u16 q_id, u16 idx, u16 wqebb_cnt,
+ u8 *wqe, u16 *wqe_size);
+
+/**
+ * Get WQE information from a receive queue.
+ *
+ * @param[in] dev
+ * Pointer to the device structure.
+ * @param[in] q_id
+ * The ID of the receive queue from which to retrieve WQE information.
+ * @param[in] idx
+ * The index of the first WQE to retrieve.
+ * @param[in] wqebb_cnt
+ * The number of WQEBBs to retrieve.
+ * @param[out] wqe
+ * Pointer to the buffer where the WQE data will be stored.
+ * @param[out] wqe_size
+ * The size (in bytes) of the retrieved WQE data.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -EINVAL if queue ID invalid.
+ * - -EFAULT if index invalid.
+ */
+int hinic3_dbg_get_rq_wqe_info(void *hwdev, u16 q_id, u16 idx, u16 wqebb_cnt,
+ u8 *wqe, u16 *wqe_size);
+
+#endif /* _HINIC3_MML_DBG_H */
diff --git a/drivers/net/hinic3/mml/hinic3_mml_cmd.c b/drivers/net/hinic3/mml/hinic3_mml_cmd.c
new file mode 100644
index 0000000000..06d20a62bd
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_cmd.c
@@ -0,0 +1,375 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#include "hinic3_mml_lib.h"
+#include "hinic3_compat.h"
+#include "hinic3_hwdev.h"
+#include "hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_mml_cmd.h"
+
+/**
+ * Compares two strings for equality.
+ *
+ * @param[in] command
+ * The first string to compare.
+ * @param[in] argument
+ * The second string to compare.
+ *
+ * @return
+ * UDA_TRUE if the strings are equal, otherwise UDA_FALSE.
+ */
+static int
+string_cmp(const char *command, const char *argument)
+{
+ const char *cmd = command;
+ const char *arg = argument;
+
+ if (!cmd || !arg)
+ return UDA_FALSE;
+
+ if (strlen(cmd) != strlen(arg))
+ return UDA_FALSE;
+
+ do {
+ if (*cmd != *arg)
+ return UDA_FALSE;
+ cmd++;
+ arg++;
+ } while (*cmd != '\0');
+
+ return UDA_TRUE;
+}
+
+static void
+show_tool_version(cmd_adapter_t *adapter)
+{
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ "hinic3 pmd version %s", HINIC3_PMD_DRV_VERSION);
+}
+
+static void
+show_tool_help(cmd_adapter_t *adapter)
+{
+ int i;
+ major_cmd_t *major_cmd = NULL;
+
+ if (!adapter)
+ return;
+
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ "\n Usage:evsadm exec dump-hinic-status <major_cmd> "
+ "[option]\n");
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ " -h, --help show help information");
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ " -v, --version show version information");
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ "\n Major Commands:\n");
+
+ for (i = 0; i < adapter->major_cmds; i++) {
+ major_cmd = adapter->p_major_cmd[i];
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ " %-23s %s", major_cmd->name,
+ major_cmd->description);
+ }
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len, "");
+}
+
+void
+major_command_option(major_cmd_t *major_cmd, const char *little,
+ const char *large, uint32_t have_param,
+ command_record_t record)
+{
+ cmd_option_t *option = NULL;
+
+ if (major_cmd == NULL || (little == NULL && large == NULL) || !record) {
+ PMD_DRV_LOG(ERR, "Invalid input parameter.");
+ return;
+ }
+
+ if (major_cmd->option_count >= COMMAND_MAX_OPTIONS) {
+ PMD_DRV_LOG(ERR, "Do not support more than %d options",
+ COMMAND_MAX_OPTIONS);
+ return;
+ }
+
+ option = &major_cmd->options[major_cmd->option_count];
+ major_cmd->options_repeat_flag[major_cmd->option_count] = 0;
+ major_cmd->option_count++;
+
+ option->record = record;
+ option->little = little;
+ option->large = large;
+ option->have_param = have_param;
+}
+
+void
+major_command_register(cmd_adapter_t *adapter, major_cmd_t *major_cmd)
+{
+ int i = 0;
+
+ if (adapter == NULL || major_cmd == NULL) {
+ PMD_DRV_LOG(ERR, "Invalid input parameter.");
+ return;
+ }
+
+ if (adapter->major_cmds >= COMMAND_MAX_MAJORS) {
+ PMD_DRV_LOG(ERR, "Major Commands is full");
+ return;
+ }
+ while (adapter->p_major_cmd[i] != NULL)
+ i++;
+ adapter->p_major_cmd[i] = major_cmd;
+ adapter->major_cmds++;
+ major_cmd->adapter = adapter;
+ major_cmd->err_no = UDA_SUCCESS;
+ (void)memset(major_cmd->err_str, 0, sizeof(major_cmd->err_str));
+}
+
+static int
+is_help_version(cmd_adapter_t *adapter, int argc, char *arg)
+{
+ if (COMMAND_HELP_POSTION(argc) &&
+ (string_cmp("-h", arg) || string_cmp("--help", arg))) {
+ show_tool_help(adapter);
+ return UDA_TRUE;
+ }
+
+ if (COMMAND_VERSION_POSTION(argc) &&
+ (string_cmp("-v", arg) || string_cmp("--version", arg))) {
+ show_tool_version(adapter);
+ return UDA_TRUE;
+ }
+
+ return UDA_FALSE;
+}
+
+static int
+check_command_length(int argc, char **argv)
+{
+ int i;
+ unsigned long long str_len = 0;
+
+ for (i = 1; i < argc; i++)
+ str_len += strlen(argv[i]);
+
+ if (str_len > COMMAND_MAX_STRING)
+ return -UDA_EINVAL;
+
+ return UDA_SUCCESS;
+}
+
+static inline int
+char_check(const char cmd)
+{
+ if (cmd >= 'a' && cmd <= 'z')
+ return UDA_SUCCESS;
+
+ if (cmd >= 'A' && cmd <= 'Z')
+ return UDA_SUCCESS;
+ return UDA_FAIL;
+}
+
+static int
+major_command_check_param(cmd_option_t *option, char *arg)
+{
+ if (!option)
+ return -UDA_EINVAL;
+ if (option->have_param != 0) {
+ if (!arg || ((arg[0] == '-') && char_check(arg[1])))
+ return -UDA_EINVAL;
+ return UDA_SUCCESS;
+ }
+
+ return -UDA_ENOOBJ;
+}
+
+static int
+major_cmd_repeat_option_set(major_cmd_t *major_cmd, const cmd_option_t *option,
+ u32 *options_repeat_flag)
+{
+ int err;
+
+ if (*options_repeat_flag != 0) {
+ major_cmd->err_no = -UDA_EINVAL;
+ err = snprintf(major_cmd->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Repeated option %s|%s.", option->little,
+ option->large);
+ if (err <= 0) {
+ PMD_DRV_LOG(ERR,
+ "snprintf cmd repeat option failed, err: %d.",
+ err);
+ }
+ return -UDA_EINVAL;
+ }
+ *options_repeat_flag = 1;
+ return UDA_SUCCESS;
+}
+
+static int
+major_cmd_option_check(major_cmd_t *major_cmd, char **argv, int *index)
+{
+ int j, ret, err, option_ok, intermediate_var;
+ cmd_option_t *option = NULL;
+ char *arg = argv[*index];
+
+ /* Find command. */
+ for (j = 0; j < major_cmd->option_count; j++) {
+ option = &major_cmd->options[j];
+ option_ok = (((option->little != NULL) &&
+ string_cmp(option->little, arg)) ||
+ ((option->large != NULL) &&
+ string_cmp(option->large, arg)));
+ if (!option_ok)
+ continue;
+ /* Find same option. */
+ ret = major_cmd_repeat_option_set(major_cmd,
+ option, &major_cmd->options_repeat_flag[j]);
+ if (ret != UDA_SUCCESS)
+ return ret;
+
+ arg = NULL;
+ /* If this option need parameters. */
+ intermediate_var = (*index) + 1;
+ ret = major_command_check_param(option, argv[intermediate_var]);
+ if (ret == UDA_SUCCESS) {
+ (*index)++;
+ arg = argv[*index];
+ } else if (ret == -UDA_EINVAL) {
+ major_cmd->err_no = -UDA_EINVAL;
+ err = snprintf(major_cmd->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "%s|%s option need parameter.",
+ option->little, option->large);
+ if (err <= 0) {
+ PMD_DRV_LOG(ERR,
+ "snprintf cmd option need para "
+ "failed, err: %d.",
+ err);
+ }
+ return -UDA_EINVAL;
+ }
+
+ /* Record messages. */
+ ret = option->record(major_cmd, arg);
+ if (ret != UDA_SUCCESS)
+ return ret;
+ break;
+ }
+
+ /* Illegal option. */
+ if (j == major_cmd->option_count) {
+ major_cmd->err_no = -UDA_EINVAL;
+ err = snprintf(major_cmd->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "%s is not option needed.", arg);
+ if (err <= 0) {
+ PMD_DRV_LOG(ERR,
+ "snprintf cmd option invalid failed, err: %d.",
+ err);
+ }
+ return -UDA_EINVAL;
+ }
+ return UDA_SUCCESS;
+}
+
+static int
+major_command_parse(major_cmd_t *major_cmd, int argc, char **argv)
+{
+ int i, err;
+
+ for (i = 0; i < argc; i++) {
+ err = major_cmd_option_check(major_cmd, argv, &i);
+ if (err != UDA_SUCCESS)
+ return err;
+ }
+
+ return UDA_SUCCESS;
+}
+
+static int
+copy_reslut_to_buffer(void *buf_out, char *reslut, int len)
+{
+ int ret;
+
+ ret = snprintf(buf_out, len - 1, "%s", reslut);
+ if (ret <= 0)
+ return 0;
+
+ return ret + 1;
+}
+
+void
+command_parse(cmd_adapter_t *adapter, int argc, char **argv, void *buf_out,
+ uint32_t *out_len)
+{
+ int i;
+ major_cmd_t *major_cmd = NULL;
+ char *arg = argv[1];
+
+ if (is_help_version(adapter, argc, arg) == UDA_TRUE) {
+ *out_len = (u32)copy_reslut_to_buffer(buf_out,
+ adapter->show_str, MAX_SHOW_STR_LEN);
+ return;
+ }
+
+ for (i = 0; i < adapter->major_cmds; i++) {
+ major_cmd = adapter->p_major_cmd[i];
+
+ /* Find major command. */
+ if (!string_cmp(major_cmd->name, arg))
+ continue;
+ if (check_command_length(argc, argv) != UDA_SUCCESS) {
+ major_cmd->err_no = -UDA_EINVAL;
+ (void)snprintf(major_cmd->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "Command input too long.");
+ break;
+ }
+
+ /* Deal sub command. */
+ if (argc > SUB_COMMAND_OFFSET) {
+ if (major_command_parse(major_cmd,
+ argc - SUB_COMMAND_OFFSET,
+ argv + SUB_COMMAND_OFFSET) != UDA_SUCCESS) {
+ goto PARSE_OUT;
+ }
+ }
+
+ /* Command exec. */
+ major_cmd->execute(major_cmd);
+ break;
+ }
+
+ /* Not find command. */
+ if (i == adapter->major_cmds) {
+ hinic3_pmd_mml_log(adapter->show_str, &adapter->show_len,
+ "Unknown major command, assign 'evsadm exec "
+ "dump-hinic-status -h' for help.");
+ *out_len = (u32)copy_reslut_to_buffer(buf_out,
+ adapter->show_str, MAX_SHOW_STR_LEN);
+ return;
+ }
+
+PARSE_OUT:
+ if (major_cmd->err_no != UDA_SUCCESS &&
+ major_cmd->err_no != -UDA_CANCEL) {
+ PMD_DRV_LOG(ERR, "%s command error(%d): %s", major_cmd->name,
+ major_cmd->err_no, major_cmd->err_str);
+
+ hinic3_pmd_mml_log(major_cmd->show_str, &major_cmd->show_len,
+ "%s command error(%d): %s",
+ major_cmd->name, major_cmd->err_no,
+ major_cmd->err_str);
+ }
+ *out_len = (u32)copy_reslut_to_buffer(buf_out, major_cmd->show_str,
+ MAX_SHOW_STR_LEN);
+}
+
+void
+tool_target_init(int *bus_num, char *dev_name, int len)
+{
+ *bus_num = TRGET_UNKNOWN_BUS_NUM;
+ (void)memset(dev_name, 0, len);
+}
diff --git a/drivers/net/hinic3/mml/hinic3_mml_cmd.h b/drivers/net/hinic3/mml/hinic3_mml_cmd.h
new file mode 100644
index 0000000000..0e1ece38f0
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_cmd.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#ifndef _HINIC3_MML_CMD
+#define _HINIC3_MML_CMD
+
+#include <stdint.h>
+
+#define COMMAND_HELP_POSTION(argc) \
+ ({ \
+ typeof(argc) __argc = (argc); \
+ (__argc == 1 || __argc == 2); \
+ })
+#define COMMAND_VERSION_POSTION(argc) ((argc) == 2)
+#define SUB_COMMAND_OFFSET 2
+
+#define COMMAND_MAX_MAJORS 128
+#define COMMAND_MAX_OPTIONS 64
+#define PARAM_MAX_STRING 128
+#define COMMAND_MAX_STRING 512
+#define COMMANDER_ERR_MAX_STRING 128
+
+#define MAX_NAME_LEN 32
+#define MAX_DES_LEN 128
+#define MAX_SHOW_STR_LEN 2048
+
+struct tag_major_cmd_t;
+struct tag_cmd_adapter_t;
+
+typedef int (*command_record_t)(struct tag_major_cmd_t *major, char *param);
+typedef void (*command_executeute_t)(struct tag_major_cmd_t *major);
+
+typedef struct {
+ const char *little;
+ const char *large;
+ unsigned int have_param;
+ command_record_t record;
+} cmd_option_t;
+
+/* Major command structure for save command details and options. */
+typedef struct tag_major_cmd_t {
+ struct tag_cmd_adapter_t *adapter;
+ char name[MAX_NAME_LEN];
+ int option_count;
+ cmd_option_t options[COMMAND_MAX_OPTIONS];
+ uint32_t options_repeat_flag[COMMAND_MAX_OPTIONS];
+ command_executeute_t execute;
+ int err_no;
+ char err_str[COMMANDER_ERR_MAX_STRING];
+ char show_str[MAX_SHOW_STR_LEN];
+ int show_len;
+ char description[MAX_DES_LEN];
+ void *cmd_st; /**< Command show queue state structure. */
+} major_cmd_t;
+
+typedef struct tag_cmd_adapter_t {
+ const char *name;
+ const char *version;
+ major_cmd_t *p_major_cmd[COMMAND_MAX_MAJORS];
+ int major_cmds;
+ char show_str[MAX_SHOW_STR_LEN];
+ int show_len;
+ char *cmd_buf;
+} cmd_adapter_t;
+
+/**
+ * Add an option to a major command.
+ *
+ * This function adds a command option with its short and long forms, whether it
+ * requires a parameter, and the function to handle it.
+ *
+ * @param[in] major_cmd
+ * Pointer to the major command structure.
+ * @param[in] little
+ * Short form of the option.
+ * @param[in] large
+ * Long form of the option.
+ * @param[in] have_param
+ * Flag indicating whether the option requires a parameter.
+ * @param[in] record
+ * Function to handle the option's action.
+ */
+void major_command_option(major_cmd_t *major_cmd, const char *little,
+ const char *large, uint32_t have_param,
+ command_record_t record);
+
+/**
+ * Register a major command with adapter.
+ *
+ * @param[in] adapter
+ * Pointer to command adapter.
+ * @param[in] major_cmd
+ * The major command to be registered with the adapter.
+ */
+void major_command_register(cmd_adapter_t *adapter, major_cmd_t *major_cmd);
+
+/**
+ * Parse and execute commands.
+ *
+ * @param[in] adapter
+ * Pointer to command adapter.
+ * @param[in] argc
+ * The number of command arguments.
+ * @param[in] argv
+ * The array of command arguments.
+ * @param[out] buf_out
+ * The buffer used to store the output result.
+ * @param[out] out_len
+ * The length (in bytes) of the output result.
+ */
+void command_parse(cmd_adapter_t *adapter, int argc, char **argv, void *buf_out,
+ uint32_t *out_len);
+
+/**
+ * Initialize the target bus number and device name.
+ *
+ * @param[out] bus_num
+ * Pointer to the bus number, which will be set to a default unknown value.
+ * @param[out] dev_name
+ * Pointer to the device name buffer, which will be cleared (set to zeros).
+ * @param[in] len
+ * The length of the device name buffer.
+ */
+void tool_target_init(int *bus_num, char *dev_name, int len);
+
+int cmd_show_q_init(cmd_adapter_t *adapter);
+int cmd_show_xstats_init(cmd_adapter_t *adapter);
+int cmd_show_dump_init(cmd_adapter_t *adapter);
+
+#endif /* _HINIC3_MML_CMD */
diff --git a/drivers/net/hinic3/mml/hinic3_mml_ioctl.c b/drivers/net/hinic3/mml/hinic3_mml_ioctl.c
new file mode 100644
index 0000000000..0fd6b97f5e
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_ioctl.c
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+#include <rte_ethdev.h>
+#include <rte_pci.h>
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_ether.h>
+#include <rte_ethdev_core.h>
+#include "hinic3_mml_lib.h"
+#include "hinic3_dbg.h"
+#include "hinic3_compat.h"
+#include "hinic3_csr.h"
+#include "hinic3_hwdev.h"
+#include "hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+
+static int
+get_tx_info(struct rte_eth_dev *dev, void *buf_in, uint16_t in_size,
+ void *buf_out, uint16_t *out_size)
+{
+ uint16_t q_id = *((uint16_t *)buf_in);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (in_size != sizeof(int))
+ return -UDA_EINVAL;
+
+ return hinic3_dbg_get_sq_info(nic_dev->hwdev, q_id, buf_out, out_size);
+}
+
+static int
+get_tx_wqe_info(struct rte_eth_dev *dev, void *buf_in, uint16_t in_size,
+ void *buf_out, uint16_t *out_size)
+{
+ struct hinic_wqe_info *wqe_info = (struct hinic_wqe_info *)buf_in;
+ uint16_t q_id = (uint16_t)wqe_info->q_id;
+ uint16_t idx = (uint16_t)wqe_info->wqe_id;
+ uint16_t wqebb_cnt = (uint16_t)wqe_info->wqebb_cnt;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (in_size != sizeof(struct hinic_wqe_info))
+ return -UDA_EINVAL;
+
+ return hinic3_dbg_get_sq_wqe_info(nic_dev->hwdev, q_id, idx, wqebb_cnt,
+ buf_out, out_size);
+}
+
+static int
+get_rx_info(struct rte_eth_dev *dev, void *buf_in, uint16_t in_size,
+ void *buf_out, uint16_t *out_size)
+{
+ uint16_t q_id = *((uint16_t *)buf_in);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (in_size != sizeof(int))
+ return -UDA_EINVAL;
+
+ return hinic3_dbg_get_rq_info(nic_dev->hwdev, q_id, buf_out, out_size);
+}
+
+static int
+get_rx_wqe_info(struct rte_eth_dev *dev, void *buf_in, uint16_t in_size,
+ void *buf_out, uint16_t *out_size)
+{
+ struct hinic_wqe_info *wqe_info = (struct hinic_wqe_info *)buf_in;
+ uint16_t q_id = (uint16_t)wqe_info->q_id;
+ uint16_t idx = (uint16_t)wqe_info->wqe_id;
+ uint16_t wqebb_cnt = (uint16_t)wqe_info->wqebb_cnt;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (in_size != sizeof(struct hinic_wqe_info))
+ return -UDA_EINVAL;
+
+ return hinic3_dbg_get_rq_wqe_info(nic_dev->hwdev, q_id, idx, wqebb_cnt,
+ buf_out, out_size);
+}
+
+static int
+get_rx_cqe_info(struct rte_eth_dev *dev, void *buf_in, uint16_t in_size,
+ void *buf_out, uint16_t *out_size)
+{
+ struct hinic_wqe_info *wqe_info = (struct hinic_wqe_info *)buf_in;
+ uint16_t q_id = (uint16_t)wqe_info->q_id;
+ uint16_t idx = (uint16_t)wqe_info->wqe_id;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (in_size != sizeof(struct hinic_wqe_info))
+ return -UDA_EINVAL;
+
+ return hinic3_dbg_get_rx_cqe_info(nic_dev->hwdev, q_id, idx, buf_out,
+ out_size);
+}
+
+typedef int (*nic_drv_module)(struct rte_eth_dev *dev, void *buf_in,
+ uint16_t in_size, void *buf_out,
+ uint16_t *out_size);
+
+struct nic_drv_module_handle {
+ enum driver_cmd_type drv_cmd_name;
+ nic_drv_module drv_func;
+};
+
+const struct nic_drv_module_handle g_nic_drv_module_cmd_handle[] = {
+ {TX_INFO, get_tx_info}, {TX_WQE_INFO, get_tx_wqe_info},
+ {RX_INFO, get_rx_info}, {RX_WQE_INFO, get_rx_wqe_info},
+ {RX_CQE_INFO, get_rx_cqe_info},
+};
+
+static int
+send_to_nic_driver(struct rte_eth_dev *dev, struct msg_module *nt_msg)
+{
+ int index;
+ int err = 0;
+ enum driver_cmd_type cmd_type =
+ (enum driver_cmd_type)nt_msg->msg_formate;
+ int num_cmds = sizeof(g_nic_drv_module_cmd_handle) /
+ sizeof(g_nic_drv_module_cmd_handle[0]);
+
+ for (index = 0; index < num_cmds; index++) {
+ if (cmd_type ==
+ g_nic_drv_module_cmd_handle[index].drv_cmd_name) {
+ err = g_nic_drv_module_cmd_handle[index].drv_func(dev,
+ nt_msg->in_buf,
+ (uint16_t)nt_msg->buf_in_size, nt_msg->out_buf,
+ (uint16_t *)&nt_msg->buf_out_size);
+ break;
+ }
+ }
+
+ if (index == num_cmds) {
+ PMD_DRV_LOG(ERR, "Unknown nic driver cmd: %d", cmd_type);
+ err = -UDA_EINVAL;
+ }
+
+ return err;
+}
+
+static int
+hinic3_msg_handle(struct rte_eth_dev *dev, struct msg_module *nt_msg)
+{
+ int err;
+
+ switch (nt_msg->module) {
+ case SEND_TO_NIC_DRIVER:
+ err = send_to_nic_driver(dev, nt_msg);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Send message to driver failed");
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Unknown message module: %d", nt_msg->module);
+ err = -UDA_EINVAL;
+ break;
+ }
+
+ return err;
+}
+
+static struct rte_eth_dev *
+get_eth_dev_by_pci_addr(char *pci_addr, __rte_unused int len)
+{
+ uint32_t i;
+ struct rte_eth_dev *eth_dev = NULL;
+ struct rte_pci_device *pci_dev = NULL;
+ int ret;
+ uint32_t bus, devid, function;
+
+ ret = sscanf(pci_addr, "%02x:%02x.%x", &bus, &devid, &function);
+ if (ret <= 0) {
+ PMD_DRV_LOG(ERR,
+ "Get pci bus devid and function id fail, err: %d",
+ ret);
+ return NULL;
+ }
+
+ for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+ eth_dev = &rte_eth_devices[i];
+ if (eth_dev->state != RTE_ETH_DEV_ATTACHED)
+ continue;
+
+ pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+
+#ifdef CONFIG_SP_VID_DID
+ if (pci_dev->id.vendor_id == PCI_VENDOR_ID_SPNIC &&
+ (pci_dev->id.device_id == HINIC3_DEV_ID_STANDARD ||
+ pci_dev->id.device_id == HINIC3_DEV_ID_VF) &&
+#else
+ if (pci_dev->id.vendor_id == PCI_VENDOR_ID_HUAWEI &&
+ (pci_dev->id.device_id == HINIC3_DEV_ID_STANDARD ||
+ pci_dev->id.device_id == HINIC3_DEV_ID_VF) &&
+#endif
+ pci_dev->addr.bus == bus && pci_dev->addr.devid == devid &&
+ pci_dev->addr.function == function) {
+ return eth_dev;
+ }
+ }
+
+ return NULL;
+}
+
+int
+hinic3_pmd_mml_ioctl(void *msg)
+{
+ struct msg_module *nt_msg = msg;
+ struct rte_eth_dev *dev;
+
+ dev = get_eth_dev_by_pci_addr(nt_msg->device_name,
+ sizeof(nt_msg->device_name));
+ if (!dev) {
+ PMD_DRV_LOG(ERR, "Can not get the device %s correctly",
+ nt_msg->device_name);
+ return UDA_FAIL;
+ }
+
+ return hinic3_msg_handle(dev, nt_msg);
+}
diff --git a/drivers/net/hinic3/mml/hinic3_mml_lib.c b/drivers/net/hinic3/mml/hinic3_mml_lib.c
new file mode 100644
index 0000000000..dae2efc54b
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_lib.c
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+#include "hinic3_compat.h"
+#include "hinic3_mml_lib.h"
+
+int
+tool_get_valid_target(char *name, struct tool_target *target)
+{
+ int ret = UDA_SUCCESS;
+
+ if (strlen(name) >= MAX_DEV_LEN) {
+ PMD_DRV_LOG(ERR,
+ "Input parameter of device name is too long.");
+ ret = -UDA_ELEN;
+ } else {
+ (void)memcpy(target->dev_name, name, strlen(name));
+ target->bus_num = 0;
+ }
+
+ return ret;
+}
+
+static void
+fill_ioctl_msg_hd(struct msg_module *msg, unsigned int module,
+ unsigned int msg_formate, unsigned int in_buff_len,
+ unsigned int out_buff_len, char *dev_name, int bus_num)
+{
+ (void)memcpy(msg->device_name, dev_name, strlen(dev_name) + 1);
+
+ msg->module = module;
+ msg->msg_formate = msg_formate;
+ msg->buf_in_size = in_buff_len;
+ msg->buf_out_size = out_buff_len;
+ msg->bus_num = bus_num;
+}
+
+static int
+lib_ioctl(struct msg_module *in_buf, void *out_buf)
+{
+ in_buf->out_buf = out_buf;
+
+ return hinic3_pmd_mml_ioctl(in_buf);
+}
+
+int
+lib_tx_sq_info_get(struct tool_target target, struct nic_sq_info *sq_info,
+ int sq_id)
+{
+ struct msg_module msg_to_kernel;
+
+ (void)memset(&msg_to_kernel, 0, sizeof(msg_to_kernel));
+ fill_ioctl_msg_hd(&msg_to_kernel, SEND_TO_NIC_DRIVER, TX_INFO,
+ (unsigned int)sizeof(int),
+ (unsigned int)sizeof(struct nic_sq_info),
+ target.dev_name, target.bus_num);
+ msg_to_kernel.in_buf = (void *)&sq_id;
+
+ return lib_ioctl(&msg_to_kernel, sq_info);
+}
+
+int
+lib_tx_wqe_info_get(struct tool_target target, struct nic_sq_info *sq_info,
+ int sq_id, int wqe_id, void *nwqe, int nwqe_size)
+{
+ struct msg_module msg_to_kernel;
+ struct hinic_wqe_info wqe = {0};
+
+ wqe.wqe_id = wqe_id;
+ wqe.q_id = sq_id;
+ wqe.wqebb_cnt = nwqe_size / sq_info->sq_wqebb_size;
+
+ (void)memset(&msg_to_kernel, 0, sizeof(msg_to_kernel));
+ fill_ioctl_msg_hd(&msg_to_kernel, SEND_TO_NIC_DRIVER, TX_WQE_INFO,
+ (unsigned int)(sizeof(struct hinic_wqe_info)),
+ nwqe_size, target.dev_name, target.bus_num);
+ msg_to_kernel.in_buf = (void *)&wqe;
+
+ return lib_ioctl(&msg_to_kernel, nwqe);
+}
+
+int
+lib_rx_rq_info_get(struct tool_target target, struct nic_rq_info *rq_info,
+ int rq_id)
+{
+ struct msg_module msg_to_kernel;
+
+ (void)memset(&msg_to_kernel, 0, sizeof(msg_to_kernel));
+ fill_ioctl_msg_hd(&msg_to_kernel, SEND_TO_NIC_DRIVER, RX_INFO,
+ (unsigned int)(sizeof(int)),
+ (unsigned int)sizeof(struct nic_rq_info),
+ target.dev_name, target.bus_num);
+ msg_to_kernel.in_buf = &rq_id;
+
+ return lib_ioctl(&msg_to_kernel, rq_info);
+}
+
+int
+lib_rx_wqe_info_get(struct tool_target target, struct nic_rq_info *rq_info,
+ int rq_id, int wqe_id, void *nwqe, int nwqe_size)
+{
+ struct msg_module msg_to_kernel;
+ struct hinic_wqe_info wqe = {0};
+
+ wqe.wqe_id = wqe_id;
+ wqe.q_id = rq_id;
+ wqe.wqebb_cnt = nwqe_size / rq_info->rq_wqebb_size;
+
+ (void)memset(&msg_to_kernel, 0, sizeof(msg_to_kernel));
+ fill_ioctl_msg_hd(&msg_to_kernel, SEND_TO_NIC_DRIVER, RX_WQE_INFO,
+ (unsigned int)(sizeof(struct hinic_wqe_info)),
+ nwqe_size, target.dev_name, target.bus_num);
+ msg_to_kernel.in_buf = (void *)&wqe;
+
+ return lib_ioctl(&msg_to_kernel, nwqe);
+}
+
+int
+lib_rx_cqe_info_get(struct tool_target target,
+ __rte_unused struct nic_rq_info *rq_info, int rq_id,
+ int wqe_id, void *nwqe, int nwqe_size)
+{
+ struct msg_module msg_to_kernel;
+ struct hinic_wqe_info wqe = {0};
+
+ wqe.wqe_id = wqe_id;
+ wqe.q_id = rq_id;
+
+ (void)memset(&msg_to_kernel, 0, sizeof(msg_to_kernel));
+ fill_ioctl_msg_hd(&msg_to_kernel, SEND_TO_NIC_DRIVER, RX_CQE_INFO,
+ (unsigned int)(sizeof(struct hinic_wqe_info)),
+ nwqe_size, target.dev_name, target.bus_num);
+ msg_to_kernel.in_buf = (void *)&wqe;
+
+ return lib_ioctl(&msg_to_kernel, nwqe);
+}
diff --git a/drivers/net/hinic3/mml/hinic3_mml_lib.h b/drivers/net/hinic3/mml/hinic3_mml_lib.h
new file mode 100644
index 0000000000..42c365922f
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_lib.h
@@ -0,0 +1,275 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#ifndef _HINIC3_MML_LIB
+#define _HINIC3_MML_LIB
+
+#include <string.h>
+#include <stdint.h>
+
+#include "hinic3_mml_cmd.h"
+#include "hinic3_compat.h"
+#include "hinic3_mgmt.h"
+
+#define MAX_DEV_LEN 16
+#define TRGET_UNKNOWN_BUS_NUM (-1)
+
+#ifndef DEV_NAME_LEN
+#define DEV_NAME_LEN 64
+#endif
+
+enum {
+ UDA_SUCCESS = 0x0,
+ UDA_FAIL,
+ UDA_ENXIO,
+ UDA_ENONMEM,
+ UDA_EBUSY,
+ UDA_ECRC,
+ UDA_EINVAL,
+ UDA_EFAULT,
+ UDA_ELEN,
+ UDA_ECMD,
+ UDA_ENODRIVER,
+ UDA_EXIST,
+ UDA_EOVERSTEP,
+ UDA_ENOOBJ,
+ UDA_EOBJ,
+ UDA_ENOMATCH,
+ UDA_ETIMEOUT,
+
+ UDA_CONTOP,
+
+ UDA_REBOOT = 0xFD,
+ UDA_CANCEL = 0xFE,
+ UDA_KILLED = 0xFF,
+};
+
+#define PARAM_NEED 1
+#define PARAM_NOT_NEED 0
+
+#define BASE_ALL 0
+#define BASE_8 8
+#define BASE_10 10
+#define BASE_16 16
+
+enum module_name {
+ SEND_TO_NPU = 1,
+ SEND_TO_MPU,
+ SEND_TO_SM,
+
+ SEND_TO_HW_DRIVER,
+ SEND_TO_NIC_DRIVER,
+ SEND_TO_OVS_DRIVER,
+ SEND_TO_ROCE_DRIVER,
+ SEND_TO_TOE_DRIVER,
+ SEND_TO_IWAP_DRIVER,
+ SEND_TO_FC_DRIVER,
+ SEND_FCOE_DRIVER,
+};
+
+enum driver_cmd_type {
+ TX_INFO = 1,
+ Q_NUM,
+ TX_WQE_INFO,
+ TX_MAPPING,
+ RX_INFO,
+ RX_WQE_INFO,
+ RX_CQE_INFO
+};
+
+struct tool_target {
+ int bus_num;
+ char dev_name[MAX_DEV_LEN];
+ void *pri;
+};
+
+struct nic_tx_hw_page {
+ long long phy_addr;
+ long long *map_addr;
+};
+
+struct nic_sq_info {
+ unsigned short q_id;
+ unsigned short pi; /**< Ring buffer queue producer point. */
+ unsigned short ci; /**< Ring buffer queue consumer point. */
+ unsigned short fi; /**< Ring buffer queue complete point. */
+ unsigned int sq_depth;
+ unsigned short sq_wqebb_size;
+ unsigned short *ci_addr;
+ unsigned long long cla_addr;
+
+ struct nic_tx_hw_page doorbell;
+ unsigned int page_idx;
+};
+
+struct comm_info_l2nic_sq_ci_attr {
+ struct mgmt_msg_head msg_head;
+
+ uint16_t func_idx;
+ uint8_t dma_attr_off;
+ uint8_t pending_limit;
+
+ uint8_t coalescing_time;
+ uint8_t int_en;
+ uint16_t int_offset;
+
+ uint32_t l2nic_sqn;
+ uint32_t rsv;
+ uint64_t ci_addr;
+};
+
+struct nic_rq_info {
+ unsigned short q_id; /**< Queue id in current function, 0, 1, 2... */
+
+ unsigned short hw_pi; /**< Where pkt buf allocated. */
+ unsigned short ci; /**< Where hw pkt received, owned by hw. */
+ unsigned short sw_pi; /**< Where driver begin receive pkt. */
+ unsigned short rq_wqebb_size; /**< wqebb size, default to 32 bytes. */
+
+ unsigned short rq_depth;
+ unsigned short buf_len; /**< 2K. */
+ void *ci_wqe_page_addr; /**< For queue context init. */
+ void *ci_cla_tbl_addr;
+ unsigned short int_num; /**< RSS support should consider int_num. */
+ unsigned int msix_vector; /**< For debug. */
+};
+
+struct hinic_wqe_info {
+ int q_id;
+ void *slq_handle;
+ uint32_t wqe_id;
+ uint32_t wqebb_cnt;
+};
+
+struct npu_cmd_st {
+ uint32_t mod : 8;
+ uint32_t cmd : 8;
+ uint32_t ack_type : 3;
+ uint32_t direct_resp : 1;
+ uint32_t len : 12;
+};
+
+struct mpu_cmd_st {
+ uint32_t api_type : 8;
+ uint32_t mod : 8;
+ uint32_t cmd : 16;
+};
+
+struct msg_module {
+ char device_name[DEV_NAME_LEN];
+ uint32_t module;
+ union {
+ uint32_t msg_formate; /**< For driver. */
+ struct npu_cmd_st npu_cmd;
+ struct mpu_cmd_st mpu_cmd;
+ };
+ uint32_t timeout; /**< For mpu/npu cmd. */
+ uint32_t func_idx;
+ uint32_t buf_in_size;
+ uint32_t buf_out_size;
+ void *in_buf;
+ void *out_buf;
+ int bus_num;
+ uint32_t rsvd2[5];
+};
+
+/**
+ * Convert the provided string into `uint32_t` according to the specified base.
+ *
+ * @param[in] nptr
+ * The string to be converted.
+ * @param[in] base
+ * The base to use for conversion (e.g., 10 for decimal, 16 for hexadecimal).
+ * @param[out] value
+ * The output variable where the converted `uint32_t` value will be stored.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -UDA_EINVAL if the string is invalid or the value is out of range.
+ */
+static inline int
+string_toui(const char *nptr, int base, uint32_t *value)
+{
+ char *endptr = NULL;
+ long tmp_value;
+
+ tmp_value = strtol(nptr, &endptr, base);
+ if ((*endptr != 0) || tmp_value >= 0x7FFFFFFF || tmp_value < 0)
+ return -UDA_EINVAL;
+ *value = (uint32_t)tmp_value;
+ return UDA_SUCCESS;
+}
+
+#define UDA_TRUE 1
+#define UDA_FALSE 0
+
+/**
+ * Format and append a log message to a string buffer.
+ *
+ * @param[out] show_str
+ * The string buffer where the formatted message will be appended.
+ * @param[out] show_len
+ * The current length of the string in the buffer. It is updated after
+ * appending.
+ * @param[in] fmt
+ * The format string that specifies how to format the log message.
+ * @param[in] args
+ * The variable arguments to be formatted according to the format string.
+ */
+static inline void
+hinic3_pmd_mml_log(char *show_str, int *show_len, const char *fmt, ...)
+{
+ va_list args;
+ int ret = 0;
+
+ va_start(args, fmt);
+ ret = vsprintf(show_str + *show_len, fmt, args);
+ va_end(args);
+
+ if (ret > 0) {
+ *show_len += ret;
+ } else {
+ PMD_DRV_LOG(ERR, "MML show string snprintf failed, err: %d",
+ ret);
+ }
+}
+
+/**
+ * Get a valid target device based on the given name.
+ *
+ * This function checks if the device name is valid (within the length limit)
+ * and then stores it in the target structure. The bus number is initialized to
+ * 0.
+ *
+ * @param[in] name
+ * The device name to be validated and stored.
+ * @param[out] target
+ * The structure where the device name and bus number will be stored.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int tool_get_valid_target(char *name, struct tool_target *target);
+
+int hinic3_pmd_mml_ioctl(void *msg);
+
+int lib_tx_sq_info_get(struct tool_target target, struct nic_sq_info *sq_info,
+ int sq_id);
+
+int lib_tx_wqe_info_get(struct tool_target target, struct nic_sq_info *sq_info,
+ int sq_id, int wqe_id, void *nwqe, int nwqe_size);
+
+int lib_rx_rq_info_get(struct tool_target target, struct nic_rq_info *rq_info,
+ int rq_id);
+
+int lib_rx_wqe_info_get(struct tool_target target, struct nic_rq_info *rq_info,
+ int rq_id, int wqe_id, void *nwqe, int nwqe_size);
+
+int lib_rx_cqe_info_get(struct tool_target target, struct nic_rq_info *rq_info,
+ int rq_id, int wqe_id, void *nwqe, int nwqe_size);
+
+int hinic3_pmd_mml_lib(const char *buf_in, uint32_t in_size, char *buf_out,
+ uint32_t *out_len, uint32_t max_buf_out_len);
+
+#endif /* _HINIC3_MML_LIB */
diff --git a/drivers/net/hinic3/mml/hinic3_mml_main.c b/drivers/net/hinic3/mml/hinic3_mml_main.c
new file mode 100644
index 0000000000..7830df479e
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_main.c
@@ -0,0 +1,167 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#include "hinic3_mml_lib.h"
+#include "hinic3_mml_cmd.h"
+
+#define MAX_ARGC 20
+
+/**
+ * Free all memory associated with the command adapter, including the command
+ * states and command buffer.
+ *
+ * @param[in] adapter
+ * Pointer to command adapter.
+ */
+static void
+cmd_deinit(cmd_adapter_t *adapter)
+{
+ int i;
+
+ for (i = 0; i < COMMAND_MAX_MAJORS; i++) {
+ if (adapter->p_major_cmd[i]) {
+ if (adapter->p_major_cmd[i]->cmd_st) {
+ free(adapter->p_major_cmd[i]->cmd_st);
+ adapter->p_major_cmd[i]->cmd_st = NULL;
+ }
+
+ free(adapter->p_major_cmd[i]);
+ adapter->p_major_cmd[i] = NULL;
+ }
+ }
+
+ if (adapter->cmd_buf) {
+ free(adapter->cmd_buf);
+ adapter->cmd_buf = NULL;
+ }
+}
+
+static int
+cmd_init(cmd_adapter_t *adapter)
+{
+ int err;
+
+ err = cmd_show_q_init(adapter);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Init cmd show queue fail");
+ return err;
+ }
+
+ return UDA_SUCCESS;
+}
+
+/**
+ * Separate the input command string into arguments.
+ *
+ * @param[in] adapter
+ * Pointer to command adapter.
+ * @param[in] buf_in
+ * The input command string.
+ * @param[in] in_size
+ * The size of the input command string.
+ * @param[out] argv
+ * The array to store separated arguments.
+ *
+ * @return
+ * The number of arguments on success, a negative error code otherwise.
+ */
+static int
+cmd_separate(cmd_adapter_t *adapter, const char *buf_in, uint32_t in_size,
+ char **argv)
+{
+ char *cmd_buf = NULL;
+ char *tmp = NULL;
+ char *saveptr = NULL;
+ int i;
+
+ cmd_buf = calloc(1, in_size + 1);
+ if (!cmd_buf) {
+ PMD_DRV_LOG(ERR, "Failed to allocate cmd_buf");
+ return -UDA_ENONMEM;
+ }
+
+ (void)memcpy(cmd_buf, buf_in, in_size);
+
+ tmp = cmd_buf;
+ for (i = 1; i < MAX_ARGC; i++) {
+ argv[i] = strtok_r(tmp, " ", &saveptr);
+ if (!argv[i])
+ break;
+ tmp = NULL;
+ }
+
+ if (i == MAX_ARGC) {
+ PMD_DRV_LOG(ERR, "Parameters is too many");
+ free(cmd_buf);
+ return -UDA_FAIL;
+ }
+
+ adapter->cmd_buf = cmd_buf;
+ return i;
+}
+
+/**
+ * Process the input command string, parse arguments, and return the result.
+ *
+ * @param[in] buf_in
+ * The input command string.
+ * @param[in] in_size
+ * The size of the input command string.
+ * @param[out] buf_out
+ * The output buffer to store the command result.
+ * @param[out] out_len
+ * The length of the output buffer.
+ * @param[in] max_buf_out_len
+ * The maximum size of the output buffer.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int
+hinic3_pmd_mml_lib(const char *buf_in, uint32_t in_size, char *buf_out,
+ uint32_t *out_len, uint32_t max_buf_out_len)
+{
+ cmd_adapter_t *adapter = NULL;
+ char *argv[MAX_ARGC];
+ int argc;
+ int err = -UDA_EINVAL;
+
+ if (!buf_in || !in_size) {
+ PMD_DRV_LOG(ERR, "Invalid param, buf_in: %d, in_size: 0x%x",
+ !!buf_in, in_size);
+ return err;
+ }
+
+ if (!buf_out || max_buf_out_len < MAX_SHOW_STR_LEN) {
+ PMD_DRV_LOG(ERR,
+ "Invalid param, buf_out: %d, max_buf_out_len: 0x%x",
+ !!buf_out, max_buf_out_len);
+ return err;
+ }
+
+ adapter = calloc(1, sizeof(cmd_adapter_t));
+ if (!adapter) {
+ PMD_DRV_LOG(ERR, "Failed to allocate cmd adapter");
+ return -UDA_ENONMEM;
+ }
+
+ err = cmd_init(adapter);
+ if (err != 0)
+ goto parse_cmd_fail;
+
+ argc = cmd_separate(adapter, buf_in, in_size, argv);
+ if (argc < 0) {
+ err = -UDA_FAIL;
+ goto parse_cmd_fail;
+ }
+
+ (void)memset(buf_out, 0, max_buf_out_len);
+ command_parse(adapter, argc, argv, buf_out, out_len);
+
+parse_cmd_fail:
+ cmd_deinit(adapter);
+ free(adapter);
+
+ return err;
+}
diff --git a/drivers/net/hinic3/mml/hinic3_mml_queue.c b/drivers/net/hinic3/mml/hinic3_mml_queue.c
new file mode 100644
index 0000000000..7d29c7ea52
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_queue.c
@@ -0,0 +1,749 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ */
+
+#include "hinic3_mml_lib.h"
+#include "hinic3_mml_cmd.h"
+#include "hinic3_mml_queue.h"
+
+#define ADDR_HI_BIT 32
+
+/**
+ * This function perform similar operations as `hinic3_pmd_mml_log`, but it
+ * return a code.
+ *
+ * @param[out] show_str
+ * The string buffer where the formatted message will be appended.
+ * @param[out] show_len
+ * The current length of the string in the buffer. It is updated after
+ * appending.
+ * @param[in] fmt
+ * The format string that specifies how to format the log message.
+ * @param[in] args
+ * The variable arguments to be formatted according to the format string.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - `-UDA_EINVAL` if an error occurs during the formatting process.
+ *
+ * @see hinic3_pmd_mml_log
+ */
+static int
+hinic3_pmd_mml_log_ret(char *show_str, int *show_len, const char *fmt, ...)
+{
+ va_list args;
+ int ret = 0;
+
+ va_start(args, fmt);
+ ret = vsprintf(show_str + *show_len, fmt, args);
+ va_end(args);
+
+ if (ret > 0) {
+ *show_len += ret;
+ } else {
+ PMD_DRV_LOG(ERR, "MML show string snprintf failed, err: %d",
+ ret);
+ return -UDA_EINVAL;
+ }
+
+ return UDA_SUCCESS;
+}
+
+/**
+ * Format and log the information about the RQ by appending details such as
+ * queue ID, ci, sw pi, RQ depth, RQ WQE buffer size, buffer length, interrupt
+ * number, and MSIX vector to the output buffer.
+ *
+ * @param[in] self
+ * Pointer to major command structure.
+ * @param[in] rq_info
+ * The receive queue information to be displayed, which includes various
+ * properties like queue ID, depth, interrupt number, etc.
+ */
+static void
+rx_show_rq_info(major_cmd_t *self, struct nic_rq_info *rq_info)
+{
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "Receive queue information:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "queue_id:%u",
+ rq_info->q_id);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "ci:%u",
+ rq_info->ci);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "sw_pi:%u",
+ rq_info->sw_pi);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rq_depth:%u",
+ rq_info->rq_depth);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "rq_wqebb_size:%u", rq_info->rq_wqebb_size);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "buf_len:%u",
+ rq_info->buf_len);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "int_num:%u",
+ rq_info->int_num);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "msix_vector:%u",
+ rq_info->msix_vector);
+}
+
+static void
+rx_show_wqe(major_cmd_t *self, nic_rq_wqe *wqe)
+{
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "Rx buffer section information:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "buf_addr:0x%" PRIx64,
+ (((uint64_t)wqe->buf_desc.pkt_buf_addr_high) << ADDR_HI_BIT) |
+ wqe->buf_desc.pkt_buf_addr_low);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "buf_len:%u",
+ wqe->buf_desc.len);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd0:%u",
+ wqe->rsvd0);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "Cqe buffer section information:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "buf_hi:0x%" PRIx64,
+ (((uint64_t)wqe->cqe_sect.pkt_buf_addr_high) << ADDR_HI_BIT) |
+ wqe->cqe_sect.pkt_buf_addr_low);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "buf_len:%u",
+ wqe->cqe_sect.len);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd1:%u",
+ wqe->rsvd1);
+}
+
+static void
+rx_show_cqe_info(major_cmd_t *self, struct tag_l2nic_rx_cqe *wqe_cs)
+{
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "Rx cqe info:");
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw0:0x%08x",
+ wqe_cs->dw0.value);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rx_done:0x%x",
+ wqe_cs->dw0.bs.rx_done);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "bp_en:0x%x",
+ wqe_cs->dw0.bs.bp_en);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "decry_pkt:0x%x",
+ wqe_cs->dw0.bs.decry_pkt);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "flush:0x%x",
+ wqe_cs->dw0.bs.flush);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "spec_flags:0x%x",
+ wqe_cs->dw0.bs.spec_flags);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd0:0x%x",
+ wqe_cs->dw0.bs.rsvd0);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "lro_num:0x%x",
+ wqe_cs->dw0.bs.lro_num);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "checksum_err:0x%x", wqe_cs->dw0.bs.checksum_err);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw1:0x%08x",
+ wqe_cs->dw1.value);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "length:%u",
+ wqe_cs->dw1.bs.length);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "vlan:0x%x",
+ wqe_cs->dw1.bs.vlan);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw2:0x%08x",
+ wqe_cs->dw2.value);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rss_type:0x%x",
+ wqe_cs->dw2.bs.rss_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd0:0x%x",
+ wqe_cs->dw2.bs.rsvd0);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "vlan_offload_en:0x%x",
+ wqe_cs->dw2.bs.vlan_offload_en);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "umbcast:0x%x",
+ wqe_cs->dw2.bs.umbcast);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd1:0x%x",
+ wqe_cs->dw2.bs.rsvd1);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "pkt_types:0x%x",
+ wqe_cs->dw2.bs.pkt_types);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "rss_hash_value:0x%08x",
+ wqe_cs->dw3.bs.rss_hash_value);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw4:0x%08x",
+ wqe_cs->dw4.value);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw5:0x%08x",
+ wqe_cs->dw5.value);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "mac_type:0x%x",
+ wqe_cs->dw5.ovs_bs.mac_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "l3_type:0x%x",
+ wqe_cs->dw5.ovs_bs.l3_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "l4_type:0x%x",
+ wqe_cs->dw5.ovs_bs.l4_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd0:0x%x",
+ wqe_cs->dw5.ovs_bs.rsvd0);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "traffic_type:0x%x",
+ wqe_cs->dw5.ovs_bs.traffic_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "traffic_from:0x%x",
+ wqe_cs->dw5.ovs_bs.traffic_from);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "cs_dw6:0x%08x",
+ wqe_cs->dw6.value);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "localtag:0x%08x",
+ wqe_cs->dw7.ovs_bs.localtag);
+}
+
+#define HINIC3_PMD_MML_LOG_RET(fmt, ...) \
+ hinic3_pmd_mml_log_ret(self->show_str, &self->show_len, fmt, \
+ ##__VA_ARGS__)
+
+/**
+ * Display help information for queue command.
+ *
+ * @param[in] self
+ * Pointer to major command structure.
+ * @param[in] argc
+ * A string representing the value associated with the command option (unused_).
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+cmd_queue_help(major_cmd_t *self, __rte_unused char *argc)
+{
+ int ret;
+ ret = HINIC3_PMD_MML_LOG_RET("") ||
+ HINIC3_PMD_MML_LOG_RET(" Usage: %s %s", self->name,
+ "-i <device> -d <tx or rx> -t <type> "
+ "-q <queue id> [-w <wqe id>]") ||
+ HINIC3_PMD_MML_LOG_RET("\n %s", self->description) ||
+ HINIC3_PMD_MML_LOG_RET("\n Options:\n") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-h", "--help",
+ "display this help and exit") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-i",
+ "--device=<device>",
+ "device target, e.g. 08:00.0") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-d", "--direction",
+ "tx or rx") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", " ", "", "0: tx") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", " ", "", "1: rx") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-t", "--type", "") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", " ", "",
+ "0: queue info") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", " ", "",
+ "1: wqe info") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", " ", "",
+ "2: cqe info(only for rx)") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-q", "--queue_id",
+ "") ||
+ HINIC3_PMD_MML_LOG_RET(" %s, %-25s %s", "-w", "--wqe_id", "") ||
+ HINIC3_PMD_MML_LOG_RET("");
+
+ return ret;
+}
+
+static void
+tx_show_sq_info(major_cmd_t *self, struct nic_sq_info *sq_info)
+{
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "Send queue information:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "queue_id:%u",
+ sq_info->q_id);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "pi:%u",
+ sq_info->pi);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "ci:%u",
+ sq_info->ci);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "fi:%u",
+ sq_info->fi);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "sq_depth:%u",
+ sq_info->sq_depth);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "sq_wqebb_size:%u", sq_info->sq_wqebb_size);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "cla_addr:0x%" PRIu64,
+ sq_info->cla_addr);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "doorbell phy_addr:0x%" PRId64,
+ (uintptr_t)sq_info->doorbell.phy_addr);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "page_idx:%u",
+ sq_info->page_idx);
+}
+
+static void
+tx_show_wqe(major_cmd_t *self, struct nic_tx_wqe_desc *wqe)
+{
+ struct nic_tx_ctrl_section *control = NULL;
+ struct nic_tx_task_section *task = NULL;
+ unsigned int *val = (unsigned int *)wqe;
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw0:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw1:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw2:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw3:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw4:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw5:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw6:0x%08x",
+ *(val++));
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "dw7:0x%08x",
+ *(val++));
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "\nWqe may analyse as follows:");
+ control = &wqe->control;
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "\nInformation about wqe control section:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "ctrl_format:0x%08x", control->ctrl_format);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "owner:%u",
+ control->ctrl_sec.o);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "extended_compact:%u", control->ctrl_sec.ec);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "direct_normal:%u", control->ctrl_sec.dn);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "inline_sgl:%u",
+ control->ctrl_sec.df);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "ts_size:%u",
+ control->ctrl_sec.tss);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "bds_len:%u",
+ control->ctrl_sec.bdsl);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd:%u",
+ control->ctrl_sec.r);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "1st_buf_len:%u",
+ control->ctrl_sec.len);
+
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "queue_info:0x%08x", control->queue_info);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "pri:%u",
+ control->qsf.pri);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "uc:%u",
+ control->qsf.uc);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "sctp:%u",
+ control->qsf.sctp);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "mss:%u",
+ control->qsf.mss);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "tcp_udp_cs:%u",
+ control->qsf.tcp_udp_cs);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "tso:%u",
+ control->qsf.tso);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "ufo:%u",
+ control->qsf.ufo);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "payload_offset:%u", control->qsf.payload_offset);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "pkt_type:%u",
+ control->qsf.pkt_type);
+
+ /* First buffer section. */
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "bd0_hi_addr:0x%08x", wqe->bd0_hi_addr);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "bd0_lo_addr:0x%08x", wqe->bd0_lo_addr);
+
+ /* Show the task section. */
+ task = &wqe->task;
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "\nInformation about wqe task section:");
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "vport_id:%u",
+ task->bs2.vport_id);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "vport_type:%u",
+ task->bs2.vport_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "traffic_type:%u",
+ task->bs2.traffic_type);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "slave_port_id:%u", task->bs2.slave_port_id);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "rsvd0:%u",
+ task->bs2.rsvd0);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "crypto_en:%u",
+ task->bs2.crypto_en);
+ hinic3_pmd_mml_log(self->show_str, &self->show_len, "pkt_type:%u",
+ task->bs2.pkt_type);
+}
+
+static int
+cmd_queue_target(major_cmd_t *self, char *argc)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ int ret;
+
+ if (tool_get_valid_target(argc, &show_q->target) != UDA_SUCCESS) {
+ self->err_no = -UDA_EINVAL;
+ ret = snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Unknown device %s.", argc);
+ if (ret <= 0) {
+ PMD_DRV_LOG(ERR,
+ "snprintf queue err msg failed, ret: %d",
+ ret);
+ }
+ return -UDA_EINVAL;
+ }
+
+ return UDA_SUCCESS;
+}
+
+static int
+get_queue_type(major_cmd_t *self, char *argc)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ unsigned int num = 0;
+
+ if (string_toui(argc, BASE_10, &num) != UDA_SUCCESS) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Unknown queuetype %u.", num);
+ return -UDA_EINVAL;
+ }
+
+ show_q->qobj = (int)num;
+ return UDA_SUCCESS;
+}
+
+static int
+get_queue_id(major_cmd_t *self, char *argc)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ unsigned int num = 0;
+
+ if (string_toui(argc, BASE_10, &num) != UDA_SUCCESS) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Invalid queue id.");
+ return -UDA_EINVAL;
+ }
+
+ show_q->q_id = (int)num;
+ return UDA_SUCCESS;
+}
+
+static int
+get_q_wqe_id(major_cmd_t *self, char *argc)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ unsigned int num = 0;
+
+ if (string_toui(argc, BASE_10, &num) != UDA_SUCCESS) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Invalid wqe id.");
+ return -UDA_EINVAL;
+ }
+
+ show_q->wqe_id = (int)num;
+ return UDA_SUCCESS;
+}
+
+/**
+ * Set direction for queue query.
+ *
+ * @param[in] self
+ * Pointer to major command structure.
+ * @param[in] argc
+ * The input argument representing the direction (as a string).
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -UDA_EINVAL If the input is invalid (not a number or out of range), it sets
+ * an error in `err_no` and `err_str`.
+ */
+static int
+get_direction(major_cmd_t *self, char *argc)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ unsigned int num = 0;
+
+ if (string_toui(argc, BASE_10, &num) != UDA_SUCCESS || num > 1) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Unknown mode.");
+ return -UDA_EINVAL;
+ }
+
+ show_q->direction = (int)num;
+ return UDA_SUCCESS;
+}
+
+static int
+rx_param_check(major_cmd_t *self, struct cmd_show_q_st *rx_param)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+
+ if (rx_param->target.bus_num == TRGET_UNKNOWN_BUS_NUM) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Need device name.");
+ return self->err_no;
+ }
+
+ if (show_q->qobj > OBJ_CQE_INFO || show_q->qobj < OBJ_Q_INFO) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Unknown queue type.");
+ return self->err_no;
+ }
+
+ if (show_q->q_id == -1) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Need queue id.");
+ return self->err_no;
+ }
+
+ if (show_q->qobj != OBJ_Q_INFO && show_q->wqe_id == -1) {
+ self->err_no = -UDA_FAIL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get cqe_info or wqe_info, must set wqeid.");
+ return -UDA_FAIL;
+ }
+
+ if (show_q->qobj == OBJ_Q_INFO && show_q->wqe_id != -1) {
+ self->err_no = -UDA_FAIL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get queue info, need not set wqeid.");
+ return -UDA_FAIL;
+ }
+
+ return UDA_SUCCESS;
+}
+
+static int
+tx_param_check(major_cmd_t *self, struct cmd_show_q_st *tx_param)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+
+ if (tx_param->target.bus_num == TRGET_UNKNOWN_BUS_NUM) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Need device name.");
+ return self->err_no;
+ }
+
+ if (show_q->qobj > OBJ_WQE_INFO || show_q->qobj < OBJ_Q_INFO) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Unknown queue type.");
+ return self->err_no;
+ }
+
+ if (show_q->q_id == -1) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Need queue id.");
+ return self->err_no;
+ }
+
+ if (show_q->qobj == OBJ_WQE_INFO && show_q->wqe_id == -1) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get wqe_info, must set wqeid.");
+ return self->err_no;
+ }
+
+ if (show_q->qobj != OBJ_WQE_INFO && show_q->wqe_id != -1) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get queue info, need not set wqeid.");
+ return self->err_no;
+ }
+
+ return UDA_SUCCESS;
+}
+
+static void
+cmd_tx_execute(major_cmd_t *self)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+ int ret;
+ struct nic_sq_info sq_info = {0};
+ struct nic_tx_wqe_desc nwqe;
+
+ if (tx_param_check(self, show_q) != UDA_SUCCESS)
+ return;
+
+ if (show_q->qobj == OBJ_Q_INFO || show_q->qobj == OBJ_WQE_INFO) {
+ ret = lib_tx_sq_info_get(show_q->target, (void *)&sq_info,
+ show_q->q_id);
+ if (ret != UDA_SUCCESS) {
+ self->err_no = ret;
+ (void)snprintf(self->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "Get tx sq_info failed.");
+ return;
+ }
+
+ if (show_q->qobj == OBJ_Q_INFO) {
+ tx_show_sq_info(self, &sq_info);
+ return;
+ }
+
+ if (show_q->wqe_id >= (int)sq_info.sq_depth) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "Max wqe id is %u.",
+ sq_info.sq_depth - 1);
+ return;
+ }
+
+ (void)memset(&nwqe, 0, sizeof(nwqe));
+ ret = lib_tx_wqe_info_get(show_q->target, &sq_info,
+ show_q->q_id, show_q->wqe_id,
+ (void *)&nwqe, sizeof(nwqe));
+ if (ret != UDA_SUCCESS) {
+ self->err_no = ret;
+ (void)snprintf(self->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "Get tx wqe_info failed.");
+ return;
+ }
+
+ tx_show_wqe(self, &nwqe);
+ return;
+ }
+}
+
+static void
+cmd_rx_execute(major_cmd_t *self)
+{
+ int ret;
+ struct nic_rq_info rq_info = {0};
+ struct tag_l2nic_rx_cqe cqe;
+ nic_rq_wqe wqe;
+ struct cmd_show_q_st *show_q = self->cmd_st;
+
+ if (rx_param_check(self, show_q) != UDA_SUCCESS)
+ return;
+
+ ret = lib_rx_rq_info_get(show_q->target, &rq_info, show_q->q_id);
+ if (ret != UDA_SUCCESS) {
+ self->err_no = ret;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get rx rq_info failed.");
+ return;
+ }
+
+ if (show_q->qobj == OBJ_Q_INFO) {
+ rx_show_rq_info(self, &rq_info);
+ return;
+ }
+
+ if ((uint32_t)show_q->wqe_id >= rq_info.rq_depth) {
+ self->err_no = -UDA_EINVAL;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Max wqe id is %u.", rq_info.rq_depth - 1);
+ return;
+ }
+
+ if (show_q->qobj == OBJ_WQE_INFO) {
+ (void)memset(&wqe, 0, sizeof(wqe));
+ ret = lib_rx_wqe_info_get(show_q->target, &rq_info,
+ show_q->q_id, show_q->wqe_id,
+ (void *)&wqe, sizeof(wqe));
+ if (ret != UDA_SUCCESS) {
+ self->err_no = ret;
+ (void)snprintf(self->err_str,
+ COMMANDER_ERR_MAX_STRING - 1,
+ "Get rx wqe_info failed.");
+ return;
+ }
+
+ rx_show_wqe(self, &wqe);
+ return;
+ }
+
+ /* OBJ_CQE_INFO */
+ (void)memset(&cqe, 0, sizeof(cqe));
+ ret = lib_rx_cqe_info_get(show_q->target, &rq_info, show_q->q_id,
+ show_q->wqe_id, (void *)&cqe, sizeof(cqe));
+ if (ret != UDA_SUCCESS) {
+ self->err_no = ret;
+ (void)snprintf(self->err_str, COMMANDER_ERR_MAX_STRING - 1,
+ "Get rx cqe_info failed.");
+ return;
+ }
+
+ rx_show_cqe_info(self, &cqe);
+}
+
+/**
+ * Execute the NIC queue query command based on the direction.
+ *
+ * @param[in] self
+ * Pointer to major command structure.
+ */
+static void
+cmd_nic_queue_execute(major_cmd_t *self)
+{
+ struct cmd_show_q_st *show_q = self->cmd_st;
+
+ if (show_q->direction == -1) {
+ hinic3_pmd_mml_log(self->show_str, &self->show_len,
+ "Need -d parameter.");
+ return;
+ }
+
+ if (show_q->direction == 0)
+ cmd_tx_execute(self);
+ else
+ cmd_rx_execute(self);
+}
+
+/**
+ * Initialize and register the queue query command.
+ *
+ * @param[in] adapter
+ * The command adapter, which holds the registered commands and their states.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ * - -UDA_ENONMEM if memory allocation fail or an error occur.
+ */
+int
+cmd_show_q_init(cmd_adapter_t *adapter)
+{
+ struct cmd_show_q_st *show_q = NULL;
+ major_cmd_t *show_q_cmd;
+
+ show_q_cmd = calloc(1, sizeof(*show_q_cmd));
+ if (!show_q_cmd) {
+ PMD_DRV_LOG(ERR, "Failed to allocate queue cmd");
+ return -UDA_ENONMEM;
+ }
+
+ (void)snprintf(show_q_cmd->name, MAX_NAME_LEN - 1, "%s", "nic_queue");
+ (void)snprintf(show_q_cmd->description,
+ MAX_DES_LEN - 1, "%s",
+ "Query the rx/tx queue information of a specified pci_addr");
+
+ show_q_cmd->option_count = 0;
+ show_q_cmd->execute = cmd_nic_queue_execute;
+
+ show_q = calloc(1, sizeof(*show_q));
+ if (!show_q) {
+ free(show_q_cmd);
+ PMD_DRV_LOG(ERR, "Failed to allocate show queue");
+ return -UDA_ENONMEM;
+ }
+
+ show_q->qobj = -1;
+ show_q->q_id = -1;
+ show_q->wqe_id = -1;
+ show_q->direction = -1;
+
+ show_q_cmd->cmd_st = show_q;
+
+ tool_target_init(&show_q->target.bus_num, show_q->target.dev_name,
+ MAX_DEV_LEN);
+
+ major_command_option(show_q_cmd, "-h", "--help", PARAM_NOT_NEED,
+ cmd_queue_help);
+ major_command_option(show_q_cmd, "-i", "--device", PARAM_NEED,
+ cmd_queue_target);
+ major_command_option(show_q_cmd, "-t", "--type", PARAM_NEED,
+ get_queue_type);
+ major_command_option(show_q_cmd, "-q", "--queue_id", PARAM_NEED,
+ get_queue_id);
+ major_command_option(show_q_cmd, "-w", "--wqe_id", PARAM_NEED,
+ get_q_wqe_id);
+ major_command_option(show_q_cmd, "-d", "--direction", PARAM_NEED,
+ get_direction);
+
+ major_command_register(adapter, show_q_cmd);
+
+ return UDA_SUCCESS;
+}
diff --git a/drivers/net/hinic3/mml/hinic3_mml_queue.h b/drivers/net/hinic3/mml/hinic3_mml_queue.h
new file mode 100644
index 0000000000..633b1db50c
--- /dev/null
+++ b/drivers/net/hinic3/mml/hinic3_mml_queue.h
@@ -0,0 +1,256 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) Huawei Technologies Co., Ltd. 2025. All rights reserved.
+ * Description : hinic3 mml for queue
+ */
+
+#ifndef _HINIC3_MML_QUEUE
+#define _HINIC3_MML_QUEUE
+
+#define OBJ_Q_INFO 0
+#define OBJ_WQE_INFO 1
+#define OBJ_CQE_INFO 2
+
+/* TX. */
+struct nic_tx_ctrl_section {
+ union {
+ struct {
+ unsigned int len : 18;
+ unsigned int r : 1;
+ unsigned int bdsl : 8;
+ unsigned int tss : 1;
+ unsigned int df : 1;
+ unsigned int dn : 1;
+ unsigned int ec : 1;
+ unsigned int o : 1;
+ } ctrl_sec;
+ unsigned int ctrl_format;
+ };
+ union {
+ struct {
+ unsigned int pkt_type : 2;
+ unsigned int payload_offset : 8;
+ unsigned int ufo : 1;
+ unsigned int tso : 1;
+ unsigned int tcp_udp_cs : 1;
+ unsigned int mss : 14;
+ unsigned int sctp : 1;
+ unsigned int uc : 1;
+ unsigned int pri : 3;
+ } qsf;
+ unsigned int queue_info;
+ };
+};
+
+struct nic_tx_task_section {
+ unsigned int dw0;
+ unsigned int dw1;
+
+ /* dw2. */
+ union {
+ struct {
+ /*
+ * When TX direct, output bond id;
+ * when RX direct, output function id.
+ */
+ unsigned int vport_id : 12;
+ unsigned int vport_type : 4;
+ unsigned int traffic_type : 6;
+ /*
+ * Only used in TX direct, ctrl pkt(LACP\LLDP) output
+ * port id.
+ */
+ unsigned int slave_port_id : 2;
+ unsigned int rsvd0 : 6;
+ unsigned int crypto_en : 1;
+ unsigned int pkt_type : 1;
+ } bs2;
+ unsigned int dw2;
+ };
+
+ unsigned int dw3;
+};
+
+struct nic_tx_sge {
+ union {
+ struct {
+ unsigned int length : 31; /**< SGE length. */
+ unsigned int rsvd : 1;
+ } bs0;
+ unsigned int dw0;
+ };
+
+ union {
+ struct {
+ /* Key or unused. */
+ unsigned int key : 30;
+ /* 0:normal, 1:pointer to next SGE. */
+ unsigned int extension : 1;
+ /* 0:list, 1:last. */
+ unsigned int list : 1;
+ } bs1;
+ unsigned int dw1;
+ };
+
+ unsigned int dma_addr_high;
+ unsigned int dma_addr_low;
+};
+
+struct nic_tx_wqe_desc {
+ struct nic_tx_ctrl_section control;
+ struct nic_tx_task_section task;
+ unsigned int bd0_hi_addr;
+ unsigned int bd0_lo_addr;
+};
+
+/* RX. */
+typedef struct tag_l2nic_rx_cqe {
+ union {
+ struct {
+ unsigned int checksum_err : 16;
+ unsigned int lro_num : 8;
+ unsigned int rsvd0 : 1;
+ unsigned int spec_flags : 3;
+ unsigned int flush : 1;
+ unsigned int decry_pkt : 1;
+ unsigned int bp_en : 1;
+ unsigned int rx_done : 1;
+ } bs;
+ unsigned int value;
+ } dw0;
+
+ union {
+ struct {
+ unsigned int vlan : 16;
+ unsigned int length : 16;
+ } bs;
+ unsigned int value;
+ } dw1;
+
+ union {
+ struct {
+ unsigned int pkt_types : 12;
+ unsigned int rsvd1 : 7;
+ unsigned int umbcast : 2;
+ unsigned int vlan_offload_en : 1;
+ unsigned int rsvd0 : 2;
+ unsigned int rss_type : 8;
+ } bs;
+ unsigned int value;
+ } dw2;
+
+ union {
+ struct {
+ unsigned int rss_hash_value;
+ } bs;
+ unsigned int value;
+ } dw3;
+
+ /* dw4~dw7 field for nic/ovs multipexing. */
+ union {
+ struct { /**< For nic. */
+ unsigned int tx_ts_seq : 16;
+ unsigned int msg_1588_offset : 8;
+ unsigned int msg_1588_type : 4;
+ unsigned int rsvd : 1;
+ unsigned int if_rx_ts : 1;
+ unsigned int if_tx_ts : 1;
+ unsigned int if_1588 : 1;
+ } bs;
+
+ struct { /**< For ovs. */
+ unsigned int reserved;
+ } ovs_bs;
+
+ struct {
+ unsigned int xid;
+ } crypt_bs;
+
+ unsigned int value;
+ } dw4;
+
+ union {
+ struct { /**< For nic. */
+ unsigned int msg_1588_ts;
+ } bs;
+
+ struct { /**< For ovs. */
+ unsigned int traffic_from : 16;
+ unsigned int traffic_type : 6;
+ unsigned int rsvd0 : 2;
+ unsigned int l4_type : 3;
+ unsigned int l3_type : 3;
+ unsigned int mac_type : 2;
+ } ovs_bs;
+
+ struct { /**< For crypt. */
+ unsigned int esp_next_head : 8;
+ unsigned int decrypt_status : 8;
+ unsigned int rsvd : 16;
+ } crypt_bs;
+
+ unsigned int value;
+ } dw5;
+
+ union {
+ struct { /**< For nic. */
+ unsigned int lro_ts;
+ } bs;
+
+ struct { /**< For ovs. */
+ unsigned int reserved;
+ } ovs_bs;
+
+ unsigned int value;
+ } dw6;
+
+ union {
+ struct { /**< For nic. */
+ /* Data len of the first or middle pkt size. */
+ unsigned int first_len : 13;
+ /* Data len of the last pkt size. */
+ unsigned int last_len : 13;
+ /* The number of packet. */
+ unsigned int pkt_num : 5;
+ /* Only this bit = 1, other dw fields is valid. */
+ unsigned int super_cqe_en : 1;
+ } bs;
+
+ struct { /**< For ovs. */
+ unsigned int localtag;
+ } ovs_bs;
+
+ unsigned int value;
+ } dw7;
+} l2nic_rx_cqe_s;
+
+struct nic_rq_bd_sec {
+ unsigned int pkt_buf_addr_high; /**< Packet buffer address high. */
+ unsigned int pkt_buf_addr_low; /**< Packet buffer address low. */
+ unsigned int len;
+};
+
+typedef struct _nic_rq_wqe {
+ /* RX buffer SGE. Notes, buf_desc.len limit in bit 0~13. */
+ struct nic_rq_bd_sec buf_desc;
+ /* Reserved field 0 for 16B align. */
+ unsigned int rsvd0;
+ /*
+ * CQE buffer SGE. Notes, cqe_sect.len is in unit of 16B and limit in
+ * bit 0~4.
+ */
+ struct nic_rq_bd_sec cqe_sect;
+ /* Reserved field 1 for unused. */
+ unsigned int rsvd1;
+} nic_rq_wqe;
+
+/* CMD. */
+struct cmd_show_q_st {
+ struct tool_target target;
+
+ int qobj;
+ int q_id;
+ int wqe_id;
+ int direction;
+};
+
+#endif /* _HINIC3_MML_QUEUE */
diff --git a/drivers/net/hinic3/mml/meson.build b/drivers/net/hinic3/mml/meson.build
new file mode 100644
index 0000000000..f8d2650d8d
--- /dev/null
+++ b/drivers/net/hinic3/mml/meson.build
@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Huawei Technologies Co., Ltd
+
+sources = files(
+ 'hinic3_dbg.c',
+ 'hinic3_mml_cmd.c',
+ 'hinic3_mml_ioctl.c',
+ 'hinic3_mml_lib.c',
+ 'hinic3_mml_main.c',
+ 'hinic3_mml_queue.c',
+)
+
+extra_flags = [
+ '-Wno-cast-qual',
+ '-Wno-format',
+ '-Wno-format-nonliteral',
+ '-Wno-format-security',
+ '-Wno-missing-braces',
+ '-Wno-missing-field-initializers',
+ '-Wno-missing-prototypes',
+ '-Wno-pointer-sign',
+ '-Wno-pointer-to-int-cast',
+ '-Wno-sign-compare',
+ '-Wno-strict-aliasing',
+ '-Wno-unused-parameter',
+ '-Wno-unused-value',
+ '-Wno-unused-variable',
+]
+
+# The driver runs only on arch64 machine, remove 32bit warnings
+if not dpdk_conf.get('RTE_ARCH_64')
+ extra_flags += [
+ '-Wno-int-to-pointer-cast',
+ '-Wno-pointer-to-int-cast',
+ ]
+endif
+
+foreach flag: extra_flags
+ if cc.has_argument(flag)
+ cflags += flag
+ endif
+endforeach
+
+deps += ['hash']
+
+c_args = cflags
+includes += include_directories('../')
+includes += include_directories('../base/')
+
+mml_lib = static_library(
+ 'hinic3_mml',
+ sources,
+ dependencies: [
+ static_rte_eal,
+ static_rte_ethdev,
+ static_rte_bus_pci,
+ static_rte_hash,
+ ],
+ include_directories: includes,
+ c_args: c_args,
+)
+mml_objs = mml_lib.extract_all_objects()
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 16/18] net/hinic3: add RSS promiscuous ops
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (6 preceding siblings ...)
2025-04-18 7:02 ` [RFC 15/18] net/hinic3: add MML and EEPROM access feature Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 17/18] net/hinic3: add FDIR flow control module Feifei Wang
` (2 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Xin Wang, Feifei Wang, Yi Chen
From: Xin Wang <wangxin679@h-partners.com>
Add RSS and promiscuous ops related function codes.
Signed-off-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
Reviewed-by: Yi Chen <chenyi221@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 370 +++++++++++++++++++++++++++++
drivers/net/hinic3/hinic3_ethdev.h | 31 +++
2 files changed, 401 insertions(+)
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index 9c5decb867..9d2dcf95f7 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -2277,6 +2277,281 @@ hinic3_dev_allmulticast_disable(struct rte_eth_dev *dev)
return 0;
}
+/**
+ * Enable promiscuous mode.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_promiscuous_enable(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 rx_mode;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->rx_mode_mutex);
+ if (err)
+ return err;
+
+ rx_mode = nic_dev->rx_mode | HINIC3_RX_MODE_PROMISC;
+
+ err = hinic3_set_rx_mode(nic_dev->hwdev, rx_mode);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+ PMD_DRV_LOG(ERR, "Enable promiscuous failed");
+ return err;
+ }
+
+ nic_dev->rx_mode = rx_mode;
+
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+
+ PMD_DRV_LOG(INFO,
+ "Enable promiscuous, nic_dev: %s, port_id: %d, promisc: %d",
+ nic_dev->dev_name, dev->data->port_id,
+ dev->data->promiscuous);
+ return 0;
+}
+
+/**
+ * Disable promiscuous mode.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_promiscuous_disable(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 rx_mode;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->rx_mode_mutex);
+ if (err)
+ return err;
+
+ rx_mode = nic_dev->rx_mode & (~HINIC3_RX_MODE_PROMISC);
+
+ err = hinic3_set_rx_mode(nic_dev->hwdev, rx_mode);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+ PMD_DRV_LOG(ERR, "Disable promiscuous failed");
+ return err;
+ }
+
+ nic_dev->rx_mode = rx_mode;
+
+ (void)hinic3_mutex_unlock(&nic_dev->rx_mode_mutex);
+
+ PMD_DRV_LOG(INFO,
+ "Disable promiscuous, nic_dev: %s, port_id: %d, promisc: %d",
+ nic_dev->dev_name, dev->data->port_id, dev->data->promiscuous);
+ return 0;
+}
+
+/**
+ * Get flow control configuration, including auto-negotiation and RX/TX pause
+ * settings.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ *
+ * @param[out] fc_conf
+ * The flow control configuration to be filled.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+
+/**
+ * Update the RSS hash key and RSS hash type.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] rss_conf
+ * RSS configuration data.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_rss_type rss_type = {0};
+ u64 rss_hf = rss_conf->rss_hf;
+ int err = 0;
+
+ if (nic_dev->rss_state == HINIC3_RSS_DISABLE) {
+ if (rss_hf != 0)
+ return -EINVAL;
+
+ PMD_DRV_LOG(INFO, "RSS is not enabled");
+ return 0;
+ }
+
+ if (rss_conf->rss_key_len > HINIC3_RSS_KEY_SIZE) {
+ PMD_DRV_LOG(ERR, "Invalid RSS key, rss_key_len: %d",
+ rss_conf->rss_key_len);
+ return -EINVAL;
+ }
+
+ if (rss_conf->rss_key) {
+ err = hinic3_rss_set_hash_key(nic_dev->hwdev, nic_dev->rss_key,
+ HINIC3_RSS_KEY_SIZE);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Set RSS hash key failed");
+ return err;
+ }
+ memcpy((void *)nic_dev->rss_key, (void *)rss_conf->rss_key,
+ (size_t)rss_conf->rss_key_len);
+ }
+
+ rss_type.ipv4 = (rss_hf & (RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4 |
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER))
+ ? 1
+ : 0;
+ rss_type.tcp_ipv4 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP) ? 1 : 0;
+ rss_type.ipv6 = (rss_hf & (RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6 |
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER))
+ ? 1
+ : 0;
+ rss_type.ipv6_ext = (rss_hf & RTE_ETH_RSS_IPV6_EX) ? 1 : 0;
+ rss_type.tcp_ipv6 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP) ? 1 : 0;
+ rss_type.tcp_ipv6_ext = (rss_hf & RTE_ETH_RSS_IPV6_TCP_EX) ? 1 : 0;
+ rss_type.udp_ipv4 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP) ? 1 : 0;
+ rss_type.udp_ipv6 = (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP) ? 1 : 0;
+
+ err = hinic3_set_rss_type(nic_dev->hwdev, rss_type);
+ if (err)
+ PMD_DRV_LOG(ERR, "Set RSS type failed");
+
+ return err;
+}
+
+/**
+ * Get the RSS hash configuration.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] rss_conf
+ * RSS configuration data.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_rss_conf_get(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_rss_type rss_type = {0};
+ int err;
+
+ if (!rss_conf)
+ return -EINVAL;
+
+ if (nic_dev->rss_state == HINIC3_RSS_DISABLE) {
+ rss_conf->rss_hf = 0;
+ PMD_DRV_LOG(INFO, "RSS is not enabled");
+ return 0;
+ }
+
+ if (rss_conf->rss_key && rss_conf->rss_key_len >= HINIC3_RSS_KEY_SIZE) {
+ /*
+ * Get RSS key from driver to reduce the frequency of the MPU
+ * accessing the RSS memory.
+ */
+ rss_conf->rss_key_len = sizeof(nic_dev->rss_key);
+ memcpy((void *)rss_conf->rss_key, (void *)nic_dev->rss_key,
+ (size_t)rss_conf->rss_key_len);
+ }
+
+ err = hinic3_get_rss_type(nic_dev->hwdev, &rss_type);
+ if (err)
+ return err;
+
+ rss_conf->rss_hf = 0;
+ rss_conf->rss_hf |=
+ rss_type.ipv4 ? (RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4 |
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER)
+ : 0;
+ rss_conf->rss_hf |= rss_type.tcp_ipv4 ? RTE_ETH_RSS_NONFRAG_IPV4_TCP
+ : 0;
+ rss_conf->rss_hf |=
+ rss_type.ipv6 ? (RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6 |
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+ : 0;
+ rss_conf->rss_hf |= rss_type.ipv6_ext ? RTE_ETH_RSS_IPV6_EX : 0;
+ rss_conf->rss_hf |= rss_type.tcp_ipv6 ? RTE_ETH_RSS_NONFRAG_IPV6_TCP
+ : 0;
+ rss_conf->rss_hf |= rss_type.tcp_ipv6_ext ? RTE_ETH_RSS_IPV6_TCP_EX : 0;
+ rss_conf->rss_hf |= rss_type.udp_ipv4 ? RTE_ETH_RSS_NONFRAG_IPV4_UDP
+ : 0;
+ rss_conf->rss_hf |= rss_type.udp_ipv6 ? RTE_ETH_RSS_NONFRAG_IPV6_UDP
+ : 0;
+
+ return 0;
+}
+
+/**
+ * Get the RETA indirection table.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] reta_conf
+ * Pointer to RETA configuration structure array.
+ * @param[in] reta_size
+ * Size of the RETA table.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 indirtbl[HINIC3_RSS_INDIR_SIZE] = {0};
+ u16 idx, shift;
+ u16 i;
+ int err;
+
+ if (nic_dev->rss_state == HINIC3_RSS_DISABLE) {
+ PMD_DRV_LOG(INFO, "RSS is not enabled");
+ return 0;
+ }
+
+ if (reta_size != HINIC3_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Invalid reta size, reta_size: %d", reta_size);
+ return -EINVAL;
+ }
+
+ err = hinic3_rss_get_indir_tbl(nic_dev->hwdev, indirtbl,
+ HINIC3_RSS_INDIR_SIZE);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Get RSS retas table failed, error: %d", err);
+ return err;
+ }
+
+ for (i = 0; i < reta_size; i++) {
+ idx = i / RTE_ETH_RETA_GROUP_SIZE;
+ shift = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[idx].mask & (1ULL << shift))
+ reta_conf[idx].reta[shift] = (uint16_t)indirtbl[i];
+ }
+
+ return 0;
+}
+
static int
hinic3_get_eeprom(__rte_unused struct rte_eth_dev *dev,
struct rte_dev_eeprom_info *info)
@@ -2287,6 +2562,68 @@ hinic3_get_eeprom(__rte_unused struct rte_eth_dev *dev,
&info->length, MAX_BUF_OUT_LEN);
}
+/**
+ * Update the RETA indirection table.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] reta_conf
+ * Pointer to RETA configuration structure array.
+ * @param[in] reta_size
+ * Size of the RETA table.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u32 indirtbl[HINIC3_RSS_INDIR_SIZE] = {0};
+ u16 idx, shift;
+ u16 i;
+ int err;
+
+ if (nic_dev->rss_state == HINIC3_RSS_DISABLE)
+ return 0;
+
+ if (reta_size != HINIC3_RSS_INDIR_SIZE) {
+ PMD_DRV_LOG(ERR, "Invalid reta size, reta_size: %d", reta_size);
+ return -EINVAL;
+ }
+
+ err = hinic3_rss_get_indir_tbl(nic_dev->hwdev, indirtbl,
+ HINIC3_RSS_INDIR_SIZE);
+ if (err)
+ return err;
+
+ /* Update RSS reta table. */
+ for (i = 0; i < reta_size; i++) {
+ idx = i / RTE_ETH_RETA_GROUP_SIZE;
+ shift = i % RTE_ETH_RETA_GROUP_SIZE;
+ if (reta_conf[idx].mask & (1ULL << shift))
+ indirtbl[i] = reta_conf[idx].reta[shift];
+ }
+
+ for (i = 0; i < reta_size; i++) {
+ if (indirtbl[i] >= nic_dev->num_rqs) {
+ PMD_DRV_LOG(ERR,
+ "Invalid reta entry, index: %d, num_rqs: %d",
+ indirtbl[i], nic_dev->num_rqs);
+ return -EFAULT;
+ }
+ }
+
+ err = hinic3_rss_set_indir_tbl(nic_dev->hwdev, indirtbl,
+ HINIC3_RSS_INDIR_SIZE);
+ if (err)
+ PMD_DRV_LOG(ERR, "Set RSS reta table failed");
+
+ return err;
+}
+
/**
* Get device generic statistics.
*
@@ -2857,6 +3194,29 @@ hinic3_set_mc_addr_list(struct rte_eth_dev *dev,
return 0;
}
+/**
+ * Manage flow director filter operations.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] filter_type
+ * Filter type.
+ * @param[in] filter_op
+ * Operation to perform.
+ * @param[in] arg
+ * Pointer to operation-specific structure.
+ *
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_dev_filter_ctrl(struct rte_eth_dev *dev, const struct rte_flow_ops **arg)
+{
+ RTE_SET_USED(dev);
+ *arg = &hinic3_flow_ops;
+ return 0;
+}
+
static int
hinic3_get_reg(__rte_unused struct rte_eth_dev *dev,
__rte_unused struct rte_dev_reg_info *regs)
@@ -2890,6 +3250,12 @@ static const struct eth_dev_ops hinic3_pmd_ops = {
.vlan_offload_set = hinic3_vlan_offload_set,
.allmulticast_enable = hinic3_dev_allmulticast_enable,
.allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .promiscuous_enable = hinic3_dev_promiscuous_enable,
+ .promiscuous_disable = hinic3_dev_promiscuous_disable,
+ .rss_hash_update = hinic3_rss_hash_update,
+ .rss_hash_conf_get = hinic3_rss_conf_get,
+ .reta_update = hinic3_rss_reta_update,
+ .reta_query = hinic3_rss_reta_query,
.get_eeprom = hinic3_get_eeprom,
.stats_get = hinic3_dev_stats_get,
.stats_reset = hinic3_dev_stats_reset,
@@ -2931,6 +3297,10 @@ static const struct eth_dev_ops hinic3_pmd_vf_ops = {
.vlan_offload_set = hinic3_vlan_offload_set,
.allmulticast_enable = hinic3_dev_allmulticast_enable,
.allmulticast_disable = hinic3_dev_allmulticast_disable,
+ .rss_hash_update = hinic3_rss_hash_update,
+ .rss_hash_conf_get = hinic3_rss_conf_get,
+ .reta_update = hinic3_rss_reta_update,
+ .reta_query = hinic3_rss_reta_query,
.get_eeprom = hinic3_get_eeprom,
.stats_get = hinic3_dev_stats_get,
.stats_reset = hinic3_dev_stats_reset,
diff --git a/drivers/net/hinic3/hinic3_ethdev.h b/drivers/net/hinic3/hinic3_ethdev.h
index a69cf972e7..5dd7c7821a 100644
--- a/drivers/net/hinic3/hinic3_ethdev.h
+++ b/drivers/net/hinic3/hinic3_ethdev.h
@@ -97,6 +97,10 @@ struct hinic3_nic_dev {
u16 rx_buff_len;
u16 mtu_size;
+ u16 rss_state;
+ u8 num_rss; /**< Number of RSS queues. */
+ u8 rsvd0; /**< Reserved field 0. */
+
u32 rx_mode;
u8 rx_queue_list[HINIC3_MAX_QUEUE_NUM];
rte_spinlock_t queue_list_lock;
@@ -106,6 +110,8 @@ struct hinic3_nic_dev {
u32 default_cos;
u32 rx_csum_en;
+ u8 rss_key[HINIC3_RSS_KEY_SIZE];
+
unsigned long dev_status;
struct rte_ether_addr default_addr;
@@ -116,4 +122,29 @@ struct hinic3_nic_dev {
u32 vfta[HINIC3_VFTA_SIZE]; /**< VLAN bitmap. */
};
+/**
+ * Enable interrupt for the specified RX queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] queue_id
+ * The ID of the receive queue for which the interrupt is being enabled.
+ * @return
+ * 0 on success, a negative error code on failure.
+ */
+int hinic3_dev_rx_queue_intr_enable(struct rte_eth_dev *dev, uint16_t queue_id);
+
+/**
+ * Disable interrupt for the specified RX queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] queue_id
+ * The ID of the receive queue for which the interrupt is being disabled.
+ * @return
+ * 0 on success, a negative error code on failure.
+ */
+int hinic3_dev_rx_queue_intr_disable(struct rte_eth_dev *dev,
+ uint16_t queue_id);
+
#endif /* _HINIC3_ETHDEV_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 17/18] net/hinic3: add FDIR flow control module
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (7 preceding siblings ...)
2025-04-18 7:02 ` [RFC 16/18] net/hinic3: add RSS promiscuous ops Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-04-18 7:02 ` [RFC 18/18] drivers/net: add hinic3 PMD build and doc files Feifei Wang
2025-05-20 15:47 ` [RFC 00/18] add hinic3 PMD driver Stephen Hemminger
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Yi Chen, Xin Wang, Feifei Wang
From: Yi Chen <chenyi221@huawei.com>
Added support for flow director filters, including ethertype, IPv4,
IPv6, and tunnel VXLAN. In addition, user can add or delete filters.
Signed-off-by: Yi Chen <chenyi221@huawei.com>
Reviewed-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 82 ++
drivers/net/hinic3/hinic3_ethdev.h | 17 +
drivers/net/hinic3/hinic3_fdir.c | 1394 +++++++++++++++++++++++
drivers/net/hinic3/hinic3_fdir.h | 398 +++++++
drivers/net/hinic3/hinic3_flow.c | 1700 ++++++++++++++++++++++++++++
drivers/net/hinic3/hinic3_flow.h | 80 ++
6 files changed, 3671 insertions(+)
create mode 100644 drivers/net/hinic3/hinic3_fdir.c
create mode 100644 drivers/net/hinic3/hinic3_fdir.h
create mode 100644 drivers/net/hinic3/hinic3_flow.c
create mode 100644 drivers/net/hinic3/hinic3_flow.h
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index 9d2dcf95f7..2b8d2dc7a7 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -2369,6 +2369,84 @@ hinic3_dev_promiscuous_disable(struct rte_eth_dev *dev)
* @return
* 0 on success, non-zero on failure.
*/
+static int
+hinic3_dev_flow_ctrl_get(struct rte_eth_dev *dev,
+ struct rte_eth_fc_conf *fc_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct nic_pause_config nic_pause;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->pause_mutuex);
+ if (err)
+ return err;
+
+ memset(&nic_pause, 0, sizeof(nic_pause));
+ err = hinic3_get_pause_info(nic_dev->hwdev, &nic_pause);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->pause_mutuex);
+ return err;
+ }
+
+ if (nic_dev->pause_set || !nic_pause.auto_neg) {
+ nic_pause.rx_pause = nic_dev->nic_pause.rx_pause;
+ nic_pause.tx_pause = nic_dev->nic_pause.tx_pause;
+ }
+
+ fc_conf->autoneg = nic_pause.auto_neg;
+
+ if (nic_pause.tx_pause && nic_pause.rx_pause)
+ fc_conf->mode = RTE_ETH_FC_FULL;
+ else if (nic_pause.tx_pause)
+ fc_conf->mode = RTE_ETH_FC_TX_PAUSE;
+ else if (nic_pause.rx_pause)
+ fc_conf->mode = RTE_ETH_FC_RX_PAUSE;
+ else
+ fc_conf->mode = RTE_ETH_FC_NONE;
+
+ (void)hinic3_mutex_unlock(&nic_dev->pause_mutuex);
+ return 0;
+}
+
+static int
+hinic3_dev_flow_ctrl_set(struct rte_eth_dev *dev,
+ struct rte_eth_fc_conf *fc_conf)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct nic_pause_config nic_pause;
+ int err;
+
+ err = hinic3_mutex_lock(&nic_dev->pause_mutuex);
+ if (err)
+ return err;
+
+ memset(&nic_pause, 0, sizeof(nic_pause));
+ if ((fc_conf->mode & RTE_ETH_FC_FULL) == RTE_ETH_FC_FULL ||
+ (fc_conf->mode & RTE_ETH_FC_TX_PAUSE))
+ nic_pause.tx_pause = true;
+
+ if ((fc_conf->mode & RTE_ETH_FC_FULL) == RTE_ETH_FC_FULL ||
+ (fc_conf->mode & RTE_ETH_FC_RX_PAUSE))
+ nic_pause.rx_pause = true;
+
+ err = hinic3_set_pause_info(nic_dev->hwdev, nic_pause);
+ if (err) {
+ (void)hinic3_mutex_unlock(&nic_dev->pause_mutuex);
+ return err;
+ }
+
+ nic_dev->pause_set = true;
+ nic_dev->nic_pause.rx_pause = nic_pause.rx_pause;
+ nic_dev->nic_pause.tx_pause = nic_pause.tx_pause;
+
+ PMD_DRV_LOG(INFO,
+ "Just support set tx or rx pause info, tx: %s, rx: %s",
+ nic_pause.tx_pause ? "on" : "off",
+ nic_pause.rx_pause ? "on" : "off");
+
+ (void)hinic3_mutex_unlock(&nic_dev->pause_mutuex);
+ return 0;
+}
/**
* Update the RSS hash key and RSS hash type.
@@ -3252,6 +3330,8 @@ static const struct eth_dev_ops hinic3_pmd_ops = {
.allmulticast_disable = hinic3_dev_allmulticast_disable,
.promiscuous_enable = hinic3_dev_promiscuous_enable,
.promiscuous_disable = hinic3_dev_promiscuous_disable,
+ .flow_ctrl_get = hinic3_dev_flow_ctrl_get,
+ .flow_ctrl_set = hinic3_dev_flow_ctrl_set,
.rss_hash_update = hinic3_rss_hash_update,
.rss_hash_conf_get = hinic3_rss_conf_get,
.reta_update = hinic3_rss_reta_update,
@@ -3269,6 +3349,7 @@ static const struct eth_dev_ops hinic3_pmd_ops = {
.mac_addr_remove = hinic3_mac_addr_remove,
.mac_addr_add = hinic3_mac_addr_add,
.set_mc_addr_list = hinic3_set_mc_addr_list,
+ .flow_ops_get = hinic3_dev_filter_ctrl,
.get_reg = hinic3_get_reg,
};
@@ -3313,6 +3394,7 @@ static const struct eth_dev_ops hinic3_pmd_vf_ops = {
.mac_addr_remove = hinic3_mac_addr_remove,
.mac_addr_add = hinic3_mac_addr_add,
.set_mc_addr_list = hinic3_set_mc_addr_list,
+ .flow_ops_get = hinic3_dev_filter_ctrl,
};
/**
diff --git a/drivers/net/hinic3/hinic3_ethdev.h b/drivers/net/hinic3/hinic3_ethdev.h
index 5dd7c7821a..07e24e971c 100644
--- a/drivers/net/hinic3/hinic3_ethdev.h
+++ b/drivers/net/hinic3/hinic3_ethdev.h
@@ -8,6 +8,8 @@
#include <rte_ethdev.h>
#include <rte_ethdev_core.h>
+#include "hinic3_fdir.h"
+
#define HINIC3_PMD_DRV_VERSION "B106"
#define PCI_DEV_TO_INTR_HANDLE(pci_dev) ((pci_dev)->intr_handle)
@@ -83,6 +85,9 @@ enum nic_feature_cap {
#define DEFAULT_DRV_FEATURE 0x3FFF
+TAILQ_HEAD(hinic3_ethertype_filter_list, rte_flow);
+TAILQ_HEAD(hinic3_fdir_rule_filter_list, rte_flow);
+
struct hinic3_nic_dev {
struct hinic3_hwdev *hwdev; /**< Hardware device. */
struct hinic3_txq **txqs;
@@ -114,14 +119,26 @@ struct hinic3_nic_dev {
unsigned long dev_status;
+ u8 pause_set; /**< Flag of PAUSE frame setting. */
+ pthread_mutex_t pause_mutuex;
+ struct nic_pause_config nic_pause;
+
struct rte_ether_addr default_addr;
struct rte_ether_addr *mc_list;
char dev_name[HINIC3_DEV_NAME_LEN];
u64 feature_cap;
u32 vfta[HINIC3_VFTA_SIZE]; /**< VLAN bitmap. */
+
+ u16 tcam_rule_nums;
+ u16 ethertype_rule_nums;
+ struct hinic3_tcam_info tcam;
+ struct hinic3_ethertype_filter_list filter_ethertype_list;
+ struct hinic3_fdir_rule_filter_list filter_fdir_rule_list;
};
+extern const struct rte_flow_ops hinic3_flow_ops;
+
/**
* Enable interrupt for the specified RX queue.
*
diff --git a/drivers/net/hinic3/hinic3_fdir.c b/drivers/net/hinic3/hinic3_fdir.c
new file mode 100644
index 0000000000..e36050f263
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_fdir.c
@@ -0,0 +1,1394 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_ethdev.h>
+#include <rte_ether.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_malloc.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_hwdev.h"
+#include "base/hinic3_hwif.h"
+#include "base/hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_fdir.h"
+
+#define HINIC3_UINT1_MAX 0x1
+#define HINIC3_UINT4_MAX 0xf
+#define HINIC3_UINT15_MAX 0x7fff
+
+#define HINIC3_DEV_PRIVATE_TO_TCAM_INFO(nic_dev) \
+ (&((struct hinic3_nic_dev *)(nic_dev))->tcam)
+
+/**
+ * Perform a bitwise AND operation on the input key value and mask, and stores
+ * the result in the key_y array.
+ *
+ * @param[out] key_y
+ * Array for storing results.
+ * @param[in] src_input
+ * Input key array.
+ * @param[in] mask
+ * Mask array.
+ * @param[in] len
+ * Length of the key value and mask.
+ */
+static void
+tcam_translate_key_y(u8 *key_y, u8 *src_input, u8 *mask, u8 len)
+{
+ u8 idx;
+
+ for (idx = 0; idx < len; idx++)
+ key_y[idx] = src_input[idx] & mask[idx];
+}
+
+/**
+ * Convert key_y to key_x using the exclusive OR operation.
+ *
+ * @param[out] key_x
+ * Array for storing results.
+ * @param[in] key_y
+ * Input key array.
+ * @param[in] mask
+ * Mask array.
+ * @param[in] len
+ * Length of the key value and mask.
+ */
+static void
+tcam_translate_key_x(u8 *key_x, u8 *key_y, u8 *mask, u8 len)
+{
+ u8 idx;
+
+ for (idx = 0; idx < len; idx++)
+ key_x[idx] = key_y[idx] ^ mask[idx];
+}
+
+static void
+tcam_key_calculate(struct hinic3_tcam_key *tcam_key,
+ struct hinic3_tcam_cfg_rule *fdir_tcam_rule)
+{
+ tcam_translate_key_y(fdir_tcam_rule->key.y, (u8 *)(&tcam_key->key_info),
+ (u8 *)(&tcam_key->key_mask),
+ HINIC3_TCAM_FLOW_KEY_SIZE);
+ tcam_translate_key_x(fdir_tcam_rule->key.x, fdir_tcam_rule->key.y,
+ (u8 *)(&tcam_key->key_mask),
+ HINIC3_TCAM_FLOW_KEY_SIZE);
+}
+
+static void
+hinic3_fdir_tcam_ipv4_init(struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ /* Fill type of ip. */
+ tcam_key->key_mask.ip_type = HINIC3_UINT1_MAX;
+ tcam_key->key_info.ip_type = HINIC3_FDIR_IP_TYPE_IPV4;
+
+ /* Fill src IPv4. */
+ tcam_key->key_mask.sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv4.src_ip);
+ tcam_key->key_mask.sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv4.src_ip);
+ tcam_key->key_info.sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv4.src_ip);
+ tcam_key->key_info.sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv4.src_ip);
+
+ /* Fill dst IPv4. */
+ tcam_key->key_mask.dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv4.dst_ip);
+ tcam_key->key_mask.dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv4.dst_ip);
+ tcam_key->key_info.dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv4.dst_ip);
+ tcam_key->key_info.dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv4.dst_ip);
+}
+
+static void
+hinic3_fdir_tcam_ipv6_init(struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ /* Fill type of ip. */
+ tcam_key->key_mask_ipv6.ip_type = HINIC3_UINT1_MAX;
+ tcam_key->key_info_ipv6.ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+
+ /* Fill src IPv6. */
+ tcam_key->key_mask_ipv6.sipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0]);
+ tcam_key->key_mask_ipv6.sipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0]);
+ tcam_key->key_mask_ipv6.sipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x1]);
+ tcam_key->key_mask_ipv6.sipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x1]);
+ tcam_key->key_mask_ipv6.sipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x2]);
+ tcam_key->key_mask_ipv6.sipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x2]);
+ tcam_key->key_mask_ipv6.sipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x3]);
+ tcam_key->key_mask_ipv6.sipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x3]);
+ tcam_key->key_info_ipv6.sipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0]);
+ tcam_key->key_info_ipv6.sipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0]);
+ tcam_key->key_info_ipv6.sipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x1]);
+ tcam_key->key_info_ipv6.sipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x1]);
+ tcam_key->key_info_ipv6.sipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x2]);
+ tcam_key->key_info_ipv6.sipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x2]);
+ tcam_key->key_info_ipv6.sipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x3]);
+ tcam_key->key_info_ipv6.sipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x3]);
+
+ /* Fill dst IPv6. */
+ tcam_key->key_mask_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0]);
+ tcam_key->key_mask_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0]);
+ tcam_key->key_mask_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x3]);
+ tcam_key->key_mask_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x3]);
+ tcam_key->key_info_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0]);
+ tcam_key->key_info_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0]);
+ tcam_key->key_info_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x1]);
+ tcam_key->key_info_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x1]);
+ tcam_key->key_info_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x2]);
+ tcam_key->key_info_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x2]);
+ tcam_key->key_info_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x3]);
+ tcam_key->key_info_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x3]);
+}
+
+/**
+ * Set the TCAM information in notunnel scenario.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] rule
+ * Pointer to the filtering rule.
+ * @param[in] tcam_key
+ * Pointer to the TCAM key.
+ */
+static void
+hinic3_fdir_tcam_notunnel_init(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ /* Fill tcam_key info. */
+ tcam_key->key_mask.sport = rule->key_mask.src_port;
+ tcam_key->key_info.sport = rule->key_spec.src_port;
+
+ tcam_key->key_mask.dport = rule->key_mask.dst_port;
+ tcam_key->key_info.dport = rule->key_spec.dst_port;
+
+ tcam_key->key_mask.tunnel_type = HINIC3_UINT4_MAX;
+ tcam_key->key_info.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_NORMAL;
+
+ tcam_key->key_mask.function_id = HINIC3_UINT15_MAX;
+ tcam_key->key_info.function_id = hinic3_global_func_id(nic_dev->hwdev) &
+ HINIC3_UINT15_MAX;
+
+ tcam_key->key_mask.ip_proto = rule->key_mask.proto;
+ tcam_key->key_info.ip_proto = rule->key_spec.proto;
+
+ if (rule->ip_type == HINIC3_FDIR_IP_TYPE_IPV4)
+ hinic3_fdir_tcam_ipv4_init(rule, tcam_key);
+ else if (rule->ip_type == HINIC3_FDIR_IP_TYPE_IPV6)
+ hinic3_fdir_tcam_ipv6_init(rule, tcam_key);
+}
+
+static void
+hinic3_fdir_tcam_vxlan_ipv4_init(struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ /* Fill type of ip. */
+ tcam_key->key_mask.ip_type = HINIC3_UINT1_MAX;
+ tcam_key->key_info.ip_type = HINIC3_FDIR_IP_TYPE_IPV4;
+
+ /* Fill src ipv4. */
+ tcam_key->key_mask.sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv4.src_ip);
+ tcam_key->key_mask.sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv4.src_ip);
+ tcam_key->key_info.sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv4.src_ip);
+ tcam_key->key_info.sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv4.src_ip);
+
+ /* Fill dst ipv4. */
+ tcam_key->key_mask.dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv4.dst_ip);
+ tcam_key->key_mask.dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv4.dst_ip);
+ tcam_key->key_info.dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv4.dst_ip);
+ tcam_key->key_info.dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv4.dst_ip);
+}
+
+static void
+hinic3_fdir_tcam_vxlan_ipv6_init(struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ /* Fill type of ip. */
+ tcam_key->key_mask_vxlan_ipv6.ip_type = HINIC3_UINT1_MAX;
+ tcam_key->key_info_vxlan_ipv6.ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+
+ /* Use inner dst ipv6 to fill the dst ipv6 of tcam_key. */
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x3]);
+ tcam_key->key_mask_vxlan_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.inner_ipv6.dst_ip[0x3]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x1]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x1]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x2]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x2]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x3]);
+ tcam_key->key_info_vxlan_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.inner_ipv6.dst_ip[0x3]);
+}
+
+static void
+hinic3_fdir_tcam_outer_ipv6_init(struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ tcam_key->key_mask_ipv6.sipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0]);
+ tcam_key->key_mask_ipv6.sipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0]);
+ tcam_key->key_mask_ipv6.sipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x1]);
+ tcam_key->key_mask_ipv6.sipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x1]);
+ tcam_key->key_mask_ipv6.sipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x2]);
+ tcam_key->key_mask_ipv6.sipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x2]);
+ tcam_key->key_mask_ipv6.sipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.src_ip[0x3]);
+ tcam_key->key_mask_ipv6.sipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.src_ip[0x3]);
+ tcam_key->key_info_ipv6.sipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0]);
+ tcam_key->key_info_ipv6.sipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0]);
+ tcam_key->key_info_ipv6.sipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x1]);
+ tcam_key->key_info_ipv6.sipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x1]);
+ tcam_key->key_info_ipv6.sipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x2]);
+ tcam_key->key_info_ipv6.sipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x2]);
+ tcam_key->key_info_ipv6.sipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.src_ip[0x3]);
+ tcam_key->key_info_ipv6.sipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.src_ip[0x3]);
+
+ tcam_key->key_mask_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0]);
+ tcam_key->key_mask_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0]);
+ tcam_key->key_mask_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x1]);
+ tcam_key->key_mask_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x2]);
+ tcam_key->key_mask_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv6.dst_ip[0x3]);
+ tcam_key->key_mask_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv6.dst_ip[0x3]);
+ tcam_key->key_info_ipv6.dipv6_key0 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0]);
+ tcam_key->key_info_ipv6.dipv6_key1 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0]);
+ tcam_key->key_info_ipv6.dipv6_key2 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x1]);
+ tcam_key->key_info_ipv6.dipv6_key3 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x1]);
+ tcam_key->key_info_ipv6.dipv6_key4 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x2]);
+ tcam_key->key_info_ipv6.dipv6_key5 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x2]);
+ tcam_key->key_info_ipv6.dipv6_key6 =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv6.dst_ip[0x3]);
+ tcam_key->key_info_ipv6.dipv6_key7 =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv6.dst_ip[0x3]);
+}
+
+static void
+hinic3_fdir_tcam_ipv6_vxlan_init(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ tcam_key->key_mask_ipv6.ip_proto = rule->key_mask.proto;
+ tcam_key->key_info_ipv6.ip_proto = rule->key_spec.proto;
+
+ tcam_key->key_mask_ipv6.tunnel_type = HINIC3_UINT4_MAX;
+ tcam_key->key_info_ipv6.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_VXLAN;
+
+ tcam_key->key_mask_ipv6.outer_ip_type = HINIC3_UINT1_MAX;
+ tcam_key->key_info_ipv6.outer_ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+
+ tcam_key->key_mask_ipv6.function_id = HINIC3_UINT15_MAX;
+ tcam_key->key_info_ipv6.function_id =
+ hinic3_global_func_id(nic_dev->hwdev) & HINIC3_UINT15_MAX;
+
+ tcam_key->key_mask_ipv6.dport = rule->key_mask.dst_port;
+ tcam_key->key_info_ipv6.dport = rule->key_spec.dst_port;
+
+ tcam_key->key_mask_ipv6.sport = rule->key_mask.src_port;
+ tcam_key->key_info_ipv6.sport = rule->key_spec.src_port;
+
+ if (rule->ip_type == HINIC3_FDIR_IP_TYPE_ANY)
+ hinic3_fdir_tcam_outer_ipv6_init(rule, tcam_key);
+}
+
+/**
+ * Sets the TCAM information in the VXLAN scenario.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] rule
+ * Pointer to the filtering rule.
+ * @param[in] tcam_key
+ * Pointer to the TCAM key.
+ */
+static void
+hinic3_fdir_tcam_vxlan_init(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (rule->outer_ip_type == HINIC3_FDIR_IP_TYPE_IPV6) {
+ hinic3_fdir_tcam_ipv6_vxlan_init(dev, rule, tcam_key);
+ return;
+ }
+
+ tcam_key->key_mask.ip_proto = rule->key_mask.proto;
+ tcam_key->key_info.ip_proto = rule->key_spec.proto;
+
+ tcam_key->key_mask.sport = rule->key_mask.src_port;
+ tcam_key->key_info.sport = rule->key_spec.src_port;
+
+ tcam_key->key_mask.dport = rule->key_mask.dst_port;
+ tcam_key->key_info.dport = rule->key_spec.dst_port;
+
+ tcam_key->key_mask.outer_sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv4.src_ip);
+ tcam_key->key_mask.outer_sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv4.src_ip);
+ tcam_key->key_info.outer_sipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv4.src_ip);
+ tcam_key->key_info.outer_sipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv4.src_ip);
+
+ tcam_key->key_mask.outer_dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.ipv4.dst_ip);
+ tcam_key->key_mask.outer_dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.ipv4.dst_ip);
+ tcam_key->key_info.outer_dipv4_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.ipv4.dst_ip);
+ tcam_key->key_info.outer_dipv4_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.ipv4.dst_ip);
+
+ tcam_key->key_mask.vni_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_mask.tunnel.tunnel_id);
+ tcam_key->key_mask.vni_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_mask.tunnel.tunnel_id);
+ tcam_key->key_info.vni_h =
+ HINIC3_32_UPPER_16_BITS(rule->key_spec.tunnel.tunnel_id);
+ tcam_key->key_info.vni_l =
+ HINIC3_32_LOWER_16_BITS(rule->key_spec.tunnel.tunnel_id);
+
+ tcam_key->key_mask.tunnel_type = HINIC3_UINT4_MAX;
+ tcam_key->key_info.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_VXLAN;
+
+ tcam_key->key_mask.function_id = HINIC3_UINT15_MAX;
+ tcam_key->key_info.function_id = hinic3_global_func_id(nic_dev->hwdev) &
+ HINIC3_UINT15_MAX;
+
+ if (rule->ip_type == HINIC3_FDIR_IP_TYPE_IPV4)
+ hinic3_fdir_tcam_vxlan_ipv4_init(rule, tcam_key);
+
+ else if (rule->ip_type == HINIC3_FDIR_IP_TYPE_IPV6)
+ hinic3_fdir_tcam_vxlan_ipv6_init(rule, tcam_key);
+}
+
+static void
+hinic3_fdir_tcam_info_init(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *rule,
+ struct hinic3_tcam_key *tcam_key,
+ struct hinic3_tcam_cfg_rule *fdir_tcam_rule)
+{
+ /* Initialize the TCAM based on the tunnel type. */
+ if (rule->tunnel_type == HINIC3_FDIR_TUNNEL_MODE_NORMAL)
+ hinic3_fdir_tcam_notunnel_init(dev, rule, tcam_key);
+ else
+ hinic3_fdir_tcam_vxlan_init(dev, rule, tcam_key);
+
+ /* Set the queue index. */
+ fdir_tcam_rule->data.qid = rule->rq_index;
+ /* Calculate key of TCAM. */
+ tcam_key_calculate(tcam_key, fdir_tcam_rule);
+}
+
+/**
+ * Find filter in given ethertype filter list.
+ *
+ * @param[in] filter_list
+ * Point to the Ether filter list.
+ * @param[in] key
+ * The tcam key to find.
+ * @return
+ * If a matching filter is found, the filter is returned, otherwise
+ * RTE_ETH_FILTER_NONE.
+ */
+static inline uint16_t
+hinic3_ethertype_filter_lookup(struct hinic3_ethertype_filter_list *ethertype_list,
+ uint16_t type)
+{
+ struct rte_flow *it;
+ struct hinic3_filter_t *filter_rules;
+
+ TAILQ_FOREACH(it, ethertype_list, node) {
+ filter_rules = it->rule;
+ if (type == filter_rules->ethertype_filter.ether_type)
+ return filter_rules->ethertype_filter.ether_type;
+ }
+
+ return RTE_ETH_FILTER_NONE;
+}
+
+/**
+ * Find the filter that matches the given key in the TCAM filter list.
+ *
+ * @param[in] filter_list
+ * Point to the tcam filter list.
+ * @param[in] key
+ * The tcam key to find.
+ * @return
+ * If a matching filter is found, the filter is returned, otherwise NULL.
+ */
+static inline struct hinic3_tcam_filter *
+hinic3_tcam_filter_lookup(struct hinic3_tcam_filter_list *filter_list,
+ struct hinic3_tcam_key *key)
+{
+ struct hinic3_tcam_filter *it;
+
+ TAILQ_FOREACH(it, filter_list, entries) {
+ if (memcmp(key, &it->tcam_key,
+ sizeof(struct hinic3_tcam_key)) == 0) {
+ return it;
+ }
+ }
+
+ return NULL;
+}
+/**
+ * Allocate memory for dynamic blocks and then add them to the queue.
+ *
+ * @param[in] tcam_info
+ * Point to TCAM information.
+ * @param[in] dynamic_block_id
+ * Indicate the ID of a dynamic block.
+ * @return
+ * Return the pointer to the dynamic block, or NULL if the allocation fails.
+ */
+static struct hinic3_tcam_dynamic_block *
+hinic3_alloc_dynamic_block_resource(struct hinic3_tcam_info *tcam_info,
+ u16 dynamic_block_id)
+{
+ struct hinic3_tcam_dynamic_block *dynamic_block_ptr = NULL;
+
+ dynamic_block_ptr =
+ rte_zmalloc("hinic3_tcam_dynamic_mem",
+ sizeof(struct hinic3_tcam_dynamic_block), 0);
+ if (dynamic_block_ptr == NULL) {
+ PMD_DRV_LOG(ERR,
+ "Alloc fdir filter dynamic block index %d memory "
+ "failed!",
+ dynamic_block_id);
+ return NULL;
+ }
+
+ dynamic_block_ptr->dynamic_block_id = dynamic_block_id;
+
+ /* Add new block to the end of the TCAM dynamic block list. */
+ TAILQ_INSERT_TAIL(&tcam_info->tcam_dynamic_info.tcam_dynamic_list,
+ dynamic_block_ptr, entries);
+
+ tcam_info->tcam_dynamic_info.dynamic_block_cnt++;
+
+ return dynamic_block_ptr;
+}
+
+static void
+hinic3_free_dynamic_block_resource(struct hinic3_tcam_info *tcam_info,
+ struct hinic3_tcam_dynamic_block *dynamic_block_ptr)
+{
+ if (dynamic_block_ptr == NULL)
+ return;
+
+ /* Remove the incoming dynamic block from the TCAM dynamic list. */
+ TAILQ_REMOVE(&tcam_info->tcam_dynamic_info.tcam_dynamic_list,
+ dynamic_block_ptr, entries);
+ rte_free(dynamic_block_ptr);
+
+ tcam_info->tcam_dynamic_info.dynamic_block_cnt--;
+}
+
+/**
+ * Check whether there are free positions in the dynamic TCAM filter.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] fdir_tcam_rule
+ * Indicate the filtering rule to be searched for.
+ * @param[in] tcam_info
+ * Ternary Content-Addressable Memory (TCAM) information.
+ * @param[in] tcam_filter
+ * Point to the TCAM filter.
+ * @param[out] tcam_index
+ * Indicate the TCAM index to be searched for.
+ * @result
+ * Pointer to the TCAM dynamic block. If the search fails, NULL is returned.
+ */
+static struct hinic3_tcam_dynamic_block *
+hinic3_dynamic_lookup_tcam_filter(struct rte_eth_dev *dev,
+ struct hinic3_tcam_cfg_rule *fdir_tcam_rule,
+ struct hinic3_tcam_info *tcam_info,
+ struct hinic3_tcam_filter *tcam_filter,
+ u16 *tcam_index)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u16 block_cnt = tcam_info->tcam_dynamic_info.dynamic_block_cnt;
+ struct hinic3_tcam_dynamic_block *dynamic_block_ptr = NULL;
+ struct hinic3_tcam_dynamic_block *tmp = NULL;
+ u16 rule_nums = nic_dev->tcam_rule_nums;
+ int block_alloc_flag = 0;
+ u16 dynamic_block_id = 0;
+ u16 index;
+ int err;
+
+ /*
+ * Check whether the number of filtering rules reaches the maximum
+ * capacity of dynamic TCAM blocks.
+ */
+ if (rule_nums >= block_cnt * HINIC3_TCAM_DYNAMIC_BLOCK_SIZE) {
+ if (block_cnt >= (HINIC3_TCAM_DYNAMIC_MAX_FILTERS /
+ HINIC3_TCAM_DYNAMIC_BLOCK_SIZE)) {
+ PMD_DRV_LOG(ERR,
+ "Dynamic tcam block is full, alloc failed!");
+ goto failed;
+ }
+ /*
+ * The TCAM blocks are insufficient.
+ * Apply for a new TCAM block.
+ */
+ err = hinic3_alloc_tcam_block(nic_dev->hwdev,
+ &dynamic_block_id);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Fdir filter dynamic tcam alloc block failed!");
+ goto failed;
+ }
+
+ block_alloc_flag = 1;
+
+ /* Applying for Memory. */
+ dynamic_block_ptr =
+ hinic3_alloc_dynamic_block_resource(tcam_info,
+ dynamic_block_id);
+ if (dynamic_block_ptr == NULL) {
+ PMD_DRV_LOG(ERR, "Fdir filter dynamic alloc block "
+ "memory failed!");
+ goto block_alloc_failed;
+ }
+ }
+
+ /*
+ * Find the first dynamic TCAM block that meets dynamci_index_cnt <
+ * HINIC3_TCAM_DYNAMIC_BLOCK_SIZE.
+ */
+ TAILQ_FOREACH(tmp, &tcam_info->tcam_dynamic_info.tcam_dynamic_list,
+ entries) {
+ if (tmp->dynamic_index_cnt < HINIC3_TCAM_DYNAMIC_BLOCK_SIZE)
+ break;
+ }
+
+ if (tmp == NULL ||
+ tmp->dynamic_index_cnt >= HINIC3_TCAM_DYNAMIC_BLOCK_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "Fdir filter dynamic lookup for index failed!");
+ goto look_up_failed;
+ }
+
+ for (index = 0; index < HINIC3_TCAM_DYNAMIC_BLOCK_SIZE; index++) {
+ if (tmp->dynamic_index[index] == 0)
+ break;
+ }
+
+ /* Find the first free position. */
+ if (index == HINIC3_TCAM_DYNAMIC_BLOCK_SIZE) {
+ PMD_DRV_LOG(ERR,
+ "tcam block 0x%x supports filter rules is full!",
+ tmp->dynamic_block_id);
+ goto look_up_failed;
+ }
+
+ tcam_filter->dynamic_block_id = tmp->dynamic_block_id;
+ tcam_filter->index = index;
+ *tcam_index = index;
+
+ fdir_tcam_rule->index =
+ HINIC3_PKT_TCAM_DYNAMIC_INDEX_START(tmp->dynamic_block_id) +
+ index;
+
+ return tmp;
+
+look_up_failed:
+ if (dynamic_block_ptr != NULL)
+ hinic3_free_dynamic_block_resource(tcam_info,
+ dynamic_block_ptr);
+
+block_alloc_failed:
+ if (block_alloc_flag == 1)
+ (void)hinic3_free_tcam_block(nic_dev->hwdev, &dynamic_block_id);
+
+failed:
+ return NULL;
+}
+
+/**
+ * Add a TCAM filter.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] tcam_key
+ * Pointer to the TCAM key.
+ * @param[in] fdir_tcam_rule
+ * Pointer to the TCAM filtering rule.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_add_tcam_filter(struct rte_eth_dev *dev,
+ struct hinic3_tcam_key *tcam_key,
+ struct hinic3_tcam_cfg_rule *fdir_tcam_rule)
+{
+ struct hinic3_tcam_info *tcam_info =
+ HINIC3_DEV_PRIVATE_TO_TCAM_INFO(dev->data->dev_private);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_tcam_dynamic_block *dynamic_block_ptr = NULL;
+ struct hinic3_tcam_dynamic_block *tmp = NULL;
+ struct hinic3_tcam_filter *tcam_filter;
+ u16 tcam_block_index = 0;
+ u16 index = 0;
+ int err;
+
+ /* Alloc TCAM filter memory. */
+ tcam_filter = rte_zmalloc("hinic3_fdir_filter",
+ sizeof(struct hinic3_tcam_filter), 0);
+ if (tcam_filter == NULL)
+ return -ENOMEM;
+ (void)rte_memcpy(&tcam_filter->tcam_key, tcam_key,
+ sizeof(struct hinic3_tcam_key));
+ tcam_filter->queue = (u16)(fdir_tcam_rule->data.qid);
+
+ /* Add new TCAM rules. */
+ if (nic_dev->tcam_rule_nums == 0) {
+ err = hinic3_alloc_tcam_block(nic_dev->hwdev,
+ &tcam_block_index);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Fdir filter tcam alloc block failed!");
+ goto failed;
+ }
+
+ dynamic_block_ptr =
+ hinic3_alloc_dynamic_block_resource(tcam_info,
+ tcam_block_index);
+ if (dynamic_block_ptr == NULL) {
+ PMD_DRV_LOG(ERR, "Fdir filter alloc dynamic first "
+ "block memory failed!");
+ goto alloc_block_failed;
+ }
+ }
+
+ /*
+ * Look for an available index in the dynamic block to store the new
+ * TCAM filter.
+ */
+ tmp = hinic3_dynamic_lookup_tcam_filter(dev, fdir_tcam_rule, tcam_info,
+ tcam_filter, &index);
+ if (tmp == NULL) {
+ PMD_DRV_LOG(ERR, "Dynamic lookup tcam filter failed!");
+ goto lookup_tcam_index_failed;
+ }
+
+ /* Add a new TCAM rule to the network device. */
+ err = hinic3_add_tcam_rule(nic_dev->hwdev, fdir_tcam_rule,
+ TCAM_RULE_FDIR_TYPE);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fdir_tcam_rule add failed!");
+ goto add_tcam_rules_failed;
+ }
+
+ /* If there are no rules, TCAM filtering is enabled. */
+ if (!(nic_dev->ethertype_rule_nums + nic_dev->tcam_rule_nums)) {
+ err = hinic3_set_fdir_tcam_rule_filter(nic_dev->hwdev, true);
+ if (err)
+ goto enable_failed;
+ }
+
+ /* Add a filter to the end of the queue. */
+ TAILQ_INSERT_TAIL(&tcam_info->tcam_list, tcam_filter, entries);
+
+ /* Update dynamic index. */
+ tmp->dynamic_index[index] = 1;
+ tmp->dynamic_index_cnt++;
+
+ nic_dev->tcam_rule_nums++;
+
+ PMD_DRV_LOG(INFO,
+ "Add fdir tcam rule, function_id: 0x%x, "
+ "tcam_block_id: %d, local_index: %d, global_index: %d, "
+ "queue: %d, "
+ "tcam_rule_nums: %d succeed",
+ hinic3_global_func_id(nic_dev->hwdev),
+ tcam_filter->dynamic_block_id, index, fdir_tcam_rule->index,
+ fdir_tcam_rule->data.qid, nic_dev->tcam_rule_nums);
+
+ return 0;
+
+enable_failed:
+ (void)hinic3_del_tcam_rule(nic_dev->hwdev, fdir_tcam_rule->index,
+ TCAM_RULE_FDIR_TYPE);
+
+add_tcam_rules_failed:
+lookup_tcam_index_failed:
+ if (nic_dev->tcam_rule_nums == 0 && dynamic_block_ptr != NULL)
+ hinic3_free_dynamic_block_resource(tcam_info,
+ dynamic_block_ptr);
+
+alloc_block_failed:
+ if (nic_dev->tcam_rule_nums == 0)
+ (void)hinic3_free_tcam_block(nic_dev->hwdev, &tcam_block_index);
+
+failed:
+ rte_free(tcam_filter);
+ return -EFAULT;
+}
+
+/**
+ * Delete a TCAM filter.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] tcam_filter
+ * TCAM Filters to Delete.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_del_dynamic_tcam_filter(struct rte_eth_dev *dev,
+ struct hinic3_tcam_filter *tcam_filter)
+{
+ struct hinic3_tcam_info *tcam_info =
+ HINIC3_DEV_PRIVATE_TO_TCAM_INFO(dev->data->dev_private);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ u16 dynamic_block_id = tcam_filter->dynamic_block_id;
+ struct hinic3_tcam_dynamic_block *tmp = NULL;
+ u32 index = 0;
+ int err;
+
+ /* Traverse to find the block that matches the given ID. */
+ TAILQ_FOREACH(tmp, &tcam_info->tcam_dynamic_info.tcam_dynamic_list,
+ entries) {
+ if (tmp->dynamic_block_id == dynamic_block_id)
+ break;
+ }
+
+ if (tmp == NULL || tmp->dynamic_block_id != dynamic_block_id) {
+ PMD_DRV_LOG(ERR,
+ "Fdir filter del dynamic lookup for block failed!");
+ return -EINVAL;
+ }
+ /* Calculate TCAM index. */
+ index = HINIC3_PKT_TCAM_DYNAMIC_INDEX_START(tmp->dynamic_block_id) +
+ tcam_filter->index;
+
+ /* Delete a specified rule. */
+ err = hinic3_del_tcam_rule(nic_dev->hwdev, index, TCAM_RULE_FDIR_TYPE);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fdir tcam rule del failed!");
+ return -EFAULT;
+ }
+
+ PMD_DRV_LOG(INFO,
+ "Del fdir_tcam_dynamic_rule function_id: 0x%x, "
+ "tcam_block_id: %d, local_index: %d, global_index: %d, "
+ "local_rules_nums: %d, global_rule_nums: %d succeed",
+ hinic3_global_func_id(nic_dev->hwdev), dynamic_block_id,
+ tcam_filter->index, index, tmp->dynamic_index_cnt - 1,
+ nic_dev->tcam_rule_nums - 1);
+
+ tmp->dynamic_index[tcam_filter->index] = 0;
+ tmp->dynamic_index_cnt--;
+ nic_dev->tcam_rule_nums--;
+ if (tmp->dynamic_index_cnt == 0) {
+ (void)hinic3_free_tcam_block(nic_dev->hwdev, &dynamic_block_id);
+
+ hinic3_free_dynamic_block_resource(tcam_info, tmp);
+ }
+
+ /* If the number of rules is 0, the TCAM filter is disabled. */
+ if (!(nic_dev->ethertype_rule_nums + nic_dev->tcam_rule_nums))
+ (void)hinic3_set_fdir_tcam_rule_filter(nic_dev->hwdev, false);
+
+ return 0;
+}
+
+static int
+hinic3_del_tcam_filter(struct rte_eth_dev *dev,
+ struct hinic3_tcam_filter *tcam_filter)
+{
+ struct hinic3_tcam_info *tcam_info =
+ HINIC3_DEV_PRIVATE_TO_TCAM_INFO(dev->data->dev_private);
+ int err;
+
+ err = hinic3_del_dynamic_tcam_filter(dev, tcam_filter);
+ if (err < 0) {
+ PMD_DRV_LOG(ERR, "Del dynamic tcam filter failed!");
+ return err;
+ }
+
+ /* Remove the filter from the TCAM list. */
+ TAILQ_REMOVE(&tcam_info->tcam_list, tcam_filter, entries);
+
+ rte_free(tcam_filter);
+
+ return 0;
+}
+
+/**
+ * Add or deletes an fdir filter rule. This is the core function for operating
+ * filters.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] fdir_filter
+ * Pointer to the fdir filter.
+ * @param[in] add
+ * This is a Boolean value (of the bool type) indicating whether the action to
+ * be performed is to add (true) or delete (false) the filter rule.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int
+hinic3_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *fdir_filter,
+ bool add)
+{
+ struct hinic3_tcam_info *tcam_info =
+ HINIC3_DEV_PRIVATE_TO_TCAM_INFO(dev->data->dev_private);
+ struct hinic3_tcam_filter *tcam_filter;
+ struct hinic3_tcam_cfg_rule fdir_tcam_rule;
+ struct hinic3_tcam_key tcam_key;
+ int ret;
+
+ memset(&fdir_tcam_rule, 0, sizeof(struct hinic3_tcam_cfg_rule));
+ memset((void *)&tcam_key, 0, sizeof(struct hinic3_tcam_key));
+
+ hinic3_fdir_tcam_info_init(dev, fdir_filter, &tcam_key,
+ &fdir_tcam_rule);
+ /* Search for a filter. */
+ tcam_filter =
+ hinic3_tcam_filter_lookup(&tcam_info->tcam_list, &tcam_key);
+ if (tcam_filter != NULL && add) {
+ PMD_DRV_LOG(ERR, "Filter exists.");
+ return -EEXIST;
+ }
+ if (tcam_filter == NULL && !add) {
+ PMD_DRV_LOG(ERR, "Filter doesn't exist.");
+ return -ENOENT;
+ }
+
+ /*
+ * If the value of Add is true, the system performs the adding
+ * operation.
+ */
+ if (add) {
+ ret = hinic3_add_tcam_filter(dev, &tcam_key, &fdir_tcam_rule);
+ if (ret)
+ goto cfg_tcam_filter_err;
+
+ fdir_filter->tcam_index = (int)(fdir_tcam_rule.index);
+ } else {
+ PMD_DRV_LOG(INFO, "begin to del tcam filter");
+ ret = hinic3_del_tcam_filter(dev, tcam_filter);
+ if (ret)
+ goto cfg_tcam_filter_err;
+ }
+
+ return 0;
+
+cfg_tcam_filter_err:
+
+ return ret;
+}
+
+/**
+ * Enable or disable the TCAM filter for the receive queue.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] able
+ * Flag to enable or disable the filter.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int
+hinic3_enable_rxq_fdir_filter(struct rte_eth_dev *dev, u32 queue_id, u32 able)
+{
+ struct hinic3_tcam_info *tcam_info =
+ HINIC3_DEV_PRIVATE_TO_TCAM_INFO(dev->data->dev_private);
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_tcam_filter *it;
+ struct hinic3_tcam_cfg_rule fdir_tcam_rule;
+ int ret;
+ u32 queue_res;
+ uint16_t index;
+
+ memset(&fdir_tcam_rule, 0, sizeof(struct hinic3_tcam_cfg_rule));
+
+ if (able) {
+ TAILQ_FOREACH(it, &tcam_info->tcam_list, entries) {
+ if (queue_id == it->queue) {
+ index = (u16)(HINIC3_PKT_TCAM_DYNAMIC_INDEX_START
+ (it->dynamic_block_id) + it->index);
+
+ /*
+ * When the rxq is start, find invalid rxq_id
+ * and delete the fdir rule from the tcam.
+ */
+ ret = hinic3_del_tcam_rule(nic_dev->hwdev,
+ index,
+ TCAM_RULE_FDIR_TYPE);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "del invalid tcam "
+ "rule failed!");
+ return -EFAULT;
+ }
+
+ fdir_tcam_rule.index = index;
+ fdir_tcam_rule.data.qid = queue_id;
+ tcam_key_calculate(&it->tcam_key,
+ &fdir_tcam_rule);
+
+ /* To enable a rule, add a rule. */
+ ret = hinic3_add_tcam_rule(nic_dev->hwdev,
+ &fdir_tcam_rule,
+ TCAM_RULE_FDIR_TYPE);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "add correct tcam "
+ "rule failed!");
+ return -EFAULT;
+ }
+ }
+ }
+ } else {
+ queue_res = HINIC3_INVALID_QID_BASE | queue_id;
+
+ TAILQ_FOREACH(it, &tcam_info->tcam_list, entries) {
+ if (queue_id == it->queue) {
+ index = (u16)(HINIC3_PKT_TCAM_DYNAMIC_INDEX_START
+ (it->dynamic_block_id) + it->index);
+
+ /*
+ * When the rxq is stop, delete the fdir rule
+ * from the tcam and add the corret fdir rule
+ * from the tcam.
+ */
+ ret = hinic3_del_tcam_rule(nic_dev->hwdev,
+ index,
+ TCAM_RULE_FDIR_TYPE);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "del correct tcam "
+ "rule failed!");
+ return -EFAULT;
+ }
+
+ fdir_tcam_rule.index = index;
+ fdir_tcam_rule.data.qid = queue_res;
+ tcam_key_calculate(&it->tcam_key,
+ &fdir_tcam_rule);
+
+ /* Add the corret fdir rule from the tcam. */
+ ret = hinic3_add_tcam_rule(nic_dev->hwdev,
+ &fdir_tcam_rule,
+ TCAM_RULE_FDIR_TYPE);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "add invalid tcam "
+ "rule failed!");
+ return -EFAULT;
+ }
+ }
+ }
+ }
+
+ return ret;
+}
+
+void
+hinic3_free_fdir_filter(struct rte_eth_dev *dev)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ (void)hinic3_set_fdir_tcam_rule_filter(nic_dev->hwdev, false);
+
+ (void)hinic3_flush_tcam_rule(nic_dev->hwdev);
+}
+
+static int
+hinic3_flow_set_arp_filter(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int ret;
+
+ /* Setting the ARP Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ARP,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s fdir ethertype rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ return ret;
+ }
+
+ /* Setting the ARP Request Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ARP_REQ,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s arp request rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ goto set_arp_req_failed;
+ }
+
+ /* Setting the ARP Response Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ARP_REP,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s arp response rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ goto set_arp_rep_failed;
+ }
+
+ return 0;
+
+set_arp_rep_failed:
+ (void)hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ARP_REQ,
+ ethertype_filter->queue, !add);
+
+set_arp_req_failed:
+ (void)hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ARP,
+ ethertype_filter->queue, !add);
+
+ return ret;
+}
+
+static int
+hinic3_flow_set_slow_filter(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int ret;
+
+ /* Setting the LACP Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_LACP,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s lacp fdir rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ return ret;
+ }
+
+ /* Setting the OAM Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_OAM,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s oam rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ goto set_arp_oam_failed;
+ }
+
+ return 0;
+
+set_arp_oam_failed:
+ (void)hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_LACP,
+ ethertype_filter->queue, !add);
+
+ return ret;
+}
+
+static int
+hinic3_flow_set_lldp_filter(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int ret;
+
+ /* Setting the LLDP Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_LLDP,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s lldp fdir rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ return ret;
+ }
+
+ /* Setting the CDCP Filter. */
+ ret = hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_CDCP,
+ ethertype_filter->queue, add);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s cdcp fdir rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ goto set_arp_cdcp_failed;
+ }
+
+ return 0;
+
+set_arp_cdcp_failed:
+ (void)hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_LLDP,
+ ethertype_filter->queue, !add);
+
+ return ret;
+}
+
+static int
+hinic3_flow_add_del_ethertype_filter_rule(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_ethertype_filter_list *ethertype_list =
+ &nic_dev->filter_ethertype_list;
+
+ /* Check whether the transferred rule exists. */
+ if (hinic3_ethertype_filter_lookup(ethertype_list,
+ ethertype_filter->ether_type)) {
+ if (add) {
+ PMD_DRV_LOG(ERR,
+ "The rule already exists, can not to be added");
+ return -EPERM;
+ }
+ } else {
+ if (!add) {
+ PMD_DRV_LOG(ERR,
+ "The rule not exists, can not to be delete");
+ return -EPERM;
+ }
+ }
+ /* Create a filter based on the protocol type. */
+ switch (ethertype_filter->ether_type) {
+ case RTE_ETHER_TYPE_ARP:
+ return hinic3_flow_set_arp_filter(dev, ethertype_filter, add);
+ case RTE_ETHER_TYPE_RARP:
+ return hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_RARP, ethertype_filter->queue, add);
+
+ case RTE_ETHER_TYPE_SLOW:
+ return hinic3_flow_set_slow_filter(dev, ethertype_filter, add);
+
+ case RTE_ETHER_TYPE_LLDP:
+ return hinic3_flow_set_lldp_filter(dev, ethertype_filter, add);
+
+ case RTE_ETHER_TYPE_CNM:
+ return hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_CNM, ethertype_filter->queue, add);
+
+ case RTE_ETHER_TYPE_ECP:
+ return hinic3_set_fdir_ethertype_filter(nic_dev->hwdev,
+ HINIC3_PKT_TYPE_ECP, ethertype_filter->queue, add);
+
+ default:
+ PMD_DRV_LOG(ERR, "Unknown ethertype %d queue_id %d",
+ ethertype_filter->ether_type,
+ ethertype_filter->queue);
+ return -EPERM;
+ }
+}
+
+static int
+hinic3_flow_ethertype_rule_nums(struct rte_eth_ethertype_filter *ethertype_filter)
+{
+ switch (ethertype_filter->ether_type) {
+ case RTE_ETHER_TYPE_ARP:
+ return HINIC3_ARP_RULE_NUM;
+ case RTE_ETHER_TYPE_RARP:
+ return HINIC3_RARP_RULE_NUM;
+ case RTE_ETHER_TYPE_SLOW:
+ return HINIC3_SLOW_RULE_NUM;
+ case RTE_ETHER_TYPE_LLDP:
+ return HINIC3_LLDP_RULE_NUM;
+ case RTE_ETHER_TYPE_CNM:
+ return HINIC3_CNM_RULE_NUM;
+ case RTE_ETHER_TYPE_ECP:
+ return HINIC3_ECP_RULE_NUM;
+
+ default:
+ PMD_DRV_LOG(ERR, "Unknown ethertype %d",
+ ethertype_filter->ether_type);
+ return 0;
+ }
+}
+
+/**
+ * Add or delete an Ethernet type filter rule.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] ethertype_filter
+ * Pointer to ethertype filter.
+ * @param[in] add
+ * This is a Boolean value (of the bool type) indicating whether the action to
+ * be performed is to add (true) or delete (false) the Ethernet type filter
+ * rule.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+int
+hinic3_flow_add_del_ethertype_filter(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add)
+{
+ /* Get dev private info. */
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ int ret;
+ /* Add or remove an Ethernet type filter rule. */
+ ret = hinic3_flow_add_del_ethertype_filter_rule(dev, ethertype_filter,
+ add);
+
+ if (ret) {
+ PMD_DRV_LOG(ERR, "%s fdir ethertype rule failed, err: %d",
+ add ? "Add" : "Del", ret);
+ return ret;
+ }
+ /*
+ * If a rule is added and the rule is the first rule, rule filtering is
+ * enabled. If a rule is deleted and the rule is the last one, rule
+ * filtering is disabled.
+ */
+ if (add) {
+ if (nic_dev->ethertype_rule_nums == 0) {
+ ret = hinic3_set_fdir_tcam_rule_filter(nic_dev->hwdev,
+ true);
+ if (ret) {
+ PMD_DRV_LOG(ERR,
+ "enable fdir rule failed, err: %d",
+ ret);
+ goto enable_fdir_failed;
+ }
+ }
+ nic_dev->ethertype_rule_nums =
+ nic_dev->ethertype_rule_nums +
+ hinic3_flow_ethertype_rule_nums(ethertype_filter);
+ } else {
+ nic_dev->ethertype_rule_nums =
+ nic_dev->ethertype_rule_nums -
+ hinic3_flow_ethertype_rule_nums(ethertype_filter);
+
+ if (!(nic_dev->ethertype_rule_nums + nic_dev->tcam_rule_nums)) {
+ ret = hinic3_set_fdir_tcam_rule_filter(nic_dev->hwdev,
+ false);
+ if (ret) {
+ PMD_DRV_LOG(ERR,
+ "disable fdir rule failed, err: %d",
+ ret);
+ }
+ }
+ }
+
+ return 0;
+
+enable_fdir_failed:
+ (void)hinic3_flow_add_del_ethertype_filter_rule(dev, ethertype_filter,
+ !add);
+ return ret;
+}
diff --git a/drivers/net/hinic3/hinic3_fdir.h b/drivers/net/hinic3/hinic3_fdir.h
new file mode 100644
index 0000000000..fbb2461a44
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_fdir.h
@@ -0,0 +1,398 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_FDIR_H_
+#define _HINIC3_FDIR_H_
+
+#define HINIC3_FLOW_MAX_PATTERN_NUM 16
+
+#define HINIC3_TCAM_DYNAMIC_BLOCK_SIZE 16
+
+#define HINIC3_TCAM_DYNAMIC_MAX_FILTERS 1024
+
+#define HINIC3_PKT_TCAM_DYNAMIC_INDEX_START(block_index) \
+ (HINIC3_TCAM_DYNAMIC_BLOCK_SIZE * (block_index))
+
+/* Indicate a traffic filtering rule. */
+struct rte_flow {
+ TAILQ_ENTRY(rte_flow) node;
+ enum rte_filter_type filter_type;
+ void *rule;
+};
+
+struct hinic3_fdir_rule_key {
+ struct rte_eth_ipv4_flow ipv4;
+ struct rte_eth_ipv6_flow ipv6;
+ struct rte_eth_ipv4_flow inner_ipv4;
+ struct rte_eth_ipv6_flow inner_ipv6;
+ struct rte_eth_tunnel_flow tunnel;
+ uint16_t src_port;
+ uint16_t dst_port;
+ uint8_t proto;
+};
+
+struct hinic3_fdir_filter {
+ int tcam_index;
+ uint8_t ip_type; /**< Inner ip type. */
+ uint8_t outer_ip_type;
+ uint8_t tunnel_type;
+ struct hinic3_fdir_rule_key key_mask;
+ struct hinic3_fdir_rule_key key_spec;
+ uint32_t rq_index; /**< Queue assigned when matched. */
+};
+
+/* This structure is used to describe a basic filter type. */
+struct hinic3_filter_t {
+ u16 filter_rule_nums;
+ enum rte_filter_type filter_type;
+ struct rte_eth_ethertype_filter ethertype_filter;
+ struct hinic3_fdir_filter fdir_filter;
+};
+
+enum hinic3_fdir_tunnel_mode {
+ HINIC3_FDIR_TUNNEL_MODE_NORMAL = 0,
+ HINIC3_FDIR_TUNNEL_MODE_VXLAN = 1,
+};
+
+enum hinic3_fdir_ip_type {
+ HINIC3_FDIR_IP_TYPE_IPV4 = 0,
+ HINIC3_FDIR_IP_TYPE_IPV6 = 1,
+ HINIC3_FDIR_IP_TYPE_ANY = 2,
+};
+
+/* Describe the key structure of the TCAM. */
+struct hinic3_tcam_key_mem {
+#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN)
+ u32 rsvd0 : 16;
+ u32 ip_proto : 8;
+ u32 tunnel_type : 4;
+ u32 rsvd1 : 4;
+
+ u32 function_id : 15;
+ u32 ip_type : 1;
+
+ u32 sipv4_h : 16;
+ u32 sipv4_l : 16;
+
+ u32 dipv4_h : 16;
+ u32 dipv4_l : 16;
+ u32 rsvd2 : 16;
+
+ u32 rsvd3;
+
+ u32 rsvd4 : 16;
+ u32 dport : 16;
+
+ u32 sport : 16;
+ u32 rsvd5 : 16;
+
+ u32 rsvd6 : 16;
+ u32 outer_sipv4_h : 16;
+ u32 outer_sipv4_l : 16;
+
+ u32 outer_dipv4_h : 16;
+ u32 outer_dipv4_l : 16;
+ u32 vni_h : 16;
+
+ u32 vni_l : 16;
+ u32 rsvd7 : 16;
+#else
+ u32 rsvd1 : 4;
+ u32 tunnel_type : 4;
+ u32 ip_proto : 8;
+ u32 rsvd0 : 16;
+
+ u32 sipv4_h : 16;
+ u32 ip_type : 1;
+ u32 function_id : 15;
+
+ u32 dipv4_h : 16;
+ u32 sipv4_l : 16;
+
+ u32 rsvd2 : 16;
+ u32 dipv4_l : 16;
+
+ u32 rsvd3;
+
+ u32 dport : 16;
+ u32 rsvd4 : 16;
+
+ u32 rsvd5 : 16;
+ u32 sport : 16;
+
+ u32 outer_sipv4_h : 16;
+ u32 rsvd6 : 16;
+
+ u32 outer_dipv4_h : 16;
+ u32 outer_sipv4_l : 16;
+
+ u32 vni_h : 16;
+ u32 outer_dipv4_l : 16;
+
+ u32 rsvd7 : 16;
+ u32 vni_l : 16;
+#endif
+};
+
+/*
+ * Define the IPv6-related TCAM key data structure in common
+ * scenarios or IPv6 tunnel scenarios.
+ */
+struct hinic3_tcam_key_ipv6_mem {
+#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN)
+ u32 rsvd0 : 16;
+ /* Indicates the normal IPv6 nextHdr or inner IPv4/IPv6 next proto. */
+ u32 ip_proto : 8;
+ u32 tunnel_type : 4;
+ u32 outer_ip_type : 1;
+ u32 rsvd1 : 3;
+
+ u32 function_id : 15;
+ u32 ip_type : 1;
+ u32 sipv6_key0 : 16;
+
+ u32 sipv6_key1 : 16;
+ u32 sipv6_key2 : 16;
+
+ u32 sipv6_key3 : 16;
+ u32 sipv6_key4 : 16;
+
+ u32 sipv6_key5 : 16;
+ u32 sipv6_key6 : 16;
+
+ u32 sipv6_key7 : 16;
+ u32 dport : 16;
+
+ u32 sport : 16;
+ u32 dipv6_key0 : 16;
+
+ u32 dipv6_key1 : 16;
+ u32 dipv6_key2 : 16;
+
+ u32 dipv6_key3 : 16;
+ u32 dipv6_key4 : 16;
+
+ u32 dipv6_key5 : 16;
+ u32 dipv6_key6 : 16;
+
+ u32 dipv6_key7 : 16;
+ u32 rsvd2 : 16;
+#else
+ u32 rsvd1 : 3;
+ u32 outer_ip_type : 1;
+ u32 tunnel_type : 4;
+ u32 ip_proto : 8;
+ u32 rsvd0 : 16;
+
+ u32 sipv6_key0 : 16;
+ u32 ip_type : 1;
+ u32 function_id : 15;
+
+ u32 sipv6_key2 : 16;
+ u32 sipv6_key1 : 16;
+
+ u32 sipv6_key4 : 16;
+ u32 sipv6_key3 : 16;
+
+ u32 sipv6_key6 : 16;
+ u32 sipv6_key5 : 16;
+
+ u32 dport : 16;
+ u32 sipv6_key7 : 16;
+
+ u32 dipv6_key0 : 16;
+ u32 sport : 16;
+
+ u32 dipv6_key2 : 16;
+ u32 dipv6_key1 : 16;
+
+ u32 dipv6_key4 : 16;
+ u32 dipv6_key3 : 16;
+
+ u32 dipv6_key6 : 16;
+ u32 dipv6_key5 : 16;
+
+ u32 rsvd2 : 16;
+ u32 dipv6_key7 : 16;
+#endif
+};
+
+/*
+ * Define the tcam key value data structure related to IPv6 in
+ * the VXLAN scenario.
+ */
+struct hinic3_tcam_key_vxlan_ipv6_mem {
+#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN)
+ u32 rsvd0 : 16;
+ u32 ip_proto : 8;
+ u32 tunnel_type : 4;
+ u32 rsvd1 : 4;
+
+ u32 function_id : 15;
+ u32 ip_type : 1;
+ u32 dipv6_key0 : 16;
+
+ u32 dipv6_key1 : 16;
+ u32 dipv6_key2 : 16;
+
+ u32 dipv6_key3 : 16;
+ u32 dipv6_key4 : 16;
+
+ u32 dipv6_key5 : 16;
+ u32 dipv6_key6 : 16;
+
+ u32 dipv6_key7 : 16;
+ u32 dport : 16;
+
+ u32 sport : 16;
+ u32 rsvd2 : 16;
+
+ u32 rsvd3 : 16;
+ u32 outer_sipv4_h : 16;
+
+ u32 outer_sipv4_l : 16;
+ u32 outer_dipv4_h : 16;
+
+ u32 outer_dipv4_l : 16;
+ u32 vni_h : 16;
+
+ u32 vni_l : 16;
+ u32 rsvd4 : 16;
+#else
+ u32 rsvd1 : 4;
+ u32 tunnel_type : 4;
+ u32 ip_proto : 8;
+ u32 rsvd0 : 16;
+
+ u32 dipv6_key0 : 16;
+ u32 ip_type : 1;
+ u32 function_id : 15;
+
+ u32 dipv6_key2 : 16;
+ u32 dipv6_key1 : 16;
+
+ u32 dipv6_key4 : 16;
+ u32 dipv6_key3 : 16;
+
+ u32 dipv6_key6 : 16;
+ u32 dipv6_key5 : 16;
+
+ u32 dport : 16;
+ u32 dipv6_key7 : 16;
+
+ u32 rsvd2 : 16;
+ u32 sport : 16;
+
+ u32 outer_sipv4_h : 16;
+ u32 rsvd3 : 16;
+
+ u32 outer_dipv4_h : 16;
+ u32 outer_sipv4_l : 16;
+
+ u32 vni_h : 16;
+ u32 outer_dipv4_l : 16;
+
+ u32 rsvd4 : 16;
+ u32 vni_l : 16;
+#endif
+};
+
+/*
+ * TCAM key structure. The two unions indicate the key and mask respectively.
+ * The TCAM key is consistent with the TCAM entry.
+ */
+struct hinic3_tcam_key {
+ union {
+ struct hinic3_tcam_key_mem key_info;
+ struct hinic3_tcam_key_ipv6_mem key_info_ipv6;
+ struct hinic3_tcam_key_vxlan_ipv6_mem key_info_vxlan_ipv6;
+ };
+ union {
+ struct hinic3_tcam_key_mem key_mask;
+ struct hinic3_tcam_key_ipv6_mem key_mask_ipv6;
+ struct hinic3_tcam_key_vxlan_ipv6_mem key_mask_vxlan_ipv6;
+ };
+};
+
+/* Structure indicates the TCAM filter. */
+struct hinic3_tcam_filter {
+ TAILQ_ENTRY(hinic3_tcam_filter)
+ entries; /**< Filter entry, used for linked list operations. */
+ uint16_t dynamic_block_id; /**< Dynamic block ID. */
+ uint16_t index; /**< TCAM index. */
+ struct hinic3_tcam_key tcam_key; /**< Indicate TCAM key. */
+ uint16_t queue; /**< Allocated RX queue. */
+};
+
+/* Define a linked list header for storing hinic3_tcam_filter data. */
+TAILQ_HEAD(hinic3_tcam_filter_list, hinic3_tcam_filter);
+
+struct hinic3_tcam_dynamic_block {
+ TAILQ_ENTRY(hinic3_tcam_dynamic_block) entries;
+ u16 dynamic_block_id;
+ u16 dynamic_index_cnt;
+ u8 dynamic_index[HINIC3_TCAM_DYNAMIC_BLOCK_SIZE];
+};
+
+/* Define a linked list header for storing hinic3_tcam_dynamic_block data. */
+TAILQ_HEAD(hinic3_tcam_dynamic_filter_list, hinic3_tcam_dynamic_block);
+
+/* Indicate TCAM dynamic block info. */
+struct hinic3_tcam_dynamic_block_info {
+ struct hinic3_tcam_dynamic_filter_list tcam_dynamic_list;
+ u16 dynamic_block_cnt;
+};
+
+/* Structure is used to store TCAM information. */
+struct hinic3_tcam_info {
+ struct hinic3_tcam_filter_list tcam_list;
+ struct hinic3_tcam_dynamic_block_info tcam_dynamic_info;
+};
+
+/* Obtain the upper and lower 16 bits. */
+#define HINIC3_32_UPPER_16_BITS(n) ((((n) >> 16)) & 0xffff)
+#define HINIC3_32_LOWER_16_BITS(n) ((n) & 0xffff)
+
+/* Number of protocol rules */
+#define HINIC3_ARP_RULE_NUM 3
+#define HINIC3_RARP_RULE_NUM 1
+#define HINIC3_SLOW_RULE_NUM 2
+#define HINIC3_LLDP_RULE_NUM 2
+#define HINIC3_CNM_RULE_NUM 1
+#define HINIC3_ECP_RULE_NUM 2
+
+/* Define Ethernet type. */
+#define RTE_ETHER_TYPE_CNM 0x22e7
+#define RTE_ETHER_TYPE_ECP 0x8940
+
+/* Protocol type of the data packet. */
+enum hinic3_ether_type {
+ HINIC3_PKT_TYPE_ARP = 1,
+ HINIC3_PKT_TYPE_ARP_REQ,
+ HINIC3_PKT_TYPE_ARP_REP,
+ HINIC3_PKT_TYPE_RARP,
+ HINIC3_PKT_TYPE_LACP,
+ HINIC3_PKT_TYPE_LLDP,
+ HINIC3_PKT_TYPE_OAM,
+ HINIC3_PKT_TYPE_CDCP,
+ HINIC3_PKT_TYPE_CNM,
+ HINIC3_PKT_TYPE_ECP = 10,
+
+ HINIC3_PKT_UNKNOWN = 31,
+};
+
+int hinic3_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
+ struct hinic3_fdir_filter *fdir_filter,
+ bool add);
+int hinic3_flow_add_del_ethertype_filter(struct rte_eth_dev *dev,
+ struct rte_eth_ethertype_filter *ethertype_filter,
+ bool add);
+
+void hinic3_free_fdir_filter(struct rte_eth_dev *dev);
+int hinic3_enable_rxq_fdir_filter(struct rte_eth_dev *dev, u32 queue_id,
+ u32 able);
+int hinic3_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error);
+
+#endif /**< _HINIC3_FDIR_H_ */
diff --git a/drivers/net/hinic3/hinic3_flow.c b/drivers/net/hinic3/hinic3_flow.c
new file mode 100644
index 0000000000..b310848530
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_flow.c
@@ -0,0 +1,1700 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_malloc.h>
+
+#include "base/hinic3_compat.h"
+#include "base/hinic3_hwdev.h"
+#include "base/hinic3_nic_cfg.h"
+#include "hinic3_ethdev.h"
+#include "hinic3_fdir.h"
+#include "hinic3_flow.h"
+
+#define HINIC3_UINT8_MAX 0xff
+
+/* Indicate the type of the IPv4 ICPM matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_icmp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_ICMP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 any protocol matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_any[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_ANY,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the Ether matching pattern. */
+static enum rte_flow_item_type pattern_ethertype[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the TCP matching pattern. */
+static enum rte_flow_item_type pattern_ethertype_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_TCP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the UDP matching pattern. */
+static enum rte_flow_item_type pattern_ethertype_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_UDP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan any protocol matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_any[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ANY, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_TCP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv4 matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv4[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv4 TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv4_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_TCP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv4 UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv4_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv6 matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv6[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv6 TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv6_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_TCP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 vxlan IPv6 UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_vxlan_ipv6_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 matching pattern. */
+static enum rte_flow_item_type pattern_ipv4[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_UDP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv4 TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv4_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV4,
+ HINIC3_FLOW_ITEM_TYPE_TCP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 matching pattern. */
+static enum rte_flow_item_type pattern_ipv6[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH,
+ HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_TCP,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_vxlan[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 VXLAN any protocol matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_vxlan_any[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_ANY, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 VXLAN TCP matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_vxlan_tcp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_TCP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+/* Indicate the type of the IPv6 VXLAN UDP matching pattern. */
+static enum rte_flow_item_type pattern_ipv6_vxlan_udp[] = {
+ HINIC3_FLOW_ITEM_TYPE_ETH, HINIC3_FLOW_ITEM_TYPE_IPV6,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_VXLAN,
+ HINIC3_FLOW_ITEM_TYPE_UDP, HINIC3_FLOW_ITEM_TYPE_END,
+};
+
+typedef int (*hinic3_parse_filter_t)(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter);
+
+/* Indicate valid filter mode . */
+struct hinic3_valid_pattern {
+ enum rte_flow_item_type *items;
+ hinic3_parse_filter_t parse_filter;
+};
+
+static int hinic3_flow_parse_fdir_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter);
+
+static int hinic3_flow_parse_ethertype_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter);
+
+static int hinic3_flow_parse_fdir_vxlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter);
+
+/*
+ * Define a supported pattern array, including the matching patterns of
+ * various network protocols and corresponding parsing functions.
+ */
+static const struct hinic3_valid_pattern hinic3_supported_patterns[] = {
+ /* Support ethertype. */
+ {pattern_ethertype, hinic3_flow_parse_ethertype_filter},
+ /* Support ipv4 but not tunnel, and any field can be masked. */
+ {pattern_ipv4, hinic3_flow_parse_fdir_filter},
+ {pattern_ipv4_any, hinic3_flow_parse_fdir_filter},
+ /* Support ipv4 + l4 but not tunnel, and any field can be masked. */
+ {pattern_ipv4_udp, hinic3_flow_parse_fdir_filter},
+ {pattern_ipv4_tcp, hinic3_flow_parse_fdir_filter},
+ /* Support ipv4 + icmp not tunnel, and any field can be masked. */
+ {pattern_ipv4_icmp, hinic3_flow_parse_fdir_filter},
+
+ /* Support ipv4 + l4 but not tunnel, and any field can be masked. */
+ {pattern_ethertype_udp, hinic3_flow_parse_fdir_filter},
+ {pattern_ethertype_tcp, hinic3_flow_parse_fdir_filter},
+
+ /* Support ipv4 + vxlan + any, and any field can be masked. */
+ {pattern_ipv4_vxlan, hinic3_flow_parse_fdir_vxlan_filter},
+ /* Support ipv4 + vxlan + ipv4, and any field can be masked. */
+ {pattern_ipv4_vxlan_ipv4, hinic3_flow_parse_fdir_vxlan_filter},
+ /* Support ipv4 + vxlan + ipv4 + l4, and any field can be masked. */
+ {pattern_ipv4_vxlan_ipv4_tcp, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv4_vxlan_ipv4_udp, hinic3_flow_parse_fdir_vxlan_filter},
+ /* Support ipv4 + vxlan + ipv6, and any field can be masked. */
+ {pattern_ipv4_vxlan_ipv6, hinic3_flow_parse_fdir_vxlan_filter},
+ /* Support ipv4 + vxlan + ipv6 + l4, and any field can be masked. */
+ {pattern_ipv4_vxlan_ipv6_tcp, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv4_vxlan_ipv6_udp, hinic3_flow_parse_fdir_vxlan_filter},
+ /* Support ipv4 + vxlan + l4, and any field can be masked. */
+ {pattern_ipv4_vxlan_tcp, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv4_vxlan_udp, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv4_vxlan_any, hinic3_flow_parse_fdir_vxlan_filter},
+
+ /* Support ipv6 but not tunnel, and any field can be masked. */
+ {pattern_ipv6, hinic3_flow_parse_fdir_filter},
+ /* Support ipv6 + l4 but not tunnel, and any field can be masked. */
+ {pattern_ipv6_udp, hinic3_flow_parse_fdir_filter},
+ {pattern_ipv6_tcp, hinic3_flow_parse_fdir_filter},
+
+ /* Support ipv6 + vxlan + any, and any field can be masked. */
+ {pattern_ipv6_vxlan, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv6_vxlan_any, hinic3_flow_parse_fdir_vxlan_filter},
+
+ /* Support ipv6 + vxlan + l4, and any field can be masked. */
+ {pattern_ipv6_vxlan_tcp, hinic3_flow_parse_fdir_vxlan_filter},
+ {pattern_ipv6_vxlan_udp, hinic3_flow_parse_fdir_vxlan_filter},
+
+};
+
+static inline void
+net_addr_to_host(uint32_t *dst, const uint32_t *src, size_t len)
+{
+ size_t i;
+ for (i = 0; i < len; i++)
+ dst[i] = rte_be_to_cpu_32(src[i]);
+}
+
+static bool
+hinic3_match_pattern(enum rte_flow_item_type *item_array,
+ const struct rte_flow_item *pattern)
+{
+ const struct rte_flow_item *item = pattern;
+
+ /* skip the first void item. */
+ while (item->type == HINIC3_FLOW_ITEM_TYPE_VOID)
+ item++;
+
+ /* Find no void item. */
+ while (((*item_array == item->type) &&
+ (*item_array != HINIC3_FLOW_ITEM_TYPE_END)) ||
+ (item->type == HINIC3_FLOW_ITEM_TYPE_VOID)) {
+ if (item->type == HINIC3_FLOW_ITEM_TYPE_VOID) {
+ item++;
+ } else {
+ item_array++;
+ item++;
+ }
+ }
+
+ return (*item_array == HINIC3_FLOW_ITEM_TYPE_END &&
+ item->type == HINIC3_FLOW_ITEM_TYPE_END);
+}
+
+/**
+ * Find matching parsing filter functions.
+ *
+ * @param[in] pattern
+ * Pattern to match.
+ * @return
+ * Matched resolution filter. If no resolution filter is found, return NULL.
+ */
+static hinic3_parse_filter_t
+hinic3_find_parse_filter_func(const struct rte_flow_item *pattern)
+{
+ hinic3_parse_filter_t parse_filter = NULL;
+ uint8_t i;
+ /* Traverse all supported patterns. */
+ for (i = 0; i < RTE_DIM(hinic3_supported_patterns); i++) {
+ if (hinic3_match_pattern(hinic3_supported_patterns[i].items,
+ pattern)) {
+ parse_filter =
+ hinic3_supported_patterns[i].parse_filter;
+ break;
+ }
+ }
+
+ return parse_filter;
+}
+
+/**
+ * Action for parsing and processing Ethernet types.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, its used to store and manipulate packet filtering rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ const struct rte_flow_action_queue *act_q;
+ const struct rte_flow_action *act = actions;
+
+ /* skip the first void item. */
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID)
+ act++;
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->fdir_filter.rq_index = act_q->index;
+ if (filter->fdir_filter.rq_index >= dev->data->nb_rx_queues) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action param.");
+ return -rte_errno;
+ }
+ break;
+ default:
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ACTION,
+ act, "Invalid action type.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+int
+hinic3_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ /* Not supported. */
+ if (!attr->ingress || attr->egress || attr->priority || attr->group) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_UNSPECIFIED, attr,
+ "Only support ingress.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_ipv4(const struct rte_flow_item *flow_item,
+ struct hinic3_filter_t *filter,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv4 *spec_ipv4, *mask_ipv4;
+
+ mask_ipv4 = (const struct rte_flow_item_ipv4 *)flow_item->mask;
+ spec_ipv4 = (const struct rte_flow_item_ipv4 *)flow_item->spec;
+ if (!mask_ipv4 || !spec_ipv4) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter ipv4 mask or spec");
+ return -rte_errno;
+ }
+
+ /*
+ * Only support src address , dst addresses, proto,
+ * others should be masked.
+ */
+ if (mask_ipv4->hdr.version_ihl || mask_ipv4->hdr.type_of_service ||
+ mask_ipv4->hdr.total_length || mask_ipv4->hdr.packet_id ||
+ mask_ipv4->hdr.fragment_offset || mask_ipv4->hdr.time_to_live ||
+ mask_ipv4->hdr.hdr_checksum) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, ipv4 only "
+ "support src ip, dst ip, proto");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_IPV4;
+ filter->fdir_filter.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_NORMAL;
+ filter->fdir_filter.key_mask.ipv4.src_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_spec.ipv4.src_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_mask.ipv4.dst_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.dst_addr);
+ filter->fdir_filter.key_spec.ipv4.dst_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.dst_addr);
+ filter->fdir_filter.key_mask.proto = mask_ipv4->hdr.next_proto_id;
+ filter->fdir_filter.key_spec.proto = spec_ipv4->hdr.next_proto_id;
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_ipv6(const struct rte_flow_item *flow_item,
+ struct hinic3_filter_t *filter,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv6 *spec_ipv6, *mask_ipv6;
+
+ mask_ipv6 = (const struct rte_flow_item_ipv6 *)flow_item->mask;
+ spec_ipv6 = (const struct rte_flow_item_ipv6 *)flow_item->spec;
+ if (!mask_ipv6 || !spec_ipv6) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter ipv6 mask or spec");
+ return -rte_errno;
+ }
+
+ /* Only support dst addresses, src addresses, proto. */
+ if (mask_ipv6->hdr.vtc_flow || mask_ipv6->hdr.payload_len ||
+ mask_ipv6->hdr.hop_limits) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, ipv6 only "
+ "support src ip, dst ip, proto");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+ filter->fdir_filter.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_NORMAL;
+ net_addr_to_host(filter->fdir_filter.key_mask.ipv6.src_ip,
+ (const uint32_t *)mask_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.ipv6.src_ip,
+ (const uint32_t *)spec_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_mask.ipv6.dst_ip,
+ (const uint32_t *)mask_ipv6->hdr.dst_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.ipv6.dst_ip,
+ (const uint32_t *)spec_ipv6->hdr.dst_addr.a, 4);
+ filter->fdir_filter.key_mask.proto = mask_ipv6->hdr.proto;
+ filter->fdir_filter.key_spec.proto = spec_ipv6->hdr.proto;
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_tcp(const struct rte_flow_item *flow_item,
+ struct hinic3_filter_t *filter,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_tcp *spec_tcp, *mask_tcp;
+
+ mask_tcp = (const struct rte_flow_item_tcp *)flow_item->mask;
+ spec_tcp = (const struct rte_flow_item_tcp *)flow_item->spec;
+
+ filter->fdir_filter.key_mask.proto = HINIC3_UINT8_MAX;
+ filter->fdir_filter.key_spec.proto = IPPROTO_TCP;
+
+ if (!mask_tcp && !spec_tcp)
+ return 0;
+
+ if (!mask_tcp || !spec_tcp) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter tcp mask or spec");
+ return -rte_errno;
+ }
+
+ /* Only support src, dst ports, others should be masked. */
+ if (mask_tcp->hdr.sent_seq || mask_tcp->hdr.recv_ack ||
+ mask_tcp->hdr.data_off || mask_tcp->hdr.rx_win ||
+ mask_tcp->hdr.tcp_flags || mask_tcp->hdr.cksum ||
+ mask_tcp->hdr.tcp_urp) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, tcp only "
+ "support src port, dst port");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.src_port =
+ (u16)rte_be_to_cpu_16(mask_tcp->hdr.src_port);
+ filter->fdir_filter.key_spec.src_port =
+ (u16)rte_be_to_cpu_16(spec_tcp->hdr.src_port);
+ filter->fdir_filter.key_mask.dst_port =
+ (u16)rte_be_to_cpu_16(mask_tcp->hdr.dst_port);
+ filter->fdir_filter.key_spec.dst_port =
+ (u16)rte_be_to_cpu_16(spec_tcp->hdr.dst_port);
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_udp(const struct rte_flow_item *flow_item,
+ struct hinic3_filter_t *filter,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_udp *spec_udp, *mask_udp;
+
+ mask_udp = (const struct rte_flow_item_udp *)flow_item->mask;
+ spec_udp = (const struct rte_flow_item_udp *)flow_item->spec;
+
+ filter->fdir_filter.key_mask.proto = HINIC3_UINT8_MAX;
+ filter->fdir_filter.key_spec.proto = IPPROTO_UDP;
+
+ if (!mask_udp && !spec_udp)
+ return 0;
+
+ if (!mask_udp || !spec_udp) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter udp mask or spec");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.src_port =
+ (u16)rte_be_to_cpu_16(mask_udp->hdr.src_port);
+ filter->fdir_filter.key_spec.src_port =
+ (u16)rte_be_to_cpu_16(spec_udp->hdr.src_port);
+ filter->fdir_filter.key_mask.dst_port =
+ (u16)rte_be_to_cpu_16(mask_udp->hdr.dst_port);
+ filter->fdir_filter.key_spec.dst_port =
+ (u16)rte_be_to_cpu_16(spec_udp->hdr.dst_port);
+
+ return 0;
+}
+
+/**
+ * Parse the pattern of network traffic and apply the parsing result to the
+ * traffic filter.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, Its used to store and manipulate packet filtering rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse_fdir_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ const struct rte_flow_item *flow_item = pattern;
+ enum rte_flow_item_type type;
+ int err;
+
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_ANY;
+ /* Traverse all modes until HINIC3_FLOW_ITEM_TYPE_END is reached. */
+ for (; flow_item->type != HINIC3_FLOW_ITEM_TYPE_END; flow_item++) {
+ if (flow_item->last) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item, "Not support range");
+ return -rte_errno;
+ }
+ type = flow_item->type;
+ switch (type) {
+ case HINIC3_FLOW_ITEM_TYPE_ETH:
+ if (flow_item->spec || flow_item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir "
+ "filter, not support mac");
+ return -rte_errno;
+ }
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_IPV4:
+ err = hinic3_flow_fdir_ipv4(flow_item, filter, error);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_IPV6:
+ err = hinic3_flow_fdir_ipv6(flow_item, filter, error);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_TCP:
+ err = hinic3_flow_fdir_tcp(flow_item, filter, error);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_UDP:
+ err = hinic3_flow_fdir_udp(flow_item, filter, error);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Resolve rules for network traffic filters.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] attr
+ * Indicates the attribute of a flow rule.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, Its used to store and manipulate packet filtering rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse_fdir_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ int ret;
+
+ ret = hinic3_flow_parse_fdir_pattern(dev, pattern, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_action(dev, actions, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_attr(attr, error);
+ if (ret)
+ return ret;
+
+ filter->filter_type = RTE_ETH_FILTER_FDIR;
+
+ return 0;
+}
+
+/**
+ * Parse and process the actions of the Ethernet type.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, Its used to store and manipulate packet filtering rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse_ethertype_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ const struct rte_flow_action *act = actions;
+ const struct rte_flow_action_queue *act_q;
+
+ /* Skip the firset void item. */
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID)
+ act++;
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->ethertype_filter.queue = act_q->index;
+ if (filter->ethertype_filter.queue >= dev->data->nb_rx_queues) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action param.");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ACTION,
+ act, "Invalid action type.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+hinic3_flow_parse_ethertype_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ const struct rte_flow_item_eth *ether_spec, *ether_mask;
+ const struct rte_flow_item *flow_item = pattern;
+ enum rte_flow_item_type type;
+
+ /* Traverse all modes until HINIC3_FLOW_ITEM_TYPE_END is reached. */
+ for (; flow_item->type != HINIC3_FLOW_ITEM_TYPE_END; flow_item++) {
+ if (flow_item->last) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item, "Not support range");
+ return -rte_errno;
+ }
+ type = flow_item->type;
+ switch (type) {
+ case HINIC3_FLOW_ITEM_TYPE_ETH:
+ /* Obtaining Ethernet Specifications and Masks. */
+ ether_spec = (const struct rte_flow_item_eth *)
+ flow_item->spec;
+ ether_mask = (const struct rte_flow_item_eth *)
+ flow_item->mask;
+ if (!ether_spec || !ether_mask) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "NULL ETH spec/mask");
+ return -rte_errno;
+ }
+
+ /*
+ * Mask bits of source MAC address must be full of 0.
+ * Mask bits of destination MAC address must be full 0.
+ * Filters traffic based on the type of Ethernet.
+ */
+ if (!rte_is_zero_ether_addr(ðer_mask->src) ||
+ (!rte_is_zero_ether_addr(ðer_mask->dst))) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid ether address mask");
+ return -rte_errno;
+ }
+
+ if ((ether_mask->type & UINT16_MAX) != UINT16_MAX) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid ethertype mask");
+ return -rte_errno;
+ }
+
+ filter->ethertype_filter.ether_type =
+ (u16)rte_be_to_cpu_16(ether_spec->type);
+
+ switch (filter->ethertype_filter.ether_type) {
+ case RTE_ETHER_TYPE_SLOW:
+ break;
+
+ case RTE_ETHER_TYPE_ARP:
+ break;
+
+ case RTE_ETHER_TYPE_RARP:
+ break;
+
+ case RTE_ETHER_TYPE_LLDP:
+ break;
+
+ default:
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Unsupported ether_type in"
+ " control packet filter.");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static int
+hinic3_flow_parse_ethertype_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ int ret;
+
+ ret = hinic3_flow_parse_ethertype_pattern(dev, pattern, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_ethertype_action(dev, actions, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_attr(attr, error);
+ if (ret)
+ return ret;
+
+ filter->filter_type = RTE_ETH_FILTER_ETHERTYPE;
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_tunnel_ipv4(struct rte_flow_error *error,
+ struct hinic3_filter_t *filter,
+ const struct rte_flow_item *flow_item,
+ enum hinic3_fdir_tunnel_mode tunnel_mode)
+{
+ const struct rte_flow_item_ipv4 *spec_ipv4, *mask_ipv4;
+ mask_ipv4 = (const struct rte_flow_item_ipv4 *)flow_item->mask;
+ spec_ipv4 = (const struct rte_flow_item_ipv4 *)flow_item->spec;
+
+ if (tunnel_mode == HINIC3_FDIR_TUNNEL_MODE_NORMAL) {
+ filter->fdir_filter.outer_ip_type = HINIC3_FDIR_IP_TYPE_IPV4;
+
+ if (!mask_ipv4 && !spec_ipv4)
+ return 0;
+
+ if (!mask_ipv4 || !spec_ipv4) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter, vxlan outer "
+ "ipv4 mask or spec");
+ return -rte_errno;
+ }
+
+ /*
+ * Only support src address , dst addresses, others should be
+ * masked.
+ */
+ if (mask_ipv4->hdr.version_ihl ||
+ mask_ipv4->hdr.type_of_service ||
+ mask_ipv4->hdr.total_length || mask_ipv4->hdr.packet_id ||
+ mask_ipv4->hdr.fragment_offset ||
+ mask_ipv4->hdr.time_to_live ||
+ mask_ipv4->hdr.next_proto_id ||
+ mask_ipv4->hdr.hdr_checksum) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, "
+ "vxlan outer ipv4 only support "
+ "src ip, dst ip");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.ipv4.src_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_spec.ipv4.src_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_mask.ipv4.dst_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.dst_addr);
+ filter->fdir_filter.key_spec.ipv4.dst_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.dst_addr);
+ } else {
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_IPV4;
+
+ if (!mask_ipv4 && !spec_ipv4)
+ return 0;
+
+ if (!mask_ipv4 || !spec_ipv4) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter, vxlan inner "
+ "ipv4 mask or spec");
+ return -rte_errno;
+ }
+
+ /*
+ * Only support src addr , dst addr, ip proto, others should be
+ * masked.
+ */
+ if (mask_ipv4->hdr.version_ihl ||
+ mask_ipv4->hdr.type_of_service ||
+ mask_ipv4->hdr.total_length || mask_ipv4->hdr.packet_id ||
+ mask_ipv4->hdr.fragment_offset ||
+ mask_ipv4->hdr.time_to_live ||
+ mask_ipv4->hdr.hdr_checksum) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, "
+ "vxlan inner ipv4 only support "
+ "src ip, dst ip, proto");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.inner_ipv4.src_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_spec.inner_ipv4.src_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.src_addr);
+ filter->fdir_filter.key_mask.inner_ipv4.dst_ip =
+ rte_be_to_cpu_32(mask_ipv4->hdr.dst_addr);
+ filter->fdir_filter.key_spec.inner_ipv4.dst_ip =
+ rte_be_to_cpu_32(spec_ipv4->hdr.dst_addr);
+ filter->fdir_filter.key_mask.proto =
+ mask_ipv4->hdr.next_proto_id;
+ filter->fdir_filter.key_spec.proto =
+ spec_ipv4->hdr.next_proto_id;
+ }
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_tunnel_ipv6(struct rte_flow_error *error,
+ struct hinic3_filter_t *filter,
+ const struct rte_flow_item *flow_item,
+ enum hinic3_fdir_tunnel_mode tunnel_mode)
+{
+ const struct rte_flow_item_ipv6 *spec_ipv6, *mask_ipv6;
+
+ mask_ipv6 = (const struct rte_flow_item_ipv6 *)flow_item->mask;
+ spec_ipv6 = (const struct rte_flow_item_ipv6 *)flow_item->spec;
+
+ if (tunnel_mode == HINIC3_FDIR_TUNNEL_MODE_NORMAL) {
+ filter->fdir_filter.outer_ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+
+ if (!mask_ipv6 && !spec_ipv6)
+ return 0;
+
+ if (!mask_ipv6 || !spec_ipv6) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM, flow_item,
+ "Invalid fdir filter ipv6 mask or spec");
+ return -rte_errno;
+ }
+
+ /* Only support dst addresses, src addresses. */
+ if (mask_ipv6->hdr.vtc_flow || mask_ipv6->hdr.payload_len ||
+ mask_ipv6->hdr.hop_limits || mask_ipv6->hdr.proto) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM, flow_item,
+ "Not supported by fdir filter, ipv6 only "
+ "support src ip, dst ip, proto");
+ return -rte_errno;
+ }
+
+ net_addr_to_host(filter->fdir_filter.key_mask.ipv6.src_ip,
+ (const uint32_t *)mask_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.ipv6.src_ip,
+ (const uint32_t *)spec_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_mask.ipv6.dst_ip,
+ (const uint32_t *)mask_ipv6->hdr.dst_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.ipv6.dst_ip,
+ (const uint32_t *)spec_ipv6->hdr.dst_addr.a, 4);
+ } else {
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_IPV6;
+
+ if (!mask_ipv6 && !spec_ipv6)
+ return 0;
+
+ if (!mask_ipv6 || !spec_ipv6) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM, flow_item,
+ "Invalid fdir filter ipv6 mask or spec");
+ return -rte_errno;
+ }
+
+ /* Only support dst addresses, src addresses, proto. */
+ if (mask_ipv6->hdr.vtc_flow || mask_ipv6->hdr.payload_len ||
+ mask_ipv6->hdr.hop_limits) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM, flow_item,
+ "Not supported by fdir filter, ipv6 only "
+ "support src ip, dst ip, proto");
+ return -rte_errno;
+ }
+
+ net_addr_to_host(filter->fdir_filter.key_mask.inner_ipv6.src_ip,
+ (const uint32_t *)mask_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.inner_ipv6.src_ip,
+ (const uint32_t *)spec_ipv6->hdr.src_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_mask.inner_ipv6.dst_ip,
+ (const uint32_t *)mask_ipv6->hdr.dst_addr.a, 4);
+ net_addr_to_host(filter->fdir_filter.key_spec.inner_ipv6.dst_ip,
+ (const uint32_t *)spec_ipv6->hdr.dst_addr.a, 4);
+
+ filter->fdir_filter.key_mask.proto = mask_ipv6->hdr.proto;
+ filter->fdir_filter.key_spec.proto = spec_ipv6->hdr.proto;
+ }
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_tunnel_tcp(struct rte_flow_error *error,
+ struct hinic3_filter_t *filter,
+ enum hinic3_fdir_tunnel_mode tunnel_mode,
+ const struct rte_flow_item *flow_item)
+{
+ const struct rte_flow_item_tcp *spec_tcp, *mask_tcp;
+
+ if (tunnel_mode == HINIC3_FDIR_TUNNEL_MODE_NORMAL) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, vxlan only "
+ "support inner tcp");
+ return -rte_errno;
+ }
+
+ filter->fdir_filter.key_mask.proto = HINIC3_UINT8_MAX;
+ filter->fdir_filter.key_spec.proto = IPPROTO_TCP;
+
+ mask_tcp = (const struct rte_flow_item_tcp *)flow_item->mask;
+ spec_tcp = (const struct rte_flow_item_tcp *)flow_item->spec;
+ if (!mask_tcp && !spec_tcp)
+ return 0;
+ if (!mask_tcp || !spec_tcp) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter tcp mask or spec");
+ return -rte_errno;
+ }
+
+ /* Only support src, dst ports, others should be masked. */
+ if (mask_tcp->hdr.sent_seq || mask_tcp->hdr.recv_ack ||
+ mask_tcp->hdr.data_off || mask_tcp->hdr.rx_win ||
+ mask_tcp->hdr.tcp_flags || mask_tcp->hdr.cksum ||
+ mask_tcp->hdr.tcp_urp) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir filter, vxlan inner "
+ "tcp only support src port,dst port");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.src_port =
+ (u16)rte_be_to_cpu_16(mask_tcp->hdr.src_port);
+ filter->fdir_filter.key_spec.src_port =
+ (u16)rte_be_to_cpu_16(spec_tcp->hdr.src_port);
+ filter->fdir_filter.key_mask.dst_port =
+ (u16)rte_be_to_cpu_16(mask_tcp->hdr.dst_port);
+ filter->fdir_filter.key_spec.dst_port =
+ (u16)rte_be_to_cpu_16(spec_tcp->hdr.dst_port);
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_tunnel_udp(struct rte_flow_error *error,
+ struct hinic3_filter_t *filter,
+ enum hinic3_fdir_tunnel_mode tunnel_mode,
+ const struct rte_flow_item *flow_item)
+{
+ const struct rte_flow_item_udp *spec_udp, *mask_udp;
+
+ mask_udp = (const struct rte_flow_item_udp *)flow_item->mask;
+ spec_udp = (const struct rte_flow_item_udp *)flow_item->spec;
+
+ if (tunnel_mode == HINIC3_FDIR_TUNNEL_MODE_NORMAL) {
+ /*
+ * UDP is used to describe protocol,
+ * spec and mask should be NULL.
+ */
+ if (flow_item->spec || flow_item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item, "Invalid UDP item");
+ return -rte_errno;
+ }
+ } else {
+ filter->fdir_filter.key_mask.proto = HINIC3_UINT8_MAX;
+ filter->fdir_filter.key_spec.proto = IPPROTO_UDP;
+ if (!mask_udp && !spec_udp)
+ return 0;
+
+ if (!mask_udp || !spec_udp) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter vxlan inner "
+ "udp mask or spec");
+ return -rte_errno;
+ }
+
+ /* Set the filter information. */
+ filter->fdir_filter.key_mask.src_port =
+ (u16)rte_be_to_cpu_16(mask_udp->hdr.src_port);
+ filter->fdir_filter.key_spec.src_port =
+ (u16)rte_be_to_cpu_16(spec_udp->hdr.src_port);
+ filter->fdir_filter.key_mask.dst_port =
+ (u16)rte_be_to_cpu_16(mask_udp->hdr.dst_port);
+ filter->fdir_filter.key_spec.dst_port =
+ (u16)rte_be_to_cpu_16(spec_udp->hdr.dst_port);
+ }
+
+ return 0;
+}
+
+static int
+hinic3_flow_fdir_vxlan(struct rte_flow_error *error,
+ struct hinic3_filter_t *filter,
+ const struct rte_flow_item *flow_item)
+{
+ const struct rte_flow_item_vxlan *spec_vxlan, *mask_vxlan;
+ uint32_t vxlan_vni_id = 0;
+
+ spec_vxlan = (const struct rte_flow_item_vxlan *)flow_item->spec;
+ mask_vxlan = (const struct rte_flow_item_vxlan *)flow_item->mask;
+
+ filter->fdir_filter.tunnel_type = HINIC3_FDIR_TUNNEL_MODE_VXLAN;
+
+ if (!spec_vxlan && !mask_vxlan) {
+ return 0;
+ } else if (filter->fdir_filter.outer_ip_type == HINIC3_FDIR_IP_TYPE_IPV6) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter vxlan mask or spec, "
+ "ipv6 vxlan, don't support vni");
+ return -rte_errno;
+ }
+
+ if (!spec_vxlan || !mask_vxlan) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Invalid fdir filter vxlan mask or spec");
+ return -rte_errno;
+ }
+
+ rte_memcpy(((uint8_t *)&vxlan_vni_id + 1), spec_vxlan->vni, 3);
+ filter->fdir_filter.key_mask.tunnel.tunnel_id =
+ rte_be_to_cpu_32(vxlan_vni_id);
+ return 0;
+}
+
+static int
+hinic3_flow_parse_fdir_vxlan_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ const struct rte_flow_item *flow_item = pattern;
+ enum hinic3_fdir_tunnel_mode tunnel_mode =
+ HINIC3_FDIR_TUNNEL_MODE_NORMAL;
+ enum rte_flow_item_type type;
+ int err;
+
+ /* Inner and outer ip type, set it to any by default */
+ filter->fdir_filter.ip_type = HINIC3_FDIR_IP_TYPE_ANY;
+ filter->fdir_filter.outer_ip_type = HINIC3_FDIR_IP_TYPE_ANY;
+
+ for (; flow_item->type != HINIC3_FLOW_ITEM_TYPE_END; flow_item++) {
+ if (flow_item->last) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item, "Not support range");
+ return -rte_errno;
+ }
+
+ type = flow_item->type;
+ switch (type) {
+ case HINIC3_FLOW_ITEM_TYPE_ETH:
+ /* All should be masked. */
+ if (flow_item->spec || flow_item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_ITEM,
+ flow_item,
+ "Not supported by fdir "
+ "filter, not support mac");
+ return -rte_errno;
+ }
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_IPV4:
+ err = hinic3_flow_fdir_tunnel_ipv4(error,
+ filter, flow_item, tunnel_mode);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_IPV6:
+ err = hinic3_flow_fdir_tunnel_ipv6(error,
+ filter, flow_item, tunnel_mode);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_TCP:
+ err = hinic3_flow_fdir_tunnel_tcp(error,
+ filter, tunnel_mode, flow_item);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_UDP:
+ err = hinic3_flow_fdir_tunnel_udp(error,
+ filter, tunnel_mode, flow_item);
+ if (err != 0)
+ return -rte_errno;
+ break;
+
+ case HINIC3_FLOW_ITEM_TYPE_VXLAN:
+ err = hinic3_flow_fdir_vxlan(error, filter, flow_item);
+ if (err != 0)
+ return -rte_errno;
+ tunnel_mode = HINIC3_FDIR_TUNNEL_MODE_VXLAN;
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Resolve VXLAN Filters in Flow Filters.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] attr
+ * Indicates the attribute of a flow rule.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, its used to store and manipulate packet filterig rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse_fdir_vxlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ struct hinic3_filter_t *filter)
+{
+ int ret;
+
+ ret = hinic3_flow_parse_fdir_vxlan_pattern(dev, pattern, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_action(dev, actions, error, filter);
+ if (ret)
+ return ret;
+
+ ret = hinic3_flow_parse_attr(attr, error);
+ if (ret)
+ return ret;
+
+ filter->filter_type = RTE_ETH_FILTER_FDIR;
+
+ return 0;
+}
+
+/**
+ * Parse patterns and actions of network traffic.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] attr
+ * Indicates the attribute of a flow rule.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @param[out] filter
+ * Filter information, its used to store and manipulate packet filterig rules.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_parse(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error, struct hinic3_filter_t *filter)
+{
+ hinic3_parse_filter_t parse_filter;
+ uint32_t pattern_num = 0;
+ int ret = 0;
+ /* Check whether the parameter is valid. */
+ if (!pattern || !actions || !attr) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "NULL param.");
+ return -rte_errno;
+ }
+
+ while ((pattern + pattern_num)->type != HINIC3_FLOW_ITEM_TYPE_END) {
+ pattern_num++;
+ if (pattern_num > HINIC3_FLOW_MAX_PATTERN_NUM) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_MAX_PATTERN_NUM, NULL,
+ "Too many patterns.");
+ return -rte_errno;
+ }
+ }
+ /*
+ * The corresponding filter is returned. If the filter is not found,
+ * NULL is returned.
+ */
+ parse_filter = hinic3_find_parse_filter_func(pattern);
+ if (!parse_filter) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_ITEM,
+ pattern, "Unsupported pattern");
+ return -rte_errno;
+ }
+ /* Parsing with filters. */
+ ret = parse_filter(dev, attr, pattern, actions, error, filter);
+
+ return ret;
+}
+
+/**
+ * Check whether the traffic rule provided by the user is valid.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] attr
+ * Indicates the attribute of a flow rule.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct hinic3_filter_t filter_rules = {0};
+
+ return hinic3_flow_parse(dev, attr, pattern, actions, error,
+ &filter_rules);
+}
+
+/**
+ * Create a flow item.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[in] attr
+ * Indicates the attribute of a flow rule.
+ * @param[in] pattern
+ * Indicates the pattern or matching condition of a traffic rule.
+ * @param[in] actions
+ * Indicates the action to be taken on the matched traffic.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @return
+ * If the operation is successful, the created flow is returned. Otherwise, NULL
+ * is returned.
+ *
+ */
+static struct rte_flow *
+hinic3_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct hinic3_filter_t *filter_rules = NULL;
+ struct rte_flow *flow = NULL;
+ int ret;
+
+ filter_rules =
+ rte_zmalloc("filter_rules", sizeof(struct hinic3_filter_t), 0);
+ if (!filter_rules) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL,
+ "Failed to allocate filter rules memory.");
+ return NULL;
+ }
+
+ flow = rte_zmalloc("hinic3_rte_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL, "Failed to allocate flow memory.");
+ rte_free(filter_rules);
+ return NULL;
+ }
+ /* Parses the flow rule to be created and generates a filter. */
+ ret = hinic3_flow_parse(dev, attr, pattern, actions, error,
+ filter_rules);
+ if (ret < 0)
+ goto free_flow;
+
+ switch (filter_rules->filter_type) {
+ case RTE_ETH_FILTER_ETHERTYPE:
+ ret = hinic3_flow_add_del_ethertype_filter(dev,
+ &filter_rules->ethertype_filter, true);
+ if (ret) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Create ethertype filter failed.");
+ goto free_flow;
+ }
+
+ flow->rule = filter_rules;
+ flow->filter_type = filter_rules->filter_type;
+ TAILQ_INSERT_TAIL(&nic_dev->filter_ethertype_list, flow, node);
+ break;
+
+ case RTE_ETH_FILTER_FDIR:
+ ret = hinic3_flow_add_del_fdir_filter(dev,
+ &filter_rules->fdir_filter, true);
+ if (ret) {
+ rte_flow_error_set(error, EINVAL,
+ HINIC3_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Create fdir filter failed.");
+ goto free_flow;
+ }
+
+ flow->rule = filter_rules;
+ flow->filter_type = filter_rules->filter_type;
+ TAILQ_INSERT_TAIL(&nic_dev->filter_fdir_rule_list, flow, node);
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Filter type %d not supported",
+ filter_rules->filter_type);
+ rte_flow_error_set(error, EINVAL, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL, "Unsupport filter type.");
+ goto free_flow;
+ }
+
+ return flow;
+
+free_flow:
+ rte_free(flow);
+ rte_free(filter_rules);
+
+ return NULL;
+}
+
+static int
+hinic3_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow,
+ struct rte_flow_error *error)
+{
+ int ret = -EINVAL;
+ enum rte_filter_type type;
+ struct hinic3_filter_t *rules = NULL;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+
+ if (!flow) {
+ PMD_DRV_LOG(ERR, "Invalid flow parameter!");
+ return -EPERM;
+ }
+
+ type = flow->filter_type;
+ rules = (struct hinic3_filter_t *)flow->rule;
+ /* Perform operations based on the type. */
+ switch (type) {
+ case RTE_ETH_FILTER_ETHERTYPE:
+ ret = hinic3_flow_add_del_ethertype_filter(dev,
+ &rules->ethertype_filter, false);
+ if (!ret)
+ TAILQ_REMOVE(&nic_dev->filter_ethertype_list, flow,
+ node);
+
+ flow->rule = rules;
+ flow->filter_type = rules->filter_type;
+ TAILQ_REMOVE(&nic_dev->filter_ethertype_list, flow, node);
+ break;
+
+ case RTE_ETH_FILTER_FDIR:
+ ret = hinic3_flow_add_del_fdir_filter(dev, &rules->fdir_filter,
+ false);
+ if (!ret)
+ TAILQ_REMOVE(&nic_dev->filter_fdir_rule_list, flow,
+ node);
+ break;
+ default:
+ PMD_DRV_LOG(WARNING, "Filter type %d not supported", type);
+ ret = -EINVAL;
+ break;
+ }
+
+ /* Deleted successfully. Resources are released. */
+ if (!ret) {
+ rte_free(rules);
+ rte_free(flow);
+ } else {
+ rte_flow_error_set(error, -ret, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL, "Failed to destroy flow.");
+ }
+
+ return ret;
+}
+
+/**
+ * Clear all fdir type flow rules on the network device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_flush_fdir_filter(struct rte_eth_dev *dev)
+{
+ int ret = 0;
+ struct hinic3_filter_t *filter_rules = NULL;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_flow *flow;
+
+ while (true) {
+ flow = TAILQ_FIRST(&nic_dev->filter_fdir_rule_list);
+ if (flow == NULL)
+ break;
+ filter_rules = (struct hinic3_filter_t *)flow->rule;
+
+ /* Delete flow rules. */
+ ret = hinic3_flow_add_del_fdir_filter(dev,
+ &filter_rules->fdir_filter, false);
+
+ if (ret)
+ return ret;
+
+ TAILQ_REMOVE(&nic_dev->filter_fdir_rule_list, flow, node);
+ rte_free(filter_rules);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+/**
+ * Clear all ether type flow rules on the network device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_flush_ethertype_filter(struct rte_eth_dev *dev)
+{
+ struct hinic3_filter_t *filter_rules = NULL;
+ struct hinic3_nic_dev *nic_dev = HINIC3_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
+ struct rte_flow *flow;
+ int ret = 0;
+
+ while (true) {
+ flow = TAILQ_FIRST(&nic_dev->filter_ethertype_list);
+ if (flow == NULL)
+ break;
+ filter_rules = (struct hinic3_filter_t *)flow->rule;
+
+ /* Delete flow rules. */
+ ret = hinic3_flow_add_del_ethertype_filter(dev,
+ &filter_rules->ethertype_filter, false);
+
+ if (ret)
+ return ret;
+
+ TAILQ_REMOVE(&nic_dev->filter_ethertype_list, flow, node);
+ rte_free(filter_rules);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+/**
+ * Clear all flow rules on the network device.
+ *
+ * @param[in] dev
+ * Pointer to ethernet device structure.
+ * @param[out] error
+ * Structure that contains error information, such as error code and error
+ * description.
+ * @return
+ * 0 on success, non-zero on failure.
+ */
+static int
+hinic3_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ int ret;
+
+ ret = hinic3_flow_flush_fdir_filter(dev);
+ if (ret) {
+ rte_flow_error_set(error, -ret, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL, "Failed to flush fdir flows.");
+ return -rte_errno;
+ }
+
+ ret = hinic3_flow_flush_ethertype_filter(dev);
+ if (ret) {
+ rte_flow_error_set(error, -ret, HINIC3_FLOW_ERROR_TYPE_HANDLE,
+ NULL, "Failed to flush ethertype flows.");
+ return -rte_errno;
+ }
+ return ret;
+}
+
+/* Structure for managing flow table operations. */
+const struct rte_flow_ops hinic3_flow_ops = {
+ .validate = hinic3_flow_validate,
+ .create = hinic3_flow_create,
+ .destroy = hinic3_flow_destroy,
+ .flush = hinic3_flow_flush,
+};
diff --git a/drivers/net/hinic3/hinic3_flow.h b/drivers/net/hinic3/hinic3_flow.h
new file mode 100644
index 0000000000..9104337544
--- /dev/null
+++ b/drivers/net/hinic3/hinic3_flow.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Huawei Technologies Co., Ltd
+ */
+
+#ifndef _HINIC3_FLOW_H_
+#define _HINIC3_FLOW_H_
+
+#include <rte_flow.h>
+
+/* Flow item type. */
+#define HINIC3_FLOW_ITEM_TYPE_END RTE_FLOW_ITEM_TYPE_END
+#define HINIC3_FLOW_ITEM_TYPE_VOID RTE_FLOW_ITEM_TYPE_VOID
+#define HINIC3_FLOW_ITEM_TYPE_INVERT RTE_FLOW_ITEM_TYPE_INVERT
+#define HINIC3_FLOW_ITEM_TYPE_ANY RTE_FLOW_ITEM_TYPE_ANY
+#define HINIC3_FLOW_ITEM_TYPE_PF RTE_FLOW_ITEM_TYPE_PF
+#define HINIC3_FLOW_ITEM_TYPE_VF RTE_FLOW_ITEM_TYPE_VF
+#define HINIC3_FLOW_ITEM_TYPE_PHY_PORT RTE_FLOW_ITEM_TYPE_PHY_PORT
+#define HINIC3_FLOW_ITEM_TYPE_PORT_ID RTE_FLOW_ITEM_TYPE_PORT_ID
+#define HINIC3_FLOW_ITEM_TYPE_RAW RTE_FLOW_ITEM_TYPE_RAW
+#define HINIC3_FLOW_ITEM_TYPE_ETH RTE_FLOW_ITEM_TYPE_ETH
+#define HINIC3_FLOW_ITEM_TYPE_VLAN RTE_FLOW_ITEM_TYPE_VLAN
+#define HINIC3_FLOW_ITEM_TYPE_IPV4 RTE_FLOW_ITEM_TYPE_IPV4
+#define HINIC3_FLOW_ITEM_TYPE_IPV6 RTE_FLOW_ITEM_TYPE_IPV6
+#define HINIC3_FLOW_ITEM_TYPE_ICMP RTE_FLOW_ITEM_TYPE_ICMP
+#define HINIC3_FLOW_ITEM_TYPE_UDP RTE_FLOW_ITEM_TYPE_UDP
+#define HINIC3_FLOW_ITEM_TYPE_TCP RTE_FLOW_ITEM_TYPE_TCP
+#define HINIC3_FLOW_ITEM_TYPE_SCTP RTE_FLOW_ITEM_TYPE_SCTP
+#define HINIC3_FLOW_ITEM_TYPE_VXLAN RTE_FLOW_ITEM_TYPE_VXLAN
+#define HINIC3_FLOW_ITEM_TYPE_E_TAG RTE_FLOW_ITEM_TYPE_E_TAG
+#define HINIC3_FLOW_ITEM_TYPE_NVGRE RTE_FLOW_ITEM_TYPE_NVGRE
+#define HINIC3_FLOW_ITEM_TYPE_MPLS RTE_FLOW_ITEM_TYPE_MPLS
+#define HINIC3_FLOW_ITEM_TYPE_GRE RTE_FLOW_ITEM_TYPE_GRE
+#define HINIC3_FLOW_ITEM_TYPE_FUZZY RTE_FLOW_ITEM_TYPE_FUZZY
+#define HINIC3_FLOW_ITEM_TYPE_GTP RTE_FLOW_ITEM_TYPE_GTP
+#define HINIC3_FLOW_ITEM_TYPE_GTPC RTE_FLOW_ITEM_TYPE_GTPC
+#define HINIC3_FLOW_ITEM_TYPE_GTPU RTE_FLOW_ITEM_TYPE_GTPU
+#define HINIC3_FLOW_ITEM_TYPE_ESP RTE_FLOW_ITEM_TYPE_ESP
+#define HINIC3_FLOW_ITEM_TYPE_GENEVE RTE_FLOW_ITEM_TYPE_GENEVE
+#define HINIC3_FLOW_ITEM_TYPE_VXLAN_GPE RTE_FLOW_ITEM_TYPE_VXLAN_GPE
+#define HINIC3_FLOW_ITEM_TYPE_ARP_ETH_IPV4 RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4
+#define HINIC3_FLOW_ITEM_TYPE_IPV6_EXT RTE_FLOW_ITEM_TYPE_IPV6_EXT
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6 RTE_FLOW_ITEM_TYPE_ICMP6
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6_ND_NS RTE_FLOW_ITEM_TYPE_ICMP6_ND_NS
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6_ND_NA RTE_FLOW_ITEM_TYPE_ICMP6_ND_NA
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6_ND_OPT RTE_FLOW_ITEM_TYPE_ICMP6_ND_OPT
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6_ND_OPT_SLA_ETH RTE_FLOW_ITEM_TYPE_ICMP6_ND_OPT_SLA_ETH
+#define HINIC3_FLOW_ITEM_TYPE_ICMP6_ND_OPT_TLA_ETH RTE_FLOW_ITEM_TYPE_ICMP6_ND_OPT_TLA_ETH
+#define HINIC3_FLOW_ITEM_TYPE_MARK RTE_FLOW_ITEM_TYPE_MARK
+#define HINIC3_FLOW_ITEM_TYPE_META RTE_FLOW_ITEM_TYPE_META
+#define HINIC3_FLOW_ITEM_TYPE_GRE_KEY RTE_FLOW_ITEM_TYPE_GRE_KEY
+#define HINIC3_FLOW_ITEM_TYPE_GTP_PSC RTE_FLOW_ITEM_TYPE_GTP_PSC
+#define HINIC3_FLOW_ITEM_TYPE_PPPOES RTE_FLOW_ITEM_TYPE_PPPOES
+#define HINIC3_FLOW_ITEM_TYPE_PPPOED RTE_FLOW_ITEM_TYPE_PPPOED
+#define HINIC3_FLOW_ITEM_TYPE_PPPOE_PROTO_ID RTE_FLOW_ITEM_TYPE_PPPOE_PROTO_ID
+#define HINIC3_FLOW_ITEM_TYPE_NSH RTE_FLOW_ITEM_TYPE_NSH
+#define HINIC3_FLOW_ITEM_TYPE_IGMP RTE_FLOW_ITEM_TYPE_IGMP
+#define HINIC3_FLOW_ITEM_TYPE_AH RTE_FLOW_ITEM_TYPE_AH
+#define HINIC3_FLOW_ITEM_TYPE_HIGIG2 RTE_FLOW_ITEM_TYPE_HIGIG2
+#define HINIC3_FLOW_ITEM_TYPE_TAG RTE_FLOW_ITEM_TYPE_TAG
+
+/* Flow error type. */
+#define HINIC3_FLOW_ERROR_TYPE_NONE RTE_FLOW_ERROR_TYPE_NONE
+#define HINIC3_FLOW_ERROR_TYPE_UNSPECIFIED RTE_FLOW_ERROR_TYPE_UNSPECIFIED
+#define HINIC3_FLOW_ERROR_TYPE_HANDLE RTE_FLOW_ERROR_TYPE_HANDLE
+#define HINIC3_FLOW_ERROR_TYPE_ATTR_GROUP RTE_FLOW_ERROR_TYPE_ATTR_GROUP
+#define HINIC3_FLOW_ERROR_TYPE_ATTR_PRIORITY RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY
+#define HINIC3_FLOW_ERROR_TYPE_ATTR_INGRESS RTE_FLOW_ERROR_TYPE_ATTR_INGRESS
+#define HINIC3_FLOW_ERROR_TYPE_ATTR_EGRESS RTE_FLOW_ERROR_TYPE_ATTR_EGRESS
+#define HINIC3_FLOW_ERROR_TYPE_ATTR_TRANSFER RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER
+#define HINIC3_FLOW_ERROR_TYPE_ATTR RTE_FLOW_ERROR_TYPE_ATTR
+#define HINIC3_FLOW_ERROR_TYPE_ITEM_NUM RTE_FLOW_ERROR_TYPE_ITEM_NUM
+#define HINIC3_FLOW_ERROR_TYPE_ITEM_SPEC RTE_FLOW_ERROR_TYPE_ITEM_SPEC
+#define HINIC3_FLOW_ERROR_TYPE_ITEM_LAST RTE_FLOW_ERROR_TYPE_ITEM_LAST
+#define HINIC3_FLOW_ERROR_TYPE_ITEM_MASK RTE_FLOW_ERROR_TYPE_ITEM_MASK
+#define HINIC3_FLOW_ERROR_TYPE_ITEM RTE_FLOW_ERROR_TYPE_ITEM
+#define HINIC3_FLOW_ERROR_TYPE_ACTION_NUM RTE_FLOW_ERROR_TYPE_ACTION_NUM
+#define HINIC3_FLOW_ERROR_TYPE_ACTION_CONF RTE_FLOW_ERROR_TYPE_ACTION_CONF
+#define HINIC3_FLOW_ERROR_TYPE_ACTION RTE_FLOW_ERROR_TYPE_ACTION
+
+#endif /**< _HINIC3_FLOW_H_ */
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [RFC 18/18] drivers/net: add hinic3 PMD build and doc files
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (8 preceding siblings ...)
2025-04-18 7:02 ` [RFC 17/18] net/hinic3: add FDIR flow control module Feifei Wang
@ 2025-04-18 7:02 ` Feifei Wang
2025-05-20 15:47 ` [RFC 00/18] add hinic3 PMD driver Stephen Hemminger
10 siblings, 0 replies; 15+ messages in thread
From: Feifei Wang @ 2025-04-18 7:02 UTC (permalink / raw)
To: dev; +Cc: Yi Chen, Xin Wang, Feifei Wang
From: Yi Chen <chenyi221@huawei.com>
The meson.build file is added to this patch to enable
the hinic3 compilation function.
Signed-off-by: Yi Chen <chenyi221@huawei.com>
Reviewed-by: Xin Wang <wangxin679@h-partners.com>
Reviewed-by: Feifei Wang <wangfeifei40@huawei.com>
---
doc/guides/nics/features/hinic3.ini | 9 ++++++
drivers/net/hinic3/base/meson.build | 50 +++++++++++++++++++++++++++++
drivers/net/hinic3/meson.build | 44 +++++++++++++++++++++++++
drivers/net/meson.build | 1 +
4 files changed, 104 insertions(+)
create mode 100644 doc/guides/nics/features/hinic3.ini
create mode 100644 drivers/net/hinic3/base/meson.build
create mode 100644 drivers/net/hinic3/meson.build
diff --git a/doc/guides/nics/features/hinic3.ini b/doc/guides/nics/features/hinic3.ini
new file mode 100644
index 0000000000..8bafd49090
--- /dev/null
+++ b/doc/guides/nics/features/hinic3.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'hinic3' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-64 = Y
+ARMv8 = Y
diff --git a/drivers/net/hinic3/base/meson.build b/drivers/net/hinic3/base/meson.build
new file mode 100644
index 0000000000..948f5efac2
--- /dev/null
+++ b/drivers/net/hinic3/base/meson.build
@@ -0,0 +1,50 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Huawei Technologies Co., Ltd
+
+sources = files(
+ 'hinic3_cmdq.c',
+ 'hinic3_eqs.c',
+ 'hinic3_hw_cfg.c',
+ 'hinic3_hw_comm.c',
+ 'hinic3_hwdev.c',
+ 'hinic3_hwif.c',
+ 'hinic3_mbox.c',
+ 'hinic3_mgmt.c',
+ 'hinic3_nic_cfg.c',
+ 'hinic3_nic_event.c',
+ 'hinic3_wq.c',
+)
+
+extra_flags = []
+
+# The driver runs only on arch64 machine, remove 32bit warnings
+if not dpdk_conf.get('RTE_ARCH_64')
+ extra_flags += [
+ '-Wno-int-to-pointer-cast',
+ '-Wno-pointer-to-int-cast',
+ ]
+endif
+
+foreach flag: extra_flags
+ if cc.has_argument(flag)
+ cflags += flag
+ endif
+endforeach
+
+deps += ['hash']
+c_args = cflags
+includes += include_directories('../')
+
+base_lib = static_library(
+ 'spnic_base',
+ sources,
+ dependencies: [
+ static_rte_eal,
+ static_rte_ethdev,
+ static_rte_bus_pci,
+ static_rte_hash,
+ ],
+ include_directories: includes,
+ c_args: c_args,
+)
+base_objs = base_lib.extract_all_objects()
diff --git a/drivers/net/hinic3/meson.build b/drivers/net/hinic3/meson.build
new file mode 100644
index 0000000000..231e966b36
--- /dev/null
+++ b/drivers/net/hinic3/meson.build
@@ -0,0 +1,44 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Huawei Technologies Co., Ltd
+
+if not is_linux
+ build = false
+ reason = 'only supported on Linux'
+ subdir_done()
+endif
+
+if (arch_subdir != 'x86' and arch_subdir != 'arm'
+ or not dpdk_conf.get('RTE_ARCH_64'))
+ build = false
+ reason = 'only supported on x86_64 and aarch64'
+ subdir_done()
+endif
+
+cflags += [
+ '-DHW_CONVERT_ENDIAN',
+ '-D__HINIC_HUAWEI_SECUREC__',
+ '-fPIC',
+ '-fstack-protector-strong',
+]
+
+subdir('base')
+subdir('mml')
+objs = [base_objs] + [mml_objs]
+
+sources = files(
+ 'hinic3_ethdev.c',
+ 'hinic3_fdir.c',
+ 'hinic3_flow.c',
+ 'hinic3_nic_io.c',
+ 'hinic3_rx.c',
+ 'hinic3_tx.c',
+)
+
+if arch_subdir == 'arm' and dpdk_conf.get('RTE_ARCH_64')
+ cflags += ['-D__ARM64_NEON__']
+else
+ cflags += ['-D__X86_64_SSE__']
+endif
+
+includes += include_directories('base')
+includes += include_directories('mml')
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 460eb69e5b..b5442349d4 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -23,6 +23,7 @@ drivers = [
'failsafe',
'gve',
'hinic',
+ 'hinic3',
'hns3',
'intel/e1000',
'intel/fm10k',
--
2.47.0.windows.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [RFC 00/18] add hinic3 PMD driver
2025-04-18 7:02 [RFC 00/18] add hinic3 PMD driver Feifei Wang
` (9 preceding siblings ...)
2025-04-18 7:02 ` [RFC 18/18] drivers/net: add hinic3 PMD build and doc files Feifei Wang
@ 2025-05-20 15:47 ` Stephen Hemminger
10 siblings, 0 replies; 15+ messages in thread
From: Stephen Hemminger @ 2025-05-20 15:47 UTC (permalink / raw)
To: Feifei Wang; +Cc: dev
On Fri, 18 Apr 2025 15:02:38 +0800
Feifei Wang <wff_light@vip.163.com> wrote:
> *** BLURB HERE ***
> The hinic3 PMD (**librte_net_hinic3**) provides poll mode driver support
> for 25Gbps/100Gbps/200Gbps Huawei SPx series Network Adapters.
>
> Feifei Wang (3):
> net/hinic3: add intro doc for hinic3
> net/hinic3: add dev ops
> net/hinic3: add Rx/Tx functions
>
> Xin Wang (7):
> net/hinic3: add basic header files
> net/hinic3: add support for cmdq mechanism
> net/hinic3: add NIC event module
> net/hinic3: add context and work queue support
> net/hinic3: add device initailization
> net/hinic3: add MML and EEPROM access feature
> net/hinic3: add RSS promiscuous ops
>
> Yi Chen (8):
> net/hinic3: add hardware interfaces of BAR operation
> net/hinic3: add eq mechanism function code
> net/hinic3: add mgmt module function code
> net/hinic3: add module about hardware operation
> net/hinic3: add a NIC business configuration module
> net/hinic3: add a mailbox communication module
> net/hinic3: add FDIR flow control module
> drivers/net: add hinic3 PMD build and doc files
>
> .mailmap | 4 +-
> MAINTAINERS | 6 +
> doc/guides/nics/features/hinic3.ini | 9 +
> doc/guides/nics/hinic3.rst | 52 +
> doc/guides/nics/index.rst | 1 +
> doc/guides/rel_notes/release_25_07.rst | 32 +-
> drivers/net/hinic3/base/hinic3_cmd.h | 231 ++
> drivers/net/hinic3/base/hinic3_cmdq.c | 975 +++++
> drivers/net/hinic3/base/hinic3_cmdq.h | 230 ++
> drivers/net/hinic3/base/hinic3_compat.h | 266 ++
> drivers/net/hinic3/base/hinic3_csr.h | 108 +
> drivers/net/hinic3/base/hinic3_eqs.c | 719 ++++
> drivers/net/hinic3/base/hinic3_eqs.h | 98 +
> drivers/net/hinic3/base/hinic3_hw_cfg.c | 240 ++
> drivers/net/hinic3/base/hinic3_hw_cfg.h | 121 +
> drivers/net/hinic3/base/hinic3_hw_comm.c | 452 +++
> drivers/net/hinic3/base/hinic3_hw_comm.h | 366 ++
> drivers/net/hinic3/base/hinic3_hwdev.c | 573 +++
> drivers/net/hinic3/base/hinic3_hwdev.h | 177 +
> drivers/net/hinic3/base/hinic3_hwif.c | 779 ++++
> drivers/net/hinic3/base/hinic3_hwif.h | 142 +
> drivers/net/hinic3/base/hinic3_mbox.c | 1392 +++++++
> drivers/net/hinic3/base/hinic3_mbox.h | 199 +
> drivers/net/hinic3/base/hinic3_mgmt.c | 392 ++
> drivers/net/hinic3/base/hinic3_mgmt.h | 121 +
> drivers/net/hinic3/base/hinic3_nic_cfg.c | 1828 +++++++++
> drivers/net/hinic3/base/hinic3_nic_cfg.h | 1527 ++++++++
> drivers/net/hinic3/base/hinic3_nic_event.c | 433 +++
> drivers/net/hinic3/base/hinic3_nic_event.h | 39 +
> drivers/net/hinic3/base/hinic3_wq.c | 148 +
> drivers/net/hinic3/base/hinic3_wq.h | 109 +
> drivers/net/hinic3/base/meson.build | 50 +
> drivers/net/hinic3/hinic3_ethdev.c | 3866 ++++++++++++++++++++
> drivers/net/hinic3/hinic3_ethdev.h | 167 +
> drivers/net/hinic3/hinic3_fdir.c | 1394 +++++++
> drivers/net/hinic3/hinic3_fdir.h | 398 ++
> drivers/net/hinic3/hinic3_flow.c | 1700 +++++++++
> drivers/net/hinic3/hinic3_flow.h | 80 +
> drivers/net/hinic3/hinic3_nic_io.c | 827 +++++
> drivers/net/hinic3/hinic3_nic_io.h | 169 +
> drivers/net/hinic3/hinic3_rx.c | 1096 ++++++
> drivers/net/hinic3/hinic3_rx.h | 356 ++
> drivers/net/hinic3/hinic3_tx.c | 1028 ++++++
> drivers/net/hinic3/hinic3_tx.h | 315 ++
> drivers/net/hinic3/meson.build | 44 +
> drivers/net/hinic3/mml/hinic3_dbg.c | 171 +
> drivers/net/hinic3/mml/hinic3_dbg.h | 160 +
> drivers/net/hinic3/mml/hinic3_mml_cmd.c | 375 ++
> drivers/net/hinic3/mml/hinic3_mml_cmd.h | 131 +
> drivers/net/hinic3/mml/hinic3_mml_ioctl.c | 215 ++
> drivers/net/hinic3/mml/hinic3_mml_lib.c | 136 +
> drivers/net/hinic3/mml/hinic3_mml_lib.h | 275 ++
> drivers/net/hinic3/mml/hinic3_mml_main.c | 167 +
> drivers/net/hinic3/mml/hinic3_mml_queue.c | 749 ++++
> drivers/net/hinic3/mml/hinic3_mml_queue.h | 256 ++
> drivers/net/hinic3/mml/meson.build | 62 +
> drivers/net/meson.build | 1 +
> 57 files changed, 25926 insertions(+), 31 deletions(-)
> create mode 100644 doc/guides/nics/features/hinic3.ini
> create mode 100644 doc/guides/nics/hinic3.rst
> create mode 100644 drivers/net/hinic3/base/hinic3_cmd.h
> create mode 100644 drivers/net/hinic3/base/hinic3_cmdq.c
> create mode 100644 drivers/net/hinic3/base/hinic3_cmdq.h
> create mode 100644 drivers/net/hinic3/base/hinic3_compat.h
> create mode 100644 drivers/net/hinic3/base/hinic3_csr.h
> create mode 100644 drivers/net/hinic3/base/hinic3_eqs.c
> create mode 100644 drivers/net/hinic3/base/hinic3_eqs.h
> create mode 100644 drivers/net/hinic3/base/hinic3_hw_cfg.c
> create mode 100644 drivers/net/hinic3/base/hinic3_hw_cfg.h
> create mode 100644 drivers/net/hinic3/base/hinic3_hw_comm.c
> create mode 100644 drivers/net/hinic3/base/hinic3_hw_comm.h
> create mode 100644 drivers/net/hinic3/base/hinic3_hwdev.c
> create mode 100644 drivers/net/hinic3/base/hinic3_hwdev.h
> create mode 100644 drivers/net/hinic3/base/hinic3_hwif.c
> create mode 100644 drivers/net/hinic3/base/hinic3_hwif.h
> create mode 100644 drivers/net/hinic3/base/hinic3_mbox.c
> create mode 100644 drivers/net/hinic3/base/hinic3_mbox.h
> create mode 100644 drivers/net/hinic3/base/hinic3_mgmt.c
> create mode 100644 drivers/net/hinic3/base/hinic3_mgmt.h
> create mode 100644 drivers/net/hinic3/base/hinic3_nic_cfg.c
> create mode 100644 drivers/net/hinic3/base/hinic3_nic_cfg.h
> create mode 100644 drivers/net/hinic3/base/hinic3_nic_event.c
> create mode 100644 drivers/net/hinic3/base/hinic3_nic_event.h
> create mode 100644 drivers/net/hinic3/base/hinic3_wq.c
> create mode 100644 drivers/net/hinic3/base/hinic3_wq.h
> create mode 100644 drivers/net/hinic3/base/meson.build
> create mode 100644 drivers/net/hinic3/hinic3_ethdev.c
> create mode 100644 drivers/net/hinic3/hinic3_ethdev.h
> create mode 100644 drivers/net/hinic3/hinic3_fdir.c
> create mode 100644 drivers/net/hinic3/hinic3_fdir.h
> create mode 100644 drivers/net/hinic3/hinic3_flow.c
> create mode 100644 drivers/net/hinic3/hinic3_flow.h
> create mode 100644 drivers/net/hinic3/hinic3_nic_io.c
> create mode 100644 drivers/net/hinic3/hinic3_nic_io.h
> create mode 100644 drivers/net/hinic3/hinic3_rx.c
> create mode 100644 drivers/net/hinic3/hinic3_rx.h
> create mode 100644 drivers/net/hinic3/hinic3_tx.c
> create mode 100644 drivers/net/hinic3/hinic3_tx.h
> create mode 100644 drivers/net/hinic3/meson.build
> create mode 100644 drivers/net/hinic3/mml/hinic3_dbg.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_dbg.h
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_cmd.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_cmd.h
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_ioctl.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_lib.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_lib.h
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_main.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_queue.c
> create mode 100644 drivers/net/hinic3/mml/hinic3_mml_queue.h
> create mode 100644 drivers/net/hinic3/mml/meson.build
>
If you can in future, please try to fix spelling in base/ files.
It would reduce the checkpatch warnings.
There are spelling errors in non-base files as well.
### [PATCH] net/hinic3: add dev ops
WARNING:TYPO_SPELLING: 'regsitered' may be misspelled - perhaps 'registered'?
#327: FILE: drivers/net/hinic3/hinic3_ethdev.c:331:
+ * The address of parameter (struct rte_eth_dev *) regsitered before.
^^^^^^^^^^
WARNING:TYPO_SPELLING: 'resourece' may be misspelled - perhaps 'resource'?
#1893: FILE: drivers/net/hinic3/hinic3_ethdev.c:1897:
+ * and free io resourece.
^^^^^^^^^
CHECK:CAMELCASE: Avoid CamelCase: <PRIu64>
#3522: FILE: drivers/net/hinic3/hinic3_nic_io.c:550:
+ "err: %d, out_param: %" PRIu64,
CHECK:CAMELCASE: Avoid CamelCase: <PRIx64>
#3787: FILE: drivers/net/hinic3/hinic3_nic_io.c:815:
+ PMD_DRV_LOG(INFO, "Update nic feature to 0x%" PRIx64,
WARNING:TYPO_SPELLING: 'coverting' may be misspelled - perhaps 'converting'?
#3905: FILE: drivers/net/hinic3/hinic3_nic_io.h:100:
+ /* Hardware will do endianness coverting. */
^^^^^^^^^
WARNING:TYPO_SPELLING: 'indrect' may be misspelled - perhaps 'indirect'?
#4428: FILE: drivers/net/hinic3/hinic3_rx.c:448:
+ "Set indrect table failed, eth_dev:%s, queue_idx:%d",
^^^^^^^
WARNING:TYPO_SPELLING: 'Does't' may be misspelled - perhaps 'Doesn't'?
#4471: FILE: drivers/net/hinic3/hinic3_rx.c:491:
+ PMD_DRV_LOG(ERR, "Does't support rss hash type: %" PRIu64,
^^^^^^
WARNING:TYPO_SPELLING: 'possition' may be misspelled - perhaps 'position'?
#4519: FILE: drivers/net/hinic3/hinic3_rx.c:539:
+ * Search given queue array to find possition of given id.
^^^^^^^^^
WARNING:TYPO_SPELLING: 'pakcet' may be misspelled - perhaps 'packet'?
#4704: FILE: drivers/net/hinic3/hinic3_rx.c:724:
+ * If RSS is disable, no mbuf in rq, pakcet will be dropped.
^^^^^^
WARNING:TYPO_SPELLING: 'indrect' may be misspelled - perhaps 'indirect'?
#4771: FILE: drivers/net/hinic3/hinic3_rx.c:791:
+ "Refill rq to indrect table failed, "
^^^^^^^
total: 0 errors, 8 warnings, 2 checks, 5700 lines checked
### [PATCH] net/hinic3: add Rx/Tx functions
WARNING:TYPO_SPELLING: 'indrect' may be misspelled - perhaps 'indirect'?
#213: FILE: drivers/net/hinic3/hinic3_rx.c:955:
+ "Refill rq to indrect table failed, "
^^^^^^^
WARNING:TYPO_SPELLING: 'warpped' may be misspelled - perhaps 'wrapped'?
#1068: FILE: drivers/net/hinic3/hinic3_tx.c:921:
+ /* Task or bd section maybe warpped for one wqe. */
^^^^^^^
total: 0 errors, 2 warnings, 0 checks, 1104 lines checked
### [PATCH] net/hinic3: add MML and EEPROM access feature
WARNING:TYPO_SPELLING: 'reslut' may be misspelled - perhaps 'result'?
#697: FILE: drivers/net/hinic3/mml/hinic3_mml_cmd.c:292:
+copy_reslut_to_buffer(void *buf_out, char *reslut, int len)
^^^^^^
WARNING:TYPO_SPELLING: 'reslut' may be misspelled - perhaps 'result'?
#701: FILE: drivers/net/hinic3/mml/hinic3_mml_cmd.c:296:
+ ret = snprintf(buf_out, len - 1, "%s", reslut);
^^^^^^
I notice the rte_flow check fails:
$ ./devtools/check-doc-vs-code.sh
rte_flow doc out of sync for hinic3
item ah
item any
item arp_eth_ipv4
item e_tag
item esp
item eth
item fuzzy
item geneve
item gre
item gre_key
item gtp
item gtp_psc
item gtpc
item gtpu
item higig2
item icmp
item icmp6
item icmp6_nd_na
item icmp6_nd_ns
item icmp6_nd_opt
item icmp6_nd_opt_sla_eth
item icmp6_nd_opt_tla_eth
item igmp
item invert
item ipv4
item ipv6
item ipv6_ext
item mark
item meta
item mpls
item nsh
item nvgre
item pf
item phy_port
item port_id
item pppoe_proto_id
item pppoed
item pppoes
item raw
item sctp
item tag
item tcp
item udp
item vf
item vlan
item vxlan
item vxlan_gpe
action queue
^ permalink raw reply [flat|nested] 15+ messages in thread