* [PATCH v10 00/23] et/sxe2: added Linkdata sxe2 ethernet driver
From: liujie5 @ 2026-06-27 4:04 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260626064954.361771-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch set implements core functionality for the SXE2 PMD,
including basic driver framework, data path setup, and advanced
offload features (VLAN, RSS,TM, PTP etc.).
V10:
Introduce the 'sxe2_txrx_check_mbuf' helper function to validate
outgoing mbuf tunnel type flags when RTE_ETHDEV_DEBUG_TX is enabled.
The function checks that the RTE_MBUF_F_TX_TUNNEL_x flag in mbuf
ol_flags matches the actual tunnel protocol detected in the packet
(GTP, VXLAN, VXLAN-GPE, Geneve, GRE, or IPIP).
Jie Liu (23):
net/sxe2: remove software statistics devargs
net/sxe2: add Rx framework and packet types callback
net/sxe2: support AVX512 vectorized path for Rx and Tx
net/sxe2: add AVX2 vector data path for Rx and Tx
net/sxe2: add link update callback
net/sxe2: support L2 filtering and MAC config
drivers: support RSS feature
net/sxe2: support TM hierarchy and shaping
net/sxe2: support IPsec inline protocol offload
net/sxe2: support statistics and multi-process
drivers: interrupt handling
net/sxe2: add NEON vec Rx/Tx burst functions
drivers: add support for VF representors
net/sxe2: add support for custom UDP tunnel ports
net/sxe2: support firmware version reading
net/sxe2: implement get monitor address
common/sxe2: add shared SFP module definitions
net/sxe2: support SFP module info and EEPROM access
net/sxe2: add mbuf validation in Tx debug mode
common/sxe2: add callback for memory event handling
net/sxe2: add private devargs parsing
net/sxe2: implement private dump info
net/sxe2: update sxe2 feature matrix docs
doc/guides/nics/features/sxe2.ini | 56 +
doc/guides/nics/sxe2.rst | 186 ++
drivers/common/sxe2/sxe2_common.c | 156 ++
drivers/common/sxe2/sxe2_common.h | 4 +
drivers/common/sxe2/sxe2_flow_public.h | 633 ++++++
drivers/common/sxe2/sxe2_ioctl_chnl.c | 178 +-
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 18 +
drivers/common/sxe2/sxe2_msg.h | 118 ++
drivers/net/sxe2/meson.build | 51 +
drivers/net/sxe2/sxe2_cmd_chnl.c | 1587 ++++++++++++++-
drivers/net/sxe2/sxe2_cmd_chnl.h | 139 ++
drivers/net/sxe2/sxe2_drv_cmd.h | 523 ++++-
drivers/net/sxe2/sxe2_dump.c | 287 +++
drivers/net/sxe2/sxe2_dump.h | 12 +
drivers/net/sxe2/sxe2_ethdev.c | 1496 +++++++++++++-
drivers/net/sxe2/sxe2_ethdev.h | 111 +-
drivers/net/sxe2/sxe2_ethdev_repr.c | 609 ++++++
drivers/net/sxe2/sxe2_ethdev_repr.h | 32 +
drivers/net/sxe2/sxe2_filter.c | 895 +++++++++
drivers/net/sxe2/sxe2_filter.h | 100 +
drivers/net/sxe2/sxe2_flow.c | 1391 +++++++++++++
drivers/net/sxe2/sxe2_flow.h | 30 +
drivers/net/sxe2/sxe2_flow_define.h | 144 ++
drivers/net/sxe2/sxe2_flow_parse_action.c | 1182 +++++++++++
drivers/net/sxe2/sxe2_flow_parse_action.h | 23 +
drivers/net/sxe2/sxe2_flow_parse_engine.c | 106 +
drivers/net/sxe2/sxe2_flow_parse_engine.h | 13 +
drivers/net/sxe2/sxe2_flow_parse_pattern.c | 1935 ++++++++++++++++++
drivers/net/sxe2/sxe2_flow_parse_pattern.h | 46 +
drivers/net/sxe2/sxe2_ipsec.c | 1565 +++++++++++++++
drivers/net/sxe2/sxe2_ipsec.h | 254 +++
drivers/net/sxe2/sxe2_irq.c | 1026 ++++++++++
drivers/net/sxe2/sxe2_irq.h | 25 +
drivers/net/sxe2/sxe2_mac.c | 530 +++++
drivers/net/sxe2/sxe2_mac.h | 84 +
drivers/net/sxe2/sxe2_mp.c | 414 ++++
drivers/net/sxe2/sxe2_mp.h | 67 +
drivers/net/sxe2/sxe2_queue.c | 17 +-
drivers/net/sxe2/sxe2_queue.h | 15 +-
drivers/net/sxe2/sxe2_rss.c | 584 ++++++
drivers/net/sxe2/sxe2_rss.h | 81 +
drivers/net/sxe2/sxe2_rx.c | 93 +-
drivers/net/sxe2/sxe2_rx.h | 2 +
drivers/net/sxe2/sxe2_security.c | 335 ++++
drivers/net/sxe2/sxe2_security.h | 77 +
drivers/net/sxe2/sxe2_stats.c | 586 ++++++
drivers/net/sxe2/sxe2_stats.h | 39 +
drivers/net/sxe2/sxe2_switchdev.c | 332 +++
drivers/net/sxe2/sxe2_switchdev.h | 33 +
drivers/net/sxe2/sxe2_tm.c | 1151 +++++++++++
drivers/net/sxe2/sxe2_tm.h | 76 +
drivers/net/sxe2/sxe2_tx.c | 7 +
drivers/net/sxe2/sxe2_txrx.c | 2106 +++++++++++++++++++-
drivers/net/sxe2/sxe2_txrx.h | 8 +
drivers/net/sxe2/sxe2_txrx_poll.c | 284 ++-
drivers/net/sxe2/sxe2_txrx_vec.c | 46 +-
drivers/net/sxe2/sxe2_txrx_vec.h | 38 +-
drivers/net/sxe2/sxe2_txrx_vec_avx2.c | 747 +++++++
drivers/net/sxe2/sxe2_txrx_vec_avx512.c | 867 ++++++++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 54 +-
drivers/net/sxe2/sxe2_txrx_vec_neon.c | 689 +++++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 38 +-
drivers/net/sxe2/sxe2_vsi.c | 146 ++
drivers/net/sxe2/sxe2_vsi.h | 12 +-
drivers/net/sxe2/sxe2vf_regs.h | 85 +
65 files changed, 24303 insertions(+), 271 deletions(-)
create mode 100644 drivers/common/sxe2/sxe2_flow_public.h
create mode 100644 drivers/common/sxe2/sxe2_msg.h
create mode 100644 drivers/net/sxe2/sxe2_dump.c
create mode 100644 drivers/net/sxe2/sxe2_dump.h
create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.c
create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.h
create mode 100644 drivers/net/sxe2/sxe2_filter.c
create mode 100644 drivers/net/sxe2/sxe2_filter.h
create mode 100644 drivers/net/sxe2/sxe2_flow.c
create mode 100644 drivers/net/sxe2/sxe2_flow.h
create mode 100644 drivers/net/sxe2/sxe2_flow_define.h
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.c
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.h
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.c
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.h
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.c
create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.h
create mode 100644 drivers/net/sxe2/sxe2_ipsec.c
create mode 100644 drivers/net/sxe2/sxe2_ipsec.h
create mode 100644 drivers/net/sxe2/sxe2_irq.c
create mode 100644 drivers/net/sxe2/sxe2_mac.c
create mode 100644 drivers/net/sxe2/sxe2_mac.h
create mode 100644 drivers/net/sxe2/sxe2_mp.c
create mode 100644 drivers/net/sxe2/sxe2_mp.h
create mode 100644 drivers/net/sxe2/sxe2_rss.c
create mode 100644 drivers/net/sxe2/sxe2_rss.h
create mode 100644 drivers/net/sxe2/sxe2_security.c
create mode 100644 drivers/net/sxe2/sxe2_security.h
create mode 100644 drivers/net/sxe2/sxe2_stats.c
create mode 100644 drivers/net/sxe2/sxe2_stats.h
create mode 100644 drivers/net/sxe2/sxe2_switchdev.c
create mode 100644 drivers/net/sxe2/sxe2_switchdev.h
create mode 100644 drivers/net/sxe2/sxe2_tm.c
create mode 100644 drivers/net/sxe2/sxe2_tm.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx2.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx512.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_neon.c
create mode 100644 drivers/net/sxe2/sxe2vf_regs.h
--
2.52.0
^ permalink raw reply
* Re: Re: [PATCH v9 19/23] net/sxe2: add mbuf validation in Tx debug mode
From: liujie5 @ 2026-06-27 4:00 UTC (permalink / raw)
To: stephen; +Cc: dev
In-Reply-To: <20260626112114.1eddd05e@phoenix.local>
[-- Attachment #1: Type: text/plain, Size: 4963 bytes --]
Hi Stephen,
Introduce the 'sxe2_txrx_check_mbuf' helper function to validate
outgoing mbuf tunnel type flags when RTE_ETHDEV_DEBUG_TX is enabled.
The function checks that the RTE_MBUF_F_TX_TUNNEL_x flag in mbuf
ol_flags matches the actual tunnel protocol detected in the packet
(GTP, VXLAN, VXLAN-GPE, Geneve, GRE, or IPIP).
The validation uses only UDP port matching and L4 protocol detection;
it does NOT re-derive header lengths or checksum fields already
verified by rte_validate_tx_offload() and rte_net_intel_cksum_prepare().
All code is wrapped inside RTE_ETHDEV_DEBUG_TX conditional compilation
to ensure zero overhead in production builds.
liujie5@linkdatatechnology.com
From: Stephen Hemminger
Date: 2026-06-27 02:21
To: liujie5
CC: dev
Subject: Re: [PATCH v9 19/23] net/sxe2: add mbuf validation in Tx debug mode
On Fri, 26 Jun 2026 14:47:51 +0800
liujie5@linkdatatechnology.com wrote:
> From: Jie Liu <liujie5@linkdatatechnology.com>
>
> Introduce the `sxe2_txrx_check_mbuf` helper function to validate outgoing
> mbufs when `RTE_ETHDEV_DEBUG_TX` is enabled. This helps developers catch
> malformed mbufs (e.g., invalid segment lengths, bad offload flags, or
> unaligned buffers) before passing them to the hardware rings, avoiding
> potential hardware hangs or silent packet drops.
>
> The validation is fully wrapped inside `RTE_ETHDEV_DEBUG_TX` conditional
> compilation blocks to ensure zero performance overhead in standard
> production builds.
>
> Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
> ---
I don't think I was clear enough before, the verbose description is:
Current state (v9):
The new sxe2_txrx_check_mbuf() helper added in 19/23 is called only
from sxe2_tx_pkts_prepare(), which is the driver's tx_prepare
callback. It is not called from any tx_burst path (sxe2_tx_pkts, the
vec paths, the poll path).
The commit message claims the validation is "fully wrapped inside
RTE_ETHDEV_DEBUG_TX conditional compilation blocks to ensure zero
performance overhead in standard production builds." This is not what
the code does. There is no #ifdef RTE_ETHDEV_DEBUG_TX anywhere in the
new file or around the new call site. In fact the patch *removes* an
existing #ifdef RTE_ETHDEV_DEBUG_TX that previously gated
rte_validate_tx_offload() in the same loop, making that always-on too.
The new function is marked __rte_unused both in the header
declaration and in the .c definition, even though it has exactly one
caller. That attribute was likely a leftover from an earlier draft
where the function was only conditionally referenced.
The validation itself (595 lines) re-derives outer/inner L2/L3/L4
lengths from the packet bytes and compares them against what the mbuf
header declares - i.e., it's verifying that the application
correctly set mbuf->l2_len, l3_len, outer_l2_len, etc. before
submitting. Much of this overlaps what rte_validate_tx_offload() and
rte_net_intel_cksum_prepare() already check, both of which are called
in the same tx_prepare loop.
Desired state:
tx_burst should not contain validation checks for application errors.
The fast path runs once per packet at line rate, and any branch that
exists to catch a caller bug is paid on every well-behaved packet
forever. For driver assumptions that an application must satisfy,
the three sensible options are:
1. RTE_ASSERT() - compiles out in release builds, surfaces the
assumption to static analyzers, and fires for users running
debug builds.
2. unlikely() with drop-and-counter - if the check is cheap and
the assumption can be violated in production. The bad packet
is dropped, a queue stat is incremented, but the burst loop
continues.
3. Document the assumption in the API contract and don't check it
at runtime.
tx_prepare is where validation belongs, and an unconditional check
there is acceptable - tx_prepare is opt-in, and an application that
calls it has already agreed to pay for inspection. The 595-line
function as written is mostly fine in that context. What's wrong is:
(a) The commit message says one thing and the code does another.
The author should either add the #ifdef RTE_ETHDEV_DEBUG_TX
they claimed to add, or rewrite the commit message to describe
what the patch actually does.
(b) Driver-specific tx_prepare validation should cover sxe2-
specific hardware constraints (descriptor count limits, buffer
alignment, segment size limits) - not redo the generic offload-
flag and length-consistency checks that rte_validate_tx_offload()
and rte_net_intel_cksum_prepare() perform. As currently written,
every packet going through tx_prepare gets its offload flags
validated roughly twice.
(c) __rte_unused on the function should be dropped.
Redo this with one of the above and resubmit
[-- Attachment #2: Type: text/html, Size: 7959 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/3] dma/ae4dma: add data path operations
From: fengchengwen @ 2026-06-27 0:23 UTC (permalink / raw)
To: Raghavendra Ningoji, dev
Cc: david.marchand, bruce.richardson, Selwin.Sebastian,
bhagyada.modali, rjarry, thomas
In-Reply-To: <20260625184728.1678328-4-raghavendra.ningoji@amd.com>
On 6/26/2026 2:47 AM, Raghavendra Ningoji wrote:
> Implement the dmadev fast path for the AMD AE4DMA PMD.
>
> This commit adds:
> - copy enqueue (rte_dma_copy): write an AE4DMA descriptor for a
> memory-to-memory transfer; on RTE_DMA_OP_FLAG_SUBMIT the doorbell
> is rung immediately.
> - submit (rte_dma_submit): advance the per-queue write_idx
> register to expose pending descriptors to the hardware.
> - completion (rte_dma_completed / rte_dma_completed_status):
> completion is detected via the hardware's per-queue read_idx
> register, which the engine advances as it processes descriptors.
> The descriptor status / err_code bytes are read only to classify
> each drained slot as success or failure, and HW error codes are
> translated to the dmadev RTE_DMA_STATUS_* enumeration.
> - burst capacity (rte_dma_burst_capacity): report the number of
> free descriptor slots, taking into account the one slot reserved
> to distinguish full from empty on the power-of-two ring.
I don't think it's necessary to write in such detail because the ops
implemented are defined by the framework. If needed, you can supplement
by explaining what special features this driver has.
>
> The fast path entry points are wired through fp_obj in
> ae4dma_dmadev_create(). The fill capability is not advertised;
> fp_obj->fill is left zero-initialised.
>
> Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
> ---
> doc/guides/dmadevs/ae4dma.rst | 22 +++
> drivers/dma/ae4dma/ae4dma_dmadev.c | 287 +++++++++++++++++++++++++++++
> 2 files changed, 309 insertions(+)
>
> diff --git a/doc/guides/dmadevs/ae4dma.rst b/doc/guides/dmadevs/ae4dma.rst
> index a85c1d92ca..37a2096ccf 100644
> --- a/doc/guides/dmadevs/ae4dma.rst
> +++ b/doc/guides/dmadevs/ae4dma.rst
> @@ -51,3 +51,25 @@ On probe the PMD performs the following steps for each PCI function:
> IOVA-contiguous memory, programs the queue base address and ring
> depth into the per-queue registers, and enables the queue.
> * Interrupts are masked; completion is polled by the application.
> +
> +Usage
> +-----
> +
> +Once a dmadev has been started, copies are submitted with
> +``rte_dma_copy()`` and completions are reaped with ``rte_dma_completed()``
> +or ``rte_dma_completed_status()``. See the
> +:ref:`Enqueue / Dequeue API <dmadev_enqueue_dequeue>` section of the
> +dmadev library documentation for details.
> +
> +Limitations
> +-----------
> +
> +* Only memory-to-memory copies are supported. Fill, scatter-gather and
> + any other operation types are not advertised in
> + ``rte_dma_info::dev_capa``.
> +* The maximum number of descriptors per virtual channel is fixed by
> + hardware at 32. The PMD rounds the requested ring size up to a
> + power of two and clamps it to 32.
> +* Only a single virtual channel per dmadev is supported; use the 16
> + per-PCI-function dmadevs to obtain channel-level parallelism.
> +* Interrupt-driven completion is not supported.
> diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
> index 607f288623..da3ec42233 100644
> --- a/drivers/dma/ae4dma/ae4dma_dmadev.c
> +++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
> @@ -158,6 +158,72 @@ ae4dma_dev_close(struct rte_dma_dev *dev)
> return 0;
> }
>
> +/* trigger h/w to process enqued desc:doorbell - by next_write */
> +static inline void
> +__submit(struct ae4dma_dmadev *ae4dma)
> +{
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t write_idx = cmd_q->next_write;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> +
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->write_idx, write_idx);
> + if (nb != 0)
> + cmd_q->stats.submitted += (uint16_t)((cmd_q->next_write - cmd_q->last_write +
> + nb) % nb);
> + cmd_q->last_write = cmd_q->next_write;
> +}
> +
> +static int
> +ae4dma_submit(void *dev_private, uint16_t vchan __rte_unused)
> +{
> + struct ae4dma_dmadev *ae4dma = dev_private;
> +
> + __submit(ae4dma);
> + return 0;
> +}
> +
> +/* Write descriptor for enqueue (copy only). */
> +static inline int
> +__write_desc_copy(void *dev_private, rte_iova_t src, rte_iova_t dst,
> + uint32_t len, uint64_t flags)
> +{
> + struct ae4dma_dmadev *ae4dma = dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + struct ae4dma_desc *dma_desc;
> + uint16_t ret;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> + uint16_t write = cmd_q->next_write;
> +
> + if (nb == 0)
> + return -EINVAL;
> +
> + /* Reserve one slot to distinguish full from empty (power-of-two ring). */
> + if ((uint32_t)cmd_q->ring_buff_count >= (uint32_t)(nb - 1))
> + return -ENOSPC;
> +
> + dma_desc = &cmd_q->qbase_desc[write];
> + memset(dma_desc, 0, sizeof(*dma_desc));
> + dma_desc->length = len;
> + dma_desc->src_hi = upper_32_bits(src);
> + dma_desc->src_lo = lower_32_bits(src);
> + dma_desc->dst_hi = upper_32_bits(dst);
> + dma_desc->dst_lo = lower_32_bits(dst);
> + cmd_q->ring_buff_count++;
> + cmd_q->next_write = (uint16_t)((write + 1) % nb);
the next_write is [0, nb_desc-1], and it will as return value as copy,
but the dmadev framework expect as [0, 0xFFFF], I doubt your drvier was
not passed in any DMA test (e.g. dpdk-test, dpdk-dma-perf or examples/dma)
> + ret = write;
> + if (flags & RTE_DMA_OP_FLAG_SUBMIT)
> + __submit(ae4dma);
> + return ret;
> +}
> +
> +/* Enqueue a copy operation onto the ae4dma device. */
> +static int
> +ae4dma_enqueue_copy(void *dev_private, uint16_t vchan __rte_unused,
> + rte_iova_t src, rte_iova_t dst, uint32_t length, uint64_t flags)
> +{
> + return __write_desc_copy(dev_private, src, dst, length, flags);
> +}
> +
> static int
> ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
> {
> @@ -187,6 +253,220 @@ ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
> cmd_q->stats.errors);
> return 0;
> }
> +
> +/* Translates AE4DMA ChanERRs to DMA error codes. */
> +static inline enum rte_dma_status_code
> +__translate_status_ae4dma_to_dma(enum ae4dma_dma_err status)
> +{
> + AE4DMA_PMD_DEBUG("ae4dma desc status = %d", status);
> +
> + switch (status) {
> + case AE4DMA_DMA_ERR_NO_ERR:
> + return RTE_DMA_STATUS_SUCCESSFUL;
> + case AE4DMA_DMA_ERR_INV_LEN:
> + return RTE_DMA_STATUS_INVALID_LENGTH;
> + case AE4DMA_DMA_ERR_INV_SRC:
> + return RTE_DMA_STATUS_INVALID_SRC_ADDR;
> + case AE4DMA_DMA_ERR_INV_DST:
> + return RTE_DMA_STATUS_INVALID_DST_ADDR;
> + case AE4DMA_DMA_ERR_INV_ALIGN:
> + /* Name matches DPDK public enum spelling. */
> + return RTE_DMA_STATUS_DATA_POISION;
Suggest add RTE_DMA_STATUS_INVALID_ALIGN enum in rte_dmadev.h
> + case AE4DMA_DMA_ERR_INV_HEADER:
> + case AE4DMA_DMA_ERR_INV_STATUS:
> + return RTE_DMA_STATUS_ERROR_UNKNOWN;
> + default:
> + return RTE_DMA_STATUS_ERROR_UNKNOWN;
> + }
> +}
> +
> +/*
> + * Scan HW queue for completed descriptors (non-blocking).
> + *
> + * The AE4DMA engine signals completion by advancing the per-queue
> + * `read_idx` register; it does not (reliably) write a status value
> + * back into the descriptor. We therefore use the HW `read_idx`
> + * register as the source of truth and only inspect the descriptor's
> + * `dw1.err_code` byte to classify each completion as success or
> + * failure.
> + *
> + * @param cmd_q
> + * The AE4DMA command queue.
> + * @param max_ops
> + * Maximum descriptors to process this call.
> + * @param[out] failed_count
> + * Number of completed descriptors that did not report success.
> + * @return
> + * Number of descriptors completed (success + failure), <= max_ops.
> + */
> +static inline uint16_t
> +ae4dma_scan_hwq(struct ae4dma_cmd_queue *cmd_q, uint16_t max_ops,
> + uint16_t *failed_count)
> +{
> + volatile struct ae4dma_desc *hw_desc;
> + uint16_t events_count = 0, fails = 0;
> + uint16_t tail;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> + uint16_t mask;
> + uint16_t hw_read_idx;
> + uint16_t in_flight;
> + uint16_t scan_cap;
> +
> + if (nb == 0 || cmd_q->ring_buff_count == 0) {
> + *failed_count = 0;
> + return 0;
> + }
> + mask = nb - 1;
> +
> + hw_read_idx = (uint16_t)(AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx) & mask);
> + tail = cmd_q->next_read;
> +
> + /*
> + * Descriptors completed since our last visit live in the
> + * half-open ring range [tail, hw_read_idx). If HW hasn't
> + * moved we have nothing to do.
> + */
> + in_flight = (uint16_t)((hw_read_idx - tail) & mask);
> + if (in_flight == 0) {
> + *failed_count = 0;
> + return 0;
> + }
> +
> + scan_cap = max_ops;
> + if (scan_cap > AE4DMA_DESCRIPTORS_PER_CMDQ)
> + scan_cap = AE4DMA_DESCRIPTORS_PER_CMDQ;
> + if (scan_cap > in_flight)
> + scan_cap = in_flight;
> + if (scan_cap > cmd_q->ring_buff_count)
> + scan_cap = (uint16_t)cmd_q->ring_buff_count;
> +
> + while (events_count < scan_cap) {
> + uint8_t hw_status;
> + uint8_t hw_err;
> +
> + hw_desc = &cmd_q->qbase_desc[tail];
> + hw_status = hw_desc->dw1.status;
> + hw_err = hw_desc->dw1.err_code;
> +
> + /*
> + * read_idx advancing is the definitive completion
> + * signal. The per-descriptor status byte is informational
> + * and may not yet be written when we observe it:
> + *
> + * AE4DMA_DMA_DESC_ERROR (4)
> + * Hard failure - err_code names the precise cause.
> + * AE4DMA_DMA_DESC_COMPLETED (3) or 0
> + * Success.
> + * AE4DMA_DMA_DESC_VALIDATED (1) / _PROCESSED (2)
> + * Benign race: HW had not finished updating the
> + * status byte at the instant we read it. Since
> + * read_idx has moved past this slot, treat it as
> + * success unless err_code says otherwise.
> + *
> + * A non-zero err_code is treated as a failure regardless
> + * of the observed status value.
> + */
> + if (hw_status == AE4DMA_DMA_DESC_ERROR ||
> + hw_err != AE4DMA_DMA_ERR_NO_ERR) {
> + fails++;
> + AE4DMA_PMD_WARN("Desc failed: status=%u err=%u",
> + hw_status, hw_err);
> + }
> + cmd_q->status[events_count] = (enum ae4dma_dma_err)hw_err;
> + cmd_q->ring_buff_count--;
> + events_count++;
> + tail = (tail + 1) & mask;
> + }
> +
> + cmd_q->stats.completed += events_count;
> + cmd_q->stats.errors += fails;
> + cmd_q->next_read = tail;
> + *failed_count = fails;
> + return events_count;
> +}
> +
> +/* Returns successful operations count and sets error flag if any errors. */
> +static uint16_t
> +ae4dma_completed(void *dev_private, uint16_t vchan __rte_unused,
> + const uint16_t max_ops, uint16_t *last_idx, bool *has_error)
> +{
> + struct ae4dma_dmadev *ae4dma = dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t cpl_count, sl_count;
> + uint16_t err_count = 0;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> +
> + *has_error = false;
> +
> + cpl_count = ae4dma_scan_hwq(cmd_q, max_ops, &err_count);
> +
> + if (cpl_count > max_ops)
> + cpl_count = max_ops;
> +
> + if (cpl_count > 0 && last_idx != NULL)
> + *last_idx = (uint16_t)((cmd_q->next_read - 1 + nb) % nb);
the last_idx should be in range of [0, 0xFFFF]
> +
> + sl_count = cpl_count - err_count;
> + if (err_count)
> + *has_error = true;
> +
> + return sl_count;
> +}
> +
> +static uint16_t
> +ae4dma_completed_status(void *dev_private, uint16_t vchan __rte_unused,
> + uint16_t max_ops, uint16_t *last_idx,
> + enum rte_dma_status_code *status)
> +{
> + struct ae4dma_dmadev *ae4dma = dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t cpl_count;
> + uint16_t i;
> + uint16_t err_count = 0;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> +
> + cpl_count = ae4dma_scan_hwq(cmd_q, max_ops, &err_count);
> +
> + if (cpl_count > max_ops)
> + cpl_count = max_ops;
> +
> + if (cpl_count > 0 && last_idx != NULL)
> + *last_idx = (uint16_t)((cmd_q->next_read - 1 + nb) % nb);
> +
> + if (likely(err_count == 0)) {
> + for (i = 0; i < cpl_count; i++)
> + status[i] = RTE_DMA_STATUS_SUCCESSFUL;
> + } else {
> + for (i = 0; i < cpl_count; i++)
> + status[i] = __translate_status_ae4dma_to_dma(cmd_q->status[i]);
> + }
> +
> + return cpl_count;
> +}
> +
> +/* Get the remaining capacity of the ring. */
> +static uint16_t
> +ae4dma_burst_capacity(const void *dev_private, uint16_t vchan __rte_unused)
> +{
> + const struct ae4dma_dmadev *ae4dma = dev_private;
> + const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> + uint16_t mask;
> + uint16_t read_idx = cmd_q->next_read;
> + uint16_t write_idx = cmd_q->next_write;
> + uint16_t used;
> +
> + if (nb < 2 || !rte_is_power_of_2(nb))
> + return 0;
No need to check this
> +
> + mask = nb - 1;
> + used = (uint16_t)((write_idx - read_idx) & mask);
> + /* One slot reserved (same rule as enqueue). */
> + if (used >= nb - 1)
> + return 0;
> + return (uint16_t)(nb - 1 - used);
> +}
> +
> static int
> ae4dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
> struct rte_dma_stats *rte_stats, uint32_t size)
> @@ -342,6 +622,13 @@ ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
> dmadev->fp_obj->dev_private = dmadev->data->dev_private;
> dmadev->dev_ops = &ae4dma_dmadev_ops;
>
> + dmadev->fp_obj->burst_capacity = ae4dma_burst_capacity;
> + dmadev->fp_obj->completed = ae4dma_completed;
> + dmadev->fp_obj->completed_status = ae4dma_completed_status;
> + dmadev->fp_obj->copy = ae4dma_enqueue_copy;
> + dmadev->fp_obj->submit = ae4dma_submit;
> + /* fill capability not advertised: leave fp_obj->fill as zero-initialised. */
> +
> ae4dma = dmadev->data->dev_private;
>
> if (ae4dma_add_queue(ae4dma, dev, qn, name) != 0)
^ permalink raw reply
* Re: [PATCH v3 2/3] dma/ae4dma: add control path operations
From: fengchengwen @ 2026-06-27 0:09 UTC (permalink / raw)
To: Raghavendra Ningoji, dev
Cc: david.marchand, bruce.richardson, Selwin.Sebastian,
bhagyada.modali, rjarry, thomas
In-Reply-To: <20260625184728.1678328-3-raghavendra.ningoji@amd.com>
On 6/26/2026 2:47 AM, Raghavendra Ningoji wrote:
> Implement the dmadev control path for the AMD AE4DMA PMD.
>
> This commit adds:
> - dev_configure / vchan_setup: accept a single virtual channel per
> dmadev and clamp the requested ring size to the hardware maximum
> of 32 descriptors (rounded up to a power of two).
> - dev_start / dev_stop / dev_close: program the per-queue control
> register to enable/disable the hardware queue and release the
> descriptor ring memzone on close.
> - dev_info_get: advertise RTE_DMA_CAPA_MEM_TO_MEM and the fixed
> ring depth.
It seemed declare support 2~32 depth, not fixed
> - dev_dump: print the queue identifiers, ring layout and software
> completion counters.
> - stats_get / stats_reset: expose submitted / completed / errors
> counters maintained by the driver.
> - vchan_status: report IDLE / ACTIVE based on hardware read_idx vs
> write_idx, and HALTED_ERROR when the queue is not enabled.
>
> The dmadev framework is wired through dev_ops in ae4dma_dmadev_create().
>
> Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
> ---
> drivers/dma/ae4dma/ae4dma_dmadev.c | 211 +++++++++++++++++++++++++++++
> 1 file changed, 211 insertions(+)
>
> diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
> index 3d82f86906..607f288623 100644
> --- a/drivers/dma/ae4dma/ae4dma_dmadev.c
> +++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
> @@ -53,6 +53,203 @@ ae4dma_queue_dma_zone_reserve(const char *queue_name,
> socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
> }
>
> +static int
> +ae4dma_dev_configure(struct rte_dma_dev *dev __rte_unused,
> + const struct rte_dma_conf *dev_conf,
> + uint32_t conf_sz)
> +{
> + if (sizeof(struct rte_dma_conf) != conf_sz)
> + return -EINVAL;
This may break ABI compatible
> +
> + if (dev_conf->nb_vchans != 1)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +/* Setup a virtual channel for AE4DMA, only 1 vchan is supported per dmadev. */
> +static int
> +ae4dma_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
> + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t max_desc = qconf->nb_desc;
> +
> + if (sizeof(struct rte_dma_vchan_conf) != qconf_sz)
> + return -EINVAL;
This may break ABI compatible
> +
> + if (max_desc < 2)
> + return -EINVAL;
No need to do this because rte_dma_vchan_setup already do it.
> +
> + if (!rte_is_power_of_2(max_desc))
> + max_desc = rte_align32pow2(max_desc);
> +
> + if (max_desc > AE4DMA_DESCRIPTORS_PER_CMDQ) {
> + AE4DMA_PMD_DEBUG("DMA dev %u nb_desc clamped to %u",
> + dev->data->dev_id, AE4DMA_DESCRIPTORS_PER_CMDQ);
> + max_desc = AE4DMA_DESCRIPTORS_PER_CMDQ;
> + }
No need to do this because rte_dma_vchan_setup already do it.
> +
> + cmd_q->qcfg = *qconf;
> + cmd_q->qcfg.nb_desc = max_desc;
> +
> + /* Ensure all counters are reset, if reconfiguring/restarting device. */
> + memset(&cmd_q->stats, 0, sizeof(cmd_q->stats));
> + return 0;
> +}
> +
> +static int
> +ae4dma_dev_start(struct rte_dma_dev *dev)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint16_t nb = cmd_q->qcfg.nb_desc;
> +
> + if (nb == 0)
> + return -EBUSY;
> +
> + /* Program ring depth expected by hardware. */
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->max_idx, nb);
> + return 0;
> +}
> +
> +static int
> +ae4dma_dev_stop(struct rte_dma_dev *dev)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> +
> + if (cmd_q->hwq_regs != NULL)
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
> + AE4DMA_CMD_QUEUE_DISABLE);
> + return 0;
> +}
> +
> +static int
> +ae4dma_dev_info_get(const struct rte_dma_dev *dev __rte_unused,
> + struct rte_dma_info *info, uint32_t size)
> +{
> + if (size < sizeof(*info))
> + return -EINVAL;
> + info->dev_capa = RTE_DMA_CAPA_MEM_TO_MEM;
You need also decalre support RTE_DMA_CAP_OPS_COPY, please use dpdk-test dmadev_autotest
to test it.
The dpdk-dma-perf could also test dmadev.
> + info->max_vchans = 1;
> + info->min_desc = 2;
> + info->max_desc = AE4DMA_DESCRIPTORS_PER_CMDQ;
> + info->nb_vchans = 1;
> + return 0;
> +}
> +
> +static int
> +ae4dma_dev_close(struct rte_dma_dev *dev)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> +
> + if (cmd_q->hwq_regs != NULL)
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
> + AE4DMA_CMD_QUEUE_DISABLE);
> +
> + rte_memzone_free(cmd_q->mz);
> + cmd_q->mz = NULL;
> + cmd_q->qbase_desc = NULL;
> + cmd_q->qbase_addr = NULL;
> + cmd_q->qbase_phys_addr = 0;
> + return 0;
> +}
> +
> +static int
> +ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q;
> + void *ae4dma_mmio_base_addr = (uint8_t *)ae4dma->io_regs;
> +
> + cmd_q = &ae4dma->cmd_q;
> + fprintf(f, "cmd_q->id = %" PRIx64 "\n", cmd_q->id);
> + fprintf(f, "cmd_q->qidx = %" PRIx64 "\n", cmd_q->qidx);
> + fprintf(f, "cmd_q->qsize = %" PRIx64 "\n", cmd_q->qsize);
> + fprintf(f, "mmio_base_addr = %p\n", ae4dma_mmio_base_addr);
> + fprintf(f, "queues per ae4dma engine = %d\n", AE4DMA_READ_REG_OFFSET(
> + ae4dma_mmio_base_addr, AE4DMA_COMMON_CONFIG_OFFSET));
> + fprintf(f, "== Private Data ==\n");
> + fprintf(f, " Config: { ring_size: %u }\n", cmd_q->qcfg.nb_desc);
> + fprintf(f, " Ring virt: %p\tphys: %#" PRIx64 "\n",
> + (void *)cmd_q->qbase_desc,
> + (uint64_t)cmd_q->qbase_phys_addr);
> + fprintf(f, " Next write: %u\n", cmd_q->next_write);
> + fprintf(f, " Next read: %u\n", cmd_q->next_read);
> + fprintf(f, " current queue depth: %u\n", cmd_q->ring_buff_count);
> + fprintf(f, " }\n");
> + fprintf(f, " Key Stats { submitted: %" PRIu64 ", comp: %" PRIu64 ", failed: %" PRIu64 " }\n",
> + cmd_q->stats.submitted,
> + cmd_q->stats.completed,
> + cmd_q->stats.errors);
> + return 0;
> +}
> +static int
> +ae4dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
> + struct rte_dma_stats *rte_stats, uint32_t size)
> +{
> + const struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + const struct rte_dma_stats *stats = &cmd_q->stats;
> +
> + if (size < sizeof(*rte_stats))
> + return -EINVAL;
> + if (rte_stats == NULL)
> + return -EINVAL;
No need to do this check because rte_dma_stats_get already check it
Please make such check on other ops.
> +
> + *rte_stats = *stats;
> + return 0;
> +}
> +
> +static int
> +ae4dma_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused)
> +{
> + struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> +
> + memset(&cmd_q->stats, 0, sizeof(cmd_q->stats));
> + return 0;
> +}
> +
> +/*
> + * Report channel state to the dmadev framework.
> + *
> + * RTE_DMA_VCHAN_HALTED_ERROR - HW queue is disabled (never started, or
> + * stopped via dev_stop()).
> + * RTE_DMA_VCHAN_IDLE - HW has caught up: read_idx == write_idx,
> + * no descriptors in flight.
> + * RTE_DMA_VCHAN_ACTIVE - HW still has descriptors to process.
> + */
> +static int
> +ae4dma_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
> + enum rte_dma_vchan_status *status)
> +{
> + const struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
> + const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
> + uint32_t ctrl, hw_read, hw_write;
> +
> + if (cmd_q->hwq_regs == NULL) {
> + *status = RTE_DMA_VCHAN_HALTED_ERROR;
> + return 0;
> + }
> +
> + ctrl = AE4DMA_READ_REG(&cmd_q->hwq_regs->control_reg.control_raw);
> + if ((ctrl & AE4DMA_CMD_QUEUE_ENABLE) == 0) {
> + *status = RTE_DMA_VCHAN_HALTED_ERROR;
> + return 0;
> + }
> +
> + hw_read = AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx);
> + hw_write = AE4DMA_READ_REG(&cmd_q->hwq_regs->write_idx);
> +
> + *status = (hw_read == hw_write) ? RTE_DMA_VCHAN_IDLE
> + : RTE_DMA_VCHAN_ACTIVE;
> + return 0;
> +}
> +
> static int
> ae4dma_add_queue(struct ae4dma_dmadev *dev, struct rte_pci_device *pci,
> uint8_t qn, const char *pci_name)
> @@ -115,6 +312,19 @@ ae4dma_channel_dev_name(char *out, size_t outlen, const char *pci_name,
> static int
> ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
> {
> + static const struct rte_dma_dev_ops ae4dma_dmadev_ops = {
> + .dev_close = ae4dma_dev_close,
> + .dev_configure = ae4dma_dev_configure,
> + .dev_dump = ae4dma_dev_dump,
> + .dev_info_get = ae4dma_dev_info_get,
> + .dev_start = ae4dma_dev_start,
> + .dev_stop = ae4dma_dev_stop,
> + .stats_get = ae4dma_stats_get,
> + .stats_reset = ae4dma_stats_reset,
> + .vchan_status = ae4dma_vchan_status,
> + .vchan_setup = ae4dma_vchan_setup,
> + };
> +
> struct rte_dma_dev *dmadev;
> struct ae4dma_dmadev *ae4dma;
> char hwq_dev_name[RTE_DEV_NAME_MAX_LEN];
> @@ -130,6 +340,7 @@ ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
> }
> dmadev->device = &dev->device;
> dmadev->fp_obj->dev_private = dmadev->data->dev_private;
> + dmadev->dev_ops = &ae4dma_dmadev_ops;
>
> ae4dma = dmadev->data->dev_private;
>
^ permalink raw reply
* Re: [PATCH v3 1/3] dma/ae4dma: introduce AMD AE4DMA DMA PMD
From: fengchengwen @ 2026-06-27 0:01 UTC (permalink / raw)
To: Raghavendra Ningoji, dev
Cc: david.marchand, bruce.richardson, Selwin.Sebastian,
bhagyada.modali, rjarry, thomas
In-Reply-To: <20260625184728.1678328-2-raghavendra.ningoji@amd.com>
On 6/26/2026 2:47 AM, Raghavendra Ningoji wrote:
> Add the skeleton of a new dmadev poll-mode driver for the AMD AE4DMA
> hardware DMA engine, providing only PCI probe/remove and per-queue
> hardware initialisation. An AE4DMA engine exposes 16 hardware command
> queues, each with a 32-entry descriptor ring; the PMD maps each
> hardware channel to its own dmadev with a single virtual channel,
> so a PCI function appears as 16 dmadevs named "<pci-bdf>-ch0" ..
> "<pci-bdf>-ch15".
>
> This patch only registers the PCI driver, allocates the dmadev
> objects, reserves the per-queue descriptor rings and programs the
> hardware queue base addresses. Control and data path operations are
> added in subsequent patches.
>
> Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
> ---
> .mailmap | 1 +
> MAINTAINERS | 5 +
> doc/guides/dmadevs/ae4dma.rst | 53 ++++++
> doc/guides/dmadevs/index.rst | 1 +
> doc/guides/rel_notes/release_26_07.rst | 7 +
> drivers/dma/ae4dma/ae4dma_dmadev.c | 220 +++++++++++++++++++++++++
> drivers/dma/ae4dma/ae4dma_hw_defs.h | 154 +++++++++++++++++
> drivers/dma/ae4dma/ae4dma_internal.h | 97 +++++++++++
> drivers/dma/ae4dma/meson.build | 7 +
> drivers/dma/meson.build | 1 +
> usertools/dpdk-devbind.py | 5 +-
> 11 files changed, 550 insertions(+), 1 deletion(-)
> create mode 100644 doc/guides/dmadevs/ae4dma.rst
> create mode 100644 drivers/dma/ae4dma/ae4dma_dmadev.c
> create mode 100644 drivers/dma/ae4dma/ae4dma_hw_defs.h
> create mode 100644 drivers/dma/ae4dma/ae4dma_internal.h
> create mode 100644 drivers/dma/ae4dma/meson.build
>
> diff --git a/.mailmap b/.mailmap
> index 89ba6ffccc..71a62564fa 100644
> --- a/.mailmap
> +++ b/.mailmap
> @@ -1329,6 +1329,7 @@ Radu Bulie <radu-andrei.bulie@nxp.com>
> Radu Nicolau <radu.nicolau@intel.com>
> Rafael Ávila de Espíndola <espindola@scylladb.com>
> Rafal Kozik <rk@semihalf.com>
> +Raghavendra Ningoji <raghavendra.ningoji@amd.com>
> Ragothaman Jayaraman <rjayaraman@caviumnetworks.com>
> Rahul Bhansali <rbhansali@marvell.com>
> Rahul Gupta <rahul.gupta@broadcom.com>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9143d028bc..2e27af49f4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1361,6 +1361,11 @@ F: doc/guides/compressdevs/features/zsda.ini
> DMAdev Drivers
> --------------
>
> +AMD AE4DMA
> +M: Bhagyada Modali <bhagyada.modali@amd.com>
> +F: drivers/dma/ae4dma/
> +F: doc/guides/dmadevs/ae4dma.rst
> +
> Intel IDXD - EXPERIMENTAL
> M: Bruce Richardson <bruce.richardson@intel.com>
> M: Kevin Laatz <kevin.laatz@intel.com>
> diff --git a/doc/guides/dmadevs/ae4dma.rst b/doc/guides/dmadevs/ae4dma.rst
> new file mode 100644
> index 0000000000..a85c1d92ca
> --- /dev/null
> +++ b/doc/guides/dmadevs/ae4dma.rst
> @@ -0,0 +1,53 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> + Copyright(c) 2025 Advanced Micro Devices, Inc.
2025 -> 2026?
> +
> +.. include:: <isonum.txt>
> +
> +AMD AE4DMA DMA Device Driver
> +============================
> +
> +The ``ae4dma`` dmadev driver is a poll-mode driver (PMD) for the
> +AMD AE4DMA hardware DMA engine. The engine exposes 16 independent
> +hardware command queues, each with a ring of 32 descriptors. The PMD
> +maps each hardware command queue to a separate DPDK dmadev with a
> +single virtual channel, so a single PCI function appears as 16 dmadevs
> +named ``<pci-bdf>-ch0`` through ``<pci-bdf>-ch15``.
> +
> +The driver supports memory-to-memory copy operations only.
> +
> +Hardware Requirements
> +---------------------
> +
> +The ``dpdk-devbind.py`` script can be used to list AE4DMA devices on
> +the system::
> +
> + dpdk-devbind.py --status-dev dma
> +
> +AE4DMA devices appear with vendor ID ``0x1022`` and device ID
> +``0x149b``.
> +
> +Compilation
> +-----------
> +
> +The driver is built as part of the standard DPDK build on x86 platforms
> +using ``meson`` and ``ninja``; no extra configuration is required.
> +
> +Device Setup
> +------------
> +
> +The AE4DMA device must be bound to a DPDK-compatible kernel module such
> +as ``vfio-pci`` before it can be used::
> +
> + dpdk-devbind.py -b vfio-pci <pci-bdf>
> +
> +Initialization
> +~~~~~~~~~~~~~~
> +
> +On probe the PMD performs the following steps for each PCI function:
> +
> +* Reads BAR0 and programs the common configuration register with the
> + number of hardware queues to enable (16).
> +* For each hardware queue it allocates a 32-entry descriptor ring in
> + IOVA-contiguous memory, programs the queue base address and ring
> + depth into the per-queue registers, and enables the queue.
> +* Interrupts are masked; completion is polled by the application.
> diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
> index 56beb1733f..97399590f6 100644
> --- a/doc/guides/dmadevs/index.rst
> +++ b/doc/guides/dmadevs/index.rst
> @@ -11,6 +11,7 @@ an application through DMA API.
> :maxdepth: 1
> :numbered:
>
> + ae4dma
> cnxk
> dpaa
> dpaa2
> diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
> index f012d47a4b..9a78a7ef62 100644
> --- a/doc/guides/rel_notes/release_26_07.rst
> +++ b/doc/guides/rel_notes/release_26_07.rst
> @@ -63,6 +63,13 @@ New Features
> ``rte_eal_init`` and the application is responsible for probing each device,
> * ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
>
> +* **Added AMD AE4DMA DMA PMD.**
> +
> + Added a new ``dma/ae4dma`` driver for the AMD AE4DMA hardware DMA engine.
> + Each PCI function exposes 16 hardware command queues; the PMD registers one
> + dmadev per channel with a single virtual channel and supports
> + memory-to-memory copy operations.
> +
>
> Removed Items
> -------------
> diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
> new file mode 100644
> index 0000000000..3d82f86906
> --- /dev/null
> +++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
> @@ -0,0 +1,220 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
> + */
> +
> +#include <errno.h>
> +#include <inttypes.h>
> +#include <stdio.h>
> +#include <string.h>
> +
> +#include <rte_bus_pci.h>
> +#include <bus_pci_driver.h>
> +#include <rte_dmadev_pmd.h>
> +#include <rte_malloc.h>
> +
> +#include "ae4dma_internal.h"
> +
> +/*
> + * One dmadev per AE4DMA hardware channel; each dmadev has exactly one
> + * virtual channel. The HW's per-queue register block must be densely
> + * packed right after the engine-common config register at BAR0+0; the
> + * build-time check below catches an accidental layout change.
> + */
> +static_assert(sizeof(struct ae4dma_hwq_regs) == 32,
> + "ae4dma_hwq_regs stride changed; per-queue offset math will break");
> +
> +RTE_LOG_REGISTER_DEFAULT(ae4dma_pmd_logtype, INFO);
> +
> +#define AE4DMA_PMD_NAME dmadev_ae4dma
> +
> +static const struct rte_memzone *
> +ae4dma_queue_dma_zone_reserve(const char *queue_name,
> + uint32_t queue_size, int socket_id)
> +{
> + const struct rte_memzone *mz;
> +
> + mz = rte_memzone_lookup(queue_name);
> + if (mz != NULL) {
> + if (((size_t)queue_size <= mz->len) &&
> + ((socket_id == SOCKET_ID_ANY) ||
> + (socket_id == mz->socket_id))) {
> + AE4DMA_PMD_INFO("reuse memzone already "
> + "allocated for %s", queue_name);
> + return mz;
> + }
> + AE4DMA_PMD_ERR("Incompatible memzone already "
> + "allocated %s, size %u, socket %d. "
> + "Requested size %u, socket %u",
> + queue_name, (uint32_t)mz->len,
> + mz->socket_id, queue_size, socket_id);
> + return NULL;
> + }
> + return rte_memzone_reserve_aligned(queue_name, queue_size,
> + socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
No need to do such reuse, and this resource could setup in vchan_setup ops,
but your dmadev has max 32 descriptors and only 1 vchan per-dmadev, so I think
it's ok to setup in the probe.
> +}
> +
> +static int
> +ae4dma_add_queue(struct ae4dma_dmadev *dev, struct rte_pci_device *pci,
> + uint8_t qn, const char *pci_name)
> +{
> + uint32_t dma_addr_lo, dma_addr_hi;
> + struct ae4dma_cmd_queue *cmd_q;
> + const struct rte_memzone *q_mz;
> +
> + dev->io_regs = pci->mem_resource[AE4DMA_PCIE_BAR].addr;
> +
> + cmd_q = &dev->cmd_q;
> + cmd_q->id = qn;
> + cmd_q->qidx = 0;
> + cmd_q->qsize = AE4DMA_QUEUE_SIZE(AE4DMA_QUEUE_DESC_SIZE);
> + cmd_q->hwq_regs = (volatile struct ae4dma_hwq_regs *)dev->io_regs + (qn + 1);
> +
> + /*
> + * Memzone name must be globally unique. Embed PCI BDF so multiple
> + * PCI functions probed concurrently don't collide.
> + */
> + snprintf(cmd_q->memz_name, sizeof(cmd_q->memz_name),
> + "ae4dma_%s_q%u", pci_name, (unsigned int)qn);
> +
> + q_mz = ae4dma_queue_dma_zone_reserve(cmd_q->memz_name,
> + cmd_q->qsize, rte_socket_id());
> + if (q_mz == NULL) {
> + AE4DMA_PMD_ERR("memzone reserve failed for %s", cmd_q->memz_name);
> + return -ENOMEM;
> + }
> +
> + cmd_q->mz = q_mz;
> + cmd_q->qbase_addr = q_mz->addr;
> + cmd_q->qbase_desc = q_mz->addr;
> + cmd_q->qbase_phys_addr = q_mz->iova;
> +
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->max_idx, AE4DMA_DESCRIPTORS_PER_CMDQ);
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
> + AE4DMA_CMD_QUEUE_ENABLE);
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->intr_status_reg.intr_status_raw,
> + AE4DMA_DISABLE_INTR);
> + cmd_q->next_write = AE4DMA_READ_REG(&cmd_q->hwq_regs->write_idx);
> + cmd_q->next_read = AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx);
> + cmd_q->ring_buff_count = 0;
> +
> + dma_addr_lo = lower_32_bits(cmd_q->qbase_phys_addr);
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->qbase_lo, dma_addr_lo);
> + dma_addr_hi = upper_32_bits(cmd_q->qbase_phys_addr);
> + AE4DMA_WRITE_REG(&cmd_q->hwq_regs->qbase_hi, dma_addr_hi);
> +
> + return 0;
> +}
> +
> +static void
> +ae4dma_channel_dev_name(char *out, size_t outlen, const char *pci_name,
> + unsigned int ch)
> +{
> + snprintf(out, outlen, "%s-ch%u", pci_name, ch);
> +}
> +
> +static int
> +ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
> +{
> + struct rte_dma_dev *dmadev;
> + struct ae4dma_dmadev *ae4dma;
> + char hwq_dev_name[RTE_DEV_NAME_MAX_LEN];
Please define local variables in a descending order, with longer ones
placed at the front. It is recommended to modify the entire driver in
this way.
> +
> + memset(hwq_dev_name, 0, sizeof(hwq_dev_name));
why not char hwq_dev_name[RTE_DEV_NAME_MAX_LEN] = {0};
> + ae4dma_channel_dev_name(hwq_dev_name, sizeof(hwq_dev_name), name, qn);
> +
> + dmadev = rte_dma_pmd_allocate(hwq_dev_name, dev->device.numa_node,
> + sizeof(struct ae4dma_dmadev));
> + if (dmadev == NULL) {
> + AE4DMA_PMD_ERR("Unable to allocate dma device");
> + return -ENOMEM;
> + }
> + dmadev->device = &dev->device;
> + dmadev->fp_obj->dev_private = dmadev->data->dev_private;
> +
> + ae4dma = dmadev->data->dev_private;
> +
> + if (ae4dma_add_queue(ae4dma, dev, qn, name) != 0)
> + goto init_error;
> + return 0;
> +
> +init_error:
> + AE4DMA_PMD_ERR("failed");
why not add more info, e.g. Probe failed!
> + rte_dma_pmd_release(hwq_dev_name);
> + return -ENOMEM;
> +}
> +
> +static int
> +ae4dma_dmadev_probe(struct rte_pci_driver *drv __rte_unused,
> + struct rte_pci_device *dev)
> +{
> + char name[32];
> + char chname[RTE_DEV_NAME_MAX_LEN];
> + void *mmio_base;
> + uint32_t q_per_eng;
> + int ret = 0;
> + uint8_t i;
> +
> + rte_pci_device_name(&dev->addr, name, sizeof(name));
> + AE4DMA_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node);
> +
> + mmio_base = dev->mem_resource[AE4DMA_PCIE_BAR].addr;
> + if (mmio_base == NULL) {
> + AE4DMA_PMD_ERR("%s: BAR%d not mapped", name, AE4DMA_PCIE_BAR);
> + return -ENODEV;
> + }
> +
> + /* Program the per-engine HW queue count once. */
> + AE4DMA_WRITE_REG_OFFSET(mmio_base, AE4DMA_COMMON_CONFIG_OFFSET,
> + AE4DMA_MAX_HW_QUEUES);
> + q_per_eng = AE4DMA_READ_REG_OFFSET(mmio_base, AE4DMA_COMMON_CONFIG_OFFSET);
> + AE4DMA_PMD_INFO("%s: AE4DMA queues per engine = %u", name, q_per_eng);
> +
> + for (i = 0; i < AE4DMA_MAX_HW_QUEUES; i++) {
> + ret = ae4dma_dmadev_create(name, dev, i);
> + if (ret != 0) {
> + AE4DMA_PMD_ERR("%s create dmadev %u failed!", name, i);
> + while (i > 0) {
> + i--;
> + ae4dma_channel_dev_name(chname, sizeof(chname), name, i);
> + rte_dma_pmd_release(chname);
> + }
> + break;
> + }
> + }
> + return ret;
> +}
> +
> +static int
> +ae4dma_dmadev_remove(struct rte_pci_device *dev)
> +{
> + char name[32];
> + char chname[RTE_DEV_NAME_MAX_LEN];
> + unsigned int i;
> +
> + rte_pci_device_name(&dev->addr, name, sizeof(name));
> +
> + AE4DMA_PMD_INFO("Closing %s on NUMA node %d",
> + name, dev->device.numa_node);
> +
> + for (i = 0; i < AE4DMA_MAX_HW_QUEUES; i++) {
> + ae4dma_channel_dev_name(chname, sizeof(chname), name, i);
> + rte_dma_pmd_release(chname);
> + }
> + return 0;
> +}
> +
> +static const struct rte_pci_id pci_id_ae4dma_map[] = {
> + { RTE_PCI_DEVICE(AMD_VENDOR_ID, AE4DMA_DEVICE_ID) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +static struct rte_pci_driver ae4dma_pmd_drv = {
> + .id_table = pci_id_ae4dma_map,
> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
> + .probe = ae4dma_dmadev_probe,
> + .remove = ae4dma_dmadev_remove,
> +};
> +
> +RTE_PMD_REGISTER_PCI(AE4DMA_PMD_NAME, ae4dma_pmd_drv);
> +RTE_PMD_REGISTER_PCI_TABLE(AE4DMA_PMD_NAME, pci_id_ae4dma_map);
> +RTE_PMD_REGISTER_KMOD_DEP(AE4DMA_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci");
> diff --git a/drivers/dma/ae4dma/ae4dma_hw_defs.h b/drivers/dma/ae4dma/ae4dma_hw_defs.h
> new file mode 100644
> index 0000000000..e7798be09b
> --- /dev/null
> +++ b/drivers/dma/ae4dma/ae4dma_hw_defs.h
> @@ -0,0 +1,154 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
> + */
> +
> +#ifndef __AE4DMA_HW_DEFS_H__
> +#define __AE4DMA_HW_DEFS_H__
> +
> +#include <stdint.h>
> +
> +#include <rte_bus_pci.h>
> +#include <rte_byteorder.h>
> +#include <rte_io.h>
> +#include <rte_pci.h>
> +#include <rte_memzone.h>
Some of the include file are not need for this head-file.
> +
> +#define AE4DMA_BIT(nr) (1UL << (nr))
> +
> +/* ae4dma device details */
> +#define AMD_VENDOR_ID 0x1022
> +#define AE4DMA_DEVICE_ID 0x149b
> +#define AE4DMA_PCIE_BAR 0
> +
> +/*
> + * An AE4DMA engine has 16 DMA queues. Each queue supports 32 descriptors.
> + */
> +#define AE4DMA_MAX_HW_QUEUES 16
> +#define AE4DMA_QUEUE_START_INDEX 0
> +#define AE4DMA_CMD_QUEUE_ENABLE 0x1
> +#define AE4DMA_CMD_QUEUE_DISABLE 0x0
> +
> +/* Common to all queues */
> +#define AE4DMA_COMMON_CONFIG_OFFSET 0x00
> +
> +#define AE4DMA_DISABLE_INTR 0x01
> +
> +/* Descriptor status */
> +enum ae4dma_dma_status {
> + AE4DMA_DMA_DESC_SUBMITTED = 0,
> + AE4DMA_DMA_DESC_VALIDATED = 1,
> + AE4DMA_DMA_DESC_PROCESSED = 2,
> + AE4DMA_DMA_DESC_COMPLETED = 3,
> + AE4DMA_DMA_DESC_ERROR = 4,
> +};
> +
> +/* Descriptor error-code */
> +enum ae4dma_dma_err {
> + AE4DMA_DMA_ERR_NO_ERR = 0,
> + AE4DMA_DMA_ERR_INV_HEADER = 1,
> + AE4DMA_DMA_ERR_INV_STATUS = 2,
> + AE4DMA_DMA_ERR_INV_LEN = 3,
> + AE4DMA_DMA_ERR_INV_SRC = 4,
> + AE4DMA_DMA_ERR_INV_DST = 5,
> + AE4DMA_DMA_ERR_INV_ALIGN = 6,
> + AE4DMA_DMA_ERR_UNKNOWN = 7,
> +};
> +
> +/* HW Queue status */
> +enum ae4dma_hwqueue_status {
> + AE4DMA_HWQUEUE_EMPTY = 0,
> + AE4DMA_HWQUEUE_FULL = 1,
> + AE4DMA_HWQUEUE_NOT_EMPTY = 4,
> +};
> +/*
> + * descriptor for AE4DMA commands
> + * 8 32-bit words:
> + * word 0: source memory type; destination memory type ; control bits
> + * word 1: desc_id; error code; status
> + * word 2: length
> + * word 3: reserved
> + * word 4: upper 32 bits of source pointer
> + * word 5: low 32 bits of source pointer
> + * word 6: upper 32 bits of destination pointer
> + * word 7: low 32 bits of destination pointer
> + */
> +
> +/* AE4DMA Descriptor - DWORD0 - Controls bits: Reserved for future use */
> +#define AE4DMA_DWORD0_STOP_ON_COMPLETION AE4DMA_BIT(0)
> +#define AE4DMA_DWORD0_INTERRUPT_ON_COMPLETION AE4DMA_BIT(1)
> +#define AE4DMA_DWORD0_START_OF_MESSAGE AE4DMA_BIT(3)
> +#define AE4DMA_DWORD0_END_OF_MESSAGE AE4DMA_BIT(4)
> +#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE RTE_GENMASK64(5, 4)
> +#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE RTE_GENMASK64(7, 6)
> +
> +#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE_MEMORY (0x0)
> +#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE_IOMEMORY (1<<4)
> +#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE_MEMORY (0x0)
> +#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE_IOMEMORY (1<<6)
> +
> +struct ae4dma_desc_dword0 {
> + uint8_t byte0;
> + uint8_t byte1;
> + uint16_t timestamp;
> +};
> +
> +struct ae4dma_desc_dword1 {
> + uint8_t status;
> + uint8_t err_code;
> + uint16_t desc_id;
> +};
> +
> +struct ae4dma_desc {
> + struct ae4dma_desc_dword0 dw0;
> + struct ae4dma_desc_dword1 dw1;
> + uint32_t length;
> + uint32_t reserved;
> + uint32_t src_lo;
> + uint32_t src_hi;
> + uint32_t dst_lo;
> + uint32_t dst_hi;
> +};
> +
> +/*
> + * Registers for each queue :4 bytes length
> + * Effective address : offset + reg
> + */
> +struct ae4dma_hwq_regs {
> + union {
> + uint32_t control_raw;
> + struct {
> + uint32_t queue_enable: 1;
> + uint32_t reserved_internal: 31;
> + } control;
> + } control_reg;
> +
> + union {
> + uint32_t status_raw;
> + struct {
> + uint32_t reserved0: 1;
> + /* 0–empty, 1–full, 2–stopped, 3–error , 4–Not Empty */
> + uint32_t queue_status: 2;
> + uint32_t reserved1: 21;
> + uint32_t interrupt_type: 4;
> + uint32_t reserved2: 4;
> + } status;
> + } status_reg;
> +
> + uint32_t max_idx;
> + uint32_t read_idx;
> + uint32_t write_idx;
> +
> + union {
> + uint32_t intr_status_raw;
> + struct {
> + uint32_t intr_status: 1;
> + uint32_t reserved: 31;
> + } intr_status;
> + } intr_status_reg;
> +
> + uint32_t qbase_lo;
> + uint32_t qbase_hi;
> +
> +};
> +
> +#endif /* AE4DMA_HW_DEFS_H */
> diff --git a/drivers/dma/ae4dma/ae4dma_internal.h b/drivers/dma/ae4dma/ae4dma_internal.h
> new file mode 100644
> index 0000000000..7f149c97b5
> --- /dev/null
> +++ b/drivers/dma/ae4dma/ae4dma_internal.h
> @@ -0,0 +1,97 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
> + */
> +
> +#ifndef _AE4DMA_INTERNAL_H_
> +#define _AE4DMA_INTERNAL_H_
> +
> +#include <stdint.h>
> +
> +#include "ae4dma_hw_defs.h"
> +
> +/* Return bits 32-63 of a 64-bit number. */
> +#define upper_32_bits(n) ((uint32_t)(((n) >> 16) >> 16))
> +
> +/* Return bits 0-31 of a 64-bit number. */
> +#define lower_32_bits(n) ((uint32_t)((n) & 0xffffffff))
> +
> +/* Hardware ring depth (slots per queue); must be power of two. */
> +#define AE4DMA_DESCRIPTORS_PER_CMDQ 32
> +#define AE4DMA_QUEUE_DESC_SIZE sizeof(struct ae4dma_desc)
> +#define AE4DMA_QUEUE_SIZE(n) (AE4DMA_DESCRIPTORS_PER_CMDQ * (n))
> +
two blank lines
> +
> +/* AE4DMA registers Write/Read */
> +static inline void ae4dma_pci_reg_write(void *base, int offset,
> + uint32_t value)
> +{
> + volatile void *reg_addr = ((uint8_t *)base + offset);
> +
> + rte_write32((rte_cpu_to_le_32(value)), reg_addr);
> +}
> +
> +static inline uint32_t ae4dma_pci_reg_read(void *base, int offset)
> +{
> + volatile void *reg_addr = ((uint8_t *)base + offset);
> +
> + return rte_le_to_cpu_32(rte_read32(reg_addr));
> +}
> +
> +#define AE4DMA_READ_REG_OFFSET(hw_addr, reg_offset) \
> + ae4dma_pci_reg_read(hw_addr, reg_offset)
> +
> +#define AE4DMA_WRITE_REG_OFFSET(hw_addr, reg_offset, value) \
> + ae4dma_pci_reg_write(hw_addr, reg_offset, value)
> +
> +
two blank lines
> +#define AE4DMA_READ_REG(hw_addr) \
> + ae4dma_pci_reg_read((void *)(uintptr_t)(hw_addr), 0)
> +
> +#define AE4DMA_WRITE_REG(hw_addr, value) \
> + ae4dma_pci_reg_write((void *)(uintptr_t)(hw_addr), 0, value)
> +
> +/* A structure describing an AE4DMA command queue. */
> +struct __rte_cache_aligned ae4dma_cmd_queue {
> + char memz_name[RTE_MEMZONE_NAMESIZE];
> + const struct rte_memzone *mz;
> + volatile struct ae4dma_hwq_regs *hwq_regs;
> +
> + struct rte_dma_vchan_conf qcfg;
> + struct rte_dma_stats stats;
> + /* Queue address */
> + struct ae4dma_desc *qbase_desc;
> + void *qbase_addr;
> + rte_iova_t qbase_phys_addr;
> + enum ae4dma_dma_err status[AE4DMA_DESCRIPTORS_PER_CMDQ];
> + /* Queue identifier */
> + uint64_t id; /* queue id */
> + uint64_t qidx; /* queue index */
> + uint64_t qsize; /* queue size */
> + uint32_t ring_buff_count;
> + uint16_t next_read;
> + uint16_t next_write;
> + uint16_t last_write; /* Used to compute submitted count. */
> +};
> +
> +/*
> + * One dmadev per AE4DMA hardware channel: probe creates AE4DMA_MAX_HW_QUEUES
> + * dmadevs per PCI function, each owning a single HW command queue.
> + */
> +struct ae4dma_dmadev {
> + void *io_regs;
> + struct ae4dma_cmd_queue cmd_q; /* single HW queue owned by this dmadev */
> +};
> +
> +
two blank line
> +extern int ae4dma_pmd_logtype;
> +#define RTE_LOGTYPE_AE4DMA_PMD ae4dma_pmd_logtype
> +
> +#define AE4DMA_PMD_LOG(level, ...) \
> + RTE_LOG_LINE_PREFIX(level, AE4DMA_PMD, "%s(): ", __func__, __VA_ARGS__)
> +
> +#define AE4DMA_PMD_DEBUG(...) AE4DMA_PMD_LOG(DEBUG, __VA_ARGS__)
> +#define AE4DMA_PMD_INFO(...) AE4DMA_PMD_LOG(INFO, __VA_ARGS__)
> +#define AE4DMA_PMD_ERR(...) AE4DMA_PMD_LOG(ERR, __VA_ARGS__)
> +#define AE4DMA_PMD_WARN(...) AE4DMA_PMD_LOG(WARNING, __VA_ARGS__)
> +
> +#endif /* _AE4DMA_INTERNAL_H_ */
> diff --git a/drivers/dma/ae4dma/meson.build b/drivers/dma/ae4dma/meson.build
> new file mode 100644
> index 0000000000..e48ab0d561
> --- /dev/null
> +++ b/drivers/dma/ae4dma/meson.build
> @@ -0,0 +1,7 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright 2024 Advanced Micro Devices, Inc. All rights reserved.
2024 -> 2026
Does this also support run BSD or Windows, if not please add following instruments:
if not is_linux
build = false
reason = 'only supported on Linux'
subdir_done()
endif
> +
> +build = dpdk_conf.has('RTE_ARCH_X86')
> +reason = 'only supported on x86'
> +sources = files('ae4dma_dmadev.c')
> +deps += ['bus_pci', 'dmadev']
> diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
> index e0d94db967..c230ac5a06 100644
> --- a/drivers/dma/meson.build
> +++ b/drivers/dma/meson.build
> @@ -2,6 +2,7 @@
> # Copyright 2021 HiSilicon Limited
>
> drivers = [
> + 'ae4dma',
> 'cnxk',
> 'dpaa',
> 'dpaa2',
> diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
> index 93f2383dff..7d09f155de 100755
> --- a/usertools/dpdk-devbind.py
> +++ b/usertools/dpdk-devbind.py
> @@ -86,6 +86,9 @@
> cn9k_ree = {'Class': '08', 'Vendor': '177d', 'Device': 'a0f4',
> 'SVendor': None, 'SDevice': None}
>
> +amd_ae4dma = {'Class': '08', 'Vendor': '1022', 'Device': '149b',
> + 'SVendor': None, 'SDevice': None}
> +
> virtio_blk = {'Class': '01', 'Vendor': "1af4", 'Device': '1001,1042',
> 'SVendor': None, 'SDevice': None}
>
> @@ -95,7 +98,7 @@
> network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class]
> baseband_devices = [acceleration_class]
> crypto_devices = [encryption_class, intel_processor_class]
> -dma_devices = [cnxk_dma, hisilicon_dma,
> +dma_devices = [amd_ae4dma, cnxk_dma, hisilicon_dma,
> intel_idxd_gnrd, intel_idxd_dmr, intel_idxd_spr,
> intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx,
> odm_dma]
^ permalink raw reply
* Re: [PATCH] examples: Fix vm_power_manager scratch area to /run/dpdk/powermonitor
From: Sudheendra S @ 2026-06-26 18:46 UTC (permalink / raw)
To: Bruce Richardson
Cc: Stephen Hemminger, dev@dpdk.org, Anatoly Burakov,
Sivaprasad Tummala
In-Reply-To: <CADggw7J2CUQOD5AAF3Uv6kTK25_OV2q3xQfS3zKsSMdSO0R0FA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4490 bytes --]
Hi Stephen,
I appreciate any feedback so that I can make changes and send out the patch.
Thanks
On Thu, Jun 18, 2026 at 12:24 PM Sudheendra S <giveback4fun@gmail.com>
wrote:
> *Note: Changed subject line from /run/dpdk/powermanager ->
> /run/dpdk/powermonitor*
>
> Hi All,
>
> Apologies for the delay.
>
> So, I am following the above thread and below are what I understand are
> the changes that are needed :
>
>
> 1. I see that rte_eal_get_runtime_dir() is RTE_EXPORTED_SYMBOL. I
> will verify that channel_manager has access to this symbol.
> 2. Upon successful creation of /run/dpdk, I can mkdir
> CHANNEL_MGR_SOCKET_PATH (/var/run/dpdk/powermonitor) and opendir() that for
> subsequent processing.
> 3. Fixup the documentation to reflect correct powermonitor path
>
> Please let me know if I am missing anything or if I misinterpreting any
> part of the email exchange.
>
> Thanks for all your help and support.
>
> -Sudheendra G Sampath
>
> On Fri, May 29, 2026 at 9:01 AM Bruce Richardson <
> bruce.richardson@intel.com> wrote:
>
>> On Fri, May 29, 2026 at 08:23:24AM -0700, Stephen Hemminger wrote:
>> > On Fri, 29 May 2026 09:01:34 +0100
>> > Bruce Richardson <bruce.richardson@intel.com> wrote:
>> >
>> > > On Thu, May 28, 2026 at 07:04:48PM +0000, Sudheendra Sampath wrote:
>> > > > This patch for bug 1832 will do the following:
>> > > > 1. If /run/dpdk is not present, it will create it first with and
>> > > > then create powermanager directory underneath it.
>> > > > 2. If /run/dpdk is present, it will verify it is actually a
>> directory
>> > > > before creating subdirectory, powermanager.
>> > > >
>> > > I would suggest using $XDG_RUNTIME_DIR for the directory path, rather
>> than
>> > > hardcoding it by default. If XDG_RUNTIME_DIR is not set, then maybe
>> > > consider using /run/dpdk. However, rather than /run/dpdk, I'd suggest
>> using
>> > > the normal runtime dir path on most distros as the default:
>> > > /run/user/<uid>.
>> > >
>> > > /Bruce
>> >
>> > The login in EAL is a little more detailed.
>> > The choice is from systemd conventions which follows filesystem
>> hierarchy.
>> >
>> >
>> > int eal_create_runtime_dir(void)
>> > {
>> > const char *directory;
>> > char run_dir[PATH_MAX];
>> > char tmp[PATH_MAX];
>> > int ret;
>> >
>> > /* from RuntimeDirectory= see systemd.exec */
>> > directory = getenv("RUNTIME_DIRECTORY");
>> > if (directory == NULL) {
>> > /*
>> > * Used standard convention defined in
>> > * XDG Base Directory Specification and
>> > * Filesystem Hierarchy Standard.
>> > */
>> > if (getuid() == 0)
>> > directory = "/var/run";
>> > else
>> > directory = getenv("XDG_RUNTIME_DIR") ? : "/tmp";
>> > }
>> >
>> > /* create DPDK subdirectory under runtime dir */
>> > ret = snprintf(tmp, sizeof(tmp), "%s/dpdk", directory);
>> > if (ret < 0 || ret == sizeof(tmp)) {
>> > EAL_LOG(ERR, "Error creating DPDK runtime path name");
>> > return -1;
>> > }
>> >
>> > /* create prefix-specific subdirectory under DPDK runtime dir */
>> > ret = snprintf(run_dir, sizeof(run_dir), "%s/%s",
>> > tmp, eal_get_hugefile_prefix());
>> > if (ret < 0 || ret == sizeof(run_dir)) {
>> > EAL_LOG(ERR, "Error creating prefix-specific runtime path
>> name");
>> > return -1;
>> > }
>> >
>> > /* create the path if it doesn't exist. no "mkdir -p" here, so do
>> it
>> > * step by step.
>> > */
>> > ret = mkdir(tmp, 0700);
>> > if (ret < 0 && errno != EEXIST) {
>> > EAL_LOG(ERR, "Error creating '%s': %s",
>> > tmp, strerror(errno));
>> > return -1;
>> > }
>> >
>> > ret = mkdir(run_dir, 0700);
>> > if (ret < 0 && errno != EEXIST) {
>> > EAL_LOG(ERR, "Error creating '%s': %s",
>> > run_dir, strerror(errno));
>> > return -1;
>> > }
>>
>> Yes. Can the power manager call the rte_eal_get_runtime_dir() API and use
>> that as a basis for its working directory? Save duplicating all this
>> logic.
>>
>> /Bruce
>>
>
[-- Attachment #2: Type: text/html, Size: 6122 bytes --]
^ permalink raw reply
* Re: [PATCH v9 19/23] net/sxe2: add mbuf validation in Tx debug mode
From: Stephen Hemminger @ 2026-06-26 18:21 UTC (permalink / raw)
To: liujie5; +Cc: dev
In-Reply-To: <20260626064751.361466-1-liujie5@linkdatatechnology.com>
On Fri, 26 Jun 2026 14:47:51 +0800
liujie5@linkdatatechnology.com wrote:
> From: Jie Liu <liujie5@linkdatatechnology.com>
>
> Introduce the `sxe2_txrx_check_mbuf` helper function to validate outgoing
> mbufs when `RTE_ETHDEV_DEBUG_TX` is enabled. This helps developers catch
> malformed mbufs (e.g., invalid segment lengths, bad offload flags, or
> unaligned buffers) before passing them to the hardware rings, avoiding
> potential hardware hangs or silent packet drops.
>
> The validation is fully wrapped inside `RTE_ETHDEV_DEBUG_TX` conditional
> compilation blocks to ensure zero performance overhead in standard
> production builds.
>
> Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
> ---
I don't think I was clear enough before, the verbose description is:
Current state (v9):
The new sxe2_txrx_check_mbuf() helper added in 19/23 is called only
from sxe2_tx_pkts_prepare(), which is the driver's tx_prepare
callback. It is not called from any tx_burst path (sxe2_tx_pkts, the
vec paths, the poll path).
The commit message claims the validation is "fully wrapped inside
RTE_ETHDEV_DEBUG_TX conditional compilation blocks to ensure zero
performance overhead in standard production builds." This is not what
the code does. There is no #ifdef RTE_ETHDEV_DEBUG_TX anywhere in the
new file or around the new call site. In fact the patch *removes* an
existing #ifdef RTE_ETHDEV_DEBUG_TX that previously gated
rte_validate_tx_offload() in the same loop, making that always-on too.
The new function is marked __rte_unused both in the header
declaration and in the .c definition, even though it has exactly one
caller. That attribute was likely a leftover from an earlier draft
where the function was only conditionally referenced.
The validation itself (595 lines) re-derives outer/inner L2/L3/L4
lengths from the packet bytes and compares them against what the mbuf
header declares - i.e., it's verifying that the application
correctly set mbuf->l2_len, l3_len, outer_l2_len, etc. before
submitting. Much of this overlaps what rte_validate_tx_offload() and
rte_net_intel_cksum_prepare() already check, both of which are called
in the same tx_prepare loop.
Desired state:
tx_burst should not contain validation checks for application errors.
The fast path runs once per packet at line rate, and any branch that
exists to catch a caller bug is paid on every well-behaved packet
forever. For driver assumptions that an application must satisfy,
the three sensible options are:
1. RTE_ASSERT() - compiles out in release builds, surfaces the
assumption to static analyzers, and fires for users running
debug builds.
2. unlikely() with drop-and-counter - if the check is cheap and
the assumption can be violated in production. The bad packet
is dropped, a queue stat is incremented, but the burst loop
continues.
3. Document the assumption in the API contract and don't check it
at runtime.
tx_prepare is where validation belongs, and an unconditional check
there is acceptable - tx_prepare is opt-in, and an application that
calls it has already agreed to pay for inspection. The 595-line
function as written is mostly fine in that context. What's wrong is:
(a) The commit message says one thing and the code does another.
The author should either add the #ifdef RTE_ETHDEV_DEBUG_TX
they claimed to add, or rewrite the commit message to describe
what the patch actually does.
(b) Driver-specific tx_prepare validation should cover sxe2-
specific hardware constraints (descriptor count limits, buffer
alignment, segment size limits) - not redo the generic offload-
flag and length-consistency checks that rte_validate_tx_offload()
and rte_net_intel_cksum_prepare() perform. As currently written,
every packet going through tx_prepare gets its offload flags
validated roughly twice.
(c) __rte_unused on the function should be dropped.
Redo this with one of the above and resubmit
^ permalink raw reply
* RE: [EXTERNAL] Re: [PATCH v2] eal: return error on devargs truncation in hotplug MP messages
From: Long Li @ 2026-06-26 18:20 UTC (permalink / raw)
To: David Marchand
Cc: dev@dpdk.org, stable@dpdk.org, Thomas Monjalon, Bruce Richardson,
Stephen Hemminger, Burakov, Anatoly
In-Reply-To: <CAJFAV8yScD-2zevEo30YrA+93NwYi0qPseHZUbw2riwmnDSk3g@mail.gmail.com>
>
> On Wed, 24 Jun 2026 at 02:11, Long Li <longli@microsoft.com> wrote:
> >
> > The EAL hotplug multi-process messaging uses a fixed-size buffer
> > (EAL_DEV_MP_DEV_ARGS_MAX_LEN, 128 bytes) for device arguments.
> > When devargs exceeds this limit, strlcpy silently truncates the
> > string. This causes secondary processes to receive incomplete devargs
> > during hotplug re-add, leading to failed port re-initialization.
> >
> > For example, a MANA PCI device with 6 mac= arguments:
> >
> > mac=AA:BB:CC:DD:EE:01,mac=AA:BB:CC:DD:EE:02,
> > mac=AA:BB:CC:DD:EE:03,mac=AA:BB:CC:DD:EE:04,
> > mac=AA:BB:CC:DD:EE:05,mac=AA:BB:CC:DD:EE:06
> >
> > produces a 131-byte devargs string that gets silently truncated to 127
> > bytes, losing the last MAC address.
> >
> > Return -E2BIG from rte_dev_probe() when devargs would be truncated,
> > instead of silently corrupting data. rte_dev_remove() does not need
> > the same check because the length was already validated at probe time.
> >
> > Fixes: 244d5130719c ("eal: enable hotplug on multi-process")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Long Li <longli@microsoft.com>
>
> Re-reading the function, I have one concern about the fix.
>
> I agree there is a bug with multiprocess.
> But this change here also imposes a limit to 128 that was not there before,
> even if multiprocess is disabled.
> It may not be a big problem, but we are calling the the multi process
> machinerie when unneeded (it ends up with a ENOTSUP).
>
> I sent a small patch on this topic, could you have a look please?
Thank you. I sent a review comment. This patch looks good.
I'll modify this patch to have the length check inside if (do_mp) block.
Long
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Finbox.
> dpdk.org%2Fdev%2F20260625072254.4190227-1-
> david.marchand%40redhat.com%2F&data=05%7C02%7Clongli%40microsoft.c
> om%7Cf1cf337c1f014cc8b83b08ded28ab9d4%7C72f988bf86f141af91ab2d7cd
> 011db47%7C1%7C0%7C639179690431011725%7CUnknown%7CTWFpbGZsb3
> d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOI
> joiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=7otoCpQbXZvcj5tSD
> QAJ77sdfu8U%2BgPwZZyYeTC%2F2lw%3D&reserved=0
>
>
> --
> David Marchand
^ permalink raw reply
* Re: [PATCH v9 00/23] net/sxe2: added Linkdata sxe2 ethernet driver
From: Stephen Hemminger @ 2026-06-26 18:18 UTC (permalink / raw)
To: liujie5; +Cc: dev
In-Reply-To: <20260626063800.359448-1-liujie5@linkdatatechnology.com>
On Fri, 26 Jun 2026 14:38:00 +0800
liujie5@linkdatatechnology.com wrote:
> From: Jie Liu <liujie5@linkdatatechnology.com>
>
> This patch set implements core functionality for the SXE2 PMD,
> including basic driver framework, data path setup, and advanced
> offload features (VLAN, RSS,TM, PTP etc.).
>
> v9
> Clarify the behavior of the flow-duplicate-pattern parameter
>
> *** BLURB HERE ***
>
> Jie Liu (23):
> net/sxe2: remove software statistics devargs
> net/sxe2: add Rx framework and packet types callback
> net/sxe2: support AVX512 vectorized path for Rx and Tx
> net/sxe2: add AVX2 vector data path for Rx and Tx
> net/sxe2: add link update callback
> net/sxe2: support L2 filtering and MAC config
> drivers: support RSS feature
> net/sxe2: support TM hierarchy and shaping
> net/sxe2: support IPsec inline protocol offload
> net/sxe2: support statistics and multi-process
> drivers: interrupt handling
> net/sxe2: add NEON vec Rx/Tx burst functions
> drivers: add support for VF representors
> net/sxe2: add support for custom UDP tunnel ports
> net/sxe2: support firmware version reading
> net/sxe2: implement get monitor address
> common/sxe2: add shared SFP module definitions
> net/sxe2: support SFP module info and EEPROM access
> net/sxe2: add mbuf validation in Tx debug mode
> common/sxe2: add callback for memory event handling
> net/sxe2: add private devargs parsing
> net/sxe2: implement private dump info
> net/sxe2: update sxe2 feature matrix docs
>
> doc/guides/nics/features/sxe2.ini | 56 +
> doc/guides/nics/sxe2.rst | 186 ++
> drivers/common/sxe2/sxe2_common.c | 156 ++
> drivers/common/sxe2/sxe2_common.h | 4 +
> drivers/common/sxe2/sxe2_flow_public.h | 633 +++++++
> drivers/common/sxe2/sxe2_ioctl_chnl.c | 178 +-
> drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 18 +
> drivers/common/sxe2/sxe2_msg.h | 118 ++
> drivers/net/sxe2/meson.build | 52 +
> drivers/net/sxe2/sxe2_cmd_chnl.c | 1587 +++++++++++++++-
> drivers/net/sxe2/sxe2_cmd_chnl.h | 139 ++
> drivers/net/sxe2/sxe2_drv_cmd.h | 523 +++++-
> drivers/net/sxe2/sxe2_dump.c | 287 +++
> drivers/net/sxe2/sxe2_dump.h | 12 +
> drivers/net/sxe2/sxe2_ethdev.c | 1496 ++++++++++++++-
> drivers/net/sxe2/sxe2_ethdev.h | 111 +-
> drivers/net/sxe2/sxe2_ethdev_repr.c | 609 ++++++
> drivers/net/sxe2/sxe2_ethdev_repr.h | 32 +
> drivers/net/sxe2/sxe2_filter.c | 895 +++++++++
> drivers/net/sxe2/sxe2_filter.h | 100 +
> drivers/net/sxe2/sxe2_flow.c | 1391 ++++++++++++++
> drivers/net/sxe2/sxe2_flow.h | 30 +
> drivers/net/sxe2/sxe2_flow_define.h | 144 ++
> drivers/net/sxe2/sxe2_flow_parse_action.c | 1182 ++++++++++++
> drivers/net/sxe2/sxe2_flow_parse_action.h | 23 +
> drivers/net/sxe2/sxe2_flow_parse_engine.c | 106 ++
> drivers/net/sxe2/sxe2_flow_parse_engine.h | 13 +
> drivers/net/sxe2/sxe2_flow_parse_pattern.c | 1935 +++++++++++++++++++
> drivers/net/sxe2/sxe2_flow_parse_pattern.h | 46 +
> drivers/net/sxe2/sxe2_ipsec.c | 1565 ++++++++++++++++
> drivers/net/sxe2/sxe2_ipsec.h | 254 +++
> drivers/net/sxe2/sxe2_irq.c | 1026 ++++++++++
> drivers/net/sxe2/sxe2_irq.h | 25 +
> drivers/net/sxe2/sxe2_mac.c | 530 ++++++
> drivers/net/sxe2/sxe2_mac.h | 84 +
> drivers/net/sxe2/sxe2_mp.c | 414 +++++
> drivers/net/sxe2/sxe2_mp.h | 67 +
> drivers/net/sxe2/sxe2_queue.c | 17 +-
> drivers/net/sxe2/sxe2_queue.h | 15 +-
> drivers/net/sxe2/sxe2_rss.c | 584 ++++++
> drivers/net/sxe2/sxe2_rss.h | 81 +
> drivers/net/sxe2/sxe2_rx.c | 93 +-
> drivers/net/sxe2/sxe2_rx.h | 2 +
> drivers/net/sxe2/sxe2_security.c | 335 ++++
> drivers/net/sxe2/sxe2_security.h | 77 +
> drivers/net/sxe2/sxe2_stats.c | 586 ++++++
> drivers/net/sxe2/sxe2_stats.h | 39 +
> drivers/net/sxe2/sxe2_switchdev.c | 332 ++++
> drivers/net/sxe2/sxe2_switchdev.h | 33 +
> drivers/net/sxe2/sxe2_tm.c | 1151 ++++++++++++
> drivers/net/sxe2/sxe2_tm.h | 76 +
> drivers/net/sxe2/sxe2_tx.c | 7 +
> drivers/net/sxe2/sxe2_txrx.c | 1958 +++++++++++++++++++-
> drivers/net/sxe2/sxe2_txrx.h | 8 +
> drivers/net/sxe2/sxe2_txrx_check_mbuf.c | 595 ++++++
> drivers/net/sxe2/sxe2_txrx_check_mbuf.h | 38 +
> drivers/net/sxe2/sxe2_txrx_poll.c | 284 ++-
> drivers/net/sxe2/sxe2_txrx_vec.c | 46 +-
> drivers/net/sxe2/sxe2_txrx_vec.h | 38 +-
> drivers/net/sxe2/sxe2_txrx_vec_avx2.c | 747 ++++++++
> drivers/net/sxe2/sxe2_txrx_vec_avx512.c | 867 +++++++++
> drivers/net/sxe2/sxe2_txrx_vec_common.h | 54 +-
> drivers/net/sxe2/sxe2_txrx_vec_neon.c | 689 +++++++
> drivers/net/sxe2/sxe2_txrx_vec_sse.c | 38 +-
> drivers/net/sxe2/sxe2_vsi.c | 146 ++
> drivers/net/sxe2/sxe2_vsi.h | 12 +-
> drivers/net/sxe2/sxe2vf_regs.h | 85 +
> 67 files changed, 24787 insertions(+), 273 deletions(-)
> create mode 100644 drivers/common/sxe2/sxe2_flow_public.h
> create mode 100644 drivers/common/sxe2/sxe2_msg.h
> create mode 100644 drivers/net/sxe2/sxe2_dump.c
> create mode 100644 drivers/net/sxe2/sxe2_dump.h
> create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.c
> create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.h
> create mode 100644 drivers/net/sxe2/sxe2_filter.c
> create mode 100644 drivers/net/sxe2/sxe2_filter.h
> create mode 100644 drivers/net/sxe2/sxe2_flow.c
> create mode 100644 drivers/net/sxe2/sxe2_flow.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_define.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.h
> create mode 100644 drivers/net/sxe2/sxe2_ipsec.c
> create mode 100644 drivers/net/sxe2/sxe2_ipsec.h
> create mode 100644 drivers/net/sxe2/sxe2_irq.c
> create mode 100644 drivers/net/sxe2/sxe2_mac.c
> create mode 100644 drivers/net/sxe2/sxe2_mac.h
> create mode 100644 drivers/net/sxe2/sxe2_mp.c
> create mode 100644 drivers/net/sxe2/sxe2_mp.h
> create mode 100644 drivers/net/sxe2/sxe2_rss.c
> create mode 100644 drivers/net/sxe2/sxe2_rss.h
> create mode 100644 drivers/net/sxe2/sxe2_security.c
> create mode 100644 drivers/net/sxe2/sxe2_security.h
> create mode 100644 drivers/net/sxe2/sxe2_stats.c
> create mode 100644 drivers/net/sxe2/sxe2_stats.h
> create mode 100644 drivers/net/sxe2/sxe2_switchdev.c
> create mode 100644 drivers/net/sxe2/sxe2_switchdev.h
> create mode 100644 drivers/net/sxe2/sxe2_tm.c
> create mode 100644 drivers/net/sxe2/sxe2_tm.h
> create mode 100644 drivers/net/sxe2/sxe2_txrx_check_mbuf.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_check_mbuf.h
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx2.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx512.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_neon.c
> create mode 100644 drivers/net/sxe2/sxe2vf_regs.h
The devargs parsing looks fixed now.
AI review from long running chat..
[PATCH v9 00/23] sxe2 driver feature additions
Only two patches changed v8 -> v9 (devargs parsing and feature matrix
docs), and they address the flow-duplicate-pattern alignment correctly
this time.
[PATCH v9 21/23] devargs parsing - flow-duplicate-pattern now aligns
The user-facing semantics for value 0 and value 1 now match mlx5's
allow_duplicate_pattern:
Value mlx5 allow_duplicate_pattern sxe2 v9 flow-duplicate-pattern
----- -------------------------- --------------------------------
0 reject duplicates (EEXIST) reject duplicates (EEXIST)
1 allow; first rule active (FIFO) allow; first rule active (FIFO)
2 (not supported) allow; last rule active (LIFO)
Default is 1 in both PMDs. A user who sets flow-duplicate-pattern=1 on
sxe2 now gets the same behaviour as allow_duplicate_pattern=1 on mlx5.
Value 2 (LIFO) is an sxe2-specific extension that mlx5 doesn't have,
and the documentation says so. This is the right shape - aligned where
the capabilities overlap, extended where sxe2 has a real additional
capability.
The implementation uses a small lookup table to map the user value to
the internal enum sxe2_flow_sw_pattern_type (whose numeric ordering
{ONLY=0, LAST=1, FIRST=2} doesn't match the user-facing ordering):
static const uint8_t udc_tbl[] = {
SXE2_FLOW_SW_PATTERN_ONLY, /* user 0 -> 0 */
SXE2_FLOW_SW_PATTERN_FIRST, /* user 1 -> 2 */
SXE2_FLOW_SW_PATTERN_LAST, /* user 2 -> 1 */
};
The comment block above the table explains the mapping clearly. The
table name `udc_tbl` is a bit opaque - `user_to_internal[]` or
`mode_map[]` would read better - but the meaning is documented in the
comment so this is a nit, not a blocker.
The orphan ternary in sxe2_flow.c that picked between "Duplicate flow
pattern." and "Duplicate flow pattern is not allowed." based on the
mode value (raised against v8) is gone. Error message is now a single
"Duplicate flow." string, which is appropriate - the rejection path
only fires in mode 0 anyway.
[PATCH v9 23/23] feature matrix docs
Documentation for flow-duplicate-pattern is rewritten to closely match
mlx5.rst's style. The mode 0 and mode 1 paragraphs are essentially
verbatim from mlx5.rst (which is fine - same semantic, same wording is
clearer for users), with mode 2 added as the sxe2-specific extension.
The new paragraph explaining cross-port vs single-port behaviour is
sxe2-specific context that mlx5 doesn't need, and it's a useful clarification.
The default-value note now says "1 (allow duplicates, first-added takes
effect)" - explicit about which mode the default is, which the v8
documentation glossed.
^ permalink raw reply
* RE: [EXTERNAL] [PATCH] dev: skip multi-process in hotplug
From: Long Li @ 2026-06-26 18:09 UTC (permalink / raw)
To: David Marchand, dev@dpdk.org
In-Reply-To: <20260625072254.4190227-1-david.marchand@redhat.com>
> If multi-process is disabled (--no-shconf/--in-memory), there is no need to
> build a multi-process message.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> lib/eal/common/eal_common_dev.c | 46 ++++++++++++++++++++++++---------
> 1 file changed, 34 insertions(+), 12 deletions(-)
>
> diff --git a/lib/eal/common/eal_common_dev.c
> b/lib/eal/common/eal_common_dev.c index 48b631532a..b0ec2b0601
> 100644
> --- a/lib/eal/common/eal_common_dev.c
> +++ b/lib/eal/common/eal_common_dev.c
> @@ -267,10 +267,16 @@ RTE_EXPORT_SYMBOL(rte_dev_probe) int
> rte_dev_probe(const char *devargs) {
> + const struct internal_config *internal_conf =
> + eal_get_internal_configuration();
> + bool do_mp = internal_conf->no_shconf == 0;
> struct eal_dev_mp_req req;
> struct rte_device *dev;
> int ret;
>
> + if (!do_mp)
> + goto skip_mp_req;
> +
> memset(&req, 0, sizeof(req));
> req.t = EAL_DEV_REQ_TYPE_ATTACH;
> strlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN);
> @@ -294,6 +300,7 @@ rte_dev_probe(const char *devargs)
>
> /* attach a shared device from primary start from here: */
>
> +skip_mp_req:
> /* primary attach the new device itself. */
> ret = local_dev_probe(devargs, &dev);
>
> @@ -311,6 +318,9 @@ rte_dev_probe(const char *devargs)
> return ret;
> }
>
> + if (!do_mp)
> + goto skip_mp_notify;
> +
> /* primary send attach sync request to secondary. */
> ret = eal_dev_hotplug_request_to_secondary(&req);
>
> @@ -337,16 +347,19 @@ rte_dev_probe(const char *devargs)
> goto rollback;
> }
>
> +skip_mp_notify:
> return 0;
>
> rollback:
> - req.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK;
> + if (do_mp) {
> + req.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK;
>
> - /* primary send rollback request to secondary. */
> - if (eal_dev_hotplug_request_to_secondary(&req) != 0)
> - EAL_LOG(WARNING,
> - "Failed to rollback device attach on secondary."
> - "Devices in secondary may not sync with primary");
> + /* primary send rollback request to secondary. */
> + if (eal_dev_hotplug_request_to_secondary(&req) != 0)
> + EAL_LOG(WARNING,
> + "Failed to rollback device attach on
> secondary."
> + "Devices in secondary may not sync with
> primary");
> + }
>
> /* primary rollback itself. */
> if (local_dev_remove(dev) != 0)
> @@ -405,6 +418,9 @@ RTE_EXPORT_SYMBOL(rte_dev_remove) int
> rte_dev_remove(struct rte_device *dev) {
> + const struct internal_config *internal_conf =
> + eal_get_internal_configuration();
> + bool do_mp = internal_conf->no_shconf == 0;
> struct eal_dev_mp_req req;
> char *devargs;
> int ret;
> @@ -418,6 +434,9 @@ rte_dev_remove(struct rte_device *dev)
> if (ret != 0)
> return ret;
>
> + if (!do_mp)
> + goto skip_mp_req;
> +
Need to call free(devargs) as it no longer used.
> memset(&req, 0, sizeof(req));
> req.t = EAL_DEV_REQ_TYPE_DETACH;
> strlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN);
> @@ -472,6 +491,7 @@ rte_dev_remove(struct rte_device *dev)
> goto rollback;
> }
>
> +skip_mp_req:
> /* primary detach the device itself. */
> ret = local_dev_remove(dev);
>
> @@ -488,13 +508,15 @@ rte_dev_remove(struct rte_device *dev)
> return 0;
>
> rollback:
> - req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK;
> + if (do_mp) {
> + req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK;
>
> - /* primary send rollback request to secondary. */
> - if (eal_dev_hotplug_request_to_secondary(&req) != 0)
> - EAL_LOG(WARNING,
> - "Failed to rollback device detach on secondary."
> - "Devices in secondary may not sync with primary");
> + /* primary send rollback request to secondary. */
> + if (eal_dev_hotplug_request_to_secondary(&req) != 0)
> + EAL_LOG(WARNING,
> + "Failed to rollback device detach on
> secondary."
> + "Devices in secondary may not sync with
> primary");
> + }
>
> return ret;
> }
> --
> 2.54.0
^ permalink raw reply
* Re: [PATCH 4/5] crypto/armv8: use timing-safe digest comparison
From: Jack Bond-Preston @ 2026-06-26 17:11 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: stable, Siraj Luthfi Ananda
In-Reply-To: <20260625160200.24170-5-stephen@networkplumber.org>
Acked-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
On 25/06/2026 16:56, Stephen Hemminger wrote:
> The chained-op verify path compared the computed and expected MAC
> with memcmp(), whose run time depends on the number of matching
> leading bytes and can leak the digest to an attacker submitting
> forged values.
>
> Use rte_memeq_timingsafe() for the verify comparison.
>
> Bugzilla ID: 1773
> Fixes: 169ca3db550c ("crypto/armv8: add PMD optimized for ARMv8 processors")
> Cc: stable@dpdk.org
>
> Reported-by: Siraj Luthfi Ananda <sirajluthfi@gmail.com>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> drivers/crypto/armv8/rte_armv8_pmd.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/crypto/armv8/rte_armv8_pmd.c b/drivers/crypto/armv8/rte_armv8_pmd.c
> index 320e2d4b3b..a7caac186d 100644
> --- a/drivers/crypto/armv8/rte_armv8_pmd.c
> +++ b/drivers/crypto/armv8/rte_armv8_pmd.c
> @@ -631,8 +631,8 @@ process_armv8_chained_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op,
>
> op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
> if (sess->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
> - if (memcmp(adst, op->sym->auth.digest.data,
> - sess->auth.digest_length) != 0) {
> + if (!rte_memeq_timingsafe(adst, op->sym->auth.digest.data,
> + sess->auth.digest_length)) {
> op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
> }
> }
^ permalink raw reply
* Re: [PATCH v5 17/19] drivers: release DPAA bpid on driver destructor
From: Stephen Hemminger @ 2026-06-26 16:59 UTC (permalink / raw)
To: Hemant Agrawal; +Cc: david.marchand, dev, Jun Yang
In-Reply-To: <20260626065655.279742-18-hemant.agrawal@nxp.com>
On Fri, 26 Jun 2026 12:26:53 +0530
Hemant Agrawal <hemant.agrawal@nxp.com> wrote:
> From: Jun Yang <jun.yang@nxp.com>
>
> Track allocated BPIDs in a static per-BPID flag table and register a
> driver destructor that releases any BPIDs still marked as in use at
> process exit. This prevents BPID leaks when an application exits without
> calling rte_mempool_free(). Also tune the per-lcore mempool cache flush
> threshold to match the hardware bulk release size (DPAA_MBUF_MAX_ACQ_REL)
> so that buffers are returned to HW in optimal burst sizes.
>
> Signed-off-by: Jun Yang <jun.yang@nxp.com>
> ---
Build with ASAN fails with this patch.
Simple fix is usually replacing all use of rte_memcpy with memcpy.
Rte_memcpy on x86 confuses ASAN and it thinks certain sizes reference past
end of buffer. I would just do a global replace of rte_memcpy with memcpy
across all of drivers/bus/dpaa none of this is performance critical on old
versions of Gcc which is the only reason rte_memcpy still exists.
This needs to be fixed before merging.
In file included from /usr/lib/gcc/x86_64-linux-gnu/15/include/immintrin.h:43,
from /usr/lib/gcc/x86_64-linux-gnu/15/include/x86intrin.h:32,
from ../lib/eal/x86/include/rte_vect.h:24,
from ../lib/eal/x86/include/rte_memcpy.h:17,
from ../drivers/bus/dpaa/base/qbman/bman.c:7:
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:158:2,
inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:169:2,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:389:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript ‘__m256i_u[1]’ is partly outside array bounds of ‘struct bm_buffer[8]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset 40 into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:157:2,
inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:170:2,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:389:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript 2 is outside array bounds of ‘struct bm_buffer[8]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset 72 into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:158:2,
inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:170:2,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:389:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript 3 is outside array bounds of ‘struct bm_buffer[8]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset 104 into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:158:2,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:396:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript ‘__m256i_u[1]’ is partly outside array bounds of ‘const void[64]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [40, 48] into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [41, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [168, 176] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [40, 48] into object ‘bm_bufs’ of size 64
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:401:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript [2, 66] is outside array bounds of ‘const void[64]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [8, 64] into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [9, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [136, 208] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [8, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [9, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [136, 208] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [8, 64] into object ‘bm_bufs’ of size 64
In function ‘_mm256_loadu_si256’,
inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:119:9,
inlined from ‘rte_memcpy_generic_more_than_64’ at ../lib/eal/x86/include/rte_memcpy.h:402:4,
inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:715:10,
inlined from ‘bman_release_fast’ at ../drivers/bus/dpaa/base/qbman/bman.c:296:3:
/usr/lib/gcc/x86_64-linux-gnu/15/include/avxintrin.h:873:10: warning: array subscript [2, 67] is outside array bounds of ‘const void[64]’ [-Warray-bounds=]
873 | return *__P;
| ^~~~
../drivers/bus/dpaa/base/qbman/bman.c: In function ‘bman_release_fast’:
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [9, 64] into object ‘bm_bufs’ of size 64
270 | struct bm_buffer bm_bufs[FSL_BM_BURST_MAX];
| ^~~~~~~
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [10, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [137, 209] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [9, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [10, 64] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [137, 209] into object ‘bm_bufs’ of size 64
../drivers/bus/dpaa/base/qbman/bman.c:270:26: note: at offset [9, 64] into object ‘bm_bufs’ of size 64
^ permalink raw reply
* Re: [PATCH v7 2/6] eal: use request ID instead of pointers
From: Stephen Hemminger @ 2026-06-26 15:08 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev, Jianfeng Tan
In-Reply-To: <4f373899b3b4dc3e529e31f5238594b4857b41c3.1782469943.git.anatoly.burakov@intel.com>
On Fri, 26 Jun 2026 11:33:57 +0100
Anatoly Burakov <anatoly.burakov@intel.com> wrote:
> @@ -1178,6 +1203,7 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
> * it, and put it on the queue if we don't send any requests.
> */
> dummy->type = REQUEST_TYPE_ASYNC;
> + dummy->id = ++next_request_id;
> dummy->request = copy;
> dummy->reply = NULL;
> dummy->async.param = param;
> --
Is there any chance of two async requests being generated on different threads.
Would be safer to use an atomic for this?
^ permalink raw reply
* Re: [v5] pcapng: add user-supplied timestamp support
From: Stephen Hemminger @ 2026-06-26 15:04 UTC (permalink / raw)
To: Dawid Wesierski; +Cc: dev, marek.kasiewicz, thomas, mb
In-Reply-To: <20260626102213.776257-1-dawid.wesierski@intel.com>
On Fri, 26 Jun 2026 06:22:04 -0400
Dawid Wesierski <dawid.wesierski@intel.com> wrote:
> Thanks again for the review,
>
> For v6 I have added the input validation we ccheck if the timestamp
> PCAPNG_TSC_FLAG is not set on caller-provided input.
>
> Regarding API versioning my concern is around source-level compatibility.
> Any application that calls rte_pcapng_copy() would need to be updated to
> pass the additional timestamp argument right?
>
> Is breaking source-level compatibility for existing callers of
> rte_pcapng_copy() acceptable here?
DPDK does not guarantee API compatibility across releases; only ABI.
As long as there is a release note this is fine.
At this point, might have to wait for 26.11 release anyway.
Also, this is not an API that is commonly used by applications.
^ permalink raw reply
* Re: [PATCH v6 0/9] bpf: JIT related bug fixes
From: Stephen Hemminger @ 2026-06-26 15:01 UTC (permalink / raw)
To: Marat Khalili; +Cc: dev@dpdk.org
In-Reply-To: <cf0e1f65fc574f2a9ffebd5fac7ca60c@huawei.com>
On Fri, 26 Jun 2026 10:35:47 +0000
Marat Khalili <marat.khalili@huawei.com> wrote:
> Series-tested-by: Marat Khalili <marat.khalili@huawei.com>
>
> (Maybe it actually deserves a release note at this point?)
Maybe just a single release note about BPF enhancements/fixes
and add sub bullets for your loader and verifier changes.
^ permalink raw reply
* [PATCH v1 1/1] net/iavf: workaround for IRQ map size mismatch
From: Anatoly Burakov @ 2026-06-26 14:25 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Some out-of-tree ICE kernel driver versions have a known issue where they
expect size of IRQ queue vector map command to be bigger than needed, and
because the driver rejects the correct size as invalid, a workaround is
needed to send down a "wrong" size and configure the IRQ maps.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/iavf/iavf_vchnl.c | 30 ++++++++++++++++++++++++-----
1 file changed, 25 insertions(+), 5 deletions(-)
diff --git a/drivers/net/intel/iavf/iavf_vchnl.c b/drivers/net/intel/iavf/iavf_vchnl.c
index 0643a835d5..408ff31aa5 100644
--- a/drivers/net/intel/iavf/iavf_vchnl.c
+++ b/drivers/net/intel/iavf/iavf_vchnl.c
@@ -1566,7 +1566,8 @@ iavf_config_irq_map(struct iavf_adapter *adapter)
static int
iavf_config_irq_map_lv_chunk(struct iavf_adapter *adapter,
uint16_t chunk_sz,
- uint16_t chunk_start)
+ uint16_t chunk_start,
+ bool wrong_size)
{
struct {
struct virtchnl_queue_vector_maps map_info;
@@ -1593,9 +1594,19 @@ iavf_config_irq_map_lv_chunk(struct iavf_adapter *adapter,
qv_maps->vector_id = vf->qv_map[chunk_start + i].vector_id;
}
- /* for some reason PF side checks for buffer being too big, so adjust it down */
- buf_len = sizeof(struct virtchnl_queue_vector_maps) +
- sizeof(struct virtchnl_queue_vector) * (chunk_sz - 1);
+ /*
+ * in out-of-tree kernel driver versions 2.6.4 and 2.6.6 there is a
+ * known issue where the PF side expects the buffer size to be bigger
+ * than necessary. this is a workaround for that issue.
+ */
+ if (wrong_size)
+ /* send N+1 instead of N to workaround PF side bug */
+ buf_len = sizeof(struct virtchnl_queue_vector_maps) +
+ sizeof(struct virtchnl_queue_vector) * (chunk_sz);
+ else
+ /* virtchnl_queue_vector_maps already contains 1 queue_vector */
+ buf_len = sizeof(struct virtchnl_queue_vector_maps) +
+ sizeof(struct virtchnl_queue_vector) * (chunk_sz - 1);
args.ops = VIRTCHNL_OP_MAP_QUEUE_VECTOR;
args.in_args = (u8 *)map_info;
@@ -1609,12 +1620,21 @@ iavf_config_irq_map_lv_chunk(struct iavf_adapter *adapter,
int
iavf_config_irq_map_lv(struct iavf_adapter *adapter, uint16_t num)
{
+ static bool wrong_size;
uint16_t c;
int err;
for (c = 0; c < num; c += IAVF_CFG_Q_NUM_PER_BUF) {
uint16_t chunk_sz = RTE_MIN(num - c, IAVF_CFG_Q_NUM_PER_BUF);
- err = iavf_config_irq_map_lv_chunk(adapter, chunk_sz, c);
+ err = iavf_config_irq_map_lv_chunk(adapter, chunk_sz, c, wrong_size);
+ if (err) {
+ /* try workaround for a known issue */
+ err = iavf_config_irq_map_lv_chunk(adapter, chunk_sz, c, true);
+ if (err == 0) {
+ PMD_DRV_LOG(DEBUG, "Using workaround for IRQ map");
+ wrong_size = true;
+ }
+ }
if (err) {
PMD_DRV_LOG(ERR, "Failed to configure irq map chunk [%u, %u)",
c, c + chunk_sz);
--
2.47.3
^ permalink raw reply related
* [PATCH] net/idpf: fix Tx of large mbuf segments
From: Bruce Richardson @ 2026-06-26 13:43 UTC (permalink / raw)
To: dev
Cc: anatoly.burakov, Bruce Richardson, stable, Jingjing Wu,
Praveen Shetty, Xiaoyun Li, Beilei Xing, Junfeng Guo
In-Reply-To: <20260303150026.1601461-1-bruce.richardson@intel.com>
When TSO is enabled, and we get a packet to transmit where an individual
mbuf segment is longer than 16k, we need to split that segment across
multiple Tx descriptors. This support is present in the single-queue
mode of idpf - since it shares common code with the other Intel drivers
- but was missing from the splitq path.
This patch adds the proper data path handling. Previous work ensured
that the descriptor count calculation took over-sized segments into
account but the actual descriptor writing part was overlooked.
Fixes: 770f4dfe0f79 ("net/idpf: support basic Tx data path")
Fixes: 2904020f8313 ("net/intel: add common function to calculate needed descs")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
v2: rework implementation based on changes to the idpf driver
since v1 was submitted.
---
drivers/net/intel/idpf/idpf_common_rxtx.c | 37 ++++++++++++++++++++---
1 file changed, 32 insertions(+), 5 deletions(-)
diff --git a/drivers/net/intel/idpf/idpf_common_rxtx.c b/drivers/net/intel/idpf/idpf_common_rxtx.c
index a123d969ee..8aa61a2af4 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx.c
@@ -994,16 +994,43 @@ idpf_dp_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t first_sw_id = sw_id;
do {
+ uint16_t slen = tx_pkt->data_len;
+ rte_iova_t buf_dma_addr = rte_mbuf_data_iova(tx_pkt);
+
+ /* Split segment across multiple descriptors if needed
+ * for TSO packets where segment exceeds max buf size.
+ */
+ while ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) &&
+ unlikely(slen > CI_MAX_DATA_PER_TXD)) {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = NULL;
+
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.cmd_dtype = cmd_dtype |
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE;
+ txd->qw1.rxr_bufsize = CI_MAX_DATA_PER_TXD;
+ txd->qw1.compl_tag = sw_id;
+
+ buf_dma_addr += CI_MAX_DATA_PER_TXD;
+ slen -= CI_MAX_DATA_PER_TXD;
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ }
+
txd = &txr[tx_id];
txn = &sw_ring[txe->next_id];
txe->mbuf = tx_pkt;
/* Setup TX descriptor */
- txd->buf_addr =
- rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
- cmd_dtype |= IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE;
- txd->qw1.cmd_dtype = cmd_dtype;
- txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.cmd_dtype = cmd_dtype |
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE;
+ txd->qw1.rxr_bufsize = slen;
txd->qw1.compl_tag = sw_id;
tx_id++;
if (tx_id == txq->nb_tx_desc)
--
2.53.0
^ permalink raw reply related
* [PATCH v2] vhost/crypto: fix segfault
From: Radu Nicolau @ 2026-06-26 10:55 UTC (permalink / raw)
To: dev
Cc: david.marchand, Radu Nicolau, stable, Maxime Coquelin, Chenbo Xia,
Jay Zhou, Fan Zhang
In-Reply-To: <20260624142023.1525497-1-radu.nicolau@intel.com>
Fix potential call with dev->mem uninitialized, one common usecase
example being running the autotest with more than one device.
Fixes: 3bb595ecd682 ("vhost/crypto: add request handler")
Cc: stable@dpdk.org
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
v2: removed unncessary ()
lib/vhost/vhost_crypto.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vhost/vhost_crypto.c b/lib/vhost/vhost_crypto.c
index 648e2d731b..a19cdd61cc 100644
--- a/lib/vhost/vhost_crypto.c
+++ b/lib/vhost/vhost_crypto.c
@@ -1512,6 +1512,10 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
VC_LOG_ERR("Invalid descriptor");
return -1;
}
+ if (unlikely(vc_req->dev->mem == NULL)) {
+ VC_LOG_ERR("Uninitialized vhost device");
+ return -1;
+ }
dlen = head->len;
src_desc = IOVA_TO_VVA(struct vring_desc *, vc_req->dev, vq,
--
2.52.0
^ permalink raw reply related
* RE: [PATCH v6 0/9] bpf: JIT related bug fixes
From: Marat Khalili @ 2026-06-26 10:35 UTC (permalink / raw)
To: Stephen Hemminger, dev@dpdk.org
In-Reply-To: <20260625173231.216074-1-stephen@networkplumber.org>
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Thursday 25 June 2026 18:30
> To: dev@dpdk.org
> Cc: Stephen Hemminger <stephen@networkplumber.org>
> Subject: [PATCH v6 0/9] bpf: JIT related bug fixes
>
> While implementing JIT for packet capture ran into several issues:
> 1. x86 JIT had a pre-existing bug which would crash.
> 2. The arm64 JIT was missing the packet-access instructions, found
> previously [1].
> 3. Shift counts were not masked to the operand width as RFC 9669
> requires: undefined behavior in the interpreter and an encoding
> failure in the arm64 JIT.
> 4. Tests related to JIT were not being run or were missing coverage.
>
> Fixed all of these. Patches are ordered with the most urgent fix (the
> x86 crash) first, each fix followed by the test that exercises it. The
> arm64 packet-load support is kept ahead of the "check JIT was generated"
> patch so the series bisects cleanly on arm64.
>
> The arm64 epilogue branch fix (patch 6) was originally posted by
> Christophe Fontaine [1]; that series stalled, so it is carried here with
> his authorship.
>
> [1] https://inbox.dpdk.org/dev/20260319114500.9757-2-cfontain@redhat.com/
>
> v6:
> - address Marat's review ARM64 JIT of LD IND instructions
> need to handle offsets outside of 32 bit range.
>
>
> Christophe Fontaine (1):
> bpf/arm64: fix offset type to allow a negative jump
>
> Stephen Hemminger (8):
> bpf/x86: fix JIT encoding of fixed-width immediates
> test/bpf: add JSET test with small immediate
> bpf: mask shift count in interpreter per RFC 9669
> bpf/arm64: mask shift count per RFC 9669
> test/bpf: add test for large shift
> bpf/arm64: add BPF_ABS/BPF_IND packet load support
> test/bpf: check that JIT was generated
> test/bpf: check that bpf_convert can be JIT'd
>
> app/test/test_bpf.c | 320 +++++++++++++++++++++++++++++++---------
> lib/bpf/bpf_exec.c | 31 ++--
> lib/bpf/bpf_jit_arm64.c | 185 ++++++++++++++++++++++-
> lib/bpf/bpf_jit_x86.c | 6 +-
> lib/bpf/meson.build | 2 +
> 5 files changed, 456 insertions(+), 88 deletions(-)
>
> --
> 2.53.0
Series-tested-by: Marat Khalili <marat.khalili@huawei.com>
(Maybe it actually deserves a release note at this point?)
^ permalink raw reply
* RE: [PATCH v6 7/9] bpf/arm64: add BPF_ABS/BPF_IND packet load support
From: Marat Khalili @ 2026-06-26 10:34 UTC (permalink / raw)
To: Stephen Hemminger, dev@dpdk.org; +Cc: Wathsala Vithanage, Konstantin Ananyev
In-Reply-To: <20260625173231.216074-8-stephen@networkplumber.org>
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Thursday 25 June 2026 18:30
> To: dev@dpdk.org
> Cc: Stephen Hemminger <stephen@networkplumber.org>; Wathsala Vithanage <wathsala.vithanage@arm.com>;
> Konstantin Ananyev <konstantin.ananyev@huawei.com>; Marat Khalili <marat.khalili@huawei.com>
> Subject: [PATCH v6 7/9] bpf/arm64: add BPF_ABS/BPF_IND packet load support
>
> The arm64 JIT rejected BPF_LD | BPF_ABS and BPF_LD | BPF_IND with
> "invalid opcode", so cBPF programs converted by rte_bpf_convert() could
> not be JITed. Add these opcodes, mirroring the x86 JIT: a fast path for
> data held in the first mbuf segment, and a __rte_pktmbuf_read() slow
> path for everything else.
>
> The forward branches over the call cannot use fixed distances:
> emit_call() materializes the helper address with a variable number of
> mov/movk instructions, so the block sizes are not known up front. Size
> the three blocks (fast path, slow path, common tail) in a dry run, then
> emit for real with the branches resolved from the measured offsets.
>
> The effective offset is validated before use: src is a runtime value for
> BPF_IND, so a negative offset is routed to the slow path rather than
> read from the first segment, and the offset is bounded to UINT32_MAX
> before __rte_pktmbuf_read(), whose off argument is uint32_t.
>
> Programs using these opcodes use the call register layout, since the
> slow path makes a function call.
>
> For example, BPF_LD | BPF_IND | BPF_W (4-byte indirect load, mbuf in
> R6/x19, effective offset kept in x9) emits:
>
> mov x9, #imm // off = imm
> add x9, x9, src // off += src (BPF_IND)
> cmp x9, xzr // reject negative
> b.mi slow // effective offset
> mov x10, #data_len_ofs
> ldrh w10, [x19, x10] // mbuf->data_len
> sub x10, x10, x9 // data_len - off
> mov x11, #sz
> cmp x10, x11
> b.lt slow // not in first segment
> mov x10, #data_off_ofs
> ldrh w10, [x19, x10] // mbuf->data_off
> mov x7, #buf_addr_ofs
> ldr x7, [x19, x7] // mbuf->buf_addr
> add x7, x7, x10
> add x7, x7, x9 // ptr = buf_addr + data_off + off
> b load
> slow:
> mov x10, #UINT32_MAX
> cmp x9, x10
> b.ls 1f // off fits uint32_t ...
> mov x7, #0 // else return 0
> b epilogue
> 1: mov x1, x9 // __rte_pktmbuf_read(mbuf, off, sz, buf)
> mov x0, x19
> mov w2, #sz
> sub x3, x25, #stack_ofs
> mov x9, #<helper lo>
> movk x9, #<helper hi>
> blr x9
> mov x7, x0 // ptr = return value
> cbnz x7, load // non-NULL -> common tail
> mov x7, #0 // else return 0
> b epilogue
> load:
> ldr w7, [x7, xzr] // *(uint32_t *)ptr (size varies)
> rev32 x7, x7 // ntoh (size varies; omitted for BPF_B)
>
> For BPF_ABS the "add x9, x9, src" is omitted; the final load/byte-swap
> vary with the access size.
>
> Bugzilla ID: 1427
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Marat Khalili <marat.khalili@huawei.com>
> ---
> lib/bpf/bpf_jit_arm64.c | 169 +++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 168 insertions(+), 1 deletion(-)
>
> diff --git a/lib/bpf/bpf_jit_arm64.c b/lib/bpf/bpf_jit_arm64.c
> index 51906c7f0d..6d531dc83d 100644
> --- a/lib/bpf/bpf_jit_arm64.c
> +++ b/lib/bpf/bpf_jit_arm64.c
> @@ -1133,6 +1133,155 @@ emit_branch(struct a64_jit_ctx *ctx, uint8_t op, uint32_t i, int16_t off)
> emit_b_cond(ctx, ebpf_to_a64_cond(op), jump_offset_get(ctx, i, off));
> }
>
> +/* LD_ABS/LD_IND code block offsets (in arm64 instructions) */
> +enum {
> + LDMB_FAST_OFS, /* fast path */
> + LDMB_SLOW_OFS, /* slow path */
> + LDMB_FIN_OFS, /* common tail */
> + LDMB_OFS_NUM
> +};
> +
> +/*
> + * Helper for emit_ld_mbuf(): fast path.
> + * Compute the packet offset; if it lies inside the first segment leave the
> + * data pointer in R0, otherwise branch to the slow path.
> + */
> +static void
> +emit_ldmb_fast_path(struct a64_jit_ctx *ctx, uint8_t src, uint8_t mode,
> + uint32_t sz, int32_t imm, const uint32_t ofs[LDMB_OFS_NUM])
> +{
> + uint8_t r0 = ebpf_to_a64_reg(ctx, EBPF_REG_0);
> + uint8_t r6 = ebpf_to_a64_reg(ctx, EBPF_REG_6);
> + uint8_t tmp1 = ebpf_to_a64_reg(ctx, TMP_REG_1);
> + uint8_t tmp2 = ebpf_to_a64_reg(ctx, TMP_REG_2);
> + uint8_t tmp3 = ebpf_to_a64_reg(ctx, TMP_REG_3);
> +
> + /* off = imm (+ src for BPF_IND) */
> + emit_mov_imm(ctx, 1, tmp1, imm);
> + if (mode == BPF_IND)
> + emit_add(ctx, 1, tmp1, src);
> +
> + /*
> + * A negative effective offset (src can be < 0 for BPF_IND) would pass
> + * the signed check below and read before the segment, so route it to
> + * the slow path, which rejects it via the uint32_t bound on off.
> + */
> + emit_cmp(ctx, 1, tmp1, A64_ZR);
> + emit_b_cond(ctx, A64_MI, (int32_t)(ofs[LDMB_SLOW_OFS] - ctx->idx));
> +
> + /* if ((int64_t)(mbuf->data_len - off) < sz) goto slow_path */
> + emit_mov_imm(ctx, 1, tmp2, offsetof(struct rte_mbuf, data_len));
> + emit_ldr(ctx, BPF_H, tmp2, r6, tmp2);
> + emit_sub(ctx, 1, tmp2, tmp1);
> + emit_mov_imm(ctx, 1, tmp3, sz);
> + emit_cmp(ctx, 1, tmp2, tmp3);
> + emit_b_cond(ctx, A64_LT, (int32_t)(ofs[LDMB_SLOW_OFS] - ctx->idx));
> +
> + /* R0 = mbuf->buf_addr + mbuf->data_off + off */
> + emit_mov_imm(ctx, 1, tmp2, offsetof(struct rte_mbuf, data_off));
> + emit_ldr(ctx, BPF_H, tmp2, r6, tmp2);
> + emit_mov_imm(ctx, 1, r0, offsetof(struct rte_mbuf, buf_addr));
> + emit_ldr(ctx, EBPF_DW, r0, r6, r0);
> + emit_add(ctx, 1, r0, tmp2);
> + emit_add(ctx, 1, r0, tmp1);
> +
> + emit_b(ctx, (int32_t)(ofs[LDMB_FIN_OFS] - ctx->idx));
> +}
> +
> +/*
> + * Helper for emit_ld_mbuf(): slow path.
> + * R0 = __rte_pktmbuf_read(mbuf, off, sz, buf); return 0 if NULL.
> + * The scratch buffer is the space reserved by __rte_bpf_validate() at the
> + * bottom of the eBPF stack frame, i.e. (frame_pointer - stack_ofs).
> + */
> +static void
> +emit_ldmb_slow_path(struct a64_jit_ctx *ctx, uint32_t sz, uint32_t stack_ofs)
> +{
> + uint8_t r0 = ebpf_to_a64_reg(ctx, EBPF_REG_0);
> + uint8_t r6 = ebpf_to_a64_reg(ctx, EBPF_REG_6);
> + uint8_t fp = ebpf_to_a64_reg(ctx, EBPF_FP);
> + uint8_t tmp1 = ebpf_to_a64_reg(ctx, TMP_REG_1);
> + uint8_t tmp2 = ebpf_to_a64_reg(ctx, TMP_REG_2);
> +
> + /*
> + * __rte_pktmbuf_read() takes a uint32_t off, so a 64-bit off that does
> + * not fit would be silently truncated. Return 0 if it is out of range;
> + * this also catches the negative off routed here by the fast path.
> + */
> + emit_mov_imm(ctx, 1, tmp2, UINT32_MAX);
> + emit_cmp(ctx, 1, tmp1, tmp2);
> + emit_b_cond(ctx, A64_LS, 3); /* off <= UINT32_MAX: do the call */
> + emit_mov_imm(ctx, 1, r0, 0);
> + emit_b(ctx, (ctx->program_start + ctx->program_sz) - ctx->idx);
> +
> + /* arguments of __rte_pktmbuf_read(mbuf, off, len, buf) */
> + emit_mov_64(ctx, A64_R(1), tmp1); /* off (held in tmp1) */
> + emit_mov_64(ctx, A64_R(0), r6); /* mbuf */
> + emit_mov_imm(ctx, 0, A64_R(2), sz); /* len */
> + emit_sub_imm_64(ctx, A64_R(3), fp, stack_ofs); /* buf */
> +
> + emit_call(ctx, tmp1, (void *)(uintptr_t)__rte_pktmbuf_read);
> + emit_return_zero_if_src_zero(ctx, 1, r0);
> +}
> +
> +/*
> + * Helper for emit_ld_mbuf(): common tail.
> + * Load the value pointed to by R0 and convert from network byte order.
> + */
> +static void
> +emit_ldmb_fin(struct a64_jit_ctx *ctx, uint8_t opsz, uint32_t sz)
> +{
> + uint8_t r0 = ebpf_to_a64_reg(ctx, EBPF_REG_0);
> +
> + emit_ldr(ctx, opsz, r0, r0, A64_ZR);
> + if (opsz != BPF_B)
> + emit_be(ctx, r0, sz * 8);
> +}
> +
> +/*
> + * Emit code for BPF_LD | BPF_ABS and BPF_LD | BPF_IND packet loads:
> + *
> + * off = imm (+ src for BPF_IND)
> + * if (off >= 0 && mbuf->data_len - off >= sz) -- fast path
> + * ptr = mbuf->buf_addr + mbuf->data_off + off;
> + * else -- slow path
> + * if ((uint64_t)off > UINT32_MAX)
> + * return 0;
> + * ptr = __rte_pktmbuf_read(mbuf, off, sz, buf);
> + * if (ptr == NULL)
> + * return 0;
> + * R0 = ntoh(*(size *)ptr); -- common tail
> + *
> + * The three blocks are sized in a dry run so the forward branches can be
> + * resolved, then emitted for real (arm64 instructions are fixed width, so
> + * the dry run reproduces the real instruction count exactly).
> + */
> +static void
> +emit_ld_mbuf(struct a64_jit_ctx *ctx, uint8_t op, uint8_t src, int32_t imm,
> + uint32_t stack_ofs)
> +{
> + uint8_t mode = BPF_MODE(op);
> + uint8_t opsz = BPF_SIZE(op);
> + uint32_t sz = bpf_size(opsz);
> + uint32_t ofs[LDMB_OFS_NUM];
> +
> + /* seed offsets so the dry-run branches stay in range */
> + ofs[LDMB_FAST_OFS] = ofs[LDMB_SLOW_OFS] = ofs[LDMB_FIN_OFS] = ctx->idx;
> +
> + /* dry run to record block offsets */
> + emit_ldmb_fast_path(ctx, src, mode, sz, imm, ofs);
> + ofs[LDMB_SLOW_OFS] = ctx->idx;
> + emit_ldmb_slow_path(ctx, sz, stack_ofs);
> + ofs[LDMB_FIN_OFS] = ctx->idx;
> + emit_ldmb_fin(ctx, opsz, sz);
> +
> + /* rewind and emit for real with resolved offsets */
> + ctx->idx = ofs[LDMB_FAST_OFS];
> + emit_ldmb_fast_path(ctx, src, mode, sz, imm, ofs);
> + emit_ldmb_slow_path(ctx, sz, stack_ofs);
> + emit_ldmb_fin(ctx, opsz, sz);
> +}
> +
> static void
> check_program_has_call(struct a64_jit_ctx *ctx, struct rte_bpf *bpf)
> {
> @@ -1145,8 +1294,17 @@ check_program_has_call(struct a64_jit_ctx *ctx, struct rte_bpf *bpf)
> op = ins->code;
>
> switch (op) {
> - /* Call imm */
> + /*
> + * BPF_ABS/BPF_IND can fall through to __rte_pktmbuf_read(),
> + * so they need the call-clobbered register layout as well.
> + */
> case (BPF_JMP | EBPF_CALL):
> + case (BPF_LD | BPF_ABS | BPF_B):
> + case (BPF_LD | BPF_ABS | BPF_H):
> + case (BPF_LD | BPF_ABS | BPF_W):
> + case (BPF_LD | BPF_IND | BPF_B):
> + case (BPF_LD | BPF_IND | BPF_H):
> + case (BPF_LD | BPF_IND | BPF_W):
> ctx->foundcall = 1;
> return;
> }
> @@ -1348,6 +1506,15 @@ emit(struct a64_jit_ctx *ctx, struct rte_bpf *bpf)
> emit_mov_imm(ctx, 1, dst, u64);
> i++;
> break;
> + /* R0 = ntoh(*(size *)(mbuf data + (src) + imm)) */
> + case (BPF_LD | BPF_ABS | BPF_B):
> + case (BPF_LD | BPF_ABS | BPF_H):
> + case (BPF_LD | BPF_ABS | BPF_W):
> + case (BPF_LD | BPF_IND | BPF_B):
> + case (BPF_LD | BPF_IND | BPF_H):
> + case (BPF_LD | BPF_IND | BPF_W):
> + emit_ld_mbuf(ctx, op, src, imm, bpf->stack_sz);
> + break;
> /* *(size *)(dst + off) = src */
> case (BPF_STX | BPF_MEM | BPF_B):
> case (BPF_STX | BPF_MEM | BPF_H):
> --
> 2.53.0
^ permalink raw reply
* [PATCH v7 6/6] eal: fix async IPC callback not fired when no peers
From: Anatoly Burakov @ 2026-06-26 10:34 UTC (permalink / raw)
To: dev, Jianfeng Tan
In-Reply-To: <cover.1782469943.git.anatoly.burakov@intel.com>
Currently, when rte_mp_request_async() is called and no peer processes
are connected (nb_sent == 0), the user callback is never invoked.
The original implementation used a dedicated background thread and
pthread_cond_signal() to wake it after queuing the dummy request. When
that thread was replaced with per-message alarms, no alarm was set for
the dummy request, silently breaking the nb_sent == 0 path.
This was not noticed because async requests are usually used while handling
secondary process requests, where peers are typically already present.
Fix it by setting a 1us alarm on the dummy request, so the callback path
immediately triggers and processes it.
Fixes: daf9bfca717e ("ipc: remove thread for async requests")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_proc.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/lib/eal/common/eal_common_proc.c b/lib/eal/common/eal_common_proc.c
index 8902159386..b0f7681547 100644
--- a/lib/eal/common/eal_common_proc.c
+++ b/lib/eal/common/eal_common_proc.c
@@ -1200,11 +1200,22 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
ret = mp_request_async(eal_mp_socket_path(), copy, param, ts);
- /* if we didn't send anything, put dummy request on the queue */
+ /* if we didn't send anything, put dummy request on the queue
+ * and set a minimum-delay alarm so the callback fires immediately.
+ */
if (ret == 0 && reply->nb_sent == 0) {
TAILQ_INSERT_TAIL(&pending_requests.requests, dummy,
next);
dummy_used = true;
+ if (rte_eal_alarm_set(1, async_reply_handle,
+ (void *)(uintptr_t)dummy->id) < 0) {
+ EAL_LOG(ERR, "Fail to set alarm for dummy request");
+ /* roll back the changes */
+ TAILQ_REMOVE(&pending_requests.requests, dummy, next);
+ dummy_used = false;
+ ret = -1;
+ goto unlock_fail;
+ }
}
pthread_mutex_unlock(&pending_requests.lock);
@@ -1273,10 +1284,22 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
reply->nb_sent = 0;
}
- /* if we didn't send anything, put dummy request on the queue */
+ /* if we didn't send anything, put dummy request on the queue
+ * and set a minimum-delay alarm so the callback fires immediately.
+ */
if (ret == 0 && reply->nb_sent == 0) {
TAILQ_INSERT_HEAD(&pending_requests.requests, dummy, next);
dummy_used = true;
+
+ if (rte_eal_alarm_set(1, async_reply_handle,
+ (void *)(uintptr_t)dummy->id) < 0) {
+ EAL_LOG(ERR, "Fail to set alarm for dummy request");
+ /* roll back the changes */
+ TAILQ_REMOVE(&pending_requests.requests, dummy, next);
+ dummy_used = false;
+ ret = -1;
+ goto closedir_fail;
+ }
}
/* finally, unlock the queue */
--
2.47.3
^ permalink raw reply related
* [PATCH v7 5/6] eal: fix memory leak in async IPC secondary path
From: Anatoly Burakov @ 2026-06-26 10:34 UTC (permalink / raw)
To: dev, Jianfeng Tan
In-Reply-To: <cover.1782469943.git.anatoly.burakov@intel.com>
When rte_mp_request_async() succeeds on the secondary process path, the
dummy request is freed only if it was inserted into the queue. However,
when the actual request was sent successfully (nb_sent > 0), the dummy is
not used and the function returns without freeing it.
Free dummy before returning on the success path when it was not inserted
into the queue.
Fixes: f05e26051c15 ("eal: add IPC asynchronous request")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_proc.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/eal/common/eal_common_proc.c b/lib/eal/common/eal_common_proc.c
index 0cffc7a127..8902159386 100644
--- a/lib/eal/common/eal_common_proc.c
+++ b/lib/eal/common/eal_common_proc.c
@@ -1212,6 +1212,8 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
/* if we couldn't send anything, clean up */
if (ret != 0)
goto fail;
+ if (!dummy_used)
+ free(dummy);
return 0;
}
--
2.47.3
^ permalink raw reply related
* [PATCH v7 4/6] eal: fix async IPC memory leaks on partial failure
From: Anatoly Burakov @ 2026-06-26 10:33 UTC (permalink / raw)
To: dev, Jianfeng Tan
In-Reply-To: <cover.1782469943.git.anatoly.burakov@intel.com>
When rte_mp_request_async() fails to send requests to all peers,
copy and param can lose ownership and leak.
However, we cannot simply free them unconditionally, as "partial failure"
means some requests were already queued and thus still reference `copy` and
`param`, so freeing them directly on the error path can cause
use-after-free when those requests are later handled by the async timeout.
Fix this by rolling back queued requests from the current batch, and reset
nb_sent to 0. Freeing the requests is now safe even if some requests were
sent, as any responses or timeouts will not find the request ID in the
queue and will safely exit without doing anything.
Coverity issue: 501503
Fixes: f05e26051c15 ("eal: add IPC asynchronous request")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_proc.c | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/lib/eal/common/eal_common_proc.c b/lib/eal/common/eal_common_proc.c
index 991bf215a3..0cffc7a127 100644
--- a/lib/eal/common/eal_common_proc.c
+++ b/lib/eal/common/eal_common_proc.c
@@ -1245,6 +1245,32 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
} else if (mp_request_async(path, copy, param, ts))
ret = -1;
}
+
+ /*
+ * On partial failure, roll back all queued requests. We hold the lock
+ * so no one else touches the queue. All requests in this batch share
+ * the same param pointer. Stale alarms will fire and harmlessly find
+ * nothing via ID-based lookup.
+ */
+ if (ret != 0 && reply->nb_sent > 0) {
+ struct pending_request *r, *next;
+
+ for (r = TAILQ_FIRST(&pending_requests.requests);
+ r != NULL; r = next) {
+ next = TAILQ_NEXT(r, next);
+ if (r->type == REQUEST_TYPE_ASYNC &&
+ r->async.param == param) {
+ TAILQ_REMOVE(&pending_requests.requests,
+ r, next);
+ free(r->reply);
+ /* r->request == copy, freed below after the loop */
+ free(r);
+ }
+ }
+ /* requests on the queue were removed so keep things consistent */
+ reply->nb_sent = 0;
+ }
+
/* if we didn't send anything, put dummy request on the queue */
if (ret == 0 && reply->nb_sent == 0) {
TAILQ_INSERT_HEAD(&pending_requests.requests, dummy, next);
@@ -1263,6 +1289,11 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
/* if dummy was unused, free it */
if (!dummy_used)
free(dummy);
+ /* if nothing was sent, nobody owns copy/param */
+ if (ret != 0) {
+ free(param);
+ free(copy);
+ }
return ret;
closedir_fail:
--
2.47.3
^ permalink raw reply related
* [PATCH v7 3/6] eal: avoid deadlock in async IPC alarm callback
From: Anatoly Burakov @ 2026-06-26 10:33 UTC (permalink / raw)
To: dev, Jianfeng Tan
In-Reply-To: <cover.1782469943.git.anatoly.burakov@intel.com>
async_reply_handle_thread_unsafe() can run while holding
pending_requests.lock and currently calls rte_eal_alarm_cancel().
rte_eal_alarm_cancel() may spin-wait for an executing callback, which can
deadlock if that callback is blocked on the same lock.
Remove callback-side alarm cancellation. It is safe to do so, because any
callback triggered without a pending request becomes a noop due to the
async request lookup now using numerical ID.
Fixes: daf9bfca717e ("ipc: remove thread for async requests")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_proc.c | 13 -------------
1 file changed, 13 deletions(-)
diff --git a/lib/eal/common/eal_common_proc.c b/lib/eal/common/eal_common_proc.c
index 347a174d22..991bf215a3 100644
--- a/lib/eal/common/eal_common_proc.c
+++ b/lib/eal/common/eal_common_proc.c
@@ -549,19 +549,6 @@ async_reply_handle_thread_unsafe(struct pending_request *req)
TAILQ_REMOVE(&pending_requests.requests, req, next);
- if (rte_eal_alarm_cancel(async_reply_handle,
- (void *)(uintptr_t)req->id) < 0) {
- /* if we failed to cancel the alarm because it's already in
- * progress, don't proceed because otherwise we will end up
- * handling the same message twice.
- */
- if (rte_errno == EINPROGRESS) {
- EAL_LOG(DEBUG, "Request handling is already in progress");
- goto no_trigger;
- }
- EAL_LOG(ERR, "Failed to cancel alarm");
- }
-
if (action == ACTION_TRIGGER)
return req;
no_trigger:
--
2.47.3
^ permalink raw reply related
* [PATCH v7 2/6] eal: use request ID instead of pointers
From: Anatoly Burakov @ 2026-06-26 10:33 UTC (permalink / raw)
To: dev, Jianfeng Tan
In-Reply-To: <cover.1782469943.git.anatoly.burakov@intel.com>
Initial implementation of async IPC request handling was using request
pointers directly. Because of the nature of how IPC is meant to work and
that requests ownership is disconnected from their creation (as in, freeing
a request may happen due to timeout, or due to received response, or due
to rollback because of a later failure), using pointers as identity is not
safe.
Use numeric request ID for async request lookup instead. This way, we can
safely free requests even if we are already waiting on responses/timeouts
for them, as the pointers themselves will not be referenced directly by
the response/timeout.
Fixes: f05e26051c15 ("eal: add IPC asynchronous request")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_proc.c | 66 ++++++++++++++++++++++----------
1 file changed, 46 insertions(+), 20 deletions(-)
diff --git a/lib/eal/common/eal_common_proc.c b/lib/eal/common/eal_common_proc.c
index 799c6e81b0..347a174d22 100644
--- a/lib/eal/common/eal_common_proc.c
+++ b/lib/eal/common/eal_common_proc.c
@@ -74,6 +74,7 @@ struct async_request_param {
struct pending_request {
TAILQ_ENTRY(pending_request) next;
+ unsigned long id;
enum {
REQUEST_TYPE_SYNC,
REQUEST_TYPE_ASYNC
@@ -92,6 +93,8 @@ struct pending_request {
};
};
+static unsigned long next_request_id;
+
TAILQ_HEAD(pending_request_list, pending_request);
static struct {
@@ -111,15 +114,15 @@ mp_send(struct rte_mp_msg *msg, const char *peer, int type);
static void
async_reply_handle(void *arg);
-/* for use with process_msg */
+/* for use with alarm callback and process_msg */
static struct pending_request *
-async_reply_handle_thread_unsafe(void *arg);
+async_reply_handle_thread_unsafe(struct pending_request *req);
static void
trigger_async_action(struct pending_request *req);
static struct pending_request *
-find_pending_request(const char *dst, const char *act_name)
+find_request_by_name(const char *dst, const char *act_name)
{
struct pending_request *r;
@@ -132,6 +135,19 @@ find_pending_request(const char *dst, const char *act_name)
return r;
}
+static struct pending_request *
+find_async_request_by_id(unsigned long id)
+{
+ struct pending_request *r;
+
+ TAILQ_FOREACH(r, &pending_requests.requests, next) {
+ if (r->id == id && r->type == REQUEST_TYPE_ASYNC)
+ return r;
+ }
+
+ return NULL;
+}
+
/*
* Combine prefix and name(optional) to return unix domain socket path
* return the number of characters that would have been put into buffer.
@@ -354,7 +370,7 @@ process_msg(struct mp_msg_internal *m, struct sockaddr_un *s)
struct pending_request *req = NULL;
pthread_mutex_lock(&pending_requests.lock);
- pending_req = find_pending_request(s->sun_path, msg->name);
+ pending_req = find_request_by_name(s->sun_path, msg->name);
if (pending_req) {
memcpy(pending_req->reply, msg, sizeof(*msg));
/* -1 indicates that we've been asked to ignore */
@@ -519,9 +535,8 @@ trigger_async_action(struct pending_request *sr)
}
static struct pending_request *
-async_reply_handle_thread_unsafe(void *arg)
+async_reply_handle_thread_unsafe(struct pending_request *req)
{
- struct pending_request *req = (struct pending_request *)arg;
enum async_action action;
struct timespec ts_now;
@@ -534,7 +549,8 @@ async_reply_handle_thread_unsafe(void *arg)
TAILQ_REMOVE(&pending_requests.requests, req, next);
- if (rte_eal_alarm_cancel(async_reply_handle, req) < 0) {
+ if (rte_eal_alarm_cancel(async_reply_handle,
+ (void *)(uintptr_t)req->id) < 0) {
/* if we failed to cancel the alarm because it's already in
* progress, don't proceed because otherwise we will end up
* handling the same message twice.
@@ -557,9 +573,16 @@ static void
async_reply_handle(void *arg)
{
struct pending_request *req;
+ /* alarm arg carries the request ID packed into a void * via uintptr_t */
+ unsigned long id = (uintptr_t)arg;
+
+ /* ensure request ID matches pointer size */
+ RTE_BUILD_BUG_ON(sizeof(next_request_id) != sizeof(uintptr_t));
pthread_mutex_lock(&pending_requests.lock);
- req = async_reply_handle_thread_unsafe(arg);
+ req = find_async_request_by_id(id);
+ if (req != NULL)
+ req = async_reply_handle_thread_unsafe(req);
pthread_mutex_unlock(&pending_requests.lock);
if (req != NULL)
@@ -878,8 +901,18 @@ mp_request_async(const char *dst, struct rte_mp_msg *req,
{
struct rte_mp_msg *reply_msg;
struct pending_request *pending_req, *exist;
+ unsigned long id;
int ret = -1;
+ /* queue already locked by caller */
+
+ exist = find_request_by_name(dst, req->name);
+ if (exist) {
+ EAL_LOG(ERR, "A pending request %s:%s", dst, req->name);
+ rte_errno = EEXIST;
+ return -1;
+ }
+
pending_req = calloc(1, sizeof(*pending_req));
reply_msg = calloc(1, sizeof(*reply_msg));
if (pending_req == NULL || reply_msg == NULL) {
@@ -889,22 +922,14 @@ mp_request_async(const char *dst, struct rte_mp_msg *req,
goto fail;
}
+ id = ++next_request_id;
pending_req->type = REQUEST_TYPE_ASYNC;
+ pending_req->id = id;
strlcpy(pending_req->dst, dst, sizeof(pending_req->dst));
pending_req->request = req;
pending_req->reply = reply_msg;
pending_req->async.param = param;
- /* queue already locked by caller */
-
- exist = find_pending_request(dst, req->name);
- if (exist) {
- EAL_LOG(ERR, "A pending request %s:%s", dst, req->name);
- rte_errno = EEXIST;
- ret = -1;
- goto fail;
- }
-
ret = send_msg(dst, req, MP_REQ);
if (ret < 0) {
EAL_LOG(ERR, "Fail to send request %s:%s",
@@ -919,7 +944,7 @@ mp_request_async(const char *dst, struct rte_mp_msg *req,
/* if alarm set fails, we simply ignore the reply */
if (rte_eal_alarm_set(ts->tv_sec * 1000000 + ts->tv_nsec / 1000,
- async_reply_handle, pending_req) < 0) {
+ async_reply_handle, (void *)(uintptr_t)id) < 0) {
EAL_LOG(ERR, "Fail to set alarm for request %s:%s",
dst, req->name);
ret = -1;
@@ -952,7 +977,7 @@ mp_request_sync(const char *dst, struct rte_mp_msg *req,
pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);
pthread_cond_init(&pending_req.sync.cond, &attr);
- exist = find_pending_request(dst, req->name);
+ exist = find_request_by_name(dst, req->name);
if (exist) {
EAL_LOG(ERR, "A pending request %s:%s", dst, req->name);
rte_errno = EEXIST;
@@ -1178,6 +1203,7 @@ rte_mp_request_async(struct rte_mp_msg *req, const struct timespec *ts,
* it, and put it on the queue if we don't send any requests.
*/
dummy->type = REQUEST_TYPE_ASYNC;
+ dummy->id = ++next_request_id;
dummy->request = copy;
dummy->reply = NULL;
dummy->async.param = param;
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox