* [v3 1/1] net/hinic3: Fix VXLAN TSO issue
From: Feifei Wang @ 2026-05-26 3:31 UTC (permalink / raw)
To: dev; +Cc: Feifei Wang
In-Reply-To: <20260526033147.1045-1-wff_light@vip.163.com>
From: Feifei Wang <wangfeifei40@huawei.com>
VXLAN TSO lacks a flag bit, causing the processing function
to determine that the hardware does not support it, leading
to improper handling.
Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index f4eb788..5071b64 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -652,6 +652,8 @@ hinic3_dev_configure(struct rte_eth_dev *dev)
static void
hinic3_dev_tnl_tso_support(struct rte_eth_dev_info *info, struct hinic3_nic_dev *nic_dev)
{
+ if (HINIC3_SUPPORT_VXLAN_OFFLOAD(nic_dev))
+ info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO;
if (HINIC3_SUPPORT_GENEVE_OFFLOAD(nic_dev))
info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
if (HINIC3_SUPPORT_IPXIP_OFFLOAD(nic_dev))
@@ -698,7 +700,6 @@ hinic3_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
RTE_ETH_TX_OFFLOAD_TCP_TSO | RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
- if (nic_dev->feature_cap & NIC_F_HTN_CMDQ)
hinic3_dev_tnl_tso_support(info, nic_dev);
info->hash_key_size = HINIC3_RSS_KEY_SIZE;
--
2.47.0.windows.2
^ permalink raw reply related
* [v3 0/1] net/hinic3: Fix VXLAN TSO issue
From: Feifei Wang @ 2026-05-26 3:31 UTC (permalink / raw)
To: dev
In-Reply-To: <20260520065817.931-2-wff_light@vip.163.com>
v1: The RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO flag is added to support the VXLAN TSO function
v2: Modify the commit information issue and supplement the commit information
v3: Revise review comments. First, deterine whether the hardware supports it, then add the flag bit.
Feifei Wang (1):
net/hinic3: Fix VXLAN TSO issue
drivers/net/hinic3/hinic3_ethdev.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
2.47.0.windows.2
^ permalink raw reply
* Re: [PATCH] eal/riscv: implement prefetch using __builtin_prefetch
From: sunyuechi @ 2026-05-26 3:30 UTC (permalink / raw)
To: shiwei dang; +Cc: dev, stanislaw.kardach, stephen, david.marchand
In-Reply-To: <CAF-WEyHKBEwYW_E6UvjKwEL7LQO4pfFS7qtv-3gjz1B7c5cNYQ@mail.gmail.com>
On 3/29/26 12:38 AM, shiwei dang wrote:
> Hi ,
>
> Thanks for the review. I'll address both points (real name in
> Signed-off-by and test results) together in v2 once I have access to
> physical RISC-V hardware.
>
> Best regards, Dang Shiwei
>
>
Hi,
Gentle ping — any update on v2?
^ permalink raw reply
* Re: [RFC 0/3] lib/fastmem: fast small-object allocator
From: Stephen Hemminger @ 2026-05-25 22:18 UTC (permalink / raw)
To: Mattias Rönnblom
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <af3a4738-df30-43c4-b3d9-5ec1912a307a@lysator.liu.se>
On Mon, 25 May 2026 21:39:20 +0200
Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> On 5/25/26 16:30, Stephen Hemminger wrote:
> > On Mon, 25 May 2026 12:36:39 +0200
> > Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> >
> >> This RFC introduces fastmem, a general-purpose small-object allocator
> >> for DPDK. It is intended to replace per-type mempools with a single
> >> allocator that handles arbitrary sizes, grows on demand, and matches
> >> mempool-level performance on the hot path.
> >
> > Makes sense, what a simple wrapper inline to allow full replacement
> > testing/performance A/B comparison?
>
> Do you mean a mempool or a heap wrapper? Or both?
>
> I haven't looked into what options there are with mempools. A mempool
> driver should be possible, but then I guess one might attempt a
> whole-sale mempool-compatible API as well.
My thinking is a yet another allocator in DPDK is just another source
of confusion and bugs. BUT if it can consolidate and fully replace
one or more existing allocators then it would be great improvement.
Mempools are fast, but fixed and space inefficient.
Rte_malloc is slow, but flexible.
Also, need to make whatever is added play well with static
and dynamic checkers.
^ permalink raw reply
* Re: [PATCH] dma/ae4dma: add AMD AE4DMA DMA PMD
From: Ningoji, Raghavendra @ 2026-05-25 19:57 UTC (permalink / raw)
To: david.marchand@redhat.com
Cc: Modali, Bhagyada, Sebastian, Selwin, dev@dpdk.org,
fengchengwen@huawei.com, Ningoji, Raghavendra, rjarry@redhat.com,
thomas@monjalon.net
[-- Attachment #1: Type: text/plain, Size: 656 bytes --]
Hi David,
Thanks for the review.
- Please fix the below warnings raised by checkpatches.sh, and run this script before submitting a new revision
- Please also checks the copyright years.
- Globally in those changes, rte_iova_t should probably be used instead of phys_addr_t.
>>All the review comments have been addressed in v2 series . Now all the patches are clean and clears patch checks.
- The patch is big, splitting it into logical patches introducing one feature at a time would help.
>>As you suggested to refactor the v1 patch into logical multiple patches, the v2 patch has been split into 3 patches.
Thanks,
Raghavendra
[-- Attachment #2: Type: text/html, Size: 3102 bytes --]
^ permalink raw reply
* Re: [RFC 0/3] lib/fastmem: fast small-object allocator
From: Mattias Rönnblom @ 2026-05-25 19:43 UTC (permalink / raw)
To: Stephen Hemminger
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <20260525113648.5bf540ca@phoenix.local>
On 5/25/26 20:36, Stephen Hemminger wrote:
> On Mon, 25 May 2026 12:36:39 +0200
> Mattias Rönnblom <hofors@lysator.liu.se> wrote:
>
>> This RFC introduces fastmem, a general-purpose small-object allocator
>> for DPDK. It is intended to replace per-type mempools with a single
>> allocator that handles arbitrary sizes, grows on demand, and matches
>> mempool-level performance on the hot path.
>>
>> Motivation
>> ----------
>>
>> DPDK applications commonly maintain many mempools — one per object
>> type (connections, sessions, timers, work items). Each must be sized
>> up front, wastes memory when over-provisioned, and cannot serve
>> objects of a different size. Fastmem eliminates this by accepting
>> arbitrary sizes at runtime, backed by a slab allocator that
>> repurposes memory across size classes as demand shifts.
>>
>> Design
>> ------
>>
>> Three-layer architecture:
>>
>> 1. Backing memory: 128 MiB IOVA-contiguous memzones from EAL,
>> reserved lazily (or pre-reserved for deterministic latency).
>>
>> 2. Slabs: 2 MiB, 2 MiB-aligned regions carved from memzones.
>> The alignment enables O(1) slab lookup from any object pointer
>> via bitmask — no radix tree or index structure. Slabs move
>> freely between 18 power-of-2 size classes (8 B to 1 MiB).
>>
>> 3. Per-lcore caches: bounded LIFO stacks (no locks on the hot
>> path). Cache misses trigger bulk transfers to/from the shared
>> bin under a spinlock.
>>
>> Key properties:
>>
>> - Zero per-object metadata in the production build.
>> - NUMA-aware, with per-socket bins and free-slab pools.
>> - DMA-usable memory with O(1) virt-to-IOVA translation.
>> - Bulk alloc/free with all-or-nothing semantics.
>> - Backing memory never returned during lifetime (slabs recycled).
>> - Non-EAL threads supported (bypass cache, take bin lock).
>>
>> API surface
>> -----------
>>
>> rte_fastmem_init / deinit
>> rte_fastmem_reserve
>> rte_fastmem_set_limit / get_limit
>> rte_fastmem_alloc / alloc_socket
>> rte_fastmem_alloc_bulk / alloc_bulk_socket
>> rte_fastmem_free / free_bulk
>> rte_fastmem_virt2iova
>> rte_fastmem_cache_flush
>> rte_fastmem_max_size / classes
>> rte_fastmem_stats / stats_class / stats_lcore / stats_lcore_class
>> rte_fastmem_stats_reset
>>
>> All APIs are marked __rte_experimental.
>>
>> Performance
>> -----------
>>
>> The single-object hot path is roughly 2-3x the cost of mempool
>> and an order of magnitude faster than rte_malloc. Under
>> multi-lcore contention, fastmem scales similarly to mempool,
>> while rte_malloc collapses.
>>
>> Limitations
>> -----------
>>
>> - Maximum allocation: 1 MiB. Larger requests should use rte_malloc.
>> - Power-of-2 classes only; worst-case internal fragmentation ~50%.
>> - Backing memory not reclaimable short of deinit.
>>
>> Future work
>> -----------
>>
>> - Lcore-affine allocations (false-sharing-free by construction).
>> - Mempool ops driver for transparent drop-in use.
>> - Pre-resolved allocator handle binding size class and socket,
>> eliminating per-call class lookup and enabling an inline
>> cache-hit fast path.
>> - Debug mode (cookies, double-free detection, poison-on-free).
>> - Telemetry integration.
>> - EAL integration, allowing EAL-internal subsystems to use
>> fastmem for their small-object allocations.
>>
>> Mattias Rönnblom (3):
>> doc: add fastmem programming guide
>> lib: add fastmem library
>> app/test: add fastmem test suite
>>
>> app/test/meson.build | 3 +
>> app/test/test_fastmem.c | 1682 +++++++++++++++++++++++++
>> app/test/test_fastmem_perf.c | 997 +++++++++++++++
>> app/test/test_fastmem_profile.c | 157 +++
>> doc/api/doxy-api-index.md | 1 +
>> doc/api/doxy-api.conf.in | 1 +
>> doc/guides/prog_guide/fastmem_lib.rst | 301 +++++
>> doc/guides/prog_guide/index.rst | 1 +
>> lib/fastmem/meson.build | 6 +
>> lib/fastmem/rte_fastmem.c | 1486 ++++++++++++++++++++++
>> lib/fastmem/rte_fastmem.h | 644 ++++++++++
>> lib/meson.build | 1 +
>> 12 files changed, 5280 insertions(+)
>> create mode 100644 app/test/test_fastmem.c
>> create mode 100644 app/test/test_fastmem_perf.c
>> create mode 100644 app/test/test_fastmem_profile.c
>> create mode 100644 doc/guides/prog_guide/fastmem_lib.rst
>> create mode 100644 lib/fastmem/meson.build
>> create mode 100644 lib/fastmem/rte_fastmem.c
>> create mode 100644 lib/fastmem/rte_fastmem.h
>>
>
> Largish patchset so did AI review with full claude model.
>
> Series review: [RFC 0/3] add fastmem allocator
> Reviewed against the v1 RFC posted 2026-05-25.
>
>
> [RFC 1/3] doc: add fastmem programming guide
>
> Info: doc/guides/prog_guide/fastmem_lib.rst -- "\ No newline at end of file"
> The new RST file does not end with a newline.
>
>
> [RFC 2/3] lib: add fastmem library
>
> Error: lib/fastmem/rte_fastmem.c -- use-after-free during rte_fastmem_deinit()
> when caches were allocated cross-socket.
>
> cache_create() places the cache struct on the *calling thread's* socket,
> not on the socket the cache serves:
>
> unsigned int own_socket = rte_socket_id();
> ...
> alloc_socket = &fastmem->sockets[own_socket];
> cache = bin_alloc_one(&alloc_socket->bins[cache_class]);
> ...
> *slot = cache; /* slot is in socket K's caches[][] */
>
> So an lcore on socket S that calls rte_fastmem_alloc_socket(..., K) with
> S != K creates a cache whose memory lives in socket S's memzone but is
> reachable through socket K's caches[lcore][class].
>
> rte_fastmem_deinit() then walks sockets in index order:
>
> for (i = 0; i < RTE_MAX_NUMA_NODES; i++)
> release_socket(&fastmem->sockets[i]);
>
> and release_socket() does, in this order:
>
> socket_release_caches(socket); /* (1) */
> for (c...) bin_release(&socket->bins[c], socket); /* (2) */
> for (i...) rte_memzone_free(socket->memzones[i]); /* (3) */
>
> When i = S, step (3) frees socket S's memzones. When i = K (K > S),
> socket_release_caches(K) runs:
>
> cache_slab = slab_of(cache); /* in socket S's freed mz */
> bin_free_one(cache_slab->bin, cache); /* reads cache_slab->bin */
>
> cache_slab points into a freed memzone, so cache_slab->bin and the
> subsequent push (slab->free_head = obj; slab->free_count++; in
> bin_push_locked()) read and write released memory. slab_release() may
> then re-attach the slab to socket S's free_head, which was zeroed and
> whose backing is gone.
>
> This is triggered by any application that allocates from a non-local
> socket via SOCKET_ID_ANY fallback or explicit socket_id, which the
> programming guide describes as a normal mode of operation. The
> existing test_alloc_socket and test_alloc_socket_numa_placement use
> rte_socket_id_by_idx(0) (the local socket) so the bug is not
> exercised by the test suite.
>
> Either order the teardown in three phases (all caches across all
> sockets first, then all bins, then all memzones), or allocate the
> cache struct from the socket it serves rather than the calling
> thread's socket.
>
> Warning: lib/fastmem/rte_fastmem.c -- non-atomic access to shared 64-bit
> statistics counters.
>
> cache->alloc_cache_hits, alloc_cache_misses, alloc_nomem,
> free_cache_hits, free_cache_misses, and the bin counters
> slab_acquires, slab_releases, slabs_partial, slabs_full are
> incremented as plain C reads/writes by the owning lcore and read
> from another thread via rte_fastmem_stats(), rte_fastmem_stats_class(),
> rte_fastmem_stats_lcore(), and rte_fastmem_stats_lcore_class(). On
> architectures where uint64_t is not naturally atomic (and per the C
> standard generally) this is a data race; even on x86-64 it is
> undefined behavior under -fsanitize=thread.
>
> Use rte_atomic_fetch_add_explicit() with rte_memory_order_relaxed on
> the producer side and rte_atomic_load_explicit() with relaxed
> ordering on the reader side. Per AGENTS.md / the DPDK convention,
> relaxed ordering is appropriate for these counters.
>
> Warning: lib/fastmem/rte_fastmem.c -- pointer publish in cache_create()
> without release ordering.
>
> *slot = cache;
> return cache;
>
> The struct fields (count, capacity, target, the stats counters) are
> written before this store but with no fence or release barrier. A
> concurrent stats reader doing socket->caches[l][c] followed by
> cache->* could observe the pointer but not all initialized fields.
> Even ignoring the stats reader, rte_fastmem_cache_flush() invoked
> from a different lcore on the same cache (not currently possible by
> API contract, but the field is technically reachable) would race.
> Pair with rte_atomic_store_explicit(..., rte_memory_order_release)
> and a matching acquire load on the reader path.
>
> Warning: lib/fastmem/rte_fastmem.c -- spurious ENOMEM window during slab
> release.
>
> bin_push_locked() removes a fully-drained slab from bin->partial
> before bin_free_one() drops the bin lock; slab_release() then puts
> it on socket->free_head under the socket lock. Between the unlock
> and slab_release(), another lcore allocating in any class on the
> same socket can see free_head == NULL, hit the memory_limit (or
> FASTMEM_MAX_MEMZONES_PER_SOCKET) check in grow_socket(), and return
> ENOMEM even though the slab is about to become available. Not a
> correctness issue but visible to applications that pin tightly to
> their limit.
>
> Info: lib/fastmem/rte_fastmem.c local_socket_id() final fallback:
>
> return (unsigned int)rte_socket_id_by_idx(0);
>
> rte_socket_id_by_idx() returns int and is documented to return -1 on
> error. If there are zero configured sockets the cast yields UINT_MAX
> and fastmem->sockets[UINT_MAX] is out of bounds. Realistically there
> is always at least one socket, but a defensive check (return 0, or
> fail allocation explicitly) would avoid the corner case.
>
> Info: lib/fastmem/rte_fastmem.c cache_pop() refills to cache->target
> (half capacity) rather than to capacity. Subsequent single-object
> allocs only get target-1 hits before the next bin trip. Likely
> intentional for fairness with bulk callers, but worth a comment.
>
> Info: lib/meson.build inserts 'fastmem' between 'dispatcher' and
> 'gpudev'. The natural alphabetical position is between 'efd' and
> 'fib'; fastmem has no dependency on dispatcher.
>
>
> [RFC 3/3] app/test: add fastmem test suite
>
> Warning: app/test/test_fastmem.c -- REGISTER_FAST_TEST uses NOHUGE_OK
> but the functional tests need real memzone-backed memory.
>
> REGISTER_FAST_TEST(fastmem_autotest, NOHUGE_OK, ASAN_OK,
> test_fastmem);
>
> test_fastmem runs both the lifecycle suite (no allocations) and the
> functional suite, which requests 128 MiB IOVA-contiguous memzones.
> In --no-huge mode IOVA-contiguous reservation of that size is not
> reliable, so NOHUGE_SKIP is more honest. If you want the lifecycle
> tests to remain no-huge-friendly, register them as a separate
> test command.
>
> Warning: app/test/test_fastmem.c -- the suite never exercises
> cross-socket cache allocation.
>
> test_alloc_socket and test_alloc_socket_numa_placement both use
> rte_socket_id_by_idx(0) (the local socket). Add a test that runs on
> a worker lcore whose rte_socket_id() differs from the target
> socket_id passed to rte_fastmem_alloc_socket(), then calls
> rte_fastmem_deinit(). This would have caught the deinit UAF above.
>
> Info: app/test/test_fastmem.c -- several test functions declare an
> uninitialized `int rc;` that is never read or written (e.g.
> test_alloc_too_big, test_alloc_invalid_align, test_alloc_free_small,
> test_alloc_alignment, test_alloc_socket, test_alloc_block_repurposing
> and others). Drop the declarations.
>
> Info: app/test/test_fastmem.c trailing blank-line clusters (two blank
> lines before "return TEST_SUCCESS;" in test_reserve_multiple_memzones,
> test_reserve_cumulative, test_reserve_invalid_socket,
> test_reserve_any_socket, test_alloc_too_big, ...). Drop the extra
> blank line.
Thanks. I've addressed the above issues and the fixes will be available
as an RFC v2, except:
#2 - Non-atomic stats counters
Diagnostic counters read cross-thread. On all DPDK-supported
architectures, aligned uint64_t stores are atomic in practice;
a torn read (e.g., on 32-bit x86) at worst yields a slightly
stale counter value. Not worth the ceremony.
#3 - Pointer publish without release ordering
On weakly-ordered architectures a stats reader could briefly see
uninitialized counter values for a newly-created cache. Acceptable
for diagnostic data.
#4 - Spurious ENOMEM window during slab release
Narrow timing window, not a correctness bug. Closing it would
require holding the bin lock across slab_release(), reintroducing
the contention the design avoids.
^ permalink raw reply
* Re: [RFC 0/3] lib/fastmem: fast small-object allocator
From: Mattias Rönnblom @ 2026-05-25 19:39 UTC (permalink / raw)
To: Stephen Hemminger
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <20260525073029.235b4d40@phoenix.local>
On 5/25/26 16:30, Stephen Hemminger wrote:
> On Mon, 25 May 2026 12:36:39 +0200
> Mattias Rönnblom <hofors@lysator.liu.se> wrote:
>
>> This RFC introduces fastmem, a general-purpose small-object allocator
>> for DPDK. It is intended to replace per-type mempools with a single
>> allocator that handles arbitrary sizes, grows on demand, and matches
>> mempool-level performance on the hot path.
>
> Makes sense, what a simple wrapper inline to allow full replacement
> testing/performance A/B comparison?
Do you mean a mempool or a heap wrapper? Or both?
I haven't looked into what options there are with mempools. A mempool
driver should be possible, but then I guess one might attempt a
whole-sale mempool-compatible API as well.
The role(s) fastmem could serve are:
a) An lcore/fast path small-object allocator when you don't know the
object size and/or count beforehand (i.e., what the cover letter says).
b) Do what mempools do and a.
c) Do what the rte_malloc heap does, but lcore/fast path-friendly. In
other words, option a but with larger objects too.
e) Something that's both b and c.
I haven't really formed an opinion yet, other than that option a seems
like a natural first step.
Fastmem is significantly slower than mempools for the moment. Claude
will tell you to inline, but that doesn't help (at least not in the
micro benchmarks). Then it will tell you to go remove the statistics,
which also doesn't help. (Latency is data dependency-driven, so stats
load/store/compute runs on resources that otherwise would have been idle.)
What does help however is pre-compute socket and bin-related info and
put into a handle, which the application may optionally use to quickly
retrieve objects of-a-certain-size/from-a-certain-socket. Still slower
than mempool though.
> === Scenario 1: Single-object hot path — cycles per (alloc + free) ===
> allocator 8 B 64 B 256 B 1024 B 4096 B
> fastmem 16.9 16.7 17.7 17.6 17.9
> fastmem_h 9.5 9.4 9.5 9.5 9.4
> mempool 6.9 6.9 6.9 7.0 6.6
> rte_malloc 93.7 93.8 94.8 100.1 130.0
> libc 118.8 119.2 20.4 20.4 111.0
>
> === Scenario 2: Batch alloc, FIFO free — cycles per alloc ===
> allocator 8 B 64 B 256 B 1024 B 4096 B
> fastmem 10.1 10.2 10.8 12.7 14.7
> fastmem_h 6.8 6.7 7.4 9.3 11.4
> mempool 4.2 4.1 4.1 4.1 4.1
> rte_malloc 58.6 58.5 62.1 67.5 68.5
> libc 104.8 104.6 73.7 203.9 1254.0
Intel(R) Xeon(R) Gold 6421N / Ubuntu 24.04 / clang
Best regards,
Mattias
^ permalink raw reply
* [PATCH v2 3/3] dma/ae4dma: add data path operations
From: Raghavendra Ningoji @ 2026-05-25 18:42 UTC (permalink / raw)
To: dev
Cc: Thomas Monjalon, Bhagyada Modali, Robin Jarry, Selwin.Sebastian,
david.marchand, Raghavendra Ningoji
In-Reply-To: <20260525184244.1758825-1-raghavendra.ningoji@amd.com>
Implement the dmadev fast path for the AMD AE4DMA PMD.
This commit adds:
- copy enqueue (rte_dma_copy): write an AE4DMA descriptor for a
memory-to-memory transfer; on RTE_DMA_OP_FLAG_SUBMIT the doorbell
is rung immediately.
- submit (rte_dma_submit): advance the per-queue write_idx
register to expose pending descriptors to the hardware.
- completion (rte_dma_completed / rte_dma_completed_status):
completion is detected via the hardware's per-queue read_idx
register, which the engine advances as it processes descriptors.
The descriptor status / err_code bytes are read only to classify
each drained slot as success or failure, and HW error codes are
translated to the dmadev RTE_DMA_STATUS_* enumeration.
- burst capacity (rte_dma_burst_capacity): report the number of
free descriptor slots, taking into account the one slot reserved
to distinguish full from empty on the power-of-two ring.
The fast path entry points are wired through fp_obj in
ae4dma_dmadev_create(). The fill capability is not advertised;
fp_obj->fill is left zero-initialised.
Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
doc/guides/dmadevs/ae4dma.rst | 22 +++
drivers/dma/ae4dma/ae4dma_dmadev.c | 288 +++++++++++++++++++++++++++++
2 files changed, 310 insertions(+)
diff --git a/doc/guides/dmadevs/ae4dma.rst b/doc/guides/dmadevs/ae4dma.rst
index a85c1d92ca..37a2096ccf 100644
--- a/doc/guides/dmadevs/ae4dma.rst
+++ b/doc/guides/dmadevs/ae4dma.rst
@@ -51,3 +51,25 @@ On probe the PMD performs the following steps for each PCI function:
IOVA-contiguous memory, programs the queue base address and ring
depth into the per-queue registers, and enables the queue.
* Interrupts are masked; completion is polled by the application.
+
+Usage
+-----
+
+Once a dmadev has been started, copies are submitted with
+``rte_dma_copy()`` and completions are reaped with ``rte_dma_completed()``
+or ``rte_dma_completed_status()``. See the
+:ref:`Enqueue / Dequeue API <dmadev_enqueue_dequeue>` section of the
+dmadev library documentation for details.
+
+Limitations
+-----------
+
+* Only memory-to-memory copies are supported. Fill, scatter-gather and
+ any other operation types are not advertised in
+ ``rte_dma_info::dev_capa``.
+* The maximum number of descriptors per virtual channel is fixed by
+ hardware at 32. The PMD rounds the requested ring size up to a
+ power of two and clamps it to 32.
+* Only a single virtual channel per dmadev is supported; use the 16
+ per-PCI-function dmadevs to obtain channel-level parallelism.
+* Interrupt-driven completion is not supported.
diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
index dfda723c13..0f223fc40c 100644
--- a/drivers/dma/ae4dma/ae4dma_dmadev.c
+++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
@@ -167,6 +167,73 @@ ae4dma_dev_close(struct rte_dma_dev *dev)
cmd_q->qbase_phys_addr = 0;
return 0;
}
+
+/* trigger h/w to process enqued desc:doorbell - by next_write */
+static inline void
+__submit(struct ae4dma_dmadev *ae4dma)
+{
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t write_idx = cmd_q->next_write;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->write_idx, write_idx);
+ if (nb != 0)
+ cmd_q->stats.submitted += (uint16_t)((cmd_q->next_write - cmd_q->last_write +
+ nb) % nb);
+ cmd_q->last_write = cmd_q->next_write;
+}
+
+static int
+ae4dma_submit(void *dev_private, uint16_t vchan __rte_unused)
+{
+ struct ae4dma_dmadev *ae4dma = dev_private;
+
+ __submit(ae4dma);
+ return 0;
+}
+
+/* Write descriptor for enqueue (copy only). */
+static inline int
+__write_desc_copy(void *dev_private, rte_iova_t src, rte_iova_t dst,
+ uint32_t len, uint64_t flags)
+{
+ struct ae4dma_dmadev *ae4dma = dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ struct ae4dma_desc *dma_desc;
+ uint16_t ret;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+ uint16_t write = cmd_q->next_write;
+
+ if (nb == 0)
+ return -EINVAL;
+
+ /* Reserve one slot to distinguish full from empty (power-of-two ring). */
+ if ((uint32_t)cmd_q->ring_buff_count >= (uint32_t)(nb - 1))
+ return -ENOSPC;
+
+ dma_desc = &cmd_q->qbase_desc[write];
+ memset(dma_desc, 0, sizeof(*dma_desc));
+ dma_desc->length = len;
+ dma_desc->src_hi = upper_32_bits(src);
+ dma_desc->src_lo = lower_32_bits(src);
+ dma_desc->dst_hi = upper_32_bits(dst);
+ dma_desc->dst_lo = lower_32_bits(dst);
+ cmd_q->ring_buff_count++;
+ cmd_q->next_write = (uint16_t)((write + 1) % nb);
+ ret = write;
+ if (flags & RTE_DMA_OP_FLAG_SUBMIT)
+ __submit(ae4dma);
+ return ret;
+}
+
+/* Enqueue a copy operation onto the ae4dma device. */
+static int
+ae4dma_enqueue_copy(void *dev_private, uint16_t vchan __rte_unused,
+ rte_iova_t src, rte_iova_t dst, uint32_t length, uint64_t flags)
+{
+ return __write_desc_copy(dev_private, src, dst, length, flags);
+}
+
/* Dump DMA device info. */
static int
ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
@@ -197,6 +264,220 @@ ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
cmd_q->stats.errors);
return 0;
}
+
+/* Translates AE4DMA ChanERRs to DMA error codes. */
+static inline enum rte_dma_status_code
+__translate_status_ae4dma_to_dma(enum ae4dma_dma_err status)
+{
+ AE4DMA_PMD_DEBUG("ae4dma desc status = %d", status);
+
+ switch (status) {
+ case AE4DMA_DMA_ERR_NO_ERR:
+ return RTE_DMA_STATUS_SUCCESSFUL;
+ case AE4DMA_DMA_ERR_INV_LEN:
+ return RTE_DMA_STATUS_INVALID_LENGTH;
+ case AE4DMA_DMA_ERR_INV_SRC:
+ return RTE_DMA_STATUS_INVALID_SRC_ADDR;
+ case AE4DMA_DMA_ERR_INV_DST:
+ return RTE_DMA_STATUS_INVALID_DST_ADDR;
+ case AE4DMA_DMA_ERR_INV_ALIGN:
+ /* Name matches DPDK public enum spelling. */
+ return RTE_DMA_STATUS_DATA_POISION;
+ case AE4DMA_DMA_ERR_INV_HEADER:
+ case AE4DMA_DMA_ERR_INV_STATUS:
+ return RTE_DMA_STATUS_ERROR_UNKNOWN;
+ default:
+ return RTE_DMA_STATUS_ERROR_UNKNOWN;
+ }
+}
+
+/*
+ * Scan HW queue for completed descriptors (non-blocking).
+ *
+ * The AE4DMA engine signals completion by advancing the per-queue
+ * `read_idx` register; it does not (reliably) write a status value
+ * back into the descriptor. We therefore use the HW `read_idx`
+ * register as the source of truth and only inspect the descriptor's
+ * `dw1.err_code` byte to classify each completion as success or
+ * failure.
+ *
+ * @param cmd_q
+ * The AE4DMA command queue.
+ * @param max_ops
+ * Maximum descriptors to process this call.
+ * @param[out] failed_count
+ * Number of completed descriptors that did not report success.
+ * @return
+ * Number of descriptors completed (success + failure), <= max_ops.
+ */
+static inline uint16_t
+ae4dma_scan_hwq(struct ae4dma_cmd_queue *cmd_q, uint16_t max_ops,
+ uint16_t *failed_count)
+{
+ volatile struct ae4dma_desc *hw_desc;
+ uint16_t events_count = 0, fails = 0;
+ uint16_t tail;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+ uint16_t mask;
+ uint16_t hw_read_idx;
+ uint16_t in_flight;
+ uint16_t scan_cap;
+
+ if (nb == 0 || cmd_q->ring_buff_count == 0) {
+ *failed_count = 0;
+ return 0;
+ }
+ mask = nb - 1;
+
+ hw_read_idx = (uint16_t)(AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx) & mask);
+ tail = cmd_q->next_read;
+
+ /*
+ * Descriptors completed since our last visit live in the
+ * half-open ring range [tail, hw_read_idx). If HW hasn't
+ * moved we have nothing to do.
+ */
+ in_flight = (uint16_t)((hw_read_idx - tail) & mask);
+ if (in_flight == 0) {
+ *failed_count = 0;
+ return 0;
+ }
+
+ scan_cap = max_ops;
+ if (scan_cap > AE4DMA_DESCRIPTORS_PER_CMDQ)
+ scan_cap = AE4DMA_DESCRIPTORS_PER_CMDQ;
+ if (scan_cap > in_flight)
+ scan_cap = in_flight;
+ if (scan_cap > cmd_q->ring_buff_count)
+ scan_cap = (uint16_t)cmd_q->ring_buff_count;
+
+ while (events_count < scan_cap) {
+ uint8_t hw_status;
+ uint8_t hw_err;
+
+ hw_desc = &cmd_q->qbase_desc[tail];
+ hw_status = hw_desc->dw1.status;
+ hw_err = hw_desc->dw1.err_code;
+
+ /*
+ * read_idx advancing is the definitive completion
+ * signal. The per-descriptor status byte is informational
+ * and may not yet be written when we observe it:
+ *
+ * AE4DMA_DMA_DESC_ERROR (4)
+ * Hard failure - err_code names the precise cause.
+ * AE4DMA_DMA_DESC_COMPLETED (3) or 0
+ * Success.
+ * AE4DMA_DMA_DESC_VALIDATED (1) / _PROCESSED (2)
+ * Benign race: HW had not finished updating the
+ * status byte at the instant we read it. Since
+ * read_idx has moved past this slot, treat it as
+ * success unless err_code says otherwise.
+ *
+ * A non-zero err_code is treated as a failure regardless
+ * of the observed status value.
+ */
+ if (hw_status == AE4DMA_DMA_DESC_ERROR ||
+ hw_err != AE4DMA_DMA_ERR_NO_ERR) {
+ fails++;
+ AE4DMA_PMD_WARN("Desc failed: status=%u err=%u",
+ hw_status, hw_err);
+ }
+ cmd_q->status[events_count] = (enum ae4dma_dma_err)hw_err;
+ cmd_q->ring_buff_count--;
+ events_count++;
+ tail = (tail + 1) & mask;
+ }
+
+ cmd_q->stats.completed += events_count;
+ cmd_q->stats.errors += fails;
+ cmd_q->next_read = tail;
+ *failed_count = fails;
+ return events_count;
+}
+
+/* Returns successful operations count and sets error flag if any errors. */
+static uint16_t
+ae4dma_completed(void *dev_private, uint16_t vchan __rte_unused,
+ const uint16_t max_ops, uint16_t *last_idx, bool *has_error)
+{
+ struct ae4dma_dmadev *ae4dma = dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t cpl_count, sl_count;
+ uint16_t err_count = 0;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+
+ *has_error = false;
+
+ cpl_count = ae4dma_scan_hwq(cmd_q, max_ops, &err_count);
+
+ if (cpl_count > max_ops)
+ cpl_count = max_ops;
+
+ if (cpl_count > 0 && last_idx != NULL)
+ *last_idx = (uint16_t)((cmd_q->next_read - 1 + nb) % nb);
+
+ sl_count = cpl_count - err_count;
+ if (err_count)
+ *has_error = true;
+
+ return sl_count;
+}
+
+static uint16_t
+ae4dma_completed_status(void *dev_private, uint16_t vchan __rte_unused,
+ uint16_t max_ops, uint16_t *last_idx,
+ enum rte_dma_status_code *status)
+{
+ struct ae4dma_dmadev *ae4dma = dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t cpl_count;
+ uint16_t i;
+ uint16_t err_count = 0;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+
+ cpl_count = ae4dma_scan_hwq(cmd_q, max_ops, &err_count);
+
+ if (cpl_count > max_ops)
+ cpl_count = max_ops;
+
+ if (cpl_count > 0 && last_idx != NULL)
+ *last_idx = (uint16_t)((cmd_q->next_read - 1 + nb) % nb);
+
+ if (likely(err_count == 0)) {
+ for (i = 0; i < cpl_count; i++)
+ status[i] = RTE_DMA_STATUS_SUCCESSFUL;
+ } else {
+ for (i = 0; i < cpl_count; i++)
+ status[i] = __translate_status_ae4dma_to_dma(cmd_q->status[i]);
+ }
+
+ return cpl_count;
+}
+
+/* Get the remaining capacity of the ring. */
+static uint16_t
+ae4dma_burst_capacity(const void *dev_private, uint16_t vchan __rte_unused)
+{
+ const struct ae4dma_dmadev *ae4dma = dev_private;
+ const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+ uint16_t mask;
+ uint16_t read_idx = cmd_q->next_read;
+ uint16_t write_idx = cmd_q->next_write;
+ uint16_t used;
+
+ if (nb < 2 || !rte_is_power_of_2(nb))
+ return 0;
+
+ mask = nb - 1;
+ used = (uint16_t)((write_idx - read_idx) & mask);
+ /* One slot reserved (same rule as enqueue). */
+ if (used >= nb - 1)
+ return 0;
+ return (uint16_t)(nb - 1 - used);
+}
+
/* Retrieve the generic stats of a DMA device. */
static int
ae4dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
@@ -357,6 +638,13 @@ ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
dmadev->fp_obj->dev_private = dmadev->data->dev_private;
dmadev->dev_ops = &ae4dma_dmadev_ops;
+ dmadev->fp_obj->burst_capacity = ae4dma_burst_capacity;
+ dmadev->fp_obj->completed = ae4dma_completed;
+ dmadev->fp_obj->completed_status = ae4dma_completed_status;
+ dmadev->fp_obj->copy = ae4dma_enqueue_copy;
+ dmadev->fp_obj->submit = ae4dma_submit;
+ /* fill capability not advertised: leave fp_obj->fill as zero-initialised. */
+
ae4dma = dmadev->data->dev_private;
ae4dma->dmadev = dmadev;
ae4dma->pci = dev;
--
2.34.1
^ permalink raw reply related
* [PATCH v2 2/3] dma/ae4dma: add control path operations
From: Raghavendra Ningoji @ 2026-05-25 18:42 UTC (permalink / raw)
To: dev
Cc: Thomas Monjalon, Bhagyada Modali, Robin Jarry, Selwin.Sebastian,
david.marchand, Raghavendra Ningoji
In-Reply-To: <20260525184244.1758825-1-raghavendra.ningoji@amd.com>
Implement the dmadev control path for the AMD AE4DMA PMD.
This commit adds:
- dev_configure / vchan_setup: accept a single virtual channel per
dmadev and clamp the requested ring size to the hardware maximum
of 32 descriptors (rounded up to a power of two).
- dev_start / dev_stop / dev_close: program the per-queue control
register to enable/disable the hardware queue and release the
descriptor ring memzone on close.
- dev_info_get: advertise RTE_DMA_CAPA_MEM_TO_MEM and the fixed
ring depth.
- dev_dump: print the queue identifiers, ring layout and software
completion counters.
- stats_get / stats_reset: expose submitted / completed / errors
counters maintained by the driver.
- vchan_status: report IDLE / ACTIVE based on hardware read_idx vs
write_idx, and HALTED_ERROR when the queue is not enabled.
The dmadev framework is wired through dev_ops in ae4dma_dmadev_create().
Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
drivers/dma/ae4dma/ae4dma_dmadev.c | 223 +++++++++++++++++++++++++++++
1 file changed, 223 insertions(+)
diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
index 76de2cde45..dfda723c13 100644
--- a/drivers/dma/ae4dma/ae4dma_dmadev.c
+++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
@@ -53,6 +53,215 @@ ae4dma_queue_dma_zone_reserve(const char *queue_name,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
+/* Configure a device. */
+static int
+ae4dma_dev_configure(struct rte_dma_dev *dev __rte_unused,
+ const struct rte_dma_conf *dev_conf,
+ uint32_t conf_sz)
+{
+ if (sizeof(struct rte_dma_conf) != conf_sz)
+ return -EINVAL;
+
+ if (dev_conf->nb_vchans != 1)
+ return -EINVAL;
+
+ return 0;
+}
+
+/* Setup a virtual channel for AE4DMA, only 1 vchan is supported per dmadev. */
+static int
+ae4dma_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
+ const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t max_desc = qconf->nb_desc;
+
+ if (sizeof(struct rte_dma_vchan_conf) != qconf_sz)
+ return -EINVAL;
+
+ if (max_desc < 2)
+ return -EINVAL;
+
+ if (!rte_is_power_of_2(max_desc))
+ max_desc = rte_align32pow2(max_desc);
+
+ if (max_desc > AE4DMA_DESCRIPTORS_PER_CMDQ) {
+ AE4DMA_PMD_DEBUG("DMA dev %u nb_desc clamped to %u",
+ dev->data->dev_id, AE4DMA_DESCRIPTORS_PER_CMDQ);
+ max_desc = AE4DMA_DESCRIPTORS_PER_CMDQ;
+ }
+
+ cmd_q->qcfg = *qconf;
+ cmd_q->qcfg.nb_desc = max_desc;
+
+ /* Ensure all counters are reset, if reconfiguring/restarting device. */
+ memset(&cmd_q->stats, 0, sizeof(cmd_q->stats));
+ return 0;
+}
+
+/* Start a configured device. */
+static int
+ae4dma_dev_start(struct rte_dma_dev *dev)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint16_t nb = cmd_q->qcfg.nb_desc;
+
+ if (nb == 0)
+ return -EBUSY;
+
+ /* Program ring depth expected by hardware. */
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->max_idx, nb);
+ return 0;
+}
+
+/* Stop a configured device. */
+static int
+ae4dma_dev_stop(struct rte_dma_dev *dev)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+
+ if (cmd_q->hwq_regs != NULL)
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
+ AE4DMA_CMD_QUEUE_DISABLE);
+ return 0;
+}
+
+/* Get device information of a device. */
+static int
+ae4dma_dev_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info,
+ uint32_t size)
+{
+ if (size < sizeof(*info))
+ return -EINVAL;
+ info->dev_name = dev->device->name;
+ info->dev_capa = RTE_DMA_CAPA_MEM_TO_MEM;
+ info->max_vchans = 1;
+ info->min_desc = 2;
+ info->max_desc = AE4DMA_DESCRIPTORS_PER_CMDQ;
+ info->nb_vchans = 1;
+ return 0;
+}
+
+/* Close a configured device. */
+static int
+ae4dma_dev_close(struct rte_dma_dev *dev)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+
+ if (cmd_q->hwq_regs != NULL)
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
+ AE4DMA_CMD_QUEUE_DISABLE);
+
+ if (cmd_q->memz_name[0] != '\0') {
+ const struct rte_memzone *mz = rte_memzone_lookup(cmd_q->memz_name);
+
+ if (mz != NULL)
+ rte_memzone_free(mz);
+ }
+ cmd_q->qbase_desc = NULL;
+ cmd_q->qbase_addr = NULL;
+ cmd_q->qbase_phys_addr = 0;
+ return 0;
+}
+/* Dump DMA device info. */
+static int
+ae4dma_dev_dump(const struct rte_dma_dev *dev, FILE *f)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q;
+ void *ae4dma_mmio_base_addr = (uint8_t *)ae4dma->io_regs;
+
+ cmd_q = &ae4dma->cmd_q;
+ fprintf(f, "cmd_q->id = %" PRIx64 "\n", cmd_q->id);
+ fprintf(f, "cmd_q->qidx = %" PRIx64 "\n", cmd_q->qidx);
+ fprintf(f, "cmd_q->qsize = %" PRIx64 "\n", cmd_q->qsize);
+ fprintf(f, "mmio_base_addr = %p\n", ae4dma_mmio_base_addr);
+ fprintf(f, "queues per ae4dma engine = %d\n", AE4DMA_READ_REG_OFFSET(
+ ae4dma_mmio_base_addr, AE4DMA_COMMON_CONFIG_OFFSET));
+ fprintf(f, "== Private Data ==\n");
+ fprintf(f, " Config: { ring_size: %u }\n", cmd_q->qcfg.nb_desc);
+ fprintf(f, " Ring virt: %p\tphys: %#" PRIx64 "\n",
+ (void *)cmd_q->qbase_desc,
+ (uint64_t)cmd_q->qbase_phys_addr);
+ fprintf(f, " Next write: %u\n", cmd_q->next_write);
+ fprintf(f, " Next read: %u\n", cmd_q->next_read);
+ fprintf(f, " current queue depth: %u\n", cmd_q->ring_buff_count);
+ fprintf(f, " }\n");
+ fprintf(f, " Key Stats { submitted: %" PRIu64 ", comp: %" PRIu64 ", failed: %" PRIu64 " }\n",
+ cmd_q->stats.submitted,
+ cmd_q->stats.completed,
+ cmd_q->stats.errors);
+ return 0;
+}
+/* Retrieve the generic stats of a DMA device. */
+static int
+ae4dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
+ struct rte_dma_stats *rte_stats, uint32_t size)
+{
+ const struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ const struct rte_dma_stats *stats = &cmd_q->stats;
+
+ if (size < sizeof(*rte_stats))
+ return -EINVAL;
+ if (rte_stats == NULL)
+ return -EINVAL;
+
+ *rte_stats = *stats;
+ return 0;
+}
+
+/* Reset the generic stat counters for the DMA device. */
+static int
+ae4dma_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused)
+{
+ struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+
+ memset(&cmd_q->stats, 0, sizeof(cmd_q->stats));
+ return 0;
+}
+
+/*
+ * Report channel state to the dmadev framework.
+ *
+ * RTE_DMA_VCHAN_HALTED_ERROR - HW queue is disabled (never started, or
+ * stopped via dev_stop()).
+ * RTE_DMA_VCHAN_IDLE - HW has caught up: read_idx == write_idx,
+ * no descriptors in flight.
+ * RTE_DMA_VCHAN_ACTIVE - HW still has descriptors to process.
+ */
+static int
+ae4dma_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused,
+ enum rte_dma_vchan_status *status)
+{
+ const struct ae4dma_dmadev *ae4dma = dev->fp_obj->dev_private;
+ const struct ae4dma_cmd_queue *cmd_q = &ae4dma->cmd_q;
+ uint32_t ctrl, hw_read, hw_write;
+
+ if (cmd_q->hwq_regs == NULL) {
+ *status = RTE_DMA_VCHAN_HALTED_ERROR;
+ return 0;
+ }
+
+ ctrl = AE4DMA_READ_REG(&cmd_q->hwq_regs->control_reg.control_raw);
+ if ((ctrl & AE4DMA_CMD_QUEUE_ENABLE) == 0) {
+ *status = RTE_DMA_VCHAN_HALTED_ERROR;
+ return 0;
+ }
+
+ hw_read = AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx);
+ hw_write = AE4DMA_READ_REG(&cmd_q->hwq_regs->write_idx);
+
+ *status = (hw_read == hw_write) ? RTE_DMA_VCHAN_IDLE
+ : RTE_DMA_VCHAN_ACTIVE;
+ return 0;
+}
+
static int
ae4dma_add_queue(struct ae4dma_dmadev *dev, uint8_t qn, const char *pci_name)
{
@@ -114,6 +323,19 @@ ae4dma_channel_dev_name(char *out, size_t outlen, const char *pci_name,
static int
ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
{
+ static const struct rte_dma_dev_ops ae4dma_dmadev_ops = {
+ .dev_close = ae4dma_dev_close,
+ .dev_configure = ae4dma_dev_configure,
+ .dev_dump = ae4dma_dev_dump,
+ .dev_info_get = ae4dma_dev_info_get,
+ .dev_start = ae4dma_dev_start,
+ .dev_stop = ae4dma_dev_stop,
+ .stats_get = ae4dma_stats_get,
+ .stats_reset = ae4dma_stats_reset,
+ .vchan_status = ae4dma_vchan_status,
+ .vchan_setup = ae4dma_vchan_setup,
+ };
+
struct rte_dma_dev *dmadev = NULL;
struct ae4dma_dmadev *ae4dma = NULL;
char hwq_dev_name[RTE_DEV_NAME_MAX_LEN];
@@ -133,6 +355,7 @@ ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
}
dmadev->device = &dev->device;
dmadev->fp_obj->dev_private = dmadev->data->dev_private;
+ dmadev->dev_ops = &ae4dma_dmadev_ops;
ae4dma = dmadev->data->dev_private;
ae4dma->dmadev = dmadev;
--
2.34.1
^ permalink raw reply related
* [PATCH v2 1/3] dma/ae4dma: introduce AMD AE4DMA DMA PMD
From: Raghavendra Ningoji @ 2026-05-25 18:42 UTC (permalink / raw)
To: dev
Cc: Thomas Monjalon, Bhagyada Modali, Robin Jarry, Selwin.Sebastian,
david.marchand, Raghavendra Ningoji
In-Reply-To: <20260525184244.1758825-1-raghavendra.ningoji@amd.com>
Add the skeleton of a new dmadev poll-mode driver for the AMD AE4DMA
hardware DMA engine, providing only PCI probe/remove and per-queue
hardware initialisation. An AE4DMA engine exposes 16 hardware command
queues, each with a 32-entry descriptor ring; the PMD maps each
hardware channel to its own dmadev with a single virtual channel,
so a PCI function appears as 16 dmadevs named "<pci-bdf>-ch0" ..
"<pci-bdf>-ch15".
This patch only registers the PCI driver, allocates the dmadev
objects, reserves the per-queue descriptor rings and programs the
hardware queue base addresses. Control and data path operations are
added in subsequent patches.
Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
.mailmap | 1 +
MAINTAINERS | 5 +
doc/guides/dmadevs/ae4dma.rst | 53 ++++++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_26_07.rst | 7 +
drivers/dma/ae4dma/ae4dma_dmadev.c | 227 +++++++++++++++++++++++++
drivers/dma/ae4dma/ae4dma_hw_defs.h | 160 +++++++++++++++++
drivers/dma/ae4dma/ae4dma_internal.h | 118 +++++++++++++
drivers/dma/ae4dma/meson.build | 7 +
drivers/dma/meson.build | 1 +
usertools/dpdk-devbind.py | 5 +-
11 files changed, 584 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/dmadevs/ae4dma.rst
create mode 100644 drivers/dma/ae4dma/ae4dma_dmadev.c
create mode 100644 drivers/dma/ae4dma/ae4dma_hw_defs.h
create mode 100644 drivers/dma/ae4dma/ae4dma_internal.h
create mode 100644 drivers/dma/ae4dma/meson.build
diff --git a/.mailmap b/.mailmap
index 89ba6ffccc..60180818f9 100644
--- a/.mailmap
+++ b/.mailmap
@@ -203,6 +203,7 @@ Benoît Ganne <bganne@cisco.com>
Bernard Iremonger <bernard.iremonger@intel.com>
Bert van Leeuwen <bert.vanleeuwen@netronome.com>
Bhagyada Modali <bhagyada.modali@amd.com>
+Raghavendra Ningoji <raghavendra.ningoji@amd.com>
Bharat Mota <bharat.mota@broadcom.com> <bmota@vmware.com>
Bhuvan Mital <bhuvan.mital@amd.com>
Bibo Mao <maobibo@loongson.cn>
diff --git a/MAINTAINERS b/MAINTAINERS
index 9143d028bc..2e27af49f4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1361,6 +1361,11 @@ F: doc/guides/compressdevs/features/zsda.ini
DMAdev Drivers
--------------
+AMD AE4DMA
+M: Bhagyada Modali <bhagyada.modali@amd.com>
+F: drivers/dma/ae4dma/
+F: doc/guides/dmadevs/ae4dma.rst
+
Intel IDXD - EXPERIMENTAL
M: Bruce Richardson <bruce.richardson@intel.com>
M: Kevin Laatz <kevin.laatz@intel.com>
diff --git a/doc/guides/dmadevs/ae4dma.rst b/doc/guides/dmadevs/ae4dma.rst
new file mode 100644
index 0000000000..a85c1d92ca
--- /dev/null
+++ b/doc/guides/dmadevs/ae4dma.rst
@@ -0,0 +1,53 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2025 Advanced Micro Devices, Inc.
+
+.. include:: <isonum.txt>
+
+AMD AE4DMA DMA Device Driver
+============================
+
+The ``ae4dma`` dmadev driver is a poll-mode driver (PMD) for the
+AMD AE4DMA hardware DMA engine. The engine exposes 16 independent
+hardware command queues, each with a ring of 32 descriptors. The PMD
+maps each hardware command queue to a separate DPDK dmadev with a
+single virtual channel, so a single PCI function appears as 16 dmadevs
+named ``<pci-bdf>-ch0`` through ``<pci-bdf>-ch15``.
+
+The driver supports memory-to-memory copy operations only.
+
+Hardware Requirements
+---------------------
+
+The ``dpdk-devbind.py`` script can be used to list AE4DMA devices on
+the system::
+
+ dpdk-devbind.py --status-dev dma
+
+AE4DMA devices appear with vendor ID ``0x1022`` and device ID
+``0x149b``.
+
+Compilation
+-----------
+
+The driver is built as part of the standard DPDK build on x86 platforms
+using ``meson`` and ``ninja``; no extra configuration is required.
+
+Device Setup
+------------
+
+The AE4DMA device must be bound to a DPDK-compatible kernel module such
+as ``vfio-pci`` before it can be used::
+
+ dpdk-devbind.py -b vfio-pci <pci-bdf>
+
+Initialization
+~~~~~~~~~~~~~~
+
+On probe the PMD performs the following steps for each PCI function:
+
+* Reads BAR0 and programs the common configuration register with the
+ number of hardware queues to enable (16).
+* For each hardware queue it allocates a 32-entry descriptor ring in
+ IOVA-contiguous memory, programs the queue base address and ring
+ depth into the per-queue registers, and enables the queue.
+* Interrupts are masked; completion is polled by the application.
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
index 56beb1733f..97399590f6 100644
--- a/doc/guides/dmadevs/index.rst
+++ b/doc/guides/dmadevs/index.rst
@@ -11,6 +11,7 @@ an application through DMA API.
:maxdepth: 1
:numbered:
+ ae4dma
cnxk
dpaa
dpaa2
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index f012d47a4b..9a78a7ef62 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -63,6 +63,13 @@ New Features
``rte_eal_init`` and the application is responsible for probing each device,
* ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
+* **Added AMD AE4DMA DMA PMD.**
+
+ Added a new ``dma/ae4dma`` driver for the AMD AE4DMA hardware DMA engine.
+ Each PCI function exposes 16 hardware command queues; the PMD registers one
+ dmadev per channel with a single virtual channel and supports
+ memory-to-memory copy operations.
+
Removed Items
-------------
diff --git a/drivers/dma/ae4dma/ae4dma_dmadev.c b/drivers/dma/ae4dma/ae4dma_dmadev.c
new file mode 100644
index 0000000000..76de2cde45
--- /dev/null
+++ b/drivers/dma/ae4dma/ae4dma_dmadev.c
@@ -0,0 +1,227 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
+ */
+
+#include <errno.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <rte_bus_pci.h>
+#include <bus_pci_driver.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_malloc.h>
+
+#include "ae4dma_internal.h"
+
+/*
+ * One dmadev per AE4DMA hardware channel; each dmadev has exactly one
+ * virtual channel. The HW's per-queue register block must be densely
+ * packed right after the engine-common config register at BAR0+0; the
+ * build-time check below catches an accidental layout change.
+ */
+static_assert(sizeof(struct ae4dma_hwq_regs) == 32,
+ "ae4dma_hwq_regs stride changed; per-queue offset math will break");
+
+RTE_LOG_REGISTER_DEFAULT(ae4dma_pmd_logtype, INFO);
+
+#define AE4DMA_PMD_NAME dmadev_ae4dma
+
+static const struct rte_memzone *
+ae4dma_queue_dma_zone_reserve(const char *queue_name,
+ uint32_t queue_size, int socket_id)
+{
+ const struct rte_memzone *mz;
+
+ mz = rte_memzone_lookup(queue_name);
+ if (mz != NULL) {
+ if (((size_t)queue_size <= mz->len) &&
+ ((socket_id == SOCKET_ID_ANY) ||
+ (socket_id == mz->socket_id))) {
+ AE4DMA_PMD_INFO("reuse memzone already "
+ "allocated for %s", queue_name);
+ return mz;
+ }
+ AE4DMA_PMD_ERR("Incompatible memzone already "
+ "allocated %s, size %u, socket %d. "
+ "Requested size %u, socket %u",
+ queue_name, (uint32_t)mz->len,
+ mz->socket_id, queue_size, socket_id);
+ return NULL;
+ }
+ return rte_memzone_reserve_aligned(queue_name, queue_size,
+ socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
+}
+
+static int
+ae4dma_add_queue(struct ae4dma_dmadev *dev, uint8_t qn, const char *pci_name)
+{
+ uint32_t dma_addr_lo, dma_addr_hi;
+ struct ae4dma_cmd_queue *cmd_q;
+ const struct rte_memzone *q_mz;
+
+ dev->io_regs = dev->pci->mem_resource[AE4DMA_PCIE_BAR].addr;
+
+ cmd_q = &dev->cmd_q;
+ cmd_q->id = qn;
+ cmd_q->qidx = 0;
+ cmd_q->qsize = AE4DMA_QUEUE_SIZE(AE4DMA_QUEUE_DESC_SIZE);
+ cmd_q->hwq_regs = (volatile struct ae4dma_hwq_regs *)dev->io_regs + (qn + 1);
+
+ /*
+ * Memzone name must be globally unique. Embed PCI BDF so multiple
+ * PCI functions probed concurrently don't collide.
+ */
+ snprintf(cmd_q->memz_name, sizeof(cmd_q->memz_name),
+ "ae4dma_%s_q%u", pci_name, (unsigned int)qn);
+
+ q_mz = ae4dma_queue_dma_zone_reserve(cmd_q->memz_name,
+ cmd_q->qsize, rte_socket_id());
+ if (q_mz == NULL) {
+ AE4DMA_PMD_ERR("memzone reserve failed for %s", cmd_q->memz_name);
+ return -ENOMEM;
+ }
+
+ cmd_q->qbase_addr = (void *)q_mz->addr;
+ cmd_q->qbase_desc = (struct ae4dma_desc *)q_mz->addr;
+ cmd_q->qbase_phys_addr = q_mz->iova;
+
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->max_idx, AE4DMA_DESCRIPTORS_PER_CMDQ);
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->control_reg.control_raw,
+ AE4DMA_CMD_QUEUE_ENABLE);
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->intr_status_reg.intr_status_raw,
+ AE4DMA_DISABLE_INTR);
+ cmd_q->next_write = (uint16_t)AE4DMA_READ_REG(&cmd_q->hwq_regs->write_idx);
+ cmd_q->next_read = (uint16_t)AE4DMA_READ_REG(&cmd_q->hwq_regs->read_idx);
+ cmd_q->ring_buff_count = 0;
+
+ dma_addr_lo = low32_value(cmd_q->qbase_phys_addr);
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->qbase_lo, dma_addr_lo);
+ dma_addr_hi = high32_value(cmd_q->qbase_phys_addr);
+ AE4DMA_WRITE_REG(&cmd_q->hwq_regs->qbase_hi, dma_addr_hi);
+
+ return 0;
+}
+
+static void
+ae4dma_channel_dev_name(char *out, size_t outlen, const char *pci_name,
+ unsigned int ch)
+{
+ snprintf(out, outlen, "%s-ch%u", pci_name, ch);
+}
+
+/* Create a dmadev(dpdk DMA device) */
+static int
+ae4dma_dmadev_create(const char *name, struct rte_pci_device *dev, uint8_t qn)
+{
+ struct rte_dma_dev *dmadev = NULL;
+ struct ae4dma_dmadev *ae4dma = NULL;
+ char hwq_dev_name[RTE_DEV_NAME_MAX_LEN];
+
+ if (!name) {
+ AE4DMA_PMD_ERR("Invalid name of the device!");
+ return -EINVAL;
+ }
+ memset(hwq_dev_name, 0, sizeof(hwq_dev_name));
+ ae4dma_channel_dev_name(hwq_dev_name, sizeof(hwq_dev_name), name, qn);
+
+ dmadev = rte_dma_pmd_allocate(hwq_dev_name, dev->device.numa_node,
+ sizeof(struct ae4dma_dmadev));
+ if (dmadev == NULL) {
+ AE4DMA_PMD_ERR("Unable to allocate dma device");
+ return -ENOMEM;
+ }
+ dmadev->device = &dev->device;
+ dmadev->fp_obj->dev_private = dmadev->data->dev_private;
+
+ ae4dma = dmadev->data->dev_private;
+ ae4dma->dmadev = dmadev;
+ ae4dma->pci = dev;
+
+ if (ae4dma_add_queue(ae4dma, qn, name) != 0)
+ goto init_error;
+ return 0;
+
+init_error:
+ AE4DMA_PMD_ERR("driver %s(): failed", __func__);
+ rte_dma_pmd_release(hwq_dev_name);
+ return -ENOMEM;
+}
+
+/* Probe DMA device. */
+static int
+ae4dma_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev)
+{
+ char name[32];
+ char chname[RTE_DEV_NAME_MAX_LEN];
+ void *mmio_base;
+ uint32_t q_per_eng;
+ int ret = 0;
+ uint8_t i;
+
+ rte_pci_device_name(&dev->addr, name, sizeof(name));
+ AE4DMA_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node);
+ dev->device.driver = &drv->driver;
+
+ mmio_base = dev->mem_resource[AE4DMA_PCIE_BAR].addr;
+ if (mmio_base == NULL) {
+ AE4DMA_PMD_ERR("%s: BAR%d not mapped", name, AE4DMA_PCIE_BAR);
+ return -ENODEV;
+ }
+
+ /* Program the per-engine HW queue count once. */
+ AE4DMA_WRITE_REG_OFFSET(mmio_base, AE4DMA_COMMON_CONFIG_OFFSET,
+ AE4DMA_MAX_HW_QUEUES);
+ q_per_eng = AE4DMA_READ_REG_OFFSET(mmio_base, AE4DMA_COMMON_CONFIG_OFFSET);
+ AE4DMA_PMD_INFO("%s: AE4DMA queues per engine = %u", name, q_per_eng);
+
+ for (i = 0; i < AE4DMA_MAX_HW_QUEUES; i++) {
+ ret = ae4dma_dmadev_create(name, dev, i);
+ if (ret != 0) {
+ AE4DMA_PMD_ERR("%s create dmadev %u failed!", name, i);
+ while (i > 0) {
+ i--;
+ ae4dma_channel_dev_name(chname, sizeof(chname), name, i);
+ rte_dma_pmd_release(chname);
+ }
+ break;
+ }
+ }
+ return ret;
+}
+
+/* Remove DMA device. */
+static int
+ae4dma_dmadev_remove(struct rte_pci_device *dev)
+{
+ char name[32];
+ char chname[RTE_DEV_NAME_MAX_LEN];
+ unsigned int i;
+
+ rte_pci_device_name(&dev->addr, name, sizeof(name));
+
+ AE4DMA_PMD_INFO("Closing %s on NUMA node %d",
+ name, dev->device.numa_node);
+
+ for (i = 0; i < AE4DMA_MAX_HW_QUEUES; i++) {
+ ae4dma_channel_dev_name(chname, sizeof(chname), name, i);
+ rte_dma_pmd_release(chname);
+ }
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_ae4dma_map[] = {
+ { RTE_PCI_DEVICE(AMD_VENDOR_ID, AE4DMA_DEVICE_ID) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+static struct rte_pci_driver ae4dma_pmd_drv = {
+ .id_table = pci_id_ae4dma_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = ae4dma_dmadev_probe,
+ .remove = ae4dma_dmadev_remove,
+};
+
+RTE_PMD_REGISTER_PCI(AE4DMA_PMD_NAME, ae4dma_pmd_drv);
+RTE_PMD_REGISTER_PCI_TABLE(AE4DMA_PMD_NAME, pci_id_ae4dma_map);
+RTE_PMD_REGISTER_KMOD_DEP(AE4DMA_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci");
diff --git a/drivers/dma/ae4dma/ae4dma_hw_defs.h b/drivers/dma/ae4dma/ae4dma_hw_defs.h
new file mode 100644
index 0000000000..62b6a1b30b
--- /dev/null
+++ b/drivers/dma/ae4dma/ae4dma_hw_defs.h
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
+ */
+
+#ifndef __AE4DMA_HW_DEFS_H__
+#define __AE4DMA_HW_DEFS_H__
+
+#include <rte_bus_pci.h>
+#include <rte_byteorder.h>
+#include <rte_io.h>
+#include <rte_pci.h>
+#include <rte_memzone.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define AE4DMA_BIT(nr) (1UL << (nr))
+
+/* ae4dma device details */
+#define AMD_VENDOR_ID 0x1022
+#define AE4DMA_DEVICE_ID 0x149b
+#define AE4DMA_PCIE_BAR 0
+
+/*
+ * An AE4DMA engine has 16 DMA queues. Each queue supports 32 descriptors.
+ */
+#define AE4DMA_MAX_HW_QUEUES 16
+#define AE4DMA_QUEUE_START_INDEX 0
+#define AE4DMA_CMD_QUEUE_ENABLE 0x1
+#define AE4DMA_CMD_QUEUE_DISABLE 0x0
+
+/* Common to all queues */
+#define AE4DMA_COMMON_CONFIG_OFFSET 0x00
+
+#define AE4DMA_DISABLE_INTR 0x01
+
+/* Descriptor status */
+enum ae4dma_dma_status {
+ AE4DMA_DMA_DESC_SUBMITTED = 0,
+ AE4DMA_DMA_DESC_VALIDATED = 1,
+ AE4DMA_DMA_DESC_PROCESSED = 2,
+ AE4DMA_DMA_DESC_COMPLETED = 3,
+ AE4DMA_DMA_DESC_ERROR = 4,
+};
+
+/* Descriptor error-code */
+enum ae4dma_dma_err {
+ AE4DMA_DMA_ERR_NO_ERR = 0,
+ AE4DMA_DMA_ERR_INV_HEADER = 1,
+ AE4DMA_DMA_ERR_INV_STATUS = 2,
+ AE4DMA_DMA_ERR_INV_LEN = 3,
+ AE4DMA_DMA_ERR_INV_SRC = 4,
+ AE4DMA_DMA_ERR_INV_DST = 5,
+ AE4DMA_DMA_ERR_INV_ALIGN = 6,
+ AE4DMA_DMA_ERR_UNKNOWN = 7,
+};
+
+/* HW Queue status */
+enum ae4dma_hwqueue_status {
+ AE4DMA_HWQUEUE_EMPTY = 0,
+ AE4DMA_HWQUEUE_FULL = 1,
+ AE4DMA_HWQUEUE_NOT_EMPTY = 4
+};
+/*
+ * descriptor for AE4DMA commands
+ * 8 32-bit words:
+ * word 0: source memory type; destination memory type ; control bits
+ * word 1: desc_id; error code; status
+ * word 2: length
+ * word 3: reserved
+ * word 4: upper 32 bits of source pointer
+ * word 5: low 32 bits of source pointer
+ * word 6: upper 32 bits of destination pointer
+ * word 7: low 32 bits of destination pointer
+ */
+
+/* AE4DMA Descriptor - DWORD0 - Controls bits: Reserved for future use */
+#define AE4DMA_DWORD0_STOP_ON_COMPLETION AE4DMA_BIT(0)
+#define AE4DMA_DWORD0_INTERRUPT_ON_COMPLETION AE4DMA_BIT(1)
+#define AE4DMA_DWORD0_START_OF_MESSAGE AE4DMA_BIT(3)
+#define AE4DMA_DWORD0_END_OF_MESSAGE AE4DMA_BIT(4)
+#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE RTE_GENMASK64(5, 4)
+#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE RTE_GENMASK64(7, 6)
+
+#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE_MEMORY (0x0)
+#define AE4DMA_DWORD0_DESTINATION_MEMORY_TYPE_IOMEMORY (1<<4)
+#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE_MEMORY (0x0)
+#define AE4DMA_DWORD0_SOURCE_MEMEORY_TYPE_IOMEMORY (1<<6)
+
+struct ae4dma_desc_dword0 {
+ uint8_t byte0;
+ uint8_t byte1;
+ uint16_t timestamp;
+};
+
+struct ae4dma_desc_dword1 {
+ uint8_t status;
+ uint8_t err_code;
+ uint16_t desc_id;
+};
+
+struct ae4dma_desc {
+ struct ae4dma_desc_dword0 dw0;
+ struct ae4dma_desc_dword1 dw1;
+ uint32_t length;
+ uint32_t reserved;
+ uint32_t src_lo;
+ uint32_t src_hi;
+ uint32_t dst_lo;
+ uint32_t dst_hi;
+};
+
+/*
+ * Registers for each queue :4 bytes length
+ * Effective address : offset + reg
+ */
+struct ae4dma_hwq_regs {
+ union {
+ uint32_t control_raw;
+ struct {
+ uint32_t queue_enable: 1;
+ uint32_t reserved_internal: 31;
+ } control;
+ } control_reg;
+
+ union {
+ uint32_t status_raw;
+ struct {
+ uint32_t reserved0: 1;
+ /* 0–empty, 1–full, 2–stopped, 3–error , 4–Not Empty */
+ uint32_t queue_status: 2;
+ uint32_t reserved1: 21;
+ uint32_t interrupt_type: 4;
+ uint32_t reserved2: 4;
+ } status;
+ } status_reg;
+
+ uint32_t max_idx;
+ uint32_t read_idx;
+ uint32_t write_idx;
+
+ union {
+ uint32_t intr_status_raw;
+ struct {
+ uint32_t intr_status: 1;
+ uint32_t reserved: 31;
+ } intr_status;
+ } intr_status_reg;
+
+ uint32_t qbase_lo;
+ uint32_t qbase_hi;
+
+};
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* AE4DMA_HW_DEFS_H */
diff --git a/drivers/dma/ae4dma/ae4dma_internal.h b/drivers/dma/ae4dma/ae4dma_internal.h
new file mode 100644
index 0000000000..9892d6697f
--- /dev/null
+++ b/drivers/dma/ae4dma/ae4dma_internal.h
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
+ */
+
+#ifndef _AE4DMA_INTERNAL_H_
+#define _AE4DMA_INTERNAL_H_
+
+#include <stdint.h>
+
+#include "ae4dma_hw_defs.h"
+
+/**
+ * upper_32_bits - return bits 32-63 of a number
+ * @n: the number we're accessing
+ */
+#define upper_32_bits(n) ((uint32_t)(((n) >> 16) >> 16))
+
+/**
+ * lower_32_bits - return bits 0-31 of a number
+ * @n: the number we're accessing
+ */
+#define lower_32_bits(n) ((uint32_t)((n) & 0xffffffff))
+
+/** Hardware ring depth (slots per queue); must be power of two. */
+#define AE4DMA_DESCRIPTORS_PER_CMDQ 32
+#define AE4DMA_QUEUE_DESC_SIZE sizeof(struct ae4dma_desc)
+#define AE4DMA_QUEUE_SIZE(n) (AE4DMA_DESCRIPTORS_PER_CMDQ * (n))
+
+
+/** AE4DMA registers Write/Read */
+static inline void ae4dma_pci_reg_write(void *base, int offset,
+ uint32_t value)
+{
+ volatile void *reg_addr = ((uint8_t *)base + offset);
+
+ rte_write32((rte_cpu_to_le_32(value)), reg_addr);
+}
+
+static inline uint32_t ae4dma_pci_reg_read(void *base, int offset)
+{
+ volatile void *reg_addr = ((uint8_t *)base + offset);
+
+ return rte_le_to_cpu_32(rte_read32(reg_addr));
+}
+
+#define AE4DMA_READ_REG_OFFSET(hw_addr, reg_offset) \
+ ae4dma_pci_reg_read(hw_addr, reg_offset)
+
+#define AE4DMA_WRITE_REG_OFFSET(hw_addr, reg_offset, value) \
+ ae4dma_pci_reg_write(hw_addr, reg_offset, value)
+
+
+#define AE4DMA_READ_REG(hw_addr) \
+ ae4dma_pci_reg_read((void *)(uintptr_t)(hw_addr), 0)
+
+#define AE4DMA_WRITE_REG(hw_addr, value) \
+ ae4dma_pci_reg_write((void *)(uintptr_t)(hw_addr), 0, value)
+
+static inline uint32_t
+low32_value(unsigned long addr)
+{
+ return ((uint64_t)addr) & 0xffffffffUL;
+}
+
+static inline uint32_t
+high32_value(unsigned long addr)
+{
+ return (uint32_t)(((uint64_t)addr) >> 32);
+}
+
+/**
+ * A structure describing a AE4DMA command queue.
+ */
+struct __rte_cache_aligned ae4dma_cmd_queue {
+ char memz_name[RTE_MEMZONE_NAMESIZE];
+ volatile struct ae4dma_hwq_regs *hwq_regs;
+
+ struct rte_dma_vchan_conf qcfg;
+ struct rte_dma_stats stats;
+ /* Queue address */
+ struct ae4dma_desc *qbase_desc;
+ void *qbase_addr;
+ rte_iova_t qbase_phys_addr;
+ enum ae4dma_dma_err status[AE4DMA_DESCRIPTORS_PER_CMDQ];
+ /* Queue identifier */
+ uint64_t id; /**< queue id */
+ uint64_t qidx; /**< queue index */
+ uint64_t qsize; /**< queue size */
+ uint32_t ring_buff_count;
+ unsigned short next_read;
+ unsigned short next_write;
+ unsigned short last_write; /* Used to compute submitted count. */
+};
+
+/*
+ * One dmadev per AE4DMA hardware channel: probe creates AE4DMA_MAX_HW_QUEUES
+ * dmadevs per PCI function, each owning a single HW command queue.
+ */
+struct ae4dma_dmadev {
+ struct rte_dma_dev *dmadev;
+ void *io_regs;
+ struct ae4dma_cmd_queue cmd_q; /**< single HW queue owned by this dmadev */
+ struct rte_pci_device *pci; /**< owning PCI device (not owned) */
+};
+
+
+extern int ae4dma_pmd_logtype;
+#define RTE_LOGTYPE_AE4DMA_PMD ae4dma_pmd_logtype
+
+#define AE4DMA_PMD_LOG(level, ...) \
+ RTE_LOG_LINE_PREFIX(level, AE4DMA_PMD, "%s(): ", __func__, __VA_ARGS__)
+
+#define AE4DMA_PMD_DEBUG(...) AE4DMA_PMD_LOG(DEBUG, __VA_ARGS__)
+#define AE4DMA_PMD_INFO(...) AE4DMA_PMD_LOG(INFO, __VA_ARGS__)
+#define AE4DMA_PMD_ERR(...) AE4DMA_PMD_LOG(ERR, __VA_ARGS__)
+#define AE4DMA_PMD_WARN(...) AE4DMA_PMD_LOG(WARNING, __VA_ARGS__)
+
+#endif /* _AE4DMA_INTERNAL_H_ */
diff --git a/drivers/dma/ae4dma/meson.build b/drivers/dma/ae4dma/meson.build
new file mode 100644
index 0000000000..e48ab0d561
--- /dev/null
+++ b/drivers/dma/ae4dma/meson.build
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2024 Advanced Micro Devices, Inc. All rights reserved.
+
+build = dpdk_conf.has('RTE_ARCH_X86')
+reason = 'only supported on x86'
+sources = files('ae4dma_dmadev.c')
+deps += ['bus_pci', 'dmadev']
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
index e0d94db967..c230ac5a06 100644
--- a/drivers/dma/meson.build
+++ b/drivers/dma/meson.build
@@ -2,6 +2,7 @@
# Copyright 2021 HiSilicon Limited
drivers = [
+ 'ae4dma',
'cnxk',
'dpaa',
'dpaa2',
diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 93f2383dff..7d09f155de 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -86,6 +86,9 @@
cn9k_ree = {'Class': '08', 'Vendor': '177d', 'Device': 'a0f4',
'SVendor': None, 'SDevice': None}
+amd_ae4dma = {'Class': '08', 'Vendor': '1022', 'Device': '149b',
+ 'SVendor': None, 'SDevice': None}
+
virtio_blk = {'Class': '01', 'Vendor': "1af4", 'Device': '1001,1042',
'SVendor': None, 'SDevice': None}
@@ -95,7 +98,7 @@
network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class]
baseband_devices = [acceleration_class]
crypto_devices = [encryption_class, intel_processor_class]
-dma_devices = [cnxk_dma, hisilicon_dma,
+dma_devices = [amd_ae4dma, cnxk_dma, hisilicon_dma,
intel_idxd_gnrd, intel_idxd_dmr, intel_idxd_spr,
intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx,
odm_dma]
--
2.34.1
^ permalink raw reply related
* [PATCH v2 0/3] dma/ae4dma: add AMD AE4DMA DMA PMD
From: Raghavendra Ningoji @ 2026-05-25 18:42 UTC (permalink / raw)
To: dev
Cc: Thomas Monjalon, Bhagyada Modali, Robin Jarry, Selwin.Sebastian,
david.marchand, Raghavendra Ningoji
In-Reply-To: <20260518181856.1228373-1-raghavendra.ningoji@amd.com>
This series adds a new dmadev poll-mode driver for the AMD AE4DMA
hardware DMA engine. An AE4DMA engine exposes 16 hardware command
queues, each with a 32-entry descriptor ring; the PMD maps each
hardware channel to its own dmadev with a single virtual channel,
so a PCI function appears as 16 dmadevs named "<pci-bdf>-ch0" ..
"<pci-bdf>-ch15".
Driver characteristics:
- Memory-to-memory copy operations only (RTE_DMA_CAPA_MEM_TO_MEM).
- Completion is detected via the hardware's per-queue read_idx
register, which the engine advances as it processes descriptors.
The descriptor status / err_code bytes are read only to classify
each drained slot as success or failure.
- vchan_status reports IDLE/ACTIVE based on HW read_idx vs write_idx
and HALTED_ERROR when the queue is not enabled.
- depends on bus_pci and dmadev.
The v1 was submitted as a single patch. Per review feedback the
driver is now introduced in three logical patches, following the
pattern of the recent hisi_acc dmadev driver:
1/3 - introduce driver (probe, remove, per-queue HW init)
2/3 - add control path operations (dev_ops)
3/3 - add data path operations (copy, submit, completion)
---
Changes in v2:
- Split the monolithic v1 patch into three logical patches
(introduce / control path / data path), mirroring the
structure used by drivers/dma/hisi_acc.
- Fix checkpatches.sh warnings in drivers/dma/ae4dma/ae4dma_internal.h:
* Use RTE_LOG_LINE_PREFIX (with RTE_LOGTYPE_AE4DMA_PMD) instead
of the deprecated rte_log() call form.
* Replace the GCC variadic argument-pack extension ("args...")
with C99 __VA_ARGS__ in the AE4DMA_PMD_{LOG,DEBUG,INFO,ERR,
WARN} macros.
- Move __rte_cache_aligned to the "struct" keyword position on
struct ae4dma_cmd_queue, as required by checkpatches.sh.
v1:https://patches.dpdk.org/project/dpdk/patch/20260518181856.1228373-1-raghavendra.ningoji@amd.com/
Raghavendra Ningoji (3):
dma/ae4dma: introduce AMD AE4DMA DMA PMD
dma/ae4dma: add control path operations
dma/ae4dma: add data path operations
.mailmap | 1 +
MAINTAINERS | 5 +
doc/guides/dmadevs/ae4dma.rst | 75 +++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_26_07.rst | 7 +
drivers/dma/ae4dma/ae4dma_dmadev.c | 738 +++++++++++++++++++++++++
drivers/dma/ae4dma/ae4dma_hw_defs.h | 160 ++++++
drivers/dma/ae4dma/ae4dma_internal.h | 118 ++++
drivers/dma/ae4dma/meson.build | 7 +
drivers/dma/meson.build | 1 +
usertools/dpdk-devbind.py | 5 +-
11 files changed, 1117 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/dmadevs/ae4dma.rst
create mode 100644 drivers/dma/ae4dma/ae4dma_dmadev.c
create mode 100644 drivers/dma/ae4dma/ae4dma_hw_defs.h
create mode 100644 drivers/dma/ae4dma/ae4dma_internal.h
create mode 100644 drivers/dma/ae4dma/meson.build
base-commit: f724d1c0d1c1636b9c171c34db3f17c3defaa2f3
--
2.34.1
^ permalink raw reply
* Re: [RFC 0/3] lib/fastmem: fast small-object allocator
From: Stephen Hemminger @ 2026-05-25 18:36 UTC (permalink / raw)
To: Mattias Rönnblom
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <20260525103642.55255-1-hofors@lysator.liu.se>
On Mon, 25 May 2026 12:36:39 +0200
Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> This RFC introduces fastmem, a general-purpose small-object allocator
> for DPDK. It is intended to replace per-type mempools with a single
> allocator that handles arbitrary sizes, grows on demand, and matches
> mempool-level performance on the hot path.
>
> Motivation
> ----------
>
> DPDK applications commonly maintain many mempools — one per object
> type (connections, sessions, timers, work items). Each must be sized
> up front, wastes memory when over-provisioned, and cannot serve
> objects of a different size. Fastmem eliminates this by accepting
> arbitrary sizes at runtime, backed by a slab allocator that
> repurposes memory across size classes as demand shifts.
>
> Design
> ------
>
> Three-layer architecture:
>
> 1. Backing memory: 128 MiB IOVA-contiguous memzones from EAL,
> reserved lazily (or pre-reserved for deterministic latency).
>
> 2. Slabs: 2 MiB, 2 MiB-aligned regions carved from memzones.
> The alignment enables O(1) slab lookup from any object pointer
> via bitmask — no radix tree or index structure. Slabs move
> freely between 18 power-of-2 size classes (8 B to 1 MiB).
>
> 3. Per-lcore caches: bounded LIFO stacks (no locks on the hot
> path). Cache misses trigger bulk transfers to/from the shared
> bin under a spinlock.
>
> Key properties:
>
> - Zero per-object metadata in the production build.
> - NUMA-aware, with per-socket bins and free-slab pools.
> - DMA-usable memory with O(1) virt-to-IOVA translation.
> - Bulk alloc/free with all-or-nothing semantics.
> - Backing memory never returned during lifetime (slabs recycled).
> - Non-EAL threads supported (bypass cache, take bin lock).
>
> API surface
> -----------
>
> rte_fastmem_init / deinit
> rte_fastmem_reserve
> rte_fastmem_set_limit / get_limit
> rte_fastmem_alloc / alloc_socket
> rte_fastmem_alloc_bulk / alloc_bulk_socket
> rte_fastmem_free / free_bulk
> rte_fastmem_virt2iova
> rte_fastmem_cache_flush
> rte_fastmem_max_size / classes
> rte_fastmem_stats / stats_class / stats_lcore / stats_lcore_class
> rte_fastmem_stats_reset
>
> All APIs are marked __rte_experimental.
>
> Performance
> -----------
>
> The single-object hot path is roughly 2-3x the cost of mempool
> and an order of magnitude faster than rte_malloc. Under
> multi-lcore contention, fastmem scales similarly to mempool,
> while rte_malloc collapses.
>
> Limitations
> -----------
>
> - Maximum allocation: 1 MiB. Larger requests should use rte_malloc.
> - Power-of-2 classes only; worst-case internal fragmentation ~50%.
> - Backing memory not reclaimable short of deinit.
>
> Future work
> -----------
>
> - Lcore-affine allocations (false-sharing-free by construction).
> - Mempool ops driver for transparent drop-in use.
> - Pre-resolved allocator handle binding size class and socket,
> eliminating per-call class lookup and enabling an inline
> cache-hit fast path.
> - Debug mode (cookies, double-free detection, poison-on-free).
> - Telemetry integration.
> - EAL integration, allowing EAL-internal subsystems to use
> fastmem for their small-object allocations.
>
> Mattias Rönnblom (3):
> doc: add fastmem programming guide
> lib: add fastmem library
> app/test: add fastmem test suite
>
> app/test/meson.build | 3 +
> app/test/test_fastmem.c | 1682 +++++++++++++++++++++++++
> app/test/test_fastmem_perf.c | 997 +++++++++++++++
> app/test/test_fastmem_profile.c | 157 +++
> doc/api/doxy-api-index.md | 1 +
> doc/api/doxy-api.conf.in | 1 +
> doc/guides/prog_guide/fastmem_lib.rst | 301 +++++
> doc/guides/prog_guide/index.rst | 1 +
> lib/fastmem/meson.build | 6 +
> lib/fastmem/rte_fastmem.c | 1486 ++++++++++++++++++++++
> lib/fastmem/rte_fastmem.h | 644 ++++++++++
> lib/meson.build | 1 +
> 12 files changed, 5280 insertions(+)
> create mode 100644 app/test/test_fastmem.c
> create mode 100644 app/test/test_fastmem_perf.c
> create mode 100644 app/test/test_fastmem_profile.c
> create mode 100644 doc/guides/prog_guide/fastmem_lib.rst
> create mode 100644 lib/fastmem/meson.build
> create mode 100644 lib/fastmem/rte_fastmem.c
> create mode 100644 lib/fastmem/rte_fastmem.h
>
Largish patchset so did AI review with full claude model.
Series review: [RFC 0/3] add fastmem allocator
Reviewed against the v1 RFC posted 2026-05-25.
[RFC 1/3] doc: add fastmem programming guide
Info: doc/guides/prog_guide/fastmem_lib.rst -- "\ No newline at end of file"
The new RST file does not end with a newline.
[RFC 2/3] lib: add fastmem library
Error: lib/fastmem/rte_fastmem.c -- use-after-free during rte_fastmem_deinit()
when caches were allocated cross-socket.
cache_create() places the cache struct on the *calling thread's* socket,
not on the socket the cache serves:
unsigned int own_socket = rte_socket_id();
...
alloc_socket = &fastmem->sockets[own_socket];
cache = bin_alloc_one(&alloc_socket->bins[cache_class]);
...
*slot = cache; /* slot is in socket K's caches[][] */
So an lcore on socket S that calls rte_fastmem_alloc_socket(..., K) with
S != K creates a cache whose memory lives in socket S's memzone but is
reachable through socket K's caches[lcore][class].
rte_fastmem_deinit() then walks sockets in index order:
for (i = 0; i < RTE_MAX_NUMA_NODES; i++)
release_socket(&fastmem->sockets[i]);
and release_socket() does, in this order:
socket_release_caches(socket); /* (1) */
for (c...) bin_release(&socket->bins[c], socket); /* (2) */
for (i...) rte_memzone_free(socket->memzones[i]); /* (3) */
When i = S, step (3) frees socket S's memzones. When i = K (K > S),
socket_release_caches(K) runs:
cache_slab = slab_of(cache); /* in socket S's freed mz */
bin_free_one(cache_slab->bin, cache); /* reads cache_slab->bin */
cache_slab points into a freed memzone, so cache_slab->bin and the
subsequent push (slab->free_head = obj; slab->free_count++; in
bin_push_locked()) read and write released memory. slab_release() may
then re-attach the slab to socket S's free_head, which was zeroed and
whose backing is gone.
This is triggered by any application that allocates from a non-local
socket via SOCKET_ID_ANY fallback or explicit socket_id, which the
programming guide describes as a normal mode of operation. The
existing test_alloc_socket and test_alloc_socket_numa_placement use
rte_socket_id_by_idx(0) (the local socket) so the bug is not
exercised by the test suite.
Either order the teardown in three phases (all caches across all
sockets first, then all bins, then all memzones), or allocate the
cache struct from the socket it serves rather than the calling
thread's socket.
Warning: lib/fastmem/rte_fastmem.c -- non-atomic access to shared 64-bit
statistics counters.
cache->alloc_cache_hits, alloc_cache_misses, alloc_nomem,
free_cache_hits, free_cache_misses, and the bin counters
slab_acquires, slab_releases, slabs_partial, slabs_full are
incremented as plain C reads/writes by the owning lcore and read
from another thread via rte_fastmem_stats(), rte_fastmem_stats_class(),
rte_fastmem_stats_lcore(), and rte_fastmem_stats_lcore_class(). On
architectures where uint64_t is not naturally atomic (and per the C
standard generally) this is a data race; even on x86-64 it is
undefined behavior under -fsanitize=thread.
Use rte_atomic_fetch_add_explicit() with rte_memory_order_relaxed on
the producer side and rte_atomic_load_explicit() with relaxed
ordering on the reader side. Per AGENTS.md / the DPDK convention,
relaxed ordering is appropriate for these counters.
Warning: lib/fastmem/rte_fastmem.c -- pointer publish in cache_create()
without release ordering.
*slot = cache;
return cache;
The struct fields (count, capacity, target, the stats counters) are
written before this store but with no fence or release barrier. A
concurrent stats reader doing socket->caches[l][c] followed by
cache->* could observe the pointer but not all initialized fields.
Even ignoring the stats reader, rte_fastmem_cache_flush() invoked
from a different lcore on the same cache (not currently possible by
API contract, but the field is technically reachable) would race.
Pair with rte_atomic_store_explicit(..., rte_memory_order_release)
and a matching acquire load on the reader path.
Warning: lib/fastmem/rte_fastmem.c -- spurious ENOMEM window during slab
release.
bin_push_locked() removes a fully-drained slab from bin->partial
before bin_free_one() drops the bin lock; slab_release() then puts
it on socket->free_head under the socket lock. Between the unlock
and slab_release(), another lcore allocating in any class on the
same socket can see free_head == NULL, hit the memory_limit (or
FASTMEM_MAX_MEMZONES_PER_SOCKET) check in grow_socket(), and return
ENOMEM even though the slab is about to become available. Not a
correctness issue but visible to applications that pin tightly to
their limit.
Info: lib/fastmem/rte_fastmem.c local_socket_id() final fallback:
return (unsigned int)rte_socket_id_by_idx(0);
rte_socket_id_by_idx() returns int and is documented to return -1 on
error. If there are zero configured sockets the cast yields UINT_MAX
and fastmem->sockets[UINT_MAX] is out of bounds. Realistically there
is always at least one socket, but a defensive check (return 0, or
fail allocation explicitly) would avoid the corner case.
Info: lib/fastmem/rte_fastmem.c cache_pop() refills to cache->target
(half capacity) rather than to capacity. Subsequent single-object
allocs only get target-1 hits before the next bin trip. Likely
intentional for fairness with bulk callers, but worth a comment.
Info: lib/meson.build inserts 'fastmem' between 'dispatcher' and
'gpudev'. The natural alphabetical position is between 'efd' and
'fib'; fastmem has no dependency on dispatcher.
[RFC 3/3] app/test: add fastmem test suite
Warning: app/test/test_fastmem.c -- REGISTER_FAST_TEST uses NOHUGE_OK
but the functional tests need real memzone-backed memory.
REGISTER_FAST_TEST(fastmem_autotest, NOHUGE_OK, ASAN_OK,
test_fastmem);
test_fastmem runs both the lifecycle suite (no allocations) and the
functional suite, which requests 128 MiB IOVA-contiguous memzones.
In --no-huge mode IOVA-contiguous reservation of that size is not
reliable, so NOHUGE_SKIP is more honest. If you want the lifecycle
tests to remain no-huge-friendly, register them as a separate
test command.
Warning: app/test/test_fastmem.c -- the suite never exercises
cross-socket cache allocation.
test_alloc_socket and test_alloc_socket_numa_placement both use
rte_socket_id_by_idx(0) (the local socket). Add a test that runs on
a worker lcore whose rte_socket_id() differs from the target
socket_id passed to rte_fastmem_alloc_socket(), then calls
rte_fastmem_deinit(). This would have caught the deinit UAF above.
Info: app/test/test_fastmem.c -- several test functions declare an
uninitialized `int rc;` that is never read or written (e.g.
test_alloc_too_big, test_alloc_invalid_align, test_alloc_free_small,
test_alloc_alignment, test_alloc_socket, test_alloc_block_repurposing
and others). Drop the declarations.
Info: app/test/test_fastmem.c trailing blank-line clusters (two blank
lines before "return TEST_SUCCESS;" in test_reserve_multiple_memzones,
test_reserve_cumulative, test_reserve_invalid_socket,
test_reserve_any_socket, test_alloc_too_big, ...). Drop the extra
blank line.
^ permalink raw reply
* Re: [PATCH 2/2] eal/freebsd: implement virtual to IOVA translation fn
From: Lewis Donzis @ 2026-05-25 18:23 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
In-Reply-To: <20260525171340.1701509-3-bruce.richardson@intel.com>
Tested with igc driver, which previously failed and now works properly.
----- On May 25, 2026, at 12:13 PM, Bruce Richardson bruce.richardson@intel.com wrote:
> The function rte_mem_virt2iova() function was always returning BAD_IOVA
> on FreeBSD as it was not implemented. Unfortunately, this function was
> used by some drivers - expecting it to work as they used addresses from
> rte_malloc calls. Implement this call on FreeBSD to fix things and
> enable more drivers on BSD. To verify the function works as expected
> add a basic unit test to the memory_autotest for it.
>
> At the same time, we can trivially use this to implement the
> rte_mem_virt2phy function for the case of using contigmem where IOVAs
> are physical addresses.
>
> Reported-by: Lewis Donzis <lew@perftech.com>
Tested-by: Lewis Donzis <lew@perftech.com>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>
> NOTE: This is one of those patches that could be either a bugfix or a
> feature. For now, I'm treating it as adding new feature support, though
> in doing so it enables previously broken drivers.
> ---
> app/test/test_memory.c | 67 ++++++++++++++++++++++++++++++++++++
> lib/eal/freebsd/eal_memory.c | 22 +++++++++---
> 2 files changed, 84 insertions(+), 5 deletions(-)
>
> diff --git a/app/test/test_memory.c b/app/test/test_memory.c
> index 628714c0b8..260d5ed8f8 100644
> --- a/app/test/test_memory.c
> +++ b/app/test/test_memory.c
> @@ -8,6 +8,7 @@
> #include <rte_eal.h>
> #include <rte_errno.h>
> #include <rte_memory.h>
> +#include <rte_malloc.h>
> #include <rte_common.h>
> #include <rte_memzone.h>
>
> @@ -75,6 +76,68 @@ check_seg_fds(const struct rte_memseg_list *msl, const struct
> rte_memseg *ms,
> return 0;
> }
>
> +static int
> +check_malloc_virt2iova(void)
> +{
> + const size_t alloc_sz = 128;
> + const size_t off = 32;
> + struct rte_memseg *ms;
> + char *p;
> + rte_iova_t iova, iova_off;
> +
> + p = rte_malloc("memory_autotest", alloc_sz, RTE_CACHE_LINE_SIZE);
> + if (p == NULL) {
> + printf("rte_malloc failed\n");
> + return -1;
> + }
> +
> + iova = rte_mem_virt2iova(p);
> + if (iova == RTE_BAD_IOVA) {
> + printf("rte_mem_virt2iova failed for rte_malloc pointer\n");
> + rte_free(p);
> + return -1;
> + }
> +
> + ms = rte_mem_virt2memseg(p, NULL);
> + if (ms == NULL) {
> + printf("failed to resolve memseg for rte_malloc pointer\n");
> + rte_free(p);
> + return -1;
> + }
> +
> + if (rte_eal_iova_mode() == RTE_IOVA_PA) {
> + if (ms->iova == RTE_BAD_IOVA || iova < ms->iova ||
> + iova >= ms->iova + ms->len) {
> + printf("translated iova is outside memseg range\n");
> + rte_free(p);
> + return -1;
> + }
> +
> + phys_addr_t physaddr = rte_mem_virt2phy(p);
> + if (physaddr == RTE_BAD_PHYS_ADDR || physaddr != iova) {
> + printf("rte_mem_virt2phy failed for rte_malloc pointer\n");
> + rte_free(p);
> + return -1;
> + }
> + } else if (rte_eal_iova_mode() == RTE_IOVA_VA) {
> + if (iova != (uintptr_t)p) {
> + printf("rte_mem_virt2iova did not return VA in VA mode\n");
> + rte_free(p);
> + return -1;
> + }
> + }
> +
> + iova_off = rte_mem_virt2iova(p + off);
> + if (iova_off == RTE_BAD_IOVA || iova_off != iova + off) {
> + printf("translated iova for interior pointer is invalid\n");
> + rte_free(p);
> + return -1;
> + }
> +
> + rte_free(p);
> + return 0;
> +}
> +
> static int
> test_memory(void)
> {
> @@ -107,6 +170,10 @@ test_memory(void)
> return -1;
> }
>
> + ret = check_malloc_virt2iova();
> + if (ret < 0)
> + return ret;
> +
> return 0;
> }
>
> diff --git a/lib/eal/freebsd/eal_memory.c b/lib/eal/freebsd/eal_memory.c
> index e56d149a22..6952b38630 100644
> --- a/lib/eal/freebsd/eal_memory.c
> +++ b/lib/eal/freebsd/eal_memory.c
> @@ -40,16 +40,28 @@ RTE_EXPORT_SYMBOL(rte_mem_virt2phy)
> phys_addr_t
> rte_mem_virt2phy(const void *virtaddr)
> {
> - /* XXX not implemented. This function is only used by
> - * rte_mempool_virt2iova() when hugepages are disabled. */
> - (void)virtaddr;
> - return RTE_BAD_IOVA;
> + /* not implemented for FreeBSD when not using contigmem memory */
> + if (virtaddr == NULL || rte_eal_iova_mode() != RTE_IOVA_PA)
> + return RTE_BAD_IOVA;
> + /* when IOVA == PA, return the IOVA */
> + return rte_mem_virt2iova(virtaddr);
> }
> +
> RTE_EXPORT_SYMBOL(rte_mem_virt2iova)
> rte_iova_t
> rte_mem_virt2iova(const void *virtaddr)
> {
> - return rte_mem_virt2phy(virtaddr);
> + if (virtaddr == NULL)
> + return RTE_BAD_IOVA;
> +
> + if (rte_eal_iova_mode() == RTE_IOVA_VA)
> + return (uintptr_t)virtaddr;
> +
> + const struct rte_memseg *ms = rte_mem_virt2memseg(virtaddr, NULL);
> + if (ms != NULL && ms->iova != RTE_BAD_IOVA)
> + return ms->iova + RTE_PTR_DIFF(virtaddr, ms->addr);
> +
> + return RTE_BAD_IOVA;
> }
>
> int
> --
> 2.53.0
^ permalink raw reply
* [PATCH 2/2] eal/freebsd: implement virtual to IOVA translation fn
From: Bruce Richardson @ 2026-05-25 17:13 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Lewis Donzis
In-Reply-To: <20260525171340.1701509-1-bruce.richardson@intel.com>
The function rte_mem_virt2iova() function was always returning BAD_IOVA
on FreeBSD as it was not implemented. Unfortunately, this function was
used by some drivers - expecting it to work as they used addresses from
rte_malloc calls. Implement this call on FreeBSD to fix things and
enable more drivers on BSD. To verify the function works as expected
add a basic unit test to the memory_autotest for it.
At the same time, we can trivially use this to implement the
rte_mem_virt2phy function for the case of using contigmem where IOVAs
are physical addresses.
Reported-by: Lewis Donzis <lew@perftech.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
NOTE: This is one of those patches that could be either a bugfix or a
feature. For now, I'm treating it as adding new feature support, though
in doing so it enables previously broken drivers.
---
app/test/test_memory.c | 67 ++++++++++++++++++++++++++++++++++++
lib/eal/freebsd/eal_memory.c | 22 +++++++++---
2 files changed, 84 insertions(+), 5 deletions(-)
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 628714c0b8..260d5ed8f8 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -8,6 +8,7 @@
#include <rte_eal.h>
#include <rte_errno.h>
#include <rte_memory.h>
+#include <rte_malloc.h>
#include <rte_common.h>
#include <rte_memzone.h>
@@ -75,6 +76,68 @@ check_seg_fds(const struct rte_memseg_list *msl, const struct rte_memseg *ms,
return 0;
}
+static int
+check_malloc_virt2iova(void)
+{
+ const size_t alloc_sz = 128;
+ const size_t off = 32;
+ struct rte_memseg *ms;
+ char *p;
+ rte_iova_t iova, iova_off;
+
+ p = rte_malloc("memory_autotest", alloc_sz, RTE_CACHE_LINE_SIZE);
+ if (p == NULL) {
+ printf("rte_malloc failed\n");
+ return -1;
+ }
+
+ iova = rte_mem_virt2iova(p);
+ if (iova == RTE_BAD_IOVA) {
+ printf("rte_mem_virt2iova failed for rte_malloc pointer\n");
+ rte_free(p);
+ return -1;
+ }
+
+ ms = rte_mem_virt2memseg(p, NULL);
+ if (ms == NULL) {
+ printf("failed to resolve memseg for rte_malloc pointer\n");
+ rte_free(p);
+ return -1;
+ }
+
+ if (rte_eal_iova_mode() == RTE_IOVA_PA) {
+ if (ms->iova == RTE_BAD_IOVA || iova < ms->iova ||
+ iova >= ms->iova + ms->len) {
+ printf("translated iova is outside memseg range\n");
+ rte_free(p);
+ return -1;
+ }
+
+ phys_addr_t physaddr = rte_mem_virt2phy(p);
+ if (physaddr == RTE_BAD_PHYS_ADDR || physaddr != iova) {
+ printf("rte_mem_virt2phy failed for rte_malloc pointer\n");
+ rte_free(p);
+ return -1;
+ }
+ } else if (rte_eal_iova_mode() == RTE_IOVA_VA) {
+ if (iova != (uintptr_t)p) {
+ printf("rte_mem_virt2iova did not return VA in VA mode\n");
+ rte_free(p);
+ return -1;
+ }
+ }
+
+ iova_off = rte_mem_virt2iova(p + off);
+ if (iova_off == RTE_BAD_IOVA || iova_off != iova + off) {
+ printf("translated iova for interior pointer is invalid\n");
+ rte_free(p);
+ return -1;
+ }
+
+ rte_free(p);
+ return 0;
+}
+
static int
test_memory(void)
{
@@ -107,6 +170,10 @@ test_memory(void)
return -1;
}
+ ret = check_malloc_virt2iova();
+ if (ret < 0)
+ return ret;
+
return 0;
}
diff --git a/lib/eal/freebsd/eal_memory.c b/lib/eal/freebsd/eal_memory.c
index e56d149a22..6952b38630 100644
--- a/lib/eal/freebsd/eal_memory.c
+++ b/lib/eal/freebsd/eal_memory.c
@@ -40,16 +40,28 @@ RTE_EXPORT_SYMBOL(rte_mem_virt2phy)
phys_addr_t
rte_mem_virt2phy(const void *virtaddr)
{
- /* XXX not implemented. This function is only used by
- * rte_mempool_virt2iova() when hugepages are disabled. */
- (void)virtaddr;
- return RTE_BAD_IOVA;
+ /* not implemented for FreeBSD when not using contigmem memory */
+ if (virtaddr == NULL || rte_eal_iova_mode() != RTE_IOVA_PA)
+ return RTE_BAD_IOVA;
+ /* when IOVA == PA, return the IOVA */
+ return rte_mem_virt2iova(virtaddr);
}
+
RTE_EXPORT_SYMBOL(rte_mem_virt2iova)
rte_iova_t
rte_mem_virt2iova(const void *virtaddr)
{
- return rte_mem_virt2phy(virtaddr);
+ if (virtaddr == NULL)
+ return RTE_BAD_IOVA;
+
+ if (rte_eal_iova_mode() == RTE_IOVA_VA)
+ return (uintptr_t)virtaddr;
+
+ const struct rte_memseg *ms = rte_mem_virt2memseg(virtaddr, NULL);
+ if (ms != NULL && ms->iova != RTE_BAD_IOVA)
+ return ms->iova + RTE_PTR_DIFF(virtaddr, ms->addr);
+
+ return RTE_BAD_IOVA;
}
int
--
2.53.0
^ permalink raw reply related
* [PATCH 1/2] eal/freebsd: fix memseg addresses in EAL VA mode
From: Bruce Richardson @ 2026-05-25 17:13 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, stable
In-Reply-To: <20260525171340.1701509-1-bruce.richardson@intel.com>
In BSD, when mapping the contigmem segments, the iova is unconditionally
set to the physical address, irrespective of the actual IOVA mode.
Change this to use virtual addresses when iova mode is IOVA_AS_VA.
Fixes: 764bf26873b9 ("add FreeBSD support")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
lib/eal/freebsd/eal_memory.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/eal/freebsd/eal_memory.c b/lib/eal/freebsd/eal_memory.c
index cd608db9f9..e56d149a22 100644
--- a/lib/eal/freebsd/eal_memory.c
+++ b/lib/eal/freebsd/eal_memory.c
@@ -214,7 +214,10 @@ rte_eal_hugepage_init(void)
}
seg->addr = addr;
- seg->iova = physaddr;
+ if (rte_eal_iova_mode() == RTE_IOVA_VA)
+ seg->iova = (uintptr_t)addr;
+ else
+ seg->iova = physaddr;
seg->hugepage_sz = page_sz;
seg->len = page_sz;
seg->nchannel = mcfg->nchannel;
--
2.53.0
^ permalink raw reply related
* [PATCH 0/2] implement missing APIs on FreeBSD
From: Bruce Richardson @ 2026-05-25 17:13 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
The APIs for converting a virtual address to an IOVA or a physical
address were implemented only as stubs on FreeBSD, but unfortunately
they are used by drivers such as "igc", part of e1000, causing the
drivers to fail on BSD. A workaround would be to change the use of these
functions with the rte_malloc equivalents (e.g. rte_malloc_virt2iova) -
which do work on FreeBSD - but a better solution is just to replace the
stubs with proper implementations in FreeBSD-specific EAL flags.
Bruce Richardson (2):
eal/freebsd: fix memseg addresses in IOVA_AS_VA mode
eal/freebsd: implement virtual to IOVA translation fn
app/test/test_memory.c | 67 ++++++++++++++++++++++++++++++++++++
lib/eal/freebsd/eal_memory.c | 27 +++++++++++----
2 files changed, 88 insertions(+), 6 deletions(-)
--
2.53.0
^ permalink raw reply
* RE: [PATCH v3 03/27] ring: use compare-and-swap wrapper
From: Morten Brørup @ 2026-05-25 15:47 UTC (permalink / raw)
To: Stephen Hemminger, Konstantin Ananyev; +Cc: dev
In-Reply-To: <20260525083509.00cf76a1@phoenix.local>
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Monday, 25 May 2026 17.35
>
> On Mon, 25 May 2026 07:41:13 +0000
> Konstantin Ananyev <konstantin.ananyev@huawei.com> wrote:
>
> > Hi Stephen,
> >
> > > The rte_atomic32_cmpset is deprecated. Initial attempts at
> > > changing this with direct conversion to
> > > rte_atomic_compare_exchange_weak_explicit()
> > > regressed MP/MC contended performance on x86 by 10-30%,
> > > because the C11 builtin's failure-writeback semantic forces
> > > GCC to emit extra instructions on the CAS critical path.
> > >
> > > Add an internal __rte_ring_compare_and_swap() wrapper that calls
> > > __sync_bool_compare_and_swap() directly, which keeps the original
> > > instruction sequence. Add equivalent function for MSVC.
> >
> > In fact, in rte_ring we do have 2 implementations of the same core
> functions:
> > lib/ring/rte_ring_c11_pvt.h - uses C11 atomics
> > lib/ring/rte_ring_generic_pvt.h - uses legacy instructions (smp_mb,
> extra),
> > If we going remove these legacy instructions anyway (or
> reimplementing them using C11 atomics),
> > then there is probably no point to keep rte_ring_generic_pvt.h.
> > Konstantin
>
> Have been deep diving into why C11 atomics give 20-30% performance
> drop versus atomic32 version. So far it comes down to GCC optimizer
> not doing as well with C11 versus assembly. The C11 form with the
> excessive use of always_inline consumes more registers.
Just an idea:
Perhaps adding "const" and/or "restrict" to relevant parameters will give the optimizer the information it needs?
^ permalink raw reply
* Re: [PATCH v3 03/27] ring: use compare-and-swap wrapper
From: Stephen Hemminger @ 2026-05-25 15:35 UTC (permalink / raw)
To: Konstantin Ananyev; +Cc: dev@dpdk.org
In-Reply-To: <b827bdd2f5994f24b9806e5cbae82cd3@huawei.com>
On Mon, 25 May 2026 07:41:13 +0000
Konstantin Ananyev <konstantin.ananyev@huawei.com> wrote:
> Hi Stephen,
>
> > The rte_atomic32_cmpset is deprecated. Initial attempts at
> > changing this with direct conversion to
> > rte_atomic_compare_exchange_weak_explicit()
> > regressed MP/MC contended performance on x86 by 10-30%,
> > because the C11 builtin's failure-writeback semantic forces
> > GCC to emit extra instructions on the CAS critical path.
> >
> > Add an internal __rte_ring_compare_and_swap() wrapper that calls
> > __sync_bool_compare_and_swap() directly, which keeps the original
> > instruction sequence. Add equivalent function for MSVC.
>
> In fact, in rte_ring we do have 2 implementations of the same core functions:
> lib/ring/rte_ring_c11_pvt.h - uses C11 atomics
> lib/ring/rte_ring_generic_pvt.h - uses legacy instructions (smp_mb, extra),
> If we going remove these legacy instructions anyway (or reimplementing them using C11 atomics),
> then there is probably no point to keep rte_ring_generic_pvt.h.
> Konstantin
Have been deep diving into why C11 atomics give 20-30% performance
drop versus atomic32 version. So far it comes down to GCC optimizer
not doing as well with C11 versus assembly. The C11 form with the
excessive use of always_inline consumes more registers.
^ permalink raw reply
* Re: [PATCH v3 03/27] ring: use compare-and-swap wrapper
From: Stephen Hemminger @ 2026-05-25 14:31 UTC (permalink / raw)
To: Konstantin Ananyev; +Cc: dev@dpdk.org
In-Reply-To: <b827bdd2f5994f24b9806e5cbae82cd3@huawei.com>
On Mon, 25 May 2026 07:41:13 +0000
Konstantin Ananyev <konstantin.ananyev@huawei.com> wrote:
> In fact, in rte_ring we do have 2 implementations of the same core functions:
> lib/ring/rte_ring_c11_pvt.h - uses C11 atomics
> lib/ring/rte_ring_generic_pvt.h - uses legacy instructions (smp_mb, extra),
> If we going remove these legacy instructions anyway (or reimplementing them using C11 atomics),
> then there is probably no point to keep rte_ring_generic_pvt.h.
> Konstantin
Good point will try heading that way.
^ permalink raw reply
* Re: [RFC 0/3] lib/fastmem: fast small-object allocator
From: Stephen Hemminger @ 2026-05-25 14:30 UTC (permalink / raw)
To: Mattias Rönnblom
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <20260525103642.55255-1-hofors@lysator.liu.se>
On Mon, 25 May 2026 12:36:39 +0200
Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> This RFC introduces fastmem, a general-purpose small-object allocator
> for DPDK. It is intended to replace per-type mempools with a single
> allocator that handles arbitrary sizes, grows on demand, and matches
> mempool-level performance on the hot path.
Makes sense, what a simple wrapper inline to allow full replacement
testing/performance A/B comparison?
^ permalink raw reply
* [PATCH v5 27/27] net/ice: use common action checks for ACL
From: Anatoly Burakov @ 2026-05-25 14:06 UTC (permalink / raw)
To: dev, Bruce Richardson
In-Reply-To: <cover.1779717943.git.anatoly.burakov@intel.com>
From: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Use the common flow action checking parsing infrastructure for checking
flow actions for ACL filter.
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
---
drivers/net/intel/ice/ice_acl_filter.c | 143 +++++++++++++++----------
1 file changed, 84 insertions(+), 59 deletions(-)
diff --git a/drivers/net/intel/ice/ice_acl_filter.c b/drivers/net/intel/ice/ice_acl_filter.c
index 0b50f06c46..30ff1254c9 100644
--- a/drivers/net/intel/ice/ice_acl_filter.c
+++ b/drivers/net/intel/ice/ice_acl_filter.c
@@ -645,60 +645,6 @@ ice_acl_filter_free(struct rte_flow *flow)
flow->rule = NULL;
}
-static int
-ice_acl_parse_action(__rte_unused struct ice_adapter *ad,
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct ice_acl_conf *filter)
-{
- struct ice_pf *pf = &ad->pf;
- const struct rte_flow_action_queue *act_q;
- uint32_t dest_num = 0;
-
- for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) {
- switch (actions->type) {
- case RTE_FLOW_ACTION_TYPE_VOID:
- break;
- case RTE_FLOW_ACTION_TYPE_DROP:
- dest_num++;
-
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DROP_PKT;
- break;
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- dest_num++;
-
- act_q = actions->conf;
- filter->input.q_index = act_q->index;
- if (filter->input.q_index >=
- pf->dev_data->nb_rx_queues) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid queue for FDIR.");
- return -rte_errno;
- }
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QINDEX;
- break;
- default:
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Invalid action.");
- return -rte_errno;
- }
- }
-
- if (dest_num == 0 || dest_num >= 2) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Unsupported action combination");
- return -rte_errno;
- }
-
- return 0;
-}
-
static int
ice_acl_parse_pattern(__rte_unused struct ice_adapter *ad,
const struct rte_flow_item pattern[],
@@ -966,6 +912,69 @@ ice_acl_parse_pattern(__rte_unused struct ice_adapter *ad,
return 0;
}
+static int
+ice_acl_parse_action(const struct ci_flow_actions *actions,
+ struct ice_acl_conf *filter,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action *act = actions->actions[0];
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DROP_PKT;
+ break;
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q = act->conf;
+
+ filter->input.q_index = act_q->index;
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QINDEX;
+ break;
+ }
+ default:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action.");
+ }
+
+ return 0;
+}
+
+static int
+ice_acl_parse_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ struct ice_adapter *ad = param->driver_ctx;
+ struct ice_pf *pf = &ad->pf;
+ const struct rte_flow_action *act = actions->actions[0];
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ break;
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q = act->conf;
+
+ if (act_q->index >= pf->dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid queue for ACL.");
+ }
+ break;
+ }
+ default:
+ /* shouldn't happen */
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action.");
+ }
+
+ return 0;
+}
+
static int
ice_acl_parse(struct ice_adapter *ad,
struct ice_pattern_match_item *array,
@@ -976,17 +985,33 @@ ice_acl_parse(struct ice_adapter *ad,
void **meta,
struct rte_flow_error *error)
{
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]){
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .check = ice_acl_parse_action_check,
+ .driver_ctx = ad,
+ };
struct ice_pf *pf = &ad->pf;
struct ice_acl_conf *filter = &pf->acl.conf;
struct ice_pattern_match_item *item = NULL;
uint64_t input_set;
int ret;
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
memset(filter, 0, sizeof(*filter));
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret)
+ return ret;
+
item = ice_search_pattern_match_item(ad, pattern, array, array_len,
error);
if (!item)
@@ -1005,7 +1030,7 @@ ice_acl_parse(struct ice_adapter *ad,
goto error;
}
- ret = ice_acl_parse_action(ad, actions, error, filter);
+ ret = ice_acl_parse_action(&parsed_actions, filter, error);
if (ret)
goto error;
--
2.47.3
^ permalink raw reply related
* [PATCH v5 26/27] net/ice: use common action checks for switch
From: Anatoly Burakov @ 2026-05-25 14:06 UTC (permalink / raw)
To: dev, Bruce Richardson
In-Reply-To: <cover.1779717943.git.anatoly.burakov@intel.com>
From: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Use the common flow action checking parsing infrastructure for checking
flow actions for switch filter.
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
---
drivers/net/intel/ice/ice_switch_filter.c | 371 +++++++++++-----------
1 file changed, 185 insertions(+), 186 deletions(-)
diff --git a/drivers/net/intel/ice/ice_switch_filter.c b/drivers/net/intel/ice/ice_switch_filter.c
index 67943b591c..5d9e0062af 100644
--- a/drivers/net/intel/ice/ice_switch_filter.c
+++ b/drivers/net/intel/ice/ice_switch_filter.c
@@ -35,6 +35,8 @@
#define ICE_IPV4_PROTO_NVGRE 0x002F
#define ICE_SW_PRI_BASE 6
+#define ICE_SW_MAX_QUEUES 128
+
#define ICE_SW_INSET_ETHER ( \
ICE_INSET_DMAC | ICE_INSET_SMAC | ICE_INSET_ETHERTYPE)
#define ICE_SW_INSET_MAC_VLAN ( \
@@ -1527,85 +1529,38 @@ ice_switch_parse_pattern(const struct rte_flow_item pattern[],
}
static int
-ice_switch_parse_dcf_action(struct ice_dcf_adapter *ad,
- const struct rte_flow_action *actions,
+ice_switch_parse_dcf_action(const struct rte_flow_action *action,
uint32_t priority,
struct rte_flow_error *error,
struct ice_adv_rule_info *rule_info)
{
const struct rte_flow_action_ethdev *act_ethdev;
- const struct rte_flow_action *action;
const struct rte_eth_dev *repr_dev;
enum rte_flow_action_type action_type;
- uint16_t rule_port_id, backer_port_id;
- for (action = actions; action->type !=
- RTE_FLOW_ACTION_TYPE_END; action++) {
- action_type = action->type;
- switch (action_type) {
- case RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR:
- rule_info->sw_act.fltr_act = ICE_FWD_TO_VSI;
- act_ethdev = action->conf;
-
- if (!rte_eth_dev_is_valid_port(act_ethdev->port_id))
- goto invalid_port_id;
-
- /* For traffic to original DCF port */
- rule_port_id = ad->parent.pf.dev_data->port_id;
-
- if (rule_port_id != act_ethdev->port_id)
- goto invalid_port_id;
-
- rule_info->sw_act.vsi_handle = 0;
-
- break;
-
-invalid_port_id:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid port_id");
- return -rte_errno;
-
- case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
- rule_info->sw_act.fltr_act = ICE_FWD_TO_VSI;
- act_ethdev = action->conf;
-
- if (!rte_eth_dev_is_valid_port(act_ethdev->port_id))
- goto invalid;
-
- repr_dev = &rte_eth_devices[act_ethdev->port_id];
-
- if (!repr_dev->data)
- goto invalid;
-
- rule_port_id = ad->parent.pf.dev_data->port_id;
- backer_port_id = repr_dev->data->backer_port_id;
-
- if (backer_port_id != rule_port_id)
- goto invalid;
-
- rule_info->sw_act.vsi_handle = repr_dev->data->representor_id;
- break;
-
-invalid:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid ethdev_port_id");
- return -rte_errno;
-
- case RTE_FLOW_ACTION_TYPE_DROP:
- rule_info->sw_act.fltr_act = ICE_DROP_PACKET;
- break;
-
- default:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid action type");
- return -rte_errno;
- }
+ action_type = action->type;
+ switch (action_type) {
+ case RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR:
+ rule_info->sw_act.fltr_act = ICE_FWD_TO_VSI;
+ rule_info->sw_act.vsi_handle = 0;
+ break;
+
+ case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
+ rule_info->sw_act.fltr_act = ICE_FWD_TO_VSI;
+ act_ethdev = action->conf;
+ repr_dev = &rte_eth_devices[act_ethdev->port_id];
+ rule_info->sw_act.vsi_handle = repr_dev->data->representor_id;
+ break;
+
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ rule_info->sw_act.fltr_act = ICE_DROP_PACKET;
+ break;
+
+ default:
+ /* Should never reach */
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Invalid action type");
+ return -rte_errno;
}
rule_info->sw_act.src = rule_info->sw_act.vsi_handle;
@@ -1621,73 +1576,38 @@ ice_switch_parse_dcf_action(struct ice_dcf_adapter *ad,
static int
ice_switch_parse_action(struct ice_pf *pf,
- const struct rte_flow_action *actions,
+ const struct rte_flow_action *action,
uint32_t priority,
struct rte_flow_error *error,
struct ice_adv_rule_info *rule_info)
{
struct ice_vsi *vsi = pf->main_vsi;
- struct rte_eth_dev_data *dev_data = pf->adapter->pf.dev_data;
const struct rte_flow_action_queue *act_q;
const struct rte_flow_action_rss *act_qgrop;
- uint16_t base_queue, i;
- const struct rte_flow_action *action;
+ uint16_t base_queue;
enum rte_flow_action_type action_type;
- uint16_t valid_qgrop_number[MAX_QGRP_NUM_TYPE] = {
- 2, 4, 8, 16, 32, 64, 128};
base_queue = pf->base_queue + vsi->base_queue;
- for (action = actions; action->type !=
- RTE_FLOW_ACTION_TYPE_END; action++) {
- action_type = action->type;
- switch (action_type) {
- case RTE_FLOW_ACTION_TYPE_RSS:
- act_qgrop = action->conf;
- if (act_qgrop->queue_num <= 1)
- goto error;
- rule_info->sw_act.fltr_act =
- ICE_FWD_TO_QGRP;
- rule_info->sw_act.fwd_id.q_id =
- base_queue + act_qgrop->queue[0];
- for (i = 0; i < MAX_QGRP_NUM_TYPE; i++) {
- if (act_qgrop->queue_num ==
- valid_qgrop_number[i])
- break;
- }
- if (i == MAX_QGRP_NUM_TYPE)
- goto error;
- if ((act_qgrop->queue[0] +
- act_qgrop->queue_num) >
- dev_data->nb_rx_queues)
- goto error1;
- for (i = 0; i < act_qgrop->queue_num - 1; i++)
- if (act_qgrop->queue[i + 1] !=
- act_qgrop->queue[i] + 1)
- goto error2;
- rule_info->sw_act.qgrp_size =
- act_qgrop->queue_num;
- break;
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- act_q = action->conf;
- if (act_q->index >= dev_data->nb_rx_queues)
- goto error;
- rule_info->sw_act.fltr_act =
- ICE_FWD_TO_Q;
- rule_info->sw_act.fwd_id.q_id =
- base_queue + act_q->index;
- break;
-
- case RTE_FLOW_ACTION_TYPE_DROP:
- rule_info->sw_act.fltr_act =
- ICE_DROP_PACKET;
- break;
-
- case RTE_FLOW_ACTION_TYPE_VOID:
- break;
-
- default:
- goto error;
- }
+ action_type = action->type;
+ switch (action_type) {
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ act_qgrop = action->conf;
+ rule_info->sw_act.fltr_act = ICE_FWD_TO_QGRP;
+ rule_info->sw_act.fwd_id.q_id = base_queue + act_qgrop->queue[0];
+ rule_info->sw_act.qgrp_size = act_qgrop->queue_num;
+ break;
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ act_q = action->conf;
+ rule_info->sw_act.fltr_act = ICE_FWD_TO_Q;
+ rule_info->sw_act.fwd_id.q_id = base_queue + act_q->index;
+ break;
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ rule_info->sw_act.fltr_act = ICE_DROP_PACKET;
+ break;
+ default:
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Invalid action type or queue number");
+ return -rte_errno;
}
rule_info->sw_act.vsi_handle = vsi->idx;
@@ -1699,65 +1619,120 @@ ice_switch_parse_action(struct ice_pf *pf,
rule_info->priority = ICE_SW_PRI_BASE - priority;
return 0;
-
-error:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid action type or queue number");
- return -rte_errno;
-
-error1:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid queue region indexes");
- return -rte_errno;
-
-error2:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Discontinuous queue region");
- return -rte_errno;
}
static int
-ice_switch_check_action(const struct rte_flow_action *actions,
- struct rte_flow_error *error)
+ice_switch_dcf_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
{
+ struct ice_dcf_adapter *ad = param->driver_ctx;
const struct rte_flow_action *action;
enum rte_flow_action_type action_type;
- uint16_t actions_num = 0;
-
- for (action = actions; action->type !=
- RTE_FLOW_ACTION_TYPE_END; action++) {
- action_type = action->type;
- switch (action_type) {
- case RTE_FLOW_ACTION_TYPE_RSS:
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- case RTE_FLOW_ACTION_TYPE_DROP:
- case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
- case RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR:
- actions_num++;
- break;
- case RTE_FLOW_ACTION_TYPE_VOID:
- continue;
- default:
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid action type");
- return -rte_errno;
+ const struct rte_flow_action_ethdev *act_ethdev;
+ const struct rte_eth_dev *repr_dev;
+
+ action = actions->actions[0];
+ action_type = action->type;
+
+ switch (action_type) {
+ case RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR:
+ case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
+ {
+ uint16_t expected_port_id, backer_port_id;
+ act_ethdev = action->conf;
+
+ if (!rte_eth_dev_is_valid_port(act_ethdev->port_id))
+ goto invalid_port_id;
+
+ expected_port_id = ad->parent.pf.dev_data->port_id;
+
+ if (action_type == RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR) {
+ if (expected_port_id != act_ethdev->port_id)
+ goto invalid_port_id;
+ } else {
+ repr_dev = &rte_eth_devices[act_ethdev->port_id];
+
+ if (!repr_dev->data)
+ goto invalid_port_id;
+
+ backer_port_id = repr_dev->data->backer_port_id;
+
+ if (backer_port_id != expected_port_id)
+ goto invalid_port_id;
}
+
+ break;
+invalid_port_id:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid port ID");
+ }
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ break;
+ default:
+ /* Should never reach */
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid action type");
}
- if (actions_num != 1) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid action number");
- return -rte_errno;
+ return 0;
+}
+
+static int
+ice_switch_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ struct ice_adapter *ad = param->driver_ctx;
+ struct ice_pf *pf = &ad->pf;
+ struct rte_eth_dev_data *dev_data = pf->dev_data;
+ const struct rte_flow_action *action = actions->actions[0];
+
+ switch (action->type) {
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ {
+ const struct rte_flow_action_rss *act_qgrop;
+ act_qgrop = action->conf;
+
+ /* Check bounds on number of queues */
+ if (act_qgrop->queue_num < 2 || act_qgrop->queue_num > ICE_SW_MAX_QUEUES)
+ goto err_rss;
+
+ /* must be power of 2 */
+ if (!rte_is_power_of_2(act_qgrop->queue_num))
+ goto err_rss;
+
+ /* queues are monotonous and contiguous so check last queue */
+ if ((act_qgrop->queue[0] + act_qgrop->queue_num) > dev_data->nb_rx_queues)
+ goto err_rss;
+
+ break;
+err_rss:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid queue region");
+ }
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q;
+ act_q = action->conf;
+ if (act_q->index >= dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid queue");
+ }
+
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ break;
+ default:
+ /* Should never reach */
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid action type");
}
return 0;
@@ -1788,11 +1763,39 @@ ice_switch_parse_pattern_action(struct ice_adapter *ad,
struct ci_flow_attr_check_param attr_param = {
.allow_priority = true,
};
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param dcf_param = {
+ .allowed_types = (enum rte_flow_action_type[]){
+ RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+ RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .check = ice_switch_dcf_action_check,
+ };
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]){
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .check = ice_switch_action_check,
+ .driver_ctx = ad,
+ .rss_queues_contig = true,
+ };
ret = ci_flow_check_attr(attr, &attr_param, error);
if (ret)
return ret;
+ ret = ci_flow_check_actions(actions, (ad->hw.dcf_enabled) ? &dcf_param : ¶m,
+ &parsed_actions, error);
+ if (ret)
+ goto error;
+
/* Allow only two priority values - 0 or 1 */
if (attr->priority > 1) {
rte_flow_error_set(error, EINVAL,
@@ -1870,16 +1873,12 @@ ice_switch_parse_pattern_action(struct ice_adapter *ad,
memset(&rule_info, 0, sizeof(rule_info));
rule_info.tun_type = tun_type;
- ret = ice_switch_check_action(actions, error);
- if (ret)
- goto error;
-
if (ad->hw.dcf_enabled)
- ret = ice_switch_parse_dcf_action((void *)ad, actions, attr->priority,
- error, &rule_info);
+ ret = ice_switch_parse_dcf_action(parsed_actions.actions[0],
+ attr->priority, error, &rule_info);
else
- ret = ice_switch_parse_action(pf, actions, attr->priority, error,
- &rule_info);
+ ret = ice_switch_parse_action(pf, parsed_actions.actions[0],
+ attr->priority, error, &rule_info);
if (ret)
goto error;
--
2.47.3
^ permalink raw reply related
* [PATCH v5 25/27] net/ice: use common action checks for FDIR
From: Anatoly Burakov @ 2026-05-25 14:06 UTC (permalink / raw)
To: dev, Bruce Richardson
In-Reply-To: <cover.1779717943.git.anatoly.burakov@intel.com>
From: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Use the common flow action checking parsing infrastructure for checking
flow actions for FDIR filter.
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
---
drivers/net/intel/ice/ice_fdir_filter.c | 376 +++++++++++++-----------
1 file changed, 204 insertions(+), 172 deletions(-)
diff --git a/drivers/net/intel/ice/ice_fdir_filter.c b/drivers/net/intel/ice/ice_fdir_filter.c
index 4efa078aca..2b4c978dd6 100644
--- a/drivers/net/intel/ice/ice_fdir_filter.c
+++ b/drivers/net/intel/ice/ice_fdir_filter.c
@@ -1695,177 +1695,6 @@ static struct ice_flow_engine ice_fdir_engine = {
.rule_size = sizeof(struct ice_fdir_filter_conf),
};
-static int
-ice_fdir_parse_action_qregion(struct ice_pf *pf,
- struct rte_flow_error *error,
- const struct rte_flow_action *act,
- struct ice_fdir_filter_conf *filter)
-{
- const struct rte_flow_action_rss *rss = act->conf;
- uint32_t i;
-
- if (act->type != RTE_FLOW_ACTION_TYPE_RSS) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, act,
- "Invalid action.");
- return -rte_errno;
- }
-
- if (rss->queue_num <= 1) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, act,
- "Queue region size can't be 0 or 1.");
- return -rte_errno;
- }
-
- /* check if queue index for queue region is continuous */
- for (i = 0; i < rss->queue_num - 1; i++) {
- if (rss->queue[i + 1] != rss->queue[i] + 1) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, act,
- "Discontinuous queue region");
- return -rte_errno;
- }
- }
-
- if (rss->queue[rss->queue_num - 1] >= pf->dev_data->nb_rx_queues) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, act,
- "Invalid queue region indexes.");
- return -rte_errno;
- }
-
- if (!(rte_is_power_of_2(rss->queue_num) &&
- (rss->queue_num <= ICE_FDIR_MAX_QREGION_SIZE))) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, act,
- "The region size should be any of the following values:"
- "1, 2, 4, 8, 16, 32, 64, 128 as long as the total number "
- "of queues do not exceed the VSI allocation.");
- return -rte_errno;
- }
-
- filter->input.q_index = rss->queue[0];
- filter->input.q_region = rte_fls_u32(rss->queue_num) - 1;
- filter->input.dest_ctl = ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QGROUP;
-
- return 0;
-}
-
-static int
-ice_fdir_parse_action(struct ice_adapter *ad,
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct ice_fdir_filter_conf *filter)
-{
- struct ice_pf *pf = &ad->pf;
- const struct rte_flow_action_queue *act_q;
- const struct rte_flow_action_mark *mark_spec = NULL;
- const struct rte_flow_action_count *act_count;
- uint32_t dest_num = 0;
- uint32_t mark_num = 0;
- uint32_t counter_num = 0;
- int ret;
-
- for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) {
- switch (actions->type) {
- case RTE_FLOW_ACTION_TYPE_VOID:
- break;
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- dest_num++;
-
- act_q = actions->conf;
- filter->input.q_index = act_q->index;
- if (filter->input.q_index >=
- pf->dev_data->nb_rx_queues) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "Invalid queue for FDIR.");
- return -rte_errno;
- }
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QINDEX;
- break;
- case RTE_FLOW_ACTION_TYPE_DROP:
- dest_num++;
-
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DROP_PKT;
- break;
- case RTE_FLOW_ACTION_TYPE_PASSTHRU:
- dest_num++;
-
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_OTHER;
- break;
- case RTE_FLOW_ACTION_TYPE_RSS:
- dest_num++;
-
- ret = ice_fdir_parse_action_qregion(pf,
- error, actions, filter);
- if (ret)
- return ret;
- break;
- case RTE_FLOW_ACTION_TYPE_MARK:
- mark_num++;
- filter->mark_flag = 1;
- mark_spec = actions->conf;
- filter->input.fltr_id = mark_spec->id;
- filter->input.fdid_prio = ICE_FXD_FLTR_QW1_FDID_PRI_ONE;
- break;
- case RTE_FLOW_ACTION_TYPE_COUNT:
- counter_num++;
-
- act_count = actions->conf;
- filter->input.cnt_ena = ICE_FXD_FLTR_QW0_STAT_ENA_PKTS;
- memcpy(&filter->act_count, act_count,
- sizeof(filter->act_count));
-
- break;
- default:
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Invalid action.");
- return -rte_errno;
- }
- }
-
- if (dest_num >= 2) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Unsupported action combination");
- return -rte_errno;
- }
-
- if (mark_num >= 2) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Too many mark actions");
- return -rte_errno;
- }
-
- if (counter_num >= 2) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Too many count actions");
- return -rte_errno;
- }
-
- if (dest_num + mark_num + counter_num == 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "Empty action");
- return -rte_errno;
- }
-
- /* set default action to PASSTHRU mode, in "mark/count only" case. */
- if (dest_num == 0)
- filter->input.dest_ctl =
- ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_OTHER;
-
- return 0;
-}
static int
ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
@@ -2804,6 +2633,188 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
return 0;
}
+static int
+ice_fdir_parse_action(struct ice_adapter *ad,
+ const struct ci_flow_actions *actions,
+ struct rte_flow_error *error)
+{
+ struct ice_pf *pf = &ad->pf;
+ struct ice_fdir_filter_conf *filter = &pf->fdir.conf;
+ bool dest_set = false;
+ size_t i;
+
+ for (i = 0; i < actions->count; i++) {
+ const struct rte_flow_action *act = actions->actions[i];
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q = act->conf;
+ dest_set = true;
+
+ filter->input.q_index = act_q->index;
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QINDEX;
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ dest_set = true;
+
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DROP_PKT;
+ break;
+ case RTE_FLOW_ACTION_TYPE_PASSTHRU:
+ dest_set = true;
+
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_OTHER;
+ break;
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ {
+ const struct rte_flow_action_rss *rss = act->conf;
+ dest_set = true;
+
+ filter->input.q_index = rss->queue[0];
+ filter->input.q_region = rte_fls_u32(rss->queue_num) - 1;
+ filter->input.dest_ctl = ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QGROUP;
+
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_MARK:
+ {
+ const struct rte_flow_action_mark *mark_spec = act->conf;
+ filter->mark_flag = 1;
+ filter->input.fltr_id = mark_spec->id;
+ filter->input.fdid_prio = ICE_FXD_FLTR_QW1_FDID_PRI_ONE;
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_COUNT:
+ {
+ const struct rte_flow_action_count *act_count = act->conf;
+
+ filter->input.cnt_ena = ICE_FXD_FLTR_QW0_STAT_ENA_PKTS;
+ rte_memcpy(&filter->act_count, act_count,
+ sizeof(filter->act_count));
+ break;
+ }
+ default:
+ /* Should not happen */
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action.");
+ }
+ }
+
+ /* set default action to PASSTHRU mode, in "mark/count only" case. */
+ if (!dest_set) {
+ filter->input.dest_ctl =
+ ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_OTHER;
+ }
+
+ return 0;
+}
+
+static int
+ice_fdir_check_action_qregion(struct ice_pf *pf,
+ struct rte_flow_error *error,
+ const struct rte_flow_action_rss *rss)
+{
+ if (rss->queue_num <= 1) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "Queue region size can't be 0 or 1.");
+ }
+
+ if (rss->queue[rss->queue_num - 1] >= pf->dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "Invalid queue region indexes.");
+ }
+
+ if (!(rte_is_power_of_2(rss->queue_num) &&
+ (rss->queue_num <= ICE_FDIR_MAX_QREGION_SIZE)))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "The region size should be any of the following values:"
+ "2, 4, 8, 16, 32, 64, 128 as long as the total number "
+ "of queues do not exceed the VSI allocation.");
+
+ return 0;
+}
+
+static int
+ice_fdir_check_action(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ struct ice_adapter *ad = param->driver_ctx;
+ struct ice_pf *pf = &ad->pf;
+ uint32_t dest_num = 0;
+ uint32_t mark_num = 0;
+ uint32_t counter_num = 0;
+ size_t i;
+ int ret;
+
+ for (i = 0; i < actions->count; i++) {
+ const struct rte_flow_action *act = actions->actions[i];
+
+ switch (act->type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q = act->conf;
+ dest_num++;
+
+ if (act_q->index >= pf->dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ act,
+ "Invalid queue for FDIR.");
+ }
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ dest_num++;
+ break;
+ case RTE_FLOW_ACTION_TYPE_PASSTHRU:
+ dest_num++;
+ break;
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ {
+ const struct rte_flow_action_rss *rss = act->conf;
+
+ dest_num++;
+ ret = ice_fdir_check_action_qregion(pf, error, rss);
+ if (ret)
+ return ret;
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_MARK:
+ {
+ mark_num++;
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_COUNT:
+ {
+ counter_num++;
+ break;
+ }
+ default:
+ /* Should not happen */
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, act,
+ "Invalid action.");
+ }
+ }
+
+ if (dest_num > 1 || mark_num > 1 || counter_num > 1) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "Unsupported action combination");
+ }
+
+ return 0;
+}
+
static int
ice_fdir_parse(struct ice_adapter *ad,
struct ice_pattern_match_item *array,
@@ -2814,6 +2825,22 @@ ice_fdir_parse(struct ice_adapter *ad,
void **meta,
struct rte_flow_error *error)
{
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]){
+ RTE_FLOW_ACTION_TYPE_PASSTHRU,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_MARK,
+ RTE_FLOW_ACTION_TYPE_COUNT,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 3,
+ .check = ice_fdir_check_action,
+ .driver_ctx = ad,
+ .rss_queues_contig = true,
+ };
struct ice_pf *pf = &ad->pf;
struct ice_fdir_filter_conf *filter = &pf->fdir.conf;
struct ice_pattern_match_item *item = NULL;
@@ -2826,6 +2853,11 @@ ice_fdir_parse(struct ice_adapter *ad,
return ret;
memset(filter, 0, sizeof(*filter));
+
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret)
+ return ret;
+
item = ice_search_pattern_match_item(ad, pattern, array, array_len,
error);
@@ -2854,7 +2886,7 @@ ice_fdir_parse(struct ice_adapter *ad,
goto error;
}
- ret = ice_fdir_parse_action(ad, actions, error, filter);
+ ret = ice_fdir_parse_action(ad, &parsed_actions, error);
if (ret)
goto error;
--
2.47.3
^ permalink raw reply related
* [PATCH v5 24/27] net/ice: use common action checks for hash
From: Anatoly Burakov @ 2026-05-25 14:06 UTC (permalink / raw)
To: dev, Bruce Richardson
In-Reply-To: <cover.1779717943.git.anatoly.burakov@intel.com>
From: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Use the common flow action checking parsing infrastructure for checking
flow actions for hash filter.
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
---
drivers/net/intel/ice/ice_hash.c | 178 +++++++++++++++++--------------
1 file changed, 95 insertions(+), 83 deletions(-)
diff --git a/drivers/net/intel/ice/ice_hash.c b/drivers/net/intel/ice/ice_hash.c
index 9a56d7000c..7a9a4f1a04 100644
--- a/drivers/net/intel/ice/ice_hash.c
+++ b/drivers/net/intel/ice/ice_hash.c
@@ -1092,94 +1092,92 @@ ice_any_invalid_rss_type(enum rte_eth_hash_function rss_func,
}
static int
-ice_hash_parse_action(struct ice_pattern_match_item *pattern_match_item,
- const struct rte_flow_action actions[],
+ice_hash_parse_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param __rte_unused,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_rss *rss;
+
+ rss = actions->actions[0]->conf;
+
+ switch (rss->func) {
+ case RTE_ETH_HASH_FUNCTION_DEFAULT:
+ case RTE_ETH_HASH_FUNCTION_TOEPLITZ:
+ case RTE_ETH_HASH_FUNCTION_SIMPLE_XOR:
+ case RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ:
+ break;
+ default:
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "Selected RSS hash function not supported");
+ }
+
+ if (rss->level)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "a nonzero RSS encapsulation level is not supported");
+
+ if (rss->key_len)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "a nonzero RSS key_len is not supported");
+
+ if (rss->queue)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "a non-NULL RSS queue is not supported");
+
+ return 0;
+}
+
+static int
+ice_hash_parse_rss_action(struct ice_pattern_match_item *pattern_match_item,
+ const struct rte_flow_action_rss *rss,
uint64_t pattern_hint, struct ice_rss_meta *rss_meta,
struct rte_flow_error *error)
{
struct ice_rss_hash_cfg *cfg = pattern_match_item->meta;
- enum rte_flow_action_type action_type;
- const struct rte_flow_action_rss *rss;
- const struct rte_flow_action *action;
uint64_t rss_type;
+ bool symm = false;
- /* Supported action is RSS. */
- for (action = actions; action->type !=
- RTE_FLOW_ACTION_TYPE_END; action++) {
- action_type = action->type;
- switch (action_type) {
- case RTE_FLOW_ACTION_TYPE_RSS:
- rss = action->conf;
- rss_type = rss->types;
-
- /* Check hash function and save it to rss_meta. */
- if (pattern_match_item->pattern_list !=
- pattern_empty && rss->func ==
- RTE_ETH_HASH_FUNCTION_SIMPLE_XOR) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "Not supported flow");
- } else if (rss->func ==
- RTE_ETH_HASH_FUNCTION_SIMPLE_XOR){
- rss_meta->hash_function =
- RTE_ETH_HASH_FUNCTION_SIMPLE_XOR;
- return 0;
- } else if (rss->func ==
- RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ) {
- rss_meta->hash_function =
- RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ;
- if (pattern_hint == ICE_PHINT_RAW)
- rss_meta->raw.symm = true;
- else
- cfg->symm = true;
- }
-
- if (rss->level)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "a nonzero RSS encapsulation level is not supported");
-
- if (rss->key_len)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "a nonzero RSS key_len is not supported");
-
- if (rss->queue)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "a non-NULL RSS queue is not supported");
-
- /* If pattern type is raw, no need to refine rss type */
- if (pattern_hint == ICE_PHINT_RAW)
- break;
-
- /**
- * Check simultaneous use of SRC_ONLY and DST_ONLY
- * of the same level.
- */
- rss_type = rte_eth_rss_hf_refine(rss_type);
-
- if (ice_any_invalid_rss_type(rss->func, rss_type,
- pattern_match_item->input_set_mask_o))
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION,
- action, "RSS type not supported");
-
- rss_meta->cfg = *cfg;
- ice_refine_hash_cfg(&rss_meta->cfg,
- rss_type, pattern_hint);
- break;
- case RTE_FLOW_ACTION_TYPE_END:
- break;
-
- default:
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "Invalid action.");
- return -rte_errno;
+ if (rss->func == RTE_ETH_HASH_FUNCTION_SIMPLE_XOR) {
+ if (pattern_match_item->pattern_list != pattern_empty) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss,
+ "XOR hash function is only supported for empty pattern");
}
+ rss_meta->hash_function = RTE_ETH_HASH_FUNCTION_SIMPLE_XOR;
+ return 0;
}
+ if (rss->func == RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ) {
+ rss_meta->hash_function = RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ;
+ symm = true;
+ }
+
+ /* If pattern type is raw, no need to refine rss type */
+ if (pattern_hint == ICE_PHINT_RAW) {
+ rss_meta->raw.symm = symm;
+ return 0;
+ }
+ cfg->symm = symm;
+
+ /**
+ * Check simultaneous use of SRC_ONLY and DST_ONLY
+ * of the same level.
+ */
+ rss_type = rte_eth_rss_hf_refine(rss->types);
+
+ if (ice_any_invalid_rss_type(rss->func, rss_type,
+ pattern_match_item->input_set_mask_o))
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ rss, "RSS type not supported");
+
+ rss_meta->cfg = *cfg;
+ ice_refine_hash_cfg(&rss_meta->cfg,
+ rss_type, pattern_hint);
+
return 0;
}
@@ -1193,15 +1191,29 @@ ice_hash_parse_pattern_action(__rte_unused struct ice_adapter *ad,
void **meta,
struct rte_flow_error *error)
{
- int ret = 0;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]){
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .check = ice_hash_parse_action_check,
+ };
+ const struct rte_flow_action_rss *rss;
struct ice_pattern_match_item *pattern_match_item;
struct ice_rss_meta *rss_meta_ptr;
uint64_t phint = ICE_PHINT_NONE;
+ int ret = 0;
ret = ci_flow_check_attr(attr, NULL, error);
if (ret)
return ret;
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret)
+ return ret;
+
rss_meta_ptr = rte_zmalloc(NULL, sizeof(*rss_meta_ptr), 0);
if (!rss_meta_ptr) {
rte_flow_error_set(error, EINVAL,
@@ -1233,9 +1245,9 @@ ice_hash_parse_pattern_action(__rte_unused struct ice_adapter *ad,
}
}
- /* Check rss action. */
- ret = ice_hash_parse_action(pattern_match_item, actions, phint,
- rss_meta_ptr, error);
+ rss = parsed_actions.actions[0]->conf;
+ ret = ice_hash_parse_rss_action(pattern_match_item, rss, phint,
+ rss_meta_ptr, error);
error:
if (!ret && meta)
--
2.47.3
^ permalink raw reply related
* [PATCH v5 23/27] net/ice: use common flow attribute checks
From: Anatoly Burakov @ 2026-05-25 14:06 UTC (permalink / raw)
To: dev, Bruce Richardson
In-Reply-To: <cover.1779717943.git.anatoly.burakov@intel.com>
From: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Replace custom attr checks with a call to common checks. Switch engine
supports priority (0 or 1) but other engines don't, so we move the
attribute checks into the engines.
Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
---
drivers/net/intel/ice/ice_acl_filter.c | 9 ++--
drivers/net/intel/ice/ice_fdir_filter.c | 8 ++-
drivers/net/intel/ice/ice_generic_flow.c | 59 +++--------------------
drivers/net/intel/ice/ice_generic_flow.h | 2 +-
drivers/net/intel/ice/ice_hash.c | 11 +++--
drivers/net/intel/ice/ice_switch_filter.c | 22 +++++++--
6 files changed, 48 insertions(+), 63 deletions(-)
diff --git a/drivers/net/intel/ice/ice_acl_filter.c b/drivers/net/intel/ice/ice_acl_filter.c
index 3939579598..0b50f06c46 100644
--- a/drivers/net/intel/ice/ice_acl_filter.c
+++ b/drivers/net/intel/ice/ice_acl_filter.c
@@ -27,6 +27,8 @@
#include "ice_generic_flow.h"
#include "base/ice_flow.h"
+#include "../common/flow_check.h"
+
#define MAX_ACL_SLOTS_ID 2048
#define ICE_ACL_INSET_ETH_IPV4 ( \
@@ -970,7 +972,7 @@ ice_acl_parse(struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error)
{
@@ -980,8 +982,9 @@ ice_acl_parse(struct ice_adapter *ad,
uint64_t input_set;
int ret;
- if (priority >= 1)
- return -rte_errno;
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
memset(filter, 0, sizeof(*filter));
item = ice_search_pattern_match_item(ad, pattern, array, array_len,
diff --git a/drivers/net/intel/ice/ice_fdir_filter.c b/drivers/net/intel/ice/ice_fdir_filter.c
index 2511aecc7e..4efa078aca 100644
--- a/drivers/net/intel/ice/ice_fdir_filter.c
+++ b/drivers/net/intel/ice/ice_fdir_filter.c
@@ -15,6 +15,8 @@
#include "ice_rxtx.h"
#include "ice_generic_flow.h"
+#include "../common/flow_check.h"
+
#define ICE_FDIR_IPV6_TC_OFFSET 20
#define ICE_IPV6_TC_MASK (0xFF << ICE_FDIR_IPV6_TC_OFFSET)
@@ -2808,7 +2810,7 @@ ice_fdir_parse(struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority __rte_unused,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error)
{
@@ -2819,6 +2821,10 @@ ice_fdir_parse(struct ice_adapter *ad,
bool raw = false;
int ret;
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
memset(filter, 0, sizeof(*filter));
item = ice_search_pattern_match_item(ad, pattern, array, array_len,
error);
diff --git a/drivers/net/intel/ice/ice_generic_flow.c b/drivers/net/intel/ice/ice_generic_flow.c
index 0880542851..cbc3d78079 100644
--- a/drivers/net/intel/ice/ice_generic_flow.c
+++ b/drivers/net/intel/ice/ice_generic_flow.c
@@ -17,6 +17,7 @@
#include <rte_malloc.h>
#include <rte_tailq.h>
+#include "../common/flow_check.h"
#include "ice_ethdev.h"
#include "ice_generic_flow.h"
@@ -1965,7 +1966,7 @@ enum rte_flow_item_type pattern_eth_ipv6_udp_l2tpv2_ppp_ipv6_tcp[] = {
typedef bool (*parse_engine_t)(struct ice_adapter *ad,
struct rte_flow *flow,
struct ice_flow_parser *parser,
- uint32_t priority,
+ const struct rte_flow_attr *attr,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
struct rte_flow_error *error);
@@ -2051,44 +2052,6 @@ ice_flow_uninit(struct ice_adapter *ad)
}
}
-static int
-ice_flow_valid_attr(const struct rte_flow_attr *attr,
- struct rte_flow_error *error)
-{
- /* Must be input direction */
- if (!attr->ingress) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
- attr, "Only support ingress.");
- return -rte_errno;
- }
-
- /* Not supported */
- if (attr->egress) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_EGRESS,
- attr, "Not support egress.");
- return -rte_errno;
- }
-
- /* Not supported */
- if (attr->transfer) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
- attr, "Not support transfer.");
- return -rte_errno;
- }
-
- if (attr->priority > 1) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
- attr, "Only support priority 0 and 1.");
- return -rte_errno;
- }
-
- return 0;
-}
-
/* Find the first VOID or non-VOID item pointer */
static const struct rte_flow_item *
ice_find_first_item(const struct rte_flow_item *item, bool is_void)
@@ -2366,7 +2329,7 @@ static bool
ice_parse_engine_create(struct ice_adapter *ad,
struct rte_flow *flow,
struct ice_flow_parser *parser,
- uint32_t priority,
+ const struct rte_flow_attr *attr,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
struct rte_flow_error *error)
@@ -2384,7 +2347,7 @@ ice_parse_engine_create(struct ice_adapter *ad,
if (parser->parse_pattern_action(ad,
parser->array,
parser->array_len,
- pattern, actions, priority, &meta, error) < 0)
+ pattern, actions, attr, &meta, error) < 0)
return false;
RTE_ASSERT(parser->engine->create != NULL);
@@ -2396,7 +2359,7 @@ static bool
ice_parse_engine_validate(struct ice_adapter *ad,
struct rte_flow *flow __rte_unused,
struct ice_flow_parser *parser,
- uint32_t priority,
+ const struct rte_flow_attr *attr,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
struct rte_flow_error *error)
@@ -2413,7 +2376,7 @@ ice_parse_engine_validate(struct ice_adapter *ad,
return parser->parse_pattern_action(ad,
parser->array,
parser->array_len,
- pattern, actions, priority,
+ pattern, actions, attr,
NULL, error) >= 0;
}
@@ -2441,7 +2404,6 @@ ice_flow_process_filter(struct rte_eth_dev *dev,
parse_engine_t ice_parse_engine,
struct rte_flow_error *error)
{
- int ret = ICE_ERR_NOT_SUPPORTED;
struct ice_adapter *ad =
ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ice_flow_parser *parser;
@@ -2466,15 +2428,10 @@ ice_flow_process_filter(struct rte_eth_dev *dev,
return -rte_errno;
}
- ret = ice_flow_valid_attr(attr, error);
- if (ret)
- return ret;
-
*engine = NULL;
/* always try hash engine first */
if (ice_parse_engine(ad, flow, &ice_hash_parser,
- attr->priority, pattern,
- actions, error)) {
+ attr, pattern, actions, error)) {
*engine = ice_hash_parser.engine;
return 0;
}
@@ -2495,7 +2452,7 @@ ice_flow_process_filter(struct rte_eth_dev *dev,
return -rte_errno;
}
- if (ice_parse_engine(ad, flow, parser, attr->priority,
+ if (ice_parse_engine(ad, flow, parser, attr,
pattern, actions, error)) {
*engine = parser->engine;
return 0;
diff --git a/drivers/net/intel/ice/ice_generic_flow.h b/drivers/net/intel/ice/ice_generic_flow.h
index dbb21c47ab..650b43633f 100644
--- a/drivers/net/intel/ice/ice_generic_flow.h
+++ b/drivers/net/intel/ice/ice_generic_flow.h
@@ -525,7 +525,7 @@ typedef int (*parse_pattern_action_t)(struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error);
diff --git a/drivers/net/intel/ice/ice_hash.c b/drivers/net/intel/ice/ice_hash.c
index 702f5d8b0c..9a56d7000c 100644
--- a/drivers/net/intel/ice/ice_hash.c
+++ b/drivers/net/intel/ice/ice_hash.c
@@ -26,6 +26,8 @@
#include "ice_ethdev.h"
#include "ice_generic_flow.h"
+#include "../common/flow_check.h"
+
#define ICE_PHINT_NONE 0
#define ICE_PHINT_VLAN BIT_ULL(0)
#define ICE_PHINT_PPPOE BIT_ULL(1)
@@ -107,7 +109,7 @@ ice_hash_parse_pattern_action(struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error);
@@ -1187,7 +1189,7 @@ ice_hash_parse_pattern_action(__rte_unused struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error)
{
@@ -1196,8 +1198,9 @@ ice_hash_parse_pattern_action(__rte_unused struct ice_adapter *ad,
struct ice_rss_meta *rss_meta_ptr;
uint64_t phint = ICE_PHINT_NONE;
- if (priority >= 1)
- return -rte_errno;
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
rss_meta_ptr = rte_zmalloc(NULL, sizeof(*rss_meta_ptr), 0);
if (!rss_meta_ptr) {
diff --git a/drivers/net/intel/ice/ice_switch_filter.c b/drivers/net/intel/ice/ice_switch_filter.c
index 2da8c5c3c8..67943b591c 100644
--- a/drivers/net/intel/ice/ice_switch_filter.c
+++ b/drivers/net/intel/ice/ice_switch_filter.c
@@ -26,6 +26,7 @@
#include "ice_generic_flow.h"
#include "ice_dcf_ethdev.h"
+#include "../common/flow_check.h"
#define MAX_QGRP_NUM_TYPE 7
#define MAX_INPUT_SET_BYTE 32
@@ -1768,7 +1769,7 @@ ice_switch_parse_pattern_action(struct ice_adapter *ad,
uint32_t array_len,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
- uint32_t priority,
+ const struct rte_flow_attr *attr,
void **meta,
struct rte_flow_error *error)
{
@@ -1784,6 +1785,21 @@ ice_switch_parse_pattern_action(struct ice_adapter *ad,
enum ice_sw_tunnel_type tun_type =
ICE_NON_TUN;
struct ice_pattern_match_item *pattern_match_item = NULL;
+ struct ci_flow_attr_check_param attr_param = {
+ .allow_priority = true,
+ };
+
+ ret = ci_flow_check_attr(attr, &attr_param, error);
+ if (ret)
+ return ret;
+
+ /* Allow only two priority values - 0 or 1 */
+ if (attr->priority > 1) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, NULL,
+ "Invalid priority for switch filter");
+ return -rte_errno;
+ }
for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
item_num++;
@@ -1859,10 +1875,10 @@ ice_switch_parse_pattern_action(struct ice_adapter *ad,
goto error;
if (ad->hw.dcf_enabled)
- ret = ice_switch_parse_dcf_action((void *)ad, actions, priority,
+ ret = ice_switch_parse_dcf_action((void *)ad, actions, attr->priority,
error, &rule_info);
else
- ret = ice_switch_parse_action(pf, actions, priority, error,
+ ret = ice_switch_parse_action(pf, actions, attr->priority, error,
&rule_info);
if (ret)
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox