* [PATCH v2 01/19] ethdev: add flow graph API
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 02/19] net/intel/common: add flow engines infrastructure Anatoly Burakov
` (18 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Thomas Monjalon, Andrew Rybchenko
This commit adds a flow graph parsing API. This is a helper API intended to
help ethdev drivers implement rte_flow parsers, as common usages map to
graph traversal problem very well.
Features provided by the API:
- Flow graph, edge, and node definitions
- Graph traversal logic
- Declarative validation against common flow item types
- Per-node validation and state processing callbacks
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
doc/guides/prog_guide/ethdev/flow_graph.rst | 748 ++++++++++++++++++++
doc/guides/prog_guide/ethdev/index.rst | 1 +
doc/guides/rel_notes/release_26_11.rst | 5 +
lib/ethdev/flow_graph.h | 507 +++++++++++++
lib/ethdev/meson.build | 1 +
5 files changed, 1262 insertions(+)
create mode 100644 doc/guides/prog_guide/ethdev/flow_graph.rst
create mode 100644 lib/ethdev/flow_graph.h
diff --git a/doc/guides/prog_guide/ethdev/flow_graph.rst b/doc/guides/prog_guide/ethdev/flow_graph.rst
new file mode 100644
index 0000000000..87b6b80f1d
--- /dev/null
+++ b/doc/guides/prog_guide/ethdev/flow_graph.rst
@@ -0,0 +1,748 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2026 Intel Corporation
+
+Flow Graph Parser
+=================
+
+Introduction
+------------
+
+The flow graph parser is a helper library for PMD drivers that implements ``rte_flow`` pattern matching.
+It lets a driver declare the protocol sequences it supports as a directed graph of nodes and edges.
+It then validates and extracts fields from an ``rte_flow_item`` pattern in a single traversal.
+
+The library is defined in ``flow_graph.h`` and is header-only.
+
+Scope and Limitations
+~~~~~~~~~~~~~~~~~~~~~
+
+Because the parser is graph-based, it is well suited for matching *protocol stacks*.
+These are sequences of protocol headers such as ``ETH / IPv4 / TCP``.
+Any pattern that can be expressed as "from protocol A, transitions to protocol B or C are allowed" fits naturally into the graph model.
+
+The library is **not** designed to cover every ``rte_flow`` item type.
+Items that do not represent a position in a protocol stack do not have a natural place in a protocol graph.
+This includes conntrack state, meter color, and other metadata items.
+Such items are best handled outside the graph, either before or after the graph parse call.
+
+Defining a Graph
+----------------
+
+A graph consists of three parts:
+
+1. An **enum** that assigns a numeric index to every node.
+2. A **node array** (``struct flow_graph_node[]``) indexed by that enum.
+3. An **edge array** (``struct flow_graph_edge[]``) also indexed by that enum, describing allowed transitions.
+
+These parts are bundled together in a ``struct flow_graph``.
+
+The running example used throughout this guide models the following protocol graph::
+
+ START -> ETH -> [VLAN] -> (IPv4 | IPv6) -> [(TCP | UDP | SCTP)] -> END
+
+Brackets ``[...]`` denote optional items.
+Parentheses ``(...)`` denote a required choice between alternatives.
+The key ideas are:
+
+* ``ETH`` is required after ``START``.
+* After ``ETH``, an optional ``VLAN`` may appear, but the pattern must then see an IP layer.
+* After an IP layer, an optional transport layer may appear; it may be TCP, UDP, or SCTP, after which the pattern reaches ``END``.
+
+Node Enum
+~~~~~~~~~
+
+Every node needs a stable index.
+The first node **must** be at index ``FLOW_GRAPH_NODE_FIRST`` (which is 0).
+This is the *start node*.
+It is used only as a traversal anchor and must not carry callbacks.
+
+.. code-block:: c
+
+ enum example_node_id {
+ EXAMPLE_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ EXAMPLE_NODE_ETH,
+ EXAMPLE_NODE_VLAN,
+ EXAMPLE_NODE_IPV4,
+ EXAMPLE_NODE_IPV6,
+ EXAMPLE_NODE_TCP,
+ EXAMPLE_NODE_UDP,
+ EXAMPLE_NODE_SCTP,
+ EXAMPLE_NODE_END,
+ /* keep last */
+ EXAMPLE_NODE_MAX,
+ };
+
+Node Definitions
+~~~~~~~~~~~~~~~~
+
+Each node maps to one ``rte_flow_item_type``.
+It can also carry a *validate* callback, a *process* callback, and a set of *constraints*.
+The the ``END`` node can also have callbacks to perform end-of-match processing.
+
+A minimal skeleton (callbacks and constraints are added in later sections):
+
+.. code-block:: c
+
+ const struct flow_graph example_graph = {
+ .nodes = (struct flow_graph_node[]){
+ [EXAMPLE_NODE_START] = {
+ .name = "START",
+ /* Start node: no type, no callbacks */
+ },
+ [EXAMPLE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ },
+ [EXAMPLE_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ },
+ [EXAMPLE_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ },
+ [EXAMPLE_NODE_IPV6] = {
+ .name = "IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ },
+ [EXAMPLE_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ },
+ [EXAMPLE_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ },
+ [EXAMPLE_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ },
+ [EXAMPLE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ };
+
+Edge Definitions
+~~~~~~~~~~~~~~~~
+
+Edges express which nodes may follow the current one.
+Every edge list is terminated by the ``FLOW_GRAPH_NODE_EDGE_END`` sentinel.
+All non-``END`` nodes **must** have an edge list.
+The ``END`` node itself does not need one.
+
+.. code-block:: c
+
+ const struct flow_graph example_graph = {
+ .nodes = (struct flow_graph_node[]){
+ /* ... same nodes as above ... */
+ },
+ .edges = (struct flow_graph_edge[]){
+ [EXAMPLE_NODE_START] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_ETH] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_VLAN,
+ EXAMPLE_NODE_IPV4,
+ EXAMPLE_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_VLAN] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_IPV4,
+ EXAMPLE_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_IPV4] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_TCP,
+ EXAMPLE_NODE_UDP,
+ EXAMPLE_NODE_SCTP,
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_IPV6] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_TCP,
+ EXAMPLE_NODE_UDP,
+ EXAMPLE_NODE_SCTP,
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_TCP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_UDP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_SCTP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ },
+ };
+
+Reading the edges back:
+
+* From ``START``, the parser can only reach ``ETH``, which makes ``ETH`` required.
+* From ``ETH``, the parser can reach ``VLAN``, ``IPV4``, or ``IPV6``, which makes ``VLAN`` optional.
+* From ``IPV4`` or ``IPV6``, the parser can reach ``TCP``, ``UDP``, ``SCTP``, or ``END``, which makes the transport layer optional.
+
+Assembling the Graph
+~~~~~~~~~~~~~~~~~~~~
+
+With nodes and edges defined inline, assembling the graph is just a matter of
+combining the two arrays into a single compound literal:
+
+.. code-block:: c
+
+ const struct flow_graph example_graph = {
+ .nodes = (struct flow_graph_node[]){
+ [EXAMPLE_NODE_START] = {
+ .name = "START"
+ },
+ [EXAMPLE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH
+ },
+ /* ... remaining nodes ... */
+ [EXAMPLE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END
+ },
+ },
+ .edges = (struct flow_graph_edge[]){
+ [EXAMPLE_NODE_START] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ /* ... remaining edges ... */
+ },
+ };
+
+Callbacks
+---------
+
+The graph calls up to two callbacks on every visited node: *validate* and *process*.
+
+Both callbacks share the same return convention.
+On success they must return ``0``.
+On failure they must call ``rte_flow_error_set`` to record a descriptive error and return its result.
+
+Validate Callback
+~~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ typedef int (*flow_graph_node_validate_fn)(
+ const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error);
+
+This callback receives a **read-only** context pointer.
+The canonical intent is that it should check whether the item's spec, mask, and last values are acceptable for the driver.
+On failure it returns the result of ``rte_flow_error_set`` (see the return convention above).
+
+It is recommended to use this callback for **all checks that can reject a rule**.
+This includes unsupported mask bits, conflicting field combinations, hardware limitations, and other applicable criteria.
+
+Process Callback
+~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ typedef int (*flow_graph_node_process_fn)(
+ void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error);
+
+This callback receives a **mutable** context pointer.
+The canonical expectation is that it should extract the fields needed for hardware programming.
+It should then store extracted data in the driver's context structure.
+On the rare failure path it returns the result of ``rte_flow_error_set`` (see the return convention above).
+
+It is recommended to use this callback for the **happy path**.
+For example, it can copy addresses, ports, and protocol IDs into the driver context so they can be programmed later.
+
+Defining a Context Structure
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The opaque ``ctx`` pointer passed to every callback is driver-defined.
+A typical context accumulates the parsed protocol fields:
+
+.. code-block:: c
+
+ struct example_parsed_flow {
+ /* L2 */
+ struct rte_ether_addr dst_mac;
+ bool has_vlan;
+ uint16_t vlan_tci;
+
+ /* L3 */
+ bool is_ipv6;
+ rte_be32_t ipv4_src;
+ rte_be32_t ipv4_dst;
+ uint8_t ipv6_src[16];
+ uint8_t ipv6_dst[16];
+
+ /* L4 */
+ enum rte_flow_item_type l4_proto;
+ rte_be16_t src_port;
+ rte_be16_t dst_port;
+ };
+
+These fields are meant to reflect the structure used by the driver to programming hardware with.
+
+Callback Example
+~~~~~~~~~~~~~~~~
+
+Below is a validate/process pair for the IPv4 node.
+The validate callback rejects unsupported mask bits.
+The process callback copies addresses into the context:
+
+.. code-block:: c
+
+ static int
+ example_validate_ipv4(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+ {
+ const struct rte_flow_item_ipv4 *mask = item->mask;
+
+ if (mask->hdr.version_ihl ||
+ mask->hdr.type_of_service ||
+ mask->hdr.total_length ||
+ mask->hdr.packet_id ||
+ mask->hdr.fragment_offset ||
+ mask->hdr.time_to_live ||
+ mask->hdr.next_proto_id ||
+ mask->hdr.hdr_checksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst addresses supported");
+ }
+ return 0;
+ }
+
+ static int
+ example_process_ipv4(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+ {
+ struct example_parsed_flow *parsed = ctx;
+ const struct rte_flow_item_ipv4 *spec = item->spec;
+
+ parsed->is_ipv6 = false;
+ if (spec != NULL) {
+ parsed->ipv4_src = spec->hdr.src_addr;
+ parsed->ipv4_dst = spec->hdr.dst_addr;
+ }
+ return 0;
+ }
+
+Add the callbacks to the node definition:
+
+.. code-block:: c
+
+ [EXAMPLE_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .validate = example_validate_ipv4,
+ .process = example_process_ipv4,
+ },
+
+Node Constraints
+----------------
+
+Many nodes share common requirements about which combination of ``spec``, ``mask``, and ``last`` pointers an item must carry.
+Instead of checking these in every validate callback, they can be declared via the ``constraints`` field.
+The field uses ``flow_graph_node_expect`` flags.
+
+Available constraint flags (may be ORed together):
+
+``FLOW_GRAPH_NODE_EXPECT_EMPTY``
+ The item must have ``spec == NULL``, ``mask == NULL``, and
+ ``last == NULL``.
+
+``FLOW_GRAPH_NODE_EXPECT_SPEC``
+ ``spec`` is required; ``mask`` and ``last`` must be NULL.
+
+``FLOW_GRAPH_NODE_EXPECT_MASK``
+ ``mask`` is required; ``spec`` and ``last`` must be NULL.
+
+``FLOW_GRAPH_NODE_EXPECT_SPEC_MASK``
+ Both ``spec`` and ``mask`` are required; ``last`` must be NULL.
+
+``FLOW_GRAPH_NODE_EXPECT_RANGE``
+ All three (``spec``, ``mask``, ``last``) are required.
+
+``FLOW_GRAPH_NODE_EXPECT_NOT_RANGE``
+ ``last`` must be NULL (``spec`` and ``mask`` are unconstrained).
+
+Multiple flags can be ORed together.
+The item is accepted if **any one** of the flagged constraints is satisfied.
+
+For example, an IPv4 node that accepts either a mask-only item or a spec+mask item:
+
+.. code-block:: c
+
+ [EXAMPLE_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .validate = example_validate_ipv4,
+ .process = example_process_ipv4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+
+An Ethernet node that may appear empty (no spec/mask) or with spec+mask:
+
+.. code-block:: c
+
+ [EXAMPLE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .validate = example_validate_eth,
+ .process = example_process_eth,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+
+Constraints are checked **before** the validate callback is invoked.
+
+Ignoring Item Types
+~~~~~~~~~~~~~~~~~~~
+
+The ``ignore_nodes`` field on ``struct flow_graph`` is an optional complement to node constraints.
+When the pattern may contain item types that are irrelevant to the driver, list them in ``ignore_nodes``.
+For example, metadata items like ``RTE_FLOW_ITEM_TYPE_MARK`` do not represent a protocol header.
+The parser skips ignored items silently without advancing the current graph position:
+
+.. code-block:: c
+
+ const struct flow_graph example_graph = {
+ /* ... nodes and edges ... */
+ .ignore_nodes = (const enum rte_flow_item_type[]){
+ RTE_FLOW_ITEM_TYPE_MARK,
+ RTE_FLOW_ITEM_TYPE_END,
+ },
+ };
+
+``RTE_FLOW_ITEM_TYPE_VOID`` is always ignored regardless of this list.
+Omit ``ignore_nodes`` entirely when no additional item types need to be skipped.
+
+Calling the Parser
+------------------
+
+``flow_graph_parse`` walks the pattern against the graph:
+
+.. code-block:: c
+
+ int
+ flow_graph_parse(const struct flow_graph *graph,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ void *ctx);
+
+A typical call site looks like this:
+
+.. code-block:: c
+
+ struct example_parsed_flow parsed;
+ int ret;
+
+ memset(&parsed, 0, sizeof(parsed));
+
+ ret = flow_graph_parse(&example_graph, pattern, error, &parsed);
+ if (ret != 0)
+ return ret;
+
+ /* 'parsed' now contains the extracted protocol fields */
+
+The function returns success or failure, with ``error`` populated.
+
+Error conditions:
+
+* **Graph is NULL** — for example, when the graph pointer itself is not provided.
+* **Pattern is NULL**.
+* **Unsupported transition** — when an item type has no matching edge from the current node.
+ This is the primary way the graph rejects unsupported protocol sequences.
+* **Constraint failure** — when the spec, mask, and last combination does not satisfy the node's declared constraints.
+* **Validate callback failure** — when driver-specific validation rejects the item.
+* **Process callback failure** — when driver-specific extraction path fails.
+
+.. warning::
+
+ Malformed graph tables (for example invalid node indices, missing sentinels,
+ or otherwise inconsistent driver-defined graph structures) are considered to be a driver implementation bug.
+ Graphs are trusted by default: driver-owned graph structures are expected to be valid and are not fully validated.
+
+The traversal processes items in order, skipping ignored types.
+After the last non-``END`` item, the parser looks for an ``END`` node reachable from the current position.
+It then visits that node and runs its callbacks, if any.
+This means drivers can attach a process callback to the ``END`` node for post-traversal finalization.
+
+Putting It All Together
+-----------------------
+
+The complete graph definition with callbacks and constraints:
+
+.. code-block:: c
+
+ const struct flow_graph example_graph = {
+ .nodes = (struct flow_graph_node[]){
+ [EXAMPLE_NODE_START] = {
+ .name = "START",
+ },
+ [EXAMPLE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .validate = example_validate_eth,
+ .process = example_process_eth,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .process = example_process_vlan,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .validate = example_validate_ipv4,
+ .process = example_process_ipv4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_IPV6] = {
+ .name = "IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .validate = example_validate_ipv6,
+ .process = example_process_ipv6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .validate = example_validate_tcp,
+ .process = example_process_tcp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .validate = example_validate_udp,
+ .process = example_process_udp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .process = example_process_sctp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY
+ | FLOW_GRAPH_NODE_EXPECT_MASK
+ | FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [EXAMPLE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]){
+ [EXAMPLE_NODE_START] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_ETH] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_VLAN,
+ EXAMPLE_NODE_IPV4,
+ EXAMPLE_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_VLAN] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_IPV4,
+ EXAMPLE_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_IPV4] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_TCP,
+ EXAMPLE_NODE_UDP,
+ EXAMPLE_NODE_SCTP,
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_IPV6] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_TCP,
+ EXAMPLE_NODE_UDP,
+ EXAMPLE_NODE_SCTP,
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_TCP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_UDP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [EXAMPLE_NODE_SCTP] = {
+ .next = (size_t[]){
+ EXAMPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ },
+ };
+
+
+Tunnel Graphs and Repeated Item Types
+-------------------------------------
+
+Tunneled patterns often repeat the same ``rte_flow_item_type`` in outer and inner headers.
+A simple representative example is a TCP-IPv4 over GTP-U pattern::
+
+ ETH -> IPV4 -> UDP -> GTPU -> IPV4 -> TCP
+
+The graph library supports this naturally.
+Multiple nodes may use the same ``type`` value, as long as they are distinct nodes in the graph and reached through different edges.
+
+For tunnel parsing, the recommended style is to model repeated protocol types as separate inner/outer nodes, for example ``OUTER_IPV4`` and ``INNER_IPV4``.
+This makes the graph intent explicit, keeps callback logic clear, and avoids unexpected graph paths due to loops.
+
+.. code-block:: c
+
+ enum tunnel_node_id {
+ TUNNEL_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ TUNNEL_NODE_ETH,
+ TUNNEL_NODE_OUTER_IPV4,
+ TUNNEL_NODE_TCP,
+ TUNNEL_NODE_UDP,
+ TUNNEL_NODE_GTPU,
+ TUNNEL_NODE_INNER_IPV4,
+ TUNNEL_NODE_END,
+ TUNNEL_NODE_MAX,
+ };
+
+ const struct flow_graph tunnel_graph = {
+ .nodes = (struct flow_graph_node[]){
+ /* Minimal topology example: callbacks and constraints are omitted. */
+ [TUNNEL_NODE_START] = { .name = "START" },
+ [TUNNEL_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ },
+ [TUNNEL_NODE_OUTER_IPV4] = {
+ .name = "OUTER_IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ },
+ [TUNNEL_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ },
+ [TUNNEL_NODE_GTPU] = {
+ .name = "GTPU",
+ .type = RTE_FLOW_ITEM_TYPE_GTPU,
+ },
+ [TUNNEL_NODE_INNER_IPV4] = {
+ .name = "INNER_IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ },
+ [TUNNEL_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ },
+ [TUNNEL_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]){
+ [TUNNEL_NODE_START] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_ETH] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_OUTER_IPV4,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_OUTER_IPV4] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_UDP,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_UDP] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_GTPU,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_GTPU] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_INNER_IPV4,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_INNER_IPV4] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_TCP,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [TUNNEL_NODE_TCP] = {
+ .next = (size_t[]){
+ TUNNEL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ },
+ };
+
+In other words, traversal follows graph edges (node-to-node), while node matching is done against each candidate node's ``rte_flow_item_type``.
+That combination allows repeated protocol layers to be represented cleanly with separate nodes for different parsing contexts.
+
+Although arbitrary loops are possible in the graph, tunnel protocol graphs are usually easier to reason about when repeated item types are split into explicit inner/outer nodes.
+It is not recommended to create loops in the graph, as these loops will be unbounded.
diff --git a/doc/guides/prog_guide/ethdev/index.rst b/doc/guides/prog_guide/ethdev/index.rst
index 392ced0a2e..b41fd045bb 100644
--- a/doc/guides/prog_guide/ethdev/index.rst
+++ b/doc/guides/prog_guide/ethdev/index.rst
@@ -10,6 +10,7 @@ Ethernet Device Library
ethdev
switch_representation
flow_offload
+ flow_graph
traffic_metering_and_policing
traffic_management
qos_framework
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 907f9013ff..663674353d 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -64,6 +64,11 @@ New Features
* Renamed the ``enable_ptype_lldp`` devarg to ``enable_lldp``.
The old name is no longer accepted.
+* **Added internal ethdev flow graph parser helper API.**
+
+ Added ``flow_graph`` helper definitions in ``flow_graph.h``
+ for PMD drivers to build graph-based flow pattern parsers.
+
Removed Items
-------------
diff --git a/lib/ethdev/flow_graph.h b/lib/ethdev/flow_graph.h
new file mode 100644
index 0000000000..0472b3b04f
--- /dev/null
+++ b/lib/ethdev/flow_graph.h
@@ -0,0 +1,507 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Intel Corporation
+ */
+
+#ifndef _FLOW_GRAPH_H_
+#define _FLOW_GRAPH_H_
+
+/**
+ * @file
+ * Flow Graph
+ *
+ * This file provides a graph-based rte_flow pattern parser for drivers.
+ * It defines structures and functions to validate and process rte_flow
+ * patterns using a directed graph representation.
+ *
+ * @warning
+ * This is an internal API for drivers only. Applications must not use it.
+ */
+
+#include <rte_flow.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * Logging for flow graph parse errors. This is an internal driver API;
+ * FLOW_GRAPH_LOG requires RTE_COMPONENT_NAME (set by meson for drivers)
+ * and the corresponding driver logtype variable to be registered.
+ */
+#ifdef RTE_COMPONENT_NAME
+extern int RTE_CONCAT(RTE_COMPONENT_NAME, _logtype_driver);
+#define FLOW_GRAPH_LOG(level, fmt, ...) \
+ rte_log(RTE_LOG_##level, RTE_CONCAT(RTE_COMPONENT_NAME, _logtype_driver), \
+ "ETHDEV FLOW GRAPH: %s(): " fmt "\n", __func__, ##__VA_ARGS__)
+#else
+/* Use ETHDEV log level when included outside driver context */
+#define FLOW_GRAPH_LOG(level, fmt, ...) \
+ rte_log(RTE_LOG_##level, \
+ rte_eth_dev_logtype, \
+ "ETHDEV FLOW GRAPH: %s(): " fmt "\n", __func__, ##__VA_ARGS__)
+#endif
+
+#define FLOW_GRAPH_NODE_FIRST (0)
+/* Edge array termination sentinel (not a valid node index). */
+#define FLOW_GRAPH_NODE_EDGE_END SIZE_MAX
+
+static inline const char *
+flow_graph_item_type_to_str(enum rte_flow_item_type type)
+{
+ const char *name;
+ int ret;
+
+ ret = rte_flow_conv(RTE_FLOW_CONV_OP_ITEM_NAME_PTR,
+ &name, sizeof(name), (const void *)(uintptr_t)type, NULL);
+ if (ret < 0)
+ return "UNKNOWN";
+
+ return name;
+}
+
+/**
+ * For a lot of nodes, there are multiple common patterns of validation behavior.
+ * This enum allows marking nodes as implementing one of these common behaviors
+ * without need for expressing that in validation code. Can be ORed together to
+ * express support for multiple node types. These checks are not combined (any
+ * one of them being satisfied is sufficient).
+ */
+enum flow_graph_node_expect {
+ FLOW_GRAPH_NODE_EXPECT_NONE = 0, /**< No special constraints. */
+ FLOW_GRAPH_NODE_EXPECT_EMPTY = (1 << 0), /**< spec, mask, last must be NULL. */
+ FLOW_GRAPH_NODE_EXPECT_SPEC = (1 << 1), /**< spec is required, mask and last must be NULL. */
+ FLOW_GRAPH_NODE_EXPECT_MASK = (1 << 2), /**< mask is required, spec and last must be NULL. */
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK = (1 << 3), /**< spec and mask required, last must be NULL. */
+ FLOW_GRAPH_NODE_EXPECT_RANGE = (1 << 4), /**< spec, mask, and last are required. */
+ FLOW_GRAPH_NODE_EXPECT_NOT_RANGE = (1 << 5), /**< last must be NULL. */
+};
+
+/**
+ * Node validation callback.
+ *
+ * Called when the graph traversal reaches this node. Validates the
+ * rte_flow_item (spec, mask, last) against driver-specific constraints.
+ *
+ * Drivers are suggested to perform all checks in this callback.
+ *
+ * @param ctx
+ * Opaque driver context for accumulating parsed state.
+ * @param item
+ * Pointer to the rte_flow_item being validated.
+ * @param error
+ * Pointer to rte_flow_error structure for reporting failures.
+ * @return
+ * 0 on success, or the value returned by rte_flow_error_set() on failure.
+ * On failure the callback must report the error with rte_flow_error_set().
+ */
+typedef int (*flow_graph_node_validate_fn)(
+ const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error);
+
+/**
+ * Node processing callback.
+ *
+ * Called after validation succeeds. Extracts fields from the rte_flow_item
+ * and stores them in driver-specific state for later hardware programming.
+ *
+ * Drivers are suggested to implement "happy path" in this callback.
+ *
+ * @param ctx
+ * Opaque driver context for accumulating parsed state.
+ * @param item
+ * Pointer to the rte_flow_item to process.
+ * @param error
+ * Pointer to rte_flow_error structure for reporting failures.
+ * @return
+ * 0 on success, or the value returned by rte_flow_error_set() on failure.
+ * On failure the callback must report the error with rte_flow_error_set().
+ */
+typedef int (*flow_graph_node_process_fn)(
+ void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error);
+
+/**
+ * Graph node definition.
+ *
+ * Node validity rules:
+ * - all nodes must define a name,
+ * - all non-END nodes must define an edge list,
+ * - start node must not define validation/processing callbacks.
+ */
+struct flow_graph_node {
+ const char *name; /**< Node name. */
+ enum rte_flow_item_type type; /**< Flow item type to match. */
+ enum flow_graph_node_expect constraints; /**< Common validation constraints (ORed). */
+ flow_graph_node_validate_fn validate; /**< Validation callback (NULL if unsupported). */
+ flow_graph_node_process_fn process; /**< Processing callback (NULL if no extraction needed). */
+};
+
+/**
+ * Graph edge definition.
+ *
+ * Describes allowed transitions from one node to others. The 'next' array
+ * lists all valid successor node types and is terminated by FLOW_GRAPH_NODE_EDGE_END.
+ * Drivers define edges to express their supported protocol sequences. Edges
+ * must be unique, as split path following is not supported.
+ */
+struct flow_graph_edge {
+ size_t *next; /**< Array of valid successor nodes, terminated by FLOW_GRAPH_NODE_EDGE_END. */
+};
+
+/**
+ * Flow graph to be implemented by drivers.
+ *
+ * Graph contents are expected to be well-formed. This library validates
+ * traversal semantics for pattern items, but does not attempt to harden
+ * against arbitrary malformed node/edge table definitions.
+ */
+struct flow_graph {
+ struct flow_graph_node *nodes;
+ struct flow_graph_edge *edges;
+ enum rte_flow_item_type *ignore_nodes; /**< Additional node types to ignore, terminated by RTE_FLOW_ITEM_TYPE_END. */
+};
+
+static inline bool
+_flow_graph_node_check_constraint(enum flow_graph_node_expect c,
+ bool has_spec, bool has_mask, bool has_last)
+{
+ bool empty = !has_spec && !has_mask && !has_last;
+
+ if ((c & FLOW_GRAPH_NODE_EXPECT_EMPTY) && empty)
+ return true;
+ if ((c & FLOW_GRAPH_NODE_EXPECT_NOT_RANGE) && !has_last)
+ return true;
+ if ((c & FLOW_GRAPH_NODE_EXPECT_SPEC) && has_spec && !has_mask && !has_last)
+ return true;
+ if ((c & FLOW_GRAPH_NODE_EXPECT_MASK) && has_mask && !has_spec && !has_last)
+ return true;
+ if ((c & FLOW_GRAPH_NODE_EXPECT_SPEC_MASK) && has_spec && has_mask && !has_last)
+ return true;
+ if ((c & FLOW_GRAPH_NODE_EXPECT_RANGE) && has_mask && has_spec && has_last)
+ return true;
+
+ return false;
+}
+
+static inline bool
+_flow_graph_node_is_expected(const struct flow_graph_node *node,
+ const struct rte_flow_item *item, struct rte_flow_error *error)
+{
+ enum flow_graph_node_expect c = node->constraints;
+
+ if (c == FLOW_GRAPH_NODE_EXPECT_NONE)
+ return true;
+
+ bool has_spec = (item->spec != NULL);
+ bool has_mask = (item->mask != NULL);
+ bool has_last = (item->last != NULL);
+
+ if (_flow_graph_node_check_constraint(c, has_spec, has_mask, has_last))
+ return true;
+
+ /*
+ * In the interest of everyone debugging flow parsing code, we should
+ * provide the user with meaningful messages about exactly what failed,
+ * as no one likes non-descript "node constraints not met" errors with
+ * no clear indication of where this is even coming from. What follows
+ * is us building said meaningful error messages. It's a bit ugly, but
+ * it is for the greater good.
+ */
+ const char *msg;
+
+ /* for empty items, we know exactly what went wrong */
+ if (c == FLOW_GRAPH_NODE_EXPECT_EMPTY) {
+ if (has_spec)
+ msg = "Unexpected spec in flow item";
+ else if (has_mask)
+ msg = "Unexpected mask in flow item";
+ else /* has_last */
+ msg = "Unexpected last in flow item";
+ } else {
+ /*
+ * for non-empty constraints, we need to figure out the one
+ * thing user is missing (or has extra) that would've satisfied
+ * the constraints. We do that by flipping each presence bit in
+ * turn and seeing whether that single change would have
+ * satisfied the node constraints.
+ */
+
+ /* check spec first */
+ if (!has_spec && _flow_graph_node_check_constraint(c, true, has_mask, has_last)) {
+ msg = "Missing spec in flow item";
+ } else if (has_spec && _flow_graph_node_check_constraint(c, false, has_mask, has_last)) {
+ msg = "Unexpected spec in flow item";
+ }
+ /* check mask next */
+ else if (!has_mask && _flow_graph_node_check_constraint(c, has_spec, true, has_last)) {
+ msg = "Missing mask in flow item";
+ } else if (has_mask && _flow_graph_node_check_constraint(c, has_spec, false, has_last)) {
+ msg = "Unexpected mask in flow item";
+ }
+ /* finally, check range */
+ else if (!has_last && _flow_graph_node_check_constraint(c, has_spec, has_mask, true)) {
+ msg = "Missing last in flow item";
+ } else if (has_last && _flow_graph_node_check_constraint(c, has_spec, has_mask, false)) {
+ msg = "Unexpected last in flow item";
+ /* multiple things are wrong with the constraint, so just output a generic error */
+ } else {
+ msg = "Flow item does not meet node constraints";
+ }
+ }
+
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item, msg);
+
+ return false;
+}
+
+/**
+ * Check if a flow item type should be ignored by the graph.
+ *
+ * Checks if the item type is in the graph's ignore list.
+ */
+static inline bool
+_flow_graph_node_is_ignored(const struct flow_graph *graph,
+ enum rte_flow_item_type fi_type)
+{
+ const enum rte_flow_item_type *ignored;
+
+ /* Always skip VOID items */
+ if (fi_type == RTE_FLOW_ITEM_TYPE_VOID)
+ return true;
+
+ if (graph->ignore_nodes == NULL)
+ return false;
+
+ for (ignored = graph->ignore_nodes; *ignored != RTE_FLOW_ITEM_TYPE_END; ignored++) {
+ if (*ignored == fi_type)
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * Get the index of a node within a graph.
+ */
+static inline size_t
+_flow_graph_get_node_index(const struct flow_graph *graph, const struct flow_graph_node *node)
+{
+ return (size_t)(node - graph->nodes);
+}
+
+/**
+ * Check if a graph node is valid.
+ */
+static inline bool
+_flow_graph_node_is_valid(const struct flow_graph *graph,
+ const struct flow_graph_node *node,
+ struct rte_flow_error *error)
+{
+ size_t node_idx;
+
+ if (node == NULL) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Flow graph node pointer is NULL");
+ return false;
+ }
+
+ if (node->name == NULL) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, node,
+ "Flow graph node name is not defined");
+ return false;
+ }
+
+ node_idx = _flow_graph_get_node_index(graph, node);
+
+ /* first node can't have callbacks because there's no item */
+ if (node_idx == FLOW_GRAPH_NODE_FIRST &&
+ (node->validate != NULL || node->process != NULL)) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, node,
+ "Flow graph start node callbacks are not allowed");
+ return false;
+ }
+
+ /* all non-END nodes must have edges */
+ if (node->type != RTE_FLOW_ITEM_TYPE_END &&
+ graph->edges[node_idx].next == NULL) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, node,
+ "Flow graph edge list is not defined for non-END node");
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * Find the next node in the graph matching the given item type.
+ */
+static inline const struct flow_graph_node *
+_flow_graph_find_next_node(const struct flow_graph *graph,
+ const struct flow_graph_node *cur_node,
+ enum rte_flow_item_type next_type,
+ struct rte_flow_error *error)
+{
+ const size_t *next_nodes;
+ size_t cur_idx, edge_idx;
+
+ if (!_flow_graph_node_is_valid(graph, cur_node, error))
+ return NULL;
+
+ cur_idx = _flow_graph_get_node_index(graph, cur_node);
+ next_nodes = graph->edges[cur_idx].next;
+
+ for (edge_idx = 0; next_nodes[edge_idx] != FLOW_GRAPH_NODE_EDGE_END; edge_idx++) {
+ const struct flow_graph_node *tmp =
+ &graph->nodes[next_nodes[edge_idx]];
+ /* if node is invalid, graph is broken */
+ if (!_flow_graph_node_is_valid(graph, tmp, error))
+ return NULL;
+ if (tmp->type == next_type)
+ return tmp;
+ }
+
+ return NULL;
+}
+
+/**
+ * Visit (validate and extract) a node's item.
+ */
+static inline int
+_flow_graph_visit_node(const struct flow_graph_node *node, void *ctx,
+ const struct rte_flow_item *item, struct rte_flow_error *error)
+{
+ int ret;
+
+ /* if we expect a certain type of node, check for it */
+ if (item != NULL && !_flow_graph_node_is_expected(node, item, error))
+ return -EINVAL;
+
+ /* Does this node fit driver's criteria? */
+ if (node->validate != NULL) {
+ ret = node->validate(ctx, item, error);
+ if (ret != 0)
+ return ret;
+ }
+
+ /* Extract data from this item */
+ if (node->process != NULL) {
+ ret = node->process(ctx, item, error);
+ if (ret != 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * Parse and validate a flow pattern using the flow graph.
+ *
+ * Traverses the pattern items and validates them against the driver's graph
+ * structure. For each item, checks that the transition from the current node
+ * is allowed, then invokes validation and processing callbacks.
+ *
+ * @param graph
+ * Pointer to the driver's flow graph definition with nodes and edges.
+ * @param pattern
+ * Array of rte_flow_item structures to parse, terminated by RTE_FLOW_ITEM_TYPE_END.
+ * @param error
+ * Pointer to rte_flow_error structure for reporting failures.
+ * @param ctx
+ * Opaque driver context for accumulating parsed state.
+ * @return
+ * 0 on success, negative errno on failure (error is set).
+ */
+static inline int
+flow_graph_parse(const struct flow_graph *graph, const struct rte_flow_item *pattern,
+ struct rte_flow_error *error, void *ctx)
+{
+ const struct flow_graph_node *cur_node;
+ const struct rte_flow_item *item;
+ int ret;
+
+ if (graph == NULL || graph->nodes == NULL || graph->edges == NULL) {
+ FLOW_GRAPH_LOG(DEBUG, "flow graph is not defined");
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Flow graph is not defined");
+ }
+ if (pattern == NULL) {
+ FLOW_GRAPH_LOG(DEBUG, "flow pattern is NULL");
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+ "Flow pattern is NULL");
+ }
+
+ /* use start node as traversal anchor */
+ cur_node = &graph->nodes[FLOW_GRAPH_NODE_FIRST];
+
+ /* is the node valid? */
+ if (!_flow_graph_node_is_valid(graph, cur_node, error)) {
+ /* error may be NULL */
+ if (error != NULL)
+ FLOW_GRAPH_LOG(DEBUG, "%s", error->message);
+ return -EINVAL;
+ }
+
+ /* Traverse pattern items */
+ for (item = pattern; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+
+ /* Skip items in the graph's ignore list */
+ if (_flow_graph_node_is_ignored(graph, item->type)) {
+ FLOW_GRAPH_LOG(DEBUG, "ignored item %s",
+ flow_graph_item_type_to_str(item->type));
+ continue;
+ }
+
+ /* Find the next graph node for this item type */
+ cur_node = _flow_graph_find_next_node(graph, cur_node,
+ item->type, error);
+ if (cur_node == NULL) {
+ FLOW_GRAPH_LOG(DEBUG, "cannot traverse to item %s",
+ flow_graph_item_type_to_str(item->type));
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Pattern item not supported");
+ }
+ FLOW_GRAPH_LOG(DEBUG, "processing %s", cur_node->name);
+ /* Validate and process the current item at this node */
+ ret = _flow_graph_visit_node(cur_node, ctx, item, error);
+ if (ret != 0) {
+ /* error may be NULL */
+ if (error != NULL)
+ FLOW_GRAPH_LOG(DEBUG, "%s", error->message);
+ return ret;
+ }
+ }
+
+ /* Pattern items have ended but we still need to process the end */
+ cur_node = _flow_graph_find_next_node(graph, cur_node, item->type, error);
+ if (cur_node == NULL) {
+ FLOW_GRAPH_LOG(DEBUG, "cannot traverse to item %s",
+ flow_graph_item_type_to_str(item->type));
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Pattern item not supported");
+ }
+ ret = _flow_graph_visit_node(cur_node, ctx, item, error);
+ if (ret != 0) {
+ /* error may be NULL */
+ if (error != NULL)
+ FLOW_GRAPH_LOG(DEBUG, "%s", error->message);
+ return ret;
+ }
+
+ return 0;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _FLOW_GRAPH_H_ */
diff --git a/lib/ethdev/meson.build b/lib/ethdev/meson.build
index 8ba6c708a2..99ff3c990c 100644
--- a/lib/ethdev/meson.build
+++ b/lib/ethdev/meson.build
@@ -40,6 +40,7 @@ driver_sdk_headers += files(
'ethdev_pci.h',
'ethdev_vdev.h',
'rte_flow_driver.h',
+ 'flow_graph.h',
'rte_mtr_driver.h',
'rte_tm_driver.h',
)
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 02/19] net/intel/common: add flow engines infrastructure
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 01/19] ethdev: add flow graph API Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 03/19] net/intel/common: add utility functions Anatoly Burakov
` (17 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Current implementation of flow engines in various drivers have a few issues
that need to be corrected.
For one, some of the are fundamentally incompatible with secondary
processes, because the flow engine registration and creation will
allocate structures in shared memory but use process-local pointers to
point to flow engines and pattern tables.
For another, a lot of them are needlessly complicated and rely on a
separation between patterns and parsing that is hard to reason about and
maintain: they do not define memory ownership model, they do not define the
way in which we approach parameter and pattern parsing, and they
occasionally do weird things like passing around pointers-to-void-pointers
or even using pointers as integer values.
Another common problem is extremely convoluted internal tracking, flow
installation, flow replay, and cleanup code. This infrastructure is usually
done in an ad-hoc manner that has a lot of boilerplate.
These issues can be corrected, but because of how much code there is to the
current infrastructure and how tightly coupled it is, it would be easier to
just build new one from scratch, and gradually migrate all engines to use
it. This patch is intended as a first step towards that goal, and defines
both common data types to be used by all rte_flow parsers, as well as the
interaction model that is to be followed by all drivers.
We define a set of structures that will represent:
- Defined rte_flow parsing interaction model and code flow (ops struct)
- Defined memory allocation and ownership model for all engines
- Scratch space format for all engines (variably allocated typed struct)
- Flow rule format for all engines (variably allocated typed struct)
- Engine definitions that are compatible with secondary process model
- Implementations of common rte_flow operations
- Various supporting infrastructure for parser customization, e.g. hooks
- Support for using custom allocation (e.g. for mempool-based alloc)
- Support for replaying all flows to restore HW state
- Support for removing all flows without modifying HW state
The design intent is heavily documented right inside the header and is to
be considered authoritative design document for how to build rte_flow
parsers for Intel Ethernet drivers going forward.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/common/flow_engine.h | 1613 ++++++++++++++++++++++++
1 file changed, 1613 insertions(+)
create mode 100644 drivers/net/intel/common/flow_engine.h
diff --git a/drivers/net/intel/common/flow_engine.h b/drivers/net/intel/common/flow_engine.h
new file mode 100644
index 0000000000..3a1d93f55c
--- /dev/null
+++ b/drivers/net/intel/common/flow_engine.h
@@ -0,0 +1,1613 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#ifndef _COMMON_INTEL_FLOW_ENGINE_H_
+#define _COMMON_INTEL_FLOW_ENGINE_H_
+
+#include <stddef.h>
+#include <stdio.h>
+#include <sys/queue.h>
+
+#include <rte_bitops.h>
+#include <rte_malloc.h>
+
+#include <ethdev_driver.h>
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_tailq.h>
+#include <rte_rwlock.h>
+#include <rte_hexdump.h>
+
+#include "log.h"
+
+/*
+ * This is a common header for Intel Ethernet drivers' flow engine
+ * implementations. It defines the interfaces and data structures required to
+ * implement flow rule engines that can be plugged into the drivers' flow
+ * handling logic.
+ *
+ * Design considerations:
+ *
+ * 1. Ease of implementation
+ *
+ * The flow engine interface is designed to be as simple as possible with
+ * obvious defaults (i.e. not specifying something leads to behavior that
+ * would've been the most expected in context). The point is not to produce a
+ * monstrous driver-within-a-driver framework, but rather to make engine
+ * definitions follow semantic expectations of what the engine actually does.
+ *
+ * All the boilerplate (flow management, engine enablement tracking, etc.) is
+ * handled by the common flow infrastructure, so the engine implementation only
+ * needs to focus on the actual logic of parsing and installing/uninstalling
+ * flow rules, and defining each step of the process as it pertains to each flow
+ * engine.
+ *
+ * It is expected that drivers will use other utility functions from the common
+ * flow-related code where applicable (e.g. flow_util.h, flow_check.h, etc.).
+ *
+ * 2. Full secondary process compatibility
+ *
+ * In order to support rte_flow operations in secondary processes, we need to
+ * store which engines are enabled for particular driver instance, and resolve
+ * them at runtime. The engine index (its position in the engine list) is used as
+ * a bit position in a driver-specific 64-bit field of enabled engines. This
+ * way, the engine definitions can be stored in read-only memory, and referenced
+ * by both primary and secondary processes without issues.
+ *
+ * For this to remain safe, flow engine lists and engine definitions must be
+ * immutable for process lifetime (declare them as const).
+ *
+ * Note that this does not imply that all drivers are therefore able to support
+ * rte_flow-related operations in secondary processes - that is still up to each
+ * driver to implement. This just ensures that the flow engine framework does
+ * not prevent it.
+ *
+ * Engine callbacks must not access or retain an `struct rte_eth_dev *` pointer,
+ * as that object is process-local; use the process-independent
+ * `struct rte_eth_dev_data *` provided by the framework instead.
+ *
+ * The per-instance engine configuration is set up and torn down exclusively by
+ * `ci_flow_engine_conf_init()` and `ci_flow_engine_conf_reset()`. These functions
+ * should only be called at device setup/teardown by primary process.
+ *
+ * 3. Flow object lifecycle is framework-owned
+ *
+ * Engines are expected to treat framework-provided context and flow objects as
+ * storage they fill in, not storage they own. In other words, engine logic
+ * should focus on contents of flow data, while object lifetime is managed by
+ * the framework. Engines may still allocate auxiliary data, but only in places
+ * where the framework guarantees a matching teardown call, which will give the
+ * engine the opportunity to release said auxiliary data.
+ *
+ * 4. Pattern parsing: flow_graph and pattern_parse callback
+ *
+ * The flow engine framework is designed to work hand-in-hand with the
+ * `flow_graph` parsing infrastructure. Each engine may provide a pattern
+ * graph that is used to match the flow pattern, and extract relevant data
+ * into the engine context provided by the framework.
+ *
+ * Engines may also provide a `pattern_parse` callback that is invoked before
+ * the graph parser runs. This allows engines to handle pattern items that
+ * don't fit neatly into the graph model (e.g. FUZZY items that can appear at
+ * any position), as well as ignoring the graph parser entirely and implementing
+ * custom pattern parsing.
+ *
+ * There is no way to completely ignore pattern contents for the engine except
+ * for defining a noop `pattern_parse` callback. This is by design, as such case
+ * is considered rte_flow API misuse. By default, even for empty fallback case,
+ * a meaningful pattern (one that is not empty or ANY) will be treated as error.
+ *
+ * 5. Setup, teardown, and flow list lifecycle ordering
+ *
+ * `ci_flow_engine_conf_init()` and `ci_flow_engine_conf_reset()` are
+ * primary-process-only (see point 2), and the framework does not serialize
+ * them against concurrent flow operations or against each other - the driver
+ * must do so.
+ *
+ * The expected sequence of calls for a driver instance is:
+ *
+ * - `ci_flow_engine_conf_init()` should run from the driver's `dev_init` path.
+ *
+ * - At `dev_close`, `ci_flow_cleanup()` should be run first to drop all flows
+ * and their internal tracking. Then, `ci_flow_engine_conf_reset()` can be run.
+ *
+ * - Devices may or may not advertise `RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP`,
+ * i.e. support for keeping flow rules across a `dev_stop`/`dev_start`
+ * cycle.
+ *
+ * - If the device does not advertise this capability, flows must be
+ * flushed via `ci_flow_flush()` as the first step of `dev_stop()` (doing so
+ * later may interfere with flow uninstall).
+ *
+ * - If rule replay is needed (i.e. flows were kept rather than flushed at
+ * `dev_stop`), the driver should call `ci_flow_replay()` from `dev_start`
+ * to re-install the kept flows to hardware as last step.
+ */
+
+/* forward declarations for flow engine data types */
+struct ci_flow_engine_ops;
+struct ci_flow_engine_ctx;
+struct ci_flow_engine;
+struct ci_flow_engine_ref;
+struct ci_flow_engine_list;
+struct ci_flow_engine_conf;
+struct ci_flow;
+
+/*
+ * Flow engine ops.
+ *
+ * Each flow engine must provide a set of operations to handle common
+ * operations, such as:
+ *
+ * - Initialize and clean up engine resources (engine_init/engine_uninit)
+ * - Allocate memory for flow rules (flow_alloc)
+ * - Process flow attributes and actions into engine-specific context (ctx_init)
+ * - Parse pattern into engine-specific context (pattern_parse)
+ * - Pattern graph to use when parsing flow patterns (see ethdev's `flow_graph`)
+ * - Any final post-parse checks or actions (ctx_finalize)
+ * - Build the actual flow rule structure from the parsed context (ctx_to_flow)
+ * - Track/untrack the flow rule in driver-internal state (flow_register/flow_unregister)
+ * - Install/remove the flow rule to/from hardware (flow_install/flow_uninstall)
+ * - Query data for the flow rule (flow_query)
+ *
+ * What follows is a description of the flow engine pipeline as it relates to
+ * the flow engine ops struct. Each stage has a setup step (run when a flow is
+ * created) and, where applicable, a teardown step (run when a flow is removed).
+ * Each operation is then a composition of stages: setup steps run in forward
+ * order, teardown steps in reverse. Optional callbacks are shown in [brackets].
+ *
+ * - Allocation stage
+ * setup: [flow_alloc] teardown: [flow_free]
+ * Provides engine-specific flow object allocation. Skipped on the validate path.
+ *
+ * - Parse stage
+ * setup: ctx_init -> [pattern_parse] -> [graph parser] ->
+ * [ctx_finalize] -> [ctx_to_flow]
+ * teardown: (none)
+ * Interprets the request into the flow structure. Must NOT allocate.
+ *
+ * - Track stage
+ * setup: [flow_register] teardown: [flow_unregister]
+ * Updates driver-internal state. Permitted to allocate.
+ *
+ * - Apply stage
+ * setup: [flow_install] teardown: [flow_uninstall]
+ * Programs the hardware. Must NOT allocate.
+ *
+ * Typical operation sequences:
+ *
+ * - rte_flow_create: Allocation -> Parse -> Track -> Apply (setup)
+ * - rte_flow_validate: Parse
+ * - rte_flow_destroy: Apply -> Track -> Allocation (teardown)
+ * - rte_flow_flush: Apply -> Track -> Allocation (teardown, per flow)
+ * - ci_flow_replay: Apply (setup, per flow)
+ * - ci_flow_cleanup: Track -> Allocation (teardown, per flow)
+ *
+ * 1) Engine availability and lifecycle
+ *
+ * The engine availability can be checked by the driver at init time. The exact
+ * mechanics of this is left up to each individual driver - it may be hardware
+ * capability bits, PHY type check, devargs, or any other criteria that makes
+ * sense in the context of driver/adapter. If the engine_init callback is not
+ * implemented, the engine is assumed to be always available.
+ *
+ * If `priv_size` is non-zero, the framework allocates a zeroed private block
+ * and passes it to engine_init/engine_uninit. Each allocated flow will also
+ * carry a pointer to this per-device private data in `flow->engine_priv`.
+ *
+ * 2) Input parsing
+ *
+ * `ctx_init` is mandatory and acts as the main gateway for processing actions and
+ * attributes into engine context.
+ *
+ * Patterns are matched either against the pattern graph provided by the
+ * `ci_flow_engine` structure, or by the `pattern_parse` callback, or both.
+ *
+ * Pattern handling mode is selected from graph/callback presence:
+ *
+ * - graph only: graph parser runs
+ * - callback only: pattern_parse runs
+ * - callback + graph: pattern_parse runs, then graph parser runs
+ * - neither graph nor callback: empty fallback matcher is used
+ *
+ * Empty fallback matcher accepts:
+ *
+ * - NULL pattern
+ * - empty pattern (start -> end)
+ * - ANY-only pattern (start -> any -> end)
+ *
+ * Any other patterns will be rejected by the empty path.
+ *
+ * In callback + graph mode, the callback does not remove items from the
+ * original pattern stream. The graph parser still sees the same pattern, so the
+ * graph should define its `ignore_nodes` list appropriately.
+ *
+ * 3) Rule finalization
+ *
+ * `ctx_finalize` is an optional final check for consistency between parsed
+ * action/attribute data and pattern-derived data, as well as a chance to perform
+ * any final post-parse operations.
+ *
+ * 4) Rule materialization
+ *
+ * `ctx_to_flow` translates parsed context into the concrete engine-specific
+ * flow rule representation. As a general guideline, flow rule should not
+ * contain anything that isn't useful for rule installation or querying. All
+ * of the temporary data should be stored in the flow building context, while
+ * the rule should only contain data that is pertinent to flow programming.
+ *
+ * 5) Rule lifecycle hooks:
+ *
+ * `flow_alloc`/`flow_free` are optional custom object lifecycle callbacks
+ * (Allocation stage). If `flow_alloc` is not provided (or returns NULL), the
+ * framework falls back to rte_zmalloc-based allocation. If `flow_alloc` callback
+ * implementation exists, `flow_free` implementation must exist as well.
+ *
+ * `flow_register`/`flow_unregister` are the optional Track stage callbacks.
+ * `flow_register` builds driver-internal state for the flow; `flow_unregister`
+ * tears it down. As with alloc/free, they must both be defined or both be NULL.
+ *
+ * `flow_install`/`flow_uninstall` are the optional Apply stage callbacks that
+ * program and unprogram the hardware. They must not allocate.
+ *
+ * The engines own only engine-specific fields. The common `ci_flow` fields
+ * (engine_idx, engine_priv, fallback_alloc, etc.) are owned and managed by the
+ * framework and are not to be modified.
+ *
+ * IMPORTANT: by the time the Track and Apply stages run, the engine has already
+ * accepted the flow through `ctx_init`/`ctx_finalize`/`ctx_to_flow`. A failure
+ * in `flow_register` or `flow_install` (e.g. hardware/software resource
+ * exhaustion) is treated as a hard failure: no other engines will be tried. It
+ * is therefore implied that if the flow parser has accepted the flow, it should
+ * in principle be trackable and installable - any checks intrinsic to the pattern
+ * itself that would have prevented that should have been done at the parse stage.
+ *
+ * 6) Memory ownership and allocation policy:
+ *
+ * There are exactly three places where an engine is permitted to do allocations:
+ *
+ * - engine init
+ * - flow alloc (Allocation stage)
+ * - flow register (Track stage)
+ *
+ * In engine initialization, the engine may allocate any engine-wide resources
+ * that will later be utilized by other callbacks (such as creating a memory
+ * pool from which `flow_alloc` will allocate flow objects).
+ *
+ * The flow allocation callback is an allocation-policy callback only. That is,
+ * a well-behaved implementation for this callback may allocate memory *for the
+ * flow itself* (such as getting an object from a mempool), and it may update
+ * allocator-associated state (such as tracking allocated objects), but it
+ * otherwise must not produce any artifacts to be used by the parsing stage.
+ * This is because for `rte_flow_validate()`, the `flow_alloc` callbacks are
+ * bypassed, so parse stage cannot rely on anything `flow_alloc` may have done.
+ *
+ * The flow register callback is where the transition from "parsing the flow" to
+ * "updating driver-internal state" happens, and is the only per-flow callback
+ * where a well-behaved implementation may allocate memory and store a pointer
+ * in the flow object or in any internal driver structure.
+ *
+ * The flow install callback must NOT allocate: it only programs the hardware.
+ *
+ * Parsing-related callbacks (`ctx_init`, `pattern_parse`, `ctx_finalize`,
+ * `ctx_to_flow`) must not allocate any memory, as the parsing control flow may
+ * fail at any of these steps without there being any teardown.
+ *
+ * 7) Flow query
+ *
+ * For querying data, the flow_query function is provided to query data for the
+ * flow rule. The engine may not provide this function, in which case any
+ * attempt to query the rule will result in failure.
+ *
+ * 8) Concurrency model
+ *
+ * The framework serializes access to each driver instance's flow state with a
+ * single rwlock. Create, destroy and flush take it exclusively (write);
+ * validate and query take it shared (read). Consequently the parse-phase
+ * callbacks (ctx_init, pattern_parse, ctx_finalize, ctx_to_flow) can run
+ * concurrently with each other on the validate path, and flow_query can run
+ * concurrently with other queries. These callbacks must treat shared engine
+ * state - including engine_priv - as read-only and must not mutate it. State
+ * mutation belongs in the write-locked callbacks (engine_init/engine_uninit,
+ * flow_alloc/flow_free, flow_register/flow_unregister,
+ * flow_install/flow_uninstall).
+ */
+struct ci_flow_engine_ops {
+ /* engine init callback - can be NULL */
+ int (*engine_init)(const struct ci_flow_engine *engine,
+ struct rte_eth_dev_data *dev_data,
+ void *priv);
+ /* engine uninit callback - can be NULL */
+ void (*engine_uninit)(const struct ci_flow_engine *engine,
+ void *priv);
+ /* allocation callback for flow rules - can be NULL */
+ struct ci_flow *(*flow_alloc)(const struct ci_flow_engine *engine,
+ struct rte_eth_dev_data *dev_data,
+ void *priv);
+ /* deallocation callback for flow rules - can be NULL */
+ void (*flow_free)(struct ci_flow *flow,
+ struct rte_eth_dev_data *dev_data,
+ void *priv);
+ /* initialize engine context from flow attr/actions - mandatory */
+ int (*ctx_init)(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error);
+ /* pattern parsing callback - can be NULL */
+ int (*pattern_parse)(struct ci_flow_engine_ctx *ctx,
+ const struct rte_flow_item pattern[],
+ struct rte_flow_error *error);
+ /* final pass before converting context to flow - can be NULL */
+ int (*ctx_finalize)(struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error);
+ /* initialize flow rule from parsed context - can be NULL */
+ int (*ctx_to_flow)(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error);
+ /* track a flow rule in driver-internal state - can be NULL */
+ int (*flow_register)(struct ci_flow *flow,
+ struct rte_flow_error *error);
+ /* untrack a flow rule from driver-internal state - can be NULL */
+ int (*flow_unregister)(struct ci_flow *flow,
+ struct rte_flow_error *error);
+ /* install a flow rule to hardware - can be NULL */
+ int (*flow_install)(struct ci_flow *flow,
+ struct rte_flow_error *error);
+ /* uninstall a flow rule from hardware - can be NULL */
+ int (*flow_uninstall)(struct ci_flow *flow,
+ struct rte_flow_error *error);
+ /* query flow - can be NULL */
+ int (*flow_query)(struct ci_flow *flow,
+ const struct rte_flow_action *action,
+ void *data,
+ struct rte_flow_error *error);
+};
+
+/*
+ * common definition for flow engine context.
+ * each engine will define its own context structure that
+ * *must* start with this base structure.
+ */
+struct ci_flow_engine_ctx {
+ /* ethernet device this context belongs to */
+ struct rte_eth_dev_data *dev_data;
+ /* original flow attributes, as passed by the caller */
+ const struct rte_flow_attr *attr;
+ /* original flow pattern, as passed by the caller */
+ const struct rte_flow_item *pattern;
+ /* original flow actions, as passed by the caller */
+ const struct rte_flow_action *actions;
+};
+
+/*
+ * Common definition for flow rule.
+ *
+ * For flow rules, there are three parts to consider:
+ *
+ * 1) Common data
+ * 2) Driver-specific data
+ * 3) Engine-specific data
+ *
+ * The common data is defined here as the `ci_flow` structure. It contains
+ * fields that are common to all flow rules, regardless of driver or engine.
+ * This includes a linked list node for managing flow rules in a list, a pointer
+ * to the device (driver instance) the flow belongs to, and the engine index
+ * that created the flow.
+ *
+ * With rte_flow API, each driver is meant to define its own rte_flow structure
+ * that contains driver-specific data. This structure must start with the
+ * `ci_flow` structure defined here, followed by driver-specific fields.
+ *
+ * Additionally, each *engine* may want to define its own flow rule structure
+ * that contains actual engine-specific data. This structure must start with the
+ * driver-wide `rte_flow` structure such that it contains everything before it,
+ * followed by engine-specific fields.
+ *
+ * IMPORTANT:
+ *
+ * All of these structures will be referred to by the same pointer and can be
+ * freely (and safely) cast between each other *as long as* each structure
+ * definition has the parent structure as its first member. E.g. the common flow
+ * struct is `ci_flow`, and the driver-specific `rte_flow` must be defined as
+ * follows:
+ *
+ * struct rte_flow {
+ * struct ci_flow base;
+ * ...any driver-specific fields...
+ * }
+ *
+ * If the engine needs to define its own flow structure, in turn it should be
+ * defined as follows:
+ *
+ * struct ixgbe_fdir_flow {
+ * struct rte_flow base;
+ * ...any engine-specific fields...
+ * }
+ *
+ * This ensures pointer conversion safety between all three types:
+ *
+ * struct ci_flow *flow = ...;
+ * struct rte_flow *rte_flow = (struct rte_flow *)flow;
+ * struct engine_specific_flow *es_flow = (struct engine_specific_flow *)flow;
+ *
+ * The engine structure provides a `flow_size` field that indicates how much
+ * memory is required for a particular engine's flow structure. The driver must
+ * provide that value for each engine, as it will be used to size flow structure
+ * allocations. If the engine does not require any memory beyond the `rte_flow`
+ * structure, this value should be set to `sizeof(rte_flow)` for those engines,
+ * thus making it equal to sizeof(ci_flow) plus any driver-specific data.
+ *
+ * Engine references and engine_idx:
+ *
+ * The engine index is treated as the engine type discriminator and is stored in
+ * every flow. Runtime code then reconstructs a `ci_flow_engine_ref`
+ * (engine pointer + index) from the immutable engine list and this stored
+ * index. This gives two important guarantees:
+ *
+ * - The framework works only with immutable engine definitions/lists.
+ * - We avoid repeated pointer-to-index lookups after flow creation; rebuilding
+ * a ref from index is direct and keeps call sites explicit about both values.
+ *
+ * This is particularly important for secondary-process compatibility where the
+ * index is the stable identity and pointer values are only meaningful in the
+ * context of the shared immutable engine list.
+ *
+ * Code that works with engine pointers and engine indices should be careful,
+ * and should treat engine references as the basic building block.
+ *
+ * Per-engine private data (engine_priv):
+ *
+ * Each flow will also carry a pointer to engine-private data, to allow for
+ * reaching engine-specific data, such as custom allocation structures to
+ * be used with flow_alloc/flow_free at allocation time.
+ *
+ * Fallback allocation flag:
+ *
+ * When `flow_alloc` is provided by the driver, but it returns NULL, the framework
+ * will fall back to rte_zmalloc()-based allocation. The `fallback_alloc` flag is
+ * then used to route subsequent deallocation to rte_free(). This implies that for
+ * an engine, `flow_alloc` returning NULL *is not considered to indicate an error*.
+ */
+struct ci_flow {
+ TAILQ_ENTRY(ci_flow) node;
+ /* device this flow belongs to */
+ struct rte_eth_dev_data *dev_data;
+ /* index of engine this flow was created by */
+ size_t engine_idx;
+ /* per-engine private data pointer, set by the framework at alloc time */
+ void *engine_priv;
+ /* set if engine allocator callback existed but fallback allocator was used */
+ bool fallback_alloc;
+};
+
+/* flow engine definition */
+struct ci_flow_engine {
+ /* engine name */
+ const char *name;
+ /* size of scratch space structure, can be 0 */
+ size_t ctx_size;
+ /* size of flow rule structure, must not be 0 */
+ size_t flow_size;
+ /* size of per-device engine private data, can be 0 */
+ size_t priv_size;
+ /* ops for this flow engine */
+ const struct ci_flow_engine_ops *ops;
+ /* pattern graph this engine supports - can be NULL */
+ const struct flow_graph *graph;
+};
+
+/* engine reference: immutable engine pointer paired with its list index */
+struct ci_flow_engine_ref {
+ const struct ci_flow_engine *engine;
+ size_t engine_idx;
+};
+
+#define CI_FLOW_ENGINE_MAX 64
+
+/* flow engine list definition */
+struct ci_flow_engine_list {
+ /* NULL-terminated immutable array of flow engine pointers */
+ const struct ci_flow_engine * const engines[CI_FLOW_ENGINE_MAX];
+};
+
+/* flow engine configuration - each device must have its own instance */
+struct ci_flow_engine_conf {
+ /* lock to protect config */
+ rte_rwlock_t config_lock;
+ /* list of flows created on this device */
+ TAILQ_HEAD(ci_flow_list, ci_flow) flows;
+ /* bitmask of enabled engines */
+ uint64_t enabled_engines;
+ /* back-reference to device structure */
+ struct rte_eth_dev_data *dev_data;
+ /* reference to the driver's engine list */
+ const struct ci_flow_engine_list *engines;
+ /* per-engine private data pointers, indexed by engine index */
+ void *engine_priv[CI_FLOW_ENGINE_MAX];
+};
+
+/* build engine ref from engine config and engine index - caller must hold config lock */
+static inline struct ci_flow_engine_ref
+ci_flow_engine_ref_from_idx(const struct ci_flow_engine_conf *engine_conf,
+ const size_t engine_idx)
+{
+ /* check if valid reference can be constructed at all */
+ if (engine_conf == NULL || engine_conf->engines == NULL ||
+ engine_idx >= CI_FLOW_ENGINE_MAX) {
+ return (struct ci_flow_engine_ref) {
+ .engine = NULL,
+ .engine_idx = CI_FLOW_ENGINE_MAX
+ };
+ }
+ /* the index itself is valid, but there may be no engine */
+ if (engine_conf->engines->engines[engine_idx] == NULL) {
+ return (struct ci_flow_engine_ref) {
+ .engine = NULL,
+ .engine_idx = CI_FLOW_ENGINE_MAX
+ };
+ }
+
+ return (struct ci_flow_engine_ref) {
+ .engine = engine_conf->engines->engines[engine_idx],
+ .engine_idx = engine_idx,
+ };
+}
+
+/* helper macro to iterate over list of engines as engine refs */
+#define CI_FLOW_ENGINE_LIST_FOREACH(engine_ref, engine_conf) \
+ for (size_t __ci_flow_engine_idx = 0; \
+ __ci_flow_engine_idx < CI_FLOW_ENGINE_MAX && \
+ (((engine_ref) = ci_flow_engine_ref_from_idx((engine_conf), \
+ __ci_flow_engine_idx)).engine != NULL); \
+ __ci_flow_engine_idx++)
+
+/* basic checks for flow engine validity */
+static inline bool
+ci_flow_engine_is_valid(const struct ci_flow_engine *engine)
+{
+ /* is the pointer valid? */
+ if (engine == NULL)
+ return false;
+ /* does the engine have a name? */
+ if (engine->name == NULL)
+ return false;
+ /* does the engine have ops? */
+ if (engine->ops == NULL)
+ return false;
+ /* does the engine have mandatory ctx_init op? */
+ if (engine->ops->ctx_init == NULL)
+ return false;
+ /* flow size cannot be less than ci_flow */
+ if (engine->flow_size < sizeof(struct ci_flow))
+ return false;
+ /* alloc and free must both be defined or NULL */
+ if ((engine->ops->flow_alloc == NULL) != (engine->ops->flow_free == NULL))
+ return false;
+ /* register and unregister must both be defined or NULL */
+ if ((engine->ops->flow_register == NULL) != (engine->ops->flow_unregister == NULL))
+ return false;
+ /* engine looks valid */
+ return true;
+}
+
+/* helper to check whether an engine is enabled in the bitmask - caller must hold config lock */
+static inline bool
+ci_flow_engine_is_enabled(const struct ci_flow_engine_conf *conf,
+ const size_t engine_idx)
+{
+ return (conf->enabled_engines & RTE_BIT64(engine_idx)) != 0;
+}
+
+/* helper to enable an engine in the bitmask - caller must hold config lock */
+static inline void
+ci_flow_engine_set_enabled(struct ci_flow_engine_conf *conf, const size_t engine_idx,
+ bool enabled)
+{
+ if (enabled)
+ conf->enabled_engines |= RTE_BIT64(engine_idx);
+ else
+ conf->enabled_engines &= ~RTE_BIT64(engine_idx);
+}
+
+static inline struct ci_flow *
+ci_flow_alloc(const struct ci_flow_engine_conf *engine_conf,
+ struct ci_flow_engine_ref engine_ref)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+ void *priv = engine_conf->engine_priv[engine_ref.engine_idx];
+ struct ci_flow *flow = NULL;
+ bool fallback = false;
+
+ /* if engine has an allocator callback, try it first */
+ if (engine->ops->flow_alloc != NULL)
+ flow = engine->ops->flow_alloc(engine, engine_conf->dev_data, priv);
+ /* if allocator callback is not defined or failed, use default allocator */
+ if (flow == NULL) {
+ flow = (struct ci_flow *)rte_zmalloc(NULL, engine->flow_size, 0);
+
+ /* if callback exists and we're here, allocation has fallen back */
+ if (flow != NULL && engine->ops->flow_alloc != NULL)
+ fallback = true;
+ }
+ /* set the engine data to enable correct deallocation in case of failure */
+ if (flow != NULL) {
+ /* erase the common parts - the rest is left up to the engine */
+ memset(flow, 0, sizeof(struct ci_flow));
+ flow->fallback_alloc = fallback;
+ flow->engine_idx = engine_ref.engine_idx;
+ flow->dev_data = engine_conf->dev_data;
+ flow->engine_priv = priv;
+ }
+ return flow;
+}
+
+static inline void
+ci_flow_free(struct ci_flow_engine_ref engine_ref, struct ci_flow *flow)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+
+ if (engine->ops->flow_free != NULL && !flow->fallback_alloc)
+ engine->ops->flow_free(flow, flow->dev_data, flow->engine_priv);
+ else
+ rte_free(flow);
+}
+
+/* track a flow in driver-internal state (Track stage) - caller must hold config lock */
+static inline int
+ci_flow_register(struct ci_flow_engine_ref engine_ref,
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+
+ if (engine->ops->flow_register != NULL)
+ return engine->ops->flow_register(flow, error);
+
+ return 0;
+}
+
+/* untrack a flow from driver-internal state (Track stage) - caller must hold config lock */
+static inline int
+ci_flow_unregister(struct ci_flow_engine_ref engine_ref,
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+
+ if (engine->ops->flow_unregister != NULL)
+ return engine->ops->flow_unregister(flow, error);
+
+ return 0;
+}
+
+/* allocate per-device engine private data and call init - caller must hold config lock */
+static inline int
+ci_flow_engine_init(struct ci_flow_engine_conf *engine_conf,
+ struct ci_flow_engine_ref engine_ref)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+ void *priv = NULL;
+ int ret;
+
+ if (engine->priv_size > 0) {
+ priv = rte_zmalloc(engine->name, engine->priv_size, 0);
+ if (priv == NULL) {
+ ret = -ENOMEM;
+ goto err;
+ }
+ }
+
+ if (engine->ops->engine_init != NULL) {
+ ret = engine->ops->engine_init(engine, engine_conf->dev_data, priv);
+ if (ret != 0)
+ goto err;
+ }
+ engine_conf->engine_priv[engine_ref.engine_idx] = priv;
+ return 0;
+err:
+ rte_free(priv);
+ return ret;
+}
+
+/* call uninit and free per-device engine private data - caller must hold config lock */
+static inline void
+ci_flow_engine_uninit(struct ci_flow_engine_conf *engine_conf,
+ struct ci_flow_engine_ref engine_ref)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+ void *priv;
+
+ priv = engine_conf->engine_priv[engine_ref.engine_idx];
+
+ CI_DRV_LOG(DEBUG, "engine '%s': uninit", engine->name);
+ /* ignore uninit errors */
+ if (engine->ops->engine_uninit != NULL)
+ engine->ops->engine_uninit(engine, priv);
+
+ rte_free(priv);
+ engine_conf->engine_priv[engine_ref.engine_idx] = NULL;
+}
+
+/* disable all engines for a specific driver instance - caller must serialize teardown */
+static inline void
+ci_flow_engine_conf_reset(struct ci_flow_engine_conf *engine_conf)
+{
+ struct ci_flow_engine_ref engine_ref;
+ struct ci_flow *flow, *tmp;
+
+ /* free all flows - shouldn't have any at this point */
+ RTE_TAILQ_FOREACH_SAFE(flow, &engine_conf->flows, node, tmp) {
+ struct rte_flow_error unreg_error = { 0 };
+ int unreg_ret;
+
+ TAILQ_REMOVE(&engine_conf->flows, flow, node);
+ /* can't make it into the list if this was invalid, so no checks */
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf, flow->engine_idx);
+ /* untrack before free; ignore errors on this teardown path */
+ unreg_ret = ci_flow_unregister(engine_ref, flow, &unreg_error);
+ if (unreg_ret != 0)
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to unregister flow: %s",
+ engine_ref.engine->name, unreg_error.message);
+ ci_flow_free(engine_ref, flow);
+ }
+
+ CI_FLOW_ENGINE_LIST_FOREACH(engine_ref, engine_conf) {
+ if (!ci_flow_engine_is_enabled(engine_conf, engine_ref.engine_idx))
+ continue;
+ /* ignore errors */
+ ci_flow_engine_uninit(engine_conf, engine_ref);
+ ci_flow_engine_set_enabled(engine_conf,
+ engine_ref.engine_idx, false);
+ }
+
+ /* erase device pointer */
+ engine_conf->dev_data = NULL;
+ engine_conf->engines = NULL;
+}
+
+/* enable all engines for a specific driver instance - caller must serialize initialization */
+static inline int
+ci_flow_engine_conf_init(struct ci_flow_engine_conf *engine_conf,
+ const struct ci_flow_engine_list *engine_list,
+ struct rte_eth_dev_data *dev_data)
+{
+ struct ci_flow_engine_ref engine_ref;
+
+ /* reject invalid configuration */
+ if (engine_conf == NULL || engine_list == NULL || dev_data == NULL)
+ return -1;
+
+ /* init the lock */
+ rte_rwlock_init(&engine_conf->config_lock);
+
+ /* store data in conf */
+ engine_conf->dev_data = dev_data;
+ engine_conf->engines = engine_list;
+
+ /* init the flow list */
+ TAILQ_INIT(&engine_conf->flows);
+
+ /* enable all engines */
+ CI_FLOW_ENGINE_LIST_FOREACH(engine_ref, engine_conf) {
+ /* skip invalid engines */
+ if (!ci_flow_engine_is_valid(engine_ref.engine)) {
+ CI_DRV_LOG(DEBUG, "engine[%zu]: invalid, skipping",
+ engine_ref.engine_idx);
+ continue;
+ }
+ if (ci_flow_engine_init(engine_conf, engine_ref) != 0) {
+ CI_DRV_LOG(DEBUG, "engine '%s': init failed, skipping",
+ engine_ref.engine->name);
+ continue;
+ }
+
+ ci_flow_engine_set_enabled(engine_conf,
+ engine_ref.engine_idx, true);
+ CI_DRV_LOG(DEBUG, "engine '%s': enabled", engine_ref.engine->name);
+ }
+ return 0;
+}
+
+/* check whether a flow is valid for a specific engine configuration - caller must hold config lock */
+static inline bool
+ci_flow_is_valid(const struct ci_flow *flow,
+ const struct ci_flow_engine_conf *engine_conf)
+{
+ /* is the pointer valid? */
+ if (flow == NULL)
+ return false;
+ /* is the conf initialized? */
+ if (engine_conf == NULL || engine_conf->dev_data == NULL ||
+ engine_conf->engines == NULL)
+ return false;
+ /* does the flow belong to this device? */
+ if (flow->dev_data != engine_conf->dev_data)
+ return false;
+ /* can we find the engine that created this flow? */
+ if (flow->engine_idx >= CI_FLOW_ENGINE_MAX)
+ return false;
+ /* engine must be enabled */
+ if (!ci_flow_engine_is_enabled(engine_conf, flow->engine_idx))
+ return false;
+ /* flow looks valid */
+ return true;
+}
+
+/* default empty pattern graph definitions */
+enum ci_flow_empty_graph_node_id {
+ CI_FLOW_EMPTY_GRAPH_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ CI_FLOW_EMPTY_GRAPH_NODE_ANY,
+ CI_FLOW_EMPTY_GRAPH_NODE_END,
+};
+
+static const struct flow_graph ci_flow_empty_graph = {
+ .nodes = (struct flow_graph_node []) {
+ [CI_FLOW_EMPTY_GRAPH_NODE_START] = {
+ .name = "START",
+ },
+ [CI_FLOW_EMPTY_GRAPH_NODE_ANY] = {
+ .name = "ANY",
+ .type = RTE_FLOW_ITEM_TYPE_ANY,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [CI_FLOW_EMPTY_GRAPH_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge []) {
+ [CI_FLOW_EMPTY_GRAPH_NODE_START] = {
+ .next = (size_t []) {
+ CI_FLOW_EMPTY_GRAPH_NODE_ANY,
+ CI_FLOW_EMPTY_GRAPH_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ [CI_FLOW_EMPTY_GRAPH_NODE_ANY] = {
+ .next = (size_t []) {
+ CI_FLOW_EMPTY_GRAPH_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END,
+ },
+ },
+ }
+};
+
+/* concrete pattern-matching mode selected from graph/callback presence */
+enum ci_match_type {
+ CI_MATCH_EMPTY, /* no graph, no callback */
+ CI_MATCH_CALLBACK, /* callback only */
+ CI_MATCH_GRAPH, /* graph only */
+ CI_MATCH_ALL, /* callback + graph */
+};
+
+/*
+ * helper to match a pattern against an engine's pattern graph and/or callback,
+ * depending on the match type. Caller must hold config lock.
+ */
+static inline int
+ci_flow_match(const struct ci_flow_engine *engine,
+ const struct rte_flow_item pattern[],
+ struct ci_flow_engine_ctx *ctx,
+ enum ci_match_type match_type,
+ struct rte_flow_error *error)
+{
+ switch (match_type) {
+ case CI_MATCH_EMPTY:
+ /* for empty matching, NULL pattern is not an error */
+ if (pattern != NULL) {
+ return flow_graph_parse(&ci_flow_empty_graph,
+ pattern, error, ctx);
+ }
+ return 0;
+
+ case CI_MATCH_CALLBACK:
+ /* for callback matching, pattern cannot be NULL */
+ if (pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+ "Pattern cannot be NULL");
+ }
+ return engine->ops->pattern_parse(ctx, pattern, error);
+
+ case CI_MATCH_GRAPH:
+ /* for graph matching, pattern cannot be NULL */
+ if (pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+ "Pattern cannot be NULL");
+ }
+ return flow_graph_parse(engine->graph, pattern, error, ctx);
+
+ case CI_MATCH_ALL:
+ {
+ int ret;
+
+ /* for callback + graph matching, pattern cannot be NULL */
+ if (pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+ "Pattern cannot be NULL");
+ }
+ ret = engine->ops->pattern_parse(ctx, pattern, error);
+ if (ret != 0)
+ return ret;
+ return flow_graph_parse(engine->graph, pattern, error, ctx);
+ }
+
+ default:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+ "Invalid match type");
+ }
+}
+
+/* parse a flow using a specific engine - caller must hold config lock */
+static inline int
+ci_flow_parse(const struct ci_flow_engine_conf *engine_conf,
+ const struct ci_flow_engine *engine,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ enum ci_match_type match_type;
+ struct ci_flow_engine_ctx *ctx;
+ int ret = 0;
+
+ /*
+ * Determine the type of matching we are going to perform based on the
+ * presence of pattern graph and pattern_parse callback. The logic is as
+ * follows:
+ *
+ * - if graph but no callback, match against graph
+ *
+ * Expected default case: pattern matching is graph based, no special
+ * handling for any pattern items.
+ *
+ * - if both graph and callback, match against callback + graph
+ *
+ * Preprocessor case, i.e. preprocess the pattern with the callback
+ * before handling the matching to the graph engine. The assumption is
+ * that the graph will be set up with a proper ignore list to skip over
+ * nodes that weren't meant for the graph processing.
+ *
+ * - if no graph but callback, match against callback
+ *
+ * Fully custom pattern parsing case.
+ *
+ * - if no graph and no callback, match against empty graph
+ *
+ * "Pattern is not meaningful" case, for engines that do not care about
+ * the pattern at all. A default matching behavior against empty
+ * patterns is provided (i.e. allow NULL pattern, and allow END or ANY
+ * -> END patterns). Note that this is not the same as ignoring pattern
+ * entirely: the engine will still reject patterns that are not empty.
+ */
+ match_type = engine->graph == NULL ?
+ (engine->ops->pattern_parse == NULL ? CI_MATCH_EMPTY : CI_MATCH_CALLBACK) :
+ (engine->ops->pattern_parse == NULL ? CI_MATCH_GRAPH : CI_MATCH_ALL);
+
+ CI_DRV_LOG(DEBUG, "engine '%s': parsing flow", engine->name);
+
+ /* allocate context */
+ ctx = (struct ci_flow_engine_ctx *)calloc(1,
+ RTE_MAX(engine->ctx_size, sizeof(struct ci_flow_engine_ctx)));
+ if (ctx == NULL) {
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory for rule engine context");
+ }
+ ctx->dev_data = engine_conf->dev_data;
+ ctx->attr = attr;
+ ctx->pattern = pattern;
+ ctx->actions = actions;
+ flow->dev_data = engine_conf->dev_data;
+
+ /* parse flow parameters */
+ ret = engine->ops->ctx_init(actions, attr, ctx, error);
+
+ /* context init failed - that means engine can't be used for this flow */
+ if (ret != 0)
+ goto free_ctx;
+
+ /* match the pattern */
+ ret = ci_flow_match(engine, pattern, ctx, match_type, error);
+
+ /* check if pattern didn't match */
+ if (ret != 0)
+ goto free_ctx;
+
+ /* final verification, if the operation is defined */
+ if (engine->ops->ctx_finalize != NULL)
+ ret = engine->ops->ctx_finalize(ctx, error);
+
+ /* finalization failed - mismatch between parsed data and context data */
+ if (ret != 0)
+ goto free_ctx;
+
+ /* if we need to build rules from context, do it */
+ if (engine->ops->ctx_to_flow != NULL) {
+ ret = engine->ops->ctx_to_flow(ctx, flow, error);
+
+ /* flow building failed - something wrong with context data */
+ if (ret != 0)
+ goto free_ctx;
+ }
+ /* success */
+ ret = 0;
+
+free_ctx:
+ free(ctx);
+ return ret;
+}
+
+/* install a flow using its appropriate engine - caller must hold config lock */
+static inline int
+ci_flow_install(struct ci_flow_engine_ref engine_ref,
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+
+ if (engine->ops->flow_install != NULL)
+ return engine->ops->flow_install(flow, error);
+
+ return 0;
+}
+
+/* uninstall a flow using its appropriate engine - caller must hold config lock */
+static inline int
+ci_flow_uninstall(struct ci_flow_engine_ref engine_ref,
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ const struct ci_flow_engine *engine = engine_ref.engine;
+
+ /* uninstall the flow if required */
+ if (engine->ops->flow_uninstall != NULL)
+ return engine->ops->flow_uninstall(flow, error);
+
+ return 0;
+}
+
+/*
+ * The following functions are designed to be called from the context of
+ * rte_flow API implementations and are provided as default implementations.
+ *
+ * These default implementations take the config lock internally.
+ */
+
+/* default implementation of rte_flow_create using flow engines */
+static inline struct rte_flow *
+ci_flow_create(struct ci_flow_engine_conf *engine_conf,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct ci_flow_engine_ref engine_ref;
+ struct ci_flow *flow = NULL;
+ int ret;
+
+ if (attr == NULL || actions == NULL) {
+ CI_DRV_LOG(DEBUG, "attr or actions is NULL");
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR, NULL,
+ "Attributes and actions cannot be NULL");
+ return NULL;
+ }
+
+ /* lock the config for writing */
+ rte_rwlock_write_lock(&engine_conf->config_lock);
+
+ /* find an engine that can handle this flow */
+ CI_FLOW_ENGINE_LIST_FOREACH(engine_ref, engine_conf) {
+ if (!ci_flow_engine_is_enabled(engine_conf, engine_ref.engine_idx))
+ continue;
+
+ flow = ci_flow_alloc(engine_conf, engine_ref);
+ if (flow == NULL) {
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to allocate flow",
+ engine_ref.engine->name);
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory for flow rule");
+ /* this is a serious error so don't continue */
+ goto unlock;
+ }
+
+ ret = ci_flow_parse(engine_conf, engine_ref.engine, attr, pattern,
+ actions, flow, error);
+
+ /* parsing failed - free the flow and try next engine */
+ if (ret != 0) {
+ ci_flow_free(engine_ref, flow);
+ if (error != NULL)
+ CI_DRV_LOG(DEBUG, "engine '%s' rejected flow: %s",
+ engine_ref.engine->name, error->message);
+ else
+ CI_DRV_LOG(DEBUG, "engine '%s' rejected flow",
+ engine_ref.engine->name);
+ continue;
+ }
+
+ CI_DRV_LOG(DEBUG, "engine '%s' accepted flow, committing",
+ engine_ref.engine->name);
+
+ /* engine accepted the flow - Track then Apply are a hard commitment */
+ ret = ci_flow_register(engine_ref, flow, error);
+ if (ret != 0) {
+ /* track failed after parse accepted - do not try other engines */
+ if (error != NULL)
+ CI_DRV_LOG(DEBUG, "engine '%s' register failed: %s",
+ engine_ref.engine->name, error->message);
+ else
+ CI_DRV_LOG(DEBUG, "engine '%s' register failed",
+ engine_ref.engine->name);
+ ci_flow_free(engine_ref, flow);
+ flow = NULL;
+ goto unlock;
+ }
+
+ ret = ci_flow_install(engine_ref, flow, error);
+ if (ret != 0) {
+ struct rte_flow_error unreg_error = { 0 };
+ int unreg_ret;
+
+ /* install failed after parse accepted - this is an
+ * engine bug, do not try other engines
+ */
+ if (error != NULL)
+ CI_DRV_LOG(DEBUG, "engine '%s' install failed: %s",
+ engine_ref.engine->name, error->message);
+ else
+ CI_DRV_LOG(DEBUG, "engine '%s' install failed",
+ engine_ref.engine->name);
+ /* roll back the Track stage, preserving the install error */
+ unreg_ret = ci_flow_unregister(engine_ref, flow, &unreg_error);
+ if (unreg_ret != 0)
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to unregister: %s",
+ engine_ref.engine->name, unreg_error.message);
+ ci_flow_free(engine_ref, flow);
+ flow = NULL;
+ goto unlock;
+ }
+
+ CI_DRV_LOG(DEBUG, "flow installed by engine '%s'",
+ engine_ref.engine->name);
+ /* success */
+ TAILQ_INSERT_TAIL(&engine_conf->flows, flow, node);
+ goto unlock;
+ }
+
+ /* no engine could handle this flow */
+ CI_DRV_LOG(DEBUG, "no engine accepted the flow");
+ flow = NULL;
+ rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "No flow engine could handle the requested flow");
+unlock:
+ rte_rwlock_write_unlock(&engine_conf->config_lock);
+
+ return (struct rte_flow *)flow;
+}
+
+/* default implementation of rte_flow_validate using flow engines */
+static inline int
+ci_flow_validate(struct ci_flow_engine_conf *engine_conf,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct ci_flow_engine_ref engine_ref;
+ int ret;
+
+ if (attr == NULL || actions == NULL) {
+ CI_DRV_LOG(DEBUG, "attr or actions is NULL");
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR, NULL,
+ "Attributes and actions cannot be NULL");
+ }
+
+ /* lock the config for reading */
+ rte_rwlock_read_lock(&engine_conf->config_lock);
+
+ /* find an engine that can handle this flow */
+ CI_FLOW_ENGINE_LIST_FOREACH(engine_ref, engine_conf) {
+ struct ci_flow *flow;
+
+ if (!ci_flow_engine_is_enabled(engine_conf, engine_ref.engine_idx))
+ continue;
+
+ /* use OS allocator as we're not keeping the flow */
+ flow = (struct ci_flow *)calloc(1, engine_ref.engine->flow_size);
+ if (flow == NULL) {
+ /* this is a serious error so don't continue */
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to allocate flow",
+ engine_ref.engine->name);
+ ret = rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory for flow rule");
+ goto unlock;
+ }
+ /* set up the flow fields */
+ flow->fallback_alloc = false;
+ flow->engine_idx = engine_ref.engine_idx;
+ flow->dev_data = engine_conf->dev_data;
+ flow->engine_priv = engine_conf->engine_priv[engine_ref.engine_idx];
+
+ /* try to parse the flow with this engine */
+ ret = ci_flow_parse(engine_conf, engine_ref.engine, attr, pattern,
+ actions, flow, error);
+ free(flow);
+
+ if (ret == 0) {
+ CI_DRV_LOG(DEBUG, "engine '%s' accepted flow",
+ engine_ref.engine->name);
+ goto unlock;
+ } else if (error != NULL) {
+ CI_DRV_LOG(DEBUG, "engine '%s' rejected flow: %s",
+ engine_ref.engine->name, error->message);
+ } else {
+ CI_DRV_LOG(DEBUG, "engine '%s' rejected flow",
+ engine_ref.engine->name);
+ }
+ }
+ /* no engine could handle this flow */
+ CI_DRV_LOG(DEBUG, "no engine accepted the flow");
+ ret = rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "No flow engine could handle the requested flow");
+unlock:
+ rte_rwlock_read_unlock(&engine_conf->config_lock);
+ return ret;
+}
+
+/* default implementation of rte_flow_destroy using flow engines. */
+static inline int
+ci_flow_destroy(struct ci_flow_engine_conf *engine_conf,
+ struct rte_flow *rte_flow,
+ struct rte_flow_error *error)
+{
+ struct ci_flow *flow = (struct ci_flow *)rte_flow;
+ struct ci_flow_engine_ref engine_ref;
+ struct rte_flow_error unreg_error = {0};
+ int unreg_ret, ret = 0;
+
+ if (rte_flow == NULL) {
+ CI_DRV_LOG(DEBUG, "flow handle is NULL");
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Flow handle cannot be NULL");
+ }
+
+ /* lock the config for writing */
+ rte_rwlock_write_lock(&engine_conf->config_lock);
+
+ /* validate the flow */
+ if (!ci_flow_is_valid(flow, engine_conf)) {
+ CI_DRV_LOG(DEBUG, "invalid flow handle");
+ ret = rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Invalid flow handle");
+ goto unlock;
+ }
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf, flow->engine_idx);
+ CI_DRV_LOG(DEBUG, "uninstalling flow (engine '%s')",
+ engine_ref.engine->name);
+
+ ret = ci_flow_uninstall(engine_ref, flow, error);
+
+ if (ret != 0) {
+ if (error != NULL)
+ CI_DRV_LOG(DEBUG, "uninstall failed: %s", error->message);
+ else
+ CI_DRV_LOG(DEBUG, "uninstall failed");
+ goto unlock;
+ }
+
+ /* untrack after HW removal; ignore errors on this teardown path */
+ unreg_ret = ci_flow_unregister(engine_ref, flow, &unreg_error);
+ if (unreg_ret != 0)
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to unregister flow: %s",
+ engine_ref.engine->name, unreg_error.message);
+
+ /* remove the flow from the list and free it */
+ TAILQ_REMOVE(&engine_conf->flows, flow, node);
+ ci_flow_free(engine_ref, flow);
+unlock:
+ rte_rwlock_write_unlock(&engine_conf->config_lock);
+
+ return ret;
+}
+
+/* default implementation of rte_flow_flush using flow engines */
+static inline int
+ci_flow_flush(struct ci_flow_engine_conf *engine_conf,
+ struct rte_flow_error *error)
+{
+ struct ci_flow *flow, *tmp;
+
+ CI_DRV_LOG(DEBUG, "removing all flows");
+
+ /* lock the config for writing */
+ rte_rwlock_write_lock(&engine_conf->config_lock);
+
+ /* iterate over all flows and uninstall them */
+ RTE_TAILQ_FOREACH_SAFE(flow, &engine_conf->flows, node, tmp) {
+ struct ci_flow_engine_ref engine_ref;
+ struct rte_flow_error unreg_error = { 0 };
+ int uninstall_ret;
+ int unreg_ret;
+
+ /* this shouldn't happen */
+ if (!ci_flow_is_valid(flow, engine_conf))
+ continue;
+
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf,
+ flow->engine_idx);
+
+ uninstall_ret = ci_flow_uninstall(engine_ref, flow, error);
+
+ /* if uninstall failed, log but ignore failure */
+ if (uninstall_ret != 0) {
+ if (error != NULL)
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to uninstall flow: %s",
+ engine_ref.engine->name, error->message);
+ else
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to uninstall flow",
+ engine_ref.engine->name);
+ }
+
+ /* untrack after HW removal; ignore errors on this teardown path, but log them */
+ unreg_ret = ci_flow_unregister(engine_ref, flow, &unreg_error);
+ if (unreg_ret != 0)
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to untrack flow: %s",
+ engine_ref.engine->name, unreg_error.message);
+
+ TAILQ_REMOVE(&engine_conf->flows, flow, node);
+ ci_flow_free(engine_ref, flow);
+ }
+
+ rte_rwlock_write_unlock(&engine_conf->config_lock);
+
+ return 0;
+}
+
+/* default implementation of rte_flow_query using flow engines */
+static inline int
+ci_flow_query(struct ci_flow_engine_conf *engine_conf,
+ struct rte_flow *rte_flow,
+ const struct rte_flow_action *action,
+ void *data,
+ struct rte_flow_error *error)
+{
+ struct ci_flow *flow = (struct ci_flow *)rte_flow;
+ struct ci_flow_engine_ref engine_ref;
+ int ret;
+
+ if (action == NULL || data == NULL) {
+ CI_DRV_LOG(DEBUG, "action or data is NULL");
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "Action or data cannot be NULL");
+ }
+
+ /* lock the config for reading */
+ rte_rwlock_read_lock(&engine_conf->config_lock);
+
+ /* validate the flow first */
+ if (!ci_flow_is_valid(flow, engine_conf)) {
+ CI_DRV_LOG(DEBUG, "invalid flow handle");
+ ret = rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Invalid flow handle");
+ goto unlock;
+ }
+ /* find the engine that created this flow */
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf, flow->engine_idx);
+ /* query the flow if supported */
+ if (engine_ref.engine->ops->flow_query != NULL) {
+ ret = engine_ref.engine->ops->flow_query(flow, action, data, error);
+
+ if (ret != 0 && error != NULL)
+ CI_DRV_LOG(DEBUG, "engine '%s' query failed: %s",
+ engine_ref.engine->name, error->message);
+ else if (ret != 0)
+ CI_DRV_LOG(DEBUG, "engine '%s' query failed",
+ engine_ref.engine->name);
+ } else {
+ CI_DRV_LOG(DEBUG, "engine '%s' does not support querying",
+ engine_ref.engine->name);
+ ret = rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Flow engine does not support querying");
+ }
+unlock:
+ rte_rwlock_read_unlock(&engine_conf->config_lock);
+
+ return ret;
+}
+
+/* dump flow rule chunks using memdump - helper for ci_flow_dump */
+#define CI_FLOW_DUMP_CHUNK_BYTES 32
+
+static inline void
+ci_flow_dump_one(FILE *file, const char *driver, const char *engine,
+ const void *data, size_t data_len)
+{
+ const uint8_t *raw = (const uint8_t *)data;
+ const size_t nchunks =
+ (data_len + CI_FLOW_DUMP_CHUNK_BYTES - 1) /
+ CI_FLOW_DUMP_CHUNK_BYTES;
+ char title[64];
+ size_t ci;
+
+ fprintf(file, "FLOW DUMP: driver=%s engine=%s\n", driver, engine);
+ fprintf(file, "FLOW DUMP: DATA size=%zu chunks=%zu chunk_bytes=%d\n",
+ data_len, nchunks, CI_FLOW_DUMP_CHUNK_BYTES);
+
+ for (ci = 0; ci < nchunks; ci++) {
+ const size_t off = ci * CI_FLOW_DUMP_CHUNK_BYTES;
+ const size_t clen =
+ RTE_MIN((size_t)CI_FLOW_DUMP_CHUNK_BYTES,
+ data_len - off);
+ snprintf(title, sizeof(title), "FLOW DUMP: chunk %03zu/%03zu",
+ ci + 1, nchunks);
+ rte_memdump(file, title, raw + off, clen);
+ }
+}
+
+/* default implementation of rte_flow_dev_dump using flow engines */
+static inline int
+ci_flow_dump(struct ci_flow_engine_conf *engine_conf,
+ struct rte_flow *flow,
+ FILE *file,
+ struct rte_flow_error *error)
+{
+ struct ci_flow_engine_ref engine_ref;
+ struct ci_flow *cur;
+ bool found = false;
+ const char *driver_name =
+#ifdef RTE_COMPONENT_NAME
+ RTE_STR(RTE_COMPONENT_NAME);
+#else
+ "unknown";
+#endif
+
+ rte_rwlock_read_lock(&engine_conf->config_lock);
+
+ TAILQ_FOREACH(cur, &engine_conf->flows, node) {
+ const void *data;
+ size_t data_len;
+
+ if (flow != NULL && (struct rte_flow *)cur != flow)
+ continue;
+
+ /* is this flow valid? */
+ if (!ci_flow_is_valid(cur, engine_conf)) {
+ CI_DRV_LOG(DEBUG, "invalid flow handle: %p, skipping", cur);
+ continue;
+ }
+
+ found = true;
+
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf, cur->engine_idx);
+
+ data = RTE_PTR_ADD(cur, sizeof(struct ci_flow));
+ data_len = engine_ref.engine->flow_size - sizeof(struct ci_flow);
+
+ /*
+ * when data_len is 0 the dump loop would not access the data
+ * pointer, but static analysis tools may flag this as a
+ * potential NULL dereference, so skip dump when data_len is 0.
+ */
+ if (data_len == 0) {
+ CI_DRV_LOG(DEBUG, "flow data length is 0, skipping dump");
+ continue;
+ }
+
+ ci_flow_dump_one(file, driver_name, engine_ref.engine->name,
+ data, data_len);
+ }
+
+ rte_rwlock_read_unlock(&engine_conf->config_lock);
+
+ if (flow != NULL && !found) {
+ return rte_flow_error_set(error, ENOENT,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Flow not found");
+ }
+
+ return 0;
+}
+
+/*
+ * The following functions are designed to be called from the context of
+ * the driver lifecycle functions such as `dev_start`.
+ *
+ * These default implementations take the config lock internally.
+ */
+
+/* re-install all flows */
+static inline int
+ci_flow_replay(struct ci_flow_engine_conf *engine_conf)
+{
+ struct rte_flow_error error = {0};
+ struct ci_flow *flow;
+
+ CI_DRV_LOG(DEBUG, "replaying all flows");
+
+ /* lock the config for writing */
+ rte_rwlock_write_lock(&engine_conf->config_lock);
+
+ TAILQ_FOREACH(flow, &engine_conf->flows, node) {
+ struct ci_flow_engine_ref engine_ref;
+ int install_ret;
+
+ /* this shouldn't happen */
+ if (!ci_flow_is_valid(flow, engine_conf))
+ continue;
+
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf,
+ flow->engine_idx);
+
+ install_ret = ci_flow_install(engine_ref, flow, &error);
+
+ /* if install failed, log but ignore failure */
+ if (install_ret != 0) {
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to install flow: %s",
+ engine_ref.engine->name, error.message);
+ }
+ }
+
+ rte_rwlock_write_unlock(&engine_conf->config_lock);
+
+ return 0;
+}
+
+/* remove all flows without uninstalling */
+static inline int
+ci_flow_cleanup(struct ci_flow_engine_conf *engine_conf)
+{
+ struct rte_flow_error error = {0};
+ struct ci_flow *flow, *tmp;
+
+ CI_DRV_LOG(DEBUG, "cleaning up all flows");
+
+ /* lock the config for writing */
+ rte_rwlock_write_lock(&engine_conf->config_lock);
+
+ RTE_TAILQ_FOREACH_SAFE(flow, &engine_conf->flows, node, tmp) {
+ struct ci_flow_engine_ref engine_ref;
+ int unregister_ret;
+
+ /* this shouldn't happen */
+ if (!ci_flow_is_valid(flow, engine_conf))
+ continue;
+
+ engine_ref = ci_flow_engine_ref_from_idx(engine_conf,
+ flow->engine_idx);
+
+ /* untrack only; the hardware is not touched */
+ unregister_ret = ci_flow_unregister(engine_ref, flow, &error);
+
+ /* if untrack failed, log but ignore failure */
+ if (unregister_ret != 0) {
+ CI_DRV_LOG(DEBUG, "engine '%s': failed to untrack flow: %s",
+ engine_ref.engine->name, error.message);
+ }
+
+ TAILQ_REMOVE(&engine_conf->flows, flow, node);
+ ci_flow_free(engine_ref, flow);
+ }
+
+ rte_rwlock_write_unlock(&engine_conf->config_lock);
+
+ return 0;
+}
+
+#endif /* _COMMON_INTEL_FLOW_ENGINE_H_ */
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 03/19] net/intel/common: add utility functions
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 01/19] ethdev: add flow graph API Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 02/19] net/intel/common: add flow engines infrastructure Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 04/19] net/ixgbe: add support for common flow parsing Anatoly Burakov
` (16 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
A lot of parsers will rely on doing the same things over and over, so
create a header with utility functions to aid in writing parsers.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/common/flow_util.h | 183 +++++++++++++++++++++++++++
1 file changed, 183 insertions(+)
create mode 100644 drivers/net/intel/common/flow_util.h
diff --git a/drivers/net/intel/common/flow_util.h b/drivers/net/intel/common/flow_util.h
new file mode 100644
index 0000000000..57f12c4637
--- /dev/null
+++ b/drivers/net/intel/common/flow_util.h
@@ -0,0 +1,183 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2025 Intel Corporation
+ */
+
+#ifndef _COMMON_INTEL_FLOW_UTIL_H_
+#define _COMMON_INTEL_FLOW_UTIL_H_
+
+#include <stdint.h>
+#include <stdbool.h>
+#include <string.h>
+
+/*
+ * Utility functions primarily intended for flow parsers.
+ */
+
+/**
+ * Check if memory region is filled with a specific byte value.
+ *
+ * @param ptr
+ * Pointer to memory region.
+ * @param len
+ * Length in bytes.
+ * @param val
+ * Byte value to check (e.g. 0x00 or 0xFF).
+ * @return
+ * true if all bytes equal val, false otherwise.
+ */
+static inline bool
+ci_is_all_byte(const void *ptr, size_t len, uint8_t val)
+{
+ const uint8_t *bytes = (const uint8_t *)ptr;
+ const uint32_t pattern32 = 0x01010101U * val;
+ size_t i = 0;
+
+ /* Process 4-byte chunks using memcpy */
+ for (; i + 4 <= len; i += 4) {
+ uint32_t chunk;
+ memcpy(&chunk, bytes + i, 4);
+ if (chunk != pattern32)
+ return false;
+ }
+
+ /* Process remaining bytes */
+ for (; i < len; i++) {
+ if (bytes[i] != val)
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * Check if bytes are all 0x00 OR all 0xFF.
+ *
+ * @param ptr
+ * Pointer to memory region.
+ * @param len
+ * Length in bytes.
+ * @return
+ * true if all bytes are 0x00 OR all bytes are 0xFF, false otherwise.
+ */
+static inline bool
+ci_is_all_zero_or_masked(const void *ptr, size_t len)
+{
+ const uint8_t *bytes = (const uint8_t *)ptr;
+ uint8_t first_val;
+
+ /* zero length cannot be valid */
+ if (len == 0)
+ return false;
+
+ first_val = bytes[0];
+
+ if (first_val != 0x00 && first_val != 0xFF)
+ return false;
+
+ return ci_is_all_byte(ptr, len, first_val);
+}
+
+/**
+ * Check if a value has no bits outside the mask, and within the mask is
+ * either all-zero or all-one.
+ *
+ * This is intended for bitfields e.g. VLAN_TCI. For byte-aligned fields,
+ * use CI_FIELD_IS_ZERO_OR_MASKED below.
+ *
+ * @param value
+ * Data value to check.
+ * @param mask
+ * Mask to compare against.
+ * @return
+ * true if (value & ~mask) == 0 AND (value & mask) is 0 or mask,
+ * false otherwise.
+ */
+static inline bool
+ci_is_zero_or_masked(uint64_t value, uint64_t mask)
+{
+ uint64_t masked = value & mask;
+ uint64_t unmasked = value & ~mask;
+
+ return unmasked == 0 && (masked == 0 || masked == mask);
+}
+
+/**
+ * Check if a struct field is fully masked or unmasked.
+ *
+ * @param field_ptr
+ * Pointer to the mask field (e.g. ð_mask->hdr.src_addr).
+ */
+#define CI_FIELD_IS_ZERO_OR_MASKED(field_ptr) \
+ ci_is_all_zero_or_masked((field_ptr), sizeof(*(field_ptr)))
+
+/**
+ * Check if a struct field is all 0x00.
+ *
+ * @param field_ptr
+ * Pointer to the mask field (e.g. ð_mask->hdr.src_addr).
+ */
+#define CI_FIELD_IS_ZERO(field_ptr) \
+ ci_is_all_byte((field_ptr), sizeof(*(field_ptr)), 0x00)
+
+/**
+ * Check if a struct field is all 0xFF.
+ *
+ * @param field_ptr
+ * Pointer to the mask field (e.g. ð_mask->hdr.src_addr).
+ */
+#define CI_FIELD_IS_MASKED(field_ptr) \
+ ci_is_all_byte((field_ptr), sizeof(*(field_ptr)), 0xFF)
+
+/**
+ * Convert 24-bit big-endian value to host byte order.
+ *
+ * Used to extract 24-bit big-endian values (e.g. VXLAN VNI).
+ *
+ * @param val
+ * Pointer to 3-byte big-endian value.
+ * @return
+ * Value in host byte order.
+ */
+static inline uint32_t
+ci_be24_to_cpu(const uint8_t val[3])
+{
+ return (val[0] << 16) | (val[1] << 8) | val[2];
+}
+
+/**
+ * Check if a character is a valid hexadecimal digit.
+ *
+ * @param c
+ * Character to check.
+ * @return
+ * true if c is in [0-9a-fA-F], false otherwise.
+ */
+static inline bool
+ci_is_hex_char(unsigned char c)
+{
+ return ((c >= '0' && c <= '9') ||
+ (c >= 'a' && c <= 'f') ||
+ (c >= 'A' && c <= 'F'));
+}
+
+/**
+ * Convert hex character to 4-bit value.
+ *
+ * @param c
+ * Hex character ('0'-'9', 'a'-'f', 'A'-'F').
+ * @return
+ * Value 0-15, or 0 if invalid.
+ */
+static inline unsigned char
+ci_hex_char_to_nibble(unsigned char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+ return 0;
+}
+
+#endif /* _INTEL_COMMON_FLOW_UTIL_H_ */
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 04/19] net/ixgbe: add support for common flow parsing
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (2 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 03/19] net/intel/common: add utility functions Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 05/19] net/ixgbe: reimplement ethertype parser Anatoly Burakov
` (15 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Implement support for common flow parsing infrastructure in preparation for
migration of flow engines. The following features are enabled:
- Conf init on dev_init
- Conf uninit on dev_close
- Flow cleanup on dev_close
- Flow replay on dev_start
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 22 ++++++++++-
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 5 +++
drivers/net/intel/ixgbe/ixgbe_flow.c | 54 +++++++++++++++++++++++++-
drivers/net/intel/ixgbe/ixgbe_flow.h | 12 ++++++
4 files changed, 90 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow.h
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 4b7f51ce13..20807534cd 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -46,6 +46,7 @@
#include "base/ixgbe_phy.h"
#include "ixgbe_osdep.h"
#include "ixgbe_regs.h"
+#include "ixgbe_flow.h"
/*
* High threshold controlling when to start sending XOFF frames. Must be at
@@ -1334,6 +1335,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
if (ret)
goto err_l2_tn_filter_init;
+ /* initialize flow engine configuration */
+ ret = ci_flow_engine_conf_init(&ad->flow_engine_conf,
+ &ixgbe_flow_engine_list, eth_dev->data);
+ if (ret)
+ goto err_flow_engine_conf_init;
+
/* initialize flow filter lists */
ixgbe_filterlist_init(eth_dev);
@@ -1345,6 +1352,8 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
return 0;
+err_flow_engine_conf_init:
+ ixgbe_l2_tn_filter_uninit(eth_dev);
err_l2_tn_filter_init:
ixgbe_fdir_filter_uninit(eth_dev);
err_fdir_filter_init:
@@ -2910,6 +2919,9 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
if (macsec_setting->offload_en)
ixgbe_dev_macsec_register_enable(dev, macsec_setting);
+ /* re-install kept flows to hardware */
+ ci_flow_replay(&adapter->flow_engine_conf);
+
return 0;
error:
@@ -3089,8 +3101,8 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
static int
ixgbe_dev_close(struct rte_eth_dev *dev)
{
- struct ixgbe_hw *hw =
- IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct ixgbe_adapter *ad = dev->data->dev_private;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(ad);
struct rte_pci_device *pci_dev = RTE_CLASS_TO_BUS_DEVICE(dev, *pci_dev);
struct rte_intr_handle *intr_handle = pci_dev->intr_handle;
int retries = 0;
@@ -3157,6 +3169,12 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
rte_free(dev->security_ctx);
dev->security_ctx = NULL;
+ /* drop all flows */
+ ci_flow_cleanup(&ad->flow_engine_conf);
+
+ /* reset flow engines */
+ ci_flow_engine_conf_reset(&ad->flow_engine_conf);
+
return ret;
}
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 5d3243cb4d..cde0ee8fda 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -22,6 +22,8 @@
#include <bus_pci_driver.h>
#include <rte_tm_driver.h>
+#include "../common/flow_engine.h"
+
/* need update link, bit flag */
#define IXGBE_FLAG_NEED_LINK_UPDATE (uint32_t)(1 << 0)
#define IXGBE_FLAG_MAILBOX (uint32_t)(1 << 1)
@@ -346,6 +348,7 @@ struct ixgbe_l2_tn_info {
};
struct rte_flow {
+ struct ci_flow flow;
enum rte_filter_type filter_type;
/* security flows are not rte_filter_type */
bool is_security;
@@ -492,6 +495,8 @@ struct ixgbe_adapter {
struct rte_timecounter tx_tstamp_tc;
struct ixgbe_tm_conf tm_conf;
+ struct ci_flow_engine_conf flow_engine_conf;
+
/* For RSS reta table update */
uint8_t rss_reta_updated;
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index c702119eb0..9babe8eea6 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -47,7 +47,8 @@
#include "rte_pmd_ixgbe.h"
#include "../common/flow_check.h"
-
+#include "../common/flow_engine.h"
+#include "ixgbe_flow.h"
#define IXGBE_MIN_N_TUPLE_PRIO 1
#define IXGBE_MAX_N_TUPLE_PRIO 7
@@ -93,6 +94,8 @@ struct ixgbe_flow_mem {
struct rte_flow *flow;
};
+const struct ci_flow_engine_list ixgbe_flow_engine_list = {0};
+
/**
* Endless loop will never happen with below assumption
* 1. there is at least one no-void item(END)
@@ -2828,6 +2831,13 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct ixgbe_rss_conf_ele *rss_filter_ptr;
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr;
+ /* try the new flow engine first */
+ flow = ci_flow_create(&adapter->flow_engine_conf, attr, pattern, actions, error);
+ if (flow != NULL)
+ return flow;
+
+ /* fall back to legacy flow engines */
+
flow = rte_zmalloc("ixgbe_rte_flow", sizeof(struct rte_flow), 0);
if (!flow) {
PMD_DRV_LOG(ERR, "failed to allocate memory");
@@ -3011,6 +3021,7 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
const struct rte_flow_action actions[],
struct rte_flow_error *error)
{
+ struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
struct rte_eth_ethertype_filter ethertype_filter;
struct rte_eth_syn_filter syn_filter;
@@ -3019,6 +3030,13 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
struct ixgbe_rte_flow_rss_conf rss_conf;
int ret;
+ /* try the new flow engine first */
+ ret = ci_flow_validate(&ad->flow_engine_conf, attr, pattern, actions, error);
+ if (ret == 0)
+ return ret;
+
+ /* fall back to legacy engines */
+
/**
* Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY
*/
@@ -3090,6 +3108,13 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
struct ixgbe_rss_conf_ele *rss_filter_ptr;
+ /* try the new flow engine first */
+ ret = ci_flow_destroy(&adapter->flow_engine_conf, flow, error);
+ if (ret == 0)
+ return 0;
+
+ /* fall back to legacy engines */
+
/* Validate ownership before touching HW/SW state. */
TAILQ_FOREACH(flow_mem_base, &adapter->flow_list, entries) {
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr =
@@ -3193,8 +3218,16 @@ static int
ixgbe_flow_flush(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
+ struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
int ret = 0;
+ /* flush all flows from the new flow engine */
+ ret = ci_flow_flush(&ad->flow_engine_conf, error);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Failed to flush flow");
+ return ret;
+ }
+
ixgbe_clear_all_ntuple_filter(dev);
ixgbe_clear_all_ethertype_filter(dev);
ixgbe_clear_syn_filter(dev);
@@ -3314,6 +3347,25 @@ ixgbe_flow_dev_dump(struct rte_eth_dev *dev,
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ixgbe_filter_ele_base *flow_mem_base;
bool found = false;
+ int ret;
+
+ /* try the new flow engine first */
+ ret = ci_flow_dump(&ad->flow_engine_conf, flow, file, error);
+
+ /*
+ * There are multiple possible situations here:
+ *
+ * - User requested to dump all flows
+ * - User requested to dump a specific flow
+ *
+ * For the first case, we keep going because legacy engines might still
+ * have flows we want to dump.
+ *
+ * For the second case, we only stop if the flow we were asked to dump
+ * was found in the new engines, otherwise we keep looking.
+ */
+ if (flow != NULL && ret == 0)
+ return 0;
TAILQ_FOREACH(flow_mem_base, &ad->flow_list, entries) {
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr =
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
new file mode 100644
index 0000000000..5e68c9886c
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#ifndef _IXGBE_FLOW_H_
+#define _IXGBE_FLOW_H_
+
+#include "../common/flow_engine.h"
+
+extern const struct ci_flow_engine_list ixgbe_flow_engine_list;
+
+#endif /* _IXGBE_FLOW_H_ */
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 05/19] net/ixgbe: reimplement ethertype parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (3 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 04/19] net/ixgbe: add support for common flow parsing Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 06/19] net/ixgbe: reimplement syn parser Anatoly Burakov
` (14 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for ethertype.
The old ethertype parser was accepting certain things that were later
rejected by the actual ethertype installation code, in particular DROP
action as well as dst MAC address filtering. This was removed from the
graph parser.
The ethertype filter tracking table is used by the rte_flow ethertype
engine, but it is also in use by other features, so the filter tracking
is refactored to be properly shared between the engine and other features
that write into the same table.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 186 +++++------
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 89 ++----
drivers/net/intel/ixgbe/ixgbe_flow.c | 219 +------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 8 +
.../net/intel/ixgbe/ixgbe_flow_ethertype.c | 295 ++++++++++++++++++
drivers/net/intel/ixgbe/ixgbe_pf.c | 49 +--
drivers/net/intel/ixgbe/meson.build | 1 +
7 files changed, 429 insertions(+), 418 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_ethertype.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 20807534cd..8cd57c1ee7 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -2961,6 +2961,10 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return -E_RTE_SECONDARY;
+ /* disable timestamping; the application must re-enable it after restart */
+ if (adapter->filter.timesync_installed)
+ ixgbe_timesync_disable(dev);
+
ixgbe_dev_wait_setup_link_complete(dev, 0);
/* disable interrupts */
@@ -6852,76 +6856,51 @@ ixgbe_add_del_ntuple_filter(struct ixgbe_adapter *adapter,
return 0;
}
-int
-ixgbe_add_del_ethertype_filter(struct ixgbe_adapter *adapter,
- struct rte_eth_ethertype_filter *filter,
- bool add)
+void
+ixgbe_ethertype_filter_program(struct ixgbe_hw *hw, uint8_t idx,
+ uint32_t etqf, uint32_t etqs)
{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
- uint32_t etqf = 0;
- uint32_t etqs = 0;
- int ret;
- struct ixgbe_ethertype_filter ethertype_filter;
-
- if (filter->queue >= IXGBE_MAX_RX_QUEUE_NUM)
- return -EINVAL;
-
- if (filter->ether_type == RTE_ETHER_TYPE_IPV4 ||
- filter->ether_type == RTE_ETHER_TYPE_IPV6) {
- PMD_DRV_LOG(ERR, "unsupported ether_type(0x%04x) in"
- " ethertype filter.", filter->ether_type);
- return -EINVAL;
- }
-
- if (filter->flags & RTE_ETHTYPE_FLAGS_MAC) {
- PMD_DRV_LOG(ERR, "mac compare is unsupported.");
- return -EINVAL;
- }
- if (filter->flags & RTE_ETHTYPE_FLAGS_DROP) {
- PMD_DRV_LOG(ERR, "drop option is unsupported.");
- return -EINVAL;
- }
-
- ret = ixgbe_ethertype_filter_lookup(filter_info, filter->ether_type);
- if (ret >= 0 && add) {
- PMD_DRV_LOG(ERR, "ethertype (0x%04x) filter exists.",
- filter->ether_type);
- return -EEXIST;
- }
- if (ret < 0 && !add) {
- PMD_DRV_LOG(ERR, "ethertype (0x%04x) filter doesn't exist.",
- filter->ether_type);
- return -ENOENT;
- }
-
- if (add) {
- etqf = IXGBE_ETQF_FILTER_EN;
- etqf |= (uint32_t)filter->ether_type;
- etqs |= (uint32_t)((filter->queue <<
- IXGBE_ETQS_RX_QUEUE_SHIFT) &
- IXGBE_ETQS_RX_QUEUE);
- etqs |= IXGBE_ETQS_QUEUE_EN;
-
- ethertype_filter.ethertype = filter->ether_type;
- ethertype_filter.etqf = etqf;
- ethertype_filter.etqs = etqs;
- ethertype_filter.conf = FALSE;
- ret = ixgbe_ethertype_filter_insert(filter_info,
- ðertype_filter);
- if (ret < 0) {
- PMD_DRV_LOG(ERR, "ethertype filters are full.");
- return -ENOSPC;
- }
- } else {
- ret = ixgbe_ethertype_filter_remove(filter_info, (uint8_t)ret);
- if (ret < 0)
- return -ENOSYS;
- }
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(ret), etqf);
- IXGBE_WRITE_REG(hw, IXGBE_ETQS(ret), etqs);
+ IXGBE_WRITE_REG(hw, IXGBE_ETQF(idx), etqf);
+ IXGBE_WRITE_REG(hw, IXGBE_ETQS(idx), etqs);
IXGBE_WRITE_FLUSH(hw);
+}
+
+int
+ixgbe_ethertype_table_add(struct ixgbe_ethertype_table *table,
+ uint16_t ethertype, uint32_t etqf, uint32_t etqs)
+{
+ int free_idx = -1;
+ int i;
+
+ for (i = 0; i < IXGBE_MAX_ETQF_FILTERS; i++) {
+ if (table->mask & (1u << i)) {
+ if (table->entries[i].ethertype == ethertype)
+ return -EEXIST;
+ } else if (free_idx < 0) {
+ free_idx = i;
+ }
+ }
+ if (free_idx < 0)
+ return -ENOSPC;
+
+ table->mask |= 1u << free_idx;
+ table->entries[free_idx].ethertype = ethertype;
+ table->entries[free_idx].etqf = etqf;
+ table->entries[free_idx].etqs = etqs;
+
+ return free_idx;
+}
+
+int
+ixgbe_ethertype_table_del(struct ixgbe_ethertype_table *table, uint8_t idx)
+{
+ if (idx >= IXGBE_MAX_ETQF_FILTERS || !(table->mask & (1u << idx)))
+ return -ENOENT;
+
+ table->mask &= ~(1u << idx);
+ table->entries[idx].ethertype = 0;
+ table->entries[idx].etqf = 0;
+ table->entries[idx].etqs = 0;
return 0;
}
@@ -7159,8 +7138,11 @@ static int
ixgbe_timesync_enable(struct rte_eth_dev *dev)
{
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct ixgbe_filter_info *filter_info =
+ IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
uint32_t tsync_ctl;
uint32_t tsauxc;
+ uint32_t etqf;
struct timespec ts;
memset(&ts, 0, sizeof(struct timespec));
@@ -7182,10 +7164,19 @@ ixgbe_timesync_enable(struct rte_eth_dev *dev)
ixgbe_start_timecounters(dev);
/* Enable L2 filtering of IEEE1588/802.1AS Ethernet frame types. */
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(IXGBE_ETQF_FILTER_1588),
- (RTE_ETHER_TYPE_1588 |
- IXGBE_ETQF_FILTER_EN |
- IXGBE_ETQF_1588));
+ etqf = RTE_ETHER_TYPE_1588 | IXGBE_ETQF_FILTER_EN | IXGBE_ETQF_1588;
+ if (!filter_info->timesync_installed) {
+ int idx = ixgbe_ethertype_table_add(&filter_info->ethertype_table,
+ RTE_ETHER_TYPE_1588, etqf, 0);
+
+ if (idx < 0) {
+ PMD_DRV_LOG(ERR, "no free ETQF slot for 1588 timestamping");
+ return idx;
+ }
+ filter_info->timesync_idx = idx;
+ filter_info->timesync_installed = true;
+ }
+ ixgbe_ethertype_filter_program(hw, filter_info->timesync_idx, etqf, 0);
/* Enable timestamping of received PTP packets. */
tsync_ctl = IXGBE_READ_REG(hw, IXGBE_TSYNCRXCTL);
@@ -7209,6 +7200,8 @@ static int
ixgbe_timesync_disable(struct rte_eth_dev *dev)
{
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct ixgbe_filter_info *filter_info =
+ IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
uint32_t tsync_ctl;
/* Disable timestamping of transmitted PTP packets. */
@@ -7222,7 +7215,12 @@ ixgbe_timesync_disable(struct rte_eth_dev *dev)
IXGBE_WRITE_REG(hw, IXGBE_TSYNCRXCTL, tsync_ctl);
/* Disable L2 filtering of IEEE1588/802.1AS Ethernet frame types. */
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(IXGBE_ETQF_FILTER_1588), 0);
+ if (filter_info->timesync_installed) {
+ ixgbe_ethertype_filter_program(hw, filter_info->timesync_idx, 0, 0);
+ ixgbe_ethertype_table_del(&filter_info->ethertype_table,
+ filter_info->timesync_idx);
+ filter_info->timesync_installed = false;
+ }
/* Stop incrementing the System Time registers. */
IXGBE_WRITE_REG(hw, IXGBE_TIMINCA, 0);
@@ -8364,26 +8362,6 @@ ixgbe_ntuple_filter_restore(struct rte_eth_dev *dev)
}
}
-/* restore ethernet type filter */
-static inline void
-ixgbe_ethertype_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
- int i;
-
- for (i = 0; i < IXGBE_MAX_ETQF_FILTERS; i++) {
- if (filter_info->ethertype_mask & (1 << i)) {
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(i),
- filter_info->ethertype_filters[i].etqf);
- IXGBE_WRITE_REG(hw, IXGBE_ETQS(i),
- filter_info->ethertype_filters[i].etqs);
- IXGBE_WRITE_FLUSH(hw);
- }
- }
-}
-
/* restore SYN filter */
static inline void
ixgbe_syn_filter_restore(struct rte_eth_dev *dev)
@@ -8439,7 +8417,6 @@ static int
ixgbe_filter_restore(struct rte_eth_dev *dev)
{
ixgbe_ntuple_filter_restore(dev);
- ixgbe_ethertype_filter_restore(dev);
ixgbe_syn_filter_restore(dev);
ixgbe_fdir_filter_restore(dev);
ixgbe_l2_tn_filter_restore(dev);
@@ -8478,27 +8455,6 @@ ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev)
ixgbe_remove_5tuple_filter(adapter, p_5tuple);
}
-/* remove all the ether type filters */
-void
-ixgbe_clear_all_ethertype_filter(struct rte_eth_dev *dev)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
- int i;
-
- for (i = 0; i < IXGBE_MAX_ETQF_FILTERS; i++) {
- if (filter_info->ethertype_mask & (1 << i) &&
- !filter_info->ethertype_filters[i].conf) {
- (void)ixgbe_ethertype_filter_remove(filter_info,
- (uint8_t)i);
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(i), 0);
- IXGBE_WRITE_REG(hw, IXGBE_ETQS(i), 0);
- IXGBE_WRITE_FLUSH(hw);
- }
- }
-}
-
/* remove the SYN filter */
void
ixgbe_clear_syn_filter(struct rte_eth_dev *dev)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index cde0ee8fda..3f15b5a0c9 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -298,24 +298,20 @@ struct ixgbe_5tuple_filter {
(RTE_ALIGN(IXGBE_MAX_FTQF_FILTERS, (sizeof(uint32_t) * NBBY)) / \
(sizeof(uint32_t) * NBBY))
-struct ixgbe_ethertype_filter {
- uint16_t ethertype;
- uint32_t etqf;
- uint32_t etqs;
- /**
- * If this filter is added by configuration,
- * it should not be removed.
- */
- bool conf;
+/* Shared EtherType (ETQF) filter table. */
+struct ixgbe_ethertype_table {
+ uint32_t mask; /* bitmask of used ETQF slots */
+ struct ixgbe_ethertype_entry {
+ uint16_t ethertype; /* ethertype, for dedup */
+ uint32_t etqf; /* ETQF register value */
+ uint32_t etqs; /* ETQS register value */
+ } entries[IXGBE_MAX_ETQF_FILTERS];
};
/*
* Structure to store filters' info.
*/
struct ixgbe_filter_info {
- uint8_t ethertype_mask; /* Bit mask for every used ethertype filter */
- /* store used ethertype filters*/
- struct ixgbe_ethertype_filter ethertype_filters[IXGBE_MAX_ETQF_FILTERS];
/* Bit mask for every used 5tuple filter */
uint32_t fivetuple_mask[IXGBE_5TUPLE_ARRAY_SIZE];
struct ixgbe_5tuple_filter_list fivetuple_list;
@@ -323,6 +319,14 @@ struct ixgbe_filter_info {
uint32_t syn_info;
/* store the rss filter info */
struct ixgbe_rte_flow_rss_conf rss_info;
+ /* shared EtherType (ETQF) slot table */
+ struct ixgbe_ethertype_table ethertype_table;
+ /* 1588 timestamping ETQF slot (valid when timesync_installed) */
+ bool timesync_installed;
+ uint8_t timesync_idx;
+ /* Tx anti-spoof ETQF slot (valid when antispoof_installed) */
+ bool antispoof_installed;
+ uint8_t antispoof_idx;
};
struct ixgbe_l2_tn_key {
@@ -680,13 +684,16 @@ bool ixgbe_rss_update_sp(enum ixgbe_mac_type mac_type);
int ixgbe_add_del_ntuple_filter(struct ixgbe_adapter *adapter,
struct rte_eth_ntuple_filter *filter,
bool add);
-int ixgbe_add_del_ethertype_filter(struct ixgbe_adapter *adapter,
- struct rte_eth_ethertype_filter *filter,
- bool add);
int ixgbe_syn_filter_set(struct ixgbe_adapter *adapter,
struct rte_eth_syn_filter *filter,
bool add);
+void ixgbe_ethertype_filter_program(struct ixgbe_hw *hw, uint8_t idx,
+ uint32_t etqf, uint32_t etqs);
+int ixgbe_ethertype_table_add(struct ixgbe_ethertype_table *table,
+ uint16_t ethertype, uint32_t etqf, uint32_t etqs);
+int ixgbe_ethertype_table_del(struct ixgbe_ethertype_table *table, uint8_t idx);
+
/**
* l2 tunnel configuration.
*/
@@ -757,7 +764,6 @@ int ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev);
extern const struct rte_flow_ops ixgbe_flow_ops;
-void ixgbe_clear_all_ethertype_filter(struct rte_eth_dev *dev);
void ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev);
void ixgbe_clear_syn_filter(struct rte_eth_dev *dev);
int ixgbe_clear_all_l2_tn_filter(struct rte_eth_dev *dev);
@@ -792,55 +798,4 @@ void ixgbe_dev_macsec_setting_save(struct rte_eth_dev *dev,
void ixgbe_dev_macsec_setting_reset(struct rte_eth_dev *dev);
-static inline int
-ixgbe_ethertype_filter_lookup(struct ixgbe_filter_info *filter_info,
- uint16_t ethertype)
-{
- int i;
-
- for (i = 0; i < IXGBE_MAX_ETQF_FILTERS; i++) {
- if (filter_info->ethertype_filters[i].ethertype == ethertype &&
- (filter_info->ethertype_mask & (1 << i)))
- return i;
- }
- return -1;
-}
-
-static inline int
-ixgbe_ethertype_filter_insert(struct ixgbe_filter_info *filter_info,
- struct ixgbe_ethertype_filter *ethertype_filter)
-{
- int i;
-
- for (i = 0; i < IXGBE_MAX_ETQF_FILTERS; i++) {
- if (!(filter_info->ethertype_mask & (1 << i))) {
- filter_info->ethertype_mask |= 1 << i;
- filter_info->ethertype_filters[i].ethertype =
- ethertype_filter->ethertype;
- filter_info->ethertype_filters[i].etqf =
- ethertype_filter->etqf;
- filter_info->ethertype_filters[i].etqs =
- ethertype_filter->etqs;
- filter_info->ethertype_filters[i].conf =
- ethertype_filter->conf;
- return i;
- }
- }
- return -1;
-}
-
-static inline int
-ixgbe_ethertype_filter_remove(struct ixgbe_filter_info *filter_info,
- uint8_t idx)
-{
- if (idx >= IXGBE_MAX_ETQF_FILTERS)
- return -1;
- filter_info->ethertype_mask &= ~(1 << idx);
- filter_info->ethertype_filters[idx].ethertype = 0;
- filter_info->ethertype_filters[idx].etqf = 0;
- filter_info->ethertype_filters[idx].etqs = 0;
- filter_info->ethertype_filters[idx].etqs = FALSE;
- return idx;
-}
-
#endif /* _IXGBE_ETHDEV_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 9babe8eea6..bde8759bf7 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -63,11 +63,6 @@ struct ixgbe_ntuple_filter_ele {
struct ixgbe_filter_ele_base base;
struct rte_eth_ntuple_filter filter_info;
};
-/* ethertype filter list structure */
-struct ixgbe_ethertype_filter_ele {
- struct ixgbe_filter_ele_base base;
- struct rte_eth_ethertype_filter filter_info;
-};
/* syn filter list structure */
struct ixgbe_eth_syn_filter_ele {
struct ixgbe_filter_ele_base base;
@@ -94,7 +89,11 @@ struct ixgbe_flow_mem {
struct rte_flow *flow;
};
-const struct ci_flow_engine_list ixgbe_flow_engine_list = {0};
+const struct ci_flow_engine_list ixgbe_flow_engine_list = {
+ {
+ &ixgbe_ethertype_flow_engine,
+ }
+};
/**
* Endless loop will never happen with below assumption
@@ -118,7 +117,7 @@ const struct rte_flow_item *next_no_void_pattern(
/*
* All ixgbe engines mostly check the same stuff, so use a common check.
*/
-static int
+int
ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
const struct ci_flow_actions_check_param *param,
struct rte_flow_error *error)
@@ -667,169 +666,6 @@ ixgbe_parse_ntuple_filter(struct rte_eth_dev *dev,
return 0;
}
-/**
- * Parse the rule to see if it is a ethertype rule.
- * And get the ethertype filter info BTW.
- * pattern:
- * The first not void item can be ETH.
- * The next not void item must be END.
- * action:
- * The first not void action should be QUEUE.
- * The next not void action should be END.
- * pattern example:
- * ITEM Spec Mask
- * ETH type 0x0807 0xFFFF
- * END
- * other members in mask and spec should set to 0x00.
- * item->last should be NULL.
- */
-static int
-cons_parse_ethertype_filter(const struct rte_flow_item *pattern,
- const struct rte_flow_action *action,
- struct rte_eth_ethertype_filter *filter,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
-
- item = next_no_void_pattern(pattern, NULL);
- /* The first non-void item should be MAC. */
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ethertype filter");
- return -rte_errno;
- }
-
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Get the MAC info. */
- if (!item->spec || !item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ethertype filter");
- return -rte_errno;
- }
-
- eth_spec = item->spec;
- eth_mask = item->mask;
-
- /* Mask bits of source MAC address must be full of 0.
- * Mask bits of destination MAC address must be full
- * of 1 or full of 0.
- */
- if (!rte_is_zero_ether_addr(ð_mask->hdr.src_addr) ||
- (!rte_is_zero_ether_addr(ð_mask->hdr.dst_addr) &&
- !rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr))) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid ether address mask");
- return -rte_errno;
- }
-
- if ((eth_mask->hdr.ether_type & UINT16_MAX) != UINT16_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid ethertype mask");
- return -rte_errno;
- }
-
- /* If mask bits of destination MAC address
- * are full of 1, set RTE_ETHTYPE_FLAGS_MAC.
- */
- if (rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr)) {
- filter->mac_addr = eth_spec->hdr.dst_addr;
- filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
- } else {
- filter->flags &= ~RTE_ETHTYPE_FLAGS_MAC;
- }
- filter->ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
-
- /* Check if the next non-void item is END. */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ethertype filter.");
- return -rte_errno;
- }
-
- filter->queue = ((const struct rte_flow_action_queue *)action->conf)->index;
-
- return 0;
-}
-
-static int
-ixgbe_parse_ethertype_filter(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[], const struct rte_flow_action actions[],
- struct rte_eth_ethertype_filter *filter, struct rte_flow_error *error)
-{
- int ret;
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only queue is allowed here */
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_END
- },
- .max_actions = 1,
- .driver_ctx = dev->data,
- .check = ixgbe_flow_actions_check
- };
- const struct rte_flow_action *action;
-
- if (hw->mac.type != ixgbe_mac_82599EB &&
- hw->mac.type != ixgbe_mac_X540 &&
- hw->mac.type != ixgbe_mac_X550 &&
- hw->mac.type != ixgbe_mac_X550EM_x &&
- hw->mac.type != ixgbe_mac_X550EM_a &&
- hw->mac.type != ixgbe_mac_E610)
- return -ENOTSUP;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
-
- action = parsed_actions.actions[0];
-
- ret = cons_parse_ethertype_filter(pattern, action, filter, error);
- if (ret)
- return ret;
-
- if (filter->ether_type == RTE_ETHER_TYPE_IPV4 ||
- filter->ether_type == RTE_ETHER_TYPE_IPV6) {
- memset(filter, 0, sizeof(struct rte_eth_ethertype_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "IPv4/IPv6 not supported by ethertype filter");
- return -rte_errno;
- }
-
- if (filter->flags & RTE_ETHTYPE_FLAGS_MAC) {
- memset(filter, 0, sizeof(struct rte_eth_ethertype_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "mac compare is unsupported");
- return -rte_errno;
- }
-
- return 0;
-}
-
/**
* Parse the rule to see if it is a TCP SYN rule.
* And get the TCP SYN filter info BTW.
@@ -2815,7 +2651,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct ixgbe_adapter *adapter =
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_ethertype_filter ethertype_filter;
struct rte_eth_syn_filter syn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
@@ -2824,7 +2659,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct ixgbe_rte_flow_rss_conf rss_conf;
struct rte_flow *flow = NULL;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_ethertype_filter_ele *ethertype_filter_ptr;
struct ixgbe_eth_syn_filter_ele *syn_filter_ptr;
struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
@@ -2884,28 +2718,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
goto out;
}
- memset(ðertype_filter, 0, sizeof(struct rte_eth_ethertype_filter));
- ret = ixgbe_parse_ethertype_filter(dev, attr, pattern,
- actions, ðertype_filter, error);
- if (!ret) {
- ret = ixgbe_add_del_ethertype_filter(adapter,
- ðertype_filter, TRUE);
- if (!ret) {
- ethertype_filter_ptr = rte_zmalloc(
- "ixgbe_ethertype_filter",
- sizeof(struct ixgbe_ethertype_filter_ele), 0);
- if (!ethertype_filter_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- ethertype_filter_ptr->filter_info = ethertype_filter;
- flow->rule = ethertype_filter_ptr;
- flow->filter_type = RTE_ETH_FILTER_ETHERTYPE;
- return flow;
- }
- goto out;
- }
-
memset(&syn_filter, 0, sizeof(struct rte_eth_syn_filter));
ret = ixgbe_parse_syn_filter(dev, attr, pattern,
actions, &syn_filter, error);
@@ -3023,7 +2835,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_ethertype_filter ethertype_filter;
struct rte_eth_syn_filter syn_filter;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_fdir_rule fdir_rule;
@@ -3050,12 +2861,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
if (!ret)
return 0;
- memset(ðertype_filter, 0, sizeof(struct rte_eth_ethertype_filter));
- ret = ixgbe_parse_ethertype_filter(dev, attr, pattern,
- actions, ðertype_filter, error);
- if (!ret)
- return 0;
-
memset(&syn_filter, 0, sizeof(struct rte_eth_syn_filter));
ret = ixgbe_parse_syn_filter(dev, attr, pattern,
actions, &syn_filter, error);
@@ -3093,12 +2898,10 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
struct rte_flow *pmd_flow = flow;
enum rte_filter_type filter_type = pmd_flow->filter_type;
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_ethertype_filter ethertype_filter;
struct rte_eth_syn_filter syn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_ethertype_filter_ele *ethertype_filter_ptr;
struct ixgbe_eth_syn_filter_ele *syn_filter_ptr;
struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
@@ -3144,15 +2947,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
if (!ret)
rte_free(ntuple_filter_ptr);
break;
- case RTE_ETH_FILTER_ETHERTYPE:
- ethertype_filter_ptr = (struct ixgbe_ethertype_filter_ele *)
- pmd_flow->rule;
- ethertype_filter = ethertype_filter_ptr->filter_info;
- ret = ixgbe_add_del_ethertype_filter(adapter,
- ðertype_filter, FALSE);
- if (!ret)
- rte_free(ethertype_filter_ptr);
- break;
case RTE_ETH_FILTER_SYN:
syn_filter_ptr = (struct ixgbe_eth_syn_filter_ele *)
pmd_flow->rule;
@@ -3229,7 +3023,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
}
ixgbe_clear_all_ntuple_filter(dev);
- ixgbe_clear_all_ethertype_filter(dev);
ixgbe_clear_syn_filter(dev);
ret = ixgbe_clear_all_fdir_filter(dev);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index 5e68c9886c..d7694283a5 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -5,8 +5,16 @@
#ifndef _IXGBE_FLOW_H_
#define _IXGBE_FLOW_H_
+#include "../common/flow_check.h"
#include "../common/flow_engine.h"
+int
+ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error);
+
extern const struct ci_flow_engine_list ixgbe_flow_engine_list;
+extern const struct ci_flow_engine ixgbe_ethertype_flow_engine;
+
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_ethertype.c b/drivers/net/intel/ixgbe/ixgbe_flow_ethertype.c
new file mode 100644
index 0000000000..d02d52d538
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_ethertype.c
@@ -0,0 +1,295 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_ethertype_flow {
+ struct rte_flow flow;
+ uint16_t ether_type;
+ uint16_t queue;
+ uint32_t etqf;
+ uint32_t etqs;
+ uint8_t index; /* assigned HW slot */
+};
+
+struct ixgbe_ethertype_ctx {
+ struct ci_flow_engine_ctx base;
+ struct rte_eth_ethertype_filter filter;
+};
+
+/**
+ * Ethertype filter graph implementation
+ * Pattern: START -> ETH -> END
+ */
+
+enum ixgbe_ethertype_node_id {
+ IXGBE_ETHERTYPE_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_ETHERTYPE_NODE_ETH,
+ IXGBE_ETHERTYPE_NODE_END,
+ IXGBE_ETHERTYPE_NODE_MAX,
+};
+
+static int
+ixgbe_ethertype_node_eth_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_eth *eth_spec;
+ const struct rte_flow_item_eth *eth_mask;
+
+ eth_spec = item->spec;
+ eth_mask = item->mask;
+
+ /* Source MAC mask must be all zeros */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Source MAC filtering not supported");
+ }
+
+ /* Dest MAC mask must be all zeros */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Destination MAC filtering not supported");
+ }
+
+ /* Ethertype mask must be exact match */
+ if (!CI_FIELD_IS_MASKED(ð_mask->hdr.ether_type)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Ethertype must be exactly matched");
+ }
+
+ /* IPv4/IPv6 ethertypes not supported by hardware */
+ uint16_t ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+ if (ether_type == RTE_ETHER_TYPE_IPV4 || ether_type == RTE_ETHER_TYPE_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4/IPv6 not supported by ethertype filter");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_ethertype_node_eth_process(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ethertype_ctx *graph_ctx = ctx;
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+
+ graph_ctx->filter.ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_ethertype_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_ETHERTYPE_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_ETHERTYPE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_ethertype_node_eth_validate,
+ .process = ixgbe_ethertype_node_eth_process,
+ },
+ [IXGBE_ETHERTYPE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_ETHERTYPE_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_ETHERTYPE_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_ETHERTYPE_NODE_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_ETHERTYPE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+ixgbe_flow_ethertype_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only queue is allowed here */
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_flow_actions_check
+ };
+ struct ixgbe_ethertype_ctx *ethertype_ctx = (struct ixgbe_ethertype_ctx *)ctx;
+ const struct rte_flow_action_queue *q_act;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ q_act = (const struct rte_flow_action_queue *)parsed_actions.actions[0]->conf;
+
+ /* set up filter action */
+ ethertype_ctx->filter.queue = q_act->index;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_ethertype_ctx *ethertype_ctx = (const struct ixgbe_ethertype_ctx *)ctx;
+ struct ixgbe_ethertype_flow *ethertype_flow = (struct ixgbe_ethertype_flow *)flow;
+
+ /* copy filter configuration */
+ ethertype_flow->ether_type = ethertype_ctx->filter.ether_type;
+ ethertype_flow->queue = ethertype_ctx->filter.queue;
+
+ /* build the ETQF/ETQS register values from the parsed filter */
+ ethertype_flow->etqf = IXGBE_ETQF_FILTER_EN | (uint32_t)ethertype_flow->ether_type;
+ ethertype_flow->etqs = ((uint32_t)ethertype_flow->queue <<
+ IXGBE_ETQS_RX_QUEUE_SHIFT) & IXGBE_ETQS_RX_QUEUE;
+ ethertype_flow->etqs |= IXGBE_ETQS_QUEUE_EN;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_ethertype_flow *ethertype_flow = (struct ixgbe_ethertype_flow *)flow;
+ struct ixgbe_filter_info *filter_info =
+ IXGBE_DEV_PRIVATE_TO_FILTER_INFO(flow->dev_data->dev_private);
+ int idx;
+
+ idx = ixgbe_ethertype_table_add(&filter_info->ethertype_table,
+ ethertype_flow->ether_type, ethertype_flow->etqf,
+ ethertype_flow->etqs);
+ if (idx == -EEXIST) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ethertype filter already exists");
+ }
+ if (idx == -ENOSPC) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ethertype filters are full");
+ }
+ ethertype_flow->index = idx;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_ethertype_flow *ethertype_flow = (struct ixgbe_ethertype_flow *)flow;
+ struct ixgbe_filter_info *filter_info =
+ IXGBE_DEV_PRIVATE_TO_FILTER_INFO(flow->dev_data->dev_private);
+ int ret;
+
+ ret = ixgbe_ethertype_table_del(&filter_info->ethertype_table,
+ ethertype_flow->index);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ethertype filter slot not found on unregister");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_install(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ethertype_flow *ethertype_flow = (struct ixgbe_ethertype_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_ethertype_filter_program(hw, ethertype_flow->index,
+ ethertype_flow->etqf, ethertype_flow->etqs);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ethertype_flow *ethertype_flow = (struct ixgbe_ethertype_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_ethertype_filter_program(hw, ethertype_flow->index, 0, 0);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ethertype_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ /* Ethertype filtering (ETQF) is only available on these MACs. */
+ if (hw->mac.type == ixgbe_mac_82599EB ||
+ hw->mac.type == ixgbe_mac_X540 ||
+ hw->mac.type == ixgbe_mac_X550 ||
+ hw->mac.type == ixgbe_mac_X550EM_x ||
+ hw->mac.type == ixgbe_mac_X550EM_a ||
+ hw->mac.type == ixgbe_mac_E610)
+ return 0;
+
+ return -ENOTSUP;
+}
+
+static const struct ci_flow_engine_ops ixgbe_ethertype_ops = {
+ .engine_init = ixgbe_flow_ethertype_engine_init,
+ .ctx_init = ixgbe_flow_ethertype_ctx_init,
+ .ctx_to_flow = ixgbe_flow_ethertype_ctx_to_flow,
+ .flow_register = ixgbe_flow_ethertype_register,
+ .flow_unregister = ixgbe_flow_ethertype_unregister,
+ .flow_install = ixgbe_flow_ethertype_install,
+ .flow_uninstall = ixgbe_flow_ethertype_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_ethertype_flow_engine = {
+ .name = "ethertype",
+ .ctx_size = sizeof(struct ixgbe_ethertype_ctx),
+ .flow_size = sizeof(struct ixgbe_ethertype_flow),
+ .graph = &ixgbe_ethertype_graph,
+ .ops = &ixgbe_ethertype_ops,
+};
diff --git a/drivers/net/intel/ixgbe/ixgbe_pf.c b/drivers/net/intel/ixgbe/ixgbe_pf.c
index 939e7d1417..7616169e08 100644
--- a/drivers/net/intel/ixgbe/ixgbe_pf.c
+++ b/drivers/net/intel/ixgbe/ixgbe_pf.c
@@ -133,12 +133,24 @@ int ixgbe_pf_host_init(struct rte_eth_dev *eth_dev)
void ixgbe_pf_host_uninit(struct rte_eth_dev *eth_dev)
{
+ struct ixgbe_filter_info *filter_info =
+ IXGBE_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
+ struct ixgbe_hw *hw =
+ IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
struct ixgbe_vf_info **vfinfo;
uint16_t vf_num;
int ret;
PMD_INIT_FUNC_TRACE();
+ /* release the Tx anti-spoof ETQF slot */
+ if (filter_info->antispoof_installed) {
+ ixgbe_ethertype_filter_program(hw, filter_info->antispoof_idx, 0, 0);
+ ixgbe_ethertype_table_del(&filter_info->ethertype_table,
+ filter_info->antispoof_idx);
+ filter_info->antispoof_installed = false;
+ }
+
RTE_ETH_DEV_SRIOV(eth_dev).active = 0;
RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool = 0;
RTE_ETH_DEV_SRIOV(eth_dev).def_vmdq_idx = 0;
@@ -168,38 +180,29 @@ ixgbe_add_tx_flow_control_drop_filter(struct rte_eth_dev *eth_dev)
struct ixgbe_filter_info *filter_info =
IXGBE_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
uint16_t vf_num;
+ uint32_t etqf, etqs;
int i;
- struct ixgbe_ethertype_filter ethertype_filter;
if (!hw->mac.ops.set_ethertype_anti_spoofing) {
PMD_DRV_LOG(INFO, "ether type anti-spoofing is not supported.");
return;
}
- i = ixgbe_ethertype_filter_lookup(filter_info,
- IXGBE_ETHERTYPE_FLOW_CTRL);
- if (i >= 0) {
- PMD_DRV_LOG(ERR, "A ether type filter entity for flow control already exists!");
- return;
- }
+ etqf = IXGBE_ETQF_FILTER_EN | IXGBE_ETQF_TX_ANTISPOOF |
+ IXGBE_ETHERTYPE_FLOW_CTRL;
+ etqs = 0;
+ if (!filter_info->antispoof_installed) {
+ int idx = ixgbe_ethertype_table_add(&filter_info->ethertype_table,
+ IXGBE_ETHERTYPE_FLOW_CTRL, etqf, etqs);
- ethertype_filter.ethertype = IXGBE_ETHERTYPE_FLOW_CTRL;
- ethertype_filter.etqf = IXGBE_ETQF_FILTER_EN |
- IXGBE_ETQF_TX_ANTISPOOF |
- IXGBE_ETHERTYPE_FLOW_CTRL;
- ethertype_filter.etqs = 0;
- ethertype_filter.conf = TRUE;
- i = ixgbe_ethertype_filter_insert(filter_info,
- ðertype_filter);
- if (i < 0) {
- PMD_DRV_LOG(ERR, "Cannot find an unused ether type filter entity for flow control.");
- return;
+ if (idx < 0) {
+ PMD_DRV_LOG(ERR, "no free ETQF slot for Tx anti-spoof filter");
+ return;
+ }
+ filter_info->antispoof_idx = idx;
+ filter_info->antispoof_installed = true;
}
-
- IXGBE_WRITE_REG(hw, IXGBE_ETQF(i),
- (IXGBE_ETQF_FILTER_EN |
- IXGBE_ETQF_TX_ANTISPOOF |
- IXGBE_ETHERTYPE_FLOW_CTRL));
+ ixgbe_ethertype_filter_program(hw, filter_info->antispoof_idx, etqf, etqs);
vf_num = dev_num_vf(eth_dev);
for (i = 0; i < vf_num; i++)
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 0531d37acd..f2857feab7 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -26,6 +26,7 @@ sources += files(
'ixgbe_ethdev.c',
'ixgbe_fdir.c',
'ixgbe_flow.c',
+ 'ixgbe_flow_ethertype.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 06/19] net/ixgbe: reimplement syn parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (4 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 05/19] net/ixgbe: reimplement ethertype parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 07/19] net/ixgbe: reimplement L2 tunnel parser Anatoly Burakov
` (13 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for SYN.
As a result of this migration, queue index validation has changed:
- queue is now validated at parse time against the configured number of Rx
queues (nb_rx_queues), rather than at install time against the hardware
maximum (IXGBE_MAX_RX_QUEUE_NUM)
- the per-function queue bound check in ixgbe_syn_filter_set() has been
removed as it is no longer needed
The syn filter tracking infrastructure is moved completely inside the new
engine and is removed from the rest of the driver.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 70 +----
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 7 +-
drivers/net/intel/ixgbe/ixgbe_flow.c | 247 +-----------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 1 +
drivers/net/intel/ixgbe/ixgbe_flow_syn.c | 312 +++++++++++++++++++++++
drivers/net/intel/ixgbe/meson.build | 1 +
6 files changed, 319 insertions(+), 319 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_syn.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 8cd57c1ee7..e4576b3844 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -6478,43 +6478,11 @@ ixgbevf_set_default_mac_addr(struct rte_eth_dev *dev,
return 0;
}
-int
-ixgbe_syn_filter_set(struct ixgbe_adapter *adapter,
- struct rte_eth_syn_filter *filter,
- bool add)
+void
+ixgbe_syn_filter_program(struct ixgbe_hw *hw, uint32_t synqf)
{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
- uint32_t syn_info;
- uint32_t synqf;
-
- if (filter->queue >= IXGBE_MAX_RX_QUEUE_NUM)
- return -EINVAL;
-
- syn_info = filter_info->syn_info;
-
- if (add) {
- if (syn_info & IXGBE_SYN_FILTER_ENABLE)
- return -EINVAL;
- synqf = (uint32_t)(((filter->queue << IXGBE_SYN_FILTER_QUEUE_SHIFT) &
- IXGBE_SYN_FILTER_QUEUE) | IXGBE_SYN_FILTER_ENABLE);
-
- if (filter->hig_pri)
- synqf |= IXGBE_SYN_FILTER_SYNQFP;
- else
- synqf &= ~IXGBE_SYN_FILTER_SYNQFP;
- } else {
- synqf = IXGBE_READ_REG(hw, IXGBE_SYNQF);
- if (!(syn_info & IXGBE_SYN_FILTER_ENABLE))
- return -ENOENT;
- synqf &= ~(IXGBE_SYN_FILTER_QUEUE | IXGBE_SYN_FILTER_ENABLE);
- }
-
- filter_info->syn_info = synqf;
IXGBE_WRITE_REG(hw, IXGBE_SYNQF, synqf);
IXGBE_WRITE_FLUSH(hw);
- return 0;
}
@@ -8362,23 +8330,6 @@ ixgbe_ntuple_filter_restore(struct rte_eth_dev *dev)
}
}
-/* restore SYN filter */
-static inline void
-ixgbe_syn_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
- uint32_t synqf;
-
- synqf = filter_info->syn_info;
-
- if (synqf & IXGBE_SYN_FILTER_ENABLE) {
- IXGBE_WRITE_REG(hw, IXGBE_SYNQF, synqf);
- IXGBE_WRITE_FLUSH(hw);
- }
-}
-
/* restore L2 tunnel filter */
static inline void
ixgbe_l2_tn_filter_restore(struct rte_eth_dev *dev)
@@ -8417,7 +8368,6 @@ static int
ixgbe_filter_restore(struct rte_eth_dev *dev)
{
ixgbe_ntuple_filter_restore(dev);
- ixgbe_syn_filter_restore(dev);
ixgbe_fdir_filter_restore(dev);
ixgbe_l2_tn_filter_restore(dev);
ixgbe_rss_filter_restore(dev);
@@ -8455,22 +8405,6 @@ ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev)
ixgbe_remove_5tuple_filter(adapter, p_5tuple);
}
-/* remove the SYN filter */
-void
-ixgbe_clear_syn_filter(struct rte_eth_dev *dev)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
-
- if (filter_info->syn_info & IXGBE_SYN_FILTER_ENABLE) {
- filter_info->syn_info = 0;
-
- IXGBE_WRITE_REG(hw, IXGBE_SYNQF, 0);
- IXGBE_WRITE_FLUSH(hw);
- }
-}
-
/* remove all the L2 tunnel filters */
int
ixgbe_clear_all_l2_tn_filter(struct rte_eth_dev *dev)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 3f15b5a0c9..59d58f6160 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -315,8 +315,6 @@ struct ixgbe_filter_info {
/* Bit mask for every used 5tuple filter */
uint32_t fivetuple_mask[IXGBE_5TUPLE_ARRAY_SIZE];
struct ixgbe_5tuple_filter_list fivetuple_list;
- /* store the SYN filter info */
- uint32_t syn_info;
/* store the rss filter info */
struct ixgbe_rte_flow_rss_conf rss_info;
/* shared EtherType (ETQF) slot table */
@@ -684,15 +682,13 @@ bool ixgbe_rss_update_sp(enum ixgbe_mac_type mac_type);
int ixgbe_add_del_ntuple_filter(struct ixgbe_adapter *adapter,
struct rte_eth_ntuple_filter *filter,
bool add);
-int ixgbe_syn_filter_set(struct ixgbe_adapter *adapter,
- struct rte_eth_syn_filter *filter,
- bool add);
void ixgbe_ethertype_filter_program(struct ixgbe_hw *hw, uint8_t idx,
uint32_t etqf, uint32_t etqs);
int ixgbe_ethertype_table_add(struct ixgbe_ethertype_table *table,
uint16_t ethertype, uint32_t etqf, uint32_t etqs);
int ixgbe_ethertype_table_del(struct ixgbe_ethertype_table *table, uint8_t idx);
+void ixgbe_syn_filter_program(struct ixgbe_hw *hw, uint32_t synqf);
/**
* l2 tunnel configuration.
@@ -765,7 +761,6 @@ int ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev);
extern const struct rte_flow_ops ixgbe_flow_ops;
void ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev);
-void ixgbe_clear_syn_filter(struct rte_eth_dev *dev);
int ixgbe_clear_all_l2_tn_filter(struct rte_eth_dev *dev);
int ixgbe_disable_sec_tx_path_generic(struct ixgbe_hw *hw);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index bde8759bf7..148f7e6d26 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -63,11 +63,6 @@ struct ixgbe_ntuple_filter_ele {
struct ixgbe_filter_ele_base base;
struct rte_eth_ntuple_filter filter_info;
};
-/* syn filter list structure */
-struct ixgbe_eth_syn_filter_ele {
- struct ixgbe_filter_ele_base base;
- struct rte_eth_syn_filter filter_info;
-};
/* fdir filter list structure */
struct ixgbe_fdir_rule_ele {
struct ixgbe_filter_ele_base base;
@@ -92,7 +87,8 @@ struct ixgbe_flow_mem {
const struct ci_flow_engine_list ixgbe_flow_engine_list = {
{
&ixgbe_ethertype_flow_engine,
- }
+ &ixgbe_syn_flow_engine,
+ },
};
/**
@@ -666,205 +662,6 @@ ixgbe_parse_ntuple_filter(struct rte_eth_dev *dev,
return 0;
}
-/**
- * Parse the rule to see if it is a TCP SYN rule.
- * And get the TCP SYN filter info BTW.
- * pattern:
- * The first not void item must be ETH.
- * The second not void item must be IPV4 or IPV6.
- * The third not void item must be TCP.
- * The next not void item must be END.
- * action:
- * The first not void action should be QUEUE.
- * The next not void action should be END.
- * pattern example:
- * ITEM Spec Mask
- * ETH NULL NULL
- * IPV4/IPV6 NULL NULL
- * TCP tcp_flags 0x02 0xFF
- * END
- * other members in mask and spec should set to 0x00.
- * item->last should be NULL.
- */
-static int
-cons_parse_syn_filter(const struct rte_flow_attr *attr, const struct rte_flow_item pattern[],
- const struct rte_flow_action_queue *q_act, struct rte_eth_syn_filter *filter,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_tcp *tcp_spec;
- const struct rte_flow_item_tcp *tcp_mask;
-
-
- /* the first not void item should be MAC or IPv4 or IPv6 or TCP */
- item = next_no_void_pattern(pattern, NULL);
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6 &&
- item->type != RTE_FLOW_ITEM_TYPE_TCP) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by syn filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Skip Ethernet */
- if (item->type == RTE_FLOW_ITEM_TYPE_ETH) {
- /* if the item is MAC, the content should be NULL */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid SYN address mask");
- return -rte_errno;
- }
-
- /* check if the next not void item is IPv4 or IPv6 */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by syn filter");
- return -rte_errno;
- }
- }
-
- /* Skip IP */
- if (item->type == RTE_FLOW_ITEM_TYPE_IPV4 ||
- item->type == RTE_FLOW_ITEM_TYPE_IPV6) {
- /* if the item is IP, the content should be NULL */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid SYN mask");
- return -rte_errno;
- }
-
- /* check if the next not void item is TCP */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_TCP) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by syn filter");
- return -rte_errno;
- }
- }
-
- /* Get the TCP info. Only support SYN. */
- if (!item->spec || !item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid SYN mask");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- tcp_spec = item->spec;
- tcp_mask = item->mask;
- if (!(tcp_spec->hdr.tcp_flags & RTE_TCP_SYN_FLAG) ||
- tcp_mask->hdr.src_port ||
- tcp_mask->hdr.dst_port ||
- tcp_mask->hdr.sent_seq ||
- tcp_mask->hdr.recv_ack ||
- tcp_mask->hdr.data_off ||
- tcp_mask->hdr.tcp_flags != RTE_TCP_SYN_FLAG ||
- tcp_mask->hdr.rx_win ||
- tcp_mask->hdr.cksum ||
- tcp_mask->hdr.tcp_urp) {
- memset(filter, 0, sizeof(struct rte_eth_syn_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by syn filter");
- return -rte_errno;
- }
-
- /* check if the next not void item is END */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(filter, 0, sizeof(struct rte_eth_syn_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by syn filter");
- return -rte_errno;
- }
-
- filter->queue = q_act->index;
-
- /* Support 2 priorities, the lowest or highest. */
- if (!attr->priority) {
- filter->hig_pri = 0;
- } else if (attr->priority == (uint32_t)~0U) {
- filter->hig_pri = 1;
- } else {
- memset(filter, 0, sizeof(struct rte_eth_syn_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
- attr, "Priority can be 0 or 0xFFFFFFFF");
- return -rte_errno;
- }
-
- return 0;
-}
-
-static int
-ixgbe_parse_syn_filter(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[], const struct rte_flow_action actions[],
- struct rte_eth_syn_filter *filter, struct rte_flow_error *error)
-{
- int ret;
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only queue is allowed here */
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_END
- },
- .driver_ctx = dev->data,
- .check = ixgbe_flow_actions_check,
- .max_actions = 1,
- };
- struct ci_flow_attr_check_param attr_param = {
- .allow_priority = true,
- };
- const struct rte_flow_action *action;
-
- if (hw->mac.type != ixgbe_mac_82599EB &&
- hw->mac.type != ixgbe_mac_X540 &&
- hw->mac.type != ixgbe_mac_X550 &&
- hw->mac.type != ixgbe_mac_X550EM_x &&
- hw->mac.type != ixgbe_mac_X550EM_a &&
- hw->mac.type != ixgbe_mac_E610)
- return -ENOTSUP;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, &attr_param, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
-
- action = parsed_actions.actions[0];
-
- return cons_parse_syn_filter(attr, pattern, action->conf, filter, error);
-}
-
/**
* Parse the rule to see if it is a L2 tunnel rule.
* And get the L2 tunnel filter info BTW.
@@ -2651,7 +2448,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct ixgbe_adapter *adapter =
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_syn_filter syn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_hw_fdir_info *fdir_info =
@@ -2659,7 +2455,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct ixgbe_rte_flow_rss_conf rss_conf;
struct rte_flow *flow = NULL;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_eth_syn_filter_ele *syn_filter_ptr;
struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_rss_conf_ele *rss_filter_ptr;
@@ -2718,26 +2513,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
goto out;
}
- memset(&syn_filter, 0, sizeof(struct rte_eth_syn_filter));
- ret = ixgbe_parse_syn_filter(dev, attr, pattern,
- actions, &syn_filter, error);
- if (!ret) {
- ret = ixgbe_syn_filter_set(adapter, &syn_filter, TRUE);
- if (!ret) {
- syn_filter_ptr = rte_zmalloc("ixgbe_syn_filter",
- sizeof(struct ixgbe_eth_syn_filter_ele), 0);
- if (!syn_filter_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- syn_filter_ptr->filter_info = syn_filter;
- flow->rule = syn_filter_ptr;
- flow->filter_type = RTE_ETH_FILTER_SYN;
- return flow;
- }
- goto out;
- }
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2835,7 +2610,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_syn_filter syn_filter;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_rte_flow_rss_conf rss_conf;
@@ -2861,12 +2635,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
if (!ret)
return 0;
- memset(&syn_filter, 0, sizeof(struct rte_eth_syn_filter));
- ret = ixgbe_parse_syn_filter(dev, attr, pattern,
- actions, &syn_filter, error);
- if (!ret)
- return 0;
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2898,11 +2666,9 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
struct rte_flow *pmd_flow = flow;
enum rte_filter_type filter_type = pmd_flow->filter_type;
struct rte_eth_ntuple_filter ntuple_filter;
- struct rte_eth_syn_filter syn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_eth_syn_filter_ele *syn_filter_ptr;
struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_filter_ele_base *flow_mem_base;
@@ -2947,14 +2713,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
if (!ret)
rte_free(ntuple_filter_ptr);
break;
- case RTE_ETH_FILTER_SYN:
- syn_filter_ptr = (struct ixgbe_eth_syn_filter_ele *)
- pmd_flow->rule;
- syn_filter = syn_filter_ptr->filter_info;
- ret = ixgbe_syn_filter_set(adapter, &syn_filter, FALSE);
- if (!ret)
- rte_free(syn_filter_ptr);
- break;
case RTE_ETH_FILTER_FDIR:
fdir_rule_ptr = (struct ixgbe_fdir_rule_ele *)pmd_flow->rule;
fdir_rule = fdir_rule_ptr->filter_info;
@@ -3023,7 +2781,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
}
ixgbe_clear_all_ntuple_filter(dev);
- ixgbe_clear_syn_filter(dev);
ret = ixgbe_clear_all_fdir_filter(dev);
if (ret < 0) {
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index d7694283a5..453a23d3b6 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -16,5 +16,6 @@ ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
extern const struct ci_flow_engine_list ixgbe_flow_engine_list;
extern const struct ci_flow_engine ixgbe_ethertype_flow_engine;
+extern const struct ci_flow_engine ixgbe_syn_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_syn.c b/drivers/net/intel/ixgbe/ixgbe_flow_syn.c
new file mode 100644
index 0000000000..17dd9f66da
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_syn.c
@@ -0,0 +1,312 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_syn_flow {
+ struct rte_flow flow;
+ struct rte_eth_syn_filter syn;
+};
+
+struct ixgbe_syn_ctx {
+ struct ci_flow_engine_ctx base;
+ struct rte_eth_syn_filter syn;
+};
+
+struct ixgbe_syn_priv {
+ bool installed; /* hardware supports a single SYN filter */
+};
+
+/**
+ * SYN filter graph implementation
+ * Pattern: START -> [ETH -> (IPV4|IPV6)] -> TCP -> END
+ */
+
+enum ixgbe_syn_node_id {
+ IXGBE_SYN_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_SYN_NODE_ETH,
+ IXGBE_SYN_NODE_IPV4,
+ IXGBE_SYN_NODE_IPV6,
+ IXGBE_SYN_NODE_TCP,
+ IXGBE_SYN_NODE_END,
+ IXGBE_SYN_NODE_MAX,
+};
+
+static int
+ixgbe_validate_syn_tcp(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_tcp *tcp_spec;
+ const struct rte_flow_item_tcp *tcp_mask;
+
+ tcp_spec = item->spec;
+ tcp_mask = item->mask;
+
+ /* SYN flag must be set in spec */
+ if (!(tcp_spec->hdr.tcp_flags & RTE_TCP_SYN_FLAG)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "TCP SYN flag must be set");
+ }
+
+ /* Mask must match only SYN flag */
+ if (tcp_mask->hdr.tcp_flags != RTE_TCP_SYN_FLAG) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "TCP flags mask must match SYN only");
+ }
+
+ /* All other TCP fields must have zero mask */
+ if (tcp_mask->hdr.src_port ||
+ tcp_mask->hdr.dst_port ||
+ tcp_mask->hdr.sent_seq ||
+ tcp_mask->hdr.recv_ack ||
+ tcp_mask->hdr.data_off ||
+ tcp_mask->hdr.rx_win ||
+ tcp_mask->hdr.cksum ||
+ tcp_mask->hdr.tcp_urp) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only TCP flags filtering supported");
+ }
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_syn_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_SYN_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_SYN_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_SYN_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_SYN_NODE_IPV6] = {
+ .name = "IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_SYN_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_syn_tcp,
+ },
+ [IXGBE_SYN_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_SYN_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_SYN_NODE_ETH,
+ IXGBE_SYN_NODE_IPV4,
+ IXGBE_SYN_NODE_IPV6,
+ IXGBE_SYN_NODE_TCP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SYN_NODE_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_SYN_NODE_IPV4,
+ IXGBE_SYN_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SYN_NODE_IPV4] = {
+ .next = (size_t[]) {
+ IXGBE_SYN_NODE_TCP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SYN_NODE_IPV6] = {
+ .next = (size_t[]) {
+ IXGBE_SYN_NODE_TCP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SYN_NODE_TCP] = {
+ .next = (size_t[]) {
+ IXGBE_SYN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+ixgbe_flow_syn_ctx_init(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_syn_ctx *syn_ctx = (struct ixgbe_syn_ctx *)ctx;
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only queue is allowed here */
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_flow_actions_check,
+ .max_actions = 1,
+ };
+ struct ci_flow_attr_check_param attr_param = {
+ .allow_priority = true,
+ };
+ const struct rte_flow_action_queue *q_act;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, &attr_param, error);
+ if (ret)
+ return ret;
+
+ /* check priority */
+ if (attr->priority != 0 && attr->priority != (uint32_t)~0U) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
+ attr, "Priority can be 0 or 0xFFFFFFFF");
+ }
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ q_act = parsed_actions.actions[0]->conf;
+
+ syn_ctx->syn.queue = q_act->index;
+
+ /* Support 2 priorities. rte_flow priority 0 is highest */
+ syn_ctx->syn.hig_pri = attr->priority == 0;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_syn_ctx *syn_ctx = (const struct ixgbe_syn_ctx *)ctx;
+ struct ixgbe_syn_flow *syn_flow = (struct ixgbe_syn_flow *)flow;
+
+ syn_flow->syn = syn_ctx->syn;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_flow_register(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_syn_priv *priv = flow->engine_priv;
+
+ /* hardware supports a single SYN filter */
+ if (priv->installed) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "SYN filter already exists");
+ }
+
+ priv->installed = true;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_flow_unregister(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_syn_priv *priv = flow->engine_priv;
+
+ priv->installed = false;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_syn_flow *syn_flow = (struct ixgbe_syn_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+ uint32_t synqf;
+
+ synqf = ((syn_flow->syn.queue << IXGBE_SYN_FILTER_QUEUE_SHIFT) &
+ IXGBE_SYN_FILTER_QUEUE) | IXGBE_SYN_FILTER_ENABLE;
+ if (syn_flow->syn.hig_pri)
+ synqf |= IXGBE_SYN_FILTER_SYNQFP;
+
+ ixgbe_syn_filter_program(hw, synqf);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_syn_filter_program(hw, 0);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_syn_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ if (hw->mac.type == ixgbe_mac_82599EB ||
+ hw->mac.type == ixgbe_mac_X540 ||
+ hw->mac.type == ixgbe_mac_X550 ||
+ hw->mac.type == ixgbe_mac_X550EM_x ||
+ hw->mac.type == ixgbe_mac_X550EM_a ||
+ hw->mac.type == ixgbe_mac_E610)
+ return 0;
+
+ return -ENOTSUP;
+}
+
+static const struct ci_flow_engine_ops ixgbe_syn_ops = {
+ .engine_init = ixgbe_flow_syn_engine_init,
+ .ctx_init = ixgbe_flow_syn_ctx_init,
+ .ctx_to_flow = ixgbe_flow_syn_ctx_to_flow,
+ .flow_register = ixgbe_flow_syn_flow_register,
+ .flow_unregister = ixgbe_flow_syn_flow_unregister,
+ .flow_install = ixgbe_flow_syn_flow_install,
+ .flow_uninstall = ixgbe_flow_syn_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_syn_flow_engine = {
+ .name = "syn",
+ .ctx_size = sizeof(struct ixgbe_syn_ctx),
+ .flow_size = sizeof(struct ixgbe_syn_flow),
+ .priv_size = sizeof(struct ixgbe_syn_priv),
+ .ops = &ixgbe_syn_ops,
+ .graph = &ixgbe_syn_graph,
+};
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index f2857feab7..1ef818fa67 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -27,6 +27,7 @@ sources += files(
'ixgbe_fdir.c',
'ixgbe_flow.c',
'ixgbe_flow_ethertype.c',
+ 'ixgbe_flow_syn.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 07/19] net/ixgbe: reimplement L2 tunnel parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (5 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 06/19] net/ixgbe: reimplement syn parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 08/19] net/ixgbe: reimplement ntuple parser Anatoly Burakov
` (12 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for L2 tunnel.
There are two L2-tag-related features in the driver: the global per-port
one, and the one that can direct traffic to specific queues (the one the
engine targets). The former is left completely untouched, while the latter
one is migrated to use engine infrastructure and use internal tracking.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 248 +---------------
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 26 +-
drivers/net/intel/ixgbe/ixgbe_flow.c | 206 +------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 1 +
drivers/net/intel/ixgbe/ixgbe_flow_l2tun.c | 317 +++++++++++++++++++++
drivers/net/intel/ixgbe/meson.build | 1 +
6 files changed, 326 insertions(+), 473 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_l2tun.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index e4576b3844..63df23f8e4 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,6 @@ static int eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev);
static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev);
-static int ixgbe_l2_tn_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_ntuple_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_dev_configure(struct rte_eth_dev *dev);
static int ixgbe_dev_start(struct rte_eth_dev *dev);
@@ -1353,7 +1352,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
return 0;
err_flow_engine_conf_init:
- ixgbe_l2_tn_filter_uninit(eth_dev);
err_l2_tn_filter_init:
ixgbe_fdir_filter_uninit(eth_dev);
err_fdir_filter_init:
@@ -1423,25 +1421,6 @@ static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev)
return 0;
}
-static int ixgbe_l2_tn_filter_uninit(struct rte_eth_dev *eth_dev)
-{
- struct ixgbe_l2_tn_info *l2_tn_info =
- IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(eth_dev->data->dev_private);
- struct ixgbe_l2_tn_filter *l2_tn_filter;
-
- rte_free(l2_tn_info->hash_map);
- rte_hash_free(l2_tn_info->hash_handle);
-
- while ((l2_tn_filter = TAILQ_FIRST(&l2_tn_info->l2_tn_list))) {
- TAILQ_REMOVE(&l2_tn_info->l2_tn_list,
- l2_tn_filter,
- entries);
- rte_free(l2_tn_filter);
- }
-
- return 0;
-}
-
static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev)
{
struct ixgbe_hw_fdir_info *fdir_info =
@@ -1487,34 +1466,7 @@ static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev)
{
struct ixgbe_l2_tn_info *l2_tn_info =
IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(eth_dev->data->dev_private);
- char l2_tn_hash_name[RTE_HASH_NAMESIZE];
- struct rte_hash_parameters l2_tn_hash_params = {
- .name = l2_tn_hash_name,
- .entries = IXGBE_MAX_L2_TN_FILTER_NUM,
- .key_len = sizeof(struct ixgbe_l2_tn_key),
- .hash_func = rte_hash_crc,
- .hash_func_init_val = 0,
- .socket_id = rte_socket_id(),
- };
- TAILQ_INIT(&l2_tn_info->l2_tn_list);
- snprintf(l2_tn_hash_name, RTE_HASH_NAMESIZE,
- "l2_tn_%s", eth_dev->device->name);
- l2_tn_info->hash_handle = rte_hash_create(&l2_tn_hash_params);
- if (!l2_tn_info->hash_handle) {
- PMD_INIT_LOG(ERR, "Failed to create L2 TN hash table!");
- return -EINVAL;
- }
- l2_tn_info->hash_map = rte_zmalloc("ixgbe",
- sizeof(struct ixgbe_l2_tn_filter *) *
- IXGBE_MAX_L2_TN_FILTER_NUM,
- 0);
- if (!l2_tn_info->hash_map) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for L2 TN hash map!");
- rte_hash_free(l2_tn_info->hash_handle);
- return -ENOMEM;
- }
l2_tn_info->e_tag_en = FALSE;
l2_tn_info->e_tag_fwd_en = FALSE;
l2_tn_info->e_tag_ether_type = RTE_ETHER_TYPE_ETAG;
@@ -3158,9 +3110,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
/* remove all the fdir filters & hash */
ixgbe_fdir_filter_uninit(dev);
- /* remove all the L2 tunnel filters & hash */
- ixgbe_l2_tn_filter_uninit(dev);
-
/* Remove all ntuple filters of the device */
ixgbe_ntuple_filter_uninit(dev);
@@ -7706,7 +7655,7 @@ ixgbe_e_tag_enable(struct ixgbe_hw *hw)
return 0;
}
-static int
+int
ixgbe_e_tag_filter_del(struct ixgbe_adapter *adapter,
struct ixgbe_l2_tunnel_conf *l2_tunnel)
{
@@ -7743,7 +7692,7 @@ ixgbe_e_tag_filter_del(struct ixgbe_adapter *adapter,
return ret;
}
-static int
+int
ixgbe_e_tag_filter_add(struct ixgbe_adapter *adapter,
struct ixgbe_l2_tunnel_conf *l2_tunnel)
{
@@ -7785,154 +7734,6 @@ ixgbe_e_tag_filter_add(struct ixgbe_adapter *adapter,
return -EINVAL;
}
-static inline struct ixgbe_l2_tn_filter *
-ixgbe_l2_tn_filter_lookup(struct ixgbe_l2_tn_info *l2_tn_info,
- struct ixgbe_l2_tn_key *key)
-{
- int ret;
-
- ret = rte_hash_lookup(l2_tn_info->hash_handle, (const void *)key);
- if (ret < 0)
- return NULL;
-
- return l2_tn_info->hash_map[ret];
-}
-
-static inline int
-ixgbe_insert_l2_tn_filter(struct ixgbe_l2_tn_info *l2_tn_info,
- struct ixgbe_l2_tn_filter *l2_tn_filter)
-{
- int ret;
-
- ret = rte_hash_add_key(l2_tn_info->hash_handle,
- &l2_tn_filter->key);
-
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to insert L2 tunnel filter"
- " to hash table %d!",
- ret);
- return ret;
- }
-
- l2_tn_info->hash_map[ret] = l2_tn_filter;
-
- TAILQ_INSERT_TAIL(&l2_tn_info->l2_tn_list, l2_tn_filter, entries);
-
- return 0;
-}
-
-static inline int
-ixgbe_remove_l2_tn_filter(struct ixgbe_l2_tn_info *l2_tn_info,
- struct ixgbe_l2_tn_key *key)
-{
- int ret;
- struct ixgbe_l2_tn_filter *l2_tn_filter;
-
- ret = rte_hash_del_key(l2_tn_info->hash_handle, key);
-
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "No such L2 tunnel filter to delete %d!",
- ret);
- return ret;
- }
-
- l2_tn_filter = l2_tn_info->hash_map[ret];
- l2_tn_info->hash_map[ret] = NULL;
-
- TAILQ_REMOVE(&l2_tn_info->l2_tn_list, l2_tn_filter, entries);
- rte_free(l2_tn_filter);
-
- return 0;
-}
-
-/* Add l2 tunnel filter */
-int
-ixgbe_dev_l2_tunnel_filter_add(struct ixgbe_adapter *adapter,
- struct ixgbe_l2_tunnel_conf *l2_tunnel,
- bool restore)
-{
- int ret;
- struct ixgbe_l2_tn_info *l2_tn_info =
- IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(adapter);
- struct ixgbe_l2_tn_key key;
- struct ixgbe_l2_tn_filter *node;
-
- if (!restore) {
- key.l2_tn_type = l2_tunnel->l2_tunnel_type;
- key.tn_id = l2_tunnel->tunnel_id;
-
- node = ixgbe_l2_tn_filter_lookup(l2_tn_info, &key);
-
- if (node) {
- PMD_DRV_LOG(ERR,
- "The L2 tunnel filter already exists!");
- return -EINVAL;
- }
-
- node = rte_zmalloc("ixgbe_l2_tn",
- sizeof(struct ixgbe_l2_tn_filter),
- 0);
- if (!node)
- return -ENOMEM;
-
- memcpy(&node->key,
- &key,
- sizeof(struct ixgbe_l2_tn_key));
- node->pool = l2_tunnel->pool;
- ret = ixgbe_insert_l2_tn_filter(l2_tn_info, node);
- if (ret < 0) {
- rte_free(node);
- return ret;
- }
- }
-
- switch (l2_tunnel->l2_tunnel_type) {
- case RTE_ETH_L2_TUNNEL_TYPE_E_TAG:
- ret = ixgbe_e_tag_filter_add(adapter, l2_tunnel);
- break;
- default:
- PMD_DRV_LOG(ERR, "Invalid tunnel type");
- ret = -EINVAL;
- break;
- }
-
- if ((!restore) && (ret < 0))
- (void)ixgbe_remove_l2_tn_filter(l2_tn_info, &key);
-
- return ret;
-}
-
-/* Delete l2 tunnel filter */
-int
-ixgbe_dev_l2_tunnel_filter_del(struct ixgbe_adapter *adapter,
- struct ixgbe_l2_tunnel_conf *l2_tunnel)
-{
- int ret;
- struct ixgbe_l2_tn_info *l2_tn_info =
- IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(adapter);
- struct ixgbe_l2_tn_key key;
-
- key.l2_tn_type = l2_tunnel->l2_tunnel_type;
- key.tn_id = l2_tunnel->tunnel_id;
- ret = ixgbe_remove_l2_tn_filter(l2_tn_info, &key);
- if (ret < 0)
- return ret;
-
- switch (l2_tunnel->l2_tunnel_type) {
- case RTE_ETH_L2_TUNNEL_TYPE_E_TAG:
- ret = ixgbe_e_tag_filter_del(adapter, l2_tunnel);
- break;
- default:
- PMD_DRV_LOG(ERR, "Invalid tunnel type");
- ret = -EINVAL;
- break;
- }
-
- return ret;
-}
-
static int
ixgbe_e_tag_forwarding_en_dis(struct rte_eth_dev *dev, bool en)
{
@@ -8330,26 +8131,6 @@ ixgbe_ntuple_filter_restore(struct rte_eth_dev *dev)
}
}
-/* restore L2 tunnel filter */
-static inline void
-ixgbe_l2_tn_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_l2_tn_info *l2_tn_info =
- IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(adapter);
- struct ixgbe_l2_tn_filter *node;
- struct ixgbe_l2_tunnel_conf l2_tn_conf;
-
- TAILQ_FOREACH(node, &l2_tn_info->l2_tn_list, entries) {
- l2_tn_conf.l2_tunnel_type = node->key.l2_tn_type;
- l2_tn_conf.tunnel_id = node->key.tn_id;
- l2_tn_conf.pool = node->pool;
- (void)ixgbe_dev_l2_tunnel_filter_add(adapter,
- &l2_tn_conf, TRUE);
- }
-}
-
/* restore rss filter */
static inline void
ixgbe_rss_filter_restore(struct rte_eth_dev *dev)
@@ -8369,7 +8150,6 @@ ixgbe_filter_restore(struct rte_eth_dev *dev)
{
ixgbe_ntuple_filter_restore(dev);
ixgbe_fdir_filter_restore(dev);
- ixgbe_l2_tn_filter_restore(dev);
ixgbe_rss_filter_restore(dev);
return 0;
@@ -8405,30 +8185,6 @@ ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev)
ixgbe_remove_5tuple_filter(adapter, p_5tuple);
}
-/* remove all the L2 tunnel filters */
-int
-ixgbe_clear_all_l2_tn_filter(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_l2_tn_info *l2_tn_info =
- IXGBE_DEV_PRIVATE_TO_L2_TN_INFO(dev->data->dev_private);
- struct ixgbe_l2_tn_filter *l2_tn_filter;
- struct ixgbe_l2_tunnel_conf l2_tn_conf;
- int ret = 0;
-
- while ((l2_tn_filter = TAILQ_FIRST(&l2_tn_info->l2_tn_list))) {
- l2_tn_conf.l2_tunnel_type = l2_tn_filter->key.l2_tn_type;
- l2_tn_conf.tunnel_id = l2_tn_filter->key.tn_id;
- l2_tn_conf.pool = l2_tn_filter->pool;
- ret = ixgbe_dev_l2_tunnel_filter_del(adapter, &l2_tn_conf);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
-
void
ixgbe_dev_macsec_setting_save(struct rte_eth_dev *dev,
struct ixgbe_macsec_setting *macsec_setting)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 59d58f6160..7713932236 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -327,23 +327,7 @@ struct ixgbe_filter_info {
uint8_t antispoof_idx;
};
-struct ixgbe_l2_tn_key {
- enum rte_eth_tunnel_type l2_tn_type;
- uint32_t tn_id;
-};
-
-struct ixgbe_l2_tn_filter {
- TAILQ_ENTRY(ixgbe_l2_tn_filter) entries;
- struct ixgbe_l2_tn_key key;
- uint32_t pool;
-};
-
-TAILQ_HEAD(ixgbe_l2_tn_filter_list, ixgbe_l2_tn_filter);
-
struct ixgbe_l2_tn_info {
- struct ixgbe_l2_tn_filter_list l2_tn_list;
- struct ixgbe_l2_tn_filter **hash_map;
- struct rte_hash *hash_handle;
bool e_tag_en; /* e-tag enabled */
bool e_tag_fwd_en; /* e-tag based forwarding enabled */
uint16_t e_tag_ether_type; /* ether type for e-tag */
@@ -702,12 +686,11 @@ struct ixgbe_l2_tunnel_conf {
};
int
-ixgbe_dev_l2_tunnel_filter_add(struct ixgbe_adapter *adapter,
- struct ixgbe_l2_tunnel_conf *l2_tunnel,
- bool restore);
+ixgbe_e_tag_filter_add(struct ixgbe_adapter *adapter,
+ struct ixgbe_l2_tunnel_conf *l2_tunnel);
int
-ixgbe_dev_l2_tunnel_filter_del(struct ixgbe_adapter *adapter,
- struct ixgbe_l2_tunnel_conf *l2_tunnel);
+ixgbe_e_tag_filter_del(struct ixgbe_adapter *adapter,
+ struct ixgbe_l2_tunnel_conf *l2_tunnel);
void ixgbe_filterlist_init(struct rte_eth_dev *dev);
void ixgbe_filterlist_flush(struct rte_eth_dev *dev);
/*
@@ -761,7 +744,6 @@ int ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev);
extern const struct rte_flow_ops ixgbe_flow_ops;
void ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev);
-int ixgbe_clear_all_l2_tn_filter(struct rte_eth_dev *dev);
int ixgbe_disable_sec_tx_path_generic(struct ixgbe_hw *hw);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 148f7e6d26..46e4fd2bba 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -68,11 +68,6 @@ struct ixgbe_fdir_rule_ele {
struct ixgbe_filter_ele_base base;
struct ixgbe_fdir_rule filter_info;
};
-/* l2_tunnel filter list structure */
-struct ixgbe_eth_l2_tunnel_conf_ele {
- struct ixgbe_filter_ele_base base;
- struct ixgbe_l2_tunnel_conf filter_info;
-};
/* rss filter list structure */
struct ixgbe_rss_conf_ele {
struct ixgbe_filter_ele_base base;
@@ -88,6 +83,7 @@ const struct ci_flow_engine_list ixgbe_flow_engine_list = {
{
&ixgbe_ethertype_flow_engine,
&ixgbe_syn_flow_engine,
+ &ixgbe_l2_tunnel_flow_engine,
},
};
@@ -662,161 +658,6 @@ ixgbe_parse_ntuple_filter(struct rte_eth_dev *dev,
return 0;
}
-/**
- * Parse the rule to see if it is a L2 tunnel rule.
- * And get the L2 tunnel filter info BTW.
- * Only support E-tag now.
- * pattern:
- * The first not void item can be E_TAG.
- * The next not void item must be END.
- * action:
- * The first not void action should be VF or PF.
- * The next not void action should be END.
- * pattern example:
- * ITEM Spec Mask
- * E_TAG grp 0x1 0x3
- e_cid_base 0x309 0xFFF
- * END
- * other members in mask and spec should set to 0x00.
- * item->last should be NULL.
- */
-static int
-cons_parse_l2_tn_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action *action,
- struct ixgbe_l2_tunnel_conf *filter,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_e_tag *e_tag_spec;
- const struct rte_flow_item_e_tag *e_tag_mask;
- struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
-
- /* The first not void item should be e-tag. */
- item = next_no_void_pattern(pattern, NULL);
- if (item->type != RTE_FLOW_ITEM_TYPE_E_TAG) {
- memset(filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by L2 tunnel filter");
- return -rte_errno;
- }
-
- if (!item->spec || !item->mask) {
- memset(filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by L2 tunnel filter");
- return -rte_errno;
- }
-
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- e_tag_spec = item->spec;
- e_tag_mask = item->mask;
-
- /* Only care about GRP and E cid base. */
- if (e_tag_mask->epcp_edei_in_ecid_b ||
- e_tag_mask->in_ecid_e ||
- e_tag_mask->ecid_e ||
- e_tag_mask->rsvd_grp_ecid_b != rte_cpu_to_be_16(0x3FFF)) {
- memset(filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by L2 tunnel filter");
- return -rte_errno;
- }
-
- filter->l2_tunnel_type = RTE_ETH_L2_TUNNEL_TYPE_E_TAG;
- /**
- * grp and e_cid_base are bit fields and only use 14 bits.
- * e-tag id is taken as little endian by HW.
- */
- filter->tunnel_id = rte_be_to_cpu_16(e_tag_spec->rsvd_grp_ecid_b);
-
- /* check if the next not void item is END */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by L2 tunnel filter");
- return -rte_errno;
- }
-
- if (action->type == RTE_FLOW_ACTION_TYPE_VF) {
- const struct rte_flow_action_vf *act_vf = action->conf;
- filter->pool = act_vf->id;
- } else {
- filter->pool = ad->max_vfs;
- }
-
- return 0;
-}
-
-static int
-ixgbe_parse_l2_tn_filter(struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct ixgbe_l2_tunnel_conf *l2_tn_filter,
- struct rte_flow_error *error)
-{
- struct rte_eth_dev_data *dev_data = dev->data;
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only vf/pf is allowed here */
- RTE_FLOW_ACTION_TYPE_VF,
- RTE_FLOW_ACTION_TYPE_PF,
- RTE_FLOW_ACTION_TYPE_END
- },
- .driver_ctx = dev_data,
- .check = ixgbe_flow_actions_check,
- .max_actions = 1,
- };
- int ret = 0;
- const struct rte_flow_action *action;
-
- if (hw->mac.type != ixgbe_mac_X550 &&
- hw->mac.type != ixgbe_mac_X550EM_x &&
- hw->mac.type != ixgbe_mac_X550EM_a &&
- hw->mac.type != ixgbe_mac_E610) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "Not supported by L2 tunnel filter");
- return -rte_errno;
- }
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
-
- /* only one action is supported */
- if (parsed_actions.count > 1) {
- return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- parsed_actions.actions[1],
- "Only one action can be specified at a time");
- }
- action = parsed_actions.actions[0];
-
- ret = cons_parse_l2_tn_filter(dev, pattern, action, l2_tn_filter, error);
-
- return ret;
-}
-
/* search next no void pattern and skip fuzzy */
static inline
const struct rte_flow_item *next_no_fuzzy_pattern(
@@ -2449,13 +2290,11 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
struct ixgbe_fdir_rule fdir_rule;
- struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_hw_fdir_info *fdir_info =
IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
struct ixgbe_rte_flow_rss_conf rss_conf;
struct rte_flow *flow = NULL;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_rss_conf_ele *rss_filter_ptr;
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr;
@@ -2546,25 +2385,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
return flow;
}
- memset(&l2_tn_filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- ret = ixgbe_parse_l2_tn_filter(dev, attr, pattern,
- actions, &l2_tn_filter, error);
- if (!ret) {
- ret = ixgbe_dev_l2_tunnel_filter_add(adapter, &l2_tn_filter, FALSE);
- if (!ret) {
- l2_tn_filter_ptr = rte_zmalloc("ixgbe_l2_tn_filter",
- sizeof(struct ixgbe_eth_l2_tunnel_conf_ele), 0);
- if (!l2_tn_filter_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- l2_tn_filter_ptr->filter_info = l2_tn_filter;
- flow->rule = l2_tn_filter_ptr;
- flow->filter_type = RTE_ETH_FILTER_L2_TUNNEL;
- return flow;
- }
- }
-
memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
ret = ixgbe_parse_rss_filter(dev, attr,
actions, &rss_conf, error);
@@ -2610,7 +2430,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_eth_ntuple_filter ntuple_filter;
- struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_rte_flow_rss_conf rss_conf;
int ret;
@@ -2641,12 +2460,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
if (!ret)
return 0;
- memset(&l2_tn_filter, 0, sizeof(struct ixgbe_l2_tunnel_conf));
- ret = ixgbe_parse_l2_tn_filter(dev, attr, pattern,
- actions, &l2_tn_filter, error);
- if (!ret)
- return 0;
-
memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
ret = ixgbe_parse_rss_filter(dev, attr,
actions, &rss_conf, error);
@@ -2667,9 +2480,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
enum rte_filter_type filter_type = pmd_flow->filter_type;
struct rte_eth_ntuple_filter ntuple_filter;
struct ixgbe_fdir_rule fdir_rule;
- struct ixgbe_l2_tunnel_conf l2_tn_filter;
struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
- struct ixgbe_eth_l2_tunnel_conf_ele *l2_tn_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_filter_ele_base *flow_mem_base;
struct ixgbe_hw_fdir_info *fdir_info =
@@ -2727,14 +2538,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
}
}
break;
- case RTE_ETH_FILTER_L2_TUNNEL:
- l2_tn_filter_ptr = (struct ixgbe_eth_l2_tunnel_conf_ele *)
- pmd_flow->rule;
- l2_tn_filter = l2_tn_filter_ptr->filter_info;
- ret = ixgbe_dev_l2_tunnel_filter_del(adapter, &l2_tn_filter);
- if (!ret)
- rte_free(l2_tn_filter_ptr);
- break;
case RTE_ETH_FILTER_HASH:
rss_filter_ptr = (struct ixgbe_rss_conf_ele *)
pmd_flow->rule;
@@ -2789,13 +2592,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
return ret;
}
- ret = ixgbe_clear_all_l2_tn_filter(dev);
- if (ret < 0) {
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Failed to flush rule");
- return ret;
- }
-
ixgbe_clear_rss_filter(dev);
ixgbe_filterlist_flush(dev);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index 453a23d3b6..ba0486b2c0 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -17,5 +17,6 @@ extern const struct ci_flow_engine_list ixgbe_flow_engine_list;
extern const struct ci_flow_engine ixgbe_ethertype_flow_engine;
extern const struct ci_flow_engine ixgbe_syn_flow_engine;
+extern const struct ci_flow_engine ixgbe_l2_tunnel_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_l2tun.c b/drivers/net/intel/ixgbe/ixgbe_flow_l2tun.c
new file mode 100644
index 0000000000..c95eafa37a
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_l2tun.c
@@ -0,0 +1,317 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_l2_tunnel_flow {
+ struct rte_flow flow;
+ struct ixgbe_l2_tunnel_conf l2_tunnel;
+};
+
+struct ixgbe_l2_tunnel_ctx {
+ struct ci_flow_engine_ctx base;
+ struct ixgbe_l2_tunnel_conf l2_tunnel;
+};
+
+/* per-device dedup set of installed tunnel ids (hardware matches by id) */
+struct ixgbe_l2_tunnel_priv {
+ uint32_t count;
+ uint32_t tunnel_id[IXGBE_MAX_L2_TN_FILTER_NUM];
+};
+
+/**
+ * L2 tunnel filter graph implementation (E-TAG)
+ * Pattern: START -> E_TAG -> END
+ */
+
+enum ixgbe_l2_tunnel_node_id {
+ IXGBE_L2_TUNNEL_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_L2_TUNNEL_NODE_E_TAG,
+ IXGBE_L2_TUNNEL_NODE_END,
+ IXGBE_L2_TUNNEL_NODE_MAX,
+};
+
+static int
+ixgbe_validate_l2_tunnel_e_tag(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_e_tag *e_tag_mask;
+
+ e_tag_mask = item->mask;
+
+ /* Only GRP and E-CID base supported (rsvd_grp_ecid_b field) */
+ if (e_tag_mask->epcp_edei_in_ecid_b ||
+ e_tag_mask->in_ecid_e ||
+ e_tag_mask->ecid_e ||
+ rte_be_to_cpu_16(e_tag_mask->rsvd_grp_ecid_b) != 0x3FFF) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only GRP and E-CID base (14 bits) supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_l2_tunnel_e_tag(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_l2_tunnel_ctx *l2tun_ctx = ctx;
+ const struct rte_flow_item_e_tag *e_tag_spec = item->spec;
+
+ l2tun_ctx->l2_tunnel.l2_tunnel_type = RTE_ETH_L2_TUNNEL_TYPE_E_TAG;
+ l2tun_ctx->l2_tunnel.tunnel_id = rte_be_to_cpu_16(e_tag_spec->rsvd_grp_ecid_b);
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_l2_tunnel_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_L2_TUNNEL_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_L2_TUNNEL_NODE_E_TAG] = {
+ .name = "E_TAG",
+ .type = RTE_FLOW_ITEM_TYPE_E_TAG,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_l2_tunnel_e_tag,
+ .process = ixgbe_process_l2_tunnel_e_tag,
+ },
+ [IXGBE_L2_TUNNEL_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_L2_TUNNEL_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_L2_TUNNEL_NODE_E_TAG,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_L2_TUNNEL_NODE_E_TAG] = {
+ .next = (size_t[]) {
+ IXGBE_L2_TUNNEL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+ixgbe_flow_l2_tunnel_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_l2_tunnel_ctx *l2tun_ctx = (struct ixgbe_l2_tunnel_ctx *)ctx;
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(ctx->dev_data->dev_private);
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only vf/pf is allowed here */
+ RTE_FLOW_ACTION_TYPE_VF,
+ RTE_FLOW_ACTION_TYPE_PF,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_flow_actions_check,
+ .max_actions = 1,
+ };
+ const struct rte_flow_action *action;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ action = parsed_actions.actions[0];
+
+ if (action->type == RTE_FLOW_ACTION_TYPE_VF) {
+ const struct rte_flow_action_vf *vf = action->conf;
+ l2tun_ctx->l2_tunnel.pool = vf->id;
+ } else {
+ l2tun_ctx->l2_tunnel.pool = adapter->max_vfs;
+ }
+
+ return ret;
+}
+
+static int
+ixgbe_flow_l2_tunnel_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_l2_tunnel_ctx *l2tun_ctx = (const struct ixgbe_l2_tunnel_ctx *)ctx;
+ struct ixgbe_l2_tunnel_flow *l2tun_flow = (struct ixgbe_l2_tunnel_flow *)flow;
+
+ l2tun_flow->l2_tunnel = l2tun_ctx->l2_tunnel;
+
+ return 0;
+}
+
+static int
+ixgbe_l2_tunnel_id_find(const struct ixgbe_l2_tunnel_priv *priv, uint32_t tunnel_id)
+{
+ uint32_t i;
+
+ for (i = 0; i < priv->count; i++) {
+ if (priv->tunnel_id[i] == tunnel_id)
+ return (int)i;
+ }
+
+ return -ENOENT;
+}
+
+static int
+ixgbe_l2_tunnel_id_add(struct ixgbe_l2_tunnel_priv *priv, uint32_t tunnel_id)
+{
+ if (priv->count >= IXGBE_MAX_L2_TN_FILTER_NUM)
+ return -ENOSPC;
+ if (ixgbe_l2_tunnel_id_find(priv, tunnel_id) >= 0)
+ return -EEXIST;
+
+ priv->tunnel_id[priv->count++] = tunnel_id;
+
+ return 0;
+}
+
+static int
+ixgbe_l2_tunnel_id_del(struct ixgbe_l2_tunnel_priv *priv, uint32_t tunnel_id)
+{
+ int idx = ixgbe_l2_tunnel_id_find(priv, tunnel_id);
+
+ if (idx < 0)
+ return -ENOENT;
+
+ /* keep a dense set, order doesn't matter */
+ priv->tunnel_id[idx] = priv->tunnel_id[--priv->count];
+
+ return 0;
+}
+
+static int
+ixgbe_flow_l2_tunnel_flow_register(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_l2_tunnel_flow *l2tun_flow = (struct ixgbe_l2_tunnel_flow *)flow;
+ struct ixgbe_l2_tunnel_priv *priv = flow->engine_priv;
+ int ret;
+
+ ret = ixgbe_l2_tunnel_id_add(priv, l2tun_flow->l2_tunnel.tunnel_id);
+ if (ret == -ENOSPC) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "L2 tunnel filter table is full");
+ }
+ if (ret == -EEXIST) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "L2 tunnel filter already exists");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_l2_tunnel_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_l2_tunnel_flow *l2tun_flow = (struct ixgbe_l2_tunnel_flow *)flow;
+ struct ixgbe_l2_tunnel_priv *priv = flow->engine_priv;
+
+ if (ixgbe_l2_tunnel_id_del(priv, l2tun_flow->l2_tunnel.tunnel_id) != 0) {
+ return rte_flow_error_set(error, ENOENT,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "L2 tunnel filter not found on unregister");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_l2_tunnel_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_l2_tunnel_flow *l2tun_flow = (struct ixgbe_l2_tunnel_flow *)flow;
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(flow->dev_data->dev_private);
+ int ret;
+
+ ret = ixgbe_e_tag_filter_add(adapter, &l2tun_flow->l2_tunnel);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Failed to add L2 tunnel filter");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_l2_tunnel_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_l2_tunnel_flow *l2tun_flow = (struct ixgbe_l2_tunnel_flow *)flow;
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(flow->dev_data->dev_private);
+ int ret;
+
+ ret = ixgbe_e_tag_filter_del(adapter, &l2tun_flow->l2_tunnel);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to remove L2 tunnel filter");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_l2_tunnel_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ if (hw->mac.type == ixgbe_mac_X550 ||
+ hw->mac.type == ixgbe_mac_X550EM_x ||
+ hw->mac.type == ixgbe_mac_X550EM_a ||
+ hw->mac.type == ixgbe_mac_E610)
+ return 0;
+
+ return -ENOTSUP;
+}
+
+static const struct ci_flow_engine_ops ixgbe_l2_tunnel_ops = {
+ .engine_init = ixgbe_flow_l2_tunnel_engine_init,
+ .ctx_init = ixgbe_flow_l2_tunnel_ctx_init,
+ .ctx_to_flow = ixgbe_flow_l2_tunnel_ctx_to_flow,
+ .flow_register = ixgbe_flow_l2_tunnel_flow_register,
+ .flow_unregister = ixgbe_flow_l2_tunnel_flow_unregister,
+ .flow_install = ixgbe_flow_l2_tunnel_flow_install,
+ .flow_uninstall = ixgbe_flow_l2_tunnel_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_l2_tunnel_flow_engine = {
+ .name = "l2_tunnel",
+ .ctx_size = sizeof(struct ixgbe_l2_tunnel_ctx),
+ .flow_size = sizeof(struct ixgbe_l2_tunnel_flow),
+ .priv_size = sizeof(struct ixgbe_l2_tunnel_priv),
+ .ops = &ixgbe_l2_tunnel_ops,
+ .graph = &ixgbe_l2_tunnel_graph,
+};
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 1ef818fa67..91ec261154 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -28,6 +28,7 @@ sources += files(
'ixgbe_flow.c',
'ixgbe_flow_ethertype.c',
'ixgbe_flow_syn.c',
+ 'ixgbe_flow_l2tun.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 08/19] net/ixgbe: reimplement ntuple parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (6 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 07/19] net/ixgbe: reimplement L2 tunnel parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 09/19] net/ixgbe: reimplement security parser Anatoly Burakov
` (11 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for ntuple.
The 5tuple filter tracking infrastructure is moved completely inside the
new engine and is removed from the rest of the driver.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 348 +-----------
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 28 +-
drivers/net/intel/ixgbe/ixgbe_flow.c | 463 +--------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 1 +
drivers/net/intel/ixgbe/ixgbe_flow_ntuple.c | 597 ++++++++++++++++++++
drivers/net/intel/ixgbe/meson.build | 1 +
6 files changed, 627 insertions(+), 811 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_ntuple.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 63df23f8e4..f1541a1554 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,6 @@ static int eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev);
static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev);
-static int ixgbe_ntuple_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_dev_configure(struct rte_eth_dev *dev);
static int ixgbe_dev_start(struct rte_eth_dev *dev);
static int ixgbe_dev_stop(struct rte_eth_dev *dev);
@@ -302,10 +301,6 @@ static int ixgbevf_add_mac_addr(struct rte_eth_dev *dev,
static void ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index);
static int ixgbevf_set_default_mac_addr(struct rte_eth_dev *dev,
struct rte_ether_addr *mac_addr);
-static int ixgbe_add_5tuple_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_5tuple_filter *filter);
-static void ixgbe_remove_5tuple_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_5tuple_filter *filter);
static int ixgbe_dev_flow_ops_get(struct rte_eth_dev *dev,
const struct rte_flow_ops **ops);
static int ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);
@@ -1321,9 +1316,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
memset(filter_info, 0,
sizeof(struct ixgbe_filter_info));
- /* initialize 5tuple filter list */
- TAILQ_INIT(&filter_info->fivetuple_list);
-
/* initialize flow director filter list & hash */
ret = ixgbe_fdir_filter_init(eth_dev);
if (ret)
@@ -1384,24 +1376,6 @@ eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
return 0;
}
-static int ixgbe_ntuple_filter_uninit(struct rte_eth_dev *eth_dev)
-{
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
- struct ixgbe_5tuple_filter *p_5tuple;
-
- while ((p_5tuple = TAILQ_FIRST(&filter_info->fivetuple_list))) {
- TAILQ_REMOVE(&filter_info->fivetuple_list,
- p_5tuple,
- entries);
- rte_free(p_5tuple);
- }
- memset(filter_info->fivetuple_mask, 0,
- sizeof(uint32_t) * IXGBE_5TUPLE_ARRAY_SIZE);
-
- return 0;
-}
-
static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev)
{
struct ixgbe_hw_fdir_info *fdir_info =
@@ -3110,9 +3084,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
/* remove all the fdir filters & hash */
ixgbe_fdir_filter_uninit(dev);
- /* Remove all ntuple filters of the device */
- ixgbe_ntuple_filter_uninit(dev);
-
/* clear all the filters list */
ixgbe_filterlist_flush(dev);
@@ -6434,134 +6405,50 @@ ixgbe_syn_filter_program(struct ixgbe_hw *hw, uint32_t synqf)
IXGBE_WRITE_FLUSH(hw);
}
-
-static inline enum ixgbe_5tuple_protocol
-convert_protocol_type(uint8_t protocol_value)
+/* program a single ntuple (5-tuple) filter slot into hardware */
+void
+ixgbe_ntuple_filter_program(struct ixgbe_hw *hw, uint16_t index,
+ const struct ixgbe_5tuple_filter_info *filter_info,
+ uint16_t queue)
{
- if (protocol_value == IPPROTO_TCP)
- return IXGBE_FILTER_PROTOCOL_TCP;
- else if (protocol_value == IPPROTO_UDP)
- return IXGBE_FILTER_PROTOCOL_UDP;
- else if (protocol_value == IPPROTO_SCTP)
- return IXGBE_FILTER_PROTOCOL_SCTP;
- else
- return IXGBE_FILTER_PROTOCOL_NONE;
-}
-
-/* inject a 5-tuple filter to HW */
-static inline void
-ixgbe_inject_5tuple_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_5tuple_filter *filter)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
- int i;
uint32_t ftqf, sdpqf;
uint32_t l34timir = 0;
uint8_t mask = 0xff;
- i = filter->index;
+ sdpqf = (uint32_t)(filter_info->dst_port << IXGBE_SDPQF_DSTPORT_SHIFT);
+ sdpqf = sdpqf | (filter_info->src_port & IXGBE_SDPQF_SRCPORT);
- sdpqf = (uint32_t)(filter->filter_info.dst_port <<
- IXGBE_SDPQF_DSTPORT_SHIFT);
- sdpqf = sdpqf | (filter->filter_info.src_port & IXGBE_SDPQF_SRCPORT);
-
- ftqf = (uint32_t)(filter->filter_info.proto &
- IXGBE_FTQF_PROTOCOL_MASK);
- ftqf |= (uint32_t)((filter->filter_info.priority &
+ ftqf = (uint32_t)(filter_info->proto & IXGBE_FTQF_PROTOCOL_MASK);
+ ftqf |= (uint32_t)((filter_info->priority &
IXGBE_FTQF_PRIORITY_MASK) << IXGBE_FTQF_PRIORITY_SHIFT);
- if (filter->filter_info.src_ip_mask == 0) /* 0 means compare. */
+ if (filter_info->src_ip_mask == 0) /* 0 means compare. */
mask &= IXGBE_FTQF_SOURCE_ADDR_MASK;
- if (filter->filter_info.dst_ip_mask == 0)
+ if (filter_info->dst_ip_mask == 0)
mask &= IXGBE_FTQF_DEST_ADDR_MASK;
- if (filter->filter_info.src_port_mask == 0)
+ if (filter_info->src_port_mask == 0)
mask &= IXGBE_FTQF_SOURCE_PORT_MASK;
- if (filter->filter_info.dst_port_mask == 0)
+ if (filter_info->dst_port_mask == 0)
mask &= IXGBE_FTQF_DEST_PORT_MASK;
- if (filter->filter_info.proto_mask == 0)
+ if (filter_info->proto_mask == 0)
mask &= IXGBE_FTQF_PROTOCOL_COMP_MASK;
ftqf |= mask << IXGBE_FTQF_5TUPLE_MASK_SHIFT;
ftqf |= IXGBE_FTQF_POOL_MASK_EN;
ftqf |= IXGBE_FTQF_QUEUE_ENABLE;
- IXGBE_WRITE_REG(hw, IXGBE_DAQF(i), filter->filter_info.dst_ip);
- IXGBE_WRITE_REG(hw, IXGBE_SAQF(i), filter->filter_info.src_ip);
- IXGBE_WRITE_REG(hw, IXGBE_SDPQF(i), sdpqf);
- IXGBE_WRITE_REG(hw, IXGBE_FTQF(i), ftqf);
+ IXGBE_WRITE_REG(hw, IXGBE_DAQF(index), filter_info->dst_ip);
+ IXGBE_WRITE_REG(hw, IXGBE_SAQF(index), filter_info->src_ip);
+ IXGBE_WRITE_REG(hw, IXGBE_SDPQF(index), sdpqf);
+ IXGBE_WRITE_REG(hw, IXGBE_FTQF(index), ftqf);
l34timir |= IXGBE_L34T_IMIR_RESERVE;
- l34timir |= (uint32_t)(filter->queue <<
- IXGBE_L34T_IMIR_QUEUE_SHIFT);
- IXGBE_WRITE_REG(hw, IXGBE_L34T_IMIR(i), l34timir);
+ l34timir |= (uint32_t)(queue << IXGBE_L34T_IMIR_QUEUE_SHIFT);
+ IXGBE_WRITE_REG(hw, IXGBE_L34T_IMIR(index), l34timir);
}
-/*
- * add a 5tuple filter
- *
- * @param
- * dev: Pointer to struct rte_eth_dev.
- * index: the index the filter allocates.
- * filter: pointer to the filter that will be added.
- * rx_queue: the queue id the filter assigned to.
- *
- * @return
- * - On success, zero.
- * - On failure, a negative value.
- */
-static int
-ixgbe_add_5tuple_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_5tuple_filter *filter)
+/* clear a single ntuple (5-tuple) filter slot from hardware */
+void
+ixgbe_ntuple_filter_clear(struct ixgbe_hw *hw, uint16_t index)
{
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
- int i, idx, shift;
-
- /*
- * look for an unused 5tuple filter index,
- * and insert the filter to list.
- */
- for (i = 0; i < IXGBE_MAX_FTQF_FILTERS; i++) {
- idx = i / (sizeof(uint32_t) * NBBY);
- shift = i % (sizeof(uint32_t) * NBBY);
- if (!(filter_info->fivetuple_mask[idx] & (1 << shift))) {
- filter_info->fivetuple_mask[idx] |= 1 << shift;
- filter->index = i;
- TAILQ_INSERT_TAIL(&filter_info->fivetuple_list,
- filter,
- entries);
- break;
- }
- }
- if (i >= IXGBE_MAX_FTQF_FILTERS) {
- PMD_DRV_LOG(ERR, "5tuple filters are full.");
- return -ENOSYS;
- }
-
- ixgbe_inject_5tuple_filter(adapter, filter);
-
- return 0;
-}
-
-/*
- * remove a 5tuple filter
- *
- * @param
- * dev: Pointer to struct rte_eth_dev.
- * filter: the pointer of the filter will be removed.
- */
-static void
-ixgbe_remove_5tuple_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_5tuple_filter *filter)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
- uint16_t index = filter->index;
-
- filter_info->fivetuple_mask[index / (sizeof(uint32_t) * NBBY)] &=
- ~(1 << (index % (sizeof(uint32_t) * NBBY)));
- TAILQ_REMOVE(&filter_info->fivetuple_list, filter, entries);
- rte_free(filter);
-
IXGBE_WRITE_REG(hw, IXGBE_DAQF(index), 0);
IXGBE_WRITE_REG(hw, IXGBE_SAQF(index), 0);
IXGBE_WRITE_REG(hw, IXGBE_SDPQF(index), 0);
@@ -6614,165 +6501,6 @@ ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
return 0;
}
-static inline struct ixgbe_5tuple_filter *
-ixgbe_5tuple_filter_lookup(struct ixgbe_5tuple_filter_list *filter_list,
- struct ixgbe_5tuple_filter_info *key)
-{
- struct ixgbe_5tuple_filter *it;
-
- TAILQ_FOREACH(it, filter_list, entries) {
- if (memcmp(key, &it->filter_info,
- sizeof(struct ixgbe_5tuple_filter_info)) == 0) {
- return it;
- }
- }
- return NULL;
-}
-
-/* translate elements in struct rte_eth_ntuple_filter to struct ixgbe_5tuple_filter_info*/
-static inline int
-ntuple_filter_to_5tuple(struct rte_eth_ntuple_filter *filter,
- struct ixgbe_5tuple_filter_info *filter_info)
-{
- if (filter->queue >= IXGBE_MAX_RX_QUEUE_NUM ||
- filter->priority > IXGBE_5TUPLE_MAX_PRI ||
- filter->priority < IXGBE_5TUPLE_MIN_PRI)
- return -EINVAL;
-
- switch (filter->dst_ip_mask) {
- case UINT32_MAX:
- filter_info->dst_ip_mask = 0;
- filter_info->dst_ip = filter->dst_ip;
- break;
- case 0:
- filter_info->dst_ip_mask = 1;
- break;
- default:
- PMD_DRV_LOG(ERR, "invalid dst_ip mask.");
- return -EINVAL;
- }
-
- switch (filter->src_ip_mask) {
- case UINT32_MAX:
- filter_info->src_ip_mask = 0;
- filter_info->src_ip = filter->src_ip;
- break;
- case 0:
- filter_info->src_ip_mask = 1;
- break;
- default:
- PMD_DRV_LOG(ERR, "invalid src_ip mask.");
- return -EINVAL;
- }
-
- switch (filter->dst_port_mask) {
- case UINT16_MAX:
- filter_info->dst_port_mask = 0;
- filter_info->dst_port = filter->dst_port;
- break;
- case 0:
- filter_info->dst_port_mask = 1;
- break;
- default:
- PMD_DRV_LOG(ERR, "invalid dst_port mask.");
- return -EINVAL;
- }
-
- switch (filter->src_port_mask) {
- case UINT16_MAX:
- filter_info->src_port_mask = 0;
- filter_info->src_port = filter->src_port;
- break;
- case 0:
- filter_info->src_port_mask = 1;
- break;
- default:
- PMD_DRV_LOG(ERR, "invalid src_port mask.");
- return -EINVAL;
- }
-
- switch (filter->proto_mask) {
- case UINT8_MAX:
- filter_info->proto_mask = 0;
- filter_info->proto =
- convert_protocol_type(filter->proto);
- break;
- case 0:
- filter_info->proto_mask = 1;
- break;
- default:
- PMD_DRV_LOG(ERR, "invalid protocol mask.");
- return -EINVAL;
- }
-
- filter_info->priority = (uint8_t)filter->priority;
- return 0;
-}
-
-/*
- * add or delete a ntuple filter
- *
- * @param
- * dev: Pointer to struct rte_eth_dev.
- * ntuple_filter: Pointer to struct rte_eth_ntuple_filter
- * add: if true, add filter, if false, remove filter
- *
- * @return
- * - On success, zero.
- * - On failure, a negative value.
- */
-int
-ixgbe_add_del_ntuple_filter(struct ixgbe_adapter *adapter,
- struct rte_eth_ntuple_filter *ntuple_filter,
- bool add)
-{
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
- struct ixgbe_5tuple_filter_info filter_5tuple;
- struct ixgbe_5tuple_filter *filter;
- int ret;
-
- if (ntuple_filter->flags != RTE_5TUPLE_FLAGS) {
- PMD_DRV_LOG(ERR, "only 5tuple is supported.");
- return -EINVAL;
- }
-
- memset(&filter_5tuple, 0, sizeof(struct ixgbe_5tuple_filter_info));
- ret = ntuple_filter_to_5tuple(ntuple_filter, &filter_5tuple);
- if (ret < 0)
- return ret;
-
- filter = ixgbe_5tuple_filter_lookup(&filter_info->fivetuple_list,
- &filter_5tuple);
- if (filter != NULL && add) {
- PMD_DRV_LOG(ERR, "filter exists.");
- return -EEXIST;
- }
- if (filter == NULL && !add) {
- PMD_DRV_LOG(ERR, "filter doesn't exist.");
- return -ENOENT;
- }
-
- if (add) {
- filter = rte_zmalloc("ixgbe_5tuple_filter",
- sizeof(struct ixgbe_5tuple_filter), 0);
- if (filter == NULL)
- return -ENOMEM;
- memcpy(&filter->filter_info,
- &filter_5tuple,
- sizeof(struct ixgbe_5tuple_filter_info));
- filter->queue = ntuple_filter->queue;
- ret = ixgbe_add_5tuple_filter(adapter, filter);
- if (ret < 0) {
- rte_free(filter);
- return ret;
- }
- } else
- ixgbe_remove_5tuple_filter(adapter, filter);
-
- return 0;
-}
-
void
ixgbe_ethertype_filter_program(struct ixgbe_hw *hw, uint8_t idx,
uint32_t etqf, uint32_t etqs)
@@ -8116,21 +7844,6 @@ int ixgbe_enable_sec_tx_path_generic(struct ixgbe_hw *hw)
return IXGBE_SUCCESS;
}
-/* restore n-tuple filter */
-static inline void
-ixgbe_ntuple_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
- struct ixgbe_5tuple_filter *node;
-
- TAILQ_FOREACH(node, &filter_info->fivetuple_list, entries) {
- ixgbe_inject_5tuple_filter(adapter, node);
- }
-}
-
/* restore rss filter */
static inline void
ixgbe_rss_filter_restore(struct rte_eth_dev *dev)
@@ -8148,7 +7861,6 @@ ixgbe_rss_filter_restore(struct rte_eth_dev *dev)
static int
ixgbe_filter_restore(struct rte_eth_dev *dev)
{
- ixgbe_ntuple_filter_restore(dev);
ixgbe_fdir_filter_restore(dev);
ixgbe_rss_filter_restore(dev);
@@ -8171,20 +7883,6 @@ ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev)
(void)ixgbe_update_e_tag_eth_type(hw, l2_tn_info->e_tag_ether_type);
}
-/* remove all the n-tuple filters */
-void
-ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
- struct ixgbe_5tuple_filter *p_5tuple;
-
- while ((p_5tuple = TAILQ_FIRST(&filter_info->fivetuple_list)))
- ixgbe_remove_5tuple_filter(adapter, p_5tuple);
-}
-
void
ixgbe_dev_macsec_setting_save(struct rte_eth_dev *dev,
struct ixgbe_macsec_setting *macsec_setting)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 7713932236..11e441fbda 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -99,8 +99,6 @@
#define IXGBE_L34T_IMIR_LLI 0x00100000
#define IXGBE_L34T_IMIR_QUEUE 0x0FE00000
#define IXGBE_L34T_IMIR_QUEUE_SHIFT 21
-#define IXGBE_5TUPLE_MAX_PRI 7
-#define IXGBE_5TUPLE_MIN_PRI 1
/* The overhead from MTU to max frame size. */
#define IXGBE_ETH_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN)
@@ -269,8 +267,6 @@ enum ixgbe_5tuple_protocol {
IXGBE_FILTER_PROTOCOL_NONE,
};
-TAILQ_HEAD(ixgbe_5tuple_filter_list, ixgbe_5tuple_filter);
-
struct ixgbe_5tuple_filter_info {
uint32_t dst_ip;
uint32_t src_ip;
@@ -286,18 +282,6 @@ struct ixgbe_5tuple_filter_info {
proto_mask:1; /* if mask is 1b, do not compare protocol. */
};
-/* 5tuple filter structure */
-struct ixgbe_5tuple_filter {
- TAILQ_ENTRY(ixgbe_5tuple_filter) entries;
- uint16_t index; /* the index of 5tuple filter */
- struct ixgbe_5tuple_filter_info filter_info;
- uint16_t queue; /* rx queue assigned to */
-};
-
-#define IXGBE_5TUPLE_ARRAY_SIZE \
- (RTE_ALIGN(IXGBE_MAX_FTQF_FILTERS, (sizeof(uint32_t) * NBBY)) / \
- (sizeof(uint32_t) * NBBY))
-
/* Shared EtherType (ETQF) filter table. */
struct ixgbe_ethertype_table {
uint32_t mask; /* bitmask of used ETQF slots */
@@ -312,9 +296,6 @@ struct ixgbe_ethertype_table {
* Structure to store filters' info.
*/
struct ixgbe_filter_info {
- /* Bit mask for every used 5tuple filter */
- uint32_t fivetuple_mask[IXGBE_5TUPLE_ARRAY_SIZE];
- struct ixgbe_5tuple_filter_list fivetuple_list;
/* store the rss filter info */
struct ixgbe_rte_flow_rss_conf rss_info;
/* shared EtherType (ETQF) slot table */
@@ -663,9 +644,10 @@ uint32_t ixgbe_rssrk_reg_get(enum ixgbe_mac_type mac_type, uint8_t i);
bool ixgbe_rss_update_sp(enum ixgbe_mac_type mac_type);
-int ixgbe_add_del_ntuple_filter(struct ixgbe_adapter *adapter,
- struct rte_eth_ntuple_filter *filter,
- bool add);
+void ixgbe_ntuple_filter_program(struct ixgbe_hw *hw, uint16_t index,
+ const struct ixgbe_5tuple_filter_info *filter_info,
+ uint16_t queue);
+void ixgbe_ntuple_filter_clear(struct ixgbe_hw *hw, uint16_t index);
void ixgbe_ethertype_filter_program(struct ixgbe_hw *hw, uint8_t idx,
uint32_t etqf, uint32_t etqs);
@@ -743,8 +725,6 @@ int ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev);
extern const struct rte_flow_ops ixgbe_flow_ops;
-void ixgbe_clear_all_ntuple_filter(struct rte_eth_dev *dev);
-
int ixgbe_disable_sec_tx_path_generic(struct ixgbe_hw *hw);
int ixgbe_enable_sec_tx_path_generic(struct ixgbe_hw *hw);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 46e4fd2bba..ec5998566e 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -50,19 +50,12 @@
#include "../common/flow_engine.h"
#include "ixgbe_flow.h"
-#define IXGBE_MIN_N_TUPLE_PRIO 1
-#define IXGBE_MAX_N_TUPLE_PRIO 7
#define IXGBE_MAX_FLX_SOURCE_OFF 62
struct ixgbe_filter_ele_base {
TAILQ_ENTRY(ixgbe_filter_ele_base) entries;
};
-/* ntuple filter list structure */
-struct ixgbe_ntuple_filter_ele {
- struct ixgbe_filter_ele_base base;
- struct rte_eth_ntuple_filter filter_info;
-};
/* fdir filter list structure */
struct ixgbe_fdir_rule_ele {
struct ixgbe_filter_ele_base base;
@@ -84,6 +77,7 @@ const struct ci_flow_engine_list ixgbe_flow_engine_list = {
&ixgbe_ethertype_flow_engine,
&ixgbe_syn_flow_engine,
&ixgbe_l2_tunnel_flow_engine,
+ &ixgbe_ntuple_flow_engine,
},
};
@@ -161,355 +155,6 @@ ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
* normally the packets should use network order.
*/
-/**
- * Parse the rule to see if it is a n-tuple rule.
- * And get the n-tuple filter info BTW.
- * pattern:
- * The first not void item can be ETH or IPV4.
- * The second not void item must be IPV4 if the first one is ETH.
- * The third not void item must be UDP or TCP.
- * The next not void item must be END.
- * action:
- * The first not void action should be QUEUE.
- * The next not void action should be END.
- * pattern example:
- * ITEM Spec Mask
- * ETH NULL NULL
- * IPV4 src_addr 192.168.1.20 0xFFFFFFFF
- * dst_addr 192.167.3.50 0xFFFFFFFF
- * next_proto_id 17 0xFF
- * UDP/TCP/ src_port 80 0xFFFF
- * SCTP dst_port 80 0xFFFF
- * END
- * other members in mask and spec should set to 0x00.
- * item->last should be NULL.
- *
- * Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY.
- *
- */
-static int
-cons_parse_ntuple_filter(const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action_queue *q_act,
- struct rte_eth_ntuple_filter *filter,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_ipv4 *ipv4_spec;
- const struct rte_flow_item_ipv4 *ipv4_mask;
- const struct rte_flow_item_tcp *tcp_spec;
- const struct rte_flow_item_tcp *tcp_mask;
- const struct rte_flow_item_udp *udp_spec;
- const struct rte_flow_item_udp *udp_mask;
- const struct rte_flow_item_sctp *sctp_spec;
- const struct rte_flow_item_sctp *sctp_mask;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- const struct rte_flow_item_vlan *vlan_spec;
- const struct rte_flow_item_vlan *vlan_mask;
- struct rte_flow_item_eth eth_null;
- struct rte_flow_item_vlan vlan_null;
-
- /* Priority must be 16-bit */
- if (attr->priority > UINT16_MAX) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, attr,
- "Priority must be 16-bit");
- }
-
- memset(ð_null, 0, sizeof(struct rte_flow_item_eth));
- memset(&vlan_null, 0, sizeof(struct rte_flow_item_vlan));
-
- /* the first not void item can be MAC or IPv4 */
- item = next_no_void_pattern(pattern, NULL);
-
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV4) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- /* Skip Ethernet */
- if (item->type == RTE_FLOW_ITEM_TYPE_ETH) {
- eth_spec = item->spec;
- eth_mask = item->mask;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error,
- EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
-
- }
- /* if the first item is MAC, the content should be NULL */
- if ((item->spec != NULL && memcmp(eth_spec, ð_null, sizeof(eth_null)) != 0) ||
- (item->mask != NULL && memcmp(eth_mask, ð_null, sizeof(eth_null)) != 0)) {
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
- "Not supported by ntuple filter");
- return -rte_errno;
- }
- /* check if the next not void item is IPv4 or Vlan */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_VLAN) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- }
-
- if (item->type == RTE_FLOW_ITEM_TYPE_VLAN) {
- vlan_spec = item->spec;
- vlan_mask = item->mask;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error,
- EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- /* the content should be NULL */
- if ((item->spec != NULL && memcmp(vlan_spec, &vlan_null, sizeof(vlan_null)) != 0) ||
- (item->mask != NULL && memcmp(vlan_mask, &vlan_null, sizeof(vlan_null)) != 0)) {
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
- "Not supported by ntuple filter");
- return -rte_errno;
- }
- /* check if the next not void item is IPv4 */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_IPV4) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- }
-
- if (item->mask) {
- /* get the IPv4 info */
- if (!item->spec || !item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid ntuple mask");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- ipv4_mask = item->mask;
- /**
- * Only support src & dst addresses, protocol,
- * others should be masked.
- */
- if (ipv4_mask->hdr.version_ihl ||
- ipv4_mask->hdr.type_of_service ||
- ipv4_mask->hdr.total_length ||
- ipv4_mask->hdr.packet_id ||
- ipv4_mask->hdr.fragment_offset ||
- ipv4_mask->hdr.time_to_live ||
- ipv4_mask->hdr.hdr_checksum) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- if ((ipv4_mask->hdr.src_addr != 0 &&
- ipv4_mask->hdr.src_addr != UINT32_MAX) ||
- (ipv4_mask->hdr.dst_addr != 0 &&
- ipv4_mask->hdr.dst_addr != UINT32_MAX) ||
- (ipv4_mask->hdr.next_proto_id != UINT8_MAX &&
- ipv4_mask->hdr.next_proto_id != 0)) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- filter->dst_ip_mask = ipv4_mask->hdr.dst_addr;
- filter->src_ip_mask = ipv4_mask->hdr.src_addr;
- filter->proto_mask = ipv4_mask->hdr.next_proto_id;
-
- ipv4_spec = item->spec;
- filter->dst_ip = ipv4_spec->hdr.dst_addr;
- filter->src_ip = ipv4_spec->hdr.src_addr;
- filter->proto = ipv4_spec->hdr.next_proto_id;
- }
-
- /* check if the next not void item is TCP or UDP */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_TCP &&
- item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_SCTP &&
- item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- if ((item->type != RTE_FLOW_ITEM_TYPE_END) &&
- (!item->spec && !item->mask)) {
- goto action;
- }
-
- /* get the TCP/UDP/SCTP info */
- if (item->type != RTE_FLOW_ITEM_TYPE_END &&
- (!item->spec || !item->mask)) {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Invalid ntuple mask");
- return -rte_errno;
- }
-
- /*Not supported last point for range*/
- if (item->last) {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
-
- }
-
- if (item->type == RTE_FLOW_ITEM_TYPE_TCP) {
- tcp_mask = item->mask;
-
- /**
- * Only support src & dst ports, tcp flags,
- * others should be masked.
- */
- if (tcp_mask->hdr.sent_seq ||
- tcp_mask->hdr.recv_ack ||
- tcp_mask->hdr.data_off ||
- tcp_mask->hdr.rx_win ||
- tcp_mask->hdr.cksum ||
- tcp_mask->hdr.tcp_urp) {
- memset(filter, 0,
- sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- if ((tcp_mask->hdr.src_port != 0 &&
- tcp_mask->hdr.src_port != UINT16_MAX) ||
- (tcp_mask->hdr.dst_port != 0 &&
- tcp_mask->hdr.dst_port != UINT16_MAX)) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- filter->dst_port_mask = tcp_mask->hdr.dst_port;
- filter->src_port_mask = tcp_mask->hdr.src_port;
- if (tcp_mask->hdr.tcp_flags == 0xFF) {
- filter->flags |= RTE_NTUPLE_FLAGS_TCP_FLAG;
- } else if (!tcp_mask->hdr.tcp_flags) {
- filter->flags &= ~RTE_NTUPLE_FLAGS_TCP_FLAG;
- } else {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- tcp_spec = item->spec;
- filter->dst_port = tcp_spec->hdr.dst_port;
- filter->src_port = tcp_spec->hdr.src_port;
- filter->tcp_flags = tcp_spec->hdr.tcp_flags;
- } else if (item->type == RTE_FLOW_ITEM_TYPE_UDP) {
- udp_mask = item->mask;
-
- /**
- * Only support src & dst ports,
- * others should be masked.
- */
- if (udp_mask->hdr.dgram_len ||
- udp_mask->hdr.dgram_cksum) {
- memset(filter, 0,
- sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
- if ((udp_mask->hdr.src_port != 0 &&
- udp_mask->hdr.src_port != UINT16_MAX) ||
- (udp_mask->hdr.dst_port != 0 &&
- udp_mask->hdr.dst_port != UINT16_MAX)) {
- rte_flow_error_set(error,
- EINVAL, RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- filter->dst_port_mask = udp_mask->hdr.dst_port;
- filter->src_port_mask = udp_mask->hdr.src_port;
-
- udp_spec = item->spec;
- filter->dst_port = udp_spec->hdr.dst_port;
- filter->src_port = udp_spec->hdr.src_port;
- } else if (item->type == RTE_FLOW_ITEM_TYPE_SCTP) {
- sctp_mask = item->mask;
-
- /**
- * Only support src & dst ports,
- * others should be masked.
- */
- if (sctp_mask->hdr.tag ||
- sctp_mask->hdr.cksum) {
- memset(filter, 0,
- sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- filter->dst_port_mask = sctp_mask->hdr.dst_port;
- filter->src_port_mask = sctp_mask->hdr.src_port;
-
- sctp_spec = item->spec;
- filter->dst_port = sctp_spec->hdr.dst_port;
- filter->src_port = sctp_spec->hdr.src_port;
- } else {
- goto action;
- }
-
- /* check if the next not void item is END */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
-action:
-
- filter->queue = q_act->index;
-
- filter->priority = (uint16_t)attr->priority;
- if (attr->priority < IXGBE_MIN_N_TUPLE_PRIO || attr->priority > IXGBE_MAX_N_TUPLE_PRIO)
- filter->priority = 1;
-
- return 0;
-}
-
static int
ixgbe_parse_security_filter(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
const struct rte_flow_item pattern[], const struct rte_flow_action actions[],
@@ -598,66 +243,6 @@ ixgbe_parse_security_filter(struct rte_eth_dev *dev, const struct rte_flow_attr
return 0;
}
-/* a specific function for ixgbe because the flags is specific */
-static int
-ixgbe_parse_ntuple_filter(struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_eth_ntuple_filter *filter,
- struct rte_flow_error *error)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ci_flow_attr_check_param attr_param = {
- .allow_priority = true,
- };
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only queue is allowed here */
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_END
- },
- .driver_ctx = dev->data,
- .check = ixgbe_flow_actions_check,
- .max_actions = 1,
- };
- const struct rte_flow_action *action;
- int ret;
-
- if (hw->mac.type != ixgbe_mac_82599EB &&
- hw->mac.type != ixgbe_mac_X540)
- return -ENOTSUP;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, &attr_param, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
- action = parsed_actions.actions[0];
-
- ret = cons_parse_ntuple_filter(attr, pattern, action->conf, filter, error);
- if (ret)
- return ret;
-
- /* Ixgbe doesn't support tcp flags. */
- if (filter->flags & RTE_NTUPLE_FLAGS_TCP_FLAG) {
- memset(filter, 0, sizeof(struct rte_eth_ntuple_filter));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "Not supported by ntuple filter");
- return -rte_errno;
- }
-
- /* fixed value for ixgbe */
- filter->flags = RTE_5TUPLE_FLAGS;
- return 0;
-}
-
/* search next no void pattern and skip fuzzy */
static inline
const struct rte_flow_item *next_no_fuzzy_pattern(
@@ -2288,13 +1873,11 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
int ret;
struct ixgbe_adapter *adapter =
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct rte_eth_ntuple_filter ntuple_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_hw_fdir_info *fdir_info =
IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
struct ixgbe_rte_flow_rss_conf rss_conf;
struct rte_flow *flow = NULL;
- struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_rss_conf_ele *rss_filter_ptr;
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr;
@@ -2331,27 +1914,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
return flow;
}
- memset(&ntuple_filter, 0, sizeof(struct rte_eth_ntuple_filter));
- ret = ixgbe_parse_ntuple_filter(dev, attr, pattern,
- actions, &ntuple_filter, error);
-
- if (!ret) {
- ret = ixgbe_add_del_ntuple_filter(adapter, &ntuple_filter, TRUE);
- if (!ret) {
- ntuple_filter_ptr = rte_zmalloc("ixgbe_ntuple_filter",
- sizeof(struct ixgbe_ntuple_filter_ele), 0);
- if (!ntuple_filter_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- ntuple_filter_ptr->filter_info = ntuple_filter;
- flow->rule = ntuple_filter_ptr;
- flow->filter_type = RTE_ETH_FILTER_NTUPLE;
- return flow;
- }
- goto out;
- }
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2429,7 +1991,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct rte_eth_ntuple_filter ntuple_filter;
struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_rte_flow_rss_conf rss_conf;
int ret;
@@ -2448,12 +2009,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
if (!ret)
return 0;
- memset(&ntuple_filter, 0, sizeof(struct rte_eth_ntuple_filter));
- ret = ixgbe_parse_ntuple_filter(dev, attr, pattern,
- actions, &ntuple_filter, error);
- if (!ret)
- return 0;
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2478,9 +2033,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_flow *pmd_flow = flow;
enum rte_filter_type filter_type = pmd_flow->filter_type;
- struct rte_eth_ntuple_filter ntuple_filter;
struct ixgbe_fdir_rule fdir_rule;
- struct ixgbe_ntuple_filter_ele *ntuple_filter_ptr;
struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_filter_ele_base *flow_mem_base;
struct ixgbe_hw_fdir_info *fdir_info =
@@ -2516,14 +2069,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
}
switch (filter_type) {
- case RTE_ETH_FILTER_NTUPLE:
- ntuple_filter_ptr = (struct ixgbe_ntuple_filter_ele *)
- pmd_flow->rule;
- ntuple_filter = ntuple_filter_ptr->filter_info;
- ret = ixgbe_add_del_ntuple_filter(adapter, &ntuple_filter, FALSE);
- if (!ret)
- rte_free(ntuple_filter_ptr);
- break;
case RTE_ETH_FILTER_FDIR:
fdir_rule_ptr = (struct ixgbe_fdir_rule_ele *)pmd_flow->rule;
fdir_rule = fdir_rule_ptr->filter_info;
@@ -2583,8 +2128,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
return ret;
}
- ixgbe_clear_all_ntuple_filter(dev);
-
ret = ixgbe_clear_all_fdir_filter(dev);
if (ret < 0) {
rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE,
@@ -2608,8 +2151,6 @@ ixgbe_flow_rule_engine_name(const struct rte_flow *flow)
return "security";
switch (flow->filter_type) {
- case RTE_ETH_FILTER_NTUPLE:
- return "ntuple";
case RTE_ETH_FILTER_ETHERTYPE:
return "ethertype";
case RTE_ETH_FILTER_SYN:
@@ -2632,8 +2173,6 @@ ixgbe_flow_rule_size(const struct rte_flow *flow)
return 0;
switch (flow->filter_type) {
- case RTE_ETH_FILTER_NTUPLE:
- return sizeof(struct rte_eth_ntuple_filter);
case RTE_ETH_FILTER_ETHERTYPE:
return sizeof(struct rte_eth_ethertype_filter);
case RTE_ETH_FILTER_SYN:
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index ba0486b2c0..6f082e9402 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -18,5 +18,6 @@ extern const struct ci_flow_engine_list ixgbe_flow_engine_list;
extern const struct ci_flow_engine ixgbe_ethertype_flow_engine;
extern const struct ci_flow_engine ixgbe_syn_flow_engine;
extern const struct ci_flow_engine ixgbe_l2_tunnel_flow_engine;
+extern const struct ci_flow_engine ixgbe_ntuple_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_ntuple.c b/drivers/net/intel/ixgbe/ixgbe_flow_ntuple.c
new file mode 100644
index 0000000000..6149f354b4
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_ntuple.c
@@ -0,0 +1,597 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+#define IXGBE_MIN_N_TUPLE_PRIO 1
+#define IXGBE_MAX_N_TUPLE_PRIO 7
+
+struct ixgbe_ntuple_flow {
+ struct rte_flow flow;
+ struct ixgbe_5tuple_filter_info key; /* HW-shaped match key */
+ uint16_t queue;
+ uint16_t index; /* HW filter slot */
+};
+
+/* per-device table of registered filters, indexed by assigned HW slot */
+struct ixgbe_ntuple_priv {
+ struct ixgbe_ntuple_flow *slots[IXGBE_MAX_FTQF_FILTERS];
+};
+
+struct ixgbe_ntuple_ctx {
+ struct ci_flow_engine_ctx base;
+ struct rte_eth_ntuple_filter ntuple;
+};
+
+/**
+ * Ntuple filter graph implementation
+ * Pattern: START -> [ETH] -> [VLAN] -> IPV4 -> [TCP|UDP|SCTP] -> END
+ */
+
+enum ixgbe_ntuple_node_id {
+ IXGBE_NTUPLE_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_NTUPLE_NODE_ETH,
+ IXGBE_NTUPLE_NODE_VLAN,
+ IXGBE_NTUPLE_NODE_IPV4,
+ IXGBE_NTUPLE_NODE_TCP,
+ IXGBE_NTUPLE_NODE_UDP,
+ IXGBE_NTUPLE_NODE_SCTP,
+ IXGBE_NTUPLE_NODE_END,
+ IXGBE_NTUPLE_NODE_MAX,
+};
+
+static int
+ixgbe_validate_ntuple_ipv4(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv4 *ipv4_mask;
+
+ ipv4_mask = item->mask;
+
+ /* Only src/dst addresses and protocol supported */
+ if (ipv4_mask->hdr.version_ihl ||
+ ipv4_mask->hdr.type_of_service ||
+ ipv4_mask->hdr.total_length ||
+ ipv4_mask->hdr.packet_id ||
+ ipv4_mask->hdr.fragment_offset ||
+ ipv4_mask->hdr.time_to_live ||
+ ipv4_mask->hdr.hdr_checksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst IP and protocol supported");
+ }
+
+ /* Masks must be 0 or all-ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.src_addr) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.dst_addr) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.next_proto_id)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial masks not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_ntuple_ipv4(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_ctx *ntuple_ctx = ctx;
+ const struct rte_flow_item_ipv4 *ipv4_spec = item->spec;
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+
+ ntuple_ctx->ntuple.dst_ip = ipv4_spec->hdr.dst_addr;
+ ntuple_ctx->ntuple.src_ip = ipv4_spec->hdr.src_addr;
+ ntuple_ctx->ntuple.proto = ipv4_spec->hdr.next_proto_id;
+
+ ntuple_ctx->ntuple.dst_ip_mask = ipv4_mask->hdr.dst_addr;
+ ntuple_ctx->ntuple.src_ip_mask = ipv4_mask->hdr.src_addr;
+ ntuple_ctx->ntuple.proto_mask = ipv4_mask->hdr.next_proto_id;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_ntuple_tcp(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_tcp *tcp_mask;
+
+ tcp_mask = item->mask;
+
+ /* Only src/dst ports and tcp_flags supported */
+ if (tcp_mask->hdr.sent_seq ||
+ tcp_mask->hdr.recv_ack ||
+ tcp_mask->hdr.data_off ||
+ tcp_mask->hdr.rx_win ||
+ tcp_mask->hdr.cksum ||
+ tcp_mask->hdr.tcp_urp) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst ports and flags supported");
+ }
+
+ /* Port masks must be 0 or all-ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial port masks not supported");
+ }
+
+ /* TCP flags not supported by hardware */
+ if (!CI_FIELD_IS_ZERO(&tcp_mask->hdr.tcp_flags)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "TCP flags filtering not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_ntuple_tcp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_ctx *ntuple_ctx = ctx;
+ const struct rte_flow_item_tcp *tcp_spec = item->spec;
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+
+ ntuple_ctx->ntuple.dst_port = tcp_spec->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port = tcp_spec->hdr.src_port;
+
+ ntuple_ctx->ntuple.dst_port_mask = tcp_mask->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port_mask = tcp_mask->hdr.src_port;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_ntuple_udp(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_udp *udp_mask;
+
+ udp_mask = item->mask;
+
+ /* Only src/dst ports supported */
+ if (udp_mask->hdr.dgram_len ||
+ udp_mask->hdr.dgram_cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst ports supported");
+ }
+
+ /* Port masks must be 0 or all-ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial port masks not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_ntuple_udp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_ctx *ntuple_ctx = ctx;
+ const struct rte_flow_item_udp *udp_spec = item->spec;
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+
+ ntuple_ctx->ntuple.dst_port = udp_spec->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port = udp_spec->hdr.src_port;
+
+ ntuple_ctx->ntuple.dst_port_mask = udp_mask->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port_mask = udp_mask->hdr.src_port;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_ntuple_sctp(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_sctp *sctp_mask;
+
+ sctp_mask = item->mask;
+
+ /* Only src/dst ports supported */
+ if (sctp_mask->hdr.tag ||
+ sctp_mask->hdr.cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst ports supported");
+ }
+
+ /* Port masks must be 0 or all-ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial port masks not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_ntuple_sctp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_ctx *ntuple_ctx = ctx;
+ const struct rte_flow_item_sctp *sctp_spec = item->spec;
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+
+ ntuple_ctx->ntuple.dst_port = sctp_spec->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port = sctp_spec->hdr.src_port;
+
+ ntuple_ctx->ntuple.dst_port_mask = sctp_mask->hdr.dst_port;
+ ntuple_ctx->ntuple.src_port_mask = sctp_mask->hdr.src_port;
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_ntuple_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_NTUPLE_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_NTUPLE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_NTUPLE_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_NTUPLE_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_ntuple_ipv4,
+ .process = ixgbe_process_ntuple_ipv4,
+ },
+ [IXGBE_NTUPLE_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_ntuple_tcp,
+ .process = ixgbe_process_ntuple_tcp,
+ },
+ [IXGBE_NTUPLE_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_ntuple_udp,
+ .process = ixgbe_process_ntuple_udp,
+ },
+ [IXGBE_NTUPLE_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_ntuple_sctp,
+ .process = ixgbe_process_ntuple_sctp,
+ },
+ [IXGBE_NTUPLE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_NTUPLE_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_ETH,
+ IXGBE_NTUPLE_NODE_IPV4,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_VLAN,
+ IXGBE_NTUPLE_NODE_IPV4,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_VLAN] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_IPV4,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_IPV4] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_TCP,
+ IXGBE_NTUPLE_NODE_UDP,
+ IXGBE_NTUPLE_NODE_SCTP,
+ IXGBE_NTUPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_TCP] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_UDP] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_NTUPLE_NODE_SCTP] = {
+ .next = (size_t[]) {
+ IXGBE_NTUPLE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+ixgbe_flow_ntuple_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_ntuple_ctx *ntuple_ctx = (struct ixgbe_ntuple_ctx *)ctx;
+ struct ci_flow_attr_check_param attr_param = {
+ .allow_priority = true,
+ };
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only queue is allowed here */
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_flow_actions_check,
+ .max_actions = 1,
+ };
+ const struct rte_flow_action_queue *q_act;
+ uint16_t priority;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, &attr_param, error);
+ if (ret)
+ return ret;
+
+ /* Priority must be 16-bit */
+ if (attr->priority > UINT16_MAX) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, attr,
+ "Priority must be 16-bit");
+ }
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ q_act = (const struct rte_flow_action_queue *)parsed_actions.actions[0]->conf;
+
+ ntuple_ctx->ntuple.queue = q_act->index;
+
+ /*
+ * rte_flow priority: 0 through UINT32_MAX, 0 is highest
+ *
+ * ntuple priority: 001b through 111b, 111b is highest
+ *
+ * which means we need to transform priority from rte_flow to ntuple:
+ *
+ * 1) clamp max value
+ * 2) reverse
+ * 3) add min value
+ */
+ priority = RTE_MIN(IXGBE_MAX_N_TUPLE_PRIO - 1, (uint16_t)attr->priority);
+ priority = IXGBE_MAX_N_TUPLE_PRIO - 1 - priority;
+ priority += IXGBE_MIN_N_TUPLE_PRIO;
+ ntuple_ctx->ntuple.priority = priority;
+
+ /* fixed value for ixgbe */
+ ntuple_ctx->ntuple.flags = RTE_5TUPLE_FLAGS;
+
+ return 0;
+}
+
+static enum ixgbe_5tuple_protocol
+convert_protocol_type(uint8_t protocol_value)
+{
+ if (protocol_value == IPPROTO_TCP)
+ return IXGBE_FILTER_PROTOCOL_TCP;
+ else if (protocol_value == IPPROTO_UDP)
+ return IXGBE_FILTER_PROTOCOL_UDP;
+ else if (protocol_value == IPPROTO_SCTP)
+ return IXGBE_FILTER_PROTOCOL_SCTP;
+ else
+ return IXGBE_FILTER_PROTOCOL_NONE;
+}
+
+static int
+ixgbe_flow_ntuple_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_ntuple_ctx *ntuple_ctx = (const struct ixgbe_ntuple_ctx *)ctx;
+ struct ixgbe_ntuple_flow *ntuple_flow = (struct ixgbe_ntuple_flow *)flow;
+ const struct rte_eth_ntuple_filter *ntuple = &ntuple_ctx->ntuple;
+ struct ixgbe_5tuple_filter_info *key = &ntuple_flow->key;
+
+ /* mask shape (0 or all-ones) is already guaranteed by the graph */
+ memset(key, 0, sizeof(*key));
+
+ key->dst_ip_mask = ntuple->dst_ip_mask == 0;
+ key->dst_ip = ntuple->dst_ip;
+ key->src_ip_mask = ntuple->src_ip_mask == 0;
+ key->src_ip = ntuple->src_ip;
+ key->dst_port_mask = ntuple->dst_port_mask == 0;
+ key->dst_port = ntuple->dst_port;
+ key->src_port_mask = ntuple->src_port_mask == 0;
+ key->src_port = ntuple->src_port;
+ key->proto_mask = ntuple->proto_mask == 0;
+ key->proto = convert_protocol_type(ntuple->proto);
+ key->priority = (uint8_t)ntuple->priority;
+
+ ntuple_flow->queue = ntuple->queue;
+
+ return 0;
+}
+
+/* two ntuple filters are the same rule if they match the same 5-tuple key */
+static bool
+ixgbe_ntuple_key_equal(const struct ixgbe_5tuple_filter_info *lhs,
+ const struct ixgbe_5tuple_filter_info *rhs)
+{
+ return memcmp(lhs, rhs, sizeof(*lhs)) == 0;
+}
+
+static int
+ixgbe_ntuple_slot_find(const struct ixgbe_ntuple_priv *priv,
+ const struct ixgbe_ntuple_flow *ntuple_flow)
+{
+ uint16_t free_idx = IXGBE_MAX_FTQF_FILTERS;
+ uint16_t idx;
+
+ for (idx = 0; idx < IXGBE_MAX_FTQF_FILTERS; idx++) {
+ const struct ixgbe_ntuple_flow *registered = priv->slots[idx];
+
+ if (registered == NULL) {
+ if (free_idx == IXGBE_MAX_FTQF_FILTERS)
+ free_idx = idx;
+ continue;
+ }
+ if (ixgbe_ntuple_key_equal(®istered->key, &ntuple_flow->key))
+ return -EEXIST;
+ }
+ if (free_idx == IXGBE_MAX_FTQF_FILTERS)
+ return -ENOSPC;
+
+ return (int)free_idx;
+}
+
+static int
+ixgbe_flow_ntuple_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_ntuple_flow *ntuple_flow = (struct ixgbe_ntuple_flow *)flow;
+ struct ixgbe_ntuple_priv *priv = flow->engine_priv;
+ int idx;
+
+ idx = ixgbe_ntuple_slot_find(priv, ntuple_flow);
+ if (idx == -EEXIST) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ntuple filter already exists");
+ }
+ if (idx == -ENOSPC) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ntuple filter table is full");
+ }
+
+ priv->slots[idx] = ntuple_flow;
+ ntuple_flow->index = idx;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ntuple_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_ntuple_flow *ntuple_flow = (struct ixgbe_ntuple_flow *)flow;
+ struct ixgbe_ntuple_priv *priv = flow->engine_priv;
+
+ if (ntuple_flow->index >= IXGBE_MAX_FTQF_FILTERS ||
+ priv->slots[ntuple_flow->index] != ntuple_flow) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ntuple filter slot does not match on unregister");
+ }
+
+ priv->slots[ntuple_flow->index] = NULL;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ntuple_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_flow *ntuple_flow = (struct ixgbe_ntuple_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_ntuple_filter_program(hw, ntuple_flow->index, &ntuple_flow->key,
+ ntuple_flow->queue);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ntuple_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_ntuple_flow *ntuple_flow = (struct ixgbe_ntuple_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_ntuple_filter_clear(hw, ntuple_flow->index);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_ntuple_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ /* only 82599 and X540 have L3/L4 5-tuple (ntuple) filters */
+ if (hw->mac.type == ixgbe_mac_82599EB ||
+ hw->mac.type == ixgbe_mac_X540)
+ return 0;
+
+ return -ENOTSUP;
+}
+
+static const struct ci_flow_engine_ops ixgbe_ntuple_ops = {
+ .engine_init = ixgbe_flow_ntuple_engine_init,
+ .ctx_init = ixgbe_flow_ntuple_ctx_init,
+ .ctx_to_flow = ixgbe_flow_ntuple_ctx_to_flow,
+ .flow_register = ixgbe_flow_ntuple_flow_register,
+ .flow_unregister = ixgbe_flow_ntuple_flow_unregister,
+ .flow_install = ixgbe_flow_ntuple_flow_install,
+ .flow_uninstall = ixgbe_flow_ntuple_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_ntuple_flow_engine = {
+ .name = "ntuple",
+ .ctx_size = sizeof(struct ixgbe_ntuple_ctx),
+ .flow_size = sizeof(struct ixgbe_ntuple_flow),
+ .priv_size = sizeof(struct ixgbe_ntuple_priv),
+ .ops = &ixgbe_ntuple_ops,
+ .graph = &ixgbe_ntuple_graph,
+};
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 91ec261154..90cc88f002 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -29,6 +29,7 @@ sources += files(
'ixgbe_flow_ethertype.c',
'ixgbe_flow_syn.c',
'ixgbe_flow_l2tun.c',
+ 'ixgbe_flow_ntuple.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 09/19] net/ixgbe: reimplement security parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (7 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 08/19] net/ixgbe: reimplement ntuple parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 10/19] net/ixgbe: reimplement FDIR parser Anatoly Burakov
` (10 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and common flow engine infrastructure to
implement flow parser for security filter. As a result, flow item checks
have become more stringent:
- Mask is now explicitly validated to not have unsupported items in it,
when previously they were ignored
- Mask is also validated to mask src/dst addresses, as otherwise it is
inconsistent with rte_flow API
Previously, security parser was a special case, now it is a first class
citizen. All decryption SA tracking has been moved into the engine, as the
engine is now the authoritative source of new Rx flows and the state of
the Rx SA table (the Tx SA table is still up to IPsec code to manage).
Because IPsec code does not manage the Rx SA table now, a synchronization
mechanism is needed to prevent IPsec code from deallocating a security
session when it is still referenced by security flows, so the security
session is now atomically refcounted. For Tx, the refcount is effectively
a noop, whereas for Rx it is now managed by rte_flow security engine.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 2 -
drivers/net/intel/ixgbe/ixgbe_flow.c | 120 +---
drivers/net/intel/ixgbe/ixgbe_flow.h | 1 +
drivers/net/intel/ixgbe/ixgbe_flow_security.c | 537 ++++++++++++++++++
drivers/net/intel/ixgbe/ixgbe_ipsec.c | 359 ++++++------
drivers/net/intel/ixgbe/ixgbe_ipsec.h | 50 +-
drivers/net/intel/ixgbe/meson.build | 1 +
7 files changed, 723 insertions(+), 347 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_security.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 11e441fbda..bd66d4afd9 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -317,8 +317,6 @@ struct ixgbe_l2_tn_info {
struct rte_flow {
struct ci_flow flow;
enum rte_filter_type filter_type;
- /* security flows are not rte_filter_type */
- bool is_security;
void *rule;
};
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index ec5998566e..4c268ac7f1 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -78,6 +78,7 @@ const struct ci_flow_engine_list ixgbe_flow_engine_list = {
&ixgbe_syn_flow_engine,
&ixgbe_l2_tunnel_flow_engine,
&ixgbe_ntuple_flow_engine,
+ &ixgbe_security_flow_engine,
},
};
@@ -155,94 +156,6 @@ ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
* normally the packets should use network order.
*/
-static int
-ixgbe_parse_security_filter(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[], const struct rte_flow_action actions[],
- struct rte_flow_error *error)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- const struct rte_flow_action_security *security;
- struct rte_security_session *session;
- const struct rte_flow_item *item;
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only security is allowed here */
- RTE_FLOW_ACTION_TYPE_SECURITY,
- RTE_FLOW_ACTION_TYPE_END
- },
- .max_actions = 1,
- };
- const struct rte_flow_action *action;
- struct ip_spec spec;
- int ret;
-
- if (hw->mac.type != ixgbe_mac_82599EB &&
- hw->mac.type != ixgbe_mac_X540 &&
- hw->mac.type != ixgbe_mac_X550 &&
- hw->mac.type != ixgbe_mac_X550EM_x &&
- hw->mac.type != ixgbe_mac_X550EM_a &&
- hw->mac.type != ixgbe_mac_E610)
- return -ENOTSUP;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
-
- action = parsed_actions.actions[0];
- security = action->conf;
-
- /* get the IP pattern*/
- item = next_no_void_pattern(pattern, NULL);
- while (item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6) {
- if (item->last || item->type == RTE_FLOW_ITEM_TYPE_END) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "IP pattern missing.");
- return -rte_errno;
- }
- item = next_no_void_pattern(pattern, item);
- }
- if (item->spec == NULL) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM_SPEC, item,
- "NULL IP pattern.");
- return -rte_errno;
- }
- spec.is_ipv6 = item->type == RTE_FLOW_ITEM_TYPE_IPV6;
- if (spec.is_ipv6) {
- const struct rte_flow_item_ipv6 *ipv6 = item->spec;
- spec.spec.ipv6 = *ipv6;
- } else {
- const struct rte_flow_item_ipv4 *ipv4 = item->spec;
- spec.spec.ipv4 = *ipv4;
- }
-
- /*
- * we get pointer to security session from security action, which is
- * const. however, we do need to act on the session, so either we do
- * some kind of pointer based lookup to get session pointer internally
- * (which quickly gets unwieldy for lots of flows case), or we simply
- * cast away constness. the latter path was chosen.
- */
- session = RTE_CAST_PTR(struct rte_security_session *, security->security_session);
- ret = ixgbe_crypto_add_ingress_sa_from_flow(session, &spec);
- if (ret) {
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "Failed to add security session.");
- return -rte_errno;
- }
- return 0;
-}
-
/* search next no void pattern and skip fuzzy */
static inline
const struct rte_flow_item *next_no_fuzzy_pattern(
@@ -1905,15 +1818,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
TAILQ_INSERT_TAIL(&adapter->flow_list,
&ixgbe_flow_mem_ptr->base, entries);
- /**
- * Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY
- */
- ret = ixgbe_parse_security_filter(dev, attr, pattern, actions, error);
- if (!ret) {
- flow->is_security = true;
- return flow;
- }
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2002,13 +1906,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
/* fall back to legacy engines */
- /**
- * Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY
- */
- ret = ixgbe_parse_security_filter(dev, attr, pattern, actions, error);
- if (!ret)
- return 0;
-
memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
actions, &fdir_rule, error);
@@ -2062,12 +1959,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
"Flow not found for this port");
}
- /* Special case for SECURITY flows */
- if (flow->is_security) {
- ret = 0;
- goto free;
- }
-
switch (filter_type) {
case RTE_ETH_FILTER_FDIR:
fdir_rule_ptr = (struct ixgbe_fdir_rule_ele *)pmd_flow->rule;
@@ -2105,7 +1996,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
return ret;
}
-free:
TAILQ_REMOVE(&adapter->flow_list, flow_mem_base, entries);
rte_free(flow_mem_base);
rte_free(flow);
@@ -2147,9 +2037,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
static const char *
ixgbe_flow_rule_engine_name(const struct rte_flow *flow)
{
- if (flow->is_security)
- return "security";
-
switch (flow->filter_type) {
case RTE_ETH_FILTER_ETHERTYPE:
return "ethertype";
@@ -2169,9 +2056,6 @@ ixgbe_flow_rule_engine_name(const struct rte_flow *flow)
static size_t
ixgbe_flow_rule_size(const struct rte_flow *flow)
{
- if (flow->is_security)
- return 0;
-
switch (flow->filter_type) {
case RTE_ETH_FILTER_ETHERTYPE:
return sizeof(struct rte_eth_ethertype_filter);
@@ -2191,7 +2075,7 @@ ixgbe_flow_rule_size(const struct rte_flow *flow)
static const void *
ixgbe_flow_rule_data(const struct rte_flow *flow)
{
- if (flow->is_security || flow->rule == NULL)
+ if (flow->rule == NULL)
return NULL;
return RTE_PTR_ADD(flow->rule, sizeof(struct ixgbe_filter_ele_base));
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index 6f082e9402..87cf028245 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -19,5 +19,6 @@ extern const struct ci_flow_engine ixgbe_ethertype_flow_engine;
extern const struct ci_flow_engine ixgbe_syn_flow_engine;
extern const struct ci_flow_engine ixgbe_l2_tunnel_flow_engine;
extern const struct ci_flow_engine ixgbe_ntuple_flow_engine;
+extern const struct ci_flow_engine ixgbe_security_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_security.c b/drivers/net/intel/ixgbe/ixgbe_flow_security.c
new file mode 100644
index 0000000000..db66ef35da
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_security.c
@@ -0,0 +1,537 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+#include <rte_security_driver.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_security_ip_spec {
+ bool is_ipv6;
+ union {
+ struct rte_flow_item_ipv4 ipv4;
+ struct rte_flow_item_ipv6 ipv6;
+ } spec;
+};
+
+struct ixgbe_security_filter {
+ struct rte_security_session *session;
+ struct ixgbe_crypto_rx_sa sa;
+};
+
+struct ixgbe_security_flow {
+ struct rte_flow flow;
+ struct ixgbe_security_filter security;
+};
+
+struct ixgbe_security_ctx {
+ struct ci_flow_engine_ctx base;
+ struct ixgbe_security_ip_spec spec;
+ struct rte_security_session *session;
+};
+
+struct ixgbe_security_ip_slot {
+ struct ipaddr ip;
+ uint16_t ref_count;
+};
+
+struct ixgbe_security_priv {
+ struct ixgbe_security_ip_slot ip_slots[IPSEC_MAX_RX_IP_COUNT];
+ struct ixgbe_security_flow *sa_slots[IPSEC_MAX_SA_COUNT];
+};
+
+/**
+ * Ntuple security filter graph implementation
+ * Pattern: START -> IPV4 | IPV6 -> END
+ */
+
+enum ixgbe_security_node_id {
+ IXGBE_SECURITY_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_SECURITY_NODE_IPV4,
+ IXGBE_SECURITY_NODE_IPV6,
+ IXGBE_SECURITY_NODE_END,
+ IXGBE_SECURITY_NODE_MAX,
+};
+
+static int
+ixgbe_validate_security_ipv4(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+
+ /* only src/dst addresses are supported */
+ if (ipv4_mask->hdr.version_ihl ||
+ ipv4_mask->hdr.type_of_service ||
+ ipv4_mask->hdr.total_length ||
+ ipv4_mask->hdr.packet_id ||
+ ipv4_mask->hdr.fragment_offset ||
+ ipv4_mask->hdr.next_proto_id ||
+ ipv4_mask->hdr.time_to_live ||
+ ipv4_mask->hdr.hdr_checksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 mask");
+ }
+
+ /* both src/dst addresses must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&ipv4_mask->hdr.src_addr) ||
+ !CI_FIELD_IS_MASKED(&ipv4_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 mask");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_security_ipv4(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_security_ctx *sec_ctx = (struct ixgbe_security_ctx *)ctx;
+ const struct rte_flow_item_ipv4 *ipv4_spec = item->spec;
+
+ /* copy entire spec */
+ sec_ctx->spec.spec.ipv4 = *ipv4_spec;
+ sec_ctx->spec.is_ipv6 = false;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_security_ipv6(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv6 *ipv6_mask = item->mask;
+
+ /* only src/dst addresses are supported */
+ if (ipv6_mask->hdr.vtc_flow ||
+ ipv6_mask->hdr.payload_len ||
+ ipv6_mask->hdr.proto ||
+ ipv6_mask->hdr.hop_limits) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 mask");
+ }
+ /* both src/dst addresses must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&ipv6_mask->hdr.src_addr) ||
+ !CI_FIELD_IS_MASKED(&ipv6_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 mask");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_security_ipv6(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_security_ctx *sec_ctx = (struct ixgbe_security_ctx *)ctx;
+ const struct rte_flow_item_ipv6 *ipv6_spec = item->spec;
+
+ /* copy entire spec */
+ sec_ctx->spec.spec.ipv6 = *ipv6_spec;
+ sec_ctx->spec.is_ipv6 = true;
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_security_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_SECURITY_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_SECURITY_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_security_ipv4,
+ .process = ixgbe_process_security_ipv4,
+ },
+ [IXGBE_SECURITY_NODE_IPV6] = {
+ .name = "IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_security_ipv6,
+ .process = ixgbe_process_security_ipv6,
+ },
+ [IXGBE_SECURITY_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_SECURITY_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_SECURITY_NODE_IPV4,
+ IXGBE_SECURITY_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SECURITY_NODE_IPV4] = {
+ .next = (size_t[]) {
+ IXGBE_SECURITY_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_SECURITY_NODE_IPV6] = {
+ .next = (size_t[]) {
+ IXGBE_SECURITY_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+ixgbe_flow_security_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_security_ctx *sec_ctx = (struct ixgbe_security_ctx *)ctx;
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only security is allowed here */
+ RTE_FLOW_ACTION_TYPE_SECURITY,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ };
+ const struct rte_flow_action_security *security;
+ struct rte_security_session *session;
+ const struct ixgbe_crypto_session *ic_session;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ security = (const struct rte_flow_action_security *)parsed_actions.actions[0]->conf;
+
+ if (security->security_session == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, parsed_actions.actions[0],
+ "NULL security session");
+ }
+
+ /* cast away constness since we need to store the session pointer in the context */
+ session = RTE_CAST_PTR(struct rte_security_session *, security->security_session);
+
+ /* verify that the session is of a correct type */
+ ic_session = SECURITY_GET_SESS_PRIV(session);
+ if (ic_session->dev_data != ctx->dev_data) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, parsed_actions.actions[0],
+ "Security session was created for a different device");
+ }
+ if (ic_session->op != IXGBE_OP_AUTHENTICATED_DECRYPTION) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, parsed_actions.actions[0],
+ "Only authenticated decryption is supported");
+ }
+ sec_ctx->session = session;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_security_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_security_ctx *security_ctx = (const struct ixgbe_security_ctx *)ctx;
+ struct ixgbe_security_flow *security_flow = (struct ixgbe_security_flow *)flow;
+ struct ixgbe_security_filter *filter = &security_flow->security;
+ const struct ixgbe_crypto_session *ic_session;
+
+ filter->session = security_ctx->session;
+ ic_session = SECURITY_GET_SESS_PRIV(filter->session);
+ filter->sa.key = ic_session->key;
+ filter->sa.salt = ic_session->salt;
+ filter->sa.spi = ic_session->spi;
+ filter->sa.mode = IPSRXMOD_VALID | IPSRXMOD_PROTO | IPSRXMOD_DECRYPT;
+ /* slot indices stay out of range until the flow is registered */
+ filter->sa.sa_idx = IPSEC_MAX_SA_COUNT;
+ filter->sa.ip_idx = IPSEC_MAX_RX_IP_COUNT;
+ if (security_ctx->spec.is_ipv6) {
+ filter->sa.dst_ip.type = IPv6;
+ filter->sa.mode |= IPSRXMOD_IPV6;
+ memcpy(filter->sa.dst_ip.ipv6,
+ &security_ctx->spec.spec.ipv6.hdr.dst_addr,
+ sizeof(filter->sa.dst_ip.ipv6));
+ } else {
+ filter->sa.dst_ip.type = IPv4;
+ filter->sa.dst_ip.ipv4 = security_ctx->spec.spec.ipv4.hdr.dst_addr;
+ }
+
+ return 0;
+}
+
+static bool
+ixgbe_security_ip_equal(const struct ipaddr *lhs, const struct ipaddr *rhs)
+{
+ if (lhs->type != rhs->type)
+ return false;
+ if (lhs->type == IPv4)
+ return lhs->ipv4 == rhs->ipv4;
+
+ return memcmp(lhs->ipv6, rhs->ipv6, sizeof(lhs->ipv6)) == 0;
+}
+
+/* two SA filters are the same rule if they match the same SPI and destination IP */
+static bool
+ixgbe_security_sa_key_equal(const struct ixgbe_crypto_rx_sa *lhs,
+ const struct ixgbe_crypto_rx_sa *rhs)
+{
+ return lhs->spi == rhs->spi &&
+ ixgbe_security_ip_equal(&lhs->dst_ip, &rhs->dst_ip);
+}
+
+static int
+ixgbe_security_sa_slot_find(const struct ixgbe_security_priv *priv,
+ const struct ixgbe_security_flow *security_flow)
+{
+ uint32_t free_idx = IPSEC_MAX_SA_COUNT;
+ uint32_t idx;
+
+ for (idx = 0; idx < IPSEC_MAX_SA_COUNT; idx++) {
+ const struct ixgbe_security_flow *registered = priv->sa_slots[idx];
+
+ if (registered == NULL) {
+ if (free_idx == IPSEC_MAX_SA_COUNT)
+ free_idx = idx;
+ continue;
+ }
+ if (ixgbe_security_sa_key_equal(®istered->security.sa,
+ &security_flow->security.sa))
+ return -EEXIST;
+ }
+ if (free_idx == IPSEC_MAX_SA_COUNT)
+ return -ENOSPC;
+
+ return (int)free_idx;
+}
+
+static int
+ixgbe_security_ip_slot_find(const struct ixgbe_security_priv *priv,
+ const struct ipaddr *ip)
+{
+ uint32_t free_idx = IPSEC_MAX_RX_IP_COUNT;
+ uint32_t idx;
+
+ for (idx = 0; idx < IPSEC_MAX_RX_IP_COUNT; idx++) {
+ if (priv->ip_slots[idx].ref_count == 0) {
+ if (free_idx == IPSEC_MAX_RX_IP_COUNT)
+ free_idx = idx;
+ continue;
+ }
+ if (ixgbe_security_ip_equal(&priv->ip_slots[idx].ip, ip))
+ return (int)idx;
+ }
+ if (free_idx == IPSEC_MAX_RX_IP_COUNT)
+ return -ENOSPC;
+
+ return (int)free_idx;
+}
+
+static void
+ixgbe_security_ip_slot_get(struct ixgbe_security_priv *priv, uint32_t idx,
+ const struct ipaddr *ip)
+{
+ struct ixgbe_security_ip_slot *slot = &priv->ip_slots[idx];
+
+ if (slot->ref_count == 0)
+ slot->ip = *ip;
+ slot->ref_count++;
+}
+
+static void
+ixgbe_security_ip_slot_put(struct ixgbe_security_priv *priv, uint32_t idx)
+{
+ struct ixgbe_security_ip_slot *slot = &priv->ip_slots[idx];
+
+ if (--slot->ref_count == 0)
+ *slot = (struct ixgbe_security_ip_slot){0};
+}
+
+static bool
+ixgbe_security_ip_slot_is_last(const struct ixgbe_security_priv *priv, uint32_t idx)
+{
+ return priv->ip_slots[idx].ref_count == 1;
+}
+
+static bool
+ixgbe_security_flow_is_registered(const struct ixgbe_security_flow *security_flow)
+{
+ const struct ixgbe_security_priv *priv = security_flow->flow.flow.engine_priv;
+ const struct ixgbe_crypto_rx_sa *sa = &security_flow->security.sa;
+
+ return sa->ip_idx < IPSEC_MAX_RX_IP_COUNT &&
+ sa->sa_idx < IPSEC_MAX_SA_COUNT &&
+ priv->sa_slots[sa->sa_idx] == security_flow &&
+ priv->ip_slots[sa->ip_idx].ref_count != 0 &&
+ ixgbe_security_ip_equal(&priv->ip_slots[sa->ip_idx].ip,
+ &sa->dst_ip);
+}
+
+static int
+ixgbe_flow_security_flow_register(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_security_flow *security_flow = (struct ixgbe_security_flow *)flow;
+ struct ixgbe_security_priv *priv = flow->engine_priv;
+ struct ixgbe_security_filter *filter = &security_flow->security;
+ int sa_idx, ip_idx, ret;
+
+ sa_idx = ixgbe_security_sa_slot_find(priv, security_flow);
+ if (sa_idx == -EEXIST) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ingress security filter already exists");
+ }
+ if (sa_idx == -ENOSPC) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ingress security SA table is full");
+ }
+
+ ip_idx = ixgbe_security_ip_slot_find(priv, &filter->sa.dst_ip);
+ if (ip_idx == -ENOSPC) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Ingress security IP table is full");
+ }
+
+ ret = ixgbe_crypto_session_acquire(filter->session);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to acquire ingress security session");
+ }
+
+ /* every resource is acquired by this point, so nothing below may fail */
+ filter->sa.sa_idx = sa_idx;
+ filter->sa.ip_idx = ip_idx;
+ priv->sa_slots[sa_idx] = security_flow;
+ ixgbe_security_ip_slot_get(priv, ip_idx, &filter->sa.dst_ip);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_security_flow_unregister(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_security_flow *security_flow = (struct ixgbe_security_flow *)flow;
+ struct ixgbe_security_priv *priv = flow->engine_priv;
+ struct ixgbe_security_filter *filter = &security_flow->security;
+ int ret;
+
+ if (!ixgbe_security_flow_is_registered(security_flow)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Ingress security flow registration is invalid");
+ }
+ ret = ixgbe_crypto_session_release(filter->session);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to release ingress security session");
+ }
+
+ priv->sa_slots[filter->sa.sa_idx] = NULL;
+ ixgbe_security_ip_slot_put(priv, filter->sa.ip_idx);
+ return 0;
+}
+
+static int
+ixgbe_flow_security_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_security_flow *security_flow = (struct ixgbe_security_flow *)flow;
+
+ if (!ixgbe_security_flow_is_registered(security_flow)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Ingress security flow registration is invalid");
+ }
+
+ ixgbe_crypto_install_rx_sa(flow->dev_data, &security_flow->security.sa);
+ return 0;
+}
+
+static int
+ixgbe_flow_security_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_security_flow *security_flow = (struct ixgbe_security_flow *)flow;
+ struct ixgbe_security_priv *priv = flow->engine_priv;
+ const struct ixgbe_crypto_rx_sa *sa = &security_flow->security.sa;
+
+ if (!ixgbe_security_flow_is_registered(security_flow)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Ingress security flow registration is invalid");
+ }
+
+ ixgbe_crypto_uninstall_rx_sa(flow->dev_data, sa,
+ ixgbe_security_ip_slot_is_last(priv, sa->ip_idx));
+ return 0;
+}
+
+static int
+ixgbe_flow_security_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ if (hw->mac.type == ixgbe_mac_82599EB ||
+ hw->mac.type == ixgbe_mac_X540 ||
+ hw->mac.type == ixgbe_mac_X550 ||
+ hw->mac.type == ixgbe_mac_X550EM_x ||
+ hw->mac.type == ixgbe_mac_X550EM_a ||
+ hw->mac.type == ixgbe_mac_E610)
+ return 0;
+
+ return -ENOTSUP;
+}
+
+static const struct ci_flow_engine_ops ixgbe_security_ops = {
+ .engine_init = ixgbe_flow_security_engine_init,
+ .ctx_init = ixgbe_flow_security_ctx_init,
+ .ctx_to_flow = ixgbe_flow_security_ctx_to_flow,
+ .flow_register = ixgbe_flow_security_flow_register,
+ .flow_unregister = ixgbe_flow_security_flow_unregister,
+ .flow_install = ixgbe_flow_security_flow_install,
+ .flow_uninstall = ixgbe_flow_security_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_security_flow_engine = {
+ .name = "security",
+ .ctx_size = sizeof(struct ixgbe_security_ctx),
+ .flow_size = sizeof(struct ixgbe_security_flow),
+ .priv_size = sizeof(struct ixgbe_security_priv),
+ .ops = &ixgbe_security_ops,
+ .graph = &ixgbe_security_graph,
+};
diff --git a/drivers/net/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/intel/ixgbe/ixgbe_ipsec.c
index 3c35326016..256f9c5019 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ipsec.c
@@ -30,12 +30,6 @@
IXGBE_WRITE_REG_THEN_POLL_MASK(hw, IXGBE_IPSTXIDX, reg_val, \
IPSRXIDX_WRITE, IXGBE_REGISTER_POLL_WAIT_5_MS)
-#define CMP_IP(a, b) (\
- (a).ipv6[0] == (b).ipv6[0] && \
- (a).ipv6[1] == (b).ipv6[1] && \
- (a).ipv6[2] == (b).ipv6[2] && \
- (a).ipv6[3] == (b).ipv6[3])
-
static inline void
ixgbe_crypto_write_rx_ip(struct ixgbe_hw *hw, uint32_t idx,
const struct ipaddr *ip, bool enable)
@@ -137,195 +131,165 @@ ixgbe_crypto_clear_ipsec_tables(struct rte_eth_dev *dev)
ixgbe_crypto_write_tx_key(hw, i, key, 0, false);
}
- memset(priv->rx_ip_tbl, 0, sizeof(priv->rx_ip_tbl));
- memset(priv->rx_sa_tbl, 0, sizeof(priv->rx_sa_tbl));
memset(priv->tx_sa_tbl, 0, sizeof(priv->tx_sa_tbl));
}
+void
+ixgbe_crypto_install_rx_sa(struct rte_eth_dev_data *dev_data,
+ const struct ixgbe_crypto_rx_sa *sa)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+ uint32_t key[4];
+
+ ixgbe_crypto_write_rx_ip(hw, sa->ip_idx, &sa->dst_ip, true);
+ ixgbe_crypto_write_rx_spi(hw, sa->sa_idx, sa->spi, sa->ip_idx, true);
+ memcpy(key, sa->key, sizeof(key));
+ ixgbe_crypto_write_rx_key(hw, sa->sa_idx, (const uint8_t *)key,
+ sa->salt, sa->mode, true);
+ rte_memzero_explicit(key, sizeof(key));
+}
+
+void
+ixgbe_crypto_uninstall_rx_sa(struct rte_eth_dev_data *dev_data,
+ const struct ixgbe_crypto_rx_sa *sa, bool clear_ip)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+ const uint8_t key[16] = {0};
+
+ ixgbe_crypto_write_rx_spi(hw, sa->sa_idx, 0, 0, false);
+ ixgbe_crypto_write_rx_key(hw, sa->sa_idx, key, 0, 0, false);
+ if (clear_ip) {
+ const struct ipaddr ip = {0};
+
+ ixgbe_crypto_write_rx_ip(hw, sa->ip_idx, &ip, false);
+ }
+}
+
static int
-ixgbe_crypto_add_sa(struct ixgbe_crypto_session *ic_session)
+ixgbe_crypto_add_tx_sa(struct ixgbe_crypto_session *ic_session)
{
struct rte_eth_dev_data *dev_data = ic_session->dev_data;
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
struct ixgbe_ipsec *priv = IXGBE_DEV_PRIVATE_TO_IPSEC(dev_data->dev_private);
- int i, sa_index = -1;
- uint8_t key[16] = {0};
-
- if (ic_session->op == IXGBE_OP_AUTHENTICATED_DECRYPTION) {
- struct ixgbe_crypto_rx_ip_table *rxip;
- struct ixgbe_crypto_rx_sa_table *rxsa;
- int ip_index = -1, free_index = -1;
-
- /* Find a match in the IP table*/
- for (i = 0; i < IPSEC_MAX_RX_IP_COUNT; i++) {
- if (CMP_IP(priv->rx_ip_tbl[i].ip,
- ic_session->dst_ip)) {
- ip_index = i;
- break;
- }
- if (free_index == -1 && priv->rx_ip_tbl[i].ref_count == 0)
- free_index = i;
- }
- /* If no match, find a free entry in the IP table*/
- if (ip_index < 0)
- ip_index = free_index;
-
- /* Fail if no match and no free entries*/
- if (ip_index < 0) {
- PMD_DRV_LOG(ERR, "No free entry left in the Rx IP table");
- return -ENOSPC;
- }
- rxip = &priv->rx_ip_tbl[ip_index];
-
- /* Find a free entry in the SA table*/
- for (i = 0; i < IPSEC_MAX_SA_COUNT; i++) {
- if (priv->rx_sa_tbl[i].used == 0) {
- sa_index = i;
- break;
- }
- }
- /* Fail if no free entries*/
- if (sa_index < 0) {
- PMD_DRV_LOG(ERR, "No free entry left in the Rx SA table");
- return -ENOSPC;
- }
- rxsa = &priv->rx_sa_tbl[sa_index];
-
- rxip->ref_count++;
- memcpy(&rxip->ip, &ic_session->dst_ip, sizeof(rxip->ip));
-
- rxsa->spi = ic_session->spi;
- rxsa->ip_index = ip_index;
- rxsa->mode = IPSRXMOD_VALID | IPSRXMOD_PROTO | IPSRXMOD_DECRYPT;
- if (ic_session->dst_ip.type == IPv6)
- rxsa->mode |= IPSRXMOD_IPV6;
-
- rxsa->used = 1;
-
- /* write IP table entry*/
- ixgbe_crypto_write_rx_ip(hw, ip_index, &rxip->ip, true);
-
- /* write SPI table entry*/
- ixgbe_crypto_write_rx_spi(hw, sa_index, rxsa->spi, ip_index, true);
-
- /* write Key table entry*/
- memcpy(key, ic_session->key, ic_session->key_len);
-
- ixgbe_crypto_write_rx_key(hw, sa_index, key,
- ic_session->salt, rxsa->mode, true);
-
- rte_memzero_explicit(key, sizeof(key));
-
- } else { /* sess->dir == RTE_CRYPTO_OUTBOUND */
- struct ixgbe_crypto_tx_sa_table *txsa;
-
- /* Find a free entry in the SA table*/
- for (i = 0; i < IPSEC_MAX_SA_COUNT; i++) {
- if (priv->tx_sa_tbl[i].used == 0) {
- sa_index = i;
- break;
- }
+ struct ixgbe_crypto_tx_sa_table *txsa;
+ uint32_t key[4];
+ int sa_index = -1;
+ int i;
+
+ for (i = 0; i < IPSEC_MAX_SA_COUNT; i++) {
+ if (priv->tx_sa_tbl[i].used == 0) {
+ sa_index = i;
+ break;
}
- /* Fail if no free entries*/
- if (sa_index < 0) {
- PMD_DRV_LOG(ERR, "No free entry left in the Tx SA table");
- return -ENOSPC;
- }
- txsa = &priv->tx_sa_tbl[sa_index];
-
- txsa->spi = ic_session->spi;
- txsa->used = 1;
- ic_session->sa_index = sa_index;
-
- memcpy(key, ic_session->key, ic_session->key_len);
-
- /* write Key table entry*/
- ixgbe_crypto_write_tx_key(hw, sa_index, key, ic_session->salt, true);
-
- rte_memzero_explicit(key, sizeof(key));
}
+ if (sa_index < 0) {
+ PMD_DRV_LOG(ERR, "No free entry left in the Tx SA table");
+ return -ENOSPC;
+ }
+ txsa = &priv->tx_sa_tbl[sa_index];
+ txsa->spi = ic_session->spi;
+ txsa->used = 1;
+ ic_session->sa_index = sa_index;
+
+ memcpy(key, ic_session->key, ic_session->key_len);
+ ixgbe_crypto_write_tx_key(hw, sa_index, (const uint8_t *)key,
+ ic_session->salt, true);
+ rte_memzero_explicit(key, sizeof(key));
return 0;
}
static int
-ixgbe_crypto_remove_sa(struct ixgbe_crypto_session *ic_session)
+ixgbe_crypto_remove_tx_sa(struct ixgbe_crypto_session *ic_session)
{
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(ic_session->dev_data->dev_private);
struct ixgbe_ipsec *priv =
IXGBE_DEV_PRIVATE_TO_IPSEC(ic_session->dev_data->dev_private);
+ struct ixgbe_crypto_tx_sa_table *txsa;
const uint8_t key[16] = {0};
- int i, sa_index = -1;
- if (ic_session->op == IXGBE_OP_AUTHENTICATED_DECRYPTION) {
- struct ixgbe_crypto_rx_ip_table *rxip;
- struct ixgbe_crypto_rx_sa_table *rxsa;
- int ip_index = -1;
+ if (ic_session->sa_index >= IPSEC_MAX_SA_COUNT)
+ return -ENOENT;
+ txsa = &priv->tx_sa_tbl[ic_session->sa_index];
+ if (txsa->used == 0 || txsa->spi != ic_session->spi)
+ return -ENOENT;
- /* Find a match in the IP table*/
- for (i = 0; i < IPSEC_MAX_RX_IP_COUNT; i++) {
- if (CMP_IP(priv->rx_ip_tbl[i].ip, ic_session->dst_ip)) {
- ip_index = i;
- break;
- }
- }
+ ixgbe_crypto_write_tx_key(hw, ic_session->sa_index, key, 0, false);
+ *txsa = (struct ixgbe_crypto_tx_sa_table){0};
- /* Fail if no match*/
- if (ip_index < 0) {
- PMD_DRV_LOG(ERR, "Entry not found in the Rx IP table");
- return -ENOENT;
- }
- rxip = &priv->rx_ip_tbl[ip_index];
+ return 0;
+}
- /* Find a free entry in the SA table*/
- for (i = 0; i < IPSEC_MAX_SA_COUNT; i++) {
- if (priv->rx_sa_tbl[i].spi == ic_session->spi) {
- sa_index = i;
- break;
- }
- }
- /* Fail if no match*/
- if (sa_index < 0) {
- PMD_DRV_LOG(ERR, "Entry not found in the Rx SA table");
- return -ENOENT;
- }
- rxsa = &priv->rx_sa_tbl[sa_index];
+static int
+ixgbe_crypto_session_state_acquire(struct ixgbe_crypto_session *ic_session)
+{
+ uint16_t expected;
- /* Disable and clear Rx SPI and key table entries*/
- ixgbe_crypto_write_rx_spi(hw, sa_index, 0, 0, false);
- ixgbe_crypto_write_rx_key(hw, sa_index, key, 0, 0, false);
-
- /* Clear the SA table entry*/
- *rxsa = (struct ixgbe_crypto_rx_sa_table){0};
+ expected = rte_atomic_load_explicit(&ic_session->refcnt,
+ rte_memory_order_acquire);
+ while (expected != IXGBE_SECURITY_SESSION_DESTROYING) {
+ if (expected >= IXGBE_SECURITY_SESSION_REFCNT_MAX)
+ return -ENOSPC;
+ if (rte_atomic_compare_exchange_strong_explicit(&ic_session->refcnt,
+ &expected, expected + 1,
+ rte_memory_order_acq_rel, rte_memory_order_acquire))
+ return 0;
+ }
- /* If last used then clear the IP table entry*/
- rxip->ref_count--;
- if (rxip->ref_count == 0) {
- const struct ipaddr ip = {0};
- ixgbe_crypto_write_rx_ip(hw, ip_index, &ip, false);
- *rxip = (struct ixgbe_crypto_rx_ip_table){0};
- }
- } else { /* session->dir == RTE_CRYPTO_OUTBOUND */
- struct ixgbe_crypto_tx_sa_table *txsa;
+ return -EBUSY;
+}
- /* Find a match in the SA table*/
- for (i = 0; i < IPSEC_MAX_SA_COUNT; i++) {
- if (priv->tx_sa_tbl[i].spi == ic_session->spi) {
- sa_index = i;
- break;
- }
- }
- /* Fail if no match entries*/
- if (sa_index < 0) {
- PMD_DRV_LOG(ERR, "Entry not found in the Tx SA table");
- return -ENOENT;
- }
- txsa = &priv->tx_sa_tbl[sa_index];
+static int
+ixgbe_crypto_session_state_release(struct ixgbe_crypto_session *ic_session)
+{
+ uint16_t expected;
- ixgbe_crypto_write_tx_key(hw, sa_index, key, 0, false);
- *txsa = (struct ixgbe_crypto_tx_sa_table){0};
+ expected = rte_atomic_load_explicit(&ic_session->refcnt,
+ rte_memory_order_acquire);
+ while (expected != IXGBE_SECURITY_SESSION_DESTROYING) {
+ if (expected == IXGBE_SECURITY_SESSION_UNBOUND)
+ return -EINVAL;
+ if (rte_atomic_compare_exchange_strong_explicit(&ic_session->refcnt,
+ &expected, expected - 1,
+ rte_memory_order_release, rte_memory_order_relaxed))
+ return 0;
}
+ return -EINVAL;
+}
+
+int
+ixgbe_crypto_session_acquire(struct rte_security_session *session)
+{
+ struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(session);
+
+ return ixgbe_crypto_session_state_acquire(ic_session);
+}
+
+int
+ixgbe_crypto_session_release(struct rte_security_session *session)
+{
+ struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(session);
+
+ return ixgbe_crypto_session_state_release(ic_session);
+}
+
+static int
+ixgbe_crypto_session_disable(struct rte_security_session *session)
+{
+ struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(session);
+ uint16_t expected;
+
+ /*
+ * Rx sessions must have no flow references before removal. Tx sessions
+ * retain the reference acquired when their SA was allocated.
+ */
+ expected = ic_session->op == IXGBE_OP_AUTHENTICATED_DECRYPTION ?
+ IXGBE_SECURITY_SESSION_UNBOUND : 1;
+ if (!rte_atomic_compare_exchange_strong_explicit(&ic_session->refcnt,
+ &expected, IXGBE_SECURITY_SESSION_DESTROYING,
+ rte_memory_order_acq_rel, rte_memory_order_acquire))
+ return -EBUSY;
return 0;
}
@@ -338,6 +302,7 @@ ixgbe_crypto_create_session(void *device,
struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(session);
struct rte_crypto_aead_xform *aead_xform;
struct rte_eth_conf *dev_conf = ð_dev->data->dev_conf;
+ int ret;
if (conf->crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD ||
conf->crypto_xform->aead.algo !=
@@ -369,18 +334,30 @@ ixgbe_crypto_create_session(void *device,
}
}
- ic_session->key = aead_xform->key.data;
+ memcpy(ic_session->key, aead_xform->key.data, sizeof(ic_session->key));
ic_session->key_len = aead_xform->key.length;
memcpy(&ic_session->salt,
&aead_xform->key.data[aead_xform->key.length], 4);
ic_session->spi = conf->ipsec.spi;
ic_session->dev_data = eth_dev->data;
+ rte_atomic_store_explicit(&ic_session->refcnt,
+ IXGBE_SECURITY_SESSION_UNBOUND,
+ rte_memory_order_relaxed);
+
+ /* only handle tx, as rx sessions are created by rte_flow */
if (ic_session->op == IXGBE_OP_AUTHENTICATED_ENCRYPTION) {
- if (ixgbe_crypto_add_sa(ic_session)) {
+ if (ixgbe_crypto_add_tx_sa(ic_session)) {
PMD_DRV_LOG(ERR, "Failed to add SA");
+ rte_memzero_explicit(ic_session, sizeof(*ic_session));
return -EPERM;
}
+ ret = ixgbe_crypto_session_state_acquire(ic_session);
+ if (ret != 0) {
+ ixgbe_crypto_remove_tx_sa(ic_session);
+ rte_memzero_explicit(ic_session, sizeof(*ic_session));
+ return ret;
+ }
}
return 0;
@@ -398,18 +375,30 @@ ixgbe_crypto_remove_session(void *device,
{
struct rte_eth_dev *eth_dev = device;
struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(session);
+ int ret;
if (eth_dev->data != ic_session->dev_data) {
PMD_DRV_LOG(ERR, "Session not bound to this device");
return -ENODEV;
}
- if (ixgbe_crypto_remove_sa(ic_session)) {
- PMD_DRV_LOG(ERR, "Failed to remove session");
- return -EFAULT;
+ if (ixgbe_crypto_session_disable(session) < 0) {
+ PMD_DRV_LOG(ERR, "Session is still in use");
+ return -EBUSY;
}
- memset(ic_session, 0, sizeof(struct ixgbe_crypto_session));
+ if (ic_session->op == IXGBE_OP_AUTHENTICATED_ENCRYPTION) {
+ ret = ixgbe_crypto_remove_tx_sa(ic_session);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to remove session");
+ /* set refcnt back to 1 to re-enable the session */
+ rte_atomic_store_explicit(&ic_session->refcnt, 1,
+ rte_memory_order_release);
+ return ret;
+ }
+ }
+
+ rte_memzero_explicit(ic_session, sizeof(*ic_session));
return 0;
}
@@ -632,34 +621,6 @@ ixgbe_crypto_enable_ipsec(struct rte_eth_dev *dev)
return 0;
}
-int
-ixgbe_crypto_add_ingress_sa_from_flow(struct rte_security_session *sess,
- const struct ip_spec *spec)
-{
- struct ixgbe_crypto_session *ic_session = SECURITY_GET_SESS_PRIV(sess);
-
- if (ic_session->op == IXGBE_OP_AUTHENTICATED_DECRYPTION) {
- if (spec->is_ipv6) {
- const struct rte_flow_item_ipv6 *ipv6 = &spec->spec.ipv6;
- ic_session->src_ip.type = IPv6;
- ic_session->dst_ip.type = IPv6;
- memcpy(ic_session->src_ip.ipv6,
- &ipv6->hdr.src_addr, 16);
- memcpy(ic_session->dst_ip.ipv6,
- &ipv6->hdr.dst_addr, 16);
- } else {
- const struct rte_flow_item_ipv4 *ipv4 = &spec->spec.ipv4;
- ic_session->src_ip.type = IPv4;
- ic_session->dst_ip.type = IPv4;
- ic_session->src_ip.ipv4 = ipv4->hdr.src_addr;
- ic_session->dst_ip.ipv4 = ipv4->hdr.dst_addr;
- }
- return ixgbe_crypto_add_sa(ic_session);
- }
-
- return 0;
-}
-
static struct rte_security_ops ixgbe_security_ops = {
.session_create = ixgbe_crypto_create_session,
.session_update = NULL,
diff --git a/drivers/net/intel/ixgbe/ixgbe_ipsec.h b/drivers/net/intel/ixgbe/ixgbe_ipsec.h
index 1099b5f598..a872a64788 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ipsec.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ipsec.h
@@ -8,6 +8,7 @@
#include <ethdev_driver.h>
#include <rte_security.h>
#include <rte_security_driver.h>
+#include <rte_stdatomic.h>
#include <rte_flow.h>
@@ -33,6 +34,9 @@
#define IPSEC_MAX_RX_IP_COUNT 128
#define IPSEC_MAX_SA_COUNT 1024
+#define IXGBE_SECURITY_SESSION_UNBOUND 0
+#define IXGBE_SECURITY_SESSION_REFCNT_MAX IPSEC_MAX_SA_COUNT
+#define IXGBE_SECURITY_SESSION_DESTROYING UINT16_MAX
#define ESP_ICV_SIZE 16
#define ESP_TRAILER_SIZE 2
@@ -67,32 +71,30 @@ struct ipaddr {
/** inline crypto crypto private session structure */
struct __rte_cache_aligned ixgbe_crypto_session {
enum ixgbe_operation op;
- const uint8_t *key;
+ RTE_ATOMIC(uint16_t) refcnt;
+ uint8_t key[16];
uint32_t key_len;
uint32_t salt;
uint32_t sa_index;
uint32_t spi;
- struct ipaddr src_ip;
- struct ipaddr dst_ip;
struct rte_eth_dev_data *dev_data;
};
-struct ixgbe_crypto_rx_ip_table {
- struct ipaddr ip;
- uint16_t ref_count;
-};
-struct ixgbe_crypto_rx_sa_table {
- uint32_t spi;
- uint32_t ip_index;
- uint8_t mode;
- uint8_t used;
-};
-
struct ixgbe_crypto_tx_sa_table {
uint32_t spi;
uint8_t used;
};
+struct ixgbe_crypto_rx_sa {
+ struct ipaddr dst_ip;
+ const uint8_t *key;
+ uint32_t salt;
+ uint32_t spi;
+ uint32_t mode;
+ uint32_t ip_idx;
+ uint32_t sa_idx;
+};
+
union ixgbe_crypto_tx_desc_md {
uint64_t data;
struct {
@@ -106,25 +108,17 @@ union ixgbe_crypto_tx_desc_md {
};
struct ixgbe_ipsec {
- struct ixgbe_crypto_rx_ip_table rx_ip_tbl[IPSEC_MAX_RX_IP_COUNT];
- struct ixgbe_crypto_rx_sa_table rx_sa_tbl[IPSEC_MAX_SA_COUNT];
struct ixgbe_crypto_tx_sa_table tx_sa_tbl[IPSEC_MAX_SA_COUNT];
};
int ixgbe_ipsec_ctx_create(struct rte_eth_dev *dev);
int ixgbe_crypto_enable_ipsec(struct rte_eth_dev *dev);
-
-struct ip_spec {
- bool is_ipv6;
- union {
- struct rte_flow_item_ipv4 ipv4;
- struct rte_flow_item_ipv6 ipv6;
- } spec;
-};
-int ixgbe_crypto_add_ingress_sa_from_flow(struct rte_security_session *sess,
- const struct ip_spec *ip_spec);
-
-
+void ixgbe_crypto_install_rx_sa(struct rte_eth_dev_data *dev_data,
+ const struct ixgbe_crypto_rx_sa *sa);
+void ixgbe_crypto_uninstall_rx_sa(struct rte_eth_dev_data *dev_data,
+ const struct ixgbe_crypto_rx_sa *sa, bool clear_ip);
+int ixgbe_crypto_session_acquire(struct rte_security_session *session);
+int ixgbe_crypto_session_release(struct rte_security_session *session);
#endif /*IXGBE_IPSEC_H_*/
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 90cc88f002..12ba639b70 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -30,6 +30,7 @@ sources += files(
'ixgbe_flow_syn.c',
'ixgbe_flow_l2tun.c',
'ixgbe_flow_ntuple.c',
+ 'ixgbe_flow_security.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 10/19] net/ixgbe: reimplement FDIR parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (8 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 09/19] net/ixgbe: reimplement security parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 11/19] net/ixgbe: reimplement hash parser Anatoly Burakov
` (9 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for flow director.
The FDIR flow tracking is moved inside the new engine, and the FDIR code is
refactored to not mix software tracking with HW writes.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 99 +-
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 71 +-
drivers/net/intel/ixgbe/ixgbe_fdir.c | 304 +---
drivers/net/intel/ixgbe/ixgbe_flow.c | 1587 +------------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 2 +
drivers/net/intel/ixgbe/ixgbe_flow_fdir.c | 1703 +++++++++++++++++++++
drivers/net/intel/ixgbe/meson.build | 1 +
7 files changed, 1848 insertions(+), 1919 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_fdir.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index f1541a1554..94af935390 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -144,8 +144,6 @@ static const char * const ixgbevf_valid_arguments[] = {
static int eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params);
static int eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev);
-static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev);
-static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev);
static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev);
static int ixgbe_dev_configure(struct rte_eth_dev *dev);
static int ixgbe_dev_start(struct rte_eth_dev *dev);
@@ -1316,11 +1314,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
memset(filter_info, 0,
sizeof(struct ixgbe_filter_info));
- /* initialize flow director filter list & hash */
- ret = ixgbe_fdir_filter_init(eth_dev);
- if (ret)
- goto err_fdir_filter_init;
-
/* initialize l2 tunnel filter list & hash */
ret = ixgbe_l2_tn_filter_init(eth_dev);
if (ret)
@@ -1345,8 +1338,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
err_flow_engine_conf_init:
err_l2_tn_filter_init:
- ixgbe_fdir_filter_uninit(eth_dev);
-err_fdir_filter_init:
ixgbe_disable_intr(hw);
rte_intr_disable(intr_handle);
rte_intr_callback_unregister(intr_handle,
@@ -1376,32 +1367,24 @@ eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
return 0;
}
-static int ixgbe_fdir_filter_uninit(struct rte_eth_dev *eth_dev)
+void
+ixgbe_fdir_state_detach(struct ixgbe_fdir_state *state)
{
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(eth_dev->data->dev_private);
- struct ixgbe_fdir_filter *fdir_filter;
+ if (--state->refcnt > 0)
+ return;
- rte_free(fdir_info->hash_map);
- rte_hash_free(fdir_info->hash_handle);
-
- while ((fdir_filter = TAILQ_FIRST(&fdir_info->fdir_list))) {
- TAILQ_REMOVE(&fdir_info->fdir_list,
- fdir_filter,
- entries);
- rte_free(fdir_filter);
- }
-
- return 0;
+ state->adapter->fdir_state = NULL;
+ rte_hash_free(state->hash_handle);
+ rte_free(state);
}
-static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev)
+struct ixgbe_fdir_state *
+ixgbe_fdir_state_attach(struct rte_eth_dev_data *dev_data)
{
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(eth_dev->data->dev_private);
- char fdir_hash_name[RTE_HASH_NAMESIZE];
- struct rte_hash_parameters fdir_hash_params = {
- .name = fdir_hash_name,
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev_data->dev_private);
+ struct ixgbe_fdir_state *state = adapter->fdir_state;
+ struct rte_hash_parameters hash_params = {
+ .name = "ixgbe_fdir_hash",
.entries = IXGBE_MAX_FDIR_FILTER_NUM,
.key_len = sizeof(union ixgbe_atr_input),
.hash_func = rte_hash_crc,
@@ -1409,31 +1392,36 @@ static int ixgbe_fdir_filter_init(struct rte_eth_dev *eth_dev)
.socket_id = rte_socket_id(),
};
- TAILQ_INIT(&fdir_info->fdir_list);
- snprintf(fdir_hash_name, RTE_HASH_NAMESIZE,
- "fdir_%s", eth_dev->device->name);
- fdir_info->hash_handle = rte_hash_create(&fdir_hash_params);
- if (!fdir_info->hash_handle) {
- PMD_INIT_LOG(ERR, "Failed to create fdir hash table!");
- return -EINVAL;
- }
- fdir_info->hash_map = rte_zmalloc("ixgbe",
- sizeof(struct ixgbe_fdir_filter *) *
- IXGBE_MAX_FDIR_FILTER_NUM,
- 0);
- if (!fdir_info->hash_map) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for fdir hash map!");
- rte_hash_free(fdir_info->hash_handle);
- return -ENOMEM;
+ if (state == NULL) {
+ state = rte_zmalloc("ixgbe_fdir_state", sizeof(*state), 0);
+ if (state == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate fdir state!");
+ return NULL;
+ }
+
+ state->hash_handle = rte_hash_create(&hash_params);
+ if (state->hash_handle == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to create fdir hash table!");
+ goto fail;
+ }
+
+ state->adapter = adapter;
+ state->mode = RTE_FDIR_MODE_NONE;
+
+ /* drop queue is always fixed */
+ IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter)->drop_queue = IXGBE_FDIR_DROP_QUEUE;
+
+ adapter->fdir_state = state;
}
- fdir_info->n_flows = 0;
- fdir_info->mask_added = FALSE;
- /* drop queue is always fixed */
- IXGBE_DEV_FDIR_CONF(eth_dev)->drop_queue = IXGBE_FDIR_DROP_QUEUE;
+ /* callers are serialized by the flow framework, so no atomics needed */
+ state->refcnt++;
- return 0;
+ return state;
+fail:
+ rte_hash_free(state->hash_handle);
+ rte_free(state);
+ return NULL;
}
static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev)
@@ -2937,6 +2925,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
/* reset hierarchy commit */
tm_conf->committed = false;
+ /* the reset above wiped the flow director setup, kept flows are replayed on start */
+ ixgbe_fdir_hw_invalidate(dev);
+
adapter->rss_reta_updated = 0;
hw->adapter_stopped = true;
@@ -3081,9 +3072,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
/* uninitialize PF if max_vfs not zero */
ixgbe_pf_host_uninit(dev);
- /* remove all the fdir filters & hash */
- ixgbe_fdir_filter_uninit(dev);
-
/* clear all the filters list */
ixgbe_filterlist_flush(dev);
@@ -7861,7 +7849,6 @@ ixgbe_rss_filter_restore(struct rte_eth_dev *dev)
static int
ixgbe_filter_restore(struct rte_eth_dev *dev)
{
- ixgbe_fdir_filter_restore(dev);
ixgbe_rss_filter_restore(dev);
return 0;
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index bd66d4afd9..8c1f421651 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -50,6 +50,7 @@
#define IXGBE_VMDQ_DCB_NB_QUEUES IXGBE_MAX_RX_QUEUE_NUM
#define IXGBE_DCB_NB_QUEUES IXGBE_MAX_RX_QUEUE_NUM
#define IXGBE_NONE_MODE_TX_NB_QUEUES 64
+#define IXGBE_MAX_FLX_SOURCE_OFF 62
#ifndef NBBY
#define NBBY 8 /* number of bits in a byte */
@@ -160,17 +161,6 @@ struct ixgbe_hw_fdir_mask {
uint8_t tunnel_type_mask;
};
-struct ixgbe_fdir_filter {
- TAILQ_ENTRY(ixgbe_fdir_filter) entries;
- union ixgbe_atr_input ixgbe_fdir; /* key of fdir filter*/
- uint32_t fdirflags; /* drop or forward */
- uint32_t fdirhash; /* hash value for fdir */
- uint8_t queue; /* assigned rx queue */
-};
-
-/* list of fdir filters */
-TAILQ_HEAD(ixgbe_fdir_filter_list, ixgbe_fdir_filter);
-
struct ixgbe_fdir_rule {
struct ixgbe_hw_fdir_mask mask;
union ixgbe_atr_input ixgbe_fdir; /* key of fdir filter*/
@@ -194,12 +184,43 @@ struct ixgbe_hw_fdir_info {
uint64_t remove;
uint64_t f_add;
uint64_t f_remove;
- struct ixgbe_fdir_filter_list fdir_list; /* filter list*/
- /* store the pointers of the filters, index is the hash value. */
- struct ixgbe_fdir_filter **hash_map;
+};
+
+/* per-flow object of the fdir flow engines, private to ixgbe_flow_fdir.c */
+struct ixgbe_fdir_flow;
+
+/*
+ * Per-device state owned by the flow director flow engines.
+ *
+ * The fdir and fdir_tunnel engines drive the same hardware block, so this is
+ * shared between them: it is created by whichever of the two initializes first
+ * and destroyed by whichever uninitializes last, tracked by `refcnt`. Access is
+ * serialized by the flow framework's per-device config lock.
+ *
+ * `hash_handle` tracks every registered filter, keyed by `union ixgbe_atr_input`,
+ * and exists to reject duplicate filters: two filters with the same key would
+ * otherwise collide in the same hardware slot.
+ *
+ * The hardware has only one global input mask for all filters, so `mask`,
+ * `flex_bytes_offset` and `mode` are claimed by the first registered filter and
+ * every later filter has to agree with them. They are released once the last
+ * filter is unregistered.
+ */
+struct ixgbe_fdir_state {
+ /* device this state belongs to, needed to detach on engine uninit */
+ struct ixgbe_adapter *adapter;
struct rte_hash *hash_handle; /* cuckoo hash handler */
- uint32_t n_flows;
- bool mask_added; /* If already got mask from consistent filter */
+ struct ixgbe_hw_fdir_mask mask;
+ enum rte_fdir_mode mode;
+ uint8_t flex_bytes_offset;
+ /* true once a filter that actually carries a mask has claimed the above */
+ bool mask_added;
+ /* false whenever the hardware may have lost the global fdir setup */
+ bool hw_configured;
+ /* false when the claimed mask has not been written to hardware yet */
+ bool mask_programmed;
+ uint32_t nb_registered;
+ uint16_t refcnt;
};
struct ixgbe_rte_flow_rss_conf {
@@ -438,6 +459,8 @@ struct ixgbe_adapter {
struct ixgbe_macsec_setting macsec_setting;
struct rte_eth_fdir_conf fdir_conf;
struct ixgbe_hw_fdir_info fdir;
+ /* shared by the fdir and fdir_tunnel flow engines, NULL if neither is enabled */
+ struct ixgbe_fdir_state *fdir_state;
struct ixgbe_interrupt intr;
struct ixgbe_stat_mapping_registers stat_mappings;
struct ixgbe_vfta shadow_vfta;
@@ -684,10 +707,19 @@ int ixgbe_fdir_set_input_mask(struct ixgbe_adapter *adapter,
enum rte_fdir_mode mode);
int ixgbe_fdir_set_flexbytes_offset(struct ixgbe_adapter *adapter,
uint16_t offset);
+struct ixgbe_fdir_state *ixgbe_fdir_state_attach(struct rte_eth_dev_data *dev_data);
+void ixgbe_fdir_state_detach(struct ixgbe_fdir_state *state);
+void ixgbe_fdir_hw_invalidate(struct rte_eth_dev *dev);
+bool ixgbe_fdir_mode_is_perfect(enum rte_fdir_mode mode);
+uint32_t ixgbe_fdir_compute_hash(struct ixgbe_adapter *adapter,
+ struct ixgbe_fdir_rule *rule);
int ixgbe_fdir_filter_program(struct ixgbe_adapter *adapter,
- struct rte_eth_fdir_conf *fdir_conf,
struct ixgbe_fdir_rule *rule,
- bool del, bool update);
+ uint8_t queue,
+ uint32_t fdircmd_flags,
+ uint32_t fdirhash);
+int ixgbe_fdir_filter_clear(struct ixgbe_adapter *adapter, uint32_t fdirhash);
+int ixgbe_fdir_reset_tables(struct ixgbe_adapter *adapter);
void ixgbe_fdir_info_get(struct rte_eth_dev *dev,
struct rte_eth_fdir_info *fdir_info);
void ixgbe_fdir_stats_get(struct rte_eth_dev *dev,
@@ -718,9 +750,6 @@ int ixgbe_pf_host_configure(struct rte_eth_dev *eth_dev);
uint32_t ixgbe_convert_vm_rx_mask_to_val(uint16_t rx_mask, uint32_t orig_val);
-void ixgbe_fdir_filter_restore(struct rte_eth_dev *dev);
-int ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev);
-
extern const struct rte_flow_ops ixgbe_flow_ops;
int ixgbe_disable_sec_tx_path_generic(struct ixgbe_hw *hw);
diff --git a/drivers/net/intel/ixgbe/ixgbe_fdir.c b/drivers/net/intel/ixgbe/ixgbe_fdir.c
index b32dc54287..1adaf739c5 100644
--- a/drivers/net/intel/ixgbe/ixgbe_fdir.c
+++ b/drivers/net/intel/ixgbe/ixgbe_fdir.c
@@ -36,7 +36,6 @@
#define SIG_BUCKET_256KB_HASH_MASK 0x7FFF /* 15 bits */
#define IXGBE_DEFAULT_FLEXBYTES_OFFSET 12 /* default flexbytes offset in bytes */
#define IXGBE_FDIR_MAX_FLEX_LEN 2 /* len in bytes of flexbytes */
-#define IXGBE_MAX_FLX_SOURCE_OFF 62
#define IXGBE_FDIRCTRL_FLEX_MASK (0x1F << IXGBE_FDIRCTRL_FLEX_SHIFT)
#define IXGBE_FDIRCMD_CMD_INTERVAL_US 10
@@ -101,7 +100,6 @@ static int fdir_write_perfect_filter_82599(struct ixgbe_hw *hw,
static int fdir_add_signature_filter_82599(struct ixgbe_hw *hw,
union ixgbe_atr_input *input, u8 queue, uint32_t fdircmd,
uint32_t fdirhash);
-static int ixgbe_fdir_flush(struct rte_eth_dev *dev);
/**
* This function is based on ixgbe_fdir_enable_82599() in base/ixgbe_82599.c.
@@ -974,218 +972,81 @@ fdir_erase_filter_82599(struct ixgbe_hw *hw, uint32_t fdirhash)
}
-static inline struct ixgbe_fdir_filter *
-ixgbe_fdir_filter_lookup(struct ixgbe_hw_fdir_info *fdir_info,
- union ixgbe_atr_input *key)
+/* hardware loses its flow director setup across a stop/start cycle */
+void
+ixgbe_fdir_hw_invalidate(struct rte_eth_dev *dev)
{
- int ret;
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
+ struct ixgbe_fdir_state *state = adapter->fdir_state;
- ret = rte_hash_lookup(fdir_info->hash_handle, (const void *)key);
- if (ret < 0)
- return NULL;
-
- return fdir_info->hash_map[ret];
+ if (state != NULL) {
+ state->hw_configured = false;
+ state->mask_programmed = false;
+ }
}
-static inline int
-ixgbe_insert_fdir_filter(struct ixgbe_hw_fdir_info *fdir_info,
- struct ixgbe_fdir_filter *fdir_filter)
+bool
+ixgbe_fdir_mode_is_perfect(enum rte_fdir_mode mode)
{
- int ret;
-
- ret = rte_hash_add_key(fdir_info->hash_handle,
- &fdir_filter->ixgbe_fdir);
-
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to insert fdir filter to hash table %d!",
- ret);
- return ret;
- }
-
- fdir_info->hash_map[ret] = fdir_filter;
-
- TAILQ_INSERT_TAIL(&fdir_info->fdir_list, fdir_filter, entries);
-
- return 0;
+ return mode >= RTE_FDIR_MODE_PERFECT && mode <= RTE_FDIR_MODE_PERFECT_TUNNEL;
}
-static inline int
-ixgbe_remove_fdir_filter(struct ixgbe_hw_fdir_info *fdir_info,
- union ixgbe_atr_input *key)
+uint32_t
+ixgbe_fdir_compute_hash(struct ixgbe_adapter *adapter, struct ixgbe_fdir_rule *rule)
{
- int ret;
- struct ixgbe_fdir_filter *fdir_filter;
-
- ret = rte_hash_del_key(fdir_info->hash_handle, key);
-
- if (ret < 0) {
- PMD_DRV_LOG(ERR, "No such fdir filter to delete %d!", ret);
- return ret;
- }
+ const struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
+ uint32_t fdirhash;
- fdir_filter = fdir_info->hash_map[ret];
- fdir_info->hash_map[ret] = NULL;
+ if (!ixgbe_fdir_mode_is_perfect(rule->mode))
+ return atr_compute_sig_hash_82599(&rule->ixgbe_fdir, fdir_conf->pballoc);
- TAILQ_REMOVE(&fdir_info->fdir_list, fdir_filter, entries);
- rte_free(fdir_filter);
+ fdirhash = atr_compute_perfect_hash_82599(&rule->ixgbe_fdir, fdir_conf->pballoc);
+ fdirhash |= rule->soft_id << IXGBE_FDIRHASH_SIG_SW_INDEX_SHIFT;
- return 0;
+ return fdirhash;
}
int
ixgbe_fdir_filter_program(struct ixgbe_adapter *adapter,
- struct rte_eth_fdir_conf *fdir_conf,
struct ixgbe_fdir_rule *rule,
- bool del,
- bool update)
+ uint8_t queue,
+ uint32_t fdircmd_flags,
+ uint32_t fdirhash)
{
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
- uint32_t fdircmd_flags;
- uint32_t fdirhash;
- uint8_t queue;
- bool is_perfect = FALSE;
int err;
- struct ixgbe_hw_fdir_info *info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
- enum rte_fdir_mode fdir_mode = fdir_conf->mode;
- struct ixgbe_fdir_filter *node;
- bool add_node = FALSE;
- if (fdir_mode == RTE_FDIR_MODE_NONE ||
- fdir_mode != rule->mode)
- return -ENOTSUP;
-
- /*
- * Sanity check for x550 and E610.
- * When adding a new filter with flow type set to IPv4,
- * the flow director mask should be configed before,
- * and the L4 protocol and ports are masked.
- */
- if ((!del) &&
- (hw->mac.type == ixgbe_mac_X550 ||
- hw->mac.type == ixgbe_mac_X550EM_x ||
- hw->mac.type == ixgbe_mac_X550EM_a ||
- hw->mac.type == ixgbe_mac_E610) &&
- (rule->ixgbe_fdir.formatted.flow_type ==
- IXGBE_ATR_FLOW_TYPE_IPV4 ||
- rule->ixgbe_fdir.formatted.flow_type ==
- IXGBE_ATR_FLOW_TYPE_IPV6) &&
- (info->mask.src_port_mask != 0 ||
- info->mask.dst_port_mask != 0) &&
- (rule->mode != RTE_FDIR_MODE_PERFECT_MAC_VLAN &&
- rule->mode != RTE_FDIR_MODE_PERFECT_TUNNEL)) {
- PMD_DRV_LOG(ERR, "By this device,"
- " IPv4 is not supported without"
- " L4 protocol and ports masked!");
- return -ENOTSUP;
- }
-
- if (fdir_mode >= RTE_FDIR_MODE_PERFECT &&
- fdir_mode <= RTE_FDIR_MODE_PERFECT_TUNNEL)
- is_perfect = TRUE;
-
- if (is_perfect) {
- if (rule->ixgbe_fdir.formatted.flow_type &
- IXGBE_ATR_L4TYPE_IPV6_MASK) {
- PMD_DRV_LOG(ERR, "IPv6 is not supported in"
- " perfect mode!");
- return -ENOTSUP;
- }
- fdirhash = atr_compute_perfect_hash_82599(&rule->ixgbe_fdir,
- fdir_conf->pballoc);
- fdirhash |= rule->soft_id <<
- IXGBE_FDIRHASH_SIG_SW_INDEX_SHIFT;
- } else
- fdirhash = atr_compute_sig_hash_82599(&rule->ixgbe_fdir,
- fdir_conf->pballoc);
-
- if (del) {
- err = ixgbe_remove_fdir_filter(info, &rule->ixgbe_fdir);
- if (err < 0)
- return err;
-
- err = fdir_erase_filter_82599(hw, fdirhash);
- if (err < 0)
- PMD_DRV_LOG(ERR, "Fail to delete FDIR filter!");
- else
- PMD_DRV_LOG(DEBUG, "Success to delete FDIR filter!");
- return err;
- }
- /* add or update an fdir filter*/
- fdircmd_flags = (update) ? IXGBE_FDIRCMD_FILTER_UPDATE : 0;
- if (rule->fdirflags & IXGBE_FDIRCMD_DROP) {
- if (is_perfect) {
- queue = fdir_conf->drop_queue;
- fdircmd_flags |= IXGBE_FDIRCMD_DROP;
- } else {
- PMD_DRV_LOG(ERR, "Drop option is not supported in"
- " signature mode.");
- return -EINVAL;
- }
- } else if (rule->queue < IXGBE_MAX_RX_QUEUE_NUM)
- queue = (uint8_t)rule->queue;
+ if (ixgbe_fdir_mode_is_perfect(rule->mode))
+ err = fdir_write_perfect_filter_82599(hw, &rule->ixgbe_fdir, queue,
+ fdircmd_flags, fdirhash, rule->mode);
else
- return -EINVAL;
+ err = fdir_add_signature_filter_82599(hw, &rule->ixgbe_fdir, queue,
+ fdircmd_flags, fdirhash);
- node = ixgbe_fdir_filter_lookup(info, &rule->ixgbe_fdir);
- if (node) {
- if (update) {
- node->fdirflags = fdircmd_flags;
- node->fdirhash = fdirhash;
- node->queue = queue;
- } else {
- PMD_DRV_LOG(ERR, "Conflict with existing fdir filter!");
- return -EINVAL;
- }
- } else {
- add_node = TRUE;
- node = rte_zmalloc("ixgbe_fdir",
- sizeof(struct ixgbe_fdir_filter),
- 0);
- if (!node)
- return -ENOMEM;
- memcpy(&node->ixgbe_fdir,
- &rule->ixgbe_fdir,
- sizeof(union ixgbe_atr_input));
- node->fdirflags = fdircmd_flags;
- node->fdirhash = fdirhash;
- node->queue = queue;
-
- err = ixgbe_insert_fdir_filter(info, node);
- if (err < 0) {
- rte_free(node);
- return err;
- }
- }
-
- if (is_perfect) {
- err = fdir_write_perfect_filter_82599(hw, &rule->ixgbe_fdir,
- queue, fdircmd_flags,
- fdirhash, fdir_mode);
- } else {
- err = fdir_add_signature_filter_82599(hw, &rule->ixgbe_fdir,
- queue, fdircmd_flags,
- fdirhash);
- }
- if (err < 0) {
+ if (err < 0)
PMD_DRV_LOG(ERR, "Fail to add FDIR filter!");
- if (add_node)
- (void)ixgbe_remove_fdir_filter(info, &rule->ixgbe_fdir);
- } else {
- PMD_DRV_LOG(DEBUG, "Success to add FDIR filter");
- }
+ return err;
+}
+
+int
+ixgbe_fdir_filter_clear(struct ixgbe_adapter *adapter, uint32_t fdirhash)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
+ int err;
+
+ err = fdir_erase_filter_82599(hw, fdirhash);
+ if (err < 0)
+ PMD_DRV_LOG(ERR, "Fail to delete FDIR filter!");
return err;
}
-static int
-ixgbe_fdir_flush(struct rte_eth_dev *dev)
+int
+ixgbe_fdir_reset_tables(struct ixgbe_adapter *adapter)
{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_hw_fdir_info *info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(dev->data->dev_private);
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
+ struct ixgbe_hw_fdir_info *info = IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
int ret;
ret = ixgbe_reinit_fdir_tables_82599(hw);
@@ -1318,76 +1179,3 @@ ixgbe_fdir_stats_get(struct rte_eth_dev *dev, struct rte_eth_fdir_stats *fdir_st
fdir_stats->guarant_cnt = max_num * 4 - fdir_stats->free;
}
-
-/* restore flow director filter */
-void
-ixgbe_fdir_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(dev->data->dev_private);
- struct ixgbe_fdir_filter *node;
- bool is_perfect = FALSE;
- enum rte_fdir_mode fdir_mode = fdir_conf->mode;
-
- if (fdir_mode >= RTE_FDIR_MODE_PERFECT &&
- fdir_mode <= RTE_FDIR_MODE_PERFECT_TUNNEL)
- is_perfect = TRUE;
-
- if (is_perfect) {
- TAILQ_FOREACH(node, &fdir_info->fdir_list, entries) {
- (void)fdir_write_perfect_filter_82599(hw,
- &node->ixgbe_fdir,
- node->queue,
- node->fdirflags,
- node->fdirhash,
- fdir_mode);
- }
- } else {
- TAILQ_FOREACH(node, &fdir_info->fdir_list, entries) {
- (void)fdir_add_signature_filter_82599(hw,
- &node->ixgbe_fdir,
- node->queue,
- node->fdirflags,
- node->fdirhash);
- }
- }
-}
-
-/* remove all the flow director filters */
-int
-ixgbe_clear_all_fdir_filter(struct rte_eth_dev *dev)
-{
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(dev->data->dev_private);
- struct ixgbe_fdir_filter *fdir_filter;
- bool had_flows;
- int ret = 0;
-
- had_flows = (fdir_info->n_flows != 0);
-
- /* flush flow director */
- rte_hash_reset(fdir_info->hash_handle);
- memset(fdir_info->hash_map, 0,
- sizeof(struct ixgbe_fdir_filter *) * IXGBE_MAX_FDIR_FILTER_NUM);
- while ((fdir_filter = TAILQ_FIRST(&fdir_info->fdir_list))) {
- TAILQ_REMOVE(&fdir_info->fdir_list,
- fdir_filter,
- entries);
- rte_free(fdir_filter);
- }
- fdir_info->n_flows = 0;
-
- /* reset internal FDIR state */
- fdir_info->mask = (struct ixgbe_hw_fdir_mask){0};
- fdir_info->flex_bytes_offset = 0;
- fdir_info->mask_added = FALSE;
- fdir_conf->mode = RTE_FDIR_MODE_NONE;
-
- if (had_flows)
- ret = ixgbe_fdir_flush(dev);
-
- return ret;
-}
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 4c268ac7f1..868d65b2a8 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -50,17 +50,10 @@
#include "../common/flow_engine.h"
#include "ixgbe_flow.h"
-#define IXGBE_MAX_FLX_SOURCE_OFF 62
-
struct ixgbe_filter_ele_base {
TAILQ_ENTRY(ixgbe_filter_ele_base) entries;
};
-/* fdir filter list structure */
-struct ixgbe_fdir_rule_ele {
- struct ixgbe_filter_ele_base base;
- struct ixgbe_fdir_rule filter_info;
-};
/* rss filter list structure */
struct ixgbe_rss_conf_ele {
struct ixgbe_filter_ele_base base;
@@ -79,28 +72,10 @@ const struct ci_flow_engine_list ixgbe_flow_engine_list = {
&ixgbe_l2_tunnel_flow_engine,
&ixgbe_ntuple_flow_engine,
&ixgbe_security_flow_engine,
+ &ixgbe_fdir_flow_engine,
+ &ixgbe_fdir_tunnel_flow_engine,
},
};
-
-/**
- * Endless loop will never happen with below assumption
- * 1. there is at least one no-void item(END)
- * 2. cur is before END.
- */
-static inline
-const struct rte_flow_item *next_no_void_pattern(
- const struct rte_flow_item pattern[],
- const struct rte_flow_item *cur)
-{
- const struct rte_flow_item *next =
- cur ? cur + 1 : &pattern[0];
- while (1) {
- if (next->type != RTE_FLOW_ITEM_TYPE_VOID)
- return next;
- next++;
- }
-}
-
/*
* All ixgbe engines mostly check the same stuff, so use a common check.
*/
@@ -156,1491 +131,6 @@ ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
* normally the packets should use network order.
*/
-/* search next no void pattern and skip fuzzy */
-static inline
-const struct rte_flow_item *next_no_fuzzy_pattern(
- const struct rte_flow_item pattern[],
- const struct rte_flow_item *cur)
-{
- const struct rte_flow_item *next =
- next_no_void_pattern(pattern, cur);
- while (1) {
- if (next->type != RTE_FLOW_ITEM_TYPE_FUZZY)
- return next;
- next = next_no_void_pattern(pattern, next);
- }
-}
-
-static inline uint8_t signature_match(const struct rte_flow_item pattern[])
-{
- const struct rte_flow_item_fuzzy *spec, *last, *mask;
- const struct rte_flow_item *item;
- uint32_t sh, lh, mh;
- int i = 0;
-
- while (1) {
- item = pattern + i;
- if (item->type == RTE_FLOW_ITEM_TYPE_END)
- break;
-
- if (item->type == RTE_FLOW_ITEM_TYPE_FUZZY) {
- spec = item->spec;
- last = item->last;
- mask = item->mask;
-
- if (!spec || !mask)
- return 0;
-
- sh = spec->thresh;
-
- if (!last)
- lh = sh;
- else
- lh = last->thresh;
-
- mh = mask->thresh;
- sh = sh & mh;
- lh = lh & mh;
-
- if (!sh || sh > lh)
- return 0;
-
- return 1;
- }
-
- i++;
- }
-
- return 0;
-}
-
-/**
- * Parse the rule to see if it is a IP or MAC VLAN flow director rule.
- * And get the flow director filter info BTW.
- * UDP/TCP/SCTP PATTERN:
- * The first not void item can be ETH or IPV4 or IPV6
- * The second not void item must be IPV4 or IPV6 if the first one is ETH.
- * The next not void item could be UDP or TCP or SCTP (optional)
- * The next not void item could be RAW (for flexbyte, optional)
- * The next not void item must be END.
- * A Fuzzy Match pattern can appear at any place before END.
- * Fuzzy Match is optional for IPV4 but is required for IPV6
- * MAC VLAN PATTERN:
- * The first not void item must be ETH.
- * The second not void item must be MAC VLAN.
- * The next not void item must be END.
- * ACTION:
- * The first not void action should be QUEUE or DROP.
- * The second not void optional action should be MARK,
- * mark_id is a uint32_t number.
- * The next not void action should be END.
- * UDP/TCP/SCTP pattern example:
- * ITEM Spec Mask
- * ETH NULL NULL
- * IPV4 src_addr 192.168.1.20 0xFFFFFFFF
- * dst_addr 192.167.3.50 0xFFFFFFFF
- * UDP/TCP/SCTP src_port 80 0xFFFF
- * dst_port 80 0xFFFF
- * FLEX relative 0 0x1
- * search 0 0x1
- * reserved 0 0
- * offset 12 0xFFFFFFFF
- * limit 0 0xFFFF
- * length 2 0xFFFF
- * pattern[0] 0x86 0xFF
- * pattern[1] 0xDD 0xFF
- * END
- * MAC VLAN pattern example:
- * ITEM Spec Mask
- * ETH dst_addr
- {0xAC, 0x7B, 0xA1, {0xFF, 0xFF, 0xFF,
- 0x2C, 0x6D, 0x36} 0xFF, 0xFF, 0xFF}
- * MAC VLAN tci 0x2016 0xEFFF
- * END
- * Other members in mask and spec should set to 0x00.
- * Item->last should be NULL.
- */
-static int
-ixgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct ci_flow_actions *parsed_actions,
- struct ixgbe_fdir_rule *rule,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- const struct rte_flow_item_ipv4 *ipv4_spec;
- const struct rte_flow_item_ipv4 *ipv4_mask;
- const struct rte_flow_item_ipv6 *ipv6_spec;
- const struct rte_flow_item_ipv6 *ipv6_mask;
- const struct rte_flow_item_tcp *tcp_spec;
- const struct rte_flow_item_tcp *tcp_mask;
- const struct rte_flow_item_udp *udp_spec;
- const struct rte_flow_item_udp *udp_mask;
- const struct rte_flow_item_sctp *sctp_spec;
- const struct rte_flow_item_sctp *sctp_mask;
- const struct rte_flow_item_vlan *vlan_spec;
- const struct rte_flow_item_vlan *vlan_mask;
- const struct rte_flow_item_raw *raw_mask;
- const struct rte_flow_item_raw *raw_spec;
- const struct rte_flow_action *fwd_action, *aux_action;
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- uint8_t j;
-
- fwd_action = parsed_actions->actions[0];
- /* can be NULL */
- aux_action = parsed_actions->actions[1];
-
- /**
- * Some fields may not be provided. Set spec to 0 and mask to default
- * value. So, we need not do anything for the not provided fields later.
- */
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- memset(&rule->mask, 0xFF, sizeof(struct ixgbe_hw_fdir_mask));
- rule->mask.vlan_tci_mask = 0;
- rule->mask.flex_bytes_mask = 0;
- rule->mask.l4_proto_match = 0;
- rule->mask.dst_port_mask = 0;
- rule->mask.src_port_mask = 0;
-
- /* check if this is a signature match */
- if (signature_match(pattern))
- rule->mode = RTE_FDIR_MODE_SIGNATURE;
- else
- rule->mode = RTE_FDIR_MODE_PERFECT;
-
- /* set up action */
- if (fwd_action->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
- const struct rte_flow_action_queue *q_act = fwd_action->conf;
- rule->queue = q_act->index;
- } else {
- /* signature mode does not support drop action. */
- if (rule->mode == RTE_FDIR_MODE_SIGNATURE) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, fwd_action,
- "Signature mode does not support drop action.");
- return -rte_errno;
- }
- rule->fdirflags = IXGBE_FDIRCMD_DROP;
- }
-
- /* set up mark action */
- if (aux_action != NULL && aux_action->type == RTE_FLOW_ACTION_TYPE_MARK) {
- const struct rte_flow_action_mark *m_act = aux_action->conf;
- rule->soft_id = m_act->id;
- }
-
- /**
- * The first not void item should be
- * MAC or IPv4 or TCP or UDP or SCTP.
- */
- item = next_no_fuzzy_pattern(pattern, NULL);
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6 &&
- item->type != RTE_FLOW_ITEM_TYPE_TCP &&
- item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_SCTP) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Get the MAC info. */
- if (item->type == RTE_FLOW_ITEM_TYPE_ETH) {
- /**
- * Only support vlan and dst MAC address,
- * others should be masked.
- */
- if (item->spec && !item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- if (item->spec) {
- rule->b_spec = TRUE;
- eth_spec = item->spec;
-
- /* Get the dst MAC. */
- for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
- rule->ixgbe_fdir.formatted.inner_mac[j] =
- eth_spec->hdr.dst_addr.addr_bytes[j];
- }
- }
-
-
- if (item->mask) {
-
- rule->b_mask = TRUE;
- eth_mask = item->mask;
-
- /* Ether type should be masked. */
- if (eth_mask->hdr.ether_type ||
- rule->mode == RTE_FDIR_MODE_SIGNATURE) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /* If ethernet has meaning, it means MAC VLAN mode. */
- rule->mode = RTE_FDIR_MODE_PERFECT_MAC_VLAN;
-
- /**
- * src MAC address must be masked,
- * and don't support dst MAC address mask.
- */
- for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
- if (eth_mask->hdr.src_addr.addr_bytes[j] ||
- eth_mask->hdr.dst_addr.addr_bytes[j] != 0xFF) {
- memset(rule, 0,
- sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* When no VLAN, considered as full mask. */
- rule->mask.vlan_tci_mask = rte_cpu_to_be_16(0xEFFF);
- }
- /*** If both spec and mask are item,
- * it means don't care about ETH.
- * Do nothing.
- */
-
- /**
- * Check if the next not void item is vlan or ipv4.
- * IPv6 is not supported.
- */
- item = next_no_fuzzy_pattern(pattern, item);
- if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
- if (item->type != RTE_FLOW_ITEM_TYPE_VLAN) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- } else {
- if (item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_VLAN) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
- }
-
- if (item->type == RTE_FLOW_ITEM_TYPE_VLAN) {
- if (!(item->spec && item->mask)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- vlan_spec = item->spec;
- vlan_mask = item->mask;
-
- rule->ixgbe_fdir.formatted.vlan_id = vlan_spec->hdr.vlan_tci;
-
- rule->mask.vlan_tci_mask = vlan_mask->hdr.vlan_tci;
- rule->mask.vlan_tci_mask &= rte_cpu_to_be_16(0xEFFF);
- /* More than one tags are not supported. */
-
- /* Next not void item must be END */
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Get the IPV4 info. */
- if (item->type == RTE_FLOW_ITEM_TYPE_IPV4) {
- /**
- * Set the flow type even if there's no content
- * as we must have a flow type.
- */
- rule->ixgbe_fdir.formatted.flow_type =
- IXGBE_ATR_FLOW_TYPE_IPV4;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- /**
- * Only care about src & dst addresses,
- * others should be masked.
- */
- if (!item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- rule->b_mask = TRUE;
- ipv4_mask = item->mask;
- if (ipv4_mask->hdr.version_ihl ||
- ipv4_mask->hdr.type_of_service ||
- ipv4_mask->hdr.total_length ||
- ipv4_mask->hdr.packet_id ||
- ipv4_mask->hdr.fragment_offset ||
- ipv4_mask->hdr.time_to_live ||
- ipv4_mask->hdr.next_proto_id ||
- ipv4_mask->hdr.hdr_checksum) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- rule->mask.dst_ipv4_mask = ipv4_mask->hdr.dst_addr;
- rule->mask.src_ipv4_mask = ipv4_mask->hdr.src_addr;
-
- if (item->spec) {
- rule->b_spec = TRUE;
- ipv4_spec = item->spec;
- rule->ixgbe_fdir.formatted.dst_ip[0] =
- ipv4_spec->hdr.dst_addr;
- rule->ixgbe_fdir.formatted.src_ip[0] =
- ipv4_spec->hdr.src_addr;
- }
-
- /**
- * Check if the next not void item is
- * TCP or UDP or SCTP or END.
- */
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_TCP &&
- item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_SCTP &&
- item->type != RTE_FLOW_ITEM_TYPE_END &&
- item->type != RTE_FLOW_ITEM_TYPE_RAW) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Get the IPV6 info. */
- if (item->type == RTE_FLOW_ITEM_TYPE_IPV6) {
- /**
- * Set the flow type even if there's no content
- * as we must have a flow type.
- */
- rule->ixgbe_fdir.formatted.flow_type =
- IXGBE_ATR_FLOW_TYPE_IPV6;
-
- /**
- * 1. must signature match
- * 2. not support last
- * 3. mask must not null
- */
- if (rule->mode != RTE_FDIR_MODE_SIGNATURE ||
- item->last ||
- !item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- rule->b_mask = TRUE;
- ipv6_mask = item->mask;
- if (ipv6_mask->hdr.vtc_flow ||
- ipv6_mask->hdr.payload_len ||
- ipv6_mask->hdr.proto ||
- ipv6_mask->hdr.hop_limits) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /* check src addr mask */
- for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.src_addr.a[j] == 0) {
- rule->mask.src_ipv6_mask &= ~(1 << j);
- } else if (ipv6_mask->hdr.src_addr.a[j] != UINT8_MAX) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* check dst addr mask */
- for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.dst_addr.a[j] == 0) {
- rule->mask.dst_ipv6_mask &= ~(1 << j);
- } else if (ipv6_mask->hdr.dst_addr.a[j] != UINT8_MAX) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- if (item->spec) {
- rule->b_spec = TRUE;
- ipv6_spec = item->spec;
- memcpy(rule->ixgbe_fdir.formatted.src_ip,
- &ipv6_spec->hdr.src_addr, 16);
- memcpy(rule->ixgbe_fdir.formatted.dst_ip,
- &ipv6_spec->hdr.dst_addr, 16);
- }
-
- /**
- * Check if the next not void item is
- * TCP or UDP or SCTP or END.
- */
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_TCP &&
- item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_SCTP &&
- item->type != RTE_FLOW_ITEM_TYPE_END &&
- item->type != RTE_FLOW_ITEM_TYPE_RAW) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Get the TCP info. */
- if (item->type == RTE_FLOW_ITEM_TYPE_TCP) {
- /**
- * Set the flow type even if there's no content
- * as we must have a flow type.
- */
- rule->ixgbe_fdir.formatted.flow_type |=
- IXGBE_ATR_L4TYPE_TCP;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- tcp_mask = item->mask;
- if (tcp_mask != NULL) {
- /**
- * Only care about src & dst ports,
- * others should be masked.
- */
- rule->b_mask = TRUE;
- if (tcp_mask->hdr.sent_seq ||
- tcp_mask->hdr.recv_ack ||
- tcp_mask->hdr.data_off ||
- tcp_mask->hdr.tcp_flags ||
- tcp_mask->hdr.rx_win ||
- tcp_mask->hdr.cksum ||
- tcp_mask->hdr.tcp_urp) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- rule->mask.src_port_mask = tcp_mask->hdr.src_port;
- rule->mask.dst_port_mask = tcp_mask->hdr.dst_port;
-
- if (item->spec) {
- rule->b_spec = TRUE;
- tcp_spec = item->spec;
- rule->ixgbe_fdir.formatted.src_port =
- tcp_spec->hdr.src_port;
- rule->ixgbe_fdir.formatted.dst_port =
- tcp_spec->hdr.dst_port;
- }
- } else if (item->spec != NULL) {
- /* No port mask means protocol-only match; spec is invalid. */
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_RAW &&
- item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- }
-
- /* Get the UDP info */
- if (item->type == RTE_FLOW_ITEM_TYPE_UDP) {
- /**
- * Set the flow type even if there's no content
- * as we must have a flow type.
- */
- rule->ixgbe_fdir.formatted.flow_type |=
- IXGBE_ATR_L4TYPE_UDP;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- udp_mask = item->mask;
- if (udp_mask != NULL) {
- /**
- * Only care about src & dst ports,
- * others should be masked.
- */
- rule->b_mask = TRUE;
- if (udp_mask->hdr.dgram_len ||
- udp_mask->hdr.dgram_cksum) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- rule->mask.src_port_mask = udp_mask->hdr.src_port;
- rule->mask.dst_port_mask = udp_mask->hdr.dst_port;
-
- if (item->spec) {
- rule->b_spec = TRUE;
- udp_spec = item->spec;
- rule->ixgbe_fdir.formatted.src_port =
- udp_spec->hdr.src_port;
- rule->ixgbe_fdir.formatted.dst_port =
- udp_spec->hdr.dst_port;
- }
- } else if (item->spec != NULL) {
- /* No port mask means protocol-only match; spec is invalid. */
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_RAW &&
- item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- }
-
- /* Get the SCTP info */
- if (item->type == RTE_FLOW_ITEM_TYPE_SCTP) {
- /**
- * Set the flow type even if there's no content
- * as we must have a flow type.
- */
- rule->ixgbe_fdir.formatted.flow_type |=
- IXGBE_ATR_L4TYPE_SCTP;
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- sctp_mask = item->mask;
- if (sctp_mask != NULL) {
- /* only some mac types support sctp port masking */
- if (hw->mac.type == ixgbe_mac_X550 ||
- hw->mac.type == ixgbe_mac_X550EM_x ||
- hw->mac.type == ixgbe_mac_X550EM_a ||
- hw->mac.type == ixgbe_mac_E610) {
- /**
- * Only care about src & dst ports,
- * others should be masked.
- */
- rule->b_mask = TRUE;
- if (sctp_mask->hdr.tag ||
- sctp_mask->hdr.cksum) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- rule->mask.src_port_mask = sctp_mask->hdr.src_port;
- rule->mask.dst_port_mask = sctp_mask->hdr.dst_port;
-
- if (item->spec) {
- rule->b_spec = TRUE;
- sctp_spec = item->spec;
- rule->ixgbe_fdir.formatted.src_port =
- sctp_spec->hdr.src_port;
- rule->ixgbe_fdir.formatted.dst_port =
- sctp_spec->hdr.dst_port;
- }
- /* others even sctp port masking is not supported */
- } else if (sctp_mask->hdr.src_port ||
- sctp_mask->hdr.dst_port ||
- sctp_mask->hdr.tag ||
- sctp_mask->hdr.cksum) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- } else if (item->spec != NULL) {
- /* No port mask means protocol-only match; spec is invalid. */
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_RAW &&
- item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Get the flex byte info */
- if (item->type == RTE_FLOW_ITEM_TYPE_RAW) {
- /* Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- /* mask should not be null */
- if (!item->mask || !item->spec) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- raw_mask = item->mask;
-
- /* check mask */
- if (raw_mask->relative != 0x1 ||
- raw_mask->search != 0x1 ||
- raw_mask->reserved != 0x0 ||
- (uint32_t)raw_mask->offset != 0xffffffff ||
- raw_mask->limit != 0xffff ||
- raw_mask->length != 0xffff) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- raw_spec = item->spec;
-
- /* check spec */
- if (raw_spec->relative != 0 ||
- raw_spec->search != 0 ||
- raw_spec->reserved != 0 ||
- raw_spec->offset > IXGBE_MAX_FLX_SOURCE_OFF ||
- raw_spec->offset % 2 ||
- raw_spec->limit != 0 ||
- raw_spec->length != 2 ||
- /* pattern can't be 0xffff */
- (raw_spec->pattern[0] == 0xff &&
- raw_spec->pattern[1] == 0xff)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /* check pattern mask */
- if (raw_mask->pattern[0] != 0xff ||
- raw_mask->pattern[1] != 0xff) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- rule->mask.flex_bytes_mask = 0xffff;
- rule->ixgbe_fdir.formatted.flex_bytes =
- (((uint16_t)raw_spec->pattern[1]) << 8) |
- raw_spec->pattern[0];
- rule->flex_bytes_offset = raw_spec->offset;
- }
-
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- /* check if the next not void item is END */
- item = next_no_fuzzy_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* L4 protocol matching is enabled when parser selected an L4 type. */
- rule->mask.l4_proto_match =
- (rule->ixgbe_fdir.formatted.flow_type & IXGBE_ATR_L4TYPE_MASK) != 0;
-
- return 0;
-}
-
-#define NVGRE_PROTOCOL 0x6558
-
-/**
- * Parse the rule to see if it is a VxLAN or NVGRE flow director rule.
- * And get the flow director filter info BTW.
- * VxLAN PATTERN:
- * The first not void item must be ETH.
- * The second not void item must be IPV4/ IPV6.
- * The third not void item must be NVGRE.
- * The next not void item must be END.
- * NVGRE PATTERN:
- * The first not void item must be ETH.
- * The second not void item must be IPV4/ IPV6.
- * The third not void item must be NVGRE.
- * The next not void item must be END.
- * ACTION:
- * The first not void action should be QUEUE or DROP.
- * The second not void optional action should be MARK,
- * mark_id is a uint32_t number.
- * The next not void action should be END.
- * VxLAN pattern example:
- * ITEM Spec Mask
- * ETH NULL NULL
- * IPV4/IPV6 NULL NULL
- * UDP NULL NULL
- * VxLAN vni{0x00, 0x32, 0x54} {0xFF, 0xFF, 0xFF}
- * MAC VLAN tci 0x2016 0xEFFF
- * END
- * NEGRV pattern example:
- * ITEM Spec Mask
- * ETH NULL NULL
- * IPV4/IPV6 NULL NULL
- * NVGRE protocol 0x6558 0xFFFF
- * tni{0x00, 0x32, 0x54} {0xFF, 0xFF, 0xFF}
- * MAC VLAN tci 0x2016 0xEFFF
- * END
- * other members in mask and spec should set to 0x00.
- * item->last should be NULL.
- */
-static int
-ixgbe_parse_fdir_filter_tunnel(const struct rte_flow_item pattern[],
- const struct ci_flow_actions *parsed_actions,
- struct ixgbe_fdir_rule *rule,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item *item;
- const struct rte_flow_item_vxlan *vxlan_spec;
- const struct rte_flow_item_vxlan *vxlan_mask;
- const struct rte_flow_item_nvgre *nvgre_spec;
- const struct rte_flow_item_nvgre *nvgre_mask;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- const struct rte_flow_item_vlan *vlan_spec;
- const struct rte_flow_item_vlan *vlan_mask;
- const struct rte_flow_action *fwd_action, *aux_action;
- uint32_t j;
-
- fwd_action = parsed_actions->actions[0];
- /* can be NULL */
- aux_action = parsed_actions->actions[1];
-
- /**
- * Some fields may not be provided. Set spec to 0 and mask to default
- * value. So, we need not do anything for the not provided fields later.
- */
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- memset(&rule->mask, 0xFF, sizeof(struct ixgbe_hw_fdir_mask));
- rule->mask.vlan_tci_mask = 0;
-
- /* set up queue/drop action */
- if (fwd_action->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
- const struct rte_flow_action_queue *q_act = fwd_action->conf;
- rule->queue = q_act->index;
- } else {
- rule->fdirflags = IXGBE_FDIRCMD_DROP;
- }
-
- /* set up mark action */
- if (aux_action != NULL && aux_action->type == RTE_FLOW_ACTION_TYPE_MARK) {
- const struct rte_flow_action_mark *mark = aux_action->conf;
- rule->soft_id = mark->id;
- }
-
- /**
- * The first not void item should be
- * MAC or IPv4 or IPv6 or UDP or VxLAN.
- */
- item = next_no_void_pattern(pattern, NULL);
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6 &&
- item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_VXLAN &&
- item->type != RTE_FLOW_ITEM_TYPE_NVGRE) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- rule->mode = RTE_FDIR_MODE_PERFECT_TUNNEL;
-
- /* Skip MAC. */
- if (item->type == RTE_FLOW_ITEM_TYPE_ETH) {
- /* Only used to describe the protocol stack. */
- if (item->spec || item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /* Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Check if the next not void item is IPv4 or IPv6. */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_IPV4 &&
- item->type != RTE_FLOW_ITEM_TYPE_IPV6) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Skip IP. */
- if (item->type == RTE_FLOW_ITEM_TYPE_IPV4 ||
- item->type == RTE_FLOW_ITEM_TYPE_IPV6) {
- /* Only used to describe the protocol stack. */
- if (item->spec || item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Check if the next not void item is UDP or NVGRE. */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_UDP &&
- item->type != RTE_FLOW_ITEM_TYPE_NVGRE) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Skip UDP. */
- if (item->type == RTE_FLOW_ITEM_TYPE_UDP) {
- /* Only used to describe the protocol stack. */
- if (item->spec || item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- /* Check if the next not void item is VxLAN. */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_VXLAN) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* Get the VxLAN info */
- if (item->type == RTE_FLOW_ITEM_TYPE_VXLAN) {
- rule->ixgbe_fdir.formatted.tunnel_type =
- IXGBE_FDIR_VXLAN_TUNNEL_TYPE;
-
- /* Only care about VNI, others should be masked. */
- if (!item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- rule->b_mask = TRUE;
-
- /* Tunnel type is always meaningful. */
- rule->mask.tunnel_type_mask = 1;
-
- vxlan_mask = item->mask;
- if (vxlan_mask->hdr.flags) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /* VNI must be totally masked or not. */
- if ((vxlan_mask->hdr.vni[0] || vxlan_mask->hdr.vni[1] ||
- vxlan_mask->hdr.vni[2]) &&
- ((vxlan_mask->hdr.vni[0] != 0xFF) ||
- (vxlan_mask->hdr.vni[1] != 0xFF) ||
- (vxlan_mask->hdr.vni[2] != 0xFF))) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- memcpy(&rule->mask.tunnel_id_mask, vxlan_mask->hdr.vni,
- RTE_DIM(vxlan_mask->hdr.vni));
-
- if (item->spec) {
- rule->b_spec = TRUE;
- vxlan_spec = item->spec;
- memcpy(((uint8_t *)
- &rule->ixgbe_fdir.formatted.tni_vni),
- vxlan_spec->hdr.vni, RTE_DIM(vxlan_spec->hdr.vni));
- }
- }
-
- /* Get the NVGRE info */
- if (item->type == RTE_FLOW_ITEM_TYPE_NVGRE) {
- rule->ixgbe_fdir.formatted.tunnel_type =
- IXGBE_FDIR_NVGRE_TUNNEL_TYPE;
-
- /**
- * Only care about flags0, flags1, protocol and TNI,
- * others should be masked.
- */
- if (!item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- rule->b_mask = TRUE;
-
- /* Tunnel type is always meaningful. */
- rule->mask.tunnel_type_mask = 1;
-
- nvgre_mask = item->mask;
- if (nvgre_mask->flow_id) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- if (nvgre_mask->protocol &&
- nvgre_mask->protocol != 0xFFFF) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- if (nvgre_mask->c_k_s_rsvd0_ver &&
- nvgre_mask->c_k_s_rsvd0_ver !=
- rte_cpu_to_be_16(0xFFFF)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /* TNI must be totally masked or not. */
- if (nvgre_mask->tni[0] &&
- ((nvgre_mask->tni[0] != 0xFF) ||
- (nvgre_mask->tni[1] != 0xFF) ||
- (nvgre_mask->tni[2] != 0xFF))) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /* tni is a 24-bits bit field */
- memcpy(&rule->mask.tunnel_id_mask, nvgre_mask->tni,
- RTE_DIM(nvgre_mask->tni));
- rule->mask.tunnel_id_mask <<= 8;
-
- if (item->spec) {
- rule->b_spec = TRUE;
- nvgre_spec = item->spec;
- if (nvgre_spec->c_k_s_rsvd0_ver !=
- rte_cpu_to_be_16(0x2000) &&
- nvgre_mask->c_k_s_rsvd0_ver) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- if (nvgre_mask->protocol &&
- nvgre_spec->protocol !=
- rte_cpu_to_be_16(NVGRE_PROTOCOL)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /* tni is a 24-bits bit field */
- memcpy(&rule->ixgbe_fdir.formatted.tni_vni,
- nvgre_spec->tni, RTE_DIM(nvgre_spec->tni));
- }
- }
-
- /* check if the next not void item is MAC */
- item = next_no_void_pattern(pattern, item);
- if (item->type != RTE_FLOW_ITEM_TYPE_ETH) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /**
- * Only support vlan and dst MAC address,
- * others should be masked.
- */
-
- if (!item->mask) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
- rule->b_mask = TRUE;
- eth_mask = item->mask;
-
- /* Ether type should be masked. */
- if (eth_mask->hdr.ether_type) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- /* src MAC address should be masked. */
- for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
- if (eth_mask->hdr.src_addr.addr_bytes[j]) {
- memset(rule, 0,
- sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
- rule->mask.mac_addr_byte_mask = 0;
- for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
- /* It's a per byte mask. */
- if (eth_mask->hdr.dst_addr.addr_bytes[j] == 0xFF) {
- rule->mask.mac_addr_byte_mask |= 0x1 << j;
- } else if (eth_mask->hdr.dst_addr.addr_bytes[j]) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /* When no vlan, considered as full mask. */
- rule->mask.vlan_tci_mask = rte_cpu_to_be_16(0xEFFF);
-
- if (item->spec) {
- rule->b_spec = TRUE;
- eth_spec = item->spec;
-
- /* Get the dst MAC. */
- for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
- rule->ixgbe_fdir.formatted.inner_mac[j] =
- eth_spec->hdr.dst_addr.addr_bytes[j];
- }
- }
-
- /**
- * Check if the next not void item is vlan or ipv4.
- * IPv6 is not supported.
- */
- item = next_no_void_pattern(pattern, item);
- if ((item->type != RTE_FLOW_ITEM_TYPE_VLAN) &&
- (item->type != RTE_FLOW_ITEM_TYPE_IPV4)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- /*Not supported last point for range*/
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- item, "Not supported last point for range");
- return -rte_errno;
- }
-
- if (item->type == RTE_FLOW_ITEM_TYPE_VLAN) {
- if (!(item->spec && item->mask)) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
-
- vlan_spec = item->spec;
- vlan_mask = item->mask;
-
- rule->ixgbe_fdir.formatted.vlan_id = vlan_spec->hdr.vlan_tci;
-
- rule->mask.vlan_tci_mask = vlan_mask->hdr.vlan_tci;
- rule->mask.vlan_tci_mask &= rte_cpu_to_be_16(0xEFFF);
- /* More than one tags are not supported. */
-
- /* check if the next not void item is END */
- item = next_no_void_pattern(pattern, item);
-
- if (item->type != RTE_FLOW_ITEM_TYPE_END) {
- memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item, "Not supported by fdir filter");
- return -rte_errno;
- }
- }
-
- /**
- * If the tags is 0, it means don't care about the VLAN.
- * Do nothing.
- */
-
- return 0;
-}
-
-/*
- * Check flow director actions
- */
-static int
-ixgbe_fdir_actions_check(const struct ci_flow_actions *parsed_actions,
- const struct ci_flow_actions_check_param *param __rte_unused,
- struct rte_flow_error *error)
-{
- const enum rte_flow_action_type fwd_actions[] = {
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_DROP,
- RTE_FLOW_ACTION_TYPE_END
- };
- const struct rte_flow_action *action;
-
- /* do the generic checks first */
- int ret = ixgbe_flow_actions_check(parsed_actions, param, error);
- if (ret)
- return ret;
-
- /* first action must be a forwarding action */
- action = parsed_actions->actions[0];
- if (!ci_flow_action_type_in_list(action->type, fwd_actions)) {
- return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- action, "First action must be QUEUE or DROP");
- }
-
- /* second action, if specified, must not be a forwarding action */
- action = parsed_actions->actions[1];
- if (action != NULL && ci_flow_action_type_in_list(action->type, fwd_actions)) {
- return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
- action, "Conflicting actions");
- }
- return 0;
-}
-
-static int
-ixgbe_parse_fdir_filter(struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct ixgbe_fdir_rule *rule,
- struct rte_flow_error *error)
-{
- int ret;
- struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* queue/mark/drop allowed here */
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_DROP,
- RTE_FLOW_ACTION_TYPE_MARK,
- RTE_FLOW_ACTION_TYPE_END
- },
- .driver_ctx = dev->data,
- .check = ixgbe_fdir_actions_check
- };
-
- if (hw->mac.type != ixgbe_mac_82599EB &&
- hw->mac.type != ixgbe_mac_X540 &&
- hw->mac.type != ixgbe_mac_X550 &&
- hw->mac.type != ixgbe_mac_X550EM_x &&
- hw->mac.type != ixgbe_mac_X550EM_a &&
- hw->mac.type != ixgbe_mac_E610)
- return -ENOTSUP;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
-
- fdir_conf->drop_queue = IXGBE_FDIR_DROP_QUEUE;
-
- ret = ixgbe_parse_fdir_filter_normal(dev, pattern, &parsed_actions, rule, error);
- if (!ret)
- return 0;
-
- return ixgbe_parse_fdir_filter_tunnel(pattern, &parsed_actions,
- rule, error);
-}
-
-static int
-ixgbe_fdir_process_rule(struct ixgbe_adapter *adapter,
- struct ixgbe_hw_fdir_info *fdir_info,
- struct ixgbe_fdir_rule *fdir_rule,
- bool *first_mask,
- struct rte_flow_error *error)
-{
- bool flex_byte_offset_changed;
- int ret;
-
- /* rule must have a spec to be valid */
- if (!fdir_rule->b_spec)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "No filter spec");
-
- /* if rule doesn't have a mask, nothing to be done */
- if (!fdir_rule->b_mask)
- return 0;
-
- /* if we already have a mask, check if it's compatible */
- if (fdir_info->mask_added) {
- ret = memcmp(&fdir_info->mask, &fdir_rule->mask,
- sizeof(struct ixgbe_hw_fdir_mask));
- if (ret)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Mask mismatch");
-
- if (fdir_rule->mask.flex_bytes_mask &&
- fdir_info->flex_bytes_offset !=
- fdir_rule->flex_bytes_offset)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Flex bytes offset mismatch");
-
- /* success */
- return 0;
- }
-
- /* we don't have a mask yet, so set it up based on this rule */
- flex_byte_offset_changed =
- fdir_info->flex_bytes_offset !=
- fdir_rule->flex_bytes_offset;
-
- if (fdir_rule->mask.flex_bytes_mask != 0 && flex_byte_offset_changed) {
- ret = ixgbe_fdir_set_flexbytes_offset(adapter,
- fdir_rule->flex_bytes_offset);
- if (ret)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL,
- "Failed to set flex bytes offset");
- }
-
- ret = ixgbe_fdir_set_input_mask(adapter, &fdir_rule->mask,
- fdir_rule->mode);
- if (ret)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Failed to set fdir mask");
-
- /* let the caller know that we've installed a mask */
- *first_mask = true;
-
- return 0;
-}
-
-static int
-ixgbe_fdir_flow_program(struct rte_eth_dev *dev,
- struct ixgbe_adapter *adapter,
- struct ixgbe_fdir_rule *fdir_rule,
- bool *first_mask,
- struct rte_flow_error *error)
-{
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
- struct rte_eth_fdir_conf local_fdir_conf = *fdir_conf;
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
- int ret;
-
- if (fdir_rule->queue >= dev->data->nb_rx_queues) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION,
- NULL, "queue id > max number of queues");
- }
-
- local_fdir_conf.mode = fdir_rule->mode;
-
- /* Configure FDIR mode if this is the first filter */
- if (fdir_conf->mode == RTE_FDIR_MODE_NONE) {
- ret = ixgbe_fdir_configure(adapter, &local_fdir_conf, &fdir_rule->mask);
- if (ret) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Failed to configure fdir mode");
- }
- } else if (fdir_conf->mode != fdir_rule->mode) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Conflict with existing fdir mode");
- }
-
- /* Process and validate rule spec and mask */
- ret = ixgbe_fdir_process_rule(adapter, fdir_info, fdir_rule,
- first_mask, error);
- if (ret)
- return ret;
-
- /* Program the filter */
- ret = ixgbe_fdir_filter_program(adapter, &local_fdir_conf,
- fdir_rule, FALSE, FALSE);
- if (ret)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "Failed to add fdir filter");
-
- return 0;
-}
-
/* Flow actions check specific to RSS filter */
static int
ixgbe_flow_actions_check_rss(const struct ci_flow_actions *parsed_actions,
@@ -1784,14 +274,9 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
int ret;
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_fdir_rule fdir_rule;
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ixgbe_rte_flow_rss_conf rss_conf;
struct rte_flow *flow = NULL;
- struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_rss_conf_ele *rss_filter_ptr;
struct ixgbe_flow_mem *ixgbe_flow_mem_ptr;
@@ -1818,39 +303,6 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
TAILQ_INSERT_TAIL(&adapter->flow_list,
&ixgbe_flow_mem_ptr->base, entries);
- memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
- ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
- actions, &fdir_rule, error);
- if (!ret) {
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
- bool first_mask = false;
-
- ret = ixgbe_fdir_flow_program(dev, adapter, &fdir_rule,
- &first_mask, error);
- if (ret)
- goto out;
-
- fdir_rule_ptr = rte_zmalloc("ixgbe_fdir_filter",
- sizeof(struct ixgbe_fdir_rule_ele), 0);
- if (!fdir_rule_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- /* update global state */
- if (first_mask) {
- fdir_info->mask_added = TRUE;
- fdir_info->mask = fdir_rule.mask;
- fdir_info->flex_bytes_offset = fdir_rule.flex_bytes_offset;
- }
- fdir_info->n_flows++;
- fdir_conf->mode = fdir_rule.mode;
-
- fdir_rule_ptr->filter_info = fdir_rule;
- flow->rule = fdir_rule_ptr;
- flow->filter_type = RTE_ETH_FILTER_FDIR;
- return flow;
- }
-
memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
ret = ixgbe_parse_rss_filter(dev, attr,
actions, &rss_conf, error);
@@ -1895,7 +347,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_fdir_rule fdir_rule;
struct ixgbe_rte_flow_rss_conf rss_conf;
int ret;
@@ -1906,12 +357,6 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
/* fall back to legacy engines */
- memset(&fdir_rule, 0, sizeof(struct ixgbe_fdir_rule));
- ret = ixgbe_parse_fdir_filter(dev, attr, pattern,
- actions, &fdir_rule, error);
- if (!ret)
- return 0;
-
memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
ret = ixgbe_parse_rss_filter(dev, attr,
actions, &rss_conf, error);
@@ -1930,12 +375,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct rte_flow *pmd_flow = flow;
enum rte_filter_type filter_type = pmd_flow->filter_type;
- struct ixgbe_fdir_rule fdir_rule;
- struct ixgbe_fdir_rule_ele *fdir_rule_ptr;
struct ixgbe_filter_ele_base *flow_mem_base;
- struct ixgbe_hw_fdir_info *fdir_info =
- IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
- struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_FDIR_CONF(dev);
struct ixgbe_rss_conf_ele *rss_filter_ptr;
/* try the new flow engine first */
@@ -1960,20 +400,6 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
}
switch (filter_type) {
- case RTE_ETH_FILTER_FDIR:
- fdir_rule_ptr = (struct ixgbe_fdir_rule_ele *)pmd_flow->rule;
- fdir_rule = fdir_rule_ptr->filter_info;
- ret = ixgbe_fdir_filter_program(adapter, fdir_conf, &fdir_rule, TRUE, FALSE);
- if (!ret) {
- rte_free(fdir_rule_ptr);
- if (fdir_info->n_flows > 0 && --(fdir_info->n_flows) == 0) {
- fdir_info->mask_added = false;
- fdir_info->mask = (struct ixgbe_hw_fdir_mask){0};
- fdir_info->flex_bytes_offset = 0;
- fdir_conf->mode = RTE_FDIR_MODE_NONE;
- }
- }
- break;
case RTE_ETH_FILTER_HASH:
rss_filter_ptr = (struct ixgbe_rss_conf_ele *)
pmd_flow->rule;
@@ -2018,13 +444,6 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
return ret;
}
- ret = ixgbe_clear_all_fdir_filter(dev);
- if (ret < 0) {
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Failed to flush rule");
- return ret;
- }
-
ixgbe_clear_rss_filter(dev);
ixgbe_filterlist_flush(dev);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index 87cf028245..256e0478f5 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -20,5 +20,7 @@ extern const struct ci_flow_engine ixgbe_syn_flow_engine;
extern const struct ci_flow_engine ixgbe_l2_tunnel_flow_engine;
extern const struct ci_flow_engine ixgbe_ntuple_flow_engine;
extern const struct ci_flow_engine ixgbe_security_flow_engine;
+extern const struct ci_flow_engine ixgbe_fdir_flow_engine;
+extern const struct ci_flow_engine ixgbe_fdir_tunnel_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_fdir.c b/drivers/net/intel/ixgbe/ixgbe_flow_fdir.c
new file mode 100644
index 0000000000..90e3cf69ef
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_fdir.c
@@ -0,0 +1,1703 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+#include <rte_hash.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_fdir_flow {
+ struct rte_flow flow;
+ struct ixgbe_fdir_rule rule;
+ /* resolved at ctx_to_flow time, written to hardware at install time */
+ uint32_t fdirhash;
+ uint32_t fdircmd_flags;
+ uint8_t queue;
+};
+
+/* both fdir engines get their own priv, all pointing at the same shared state */
+struct ixgbe_fdir_priv {
+ struct ixgbe_fdir_state *state;
+};
+
+struct ixgbe_fdir_ctx {
+ struct ci_flow_engine_ctx base;
+ struct ixgbe_fdir_rule rule;
+ bool supports_sctp_ports;
+ const struct rte_flow_action *fwd_action;
+ const struct rte_flow_action *aux_action;
+};
+
+#define IXGBE_FDIR_VLAN_TCI_MASK rte_cpu_to_be_16(0xEFFF)
+#define NVGRE_FLAGS 0x2000
+#define NVGRE_PROTOCOL 0x6558
+
+/**
+ * FDIR normal graph implementation
+ * Pattern: START -> [ETH] -> (IPv4|IPv6) -> [TCP|UDP|SCTP] -> [RAW] -> END
+ * Pattern: START -> ETH -> VLAN -> END
+ */
+
+enum ixgbe_fdir_normal_node_id {
+ IXGBE_FDIR_NORMAL_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_FDIR_NORMAL_NODE_ETH,
+ IXGBE_FDIR_NORMAL_NODE_VLAN,
+ IXGBE_FDIR_NORMAL_NODE_IPV4,
+ IXGBE_FDIR_NORMAL_NODE_IPV6,
+ IXGBE_FDIR_NORMAL_NODE_TCP,
+ IXGBE_FDIR_NORMAL_NODE_UDP,
+ IXGBE_FDIR_NORMAL_NODE_SCTP,
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ IXGBE_FDIR_NORMAL_NODE_MAX,
+};
+
+static int
+ixgbe_validate_fdir_normal_eth(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct ixgbe_fdir_ctx *fdir_ctx = (const struct ixgbe_fdir_ctx *)ctx;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+
+ if (item->spec == NULL && item->mask == NULL)
+ return 0;
+
+ /* we cannot have ETH item in signature mode */
+ if (fdir_ctx->rule.mode == RTE_FDIR_MODE_SIGNATURE) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "ETH item not supported in signature mode");
+ }
+ /* ethertype isn't supported by FDIR */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.ether_type)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Ethertype filtering not supported");
+ }
+ /* source address mask must be all zeroes */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Source MAC filtering not supported");
+ }
+ /* destination address mask must be all ones */
+ if (!CI_FIELD_IS_MASKED(ð_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Destination MAC filtering must be exact match");
+ }
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_eth(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+
+
+ if (eth_spec == NULL && eth_mask == NULL)
+ return 0;
+
+ /* copy dst MAC */
+ rule->b_spec = TRUE;
+ memcpy(rule->ixgbe_fdir.formatted.inner_mac, eth_spec->hdr.dst_addr.addr_bytes,
+ RTE_ETHER_ADDR_LEN);
+
+ /* set tunnel type */
+ rule->mode = RTE_FDIR_MODE_PERFECT_MAC_VLAN;
+ /* when no VLAN specified, set full mask */
+ rule->b_mask = TRUE;
+ rule->mask.vlan_tci_mask = IXGBE_FDIR_VLAN_TCI_MASK;
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_vlan(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.vlan_id = vlan_spec->hdr.vlan_tci;
+
+ rule->mask.vlan_tci_mask = vlan_mask->hdr.vlan_tci;
+ rule->mask.vlan_tci_mask &= IXGBE_FDIR_VLAN_TCI_MASK;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_ipv4(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+ const struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4 not supported with ETH/VLAN items");
+ }
+
+ if (ipv4_mask->hdr.version_ihl ||
+ ipv4_mask->hdr.type_of_service ||
+ ipv4_mask->hdr.total_length ||
+ ipv4_mask->hdr.packet_id ||
+ ipv4_mask->hdr.fragment_offset ||
+ ipv4_mask->hdr.time_to_live ||
+ ipv4_mask->hdr.next_proto_id ||
+ ipv4_mask->hdr.hdr_checksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst addresses supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_ipv4(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_ipv4 *ipv4_spec = item->spec;
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.flow_type = IXGBE_ATR_FLOW_TYPE_IPV4;
+
+ /* spec may not be present */
+ if (ipv4_spec) {
+ rule->b_spec = TRUE;
+ rule->ixgbe_fdir.formatted.dst_ip[0] = ipv4_spec->hdr.dst_addr;
+ rule->ixgbe_fdir.formatted.src_ip[0] = ipv4_spec->hdr.src_addr;
+ }
+
+ rule->b_mask = TRUE;
+ rule->mask.dst_ipv4_mask = ipv4_mask->hdr.dst_addr;
+ rule->mask.src_ipv4_mask = ipv4_mask->hdr.src_addr;
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_ipv6(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_ipv6 *ipv6_mask = item->mask;
+ const struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv6 not supported with ETH/VLAN items");
+ }
+
+ if (rule->mode != RTE_FDIR_MODE_SIGNATURE) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv6 only supported in signature mode");
+ }
+
+ ipv6_mask = item->mask;
+
+ if (ipv6_mask->hdr.vtc_flow ||
+ ipv6_mask->hdr.payload_len ||
+ ipv6_mask->hdr.proto ||
+ ipv6_mask->hdr.hop_limits) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst addresses supported");
+ }
+
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial src address masks not supported");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial dst address masks not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_ipv6(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_ipv6 *ipv6_spec = item->spec;
+ const struct rte_flow_item_ipv6 *ipv6_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+ uint8_t j;
+
+ rule->ixgbe_fdir.formatted.flow_type = IXGBE_ATR_FLOW_TYPE_IPV6;
+
+ /* spec may not be present */
+ if (ipv6_spec) {
+ rule->b_spec = TRUE;
+ memcpy(rule->ixgbe_fdir.formatted.src_ip, &ipv6_spec->hdr.src_addr,
+ sizeof(struct rte_ipv6_addr));
+ memcpy(rule->ixgbe_fdir.formatted.dst_ip, &ipv6_spec->hdr.dst_addr,
+ sizeof(struct rte_ipv6_addr));
+ }
+
+ rule->b_mask = TRUE;
+ for (j = 0; j < sizeof(struct rte_ipv6_addr); j++) {
+ if (ipv6_mask->hdr.src_addr.a[j] == 0)
+ rule->mask.src_ipv6_mask &= ~(1 << j);
+ if (ipv6_mask->hdr.dst_addr.a[j] == 0)
+ rule->mask.dst_ipv6_mask &= ~(1 << j);
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_tcp(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+ const struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "TCP not supported with ETH/VLAN items");
+ }
+
+ if (tcp_mask == NULL)
+ return 0;
+
+ if (tcp_mask->hdr.sent_seq ||
+ tcp_mask->hdr.recv_ack ||
+ tcp_mask->hdr.data_off ||
+ tcp_mask->hdr.tcp_flags ||
+ tcp_mask->hdr.rx_win ||
+ tcp_mask->hdr.cksum ||
+ tcp_mask->hdr.tcp_urp) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst ports supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_tcp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_tcp *tcp_spec = item->spec;
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.flow_type |= IXGBE_ATR_L4TYPE_TCP;
+
+ if (tcp_mask != NULL) {
+ rule->b_mask = TRUE;
+ rule->mask.src_port_mask = tcp_mask->hdr.src_port;
+ rule->mask.dst_port_mask = tcp_mask->hdr.dst_port;
+ }
+
+ if (tcp_spec != NULL) {
+ rule->b_spec = TRUE;
+ rule->ixgbe_fdir.formatted.src_port = tcp_spec->hdr.src_port;
+ rule->ixgbe_fdir.formatted.dst_port = tcp_spec->hdr.dst_port;
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_udp(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+ const struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "UDP not supported with ETH/VLAN items");
+ }
+
+ if (udp_mask == NULL)
+ return 0;
+
+ if (udp_mask->hdr.dgram_len ||
+ udp_mask->hdr.dgram_cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Only src/dst ports supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_udp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_udp *udp_spec = item->spec;
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.flow_type |= IXGBE_ATR_L4TYPE_UDP;
+
+ if (udp_mask != NULL) {
+ rule->b_mask = TRUE;
+ rule->mask.src_port_mask = udp_mask->hdr.src_port;
+ rule->mask.dst_port_mask = udp_mask->hdr.dst_port;
+ }
+
+ if (udp_spec != NULL) {
+ rule->b_spec = TRUE;
+ rule->ixgbe_fdir.formatted.src_port = udp_spec->hdr.src_port;
+ rule->ixgbe_fdir.formatted.dst_port = udp_spec->hdr.dst_port;
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_sctp(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+ const struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ if (rule->mode == RTE_FDIR_MODE_PERFECT_MAC_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "SCTP not supported with ETH/VLAN items");
+ }
+
+ if (sctp_mask == NULL)
+ return 0;
+
+
+ /* Tag and checksum not supported */
+ if (sctp_mask->hdr.tag ||
+ sctp_mask->hdr.cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "SCTP tag/cksum not supported");
+ }
+
+ /*
+ * SCTP mask is not NULL, which means we are potentially looking at
+ * masking SCTP ports, so check hardware support.
+ */
+ if (!fdir_ctx->supports_sctp_ports) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "SCTP port filtering not supported by hardware");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_sctp(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_sctp *sctp_spec = item->spec;
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.flow_type |= IXGBE_ATR_L4TYPE_SCTP;
+
+ if (sctp_mask != NULL) {
+ rule->b_mask = TRUE;
+ rule->mask.src_port_mask = sctp_mask->hdr.src_port;
+ rule->mask.dst_port_mask = sctp_mask->hdr.dst_port;
+ }
+
+ if (sctp_spec != NULL) {
+ rule->b_spec = TRUE;
+ rule->ixgbe_fdir.formatted.src_port = sctp_spec->hdr.src_port;
+ rule->ixgbe_fdir.formatted.dst_port = sctp_spec->hdr.dst_port;
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_normal_raw(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_raw *raw_spec;
+ const struct rte_flow_item_raw *raw_mask;
+
+ raw_spec = item->spec;
+ raw_mask = item->mask;
+
+ if (raw_spec->pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid RAW spec");
+ }
+
+ if (raw_mask->pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid RAW mask");
+ }
+
+ if (raw_mask->length != raw_spec->length &&
+ raw_mask->length != 0xffff) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid RAW mask");
+ }
+
+ if (raw_mask->relative != 0x1 ||
+ raw_mask->search != 0x1 ||
+ raw_mask->reserved != 0x0 ||
+ (uint32_t)raw_mask->offset != 0xffffffff ||
+ raw_mask->limit != 0xffff) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid RAW mask");
+ }
+
+ if (raw_spec->relative != 0 ||
+ raw_spec->search != 0 ||
+ raw_spec->reserved != 0 ||
+ raw_spec->offset > IXGBE_MAX_FLX_SOURCE_OFF ||
+ raw_spec->offset % 2 ||
+ raw_spec->limit != 0 ||
+ raw_spec->length != 2 ||
+ (raw_spec->pattern[0] == 0xff &&
+ raw_spec->pattern[1] == 0xff)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid RAW spec");
+ }
+
+ if (raw_mask->pattern[0] != 0xff ||
+ raw_mask->pattern[1] != 0xff) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW pattern must be fully masked");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_raw(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_raw *raw_spec = item->spec;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->b_spec = TRUE;
+ rule->ixgbe_fdir.formatted.flex_bytes =
+ (((uint16_t)raw_spec->pattern[1]) << 8) | raw_spec->pattern[0];
+ rule->flex_bytes_offset = raw_spec->offset;
+
+ rule->b_mask = TRUE;
+ rule->mask.flex_bytes_mask = 0xffff;
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_normal_end(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+
+ /* check if we need L4 protocol */
+ if (fdir_ctx->rule.ixgbe_fdir.formatted.flow_type & IXGBE_ATR_L4TYPE_MASK)
+ fdir_ctx->rule.mask.l4_proto_match = 1;
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_fdir_normal_graph = {
+ .ignore_nodes = (enum rte_flow_item_type[]) {
+ RTE_FLOW_ITEM_TYPE_FUZZY,
+ RTE_FLOW_ITEM_TYPE_END,
+ },
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_FDIR_NORMAL_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_FDIR_NORMAL_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .validate = ixgbe_validate_fdir_normal_eth,
+ .process = ixgbe_process_fdir_normal_eth,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .process = ixgbe_process_fdir_normal_vlan,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_IPV4] = {
+ .name = "IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .validate = ixgbe_validate_fdir_normal_ipv4,
+ .process = ixgbe_process_fdir_normal_ipv4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_IPV6] = {
+ .name = "IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .validate = ixgbe_validate_fdir_normal_ipv6,
+ .process = ixgbe_process_fdir_normal_ipv6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .validate = ixgbe_validate_fdir_normal_tcp,
+ .process = ixgbe_process_fdir_normal_tcp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .validate = ixgbe_validate_fdir_normal_udp,
+ .process = ixgbe_process_fdir_normal_udp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .validate = ixgbe_validate_fdir_normal_sctp,
+ .process = ixgbe_process_fdir_normal_sctp,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_RAW] = {
+ .name = "RAW",
+ .type = RTE_FLOW_ITEM_TYPE_RAW,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = ixgbe_validate_fdir_normal_raw,
+ .process = ixgbe_process_fdir_normal_raw,
+ },
+ [IXGBE_FDIR_NORMAL_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .process = ixgbe_process_fdir_normal_end,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_FDIR_NORMAL_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_ETH,
+ IXGBE_FDIR_NORMAL_NODE_IPV4,
+ IXGBE_FDIR_NORMAL_NODE_IPV6,
+ IXGBE_FDIR_NORMAL_NODE_TCP,
+ IXGBE_FDIR_NORMAL_NODE_UDP,
+ IXGBE_FDIR_NORMAL_NODE_SCTP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_VLAN,
+ IXGBE_FDIR_NORMAL_NODE_IPV4,
+ IXGBE_FDIR_NORMAL_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_VLAN] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_IPV4] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_TCP,
+ IXGBE_FDIR_NORMAL_NODE_UDP,
+ IXGBE_FDIR_NORMAL_NODE_SCTP,
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_IPV6] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_TCP,
+ IXGBE_FDIR_NORMAL_NODE_UDP,
+ IXGBE_FDIR_NORMAL_NODE_SCTP,
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_TCP] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_UDP] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_SCTP] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_RAW,
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_NORMAL_NODE_RAW] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_NORMAL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * FDIR tunnel graph implementation (VxLAN and NVGRE)
+ * Pattern: START -> [OUTER_ETH] -> (OUTER_IPv4|OUTER_IPv6) -> [UDP] -> (VXLAN|NVGRE) -> INNER_ETH -> [VLAN] -> END
+ * VxLAN: START -> [OUTER_ETH] -> (OUTER_IPv4|OUTER_IPv6) -> UDP -> VXLAN -> INNER_ETH -> [VLAN] -> END
+ * NVGRE: START -> [OUTER_ETH] -> (OUTER_IPv4|OUTER_IPv6) -> NVGRE -> INNER_ETH -> [VLAN] -> END
+ */
+
+enum ixgbe_fdir_tunnel_node_id {
+ IXGBE_FDIR_TUNNEL_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_ETH,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV4,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV6,
+ IXGBE_FDIR_TUNNEL_NODE_UDP,
+ IXGBE_FDIR_TUNNEL_NODE_VXLAN,
+ IXGBE_FDIR_TUNNEL_NODE_NVGRE,
+ IXGBE_FDIR_TUNNEL_NODE_INNER_ETH,
+ IXGBE_FDIR_TUNNEL_NODE_INNER_IPV4,
+ IXGBE_FDIR_TUNNEL_NODE_VLAN,
+ IXGBE_FDIR_TUNNEL_NODE_END,
+ IXGBE_FDIR_TUNNEL_NODE_MAX,
+};
+
+static int
+ixgbe_validate_fdir_tunnel_vxlan(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vxlan *vxlan_mask = item->mask;
+
+ if (vxlan_mask->hdr.flags) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "VxLAN flags must be masked");
+ }
+
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&vxlan_mask->hdr.vni)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial VNI mask not supported");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_tunnel_vxlan(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_vxlan *vxlan_spec = item->spec;
+ const struct rte_flow_item_vxlan *vxlan_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.tunnel_type = IXGBE_FDIR_VXLAN_TUNNEL_TYPE;
+
+ /* spec is optional */
+ if (vxlan_spec != NULL) {
+ rule->b_spec = TRUE;
+ memcpy(((uint8_t *)&rule->ixgbe_fdir.formatted.tni_vni), vxlan_spec->hdr.vni,
+ RTE_DIM(vxlan_spec->hdr.vni));
+ }
+
+ rule->b_mask = TRUE;
+ rule->mask.tunnel_type_mask = 1;
+ memcpy(&rule->mask.tunnel_id_mask, vxlan_mask->hdr.vni, RTE_DIM(vxlan_mask->hdr.vni));
+
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_tunnel_nvgre(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_nvgre *nvgre_mask;
+
+ nvgre_mask = item->mask;
+
+ if (nvgre_mask->flow_id) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "NVGRE flow ID must not be masked");
+ }
+
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&nvgre_mask->protocol)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "NVGRE protocol must be fully masked or unmasked");
+ }
+
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&nvgre_mask->c_k_s_rsvd0_ver)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "NVGRE flags must be fully masked or unmasked");
+ }
+
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&nvgre_mask->tni)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial TNI mask not supported");
+ }
+
+ /* if spec is present, validate flags and protocol values */
+ if (item->spec) {
+ const struct rte_flow_item_nvgre *nvgre_spec = item->spec;
+
+ if (nvgre_mask->c_k_s_rsvd0_ver &&
+ nvgre_spec->c_k_s_rsvd0_ver != rte_cpu_to_be_16(NVGRE_FLAGS)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "NVGRE flags must be 0x2000");
+ }
+ if (nvgre_mask->protocol &&
+ nvgre_spec->protocol != rte_cpu_to_be_16(NVGRE_PROTOCOL)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "NVGRE protocol must be 0x6558");
+ }
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_tunnel_nvgre(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_nvgre *nvgre_spec = item->spec;
+ const struct rte_flow_item_nvgre *nvgre_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.tunnel_type = IXGBE_FDIR_NVGRE_TUNNEL_TYPE;
+
+ /* spec is optional */
+ if (nvgre_spec != NULL) {
+ rule->b_spec = TRUE;
+ memcpy(&fdir_ctx->rule.ixgbe_fdir.formatted.tni_vni,
+ nvgre_spec->tni, RTE_DIM(nvgre_spec->tni));
+ }
+
+ rule->b_mask = TRUE;
+ rule->mask.tunnel_type_mask = 1;
+ memcpy(&rule->mask.tunnel_id_mask, nvgre_mask->tni, RTE_DIM(nvgre_mask->tni));
+ rule->mask.tunnel_id_mask <<= 8;
+ return 0;
+}
+
+static int
+ixgbe_validate_fdir_tunnel_inner_eth(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+
+ if (eth_mask->hdr.ether_type != 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Ether type mask not supported");
+ }
+
+ /* src addr must not be masked */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Masking not supported for src MAC address");
+ }
+
+ /* dst addr must be either fully masked or fully unmasked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(ð_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Partial masks not supported for dst MAC address");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_tunnel_inner_eth(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+ uint8_t j;
+
+ /* spec is optional */
+ if (eth_spec != NULL) {
+ rule->b_spec = TRUE;
+ memcpy(&rule->ixgbe_fdir.formatted.inner_mac,
+ eth_spec->hdr.dst_addr.addr_bytes,
+ RTE_ETHER_ADDR_LEN);
+ }
+
+ rule->b_mask = TRUE;
+ rule->mask.mac_addr_byte_mask = 0;
+ for (j = 0; j < RTE_ETHER_ADDR_LEN; j++) {
+ if (eth_mask->hdr.dst_addr.addr_bytes[j] == 0xFF) {
+ rule->mask.mac_addr_byte_mask |= 0x1 << j;
+ }
+ }
+
+ /* When no vlan, considered as full mask. */
+ rule->mask.vlan_tci_mask = IXGBE_FDIR_VLAN_TCI_MASK;
+
+ return 0;
+}
+
+static int
+ixgbe_process_fdir_tunnel_vlan(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = ctx;
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+
+ rule->ixgbe_fdir.formatted.vlan_id = vlan_spec->hdr.vlan_tci;
+
+ rule->mask.vlan_tci_mask = vlan_mask->hdr.vlan_tci;
+ rule->mask.vlan_tci_mask &= IXGBE_FDIR_VLAN_TCI_MASK;
+
+ return 0;
+}
+
+static const struct flow_graph ixgbe_fdir_tunnel_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [IXGBE_FDIR_TUNNEL_NODE_START] = {
+ .name = "START",
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_ETH] = {
+ .name = "OUTER_ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV4] = {
+ .name = "OUTER_IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV6] = {
+ .name = "OUTER_IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_VXLAN] = {
+ .name = "VXLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VXLAN,
+ .validate = ixgbe_validate_fdir_tunnel_vxlan,
+ .process = ixgbe_process_fdir_tunnel_vxlan,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_NVGRE] = {
+ .name = "NVGRE",
+ .type = RTE_FLOW_ITEM_TYPE_NVGRE,
+ .validate = ixgbe_validate_fdir_tunnel_nvgre,
+ .process = ixgbe_process_fdir_tunnel_nvgre,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_INNER_ETH] = {
+ .name = "INNER_ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .validate = ixgbe_validate_fdir_tunnel_inner_eth,
+ .process = ixgbe_process_fdir_tunnel_inner_eth,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_MASK |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_INNER_IPV4] = {
+ .name = "INNER_IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .process = ixgbe_process_fdir_tunnel_vlan,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [IXGBE_FDIR_TUNNEL_NODE_START] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_ETH,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV4,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV6,
+ IXGBE_FDIR_TUNNEL_NODE_UDP,
+ IXGBE_FDIR_TUNNEL_NODE_VXLAN,
+ IXGBE_FDIR_TUNNEL_NODE_NVGRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV4,
+ IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV4] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_UDP,
+ IXGBE_FDIR_TUNNEL_NODE_NVGRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_OUTER_IPV6] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_UDP,
+ IXGBE_FDIR_TUNNEL_NODE_NVGRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_UDP] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_VXLAN,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_VXLAN] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_INNER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_NVGRE] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_INNER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_INNER_ETH] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_VLAN,
+ IXGBE_FDIR_TUNNEL_NODE_INNER_IPV4,
+ IXGBE_FDIR_TUNNEL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [IXGBE_FDIR_TUNNEL_NODE_VLAN] = {
+ .next = (size_t[]) {
+ IXGBE_FDIR_TUNNEL_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static inline uint8_t
+signature_match(const struct rte_flow_item *item)
+{
+ const struct rte_flow_item_fuzzy *spec, *last, *mask;
+ uint32_t sh, lh, mh;
+
+ spec = item->spec;
+ last = item->last;
+ mask = item->mask;
+
+ if (spec == NULL || mask == NULL)
+ return 0;
+
+ sh = spec->thresh;
+
+ if (last == NULL)
+ lh = sh;
+ else
+ lh = last->thresh;
+
+ mh = mask->thresh;
+ sh = sh & mh;
+ lh = lh & mh;
+
+ /*
+ * A fuzzy item selects signature mode only when the masked threshold range
+ * is non-empty. Otherwise this stays a perfect-match rule.
+ */
+ if (!sh || sh > lh)
+ return 0;
+
+ return 1;
+}
+
+/* pre-parse pattern to determine if this is a signature or perfect match rule */
+static int
+ixgbe_fdir_pattern_parse(struct ci_flow_engine_ctx *ctx,
+ const struct rte_flow_item pattern[],
+ struct rte_flow_error *error)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = (struct ixgbe_fdir_ctx *)ctx;
+ const struct rte_flow_item *item;
+ bool found = false;
+
+ fdir_ctx->rule.mode = RTE_FDIR_MODE_PERFECT;
+
+ for (item = pattern; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->type != RTE_FLOW_ITEM_TYPE_FUZZY)
+ continue;
+
+ if (found) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Multiple FUZZY items not supported");
+ }
+ found = true;
+
+ if (signature_match(item))
+ fdir_ctx->rule.mode = RTE_FDIR_MODE_SIGNATURE;
+
+ break;
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_fdir_actions_check(const struct ci_flow_actions *parsed_actions,
+ const struct ci_flow_actions_check_param *param __rte_unused,
+ struct rte_flow_error *error)
+{
+ const enum rte_flow_action_type fwd_actions[] = {
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_END
+ };
+ const struct rte_flow_action *action;
+
+ /* do the generic checks first */
+ int ret = ixgbe_flow_actions_check(parsed_actions, param, error);
+ if (ret)
+ return ret;
+
+ /* first action must be a forwarding action */
+ action = parsed_actions->actions[0];
+ if (!ci_flow_action_type_in_list(action->type, fwd_actions)) {
+ return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "First action must be QUEUE or DROP");
+ }
+ /* second action, if specified, must not be a forwarding action */
+ action = parsed_actions->actions[1];
+ if (action != NULL && ci_flow_action_type_in_list(action->type, fwd_actions)) {
+ return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Conflicting actions");
+ }
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_ctx_finalize(struct ci_flow_engine_ctx *ctx, struct rte_flow_error *error)
+{
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(ctx->dev_data->dev_private);
+ struct ixgbe_fdir_ctx *fdir_ctx = (struct ixgbe_fdir_ctx *)ctx;
+ struct rte_eth_fdir_conf *global_fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
+
+ /* DROP action should not be used with signature matches */
+ if ((fdir_ctx->rule.mode == RTE_FDIR_MODE_SIGNATURE) &&
+ (fdir_ctx->fwd_action->type == RTE_FLOW_ACTION_TYPE_DROP)) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "DROP action not allowed with signature mode");
+ }
+
+ /* check for conflicting filter modes */
+ if (global_fdir_conf->mode != RTE_FDIR_MODE_NONE &&
+ global_fdir_conf->mode != fdir_ctx->rule.mode) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Conflicting filter modes");
+ }
+
+ /* rules without spec aren't allowed */
+ if (!fdir_ctx->rule.b_spec) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Rule spec cannot be empty");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_ctx_init_common(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = (struct ixgbe_fdir_ctx *)ctx;
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* queue/mark/drop allowed here */
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_MARK,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_fdir_actions_check
+ };
+ struct ixgbe_fdir_rule *rule = &fdir_ctx->rule;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ fdir_ctx->fwd_action = parsed_actions.actions[0];
+ /* can be NULL */
+ fdir_ctx->aux_action = parsed_actions.actions[1];
+
+ /* set up forward/drop action */
+ if (fdir_ctx->fwd_action->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ const struct rte_flow_action_queue *q_act = fdir_ctx->fwd_action->conf;
+ rule->queue = q_act->index;
+ } else {
+ rule->fdirflags = IXGBE_FDIRCMD_DROP;
+ }
+
+ /* set up mark action */
+ if (fdir_ctx->aux_action != NULL && fdir_ctx->aux_action->type == RTE_FLOW_ACTION_TYPE_MARK) {
+ const struct rte_flow_action_mark *m_act = fdir_ctx->aux_action->conf;
+ rule->soft_id = m_act->id;
+ }
+
+ return ret;
+}
+
+static int
+ixgbe_flow_fdir_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(ctx->dev_data->dev_private);
+ struct ixgbe_fdir_ctx *fdir_ctx = (struct ixgbe_fdir_ctx *)ctx;
+ int ret;
+
+ /* call into common part first */
+ ret = ixgbe_flow_fdir_ctx_init_common(actions, attr, ctx, error);
+ if (ret)
+ return ret;
+
+ /* some hardware does not support SCTP matching */
+ if (hw->mac.type == ixgbe_mac_X550 ||
+ hw->mac.type == ixgbe_mac_X550EM_x ||
+ hw->mac.type == ixgbe_mac_X550EM_a ||
+ hw->mac.type == ixgbe_mac_E610)
+ fdir_ctx->supports_sctp_ports = true;
+
+ /*
+ * Some fields may not be provided. Set spec to 0 and mask to default
+ * value. So, we need not do anything for the not provided fields later.
+ */
+ memset(&fdir_ctx->rule.mask, 0xFF, sizeof(struct ixgbe_hw_fdir_mask));
+ fdir_ctx->rule.mask.vlan_tci_mask = 0;
+ fdir_ctx->rule.mask.flex_bytes_mask = 0;
+ fdir_ctx->rule.mask.dst_port_mask = 0;
+ fdir_ctx->rule.mask.src_port_mask = 0;
+ fdir_ctx->rule.mask.l4_proto_match = 0;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_tunnel_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_fdir_ctx *fdir_ctx = (struct ixgbe_fdir_ctx *)ctx;
+ int ret;
+
+ /* call into common part first */
+ ret = ixgbe_flow_fdir_ctx_init_common(actions, attr, ctx, error);
+ if (ret)
+ return ret;
+
+ /**
+ * Some fields may not be provided. Set spec to 0 and mask to default
+ * value. So, we need not do anything for the not provided fields later.
+ */
+ memset(&fdir_ctx->rule.mask, 0xFF, sizeof(struct ixgbe_hw_fdir_mask));
+ fdir_ctx->rule.mask.vlan_tci_mask = 0;
+ fdir_ctx->rule.mask.flex_bytes_mask = 0;
+ fdir_ctx->rule.mask.dst_port_mask = 0;
+ fdir_ctx->rule.mask.src_port_mask = 0;
+ fdir_ctx->rule.mask.l4_proto_match = 0;
+
+ fdir_ctx->rule.mode = RTE_FDIR_MODE_PERFECT_TUNNEL;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(ctx->dev_data->dev_private);
+ const struct rte_eth_fdir_conf *fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
+ const struct ixgbe_fdir_ctx *fdir_ctx = (const struct ixgbe_fdir_ctx *)ctx;
+ struct ixgbe_fdir_flow *fdir_flow = (struct ixgbe_fdir_flow *)flow;
+
+ fdir_flow->rule = fdir_ctx->rule;
+
+ if (fdir_flow->rule.fdirflags & IXGBE_FDIRCMD_DROP) {
+ fdir_flow->fdircmd_flags = IXGBE_FDIRCMD_DROP;
+ fdir_flow->queue = fdir_conf->drop_queue;
+ } else {
+ fdir_flow->fdircmd_flags = 0;
+ fdir_flow->queue = fdir_flow->rule.queue;
+ }
+
+ fdir_flow->fdirhash = ixgbe_fdir_compute_hash(adapter, &fdir_flow->rule);
+
+ return 0;
+}
+
+static bool
+ixgbe_fdir_table_contains(const struct ixgbe_fdir_state *state,
+ const struct ixgbe_fdir_rule *rule)
+{
+ return rte_hash_lookup(state->hash_handle, &rule->ixgbe_fdir) >= 0;
+}
+
+static int
+ixgbe_fdir_table_add(struct ixgbe_fdir_state *state,
+ struct ixgbe_fdir_flow *fdir_flow)
+{
+ if (rte_hash_add_key(state->hash_handle, &fdir_flow->rule.ixgbe_fdir) < 0)
+ return -ENOSPC;
+
+ return 0;
+}
+
+static int
+ixgbe_fdir_table_del(struct ixgbe_fdir_state *state,
+ struct ixgbe_fdir_flow *fdir_flow)
+{
+ if (rte_hash_del_key(state->hash_handle, &fdir_flow->rule.ixgbe_fdir) < 0)
+ return -ENOENT;
+
+ return 0;
+}
+
+/*
+ * The hardware has a single global input mask, so the first filter to carry one
+ * claims it and every later filter has to agree with it.
+ */
+static bool
+ixgbe_fdir_mask_is_compatible(const struct ixgbe_fdir_state *state,
+ const struct ixgbe_fdir_rule *rule)
+{
+ if (!state->mask_added || !rule->b_mask)
+ return true;
+ if (memcmp(&state->mask, &rule->mask, sizeof(state->mask)) != 0)
+ return false;
+
+ return rule->mask.flex_bytes_mask == 0 ||
+ state->flex_bytes_offset == rule->flex_bytes_offset;
+}
+
+/* take a reference on the global mode and mask on behalf of a new filter */
+static void
+ixgbe_fdir_mask_claim(struct ixgbe_fdir_state *state,
+ const struct ixgbe_fdir_rule *rule)
+{
+ if (state->nb_registered == 0)
+ state->mode = rule->mode;
+
+ /*
+ * Take the mask from the first filter so the block gets configured with
+ * something valid, then let the first filter that carries a real mask
+ * claim it for good.
+ */
+ if (state->nb_registered == 0 || (rule->b_mask && !state->mask_added)) {
+ state->mask = rule->mask;
+ state->flex_bytes_offset = rule->flex_bytes_offset;
+ state->mask_added = rule->b_mask;
+ state->mask_programmed = false;
+ }
+ state->nb_registered++;
+}
+
+/* drop a reference; the next filter to be registered gets to pick the mask again */
+static void
+ixgbe_fdir_mask_release(struct ixgbe_fdir_state *state)
+{
+ if (--state->nb_registered > 0)
+ return;
+
+ state->mask = (struct ixgbe_hw_fdir_mask){ 0 };
+ state->flex_bytes_offset = 0;
+ state->mask_added = false;
+ state->mask_programmed = false;
+ state->mode = RTE_FDIR_MODE_NONE;
+}
+
+static int
+ixgbe_flow_fdir_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_fdir_flow *fdir_flow = (struct ixgbe_fdir_flow *)flow;
+ struct ixgbe_fdir_priv *priv = flow->engine_priv;
+ struct ixgbe_fdir_state *state = priv->state;
+ struct ixgbe_fdir_rule *rule = &fdir_flow->rule;
+
+ if (ixgbe_fdir_table_contains(state, rule)) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Conflict with existing flow director filter");
+ }
+
+ if (!ixgbe_fdir_mask_is_compatible(state, rule)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Flow mask is incompatible with existing rules");
+ }
+
+ if (ixgbe_fdir_table_add(state, fdir_flow) != 0) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Flow director filter table is full");
+ }
+
+ ixgbe_fdir_mask_claim(state, rule);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_fdir_flow *fdir_flow = (struct ixgbe_fdir_flow *)flow;
+ struct ixgbe_fdir_priv *priv = flow->engine_priv;
+ struct ixgbe_fdir_state *state = priv->state;
+ int ret;
+
+ ret = ixgbe_fdir_table_del(state, fdir_flow);
+ if (ret == -ENOENT) {
+ return rte_flow_error_set(error, ENOENT,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Flow director filter is missing from the filter table");
+ }
+
+ ixgbe_fdir_mask_release(state);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_configure_hw(struct ixgbe_adapter *adapter,
+ struct ixgbe_fdir_state *state,
+ struct rte_flow_error *error)
+{
+ struct rte_eth_fdir_conf *global_fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
+ struct rte_eth_fdir_conf local_fdir_conf = *global_fdir_conf;
+ int ret;
+
+ local_fdir_conf.mode = state->mode;
+
+ ret = ixgbe_fdir_configure(adapter, &local_fdir_conf, &state->mask);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Failed to configure flow director");
+ }
+
+ global_fdir_conf->mode = state->mode;
+ state->hw_configured = true;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_program_mask(struct ixgbe_adapter *adapter,
+ struct ixgbe_fdir_state *state,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_hw_fdir_info *global_fdir_info = IXGBE_DEV_PRIVATE_TO_FDIR_INFO(adapter);
+ int ret;
+
+ if (state->mask.flex_bytes_mask != 0) {
+ ret = ixgbe_fdir_set_flexbytes_offset(adapter, state->flex_bytes_offset);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Failed to set flex bytes offset");
+ }
+ }
+
+ ret = ixgbe_fdir_set_input_mask(adapter, &state->mask, state->mode);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Failed to set input mask");
+ }
+
+ /* record what is now in hardware for ixgbe_fdir_info_get() */
+ global_fdir_info->mask = state->mask;
+ global_fdir_info->flex_bytes_offset = state->flex_bytes_offset;
+ state->mask_programmed = true;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(flow->dev_data->dev_private);
+ struct ixgbe_fdir_flow *fdir_flow = (struct ixgbe_fdir_flow *)flow;
+ struct ixgbe_fdir_priv *priv = flow->engine_priv;
+ struct ixgbe_fdir_state *state = priv->state;
+ int ret;
+
+ if (!state->hw_configured) {
+ ret = ixgbe_flow_fdir_configure_hw(adapter, state, error);
+ if (ret != 0)
+ return ret;
+ }
+
+ if (!state->mask_programmed) {
+ ret = ixgbe_flow_fdir_program_mask(adapter, state, error);
+ if (ret != 0)
+ return ret;
+ }
+
+ ret = ixgbe_fdir_filter_program(adapter, &fdir_flow->rule, fdir_flow->queue,
+ fdir_flow->fdircmd_flags, fdir_flow->fdirhash);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Failed to program flow director filter");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_fdir_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(flow->dev_data->dev_private);
+ struct rte_eth_fdir_conf *global_fdir_conf = IXGBE_DEV_PRIVATE_TO_FDIR_CONF(adapter);
+ struct ixgbe_fdir_flow *fdir_flow = (struct ixgbe_fdir_flow *)flow;
+ struct ixgbe_fdir_priv *priv = flow->engine_priv;
+ struct ixgbe_fdir_state *state = priv->state;
+ int ret;
+
+ ret = ixgbe_fdir_filter_clear(adapter, fdir_flow->fdirhash);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to remove flow director filter");
+ }
+
+ /* unregister has not run yet, so this filter is still counted */
+ if (state->nb_registered > 1)
+ return 0;
+
+ state->hw_configured = false;
+ state->mask_programmed = false;
+ global_fdir_conf->mode = RTE_FDIR_MODE_NONE;
+
+ ret = ixgbe_fdir_reset_tables(adapter);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to reset flow director tables");
+ }
+
+ return 0;
+}
+
+static void
+ixgbe_flow_fdir_engine_uninit(const struct ci_flow_engine *engine __rte_unused,
+ void *priv)
+{
+ struct ixgbe_fdir_priv *fdir_priv = priv;
+
+ ixgbe_fdir_state_detach(fdir_priv->state);
+}
+
+static int
+ixgbe_flow_fdir_state_attach(struct rte_eth_dev_data *dev_data, void *priv)
+{
+ struct ixgbe_fdir_priv *fdir_priv = priv;
+
+ fdir_priv->state = ixgbe_fdir_state_attach(dev_data);
+
+ return fdir_priv->state == NULL ? -ENOMEM : 0;
+}
+
+static int
+ixgbe_flow_fdir_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ if (hw->mac.type != ixgbe_mac_82599EB &&
+ hw->mac.type != ixgbe_mac_X540 &&
+ hw->mac.type != ixgbe_mac_X550 &&
+ hw->mac.type != ixgbe_mac_X550EM_x &&
+ hw->mac.type != ixgbe_mac_X550EM_a &&
+ hw->mac.type != ixgbe_mac_E610)
+ return -ENOTSUP;
+
+ return ixgbe_flow_fdir_state_attach(dev_data, priv);
+}
+
+static int
+ixgbe_flow_fdir_tunnel_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+
+ if (hw->mac.type != ixgbe_mac_X550 &&
+ hw->mac.type != ixgbe_mac_X550EM_x &&
+ hw->mac.type != ixgbe_mac_X550EM_a &&
+ hw->mac.type != ixgbe_mac_E610)
+ return -ENOTSUP;
+
+ return ixgbe_flow_fdir_state_attach(dev_data, priv);
+}
+
+static const struct ci_flow_engine_ops ixgbe_fdir_ops = {
+ .engine_init = ixgbe_flow_fdir_engine_init,
+ .engine_uninit = ixgbe_flow_fdir_engine_uninit,
+ .ctx_init = ixgbe_flow_fdir_ctx_init,
+ .pattern_parse = ixgbe_fdir_pattern_parse,
+ .ctx_finalize = ixgbe_flow_fdir_ctx_finalize,
+ .ctx_to_flow = ixgbe_flow_fdir_ctx_to_flow,
+ .flow_register = ixgbe_flow_fdir_flow_register,
+ .flow_unregister = ixgbe_flow_fdir_flow_unregister,
+ .flow_install = ixgbe_flow_fdir_flow_install,
+ .flow_uninstall = ixgbe_flow_fdir_flow_uninstall,
+};
+
+static const struct ci_flow_engine_ops ixgbe_fdir_tunnel_ops = {
+ .engine_init = ixgbe_flow_fdir_tunnel_engine_init,
+ .engine_uninit = ixgbe_flow_fdir_engine_uninit,
+ .ctx_init = ixgbe_flow_fdir_tunnel_ctx_init,
+ .ctx_finalize = ixgbe_flow_fdir_ctx_finalize,
+ .ctx_to_flow = ixgbe_flow_fdir_ctx_to_flow,
+ .flow_register = ixgbe_flow_fdir_flow_register,
+ .flow_unregister = ixgbe_flow_fdir_flow_unregister,
+ .flow_install = ixgbe_flow_fdir_flow_install,
+ .flow_uninstall = ixgbe_flow_fdir_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_fdir_flow_engine = {
+ .name = "fdir",
+ .ctx_size = sizeof(struct ixgbe_fdir_ctx),
+ .flow_size = sizeof(struct ixgbe_fdir_flow),
+ .priv_size = sizeof(struct ixgbe_fdir_priv),
+ .ops = &ixgbe_fdir_ops,
+ .graph = &ixgbe_fdir_normal_graph,
+};
+
+const struct ci_flow_engine ixgbe_fdir_tunnel_flow_engine = {
+ .name = "fdir_tunnel",
+ .ctx_size = sizeof(struct ixgbe_fdir_ctx),
+ .flow_size = sizeof(struct ixgbe_fdir_flow),
+ .priv_size = sizeof(struct ixgbe_fdir_priv),
+ .ops = &ixgbe_fdir_tunnel_ops,
+ .graph = &ixgbe_fdir_tunnel_graph,
+};
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 12ba639b70..2487f6a522 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -31,6 +31,7 @@ sources += files(
'ixgbe_flow_l2tun.c',
'ixgbe_flow_ntuple.c',
'ixgbe_flow_security.c',
+ 'ixgbe_flow_fdir.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 11/19] net/ixgbe: reimplement hash parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (9 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 10/19] net/ixgbe: reimplement FDIR parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 12/19] net/ixgbe: advertise flow keep capability Anatoly Burakov
` (8 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
Use the new flow graph API and the common parsing framework to implement
flow parser for RSS configuration.
RSS flow parser doesn't really parse any "flows", it was completely
ignoring the flow patterns and was only looking at actions. It will
therefore not specify a pattern graph and will match NULL pattern, empty
patterns (START -> END), and ANY patterns (START -> ANY -> END).
RSS was the last engine using "filter list", so that is now removed. The
RSS engine filter tracking was also moved completely into the new engine,
and all of the "filter list" infrastructure is removed for good.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 30 --
drivers/net/intel/ixgbe/ixgbe_ethdev.h | 18 +-
drivers/net/intel/ixgbe/ixgbe_flow.c | 415 +---------------------
drivers/net/intel/ixgbe/ixgbe_flow.h | 1 +
drivers/net/intel/ixgbe/ixgbe_flow_hash.c | 212 +++++++++++
drivers/net/intel/ixgbe/ixgbe_rxtx.c | 48 +--
drivers/net/intel/ixgbe/meson.build | 1 +
7 files changed, 235 insertions(+), 490 deletions(-)
create mode 100644 drivers/net/intel/ixgbe/ixgbe_flow_hash.c
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 94af935390..0f2188f6a9 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -345,7 +345,6 @@ static int ixgbe_dev_udp_tunnel_port_add(struct rte_eth_dev *dev,
struct rte_eth_udp_tunnel *udp_tunnel);
static int ixgbe_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
struct rte_eth_udp_tunnel *udp_tunnel);
-static int ixgbe_filter_restore(struct rte_eth_dev *dev);
static void ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev);
static int ixgbe_wait_for_link_up(struct ixgbe_hw *hw);
static int devarg_handle_int(__rte_unused const char *key, const char *value,
@@ -1325,9 +1324,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
if (ret)
goto err_flow_engine_conf_init;
- /* initialize flow filter lists */
- ixgbe_filterlist_init(eth_dev);
-
/* initialize bandwidth configuration info */
memset(bw_conf, 0, sizeof(struct ixgbe_bw_conf));
@@ -2811,7 +2807,6 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
/* resume enabled intr since hw reset */
ixgbe_enable_intr(dev);
ixgbe_l2_tunnel_conf(dev);
- ixgbe_filter_restore(dev);
if (tm_conf->root && !tm_conf->committed)
PMD_DRV_LOG(WARNING,
@@ -3072,9 +3067,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
/* uninitialize PF if max_vfs not zero */
ixgbe_pf_host_uninit(dev);
- /* clear all the filters list */
- ixgbe_filterlist_flush(dev);
-
/* Remove all Traffic Manager configuration */
ixgbe_tm_conf_uninit(dev);
@@ -7832,28 +7824,6 @@ int ixgbe_enable_sec_tx_path_generic(struct ixgbe_hw *hw)
return IXGBE_SUCCESS;
}
-/* restore rss filter */
-static inline void
-ixgbe_rss_filter_restore(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
-
- if (filter_info->rss_info.conf.queue_num)
- ixgbe_config_rss_filter(adapter,
- &filter_info->rss_info, TRUE);
-}
-
-static int
-ixgbe_filter_restore(struct rte_eth_dev *dev)
-{
- ixgbe_rss_filter_restore(dev);
-
- return 0;
-}
-
static void
ixgbe_l2_tunnel_conf(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.h b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
index 8c1f421651..cce8363a70 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.h
@@ -317,8 +317,6 @@ struct ixgbe_ethertype_table {
* Structure to store filters' info.
*/
struct ixgbe_filter_info {
- /* store the rss filter info */
- struct ixgbe_rte_flow_rss_conf rss_info;
/* shared EtherType (ETQF) slot table */
struct ixgbe_ethertype_table ethertype_table;
/* 1588 timestamping ETQF slot (valid when timesync_installed) */
@@ -335,10 +333,9 @@ struct ixgbe_l2_tn_info {
uint16_t e_tag_ether_type; /* ether type for e-tag */
};
+/* no driver-specific data needed */
struct rte_flow {
struct ci_flow flow;
- enum rte_filter_type filter_type;
- void *rule;
};
struct ixgbe_macsec_setting {
@@ -446,9 +443,6 @@ struct ixgbe_tm_conf {
bool committed;
};
-struct ixgbe_filter_ele_base;
-TAILQ_HEAD(ixgbe_filter_ele_list, ixgbe_filter_ele_base);
-
/*
* Structure to store private data for each driver instance (for each port).
*/
@@ -472,7 +466,6 @@ struct ixgbe_adapter {
struct ixgbe_bypass_info bps;
#endif /* RTE_LIBRTE_IXGBE_BYPASS */
struct ixgbe_filter_info filter;
- struct ixgbe_filter_ele_list flow_list;
struct ixgbe_l2_tn_info l2_tn;
struct ixgbe_bw_conf bw_conf;
struct ixgbe_ipsec ipsec;
@@ -694,8 +687,6 @@ ixgbe_e_tag_filter_add(struct ixgbe_adapter *adapter,
int
ixgbe_e_tag_filter_del(struct ixgbe_adapter *adapter,
struct ixgbe_l2_tunnel_conf *l2_tunnel);
-void ixgbe_filterlist_init(struct rte_eth_dev *dev);
-void ixgbe_filterlist_flush(struct rte_eth_dev *dev);
/*
* Flow director function prototypes
*/
@@ -767,10 +758,9 @@ int ixgbe_set_queue_rate_limit(struct rte_eth_dev *dev, uint16_t queue_idx,
uint32_t tx_rate);
int ixgbe_rss_conf_init(struct ixgbe_rte_flow_rss_conf *out,
const struct rte_flow_action_rss *in);
-int ixgbe_action_rss_same(const struct rte_flow_action_rss *comp,
- const struct rte_flow_action_rss *with);
-int ixgbe_config_rss_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_rte_flow_rss_conf *conf, bool add);
+void ixgbe_hw_rss_filter_program(struct ixgbe_hw *hw,
+ struct ixgbe_rte_flow_rss_conf *conf);
+void ixgbe_hw_rss_filter_clear(struct ixgbe_hw *hw);
void ixgbe_dev_macsec_register_enable(struct rte_eth_dev *dev,
struct ixgbe_macsec_setting *macsec_setting);
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 868d65b2a8..f853e71d5c 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -50,21 +50,6 @@
#include "../common/flow_engine.h"
#include "ixgbe_flow.h"
-struct ixgbe_filter_ele_base {
- TAILQ_ENTRY(ixgbe_filter_ele_base) entries;
-};
-
-/* rss filter list structure */
-struct ixgbe_rss_conf_ele {
- struct ixgbe_filter_ele_base base;
- struct ixgbe_rte_flow_rss_conf filter_info;
-};
-/* ixgbe_flow memory list structure */
-struct ixgbe_flow_mem {
- struct ixgbe_filter_ele_base base;
- struct rte_flow *flow;
-};
-
const struct ci_flow_engine_list ixgbe_flow_engine_list = {
{
&ixgbe_ethertype_flow_engine,
@@ -74,6 +59,7 @@ const struct ci_flow_engine_list ixgbe_flow_engine_list = {
&ixgbe_security_flow_engine,
&ixgbe_fdir_flow_engine,
&ixgbe_fdir_tunnel_flow_engine,
+ &ixgbe_hash_flow_engine,
},
};
/*
@@ -131,135 +117,6 @@ ixgbe_flow_actions_check(const struct ci_flow_actions *actions,
* normally the packets should use network order.
*/
-/* Flow actions check specific to RSS filter */
-static int
-ixgbe_flow_actions_check_rss(const struct ci_flow_actions *parsed_actions,
- const struct ci_flow_actions_check_param *param,
- struct rte_flow_error *error)
-{
- const struct rte_flow_action *action = parsed_actions->actions[0];
- const struct rte_flow_action_rss *rss_act = action->conf;
- struct rte_eth_dev_data *dev_data = param->driver_ctx;
- const size_t rss_key_len = sizeof(((struct ixgbe_rte_flow_rss_conf *)0)->key);
- size_t q_idx, q;
-
- /* check if queue list is not empty */
- if (rss_act->queue_num == 0) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS queue list is empty");
- }
-
- /* check if each RSS queue is valid */
- for (q_idx = 0; q_idx < rss_act->queue_num; q_idx++) {
- q = rss_act->queue[q_idx];
- if (q >= dev_data->nb_rx_queues) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "Invalid RSS queue specified");
- }
- }
-
- /* only support default hash function */
- if (rss_act->func != RTE_ETH_HASH_FUNCTION_DEFAULT) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "Non-default RSS hash functions are not supported");
- }
- /* levels aren't supported */
- if (rss_act->level) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "A nonzero RSS encapsulation level is not supported");
- }
- /* check key length */
- if (rss_act->key_len != 0 && rss_act->key_len != rss_key_len) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS key must be exactly 40 bytes long");
- }
- return 0;
-}
-
-static int
-ixgbe_parse_rss_filter(struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_action actions[],
- struct ixgbe_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ap_param = {
- .allowed_types = (const enum rte_flow_action_type[]){
- /* only rss allowed here */
- RTE_FLOW_ACTION_TYPE_RSS,
- RTE_FLOW_ACTION_TYPE_END
- },
- .driver_ctx = dev->data,
- .check = ixgbe_flow_actions_check_rss,
- .max_actions = 1,
- };
- int ret;
- const struct rte_flow_action *action;
-
- /* validate attributes */
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret)
- return ret;
-
- /* parse requested actions */
- ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
- if (ret)
- return ret;
- action = parsed_actions.actions[0];
-
- if (ixgbe_rss_conf_init(rss_conf, action->conf))
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, NULL,
- "RSS context initialization failure");
-
- return 0;
-}
-
-/* remove the rss filter */
-static void
-ixgbe_clear_rss_filter(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
-
- if (filter_info->rss_info.conf.queue_num)
- ixgbe_config_rss_filter(adapter, &filter_info->rss_info, FALSE);
-}
-
-void
-ixgbe_filterlist_init(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
-
- TAILQ_INIT(&adapter->flow_list);
-}
-
-void
-ixgbe_filterlist_flush(struct rte_eth_dev *dev)
-{
- struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_ele_base *ele, *tmp;
-
- RTE_TAILQ_FOREACH_SAFE(ele, &adapter->flow_list, entries, tmp) {
- struct ixgbe_flow_mem *ixgbe_flow_mem_ptr =
- (struct ixgbe_flow_mem *)ele;
- struct rte_flow *flow = ixgbe_flow_mem_ptr->flow;
-
- TAILQ_REMOVE(&adapter->flow_list, ele, entries);
- rte_free(flow->rule);
- rte_free(flow);
- rte_free(ele);
- }
-}
-
/**
* Create or destroy a flow rule.
* Theorically one rule can match more than one filters.
@@ -273,65 +130,9 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
const struct rte_flow_action actions[],
struct rte_flow_error *error)
{
- int ret;
- struct ixgbe_adapter *adapter = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_rte_flow_rss_conf rss_conf;
- struct rte_flow *flow = NULL;
- struct ixgbe_rss_conf_ele *rss_filter_ptr;
- struct ixgbe_flow_mem *ixgbe_flow_mem_ptr;
+ struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- /* try the new flow engine first */
- flow = ci_flow_create(&adapter->flow_engine_conf, attr, pattern, actions, error);
- if (flow != NULL)
- return flow;
-
- /* fall back to legacy flow engines */
-
- flow = rte_zmalloc("ixgbe_rte_flow", sizeof(struct rte_flow), 0);
- if (!flow) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- return (struct rte_flow *)flow;
- }
- ixgbe_flow_mem_ptr = rte_zmalloc("ixgbe_flow_mem",
- sizeof(struct ixgbe_flow_mem), 0);
- if (!ixgbe_flow_mem_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- rte_free(flow);
- return NULL;
- }
- ixgbe_flow_mem_ptr->flow = flow;
- TAILQ_INSERT_TAIL(&adapter->flow_list,
- &ixgbe_flow_mem_ptr->base, entries);
-
- memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
- ret = ixgbe_parse_rss_filter(dev, attr,
- actions, &rss_conf, error);
- if (!ret) {
- ret = ixgbe_config_rss_filter(adapter, &rss_conf, TRUE);
- if (!ret) {
- rss_filter_ptr = rte_zmalloc("ixgbe_rss_filter",
- sizeof(struct ixgbe_rss_conf_ele), 0);
- if (!rss_filter_ptr) {
- PMD_DRV_LOG(ERR, "failed to allocate memory");
- goto out;
- }
- ixgbe_rss_conf_init(&rss_filter_ptr->filter_info,
- &rss_conf.conf);
- flow->rule = rss_filter_ptr;
- flow->filter_type = RTE_ETH_FILTER_HASH;
- return flow;
- }
- }
-
-out:
- TAILQ_REMOVE(&adapter->flow_list,
- &ixgbe_flow_mem_ptr->base, entries);
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to create flow.");
- rte_free(ixgbe_flow_mem_ptr);
- rte_free(flow);
- return NULL;
+ return ci_flow_create(&ad->flow_engine_conf, attr, pattern, actions, error);
}
/**
@@ -347,21 +148,8 @@ ixgbe_flow_validate(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_rte_flow_rss_conf rss_conf;
- int ret;
- /* try the new flow engine first */
- ret = ci_flow_validate(&ad->flow_engine_conf, attr, pattern, actions, error);
- if (ret == 0)
- return ret;
-
- /* fall back to legacy engines */
-
- memset(&rss_conf, 0, sizeof(struct ixgbe_rte_flow_rss_conf));
- ret = ixgbe_parse_rss_filter(dev, attr,
- actions, &rss_conf, error);
-
- return ret;
+ return ci_flow_validate(&ad->flow_engine_conf, attr, pattern, actions, error);
}
/* Destroy a flow rule on ixgbe. */
@@ -370,63 +158,9 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
struct rte_flow *flow,
struct rte_flow_error *error)
{
- int ret;
- struct ixgbe_adapter *adapter =
- IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct rte_flow *pmd_flow = flow;
- enum rte_filter_type filter_type = pmd_flow->filter_type;
- struct ixgbe_filter_ele_base *flow_mem_base;
- struct ixgbe_rss_conf_ele *rss_filter_ptr;
+ struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- /* try the new flow engine first */
- ret = ci_flow_destroy(&adapter->flow_engine_conf, flow, error);
- if (ret == 0)
- return 0;
-
- /* fall back to legacy engines */
-
- /* Validate ownership before touching HW/SW state. */
- TAILQ_FOREACH(flow_mem_base, &adapter->flow_list, entries) {
- struct ixgbe_flow_mem *ixgbe_flow_mem_ptr =
- (struct ixgbe_flow_mem *)flow_mem_base;
-
- if (ixgbe_flow_mem_ptr->flow == pmd_flow)
- break;
- }
- if (flow_mem_base == NULL) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Flow not found for this port");
- }
-
- switch (filter_type) {
- case RTE_ETH_FILTER_HASH:
- rss_filter_ptr = (struct ixgbe_rss_conf_ele *)
- pmd_flow->rule;
- ret = ixgbe_config_rss_filter(adapter,
- &rss_filter_ptr->filter_info, FALSE);
- if (!ret)
- rte_free(rss_filter_ptr);
- break;
- default:
- PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
- filter_type);
- ret = -EINVAL;
- break;
- }
-
- if (ret) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Failed to destroy flow");
- return ret;
- }
-
- TAILQ_REMOVE(&adapter->flow_list, flow_mem_base, entries);
- rte_free(flow_mem_base);
- rte_free(flow);
-
- return ret;
+ return ci_flow_destroy(&ad->flow_engine_conf, flow, error);
}
/* Destroy all flow rules associated with a port on ixgbe. */
@@ -437,95 +171,16 @@ ixgbe_flow_flush(struct rte_eth_dev *dev,
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
int ret = 0;
- /* flush all flows from the new flow engine */
+ /* flush the flow engine */
ret = ci_flow_flush(&ad->flow_engine_conf, error);
if (ret) {
PMD_DRV_LOG(ERR, "Failed to flush flow");
return ret;
}
- ixgbe_clear_rss_filter(dev);
-
- ixgbe_filterlist_flush(dev);
-
return 0;
}
-#define IXGBE_FLOW_DUMP_CHUNK_BYTES 32
-
-static const char *
-ixgbe_flow_rule_engine_name(const struct rte_flow *flow)
-{
- switch (flow->filter_type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- return "ethertype";
- case RTE_ETH_FILTER_SYN:
- return "syn";
- case RTE_ETH_FILTER_FDIR:
- return "fdir";
- case RTE_ETH_FILTER_L2_TUNNEL:
- return "l2_tunnel";
- case RTE_ETH_FILTER_HASH:
- return "hash";
- default:
- return "unknown";
- }
-}
-
-static size_t
-ixgbe_flow_rule_size(const struct rte_flow *flow)
-{
- switch (flow->filter_type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- return sizeof(struct rte_eth_ethertype_filter);
- case RTE_ETH_FILTER_SYN:
- return sizeof(struct rte_eth_syn_filter);
- case RTE_ETH_FILTER_FDIR:
- return sizeof(struct ixgbe_fdir_rule);
- case RTE_ETH_FILTER_L2_TUNNEL:
- return sizeof(struct ixgbe_l2_tunnel_conf);
- case RTE_ETH_FILTER_HASH:
- return sizeof(struct ixgbe_rte_flow_rss_conf);
- default:
- return 0;
- }
-}
-
-static const void *
-ixgbe_flow_rule_data(const struct rte_flow *flow)
-{
- if (flow->rule == NULL)
- return NULL;
-
- return RTE_PTR_ADD(flow->rule, sizeof(struct ixgbe_filter_ele_base));
-}
-
-static void
-ixgbe_flow_dump_blob(FILE *file, const char *engine,
- const void *data, size_t data_len)
-{
- const uint8_t *raw = (const uint8_t *)data;
- const size_t nchunks =
- (data_len + IXGBE_FLOW_DUMP_CHUNK_BYTES - 1) /
- IXGBE_FLOW_DUMP_CHUNK_BYTES;
- char title[64];
- size_t ci;
-
- fprintf(file, "FLOW DUMP: driver=ixgbe engine=%s\n", engine);
- fprintf(file, "FLOW DUMP: DATA size=%zu chunks=%zu chunk_bytes=%d\n",
- data_len, nchunks, IXGBE_FLOW_DUMP_CHUNK_BYTES);
-
- for (ci = 0; ci < nchunks; ci++) {
- const size_t off = ci * IXGBE_FLOW_DUMP_CHUNK_BYTES;
- const size_t clen =
- RTE_MIN((size_t)IXGBE_FLOW_DUMP_CHUNK_BYTES, data_len - off);
-
- snprintf(title, sizeof(title), "FLOW DUMP: chunk %03zu/%03zu",
- ci + 1, nchunks);
- rte_memdump(file, title, raw + off, clen);
- }
-}
-
static int
ixgbe_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow *flow,
@@ -533,62 +188,8 @@ ixgbe_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct ixgbe_adapter *ad = IXGBE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
- struct ixgbe_filter_ele_base *flow_mem_base;
- bool found = false;
- int ret;
- /* try the new flow engine first */
- ret = ci_flow_dump(&ad->flow_engine_conf, flow, file, error);
-
- /*
- * There are multiple possible situations here:
- *
- * - User requested to dump all flows
- * - User requested to dump a specific flow
- *
- * For the first case, we keep going because legacy engines might still
- * have flows we want to dump.
- *
- * For the second case, we only stop if the flow we were asked to dump
- * was found in the new engines, otherwise we keep looking.
- */
- if (flow != NULL && ret == 0)
- return 0;
-
- TAILQ_FOREACH(flow_mem_base, &ad->flow_list, entries) {
- struct ixgbe_flow_mem *ixgbe_flow_mem_ptr =
- (struct ixgbe_flow_mem *)flow_mem_base;
- struct rte_flow *p_flow = ixgbe_flow_mem_ptr->flow;
- const void *rule_data = NULL;
- const char *engine_name;
- size_t rule_size = 0;
-
- if (flow != NULL && p_flow != flow)
- continue;
-
- /* this should not happen */
- if (p_flow->rule == NULL) {
- PMD_DRV_LOG(DEBUG, "Invalid flow");
- continue;
- }
-
- rule_size = ixgbe_flow_rule_size(p_flow);
- if (rule_size == 0)
- continue;
-
- found = true;
- rule_data = ixgbe_flow_rule_data(p_flow);
- engine_name = ixgbe_flow_rule_engine_name(p_flow);
- ixgbe_flow_dump_blob(file, engine_name,
- rule_data, rule_size);
- }
-
- if (flow != NULL && !found)
- return rte_flow_error_set(error, ENOENT,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Flow not found");
-
- return 0;
+ return ci_flow_dump(&ad->flow_engine_conf, flow, file, error);
}
const struct rte_flow_ops ixgbe_flow_ops = {
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.h b/drivers/net/intel/ixgbe/ixgbe_flow.h
index 256e0478f5..959b2c9594 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.h
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.h
@@ -22,5 +22,6 @@ extern const struct ci_flow_engine ixgbe_ntuple_flow_engine;
extern const struct ci_flow_engine ixgbe_security_flow_engine;
extern const struct ci_flow_engine ixgbe_fdir_flow_engine;
extern const struct ci_flow_engine ixgbe_fdir_tunnel_flow_engine;
+extern const struct ci_flow_engine ixgbe_hash_flow_engine;
#endif /* _IXGBE_FLOW_H_ */
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow_hash.c b/drivers/net/intel/ixgbe/ixgbe_flow_hash.c
new file mode 100644
index 0000000000..c1fb893ac2
--- /dev/null
+++ b/drivers/net/intel/ixgbe/ixgbe_flow_hash.c
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include <rte_flow.h>
+#include <flow_graph.h>
+#include <rte_ether.h>
+
+#include "ixgbe_ethdev.h"
+#include "ixgbe_flow.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+#include "../common/flow_engine.h"
+
+struct ixgbe_hash_flow {
+ struct rte_flow flow;
+ struct ixgbe_rte_flow_rss_conf rss_conf;
+};
+
+struct ixgbe_hash_ctx {
+ struct ci_flow_engine_ctx base;
+ struct ixgbe_rte_flow_rss_conf rss_conf;
+};
+
+struct ixgbe_hash_priv {
+ /* the single installed RSS flow, NULL when none is configured */
+ struct ixgbe_hash_flow *active;
+};
+
+/* Flow actions check specific to RSS filter */
+static int
+ixgbe_flow_actions_check_rss(const struct ci_flow_actions *parsed_actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action *action = parsed_actions->actions[0];
+ const struct rte_flow_action_rss *rss_act = action->conf;
+ const struct rte_eth_dev_data *dev_data = param->driver_ctx;
+ const size_t rss_key_len = sizeof(((struct ixgbe_rte_flow_rss_conf *)0)->key);
+ unsigned i;
+
+ /* check if queue list is not empty */
+ if (rss_act->queue_num == 0) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS queue list is empty");
+ }
+
+ /* check if all queues are valid */
+ for (i = 0; i < rss_act->queue_num; i++) {
+ if (rss_act->queue[i] >= dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "Invalid RSS queue specified");
+ }
+ }
+
+ /* only support default hash function */
+ if (rss_act->func != RTE_ETH_HASH_FUNCTION_DEFAULT) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "Non-default RSS hash functions are not supported");
+ }
+ /* levels aren't supported */
+ if (rss_act->level) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "A nonzero RSS encapsulation level is not supported");
+ }
+ /* check key length */
+ if (rss_act->key_len != 0 && rss_act->key_len != rss_key_len) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS key must be exactly 40 bytes long");
+ }
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct ci_flow_actions parsed_actions;
+ struct ci_flow_actions_check_param ap_param = {
+ .allowed_types = (const enum rte_flow_action_type[]){
+ /* only rss allowed here */
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .driver_ctx = ctx->dev_data,
+ .check = ixgbe_flow_actions_check_rss,
+ .max_actions = 1,
+ };
+ struct ixgbe_hash_ctx *hash_ctx = (struct ixgbe_hash_ctx *)ctx;
+ const struct rte_flow_action_rss *rss_conf;
+ int ret;
+
+ /* validate attributes */
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ /* parse requested actions */
+ ret = ci_flow_check_actions(actions, &ap_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ rss_conf = parsed_actions.actions[0]->conf;
+
+ ret = ixgbe_rss_conf_init(&hash_ctx->rss_conf, rss_conf);
+ if (ret) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, rss_conf,
+ "RSS context initialization failure");
+ }
+
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct ixgbe_hash_ctx *hash_ctx = (const struct ixgbe_hash_ctx *)ctx;
+ struct ixgbe_hash_flow *hash_flow = (struct ixgbe_hash_flow *)flow;
+
+ hash_flow->rss_conf = hash_ctx->rss_conf;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct ixgbe_hash_flow *hash_flow = (struct ixgbe_hash_flow *)flow;
+ struct ixgbe_hash_priv *priv = flow->engine_priv;
+
+ /* hardware supports a single RSS filter at a time */
+ if (priv->active != NULL) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "An RSS filter is already configured");
+ }
+
+ priv->active = hash_flow;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_flow_unregister(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct ixgbe_hash_flow *hash_flow = (struct ixgbe_hash_flow *)flow;
+ struct ixgbe_hash_priv *priv = flow->engine_priv;
+
+ if (priv->active != hash_flow) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "RSS filter is not registered");
+ }
+
+ priv->active = NULL;
+
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_hash_flow *hash_flow = (struct ixgbe_hash_flow *)flow;
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_hw_rss_filter_program(hw, &hash_flow->rss_conf);
+
+ return 0;
+}
+
+static int
+ixgbe_flow_hash_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(flow->dev_data->dev_private);
+
+ ixgbe_hw_rss_filter_clear(hw);
+
+ return 0;
+}
+
+static const struct ci_flow_engine_ops ixgbe_hash_ops = {
+ /* RSS engine always available */
+ .ctx_init = ixgbe_flow_hash_ctx_init,
+ .ctx_to_flow = ixgbe_flow_hash_ctx_to_flow,
+ .flow_register = ixgbe_flow_hash_flow_register,
+ .flow_unregister = ixgbe_flow_hash_flow_unregister,
+ .flow_install = ixgbe_flow_hash_flow_install,
+ .flow_uninstall = ixgbe_flow_hash_flow_uninstall,
+};
+
+const struct ci_flow_engine ixgbe_hash_flow_engine = {
+ .name = "hash",
+ .ctx_size = sizeof(struct ixgbe_hash_ctx),
+ .flow_size = sizeof(struct ixgbe_hash_flow),
+ .priv_size = sizeof(struct ixgbe_hash_priv),
+ .ops = &ixgbe_hash_ops,
+ /* RSS does not accept patterns */
+};
diff --git a/drivers/net/intel/ixgbe/ixgbe_rxtx.c b/drivers/net/intel/ixgbe/ixgbe_rxtx.c
index 60222693fe..7e251dff36 100644
--- a/drivers/net/intel/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/intel/ixgbe/ixgbe_rxtx.c
@@ -6117,25 +6117,10 @@ ixgbe_rss_conf_init(struct ixgbe_rte_flow_rss_conf *out,
return 0;
}
-int
-ixgbe_action_rss_same(const struct rte_flow_action_rss *comp,
- const struct rte_flow_action_rss *with)
+void
+ixgbe_hw_rss_filter_program(struct ixgbe_hw *hw,
+ struct ixgbe_rte_flow_rss_conf *conf)
{
- return (comp->func == with->func &&
- comp->level == with->level &&
- comp->types == with->types &&
- comp->key_len == with->key_len &&
- comp->queue_num == with->queue_num &&
- !memcmp(comp->key, with->key, with->key_len) &&
- !memcmp(comp->queue, with->queue,
- sizeof(*with->queue) * with->queue_num));
-}
-
-int
-ixgbe_config_rss_filter(struct ixgbe_adapter *adapter,
- struct ixgbe_rte_flow_rss_conf *conf, bool add)
-{
- struct ixgbe_hw *hw;
uint32_t reta;
uint16_t i;
uint16_t j;
@@ -6147,27 +6132,11 @@ ixgbe_config_rss_filter(struct ixgbe_adapter *adapter,
.rss_key_len = conf->conf.key_len,
.rss_hf = conf->conf.types,
};
- struct ixgbe_filter_info *filter_info =
- IXGBE_DEV_PRIVATE_TO_FILTER_INFO(adapter);
PMD_INIT_FUNC_TRACE();
- hw = IXGBE_DEV_PRIVATE_TO_HW(adapter);
sp_reta_size = ixgbe_reta_size_get(hw->mac.type);
- if (!add) {
- if (ixgbe_action_rss_same(&filter_info->rss_info.conf,
- &conf->conf)) {
- ixgbe_mrqc_rss_remove(hw);
- memset(&filter_info->rss_info, 0,
- sizeof(struct ixgbe_rte_flow_rss_conf));
- return 0;
- }
- return -EINVAL;
- }
-
- if (filter_info->rss_info.conf.queue_num)
- return -EINVAL;
/* Fill in redirection table
* The byte-swap is needed because NIC registers are in
* little-endian order.
@@ -6189,14 +6158,15 @@ ixgbe_config_rss_filter(struct ixgbe_adapter *adapter,
*/
if ((rss_conf.rss_hf & IXGBE_RSS_OFFLOAD_ALL) == 0) {
ixgbe_mrqc_rss_remove(hw);
- return 0;
+ return;
}
if (rss_conf.rss_key == NULL)
rss_conf.rss_key = rss_intel_key; /* Default hash key */
ixgbe_hw_rss_hash_set(hw, &rss_conf);
+}
- if (ixgbe_rss_conf_init(&filter_info->rss_info, &conf->conf))
- return -EINVAL;
-
- return 0;
+void
+ixgbe_hw_rss_filter_clear(struct ixgbe_hw *hw)
+{
+ ixgbe_mrqc_rss_remove(hw);
}
diff --git a/drivers/net/intel/ixgbe/meson.build b/drivers/net/intel/ixgbe/meson.build
index 2487f6a522..7e26700835 100644
--- a/drivers/net/intel/ixgbe/meson.build
+++ b/drivers/net/intel/ixgbe/meson.build
@@ -32,6 +32,7 @@ sources += files(
'ixgbe_flow_ntuple.c',
'ixgbe_flow_security.c',
'ixgbe_flow_fdir.c',
+ 'ixgbe_flow_hash.c',
'ixgbe_ipsec.c',
'ixgbe_pf.c',
'ixgbe_rxtx.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 12/19] net/ixgbe: advertise flow keep capability
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (10 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 11/19] net/ixgbe: reimplement hash parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 13/19] net/i40e: add support for common flow parsing Anatoly Burakov
` (7 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Vladimir Medvedkin
The ixgbe driver's rte_flow implementation has always endeavored to keep
flows across device restart, however it never advertised this capability.
Advertise it explicitly.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 0f2188f6a9..b8bcd12b6a 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -3968,6 +3968,9 @@ ixgbe_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->reta_size = ixgbe_reta_size_get(hw->mac.type);
dev_info->flow_type_rss_offloads = IXGBE_RSS_OFFLOAD_ALL;
+ /* rte_flow rules are kept across dev_stop/dev_start and replayed on start */
+ dev_info->dev_capa |= RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+
dev_info->speed_capa = RTE_ETH_LINK_SPEED_1G | RTE_ETH_LINK_SPEED_10G;
if (hw->device_id == IXGBE_DEV_ID_X550EM_A_1G_T ||
hw->device_id == IXGBE_DEV_ID_X550EM_A_1G_T_L)
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 13/19] net/i40e: add support for common flow parsing
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (11 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 12/19] net/ixgbe: advertise flow keep capability Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 14/19] net/i40e: reimplement ethertype parser Anatoly Burakov
` (6 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Implement support for common flow parsing infrastructure in preparation for
migration of flow engines.
Currently, i40e explicitly clears KEEP_FLOW capability flag, meaning that
it advertises that all flows are flushed on dev stop. However, in
practice i40e actually restores all flows on dev start, so while we will
not (yet) advertise the KEEP_FLOW capability flag, we will also not flush
the flows on dev stop with the new engines, and instead replay them.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 21 +++++++++++
drivers/net/intel/i40e/i40e_ethdev.h | 5 +++
drivers/net/intel/i40e/i40e_flow.c | 53 +++++++++++++++++++++++++++-
drivers/net/intel/i40e/i40e_flow.h | 12 +++++++
4 files changed, 90 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/intel/i40e/i40e_flow.h
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 86cba68430..bff1fbcb5b 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -42,6 +42,9 @@
#include "i40e_regs.h"
#include "rte_pmd_i40e.h"
#include "i40e_hash.h"
+#include "i40e_flow.h"
+
+#include "../common/flow_engine.h"
#define ETH_I40E_FLOATING_VEB_ARG "enable_floating_veb"
#define ETH_I40E_FLOATING_VEB_LIST_ARG "floating_veb_list"
@@ -1838,6 +1841,12 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
if (ret < 0)
goto err_init_fdir_filter_list;
+ /* initialize flow engine configuration */
+ ret = ci_flow_engine_conf_init(&pf->flow_engine_conf,
+ &i40e_flow_engine_list, dev->data);
+ if (ret < 0)
+ goto err_flow_engine_conf_init;
+
/* initialize queue region configuration */
i40e_init_queue_region_conf(dev);
@@ -1846,6 +1855,12 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return 0;
+err_flow_engine_conf_init:
+ rte_free(pf->fdir.fdir_flow_pool.bitmap);
+ rte_free(pf->fdir.fdir_flow_pool.pool);
+ rte_free(pf->fdir.fdir_filter_array);
+ rte_free(pf->fdir.hash_map);
+ rte_hash_free(pf->fdir.hash_table);
err_init_fdir_filter_list:
rte_hash_free(pf->tunnel.hash_table);
rte_free(pf->tunnel.hash_map);
@@ -2613,6 +2628,8 @@ i40e_dev_start(struct rte_eth_dev *dev)
/* Set the max frame size to HW*/
i40e_aq_set_mac_config(hw, max_frame_size, TRUE, false, 0, NULL);
+ ci_flow_replay(&pf->flow_engine_conf);
+
return I40E_SUCCESS;
tx_err:
@@ -2715,6 +2732,10 @@ i40e_dev_close(struct rte_eth_dev *dev)
ret = i40e_dev_stop(dev);
+ /* free the flows and reset flow config */
+ ci_flow_cleanup(&pf->flow_engine_conf);
+ ci_flow_engine_conf_reset(&pf->flow_engine_conf);
+
i40e_dev_free_queues(dev);
/* Disable interrupt */
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index 1e64a2d280..16b67268f7 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -21,6 +21,8 @@
#include "base/i40e_type.h"
#include "base/virtchnl.h"
+#include "../common/flow_engine.h"
+
#define I40E_AQ_LEN 32
#define I40E_AQ_BUF_SZ 4096
/* Number of queues per TC should be one of 1, 2, 4, 8, 16, 32, 64 */
@@ -278,6 +280,7 @@ enum i40e_flxpld_layer_idx {
* Struct to store flow created.
*/
struct rte_flow {
+ struct ci_flow base;
TAILQ_ENTRY(rte_flow) node;
enum rte_filter_type filter_type;
void *rule;
@@ -1182,6 +1185,8 @@ struct i40e_pf {
/* The floating enable flag for the specific VF */
bool floating_veb_list[I40E_MAX_VF];
struct i40e_flow_list flow_list;
+ /* flow engine configuration */
+ struct ci_flow_engine_conf flow_engine_conf;
bool mpls_replace_flag; /* 1 - MPLS filter replace is done */
bool gtp_replace_flag; /* 1 - GTP-C/U filter replace is done */
bool qinq_replace_flag; /* QINQ filter replace is done */
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index a017d3cd44..7d410b6c33 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -26,9 +26,12 @@
#include "base/i40e_prototype.h"
#include "i40e_ethdev.h"
#include "i40e_hash.h"
+#include "i40e_flow.h"
#include "../common/flow_check.h"
+const struct ci_flow_engine_list i40e_flow_engine_list = {0};
+
#define I40E_IPV6_TC_MASK (0xFF << I40E_FDIR_IPv6_TC_OFFSET)
#define I40E_IPV6_FRAG_HEADER 44
#define I40E_TENANT_ARRAY_NUM 3
@@ -1268,6 +1271,25 @@ i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct rte_flow *p_flow;
bool found = false;
+ int ret;
+
+ /* try the new flow engine first */
+ ret = ci_flow_dump(&pf->flow_engine_conf, flow, file, error);
+
+ /*
+ * There are multiple possible situations here:
+ *
+ * - User requested to dump all flows
+ * - User requested to dump a specific flow
+ *
+ * For the first case, we keep going because legacy engines might still
+ * have flows we want to dump.
+ *
+ * For the second case, we only keep going if the flow we were asked to
+ * dump was not found in the new engines.
+ */
+ if (flow != NULL && ret == 0)
+ return 0;
TAILQ_FOREACH(p_flow, &pf->flow_list, node) {
size_t rule_size = 0;
@@ -3892,8 +3914,15 @@ i40e_flow_validate(struct rte_eth_dev *dev,
const struct rte_flow_action actions[],
struct rte_flow_error *error)
{
+ struct i40e_pf *pf = dev->data->dev_private;
/* creates dummy context */
struct i40e_filter_ctx filter_ctx = {0};
+ int ret;
+
+ /* try the new engine first */
+ ret = ci_flow_validate(&pf->flow_engine_conf, attr, pattern, actions, error);
+ if (ret == 0)
+ return 0;
return i40e_flow_check(dev, attr, pattern, actions, &filter_ctx, error);
}
@@ -3911,6 +3940,11 @@ i40e_flow_create(struct rte_eth_dev *dev,
struct i40e_fdir_info *fdir_info = &pf->fdir;
int ret;
+ /* try the new engine first */
+ flow = ci_flow_create(&pf->flow_engine_conf, attr, pattern, actions, error);
+ if (flow != NULL)
+ return flow;
+
ret = i40e_flow_check(dev, attr, pattern, actions, &filter_ctx, error);
if (ret < 0)
return NULL;
@@ -4017,6 +4051,11 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
struct i40e_fdir_info *fdir_info = &pf->fdir;
int ret = 0;
+ /* try the new engine first */
+ ret = ci_flow_destroy(&pf->flow_engine_conf, flow, error);
+ if (ret == 0)
+ return 0;
+
switch (filter_type) {
case RTE_ETH_FILTER_ETHERTYPE:
ret = i40e_flow_destroy_ethertype_filter(pf,
@@ -4161,6 +4200,11 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
int ret;
+ /* flush the new engine first */
+ ret = ci_flow_flush(&pf->flow_engine_conf, error);
+ if (ret != 0)
+ return ret;
+
ret = i40e_flow_flush_fdir_filter(pf);
if (ret) {
rte_flow_error_set(error, -ret,
@@ -4310,14 +4354,21 @@ i40e_flow_flush_tunnel_filter(struct i40e_pf *pf)
}
static int
-i40e_flow_query(struct rte_eth_dev *dev __rte_unused,
+i40e_flow_query(struct rte_eth_dev *dev,
struct rte_flow *flow,
const struct rte_flow_action *actions,
void *data, struct rte_flow_error *error)
{
+ struct i40e_pf *pf = dev->data->dev_private;
struct i40e_rss_filter *rss_rule = (struct i40e_rss_filter *)flow->rule;
enum rte_filter_type filter_type = flow->filter_type;
struct rte_flow_action_rss *rss_conf = data;
+ int ret;
+
+ /* try the new engine first */
+ ret = ci_flow_query(&pf->flow_engine_conf, flow, actions, data, error);
+ if (ret == 0)
+ return 0;
if (!rss_rule) {
rte_flow_error_set(error, EINVAL,
diff --git a/drivers/net/intel/i40e/i40e_flow.h b/drivers/net/intel/i40e/i40e_flow.h
new file mode 100644
index 0000000000..c958868661
--- /dev/null
+++ b/drivers/net/intel/i40e/i40e_flow.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#ifndef _I40E_FLOW_H_
+#define _I40E_FLOW_H_
+
+#include "../common/flow_engine.h"
+
+extern const struct ci_flow_engine_list i40e_flow_engine_list;
+
+#endif /* _I40E_FLOW_H_ */
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 14/19] net/i40e: reimplement ethertype parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (12 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 13/19] net/i40e: add support for common flow parsing Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 15/19] net/i40e: refactor FDIR engine infrastructure Anatoly Burakov
` (5 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Use the new flow graph API and the common parsing framework to implement
flow parser for Ethertype.
The ethertype filter tracking has been moved completely inside the new
engine, and the ethertype code is refactored to enabled that use case by
decoupling hardware writes from software tracking.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 247 +------------
drivers/net/intel/i40e/i40e_ethdev.h | 30 +-
drivers/net/intel/i40e/i40e_flow.c | 298 +---------------
drivers/net/intel/i40e/i40e_flow.h | 4 +
drivers/net/intel/i40e/i40e_flow_ethertype.c | 348 +++++++++++++++++++
drivers/net/intel/i40e/meson.build | 1 +
6 files changed, 367 insertions(+), 561 deletions(-)
create mode 100644 drivers/net/intel/i40e/i40e_flow_ethertype.c
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index bff1fbcb5b..8c009e98ba 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -392,12 +392,6 @@ static int i40e_set_default_mac_addr(struct rte_eth_dev *dev,
static int i40e_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
-static int i40e_ethertype_filter_convert(
- const struct rte_eth_ethertype_filter *input,
- struct i40e_ethertype_filter *filter);
-static int i40e_sw_ethertype_filter_insert(struct i40e_pf *pf,
- struct i40e_ethertype_filter *filter);
-
static int i40e_tunnel_filter_convert(
struct i40e_aqc_cloud_filters_element_bb *cld_filter,
struct i40e_tunnel_filter *tunnel_filter);
@@ -405,7 +399,6 @@ static int i40e_sw_tunnel_filter_insert(struct i40e_pf *pf,
struct i40e_tunnel_filter *tunnel_filter);
static int i40e_cloud_filter_qinq_create(struct i40e_pf *pf);
-static void i40e_ethertype_filter_restore(struct i40e_pf *pf);
static void i40e_tunnel_filter_restore(struct i40e_pf *pf);
static void i40e_filter_restore(struct i40e_pf *pf);
static void i40e_notify_all_vfs_link_status(struct rte_eth_dev *dev);
@@ -1003,51 +996,6 @@ config_floating_veb(struct rte_eth_dev *dev)
#define I40E_L2_TAGS_S_TAG_SHIFT 1
#define I40E_L2_TAGS_S_TAG_MASK I40E_MASK(0x1, I40E_L2_TAGS_S_TAG_SHIFT)
-static int
-i40e_init_ethtype_filter_list(struct rte_eth_dev *dev)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct i40e_ethertype_rule *ethertype_rule = &pf->ethertype;
- char ethertype_hash_name[RTE_HASH_NAMESIZE];
- int ret;
-
- struct rte_hash_parameters ethertype_hash_params = {
- .name = ethertype_hash_name,
- .entries = I40E_MAX_ETHERTYPE_FILTER_NUM,
- .key_len = sizeof(struct i40e_ethertype_filter_input),
- .hash_func = rte_hash_crc,
- .hash_func_init_val = 0,
- .socket_id = rte_socket_id(),
- };
-
- /* Initialize ethertype filter rule list and hash */
- TAILQ_INIT(ðertype_rule->ethertype_list);
- snprintf(ethertype_hash_name, RTE_HASH_NAMESIZE,
- "ethertype_%s", dev->device->name);
- ethertype_rule->hash_table = rte_hash_create(ðertype_hash_params);
- if (!ethertype_rule->hash_table) {
- PMD_INIT_LOG(ERR, "Failed to create ethertype hash table!");
- return -EINVAL;
- }
- ethertype_rule->hash_map = rte_zmalloc("i40e_ethertype_hash_map",
- sizeof(struct i40e_ethertype_filter *) *
- I40E_MAX_ETHERTYPE_FILTER_NUM,
- 0);
- if (!ethertype_rule->hash_map) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for ethertype hash map!");
- ret = -ENOMEM;
- goto err_ethertype_hash_map_alloc;
- }
-
- return 0;
-
-err_ethertype_hash_map_alloc:
- rte_hash_free(ethertype_rule->hash_table);
-
- return ret;
-}
-
static int
i40e_init_tunnel_filter_list(struct rte_eth_dev *dev)
{
@@ -1831,9 +1779,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
/* Initialize the filter invalidation configuration */
i40e_init_filter_invalidation(pf);
- ret = i40e_init_ethtype_filter_list(dev);
- if (ret < 0)
- goto err_init_ethtype_filter_list;
ret = i40e_init_tunnel_filter_list(dev);
if (ret < 0)
goto err_init_tunnel_filter_list;
@@ -1865,9 +1810,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
rte_hash_free(pf->tunnel.hash_table);
rte_free(pf->tunnel.hash_map);
err_init_tunnel_filter_list:
- rte_hash_free(pf->ethertype.hash_table);
- rte_free(pf->ethertype.hash_map);
-err_init_ethtype_filter_list:
rte_intr_callback_unregister(intr_handle,
i40e_dev_interrupt_handler, dev);
rte_free(dev->data->mac_addrs);
@@ -1890,24 +1832,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return ret;
}
-static void
-i40e_rm_ethtype_filter_list(struct i40e_pf *pf)
-{
- struct i40e_ethertype_filter *p_ethertype;
- struct i40e_ethertype_rule *ethertype_rule;
-
- ethertype_rule = &pf->ethertype;
- /* Remove all ethertype filter rules and hash */
- rte_free(ethertype_rule->hash_map);
- rte_hash_free(ethertype_rule->hash_table);
-
- while ((p_ethertype = TAILQ_FIRST(ðertype_rule->ethertype_list))) {
- TAILQ_REMOVE(ðertype_rule->ethertype_list,
- p_ethertype, rules);
- rte_free(p_ethertype);
- }
-}
-
static void
i40e_rm_tunnel_filter_list(struct i40e_pf *pf)
{
@@ -2805,7 +2729,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
i40e_msec_delay(500);
} while (retries++ < 5);
- i40e_rm_ethtype_filter_list(pf);
i40e_rm_tunnel_filter_list(pf);
i40e_rm_fdir_filter_list(pf);
@@ -9907,130 +9830,16 @@ i40e_set_hash_inset(struct i40e_hw *hw, uint64_t input_set,
return 0;
}
-/* Convert ethertype filter structure */
-static int
-i40e_ethertype_filter_convert(const struct rte_eth_ethertype_filter *input,
- struct i40e_ethertype_filter *filter)
-{
- memcpy(&filter->input.mac_addr, &input->mac_addr,
- RTE_ETHER_ADDR_LEN);
- filter->input.ether_type = input->ether_type;
- filter->flags = input->flags;
- filter->queue = input->queue;
-
- return 0;
-}
-
-/* Check if there exists the ethertype filter */
-struct i40e_ethertype_filter *
-i40e_sw_ethertype_filter_lookup(struct i40e_ethertype_rule *ethertype_rule,
- const struct i40e_ethertype_filter_input *input)
-{
- int ret;
-
- ret = rte_hash_lookup(ethertype_rule->hash_table, (const void *)input);
- if (ret < 0)
- return NULL;
-
- return ethertype_rule->hash_map[ret];
-}
-
-/* Add ethertype filter in SW list */
-static int
-i40e_sw_ethertype_filter_insert(struct i40e_pf *pf,
- struct i40e_ethertype_filter *filter)
-{
- struct i40e_ethertype_rule *rule = &pf->ethertype;
- int ret;
-
- ret = rte_hash_add_key(rule->hash_table, &filter->input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to insert ethertype filter"
- " to hash table %d!",
- ret);
- return ret;
- }
- rule->hash_map[ret] = filter;
-
- TAILQ_INSERT_TAIL(&rule->ethertype_list, filter, rules);
-
- return 0;
-}
-
-/* Delete ethertype filter in SW list */
int
-i40e_sw_ethertype_filter_del(struct i40e_pf *pf,
- struct i40e_ethertype_filter_input *input)
-{
- struct i40e_ethertype_rule *rule = &pf->ethertype;
- struct i40e_ethertype_filter *filter;
- int ret;
-
- ret = rte_hash_del_key(rule->hash_table, input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to delete ethertype filter"
- " to hash table %d!",
- ret);
- return ret;
- }
- filter = rule->hash_map[ret];
- rule->hash_map[ret] = NULL;
-
- TAILQ_REMOVE(&rule->ethertype_list, filter, rules);
- rte_free(filter);
-
- return 0;
-}
-
-/*
- * Configure ethertype filter, which can director packet by filtering
- * with mac address and ether_type or only ether_type
- */
-int
-i40e_ethertype_filter_set(struct i40e_pf *pf,
+i40e_ethertype_filter_program(struct i40e_pf *pf,
struct rte_eth_ethertype_filter *filter,
bool add)
{
struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_ethertype_rule *ethertype_rule = &pf->ethertype;
- struct i40e_ethertype_filter *ethertype_filter, *node;
- struct i40e_ethertype_filter check_filter;
struct i40e_control_filter_stats stats;
uint16_t flags = 0;
int ret;
- if (filter->queue >= pf->dev_data->nb_rx_queues) {
- PMD_DRV_LOG(ERR, "Invalid queue ID");
- return -EINVAL;
- }
- if (filter->ether_type == RTE_ETHER_TYPE_IPV4 ||
- filter->ether_type == RTE_ETHER_TYPE_IPV6) {
- PMD_DRV_LOG(ERR,
- "unsupported ether_type(0x%04x) in control packet filter.",
- filter->ether_type);
- return -EINVAL;
- }
- if (filter->ether_type == RTE_ETHER_TYPE_VLAN)
- PMD_DRV_LOG(WARNING,
- "filter vlan ether_type in first tag is not supported.");
-
- /* Check if there is the filter in SW list */
- memset(&check_filter, 0, sizeof(check_filter));
- i40e_ethertype_filter_convert(filter, &check_filter);
- node = i40e_sw_ethertype_filter_lookup(ethertype_rule,
- &check_filter.input);
- if (add && node) {
- PMD_DRV_LOG(ERR, "Conflict with existing ethertype rules!");
- return -EINVAL;
- }
-
- if (!add && !node) {
- PMD_DRV_LOG(ERR, "There's no corresponding ethertype filter!");
- return -EINVAL;
- }
-
if (!(filter->flags & RTE_ETHTYPE_FLAGS_MAC))
flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_IGNORE_MAC;
if (filter->flags & RTE_ETHTYPE_FLAGS_DROP)
@@ -10051,25 +9860,7 @@ i40e_ethertype_filter_set(struct i40e_pf *pf,
if (ret < 0)
return -ENOSYS;
- /* Add or delete a filter in SW list */
- if (add) {
- ethertype_filter = rte_zmalloc("ethertype_filter",
- sizeof(*ethertype_filter), 0);
- if (ethertype_filter == NULL) {
- PMD_DRV_LOG(ERR, "Failed to alloc memory.");
- return -ENOMEM;
- }
-
- memcpy(ethertype_filter, &check_filter,
- sizeof(check_filter));
- ret = i40e_sw_ethertype_filter_insert(pf, ethertype_filter);
- if (ret < 0)
- rte_free(ethertype_filter);
- } else {
- ret = i40e_sw_ethertype_filter_del(pf, &node->input);
- }
-
- return ret;
+ return 0;
}
static int
@@ -11640,39 +11431,6 @@ i40e_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
return 0;
}
-/* Restore ethertype filter */
-static void
-i40e_ethertype_filter_restore(struct i40e_pf *pf)
-{
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_ethertype_filter_list
- *ethertype_list = &pf->ethertype.ethertype_list;
- struct i40e_ethertype_filter *f;
- struct i40e_control_filter_stats stats;
- uint16_t flags;
-
- TAILQ_FOREACH(f, ethertype_list, rules) {
- flags = 0;
- if (!(f->flags & RTE_ETHTYPE_FLAGS_MAC))
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_IGNORE_MAC;
- if (f->flags & RTE_ETHTYPE_FLAGS_DROP)
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_DROP;
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_TO_QUEUE;
-
- memset(&stats, 0, sizeof(stats));
- i40e_aq_add_rem_control_packet_filter(hw,
- f->input.mac_addr.addr_bytes,
- f->input.ether_type,
- flags, pf->main_vsi->seid,
- f->queue, 1, &stats, NULL);
- }
- PMD_DRV_LOG(INFO, "Ethertype filter:"
- " mac_etype_used = %u, etype_used = %u,"
- " mac_etype_free = %u, etype_free = %u",
- stats.mac_etype_used, stats.etype_used,
- stats.mac_etype_free, stats.etype_free);
-}
-
/* Restore tunnel filter */
static void
i40e_tunnel_filter_restore(struct i40e_pf *pf)
@@ -11731,7 +11489,6 @@ i40e_tunnel_filter_restore(struct i40e_pf *pf)
static void
i40e_filter_restore(struct i40e_pf *pf)
{
- i40e_ethertype_filter_restore(pf);
i40e_tunnel_filter_restore(pf);
i40e_fdir_filter_restore(pf);
(void)i40e_hash_filter_restore(pf);
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index 16b67268f7..f674f7995d 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -805,27 +805,6 @@ struct i40e_fdir_info {
/* Ethertype filter number HW supports */
#define I40E_MAX_ETHERTYPE_FILTER_NUM 768
-/* Ethertype filter struct */
-struct i40e_ethertype_filter_input {
- struct rte_ether_addr mac_addr; /* Mac address to match */
- uint16_t ether_type; /* Ether type to match */
-};
-
-struct i40e_ethertype_filter {
- TAILQ_ENTRY(i40e_ethertype_filter) rules;
- struct i40e_ethertype_filter_input input;
- uint16_t flags; /* Flags from RTE_ETHTYPE_FLAGS_* */
- uint16_t queue; /* Queue assigned to when match */
-};
-
-TAILQ_HEAD(i40e_ethertype_filter_list, i40e_ethertype_filter);
-
-struct i40e_ethertype_rule {
- struct i40e_ethertype_filter_list ethertype_list;
- struct i40e_ethertype_filter **hash_map;
- struct rte_hash *hash_table;
-};
-
/* queue region info */
struct i40e_queue_region_info {
/* the region id for this configuration */
@@ -1176,7 +1155,6 @@ struct i40e_pf {
struct i40e_vmdq_info *vmdq;
struct i40e_fdir_info fdir; /* flow director info */
- struct i40e_ethertype_rule ethertype; /* Ethertype filter rule */
struct i40e_tunnel_rule tunnel; /* Tunnel filter rule */
struct i40e_rss_conf_list rss_config_list; /* RSS rule list */
struct i40e_queue_regions queue_region; /* queue region info */
@@ -1325,7 +1303,6 @@ extern const struct rte_flow_ops i40e_flow_ops;
struct i40e_filter_ctx {
union {
- struct rte_eth_ethertype_filter ethertype_filter;
struct i40e_fdir_filter_conf fdir_filter;
struct i40e_tunnel_filter_conf consistent_tunnel_filter;
struct i40e_rte_flow_rss_conf rss_conf;
@@ -1407,11 +1384,6 @@ int i40e_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
-struct i40e_ethertype_filter *
-i40e_sw_ethertype_filter_lookup(struct i40e_ethertype_rule *ethertype_rule,
- const struct i40e_ethertype_filter_input *input);
-int i40e_sw_ethertype_filter_del(struct i40e_pf *pf,
- struct i40e_ethertype_filter_input *input);
int i40e_sw_fdir_filter_del(struct i40e_pf *pf,
struct i40e_fdir_input *input);
struct i40e_tunnel_filter *
@@ -1420,7 +1392,7 @@ i40e_sw_tunnel_filter_lookup(struct i40e_tunnel_rule *tunnel_rule,
int i40e_sw_tunnel_filter_del(struct i40e_pf *pf,
struct i40e_tunnel_filter_input *input);
uint64_t i40e_get_default_input_set(uint16_t pctype);
-int i40e_ethertype_filter_set(struct i40e_pf *pf,
+int i40e_ethertype_filter_program(struct i40e_pf *pf,
struct rte_eth_ethertype_filter *filter,
bool add);
struct rte_flow *
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 7d410b6c33..8f09dfeb11 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -30,7 +30,11 @@
#include "../common/flow_check.h"
-const struct ci_flow_engine_list i40e_flow_engine_list = {0};
+const struct ci_flow_engine_list i40e_flow_engine_list = {
+ {
+ &i40e_flow_engine_ethertype,
+ }
+};
#define I40E_IPV6_TC_MASK (0xFF << I40E_FDIR_IPv6_TC_OFFSET)
#define I40E_IPV6_FRAG_HEADER 44
@@ -60,15 +64,6 @@ static int i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow *flow,
FILE *file,
struct rte_flow_error *error);
-static int
-i40e_flow_parse_ethertype_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct rte_eth_ethertype_filter *filter);
-static int i40e_flow_parse_ethertype_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct rte_eth_ethertype_filter *filter);
static int i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
const struct rte_flow_item *pattern,
struct rte_flow_error *error,
@@ -81,11 +76,6 @@ static int i40e_flow_parse_tunnel_action(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
struct rte_flow_error *error,
struct i40e_tunnel_filter_conf *filter);
-static int i40e_flow_parse_ethertype_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
static int i40e_flow_parse_fdir_filter(struct rte_eth_dev *dev,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
@@ -111,12 +101,9 @@ static int i40e_flow_parse_gtp_filter(struct rte_eth_dev *dev,
const struct rte_flow_action actions[],
struct rte_flow_error *error,
struct i40e_filter_ctx *filter);
-static int i40e_flow_destroy_ethertype_filter(struct i40e_pf *pf,
- struct i40e_ethertype_filter *filter);
static int i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
struct i40e_tunnel_filter *filter);
static int i40e_flow_flush_fdir_filter(struct i40e_pf *pf);
-static int i40e_flow_flush_ethertype_filter(struct i40e_pf *pf);
static int i40e_flow_flush_tunnel_filter(struct i40e_pf *pf);
static int
i40e_flow_parse_qinq_filter(struct rte_eth_dev *dev,
@@ -987,8 +974,6 @@ static enum rte_flow_item_type pattern_fdir_ipv6_udp_esp[] = {
};
static struct i40e_valid_pattern i40e_supported_patterns[] = {
- /* Ethertype */
- { pattern_ethertype, i40e_flow_parse_ethertype_filter },
/* FDIR - support default flow type without flexible payload*/
{ pattern_ethertype, i40e_flow_parse_fdir_filter },
{ pattern_fdir_ipv4, i40e_flow_parse_fdir_filter },
@@ -1206,8 +1191,6 @@ static const char *
i40e_flow_rule_name(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- return "ethertype";
case RTE_ETH_FILTER_FDIR:
return "fdir";
case RTE_ETH_FILTER_TUNNEL:
@@ -1223,8 +1206,6 @@ static size_t
i40e_flow_rule_size(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- return sizeof(struct i40e_ethertype_filter);
case RTE_ETH_FILTER_FDIR:
return sizeof(struct i40e_fdir_filter);
case RTE_ETH_FILTER_TUNNEL:
@@ -1324,11 +1305,11 @@ i40e_flow_dev_dump(struct rte_eth_dev *dev,
return 0;
}
-static int
-i40e_get_outer_vlan(struct rte_eth_dev *dev, uint16_t *tpid)
+int
+i40e_get_outer_vlan(struct i40e_pf *pf, uint16_t *tpid)
{
- struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- int qinq = dev->data->dev_conf.rxmode.offloads &
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
+ int qinq = pf->dev_data->dev_conf.rxmode.offloads &
RTE_ETH_RX_OFFLOAD_VLAN_EXTEND;
uint64_t reg_r = 0;
uint16_t reg_id;
@@ -1351,181 +1332,6 @@ i40e_get_outer_vlan(struct rte_eth_dev *dev, uint16_t *tpid)
return 0;
}
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: MAC_ETHTYPE and ETHTYPE.
- * 3. SRC mac_addr mask should be 00:00:00:00:00:00.
- * 4. DST mac_addr mask should be 00:00:00:00:00:00 or
- * FF:FF:FF:FF:FF:FF
- * 5. Ether_type mask should be 0xFFFF.
- */
-static int
-i40e_flow_parse_ethertype_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct rte_eth_ethertype_filter *filter)
-{
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- enum rte_flow_item_type item_type;
- int ret;
- uint16_t tpid;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- eth_spec = item->spec;
- eth_mask = item->mask;
- /* Get the MAC info. */
- if (!eth_spec || !eth_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "NULL ETH spec/mask");
- return -rte_errno;
- }
-
- /* Mask bits of source MAC address must be full of 0.
- * Mask bits of destination MAC address must be full
- * of 1 or full of 0.
- */
- if (!rte_is_zero_ether_addr(ð_mask->hdr.src_addr) ||
- (!rte_is_zero_ether_addr(ð_mask->hdr.dst_addr) &&
- !rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr))) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid MAC_addr mask");
- return -rte_errno;
- }
-
- if ((eth_mask->hdr.ether_type & UINT16_MAX) != UINT16_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ethertype mask");
- return -rte_errno;
- }
-
- /* If mask bits of destination MAC address
- * are full of 1, set RTE_ETHTYPE_FLAGS_MAC.
- */
- if (rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr)) {
- filter->mac_addr = eth_spec->hdr.dst_addr;
- filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
- } else {
- filter->flags &= ~RTE_ETHTYPE_FLAGS_MAC;
- }
- filter->ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
-
- if (filter->ether_type == RTE_ETHER_TYPE_IPV4 ||
- filter->ether_type == RTE_ETHER_TYPE_IPV6 ||
- filter->ether_type == RTE_ETHER_TYPE_LLDP) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ether_type in control packet filter.");
- return -rte_errno;
- }
-
- ret = i40e_get_outer_vlan(dev, &tpid);
- if (ret != 0) {
- rte_flow_error_set(error, EIO,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Can not get the Ethertype identifying the L2 tag");
- return -rte_errno;
- }
- if (filter->ether_type == tpid) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ether_type in"
- " control packet filter.");
- return -rte_errno;
- }
-
- break;
- default:
- break;
- }
- }
-
- return 0;
-}
-
-/* Ethertype action only supports QUEUE or DROP. */
-static int
-i40e_flow_parse_ethertype_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct rte_eth_ethertype_filter *filter)
-{
- struct ci_flow_actions parsed_actions = {0};
- struct ci_flow_actions_check_param ac_param = {
- .allowed_types = (enum rte_flow_action_type[]) {
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_DROP,
- RTE_FLOW_ACTION_TYPE_END,
- },
- .max_actions = 1,
- };
- const struct rte_flow_action *action;
- int ret;
-
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- action = parsed_actions.actions[0];
-
- if (action->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
- const struct rte_flow_action_queue *act_q = action->conf;
- /* check queue index */
- if (act_q->index >= dev->data->nb_rx_queues) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, action,
- "Invalid queue index");
- }
- filter->queue = act_q->index;
- } else if (action->type == RTE_FLOW_ACTION_TYPE_DROP) {
- filter->flags |= RTE_ETHTYPE_FLAGS_DROP;
- }
- return 0;
-}
-
-static int
-i40e_flow_parse_ethertype_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct rte_eth_ethertype_filter *ethertype_filter = &filter->ethertype_filter;
- int ret;
-
- ret = i40e_flow_parse_ethertype_pattern(dev, pattern, error,
- ethertype_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_ethertype_action(dev, actions, error,
- ethertype_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_ETHERTYPE;
-
- return ret;
-}
-
static int
i40e_flow_check_raw_item(const struct rte_flow_item *item,
const struct rte_flow_item_raw *raw_spec,
@@ -1786,7 +1592,7 @@ i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
"Unsupported ether_type.");
return -rte_errno;
}
- ret = i40e_get_outer_vlan(dev, &tpid);
+ ret = i40e_get_outer_vlan(pf, &tpid);
if (ret != 0) {
rte_flow_error_set(error, EIO,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -1852,7 +1658,7 @@ i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
"Unsupported inner_type.");
return -rte_errno;
}
- ret = i40e_get_outer_vlan(dev, &tpid);
+ ret = i40e_get_outer_vlan(pf, &tpid);
if (ret != 0) {
rte_flow_error_set(error, EIO,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -3991,13 +3797,6 @@ i40e_flow_create(struct rte_eth_dev *dev,
}
switch (filter_ctx.type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- ret = i40e_ethertype_filter_set(pf, &filter_ctx.ethertype_filter, 1);
- if (ret)
- goto free_flow;
- flow->rule = TAILQ_LAST(&pf->ethertype.ethertype_list,
- i40e_ethertype_filter_list);
- break;
case RTE_ETH_FILTER_FDIR:
ret = i40e_flow_add_del_fdir_filter(dev, &filter_ctx.fdir_filter, 1);
if (ret)
@@ -4057,10 +3856,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
return 0;
switch (filter_type) {
- case RTE_ETH_FILTER_ETHERTYPE:
- ret = i40e_flow_destroy_ethertype_filter(pf,
- (struct i40e_ethertype_filter *)flow->rule);
- break;
case RTE_ETH_FILTER_TUNNEL:
ret = i40e_flow_destroy_tunnel_filter(pf,
(struct i40e_tunnel_filter *)flow->rule);
@@ -4100,41 +3895,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
return ret;
}
-static int
-i40e_flow_destroy_ethertype_filter(struct i40e_pf *pf,
- struct i40e_ethertype_filter *filter)
-{
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_ethertype_rule *ethertype_rule = &pf->ethertype;
- struct i40e_ethertype_filter *node;
- struct i40e_control_filter_stats stats;
- uint16_t flags = 0;
- int ret = 0;
-
- if (!(filter->flags & RTE_ETHTYPE_FLAGS_MAC))
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_IGNORE_MAC;
- if (filter->flags & RTE_ETHTYPE_FLAGS_DROP)
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_DROP;
- flags |= I40E_AQC_ADD_CONTROL_PACKET_FLAGS_TO_QUEUE;
-
- memset(&stats, 0, sizeof(stats));
- ret = i40e_aq_add_rem_control_packet_filter(hw,
- filter->input.mac_addr.addr_bytes,
- filter->input.ether_type,
- flags, pf->main_vsi->seid,
- filter->queue, 0, &stats, NULL);
- if (ret < 0)
- return ret;
-
- node = i40e_sw_ethertype_filter_lookup(ethertype_rule, &filter->input);
- if (!node)
- return -EINVAL;
-
- ret = i40e_sw_ethertype_filter_del(pf, &node->input);
-
- return ret;
-}
-
static int
i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
struct i40e_tunnel_filter *filter)
@@ -4213,14 +3973,6 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
return -rte_errno;
}
- ret = i40e_flow_flush_ethertype_filter(pf);
- if (ret) {
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to ethertype flush flows.");
- return -rte_errno;
- }
-
ret = i40e_flow_flush_tunnel_filter(pf);
if (ret) {
rte_flow_error_set(error, -ret,
@@ -4297,34 +4049,6 @@ i40e_flow_flush_fdir_filter(struct i40e_pf *pf)
return ret;
}
-/* Flush all ethertype filters */
-static int
-i40e_flow_flush_ethertype_filter(struct i40e_pf *pf)
-{
- struct i40e_ethertype_filter_list
- *ethertype_list = &pf->ethertype.ethertype_list;
- struct i40e_ethertype_filter *filter;
- struct rte_flow *flow;
- void *temp;
- int ret = 0;
-
- while ((filter = TAILQ_FIRST(ethertype_list))) {
- ret = i40e_flow_destroy_ethertype_filter(pf, filter);
- if (ret)
- return ret;
- }
-
- /* Delete ethertype flows in flow list. */
- RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, temp) {
- if (flow->filter_type == RTE_ETH_FILTER_ETHERTYPE) {
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- rte_free(flow);
- }
- }
-
- return ret;
-}
-
/* Flush all tunnel filters */
static int
i40e_flow_flush_tunnel_filter(struct i40e_pf *pf)
diff --git a/drivers/net/intel/i40e/i40e_flow.h b/drivers/net/intel/i40e/i40e_flow.h
index c958868661..11d13a76fe 100644
--- a/drivers/net/intel/i40e/i40e_flow.h
+++ b/drivers/net/intel/i40e/i40e_flow.h
@@ -7,6 +7,10 @@
#include "../common/flow_engine.h"
+int i40e_get_outer_vlan(struct i40e_pf *pf, uint16_t *tpid);
+
extern const struct ci_flow_engine_list i40e_flow_engine_list;
+extern const struct ci_flow_engine i40e_flow_engine_ethertype;
+
#endif /* _I40E_FLOW_H_ */
diff --git a/drivers/net/intel/i40e/i40e_flow_ethertype.c b/drivers/net/intel/i40e/i40e_flow_ethertype.c
new file mode 100644
index 0000000000..8d28dfc7d6
--- /dev/null
+++ b/drivers/net/intel/i40e/i40e_flow_ethertype.c
@@ -0,0 +1,348 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include <rte_hash.h>
+#include <rte_hash_crc.h>
+
+#include "i40e_ethdev.h"
+#include "i40e_flow.h"
+
+#include "../common/flow_engine.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+
+struct i40e_ethertype_ctx {
+ struct ci_flow_engine_ctx base;
+ struct rte_eth_ethertype_filter ethertype;
+};
+
+struct i40e_ethertype_flow {
+ struct rte_flow base;
+ struct rte_eth_ethertype_filter ethertype;
+};
+
+/* leading fields of struct rte_eth_ethertype_filter, used as the dedup hash key */
+struct i40e_ethertype_key {
+ struct rte_ether_addr mac_addr;
+ uint16_t ether_type;
+};
+
+struct i40e_ethertype_priv {
+ struct rte_hash *hash_table;
+};
+
+/**
+ * Ethertype filter graph implementation
+ * Pattern: START -> ETH -> END
+ */
+
+enum i40e_ethertype_node_id {
+ I40E_ETHERTYPE_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_ETHERTYPE_NODE_ETH,
+ I40E_ETHERTYPE_NODE_END,
+ I40E_ETHERTYPE_NODE_MAX,
+};
+
+static int
+i40e_ethertype_node_eth_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item, struct rte_flow_error *error)
+{
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ uint16_t ether_type;
+
+ /* Source MAC mask must be all zeros */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Source MAC filtering not supported");
+ }
+
+ /* Dest MAC mask must be all zeros or all ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(ð_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Dest MAC filtering not supported");
+ }
+
+ /* Ethertype mask must be exact match */
+ if (!CI_FIELD_IS_MASKED(ð_mask->hdr.ether_type)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Ethertype must be exactly matched");
+ }
+
+ /* Check for valid ethertype (not IPv4/IPv6/LLDP/VLAN) */
+ ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+ if (ether_type == RTE_ETHER_TYPE_IPV4 ||
+ ether_type == RTE_ETHER_TYPE_IPV6 ||
+ ether_type == RTE_ETHER_TYPE_LLDP ||
+ ether_type == RTE_ETHER_TYPE_VLAN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4/IPv6/LLDP/VLAN not supported by ethertype filter");
+ }
+
+ return 0;
+}
+
+static int
+i40e_ethertype_node_eth_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ struct i40e_ethertype_ctx *ethertype_ctx = ctx;
+ struct rte_eth_ethertype_filter *filter = ðertype_ctx->ethertype;
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ uint16_t ether_type, tpid;
+ /* pf cannot be const so it's here rather than in validate() */
+ struct rte_eth_dev_data *dev_data = ethertype_ctx->base.dev_data;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+
+ ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+
+ if (CI_FIELD_IS_MASKED(ð_mask->hdr.dst_addr)) {
+ filter->mac_addr = eth_spec->hdr.dst_addr;
+ filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
+ }
+
+ /* Cannot match currently installed VLAN ethertype */
+ if (i40e_get_outer_vlan(pf, &tpid) != 0) {
+ return rte_flow_error_set(error, EIO,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Can not get the Ethertype identifying the L2 tag");
+ }
+ if (ether_type == tpid) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported ether_type in control packet filter.");
+ }
+
+ filter->ether_type = ether_type;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_ethertype_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_ETHERTYPE_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_ETHERTYPE_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_ethertype_node_eth_validate,
+ .process = i40e_ethertype_node_eth_process,
+ },
+ [I40E_ETHERTYPE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_ETHERTYPE_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_ETHERTYPE_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_ETHERTYPE_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_ETHERTYPE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+i40e_flow_ethertype_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_ethertype_ctx *ethertype_ctx = (struct i40e_ethertype_ctx *)ctx;
+ struct rte_eth_dev_data *dev_data = ethertype_ctx->base.dev_data;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param ac_param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_END,
+ },
+ .max_actions = 1,
+ };
+ const struct rte_flow_action *action;
+ int ret;
+
+ ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ action = parsed_actions.actions[0];
+
+ if (action->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ const struct rte_flow_action_queue *act_q = action->conf;
+ /* check queue index */
+ if (act_q->index >= dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Invalid queue index");
+ }
+ ethertype_ctx->ethertype.queue = act_q->index;
+ } else if (action->type == RTE_FLOW_ACTION_TYPE_DROP) {
+ ethertype_ctx->ethertype.flags |= RTE_ETHTYPE_FLAGS_DROP;
+ }
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct i40e_ethertype_ctx *ethertype_ctx = (const struct i40e_ethertype_ctx *)ctx;
+ struct i40e_ethertype_flow *ethertype_flow = (struct i40e_ethertype_flow *)flow;
+
+ /* copy ethertype filter configuration to flow */
+ ethertype_flow->ethertype = ethertype_ctx->ethertype;
+
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_ethertype_flow *ethertype_flow = (struct i40e_ethertype_flow *)flow;
+ struct i40e_ethertype_priv *priv = flow->engine_priv;
+ struct i40e_ethertype_key key;
+ int ret;
+
+ memcpy(&key, ðertype_flow->ethertype, sizeof(key));
+
+ if (rte_hash_lookup(priv->hash_table, &key) >= 0) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Conflict with existing ethertype filter");
+ }
+
+ ret = rte_hash_add_key(priv->hash_table, &key);
+ if (ret < 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to register ethertype filter");
+ }
+
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_ethertype_flow *ethertype_flow = (struct i40e_ethertype_flow *)flow;
+ struct i40e_ethertype_priv *priv = flow->engine_priv;
+ struct i40e_ethertype_key key;
+ int ret;
+
+ memcpy(&key, ðertype_flow->ethertype, sizeof(key));
+
+ ret = rte_hash_del_key(priv->hash_table, &key);
+ if (ret < 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Ethertype filter not found on unregister");
+ }
+
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_install(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_ethertype_flow *ethertype_flow = (struct i40e_ethertype_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ int ret;
+
+ ret = i40e_ethertype_filter_program(pf, ðertype_flow->ethertype, true);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to install ethertype filter");
+ }
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_uninstall(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_ethertype_flow *ethertype_flow = (struct i40e_ethertype_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ int ret;
+
+ ret = i40e_ethertype_filter_program(pf, ðertype_flow->ethertype, false);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to delete ethertype filter");
+ }
+ return 0;
+}
+
+static int
+i40e_flow_ethertype_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data, void *priv)
+{
+ char ethertype_hash_name[RTE_HASH_NAMESIZE];
+ struct i40e_ethertype_priv *etype_priv = priv;
+ struct rte_hash_parameters params = {
+ .name = ethertype_hash_name,
+ .entries = I40E_MAX_ETHERTYPE_FILTER_NUM,
+ .key_len = sizeof(struct i40e_ethertype_key),
+ .hash_func = rte_hash_crc,
+ .hash_func_init_val = 0,
+ .socket_id = rte_socket_id(),
+ };
+
+ snprintf(ethertype_hash_name, RTE_HASH_NAMESIZE, "i40e_ethertype_hash_%p", dev_data->name);
+
+ etype_priv->hash_table = rte_hash_create(¶ms);
+ if (etype_priv->hash_table == NULL)
+ return -rte_errno;
+
+ return 0;
+}
+
+static void
+i40e_flow_ethertype_engine_uninit(const struct ci_flow_engine *engine __rte_unused,
+ void *priv)
+{
+ struct i40e_ethertype_priv *etype_priv = priv;
+
+ rte_hash_free(etype_priv->hash_table);
+}
+
+static const struct ci_flow_engine_ops i40e_flow_engine_ethertype_ops = {
+ .engine_init = i40e_flow_ethertype_engine_init,
+ .engine_uninit = i40e_flow_ethertype_engine_uninit,
+ .ctx_init = i40e_flow_ethertype_ctx_init,
+ .ctx_to_flow = i40e_flow_ethertype_ctx_to_flow,
+ .flow_register = i40e_flow_ethertype_register,
+ .flow_unregister = i40e_flow_ethertype_unregister,
+ .flow_install = i40e_flow_ethertype_install,
+ .flow_uninstall = i40e_flow_ethertype_uninstall,
+};
+
+const struct ci_flow_engine i40e_flow_engine_ethertype = {
+ .name = "ethertype",
+ .ctx_size = sizeof(struct i40e_ethertype_ctx),
+ .flow_size = sizeof(struct i40e_ethertype_flow),
+ .priv_size = sizeof(struct i40e_ethertype_priv),
+ .ops = &i40e_flow_engine_ethertype_ops,
+ .graph = &i40e_ethertype_graph,
+};
diff --git a/drivers/net/intel/i40e/meson.build b/drivers/net/intel/i40e/meson.build
index 3229233f50..ddc97f9b3b 100644
--- a/drivers/net/intel/i40e/meson.build
+++ b/drivers/net/intel/i40e/meson.build
@@ -33,6 +33,7 @@ sources += files(
'i40e_pf.c',
'i40e_fdir.c',
'i40e_flow.c',
+ 'i40e_flow_ethertype.c',
'i40e_tm.c',
'i40e_hash.c',
'i40e_vf_representor.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 15/19] net/i40e: refactor FDIR engine infrastructure
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (13 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 14/19] net/i40e: reimplement ethertype parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 16/19] net/i40e: reimplement FDIR parser Anatoly Burakov
` (4 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Currently, there are multiple problems with how i40e flow directory feature
is implemented, both in terms of how it works with rte_flow, and how it
integrates with the PMD-specific packet template API.
For one, these two subsystems, while using shared infrastructure, do not
really interact or cooperate, and are built on top of special cases in FDIR
path. More specifically, the packet template code does not store its
packet in the hash map, and has a different hashing scheme, yet it still
registers itself in FDIR flow list, hash table, and hash map. This list
is then used by `dev_start` to restore FDIR filters that user has
inserted into the list. These filters, as written, cannot be reprogrammed
that way because the information a filter restore function would need is
lost on insert (the packet pointer is not added to the hash map).
Another issue is that while rte_flow FDIR code does lazy FDIR init on first
added flow, the packet template API does not, even though it too relies on
the same hardware feature, nor does it ever do teardown on last FDIR flow.
Yet another issue is how the "filter restore" code itself is implemented,
namely that currently it simply does not work. When doing filter restore,
the driver will walk every FDIR filter stored in the TAILQ, and attempt to
program it. However, inside the program function, there is a deduplication
check (to see if flow being installed is already present in the flow hash
table), which fails because the flows we are programming come from the same
list that is being checked for deduplication, which makes the entire filter
restore a no-op.
The FDIR filter programming code itself also has a number of readability
problems as well as being otherwise hard to use - SW bookkeeping,
validation, and flow programming is interspersed within the code, and it
is difficult to reason about what happens when the code is called from
this or that context.
So, this refactor does the following:
- Reorganize FDIR internals to track packet templates and rte_flow FDIR
flows separately
- Refactor FDIR init/teardown to always happen on first/last rule, so that
whichever API happens to call FDIR first, the state is consistent
- Rework the FDIR code to disentangle FDIR flow rule programming, Flex PIT
checks, SW bookkeeping, etc. from each other
- Remove both the rte_flow FDIR TAILQ and the hash map (filter array)
structure, because they are redundant (information about the flow is
already stored in the rte_flow flow list, and hash_map structure only
stored pointers to data we also have in that same list)
- Fix FDIR filter restore to replay all configuration correctly, as well as
re-init the FDIR queue enablement tracking
- Rework the internal FDIR global state data structure to make a little
more sense by grouping things that belong together into structures
Additionally, there was a delay mechanism at flow director rule program
time, as when programming a rule we might not know if it's actually
possible to install the rule, because space for the rules may come either
from our own pool, or it may come from a pool that is shared with other
VSI's. However, it only makes sense to wait on rule create (i.e. when it is
programmed into the hardware for the first time), but not when we are
replaying or removing these rules. So, adjust the waiting mechanism to only
wait on FDIR rule creation.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 168 +---
drivers/net/intel/i40e/i40e_ethdev.h | 126 ++-
drivers/net/intel/i40e/i40e_fdir.c | 1134 +++++++++++++++++--------
drivers/net/intel/i40e/i40e_flow.c | 131 +--
drivers/net/intel/i40e/rte_pmd_i40e.c | 4 +-
5 files changed, 928 insertions(+), 635 deletions(-)
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 8c009e98ba..25062fe695 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -1041,129 +1041,6 @@ i40e_init_tunnel_filter_list(struct rte_eth_dev *dev)
return ret;
}
-static int
-i40e_init_fdir_filter_list(struct rte_eth_dev *dev)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_fdir_info *fdir_info = &pf->fdir;
- char fdir_hash_name[RTE_HASH_NAMESIZE];
- uint32_t alloc = hw->func_caps.fd_filters_guaranteed;
- uint32_t best = hw->func_caps.fd_filters_best_effort;
- enum i40e_filter_pctype pctype;
- struct rte_bitmap *bmp = NULL;
- uint32_t bmp_size;
- void *mem = NULL;
- uint32_t i = 0;
- int ret;
-
- struct rte_hash_parameters fdir_hash_params = {
- .name = fdir_hash_name,
- .entries = I40E_MAX_FDIR_FILTER_NUM,
- .key_len = sizeof(struct i40e_fdir_input),
- .hash_func = rte_hash_crc,
- .hash_func_init_val = 0,
- .socket_id = rte_socket_id(),
- };
-
- /* Initialize flow director filter rule list and hash */
- TAILQ_INIT(&fdir_info->fdir_list);
- snprintf(fdir_hash_name, RTE_HASH_NAMESIZE,
- "fdir_%s", dev->device->name);
- fdir_info->hash_table = rte_hash_create(&fdir_hash_params);
- if (!fdir_info->hash_table) {
- PMD_INIT_LOG(ERR, "Failed to create fdir hash table!");
- return -EINVAL;
- }
-
- fdir_info->hash_map = rte_zmalloc("i40e_fdir_hash_map",
- sizeof(struct i40e_fdir_filter *) *
- I40E_MAX_FDIR_FILTER_NUM,
- 0);
- if (!fdir_info->hash_map) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for fdir hash map!");
- ret = -ENOMEM;
- goto err_fdir_hash_map_alloc;
- }
-
- fdir_info->fdir_filter_array = rte_zmalloc("fdir_filter",
- sizeof(struct i40e_fdir_filter) *
- I40E_MAX_FDIR_FILTER_NUM,
- 0);
-
- if (!fdir_info->fdir_filter_array) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for fdir filter array!");
- ret = -ENOMEM;
- goto err_fdir_filter_array_alloc;
- }
-
- for (pctype = I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
- pctype <= I40E_FILTER_PCTYPE_L2_PAYLOAD; pctype++)
- pf->fdir.flow_count[pctype] = 0;
-
- fdir_info->fdir_space_size = alloc + best;
- fdir_info->fdir_actual_cnt = 0;
- fdir_info->fdir_guarantee_total_space = alloc;
- fdir_info->fdir_guarantee_free_space =
- fdir_info->fdir_guarantee_total_space;
-
- PMD_DRV_LOG(INFO, "FDIR guarantee space: %u, best_effort space %u.", alloc, best);
-
- fdir_info->fdir_flow_pool.pool =
- rte_zmalloc("i40e_fdir_entry",
- sizeof(struct i40e_fdir_entry) *
- fdir_info->fdir_space_size,
- 0);
-
- if (!fdir_info->fdir_flow_pool.pool) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for bitmap flow!");
- ret = -ENOMEM;
- goto err_fdir_bitmap_flow_alloc;
- }
-
- for (i = 0; i < fdir_info->fdir_space_size; i++)
- fdir_info->fdir_flow_pool.pool[i].idx = i;
-
- bmp_size =
- rte_bitmap_get_memory_footprint(fdir_info->fdir_space_size);
- mem = rte_zmalloc("fdir_bmap", bmp_size, RTE_CACHE_LINE_SIZE);
- if (mem == NULL) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for fdir bitmap!");
- ret = -ENOMEM;
- goto err_fdir_mem_alloc;
- }
- bmp = rte_bitmap_init(fdir_info->fdir_space_size, mem, bmp_size);
- if (bmp == NULL) {
- PMD_INIT_LOG(ERR,
- "Failed to initialization fdir bitmap!");
- ret = -ENOMEM;
- goto err_fdir_bmp_alloc;
- }
- for (i = 0; i < fdir_info->fdir_space_size; i++)
- rte_bitmap_set(bmp, i);
-
- fdir_info->fdir_flow_pool.bitmap = bmp;
-
- return 0;
-
-err_fdir_bmp_alloc:
- rte_free(mem);
-err_fdir_mem_alloc:
- rte_free(fdir_info->fdir_flow_pool.pool);
-err_fdir_bitmap_flow_alloc:
- rte_free(fdir_info->fdir_filter_array);
-err_fdir_filter_array_alloc:
- rte_free(fdir_info->hash_map);
-err_fdir_hash_map_alloc:
- rte_hash_free(fdir_info->hash_table);
-
- return ret;
-}
-
static void
i40e_init_customized_info(struct i40e_pf *pf)
{
@@ -1782,9 +1659,9 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
ret = i40e_init_tunnel_filter_list(dev);
if (ret < 0)
goto err_init_tunnel_filter_list;
- ret = i40e_init_fdir_filter_list(dev);
+ ret = i40e_fdir_flow_store_init(dev);
if (ret < 0)
- goto err_init_fdir_filter_list;
+ goto err_init_fdir_flow_store;
/* initialize flow engine configuration */
ret = ci_flow_engine_conf_init(&pf->flow_engine_conf,
@@ -1801,12 +1678,8 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return 0;
err_flow_engine_conf_init:
- rte_free(pf->fdir.fdir_flow_pool.bitmap);
- rte_free(pf->fdir.fdir_flow_pool.pool);
- rte_free(pf->fdir.fdir_filter_array);
- rte_free(pf->fdir.hash_map);
- rte_hash_free(pf->fdir.hash_table);
-err_init_fdir_filter_list:
+ i40e_fdir_flow_store_free(&pf->fdir);
+err_init_fdir_flow_store:
rte_hash_free(pf->tunnel.hash_table);
rte_free(pf->tunnel.hash_map);
err_init_tunnel_filter_list:
@@ -1849,32 +1722,13 @@ i40e_rm_tunnel_filter_list(struct i40e_pf *pf)
}
}
-static void
-i40e_rm_fdir_filter_list(struct i40e_pf *pf)
-{
- struct i40e_fdir_filter *p_fdir;
- struct i40e_fdir_info *fdir_info;
-
- fdir_info = &pf->fdir;
-
- /* Remove all flow director rules */
- while ((p_fdir = TAILQ_FIRST(&fdir_info->fdir_list)))
- TAILQ_REMOVE(&fdir_info->fdir_list, p_fdir, rules);
-}
-
static void
i40e_fdir_memory_cleanup(struct i40e_pf *pf)
{
- struct i40e_fdir_info *fdir_info;
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
- fdir_info = &pf->fdir;
-
- /* flow director memory cleanup */
- rte_free(fdir_info->hash_map);
- rte_hash_free(fdir_info->hash_table);
- rte_free(fdir_info->fdir_flow_pool.bitmap);
- rte_free(fdir_info->fdir_flow_pool.pool);
- rte_free(fdir_info->fdir_filter_array);
+ i40e_fdir_flow_store_free(fdir_info);
+ i40e_fdir_tmpl_store_free(fdir_info);
}
void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
@@ -2730,7 +2584,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
} while (retries++ < 5);
i40e_rm_tunnel_filter_list(pf);
- i40e_rm_fdir_filter_list(pf);
/* Remove all flows */
while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
@@ -9774,7 +9627,12 @@ i40e_filter_input_set_init(struct i40e_pf *pf)
/* store the default input set */
if (!pf->support_multi_driver)
pf->hash_input_set[pctype] = input_set;
- pf->fdir.input_set[pctype] = input_set;
+ {
+ struct i40e_fdir_pctype_state *state =
+ &pf->fdir.flows.pctype[pctype];
+
+ state->input_set = input_set;
+ }
}
}
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index f674f7995d..3b6868fe7c 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -693,7 +693,7 @@ struct i40e_fdir_action {
/* A structure used to define the flow director filter entry by filter_ctrl API
* It supports RTE_ETH_FILTER_FDIR data representation.
*/
-struct i40e_fdir_filter_conf {
+struct i40e_fdir_filter {
uint32_t soft_id;
/* ID, an unique value is required when deal with FDIR entry */
struct i40e_fdir_input input; /* Input set */
@@ -713,9 +713,26 @@ struct i40e_fdir_flex_mask {
#define I40E_FILTER_PCTYPE_MAX 64
#define I40E_MAX_FDIR_FILTER_NUM (1024 * 8)
-struct i40e_fdir_filter {
- TAILQ_ENTRY(i40e_fdir_filter) rules;
- struct i40e_fdir_filter_conf fdir;
+/*
+ * A filter added through the PMD packet template API. It owns the raw packet,
+ * and fdir.input.flow.raw_flow.packet points at that copy, so the filter stays
+ * self-contained and can be reprogrammed at any time.
+ */
+struct i40e_fdir_tmpl_filter {
+ TAILQ_ENTRY(i40e_fdir_tmpl_filter) rules;
+ struct i40e_fdir_filter fdir;
+ uint8_t *packet;
+};
+
+/*
+ * Packet templates are keyed on their contents, which are too big for a hash
+ * key, so the key borrows the packet and a custom comparison walks it. The
+ * hash signature is computed by the caller and passed to the _with_hash() API.
+ */
+struct i40e_fdir_tmpl_key {
+ uint16_t pctype;
+ uint32_t length;
+ const uint8_t *packet;
};
/* fdir memory pool entry */
@@ -735,7 +752,46 @@ struct i40e_fdir_flow_pool {
#define FLOW_TO_FLOW_BITMAP(f) \
container_of((f), struct i40e_fdir_entry, flow)
-TAILQ_HEAD(i40e_fdir_filter_list, i40e_fdir_filter);
+TAILQ_HEAD(i40e_fdir_tmpl_list, i40e_fdir_tmpl_filter);
+
+/* tracking for rte_flow-backed filters */
+struct i40e_fdir_pctype_state {
+ /* input set bits for this pctype */
+ uint64_t input_set;
+ uint32_t flow_count;
+ struct i40e_fdir_flex_mask flex_mask;
+ bool flex_mask_flag;
+};
+
+struct i40e_fdir_layer_state {
+ /*
+ * The rule for extracting a byte stream as flexible payload. Each layer
+ * can have up to three elements, and all filters sharing the same layer
+ * reuse the same programmed layout.
+ */
+ struct i40e_fdir_flex_pit flex_set[I40E_MAX_FLXPLD_FIED];
+ bool flex_pit_flag;
+ uint32_t flex_flow_count;
+};
+
+struct i40e_fdir_flow_store {
+ struct rte_hash *hash_table;
+ /* the pre-allocated pool of the rte_flow */
+ struct i40e_fdir_flow_pool flow_pool;
+
+ struct i40e_fdir_pctype_state pctype[I40E_FILTER_PCTYPE_MAX];
+ struct i40e_fdir_layer_state layer[I40E_MAX_FLXPLD_LAYER];
+};
+
+/* tracking for packet template filters; allocated on first use */
+struct i40e_fdir_tmpl_store {
+ struct rte_hash *hash_table;
+ /* filters indexed by their hash table slot */
+ struct i40e_fdir_tmpl_filter *filter_array;
+ /* these filters have no other owner, so they are enumerated here */
+ struct i40e_fdir_tmpl_list list;
+};
+
/*
* A structure used to define fields of a FDIR related info.
*/
@@ -752,20 +808,8 @@ struct i40e_fdir_info {
*/
int txq_available_buf_count;
- /* input set bits for each pctype */
- uint64_t input_set[I40E_FILTER_PCTYPE_MAX];
- /*
- * the rule how bytes stream is extracted as flexible payload
- * for each payload layer, the setting can up to three elements
- */
- struct i40e_fdir_flex_pit flex_set[I40E_MAX_FLXPLD_LAYER * I40E_MAX_FLXPLD_FIED];
- struct i40e_fdir_flex_mask flex_mask[I40E_FILTER_PCTYPE_MAX];
-
- struct i40e_fdir_filter_list fdir_list;
- struct i40e_fdir_filter **hash_map;
- struct rte_hash *hash_table;
- /* An array to store the inserted rules input */
- struct i40e_fdir_filter *fdir_filter_array;
+ struct i40e_fdir_flow_store flows;
+ struct i40e_fdir_tmpl_store tmpls;
/*
* Priority ordering at filter invalidation(destroying a flow) between
@@ -784,22 +828,10 @@ struct i40e_fdir_info {
* shared space
*/
uint32_t fdir_space_size;
- /* the actual number of the fdir rules in hardware, initialized as 0 */
+ /* number of filters in hardware, across both stores */
uint32_t fdir_actual_cnt;
- /* the free guaranteed space of the fdir */
- uint32_t fdir_guarantee_free_space;
/* the fdir total guaranteed space */
uint32_t fdir_guarantee_total_space;
- /* the pre-allocated pool of the rte_flow */
- struct i40e_fdir_flow_pool fdir_flow_pool;
-
- /* Mark if flex pit and mask is set */
- bool flex_pit_flag[I40E_MAX_FLXPLD_LAYER];
- bool flex_mask_flag[I40E_FILTER_PCTYPE_MAX];
-
- uint32_t flow_count[I40E_FILTER_PCTYPE_MAX];
-
- uint32_t flex_flow_count[I40E_MAX_FLXPLD_LAYER];
};
/* Ethertype filter number HW supports */
@@ -1303,7 +1335,7 @@ extern const struct rte_flow_ops i40e_flow_ops;
struct i40e_filter_ctx {
union {
- struct i40e_fdir_filter_conf fdir_filter;
+ struct i40e_fdir_filter fdir_filter;
struct i40e_tunnel_filter_conf consistent_tunnel_filter;
struct i40e_rte_flow_rss_conf rss_conf;
};
@@ -1353,7 +1385,7 @@ const struct rte_memzone *i40e_memzone_reserve(const char *name,
uint32_t len,
int socket_id);
int i40e_fdir_configure(struct rte_eth_dev *dev);
-void i40e_fdir_rx_proc_enable(struct rte_eth_dev *dev, bool on);
+void i40e_fdir_rx_proc_sync(struct rte_eth_dev *dev);
void i40e_fdir_teardown(struct i40e_pf *pf);
enum i40e_filter_pctype
i40e_flowtype_to_pctype(const struct i40e_adapter *adapter,
@@ -1384,8 +1416,6 @@ int i40e_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
-int i40e_sw_fdir_filter_del(struct i40e_pf *pf,
- struct i40e_fdir_input *input);
struct i40e_tunnel_filter *
i40e_sw_tunnel_filter_lookup(struct i40e_tunnel_rule *tunnel_rule,
const struct i40e_tunnel_filter_input *input);
@@ -1399,9 +1429,27 @@ struct rte_flow *
i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info);
void i40e_fdir_entry_pool_put(struct i40e_fdir_info *fdir_info,
struct rte_flow *flow);
-int i40e_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
- const struct i40e_fdir_filter_conf *filter,
- bool add);
+int i40e_fdir_tmpl_add_del(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ bool add);
+struct i40e_fdir_filter *
+i40e_fdir_filter_lookup(struct i40e_fdir_info *fdir_info,
+ const struct i40e_fdir_input *input);
+int i40e_fdir_filter_validate(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter);
+int i40e_fdir_filter_register(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ struct i40e_fdir_filter **node);
+int i40e_fdir_filter_unregister(struct rte_eth_dev *dev,
+ struct i40e_fdir_filter *node);
+int i40e_fdir_filter_program(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ bool add, bool wait_status);
+bool i40e_fdir_filter_needs_status_wait(const struct i40e_pf *pf,
+ uint32_t filter_count);
+void i40e_fdir_tmpl_store_free(struct i40e_fdir_info *fdir_info);
+int i40e_fdir_flow_store_init(struct rte_eth_dev *dev);
+void i40e_fdir_flow_store_free(struct i40e_fdir_info *fdir_info);
int i40e_dev_tunnel_filter_set(struct i40e_pf *pf,
struct rte_eth_tunnel_filter_conf *tunnel_filter,
uint8_t add);
diff --git a/drivers/net/intel/i40e/i40e_fdir.c b/drivers/net/intel/i40e/i40e_fdir.c
index 8a233f8a97..182894cdde 100644
--- a/drivers/net/intel/i40e/i40e_fdir.c
+++ b/drivers/net/intel/i40e/i40e_fdir.c
@@ -86,17 +86,10 @@
(1ULL << RTE_ETH_FLOW_NONFRAG_IPV6_OTHER) | \
(1ULL << RTE_ETH_FLOW_L2_PAYLOAD))
-static int i40e_fdir_filter_convert(const struct i40e_fdir_filter_conf *input,
- struct i40e_fdir_filter *filter);
-static struct i40e_fdir_filter *
-i40e_sw_fdir_filter_lookup(struct i40e_fdir_info *fdir_info,
- const struct i40e_fdir_input *input);
-static int i40e_sw_fdir_filter_insert(struct i40e_pf *pf,
- struct i40e_fdir_filter *filter);
static int
i40e_flow_fdir_filter_programming(struct i40e_pf *pf,
enum i40e_filter_pctype pctype,
- const struct i40e_fdir_filter_conf *filter,
+ const struct i40e_fdir_filter *filter,
bool add, bool wait_status);
static int
@@ -256,8 +249,6 @@ i40e_fdir_setup(struct i40e_pf *pf)
pf->fdir.match_counter_index = I40E_COUNTER_INDEX_FDIR(hw->pf_id);
pf->fdir.fdir_actual_cnt = 0;
- pf->fdir.fdir_guarantee_free_space =
- pf->fdir.fdir_guarantee_total_space;
PMD_DRV_LOG(INFO, "FDIR setup successfully, with programming queue %u.",
vsi->base_queue);
@@ -344,46 +335,52 @@ i40e_init_flx_pld(struct i40e_pf *pf)
* of payload as flexible payload.
*/
for (i = I40E_FLXPLD_L2_IDX; i < I40E_MAX_FLXPLD_LAYER; i++) {
+ struct i40e_fdir_layer_state *layer = &pf->fdir.flows.layer[i];
+ struct i40e_fdir_flex_pit *flex_set = &layer->flex_set[0];
index = i * I40E_MAX_FLXPLD_FIED;
- pf->fdir.flex_set[index].src_offset = 0;
- pf->fdir.flex_set[index].size = I40E_FDIR_MAX_FLEXWORD_NUM;
- pf->fdir.flex_set[index].dst_offset = 0;
+ flex_set->src_offset = 0;
+ flex_set->size = I40E_FDIR_MAX_FLEXWORD_NUM;
+ flex_set->dst_offset = 0;
I40E_WRITE_REG(hw, I40E_PRTQF_FLX_PIT(index), 0x0000C900);
I40E_WRITE_REG(hw,
I40E_PRTQF_FLX_PIT(index + 1), 0x0000FC29);/*non-used*/
I40E_WRITE_REG(hw,
I40E_PRTQF_FLX_PIT(index + 2), 0x0000FC2A);/*non-used*/
- pf->fdir.flex_pit_flag[i] = 0;
+ layer->flex_pit_flag = false;
}
/* initialize the masks */
for (pctype = I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
pctype <= I40E_FILTER_PCTYPE_L2_PAYLOAD; pctype++) {
+ struct i40e_fdir_pctype_state *state = &pf->fdir.flows.pctype[pctype];
flow_type = i40e_pctype_to_flowtype(pf->adapter, pctype);
if (flow_type == RTE_ETH_FLOW_UNKNOWN)
continue;
- pf->fdir.flex_mask[pctype].word_mask = 0;
+ state->flex_mask.word_mask = 0;
i40e_write_rx_ctl(hw, I40E_PRTQF_FD_FLXINSET(pctype), 0);
for (i = 0; i < I40E_FDIR_BITMASK_NUM_WORD; i++) {
- pf->fdir.flex_mask[pctype].bitmask[i].offset = 0;
- pf->fdir.flex_mask[pctype].bitmask[i].mask = 0;
+ state->flex_mask.bitmask[i].offset = 0;
+ state->flex_mask.bitmask[i].mask = 0;
i40e_write_rx_ctl(hw, I40E_PRTQF_FD_MSK(pctype, i), 0);
}
}
}
/*
- * Enable/disable flow director RX processing in vector routines.
+ * Match flow director RX processing to whether any filter is registered.
*/
void
-i40e_fdir_rx_proc_enable(struct rte_eth_dev *dev, bool on)
+i40e_fdir_rx_proc_sync(struct rte_eth_dev *dev)
{
- int32_t i;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ bool on = pf->fdir.fdir_actual_cnt > 0;
+ uint16_t i;
for (i = 0; i < dev->data->nb_rx_queues; i++) {
struct ci_rx_queue *rxq = dev->data->rx_queues[i];
- if (!rxq)
+
+ if (rxq == NULL)
continue;
rxq->fdir_enabled = on;
}
@@ -421,12 +418,37 @@ i40e_fdir_configure(struct rte_eth_dev *dev)
i40e_init_flx_pld(pf); /* set flex config to default value */
- /* Enable FDIR processing in RX routines */
- i40e_fdir_rx_proc_enable(dev, 1);
-
return ret;
}
+/*
+ * Bring up the flow director engine on first use.
+ */
+static int
+i40e_fdir_engine_init(struct rte_eth_dev *dev)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ int ret;
+
+ if (pf->fdir.fdir_vsi != NULL)
+ return 0;
+
+ ret = i40e_fdir_setup(pf);
+ if (ret != I40E_SUCCESS) {
+ PMD_DRV_LOG(ERR, "Failed to setup fdir.");
+ return -ENOTSUP;
+ }
+
+ ret = i40e_fdir_configure(dev);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure fdir.");
+ i40e_fdir_teardown(pf);
+ return ret;
+ }
+
+ return 0;
+}
+
static struct i40e_customized_pctype *
i40e_flow_fdir_find_customized_pctype(struct i40e_pf *pf, uint8_t pctype)
@@ -642,7 +664,7 @@ i40e_flow_fdir_construct_pkt(struct i40e_pf *pf,
struct rte_ipv6_hdr *esp_ipv6;
uint8_t size, dst = 0;
- uint8_t i, pit_idx, set_idx = I40E_FLXPLD_L4_IDX; /* use l4 by default*/
+ uint8_t i, set_idx = I40E_FLXPLD_L4_IDX; /* use l4 by default*/
int len;
uint8_t pctype = fdir_input->pctype;
struct i40e_customized_pctype *cus_pctype;
@@ -894,13 +916,15 @@ i40e_flow_fdir_construct_pkt(struct i40e_pf *pf,
/* fill the flexbytes to payload */
for (i = 0; i < I40E_MAX_FLXPLD_FIED; i++) {
- pit_idx = set_idx * I40E_MAX_FLXPLD_FIED + i;
- size = pf->fdir.flex_set[pit_idx].size;
+ const struct i40e_fdir_flex_pit *pit;
+ struct i40e_fdir_layer_state *layer = &pf->fdir.flows.layer[set_idx];
+
+ pit = &layer->flex_set[i];
+ size = pit->size;
if (size == 0)
continue;
- dst = pf->fdir.flex_set[pit_idx].dst_offset * sizeof(uint16_t);
- ptr = payload +
- pf->fdir.flex_set[pit_idx].src_offset * sizeof(uint16_t);
+ dst = pit->dst_offset * sizeof(uint16_t);
+ ptr = payload + pit->src_offset * sizeof(uint16_t);
(void)memcpy(ptr,
&fdir_input->flow_ext.flexbytes[dst],
size * sizeof(uint16_t));
@@ -997,56 +1021,23 @@ i40e_fdir_programming_status_cleanup(struct ci_rx_queue *rxq)
PMD_DRV_LOG(INFO, "error report captured.");
}
+/* Add a flow director filter into the SW hash table */
static int
-i40e_fdir_filter_convert(const struct i40e_fdir_filter_conf *input,
- struct i40e_fdir_filter *filter)
+i40e_fdir_filter_hash_add(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ struct i40e_fdir_filter **node)
{
- memcpy(&filter->fdir, input, sizeof(struct i40e_fdir_filter_conf));
- if (input->input.flow_ext.pkt_template) {
- filter->fdir.input.flow.raw_flow.packet = NULL;
- filter->fdir.input.flow.raw_flow.length =
- rte_hash_crc(input->input.flow.raw_flow.packet,
- input->input.flow.raw_flow.length,
- input->input.flow.raw_flow.pctype);
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ int ret;
+
+ ret = rte_hash_lookup(flows->hash_table, &filter->input);
+ if (ret >= 0) {
+ PMD_DRV_LOG(ERR, "Failed to add fdir filter to hash table %d!",
+ ret);
+ return -EEXIST;
}
- return 0;
-}
-/* Check if there exists the flow director filter */
-static struct i40e_fdir_filter *
-i40e_sw_fdir_filter_lookup(struct i40e_fdir_info *fdir_info,
- const struct i40e_fdir_input *input)
-{
- int ret;
-
- if (input->flow_ext.pkt_template)
- ret = rte_hash_lookup_with_hash(fdir_info->hash_table,
- (const void *)input,
- input->flow.raw_flow.length);
- else
- ret = rte_hash_lookup(fdir_info->hash_table,
- (const void *)input);
- if (ret < 0)
- return NULL;
-
- return fdir_info->hash_map[ret];
-}
-
-/* Add a flow director filter into the SW list */
-static int
-i40e_sw_fdir_filter_insert(struct i40e_pf *pf, struct i40e_fdir_filter *filter)
-{
- struct i40e_fdir_info *fdir_info = &pf->fdir;
- struct i40e_fdir_filter *hash_filter;
- int ret;
-
- if (filter->fdir.input.flow_ext.pkt_template)
- ret = rte_hash_add_key_with_hash(fdir_info->hash_table,
- &filter->fdir.input,
- filter->fdir.input.flow.raw_flow.length);
- else
- ret = rte_hash_add_key(fdir_info->hash_table,
- &filter->fdir.input);
+ ret = rte_hash_add_key(flows->hash_table, &filter->input);
if (ret < 0) {
PMD_DRV_LOG(ERR,
"Failed to insert fdir filter to hash table %d!",
@@ -1054,48 +1045,247 @@ i40e_sw_fdir_filter_insert(struct i40e_pf *pf, struct i40e_fdir_filter *filter)
return ret;
}
- if (fdir_info->hash_map[ret])
- return -1;
-
- hash_filter = &fdir_info->fdir_filter_array[ret];
- memcpy(hash_filter, filter, sizeof(*filter));
- fdir_info->hash_map[ret] = hash_filter;
- TAILQ_INSERT_TAIL(&fdir_info->fdir_list, hash_filter, rules);
+ **node = *filter;
return 0;
}
-/* Delete a flow director filter from the SW list */
-int
-i40e_sw_fdir_filter_del(struct i40e_pf *pf, struct i40e_fdir_input *input)
+/* Delete a flow director filter from the SW hash table */
+static int
+i40e_fdir_filter_hash_del(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *node)
{
- struct i40e_fdir_info *fdir_info = &pf->fdir;
- struct i40e_fdir_filter *filter;
int ret;
- if (input->flow_ext.pkt_template)
- ret = rte_hash_del_key_with_hash(fdir_info->hash_table,
- input,
- input->flow.raw_flow.length);
- else
- ret = rte_hash_del_key(fdir_info->hash_table, input);
+ ret = rte_hash_del_key(pf->fdir.flows.hash_table, &node->input);
if (ret < 0) {
PMD_DRV_LOG(ERR,
- "Failed to delete fdir filter to hash table %d!",
+ "Failed to delete fdir filter from hash table %d!",
ret);
return ret;
}
- filter = fdir_info->hash_map[ret];
- fdir_info->hash_map[ret] = NULL;
- TAILQ_REMOVE(&fdir_info->fdir_list, filter, rules);
+ return 0;
+}
+
+int
+i40e_fdir_flow_store_init(struct rte_eth_dev *dev)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
+ struct i40e_fdir_flow_store *flows = &fdir_info->flows;
+ char fdir_hash_name[RTE_HASH_NAMESIZE];
+ uint32_t alloc = hw->func_caps.fd_filters_guaranteed;
+ uint32_t best = hw->func_caps.fd_filters_best_effort;
+ struct rte_bitmap *bmp = NULL;
+ uint32_t bmp_size;
+ void *mem = NULL;
+ uint32_t i = 0;
+ int ret;
+
+ struct rte_hash_parameters fdir_hash_params = {
+ .name = fdir_hash_name,
+ .entries = I40E_MAX_FDIR_FILTER_NUM,
+ .key_len = sizeof(struct i40e_fdir_input),
+ .hash_func = rte_hash_crc,
+ .hash_func_init_val = 0,
+ .socket_id = rte_socket_id(),
+ };
+
+ snprintf(fdir_hash_name, RTE_HASH_NAMESIZE,
+ "fdir_%s", dev->device->name);
+ flows->hash_table = rte_hash_create(&fdir_hash_params);
+ if (!flows->hash_table) {
+ PMD_INIT_LOG(ERR, "Failed to create fdir hash table!");
+ return -EINVAL;
+ }
+
+ fdir_info->fdir_space_size = alloc + best;
+ fdir_info->fdir_actual_cnt = 0;
+ fdir_info->fdir_guarantee_total_space = alloc;
+
+ PMD_DRV_LOG(INFO, "FDIR guarantee space: %u, best_effort space %u.", alloc, best);
+
+ flows->flow_pool.pool =
+ rte_zmalloc("i40e_fdir_entry",
+ sizeof(struct i40e_fdir_entry) *
+ fdir_info->fdir_space_size,
+ 0);
+
+ if (!flows->flow_pool.pool) {
+ PMD_INIT_LOG(ERR,
+ "Failed to allocate memory for bitmap flow!");
+ ret = -ENOMEM;
+ goto err_fdir_bitmap_flow_alloc;
+ }
+
+ for (i = 0; i < fdir_info->fdir_space_size; i++)
+ flows->flow_pool.pool[i].idx = i;
+
+ bmp_size =
+ rte_bitmap_get_memory_footprint(fdir_info->fdir_space_size);
+ mem = rte_zmalloc("fdir_bmap", bmp_size, RTE_CACHE_LINE_SIZE);
+ if (mem == NULL) {
+ PMD_INIT_LOG(ERR,
+ "Failed to allocate memory for fdir bitmap!");
+ ret = -ENOMEM;
+ goto err_fdir_mem_alloc;
+ }
+ bmp = rte_bitmap_init(fdir_info->fdir_space_size, mem, bmp_size);
+ if (bmp == NULL) {
+ PMD_INIT_LOG(ERR,
+ "Failed to initialization fdir bitmap!");
+ ret = -ENOMEM;
+ goto err_fdir_bmp_alloc;
+ }
+ for (i = 0; i < fdir_info->fdir_space_size; i++)
+ rte_bitmap_set(bmp, i);
+
+ flows->flow_pool.bitmap = bmp;
return 0;
+
+err_fdir_bmp_alloc:
+ rte_free(mem);
+err_fdir_mem_alloc:
+ rte_free(flows->flow_pool.pool);
+err_fdir_bitmap_flow_alloc:
+ rte_hash_free(flows->hash_table);
+
+ return ret;
+}
+
+void
+i40e_fdir_flow_store_free(struct i40e_fdir_info *fdir_info)
+{
+ struct i40e_fdir_flow_store *flows = &fdir_info->flows;
+
+ rte_free(flows->flow_pool.bitmap);
+ rte_free(flows->flow_pool.pool);
+ rte_hash_free(flows->hash_table);
+}
+
+static uint32_t
+i40e_fdir_tmpl_sig(const struct i40e_raw_flow *raw)
+{
+ return rte_hash_crc(raw->packet, raw->length, raw->pctype);
+}
+
+static int
+i40e_fdir_tmpl_cmp(const void *key1, const void *key2,
+ size_t key_len __rte_unused)
+{
+ const struct i40e_fdir_tmpl_key *k1 = key1;
+ const struct i40e_fdir_tmpl_key *k2 = key2;
+
+ if (k1->pctype != k2->pctype || k1->length != k2->length)
+ return 1;
+
+ return memcmp(k1->packet, k2->packet, k1->length);
+}
+
+static void
+i40e_fdir_tmpl_key_fill(struct i40e_fdir_tmpl_key *key,
+ const struct i40e_raw_flow *raw,
+ const uint8_t *packet)
+{
+ memset(key, 0, sizeof(*key));
+ key->pctype = raw->pctype;
+ key->length = raw->length;
+ key->packet = packet;
+}
+
+/* The template store is only worth its memory once the API is actually used */
+static int
+i40e_fdir_tmpl_store_init(struct rte_eth_dev *dev)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct i40e_fdir_tmpl_store *tmpls = &pf->fdir.tmpls;
+ char name[RTE_HASH_NAMESIZE];
+ struct rte_hash_parameters params = {
+ .name = name,
+ .entries = I40E_MAX_FDIR_FILTER_NUM,
+ .key_len = sizeof(struct i40e_fdir_tmpl_key),
+ .hash_func = rte_hash_crc,
+ .hash_func_init_val = 0,
+ .socket_id = rte_socket_id(),
+ };
+
+ if (tmpls->hash_table != NULL)
+ return 0;
+
+ /* the packet template store is created on first use */
+ TAILQ_INIT(&tmpls->list);
+
+ snprintf(name, sizeof(name), "fdir_tmpl_%s", dev->device->name);
+ tmpls->hash_table = rte_hash_create(¶ms);
+ if (tmpls->hash_table == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to create fdir template hash table.");
+ return -ENOMEM;
+ }
+ /* the key only borrows the packet, so contents drive the comparison */
+ rte_hash_set_cmp_func(tmpls->hash_table, i40e_fdir_tmpl_cmp);
+
+ tmpls->filter_array = rte_zmalloc("fdir_tmpl_filter",
+ sizeof(*tmpls->filter_array) * I40E_MAX_FDIR_FILTER_NUM,
+ 0);
+ if (tmpls->filter_array == NULL) {
+ PMD_DRV_LOG(ERR,
+ "Failed to allocate fdir template filter array.");
+ rte_hash_free(tmpls->hash_table);
+ tmpls->hash_table = NULL;
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+i40e_fdir_tmpl_store_free(struct i40e_fdir_info *fdir_info)
+{
+ struct i40e_fdir_tmpl_store *tmpls = &fdir_info->tmpls;
+ struct i40e_fdir_tmpl_filter *tmpl, *tmp;
+ uint32_t i;
+
+ /* rules in this list point into filter array so no free() needed */
+ RTE_TAILQ_FOREACH_SAFE(tmpl, &tmpls->list, rules, tmp)
+ TAILQ_REMOVE(&tmpls->list, tmpl, rules);
+
+ if (tmpls->filter_array != NULL) {
+ for (i = 0; i < I40E_MAX_FDIR_FILTER_NUM; i++)
+ rte_free(tmpls->filter_array[i].packet);
+ rte_free(tmpls->filter_array);
+ tmpls->filter_array = NULL;
+ }
+ rte_hash_free(tmpls->hash_table);
+ tmpls->hash_table = NULL;
+}
+
+static struct i40e_fdir_tmpl_filter *
+i40e_fdir_tmpl_lookup(struct i40e_fdir_info *fdir_info,
+ const struct i40e_raw_flow *raw)
+{
+ struct i40e_fdir_tmpl_key key;
+ int ret;
+
+ if (fdir_info->tmpls.hash_table == NULL)
+ return NULL;
+
+ i40e_fdir_tmpl_key_fill(&key, raw, raw->packet);
+
+ ret = rte_hash_lookup_with_hash(fdir_info->tmpls.hash_table, &key,
+ i40e_fdir_tmpl_sig(raw));
+ if (ret < 0)
+ return NULL;
+
+ return &fdir_info->tmpls.filter_array[ret];
}
struct rte_flow *
i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info)
{
+ struct i40e_fdir_flow_pool *pool = &fdir_info->flows.flow_pool;
struct rte_flow *flow = NULL;
uint64_t slab = 0;
uint32_t pos = 0;
@@ -1108,8 +1298,7 @@ i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info)
return NULL;
}
- ret = rte_bitmap_scan(fdir_info->fdir_flow_pool.bitmap, &pos,
- &slab);
+ ret = rte_bitmap_scan(pool->bitmap, &pos, &slab);
/* normally this won't happen as the fdir_actual_cnt should be
* same with the number of the set bits in fdir_flow_pool,
@@ -1122,8 +1311,8 @@ i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info)
i = rte_bsf64(slab);
pos += i;
- rte_bitmap_clear(fdir_info->fdir_flow_pool.bitmap, pos);
- flow = &fdir_info->fdir_flow_pool.pool[pos].flow;
+ rte_bitmap_clear(pool->bitmap, pos);
+ flow = &pool->pool[pos].flow;
memset(flow, 0, sizeof(struct rte_flow));
@@ -1137,48 +1326,47 @@ i40e_fdir_entry_pool_put(struct i40e_fdir_info *fdir_info,
struct i40e_fdir_entry *f;
f = FLOW_TO_FLOW_BITMAP(flow);
- rte_bitmap_set(fdir_info->fdir_flow_pool.bitmap, f->idx);
+ rte_bitmap_set(fdir_info->flows.flow_pool.bitmap, f->idx);
}
static int
-i40e_flow_store_flex_pit(struct i40e_pf *pf,
- struct i40e_fdir_flex_pit *flex_pit,
+i40e_fdir_check_flex_pit(struct i40e_pf *pf,
+ const struct i40e_fdir_flex_pit *flex_pit,
enum i40e_flxpld_layer_idx layer_idx,
uint8_t raw_id)
{
- uint8_t field_idx;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_layer_state *layer = &flows->layer[layer_idx];
- field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + raw_id;
/* Check if the configuration is conflicted */
- if (pf->fdir.flex_pit_flag[layer_idx] &&
- (pf->fdir.flex_set[field_idx].src_offset != flex_pit->src_offset ||
- pf->fdir.flex_set[field_idx].size != flex_pit->size ||
- pf->fdir.flex_set[field_idx].dst_offset != flex_pit->dst_offset))
+ if (layer->flex_pit_flag &&
+ (layer->flex_set[raw_id].src_offset !=
+ flex_pit->src_offset ||
+ layer->flex_set[raw_id].size != flex_pit->size ||
+ layer->flex_set[raw_id].dst_offset !=
+ flex_pit->dst_offset))
return -1;
/* Check if the configuration exists. */
- if (pf->fdir.flex_pit_flag[layer_idx] &&
- (pf->fdir.flex_set[field_idx].src_offset == flex_pit->src_offset &&
- pf->fdir.flex_set[field_idx].size == flex_pit->size &&
- pf->fdir.flex_set[field_idx].dst_offset == flex_pit->dst_offset))
+ if (layer->flex_pit_flag &&
+ (layer->flex_set[raw_id].src_offset ==
+ flex_pit->src_offset &&
+ layer->flex_set[raw_id].size == flex_pit->size &&
+ layer->flex_set[raw_id].dst_offset ==
+ flex_pit->dst_offset))
return 1;
- pf->fdir.flex_set[field_idx].src_offset =
- flex_pit->src_offset;
- pf->fdir.flex_set[field_idx].size =
- flex_pit->size;
- pf->fdir.flex_set[field_idx].dst_offset =
- flex_pit->dst_offset;
-
return 0;
}
static void
-i40e_flow_set_fdir_flex_pit(struct i40e_pf *pf,
- enum i40e_flxpld_layer_idx layer_idx,
- uint8_t raw_id)
+i40e_fdir_flex_pit_program(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter)
{
+ enum i40e_flxpld_layer_idx layer_idx = filter->input.flow_ext.layer_idx;
+ uint8_t raw_id = filter->input.flow_ext.raw_id;
struct i40e_hw *hw = I40E_PF_TO_HW(pf);
+ const struct i40e_fdir_flex_pit *pit;
uint32_t flx_pit, flx_ort;
uint16_t min_next_off = 0;
uint8_t field_idx;
@@ -1194,13 +1382,12 @@ i40e_flow_set_fdir_flex_pit(struct i40e_pf *pf,
/* Set flex pit */
for (i = 0; i < raw_id; i++) {
field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + i;
- flx_pit = MK_FLX_PIT(pf->fdir.flex_set[field_idx].src_offset,
- pf->fdir.flex_set[field_idx].size,
- pf->fdir.flex_set[field_idx].dst_offset);
+ pit = &filter->input.flow_ext.flex_pit[field_idx];
+ flx_pit = MK_FLX_PIT(pit->src_offset, pit->size,
+ pit->dst_offset);
I40E_WRITE_REG(hw, I40E_PRTQF_FLX_PIT(field_idx), flx_pit);
- min_next_off = pf->fdir.flex_set[field_idx].src_offset +
- pf->fdir.flex_set[field_idx].size;
+ min_next_off = pit->src_offset + pit->size;
}
for (; i < I40E_MAX_FLXPLD_FIED; i++) {
@@ -1213,104 +1400,181 @@ i40e_flow_set_fdir_flex_pit(struct i40e_pf *pf,
}
}
+/* Translate the byte-granular mask supplied by the caller into register form */
static int
-i40e_flow_store_flex_mask(struct i40e_pf *pf,
- enum i40e_filter_pctype pctype,
- uint8_t *mask)
+i40e_fdir_flex_mask_convert(const uint8_t *mask,
+ struct i40e_fdir_flex_mask *flex_mask)
{
- struct i40e_fdir_flex_mask flex_mask;
uint8_t nb_bitmask = 0;
uint16_t mask_tmp;
uint8_t i;
- memset(&flex_mask, 0, sizeof(struct i40e_fdir_flex_mask));
+ memset(flex_mask, 0, sizeof(*flex_mask));
for (i = 0; i < I40E_FDIR_MAX_FLEX_LEN; i += sizeof(uint16_t)) {
mask_tmp = I40E_WORD(mask[i], mask[i + 1]);
if (mask_tmp) {
- flex_mask.word_mask |=
+ flex_mask->word_mask |=
I40E_FLEX_WORD_MASK(i / sizeof(uint16_t));
if (mask_tmp != UINT16_MAX) {
if (nb_bitmask >= I40E_FDIR_BITMASK_NUM_WORD)
return -1;
- flex_mask.bitmask[nb_bitmask].mask = ~mask_tmp;
- flex_mask.bitmask[nb_bitmask].offset =
+ flex_mask->bitmask[nb_bitmask].mask = ~mask_tmp;
+ flex_mask->bitmask[nb_bitmask].offset =
i / sizeof(uint16_t);
nb_bitmask++;
}
}
}
- flex_mask.nb_bitmask = nb_bitmask;
+ flex_mask->nb_bitmask = nb_bitmask;
- if (pf->fdir.flex_mask_flag[pctype] &&
- (memcmp(&flex_mask, &pf->fdir.flex_mask[pctype],
+ return 0;
+}
+
+static int
+i40e_fdir_check_flex_mask(struct i40e_pf *pf,
+ enum i40e_filter_pctype pctype,
+ const uint8_t *mask,
+ struct i40e_fdir_flex_mask *flex_mask)
+{
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+
+ if (i40e_fdir_flex_mask_convert(mask, flex_mask) < 0)
+ return -1;
+
+ if (state->flex_mask_flag &&
+ (memcmp(flex_mask, &state->flex_mask,
sizeof(struct i40e_fdir_flex_mask))))
return -2;
- else if (pf->fdir.flex_mask_flag[pctype] &&
- !(memcmp(&flex_mask, &pf->fdir.flex_mask[pctype],
- sizeof(struct i40e_fdir_flex_mask))))
- return 1;
- pf->fdir.flex_mask[pctype] = flex_mask;
return 0;
}
-static void
-i40e_flow_set_fdir_flex_msk(struct i40e_pf *pf,
- enum i40e_filter_pctype pctype)
+static int
+i40e_fdir_flex_msk_program(struct i40e_pf *pf,
+ enum i40e_filter_pctype pctype,
+ const struct i40e_fdir_filter *filter)
{
struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_fdir_flex_mask *flex_mask;
+ struct i40e_fdir_flex_mask flex_mask;
uint32_t flxinset, fd_mask;
uint8_t i;
- /* Set flex mask */
- flex_mask = &pf->fdir.flex_mask[pctype];
- flxinset = (flex_mask->word_mask <<
+ if (i40e_fdir_flex_mask_convert(filter->input.flow_ext.flex_mask,
+ &flex_mask) < 0)
+ return -EINVAL;
+
+ flxinset = (flex_mask.word_mask <<
I40E_PRTQF_FD_FLXINSET_INSET_SHIFT) &
I40E_PRTQF_FD_FLXINSET_INSET_MASK;
i40e_write_rx_ctl(hw, I40E_PRTQF_FD_FLXINSET(pctype), flxinset);
- for (i = 0; i < flex_mask->nb_bitmask; i++) {
- fd_mask = (flex_mask->bitmask[i].mask <<
+ for (i = 0; i < flex_mask.nb_bitmask; i++) {
+ fd_mask = (flex_mask.bitmask[i].mask <<
I40E_PRTQF_FD_MSK_MASK_SHIFT) &
I40E_PRTQF_FD_MSK_MASK_MASK;
- fd_mask |= ((flex_mask->bitmask[i].offset +
+ fd_mask |= ((flex_mask.bitmask[i].offset +
I40E_FLX_OFFSET_IN_FIELD_VECTOR) <<
I40E_PRTQF_FD_MSK_OFFSET_SHIFT) &
I40E_PRTQF_FD_MSK_OFFSET_MASK;
i40e_write_rx_ctl(hw, I40E_PRTQF_FD_MSK(pctype, i), fd_mask);
}
- pf->fdir.flex_mask_flag[pctype] = 1;
+ return 0;
}
static int
-i40e_flow_set_fdir_inset(struct i40e_pf *pf,
- enum i40e_filter_pctype pctype,
- uint64_t input_set)
+i40e_fdir_flex_check(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ enum i40e_filter_pctype pctype,
+ struct i40e_fdir_flex_mask *flex_mask)
{
- uint32_t mask_reg[I40E_INSET_MASK_NUM_REG] = {0};
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- uint64_t inset_reg = 0;
- int i, num;
+ enum i40e_flxpld_layer_idx layer_idx = filter->input.flow_ext.layer_idx;
+ int ret;
+ int i;
+
+ for (i = 0; i < filter->input.flow_ext.raw_id; i++) {
+ uint8_t field_idx;
+
+ field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + i;
+ ret = i40e_fdir_check_flex_pit(pf,
+ &filter->input.flow_ext.flex_pit[field_idx],
+ layer_idx, i);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR,
+ "Conflict with the first flexible rule.");
+ return -EINVAL;
+ }
+ }
+
+ ret = i40e_fdir_check_flex_mask(pf, pctype,
+ filter->input.flow_ext.flex_mask,
+ flex_mask);
+ if (ret == -1) {
+ PMD_DRV_LOG(ERR, "Exceed maximal number of bitmasks");
+ return -EINVAL;
+ } else if (ret == -2) {
+ PMD_DRV_LOG(ERR, "Conflict with the first flexible rule");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+i40e_fdir_flex_store(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ enum i40e_filter_pctype pctype,
+ const struct i40e_fdir_flex_mask *flex_mask)
+{
+ enum i40e_flxpld_layer_idx layer_idx = filter->input.flow_ext.layer_idx;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+ struct i40e_fdir_layer_state *layer = &flows->layer[layer_idx];
+ int i;
+
+ for (i = 0; i < filter->input.flow_ext.raw_id; i++) {
+ uint8_t field_idx;
+
+ field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + i;
+ layer->flex_set[i] = filter->input.flow_ext.flex_pit[field_idx];
+ }
+
+ state->flex_mask = *flex_mask;
+}
+
+/* Validate an input set against what earlier filters on this pctype established */
+static int
+i40e_fdir_inset_check(struct i40e_pf *pf,
+ enum i40e_filter_pctype pctype,
+ uint64_t input_set)
+{
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
- /* Check if the input set is valid */
if (i40e_validate_input_set(pctype, RTE_ETH_FILTER_FDIR,
input_set) != 0) {
PMD_DRV_LOG(ERR, "Invalid input set");
return -EINVAL;
}
- /* Check if the configuration is conflicted */
- if (pf->fdir.flow_count[pctype] &&
- memcmp(&pf->fdir.input_set[pctype], &input_set, sizeof(uint64_t))) {
+ if (state->flow_count && state->input_set != input_set) {
PMD_DRV_LOG(ERR, "Conflict with the first rule's input set.");
return -EINVAL;
}
- if (pf->fdir.flow_count[pctype] &&
- !memcmp(&pf->fdir.input_set[pctype], &input_set, sizeof(uint64_t)))
- return 0;
+ return 0;
+}
+
+static int
+i40e_fdir_inset_program(struct i40e_pf *pf,
+ enum i40e_filter_pctype pctype,
+ uint64_t input_set)
+{
+ uint32_t mask_reg[I40E_INSET_MASK_NUM_REG] = {0};
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
+ uint64_t inset_reg = 0;
+ int i, num;
num = i40e_generate_inset_mask_reg(hw, input_set, mask_reg,
I40E_INSET_MASK_NUM_REG);
@@ -1360,7 +1624,6 @@ i40e_flow_set_fdir_inset(struct i40e_pf *pf,
I40E_WRITE_FLUSH(hw);
- pf->fdir.input_set[pctype] = input_set;
return 0;
}
@@ -1403,166 +1666,298 @@ i40e_find_available_buffer(struct rte_eth_dev *dev)
return (unsigned char *)fdir_info->prg_pkt[txq->tx_tail >> 1];
}
+static enum i40e_filter_pctype
+i40e_fdir_filter_pctype(const struct i40e_fdir_filter *filter)
+{
+ if (filter->input.flow_ext.pkt_template)
+ return filter->input.flow.raw_flow.pctype;
+
+ return filter->input.pctype;
+}
+
+bool
+i40e_fdir_filter_needs_status_wait(const struct i40e_pf *pf,
+ uint32_t filter_count)
+{
+ if (pf->fdir.fdir_invalprio != 1)
+ return true;
+
+ return filter_count >= pf->fdir.fdir_guarantee_total_space;
+}
+
/**
- * i40e_flow_add_del_fdir_filter - add or remove a flow director filter.
- * @pf: board private structure
- * @filter: fdir filter entry
- * @add: 0 - delete, 1 - add
+ * i40e_fdir_filter_validate - check whether a filter can be accepted at all.
+ *
+ * Only inspects the filter itself; conflicts against already registered
+ * filters are detected by i40e_fdir_filter_register().
*/
int
-i40e_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
- const struct i40e_fdir_filter_conf *filter,
- bool add)
+i40e_fdir_filter_validate(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter)
{
- struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- enum i40e_flxpld_layer_idx layer_idx = I40E_FLXPLD_L2_IDX;
- struct i40e_fdir_info *fdir_info = &pf->fdir;
- uint8_t flex_mask[I40E_FDIR_MAX_FLEX_LEN];
- struct i40e_fdir_filter check_filter; /* Check if the filter exists */
- struct i40e_fdir_flex_pit flex_pit;
- enum i40e_filter_pctype pctype;
- struct i40e_fdir_filter *node;
- unsigned char *pkt = NULL;
- bool cfg_flex_pit = true;
- bool wait_status = true;
- uint8_t field_idx;
- int ret = 0;
- int i;
-
- if (pf->fdir.fdir_vsi == NULL) {
- PMD_DRV_LOG(ERR, "FDIR is not enabled");
- return -ENOTSUP;
- }
if (filter->action.rx_queue >= pf->dev_data->nb_rx_queues) {
PMD_DRV_LOG(ERR, "Invalid queue ID");
return -EINVAL;
}
+
if (filter->input.flow_ext.is_vf &&
filter->input.flow_ext.dst_id >= pf->vf_num) {
PMD_DRV_LOG(ERR, "Invalid VF ID");
return -EINVAL;
}
- if (filter->input.flow_ext.pkt_template) {
- if (filter->input.flow.raw_flow.length > I40E_FDIR_PKT_LEN ||
- !filter->input.flow.raw_flow.packet) {
- PMD_DRV_LOG(ERR, "Invalid raw packet template"
- " flow filter parameters!");
- return -EINVAL;
- }
- pctype = filter->input.flow.raw_flow.pctype;
- } else {
- pctype = filter->input.pctype;
- }
-
- /* Check if there is the filter in SW list */
- memset(&check_filter, 0, sizeof(check_filter));
- i40e_fdir_filter_convert(filter, &check_filter);
+
+ if (filter->input.flow_ext.pkt_template &&
+ (filter->input.flow.raw_flow.length > I40E_FDIR_PKT_LEN ||
+ filter->input.flow.raw_flow.packet == NULL)) {
+ PMD_DRV_LOG(ERR,
+ "Invalid raw packet template flow filter parameters!");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/**
+ * Records the filter and everything derived from it.
+ */
+int
+i40e_fdir_filter_register(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ struct i40e_fdir_filter **node)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
+ uint64_t input_set = filter->input.flow_ext.input_set;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+ struct i40e_fdir_layer_state *layer = NULL;
+ struct i40e_fdir_flex_mask flex_mask;
+ bool common_pctype;
+ int ret;
+
+ ret = i40e_fdir_engine_init(dev);
+ if (ret < 0)
+ return ret;
+
+ /* check input set if the packet type is common */
+ common_pctype = !filter->input.flow_ext.customized_pctype;
+
+ if (common_pctype) {
+ ret = i40e_fdir_inset_check(pf, pctype, input_set);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* check if flex flow configuration is valid */
+ if (filter->input.flow_ext.is_flex_flow) {
+ ret = i40e_fdir_flex_check(pf, filter, pctype, &flex_mask);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* register the flow with the hash table */
+ ret = i40e_fdir_filter_hash_add(pf, filter, node);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Conflict with existing flow director rules!");
+ return ret;
+ }
+
+ /* save additional configuration */
+ if (common_pctype)
+ state->input_set = input_set;
+
+ if (filter->input.flow_ext.is_flex_flow) {
+ i40e_fdir_flex_store(pf, filter, pctype, &flex_mask);
+ layer = &flows->layer[filter->input.flow_ext.layer_idx];
+ layer->flex_flow_count++;
+ layer->flex_pit_flag = true;
+ state->flex_mask_flag = true;
+ }
+
+ state->flow_count++;
+ pf->fdir.fdir_actual_cnt++;
+
+ i40e_fdir_rx_proc_sync(dev);
+
+ return 0;
+}
+
+/**
+ * i40e_fdir_filter_unregister - drop software ownership of a filter.
+ */
+int
+i40e_fdir_filter_unregister(struct rte_eth_dev *dev,
+ struct i40e_fdir_filter *node)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(node);
+ enum i40e_flxpld_layer_idx layer_idx = node->input.flow_ext.layer_idx;
+ bool is_flex_flow = node->input.flow_ext.is_flex_flow;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+ struct i40e_fdir_layer_state *layer = &flows->layer[layer_idx];
+ int ret;
+
+ ret = i40e_fdir_filter_hash_del(pf, node);
+ if (ret < 0)
+ return ret;
+
+ if (is_flex_flow && --layer->flex_flow_count == 0)
+ layer->flex_pit_flag = false;
+
+ if (--state->flow_count == 0)
+ state->flex_mask_flag = false;
+
+ pf->fdir.fdir_actual_cnt--;
+
+ i40e_fdir_rx_proc_sync(dev);
+
+ return 0;
+}
+
+/* Take software ownership of a packet template filter, owning its packet */
+static int
+i40e_fdir_tmpl_register(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ struct i40e_fdir_tmpl_filter **node)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ const struct i40e_raw_flow *raw = &filter->input.flow.raw_flow;
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
+ struct i40e_fdir_tmpl_filter *tmpl;
+ struct i40e_fdir_tmpl_key key;
+ uint8_t *packet;
+ int ret;
+
+ ret = i40e_fdir_engine_init(dev);
+ if (ret < 0)
+ return ret;
+
+ ret = i40e_fdir_tmpl_store_init(dev);
+ if (ret < 0)
+ return ret;
+
+ if (i40e_fdir_tmpl_lookup(fdir_info, raw) != NULL) {
+ PMD_DRV_LOG(ERR, "Conflict with existing flow director rules!");
+ return -EEXIST;
+ }
+
+ packet = rte_malloc("fdir_tmpl_packet", raw->length, 0);
+ if (packet == NULL)
+ return -ENOMEM;
+ memcpy(packet, raw->packet, raw->length);
+
+ /* the stored key borrows the packet, so it must point at our copy */
+ i40e_fdir_tmpl_key_fill(&key, raw, packet);
+
+ ret = rte_hash_add_key_with_hash(fdir_info->tmpls.hash_table, &key,
+ i40e_fdir_tmpl_sig(raw));
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to insert fdir template to hash table %d!",
+ ret);
+ rte_free(packet);
+ return ret;
+ }
+
+ tmpl = &fdir_info->tmpls.filter_array[ret];
+ tmpl->fdir = *filter;
+ tmpl->packet = packet;
+ tmpl->fdir.input.flow.raw_flow.packet = packet;
+
+ fdir_info->fdir_actual_cnt++;
+
+ i40e_fdir_rx_proc_sync(dev);
+
+ *node = tmpl;
+
+ return 0;
+}
+
+static int
+i40e_fdir_tmpl_unregister(struct rte_eth_dev *dev,
+ struct i40e_fdir_tmpl_filter *node)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ const struct i40e_raw_flow *raw = &node->fdir.input.flow.raw_flow;
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
+ struct i40e_fdir_tmpl_key key;
+ int ret;
+
+ i40e_fdir_tmpl_key_fill(&key, raw, raw->packet);
+
+ ret = rte_hash_del_key_with_hash(fdir_info->tmpls.hash_table, &key,
+ i40e_fdir_tmpl_sig(raw));
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to delete fdir template from hash table %d!",
+ ret);
+ return ret;
+ }
+
+ rte_free(node->packet);
+ node->packet = NULL;
+
+ fdir_info->fdir_actual_cnt--;
+
+ i40e_fdir_rx_proc_sync(dev);
+
+ return 0;
+}
+
+/**
+ * i40e_fdir_filter_program - apply a filter to the hardware.
+ *
+ * Derives everything it needs from the filter itself and touches no software
+ * bookkeeping, so it may be called repeatedly to reapply an already registered
+ * filter. The caller is responsible for having validated and registered it.
+ */
+int
+i40e_fdir_filter_program(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ bool add, bool wait_status)
+{
+ struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
+ unsigned char *pkt;
+ int ret;
+
+ if (pf->fdir.fdir_vsi == NULL) {
+ PMD_DRV_LOG(ERR, "FDIR is not enabled");
+ return -ENOTSUP;
+ }
if (add) {
- /* configure the input set for common PCTYPEs*/
if (!filter->input.flow_ext.customized_pctype &&
!filter->input.flow_ext.pkt_template) {
- ret = i40e_flow_set_fdir_inset(pf, pctype,
+ ret = i40e_fdir_inset_program(pf, pctype,
filter->input.flow_ext.input_set);
if (ret < 0)
return ret;
}
if (filter->input.flow_ext.is_flex_flow) {
- for (i = 0; i < filter->input.flow_ext.raw_id; i++) {
- layer_idx = filter->input.flow_ext.layer_idx;
- field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + i;
- flex_pit = filter->input.flow_ext.flex_pit[field_idx];
-
- /* Store flex pit to SW */
- ret = i40e_flow_store_flex_pit(pf, &flex_pit,
- layer_idx, i);
- if (ret < 0) {
- PMD_DRV_LOG(ERR, "Conflict with the"
- " first flexible rule.");
- return -EINVAL;
- } else if (ret > 0) {
- cfg_flex_pit = false;
- }
- }
-
- /* Store flex mask to SW */
- for (i = 0; i < I40E_FDIR_MAX_FLEX_LEN; i++)
- flex_mask[i] =
- filter->input.flow_ext.flex_mask[i];
-
- /* Validate the flex mask before writing any hardware
- * register. i40e_flow_set_fdir_flex_pit() below programs
- * the global GLQF_ORT register, which is shared by all
- * PFs on the NIC, so it must not be touched for a rule
- * that is going to be rejected.
- */
- ret = i40e_flow_store_flex_mask(pf, pctype, flex_mask);
- if (ret == -1) {
- PMD_DRV_LOG(ERR, "Exceed maximal"
- " number of bitmasks");
- return -EINVAL;
- } else if (ret == -2) {
- PMD_DRV_LOG(ERR, "Conflict with the"
- " first flexible rule");
- return -EINVAL;
- }
-
- if (cfg_flex_pit)
- i40e_flow_set_fdir_flex_pit(pf, layer_idx,
- filter->input.flow_ext.raw_id);
-
- if (ret == 0)
- i40e_flow_set_fdir_flex_msk(pf, pctype);
+ i40e_fdir_flex_pit_program(pf, filter);
+ ret = i40e_fdir_flex_msk_program(pf, pctype, filter);
+ if (ret < 0)
+ return ret;
}
-
- ret = i40e_sw_fdir_filter_insert(pf, &check_filter);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Conflict with existing flow director rules!");
- return -EINVAL;
- }
-
- if (fdir_info->fdir_invalprio == 1 &&
- fdir_info->fdir_guarantee_free_space > 0)
- wait_status = false;
- } else {
- if (filter->input.flow_ext.is_flex_flow)
- layer_idx = filter->input.flow_ext.layer_idx;
-
- node = i40e_sw_fdir_filter_lookup(fdir_info,
- &check_filter.fdir.input);
- if (!node) {
- PMD_DRV_LOG(ERR,
- "There's no corresponding flow director filter!");
- return -EINVAL;
- }
-
- ret = i40e_sw_fdir_filter_del(pf, &node->fdir.input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Error deleting fdir rule from hash table!");
- return -EINVAL;
- }
-
- pf->fdir.flex_mask_flag[pctype] = 0;
-
- if (fdir_info->fdir_invalprio == 1)
- wait_status = false;
}
- /* find a buffer to store the pkt */
pkt = i40e_find_available_buffer(dev);
- if (pkt == NULL)
- goto error_op;
+ if (pkt == NULL) {
+ PMD_DRV_LOG(ERR, "No buffer available to program fdir filter.");
+ return -ENOSPC;
+ }
memset(pkt, 0, I40E_FDIR_PKT_LEN);
ret = i40e_flow_fdir_construct_pkt(pf, &filter->input, pkt);
if (ret < 0) {
PMD_DRV_LOG(ERR, "construct packet for fdir fails.");
- goto error_op;
+ return ret;
}
if (hw->mac.type == I40E_MAC_X722) {
@@ -1576,45 +1971,72 @@ i40e_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
if (ret < 0) {
PMD_DRV_LOG(ERR, "fdir programming fails for PCTYPE(%u).",
pctype);
- goto error_op;
+ return ret;
}
- if (filter->input.flow_ext.is_flex_flow) {
- if (add) {
- fdir_info->flex_flow_count[layer_idx]++;
- pf->fdir.flex_pit_flag[layer_idx] = 1;
- } else {
- fdir_info->flex_flow_count[layer_idx]--;
- if (!fdir_info->flex_flow_count[layer_idx])
- pf->fdir.flex_pit_flag[layer_idx] = 0;
- }
- }
+ return 0;
+}
+
+/**
+ * i40e_fdir_tmpl_add_del - add or remove a packet template filter.
+ * @dev: ethernet device
+ * @filter: fdir filter entry
+ * @add: 0 - delete, 1 - add
+ *
+ * Entry point for the PMD packet template API, whose filters have no rte_flow
+ * handle and so are tracked entirely here.
+ */
+int
+i40e_fdir_tmpl_add_del(struct rte_eth_dev *dev,
+ const struct i40e_fdir_filter *filter,
+ bool add)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct i40e_fdir_tmpl_filter *node;
+ int ret;
+
+ ret = i40e_fdir_filter_validate(dev, filter);
+ if (ret < 0)
+ return ret;
if (add) {
- fdir_info->flow_count[pctype]++;
- fdir_info->fdir_actual_cnt++;
- if (fdir_info->fdir_invalprio == 1 &&
- fdir_info->fdir_guarantee_free_space > 0)
- fdir_info->fdir_guarantee_free_space--;
- } else {
- fdir_info->flow_count[pctype]--;
- fdir_info->fdir_actual_cnt--;
- if (fdir_info->fdir_invalprio == 1 &&
- fdir_info->fdir_guarantee_free_space <
- fdir_info->fdir_guarantee_total_space)
- fdir_info->fdir_guarantee_free_space++;
+ /* register updates count, so store it */
+ uint32_t cnt = pf->fdir.fdir_actual_cnt;
+
+ ret = i40e_fdir_tmpl_register(dev, filter, &node);
+ if (ret < 0)
+ return ret;
+
+ ret = i40e_fdir_filter_program(dev, &node->fdir, true,
+ i40e_fdir_filter_needs_status_wait(pf, cnt));
+ if (ret < 0) {
+ i40e_fdir_tmpl_unregister(dev, node);
+ return ret;
+ }
+
+ TAILQ_INSERT_TAIL(&pf->fdir.tmpls.list, node, rules);
+
+ return 0;
}
- return ret;
+ node = i40e_fdir_tmpl_lookup(&pf->fdir, &filter->input.flow.raw_flow);
+ if (node == NULL) {
+ PMD_DRV_LOG(ERR,
+ "There's no corresponding flow director filter!");
+ return -EINVAL;
+ }
+
+ ret = i40e_fdir_filter_program(dev, &node->fdir, false, false);
+ if (ret < 0)
+ return ret;
+
+ ret = i40e_fdir_tmpl_unregister(dev, node);
+ if (ret < 0)
+ return ret;
-error_op:
- /* roll back */
- if (add)
- i40e_sw_fdir_filter_del(pf, &check_filter.fdir.input);
- else
- i40e_sw_fdir_filter_insert(pf, &check_filter);
+ TAILQ_REMOVE(&pf->fdir.tmpls.list, node, rules);
- return ret;
+ return 0;
}
/*
@@ -1629,7 +2051,7 @@ i40e_flow_add_del_fdir_filter(struct rte_eth_dev *dev,
static int
i40e_flow_fdir_filter_programming(struct i40e_pf *pf,
enum i40e_filter_pctype pctype,
- const struct i40e_fdir_filter_conf *filter,
+ const struct i40e_fdir_filter *filter,
bool add, bool wait_status)
{
struct ci_tx_queue *txq = pf->fdir.txq;
@@ -1813,6 +2235,9 @@ i40e_fdir_info_get_flex_set(struct i40e_pf *pf,
for (layer_idx = I40E_FLXPLD_L2_IDX;
layer_idx <= I40E_FLXPLD_L4_IDX;
layer_idx++) {
+ struct i40e_fdir_layer_state *layer =
+ &pf->fdir.flows.layer[layer_idx];
+
if (layer_idx == I40E_FLXPLD_L2_IDX)
ptr->type = RTE_ETH_L2_PAYLOAD;
else if (layer_idx == I40E_FLXPLD_L3_IDX)
@@ -1821,8 +2246,7 @@ i40e_fdir_info_get_flex_set(struct i40e_pf *pf,
ptr->type = RTE_ETH_L4_PAYLOAD;
for (i = 0; i < I40E_MAX_FLXPLD_FIED; i++) {
- flex_pit = &pf->fdir.flex_set[layer_idx *
- I40E_MAX_FLXPLD_FIED + i];
+ flex_pit = &layer->flex_set[i];
if (flex_pit->size == 0)
continue;
src = flex_pit->src_offset * sizeof(uint16_t);
@@ -1850,7 +2274,9 @@ i40e_fdir_info_get_flex_mask(struct i40e_pf *pf,
for (i = I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
i <= I40E_FILTER_PCTYPE_L2_PAYLOAD;
i++) {
- mask = &pf->fdir.flex_mask[i];
+ struct i40e_fdir_pctype_state *state = &pf->fdir.flows.pctype[i];
+
+ mask = &state->flex_mask;
flow_type = i40e_pctype_to_flowtype(pf->adapter,
(enum i40e_filter_pctype)i);
if (flow_type == RTE_ETH_FLOW_UNKNOWN)
@@ -1941,29 +2367,35 @@ i40e_fdir_stats_get(struct rte_eth_dev *dev, struct rte_eth_fdir_stats *stat)
I40E_PFQF_FDSTAT_BEST_CNT_SHIFT);
}
-/* Restore flow director filter */
void
i40e_fdir_filter_restore(struct i40e_pf *pf)
{
struct rte_eth_dev *dev = I40E_VSI_TO_ETH_DEV(pf->main_vsi);
- struct i40e_fdir_filter_list *fdir_list = &pf->fdir.fdir_list;
- struct i40e_fdir_filter *f;
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- uint32_t fdstat;
- uint32_t guarant_cnt; /**< Number of filters in guaranteed spaces. */
- uint32_t best_cnt; /**< Number of filters in best effort spaces. */
+ struct i40e_fdir_tmpl_filter *tmpl;
+ struct rte_flow *flow;
+ int ret;
- TAILQ_FOREACH(f, fdir_list, rules)
- i40e_flow_add_del_fdir_filter(dev, &f->fdir, TRUE);
+ i40e_fdir_rx_proc_sync(dev);
- fdstat = I40E_READ_REG(hw, I40E_PFQF_FDSTAT);
- guarant_cnt =
- (uint32_t)((fdstat & I40E_PFQF_FDSTAT_GUARANT_CNT_MASK) >>
- I40E_PFQF_FDSTAT_GUARANT_CNT_SHIFT);
- best_cnt =
- (uint32_t)((fdstat & I40E_PFQF_FDSTAT_BEST_CNT_MASK) >>
- I40E_PFQF_FDSTAT_BEST_CNT_SHIFT);
+ if (pf->fdir.fdir_vsi == NULL)
+ return;
- PMD_DRV_LOG(INFO, "FDIR: Guarant count: %d, Best count: %d",
- guarant_cnt, best_cnt);
+ TAILQ_FOREACH(flow, &pf->flow_list, node) {
+ struct i40e_fdir_filter *node = flow->rule;
+
+ if (flow->filter_type != RTE_ETH_FILTER_FDIR)
+ continue;
+
+ ret = i40e_fdir_filter_program(dev, node, true, false);
+ if (ret < 0)
+ PMD_DRV_LOG(ERR,
+ "Failed to restore flow director filter: %d", ret);
+ }
+
+ TAILQ_FOREACH(tmpl, &pf->fdir.tmpls.list, rules) {
+ ret = i40e_fdir_filter_program(dev, &tmpl->fdir, true, false);
+ if (ret < 0)
+ PMD_DRV_LOG(ERR,
+ "Failed to restore flow director template: %d", ret);
+ }
}
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 8f09dfeb11..200de80863 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -67,11 +67,11 @@ static int i40e_flow_dev_dump(struct rte_eth_dev *dev,
static int i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
const struct rte_flow_item *pattern,
struct rte_flow_error *error,
- struct i40e_fdir_filter_conf *filter);
+ struct i40e_fdir_filter *filter);
static int i40e_flow_parse_fdir_action(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
struct rte_flow_error *error,
- struct i40e_fdir_filter_conf *filter);
+ struct i40e_fdir_filter *filter);
static int i40e_flow_parse_tunnel_action(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
struct rte_flow_error *error,
@@ -1375,7 +1375,7 @@ i40e_flow_check_raw_item(const struct rte_flow_item *item,
static uint8_t
i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
enum rte_flow_item_type item_type,
- struct i40e_fdir_filter_conf *filter)
+ struct i40e_fdir_filter *filter)
{
struct i40e_customized_pctype *cus_pctype = NULL;
@@ -1440,7 +1440,7 @@ i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
}
static void
-i40e_flow_set_filter_spi(struct i40e_fdir_filter_conf *filter,
+i40e_flow_set_filter_spi(struct i40e_fdir_filter *filter,
const struct rte_flow_item_esp *esp_spec)
{
if (filter->input.flow_ext.oip_type ==
@@ -1478,7 +1478,7 @@ static int
i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
const struct rte_flow_item *pattern,
struct rte_flow_error *error,
- struct i40e_fdir_filter_conf *filter)
+ struct i40e_fdir_filter *filter)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
const struct rte_flow_item *item = pattern;
@@ -2310,7 +2310,7 @@ static int
i40e_flow_parse_fdir_action(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
struct rte_flow_error *error,
- struct i40e_fdir_filter_conf *filter)
+ struct i40e_fdir_filter *filter)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct ci_flow_actions parsed_actions = {0};
@@ -2430,7 +2430,7 @@ i40e_flow_parse_fdir_filter(struct rte_eth_dev *dev,
struct rte_flow_error *error,
struct i40e_filter_ctx *filter)
{
- struct i40e_fdir_filter_conf *fdir_filter = &filter->fdir_filter;
+ struct i40e_fdir_filter *fdir_filter = &filter->fdir_filter;
int ret;
ret = i40e_flow_parse_fdir_pattern(dev, pattern, error, fdir_filter);
@@ -3756,28 +3756,6 @@ i40e_flow_create(struct rte_eth_dev *dev,
return NULL;
if (filter_ctx.type == RTE_ETH_FILTER_FDIR) {
- /* if this is the first time we're creating an fdir flow */
- if (pf->fdir.fdir_vsi == NULL) {
- ret = i40e_fdir_setup(pf);
- if (ret != I40E_SUCCESS) {
- rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Failed to setup fdir.");
- return NULL;
- }
- ret = i40e_fdir_configure(dev);
- if (ret < 0) {
- rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Failed to configure fdir.");
- i40e_fdir_teardown(pf);
- return NULL;
- }
- }
- /* If create the first fdir rule, enable fdir check for rx queues */
- if (TAILQ_EMPTY(&pf->fdir.fdir_list))
- i40e_fdir_rx_proc_enable(dev, 1);
-
flow = i40e_fdir_entry_pool_get(fdir_info);
if (flow == NULL) {
rte_flow_error_set(error, ENOBUFS,
@@ -3797,13 +3775,26 @@ i40e_flow_create(struct rte_eth_dev *dev,
}
switch (filter_ctx.type) {
- case RTE_ETH_FILTER_FDIR:
- ret = i40e_flow_add_del_fdir_filter(dev, &filter_ctx.fdir_filter, 1);
+ case RTE_ETH_FILTER_FDIR: {
+ struct i40e_fdir_filter *node;
+
+ ret = i40e_fdir_filter_validate(dev, &filter_ctx.fdir_filter);
if (ret)
goto free_flow;
- flow->rule = TAILQ_LAST(&pf->fdir.fdir_list,
- i40e_fdir_filter_list);
+ ret = i40e_fdir_filter_register(dev, &filter_ctx.fdir_filter,
+ &node);
+ if (ret)
+ goto free_flow;
+ ret = i40e_fdir_filter_program(dev, node, 1,
+ i40e_fdir_filter_needs_status_wait(pf,
+ fdir_info->fdir_actual_cnt - 1));
+ if (ret) {
+ i40e_fdir_filter_unregister(dev, node);
+ goto free_flow;
+ }
+ flow->rule = node;
break;
+ }
case RTE_ETH_FILTER_TUNNEL:
ret = i40e_dev_consistent_tunnel_filter_set(pf,
&filter_ctx.consistent_tunnel_filter, 1);
@@ -3860,16 +3851,15 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
ret = i40e_flow_destroy_tunnel_filter(pf,
(struct i40e_tunnel_filter *)flow->rule);
break;
- case RTE_ETH_FILTER_FDIR:
- ret = i40e_flow_add_del_fdir_filter(dev,
- &((struct i40e_fdir_filter *)flow->rule)->fdir,
- 0);
+ case RTE_ETH_FILTER_FDIR: {
+ struct i40e_fdir_filter *node = flow->rule;
- /* If the last flow is destroyed, disable fdir. */
- if (!ret && TAILQ_EMPTY(&pf->fdir.fdir_list)) {
- i40e_fdir_rx_proc_enable(dev, 0);
- }
+ ret = i40e_fdir_filter_program(dev, node, 0, false);
+ if (ret)
+ break;
+ ret = i40e_fdir_filter_unregister(dev, node);
break;
+ }
case RTE_ETH_FILTER_HASH:
ret = i40e_hash_filter_destroy(pf, flow->rule);
break;
@@ -3994,59 +3984,24 @@ i40e_flow_flush_fdir_filter(struct i40e_pf *pf)
{
struct rte_eth_dev *dev = &rte_eth_devices[pf->dev_data->port_id];
struct i40e_fdir_info *fdir_info = &pf->fdir;
- struct i40e_fdir_filter *fdir_filter;
- enum i40e_filter_pctype pctype;
struct rte_flow *flow;
void *temp;
int ret;
- uint32_t i = 0;
- ret = i40e_fdir_flush(dev);
- if (!ret) {
- /* Delete FDIR filters in FDIR list. */
- while ((fdir_filter = TAILQ_FIRST(&fdir_info->fdir_list))) {
- ret = i40e_sw_fdir_filter_del(pf,
- &fdir_filter->fdir.input);
- if (ret < 0)
- return ret;
- }
-
- /* Delete FDIR flows in flow list. */
- RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, temp) {
- if (flow->filter_type == RTE_ETH_FILTER_FDIR) {
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- }
- }
-
- /* reset bitmap */
- rte_bitmap_reset(fdir_info->fdir_flow_pool.bitmap);
- for (i = 0; i < fdir_info->fdir_space_size; i++) {
- fdir_info->fdir_flow_pool.pool[i].idx = i;
- rte_bitmap_set(fdir_info->fdir_flow_pool.bitmap, i);
- }
-
- fdir_info->fdir_actual_cnt = 0;
- fdir_info->fdir_guarantee_free_space =
- fdir_info->fdir_guarantee_total_space;
- memset(fdir_info->fdir_filter_array,
- 0,
- sizeof(struct i40e_fdir_filter) *
- I40E_MAX_FDIR_FILTER_NUM);
-
- for (pctype = I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
- pctype <= I40E_FILTER_PCTYPE_L2_PAYLOAD; pctype++) {
- pf->fdir.flow_count[pctype] = 0;
- pf->fdir.flex_mask_flag[pctype] = 0;
- }
-
- for (i = 0; i < I40E_MAX_FLXPLD_LAYER; i++)
- pf->fdir.flex_pit_flag[i] = 0;
-
- /* Disable FDIR processing as all FDIR rules are now flushed */
- i40e_fdir_rx_proc_enable(dev, 0);
+ RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, temp) {
+ if (flow->filter_type != RTE_ETH_FILTER_FDIR)
+ continue;
+ ret = i40e_fdir_filter_program(dev, flow->rule, false, false);
+ if (ret < 0)
+ return ret;
+ ret = i40e_fdir_filter_unregister(dev, flow->rule);
+ if (ret < 0)
+ return ret;
+ TAILQ_REMOVE(&pf->flow_list, flow, node);
+ i40e_fdir_entry_pool_put(fdir_info, flow);
}
- return ret;
+ return 0;
}
/* Flush all tunnel filters */
diff --git a/drivers/net/intel/i40e/rte_pmd_i40e.c b/drivers/net/intel/i40e/rte_pmd_i40e.c
index 2e7943ef8b..5b9234f8e5 100644
--- a/drivers/net/intel/i40e/rte_pmd_i40e.c
+++ b/drivers/net/intel/i40e/rte_pmd_i40e.c
@@ -3044,7 +3044,7 @@ int rte_pmd_i40e_flow_add_del_packet_template(
uint8_t add)
{
struct rte_eth_dev *dev = &rte_eth_devices[port];
- struct i40e_fdir_filter_conf filter_conf;
+ struct i40e_fdir_filter filter_conf;
RTE_ETH_VALID_PORTID_OR_ERR_RET(port, -ENODEV);
@@ -3068,7 +3068,7 @@ int rte_pmd_i40e_flow_add_del_packet_template(
(enum i40e_fdir_status)conf->action.report_status;
filter_conf.action.flex_off = conf->action.flex_off;
- return i40e_flow_add_del_fdir_filter(dev, &filter_conf, add);
+ return i40e_fdir_tmpl_add_del(dev, &filter_conf, add);
}
RTE_EXPORT_SYMBOL(rte_pmd_i40e_inset_get)
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 16/19] net/i40e: reimplement FDIR parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (14 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 15/19] net/i40e: refactor FDIR engine infrastructure Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:20 ` [PATCH v2 17/19] net/i40e: reimplement tunnel parsers Anatoly Burakov
` (3 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Use the new flow graph API and the common parsing framework to implement
flow parser for flow director.
As a result of transitioning to more formalized validation, some checks
have become more stringent. In particular, some protocols (such as SCTP)
were previously accepting non-zero or invalid masks and either ignoring
them or misinterpreting them to mean "fully masked". This has now been
corrected.
The FDIR engine in i40e has also relied on a custom memory allocation
scheme for fdir flows - also migrated to the new infrastructure.
Finally, some of the FDIR infrastructure has been migrated into the new
flow engine, so that the engine itself is managing its own allocator and
its own flow deduplication scheme.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 11 +-
drivers/net/intel/i40e/i40e_ethdev.h | 64 +-
drivers/net/intel/i40e/i40e_fdir.c | 351 +---
drivers/net/intel/i40e/i40e_flow.c | 1916 +--------------------
drivers/net/intel/i40e/i40e_flow.h | 5 +
drivers/net/intel/i40e/i40e_flow_fdir.c | 2056 +++++++++++++++++++++++
drivers/net/intel/i40e/meson.build | 1 +
7 files changed, 2126 insertions(+), 2278 deletions(-)
create mode 100644 drivers/net/intel/i40e/i40e_flow_fdir.c
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 25062fe695..572dfff13a 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -1659,9 +1659,7 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
ret = i40e_init_tunnel_filter_list(dev);
if (ret < 0)
goto err_init_tunnel_filter_list;
- ret = i40e_fdir_flow_store_init(dev);
- if (ret < 0)
- goto err_init_fdir_flow_store;
+ i40e_fdir_flow_store_init(dev);
/* initialize flow engine configuration */
ret = ci_flow_engine_conf_init(&pf->flow_engine_conf,
@@ -1678,8 +1676,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return 0;
err_flow_engine_conf_init:
- i40e_fdir_flow_store_free(&pf->fdir);
-err_init_fdir_flow_store:
rte_hash_free(pf->tunnel.hash_table);
rte_free(pf->tunnel.hash_map);
err_init_tunnel_filter_list:
@@ -1727,7 +1723,6 @@ i40e_fdir_memory_cleanup(struct i40e_pf *pf)
{
struct i40e_fdir_info *fdir_info = &pf->fdir;
- i40e_fdir_flow_store_free(fdir_info);
i40e_fdir_tmpl_store_free(fdir_info);
}
@@ -2588,9 +2583,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
/* Remove all flows */
while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
TAILQ_REMOVE(&pf->flow_list, p_flow, node);
- /* Do not free FDIR flows since they are static allocated */
- if (p_flow->filter_type != RTE_ETH_FILTER_FDIR)
- rte_free(p_flow);
+ rte_free(p_flow);
}
/* release the fdir static allocated memory */
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index 3b6868fe7c..7a326fa75b 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -522,6 +522,9 @@ struct i40e_vmdq_info {
#define I40E_WORD(hi, lo) (uint16_t)((((hi) << 8) & 0xFF00) | ((lo) & 0xFF))
#define I40E_FLEX_WORD_MASK(off) (0x80 >> (off))
#define I40E_FDIR_IPv6_TC_OFFSET 20
+#define I40E_IPV6_TC_MASK (0xFF << I40E_FDIR_IPv6_TC_OFFSET)
+#define I40E_IPV6_FRAG_HEADER 44
+#define I40E_VLAN_TCI_MASK (RTE_VLAN_PRI_MASK | RTE_VLAN_DEI_MASK | RTE_VLAN_ID_MASK)
/* A structure used to define the input for GTP flow */
struct i40e_gtp_flow {
@@ -735,23 +738,6 @@ struct i40e_fdir_tmpl_key {
const uint8_t *packet;
};
-/* fdir memory pool entry */
-struct i40e_fdir_entry {
- struct rte_flow flow;
- uint32_t idx;
-};
-
-/* pre-allocated fdir memory pool */
-struct i40e_fdir_flow_pool {
- /* a bitmap to manage the fdir pool */
- struct rte_bitmap *bitmap;
- /* the size the pool is pf->fdir->fdir_space_size */
- struct i40e_fdir_entry *pool;
-};
-
-#define FLOW_TO_FLOW_BITMAP(f) \
- container_of((f), struct i40e_fdir_entry, flow)
-
TAILQ_HEAD(i40e_fdir_tmpl_list, i40e_fdir_tmpl_filter);
/* tracking for rte_flow-backed filters */
@@ -774,11 +760,8 @@ struct i40e_fdir_layer_state {
uint32_t flex_flow_count;
};
+/* global flex PIT configuration tracking */
struct i40e_fdir_flow_store {
- struct rte_hash *hash_table;
- /* the pre-allocated pool of the rte_flow */
- struct i40e_fdir_flow_pool flow_pool;
-
struct i40e_fdir_pctype_state pctype[I40E_FILTER_PCTYPE_MAX];
struct i40e_fdir_layer_state layer[I40E_MAX_FLXPLD_LAYER];
};
@@ -1335,7 +1318,6 @@ extern const struct rte_flow_ops i40e_flow_ops;
struct i40e_filter_ctx {
union {
- struct i40e_fdir_filter fdir_filter;
struct i40e_tunnel_filter_conf consistent_tunnel_filter;
struct i40e_rte_flow_rss_conf rss_conf;
};
@@ -1384,8 +1366,9 @@ void i40e_vsi_enable_queues_intr(struct i40e_vsi *vsi);
const struct rte_memzone *i40e_memzone_reserve(const char *name,
uint32_t len,
int socket_id);
-int i40e_fdir_configure(struct rte_eth_dev *dev);
-void i40e_fdir_rx_proc_sync(struct rte_eth_dev *dev);
+int i40e_fdir_configure(struct i40e_pf *pf);
+void i40e_fdir_rx_proc_sync(struct rte_eth_dev_data *dev_data);
+int i40e_fdir_engine_init(struct i40e_pf *pf);
void i40e_fdir_teardown(struct i40e_pf *pf);
enum i40e_filter_pctype
i40e_flowtype_to_pctype(const struct i40e_adapter *adapter,
@@ -1425,38 +1408,35 @@ uint64_t i40e_get_default_input_set(uint16_t pctype);
int i40e_ethertype_filter_program(struct i40e_pf *pf,
struct rte_eth_ethertype_filter *filter,
bool add);
-struct rte_flow *
-i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info);
-void i40e_fdir_entry_pool_put(struct i40e_fdir_info *fdir_info,
- struct rte_flow *flow);
int i40e_fdir_tmpl_add_del(struct rte_eth_dev *dev,
const struct i40e_fdir_filter *filter,
bool add);
-struct i40e_fdir_filter *
-i40e_fdir_filter_lookup(struct i40e_fdir_info *fdir_info,
- const struct i40e_fdir_input *input);
-int i40e_fdir_filter_validate(struct rte_eth_dev *dev,
- const struct i40e_fdir_filter *filter);
-int i40e_fdir_filter_register(struct rte_eth_dev *dev,
- const struct i40e_fdir_filter *filter,
- struct i40e_fdir_filter **node);
-int i40e_fdir_filter_unregister(struct rte_eth_dev *dev,
- struct i40e_fdir_filter *node);
-int i40e_fdir_filter_program(struct rte_eth_dev *dev,
+int i40e_fdir_filter_program(struct i40e_pf *pf,
const struct i40e_fdir_filter *filter,
bool add, bool wait_status);
bool i40e_fdir_filter_needs_status_wait(const struct i40e_pf *pf,
uint32_t filter_count);
+enum i40e_filter_pctype i40e_fdir_filter_pctype(const struct i40e_fdir_filter *filter);
void i40e_fdir_tmpl_store_free(struct i40e_fdir_info *fdir_info);
-int i40e_fdir_flow_store_init(struct rte_eth_dev *dev);
-void i40e_fdir_flow_store_free(struct i40e_fdir_info *fdir_info);
+void i40e_fdir_flow_store_init(struct rte_eth_dev *dev);
+int i40e_fdir_inset_check(struct i40e_pf *pf,
+ enum i40e_filter_pctype pctype,
+ uint64_t input_set);
+int i40e_fdir_flex_check(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ enum i40e_filter_pctype pctype,
+ struct i40e_fdir_flex_mask *flex_mask);
+void i40e_fdir_flex_store(struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ enum i40e_filter_pctype pctype,
+ const struct i40e_fdir_flex_mask *flex_mask);
int i40e_dev_tunnel_filter_set(struct i40e_pf *pf,
struct rte_eth_tunnel_filter_conf *tunnel_filter,
uint8_t add);
int i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
struct i40e_tunnel_filter_conf *tunnel_filter,
uint8_t add);
-int i40e_fdir_flush(struct rte_eth_dev *dev);
+int i40e_fdir_flush(struct i40e_pf *pf);
int i40e_find_all_vlan_for_mac(struct i40e_vsi *vsi,
struct i40e_macvlan_filter *mv_f,
int num, struct rte_ether_addr *addr);
diff --git a/drivers/net/intel/i40e/i40e_fdir.c b/drivers/net/intel/i40e/i40e_fdir.c
index 182894cdde..750613af89 100644
--- a/drivers/net/intel/i40e/i40e_fdir.c
+++ b/drivers/net/intel/i40e/i40e_fdir.c
@@ -371,14 +371,14 @@ i40e_init_flx_pld(struct i40e_pf *pf)
* Match flow director RX processing to whether any filter is registered.
*/
void
-i40e_fdir_rx_proc_sync(struct rte_eth_dev *dev)
+i40e_fdir_rx_proc_sync(struct rte_eth_dev_data *data)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(data->dev_private);
bool on = pf->fdir.fdir_actual_cnt > 0;
uint16_t i;
- for (i = 0; i < dev->data->nb_rx_queues; i++) {
- struct ci_rx_queue *rxq = dev->data->rx_queues[i];
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ struct ci_rx_queue *rxq = data->rx_queues[i];
if (rxq == NULL)
continue;
@@ -391,10 +391,9 @@ i40e_fdir_rx_proc_sync(struct rte_eth_dev *dev)
* Configure flow director related setting
*/
int
-i40e_fdir_configure(struct rte_eth_dev *dev)
+i40e_fdir_configure(struct i40e_pf *pf)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
uint32_t val;
int ret = 0;
@@ -404,7 +403,7 @@ i40e_fdir_configure(struct rte_eth_dev *dev)
* If filters exist, flush them.
*/
if (i40e_fdir_empty(hw) < 0) {
- ret = i40e_fdir_flush(dev);
+ ret = i40e_fdir_flush(pf);
if (ret) {
PMD_DRV_LOG(ERR, "failed to flush fdir table.");
return ret;
@@ -424,10 +423,9 @@ i40e_fdir_configure(struct rte_eth_dev *dev)
/*
* Bring up the flow director engine on first use.
*/
-static int
-i40e_fdir_engine_init(struct rte_eth_dev *dev)
+int
+i40e_fdir_engine_init(struct i40e_pf *pf)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
int ret;
if (pf->fdir.fdir_vsi != NULL)
@@ -439,7 +437,7 @@ i40e_fdir_engine_init(struct rte_eth_dev *dev)
return -ENOTSUP;
}
- ret = i40e_fdir_configure(dev);
+ ret = i40e_fdir_configure(pf);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to configure fdir.");
i40e_fdir_teardown(pf);
@@ -1021,149 +1019,20 @@ i40e_fdir_programming_status_cleanup(struct ci_rx_queue *rxq)
PMD_DRV_LOG(INFO, "error report captured.");
}
-/* Add a flow director filter into the SW hash table */
-static int
-i40e_fdir_filter_hash_add(struct i40e_pf *pf,
- const struct i40e_fdir_filter *filter,
- struct i40e_fdir_filter **node)
-{
- struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
- int ret;
-
- ret = rte_hash_lookup(flows->hash_table, &filter->input);
- if (ret >= 0) {
- PMD_DRV_LOG(ERR, "Failed to add fdir filter to hash table %d!",
- ret);
- return -EEXIST;
- }
-
- ret = rte_hash_add_key(flows->hash_table, &filter->input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to insert fdir filter to hash table %d!",
- ret);
- return ret;
- }
-
- **node = *filter;
-
- return 0;
-}
-
-/* Delete a flow director filter from the SW hash table */
-static int
-i40e_fdir_filter_hash_del(struct i40e_pf *pf,
- const struct i40e_fdir_filter *node)
-{
- int ret;
-
- ret = rte_hash_del_key(pf->fdir.flows.hash_table, &node->input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to delete fdir filter from hash table %d!",
- ret);
- return ret;
- }
-
- return 0;
-}
-
-int
+void
i40e_fdir_flow_store_init(struct rte_eth_dev *dev)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct i40e_hw *hw = I40E_PF_TO_HW(pf);
struct i40e_fdir_info *fdir_info = &pf->fdir;
- struct i40e_fdir_flow_store *flows = &fdir_info->flows;
- char fdir_hash_name[RTE_HASH_NAMESIZE];
uint32_t alloc = hw->func_caps.fd_filters_guaranteed;
uint32_t best = hw->func_caps.fd_filters_best_effort;
- struct rte_bitmap *bmp = NULL;
- uint32_t bmp_size;
- void *mem = NULL;
- uint32_t i = 0;
- int ret;
-
- struct rte_hash_parameters fdir_hash_params = {
- .name = fdir_hash_name,
- .entries = I40E_MAX_FDIR_FILTER_NUM,
- .key_len = sizeof(struct i40e_fdir_input),
- .hash_func = rte_hash_crc,
- .hash_func_init_val = 0,
- .socket_id = rte_socket_id(),
- };
-
- snprintf(fdir_hash_name, RTE_HASH_NAMESIZE,
- "fdir_%s", dev->device->name);
- flows->hash_table = rte_hash_create(&fdir_hash_params);
- if (!flows->hash_table) {
- PMD_INIT_LOG(ERR, "Failed to create fdir hash table!");
- return -EINVAL;
- }
fdir_info->fdir_space_size = alloc + best;
fdir_info->fdir_actual_cnt = 0;
fdir_info->fdir_guarantee_total_space = alloc;
PMD_DRV_LOG(INFO, "FDIR guarantee space: %u, best_effort space %u.", alloc, best);
-
- flows->flow_pool.pool =
- rte_zmalloc("i40e_fdir_entry",
- sizeof(struct i40e_fdir_entry) *
- fdir_info->fdir_space_size,
- 0);
-
- if (!flows->flow_pool.pool) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for bitmap flow!");
- ret = -ENOMEM;
- goto err_fdir_bitmap_flow_alloc;
- }
-
- for (i = 0; i < fdir_info->fdir_space_size; i++)
- flows->flow_pool.pool[i].idx = i;
-
- bmp_size =
- rte_bitmap_get_memory_footprint(fdir_info->fdir_space_size);
- mem = rte_zmalloc("fdir_bmap", bmp_size, RTE_CACHE_LINE_SIZE);
- if (mem == NULL) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for fdir bitmap!");
- ret = -ENOMEM;
- goto err_fdir_mem_alloc;
- }
- bmp = rte_bitmap_init(fdir_info->fdir_space_size, mem, bmp_size);
- if (bmp == NULL) {
- PMD_INIT_LOG(ERR,
- "Failed to initialization fdir bitmap!");
- ret = -ENOMEM;
- goto err_fdir_bmp_alloc;
- }
- for (i = 0; i < fdir_info->fdir_space_size; i++)
- rte_bitmap_set(bmp, i);
-
- flows->flow_pool.bitmap = bmp;
-
- return 0;
-
-err_fdir_bmp_alloc:
- rte_free(mem);
-err_fdir_mem_alloc:
- rte_free(flows->flow_pool.pool);
-err_fdir_bitmap_flow_alloc:
- rte_hash_free(flows->hash_table);
-
- return ret;
-}
-
-void
-i40e_fdir_flow_store_free(struct i40e_fdir_info *fdir_info)
-{
- struct i40e_fdir_flow_store *flows = &fdir_info->flows;
-
- rte_free(flows->flow_pool.bitmap);
- rte_free(flows->flow_pool.pool);
- rte_hash_free(flows->hash_table);
}
static uint32_t
@@ -1282,53 +1151,6 @@ i40e_fdir_tmpl_lookup(struct i40e_fdir_info *fdir_info,
return &fdir_info->tmpls.filter_array[ret];
}
-struct rte_flow *
-i40e_fdir_entry_pool_get(struct i40e_fdir_info *fdir_info)
-{
- struct i40e_fdir_flow_pool *pool = &fdir_info->flows.flow_pool;
- struct rte_flow *flow = NULL;
- uint64_t slab = 0;
- uint32_t pos = 0;
- uint32_t i = 0;
- int ret;
-
- if (fdir_info->fdir_actual_cnt >=
- fdir_info->fdir_space_size) {
- PMD_DRV_LOG(ERR, "Fdir space full");
- return NULL;
- }
-
- ret = rte_bitmap_scan(pool->bitmap, &pos, &slab);
-
- /* normally this won't happen as the fdir_actual_cnt should be
- * same with the number of the set bits in fdir_flow_pool,
- * but anyway handle this error condition here for safe
- */
- if (ret == 0) {
- PMD_DRV_LOG(ERR, "fdir_actual_cnt out of sync");
- return NULL;
- }
-
- i = rte_bsf64(slab);
- pos += i;
- rte_bitmap_clear(pool->bitmap, pos);
- flow = &pool->pool[pos].flow;
-
- memset(flow, 0, sizeof(struct rte_flow));
-
- return flow;
-}
-
-void
-i40e_fdir_entry_pool_put(struct i40e_fdir_info *fdir_info,
- struct rte_flow *flow)
-{
- struct i40e_fdir_entry *f;
-
- f = FLOW_TO_FLOW_BITMAP(flow);
- rte_bitmap_set(fdir_info->flows.flow_pool.bitmap, f->idx);
-}
-
static int
i40e_fdir_check_flex_pit(struct i40e_pf *pf,
const struct i40e_fdir_flex_pit *flex_pit,
@@ -1483,7 +1305,7 @@ i40e_fdir_flex_msk_program(struct i40e_pf *pf,
return 0;
}
-static int
+int
i40e_fdir_flex_check(struct i40e_pf *pf,
const struct i40e_fdir_filter *filter,
enum i40e_filter_pctype pctype,
@@ -1521,7 +1343,7 @@ i40e_fdir_flex_check(struct i40e_pf *pf,
return 0;
}
-static void
+void
i40e_fdir_flex_store(struct i40e_pf *pf,
const struct i40e_fdir_filter *filter,
enum i40e_filter_pctype pctype,
@@ -1544,7 +1366,7 @@ i40e_fdir_flex_store(struct i40e_pf *pf,
}
/* Validate an input set against what earlier filters on this pctype established */
-static int
+int
i40e_fdir_inset_check(struct i40e_pf *pf,
enum i40e_filter_pctype pctype,
uint64_t input_set)
@@ -1628,9 +1450,8 @@ i40e_fdir_inset_program(struct i40e_pf *pf,
}
static inline unsigned char *
-i40e_find_available_buffer(struct rte_eth_dev *dev)
+i40e_find_available_buffer(struct i40e_pf *pf)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct i40e_fdir_info *fdir_info = &pf->fdir;
struct ci_tx_queue *txq = pf->fdir.txq;
@@ -1666,7 +1487,7 @@ i40e_find_available_buffer(struct rte_eth_dev *dev)
return (unsigned char *)fdir_info->prg_pkt[txq->tx_tail >> 1];
}
-static enum i40e_filter_pctype
+enum i40e_filter_pctype
i40e_fdir_filter_pctype(const struct i40e_fdir_filter *filter)
{
if (filter->input.flow_ext.pkt_template)
@@ -1691,7 +1512,7 @@ i40e_fdir_filter_needs_status_wait(const struct i40e_pf *pf,
* Only inspects the filter itself; conflicts against already registered
* filters are detected by i40e_fdir_filter_register().
*/
-int
+static int
i40e_fdir_filter_validate(struct rte_eth_dev *dev,
const struct i40e_fdir_filter *filter)
{
@@ -1719,111 +1540,14 @@ i40e_fdir_filter_validate(struct rte_eth_dev *dev,
return 0;
}
-/**
- * Records the filter and everything derived from it.
- */
-int
-i40e_fdir_filter_register(struct rte_eth_dev *dev,
- const struct i40e_fdir_filter *filter,
- struct i40e_fdir_filter **node)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
- uint64_t input_set = filter->input.flow_ext.input_set;
- struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
- struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
- struct i40e_fdir_layer_state *layer = NULL;
- struct i40e_fdir_flex_mask flex_mask;
- bool common_pctype;
- int ret;
-
- ret = i40e_fdir_engine_init(dev);
- if (ret < 0)
- return ret;
-
- /* check input set if the packet type is common */
- common_pctype = !filter->input.flow_ext.customized_pctype;
-
- if (common_pctype) {
- ret = i40e_fdir_inset_check(pf, pctype, input_set);
- if (ret < 0)
- return ret;
- }
-
- /* check if flex flow configuration is valid */
- if (filter->input.flow_ext.is_flex_flow) {
- ret = i40e_fdir_flex_check(pf, filter, pctype, &flex_mask);
- if (ret < 0)
- return ret;
- }
-
- /* register the flow with the hash table */
- ret = i40e_fdir_filter_hash_add(pf, filter, node);
- if (ret < 0) {
- PMD_DRV_LOG(ERR, "Conflict with existing flow director rules!");
- return ret;
- }
-
- /* save additional configuration */
- if (common_pctype)
- state->input_set = input_set;
-
- if (filter->input.flow_ext.is_flex_flow) {
- i40e_fdir_flex_store(pf, filter, pctype, &flex_mask);
- layer = &flows->layer[filter->input.flow_ext.layer_idx];
- layer->flex_flow_count++;
- layer->flex_pit_flag = true;
- state->flex_mask_flag = true;
- }
-
- state->flow_count++;
- pf->fdir.fdir_actual_cnt++;
-
- i40e_fdir_rx_proc_sync(dev);
-
- return 0;
-}
-
-/**
- * i40e_fdir_filter_unregister - drop software ownership of a filter.
- */
-int
-i40e_fdir_filter_unregister(struct rte_eth_dev *dev,
- struct i40e_fdir_filter *node)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(node);
- enum i40e_flxpld_layer_idx layer_idx = node->input.flow_ext.layer_idx;
- bool is_flex_flow = node->input.flow_ext.is_flex_flow;
- struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
- struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
- struct i40e_fdir_layer_state *layer = &flows->layer[layer_idx];
- int ret;
-
- ret = i40e_fdir_filter_hash_del(pf, node);
- if (ret < 0)
- return ret;
-
- if (is_flex_flow && --layer->flex_flow_count == 0)
- layer->flex_pit_flag = false;
-
- if (--state->flow_count == 0)
- state->flex_mask_flag = false;
-
- pf->fdir.fdir_actual_cnt--;
-
- i40e_fdir_rx_proc_sync(dev);
-
- return 0;
-}
-
/* Take software ownership of a packet template filter, owning its packet */
static int
i40e_fdir_tmpl_register(struct rte_eth_dev *dev,
const struct i40e_fdir_filter *filter,
struct i40e_fdir_tmpl_filter **node)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct rte_eth_dev_data *dev_data = dev->data;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
const struct i40e_raw_flow *raw = &filter->input.flow.raw_flow;
struct i40e_fdir_info *fdir_info = &pf->fdir;
struct i40e_fdir_tmpl_filter *tmpl;
@@ -1831,7 +1555,7 @@ i40e_fdir_tmpl_register(struct rte_eth_dev *dev,
uint8_t *packet;
int ret;
- ret = i40e_fdir_engine_init(dev);
+ ret = i40e_fdir_engine_init(pf);
if (ret < 0)
return ret;
@@ -1869,7 +1593,7 @@ i40e_fdir_tmpl_register(struct rte_eth_dev *dev,
fdir_info->fdir_actual_cnt++;
- i40e_fdir_rx_proc_sync(dev);
+ i40e_fdir_rx_proc_sync(dev_data);
*node = tmpl;
@@ -1902,7 +1626,7 @@ i40e_fdir_tmpl_unregister(struct rte_eth_dev *dev,
fdir_info->fdir_actual_cnt--;
- i40e_fdir_rx_proc_sync(dev);
+ i40e_fdir_rx_proc_sync(dev->data);
return 0;
}
@@ -1915,12 +1639,11 @@ i40e_fdir_tmpl_unregister(struct rte_eth_dev *dev,
* filter. The caller is responsible for having validated and registered it.
*/
int
-i40e_fdir_filter_program(struct rte_eth_dev *dev,
+i40e_fdir_filter_program(struct i40e_pf *pf,
const struct i40e_fdir_filter *filter,
bool add, bool wait_status)
{
- struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
unsigned char *pkt;
int ret;
@@ -1947,7 +1670,7 @@ i40e_fdir_filter_program(struct rte_eth_dev *dev,
}
}
- pkt = i40e_find_available_buffer(dev);
+ pkt = i40e_find_available_buffer(pf);
if (pkt == NULL) {
PMD_DRV_LOG(ERR, "No buffer available to program fdir filter.");
return -ENOSPC;
@@ -2007,7 +1730,7 @@ i40e_fdir_tmpl_add_del(struct rte_eth_dev *dev,
if (ret < 0)
return ret;
- ret = i40e_fdir_filter_program(dev, &node->fdir, true,
+ ret = i40e_fdir_filter_program(pf, &node->fdir, true,
i40e_fdir_filter_needs_status_wait(pf, cnt));
if (ret < 0) {
i40e_fdir_tmpl_unregister(dev, node);
@@ -2026,7 +1749,7 @@ i40e_fdir_tmpl_add_del(struct rte_eth_dev *dev,
return -EINVAL;
}
- ret = i40e_fdir_filter_program(dev, &node->fdir, false, false);
+ ret = i40e_fdir_filter_program(pf, &node->fdir, false, false);
if (ret < 0)
return ret;
@@ -2187,9 +1910,8 @@ i40e_flow_fdir_filter_programming(struct i40e_pf *pf,
* @pf: board private structure
*/
int
-i40e_fdir_flush(struct rte_eth_dev *dev)
+i40e_fdir_flush(struct i40e_pf *pf)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct i40e_hw *hw = I40E_PF_TO_HW(pf);
uint32_t reg;
uint16_t guarant_cnt, best_cnt;
@@ -2372,28 +2094,15 @@ i40e_fdir_filter_restore(struct i40e_pf *pf)
{
struct rte_eth_dev *dev = I40E_VSI_TO_ETH_DEV(pf->main_vsi);
struct i40e_fdir_tmpl_filter *tmpl;
- struct rte_flow *flow;
int ret;
- i40e_fdir_rx_proc_sync(dev);
+ i40e_fdir_rx_proc_sync(dev->data);
if (pf->fdir.fdir_vsi == NULL)
return;
- TAILQ_FOREACH(flow, &pf->flow_list, node) {
- struct i40e_fdir_filter *node = flow->rule;
-
- if (flow->filter_type != RTE_ETH_FILTER_FDIR)
- continue;
-
- ret = i40e_fdir_filter_program(dev, node, true, false);
- if (ret < 0)
- PMD_DRV_LOG(ERR,
- "Failed to restore flow director filter: %d", ret);
- }
-
TAILQ_FOREACH(tmpl, &pf->fdir.tmpls.list, rules) {
- ret = i40e_fdir_filter_program(dev, &tmpl->fdir, true, false);
+ ret = i40e_fdir_filter_program(pf, &tmpl->fdir, true, false);
if (ret < 0)
PMD_DRV_LOG(ERR,
"Failed to restore flow director template: %d", ret);
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 200de80863..104749eb8c 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -33,14 +33,10 @@
const struct ci_flow_engine_list i40e_flow_engine_list = {
{
&i40e_flow_engine_ethertype,
+ &i40e_flow_engine_fdir,
}
};
-#define I40E_IPV6_TC_MASK (0xFF << I40E_FDIR_IPv6_TC_OFFSET)
-#define I40E_IPV6_FRAG_HEADER 44
-#define I40E_TENANT_ARRAY_NUM 3
-#define I40E_VLAN_TCI_MASK 0xFFFF
-
static int i40e_flow_validate(struct rte_eth_dev *dev,
const struct rte_flow_attr *attr,
const struct rte_flow_item pattern[],
@@ -64,23 +60,10 @@ static int i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow *flow,
FILE *file,
struct rte_flow_error *error);
-static int i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_fdir_filter *filter);
-static int i40e_flow_parse_fdir_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct i40e_fdir_filter *filter);
static int i40e_flow_parse_tunnel_action(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
struct rte_flow_error *error,
struct i40e_tunnel_filter_conf *filter);
-static int i40e_flow_parse_fdir_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
static int i40e_flow_parse_vxlan_filter(struct rte_eth_dev *dev,
const struct rte_flow_item pattern[],
const struct rte_flow_action actions[],
@@ -103,7 +86,6 @@ static int i40e_flow_parse_gtp_filter(struct rte_eth_dev *dev,
struct i40e_filter_ctx *filter);
static int i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
struct i40e_tunnel_filter *filter);
-static int i40e_flow_flush_fdir_filter(struct i40e_pf *pf);
static int i40e_flow_flush_tunnel_filter(struct i40e_pf *pf);
static int
i40e_flow_parse_qinq_filter(struct rte_eth_dev *dev,
@@ -131,19 +113,6 @@ const struct rte_flow_ops i40e_flow_ops = {
.dev_dump = i40e_flow_dev_dump,
};
-/* Pattern matched ethertype filter */
-static enum rte_flow_item_type pattern_ethertype[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-/* Pattern matched flow director filter */
-static enum rte_flow_item_type pattern_fdir_ipv4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
static enum rte_flow_item_type pattern_fdir_ipv4_udp[] = {
RTE_FLOW_ITEM_TYPE_ETH,
RTE_FLOW_ITEM_TYPE_IPV4,
@@ -181,30 +150,6 @@ static enum rte_flow_item_type pattern_fdir_ipv4_gtpu[] = {
RTE_FLOW_ITEM_TYPE_END,
};
-static enum rte_flow_item_type pattern_fdir_ipv4_gtpu_ipv4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_gtpu_ipv6[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
static enum rte_flow_item_type pattern_fdir_ipv6_udp[] = {
RTE_FLOW_ITEM_TYPE_ETH,
RTE_FLOW_ITEM_TYPE_IPV6,
@@ -242,581 +187,6 @@ static enum rte_flow_item_type pattern_fdir_ipv6_gtpu[] = {
RTE_FLOW_ITEM_TYPE_END,
};
-static enum rte_flow_item_type pattern_fdir_ipv6_gtpu_ipv4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_gtpu_ipv6[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_udp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_udp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_udp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_tcp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_tcp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_tcp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_sctp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_sctp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_sctp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_udp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_udp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_udp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_tcp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_tcp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_tcp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_sctp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_sctp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_sctp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_vlan[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_udp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_tcp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_sctp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_udp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_tcp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_sctp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_vlan_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_vlan_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ethertype_vlan_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_udp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_udp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_udp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_tcp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_tcp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_tcp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_sctp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_sctp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv4_sctp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_udp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_udp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_udp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_tcp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_tcp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_tcp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_sctp_raw_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_sctp_raw_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_vlan_ipv6_sctp_raw_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_RAW,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
/* Pattern matched tunnel filter */
static enum rte_flow_item_type pattern_vxlan_1[] = {
RTE_FLOW_ITEM_TYPE_ETH,
@@ -929,138 +299,7 @@ static enum rte_flow_item_type pattern_qinq_1[] = {
RTE_FLOW_ITEM_TYPE_END,
};
-static enum rte_flow_item_type pattern_fdir_ipv4_l2tpv3oip[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_L2TPV3OIP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_l2tpv3oip[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_L2TPV3OIP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_esp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_ESP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_esp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_ESP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_udp_esp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_ESP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_udp_esp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_ESP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
static struct i40e_valid_pattern i40e_supported_patterns[] = {
- /* FDIR - support default flow type without flexible payload*/
- { pattern_ethertype, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_udp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_tcp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_sctp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_gtpc, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_gtpu, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_gtpu_ipv4, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_gtpu_ipv6, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_esp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_udp_esp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_udp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_tcp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_sctp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_gtpc, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_gtpu, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_gtpu_ipv4, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_gtpu_ipv6, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_esp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_udp_esp, i40e_flow_parse_fdir_filter },
- /* FDIR - support default flow type with flexible payload */
- { pattern_fdir_ethertype_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ethertype_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ethertype_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_udp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_udp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_udp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_tcp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_tcp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_tcp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_sctp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_sctp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv4_sctp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_udp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_udp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_udp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_tcp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_tcp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_tcp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_sctp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_sctp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_sctp_raw_3, i40e_flow_parse_fdir_filter },
- /* FDIR - support single vlan input set */
- { pattern_fdir_ethertype_vlan, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_udp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_tcp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_sctp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_udp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_tcp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_sctp, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ethertype_vlan_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ethertype_vlan_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ethertype_vlan_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_udp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_udp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_udp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_tcp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_tcp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_tcp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_sctp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_sctp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv4_sctp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_udp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_udp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_udp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_tcp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_tcp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_tcp_raw_3, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_sctp_raw_1, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_sctp_raw_2, i40e_flow_parse_fdir_filter },
- { pattern_fdir_vlan_ipv6_sctp_raw_3, i40e_flow_parse_fdir_filter },
/* VXLAN */
{ pattern_vxlan_1, i40e_flow_parse_vxlan_filter },
{ pattern_vxlan_2, i40e_flow_parse_vxlan_filter },
@@ -1083,9 +322,6 @@ static struct i40e_valid_pattern i40e_supported_patterns[] = {
{ pattern_fdir_ipv6_gtpu, i40e_flow_parse_gtp_filter },
/* QINQ */
{ pattern_qinq_1, i40e_flow_parse_qinq_filter },
- /* L2TPv3 over IP */
- { pattern_fdir_ipv4_l2tpv3oip, i40e_flow_parse_fdir_filter },
- { pattern_fdir_ipv6_l2tpv3oip, i40e_flow_parse_fdir_filter },
/* L4 over port */
{ pattern_fdir_ipv4_udp, i40e_flow_parse_l4_cloud_filter },
{ pattern_fdir_ipv4_tcp, i40e_flow_parse_l4_cloud_filter },
@@ -1191,8 +427,6 @@ static const char *
i40e_flow_rule_name(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_FDIR:
- return "fdir";
case RTE_ETH_FILTER_TUNNEL:
return "tunnel";
case RTE_ETH_FILTER_HASH:
@@ -1206,8 +440,6 @@ static size_t
i40e_flow_rule_size(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_FDIR:
- return sizeof(struct i40e_fdir_filter);
case RTE_ETH_FILTER_TUNNEL:
return sizeof(struct i40e_tunnel_filter);
case RTE_ETH_FILTER_HASH:
@@ -1332,47 +564,7 @@ i40e_get_outer_vlan(struct i40e_pf *pf, uint16_t *tpid)
return 0;
}
-static int
-i40e_flow_check_raw_item(const struct rte_flow_item *item,
- const struct rte_flow_item_raw *raw_spec,
- struct rte_flow_error *error)
-{
- if (!raw_spec->relative) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Relative should be 1.");
- return -rte_errno;
- }
-
- if (raw_spec->offset % sizeof(uint16_t)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Offset should be even.");
- return -rte_errno;
- }
-
- if (raw_spec->search || raw_spec->limit) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "search or limit is not supported.");
- return -rte_errno;
- }
-
- if (raw_spec->offset < 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Offset should be non-negative.");
- return -rte_errno;
- }
- return 0;
-}
-
-
-static uint8_t
+uint8_t
i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
enum rte_flow_item_type item_type,
struct i40e_fdir_filter *filter)
@@ -1439,1013 +631,6 @@ i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
return I40E_FILTER_PCTYPE_INVALID;
}
-static void
-i40e_flow_set_filter_spi(struct i40e_fdir_filter *filter,
- const struct rte_flow_item_esp *esp_spec)
-{
- if (filter->input.flow_ext.oip_type ==
- I40E_FDIR_IPTYPE_IPV4) {
- if (filter->input.flow_ext.is_udp)
- filter->input.flow.esp_ipv4_udp_flow.spi =
- esp_spec->hdr.spi;
- else
- filter->input.flow.esp_ipv4_flow.spi =
- esp_spec->hdr.spi;
- }
- if (filter->input.flow_ext.oip_type ==
- I40E_FDIR_IPTYPE_IPV6) {
- if (filter->input.flow_ext.is_udp)
- filter->input.flow.esp_ipv6_udp_flow.spi =
- esp_spec->hdr.spi;
- else
- filter->input.flow.esp_ipv6_flow.spi =
- esp_spec->hdr.spi;
- }
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported patterns: refer to array i40e_supported_patterns.
- * 3. Default supported flow type and input set: refer to array
- * valid_fdir_inset_table in i40e_ethdev.c.
- * 4. Mask of fields which need to be matched should be
- * filled with 1.
- * 5. Mask of fields which needn't to be matched should be
- * filled with 0.
- * 6. GTP profile supports GTPv1 only.
- * 7. GTP-C response message ('source_port' = 2123) is not supported.
- */
-static int
-i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_fdir_filter *filter)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_eth *eth_spec, *eth_mask;
- const struct rte_flow_item_vlan *vlan_spec, *vlan_mask;
- const struct rte_flow_item_ipv4 *ipv4_spec, *ipv4_last, *ipv4_mask;
- const struct rte_flow_item_ipv6 *ipv6_spec, *ipv6_mask;
- const struct rte_flow_item_tcp *tcp_spec, *tcp_mask;
- const struct rte_flow_item_udp *udp_spec, *udp_mask;
- const struct rte_flow_item_sctp *sctp_spec, *sctp_mask;
- const struct rte_flow_item_gtp *gtp_spec, *gtp_mask;
- const struct rte_flow_item_esp *esp_spec, *esp_mask;
- const struct rte_flow_item_raw *raw_spec, *raw_mask;
- const struct rte_flow_item_l2tpv3oip *l2tpv3oip_spec, *l2tpv3oip_mask;
-
- uint8_t pctype = 0;
- uint64_t input_set = I40E_INSET_NONE;
- enum rte_flow_item_type item_type;
- enum rte_flow_item_type next_type;
- enum rte_flow_item_type l3 = RTE_FLOW_ITEM_TYPE_END;
- enum rte_flow_item_type cus_proto = RTE_FLOW_ITEM_TYPE_END;
- uint32_t i, j;
- uint8_t ipv6_addr_mask[16] = {
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
- enum i40e_flxpld_layer_idx layer_idx = I40E_FLXPLD_L2_IDX;
- uint8_t raw_id = 0;
- int32_t off_arr[I40E_MAX_FLXPLD_FIED];
- uint16_t len_arr[I40E_MAX_FLXPLD_FIED];
- struct i40e_fdir_flex_pit flex_pit;
- uint8_t next_dst_off = 0;
- uint16_t flex_size;
- uint16_t ether_type;
- uint32_t vtc_flow_cpu;
- bool outer_ip = true;
- uint8_t field_idx;
- int ret;
- uint16_t tpid;
-
- memset(off_arr, 0, sizeof(off_arr));
- memset(len_arr, 0, sizeof(len_arr));
- filter->input.flow_ext.customized_pctype = false;
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last && item->type != RTE_FLOW_ITEM_TYPE_IPV4) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- eth_spec = item->spec;
- eth_mask = item->mask;
- next_type = (item + 1)->type;
-
- if (next_type == RTE_FLOW_ITEM_TYPE_END &&
- (!eth_spec || !eth_mask)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "NULL eth spec/mask.");
- return -rte_errno;
- }
-
- if (eth_spec && eth_mask) {
- if (rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr) &&
- rte_is_zero_ether_addr(ð_mask->hdr.src_addr)) {
- filter->input.flow.l2_flow.dst =
- eth_spec->hdr.dst_addr;
- input_set |= I40E_INSET_DMAC;
- } else if (rte_is_zero_ether_addr(ð_mask->hdr.dst_addr) &&
- rte_is_broadcast_ether_addr(ð_mask->hdr.src_addr)) {
- filter->input.flow.l2_flow.src =
- eth_spec->hdr.src_addr;
- input_set |= I40E_INSET_SMAC;
- } else if (rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr) &&
- rte_is_broadcast_ether_addr(ð_mask->hdr.src_addr)) {
- filter->input.flow.l2_flow.dst =
- eth_spec->hdr.dst_addr;
- filter->input.flow.l2_flow.src =
- eth_spec->hdr.src_addr;
- input_set |= (I40E_INSET_DMAC | I40E_INSET_SMAC);
- } else if (!rte_is_zero_ether_addr(ð_mask->hdr.src_addr) ||
- !rte_is_zero_ether_addr(ð_mask->hdr.dst_addr)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid MAC_addr mask.");
- return -rte_errno;
- }
- }
- if (eth_spec && eth_mask &&
- next_type == RTE_FLOW_ITEM_TYPE_END) {
- if (eth_mask->hdr.ether_type != RTE_BE16(0xffff)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid type mask.");
- return -rte_errno;
- }
-
- ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
-
- if (ether_type == RTE_ETHER_TYPE_IPV4 ||
- ether_type == RTE_ETHER_TYPE_IPV6) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ether_type.");
- return -rte_errno;
- }
- ret = i40e_get_outer_vlan(pf, &tpid);
- if (ret != 0) {
- rte_flow_error_set(error, EIO,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Can not get the Ethertype identifying the L2 tag");
- return -rte_errno;
- }
- if (ether_type == tpid) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ether_type.");
- return -rte_errno;
- }
-
- input_set |= I40E_INSET_LAST_ETHER_TYPE;
- filter->input.flow.l2_flow.ether_type =
- eth_spec->hdr.ether_type;
- }
-
- pctype = I40E_FILTER_PCTYPE_L2_PAYLOAD;
- layer_idx = I40E_FLXPLD_L2_IDX;
-
- break;
- case RTE_FLOW_ITEM_TYPE_VLAN:
- vlan_spec = item->spec;
- vlan_mask = item->mask;
-
- RTE_ASSERT(!(input_set & I40E_INSET_LAST_ETHER_TYPE));
- if (vlan_spec && vlan_mask) {
- if (vlan_mask->hdr.vlan_tci != 0 &&
- vlan_mask->hdr.vlan_tci !=
- rte_cpu_to_be_16(I40E_VLAN_TCI_MASK)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported TCI mask.");
- return -rte_errno;
- }
- if (vlan_mask->hdr.vlan_tci != 0) {
- input_set |= I40E_INSET_VLAN_INNER;
- filter->input.flow_ext.vlan_tci = vlan_spec->hdr.vlan_tci;
- }
- }
- if (vlan_spec && vlan_mask && vlan_mask->hdr.eth_proto) {
- if (vlan_mask->hdr.eth_proto != RTE_BE16(0xffff)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid inner_type"
- " mask.");
- return -rte_errno;
- }
-
- ether_type =
- rte_be_to_cpu_16(vlan_spec->hdr.eth_proto);
-
- if (ether_type == RTE_ETHER_TYPE_IPV4 ||
- ether_type == RTE_ETHER_TYPE_IPV6) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported inner_type.");
- return -rte_errno;
- }
- ret = i40e_get_outer_vlan(pf, &tpid);
- if (ret != 0) {
- rte_flow_error_set(error, EIO,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Can not get the Ethertype identifying the L2 tag");
- return -rte_errno;
- }
- if (ether_type == tpid) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ether_type.");
- return -rte_errno;
- }
-
- input_set |= I40E_INSET_LAST_ETHER_TYPE;
- filter->input.flow.l2_flow.ether_type =
- vlan_spec->hdr.eth_proto;
- }
-
- pctype = I40E_FILTER_PCTYPE_L2_PAYLOAD;
- layer_idx = I40E_FLXPLD_L2_IDX;
-
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- l3 = RTE_FLOW_ITEM_TYPE_IPV4;
- ipv4_spec = item->spec;
- ipv4_mask = item->mask;
- ipv4_last = item->last;
- pctype = I40E_FILTER_PCTYPE_NONF_IPV4_OTHER;
- layer_idx = I40E_FLXPLD_L3_IDX;
-
- if (ipv4_last) {
- if (!ipv4_spec || !ipv4_mask || !outer_ip) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- /* Only fragment_offset supports range */
- if (ipv4_last->hdr.version_ihl ||
- ipv4_last->hdr.type_of_service ||
- ipv4_last->hdr.total_length ||
- ipv4_last->hdr.packet_id ||
- ipv4_last->hdr.time_to_live ||
- ipv4_last->hdr.next_proto_id ||
- ipv4_last->hdr.hdr_checksum ||
- ipv4_last->hdr.src_addr ||
- ipv4_last->hdr.dst_addr) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- }
- if (ipv4_spec && ipv4_mask && outer_ip) {
- /* Check IPv4 mask and update input set */
- if (ipv4_mask->hdr.version_ihl ||
- ipv4_mask->hdr.total_length ||
- ipv4_mask->hdr.packet_id ||
- ipv4_mask->hdr.hdr_checksum) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 mask.");
- return -rte_errno;
- }
-
- if (ipv4_mask->hdr.src_addr == UINT32_MAX)
- input_set |= I40E_INSET_IPV4_SRC;
- if (ipv4_mask->hdr.dst_addr == UINT32_MAX)
- input_set |= I40E_INSET_IPV4_DST;
- if (ipv4_mask->hdr.type_of_service == UINT8_MAX)
- input_set |= I40E_INSET_IPV4_TOS;
- if (ipv4_mask->hdr.time_to_live == UINT8_MAX)
- input_set |= I40E_INSET_IPV4_TTL;
- if (ipv4_mask->hdr.next_proto_id == UINT8_MAX)
- input_set |= I40E_INSET_IPV4_PROTO;
-
- /* Check if it is fragment. */
- uint16_t frag_mask =
- ipv4_mask->hdr.fragment_offset;
- uint16_t frag_spec =
- ipv4_spec->hdr.fragment_offset;
- uint16_t frag_last = 0;
- if (ipv4_last)
- frag_last =
- ipv4_last->hdr.fragment_offset;
- if (frag_mask) {
- frag_mask = rte_be_to_cpu_16(frag_mask);
- frag_spec = rte_be_to_cpu_16(frag_spec);
- frag_last = rte_be_to_cpu_16(frag_last);
- /* frag_off mask has to be 0x3fff */
- if (frag_mask !=
- (RTE_IPV4_HDR_OFFSET_MASK |
- RTE_IPV4_HDR_MF_FLAG)) {
- rte_flow_error_set(error,
- EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 fragment_offset mask");
- return -rte_errno;
- }
- /*
- * non-frag rule:
- * mask=0x3fff,spec=0
- * frag rule:
- * mask=0x3fff,spec=0x8,last=0x2000
- */
- if (frag_spec ==
- (1 << RTE_IPV4_HDR_FO_SHIFT) &&
- frag_last == RTE_IPV4_HDR_MF_FLAG) {
- pctype =
- I40E_FILTER_PCTYPE_FRAG_IPV4;
- } else if (frag_spec || frag_last) {
- rte_flow_error_set(error,
- EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 fragment_offset rule");
- return -rte_errno;
- }
- } else if (frag_spec || frag_last) {
- rte_flow_error_set(error,
- EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid fragment_offset");
- return -rte_errno;
- }
-
- if (input_set & (I40E_INSET_DMAC | I40E_INSET_SMAC)) {
- if (input_set & (I40E_INSET_IPV4_SRC |
- I40E_INSET_IPV4_DST | I40E_INSET_IPV4_TOS |
- I40E_INSET_IPV4_TTL | I40E_INSET_IPV4_PROTO)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "L2 and L3 input set are exclusive.");
- return -rte_errno;
- }
- } else {
- /* Get the filter info */
- filter->input.flow.ip4_flow.proto =
- ipv4_spec->hdr.next_proto_id;
- filter->input.flow.ip4_flow.tos =
- ipv4_spec->hdr.type_of_service;
- filter->input.flow.ip4_flow.ttl =
- ipv4_spec->hdr.time_to_live;
- filter->input.flow.ip4_flow.src_ip =
- ipv4_spec->hdr.src_addr;
- filter->input.flow.ip4_flow.dst_ip =
- ipv4_spec->hdr.dst_addr;
-
- filter->input.flow_ext.inner_ip = false;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV4;
- }
- } else if (!ipv4_spec && !ipv4_mask && !outer_ip) {
- filter->input.flow_ext.inner_ip = true;
- filter->input.flow_ext.iip_type =
- I40E_FDIR_IPTYPE_IPV4;
- } else if (!ipv4_spec && !ipv4_mask && outer_ip) {
- filter->input.flow_ext.inner_ip = false;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV4;
- } else if ((ipv4_spec || ipv4_mask) && !outer_ip) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid inner IPv4 mask.");
- return -rte_errno;
- }
-
- if (outer_ip)
- outer_ip = false;
-
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- l3 = RTE_FLOW_ITEM_TYPE_IPV6;
- ipv6_spec = item->spec;
- ipv6_mask = item->mask;
- pctype = I40E_FILTER_PCTYPE_NONF_IPV6_OTHER;
- layer_idx = I40E_FLXPLD_L3_IDX;
-
- if (ipv6_spec && ipv6_mask && outer_ip) {
- /* Check IPv6 mask and update input set */
- if (ipv6_mask->hdr.payload_len) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 mask");
- return -rte_errno;
- }
-
- if (!memcmp(&ipv6_mask->hdr.src_addr,
- ipv6_addr_mask,
- sizeof(ipv6_mask->hdr.src_addr)))
- input_set |= I40E_INSET_IPV6_SRC;
- if (!memcmp(&ipv6_mask->hdr.dst_addr,
- ipv6_addr_mask,
- sizeof(ipv6_mask->hdr.dst_addr)))
- input_set |= I40E_INSET_IPV6_DST;
-
- if ((ipv6_mask->hdr.vtc_flow &
- rte_cpu_to_be_32(I40E_IPV6_TC_MASK))
- == rte_cpu_to_be_32(I40E_IPV6_TC_MASK))
- input_set |= I40E_INSET_IPV6_TC;
- if (ipv6_mask->hdr.proto == UINT8_MAX)
- input_set |= I40E_INSET_IPV6_NEXT_HDR;
- if (ipv6_mask->hdr.hop_limits == UINT8_MAX)
- input_set |= I40E_INSET_IPV6_HOP_LIMIT;
-
- /* Get filter info */
- vtc_flow_cpu =
- rte_be_to_cpu_32(ipv6_spec->hdr.vtc_flow);
- filter->input.flow.ipv6_flow.tc =
- (uint8_t)(vtc_flow_cpu >>
- I40E_FDIR_IPv6_TC_OFFSET);
- filter->input.flow.ipv6_flow.proto =
- ipv6_spec->hdr.proto;
- filter->input.flow.ipv6_flow.hop_limits =
- ipv6_spec->hdr.hop_limits;
-
- filter->input.flow_ext.inner_ip = false;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV6;
-
- memcpy(filter->input.flow.ipv6_flow.src_ip,
- &ipv6_spec->hdr.src_addr, 16);
- memcpy(filter->input.flow.ipv6_flow.dst_ip,
- &ipv6_spec->hdr.dst_addr, 16);
-
- /* Check if it is fragment. */
- if (ipv6_spec->hdr.proto ==
- I40E_IPV6_FRAG_HEADER)
- pctype = I40E_FILTER_PCTYPE_FRAG_IPV6;
- } else if (!ipv6_spec && !ipv6_mask && !outer_ip) {
- filter->input.flow_ext.inner_ip = true;
- filter->input.flow_ext.iip_type =
- I40E_FDIR_IPTYPE_IPV6;
- } else if (!ipv6_spec && !ipv6_mask && outer_ip) {
- filter->input.flow_ext.inner_ip = false;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV6;
- } else if ((ipv6_spec || ipv6_mask) && !outer_ip) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid inner IPv6 mask");
- return -rte_errno;
- }
-
- if (outer_ip)
- outer_ip = false;
- break;
- case RTE_FLOW_ITEM_TYPE_TCP:
- tcp_spec = item->spec;
- tcp_mask = item->mask;
-
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV4_TCP;
- else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV6_TCP;
- if (tcp_spec && tcp_mask) {
- /* Check TCP mask and update input set */
- if (tcp_mask->hdr.sent_seq ||
- tcp_mask->hdr.recv_ack ||
- tcp_mask->hdr.data_off ||
- tcp_mask->hdr.tcp_flags ||
- tcp_mask->hdr.rx_win ||
- tcp_mask->hdr.cksum ||
- tcp_mask->hdr.tcp_urp) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid TCP mask");
- return -rte_errno;
- }
-
- if (tcp_mask->hdr.src_port == UINT16_MAX)
- input_set |= I40E_INSET_SRC_PORT;
- if (tcp_mask->hdr.dst_port == UINT16_MAX)
- input_set |= I40E_INSET_DST_PORT;
-
- if (input_set & (I40E_INSET_DMAC | I40E_INSET_SMAC)) {
- if (input_set &
- (I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "L2 and L4 input set are exclusive.");
- return -rte_errno;
- }
- } else {
- /* Get filter info */
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4) {
- filter->input.flow.tcp4_flow.src_port =
- tcp_spec->hdr.src_port;
- filter->input.flow.tcp4_flow.dst_port =
- tcp_spec->hdr.dst_port;
- } else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6) {
- filter->input.flow.tcp6_flow.src_port =
- tcp_spec->hdr.src_port;
- filter->input.flow.tcp6_flow.dst_port =
- tcp_spec->hdr.dst_port;
- }
- }
- }
-
- layer_idx = I40E_FLXPLD_L4_IDX;
-
- break;
- case RTE_FLOW_ITEM_TYPE_UDP:
- udp_spec = item->spec;
- udp_mask = item->mask;
-
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
- else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV6_UDP;
-
- if (udp_spec && udp_mask) {
- /* Check UDP mask and update input set*/
- if (udp_mask->hdr.dgram_len ||
- udp_mask->hdr.dgram_cksum) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid UDP mask");
- return -rte_errno;
- }
-
- if (udp_mask->hdr.src_port == UINT16_MAX)
- input_set |= I40E_INSET_SRC_PORT;
- if (udp_mask->hdr.dst_port == UINT16_MAX)
- input_set |= I40E_INSET_DST_PORT;
-
- if (input_set & (I40E_INSET_DMAC | I40E_INSET_SMAC)) {
- if (input_set &
- (I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "L2 and L4 input set are exclusive.");
- return -rte_errno;
- }
- } else {
- /* Get filter info */
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4) {
- filter->input.flow.udp4_flow.src_port =
- udp_spec->hdr.src_port;
- filter->input.flow.udp4_flow.dst_port =
- udp_spec->hdr.dst_port;
- } else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6) {
- filter->input.flow.udp6_flow.src_port =
- udp_spec->hdr.src_port;
- filter->input.flow.udp6_flow.dst_port =
- udp_spec->hdr.dst_port;
- }
- }
- }
- filter->input.flow_ext.is_udp = true;
- layer_idx = I40E_FLXPLD_L4_IDX;
-
- break;
- case RTE_FLOW_ITEM_TYPE_GTPC:
- case RTE_FLOW_ITEM_TYPE_GTPU:
- if (!pf->gtp_support) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported protocol");
- return -rte_errno;
- }
-
- gtp_spec = item->spec;
- gtp_mask = item->mask;
-
- if (gtp_spec && gtp_mask) {
- if (gtp_mask->hdr.gtp_hdr_info ||
- gtp_mask->hdr.msg_type ||
- gtp_mask->hdr.plen ||
- gtp_mask->hdr.teid != UINT32_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid GTP mask");
- return -rte_errno;
- }
-
- filter->input.flow.gtp_flow.teid =
- gtp_spec->hdr.teid;
- filter->input.flow_ext.customized_pctype = true;
- cus_proto = item_type;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_ESP:
- if (!pf->esp_support) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported ESP protocol");
- return -rte_errno;
- }
-
- esp_spec = item->spec;
- esp_mask = item->mask;
-
- if (!esp_spec || !esp_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ESP item");
- return -rte_errno;
- }
-
- if (esp_spec && esp_mask) {
- if (esp_mask->hdr.spi != UINT32_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ESP mask");
- return -rte_errno;
- }
- i40e_flow_set_filter_spi(filter, esp_spec);
- filter->input.flow_ext.customized_pctype = true;
- cus_proto = item_type;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_SCTP:
- sctp_spec = item->spec;
- sctp_mask = item->mask;
-
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV4_SCTP;
- else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- pctype =
- I40E_FILTER_PCTYPE_NONF_IPV6_SCTP;
-
- if (sctp_spec && sctp_mask) {
- /* Check SCTP mask and update input set */
- if (sctp_mask->hdr.cksum) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid UDP mask");
- return -rte_errno;
- }
-
- if (sctp_mask->hdr.src_port == UINT16_MAX)
- input_set |= I40E_INSET_SRC_PORT;
- if (sctp_mask->hdr.dst_port == UINT16_MAX)
- input_set |= I40E_INSET_DST_PORT;
- if (sctp_mask->hdr.tag == UINT32_MAX)
- input_set |= I40E_INSET_SCTP_VT;
-
- /* Get filter info */
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4) {
- filter->input.flow.sctp4_flow.src_port =
- sctp_spec->hdr.src_port;
- filter->input.flow.sctp4_flow.dst_port =
- sctp_spec->hdr.dst_port;
- filter->input.flow.sctp4_flow.verify_tag
- = sctp_spec->hdr.tag;
- } else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6) {
- filter->input.flow.sctp6_flow.src_port =
- sctp_spec->hdr.src_port;
- filter->input.flow.sctp6_flow.dst_port =
- sctp_spec->hdr.dst_port;
- filter->input.flow.sctp6_flow.verify_tag
- = sctp_spec->hdr.tag;
- }
- }
-
- layer_idx = I40E_FLXPLD_L4_IDX;
-
- break;
- case RTE_FLOW_ITEM_TYPE_RAW:
- raw_spec = item->spec;
- raw_mask = item->mask;
-
- if (!raw_spec || !raw_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "NULL RAW spec/mask");
- return -rte_errno;
- }
-
- if (pf->support_multi_driver) {
- rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported flexible payload.");
- return -rte_errno;
- }
-
- ret = i40e_flow_check_raw_item(item, raw_spec, error);
- if (ret < 0)
- return ret;
-
- off_arr[raw_id] = raw_spec->offset;
- len_arr[raw_id] = raw_spec->length;
-
- flex_size = 0;
- memset(&flex_pit, 0, sizeof(struct i40e_fdir_flex_pit));
- field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + raw_id;
- flex_pit.size =
- raw_spec->length / sizeof(uint16_t);
- flex_pit.dst_offset =
- next_dst_off / sizeof(uint16_t);
-
- for (i = 0; i <= raw_id; i++) {
- if (i == raw_id)
- flex_pit.src_offset +=
- raw_spec->offset /
- sizeof(uint16_t);
- else
- flex_pit.src_offset +=
- (off_arr[i] + len_arr[i]) /
- sizeof(uint16_t);
- flex_size += len_arr[i];
- }
- if (((flex_pit.src_offset + flex_pit.size) >=
- I40E_MAX_FLX_SOURCE_OFF / sizeof(uint16_t)) ||
- flex_size > I40E_FDIR_MAX_FLEXLEN) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Exceeds maximal payload limit.");
- return -rte_errno;
- }
-
- if (raw_spec->length != 0) {
- if (raw_spec->pattern == NULL) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "NULL RAW spec pattern");
- return -rte_errno;
- }
- if (raw_mask->pattern == NULL) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "NULL RAW mask pattern");
- return -rte_errno;
- }
- }
-
- for (i = 0; i < raw_spec->length; i++) {
- j = i + next_dst_off;
- if (j >= RTE_ETH_FDIR_MAX_FLEXLEN ||
- j >= I40E_FDIR_MAX_FLEX_LEN)
- break;
- filter->input.flow_ext.flexbytes[j] =
- raw_spec->pattern[i];
- filter->input.flow_ext.flex_mask[j] =
- raw_mask->pattern[i];
- }
-
- next_dst_off += raw_spec->length;
- raw_id++;
-
- filter->input.flow_ext.flex_pit[field_idx] = flex_pit;
- filter->input.flow_ext.layer_idx = layer_idx;
- filter->input.flow_ext.raw_id = raw_id;
- filter->input.flow_ext.is_flex_flow = true;
- break;
- case RTE_FLOW_ITEM_TYPE_L2TPV3OIP:
- l2tpv3oip_spec = item->spec;
- l2tpv3oip_mask = item->mask;
-
- if (!l2tpv3oip_spec || !l2tpv3oip_mask)
- break;
-
- if (l2tpv3oip_mask->session_id != UINT32_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid L2TPv3 mask");
- return -rte_errno;
- }
-
- if (l3 == RTE_FLOW_ITEM_TYPE_IPV4) {
- filter->input.flow.ip4_l2tpv3oip_flow.session_id =
- l2tpv3oip_spec->session_id;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV4;
- } else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6) {
- filter->input.flow.ip6_l2tpv3oip_flow.session_id =
- l2tpv3oip_spec->session_id;
- filter->input.flow_ext.oip_type =
- I40E_FDIR_IPTYPE_IPV6;
- }
-
- filter->input.flow_ext.customized_pctype = true;
- cus_proto = item_type;
- break;
- default:
- break;
- }
- }
-
- /* Get customized pctype value */
- if (filter->input.flow_ext.customized_pctype) {
- pctype = i40e_flow_fdir_get_pctype_value(pf, cus_proto, filter);
- if (pctype == I40E_FILTER_PCTYPE_INVALID) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Unsupported pctype");
- return -rte_errno;
- }
- }
-
- /* If customized pctype is not used, set fdir configuration.*/
- if (!filter->input.flow_ext.customized_pctype) {
- /* Check if the input set is valid */
- if (i40e_validate_input_set(pctype, RTE_ETH_FILTER_FDIR,
- input_set) != 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid input set");
- return -rte_errno;
- }
-
- filter->input.flow_ext.input_set = input_set;
- }
-
- filter->input.pctype = pctype;
-
- return 0;
-}
-
-/* Parse to get the action info of a FDIR filter.
- * FDIR action supports QUEUE or (QUEUE + MARK).
- */
-static int
-i40e_flow_parse_fdir_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct i40e_fdir_filter *filter)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct ci_flow_actions parsed_actions = {0};
- struct ci_flow_actions_check_param ac_param = {
- .allowed_types = (enum rte_flow_action_type[]) {
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_DROP,
- RTE_FLOW_ACTION_TYPE_PASSTHRU,
- RTE_FLOW_ACTION_TYPE_MARK,
- RTE_FLOW_ACTION_TYPE_FLAG,
- RTE_FLOW_ACTION_TYPE_RSS,
- RTE_FLOW_ACTION_TYPE_END
- },
- .max_actions = 2,
- };
- const struct rte_flow_action *first, *second;
- int ret;
-
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- first = parsed_actions.actions[0];
- /* can be NULL */
- second = parsed_actions.actions[1];
-
- switch (first->type) {
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- {
- const struct rte_flow_action_queue *act_q = first->conf;
- /* check against PF constraints */
- if (!filter->input.flow_ext.is_vf && act_q->index >= pf->dev_data->nb_rx_queues) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, first,
- "Invalid queue ID for FDIR");
- }
- /* check against VF constraints */
- if (filter->input.flow_ext.is_vf && act_q->index >= pf->vf_nb_qps) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, first,
- "Invalid queue ID for FDIR");
- }
- filter->action.rx_queue = act_q->index;
- filter->action.behavior = I40E_FDIR_ACCEPT;
- break;
- }
- case RTE_FLOW_ACTION_TYPE_DROP:
- filter->action.behavior = I40E_FDIR_REJECT;
- break;
- case RTE_FLOW_ACTION_TYPE_PASSTHRU:
- filter->action.behavior = I40E_FDIR_PASSTHRU;
- break;
- case RTE_FLOW_ACTION_TYPE_MARK:
- {
- const struct rte_flow_action_mark *act_m = first->conf;
- filter->action.behavior = I40E_FDIR_PASSTHRU;
- filter->action.report_status = I40E_FDIR_REPORT_ID;
- filter->soft_id = act_m->id;
- break;
- }
- default:
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, first,
- "Invalid first action for FDIR");
- }
-
- /* do we have another? */
- if (second == NULL)
- return 0;
-
- switch (second->type) {
- case RTE_FLOW_ACTION_TYPE_MARK:
- {
- const struct rte_flow_action_mark *act_m = second->conf;
- /* only one mark action can be specified */
- if (first->type == RTE_FLOW_ACTION_TYPE_MARK) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, second,
- "Invalid second action for FDIR");
- }
- filter->action.report_status = I40E_FDIR_REPORT_ID;
- filter->soft_id = act_m->id;
- break;
- }
- case RTE_FLOW_ACTION_TYPE_FLAG:
- {
- /* mark + flag is unsupported */
- if (first->type == RTE_FLOW_ACTION_TYPE_MARK) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, second,
- "Invalid second action for FDIR");
- }
- filter->action.report_status = I40E_FDIR_NO_REPORT_STATUS;
- break;
- }
- case RTE_FLOW_ACTION_TYPE_RSS:
- /* RSS filter only can be after passthru or mark */
- if (first->type != RTE_FLOW_ACTION_TYPE_PASSTHRU &&
- first->type != RTE_FLOW_ACTION_TYPE_MARK) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, second,
- "Invalid second action for FDIR");
- }
- break;
- default:
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, second,
- "Invalid second action for FDIR");
- }
-
- return 0;
-}
-
-static int
-i40e_flow_parse_fdir_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_fdir_filter *fdir_filter = &filter->fdir_filter;
- int ret;
-
- ret = i40e_flow_parse_fdir_pattern(dev, pattern, error, fdir_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_fdir_action(dev, actions, error, fdir_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_FDIR;
-
- return 0;
-}
-
/* Parse to get the action info of a tunnel filter
* Tunnel action only supports PF, VF and QUEUE.
*/
@@ -3743,7 +1928,6 @@ i40e_flow_create(struct rte_eth_dev *dev,
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct i40e_filter_ctx filter_ctx = {0};
struct rte_flow *flow = NULL;
- struct i40e_fdir_info *fdir_info = &pf->fdir;
int ret;
/* try the new engine first */
@@ -3755,46 +1939,15 @@ i40e_flow_create(struct rte_eth_dev *dev,
if (ret < 0)
return NULL;
- if (filter_ctx.type == RTE_ETH_FILTER_FDIR) {
- flow = i40e_fdir_entry_pool_get(fdir_info);
- if (flow == NULL) {
- rte_flow_error_set(error, ENOBUFS,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Fdir space full");
-
- return flow;
- }
- } else {
- flow = rte_zmalloc("i40e_flow", sizeof(struct rte_flow), 0);
- if (!flow) {
- rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to allocate memory");
- return flow;
- }
+ flow = rte_zmalloc("i40e_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return flow;
}
switch (filter_ctx.type) {
- case RTE_ETH_FILTER_FDIR: {
- struct i40e_fdir_filter *node;
-
- ret = i40e_fdir_filter_validate(dev, &filter_ctx.fdir_filter);
- if (ret)
- goto free_flow;
- ret = i40e_fdir_filter_register(dev, &filter_ctx.fdir_filter,
- &node);
- if (ret)
- goto free_flow;
- ret = i40e_fdir_filter_program(dev, node, 1,
- i40e_fdir_filter_needs_status_wait(pf,
- fdir_info->fdir_actual_cnt - 1));
- if (ret) {
- i40e_fdir_filter_unregister(dev, node);
- goto free_flow;
- }
- flow->rule = node;
- break;
- }
case RTE_ETH_FILTER_TUNNEL:
ret = i40e_dev_consistent_tunnel_filter_set(pf,
&filter_ctx.consistent_tunnel_filter, 1);
@@ -3823,10 +1976,7 @@ i40e_flow_create(struct rte_eth_dev *dev,
RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
"Failed to create flow.");
- if (filter_ctx.type != RTE_ETH_FILTER_FDIR)
- rte_free(flow);
- else
- i40e_fdir_entry_pool_put(fdir_info, flow);
+ rte_free(flow);
return NULL;
}
@@ -3838,7 +1988,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
enum rte_filter_type filter_type = flow->filter_type;
- struct i40e_fdir_info *fdir_info = &pf->fdir;
int ret = 0;
/* try the new engine first */
@@ -3851,15 +2000,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
ret = i40e_flow_destroy_tunnel_filter(pf,
(struct i40e_tunnel_filter *)flow->rule);
break;
- case RTE_ETH_FILTER_FDIR: {
- struct i40e_fdir_filter *node = flow->rule;
-
- ret = i40e_fdir_filter_program(dev, node, 0, false);
- if (ret)
- break;
- ret = i40e_fdir_filter_unregister(dev, node);
- break;
- }
case RTE_ETH_FILTER_HASH:
ret = i40e_hash_filter_destroy(pf, flow->rule);
break;
@@ -3872,10 +2012,7 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
if (!ret) {
TAILQ_REMOVE(&pf->flow_list, flow, node);
- if (filter_type == RTE_ETH_FILTER_FDIR)
- i40e_fdir_entry_pool_put(fdir_info, flow);
- else
- rte_free(flow);
+ rte_free(flow);
} else
rte_flow_error_set(error, -ret,
@@ -3955,14 +2092,6 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
if (ret != 0)
return ret;
- ret = i40e_flow_flush_fdir_filter(pf);
- if (ret) {
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to flush FDIR flows.");
- return -rte_errno;
- }
-
ret = i40e_flow_flush_tunnel_filter(pf);
if (ret) {
rte_flow_error_set(error, -ret,
@@ -3979,31 +2108,6 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
return ret;
}
-static int
-i40e_flow_flush_fdir_filter(struct i40e_pf *pf)
-{
- struct rte_eth_dev *dev = &rte_eth_devices[pf->dev_data->port_id];
- struct i40e_fdir_info *fdir_info = &pf->fdir;
- struct rte_flow *flow;
- void *temp;
- int ret;
-
- RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, temp) {
- if (flow->filter_type != RTE_ETH_FILTER_FDIR)
- continue;
- ret = i40e_fdir_filter_program(dev, flow->rule, false, false);
- if (ret < 0)
- return ret;
- ret = i40e_fdir_filter_unregister(dev, flow->rule);
- if (ret < 0)
- return ret;
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- i40e_fdir_entry_pool_put(fdir_info, flow);
- }
-
- return 0;
-}
-
/* Flush all tunnel filters */
static int
i40e_flow_flush_tunnel_filter(struct i40e_pf *pf)
diff --git a/drivers/net/intel/i40e/i40e_flow.h b/drivers/net/intel/i40e/i40e_flow.h
index 11d13a76fe..6823dbef33 100644
--- a/drivers/net/intel/i40e/i40e_flow.h
+++ b/drivers/net/intel/i40e/i40e_flow.h
@@ -8,9 +8,14 @@
#include "../common/flow_engine.h"
int i40e_get_outer_vlan(struct i40e_pf *pf, uint16_t *tpid);
+uint8_t
+i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
+ enum rte_flow_item_type item_type,
+ struct i40e_fdir_filter *filter);
extern const struct ci_flow_engine_list i40e_flow_engine_list;
extern const struct ci_flow_engine i40e_flow_engine_ethertype;
+extern const struct ci_flow_engine i40e_flow_engine_fdir;
#endif /* _I40E_FLOW_H_ */
diff --git a/drivers/net/intel/i40e/i40e_flow_fdir.c b/drivers/net/intel/i40e/i40e_flow_fdir.c
new file mode 100644
index 0000000000..1bd92fd31a
--- /dev/null
+++ b/drivers/net/intel/i40e/i40e_flow_fdir.c
@@ -0,0 +1,2056 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include "i40e_ethdev.h"
+#include "i40e_flow.h"
+
+#include <rte_bitmap.h>
+#include <rte_malloc.h>
+#include <rte_hash_crc.h>
+
+#include "../common/flow_engine.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+
+struct i40e_fdir_ctx {
+ struct ci_flow_engine_ctx base;
+ struct i40e_fdir_filter fdir_filter;
+ enum rte_flow_item_type custom_pctype;
+ struct flex_item {
+ size_t size;
+ size_t offset;
+ } flex_data[I40E_MAX_FLXPLD_FIED];
+};
+
+struct i40e_flow_engine_fdir_flow {
+ struct rte_flow base;
+ struct i40e_fdir_filter fdir_filter;
+ bool installed;
+};
+
+struct i40e_fdir_flow_pool_entry {
+ struct i40e_flow_engine_fdir_flow flow;
+ uint32_t idx;
+};
+
+struct i40e_fdir_engine_priv {
+ struct i40e_fdir_flow_pool_entry *pool;
+ struct rte_bitmap *bitmap;
+ struct rte_hash *hash_table;
+};
+
+#define I40E_FDIR_FLOW_ENTRY(flow_ptr) \
+ container_of((flow_ptr), struct i40e_fdir_flow_pool_entry, flow)
+
+/**
+ * FDIR graph implementation (non-tunnel)
+ * Pattern: START -> ETH -> [VLAN] -> (IPv4 | IPv6) -> [TCP | UDP | SCTP | ESP | L2TPv3 | GTP] -> END
+ * With RAW flexible payload support:
+ * - L2: ETH/VLAN -> RAW -> RAW -> RAW -> END
+ * - L3: IPv4/IPv6 -> RAW -> RAW -> RAW -> END
+ * - L4: TCP/UDP/SCTP -> RAW -> RAW -> RAW -> END
+ * GTP tunnel support:
+ * - IPv4/IPv6 -> UDP -> GTP -> END (GTP-C, GTP-U outer)
+ * - IPv4/IPv6 -> UDP -> GTP -> IPv4/IPv6 -> END (GTP-U with inner IP)
+ */
+
+enum i40e_fdir_node_id {
+ I40E_FDIR_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_FDIR_NODE_ETH,
+ I40E_FDIR_NODE_VLAN,
+ I40E_FDIR_NODE_IPV4,
+ I40E_FDIR_NODE_IPV6,
+ I40E_FDIR_NODE_TCP,
+ I40E_FDIR_NODE_UDP,
+ I40E_FDIR_NODE_SCTP,
+ I40E_FDIR_NODE_ESP,
+ I40E_FDIR_NODE_L2TPV3OIP,
+ I40E_FDIR_NODE_GTPC,
+ I40E_FDIR_NODE_GTPU,
+ I40E_FDIR_NODE_INNER_IPV4,
+ I40E_FDIR_NODE_INNER_IPV6,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ I40E_FDIR_NODE_MAX,
+};
+
+static int
+i40e_fdir_node_eth_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ bool no_src_mac, no_dst_mac, src_mac, dst_mac;
+
+ /* may be empty */
+ if (eth_spec == NULL && eth_mask == NULL)
+ return 0;
+
+ /* source and destination masks may be all zero or all one */
+ no_src_mac = CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr);
+ no_dst_mac = CI_FIELD_IS_ZERO(ð_mask->hdr.dst_addr);
+ src_mac = CI_FIELD_IS_MASKED(ð_mask->hdr.src_addr);
+ dst_mac = CI_FIELD_IS_MASKED(ð_mask->hdr.dst_addr);
+
+ /* can't be all zero */
+ if (no_src_mac && no_dst_mac) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item, "Invalid eth mask");
+ }
+ /* can't be neither zero nor ones */
+ if ((!no_src_mac && !src_mac) ||
+ (!no_dst_mac && !dst_mac)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item, "Invalid eth mask");
+ }
+
+ /* ethertype can either be unmasked or fully masked */
+ if (CI_FIELD_IS_ZERO(ð_mask->hdr.ether_type))
+ return 0;
+
+ if (!CI_FIELD_IS_MASKED(ð_mask->hdr.ether_type)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item, "Invalid ethertype mask");
+ }
+
+ /* Check for valid ethertype (not IPv4/IPv6) */
+ uint16_t ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+ if (ether_type == RTE_ETHER_TYPE_IPV4 ||
+ ether_type == RTE_ETHER_TYPE_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4/IPv6 not supported by ethertype filter");
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_eth_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ uint16_t tpid, ether_type;
+ uint64_t input_set = 0;
+ int ret;
+
+ /* Set layer index for L2 flexible payload (after ETH/VLAN) */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L2_IDX;
+
+ /* set packet type */
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_L2_PAYLOAD;
+
+ /* do we need to set up MAC addresses? */
+ if (eth_spec == NULL && eth_mask == NULL)
+ return 0;
+
+ /* do we care for source address? */
+ if (CI_FIELD_IS_MASKED(ð_mask->hdr.src_addr)) {
+ fdir_filter->input.flow.l2_flow.src = eth_spec->hdr.src_addr;
+ input_set |= I40E_INSET_SMAC;
+ }
+ /* do we care for destination address? */
+ if (CI_FIELD_IS_MASKED(ð_mask->hdr.dst_addr)) {
+ fdir_filter->input.flow.l2_flow.dst = eth_spec->hdr.dst_addr;
+ input_set |= I40E_INSET_DMAC;
+ }
+
+ /* do we care for ethertype? */
+ if (eth_mask->hdr.ether_type) {
+ struct i40e_pf *pf =
+ I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+
+ ether_type = rte_be_to_cpu_16(eth_spec->hdr.ether_type);
+ ret = i40e_get_outer_vlan(pf, &tpid);
+ if (ret != 0) {
+ return rte_flow_error_set(error, EIO,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Can not get the Ethertype identifying the L2 tag");
+ }
+ if (ether_type == tpid) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported ether_type in control packet filter.");
+ }
+ fdir_filter->input.flow.l2_flow.ether_type = eth_spec->hdr.ether_type;
+ input_set |= I40E_INSET_LAST_ETHER_TYPE;
+ }
+
+ fdir_filter->input.flow_ext.input_set = input_set;
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_vlan_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ uint16_t ether_type;
+
+ if (vlan_spec == NULL && vlan_mask == NULL)
+ return 0;
+
+ /* TCI mask can be either fully disabled or fully enabled. */
+ if (vlan_mask->hdr.vlan_tci != 0 &&
+ vlan_mask->hdr.vlan_tci != rte_cpu_to_be_16(I40E_VLAN_TCI_MASK)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported TCI mask");
+ }
+ if (CI_FIELD_IS_ZERO(&vlan_mask->hdr.eth_proto))
+ return 0;
+
+ if (!CI_FIELD_IS_MASKED(&vlan_mask->hdr.eth_proto)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid VLAN header mask");
+ }
+
+ /* can't match on eth_proto as we're already matching on ethertype */
+ if (fdir_filter->input.flow_ext.input_set & I40E_INSET_LAST_ETHER_TYPE) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Cannot set two ethertype filters");
+ }
+
+ ether_type = rte_be_to_cpu_16(vlan_spec->hdr.eth_proto);
+ if (ether_type == RTE_ETHER_TYPE_IPV4 ||
+ ether_type == RTE_ETHER_TYPE_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4/IPv6 not supported by VLAN protocol filter");
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_vlan_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+
+ /* Set layer index for L2 flexible payload (after ETH/VLAN) */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L2_IDX;
+
+ /* set packet type */
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_L2_PAYLOAD;
+
+ if (vlan_spec == NULL && vlan_mask == NULL)
+ return 0;
+
+ /* Store TCI value if requested */
+ if (vlan_mask->hdr.vlan_tci) {
+ fdir_filter->input.flow_ext.vlan_tci = vlan_spec->hdr.vlan_tci;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_VLAN_INNER;
+ }
+
+ /* if ethertype specified, store it */
+ if (vlan_mask->hdr.eth_proto) {
+ struct i40e_pf *pf =
+ I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+ uint16_t tpid, ether_type;
+ int ret;
+
+ ether_type = rte_be_to_cpu_16(vlan_spec->hdr.eth_proto);
+
+ ret = i40e_get_outer_vlan(pf, &tpid);
+ if (ret != 0) {
+ return rte_flow_error_set(error, EIO,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Can not get the Ethertype identifying the L2 tag");
+ }
+ if (ether_type == tpid) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported ether_type in control packet filter.");
+ }
+ fdir_filter->input.flow.l2_flow.ether_type = vlan_spec->hdr.eth_proto;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_LAST_ETHER_TYPE;
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_ipv4_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv4 *ipv4_spec = item->spec;
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+ const struct rte_flow_item_ipv4 *ipv4_last = item->last;
+
+ if (ipv4_mask == NULL && ipv4_spec == NULL)
+ return 0;
+
+ /* Validate mask fields */
+ if (ipv4_mask->hdr.version_ihl ||
+ ipv4_mask->hdr.total_length ||
+ ipv4_mask->hdr.packet_id ||
+ ipv4_mask->hdr.hdr_checksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 header mask");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.src_addr) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.dst_addr) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.type_of_service) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.time_to_live) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv4_mask->hdr.next_proto_id)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 header mask");
+ }
+
+ if (ipv4_last == NULL)
+ return 0;
+
+ /* Only fragment_offset supports range */
+ if (ipv4_last->hdr.version_ihl ||
+ ipv4_last->hdr.type_of_service ||
+ ipv4_last->hdr.total_length ||
+ ipv4_last->hdr.packet_id ||
+ ipv4_last->hdr.time_to_live ||
+ ipv4_last->hdr.next_proto_id ||
+ ipv4_last->hdr.hdr_checksum ||
+ ipv4_last->hdr.src_addr ||
+ ipv4_last->hdr.dst_addr) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4 range only supported for fragment_offset");
+ }
+
+ /* Validate fragment_offset range values */
+ uint16_t frag_mask = rte_be_to_cpu_16(ipv4_mask->hdr.fragment_offset);
+ uint16_t frag_spec = rte_be_to_cpu_16(ipv4_spec->hdr.fragment_offset);
+ uint16_t frag_last = rte_be_to_cpu_16(ipv4_last->hdr.fragment_offset);
+
+ /* Mask must be 0x3fff (fragment offset + MF flag) */
+ if (frag_mask != (RTE_IPV4_HDR_OFFSET_MASK | RTE_IPV4_HDR_MF_FLAG)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 fragment_offset mask");
+ }
+
+ /* Only allow: frag rule (spec=0x8, last=0x2000) or non-frag (spec=0, last=0) */
+ if (frag_spec == (1 << RTE_IPV4_HDR_FO_SHIFT) &&
+ frag_last == RTE_IPV4_HDR_MF_FLAG)
+ return 0; /* Fragment rule */
+
+ if (frag_spec == 0 && frag_last == 0)
+ return 0; /* Non-fragment rule */
+
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv4 fragment_offset rule");
+}
+
+static int
+i40e_fdir_node_ipv4_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_ipv4 *ipv4_spec = item->spec;
+ const struct rte_flow_item_ipv4 *ipv4_mask = item->mask;
+ const struct rte_flow_item_ipv4 *ipv4_last = item->last;
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ uint16_t frag_spec, frag_last;
+
+ /* Set layer index for L2 flexible payload (after ETH/VLAN) */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L3_IDX;
+
+ /* set packet type */
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV4_OTHER;
+
+ /* set up flow type */
+ fdir_filter->input.flow_ext.inner_ip = false;
+ fdir_filter->input.flow_ext.oip_type = I40E_FDIR_IPTYPE_IPV4;
+
+ if (ipv4_mask == NULL && ipv4_spec == NULL)
+ return 0;
+
+ /* Mark that IPv4 fields are used */
+ if (!CI_FIELD_IS_ZERO(&ipv4_mask->hdr.next_proto_id)) {
+ fdir_filter->input.flow.ip4_flow.proto = ipv4_spec->hdr.next_proto_id;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV4_PROTO;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv4_mask->hdr.type_of_service)) {
+ fdir_filter->input.flow.ip4_flow.tos = ipv4_spec->hdr.type_of_service;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV4_TOS;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv4_mask->hdr.time_to_live)) {
+ fdir_filter->input.flow.ip4_flow.ttl = ipv4_spec->hdr.time_to_live;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV4_TTL;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv4_mask->hdr.src_addr)) {
+ fdir_filter->input.flow.ip4_flow.src_ip = ipv4_spec->hdr.src_addr;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV4_SRC;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv4_mask->hdr.dst_addr)) {
+ fdir_filter->input.flow.ip4_flow.dst_ip = ipv4_spec->hdr.dst_addr;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV4_DST;
+ }
+
+ /* do we have range? */
+ if (ipv4_last == NULL)
+ return 0;
+
+ /* frag mask is already known to be non-zero */
+ frag_spec = rte_be_to_cpu_16(ipv4_spec->hdr.fragment_offset);
+ frag_last = rte_be_to_cpu_16(ipv4_last->hdr.fragment_offset);
+ /* frag spec and last are already known to be either 0 or valid */
+
+ /* is range specified for fragment_offset? */
+ if (frag_spec != 0 && frag_last != 0)
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_FRAG_IPV4;
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_ipv6_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_ipv6 *ipv6_spec = item->spec;
+ const struct rte_flow_item_ipv6 *ipv6_mask = item->mask;
+ if (ipv6_mask == NULL && ipv6_spec == NULL)
+ return 0;
+
+ /* payload len isn't supported */
+ if (ipv6_mask->hdr.payload_len) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 header mask");
+ }
+ /* source and destination mask can either be all zeroes or all ones */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 source address mask");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 destination address mask");
+ }
+
+ /* check other supported fields */
+ if (!ci_is_zero_or_masked(ipv6_mask->hdr.vtc_flow, rte_cpu_to_be_32(I40E_IPV6_TC_MASK)) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.proto) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&ipv6_mask->hdr.hop_limits)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid IPv6 header mask");
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_ipv6_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_ipv6 *ipv6_spec = item->spec;
+ const struct rte_flow_item_ipv6 *ipv6_mask = item->mask;
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+
+ /* Set layer index for L2 flexible payload (after ETH/VLAN) */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L3_IDX;
+
+ /* set packet type */
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV6_OTHER;
+
+ /* set up flow type */
+ fdir_filter->input.flow_ext.inner_ip = false;
+ fdir_filter->input.flow_ext.oip_type = I40E_FDIR_IPTYPE_IPV6;
+
+ if (ipv6_mask == NULL && ipv6_spec == NULL)
+ return 0;
+ if (CI_FIELD_IS_MASKED(&ipv6_mask->hdr.src_addr)) {
+ memcpy(&fdir_filter->input.flow.ipv6_flow.src_ip, &ipv6_spec->hdr.src_addr, sizeof(ipv6_spec->hdr.src_addr));
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV6_SRC;
+ }
+ if (CI_FIELD_IS_MASKED(&ipv6_mask->hdr.dst_addr)) {
+ memcpy(&fdir_filter->input.flow.ipv6_flow.dst_ip, &ipv6_spec->hdr.dst_addr, sizeof(ipv6_spec->hdr.dst_addr));
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV6_DST;
+ }
+
+ if (!CI_FIELD_IS_ZERO(&ipv6_mask->hdr.vtc_flow)) {
+ rte_be32_t vtc_flow = rte_be_to_cpu_32(ipv6_spec->hdr.vtc_flow);
+ uint8_t tc = (uint8_t)((vtc_flow & I40E_IPV6_TC_MASK) >> I40E_FDIR_IPv6_TC_OFFSET);
+ fdir_filter->input.flow.ipv6_flow.tc = tc;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV6_TC;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv6_mask->hdr.proto)) {
+ fdir_filter->input.flow.ipv6_flow.proto = ipv6_spec->hdr.proto;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV6_NEXT_HDR;
+ }
+ if (!CI_FIELD_IS_ZERO(&ipv6_mask->hdr.hop_limits)) {
+ fdir_filter->input.flow.ipv6_flow.hop_limits = ipv6_spec->hdr.hop_limits;
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_IPV6_HOP_LIMIT;
+ }
+ /* mark as fragment traffic if necessary */
+ if (ipv6_spec->hdr.proto == I40E_IPV6_FRAG_HEADER)
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_FRAG_IPV6;
+
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_tcp_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_tcp *tcp_spec = item->spec;
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+
+ /* cannot match both fragmented and TCP */
+ if (fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV4 ||
+ fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Cannot combine fragmented IP and TCP match");
+ }
+
+ if (tcp_spec == NULL && tcp_mask == NULL)
+ return 0;
+
+ if (tcp_mask->hdr.sent_seq ||
+ tcp_mask->hdr.recv_ack ||
+ tcp_mask->hdr.data_off ||
+ tcp_mask->hdr.tcp_flags ||
+ tcp_mask->hdr.rx_win ||
+ tcp_mask->hdr.cksum ||
+ tcp_mask->hdr.tcp_urp) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid TCP header mask");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid TCP header mask");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_node_tcp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_tcp *tcp_spec = item->spec;
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+ rte_be16_t src_spec, dst_spec, src_mask, dst_mask;
+ bool is_ipv4;
+
+ /* Set layer index for L4 flexible payload */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L4_IDX;
+
+ /* set packet type depending on L3 type */
+ if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV4_TCP;
+ } else if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV6) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV6_TCP;
+ }
+
+ if (tcp_spec == NULL && tcp_mask == NULL)
+ return 0;
+
+ src_spec = tcp_spec->hdr.src_port;
+ dst_spec = tcp_spec->hdr.dst_port;
+ src_mask = tcp_mask->hdr.src_port;
+ dst_mask = tcp_mask->hdr.dst_port;
+ is_ipv4 = fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4;
+
+ if (is_ipv4) {
+ if (src_mask != 0) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.tcp4_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.tcp4_flow.dst_port = dst_spec;
+ }
+ } else {
+ if (src_mask != 0) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.tcp6_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.tcp6_flow.dst_port = dst_spec;
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_udp_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_udp *udp_spec = item->spec;
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+
+ /* cannot match both fragmented and TCP */
+ if (fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV4 ||
+ fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Cannot combine fragmented IP and UDP match");
+ }
+
+ if (udp_spec == NULL && udp_mask == NULL)
+ return 0;
+
+ if (udp_mask->hdr.dgram_len || udp_mask->hdr.dgram_cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid UDP header mask");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid UDP header mask");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_node_udp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_udp *udp_spec = item->spec;
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+ rte_be16_t src_spec, dst_spec, src_mask, dst_mask;
+ bool is_ipv4;
+
+ /* Set layer index for L4 flexible payload */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L4_IDX;
+
+ /* set packet type depending on L3 type */
+ if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV4_UDP;
+ } else if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV6) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV6_UDP;
+ }
+
+ /* set UDP */
+ fdir_filter->input.flow_ext.is_udp = true;
+
+ if (udp_spec == NULL && udp_mask == NULL)
+ return 0;
+
+ src_spec = udp_spec->hdr.src_port;
+ dst_spec = udp_spec->hdr.dst_port;
+ src_mask = udp_mask->hdr.src_port;
+ dst_mask = udp_mask->hdr.dst_port;
+ is_ipv4 = fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4;
+
+ if (is_ipv4) {
+ if (src_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.udp4_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.udp4_flow.dst_port = dst_spec;
+ }
+ } else {
+ if (src_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.udp6_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.udp6_flow.dst_port = dst_spec;
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_sctp_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_sctp *sctp_spec = item->spec;
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+
+ /* cannot match both fragmented and TCP */
+ if (fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV4 ||
+ fdir_filter->input.pctype == I40E_FILTER_PCTYPE_FRAG_IPV6) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Cannot combine fragmented IP and SCTP match");
+ }
+
+ if (sctp_spec == NULL && sctp_mask == NULL)
+ return 0;
+
+ if (sctp_mask->hdr.cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid SCTP header mask");
+ }
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.dst_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.tag)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid SCTP header mask");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_node_sctp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_sctp *sctp_spec = item->spec;
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+ rte_be16_t src_spec, dst_spec, src_mask, dst_mask, tag_spec, tag_mask;
+ bool is_ipv4;
+
+ /* Set layer index for L4 flexible payload */
+ fdir_filter->input.flow_ext.layer_idx = I40E_FLXPLD_L4_IDX;
+
+ /* set packet type depending on L3 type */
+ if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV4_SCTP;
+ } else if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV6) {
+ fdir_filter->input.pctype = I40E_FILTER_PCTYPE_NONF_IPV6_SCTP;
+ }
+
+ if (sctp_spec == NULL && sctp_mask == NULL)
+ return 0;
+
+ if (!CI_FIELD_IS_ZERO(&sctp_mask->hdr.tag)) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SCTP_VT;
+ }
+
+ src_spec = sctp_spec->hdr.src_port;
+ dst_spec = sctp_spec->hdr.dst_port;
+ src_mask = sctp_mask->hdr.src_port;
+ dst_mask = sctp_mask->hdr.dst_port;
+ tag_spec = sctp_spec->hdr.tag;
+ tag_mask = sctp_mask->hdr.tag;
+ is_ipv4 = fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4;
+
+ if (is_ipv4) {
+ if (src_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.sctp4_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.sctp4_flow.dst_port = dst_spec;
+ }
+ if (tag_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SCTP_VT;
+ fdir_filter->input.flow.sctp4_flow.verify_tag = tag_spec;
+ }
+ } else {
+ if (src_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SRC_PORT;
+ fdir_filter->input.flow.sctp6_flow.src_port = src_spec;
+ }
+ if (dst_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_DST_PORT;
+ fdir_filter->input.flow.sctp6_flow.dst_port = dst_spec;
+ }
+ if (tag_mask) {
+ fdir_filter->input.flow_ext.input_set |= I40E_INSET_SCTP_VT;
+ fdir_filter->input.flow.sctp6_flow.verify_tag = tag_spec;
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_raw_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+ const struct rte_flow_item_raw *raw_spec = item->spec;
+ const struct rte_flow_item_raw *raw_mask = item->mask;
+ enum i40e_flxpld_layer_idx raw_id = fdir_filter->input.flow_ext.raw_id;
+ size_t spec_size, spec_offset;
+ size_t total_size, i;
+ size_t new_src_offset;
+
+ /* we shouldn't write to global registers on some hardware */
+ if (pf->support_multi_driver) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported flexible payload.");
+ }
+
+ /* Check max RAW items limit */
+ RTE_BUILD_BUG_ON(I40E_MAX_FLXPLD_LAYER != I40E_MAX_FLXPLD_FIED);
+ if (raw_id >= I40E_MAX_FLXPLD_LAYER) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Maximum 3 RAW items allowed per layer");
+ }
+
+ if (raw_spec->pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW spec pattern must not be NULL");
+ }
+
+ if (raw_mask->pattern == NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW mask pattern must not be NULL");
+ }
+
+ if (raw_mask->length != raw_spec->length &&
+ raw_mask->length != 0xffff) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW mask length must match spec length or be 0xffff");
+ }
+
+ if (raw_mask->relative || raw_mask->search ||
+ raw_mask->reserved || raw_mask->offset || raw_mask->limit) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW mask control fields are not supported");
+ }
+
+ /* Relative offset is mandatory */
+ if (!raw_spec->relative) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW relative must be 1");
+ }
+
+ /* Offset must be 16-bit aligned */
+ if (raw_spec->offset % sizeof(uint16_t)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW offset must be even");
+ }
+
+ /* Search and limit not supported */
+ if (raw_spec->search || raw_spec->limit) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW search/limit not supported");
+ }
+
+ if (raw_spec->reserved) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW reserved field must be zero");
+ }
+
+ /* Offset must be non-negative */
+ if (raw_spec->offset < 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW offset must be non-negative");
+ }
+
+ /* flex size/offset for current item (in bytes) */
+ spec_size = raw_spec->length;
+ spec_offset = raw_spec->offset;
+
+ /*
+ * RAW node can be triggered multiple times, each time we will be copying more data to the
+ * flexbyte buffer. we need to validate total size/offset against max allowed because we
+ * cannot overflow our flexbyte buffer.
+ */
+
+ /* accumulate previous raw items' size/offset */
+ total_size = 0;
+ new_src_offset = 0;
+ for (i = 0; i < raw_id; i++) {
+ const struct flex_item *fi = &fdir_ctx->flex_data[i];
+ total_size += fi->size;
+ /* offset is relative to end of previous item */
+ new_src_offset += fi->offset + fi->size;
+ }
+ /* add current item to totals */
+ total_size += spec_size;
+ new_src_offset += spec_offset;
+
+ /* validate against max offset/size */
+ if (spec_size + new_src_offset >= I40E_MAX_FLX_SOURCE_OFF) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW total offset exceeds maximum");
+ }
+ if (total_size > I40E_FDIR_MAX_FLEXLEN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "RAW total size exceeds maximum");
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_raw_process(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_raw *raw_spec = item->spec;
+ const struct rte_flow_item_raw *raw_mask = item->mask;
+ enum i40e_flxpld_layer_idx raw_id = fdir_filter->input.flow_ext.raw_id;
+ enum i40e_flxpld_layer_idx layer_idx = fdir_filter->input.flow_ext.layer_idx;
+ size_t flex_pit_field_idx = layer_idx * I40E_MAX_FLXPLD_FIED + raw_id;
+ struct i40e_fdir_flex_pit *flex_pit;
+ size_t spec_size, spec_offset, i;
+ size_t total_size, new_src_offset;
+
+ /* flex size for current item */
+ spec_size = raw_spec->length;
+ spec_offset = raw_spec->offset;
+
+ /* accumulate previous raw items' size/offset */
+ total_size = 0;
+ new_src_offset = 0;
+ for (i = 0; i < raw_id; i++) {
+ const struct flex_item *fi = &fdir_ctx->flex_data[i];
+ total_size += fi->size;
+ /* offset is relative to end of previous item */
+ new_src_offset += fi->offset + fi->size;
+ }
+ /* src offset must also include current offset */
+ new_src_offset += spec_offset;
+
+ /* store the current data */
+ fdir_ctx->flex_data[raw_id].size = spec_size;
+ fdir_ctx->flex_data[raw_id].offset = spec_offset;
+
+ /* copy bytes from current spec into the flex pit buffer */
+ for (i = 0; i < spec_size; i++) {
+ const size_t j = total_size + i;
+ fdir_filter->input.flow_ext.flexbytes[j] = raw_spec->pattern[i];
+ fdir_filter->input.flow_ext.flex_mask[j] = raw_mask->pattern[i];
+ }
+
+ /*
+ * all metadata in the flex pit is stored in units of 2 bytes (words),
+ * but all the limits are in bytes, so we need to convert sizes/offsets
+ * accordingly.
+ */
+
+ /* pick our flex pit */
+ flex_pit = &fdir_filter->input.flow_ext.flex_pit[flex_pit_field_idx];
+ /* convert to words (2-byte units) */
+ flex_pit->src_offset = (uint16_t)new_src_offset / sizeof(uint16_t);
+ flex_pit->dst_offset = (uint16_t)total_size / sizeof(uint16_t);
+ flex_pit->size = (uint16_t)spec_size / sizeof(uint16_t);
+
+ /* increment raw item index */
+ fdir_filter->input.flow_ext.raw_id++;
+
+ /* mark as flex flow */
+ fdir_filter->input.flow_ext.is_flex_flow = true;
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_esp_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+ const struct rte_flow_item_esp *esp_mask = item->mask;
+
+ if (!pf->esp_support) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Protocol not supported");
+ }
+
+ /* SPI must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&esp_mask->hdr.spi)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid ESP header mask");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_node_esp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_esp *esp_spec = item->spec;
+ bool is_ipv4 = fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4;
+ bool is_udp = fdir_filter->input.flow_ext.is_udp;
+
+ /* ESP uses customized pctype */
+ fdir_filter->input.flow_ext.customized_pctype = true;
+ fdir_ctx->custom_pctype = item->type;
+
+ if (is_ipv4) {
+ if (is_udp)
+ fdir_filter->input.flow.esp_ipv4_udp_flow.spi = esp_spec->hdr.spi;
+ else {
+ fdir_filter->input.flow.esp_ipv4_flow.spi = esp_spec->hdr.spi;
+ }
+ } else {
+ if (is_udp)
+ fdir_filter->input.flow.esp_ipv6_udp_flow.spi = esp_spec->hdr.spi;
+ else {
+ fdir_filter->input.flow.esp_ipv6_flow.spi = esp_spec->hdr.spi;
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_l2tpv3oip_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_l2tpv3oip *l2tp_mask = item->mask;
+
+ if (!CI_FIELD_IS_MASKED(&l2tp_mask->session_id)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid L2TPv3oIP header mask");
+ }
+ return 0;
+
+}
+
+static int
+i40e_fdir_node_l2tpv3oip_process(void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_l2tpv3oip *l2tp_spec = item->spec;
+
+ /* L2TPv3 uses customized pctype */
+ fdir_filter->input.flow_ext.customized_pctype = true;
+ fdir_ctx->custom_pctype = item->type;
+
+ /* Store session_id in appropriate flow union member based on IP version */
+ if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV4) {
+ fdir_filter->input.flow.ip4_l2tpv3oip_flow.session_id = l2tp_spec->session_id;
+ } else if (fdir_filter->input.flow_ext.oip_type == I40E_FDIR_IPTYPE_IPV6) {
+ fdir_filter->input.flow.ip6_l2tpv3oip_flow.session_id = l2tp_spec->session_id;
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_gtp_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+ const struct rte_flow_item_gtp *gtp_mask = item->mask;
+
+ /* DDP may not support this packet type */
+ if (!pf->gtp_support) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Protocol not supported");
+ }
+
+ if (gtp_mask->hdr.gtp_hdr_info ||
+ gtp_mask->hdr.msg_type ||
+ gtp_mask->hdr.plen) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid GTP header mask");
+ }
+ /* if GTP is specified, TEID must be masked */
+ if (!CI_FIELD_IS_MASKED(>p_mask->hdr.teid)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid GTP header mask");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_node_gtp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ const struct rte_flow_item_gtp *gtp_spec = item->spec;
+
+ /* Mark as GTP tunnel with customized pctype */
+ fdir_filter->input.flow_ext.customized_pctype = true;
+ fdir_ctx->custom_pctype = item->type;
+
+ fdir_filter->input.flow.gtp_flow.teid = gtp_spec->teid;
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_inner_ipv4_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+
+ /* Mark as inner IP */
+ fdir_filter->input.flow_ext.inner_ip = true;
+ fdir_filter->input.flow_ext.iip_type = I40E_FDIR_IPTYPE_IPV4;
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_inner_ipv6_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+
+ /* Mark as inner IP */
+ fdir_filter->input.flow_ext.inner_ip = true;
+ fdir_filter->input.flow_ext.iip_type = I40E_FDIR_IPTYPE_IPV6;
+
+ return 0;
+}
+
+/* END node validation for FDIR - performs pctype determination and input_set validation */
+static int
+i40e_fdir_node_end_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = ctx;
+ const struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ uint64_t input_set = fdir_filter->input.flow_ext.input_set;
+ enum i40e_filter_pctype pctype = fdir_filter->input.pctype;
+
+ /*
+ * Before sending the configuration down to hardware, we need to make
+ * sure that the configuration makes sense - more specifically, that the
+ * input set is a valid one that is actually supported by the hardware.
+ * This is validated for built-in ptypes, however for customized ptypes,
+ * the validation is skipped, and we have no way of validating the input
+ * set because we do not have that information at our disposal - the
+ * input set for customized packet type is not available through DDP
+ * queries.
+ *
+ * However, we do know that some things are unsupported by the hardware no matter the
+ * configuration. We can check for them here.
+ */
+ const uint64_t i40e_l2_input_set = I40E_INSET_DMAC | I40E_INSET_SMAC;
+ const uint64_t i40e_l3_input_set = (I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
+ I40E_INSET_IPV4_TOS | I40E_INSET_IPV4_TTL |
+ I40E_INSET_IPV4_PROTO);
+ const uint64_t i40e_l4_input_set = (I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT);
+ const bool l2_in_set = (input_set & i40e_l2_input_set) != 0;
+ const bool l3_in_set = (input_set & i40e_l3_input_set) != 0;
+ const bool l4_in_set = (input_set & i40e_l4_input_set) != 0;
+
+ /* if we're matching ethertype, we may be matching L2 only, and cannot have RAW patterns */
+ if ((input_set & I40E_INSET_LAST_ETHER_TYPE) != 0 &&
+ (pctype != I40E_FILTER_PCTYPE_L2_PAYLOAD ||
+ fdir_filter->input.flow_ext.is_flex_flow)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Cannot match ethertype with L3/L4 or RAW patterns");
+ }
+
+ /* L2 and L3 input sets are exclusive */
+ if (l2_in_set && l3_in_set) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Matching both L2 and L3 is not supported");
+ }
+ /* L2 and L4 input sets are exclusive */
+ if (l2_in_set && l4_in_set) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Matching both L2 and L4 is not supported");
+ }
+
+ /* if we are using one of the builtin packet types, validate it */
+ if (!fdir_filter->input.flow_ext.customized_pctype) {
+ /* validate the input set for the built-in pctype */
+ if (i40e_validate_input_set(pctype, RTE_ETH_FILTER_FDIR, input_set) != 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid input set");
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_node_end_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ struct i40e_fdir_ctx *fdir_ctx = ctx;
+ struct i40e_fdir_filter *fdir_filter = &fdir_ctx->fdir_filter;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(fdir_ctx->base.dev_data->dev_private);
+
+ /* Get customized pctype value */
+ if (fdir_filter->input.flow_ext.customized_pctype) {
+ enum i40e_filter_pctype pctype = i40e_flow_fdir_get_pctype_value(pf,
+ fdir_ctx->custom_pctype, fdir_filter);
+ if (pctype == I40E_FILTER_PCTYPE_INVALID) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported packet type");
+ }
+ /* update FDIR packet type */
+ fdir_filter->input.pctype = pctype;
+ }
+
+ return 0;
+}
+
+static const struct flow_graph i40e_fdir_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_FDIR_NODE_START] = { .name = "START" },
+ [I40E_FDIR_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_eth_validate,
+ .process = i40e_fdir_node_eth_process,
+ },
+ [I40E_FDIR_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_vlan_validate,
+ .process = i40e_fdir_node_vlan_process,
+ },
+ [I40E_FDIR_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK |
+ FLOW_GRAPH_NODE_EXPECT_RANGE,
+ .validate = i40e_fdir_node_ipv4_validate,
+ .process = i40e_fdir_node_ipv4_process,
+ },
+ [I40E_FDIR_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_ipv6_validate,
+ .process = i40e_fdir_node_ipv6_process,
+ },
+ [I40E_FDIR_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_tcp_validate,
+ .process = i40e_fdir_node_tcp_process,
+ },
+ [I40E_FDIR_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_udp_validate,
+ .process = i40e_fdir_node_udp_process,
+ },
+ [I40E_FDIR_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_sctp_validate,
+ .process = i40e_fdir_node_sctp_process,
+ },
+ [I40E_FDIR_NODE_ESP] = {
+ .name = "ESP",
+ .type = RTE_FLOW_ITEM_TYPE_ESP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_esp_validate,
+ .process = i40e_fdir_node_esp_process,
+ },
+ [I40E_FDIR_NODE_L2TPV3OIP] = {
+ .name = "L2TPV3OIP",
+ .type = RTE_FLOW_ITEM_TYPE_L2TPV3OIP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_l2tpv3oip_validate,
+ .process = i40e_fdir_node_l2tpv3oip_process,
+ },
+ [I40E_FDIR_NODE_GTPC] = {
+ .name = "GTPC",
+ .type = RTE_FLOW_ITEM_TYPE_GTPC,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_gtp_validate,
+ .process = i40e_fdir_node_gtp_process,
+ },
+ [I40E_FDIR_NODE_GTPU] = {
+ .name = "GTPU",
+ .type = RTE_FLOW_ITEM_TYPE_GTPU,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_gtp_validate,
+ .process = i40e_fdir_node_gtp_process,
+ },
+ [I40E_FDIR_NODE_INNER_IPV4] = {
+ .name = "INNER_IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .validate = i40e_fdir_node_ipv4_validate,
+ .process = i40e_fdir_node_inner_ipv4_process,
+ },
+ [I40E_FDIR_NODE_INNER_IPV6] = {
+ .name = "INNER_IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .validate = i40e_fdir_node_ipv6_validate,
+ .process = i40e_fdir_node_inner_ipv6_process,
+ },
+ [I40E_FDIR_NODE_RAW] = {
+ .name = "RAW",
+ .type = RTE_FLOW_ITEM_TYPE_RAW,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_fdir_node_raw_validate,
+ .process = i40e_fdir_node_raw_process,
+ },
+ [I40E_FDIR_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .validate = i40e_fdir_node_end_validate,
+ .process = i40e_fdir_node_end_process
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_FDIR_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_VLAN,
+ I40E_FDIR_NODE_IPV4,
+ I40E_FDIR_NODE_IPV6,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_VLAN] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_IPV4,
+ I40E_FDIR_NODE_IPV6,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_TCP,
+ I40E_FDIR_NODE_UDP,
+ I40E_FDIR_NODE_SCTP,
+ I40E_FDIR_NODE_ESP,
+ I40E_FDIR_NODE_L2TPV3OIP,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_TCP,
+ I40E_FDIR_NODE_UDP,
+ I40E_FDIR_NODE_SCTP,
+ I40E_FDIR_NODE_ESP,
+ I40E_FDIR_NODE_L2TPV3OIP,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_TCP] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_GTPC,
+ I40E_FDIR_NODE_GTPU,
+ I40E_FDIR_NODE_ESP,
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_SCTP] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_ESP] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_L2TPV3OIP] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_GTPC] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_GTPU] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_INNER_IPV4,
+ I40E_FDIR_NODE_INNER_IPV6,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_INNER_IPV4] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_INNER_IPV6] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_FDIR_NODE_RAW] = {
+ .next = (size_t[]) {
+ I40E_FDIR_NODE_RAW,
+ I40E_FDIR_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+i40e_fdir_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(param->driver_ctx);
+ const struct rte_flow_action *first, *second;
+
+ first = actions->actions[0];
+ /* can be NULL */
+ second = actions->actions[1];
+
+ switch (first->type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ {
+ const struct rte_flow_action_queue *act_q = first->conf;
+ /* check against PF constraints */
+ if (act_q->index >= pf->dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, first,
+ "Invalid queue ID for FDIR");
+ }
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ case RTE_FLOW_ACTION_TYPE_PASSTHRU:
+ case RTE_FLOW_ACTION_TYPE_MARK:
+ break;
+ default:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, first,
+ "Invalid first action for FDIR");
+ }
+
+ /* do we have another? */
+ if (second == NULL)
+ return 0;
+
+ switch (second->type) {
+ case RTE_FLOW_ACTION_TYPE_MARK:
+ {
+ /* only one mark action can be specified */
+ if (first->type == RTE_FLOW_ACTION_TYPE_MARK) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, second,
+ "Invalid second action for FDIR");
+ }
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_FLAG:
+ {
+ /* mark + flag is unsupported */
+ if (first->type == RTE_FLOW_ACTION_TYPE_MARK) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, second,
+ "Invalid second action for FDIR");
+ }
+ break;
+ }
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ /* RSS filter only can be after passthru or mark */
+ if (first->type != RTE_FLOW_ACTION_TYPE_PASSTHRU &&
+ first->type != RTE_FLOW_ACTION_TYPE_MARK) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, second,
+ "Invalid second action for FDIR");
+ }
+ break;
+ default:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, second,
+ "Invalid second action for FDIR");
+ }
+
+ return 0;
+}
+
+static int
+i40e_fdir_ctx_init(const struct rte_flow_action *actions,
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_adapter *adapter = I40E_DEV_PRIVATE_TO_ADAPTER(ctx->dev_data->dev_private);
+ struct i40e_fdir_ctx *fdir_ctx = (struct i40e_fdir_ctx *)ctx;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param ac_param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_DROP,
+ RTE_FLOW_ACTION_TYPE_PASSTHRU,
+ RTE_FLOW_ACTION_TYPE_MARK,
+ RTE_FLOW_ACTION_TYPE_FLAG,
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 2,
+ .driver_ctx = adapter,
+ .check = i40e_fdir_action_check,
+ };
+ int ret;
+ const struct rte_flow_action *first, *second;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret) {
+ return ret;
+ }
+
+ ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
+ if (ret) {
+ return ret;
+ }
+
+ first = parsed_actions.actions[0];
+ /* can be NULL */
+ second = parsed_actions.actions[1];
+
+ if (first->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ const struct rte_flow_action_queue *act_q = first->conf;
+ fdir_ctx->fdir_filter.action.rx_queue = act_q->index;
+ fdir_ctx->fdir_filter.action.behavior = I40E_FDIR_ACCEPT;
+ } else if (first->type == RTE_FLOW_ACTION_TYPE_DROP) {
+ fdir_ctx->fdir_filter.action.behavior = I40E_FDIR_REJECT;
+ } else if (first->type == RTE_FLOW_ACTION_TYPE_PASSTHRU) {
+ fdir_ctx->fdir_filter.action.behavior = I40E_FDIR_PASSTHRU;
+ } else if (first->type == RTE_FLOW_ACTION_TYPE_MARK) {
+ const struct rte_flow_action_mark *act_m = first->conf;
+ fdir_ctx->fdir_filter.action.behavior = I40E_FDIR_PASSTHRU;
+ fdir_ctx->fdir_filter.action.report_status = I40E_FDIR_REPORT_ID;
+ fdir_ctx->fdir_filter.soft_id = act_m->id;
+ }
+
+ if (second != NULL) {
+ if (second->type == RTE_FLOW_ACTION_TYPE_MARK) {
+ const struct rte_flow_action_mark *act_m = second->conf;
+ fdir_ctx->fdir_filter.action.report_status = I40E_FDIR_REPORT_ID;
+ fdir_ctx->fdir_filter.soft_id = act_m->id;
+ } else if (second->type == RTE_FLOW_ACTION_TYPE_FLAG) {
+ fdir_ctx->fdir_filter.action.report_status = I40E_FDIR_NO_REPORT_STATUS;
+ }
+ /* RSS action does nothing */
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct i40e_fdir_ctx *fdir_ctx = (const struct i40e_fdir_ctx *)ctx;
+ struct i40e_flow_engine_fdir_flow *fdir_flow = (struct i40e_flow_engine_fdir_flow *)flow;
+
+ fdir_flow->fdir_filter = fdir_ctx->fdir_filter;
+
+ return 0;
+}
+
+/* Add a flow director filter into the SW hash table */
+static int
+i40e_fdir_filter_hash_add(struct i40e_fdir_engine_priv *priv,
+ const struct i40e_fdir_input *filter_key)
+{
+ int ret;
+
+ /* try to find the filter in the hash table first */
+ ret = rte_hash_lookup(priv->hash_table, filter_key);
+ if (ret >= 0) {
+ PMD_DRV_LOG(ERR, "Filter already exists!");
+ return -EEXIST;
+ }
+
+ ret = rte_hash_add_key(priv->hash_table, filter_key);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to insert fdir filter: %s", rte_strerror(-ret));
+ return ret;
+ }
+
+ return 0;
+}
+
+/* Delete a flow director filter from the SW hash table */
+static int
+i40e_fdir_filter_hash_del(struct i40e_fdir_engine_priv *priv,
+ const struct i40e_fdir_input *filter_key)
+{
+ int ret;
+
+ ret = rte_hash_del_key(priv->hash_table, filter_key);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to delete fdir filter: %s", rte_strerror(-ret));
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * Records the filter and everything derived from it.
+ */
+static int
+i40e_fdir_filter_register(struct i40e_fdir_engine_priv *priv,
+ struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ struct rte_flow_error *error)
+{
+ struct rte_eth_dev_data *dev_data = pf->dev_data;
+ enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
+ uint64_t input_set = filter->input.flow_ext.input_set;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+ struct i40e_fdir_layer_state *layer = NULL;
+ struct i40e_fdir_flex_mask flex_mask = {0};
+ bool common_pctype, needs_teardown = false;
+ int ret;
+
+ ret = i40e_fdir_engine_init(pf);
+ if (ret < 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to initialize flow director engine");
+ }
+ /* if we are initializing, we tear it down */
+ needs_teardown = pf->fdir.fdir_actual_cnt == 0;
+
+ /* check input set if the packet type is common */
+ common_pctype = !filter->input.flow_ext.customized_pctype;
+
+ if (common_pctype) {
+ ret = i40e_fdir_inset_check(pf, pctype, input_set);
+ if (ret < 0) {
+ ret = rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Invalid input set");
+ goto err;
+ }
+ }
+
+ /* check if flex flow configuration is valid */
+ if (filter->input.flow_ext.is_flex_flow) {
+ ret = i40e_fdir_flex_check(pf, filter, pctype, &flex_mask);
+ if (ret < 0) {
+ ret = rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Invalid flex flow configuration");
+ goto err;
+ }
+ }
+
+ /* register the flow with the hash table */
+ ret = i40e_fdir_filter_hash_add(priv, &filter->input);
+ if (ret < 0) {
+ ret = rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to register fdir filter");
+ goto err;
+ }
+
+ /* save additional configuration */
+ if (common_pctype)
+ state->input_set = input_set;
+
+ if (filter->input.flow_ext.is_flex_flow) {
+ i40e_fdir_flex_store(pf, filter, pctype, &flex_mask);
+ layer = &flows->layer[filter->input.flow_ext.layer_idx];
+ layer->flex_flow_count++;
+ layer->flex_pit_flag = true;
+ state->flex_mask_flag = true;
+ }
+
+ state->flow_count++;
+ pf->fdir.fdir_actual_cnt++;
+
+ /* synchronize fdir processing queue flags */
+ i40e_fdir_rx_proc_sync(dev_data);
+
+ return 0;
+err:
+ if (needs_teardown)
+ i40e_fdir_teardown(pf);
+
+ return ret;
+}
+
+/**
+ * i40e_fdir_filter_unregister - drop software ownership of a filter.
+ */
+static int
+i40e_fdir_filter_unregister(struct i40e_fdir_engine_priv *priv,
+ struct i40e_pf *pf,
+ const struct i40e_fdir_filter *filter,
+ struct rte_flow_error *error)
+{
+ struct rte_eth_dev_data *dev_data = pf->dev_data;
+ enum i40e_filter_pctype pctype = i40e_fdir_filter_pctype(filter);
+ enum i40e_flxpld_layer_idx layer_idx = filter->input.flow_ext.layer_idx;
+ bool is_flex_flow = filter->input.flow_ext.is_flex_flow;
+ struct i40e_fdir_flow_store *flows = &pf->fdir.flows;
+ struct i40e_fdir_pctype_state *state = &flows->pctype[pctype];
+ struct i40e_fdir_layer_state *layer = &flows->layer[layer_idx];
+ int ret;
+
+ ret = i40e_fdir_filter_hash_del(priv, &filter->input);
+ if (ret < 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to unregister fdir filter");
+ }
+
+ if (is_flex_flow && --layer->flex_flow_count == 0)
+ layer->flex_pit_flag = false;
+
+ if (--state->flow_count == 0)
+ state->flex_mask_flag = false;
+
+ pf->fdir.fdir_actual_cnt--;
+
+ /* synchronize fdir processing queue flags */
+ i40e_fdir_rx_proc_sync(dev_data);
+
+ /* if there are no more FDIR flows, teardown FDIR */
+ if (pf->fdir.fdir_actual_cnt == 0)
+ i40e_fdir_teardown(pf);
+
+ return 0;
+}
+
+static int
+i40e_fdir_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_fdir_engine_priv *priv = flow->engine_priv;
+ struct i40e_flow_engine_fdir_flow *fdir_flow = (struct i40e_flow_engine_fdir_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+
+ /*
+ * fdir_space_size is a hardware capacity limit, not just the size of the
+ * allocator's SW tracking pool. If it's already reached, reject the flow
+ * here instead of letting the allocator's rte_zmalloc fallback paper
+ * over a request hardware has no room for.
+ */
+ if (pf->fdir.fdir_actual_cnt >= pf->fdir.fdir_space_size) {
+ return rte_flow_error_set(error, ENOSPC,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "FDIR filter space is full");
+ }
+
+ return i40e_fdir_filter_register(priv, pf, &fdir_flow->fdir_filter, error);
+}
+
+static int
+i40e_fdir_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_fdir_engine_priv *priv = flow->engine_priv;
+ struct i40e_flow_engine_fdir_flow *fdir_flow = (struct i40e_flow_engine_fdir_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+
+ return i40e_fdir_filter_unregister(priv, pf, &fdir_flow->fdir_filter, error);
+}
+
+static int
+i40e_fdir_flow_install(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_flow_engine_fdir_flow *fdir_flow = (struct i40e_flow_engine_fdir_flow *)flow;
+ struct rte_eth_dev_data *dev_data = flow->dev_data;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ bool needs_wait;
+ int ret;
+
+ /*
+ * when installing a flow, it may so happen that the flow is being
+ * allocated from shared fdir pool, so we need to wait for adminq to
+ * return status to know if the flow was actually installed.
+ *
+ * however, there are cases where the wait is not needed. one case is
+ * when we are re-installing the flow (HW will update existing filter so
+ * the flow is already there). another case is if we know that we are
+ * not using the shared pool.
+ */
+ needs_wait = fdir_flow->installed == false &&
+ i40e_fdir_filter_needs_status_wait(pf, pf->fdir.fdir_actual_cnt);
+
+ ret = i40e_fdir_filter_program(pf, &fdir_flow->fdir_filter, true, needs_wait);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to program fdir filter.");
+ }
+
+ /* flow is installed, don't wait next time */
+ fdir_flow->installed = true;
+
+ return 0;
+}
+
+static int
+i40e_fdir_flow_uninstall(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct rte_eth_dev_data *dev_data = flow->dev_data;
+ struct i40e_flow_engine_fdir_flow *fdir_flow = (struct i40e_flow_engine_fdir_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ int ret;
+
+ /* removal does not need wait */
+ ret = i40e_fdir_filter_program(pf, &fdir_flow->fdir_filter, false, false);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE,
+ flow, "Failed to deprogram fdir filter.");
+ }
+ return 0;
+}
+
+static int
+i40e_fdir_flow_engine_init(const struct ci_flow_engine *engine,
+ struct rte_eth_dev_data *dev_data,
+ void *priv_data)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
+ struct i40e_fdir_engine_priv *priv = priv_data;
+ char fdir_hash_name[RTE_HASH_NAMESIZE];
+ struct i40e_fdir_flow_pool_entry *pool;
+ struct rte_hash *hash_table;
+ struct rte_bitmap *bmp;
+ uint32_t bmp_size;
+ void *bmp_mem;
+ uint32_t i;
+ int ret;
+
+ snprintf(fdir_hash_name, sizeof(fdir_hash_name), "fdir_hash_%s", engine->name);
+
+ struct rte_hash_parameters fdir_hash_params = {
+ .name = fdir_hash_name,
+ .entries = I40E_MAX_FDIR_FILTER_NUM,
+ .key_len = sizeof(struct i40e_fdir_input),
+ .hash_func = rte_hash_crc,
+ .hash_func_init_val = 0,
+ .socket_id = rte_socket_id(),
+ };
+
+ hash_table = rte_hash_create(&fdir_hash_params);
+ if (hash_table == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to create fdir hash table: %s", rte_strerror(rte_errno));
+ ret = -rte_errno;
+ goto err_hash;
+ }
+
+ pool = rte_zmalloc(engine->name,
+ fdir_info->fdir_space_size * sizeof(*pool), 0);
+ if (pool == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate fdir pool");
+ ret = -ENOMEM;
+ goto err_pool;
+ }
+
+ bmp_size = rte_bitmap_get_memory_footprint(fdir_info->fdir_space_size);
+ bmp_mem = rte_zmalloc("fdir_bmap", bmp_size, RTE_CACHE_LINE_SIZE);
+ if (bmp_mem == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate fdir bitmap");
+ ret = -ENOMEM;
+ goto err_bitmap_mem;
+ }
+
+ bmp = rte_bitmap_init(fdir_info->fdir_space_size, bmp_mem, bmp_size);
+ if (bmp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to initialize fdir bitmap: %s", rte_strerror(rte_errno));
+ ret = -rte_errno;
+ goto err_bitmap;
+ }
+
+ for (i = 0; i < fdir_info->fdir_space_size; i++) {
+ pool[i].idx = i;
+ rte_bitmap_set(bmp, i);
+ }
+
+ priv->pool = pool;
+ priv->bitmap = bmp;
+ priv->hash_table = hash_table;
+
+ return 0;
+err_bitmap:
+ rte_free(bmp_mem);
+err_bitmap_mem:
+ rte_free(pool);
+err_pool:
+ rte_hash_free(hash_table);
+err_hash:
+ return ret;
+}
+
+static void
+i40e_fdir_flow_engine_uninit(const struct ci_flow_engine *engine __rte_unused,
+ void *priv_data)
+{
+ struct i40e_fdir_engine_priv *priv = priv_data;
+
+ rte_free(priv->bitmap);
+ rte_free(priv->pool);
+ rte_hash_free(priv->hash_table);
+}
+
+static struct ci_flow *
+i40e_fdir_flow_alloc(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv_data)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ struct i40e_fdir_info *fdir_info = &pf->fdir;
+ struct i40e_fdir_engine_priv *priv = priv_data;
+ struct i40e_flow_engine_fdir_flow *flow;
+ uint64_t slab = 0;
+ uint32_t pos = 0;
+ uint32_t bit;
+ size_t mem_sz;
+ int ret;
+
+ if (fdir_info->fdir_actual_cnt >= fdir_info->fdir_space_size)
+ return NULL;
+
+ ret = rte_bitmap_scan(priv->bitmap, &pos, &slab);
+ if (ret == 0)
+ return NULL;
+
+ bit = rte_bsf64(slab);
+ pos += bit;
+ rte_bitmap_clear(priv->bitmap, pos);
+
+ flow = &priv->pool[pos].flow;
+ /* do not touch ci_flow members, they are initialized by the caller */
+ mem_sz = sizeof(*flow) - sizeof(struct ci_flow);
+
+ memset(RTE_PTR_ADD(flow, sizeof(struct ci_flow)), 0, mem_sz);
+ return (struct ci_flow *)flow;
+}
+
+static void
+i40e_fdir_flow_free(struct ci_flow *flow,
+ struct rte_eth_dev_data *dev_data __rte_unused,
+ void *priv_data)
+{
+ struct i40e_fdir_engine_priv *priv = priv_data;
+ struct i40e_fdir_flow_pool_entry *entry;
+
+ entry = I40E_FDIR_FLOW_ENTRY((struct i40e_flow_engine_fdir_flow *)flow);
+ /* idx is not set at alloc, it's set at init */
+ rte_bitmap_set(priv->bitmap, entry->idx);
+}
+
+static const struct ci_flow_engine_ops i40e_flow_engine_fdir_ops = {
+ .engine_init = i40e_fdir_flow_engine_init,
+ .engine_uninit = i40e_fdir_flow_engine_uninit,
+ .flow_alloc = i40e_fdir_flow_alloc,
+ .flow_free = i40e_fdir_flow_free,
+ .ctx_init = i40e_fdir_ctx_init,
+ .ctx_to_flow = i40e_fdir_ctx_to_flow,
+ .flow_register = i40e_fdir_flow_register,
+ .flow_unregister = i40e_fdir_flow_unregister,
+ .flow_install = i40e_fdir_flow_install,
+ .flow_uninstall = i40e_fdir_flow_uninstall,
+};
+
+const struct ci_flow_engine i40e_flow_engine_fdir = {
+ .name = "fdir",
+ .ops = &i40e_flow_engine_fdir_ops,
+ .ctx_size = sizeof(struct i40e_fdir_ctx),
+ .flow_size = sizeof(struct i40e_flow_engine_fdir_flow),
+ .priv_size = sizeof(struct i40e_fdir_engine_priv),
+ .graph = &i40e_fdir_graph,
+};
diff --git a/drivers/net/intel/i40e/meson.build b/drivers/net/intel/i40e/meson.build
index ddc97f9b3b..c07257cb80 100644
--- a/drivers/net/intel/i40e/meson.build
+++ b/drivers/net/intel/i40e/meson.build
@@ -34,6 +34,7 @@ sources += files(
'i40e_fdir.c',
'i40e_flow.c',
'i40e_flow_ethertype.c',
+ 'i40e_flow_fdir.c',
'i40e_tm.c',
'i40e_hash.c',
'i40e_vf_representor.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 17/19] net/i40e: reimplement tunnel parsers
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (15 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 16/19] net/i40e: reimplement FDIR parser Anatoly Burakov
@ 2026-09-08 15:20 ` Anatoly Burakov
2026-09-08 15:21 ` [PATCH v2 18/19] net/i40e: reimplement hash parser Anatoly Burakov
` (2 subsequent siblings)
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:20 UTC (permalink / raw)
To: dev, Bruce Richardson
Use the new flow graph API and the common parsing framework to implement
flow parser for tunnel filters: QinQ, VXLAN, NVGRE, MPLS, GTP, and L4.
As a result of transitioning to more formalized validation, some
checks have become more stringent:
- VLAN TCI mask is now required to be fully masked (all-ones); previously
the mask was only checked for eth_proto and any non-zero vlan_tci mask
value was silently accepted
In addition to using the new graph infrastructure, some of the checks were
made more stringent and/or more correct. In particular:
- old code did not check for whether fields other than ports are masked
(they are now rejected)
- old code did not check for whether src/ports are fully masked (masks
other than full are now rejected)
- old code used spec to decide which port to copy (as a result, it was not
possible to match port 0 - this is now allowed)
Tunnel engine now also share a refcounted global state, and track all
flows and do deduplication inside the engine.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 548 +++----
drivers/net/intel/i40e/i40e_ethdev.h | 52 +-
drivers/net/intel/i40e/i40e_flow.c | 1702 +--------------------
drivers/net/intel/i40e/i40e_flow.h | 6 +
drivers/net/intel/i40e/i40e_flow_tunnel.c | 1590 +++++++++++++++++++
drivers/net/intel/i40e/meson.build | 1 +
6 files changed, 1829 insertions(+), 2070 deletions(-)
create mode 100644 drivers/net/intel/i40e/i40e_flow_tunnel.c
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 572dfff13a..97211b5994 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -392,14 +392,8 @@ static int i40e_set_default_mac_addr(struct rte_eth_dev *dev,
static int i40e_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
-static int i40e_tunnel_filter_convert(
- struct i40e_aqc_cloud_filters_element_bb *cld_filter,
- struct i40e_tunnel_filter *tunnel_filter);
-static int i40e_sw_tunnel_filter_insert(struct i40e_pf *pf,
- struct i40e_tunnel_filter *tunnel_filter);
static int i40e_cloud_filter_qinq_create(struct i40e_pf *pf);
-static void i40e_tunnel_filter_restore(struct i40e_pf *pf);
static void i40e_filter_restore(struct i40e_pf *pf);
static void i40e_notify_all_vfs_link_status(struct rte_eth_dev *dev);
static int i40e_fec_get_capability(struct rte_eth_dev *dev,
@@ -996,49 +990,41 @@ config_floating_veb(struct rte_eth_dev *dev)
#define I40E_L2_TAGS_S_TAG_SHIFT 1
#define I40E_L2_TAGS_S_TAG_MASK I40E_MASK(0x1, I40E_L2_TAGS_S_TAG_SHIFT)
-static int
-i40e_init_tunnel_filter_list(struct rte_eth_dev *dev)
+struct i40e_tunnel_state *
+i40e_tunnel_state_attach(struct rte_eth_dev_data *dev_data)
{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct i40e_tunnel_rule *tunnel_rule = &pf->tunnel;
- char tunnel_hash_name[RTE_HASH_NAMESIZE];
- int ret;
-
- struct rte_hash_parameters tunnel_hash_params = {
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ struct i40e_tunnel_state *state = &pf->tunnel_state;
+ char tunnel_hash_name[RTE_HASH_NAMESIZE] = {0};
+ struct rte_hash_parameters hash_params = {
.name = tunnel_hash_name,
.entries = I40E_MAX_TUNNEL_FILTER_NUM,
- .key_len = sizeof(struct i40e_tunnel_filter_input),
+ .key_len = sizeof(struct i40e_tunnel_filter_match_key),
.hash_func = rte_hash_crc,
.hash_func_init_val = 0,
.socket_id = rte_socket_id(),
};
- /* Initialize tunnel filter rule list and hash */
- TAILQ_INIT(&tunnel_rule->tunnel_list);
- snprintf(tunnel_hash_name, RTE_HASH_NAMESIZE,
- "tunnel_%s", dev->device->name);
- tunnel_rule->hash_table = rte_hash_create(&tunnel_hash_params);
- if (!tunnel_rule->hash_table) {
- PMD_INIT_LOG(ERR, "Failed to create tunnel hash table!");
- return -EINVAL;
- }
- tunnel_rule->hash_map = rte_zmalloc("i40e_tunnel_hash_map",
- sizeof(struct i40e_tunnel_filter *) *
- I40E_MAX_TUNNEL_FILTER_NUM,
- 0);
- if (!tunnel_rule->hash_map) {
- PMD_INIT_LOG(ERR,
- "Failed to allocate memory for tunnel hash map!");
- ret = -ENOMEM;
- goto err_tunnel_hash_map_alloc;
+ snprintf(tunnel_hash_name, RTE_HASH_NAMESIZE, "i40e_tunnel_hash_%d", dev_data->port_id);
+
+ if (state->refcnt == 0) {
+ state->hash_table = rte_hash_create(&hash_params);
+ if (state->hash_table == NULL)
+ return NULL;
}
- return 0;
+ state->refcnt++;
+ return state;
+}
-err_tunnel_hash_map_alloc:
- rte_hash_free(tunnel_rule->hash_table);
+void
+i40e_tunnel_state_detach(struct i40e_tunnel_state *state)
+{
+ if (--state->refcnt > 0)
+ return;
- return ret;
+ rte_hash_free(state->hash_table);
+ *state = (struct i40e_tunnel_state){0};
}
static void
@@ -1656,9 +1642,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
/* Initialize the filter invalidation configuration */
i40e_init_filter_invalidation(pf);
- ret = i40e_init_tunnel_filter_list(dev);
- if (ret < 0)
- goto err_init_tunnel_filter_list;
i40e_fdir_flow_store_init(dev);
/* initialize flow engine configuration */
@@ -1676,9 +1659,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return 0;
err_flow_engine_conf_init:
- rte_hash_free(pf->tunnel.hash_table);
- rte_free(pf->tunnel.hash_map);
-err_init_tunnel_filter_list:
rte_intr_callback_unregister(intr_handle,
i40e_dev_interrupt_handler, dev);
rte_free(dev->data->mac_addrs);
@@ -1701,23 +1681,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
return ret;
}
-static void
-i40e_rm_tunnel_filter_list(struct i40e_pf *pf)
-{
- struct i40e_tunnel_filter *p_tunnel;
- struct i40e_tunnel_rule *tunnel_rule;
-
- tunnel_rule = &pf->tunnel;
- /* Remove all tunnel director rules and hash */
- rte_free(tunnel_rule->hash_map);
- rte_hash_free(tunnel_rule->hash_table);
-
- while ((p_tunnel = TAILQ_FIRST(&tunnel_rule->tunnel_list))) {
- TAILQ_REMOVE(&tunnel_rule->tunnel_list, p_tunnel, rules);
- rte_free(p_tunnel);
- }
-}
-
static void
i40e_fdir_memory_cleanup(struct i40e_pf *pf)
{
@@ -2578,8 +2541,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
i40e_msec_delay(500);
} while (retries++ < 5);
- i40e_rm_tunnel_filter_list(pf);
-
/* Remove all flows */
while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
TAILQ_REMOVE(&pf->flow_list, p_flow, node);
@@ -7747,96 +7708,6 @@ i40e_dev_get_filter_type(uint16_t filter_type, uint16_t *flag)
return 0;
}
-/* Convert tunnel filter structure */
-static int
-i40e_tunnel_filter_convert(
- struct i40e_aqc_cloud_filters_element_bb *cld_filter,
- struct i40e_tunnel_filter *tunnel_filter)
-{
- rte_ether_addr_copy((struct rte_ether_addr *)
- &cld_filter->element.outer_mac,
- (struct rte_ether_addr *)&tunnel_filter->input.outer_mac);
- rte_ether_addr_copy((struct rte_ether_addr *)
- &cld_filter->element.inner_mac,
- (struct rte_ether_addr *)&tunnel_filter->input.inner_mac);
- tunnel_filter->input.inner_vlan = cld_filter->element.inner_vlan;
- if ((rte_le_to_cpu_16(cld_filter->element.flags) &
- I40E_AQC_ADD_CLOUD_FLAGS_IPV6) ==
- I40E_AQC_ADD_CLOUD_FLAGS_IPV6)
- tunnel_filter->input.ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- else
- tunnel_filter->input.ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- tunnel_filter->input.flags = cld_filter->element.flags;
- tunnel_filter->input.tenant_id = cld_filter->element.tenant_id;
- tunnel_filter->queue = cld_filter->element.queue_number;
- memcpy(tunnel_filter->input.general_fields,
- cld_filter->general_fields,
- sizeof(cld_filter->general_fields));
-
- return 0;
-}
-
-/* Check if there exists the tunnel filter */
-struct i40e_tunnel_filter *
-i40e_sw_tunnel_filter_lookup(struct i40e_tunnel_rule *tunnel_rule,
- const struct i40e_tunnel_filter_input *input)
-{
- int ret;
-
- ret = rte_hash_lookup(tunnel_rule->hash_table, (const void *)input);
- if (ret < 0)
- return NULL;
-
- return tunnel_rule->hash_map[ret];
-}
-
-/* Add a tunnel filter into the SW list */
-static int
-i40e_sw_tunnel_filter_insert(struct i40e_pf *pf,
- struct i40e_tunnel_filter *tunnel_filter)
-{
- struct i40e_tunnel_rule *rule = &pf->tunnel;
- int ret;
-
- ret = rte_hash_add_key(rule->hash_table, &tunnel_filter->input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to insert tunnel filter to hash table %d!",
- ret);
- return ret;
- }
- rule->hash_map[ret] = tunnel_filter;
-
- TAILQ_INSERT_TAIL(&rule->tunnel_list, tunnel_filter, rules);
-
- return 0;
-}
-
-/* Delete a tunnel filter from the SW list */
-int
-i40e_sw_tunnel_filter_del(struct i40e_pf *pf,
- struct i40e_tunnel_filter_input *input)
-{
- struct i40e_tunnel_rule *rule = &pf->tunnel;
- struct i40e_tunnel_filter *tunnel_filter;
- int ret;
-
- ret = rte_hash_del_key(rule->hash_table, input);
- if (ret < 0) {
- PMD_DRV_LOG(ERR,
- "Failed to delete tunnel filter to hash table %d!",
- ret);
- return ret;
- }
- tunnel_filter = rule->hash_map[ret];
- rule->hash_map[ret] = NULL;
-
- TAILQ_REMOVE(&rule->tunnel_list, tunnel_filter, rules);
- rte_free(tunnel_filter);
-
- return 0;
-}
-
#define I40E_AQC_REPLACE_CLOUD_CMD_INPUT_TR_WORD0 0x48
#define I40E_TR_VXLAN_GRE_KEY_MASK 0x4
#define I40E_TR_GENEVE_KEY_MASK 0x8
@@ -8275,41 +8146,35 @@ i40e_replace_port_cloud_filter(struct i40e_pf *pf,
return status;
}
-int
-i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
- struct i40e_tunnel_filter_conf *tunnel_filter,
- uint8_t add)
+static int
+i40e_tunnel_filter_convert_conf(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter,
+ struct i40e_aqc_cloud_filters_element_bb *cld_filter,
+ struct i40e_vsi **vsi, bool *big_buffer)
{
uint16_t ip_type;
uint32_t ipv4_addr, ipv4_addr_le;
uint8_t i, tun_type = 0;
/* internal variable to convert ipv6 byte order */
uint32_t convert_ipv6[4];
- int val, ret = 0;
+ int val;
struct i40e_pf_vf *vf = NULL;
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_vsi *vsi;
- struct i40e_aqc_cloud_filters_element_bb cld_filter = {0};
- struct i40e_tunnel_rule *tunnel_rule = &pf->tunnel;
- struct i40e_tunnel_filter *node;
- struct i40e_tunnel_filter check_filter; /* Check if filter exists */
uint32_t teid_le;
- bool big_buffer = 0;
rte_ether_addr_copy(&tunnel_filter->outer_mac,
- (struct rte_ether_addr *)&cld_filter.element.outer_mac);
+ (struct rte_ether_addr *)&cld_filter->element.outer_mac);
rte_ether_addr_copy(&tunnel_filter->inner_mac,
- (struct rte_ether_addr *)&cld_filter.element.inner_mac);
+ (struct rte_ether_addr *)&cld_filter->element.inner_mac);
- cld_filter.element.inner_vlan =
+ cld_filter->element.inner_vlan =
rte_cpu_to_le_16(tunnel_filter->inner_vlan);
if (tunnel_filter->ip_type == I40E_TUNNEL_IPTYPE_IPV4) {
ip_type = I40E_AQC_ADD_CLOUD_FLAGS_IPV4;
ipv4_addr = rte_be_to_cpu_32(tunnel_filter->ip_addr.ipv4_addr);
ipv4_addr_le = rte_cpu_to_le_32(ipv4_addr);
- memcpy(&cld_filter.element.ipaddr.v4.data,
+ memcpy(&cld_filter->element.ipaddr.v4.data,
&ipv4_addr_le,
- sizeof(cld_filter.element.ipaddr.v4.data));
+ sizeof(cld_filter->element.ipaddr.v4.data));
} else {
ip_type = I40E_AQC_ADD_CLOUD_FLAGS_IPV6;
for (i = 0; i < 4; i++) {
@@ -8317,9 +8182,9 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
rte_cpu_to_le_32(rte_be_to_cpu_32(
tunnel_filter->ip_addr.ipv6_addr[i]));
}
- memcpy(&cld_filter.element.ipaddr.v6.data,
+ memcpy(&cld_filter->element.ipaddr.v6.data,
&convert_ipv6,
- sizeof(cld_filter.element.ipaddr.v6.data));
+ sizeof(cld_filter->element.ipaddr.v6.data));
}
/* check tunneled type */
@@ -8334,137 +8199,96 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
tun_type = I40E_AQC_ADD_CLOUD_TNL_TYPE_IP;
break;
case I40E_TUNNEL_TYPE_MPLSoUDP:
- if (!pf->mpls_replace_flag) {
- i40e_replace_mpls_l1_filter(pf);
- i40e_replace_mpls_cloud_filter(pf);
- pf->mpls_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
teid_le >> 4;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
(teid_le & 0xF) << 12;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
0x40;
- big_buffer = 1;
+ *big_buffer = 1;
tun_type = I40E_AQC_ADD_CLOUD_TNL_TYPE_MPLSOUDP;
break;
case I40E_TUNNEL_TYPE_MPLSoGRE:
- if (!pf->mpls_replace_flag) {
- i40e_replace_mpls_l1_filter(pf);
- i40e_replace_mpls_cloud_filter(pf);
- pf->mpls_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
teid_le >> 4;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
(teid_le & 0xF) << 12;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
0x0;
- big_buffer = 1;
+ *big_buffer = 1;
tun_type = I40E_AQC_ADD_CLOUD_TNL_TYPE_MPLSOGRE;
break;
case I40E_TUNNEL_TYPE_GTPC:
- if (!pf->gtp_replace_flag) {
- i40e_replace_gtp_l1_filter(pf);
- i40e_replace_gtp_cloud_filter(pf);
- pf->gtp_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD0] =
(teid_le >> 16) & 0xFFFF;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD1] =
teid_le & 0xFFFF;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X12_WORD2] =
0x0;
- big_buffer = 1;
+ *big_buffer = 1;
break;
case I40E_TUNNEL_TYPE_GTPU:
- if (!pf->gtp_replace_flag) {
- i40e_replace_gtp_l1_filter(pf);
- i40e_replace_gtp_cloud_filter(pf);
- pf->gtp_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD0] =
(teid_le >> 16) & 0xFFFF;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD1] =
teid_le & 0xFFFF;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X13_WORD2] =
0x0;
- big_buffer = 1;
+ *big_buffer = 1;
break;
case I40E_TUNNEL_TYPE_QINQ:
- if (!pf->qinq_replace_flag) {
- ret = i40e_cloud_filter_qinq_create(pf);
- if (ret < 0)
- PMD_DRV_LOG(DEBUG,
- "QinQ tunnel filter already created.");
- pf->qinq_replace_flag = 1;
- }
/* Add in the General fields the values of
* the Outer and Inner VLAN
* Big Buffer should be set, see changes in
* i40e_aq_add_cloud_filters
*/
- cld_filter.general_fields[0] = tunnel_filter->inner_vlan;
- cld_filter.general_fields[1] = tunnel_filter->outer_vlan;
- big_buffer = 1;
+ cld_filter->general_fields[0] = tunnel_filter->inner_vlan;
+ cld_filter->general_fields[1] = tunnel_filter->outer_vlan;
+ *big_buffer = 1;
break;
case I40E_CLOUD_TYPE_UDP:
case I40E_CLOUD_TYPE_TCP:
case I40E_CLOUD_TYPE_SCTP:
if (tunnel_filter->l4_port_type == I40E_L4_PORT_TYPE_SRC) {
- if (!pf->sport_replace_flag) {
- i40e_replace_port_l1_filter(pf,
- tunnel_filter->l4_port_type);
- i40e_replace_port_cloud_filter(pf,
- tunnel_filter->l4_port_type);
- pf->sport_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD0] =
I40E_DIRECTION_INGRESS_KEY;
if (tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_UDP)
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
I40E_TR_L4_TYPE_UDP;
else if (tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_TCP)
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
I40E_TR_L4_TYPE_TCP;
else
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD1] =
I40E_TR_L4_TYPE_SCTP;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X11_WORD2] =
(teid_le >> 16) & 0xFFFF;
- big_buffer = 1;
+ *big_buffer = 1;
} else {
- if (!pf->dport_replace_flag) {
- i40e_replace_port_l1_filter(pf,
- tunnel_filter->l4_port_type);
- i40e_replace_port_cloud_filter(pf,
- tunnel_filter->l4_port_type);
- pf->dport_replace_flag = 1;
- }
teid_le = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD0] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD0] =
I40E_DIRECTION_INGRESS_KEY;
if (tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_UDP)
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
I40E_TR_L4_TYPE_UDP;
else if (tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_TCP)
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
I40E_TR_L4_TYPE_TCP;
else
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD1] =
I40E_TR_L4_TYPE_SCTP;
- cld_filter.general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD2] =
+ cld_filter->general_fields[I40E_AQC_ADD_CLOUD_FV_FLU_0X10_WORD2] =
(teid_le >> 16) & 0xFFFF;
- big_buffer = 1;
+ *big_buffer = 1;
}
break;
@@ -8475,74 +8299,191 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
}
if (tunnel_filter->tunnel_type == I40E_TUNNEL_TYPE_MPLSoUDP)
- cld_filter.element.flags =
+ cld_filter->element.flags =
I40E_AQC_ADD_CLOUD_FILTER_0X11;
else if (tunnel_filter->tunnel_type == I40E_TUNNEL_TYPE_MPLSoGRE)
- cld_filter.element.flags =
+ cld_filter->element.flags =
I40E_AQC_ADD_CLOUD_FILTER_0X12;
else if (tunnel_filter->tunnel_type == I40E_TUNNEL_TYPE_GTPC)
- cld_filter.element.flags =
+ cld_filter->element.flags =
I40E_AQC_ADD_CLOUD_FILTER_0X11;
else if (tunnel_filter->tunnel_type == I40E_TUNNEL_TYPE_GTPU)
- cld_filter.element.flags =
+ cld_filter->element.flags =
I40E_AQC_ADD_CLOUD_FILTER_0X12;
else if (tunnel_filter->tunnel_type == I40E_TUNNEL_TYPE_QINQ)
- cld_filter.element.flags |=
+ cld_filter->element.flags |=
I40E_AQC_ADD_CLOUD_FILTER_0X10;
else if (tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_UDP ||
tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_TCP ||
tunnel_filter->tunnel_type == I40E_CLOUD_TYPE_SCTP) {
if (tunnel_filter->l4_port_type == I40E_L4_PORT_TYPE_SRC)
- cld_filter.element.flags |=
+ cld_filter->element.flags |=
I40E_AQC_ADD_CLOUD_FILTER_0X11;
else
- cld_filter.element.flags |=
+ cld_filter->element.flags |=
I40E_AQC_ADD_CLOUD_FILTER_0X10;
} else {
val = i40e_dev_get_filter_type(tunnel_filter->filter_type,
- &cld_filter.element.flags);
+ &cld_filter->element.flags);
if (val < 0) {
return -EINVAL;
}
}
- cld_filter.element.flags |=
+ cld_filter->element.flags |=
rte_cpu_to_le_16(I40E_AQC_ADD_CLOUD_FLAGS_TO_QUEUE | ip_type |
(tun_type << I40E_AQC_ADD_CLOUD_TNL_TYPE_SHIFT));
- cld_filter.element.tenant_id = rte_cpu_to_le_32(tunnel_filter->tenant_id);
- cld_filter.element.queue_number =
+ cld_filter->element.tenant_id = rte_cpu_to_le_32(tunnel_filter->tenant_id);
+ cld_filter->element.queue_number =
rte_cpu_to_le_16(tunnel_filter->queue_id);
if (!tunnel_filter->is_to_vf)
- vsi = pf->main_vsi;
+ *vsi = pf->main_vsi;
else {
if (tunnel_filter->vf_id >= pf->vf_num) {
PMD_DRV_LOG(ERR, "Invalid argument.");
return -EINVAL;
}
vf = &pf->vfs[tunnel_filter->vf_id];
- vsi = vf->vsi;
+ *vsi = vf->vsi;
}
- /* Check if there is the filter in SW list */
- memset(&check_filter, 0, sizeof(check_filter));
- i40e_tunnel_filter_convert(&cld_filter, &check_filter);
- check_filter.is_to_vf = tunnel_filter->is_to_vf;
- check_filter.vf_id = tunnel_filter->vf_id;
- node = i40e_sw_tunnel_filter_lookup(tunnel_rule, &check_filter.input);
- if (add && node) {
- PMD_DRV_LOG(ERR, "Conflict with existing tunnel rules!");
- return -EINVAL;
- }
+ return 0;
+}
+
+static int
+i40e_tunnel_filter_prepare_hw(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter)
+{
+ int ret;
- if (!add && !node) {
- PMD_DRV_LOG(ERR, "There's no corresponding tunnel filter!");
- return -EINVAL;
+ switch (tunnel_filter->tunnel_type) {
+ case I40E_TUNNEL_TYPE_MPLSoUDP:
+ case I40E_TUNNEL_TYPE_MPLSoGRE:
+ if (!pf->mpls_replace_flag) {
+ ret = i40e_replace_mpls_l1_filter(pf);
+ if (ret < 0)
+ return ret;
+ ret = i40e_replace_mpls_cloud_filter(pf);
+ if (ret < 0)
+ return ret;
+ pf->mpls_replace_flag = 1;
+ }
+ break;
+ case I40E_TUNNEL_TYPE_GTPC:
+ case I40E_TUNNEL_TYPE_GTPU:
+ if (!pf->gtp_replace_flag) {
+ ret = i40e_replace_gtp_l1_filter(pf);
+ if (ret < 0)
+ return ret;
+ ret = i40e_replace_gtp_cloud_filter(pf);
+ if (ret < 0)
+ return ret;
+ pf->gtp_replace_flag = 1;
+ }
+ break;
+ case I40E_TUNNEL_TYPE_QINQ:
+ if (!pf->qinq_replace_flag) {
+ ret = i40e_cloud_filter_qinq_create(pf);
+ if (ret < 0)
+ PMD_DRV_LOG(DEBUG,
+ "QinQ tunnel filter already created.");
+ pf->qinq_replace_flag = 1;
+ }
+ break;
+ case I40E_CLOUD_TYPE_UDP:
+ case I40E_CLOUD_TYPE_TCP:
+ case I40E_CLOUD_TYPE_SCTP:
+ if (tunnel_filter->l4_port_type == I40E_L4_PORT_TYPE_SRC) {
+ if (!pf->sport_replace_flag) {
+ ret = i40e_replace_port_l1_filter(pf,
+ tunnel_filter->l4_port_type);
+ if (ret < 0)
+ return ret;
+ ret = i40e_replace_port_cloud_filter(pf,
+ tunnel_filter->l4_port_type);
+ if (ret < 0)
+ return ret;
+ pf->sport_replace_flag = 1;
+ }
+ } else if (!pf->dport_replace_flag) {
+ ret = i40e_replace_port_l1_filter(pf,
+ tunnel_filter->l4_port_type);
+ if (ret < 0)
+ return ret;
+ ret = i40e_replace_port_cloud_filter(pf,
+ tunnel_filter->l4_port_type);
+ if (ret < 0)
+ return ret;
+ pf->dport_replace_flag = 1;
+ }
+ break;
+ default:
+ break;
}
+ return 0;
+}
+
+int
+i40e_tunnel_filter_match_key_get(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter,
+ struct i40e_tunnel_filter_match_key *input)
+{
+ struct i40e_aqc_cloud_filters_element_bb cld_filter = {0};
+ struct i40e_vsi *vsi;
+ bool big_buffer = 0;
+ int ret;
+
+ ret = i40e_tunnel_filter_convert_conf(pf, tunnel_filter, &cld_filter,
+ &vsi, &big_buffer);
+ if (ret != 0)
+ return ret;
+
+ rte_ether_addr_copy((struct rte_ether_addr *)
+ &cld_filter.element.outer_mac,
+ (struct rte_ether_addr *)&input->outer_mac);
+ rte_ether_addr_copy((struct rte_ether_addr *)
+ &cld_filter.element.inner_mac,
+ (struct rte_ether_addr *)&input->inner_mac);
+ input->inner_vlan = cld_filter.element.inner_vlan;
+ if ((rte_le_to_cpu_16(cld_filter.element.flags) &
+ I40E_AQC_ADD_CLOUD_FLAGS_IPV6) ==
+ I40E_AQC_ADD_CLOUD_FLAGS_IPV6)
+ input->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
+ else
+ input->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
+ input->flags = cld_filter.element.flags;
+ input->tenant_id = cld_filter.element.tenant_id;
+ memcpy(input->general_fields, cld_filter.general_fields,
+ sizeof(cld_filter.general_fields));
+
+ return 0;
+}
+
+int
+i40e_tunnel_filter_program(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter,
+ uint8_t add)
+{
+ struct i40e_hw *hw = I40E_PF_TO_HW(pf);
+ struct i40e_aqc_cloud_filters_element_bb cld_filter = {0};
+ struct i40e_vsi *vsi;
+ bool big_buffer = 0;
+ int ret;
+
+ ret = i40e_tunnel_filter_convert_conf(pf, tunnel_filter, &cld_filter,
+ &vsi, &big_buffer);
+ if (ret != 0)
+ return ret;
+
if (add) {
- struct i40e_tunnel_filter *tunnel;
+ ret = i40e_tunnel_filter_prepare_hw(pf, tunnel_filter);
+ if (ret < 0)
+ return ret;
+ }
+ if (add) {
if (big_buffer)
ret = i40e_aq_add_cloud_filters_bb(hw,
vsi->seid, &cld_filter, 1);
@@ -8553,16 +8494,6 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
PMD_DRV_LOG(ERR, "Failed to add a tunnel filter.");
return -ENOTSUP;
}
- tunnel = rte_zmalloc("tunnel_filter", sizeof(*tunnel), 0);
- if (tunnel == NULL) {
- PMD_DRV_LOG(ERR, "Failed to alloc memory.");
- return -ENOMEM;
- }
-
- memcpy(tunnel, &check_filter, sizeof(check_filter));
- ret = i40e_sw_tunnel_filter_insert(pf, tunnel);
- if (ret < 0)
- rte_free(tunnel);
} else {
if (big_buffer)
ret = i40e_aq_rem_cloud_filters_bb(
@@ -8574,7 +8505,6 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
PMD_DRV_LOG(ERR, "Failed to delete a tunnel filter.");
return -ENOTSUP;
}
- ret = i40e_sw_tunnel_filter_del(pf, &node->input);
}
return ret;
@@ -11282,65 +11212,9 @@ i40e_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
return 0;
}
-/* Restore tunnel filter */
-static void
-i40e_tunnel_filter_restore(struct i40e_pf *pf)
-{
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_vsi *vsi;
- struct i40e_pf_vf *vf;
- struct i40e_tunnel_filter_list
- *tunnel_list = &pf->tunnel.tunnel_list;
- struct i40e_tunnel_filter *f;
- struct i40e_aqc_cloud_filters_element_bb cld_filter;
- bool big_buffer = 0;
-
- TAILQ_FOREACH(f, tunnel_list, rules) {
- if (!f->is_to_vf)
- vsi = pf->main_vsi;
- else {
- vf = &pf->vfs[f->vf_id];
- vsi = vf->vsi;
- }
- memset(&cld_filter, 0, sizeof(cld_filter));
- rte_ether_addr_copy((struct rte_ether_addr *)
- &f->input.outer_mac,
- (struct rte_ether_addr *)&cld_filter.element.outer_mac);
- rte_ether_addr_copy((struct rte_ether_addr *)
- &f->input.inner_mac,
- (struct rte_ether_addr *)&cld_filter.element.inner_mac);
- cld_filter.element.inner_vlan = f->input.inner_vlan;
- cld_filter.element.flags = f->input.flags;
- cld_filter.element.tenant_id = f->input.tenant_id;
- cld_filter.element.queue_number = f->queue;
- memcpy(cld_filter.general_fields,
- f->input.general_fields,
- sizeof(f->input.general_fields));
-
- if (((f->input.flags &
- I40E_AQC_ADD_CLOUD_FILTER_0X11) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X11) ||
- ((f->input.flags &
- I40E_AQC_ADD_CLOUD_FILTER_0X12) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X12) ||
- ((f->input.flags &
- I40E_AQC_ADD_CLOUD_FILTER_0X10) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X10))
- big_buffer = 1;
-
- if (big_buffer)
- i40e_aq_add_cloud_filters_bb(hw,
- vsi->seid, &cld_filter, 1);
- else
- i40e_aq_add_cloud_filters(hw, vsi->seid,
- &cld_filter.element, 1);
- }
-}
-
static void
i40e_filter_restore(struct i40e_pf *pf)
{
- i40e_tunnel_filter_restore(pf);
i40e_fdir_filter_restore(pf);
(void)i40e_hash_filter_restore(pf);
}
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index 7a326fa75b..9d68d8fd0f 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -877,8 +877,8 @@ enum i40e_tunnel_iptype {
I40E_TUNNEL_IPTYPE_IPV6,
};
-/* Tunnel filter struct */
-struct i40e_tunnel_filter_input {
+/* Tunnel filter hash table match key */
+struct i40e_tunnel_filter_match_key {
uint8_t outer_mac[6]; /* Outer mac address to match */
uint8_t inner_mac[6]; /* Inner mac address to match */
uint16_t inner_vlan; /* Inner vlan address to match */
@@ -888,20 +888,9 @@ struct i40e_tunnel_filter_input {
uint16_t general_fields[32]; /* Big buffer */
};
-struct i40e_tunnel_filter {
- TAILQ_ENTRY(i40e_tunnel_filter) rules;
- struct i40e_tunnel_filter_input input;
- uint8_t is_to_vf; /* 0 - to PF, 1 - to VF */
- uint16_t vf_id; /* VF id, available when is_to_vf is 1. */
- uint16_t queue; /* Queue assigned to when match */
-};
-
-TAILQ_HEAD(i40e_tunnel_filter_list, i40e_tunnel_filter);
-
-struct i40e_tunnel_rule {
- struct i40e_tunnel_filter_list tunnel_list;
- struct i40e_tunnel_filter **hash_map;
+struct i40e_tunnel_state {
struct rte_hash *hash_table;
+ uint16_t refcnt;
};
/**
@@ -1170,7 +1159,7 @@ struct i40e_pf {
struct i40e_vmdq_info *vmdq;
struct i40e_fdir_info fdir; /* flow director info */
- struct i40e_tunnel_rule tunnel; /* Tunnel filter rule */
+ struct i40e_tunnel_state tunnel_state; /* tunnel flow engine state */
struct i40e_rss_conf_list rss_config_list; /* RSS rule list */
struct i40e_queue_regions queue_region; /* queue region info */
struct i40e_fc_conf fc_conf; /* Flow control conf */
@@ -1317,23 +1306,10 @@ struct i40e_vf_representor {
extern const struct rte_flow_ops i40e_flow_ops;
struct i40e_filter_ctx {
- union {
- struct i40e_tunnel_filter_conf consistent_tunnel_filter;
- struct i40e_rte_flow_rss_conf rss_conf;
- };
+ struct i40e_rte_flow_rss_conf rss_conf;
enum rte_filter_type type;
};
-typedef int (*parse_filter_t)(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-struct i40e_valid_pattern {
- enum rte_flow_item_type *items;
- parse_filter_t parse_filter;
-};
-
int i40e_dev_switch_queues(struct i40e_pf *pf, bool on);
int i40e_vsi_release(struct i40e_vsi *vsi);
struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf,
@@ -1399,11 +1375,6 @@ int i40e_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_burst_mode *mode);
-struct i40e_tunnel_filter *
-i40e_sw_tunnel_filter_lookup(struct i40e_tunnel_rule *tunnel_rule,
- const struct i40e_tunnel_filter_input *input);
-int i40e_sw_tunnel_filter_del(struct i40e_pf *pf,
- struct i40e_tunnel_filter_input *input);
uint64_t i40e_get_default_input_set(uint16_t pctype);
int i40e_ethertype_filter_program(struct i40e_pf *pf,
struct rte_eth_ethertype_filter *filter,
@@ -1433,9 +1404,14 @@ void i40e_fdir_flex_store(struct i40e_pf *pf,
int i40e_dev_tunnel_filter_set(struct i40e_pf *pf,
struct rte_eth_tunnel_filter_conf *tunnel_filter,
uint8_t add);
-int i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
- struct i40e_tunnel_filter_conf *tunnel_filter,
- uint8_t add);
+int i40e_tunnel_filter_match_key_get(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter,
+ struct i40e_tunnel_filter_match_key *out);
+struct i40e_tunnel_state *i40e_tunnel_state_attach(struct rte_eth_dev_data *dev_data);
+void i40e_tunnel_state_detach(struct i40e_tunnel_state *state);
+int i40e_tunnel_filter_program(struct i40e_pf *pf,
+ struct i40e_tunnel_filter_conf *tunnel_filter,
+ uint8_t add);
int i40e_fdir_flush(struct i40e_pf *pf);
int i40e_find_all_vlan_for_mac(struct i40e_vsi *vsi,
struct i40e_macvlan_filter *mv_f,
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 104749eb8c..0de0c82521 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -34,6 +34,12 @@ const struct ci_flow_engine_list i40e_flow_engine_list = {
{
&i40e_flow_engine_ethertype,
&i40e_flow_engine_fdir,
+ &i40e_flow_engine_tunnel_qinq,
+ &i40e_flow_engine_tunnel_vxlan,
+ &i40e_flow_engine_tunnel_nvgre,
+ &i40e_flow_engine_tunnel_mpls,
+ &i40e_flow_engine_tunnel_gtp,
+ &i40e_flow_engine_tunnel_l4,
}
};
@@ -60,50 +66,7 @@ static int i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow *flow,
FILE *file,
struct rte_flow_error *error);
-static int i40e_flow_parse_tunnel_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter);
-static int i40e_flow_parse_vxlan_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-static int i40e_flow_parse_nvgre_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-static int i40e_flow_parse_mpls_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-static int i40e_flow_parse_gtp_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-static int i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
- struct i40e_tunnel_filter *filter);
-static int i40e_flow_flush_tunnel_filter(struct i40e_pf *pf);
-static int
-i40e_flow_parse_qinq_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
-static int
-i40e_flow_parse_qinq_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter);
-static int i40e_flow_parse_l4_cloud_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter);
const struct rte_flow_ops i40e_flow_ops = {
.validate = i40e_flow_validate,
.create = i40e_flow_create,
@@ -113,322 +76,12 @@ const struct rte_flow_ops i40e_flow_ops = {
.dev_dump = i40e_flow_dev_dump,
};
-static enum rte_flow_item_type pattern_fdir_ipv4_udp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_tcp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_sctp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_gtpc[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPC,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv4_gtpu[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_udp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_tcp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_TCP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_sctp[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_SCTP,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_gtpc[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPC,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_fdir_ipv6_gtpu[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_GTPU,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-/* Pattern matched tunnel filter */
-static enum rte_flow_item_type pattern_vxlan_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_VXLAN,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_vxlan_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_VXLAN,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_vxlan_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_VXLAN,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_vxlan_4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_VXLAN,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_nvgre_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_NVGRE,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_nvgre_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_NVGRE,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_nvgre_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_NVGRE,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_nvgre_4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_NVGRE,
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_mpls_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_MPLS,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_mpls_2[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_UDP,
- RTE_FLOW_ITEM_TYPE_MPLS,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_mpls_3[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV4,
- RTE_FLOW_ITEM_TYPE_GRE,
- RTE_FLOW_ITEM_TYPE_MPLS,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_mpls_4[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_IPV6,
- RTE_FLOW_ITEM_TYPE_GRE,
- RTE_FLOW_ITEM_TYPE_MPLS,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static enum rte_flow_item_type pattern_qinq_1[] = {
- RTE_FLOW_ITEM_TYPE_ETH,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_VLAN,
- RTE_FLOW_ITEM_TYPE_END,
-};
-
-static struct i40e_valid_pattern i40e_supported_patterns[] = {
- /* VXLAN */
- { pattern_vxlan_1, i40e_flow_parse_vxlan_filter },
- { pattern_vxlan_2, i40e_flow_parse_vxlan_filter },
- { pattern_vxlan_3, i40e_flow_parse_vxlan_filter },
- { pattern_vxlan_4, i40e_flow_parse_vxlan_filter },
- /* NVGRE */
- { pattern_nvgre_1, i40e_flow_parse_nvgre_filter },
- { pattern_nvgre_2, i40e_flow_parse_nvgre_filter },
- { pattern_nvgre_3, i40e_flow_parse_nvgre_filter },
- { pattern_nvgre_4, i40e_flow_parse_nvgre_filter },
- /* MPLSoUDP & MPLSoGRE */
- { pattern_mpls_1, i40e_flow_parse_mpls_filter },
- { pattern_mpls_2, i40e_flow_parse_mpls_filter },
- { pattern_mpls_3, i40e_flow_parse_mpls_filter },
- { pattern_mpls_4, i40e_flow_parse_mpls_filter },
- /* GTP-C & GTP-U */
- { pattern_fdir_ipv4_gtpc, i40e_flow_parse_gtp_filter },
- { pattern_fdir_ipv4_gtpu, i40e_flow_parse_gtp_filter },
- { pattern_fdir_ipv6_gtpc, i40e_flow_parse_gtp_filter },
- { pattern_fdir_ipv6_gtpu, i40e_flow_parse_gtp_filter },
- /* QINQ */
- { pattern_qinq_1, i40e_flow_parse_qinq_filter },
- /* L4 over port */
- { pattern_fdir_ipv4_udp, i40e_flow_parse_l4_cloud_filter },
- { pattern_fdir_ipv4_tcp, i40e_flow_parse_l4_cloud_filter },
- { pattern_fdir_ipv4_sctp, i40e_flow_parse_l4_cloud_filter },
- { pattern_fdir_ipv6_udp, i40e_flow_parse_l4_cloud_filter },
- { pattern_fdir_ipv6_tcp, i40e_flow_parse_l4_cloud_filter },
- { pattern_fdir_ipv6_sctp, i40e_flow_parse_l4_cloud_filter },
-};
-
-/* Find the first VOID or non-VOID item pointer */
-static const struct rte_flow_item *
-i40e_find_first_item(const struct rte_flow_item *item, bool is_void)
-{
- bool is_find;
-
- while (item->type != RTE_FLOW_ITEM_TYPE_END) {
- if (is_void)
- is_find = item->type == RTE_FLOW_ITEM_TYPE_VOID;
- else
- is_find = item->type != RTE_FLOW_ITEM_TYPE_VOID;
- if (is_find)
- break;
- item++;
- }
- return item;
-}
-
-/* Skip all VOID items of the pattern */
-static void
-i40e_pattern_skip_void_item(struct rte_flow_item *items,
- const struct rte_flow_item *pattern)
-{
- uint32_t cpy_count = 0;
- const struct rte_flow_item *pb = pattern, *pe = pattern;
-
- for (;;) {
- /* Find a non-void item first */
- pb = i40e_find_first_item(pb, false);
- if (pb->type == RTE_FLOW_ITEM_TYPE_END) {
- pe = pb;
- break;
- }
-
- /* Find a void item */
- pe = i40e_find_first_item(pb + 1, true);
-
- cpy_count = pe - pb;
- memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
-
- items += cpy_count;
-
- if (pe->type == RTE_FLOW_ITEM_TYPE_END) {
- pb = pe;
- break;
- }
-
- pb = pe + 1;
- }
- /* Copy the END item. */
- memcpy(items, pe, sizeof(struct rte_flow_item));
-}
-
-/* Check if the pattern matches a supported item type array */
-static bool
-i40e_match_pattern(enum rte_flow_item_type *item_array,
- struct rte_flow_item *pattern)
-{
- struct rte_flow_item *item = pattern;
-
- while ((*item_array == item->type) &&
- (*item_array != RTE_FLOW_ITEM_TYPE_END)) {
- item_array++;
- item++;
- }
-
- return (*item_array == RTE_FLOW_ITEM_TYPE_END &&
- item->type == RTE_FLOW_ITEM_TYPE_END);
-}
-
-/* Find if there's parse filter function matched */
-static parse_filter_t
-i40e_find_parse_filter_func(struct rte_flow_item *pattern, uint32_t *idx)
-{
- parse_filter_t parse_filter = NULL;
- uint8_t i = *idx;
-
- for (; i < RTE_DIM(i40e_supported_patterns); i++) {
- if (i40e_match_pattern(i40e_supported_patterns[i].items,
- pattern)) {
- parse_filter = i40e_supported_patterns[i].parse_filter;
- break;
- }
- }
-
- *idx = ++i;
-
- return parse_filter;
-}
-
#define I40E_FLOW_DUMP_CHUNK_BYTES 32
static const char *
i40e_flow_rule_name(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_TUNNEL:
- return "tunnel";
case RTE_ETH_FILTER_HASH:
return "hash";
default:
@@ -440,8 +93,6 @@ static size_t
i40e_flow_rule_size(enum rte_filter_type filter_type)
{
switch (filter_type) {
- case RTE_ETH_FILTER_TUNNEL:
- return sizeof(struct i40e_tunnel_filter);
case RTE_ETH_FILTER_HASH:
return sizeof(struct i40e_rss_filter);
default:
@@ -631,1190 +282,6 @@ i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
return I40E_FILTER_PCTYPE_INVALID;
}
-/* Parse to get the action info of a tunnel filter
- * Tunnel action only supports PF, VF and QUEUE.
- */
-static int
-i40e_flow_parse_tunnel_action(struct rte_eth_dev *dev,
- const struct rte_flow_action *actions,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- const struct rte_flow_action_queue *act_q;
- struct ci_flow_actions parsed_actions = {0};
- struct ci_flow_actions_check_param ac_param = {
- .allowed_types = (enum rte_flow_action_type[]) {
- RTE_FLOW_ACTION_TYPE_QUEUE,
- RTE_FLOW_ACTION_TYPE_PF,
- RTE_FLOW_ACTION_TYPE_VF,
- RTE_FLOW_ACTION_TYPE_END
- },
- .max_actions = 2,
- };
- const struct rte_flow_action *first, *second;
- int ret;
-
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- first = parsed_actions.actions[0];
- /* can be NULL */
- second = parsed_actions.actions[1];
-
- /* first action must be PF or VF */
- if (first->type == RTE_FLOW_ACTION_TYPE_VF) {
- const struct rte_flow_action_vf *vf = first->conf;
- if (vf->id >= pf->vf_num) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, first,
- "Invalid VF ID for tunnel filter");
- return -rte_errno;
- }
- filter->vf_id = vf->id;
- filter->is_to_vf = 1;
- } else if (first->type != RTE_FLOW_ACTION_TYPE_PF) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, first,
- "Unsupported action");
- }
-
- /* check if second action is QUEUE */
- if (second == NULL)
- return 0;
-
- act_q = second->conf;
- /* check queue ID for PF flow */
- if (!filter->is_to_vf && act_q->index >= pf->dev_data->nb_rx_queues) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, act_q,
- "Invalid queue ID for tunnel filter");
- }
- /* check queue ID for VF flow */
- if (filter->is_to_vf && act_q->index >= pf->vf_nb_qps) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, act_q,
- "Invalid queue ID for tunnel filter");
- }
- filter->queue_id = act_q->index;
-
- return 0;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: Source port only and Destination port only.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- */
-static int
-i40e_flow_parse_l4_pattern(const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- const struct rte_flow_item_sctp *sctp_spec, *sctp_mask;
- const struct rte_flow_item_tcp *tcp_spec, *tcp_mask;
- const struct rte_flow_item_udp *udp_spec, *udp_mask;
- const struct rte_flow_item *item = pattern;
- enum rte_flow_item_type item_type;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ETH item");
- return -rte_errno;
- }
-
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- /* IPv4 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 item");
- return -rte_errno;
- }
-
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- /* IPv6 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 item");
- return -rte_errno;
- }
-
- break;
- case RTE_FLOW_ITEM_TYPE_UDP:
- udp_spec = item->spec;
- udp_mask = item->mask;
-
- if (!udp_spec || !udp_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid udp item");
- return -rte_errno;
- }
-
- if (udp_spec->hdr.src_port != 0 &&
- udp_spec->hdr.dst_port != 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid udp spec");
- return -rte_errno;
- }
-
- if (udp_spec->hdr.src_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_SRC;
- filter->tenant_id =
- rte_be_to_cpu_32(udp_spec->hdr.src_port);
- }
-
- if (udp_spec->hdr.dst_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_DST;
- filter->tenant_id =
- rte_be_to_cpu_32(udp_spec->hdr.dst_port);
- }
-
- filter->tunnel_type = I40E_CLOUD_TYPE_UDP;
-
- break;
- case RTE_FLOW_ITEM_TYPE_TCP:
- tcp_spec = item->spec;
- tcp_mask = item->mask;
-
- if (!tcp_spec || !tcp_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid tcp item");
- return -rte_errno;
- }
-
- if (tcp_spec->hdr.src_port != 0 &&
- tcp_spec->hdr.dst_port != 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid tcp spec");
- return -rte_errno;
- }
-
- if (tcp_spec->hdr.src_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_SRC;
- filter->tenant_id =
- rte_be_to_cpu_32(tcp_spec->hdr.src_port);
- }
-
- if (tcp_spec->hdr.dst_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_DST;
- filter->tenant_id =
- rte_be_to_cpu_32(tcp_spec->hdr.dst_port);
- }
-
- filter->tunnel_type = I40E_CLOUD_TYPE_TCP;
-
- break;
- case RTE_FLOW_ITEM_TYPE_SCTP:
- sctp_spec = item->spec;
- sctp_mask = item->mask;
-
- if (!sctp_spec || !sctp_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid sctp item");
- return -rte_errno;
- }
-
- if (sctp_spec->hdr.src_port != 0 &&
- sctp_spec->hdr.dst_port != 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid sctp spec");
- return -rte_errno;
- }
-
- if (sctp_spec->hdr.src_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_SRC;
- filter->tenant_id =
- rte_be_to_cpu_32(sctp_spec->hdr.src_port);
- }
-
- if (sctp_spec->hdr.dst_port != 0) {
- filter->l4_port_type =
- I40E_L4_PORT_TYPE_DST;
- filter->tenant_id =
- rte_be_to_cpu_32(sctp_spec->hdr.dst_port);
- }
-
- filter->tunnel_type = I40E_CLOUD_TYPE_SCTP;
-
- break;
- default:
- break;
- }
- }
-
- return 0;
-}
-
-static int
-i40e_flow_parse_l4_cloud_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_l4_pattern(pattern, error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
-static uint16_t i40e_supported_tunnel_filter_types[] = {
- RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_TENID |
- RTE_ETH_TUNNEL_FILTER_IVLAN,
- RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_IVLAN,
- RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_TENID,
- RTE_ETH_TUNNEL_FILTER_OMAC | RTE_ETH_TUNNEL_FILTER_TENID |
- RTE_ETH_TUNNEL_FILTER_IMAC,
- RTE_ETH_TUNNEL_FILTER_IMAC,
-};
-
-static int
-i40e_check_tunnel_filter_type(uint8_t filter_type)
-{
- uint8_t i;
-
- for (i = 0; i < RTE_DIM(i40e_supported_tunnel_filter_types); i++) {
- if (filter_type == i40e_supported_tunnel_filter_types[i])
- return 0;
- }
-
- return -1;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: IMAC_IVLAN_TENID, IMAC_IVLAN,
- * IMAC_TENID, OMAC_TENID_IMAC and IMAC.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- */
-static int
-i40e_flow_parse_vxlan_pattern(__rte_unused struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- const struct rte_flow_item_vxlan *vxlan_spec;
- const struct rte_flow_item_vxlan *vxlan_mask;
- const struct rte_flow_item_vlan *vlan_spec;
- const struct rte_flow_item_vlan *vlan_mask;
- uint8_t filter_type = 0;
- bool is_vni_masked = 0;
- uint8_t vni_mask[] = {0xFF, 0xFF, 0xFF};
- enum rte_flow_item_type item_type;
- bool vxlan_flag = 0;
- uint32_t tenant_id_be = 0;
- int ret;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- eth_spec = item->spec;
- eth_mask = item->mask;
-
- /* Check if ETH item is used for place holder.
- * If yes, both spec and mask should be NULL.
- * If no, both spec and mask shouldn't be NULL.
- */
- if ((!eth_spec && eth_mask) ||
- (eth_spec && !eth_mask)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ether spec/mask");
- return -rte_errno;
- }
-
- if (eth_spec && eth_mask) {
- /* DST address of inner MAC shouldn't be masked.
- * SRC address of Inner MAC should be masked.
- */
- if (!rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr) ||
- !rte_is_zero_ether_addr(ð_mask->hdr.src_addr) ||
- eth_mask->hdr.ether_type) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ether spec/mask");
- return -rte_errno;
- }
-
- if (!vxlan_flag) {
- memcpy(&filter->outer_mac,
- ð_spec->hdr.dst_addr,
- RTE_ETHER_ADDR_LEN);
- filter_type |= RTE_ETH_TUNNEL_FILTER_OMAC;
- } else {
- memcpy(&filter->inner_mac,
- ð_spec->hdr.dst_addr,
- RTE_ETHER_ADDR_LEN);
- filter_type |= RTE_ETH_TUNNEL_FILTER_IMAC;
- }
- }
- break;
- case RTE_FLOW_ITEM_TYPE_VLAN:
- vlan_spec = item->spec;
- vlan_mask = item->mask;
- if (!(vlan_spec && vlan_mask) ||
- vlan_mask->hdr.eth_proto) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid vlan item");
- return -rte_errno;
- }
-
- if (vlan_spec && vlan_mask) {
- if (vlan_mask->hdr.vlan_tci ==
- rte_cpu_to_be_16(I40E_VLAN_TCI_MASK))
- filter->inner_vlan =
- rte_be_to_cpu_16(vlan_spec->hdr.vlan_tci) &
- I40E_VLAN_TCI_MASK;
- filter_type |= RTE_ETH_TUNNEL_FILTER_IVLAN;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- /* IPv4 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- /* IPv6 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_UDP:
- /* UDP is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid UDP item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_VXLAN:
- vxlan_spec = item->spec;
- vxlan_mask = item->mask;
- /* Check if VXLAN item is used to describe protocol.
- * If yes, both spec and mask should be NULL.
- * If no, both spec and mask shouldn't be NULL.
- */
- if ((!vxlan_spec && vxlan_mask) ||
- (vxlan_spec && !vxlan_mask)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid VXLAN item");
- return -rte_errno;
- }
-
- /* Check if VNI is masked. */
- if (vxlan_spec && vxlan_mask) {
- is_vni_masked =
- !!memcmp(vxlan_mask->hdr.vni, vni_mask,
- RTE_DIM(vni_mask));
- if (is_vni_masked) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid VNI mask");
- return -rte_errno;
- }
-
- memcpy(((uint8_t *)&tenant_id_be + 1),
- vxlan_spec->hdr.vni, 3);
- filter->tenant_id =
- rte_be_to_cpu_32(tenant_id_be);
- filter_type |= RTE_ETH_TUNNEL_FILTER_TENID;
- }
-
- vxlan_flag = 1;
- break;
- default:
- break;
- }
- }
-
- ret = i40e_check_tunnel_filter_type(filter_type);
- if (ret < 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL,
- "Invalid filter type");
- return -rte_errno;
- }
- filter->filter_type = filter_type;
-
- filter->tunnel_type = I40E_TUNNEL_TYPE_VXLAN;
-
- return 0;
-}
-
-static int
-i40e_flow_parse_vxlan_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_vxlan_pattern(dev, pattern,
- error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: IMAC_IVLAN_TENID, IMAC_IVLAN,
- * IMAC_TENID, OMAC_TENID_IMAC and IMAC.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- */
-static int
-i40e_flow_parse_nvgre_pattern(__rte_unused struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_eth *eth_spec;
- const struct rte_flow_item_eth *eth_mask;
- const struct rte_flow_item_nvgre *nvgre_spec;
- const struct rte_flow_item_nvgre *nvgre_mask;
- const struct rte_flow_item_vlan *vlan_spec;
- const struct rte_flow_item_vlan *vlan_mask;
- enum rte_flow_item_type item_type;
- uint8_t filter_type = 0;
- bool is_tni_masked = 0;
- uint8_t tni_mask[] = {0xFF, 0xFF, 0xFF};
- bool nvgre_flag = 0;
- uint32_t tenant_id_be = 0;
- int ret;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- eth_spec = item->spec;
- eth_mask = item->mask;
-
- /* Check if ETH item is used for place holder.
- * If yes, both spec and mask should be NULL.
- * If no, both spec and mask shouldn't be NULL.
- */
- if ((!eth_spec && eth_mask) ||
- (eth_spec && !eth_mask)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ether spec/mask");
- return -rte_errno;
- }
-
- if (eth_spec && eth_mask) {
- /* DST address of inner MAC shouldn't be masked.
- * SRC address of Inner MAC should be masked.
- */
- if (!rte_is_broadcast_ether_addr(ð_mask->hdr.dst_addr) ||
- !rte_is_zero_ether_addr(ð_mask->hdr.src_addr) ||
- eth_mask->hdr.ether_type) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ether spec/mask");
- return -rte_errno;
- }
-
- if (!nvgre_flag) {
- memcpy(&filter->outer_mac,
- ð_spec->hdr.dst_addr,
- RTE_ETHER_ADDR_LEN);
- filter_type |= RTE_ETH_TUNNEL_FILTER_OMAC;
- } else {
- memcpy(&filter->inner_mac,
- ð_spec->hdr.dst_addr,
- RTE_ETHER_ADDR_LEN);
- filter_type |= RTE_ETH_TUNNEL_FILTER_IMAC;
- }
- }
-
- break;
- case RTE_FLOW_ITEM_TYPE_VLAN:
- vlan_spec = item->spec;
- vlan_mask = item->mask;
- if (!(vlan_spec && vlan_mask) ||
- vlan_mask->hdr.eth_proto) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid vlan item");
- return -rte_errno;
- }
-
- if (vlan_spec && vlan_mask) {
- if (vlan_mask->hdr.vlan_tci ==
- rte_cpu_to_be_16(I40E_VLAN_TCI_MASK))
- filter->inner_vlan =
- rte_be_to_cpu_16(vlan_spec->hdr.vlan_tci) &
- I40E_VLAN_TCI_MASK;
- filter_type |= RTE_ETH_TUNNEL_FILTER_IVLAN;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- /* IPv4 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- /* IPv6 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_NVGRE:
- nvgre_spec = item->spec;
- nvgre_mask = item->mask;
- /* Check if NVGRE item is used to describe protocol.
- * If yes, both spec and mask should be NULL.
- * If no, both spec and mask shouldn't be NULL.
- */
- if ((!nvgre_spec && nvgre_mask) ||
- (nvgre_spec && !nvgre_mask)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid NVGRE item");
- return -rte_errno;
- }
-
- if (nvgre_spec && nvgre_mask) {
- is_tni_masked =
- !!memcmp(nvgre_mask->tni, tni_mask,
- RTE_DIM(tni_mask));
- if (is_tni_masked) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid TNI mask");
- return -rte_errno;
- }
- if (nvgre_mask->protocol &&
- nvgre_mask->protocol != 0xFFFF) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid NVGRE item");
- return -rte_errno;
- }
- if (nvgre_mask->c_k_s_rsvd0_ver &&
- nvgre_mask->c_k_s_rsvd0_ver !=
- rte_cpu_to_be_16(0xFFFF)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid NVGRE item");
- return -rte_errno;
- }
- if (nvgre_spec->c_k_s_rsvd0_ver !=
- rte_cpu_to_be_16(0x2000) &&
- nvgre_mask->c_k_s_rsvd0_ver) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid NVGRE item");
- return -rte_errno;
- }
- if (nvgre_mask->protocol &&
- nvgre_spec->protocol !=
- rte_cpu_to_be_16(0x6558)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid NVGRE item");
- return -rte_errno;
- }
- memcpy(((uint8_t *)&tenant_id_be + 1),
- nvgre_spec->tni, 3);
- filter->tenant_id =
- rte_be_to_cpu_32(tenant_id_be);
- filter_type |= RTE_ETH_TUNNEL_FILTER_TENID;
- }
-
- nvgre_flag = 1;
- break;
- default:
- break;
- }
- }
-
- ret = i40e_check_tunnel_filter_type(filter_type);
- if (ret < 0) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL,
- "Invalid filter type");
- return -rte_errno;
- }
- filter->filter_type = filter_type;
-
- filter->tunnel_type = I40E_TUNNEL_TYPE_NVGRE;
-
- return 0;
-}
-
-static int
-i40e_flow_parse_nvgre_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_nvgre_pattern(dev, pattern,
- error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: MPLS label.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- */
-static int
-i40e_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_mpls *mpls_spec;
- const struct rte_flow_item_mpls *mpls_mask;
- enum rte_flow_item_type item_type;
- bool is_mplsoudp = 0; /* 1 - MPLSoUDP, 0 - MPLSoGRE */
- const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
- uint32_t label_be = 0;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ETH item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- /* IPv4 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- /* IPv6 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_UDP:
- /* UDP is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid UDP item");
- return -rte_errno;
- }
- is_mplsoudp = 1;
- break;
- case RTE_FLOW_ITEM_TYPE_GRE:
- /* GRE is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid GRE item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_MPLS:
- mpls_spec = item->spec;
- mpls_mask = item->mask;
-
- if (!mpls_spec || !mpls_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid MPLS item");
- return -rte_errno;
- }
-
- if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid MPLS label mask");
- return -rte_errno;
- }
- memcpy(((uint8_t *)&label_be + 1),
- mpls_spec->label_tc_s, 3);
- filter->tenant_id = rte_be_to_cpu_32(label_be) >> 4;
- break;
- default:
- break;
- }
- }
-
- if (is_mplsoudp)
- filter->tunnel_type = I40E_TUNNEL_TYPE_MPLSoUDP;
- else
- filter->tunnel_type = I40E_TUNNEL_TYPE_MPLSoGRE;
-
- return 0;
-}
-
-static int
-i40e_flow_parse_mpls_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_mpls_pattern(dev, pattern,
- error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: GTP TEID.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- * 5. GTP profile supports GTPv1 only.
- * 6. GTP-C response message ('source_port' = 2123) is not supported.
- */
-static int
-i40e_flow_parse_gtp_pattern(struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_gtp *gtp_spec;
- const struct rte_flow_item_gtp *gtp_mask;
- enum rte_flow_item_type item_type;
-
- if (!pf->gtp_support) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "GTP is not supported by default.");
- return -rte_errno;
- }
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ETH item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV4:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
- /* IPv4 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv4 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_IPV6:
- filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
- /* IPv6 is used to describe protocol,
- * spec and mask should be NULL.
- */
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid IPv6 item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_UDP:
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid UDP item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_GTPC:
- case RTE_FLOW_ITEM_TYPE_GTPU:
- gtp_spec = item->spec;
- gtp_mask = item->mask;
-
- if (!gtp_spec || !gtp_mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid GTP item");
- return -rte_errno;
- }
-
- if (gtp_mask->hdr.gtp_hdr_info ||
- gtp_mask->hdr.msg_type ||
- gtp_mask->hdr.plen ||
- gtp_mask->hdr.teid != UINT32_MAX) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid GTP mask");
- return -rte_errno;
- }
-
- if (item_type == RTE_FLOW_ITEM_TYPE_GTPC)
- filter->tunnel_type = I40E_TUNNEL_TYPE_GTPC;
- else if (item_type == RTE_FLOW_ITEM_TYPE_GTPU)
- filter->tunnel_type = I40E_TUNNEL_TYPE_GTPU;
-
- filter->tenant_id = rte_be_to_cpu_32(gtp_spec->hdr.teid);
-
- break;
- default:
- break;
- }
- }
-
- return 0;
-}
-
-static int
-i40e_flow_parse_gtp_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_gtp_pattern(dev, pattern,
- error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
-/* 1. Last in item should be NULL as range is not supported.
- * 2. Supported filter types: QINQ.
- * 3. Mask of fields which need to be matched should be
- * filled with 1.
- * 4. Mask of fields which needn't to be matched should be
- * filled with 0.
- */
-static int
-i40e_flow_parse_qinq_pattern(__rte_unused struct rte_eth_dev *dev,
- const struct rte_flow_item *pattern,
- struct rte_flow_error *error,
- struct i40e_tunnel_filter_conf *filter)
-{
- const struct rte_flow_item *item = pattern;
- const struct rte_flow_item_vlan *vlan_spec = NULL;
- const struct rte_flow_item_vlan *vlan_mask = NULL;
- const struct rte_flow_item_vlan *i_vlan_spec = NULL;
- const struct rte_flow_item_vlan *i_vlan_mask = NULL;
- const struct rte_flow_item_vlan *o_vlan_spec = NULL;
- const struct rte_flow_item_vlan *o_vlan_mask = NULL;
-
- enum rte_flow_item_type item_type;
- bool vlan_flag = 0;
-
- for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
- if (item->last) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Not support range");
- return -rte_errno;
- }
- item_type = item->type;
- switch (item_type) {
- case RTE_FLOW_ITEM_TYPE_ETH:
- if (item->spec || item->mask) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid ETH item");
- return -rte_errno;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_VLAN:
- vlan_spec = item->spec;
- vlan_mask = item->mask;
-
- if (!(vlan_spec && vlan_mask) ||
- vlan_mask->hdr.eth_proto) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- item,
- "Invalid vlan item");
- return -rte_errno;
- }
-
- if (!vlan_flag) {
- o_vlan_spec = vlan_spec;
- o_vlan_mask = vlan_mask;
- vlan_flag = 1;
- } else {
- i_vlan_spec = vlan_spec;
- i_vlan_mask = vlan_mask;
- vlan_flag = 0;
- }
- break;
-
- default:
- break;
- }
- }
-
- /* Get filter specification */
- if (o_vlan_mask != NULL && i_vlan_mask != NULL) {
- filter->outer_vlan = rte_be_to_cpu_16(o_vlan_spec->hdr.vlan_tci);
- filter->inner_vlan = rte_be_to_cpu_16(i_vlan_spec->hdr.vlan_tci);
- } else {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL,
- "Invalid filter type");
- return -rte_errno;
- }
-
- filter->tunnel_type = I40E_TUNNEL_TYPE_QINQ;
- return 0;
-}
-
-static int
-i40e_flow_parse_qinq_filter(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error,
- struct i40e_filter_ctx *filter)
-{
- struct i40e_tunnel_filter_conf *tunnel_filter = &filter->consistent_tunnel_filter;
- int ret;
-
- ret = i40e_flow_parse_qinq_pattern(dev, pattern,
- error, tunnel_filter);
- if (ret)
- return ret;
-
- ret = i40e_flow_parse_tunnel_action(dev, actions, error, tunnel_filter);
- if (ret)
- return ret;
-
- filter->type = RTE_ETH_FILTER_TUNNEL;
-
- return ret;
-}
-
static int
i40e_flow_check(struct rte_eth_dev *dev,
const struct rte_flow_attr *attr,
@@ -1823,11 +290,6 @@ i40e_flow_check(struct rte_eth_dev *dev,
struct i40e_filter_ctx *filter_ctx,
struct rte_flow_error *error)
{
- struct rte_flow_item *items; /* internal pattern w/o VOID items */
- parse_filter_t parse_filter;
- uint32_t item_num = 0; /* non-void item number of pattern*/
- uint32_t i = 0;
- bool flag = false;
int ret;
ret = ci_flow_check_attr(attr, NULL, error);
@@ -1851,51 +313,8 @@ i40e_flow_check(struct rte_eth_dev *dev,
/* try parsing as RSS */
filter_ctx->type = RTE_ETH_FILTER_HASH;
- ret = i40e_hash_parse(dev, pattern, actions, &filter_ctx->rss_conf, error);
- if (!ret)
- return ret;
- i = 0;
- /* Get the non-void item number of pattern */
- while ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_END) {
- if ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_VOID)
- item_num++;
- i++;
- }
- item_num++;
- items = calloc(item_num, sizeof(struct rte_flow_item));
- if (items == NULL) {
- rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_ITEM_NUM,
- NULL,
- "No memory for PMD internal items.");
- return -ENOMEM;
- }
-
- i40e_pattern_skip_void_item(items, pattern);
-
- i = 0;
- ret = I40E_NOT_SUPPORTED;
- do {
- parse_filter = i40e_find_parse_filter_func(items, &i);
- if (!parse_filter && !flag) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM,
- pattern, "Unsupported pattern");
-
- free(items);
- return -rte_errno;
- }
-
- if (parse_filter)
- ret = parse_filter(dev, items, actions, error, filter_ctx);
-
- flag = true;
- } while ((ret < 0) && (i < RTE_DIM(i40e_supported_patterns)));
-
- free(items);
-
- return ret;
+ return i40e_hash_parse(dev, pattern, actions, &filter_ctx->rss_conf, error);
}
static int
@@ -1948,14 +367,6 @@ i40e_flow_create(struct rte_eth_dev *dev,
}
switch (filter_ctx.type) {
- case RTE_ETH_FILTER_TUNNEL:
- ret = i40e_dev_consistent_tunnel_filter_set(pf,
- &filter_ctx.consistent_tunnel_filter, 1);
- if (ret)
- goto free_flow;
- flow->rule = TAILQ_LAST(&pf->tunnel.tunnel_list,
- i40e_tunnel_filter_list);
- break;
case RTE_ETH_FILTER_HASH:
ret = i40e_hash_filter_create(pf, &filter_ctx.rss_conf);
if (ret)
@@ -1996,10 +407,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
return 0;
switch (filter_type) {
- case RTE_ETH_FILTER_TUNNEL:
- ret = i40e_flow_destroy_tunnel_filter(pf,
- (struct i40e_tunnel_filter *)flow->rule);
- break;
case RTE_ETH_FILTER_HASH:
ret = i40e_hash_filter_destroy(pf, flow->rule);
break;
@@ -2022,65 +429,6 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
return ret;
}
-static int
-i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
- struct i40e_tunnel_filter *filter)
-{
- struct i40e_hw *hw = I40E_PF_TO_HW(pf);
- struct i40e_vsi *vsi;
- struct i40e_pf_vf *vf;
- struct i40e_aqc_cloud_filters_element_bb cld_filter;
- struct i40e_tunnel_rule *tunnel_rule = &pf->tunnel;
- struct i40e_tunnel_filter *node;
- bool big_buffer = 0;
- int ret = 0;
-
- memset(&cld_filter, 0, sizeof(cld_filter));
- rte_ether_addr_copy((struct rte_ether_addr *)&filter->input.outer_mac,
- (struct rte_ether_addr *)&cld_filter.element.outer_mac);
- rte_ether_addr_copy((struct rte_ether_addr *)&filter->input.inner_mac,
- (struct rte_ether_addr *)&cld_filter.element.inner_mac);
- cld_filter.element.inner_vlan = filter->input.inner_vlan;
- cld_filter.element.flags = filter->input.flags;
- cld_filter.element.tenant_id = filter->input.tenant_id;
- cld_filter.element.queue_number = filter->queue;
- memcpy(cld_filter.general_fields,
- filter->input.general_fields,
- sizeof(cld_filter.general_fields));
-
- if (!filter->is_to_vf)
- vsi = pf->main_vsi;
- else {
- vf = &pf->vfs[filter->vf_id];
- vsi = vf->vsi;
- }
-
- if (((filter->input.flags & I40E_AQC_ADD_CLOUD_FILTER_0X11) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X11) ||
- ((filter->input.flags & I40E_AQC_ADD_CLOUD_FILTER_0X12) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X12) ||
- ((filter->input.flags & I40E_AQC_ADD_CLOUD_FILTER_0X10) ==
- I40E_AQC_ADD_CLOUD_FILTER_0X10))
- big_buffer = 1;
-
- if (big_buffer)
- ret = i40e_aq_rem_cloud_filters_bb(hw, vsi->seid,
- &cld_filter, 1);
- else
- ret = i40e_aq_rem_cloud_filters(hw, vsi->seid,
- &cld_filter.element, 1);
- if (ret < 0)
- return -ENOTSUP;
-
- node = i40e_sw_tunnel_filter_lookup(tunnel_rule, &filter->input);
- if (!node)
- return -EINVAL;
-
- ret = i40e_sw_tunnel_filter_del(pf, &node->input);
-
- return ret;
-}
-
static int
i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
{
@@ -2092,14 +440,6 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
if (ret != 0)
return ret;
- ret = i40e_flow_flush_tunnel_filter(pf);
- if (ret) {
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to flush tunnel flows.");
- return -rte_errno;
- }
-
ret = i40e_hash_filter_flush(pf);
if (ret)
rte_flow_error_set(error, -ret,
@@ -2108,34 +448,6 @@ i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
return ret;
}
-/* Flush all tunnel filters */
-static int
-i40e_flow_flush_tunnel_filter(struct i40e_pf *pf)
-{
- struct i40e_tunnel_filter_list
- *tunnel_list = &pf->tunnel.tunnel_list;
- struct i40e_tunnel_filter *filter;
- struct rte_flow *flow;
- void *temp;
- int ret = 0;
-
- while ((filter = TAILQ_FIRST(tunnel_list))) {
- ret = i40e_flow_destroy_tunnel_filter(pf, filter);
- if (ret)
- return ret;
- }
-
- /* Delete tunnel flows in flow list. */
- RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, temp) {
- if (flow->filter_type == RTE_ETH_FILTER_TUNNEL) {
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- rte_free(flow);
- }
- }
-
- return ret;
-}
-
static int
i40e_flow_query(struct rte_eth_dev *dev,
struct rte_flow *flow,
diff --git a/drivers/net/intel/i40e/i40e_flow.h b/drivers/net/intel/i40e/i40e_flow.h
index 6823dbef33..28e342210c 100644
--- a/drivers/net/intel/i40e/i40e_flow.h
+++ b/drivers/net/intel/i40e/i40e_flow.h
@@ -17,5 +17,11 @@ extern const struct ci_flow_engine_list i40e_flow_engine_list;
extern const struct ci_flow_engine i40e_flow_engine_ethertype;
extern const struct ci_flow_engine i40e_flow_engine_fdir;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_qinq;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_vxlan;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_nvgre;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_mpls;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_gtp;
+extern const struct ci_flow_engine i40e_flow_engine_tunnel_l4;
#endif /* _I40E_FLOW_H_ */
diff --git a/drivers/net/intel/i40e/i40e_flow_tunnel.c b/drivers/net/intel/i40e/i40e_flow_tunnel.c
new file mode 100644
index 0000000000..aae64af398
--- /dev/null
+++ b/drivers/net/intel/i40e/i40e_flow_tunnel.c
@@ -0,0 +1,1590 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include "i40e_ethdev.h"
+#include "i40e_flow.h"
+
+#include "../common/flow_engine.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+
+struct i40e_tunnel_priv {
+ struct i40e_tunnel_state *state;
+};
+
+struct i40e_tunnel_ctx {
+ struct ci_flow_engine_ctx base;
+ struct i40e_tunnel_filter_conf filter;
+};
+
+struct i40e_tunnel_flow {
+ struct rte_flow base;
+ struct i40e_tunnel_filter_conf filter;
+ struct i40e_tunnel_filter_match_key match_key;
+};
+
+static int
+i40e_check_tunnel_filter_type(uint8_t filter_type)
+{
+ const uint16_t i40e_supported_tunnel_filter_types[] = {
+ RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_TENID |
+ RTE_ETH_TUNNEL_FILTER_IVLAN,
+ RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_IVLAN,
+ RTE_ETH_TUNNEL_FILTER_IMAC | RTE_ETH_TUNNEL_FILTER_TENID,
+ RTE_ETH_TUNNEL_FILTER_OMAC | RTE_ETH_TUNNEL_FILTER_TENID |
+ RTE_ETH_TUNNEL_FILTER_IMAC,
+ RTE_ETH_TUNNEL_FILTER_IMAC,
+ };
+ uint8_t i;
+
+ for (i = 0; i < RTE_DIM(i40e_supported_tunnel_filter_types); i++) {
+ if (filter_type == i40e_supported_tunnel_filter_types[i])
+ return 0;
+ }
+ return -1;
+}
+
+/**
+ * QinQ tunnel filter graph implementation
+ * Pattern: START -> ETH -> OUTER_VLAN -> INNER_VLAN -> END
+ */
+enum i40e_tunnel_qinq_node_id {
+ I40E_TUNNEL_QINQ_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_QINQ_NODE_ETH,
+ I40E_TUNNEL_QINQ_NODE_OUTER_VLAN,
+ I40E_TUNNEL_QINQ_NODE_INNER_VLAN,
+ I40E_TUNNEL_QINQ_NODE_END,
+ I40E_TUNNEL_QINQ_NODE_MAX,
+};
+
+static int
+i40e_tunnel_node_vlan_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+
+ /* matching eth proto not supported */
+ if (vlan_mask->hdr.eth_proto) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid VLAN mask");
+ }
+
+ /* VLAN TCI must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&vlan_mask->hdr.vlan_tci)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid VLAN mask");
+ }
+
+ return 0;
+}
+
+/* common VLAN processing for both outer and inner VLAN nodes */
+static int
+i40e_tunnel_node_vlan_process(struct i40e_tunnel_ctx *tunnel_ctx,
+ const struct rte_flow_item *item, bool is_inner)
+{
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ /* Store the VLAN ID and set filter flag */
+ if (is_inner) {
+ tunnel_filter->inner_vlan = rte_be_to_cpu_16(vlan_spec->hdr.vlan_tci);
+ tunnel_filter->filter_type |= RTE_ETH_TUNNEL_FILTER_IVLAN;
+ } else {
+ tunnel_filter->outer_vlan = rte_be_to_cpu_16(vlan_spec->hdr.vlan_tci);
+ /* no special flag for outer VLAN matching */
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_outer_vlan_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+
+ return i40e_tunnel_node_vlan_process(tunnel_ctx, item, false);
+}
+
+static int
+i40e_tunnel_node_inner_vlan_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+
+ return i40e_tunnel_node_vlan_process(tunnel_ctx, item, true);
+}
+
+static int
+i40e_tunnel_qinq_node_end_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_QINQ;
+
+ /* QinQ filter is not meant to set this flag */
+ tunnel_filter->filter_type &= ~RTE_ETH_TUNNEL_FILTER_IVLAN;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_qinq_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_QINQ_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_QINQ_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_QINQ_NODE_OUTER_VLAN] = {
+ .name = "OUTER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_vlan_validate,
+ .process = i40e_tunnel_node_outer_vlan_process,
+ },
+ [I40E_TUNNEL_QINQ_NODE_INNER_VLAN] = {
+ .name = "INNER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_vlan_validate,
+ .process = i40e_tunnel_node_inner_vlan_process,
+ },
+ [I40E_TUNNEL_QINQ_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .process = i40e_tunnel_qinq_node_end_process,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_QINQ_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_QINQ_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_QINQ_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_QINQ_NODE_OUTER_VLAN,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_QINQ_NODE_OUTER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_QINQ_NODE_INNER_VLAN,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_QINQ_NODE_INNER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_QINQ_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * VXLAN tunnel filter graph implementation
+ * Pattern: START -> ETH -> (IPv4 | IPv6) -> UDP -> VXLAN -> ETH -> [VLAN] -> END
+ */
+enum i40e_tunnel_vxlan_node_id {
+ I40E_TUNNEL_VXLAN_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_VXLAN_NODE_OUTER_ETH,
+ I40E_TUNNEL_VXLAN_NODE_IPV4,
+ I40E_TUNNEL_VXLAN_NODE_IPV6,
+ I40E_TUNNEL_VXLAN_NODE_UDP,
+ I40E_TUNNEL_VXLAN_NODE_VXLAN,
+ I40E_TUNNEL_VXLAN_NODE_INNER_ETH,
+ I40E_TUNNEL_VXLAN_NODE_INNER_VLAN,
+ I40E_TUNNEL_VXLAN_NODE_END,
+ I40E_TUNNEL_VXLAN_NODE_MAX,
+};
+
+static int
+i40e_tunnel_node_eth_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+
+ /* spec/mask is optional */
+ if (eth_spec == NULL && eth_mask == NULL)
+ return 0;
+
+ /* matching eth type not supported */
+ if (eth_mask->hdr.ether_type) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid ETH mask");
+ }
+
+ /* source MAC must be fully unmasked */
+ if (!CI_FIELD_IS_ZERO(ð_mask->hdr.src_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid ETH mask");
+ }
+ /* destination MAC must be fully masked */
+ if (!CI_FIELD_IS_MASKED(ð_mask->hdr.dst_addr)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid ETH mask");
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_eth_process(struct i40e_tunnel_ctx *tunnel_ctx,
+ const struct rte_flow_item *item, bool is_inner)
+{
+ const struct rte_flow_item_eth *eth_spec = item->spec;
+ const struct rte_flow_item_eth *eth_mask = item->mask;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ /* eth spec/mask is optional */
+ if (eth_spec == NULL && eth_mask == NULL)
+ return 0;
+
+ /* Store the MAC addresses and set filter flags */
+ if (is_inner) {
+ memcpy(&tunnel_filter->inner_mac, ð_spec->hdr.dst_addr,
+ sizeof(tunnel_filter->inner_mac));
+ tunnel_filter->filter_type |= RTE_ETH_TUNNEL_FILTER_IMAC;
+ } else {
+ memcpy(&tunnel_filter->outer_mac, ð_spec->hdr.dst_addr,
+ sizeof(tunnel_filter->outer_mac));
+ tunnel_filter->filter_type |= RTE_ETH_TUNNEL_FILTER_OMAC;
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_outer_eth_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+
+ return i40e_tunnel_eth_process(tunnel_ctx, item, false);
+}
+
+static int
+i40e_tunnel_node_inner_eth_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+
+ return i40e_tunnel_eth_process(tunnel_ctx, item, true);
+}
+
+static int
+i40e_tunnel_node_ipv4_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->ip_type = I40E_TUNNEL_IPTYPE_IPV4;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_ipv6_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->ip_type = I40E_TUNNEL_IPTYPE_IPV6;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_vxlan_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vxlan *vxlan_spec = item->spec;
+ const struct rte_flow_item_vxlan *vxlan_mask = item->mask;
+
+ /* spec/mask are optional */
+ if (vxlan_spec == NULL && vxlan_mask == NULL)
+ return 0;
+
+ /* VNI must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&vxlan_mask->hdr.vni)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid VXLAN mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_vxlan_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_vxlan *vxlan_spec = item->spec;
+ const struct rte_flow_item_vxlan *vxlan_mask = item->mask;
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ /* spec/mask are optional */
+ if (vxlan_spec == NULL && vxlan_mask == NULL)
+ return 0;
+
+ /* Store the VNI and set filter flag */
+ tunnel_filter->tenant_id = ci_be24_to_cpu(vxlan_spec->hdr.vni);
+ tunnel_filter->filter_type |= RTE_ETH_TUNNEL_FILTER_TENID;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_end_validate(const void *ctx,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ const struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ /* this shouldn't happen but check this just in case */
+ if (i40e_check_tunnel_filter_type(tunnel_filter->filter_type) != 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid tunnel filter configuration");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_vxlan_node_end_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_VXLAN;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_vxlan_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_VXLAN_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_VXLAN_NODE_OUTER_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_eth_validate,
+ .process = i40e_tunnel_node_outer_eth_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv4_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv6_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_VXLAN] = {
+ .name = "VXLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VXLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_vxlan_validate,
+ .process = i40e_tunnel_node_vxlan_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_INNER_ETH] = {
+ .name = "INNER_ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_eth_validate,
+ .process = i40e_tunnel_node_inner_eth_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_INNER_VLAN] = {
+ .name = "INNER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_vlan_validate,
+ .process = i40e_tunnel_node_inner_vlan_process,
+ },
+ [I40E_TUNNEL_VXLAN_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .validate = i40e_tunnel_node_end_validate,
+ .process = i40e_tunnel_vxlan_node_end_process
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_VXLAN_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_OUTER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_OUTER_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_IPV4,
+ I40E_TUNNEL_VXLAN_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_UDP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_UDP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_VXLAN,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_VXLAN] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_INNER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_INNER_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_INNER_VLAN,
+ I40E_TUNNEL_VXLAN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_VXLAN_NODE_INNER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_VXLAN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * NVGRE tunnel filter graph implementation
+ * Pattern: START -> ETH -> (IPv4 | IPv6) -> NVGRE -> ETH -> [VLAN] -> END
+ */
+enum i40e_tunnel_nvgre_node_id {
+ I40E_TUNNEL_NVGRE_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_NVGRE_NODE_OUTER_ETH,
+ I40E_TUNNEL_NVGRE_NODE_IPV4,
+ I40E_TUNNEL_NVGRE_NODE_IPV6,
+ I40E_TUNNEL_NVGRE_NODE_NVGRE,
+ I40E_TUNNEL_NVGRE_NODE_INNER_ETH,
+ I40E_TUNNEL_NVGRE_NODE_INNER_VLAN,
+ I40E_TUNNEL_NVGRE_NODE_END,
+ I40E_TUNNEL_NVGRE_NODE_MAX,
+};
+
+static int
+i40e_tunnel_node_nvgre_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_nvgre *nvgre_spec = item->spec;
+ const struct rte_flow_item_nvgre *nvgre_mask = item->mask;
+
+ /* spec/mask are optional */
+ if (nvgre_spec == NULL && nvgre_mask == NULL)
+ return 0;
+
+ /* TNI must be fully masked */
+ if (!CI_FIELD_IS_MASKED(&nvgre_mask->tni)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK, item,
+ "Invalid NVGRE mask");
+ }
+ /* protocol must either be unmasked or fully masked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&nvgre_mask->protocol)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK, item,
+ "Invalid NVGRE mask");
+ }
+ /* reserved/version field must either be unmasked or fully masked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&nvgre_mask->c_k_s_rsvd0_ver)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK, item,
+ "Invalid NVGRE mask");
+ }
+ /* if reserved/version field is masked, it must be set to 0x2000 */
+ if (nvgre_mask->c_k_s_rsvd0_ver &&
+ nvgre_spec->c_k_s_rsvd0_ver != rte_cpu_to_be_16(0x2000)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK, item,
+ "Invalid NVGRE spec");
+ }
+ /* if protocol field is masked, it must be set to 0x6558 */
+ if (nvgre_mask->protocol &&
+ nvgre_spec->protocol != rte_cpu_to_be_16(0x6558)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK, item,
+ "Invalid NVGRE spec");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_nvgre_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_nvgre *nvgre_spec = item->spec;
+ const struct rte_flow_item_nvgre *nvgre_mask = item->mask;
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ /* spec/mask are optional */
+ if (nvgre_spec == NULL && nvgre_mask == NULL)
+ return 0;
+
+ /* Store the VNI and set filter flag */
+ tunnel_filter->tenant_id = ci_be24_to_cpu(nvgre_spec->tni);
+ tunnel_filter->filter_type |= RTE_ETH_TUNNEL_FILTER_TENID;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_nvgre_end_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_NVGRE;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_nvgre_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_NVGRE_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_NVGRE_NODE_OUTER_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_eth_validate,
+ .process = i40e_tunnel_node_outer_eth_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv4_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv6_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_NVGRE] = {
+ .name = "NVGRE",
+ .type = RTE_FLOW_ITEM_TYPE_NVGRE,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_nvgre_validate,
+ .process = i40e_tunnel_node_nvgre_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_INNER_ETH] = {
+ .name = "INNER_ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY |
+ FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_eth_validate,
+ .process = i40e_tunnel_node_inner_eth_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_INNER_VLAN] = {
+ .name = "INNER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_vlan_validate,
+ .process = i40e_tunnel_node_inner_vlan_process,
+ },
+ [I40E_TUNNEL_NVGRE_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .validate = i40e_tunnel_node_end_validate,
+ .process = i40e_tunnel_node_nvgre_end_process
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_NVGRE_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_OUTER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_OUTER_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_IPV4,
+ I40E_TUNNEL_NVGRE_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_NVGRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_NVGRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_NVGRE] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_INNER_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_INNER_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_INNER_VLAN,
+ I40E_TUNNEL_NVGRE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_NVGRE_NODE_INNER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_NVGRE_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * MPLS tunnel filter graph implementation
+ * Pattern: START -> ETH -> (IPv4 | IPv6) -> (UDP | GRE) -> MPLS -> END
+ */
+enum i40e_tunnel_mpls_node_id {
+ I40E_TUNNEL_MPLS_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_MPLS_NODE_ETH,
+ I40E_TUNNEL_MPLS_NODE_IPV4,
+ I40E_TUNNEL_MPLS_NODE_IPV6,
+ I40E_TUNNEL_MPLS_NODE_UDP,
+ I40E_TUNNEL_MPLS_NODE_GRE,
+ I40E_TUNNEL_MPLS_NODE_MPLS,
+ I40E_TUNNEL_MPLS_NODE_END,
+ I40E_TUNNEL_MPLS_NODE_MAX,
+};
+
+static int
+i40e_tunnel_mpls_node_udp_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_MPLSoUDP;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_mpls_node_gre_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_MPLSoGRE;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_mpls_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_mpls *mpls_mask = item->mask;
+ const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
+
+ /* MPLS label and TC must be fully masked */
+ if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid MPLS mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_mpls_process(void *ctx, const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct rte_flow_item_mpls *mpls_spec = item->spec;
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ tunnel_filter->tenant_id = ci_be24_to_cpu(mpls_spec->label_tc_s) >> 4;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_mpls_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_MPLS_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_MPLS_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_MPLS_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv4_process,
+ },
+ [I40E_TUNNEL_MPLS_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv6_process,
+ },
+ [I40E_TUNNEL_MPLS_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_mpls_node_udp_process,
+ },
+ [I40E_TUNNEL_MPLS_NODE_GRE] = {
+ .name = "GRE",
+ .type = RTE_FLOW_ITEM_TYPE_GRE,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_mpls_node_gre_process,
+ },
+ [I40E_TUNNEL_MPLS_NODE_MPLS] = {
+ .name = "MPLS",
+ .type = RTE_FLOW_ITEM_TYPE_MPLS,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_mpls_validate,
+ .process = i40e_tunnel_node_mpls_process,
+ },
+ [I40E_TUNNEL_MPLS_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_MPLS_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_IPV4,
+ I40E_TUNNEL_MPLS_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_UDP,
+ I40E_TUNNEL_MPLS_NODE_GRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_UDP,
+ I40E_TUNNEL_MPLS_NODE_GRE,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_MPLS,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_GRE] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_MPLS,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_MPLS_NODE_MPLS] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_MPLS_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * GTP tunnel filter graph implementation
+ * Pattern: START -> ETH -> (IPv4 | IPv6) -> UDP -> (GTPC | GTPU) -> END
+ */
+enum i40e_tunnel_gtp_node_id {
+ I40E_TUNNEL_GTP_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_GTP_NODE_ETH,
+ I40E_TUNNEL_GTP_NODE_IPV4,
+ I40E_TUNNEL_GTP_NODE_IPV6,
+ I40E_TUNNEL_GTP_NODE_UDP,
+ I40E_TUNNEL_GTP_NODE_GTPC,
+ I40E_TUNNEL_GTP_NODE_GTPU,
+ I40E_TUNNEL_GTP_NODE_END,
+ I40E_TUNNEL_GTP_NODE_MAX,
+};
+
+static int
+i40e_tunnel_node_gtp_validate(const void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_gtp *gtp_mask = item->mask;
+ const struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ const struct rte_eth_dev_data *dev_data = tunnel_ctx->base.dev_data;
+ const struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+
+ /* does HW support GTP? */
+ if (!pf->gtp_support) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "GTP not supported");
+ }
+
+ /* reject unsupported fields */
+ if (gtp_mask->hdr.gtp_hdr_info ||
+ gtp_mask->hdr.msg_type ||
+ gtp_mask->hdr.plen) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid GTP mask");
+ }
+
+ /* teid must be fully masked */
+ if (!CI_FIELD_IS_MASKED(>p_mask->hdr.teid)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid GTP mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_gtp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_gtp *gtp_spec = item->spec;
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+
+ if (item->type == RTE_FLOW_ITEM_TYPE_GTPC)
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_GTPC;
+ else if (item->type == RTE_FLOW_ITEM_TYPE_GTPU)
+ tunnel_filter->tunnel_type = I40E_TUNNEL_TYPE_GTPU;
+ else {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid GTP item type");
+ }
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(gtp_spec->hdr.teid);
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_gtp_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_GTP_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_GTP_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_GTP_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv4_process,
+ },
+ [I40E_TUNNEL_GTP_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv6_process,
+ },
+ [I40E_TUNNEL_GTP_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_GTP_NODE_GTPC] = {
+ .name = "GTPC",
+ .type = RTE_FLOW_ITEM_TYPE_GTPC,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_gtp_validate,
+ .process = i40e_tunnel_node_gtp_process,
+ },
+ [I40E_TUNNEL_GTP_NODE_GTPU] = {
+ .name = "GTPU",
+ .type = RTE_FLOW_ITEM_TYPE_GTPU,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_gtp_validate,
+ .process = i40e_tunnel_node_gtp_process,
+ },
+ [I40E_TUNNEL_GTP_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_GTP_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_IPV4,
+ I40E_TUNNEL_GTP_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_UDP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_UDP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_GTPC,
+ I40E_TUNNEL_GTP_NODE_GTPU,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_GTPC] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_GTP_NODE_GTPU] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_GTP_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/**
+ * L4 tunnel filter graph implementation
+ * Pattern: START -> ETH -> (IPv4 | IPv6) -> (TCP | UDP | SCTP) -> END
+ */
+enum i40e_tunnel_l4_node_id {
+ I40E_TUNNEL_L4_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_TUNNEL_L4_NODE_ETH,
+ I40E_TUNNEL_L4_NODE_IPV4,
+ I40E_TUNNEL_L4_NODE_IPV6,
+ I40E_TUNNEL_L4_NODE_TCP,
+ I40E_TUNNEL_L4_NODE_UDP,
+ I40E_TUNNEL_L4_NODE_SCTP,
+ I40E_TUNNEL_L4_NODE_END,
+ I40E_TUNNEL_L4_NODE_MAX,
+};
+
+static int
+i40e_tunnel_node_tcp_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+
+ /* only source/destination ports are supported */
+ if (tcp_mask->hdr.sent_seq ||
+ tcp_mask->hdr.recv_ack ||
+ tcp_mask->hdr.data_off ||
+ tcp_mask->hdr.tcp_flags ||
+ tcp_mask->hdr.rx_win ||
+ tcp_mask->hdr.cksum ||
+ tcp_mask->hdr.tcp_urp) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid TCP mask");
+ }
+
+ /* src/dst ports have to be fully masked or fully unmasked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&tcp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid TCP mask");
+ }
+ /* there can be only one! */
+ if (tcp_mask->hdr.src_port && tcp_mask->hdr.dst_port) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid TCP mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_tcp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+ const struct rte_flow_item_tcp *tcp_spec = item->spec;
+ const struct rte_flow_item_tcp *tcp_mask = item->mask;
+
+ if (tcp_mask->hdr.src_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_SRC;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(tcp_spec->hdr.src_port);
+ } else if (tcp_mask->hdr.dst_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_DST;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(tcp_spec->hdr.dst_port);
+ }
+ tunnel_filter->tunnel_type = I40E_CLOUD_TYPE_TCP;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_udp_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+
+ /* only source/destination ports are supported */
+ if (udp_mask->hdr.dgram_len ||
+ udp_mask->hdr.dgram_cksum) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid UDP mask");
+ }
+
+ /* src/dst ports have to be fully masked or fully unmasked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&udp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid UDP mask");
+ }
+ /* there can be only one! */
+ if (udp_mask->hdr.src_port && udp_mask->hdr.dst_port) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid UDP mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_udp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+ const struct rte_flow_item_udp *udp_spec = item->spec;
+ const struct rte_flow_item_udp *udp_mask = item->mask;
+
+ if (udp_mask->hdr.src_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_SRC;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(udp_spec->hdr.src_port);
+ } else if (udp_mask->hdr.dst_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_DST;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(udp_spec->hdr.dst_port);
+ }
+ tunnel_filter->tunnel_type = I40E_CLOUD_TYPE_UDP;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_node_sctp_validate(const void *ctx __rte_unused,
+ const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+
+ /* only source/destination ports are supported */
+ if (sctp_mask->hdr.cksum || sctp_mask->hdr.tag) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid SCTP mask");
+ }
+
+ /* src/dst ports have to be fully masked or fully unmasked */
+ if (!CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.src_port) ||
+ !CI_FIELD_IS_ZERO_OR_MASKED(&sctp_mask->hdr.dst_port)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid SCTP mask");
+ }
+ /* there can be only one! */
+ if (sctp_mask->hdr.src_port && sctp_mask->hdr.dst_port) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid SCTP mask");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_node_sctp_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = ctx;
+ struct i40e_tunnel_filter_conf *tunnel_filter = &tunnel_ctx->filter;
+ const struct rte_flow_item_sctp *sctp_spec = item->spec;
+ const struct rte_flow_item_sctp *sctp_mask = item->mask;
+
+ if (sctp_mask->hdr.src_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_SRC;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(sctp_spec->hdr.src_port);
+ } else if (sctp_mask->hdr.dst_port) {
+ tunnel_filter->l4_port_type = I40E_L4_PORT_TYPE_DST;
+ tunnel_filter->tenant_id = rte_be_to_cpu_32(sctp_spec->hdr.dst_port);
+ }
+ tunnel_filter->tunnel_type = I40E_CLOUD_TYPE_SCTP;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_tunnel_l4_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_TUNNEL_L4_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_TUNNEL_L4_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_TUNNEL_L4_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv4_process,
+ },
+ [I40E_TUNNEL_L4_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_tunnel_node_ipv6_process,
+ },
+ [I40E_TUNNEL_L4_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_tcp_validate,
+ .process = i40e_tunnel_node_tcp_process,
+ },
+ [I40E_TUNNEL_L4_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_udp_validate,
+ .process = i40e_tunnel_node_udp_process,
+ },
+ [I40E_TUNNEL_L4_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_tunnel_node_sctp_validate,
+ .process = i40e_tunnel_node_sctp_process,
+ },
+ [I40E_TUNNEL_L4_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_TUNNEL_L4_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_IPV4,
+ I40E_TUNNEL_L4_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_TCP,
+ I40E_TUNNEL_L4_NODE_UDP,
+ I40E_TUNNEL_L4_NODE_SCTP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_TCP,
+ I40E_TUNNEL_L4_NODE_UDP,
+ I40E_TUNNEL_L4_NODE_SCTP,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_TCP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_TUNNEL_L4_NODE_SCTP] = {
+ .next = (size_t[]) {
+ I40E_TUNNEL_L4_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static int
+i40e_tunnel_action_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(param->driver_ctx);
+ const struct rte_flow_action *first, *second;
+ const struct rte_flow_action_queue *act_q;
+ bool is_to_vf = false;
+
+ first = actions->actions[0];
+ /* can be NULL */
+ second = actions->actions[1];
+
+ /* first action must be PF or VF */
+ if (first->type == RTE_FLOW_ACTION_TYPE_VF) {
+ const struct rte_flow_action_vf *vf = first->conf;
+ if (vf->id >= pf->vf_num) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, first,
+ "Invalid VF ID for tunnel filter");
+ }
+ is_to_vf = true;
+ } else if (first->type != RTE_FLOW_ACTION_TYPE_PF) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, first,
+ "Unsupported action");
+ }
+
+ /* check if second action is QUEUE */
+ if (second == NULL)
+ return 0;
+
+ if (second->type != RTE_FLOW_ACTION_TYPE_QUEUE) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, second,
+ "Unsupported action");
+ }
+
+ act_q = second->conf;
+ /* check queue ID for PF flow */
+ if (!is_to_vf && act_q->index >= pf->dev_data->nb_rx_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, act_q,
+ "Invalid queue ID for tunnel filter");
+ }
+ /* check queue ID for VF flow */
+ if (is_to_vf && act_q->index >= pf->vf_nb_qps) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, act_q,
+ "Invalid queue ID for tunnel filter");
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_ctx_init(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_tunnel_ctx *tunnel_ctx = (struct i40e_tunnel_ctx *)ctx;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param ac_param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_PF,
+ RTE_FLOW_ACTION_TYPE_VF,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 2,
+ .check = i40e_tunnel_action_check,
+ .driver_ctx = ctx->dev_data->dev_private,
+ };
+ const struct rte_flow_action *first, *second;
+ const struct rte_flow_action_queue *act_q;
+ int ret;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret)
+ return ret;
+
+ ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
+ if (ret)
+ return ret;
+
+ first = parsed_actions.actions[0];
+ /* can be NULL */
+ second = parsed_actions.actions[1];
+
+ if (first->type == RTE_FLOW_ACTION_TYPE_VF) {
+ const struct rte_flow_action_vf *vf = first->conf;
+ tunnel_ctx->filter.vf_id = vf->id;
+ tunnel_ctx->filter.is_to_vf = 1;
+ } else if (first->type == RTE_FLOW_ACTION_TYPE_PF) {
+ tunnel_ctx->filter.is_to_vf = 0;
+ }
+
+ /* check if second action is QUEUE */
+ if (second == NULL)
+ return 0;
+
+ act_q = second->conf;
+ tunnel_ctx->filter.queue_id = act_q->index;
+
+ return 0;
+}
+
+static int
+i40e_tunnel_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ const struct i40e_tunnel_ctx *tunnel_ctx = (const struct i40e_tunnel_ctx *)ctx;
+ struct i40e_tunnel_flow *tunnel_flow = (struct i40e_tunnel_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ int ret;
+
+ /* copy filter configuration from context to flow */
+ tunnel_flow->filter = tunnel_ctx->filter;
+
+ /* compute hash input */
+ ret = i40e_tunnel_filter_match_key_get(pf, &tunnel_flow->filter,
+ &tunnel_flow->match_key);
+ if (ret != 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to build tunnel filter input");
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_tunnel_flow *tunnel_flow = (struct i40e_tunnel_flow *)flow;
+ struct i40e_tunnel_priv *priv = flow->engine_priv;
+ struct i40e_tunnel_state *state = priv->state;
+ int ret;
+
+ if (rte_hash_lookup(state->hash_table, &tunnel_flow->match_key) >= 0) {
+ return rte_flow_error_set(error, EEXIST,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Conflict with existing tunnel rule");
+ }
+
+ ret = rte_hash_add_key(state->hash_table, &tunnel_flow->match_key);
+ if (ret < 0) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Tunnel filter table is full");
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_tunnel_flow *tunnel_flow = (struct i40e_tunnel_flow *)flow;
+ struct i40e_tunnel_priv *priv = flow->engine_priv;
+ struct i40e_tunnel_state *state = priv->state;
+
+ if (rte_hash_del_key(state->hash_table, &tunnel_flow->match_key) < 0) {
+ return rte_flow_error_set(error, ENOENT,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Tunnel filter is missing from the filter table");
+ }
+
+ return 0;
+}
+
+static int
+i40e_tunnel_flow_install(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ struct i40e_tunnel_flow *tunnel_flow = (struct i40e_tunnel_flow *)flow;
+ int ret;
+
+ ret = i40e_tunnel_filter_program(pf, &tunnel_flow->filter, 1);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to install tunnel filter");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_flow_uninstall(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ struct i40e_tunnel_flow *tunnel_flow = (struct i40e_tunnel_flow *)flow;
+ int ret;
+
+ ret = i40e_tunnel_filter_program(pf, &tunnel_flow->filter, 0);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to uninstall tunnel filter");
+ }
+ return 0;
+}
+
+static int
+i40e_tunnel_flow_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv)
+{
+ struct i40e_tunnel_priv *tunnel_priv = priv;
+
+ tunnel_priv->state = i40e_tunnel_state_attach(dev_data);
+ return tunnel_priv->state == NULL ? -ENOMEM : 0;
+}
+
+static void
+i40e_tunnel_flow_engine_uninit(const struct ci_flow_engine *engine __rte_unused,
+ void *priv)
+{
+ struct i40e_tunnel_priv *tunnel_priv = priv;
+
+ i40e_tunnel_state_detach(tunnel_priv->state);
+}
+
+static const struct ci_flow_engine_ops i40e_flow_engine_tunnel_ops = {
+ .engine_init = i40e_tunnel_flow_engine_init,
+ .engine_uninit = i40e_tunnel_flow_engine_uninit,
+ .ctx_init = i40e_tunnel_ctx_init,
+ .ctx_to_flow = i40e_tunnel_ctx_to_flow,
+ .flow_register = i40e_tunnel_flow_register,
+ .flow_unregister = i40e_tunnel_flow_unregister,
+ .flow_install = i40e_tunnel_flow_install,
+ .flow_uninstall = i40e_tunnel_flow_uninstall,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_nvgre = {
+ .name = "tunnel_nvgre",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_nvgre_graph,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_vxlan = {
+ .name = "tunnel_vxlan",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_vxlan_graph,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_mpls = {
+ .name = "tunnel_mpls",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_mpls_graph,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_gtp = {
+ .name = "tunnel_gtp",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_gtp_graph,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_l4 = {
+ .name = "tunnel_l4",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_l4_graph,
+};
+
+const struct ci_flow_engine i40e_flow_engine_tunnel_qinq = {
+ .name = "tunnel_qinq",
+ .ops = &i40e_flow_engine_tunnel_ops,
+ .ctx_size = sizeof(struct i40e_tunnel_ctx),
+ .flow_size = sizeof(struct i40e_tunnel_flow),
+ .priv_size = sizeof(struct i40e_tunnel_priv),
+ .graph = &i40e_tunnel_qinq_graph,
+};
diff --git a/drivers/net/intel/i40e/meson.build b/drivers/net/intel/i40e/meson.build
index c07257cb80..0db60c1e99 100644
--- a/drivers/net/intel/i40e/meson.build
+++ b/drivers/net/intel/i40e/meson.build
@@ -35,6 +35,7 @@ sources += files(
'i40e_flow.c',
'i40e_flow_ethertype.c',
'i40e_flow_fdir.c',
+ 'i40e_flow_tunnel.c',
'i40e_tm.c',
'i40e_hash.c',
'i40e_vf_representor.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 18/19] net/i40e: reimplement hash parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (16 preceding siblings ...)
2026-09-08 15:20 ` [PATCH v2 17/19] net/i40e: reimplement tunnel parsers Anatoly Burakov
@ 2026-09-08 15:21 ` Anatoly Burakov
2026-09-08 15:21 ` [PATCH v2 19/19] net/i40e: advertise flow keep capability Anatoly Burakov
2026-09-09 9:08 ` [PATCH v2 00/19] Building a better rte_flow parser Burakov, Anatoly
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:21 UTC (permalink / raw)
To: dev, Bruce Richardson
Use the new flow graph API and the common parsing framework to implement
flow parser for RSS.
The RSS parser was bypassing the other generic infrastructure, probably
because it was very convoluted and did not map onto that model very well.
It has now been made a first-class citizen.
The hash parser is multiple engines that share state, so each engine will
attach to the shared refcounted RSS configuration state.
Additionally, this is the final engine in i40e, so remaining legacy flow
infrastructure has been removed.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 34 +-
drivers/net/intel/i40e/i40e_ethdev.h | 34 +-
drivers/net/intel/i40e/i40e_flow.c | 289 +----
drivers/net/intel/i40e/i40e_flow.h | 3 +
drivers/net/intel/i40e/i40e_flow_hash.c | 1441 +++++++++++++++++++++++
drivers/net/intel/i40e/i40e_hash.c | 1208 +------------------
drivers/net/intel/i40e/i40e_hash.h | 19 +-
drivers/net/intel/i40e/meson.build | 1 +
8 files changed, 1549 insertions(+), 1480 deletions(-)
create mode 100644 drivers/net/intel/i40e/i40e_flow_hash.c
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 97211b5994..8c35a64780 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -1027,6 +1027,27 @@ i40e_tunnel_state_detach(struct i40e_tunnel_state *state)
*state = (struct i40e_tunnel_state){0};
}
+struct i40e_rss_state *
+i40e_rss_state_attach(struct rte_eth_dev_data *dev_data)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ struct i40e_rss_state *state = &pf->rss_state;
+
+ if (state->refcnt == 0)
+ TAILQ_INIT(&state->list);
+
+ state->refcnt++;
+ return state;
+}
+
+void
+i40e_rss_state_detach(struct i40e_rss_state *state)
+{
+ if (--state->refcnt > 0)
+ return;
+
+ *state = (struct i40e_rss_state){0};
+}
static void
i40e_init_customized_info(struct i40e_pf *pf)
{
@@ -1630,9 +1651,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
*/
i40e_add_tx_flow_control_drop_filter(pf);
- /* initialize RSS rule list */
- TAILQ_INIT(&pf->rss_config_list);
-
/* initialize Traffic Manager configuration */
i40e_tm_conf_init(dev);
@@ -1781,8 +1799,6 @@ i40e_dev_configure(struct rte_eth_dev *dev)
}
}
- TAILQ_INIT(&pf->flow_list);
-
return 0;
err_dcb:
@@ -2450,7 +2466,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
struct rte_pci_device *pci_dev = RTE_CLASS_TO_BUS_DEVICE(dev, *pci_dev);
struct rte_intr_handle *intr_handle = pci_dev->intr_handle;
struct i40e_filter_control_settings settings;
- struct rte_flow *p_flow;
uint32_t reg;
int i;
int ret;
@@ -2541,12 +2556,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
i40e_msec_delay(500);
} while (retries++ < 5);
- /* Remove all flows */
- while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
- TAILQ_REMOVE(&pf->flow_list, p_flow, node);
- rte_free(p_flow);
- }
-
/* release the fdir static allocated memory */
i40e_fdir_memory_cleanup(pf);
@@ -11216,7 +11225,6 @@ static void
i40e_filter_restore(struct i40e_pf *pf)
{
i40e_fdir_filter_restore(pf);
- (void)i40e_hash_filter_restore(pf);
}
bool
diff --git a/drivers/net/intel/i40e/i40e_ethdev.h b/drivers/net/intel/i40e/i40e_ethdev.h
index 9d68d8fd0f..782b85e19c 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.h
+++ b/drivers/net/intel/i40e/i40e_ethdev.h
@@ -281,9 +281,6 @@ enum i40e_flxpld_layer_idx {
*/
struct rte_flow {
struct ci_flow base;
- TAILQ_ENTRY(rte_flow) node;
- enum rte_filter_type filter_type;
- void *rule;
};
/**
@@ -953,8 +950,6 @@ struct i40e_tunnel_filter_conf {
uint16_t vf_id; /**< VF id, available when is_to_vf is 1. */
};
-TAILQ_HEAD(i40e_flow_list, rte_flow);
-
/* Struct to store Traffic Manager shaper profile. */
struct i40e_tm_shaper_profile {
TAILQ_ENTRY(i40e_tm_shaper_profile) node;
@@ -1076,14 +1071,23 @@ struct i40e_rss_filter_data {
uint64_t reset_symmetric_pctypes;
};
-/* RSS filter list structure */
-struct i40e_rss_filter {
- TAILQ_ENTRY(i40e_rss_filter) next;
- struct i40e_rte_flow_rss_conf rss_filter_info;
+/*
+ * Tracking entry for a registered RSS flow: points back to the flow that
+ * owns the rss_conf, plus the metadata recording which HW state it owns.
+ */
+struct i40e_rss_flow_node {
+ TAILQ_ENTRY(i40e_rss_flow_node) next;
+ struct ci_flow *flow;
struct i40e_rss_filter_data filter_data;
};
-TAILQ_HEAD(i40e_rss_conf_list, i40e_rss_filter);
+TAILQ_HEAD(i40e_rss_node_list, i40e_rss_flow_node);
+
+/* RSS flow engine shared state, attached/detached by the RSS engines */
+struct i40e_rss_state {
+ struct i40e_rss_node_list list;
+ uint16_t refcnt;
+};
struct i40e_vf_msg_cfg {
/* maximal VF message during a statistic period */
@@ -1160,13 +1164,12 @@ struct i40e_pf {
struct i40e_fdir_info fdir; /* flow director info */
struct i40e_tunnel_state tunnel_state; /* tunnel flow engine state */
- struct i40e_rss_conf_list rss_config_list; /* RSS rule list */
+ struct i40e_rss_state rss_state; /* RSS flow engine state */
struct i40e_queue_regions queue_region; /* queue region info */
struct i40e_fc_conf fc_conf; /* Flow control conf */
bool floating_veb; /* The flag to use the floating VEB */
/* The floating enable flag for the specific VF */
bool floating_veb_list[I40E_MAX_VF];
- struct i40e_flow_list flow_list;
/* flow engine configuration */
struct ci_flow_engine_conf flow_engine_conf;
bool mpls_replace_flag; /* 1 - MPLS filter replace is done */
@@ -1305,11 +1308,6 @@ struct i40e_vf_representor {
extern const struct rte_flow_ops i40e_flow_ops;
-struct i40e_filter_ctx {
- struct i40e_rte_flow_rss_conf rss_conf;
- enum rte_filter_type type;
-};
-
int i40e_dev_switch_queues(struct i40e_pf *pf, bool on);
int i40e_vsi_release(struct i40e_vsi *vsi);
struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf,
@@ -1409,6 +1407,8 @@ int i40e_tunnel_filter_match_key_get(struct i40e_pf *pf,
struct i40e_tunnel_filter_match_key *out);
struct i40e_tunnel_state *i40e_tunnel_state_attach(struct rte_eth_dev_data *dev_data);
void i40e_tunnel_state_detach(struct i40e_tunnel_state *state);
+struct i40e_rss_state *i40e_rss_state_attach(struct rte_eth_dev_data *dev_data);
+void i40e_rss_state_detach(struct i40e_rss_state *state);
int i40e_tunnel_filter_program(struct i40e_pf *pf,
struct i40e_tunnel_filter_conf *tunnel_filter,
uint8_t add);
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 0de0c82521..e7dc21d8cb 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -40,6 +40,9 @@ const struct ci_flow_engine_list i40e_flow_engine_list = {
&i40e_flow_engine_tunnel_mpls,
&i40e_flow_engine_tunnel_gtp,
&i40e_flow_engine_tunnel_l4,
+ &i40e_flow_engine_hash_pattern,
+ &i40e_flow_engine_hash_vlan,
+ &i40e_flow_engine_hash_empty,
}
};
@@ -76,56 +79,6 @@ const struct rte_flow_ops i40e_flow_ops = {
.dev_dump = i40e_flow_dev_dump,
};
-#define I40E_FLOW_DUMP_CHUNK_BYTES 32
-
-static const char *
-i40e_flow_rule_name(enum rte_filter_type filter_type)
-{
- switch (filter_type) {
- case RTE_ETH_FILTER_HASH:
- return "hash";
- default:
- return "unknown";
- }
-}
-
-static size_t
-i40e_flow_rule_size(enum rte_filter_type filter_type)
-{
- switch (filter_type) {
- case RTE_ETH_FILTER_HASH:
- return sizeof(struct i40e_rss_filter);
- default:
- return 0;
- }
-}
-
-static void
-i40e_flow_dump_blob(FILE *file, const char *engine,
- const void *data, size_t data_len)
-{
- const uint8_t *raw = (const uint8_t *)data;
- const size_t nchunks =
- (data_len + I40E_FLOW_DUMP_CHUNK_BYTES - 1) /
- I40E_FLOW_DUMP_CHUNK_BYTES;
- char title[64];
- size_t ci;
-
- fprintf(file, "FLOW DUMP: driver=i40e engine=%s\n", engine);
- fprintf(file, "FLOW DUMP: DATA size=%zu chunks=%zu chunk_bytes=%d\n",
- data_len, nchunks, I40E_FLOW_DUMP_CHUNK_BYTES);
-
- for (ci = 0; ci < nchunks; ci++) {
- const size_t off = ci * I40E_FLOW_DUMP_CHUNK_BYTES;
- const size_t clen =
- RTE_MIN((size_t)I40E_FLOW_DUMP_CHUNK_BYTES, data_len - off);
-
- snprintf(title, sizeof(title), "FLOW DUMP: chunk %03zu/%03zu",
- ci + 1, nchunks);
- rte_memdump(file, title, raw + off, clen);
- }
-}
-
static int
i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow *flow,
@@ -133,59 +86,8 @@ i40e_flow_dev_dump(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct rte_flow *p_flow;
- bool found = false;
- int ret;
- /* try the new flow engine first */
- ret = ci_flow_dump(&pf->flow_engine_conf, flow, file, error);
-
- /*
- * There are multiple possible situations here:
- *
- * - User requested to dump all flows
- * - User requested to dump a specific flow
- *
- * For the first case, we keep going because legacy engines might still
- * have flows we want to dump.
- *
- * For the second case, we only keep going if the flow we were asked to
- * dump was not found in the new engines.
- */
- if (flow != NULL && ret == 0)
- return 0;
-
- TAILQ_FOREACH(p_flow, &pf->flow_list, node) {
- size_t rule_size = 0;
- const void *rule_data = NULL;
-
- if (flow != NULL && p_flow != flow)
- continue;
-
- /* should not happen */
- if (p_flow->rule == NULL) {
- PMD_DRV_LOG(DEBUG, "Invalid flow rule");
- continue;
- }
-
- rule_size = i40e_flow_rule_size(p_flow->filter_type);
- /* should not happen either */
- if (rule_size == 0)
- continue;
-
- found = true;
- rule_data = p_flow->rule;
- i40e_flow_dump_blob(file,
- i40e_flow_rule_name(p_flow->filter_type),
- rule_data, rule_size);
- }
-
- if (flow != NULL && !found)
- return rte_flow_error_set(error, ENOENT,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Flow not found");
-
- return 0;
+ return ci_flow_dump(&pf->flow_engine_conf, flow, file, error);
}
int
@@ -282,41 +184,6 @@ i40e_flow_fdir_get_pctype_value(struct i40e_pf *pf,
return I40E_FILTER_PCTYPE_INVALID;
}
-static int
-i40e_flow_check(struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct i40e_filter_ctx *filter_ctx,
- struct rte_flow_error *error)
-{
- int ret;
-
- ret = ci_flow_check_attr(attr, NULL, error);
- if (ret) {
- return ret;
- }
- /* action and pattern validation will happen in each respective engine */
-
- if (!pattern) {
- rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
- NULL, "NULL pattern.");
- return -rte_errno;
- }
-
- if (!actions) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_NUM,
- NULL, "NULL action.");
- return -rte_errno;
- }
-
- /* try parsing as RSS */
- filter_ctx->type = RTE_ETH_FILTER_HASH;
-
- return i40e_hash_parse(dev, pattern, actions, &filter_ctx->rss_conf, error);
-}
-
static int
i40e_flow_validate(struct rte_eth_dev *dev,
const struct rte_flow_attr *attr,
@@ -325,16 +192,8 @@ i40e_flow_validate(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct i40e_pf *pf = dev->data->dev_private;
- /* creates dummy context */
- struct i40e_filter_ctx filter_ctx = {0};
- int ret;
- /* try the new engine first */
- ret = ci_flow_validate(&pf->flow_engine_conf, attr, pattern, actions, error);
- if (ret == 0)
- return 0;
-
- return i40e_flow_check(dev, attr, pattern, actions, &filter_ctx, error);
+ return ci_flow_validate(&pf->flow_engine_conf, attr, pattern, actions, error);
}
static struct rte_flow *
@@ -345,51 +204,8 @@ i40e_flow_create(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- struct i40e_filter_ctx filter_ctx = {0};
- struct rte_flow *flow = NULL;
- int ret;
- /* try the new engine first */
- flow = ci_flow_create(&pf->flow_engine_conf, attr, pattern, actions, error);
- if (flow != NULL)
- return flow;
-
- ret = i40e_flow_check(dev, attr, pattern, actions, &filter_ctx, error);
- if (ret < 0)
- return NULL;
-
- flow = rte_zmalloc("i40e_flow", sizeof(struct rte_flow), 0);
- if (!flow) {
- rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to allocate memory");
- return flow;
- }
-
- switch (filter_ctx.type) {
- case RTE_ETH_FILTER_HASH:
- ret = i40e_hash_filter_create(pf, &filter_ctx.rss_conf);
- if (ret)
- goto free_flow;
- flow->rule = TAILQ_LAST(&pf->rss_config_list,
- i40e_rss_conf_list);
- break;
- default:
- goto free_flow;
- }
-
- flow->filter_type = filter_ctx.type;
- TAILQ_INSERT_TAIL(&pf->flow_list, flow, node);
- return flow;
-
-free_flow:
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to create flow.");
-
- rte_free(flow);
-
- return NULL;
+ return ci_flow_create(&pf->flow_engine_conf, attr, pattern, actions, error);
}
static int
@@ -398,54 +214,16 @@ i40e_flow_destroy(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- enum rte_filter_type filter_type = flow->filter_type;
- int ret = 0;
- /* try the new engine first */
- ret = ci_flow_destroy(&pf->flow_engine_conf, flow, error);
- if (ret == 0)
- return 0;
-
- switch (filter_type) {
- case RTE_ETH_FILTER_HASH:
- ret = i40e_hash_filter_destroy(pf, flow->rule);
- break;
- default:
- PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
- filter_type);
- ret = -EINVAL;
- break;
- }
-
- if (!ret) {
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- rte_free(flow);
-
- } else
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to destroy flow.");
-
- return ret;
+ return ci_flow_destroy(&pf->flow_engine_conf, flow, error);
}
static int
i40e_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
{
struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- int ret;
- /* flush the new engine first */
- ret = ci_flow_flush(&pf->flow_engine_conf, error);
- if (ret != 0)
- return ret;
-
- ret = i40e_hash_filter_flush(pf);
- if (ret)
- rte_flow_error_set(error, -ret,
- RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
- "Failed to flush RSS flows.");
- return ret;
+ return ci_flow_flush(&pf->flow_engine_conf, error);
}
static int
@@ -454,54 +232,7 @@ i40e_flow_query(struct rte_eth_dev *dev,
const struct rte_flow_action *actions,
void *data, struct rte_flow_error *error)
{
- struct i40e_pf *pf = dev->data->dev_private;
- struct i40e_rss_filter *rss_rule = (struct i40e_rss_filter *)flow->rule;
- enum rte_filter_type filter_type = flow->filter_type;
- struct rte_flow_action_rss *rss_conf = data;
- int ret;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- /* try the new engine first */
- ret = ci_flow_query(&pf->flow_engine_conf, flow, actions, data, error);
- if (ret == 0)
- return 0;
-
- if (!rss_rule) {
- rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_HANDLE,
- NULL, "Invalid rule");
- return -rte_errno;
- }
-
- for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) {
- switch (actions->type) {
- case RTE_FLOW_ACTION_TYPE_VOID:
- break;
- case RTE_FLOW_ACTION_TYPE_RSS:
- if (filter_type != RTE_ETH_FILTER_HASH) {
- rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "action not supported");
- return -rte_errno;
- }
- *rss_conf = (struct rte_flow_action_rss){
- .func = rss_rule->rss_filter_info.func,
- .types = rss_rule->rss_filter_info.types,
- .key_len = rss_rule->rss_filter_info.key_len,
- .queue_num = rss_rule->rss_filter_info.queue_num,
- .key = rss_rule->rss_filter_info.key_len ?
- rss_rule->rss_filter_info.key : NULL,
- .queue = rss_rule->rss_filter_info.queue_num ?
- rss_rule->rss_filter_info.queue : NULL,
- };
- break;
- default:
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION,
- actions,
- "action not supported");
- }
- }
-
- return 0;
+ return ci_flow_query(&pf->flow_engine_conf, flow, actions, data, error);
}
diff --git a/drivers/net/intel/i40e/i40e_flow.h b/drivers/net/intel/i40e/i40e_flow.h
index 28e342210c..31269d4d93 100644
--- a/drivers/net/intel/i40e/i40e_flow.h
+++ b/drivers/net/intel/i40e/i40e_flow.h
@@ -23,5 +23,8 @@ extern const struct ci_flow_engine i40e_flow_engine_tunnel_nvgre;
extern const struct ci_flow_engine i40e_flow_engine_tunnel_mpls;
extern const struct ci_flow_engine i40e_flow_engine_tunnel_gtp;
extern const struct ci_flow_engine i40e_flow_engine_tunnel_l4;
+extern const struct ci_flow_engine i40e_flow_engine_hash_pattern;
+extern const struct ci_flow_engine i40e_flow_engine_hash_vlan;
+extern const struct ci_flow_engine i40e_flow_engine_hash_empty;
#endif /* _I40E_FLOW_H_ */
diff --git a/drivers/net/intel/i40e/i40e_flow_hash.c b/drivers/net/intel/i40e/i40e_flow_hash.c
new file mode 100644
index 0000000000..a2ba821d4f
--- /dev/null
+++ b/drivers/net/intel/i40e/i40e_flow_hash.c
@@ -0,0 +1,1441 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2026 Intel Corporation
+ */
+
+#include "i40e_ethdev.h"
+#include "i40e_flow.h"
+#include "i40e_hash.h"
+
+#include "../common/flow_engine.h"
+#include "../common/flow_check.h"
+#include "../common/flow_util.h"
+
+struct i40e_hash_priv {
+ struct i40e_rss_state *state;
+};
+
+struct i40e_hash_ctx {
+ struct ci_flow_engine_ctx base;
+ struct i40e_rte_flow_rss_conf rss_conf;
+ uint32_t pctype;
+ bool customized_ptype;
+};
+
+struct i40e_flow_engine_hash_flow {
+ struct rte_flow base;
+ struct i40e_rte_flow_rss_conf rss_conf;
+ struct i40e_rss_flow_node *node;
+};
+
+#define I40E_HASH_L4_TYPES (RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP)
+
+#define I40E_HASH_L2_RSS_MASK (RTE_ETH_RSS_VLAN | RTE_ETH_RSS_ETH | \
+ RTE_ETH_RSS_L2_SRC_ONLY | \
+ RTE_ETH_RSS_L2_DST_ONLY)
+
+#define I40E_HASH_L23_RSS_MASK (I40E_HASH_L2_RSS_MASK | \
+ RTE_ETH_RSS_L3_SRC_ONLY | \
+ RTE_ETH_RSS_L3_DST_ONLY)
+
+#define I40E_HASH_IPV4_L23_RSS_MASK (RTE_ETH_RSS_IPV4 | I40E_HASH_L23_RSS_MASK)
+#define I40E_HASH_IPV6_L23_RSS_MASK (RTE_ETH_RSS_IPV6 | I40E_HASH_L23_RSS_MASK)
+
+#define I40E_HASH_L234_RSS_MASK (I40E_HASH_L23_RSS_MASK | \
+ RTE_ETH_RSS_PORT | RTE_ETH_RSS_L4_SRC_ONLY | \
+ RTE_ETH_RSS_L4_DST_ONLY)
+
+#define I40E_HASH_IPV4_L234_RSS_MASK (I40E_HASH_L234_RSS_MASK | RTE_ETH_RSS_IPV4)
+#define I40E_HASH_IPV6_L234_RSS_MASK (I40E_HASH_L234_RSS_MASK | RTE_ETH_RSS_IPV6)
+
+/* Structure of mapping RSS type to input set */
+struct i40e_hash_map_rss_inset {
+ uint64_t rss_type;
+ uint64_t inset;
+};
+
+static const struct i40e_hash_map_rss_inset i40e_hash_rss_inset[] = {
+ /* IPv4 */
+ { RTE_ETH_RSS_IPV4, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
+ { RTE_ETH_RSS_FRAG_IPV4, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
+
+ { RTE_ETH_RSS_NONFRAG_IPV4_OTHER,
+ I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
+
+ { RTE_ETH_RSS_NONFRAG_IPV4_TCP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
+
+ { RTE_ETH_RSS_NONFRAG_IPV4_UDP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
+
+ { RTE_ETH_RSS_NONFRAG_IPV4_SCTP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT | I40E_INSET_SCTP_VT },
+
+ /* IPv6 */
+ { RTE_ETH_RSS_IPV6, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
+ { RTE_ETH_RSS_FRAG_IPV6, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
+
+ { RTE_ETH_RSS_NONFRAG_IPV6_OTHER,
+ I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
+
+ { RTE_ETH_RSS_NONFRAG_IPV6_TCP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
+
+ { RTE_ETH_RSS_NONFRAG_IPV6_UDP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
+
+ { RTE_ETH_RSS_NONFRAG_IPV6_SCTP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
+ I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT | I40E_INSET_SCTP_VT },
+
+ /* Port */
+ { RTE_ETH_RSS_PORT, I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
+
+ /* Ether */
+ { RTE_ETH_RSS_L2_PAYLOAD, I40E_INSET_LAST_ETHER_TYPE },
+ { RTE_ETH_RSS_ETH, I40E_INSET_DMAC | I40E_INSET_SMAC },
+
+ /* VLAN */
+ { RTE_ETH_RSS_S_VLAN, I40E_INSET_VLAN_OUTER },
+ { RTE_ETH_RSS_C_VLAN, I40E_INSET_VLAN_INNER },
+};
+
+static uint64_t
+i40e_hash_get_inset(uint64_t rss_types, bool symmetric_enable)
+{
+ uint64_t mask, inset = 0;
+ int i;
+
+ for (i = 0; i < (int)RTE_DIM(i40e_hash_rss_inset); i++) {
+ if (rss_types & i40e_hash_rss_inset[i].rss_type)
+ inset |= i40e_hash_rss_inset[i].inset;
+ }
+
+ if (!inset)
+ return 0;
+
+ /* If SRC_ONLY and DST_ONLY of the same level are used simultaneously,
+ * it is the same case as none of them are added.
+ */
+ mask = rss_types & (RTE_ETH_RSS_L2_SRC_ONLY | RTE_ETH_RSS_L2_DST_ONLY);
+ if (mask == RTE_ETH_RSS_L2_SRC_ONLY)
+ inset &= ~I40E_INSET_DMAC;
+ else if (mask == RTE_ETH_RSS_L2_DST_ONLY)
+ inset &= ~I40E_INSET_SMAC;
+
+ mask = rss_types & (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY);
+ if (mask == RTE_ETH_RSS_L3_SRC_ONLY)
+ inset &= ~(I40E_INSET_IPV4_DST | I40E_INSET_IPV6_DST);
+ else if (mask == RTE_ETH_RSS_L3_DST_ONLY)
+ inset &= ~(I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC);
+
+ mask = rss_types & (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY);
+ if (mask == RTE_ETH_RSS_L4_SRC_ONLY)
+ inset &= ~I40E_INSET_DST_PORT;
+ else if (mask == RTE_ETH_RSS_L4_DST_ONLY)
+ inset &= ~I40E_INSET_SRC_PORT;
+
+ if (rss_types & I40E_HASH_L4_TYPES) {
+ uint64_t l3_mask = rss_types &
+ (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY);
+ uint64_t l4_mask = rss_types &
+ (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY);
+
+ if (l3_mask && !l4_mask)
+ inset &= ~(I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT);
+ else if (!l3_mask && l4_mask)
+ inset &= ~(I40E_INSET_IPV4_DST | I40E_INSET_IPV6_DST |
+ I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC);
+ }
+
+ /* SCTP Verification Tag is not required in hash computation for SYMMETRIC_TOEPLITZ */
+ if (symmetric_enable) {
+ mask = rss_types & RTE_ETH_RSS_NONFRAG_IPV4_SCTP;
+ if (mask == RTE_ETH_RSS_NONFRAG_IPV4_SCTP)
+ inset &= ~I40E_INSET_SCTP_VT;
+
+ mask = rss_types & RTE_ETH_RSS_NONFRAG_IPV6_SCTP;
+ if (mask == RTE_ETH_RSS_NONFRAG_IPV6_SCTP)
+ inset &= ~I40E_INSET_SCTP_VT;
+ }
+
+ return inset;
+}
+
+/*
+ * Hash pattern graph implementation
+ * Pattern: START -> ETH -> [VLAN] -> [VLAN] -> (IPv4|IPv6) -> (TCP|UDP|SCTP|ESP|L2TPV3OIP|AH)
+ * START -> ETH -> [VLAN] -> [VLAN] -> (IPv4|IPv6) frag
+ * START -> ETH -> [VLAN] -> [VLAN] -> (IPv4|IPv6) -> UDP -> (GTPC|ESP|GTPU)
+ * START -> ETH -> [VLAN] -> [VLAN] -> (IPv4|IPv6) -> UDP -> GTPU -> (IPv4|IPv6)
+ */
+enum i40e_hash_pattern_node_id {
+ I40E_HASH_PATTERN_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_HASH_PATTERN_NODE_ETH,
+ I40E_HASH_PATTERN_NODE_OUTER_VLAN,
+ I40E_HASH_PATTERN_NODE_INNER_VLAN,
+ I40E_HASH_PATTERN_NODE_IPV4,
+ I40E_HASH_PATTERN_NODE_IPV6,
+ I40E_HASH_PATTERN_NODE_IPV6_FRAG,
+ I40E_HASH_PATTERN_NODE_TCP,
+ I40E_HASH_PATTERN_NODE_UDP,
+ I40E_HASH_PATTERN_NODE_SCTP,
+ I40E_HASH_PATTERN_NODE_ESP,
+ I40E_HASH_PATTERN_NODE_GTPU,
+ I40E_HASH_PATTERN_NODE_GTPC,
+ I40E_HASH_PATTERN_NODE_L2TPV3OIP,
+ I40E_HASH_PATTERN_NODE_AH,
+ I40E_HASH_PATTERN_NODE_INNER_IPV4,
+ I40E_HASH_PATTERN_NODE_INNER_IPV6,
+ I40E_HASH_PATTERN_NODE_END,
+ I40E_HASH_PATTERN_NODE_MAX,
+};
+
+static int
+i40e_hash_pattern_node_eth_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = I40E_FILTER_PCTYPE_L2_PAYLOAD;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_ipv4_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ /* hash parser does not differentiate between frag and non-frag IPv4 until later */
+ hash_ctx->pctype = I40E_FILTER_PCTYPE_NONF_IPV4_OTHER;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_ipv6_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = I40E_FILTER_PCTYPE_NONF_IPV6_OTHER;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_ipv6_frag_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = I40E_FILTER_PCTYPE_FRAG_IPV6;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_tcp_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) ?
+ I40E_FILTER_PCTYPE_NONF_IPV4_TCP :
+ I40E_FILTER_PCTYPE_NONF_IPV6_TCP;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_udp_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) ?
+ I40E_FILTER_PCTYPE_NONF_IPV4_UDP :
+ I40E_FILTER_PCTYPE_NONF_IPV6_UDP;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_sctp_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) ?
+ I40E_FILTER_PCTYPE_NONF_IPV4_SCTP :
+ I40E_FILTER_PCTYPE_NONF_IPV6_SCTP;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_esp_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ bool ipv4 = false;
+ bool udp = false;
+
+ /* ESP can be over IP or over UDP */
+ ipv4 = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER ||
+ hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_UDP);
+ udp = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_UDP ||
+ hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV6_UDP);
+ if (udp) {
+ hash_ctx->pctype = ipv4 ? I40E_CUSTOMIZED_ESP_IPV4_UDP : I40E_CUSTOMIZED_ESP_IPV6_UDP;
+ } else {
+ hash_ctx->pctype = ipv4 ? I40E_CUSTOMIZED_ESP_IPV4 : I40E_CUSTOMIZED_ESP_IPV6;
+ }
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_gtpu_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ /* GTPU pctype does not differentiate between IPv4 and IPv6 */
+ hash_ctx->pctype = I40E_CUSTOMIZED_GTPU;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_gtpc_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ /* GTPC pctype does not differentiate between IPv4 and IPv6 */
+ hash_ctx->pctype = I40E_CUSTOMIZED_GTPC;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_l2tpv3oip_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ hash_ctx->pctype = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) ?
+ I40E_CUSTOMIZED_IPV4_L2TPV3 :
+ I40E_CUSTOMIZED_IPV6_L2TPV3;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_ah_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ hash_ctx->pctype = (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) ?
+ I40E_CUSTOMIZED_AH_IPV4 :
+ I40E_CUSTOMIZED_AH_IPV6;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_inner_ipv4_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ /* inner IP patterns are always over GTP-U */
+ hash_ctx->pctype = I40E_CUSTOMIZED_GTPU_IPV4;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_inner_ipv6_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ /* inner IP patterns are always over GTP-U */
+ hash_ctx->pctype = I40E_CUSTOMIZED_GTPU_IPV6;
+ hash_ctx->customized_ptype = true;
+ return 0;
+}
+
+static int
+i40e_hash_pattern_node_end_process(void *ctx,
+ const struct rte_flow_item *item __rte_unused,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+
+ /* if RSS hash type for IPv4 frag was requested, change pctype */
+ if (hash_ctx->pctype == I40E_FILTER_PCTYPE_NONF_IPV4_OTHER &&
+ (hash_ctx->rss_conf.types & RTE_ETH_RSS_FRAG_IPV4))
+ hash_ctx->pctype = I40E_FILTER_PCTYPE_FRAG_IPV4;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_hash_pattern_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_HASH_PATTERN_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_HASH_PATTERN_NODE_ETH] = {
+ .name = "ETH",
+ .type = RTE_FLOW_ITEM_TYPE_ETH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_eth_process,
+ },
+ [I40E_HASH_PATTERN_NODE_OUTER_VLAN] = {
+ .name = "OUTER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_VLAN] = {
+ .name = "INNER_VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ },
+ [I40E_HASH_PATTERN_NODE_IPV4] = {
+ .name = "IPv4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_ipv4_process,
+ },
+ [I40E_HASH_PATTERN_NODE_IPV6] = {
+ .name = "IPv6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_ipv6_process,
+ },
+ [I40E_HASH_PATTERN_NODE_IPV6_FRAG] = {
+ .name = "IPv6_FRAG",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_ipv6_frag_process,
+ },
+ [I40E_HASH_PATTERN_NODE_TCP] = {
+ .name = "TCP",
+ .type = RTE_FLOW_ITEM_TYPE_TCP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_tcp_process,
+ },
+ [I40E_HASH_PATTERN_NODE_UDP] = {
+ .name = "UDP",
+ .type = RTE_FLOW_ITEM_TYPE_UDP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_udp_process,
+ },
+ [I40E_HASH_PATTERN_NODE_SCTP] = {
+ .name = "SCTP",
+ .type = RTE_FLOW_ITEM_TYPE_SCTP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_sctp_process,
+ },
+ [I40E_HASH_PATTERN_NODE_ESP] = {
+ .name = "ESP",
+ .type = RTE_FLOW_ITEM_TYPE_ESP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_esp_process,
+ },
+ [I40E_HASH_PATTERN_NODE_GTPU] = {
+ .name = "GTPU",
+ .type = RTE_FLOW_ITEM_TYPE_GTPU,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_gtpu_process,
+ },
+ [I40E_HASH_PATTERN_NODE_GTPC] = {
+ .name = "GTPC",
+ .type = RTE_FLOW_ITEM_TYPE_GTPC,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_gtpc_process,
+ },
+ [I40E_HASH_PATTERN_NODE_L2TPV3OIP] = {
+ .name = "L2TPV3OIP",
+ .type = RTE_FLOW_ITEM_TYPE_L2TPV3OIP,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_l2tpv3oip_process,
+ },
+ [I40E_HASH_PATTERN_NODE_AH] = {
+ .name = "AH",
+ .type = RTE_FLOW_ITEM_TYPE_AH,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_ah_process,
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_IPV4] = {
+ .name = "INNER_IPV4",
+ .type = RTE_FLOW_ITEM_TYPE_IPV4,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_inner_ipv4_process,
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_IPV6] = {
+ .name = "INNER_IPV6",
+ .type = RTE_FLOW_ITEM_TYPE_IPV6,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_EMPTY,
+ .process = i40e_hash_pattern_node_inner_ipv6_process,
+ },
+ [I40E_HASH_PATTERN_NODE_END] = {
+ .name = "END",
+ .type = RTE_FLOW_ITEM_TYPE_END,
+ .process = i40e_hash_pattern_node_end_process,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_HASH_PATTERN_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_ETH,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_ETH] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_OUTER_VLAN,
+ I40E_HASH_PATTERN_NODE_IPV4,
+ I40E_HASH_PATTERN_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_OUTER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_INNER_VLAN,
+ I40E_HASH_PATTERN_NODE_IPV4,
+ I40E_HASH_PATTERN_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_VLAN] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_IPV4,
+ I40E_HASH_PATTERN_NODE_IPV6,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_IPV4] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_TCP,
+ I40E_HASH_PATTERN_NODE_UDP,
+ I40E_HASH_PATTERN_NODE_SCTP,
+ I40E_HASH_PATTERN_NODE_ESP,
+ I40E_HASH_PATTERN_NODE_L2TPV3OIP,
+ I40E_HASH_PATTERN_NODE_AH,
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_IPV6] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_IPV6_FRAG,
+ I40E_HASH_PATTERN_NODE_TCP,
+ I40E_HASH_PATTERN_NODE_UDP,
+ I40E_HASH_PATTERN_NODE_SCTP,
+ I40E_HASH_PATTERN_NODE_ESP,
+ I40E_HASH_PATTERN_NODE_L2TPV3OIP,
+ I40E_HASH_PATTERN_NODE_AH,
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_IPV6_FRAG] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_TCP] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_UDP] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_GTPU,
+ I40E_HASH_PATTERN_NODE_GTPC,
+ I40E_HASH_PATTERN_NODE_ESP,
+ I40E_HASH_PATTERN_NODE_END,
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_SCTP] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_ESP] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_GTPU] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_INNER_IPV4,
+ I40E_HASH_PATTERN_NODE_INNER_IPV6,
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_GTPC] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_L2TPV3OIP] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_AH] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_IPV4] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_PATTERN_NODE_INNER_IPV6] = {
+ .next = (size_t[]) {
+ I40E_HASH_PATTERN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+/*
+ * Hash VLAN graph implementation
+ * Pattern: START -> VLAN -> END
+ */
+enum i40e_hash_vlan_node_id {
+ I40E_HASH_VLAN_NODE_START = FLOW_GRAPH_NODE_FIRST,
+ I40E_HASH_VLAN_NODE_VLAN,
+ I40E_HASH_VLAN_NODE_END,
+ I40E_HASH_VLAN_NODE_MAX,
+};
+
+static int
+i40e_hash_node_vlan_validate(const void *ctx __rte_unused, const struct rte_flow_item *item,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_vlan *vlan_mask = item->mask;
+
+ /* only the VLAN priority bits may be matched */
+ if (rte_be_to_cpu_16(vlan_mask->hdr.vlan_tci) != RTE_VLAN_PRI_MASK) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Invalid VLAN mask");
+ }
+ return 0;
+}
+
+static int
+i40e_hash_node_vlan_process(void *ctx, const struct rte_flow_item *item,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_hash_ctx *hash_ctx = ctx;
+ const struct rte_flow_item_vlan *vlan_spec = item->spec;
+
+ hash_ctx->rss_conf.region_priority = rte_cpu_to_be_16(vlan_spec->hdr.vlan_tci) >> 13;
+
+ return 0;
+}
+
+static const struct flow_graph i40e_hash_vlan_graph = {
+ .nodes = (struct flow_graph_node[]) {
+ [I40E_HASH_VLAN_NODE_START] = {
+ .name = "START",
+ },
+ [I40E_HASH_VLAN_NODE_VLAN] = {
+ .name = "VLAN",
+ .type = RTE_FLOW_ITEM_TYPE_VLAN,
+ .constraints = FLOW_GRAPH_NODE_EXPECT_SPEC_MASK,
+ .validate = i40e_hash_node_vlan_validate,
+ .process = i40e_hash_node_vlan_process,
+ },
+ },
+ .edges = (struct flow_graph_edge[]) {
+ [I40E_HASH_VLAN_NODE_START] = {
+ .next = (size_t[]) {
+ I40E_HASH_VLAN_NODE_VLAN,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ [I40E_HASH_VLAN_NODE_VLAN] = {
+ .next = (size_t[]) {
+ I40E_HASH_VLAN_NODE_END,
+ FLOW_GRAPH_NODE_EDGE_END
+ }
+ },
+ },
+};
+
+static bool
+i40e_hash_validate_rss_types(uint64_t rss_types)
+{
+ uint64_t type, mask;
+
+ /* Validate L2 */
+ type = RTE_ETH_RSS_ETH & rss_types;
+ mask = (RTE_ETH_RSS_L2_SRC_ONLY | RTE_ETH_RSS_L2_DST_ONLY) & rss_types;
+ if (!type && mask)
+ return false;
+
+ /* Validate L3 */
+ type = (I40E_HASH_L4_TYPES | RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4 |
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | RTE_ETH_RSS_IPV6 |
+ RTE_ETH_RSS_FRAG_IPV6 | RTE_ETH_RSS_NONFRAG_IPV6_OTHER) & rss_types;
+ mask = (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY) & rss_types;
+ if (!type && mask)
+ return false;
+
+ /* Validate L4 */
+ type = (I40E_HASH_L4_TYPES | RTE_ETH_RSS_PORT) & rss_types;
+ mask = (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY) & rss_types;
+ if (!type && mask)
+ return false;
+
+ return true;
+}
+
+static int
+i40e_hash_validate_rss_common(const struct rte_flow_action_rss *rss_act,
+ struct rte_flow_error *error)
+{
+ /* RSS level is not supported */
+ if (rss_act->level != 0) {
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS level is not supported");
+ }
+
+ /* symmetric toeplitz is only supported when a specific pattern is provided */
+ if (rss_act->func == RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "Symmetric hash function not supported without specific patterns");
+ }
+
+ /*
+ * When RSS types is not specified in testpmd, it will set up a default
+ * RSS types value for the flow. Even though no hash engine part calling
+ * this particular function will use RSS types parameter for anything,
+ * we cannot reject having it because it is extra effort for testpmd
+ * user to avoid specifying it.
+ *
+ * So, instead, accept types value even though we are not using it for
+ * anything, but produce a warning for the user.
+ */
+ if (rss_act->types != 0)
+ PMD_DRV_LOG(WARNING, "RSS types specified but will not be used");
+
+ /* check RSS key length if it is specified */
+ if (rss_act->key_len != 0 && rss_act->key_len != I40E_RSS_KEY_LEN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS key length must be 52 bytes");
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_pattern_rss_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param __rte_unused,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
+
+ /* queue list is not supported */
+ if (rss_act->queue_num != 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS queues not supported when pattern specified");
+ }
+
+ /* disallow unsupported hash functions */
+ switch (rss_act->func) {
+ case RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ:
+ case RTE_ETH_HASH_FUNCTION_DEFAULT:
+ case RTE_ETH_HASH_FUNCTION_TOEPLITZ:
+ case RTE_ETH_HASH_FUNCTION_SIMPLE_XOR:
+ break;
+ default:
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS hash function not supported when pattern specified");
+ }
+
+ if (!i40e_hash_validate_rss_types(rss_act->types))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ rss_act, "RSS types are invalid");
+
+ /* check RSS key length if it is specified */
+ if (rss_act->key_len != 0 && rss_act->key_len != I40E_RSS_KEY_LEN) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS key length must be 52 bytes");
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_pattern_ctx_init(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_hash_ctx *hash_ctx = (struct i40e_hash_ctx *)ctx;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .check = i40e_hash_pattern_rss_check,
+ };
+ const struct rte_flow_action_rss *rss_act;
+ int ret;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret != 0)
+ return ret;
+
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret != 0)
+ return ret;
+
+ rss_act = parsed_actions.actions[0]->conf;
+
+ hash_ctx->rss_conf.symmetric_enable =
+ rss_act->func == RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ;
+ hash_ctx->rss_conf.func = rss_act->func;
+ hash_ctx->rss_conf.types = rss_act->types;
+
+ if (rss_act->key_len != 0) {
+ /* key length already checked */
+ memcpy(hash_ctx->rss_conf.key, rss_act->key, I40E_RSS_KEY_LEN);
+ hash_ctx->rss_conf.key_len = I40E_RSS_KEY_LEN;
+ }
+
+ hash_ctx->rss_conf.inset = i40e_hash_get_inset(rss_act->types,
+ hash_ctx->rss_conf.symmetric_enable);
+
+ return 0;
+}
+
+static uint64_t
+i40e_hash_get_x722_ext_pctypes(uint8_t match_pctype)
+{
+ uint64_t pctypes = 0;
+
+ switch (match_pctype) {
+ case I40E_FILTER_PCTYPE_NONF_IPV4_TCP:
+ pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_IPV4_TCP_SYN_NO_ACK);
+ break;
+
+ case I40E_FILTER_PCTYPE_NONF_IPV4_UDP:
+ pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_UNICAST_IPV4_UDP) |
+ BIT_ULL(I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV4_UDP);
+ break;
+
+ case I40E_FILTER_PCTYPE_NONF_IPV6_TCP:
+ pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_IPV6_TCP_SYN_NO_ACK);
+ break;
+
+ case I40E_FILTER_PCTYPE_NONF_IPV6_UDP:
+ pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_UNICAST_IPV6_UDP) |
+ BIT_ULL(I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV6_UDP);
+ break;
+ }
+
+ return pctypes;
+}
+
+static int
+i40e_hash_translate_gtp_inset(struct i40e_rte_flow_rss_conf *rss_conf,
+ struct rte_flow_error *error)
+{
+ if (rss_conf->inset &
+ (I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC |
+ I40E_INSET_DST_PORT | I40E_INSET_SRC_PORT))
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL,
+ "Only support external destination IP");
+
+ if (rss_conf->inset & I40E_INSET_IPV4_DST)
+ rss_conf->inset = (rss_conf->inset & ~I40E_INSET_IPV4_DST) |
+ I40E_INSET_TUNNEL_IPV4_DST;
+
+ if (rss_conf->inset & I40E_INSET_IPV6_DST)
+ rss_conf->inset = (rss_conf->inset & ~I40E_INSET_IPV6_DST) |
+ I40E_INSET_TUNNEL_IPV6_DST;
+
+ return 0;
+}
+
+static int
+i40e_hash_pattern_ctx_finalize(struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(ctx->dev_data->dev_private);
+ struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(ctx->dev_data->dev_private);
+ struct i40e_hash_ctx *hash_ctx = (struct i40e_hash_ctx *)ctx;
+ const struct hash_type_to_pattern {
+ uint32_t pctype;
+ uint64_t valid_rss_flags;
+ } valid_pctype_to_pattern[] = {
+ /* Ether */
+ {I40E_FILTER_PCTYPE_L2_PAYLOAD, I40E_HASH_L2_RSS_MASK | RTE_ETH_RSS_L2_PAYLOAD},
+ /* IP */
+ {I40E_FILTER_PCTYPE_NONF_IPV4_OTHER, RTE_ETH_RSS_NONFRAG_IPV4_OTHER | I40E_HASH_IPV4_L23_RSS_MASK},
+ {I40E_FILTER_PCTYPE_NONF_IPV6_OTHER, RTE_ETH_RSS_NONFRAG_IPV6_OTHER | I40E_HASH_IPV6_L23_RSS_MASK},
+ /* IP fragmented */
+ {I40E_FILTER_PCTYPE_FRAG_IPV4, RTE_ETH_RSS_FRAG_IPV4 | I40E_HASH_IPV4_L23_RSS_MASK},
+ {I40E_FILTER_PCTYPE_FRAG_IPV6, RTE_ETH_RSS_FRAG_IPV6 | I40E_HASH_IPV6_L23_RSS_MASK},
+ /* TCP */
+ {I40E_FILTER_PCTYPE_NONF_IPV4_TCP, RTE_ETH_RSS_NONFRAG_IPV4_TCP | I40E_HASH_IPV4_L234_RSS_MASK},
+ {I40E_FILTER_PCTYPE_NONF_IPV6_TCP, RTE_ETH_RSS_NONFRAG_IPV6_TCP | I40E_HASH_IPV6_L234_RSS_MASK},
+ /* UDP */
+ {I40E_FILTER_PCTYPE_NONF_IPV4_UDP, RTE_ETH_RSS_NONFRAG_IPV4_UDP | I40E_HASH_IPV4_L234_RSS_MASK},
+ {I40E_FILTER_PCTYPE_NONF_IPV6_UDP, RTE_ETH_RSS_NONFRAG_IPV6_UDP | I40E_HASH_IPV6_L234_RSS_MASK},
+ /* SCTP */
+ {I40E_FILTER_PCTYPE_NONF_IPV4_SCTP, RTE_ETH_RSS_NONFRAG_IPV4_SCTP | I40E_HASH_IPV4_L234_RSS_MASK},
+ {I40E_FILTER_PCTYPE_NONF_IPV6_SCTP, RTE_ETH_RSS_NONFRAG_IPV6_SCTP | I40E_HASH_IPV6_L234_RSS_MASK},
+ /* AH */
+ {I40E_CUSTOMIZED_AH_IPV4, RTE_ETH_RSS_AH},
+ {I40E_CUSTOMIZED_AH_IPV6, RTE_ETH_RSS_AH},
+ /* L2TPV3 */
+ {I40E_CUSTOMIZED_IPV4_L2TPV3, RTE_ETH_RSS_L2TPV3},
+ {I40E_CUSTOMIZED_IPV6_L2TPV3, RTE_ETH_RSS_L2TPV3},
+ /* ESP */
+ {I40E_CUSTOMIZED_ESP_IPV4, RTE_ETH_RSS_ESP},
+ {I40E_CUSTOMIZED_ESP_IPV6, RTE_ETH_RSS_ESP},
+ {I40E_CUSTOMIZED_ESP_IPV4_UDP, RTE_ETH_RSS_ESP},
+ {I40E_CUSTOMIZED_ESP_IPV6_UDP, RTE_ETH_RSS_ESP},
+ /* GTPC */
+ {I40E_CUSTOMIZED_GTPC, I40E_HASH_IPV4_L234_RSS_MASK},
+ {I40E_CUSTOMIZED_GTPC, I40E_HASH_IPV6_L234_RSS_MASK},
+ /* GTPU */
+ {I40E_CUSTOMIZED_GTPU, I40E_HASH_IPV4_L234_RSS_MASK},
+ {I40E_CUSTOMIZED_GTPU, I40E_HASH_IPV6_L234_RSS_MASK},
+ /* IP over GTPU */
+ {I40E_CUSTOMIZED_GTPU_IPV4, RTE_ETH_RSS_GTPU},
+ {I40E_CUSTOMIZED_GTPU_IPV6, RTE_ETH_RSS_GTPU},
+ };
+ size_t i;
+
+ for (i = 0; i < RTE_DIM(valid_pctype_to_pattern); i++) {
+ uint32_t pctype = valid_pctype_to_pattern[i].pctype;
+ uint64_t flags = valid_pctype_to_pattern[i].valid_rss_flags;
+
+ if (pctype != hash_ctx->pctype)
+ continue;
+
+ /* find if our ptype works with specified RSS types */
+ if ((hash_ctx->rss_conf.types & ~flags) != 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL,
+ "Some RSS types are not supported for the specified pattern");
+ }
+
+ /* for customized pctypes, find if it's supported */
+ if (hash_ctx->customized_ptype) {
+ struct i40e_customized_pctype *ct;
+
+ ct = i40e_find_customized_pctype(pf, pctype);
+
+ if (ct == NULL || !ct->valid) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL,
+ "Specified pattern is not supported by the device");
+ }
+ hash_ctx->rss_conf.config_pctypes |= BIT_ULL(ct->pctype);
+
+ /* GTPC/GTPU endpoints require special handling */
+ return i40e_hash_translate_gtp_inset(&hash_ctx->rss_conf, error);
+ } else {
+ hash_ctx->rss_conf.config_pctypes |= BIT_ULL(pctype);
+
+ /* X722 needs special handling */
+ if (hw->mac.type == I40E_MAC_X722) {
+ uint64_t types = i40e_hash_get_x722_ext_pctypes(pctype);
+ hash_ctx->rss_conf.config_pctypes |= types;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_queue_region_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
+ struct rte_eth_dev_data *dev_data = param->driver_ctx;
+ const struct i40e_pf *pf;
+ uint64_t hash_queues;
+ int ret;
+
+ ret = i40e_hash_validate_rss_common(rss_act, error);
+ if (ret)
+ return ret;
+
+ RTE_BUILD_BUG_ON(sizeof(hash_queues) != sizeof(pf->hash_enabled_queues));
+
+ /* having RSS key is not supported */
+ if (rss_act->key != NULL) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS key not supported");
+ }
+
+ /* queue region must be specified */
+ if (rss_act->queue_num == 0) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS queues missing");
+ }
+
+ /* queue region must be power of two */
+ if (!rte_is_power_of_2(rss_act->queue_num)) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS queue number must be power of two");
+ }
+
+ /* generic checks already filtered out discontiguous/non-unique RSS queues */
+
+ /* queues must not exceed maximum queues per traffic class */
+ if (rss_act->queue[rss_act->queue_num - 1] >= I40E_MAX_Q_PER_TC) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "Invalid RSS queue index");
+ }
+
+ /* queues must be in LUT */
+ pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ hash_queues = (BIT_ULL(rss_act->queue[0] + rss_act->queue_num) - 1) &
+ ~(BIT_ULL(rss_act->queue[0]) - 1);
+
+ if (hash_queues & ~pf->hash_enabled_queues) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ rss_act, "Some queues are not in LUT");
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_vlan_ctx_init(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_hash_ctx *hash_ctx = (struct i40e_hash_ctx *)ctx;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .driver_ctx = ctx->dev_data,
+ .check = i40e_hash_queue_region_check,
+ .rss_queues_contig = true,
+ };
+ const struct rte_flow_action_rss *rss_act;
+ int ret;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret != 0)
+ return ret;
+
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret != 0)
+ return ret;
+
+ rss_act = parsed_actions.actions[0]->conf;
+ hash_ctx->rss_conf.func = rss_act->func;
+ hash_ctx->rss_conf.region_queue_num = rss_act->queue_num;
+ hash_ctx->rss_conf.region_queue_start = rss_act->queue[0];
+
+ return 0;
+}
+
+static int
+i40e_hash_queue_list_check(const struct ci_flow_actions *actions,
+ const struct ci_flow_actions_check_param *param,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
+ struct rte_eth_dev_data *dev_data = param->driver_ctx;
+ struct i40e_pf *pf;
+ struct i40e_hw *hw;
+ uint16_t max_queue;
+ bool has_queue, has_key;
+ int ret;
+
+ ret = i40e_hash_validate_rss_common(rss_act, error);
+ if (ret)
+ return ret;
+
+ has_queue = rss_act->queue != NULL;
+ has_key = rss_act->key != NULL;
+
+ /* if we have queues, we must not have key */
+ if (has_queue && has_key) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "RSS key for queue region is not supported");
+ }
+
+ /* if there are no queues, no further checks needed */
+ if (!has_queue)
+ return 0;
+
+ /* check queue number limits */
+ hw = I40E_DEV_PRIVATE_TO_HW(dev_data->dev_private);
+ if (rss_act->queue_num > hw->func_caps.rss_table_size) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ rss_act, "Too many RSS queues");
+ }
+
+ pf = I40E_DEV_PRIVATE_TO_PF(dev_data->dev_private);
+ if (pf->dev_data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_VMDQ_FLAG)
+ max_queue = i40e_pf_calc_configured_queues_num(pf);
+ else
+ max_queue = pf->dev_data->nb_rx_queues;
+
+ max_queue = RTE_MIN(max_queue, I40E_MAX_Q_PER_TC);
+
+ /* we know RSS queues are monotonic so we only need to check last queue */
+ if (rss_act->queue[rss_act->queue_num - 1] >= max_queue) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
+ "Invalid RSS queue");
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_empty_ctx_init(const struct rte_flow_action actions[],
+ const struct rte_flow_attr *attr,
+ struct ci_flow_engine_ctx *ctx,
+ struct rte_flow_error *error)
+{
+ struct i40e_hash_ctx *hash_ctx = (struct i40e_hash_ctx *)ctx;
+ struct ci_flow_actions parsed_actions = {0};
+ struct ci_flow_actions_check_param param = {
+ .allowed_types = (enum rte_flow_action_type[]) {
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_END
+ },
+ .max_actions = 1,
+ .driver_ctx = ctx->dev_data,
+ .check = i40e_hash_queue_list_check,
+ };
+ const struct rte_flow_action_rss *rss_act;
+ int ret;
+
+ ret = ci_flow_check_attr(attr, NULL, error);
+ if (ret != 0)
+ return ret;
+
+ ret = ci_flow_check_actions(actions, ¶m, &parsed_actions, error);
+ if (ret != 0)
+ return ret;
+
+ rss_act = parsed_actions.actions[0]->conf;
+ hash_ctx->rss_conf.func = rss_act->func;
+
+ /* if we have queues, copy them */
+ if (rss_act->queue_num > 0) {
+ memcpy(hash_ctx->rss_conf.queue,
+ rss_act->queue,
+ rss_act->queue_num * sizeof(hash_ctx->rss_conf.queue[0]));
+ hash_ctx->rss_conf.queue_num = rss_act->queue_num;
+ /* if we have key, copy it */
+ } else if (rss_act->key_len > 0) {
+ /* key length already checked */
+ memcpy(hash_ctx->rss_conf.key, rss_act->key, I40E_RSS_KEY_LEN);
+ hash_ctx->rss_conf.key_len = I40E_RSS_KEY_LEN;
+ }
+
+ return 0;
+}
+
+static int
+i40e_hash_ctx_to_flow(const struct ci_flow_engine_ctx *ctx,
+ struct ci_flow *flow,
+ struct rte_flow_error *error __rte_unused)
+{
+ const struct i40e_hash_ctx *hash_ctx = (const struct i40e_hash_ctx *)ctx;
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+
+ memcpy(&hash_flow->rss_conf, &hash_ctx->rss_conf, sizeof(hash_flow->rss_conf));
+
+ return 0;
+}
+
+/*
+ * Strip from an earlier (prev) filter's owned reset flags anything that a
+ * later (ref) filter now also owns, since the later one takes over ownership
+ * of overlapping HW state.
+ */
+static void
+i40e_hash_invalidate_prev(const struct i40e_rte_flow_rss_conf *ref_conf,
+ const struct i40e_rss_filter_data *ref_data,
+ const struct i40e_rte_flow_rss_conf *prev_conf,
+ struct i40e_rss_filter_data *prev_data)
+{
+ uint32_t reset_flags = prev_data->misc_reset_flags;
+
+ prev_data->misc_reset_flags &= ~ref_data->misc_reset_flags;
+
+ if ((reset_flags & I40E_HASH_FLOW_RESET_FLAG_REGION) &&
+ (ref_data->misc_reset_flags & I40E_HASH_FLOW_RESET_FLAG_REGION) &&
+ (prev_conf->region_queue_start != ref_conf->region_queue_start ||
+ prev_conf->region_queue_num != ref_conf->region_queue_num))
+ prev_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_REGION;
+
+ prev_data->reset_config_pctypes &= ~ref_data->reset_config_pctypes;
+ prev_data->reset_symmetric_pctypes &= ~ref_data->reset_symmetric_pctypes;
+}
+
+/*
+ * Recompute the owned-reset-flags metadata for every currently registered
+ * RSS flow. For example, flow A sets up RSS key, flow B sets up RSS hash
+ * function, and flow C sets up a different RSS key.
+ *
+ * When flow C registers, we need to make sure that flow A's reset flags do
+ * not include RSS key, because that is now owned by flow C. This is to make
+ * sure that if flow A is removed, RSS key configuration is not affected.
+ */
+static void
+i40e_hash_reconstruct_metadata(struct i40e_rss_state *state)
+{
+ struct i40e_rss_flow_node *node, *prev;
+
+ TAILQ_FOREACH(node, &state->list, next) {
+ struct i40e_flow_engine_hash_flow *hash_flow =
+ (struct i40e_flow_engine_hash_flow *)node->flow;
+
+ i40e_hash_compute_reset_flags(&hash_flow->rss_conf, &node->filter_data);
+ }
+
+ TAILQ_FOREACH(node, &state->list, next) {
+ struct i40e_flow_engine_hash_flow *hash_flow =
+ (struct i40e_flow_engine_hash_flow *)node->flow;
+
+ TAILQ_FOREACH(prev, &state->list, next) {
+ struct i40e_flow_engine_hash_flow *prev_hash_flow;
+
+ if (prev == node)
+ break;
+
+ prev_hash_flow = (struct i40e_flow_engine_hash_flow *)prev->flow;
+ i40e_hash_invalidate_prev(&hash_flow->rss_conf, &node->filter_data,
+ &prev_hash_flow->rss_conf, &prev->filter_data);
+ }
+ }
+}
+
+static int
+i40e_hash_flow_register(struct ci_flow *flow, struct rte_flow_error *error)
+{
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+ struct i40e_hash_priv *priv = flow->engine_priv;
+ struct i40e_rss_state *state = priv->state;
+ struct i40e_rss_flow_node *node;
+
+ node = rte_zmalloc("i40e_rss_flow_node", sizeof(*node), 0);
+ if (node == NULL) {
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, flow,
+ "Failed to allocate RSS flow tracking node");
+ }
+
+ node->flow = flow;
+ hash_flow->node = node;
+ TAILQ_INSERT_TAIL(&state->list, node, next);
+
+ i40e_hash_reconstruct_metadata(state);
+
+ return 0;
+}
+
+static int
+i40e_hash_flow_unregister(struct ci_flow *flow, struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+ struct i40e_hash_priv *priv = flow->engine_priv;
+ struct i40e_rss_state *state = priv->state;
+
+ TAILQ_REMOVE(&state->list, hash_flow->node, next);
+ rte_free(hash_flow->node);
+ hash_flow->node = NULL;
+
+ i40e_hash_reconstruct_metadata(state);
+
+ return 0;
+}
+
+static int
+i40e_hash_flow_install(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ int ret;
+
+ ret = i40e_hash_program_hw(pf, &hash_flow->rss_conf);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, flow,
+ "Failed to program hash filter");
+ }
+ return 0;
+}
+
+static int
+i40e_hash_flow_uninstall(struct ci_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+ struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(flow->dev_data->dev_private);
+ int ret;
+
+ ret = i40e_hash_deprogram_hw(pf, &hash_flow->node->filter_data);
+ if (ret) {
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, flow,
+ "Failed to reset hash filter");
+ }
+ return 0;
+}
+
+static int
+i40e_hash_flow_query(struct ci_flow *flow,
+ const struct rte_flow_action *action,
+ void *data,
+ struct rte_flow_error *error __rte_unused)
+{
+ struct i40e_flow_engine_hash_flow *hash_flow = (struct i40e_flow_engine_hash_flow *)flow;
+ struct i40e_rte_flow_rss_conf *rss_conf = data;
+
+ if (action->type != RTE_FLOW_ACTION_TYPE_RSS) {
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, action,
+ "Unsupported action for query");
+ }
+
+ memcpy(rss_conf, &hash_flow->rss_conf, sizeof(*rss_conf));
+ return 0;
+}
+
+static int
+i40e_hash_flow_engine_init(const struct ci_flow_engine *engine __rte_unused,
+ struct rte_eth_dev_data *dev_data,
+ void *priv)
+{
+ struct i40e_hash_priv *hash_priv = priv;
+
+ hash_priv->state = i40e_rss_state_attach(dev_data);
+ return 0;
+}
+
+static void
+i40e_hash_flow_engine_uninit(const struct ci_flow_engine *engine __rte_unused,
+ void *priv)
+{
+ struct i40e_hash_priv *hash_priv = priv;
+
+ i40e_rss_state_detach(hash_priv->state);
+}
+
+static const struct ci_flow_engine_ops i40e_flow_engine_hash_pattern_ops = {
+ .engine_init = i40e_hash_flow_engine_init,
+ .engine_uninit = i40e_hash_flow_engine_uninit,
+ .ctx_init = i40e_hash_pattern_ctx_init,
+ .ctx_finalize = i40e_hash_pattern_ctx_finalize,
+ .ctx_to_flow = i40e_hash_ctx_to_flow,
+ .flow_register = i40e_hash_flow_register,
+ .flow_unregister = i40e_hash_flow_unregister,
+ .flow_install = i40e_hash_flow_install,
+ .flow_uninstall = i40e_hash_flow_uninstall,
+ .flow_query = i40e_hash_flow_query,
+};
+
+static const struct ci_flow_engine_ops i40e_flow_engine_hash_vlan_ops = {
+ .engine_init = i40e_hash_flow_engine_init,
+ .engine_uninit = i40e_hash_flow_engine_uninit,
+ .ctx_init = i40e_hash_vlan_ctx_init,
+ .ctx_to_flow = i40e_hash_ctx_to_flow,
+ .flow_register = i40e_hash_flow_register,
+ .flow_unregister = i40e_hash_flow_unregister,
+ .flow_install = i40e_hash_flow_install,
+ .flow_uninstall = i40e_hash_flow_uninstall,
+ .flow_query = i40e_hash_flow_query,
+};
+
+static const struct ci_flow_engine_ops i40e_flow_engine_hash_empty_ops = {
+ .engine_init = i40e_hash_flow_engine_init,
+ .engine_uninit = i40e_hash_flow_engine_uninit,
+ .ctx_init = i40e_hash_empty_ctx_init,
+ .ctx_to_flow = i40e_hash_ctx_to_flow,
+ .flow_register = i40e_hash_flow_register,
+ .flow_unregister = i40e_hash_flow_unregister,
+ .flow_install = i40e_hash_flow_install,
+ .flow_uninstall = i40e_hash_flow_uninstall,
+ .flow_query = i40e_hash_flow_query,
+};
+
+const struct ci_flow_engine i40e_flow_engine_hash_pattern = {
+ .name = "hash_pattern",
+ .ops = &i40e_flow_engine_hash_pattern_ops,
+ .graph = &i40e_hash_pattern_graph,
+ .ctx_size = sizeof(struct i40e_hash_ctx),
+ .flow_size = sizeof(struct i40e_flow_engine_hash_flow),
+ .priv_size = sizeof(struct i40e_hash_priv),
+};
+
+const struct ci_flow_engine i40e_flow_engine_hash_vlan = {
+ .name = "hash_vlan",
+ .ops = &i40e_flow_engine_hash_vlan_ops,
+ .graph = &i40e_hash_vlan_graph,
+ .ctx_size = sizeof(struct i40e_hash_ctx),
+ .flow_size = sizeof(struct i40e_flow_engine_hash_flow),
+ .priv_size = sizeof(struct i40e_hash_priv),
+};
+
+const struct ci_flow_engine i40e_flow_engine_hash_empty = {
+ .name = "hash_empty",
+ .ops = &i40e_flow_engine_hash_empty_ops,
+ .ctx_size = sizeof(struct i40e_hash_ctx),
+ .flow_size = sizeof(struct i40e_flow_engine_hash_flow),
+ .priv_size = sizeof(struct i40e_hash_priv),
+};
diff --git a/drivers/net/intel/i40e/i40e_hash.c b/drivers/net/intel/i40e/i40e_hash.c
index 17adcbaa27..60efdaea60 100644
--- a/drivers/net/intel/i40e/i40e_hash.c
+++ b/drivers/net/intel/i40e/i40e_hash.c
@@ -2,239 +2,17 @@
* Copyright(c) 2020 Intel Corporation
*/
-#include <sys/queue.h>
#include <stdio.h>
#include <errno.h>
#include <stdint.h>
#include <string.h>
-#include <assert.h>
-#include <rte_malloc.h>
-#include <rte_tailq.h>
#include "base/i40e_prototype.h"
#include "i40e_logs.h"
#include "i40e_ethdev.h"
#include "i40e_hash.h"
#include "../common/flow_check.h"
-
-#ifndef BIT
-#define BIT(n) (1UL << (n))
-#endif
-
-#ifndef BIT_ULL
-#define BIT_ULL(n) (1ULL << (n))
-#endif
-
-/* Pattern item headers */
-#define I40E_HASH_HDR_ETH 0x01ULL
-#define I40E_HASH_HDR_IPV4 0x10ULL
-#define I40E_HASH_HDR_IPV6 0x20ULL
-#define I40E_HASH_HDR_IPV6_FRAG 0x40ULL
-#define I40E_HASH_HDR_TCP 0x100ULL
-#define I40E_HASH_HDR_UDP 0x200ULL
-#define I40E_HASH_HDR_SCTP 0x400ULL
-#define I40E_HASH_HDR_ESP 0x10000ULL
-#define I40E_HASH_HDR_L2TPV3 0x20000ULL
-#define I40E_HASH_HDR_AH 0x40000ULL
-#define I40E_HASH_HDR_GTPC 0x100000ULL
-#define I40E_HASH_HDR_GTPU 0x200000ULL
-
-#define I40E_HASH_HDR_INNER_SHIFT 32
-#define I40E_HASH_HDR_IPV4_INNER (I40E_HASH_HDR_IPV4 << \
- I40E_HASH_HDR_INNER_SHIFT)
-#define I40E_HASH_HDR_IPV6_INNER (I40E_HASH_HDR_IPV6 << \
- I40E_HASH_HDR_INNER_SHIFT)
-
-/* ETH */
-#define I40E_PHINT_ETH I40E_HASH_HDR_ETH
-
-/* IPv4 */
-#define I40E_PHINT_IPV4 (I40E_HASH_HDR_ETH | I40E_HASH_HDR_IPV4)
-#define I40E_PHINT_IPV4_TCP (I40E_PHINT_IPV4 | I40E_HASH_HDR_TCP)
-#define I40E_PHINT_IPV4_UDP (I40E_PHINT_IPV4 | I40E_HASH_HDR_UDP)
-#define I40E_PHINT_IPV4_SCTP (I40E_PHINT_IPV4 | I40E_HASH_HDR_SCTP)
-
-/* IPv6 */
-#define I40E_PHINT_IPV6 (I40E_HASH_HDR_ETH | I40E_HASH_HDR_IPV6)
-#define I40E_PHINT_IPV6_FRAG (I40E_PHINT_IPV6 | \
- I40E_HASH_HDR_IPV6_FRAG)
-#define I40E_PHINT_IPV6_TCP (I40E_PHINT_IPV6 | I40E_HASH_HDR_TCP)
-#define I40E_PHINT_IPV6_UDP (I40E_PHINT_IPV6 | I40E_HASH_HDR_UDP)
-#define I40E_PHINT_IPV6_SCTP (I40E_PHINT_IPV6 | I40E_HASH_HDR_SCTP)
-
-/* ESP */
-#define I40E_PHINT_IPV4_ESP (I40E_PHINT_IPV4 | I40E_HASH_HDR_ESP)
-#define I40E_PHINT_IPV6_ESP (I40E_PHINT_IPV6 | I40E_HASH_HDR_ESP)
-#define I40E_PHINT_IPV4_UDP_ESP (I40E_PHINT_IPV4_UDP | \
- I40E_HASH_HDR_ESP)
-#define I40E_PHINT_IPV6_UDP_ESP (I40E_PHINT_IPV6_UDP | \
- I40E_HASH_HDR_ESP)
-
-/* GTPC */
-#define I40E_PHINT_IPV4_GTPC (I40E_PHINT_IPV4_UDP | \
- I40E_HASH_HDR_GTPC)
-#define I40E_PHINT_IPV6_GTPC (I40E_PHINT_IPV6_UDP | \
- I40E_HASH_HDR_GTPC)
-
-/* GTPU */
-#define I40E_PHINT_IPV4_GTPU (I40E_PHINT_IPV4_UDP | \
- I40E_HASH_HDR_GTPU)
-#define I40E_PHINT_IPV4_GTPU_IPV4 (I40E_PHINT_IPV4_GTPU | \
- I40E_HASH_HDR_IPV4_INNER)
-#define I40E_PHINT_IPV4_GTPU_IPV6 (I40E_PHINT_IPV4_GTPU | \
- I40E_HASH_HDR_IPV6_INNER)
-#define I40E_PHINT_IPV6_GTPU (I40E_PHINT_IPV6_UDP | \
- I40E_HASH_HDR_GTPU)
-#define I40E_PHINT_IPV6_GTPU_IPV4 (I40E_PHINT_IPV6_GTPU | \
- I40E_HASH_HDR_IPV4_INNER)
-#define I40E_PHINT_IPV6_GTPU_IPV6 (I40E_PHINT_IPV6_GTPU | \
- I40E_HASH_HDR_IPV6_INNER)
-
-/* L2TPV3 */
-#define I40E_PHINT_IPV4_L2TPV3 (I40E_PHINT_IPV4 | I40E_HASH_HDR_L2TPV3)
-#define I40E_PHINT_IPV6_L2TPV3 (I40E_PHINT_IPV6 | I40E_HASH_HDR_L2TPV3)
-
-/* AH */
-#define I40E_PHINT_IPV4_AH (I40E_PHINT_IPV4 | I40E_HASH_HDR_AH)
-#define I40E_PHINT_IPV6_AH (I40E_PHINT_IPV6 | I40E_HASH_HDR_AH)
-
-/* Structure of mapping RSS type to input set */
-struct i40e_hash_map_rss_inset {
- uint64_t rss_type;
- uint64_t inset;
-};
-
-const struct i40e_hash_map_rss_inset i40e_hash_rss_inset[] = {
- /* IPv4 */
- { RTE_ETH_RSS_IPV4, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
- { RTE_ETH_RSS_FRAG_IPV4, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
-
- { RTE_ETH_RSS_NONFRAG_IPV4_OTHER,
- I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST },
-
- { RTE_ETH_RSS_NONFRAG_IPV4_TCP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
-
- { RTE_ETH_RSS_NONFRAG_IPV4_UDP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
-
- { RTE_ETH_RSS_NONFRAG_IPV4_SCTP, I40E_INSET_IPV4_SRC | I40E_INSET_IPV4_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT | I40E_INSET_SCTP_VT },
-
- /* IPv6 */
- { RTE_ETH_RSS_IPV6, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
- { RTE_ETH_RSS_FRAG_IPV6, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
-
- { RTE_ETH_RSS_NONFRAG_IPV6_OTHER,
- I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST },
-
- { RTE_ETH_RSS_NONFRAG_IPV6_TCP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
-
- { RTE_ETH_RSS_NONFRAG_IPV6_UDP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
-
- { RTE_ETH_RSS_NONFRAG_IPV6_SCTP, I40E_INSET_IPV6_SRC | I40E_INSET_IPV6_DST |
- I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT | I40E_INSET_SCTP_VT },
-
- /* Port */
- { RTE_ETH_RSS_PORT, I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT },
-
- /* Ether */
- { RTE_ETH_RSS_L2_PAYLOAD, I40E_INSET_LAST_ETHER_TYPE },
- { RTE_ETH_RSS_ETH, I40E_INSET_DMAC | I40E_INSET_SMAC },
-
- /* VLAN */
- { RTE_ETH_RSS_S_VLAN, I40E_INSET_VLAN_OUTER },
- { RTE_ETH_RSS_C_VLAN, I40E_INSET_VLAN_INNER },
-};
-
-#define I40E_HASH_VOID_NEXT_ALLOW BIT_ULL(RTE_FLOW_ITEM_TYPE_ETH)
-
-#define I40E_HASH_ETH_NEXT_ALLOW (BIT_ULL(RTE_FLOW_ITEM_TYPE_IPV4) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_IPV6) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_VLAN))
-
-#define I40E_HASH_IP_NEXT_ALLOW (BIT_ULL(RTE_FLOW_ITEM_TYPE_TCP) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_UDP) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_SCTP) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_ESP) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_L2TPV3OIP) |\
- BIT_ULL(RTE_FLOW_ITEM_TYPE_AH))
-
-#define I40E_HASH_IPV6_NEXT_ALLOW (I40E_HASH_IP_NEXT_ALLOW | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_IPV6_FRAG_EXT))
-
-#define I40E_HASH_UDP_NEXT_ALLOW (BIT_ULL(RTE_FLOW_ITEM_TYPE_GTPU) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_GTPC))
-
-#define I40E_HASH_GTPU_NEXT_ALLOW (BIT_ULL(RTE_FLOW_ITEM_TYPE_IPV4) | \
- BIT_ULL(RTE_FLOW_ITEM_TYPE_IPV6))
-
-static const uint64_t pattern_next_allow_items[] = {
- [RTE_FLOW_ITEM_TYPE_VOID] = I40E_HASH_VOID_NEXT_ALLOW,
- [RTE_FLOW_ITEM_TYPE_ETH] = I40E_HASH_ETH_NEXT_ALLOW,
- [RTE_FLOW_ITEM_TYPE_IPV4] = I40E_HASH_IP_NEXT_ALLOW,
- [RTE_FLOW_ITEM_TYPE_IPV6] = I40E_HASH_IPV6_NEXT_ALLOW,
- [RTE_FLOW_ITEM_TYPE_UDP] = I40E_HASH_UDP_NEXT_ALLOW,
- [RTE_FLOW_ITEM_TYPE_GTPU] = I40E_HASH_GTPU_NEXT_ALLOW,
-};
-
-static const uint64_t pattern_item_header[] = {
- [RTE_FLOW_ITEM_TYPE_ETH] = I40E_HASH_HDR_ETH,
- [RTE_FLOW_ITEM_TYPE_IPV4] = I40E_HASH_HDR_IPV4,
- [RTE_FLOW_ITEM_TYPE_IPV6] = I40E_HASH_HDR_IPV6,
- [RTE_FLOW_ITEM_TYPE_IPV6_FRAG_EXT] = I40E_HASH_HDR_IPV6_FRAG,
- [RTE_FLOW_ITEM_TYPE_TCP] = I40E_HASH_HDR_TCP,
- [RTE_FLOW_ITEM_TYPE_UDP] = I40E_HASH_HDR_UDP,
- [RTE_FLOW_ITEM_TYPE_SCTP] = I40E_HASH_HDR_SCTP,
- [RTE_FLOW_ITEM_TYPE_ESP] = I40E_HASH_HDR_ESP,
- [RTE_FLOW_ITEM_TYPE_GTPC] = I40E_HASH_HDR_GTPC,
- [RTE_FLOW_ITEM_TYPE_GTPU] = I40E_HASH_HDR_GTPU,
- [RTE_FLOW_ITEM_TYPE_L2TPV3OIP] = I40E_HASH_HDR_L2TPV3,
- [RTE_FLOW_ITEM_TYPE_AH] = I40E_HASH_HDR_AH,
-};
-
-/* Structure of matched pattern */
-struct i40e_hash_match_pattern {
- uint64_t pattern_type;
- uint64_t rss_mask; /* Supported RSS type for this pattern */
- bool custom_pctype_flag;/* true for custom packet type */
- uint8_t pctype;
-};
-
-#define I40E_HASH_MAP_PATTERN(pattern, rss_mask, pctype) { \
- pattern, rss_mask, false, pctype }
-
-#define I40E_HASH_MAP_CUS_PATTERN(pattern, rss_mask, cus_pctype) { \
- pattern, rss_mask, true, cus_pctype }
-
-#define I40E_HASH_L2_RSS_MASK (RTE_ETH_RSS_VLAN | RTE_ETH_RSS_ETH | \
- RTE_ETH_RSS_L2_SRC_ONLY | \
- RTE_ETH_RSS_L2_DST_ONLY)
-
-#define I40E_HASH_L23_RSS_MASK (I40E_HASH_L2_RSS_MASK | \
- RTE_ETH_RSS_L3_SRC_ONLY | \
- RTE_ETH_RSS_L3_DST_ONLY)
-
-#define I40E_HASH_IPV4_L23_RSS_MASK (RTE_ETH_RSS_IPV4 | I40E_HASH_L23_RSS_MASK)
-#define I40E_HASH_IPV6_L23_RSS_MASK (RTE_ETH_RSS_IPV6 | I40E_HASH_L23_RSS_MASK)
-
-#define I40E_HASH_L234_RSS_MASK (I40E_HASH_L23_RSS_MASK | \
- RTE_ETH_RSS_PORT | RTE_ETH_RSS_L4_SRC_ONLY | \
- RTE_ETH_RSS_L4_DST_ONLY)
-
-#define I40E_HASH_IPV4_L234_RSS_MASK (I40E_HASH_L234_RSS_MASK | RTE_ETH_RSS_IPV4)
-#define I40E_HASH_IPV6_L234_RSS_MASK (I40E_HASH_L234_RSS_MASK | RTE_ETH_RSS_IPV6)
-
-#define I40E_HASH_L4_TYPES (RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
- RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
- RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
- RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
- RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
- RTE_ETH_RSS_NONFRAG_IPV6_SCTP)
-
const uint8_t i40e_rss_key_default[] = {
0x44, 0x39, 0x79, 0x6b,
0xb5, 0x4c, 0x50, 0x23,
@@ -251,395 +29,6 @@ const uint8_t i40e_rss_key_default[] = {
0x81, 0x15, 0x03, 0x66
};
-/* Current supported patterns and RSS types.
- * All items that have the same pattern types are together.
- */
-static const struct i40e_hash_match_pattern match_patterns[] = {
- /* Ether */
- I40E_HASH_MAP_PATTERN(I40E_PHINT_ETH,
- RTE_ETH_RSS_L2_PAYLOAD | I40E_HASH_L2_RSS_MASK,
- I40E_FILTER_PCTYPE_L2_PAYLOAD),
-
- /* IPv4 */
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV4,
- RTE_ETH_RSS_FRAG_IPV4 | I40E_HASH_IPV4_L23_RSS_MASK,
- I40E_FILTER_PCTYPE_FRAG_IPV4),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV4,
- RTE_ETH_RSS_NONFRAG_IPV4_OTHER |
- I40E_HASH_IPV4_L23_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV4_OTHER),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV4_TCP,
- RTE_ETH_RSS_NONFRAG_IPV4_TCP |
- I40E_HASH_IPV4_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV4_TCP),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV4_UDP,
- RTE_ETH_RSS_NONFRAG_IPV4_UDP |
- I40E_HASH_IPV4_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV4_UDP),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV4_SCTP,
- RTE_ETH_RSS_NONFRAG_IPV4_SCTP |
- I40E_HASH_IPV4_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV4_SCTP),
-
- /* IPv6 */
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6,
- RTE_ETH_RSS_FRAG_IPV6 | I40E_HASH_IPV6_L23_RSS_MASK,
- I40E_FILTER_PCTYPE_FRAG_IPV6),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6,
- RTE_ETH_RSS_NONFRAG_IPV6_OTHER |
- I40E_HASH_IPV6_L23_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV6_OTHER),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6_FRAG,
- RTE_ETH_RSS_FRAG_IPV6 | I40E_HASH_L23_RSS_MASK,
- I40E_FILTER_PCTYPE_FRAG_IPV6),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6_TCP,
- RTE_ETH_RSS_NONFRAG_IPV6_TCP |
- I40E_HASH_IPV6_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV6_TCP),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6_UDP,
- RTE_ETH_RSS_NONFRAG_IPV6_UDP |
- I40E_HASH_IPV6_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV6_UDP),
-
- I40E_HASH_MAP_PATTERN(I40E_PHINT_IPV6_SCTP,
- RTE_ETH_RSS_NONFRAG_IPV6_SCTP |
- I40E_HASH_IPV6_L234_RSS_MASK,
- I40E_FILTER_PCTYPE_NONF_IPV6_SCTP),
-
- /* ESP */
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_ESP,
- RTE_ETH_RSS_ESP, I40E_CUSTOMIZED_ESP_IPV4),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_ESP,
- RTE_ETH_RSS_ESP, I40E_CUSTOMIZED_ESP_IPV6),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_UDP_ESP,
- RTE_ETH_RSS_ESP, I40E_CUSTOMIZED_ESP_IPV4_UDP),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_UDP_ESP,
- RTE_ETH_RSS_ESP, I40E_CUSTOMIZED_ESP_IPV6_UDP),
-
- /* GTPC */
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_GTPC,
- I40E_HASH_IPV4_L234_RSS_MASK,
- I40E_CUSTOMIZED_GTPC),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_GTPC,
- I40E_HASH_IPV6_L234_RSS_MASK,
- I40E_CUSTOMIZED_GTPC),
-
- /* GTPU */
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_GTPU,
- I40E_HASH_IPV4_L234_RSS_MASK,
- I40E_CUSTOMIZED_GTPU),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_GTPU_IPV4,
- RTE_ETH_RSS_GTPU, I40E_CUSTOMIZED_GTPU_IPV4),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_GTPU_IPV6,
- RTE_ETH_RSS_GTPU, I40E_CUSTOMIZED_GTPU_IPV6),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_GTPU,
- I40E_HASH_IPV6_L234_RSS_MASK,
- I40E_CUSTOMIZED_GTPU),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_GTPU_IPV4,
- RTE_ETH_RSS_GTPU, I40E_CUSTOMIZED_GTPU_IPV4),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_GTPU_IPV6,
- RTE_ETH_RSS_GTPU, I40E_CUSTOMIZED_GTPU_IPV6),
-
- /* L2TPV3 */
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_L2TPV3,
- RTE_ETH_RSS_L2TPV3, I40E_CUSTOMIZED_IPV4_L2TPV3),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_L2TPV3,
- RTE_ETH_RSS_L2TPV3, I40E_CUSTOMIZED_IPV6_L2TPV3),
-
- /* AH */
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV4_AH, RTE_ETH_RSS_AH,
- I40E_CUSTOMIZED_AH_IPV4),
- I40E_HASH_MAP_CUS_PATTERN(I40E_PHINT_IPV6_AH, RTE_ETH_RSS_AH,
- I40E_CUSTOMIZED_AH_IPV6),
-};
-
-static int
-i40e_hash_get_pattern_type(const struct rte_flow_item pattern[],
- uint64_t *pattern_types,
- struct rte_flow_error *error)
-{
- const char *message = "Pattern not supported";
- enum rte_flow_item_type prev_item_type = RTE_FLOW_ITEM_TYPE_VOID;
- enum rte_flow_item_type last_item_type = prev_item_type;
- uint64_t item_hdr, pattern_hdrs = 0;
- bool inner_flag = false;
- int vlan_count = 0;
-
- for (; pattern->type != RTE_FLOW_ITEM_TYPE_END; pattern++) {
- if (pattern->type == RTE_FLOW_ITEM_TYPE_VOID)
- continue;
-
- if (pattern->mask || pattern->spec || pattern->last) {
- message = "Header info should not be specified";
- goto not_sup;
- }
-
- /* Check the previous item allows this sub-item. */
- if (prev_item_type >= (enum rte_flow_item_type)
- RTE_DIM(pattern_next_allow_items) ||
- !(pattern_next_allow_items[prev_item_type] &
- BIT_ULL(pattern->type)))
- goto not_sup;
-
- /* For VLAN item, it does no matter about to pattern type
- * recognition. So just count the number of VLAN and do not
- * change the value of variable `prev_item_type`.
- */
- last_item_type = pattern->type;
- if (last_item_type == RTE_FLOW_ITEM_TYPE_VLAN) {
- if (vlan_count >= 2)
- goto not_sup;
- vlan_count++;
- continue;
- }
-
- prev_item_type = last_item_type;
- if (last_item_type >= (enum rte_flow_item_type)
- RTE_DIM(pattern_item_header))
- goto not_sup;
-
- item_hdr = pattern_item_header[last_item_type];
- assert(item_hdr);
-
- if (inner_flag) {
- item_hdr <<= I40E_HASH_HDR_INNER_SHIFT;
-
- /* Inner layer should not have GTPU item */
- if (last_item_type == RTE_FLOW_ITEM_TYPE_GTPU)
- goto not_sup;
- } else {
- if (last_item_type == RTE_FLOW_ITEM_TYPE_GTPU) {
- inner_flag = true;
- vlan_count = 0;
- }
- }
-
- if (item_hdr & pattern_hdrs)
- goto not_sup;
-
- pattern_hdrs |= item_hdr;
- }
-
- if (pattern_hdrs && last_item_type != RTE_FLOW_ITEM_TYPE_VLAN) {
- *pattern_types = pattern_hdrs;
- return 0;
- }
-
-not_sup:
- return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM,
- pattern, message);
-}
-
-static uint64_t
-i40e_hash_get_x722_ext_pctypes(uint8_t match_pctype)
-{
- uint64_t pctypes = 0;
-
- switch (match_pctype) {
- case I40E_FILTER_PCTYPE_NONF_IPV4_TCP:
- pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_IPV4_TCP_SYN_NO_ACK);
- break;
-
- case I40E_FILTER_PCTYPE_NONF_IPV4_UDP:
- pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_UNICAST_IPV4_UDP) |
- BIT_ULL(I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV4_UDP);
- break;
-
- case I40E_FILTER_PCTYPE_NONF_IPV6_TCP:
- pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_IPV6_TCP_SYN_NO_ACK);
- break;
-
- case I40E_FILTER_PCTYPE_NONF_IPV6_UDP:
- pctypes = BIT_ULL(I40E_FILTER_PCTYPE_NONF_UNICAST_IPV6_UDP) |
- BIT_ULL(I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV6_UDP);
- break;
- }
-
- return pctypes;
-}
-
-static int
-i40e_hash_translate_gtp_inset(struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- if (rss_conf->inset &
- (I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC |
- I40E_INSET_DST_PORT | I40E_INSET_SRC_PORT))
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF,
- NULL,
- "Only support external destination IP");
-
- if (rss_conf->inset & I40E_INSET_IPV4_DST)
- rss_conf->inset = (rss_conf->inset & ~I40E_INSET_IPV4_DST) |
- I40E_INSET_TUNNEL_IPV4_DST;
-
- if (rss_conf->inset & I40E_INSET_IPV6_DST)
- rss_conf->inset = (rss_conf->inset & ~I40E_INSET_IPV6_DST) |
- I40E_INSET_TUNNEL_IPV6_DST;
-
- return 0;
-}
-
-static int
-i40e_hash_get_pctypes(const struct rte_eth_dev *dev,
- const struct i40e_hash_match_pattern *match,
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- if (match->custom_pctype_flag) {
- struct i40e_pf *pf;
- struct i40e_customized_pctype *custom_type;
-
- pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
- custom_type = i40e_find_customized_pctype(pf, match->pctype);
- if (!custom_type || !custom_type->valid)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "PCTYPE not supported");
-
- rss_conf->config_pctypes |= BIT_ULL(custom_type->pctype);
-
- if (match->pctype == I40E_CUSTOMIZED_GTPU ||
- match->pctype == I40E_CUSTOMIZED_GTPC)
- return i40e_hash_translate_gtp_inset(rss_conf, error);
- } else {
- struct i40e_hw *hw =
- I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
- uint64_t types;
-
- rss_conf->config_pctypes |= BIT_ULL(match->pctype);
- if (hw->mac.type == I40E_MAC_X722) {
- types = i40e_hash_get_x722_ext_pctypes(match->pctype);
- rss_conf->config_pctypes |= types;
- }
- }
-
- return 0;
-}
-
-static int
-i40e_hash_get_pattern_pctypes(const struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action_rss *rss_act,
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- uint64_t pattern_types = 0;
- bool match_flag = false;
- int i, ret;
-
- ret = i40e_hash_get_pattern_type(pattern, &pattern_types, error);
- if (ret)
- return ret;
-
- for (i = 0; i < (int)RTE_DIM(match_patterns); i++) {
- const struct i40e_hash_match_pattern *match =
- &match_patterns[i];
-
- /* Check pattern types match. All items that have the same
- * pattern types are together, so if the pattern types match
- * previous item but they doesn't match current item, it means
- * the pattern types do not match all remain items.
- */
- if (pattern_types != match->pattern_type) {
- if (match_flag)
- break;
- continue;
- }
- match_flag = true;
-
- /* Check RSS types match */
- if (!(rss_act->types & ~match->rss_mask)) {
- ret = i40e_hash_get_pctypes(dev, match,
- rss_conf, error);
- if (ret)
- return ret;
- }
- }
-
- if (rss_conf->config_pctypes)
- return 0;
-
- if (match_flag)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF,
- NULL, "RSS types not supported");
-
- return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM,
- NULL, "Pattern not supported");
-}
-
-static uint64_t
-i40e_hash_get_inset(uint64_t rss_types, bool symmetric_enable)
-{
- uint64_t mask, inset = 0;
- int i;
-
- for (i = 0; i < (int)RTE_DIM(i40e_hash_rss_inset); i++) {
- if (rss_types & i40e_hash_rss_inset[i].rss_type)
- inset |= i40e_hash_rss_inset[i].inset;
- }
-
- if (!inset)
- return 0;
-
- /* If SRC_ONLY and DST_ONLY of the same level are used simultaneously,
- * it is the same case as none of them are added.
- */
- mask = rss_types & (RTE_ETH_RSS_L2_SRC_ONLY | RTE_ETH_RSS_L2_DST_ONLY);
- if (mask == RTE_ETH_RSS_L2_SRC_ONLY)
- inset &= ~I40E_INSET_DMAC;
- else if (mask == RTE_ETH_RSS_L2_DST_ONLY)
- inset &= ~I40E_INSET_SMAC;
-
- mask = rss_types & (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY);
- if (mask == RTE_ETH_RSS_L3_SRC_ONLY)
- inset &= ~(I40E_INSET_IPV4_DST | I40E_INSET_IPV6_DST);
- else if (mask == RTE_ETH_RSS_L3_DST_ONLY)
- inset &= ~(I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC);
-
- mask = rss_types & (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY);
- if (mask == RTE_ETH_RSS_L4_SRC_ONLY)
- inset &= ~I40E_INSET_DST_PORT;
- else if (mask == RTE_ETH_RSS_L4_DST_ONLY)
- inset &= ~I40E_INSET_SRC_PORT;
-
- if (rss_types & I40E_HASH_L4_TYPES) {
- uint64_t l3_mask = rss_types &
- (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY);
- uint64_t l4_mask = rss_types &
- (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY);
-
- if (l3_mask && !l4_mask)
- inset &= ~(I40E_INSET_SRC_PORT | I40E_INSET_DST_PORT);
- else if (!l3_mask && l4_mask)
- inset &= ~(I40E_INSET_IPV4_DST | I40E_INSET_IPV6_DST |
- I40E_INSET_IPV4_SRC | I40E_INSET_IPV6_SRC);
- }
-
- /* SCTP Verification Tag is not required in hash computation for SYMMETRIC_TOEPLITZ */
- if (symmetric_enable) {
- mask = rss_types & RTE_ETH_RSS_NONFRAG_IPV4_SCTP;
- if (mask == RTE_ETH_RSS_NONFRAG_IPV4_SCTP)
- inset &= ~I40E_INSET_SCTP_VT;
-
- mask = rss_types & RTE_ETH_RSS_NONFRAG_IPV6_SCTP;
- if (mask == RTE_ETH_RSS_NONFRAG_IPV6_SCTP)
- inset &= ~I40E_INSET_SCTP_VT;
- }
-
- return inset;
-}
-
static int
i40e_hash_config_func(struct i40e_hw *hw, enum rte_eth_hash_function func)
{
@@ -825,12 +214,10 @@ i40e_hash_config_region(struct i40e_pf *pf,
return i40e_flush_queue_region_all_conf(dev, hw, pf, 1);
}
-static int
-i40e_hash_config(struct i40e_pf *pf,
- struct i40e_rss_filter *filter)
+int
+i40e_hash_program_hw(struct i40e_pf *pf,
+ struct i40e_rte_flow_rss_conf *rss_conf)
{
- struct i40e_rte_flow_rss_conf *rss_conf = &filter->rss_filter_info;
- struct i40e_rss_filter_data *filter_data = &filter->filter_data;
struct i40e_hw *hw = &pf->adapter->hw;
uint64_t pctypes;
int ret;
@@ -839,18 +226,12 @@ i40e_hash_config(struct i40e_pf *pf,
ret = i40e_hash_config_func(hw, rss_conf->func);
if (ret)
return ret;
-
- if (rss_conf->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ)
- filter_data->misc_reset_flags |=
- I40E_HASH_FLOW_RESET_FLAG_FUNC;
}
if (rss_conf->region_queue_num > 0) {
ret = i40e_hash_config_region(pf, rss_conf);
if (ret)
return ret;
-
- filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_REGION;
}
if (rss_conf->key_len > 0) {
@@ -858,8 +239,6 @@ i40e_hash_config(struct i40e_pf *pf,
rss_conf->key_len);
if (ret)
return ret;
-
- filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_KEY;
}
/* Update lookup table */
@@ -881,7 +260,6 @@ i40e_hash_config(struct i40e_pf *pf,
pf->hash_enabled_queues |= BIT_ULL(lut[i]);
pf->adapter->rss_reta_updated = 0;
- filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_QUEUE;
}
/* The codes behind configure the input sets and symmetric hash
@@ -901,504 +279,70 @@ i40e_hash_config(struct i40e_pf *pf,
do {
uint32_t idx = rte_bsf64(pctypes);
- uint64_t bit = BIT_ULL(idx);
if (rss_conf->symmetric_enable) {
ret = i40e_hash_config_pctype_symmetric(hw, idx, true);
if (ret)
return ret;
-
- filter_data->reset_symmetric_pctypes |= bit;
}
ret = i40e_hash_config_pctype(hw, rss_conf, idx);
if (ret)
return ret;
+ pctypes &= ~BIT_ULL(idx);
+ } while (pctypes);
+
+ return 0;
+}
+
+/*
+ * Compute the reset flags a filter would own if it were the only RSS filter
+ * installed. Pure function of rss_conf; used to reconstruct the ownership of
+ * shared RSS state across all currently registered filters.
+ */
+void
+i40e_hash_compute_reset_flags(const struct i40e_rte_flow_rss_conf *rss_conf,
+ struct i40e_rss_filter_data *filter_data)
+{
+ uint64_t pctypes;
+
+ *filter_data = (struct i40e_rss_filter_data){0};
+
+ if (rss_conf->func != RTE_ETH_HASH_FUNCTION_DEFAULT &&
+ rss_conf->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ)
+ filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_FUNC;
+
+ if (rss_conf->region_queue_num > 0)
+ filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_REGION;
+
+ if (rss_conf->key_len > 0)
+ filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_KEY;
+
+ if (rss_conf->queue_num > 0)
+ filter_data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_QUEUE;
+
+ pctypes = rss_conf->config_pctypes;
+ while (pctypes) {
+ uint32_t idx = rte_bsf64(pctypes);
+ uint64_t bit = BIT_ULL(idx);
+
filter_data->reset_config_pctypes |= bit;
+ if (rss_conf->symmetric_enable)
+ filter_data->reset_symmetric_pctypes |= bit;
+
pctypes &= ~bit;
- } while (pctypes);
-
- return 0;
-}
-
-static void
-i40e_hash_parse_key(const struct rte_flow_action_rss *rss_act,
- struct i40e_rte_flow_rss_conf *rss_conf)
-{
- const uint8_t *key = rss_act->key;
-
- if (key == NULL) {
- memcpy(rss_conf->key, i40e_rss_key_default, sizeof(rss_conf->key));
- } else {
- memcpy(rss_conf->key, key, sizeof(rss_conf->key));
}
-
- rss_conf->key_len = sizeof(rss_conf->key);
-}
-
-static int
-i40e_hash_parse_pattern_act(const struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action_rss *rss_act,
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- rss_conf->symmetric_enable = rss_act->func == RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ;
-
- if (rss_act->key_len)
- i40e_hash_parse_key(rss_act, rss_conf);
-
- rss_conf->func = rss_act->func;
- rss_conf->types = rss_act->types;
- rss_conf->inset = i40e_hash_get_inset(rss_act->types, rss_conf->symmetric_enable);
-
- return i40e_hash_get_pattern_pctypes(dev, pattern, rss_act,
- rss_conf, error);
-}
-
-static int
-i40e_hash_parse_queues(const struct rte_flow_action_rss *rss_act,
- struct i40e_rte_flow_rss_conf *rss_conf)
-{
- memcpy(rss_conf->queue, rss_act->queue,
- rss_act->queue_num * sizeof(rss_conf->queue[0]));
- rss_conf->queue_num = rss_act->queue_num;
- return 0;
-}
-
-static int
-i40e_hash_parse_queue_region(const struct rte_flow_item pattern[],
- const struct rte_flow_action_rss *rss_act,
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- const struct rte_flow_item_vlan *vlan_spec, *vlan_mask;
-
- vlan_spec = pattern->spec;
- vlan_mask = pattern->mask;
-
- /* VLAN must have spec and mask */
- if (vlan_spec == NULL || vlan_mask == NULL) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM, &pattern[0],
- "VLAN pattern spec and mask required");
- }
- /* for mask, VLAN/TCI must be masked appropriately */
- if ((rte_be_to_cpu_16(vlan_mask->hdr.vlan_tci) >> 13) != 0x7) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ITEM, &pattern[0],
- "VLAN pattern mask invalid");
- }
-
- /* Use a 64 bit variable to represent all queues in a region. */
- RTE_BUILD_BUG_ON(I40E_MAX_Q_PER_TC > 64);
-
- rss_conf->region_queue_num = (uint8_t)rss_act->queue_num;
- rss_conf->region_queue_start = rss_act->queue[0];
- rss_conf->region_priority = rte_be_to_cpu_16(vlan_spec->hdr.vlan_tci) >> 13;
- return 0;
-}
-
-static bool
-i40e_hash_validate_rss_types(uint64_t rss_types)
-{
- uint64_t type, mask;
-
- /* Validate L2 */
- type = RTE_ETH_RSS_ETH & rss_types;
- mask = (RTE_ETH_RSS_L2_SRC_ONLY | RTE_ETH_RSS_L2_DST_ONLY) & rss_types;
- if (!type && mask)
- return false;
-
- /* Validate L3 */
- type = (I40E_HASH_L4_TYPES | RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4 |
- RTE_ETH_RSS_NONFRAG_IPV4_OTHER | RTE_ETH_RSS_IPV6 |
- RTE_ETH_RSS_FRAG_IPV6 | RTE_ETH_RSS_NONFRAG_IPV6_OTHER) & rss_types;
- mask = (RTE_ETH_RSS_L3_SRC_ONLY | RTE_ETH_RSS_L3_DST_ONLY) & rss_types;
- if (!type && mask)
- return false;
-
- /* Validate L4 */
- type = (I40E_HASH_L4_TYPES | RTE_ETH_RSS_PORT) & rss_types;
- mask = (RTE_ETH_RSS_L4_SRC_ONLY | RTE_ETH_RSS_L4_DST_ONLY) & rss_types;
- if (!type && mask)
- return false;
-
- return true;
-}
-
-static int
-i40e_hash_validate_rss_pattern(const struct ci_flow_actions *actions,
- const struct ci_flow_actions_check_param *param __rte_unused,
- struct rte_flow_error *error)
-{
- const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
-
- /* queue list is not supported */
- if (rss_act->queue_num != 0) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS queues not supported when pattern specified");
- }
-
- /* disallow unsupported hash functions */
- switch (rss_act->func) {
- case RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ:
- case RTE_ETH_HASH_FUNCTION_DEFAULT:
- case RTE_ETH_HASH_FUNCTION_TOEPLITZ:
- case RTE_ETH_HASH_FUNCTION_SIMPLE_XOR:
- break;
- default:
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS hash function not supported when pattern specified");
- }
-
- if (!i40e_hash_validate_rss_types(rss_act->types))
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF,
- rss_act, "RSS types are invalid");
-
- /* check RSS key length if it is specified */
- if (rss_act->key_len != 0 && rss_act->key_len != I40E_RSS_KEY_LEN) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS key length must be 52 bytes");
- }
-
- return 0;
-}
-
-static int
-i40e_hash_validate_rss_common(const struct rte_flow_action_rss *rss_act,
- struct rte_flow_error *error)
-{
- /* RSS level is not supported */
- if (rss_act->level != 0) {
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS level is not supported");
- }
-
- /* for empty patterns, symmetric toeplitz is not supported */
- if (rss_act->func == RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "Symmetric hash function not supported without specific patterns");
- }
-
- /*
- * When RSS types is not specified in testpmd, it will set up a default
- * RSS types value for the flow. Even though no hash engine part calling
- * this particular function will use RSS types parameter for anything,
- * we cannot reject having it because it is extra effort for testpmd
- * user to avoid specifying it.
- *
- * So, instead, accept types value even though we are not using it for
- * anything, but produce a warning for the user.
- */
- if (rss_act->types != 0)
- PMD_DRV_LOG(WARNING, "RSS types specified but will not be used");
-
- /* check RSS key length if it is specified */
- if (rss_act->key_len != 0 && rss_act->key_len != I40E_RSS_KEY_LEN) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS key length must be 52 bytes");
- }
-
- return 0;
-}
-
-static int
-i40e_hash_validate_queue_region(const struct ci_flow_actions *actions,
- const struct ci_flow_actions_check_param *param,
- struct rte_flow_error *error)
-{
- const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
- struct i40e_adapter *adapter = I40E_DEV_PRIVATE_TO_ADAPTER(param->driver_ctx);
- struct i40e_pf *pf = &adapter->pf;
- uint64_t hash_queues;
- int ret;
-
- ret = i40e_hash_validate_rss_common(rss_act, error);
- if (ret)
- return ret;
-
- RTE_BUILD_BUG_ON(sizeof(hash_queues) != sizeof(pf->hash_enabled_queues));
-
- /* having RSS key is not supported */
- if (rss_act->key != NULL) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS key not supported");
- }
-
- /* queue region must be specified */
- if (rss_act->queue_num == 0) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS queues missing");
- }
-
- /* queue region must be power of two */
- if (!rte_is_power_of_2(rss_act->queue_num)) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS queue number must be power of two");
- }
-
- /* generic checks already filtered out discontiguous/non-unique RSS queues */
-
- /* queues must not exceed maximum queues per traffic class */
- if (rss_act->queue[rss_act->queue_num - 1] >= I40E_MAX_Q_PER_TC) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "Invalid RSS queue index");
- }
-
- /* queues must be in LUT */
- hash_queues = (BIT_ULL(rss_act->queue[0] + rss_act->queue_num) - 1) &
- ~(BIT_ULL(rss_act->queue[0]) - 1);
-
- if (hash_queues & ~pf->hash_enabled_queues) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF,
- rss_act, "Some queues are not in LUT");
- }
-
- return 0;
-}
-
-static int
-i40e_hash_validate_queue_list(const struct ci_flow_actions *actions,
- const struct ci_flow_actions_check_param *param,
- struct rte_flow_error *error)
-{
- const struct rte_flow_action_rss *rss_act = actions->actions[0]->conf;
- struct i40e_adapter *adapter = I40E_DEV_PRIVATE_TO_ADAPTER(param->driver_ctx);
- struct i40e_pf *pf;
- struct i40e_hw *hw;
- uint16_t max_queue;
- bool has_queue, has_key;
- int ret;
-
- ret = i40e_hash_validate_rss_common(rss_act, error);
- if (ret)
- return ret;
-
- has_queue = rss_act->queue != NULL;
- has_key = rss_act->key != NULL;
-
- /* if we have queues, we must not have key */
- if (has_queue && has_key) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "RSS key for queue region is not supported");
- }
-
- /* if there are no queues, no further checks needed */
- if (!has_queue)
- return 0;
-
- /* check queue number limits */
- hw = &adapter->hw;
- if (rss_act->queue_num > hw->func_caps.rss_table_size) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF,
- rss_act, "Too many RSS queues");
- }
-
- pf = &adapter->pf;
- if (pf->dev_data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_VMDQ_FLAG)
- max_queue = i40e_pf_calc_configured_queues_num(pf);
- else
- max_queue = pf->dev_data->nb_rx_queues;
-
- max_queue = RTE_MIN(max_queue, I40E_MAX_Q_PER_TC);
-
- /* we know RSS queues are contiguous so we only need to check last queue */
- if (rss_act->queue[rss_act->queue_num - 1] >= max_queue) {
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION_CONF, rss_act,
- "Invalid RSS queue");
- }
-
- return 0;
-}
-
-int
-i40e_hash_parse(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error)
-{
- struct ci_flow_actions parsed_actions;
- struct ci_flow_actions_check_param ac_param = {
- .allowed_types = (enum rte_flow_action_type[]) {
- RTE_FLOW_ACTION_TYPE_RSS,
- RTE_FLOW_ACTION_TYPE_END
- },
- .max_actions = 1,
- .driver_ctx = dev->data->dev_private,
- /* each pattern type will add specific check function */
- };
- const struct rte_flow_action_rss *rss_act;
- int ret;
-
- /*
- * We have two possible paths: global RSS configuration, and an RSS pattern action.
- *
- * For global patterns, we act on two types of flows:
- * - Empty pattern ([END])
- * - VLAN pattern ([VLAN] -> [END])
- *
- * Everything else is handled by pattern action parser.
- */
- bool is_empty, is_vlan;
-
- while (pattern->type == RTE_FLOW_ITEM_TYPE_VOID)
- pattern++;
-
- is_empty = pattern[0].type == RTE_FLOW_ITEM_TYPE_END;
- is_vlan = pattern[0].type == RTE_FLOW_ITEM_TYPE_VLAN &&
- pattern[1].type == RTE_FLOW_ITEM_TYPE_END;
-
- /* VLAN path */
- if (is_vlan) {
- ac_param.check = i40e_hash_validate_queue_region;
- /* queue regions must be contiguous */
- ac_param.rss_queues_contig = true;
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- rss_act = parsed_actions.actions[0]->conf;
- /* set up RSS functions */
- rss_conf->func = rss_act->func;
- return i40e_hash_parse_queue_region(pattern, rss_act, rss_conf, error);
- }
- /* Empty pattern path */
- if (is_empty) {
- ac_param.check = i40e_hash_validate_queue_list;
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- rss_act = parsed_actions.actions[0]->conf;
- rss_conf->func = rss_act->func;
- /* if there is a queue list, take that path */
- if (rss_act->queue != NULL)
- return i40e_hash_parse_queues(rss_act, rss_conf);
-
- /* otherwise just parse RSS key */
- if (rss_act->key != NULL)
- i40e_hash_parse_key(rss_act, rss_conf);
-
- return 0;
- }
- ac_param.check = i40e_hash_validate_rss_pattern;
- ret = ci_flow_check_actions(actions, &ac_param, &parsed_actions, error);
- if (ret)
- return ret;
- rss_act = parsed_actions.actions[0]->conf;
-
- /* pattern case */
- return i40e_hash_parse_pattern_act(dev, pattern, rss_act, rss_conf, error);
-}
-
-static void
-i40e_invalid_rss_filter(const struct i40e_rss_filter *ref,
- struct i40e_rss_filter *filter)
-{
- const struct i40e_rte_flow_rss_conf *ref_conf = &ref->rss_filter_info;
- const struct i40e_rss_filter_data *ref_data = &ref->filter_data;
- const struct i40e_rte_flow_rss_conf *conf = &filter->rss_filter_info;
- struct i40e_rss_filter_data *data = &filter->filter_data;
- uint32_t reset_flags = data->misc_reset_flags;
-
- data->misc_reset_flags &= ~ref_data->misc_reset_flags;
-
- if ((reset_flags & I40E_HASH_FLOW_RESET_FLAG_REGION) &&
- (ref_data->misc_reset_flags & I40E_HASH_FLOW_RESET_FLAG_REGION) &&
- (conf->region_queue_start != ref_conf->region_queue_start ||
- conf->region_queue_num != ref_conf->region_queue_num))
- data->misc_reset_flags |= I40E_HASH_FLOW_RESET_FLAG_REGION;
-
- data->reset_config_pctypes &= ~ref_data->reset_config_pctypes;
- data->reset_symmetric_pctypes &= ~ref_data->reset_symmetric_pctypes;
-}
-
-int
-i40e_hash_filter_restore(struct i40e_pf *pf)
-{
- struct i40e_rss_filter *filter;
- int ret;
-
- TAILQ_FOREACH(filter, &pf->rss_config_list, next) {
- struct i40e_rss_filter *prev;
-
- filter->filter_data = (struct i40e_rss_filter_data){0};
-
- ret = i40e_hash_config(pf, filter);
- if (ret) {
- pf->hash_filter_enabled = 0;
- i40e_pf_disable_rss(pf);
- PMD_DRV_LOG(ERR,
- "Re-configure RSS failed, RSS has been disabled");
- return ret;
- }
-
- /* Invalid previous RSS filter */
- TAILQ_FOREACH(prev, &pf->rss_config_list, next) {
- if (prev == filter)
- break;
- i40e_invalid_rss_filter(filter, prev);
- }
- }
-
- return 0;
}
+/*
+ * Revert to defaults only the registers this filter's filter_data records as
+ * owned. This is the inverse of i40e_hash_program_hw, but only undoes the
+ * owned subset -- it is not a full RSS wipe.
+ */
int
-i40e_hash_filter_create(struct i40e_pf *pf,
- struct i40e_rte_flow_rss_conf *rss_conf)
-{
- struct i40e_rss_filter *filter, *prev;
- struct i40e_rte_flow_rss_conf *new_conf;
- int ret;
-
- filter = rte_zmalloc("i40e_rss_filter", sizeof(*filter), 0);
- if (!filter) {
- PMD_DRV_LOG(ERR, "Failed to allocate memory.");
- return -ENOMEM;
- }
-
- new_conf = &filter->rss_filter_info;
-
- memcpy(new_conf, rss_conf, sizeof(*new_conf));
-
- ret = i40e_hash_config(pf, filter);
- if (ret) {
- rte_free(filter);
- if (i40e_pf_config_rss(pf))
- return ret;
-
- (void)i40e_hash_filter_restore(pf);
- return ret;
- }
-
- /* Invalid previous RSS filter */
- TAILQ_FOREACH(prev, &pf->rss_config_list, next)
- i40e_invalid_rss_filter(filter, prev);
-
- TAILQ_INSERT_TAIL(&pf->rss_config_list, filter, next);
- return 0;
-}
-
-static int
-i40e_hash_reset_conf(struct i40e_pf *pf,
- struct i40e_rss_filter_data *filter_data)
+i40e_hash_deprogram_hw(struct i40e_pf *pf,
+ struct i40e_rss_filter_data *filter_data)
{
struct i40e_hw *hw = &pf->adapter->hw;
struct rte_eth_dev *dev;
@@ -1468,55 +412,3 @@ i40e_hash_reset_conf(struct i40e_pf *pf,
return 0;
}
-
-int
-i40e_hash_filter_destroy(struct i40e_pf *pf,
- const struct i40e_rss_filter *rss_filter)
-{
- struct i40e_rss_filter *filter;
- int ret;
-
- TAILQ_FOREACH(filter, &pf->rss_config_list, next) {
- if (rss_filter == filter) {
- ret = i40e_hash_reset_conf(pf, &filter->filter_data);
- if (ret)
- return ret;
-
- TAILQ_REMOVE(&pf->rss_config_list, filter, next);
- rte_free(filter);
- return 0;
- }
- }
-
- return -ENOENT;
-}
-
-int
-i40e_hash_filter_flush(struct i40e_pf *pf)
-{
- struct rte_flow *flow, *next;
-
- RTE_TAILQ_FOREACH_SAFE(flow, &pf->flow_list, node, next) {
- if (flow->filter_type != RTE_ETH_FILTER_HASH)
- continue;
-
- if (flow->rule) {
- struct i40e_rss_filter *filter = flow->rule;
- int ret;
-
- ret = i40e_hash_reset_conf(pf,
- &filter->filter_data);
- if (ret)
- return ret;
-
- TAILQ_REMOVE(&pf->rss_config_list, filter, next);
- rte_free(filter);
- }
-
- TAILQ_REMOVE(&pf->flow_list, flow, node);
- rte_free(flow);
- }
-
- assert(!pf->rss_config_list.tqh_first);
- return 0;
-}
diff --git a/drivers/net/intel/i40e/i40e_hash.h b/drivers/net/intel/i40e/i40e_hash.h
index 3bf30cdee5..c1ce372b6e 100644
--- a/drivers/net/intel/i40e/i40e_hash.h
+++ b/drivers/net/intel/i40e/i40e_hash.h
@@ -13,19 +13,12 @@
extern "C" {
#endif
-int i40e_hash_parse(struct rte_eth_dev *dev,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct i40e_rte_flow_rss_conf *rss_conf,
- struct rte_flow_error *error);
-
-int i40e_hash_filter_create(struct i40e_pf *pf,
- struct i40e_rte_flow_rss_conf *rss_conf);
-
-int i40e_hash_filter_restore(struct i40e_pf *pf);
-int i40e_hash_filter_destroy(struct i40e_pf *pf,
- const struct i40e_rss_filter *rss_filter);
-int i40e_hash_filter_flush(struct i40e_pf *pf);
+int i40e_hash_program_hw(struct i40e_pf *pf,
+ struct i40e_rte_flow_rss_conf *rss_conf);
+int i40e_hash_deprogram_hw(struct i40e_pf *pf,
+ struct i40e_rss_filter_data *filter_data);
+void i40e_hash_compute_reset_flags(const struct i40e_rte_flow_rss_conf *rss_conf,
+ struct i40e_rss_filter_data *filter_data);
extern const uint8_t i40e_rss_key_default[I40E_RSS_KEY_LEN];
diff --git a/drivers/net/intel/i40e/meson.build b/drivers/net/intel/i40e/meson.build
index 0db60c1e99..07162a93b3 100644
--- a/drivers/net/intel/i40e/meson.build
+++ b/drivers/net/intel/i40e/meson.build
@@ -36,6 +36,7 @@ sources += files(
'i40e_flow_ethertype.c',
'i40e_flow_fdir.c',
'i40e_flow_tunnel.c',
+ 'i40e_flow_hash.c',
'i40e_tm.c',
'i40e_hash.c',
'i40e_vf_representor.c',
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* [PATCH v2 19/19] net/i40e: advertise flow keep capability
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (17 preceding siblings ...)
2026-09-08 15:21 ` [PATCH v2 18/19] net/i40e: reimplement hash parser Anatoly Burakov
@ 2026-09-08 15:21 ` Anatoly Burakov
2026-09-09 9:08 ` [PATCH v2 00/19] Building a better rte_flow parser Burakov, Anatoly
19 siblings, 0 replies; 52+ messages in thread
From: Anatoly Burakov @ 2026-09-08 15:21 UTC (permalink / raw)
To: dev, Bruce Richardson
The i40e driver's rte_flow implementation has always endeavored to keep
flows across device restart, however it never advertised this capability.
Advertise it explicitly.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
drivers/net/intel/i40e/i40e_ethdev.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 8c35a64780..746fdd564e 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -3662,7 +3662,8 @@ i40e_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->dev_capa =
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
- dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+ /* rte_flow rules are kept across dev_stop/dev_start and replayed on start */
+ dev_info->dev_capa |= RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
dev_info->hash_key_size = (I40E_PFQF_HKEY_MAX_INDEX + 1) *
sizeof(uint32_t);
--
2.52.0
^ permalink raw reply related [flat|nested] 52+ messages in thread* Re: [PATCH v2 00/19] Building a better rte_flow parser
2026-09-08 15:20 ` [PATCH v2 00/19] " Anatoly Burakov
` (18 preceding siblings ...)
2026-09-08 15:21 ` [PATCH v2 19/19] net/i40e: advertise flow keep capability Anatoly Burakov
@ 2026-09-09 9:08 ` Burakov, Anatoly
19 siblings, 0 replies; 52+ messages in thread
From: Burakov, Anatoly @ 2026-09-09 9:08 UTC (permalink / raw)
To: dev
On 9/8/2026 5:20 PM, Anatoly Burakov wrote:
> Most rte_flow parsers in DPDK suffer from huge implementation complexity because
> even though 99% of what people use rte_flow parsers for is parsing protocol
> graphs, no parser is written explicitly as a graph. This patchset attempts to
> suggest a viable model to build rte_flow parsers as graphs, by offering a
> lightweight header only library to build rte_flow parsering graphs without too
> much boilerplate and complexity.
>
> Most of the patchset is about Intel drivers, but they are meant as
> reimplementations as well as examples for the rest of the community to assess
> how to build parsers using this new infrastructure. I expect the first two
> patches will be of most interest to non-Intel reviewers, as they deal with
> building two reusable parser architecture pieces.
>
> The first piece is a new flow graph helper in ethdev. Its purpose is
> deliberately narrow: it targets the protocol-graph part of rte_flow pattern
> parsing, where drivers walk packet headers and validate legal item sequences and
> parameters. That does not cover all possible rte_flow features, especially more
> exotic flow items, but it does cover a large and widely shared part of what
> existing drivers need to do. Or, to put it in other words, the only flow items
> this infrastructure *doesn't* cover is things that do not lend themselves well
> to be parsed as a graph of protocol headers (e.g. conntrack items). Everything
> else should be covered or cover-able. In practice, just about all drivers will
> benefit from graph parsing as all but one of them implement only the protocol
> stack parts, which are the ones targeted by the graph helper.
>
> The second piece is a reusable flow engine framework for Intel Ethernet drivers.
> This is kept Intel-local because I do not feel it is even appropriate to define
> such a framework for all drivers to use in the first place. Even so, the intent
> is to establish a cleaner parser architecture with a defined interaction model,
> explicit memory ownership rules, locking, initialization sequence,
> implementations of rte_flow API entry points, flow replay and memory cleanup,
> and engine definitions that do not block secondary-process-safe usage. It is my
> hope that this would serve as a model for other drivers to follow, expand on,
> rework, and improve, so that maybe down the line we *might* have a common
> rte_flow infrastructure for drivers to use.
>
> Most of the rest of the series is parser reimplementation, but that is mainly
> the vehicle for demonstrating and validating those two pieces. ixgbe and i40e
> are wired into the new common parsing path, and their existing parsers are
> migrated incrementally to the graph-based model. Besides reducing ad hoc parser
> code, this also makes validation more explicit and more consistent. In a few
> places that means invalid inputs that were previously ignored, deferred, or
> interpreted loosely are now rejected earlier and more strictly, without any
> increase in code complexity (in fact, with marked *decrease* of it!).
>
Recheck-request: rebase=next-net-intel, iol-compile-amd64-testing,
iol-intel-Functional, iol-intel-Performance
--
Thanks,
Anatoly
^ permalink raw reply [flat|nested] 52+ messages in thread