* [PATCH v2 00/13] ENA release 2.15.0
@ 2026-10-18 6:36 Shai Brandes
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
0 siblings, 1 reply; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 6:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Hello all,
This series upgrades HAL and fixes two PHC correctness issues,
an LLQ feature version not communicated to the device, and
a failed LLQ configuration zeroing the negotiated tx_max_header_size.
Additionally it adds n-tuple Rx flow steering support through the rte_flow API.
Review and comments are welcome
Best regards,
Shai Brandes
---
v2:
* Fixed Fedora compilation issue
Shai Brandes (13):
net/ena: avoid using un-needed admin response buffers
net/ena/base: fix race in PHC destroy
net/ena/base: fix missing barriers in PHC timestamp read
net/ena/base: rename bounce buffer references
net/ena/base: add LLQ policy inline helper
net/ena/base: fix LLQ feature version sent to device
net/ena/base: deduplicate LLQ entry write and advance
net/ena/base: deduplicate meta caching config flag
net/ena/base: split LLQ info into datapath and config
net/ena/base: fix LLQ config failure flow
net/ena: support flow API for n-tuple Rx steering
net/ena/base: remove x86 Tx LLQ write barrier
net/ena: upgrade driver version to 2.15.0
doc/guides/nics/ena.rst | 26 +
doc/guides/nics/features/ena.ini | 10 +
doc/guides/rel_notes/release_26_11.rst | 6 +
drivers/net/ena/base/ena_com.c | 793 +++++++++++++-----
drivers/net/ena/base/ena_com.h | 115 ++-
.../net/ena/base/ena_defs/ena_admin_defs.h | 83 +-
drivers/net/ena/base/ena_eth_com.c | 131 +--
drivers/net/ena/base/ena_eth_com.h | 18 +-
drivers/net/ena/base/ena_plat_dpdk.h | 8 +
drivers/net/ena/ena_ethdev.c | 73 +-
drivers/net/ena/ena_ethdev.h | 3 +-
drivers/net/ena/ena_flow.c | 542 ++++++++++++
drivers/net/ena/ena_flow.h | 12 +
drivers/net/ena/meson.build | 1 +
14 files changed, 1512 insertions(+), 309 deletions(-)
create mode 100644 drivers/net/ena/ena_flow.c
create mode 100644 drivers/net/ena/ena_flow.h
--
2.17.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 00/13] ENA release 2.15.0
2026-10-18 6:36 [PATCH v2 00/13] ENA release 2.15.0 Shai Brandes
@ 2026-10-18 9:45 ` Shai Brandes
2026-10-18 9:46 ` [PATCH v3 01/13] net/ena: avoid using un-needed admin response buffers Shai Brandes
` (12 more replies)
0 siblings, 13 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:45 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Hello all,
This series upgrades HAL and fixes two PHC correctness issues,
an LLQ feature version not communicated to the device, and
a failed LLQ configuration zeroing the negotiated tx_max_header_size.
Additionally it adds n-tuple Rx flow steering support through the rte_flow API.
Review and comments are welcome
Best regards,
Shai Brandes
---
v2:
* Fixed Fedora compilation issue
v3:
- Fixed a minor typo in a comment
- Rebased on next-net-for-main to fix the CI apply failure.
- Placed the ENA release-notes entry alphabetically before Broadcom bnxt.
Shai Brandes (13):
net/ena: avoid using un-needed admin response buffers
net/ena/base: fix race in PHC destroy
net/ena/base: fix missing barriers in PHC timestamp read
net/ena/base: rename bounce buffer references
net/ena/base: add LLQ policy inline helper
net/ena/base: fix LLQ feature version sent to device
net/ena/base: deduplicate LLQ entry write and advance
net/ena/base: deduplicate meta caching config flag
net/ena/base: split LLQ info into datapath and config
net/ena/base: fix LLQ config failure flow
net/ena: support flow API for n-tuple Rx steering
net/ena/base: remove x86 Tx LLQ write barrier
net/ena: upgrade driver version to 2.15.0
doc/guides/nics/ena.rst | 26 +
doc/guides/nics/features/ena.ini | 10 +
doc/guides/rel_notes/release_26_11.rst | 6 +
drivers/net/ena/base/ena_com.c | 793 +++++++++++++-----
drivers/net/ena/base/ena_com.h | 115 ++-
.../net/ena/base/ena_defs/ena_admin_defs.h | 83 +-
drivers/net/ena/base/ena_eth_com.c | 131 +--
drivers/net/ena/base/ena_eth_com.h | 18 +-
drivers/net/ena/base/ena_plat_dpdk.h | 8 +
drivers/net/ena/ena_ethdev.c | 73 +-
drivers/net/ena/ena_ethdev.h | 3 +-
drivers/net/ena/ena_flow.c | 542 ++++++++++++
drivers/net/ena/ena_flow.h | 12 +
drivers/net/ena/meson.build | 1 +
14 files changed, 1512 insertions(+), 309 deletions(-)
create mode 100644 drivers/net/ena/ena_flow.c
create mode 100644 drivers/net/ena/ena_flow.h
base-commit: 3b99fca0dcd388585b24947e2a851001b159757b
--
2.17.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 01/13] net/ena: avoid using un-needed admin response buffers
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
@ 2026-10-18 9:46 ` Shai Brandes
2026-10-18 9:47 ` [PATCH v3 02/13] net/ena/base: fix race in PHC destroy Shai Brandes
` (11 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
There are admin commands that do not require a response parsing.
For these, there's no need to set a response buffer.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 356 +++++++++++++++++----------------
1 file changed, 179 insertions(+), 177 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index f0936a6262..b1fe0bcdf7 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -613,6 +613,149 @@ static int ena_com_wait_and_process_admin_cq_polling(struct ena_comp_ctx *comp_c
return ret;
}
+static int ena_com_wait_and_process_admin_cq_interrupts(struct ena_comp_ctx *comp_ctx,
+ struct ena_com_admin_queue *admin_queue)
+{
+ unsigned long flags = 0;
+ int ret;
+
+ ENA_WAIT_EVENT_WAIT(comp_ctx->wait_event,
+ admin_queue->completion_timeout);
+
+ /* In case the command wasn't completed find out the root cause.
+ * There might be 2 kinds of errors
+ * 1) No completion (timeout reached)
+ * 2) There is completion but the device didn't get any msi-x interrupt.
+ */
+ if (unlikely(comp_ctx->status == ENA_CMD_SUBMITTED)) {
+ ENA_SPINLOCK_LOCK(admin_queue->q_lock, flags);
+ ena_com_handle_admin_completion(admin_queue);
+ admin_queue->stats.no_completion++;
+ ENA_SPINLOCK_UNLOCK(admin_queue->q_lock, flags);
+
+ ret = ENA_COM_TIMER_EXPIRED;
+ /* Now that the admin queue has been polled, check whether the
+ * request was fulfilled by the device
+ */
+ if (comp_ctx->status != ENA_CMD_COMPLETED) {
+ ena_trc_err(admin_queue->ena_dev,
+ "The ena device didn't send a completion for the admin cmd %d status %d\n",
+ comp_ctx->cmd_opcode, comp_ctx->status);
+ goto close_aq;
+ }
+
+ admin_queue->is_missing_admin_interrupt = true;
+
+ ena_trc_err(admin_queue->ena_dev,
+ "The ena device sent a completion but the driver didn't receive a MSI-X interrupt (cmd %d), autopolling mode is %s\n",
+ comp_ctx->cmd_opcode, admin_queue->auto_polling ? "ON" : "OFF");
+ /* If fallback to polling is not enabled, missing an interrupt
+ * is considered an error
+ */
+ if (!admin_queue->auto_polling)
+ goto close_aq;
+
+ ena_com_set_admin_polling_mode(admin_queue->ena_dev, true);
+ } else if (unlikely(comp_ctx->status == ENA_CMD_ABORTED)) {
+ ena_trc_err(admin_queue->ena_dev, "Command was aborted\n");
+ ENA_SPINLOCK_LOCK(admin_queue->q_lock, flags);
+ admin_queue->stats.aborted_cmd++;
+ ENA_SPINLOCK_UNLOCK(admin_queue->q_lock, flags);
+ ret = ENA_COM_NO_DEVICE;
+ goto err;
+ }
+
+ ret = ena_com_admin_status_to_errno(admin_queue, comp_ctx);
+ comp_ctxt_release(admin_queue, comp_ctx);
+
+ return ret;
+close_aq:
+ admin_queue->running_state = false;
+err:
+ comp_ctxt_release(admin_queue, comp_ctx);
+
+ return ret;
+}
+
+/* There are two types to wait for completion.
+ * Polling mode - wait until the completion is available.
+ * Async mode - wait on wait queue until the completion is ready
+ * (or the timeout expired).
+ * It is expected that the IRQ called ena_com_handle_admin_completion
+ * to mark the completions.
+ */
+static int ena_com_wait_and_process_admin_cq(struct ena_comp_ctx *comp_ctx,
+ struct ena_com_admin_queue *admin_queue)
+{
+ if (admin_queue->polling)
+ return ena_com_wait_and_process_admin_cq_polling(comp_ctx,
+ admin_queue);
+
+ return ena_com_wait_and_process_admin_cq_interrupts(comp_ctx,
+ admin_queue);
+}
+
+/* ena_com_execute_admin_command - Execute admin command
+ * @admin_queue: admin queue.
+ * @cmd: the admin command to execute.
+ * @cmd_size: the command size.
+ * @cmd_completion: command completion return value.
+ * @cmd_comp_size: command completion size.
+
+ * Submit an admin command and then wait until the device returns a
+ * completion.
+ * The completion will be copied into cmd_comp.
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+int ena_com_execute_admin_command(struct ena_com_admin_queue *admin_queue,
+ struct ena_admin_aq_entry *cmd,
+ size_t cmd_size,
+ struct ena_admin_acq_entry *comp,
+ size_t comp_size)
+{
+ struct ena_comp_ctx *comp_ctx;
+ int ret;
+
+ comp_ctx = ena_com_submit_admin_cmd(admin_queue, cmd, cmd_size,
+ comp, comp_size);
+ if (IS_ERR(comp_ctx)) {
+ ret = PTR_ERR(comp_ctx);
+ if (ret != ENA_COM_NO_DEVICE)
+ ena_trc_err(admin_queue->ena_dev,
+ "Failed to submit command [%d]\n",
+ ret);
+
+ return ret;
+ }
+
+ ret = ena_com_wait_and_process_admin_cq(comp_ctx, admin_queue);
+ if (unlikely(ret)) {
+ if (admin_queue->running_state)
+ ena_trc_err(admin_queue->ena_dev,
+ "Failed to process command [%d]\n",
+ ret);
+ }
+ return ret;
+}
+
+/* ena_com_execute_admin_command_no_resp - Execute admin command without response payload
+ * @admin_queue: admin queue.
+ * @cmd: the admin command to execute.
+ * @cmd_size: the command size.
+ *
+ * Submit an admin command and then wait until the device returns a
+ * completion, no return payload is expected.
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+static int ena_com_execute_admin_command_no_resp(struct ena_com_admin_queue *admin_queue,
+ struct ena_admin_aq_entry *cmd,
+ size_t cmd_size)
+{
+ return ena_com_execute_admin_command(admin_queue, cmd, cmd_size, NULL, 0);
+}
+
/*
* Set the LLQ configurations of the firmware
*
@@ -623,7 +766,6 @@ static int ena_com_set_llq(struct ena_com_dev *ena_dev)
{
struct ena_com_llq_info *llq_info = &ena_dev->llq_info;
struct ena_com_admin_queue *admin_queue;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -642,11 +784,9 @@ static int ena_com_set_llq(struct ena_com_dev *ena_dev)
BIT(ENA_ADMIN_DISABLE_META_CACHING) |
BIT(ENA_ADMIN_LIMIT_TX_BURST);
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to set LLQ configurations: %d\n", ret);
@@ -782,62 +922,6 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
return rc;
}
-static int ena_com_wait_and_process_admin_cq_interrupts(struct ena_comp_ctx *comp_ctx,
- struct ena_com_admin_queue *admin_queue)
-{
- unsigned long flags = 0;
- int ret;
-
- ENA_WAIT_EVENT_WAIT(comp_ctx->wait_event,
- admin_queue->completion_timeout);
-
- /* In case the command wasn't completed find out the root cause.
- * There might be 2 kinds of errors
- * 1) No completion (timeout reached)
- * 2) There is completion but the device didn't get any msi-x interrupt.
- */
- if (unlikely(comp_ctx->status == ENA_CMD_SUBMITTED)) {
- ENA_SPINLOCK_LOCK(admin_queue->q_lock, flags);
- ena_com_handle_admin_completion(admin_queue);
- admin_queue->stats.no_completion++;
- ENA_SPINLOCK_UNLOCK(admin_queue->q_lock, flags);
-
- if (comp_ctx->status == ENA_CMD_COMPLETED) {
- admin_queue->is_missing_admin_interrupt = true;
- ena_trc_err(admin_queue->ena_dev,
- "The ena device sent a completion but the driver didn't receive a MSI-X interrupt (cmd %d), autopolling mode is %s\n",
- comp_ctx->cmd_opcode, admin_queue->auto_polling ? "ON" : "OFF");
- /* Check if fallback to polling is enabled */
- if (admin_queue->auto_polling)
- admin_queue->polling = true;
- } else {
- ena_trc_err(admin_queue->ena_dev,
- "The ena device didn't send a completion for the admin cmd %d status %d\n",
- comp_ctx->cmd_opcode, comp_ctx->status);
- }
- /* Check if shifted to polling mode.
- * This will happen if there is a completion without an interrupt
- * and autopolling mode is enabled. Continuing normal execution in such case
- */
- if (!admin_queue->polling) {
- admin_queue->running_state = false;
- ret = ENA_COM_TIMER_EXPIRED;
- goto err;
- }
- } else if (unlikely(comp_ctx->status == ENA_CMD_ABORTED)) {
- ena_trc_err(admin_queue->ena_dev, "Command was aborted\n");
- ENA_SPINLOCK_LOCK(admin_queue->q_lock, flags);
- admin_queue->stats.aborted_cmd++;
- ENA_SPINLOCK_UNLOCK(admin_queue->q_lock, flags);
- ret = ENA_COM_NO_DEVICE;
- goto err;
- }
- ret = ena_com_admin_status_to_errno(admin_queue, comp_ctx);
-err:
- comp_ctxt_release(admin_queue, comp_ctx);
- return ret;
-}
-
/* This method read the hardware device register through posting writes
* and waiting for response
* On timeout the function will return ENA_MMIO_READ_TIMEOUT
@@ -903,29 +987,10 @@ static u32 ena_com_reg_bar_read32(struct ena_com_dev *ena_dev, u16 offset)
return ret;
}
-/* There are two types to wait for completion.
- * Polling mode - wait until the completion is available.
- * Async mode - wait on wait queue until the completion is ready
- * (or the timeout expired).
- * It is expected that the IRQ called ena_com_handle_admin_completion
- * to mark the completions.
- */
-static int ena_com_wait_and_process_admin_cq(struct ena_comp_ctx *comp_ctx,
- struct ena_com_admin_queue *admin_queue)
-{
- if (admin_queue->polling)
- return ena_com_wait_and_process_admin_cq_polling(comp_ctx,
- admin_queue);
-
- return ena_com_wait_and_process_admin_cq_interrupts(comp_ctx,
- admin_queue);
-}
-
static int ena_com_destroy_io_sq(struct ena_com_dev *ena_dev,
struct ena_com_io_sq *io_sq)
{
struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue;
- struct ena_admin_acq_destroy_sq_resp_desc destroy_resp;
struct ena_admin_aq_destroy_sq_cmd destroy_cmd;
u8 direction;
int ret;
@@ -945,11 +1010,9 @@ static int ena_com_destroy_io_sq(struct ena_com_dev *ena_dev,
destroy_cmd.sq.sq_idx = io_sq->idx;
destroy_cmd.aq_common_descriptor.opcode = ENA_ADMIN_DESTROY_SQ;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&destroy_cmd,
- sizeof(destroy_cmd),
- (struct ena_admin_acq_entry *)&destroy_resp,
- sizeof(destroy_resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&destroy_cmd,
+ sizeof(destroy_cmd));
if (unlikely(ret && (ret != ENA_COM_NO_DEVICE)))
ena_trc_err(ena_dev, "Failed to destroy io sq error: %d\n", ret);
@@ -1152,7 +1215,6 @@ int ena_com_set_hw_timestamping_configuration(struct ena_com_dev *ena_dev,
u8 tx_enable,
u8 rx_enable)
{
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -1170,11 +1232,9 @@ int ena_com_set_hw_timestamping_configuration(struct ena_com_dev *ena_dev,
cmd.u.hw_ts.rx = rx_enable ? ENA_ADMIN_HW_TIMESTAMP_RX_SUPPORT_ALL :
ENA_ADMIN_HW_TIMESTAMP_RX_SUPPORT_NONE;
- ret = ena_com_execute_admin_command(&ena_dev->admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(&ena_dev->admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret)) {
ena_trc_err(ena_dev,
"Failed to set HW timestamping configuration, error: %d\n",
@@ -1469,37 +1529,6 @@ static void ena_com_update_intr_delay_resolution(struct ena_com_dev *ena_dev,
/******************************* API ******************************/
/*****************************************************************************/
-int ena_com_execute_admin_command(struct ena_com_admin_queue *admin_queue,
- struct ena_admin_aq_entry *cmd,
- size_t cmd_size,
- struct ena_admin_acq_entry *comp,
- size_t comp_size)
-{
- struct ena_comp_ctx *comp_ctx;
- int ret;
-
- comp_ctx = ena_com_submit_admin_cmd(admin_queue, cmd, cmd_size,
- comp, comp_size);
- if (IS_ERR(comp_ctx)) {
- ret = PTR_ERR(comp_ctx);
- if (ret != ENA_COM_NO_DEVICE)
- ena_trc_err(admin_queue->ena_dev,
- "Failed to submit command [%d]\n",
- ret);
-
- return ret;
- }
-
- ret = ena_com_wait_and_process_admin_cq(comp_ctx, admin_queue);
- if (unlikely(ret)) {
- if (admin_queue->running_state)
- ena_trc_err(admin_queue->ena_dev,
- "Failed to process command [%d]\n",
- ret);
- }
- return ret;
-}
-
int ena_com_create_io_cq(struct ena_com_dev *ena_dev,
struct ena_com_io_cq *io_cq)
{
@@ -1608,7 +1637,6 @@ int ena_com_destroy_io_cq(struct ena_com_dev *ena_dev,
struct ena_com_io_cq *io_cq)
{
struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue;
- struct ena_admin_acq_destroy_cq_resp_desc destroy_resp;
struct ena_admin_aq_destroy_cq_cmd destroy_cmd;
int ret;
@@ -1617,11 +1645,9 @@ int ena_com_destroy_io_cq(struct ena_com_dev *ena_dev,
destroy_cmd.cq_idx = io_cq->idx;
destroy_cmd.aq_common_descriptor.opcode = ENA_ADMIN_DESTROY_CQ;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&destroy_cmd,
- sizeof(destroy_cmd),
- (struct ena_admin_acq_entry *)&destroy_resp,
- sizeof(destroy_resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&destroy_cmd,
+ sizeof(destroy_cmd));
if (unlikely(ret && (ret != ENA_COM_NO_DEVICE)))
ena_trc_err(ena_dev, "Failed to destroy IO CQ. error: %d\n", ret);
@@ -1660,7 +1686,6 @@ int ena_com_set_aenq_config(struct ena_com_dev *ena_dev, u32 groups_flag)
{
struct ena_com_admin_queue *admin_queue;
struct ena_admin_get_feat_resp get_resp;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -1685,11 +1710,9 @@ int ena_com_set_aenq_config(struct ena_com_dev *ena_dev, u32 groups_flag)
cmd.feat_common.feature_id = ENA_ADMIN_AENQ_CONFIG;
cmd.u.aenq.enabled_groups = groups_flag;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to config AENQ ret: %d\n", ret);
@@ -1879,7 +1902,6 @@ int ena_com_phc_config(struct ena_com_dev *ena_dev)
{
struct ena_com_phc_info *phc = &ena_dev->phc;
struct ena_admin_get_feat_resp get_feat_resp;
- struct ena_admin_set_feat_resp set_feat_resp;
struct ena_admin_set_feat_cmd set_feat_cmd;
int ret = 0;
@@ -1934,11 +1956,9 @@ int ena_com_phc_config(struct ena_com_dev *ena_dev)
}
/* Send PHC feature command to the device */
- ret = ena_com_execute_admin_command(&ena_dev->admin_queue,
- (struct ena_admin_aq_entry *)&set_feat_cmd,
- sizeof(set_feat_cmd),
- (struct ena_admin_acq_entry *)&set_feat_resp,
- sizeof(set_feat_resp));
+ ret = ena_com_execute_admin_command_no_resp(&ena_dev->admin_queue,
+ (struct ena_admin_aq_entry *)&set_feat_cmd,
+ sizeof(set_feat_cmd));
if (unlikely(ret)) {
ena_trc_err(ena_dev, "Failed to enable PHC, error: %d\n", ret);
@@ -2822,7 +2842,6 @@ int ena_com_get_customer_metrics(struct ena_com_dev *ena_dev, char *buffer, u32
int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, u32 mtu)
{
struct ena_com_admin_queue *admin_queue;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -2839,11 +2858,9 @@ int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, u32 mtu)
cmd.feat_common.feature_id = ENA_ADMIN_MTU;
cmd.u.mtu.mtu = mtu;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to set mtu %d. error: %d\n", mtu, ret);
@@ -2859,7 +2876,6 @@ int ena_com_set_hash_function(struct ena_com_dev *ena_dev)
{
struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue;
struct ena_admin_get_feat_resp get_resp;
- struct ena_admin_set_feat_resp resp;
struct ena_rss *rss = &ena_dev->rss;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -2902,11 +2918,9 @@ int ena_com_set_hash_function(struct ena_com_dev *ena_dev)
cmd.control_buffer.length = sizeof(*rss->hash_key);
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret)) {
ena_trc_err(ena_dev, "Failed to set hash function %d. error: %d\n",
rss->hash_func, ret);
@@ -3031,7 +3045,6 @@ int ena_com_set_hash_ctrl(struct ena_com_dev *ena_dev)
struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue;
struct ena_admin_feature_rss_hash_control *hash_ctrl;
struct ena_rss *rss = &ena_dev->rss;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -3063,11 +3076,9 @@ int ena_com_set_hash_ctrl(struct ena_com_dev *ena_dev)
}
cmd.control_buffer.length = sizeof(*hash_ctrl);
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to set hash input. error: %d\n", ret);
@@ -3195,7 +3206,6 @@ int ena_com_indirect_table_set(struct ena_com_dev *ena_dev)
{
struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue;
struct ena_rss *rss = &ena_dev->rss;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -3232,11 +3242,9 @@ int ena_com_indirect_table_set(struct ena_com_dev *ena_dev)
cmd.control_buffer.length = (1ULL << rss->tbl_log_size) *
sizeof(struct ena_admin_rss_ind_table_entry);
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to set indirect table. error: %d\n", ret);
@@ -3420,7 +3428,6 @@ int ena_com_set_host_attributes(struct ena_com_dev *ena_dev)
{
struct ena_host_attribute *host_attr = &ena_dev->host_attr;
struct ena_com_admin_queue *admin_queue;
- struct ena_admin_set_feat_resp resp;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -3452,11 +3459,9 @@ int ena_com_set_host_attributes(struct ena_com_dev *ena_dev)
cmd.u.host_attr.debug_area_size = host_attr->debug_area_size;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&resp,
- sizeof(resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to set host attributes: %d\n", ret);
@@ -3579,7 +3584,6 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
int ena_com_set_frag_bypass(struct ena_com_dev *ena_dev, bool enable)
{
- struct ena_admin_set_feat_resp set_feat_resp;
struct ena_com_admin_queue *admin_queue;
struct ena_admin_set_feat_cmd cmd;
int ret;
@@ -3599,11 +3603,9 @@ int ena_com_set_frag_bypass(struct ena_com_dev *ena_dev, bool enable)
cmd.feat_common.feature_version = ENA_ADMIN_FRAG_BYPASS_FEATURE_VERSION_0;
cmd.u.frag_bypass.enable = (u8)enable;
- ret = ena_com_execute_admin_command(admin_queue,
- (struct ena_admin_aq_entry *)&cmd,
- sizeof(cmd),
- (struct ena_admin_acq_entry *)&set_feat_resp,
- sizeof(set_feat_resp));
+ ret = ena_com_execute_admin_command_no_resp(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd));
if (unlikely(ret))
ena_trc_err(ena_dev, "Failed to enable frag bypass. error: %d\n", ret);
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 02/13] net/ena/base: fix race in PHC destroy
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
2026-10-18 9:46 ` [PATCH v3 01/13] net/ena: avoid using un-needed admin response buffers Shai Brandes
@ 2026-10-18 9:47 ` Shai Brandes
2026-10-18 9:47 ` [PATCH v3 03/13] net/ena/base: fix missing barriers in PHC timestamp read Shai Brandes
` (10 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:47 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes, stable
ena_com_phc_get_timestamp() checked phc->active and cached the
resp pointer from ena_dev->phc.virt_addr before acquiring the PHC
spinlock, so neither the liveness check nor the pointer read was
serialised against teardown.
If ena_com_phc_destroy() runs in that window it clears active,
releases the lock, frees the PHC DMA memory and sets virt_addr to
NULL. The reader then dereferences either a pointer to already
freed memory or a NULL virt_addr, depending on where it was
preempted, crashing the application on PHC teardown or device
reconfiguration.
Move the phc->active check and the resp pointer assignment inside
the spinlock so that destroy cannot free the DMA memory while a
timestamp read is in progress.
Fixes: f73f53f7dc7a ("net/ena: upgrade HAL")
Cc: stable@dpdk.org
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index b1fe0bcdf7..737626bcfa 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -1996,20 +1996,23 @@ void ena_com_phc_destroy(struct ena_com_dev *ena_dev)
int ena_com_phc_get_timestamp(struct ena_com_dev *ena_dev, u64 *timestamp)
{
- volatile struct ena_admin_phc_resp *resp = ena_dev->phc.virt_addr;
const ena_time_high_res_t zero_system_time = ENA_TIME_INIT_HIGH_RES();
struct ena_com_phc_info *phc = &ena_dev->phc;
+ volatile struct ena_admin_phc_resp *resp;
ena_time_high_res_t expire_time;
ena_time_high_res_t block_time;
unsigned long flags = 0;
int ret = ENA_COM_OK;
+ ENA_SPINLOCK_LOCK(phc->lock, flags);
+
if (!phc->active) {
+ ENA_SPINLOCK_UNLOCK(phc->lock, flags);
ena_trc_err(ena_dev, "PHC feature is not active in the device\n");
return ENA_COM_UNSUPPORTED;
}
- ENA_SPINLOCK_LOCK(phc->lock, flags);
+ resp = ena_dev->phc.virt_addr;
/* Check if PHC is in blocked state */
if (unlikely(ENA_TIME_COMPARE_HIGH_RES(phc->system_time, zero_system_time))) {
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 03/13] net/ena/base: fix missing barriers in PHC timestamp read
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
2026-10-18 9:46 ` [PATCH v3 01/13] net/ena: avoid using un-needed admin response buffers Shai Brandes
2026-10-18 9:47 ` [PATCH v3 02/13] net/ena/base: fix race in PHC destroy Shai Brandes
@ 2026-10-18 9:47 ` Shai Brandes
2026-10-18 9:47 ` [PATCH v3 04/13] net/ena/base: rename bounce buffer references Shai Brandes
` (9 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:47 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes, stable
The PHC read path in ena_com_phc_get_timestamp() polls the shared
error_bound and timestamp fields for a matching req_id to detect
that the device has published a fresh reading. Both completion
checks read the payload without a barrier after observing req_id.
On weakly-ordered architectures the CPU may load the payload fields
before the req_id load is observed as updated, so a stale timestamp
or error bound can be returned while the read is reported as valid.
Add dma_rmb() after both req_id completion checks to order the
req_id load against the subsequent payload loads.
Fixes: a14be4dd8693 ("net/ena/base: add PHC error statistics")
Cc: stable@dpdk.org
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 50 +++++++++++++++++++++-------------
1 file changed, 31 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index 737626bcfa..49818c4bd0 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -2037,27 +2037,34 @@ int ena_com_phc_get_timestamp(struct ena_com_dev *ena_dev, u64 *timestamp)
"PHC get time request 0x%x failed (device error)\n",
phc->req_id);
phc->stats.phc_err_dv++;
- } else if (resp->error_flags & ENA_PHC_ERROR_FLAGS) {
- /* Device updated req_id during blocking time but got
- * a PHC error, this occurs if device:
- * - exceeded the get time request limit
- * - received an invalid timestamp
- * - received an excessively high error bound
- * - received an invalid error bound
- */
- ena_trc_err(ena_dev,
- "PHC get time request 0x%x failed (error 0x%x)\n",
- phc->req_id,
- resp->error_flags);
- phc->stats.phc_err_ts += !!(resp->error_flags &
- ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP);
- phc->stats.phc_err_eb += !!(resp->error_flags &
- ENA_ADMIN_PHC_ERROR_FLAG_ERROR_BOUND);
} else {
- /* Device updated req_id during blocking time
- * with valid timestamp and error bound
+ /* Ensure PHC payload is read after req_id update
+ * is observed
*/
- phc->stats.phc_exp++;
+ dma_rmb();
+
+ if (resp->error_flags & ENA_PHC_ERROR_FLAGS) {
+ /* Device updated req_id during blocking time
+ * but got a PHC error, this occurs if device:
+ * - exceeded the get time request limit
+ * - received an invalid timestamp
+ * - received an excessively high error bound
+ * - received an invalid error bound
+ */
+ ena_trc_err(ena_dev,
+ "PHC get time request 0x%x failed (error 0x%x)\n",
+ phc->req_id,
+ resp->error_flags);
+ phc->stats.phc_err_ts += !!(resp->error_flags &
+ ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP);
+ phc->stats.phc_err_eb += !!(resp->error_flags &
+ ENA_ADMIN_PHC_ERROR_FLAG_ERROR_BOUND);
+ } else {
+ /* Device updated req_id during blocking time
+ * with valid timestamp and error bound
+ */
+ phc->stats.phc_exp++;
+ }
}
}
@@ -2100,6 +2107,11 @@ int ena_com_phc_get_timestamp(struct ena_com_dev *ena_dev, u64 *timestamp)
continue;
}
+ /* Ensure PHC payload (timestamp, error_bound, error_flags)
+ * is read after req_id update is observed
+ */
+ dma_rmb();
+
/* req_id was updated by the device which indicates that
* PHC timestamp, error_bound and error_flags are updated too,
* checking errors before retrieving timestamp and
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 04/13] net/ena/base: rename bounce buffer references
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (2 preceding siblings ...)
2026-10-18 9:47 ` [PATCH v3 03/13] net/ena/base: fix missing barriers in PHC timestamp read Shai Brandes
@ 2026-10-18 9:47 ` Shai Brandes
2026-10-18 9:50 ` [PATCH v3 05/13] net/ena/base: add LLQ policy inline helper Shai Brandes
` (8 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:47 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Rename curr_bounce_buf pointer in sq llq packet control to
curr_llq_entry for future use of llq not only with bounce buffers.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 4 ++--
drivers/net/ena/base/ena_com.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index 49818c4bd0..e9b9de21cb 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -391,9 +391,9 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev,
sizeof(io_sq->llq_info));
/* Initiate the first bounce buffer */
- io_sq->llq_buf_ctrl.curr_bounce_buf =
+ io_sq->llq_buf_ctrl.curr_llq_entry =
ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
- memset(io_sq->llq_buf_ctrl.curr_bounce_buf,
+ memset(io_sq->llq_buf_ctrl.curr_llq_entry,
0x0, io_sq->llq_info.desc_list_entry_size);
io_sq->llq_buf_ctrl.descs_left_in_line =
io_sq->llq_info.descs_num_before_header;
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index 1d024cfea0..bd395fd90c 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -138,7 +138,7 @@ struct ena_com_io_bounce_buffer_control {
/* This struct is to keep tracking the current location of the next llq entry */
struct ena_com_llq_pkt_ctrl {
- u8 *curr_bounce_buf;
+ u8 *curr_llq_entry;
u16 idx;
u16 descs_left_in_line;
};
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 05/13] net/ena/base: add LLQ policy inline helper
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (3 preceding siblings ...)
2026-10-18 9:47 ` [PATCH v3 04/13] net/ena/base: rename bounce buffer references Shai Brandes
@ 2026-10-18 9:50 ` Shai Brandes
2026-10-18 9:51 ` [PATCH v3 06/13] net/ena/base: fix LLQ feature version sent to device Shai Brandes
` (7 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:50 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Introduce a helper to abstract the LLQ placement policy check,
replacing direct mem_queue_type comparisons across ena-com.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.h | 5 +++
drivers/net/ena/base/ena_eth_com.c | 55 +++++++++++++++---------------
drivers/net/ena/base/ena_eth_com.h | 6 ++--
3 files changed, 35 insertions(+), 31 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index bd395fd90c..c62fecdef2 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -1139,6 +1139,11 @@ static inline struct ena_com_dev *ena_com_io_cq_to_ena_dev(struct ena_com_io_cq
return container_of(io_cq, struct ena_com_dev, io_cq_queues[io_cq->qid]);
}
+static inline bool ena_com_sq_is_llq(struct ena_com_io_sq *io_sq)
+{
+ return io_sq->mem_queue_type != ENA_ADMIN_PLACEMENT_POLICY_HOST;
+}
+
static inline bool ena_com_get_adaptive_moderation_enabled(struct ena_com_dev *ena_dev)
{
return ena_dev->adaptive_coalescing;
diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c
index 8a019b3451..e3b2a753c4 100644
--- a/drivers/net/ena/base/ena_eth_com.c
+++ b/drivers/net/ena/base/ena_eth_com.c
@@ -151,16 +151,16 @@ static int ena_com_write_bounce_buffer_to_dev(struct ena_com_io_sq *io_sq,
return ENA_COM_OK;
}
-static int ena_com_write_header_to_bounce(struct ena_com_io_sq *io_sq,
- u8 *header_src,
- u16 header_len)
+static int ena_com_write_header_to_llq_entry(struct ena_com_io_sq *io_sq,
+ u8 *header_src,
+ u16 header_len)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
struct ena_com_llq_info *llq_info = &io_sq->llq_info;
- u8 *bounce_buffer = pkt_ctrl->curr_bounce_buf;
+ u8 *curr_llq_entry = pkt_ctrl->curr_llq_entry;
u16 header_offset;
- if (unlikely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST))
+ if (unlikely(!ena_com_sq_is_llq(io_sq)))
return 0;
header_offset =
@@ -172,13 +172,13 @@ static int ena_com_write_header_to_bounce(struct ena_com_io_sq *io_sq,
return ENA_COM_FAULT;
}
- if (unlikely(!bounce_buffer)) {
+ if (unlikely(!curr_llq_entry)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
- "Bounce buffer is NULL\n");
+ "LLQ entry is NULL\n");
return ENA_COM_FAULT;
}
- memcpy(bounce_buffer + header_offset, header_src, header_len);
+ memcpy(curr_llq_entry + header_offset, header_src, header_len);
return 0;
}
@@ -186,46 +186,46 @@ static int ena_com_write_header_to_bounce(struct ena_com_io_sq *io_sq,
static void *get_sq_desc_llq(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
- u8 *bounce_buffer;
+ u8 *curr_llq_entry;
void *sq_desc;
- bounce_buffer = pkt_ctrl->curr_bounce_buf;
+ curr_llq_entry = pkt_ctrl->curr_llq_entry;
- if (unlikely(!bounce_buffer)) {
+ if (unlikely(!curr_llq_entry)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
- "Bounce buffer is NULL\n");
+ "LLQ entry is NULL\n");
return NULL;
}
- sq_desc = bounce_buffer + pkt_ctrl->idx * io_sq->desc_entry_size;
+ sq_desc = curr_llq_entry + pkt_ctrl->idx * io_sq->desc_entry_size;
pkt_ctrl->idx++;
pkt_ctrl->descs_left_in_line--;
return sq_desc;
}
-static int ena_com_close_bounce_buffer(struct ena_com_io_sq *io_sq)
+static int ena_com_close_llq_entry(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
struct ena_com_llq_info *llq_info = &io_sq->llq_info;
int rc;
- if (unlikely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST))
+ if (unlikely(!ena_com_sq_is_llq(io_sq)))
return ENA_COM_OK;
/* bounce buffer was used, so write it and get a new one */
if (likely(pkt_ctrl->idx)) {
rc = ena_com_write_bounce_buffer_to_dev(io_sq,
- pkt_ctrl->curr_bounce_buf);
+ pkt_ctrl->curr_llq_entry);
if (unlikely(rc)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
"Failed to write bounce buffer to device\n");
return rc;
}
- pkt_ctrl->curr_bounce_buf =
+ pkt_ctrl->curr_llq_entry =
ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
- memset(io_sq->llq_buf_ctrl.curr_bounce_buf,
+ memset(io_sq->llq_buf_ctrl.curr_llq_entry,
0x0, llq_info->desc_list_entry_size);
}
@@ -241,7 +241,7 @@ static void *get_tx_sq_desc(struct ena_com_io_sq *io_sq)
{
void *tx_desc;
- if (likely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV))
+ if (likely(ena_com_sq_is_llq(io_sq)))
return get_sq_desc_llq(io_sq);
tx_desc = get_sq_desc_regular_queue(io_sq);
@@ -259,16 +259,16 @@ static int ena_com_sq_update_llq_tail(struct ena_com_io_sq *io_sq)
if (!pkt_ctrl->descs_left_in_line) {
rc = ena_com_write_bounce_buffer_to_dev(io_sq,
- pkt_ctrl->curr_bounce_buf);
+ pkt_ctrl->curr_llq_entry);
if (unlikely(rc)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
"Failed to write bounce buffer to device\n");
return rc;
}
- pkt_ctrl->curr_bounce_buf =
+ pkt_ctrl->curr_llq_entry =
ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
- memset(io_sq->llq_buf_ctrl.curr_bounce_buf,
+ memset(io_sq->llq_buf_ctrl.curr_llq_entry,
0x0, llq_info->desc_list_entry_size);
pkt_ctrl->idx = 0;
@@ -291,7 +291,7 @@ static int ena_com_sq_update_reqular_queue_tail(struct ena_com_io_sq *io_sq)
static int ena_com_sq_update_tail(struct ena_com_io_sq *io_sq)
{
- if (likely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV))
+ if (likely(ena_com_sq_is_llq(io_sq)))
return ena_com_sq_update_llq_tail(io_sq);
return ena_com_sq_update_reqular_queue_tail(io_sq);
@@ -514,14 +514,13 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq,
return ENA_COM_INVAL;
}
- if (unlikely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV
- && !buffer_to_push)) {
+ if (unlikely(ena_com_sq_is_llq(io_sq) && !buffer_to_push)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
"Push header wasn't provided in LLQ mode\n");
return ENA_COM_INVAL;
}
- rc = ena_com_write_header_to_bounce(io_sq, buffer_to_push, header_len);
+ rc = ena_com_write_header_to_llq_entry(io_sq, buffer_to_push, header_len);
if (unlikely(rc))
return rc;
@@ -534,7 +533,7 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq,
/* If the caller doesn't want to send packets */
if (unlikely(!num_bufs && !header_len)) {
- rc = ena_com_close_bounce_buffer(io_sq);
+ rc = ena_com_close_llq_entry(io_sq);
if (unlikely(rc))
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
"Failed to write buffers to LLQ\n");
@@ -635,7 +634,7 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq,
return rc;
}
- rc = ena_com_close_bounce_buffer(io_sq);
+ rc = ena_com_close_llq_entry(io_sq);
*nb_hw_desc = io_sq->tail - start_tail;
return rc;
diff --git a/drivers/net/ena/base/ena_eth_com.h b/drivers/net/ena/base/ena_eth_com.h
index f773699467..503cf45cf7 100644
--- a/drivers/net/ena/base/ena_eth_com.h
+++ b/drivers/net/ena/base/ena_eth_com.h
@@ -110,7 +110,7 @@ static inline bool ena_com_sq_have_enough_space(struct ena_com_io_sq *io_sq,
{
int temp;
- if (unlikely(io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST))
+ if (unlikely(!ena_com_sq_is_llq(io_sq)))
return ena_com_free_q_entries(io_sq) >= required_buffers;
/* This calculation doesn't need to be 100% accurate. So to reduce
@@ -136,8 +136,8 @@ static inline bool ena_com_meta_desc_changed(struct ena_com_io_sq *io_sq,
static inline bool is_llq_max_tx_burst_exists(struct ena_com_io_sq *io_sq)
{
- return (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) &&
- io_sq->llq_info.max_entries_in_tx_burst > 0;
+ return likely(ena_com_sq_is_llq(io_sq)) &&
+ (io_sq->llq_info.max_entries_in_tx_burst > 0);
}
static inline bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq,
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 06/13] net/ena/base: fix LLQ feature version sent to device
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (4 preceding siblings ...)
2026-10-18 9:50 ` [PATCH v3 05/13] net/ena/base: add LLQ policy inline helper Shai Brandes
@ 2026-10-18 9:51 ` Shai Brandes
2026-10-18 9:51 ` [PATCH v3 07/13] net/ena/base: deduplicate LLQ entry write and advance Shai Brandes
` (6 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:51 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes, stable
During LLQ configuration the driver reads the supported LLQ
feature version from the device with a GET feature command, but
leaves the feature_version field zeroed in the subsequent SET
feature command.
The device therefore never learns which LLQ version the driver
negotiated, and falls back to interpreting the configuration
according to the base version regardless of what the driver
actually agreed to, which can lead to a Tx descriptor layout
mismatch between driver and device.
Store the feature version reported in the GET response and echo
it back in the SET command so the negotiated version is
communicated to the device.
Fixes: b68309be44c0 ("net/ena/base: update communication layer for the ENAv2")
Cc: stable@dpdk.org
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 4 ++++
drivers/net/ena/base/ena_com.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index e9b9de21cb..d3f1063b6c 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -784,6 +784,8 @@ static int ena_com_set_llq(struct ena_com_dev *ena_dev)
BIT(ENA_ADMIN_DISABLE_META_CACHING) |
BIT(ENA_ADMIN_LIMIT_TX_BURST);
+ cmd.feat_common.feature_version = llq_info->feature_version;
+
ret = ena_com_execute_admin_command_no_resp(admin_queue,
(struct ena_admin_aq_entry *)&cmd,
sizeof(cmd));
@@ -805,6 +807,8 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
memset(llq_info, 0, sizeof(*llq_info));
+ llq_info->feature_version = llq_features->feature_version;
+
supported_feat = llq_features->header_location_ctrl_supported;
if (likely(supported_feat & llq_default_cfg->llq_header_location)) {
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index c62fecdef2..7884d0dfc3 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -93,6 +93,7 @@ struct ena_com_llq_info {
u16 descs_per_entry;
u16 max_entries_in_tx_burst;
bool disable_meta_caching;
+ u8 feature_version;
};
struct ena_com_io_cq {
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 07/13] net/ena/base: deduplicate LLQ entry write and advance
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (5 preceding siblings ...)
2026-10-18 9:51 ` [PATCH v3 06/13] net/ena/base: fix LLQ feature version sent to device Shai Brandes
@ 2026-10-18 9:51 ` Shai Brandes
2026-10-18 9:51 ` [PATCH v3 08/13] net/ena/base: deduplicate meta caching config flag Shai Brandes
` (5 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:51 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
ena_com_close_llq_entry() and ena_com_sq_update_llq_tail() duplicate
the same sequence for flushing the current LLQ entry to the device and
moving on to the next one: write the bounce buffer, fetch the next
bounce buffer and reset it.
Extract the common sequence into a new helper,
ena_com_write_and_advance_llq_entry(), and call it from both sites.
No functional change.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_eth_com.c | 49 ++++++++++++++++--------------
1 file changed, 27 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c
index e3b2a753c4..ca15306859 100644
--- a/drivers/net/ena/base/ena_eth_com.c
+++ b/drivers/net/ena/base/ena_eth_com.c
@@ -204,6 +204,29 @@ static void *get_sq_desc_llq(struct ena_com_io_sq *io_sq)
return sq_desc;
}
+static int ena_com_write_and_advance_llq_entry(struct ena_com_io_sq *io_sq)
+{
+ struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
+ struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ int rc;
+
+ rc = ena_com_write_bounce_buffer_to_dev(io_sq,
+ pkt_ctrl->curr_llq_entry);
+ if (unlikely(rc)) {
+ ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
+ "Failed to write bounce buffer to device\n");
+ return rc;
+ }
+
+ pkt_ctrl->curr_llq_entry =
+ ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
+
+ memset(io_sq->llq_buf_ctrl.curr_llq_entry,
+ 0x0, llq_info->desc_list_entry_size);
+
+ return ENA_COM_OK;
+}
+
static int ena_com_close_llq_entry(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
@@ -215,18 +238,9 @@ static int ena_com_close_llq_entry(struct ena_com_io_sq *io_sq)
/* bounce buffer was used, so write it and get a new one */
if (likely(pkt_ctrl->idx)) {
- rc = ena_com_write_bounce_buffer_to_dev(io_sq,
- pkt_ctrl->curr_llq_entry);
- if (unlikely(rc)) {
- ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
- "Failed to write bounce buffer to device\n");
+ rc = ena_com_write_and_advance_llq_entry(io_sq);
+ if (unlikely(rc))
return rc;
- }
-
- pkt_ctrl->curr_llq_entry =
- ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
- memset(io_sq->llq_buf_ctrl.curr_llq_entry,
- 0x0, llq_info->desc_list_entry_size);
}
pkt_ctrl->idx = 0;
@@ -258,18 +272,9 @@ static int ena_com_sq_update_llq_tail(struct ena_com_io_sq *io_sq)
int rc;
if (!pkt_ctrl->descs_left_in_line) {
- rc = ena_com_write_bounce_buffer_to_dev(io_sq,
- pkt_ctrl->curr_llq_entry);
- if (unlikely(rc)) {
- ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
- "Failed to write bounce buffer to device\n");
+ rc = ena_com_write_and_advance_llq_entry(io_sq);
+ if (unlikely(rc))
return rc;
- }
-
- pkt_ctrl->curr_llq_entry =
- ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
- memset(io_sq->llq_buf_ctrl.curr_llq_entry,
- 0x0, llq_info->desc_list_entry_size);
pkt_ctrl->idx = 0;
pkt_ctrl->descs_left_in_line = llq_info->descs_per_entry;
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 08/13] net/ena/base: deduplicate meta caching config flag
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (6 preceding siblings ...)
2026-10-18 9:51 ` [PATCH v3 07/13] net/ena/base: deduplicate LLQ entry write and advance Shai Brandes
@ 2026-10-18 9:51 ` Shai Brandes
2026-10-18 9:52 ` [PATCH v3 09/13] net/ena/base: split LLQ info into datapath and config Shai Brandes
` (4 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:51 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
struct ena_com_io_sq stores disable_meta_caching twice: once as a flat
field, initialised from io_sq->llq_info.disable_meta_caching, and once
inside the embedded llq_info structure. Both are assigned during queue
configuration and never diverge afterwards, making the flat field
redundant.
Remove the flat io_sq->disable_meta_caching field along with its
initialisation, and read the value from llq_info directly on the
metadata path.
No functional change.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 2 --
drivers/net/ena/base/ena_com.h | 1 -
drivers/net/ena/base/ena_eth_com.c | 2 +-
3 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index d3f1063b6c..51bcf1b321 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -397,8 +397,6 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev,
0x0, io_sq->llq_info.desc_list_entry_size);
io_sq->llq_buf_ctrl.descs_left_in_line =
io_sq->llq_info.descs_num_before_header;
- io_sq->disable_meta_caching =
- io_sq->llq_info.disable_meta_caching;
if (io_sq->llq_info.max_entries_in_tx_burst > 0)
io_sq->entries_in_tx_burst_left =
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index 7884d0dfc3..063a8f021b 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -153,7 +153,6 @@ struct ena_com_io_sq {
enum queue_direction direction;
enum ena_admin_placement_policy_type mem_queue_type;
- bool disable_meta_caching;
u32 msix_vector;
struct ena_com_tx_meta cached_tx_meta;
diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c
index ca15306859..63624d169b 100644
--- a/drivers/net/ena/base/ena_eth_com.c
+++ b/drivers/net/ena/base/ena_eth_com.c
@@ -439,7 +439,7 @@ static int ena_com_create_and_store_tx_meta_desc(struct ena_com_io_sq *io_sq,
/* When disable meta caching is set, don't bother to save the meta and
* compare it to the stored version, just create the meta
*/
- if (io_sq->disable_meta_caching) {
+ if (io_sq->llq_info.disable_meta_caching) {
*have_meta = true;
return ena_com_create_meta(io_sq, ena_meta);
}
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 09/13] net/ena/base: split LLQ info into datapath and config
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (7 preceding siblings ...)
2026-10-18 9:51 ` [PATCH v3 08/13] net/ena/base: deduplicate meta caching config flag Shai Brandes
@ 2026-10-18 9:52 ` Shai Brandes
2026-10-18 9:53 ` [PATCH v3 10/13] net/ena/base: fix LLQ config failure flow Shai Brandes
` (3 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:52 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
struct ena_com_io_sq embeds the full struct ena_com_llq_info, although
the Tx datapath reads only a small subset of its fields. The remaining
members are consumed exclusively during device configuration.
Split struct ena_com_llq_info into struct ena_com_llq_info_common,
holding the fields read on the datapath (desc_list_entry_size,
descs_num_before_header, descs_per_entry, max_entries_in_tx_burst and
disable_meta_caching), and the configuration-only members, which remain
in the original structure.
Introduce struct ena_com_llq_info_runtime, comprising the common fields
and entries_in_tx_burst_left, and embed it in struct ena_com_io_sq in
place of the full structure.
No functional change.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 66 ++++++++++++++++--------------
drivers/net/ena/base/ena_com.h | 20 ++++++---
drivers/net/ena/base/ena_eth_com.c | 18 ++++----
drivers/net/ena/base/ena_eth_com.h | 14 +++----
4 files changed, 66 insertions(+), 52 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index 51bcf1b321..1ed73c1890 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -367,7 +367,7 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev,
} else {
/* Allocate bounce buffers */
io_sq->bounce_buf_ctrl.buffer_size =
- ena_dev->llq_info.desc_list_entry_size;
+ ena_dev->llq_info.common.desc_list_entry_size;
io_sq->bounce_buf_ctrl.buffers_num =
ENA_COM_BOUNCE_BUFFER_CNTRL_CNT;
io_sq->bounce_buf_ctrl.next_to_use = 0;
@@ -387,20 +387,20 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev,
return ENA_COM_NO_MEM;
}
- memcpy(&io_sq->llq_info, &ena_dev->llq_info,
- sizeof(io_sq->llq_info));
+ memcpy(&io_sq->llq_runtime.common, &ena_dev->llq_info.common,
+ sizeof(io_sq->llq_runtime.common));
/* Initiate the first bounce buffer */
io_sq->llq_buf_ctrl.curr_llq_entry =
ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl);
memset(io_sq->llq_buf_ctrl.curr_llq_entry,
- 0x0, io_sq->llq_info.desc_list_entry_size);
+ 0x0, io_sq->llq_runtime.common.desc_list_entry_size);
io_sq->llq_buf_ctrl.descs_left_in_line =
- io_sq->llq_info.descs_num_before_header;
+ io_sq->llq_runtime.common.descs_num_before_header;
- if (io_sq->llq_info.max_entries_in_tx_burst > 0)
- io_sq->entries_in_tx_burst_left =
- io_sq->llq_info.max_entries_in_tx_burst;
+ if (io_sq->llq_runtime.common.max_entries_in_tx_burst > 0)
+ io_sq->llq_runtime.entries_in_tx_burst_left =
+ io_sq->llq_runtime.common.max_entries_in_tx_burst;
}
io_sq->tail = 0;
@@ -775,7 +775,7 @@ static int ena_com_set_llq(struct ena_com_dev *ena_dev)
cmd.u.llq.header_location_ctrl_enabled = llq_info->header_location_ctrl;
cmd.u.llq.entry_size_ctrl_enabled = llq_info->desc_list_entry_size_ctrl;
- cmd.u.llq.desc_num_before_header_enabled = llq_info->descs_num_before_header;
+ cmd.u.llq.desc_num_before_header_enabled = llq_info->common.descs_num_before_header;
cmd.u.llq.descriptors_stride_ctrl_enabled = llq_info->desc_stride_ctrl;
cmd.u.llq.accel_mode.u.set.enabled_flags =
@@ -845,17 +845,17 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
supported_feat = llq_features->entry_size_ctrl_supported;
if (likely(supported_feat & llq_default_cfg->llq_ring_entry_size)) {
llq_info->desc_list_entry_size_ctrl = llq_default_cfg->llq_ring_entry_size;
- llq_info->desc_list_entry_size = llq_default_cfg->llq_ring_entry_size_value;
+ llq_info->common.desc_list_entry_size = llq_default_cfg->llq_ring_entry_size_value;
} else {
if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_128B) {
llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_128B;
- llq_info->desc_list_entry_size = 128;
+ llq_info->common.desc_list_entry_size = 128;
} else if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_192B) {
llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_192B;
- llq_info->desc_list_entry_size = 192;
+ llq_info->common.desc_list_entry_size = 192;
} else if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_256B) {
llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_256B;
- llq_info->desc_list_entry_size = 256;
+ llq_info->common.desc_list_entry_size = 256;
} else {
ena_trc_err(ena_dev, "Invalid entry_size_ctrl, supported: 0x%x\n",
supported_feat);
@@ -865,35 +865,40 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
ena_trc_err(ena_dev, "Default llq ring entry size is not supported, performing fallback, default: 0x%x, supported: 0x%x, used: 0x%x\n",
llq_default_cfg->llq_ring_entry_size,
supported_feat,
- llq_info->desc_list_entry_size);
+ llq_info->common.desc_list_entry_size);
}
- if (unlikely(llq_info->desc_list_entry_size & 0x7)) {
+ if (unlikely(llq_info->common.desc_list_entry_size & 0x7)) {
/* The desc list entry size should be whole multiply of 8
* This requirement comes from __iowrite64_copy()
*/
ena_trc_err(ena_dev, "Illegal entry size %d\n",
- llq_info->desc_list_entry_size);
+ llq_info->common.desc_list_entry_size);
return ENA_COM_INVAL;
}
if (llq_info->desc_stride_ctrl == ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY)
- llq_info->descs_per_entry = llq_info->desc_list_entry_size /
+ llq_info->common.descs_per_entry = llq_info->common.desc_list_entry_size /
sizeof(struct ena_eth_io_tx_desc);
else
- llq_info->descs_per_entry = 1;
+ llq_info->common.descs_per_entry = 1;
supported_feat = llq_features->desc_num_before_header_supported;
if (likely(supported_feat & llq_default_cfg->llq_num_decs_before_header)) {
- llq_info->descs_num_before_header = llq_default_cfg->llq_num_decs_before_header;
+ llq_info->common.descs_num_before_header =
+ llq_default_cfg->llq_num_decs_before_header;
} else {
if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2) {
- llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2;
+ llq_info->common.descs_num_before_header =
+ ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2;
} else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1) {
- llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1;
+ llq_info->common.descs_num_before_header =
+ ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1;
} else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4) {
- llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4;
+ llq_info->common.descs_num_before_header =
+ ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4;
} else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8) {
- llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8;
+ llq_info->common.descs_num_before_header =
+ ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8;
} else {
ena_trc_err(ena_dev, "Invalid descs_num_before_header, supported: 0x%x\n",
supported_feat);
@@ -903,19 +908,19 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
ena_trc_err(ena_dev, "Default llq num descs before header is not supported, performing fallback, default: 0x%x, supported: 0x%x, used: 0x%x\n",
llq_default_cfg->llq_num_decs_before_header,
supported_feat,
- llq_info->descs_num_before_header);
+ llq_info->common.descs_num_before_header);
}
/* Check for accelerated queue supported */
llq_accel_mode_get = llq_features->accel_mode.u.get;
- llq_info->disable_meta_caching =
+ llq_info->common.disable_meta_caching =
!!(llq_accel_mode_get.supported_flags &
BIT(ENA_ADMIN_DISABLE_META_CACHING));
if (llq_accel_mode_get.supported_flags & BIT(ENA_ADMIN_LIMIT_TX_BURST))
- llq_info->max_entries_in_tx_burst =
+ llq_info->common.max_entries_in_tx_burst =
llq_accel_mode_get.max_tx_burst_size /
- llq_default_cfg->llq_ring_entry_size_value;
+ llq_info->common.desc_list_entry_size;
rc = ena_com_set_llq(ena_dev);
if (unlikely(rc))
@@ -1055,7 +1060,8 @@ static void ena_com_io_queue_free(struct ena_com_dev *ena_dev,
if (io_sq->bounce_buf_ctrl.base_buffer) {
ENA_MEM_FREE(ena_dev->dmadev,
io_sq->bounce_buf_ctrl.base_buffer,
- (io_sq->llq_info.desc_list_entry_size * ENA_COM_BOUNCE_BUFFER_CNTRL_CNT));
+ (io_sq->llq_runtime.common.desc_list_entry_size *
+ ENA_COM_BOUNCE_BUFFER_CNTRL_CNT));
io_sq->bounce_buf_ctrl.base_buffer = NULL;
}
}
@@ -3586,8 +3592,8 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
if (unlikely(rc))
return rc;
- ena_dev->tx_max_header_size = llq_info->desc_list_entry_size -
- (llq_info->descs_num_before_header * sizeof(struct ena_eth_io_tx_desc));
+ ena_dev->tx_max_header_size = llq_info->common.desc_list_entry_size -
+ (llq_info->common.descs_num_before_header * sizeof(struct ena_eth_io_tx_desc));
if (unlikely(ena_dev->tx_max_header_size == 0)) {
ena_trc_err(ena_dev, "The size of the LLQ entry is smaller than needed\n");
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index 063a8f021b..94aa285cd8 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -84,18 +84,27 @@ struct ena_com_tx_meta {
u16 l4_hdr_len; /* In words */
};
-struct ena_com_llq_info {
- u16 header_location_ctrl;
- u16 desc_stride_ctrl;
- u16 desc_list_entry_size_ctrl;
+struct ena_com_llq_info_common {
u16 desc_list_entry_size;
u16 descs_num_before_header;
u16 descs_per_entry;
u16 max_entries_in_tx_burst;
bool disable_meta_caching;
+};
+
+struct ena_com_llq_info {
+ struct ena_com_llq_info_common common;
+ u16 header_location_ctrl;
+ u16 desc_stride_ctrl;
+ u16 desc_list_entry_size_ctrl;
u8 feature_version;
};
+struct ena_com_llq_info_runtime {
+ struct ena_com_llq_info_common common;
+ u16 entries_in_tx_burst_left;
+};
+
struct ena_com_io_cq {
struct ena_com_io_desc_addr cdesc_addr;
void *bus;
@@ -156,7 +165,7 @@ struct ena_com_io_sq {
u32 msix_vector;
struct ena_com_tx_meta cached_tx_meta;
- struct ena_com_llq_info llq_info;
+ struct ena_com_llq_info_runtime llq_runtime;
struct ena_com_llq_pkt_ctrl llq_buf_ctrl;
struct ena_com_io_bounce_buffer_control bounce_buf_ctrl;
@@ -171,7 +180,6 @@ struct ena_com_io_sq {
u8 phase;
u8 desc_entry_size;
u8 dma_addr_bits;
- u16 entries_in_tx_burst_left;
} ____cacheline_aligned;
struct ena_com_admin_cq {
diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c
index 63624d169b..ea0fb374db 100644
--- a/drivers/net/ena/base/ena_eth_com.c
+++ b/drivers/net/ena/base/ena_eth_com.c
@@ -112,7 +112,7 @@ static void *get_sq_desc_regular_queue(struct ena_com_io_sq *io_sq)
static int ena_com_write_bounce_buffer_to_dev(struct ena_com_io_sq *io_sq,
u8 *bounce_buffer)
{
- struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ struct ena_com_llq_info_common *llq_info = &io_sq->llq_runtime.common;
u16 dst_tail_mask;
u32 dst_offset;
@@ -120,16 +120,16 @@ static int ena_com_write_bounce_buffer_to_dev(struct ena_com_io_sq *io_sq,
dst_offset = dst_tail_mask * llq_info->desc_list_entry_size;
if (is_llq_max_tx_burst_exists(io_sq)) {
- if (unlikely(!io_sq->entries_in_tx_burst_left)) {
+ if (unlikely(!io_sq->llq_runtime.entries_in_tx_burst_left)) {
ena_trc_err(ena_com_io_sq_to_ena_dev(io_sq),
"Error: trying to send more packets than tx burst allows\n");
return ENA_COM_NO_SPACE;
}
- io_sq->entries_in_tx_burst_left--;
+ io_sq->llq_runtime.entries_in_tx_burst_left--;
ena_trc_dbg(ena_com_io_sq_to_ena_dev(io_sq),
"Decreasing entries_in_tx_burst_left of queue %u to %u\n",
- io_sq->qid, io_sq->entries_in_tx_burst_left);
+ io_sq->qid, io_sq->llq_runtime.entries_in_tx_burst_left);
}
/* Make sure everything was written into the bounce buffer before
@@ -156,7 +156,7 @@ static int ena_com_write_header_to_llq_entry(struct ena_com_io_sq *io_sq,
u16 header_len)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
- struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ struct ena_com_llq_info_common *llq_info = &io_sq->llq_runtime.common;
u8 *curr_llq_entry = pkt_ctrl->curr_llq_entry;
u16 header_offset;
@@ -207,7 +207,7 @@ static void *get_sq_desc_llq(struct ena_com_io_sq *io_sq)
static int ena_com_write_and_advance_llq_entry(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
- struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ struct ena_com_llq_info_common *llq_info = &io_sq->llq_runtime.common;
int rc;
rc = ena_com_write_bounce_buffer_to_dev(io_sq,
@@ -230,7 +230,7 @@ static int ena_com_write_and_advance_llq_entry(struct ena_com_io_sq *io_sq)
static int ena_com_close_llq_entry(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
- struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ struct ena_com_llq_info_common *llq_info = &io_sq->llq_runtime.common;
int rc;
if (unlikely(!ena_com_sq_is_llq(io_sq)))
@@ -268,7 +268,7 @@ static void *get_tx_sq_desc(struct ena_com_io_sq *io_sq)
static int ena_com_sq_update_llq_tail(struct ena_com_io_sq *io_sq)
{
struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl;
- struct ena_com_llq_info *llq_info = &io_sq->llq_info;
+ struct ena_com_llq_info_common *llq_info = &io_sq->llq_runtime.common;
int rc;
if (!pkt_ctrl->descs_left_in_line) {
@@ -439,7 +439,7 @@ static int ena_com_create_and_store_tx_meta_desc(struct ena_com_io_sq *io_sq,
/* When disable meta caching is set, don't bother to save the meta and
* compare it to the stored version, just create the meta
*/
- if (io_sq->llq_info.disable_meta_caching) {
+ if (io_sq->llq_runtime.common.disable_meta_caching) {
*have_meta = true;
return ena_com_create_meta(io_sq, ena_meta);
}
diff --git a/drivers/net/ena/base/ena_eth_com.h b/drivers/net/ena/base/ena_eth_com.h
index 503cf45cf7..514df12e9d 100644
--- a/drivers/net/ena/base/ena_eth_com.h
+++ b/drivers/net/ena/base/ena_eth_com.h
@@ -118,7 +118,7 @@ static inline bool ena_com_sq_have_enough_space(struct ena_com_io_sq *io_sq,
* (one for the header line and one to compensate the devision
* down calculation.
*/
- temp = required_buffers / io_sq->llq_info.descs_per_entry + 2;
+ temp = required_buffers / io_sq->llq_runtime.common.descs_per_entry + 2;
return ena_com_free_q_entries(io_sq) > temp;
}
@@ -137,13 +137,13 @@ static inline bool ena_com_meta_desc_changed(struct ena_com_io_sq *io_sq,
static inline bool is_llq_max_tx_burst_exists(struct ena_com_io_sq *io_sq)
{
return likely(ena_com_sq_is_llq(io_sq)) &&
- (io_sq->llq_info.max_entries_in_tx_burst > 0);
+ (io_sq->llq_runtime.common.max_entries_in_tx_burst > 0);
}
static inline bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq,
struct ena_com_tx_ctx *ena_tx_ctx)
{
- struct ena_com_llq_info *llq_info;
+ struct ena_com_llq_info_common *llq_info;
int descs_after_first_entry;
int num_entries_needed = 1;
u16 num_descs;
@@ -151,7 +151,7 @@ static inline bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq,
if (!is_llq_max_tx_burst_exists(io_sq))
return false;
- llq_info = &io_sq->llq_info;
+ llq_info = &io_sq->llq_runtime.common;
num_descs = ena_tx_ctx->num_bufs;
if (llq_info->disable_meta_caching ||
@@ -168,7 +168,7 @@ static inline bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq,
"Queue: %d num_descs: %d num_entries_needed: %d\n",
io_sq->qid, num_descs, num_entries_needed);
- return num_entries_needed > io_sq->entries_in_tx_burst_left;
+ return num_entries_needed > io_sq->llq_runtime.entries_in_tx_burst_left;
}
static inline void ena_com_write_rx_sq_doorbell(struct ena_com_io_sq *io_sq)
@@ -184,7 +184,7 @@ static inline void ena_com_write_rx_sq_doorbell(struct ena_com_io_sq *io_sq)
static inline void ena_com_write_tx_sq_doorbell(struct ena_com_io_sq *io_sq)
{
- u16 max_entries_in_tx_burst = io_sq->llq_info.max_entries_in_tx_burst;
+ u16 max_entries_in_tx_burst = io_sq->llq_runtime.common.max_entries_in_tx_burst;
u16 tail = io_sq->tail;
ena_trc_dbg(ena_com_io_sq_to_ena_dev(io_sq),
@@ -197,7 +197,7 @@ static inline void ena_com_write_tx_sq_doorbell(struct ena_com_io_sq *io_sq)
ena_trc_dbg(ena_com_io_sq_to_ena_dev(io_sq),
"Reset available entries in tx burst for queue %d to %d\n",
io_sq->qid, max_entries_in_tx_burst);
- io_sq->entries_in_tx_burst_left = max_entries_in_tx_burst;
+ io_sq->llq_runtime.entries_in_tx_burst_left = max_entries_in_tx_burst;
}
}
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 10/13] net/ena/base: fix LLQ config failure flow
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (8 preceding siblings ...)
2026-10-18 9:52 ` [PATCH v3 09/13] net/ena/base: split LLQ info into datapath and config Shai Brandes
@ 2026-10-18 9:53 ` Shai Brandes
2026-10-18 9:53 ` [PATCH v3 11/13] net/ena: support flow API for n-tuple Rx steering Shai Brandes
` (2 subsequent siblings)
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:53 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes, stable
ena_com_config_dev_mode() computes tx_max_header_size directly
into the ena_com_dev structure and only afterwards rejects the
result if it came out zero. The error path therefore leaves the
device with tx_max_header_size == 0, overwriting the value the
device reported in ena_com_get_dev_attr_feat().
The failure is not fatal to the caller: the PMD logs it, falls
back to host placement policy and continues, so the zeroed value
stays in effect. It is then copied into the per-ring field, which
sizes the Tx intermediate header buffer to zero and clamps
header_len to zero on every transmit.
Compute the value into a local variable, validate it there, and
store it into ena_dev only once it is known good, so a failed LLQ
configuration leaves the previously negotiated value untouched.
Fixes: b2b02edeb0d6 ("net/ena/base: upgrade HAL for new HW features")
Cc: stable@dpdk.org
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/base/ena_com.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index 1ed73c1890..161d5b5ebd 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -3581,6 +3581,7 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
struct ena_llq_configurations *llq_default_cfg)
{
struct ena_com_llq_info *llq_info = &ena_dev->llq_info;
+ u32 llq_max_header_size;
int rc;
if (!llq_features->max_llq_num) {
@@ -3592,14 +3593,15 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
if (unlikely(rc))
return rc;
- ena_dev->tx_max_header_size = llq_info->common.desc_list_entry_size -
+ llq_max_header_size = llq_info->common.desc_list_entry_size -
(llq_info->common.descs_num_before_header * sizeof(struct ena_eth_io_tx_desc));
- if (unlikely(ena_dev->tx_max_header_size == 0)) {
+ if (unlikely(llq_max_header_size == 0)) {
ena_trc_err(ena_dev, "The size of the LLQ entry is smaller than needed\n");
return ENA_COM_INVAL;
}
+ ena_dev->tx_max_header_size = llq_max_header_size;
ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_DEV;
return 0;
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 11/13] net/ena: support flow API for n-tuple Rx steering
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (9 preceding siblings ...)
2026-10-18 9:53 ` [PATCH v3 10/13] net/ena/base: fix LLQ config failure flow Shai Brandes
@ 2026-10-18 9:53 ` Shai Brandes
2026-10-18 9:53 ` [PATCH v3 12/13] net/ena/base: remove x86 Tx LLQ write barrier Shai Brandes
2026-10-18 9:54 ` [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0 Shai Brandes
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:53 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Implement the flow_ops_get ethdev callback to expose hardware
n-tuple Rx steering through the rte_flow API, allowing applications
to direct selected traffic to dedicated Rx queues for workload
isolation.
Supported patterns are IPv4, IPv6, TCP and UDP, matching source and
destination addresses, source and destination ports and IP TOS/DSCP.
The protocol is derived from the TCP or UDP item type. The only
supported action is queue steering to a single Rx queue. Rules are
accepted for ingress traffic in group 0 with priority 0. Item spec
and mask are honoured, including default masks; ranges specified
through the item last field are rejected, as are unsupported
header fields.
Flow steering is reported as unsupported when the device exposes no
rule entries.
Documented limitations: rules can only be created once the device
is started, they are not retained across device stop, and a rule
referencing an Rx queue that is removed by a later reconfiguration
will cause matching packets to be dropped.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
doc/guides/nics/ena.rst | 26 +
doc/guides/nics/features/ena.ini | 10 +
doc/guides/rel_notes/release_26_11.rst | 4 +
drivers/net/ena/base/ena_com.c | 300 ++++++++++
drivers/net/ena/base/ena_com.h | 86 +++
.../net/ena/base/ena_defs/ena_admin_defs.h | 83 ++-
drivers/net/ena/ena_ethdev.c | 71 ++-
drivers/net/ena/ena_ethdev.h | 3 +-
drivers/net/ena/ena_flow.c | 542 ++++++++++++++++++
drivers/net/ena/ena_flow.h | 12 +
drivers/net/ena/meson.build | 1 +
11 files changed, 1135 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/ena/ena_flow.c
create mode 100644 drivers/net/ena/ena_flow.h
diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst
index d829e8854f..4c2f0e098a 100644
--- a/doc/guides/nics/ena.rst
+++ b/doc/guides/nics/ena.rst
@@ -33,6 +33,7 @@ Supported features
* Device reset upon failure
* Rx interrupts
* Rx HW packet timestamping
+* Rx flow steering
Overview
--------
@@ -59,6 +60,9 @@ Queue (LLQ), which saves several more microseconds.
Rx hardware timestamping is available in the ENA driver
on platforms that support this feature.
+Rx flow steering is available in the ENA driver
+on platforms that support this feature.
+
Management Interface
--------------------
@@ -248,6 +252,28 @@ There is possibility that there won't be enough event file descriptors to
handle both admin and Rx interrupts. In that situation the Rx interrupt request
will fail.
+**Rx flow steering support**
+
+ENA PMD supports steering ingress packets to a specific Rx queue
+using the generic flow API (``rte_flow``).
+A rule matches IPv4 or IPv6 packets, optionally followed by TCP or UDP,
+on any combination of source and destination addresses,
+TOS (IPv4) or traffic class (IPv6), and source and destination ports.
+Each rule has a single ``QUEUE`` action.
+
+The following limitations apply:
+
+* Only ingress rules in group 0 with priority 0 are supported.
+* The ``ETH`` item is accepted only as a wildcard, without matching fields.
+* Rules can be created only after the device is started.
+* ``RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP`` is not advertised,
+ so the application must not rely on rules being kept across device stop,
+ and should flush them with ``rte_flow_flush()`` before stopping the device.
+* The queue index is checked only when the rule is created.
+ If the port is later reconfigured with fewer Rx queues,
+ the device silently drops the packets matching the rule.
+* The maximum number of rules is defined by the device.
+
**Note about usage on \*.metal instances**
On AWS, the metal instances are supporting IOMMU for both arm64 and x86_64 hosts.
diff --git a/doc/guides/nics/features/ena.ini b/doc/guides/nics/features/ena.ini
index 037e0bad60..a8cbb6df1c 100644
--- a/doc/guides/nics/features/ena.ini
+++ b/doc/guides/nics/features/ena.ini
@@ -22,3 +22,13 @@ Linux = Y
ARMv8 = Y
x86-32 = Y
x86-64 = Y
+
+[rte_flow items]
+eth = P
+ipv4 = Y
+ipv6 = Y
+tcp = Y
+udp = Y
+
+[rte_flow actions]
+queue = Y
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 21ea1aed3d..a34b81c9b6 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -133,6 +133,10 @@ New Features
``xdp_meta_rx_ts_valid_mask``.
* Added ``read_clock`` operation to query the PTP hardware clock.
+* **Updated Amazon ENA (Elastic Network Adapter) ethernet driver.**
+
+ * Added Rx flow steering support using the generic flow API.
+
* **Updated Broadcom bnxt driver.**
* Added ``tx_dma_err_cmpl`` statistics to report Tx completions
diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c
index 161d5b5ebd..741a2dbbd8 100644
--- a/drivers/net/ena/base/ena_com.c
+++ b/drivers/net/ena/base/ena_com.c
@@ -3345,6 +3345,62 @@ void ena_com_rss_destroy(struct ena_com_dev *ena_dev)
memset(&ena_dev->rss, 0x0, sizeof(ena_dev->rss));
}
+int ena_com_flow_steering_init(struct ena_com_dev *ena_dev, u16 flow_steering_entries)
+{
+ u32 tbl_size_in_bytes =
+ flow_steering_entries * sizeof(struct ena_com_flow_steering_table_entry);
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+
+ memset(flow_steering, 0x0, sizeof(*flow_steering));
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG)))
+ return ENA_COM_UNSUPPORTED;
+
+ flow_steering->tbl_size = flow_steering_entries;
+
+ flow_steering->flow_steering_tbl = ENA_MEM_ALLOC(ena_dev->dmadev, tbl_size_in_bytes);
+ if (unlikely(!flow_steering->flow_steering_tbl)) {
+ ena_trc_err(ena_dev, "Flow steering table memory allocation failed\n");
+ return ENA_COM_NO_MEM;
+ }
+
+ ENA_MEM_ALLOC_COHERENT(ena_dev->dmadev,
+ sizeof(struct ena_admin_flow_steering_rule_params),
+ flow_steering->requested_rule,
+ flow_steering->requested_rule_dma_addr,
+ flow_steering->requested_rule_mem_handle);
+ if (unlikely(!flow_steering->requested_rule)) {
+ ena_trc_err(ena_dev, "Flow steering dma-able params memory allocation failed\n");
+ goto err;
+ }
+
+ return 0;
+err:
+ ena_com_flow_steering_destroy(ena_dev);
+
+ return ENA_COM_NO_MEM;
+}
+
+void ena_com_flow_steering_destroy(struct ena_com_dev *ena_dev)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+
+ if (flow_steering->requested_rule) {
+ ENA_MEM_FREE_COHERENT(ena_dev->dmadev,
+ sizeof(struct ena_admin_flow_steering_rule_params),
+ flow_steering->requested_rule,
+ flow_steering->requested_rule_dma_addr,
+ flow_steering->requested_rule_mem_handle);
+ }
+
+ if (flow_steering->flow_steering_tbl) {
+ ENA_MEM_FREE(ena_dev->dmadev,
+ flow_steering->flow_steering_tbl,
+ flow_steering->tbl_size *
+ sizeof(struct ena_com_flow_steering_table_entry));
+ }
+}
+
int ena_com_allocate_host_info(struct ena_com_dev *ena_dev)
{
struct ena_host_attribute *host_attr = &ena_dev->host_attr;
@@ -3607,6 +3663,250 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
return 0;
}
+int ena_com_flow_steering_add_rule(struct ena_com_dev *ena_dev,
+ struct ena_com_flow_steering_rule_params *configure_params,
+ u16 *rule_idx)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+ struct ena_com_admin_queue *admin_queue;
+ struct ena_admin_set_feat_resp resp;
+ struct ena_admin_set_feat_cmd cmd;
+ int ret;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG))) {
+ ena_trc_err(ena_dev, "Flow steering rules are not supported by this device\n");
+ return ENA_COM_UNSUPPORTED;
+ }
+
+ if ((*rule_idx >= flow_steering->tbl_size) &&
+ (*rule_idx != ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION)) {
+ ena_trc_err(ena_dev, "Failed to add a flow steering rule, index %u out of bounds\n",
+ *rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ if ((*rule_idx != ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION) &&
+ flow_steering->flow_steering_tbl[*rule_idx].in_use) {
+ ena_trc_err(ena_dev,
+ "Failed to configure Flow steering rule to index %u with currently active rule in it\n",
+ *rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ memset(&cmd, 0x0, sizeof(cmd));
+ admin_queue = &ena_dev->admin_queue;
+
+ cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE;
+ cmd.aq_common_descriptor.flags =
+ ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_INDIRECT_MASK;
+ cmd.feat_common.feature_id = ENA_ADMIN_FLOW_STEERING_CONFIG;
+ cmd.u.flow_steering.action = ENA_ADMIN_FLOW_STEERING_ADD_RULE;
+ cmd.u.flow_steering.flow_type = configure_params->flow_type;
+ cmd.u.flow_steering.rx_q_idx = configure_params->qid;
+ cmd.u.flow_steering.rule_location = *rule_idx;
+ cmd.u.flow_steering.flags = 0;
+
+ memcpy(flow_steering->requested_rule,
+ &configure_params->flow_params,
+ sizeof(struct ena_admin_flow_steering_rule_params));
+
+ ret = ena_com_mem_addr_set(ena_dev,
+ &cmd.control_buffer.address,
+ flow_steering->requested_rule_dma_addr);
+ if (unlikely(ret)) {
+ ena_trc_err(ena_dev, "Memory address set failed\n");
+ return ret;
+ }
+
+ cmd.control_buffer.length = sizeof(struct ena_admin_flow_steering_rule_params);
+
+ ret = ena_com_execute_admin_command(admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd),
+ (struct ena_admin_acq_entry *)&resp,
+ sizeof(resp));
+ if (unlikely(ret)) {
+ ena_trc_err(ena_dev, "Failed to add a new flow steering rule: %d\n", ret);
+ return ret;
+ }
+
+ /* If the rule index needs to be chosen by the device,
+ * set it to the rule_idx from the response
+ */
+ if (*rule_idx == ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION) {
+ *rule_idx = resp.u.flow_steering.rule_location;
+ if (*rule_idx >= flow_steering->tbl_size) {
+ ena_trc_err(ena_dev, "Flow steering rule configured to invalid index: %d\n",
+ *rule_idx);
+ return ENA_COM_FAULT;
+ }
+ }
+
+ flow_steering->flow_steering_tbl[*rule_idx].rule_params = *configure_params;
+ flow_steering->flow_steering_tbl[*rule_idx].in_use = true;
+
+ flow_steering->active_rules_cnt++;
+
+ return 0;
+}
+
+int ena_com_flow_steering_remove_rule(struct ena_com_dev *ena_dev, u16 rule_idx)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+ struct ena_admin_set_feat_resp resp;
+ struct ena_admin_set_feat_cmd cmd;
+ int ret;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG))) {
+ ena_trc_err(ena_dev, "Flow steering rules are not supported by this device\n");
+ return ENA_COM_UNSUPPORTED;
+ }
+
+ if (rule_idx >= flow_steering->tbl_size) {
+ ena_trc_err(ena_dev, "Failed to remove a flow steering rule, index %u out of bounds\n",
+ rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ if (!flow_steering->flow_steering_tbl[rule_idx].in_use) {
+ ena_trc_err(ena_dev,
+ "Failed to remove a flow steering rule, no rule configured in index %u\n",
+ rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ memset(&cmd, 0x0, sizeof(cmd));
+
+ cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE;
+ cmd.feat_common.feature_id = ENA_ADMIN_FLOW_STEERING_CONFIG;
+ cmd.u.flow_steering.action = ENA_ADMIN_FLOW_STEERING_REMOVE_RULE;
+ cmd.u.flow_steering.rule_location = rule_idx;
+
+ ret = ena_com_execute_admin_command(&ena_dev->admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd),
+ (struct ena_admin_acq_entry *)&resp,
+ sizeof(resp));
+ if (unlikely(ret)) {
+ ena_trc_err(ena_dev, "Failed to remove a flow steering rule: %d\n", ret);
+ return ret;
+ }
+
+ memset(&flow_steering->flow_steering_tbl[rule_idx].rule_params, 0,
+ sizeof(struct ena_admin_flow_steering_rule_params));
+ flow_steering->flow_steering_tbl[rule_idx].in_use = false;
+
+ flow_steering->active_rules_cnt--;
+
+ return 0;
+}
+
+int ena_com_flow_steering_remove_all_rules(struct ena_com_dev *ena_dev)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+ struct ena_admin_set_feat_resp resp;
+ struct ena_admin_set_feat_cmd cmd;
+ int ret;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG))) {
+ ena_trc_err(ena_dev, "Flow steering rules are not supported by this device\n");
+ return ENA_COM_UNSUPPORTED;
+ }
+
+ memset(&cmd, 0x0, sizeof(cmd));
+
+ cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE;
+ cmd.feat_common.feature_id = ENA_ADMIN_FLOW_STEERING_CONFIG;
+ cmd.u.flow_steering.action = ENA_ADMIN_FLOW_STEERING_REMOVE_ALL_RULES;
+
+ ret = ena_com_execute_admin_command(&ena_dev->admin_queue,
+ (struct ena_admin_aq_entry *)&cmd,
+ sizeof(cmd),
+ (struct ena_admin_acq_entry *)&resp,
+ sizeof(resp));
+ if (unlikely(ret)) {
+ ena_trc_err(ena_dev, "Failed to remove all flow steering rules: %d\n", ret);
+ return ret;
+ }
+
+ memset(flow_steering->flow_steering_tbl, 0,
+ flow_steering->tbl_size * sizeof(struct ena_com_flow_steering_table_entry));
+ flow_steering->active_rules_cnt = 0;
+
+ return 0;
+}
+
+int ena_com_flow_steering_get_rule(struct ena_com_dev *ena_dev,
+ struct ena_com_flow_steering_rule_params *configure_params,
+ u16 rule_idx)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+ struct ena_com_flow_steering_table_entry *entry;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG))) {
+ ena_trc_err(ena_dev, "Flow steering rules are not supported by this device\n");
+ return ENA_COM_UNSUPPORTED;
+ }
+
+ if (rule_idx >= flow_steering->tbl_size) {
+ ena_trc_err(ena_dev, "Failed to get a flow steering rule, index %u out bounds\n",
+ rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ entry = &flow_steering->flow_steering_tbl[rule_idx];
+
+ if (!entry->in_use) {
+ ena_trc_err(ena_dev, "Failed to get a flow steering rule in index %u, entry not in use\n",
+ rule_idx);
+ return ENA_COM_INVAL;
+ }
+
+ *configure_params = entry->rule_params;
+
+ return 0;
+}
+
+int ena_com_flow_steering_restore_device_rules(struct ena_com_dev *ena_dev)
+{
+ struct ena_com_flow_steering *flow_steering = &ena_dev->flow_steering;
+ struct ena_com_flow_steering_table_entry *rule_entry;
+ u16 rule_idx;
+ int ret;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG)))
+ return ENA_COM_UNSUPPORTED;
+
+ /* no rules to restore */
+ if (flow_steering->active_rules_cnt == 0)
+ return 0;
+
+ /* set the amount of active rules to zero, will count them again while restoring */
+ flow_steering->active_rules_cnt = 0;
+
+ for (rule_idx = 0; rule_idx < flow_steering->tbl_size; rule_idx++) {
+ rule_entry = &flow_steering->flow_steering_tbl[rule_idx];
+
+ if (rule_entry->in_use) {
+ /* mark the entry as not in use before attempt to reconfigure it
+ * so it will be counted as new rule
+ */
+ rule_entry->in_use = false;
+
+ ret = ena_com_flow_steering_add_rule(ena_dev, &rule_entry->rule_params,
+ &rule_idx);
+ if (unlikely(ret)) {
+ ena_trc_err(ena_dev,
+ "Failed to restore flow steering rule in index %d\n",
+ rule_idx);
+ return ENA_COM_FAULT;
+ }
+ }
+ }
+
+ return 0;
+}
+
int ena_com_set_frag_bypass(struct ena_com_dev *ena_dev, bool enable)
{
struct ena_com_admin_queue *admin_queue;
diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h
index 94aa285cd8..183c9c2b22 100644
--- a/drivers/net/ena/base/ena_com.h
+++ b/drivers/net/ena/base/ena_com.h
@@ -368,6 +368,28 @@ struct ena_host_attribute {
ena_mem_handle_t host_info_dma_handle;
};
+struct ena_com_flow_steering_rule_params {
+ struct ena_admin_flow_steering_rule_params flow_params;
+ u16 qid;
+ /* Specific flow type as defined in enum ena_admin_flow_steering_type */
+ u8 flow_type;
+};
+
+struct ena_com_flow_steering_table_entry {
+ struct ena_com_flow_steering_rule_params rule_params;
+ bool in_use;
+};
+
+struct ena_com_flow_steering {
+ struct ena_com_flow_steering_table_entry *flow_steering_tbl;
+ u16 tbl_size;
+ u16 active_rules_cnt;
+
+ ena_mem_handle_t requested_rule_mem_handle;
+ struct ena_admin_flow_steering_rule_params *requested_rule;
+ dma_addr_t requested_rule_dma_addr;
+};
+
/* Each ena_dev is a PCI function. */
struct ena_com_dev {
struct ena_com_admin_queue admin_queue;
@@ -410,6 +432,9 @@ struct ena_com_dev {
struct ena_com_llq_info llq_info;
struct ena_customer_metrics customer_metrics;
+
+ struct ena_com_flow_steering flow_steering;
+
bool use_extended_tx_cdesc;
bool use_extended_rx_cdesc;
};
@@ -799,6 +824,23 @@ int ena_com_rss_init(struct ena_com_dev *ena_dev);
*/
void ena_com_rss_destroy(struct ena_com_dev *ena_dev);
+/* ena_com_flow_steering_init - Init Flow steering
+ * @ena_dev: ENA communication layer struct
+ * @flow_steering_entries: Number of Flow steering entries to use.
+ *
+ * Allocate the table to hold the steering rules context.
+ *
+ * @return: 0 on success and negative value otherwise.
+ */
+int ena_com_flow_steering_init(struct ena_com_dev *ena_dev, u16 flow_steering_entries);
+
+/* ena_com_flow_steering_destroy - Destroy flow steering
+ * @ena_dev: ENA communication layer struct
+ *
+ * Free the flow steering allocated resources.
+ */
+void ena_com_flow_steering_destroy(struct ena_com_dev *ena_dev);
+
/* ena_com_get_current_hash_function - Get RSS hash function
* @ena_dev: ENA communication layer struct
*
@@ -1120,6 +1162,50 @@ static inline bool ena_com_get_missing_admin_interrupt(struct ena_com_dev *ena_d
return ena_dev->admin_queue.is_missing_admin_interrupt;
}
+/* ena_com_flow_steering_add_rule - configure new Rx flow steering rule
+ * @ena_dev: ENA communication layer struct
+ * @configure_params: Steering rule params, including queue and flow type.
+ * @rule_idx: Index in rules table to configure the rule into, on return
+ * it will hold the actual index of the configured rule.
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+int ena_com_flow_steering_add_rule(struct ena_com_dev *ena_dev,
+ struct ena_com_flow_steering_rule_params *configure_params,
+ u16 *rule_idx);
+
+/* ena_com_flow_steering_remove_rule - Remove an existing RX flow steering rule
+ * @ena_dev: ENA communication layer struct
+ * @rule_idx: Rule table index to delete
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+int ena_com_flow_steering_remove_rule(struct ena_com_dev *ena_dev, u16 rule_idx);
+
+/* ena_com_flow_steering_remove_all_rules - Remove all flow steering rules
+ * @ena_dev: ENA communication layer struct
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+int ena_com_flow_steering_remove_all_rules(struct ena_com_dev *ena_dev);
+
+/* ena_com_flow_steering_get_rule - retrieve info about specific steering rule
+ * @ena_dev: ENA communication layer struct
+ * @configure_params: pointer to be filled with the steering rule parameters.
+ * @rule_idx: Rule index to get info about.
+ *
+ * @return - 0 on success, negative value on failure.
+ */
+int ena_com_flow_steering_get_rule(struct ena_com_dev *ena_dev,
+ struct ena_com_flow_steering_rule_params *configure_params,
+ u16 rule_idx);
+
+/* ena_com_flow_steering_restore_device_rules - reconfigure the existing rules to the device
+ * after resets caused by errors
+ * @ena_dev: ENA communication layer struct
+ */
+int ena_com_flow_steering_restore_device_rules(struct ena_com_dev *ena_dev);
+
/* ena_com_set_frag_bypass - set fragment bypass
* @ena_dev: ENA communication layer struct
* @enable: true if fragment bypass is enabled, false otherwise.
diff --git a/drivers/net/ena/base/ena_defs/ena_admin_defs.h b/drivers/net/ena/base/ena_defs/ena_admin_defs.h
index 717d21f9b9..64b4f8cb5f 100644
--- a/drivers/net/ena/base/ena_defs/ena_admin_defs.h
+++ b/drivers/net/ena/base/ena_defs/ena_admin_defs.h
@@ -9,6 +9,8 @@
#define ENA_ADMIN_EXTRA_PROPERTIES_STRING_LEN 32
#define ENA_ADMIN_EXTRA_PROPERTIES_COUNT 32
+#define ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION 0xFFFF
+
#define ENA_ADMIN_RSS_KEY_PARTS 10
#define ENA_ADMIN_CUSTOMER_METRICS_SUPPORT_MASK 0x3F
@@ -68,6 +70,7 @@ enum ena_admin_aq_feature_id {
ENA_ADMIN_LINK_CONFIG = 27,
ENA_ADMIN_HOST_ATTR_CONFIG = 28,
ENA_ADMIN_PHC_CONFIG = 29,
+ ENA_ADMIN_FLOW_STEERING_CONFIG = 30,
ENA_ADMIN_HW_TIMESTAMP = 31,
ENA_ADMIN_FEATURES_OPCODE_NUM = 32,
};
@@ -600,7 +603,7 @@ struct ena_admin_device_attr_feature_desc {
/* unicast MAC address (in Network byte order) */
uint8_t mac_addr[6];
- uint8_t reserved7[2];
+ uint16_t flow_steering_max_entries;
uint32_t max_mtu;
};
@@ -751,6 +754,78 @@ struct ena_admin_feature_hw_ts_desc {
uint8_t rx;
};
+struct ena_admin_flow_steering_rule_params {
+ uint8_t dst_ip[16];
+
+ uint8_t dst_ip_mask[16];
+
+ uint8_t src_ip[16];
+
+ uint8_t src_ip_mask[16];
+
+ uint16_t dst_port;
+
+ uint16_t dst_port_mask;
+
+ uint16_t src_port;
+
+ uint16_t src_port_mask;
+
+ uint8_t tos;
+
+ uint8_t tos_mask;
+
+ /* Reserved */
+ uint8_t reserved1[54];
+};
+
+/* add new rule or remove existing one */
+enum ena_admin_flow_steering_action {
+ ENA_ADMIN_FLOW_STEERING_ADD_RULE = 1,
+ ENA_ADMIN_FLOW_STEERING_REMOVE_RULE = 2,
+ ENA_ADMIN_FLOW_STEERING_REMOVE_ALL_RULES = 3,
+};
+
+/* type of traffic the rule applied on */
+enum ena_admin_flow_steering_type {
+ ENA_ADMIN_FLOW_INVALID = 0,
+ ENA_ADMIN_FLOW_IPV4 = 1,
+ ENA_ADMIN_FLOW_IPV6 = 2,
+ ENA_ADMIN_FLOW_IPV4_TCP = 3,
+ ENA_ADMIN_FLOW_IPV6_TCP = 4,
+ ENA_ADMIN_FLOW_IPV4_UDP = 5,
+ ENA_ADMIN_FLOW_IPV6_UDP = 6,
+ ENA_ADMIN_FLOW_LAST = 7,
+};
+
+struct ena_admin_set_feature_flow_steering_desc_req {
+ /* specific command action as defined in enum
+ * ena_admin_flow_steering_action
+ */
+ uint8_t action;
+
+ /* specific flow type as defined in enum ena_admin_flow_steering_type */
+ uint8_t flow_type;
+
+ uint16_t rx_q_idx;
+
+ /* if value is ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION it means
+ * the device should find unused location for this rule
+ */
+ uint16_t rule_location;
+
+ uint16_t reserved;
+
+ /* 31:0 : reserved */
+ uint32_t flags;
+};
+
+struct ena_admin_set_feature_flow_steering_desc_resp {
+ uint16_t rule_location;
+
+ uint16_t reserved;
+};
+
struct ena_admin_feature_frag_bypass_desc {
/* Enable frag_bypass */
uint8_t enable;
@@ -1234,6 +1309,9 @@ struct ena_admin_set_feat_cmd {
/* PHC configuration */
struct ena_admin_feature_phc_desc phc;
+ /* Flow steering configuration */
+ struct ena_admin_set_feature_flow_steering_desc_req flow_steering;
+
/* Fragment bypass configuration */
struct ena_admin_feature_frag_bypass_desc frag_bypass;
@@ -1246,6 +1324,9 @@ struct ena_admin_set_feat_resp {
union {
uint32_t raw[14];
+
+ /* Flow steering configuration */
+ struct ena_admin_set_feature_flow_steering_desc_resp flow_steering;
} u;
};
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index c2d0194e51..3eee5e2545 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -240,6 +240,7 @@ static void ena_tx_map_mbuf(struct ena_ring *tx_ring,
uint16_t *header_len);
static int ena_xmit_mbuf(struct ena_ring *tx_ring, struct rte_mbuf *mbuf);
static int ena_tx_cleanup(void *txp, uint32_t free_pkt_cnt);
+static int ena_flow_ops_get(struct rte_eth_dev *dev, const struct rte_flow_ops **ops);
static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
static uint16_t eth_ena_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
@@ -360,6 +361,7 @@ static const struct eth_dev_ops ena_dev_ops = {
.rss_hash_update = ena_rss_hash_update,
.rss_hash_conf_get = ena_rss_hash_conf_get,
.tx_done_cleanup = ena_tx_cleanup,
+ .flow_ops_get = ena_flow_ops_get,
};
/*********************************************************************
@@ -944,6 +946,12 @@ static int ena_close(struct rte_eth_dev *dev)
rte_eal_alarm_cancel(ena_control_path_poll_handler, dev);
}
+ if (adapter->flow_rule_count) {
+ PMD_INIT_LOG_LINE(WARNING,
+ "Closing device with %u active flow rules",
+ adapter->flow_rule_count);
+ adapter->flow_rule_count = 0;
+ }
ena_rx_queue_release_all(dev);
ena_tx_queue_release_all(dev);
@@ -952,6 +960,7 @@ static int ena_close(struct rte_eth_dev *dev)
adapter->drv_stats = NULL;
ena_com_set_admin_running_state(ena_dev, false);
ena_com_rss_destroy(ena_dev);
+ ena_com_flow_steering_destroy(ena_dev);
ena_com_delete_debug_area(ena_dev);
ena_com_delete_host_info(ena_dev);
ena_com_abort_admin_commands(ena_dev);
@@ -1460,6 +1469,17 @@ static int ena_stop(struct rte_eth_dev *dev)
PMD_DRV_LOG_LINE(ERR, "Device reset failed, rc: %d", rc);
}
+ /*
+ * Warn if application didn't flush rules before stopping.
+ * RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP is not advertised, so applications must
+ * not rely on rules being kept across device stop. The PMD doesn't flush
+ * them either, so the remaining rules stay active until destroyed.
+ */
+ if (adapter->flow_rule_count)
+ PMD_DRV_LOG_LINE(WARNING,
+ "Stopping device with %u active flow rules. "
+ "Application should call rte_flow_flush() before stop.",
+ adapter->flow_rule_count);
rte_intr_disable(intr_handle);
rte_intr_efd_disable(intr_handle);
@@ -2546,6 +2566,19 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
goto err_rss_destroy;
}
}
+
+ /* A device that reports no rule entries can't hold any rule */
+ if (!get_feat_ctx.dev_attr.flow_steering_max_entries)
+ ena_dev->supported_features &= ~BIT(ENA_ADMIN_FLOW_STEERING_CONFIG);
+
+ rc = ena_com_flow_steering_init(ena_dev,
+ get_feat_ctx.dev_attr.flow_steering_max_entries);
+ if (rc && rc != ENA_COM_UNSUPPORTED) {
+ PMD_DRV_LOG_LINE(ERR, "Failed to init flow steering rules: %d", rc);
+ goto err_indirect_table_destroy;
+ }
+ adapter->flow_rule_count = 0;
+
adapter->drv_stats = rte_zmalloc("adapter stats",
sizeof(*adapter->drv_stats),
RTE_CACHE_LINE_SIZE);
@@ -2553,7 +2586,7 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
PMD_DRV_LOG_LINE(ERR,
"Failed to allocate memory for adapter statistics");
rc = -ENOMEM;
- goto err_indirect_table_destroy;
+ goto err_rx_flow_steering_destroy;
}
rte_spinlock_init(&adapter->admin_lock);
@@ -2597,6 +2630,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
}
err_stats_destroy:
rte_free(adapter->drv_stats);
+err_rx_flow_steering_destroy:
+ ena_com_flow_steering_destroy(ena_dev);
err_indirect_table_destroy:
ena_indirect_table_release(adapter);
err_rss_destroy:
@@ -2818,6 +2853,24 @@ static int ena_infos_get(struct rte_eth_dev *dev,
dev_info->err_handle_mode = RTE_ETH_ERROR_HANDLE_MODE_PASSIVE;
+ /*
+ * Flow rule lifecycle compliance:
+ * rte_flow specification states that if this flag is not advertised,
+ * rules cannot be created until the device is started for the
+ * first time and cannot be kept when the device is stopped.
+ * Explicitly clear RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP to inform applications
+ * that they must not rely on rules being kept across device stop, and
+ * should call rte_flow_flush() before stopping the device.
+ */
+ dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+ /*
+ * ENA PMD does not implement internal locking for flow operations.
+ * We clear RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE to tell DPDK that the
+ * rte_flow API layer must provide thread-safety protection (via mutex)
+ * when multiple threads call flow operations concurrently.
+ */
+ dev->data->dev_flags &= ~RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE;
+
return 0;
}
@@ -4354,3 +4407,19 @@ static bool ena_use_large_llq_hdr(struct ena_adapter *adapter, uint8_t recommend
}
return false;
}
+
+static int
+ena_flow_ops_get(struct rte_eth_dev *dev, const struct rte_flow_ops **ops)
+{
+ struct ena_adapter *adapter = dev->data->dev_private;
+ struct ena_com_dev *ena_dev = &adapter->ena_dev;
+
+ if (!(ena_dev->supported_features & BIT(ENA_ADMIN_FLOW_STEERING_CONFIG))) {
+ PMD_DRV_LOG_LINE(WARNING, "Flow steering not supported");
+ *ops = NULL;
+ rte_errno = ENOTSUP;
+ return -rte_errno;
+ }
+ *ops = &ena_flow_ops;
+ return 0;
+}
diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h
index b204b07767..7a7d4a4036 100644
--- a/drivers/net/ena/ena_ethdev.h
+++ b/drivers/net/ena/ena_ethdev.h
@@ -15,7 +15,7 @@
#include <rte_timer.h>
#include <rte_dev.h>
#include <rte_net.h>
-
+#include "ena_flow.h"
#include "ena_com.h"
#define ENA_REGS_BAR 0
@@ -343,6 +343,7 @@ struct ena_adapter {
struct ena_stats_dev dev_stats;
struct ena_admin_basic_stats basic_stats;
+ uint16_t flow_rule_count;
u32 *indirect_table;
size_t indirect_table_size;
diff --git a/drivers/net/ena/ena_flow.c b/drivers/net/ena/ena_flow.c
new file mode 100644
index 0000000000..1594c02f87
--- /dev/null
+++ b/drivers/net/ena/ena_flow.c
@@ -0,0 +1,542 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Amazon.com, Inc. or its affiliates.
+ */
+
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_malloc.h>
+
+#include "ena_ethdev.h"
+#include "ena_logs.h"
+#include "base/ena_com.h"
+#include "ena_flow.h"
+
+/* Flow handle encoding - no heap allocation needed */
+static inline struct rte_flow *
+ena_flow_handle_encode(uint16_t rule_idx)
+{
+ return (struct rte_flow *)(uintptr_t)(rule_idx + 1);
+}
+
+static inline int
+ena_flow_handle_decode(const struct rte_flow *flow, uint16_t *rule_idx)
+{
+ uintptr_t v = (uintptr_t)flow;
+ if (v == 0 || v > (uintptr_t)UINT16_MAX + 1)
+ return -EINVAL;
+ *rule_idx = (uint16_t)(v - 1);
+ return 0;
+}
+
+/* Convert to ENA flow type - note: returns uint8_t for flow_type field */
+static uint8_t
+ena_flow_type_convert(bool ipv4, bool ipv6, bool tcp, bool udp)
+{
+ if (ipv4 && tcp)
+ return ENA_ADMIN_FLOW_IPV4_TCP;
+ if (ipv4 && udp)
+ return ENA_ADMIN_FLOW_IPV4_UDP;
+ if (ipv4)
+ return ENA_ADMIN_FLOW_IPV4;
+ if (ipv6 && tcp)
+ return ENA_ADMIN_FLOW_IPV6_TCP;
+ if (ipv6 && udp)
+ return ENA_ADMIN_FLOW_IPV6_UDP;
+ if (ipv6)
+ return ENA_ADMIN_FLOW_IPV6;
+
+ return ENA_ADMIN_FLOW_INVALID;
+}
+
+/* Parse rte_flow pattern and actions into ENA format */
+static int
+ena_flow_parse(struct rte_eth_dev *dev,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct ena_com_flow_steering_rule_params *cfg,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item *item;
+ const struct rte_flow_action *action;
+ bool has_ipv4 = false, has_ipv6 = false;
+ bool has_tcp = false, has_udp = false;
+ bool has_queue = false;
+ uint8_t flow_type;
+
+ if (!pattern || !actions || !cfg)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "Null flow input");
+
+ memset(cfg, 0, sizeof(*cfg));
+
+ /* Parse pattern items - spec/mask semantics, ranges (last) are rejected */
+ for (item = pattern; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->type == RTE_FLOW_ITEM_TYPE_VOID)
+ continue;
+
+ switch (item->type) {
+ case RTE_FLOW_ITEM_TYPE_ETH: {
+ const struct rte_flow_item_eth *spec = item->spec;
+ const struct rte_flow_item_eth *mask = item->mask;
+
+ /* Reject mask/last without spec */
+ if (!spec && (mask || item->last))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "mask/last without spec");
+
+ if (!spec)
+ break; /* wildcard ETH */
+
+ /* Use default mask if not provided */
+ if (!mask)
+ mask = &rte_flow_item_eth_mask;
+
+ /* Reject any ETH field matching (not supported) */
+ if (mask->hdr.ether_type != 0)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "EtherType matching not supported");
+
+ for (int i = 0; i < 6; i++) {
+ if (mask->hdr.dst_addr.addr_bytes[i] ||
+ mask->hdr.src_addr.addr_bytes[i])
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "MAC address matching not supported");
+ }
+ break;
+ }
+
+ case RTE_FLOW_ITEM_TYPE_IPV4: {
+ const struct rte_flow_item_ipv4 *spec = item->spec;
+ const struct rte_flow_item_ipv4 *mask = item->mask;
+
+ /* Reject mask/last without spec */
+ if (!spec && (mask || item->last))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "mask/last without spec");
+
+ if (!spec)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv4 item requires spec");
+
+ if (item->last)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Range (last) not supported");
+
+ has_ipv4 = true;
+ if (has_ipv6)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Mixed IPv4/IPv6 not supported");
+
+ /* Use default mask if not provided */
+ if (!mask)
+ mask = &rte_flow_item_ipv4_mask;
+
+ /* Reject unsupported mask fields */
+ if (mask->hdr.version_ihl || mask->hdr.total_length ||
+ mask->hdr.packet_id || mask->hdr.fragment_offset ||
+ mask->hdr.time_to_live || mask->hdr.next_proto_id ||
+ mask->hdr.hdr_checksum)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported IPv4 header mask fields");
+
+ /* Copy supported fields: src/dst + tos */
+ memcpy(cfg->flow_params.src_ip, &spec->hdr.src_addr, 4);
+ memcpy(cfg->flow_params.src_ip_mask, &mask->hdr.src_addr, 4);
+ memcpy(cfg->flow_params.dst_ip, &spec->hdr.dst_addr, 4);
+ memcpy(cfg->flow_params.dst_ip_mask, &mask->hdr.dst_addr, 4);
+ cfg->flow_params.tos = spec->hdr.type_of_service;
+ cfg->flow_params.tos_mask = mask->hdr.type_of_service;
+ break;
+ }
+
+ case RTE_FLOW_ITEM_TYPE_IPV6: {
+ const struct rte_flow_item_ipv6 *spec = item->spec;
+ const struct rte_flow_item_ipv6 *mask = item->mask;
+
+ /* Reject mask/last without spec */
+ if (!spec && (mask || item->last))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "mask/last without spec");
+
+ if (!spec)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv6 item requires spec");
+
+ if (item->last)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Range (last) not supported");
+
+ has_ipv6 = true;
+ if (has_ipv4)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Mixed IPv4/IPv6 not supported");
+
+ /* Use default mask if not provided */
+ if (!mask)
+ mask = &rte_flow_item_ipv6_mask;
+
+ /* Reject unsupported IPv6 mask fields */
+ if (mask->hdr.payload_len || mask->hdr.proto || mask->hdr.hop_limits)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported IPv6 header mask fields");
+
+ /* Only allow traffic class bits in vtc_flow mask */
+ uint32_t vtc_m = rte_be_to_cpu_32(mask->hdr.vtc_flow);
+ if (vtc_m & 0xF0000000u) /* version bits */
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv6 version matching not supported");
+ if (vtc_m & 0x000FFFFFu) /* flowlabel bits */
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "IPv6 flowlabel matching not supported");
+
+ memcpy(cfg->flow_params.src_ip, spec->hdr.src_addr.a, 16);
+ memcpy(cfg->flow_params.src_ip_mask, mask->hdr.src_addr.a, 16);
+ memcpy(cfg->flow_params.dst_ip, spec->hdr.dst_addr.a, 16);
+ memcpy(cfg->flow_params.dst_ip_mask, mask->hdr.dst_addr.a, 16);
+
+ uint32_t vtc = rte_be_to_cpu_32(spec->hdr.vtc_flow);
+ cfg->flow_params.tos = (uint8_t)((vtc >> 20) & 0xFF);
+ cfg->flow_params.tos_mask = (uint8_t)((vtc_m >> 20) & 0xFF);
+ break;
+ }
+
+ case RTE_FLOW_ITEM_TYPE_TCP: {
+ const struct rte_flow_item_tcp *spec = item->spec;
+ const struct rte_flow_item_tcp *mask = item->mask;
+
+ /* Reject mask/last without spec */
+ if (!spec && (mask || item->last))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "mask/last without spec");
+
+ if (!spec)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "TCP item requires spec");
+
+ if (item->last)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Range (last) not supported");
+
+ has_tcp = true;
+ if (has_udp)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Mixed TCP/UDP not supported");
+
+ /* Use default mask if not provided */
+ if (!mask)
+ mask = &rte_flow_item_tcp_mask;
+
+ /* Reject unsupported TCP mask fields */
+ if (mask->hdr.sent_seq || mask->hdr.recv_ack ||
+ mask->hdr.data_off || mask->hdr.tcp_flags ||
+ mask->hdr.rx_win || mask->hdr.cksum || mask->hdr.tcp_urp)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported TCP header mask fields");
+
+ cfg->flow_params.src_port = rte_be_to_cpu_16(spec->hdr.src_port);
+ cfg->flow_params.src_port_mask = rte_be_to_cpu_16(mask->hdr.src_port);
+ cfg->flow_params.dst_port = rte_be_to_cpu_16(spec->hdr.dst_port);
+ cfg->flow_params.dst_port_mask = rte_be_to_cpu_16(mask->hdr.dst_port);
+ break;
+ }
+
+ case RTE_FLOW_ITEM_TYPE_UDP: {
+ const struct rte_flow_item_udp *spec = item->spec;
+ const struct rte_flow_item_udp *mask = item->mask;
+
+ /* Reject mask/last without spec */
+ if (!spec && (mask || item->last))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "mask/last without spec");
+
+ if (!spec)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "UDP item requires spec");
+
+ if (item->last)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Range (last) not supported");
+
+ has_udp = true;
+ if (has_tcp)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Mixed TCP/UDP not supported");
+
+ /* Use default mask if not provided */
+ if (!mask)
+ mask = &rte_flow_item_udp_mask;
+
+ /* Reject unsupported UDP mask fields */
+ if (mask->hdr.dgram_len || mask->hdr.dgram_cksum)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "Unsupported UDP header mask fields");
+
+ cfg->flow_params.src_port = rte_be_to_cpu_16(spec->hdr.src_port);
+ cfg->flow_params.src_port_mask = rte_be_to_cpu_16(mask->hdr.src_port);
+ cfg->flow_params.dst_port = rte_be_to_cpu_16(spec->hdr.dst_port);
+ cfg->flow_params.dst_port_mask = rte_be_to_cpu_16(mask->hdr.dst_port);
+ break;
+ }
+
+ default:
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Unsupported pattern item");
+ }
+ }
+
+ /* Determine flow type */
+ flow_type = ena_flow_type_convert(has_ipv4, has_ipv6, has_tcp, has_udp);
+ if (flow_type == ENA_ADMIN_FLOW_INVALID)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ NULL, "Invalid pattern: requires IPv4 or IPv6");
+
+ cfg->flow_type = flow_type;
+
+ /* Parse actions: exactly one QUEUE, no other non-VOID actions */
+ for (action = actions; action->type != RTE_FLOW_ACTION_TYPE_END; action++) {
+ switch (action->type) {
+ case RTE_FLOW_ACTION_TYPE_VOID:
+ break;
+
+ case RTE_FLOW_ACTION_TYPE_QUEUE: {
+ const struct rte_flow_action_queue *queue = action->conf;
+
+ if (!queue)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Missing queue configuration");
+
+ if (has_queue)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Multiple QUEUE actions not supported");
+
+ /*
+ * Validate queue index. It is checked only here: if the port is
+ * later reconfigured with fewer Rx queues, the device silently
+ * drops the packets matching this rule.
+ */
+ if (queue->index >= dev->data->nb_rx_queues)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Queue index out of range");
+
+ cfg->qid = queue->index;
+ has_queue = true;
+ break;
+ }
+
+ default:
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Unsupported action type");
+ }
+ }
+
+ /* QUEUE action is mandatory */
+ if (!has_queue)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL, "QUEUE action required");
+
+ return 0;
+}
+
+/* Validate flow rule */
+static int
+ena_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct ena_com_flow_steering_rule_params cfg;
+
+ /* Check device is started - use STATE error type, not EAGAIN */
+ if (!dev->data->dev_started)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_STATE, dev,
+ "Device not started; flow rules cannot be created in this state");
+
+ /* Validate attributes */
+ if (attr->group != 0)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+ attr, "Only group 0 supported");
+
+ if (attr->priority != 0)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
+ attr, "Priority not supported");
+
+ if (!attr->ingress || attr->egress || attr->transfer)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR,
+ attr, "Only ingress supported");
+
+ /* Validate pattern and actions */
+ return ena_flow_parse(dev, pattern, actions, &cfg, error);
+}
+
+/* Create flow rule */
+static struct rte_flow *
+ena_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct ena_adapter *adapter = dev->data->dev_private;
+ struct ena_com_dev *ena_dev = &adapter->ena_dev;
+ struct ena_com_flow_steering_rule_params cfg;
+ uint16_t rule_idx = ENA_ADMIN_FLOW_STEERING_DEVICE_CHOOSE_LOCATION;
+ int ret;
+
+ /* Validate attributes, parse pattern and actions into cfg */
+ ret = ena_flow_validate(dev, attr, pattern, actions, error);
+ if (ret)
+ return NULL;
+
+ /* Parse pattern and actions */
+ ret = ena_flow_parse(dev, pattern, actions, &cfg, error);
+ if (ret)
+ return NULL;
+
+ /* Add rule to hardware via base layer */
+ ret = ena_com_flow_steering_add_rule(ena_dev, &cfg, &rule_idx);
+ if (ret) {
+ PMD_DRV_LOG_LINE(ERR, "Failed to add flow steering rule: %d", ret);
+ rte_flow_error_set(error, (ret == -EAGAIN) ? EBUSY : -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to add flow steering rule");
+ return NULL;
+ }
+
+ /* Increment flow counter for lifecycle tracking */
+ adapter->flow_rule_count++;
+
+ PMD_DRV_LOG_LINE(DEBUG, "Flow rule created at HW location %u (total: %u)",
+ rule_idx, adapter->flow_rule_count);
+
+ /* Return encoded handle (no heap allocation) */
+ return ena_flow_handle_encode(rule_idx);
+}
+
+/* Destroy flow rule */
+static int
+ena_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *rte_flow,
+ struct rte_flow_error *error)
+{
+ struct ena_adapter *adapter = dev->data->dev_private;
+ struct ena_com_dev *ena_dev = &adapter->ena_dev;
+ uint16_t rule_idx;
+ int ret;
+
+ /* Decode handle */
+ ret = ena_flow_handle_decode(rte_flow, &rule_idx);
+ if (ret)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Invalid flow handle");
+
+ /* Remove rule from hardware */
+ ret = ena_com_flow_steering_remove_rule(ena_dev, rule_idx);
+ if (ret) {
+ PMD_DRV_LOG_LINE(ERR, "Failed to remove flow rule %u: %d",
+ rule_idx, ret);
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to remove rule from hardware");
+ }
+
+ /* Decrement flow counter */
+ if (adapter->flow_rule_count > 0)
+ adapter->flow_rule_count--;
+
+ PMD_DRV_LOG_LINE(DEBUG, "Flow rule %u destroyed (remaining: %u)",
+ rule_idx, adapter->flow_rule_count);
+
+ /* No free needed - handle is encoded, not allocated */
+ return 0;
+}
+
+/* Flush all flow rules */
+static int
+ena_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ struct ena_adapter *adapter = dev->data->dev_private;
+ struct ena_com_dev *ena_dev = &adapter->ena_dev;
+ int ret;
+
+ /* Remove all rules from hardware */
+ ret = ena_com_flow_steering_remove_all_rules(ena_dev);
+ if (ret) {
+ PMD_DRV_LOG_LINE(ERR, "Failed to flush all flow rules: %d", ret);
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "Failed to flush rules from hardware");
+ }
+
+ PMD_DRV_LOG_LINE(DEBUG, "Flushed %u flow rules", adapter->flow_rule_count);
+
+ /* Reset flow counter */
+ adapter->flow_rule_count = 0;
+
+ /* No handles to free - all were encoded, not allocated */
+ return 0;
+}
+
+/* Query flow rule - not supported */
+static int
+ena_flow_query(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ const struct rte_flow_action *action,
+ void *data,
+ struct rte_flow_error *error)
+{
+ (void)dev;
+ (void)flow;
+ (void)data;
+
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ action, "Query not supported");
+}
+
+/* Flow operations structure */
+const struct rte_flow_ops ena_flow_ops = {
+ .validate = ena_flow_validate,
+ .create = ena_flow_create,
+ .destroy = ena_flow_destroy,
+ .flush = ena_flow_flush,
+ .query = ena_flow_query,
+};
diff --git a/drivers/net/ena/ena_flow.h b/drivers/net/ena/ena_flow.h
new file mode 100644
index 0000000000..e83aa56e87
--- /dev/null
+++ b/drivers/net/ena/ena_flow.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Amazon.com, Inc. or its affiliates.
+ */
+#ifndef _ENA_FLOW_H_
+#define _ENA_FLOW_H_
+
+#include <rte_flow.h>
+
+/* Flow operations */
+extern const struct rte_flow_ops ena_flow_ops;
+
+#endif /* _ENA_FLOW_H_ */
diff --git a/drivers/net/ena/meson.build b/drivers/net/ena/meson.build
index 2b18d7add7..244775bf16 100644
--- a/drivers/net/ena/meson.build
+++ b/drivers/net/ena/meson.build
@@ -10,6 +10,7 @@ endif
sources = files(
'ena_ethdev.c',
'ena_rss.c',
+ 'ena_flow.c',
'base/ena_com.c',
'base/ena_eth_com.c',
)
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 12/13] net/ena/base: remove x86 Tx LLQ write barrier
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (10 preceding siblings ...)
2026-10-18 9:53 ` [PATCH v3 11/13] net/ena: support flow API for n-tuple Rx steering Shai Brandes
@ 2026-10-18 9:53 ` Shai Brandes
2026-10-18 9:54 ` [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0 Shai Brandes
12 siblings, 0 replies; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:53 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes, Amit Bernstein
ena_com_write_bounce_buffer_to_dev() issues a wmb() before copying each
LLQ entry from the bounce buffer to the write-combining (WC) LLQ memory
region. It was intended to ensure the bounce buffer is fully written
before the copy reads it. That is never needed: a CPU always observes
its own prior stores, on every architecture. wmb() also orders stores
against later stores, not against later loads, so it could not provide
that guarantee anyway.
The barrier is needed on ARM for a different reason: it orders one LLQ
entry's stores to the device ahead of the next entry's. The copy uses
relaxed stores, which on ARM may reach the device partially and out of
order while the burst is still being written. The barrier in the
doorbell write cannot prevent this, as it only orders the entries
against the doorbell, not against each other. With the barrier removed,
c8gn at 32 queues fails with IPP_REASS_ERROR_PRE_VALID_DB ("Data Out of
Window") before the doorbell is written.
On x86 the barrier is redundant. WC stores are weakly ordered there
too, so it never ordered entries against each other. The entries reach
the device before the doorbell because of the memory type, not a fence:
each entry is written as complete 64-byte WC lines, and on Intel the UC
doorbell write flushes pending WC buffers, while on AMD each 64-byte WC
line is flushed once it is fully written. The per-entry SFENCE made the
CPU wait for the previous entry to reach the device before copying the
next one, defeating the purpose of write combining.
Introduce arm_wmb(), a write barrier on ARM and a no-op elsewhere, and
use it in place of wmb() before the bounce buffer copy.
Signed-off-by: Amit Bernstein <amitbern@amazon.com>
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
doc/guides/rel_notes/release_26_11.rst | 2 ++
drivers/net/ena/base/ena_eth_com.c | 23 ++++++++++++++++++++---
drivers/net/ena/base/ena_plat_dpdk.h | 8 ++++++++
3 files changed, 30 insertions(+), 3 deletions(-)
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index a34b81c9b6..9b8e5ccc90 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -136,6 +136,8 @@ New Features
* **Updated Amazon ENA (Elastic Network Adapter) ethernet driver.**
* Added Rx flow steering support using the generic flow API.
+ * Restrict the Tx LLQ write barrier to ARM, where it is required for correct entry ordering,
+ and remove it on x86, where it is redundant and limits throughput.
* **Updated Broadcom bnxt driver.**
diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c
index ea0fb374db..5dbb432acc 100644
--- a/drivers/net/ena/base/ena_eth_com.c
+++ b/drivers/net/ena/base/ena_eth_com.c
@@ -132,10 +132,27 @@ static int ena_com_write_bounce_buffer_to_dev(struct ena_com_io_sq *io_sq,
io_sq->qid, io_sq->llq_runtime.entries_in_tx_burst_left);
}
- /* Make sure everything was written into the bounce buffer before
- * writing the bounce buffer to the device
+ /* Ensure the previous LLQ entry reaches the device before this one.
+ *
+ * The original wmb() was intended to ensure the bounce buffer is
+ * fully written before it is copied to the device. This is not
+ * required: a CPU always observes its own prior stores, on every
+ * architecture. Moreover, wmb() orders stores against later stores,
+ * not against later loads, so it could not provide this guarantee.
+ *
+ * The copy below uses relaxed stores to the WC LLQ region: plain
+ * stores with no barrier, which the CPU may send to the device in any
+ * order. On ARM, this means an entry may reach the device partially
+ * and out of order before the doorbell is written. In addition, the
+ * barrier in the doorbell write cannot prevent this, as it only orders
+ * the entries against the doorbell, not against each other.
+ *
+ * On x86, this barrier is not needed. Each entry is written as complete
+ * 64-byte WC lines, all of which reach the device before the
+ * doorbell: on Intel, the UC doorbell write flushes pending WC
+ * buffers; on AMD, each line is flushed once it is fully written.
*/
- wmb();
+ arm_wmb();
/* The line is completed. Copy it to dev */
ENA_MEMCPY_TO_DEVICE_64(io_sq->desc_addr.pbuf_dev_addr + dst_offset,
diff --git a/drivers/net/ena/base/ena_plat_dpdk.h b/drivers/net/ena/base/ena_plat_dpdk.h
index 4a6d53d0a2..12f1643a85 100644
--- a/drivers/net/ena/base/ena_plat_dpdk.h
+++ b/drivers/net/ena/base/ena_plat_dpdk.h
@@ -73,6 +73,14 @@ typedef RTE_ATOMIC(int32_t) ena_atomic32_t;
#define rmb rte_rmb
#define mb rte_mb
#define mmiowb rte_io_wmb
+
+/* Write memory barrier for ARM architectures - covers 32 and 64 bit ARM */
+#if defined(RTE_ARCH_ARM)
+#define arm_wmb rte_wmb
+#else
+static inline void arm_wmb(void) { }
+#endif
+
#define __iomem
#ifndef READ_ONCE
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
` (11 preceding siblings ...)
2026-10-18 9:53 ` [PATCH v3 12/13] net/ena/base: remove x86 Tx LLQ write barrier Shai Brandes
@ 2026-10-18 9:54 ` Shai Brandes
2026-10-18 12:53 ` Brandes, Shai
12 siblings, 1 reply; 16+ messages in thread
From: Shai Brandes @ 2026-10-18 9:54 UTC (permalink / raw)
To: stephen; +Cc: dev, Shai Brandes
Upgrade driver version to 2.15.0
Signed-off-by: Shai Brandes <shaibran@amazon.com>
---
drivers/net/ena/ena_ethdev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 3eee5e2545..435c2af477 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -23,7 +23,7 @@
#include <ena_eth_io_defs.h>
#define DRV_MODULE_VER_MAJOR 2
-#define DRV_MODULE_VER_MINOR 14
+#define DRV_MODULE_VER_MINOR 15
#define DRV_MODULE_VER_SUBMINOR 0
#define __MERGE_64B_H_L(h, l) (((uint64_t)h << 32) | l)
--
2.17.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* RE: [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0
2026-10-18 9:54 ` [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0 Shai Brandes
@ 2026-10-18 12:53 ` Brandes, Shai
0 siblings, 0 replies; 16+ messages in thread
From: Brandes, Shai @ 2026-10-18 12:53 UTC (permalink / raw)
To: dev@dpdk.org; +Cc: stephen@networkplumber.org
Recheck-request: iol-unit-arm64-testing
It seems that the failure is environmental. Rechecking may fail again as I see other patches also fail on that test suite today.
> -----Original Message-----
> From: Brandes, Shai <shaibran@amazon.com>
> Sent: Sunday, October 18, 2026 12:54 PM
> To: stephen@networkplumber.org
> Cc: dev@dpdk.org; Brandes, Shai <shaibran@amazon.com>
> Subject: [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0
>
> Upgrade driver version to 2.15.0
>
> Signed-off-by: Shai Brandes <shaibran@amazon.com>
> ---
> drivers/net/ena/ena_ethdev.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index 3eee5e2545..435c2af477 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -23,7 +23,7 @@
> #include <ena_eth_io_defs.h>
>
> #define DRV_MODULE_VER_MAJOR 2
> -#define DRV_MODULE_VER_MINOR 14
> +#define DRV_MODULE_VER_MINOR 15
> #define DRV_MODULE_VER_SUBMINOR 0
>
> #define __MERGE_64B_H_L(h, l) (((uint64_t)h << 32) | l)
> --
> 2.17.1
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2026-10-18 12:54 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-18 6:36 [PATCH v2 00/13] ENA release 2.15.0 Shai Brandes
2026-10-18 9:45 ` [PATCH v3 " Shai Brandes
2026-10-18 9:46 ` [PATCH v3 01/13] net/ena: avoid using un-needed admin response buffers Shai Brandes
2026-10-18 9:47 ` [PATCH v3 02/13] net/ena/base: fix race in PHC destroy Shai Brandes
2026-10-18 9:47 ` [PATCH v3 03/13] net/ena/base: fix missing barriers in PHC timestamp read Shai Brandes
2026-10-18 9:47 ` [PATCH v3 04/13] net/ena/base: rename bounce buffer references Shai Brandes
2026-10-18 9:50 ` [PATCH v3 05/13] net/ena/base: add LLQ policy inline helper Shai Brandes
2026-10-18 9:51 ` [PATCH v3 06/13] net/ena/base: fix LLQ feature version sent to device Shai Brandes
2026-10-18 9:51 ` [PATCH v3 07/13] net/ena/base: deduplicate LLQ entry write and advance Shai Brandes
2026-10-18 9:51 ` [PATCH v3 08/13] net/ena/base: deduplicate meta caching config flag Shai Brandes
2026-10-18 9:52 ` [PATCH v3 09/13] net/ena/base: split LLQ info into datapath and config Shai Brandes
2026-10-18 9:53 ` [PATCH v3 10/13] net/ena/base: fix LLQ config failure flow Shai Brandes
2026-10-18 9:53 ` [PATCH v3 11/13] net/ena: support flow API for n-tuple Rx steering Shai Brandes
2026-10-18 9:53 ` [PATCH v3 12/13] net/ena/base: remove x86 Tx LLQ write barrier Shai Brandes
2026-10-18 9:54 ` [PATCH v3 13/13] net/ena: upgrade driver version to 2.15.0 Shai Brandes
2026-10-18 12:53 ` Brandes, Shai
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.