* [PATCH 01/45] crypto/dpaa2_sec: fix buffer overflow in GCM decrypt
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 02/45] crypto/dpaa2_sec: fix FLE pool leak on sec FD build failure Prashant Gupta
` (43 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: stable, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
In build_authenc_gcm_fd, when both AAD (auth_only_len > 0) and decrypt
direction are active, the SGE layout occupies 8 entries plus 16 bytes of
old_icv storage at index 8. The FLE pool buffer was only 256 bytes
(8 x 32), causing old_icv to be written one entry past the end of the
allocated buffer. The resulting virtual address was not mapped by the
IOMMU, so DPAA2_VADDR_TO_IOVA returned 0 and the SEC engine received
iova=0x00000000 as the ICV buffer address, triggering an SMMU
translation fault (FSR=0x402 TF).
Additionally, the upfront bpid/IVP initialization only covered sge+3,
leaving sge+4 (the input data SGE when AAD is present) without a valid
bpid or IVP assignment.
Increase FLE_POOL_BUF_SIZE from 256 to 288 (9 x 32 bytes) to
accommodate the full layout, and extend the bpid/IVP initialization
to cover sge+4 in both branches of build_authenc_gcm_fd.
Fixes: 13273250eec5 ("crypto/dpaa2_sec: support AES-GCM and CTR")
Cc: stable@dpdk.org
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 2 ++
drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
index 3d980d096f..2a015a3d82 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
@@ -569,6 +569,7 @@ build_authenc_gcm_fd(dpaa2_sec_session *sess,
DPAA2_SET_FLE_BPID(sge + 1, bpid);
DPAA2_SET_FLE_BPID(sge + 2, bpid);
DPAA2_SET_FLE_BPID(sge + 3, bpid);
+ DPAA2_SET_FLE_BPID(sge + 4, bpid);
} else {
DPAA2_SET_FD_IVP(fd);
DPAA2_SET_FLE_IVP(fle);
@@ -577,6 +578,7 @@ build_authenc_gcm_fd(dpaa2_sec_session *sess,
DPAA2_SET_FLE_IVP((sge + 1));
DPAA2_SET_FLE_IVP((sge + 2));
DPAA2_SET_FLE_IVP((sge + 3));
+ DPAA2_SET_FLE_IVP((sge + 4));
}
/* Save the shared descriptor */
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
index 755c8e9cc3..ff32f3d860 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
@@ -17,7 +17,7 @@ extern uint8_t cryptodev_driver_id;
/* FLE_POOL_NUM_BUFS is set as per the ipsec-secgw application */
#define FLE_POOL_NUM_BUFS 32000
-#define FLE_POOL_BUF_SIZE 256
+#define FLE_POOL_BUF_SIZE 288
#define FLE_POOL_CACHE_SIZE 512
#define FLE_SG_MEM_SIZE(num) (FLE_POOL_BUF_SIZE + ((num) * 32))
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 02/45] crypto/dpaa2_sec: fix FLE pool leak on sec FD build failure
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
2026-09-03 13:53 ` [PATCH 01/45] crypto/dpaa2_sec: fix buffer overflow in GCM decrypt Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 03/45] crypto/dpaa2_sec: support AES-GMAC Prashant Gupta
` (42 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: stable, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
When build_sec_fd fails at index loop inside the enqueue burst loops,
the previously built FD entries (indices 0..loop-1) were never freed.
The cleanup loop iterated in the wrong direction, starting at the
failed index and going up to frames_to_send, which are entries that
were never built. This caused silent FLE pool exhaustion, after which
every subsequent build_sec_fd returned -ENOMEM, enqueue_burst
returned 0 indefinitely, and the crypto-perf test hung.
Fix both dpaa2_sec_enqueue_burst and dpaa2_sec_enqueue_burst_ordered
by clamping frames_to_send to loop + 1 and iterating from 0 to
free all allocated FLE buffers including the failed entry.
Fixes: 623326dded3a ("crypto/dpaa2_sec: introduce poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
index 2a015a3d82..15152cc5a1 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
@@ -1550,6 +1550,9 @@ dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,
ret = build_sec_fd(*ops, &fd_arr[loop], bpid, dpaa2_qp);
if (ret) {
DPAA2_SEC_DP_DEBUG("FD build failed");
+ frames_to_send = loop + 1;
+ for (loop = 0; loop < frames_to_send; loop++)
+ free_fle(&fd_arr[loop], dpaa2_qp);
goto skip_tx;
}
ops++;
@@ -1909,6 +1912,9 @@ dpaa2_sec_enqueue_burst_ordered(void *qp, struct rte_crypto_op **ops,
ret = build_sec_fd(*ops, &fd_arr[loop], bpid, dpaa2_qp);
if (ret) {
DPAA2_SEC_DP_DEBUG("FD build failed");
+ frames_to_send = loop + 1;
+ for (loop = 0; loop < frames_to_send; loop++)
+ free_fle(&fd_arr[loop], dpaa2_qp);
goto skip_tx;
}
ops++;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 03/45] crypto/dpaa2_sec: support AES-GMAC
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
2026-09-03 13:53 ` [PATCH 01/45] crypto/dpaa2_sec: fix buffer overflow in GCM decrypt Prashant Gupta
2026-09-03 13:53 ` [PATCH 02/45] crypto/dpaa2_sec: fix FLE pool leak on sec FD build failure Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 04/45] crypto/dpaa2_sec: increase ivsize range for AES-CTR Prashant Gupta
` (41 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Akhil Goyal, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Add AES-GMAC as a supported AEAD algorithm for IPsec protocol
offload. AES-GMAC provides NULL encryption with GMAC authentication
and maps to OP_PCL_IPSEC_AES_NULL_WITH_GMAC in the SEC protocol
control word.
When AES_GMAC is specified as an AUTH (non-AEAD) algorithm, return
-ENOTSUP with an informative message directing the user to the AEAD
path.
Add RTE_CRYPTO_AEAD_AES_GMAC to the AEAD algorithm enum and expose
the capability in dpaa2_sec_capabilities.
Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 15 +++++++++-
drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 33 ++++++++++++++++++++-
lib/cryptodev/rte_crypto_sym.h | 2 ++
3 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
index 15152cc5a1..0e4e67aa07 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*
*/
@@ -2975,6 +2975,13 @@ dpaa2_sec_ipsec_aead_init(struct rte_crypto_aead_xform *aead_xform,
aeaddata->algmode = OP_ALG_AAI_CCM;
session->aead_alg = RTE_CRYPTO_AEAD_AES_CCM;
break;
+ case RTE_CRYPTO_AEAD_AES_GMAC:
+ /**
+ * AES-GMAC is an AEAD algo with NULL encryption and GMAC
+ * authentication.
+ */
+ aeaddata->algtype = OP_PCL_IPSEC_AES_NULL_WITH_GMAC;
+ break;
default:
DPAA2_SEC_ERR("Crypto: Undefined AEAD specified %u",
aead_xform->algo);
@@ -3046,6 +3053,10 @@ dpaa2_sec_ipsec_proto_init(struct rte_crypto_cipher_xform *cipher_xform,
authdata->algtype = OP_PCL_IPSEC_HMAC_MD5_96;
authdata->algmode = OP_ALG_AAI_HMAC;
break;
+ case RTE_CRYPTO_AUTH_AES_GMAC:
+ DPAA2_SEC_ERR(
+ "AES_GMAC is supported as AEAD algo for IPSEC proto only");
+ return -ENOTSUP;
case RTE_CRYPTO_AUTH_SHA224_HMAC:
authdata->algmode = OP_ALG_AAI_HMAC;
if (session->digest_length == 6)
@@ -3217,6 +3228,7 @@ dpaa2_sec_set_ipsec_session(struct rte_cryptodev *dev,
case OP_PCL_IPSEC_AES_GCM8:
case OP_PCL_IPSEC_AES_GCM12:
case OP_PCL_IPSEC_AES_GCM16:
+ case OP_PCL_IPSEC_AES_NULL_WITH_GMAC:
memcpy(encap_pdb.gcm.salt,
(uint8_t *)&(ipsec_xform->salt), 4);
break;
@@ -3357,6 +3369,7 @@ dpaa2_sec_set_ipsec_session(struct rte_cryptodev *dev,
case OP_PCL_IPSEC_AES_GCM8:
case OP_PCL_IPSEC_AES_GCM12:
case OP_PCL_IPSEC_AES_GCM16:
+ case OP_PCL_IPSEC_AES_NULL_WITH_GMAC:
memcpy(decap_pdb.gcm.salt,
(uint8_t *)&(ipsec_xform->salt), 4);
break;
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
index ff32f3d860..1824cc4a60 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016,2020-2024 NXP
+ * Copyright 2016,2020-2026 NXP
*
*/
@@ -762,6 +762,37 @@ static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = {
}, }
}, }
},
+ { /* AES GMAC (AEAD) */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+ {.aead = {
+ .algo = RTE_CRYPTO_AEAD_AES_GMAC,
+ .block_size = 16,
+ .key_size = {
+ .min = 16,
+ .max = 32,
+ .increment = 8
+ },
+ .digest_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ },
+ .aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 12,
+ .max = 16,
+ .increment = 4
+ }
+ }, }
+ }, }
+ },
+
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
diff --git a/lib/cryptodev/rte_crypto_sym.h b/lib/cryptodev/rte_crypto_sym.h
index 630fd153bd..f65db616a0 100644
--- a/lib/cryptodev/rte_crypto_sym.h
+++ b/lib/cryptodev/rte_crypto_sym.h
@@ -508,6 +508,8 @@ enum rte_crypto_aead_algorithm {
/**< AES algorithm in NCA5 mode */
RTE_CRYPTO_AEAD_ZUC_NCA6,
/**< ZUC-256 algorithm in NCA6 mode */
+ RTE_CRYPTO_AEAD_AES_GMAC,
+ /**< AES algorithm in GMAC mode. */
};
/** Symmetric AEAD Operations */
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 04/45] crypto/dpaa2_sec: increase ivsize range for AES-CTR
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (2 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 03/45] crypto/dpaa2_sec: support AES-GMAC Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 05/45] crypto/dpaa2_sec: add missing ECN capability Prashant Gupta
` (40 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Widen the AES-CTR IV size range from the fixed 16-byte value to
[12, 16] with a 4-byte increment. This allows 96-bit IVs (the
standard NIST SP 800-38A recommendation) in addition to 128-bit
IVs, aligning with common usage and test-vector expectations.
The change applies to both dpaa2_sec_capabilities and
dpaa2_pdcp_capabilities.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
index 1824cc4a60..7e18e83858 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
@@ -625,9 +625,9 @@ static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = {
.increment = 8
},
.iv_size = {
- .min = 16,
+ .min = 12,
.max = 16,
- .increment = 0
+ .increment = 4
},
}, }
}, }
@@ -855,9 +855,9 @@ static const struct rte_cryptodev_capabilities dpaa2_pdcp_capabilities[] = {
.increment = 8
},
.iv_size = {
- .min = 16,
+ .min = 12,
.max = 16,
- .increment = 0
+ .increment = 4
}
}, }
}, }
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 05/45] crypto/dpaa2_sec: add missing ECN capability
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (3 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 04/45] crypto/dpaa2_sec: increase ivsize range for AES-CTR Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 06/45] crypto/dpaa2_sec: add support for env variables Prashant Gupta
` (39 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Set the .ecn = 1 flag in both tunnel-mode security capability
entries to advertise that the driver supports ECN (Explicit
Congestion Notification) copying during IPsec encap/decap.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
index 7e18e83858..16b8273a79 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h
@@ -1004,6 +1004,7 @@ static const struct rte_security_capability dpaa2_sec_security_cap[] = {
.copy_df = 1,
.copy_dscp = 1,
.dec_ttl = 1,
+ .ecn = 1,
.esn = 1,
},
.replay_win_sz_max = 1024
@@ -1023,6 +1024,7 @@ static const struct rte_security_capability dpaa2_sec_security_cap[] = {
.copy_df = 1,
.copy_dscp = 1,
.dec_ttl = 1,
+ .ecn = 1,
.esn = 1,
},
.replay_win_sz_max = 1024
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 06/45] crypto/dpaa2_sec: add support for env variables
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (4 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 05/45] crypto/dpaa2_sec: add missing ECN capability Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 07/45] drivers: fix double free of dpaa2 device on uninit Prashant Gupta
` (38 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Allow driver configuration via environment variables as a fallback
when devargs are not provided. After processing devargs (or when
devargs are absent), check DRIVER_STRICT_ORDER and DRIVER_DUMP_MODE
environment variables to set en_loose_ordered and dpaa2_sec_dp_dump.
This lets users configure the driver without modifying EAL arguments,
useful in environments where command-line access is restricted.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 27 ++++++++++++++++++---
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
index 0e4e67aa07..c54960820c 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
@@ -4379,25 +4379,44 @@ check_devargs_handler(const char *key, const char *value,
static void
dpaa2_sec_get_devargs(struct rte_cryptodev *cryptodev, const char *key)
{
+ struct dpaa2_sec_dev_private *internals;
struct rte_kvargs *kvlist;
struct rte_devargs *devargs;
+ int ret;
+ char *env;
+
+ internals = cryptodev->data->dev_private;
devargs = cryptodev->device->devargs;
if (!devargs)
- return;
+ goto env_set;
kvlist = rte_kvargs_parse(devargs->args, NULL);
if (!kvlist)
- return;
+ goto env_set;
if (!rte_kvargs_count(kvlist, key)) {
rte_kvargs_free(kvlist);
- return;
+ goto env_set;
}
- rte_kvargs_process(kvlist, key,
+ ret = rte_kvargs_process(kvlist, key,
check_devargs_handler, (void *)cryptodev);
rte_kvargs_free(kvlist);
+ if (!ret)
+ return;
+
+env_set:
+ env = getenv(DRIVER_STRICT_ORDER);
+ if (env)
+ internals->en_loose_ordered = !atoi(env);
+
+ env = getenv(DRIVER_DUMP_MODE);
+ if (env) {
+ dpaa2_sec_dp_dump = atoi(env);
+ if (dpaa2_sec_dp_dump > DPAA2_SEC_DP_FULL_DUMP)
+ dpaa2_sec_dp_dump = DPAA2_SEC_DP_FULL_DUMP;
+ }
}
static int
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 07/45] drivers: fix double free of dpaa2 device on uninit
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (5 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 06/45] crypto/dpaa2_sec: add support for env variables Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 14:05 ` David Marchand
2026-09-03 13:53 ` [PATCH 08/45] net/dpaa2: fix integer overflow in CCSR region mapping Prashant Gupta
` (37 subsequent siblings)
44 siblings, 1 reply; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: stable, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
dpaa2_dpdmai_dev_uninit() was called from dpaa2_qdma_close(), but
rte_dma close may run without a matching device remove, so the
uninitialization (which frees the qdma_dev allocation and the MC portal)
could run twice or leak the MC portal.
Move dpaa2_dpdmai_dev_uninit() to dpaa2_qdma_remove(), which is the
correct teardown counterpart of dpaa2_qdma_probe(). To reach the dmadev
from the remove callback, store the rte_dma_dev pointer in
struct rte_dpaa2_device at probe time and use it in remove, dropping the
now-unused forward declaration.
Fixes: 8caf8427f85a ("dma/dpaa2: introduce driver skeleton")
Cc: stable@dpdk.org
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/bus/fslmc/bus_fslmc_driver.h | 1 +
drivers/dma/dpaa2/dpaa2_qdma.c | 9 ++++-----
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/bus/fslmc/bus_fslmc_driver.h b/drivers/bus/fslmc/bus_fslmc_driver.h
index 44f81cf662..c64ef094d0 100644
--- a/drivers/bus/fslmc/bus_fslmc_driver.h
+++ b/drivers/bus/fslmc/bus_fslmc_driver.h
@@ -102,6 +102,7 @@ struct rte_dpaa2_device {
char ep_name[RTE_DEV_NAME_MAX_LEN];
struct rte_intr_handle *intr_handle; /**< Interrupt handle */
char name[FSLMC_OBJECT_MAX_LEN]; /**< DPAA2 Object name*/
+ struct rte_dma_dev *dmadev; /**< DMA device */
};
typedef int (*rte_dpaa2_obj_create_t)(int vdev_fd,
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index f7d94bb799..4a95f2a1ab 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -1455,9 +1455,6 @@ dpaa2_qdma_stop(struct rte_dma_dev *dev)
return 0;
}
-static int
-dpaa2_dpdmai_dev_uninit(struct rte_dma_dev *dev);
-
static int
dpaa2_qdma_close(struct rte_dma_dev *dev)
{
@@ -1508,8 +1505,6 @@ dpaa2_qdma_close(struct rte_dma_dev *dev)
/* Reset QDMA device structure */
qdma_dev->num_vqs = 0;
- dpaa2_dpdmai_dev_uninit(dev);
-
return 0;
}
@@ -1708,6 +1703,7 @@ dpaa2_qdma_probe(struct rte_dpaa2_driver *dpaa2_drv,
return -EINVAL;
}
+ dpaa2_dev->dmadev = dmadev;
dmadev->dev_ops = &dpaa2_qdma_ops;
dmadev->device = &dpaa2_dev->device;
dmadev->fp_obj->dev_private = dmadev->data->dev_private;
@@ -1731,10 +1727,13 @@ dpaa2_qdma_probe(struct rte_dpaa2_driver *dpaa2_drv,
static int
dpaa2_qdma_remove(struct rte_dpaa2_device *dpaa2_dev)
{
+ struct rte_dma_dev *dmadev = dpaa2_dev->dmadev;
int ret;
DPAA2_QDMA_FUNC_TRACE();
+ dpaa2_dpdmai_dev_uninit(dmadev);
+
ret = rte_dma_pmd_release(dpaa2_dev->device.name);
if (ret)
DPAA2_QDMA_ERR("Device cleanup failed");
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH 07/45] drivers: fix double free of dpaa2 device on uninit
2026-09-03 13:53 ` [PATCH 07/45] drivers: fix double free of dpaa2 device on uninit Prashant Gupta
@ 2026-09-03 14:05 ` David Marchand
0 siblings, 0 replies; 47+ messages in thread
From: David Marchand @ 2026-09-03 14:05 UTC (permalink / raw)
To: Prashant Gupta; +Cc: stephen, dev, stable, Gagandeep Singh, Hemant Agrawal
On Thu, 3 Sept 2026 at 15:55, Prashant Gupta <prashant.gupta_3@nxp.com> wrote:
>
> From: Gagandeep Singh <g.singh@nxp.com>
>
> dpaa2_dpdmai_dev_uninit() was called from dpaa2_qdma_close(), but
> rte_dma close may run without a matching device remove, so the
> uninitialization (which frees the qdma_dev allocation and the MC portal)
> could run twice or leak the MC portal.
>
> Move dpaa2_dpdmai_dev_uninit() to dpaa2_qdma_remove(), which is the
> correct teardown counterpart of dpaa2_qdma_probe(). To reach the dmadev
> from the remove callback, store the rte_dma_dev pointer in
> struct rte_dpaa2_device at probe time and use it in remove, dropping the
> now-unused forward declaration.
>
> Fixes: 8caf8427f85a ("dma/dpaa2: introduce driver skeleton")
> Cc: stable@dpdk.org
>
> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Nack.
I just cleaned this up so maybe it is probably a regression of my change.
868a3abd0ee5 ("dma/dpaa2: remove dmadev pointer from bus device")
Thank you.
--
David Marchand
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH 08/45] net/dpaa2: fix integer overflow in CCSR region mapping
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (6 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 07/45] drivers: fix double free of dpaa2 device on uninit Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 09/45] dma/dpaa2: fix array-bounds warning in dequeue path Prashant Gupta
` (36 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: stable, Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
In lsx_ccsr_map_region, PAGE_SIZE is defined as sysconf(_SC_PAGESIZE)
which may be negative on error. The value was used directly in size_t
comparisons and passed to mmap, causing an integer overflow (Coverity
CID 49765682).
Use sysconf() explicitly with a signed long, check for errors, and
compute map_len separately with unsigned arithmetic before passing to
mmap.
Coverity issue: 49765682
Fixes: f023d059769f ("net/dpaa2: support recycle loopback port")
Cc: stable@dpdk.org
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_recycle.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_recycle.c b/drivers/net/dpaa2/dpaa2_recycle.c
index f78d12362e..80c20272ce 100644
--- a/drivers/net/dpaa2/dpaa2_recycle.c
+++ b/drivers/net/dpaa2/dpaa2_recycle.c
@@ -176,6 +176,8 @@ static void *lsx_ccsr_map_region(uint64_t addr, size_t len)
void *tmp;
uint64_t start;
uint64_t offset;
+ long page_size;
+ size_t map_len;
fd = open("/dev/mem", O_RDWR);
if (fd < 0) {
@@ -183,20 +185,27 @@ static void *lsx_ccsr_map_region(uint64_t addr, size_t len)
return NULL;
}
+ page_size = sysconf(_SC_PAGESIZE);
+ if (page_size <= 0) {
+ close(fd);
+ return NULL;
+ }
start = addr & PAGE_MASK;
offset = addr - start;
len = len & PAGE_MASK;
- if (len < (size_t)PAGE_SIZE)
- len = PAGE_SIZE;
- tmp = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, start);
+ map_len = len & ~(page_size - 1);
+ if (map_len < (size_t)page_size)
+ map_len = (size_t)page_size;
+
+ tmp = mmap(NULL, map_len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, start);
close(fd);
- if (tmp != MAP_FAILED)
- return (uint8_t *)tmp + offset;
- else
+ if (tmp == MAP_FAILED)
return NULL;
+
+ return (uint8_t *)tmp + offset;
}
static const uint8_t ls_sd1_prot_idx_map[] = {
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 09/45] dma/dpaa2: fix array-bounds warning in dequeue path
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (7 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 08/45] net/dpaa2: fix integer overflow in CCSR region mapping Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 10/45] bus/fslmc: defer bus initialization to probe Prashant Gupta
` (35 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: stable, Jun Yang
From: Jun Yang <jun.yang@nxp.com>
In dpaa2_qdma_dq_fd, passing a local uint16_t variable directly as a
pointer to qdma_cntx_idx_ring_eq caused a compiler array-bounds warning
because the function takes a pointer to an element of the ring array.
Add a per-queue scratch buffer idxs[DPAA2_QDMA_MAX_DESC] to
struct qdma_virt_queue and use idxs[0] instead of a local variable
pointer. Also consolidate the fle_sdd pointer assignment for LONG and
SG FD types to avoid code duplication.
Fixes: 388e888dc082 ("dma/dpaa2: support short FD")
Cc: stable@dpdk.org
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/dma/dpaa2/dpaa2_qdma.c | 26 +++++++++-----------------
drivers/dma/dpaa2/dpaa2_qdma.h | 1 +
2 files changed, 10 insertions(+), 17 deletions(-)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index 4a95f2a1ab..45d7a99805 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -935,35 +935,27 @@ dpaa2_qdma_dq_fd(const struct qbman_fd *fd,
enum dpaa2_qdma_fd_type type;
int ret;
struct qdma_cntx_sg *cntx_sg;
- struct qdma_cntx_fle_sdd *fle_sdd;
+ struct qdma_cntx_fle_sdd *fle_sdd = NULL;
att = dpaa2_qdma_fd_get_att(fd);
type = DPAA2_QDMA_FD_ATT_TYPE(att);
- if (type == DPAA2_QDMA_FD_SHORT) {
- idx = DPAA2_QDMA_FD_ATT_CNTX(att);
- ret = qdma_cntx_idx_ring_eq(qdma_vq->ring_cntx_idx,
- &idx, 1, free_space);
- if (unlikely(ret != 1))
- return -ENOSPC;
-
- return 0;
- }
- if (type == DPAA2_QDMA_FD_LONG) {
- idx = DPAA2_QDMA_FD_ATT_CNTX(att);
- fle_sdd = (void *)(uintptr_t)DPAA2_GET_FD_FLC(fd);
+ if (type == DPAA2_QDMA_FD_LONG || type == DPAA2_QDMA_FD_SG) {
+ fle_sdd = (void *)DPAA2_GET_FD_FLC(fd);
qdma_vq->fle_elem[*fle_elem_nb] = fle_sdd;
(*fle_elem_nb)++;
+ }
+ if (type == DPAA2_QDMA_FD_SHORT ||
+ type == DPAA2_QDMA_FD_LONG) {
+ idx = DPAA2_QDMA_FD_ATT_CNTX(att);
+ qdma_vq->idxs[0] = idx;
ret = qdma_cntx_idx_ring_eq(qdma_vq->ring_cntx_idx,
- &idx, 1, free_space);
+ qdma_vq->idxs, 1, free_space);
if (unlikely(ret != 1))
return -ENOSPC;
return 0;
}
if (type == DPAA2_QDMA_FD_SG) {
- fle_sdd = (void *)(uintptr_t)DPAA2_GET_FD_FLC(fd);
- qdma_vq->fle_elem[*fle_elem_nb] = fle_sdd;
- (*fle_elem_nb)++;
cntx_sg = container_of(fle_sdd,
struct qdma_cntx_sg, fle_sdd);
ret = qdma_cntx_idx_ring_eq(qdma_vq->ring_cntx_idx,
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.h b/drivers/dma/dpaa2/dpaa2_qdma.h
index e2d2eb4fa8..98c64f7b45 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.h
+++ b/drivers/dma/dpaa2/dpaa2_qdma.h
@@ -290,6 +290,7 @@ struct qdma_virt_queue {
struct qdma_cntx_idx_ring *ring_cntx_idx;
/**Used for silent enabled*/
+ uint16_t idxs[DPAA2_QDMA_MAX_DESC];
struct qdma_cntx_sg *cntx_sg[DPAA2_QDMA_MAX_DESC];
struct qdma_cntx_fle_sdd *cntx_fle_sdd[DPAA2_QDMA_MAX_DESC];
uint16_t silent_idx;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 10/45] bus/fslmc: defer bus initialization to probe
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (8 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 09/45] dma/dpaa2: fix array-bounds warning in dequeue path Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 11/45] dma/dpaa2: validate IOVA in pre-populate helpers Prashant Gupta
` (34 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev
The fslmc bus initialization (register the sequence number mbuf
dynfield, set up the VFIO group, DMA map the memory segments) needs the
DPDK heap and the memory segment list in shared (hugepage) memory for
multi-process sharing. But these are not available during the bus scan
(EAL runs rte_bus_scan() before memzone, memory and malloc heap init),
so it fails there.
Device discovery does not need it, and get_iommu_class() only checks
sysfs paths to report the IOVA mode, so the init is not needed in scan.
Move it back to probe.
Fixes: cdefd2e980bd ("drivers/bus: initialize NXP bus specifics in scan")
Signed-off-by: Prashant Gupta <prashant.gupta_3@nxp.com>
---
drivers/bus/fslmc/fslmc_bus.c | 92 +++++++++++++++++++----------------
1 file changed, 51 insertions(+), 41 deletions(-)
diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index 3626b12316..ee162b0ed8 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -368,46 +368,6 @@ rte_fslmc_scan(void)
/* If debugging is enabled, device list is dumped to log output */
dump_device_list();
- /* Bus initialization - only if devices were found */
- if (!TAILQ_EMPTY(&rte_fslmc_bus.device_list)) {
- static const struct rte_mbuf_dynfield dpaa2_seqn_dynfield_desc = {
- .name = DPAA2_SEQN_DYNFIELD_NAME,
- .size = sizeof(dpaa2_seqn_t),
- .align = alignof(dpaa2_seqn_t),
- };
-
- dpaa2_seqn_dynfield_offset =
- rte_mbuf_dynfield_register(&dpaa2_seqn_dynfield_desc);
- if (dpaa2_seqn_dynfield_offset < 0) {
- DPAA2_BUS_ERR("Failed to register mbuf field for dpaa sequence number");
- return 0;
- }
-
- ret = fslmc_vfio_setup_group();
- if (ret) {
- DPAA2_BUS_ERR("Unable to setup VFIO %d", ret);
- return 0;
- }
-
- /* Map existing segments as well as, in case of hotpluggable memory,
- * install callback handler.
- */
- if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
- ret = fslmc_vfio_dmamap();
- if (ret) {
- DPAA2_BUS_ERR("Unable to DMA map existing VAs: (%d)", ret);
- DPAA2_BUS_ERR("FSLMC VFIO Mapping failed");
- return 0;
- }
- }
-
- ret = fslmc_vfio_process_group();
- if (ret) {
- DPAA2_BUS_ERR("Unable to setup devices %d", ret);
- return 0;
- }
- }
-
process_once = 1;
return 0;
@@ -423,6 +383,56 @@ rte_fslmc_scan(void)
return 0;
}
+/* Bus initialization needs the DPDK heap and DMA mapping of the memory
+ * segments, which EAL only sets up after the bus scan, so it is done here.
+ */
+static int
+rte_fslmc_probe(struct rte_bus *bus)
+{
+ static const struct rte_mbuf_dynfield dpaa2_seqn_dynfield_desc = {
+ .name = DPAA2_SEQN_DYNFIELD_NAME,
+ .size = sizeof(dpaa2_seqn_t),
+ .align = alignof(dpaa2_seqn_t),
+ };
+ int ret;
+
+ if (TAILQ_EMPTY(&rte_fslmc_bus.device_list))
+ return 0;
+
+ dpaa2_seqn_dynfield_offset =
+ rte_mbuf_dynfield_register(&dpaa2_seqn_dynfield_desc);
+ if (dpaa2_seqn_dynfield_offset < 0) {
+ DPAA2_BUS_ERR("Failed to register mbuf field for dpaa sequence number");
+ return 0;
+ }
+
+ ret = fslmc_vfio_setup_group();
+ if (ret) {
+ DPAA2_BUS_ERR("Unable to setup VFIO %d", ret);
+ return 0;
+ }
+
+ /* Map existing segments as well as, in case of hotpluggable memory,
+ * install callback handler.
+ */
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = fslmc_vfio_dmamap();
+ if (ret) {
+ DPAA2_BUS_ERR("Unable to DMA map existing VAs: (%d)", ret);
+ DPAA2_BUS_ERR("FSLMC VFIO Mapping failed");
+ return 0;
+ }
+ }
+
+ ret = fslmc_vfio_process_group();
+ if (ret) {
+ DPAA2_BUS_ERR("Unable to setup devices %d", ret);
+ return 0;
+ }
+
+ return rte_bus_generic_probe(bus);
+}
+
static bool
fslmc_bus_match(const struct rte_driver *drv, const struct rte_device *dev)
{
@@ -544,7 +554,7 @@ fslmc_bus_unplug_device(struct rte_device *rte_dev)
struct rte_bus rte_fslmc_bus = {
.scan = rte_fslmc_scan,
- .probe = rte_bus_generic_probe,
+ .probe = rte_fslmc_probe,
.cleanup = rte_fslmc_close,
.parse = rte_fslmc_parse,
.dev_compare = fslmc_dev_compare,
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 11/45] dma/dpaa2: validate IOVA in pre-populate helpers
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (9 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 10/45] bus/fslmc: defer bus initialization to probe Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 12/45] dma/dpaa2: optimize context index ring enqueue Prashant Gupta
` (33 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
fle_sdd_pre_populate() and fle_sdd_sg_pre_populate() converted the SDD
and SG entry virtual addresses to IOVA with DPAA2_VADDR_TO_IOVA(), which
does not verify that the range is actually mapped in the IOMMU/SMMU. An
unmapped buffer was silently programmed into the hardware descriptor,
leading to an SMMU translation fault at transfer time that is hard to
trace back to the missing mapping.
Use DPAA2_VADDR_TO_IOVA_AND_CHECK() for the SDD, source SG and
destination SG buffers and report the offending address and size when
the translation is missing, so the misconfiguration is caught early and
clearly. Both helpers now return an error code which is propagated to
the caller instead of continuing with an invalid descriptor.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/dma/dpaa2/dpaa2_qdma.c | 50 ++++++++++++++++++++++++++++------
1 file changed, 41 insertions(+), 9 deletions(-)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index 45d7a99805..4ad72c5816 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -180,14 +180,23 @@ dpaa2_qdma_multi_eq(struct qdma_virt_queue *qdma_vq)
return num_tx;
}
-static void
+static int
fle_sdd_pre_populate(struct qdma_cntx_fle_sdd *fle_sdd,
struct dpaa2_qdma_rbp *rbp, uint64_t src, uint64_t dest,
uint32_t fmt)
{
struct qbman_fle *fle = fle_sdd->fle;
struct qdma_sdd *sdd = fle_sdd->sdd;
- uint64_t sdd_iova = DPAA2_VADDR_TO_IOVA(sdd);
+ uint64_t sdd_iova, iova_size;
+
+ iova_size = sizeof(struct qdma_sdd) * DPAA2_QDMA_MAX_SDD;
+ sdd_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(sdd, iova_size);
+ if (sdd_iova == RTE_BAD_IOVA) {
+ DPAA2_QDMA_ERR("No IOMMU map for sdd(%p)(size=%" PRIx64 ")",
+ sdd, iova_size);
+
+ return -ENOMEM;
+ }
/* first frame list to source descriptor */
DPAA2_SET_FLE_ADDR(&fle[DPAA2_QDMA_SDD_FLE], sdd_iova);
@@ -256,6 +265,8 @@ fle_sdd_pre_populate(struct qdma_cntx_fle_sdd *fle_sdd,
/* Final bit: 1, for last frame list */
DPAA2_SET_FLE_FIN(&fle[DPAA2_QDMA_DST_FLE]);
+
+ return 0;
}
static void
@@ -283,22 +294,39 @@ sg_entry_pre_populate(struct qdma_cntx_sg *sg_cntx)
}
}
-static void
+static int
fle_sdd_sg_pre_populate(struct qdma_cntx_sg *sg_cntx,
struct qdma_virt_queue *qdma_vq)
{
struct qdma_sg_entry *src_sge = sg_cntx->sg_src_entry;
struct qdma_sg_entry *dst_sge = sg_cntx->sg_dst_entry;
- rte_iova_t src_sge_iova, dst_sge_iova;
+ rte_iova_t src_sge_iova, dst_sge_iova, iova_size;
struct dpaa2_qdma_rbp *rbp = &qdma_vq->rbp;
memset(sg_cntx, 0, sizeof(struct qdma_cntx_sg));
- src_sge_iova = DPAA2_VADDR_TO_IOVA(src_sge);
- dst_sge_iova = DPAA2_VADDR_TO_IOVA(dst_sge);
+ iova_size = RTE_DPAAX_QDMA_JOB_SUBMIT_MAX *
+ sizeof(struct qdma_sg_entry);
+
+ src_sge_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(src_sge, iova_size);
+ if (src_sge_iova == RTE_BAD_IOVA) {
+ DPAA2_QDMA_ERR("No IOMMU map for src_sge(%p)(size=%" PRIx64 ")",
+ src_sge, iova_size);
+
+ return -ENOMEM;
+ }
+
+ dst_sge_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(dst_sge, iova_size);
+ if (dst_sge_iova == RTE_BAD_IOVA) {
+ DPAA2_QDMA_ERR("No IOMMU map for dst_sge(%p)(size=%" PRIx64 ")",
+ dst_sge, iova_size);
+
+ return -ENOMEM;
+ }
sg_entry_pre_populate(sg_cntx);
- fle_sdd_pre_populate(&sg_cntx->fle_sdd,
+
+ return fle_sdd_pre_populate(&sg_cntx->fle_sdd,
rbp, src_sge_iova, dst_sge_iova,
QBMAN_FLE_WORD4_FMT_SGE);
}
@@ -669,7 +697,9 @@ dpaa2_qdma_copy_sg(void *dev_private,
if (qdma_vq->fle_pre_populate) {
if (unlikely(!fle[DPAA2_QDMA_SRC_FLE].length)) {
- fle_sdd_sg_pre_populate(cntx_sg, qdma_vq);
+ ret = fle_sdd_sg_pre_populate(cntx_sg, qdma_vq);
+ if (ret)
+ return ret;
if (!qdma_dev->is_silent && cntx_sg && idx_addr) {
for (i = 0; i < nb_src; i++)
cntx_sg->cntx_idx[i] = idx_addr[i];
@@ -871,9 +901,11 @@ dpaa2_qdma_long_copy(struct qdma_virt_queue *qdma_vq,
if (qdma_vq->fle_pre_populate) {
if (unlikely(!fle[DPAA2_QDMA_SRC_FLE].length)) {
- fle_sdd_pre_populate(fle_sdd,
+ ret = fle_sdd_pre_populate(fle_sdd,
&qdma_vq->rbp,
0, 0, QBMAN_FLE_WORD4_FMT_SBF);
+ if (ret)
+ return ret;
}
fle_post_populate(fle, src, dst, length);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 12/45] dma/dpaa2: optimize context index ring enqueue
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (10 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 11/45] dma/dpaa2: validate IOVA in pre-populate helpers Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 13/45] drivers: add dpaa2 DMA bypass memory translation option Prashant Gupta
` (32 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
qdma_cntx_idx_ring_eq() copied the context indices into the ring one
element at a time in a loop, updating the tail on every iteration.
Replace the per-element loop with rte_memcpy(), handling the ring
wrap-around as at most two contiguous copies, and update the tail once.
This reduces the per-burst overhead on the dequeue completion path.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/dma/dpaa2/dpaa2_qdma.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index 4ad72c5816..1e9ec463aa 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -66,15 +66,23 @@ qdma_cntx_idx_ring_eq(struct qdma_cntx_idx_ring *ring,
const uint16_t *elem, uint16_t nb,
uint16_t *free_space)
{
- uint16_t i;
-
if (unlikely(nb > ring->free_space))
return 0;
- for (i = 0; i < nb; i++) {
- ring->cntx_idx_ring[ring->tail] = elem[i];
- ring->tail = (ring->tail + 1) &
- (DPAA2_QDMA_MAX_DESC - 1);
+ if ((ring->tail + nb) < DPAA2_QDMA_MAX_DESC) {
+ rte_memcpy(&ring->cntx_idx_ring[ring->tail],
+ elem, nb * sizeof(uint16_t));
+ ring->tail += nb;
+ } else {
+ rte_memcpy(&ring->cntx_idx_ring[ring->tail],
+ elem,
+ (DPAA2_QDMA_MAX_DESC - ring->tail) *
+ sizeof(uint16_t));
+ rte_memcpy(&ring->cntx_idx_ring[0],
+ &elem[DPAA2_QDMA_MAX_DESC - ring->tail],
+ (nb - DPAA2_QDMA_MAX_DESC + ring->tail) *
+ sizeof(uint16_t));
+ ring->tail = (ring->tail + nb) & (DPAA2_QDMA_MAX_DESC - 1);
}
ring->free_space -= nb;
ring->nb_in_ring += nb;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 13/45] drivers: add dpaa2 DMA bypass memory translation option
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (11 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 12/45] dma/dpaa2: optimize context index ring enqueue Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 14/45] mempool/dpaa2: support ops index from primary in secondary Prashant Gupta
` (31 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Add RTE_DPAAX_QDMA_BMT_FLAG flag to allow callers to bypass
memory translation (BMT) for DMA operations. This is needed for
some PCIe scenarios where IOMMU translation should be skipped.
The flag is propagated to all SG entry and FD populate helpers
via a new 'bmt' parameter, defaulting to DISABLE when the flag
is not set.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/common/dpaax/rte_pmd_dpaax_qdma.h | 4 +-
drivers/dma/dpaa2/dpaa2_qdma.c | 59 ++++++++++-------------
2 files changed, 29 insertions(+), 34 deletions(-)
diff --git a/drivers/common/dpaax/rte_pmd_dpaax_qdma.h b/drivers/common/dpaax/rte_pmd_dpaax_qdma.h
index 8f2fc83f8d..6bd72b0b0e 100644
--- a/drivers/common/dpaax/rte_pmd_dpaax_qdma.h
+++ b/drivers/common/dpaax/rte_pmd_dpaax_qdma.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2021-2024 NXP
+ * Copyright 2021-2024, 2026 NXP
*/
#ifndef RTE_PMD_DPAAX_QDMA_H
@@ -7,6 +7,8 @@
#include <rte_compat.h>
+#define RTE_DPAAX_QDMA_BMT_FLAG RTE_BIT64(7)
+
#define RTE_DPAAX_QDMA_COPY_IDX_OFFSET 8
#define RTE_DPAAX_QDMA_SG_IDX_ADDR_ALIGN \
RTE_BIT64(RTE_DPAAX_QDMA_COPY_IDX_OFFSET)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index 1e9ec463aa..e73b7dbb1e 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -278,7 +278,7 @@ fle_sdd_pre_populate(struct qdma_cntx_fle_sdd *fle_sdd,
}
static void
-sg_entry_pre_populate(struct qdma_cntx_sg *sg_cntx)
+sg_entry_pre_populate(struct qdma_cntx_sg *sg_cntx, int bmt)
{
uint16_t i;
struct qdma_sg_entry *src_sge = sg_cntx->sg_src_entry;
@@ -288,23 +288,17 @@ sg_entry_pre_populate(struct qdma_cntx_sg *sg_cntx)
/* source SG */
src_sge[i].ctrl.sl = QDMA_SG_SL_LONG;
src_sge[i].ctrl.fmt = QDMA_SG_FMT_SDB;
- /** IOMMU is always on for either VA or PA mode,
- * so Bypass Memory Translation should be disabled.
- */
- src_sge[i].ctrl.bmt = QDMA_SG_BMT_DISABLE;
+ src_sge[i].ctrl.bmt = bmt;
/* destination SG */
dst_sge[i].ctrl.sl = QDMA_SG_SL_LONG;
dst_sge[i].ctrl.fmt = QDMA_SG_FMT_SDB;
- /** IOMMU is always on for either VA or PA mode,
- * so Bypass Memory Translation should be disabled.
- */
- dst_sge[i].ctrl.bmt = QDMA_SG_BMT_DISABLE;
+ dst_sge[i].ctrl.bmt = bmt;
}
}
static int
fle_sdd_sg_pre_populate(struct qdma_cntx_sg *sg_cntx,
- struct qdma_virt_queue *qdma_vq)
+ struct qdma_virt_queue *qdma_vq, int bmt)
{
struct qdma_sg_entry *src_sge = sg_cntx->sg_src_entry;
struct qdma_sg_entry *dst_sge = sg_cntx->sg_dst_entry;
@@ -332,7 +326,7 @@ fle_sdd_sg_pre_populate(struct qdma_cntx_sg *sg_cntx,
return -ENOMEM;
}
- sg_entry_pre_populate(sg_cntx);
+ sg_entry_pre_populate(sg_cntx, bmt);
return fle_sdd_pre_populate(&sg_cntx->fle_sdd,
rbp, src_sge_iova, dst_sge_iova,
@@ -398,7 +392,7 @@ sg_fle_post_populate(struct qbman_fle fle[],
static inline uint32_t
sg_entry_populate(const struct rte_dma_sge *src,
const struct rte_dma_sge *dst, struct qdma_cntx_sg *sg_cntx,
- uint16_t nb_sge)
+ uint16_t nb_sge, int bmt)
{
uint16_t i;
uint32_t total_len = 0;
@@ -414,19 +408,13 @@ sg_entry_populate(const struct rte_dma_sge *src,
src_sge->data_len.data_len_sl0 = src[i].length;
src_sge->ctrl.sl = QDMA_SG_SL_LONG;
src_sge->ctrl.fmt = QDMA_SG_FMT_SDB;
- /** IOMMU is always on for either VA or PA mode,
- * so Bypass Memory Translation should be disabled.
- */
- src_sge->ctrl.bmt = QDMA_SG_BMT_DISABLE;
+ src_sge->ctrl.bmt = bmt;
dst_sge->addr_lo = (uint32_t)dst[i].addr;
dst_sge->addr_hi = (dst[i].addr >> 32);
dst_sge->data_len.data_len_sl0 = dst[i].length;
dst_sge->ctrl.sl = QDMA_SG_SL_LONG;
dst_sge->ctrl.fmt = QDMA_SG_FMT_SDB;
- /** IOMMU is always on for either VA or PA mode,
- * so Bypass Memory Translation should be disabled.
- */
- dst_sge->ctrl.bmt = QDMA_SG_BMT_DISABLE;
+ dst_sge->ctrl.bmt = bmt;
total_len += src[i].length;
if (i == (nb_sge - 1)) {
@@ -705,7 +693,9 @@ dpaa2_qdma_copy_sg(void *dev_private,
if (qdma_vq->fle_pre_populate) {
if (unlikely(!fle[DPAA2_QDMA_SRC_FLE].length)) {
- ret = fle_sdd_sg_pre_populate(cntx_sg, qdma_vq);
+ ret = fle_sdd_sg_pre_populate(cntx_sg, qdma_vq,
+ (flags & RTE_DPAAX_QDMA_BMT_FLAG) ?
+ QDMA_SG_BMT_ENABLE : QDMA_SG_BMT_DISABLE);
if (ret)
return ret;
if (!qdma_dev->is_silent && cntx_sg && idx_addr) {
@@ -726,7 +716,9 @@ dpaa2_qdma_copy_sg(void *dev_private,
offsetof(struct qdma_cntx_sg, sg_src_entry);
dst_sge_iova = cntx_iova +
offsetof(struct qdma_cntx_sg, sg_dst_entry);
- len = sg_entry_populate(src, dst, cntx_sg, nb_src);
+ len = sg_entry_populate(src, dst, cntx_sg, nb_src,
+ (flags & RTE_DPAAX_QDMA_BMT_FLAG) ?
+ QDMA_SG_BMT_ENABLE : QDMA_SG_BMT_DISABLE);
fle_populate(fle, sdd, sdd_iova,
&qdma_vq->rbp, src_sge_iova, dst_sge_iova, len,
@@ -759,14 +751,14 @@ dpaa2_qdma_copy_sg(void *dev_private,
static inline void
qdma_populate_fd_pci(uint64_t src, uint64_t dest,
uint32_t len, struct qbman_fd *fd,
- struct dpaa2_qdma_rbp *rbp, int ser)
+ struct dpaa2_qdma_rbp *rbp, int ser, int bmt)
{
fd->simple_pci.saddr_lo = lower_32_bits(src);
fd->simple_pci.saddr_hi = upper_32_bits(src);
fd->simple_pci.len_sl = len;
- fd->simple_pci.bmt = DPAA2_QDMA_BMT_DISABLE;
+ fd->simple_pci.bmt = bmt;
fd->simple_pci.fmt = DPAA2_QDMA_FD_SHORT_FORMAT;
fd->simple_pci.sl = 1;
fd->simple_pci.ser = ser;
@@ -802,14 +794,14 @@ qdma_populate_fd_pci(uint64_t src, uint64_t dest,
static inline void
qdma_populate_fd_ddr(uint64_t src, uint64_t dest,
- uint32_t len, struct qbman_fd *fd, int ser)
+ uint32_t len, struct qbman_fd *fd, int ser, int bmt)
{
fd->simple_ddr.saddr_lo = lower_32_bits(src);
fd->simple_ddr.saddr_hi = upper_32_bits(src);
fd->simple_ddr.len = len;
- fd->simple_ddr.bmt = DPAA2_QDMA_BMT_DISABLE;
+ fd->simple_ddr.bmt = bmt;
fd->simple_ddr.fmt = DPAA2_QDMA_FD_SHORT_FORMAT;
fd->simple_ddr.sl = 1;
fd->simple_ddr.ser = ser;
@@ -847,15 +839,16 @@ dpaa2_qdma_short_copy(struct qdma_virt_queue *qdma_vq,
if (qdma_vq->rbp.drbp || qdma_vq->rbp.srbp) {
/** PCIe EP*/
- qdma_populate_fd_pci(src,
- dst, length,
- fd, &qdma_vq->rbp,
- is_silent ? 0 : 1);
+ qdma_populate_fd_pci(src, dst, length,
+ fd, &qdma_vq->rbp, is_silent ? 0 : 1,
+ (flags & RTE_DPAAX_QDMA_BMT_FLAG) ?
+ DPAA2_QDMA_BMT_ENABLE : DPAA2_QDMA_BMT_DISABLE);
} else {
/** DDR or PCIe RC*/
- qdma_populate_fd_ddr(src,
- dst, length,
- fd, is_silent ? 0 : 1);
+ qdma_populate_fd_ddr(src, dst, length,
+ fd, is_silent ? 0 : 1,
+ (flags & RTE_DPAAX_QDMA_BMT_FLAG) ?
+ DPAA2_QDMA_BMT_ENABLE : DPAA2_QDMA_BMT_DISABLE);
}
dpaa2_qdma_fd_save_att(fd, DPAA2_QDMA_IDX_FROM_FLAG(flags),
DPAA2_QDMA_FD_SHORT);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 14/45] mempool/dpaa2: support ops index from primary in secondary
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (12 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 13/45] drivers: add dpaa2 DMA bypass memory translation option Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 15/45] net/dpaa2: set Tx confirmation on device init Prashant Gupta
` (30 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang, Hemant Agrawal
From: Jun Yang <jun.yang@nxp.com>
Secondary processes call rte_dpaa2_mpool_get_ops_idx() but the ops
index is registered only in the primary process. The secondary had no
way to learn the index, causing it to use the initial sentinel value
RTE_MEMPOOL_MAX_OPS_IDX and return an error.
Register an IPC action handler in the primary during pool creation.
Secondary processes send DPAA2_POOL_OPS_IDX_REQ over the
dpaa2_pool_mp_sync channel and receive the ops index back.
Also handle the case where rte_mp_action_register returns ENOTSUP
(e.g. no secondary process support enabled at build time) by
treating it as non-fatal, and improve the error message in
rte_hw_mbuf_free_pool to include the pool name.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/mempool/dpaa2/dpaa2_hw_mempool.c | 88 +++++++++++++++++++++++-
1 file changed, 85 insertions(+), 3 deletions(-)
diff --git a/drivers/mempool/dpaa2/dpaa2_hw_mempool.c b/drivers/mempool/dpaa2/dpaa2_hw_mempool.c
index ee001d8ce0..f19b6c039a 100644
--- a/drivers/mempool/dpaa2/dpaa2_hw_mempool.c
+++ b/drivers/mempool/dpaa2/dpaa2_hw_mempool.c
@@ -34,16 +34,90 @@
#include <dpaax_iova_table.h>
+#define DPAA2_POOL_MP_SYNC "dpaa2_pool_mp_sync"
+#define DPAA2_POOL_OPS_IDX_REQ 0x100
+#define DPAA2_POOL_OPS_IDX_RSP 0x101
+
+static int s_dpaa2_pool_mp_msg_setup;
+
+struct dpaa2_pool_mp_msg {
+ uint16_t msg_type;
+ uint8_t msg_data[];
+};
+
RTE_EXPORT_INTERNAL_SYMBOL(rte_dpaa2_bpid_info)
struct dpaa2_bp_info *rte_dpaa2_bpid_info;
static struct dpaa2_bp_list *h_bp_list;
static int16_t s_dpaa2_pool_ops_idx = RTE_MEMPOOL_MAX_OPS_IDX;
+static int
+dpaa2_mbuf_pool_mp_primary(const struct rte_mp_msg *msg,
+ const void *peer)
+{
+ struct rte_mp_msg reply;
+ const struct dpaa2_pool_mp_msg *req_msg = (const void *)msg->param;
+ struct dpaa2_pool_mp_msg *rsp_msg = (void *)reply.param;
+
+ memset(&reply, 0, sizeof(reply));
+
+ switch (req_msg->msg_type) {
+ case DPAA2_POOL_OPS_IDX_REQ:
+ rsp_msg->msg_type = DPAA2_POOL_OPS_IDX_RSP;
+ rte_memcpy(rsp_msg->msg_data, &s_dpaa2_pool_ops_idx,
+ sizeof(s_dpaa2_pool_ops_idx));
+ break;
+ default:
+ DPAA2_MEMPOOL_ERR("%s received invalid request(%d)",
+ __func__, req_msg->msg_type);
+ return -ENOTSUP;
+ }
+
+ strlcpy(reply.name, DPAA2_POOL_MP_SYNC, sizeof(reply.name));
+ return rte_mp_reply(&reply, peer);
+}
+
RTE_EXPORT_INTERNAL_SYMBOL(rte_dpaa2_mpool_get_ops_idx)
int rte_dpaa2_mpool_get_ops_idx(void)
{
- return s_dpaa2_pool_ops_idx;
+ struct rte_mp_msg mp_req;
+ struct rte_mp_reply mp_reply;
+ struct timespec ts = {.tv_sec = 5, .tv_nsec = 0};
+ int ret = 0;
+ struct dpaa2_pool_mp_msg *req_msg = (void *)mp_req.param;
+ struct dpaa2_pool_mp_msg *rsp_msg;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY ||
+ s_dpaa2_pool_ops_idx != RTE_MEMPOOL_MAX_OPS_IDX)
+ return s_dpaa2_pool_ops_idx;
+
+ strlcpy(mp_req.name, DPAA2_POOL_MP_SYNC, sizeof(mp_req.name));
+ req_msg->msg_type = DPAA2_POOL_OPS_IDX_REQ;
+ memset(&mp_reply, 0, sizeof(struct rte_mp_reply));
+ ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts);
+ if (ret) {
+ DPAA2_MEMPOOL_ERR("%s Failed to get response(%d)",
+ __func__, ret);
+ return ret;
+ }
+ if (!mp_reply.msgs) {
+ DPAA2_MEMPOOL_ERR("%s Failed to get response message",
+ __func__);
+ return -EINVAL;
+ }
+ rsp_msg = (void *)mp_reply.msgs;
+ if (rsp_msg->msg_type == DPAA2_POOL_OPS_IDX_RSP) {
+ rte_memcpy(&s_dpaa2_pool_ops_idx, rsp_msg->msg_data,
+ sizeof(s_dpaa2_pool_ops_idx));
+ ret = 0;
+ } else {
+ DPAA2_MEMPOOL_ERR("%s received invalid response(%d)",
+ __func__, rsp_msg->msg_type);
+ ret = -EINVAL;
+ }
+ free(mp_reply.msgs);
+
+ return ret;
}
static int
@@ -128,10 +202,18 @@ rte_hw_mbuf_create_pool(struct rte_mempool *mp)
if (s_dpaa2_pool_ops_idx == RTE_MEMPOOL_MAX_OPS_IDX) {
s_dpaa2_pool_ops_idx = mp->ops_index;
} else if (s_dpaa2_pool_ops_idx != mp->ops_index) {
- DPAA2_MEMPOOL_ERR("Only single ops index only");
+ DPAA2_MEMPOOL_ERR("Single ops index only");
ret = -EINVAL;
goto err4;
}
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY &&
+ !s_dpaa2_pool_mp_msg_setup) {
+ ret = rte_mp_action_register(DPAA2_POOL_MP_SYNC,
+ dpaa2_mbuf_pool_mp_primary);
+ if (ret && rte_errno != ENOTSUP)
+ return ret;
+ s_dpaa2_pool_mp_msg_setup = 1;
+ }
bp_list->next = h_bp_list;
bp_list->mp = mp;
@@ -172,7 +254,7 @@ rte_hw_mbuf_free_pool(struct rte_mempool *mp)
struct dpaa2_dpbp_dev *dpbp_node;
if (!mp->pool_data) {
- DPAA2_MEMPOOL_ERR("Not a valid dpaa2 buffer pool");
+ DPAA2_MEMPOOL_ERR("Not a valid dpaa2 buffer pool %s", mp->name);
return;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 15/45] net/dpaa2: set Tx confirmation on device init
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (13 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 14/45] mempool/dpaa2: support ops index from primary in secondary Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 16/45] drivers: optimize dpaa2 Tx queue and channel mapping Prashant Gupta
` (29 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Move TX confirmation mode setting from TXQ setup to device
initialization because TXQ0 is not always set up by the user.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 40 ++++++++++++--------------------
1 file changed, 15 insertions(+), 25 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index f211357919..c95704d4ca 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1153,7 +1153,6 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
struct dpni_queue tx_conf_cfg;
struct dpni_queue tx_flow_cfg;
uint8_t options = 0, flow_id;
- uint8_t ceetm_ch_idx;
uint16_t channel_id;
struct dpni_queue_id qid;
uint32_t tc_id;
@@ -1174,30 +1173,6 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
memset(&tx_conf_cfg, 0, sizeof(struct dpni_queue));
memset(&tx_flow_cfg, 0, sizeof(struct dpni_queue));
- if (!tx_queue_id) {
- for (ceetm_ch_idx = 0;
- ceetm_ch_idx <= (priv->num_channels - 1);
- ceetm_ch_idx++) {
- /*Set tx-conf and error configuration*/
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
- ret = dpni_set_tx_confirmation_mode(dpni,
- CMD_PRI_LOW, priv->token,
- ceetm_ch_idx,
- DPNI_CONF_AFFINE);
- } else {
- ret = dpni_set_tx_confirmation_mode(dpni,
- CMD_PRI_LOW, priv->token,
- ceetm_ch_idx,
- DPNI_CONF_DISABLE);
- }
- if (ret) {
- DPAA2_PMD_ERR("Error(%d) in tx conf setting",
- ret);
- return ret;
- }
- }
- }
-
tc_id = tx_queue_id % priv->num_tx_tc;
channel_id = (uint8_t)(tx_queue_id / priv->num_tx_tc) % priv->num_channels;
flow_id = 0;
@@ -3624,6 +3599,21 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
goto init_err;
}
+ for (i = 0; i < priv->num_channels; i++) {
+ /*Set tx-conf and error configuration*/
+ if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ ret = dpni_set_tx_confirmation_mode(dpni_dev,
+ CMD_PRI_LOW, priv->token, i, DPNI_CONF_AFFINE);
+ } else {
+ ret = dpni_set_tx_confirmation_mode(dpni_dev,
+ CMD_PRI_LOW, priv->token, i, DPNI_CONF_DISABLE);
+ }
+ if (ret) {
+ DPAA2_PMD_ERR("Error(%d) in tx conf setting", ret);
+ goto init_err;
+ }
+ }
+
/* Allocate memory for storing MAC addresses.
* Table of mac_filter_entries size is allocated so that RTE ether lib
* can add MAC entries when rte_eth_dev_mac_addr_add is called.
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 16/45] drivers: optimize dpaa2 Tx queue and channel mapping
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (14 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 15/45] net/dpaa2: set Tx confirmation on device init Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 17/45] net/dpaa2: support larger burst size Prashant Gupta
` (28 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Allocate TX queues based on dpni attribute and align the
bounds check with the RX queue logic:
- compute nb_tx_queues from num_tx_tcs * num_queues (respecting
DPNI_OPT_SINGLE_SENDER) and cap at MAX_TX_QUEUES (128)
- cap nb_rx_queues at MAX_RX_QUEUES (128)
- cap num_channels at DPAA2_MAX_CHANNELS
- assign tc_index/flow_id at queue-allocation time instead of
at queue-setup time
- replace the open-coded (channel << 8 | tc) with
DPNI_BUILD_PARAM(channel, tc) throughout tx_queue_setup
- initialize fqid to DPAA2_INVALID_FQ_ID and use it as the
already-configured guard (replacing DPAA2_INVALID_FLOW_ID)
- add a per-device tx_channels[] array for channel mapping
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 1 +
drivers/net/dpaa2/dpaa2_ethdev.c | 112 ++++++++++++++----------
drivers/net/dpaa2/dpaa2_ethdev.h | 5 +-
3 files changed, 70 insertions(+), 48 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index ef65e7895a..f0bc9a3063 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -82,6 +82,7 @@
#define DPAA2_DPCI_MAX_QUEUES 2
#define DPAA2_INVALID_FLOW_ID 0xffff
+#define DPAA2_INVALID_FQ_ID ((uint32_t)(-1))
#define DPAA2_INVALID_CGID 0xff
#define SEC_FLC_DHR_OUTBOUND (-114)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index c95704d4ca..d79dfa7114 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -519,17 +519,13 @@ static int
dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- uint16_t dist_idx;
- uint32_t vq_id;
- uint8_t num_rxqueue_per_tc;
- struct dpaa2_queue *mc_q, *mcq;
+ uint8_t num_queue_per_tc;
+ struct dpaa2_queue *mc_q, *dpaa2_q;
uint32_t tot_queues;
int i, ret = 0;
- struct dpaa2_queue *dpaa2_q;
PMD_INIT_FUNC_TRACE();
- num_rxqueue_per_tc = (priv->nb_rx_queues / priv->num_rx_tc);
if (priv->flags & DPAA2_TX_CONF_ENABLE)
tot_queues = priv->nb_rx_queues + 2 * priv->nb_tx_queues;
else
@@ -541,8 +537,12 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
return -ENOBUFS;
}
+ num_queue_per_tc = (priv->nb_rx_queues / priv->num_rx_tc);
for (i = 0; i < priv->nb_rx_queues; i++) {
mc_q->eth_data = dev->data;
+ mc_q->tc_index = i / num_queue_per_tc;
+ mc_q->flow_id = i % num_queue_per_tc;
+ mc_q->fqid = DPAA2_INVALID_FQ_ID;
priv->rx_vq[i] = mc_q++;
dpaa2_q = priv->rx_vq[i];
ret = dpaa2_queue_storage_alloc(dpaa2_q,
@@ -566,9 +566,12 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
goto fail;
}
+ num_queue_per_tc = (priv->nb_tx_queues / priv->num_tx_tc);
for (i = 0; i < priv->nb_tx_queues; i++) {
mc_q->eth_data = dev->data;
- mc_q->flow_id = DPAA2_INVALID_FLOW_ID;
+ mc_q->tc_index = i / num_queue_per_tc;
+ mc_q->flow_id = i % num_queue_per_tc;
+ mc_q->fqid = DPAA2_INVALID_FQ_ID;
priv->tx_vq[i] = mc_q++;
dpaa2_q = (struct dpaa2_queue *)priv->tx_vq[i];
dpaa2_q->cscn = rte_malloc(NULL,
@@ -583,8 +586,9 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
/*Setup tx confirmation queues*/
for (i = 0; i < priv->nb_tx_queues; i++) {
mc_q->eth_data = dev->data;
- mc_q->tc_index = i;
- mc_q->flow_id = 0;
+ mc_q->tc_index = i / num_queue_per_tc;
+ mc_q->flow_id = i % num_queue_per_tc;
+ mc_q->fqid = DPAA2_INVALID_FQ_ID;
priv->tx_conf_vq[i] = mc_q++;
dpaa2_q = priv->tx_conf_vq[i];
ret = dpaa2_queue_storage_alloc(dpaa2_q,
@@ -594,14 +598,6 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
}
}
- vq_id = 0;
- for (dist_idx = 0; dist_idx < priv->nb_rx_queues; dist_idx++) {
- mcq = priv->rx_vq[vq_id];
- mcq->tc_index = dist_idx / num_rxqueue_per_tc;
- mcq->flow_id = dist_idx % num_rxqueue_per_tc;
- vq_id++;
- }
-
return 0;
fail_tx_conf:
i -= 1;
@@ -977,6 +973,12 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
return ret;
}
dpaa2_q = priv->rx_vq[rx_queue_id];
+ if (dpaa2_q->fqid != DPAA2_INVALID_FQ_ID) {
+ DPAA2_PMD_WARN("%s: RXQ[%d] has been setup",
+ dev->data->name, rx_queue_id);
+ dev->data->rx_queues[rx_queue_id] = dpaa2_q;
+ return 0;
+ }
dpaa2_q->mb_pool = mb_pool; /**< mbuf pool to populate RX ring. */
dpaa2_q->bp_array = rte_dpaa2_bpid_info;
dpaa2_q->offloads = rx_conf->offloads;
@@ -1165,7 +1167,9 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
dpaa2_q->offloads = tx_conf->offloads;
/* Return if queue already configured */
- if (dpaa2_q->flow_id != DPAA2_INVALID_FLOW_ID) {
+ if (dpaa2_q->fqid != DPAA2_INVALID_FQ_ID) {
+ DPAA2_PMD_WARN("%s: TXQ[%d] has been setup",
+ dev->data->name, tx_queue_id);
dev->data->tx_queues[tx_queue_id] = dpaa2_q;
return 0;
}
@@ -1173,26 +1177,24 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
memset(&tx_conf_cfg, 0, sizeof(struct dpni_queue));
memset(&tx_flow_cfg, 0, sizeof(struct dpni_queue));
- tc_id = tx_queue_id % priv->num_tx_tc;
- channel_id = (uint8_t)(tx_queue_id / priv->num_tx_tc) % priv->num_channels;
- flow_id = 0;
+ tc_id = dpaa2_q->tc_index;
+ flow_id = dpaa2_q->flow_id;
+ if (tc_id < priv->num_channels)
+ channel_id = priv->tx_channels[tc_id];
+ else
+ channel_id = priv->tx_channels[priv->num_channels - 1];
ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token, DPNI_QUEUE_TX,
- ((channel_id << 8) | tc_id), flow_id, options, &tx_flow_cfg);
+ DPNI_BUILD_PARAM(channel_id, tc_id), flow_id, options, &tx_flow_cfg);
if (ret) {
- DPAA2_PMD_ERR("Error in setting the tx flow: "
- "tc_id=%d, flow=%d err=%d",
- tc_id, flow_id, ret);
- return ret;
+ DPAA2_PMD_ERR("Failed(%d) to set %s's TC[%d].txq[%d]",
+ ret, dev->data->name, tc_id, flow_id);
+ return ret;
}
- dpaa2_q->flow_id = flow_id;
-
- dpaa2_q->tc_index = tc_id;
-
ret = dpni_get_queue(dpni, CMD_PRI_LOW, priv->token,
- DPNI_QUEUE_TX, ((channel_id << 8) | dpaa2_q->tc_index),
- dpaa2_q->flow_id, &tx_flow_cfg, &qid);
+ DPNI_QUEUE_TX, DPNI_BUILD_PARAM(channel_id, tc_id),
+ flow_id, &tx_flow_cfg, &qid);
if (ret) {
DPAA2_PMD_ERR("Error in getting LFQID err=%d", ret);
return ret;
@@ -1231,10 +1233,9 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
ret = dpni_set_congestion_notification(dpni,
CMD_PRI_LOW, priv->token, DPNI_QUEUE_TX,
- ((channel_id << 8) | tc_id), &cong_notif_cfg);
+ DPNI_BUILD_PARAM(channel_id, tc_id), &cong_notif_cfg);
if (ret) {
- DPAA2_PMD_ERR("Set TX congestion notification err=%d",
- ret);
+ DPAA2_PMD_ERR("Set TX congestion notification err=%d", ret);
return ret;
}
}
@@ -1242,31 +1243,32 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
dev->data->tx_queues[tx_queue_id] = dpaa2_q;
if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ tc_id = dpaa2_tx_conf_q->tc_index;
+ flow_id = dpaa2_tx_conf_q->flow_id;
dpaa2_q->tx_conf_queue = dpaa2_tx_conf_q;
options = options | DPNI_QUEUE_OPT_USER_CTX;
tx_conf_cfg.user_context = (size_t)(dpaa2_q);
ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token,
DPNI_QUEUE_TX_CONFIRM,
- ((channel_id << 8) | dpaa2_tx_conf_q->tc_index),
- dpaa2_tx_conf_q->flow_id,
- options, &tx_conf_cfg);
+ DPNI_BUILD_PARAM(channel_id, tc_id),
+ flow_id, options, &tx_conf_cfg);
if (ret) {
DPAA2_PMD_ERR("Set TC[%d].TX[%d] conf flow err=%d",
- dpaa2_tx_conf_q->tc_index,
- dpaa2_tx_conf_q->flow_id, ret);
+ tc_id, flow_id, ret);
return ret;
}
ret = dpni_get_queue(dpni, CMD_PRI_LOW, priv->token,
DPNI_QUEUE_TX_CONFIRM,
- ((channel_id << 8) | dpaa2_tx_conf_q->tc_index),
- dpaa2_tx_conf_q->flow_id, &tx_conf_cfg, &qid);
+ DPNI_BUILD_PARAM(channel_id, tc_id),
+ flow_id, &tx_conf_cfg, &qid);
if (ret) {
DPAA2_PMD_ERR("Error in getting LFQID err=%d", ret);
return ret;
}
dpaa2_tx_conf_q->fqid = qid.fqid;
}
+
return 0;
}
@@ -3553,10 +3555,28 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
for (i = 0; i < priv->max_cgs; i++)
priv->cgid_in_use[i] = 0;
- for (i = 0; i < attr.num_rx_tcs; i++)
- priv->nb_rx_queues += attr.num_queues;
-
- priv->nb_tx_queues = attr.num_tx_tcs * attr.num_channels;
+ priv->nb_rx_queues = attr.num_rx_tcs * attr.num_queues;
+ if (priv->nb_rx_queues > MAX_RX_QUEUES) {
+ DPAA2_PMD_WARN("Too many RXQs(%d) > %d, reduce it to %d",
+ priv->nb_rx_queues, MAX_RX_QUEUES, MAX_RX_QUEUES);
+ priv->nb_rx_queues = MAX_RX_QUEUES;
+ }
+ if (attr.options & DPNI_OPT_SINGLE_SENDER)
+ priv->nb_tx_queues = attr.num_tx_tcs * 1;
+ else
+ priv->nb_tx_queues = attr.num_tx_tcs * attr.num_queues;
+ if (priv->nb_tx_queues > MAX_TX_QUEUES) {
+ DPAA2_PMD_WARN("Too many TXQs(%d) > %d, reduce it to %d",
+ priv->nb_tx_queues, MAX_TX_QUEUES, MAX_TX_QUEUES);
+ priv->nb_tx_queues = MAX_TX_QUEUES;
+ }
+ if (priv->num_channels > DPAA2_MAX_CHANNELS) {
+ DPAA2_PMD_WARN("Too many TX channels(%d) > %d, reduce it to %d",
+ priv->num_channels, DPAA2_MAX_CHANNELS, DPAA2_MAX_CHANNELS);
+ priv->num_channels = DPAA2_MAX_CHANNELS;
+ }
+ for (i = 0; i < priv->num_channels; i++)
+ priv->tx_channels[i] = i;
DPAA2_PMD_DEBUG("RX-TC= %d, rx_queues= %d, tx_queues=%d, max_cgs=%d",
priv->num_rx_tc, priv->nb_rx_queues,
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 149954fb6b..0366315ec7 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -28,7 +28,7 @@
#define MAX_TCS DPNI_MAX_TC
#define MAX_RX_QUEUES 128
-#define MAX_TX_QUEUES 16
+#define MAX_TX_QUEUES 128
#define MAX_DPNI 8
#define DPAA2_MAX_CHANNELS 16
@@ -390,13 +390,14 @@ struct dpaa2_dev_priv {
int32_t hw_id;
int32_t qdid;
uint16_t token;
+ uint16_t tx_channels[DPAA2_MAX_CHANNELS];
uint8_t nb_tx_queues;
uint8_t nb_rx_queues;
uint32_t options;
void *rx_vq[MAX_RX_QUEUES];
void *tx_vq[MAX_TX_QUEUES];
struct dpaa2_bp_list *bp_list; /**<Attached buffer pool list */
- void *tx_conf_vq[MAX_TX_QUEUES * DPAA2_MAX_CHANNELS];
+ void *tx_conf_vq[MAX_TX_QUEUES];
void *rx_err_vq;
uint32_t flags; /*dpaa2 config flags */
uint8_t max_mac_filters;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 17/45] net/dpaa2: support larger burst size
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (15 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 16/45] drivers: optimize dpaa2 Tx queue and channel mapping Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 18/45] net/dpaa2: support MPLS and PPPoE flow distribution Prashant Gupta
` (27 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Make burst size checks and MC command fields SoC-aware,
enabling bursts larger than 64K on LX2160A with dpni version 8.7+.
The max_burst_size field in dpni_tx_shaping_cfg is widened to uint32_t
and the MC command struct is updated to split the value across two
16-bit fields using the new DPNI_BURST_LO/HI macros.
Signed-off-by: Prashant Gupta <prashant.gupta_3@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_tm.c | 16 +++++++++++-----
drivers/net/dpaa2/mc/dpni.c | 11 +++++++++--
drivers/net/dpaa2/mc/fsl_dpni.h | 8 ++++++--
drivers/net/dpaa2/mc/fsl_dpni_cmd.h | 3 ++-
4 files changed, 28 insertions(+), 10 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_tm.c b/drivers/net/dpaa2/dpaa2_tm.c
index 36e815c356..0e1b2c5b54 100644
--- a/drivers/net/dpaa2/dpaa2_tm.c
+++ b/drivers/net/dpaa2/dpaa2_tm.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2020-2024 NXP
+ * Copyright 2020-2026 NXP
*/
#include <rte_ethdev.h>
@@ -10,14 +10,20 @@
#include "dpaa2_pmd_logs.h"
#include <dpaa2_hw_dpio.h>
-#define DPAA2_BURST_MAX (64 * 1024)
-
#define DPAA2_SHAPER_MIN_RATE 0
#define DPAA2_SHAPER_MAX_RATE 107374182400ull
#define DPAA2_WEIGHT_MAX 24701
#define DPAA2_PKT_ADJUST_LEN_MIN 0
#define DPAA2_PKT_ADJUST_LEN_MAX 0x7ff
+static inline uint32_t dpaa2_get_burst_max(struct dpaa2_dev_priv *priv)
+{
+ if (priv->dpni_ver_major == 8 && priv->dpni_ver_minor >= 7)
+ return (dpaa2_svr_family == SVR_LX2160A) ? 229375 : (64 * 1024);
+
+ return (64 * 1024);
+}
+
int
dpaa2_tm_init(struct rte_eth_dev *dev)
{
@@ -283,7 +289,7 @@ dpaa2_shaper_profile_add(struct rte_eth_dev *dev, uint32_t shaper_profile_id,
RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PEAK_RATE,
NULL, "committed rate is out of range\n");
- if (params->committed.size > DPAA2_BURST_MAX)
+ if (params->committed.size > dpaa2_get_burst_max(priv))
return -rte_tm_error_set(error, EINVAL,
RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PEAK_SIZE,
NULL, "committed size is out of range\n");
@@ -293,7 +299,7 @@ dpaa2_shaper_profile_add(struct rte_eth_dev *dev, uint32_t shaper_profile_id,
RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PEAK_RATE,
NULL, "Peak rate is out of range\n");
- if (params->peak.size > DPAA2_BURST_MAX)
+ if (params->peak.size > dpaa2_get_burst_max(priv))
return -rte_tm_error_set(error, EINVAL,
RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PEAK_SIZE,
NULL, "Peak size is out of range\n");
diff --git a/drivers/net/dpaa2/mc/dpni.c b/drivers/net/dpaa2/mc/dpni.c
index d0a9fefd31..17275b7195 100644
--- a/drivers/net/dpaa2/mc/dpni.c
+++ b/drivers/net/dpaa2/mc/dpni.c
@@ -1106,6 +1106,7 @@ int dpni_set_tx_shaping(struct fsl_mc_io *mc_io,
{
struct dpni_cmd_set_tx_shaping *cmd_params;
struct mc_command cmd = { 0 };
+ uint32_t tx_cr_burst, tx_er_burst;
int coupled, lni_shaper;
uint8_t channel_id;
uint16_t oal;
@@ -1115,10 +1116,16 @@ int dpni_set_tx_shaping(struct fsl_mc_io *mc_io,
cmd_flags,
token);
cmd_params = (struct dpni_cmd_set_tx_shaping *)cmd.params;
+ tx_cr_burst = tx_cr_shaper->max_burst_size;
+ tx_er_burst = tx_er_shaper->max_burst_size;
cmd_params->tx_cr_max_burst_size =
- cpu_to_le16(tx_cr_shaper->max_burst_size);
+ cpu_to_le16(DPNI_BURST_LO(tx_cr_burst));
cmd_params->tx_er_max_burst_size =
- cpu_to_le16(tx_er_shaper->max_burst_size);
+ cpu_to_le16(DPNI_BURST_LO(tx_er_burst));
+ cmd_params->tx_cr_max_burst_size_hi =
+ cpu_to_le16(DPNI_BURST_HI(tx_cr_burst));
+ cmd_params->tx_er_max_burst_size_hi =
+ cpu_to_le16(DPNI_BURST_HI(tx_er_burst));
cmd_params->tx_cr_rate_limit =
cpu_to_le32(tx_cr_shaper->rate_limit);
cmd_params->tx_er_rate_limit =
diff --git a/drivers/net/dpaa2/mc/fsl_dpni.h b/drivers/net/dpaa2/mc/fsl_dpni.h
index 42d633eaf8..8913b408e7 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni.h
@@ -830,14 +830,18 @@ int dpni_get_link_state(struct fsl_mc_io *mc_io,
uint16_t token,
struct dpni_link_state *state);
+#define DPNI_BURST_LO(burst) ((burst) & GENMASK(15, 0))
+#define DPNI_BURST_HI(burst) ((burst) >> 16)
+
/**
* struct dpni_tx_shaping - Structure representing DPNI tx shaping configuration
* @rate_limit: Rate in Mbits/s
- * @max_burst_size: Burst size in bytes (up to 64KB)
+ * @max_burst_size: Burst size in bytes. Limits depend on the SoC (0x37FFF
+ * for LX2160A, 0xF7FF for all others)
*/
struct dpni_tx_shaping_cfg {
uint32_t rate_limit;
- uint16_t max_burst_size;
+ uint32_t max_burst_size;
};
/**
diff --git a/drivers/net/dpaa2/mc/fsl_dpni_cmd.h b/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
index bb4939031b..6b396ca7cc 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
@@ -394,7 +394,8 @@ struct dpni_rsp_get_link_state {
struct dpni_cmd_set_tx_shaping {
uint16_t tx_cr_max_burst_size;
uint16_t tx_er_max_burst_size;
- uint32_t pad;
+ uint16_t tx_cr_max_burst_size_hi;
+ uint16_t tx_er_max_burst_size_hi;
uint32_t tx_cr_rate_limit;
uint32_t tx_er_rate_limit;
/* from LSB: coupled:1, lni_shaper: 1*/
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 18/45] net/dpaa2: support MPLS and PPPoE flow distribution
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (16 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 17/45] net/dpaa2: support larger burst size Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 19/45] net/dpaa2: support meter and policing Prashant Gupta
` (26 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Add rte_flow pattern support for MPLS and PPPoE items in the dpaa2
PMD flow engine.
For a pattern with no spec/mask, the frame is identified by its Frame
Attribute Flag (FAF_MPLS_FRAM / FAF_PPPOE_PPP_FRAM) for both the QoS
and flow steering tables. When a spec is present, the MPLS label
(NH_FLD_MPLS_MPLSL_1) or the PPPoE session id (NH_FLD_PPPOE_SID) is
added as a header extract rule for both tables.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_flow.c | 196 +++++++++++++++++++++++++++++++++
1 file changed, 196 insertions(+)
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 2e44bff766..90e0b66258 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -80,6 +80,8 @@ enum rte_flow_item_type dpaa2_hp_supported_pattern_type[] = {
RTE_FLOW_ITEM_TYPE_GTP,
RTE_FLOW_ITEM_TYPE_ESP,
RTE_FLOW_ITEM_TYPE_AH,
+ RTE_FLOW_ITEM_TYPE_MPLS,
+ RTE_FLOW_ITEM_TYPE_PPPOES,
RTE_FLOW_ITEM_TYPE_RAW
};
@@ -180,8 +182,17 @@ static const struct rte_flow_item_gtp dpaa2_flow_item_gtp_mask = {
.teid = RTE_BE32(0xffffffff),
};
+static const struct rte_flow_item_mpls dpaa2_flow_item_mpls_mask = {
+ .label_tc_s = { 0xff, 0xff, 0xff },
+};
+
+static const struct rte_flow_item_pppoe dpaa2_flow_item_pppoe_mask = {
+ .session_id = RTE_BE16(0xffff),
+};
+
#endif
+
#define DPAA2_FLOW_DUMP printf
static inline void
@@ -1764,6 +1775,14 @@ dpaa2_flow_extract_support(const uint8_t *mask_src,
mask_support = (const char *)&dpaa2_flow_item_gtp_mask;
size = sizeof(struct rte_flow_item_gtp);
break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ mask_support = (const char *)&dpaa2_flow_item_mpls_mask;
+ size = sizeof(struct rte_flow_item_mpls);
+ break;
+ case RTE_FLOW_ITEM_TYPE_PPPOES:
+ mask_support = (const char *)&dpaa2_flow_item_pppoe_mask;
+ size = sizeof(struct rte_flow_item_pppoe);
+ break;
default:
return -EINVAL;
}
@@ -4041,6 +4060,163 @@ dpaa2_configure_flow_gtp(struct dpaa2_dev_flow *flow,
return 0;
}
+static int
+dpaa2_configure_flow_mpls(struct dpaa2_dev_flow *flow,
+ struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_dpaa2_flow_item *dpaa2_pattern,
+ const struct rte_flow_action actions[] __rte_unused,
+ struct rte_flow_error *error __rte_unused,
+ int *device_configured)
+{
+ int ret, local_cfg = 0;
+ uint32_t group;
+ const struct rte_flow_item_mpls *spec, *mask;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ const struct rte_flow_item *pattern =
+ &dpaa2_pattern->generic_item;
+
+ group = attr->group;
+
+ /* Parse pattern list to get the matching parameters */
+ spec = pattern->spec;
+ mask = pattern->mask ?
+ pattern->mask : &dpaa2_flow_item_mpls_mask;
+
+ /* Get traffic class index and flow id to be configured */
+ flow->tc_id = group;
+ flow->tc_index = attr->priority;
+
+ if (dpaa2_pattern->in_tunnel) {
+ DPAA2_PMD_ERR("Tunnel-MPLS distribution not support");
+ return -ENOTSUP;
+ }
+
+ if (!spec) {
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ FAF_MPLS_FRAM, DPAA2_FLOW_QOS_TYPE,
+ group, &local_cfg);
+ if (ret)
+ return ret;
+
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ FAF_MPLS_FRAM, DPAA2_FLOW_FS_TYPE,
+ group, &local_cfg);
+ if (ret)
+ return ret;
+
+ (*device_configured) |= local_cfg;
+ return 0;
+ }
+
+ ret = dpaa2_flow_extract_support((const uint8_t *)mask,
+ RTE_FLOW_ITEM_TYPE_MPLS);
+ if (ret) {
+ DPAA2_PMD_WARN("Extract field(s) of MPLS not support.");
+ return ret;
+ }
+
+ if (!mask->label_tc_s[0] && !mask->label_tc_s[1] &&
+ !mask->label_tc_s[2])
+ return 0;
+
+ ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_MPLS,
+ NH_FLD_MPLS_MPLSL_1, spec->label_tc_s,
+ mask->label_tc_s, sizeof(mask->label_tc_s),
+ priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
+ if (ret)
+ return ret;
+
+ ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_MPLS,
+ NH_FLD_MPLS_MPLSL_1, spec->label_tc_s,
+ mask->label_tc_s, sizeof(mask->label_tc_s),
+ priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ if (ret)
+ return ret;
+
+ (*device_configured) |= local_cfg;
+
+ return 0;
+}
+
+static int
+dpaa2_configure_flow_pppoe(struct dpaa2_dev_flow *flow,
+ struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_dpaa2_flow_item *dpaa2_pattern,
+ const struct rte_flow_action actions[] __rte_unused,
+ struct rte_flow_error *error __rte_unused,
+ int *device_configured)
+{
+ int ret, local_cfg = 0;
+ uint32_t group;
+ const struct rte_flow_item_pppoe *spec, *mask;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ const struct rte_flow_item *pattern =
+ &dpaa2_pattern->generic_item;
+
+ group = attr->group;
+
+ /* Parse pattern list to get the matching parameters */
+ spec = pattern->spec;
+ mask = pattern->mask ?
+ pattern->mask : &dpaa2_flow_item_pppoe_mask;
+
+ /* Get traffic class index and flow id to be configured */
+ flow->tc_id = group;
+ flow->tc_index = attr->priority;
+
+ if (dpaa2_pattern->in_tunnel) {
+ DPAA2_PMD_ERR("Tunnel-PPPoE distribution not support");
+ return -ENOTSUP;
+ }
+
+ if (!spec) {
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ FAF_PPPOE_PPP_FRAM, DPAA2_FLOW_QOS_TYPE,
+ group, &local_cfg);
+ if (ret)
+ return ret;
+
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ FAF_PPPOE_PPP_FRAM, DPAA2_FLOW_FS_TYPE,
+ group, &local_cfg);
+ if (ret)
+ return ret;
+
+ (*device_configured) |= local_cfg;
+ return 0;
+ }
+
+ ret = dpaa2_flow_extract_support((const uint8_t *)mask,
+ RTE_FLOW_ITEM_TYPE_PPPOES);
+ if (ret) {
+ DPAA2_PMD_WARN("Extract field(s) of PPPoE not support.");
+ return ret;
+ }
+
+ if (!mask->session_id)
+ return 0;
+
+ ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_PPPOE,
+ NH_FLD_PPPOE_SID, &spec->session_id,
+ &mask->session_id, sizeof(rte_be16_t),
+ priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
+ if (ret)
+ return ret;
+
+ ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_PPPOE,
+ NH_FLD_PPPOE_SID, &spec->session_id,
+ &mask->session_id, sizeof(rte_be16_t),
+ priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ if (ret)
+ return ret;
+
+ (*device_configured) |= local_cfg;
+
+ return 0;
+}
+
static int
dpaa2_configure_flow_raw(struct dpaa2_dev_flow *flow,
struct rte_eth_dev *dev,
@@ -4712,6 +4888,26 @@ dpaa2_generic_flow_set(struct dpaa2_dev_flow *flow,
goto end_flow_set;
}
break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ ret = dpaa2_configure_flow_mpls(flow,
+ dev, attr, &dpaa2_pattern[i],
+ actions, error,
+ &is_keycfg_configured);
+ if (ret) {
+ DPAA2_PMD_ERR("MPLS flow config failed!");
+ goto end_flow_set;
+ }
+ break;
+ case RTE_FLOW_ITEM_TYPE_PPPOES:
+ ret = dpaa2_configure_flow_pppoe(flow,
+ dev, attr, &dpaa2_pattern[i],
+ actions, error,
+ &is_keycfg_configured);
+ if (ret) {
+ DPAA2_PMD_ERR("PPPoE flow config failed!");
+ goto end_flow_set;
+ }
+ break;
case RTE_FLOW_ITEM_TYPE_RAW:
ret = dpaa2_configure_flow_raw(flow, dev, attr,
&dpaa2_pattern[i],
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 19/45] net/dpaa2: support meter and policing
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (17 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 18/45] net/dpaa2: support MPLS and PPPoE flow distribution Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 20/45] net/dpaa2: support flow drop action Prashant Gupta
` (25 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Add support for the traffic metering and policing (rte_mtr) API to
the dpaa2 PMD. The implementation registers the rte_mtr operations
through the ".mtr_ops_get" dev_ops hook and maintains software
bookkeeping of meter profiles, policies and meter objects under a
per-device spinlock.
Supported algorithms are trTCM RFC 2698 and trTCM RFC 4115 in both
byte and packet modes. Meter policies support drop and pass-through
actions on the red color.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 1 +
drivers/net/dpaa2/dpaa2_ethdev.h | 29 ++
drivers/net/dpaa2/dpaa2_meter.c | 588 +++++++++++++++++++++++++++++++
drivers/net/dpaa2/mc/fsl_dpni.h | 23 +-
drivers/net/dpaa2/meson.build | 1 +
5 files changed, 638 insertions(+), 4 deletions(-)
create mode 100644 drivers/net/dpaa2/dpaa2_meter.c
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index d79dfa7114..c8abab830f 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3322,6 +3322,7 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.rxq_info_get = dpaa2_rxq_info_get,
.txq_info_get = dpaa2_txq_info_get,
.tm_ops_get = dpaa2_tm_ops_get,
+ .mtr_ops_get = dpaa2_mtr_ops_get,
#if defined(RTE_LIBRTE_IEEE1588)
.timesync_enable = dpaa2_timesync_enable,
.timesync_disable = dpaa2_timesync_disable,
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 0366315ec7..c5aac65c28 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -385,6 +385,30 @@ struct extract_s {
uint8_t *tc_extract_param[MAX_TCS];
};
+struct dpaa2_dev_meter_profile {
+ LIST_ENTRY(dpaa2_dev_meter_profile) next;
+ uint32_t profile_id;
+ uint64_t cir;
+ uint64_t cbs;
+ uint64_t pir;
+ uint64_t pbs;
+ enum dpni_policer_mode mode;
+ enum dpni_policer_unit policer_unit;
+};
+
+struct dpaa2_dev_meter_policy {
+ LIST_ENTRY(dpaa2_dev_meter_policy) next;
+ uint32_t policy_id;
+ int red_drop;
+};
+
+struct dpaa2_dev_meter {
+ LIST_ENTRY(dpaa2_dev_meter) next;
+ uint32_t meter_id;
+ uint32_t profile_id;
+ uint32_t policy_id;
+};
+
struct dpaa2_dev_priv {
void *hw;
int32_t hw_id;
@@ -412,6 +436,7 @@ struct dpaa2_dev_priv {
uint8_t en_loose_ordered;
uint8_t max_cgs;
uint8_t cgid_in_use[MAX_RX_QUEUES];
+ rte_spinlock_t meter_lock;
/* Current hash distribution size per RX TC, written by
* dpaa2_setup_flow_dist_size() and read by reta_query / reta_update.
@@ -457,6 +482,9 @@ struct dpaa2_dev_priv {
struct dpaa2_dev_flow *curr;
LIST_HEAD(, dpaa2_dev_flow) flows;
+ LIST_HEAD(, dpaa2_dev_meter_profile) profiles;
+ LIST_HEAD(, dpaa2_dev_meter_policy) policies;
+ LIST_HEAD(, dpaa2_dev_meter) meters;
LIST_HEAD(nodes, dpaa2_tm_node) nodes;
LIST_HEAD(shaper_profiles, dpaa2_tm_shaper_profile) shaper_profiles;
};
@@ -549,6 +577,7 @@ int dpaa2_timesync_read_tx_timestamp(struct rte_eth_dev *dev,
int dpaa2_dev_recycle_config(struct rte_eth_dev *eth_dev);
int dpaa2_dev_recycle_deconfig(struct rte_eth_dev *eth_dev);
+int dpaa2_mtr_ops_get(struct rte_eth_dev *dev, void *ops);
int dpaa2_soft_parser_loaded(void);
void
diff --git a/drivers/net/dpaa2/dpaa2_meter.c b/drivers/net/dpaa2/dpaa2_meter.c
new file mode 100644
index 0000000000..aae88a4d4b
--- /dev/null
+++ b/drivers/net/dpaa2/dpaa2_meter.c
@@ -0,0 +1,588 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2025-2026 NXP
+ */
+
+#include <sys/queue.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <sys/mman.h>
+
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_flow_driver.h>
+#include <rte_tailq.h>
+#include <rte_mtr.h>
+#include <rte_mtr_driver.h>
+
+#include <fsl_dpni.h>
+#include <fsl_dpkg.h>
+
+#include <dpaa2_ethdev.h>
+#include <dpaa2_pmd_logs.h>
+
+static char s_err_msg[128];
+
+static struct rte_mtr_capabilities s_dpaa2_mtr_capa = {
+ .color_aware_trtcm_rfc2698_supported = true,
+ .color_aware_trtcm_rfc4115_supported = true,
+ .trtcm_rfc2698_byte_mode_supported = true,
+ .trtcm_rfc2698_packet_mode_supported = true,
+ .trtcm_rfc4115_byte_mode_supported = true,
+ .trtcm_rfc4115_packet_mode_supported = true
+};
+
+static int
+dpaa2_mtr_capabilities_get(struct rte_eth_dev *dev,
+ struct rte_mtr_capabilities *capa,
+ struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+
+ if (capa == NULL) {
+ return -rte_mtr_error_set(error, EINVAL,
+ RTE_MTR_ERROR_TYPE_MTR_PARAMS, NULL,
+ "NULL input parameter");
+ }
+
+ rte_spinlock_lock(&priv->meter_lock);
+ s_dpaa2_mtr_capa.n_max = priv->num_rx_tc;
+ s_dpaa2_mtr_capa.n_shared_max = priv->num_rx_tc;
+ s_dpaa2_mtr_capa.meter_trtcm_rfc2698_n_max = priv->num_rx_tc;
+ s_dpaa2_mtr_capa.meter_trtcm_rfc4115_n_max = priv->num_rx_tc;
+ s_dpaa2_mtr_capa.meter_policy_n_max = priv->num_rx_tc;
+ s_dpaa2_mtr_capa.shared_n_flows_per_mtr_max = priv->fs_entries;
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ *capa = s_dpaa2_mtr_capa;
+
+ return 0;
+}
+
+static int
+dpaa2_mtr_profile_add(struct rte_eth_dev *dev,
+ uint32_t profile_id, struct rte_mtr_meter_profile *profile,
+ struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_profile *dpaa2_profile;
+ struct dpaa2_dev_meter_profile *curr;
+ int ret = 0;
+
+ dpaa2_profile = rte_zmalloc(NULL,
+ sizeof(struct dpaa2_dev_meter_profile), 0);
+ if (dpaa2_profile == NULL) {
+ return -rte_mtr_error_set(error, ENOMEM,
+ RTE_MTR_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Meter profile memory alloc failed!");
+ }
+ if (profile->alg == RTE_MTR_NONE) {
+ dpaa2_profile->mode = DPNI_POLICER_MODE_PASS_THROUGH;
+ } else if (profile->alg == RTE_MTR_TRTCM_RFC2698) {
+ dpaa2_profile->mode = DPNI_POLICER_MODE_RFC_2698;
+ } else if (profile->alg == RTE_MTR_TRTCM_RFC4115) {
+ dpaa2_profile->mode = DPNI_POLICER_MODE_RFC_4115;
+ } else {
+ DPAA2_PMD_ERR("Policer profile alg(%d) not supported!",
+ profile->alg);
+ ret = -ENOTSUP;
+ goto err;
+ }
+
+ if (profile->alg == RTE_MTR_TRTCM_RFC2698) {
+ dpaa2_profile->cir = profile->trtcm_rfc2698.cir;
+ dpaa2_profile->cbs = profile->trtcm_rfc2698.cbs;
+ dpaa2_profile->pir = profile->trtcm_rfc2698.pir;
+ dpaa2_profile->pbs = profile->trtcm_rfc2698.pbs;
+ } else if (profile->alg == RTE_MTR_TRTCM_RFC4115) {
+ dpaa2_profile->cir = profile->trtcm_rfc4115.cir;
+ dpaa2_profile->cbs = profile->trtcm_rfc4115.cbs;
+ dpaa2_profile->pir = profile->trtcm_rfc4115.eir;
+ dpaa2_profile->pbs = profile->trtcm_rfc4115.ebs;
+ }
+
+ /** Align with DPNI policy.*/
+ if (!profile->packet_mode) {
+ dpaa2_profile->policer_unit = DPNI_POLICER_UNIT_BYTES_L3;
+ } else if (profile->packet_mode > DPNI_POLICER_UNIT_FRAMES) {
+ dpaa2_profile->policer_unit =
+ DPNI_POLICER_UNIT_BYTES_L2_WITHOUT_FCS;
+ } else {
+ dpaa2_profile->policer_unit = DPNI_POLICER_UNIT_FRAMES;
+ }
+
+ dpaa2_profile->profile_id = profile_id;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ curr = LIST_FIRST(&priv->profiles);
+ if (curr == NULL) {
+ LIST_INSERT_HEAD(&priv->profiles, dpaa2_profile, next);
+ } else {
+ while (LIST_NEXT(curr, next) != NULL)
+ curr = LIST_NEXT(curr, next);
+ LIST_INSERT_AFTER(curr, dpaa2_profile, next);
+ }
+ rte_spinlock_unlock(&priv->meter_lock);
+
+err:
+ if (ret != 0)
+ rte_free(dpaa2_profile);
+
+ return ret;
+}
+
+static int
+dpaa2_mtr_policy_add(struct rte_eth_dev *dev,
+ uint32_t policy_id, struct rte_mtr_meter_policy_params *policy,
+ struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_policy *dpaa2_policy;
+ struct dpaa2_dev_meter_policy *curr;
+ const struct rte_flow_action *red_action;
+ bool red_drop = false;
+
+ if (policy->actions[RTE_COLOR_GREEN] != NULL) {
+ return -rte_mtr_error_set(error, ENOMEM,
+ RTE_MTR_ERROR_TYPE_POLICER_ACTION_GREEN, NULL,
+ "Meter green policy action not supported!");
+ }
+ if (policy->actions[RTE_COLOR_YELLOW] != NULL) {
+ return -rte_mtr_error_set(error, ENOMEM,
+ RTE_MTR_ERROR_TYPE_POLICER_ACTION_YELLOW, NULL,
+ "Meter yellow policy action not supported!");
+ }
+
+ red_action = policy->actions[RTE_COLOR_RED];
+
+ if (red_action != NULL) {
+ if (red_action->type == RTE_FLOW_ACTION_TYPE_DROP) {
+ red_drop = true;
+ } else if (red_action->type != RTE_FLOW_ACTION_TYPE_PASSTHRU) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter red policy action(%d) NOT supported!",
+ red_action->type);
+ return -rte_mtr_error_set(error, ENOMEM,
+ RTE_MTR_ERROR_TYPE_POLICER_ACTION_RED, NULL,
+ s_err_msg);
+ }
+ }
+
+ dpaa2_policy = rte_zmalloc(NULL,
+ sizeof(struct dpaa2_dev_meter_policy), 0);
+ if (dpaa2_policy == NULL) {
+ return -rte_mtr_error_set(error, ENOMEM,
+ RTE_MTR_ERROR_TYPE_UNSPECIFIED, NULL,
+ "Meter policy memory alloc failed!");
+ }
+
+ dpaa2_policy->policy_id = policy_id;
+ dpaa2_policy->red_drop = red_drop;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ curr = LIST_FIRST(&priv->policies);
+ if (curr == NULL) {
+ LIST_INSERT_HEAD(&priv->policies, dpaa2_policy, next);
+ } else {
+ while (LIST_NEXT(curr, next) != NULL)
+ curr = LIST_NEXT(curr, next);
+ LIST_INSERT_AFTER(curr, dpaa2_policy, next);
+ }
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return 0;
+}
+
+static struct rte_flow_meter_profile *
+dpaa2_mtr_profile_get(struct rte_eth_dev *dev,
+ uint32_t meter_profile_id, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_profile *dpaa2_profile;
+
+ RTE_SET_USED(error);
+
+ rte_spinlock_lock(&priv->meter_lock);
+ dpaa2_profile = LIST_FIRST(&priv->profiles);
+ while (dpaa2_profile != NULL) {
+ if (dpaa2_profile->profile_id == meter_profile_id) {
+ rte_spinlock_unlock(&priv->meter_lock);
+ return (struct rte_flow_meter_profile *)dpaa2_profile;
+ }
+ dpaa2_profile = LIST_NEXT(dpaa2_profile, next);
+ }
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return NULL;
+}
+
+static struct rte_flow_meter_policy *
+dpaa2_mtr_policy_get(struct rte_eth_dev *dev,
+ uint32_t policy_id, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_policy *dpaa2_policy;
+
+ RTE_SET_USED(error);
+
+ rte_spinlock_lock(&priv->meter_lock);
+ dpaa2_policy = LIST_FIRST(&priv->policies);
+ while (dpaa2_policy != NULL) {
+ if (dpaa2_policy->policy_id == policy_id) {
+ rte_spinlock_unlock(&priv->meter_lock);
+ return (struct rte_flow_meter_policy *)dpaa2_policy;
+ }
+ dpaa2_policy = LIST_NEXT(dpaa2_policy, next);
+ }
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return NULL;
+}
+
+static int
+dpaa2_mtr_profile_delete(struct rte_eth_dev *dev,
+ uint32_t profile_id, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_profile *dpaa2_profile = NULL, *curr;
+ struct dpaa2_dev_meter *meter, *tmp;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ curr = LIST_FIRST(&priv->profiles);
+ while (curr != NULL) {
+ if (curr->profile_id == profile_id) {
+ dpaa2_profile = curr;
+ break;
+ }
+ curr = LIST_NEXT(curr, next);
+ }
+ if (dpaa2_profile == NULL) {
+ rte_spinlock_unlock(&priv->meter_lock);
+ return -rte_mtr_error_set(error, ENOENT,
+ RTE_MTR_ERROR_TYPE_METER_PROFILE_ID,
+ &profile_id, "Meter profile is invalid.");
+ }
+
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->profile_id == profile_id) {
+ DPAA2_PMD_INFO("Meter(id=%d) with profile(%d) is removed!",
+ meter->meter_id, profile_id);
+ tmp = meter;
+ meter = LIST_NEXT(meter, next);
+ LIST_REMOVE(tmp, next);
+ rte_free(tmp);
+ } else {
+ meter = LIST_NEXT(meter, next);
+ }
+ }
+
+ LIST_REMOVE(dpaa2_profile, next);
+ rte_free(dpaa2_profile);
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return 0;
+}
+
+static int
+dpaa2_mtr_policy_delete(struct rte_eth_dev *dev,
+ uint32_t policy_id, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_policy *dpaa2_policy = NULL, *curr;
+ struct dpaa2_dev_meter *meter, *tmp;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ curr = LIST_FIRST(&priv->policies);
+ while (curr != NULL) {
+ if (curr->policy_id == policy_id) {
+ dpaa2_policy = curr;
+ break;
+ }
+ curr = LIST_NEXT(curr, next);
+ }
+ if (dpaa2_policy == NULL) {
+ rte_spinlock_unlock(&priv->meter_lock);
+ return -rte_mtr_error_set(error, ENOENT,
+ RTE_MTR_ERROR_TYPE_METER_POLICY_ID,
+ NULL, "Meter policy is invalid.");
+ }
+
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->policy_id == policy_id) {
+ DPAA2_PMD_INFO("Meter(id=%d) with policy(%d) is removed!",
+ meter->meter_id, policy_id);
+ tmp = meter;
+ meter = LIST_NEXT(meter, next);
+ LIST_REMOVE(tmp, next);
+ rte_free(tmp);
+ } else {
+ meter = LIST_NEXT(meter, next);
+ }
+ }
+
+ LIST_REMOVE(dpaa2_policy, next);
+ rte_free(dpaa2_policy);
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return 0;
+}
+
+static int
+dpaa2_mtr_meter_create(struct rte_eth_dev *dev,
+ uint32_t mtr_id, struct rte_mtr_params *params,
+ int shared, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_profile *profile;
+ struct dpaa2_dev_meter_policy *policy;
+ struct dpaa2_dev_meter *meter, *curr;
+ uint32_t profile_id, policy_id;
+ enum rte_mtr_error_type err_type = RTE_MTR_ERROR_TYPE_NONE;
+ bool found = false;
+ int ret = 0;
+
+ RTE_SET_USED(shared);
+ profile_id = params->meter_profile_id;
+ policy_id = params->meter_policy_id;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ profile = LIST_FIRST(&priv->profiles);
+ while (profile != NULL) {
+ if (profile->profile_id == profile_id) {
+ found = true;
+ break;
+ }
+ profile = LIST_NEXT(profile, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter profile ID(%d) not exist!", profile_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_METER_PROFILE_ID;
+ goto quit;
+ }
+
+ found = false;
+ policy = LIST_FIRST(&priv->policies);
+ while (policy != NULL) {
+ if (policy->policy_id == policy_id) {
+ found = true;
+ break;
+ }
+ policy = LIST_NEXT(policy, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter policy ID(%d) not exist!", policy_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_METER_POLICY_ID;
+ goto quit;
+ }
+
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->meter_id == mtr_id) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter ID(%d) exist!", mtr_id);
+ ret = EEXIST;
+ err_type = RTE_MTR_ERROR_TYPE_MTR_ID;
+ goto quit;
+ }
+ meter = LIST_NEXT(meter, next);
+ }
+ meter = rte_zmalloc(NULL, sizeof(struct dpaa2_dev_meter), 0);
+ if (meter == NULL) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter memory alloc failed!");
+ ret = ENOMEM;
+ err_type = RTE_MTR_ERROR_TYPE_UNSPECIFIED;
+ goto quit;
+ }
+ meter->meter_id = mtr_id;
+ meter->profile_id = profile_id;
+ meter->policy_id = policy_id;
+
+ curr = LIST_FIRST(&priv->meters);
+ if (curr == NULL) {
+ LIST_INSERT_HEAD(&priv->meters, meter, next);
+ } else {
+ while (LIST_NEXT(curr, next) != NULL)
+ curr = LIST_NEXT(curr, next);
+ LIST_INSERT_AFTER(curr, meter, next);
+ }
+
+quit:
+ rte_spinlock_unlock(&priv->meter_lock);
+ if (ret != 0)
+ return -rte_mtr_error_set(error, ret, err_type, NULL, s_err_msg);
+
+ return 0;
+}
+
+static int
+dpaa2_mtr_meter_destroy(struct rte_eth_dev *dev,
+ uint32_t mtr_id, struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter *meter;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->meter_id == mtr_id) {
+ LIST_REMOVE(meter, next);
+ rte_free(meter);
+ rte_spinlock_unlock(&priv->meter_lock);
+
+ return 0;
+ }
+ meter = LIST_NEXT(meter, next);
+ }
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter ID(%d) does not exist!", mtr_id);
+ rte_spinlock_unlock(&priv->meter_lock);
+ return -rte_mtr_error_set(error, ENOENT,
+ RTE_MTR_ERROR_TYPE_MTR_ID, NULL, s_err_msg);
+}
+
+static int
+dpaa2_mtr_meter_profile_update(struct rte_eth_dev *dev,
+ uint32_t mtr_id, uint32_t profile_id,
+ struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_profile *profile;
+ struct dpaa2_dev_meter *meter;
+ enum rte_mtr_error_type err_type = RTE_MTR_ERROR_TYPE_NONE;
+ bool found = false;
+ int ret = 0;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->meter_id == mtr_id) {
+ found = true;
+ break;
+ }
+ meter = LIST_NEXT(meter, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter ID(%d) not found!", mtr_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_MTR_ID;
+ goto quit;
+ }
+
+ found = false;
+ profile = LIST_FIRST(&priv->profiles);
+ while (profile != NULL) {
+ if (profile->profile_id == profile_id) {
+ found = true;
+ break;
+ }
+ profile = LIST_NEXT(profile, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Profile ID(%d) not found!", profile_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_METER_PROFILE_ID;
+ goto quit;
+ }
+ meter->profile_id = profile_id;
+
+quit:
+ rte_spinlock_unlock(&priv->meter_lock);
+ if (ret != 0)
+ return -rte_mtr_error_set(error, ret, err_type, NULL, s_err_msg);
+
+ return 0;
+}
+
+static int
+dpaa2_mtr_meter_policy_update(struct rte_eth_dev *dev,
+ uint32_t mtr_id, uint32_t policy_id,
+ struct rte_mtr_error *error)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_meter_policy *policy;
+ struct dpaa2_dev_meter *meter;
+ enum rte_mtr_error_type err_type = RTE_MTR_ERROR_TYPE_NONE;
+ bool found = false;
+ int ret = 0;
+
+ rte_spinlock_lock(&priv->meter_lock);
+ meter = LIST_FIRST(&priv->meters);
+ while (meter != NULL) {
+ if (meter->meter_id == mtr_id) {
+ found = true;
+ break;
+ }
+ meter = LIST_NEXT(meter, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Meter ID(%d) not found!", mtr_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_MTR_ID;
+ goto quit;
+ }
+
+ found = false;
+ policy = LIST_FIRST(&priv->policies);
+ while (policy != NULL) {
+ if (policy->policy_id == policy_id) {
+ found = true;
+ break;
+ }
+ policy = LIST_NEXT(policy, next);
+ }
+ if (!found) {
+ snprintf(s_err_msg, sizeof(s_err_msg),
+ "Profile ID(%d) not found!", policy_id);
+ ret = ENOENT;
+ err_type = RTE_MTR_ERROR_TYPE_METER_POLICY_ID;
+ goto quit;
+ }
+ meter->policy_id = policy_id;
+
+quit:
+ rte_spinlock_unlock(&priv->meter_lock);
+ if (ret != 0)
+ return -rte_mtr_error_set(error, ret, err_type, NULL, s_err_msg);
+
+ return 0;
+}
+
+static const struct rte_mtr_ops dpaa2_meter_ops = {
+ .capabilities_get = dpaa2_mtr_capabilities_get,
+ .meter_profile_add = dpaa2_mtr_profile_add,
+ .meter_profile_delete = dpaa2_mtr_profile_delete,
+ .meter_policy_add = dpaa2_mtr_policy_add,
+ .meter_policy_delete = dpaa2_mtr_policy_delete,
+ .meter_profile_get = dpaa2_mtr_profile_get,
+ .meter_policy_get = dpaa2_mtr_policy_get,
+ .create = dpaa2_mtr_meter_create,
+ .destroy = dpaa2_mtr_meter_destroy,
+ .meter_profile_update = dpaa2_mtr_meter_profile_update,
+ .meter_policy_update = dpaa2_mtr_meter_policy_update,
+};
+
+int
+dpaa2_mtr_ops_get(struct rte_eth_dev *dev, void *ops)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+
+ rte_spinlock_init(&priv->meter_lock);
+
+ *(const void **)ops = &dpaa2_meter_ops;
+ return 0;
+}
diff --git a/drivers/net/dpaa2/mc/fsl_dpni.h b/drivers/net/dpaa2/mc/fsl_dpni.h
index 8913b408e7..d227025e01 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni.h
@@ -1111,6 +1111,11 @@ int dpni_set_rx_tc_dist(struct fsl_mc_io *mc_io,
*/
#define DPNI_POLICER_OPT_DISCARD_RED 0x00000002
+/**
+ * Do NOT reset counters generally for policer update
+ */
+#define DPNI_POLICER_OPT_DO_NOT_RESET_COUNTERS 0x00000004
+
/**
* enum dpni_policer_mode - selecting the policer mode
* @DPNI_POLICER_MODE_NONE: Policer is disabled
@@ -1127,12 +1132,17 @@ enum dpni_policer_mode {
/**
* enum dpni_policer_unit - DPNI policer units
- * @DPNI_POLICER_UNIT_BYTES: bytes units
+ * @DPNI_POLICER_UNIT_BYTES_L3: bytes units (for each frame takes into account
+ * FD length - L3 offset)
* @DPNI_POLICER_UNIT_FRAMES: frames units
+ * @DPNI_POLICER_UNIT_BYTES_L2_WITHOUT_FCS: bytes units (for each frame takes
+ * into account FD length - L2 offset, which means L2 size without the 4 bytes
+ * for the FCS)
*/
enum dpni_policer_unit {
- DPNI_POLICER_UNIT_BYTES = 0,
- DPNI_POLICER_UNIT_FRAMES
+ DPNI_POLICER_UNIT_BYTES_L3 = 0,
+ DPNI_POLICER_UNIT_FRAMES,
+ DPNI_POLICER_UNIT_BYTES_L2_WITHOUT_FCS,
};
/**
@@ -1173,13 +1183,18 @@ struct dpni_rx_tc_policing_cfg {
uint32_t ebs;
};
-
int dpni_set_rx_tc_policing(struct fsl_mc_io *mc_io,
uint32_t cmd_flags,
uint16_t token,
uint8_t tc_id,
const struct dpni_rx_tc_policing_cfg *cfg);
+int dpni_set_rx_tc_policing_v1(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags,
+ uint16_t token,
+ uint8_t tc_id,
+ const struct dpni_rx_tc_policing_cfg *cfg);
+
int dpni_get_rx_tc_policing(struct fsl_mc_io *mc_io,
uint32_t cmd_flags,
uint16_t token,
diff --git a/drivers/net/dpaa2/meson.build b/drivers/net/dpaa2/meson.build
index 4094ea02ba..808dad8112 100644
--- a/drivers/net/dpaa2/meson.build
+++ b/drivers/net/dpaa2/meson.build
@@ -11,6 +11,7 @@ sources = files(
'base/dpaa2_hw_dpni.c',
'base/dpaa2_tlu_hash.c',
'dpaa2_tm.c',
+ 'dpaa2_meter.c',
'dpaa2_mux.c',
'dpaa2_ethdev.c',
'dpaa2_flow.c',
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 20/45] net/dpaa2: support flow drop action
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (18 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 19/45] net/dpaa2: support meter and policing Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 21/45] net/dpaa2: set default flow miss action per device Prashant Gupta
` (24 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
Add support for the RTE_FLOW_ACTION_TYPE_DROP action in the dpaa2 flow
engine. A flow rule with a drop action is programmed into the flow
steering (FS) table with the DPNI_FS_OPT_DISCARD option so that matching
frames are discarded by hardware.
The drop action is added to the supported action list, accepted during
action verification, configured in the FS action path, handled in the
generic flow-set action loop, and cleaned up in the flow destroy path.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_flow.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 90e0b66258..b2f1778de3 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -97,7 +97,8 @@ enum rte_flow_action_type dpaa2_supported_action_type[] = {
RTE_FLOW_ACTION_TYPE_QUEUE,
RTE_FLOW_ACTION_TYPE_PORT_ID,
RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
- RTE_FLOW_ACTION_TYPE_RSS
+ RTE_FLOW_ACTION_TYPE_RSS,
+ RTE_FLOW_ACTION_TYPE_DROP
};
#ifndef __cplusplus
@@ -4421,6 +4422,7 @@ dpaa2_flow_verify_action(struct dpaa2_dev_priv *priv,
break;
case RTE_FLOW_ACTION_TYPE_PF:
/* Skip this action, have to add for vxlan */
+ case RTE_FLOW_ACTION_TYPE_DROP:
break;
case RTE_FLOW_ACTION_TYPE_END:
end_of_list = 1;
@@ -4468,6 +4470,8 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
flow->fs_action_cfg.redirect_obj_token =
dest_priv->token;
flow->fs_action_cfg.flow_id = dest_q->flow_id;
+ } else if (flow->action_type == RTE_FLOW_ACTION_TYPE_DROP) {
+ flow->fs_action_cfg.options = DPNI_FS_OPT_DISCARD;
}
return 0;
@@ -4945,6 +4949,7 @@ dpaa2_generic_flow_set(struct dpaa2_dev_flow *flow,
case RTE_FLOW_ACTION_TYPE_QUEUE:
case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ case RTE_FLOW_ACTION_TYPE_DROP:
ret = dpaa2_configure_flow_fs_action(priv, flow,
&actions[j]);
if (ret)
@@ -5346,6 +5351,7 @@ dpaa2_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *_flow,
case RTE_FLOW_ACTION_TYPE_QUEUE:
case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ case RTE_FLOW_ACTION_TYPE_DROP:
if (priv->num_rx_tc > 1) {
/* Remove entry from QoS table first */
ret = dpni_remove_qos_entry(dpni, CMD_PRI_LOW,
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 21/45] net/dpaa2: set default flow miss action per device
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (19 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 20/45] net/dpaa2: support flow drop action Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 22/45] net/dpaa2: identify Rx mbuf hash information by FLC Prashant Gupta
` (23 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
The flow steering miss flow id was a file-scope global configured only
via the DPAA2_FLOW_CONTROL_MISS_FLOW environment variable and defaulted
to 0. Replace it with a per-device default_flow computed at probe time
as the lowest priority flow of the FS table, so that unmatched frames
are directed there without depending on an environment variable.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 2 ++
drivers/net/dpaa2/dpaa2_ethdev.h | 2 ++
drivers/net/dpaa2/dpaa2_flow.c | 14 +-------------
3 files changed, 5 insertions(+), 13 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index c8abab830f..387a153036 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3545,6 +3545,8 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
priv->dist_queues = attr.num_queues;
priv->num_channels = attr.num_channels;
priv->channel_inuse = 0;
+ priv->default_flow = RTE_MIN(priv->fs_entries,
+ priv->dist_queues) - 1;
rte_spinlock_init(&priv->lpbk_qp_lock);
/* only if the custom CG is enabled */
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index c5aac65c28..4501bd9e49 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -437,6 +437,8 @@ struct dpaa2_dev_priv {
uint8_t max_cgs;
uint8_t cgid_in_use[MAX_RX_QUEUES];
rte_spinlock_t meter_lock;
+ /* Lowest priority FS flow id to receive flow steering miss frames. */
+ uint16_t default_flow;
/* Current hash distribution size per RX TC, written by
* dpaa2_setup_flow_dist_size() and read by reta_query / reta_update.
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index b2f1778de3..2e844121a8 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -24,7 +24,6 @@
#include <dpaa2_pmd_logs.h>
static char *dpaa2_flow_control_log;
-static uint16_t dpaa2_flow_miss_flow_id; /* Default miss flow id is 0. */
static int dpaa2_sp_loaded = -1;
enum dpaa2_flow_entry_size {
@@ -4590,7 +4589,7 @@ dpaa2_configure_fs_rss_table(struct dpaa2_dev_priv *priv,
return 0;
tc_cfg.enable = true;
- tc_cfg.fs_miss_flow_id = dpaa2_flow_miss_flow_id;
+ tc_cfg.fs_miss_flow_id = priv->default_flow;
ret = dpni_set_rx_fs_dist(dpni, CMD_PRI_LOW,
priv->token, &tc_cfg);
if (ret < 0) {
@@ -5224,17 +5223,6 @@ dpaa2_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
dpaa2_flow_control_log =
getenv("DPAA2_FLOW_CONTROL_LOG");
- if (getenv("DPAA2_FLOW_CONTROL_MISS_FLOW")) {
- dpaa2_flow_miss_flow_id =
- (uint16_t)atoi(getenv("DPAA2_FLOW_CONTROL_MISS_FLOW"));
- if (dpaa2_flow_miss_flow_id >= priv->dist_queues) {
- DPAA2_PMD_ERR("Missed flow ID %d >= dist size(%d)",
- dpaa2_flow_miss_flow_id,
- priv->dist_queues);
- return NULL;
- }
- }
-
flow = rte_zmalloc(NULL, sizeof(struct dpaa2_dev_flow),
RTE_CACHE_LINE_SIZE);
if (!flow) {
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 22/45] net/dpaa2: identify Rx mbuf hash information by FLC
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (20 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 21/45] net/dpaa2: set default flow miss action per device Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 23/45] net/dpaa2: add minimum key size support Prashant Gupta
` (22 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
For default RSS distribution, mbuf is marked with RTE_MBUF_F_RX_RSS_HASH
and the hash value from the high word of FLC.
Configure the flow steering action with FLC enabled to align the stashing
setting with the RSS configuration. The TC ID and flow ID are packed into
the FLC for mbuf Rx scheduling. An Rx mbuf received from the FS table is
marked with RTE_MBUF_F_RX_FDIR along with the TC and flow ID decoded from
the FLC low word.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.h | 12 +++++++++
drivers/net/dpaa2/dpaa2_flow.c | 42 ++++++++++++++++++++++++++++++++
drivers/net/dpaa2/dpaa2_rxtx.c | 18 ++++++++++++--
3 files changed, 70 insertions(+), 2 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 4501bd9e49..c693989998 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -282,6 +282,18 @@ enum dpaa2_ecpri_fafe_type {
/* ECPRI shares SP context with VXLAN*/
#define DPAA2_ECPRI_MSG_OFFSET DPAA2_VXLAN_VNI_OFFSET
+#define DPAA2_FS_FLC_FS_MARK_OFFSET \
+ (DPAA2_FLC_DATA_STASHING + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
+
+#define DPAA2_FS_FLC_TC_OFFSET \
+ (DPAA2_FS_FLC_FS_MARK_OFFSET + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
+
+#define DPAA2_FS_FLC_TC_BIT_SIZE (sizeof(uint8_t) * 8)
+#define DPAA2_FS_FLC_TC_MASK ((1 << DPAA2_FS_FLC_TC_BIT_SIZE) - 1)
+
+#define DPAA2_FS_FLC_FLOW_OFFSET \
+ (DPAA2_FS_FLC_TC_OFFSET + DPAA2_FS_FLC_TC_BIT_SIZE)
+
#define DPAA2_ECPRI_MAX_EXTRACT_NB 8
struct ipv4_sd_addr_extract_rule {
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 2e844121a8..45660df31a 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -4383,6 +4383,13 @@ dpaa2_flow_verify_action(struct dpaa2_dev_priv *priv,
switch (actions[j].type) {
case RTE_FLOW_ACTION_TYPE_QUEUE:
dest_queue = actions[j].conf;
+ if (dest_queue->index >= MAX_RX_QUEUES ||
+ !priv->rx_vq[dest_queue->index]) {
+ DPAA2_PMD_ERR("Invalid FSQ index(%d)",
+ dest_queue->index);
+
+ return -EINVAL;
+ }
rxq = priv->rx_vq[dest_queue->index];
if (attr->group != rxq->tc_index) {
DPAA2_PMD_ERR("FSQ(%d.%d) not in TC[%d]",
@@ -4445,6 +4452,7 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
struct dpaa2_dev_priv *dest_priv;
const struct rte_flow_action_queue *dest_queue;
struct dpaa2_queue *dest_q;
+ uint64_t flc = 0;
memset(&flow->fs_action_cfg, 0,
sizeof(struct dpni_fs_action_cfg));
@@ -4452,7 +4460,41 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
if (flow->action_type == RTE_FLOW_ACTION_TYPE_QUEUE) {
dest_queue = rte_action->conf;
+ if (dest_queue->index >= MAX_RX_QUEUES ||
+ !priv->rx_vq[dest_queue->index]) {
+ DPAA2_PMD_ERR("Invalid FSQ index(%d)",
+ dest_queue->index);
+
+ return -EINVAL;
+ }
dest_q = priv->rx_vq[dest_queue->index];
+ if (flow->tc_id != dest_q->tc_index) {
+ DPAA2_PMD_ERR("RXQ[%d](%d.%d) not in TC[%d]",
+ dest_queue->index,
+ dest_q->tc_index, dest_q->flow_id,
+ flow->tc_id);
+
+ return -EINVAL;
+ }
+ flow->fs_action_cfg.options =
+ DPNI_FS_OPT_SET_FLC | DPNI_FS_OPT_SET_STASH_CONTROL;
+
+ if (dest_q->data_stashing_off) {
+ dpaa2_flc_stashing_set(DPAA2_FLC_DATA_STASHING,
+ 0, &flc);
+ } else {
+ dpaa2_flc_stashing_set(DPAA2_FLC_DATA_STASHING,
+ 1, &flc);
+ }
+ if ((dpaa2_svr_family & 0xffff0000) != SVR_LX2160A) {
+ dpaa2_flc_stashing_set(DPAA2_FLC_ANNO_STASHING,
+ 1, &flc);
+ }
+
+ flc |= ((uint64_t)1) << DPAA2_FS_FLC_FS_MARK_OFFSET;
+ flc |= ((uint64_t)dest_q->tc_index) << DPAA2_FS_FLC_TC_OFFSET;
+ flc |= ((uint64_t)dest_q->flow_id) << DPAA2_FS_FLC_FLOW_OFFSET;
+ flow->fs_action_cfg.flc = flc;
flow->fs_action_cfg.flow_id = dest_q->flow_id;
} else if (flow->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
flow->action_type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) {
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index c31d6556ad..7ba6a4563f 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -58,6 +58,7 @@ dpaa2_dev_rx_parse_new(struct rte_mbuf *m, const struct qbman_fd *fd,
uint16_t frc = DPAA2_GET_FD_FRC_PARSE_SUM(fd);
struct dpaa2_annot_hdr *annotation =
(struct dpaa2_annot_hdr *)hw_annot_addr;
+ uint32_t flc_lo, tc, flow;
if (unlikely(dpaa2_print_parser_result))
dpaa2_print_parse_result(annotation);
@@ -118,8 +119,21 @@ dpaa2_dev_rx_parse_new(struct rte_mbuf *m, const struct qbman_fd *fd,
default:
m->packet_type = dpaa2_dev_rx_parse_slow(m, annotation);
}
- m->hash.rss = fd->simple.flc_hi;
- m->ol_flags |= RTE_MBUF_F_RX_RSS_HASH;
+ flc_lo = fd->simple.flc_lo;
+ if (flc_lo & (1 << DPAA2_FS_FLC_FS_MARK_OFFSET)) {
+ m->ol_flags |= RTE_MBUF_F_RX_FDIR;
+ tc = (flc_lo >> DPAA2_FS_FLC_TC_OFFSET) &
+ DPAA2_FS_FLC_TC_MASK;
+ flow = flc_lo >> DPAA2_FS_FLC_FLOW_OFFSET;
+ rte_mbuf_sched_set(m, flow, tc, 0);
+ DPAA2_PMD_DP_DEBUG("FS frame received from TC[%d]->flow%d",
+ tc, flow);
+ } else {
+ m->hash.rss = fd->simple.flc_hi;
+ m->ol_flags |= RTE_MBUF_F_RX_RSS_HASH;
+ DPAA2_PMD_DP_DEBUG("Hash frame received with RSS(%08x)",
+ m->hash.rss);
+ }
if (dpaa2_enable_ts[m->port]) {
*dpaa2_timestamp_dynfield(m) = annotation->word2;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 23/45] net/dpaa2: add minimum key size support
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (21 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 22/45] net/dpaa2: identify Rx mbuf hash information by FLC Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 24/45] net/dpaa2: restructure dpaa2 parser processing Prashant Gupta
` (21 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
The default size for a QoS key is 24 bytes. If the key size is
smaller than the maximum, use the default (minimum) key size
instead of always returning the maximum size (56 bytes). This
allows smaller flow tables to be used when the extracted key
fits within the default size.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_flow.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 45660df31a..2662c358e4 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -26,8 +26,11 @@
static char *dpaa2_flow_control_log;
static int dpaa2_sp_loaded = -1;
+/* Default size of a key */
+#define DPNI_DEFAULT_KEY_SIZE 24
+
enum dpaa2_flow_entry_size {
- DPAA2_FLOW_ENTRY_MIN_SIZE = (DPNI_MAX_KEY_SIZE / 2),
+ DPAA2_FLOW_ENTRY_MIN_SIZE = DPNI_DEFAULT_KEY_SIZE,
DPAA2_FLOW_ENTRY_MAX_SIZE = DPNI_MAX_KEY_SIZE
};
@@ -4532,8 +4535,7 @@ dpaa2_flow_entry_size(uint16_t key_max_size)
if (key_max_size > DPAA2_FLOW_ENTRY_MIN_SIZE)
return DPAA2_FLOW_ENTRY_MAX_SIZE;
- /* Current MC only support fixed entry size(56)*/
- return DPAA2_FLOW_ENTRY_MAX_SIZE;
+ return DPAA2_FLOW_ENTRY_MIN_SIZE;
}
static inline int
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 24/45] net/dpaa2: restructure dpaa2 parser processing
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (22 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 23/45] net/dpaa2: add minimum key size support Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 25/45] net/dpaa2: parse tunnel and fragmented packet types Prashant Gupta
` (20 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Move the parse-result decoding into a dedicated header so that the
hardware parser fields are described in one place and shared between
the flow code and the Rx path. No functional change is intended.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.h | 172 ++-
drivers/net/dpaa2/dpaa2_parse_dump.h | 374 ------
drivers/net/dpaa2/dpaa2_parser_decode.h | 1539 +++++++++++++++++++++++
drivers/net/dpaa2/dpaa2_pmd_logs.h | 67 +
drivers/net/dpaa2/dpaa2_rxtx.c | 332 ++---
5 files changed, 1906 insertions(+), 578 deletions(-)
delete mode 100644 drivers/net/dpaa2/dpaa2_parse_dump.h
create mode 100644 drivers/net/dpaa2/dpaa2_parser_decode.h
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index c693989998..2f552f3e29 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -21,6 +21,8 @@
#include <mc/fsl_dpmac.h>
#include "base/dpaa2_hw_dpni_annot.h"
+#include "dpaa2_parser_decode.h"
+
#define DPAA2_MIN_RX_BUF_SIZE 512
#define DPAA2_MAX_RX_PKT_LEN 10240 /*WRIOP support*/
@@ -101,6 +103,7 @@
#define DPAA2_DATA_STASHING_OFF RTE_BIT32(10)
#define DPAAX_RX_ERROR_QUEUE_FLAG RTE_BIT32(11)
+#define DPAA2_RX_PRINT_PSR_RESULT_FLAG RTE_BIT32(12)
/* DPDMUX index for DPMAC */
#define DPAA2_DPDMUX_DPMAC_IDX 0
@@ -124,31 +127,62 @@
RTE_ETH_RSS_PPPOE)
/* LX2 FRC Parsed values (Little Endian) */
-#define DPAA2_PKT_TYPE_ETHER 0x0060
-#define DPAA2_PKT_TYPE_IPV4 0x0000
-#define DPAA2_PKT_TYPE_IPV6 0x0020
+#define DPAA2_PKT_TYPE_ETHER \
+ (DPAA2_PSR_SUMMARY_NONIP << DPAA2_PSR_SUMMARY_NON_IP_L2_BIT_SIZE)
+#define DPAA2_PKT_TYPE_IPV4 \
+ (DPAA2_PSR_SUMMARY_IPV4 << DPAA2_PSR_SUMMARY_L4_BIT_SIZE)
+#define DPAA2_PKT_TYPE_IPV6 \
+ (DPAA2_PSR_SUMMARY_IPV6 << DPAA2_PSR_SUMMARY_L4_BIT_SIZE)
+
+/**Tunnel*/
+#define DPAA2_PKT_TYPE_L3_EXT \
+ (DPAA2_PSR_SUMMARY_L4_EXT << DPAA2_PSR_SUMMARY_L4_EXT_L3_POS)
+#define DPAA2_PKT_TYPE_L3_EXT_IPV4 \
+ (DPAA2_PSR_SUMMARY_IPV4 << DPAA2_PSR_SUMMARY_L4_EXT_BIT_SIZE)
+#define DPAA2_PKT_TYPE_L3_EXT_IPV6 \
+ (DPAA2_PSR_SUMMARY_IPV6 << DPAA2_PSR_SUMMARY_L4_EXT_BIT_SIZE)
#define DPAA2_PKT_TYPE_IPV4_EXT \
- (0x0001 | DPAA2_PKT_TYPE_IPV4)
+ (DPAA2_PKT_TYPE_L3_EXT | DPAA2_PKT_TYPE_L3_EXT_IPV4)
#define DPAA2_PKT_TYPE_IPV6_EXT \
- (0x0001 | DPAA2_PKT_TYPE_IPV6)
+ (DPAA2_PKT_TYPE_L3_EXT | DPAA2_PKT_TYPE_L3_EXT_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_EXT_GRE_IPV4 \
+ (DPAA2_PKT_TYPE_IPV4_EXT | DPAA2_PSR_SUMMARY_GRE_IPV4)
+#define DPAA2_PKT_TYPE_IPV4_EXT_GRE_IPV4_UDP_TCP \
+ (DPAA2_PKT_TYPE_IPV4_EXT | DPAA2_PSR_SUMMARY_GRE_IPV4_UDP_TCP)
+#define DPAA2_PKT_TYPE_IPV4_EXT_GRE_IPV6 \
+ (DPAA2_PKT_TYPE_IPV4_EXT | DPAA2_PSR_SUMMARY_GRE_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_EXT_GRE_IPV6_UDP_TCP \
+ (DPAA2_PKT_TYPE_IPV4_EXT | DPAA2_PSR_SUMMARY_GRE_IPV6_UDP_TCP)
+#define DPAA2_PKT_TYPE_IPV6_EXT_GRE_IPV4 \
+ (DPAA2_PKT_TYPE_IPV6_EXT | DPAA2_PSR_SUMMARY_GRE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_EXT_GRE_IPV4_UDP_TCP \
+ (DPAA2_PKT_TYPE_IPV6_EXT | DPAA2_PSR_SUMMARY_GRE_IPV4_UDP_TCP)
+#define DPAA2_PKT_TYPE_IPV6_EXT_GRE_IPV6 \
+ (DPAA2_PKT_TYPE_IPV6_EXT | DPAA2_PSR_SUMMARY_GRE_IPV6)
+#define DPAA2_PKT_TYPE_IPV6_EXT_GRE_IPV6_UDP_TCP \
+ (DPAA2_PKT_TYPE_IPV6_EXT | DPAA2_PSR_SUMMARY_GRE_IPV6_UDP_TCP)
+
#define DPAA2_PKT_TYPE_IPV4_TCP \
- (0x000e | DPAA2_PKT_TYPE_IPV4)
+ (DPAA2_PSR_SUMMARY_TCP | DPAA2_PKT_TYPE_IPV4)
#define DPAA2_PKT_TYPE_IPV6_TCP \
- (0x000e | DPAA2_PKT_TYPE_IPV6)
+ (DPAA2_PSR_SUMMARY_TCP | DPAA2_PKT_TYPE_IPV6)
#define DPAA2_PKT_TYPE_IPV4_UDP \
- (0x0010 | DPAA2_PKT_TYPE_IPV4)
+ (DPAA2_PSR_SUMMARY_UDP | DPAA2_PKT_TYPE_IPV4)
#define DPAA2_PKT_TYPE_IPV6_UDP \
- (0x0010 | DPAA2_PKT_TYPE_IPV6)
-#define DPAA2_PKT_TYPE_IPV4_SCTP \
- (0x000f | DPAA2_PKT_TYPE_IPV4)
-#define DPAA2_PKT_TYPE_IPV6_SCTP \
- (0x000f | DPAA2_PKT_TYPE_IPV6)
+ (DPAA2_PSR_SUMMARY_UDP | DPAA2_PKT_TYPE_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_SCTP \
+ (DPAA2_PSR_SUMMARY_SCTP | DPAA2_PKT_TYPE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_SCTP \
+ (DPAA2_PSR_SUMMARY_SCTP | DPAA2_PKT_TYPE_IPV6)
#define DPAA2_PKT_TYPE_IPV4_ICMP \
- (0x0003 | DPAA2_PKT_TYPE_IPV4_EXT)
+ (DPAA2_PSR_SUMMARY_ICMP | DPAA2_PKT_TYPE_IPV4)
#define DPAA2_PKT_TYPE_IPV6_ICMP \
- (0x0003 | DPAA2_PKT_TYPE_IPV6_EXT)
-#define DPAA2_PKT_TYPE_VLAN_1 0x0160
-#define DPAA2_PKT_TYPE_VLAN_2 0x0260
+ (DPAA2_PSR_SUMMARY_ICMP | DPAA2_PKT_TYPE_IPV6)
+
+#define DPAA2_PKT_TYPE_VLAN_1 0x0100
+#define DPAA2_PKT_TYPE_VLAN_2 0x0200
+#define DPAA2_PKT_TYPE_VLAN \
+ (DPAA2_PKT_TYPE_VLAN_1 | DPAA2_PKT_TYPE_VLAN_2)
/* mac counters */
#define DPAA2_MAC_NUM_STATS (DPMAC_CNT_EGR_CONTROL_FRAME + 1)
@@ -173,17 +207,33 @@ struct sw_buf_free {
struct rte_mbuf *seg;
};
-/* enable timestamp in mbuf*/
-extern bool dpaa2_enable_ts[];
-extern uint64_t dpaa2_timestamp_rx_dynflag;
-extern int dpaa2_timestamp_dynfield_offset;
-
/* Externally defined */
extern const struct rte_flow_ops dpaa2_flow_ops;
extern const struct rte_tm_ops dpaa2_tm_ops;
-extern bool dpaa2_print_parser_result;
+struct dpaa2_dyn_rx_protocol_pos {
+ uint8_t l3_offset;
+ uint8_t l4_offset;
+ uint8_t l5_offset;
+ uint8_t rsv;
+};
+
+#define L3_OFFSET_OF_MBUF_DYN 0
+#define L4_OFFSET_OF_MBUF_DYN 1
+#define L5_OFFSET_OF_MBUF_DYN 2
+
+#define DPAA2_FS_FLC_FS_MARK_OFFSET \
+ (DPAA2_FLC_DATA_STASHING + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
+
+#define DPAA2_FS_FLC_TC_OFFSET \
+ (DPAA2_FS_FLC_FS_MARK_OFFSET + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
+
+#define DPAA2_FS_FLC_TC_BIT_SIZE (sizeof(uint8_t) * 8)
+#define DPAA2_FS_FLC_TC_MASK ((1 << DPAA2_FS_FLC_TC_BIT_SIZE) - 1)
+
+#define DPAA2_FS_FLC_FLOW_OFFSET \
+ (DPAA2_FS_FLC_TC_OFFSET + DPAA2_FS_FLC_TC_BIT_SIZE)
#define DPAA2_FAPR_SIZE \
(sizeof(struct dpaa2_annot_hdr) - \
@@ -258,42 +308,6 @@ enum dpaa2_ecpri_fafe_type {
#define DPAA2_PR_L5_OFF_OFFSET 31
#define DPAA2_PR_NXTHDR_OFF_OFFSET 34
-/* Set by SP for vxlan distribution start*/
-#define DPAA2_VXLAN_IN_TCI_OFFSET 16
-
-#define DPAA2_VXLAN_IN_DADDR0_OFFSET 20
-#define DPAA2_VXLAN_IN_DADDR1_OFFSET 22
-#define DPAA2_VXLAN_IN_DADDR2_OFFSET 24
-#define DPAA2_VXLAN_IN_DADDR3_OFFSET 25
-#define DPAA2_VXLAN_IN_DADDR4_OFFSET 26
-#define DPAA2_VXLAN_IN_DADDR5_OFFSET 28
-
-#define DPAA2_VXLAN_IN_SADDR0_OFFSET 29
-#define DPAA2_VXLAN_IN_SADDR1_OFFSET 32
-#define DPAA2_VXLAN_IN_SADDR2_OFFSET 33
-#define DPAA2_VXLAN_IN_SADDR3_OFFSET 35
-#define DPAA2_VXLAN_IN_SADDR4_OFFSET 41
-#define DPAA2_VXLAN_IN_SADDR5_OFFSET 42
-
-#define DPAA2_VXLAN_VNI_OFFSET 43
-#define DPAA2_VXLAN_IN_TYPE_OFFSET 46
-/* Set by SP for vxlan distribution end*/
-
-/* ECPRI shares SP context with VXLAN*/
-#define DPAA2_ECPRI_MSG_OFFSET DPAA2_VXLAN_VNI_OFFSET
-
-#define DPAA2_FS_FLC_FS_MARK_OFFSET \
- (DPAA2_FLC_DATA_STASHING + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
-
-#define DPAA2_FS_FLC_TC_OFFSET \
- (DPAA2_FS_FLC_FS_MARK_OFFSET + DPAA2_FLC_STASHING_MAX_BIT_SIZE)
-
-#define DPAA2_FS_FLC_TC_BIT_SIZE (sizeof(uint8_t) * 8)
-#define DPAA2_FS_FLC_TC_MASK ((1 << DPAA2_FS_FLC_TC_BIT_SIZE) - 1)
-
-#define DPAA2_FS_FLC_FLOW_OFFSET \
- (DPAA2_FS_FLC_TC_OFFSET + DPAA2_FS_FLC_TC_BIT_SIZE)
-
#define DPAA2_ECPRI_MAX_EXTRACT_NB 8
struct ipv4_sd_addr_extract_rule {
@@ -436,6 +450,7 @@ struct dpaa2_dev_priv {
void *tx_conf_vq[MAX_TX_QUEUES];
void *rx_err_vq;
uint32_t flags; /*dpaa2 config flags */
+ int psr_dynfield_offset;
uint8_t max_mac_filters;
uint8_t max_vlan_filters;
uint8_t num_rx_tc;
@@ -475,6 +490,9 @@ struct dpaa2_dev_priv {
uint64_t rx_timestamp;
/*stores timestamp of last received tx confirmation packet on dev*/
uint64_t tx_timestamp;
+
+ int rx_ts_offset;
+ uint64_t rx_ts_flag;
/* stores pointer to next tx_conf queue that should be processed,
* it corresponds to last packet transmitted
*/
@@ -483,6 +501,7 @@ struct dpaa2_dev_priv {
struct rte_eth_dev *eth_dev; /**< Pointer back to holding ethdev */
rte_spinlock_t lpbk_qp_lock;
+ bool sp_protocol;
uint8_t channel_inuse;
/* Rx-interrupt mode latched at the first dev_configure */
uint8_t intr_mode;
@@ -514,6 +533,41 @@ static inline int dpaa2_dev_cmp_dpni_ver(struct dpaa2_dev_priv *priv,
return priv->dpni_ver_major - ver_major;
}
+static inline void
+dpaa2_dev_rx_print_parser_result(struct dpaa2_dev_priv *priv,
+ const struct qbman_fd *fd, const struct rte_mbuf *m)
+{
+ size_t fd_addr;
+ void *hw_annot_addr;
+
+ if (likely(!(priv->flags & DPAA2_RX_PRINT_PSR_RESULT_FLAG)))
+ return;
+
+ if (dpaa2_svr_family == SVR_LX2160A)
+ dpaa2_print_fd_frc(fd);
+
+ fd_addr = (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
+ hw_annot_addr = (void *)(fd_addr + DPAA2_FD_PTA_SIZE);
+ dpaa2_print_parse_result(hw_annot_addr, priv->sp_protocol);
+ if (m->ol_flags & RTE_MBUF_F_RX_FDIR) {
+ const struct rte_mbuf_sched *sched;
+ uint16_t i;
+ struct dpaa2_queue *rxq;
+
+ sched = &m->hash.sched;
+ for (i = 0; i < MAX_RX_QUEUES; i++) {
+ rxq = priv->rx_vq[i];
+ if (rxq->tc_index == sched->traffic_class &&
+ rxq->flow_id == sched->queue_id)
+ break;
+ }
+ DPAA2_PMD_DEBUG("Directed to TC%d-flow%d(rxq%d), color(%d)",
+ sched->traffic_class, sched->queue_id, i, sched->color);
+ } else if (m->ol_flags & RTE_MBUF_F_RX_RSS_HASH) {
+ DPAA2_PMD_DEBUG("Balanced with hash(0x%08x)", m->hash.rss);
+ }
+}
+
int dpaa2_distset_to_dpkg_profile_cfg(uint64_t req_dist_set,
struct dpkg_profile_cfg *kg_cfg);
diff --git a/drivers/net/dpaa2/dpaa2_parse_dump.h b/drivers/net/dpaa2/dpaa2_parse_dump.h
deleted file mode 100644
index bcab3bf0cb..0000000000
--- a/drivers/net/dpaa2/dpaa2_parse_dump.h
+++ /dev/null
@@ -1,374 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- *
- * Copyright 2022 NXP
- *
- */
-
-#ifndef _DPAA2_PARSE_DUMP_H
-#define _DPAA2_PARSE_DUMP_H
-
-#include <rte_event_eth_rx_adapter.h>
-#include <rte_pmd_dpaa2.h>
-
-#include <dpaa2_hw_pvt.h>
-#include "dpaa2_tm.h"
-
-#include <mc/fsl_dpni.h>
-#include <mc/fsl_mc_sys.h>
-
-#include "base/dpaa2_hw_dpni_annot.h"
-
-#define DPAA2_PR_PRINT printf
-
-struct dpaa2_faf_bit_info {
- const char *name;
- int position;
-};
-
-struct dpaa2_fapr_field_info {
- const char *name;
- uint16_t value;
-};
-
-struct dpaa2_fapr_array {
- union {
- uint64_t pr_64[DPAA2_FAPR_SIZE / 8];
- uint8_t pr[DPAA2_FAPR_SIZE];
- };
-};
-
-#define NEXT_HEADER_NAME "Next Header"
-#define ETH_OFF_NAME "ETH OFFSET"
-#define VLAN_TCI_OFF_NAME "VLAN TCI OFFSET"
-#define LAST_ENTRY_OFF_NAME "LAST ETYPE Offset"
-#define L3_OFF_NAME "L3 Offset"
-#define L4_OFF_NAME "L4 Offset"
-#define L5_OFF_NAME "L5 Offset"
-#define NEXT_HEADER_OFF_NAME "Next Header Offset"
-
-static const
-struct dpaa2_fapr_field_info support_dump_fields[] = {
- {
- .name = NEXT_HEADER_NAME,
- },
- {
- .name = ETH_OFF_NAME,
- },
- {
- .name = VLAN_TCI_OFF_NAME,
- },
- {
- .name = LAST_ENTRY_OFF_NAME,
- },
- {
- .name = L3_OFF_NAME,
- },
- {
- .name = L4_OFF_NAME,
- },
- {
- .name = L5_OFF_NAME,
- },
- {
- .name = NEXT_HEADER_OFF_NAME,
- }
-};
-
-static inline void
-dpaa2_print_ecpri(struct dpaa2_fapr_array *fapr)
-{
- uint8_t ecpri_type;
- struct rte_ecpri_combined_msg_hdr ecpri_msg;
-
- ecpri_type = fapr->pr[DPAA2_FAFE_PSR_OFFSET];
- if ((ecpri_type >> 1) > 7) {
- DPAA2_PR_PRINT("Invalid ECPRI type(0x%02x)\r\n",
- ecpri_type);
- } else {
- DPAA2_PR_PRINT("ECPRI type %d present\r\n",
- ecpri_type >> 1);
- if (ecpri_type == ECPRI_FAFE_TYPE_0 ||
- ecpri_type == ECPRI_FAFE_TYPE_1) {
- /* Type 0 is identical to type1*/
- ecpri_msg.type0.pc_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1];
- ecpri_msg.type0.pc_id =
- ecpri_msg.type0.pc_id << 8;
- ecpri_msg.type0.pc_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
-
- ecpri_msg.type0.seq_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 3];
- ecpri_msg.type0.seq_id =
- ecpri_msg.type0.seq_id << 8;
- ecpri_msg.type0.seq_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 2];
-
- DPAA2_PR_PRINT("pc_id(0x%04x) seq_id(0x%04x)\r\n",
- rte_be_to_cpu_16(ecpri_msg.type0.pc_id),
- rte_be_to_cpu_16(ecpri_msg.type0.seq_id));
- } else if (ecpri_type == ECPRI_FAFE_TYPE_2) {
- ecpri_msg.type2.rtc_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1];
- ecpri_msg.type2.rtc_id =
- ecpri_msg.type2.rtc_id << 8;
- ecpri_msg.type2.rtc_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
-
- ecpri_msg.type2.seq_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 3];
- ecpri_msg.type2.seq_id =
- ecpri_msg.type2.seq_id << 8;
- ecpri_msg.type2.seq_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 2];
-
- DPAA2_PR_PRINT("rtc_id(0x%04x) seq_id(0x%04x)\r\n",
- rte_be_to_cpu_16(ecpri_msg.type2.rtc_id),
- rte_be_to_cpu_16(ecpri_msg.type2.seq_id));
- } else if (ecpri_type == ECPRI_FAFE_TYPE_3) {
- DPAA2_PR_PRINT("ECPRI type3 extract not support\r\n");
- } else if (ecpri_type == ECPRI_FAFE_TYPE_4) {
- ecpri_msg.type4.rma_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
- ecpri_msg.type4.rw =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1] >> 4;
- ecpri_msg.type4.rr =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1] & 0xf;
-
- ecpri_msg.type4.ele_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 3];
- ecpri_msg.type4.ele_id =
- ecpri_msg.type4.ele_id << 8;
- ecpri_msg.type4.ele_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 2];
-
- DPAA2_PR_PRINT("rma_id(0x%02x) rw(0x%02x)",
- ecpri_msg.type4.rma_id, ecpri_msg.type4.rw);
- DPAA2_PR_PRINT(" rr(0x%02x) ele_id(0x%04x)\r\n",
- ecpri_msg.type4.rr,
- rte_be_to_cpu_16(ecpri_msg.type4.ele_id));
- } else if (ecpri_type == ECPRI_FAFE_TYPE_5) {
- ecpri_msg.type5.msr_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
- ecpri_msg.type5.act_type =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1];
-
- DPAA2_PR_PRINT("rma_id(0x%02x) rw(0x%02x)\r\n",
- ecpri_msg.type5.msr_id,
- ecpri_msg.type5.act_type);
- } else if (ecpri_type == ECPRI_FAFE_TYPE_6) {
- ecpri_msg.type6.rst_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1];
- ecpri_msg.type6.rst_id =
- ecpri_msg.type6.rst_id << 8;
- ecpri_msg.type6.rst_id |=
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
-
- ecpri_msg.type6.rst_op =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 2];
-
- DPAA2_PR_PRINT("rst_id(0x%04x) rst_op(0x%02x)\r\n",
- rte_be_to_cpu_16(ecpri_msg.type6.rst_id),
- ecpri_msg.type6.rst_op);
- } else if (ecpri_type == ECPRI_FAFE_TYPE_7) {
- ecpri_msg.type7.evt_id =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET];
- ecpri_msg.type7.evt_type =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 1];
- ecpri_msg.type7.seq =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 2];
- ecpri_msg.type7.number =
- fapr->pr[DPAA2_ECPRI_MSG_OFFSET + 3];
-
- DPAA2_PR_PRINT("evt_id(0x%02x) evt_type(0x%02x)",
- ecpri_msg.type7.evt_id,
- ecpri_msg.type7.evt_type);
- DPAA2_PR_PRINT(" seq(0x%02x) number(0x%02x)\r\n",
- ecpri_msg.type7.seq,
- ecpri_msg.type7.number);
- }
- }
-}
-
-static inline void
-dpaa2_print_faf(struct dpaa2_fapr_array *fapr)
-{
- const int faf_bit_len = DPAA2_FAF_TOTAL_SIZE * 8;
- struct dpaa2_faf_bit_info faf_bits[faf_bit_len];
- int i, byte_pos, bit_pos, vxlan = 0, vxlan_vlan = 0;
- struct rte_ether_hdr vxlan_in_eth;
- uint16_t vxlan_vlan_tci;
- int ecpri = 0;
-
- for (i = 0; i < faf_bit_len; i++) {
- faf_bits[i].position = i;
- if (i == FAFE_VXLAN_IN_VLAN_FRAM)
- faf_bits[i].name = "VXLAN VLAN Present";
- else if (i == FAFE_VXLAN_IN_IPV4_FRAM)
- faf_bits[i].name = "VXLAN IPV4 Present";
- else if (i == FAFE_VXLAN_IN_IPV6_FRAM)
- faf_bits[i].name = "VXLAN IPV6 Present";
- else if (i == FAFE_VXLAN_IN_UDP_FRAM)
- faf_bits[i].name = "VXLAN UDP Present";
- else if (i == FAFE_VXLAN_IN_TCP_FRAM)
- faf_bits[i].name = "VXLAN TCP Present";
- else if (i == FAF_VXLAN_FRAM)
- faf_bits[i].name = "VXLAN Present";
- else if (i == FAF_ETH_FRAM)
- faf_bits[i].name = "Ethernet MAC Present";
- else if (i == FAF_VLAN_FRAM)
- faf_bits[i].name = "VLAN 1 Present";
- else if (i == FAF_IPV4_FRAM)
- faf_bits[i].name = "IPv4 1 Present";
- else if (i == FAF_IPV6_FRAM)
- faf_bits[i].name = "IPv6 1 Present";
- else if (i == FAF_IP_FRAG_FRAM)
- faf_bits[i].name = "IP fragment Present";
- else if (i == FAF_UDP_FRAM)
- faf_bits[i].name = "UDP Present";
- else if (i == FAF_TCP_FRAM)
- faf_bits[i].name = "TCP Present";
- else if (i == FAFE_ECPRI_FRAM)
- faf_bits[i].name = "ECPRI Present";
- else
- faf_bits[i].name = "Check RM for this unusual frame";
- }
-
- DPAA2_PR_PRINT("Frame Annotation Flags:\r\n");
- for (i = 0; i < faf_bit_len; i++) {
- byte_pos = i / 8 + DPAA2_FAFE_PSR_OFFSET;
- bit_pos = i % 8;
- if (fapr->pr[byte_pos] & (1 << (7 - bit_pos))) {
- DPAA2_PR_PRINT("FAF bit %d : %s\r\n",
- faf_bits[i].position, faf_bits[i].name);
- if (i == FAF_VXLAN_FRAM)
- vxlan = 1;
- else if (i == FAFE_ECPRI_FRAM)
- ecpri = 1;
- }
- }
-
- if (vxlan) {
- vxlan_in_eth.dst_addr.addr_bytes[0] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR0_OFFSET];
- vxlan_in_eth.dst_addr.addr_bytes[1] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR1_OFFSET];
- vxlan_in_eth.dst_addr.addr_bytes[2] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR2_OFFSET];
- vxlan_in_eth.dst_addr.addr_bytes[3] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR3_OFFSET];
- vxlan_in_eth.dst_addr.addr_bytes[4] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR4_OFFSET];
- vxlan_in_eth.dst_addr.addr_bytes[5] =
- fapr->pr[DPAA2_VXLAN_IN_DADDR5_OFFSET];
-
- vxlan_in_eth.src_addr.addr_bytes[0] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR0_OFFSET];
- vxlan_in_eth.src_addr.addr_bytes[1] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR1_OFFSET];
- vxlan_in_eth.src_addr.addr_bytes[2] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR2_OFFSET];
- vxlan_in_eth.src_addr.addr_bytes[3] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR3_OFFSET];
- vxlan_in_eth.src_addr.addr_bytes[4] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR4_OFFSET];
- vxlan_in_eth.src_addr.addr_bytes[5] =
- fapr->pr[DPAA2_VXLAN_IN_SADDR5_OFFSET];
-
- vxlan_in_eth.ether_type =
- fapr->pr[DPAA2_VXLAN_IN_TYPE_OFFSET];
- vxlan_in_eth.ether_type =
- vxlan_in_eth.ether_type << 8;
- vxlan_in_eth.ether_type |=
- fapr->pr[DPAA2_VXLAN_IN_TYPE_OFFSET + 1];
-
- if (vxlan_in_eth.ether_type == RTE_ETHER_TYPE_VLAN)
- vxlan_vlan = 1;
- DPAA2_PR_PRINT("VXLAN inner eth:\r\n");
- DPAA2_PR_PRINT("dst addr: ");
- for (i = 0; i < RTE_ETHER_ADDR_LEN; i++) {
- if (i != 0)
- DPAA2_PR_PRINT(":");
- DPAA2_PR_PRINT("%02x",
- vxlan_in_eth.dst_addr.addr_bytes[i]);
- }
- DPAA2_PR_PRINT("\r\n");
- DPAA2_PR_PRINT("src addr: ");
- for (i = 0; i < RTE_ETHER_ADDR_LEN; i++) {
- if (i != 0)
- DPAA2_PR_PRINT(":");
- DPAA2_PR_PRINT("%02x",
- vxlan_in_eth.src_addr.addr_bytes[i]);
- }
- DPAA2_PR_PRINT("\r\n");
- DPAA2_PR_PRINT("type: 0x%04x\r\n",
- vxlan_in_eth.ether_type);
- if (vxlan_vlan) {
- vxlan_vlan_tci = fapr->pr[DPAA2_VXLAN_IN_TCI_OFFSET];
- vxlan_vlan_tci = vxlan_vlan_tci << 8;
- vxlan_vlan_tci |=
- fapr->pr[DPAA2_VXLAN_IN_TCI_OFFSET + 1];
-
- DPAA2_PR_PRINT("vlan tci: 0x%04x\r\n",
- vxlan_vlan_tci);
- }
- }
-
- if (ecpri)
- dpaa2_print_ecpri(fapr);
-}
-
-static inline void
-dpaa2_print_parse_result(struct dpaa2_annot_hdr *annotation)
-{
- struct dpaa2_fapr_array fapr;
- struct dpaa2_fapr_field_info
- fapr_fields[sizeof(support_dump_fields) /
- sizeof(struct dpaa2_fapr_field_info)];
- uint64_t len, i;
-
- memcpy(&fapr, &annotation->word3, DPAA2_FAPR_SIZE);
- for (i = 0; i < (DPAA2_FAPR_SIZE / 8); i++)
- fapr.pr_64[i] = rte_cpu_to_be_64(fapr.pr_64[i]);
-
- memcpy(fapr_fields, support_dump_fields,
- sizeof(support_dump_fields));
-
- for (i = 0;
- i < sizeof(fapr_fields) /
- sizeof(struct dpaa2_fapr_field_info);
- i++) {
- if (!strcmp(fapr_fields[i].name, NEXT_HEADER_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_NXTHDR_OFFSET];
- fapr_fields[i].value = fapr_fields[i].value << 8;
- fapr_fields[i].value |=
- fapr.pr[DPAA2_PR_NXTHDR_OFFSET + 1];
- } else if (!strcmp(fapr_fields[i].name, ETH_OFF_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_ETH_OFF_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, VLAN_TCI_OFF_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_TCI_OFF_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, LAST_ENTRY_OFF_NAME)) {
- fapr_fields[i].value =
- fapr.pr[DPAA2_PR_LAST_ETYPE_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, L3_OFF_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_L3_OFF_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, L4_OFF_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_L4_OFF_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, L5_OFF_NAME)) {
- fapr_fields[i].value = fapr.pr[DPAA2_PR_L5_OFF_OFFSET];
- } else if (!strcmp(fapr_fields[i].name, NEXT_HEADER_OFF_NAME)) {
- fapr_fields[i].value =
- fapr.pr[DPAA2_PR_NXTHDR_OFF_OFFSET];
- }
- }
-
- len = sizeof(fapr_fields) / sizeof(struct dpaa2_fapr_field_info);
- DPAA2_PR_PRINT("Parse Result:\r\n");
- for (i = 0; i < len; i++) {
- DPAA2_PR_PRINT("%21s : 0x%02x\r\n",
- fapr_fields[i].name, fapr_fields[i].value);
- }
- dpaa2_print_faf(&fapr);
-}
-
-#endif
diff --git a/drivers/net/dpaa2/dpaa2_parser_decode.h b/drivers/net/dpaa2/dpaa2_parser_decode.h
new file mode 100644
index 0000000000..f180955b15
--- /dev/null
+++ b/drivers/net/dpaa2/dpaa2_parser_decode.h
@@ -0,0 +1,1539 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright 2022-2026 NXP
+ *
+ */
+
+#ifndef _DPAA2_PARSE_DECODE_H
+#define _DPAA2_PARSE_DECODE_H
+
+#include <rte_flow.h>
+#include <rte_event_eth_rx_adapter.h>
+#include <rte_pmd_dpaa2.h>
+
+#include <bus_fslmc_driver.h>
+#include <dpaa2_hw_pvt.h>
+#include "dpaa2_tm.h"
+
+#include <mc/fsl_dpni.h>
+#include <mc/fsl_mc_sys.h>
+
+#include "base/dpaa2_hw_dpni_annot.h"
+#include "dpaa2_pmd_logs.h"
+
+#define DPAA2_PR_PRINT dpaa2_dump_print
+
+#define BIT_SIZE(t) (sizeof(t) * 8)
+#define BIT_OFFSET(t, m) (offsetof(t, m) * 8)
+#define BIT_LAST_OFFSET(t) (BIT_SIZE(t) - 1)
+
+#define ROCEV2_DEST_QP_SIZE 3
+
+enum dpaa2_parser_protocol_id {
+ DPAA2_PARSER_MAC_ID,
+ DPAA2_PARSER_VLAN_ID,
+ DPAA2_PARSER_ICMP_ID,
+ DPAA2_PARSER_IP_ID,
+ DPAA2_PARSER_IPV4_ID,
+ DPAA2_PARSER_IPV6_ID,
+ DPAA2_PARSER_IP_FRAG_ID,
+ DPAA2_PARSER_GRE_ID,
+ DPAA2_PARSER_UDP_ID,
+ DPAA2_PARSER_TCP_ID,
+ DPAA2_PARSER_SCTP_ID,
+ DPAA2_PARSER_GTP_ID,
+ DPAA2_PARSER_IPSEC_AH_ID,
+ DPAA2_PARSER_IPSEC_ESP_ID,
+ DPAA2_PARSER_VXLAN_ID,
+ DPAA2_PARSER_TUNNEL_VLAN_ID,
+ DPAA2_PARSER_TUNNEL_IPV4_ID,
+ DPAA2_PARSER_TUNNEL_IPV6_ID,
+ DPAA2_PARSER_TUNNEL_UDP_ID,
+ DPAA2_PARSER_TUNNEL_TCP_ID,
+ DPAA2_PARSER_ECPRI_ID,
+ DPAA2_PARSER_ROCEV2_ID,
+ DPAA2_PARSER_GENEVE_ID
+};
+
+#define DPAA2_PSR_SUMMARY_NONIP 6
+#define DPAA2_PSR_SUMMARY_ARP 3
+#define DPAA2_PSR_SUMMARY_NONIP_PTP 4
+#define DPAA2_PSR_SUMMARY_MPLS 6
+
+#define DPAA2_PSR_SUMMARY_L4_EXT 2
+#define DPAA2_PSR_SUMMARY_IPV4 0
+#define DPAA2_PSR_SUMMARY_IPV6 1
+
+#define DPAA2_PSR_SUMMARY_GRE_IPV4 0
+#define DPAA2_PSR_SUMMARY_GRE_IPV4_UDP_TCP 1
+#define DPAA2_PSR_SUMMARY_GRE_IPV6 2
+#define DPAA2_PSR_SUMMARY_GRE_IPV6_UDP_TCP 3
+
+#define DPAA2_PSR_SUMMARY_IP_FRAG 1
+#define DPAA2_PSR_SUMMARY_ICMP 3
+#define DPAA2_PSR_SUMMARY_IPSEC_ESP 5
+#define DPAA2_PSR_SUMMARY_TCP 14
+#define DPAA2_PSR_SUMMARY_SCTP 15
+#define DPAA2_PSR_SUMMARY_UDP 16
+#define DPAA2_PSR_SUMMARY_PTP 18
+#define DPAA2_PSR_SUMMARY_UDP_ESP_IKE 19
+#define DPAA2_PSR_SUMMARY_GTPU 20
+#define DPAA2_PSR_SUMMARY_GTPC 21
+#define DPAA2_PSR_SUMMARY_VXLAN 27
+
+#define DPAA2_PSR_SUMMARY_L4_BIT_SIZE 5
+#define DPAA2_PSR_SUMMARY_L3_BIT_SIZE 2
+
+#define DPAA2_PSR_SUMMARY_L4_EXT_BIT_SIZE 4
+#define DPAA2_PSR_SUMMARY_L4_EXT_IP_BIT_SIZE 1
+#define DPAA2_PSR_SUMMARY_L4_EXT_L3_BIT_SIZE 2
+#define DPAA2_PSR_SUMMARY_L4_EXT_L3_POS \
+ (DPAA2_PSR_SUMMARY_L4_EXT_BIT_SIZE + \
+ DPAA2_PSR_SUMMARY_L4_EXT_IP_BIT_SIZE)
+
+#define DPAA2_PSR_SUMMARY_NON_IP_L2_BIT_SIZE 4
+#define DPAA2_PSR_SUMMARY_NON_IP_NON_IP_BIT_SIZE 3
+
+union __rte_packed_begin dpaa2_psr_summary_l {
+ struct {
+ uint8_t l4:DPAA2_PSR_SUMMARY_L4_BIT_SIZE;
+ uint8_t l3:DPAA2_PSR_SUMMARY_L3_BIT_SIZE;
+ uint8_t fafe3:1;
+ } l4;
+ struct {
+ uint8_t l4_ext:DPAA2_PSR_SUMMARY_L4_EXT_BIT_SIZE;
+ uint8_t ip:DPAA2_PSR_SUMMARY_L4_EXT_IP_BIT_SIZE;
+ uint8_t l3:DPAA2_PSR_SUMMARY_L4_EXT_L3_BIT_SIZE;
+ uint8_t fafe3:1;
+ } l4_ext;
+ struct {
+ uint8_t l2:DPAA2_PSR_SUMMARY_NON_IP_L2_BIT_SIZE;
+ uint8_t non_ip:DPAA2_PSR_SUMMARY_NON_IP_NON_IP_BIT_SIZE;
+ uint8_t fafe3:1;
+ } non_ip;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_psr_summary {
+ union dpaa2_psr_summary_l sum_l;
+ uint8_t vlan:2;
+ uint8_t l2_l2_ext:2;
+ uint8_t fafe2:1;
+ uint8_t fafe1:1;
+ uint8_t fafe0:1;
+ uint8_t checksum_err:1;
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_psr_sum_16b {
+ uint16_t sum_16b;
+ struct dpaa2_psr_summary sum;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_ingress_frc {
+ uint32_t rsv0:10;
+
+ uint32_t faicfdv:1;
+ uint32_t faswov:1;
+ uint32_t faiadv:1;
+ uint32_t faprv:1;
+ uint32_t faeadv:1;
+ uint32_t fasv:1;
+ union dpaa2_psr_sum_16b psr_sum;
+} __rte_packed_end;
+
+/* Frame annotation status */
+struct __rte_packed_begin dpaa2_fas_parse {
+ uint32_t status:8;
+ uint32_t ppid:6;
+ uint32_t rsv4:2;
+ uint32_t ifpid:12;
+ uint32_t rsv3:4;
+ uint32_t l4ce:1;
+ uint32_t l4cv:1;
+ uint32_t l3ce:1;
+ uint32_t l3cv:1;
+ uint32_t ble:1;
+ uint32_t phe:1;
+ uint32_t isp:1;
+ uint32_t pte:1;
+ uint32_t rsv2:4;
+ uint32_t fpe:1;
+ uint32_t fle:1;
+ uint32_t piee:1;
+ uint32_t tide:1;
+ uint32_t mnle:1;
+ uint32_t eofhe:1;
+ uint32_t kse:1;
+ uint32_t rsv1:6;
+ uint32_t broadcast:1;
+ uint32_t multicast:1;
+ uint32_t ptp:1;
+ uint32_t rsv0:2;
+ uint32_t macsec:1;
+ uint32_t discard:1;
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_fas_parse_32b {
+ uint32_t fas_32b;
+ struct dpaa2_fas_parse fas;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_sp_fafe_ecpri {
+ uint8_t msg_type:5;
+ uint8_t non_used:2;
+ uint8_t ecpri:1; /**Always 1, FRC FAFE0*/
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_sp_fafe_rocev2_tunnel {
+ uint8_t in_vlan:1;
+ uint8_t in_udp:1;
+ uint8_t in_tcp:1;
+ uint8_t in_ipv6:1;
+ uint8_t in_ipv4:1; /**FRC FAFE3*/
+ uint8_t geneve:1; /**FRC FAFE2*/
+ uint8_t rocev2:1; /**FRC FAFE1*/
+ uint8_t non_used:1;
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_sp_fafe_parse {
+ struct dpaa2_sp_fafe_ecpri ecpri;
+ struct dpaa2_sp_fafe_rocev2_tunnel rocev2_tunnel;
+ uint8_t fafe_8b;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_faf_l_parse {
+ uint32_t arp_err:1;
+ uint32_t arp:1;
+ uint32_t mpls_err:1;
+ uint32_t mpls_n:1;
+ uint32_t mpls_1:1;
+ uint32_t pppoe_ppp_err:1;
+ uint32_t pppoe_ppp:1;
+ uint32_t vlan_err:1;
+
+ uint32_t gre_eth:1;
+ uint32_t vlan_n:1;
+ uint32_t vlan_1:1;
+ uint32_t llc_snap_err:1;
+ uint32_t unknown_llc_oui:1;
+ uint32_t llc_snap:1;
+ uint32_t eth_err:1;
+ uint32_t fip:1;
+
+ uint32_t fcoe:1;
+ uint32_t bpdu:1;
+ uint32_t broadcast:1;
+ uint32_t multicast:1;
+ uint32_t unicast:1;
+ uint32_t mac:1;
+ uint32_t psr_err:1;
+ uint32_t spsr_err:1;
+
+ uint32_t ike:1;
+ uint32_t eth_slow:1;
+ uint32_t vxlan_err:1;
+ uint32_t vxlan:1;
+ uint32_t ptp:1;
+ uint32_t vlan_prio:1;
+ uint32_t gtp_prim:1;
+ uint32_t i6_r_hdr2:1;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_faf_l_parse_be {
+ uint32_t ike:1;
+ uint32_t eth_slow:1;
+ uint32_t vxlan_err:1;
+ uint32_t vxlan:1;
+ uint32_t ptp:1;
+ uint32_t vlan_prio:1;
+ uint32_t gtp_prim:1;
+ uint32_t i6_r_hdr2:1;
+
+ uint32_t fcoe:1;
+ uint32_t bpdu:1;
+ uint32_t broadcast:1;
+ uint32_t multicast:1;
+ uint32_t unicast:1;
+ uint32_t mac:1;
+ uint32_t psr_err:1;
+ uint32_t spsr_err:1;
+
+ uint32_t gre_eth:1;
+ uint32_t vlan_n:1;
+ uint32_t vlan_1:1;
+ uint32_t llc_snap_err:1;
+ uint32_t unknown_llc_oui:1;
+ uint32_t llc_snap:1;
+ uint32_t eth_err:1;
+ uint32_t fip:1;
+
+ uint32_t arp_err:1;
+ uint32_t arp:1;
+ uint32_t mpls_err:1;
+ uint32_t mpls_n:1;
+ uint32_t mpls_1:1;
+ uint32_t pppoe_ppp_err:1;
+ uint32_t pppoe_ppp:1;
+ uint32_t vlan_err:1;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_faf_h_parse {
+ uint32_t i6_r_hdr1:1;
+ uint32_t l5_spsr:1;
+ uint32_t capwap_data:1;
+ uint32_t capwap_ctl:1;
+ uint32_t iscsi:1;
+ uint32_t esp_err:1;
+ uint32_t esp:1;
+ uint32_t gtp_err:1;
+
+ uint32_t gtp:1;
+ uint32_t l4_spsr_err:1;
+ uint32_t l4_unknown:1;
+ uint32_t dccp_err:1;
+ uint32_t dccp:1;
+ uint32_t sctp_err:1;
+ uint32_t sctp:1;
+ uint32_t ipsec_err:1;
+
+ uint32_t ipsec_ah:1;
+ uint32_t ipsec_esp:1;
+ uint32_t ipsec:1;
+ uint32_t tcp_err:1;
+ uint32_t tcp_3_5:1;
+ uint32_t tcp_6_11:1;
+ uint32_t tcp_opt:1;
+ uint32_t tcp:1;
+
+ uint32_t udp_err:1;
+ uint32_t udp:1;
+ uint32_t l3_spsr_err:1;
+ uint32_t l3_unknown:1;
+ uint32_t gre_err:1;
+ uint32_t gre_k:1;
+ uint32_t gre:1;
+ uint32_t min_encp_err:1;
+
+ uint32_t min_encp_s:1;
+ uint32_t min_encp:1;
+ uint32_t ip_n_err:1;
+ uint32_t udp_light:1;
+ uint32_t icmp_v6:1;
+ uint32_t igmp:1;
+ uint32_t icmp:1;
+ uint32_t ip_n_init_frag:1;
+
+ uint32_t ip_n_frag:1;
+ uint32_t ip_n_unknown:1;
+ uint32_t ip_n_opt:1;
+ uint32_t ip_1_err:1;
+ uint32_t ip_1_init_frag:1;
+ uint32_t ip_1_frag:1;
+ uint32_t ip_1_unknown:1;
+ uint32_t ip_1_opt:1;
+
+ uint32_t ipv6_n_multicast:1;
+ uint32_t ipv6_n_unicast:1;
+ uint32_t ipv6_n:1;
+ uint32_t ipv6_1_multicast:1;
+ uint32_t ipv6_1_unicast:1;
+ uint32_t ipv6_1:1;
+ uint32_t ipv4_n_broadcast:1;
+ uint32_t ipv4_n_multicast:1;
+
+ uint32_t ipv4_n_unicast:1;
+ uint32_t ipv4_n:1;
+ uint32_t ipv4_1_broadcast:1;
+ uint32_t ipv4_1_multicast:1;
+ uint32_t ipv4_1_unicast:1;
+ uint32_t ipv4_1:1;
+ uint32_t l2_spsr_err:1;
+ uint32_t l2_unknown:1;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_faf_h_parse_be {
+ uint32_t ipv4_n_unicast:1;
+ uint32_t ipv4_n:1;
+ uint32_t ipv4_1_broadcast:1;
+ uint32_t ipv4_1_multicast:1;
+ uint32_t ipv4_1_unicast:1;
+ uint32_t ipv4_1:1;
+ uint32_t l2_spsr_err:1;
+ uint32_t l2_unknown:1;
+
+ uint32_t ipv6_n_multicast:1;
+ uint32_t ipv6_n_unicast:1;
+ uint32_t ipv6_n:1;
+ uint32_t ipv6_1_multicast:1;
+ uint32_t ipv6_1_unicast:1;
+ uint32_t ipv6_1:1;
+ uint32_t ipv4_n_broadcast:1;
+ uint32_t ipv4_n_multicast:1;
+
+ uint32_t ip_n_frag:1;
+ uint32_t ip_n_unknown:1;
+ uint32_t ip_n_opt:1;
+ uint32_t ip_1_err:1;
+ uint32_t ip_1_init_frag:1;
+ uint32_t ip_1_frag:1;
+ uint32_t ip_1_unknown:1;
+ uint32_t ip_1_opt:1;
+
+ uint32_t min_encp_s:1;
+ uint32_t min_encp:1;
+ uint32_t ip_n_err:1;
+ uint32_t udp_light:1;
+ uint32_t icmp_v6:1;
+ uint32_t igmp:1;
+ uint32_t icmp:1;
+ uint32_t ip_n_init_frag:1;
+
+ uint32_t udp_err:1;
+ uint32_t udp:1;
+ uint32_t l3_spsr_err:1;
+ uint32_t l3_unknown:1;
+ uint32_t gre_err:1;
+ uint32_t gre_k:1;
+ uint32_t gre:1;
+ uint32_t min_encp_err:1;
+
+ uint32_t ipsec_ah:1;
+ uint32_t ipsec_esp:1;
+ uint32_t ipsec:1;
+ uint32_t tcp_err:1;
+ uint32_t tcp_3_5:1;
+ uint32_t tcp_6_11:1;
+ uint32_t tcp_opt:1;
+ uint32_t tcp:1;
+
+ uint32_t gtp:1;
+ uint32_t l4_spsr_err:1;
+ uint32_t l4_unknown:1;
+ uint32_t dccp_err:1;
+ uint32_t dccp:1;
+ uint32_t sctp_err:1;
+ uint32_t sctp:1;
+ uint32_t ipsec_err:1;
+
+ uint32_t i6_r_hdr1:1;
+ uint32_t l5_spsr:1;
+ uint32_t capwap_data:1;
+ uint32_t capwap_ctl:1;
+ uint32_t iscsi:1;
+ uint32_t esp_err:1;
+ uint32_t esp:1;
+ uint32_t gtp_err:1;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_annot_word3_parse {
+ struct dpaa2_faf_l_parse faf_l;
+ uint8_t rsv;
+ union dpaa2_sp_fafe_parse fafe;
+ rte_be16_t nxthdr;
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_ecpri_msg_rm_access {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+ uint32_t ele_id:16; /**< Element ID */
+ uint32_t rr:4; /**< Req/Resp */
+ uint32_t rw:4; /**< Read/Write */
+ uint32_t rma_id:8; /**< Remote Memory Access ID */
+#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
+ uint32_t rma_id:8; /**< Remote Memory Access ID */
+ uint32_t rw:4; /**< Read/Write */
+ uint32_t rr:4; /**< Req/Resp */
+ uint32_t ele_id:16; /**< Element ID */
+#endif
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_sp_ecpri_msg {
+ struct rte_ecpri_msg_iq_data type0;
+ struct rte_ecpri_msg_bit_seq type1;
+ struct rte_ecpri_msg_rtc_ctrl type2;
+ struct rte_ecpri_msg_bit_seq type3;
+ struct dpaa2_ecpri_msg_rm_access type4;
+ struct rte_ecpri_msg_delay_measure type5;
+ struct rte_ecpri_msg_remote_reset type6;
+ struct rte_ecpri_msg_event_ind type7;
+} __rte_packed_end;
+
+#define DPAA2_SP_PSR_CTX_LEN 5
+
+struct __rte_packed_begin dpaa2_sp_rocev2 {
+ uint8_t opcode;
+ uint8_t dest_qp[ROCEV2_DEST_QP_SIZE];
+} __rte_packed_end;
+
+struct __rte_packed_begin dpaa2_sp_geneve {
+ rte_be16_t protocol;
+ uint8_t geneve_vni[3];
+} __rte_packed_end;
+
+struct dpaa2_psr_result_word3 {
+ rte_be16_t nxthdr;
+ union dpaa2_sp_fafe_parse fafe;
+ uint8_t rsv;
+ struct dpaa2_faf_l_parse_be faf_l_be;
+};
+
+struct dpaa2_psr_result_word4 {
+ struct dpaa2_faf_h_parse_be faf_h_be;
+};
+
+struct dpaa2_psr_result_word5 {
+ union {
+ struct {
+ uint8_t shim_off1;
+ uint8_t shim_off2;
+ };
+ rte_be16_t vxlan_vlan_tci;
+ };
+ uint8_t ip_1_pid_off;
+ uint8_t eth_off;
+ union {
+ uint8_t llc_snap_off;
+ uint8_t vxlan_in_daddr0;
+ };
+ uint8_t vlan_tci_1_off;
+ union {
+ uint8_t vlan_tci_n_off;
+ uint8_t vxlan_in_daddr1;
+ };
+ uint8_t last_etype_off;
+};
+
+struct dpaa2_psr_result_word6 {
+ union {
+ uint8_t pppoe_off;
+ uint8_t vxlan_in_daddr2;
+ };
+ union {
+ uint8_t mpls_1_off;
+ uint8_t vxlan_in_daddr3;
+ };
+ union {
+ uint8_t mpls_n_off;
+ uint8_t vxlan_in_daddr4;
+ };
+ uint8_t l3_off;
+ union {
+ uint8_t ip_n_off;
+ uint8_t vxlan_in_daddr5;
+ };
+ union {
+ uint8_t gre_off;
+ uint8_t vxlan_in_saddr0;
+ };
+ uint8_t l4_off;
+ uint8_t l5_off;
+};
+
+struct dpaa2_psr_result_word7 {
+ union {
+ uint8_t route_hdr1_off;
+ uint8_t vxlan_in_saddr1;
+ };
+ union {
+ uint8_t route_hdr2_off;
+ uint8_t vxlan_in_saddr2;
+ };
+ uint8_t nxthdr_off;
+ union {
+ uint8_t ipv6_frag_off;
+ uint8_t vxlan_in_saddr3;
+ };
+ rte_be16_t gross_running_sum;
+ rte_be16_t running_sum;
+};
+
+struct dpaa2_psr_result_word8 {
+ uint8_t psr_err;
+ union {
+ uint8_t nxthdr_frag_off;
+ uint8_t vxlan_in_saddr4;
+ };
+ union {
+ uint8_t ip_n_pid_off;
+ uint8_t vxlan_in_saddr5;
+ };
+ union __rte_packed_begin {
+ uint8_t sp_psr_ctx[DPAA2_SP_PSR_CTX_LEN];
+ struct __rte_packed_begin {
+ uint8_t vxlan_vni[3];
+ rte_be16_t vxlan_eth_type;
+ } __rte_packed_end;
+ union dpaa2_sp_ecpri_msg ecpri_msg;
+ struct dpaa2_sp_rocev2 rocev2;
+ struct dpaa2_sp_geneve geneve;
+ } __rte_packed_end;
+};
+
+struct __rte_packed_begin dpaa2_psr_result_parse {
+ struct dpaa2_psr_result_word3 word3;
+ struct dpaa2_psr_result_word4 word4;
+ struct dpaa2_psr_result_word5 word5;
+ struct dpaa2_psr_result_word6 word6;
+ struct dpaa2_psr_result_word7 word7;
+ struct dpaa2_psr_result_word8 word8;
+} __rte_packed_end;
+
+#define DPAA2_PSR_RESULT_SIZE sizeof(struct dpaa2_psr_result_parse)
+
+#define DPAA2_FAFE_PSR_RESULT_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word3.fafe)
+
+union __rte_packed_begin dpaa2_sp_fafe_parse_8b {
+ uint8_t fafe_8b;
+ union dpaa2_sp_fafe_parse fafe;
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_faf_l_parse_32b {
+ uint32_t faf_l_32b;
+ struct dpaa2_faf_l_parse faf_l;
+} __rte_packed_end;
+
+union __rte_packed_begin dpaa2_faf_h_parse_64b {
+ uint64_t faf_h_64b;
+ struct dpaa2_faf_h_parse faf_h;
+} __rte_packed_end;
+
+/* Set by SP for vxlan distribution start*/
+#define DPAA2_VXLAN_IN_TCI_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word5.vxlan_vlan_tci)
+#define DPAA2_VXLAN_IN_DADDR0_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word5.vxlan_in_daddr0)
+#define DPAA2_VXLAN_IN_DADDR1_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word5.vxlan_in_daddr1)
+#define DPAA2_VXLAN_IN_DADDR2_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word6.vxlan_in_daddr2)
+#define DPAA2_VXLAN_IN_DADDR3_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word6.vxlan_in_daddr3)
+#define DPAA2_VXLAN_IN_DADDR4_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word6.vxlan_in_daddr4)
+#define DPAA2_VXLAN_IN_DADDR5_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word6.vxlan_in_daddr5)
+
+#define DPAA2_VXLAN_IN_SADDR0_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word6.vxlan_in_saddr0)
+#define DPAA2_VXLAN_IN_SADDR1_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word7.vxlan_in_saddr1)
+#define DPAA2_VXLAN_IN_SADDR2_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word7.vxlan_in_saddr2)
+#define DPAA2_VXLAN_IN_SADDR3_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word7.vxlan_in_saddr3)
+#define DPAA2_VXLAN_IN_SADDR4_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.vxlan_in_saddr4)
+#define DPAA2_VXLAN_IN_SADDR5_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.vxlan_in_saddr5)
+
+#define DPAA2_VXLAN_VNI_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.vxlan_vni[0])
+#define DPAA2_VXLAN_IN_TYPE_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.vxlan_eth_type)
+/* Set by SP for vxlan distribution end*/
+
+#define DPAA2_ECPRI_MSG_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.ecpri_msg)
+
+#define DPAA2_ROCEV2_OPCODE_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.rocev2.opcode)
+
+#define DPAA2_ROCEV2_DST_QP_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.rocev2.dest_qp[0])
+
+#define DPAA2_GENEVE_PROTOCOL_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.geneve.protocol)
+
+#define DPAA2_GENEVE_VNI_OFFSET \
+ offsetof(struct dpaa2_psr_result_parse, word8.geneve.geneve_vni[0])
+
+static inline uint32_t
+dpaa2_spsr_fafe_bit_offset(union dpaa2_sp_fafe_parse fafe)
+{
+ union dpaa2_sp_fafe_parse_8b fafe_tmp;
+ uint32_t offset = BIT_LAST_OFFSET(union dpaa2_sp_fafe_parse_8b);
+
+ memset(&fafe_tmp, 0, sizeof(union dpaa2_sp_fafe_parse_8b));
+ fafe_tmp.fafe_8b = 1;
+ while (memcmp(&fafe_tmp.fafe, &fafe,
+ sizeof(union dpaa2_sp_fafe_parse))) {
+ fafe_tmp.fafe_8b = fafe_tmp.fafe_8b << 1;
+ offset--;
+ }
+ return offset + BIT_OFFSET(struct dpaa2_psr_result_parse, word3.fafe);
+}
+
+static inline uint32_t
+dpaa2_psr_faf_l_bit_offset(struct dpaa2_faf_l_parse *faf_l)
+{
+ union dpaa2_faf_l_parse_32b faf_l_tmp;
+ uint32_t offset = BIT_LAST_OFFSET(union dpaa2_faf_l_parse_32b);
+
+ memset(&faf_l_tmp, 0, sizeof(union dpaa2_faf_l_parse_32b));
+ faf_l_tmp.faf_l_32b = 1;
+ while (memcmp(&faf_l_tmp.faf_l, faf_l,
+ sizeof(struct dpaa2_faf_l_parse))) {
+ faf_l_tmp.faf_l_32b = faf_l_tmp.faf_l_32b << 1;
+ offset--;
+ }
+ return offset + BIT_OFFSET(struct dpaa2_psr_result_parse,
+ word3.faf_l_be);
+}
+
+static inline uint32_t
+dpaa2_psr_faf_h_bit_offset(struct dpaa2_faf_h_parse *faf_h)
+{
+ union dpaa2_faf_h_parse_64b faf_h_tmp;
+ uint32_t offset = BIT_LAST_OFFSET(union dpaa2_faf_h_parse_64b);
+
+ memset(&faf_h_tmp, 0, sizeof(union dpaa2_faf_h_parse_64b));
+ faf_h_tmp.faf_h_64b = 1;
+ while (memcmp(&faf_h_tmp.faf_h, faf_h,
+ sizeof(struct dpaa2_faf_h_parse))) {
+ faf_h_tmp.faf_h_64b = faf_h_tmp.faf_h_64b << 1;
+ offset--;
+ }
+ return offset + BIT_OFFSET(struct dpaa2_psr_result_parse,
+ word4.faf_h_be);
+}
+
+static inline int
+dpaa2_protocol_psr_bit_offset(uint32_t *bit_offset,
+ enum dpaa2_parser_protocol_id protocol)
+{
+ struct dpaa2_faf_l_parse faf_l;
+ struct dpaa2_faf_h_parse faf_h;
+ union dpaa2_sp_fafe_parse fafe;
+
+ memset(&faf_l, 0, sizeof(faf_l));
+ memset(&faf_h, 0, sizeof(faf_h));
+ memset(&fafe, 0, sizeof(fafe));
+
+ if (protocol == DPAA2_PARSER_MAC_ID) {
+ faf_l.mac = 1;
+ *bit_offset = dpaa2_psr_faf_l_bit_offset(&faf_l);
+ } else if (protocol == DPAA2_PARSER_VLAN_ID) {
+ faf_l.vlan_1 = 1;
+ *bit_offset = dpaa2_psr_faf_l_bit_offset(&faf_l);
+ } else if (protocol == DPAA2_PARSER_ICMP_ID) {
+ faf_h.icmp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IP_ID) {
+ faf_h.ip_1_opt = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IPV4_ID) {
+ faf_h.ipv4_1 = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IP_FRAG_ID) {
+ faf_h.ip_1_frag = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IPV6_ID) {
+ faf_h.ipv6_1 = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_GRE_ID) {
+ faf_h.gre = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_UDP_ID) {
+ faf_h.udp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_TCP_ID) {
+ faf_h.tcp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_SCTP_ID) {
+ faf_h.sctp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_GTP_ID) {
+ faf_h.gtp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IPSEC_ESP_ID) {
+ faf_h.ipsec_esp = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_IPSEC_AH_ID) {
+ faf_h.ipsec_ah = 1;
+ *bit_offset = dpaa2_psr_faf_h_bit_offset(&faf_h);
+ } else if (protocol == DPAA2_PARSER_VXLAN_ID) {
+ faf_l.vxlan = 1;
+ *bit_offset = dpaa2_psr_faf_l_bit_offset(&faf_l);
+ } else if (protocol == DPAA2_PARSER_TUNNEL_VLAN_ID) {
+ fafe.rocev2_tunnel.in_vlan = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_TUNNEL_IPV4_ID) {
+ fafe.rocev2_tunnel.in_ipv4 = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_TUNNEL_IPV6_ID) {
+ fafe.rocev2_tunnel.in_ipv6 = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_TUNNEL_UDP_ID) {
+ fafe.rocev2_tunnel.in_udp = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_TUNNEL_TCP_ID) {
+ fafe.rocev2_tunnel.in_tcp = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_ECPRI_ID) {
+ fafe.ecpri.ecpri = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_ROCEV2_ID) {
+ fafe.rocev2_tunnel.rocev2 = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else if (protocol == DPAA2_PARSER_GENEVE_ID) {
+ fafe.rocev2_tunnel.geneve = 1;
+ *bit_offset = dpaa2_spsr_fafe_bit_offset(fafe);
+ } else {
+ DPAA2_PMD_ERR("Unsupported parser protocol(%d)", protocol);
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+static inline uint32_t __rte_hot
+dpaa2_dev_rx_parse_frc(const struct qbman_fd *fd,
+ struct rte_mbuf *m,
+ const struct dpaa2_annot_hdr *annotation)
+{
+ const struct dpaa2_ingress_frc *frc = (const void *)&fd->simple.frc;
+ const struct dpaa2_psr_summary *sum = &frc->psr_sum.sum;
+ const union dpaa2_psr_summary_l *sum_l = &sum->sum_l;
+ int ip4_6 = 0, vxlan = 0;
+ struct dpaa2_annot_hdr annot_convert;
+ const struct dpaa2_psr_result_parse *psr;
+ const struct dpaa2_fas_parse *fas;
+ uint32_t packet_type = RTE_PTYPE_L4_NONFRAG;
+ uint8_t vlan_offset;
+ rte_be16_t *vlan_tci;
+
+ /**Access annotation only for vlan and checksum error.*/
+
+ if (sum->vlan) {
+ annot_convert.word5 = annotation->word5;
+ annot_convert.word5 = rte_cpu_to_be_64(annot_convert.word5);
+ psr = (const void *)&annot_convert.word3;
+ packet_type |= RTE_PTYPE_L2_ETHER_VLAN;
+ m->ol_flags |= RTE_MBUF_F_RX_VLAN;
+ if (sum->vlan > 1) {
+ vlan_offset = psr->word5.vlan_tci_n_off;
+ packet_type |= RTE_PTYPE_L2_ETHER_QINQ;
+ m->ol_flags |= RTE_MBUF_F_RX_QINQ;
+ } else {
+ vlan_offset = psr->word5.vlan_tci_1_off;
+ }
+ vlan_tci = rte_pktmbuf_mtod_offset(m, void *, vlan_offset);
+ m->vlan_tci = rte_be_to_cpu_16(*vlan_tci);
+ }
+ if (sum_l->l4.l3 == DPAA2_PSR_SUMMARY_IPV4) {
+ packet_type |= RTE_PTYPE_L3_IPV4;
+ ip4_6 = 1;
+ } else if (sum_l->l4.l3 == DPAA2_PSR_SUMMARY_IPV6) {
+ packet_type |= RTE_PTYPE_L3_IPV6;
+ ip4_6 = 1;
+ }
+
+ if (ip4_6) {
+ if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_IP_FRAG) {
+ packet_type &= ~RTE_PTYPE_L4_NONFRAG;
+ packet_type |= RTE_PTYPE_L4_FRAG;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_ICMP) {
+ packet_type |= RTE_PTYPE_L4_ICMP;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_IPSEC_ESP) {
+ packet_type |= RTE_PTYPE_TUNNEL_ESP;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_TCP) {
+ packet_type |= RTE_PTYPE_L4_TCP;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_UDP) {
+ packet_type |= RTE_PTYPE_L4_UDP;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_SCTP) {
+ packet_type |= RTE_PTYPE_L4_SCTP;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_GTPU) {
+ packet_type |= RTE_PTYPE_TUNNEL_GTPU;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_GTPC) {
+ packet_type |= RTE_PTYPE_TUNNEL_GTPC;
+ } else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_VXLAN) {
+ packet_type |= RTE_PTYPE_TUNNEL_VXLAN;
+ vxlan = 1;
+ }
+ } else if (sum_l->l4_ext.l3 == DPAA2_PSR_SUMMARY_L4_EXT) {
+ if (sum_l->l4_ext.ip == DPAA2_PSR_SUMMARY_IPV4)
+ packet_type |= RTE_PTYPE_L3_IPV4;
+ else if (sum_l->l4_ext.ip == DPAA2_PSR_SUMMARY_IPV6)
+ packet_type |= RTE_PTYPE_L3_IPV6;
+ if (sum_l->l4_ext.l4_ext == DPAA2_PSR_SUMMARY_GRE_IPV4 ||
+ sum_l->l4_ext.l4_ext == DPAA2_PSR_SUMMARY_GRE_IPV6)
+ packet_type |= RTE_PTYPE_TUNNEL_GRE;
+ else if (sum_l->l4_ext.l4_ext ==
+ DPAA2_PSR_SUMMARY_GRE_IPV4_UDP_TCP ||
+ sum_l->l4_ext.l4_ext ==
+ DPAA2_PSR_SUMMARY_GRE_IPV6_UDP_TCP)
+ packet_type |= RTE_PTYPE_TUNNEL_NVGRE;
+ } else if (sum_l->non_ip.non_ip == DPAA2_PSR_SUMMARY_NONIP) {
+ if (sum_l->non_ip.l2 == DPAA2_PSR_SUMMARY_ARP) {
+ packet_type |= RTE_PTYPE_L2_ETHER_ARP;
+ } else if (sum_l->non_ip.l2 == DPAA2_PSR_SUMMARY_NONIP_PTP) {
+ packet_type |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ m->ol_flags |= RTE_MBUF_F_RX_IEEE1588_PTP;
+ } else if (sum_l->non_ip.l2 == DPAA2_PSR_SUMMARY_MPLS) {
+ packet_type |= RTE_PTYPE_L2_ETHER_MPLS;
+ }
+ }
+
+ if (unlikely(sum->checksum_err)) {
+ fas = (const void *)&annotation->word1;
+ if (fas->l3ce)
+ m->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ if (fas->l4ce)
+ m->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ }
+
+ /*
+ * fafe0 (eCPRI) and fafe1 (RoCEv2) are reported by the soft parser but
+ * have no matching RTE_PTYPE value, so only the VXLAN inner types are
+ * translated here.
+ */
+ if (!sum->fafe0 && !sum->fafe1 && vxlan) {
+ if (sum->fafe2) {
+ packet_type |= RTE_PTYPE_INNER_L2_ETHER;
+ packet_type |= RTE_PTYPE_INNER_L3_IPV4;
+ } else if (sum_l->l4.fafe3) {
+ packet_type |= RTE_PTYPE_INNER_L2_ETHER;
+ packet_type |= RTE_PTYPE_INNER_L3_IPV6;
+ }
+ }
+
+ return packet_type;
+}
+
+#define DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule, mask, size, offset, nb, max, \
+ rule_val, mask_val, size_val, offset_val) \
+({ \
+ uint64_t *_rule = (rule); \
+ uint64_t *_mask = (mask); \
+ uint8_t *_size = (size); \
+ uint8_t *_offset = (offset); \
+ int _ecpri_ret = 0; \
+ int _nb = (nb); \
+ \
+ if (_nb >= (max)) { \
+ _ecpri_ret = -ENOSPC; \
+ } else { \
+ _rule[_nb] = (rule_val); \
+ _mask[_nb] = (mask_val); \
+ _size[_nb] = (size_val); \
+ _offset[_nb] = (offset_val); \
+ } \
+ _ecpri_ret; \
+})
+
+static inline int
+dpaa2_parser_ecpri_extract(const struct rte_flow_item_ecpri *spec,
+ const struct rte_flow_item_ecpri *mask,
+ uint64_t rule_data[], uint64_t mask_data[],
+ uint8_t extract_size[], uint8_t extract_off[],
+ union dpaa2_sp_fafe_parse *fafe, uint8_t max_nb)
+{
+ int extract_nb = 0, ret;
+
+ if (mask->hdr.common.type != 0xff) {
+ DPAA2_PR_PRINT("ECPRI header type not specified.");
+
+ return -EINVAL;
+ }
+
+ if (spec->hdr.common.type > RTE_ECPRI_MSG_TYPE_IWF_DCTRL) {
+ DPAA2_PR_PRINT("ECPRI header type(%d) not supported.",
+ spec->hdr.common.type);
+
+ return -ENOTSUP;
+ }
+
+ /** Extract eCPRI type from FAFE.*/
+ fafe->ecpri.ecpri = 1;
+ fafe->ecpri.msg_type = spec->hdr.common.type;
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, fafe->fafe_8b, 0xff, sizeof(uint8_t),
+ DPAA2_FAFE_PSR_RESULT_OFFSET);
+ if (ret)
+ return ret;
+ extract_nb++;
+
+ if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_IQ_DATA) {
+ if (mask->hdr.type0.pc_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type0.pc_id,
+ mask->hdr.type0.pc_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_iq_data, pc_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type0.seq_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type0.seq_id,
+ mask->hdr.type0.seq_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_iq_data, seq_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_BIT_SEQ) {
+ if (mask->hdr.type1.pc_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type1.pc_id,
+ mask->hdr.type1.pc_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_bit_seq, pc_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type1.seq_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type1.seq_id,
+ mask->hdr.type1.seq_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_bit_seq, seq_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RTC_CTRL) {
+ if (mask->hdr.type2.rtc_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type2.rtc_id,
+ mask->hdr.type2.rtc_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_rtc_ctrl, rtc_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type2.seq_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type2.seq_id,
+ mask->hdr.type2.seq_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_rtc_ctrl, seq_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_GEN_DATA) {
+ if (mask->hdr.type3.pc_id || mask->hdr.type3.seq_id)
+ DPAA2_PMD_WARN("Extract type3 msg not support.");
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RM_ACC) {
+ if (mask->hdr.type4.rma_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type4.rma_id,
+ mask->hdr.type4.rma_id, sizeof(uint8_t),
+ /** Compiler can't support to take address
+ * of bit-field
+ * offsetof(struct rte_ecpri_msg_rm_access,
+ * rma_id);
+ */
+ DPAA2_ECPRI_MSG_OFFSET + 0);
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type4.ele_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type4.ele_id,
+ mask->hdr.type4.ele_id, sizeof(rte_be16_t),
+ /** Compiler can't support to take address
+ * of bit-field
+ * offsetof(struct rte_ecpri_msg_rm_access,
+ * rma_id);
+ */
+ DPAA2_ECPRI_MSG_OFFSET + 2);
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_DLY_MSR) {
+ if (mask->hdr.type5.msr_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type5.msr_id,
+ mask->hdr.type5.msr_id, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_delay_measure, msr_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type5.act_type) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type5.act_type,
+ mask->hdr.type5.act_type, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_delay_measure, act_type));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RMT_RST) {
+ if (mask->hdr.type6.rst_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type6.rst_id,
+ mask->hdr.type6.rst_id, sizeof(rte_be16_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_remote_reset, rst_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type6.rst_op) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type6.rst_op,
+ mask->hdr.type6.rst_op, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_remote_reset, rst_op));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_EVT_IND) {
+ if (mask->hdr.type7.evt_id) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type7.evt_id,
+ mask->hdr.type7.evt_id, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_event_ind, evt_id));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type7.evt_type) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type7.evt_type,
+ mask->hdr.type7.evt_type, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_event_ind, evt_type));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type7.seq) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type7.seq,
+ mask->hdr.type7.seq, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_event_ind, seq));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ if (mask->hdr.type7.number) {
+ ret = DPAA2_PARSER_ECPRI_EXTRACT_SAFE(rule_data, mask_data, extract_size,
+ extract_off, extract_nb, max_nb, spec->hdr.type7.number,
+ mask->hdr.type7.number, sizeof(uint8_t),
+ DPAA2_ECPRI_MSG_OFFSET +
+ offsetof(struct rte_ecpri_msg_event_ind, number));
+ if (ret)
+ return ret;
+ extract_nb++;
+ }
+ } else {
+ DPAA2_PR_PRINT("Unsupported ecpri header type(%d)",
+ spec->hdr.common.type);
+ return -ENOTSUP;
+ }
+
+ return extract_nb;
+}
+
+static inline void
+dpaa2_print_rocev2_parse_result(const struct dpaa2_psr_result_parse *psr)
+{
+ int i;
+
+ DPAA2_PR_PRINT("ROCEv2 opcode: 0x%02x\r\n", psr->word8.rocev2.opcode);
+ DPAA2_PR_PRINT("ROCEv2 qp: ");
+ for (i = 0; i < ROCEV2_DEST_QP_SIZE; i++)
+ DPAA2_PR_PRINT("0x%02x ", psr->word8.rocev2.dest_qp[i]);
+
+ DPAA2_PR_PRINT("\r\n");
+}
+
+static inline void
+dpaa2_print_geneve_parse_result(const struct dpaa2_psr_result_parse *psr)
+{
+ DPAA2_PR_PRINT("GENEVE protocol: 0x%04x\r\n",
+ rte_be_to_cpu_16(psr->word8.geneve.protocol));
+ DPAA2_PR_PRINT("GENEVE vni: %02x %02x %02x\r\n",
+ psr->word8.geneve.geneve_vni[0],
+ psr->word8.geneve.geneve_vni[1],
+ psr->word8.geneve.geneve_vni[2]);
+}
+
+static inline void
+dpaa2_print_vxlan_parse_result(const struct dpaa2_psr_result_parse *psr)
+{
+ struct rte_ether_hdr vxlan_in_eth;
+ uint16_t vxlan_vlan_tci, i;
+
+ vxlan_in_eth.dst_addr.addr_bytes[0] = psr->word5.vxlan_in_daddr0;
+ vxlan_in_eth.dst_addr.addr_bytes[1] = psr->word5.vxlan_in_daddr1;
+ vxlan_in_eth.dst_addr.addr_bytes[2] = psr->word6.vxlan_in_daddr2;
+ vxlan_in_eth.dst_addr.addr_bytes[3] = psr->word6.vxlan_in_daddr3;
+ vxlan_in_eth.dst_addr.addr_bytes[4] = psr->word6.vxlan_in_daddr4;
+ vxlan_in_eth.dst_addr.addr_bytes[5] = psr->word6.vxlan_in_daddr5;
+
+ vxlan_in_eth.src_addr.addr_bytes[0] = psr->word6.vxlan_in_saddr0;
+ vxlan_in_eth.src_addr.addr_bytes[1] = psr->word7.vxlan_in_saddr1;
+ vxlan_in_eth.src_addr.addr_bytes[2] = psr->word7.vxlan_in_saddr2;
+ vxlan_in_eth.src_addr.addr_bytes[3] = psr->word7.vxlan_in_saddr3;
+ vxlan_in_eth.src_addr.addr_bytes[4] = psr->word8.vxlan_in_saddr4;
+ vxlan_in_eth.src_addr.addr_bytes[5] = psr->word8.vxlan_in_saddr5;
+
+ vxlan_in_eth.ether_type = psr->word8.vxlan_eth_type;
+ vxlan_in_eth.ether_type = rte_be_to_cpu_16(vxlan_in_eth.ether_type);
+
+ DPAA2_PR_PRINT("VXLAN inner eth:\r\n");
+ DPAA2_PR_PRINT("dst addr: ");
+ for (i = 0; i < RTE_ETHER_ADDR_LEN; i++) {
+ if (i != 0)
+ DPAA2_PR_PRINT(":");
+ DPAA2_PR_PRINT("%02x", vxlan_in_eth.dst_addr.addr_bytes[i]);
+ }
+ DPAA2_PR_PRINT("\r\n");
+ DPAA2_PR_PRINT("src addr: ");
+ for (i = 0; i < RTE_ETHER_ADDR_LEN; i++) {
+ if (i != 0)
+ DPAA2_PR_PRINT(":");
+ DPAA2_PR_PRINT("%02x", vxlan_in_eth.src_addr.addr_bytes[i]);
+ }
+ DPAA2_PR_PRINT("\r\n");
+ DPAA2_PR_PRINT("type: 0x%04x\r\n", vxlan_in_eth.ether_type);
+ if (vxlan_in_eth.ether_type == RTE_ETHER_TYPE_VLAN) {
+ rte_memcpy(&vxlan_vlan_tci,
+ &psr->word5.vxlan_vlan_tci, sizeof(uint16_t));
+ vxlan_vlan_tci = rte_be_to_cpu_16(vxlan_vlan_tci);
+ DPAA2_PR_PRINT("vlan tci: 0x%04x\r\n", vxlan_vlan_tci);
+ }
+}
+
+static inline void
+dpaa2_print_ecpri_parse_result(const struct dpaa2_psr_result_parse *psr)
+{
+ uint8_t msg_type;
+ struct rte_ecpri_combined_msg_hdr ecpri_msg;
+
+ msg_type = psr->word3.fafe.ecpri.msg_type;
+ if (msg_type > RTE_ECPRI_MSG_TYPE_IWF_DCTRL) {
+ DPAA2_PR_PRINT("Invalid ECPRI message type(0x%02x)\r\n",
+ msg_type);
+ return;
+ }
+
+ DPAA2_PR_PRINT("ECPRI type %d present\r\n", msg_type);
+ rte_memcpy(ecpri_msg.dummy, psr->word8.sp_psr_ctx,
+ DPAA2_SP_PSR_CTX_LEN);
+ if (msg_type == RTE_ECPRI_MSG_TYPE_IQ_DATA ||
+ msg_type == RTE_ECPRI_MSG_TYPE_BIT_SEQ) {
+ DPAA2_PR_PRINT("pc_id(0x%04x) seq_id(0x%04x)\r\n",
+ rte_be_to_cpu_16(ecpri_msg.type0.pc_id),
+ rte_be_to_cpu_16(ecpri_msg.type0.seq_id));
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_RTC_CTRL) {
+ DPAA2_PR_PRINT("rtc_id(0x%04x) seq_id(0x%04x)\r\n",
+ rte_be_to_cpu_16(ecpri_msg.type2.rtc_id),
+ rte_be_to_cpu_16(ecpri_msg.type2.seq_id));
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_GEN_DATA) {
+ DPAA2_PR_PRINT("ECPRI type3 extract not support\r\n");
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_RM_ACC) {
+ DPAA2_PR_PRINT("rma_id(0x%02x) rw(0x%02x)",
+ ecpri_msg.type4.rma_id, ecpri_msg.type4.rw);
+ DPAA2_PR_PRINT(" rr(0x%02x) ele_id(0x%04x)\r\n",
+ ecpri_msg.type4.rr,
+ rte_be_to_cpu_16(ecpri_msg.type4.ele_id));
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_DLY_MSR) {
+ DPAA2_PR_PRINT("rma_id(0x%02x) rw(0x%02x)\r\n",
+ ecpri_msg.type5.msr_id,
+ ecpri_msg.type5.act_type);
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_RMT_RST) {
+ DPAA2_PR_PRINT("rst_id(0x%04x) rst_op(0x%02x)\r\n",
+ rte_be_to_cpu_16(ecpri_msg.type6.rst_id),
+ ecpri_msg.type6.rst_op);
+ } else if (msg_type == RTE_ECPRI_MSG_TYPE_EVT_IND) {
+ DPAA2_PR_PRINT("evt_id(0x%02x) evt_type(0x%02x)",
+ ecpri_msg.type7.evt_id,
+ ecpri_msg.type7.evt_type);
+ DPAA2_PR_PRINT(" seq(0x%02x) number(0x%02x)\r\n",
+ ecpri_msg.type7.seq,
+ ecpri_msg.type7.number);
+ }
+}
+
+static inline void
+dpaa2_print_fd_frc(const struct qbman_fd *fd)
+{
+ const struct dpaa2_ingress_frc *frc;
+ const struct dpaa2_psr_summary *sum;
+ const union dpaa2_psr_summary_l *sum_l;
+ const char *l4_nm;
+
+ frc = (const void *)&fd->simple.frc;
+ DPAA2_PR_PRINT("FRC faicfdv: %d\r\n", frc->faicfdv);
+ DPAA2_PR_PRINT("FRC faswov: %d\r\n", frc->faswov);
+ DPAA2_PR_PRINT("FRC faiadv: %d\r\n", frc->faiadv);
+ DPAA2_PR_PRINT("FRC faprv: %d\r\n", frc->faprv);
+ DPAA2_PR_PRINT("FRC faeadv: %d\r\n", frc->faeadv);
+ DPAA2_PR_PRINT("FRC fasv: %d\r\n", frc->fasv);
+
+ sum = &frc->psr_sum.sum;
+ sum_l = &sum->sum_l;
+ DPAA2_PR_PRINT("FRC Check sum err: %d\r\n", sum->checksum_err);
+ DPAA2_PR_PRINT("FRC vlan: %d\r\n", sum->vlan);
+ if (sum_l->non_ip.non_ip == DPAA2_PSR_SUMMARY_NONIP) {
+ DPAA2_PR_PRINT("FRC NON IP %s detected, l2: %d\r\n",
+ sum_l->non_ip.l2 == DPAA2_PSR_SUMMARY_ARP ?
+ "arp" :
+ sum_l->non_ip.l2 == DPAA2_PSR_SUMMARY_NONIP_PTP ?
+ "ptp" : "",
+ sum_l->non_ip.l2);
+ } else if (sum_l->l4_ext.l3 == DPAA2_PSR_SUMMARY_L4_EXT) {
+ DPAA2_PR_PRINT("FRC L4 ext %s (%d) detected\r\n",
+ sum_l->l4_ext.ip == DPAA2_PSR_SUMMARY_IPV4 ?
+ "ipv4" : "ipv6", sum_l->l4_ext.l4_ext);
+ } else {
+ if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_IP_FRAG)
+ l4_nm = "fragment";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_ICMP)
+ l4_nm = "icmp";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_IPSEC_ESP)
+ l4_nm = "IPSec ESP";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_TCP)
+ l4_nm = "tcp";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_SCTP)
+ l4_nm = "sctp";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_UDP)
+ l4_nm = "udp";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_PTP)
+ l4_nm = "ptp";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_UDP_ESP_IKE)
+ l4_nm = "udp/esp/ike";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_GTPU)
+ l4_nm = "gtpu";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_GTPC)
+ l4_nm = "gtpc";
+ else if (sum_l->l4.l4 == DPAA2_PSR_SUMMARY_VXLAN)
+ l4_nm = "vxlan";
+ else
+ l4_nm = "";
+ DPAA2_PR_PRINT("FRC %s(%d) %s(%d) detected\r\n",
+ sum_l->l4.l3 == DPAA2_PSR_SUMMARY_IPV4 ?
+ "ipv4" : "ipv6", sum_l->l4.l3,
+ l4_nm, sum_l->l4.l4);
+ }
+
+ DPAA2_PR_PRINT("FRC FAFE bit0~3: %d%d%d%d\r\n",
+ sum->fafe0, sum->fafe1, sum->fafe2, sum_l->l4.fafe3);
+}
+
+static inline void
+dpaa2_print_parse_result(const struct dpaa2_annot_hdr *annotation,
+ int is_sp)
+{
+ struct dpaa2_annot_hdr annot_convert;
+ uint64_t annot_size = sizeof(struct dpaa2_annot_hdr), offset = 0;
+ uint64_t *word;
+ const struct dpaa2_psr_result_parse *psr;
+ const struct dpaa2_fas_parse *fas;
+
+ rte_memcpy(&annot_convert, annotation, sizeof(struct dpaa2_annot_hdr));
+ while (annot_size) {
+ word = (uint64_t *)&annot_convert + offset;
+ *word = rte_cpu_to_be_64(*word);
+ offset++;
+ annot_size -= sizeof(uint64_t);
+ }
+ fas = (const void *)&annotation->word1;
+ psr = (const void *)&annot_convert.word3;
+
+ if (fas->discard)
+ DPAA2_PR_PRINT("FAS discard\r\n");
+ if (fas->macsec)
+ DPAA2_PR_PRINT("FAS MACSEC present\r\n");
+ if (fas->ptp)
+ DPAA2_PR_PRINT("FAS PTP present\r\n");
+ if (fas->multicast)
+ DPAA2_PR_PRINT("FAS Multicast present\r\n");
+ if (fas->broadcast)
+ DPAA2_PR_PRINT("FAS Broadcast present\r\n");
+ if (fas->kse)
+ DPAA2_PR_PRINT("FAS Invalid key or key size error\r\n");
+ if (fas->eofhe)
+ DPAA2_PR_PRINT("FAS extract out of frame header\r\n");
+ if (fas->mnle)
+ DPAA2_PR_PRINT("FAS Max number of chained lookups reached\r\n");
+ if (fas->tide)
+ DPAA2_PR_PRINT("FAS Invalid table ID\r\n");
+ if (fas->piee)
+ DPAA2_PR_PRINT("FAS Policer Initialization Entry Error\r\n");
+ if (fas->fle)
+ DPAA2_PR_PRINT("FAS Frame Length Error\r\n");
+ if (fas->fpe)
+ DPAA2_PR_PRINT("FAS Frame physical Error\r\n");
+ if (fas->pte)
+ DPAA2_PR_PRINT("FAS Parser terminate early\r\n");
+ if (fas->isp)
+ DPAA2_PR_PRINT("FAS Invalid SP instruction is encountered\r\n");
+ if (fas->phe)
+ DPAA2_PR_PRINT("FAS Error during parsing\r\n");
+ if (fas->ble)
+ DPAA2_PR_PRINT("FAS block limit is exceeded\r\n");
+ if (fas->l3cv)
+ DPAA2_PR_PRINT("FAS L3 checksum validation is performed\r\n");
+ if (fas->l3ce)
+ DPAA2_PR_PRINT("FAS L3 checksum error detected\r\n");
+ if (fas->l4cv)
+ DPAA2_PR_PRINT("FAS L4 checksum validation is performed\r\n");
+ if (fas->l4ce)
+ DPAA2_PR_PRINT("FAS L4 checksum error detected\r\n");
+ DPAA2_PR_PRINT("FAS IFPID:%d PPID:%d, status:%02x\r\n",
+ fas->ifpid, fas->ppid, fas->status);
+
+ /** FAF error dump.*/
+ if (psr->word3.faf_l_be.arp_err)
+ DPAA2_PR_PRINT("FAF ARP error detected\r\n");
+ if (psr->word3.faf_l_be.mpls_err)
+ DPAA2_PR_PRINT("FAF MPLS error detected\r\n");
+ if (psr->word3.faf_l_be.pppoe_ppp_err)
+ DPAA2_PR_PRINT("FAF PPPOE/PPP error detected\r\n");
+ if (psr->word3.faf_l_be.vlan_err)
+ DPAA2_PR_PRINT("FAF vLAN error detected\r\n");
+ if (psr->word3.faf_l_be.llc_snap_err)
+ DPAA2_PR_PRINT("FAF LLC/SNAP error detected\r\n");
+ if (psr->word3.faf_l_be.eth_err)
+ DPAA2_PR_PRINT("FAF Ethernet error detected\r\n");
+ if (psr->word3.faf_l_be.psr_err)
+ DPAA2_PR_PRINT("FAF Parser error detected\r\n");
+ if (psr->word3.faf_l_be.spsr_err)
+ DPAA2_PR_PRINT("FAF Soft Parser error detected\r\n");
+ if (psr->word3.faf_l_be.vxlan_err)
+ DPAA2_PR_PRINT("FAF VXLan error detected\r\n");
+ if (psr->word4.faf_h_be.esp_err)
+ DPAA2_PR_PRINT("FAF ESP error detected\r\n");
+ if (psr->word4.faf_h_be.gtp_err)
+ DPAA2_PR_PRINT("FAF GTP error detected\r\n");
+ if (psr->word4.faf_h_be.gtp_err)
+ DPAA2_PR_PRINT("FAF GTP error detected\r\n");
+ if (psr->word4.faf_h_be.l4_spsr_err)
+ DPAA2_PR_PRINT("FAF L4 soft Parser error detected\r\n");
+ if (psr->word4.faf_h_be.sctp_err)
+ DPAA2_PR_PRINT("FAF SCTP error detected\r\n");
+ if (psr->word4.faf_h_be.ipsec_err)
+ DPAA2_PR_PRINT("FAF IPSec error detected\r\n");
+ if (psr->word4.faf_h_be.tcp_err)
+ DPAA2_PR_PRINT("FAF TCP error detected\r\n");
+ if (psr->word4.faf_h_be.udp_err)
+ DPAA2_PR_PRINT("FAF UDP error detected\r\n");
+ if (psr->word4.faf_h_be.l3_spsr_err)
+ DPAA2_PR_PRINT("FAF L3 soft Parser error detected\r\n");
+ if (psr->word4.faf_h_be.gre_err)
+ DPAA2_PR_PRINT("FAF GRE error detected\r\n");
+ if (psr->word4.faf_h_be.min_encp_err)
+ DPAA2_PR_PRINT("FAF Min.Encap error detected\r\n");
+ if (psr->word4.faf_h_be.ip_n_err)
+ DPAA2_PR_PRINT("FAF IP.n error detected\r\n");
+ if (psr->word4.faf_h_be.ip_1_err)
+ DPAA2_PR_PRINT("FAF IP.1 error detected\r\n");
+ if (psr->word4.faf_h_be.l2_spsr_err)
+ DPAA2_PR_PRINT("FAF L2 soft parser error detected\r\n");
+
+ DPAA2_PR_PRINT("Next Header: %04x\r\n",
+ rte_be_to_cpu_16(psr->word3.nxthdr));
+
+ if (is_sp) {
+ if (psr->word3.fafe.ecpri.ecpri) {
+ DPAA2_PR_PRINT("FAFE ECPRI present, msg: %d\r\n",
+ psr->word3.fafe.ecpri.msg_type);
+ } else if (psr->word3.fafe.rocev2_tunnel.rocev2) {
+ DPAA2_PR_PRINT("FAFE ROCEV2 present\r\n");
+ } else if (psr->word3.fafe.rocev2_tunnel.geneve) {
+ DPAA2_PR_PRINT("FAFE GENEVE present\r\n");
+ } else {
+ if (psr->word3.fafe.rocev2_tunnel.in_vlan)
+ DPAA2_PR_PRINT("FAFE TUNNEL vLAN present\r\n");
+ if (psr->word3.fafe.rocev2_tunnel.in_ipv4)
+ DPAA2_PR_PRINT("FAFE TUNNEL IPv4 present\r\n");
+ if (psr->word3.fafe.rocev2_tunnel.in_ipv6)
+ DPAA2_PR_PRINT("FAFE TUNNEL IPv6 present\r\n");
+ if (psr->word3.fafe.rocev2_tunnel.in_udp)
+ DPAA2_PR_PRINT("FAFE TUNNEL UDP present\r\n");
+ if (psr->word3.fafe.rocev2_tunnel.in_tcp)
+ DPAA2_PR_PRINT("FAFE TUNNEL TCP present\r\n");
+ }
+ } else {
+ DPAA2_PR_PRINT("FAFE: 0x%02x\r\n", psr->word3.fafe.fafe_8b);
+ }
+
+ /** FAF popular protocol dump.*/
+ if (psr->word3.faf_l_be.arp)
+ DPAA2_PR_PRINT("FAF ARP Present\r\n");
+ if (psr->word3.faf_l_be.mac)
+ DPAA2_PR_PRINT("FAF Ethernet MAC Present\r\n");
+ if (psr->word3.faf_l_be.vxlan)
+ DPAA2_PR_PRINT("FAF VXLan Present\r\n");
+ if (psr->word3.faf_l_be.ike)
+ DPAA2_PR_PRINT("FAF IKE Present\r\n");
+ if (psr->word3.faf_l_be.ptp)
+ DPAA2_PR_PRINT("FAF PTP Present\r\n");
+ if (psr->word4.faf_h_be.gtp)
+ DPAA2_PR_PRINT("FAF GTP Present\r\n");
+ if (psr->word4.faf_h_be.sctp)
+ DPAA2_PR_PRINT("FAF SCTP Present\r\n");
+ if (psr->word4.faf_h_be.ipsec)
+ DPAA2_PR_PRINT("FAF IPSec Present\r\n");
+ if (psr->word4.faf_h_be.ipsec_esp)
+ DPAA2_PR_PRINT("FAF IPSec ESP Present\r\n");
+ if (psr->word4.faf_h_be.ipsec_ah)
+ DPAA2_PR_PRINT("FAF IPSec AH Present\r\n");
+ if (psr->word4.faf_h_be.gre)
+ DPAA2_PR_PRINT("FAF GRE Present\r\n");
+ if (psr->word4.faf_h_be.icmp)
+ DPAA2_PR_PRINT("FAF ICMP Present\r\n");
+ if (psr->word4.faf_h_be.ipv6_1)
+ DPAA2_PR_PRINT("FAF IPv6 Present\r\n");
+ if (psr->word4.faf_h_be.ipv6_n)
+ DPAA2_PR_PRINT("FAF last IPv6 Present\r\n");
+ if (psr->word4.faf_h_be.ipv4_1)
+ DPAA2_PR_PRINT("FAF IPv4 Present\r\n");
+ if (psr->word4.faf_h_be.ipv4_n)
+ DPAA2_PR_PRINT("FAF last IPv4 Present\r\n");
+ if (psr->word4.faf_h_be.ip_1_opt)
+ DPAA2_PR_PRINT("FAF IP Present\r\n");
+ if (psr->word4.faf_h_be.ip_1_init_frag)
+ DPAA2_PR_PRINT("FAF IP first fragment Present\r\n");
+ if (psr->word4.faf_h_be.ip_1_frag)
+ DPAA2_PR_PRINT("FAF IP fragment Present\r\n");
+ if (psr->word4.faf_h_be.tcp)
+ DPAA2_PR_PRINT("FAF TCP Present\r\n");
+ if (psr->word4.faf_h_be.udp)
+ DPAA2_PR_PRINT("FAF UDP Present\r\n");
+
+ DPAA2_PR_PRINT("Parser result eth header offset: %02x\r\n",
+ psr->word5.eth_off);
+ DPAA2_PR_PRINT("Parser result TCI1 offset: %02x\r\n",
+ psr->word5.vlan_tci_1_off);
+ DPAA2_PR_PRINT("Parser result last eth type offset: %02x\r\n",
+ psr->word5.last_etype_off);
+ DPAA2_PR_PRINT("Parser result l3 header offset: %02x\r\n",
+ psr->word6.l3_off);
+ DPAA2_PR_PRINT("Parser result l4 header offset: %02x\r\n",
+ psr->word6.l4_off);
+ DPAA2_PR_PRINT("Parser result l5 header offset: %02x\r\n",
+ psr->word6.l5_off);
+ DPAA2_PR_PRINT("Parser result next header offset: %02x\r\n",
+ psr->word7.nxthdr_off);
+
+ if (is_sp) {
+ if (psr->word3.faf_l_be.vxlan)
+ dpaa2_print_vxlan_parse_result(psr);
+ if (psr->word3.fafe.ecpri.ecpri)
+ dpaa2_print_ecpri_parse_result(psr);
+ if (psr->word3.fafe.rocev2_tunnel.rocev2)
+ dpaa2_print_rocev2_parse_result(psr);
+ if (psr->word3.fafe.rocev2_tunnel.geneve)
+ dpaa2_print_geneve_parse_result(psr);
+ }
+}
+
+#endif /* _DPAA2_PARSE_DECODE_H */
diff --git a/drivers/net/dpaa2/dpaa2_pmd_logs.h b/drivers/net/dpaa2/dpaa2_pmd_logs.h
index 28c9bed1ae..f1148f5b8d 100644
--- a/drivers/net/dpaa2/dpaa2_pmd_logs.h
+++ b/drivers/net/dpaa2/dpaa2_pmd_logs.h
@@ -5,6 +5,13 @@
#ifndef _DPAA2_PMD_LOGS_H_
#define _DPAA2_PMD_LOGS_H_
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_log.h>
+
extern int dpaa2_logtype_pmd;
#define RTE_LOGTYPE_DPAA2_NET dpaa2_logtype_pmd
@@ -36,4 +43,64 @@ extern int dpaa2_logtype_pmd;
#define DPAA2_PMD_DP_WARN(fmt, ...) \
DPAA2_PMD_DP_LOG(WARNING, fmt, ## __VA_ARGS__)
+/** Maximum length of a single debug dump line. */
+#define DPAA2_DUMP_LINE_SIZE 512
+
+/**
+ * Append formatted text to the per-thread debug dump line buffer.
+ *
+ * The parse result and flow dump helpers build a line from several
+ * fragments, so the text is accumulated here and handed to the regular
+ * logging framework one complete line at a time. This keeps the dump
+ * readable without writing to stdout or stderr from the driver.
+ *
+ * Carriage returns and newlines terminate a line and are not logged.
+ */
+static inline void
+dpaa2_dump_print(const char *fmt, ...) __rte_format_printf(1, 2);
+
+static inline void
+dpaa2_dump_print(const char *fmt, ...)
+{
+ static __thread char line[DPAA2_DUMP_LINE_SIZE];
+ static __thread size_t used;
+ size_t start, i;
+ va_list ap;
+ int len;
+
+ va_start(ap, fmt);
+ len = vsnprintf(line + used, sizeof(line) - used, fmt, ap);
+ va_end(ap);
+
+ if (len > 0) {
+ used += (size_t)len;
+ /* Output was truncated, keep the buffer consistent. */
+ if (used >= sizeof(line))
+ used = sizeof(line) - 1;
+ }
+
+ /* Log every complete line held in the buffer. */
+ start = 0;
+ for (i = 0; i < used; i++) {
+ if (line[i] != '\n' && line[i] != '\r')
+ continue;
+ line[i] = '\0';
+ if (i != start)
+ DPAA2_PMD_DEBUG("%s", &line[start]);
+ start = i + 1;
+ }
+
+ if (start) {
+ used -= start;
+ memmove(line, &line[start], used);
+ }
+
+ /* Flush a full buffer even without a line terminator. */
+ if (used >= sizeof(line) - 1) {
+ line[used] = '\0';
+ DPAA2_PMD_DEBUG("%s", line);
+ used = 0;
+ }
+}
+
#endif /* _DPAA2_PMD_LOGS_H_ */
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 7ba6a4563f..47c8cacdc4 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -26,19 +26,43 @@
#include "dpaa2_pmd_logs.h"
#include "dpaa2_ethdev.h"
#include "base/dpaa2_hw_dpni_annot.h"
-#include "dpaa2_parse_dump.h"
+#include "dpaa2_parser_decode.h"
-static inline uint32_t __rte_hot
-dpaa2_dev_rx_parse_slow(struct rte_mbuf *mbuf,
- struct dpaa2_annot_hdr *annotation);
+static inline void
+dpaa2_dev_rx_annot_prefetch(const struct qbman_fd *fd)
+{
+ size_t fd_addr;
+ void *hw_annot_addr;
-static void enable_tx_tstamp(struct qbman_fd *fd) __rte_unused;
+ fd_addr = (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
+ hw_annot_addr = (void *)(fd_addr + DPAA2_FD_PTA_SIZE);
+ rte_prefetch0(hw_annot_addr);
+}
-static inline rte_mbuf_timestamp_t *
-dpaa2_timestamp_dynfield(struct rte_mbuf *mbuf)
+static inline void
+dpaa2_dev_rx_parse_offset(struct dpaa2_dev_priv *priv,
+ struct rte_mbuf *mbuf, const struct qbman_fd *fd)
{
- return RTE_MBUF_DYNFIELD(mbuf,
- dpaa2_timestamp_dynfield_offset, rte_mbuf_timestamp_t *);
+ size_t fd_addr;
+ const struct dpaa2_annot_hdr *annotation;
+ uint64_t word6;
+ struct dpaa2_psr_result_word6 *decoded;
+ struct dpaa2_dyn_rx_protocol_pos *pos;
+
+ fd_addr = (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
+ annotation = (void *)(fd_addr + DPAA2_FD_PTA_SIZE);
+
+ RTE_ASSERT(priv->psr_dynfield_offset >=
+ offsetof(struct rte_mbuf, dynfield1[0]) &&
+ priv->psr_dynfield_offset < (sizeof(struct rte_mbuf) -
+ sizeof(struct dpaa2_dyn_rx_protocol_pos)));
+ pos = (void *)((uint8_t *)mbuf + priv->psr_dynfield_offset);
+
+ word6 = rte_be_to_cpu_64(annotation->word6);
+ decoded = (void *)&word6;
+ pos->l3_offset = decoded->l3_off;
+ pos->l4_offset = decoded->l4_off;
+ pos->l5_offset = decoded->l5_off;
}
#define DPAA2_MBUF_TO_CONTIG_FD(_mbuf, _fd, _bpid) do { \
@@ -51,95 +75,108 @@ dpaa2_timestamp_dynfield(struct rte_mbuf *mbuf)
DPAA2_RESET_FD_FLC(_fd); \
} while (0)
-static inline void __rte_hot
-dpaa2_dev_rx_parse_new(struct rte_mbuf *m, const struct qbman_fd *fd,
- void *hw_annot_addr)
+
+static inline void
+dpaa2_dev_rx_mbuf_sched_set(struct rte_mbuf *m,
+ const struct qbman_fd *fd)
{
- uint16_t frc = DPAA2_GET_FD_FRC_PARSE_SUM(fd);
- struct dpaa2_annot_hdr *annotation =
- (struct dpaa2_annot_hdr *)hw_annot_addr;
uint32_t flc_lo, tc, flow;
- if (unlikely(dpaa2_print_parser_result))
- dpaa2_print_parse_result(annotation);
+ flc_lo = fd->simple.flc_lo;
+ if (flc_lo & (1 << DPAA2_FS_FLC_FS_MARK_OFFSET)) {
+ m->ol_flags |= RTE_MBUF_F_RX_FDIR;
+ tc = (flc_lo >> DPAA2_FS_FLC_TC_OFFSET) &
+ DPAA2_FS_FLC_TC_MASK;
+ flow = flc_lo >> DPAA2_FS_FLC_FLOW_OFFSET;
+ rte_mbuf_sched_set(m, flow, tc, DPAA2_GET_FD_DROPP(fd));
+ DPAA2_PMD_DP_DEBUG("FS frame received from TC[%d]->flow%d",
+ tc, flow);
+ } else {
+ m->hash.rss = fd->simple.flc_hi;
+ m->ol_flags |= RTE_MBUF_F_RX_RSS_HASH;
+ DPAA2_PMD_DP_DEBUG("Hash frame received with RSS(%08x)",
+ m->hash.rss);
+ }
+}
+
+static void __rte_hot
+dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
+ struct rte_mbuf *m, const struct qbman_fd *fd,
+ void *hw_annot_addr, int is_vlan)
+{
+ uint16_t frc = DPAA2_GET_FD_FRC_PARSE_SUM(fd);
+ struct dpaa2_psr_summary *frc_parse = (void *)&frc;
+ struct dpaa2_annot_hdr *annotation = hw_annot_addr;
+ int default_parsed = false, vlan2 = false;
+ uint32_t ext_packet_type = RTE_PTYPE_UNKNOWN;
+
+ RTE_SET_USED(priv);
m->packet_type = RTE_PTYPE_UNKNOWN;
+ if (unlikely(is_vlan)) {
+ if (frc & DPAA2_PKT_TYPE_VLAN_2)
+ vlan2 = true;
+ frc &= (~DPAA2_PKT_TYPE_VLAN);
+ }
+ if (priv->sp_protocol) {
+ if (frc_parse->fafe2) {
+ frc_parse->fafe2 = 0;
+ ext_packet_type |= RTE_PTYPE_TUNNEL_GENEVE;
+ }
+ if (frc_parse->sum_l.l4.fafe3) {
+ frc_parse->sum_l.l4.fafe3 = 0;
+ ext_packet_type |= RTE_PTYPE_INNER_L3_IPV4;
+ }
+ }
switch (frc) {
- case DPAA2_PKT_TYPE_ETHER:
- m->packet_type = RTE_PTYPE_L2_ETHER;
- break;
- case DPAA2_PKT_TYPE_IPV4:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV4;
- break;
- case DPAA2_PKT_TYPE_IPV6:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV6;
- break;
- case DPAA2_PKT_TYPE_IPV4_EXT:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV4_EXT;
- break;
- case DPAA2_PKT_TYPE_IPV6_EXT:
+ case DPAA2_PKT_TYPE_IPV4_UDP:
m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV6_EXT;
+ RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_UDP;
break;
case DPAA2_PKT_TYPE_IPV4_TCP:
m->packet_type = RTE_PTYPE_L2_ETHER |
RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_TCP;
break;
- case DPAA2_PKT_TYPE_IPV6_TCP:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_TCP;
- break;
- case DPAA2_PKT_TYPE_IPV4_UDP:
+ case DPAA2_PKT_TYPE_IPV4:
m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_UDP;
+ RTE_PTYPE_L3_IPV4;
break;
case DPAA2_PKT_TYPE_IPV6_UDP:
m->packet_type = RTE_PTYPE_L2_ETHER |
RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP;
break;
- case DPAA2_PKT_TYPE_IPV4_SCTP:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_SCTP;
- break;
- case DPAA2_PKT_TYPE_IPV6_SCTP:
+ case DPAA2_PKT_TYPE_IPV6_TCP:
m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_SCTP;
+ RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_TCP;
break;
- case DPAA2_PKT_TYPE_IPV4_ICMP:
+ case DPAA2_PKT_TYPE_IPV6:
m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_ICMP;
+ RTE_PTYPE_L3_IPV6;
break;
- case DPAA2_PKT_TYPE_IPV6_ICMP:
- m->packet_type = RTE_PTYPE_L2_ETHER |
- RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_ICMP;
+ case DPAA2_PKT_TYPE_ETHER:
+ m->packet_type = RTE_PTYPE_L2_ETHER;
break;
default:
- m->packet_type = dpaa2_dev_rx_parse_slow(m, annotation);
- }
- flc_lo = fd->simple.flc_lo;
- if (flc_lo & (1 << DPAA2_FS_FLC_FS_MARK_OFFSET)) {
- m->ol_flags |= RTE_MBUF_F_RX_FDIR;
- tc = (flc_lo >> DPAA2_FS_FLC_TC_OFFSET) &
- DPAA2_FS_FLC_TC_MASK;
- flow = flc_lo >> DPAA2_FS_FLC_FLOW_OFFSET;
- rte_mbuf_sched_set(m, flow, tc, 0);
- DPAA2_PMD_DP_DEBUG("FS frame received from TC[%d]->flow%d",
- tc, flow);
- } else {
- m->hash.rss = fd->simple.flc_hi;
- m->ol_flags |= RTE_MBUF_F_RX_RSS_HASH;
- DPAA2_PMD_DP_DEBUG("Hash frame received with RSS(%08x)",
- m->hash.rss);
+ m->packet_type = dpaa2_dev_rx_parse_frc(fd, m, annotation);
+ default_parsed = true;
}
- if (dpaa2_enable_ts[m->port]) {
- *dpaa2_timestamp_dynfield(m) = annotation->word2;
- m->ol_flags |= dpaa2_timestamp_rx_dynflag;
- DPAA2_PMD_DP_DEBUG("pkt timestamp:0x%" PRIx64 "",
- *dpaa2_timestamp_dynfield(m));
+ m->packet_type |= ext_packet_type;
+
+ if (unlikely(is_vlan && !default_parsed)) {
+ struct dpaa2_psr_result_word5 word5;
+ rte_be16_t *vlan_tci = NULL;
+
+ m->ol_flags |= RTE_MBUF_F_RX_VLAN;
+ *(rte_be64_t *)&word5 = rte_cpu_to_be_64(annotation->word5);
+ if (vlan2) {
+ vlan_tci = rte_pktmbuf_mtod_offset(m, void *,
+ word5.vlan_tci_n_off);
+ } else {
+ vlan_tci = rte_pktmbuf_mtod_offset(m, void *,
+ word5.vlan_tci_1_off);
+ }
+ m->vlan_tci = rte_be_to_cpu_16(*vlan_tci);
}
DPAA2_PMD_DP_DEBUG("HW frc = 0x%x\t packet type =0x%x "
@@ -249,36 +286,18 @@ dpaa2_dev_rx_parse_slow(struct rte_mbuf *mbuf,
}
static inline uint32_t __rte_hot
-dpaa2_dev_rx_parse(struct rte_mbuf *mbuf, void *hw_annot_addr)
+dpaa2_dev_rx_parse(struct dpaa2_dev_priv *priv,
+ struct rte_mbuf *mbuf, void *hw_annot_addr)
{
- struct dpaa2_annot_hdr *annotation =
- (struct dpaa2_annot_hdr *)hw_annot_addr;
+ struct dpaa2_annot_hdr *annotation = hw_annot_addr;
DPAA2_PMD_DP_DEBUG("(fast parse) Annotation = 0x%" PRIx64 "\t",
- annotation->word4);
-
- if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_L3CE))
- mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
- else if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_L3CV))
- mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
-
- if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_L4CE))
- mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
- else if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_L4CV))
- mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
-
- if (unlikely(dpaa2_print_parser_result))
- dpaa2_print_parse_result(annotation);
+ annotation->word4);
- if (dpaa2_enable_ts[mbuf->port]) {
- *dpaa2_timestamp_dynfield(mbuf) = annotation->word2;
- mbuf->ol_flags |= dpaa2_timestamp_rx_dynflag;
- DPAA2_PMD_DP_DEBUG("pkt timestamp: 0x%" PRIx64 "",
- *dpaa2_timestamp_dynfield(mbuf));
- }
+ RTE_SET_USED(priv);
/* Check detailed parsing requirement */
- if (annotation->word3 & 0x7FFFFC3FFFF)
+ if (unlikely(annotation->word3 & 0x7FFFFC3FFFF))
return dpaa2_dev_rx_parse_slow(mbuf, annotation);
/* Return some common types from parse processing */
@@ -307,8 +326,7 @@ dpaa2_dev_rx_parse(struct rte_mbuf *mbuf, void *hw_annot_addr)
}
static inline struct rte_mbuf *__rte_hot
-eth_sg_fd_to_mbuf(const struct qbman_fd *fd,
- int port_id)
+dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
{
struct qbman_sge *sgt, *sge;
size_t sg_addr, fd_addr;
@@ -336,12 +354,14 @@ eth_sg_fd_to_mbuf(const struct qbman_fd *fd,
first_seg->pkt_len = DPAA2_GET_FD_LEN(fd);
first_seg->nb_segs = 1;
first_seg->next = NULL;
- first_seg->port = port_id;
- if (dpaa2_svr_family == SVR_LX2160A)
- dpaa2_dev_rx_parse_new(first_seg, fd, hw_annot_addr);
- else
- first_seg->packet_type =
- dpaa2_dev_rx_parse(first_seg, hw_annot_addr);
+ first_seg->port = priv->eth_dev->data->port_id;
+ if (dpaa2_svr_family == SVR_LX2160A) {
+ dpaa2_dev_rx_parse_new(priv, first_seg, fd, hw_annot_addr, false);
+ } else {
+ first_seg->packet_type = dpaa2_dev_rx_parse(priv, first_seg,
+ hw_annot_addr);
+ }
+ dpaa2_dev_rx_mbuf_sched_set(first_seg, fd);
rte_mbuf_refcnt_set(first_seg, 1);
#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
@@ -381,13 +401,22 @@ eth_sg_fd_to_mbuf(const struct qbman_fd *fd,
}
static inline struct rte_mbuf *__rte_hot
-eth_fd_to_mbuf(const struct qbman_fd *fd,
- int port_id)
+dpaa2_eth_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
{
void *v_addr = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
void *hw_annot_addr = (void *)((size_t)v_addr + DPAA2_FD_PTA_SIZE);
struct rte_mbuf *mbuf = DPAA2_INLINE_MBUF_FROM_BUF(v_addr,
- rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size);
+ rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size);
+ int is_vlan = false;
+
+ if (unlikely(dpaa2_svr_family == SVR_LX2160A &&
+ (DPAA2_GET_FD_FRC_PARSE_SUM(fd) &
+ DPAA2_PKT_TYPE_VLAN))) {
+ mbuf->data_off = DPAA2_GET_FD_OFFSET(fd);
+ rte_prefetch0(rte_pktmbuf_mtod(mbuf, void *));
+ rte_prefetch0(hw_annot_addr);
+ is_vlan = true;
+ }
/* need to repopulated some of the fields,
* as they may have changed in last transmission
@@ -397,7 +426,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
mbuf->data_off = DPAA2_GET_FD_OFFSET(fd);
mbuf->data_len = DPAA2_GET_FD_LEN(fd);
mbuf->pkt_len = mbuf->data_len;
- mbuf->port = port_id;
+ mbuf->port = priv->eth_dev->data->port_id;
mbuf->next = NULL;
mbuf->hash.sched.color = DPAA2_GET_FD_DROPP(fd);
rte_mbuf_refcnt_set(mbuf, 1);
@@ -413,9 +442,10 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
*/
if (dpaa2_svr_family == SVR_LX2160A)
- dpaa2_dev_rx_parse_new(mbuf, fd, hw_annot_addr);
+ dpaa2_dev_rx_parse_new(priv, mbuf, fd, hw_annot_addr, is_vlan);
else
- mbuf->packet_type = dpaa2_dev_rx_parse(mbuf, hw_annot_addr);
+ mbuf->packet_type = dpaa2_dev_rx_parse(priv, mbuf, hw_annot_addr);
+ dpaa2_dev_rx_mbuf_sched_set(mbuf, fd);
DPAA2_PMD_DP_DEBUG("to mbuf - mbuf =%p, mbuf->buf_addr =%p, off = %d,"
"fd_off=%d fd =%" PRIx64 ", meta = %d bpid =%d, len=%d",
@@ -674,6 +704,7 @@ dump_err_pkts(struct dpaa2_queue *dpaa2_q)
struct dpaa2_fas *fas;
struct rte_mbuf *mbuf;
char title[32];
+ struct dpaa2_dev_priv *priv = eth_data->dev_private;
if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
ret = dpaa2_affine_qbman_swp();
@@ -729,15 +760,16 @@ dump_err_pkts(struct dpaa2_queue *dpaa2_q)
hw_annot_addr = (void *)((size_t)v_addr + DPAA2_FD_PTA_SIZE);
fas = hw_annot_addr;
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_annot_prefetch(fd);
if (DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg)
- mbuf = eth_sg_fd_to_mbuf(fd, eth_data->port_id);
+ mbuf = dpaa2_eth_sg_fd_to_mbuf(priv, fd);
else
- mbuf = eth_fd_to_mbuf(fd, eth_data->port_id);
+ mbuf = dpaa2_eth_fd_to_mbuf(priv, fd);
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_parse_offset(priv, mbuf, fd);
- if (!dpaa2_print_parser_result) {
- /** Don't print parse result twice.*/
- dpaa2_print_parse_result(hw_annot_addr);
- }
+ dpaa2_dev_rx_print_parser_result(priv, fd, mbuf);
DPAA2_PMD_ERR("Err pkt on port[%d]:", eth_data->port_id);
DPAA2_PMD_ERR("FD offset: %d, FD err: %x, FAS status: %x",
@@ -792,18 +824,12 @@ dpaa2_dev_prefetch_rx_common(void *queue, struct rte_mbuf **bufs,
struct qbman_pull_desc pulldesc;
struct queue_storage_info_t *q_storage;
struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
+ struct dpaa2_dev_priv *priv = eth_data->dev_private;
q_storage = dpaa2_q->q_storage[rte_lcore_id()];
- /* the error-queue drain runs on the regular portal (DPAA2_PER_LCORE_PORTAL);
- * the channel/interrupt path owns the ethrx portal, so skip it there
- */
- if (!by_channel) {
- struct dpaa2_dev_priv *priv = eth_data->dev_private;
-
- if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
- dump_err_pkts(priv->rx_err_vq);
- }
+ if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
+ dump_err_pkts(priv->rx_err_vq);
if (unlikely(!DPAA2_PER_LCORE_ETHRX_DPIO)) {
ret = dpaa2_affine_qbman_ethrx_swp();
@@ -909,18 +935,17 @@ dpaa2_dev_prefetch_rx_common(void *queue, struct rte_mbuf **bufs,
dpaa2_dev_prefetch_next_psr(dq_storage);
fd = qbman_result_DQ_fd(dq_storage);
+
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_annot_prefetch(fd);
if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
- bufs[num_rx] = eth_sg_fd_to_mbuf(fd, eth_data->port_id);
+ bufs[num_rx] = dpaa2_eth_sg_fd_to_mbuf(priv, fd);
else
- bufs[num_rx] = eth_fd_to_mbuf(fd, eth_data->port_id);
-#if defined(RTE_LIBRTE_IEEE1588)
- if (bufs[num_rx]->ol_flags & RTE_MBUF_F_RX_IEEE1588_TMST) {
- struct dpaa2_dev_priv *priv = eth_data->dev_private;
+ bufs[num_rx] = dpaa2_eth_fd_to_mbuf(priv, fd);
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_parse_offset(priv, bufs[num_rx], fd);
- priv->rx_timestamp =
- *dpaa2_timestamp_dynfield(bufs[num_rx]);
- }
-#endif
+ dpaa2_dev_rx_print_parser_result(priv, fd, bufs[num_rx]);
dq_storage++;
num_rx++;
@@ -989,7 +1014,10 @@ dpaa2_dev_process_parallel_event(struct qbman_swp *swp,
ev->queue_id = rxq->ev.queue_id;
ev->priority = rxq->ev.priority;
- ev->mbuf = eth_fd_to_mbuf(fd, rxq->eth_data->port_id);
+ if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
+ ev->mbuf = dpaa2_eth_sg_fd_to_mbuf(rxq->eth_data->dev_private, fd);
+ else
+ ev->mbuf = dpaa2_eth_fd_to_mbuf(rxq->eth_data->dev_private, fd);
qbman_swp_dqrr_consume(swp, dq);
}
@@ -1014,7 +1042,10 @@ dpaa2_dev_process_atomic_event(struct qbman_swp *swp __rte_unused,
ev->queue_id = rxq->ev.queue_id;
ev->priority = rxq->ev.priority;
- ev->mbuf = eth_fd_to_mbuf(fd, rxq->eth_data->port_id);
+ if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
+ ev->mbuf = dpaa2_eth_sg_fd_to_mbuf(rxq->eth_data->dev_private, fd);
+ else
+ ev->mbuf = dpaa2_eth_fd_to_mbuf(rxq->eth_data->dev_private, fd);
dqrr_index = qbman_get_dqrr_idx(dq);
*dpaa2_seqn(ev->mbuf) = dqrr_index + 1;
@@ -1041,7 +1072,10 @@ dpaa2_dev_process_ordered_event(struct qbman_swp *swp,
ev->queue_id = rxq->ev.queue_id;
ev->priority = rxq->ev.priority;
- ev->mbuf = eth_fd_to_mbuf(fd, rxq->eth_data->port_id);
+ if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
+ ev->mbuf = dpaa2_eth_sg_fd_to_mbuf(rxq->eth_data->dev_private, fd);
+ else
+ ev->mbuf = dpaa2_eth_fd_to_mbuf(rxq->eth_data->dev_private, fd);
*dpaa2_seqn(ev->mbuf) = DPAA2_ENQUEUE_FLAG_ORP;
*dpaa2_seqn(ev->mbuf) |= qbman_result_DQ_odpid(dq) << DPAA2_EQCR_OPRID_SHIFT;
@@ -1151,12 +1185,15 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
dpaa2_dev_prefetch_next_psr(dq_storage);
fd = qbman_result_DQ_fd(dq_storage);
+
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_annot_prefetch(fd);
if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
- bufs[num_rx] = eth_sg_fd_to_mbuf(fd,
- eth_data->port_id);
+ bufs[num_rx] = dpaa2_eth_sg_fd_to_mbuf(priv, fd);
else
- bufs[num_rx] = eth_fd_to_mbuf(fd,
- eth_data->port_id);
+ bufs[num_rx] = dpaa2_eth_fd_to_mbuf(priv, fd);
+ if (priv->psr_dynfield_offset >= 0)
+ dpaa2_dev_rx_parse_offset(priv, bufs[num_rx], fd);
#if defined(RTE_LIBRTE_IEEE1588)
if (bufs[num_rx]->ol_flags & RTE_MBUF_F_RX_IEEE1588_TMST) {
@@ -1164,6 +1201,7 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
*dpaa2_timestamp_dynfield(bufs[num_rx]);
}
#endif
+ dpaa2_dev_rx_print_parser_result(priv, fd, bufs[num_rx]);
dq_storage++;
num_rx++;
@@ -1314,7 +1352,7 @@ uint16_t dpaa2_dev_tx_conf(void *queue)
return num_tx_conf;
}
-
+#ifdef RTE_LIBRTE_IEEE1588
/* Configure the egress frame annotation for timestamp update */
static void enable_tx_tstamp(struct qbman_fd *fd)
{
@@ -1336,7 +1374,7 @@ static void enable_tx_tstamp(struct qbman_fd *fd)
fd_faead->ctrl = DPAA2_ANNOT_FAEAD_A2V | DPAA2_ANNOT_FAEAD_UPDV |
DPAA2_ANNOT_FAEAD_UPD;
}
-
+#endif
/*
* Callback to handle sending packets through WRIOP based interface
*/
@@ -1617,8 +1655,12 @@ dpaa2_dev_free_eqresp_buf(uint16_t eqresp_ci,
fd = qbman_result_eqresp_fd(&dpio_dev->eqresp[eqresp_ci]);
/* Setting port id does not matter as we are to free the mbuf */
- m = eth_fd_to_mbuf(fd, 0);
- rte_pktmbuf_free(m);
+ if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg))
+ m = dpaa2_eth_sg_fd_to_mbuf(dpaa2_q->eth_data->dev_private, fd);
+ else
+ m = dpaa2_eth_fd_to_mbuf(dpaa2_q->eth_data->dev_private, fd);
+ if (m)
+ rte_pktmbuf_free(m);
}
static void
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 25/45] net/dpaa2: parse tunnel and fragmented packet types
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (23 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 24/45] net/dpaa2: restructure dpaa2 parser processing Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 26/45] net/dpaa2: remove unused soft parser driver Prashant Gupta
` (19 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Extend the Rx parser to identify additional packet types decoded from
the hardware parser summary: IPv4/IPv6 fragments, IPsec ESP, and GTP-U /
GTP-C tunnels. The corresponding DPAA2_PKT_TYPE_* values are added and
mapped to the matching RTE_PTYPE_* flags in the slow-path parser.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.h | 16 ++++++++++++++++
drivers/net/dpaa2/dpaa2_rxtx.c | 32 ++++++++++++++++++++++++++++++++
2 files changed, 48 insertions(+)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 2f552f3e29..6c72b9012c 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -162,6 +162,10 @@
#define DPAA2_PKT_TYPE_IPV6_EXT_GRE_IPV6_UDP_TCP \
(DPAA2_PKT_TYPE_IPV6_EXT | DPAA2_PSR_SUMMARY_GRE_IPV6_UDP_TCP)
+#define DPAA2_PKT_TYPE_IPV4_FRAG \
+ (DPAA2_PSR_SUMMARY_IP_FRAG | DPAA2_PKT_TYPE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_FRAG \
+ (DPAA2_PSR_SUMMARY_IP_FRAG | DPAA2_PKT_TYPE_IPV6)
#define DPAA2_PKT_TYPE_IPV4_TCP \
(DPAA2_PSR_SUMMARY_TCP | DPAA2_PKT_TYPE_IPV4)
#define DPAA2_PKT_TYPE_IPV6_TCP \
@@ -178,6 +182,18 @@
(DPAA2_PSR_SUMMARY_ICMP | DPAA2_PKT_TYPE_IPV4)
#define DPAA2_PKT_TYPE_IPV6_ICMP \
(DPAA2_PSR_SUMMARY_ICMP | DPAA2_PKT_TYPE_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_ESP \
+ (DPAA2_PSR_SUMMARY_IPSEC_ESP | DPAA2_PKT_TYPE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_ESP \
+ (DPAA2_PSR_SUMMARY_IPSEC_ESP | DPAA2_PKT_TYPE_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_GTPU \
+ (DPAA2_PSR_SUMMARY_GTPU | DPAA2_PKT_TYPE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_GTPU \
+ (DPAA2_PSR_SUMMARY_GTPU | DPAA2_PKT_TYPE_IPV6)
+#define DPAA2_PKT_TYPE_IPV4_GTPC \
+ (DPAA2_PSR_SUMMARY_GTPC | DPAA2_PKT_TYPE_IPV4)
+#define DPAA2_PKT_TYPE_IPV6_GTPC \
+ (DPAA2_PSR_SUMMARY_GTPC | DPAA2_PKT_TYPE_IPV6)
#define DPAA2_PKT_TYPE_VLAN_1 0x0100
#define DPAA2_PKT_TYPE_VLAN_2 0x0200
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 47c8cacdc4..26c84e7125 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -153,6 +153,38 @@ dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
m->packet_type = RTE_PTYPE_L2_ETHER |
RTE_PTYPE_L3_IPV6;
break;
+ case DPAA2_PKT_TYPE_IPV4_FRAG:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_FRAG;
+ break;
+ case DPAA2_PKT_TYPE_IPV6_FRAG:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_FRAG;
+ break;
+ case DPAA2_PKT_TYPE_IPV4_ESP:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_ESP;
+ break;
+ case DPAA2_PKT_TYPE_IPV6_ESP:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_ESP;
+ break;
+ case DPAA2_PKT_TYPE_IPV4_GTPU:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case DPAA2_PKT_TYPE_IPV6_GTPU:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case DPAA2_PKT_TYPE_IPV4_GTPC:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case DPAA2_PKT_TYPE_IPV6_GTPC:
+ m->packet_type = RTE_PTYPE_L2_ETHER |
+ RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_GTPC;
+ break;
case DPAA2_PKT_TYPE_ETHER:
m->packet_type = RTE_PTYPE_L2_ETHER;
break;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 26/45] net/dpaa2: remove unused soft parser driver
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (24 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 25/45] net/dpaa2: parse tunnel and fragmented packet types Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 27/45] drivers: refresh dpaa2 MC and SoC version info Prashant Gupta
` (18 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Softparser only supports to be loaded before Linux starts.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 28 -----
drivers/net/dpaa2/dpaa2_recycle.c | 1 -
drivers/net/dpaa2/dpaa2_sparser.c | 137 --------------------
drivers/net/dpaa2/dpaa2_sparser.h | 202 ------------------------------
drivers/net/dpaa2/meson.build | 1 -
5 files changed, 369 deletions(-)
delete mode 100644 drivers/net/dpaa2/dpaa2_sparser.c
delete mode 100644 drivers/net/dpaa2/dpaa2_sparser.h
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 387a153036..1df84e1316 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -29,7 +29,6 @@
#include <mc/fsl_dpmng.h>
#include <mc/fsl_dpcon.h>
#include "dpaa2_ethdev.h"
-#include "dpaa2_sparser.h"
#include <fsl_qbman_debug.h>
#define DRIVER_LOOPBACK_MODE "drv_loopback"
@@ -3738,33 +3737,6 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
}
eth_dev->data->mtu = RTE_ETHER_MTU;
- /*TODO To enable soft parser support DPAA2 driver needs to integrate
- * with external entity to receive byte code for software sequence
- * and same will be offload to the H/W using MC interface.
- * Currently it is assumed that DPAA2 driver has byte code by some
- * mean and same if offloaded to H/W.
- */
- if (getenv("DPAA2_ENABLE_SOFT_PARSER")) {
- WRIOP_SS_INITIALIZER(priv);
- ret = dpaa2_eth_load_wriop_soft_parser(priv, DPNI_SS_INGRESS);
- if (ret < 0) {
- DPAA2_PMD_ERR(" Error(%d) in loading softparser",
- ret);
- goto init_err;
- }
-
- ret = dpaa2_eth_enable_wriop_soft_parser(priv,
- DPNI_SS_INGRESS);
- if (ret < 0) {
- DPAA2_PMD_ERR(" Error(%d) in enabling softparser",
- ret);
- goto init_err;
- }
- }
-
- ret = dpaa2_soft_parser_loaded();
- if (ret > 0)
- DPAA2_PMD_INFO("soft parser is loaded");
DPAA2_PMD_INFO("%s: netdev created, connected to %s",
eth_dev->data->name, dpaa2_dev->ep_name);
diff --git a/drivers/net/dpaa2/dpaa2_recycle.c b/drivers/net/dpaa2/dpaa2_recycle.c
index 80c20272ce..ed038c204b 100644
--- a/drivers/net/dpaa2/dpaa2_recycle.c
+++ b/drivers/net/dpaa2/dpaa2_recycle.c
@@ -23,7 +23,6 @@
#include <dpaa2_hw_dpio.h>
#include <mc/fsl_dpmng.h>
#include "dpaa2_ethdev.h"
-#include "dpaa2_sparser.h"
#include <fsl_qbman_debug.h>
#include <rte_io.h>
diff --git a/drivers/net/dpaa2/dpaa2_sparser.c b/drivers/net/dpaa2/dpaa2_sparser.c
deleted file mode 100644
index 6c3e6eb573..0000000000
--- a/drivers/net/dpaa2/dpaa2_sparser.c
+++ /dev/null
@@ -1,137 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2018-2023 NXP
- */
-
-#include <rte_mbuf.h>
-#include <rte_ethdev.h>
-#include <rte_malloc.h>
-#include <rte_memcpy.h>
-#include <rte_string_fns.h>
-#include <dev_driver.h>
-
-#include <fslmc_logs.h>
-#include <fslmc_vfio.h>
-#include <dpaa2_hw_pvt.h>
-#include <dpaa2_hw_dpio.h>
-#include <dpaa2_hw_mempool.h>
-#include <dpaa2_pmd_logs.h>
-
-#include "dpaa2_ethdev.h"
-#include "dpaa2_sparser.h"
-#include "base/dpaa2_hw_dpni_annot.h"
-#define __STDC_FORMAT_MACROS
-#include <stdint.h>
-#include <inttypes.h>
-
-static uint8_t wriop_bytecode[] = {
- 0x00, 0x04, 0x29, 0x42, 0x03, 0xe0, 0x12, 0x00, 0x29, 0x02,
- 0x18, 0x00, 0x87, 0x3c, 0x00, 0x02, 0x18, 0x00, 0x00, 0x00
-};
-
-int dpaa2_eth_load_wriop_soft_parser(struct dpaa2_dev_priv *priv,
- enum dpni_soft_sequence_dest dest)
-{
- struct fsl_mc_io *dpni = priv->hw;
- struct dpni_load_ss_cfg cfg;
- struct dpni_drv_sparser_param sp_param;
- uint8_t *addr;
- int ret;
-
- memset(&sp_param, 0, sizeof(sp_param));
- sp_param.start_pc = priv->ss_offset;
- sp_param.byte_code = &wriop_bytecode[0];
- sp_param.size = sizeof(wriop_bytecode);
-
- cfg.dest = dest;
- cfg.ss_offset = sp_param.start_pc;
- cfg.ss_size = sp_param.size;
-
- addr = rte_malloc(NULL, sp_param.size, 64);
- if (!addr) {
- DPAA2_PMD_ERR("Memory unavailable for soft parser param");
- return -1;
- }
-
- memcpy(addr, sp_param.byte_code, sp_param.size);
- cfg.ss_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(addr, sp_param.size);
- if (cfg.ss_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("No IOMMU map for soft sequence(%p), size=%d",
- addr, sp_param.size);
- rte_free(addr);
-
- return -ENOBUFS;
- }
-
- ret = dpni_load_sw_sequence(dpni, CMD_PRI_LOW, priv->token, &cfg);
- if (ret) {
- DPAA2_PMD_ERR("dpni_load_sw_sequence failed");
- rte_free(addr);
- return ret;
- }
-
- priv->ss_iova = cfg.ss_iova;
- priv->ss_offset += sp_param.size;
- DPAA2_PMD_INFO("Soft parser loaded for dpni@%d", priv->hw_id);
-
- rte_free(addr);
- return 0;
-}
-
-int dpaa2_eth_enable_wriop_soft_parser(struct dpaa2_dev_priv *priv,
- enum dpni_soft_sequence_dest dest)
-{
- struct fsl_mc_io *dpni = priv->hw;
- struct dpni_enable_ss_cfg cfg;
- uint8_t pa[3];
- struct dpni_drv_sparser_param sp_param;
- uint8_t *param_addr = NULL;
- int ret;
-
- memset(&sp_param, 0, sizeof(sp_param));
- pa[0] = 32; /* Custom Header Length in bytes */
- sp_param.custom_header_first = 1;
- sp_param.param_offset = 32;
- sp_param.param_size = 1;
- sp_param.start_pc = priv->ss_offset;
- sp_param.param_array = (uint8_t *)&pa[0];
-
- cfg.dest = dest;
- cfg.ss_offset = sp_param.start_pc;
- cfg.set_start = sp_param.custom_header_first;
- cfg.hxs = (uint16_t)sp_param.link_to_hard_hxs;
- cfg.param_offset = sp_param.param_offset;
- cfg.param_size = sp_param.param_size;
- if (cfg.param_size) {
- param_addr = rte_malloc(NULL, cfg.param_size, 64);
- if (!param_addr) {
- DPAA2_PMD_ERR("Memory unavailable for soft parser param");
- return -1;
- }
-
- memcpy(param_addr, sp_param.param_array, cfg.param_size);
- cfg.param_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(param_addr,
- cfg.param_size);
- if (cfg.param_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for %p, size=%d",
- __func__, param_addr, cfg.param_size);
- rte_free(param_addr);
-
- return -ENOBUFS;
- }
- priv->ss_param_iova = cfg.param_iova;
- } else {
- cfg.param_iova = 0;
- }
-
- ret = dpni_enable_sw_sequence(dpni, CMD_PRI_LOW, priv->token, &cfg);
- if (ret) {
- DPAA2_PMD_ERR("Soft parser enabled for dpni@%d failed",
- priv->hw_id);
- rte_free(param_addr);
- return ret;
- }
-
- rte_free(param_addr);
- DPAA2_PMD_INFO("Soft parser enabled for dpni@%d", priv->hw_id);
- return 0;
-}
diff --git a/drivers/net/dpaa2/dpaa2_sparser.h b/drivers/net/dpaa2/dpaa2_sparser.h
deleted file mode 100644
index 0187cde113..0000000000
--- a/drivers/net/dpaa2/dpaa2_sparser.h
+++ /dev/null
@@ -1,202 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2018-2019 NXP
- */
-
-/**
- * @file dpaa2_sparser.h
- *
- * @brief Soft parser related macros & functions support for DPAA2 device
- * framework based applications.
- *
- */
-
-#ifndef _DPAA2_SPARSER_H
-#define _DPAA2_SPARSER_H
-
-#define WRIOP_SS_INITIALIZER(priv) \
-do { \
- /* Base offset of parse profile memory in WRIOP */ \
- (priv)->ss_offset = 0x20; \
- (priv)->ss_iova = (size_t)NULL; \
- (priv)->ss_param_iova = (size_t)NULL; \
-} while (0)
-
-/**************************************************************************/
-/*
- * @enum parser_starting_hxs_code
- * @Description PARSER Starting HXS code
- */
-/***************************************************************************/
-enum parser_starting_hxs_code {
- /** Ethernet Starting HXS coding */
- PARSER_ETH_STARTING_HXS = 0x0000,
- /** LLC+SNAP Starting HXS coding */
- PARSER_LLC_SNAP_STARTING_HXS = 0x0001,
- /** VLAN Starting HXS coding */
- PARSER_VLAN_STARTING_HXS = 0x0002,
- /** PPPoE+PPP Starting HXS coding */
- PARSER_PPPOE_PPP_STARTING_HXS = 0x0003,
- /** MPLS Starting HXS coding */
- PARSER_MPLS_STARTING_HXS = 0x0004,
- /** ARP Starting HXS coding */
- PARSER_ARP_STARTING_HXS = 0x0005,
- /** IP Starting HXS coding */
- PARSER_IP_STARTING_HXS = 0x0006,
- /** IPv4 Starting HXS coding */
- PARSER_IPV4_STARTING_HXS = 0x0007,
- /** IPv6 Starting HXS coding */
- PARSER_IPV6_STARTING_HXS = 0x0008,
- /** GRE Starting HXS coding */
- PARSER_GRE_STARTING_HXS = 0x0009,
- /** MinEncap Starting HXS coding */
- PARSER_MINENCAP_STARTING_HXS = 0x000A,
- /** Other L3 Shell Starting HXS coding */
- PARSER_OTHER_L3_SHELL_STARTING_HXS = 0x000B,
- /** TCP Starting HXS coding */
- PARSER_TCP_STARTING_HXS = 0x000C,
- /** UDP Starting HXS coding */
- PARSER_UDP_STARTING_HXS = 0x000D,
- /** IPSec Starting HXS coding */
- PARSER_IPSEC_STARTING_HXS = 0x000E,
- /** SCTP Starting HXS coding */
- PARSER_SCTP_STARTING_HXS = 0x000F,
- /** DCCP Starting HXS coding */
- PARSER_DCCP_STARTING_HXS = 0x0010,
- /** Other L4 Shell Starting HXS coding */
- PARSER_OTHER_L4_SHELL_STARTING_HXS = 0x0011,
- /** GTP Starting HXS coding */
- PARSER_GTP_STARTING_HXS = 0x0012,
- /** ESP Starting HXS coding */
- PARSER_ESP_STARTING_HXS = 0x0013,
- /** VXLAN Starting HXS coding */
- PARSER_VXLAN_STARTING_HXS = 0x0014,
- /** L5 (and above) Shell Starting HXS coding */
- PARSER_L5_SHELL_STARTING_HXS = 0x001E,
- /** Final Shell Starting HXS coding */
- PARSER_FINAL_SHELL_STARTING_HXS = 0x001F
-};
-
-/**************************************************************************/
-/*
- * @Description struct dpni_drv_sparser_param - Structure representing the
- * information needed to activate(enable) a Soft Parser.
- */
-/***************************************************************************/
-
-struct dpni_drv_sparser_param {
- /* The "custom_header_first" must be set if the custom header to parse
- * is the first header in the packet, otherwise "custom_header_first"
- * must be cleared.
- */
- uint8_t custom_header_first;
- /* Hard HXS on which a soft parser is activated. This must be
- * configured.
- * if the header to parse is not the first header in the packet.
- */
- enum parser_starting_hxs_code link_to_hard_hxs;
- /* Soft Sequence Start PC */
- uint16_t start_pc;
- /* Soft Sequence byte-code */
- uint8_t *byte_code;
- /* Soft Sequence size */
- uint16_t size;
- /* Pointer to the Parameters Array of the SP */
- uint8_t *param_array;
- /* Parameters offset */
- uint8_t param_offset;
- /* Parameters size */
- uint8_t param_size;
-};
-
-struct sp_parse_result {
- /* Next header */
- uint16_t nxt_hdr;
- /* Frame Attribute Flags Extension */
- uint16_t frame_attribute_flags_extension;
- /* Frame Attribute Flags (part 1) */
- uint32_t frame_attribute_flags_1;
- /* Frame Attribute Flags (part 2) */
- uint32_t frame_attribute_flags_2;
- /* Frame Attribute Flags (part 3) */
- uint32_t frame_attribute_flags_3;
- /* Shim Offset 1 */
- uint8_t shim_offset_1;
- /* Shim Offset 2 */
- uint8_t shim_offset_2;
- /* Outer IP protocol field offset */
- uint8_t ip_1_pid_offset;
- /* Ethernet offset */
- uint8_t eth_offset;
- /* LLC+SNAP offset */
- uint8_t llc_snap_offset;
- /* First VLAN's TCI field offset*/
- uint8_t vlan_tci1_offset;
- /* Last VLAN's TCI field offset*/
- uint8_t vlan_tcin_offset;
- /* Last Ethertype offset*/
- uint8_t last_etype_offset;
- /* PPPoE offset */
- uint8_t pppoe_offset;
- /* First MPLS offset */
- uint8_t mpls_offset_1;
- /* Last MPLS offset */
- uint8_t mpls_offset_n;
- /* Layer 3 (Outer IP, ARP, FCoE or FIP) offset */
- uint8_t l3_offset;
- /* Inner IP or MinEncap offset*/
- uint8_t ipn_or_minencap_offset;
- /* GRE offset */
- uint8_t gre_offset;
- /* Layer 4 offset*/
- uint8_t l4_offset;
- /* Layer 5 offset */
- uint8_t l5_offset;
- /* Routing header offset of 1st IPv6 header */
- uint8_t routing_hdr_offset1;
- /* Routing header offset of 2nd IPv6 header */
- uint8_t routing_hdr_offset2;
- /* Next header offset */
- uint8_t nxt_hdr_offset;
- /* IPv6 fragmentable part offset */
- uint8_t ipv6_frag_offset;
- /* Frame's untouched running sum, input to parser */
- uint16_t gross_running_sum;
- /* Running Sum */
- uint16_t running_sum;
- /* Parse Error code */
- uint8_t parse_error_code;
- /* Offset to the next header field before IPv6 fragment extension */
- uint8_t nxt_hdr_before_ipv6_frag_ext;
- /* Inner IP Protocol field offset */
- uint8_t ip_n_pid_offset;
- /* Reserved for Soft parsing context*/
- uint8_t soft_parsing_context[21];
-};
-
-struct frame_attr {
- const char *fld_name;
- uint8_t faf_offset;
- uint32_t fld_mask;
-};
-
-struct frame_attr_ext {
- const char *fld_name;
- uint8_t faf_ext_offset;
- uint16_t fld_mask;
-};
-
-
-struct parse_err {
- uint16_t code;
- const char *err_name;
-};
-
-/* Macro definitions */
-#define IS_ONE_BIT_FIELD(_mask) \
-(!((_mask) & ((_mask) - 1)) || (_mask == 1))
-
-int dpaa2_eth_load_wriop_soft_parser(struct dpaa2_dev_priv *priv,
- enum dpni_soft_sequence_dest dest);
-int dpaa2_eth_enable_wriop_soft_parser(struct dpaa2_dev_priv *priv,
- enum dpni_soft_sequence_dest dest);
-#endif /* _DPAA2_SPARSER_H_ */
diff --git a/drivers/net/dpaa2/meson.build b/drivers/net/dpaa2/meson.build
index 808dad8112..4f774f7c82 100644
--- a/drivers/net/dpaa2/meson.build
+++ b/drivers/net/dpaa2/meson.build
@@ -17,7 +17,6 @@ sources = files(
'dpaa2_flow.c',
'dpaa2_recycle.c',
'dpaa2_rxtx.c',
- 'dpaa2_sparser.c',
'dpaa2_ptp.c',
'mc/dprtc.c',
'mc/dpkg.c',
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 27/45] drivers: refresh dpaa2 MC and SoC version info
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (25 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 26/45] net/dpaa2: remove unused soft parser driver Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 28/45] drivers: identify dpaa2 soft parser protocol Prashant Gupta
` (17 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Fix MC revision typo to correct major to minor.
Get MC/SoC rev info only once and save them in bus info
as well as other information to reduce replicated variables.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/bus_fslmc_driver.h | 11 +-
drivers/bus/fslmc/fslmc_bus.c | 2 +
drivers/bus/fslmc/fslmc_vfio.c | 200 ++++++++++++++++++-----
drivers/bus/fslmc/fslmc_vfio.h | 30 +++-
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 79 +--------
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 7 -
drivers/bus/fslmc/private.h | 3 +
drivers/net/dpaa2/dpaa2_ethdev.c | 18 +-
drivers/net/dpaa2/dpaa2_ethdev.h | 1 +
drivers/net/dpaa2/dpaa2_rxtx.c | 10 +-
10 files changed, 225 insertions(+), 136 deletions(-)
diff --git a/drivers/bus/fslmc/bus_fslmc_driver.h b/drivers/bus/fslmc/bus_fslmc_driver.h
index c64ef094d0..b4a5a86318 100644
--- a/drivers/bus/fslmc/bus_fslmc_driver.h
+++ b/drivers/bus/fslmc/bus_fslmc_driver.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
- * Copyright 2016,2021-2023 NXP
+ * Copyright 2016,2021-2026 NXP
*
*/
@@ -89,6 +89,14 @@ enum rte_dpaa2_dev_type {
DPAA2_DEVTYPE_MAX,
};
+struct rte_fslmc_bus_info {
+ void *mem_pool;
+ int sp_protocol;
+ uint64_t mc_rev;
+ uint32_t svr;
+ uint32_t pvr;
+};
+
/**
* A structure describing a DPAA2 device.
*/
@@ -103,6 +111,7 @@ struct rte_dpaa2_device {
struct rte_intr_handle *intr_handle; /**< Interrupt handle */
char name[FSLMC_OBJECT_MAX_LEN]; /**< DPAA2 Object name*/
struct rte_dma_dev *dmadev; /**< DMA device */
+ const struct rte_fslmc_bus_info *bus_info;
};
typedef int (*rte_dpaa2_obj_create_t)(int vdev_fd,
diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index ee162b0ed8..2427421075 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -29,6 +29,7 @@
struct rte_bus rte_fslmc_bus;
static int fslmc_bus_device_count[DPAA2_DEVTYPE_MAX];
+struct rte_fslmc_bus_info fslmc_bus_info;
#define DPAA2_SEQN_DYNFIELD_NAME "dpaa2_seqn_dynfield"
RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_seqn_dynfield_offset)
@@ -199,6 +200,7 @@ scan_one_fslmc_device(char *dev_name)
goto cleanup;
}
dev->device.devargs = rte_bus_find_devargs(&rte_fslmc_bus, dev_name);
+ dev->bus_info = &fslmc_bus_info;
/* Update the device found into the device_count table */
fslmc_bus_device_count[dev->dev_type]++;
diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
index 412b70e5ae..4eb8552696 100644
--- a/drivers/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -87,6 +87,83 @@ enum {
FSLMC_VFIO_SOCKET_REQ_MEM
};
+/* Variable to store DPAA2 platform type */
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_svr_family)
+uint32_t dpaa2_svr_family;
+
+/* Variable to store DPAA2 DQRR size */
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dqrr_size)
+uint32_t dpaa2_dqrr_size;
+/* Variable to store DPAA2 EQCR size */
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_eqcr_size)
+uint32_t dpaa2_eqcr_size;
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_cluster_base)
+uint32_t dpaa2_cluster_base;
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_cluster_size)
+uint32_t dpaa2_cluster_size;
+
+struct fslmc_soc_type {
+ uint32_t svr;
+ uint32_t cluster_base;
+ uint32_t cluster_size;
+ uint32_t qm_dq_size;
+ uint32_t qm_eq_size;
+};
+
+/* For LS208X platform There are four clusters with following mapping:
+ * Cluster 1 (ID = x04) : CPU0, CPU1;
+ * Cluster 2 (ID = x05) : CPU2, CPU3;
+ * Cluster 3 (ID = x06) : CPU4, CPU5;
+ * Cluster 4 (ID = x07) : CPU6, CPU7;
+ */
+/* For LS108X platform There are two clusters with following mapping:
+ * Cluster 1 (ID = x02) : CPU0, CPU1, CPU2, CPU3;
+ * Cluster 2 (ID = x03) : CPU4, CPU5, CPU6, CPU7;
+ */
+/* For LX2160 platform There are four clusters with following mapping:
+ * Cluster 1 (ID = x00) : CPU0, CPU1;
+ * Cluster 2 (ID = x01) : CPU2, CPU3;
+ * Cluster 3 (ID = x02) : CPU4, CPU5;
+ * Cluster 4 (ID = x03) : CPU6, CPU7;
+ * Cluster 1 (ID = x04) : CPU8, CPU9;
+ * Cluster 2 (ID = x05) : CPU10, CP11;
+ * Cluster 3 (ID = x06) : CPU12, CPU13;
+ * Cluster 4 (ID = x07) : CPU14, CPU15;
+ */
+
+int
+fslmc_vfio_core_cluster_sdest(uint32_t cpu_id)
+{
+ int x = cpu_id / dpaa2_cluster_size;
+
+ if (!dpaa2_cluster_size) {
+ DPAA2_BUS_ERR("SoC type has not been initialized!");
+
+ return -ENODEV;
+ }
+
+ return dpaa2_cluster_base + x;
+}
+
+static const struct fslmc_soc_type s_soc_type[] = {
+ {
+ SVR_LS2080A, 4, 2,
+ DPAA2_DQRR_RING_SIZE, DPAA2_EQCR_RING_SIZE
+ },
+ {
+ SVR_LS2088A, 4, 2,
+ DPAA2_DQRR_RING_SIZE, DPAA2_EQCR_RING_SIZE
+ },
+ {
+ SVR_LS1080A, 2, 4,
+ DPAA2_DQRR_RING_SIZE, DPAA2_EQCR_RING_SIZE
+ },
+ {
+ SVR_LX2160A, 0, 2,
+ DPAA2_LX2_DQRR_RING_SIZE, DPAA2_LX2_EQCR_RING_SIZE
+ }
+};
+
RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_get_mcp_ptr)
void *
dpaa2_get_mcp_ptr(int portal_idx)
@@ -168,6 +245,70 @@ rte_fslmc_object_register(struct rte_dpaa2_object *object)
TAILQ_INSERT_TAIL(&dpaa2_obj_list, object, next);
}
+static int
+fslmc_vfio_get_bus_info(void *regs, struct rte_fslmc_bus_info *bus_info)
+{
+ struct fsl_mc_io mc_io;
+ struct mc_version mc_ver_info = {0};
+ struct mc_soc_version mc_plat_info;
+ int ret;
+ size_t i;
+
+ mc_io.regs = regs;
+
+ ret = mc_get_version(&mc_io, CMD_PRI_LOW, &mc_ver_info);
+ if (ret) {
+ DPAA2_BUS_ERR("Failed(%d) to obtain MC version.", ret);
+ return ret;
+ }
+ DPAA2_BUS_INFO("MC version is %d.%d.%d.", mc_ver_info.major,
+ mc_ver_info.minor, mc_ver_info.revision);
+
+ bus_info->mc_rev = RTE_FSL_MC_REV(mc_ver_info.major,
+ mc_ver_info.minor, mc_ver_info.revision);
+
+ if (bus_info->mc_rev < RTE_FSL_MC_REV(MC_VER_MAJOR, MC_VER_MINOR, 0)) {
+ DPAA2_BUS_ERR("DPAA2 MC version not compatible! %d.%d.%d < %d.%d.0",
+ mc_ver_info.major, mc_ver_info.minor, mc_ver_info.revision,
+ MC_VER_MAJOR, MC_VER_MINOR);
+ return -EPERM;
+ }
+
+ memset(&mc_plat_info, 0, sizeof(struct mc_soc_version));
+ ret = mc_get_soc_version(&mc_io, CMD_PRI_LOW, &mc_plat_info);
+ if (ret) {
+ DPAA2_BUS_ERR("Unable to get SoC version err:%d", ret);
+ return ret;
+ }
+
+ memset(&mc_plat_info, 0, sizeof(struct mc_soc_version));
+ ret = mc_get_soc_version(&mc_io, CMD_PRI_LOW, &mc_plat_info);
+ if (ret) {
+ DPAA2_BUS_ERR("Unable to get SoC version err:%d", ret);
+ return ret;
+ }
+
+ for (i = 0; i < RTE_DIM(s_soc_type); i++) {
+ if ((mc_plat_info.svr & DPAA2_SVR_MASK) == s_soc_type[i].svr) {
+ dpaa2_svr_family = s_soc_type[i].svr;
+ dpaa2_cluster_base = s_soc_type[i].cluster_base;
+ dpaa2_cluster_size = s_soc_type[i].cluster_size;
+ dpaa2_dqrr_size = s_soc_type[i].qm_dq_size;
+ dpaa2_eqcr_size = s_soc_type[i].qm_eq_size;
+ break;
+ }
+ }
+ if (i == RTE_DIM(s_soc_type)) {
+ DPAA2_BUS_ERR("Invalid SoC version: 0x%08x", mc_plat_info.svr);
+ return -ENODEV;
+ }
+
+ bus_info->svr = mc_plat_info.svr;
+ bus_info->pvr = mc_plat_info.pvr;
+
+ return 0;
+}
+
static const char *
fslmc_vfio_get_group_name(void)
{
@@ -1247,9 +1388,9 @@ fslmc_vfio_setup_device(const char *dev_addr,
dev_addr);
}
-static intptr_t vfio_map_mcp_obj(const char *mcp_obj)
+static void *fslmc_vfio_map_mcp_obj(const char *mcp_obj)
{
- intptr_t v_addr = (intptr_t)MAP_FAILED;
+ void *v_addr = MAP_FAILED;
int32_t ret, mc_fd;
struct vfio_group_status status = { .argsz = sizeof(status) };
@@ -1265,7 +1406,7 @@ static intptr_t vfio_map_mcp_obj(const char *mcp_obj)
goto MC_FAILURE;
}
- v_addr = (size_t)mmap(NULL, reg_info.size,
+ v_addr = mmap(NULL, reg_info.size,
PROT_WRITE | PROT_READ, MAP_SHARED,
mc_fd, reg_info.offset);
@@ -1388,7 +1529,7 @@ rte_dpaa2_vfio_setup_intr(struct rte_intr_handle *intr_handle,
}
static void
-fslmc_close_iodevices(struct rte_dpaa2_device *dev,
+fslmc_vfio_close_iodevices(struct rte_dpaa2_device *dev,
int vfio_fd)
{
struct rte_dpaa2_object *object = NULL;
@@ -1431,11 +1572,11 @@ fslmc_close_iodevices(struct rte_dpaa2_device *dev,
}
/*
- * fslmc_process_iodevices for processing only IO (ETH, CRYPTO, and possibly
+ * fslmc_vfio_process_iodevices for processing only IO (ETH, CRYPTO, and possibly
* EVENT) devices.
*/
static int
-fslmc_process_iodevices(struct rte_dpaa2_device *dev)
+fslmc_vfio_process_iodevices(struct rte_dpaa2_device *dev)
{
int dev_fd, ret;
struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
@@ -1477,12 +1618,10 @@ fslmc_process_iodevices(struct rte_dpaa2_device *dev)
}
static int
-fslmc_process_mcp(struct rte_dpaa2_device *dev)
+fslmc_vfio_process_mcp(struct rte_dpaa2_device *dev)
{
int ret;
- intptr_t v_addr;
- struct fsl_mc_io dpmng = {0};
- struct mc_version mc_ver_info = {0};
+ void *v_addr;
rte_mcp_ptr_list = malloc(sizeof(void *) * (MC_PORTAL_INDEX + 1));
if (!rte_mcp_ptr_list) {
@@ -1491,15 +1630,16 @@ fslmc_process_mcp(struct rte_dpaa2_device *dev)
goto cleanup;
}
- v_addr = vfio_map_mcp_obj(dev->device.name);
- if (v_addr == (intptr_t)MAP_FAILED) {
+ v_addr = fslmc_vfio_map_mcp_obj(dev->device.name);
+ if (v_addr == MAP_FAILED) {
DPAA2_BUS_ERR("Error mapping region (errno = %d)", errno);
ret = -1;
goto cleanup;
}
- /* check the MC version compatibility */
- dpmng.regs = (void *)v_addr;
+ ret = fslmc_vfio_get_bus_info(v_addr, &fslmc_bus_info);
+ if (ret)
+ goto cleanup;
/* In case of secondary processes, MC version check is no longer
* required.
@@ -1509,23 +1649,7 @@ fslmc_process_mcp(struct rte_dpaa2_device *dev)
return 0;
}
- if (mc_get_version(&dpmng, CMD_PRI_LOW, &mc_ver_info)) {
- DPAA2_BUS_ERR("Unable to obtain MC version");
- ret = -1;
- goto cleanup;
- }
-
- if ((mc_ver_info.major != MC_VER_MAJOR) ||
- (mc_ver_info.minor < MC_VER_MINOR)) {
- DPAA2_BUS_ERR("DPAA2 MC version not compatible!"
- " Expected %d.%d.x, Detected %d.%d.%d",
- MC_VER_MAJOR, MC_VER_MINOR,
- mc_ver_info.major, mc_ver_info.minor,
- mc_ver_info.revision);
- ret = -1;
- goto cleanup;
- }
- rte_mcp_ptr_list[MC_PORTAL_INDEX] = (void *)v_addr;
+ rte_mcp_ptr_list[MC_PORTAL_INDEX] = v_addr;
return 0;
@@ -1567,7 +1691,7 @@ fslmc_vfio_close_group(void)
case DPAA2_CRYPTO:
case DPAA2_QDMA:
case DPAA2_IO:
- fslmc_close_iodevices(dev, vfio_group_fd);
+ fslmc_vfio_close_iodevices(dev, vfio_group_fd);
break;
case DPAA2_CON:
case DPAA2_CI:
@@ -1576,7 +1700,7 @@ fslmc_vfio_close_group(void)
if (rte_eal_process_type() == RTE_PROC_SECONDARY)
continue;
- fslmc_close_iodevices(dev, vfio_group_fd);
+ fslmc_vfio_close_iodevices(dev, vfio_group_fd);
break;
case DPAA2_DPRTC:
default:
@@ -1639,7 +1763,7 @@ fslmc_vfio_process_group(void)
}
if (!found_mportal) {
- ret = fslmc_process_mcp(dev);
+ ret = fslmc_vfio_process_mcp(dev);
if (ret) {
DPAA2_BUS_ERR("Unable to map MC Portal");
return ret;
@@ -1668,7 +1792,7 @@ fslmc_vfio_process_group(void)
current_device = 0;
RTE_BUS_FOREACH_DEV(dev, &rte_fslmc_bus) {
if (dev->dev_type == DPAA2_DPRC) {
- ret = fslmc_process_iodevices(dev);
+ ret = fslmc_vfio_process_iodevices(dev);
if (ret) {
DPAA2_BUS_ERR("Unable to process dprc");
return ret;
@@ -1700,7 +1824,7 @@ fslmc_vfio_process_group(void)
case DPAA2_ETH:
case DPAA2_CRYPTO:
case DPAA2_QDMA:
- ret = fslmc_process_iodevices(dev);
+ ret = fslmc_vfio_process_iodevices(dev);
if (ret) {
DPAA2_BUS_DEBUG("Dev (%s) init failed",
dev->device.name);
@@ -1723,7 +1847,7 @@ fslmc_vfio_process_group(void)
/* Call the object creation routine and remove the
* device entry from device list
*/
- ret = fslmc_process_iodevices(dev);
+ ret = fslmc_vfio_process_iodevices(dev);
if (ret) {
DPAA2_BUS_DEBUG("Dev (%s) init failed",
dev->device.name);
@@ -1747,7 +1871,7 @@ fslmc_vfio_process_group(void)
}
}
- ret = fslmc_process_iodevices(dev);
+ ret = fslmc_vfio_process_iodevices(dev);
if (ret) {
DPAA2_BUS_DEBUG("Dev (%s) init failed",
dev->device.name);
diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h
index c995fd67b8..554123ffa6 100644
--- a/drivers/bus/fslmc/fslmc_vfio.h
+++ b/drivers/bus/fslmc/fslmc_vfio.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016,2019-2023 NXP
+ * Copyright 2016,2019-2026 NXP
*
*/
@@ -9,6 +9,8 @@
#define _FSLMC_VFIO_H_
#include <rte_compat.h>
+#include <rte_vfio.h>
+#include <rte_interrupts.h>
/* Pathname of FSL-MC devices directory. */
#define SYSFS_FSL_MC_DEVICES "/sys/bus/fsl-mc/devices"
@@ -44,6 +46,30 @@ struct fslmc_vfio_container {
extern char *fslmc_container;
+extern uint32_t dpaa2_svr_family;
+extern uint32_t dpaa2_dqrr_size;
+extern uint32_t dpaa2_eqcr_size;
+extern uint32_t dpaa2_cluster_base;
+extern uint32_t dpaa2_cluster_size;
+
+#define DPAA2_SVR_MASK 0xffff0000
+
+#define SVR_LS1080A 0x87030000
+#define SVR_LS2080A 0x87010000
+#define SVR_LS2088A 0x87090000
+#define SVR_LX2160A 0x87360000
+
+#define MC_MAJOR_OFFSET 32
+#define MC_MINOR_OFFSET 16
+#define MC_MINOR_MASK ((((uint64_t)1) << (MC_MAJOR_OFFSET - MC_MINOR_OFFSET)) - 1)
+#define MC_REVISION_MASK ((((uint64_t)1) << MC_MINOR_OFFSET) - 1)
+#define RTE_FSL_MC_REV(major, minor, revision) \
+ ((((uint64_t)(major)) << MC_MAJOR_OFFSET) + \
+ (((uint64_t)(minor)) << MC_MINOR_OFFSET) + (revision))
+#define RTE_FSL_MC_REV_MAJOR(rev) ((uint32_t)((rev) >> MC_MAJOR_OFFSET))
+#define RTE_FSL_MC_REV_MINOR(rev) ((uint32_t)(((rev) >> MC_MINOR_OFFSET) & MC_MINOR_MASK))
+#define RTE_FSL_MC_REV_REVISION(rev) ((uint32_t)((rev) & MC_REVISION_MASK))
+
__rte_internal
int rte_dpaa2_intr_enable(struct rte_intr_handle *intr_handle, int index);
@@ -60,4 +86,6 @@ int fslmc_vfio_close_group(void);
char *fslmc_get_container(void);
int fslmc_get_container_group(const char *group_name, int *gropuid);
int fslmc_vfio_dmamap(void);
+int fslmc_vfio_core_cluster_sdest(uint32_t cpu_id);
+
#endif /* _FSLMC_VFIO_H_ */
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
index e17050b625..20d1bb2a66 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2022 NXP
+ * Copyright 2016-2026 NXP
*
*/
#include <uapi/linux/vfio.h>
@@ -63,45 +63,9 @@ static struct dpio_dev_list dpio_dev_list
= TAILQ_HEAD_INITIALIZER(dpio_dev_list); /*!< DPIO device list */
static uint32_t io_space_count;
-/* Variable to store DPAA2 platform type */
-RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_svr_family)
-uint32_t dpaa2_svr_family;
-
-/* Variable to store DPAA2 DQRR size */
-RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dqrr_size)
-uint8_t dpaa2_dqrr_size;
-/* Variable to store DPAA2 EQCR size */
-RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_eqcr_size)
-uint8_t dpaa2_eqcr_size;
-
/* Variable to hold the portal_key, once created.*/
static pthread_key_t dpaa2_portal_key;
-/*Stashing Macros default for LS208x*/
-static int dpaa2_core_cluster_base = 0x04;
-static int dpaa2_cluster_sz = 2;
-
-/* For LS208X platform There are four clusters with following mapping:
- * Cluster 1 (ID = x04) : CPU0, CPU1;
- * Cluster 2 (ID = x05) : CPU2, CPU3;
- * Cluster 3 (ID = x06) : CPU4, CPU5;
- * Cluster 4 (ID = x07) : CPU6, CPU7;
- */
-/* For LS108X platform There are two clusters with following mapping:
- * Cluster 1 (ID = x02) : CPU0, CPU1, CPU2, CPU3;
- * Cluster 2 (ID = x03) : CPU4, CPU5, CPU6, CPU7;
- */
-/* For LX2160 platform There are four clusters with following mapping:
- * Cluster 1 (ID = x00) : CPU0, CPU1;
- * Cluster 2 (ID = x01) : CPU2, CPU3;
- * Cluster 3 (ID = x02) : CPU4, CPU5;
- * Cluster 4 (ID = x03) : CPU6, CPU7;
- * Cluster 1 (ID = x04) : CPU8, CPU9;
- * Cluster 2 (ID = x05) : CPU10, CP11;
- * Cluster 3 (ID = x06) : CPU12, CPU13;
- * Cluster 4 (ID = x07) : CPU14, CPU15;
- */
-
static struct dpaa2_dpio_dev *get_dpio_dev_from_id(int32_t dpio_id)
{
struct dpaa2_dpio_dev *dpio_dev = NULL;
@@ -141,14 +105,6 @@ dpaa2_get_core_id(void)
return cpu_id;
}
-static int
-dpaa2_core_cluster_sdest(int cpu_id)
-{
- int x = cpu_id / dpaa2_cluster_sz;
-
- return dpaa2_core_cluster_base + x;
-}
-
static void
dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)
{
@@ -307,15 +263,17 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id, bool ethrx
/* Set the STASH Destination depending on Current CPU ID.
* Valid values of SDEST are 4,5,6,7. Where,
*/
- sdest = dpaa2_core_cluster_sdest(cpu_id);
+ sdest = fslmc_vfio_core_cluster_sdest(cpu_id);
DPAA2_BUS_DEBUG("Portal= %d CPU= %u SDEST= %d",
dpio_dev->index, cpu_id, sdest);
+ if (sdest < 0)
+ return sdest;
ret = dpio_set_stashing_destination(dpio_dev->dpio, CMD_PRI_LOW,
dpio_dev->token, sdest);
if (ret) {
DPAA2_BUS_ERR("%d ERROR in SDEST", ret);
- return -1;
+ return ret;
}
#ifdef RTE_EVENT_DPAA2
@@ -542,33 +500,6 @@ dpaa2_create_dpio_device(int vdev_fd,
goto err;
}
- /* find the SoC type for the first time */
- if (!dpaa2_svr_family) {
- struct mc_soc_version mc_plat_info = {0};
-
- if (mc_get_soc_version(dpio_dev->dpio,
- CMD_PRI_LOW, &mc_plat_info)) {
- DPAA2_BUS_ERR("Unable to get SoC version information");
- } else if ((mc_plat_info.svr & 0xffff0000) == SVR_LS1080A) {
- dpaa2_core_cluster_base = 0x02;
- dpaa2_cluster_sz = 4;
- DPAA2_BUS_DEBUG("LS108x (A53) Platform Detected");
- } else if ((mc_plat_info.svr & 0xffff0000) == SVR_LX2160A) {
- dpaa2_core_cluster_base = 0x00;
- dpaa2_cluster_sz = 2;
- DPAA2_BUS_DEBUG("LX2160 Platform Detected");
- }
- dpaa2_svr_family = (mc_plat_info.svr & 0xffff0000);
-
- if (dpaa2_svr_family == SVR_LX2160A) {
- dpaa2_dqrr_size = DPAA2_LX2_DQRR_RING_SIZE;
- dpaa2_eqcr_size = DPAA2_LX2_EQCR_RING_SIZE;
- } else {
- dpaa2_dqrr_size = DPAA2_DQRR_RING_SIZE;
- dpaa2_eqcr_size = DPAA2_EQCR_RING_SIZE;
- }
- }
-
if (dpaa2_svr_family == SVR_LX2160A)
reg_info.index = DPAA2_SWP_CENA_MEM_REGION;
else
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
index 7e2271cc35..2c559e5354 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
@@ -35,13 +35,6 @@ RTE_DECLARE_PER_LCORE(struct dpaa2_io_portal_t, _dpaa2_io);
#define DPAA2_PER_LCORE_DQRR_MBUF(i) \
RTE_PER_LCORE(_dpaa2_io).dpio_dev->dpaa2_held_bufs.mbuf[i]
-/* Variable to store DPAA2 DQRR size */
-extern uint8_t dpaa2_dqrr_size;
-/* Variable to store DPAA2 EQCR size */
-extern uint8_t dpaa2_eqcr_size;
-
-extern struct dpaa2_io_portal_t dpaa2_io_portal[RTE_MAX_LCORE];
-
/* Affine a DPIO portal to current processing thread */
__rte_internal
int dpaa2_affine_qbman_swp(void);
diff --git a/drivers/bus/fslmc/private.h b/drivers/bus/fslmc/private.h
index 20a454c3fc..1c8a3e59e9 100644
--- a/drivers/bus/fslmc/private.h
+++ b/drivers/bus/fslmc/private.h
@@ -11,4 +11,7 @@
extern struct rte_bus rte_fslmc_bus;
+/* MC/SoC version and capability info, shared by the bus and VFIO code. */
+extern struct rte_fslmc_bus_info fslmc_bus_info;
+
#endif /* BUS_FSLMC_PRIVATE_H */
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 1df84e1316..e6ce29b2ea 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -329,24 +329,21 @@ dpaa2_fw_version_get(struct rte_eth_dev *dev,
char *fw_version, size_t fw_size)
{
int ret;
+ uint32_t major, minor, rev;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
struct fsl_mc_io *dpni = dev->process_private;
struct mc_soc_version mc_plat_info = {0};
- struct mc_version mc_ver_info = {0};
PMD_INIT_FUNC_TRACE();
+ major = RTE_FSL_MC_REV_MAJOR(priv->mc_rev);
+ minor = RTE_FSL_MC_REV_MINOR(priv->mc_rev);
+ rev = RTE_FSL_MC_REV_REVISION(priv->mc_rev);
if (mc_get_soc_version(dpni, CMD_PRI_LOW, &mc_plat_info))
DPAA2_PMD_WARN("\tmc_get_soc_version failed");
- if (mc_get_version(dpni, CMD_PRI_LOW, &mc_ver_info))
- DPAA2_PMD_WARN("\tmc_get_version failed");
-
- ret = snprintf(fw_version, fw_size,
- "%x-%d.%d.%d",
- mc_plat_info.svr,
- mc_ver_info.major,
- mc_ver_info.minor,
- mc_ver_info.revision);
+ ret = snprintf(fw_version, fw_size, "%x-%d.%d.%d",
+ mc_plat_info.svr, major, minor, rev);
if (ret < 0)
return -EINVAL;
@@ -3537,6 +3534,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
goto init_err;
}
+ priv->mc_rev = dpaa2_dev->bus_info->mc_rev;
priv->num_rx_tc = attr.num_rx_tcs;
priv->num_tx_tc = attr.num_tx_tcs;
priv->qos_entries = attr.qos_entries;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 6c72b9012c..f1d1967cd1 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -528,6 +528,7 @@ struct dpaa2_dev_priv {
uint32_t *cnt_idx_dma_mem;
uint64_t *cnt_values_dma_mem;
uint64_t cnt_idx_iova, cnt_values_iova;
+ uint64_t mc_rev;
struct dpaa2_dev_flow *curr;
LIST_HEAD(, dpaa2_dev_flow) flows;
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 26c84e7125..f417edf52b 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -1123,9 +1123,9 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
/* Function receive frames for a given device and VQ */
struct dpaa2_queue *dpaa2_q = queue;
struct qbman_result *dq_storage;
- uint32_t fqid = dpaa2_q->fqid;
- int ret, num_rx = 0, next_pull = nb_pkts, num_pulled;
- uint8_t pending, status;
+ uint32_t fqid = dpaa2_q->fqid, next_pull = nb_pkts;
+ int ret;
+ uint8_t pending, status, num_rx = 0, num_pulled;
struct qbman_swp *swp;
const struct qbman_fd *fd;
struct qbman_pull_desc pulldesc;
@@ -1269,8 +1269,8 @@ uint16_t dpaa2_dev_tx_conf(void *queue)
struct dpaa2_queue *dpaa2_q = (struct dpaa2_queue *)queue;
struct qbman_result *dq_storage;
uint32_t fqid = dpaa2_q->fqid;
- int ret, num_tx_conf = 0, num_pulled;
- uint8_t pending, status;
+ int ret, num_tx_conf = 0;
+ uint8_t pending, status, num_pulled;
struct qbman_swp *swp;
const struct qbman_fd *fd;
struct qbman_pull_desc pulldesc;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 28/45] drivers: identify dpaa2 soft parser protocol
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (26 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 27/45] drivers: refresh dpaa2 MC and SoC version info Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 29/45] drivers: assign dpaa2 Rx CGID per traffic class Prashant Gupta
` (16 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Identify if soft parser protocol firmware is loaded by checking
firmware magic number.
Enable flow/parser of ECPRI, GENEVE, ROCEv2 and VXLAN's spec
only with soft parser protocol firmware being loaded.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/fslmc_bus.c | 91 +++++++++++++++++++++-
drivers/net/dpaa2/dpaa2_ethdev.c | 2 +
drivers/net/dpaa2/dpaa2_flow.c | 129 ++++++++-----------------------
drivers/net/dpaa2/dpaa2_rxtx.c | 10 +++
4 files changed, 134 insertions(+), 98 deletions(-)
diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index 2427421075..c3415bc541 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
- * Copyright 2016,2018-2021 NXP
+ * Copyright 2016,2018-2026 NXP
*
*/
@@ -8,6 +8,7 @@
#include <dirent.h>
#include <stdalign.h>
#include <stdbool.h>
+#include <sys/mman.h>
#include <eal_export.h>
#include <rte_log.h>
@@ -35,6 +36,93 @@ struct rte_fslmc_bus_info fslmc_bus_info;
RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_seqn_dynfield_offset)
int dpaa2_seqn_dynfield_offset = -1;
+/** For LX2160A, LS2088A and LS1088A*/
+#define WRIOP_CCSR_BASE 0x8b80000
+#define WRIOP_CCSR_CTLU_OFFSET 0
+#define WRIOP_CCSR_CTLU_PARSER_OFFSET 0
+#define WRIOP_CCSR_CTLU_PARSER_INGRESS_OFFSET 0
+
+#define WRIOP_INGRESS_PARSER_PHY \
+ (WRIOP_CCSR_BASE + WRIOP_CCSR_CTLU_OFFSET + \
+ WRIOP_CCSR_CTLU_PARSER_OFFSET + \
+ WRIOP_CCSR_CTLU_PARSER_INGRESS_OFFSET)
+
+struct __rte_packed_begin dpaa2_parser_ccsr {
+ uint32_t psr_cfg;
+ uint32_t psr_idle;
+ uint32_t psr_pclm;
+ uint8_t psr_ver_min;
+ uint8_t psr_ver_maj;
+ uint8_t psr_id1_l;
+ uint8_t psr_id1_h;
+ uint32_t psr_rev2;
+ uint8_t rsv[0x2c];
+ uint8_t sp_ins[4032];
+} __rte_packed_end;
+
+#define SP_PROTOCOL_MAGIC_DATA 0xabcd
+#define SP_PROTOCOL_MAGIC_OFFSET 0x4
+
+/** Number of soft parser firmware bytes reported for debugging. */
+#define SP_PRINT_LEN 128
+/** Firmware bytes logged per debug line. */
+#define SP_DUMP_BYTES_PER_LINE 16
+
+static void
+fslmc_soft_parser_protocol_supported(void)
+{
+ int fd, i, pos = 0;
+ void *map_addr = NULL;
+ const struct dpaa2_parser_ccsr *parser_ccsr = NULL;
+ const uint16_t *magic_num;
+ char line[SP_DUMP_BYTES_PER_LINE * 3 + 1];
+
+ fd = open("/dev/mem", O_RDWR | O_SYNC);
+ if (fd < 0) {
+ DPAA2_BUS_ERR("open \"/dev/mem\" ERROR(%d)", fd);
+ goto exit;
+ }
+
+ map_addr = mmap(NULL, sizeof(struct dpaa2_parser_ccsr),
+ PROT_READ | PROT_WRITE, MAP_SHARED, fd,
+ WRIOP_INGRESS_PARSER_PHY);
+ parser_ccsr = map_addr;
+ if (!parser_ccsr) {
+ DPAA2_BUS_ERR("Map 0x%" PRIx64 "(size=0x%zx) failed",
+ (uint64_t)WRIOP_INGRESS_PARSER_PHY,
+ sizeof(struct dpaa2_parser_ccsr));
+ goto exit;
+ }
+
+ DPAA2_BUS_DEBUG("Soft ParserID:0x%02x%02x, Rev:maj(%02x), min(%02x)",
+ parser_ccsr->psr_id1_h, parser_ccsr->psr_id1_l,
+ parser_ccsr->psr_ver_maj, parser_ccsr->psr_ver_min);
+
+ magic_num = (const void *)&parser_ccsr->sp_ins[SP_PROTOCOL_MAGIC_OFFSET];
+ if (*magic_num == SP_PROTOCOL_MAGIC_DATA) {
+ fslmc_bus_info.sp_protocol = true;
+ DPAA2_BUS_INFO("Soft parser protocol support.");
+ DPAA2_BUS_DEBUG("First %d bytes of sp protocol firmware:",
+ SP_PRINT_LEN);
+ for (i = 0; i < SP_PRINT_LEN; i++) {
+ pos += snprintf(&line[pos], sizeof(line) - pos,
+ "%02x ", parser_ccsr->sp_ins[i]);
+ if ((i + 1) % SP_DUMP_BYTES_PER_LINE == 0) {
+ DPAA2_BUS_DEBUG("%s", line);
+ pos = 0;
+ }
+ }
+ if (pos)
+ DPAA2_BUS_DEBUG("%s", line);
+ }
+
+exit:
+ if (map_addr)
+ munmap(map_addr, sizeof(struct dpaa2_parser_ccsr));
+ if (fd >= 0)
+ close(fd);
+}
+
RTE_EXPORT_INTERNAL_SYMBOL(rte_fslmc_get_device_count)
uint32_t
rte_fslmc_get_device_count(enum rte_dpaa2_dev_type device_type)
@@ -345,6 +433,7 @@ rte_fslmc_scan(void)
DPAA2_BUS_ERR("Unable to open VFIO group directory");
goto scan_fail;
}
+ fslmc_soft_parser_protocol_supported();
/* Scan the DPRC container object */
ret = scan_one_fslmc_device(group_name);
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index e6ce29b2ea..fb1d01e209 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3735,6 +3735,8 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
}
eth_dev->data->mtu = RTE_ETHER_MTU;
+ priv->sp_protocol = dpaa2_dev->bus_info->sp_protocol;
+
DPAA2_PMD_INFO("%s: netdev created, connected to %s",
eth_dev->data->name, dpaa2_dev->ep_name);
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 2662c358e4..cde2f5a9d3 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -9,7 +9,6 @@
#include <string.h>
#include <unistd.h>
#include <stdarg.h>
-#include <sys/mman.h>
#include <rte_ethdev.h>
#include <rte_log.h>
@@ -22,9 +21,9 @@
#include <dpaa2_ethdev.h>
#include <dpaa2_pmd_logs.h>
+#include "dpaa2_parser_decode.h"
static char *dpaa2_flow_control_log;
-static int dpaa2_sp_loaded = -1;
/* Default size of a key */
#define DPNI_DEFAULT_KEY_SIZE 24
@@ -434,92 +433,6 @@ dpaa2_flow_fs_entry_log(const char *log_info,
DPAA2_FLOW_DUMP("\r\n");
}
-/** For LX2160A, LS2088A and LS1088A*/
-#define WRIOP_CCSR_BASE 0x8b80000
-#define WRIOP_CCSR_CTLU_OFFSET 0
-#define WRIOP_CCSR_CTLU_PARSER_OFFSET 0
-#define WRIOP_CCSR_CTLU_PARSER_INGRESS_OFFSET 0
-
-#define WRIOP_INGRESS_PARSER_PHY \
- (WRIOP_CCSR_BASE + WRIOP_CCSR_CTLU_OFFSET + \
- WRIOP_CCSR_CTLU_PARSER_OFFSET + \
- WRIOP_CCSR_CTLU_PARSER_INGRESS_OFFSET)
-
-struct dpaa2_parser_ccsr {
- uint32_t psr_cfg;
- uint32_t psr_idle;
- uint32_t psr_pclm;
- uint8_t psr_ver_min;
- uint8_t psr_ver_maj;
- uint8_t psr_id1_l;
- uint8_t psr_id1_h;
- uint32_t psr_rev2;
- uint8_t rsv[0x2c];
- uint8_t sp_ins[4032];
-};
-
-int
-dpaa2_soft_parser_loaded(void)
-{
- int fd, i, ret = 0;
- struct dpaa2_parser_ccsr *parser_ccsr = NULL;
-
- dpaa2_flow_control_log = getenv("DPAA2_FLOW_CONTROL_LOG");
-
- if (dpaa2_sp_loaded >= 0)
- return dpaa2_sp_loaded;
-
- fd = open("/dev/mem", O_RDWR | O_SYNC);
- if (fd < 0) {
- DPAA2_PMD_ERR("open \"/dev/mem\" ERROR(%d)", fd);
- ret = fd;
- goto exit;
- }
-
- parser_ccsr = mmap(NULL, sizeof(struct dpaa2_parser_ccsr),
- PROT_READ | PROT_WRITE, MAP_SHARED, fd,
- WRIOP_INGRESS_PARSER_PHY);
- if (!parser_ccsr) {
- DPAA2_PMD_ERR("Map 0x%" PRIx64 "(size=0x%x) failed",
- (uint64_t)WRIOP_INGRESS_PARSER_PHY,
- (uint32_t)sizeof(struct dpaa2_parser_ccsr));
- ret = -ENOBUFS;
- goto exit;
- }
-
- DPAA2_PMD_INFO("Parser ID:0x%02x%02x, Rev:major(%02x), minor(%02x)",
- parser_ccsr->psr_id1_h, parser_ccsr->psr_id1_l,
- parser_ccsr->psr_ver_maj, parser_ccsr->psr_ver_min);
-
- if (dpaa2_flow_control_log) {
- for (i = 0; i < 64; i++) {
- DPAA2_FLOW_DUMP("%02x ",
- parser_ccsr->sp_ins[i]);
- if (!((i + 1) % 16))
- DPAA2_FLOW_DUMP("\r\n");
- }
- }
-
- for (i = 0; i < 16; i++) {
- if (parser_ccsr->sp_ins[i]) {
- dpaa2_sp_loaded = 1;
- break;
- }
- }
- if (dpaa2_sp_loaded < 0)
- dpaa2_sp_loaded = 0;
-
- ret = dpaa2_sp_loaded;
-
-exit:
- if (parser_ccsr)
- munmap(parser_ccsr, sizeof(struct dpaa2_parser_ccsr));
- if (fd >= 0)
- close(fd);
-
- return ret;
-}
-
static int
dpaa2_flow_ip_address_extract(enum net_prot prot,
uint32_t field)
@@ -3538,6 +3451,11 @@ dpaa2_configure_flow_gre(struct dpaa2_dev_flow *flow,
return 0;
}
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("vXLAN flow with spec is not supported without SP.");
+ return -ENOTSUP;
+ }
+
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_GRE);
if (ret) {
@@ -3583,6 +3501,11 @@ dpaa2_configure_flow_vxlan(struct dpaa2_dev_flow *flow,
struct dpaa2_dev_priv *priv = dev->data->dev_private;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("eCPRI flow is not supported without SP.");
+ return -ENOTSUP;
+ }
+
group = attr->group;
/* Parse pattern list to get the matching parameters */
@@ -3681,6 +3604,11 @@ dpaa2_configure_flow_ecpri(struct dpaa2_dev_flow *flow,
uint8_t extract_size[DPAA2_ECPRI_MAX_EXTRACT_NB];
uint8_t extract_off[DPAA2_ECPRI_MAX_EXTRACT_NB];
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("ROCEV2 flow is not supported without SP.");
+ return -ENOTSUP;
+ }
+
group = attr->group;
/* Parse pattern list to get the matching parameters */
@@ -4000,6 +3928,11 @@ dpaa2_configure_flow_gtp(struct dpaa2_dev_flow *flow,
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("GENEVE flow is not supported without SP.");
+ return -ENOTSUP;
+ }
+
group = attr->group;
/* Parse pattern list to get the matching parameters */
@@ -4725,7 +4658,7 @@ dpaa2_configure_qos_table(struct dpaa2_dev_priv *priv,
static int
dpaa2_flow_item_convert(const struct rte_flow_item pattern[],
- struct rte_dpaa2_flow_item **dpaa2_pattern)
+ struct rte_dpaa2_flow_item **dpaa2_pattern, int sp_protocol)
{
struct rte_dpaa2_flow_item *new_pattern;
int num = 0, tunnel_start = 0;
@@ -4737,7 +4670,7 @@ dpaa2_flow_item_convert(const struct rte_flow_item pattern[],
}
new_pattern = rte_malloc(NULL, sizeof(struct rte_dpaa2_flow_item) * num,
- RTE_CACHE_LINE_SIZE);
+ RTE_CACHE_LINE_SIZE);
if (!new_pattern) {
DPAA2_PMD_ERR("Failed to alloc %d flow items", num);
return -ENOMEM;
@@ -4745,13 +4678,13 @@ dpaa2_flow_item_convert(const struct rte_flow_item pattern[],
num = 0;
while (pattern[num].type != RTE_FLOW_ITEM_TYPE_END) {
- memcpy(&new_pattern[num].generic_item, &pattern[num],
- sizeof(struct rte_flow_item));
+ rte_memcpy(&new_pattern[num].generic_item, &pattern[num],
+ sizeof(struct rte_flow_item));
new_pattern[num].in_tunnel = 0;
if (pattern[num].type == RTE_FLOW_ITEM_TYPE_VXLAN)
tunnel_start = 1;
- else if (tunnel_start)
+ else if (tunnel_start && sp_protocol)
new_pattern[num].in_tunnel = 1;
num++;
}
@@ -4788,7 +4721,8 @@ dpaa2_generic_flow_set(struct dpaa2_dev_flow *flow,
if (ret)
return ret;
- ret = dpaa2_flow_item_convert(pattern, &dpaa2_pattern);
+ ret = dpaa2_flow_item_convert(pattern, &dpaa2_pattern,
+ priv->sp_protocol);
if (ret)
return ret;
@@ -5129,7 +5063,8 @@ dpaa2_dev_verify_attr(struct dpni_attr *dpni_attr,
}
static inline int
-dpaa2_dev_verify_patterns(const struct rte_flow_item pattern[])
+dpaa2_dev_verify_patterns(const struct rte_flow_item pattern[],
+ int sp_support)
{
unsigned int i, j, is_found = 0;
int ret = 0;
@@ -5153,7 +5088,7 @@ dpaa2_dev_verify_patterns(const struct rte_flow_item pattern[])
}
if (is_found)
continue;
- if (dpaa2_sp_loaded > 0) {
+ if (sp_support) {
for (i = 0; i < sp_supported_num; i++) {
if (sp_supported[i] == pattern[j].type) {
is_found = 1;
@@ -5232,7 +5167,7 @@ dpaa2_flow_validate(struct rte_eth_dev *dev,
goto not_valid_params;
}
/* Verify input pattern list */
- ret = dpaa2_dev_verify_patterns(pattern);
+ ret = dpaa2_dev_verify_patterns(pattern, priv->sp_protocol);
if (ret < 0) {
DPAA2_PMD_ERR("Invalid pattern list is given");
rte_flow_error_set(error, EPERM,
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index f417edf52b..ccb4bbe7dc 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -128,6 +128,16 @@ dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
ext_packet_type |= RTE_PTYPE_INNER_L3_IPV4;
}
}
+ if (priv->sp_protocol) {
+ if (frc_parse->fafe2) {
+ frc_parse->fafe2 = 0;
+ ext_packet_type |= RTE_PTYPE_TUNNEL_GENEVE;
+ }
+ if (frc_parse->sum_l.l4.fafe3) {
+ frc_parse->sum_l.l4.fafe3 = 0;
+ ext_packet_type |= RTE_PTYPE_INNER_L3_IPV4;
+ }
+ }
switch (frc) {
case DPAA2_PKT_TYPE_IPV4_UDP:
m->packet_type = RTE_PTYPE_L2_ETHER |
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 29/45] drivers: assign dpaa2 Rx CGID per traffic class
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (27 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 28/45] drivers: identify dpaa2 soft parser protocol Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 30/45] drivers: inherit dpaa2 rxq config for event queue Prashant Gupta
` (15 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
RXQs in same TC should use same CGID for congestion set.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 3 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 252 ++++++++++++------------
drivers/net/dpaa2/dpaa2_ethdev.h | 3 +-
3 files changed, 126 insertions(+), 132 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index f0bc9a3063..12216750a8 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -176,7 +176,6 @@ struct __rte_cache_aligned dpaa2_queue {
uint32_t fqid; /*!< Unique ID of this queue */
uint16_t flow_id; /*!< To be used by DPAA2 framework */
uint8_t tc_index; /*!< traffic class identifier */
- uint8_t cgid; /*! < Congestion Group id for this queue */
uint64_t rx_pkts;
uint64_t tx_pkts;
uint64_t err_pkts;
@@ -186,6 +185,8 @@ struct __rte_cache_aligned dpaa2_queue {
/**Egress*/
struct qbman_result *cscn;
};
+ void *cfg;
+ uint8_t options;
struct rte_event ev;
dpaa2_queue_cb_dqrr_t *cb;
dpaa2_queue_cb_eqresp_free_t *cb_eqresp_free;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index fb1d01e209..0835296ee2 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*/
#include <time.h>
@@ -682,6 +682,9 @@ dpaa2_free_rx_tx_queues(struct rte_eth_dev *dev)
RTE_MAX_LCORE);
dpaa2_queue_storage_free(dpaa2_q,
RTE_MAX_LCORE);
+ if (dpaa2_q->cfg)
+ rte_free(dpaa2_q->cfg);
+ dpaa2_q->cfg = NULL;
priv->rx_vq[i] = NULL;
}
/* cleanup tx queue cscn */
@@ -934,11 +937,12 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
struct fsl_mc_io *dpni = dev->process_private;
bool dpcon_allocated = false;
struct dpaa2_queue *dpaa2_q;
- struct dpni_queue cfg;
- uint8_t options = 0;
- uint8_t flow_id;
+ struct dpni_queue *cfg;
+ struct dpni_taildrop taildrop;
+ uint8_t qopt = 0;
+ uint16_t flow_id;
uint32_t bpid;
- int i, ret;
+ int ret;
PMD_INIT_FUNC_TRACE();
@@ -975,6 +979,10 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
dev->data->rx_queues[rx_queue_id] = dpaa2_q;
return 0;
}
+ cfg = rte_zmalloc(NULL, sizeof(struct dpni_queue), 0);
+ if (!cfg)
+ return -ENOMEM;
+ dpaa2_q->cfg = cfg;
dpaa2_q->mb_pool = mb_pool; /**< mbuf pool to populate RX ring. */
dpaa2_q->bp_array = rte_dpaa2_bpid_info;
dpaa2_q->offloads = rx_conf->offloads;
@@ -1004,124 +1012,100 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
/*Get the flow id from given VQ id*/
flow_id = dpaa2_q->flow_id;
- memset(&cfg, 0, sizeof(struct dpni_queue));
+ memset(cfg, 0, sizeof(struct dpni_queue));
- options = options | DPNI_QUEUE_OPT_USER_CTX;
- cfg.user_context = (size_t)(dpaa2_q);
+ qopt |= DPNI_QUEUE_OPT_USER_CTX;
+ cfg->user_context = (size_t)(dpaa2_q);
/* napi: schedule the FQ to its DPCON once, when the channel is first
* grabbed; never re-bound (the MC refuses scheduled->parked at runtime)
*/
if (dpcon_allocated) {
- options |= DPNI_QUEUE_OPT_DEST;
- cfg.destination.type = DPNI_DEST_DPCON;
- cfg.destination.id = dpaa2_q->napi_dpcon->dpcon_id;
- }
-
- /* check if a private cgr available. */
- for (i = 0; i < priv->max_cgs; i++) {
- if (!priv->cgid_in_use[i]) {
- priv->cgid_in_use[i] = 1;
- break;
- }
+ qopt |= DPNI_QUEUE_OPT_DEST;
+ cfg->destination.type = DPNI_DEST_DPCON;
+ cfg->destination.id = dpaa2_q->napi_dpcon->dpcon_id;
}
- if (i < priv->max_cgs) {
- options |= DPNI_QUEUE_OPT_SET_CGID;
- cfg.cgid = i;
- dpaa2_q->cgid = cfg.cgid;
+ /** RXQs in same TC share same cgid.*/
+ if (dpaa2_q->tc_index < priv->max_cgs) {
+ qopt |= DPNI_QUEUE_OPT_SET_CGID;
+ cfg->cgid = dpaa2_q->tc_index;
+ priv->cgid_in_use[dpaa2_q->tc_index]++;
} else {
- dpaa2_q->cgid = DPAA2_INVALID_CGID;
+ cfg->cgid = DPAA2_INVALID_CGID;
}
/*if ls2088 or rev2 device, enable the stashing */
if ((dpaa2_svr_family & 0xffff0000) != SVR_LS2080A) {
- options |= DPNI_QUEUE_OPT_FLC;
- cfg.flc.stash_control = true;
- dpaa2_flc_stashing_clear_all(&cfg.flc.value);
+ qopt |= DPNI_QUEUE_OPT_FLC;
+ cfg->flc.stash_control = true;
+ dpaa2_flc_stashing_clear_all(&cfg->flc.value);
if (priv->flags & DPAA2_DATA_STASHING_OFF) {
dpaa2_flc_stashing_set(DPAA2_FLC_DATA_STASHING, 0,
- &cfg.flc.value);
+ &cfg->flc.value);
dpaa2_q->data_stashing_off = 1;
} else {
dpaa2_flc_stashing_set(DPAA2_FLC_DATA_STASHING, 1,
- &cfg.flc.value);
+ &cfg->flc.value);
dpaa2_q->data_stashing_off = 0;
}
if ((dpaa2_svr_family & 0xffff0000) != SVR_LX2160A) {
dpaa2_flc_stashing_set(DPAA2_FLC_ANNO_STASHING, 1,
- &cfg.flc.value);
+ &cfg->flc.value);
}
}
+
ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token, DPNI_QUEUE_RX,
- dpaa2_q->tc_index, flow_id, options, &cfg);
+ dpaa2_q->tc_index, flow_id, qopt, cfg);
if (ret) {
+ rte_free(dpaa2_q->cfg);
+ dpaa2_q->cfg = NULL;
DPAA2_PMD_ERR("Error in setting the rx flow: = %d", ret);
goto err_free_dpcon;
}
dpaa2_q->nb_desc = nb_rx_desc;
-
- if (!(priv->flags & DPAA2_RX_TAILDROP_OFF)) {
- struct dpni_taildrop taildrop;
-
+ memset(&taildrop, 0, sizeof(struct dpni_taildrop));
+ if (!(priv->flags & DPAA2_RX_TAILDROP_OFF))
taildrop.enable = 1;
- /* Private CGR will use tail drop length as nb_rx_desc.
- * for rest cases we can use standard byte based tail drop.
- * There is no HW restriction, but number of CGRs are limited,
- * hence this restriction is placed.
- */
- if (dpaa2_q->cgid != DPAA2_INVALID_CGID) {
- /*enabling per rx queue congestion control */
- taildrop.threshold = nb_rx_desc;
- taildrop.units = DPNI_CONGESTION_UNIT_FRAMES;
- taildrop.oal = 0;
- DPAA2_PMD_DEBUG("Enabling CG Tail Drop on queue = %d",
- rx_queue_id);
- ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
- DPNI_CP_CONGESTION_GROUP,
- DPNI_QUEUE_RX,
- dpaa2_q->tc_index,
- dpaa2_q->cgid, &taildrop);
- } else {
- /*enabling per rx queue congestion control */
- taildrop.threshold = CONG_THRESHOLD_RX_BYTES_Q;
- taildrop.units = DPNI_CONGESTION_UNIT_BYTES;
- taildrop.oal = CONG_RX_OAL;
- DPAA2_PMD_DEBUG("Enabling Byte based Drop on queue= %d",
- rx_queue_id);
- ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
- DPNI_CP_QUEUE, DPNI_QUEUE_RX,
- dpaa2_q->tc_index, flow_id,
- &taildrop);
- }
- if (ret) {
- DPAA2_PMD_ERR("Error in setting taildrop. err=(%d)",
- ret);
- goto err_free_dpcon;
- }
- } else { /* Disable tail Drop */
- struct dpni_taildrop taildrop = {0};
- DPAA2_PMD_INFO("Tail drop is disabled on queue");
-
- taildrop.enable = 0;
- if (dpaa2_q->cgid != DPAA2_INVALID_CGID) {
- ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
- DPNI_CP_CONGESTION_GROUP, DPNI_QUEUE_RX,
- dpaa2_q->tc_index,
- dpaa2_q->cgid, &taildrop);
- } else {
- ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
- DPNI_CP_QUEUE, DPNI_QUEUE_RX,
- dpaa2_q->tc_index, flow_id, &taildrop);
- }
- if (ret) {
- DPAA2_PMD_ERR("Error in setting taildrop. err=(%d)",
- ret);
- goto err_free_dpcon;
- }
+ /* Private CGR will use tail drop length as nb_rx_desc * queues per TC.
+ * For rest cases we can use standard byte based tail drop.
+ * There is no HW restriction, but number of CGRs are limited,
+ * hence this restriction is placed.
+ */
+ if (cfg->cgid != DPAA2_INVALID_CGID &&
+ priv->cgid_in_use[dpaa2_q->tc_index] == 1) {
+ /*enabling per TC congestion control */
+ taildrop.threshold = nb_rx_desc * priv->dist_queues;
+ taildrop.units = DPNI_CONGESTION_UNIT_FRAMES;
+ taildrop.oal = 0;
+ DPAA2_PMD_DEBUG("%s CG Tail Drop on TC%d",
+ taildrop.enable ? "Enabling" : "Disabling",
+ dpaa2_q->tc_index);
+ ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
+ DPNI_CP_CONGESTION_GROUP, DPNI_QUEUE_RX,
+ dpaa2_q->tc_index, cfg->cgid, &taildrop);
+ } else if (cfg->cgid == DPAA2_INVALID_CGID) {
+ /*enabling per rx queue congestion control */
+ taildrop.threshold = CONG_THRESHOLD_RX_BYTES_Q;
+ taildrop.units = DPNI_CONGESTION_UNIT_BYTES;
+ taildrop.oal = CONG_RX_OAL;
+ DPAA2_PMD_DEBUG("%s Byte based Drop on TC[%d].flow%d",
+ taildrop.enable ? "Enabling" : "Disabling",
+ dpaa2_q->tc_index, flow_id);
+ ret = dpni_set_taildrop(dpni, CMD_PRI_LOW, priv->token,
+ DPNI_CP_QUEUE, DPNI_QUEUE_RX,
+ dpaa2_q->tc_index, flow_id, &taildrop);
}
+ if (ret) {
+ rte_free(dpaa2_q->cfg);
+ dpaa2_q->cfg = NULL;
+ DPAA2_PMD_ERR("Error in setting taildrop. err=(%d)", ret);
+ return ret;
+ }
+
+ dpaa2_q->options = qopt;
dev->data->rx_queues[rx_queue_id] = dpaa2_q;
return 0;
@@ -1150,8 +1134,8 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
struct fsl_mc_io *dpni = dev->process_private;
struct dpni_queue tx_conf_cfg;
struct dpni_queue tx_flow_cfg;
- uint8_t options = 0, flow_id;
- uint16_t channel_id;
+ uint8_t qopt = 0;
+ uint16_t channel_id, flow_id;
struct dpni_queue_id qid;
uint32_t tc_id;
int ret;
@@ -1181,7 +1165,7 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
channel_id = priv->tx_channels[priv->num_channels - 1];
ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token, DPNI_QUEUE_TX,
- DPNI_BUILD_PARAM(channel_id, tc_id), flow_id, options, &tx_flow_cfg);
+ DPNI_BUILD_PARAM(channel_id, tc_id), flow_id, qopt, &tx_flow_cfg);
if (ret) {
DPAA2_PMD_ERR("Failed(%d) to set %s's TC[%d].txq[%d]",
ret, dev->data->name, tc_id, flow_id);
@@ -1190,7 +1174,7 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
ret = dpni_get_queue(dpni, CMD_PRI_LOW, priv->token,
DPNI_QUEUE_TX, DPNI_BUILD_PARAM(channel_id, tc_id),
- flow_id, &tx_flow_cfg, &qid);
+ dpaa2_q->flow_id, &tx_flow_cfg, &qid);
if (ret) {
DPAA2_PMD_ERR("Error in getting LFQID err=%d", ret);
return ret;
@@ -1242,12 +1226,12 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
tc_id = dpaa2_tx_conf_q->tc_index;
flow_id = dpaa2_tx_conf_q->flow_id;
dpaa2_q->tx_conf_queue = dpaa2_tx_conf_q;
- options = options | DPNI_QUEUE_OPT_USER_CTX;
+ qopt |= DPNI_QUEUE_OPT_USER_CTX;
tx_conf_cfg.user_context = (size_t)(dpaa2_q);
ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token,
DPNI_QUEUE_TX_CONFIRM,
DPNI_BUILD_PARAM(channel_id, tc_id),
- flow_id, options, &tx_conf_cfg);
+ flow_id, qopt, &tx_conf_cfg);
if (ret) {
DPAA2_PMD_ERR("Set TC[%d].TX[%d] conf flow err=%d",
tc_id, flow_id, ret);
@@ -1296,30 +1280,29 @@ dpaa2_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t rx_queue_id)
struct dpaa2_queue *dpaa2_q = dev->data->rx_queues[rx_queue_id];
struct dpaa2_dev_priv *priv = dpaa2_q->eth_data->dev_private;
struct fsl_mc_io *dpni = priv->eth_dev->process_private;
- uint8_t options = 0;
+ uint8_t qopt = 0;
int ret;
- struct dpni_queue cfg;
+ struct dpni_queue *cfg = dpaa2_q->cfg;
- memset(&cfg, 0, sizeof(struct dpni_queue));
PMD_INIT_FUNC_TRACE();
total_nb_rx_desc -= dpaa2_q->nb_desc;
- if (dpaa2_q->cgid != DPAA2_INVALID_CGID) {
- options = DPNI_QUEUE_OPT_CLEAR_CGID;
- cfg.cgid = dpaa2_q->cgid;
-
- ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token,
- DPNI_QUEUE_RX,
- dpaa2_q->tc_index, dpaa2_q->flow_id,
- options, &cfg);
- if (ret)
- DPAA2_PMD_ERR("Unable to clear CGR from q=%u err=%d",
- dpaa2_q->fqid, ret);
- priv->cgid_in_use[dpaa2_q->cgid] = 0;
- dpaa2_q->cgid = DPAA2_INVALID_CGID;
+ if (cfg) {
+ if (cfg->cgid != DPAA2_INVALID_CGID) {
+ qopt = DPNI_QUEUE_OPT_CLEAR_CGID;
+ ret = dpni_set_queue(dpni, CMD_PRI_LOW, priv->token,
+ DPNI_QUEUE_RX, dpaa2_q->tc_index, dpaa2_q->flow_id,
+ qopt, cfg);
+ if (ret) {
+ DPAA2_PMD_ERR("Unable to clear CGR from TC[%d].flow%d err=%d",
+ dpaa2_q->tc_index, dpaa2_q->flow_id, ret);
+ }
+ priv->cgid_in_use[cfg->cgid]--;
+ }
+ rte_free(cfg);
+ dpaa2_q->cfg = NULL;
}
-
/* keep the DPCON on the FQ across a reconfigure; freed at dev_close */
if (dpaa2_q->napi_dpcon)
dpaa2_dev_rx_queue_intr_unbind(dpaa2_q);
@@ -2833,10 +2816,12 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,
struct fsl_mc_io *dpni = (struct fsl_mc_io *)dev->process_private;
struct dpaa2_queue *dpaa2_ethq = eth_priv->rx_vq[eth_rx_queue_id];
uint8_t flow_id = dpaa2_ethq->flow_id;
- struct dpni_queue cfg;
+ struct dpni_queue *cfg;
uint8_t options, priority;
int ret;
+ cfg = dpaa2_ethq->cfg;
+
if (queue_conf->ev.sched_type == RTE_SCHED_TYPE_PARALLEL)
dpaa2_ethq->cb = dpaa2_dev_process_parallel_event;
else if (queue_conf->ev.sched_type == RTE_SCHED_TYPE_ATOMIC)
@@ -2849,15 +2834,19 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,
priority = (RTE_EVENT_DEV_PRIORITY_LOWEST / queue_conf->ev.priority) *
(dpcon->num_priorities - 1);
- memset(&cfg, 0, sizeof(struct dpni_queue));
+ if (!cfg) {
+ DPAA2_PMD_ERR("%s: %s-rxq%d was not setup yet!",
+ __func__, dev->data->name, eth_rx_queue_id);
+ return -EINVAL;
+ }
options = DPNI_QUEUE_OPT_DEST;
- cfg.destination.type = DPNI_DEST_DPCON;
- cfg.destination.id = dpcon->dpcon_id;
- cfg.destination.priority = priority;
+ cfg->destination.type = DPNI_DEST_DPCON;
+ cfg->destination.id = dpcon->dpcon_id;
+ cfg->destination.priority = priority;
if (queue_conf->ev.sched_type == RTE_SCHED_TYPE_ATOMIC) {
options |= DPNI_QUEUE_OPT_HOLD_ACTIVE;
- cfg.destination.hold_active = 1;
+ cfg->destination.hold_active = 1;
}
if (queue_conf->ev.sched_type == RTE_SCHED_TYPE_ORDERED &&
@@ -2896,16 +2885,16 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,
}
options |= DPNI_QUEUE_OPT_USER_CTX;
- cfg.user_context = (size_t)(dpaa2_ethq);
+ cfg->user_context = (size_t)(dpaa2_ethq);
ret = dpni_set_queue(dpni, CMD_PRI_LOW, eth_priv->token, DPNI_QUEUE_RX,
- dpaa2_ethq->tc_index, flow_id, options, &cfg);
+ dpaa2_ethq->tc_index, flow_id, options, cfg);
if (ret) {
DPAA2_PMD_ERR("Error in dpni_set_queue: ret: %d", ret);
return ret;
}
- memcpy(&dpaa2_ethq->ev, &queue_conf->ev, sizeof(struct rte_event));
+ rte_memcpy(&dpaa2_ethq->ev, &queue_conf->ev, sizeof(struct rte_event));
return 0;
}
@@ -2917,17 +2906,15 @@ int dpaa2_eth_eventq_detach(const struct rte_eth_dev *dev,
struct dpaa2_dev_priv *eth_priv = dev->data->dev_private;
struct fsl_mc_io *dpni = (struct fsl_mc_io *)dev->process_private;
struct dpaa2_queue *dpaa2_ethq = eth_priv->rx_vq[eth_rx_queue_id];
- uint8_t flow_id = dpaa2_ethq->flow_id;
- struct dpni_queue cfg;
- uint8_t options;
+ struct dpni_queue *cfg;
int ret;
- memset(&cfg, 0, sizeof(struct dpni_queue));
- options = DPNI_QUEUE_OPT_DEST;
- cfg.destination.type = DPNI_DEST_NONE;
+ cfg = dpaa2_ethq->cfg;
+ cfg->destination.type = DPNI_DEST_NONE;
+ dpaa2_ethq->options &= ~DPNI_QUEUE_OPT_DEST;
ret = dpni_set_queue(dpni, CMD_PRI_LOW, eth_priv->token, DPNI_QUEUE_RX,
- dpaa2_ethq->tc_index, flow_id, options, &cfg);
+ dpaa2_ethq->tc_index, dpaa2_ethq->flow_id, dpaa2_ethq->options, cfg);
if (ret)
DPAA2_PMD_ERR("Error in dpni_set_queue: ret: %d", ret);
@@ -3547,10 +3534,15 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
rte_spinlock_init(&priv->lpbk_qp_lock);
/* only if the custom CG is enabled */
- if (attr.options & DPNI_OPT_CUSTOM_CG)
+ if (attr.options & DPNI_OPT_CUSTOM_CG) {
priv->max_cgs = attr.num_cgs;
- else
+ if (priv->max_cgs < priv->num_rx_tc) {
+ DPAA2_PMD_WARN("DPNI%d has no enough cgids(%d) to set %d TCs",
+ hw_id, priv->max_cgs, priv->num_rx_tc);
+ }
+ } else {
priv->max_cgs = 0;
+ }
for (i = 0; i < priv->max_cgs; i++)
priv->cgid_in_use[i] = 0;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index f1d1967cd1..3c611652b1 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -478,7 +478,8 @@ struct dpaa2_dev_priv {
uint8_t en_ordered;
uint8_t en_loose_ordered;
uint8_t max_cgs;
- uint8_t cgid_in_use[MAX_RX_QUEUES];
+ /** RXQs in same TC share same cgid.*/
+ uint8_t cgid_in_use[MAX_TCS];
rte_spinlock_t meter_lock;
/* Lowest priority FS flow id to receive flow steering miss frames. */
uint16_t default_flow;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 30/45] drivers: inherit dpaa2 rxq config for event queue
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (28 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 29/45] drivers: assign dpaa2 Rx CGID per traffic class Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 31/45] net/dpaa2: rename Rx queue flags Prashant Gupta
` (14 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
The configuration parameter in event queue attachment should use
flc and cgid configured in rxq setup.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 2 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index 12216750a8..f87bf51d09 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2024 NXP
+ * Copyright 2016-2025 NXP
*
*/
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 0835296ee2..57c6e8e90b 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -2815,11 +2815,12 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,
struct dpaa2_dev_priv *eth_priv = dev->data->dev_private;
struct fsl_mc_io *dpni = (struct fsl_mc_io *)dev->process_private;
struct dpaa2_queue *dpaa2_ethq = eth_priv->rx_vq[eth_rx_queue_id];
- uint8_t flow_id = dpaa2_ethq->flow_id;
+ uint8_t flow_id;
struct dpni_queue *cfg;
uint8_t options, priority;
int ret;
+ flow_id = dpaa2_ethq->flow_id;
cfg = dpaa2_ethq->cfg;
if (queue_conf->ev.sched_type == RTE_SCHED_TYPE_PARALLEL)
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 31/45] net/dpaa2: rename Rx queue flags
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (29 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 30/45] drivers: inherit dpaa2 rxq config for event queue Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 32/45] drivers: rework dpaa2 Tx confirmation Prashant Gupta
` (13 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Rename the Rx-side device flags to a consistent DPAA2_RX_*_FLAG naming:
DPAAX_RX_ERROR_QUEUE_FLAG becomes DPAA2_RX_ERROR_QUEUE_FLAG and
DPAA2_DATA_STASHING_OFF becomes DPAA2_RX_DATA_STASHING_OFF_FLAG.
This is a mechanical rename with no functional change; it prepares the
flag namespace for the Tx confirmation rework that follows.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 10 +++++-----
drivers/net/dpaa2/dpaa2_ethdev.h | 4 ++--
drivers/net/dpaa2/dpaa2_rxtx.c | 4 ++--
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 57c6e8e90b..15a886b3e8 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -547,7 +547,7 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
goto fail;
}
- if (priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG) {
+ if (priv->flags & DPAA2_RX_ERROR_QUEUE_FLAG) {
priv->rx_err_vq = rte_zmalloc("dpni_rx_err",
sizeof(struct dpaa2_queue), 0);
if (!priv->rx_err_vq) {
@@ -1041,7 +1041,7 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
qopt |= DPNI_QUEUE_OPT_FLC;
cfg->flc.stash_control = true;
dpaa2_flc_stashing_clear_all(&cfg->flc.value);
- if (priv->flags & DPAA2_DATA_STASHING_OFF) {
+ if (priv->flags & DPAA2_RX_DATA_STASHING_OFF_FLAG) {
dpaa2_flc_stashing_set(DPAA2_FLC_DATA_STASHING, 0,
&cfg->flc.value);
dpaa2_q->data_stashing_off = 1;
@@ -1499,7 +1499,7 @@ dpaa2_dev_start(struct rte_eth_dev *dev)
dpaa2_q->fqid = qid.fqid;
}
- if (priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG) {
+ if (priv->flags & DPAA2_RX_ERROR_QUEUE_FLAG) {
ret = dpni_get_queue(dpni, CMD_PRI_LOW, priv->token,
DPNI_QUEUE_RX_ERR, 0, 0, &cfg, &qid);
if (ret) {
@@ -3462,7 +3462,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
if (dpaa2_get_devargs(dev->devargs, DRIVER_NO_DATA_STASHING) ||
getenv("DPAA2_DATA_STASHING_OFF")) {
- priv->flags |= DPAA2_DATA_STASHING_OFF;
+ priv->flags |= DPAA2_RX_DATA_STASHING_OFF_FLAG;
DPAA2_PMD_INFO("Data stashing disabled");
}
@@ -3592,7 +3592,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
}
if (dpaa2_get_devargs(dev->devargs, DRIVER_ERROR_QUEUE)) {
- priv->flags |= DPAAX_RX_ERROR_QUEUE_FLAG;
+ priv->flags |= DPAA2_RX_ERROR_QUEUE_FLAG;
DPAA2_PMD_INFO("Enable error queue");
}
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 3c611652b1..28623dc37d 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -100,9 +100,9 @@
#define DPAA2_TX_DYNAMIC_CONF_ENABLE RTE_BIT32(9)
/* Disable data stashing (prefetch of packet data into CPU cache) */
-#define DPAA2_DATA_STASHING_OFF RTE_BIT32(10)
+#define DPAA2_RX_DATA_STASHING_OFF_FLAG RTE_BIT32(10)
-#define DPAAX_RX_ERROR_QUEUE_FLAG RTE_BIT32(11)
+#define DPAA2_RX_ERROR_QUEUE_FLAG RTE_BIT32(11)
#define DPAA2_RX_PRINT_PSR_RESULT_FLAG RTE_BIT32(12)
/* DPDMUX index for DPMAC */
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index ccb4bbe7dc..a03e3fd8db 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -870,7 +870,7 @@ dpaa2_dev_prefetch_rx_common(void *queue, struct rte_mbuf **bufs,
q_storage = dpaa2_q->q_storage[rte_lcore_id()];
- if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
+ if (unlikely(priv->flags & DPAA2_RX_ERROR_QUEUE_FLAG))
dump_err_pkts(priv->rx_err_vq);
if (unlikely(!DPAA2_PER_LCORE_ETHRX_DPIO)) {
@@ -1142,7 +1142,7 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
struct dpaa2_dev_priv *priv = eth_data->dev_private;
- if (!by_channel && unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
+ if (!by_channel && unlikely(priv->flags & DPAA2_RX_ERROR_QUEUE_FLAG))
dump_err_pkts(priv->rx_err_vq);
if (by_channel) {
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 32/45] drivers: rework dpaa2 Tx confirmation
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (30 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 31/45] net/dpaa2: rename Rx queue flags Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 33/45] net/dpaa2: ptp enhancements Prashant Gupta
` (12 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Mbufs released by Tx confirmation may come from any mempool, so use the
RTE mbuf free API instead of returning the buffers directly to the bman
pool. The FD FLC field now carries the Tx mbuf pointer so that the
confirmation callback can release it to either the platform pool or an
external pool.
Absolute and dynamic Tx confirmation become independent settings, giving
three Tx modes:
1) No confirmation (default). Suited to traffic that is mostly sent from
the platform pool. For the few packets from other pools the driver
allocates a platform buffer, copies the data and frees the original.
2) Absolute confirmation. Suited to traffic mostly sent from non platform
pools. Every Tx packet raises a confirmation, configured at queue
setup, and the callback frees the packet.
3) Dynamic confirmation. Confirmation is raised only for packets from non
platform pools; the driver marks each packet accordingly at runtime.
This suits a roughly even mix of platform and non platform buffers.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 1 +
drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 2 +-
drivers/net/dpaa2/base/dpaa2_hw_dpni_annot.h | 6 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 63 +-
drivers/net/dpaa2/dpaa2_ethdev.h | 37 +-
drivers/net/dpaa2/dpaa2_ptp.c | 4 +-
drivers/net/dpaa2/dpaa2_rxtx.c | 1511 ++++++++----------
7 files changed, 751 insertions(+), 873 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index f87bf51d09..b3aad85d06 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -198,6 +198,7 @@ struct __rte_cache_aligned dpaa2_queue {
uint16_t tm_sw_td; /*!< TM software taildrop */
uint64_t offloads;
uint64_t lpbk_cntx;
+ int32_t ts_to_cnfd;
uint8_t data_stashing_off;
/* NAPI rx-interrupt: per-queue DPCON bound to this FQ at dev_start
* (DEST_DPCON, static); the polling worker points the channel's CDAN at
diff --git a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
index 26ad105c73..fee96b3529 100644
--- a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
+++ b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
@@ -484,7 +484,7 @@ dpaa2_attach_bp_list(struct dpaa2_dev_priv *priv,
*/
/* ... rx buffer layout ... */
- if (priv->flags & DPAA2_TX_DYNAMIC_CONF_ENABLE) {
+ if (priv->flags & DPAA2_TX_PREFETCH_DYNAMIC_CONF) {
int out_min_hdr_room, in_min_hdr_room;
/** Additional headroom layout for IPSec with TX configure
* dynamic enabled.
diff --git a/drivers/net/dpaa2/base/dpaa2_hw_dpni_annot.h b/drivers/net/dpaa2/base/dpaa2_hw_dpni_annot.h
index a670098958..b06fa3f644 100644
--- a/drivers/net/dpaa2/base/dpaa2_hw_dpni_annot.h
+++ b/drivers/net/dpaa2/base/dpaa2_hw_dpni_annot.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016,2019,2022,2024 NXP
+ * Copyright 2016,2019,2022,2024-2026 NXP
*
*/
@@ -281,6 +281,10 @@ struct dpaa2_faead {
/*UPD (Update prepended data)*/
#define DPAA2_ANNOT_FAEAD_UPD 0x00000010
+#define DPAA2_TX_CONFIRM_ENABLE \
+ (DPAA2_ANNOT_FAEAD_A4V | DPAA2_ANNOT_FAEAD_EBDD | \
+ DPAA2_ANNOT_FAEAD_EBDDV)
+
/* Debug frame, otherwise supposed to be discarded */
#define DPAA2_ETH_FAS_DISC 0x80000000
/* MACSEC frame */
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 15a886b3e8..61b7ae138c 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -34,6 +34,8 @@
#define DRIVER_LOOPBACK_MODE "drv_loopback"
#define DRIVER_NO_PREFETCH_MODE "drv_no_prefetch"
#define DRIVER_TX_CONF "drv_tx_conf"
+#define DRIVER_TX_DYNAMIC_CONF "drv_tx_dyn_conf"
+#define DRIVER_TX_DYNAMIC_CONF_PREFETCH "drv_tx_dyn_conf_pre"
#define DRIVER_RX_PARSE_ERR_DROP "drv_rx_parse_drop"
#define DRIVER_ERROR_QUEUE "drv_err_queue"
#define DRIVER_NO_TAILDROP "drv_no_taildrop"
@@ -522,7 +524,7 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
PMD_INIT_FUNC_TRACE();
- if (priv->flags & DPAA2_TX_CONF_ENABLE)
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF)
tot_queues = priv->nb_rx_queues + 2 * priv->nb_tx_queues;
else
tot_queues = priv->nb_rx_queues + priv->nb_tx_queues;
@@ -578,7 +580,7 @@ dpaa2_alloc_rx_tx_queues(struct rte_eth_dev *dev)
}
}
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
/*Setup tx confirmation queues*/
for (i = 0; i < priv->nb_tx_queues; i++) {
mc_q->eth_data = dev->data;
@@ -693,7 +695,7 @@ dpaa2_free_rx_tx_queues(struct rte_eth_dev *dev)
rte_free(dpaa2_q->cscn);
priv->tx_vq[i] = NULL;
}
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
/* cleanup tx conf queue storage */
for (i = 0; i < priv->nb_tx_queues; i++) {
dpaa2_q = priv->tx_conf_vq[i];
@@ -1143,6 +1145,13 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
PMD_INIT_FUNC_TRACE();
+ /* Tx deferred start is not supported */
+ if (tx_conf->tx_deferred_start) {
+ DPAA2_PMD_ERR("%s:Tx deferred start not supported",
+ dev->data->name);
+ return -EINVAL;
+ }
+
dpaa2_q->nb_desc = UINT16_MAX;
dpaa2_q->offloads = tx_conf->offloads;
@@ -1222,7 +1231,7 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
dpaa2_q->cb_eqresp_free = dpaa2_dev_free_eqresp_buf;
dev->data->tx_queues[tx_queue_id] = dpaa2_q;
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
tc_id = dpaa2_tx_conf_q->tc_index;
flow_id = dpaa2_tx_conf_q->flow_id;
dpaa2_q->tx_conf_queue = dpaa2_tx_conf_q;
@@ -3585,10 +3594,22 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
DPAA2_PMD_INFO("DPDK IEEE1588 is enabled");
priv->flags |= DPAA2_TX_CONF_ENABLE;
#endif
+ priv->tx_conf_type = DPAA2_TX_NO_CONF;
+
/* Used with ``fslmc:dpni.1,drv_tx_conf=1`` */
if (dpaa2_get_devargs(dev->devargs, DRIVER_TX_CONF)) {
- priv->flags |= DPAA2_TX_CONF_ENABLE;
- DPAA2_PMD_INFO("TX_CONF Enabled");
+ priv->tx_conf_type = DPAA2_TX_ABSOLUTE_CONF;
+ DPAA2_PMD_INFO("TX_ABSOLUTE_CONF Enabled");
+ } else if (dpaa2_get_devargs(dev->devargs, DRIVER_TX_DYNAMIC_CONF)) {
+ priv->tx_conf_type = DPAA2_TX_DYNAMIC_CONF;
+ DPAA2_PMD_INFO("TX_DYNAMIC_CONF Enabled");
+ priv->flags |= DPAA2_TX_PREFETCH_DYNAMIC_CONF;
+ if (dpaa2_get_devargs(dev->devargs, DRIVER_TX_DYNAMIC_CONF_PREFETCH))
+ priv->flags &= ~DPAA2_TX_PREFETCH_DYNAMIC_CONF;
+
+ DPAA2_PMD_INFO("Tx dynamic prefetch confirm %s",
+ (priv->flags & DPAA2_TX_PREFETCH_DYNAMIC_CONF) ?
+ "enabled" : "disabled");
}
if (dpaa2_get_devargs(dev->devargs, DRIVER_ERROR_QUEUE)) {
@@ -3603,7 +3624,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
}
if (getenv("DPAA2_PRINT_RX_PARSER_RESULT"))
- dpaa2_print_parser_result = 1;
+ priv->flags |= DPAA2_RX_PRINT_PSR_RESULT_FLAG;
/* Allocate memory for hardware structure for queues */
ret = dpaa2_alloc_rx_tx_queues(eth_dev);
@@ -3614,7 +3635,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
for (i = 0; i < priv->num_channels; i++) {
/*Set tx-conf and error configuration*/
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
+ if (priv->tx_conf_type == DPAA2_TX_ABSOLUTE_CONF) {
ret = dpni_set_tx_confirmation_mode(dpni_dev,
CMD_PRI_LOW, priv->token, i, DPNI_CONF_AFFINE);
} else {
@@ -3651,34 +3672,22 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
/* ... tx buffer layout ... */
memset(&layout, 0, sizeof(struct dpni_buffer_layout));
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
- layout.options = DPNI_BUF_LAYOUT_OPT_FRAME_STATUS |
- DPNI_BUF_LAYOUT_OPT_TIMESTAMP;
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
+ layout.options = DPNI_BUF_LAYOUT_OPT_TIMESTAMP;
layout.pass_timestamp = true;
- } else {
- layout.options = DPNI_BUF_LAYOUT_OPT_FRAME_STATUS;
}
+ layout.options |= DPNI_BUF_LAYOUT_OPT_FRAME_STATUS;
layout.pass_frame_status = 1;
ret = dpni_set_buffer_layout(dpni_dev, CMD_PRI_LOW, priv->token,
- DPNI_QUEUE_TX, &layout);
+ DPNI_QUEUE_TX, &layout);
if (ret) {
DPAA2_PMD_ERR("Error (%d) in setting tx buffer layout", ret);
goto init_err;
}
-
- /* ... tx-conf and error buffer layout ... */
- memset(&layout, 0, sizeof(struct dpni_buffer_layout));
- if (priv->flags & DPAA2_TX_CONF_ENABLE) {
- layout.options = DPNI_BUF_LAYOUT_OPT_TIMESTAMP;
- layout.pass_timestamp = true;
- }
- layout.options |= DPNI_BUF_LAYOUT_OPT_FRAME_STATUS;
- layout.pass_frame_status = 1;
ret = dpni_set_buffer_layout(dpni_dev, CMD_PRI_LOW, priv->token,
- DPNI_QUEUE_TX_CONFIRM, &layout);
+ DPNI_QUEUE_TX_CONFIRM, &layout);
if (ret) {
- DPAA2_PMD_ERR("Error (%d) in setting tx-conf buffer layout",
- ret);
+ DPAA2_PMD_ERR("Error (%d) in setting tx conf buffer layout", ret);
goto init_err;
}
@@ -3979,6 +3988,8 @@ RTE_PMD_REGISTER_PARAM_STRING(NET_DPAA2_PMD_DRIVER_NAME,
DRIVER_LOOPBACK_MODE "=<int> "
DRIVER_NO_PREFETCH_MODE "=<int>"
DRIVER_TX_CONF "=<int>"
+ DRIVER_TX_DYNAMIC_CONF "=<int>"
+ DRIVER_TX_DYNAMIC_CONF_PREFETCH "=<int>"
DRIVER_RX_PARSE_ERR_DROP "=<int>"
DRIVER_ERROR_QUEUE "=<int>"
DRIVER_NO_TAILDROP "=<int>"
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 28623dc37d..c6a8b4f595 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -91,18 +91,14 @@
#define DPAA2_TX_DPNI_LOOPBACK_MODE RTE_BIT32(7)
-/* Tx confirmation enabled */
-#define DPAA2_TX_CONF_ENABLE RTE_BIT32(8)
+#define DPAA2_TX_PREFETCH_DYNAMIC_CONF RTE_BIT32(8)
-/* Tx dynamic confirmation enabled,
- * only valid with Tx confirmation enabled.
- */
-#define DPAA2_TX_DYNAMIC_CONF_ENABLE RTE_BIT32(9)
+#define DPAA2_RX_ERROR_QUEUE_FLAG RTE_BIT32(9)
+
+#define DPAA2_RX_DATA_STASHING_OFF_FLAG RTE_BIT32(10)
-/* Disable data stashing (prefetch of packet data into CPU cache) */
-#define DPAA2_RX_DATA_STASHING_OFF_FLAG RTE_BIT32(10)
+#define DPAA2_RX_SCHED_STRICT_ORDER_FLAG RTE_BIT32(11)
-#define DPAA2_RX_ERROR_QUEUE_FLAG RTE_BIT32(11)
#define DPAA2_RX_PRINT_PSR_RESULT_FLAG RTE_BIT32(12)
/* DPDMUX index for DPMAC */
@@ -451,6 +447,12 @@ struct dpaa2_dev_meter {
uint32_t policy_id;
};
+enum dpaa2_tx_conf_type {
+ DPAA2_TX_NO_CONF,
+ DPAA2_TX_ABSOLUTE_CONF,
+ DPAA2_TX_DYNAMIC_CONF
+};
+
struct dpaa2_dev_priv {
void *hw;
int32_t hw_id;
@@ -466,6 +468,7 @@ struct dpaa2_dev_priv {
void *tx_conf_vq[MAX_TX_QUEUES];
void *rx_err_vq;
uint32_t flags; /*dpaa2 config flags */
+ enum dpaa2_tx_conf_type tx_conf_type;
int psr_dynfield_offset;
uint8_t max_mac_filters;
uint8_t max_vlan_filters;
@@ -510,10 +513,10 @@ struct dpaa2_dev_priv {
int rx_ts_offset;
uint64_t rx_ts_flag;
- /* stores pointer to next tx_conf queue that should be processed,
+ /* stores next tx queue to be confirmed that should be processed,
* it corresponds to last packet transmitted
*/
- struct dpaa2_queue *next_tx_conf_queue;
+ struct dpaa2_queue *next_txq_to_cnf;
struct rte_eth_dev *eth_dev; /**< Pointer back to holding ethdev */
rte_spinlock_t lpbk_qp_lock;
@@ -531,6 +534,8 @@ struct dpaa2_dev_priv {
uint64_t cnt_idx_iova, cnt_values_iova;
uint64_t mc_rev;
+ struct rte_mempool *tx_sg_pool;
+
struct dpaa2_dev_flow *curr;
LIST_HEAD(, dpaa2_dev_flow) flows;
LIST_HEAD(, dpaa2_dev_meter_profile) profiles;
@@ -646,7 +651,7 @@ uint16_t dpaa2_dev_tx_multi_txq_ordered(void **queue,
void dpaa2_dev_free_eqresp_buf(uint16_t eqresp_ci, struct dpaa2_queue *dpaa2_q);
void dpaa2_flow_clean(struct rte_eth_dev *dev);
-uint16_t dpaa2_dev_tx_conf(void *queue) __rte_unused;
+uint16_t dpaa2_dev_tx_conf(void *txq, int drain);
int dpaa2_timesync_enable(struct rte_eth_dev *dev);
int dpaa2_timesync_disable(struct rte_eth_dev *dev);
@@ -663,6 +668,14 @@ int dpaa2_timesync_read_tx_timestamp(struct rte_eth_dev *dev,
int dpaa2_dev_recycle_config(struct rte_eth_dev *eth_dev);
int dpaa2_dev_recycle_deconfig(struct rte_eth_dev *eth_dev);
+
+__rte_internal
+struct rte_mbuf *__rte_hot
+dpaa2_eth_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd);
+__rte_internal
+struct rte_mbuf *__rte_hot
+dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd);
+
int dpaa2_mtr_ops_get(struct rte_eth_dev *dev, void *ops);
int dpaa2_soft_parser_loaded(void);
diff --git a/drivers/net/dpaa2/dpaa2_ptp.c b/drivers/net/dpaa2/dpaa2_ptp.c
index 751e558c73..c845291c68 100644
--- a/drivers/net/dpaa2/dpaa2_ptp.c
+++ b/drivers/net/dpaa2/dpaa2_ptp.c
@@ -111,9 +111,9 @@ int dpaa2_timesync_read_tx_timestamp(struct rte_eth_dev *dev,
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- if (priv->next_tx_conf_queue) {
+ if (priv->next_txq_to_cnf) {
while (!priv->tx_timestamp)
- dpaa2_dev_tx_conf(priv->next_tx_conf_queue);
+ dpaa2_dev_tx_conf(priv->next_txq_to_cnf, false);
} else {
return -1;
}
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index a03e3fd8db..4512c8cf00 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2024 NXP
+ * Copyright 2016-2026 NXP
*
*/
@@ -75,6 +75,23 @@ dpaa2_dev_rx_parse_offset(struct dpaa2_dev_priv *priv,
DPAA2_RESET_FD_FLC(_fd); \
} while (0)
+/* Build a contiguous FD which is returned on the Tx confirmation queue.
+ * The buffer is not released by the hardware, so an invalid buffer pool
+ * ID is programmed and the mbuf is carried in the FLC to be freed once
+ * the confirmation is dequeued.
+ */
+static inline void
+dpaa2_mbuf_to_conf_contig_fd(struct rte_mbuf *mbuf, struct qbman_fd *fd)
+{
+ DPAA2_SET_FD_ADDR(fd, DPAA2_MBUF_VADDR_TO_IOVA(mbuf));
+ DPAA2_SET_FD_LEN(fd, mbuf->data_len);
+ DPAA2_SET_ONLY_FD_BPID(fd, MAX_BPID);
+ DPAA2_SET_FD_IVP(fd);
+ DPAA2_SET_FD_OFFSET(fd, mbuf->data_off);
+ DPAA2_SET_FD_FRC(fd, 0);
+ DPAA2_RESET_FD_CTRL(fd);
+ DPAA2_SET_FD_FLC(fd, mbuf);
+}
static inline void
dpaa2_dev_rx_mbuf_sched_set(struct rte_mbuf *m,
@@ -98,7 +115,6 @@ dpaa2_dev_rx_mbuf_sched_set(struct rte_mbuf *m,
m->hash.rss);
}
}
-
static void __rte_hot
dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
struct rte_mbuf *m, const struct qbman_fd *fd,
@@ -237,13 +253,6 @@ dpaa2_dev_rx_parse_slow(struct rte_mbuf *mbuf,
"(4)=0x%" PRIx64 "\t",
annotation->word3, annotation->word4);
-#if defined(RTE_LIBRTE_IEEE1588)
- if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_PTP)) {
- mbuf->ol_flags |= RTE_MBUF_F_RX_IEEE1588_PTP;
- mbuf->ol_flags |= RTE_MBUF_F_RX_IEEE1588_TMST;
- }
-#endif
-
if (BIT_ISSET_AT_POS(annotation->word3, L2_VLAN_1_PRESENT)) {
vlan_tci = rte_pktmbuf_mtod_offset(mbuf, uint16_t *,
(VLAN_TCI_OFFSET_1(annotation->word5) >> 16));
@@ -367,13 +376,14 @@ dpaa2_dev_rx_parse(struct dpaa2_dev_priv *priv,
return dpaa2_dev_rx_parse_slow(mbuf, annotation);
}
-static inline struct rte_mbuf *__rte_hot
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_eth_sg_fd_to_mbuf)
+struct rte_mbuf *__rte_hot
dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
{
struct qbman_sge *sgt, *sge;
- size_t sg_addr, fd_addr;
+ size_t fd_addr;
int i = 0;
- void *hw_annot_addr;
+ void *hw_annot_addr, *sg_addr;
struct rte_mbuf *first_seg, *next_seg, *cur_seg, *temp;
fd_addr = (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
@@ -383,7 +393,7 @@ dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
sgt = (struct qbman_sge *)(fd_addr + DPAA2_GET_FD_OFFSET(fd));
sge = &sgt[i++];
- sg_addr = (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FLE_ADDR(sge));
+ sg_addr = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FLE_ADDR(sge));
/* First Scatter gather entry */
first_seg = DPAA2_INLINE_MBUF_FROM_BUF(sg_addr,
@@ -413,8 +423,7 @@ dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
cur_seg = first_seg;
while (!DPAA2_SG_IS_FINAL(sge) && i < DPAA2_MAX_SGS) {
sge = &sgt[i++];
- sg_addr = (size_t)DPAA2_IOVA_TO_VADDR(
- DPAA2_GET_FLE_ADDR(sge));
+ sg_addr = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FLE_ADDR(sge));
next_seg = DPAA2_INLINE_MBUF_FROM_BUF(sg_addr,
rte_dpaa2_bpid_info[DPAA2_GET_FLE_BPID(sge)].meta_data_size);
next_seg->buf_addr = (uint8_t *)sg_addr;
@@ -442,7 +451,8 @@ dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
return (void *)first_seg;
}
-static inline struct rte_mbuf *__rte_hot
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_eth_fd_to_mbuf)
+struct rte_mbuf *__rte_hot
dpaa2_eth_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
{
void *v_addr = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
@@ -499,119 +509,296 @@ dpaa2_eth_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
return mbuf;
}
-static int __rte_noinline __rte_hot
-eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
- struct qbman_fd *fd,
- struct sw_buf_free *free_buf,
- uint32_t *free_count,
- uint32_t pkt_id,
- uint16_t bpid)
+static void dpaa2_dev_tx_enable_tstamp(struct qbman_fd *fd)
{
- struct rte_mbuf *cur_seg = mbuf, *mi, *temp;
- struct qbman_sge *sgt, *sge = NULL;
- int i, offset = 0;
+ struct dpaa2_faead *fd_faead;
+ void *fd_va = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
-#ifdef RTE_LIBRTE_IEEE1588
- /* annotation area for timestamp in first buffer */
- offset = 0x64;
-#endif
- if (RTE_MBUF_DIRECT(mbuf) &&
- (mbuf->data_off > (mbuf->nb_segs * sizeof(struct qbman_sge)
- + offset))) {
- temp = mbuf;
- if (rte_mbuf_refcnt_read(temp) > 1) {
- /* If refcnt > 1, invalid bpid is set to ensure
- * buffer is not freed by HW
- */
- fd->simple.bpid_offset = 0;
+ /* Set frame annotation status field as valid */
+ (fd)->simple.frc |= DPAA2_FD_FRC_FASV;
+
+ /* Set frame annotation egress action descriptor as valid */
+ (fd)->simple.frc |= DPAA2_FD_FRC_FAEADV;
+
+ /* Set Annotation Length as 128B */
+ (fd)->simple.ctrl |= DPAA2_FD_CTRL_ASAL;
+
+ /* enable update of confirmation frame annotation */
+ fd_faead = (void *)((size_t)fd_va +
+ DPAA2_FD_PTA_SIZE + DPAA2_FD_HW_ANNOT_FAEAD_OFFSET);
+ fd_faead->ctrl |= (DPAA2_ANNOT_FAEAD_A2V |
+ DPAA2_ANNOT_FAEAD_UPDV | DPAA2_ANNOT_FAEAD_UPD);
+}
+
+static inline struct rte_mbuf *
+dpaa2_dev_mbuf_copy_one_seg(const struct rte_mbuf *sgm, struct rte_mempool *mp)
+{
+ struct rte_mbuf *mc = rte_pktmbuf_alloc(mp);
+
+ if (!mc)
+ return NULL;
+ if ((mc->data_off + sgm->data_len) > mc->buf_len) {
+ rte_pktmbuf_free(mc);
+ return NULL;
+ }
+ rte_memcpy(rte_pktmbuf_mtod(mc, void *),
+ rte_pktmbuf_mtod(sgm, void *), sgm->data_len);
+ mc->pkt_len = sgm->data_len;
+ mc->data_len = sgm->data_len;
+ mc->next = NULL;
+
+ return mc;
+}
+
+static inline void
+dpaa2_dev_tx_config_dynamic_confirm(struct qbman_fd *fd,
+ struct dpaa2_queue *txq, int enable, int tstamp)
+{
+ struct dpaa2_faead *fd_faead;
+ void *fd_va = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
+
+ if (tstamp) {
+ DPAA2_SET_FD_FRC(fd,
+ DPAA2_GET_FD_FRC(fd) |
+ DPAA2_FD_FRC_FAEADV | DPAA2_FD_FRC_FASV);
+ fd->simple.ctrl |= DPAA2_FD_CTRL_ASAL;
+ } else {
+ DPAA2_SET_FD_FRC(fd, DPAA2_GET_FD_FRC(fd) |
+ DPAA2_FD_FRC_FAEADV);
+ }
+ /* enable update of confirmation frame annotation */
+ fd_faead = (void *)((size_t)fd_va +
+ DPAA2_FD_PTA_SIZE + DPAA2_FD_HW_ANNOT_FAEAD_OFFSET);
+ if (enable) {
+ fd_faead->fqid = txq->tx_conf_queue->fqid;
+ fd_faead->ctrl = DPAA2_TX_CONFIRM_ENABLE;
+ if (tstamp) {
+ fd_faead->ctrl |= (DPAA2_ANNOT_FAEAD_A2V |
+ DPAA2_ANNOT_FAEAD_UPDV |
+ DPAA2_ANNOT_FAEAD_UPD);
+ }
+ } else {
+ fd_faead->fqid = 0;
+ fd_faead->ctrl &= (~DPAA2_TX_CONFIRM_ENABLE);
+ }
+}
+
+static int
+dpaa2_dev_tx_no_conf_mbuf_to_sge(struct qbman_sge *sgt,
+ struct rte_mbuf *mbuf, uint16_t nb_segs, struct rte_mbuf *sg_mbuf,
+ struct rte_mempool *hw_mp)
+{
+ int i;
+ struct qbman_sge *sge = NULL;
+ struct rte_mbuf *cur_seg, *mi, *prev, *copy_mbuf;
+
+ cur_seg = mbuf;
+ prev = NULL;
+ for (i = 0; i < nb_segs; i++) {
+ sge = &sgt[i];
+ /*Resetting the buffer pool id and offset field*/
+ sge->fin_bpid_offset = 0;
+ if (unlikely(RTE_MBUF_CLONED(cur_seg))) {
+ copy_mbuf = dpaa2_dev_mbuf_copy_one_seg(cur_seg, hw_mp);
+ if (!copy_mbuf)
+ return -ENOMEM;
+ mi = rte_mbuf_from_indirect(cur_seg);
+ if (rte_mbuf_refcnt_read(mi) > 1)
+ rte_mbuf_refcnt_update(mi, -1);
+ else
+ rte_pktmbuf_free_seg(mi);
+ copy_mbuf->next = cur_seg->next;
+ if (prev)
+ prev->next = copy_mbuf;
+ rte_pktmbuf_init(cur_seg->pool, NULL, cur_seg, 0);
+ rte_pktmbuf_free_seg(cur_seg);
+ cur_seg = copy_mbuf;
+ }
+ if (unlikely(RTE_MBUF_HAS_EXTBUF(cur_seg) ||
+ cur_seg->pool->ops_index != hw_mp->ops_index)) {
+ copy_mbuf = dpaa2_dev_mbuf_copy_one_seg(cur_seg, hw_mp);
+ if (!copy_mbuf)
+ return -ENOMEM;
+
+ copy_mbuf->next = cur_seg->next;
+
+ if (prev)
+ prev->next = copy_mbuf;
+ if (rte_mbuf_refcnt_read(cur_seg) > 1)
+ rte_mbuf_refcnt_update(cur_seg, -1);
+ else
+ rte_pktmbuf_free_seg(cur_seg);
+ cur_seg = copy_mbuf;
+ }
+ DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(cur_seg));
+ DPAA2_SET_FLE_OFFSET(sge, cur_seg->data_off);
+ DPAA2_SET_FLE_LEN(sge, cur_seg->data_len);
+ if (unlikely(rte_mbuf_refcnt_read(cur_seg) > 1)) {
+ DPAA2_SET_FLE_BPID(sge, MAX_BPID);
+ DPAA2_SET_FLE_IVP(sge);
+ rte_mbuf_refcnt_update(cur_seg, -1);
+ } else {
+ DPAA2_SET_FLE_BPID(sge, mempool_to_bpid(cur_seg->pool));
+ }
+ if (sg_mbuf == cur_seg)
+ DPAA2_SG_SET_FORMAT(sge, qbman_fd_list);
+ prev = cur_seg;
+ cur_seg = cur_seg->next;
+ }
+
+ if (likely(sge))
+ DPAA2_SG_SET_FINAL(sge, true);
+
+ return 0;
+}
+
+static void
+dpaa2_dev_tx_conf_mbuf_to_sge(struct qbman_sge *sgt,
+ struct rte_mbuf *mbuf, uint16_t nb_segs, struct rte_mbuf *sg_mbuf)
+{
+ int i;
+ struct qbman_sge *sge = NULL;
+ struct rte_mbuf *cur_seg = mbuf, *mi, *prev;
+
+ prev = NULL;
+ for (i = 0; i < nb_segs; i++) {
+ sge = &sgt[i];
+ /*Resetting the buffer pool id and offset field*/
+ sge->fin_bpid_offset = 0;
+ if (unlikely(RTE_MBUF_CLONED(cur_seg))) {
+ mi = rte_mbuf_from_indirect(cur_seg);
+ mi->next = cur_seg->next;
+ if (prev)
+ prev->next = mi;
+ rte_pktmbuf_init(cur_seg->pool, NULL, cur_seg, 0);
+ rte_pktmbuf_free_seg(cur_seg);
+ cur_seg = mi;
+ }
+ DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(cur_seg));
+ DPAA2_SET_FLE_OFFSET(sge, cur_seg->data_off);
+ DPAA2_SET_FLE_LEN(sge, cur_seg->data_len);
+ DPAA2_SET_FLE_BPID(sge, MAX_BPID);
+ DPAA2_SET_FLE_IVP(sge);
+ if (sg_mbuf == cur_seg)
+ DPAA2_SG_SET_FORMAT(sge, qbman_fd_list);
+ prev = cur_seg;
+ cur_seg = cur_seg->next;
+ }
+
+ if (likely(sge))
+ DPAA2_SG_SET_FINAL(sge, true);
+}
+
+static int __rte_noinline __rte_hot
+dpaa2_dev_tx_mbuf_to_sg_fd(struct rte_mempool *hw_mp,
+ struct rte_mbuf *mbuf, struct qbman_fd *fd,
+ struct dpaa2_queue *txq, enum dpaa2_tx_conf_type conf,
+ uint8_t *dy_conf, int tstamp)
+{
+ struct rte_mbuf *cur_seg = mbuf, *sg_mbuf;
+ struct qbman_sge *sgt;
+ int offset = 0, ret, need_dyconf = false, need_conf = false;
+ uint16_t nb_segs = mbuf->nb_segs;
+ uint32_t sg_size = nb_segs * sizeof(struct qbman_sge);
+ struct rte_mempool *mp;
+ struct dpaa2_dev_priv *priv = txq->eth_data->dev_private;
+
+ if (conf == DPAA2_TX_DYNAMIC_CONF) {
+ while (cur_seg) {
+ if (!RTE_MBUF_DIRECT(cur_seg) ||
+ cur_seg->pool->ops_index != hw_mp->ops_index ||
+ rte_mbuf_refcnt_read(cur_seg) > 1 || tstamp) {
+ need_dyconf = true;
+ break;
+ }
+ cur_seg = cur_seg->next;
+ }
+ cur_seg = mbuf;
+ }
+ if (need_dyconf || conf == DPAA2_TX_ABSOLUTE_CONF)
+ need_conf = true;
+
+ if (need_dyconf || tstamp)
+ offset = DPAA2_FD_PTA_SIZE + DPAA2_DYN_TX_MIN_FD_OFFSET;
+
+ if (mbuf->pool->ops_index == hw_mp->ops_index &&
+ RTE_MBUF_DIRECT(mbuf) &&
+ mbuf->data_off > RTE_ALIGN(sg_size + offset, 8)) {
+ sg_mbuf = mbuf;
+ mp = mbuf->pool;
+ if (need_conf) {
+ DPAA2_SET_ONLY_FD_BPID(fd, MAX_BPID);
DPAA2_SET_FD_IVP(fd);
- rte_mbuf_refcnt_update(temp, -1);
} else {
- DPAA2_SET_ONLY_FD_BPID(fd, bpid);
+ if (unlikely(rte_mbuf_refcnt_read(sg_mbuf) > 1)) {
+ DPAA2_SET_ONLY_FD_BPID(fd, MAX_BPID);
+ DPAA2_SET_FD_IVP(fd);
+ rte_mbuf_refcnt_update(sg_mbuf, -1);
+ } else {
+ DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(mp));
+ }
+ }
#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- rte_mempool_check_cookies(rte_mempool_from_obj((void *)temp),
- (void **)&temp, 1, 0);
+ rte_mempool_check_cookies(rte_mempool_from_obj(sg_mbuf),
+ (void **)&sg_mbuf, 1, 0);
#endif
- }
DPAA2_SET_FD_OFFSET(fd, offset);
} else {
- temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
- if (temp == NULL) {
+ sg_mbuf = NULL;
+ if (priv->tx_sg_pool)
+ sg_mbuf = rte_pktmbuf_alloc(priv->tx_sg_pool);
+ if (!sg_mbuf) {
DPAA2_PMD_DP_DEBUG("No memory to allocate S/G table");
return -ENOMEM;
}
- DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(dpaa2_tx_sg_pool));
- DPAA2_SET_FD_OFFSET(fd, temp->data_off);
+ if (need_conf) {
+ if (need_dyconf)
+ sg_mbuf->data_off = RTE_ALIGN(offset, 8);
+ else
+ sg_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ DPAA2_SET_ONLY_FD_BPID(fd, MAX_BPID);
+ DPAA2_SET_FD_IVP(fd);
+ } else {
+ DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(priv->tx_sg_pool));
+ }
+ DPAA2_SET_FD_OFFSET(fd, sg_mbuf->data_off);
+ offset = sg_mbuf->data_off;
#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- rte_mempool_check_cookies(rte_mempool_from_obj((void *)temp),
- (void **)&temp, 1, 0);
+ rte_mempool_check_cookies(rte_mempool_from_obj(sg_mbuf),
+ (void **)&sg_mbuf, 1, 0);
#endif
}
- DPAA2_SET_FD_ADDR(fd, DPAA2_MBUF_VADDR_TO_IOVA(temp));
+
+ /*Set Scatter gather table and Scatter gather entries*/
+ sgt = (void *)((size_t)sg_mbuf->buf_addr + offset);
+ DPAA2_SET_FD_ADDR(fd, DPAA2_MBUF_VADDR_TO_IOVA(sg_mbuf));
DPAA2_SET_FD_LEN(fd, mbuf->pkt_len);
DPAA2_FD_SET_FORMAT(fd, qbman_fd_sg);
DPAA2_RESET_FD_FRC(fd);
DPAA2_RESET_FD_CTRL(fd);
- DPAA2_RESET_FD_FLC(fd);
- /*Set Scatter gather table and Scatter gather entries*/
- sgt = (struct qbman_sge *)(
- (size_t)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd))
- + DPAA2_GET_FD_OFFSET(fd));
+ if (need_conf)
+ DPAA2_SET_FD_FLC(fd, sg_mbuf);
+ else
+ DPAA2_RESET_FD_FLC(fd);
+ if (need_dyconf)
+ dpaa2_dev_tx_config_dynamic_confirm(fd, txq, true, tstamp);
+ else if (tstamp)
+ dpaa2_dev_tx_enable_tstamp(fd);
- for (i = 0; i < mbuf->nb_segs; i++) {
- sge = &sgt[i];
- /*Resetting the buffer pool id and offset field*/
- sge->fin_bpid_offset = 0;
- DPAA2_SET_FLE_ADDR(sge, rte_pktmbuf_iova(cur_seg));
- sge->length = cur_seg->data_len;
- if (RTE_MBUF_DIRECT(cur_seg)) {
- /* if we are using inline SGT in same buffers
- * set the FLE FMT as Frame Data Section
- */
- if (temp == cur_seg) {
- DPAA2_SG_SET_FORMAT(sge, qbman_fd_list);
- DPAA2_SET_FLE_IVP(sge);
- } else {
- if (rte_mbuf_refcnt_read(cur_seg) > 1) {
- /* If refcnt > 1, invalid bpid is set to ensure
- * buffer is not freed by HW
- */
- DPAA2_SET_FLE_IVP(sge);
- rte_mbuf_refcnt_update(cur_seg, -1);
- } else {
- DPAA2_SET_FLE_BPID(sge,
- mempool_to_bpid(cur_seg->pool));
-#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- rte_mempool_check_cookies(rte_mempool_from_obj((void *)cur_seg),
- (void **)&cur_seg, 1, 0);
-#endif
- }
- }
- } else if (RTE_MBUF_HAS_EXTBUF(cur_seg)) {
- free_buf[*free_count].seg = cur_seg;
- free_buf[*free_count].pkt_id = pkt_id;
- ++*free_count;
- DPAA2_SET_FLE_IVP(sge);
- } else {
- /* Get owner MBUF from indirect buffer */
- mi = rte_mbuf_from_indirect(cur_seg);
- if (rte_mbuf_refcnt_read(mi) > 1) {
- /* If refcnt > 1, invalid bpid is set to ensure
- * owner buffer is not freed by HW
- */
- DPAA2_SET_FLE_IVP(sge);
- } else {
- DPAA2_SET_FLE_BPID(sge,
- mempool_to_bpid(mi->pool));
- rte_mbuf_refcnt_update(mi, 1);
- }
- free_buf[*free_count].seg = cur_seg;
- free_buf[*free_count].pkt_id = pkt_id;
- ++*free_count;
- }
- cur_seg = cur_seg->next;
+ if (cur_seg != sg_mbuf && need_conf)
+ sg_mbuf->next = cur_seg;
+
+ if (need_conf) {
+ dpaa2_dev_tx_conf_mbuf_to_sge(sgt, cur_seg, nb_segs, sg_mbuf);
+ } else {
+ ret = dpaa2_dev_tx_no_conf_mbuf_to_sge(sgt, cur_seg, nb_segs,
+ sg_mbuf, hw_mp);
+ if (ret)
+ return ret;
}
- DPAA2_SG_SET_FINAL(sge, true);
+ if (dy_conf && need_dyconf)
+ *dy_conf = true;
+
return 0;
}
@@ -632,101 +819,59 @@ dpaa2_dev_prefetch_next_psr(const struct qbman_result *dq)
rte_prefetch0(&annotation->word3);
}
-static void
-eth_mbuf_to_fd(struct rte_mbuf *mbuf,
- struct qbman_fd *fd,
- struct sw_buf_free *buf_to_free,
- uint32_t *free_count,
- uint32_t pkt_id,
- uint16_t bpid) __rte_unused;
-
-static void __rte_noinline __rte_hot
-eth_mbuf_to_fd(struct rte_mbuf *mbuf,
- struct qbman_fd *fd,
- struct sw_buf_free *buf_to_free,
- uint32_t *free_count,
- uint32_t pkt_id,
- uint16_t bpid)
+static int __rte_noinline __rte_hot
+dpaa2_dev_tx_mbuf_to_simple_fd(struct rte_mempool *hw_mp,
+ struct rte_mbuf *mbuf, struct qbman_fd *fd,
+ struct dpaa2_queue *txq, enum dpaa2_tx_conf_type conf,
+ uint8_t *dy_conf, int tstamp)
{
- DPAA2_MBUF_TO_CONTIG_FD(mbuf, fd, bpid);
+ int need_dyconf = false, need_conf = false;
+ struct rte_mbuf *mi;
+ struct rte_mbuf *copy_mbuf;
+ int ret = 0;
- DPAA2_PMD_DP_DEBUG("mbuf =%p, mbuf->buf_addr =%p, off = %d,"
- "fd_off=%d fd =%" PRIx64 ", meta = %d bpid =%d, len=%d",
- mbuf, mbuf->buf_addr, mbuf->data_off,
- DPAA2_GET_FD_OFFSET(fd), DPAA2_GET_FD_ADDR(fd),
- rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size,
- DPAA2_GET_FD_BPID(fd), DPAA2_GET_FD_LEN(fd));
- if (RTE_MBUF_DIRECT(mbuf)) {
+ if (unlikely(RTE_MBUF_CLONED(mbuf))) {
+ mi = rte_mbuf_from_indirect(mbuf);
if (rte_mbuf_refcnt_read(mbuf) > 1) {
- DPAA2_SET_FD_IVP(fd);
rte_mbuf_refcnt_update(mbuf, -1);
+ } else {
+ rte_pktmbuf_init(mbuf->pool, NULL, mbuf, 0);
+ rte_pktmbuf_free(mbuf);
}
-#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- else
- rte_mempool_check_cookies(rte_mempool_from_obj((void *)mbuf),
- (void **)&mbuf, 1, 0);
-#endif
- } else if (RTE_MBUF_HAS_EXTBUF(mbuf)) {
- buf_to_free[*free_count].seg = mbuf;
- buf_to_free[*free_count].pkt_id = pkt_id;
- ++*free_count;
- DPAA2_SET_FD_IVP(fd);
+ mbuf = mi;
+ }
+ if (conf == DPAA2_TX_DYNAMIC_CONF &&
+ (mbuf->pool->ops_index != hw_mp->ops_index ||
+ RTE_MBUF_HAS_EXTBUF(mbuf) ||
+ rte_mbuf_refcnt_read(mbuf) > 1 || tstamp))
+ need_dyconf = true;
+ if (conf == DPAA2_TX_ABSOLUTE_CONF || need_dyconf)
+ need_conf = true;
+
+ if (need_conf) {
+ dpaa2_mbuf_to_conf_contig_fd(mbuf, fd);
+ } else if (mbuf->pool->ops_index == hw_mp->ops_index) {
+ DPAA2_MBUF_TO_CONTIG_FD(mbuf, fd, mempool_to_bpid(mbuf->pool));
} else {
- struct rte_mbuf *mi;
-
- mi = rte_mbuf_from_indirect(mbuf);
- if (rte_mbuf_refcnt_read(mi) > 1)
- DPAA2_SET_FD_IVP(fd);
- else
- rte_mbuf_refcnt_update(mi, 1);
-
- buf_to_free[*free_count].seg = mbuf;
- buf_to_free[*free_count].pkt_id = pkt_id;
- ++*free_count;
+ copy_mbuf = dpaa2_dev_mbuf_copy_one_seg(mbuf, hw_mp);
+ if (!copy_mbuf) {
+ ret = -ENOMEM;
+ goto quit;
+ }
+ DPAA2_MBUF_TO_CONTIG_FD(copy_mbuf, fd, mempool_to_bpid(hw_mp));
+quit:
+ rte_pktmbuf_free(mbuf);
}
-}
-
-static inline int __rte_hot
-eth_copy_mbuf_to_fd(struct rte_mbuf *mbuf,
- struct qbman_fd *fd, uint16_t bpid)
-{
- struct rte_mbuf *m;
- void *mb = NULL;
- if (rte_dpaa2_mbuf_alloc_bulk(
- rte_dpaa2_bpid_info[bpid].bp_list->mp, &mb, 1)) {
- DPAA2_PMD_DP_DEBUG("Unable to allocated DPAA2 buffer");
- return -1;
+ if (need_dyconf) {
+ dpaa2_dev_tx_config_dynamic_confirm(fd, txq, true, tstamp);
+ if (dy_conf)
+ *dy_conf = true;
+ } else if (tstamp) {
+ dpaa2_dev_tx_enable_tstamp(fd);
}
- m = (struct rte_mbuf *)mb;
- memcpy((char *)m->buf_addr + mbuf->data_off,
- (void *)((char *)mbuf->buf_addr + mbuf->data_off),
- mbuf->pkt_len);
-
- /* Copy required fields */
- m->data_off = mbuf->data_off;
- m->ol_flags = mbuf->ol_flags;
- m->packet_type = mbuf->packet_type;
- m->tx_offload = mbuf->tx_offload;
-
- DPAA2_MBUF_TO_CONTIG_FD(m, fd, bpid);
-
-#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- rte_mempool_check_cookies(rte_mempool_from_obj((void *)m),
- (void **)&m, 1, 0);
-#endif
- DPAA2_PMD_DP_DEBUG(
- "mbuf: %p, BMAN buf addr: %p, fdaddr: %" PRIx64 ", bpid: %d,"
- " meta: %d, off: %d, len: %d",
- (void *)mbuf,
- mbuf->buf_addr,
- DPAA2_GET_FD_ADDR(fd),
- DPAA2_GET_FD_BPID(fd),
- rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size,
- DPAA2_GET_FD_OFFSET(fd),
- DPAA2_GET_FD_LEN(fd));
-return 0;
+ return ret;
}
static void
@@ -812,7 +957,6 @@ dump_err_pkts(struct dpaa2_queue *dpaa2_q)
dpaa2_dev_rx_parse_offset(priv, mbuf, fd);
dpaa2_dev_rx_print_parser_result(priv, fd, mbuf);
-
DPAA2_PMD_ERR("Err pkt on port[%d]:", eth_data->port_id);
DPAA2_PMD_ERR("FD offset: %d, FD err: %x, FAS status: %x",
DPAA2_GET_FD_OFFSET(fd), DPAA2_GET_FD_ERR(fd),
@@ -1237,12 +1381,6 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
if (priv->psr_dynfield_offset >= 0)
dpaa2_dev_rx_parse_offset(priv, bufs[num_rx], fd);
-#if defined(RTE_LIBRTE_IEEE1588)
- if (bufs[num_rx]->ol_flags & RTE_MBUF_F_RX_IEEE1588_TMST) {
- priv->rx_timestamp =
- *dpaa2_timestamp_dynfield(bufs[num_rx]);
- }
-#endif
dpaa2_dev_rx_print_parser_result(priv, fd, bufs[num_rx]);
dq_storage++;
@@ -1273,150 +1411,208 @@ dpaa2_dev_rx_channel(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
return dpaa2_dev_rx_common(queue, bufs, nb_pkts, true);
}
-uint16_t dpaa2_dev_tx_conf(void *queue)
+static inline int
+dpaa2_dev_is_mbuf_from_spec_pool(struct rte_mempool *mp,
+ struct rte_mbuf *mbuf)
{
- /* Function receive frames for a given device and VQ */
- struct dpaa2_queue *dpaa2_q = (struct dpaa2_queue *)queue;
- struct qbman_result *dq_storage;
+ while (mbuf) {
+ if (mbuf->pool != mp)
+ return false;
+ mbuf = mbuf->next;
+ }
+
+ return true;
+}
+
+uint16_t dpaa2_dev_tx_conf(void *txq, int drain)
+{
+ /* Function receive frames for a given device and VQ*/
+ struct dpaa2_queue *dpaa2_txq = txq;
+ struct dpaa2_queue *dpaa2_q = dpaa2_txq->tx_conf_queue;
+ struct qbman_result *dq_storage, *dq_storage1 = NULL;
uint32_t fqid = dpaa2_q->fqid;
- int ret, num_tx_conf = 0;
- uint8_t pending, status, num_pulled;
+ int ret, pull_size, total = 0, bulk_free;
+ uint8_t pending, status, idx, buf_idx;
struct qbman_swp *swp;
const struct qbman_fd *fd;
struct qbman_pull_desc pulldesc;
- struct qbman_release_desc releasedesc;
- uint32_t bpid;
- uint64_t buf;
-#if defined(RTE_LIBRTE_IEEE1588)
- struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
- struct dpaa2_dev_priv *priv = eth_data->dev_private;
- struct dpaa2_annot_hdr *annotation;
- void *v_addr;
- struct rte_mbuf *mbuf;
-#endif
+ struct queue_storage_info_t *q_storage;
+ struct qbman_result *rst;
- if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
- ret = dpaa2_affine_qbman_swp();
+ struct rte_mbuf *mbufs[dpaa2_dqrr_size];
+ struct rte_mempool *mp = NULL;
+ #define swp_idx (DPAA2_PER_LCORE_ETHRX_DPIO->index)
+
+conf_again:
+ bulk_free = true;
+ idx = 0;
+ q_storage = dpaa2_q->q_storage[rte_lcore_id()];
+ if (unlikely(!DPAA2_PER_LCORE_ETHRX_DPIO)) {
+ ret = dpaa2_affine_qbman_ethrx_swp();
if (ret) {
- DPAA2_PMD_ERR(
- "Failed to allocate IO portal, tid: %d",
- rte_gettid());
+ DPAA2_PMD_ERR("Failure in affining portal");
return 0;
}
}
- swp = DPAA2_PER_LCORE_PORTAL;
- do {
- dq_storage = dpaa2_q->q_storage[0]->dq_storage[0];
+ swp = DPAA2_PER_LCORE_ETHRX_PORTAL;
+ pull_size = dpaa2_dqrr_size;
+ if (unlikely(!q_storage->active_dqs)) {
+ q_storage->toggle = 0;
+ dq_storage = q_storage->dq_storage[q_storage->toggle];
+ q_storage->last_num_pkts = pull_size;
qbman_pull_desc_clear(&pulldesc);
+ qbman_pull_desc_set_numframes(&pulldesc,
+ q_storage->last_num_pkts);
qbman_pull_desc_set_fq(&pulldesc, fqid);
qbman_pull_desc_set_storage(&pulldesc, dq_storage,
- (size_t)(DPAA2_VADDR_TO_IOVA(dq_storage)), 1);
-
- qbman_pull_desc_set_numframes(&pulldesc, dpaa2_dqrr_size);
-
+ DPAA2_VADDR_TO_IOVA(dq_storage), 1);
+ if (check_swp_active_dqs(swp_idx)) {
+ rst = get_swp_active_dqs(swp_idx);
+ while (!qbman_check_command_complete(rst))
+ ;
+ clear_swp_active_dqs(swp_idx);
+ }
while (1) {
if (qbman_swp_pull(swp, &pulldesc)) {
- DPAA2_PMD_DP_DEBUG("VDQ command is not issued."
- "QBMAN is busy");
+ DPAA2_PMD_DP_DEBUG("QBMAN is busy (1)");
/* Portal was busy, try again */
continue;
}
break;
}
+ q_storage->active_dqs = dq_storage;
+ q_storage->active_dpio_id = swp_idx;
+ set_swp_active_dqs(swp_idx, dq_storage);
+ }
- rte_prefetch0((void *)((size_t)(dq_storage + 1)));
- /* Check if the previous issued command is completed. */
- while (!qbman_check_command_complete(dq_storage))
- ;
-
- num_pulled = 0;
- pending = 1;
- do {
- /* Loop until the dq_storage is updated with
- * new token by QBMAN
- */
- while (!qbman_check_new_result(dq_storage))
- ;
- rte_prefetch0((void *)((size_t)(dq_storage + 2)));
- /* Check whether Last Pull command is Expired and
- * setting Condition for Loop termination
- */
- if (qbman_result_DQ_is_pull_complete(dq_storage)) {
- pending = 0;
- /* Check for valid frame. */
- status = qbman_result_DQ_flags(dq_storage);
- if (unlikely((status &
- QBMAN_DQ_STAT_VALIDFRAME) == 0))
- continue;
- }
- if (dpaa2_svr_family != SVR_LX2160A)
- /** Packet type is parsed from FRC for LX2160A.*/
- dpaa2_dev_prefetch_next_psr(dq_storage);
+ dq_storage = q_storage->active_dqs;
+ rte_prefetch0((void *)(dq_storage));
+ rte_prefetch0((void *)(dq_storage + 1));
- fd = qbman_result_DQ_fd(dq_storage);
- bpid = DPAA2_GET_FD_BPID(fd);
+ /* Prepare next pull descriptor. This will give space for the
+ * prefetching done on DQRR entries
+ */
+ q_storage->toggle ^= 1;
+ dq_storage1 = q_storage->dq_storage[q_storage->toggle];
+ qbman_pull_desc_clear(&pulldesc);
+ qbman_pull_desc_set_numframes(&pulldesc, pull_size);
+ qbman_pull_desc_set_fq(&pulldesc, fqid);
+ qbman_pull_desc_set_storage(&pulldesc, dq_storage1,
+ DPAA2_VADDR_TO_IOVA(dq_storage1), 1);
- /* Create a release descriptor required for releasing
- * buffers into QBMAN
- */
- qbman_release_desc_clear(&releasedesc);
- qbman_release_desc_set_bpid(&releasedesc, bpid);
+ /* Check if the previous issued command is completed.
+ * Also seems like the SWP is shared between the Ethernet Driver
+ * and the SEC driver.
+ */
+ while (!qbman_check_command_complete(dq_storage))
+ ;
+ if (dq_storage == get_swp_active_dqs(q_storage->active_dpio_id))
+ clear_swp_active_dqs(q_storage->active_dpio_id);
- buf = DPAA2_GET_FD_ADDR(fd);
- /* feed them to bman */
- do {
- ret = qbman_swp_release(swp, &releasedesc,
- &buf, 1);
- } while (ret == -EBUSY);
+ pending = 1;
- dq_storage++;
- num_tx_conf++;
- num_pulled++;
-#if defined(RTE_LIBRTE_IEEE1588)
- v_addr = DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd));
- mbuf = DPAA2_INLINE_MBUF_FROM_BUF(v_addr,
- rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size);
-
- if (mbuf->ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST) {
- annotation = (struct dpaa2_annot_hdr *)((size_t)
- DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)) +
- DPAA2_FD_PTA_SIZE);
- priv->tx_timestamp = annotation->word2;
- }
-#endif
- } while (pending);
+ do {
+ /* Loop until the dq_storage is updated with
+ * new token by QBMAN
+ */
+ while (!qbman_check_new_result(dq_storage))
+ ;
+ rte_prefetch0((void *)(dq_storage + 2));
+ /* Check whether Last Pull command is Expired and
+ * setting Condition for Loop termination
+ */
+ if (qbman_result_DQ_is_pull_complete(dq_storage)) {
+ pending = 0;
+ /* Check for valid frame. */
+ status = qbman_result_DQ_flags(dq_storage);
+ if (unlikely((status & QBMAN_DQ_STAT_VALIDFRAME) == 0))
+ continue;
+ }
- /* Last VDQ provided all packets and more packets are requested */
- } while (num_pulled == dpaa2_dqrr_size);
+ fd = qbman_result_DQ_fd(dq_storage);
+ mbufs[idx] = (void *)DPAA2_GET_FD_FLC(fd);
+ if (unlikely(!mp))
+ mp = mbufs[idx]->pool;
+ if (unlikely(rte_mbuf_refcnt_read(mbufs[idx]) > 1))
+ bulk_free = false;
+ if (bulk_free &&
+ !dpaa2_dev_is_mbuf_from_spec_pool(mp, mbufs[idx]))
+ bulk_free = false;
+ idx++;
+ dq_storage++;
+ if (idx >= dpaa2_dqrr_size)
+ break;
+ } while (pending);
- dpaa2_q->rx_pkts += num_tx_conf;
+ if (check_swp_active_dqs(swp_idx)) {
+ rst = get_swp_active_dqs(swp_idx);
+ while (!qbman_check_command_complete(rst))
+ ;
+ clear_swp_active_dqs(swp_idx);
+ }
+ /* issue a volatile dequeue command for next pull */
+ while (1) {
+ if (qbman_swp_pull(swp, &pulldesc)) {
+ DPAA2_PMD_DP_DEBUG("QBMAN is busy (2)");
+ continue;
+ }
+ break;
+ }
+ q_storage->active_dqs = dq_storage1;
+ q_storage->active_dpio_id = swp_idx;
+ set_swp_active_dqs(swp_idx, dq_storage1);
- return num_tx_conf;
-}
-#ifdef RTE_LIBRTE_IEEE1588
-/* Configure the egress frame annotation for timestamp update */
-static void enable_tx_tstamp(struct qbman_fd *fd)
-{
- struct dpaa2_faead *fd_faead;
+ if (bulk_free) {
+ rte_pktmbuf_free_bulk(mbufs, idx);
+ } else {
+ for (buf_idx = 0; buf_idx < idx; buf_idx++) {
+ if (rte_mbuf_refcnt_read(mbufs[buf_idx]) > 1)
+ rte_mbuf_refcnt_update(mbufs[buf_idx], -1);
+ else
+ rte_pktmbuf_free(mbufs[buf_idx]);
+ }
+ }
- /* Set frame annotation status field as valid */
- (fd)->simple.frc |= DPAA2_FD_FRC_FASV;
+ dpaa2_q->rx_pkts += idx;
+ total += idx;
+ if (drain && pending)
+ goto conf_again;
- /* Set frame annotation egress action descriptor as valid */
- (fd)->simple.frc |= DPAA2_FD_FRC_FAEADV;
+ return total;
+}
- /* Set Annotation Length as 128B */
- (fd)->simple.ctrl |= DPAA2_FD_CTRL_ASAL;
+static inline int
+dpaa2_dev_tx_fast_mbuf_to_fd(struct rte_eth_dev_data *dev,
+ struct rte_mbuf *buf, struct qbman_fd *fd,
+ enum dpaa2_tx_conf_type conf)
+{
+ struct rte_mempool *mp = buf->pool;
+ struct dpaa2_dev_priv *priv = dev->dev_private;
+
+ if (likely(RTE_MBUF_DIRECT(buf) &&
+ mp && mp->ops_index ==
+ priv->bp_list->dpaa2_ops_index &&
+ buf->nb_segs == 1 &&
+ rte_mbuf_refcnt_read(buf) == 1)) {
+ if (unlikely(buf->next)) {
+ DPAA2_PMD_WARN("Single mbuf has next segment(%p)",
+ buf->next);
+ }
+ if (unlikely(conf == DPAA2_TX_ABSOLUTE_CONF))
+ dpaa2_mbuf_to_conf_contig_fd(buf, fd);
+ else
+ DPAA2_MBUF_TO_CONTIG_FD(buf, fd, mempool_to_bpid(mp));
+#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
+ rte_mempool_check_cookies(rte_mempool_from_obj(buf),
+ (void **)&buf, 1, 0);
+#endif
+ return 0;
+ }
- /* enable update of confirmation frame annotation */
- fd_faead = (struct dpaa2_faead *)((size_t)
- DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)) +
- DPAA2_FD_PTA_SIZE + DPAA2_FD_HW_ANNOT_FAEAD_OFFSET);
- fd_faead->ctrl = DPAA2_ANNOT_FAEAD_A2V | DPAA2_ANNOT_FAEAD_UPDV |
- DPAA2_ANNOT_FAEAD_UPD;
+ return -EAGAIN;
}
-#endif
+
/*
* Callback to handle sending packets through WRIOP based interface
*/
@@ -1424,28 +1620,24 @@ uint16_t
dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
{
/* Function to transmit the frames to given device and VQ*/
- uint32_t loop, retry_count;
- int32_t ret;
+ uint32_t loop, retry_count, i;
+ int32_t ret, tstamp[MAX_TX_RING_SLOTS];
struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
- struct rte_mbuf *mi;
uint32_t frames_to_send;
- struct rte_mempool *mp;
struct qbman_eq_desc eqdesc;
- struct dpaa2_queue *dpaa2_q = (struct dpaa2_queue *)queue;
+ struct dpaa2_queue *dpaa2_q = queue;
struct qbman_swp *swp;
uint16_t num_tx = 0;
- uint16_t bpid;
struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
struct dpaa2_dev_priv *priv = eth_data->dev_private;
+ struct rte_mempool *hw_mp;
uint32_t flags[MAX_TX_RING_SLOTS] = {0};
- struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
- uint32_t free_count = 0;
+ uint8_t dy_conf[MAX_TX_RING_SLOTS];
if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
ret = dpaa2_affine_qbman_swp();
if (ret) {
- DPAA2_PMD_ERR(
- "Failed to allocate IO portal, tid: %d",
+ DPAA2_PMD_ERR("Failed to allocate IO portal, tid: %d",
rte_gettid());
return 0;
}
@@ -1453,19 +1645,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
swp = DPAA2_PER_LCORE_PORTAL;
DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d",
- eth_data, dpaa2_q->fqid);
+ eth_data, dpaa2_q->fqid);
-#ifdef RTE_LIBRTE_IEEE1588
- /* IEEE1588 driver need pointer to tx confirmation queue
- * corresponding to last packet transmitted for reading
- * the timestamp
- */
- if ((*bufs)->ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST) {
- priv->next_tx_conf_queue = dpaa2_q->tx_conf_queue;
- dpaa2_dev_tx_conf(dpaa2_q->tx_conf_queue);
- priv->tx_timestamp = 0;
+ if (unlikely(!priv->bp_list)) {
+ DPAA2_PMD_ERR("%s's buffer pool not initialized!",
+ eth_data->name);
+ return 0;
}
-#endif
+ hw_mp = priv->bp_list->mp;
+
+ if (dpaa2_q->tx_conf_queue)
+ dpaa2_dev_tx_conf(dpaa2_q, false);
/*Prepare enqueue descriptor*/
qbman_eq_desc_clear(&eqdesc);
@@ -1473,214 +1663,119 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
qbman_eq_desc_set_fq(&eqdesc, dpaa2_q->fqid);
/*Clear the unused FD fields before sending*/
- while (nb_pkts) {
- /*Check if the queue is congested*/
- retry_count = 0;
- while (qbman_result_SCN_state(dpaa2_q->cscn)) {
- retry_count++;
- /* Retry for some time before giving up */
- if (retry_count > CONG_RETRY_COUNT) {
- if (dpaa2_q->tm_sw_td)
- goto sw_td;
- goto skip_tx;
- }
- }
-
- frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
- dpaa2_eqcr_size : nb_pkts;
- for (loop = 0; loop < frames_to_send; loop++) {
- if (*dpaa2_seqn(*bufs)) {
- uint8_t dqrr_index = *dpaa2_seqn(*bufs) - 1;
-
- flags[loop] = QBMAN_ENQUEUE_FLAG_DCA |
- dqrr_index;
- DPAA2_PER_LCORE_DQRR_SIZE--;
- DPAA2_PER_LCORE_DQRR_HELD &= ~(UINT64_C(1) << dqrr_index);
- *dpaa2_seqn(*bufs) = DPAA2_INVALID_MBUF_SEQN;
- }
+tx_again:
+ /*Check if the queue is congested*/
+ retry_count = 0;
+ while (qbman_result_SCN_state(dpaa2_q->cscn)) {
+ retry_count++;
+ /* Retry for some time before giving up */
+ if (retry_count > CONG_RETRY_COUNT)
+ goto skip_tx;
+ }
- if (likely(RTE_MBUF_DIRECT(*bufs))) {
- mp = (*bufs)->pool;
- /* Check the basic scenario and set
- * the FD appropriately here itself.
- */
- if (likely(mp && mp->ops_index ==
- priv->bp_list->dpaa2_ops_index &&
- (*bufs)->nb_segs == 1 &&
- rte_mbuf_refcnt_read((*bufs)) == 1)) {
- if (unlikely((*bufs)->ol_flags
- & RTE_MBUF_F_TX_VLAN)) {
- ret = rte_vlan_insert(bufs);
- if (ret)
- goto send_n_return;
- }
- DPAA2_MBUF_TO_CONTIG_FD((*bufs),
- &fd_arr[loop], mempool_to_bpid(mp));
-#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
- rte_mempool_check_cookies
- (rte_mempool_from_obj((void *)*bufs),
- (void **)bufs, 1, 0);
-#endif
- bufs++;
-#ifdef RTE_LIBRTE_IEEE1588
- enable_tx_tstamp(&fd_arr[loop]);
-#endif
- continue;
- }
- } else {
- mi = rte_mbuf_from_indirect(*bufs);
- mp = mi->pool;
- }
+ frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
+ dpaa2_eqcr_size : nb_pkts;
- if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
- if (unlikely((*bufs)->nb_segs > 1)) {
- mp = (*bufs)->pool;
- if (eth_mbuf_to_sg_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop,
- mempool_to_bpid(mp)))
- goto send_n_return;
- } else {
- eth_mbuf_to_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop, 0);
- }
- bufs++;
-#ifdef RTE_LIBRTE_IEEE1588
- enable_tx_tstamp(&fd_arr[loop]);
-#endif
- continue;
- }
+ for (loop = 0; loop < frames_to_send; loop++) {
+ dy_conf[loop] = false;
+ if (*dpaa2_seqn(*bufs)) {
+ uint8_t dqrr_index = *dpaa2_seqn(*bufs) - 1;
+
+ flags[loop] = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index;
+ DPAA2_PER_LCORE_DQRR_SIZE--;
+ DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dqrr_index);
+ *dpaa2_seqn(*bufs) = DPAA2_INVALID_MBUF_SEQN;
+ }
- /* Not a hw_pkt pool allocated frame */
- if (unlikely(!mp || !priv->bp_list)) {
- DPAA2_PMD_ERR("Err: No buffer pool attached");
+ if (unlikely((*bufs)->ol_flags
+ & RTE_MBUF_F_TX_VLAN)) {
+ ret = rte_vlan_insert(bufs);
+ if (ret)
goto send_n_return;
- }
+ }
- if (unlikely((*bufs)->ol_flags & RTE_MBUF_F_TX_VLAN)) {
- int ret = rte_vlan_insert(bufs);
- if (ret)
- goto send_n_return;
- }
- if (mp->ops_index != priv->bp_list->dpaa2_ops_index) {
- DPAA2_PMD_WARN("Non DPAA2 buffer pool");
- /* alloc should be from the default buffer pool
- * attached to this interface
- */
- bpid = priv->bp_list->buf_pool.bpid;
-
- if (unlikely((*bufs)->nb_segs > 1)) {
- DPAA2_PMD_ERR("S/G support not added"
- " for non hw offload buffer");
- goto send_n_return;
- }
- if (eth_copy_mbuf_to_fd(*bufs,
- &fd_arr[loop], bpid)) {
- goto send_n_return;
- }
- /* free the original packet */
- rte_pktmbuf_free(*bufs);
- } else {
- bpid = mempool_to_bpid(mp);
- if (unlikely((*bufs)->nb_segs > 1)) {
- if (eth_mbuf_to_sg_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop,
- bpid))
- goto send_n_return;
- } else {
- eth_mbuf_to_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop, bpid);
- }
- }
-#ifdef RTE_LIBRTE_IEEE1588
- enable_tx_tstamp(&fd_arr[loop]);
-#endif
+ tstamp[loop] = false;
+
+ ret = dpaa2_dev_tx_fast_mbuf_to_fd(eth_data,
+ *bufs, &fd_arr[loop], priv->tx_conf_type);
+ if (likely(!ret)) {
bufs++;
+ continue;
}
- loop = 0;
- retry_count = 0;
- while (loop < frames_to_send) {
- ret = qbman_swp_enqueue_multiple(swp, &eqdesc,
- &fd_arr[loop], &flags[loop],
- frames_to_send - loop);
- if (unlikely(ret < 0)) {
- retry_count++;
- if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) {
- num_tx += loop;
- nb_pkts -= loop;
- goto send_n_return;
- }
- } else {
- loop += ret;
- retry_count = 0;
- }
+ if (unlikely((*bufs)->nb_segs > 1)) {
+ ret = dpaa2_dev_tx_mbuf_to_sg_fd(hw_mp, *bufs, &fd_arr[loop],
+ dpaa2_q, priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
+ } else {
+ ret = dpaa2_dev_tx_mbuf_to_simple_fd(hw_mp, *bufs, &fd_arr[loop],
+ dpaa2_q, priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
}
+ if (ret)
+ goto send_n_return;
+ bufs++;
+ }
- num_tx += loop;
- nb_pkts -= loop;
+ loop = 0;
+ retry_count = 0;
+ while (loop < frames_to_send) {
+ ret = qbman_swp_enqueue_multiple(swp, &eqdesc,
+ &fd_arr[loop], &flags[loop],
+ frames_to_send - loop);
+ if (unlikely(ret < 0)) {
+ retry_count++;
+ if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) {
+ num_tx += loop;
+ nb_pkts -= loop;
+ goto send_n_return;
+ }
+ } else {
+ loop += ret;
+ retry_count = 0;
+ }
}
- dpaa2_q->tx_pkts += num_tx;
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+ num_tx += loop;
+ nb_pkts -= loop;
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
+ for (i = 0; i < loop; i++) {
+ if (tstamp[i])
+ dpaa2_q->tx_conf_queue->ts_to_cnfd++;
+ }
}
+ if (nb_pkts > 0)
+ goto tx_again;
+
+ dpaa2_q->tx_pkts += num_tx;
return num_tx;
send_n_return:
/* send any already prepared fd */
- if (loop) {
- unsigned int i = 0;
-
- retry_count = 0;
- while (i < loop) {
- ret = qbman_swp_enqueue_multiple(swp, &eqdesc,
- &fd_arr[i],
- &flags[i],
- loop - i);
- if (unlikely(ret < 0)) {
- retry_count++;
- if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
- break;
- } else {
- i += ret;
- retry_count = 0;
- }
+ retry_count = 0;
+ i = 0;
+ while (i < loop) {
+ ret = qbman_swp_enqueue_multiple(swp, &eqdesc, &fd_arr[i],
+ &flags[i], loop - i);
+ if (unlikely(ret < 0)) {
+ retry_count++;
+ if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
+ break;
+ } else {
+ i += ret;
+ retry_count = 0;
}
- num_tx += i;
- }
-skip_tx:
- dpaa2_q->tx_pkts += num_tx;
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
}
-
- return num_tx;
-sw_td:
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+ num_tx += i;
+ if (priv->tx_conf_type != DPAA2_TX_NO_CONF) {
+ loop = i;
+ for (i = 0; i < loop; i++) {
+ if (tstamp[i])
+ dpaa2_q->tx_conf_queue->ts_to_cnfd++;
+ }
}
- /* free the pending buffers */
- rte_pktmbuf_free_bulk(bufs, nb_pkts);
-
- num_tx += nb_pkts;
+skip_tx:
dpaa2_q->tx_pkts += num_tx;
return num_tx;
@@ -1755,149 +1850,110 @@ dpaa2_set_enqueue_descriptor(struct dpaa2_queue *dpaa2_q,
RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dev_tx_multi_txq_ordered)
uint16_t
dpaa2_dev_tx_multi_txq_ordered(void **queue,
- struct rte_mbuf **bufs, uint16_t nb_pkts)
+ struct rte_mbuf **bufs, uint16_t nb_pkts)
{
/* Function to transmit the frames to multiple queues respectively.*/
- uint32_t loop, i, retry_count;
- int32_t ret;
+ uint32_t loop, retry_count, sent = 0, i;
+ int32_t ret = 0, tstamp[MAX_TX_RING_SLOTS];
struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
uint32_t frames_to_send, num_free_eq_desc = 0;
- struct rte_mempool *mp;
+ struct rte_mempool *hw_mp;
struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS];
struct dpaa2_queue *dpaa2_q[MAX_TX_RING_SLOTS];
struct qbman_swp *swp;
- uint16_t bpid;
- struct rte_mbuf *mi;
struct rte_eth_dev_data *eth_data;
struct dpaa2_dev_priv *priv;
struct dpaa2_queue *order_sendq;
- struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
- uint32_t free_count = 0;
+ uint8_t dy_conf[MAX_TX_RING_SLOTS];
if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
ret = dpaa2_affine_qbman_swp();
if (ret) {
- DPAA2_PMD_ERR(
- "Failed to allocate IO portal, tid: %d",
+ DPAA2_PMD_ERR("Failed to allocate IO portal, tid: %d",
rte_gettid());
return 0;
}
}
swp = DPAA2_PER_LCORE_PORTAL;
+tx_again:
frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
dpaa2_eqcr_size : nb_pkts;
for (loop = 0; loop < frames_to_send; loop++) {
- dpaa2_q[loop] = (struct dpaa2_queue *)queue[loop];
+ dpaa2_q[loop] = queue[loop];
+ if (dpaa2_q[loop]->tx_conf_queue)
+ dpaa2_dev_tx_conf(dpaa2_q[loop], false);
eth_data = dpaa2_q[loop]->eth_data;
priv = eth_data->dev_private;
- if (!priv->en_loose_ordered) {
- if (*dpaa2_seqn(*bufs) & DPAA2_ENQUEUE_FLAG_ORP) {
+ dy_conf[loop] = false;
+ if (unlikely(!priv->bp_list)) {
+ DPAA2_PMD_ERR("%s's buffer pool not initialized!",
+ eth_data->name);
+ ret = -ENOMEM;
+ goto send_frames;
+ }
+ hw_mp = priv->bp_list->mp;
+ if (!priv->en_loose_ordered &&
+ (*dpaa2_seqn(*bufs) & DPAA2_ENQUEUE_FLAG_ORP)) {
+ if (!num_free_eq_desc) {
+ num_free_eq_desc = dpaa2_free_eq_descriptors();
if (!num_free_eq_desc) {
- num_free_eq_desc = dpaa2_free_eq_descriptors();
- if (!num_free_eq_desc)
- goto send_frames;
+ ret = -EIO;
+ goto send_frames;
}
- num_free_eq_desc--;
}
+ num_free_eq_desc--;
}
DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d",
- eth_data, dpaa2_q[loop]->fqid);
+ eth_data, dpaa2_q[loop]->fqid);
/* Check if the queue is congested */
retry_count = 0;
while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
retry_count++;
/* Retry for some time before giving up */
- if (retry_count > CONG_RETRY_COUNT)
+ if (retry_count > CONG_RETRY_COUNT) {
+ ret = -ETIME;
goto send_frames;
+ }
}
/* Prepare enqueue descriptor */
qbman_eq_desc_clear(&eqdesc[loop]);
if (*dpaa2_seqn(*bufs) && priv->en_ordered) {
- order_sendq = (struct dpaa2_queue *)priv->tx_vq[0];
- dpaa2_set_enqueue_descriptor(order_sendq,
- (*bufs),
- &eqdesc[loop]);
+ order_sendq = priv->tx_vq[0];
+ dpaa2_set_enqueue_descriptor(order_sendq, *bufs,
+ &eqdesc[loop]);
} else {
- qbman_eq_desc_set_no_orp(&eqdesc[loop],
- DPAA2_EQ_RESP_ERR_FQ);
- qbman_eq_desc_set_fq(&eqdesc[loop],
- dpaa2_q[loop]->fqid);
+ qbman_eq_desc_set_no_orp(&eqdesc[loop], DPAA2_EQ_RESP_ERR_FQ);
+ qbman_eq_desc_set_fq(&eqdesc[loop], dpaa2_q[loop]->fqid);
}
- if (likely(RTE_MBUF_DIRECT(*bufs))) {
- mp = (*bufs)->pool;
- /* Check the basic scenario and set
- * the FD appropriately here itself.
- */
- if (likely(mp && mp->ops_index ==
- priv->bp_list->dpaa2_ops_index &&
- (*bufs)->nb_segs == 1 &&
- rte_mbuf_refcnt_read((*bufs)) == 1)) {
- if (unlikely((*bufs)->ol_flags
- & RTE_MBUF_F_TX_VLAN)) {
- ret = rte_vlan_insert(bufs);
- if (ret)
- goto send_frames;
- }
- DPAA2_MBUF_TO_CONTIG_FD((*bufs),
- &fd_arr[loop],
- mempool_to_bpid(mp));
- bufs++;
- continue;
- }
- } else {
- mi = rte_mbuf_from_indirect(*bufs);
- mp = mi->pool;
- }
- /* Not a hw_pkt pool allocated frame */
- if (unlikely(!mp || !priv->bp_list)) {
- DPAA2_PMD_ERR("Err: No buffer pool attached");
- goto send_frames;
- }
+ tstamp[loop] = false;
- if (mp->ops_index != priv->bp_list->dpaa2_ops_index) {
- DPAA2_PMD_WARN("Non DPAA2 buffer pool");
- /* alloc should be from the default buffer pool
- * attached to this interface
- */
- bpid = priv->bp_list->buf_pool.bpid;
+ ret = dpaa2_dev_tx_fast_mbuf_to_fd(eth_data,
+ *bufs, &fd_arr[loop], priv->tx_conf_type);
+ if (likely(!ret)) {
+ if (priv->tx_conf_type == DPAA2_TX_ABSOLUTE_CONF)
+ dy_conf[loop] = true;
+ bufs++;
+ continue;
+ }
- if (unlikely((*bufs)->nb_segs > 1)) {
- DPAA2_PMD_ERR(
- "S/G not supp for non hw offload buffer");
- goto send_frames;
- }
- if (eth_copy_mbuf_to_fd(*bufs,
- &fd_arr[loop], bpid)) {
- goto send_frames;
- }
- /* free the original packet */
- rte_pktmbuf_free(*bufs);
+ if (unlikely((*bufs)->nb_segs > 1)) {
+ ret = dpaa2_dev_tx_mbuf_to_sg_fd(hw_mp, *bufs, &fd_arr[loop],
+ dpaa2_q[loop], priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
} else {
- bpid = mempool_to_bpid(mp);
- if (unlikely((*bufs)->nb_segs > 1)) {
- if (eth_mbuf_to_sg_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop,
- bpid))
- goto send_frames;
- } else {
- eth_mbuf_to_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop, bpid);
- }
+ ret = dpaa2_dev_tx_mbuf_to_simple_fd(hw_mp, *bufs, &fd_arr[loop],
+ dpaa2_q[loop], priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
}
-
+ if (ret)
+ goto send_frames;
+ if (priv->tx_conf_type == DPAA2_TX_ABSOLUTE_CONF)
+ dy_conf[loop] = true;
bufs++;
}
@@ -1918,237 +1974,30 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
break;
}
}
-
- for (i = 0; i < free_count; i++) {
- if (buf_to_free[i].pkt_id < loop)
- rte_pktmbuf_free_seg(buf_to_free[i].seg);
+ nb_pkts -= loop;
+ sent += loop;
+ for (i = 0; i < loop; i++) {
+ if (tstamp[i] && dpaa2_q[i]->tx_conf_queue)
+ dpaa2_q[i]->tx_conf_queue->ts_to_cnfd++;
}
- return loop;
+ if (nb_pkts > 0 && !ret)
+ goto tx_again;
+
+ return sent;
}
/* Callback to handle sending ordered packets through WRIOP based interface */
uint16_t
-dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs,
+ uint16_t nb_pkts)
{
- /* Function to transmit the frames to given device and VQ*/
- struct dpaa2_queue *dpaa2_q = (struct dpaa2_queue *)queue;
- struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
- struct dpaa2_dev_priv *priv = eth_data->dev_private;
- struct dpaa2_queue *order_sendq = (struct dpaa2_queue *)priv->tx_vq[0];
- struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
- struct rte_mbuf *mi;
- struct rte_mempool *mp;
- struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS];
- struct qbman_swp *swp;
- uint32_t frames_to_send, num_free_eq_desc;
- uint32_t loop, retry_count;
- int32_t ret;
- uint16_t num_tx = 0;
- uint16_t bpid;
- struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
- uint32_t free_count = 0;
+ uint16_t i;
+ void *mq[nb_pkts];
- if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
- ret = dpaa2_affine_qbman_swp();
- if (ret) {
- DPAA2_PMD_ERR(
- "Failed to allocate IO portal, tid: %d",
- rte_gettid());
- return 0;
- }
- }
- swp = DPAA2_PER_LCORE_PORTAL;
+ for (i = 0; i < nb_pkts; i++)
+ mq[i] = queue;
- DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d",
- eth_data, dpaa2_q->fqid);
-
- /* This would also handle normal and atomic queues as any type
- * of packet can be enqueued when ordered queues are being used.
- */
- while (nb_pkts) {
- /*Check if the queue is congested*/
- retry_count = 0;
- while (qbman_result_SCN_state(dpaa2_q->cscn)) {
- retry_count++;
- /* Retry for some time before giving up */
- if (retry_count > CONG_RETRY_COUNT) {
- if (dpaa2_q->tm_sw_td)
- goto sw_td;
- goto skip_tx;
- }
- }
-
- frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
- dpaa2_eqcr_size : nb_pkts;
-
- if (!priv->en_loose_ordered) {
- if (*dpaa2_seqn(*bufs) & DPAA2_ENQUEUE_FLAG_ORP) {
- num_free_eq_desc = dpaa2_free_eq_descriptors();
- if (num_free_eq_desc < frames_to_send)
- frames_to_send = num_free_eq_desc;
- }
- }
-
- for (loop = 0; loop < frames_to_send; loop++) {
- /*Prepare enqueue descriptor*/
- qbman_eq_desc_clear(&eqdesc[loop]);
-
- if (*dpaa2_seqn(*bufs)) {
- /* Use only queue 0 for Tx in case of atomic/
- * ordered packets as packets can get unordered
- * when being transmitted out from the interface
- */
- dpaa2_set_enqueue_descriptor(order_sendq,
- (*bufs),
- &eqdesc[loop]);
- } else {
- qbman_eq_desc_set_no_orp(&eqdesc[loop],
- DPAA2_EQ_RESP_ERR_FQ);
- qbman_eq_desc_set_fq(&eqdesc[loop],
- dpaa2_q->fqid);
- }
-
- if (likely(RTE_MBUF_DIRECT(*bufs))) {
- mp = (*bufs)->pool;
- /* Check the basic scenario and set
- * the FD appropriately here itself.
- */
- if (likely(mp && mp->ops_index ==
- priv->bp_list->dpaa2_ops_index &&
- (*bufs)->nb_segs == 1 &&
- rte_mbuf_refcnt_read((*bufs)) == 1)) {
- if (unlikely((*bufs)->ol_flags
- & RTE_MBUF_F_TX_VLAN)) {
- ret = rte_vlan_insert(bufs);
- if (ret)
- goto send_n_return;
- }
- DPAA2_MBUF_TO_CONTIG_FD((*bufs),
- &fd_arr[loop],
- mempool_to_bpid(mp));
- bufs++;
- continue;
- }
- } else {
- mi = rte_mbuf_from_indirect(*bufs);
- mp = mi->pool;
- }
- /* Not a hw_pkt pool allocated frame */
- if (unlikely(!mp || !priv->bp_list)) {
- DPAA2_PMD_ERR("Err: No buffer pool attached");
- goto send_n_return;
- }
-
- if (mp->ops_index != priv->bp_list->dpaa2_ops_index) {
- DPAA2_PMD_WARN("Non DPAA2 buffer pool");
- /* alloc should be from the default buffer pool
- * attached to this interface
- */
- bpid = priv->bp_list->buf_pool.bpid;
-
- if (unlikely((*bufs)->nb_segs > 1)) {
- DPAA2_PMD_ERR(
- "S/G not supp for non hw offload buffer");
- goto send_n_return;
- }
- if (eth_copy_mbuf_to_fd(*bufs,
- &fd_arr[loop], bpid)) {
- goto send_n_return;
- }
- /* free the original packet */
- rte_pktmbuf_free(*bufs);
- } else {
- bpid = mempool_to_bpid(mp);
- if (unlikely((*bufs)->nb_segs > 1)) {
- if (eth_mbuf_to_sg_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop,
- bpid))
- goto send_n_return;
- } else {
- eth_mbuf_to_fd(*bufs,
- &fd_arr[loop],
- buf_to_free,
- &free_count,
- loop, bpid);
- }
- }
- bufs++;
- }
-
- loop = 0;
- retry_count = 0;
- while (loop < frames_to_send) {
- ret = qbman_swp_enqueue_multiple_desc(swp,
- &eqdesc[loop], &fd_arr[loop],
- frames_to_send - loop);
- if (unlikely(ret < 0)) {
- retry_count++;
- if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) {
- num_tx += loop;
- nb_pkts -= loop;
- goto send_n_return;
- }
- } else {
- loop += ret;
- retry_count = 0;
- }
- }
-
- num_tx += loop;
- nb_pkts -= loop;
- }
- dpaa2_q->tx_pkts += num_tx;
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
- }
-
- return num_tx;
-
-send_n_return:
- /* send any already prepared fd */
- if (loop) {
- unsigned int i = 0;
-
- retry_count = 0;
- while (i < loop) {
- ret = qbman_swp_enqueue_multiple_desc(swp,
- &eqdesc[i], &fd_arr[i], loop - i);
- if (unlikely(ret < 0)) {
- retry_count++;
- if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
- break;
- } else {
- i += ret;
- retry_count = 0;
- }
- }
- num_tx += i;
- }
-skip_tx:
- dpaa2_q->tx_pkts += num_tx;
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
- }
-
- return num_tx;
-sw_td:
- for (loop = 0; loop < free_count; loop++) {
- if (buf_to_free[loop].pkt_id < num_tx)
- rte_pktmbuf_free_seg(buf_to_free[loop].seg);
- }
-
- /* free the pending buffers */
- rte_pktmbuf_free_bulk(bufs, nb_pkts);
-
- num_tx += nb_pkts;
- dpaa2_q->tx_pkts += num_tx;
-
- return num_tx;
+ return dpaa2_dev_tx_multi_txq_ordered(mq, bufs, nb_pkts);
}
/* This function loopbacks all the received packets.*/
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 33/45] net/dpaa2: ptp enhancements
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (31 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 32/45] drivers: rework dpaa2 Tx confirmation Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 34/45] net/dpaa2: remove unused soft parser Tx code Prashant Gupta
` (11 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
1) Correct TX timestamp only for PTP sync packets.
2) Runtime configure TX timestamp offset by parsing current TX
PTP packet format.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 92 ++--------------
drivers/net/dpaa2/dpaa2_ethdev.h | 30 +++++-
drivers/net/dpaa2/dpaa2_ptp.c | 176 +++++++++++++++++++++++++------
drivers/net/dpaa2/dpaa2_rxtx.c | 77 +++++++++++++-
4 files changed, 252 insertions(+), 123 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 61b7ae138c..58b4cedee3 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -72,11 +72,6 @@ static uint64_t dev_tx_offloads_sup =
static uint64_t dev_tx_offloads_nodis =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
-/* enable timestamp in mbuf */
-bool dpaa2_enable_ts[RTE_MAX_ETHPORTS];
-uint64_t dpaa2_timestamp_rx_dynflag;
-int dpaa2_timestamp_dynfield_offset = -1;
-
bool dpaa2_print_parser_result;
/* Rx descriptor limit when DPNI loads PFDRs in PEB */
@@ -730,9 +725,6 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
int tx_l4_csum_offload = false;
int ret, tc_index;
uint32_t max_rx_pktlen;
-#if defined(RTE_LIBRTE_IEEE1588)
- uint16_t ptp_correction_offset;
-#endif
PMD_INIT_FUNC_TRACE();
@@ -814,25 +806,15 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
return ret;
}
-#if !defined(RTE_LIBRTE_IEEE1588)
- if (rx_offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
-#endif
- {
- ret = rte_mbuf_dyn_rx_timestamp_register(
- &dpaa2_timestamp_dynfield_offset,
- &dpaa2_timestamp_rx_dynflag);
- if (ret != 0) {
+ if (rx_offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ ret = rte_mbuf_dyn_rx_timestamp_register(&priv->rx_ts_offset,
+ &priv->rx_ts_flag);
+ if (ret) {
DPAA2_PMD_ERR("Error to register timestamp field/flag");
- return -rte_errno;
+ return ret;
}
- dpaa2_enable_ts[dev->data->port_id] = true;
}
-#if defined(RTE_LIBRTE_IEEE1588)
- /* By default setting ptp correction offset for Ethernet SYNC packets */
- ptp_correction_offset = RTE_ETHER_HDR_LEN + 8;
- rte_pmd_dpaa2_set_one_step_ts(dev->data->port_id, ptp_correction_offset, 0);
-#endif
if (tx_offloads & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM)
tx_l3_csum_offload = true;
@@ -3315,8 +3297,6 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.rxq_info_get = dpaa2_rxq_info_get,
.txq_info_get = dpaa2_txq_info_get,
.tm_ops_get = dpaa2_tm_ops_get,
- .mtr_ops_get = dpaa2_mtr_ops_get,
-#if defined(RTE_LIBRTE_IEEE1588)
.timesync_enable = dpaa2_timesync_enable,
.timesync_disable = dpaa2_timesync_disable,
.timesync_read_time = dpaa2_timesync_read_time,
@@ -3324,7 +3304,7 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.timesync_adjust_time = dpaa2_timesync_adjust_time,
.timesync_read_rx_timestamp = dpaa2_timesync_read_rx_timestamp,
.timesync_read_tx_timestamp = dpaa2_timesync_read_tx_timestamp,
-#endif
+ .mtr_ops_get = dpaa2_mtr_ops_get,
};
/* Populate the mac address from physically available (u-boot/firmware) and/or
@@ -3589,11 +3569,6 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
priv->options = attr.options;
priv->max_mac_filters = attr.mac_filter_entries;
priv->max_vlan_filters = attr.vlan_filter_entries;
- priv->flags = 0;
-#if defined(RTE_LIBRTE_IEEE1588)
- DPAA2_PMD_INFO("DPDK IEEE1588 is enabled");
- priv->flags |= DPAA2_TX_CONF_ENABLE;
-#endif
priv->tx_conf_type = DPAA2_TX_NO_CONF;
/* Used with ``fslmc:dpni.1,drv_tx_conf=1`` */
@@ -3803,61 +3778,6 @@ rte_pmd_dpaa2_ep_name(uint32_t eth_id)
return priv->ep_name;
}
-#if defined(RTE_LIBRTE_IEEE1588)
-RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pmd_dpaa2_get_one_step_ts, 24.11)
-int
-rte_pmd_dpaa2_get_one_step_ts(uint16_t port_id, bool mc_query)
-{
- struct rte_eth_dev *dev = &rte_eth_devices[port_id];
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct fsl_mc_io *dpni = priv->eth_dev->process_private;
- struct dpni_single_step_cfg ptp_cfg;
- int err;
-
- if (!mc_query)
- return priv->ptp_correction_offset;
-
- err = dpni_get_single_step_cfg(dpni, CMD_PRI_LOW, priv->token, &ptp_cfg);
- if (err) {
- DPAA2_PMD_ERR("Failed to retrieve onestep configuration");
- return err;
- }
-
- if (!ptp_cfg.ptp_onestep_reg_base) {
- DPAA2_PMD_ERR("1588 onestep reg not available");
- return -1;
- }
-
- priv->ptp_correction_offset = ptp_cfg.offset;
-
- return priv->ptp_correction_offset;
-}
-
-RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pmd_dpaa2_set_one_step_ts, 24.11)
-int
-rte_pmd_dpaa2_set_one_step_ts(uint16_t port_id, uint16_t offset, uint8_t ch_update)
-{
- struct rte_eth_dev *dev = &rte_eth_devices[port_id];
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct fsl_mc_io *dpni = dev->process_private;
- struct dpni_single_step_cfg cfg;
- int err;
-
- cfg.en = 1;
- cfg.ch_update = ch_update;
- cfg.offset = offset;
- cfg.peer_delay = 0;
-
- err = dpni_set_single_step_cfg(dpni, CMD_PRI_LOW, priv->token, &cfg);
- if (err)
- return err;
-
- priv->ptp_correction_offset = offset;
-
- return 0;
-}
-#endif
-
static int dpaa2_tx_sg_pool_init(void)
{
char name[RTE_MEMZONE_NAMESIZE];
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index c6a8b4f595..e7490b1e0c 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -8,6 +8,7 @@
#ifndef _DPAA2_ETHDEV_H
#define _DPAA2_ETHDEV_H
+#include <rte_time.h>
#include <rte_compat.h>
#include <rte_event_eth_rx_adapter.h>
#include <rte_pmd_dpaa2.h>
@@ -101,6 +102,11 @@
#define DPAA2_RX_PRINT_PSR_RESULT_FLAG RTE_BIT32(12)
+#define DPAA2_IEEE1588_DEBUG_FLAG RTE_BIT32(13)
+
+#define DPAA2_IEEE1588_TX_TS_FLAG RTE_BIT32(14)
+#define DPAA2_IEEE1588_RX_TS_FLAG RTE_BIT32(15)
+
/* DPDMUX index for DPMAC */
#define DPAA2_DPDMUX_DPMAC_IDX 0
@@ -507,16 +513,16 @@ struct dpaa2_dev_priv {
uint64_t ss_iova;
uint64_t ss_param_iova;
/*stores timestamp of last received packet on dev*/
- uint64_t rx_timestamp;
+ RTE_ATOMIC(uint64_t) rx_timestamp;
/*stores timestamp of last received tx confirmation packet on dev*/
- uint64_t tx_timestamp;
+ RTE_ATOMIC(uint64_t) tx_timestamp;
int rx_ts_offset;
uint64_t rx_ts_flag;
/* stores next tx queue to be confirmed that should be processed,
* it corresponds to last packet transmitted
*/
- struct dpaa2_queue *next_txq_to_cnf;
+ RTE_ATOMIC(struct dpaa2_queue *) next_txq_to_cnf;
struct rte_eth_dev *eth_dev; /**< Pointer back to holding ethdev */
rte_spinlock_t lpbk_qp_lock;
@@ -591,6 +597,21 @@ dpaa2_dev_rx_print_parser_result(struct dpaa2_dev_priv *priv,
}
}
+static inline void
+dpaa2_timestamp_debug(struct dpaa2_dev_priv *priv,
+ const char *prefix, uint64_t timestamp)
+{
+ struct timespec ts;
+
+ if (likely(!(priv->flags & DPAA2_IEEE1588_DEBUG_FLAG)))
+ return;
+
+ ts = rte_ns_to_timespec(timestamp);
+ DPAA2_PMD_DEBUG("DPAA2 TS DBG: %s: ns(%" PRIu64 ")->%" PRId64
+ " seconds/%" PRId64 " nanoseconds",
+ prefix, timestamp, (int64_t)ts.tv_sec, (int64_t)ts.tv_nsec);
+}
+
int dpaa2_distset_to_dpkg_profile_cfg(uint64_t req_dist_set,
struct dpkg_profile_cfg *kg_cfg);
@@ -653,6 +674,9 @@ void dpaa2_dev_free_eqresp_buf(uint16_t eqresp_ci, struct dpaa2_queue *dpaa2_q);
void dpaa2_flow_clean(struct rte_eth_dev *dev);
uint16_t dpaa2_dev_tx_conf(void *txq, int drain);
+void
+dpaa2_dev_tx_ptp_one_step_runtime(struct rte_eth_dev *dev,
+ struct rte_mbuf *buf, int *tstamp, int *set);
int dpaa2_timesync_enable(struct rte_eth_dev *dev);
int dpaa2_timesync_disable(struct rte_eth_dev *dev);
int dpaa2_timesync_read_time(struct rte_eth_dev *dev,
diff --git a/drivers/net/dpaa2/dpaa2_ptp.c b/drivers/net/dpaa2/dpaa2_ptp.c
index c845291c68..71b811aff6 100644
--- a/drivers/net/dpaa2/dpaa2_ptp.c
+++ b/drivers/net/dpaa2/dpaa2_ptp.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2019, 2023 NXP
+ * Copyright 2019, 2023-2026 NXP
*/
#include <sys/queue.h>
@@ -22,6 +22,7 @@
#include <dpaa2_ethdev.h>
#include <dpaa2_pmd_logs.h>
+#include <dpaax_ptp.h>
struct dpaa2_dprtc_dev {
struct fsl_mc_io dprtc; /** handle to DPRTC portal object */
@@ -30,52 +31,144 @@ struct dpaa2_dprtc_dev {
};
static struct dpaa2_dprtc_dev *dprtc_dev;
-int dpaa2_timesync_enable(struct rte_eth_dev *dev __rte_unused)
+static int
+dpaa2_timesync_set_one_step(struct rte_eth_dev *dev,
+ uint16_t offset, int udp)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct fsl_mc_io *dpni = dev->process_private;
+ struct dpni_single_step_cfg cfg;
+ int err;
+
+ memset(&cfg, 0, sizeof(struct dpni_single_step_cfg));
+ cfg.en = 1;
+ cfg.ch_update = udp;
+ cfg.offset = offset;
+ cfg.peer_delay = 0;
+
+ err = dpni_set_single_step_cfg(dpni, CMD_PRI_LOW, priv->token, &cfg);
+ if (err)
+ return err;
+
+ priv->ptp_correction_offset = offset;
+
+ return 0;
+}
+
+void
+dpaa2_dev_tx_ptp_one_step_runtime(struct rte_eth_dev *dev,
+ struct rte_mbuf *buf, int *tstamp, int *set)
{
+ uint16_t ts_offset;
+ int is_udp = false, ret = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+
+ if (tstamp)
+ *tstamp = false;
+ if (set)
+ *set = false;
+
+ if (!dpaax_timesync_ptp_parse_header(buf,
+ &ts_offset, &is_udp)) {
+ DPAA2_PMD_WARN("Tx packet is not PTP frame.");
+ ret = -EINVAL;
+ } else {
+ if (ts_offset != priv->ptp_correction_offset) {
+ ret = dpaa2_timesync_set_one_step(dev, ts_offset, is_udp);
+ if (ret) {
+ DPAA2_PMD_WARN("Change one step failed(%d)", ret);
+ } else {
+ DPAA2_PMD_INFO("Change one step from offset %d to %d",
+ priv->ptp_correction_offset, ts_offset);
+ priv->ptp_correction_offset = ts_offset;
+ if (set)
+ *set = true;
+ }
+ }
+ }
+
+ if (!ret && tstamp)
+ *tstamp = true;
+}
+
+int dpaa2_timesync_enable(struct rte_eth_dev *dev)
+{
+ uint16_t default_offset;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int ret;
+
+ default_offset = RTE_ETHER_HDR_LEN +
+ offsetof(struct rte_dpaax_ptp_header, correction);
+
+ ret = dpaa2_timesync_set_one_step(dev, default_offset, false);
+ if (ret) {
+ DPAA2_PMD_ERR("%s one step timesyc set failed(%d)",
+ dev->data->name, ret);
+ } else {
+ priv->flags |= DPAA2_IEEE1588_TX_TS_FLAG;
+ }
+ priv->flags |= DPAA2_IEEE1588_RX_TS_FLAG;
+ if (priv->tx_conf_type == DPAA2_TX_NO_CONF) {
+ DPAA2_PMD_WARN("%s Unable to read TX timestamp",
+ dev->data->name);
+ }
+ if (getenv("DPAA2_IEEE1588_DEBUG_ENABLE"))
+ priv->flags |= DPAA2_IEEE1588_DEBUG_FLAG;
+
return 0;
}
-int dpaa2_timesync_disable(struct rte_eth_dev *dev __rte_unused)
+int dpaa2_timesync_disable(struct rte_eth_dev *dev)
{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+
+ priv->ptp_correction_offset = 0;
+ priv->flags &= ~DPAA2_IEEE1588_RX_TS_FLAG;
+ priv->flags &= ~DPAA2_IEEE1588_TX_TS_FLAG;
return 0;
}
int dpaa2_timesync_read_time(struct rte_eth_dev *dev,
- struct timespec *timestamp)
+ struct timespec *timestamp)
{
uint64_t ns;
int ret = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
- RTE_SET_USED(dev);
+ if (!dprtc_dev)
+ return -ENODEV;
ret = dprtc_get_time(&dprtc_dev->dprtc, CMD_PRI_LOW,
- dprtc_dev->token, &ns);
+ dprtc_dev->token, &ns);
if (ret) {
DPAA2_PMD_ERR("dprtc_get_time failed ret: %d", ret);
return ret;
}
*timestamp = rte_ns_to_timespec(ns);
+ dpaa2_timestamp_debug(priv, __func__, ns);
return 0;
}
int dpaa2_timesync_write_time(struct rte_eth_dev *dev,
- const struct timespec *ts)
+ const struct timespec *ts)
{
uint64_t ns;
int ret = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
- RTE_SET_USED(dev);
+ if (!dprtc_dev)
+ return -ENODEV;
ns = rte_timespec_to_ns(ts);
-
ret = dprtc_set_time(&dprtc_dev->dprtc, CMD_PRI_LOW,
- dprtc_dev->token, ns);
+ dprtc_dev->token, ns);
if (ret) {
DPAA2_PMD_ERR("dprtc_set_time failed ret: %d", ret);
return ret;
}
+ dpaa2_timestamp_debug(priv, __func__, ns);
return 0;
}
@@ -84,11 +177,13 @@ int dpaa2_timesync_adjust_time(struct rte_eth_dev *dev, int64_t delta)
{
uint64_t ns;
int ret = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
- RTE_SET_USED(dev);
+ if (!dprtc_dev)
+ return -ENODEV;
ret = dprtc_get_time(&dprtc_dev->dprtc, CMD_PRI_LOW,
- dprtc_dev->token, &ns);
+ dprtc_dev->token, &ns);
if (ret) {
DPAA2_PMD_ERR("dprtc_get_time failed ret: %d", ret);
return ret;
@@ -102,64 +197,84 @@ int dpaa2_timesync_adjust_time(struct rte_eth_dev *dev, int64_t delta)
DPAA2_PMD_ERR("dprtc_set_time failed ret: %d", ret);
return ret;
}
+ dpaa2_timestamp_debug(priv, "adjust delata", delta);
+ dpaa2_timestamp_debug(priv, "adjust absolute", ns);
return 0;
}
int dpaa2_timesync_read_tx_timestamp(struct rte_eth_dev *dev,
- struct timespec *timestamp)
+ struct timespec *timestamp)
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
-
- if (priv->next_txq_to_cnf) {
- while (!priv->tx_timestamp)
- dpaa2_dev_tx_conf(priv->next_txq_to_cnf, false);
- } else {
- return -1;
+ struct dpaa2_queue *txq;
+ uint64_t ts;
+
+ if (!(priv->flags & DPAA2_IEEE1588_TX_TS_FLAG) ||
+ priv->tx_conf_type == DPAA2_TX_NO_CONF)
+ return -EINVAL;
+
+ txq = rte_atomic_load_explicit(&priv->next_txq_to_cnf,
+ rte_memory_order_relaxed);
+ while (txq && txq->ts_to_cnfd > 0) {
+ dpaa2_dev_tx_conf(txq, false);
+ txq = rte_atomic_load_explicit(&priv->next_txq_to_cnf,
+ rte_memory_order_relaxed);
}
- *timestamp = rte_ns_to_timespec(priv->tx_timestamp);
+
+ ts = rte_atomic_load_explicit(&priv->tx_timestamp,
+ rte_memory_order_relaxed);
+ *timestamp = rte_ns_to_timespec(ts);
+ dpaa2_timestamp_debug(priv, __func__, ts);
return 0;
}
int dpaa2_timesync_read_rx_timestamp(struct rte_eth_dev *dev,
- struct timespec *timestamp,
- uint32_t flags __rte_unused)
+ struct timespec *timestamp, uint32_t flags __rte_unused)
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- *timestamp = rte_ns_to_timespec(priv->rx_timestamp);
+ uint64_t ts;
+
+ if (!(priv->flags & DPAA2_IEEE1588_RX_TS_FLAG))
+ return -EINVAL;
+
+ ts = rte_atomic_load_explicit(&priv->rx_timestamp,
+ rte_memory_order_relaxed);
+ *timestamp = rte_ns_to_timespec(ts);
+ dpaa2_timestamp_debug(priv, __func__, ts);
+
return 0;
}
-#if defined(RTE_LIBRTE_IEEE1588)
static int
dpaa2_create_dprtc_device(int vdev_fd __rte_unused,
struct vfio_device_info *obj_info __rte_unused,
struct rte_dpaa2_device *obj)
{
struct dprtc_attr attr;
- int ret, dprtc_id = obj->object_id;
+ int ret = 0, dprtc_id = obj->object_id;
PMD_INIT_FUNC_TRACE();
/* Allocate DPAA2 dprtc handle */
- dprtc_dev = rte_malloc(NULL, sizeof(struct dpaa2_dprtc_dev), 0);
+ dprtc_dev = rte_zmalloc(NULL, sizeof(struct dpaa2_dprtc_dev), 0);
if (!dprtc_dev) {
DPAA2_PMD_ERR("Memory allocation failed for DPRTC Device");
- return -1;
+ return -ENOMEM;
}
/* Open the dprtc object */
dprtc_dev->dprtc.regs = dpaa2_get_mcp_ptr(MC_PORTAL_INDEX);
ret = dprtc_open(&dprtc_dev->dprtc, CMD_PRI_LOW, dprtc_id,
- &dprtc_dev->token);
+ &dprtc_dev->token);
if (ret) {
DPAA2_PMD_ERR("Unable to open dprtc object: err(%d)", ret);
goto init_err;
}
ret = dprtc_get_attributes(&dprtc_dev->dprtc, CMD_PRI_LOW,
- dprtc_dev->token, &attr);
+ dprtc_dev->token, &attr);
if (ret) {
DPAA2_PMD_ERR("Unable to get dprtc attr: err(%d)", ret);
goto init_err;
@@ -172,7 +287,7 @@ dpaa2_create_dprtc_device(int vdev_fd __rte_unused,
init_err:
rte_free(dprtc_dev);
- return -1;
+ return ret;
}
static struct rte_dpaa2_object rte_dpaa2_dprtc_obj = {
@@ -181,4 +296,3 @@ static struct rte_dpaa2_object rte_dpaa2_dprtc_obj = {
};
RTE_PMD_REGISTER_DPAA2_OBJECT(dprtc, rte_dpaa2_dprtc_obj);
-#endif
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 4512c8cf00..eda88a7bed 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -22,6 +22,7 @@
#include <dpaa2_hw_pvt.h>
#include <dpaa2_hw_dpio.h>
#include <dpaa2_hw_mempool.h>
+#include <dpaax_ptp.h>
#include "dpaa2_pmd_logs.h"
#include "dpaa2_ethdev.h"
@@ -115,6 +116,30 @@ dpaa2_dev_rx_mbuf_sched_set(struct rte_mbuf *m,
m->hash.rss);
}
}
+
+static inline void
+dpaa2_dev_rx_read_timestamp(struct dpaa2_dev_priv *priv,
+ struct rte_mbuf *m)
+{
+ struct dpaa2_annot_hdr *annotation;
+ rte_mbuf_timestamp_t *ts;
+
+ if (!(priv->flags & DPAA2_IEEE1588_RX_TS_FLAG))
+ return;
+
+ annotation = (void *)((uint8_t *)m->buf_addr + DPAA2_FD_PTA_SIZE);
+ if (BIT_ISSET_AT_POS(annotation->word1, DPAA2_ETH_FAS_PTP)) {
+ m->ol_flags |= RTE_MBUF_F_RX_IEEE1588_PTP;
+ m->ol_flags |= RTE_MBUF_F_RX_IEEE1588_TMST;
+ }
+ ts = RTE_MBUF_DYNFIELD(m, priv->rx_ts_offset, void *);
+ *ts = annotation->word2;
+ m->ol_flags |= priv->rx_ts_flag;
+ rte_atomic_store_explicit(&priv->rx_timestamp, *ts,
+ rte_memory_order_relaxed);
+ dpaa2_timestamp_debug(priv, __func__, priv->rx_timestamp);
+}
+
static void __rte_hot
dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
struct rte_mbuf *m, const struct qbman_fd *fd,
@@ -956,6 +981,8 @@ dump_err_pkts(struct dpaa2_queue *dpaa2_q)
if (priv->psr_dynfield_offset >= 0)
dpaa2_dev_rx_parse_offset(priv, mbuf, fd);
+ dpaa2_dev_rx_read_timestamp(priv, mbuf);
+
dpaa2_dev_rx_print_parser_result(priv, fd, mbuf);
DPAA2_PMD_ERR("Err pkt on port[%d]:", eth_data->port_id);
DPAA2_PMD_ERR("FD offset: %d, FD err: %x, FAS status: %x",
@@ -1131,6 +1158,8 @@ dpaa2_dev_prefetch_rx_common(void *queue, struct rte_mbuf **bufs,
if (priv->psr_dynfield_offset >= 0)
dpaa2_dev_rx_parse_offset(priv, bufs[num_rx], fd);
+ dpaa2_dev_rx_read_timestamp(priv, bufs[num_rx]);
+
dpaa2_dev_rx_print_parser_result(priv, fd, bufs[num_rx]);
dq_storage++;
@@ -1381,6 +1410,7 @@ dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
if (priv->psr_dynfield_offset >= 0)
dpaa2_dev_rx_parse_offset(priv, bufs[num_rx], fd);
+ dpaa2_dev_rx_read_timestamp(priv, bufs[num_rx]);
dpaa2_dev_rx_print_parser_result(priv, fd, bufs[num_rx]);
dq_storage++;
@@ -1424,6 +1454,8 @@ dpaa2_dev_is_mbuf_from_spec_pool(struct rte_mempool *mp,
return true;
}
+#define swp_idx (DPAA2_PER_LCORE_ETHRX_DPIO->index)
+
uint16_t dpaa2_dev_tx_conf(void *txq, int drain)
{
/* Function receive frames for a given device and VQ*/
@@ -1441,7 +1473,10 @@ uint16_t dpaa2_dev_tx_conf(void *txq, int drain)
struct rte_mbuf *mbufs[dpaa2_dqrr_size];
struct rte_mempool *mp = NULL;
- #define swp_idx (DPAA2_PER_LCORE_ETHRX_DPIO->index)
+ struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
+ struct dpaa2_dev_priv *priv = eth_data->dev_private;
+ struct dpaa2_annot_hdr *annotation;
+ void *v_addr;
conf_again:
bulk_free = true;
@@ -1539,6 +1574,16 @@ uint16_t dpaa2_dev_tx_conf(void *txq, int drain)
if (bulk_free &&
!dpaa2_dev_is_mbuf_from_spec_pool(mp, mbufs[idx]))
bulk_free = false;
+ if (unlikely(mbufs[idx]->ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST)) {
+ v_addr = mbufs[idx]->buf_addr;
+ annotation = (void *)((size_t)v_addr + DPAA2_FD_PTA_SIZE);
+ rte_atomic_store_explicit(&priv->tx_timestamp,
+ annotation->word2, rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&priv->next_txq_to_cnf,
+ dpaa2_txq, rte_memory_order_relaxed);
+ dpaa2_q->ts_to_cnfd--;
+ dpaa2_timestamp_debug(priv, __func__, priv->tx_timestamp);
+ }
idx++;
dq_storage++;
if (idx >= dpaa2_dqrr_size)
@@ -1621,7 +1666,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
{
/* Function to transmit the frames to given device and VQ*/
uint32_t loop, retry_count, i;
- int32_t ret, tstamp[MAX_TX_RING_SLOTS];
+ int32_t ret, tstamp[MAX_TX_RING_SLOTS], ptp_set_count, ptp_set;
struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
uint32_t frames_to_send;
struct qbman_eq_desc eqdesc;
@@ -1667,6 +1712,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
tx_again:
/*Check if the queue is congested*/
retry_count = 0;
+ ptp_set_count = 0;
while (qbman_result_SCN_state(dpaa2_q->cscn)) {
retry_count++;
/* Retry for some time before giving up */
@@ -1696,6 +1742,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
}
tstamp[loop] = false;
+ if (unlikely(((*bufs)->ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST) &&
+ (priv->flags & DPAA2_IEEE1588_TX_TS_FLAG))) {
+ ptp_set = false;
+ dpaa2_dev_tx_ptp_one_step_runtime(priv->eth_dev, *bufs,
+ &tstamp[loop], &ptp_set);
+ if (ptp_set)
+ ptp_set_count++;
+ if (ptp_set_count > 1)
+ DPAA2_PMD_WARN("Multiple ptp formats in burst transmission!");
+ goto skip_fast_mbuf2fd;
+ }
ret = dpaa2_dev_tx_fast_mbuf_to_fd(eth_data,
*bufs, &fd_arr[loop], priv->tx_conf_type);
@@ -1704,6 +1761,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
continue;
}
+skip_fast_mbuf2fd:
if (unlikely((*bufs)->nb_segs > 1)) {
ret = dpaa2_dev_tx_mbuf_to_sg_fd(hw_mp, *bufs, &fd_arr[loop],
dpaa2_q, priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
@@ -1854,7 +1912,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
{
/* Function to transmit the frames to multiple queues respectively.*/
uint32_t loop, retry_count, sent = 0, i;
- int32_t ret = 0, tstamp[MAX_TX_RING_SLOTS];
+ int32_t ret = 0, tstamp[MAX_TX_RING_SLOTS], ptp_set_count, ptp_set;
struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
uint32_t frames_to_send, num_free_eq_desc = 0;
struct rte_mempool *hw_mp;
@@ -1877,6 +1935,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
swp = DPAA2_PER_LCORE_PORTAL;
tx_again:
+ ptp_set_count = 0;
frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
dpaa2_eqcr_size : nb_pkts;
@@ -1933,6 +1992,17 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
}
tstamp[loop] = false;
+ if (unlikely(((*bufs)->ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST) &&
+ (priv->flags & DPAA2_IEEE1588_TX_TS_FLAG))) {
+ ptp_set = false;
+ dpaa2_dev_tx_ptp_one_step_runtime(priv->eth_dev, *bufs,
+ &tstamp[loop], &ptp_set);
+ if (ptp_set)
+ ptp_set_count++;
+ if (ptp_set_count > 1)
+ DPAA2_PMD_WARN("Multiple ptp formats in burst transmission!");
+ goto skip_fast_mbuf2fd;
+ }
ret = dpaa2_dev_tx_fast_mbuf_to_fd(eth_data,
*bufs, &fd_arr[loop], priv->tx_conf_type);
@@ -1943,6 +2013,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
continue;
}
+skip_fast_mbuf2fd:
if (unlikely((*bufs)->nb_segs > 1)) {
ret = dpaa2_dev_tx_mbuf_to_sg_fd(hw_mp, *bufs, &fd_arr[loop],
dpaa2_q[loop], priv->tx_conf_type, &dy_conf[loop], tstamp[loop]);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 34/45] net/dpaa2: remove unused soft parser Tx code
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (32 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 33/45] net/dpaa2: ptp enhancements Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 35/45] net/dpaa2: update MC dpni QoS and flow steering API Prashant Gupta
` (10 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Remove the soft parser related code from the Tx path. The soft parser
can only be loaded before Linux boots, so this code was never reachable
at runtime.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_rxtx.c | 10 ----------
1 file changed, 10 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index eda88a7bed..5c6a77d2f8 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -169,16 +169,6 @@ dpaa2_dev_rx_parse_new(struct dpaa2_dev_priv *priv,
ext_packet_type |= RTE_PTYPE_INNER_L3_IPV4;
}
}
- if (priv->sp_protocol) {
- if (frc_parse->fafe2) {
- frc_parse->fafe2 = 0;
- ext_packet_type |= RTE_PTYPE_TUNNEL_GENEVE;
- }
- if (frc_parse->sum_l.l4.fafe3) {
- frc_parse->sum_l.l4.fafe3 = 0;
- ext_packet_type |= RTE_PTYPE_INNER_L3_IPV4;
- }
- }
switch (frc) {
case DPAA2_PKT_TYPE_IPV4_UDP:
m->packet_type = RTE_PTYPE_L2_ETHER |
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 35/45] net/dpaa2: update MC dpni QoS and flow steering API
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (33 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 34/45] net/dpaa2: remove unused soft parser Tx code Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 36/45] net/dpaa2: enhance xstat implementation Prashant Gupta
` (9 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Update the MC firmware command interface for the QoS and flow steering
(FS) tables ahead of the flow engine rework:
- Version the SET_QOS_TBL / ADD_FS_ENT commands and add
dpni_set_qos_table_v2() and dpni_add_fs_entry_legacy() wrappers.
- Support a per-entry default flow id in the QoS table configuration.
- Replace the single redirect_obj_token in dpni_fs_action_cfg with a
redir_tokens[] array plus num_tokens, and program it via the new
ADD_FS_ENT command.
- Add the dpni dump-table structures.
The existing flow code is updated to populate the new redir_tokens[]
field so behaviour is unchanged.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_flow.c | 3 +-
drivers/net/dpaa2/mc/dpni.c | 127 ++++++++++++++++++++--------
drivers/net/dpaa2/mc/fsl_dpni.h | 100 ++++++++++++++++++++--
drivers/net/dpaa2/mc/fsl_dpni_cmd.h | 30 ++++---
4 files changed, 206 insertions(+), 54 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index cde2f5a9d3..c616857e98 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -4444,7 +4444,8 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
dest_q = dest_priv->tx_vq[0];
flow->fs_action_cfg.options =
DPNI_FS_OPT_REDIRECT_TO_DPNI_TX;
- flow->fs_action_cfg.redirect_obj_token =
+ flow->fs_action_cfg.num_tokens = 1;
+ flow->fs_action_cfg.redir_tokens[0] =
dest_priv->token;
flow->fs_action_cfg.flow_id = dest_q->flow_id;
} else if (flow->action_type == RTE_FLOW_ACTION_TYPE_DROP) {
diff --git a/drivers/net/dpaa2/mc/dpni.c b/drivers/net/dpaa2/mc/dpni.c
index 17275b7195..8e4114e604 100644
--- a/drivers/net/dpaa2/mc/dpni.c
+++ b/drivers/net/dpaa2/mc/dpni.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
*
* Copyright 2013-2016 Freescale Semiconductor Inc.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*
*/
#include <fsl_mc_sys.h>
@@ -1937,32 +1937,43 @@ int dpni_get_queue_tx_confirmation_mode(struct fsl_mc_io *mc_io,
*
* Return: '0' on Success; Error code otherwise.
*/
-int dpni_set_qos_table(struct fsl_mc_io *mc_io,
- uint32_t cmd_flags,
- uint16_t token,
- const struct dpni_qos_tbl_cfg *cfg)
+static int
+_dpni_set_qos_table(struct fsl_mc_io *mc_io, uint32_t cmd_flags,
+ uint16_t token, const struct dpni_qos_tbl_cfg *cfg, uint16_t cmd_id)
{
struct dpni_cmd_set_qos_table *cmd_params;
struct mc_command cmd = { 0 };
/* prepare command */
- cmd.header = mc_encode_cmd_header(DPNI_CMDID_SET_QOS_TBL,
- cmd_flags,
- token);
- cmd_params = (struct dpni_cmd_set_qos_table *)cmd.params;
+ cmd.header = mc_encode_cmd_header(cmd_id, cmd_flags, token);
+ cmd_params = (void *)cmd.params;
cmd_params->default_tc = cfg->default_tc;
cmd_params->key_cfg_iova = cpu_to_le64(cfg->key_cfg_iova);
- dpni_set_field(cmd_params->discard_on_miss,
- ENABLE,
- cfg->discard_on_miss);
- dpni_set_field(cmd_params->discard_on_miss,
- KEEP_QOS_ENTRIES,
- cfg->keep_entries);
+ if (DPNI_CMD_VER(cmd_id) > DPNI_CMD_VERSION_2)
+ cmd_params->default_flow_id = cpu_to_le16(cfg->default_flow_id);
+ dpni_set_field(cmd_params->flags, DISCARD_ON_MISS, cfg->discard_on_miss);
+ dpni_set_field(cmd_params->flags, KEEP_QOS_ENTRIES, cfg->keep_entries);
+ if (DPNI_CMD_VER(cmd_id) > DPNI_CMD_VERSION_2)
+ dpni_set_field(cmd_params->flags, SET_DEFAULT_FLOW_ID, cfg->set_default_flow_id);
/* send command to mc*/
return mc_send_command(mc_io, &cmd);
}
+int
+dpni_set_qos_table(struct fsl_mc_io *mc_io, uint32_t cmd_flags,
+ uint16_t token, const struct dpni_qos_tbl_cfg *cfg)
+{
+ return _dpni_set_qos_table(mc_io, cmd_flags, token, cfg, DPNI_CMDID_SET_QOS_TBL);
+}
+
+int
+dpni_set_qos_table_v2(struct fsl_mc_io *mc_io, uint32_t cmd_flags,
+ uint16_t token, const struct dpni_qos_tbl_cfg *cfg)
+{
+ return _dpni_set_qos_table(mc_io, cmd_flags, token, cfg, DPNI_CMDID_SET_QOS_TBL_V2);
+}
+
/**
* dpni_add_qos_entry() - Add QoS mapping entry (to select a traffic class)
* @mc_io: Pointer to MC portal's I/O object
@@ -2086,6 +2097,7 @@ int dpni_add_fs_entry(struct fsl_mc_io *mc_io,
{
struct dpni_cmd_add_fs_entry *cmd_params;
struct mc_command cmd = { 0 };
+ int i;
/* prepare command */
cmd.header = mc_encode_cmd_header(DPNI_CMDID_ADD_FS_ENT,
@@ -2100,7 +2112,36 @@ int dpni_add_fs_entry(struct fsl_mc_io *mc_io,
cmd_params->options = cpu_to_le16(action->options);
cmd_params->flow_id = cpu_to_le16(action->flow_id);
cmd_params->flc = cpu_to_le64(action->flc);
- cmd_params->redir_token = cpu_to_le16(action->redirect_obj_token);
+ cmd_params->token_num = action->num_tokens;
+ for (i = 0; i < DPNI_FS_REDIR_MAX_NUM; i++)
+ cmd_params->redir_tokens[i] = cpu_to_le16(action->redir_tokens[i]);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpni_add_fs_entry_legacy(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags, uint16_t token,
+ uint8_t tc_id, uint16_t index,
+ const struct dpni_rule_cfg *cfg,
+ const struct dpni_fs_action_cfg *action)
+{
+ struct dpni_cmd_add_fs_entry *cmd_params;
+ struct mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPNI_CMDID_ADD_FS_ENT_LEGACY,
+ cmd_flags, token);
+ cmd_params = (struct dpni_cmd_add_fs_entry *)cmd.params;
+ cmd_params->tc_id = tc_id;
+ cmd_params->key_size = cfg->key_size;
+ cmd_params->index = cpu_to_le16(index);
+ cmd_params->key_iova = cpu_to_le64(cfg->key_iova);
+ cmd_params->mask_iova = cpu_to_le64(cfg->mask_iova);
+ cmd_params->options = cpu_to_le16(action->options);
+ cmd_params->flow_id = cpu_to_le16(action->flow_id);
+ cmd_params->flc = cpu_to_le64(action->flc);
+ cmd_params->redir_tokens[0] = cpu_to_le16(action->redir_tokens[0]);
/* send command to mc*/
return mc_send_command(mc_io, &cmd);
@@ -2169,30 +2210,16 @@ int dpni_clear_fs_entries(struct fsl_mc_io *mc_io,
return mc_send_command(mc_io, &cmd);
}
-/**
- * dpni_set_rx_tc_policing() - Set Rx traffic class policing configuration
- * @mc_io: Pointer to MC portal's I/O object
- * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
- * @token: Token of DPNI object
- * @tc_id: Traffic class selection (0-7)
- * @cfg: Traffic class policing configuration
- *
- * Return: '0' on Success; error code otherwise.
- */
-int dpni_set_rx_tc_policing(struct fsl_mc_io *mc_io,
- uint32_t cmd_flags,
- uint16_t token,
- uint8_t tc_id,
- const struct dpni_rx_tc_policing_cfg *cfg)
+static int
+_dpni_set_rx_tc_policing(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags, uint16_t token, uint8_t tc_id,
+ const struct dpni_rx_tc_policing_cfg *cfg, uint16_t cmd_id)
{
struct dpni_cmd_set_rx_tc_policing *cmd_params;
struct mc_command cmd = { 0 };
- /* prepare command */
- cmd.header = mc_encode_cmd_header(DPNI_CMDID_SET_RX_TC_POLICING,
- cmd_flags,
- token);
- cmd_params = (struct dpni_cmd_set_rx_tc_policing *)cmd.params;
+ cmd.header = mc_encode_cmd_header(cmd_id, cmd_flags, token);
+ cmd_params = (void *)cmd.params;
dpni_set_field(cmd_params->mode_color, COLOR, cfg->default_color);
dpni_set_field(cmd_params->mode_color, MODE, cfg->mode);
dpni_set_field(cmd_params->units, UNITS, cfg->units);
@@ -2207,6 +2234,34 @@ int dpni_set_rx_tc_policing(struct fsl_mc_io *mc_io,
return mc_send_command(mc_io, &cmd);
}
+/**
+ * dpni_set_rx_tc_policing() - Set Rx traffic class policing configuration
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPNI object
+ * @tc_id: Traffic class selection (0-7)
+ * @cfg: Traffic class policing configuration
+ *
+ * Return: '0' on Success; error code otherwise.
+ */
+int
+dpni_set_rx_tc_policing(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags, uint16_t token, uint8_t tc_id,
+ const struct dpni_rx_tc_policing_cfg *cfg)
+{
+ return _dpni_set_rx_tc_policing(mc_io, cmd_flags, token, tc_id, cfg,
+ DPNI_CMDID_SET_RX_TC_POLICING);
+}
+
+int
+dpni_set_rx_tc_policing_v1(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags, uint16_t token, uint8_t tc_id,
+ const struct dpni_rx_tc_policing_cfg *cfg)
+{
+ return _dpni_set_rx_tc_policing(mc_io, cmd_flags, token, tc_id, cfg,
+ DPNI_CMDID_SET_RX_TC_POLICING_V1);
+}
+
/**
* dpni_get_rx_tc_policing() - Get Rx traffic class policing configuration
* @mc_io: Pointer to MC portal's I/O object
diff --git a/drivers/net/dpaa2/mc/fsl_dpni.h b/drivers/net/dpaa2/mc/fsl_dpni.h
index d227025e01..f315bb7004 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
*
* Copyright 2013-2016 Freescale Semiconductor Inc.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*
*/
#ifndef __FSL_DPNI_H
@@ -121,6 +121,9 @@ struct fsl_mc_io;
* The stashing is enabled by default.
*/
#define DPNI_OPT_STASHING_DIS 0x002000
+
+#define DPNI_OPT_V8_HAS_REPLICATION 0x00004000
+
/*
* PFDR in PEB mode (v1 layout).
* The total number of Rx descriptors is limited to 11264 in this mode.
@@ -1554,6 +1557,8 @@ struct dpni_qos_tbl_cfg {
int discard_on_miss;
int keep_entries;
uint8_t default_tc;
+ int set_default_flow_id;
+ uint16_t default_flow_id;
};
int dpni_set_qos_table(struct fsl_mc_io *mc_io,
@@ -1561,6 +1566,11 @@ int dpni_set_qos_table(struct fsl_mc_io *mc_io,
uint16_t token,
const struct dpni_qos_tbl_cfg *cfg);
+int dpni_set_qos_table_v2(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags,
+ uint16_t token,
+ const struct dpni_qos_tbl_cfg *cfg);
+
/**
* struct dpni_rule_cfg - Rule configuration for table lookup
* @key_iova: I/O virtual address of the key (must be in DMA-able memory)
@@ -1573,6 +1583,10 @@ struct dpni_rule_cfg {
uint8_t key_size;
};
+#define DPNI_QOS_OPT_SET_TC_ONLY 0x0
+#define DPNI_QOS_OPT_SET_FLOW_ID 0x1
+#define DPNI_QOS_OPT_UPDATE_IF_EXISTS 0x2
+
int dpni_add_qos_entry(struct fsl_mc_io *mc_io,
uint32_t cmd_flags,
uint16_t token,
@@ -1633,6 +1647,22 @@ int dpni_clear_qos_table(struct fsl_mc_io *mc_io,
*/
#define DPNI_FS_OPT_REDIRECT_TO_DPNI_TX 0x10
+/**
+ * Redirect matching traffic into multiple Tx queues of other dpni objects.
+ * The frame will be transmitted directly
+ */
+#define DPNI_FS_OPT_REDIRECT_TO_MULTIPLE_DPNI_TX 0x20
+
+/**
+ * In case the FS rule already exists (key and mask), update its action.
+ * Cannot be used with the actions which redirect the frame towards other DPNIs.
+ */
+#define DPNI_FS_OPT_UPDATE_IF_EXISTS 0x40
+
+#ifndef DPNI_FS_REDIR_MAX_NUM
+#define DPNI_FS_REDIR_MAX_NUM 8
+#endif
+
/**
* struct dpni_fs_action_cfg - Action configuration for table look-up
* @flc: FLC value for traffic matching this rule. Please check the Frame
@@ -1650,12 +1680,38 @@ int dpni_clear_qos_table(struct fsl_mc_io *mc_io,
* - if DPNI_FS_OPT_DISCARD is cleared the frame will be enqueued in queue with
* index provided in flow_id parameter.
* @options: Any combination of DPNI_FS_OPT_ values.
+ * @token_num: Number of tokens supplied. For DPNI_FS_OPT_REDIRECT_TO_DPNI_RX
+ * or DPNI_FS_OPT_REDIRECT_TO_DPNI_TX, the token_num must be 1 since there is
+ * only one token which is necessary. Accepted values are in the
+ * [1-8] range in case a REDIRECT option is requested.
+ * @redir_tokens: Array of tokens that identify the object where frame is redirected
+ * when this rule is hit. This parameter is used only when one
+ * of the flags DPNI_FS_OPT_REDIRECT_TO_DPNI_RX,
+ * DPNI_FS_OPT_REDIRECT_TO_DPNI_TX or
+ * DPNI_FS_OPT_REDIRECT_TO_MULTIPLE_DPNI_TX is set. The tokens
+ * are obtained using dpni_open() API call. The objects must
+ * stay open during the operation to ensure the fact that
+ * application has access on them.
+ * If the object is destroyed of closed, the following actions
+ * will take place:
+ * - In case of DPNI_FS_OPT_REDIRECT_TO_DPNI_TX and
+ * DPNI_FS_OPT_REDIRECT_TO_DPNI_RX:
+ * + if DPNI_FS_OPT_DISCARD is set the frame will be
+ * discarded by current dpni
+ * + if DPNI_FS_OPT_DISCARD is cleared the frame will be
+ * enqueued in queue with index provided in flow_id
+ * parameter.
+ * - In case of DPNI_FS_OPT_REDIRECT_TO_MULTIPLE_DPNI_TX, the
+ * frame will be redirected to the remaining opened target
+ * DPNIs. If there are no more opened target DPNIs, the frame
+ * will be discarded.
*/
struct dpni_fs_action_cfg {
uint64_t flc;
uint16_t flow_id;
- uint16_t redirect_obj_token;
uint16_t options;
+ uint16_t num_tokens;
+ uint16_t redir_tokens[DPNI_FS_REDIR_MAX_NUM];
};
int dpni_add_fs_entry(struct fsl_mc_io *mc_io,
@@ -1666,6 +1722,14 @@ int dpni_add_fs_entry(struct fsl_mc_io *mc_io,
const struct dpni_rule_cfg *cfg,
const struct dpni_fs_action_cfg *action);
+int dpni_add_fs_entry_legacy(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags,
+ uint16_t token,
+ uint8_t tc_id,
+ uint16_t index,
+ const struct dpni_rule_cfg *cfg,
+ const struct dpni_fs_action_cfg *action);
+
int dpni_remove_fs_entry(struct fsl_mc_io *mc_io,
uint32_t cmd_flags,
uint16_t token,
@@ -1928,7 +1992,7 @@ void dpni_extract_sw_sequence_layout(struct dpni_sw_sequence_layout *layout,
* When used for queue_idx in function dpni_set_rx_dist_default_queue will signal to dpni
* to drop all unclassified frames
*/
-#define DPNI_FS_MISS_DROP ((uint16_t)-1)
+#define DPNI_FS_MISS_ACTION_DROP ((uint16_t)-1)
/**
* struct dpni_rx_dist_cfg - distribution configuration
@@ -1941,9 +2005,10 @@ void dpni_extract_sw_sequence_layout(struct dpni_sw_sequence_layout *layout,
* @enable: enable/disable the distribution.
* @tc: TC id for which distribution is set
* @fs_miss_flow_id: when packet misses all rules from flow steering table and hash is
- * disabled it will be put into this queue id; use DPNI_FS_MISS_DROP to drop
- * frames. The value of this field is used only when flow steering distribution
- * is enabled and hash distribution is disabled
+ * disabled it will be put into this queue id;
+ * use DPNI_FS_MISS_ACTION_DROP to drop frames.
+ * The value of this field is used only when flow steering
+ * distribution is enabled and hash distribution is disabled.
*/
struct dpni_rx_dist_cfg {
uint16_t dist_size;
@@ -2035,6 +2100,29 @@ enum dpni_table_type {
DPNI_VLAN_TABLE = 4,
};
+struct __rte_packed_begin dpni_dump_table_header {
+ uint16_t table_type;
+ uint16_t table_num_entries;
+ uint16_t table_max_entries;
+ uint8_t default_action;
+ uint8_t match_type;
+ uint8_t reserved[24];
+} __rte_packed_end;
+
+struct __rte_packed_begin dpni_dump_table_entry {
+ uint8_t key[DPNI_MAX_KEY_SIZE];
+ uint8_t mask[DPNI_MAX_KEY_SIZE];
+ uint8_t key_action;
+ uint16_t result[3];
+ uint16_t rule_index;
+ uint8_t reserved[19];
+} __rte_packed_end;
+
+struct __rte_packed_begin dpni_dump_table_rsp {
+ struct dpni_dump_table_header hdr;
+ struct dpni_dump_table_entry entry[];
+} __rte_packed_end;
+
int dpni_dump_table(struct fsl_mc_io *mc_io,
uint32_t cmd_flags,
uint16_t token,
diff --git a/drivers/net/dpaa2/mc/fsl_dpni_cmd.h b/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
index 6b396ca7cc..36e4aacc3a 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni_cmd.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
*
* Copyright 2013-2016 Freescale Semiconductor Inc.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*
*/
#ifndef _FSL_DPNI_CMD_H
@@ -20,6 +20,7 @@
#define DPNI_CMD_VERSION_7 7
#define DPNI_CMD_ID_OFFSET 4
+#define DPNI_CMD_VER(cmd_id) ((cmd_id) & ((1 << DPNI_CMD_ID_OFFSET) - 1))
#define DPNI_CMD(id) (((id) << DPNI_CMD_ID_OFFSET) | DPNI_CMD_BASE_VERSION)
#define DPNI_CMD_V2(id) (((id) << DPNI_CMD_ID_OFFSET) | DPNI_CMD_VERSION_2)
#define DPNI_CMD_V3(id) (((id) << DPNI_CMD_ID_OFFSET) | DPNI_CMD_VERSION_3)
@@ -77,13 +78,16 @@
#define DPNI_CMDID_SET_RX_TC_DIST DPNI_CMD_V4(0x235)
-#define DPNI_CMDID_SET_RX_TC_POLICING DPNI_CMD(0x23E)
+#define DPNI_CMDID_SET_RX_TC_POLICING_V1 DPNI_CMD(0x23E)
+#define DPNI_CMDID_SET_RX_TC_POLICING DPNI_CMD_V2(0x23E)
-#define DPNI_CMDID_SET_QOS_TBL DPNI_CMD_V2(0x240)
+#define DPNI_CMDID_SET_QOS_TBL DPNI_CMD_V3(0x240)
+#define DPNI_CMDID_SET_QOS_TBL_V2 DPNI_CMD_V2(0x240)
#define DPNI_CMDID_ADD_QOS_ENT DPNI_CMD_V2(0x241)
#define DPNI_CMDID_REMOVE_QOS_ENT DPNI_CMD(0x242)
#define DPNI_CMDID_CLR_QOS_TBL DPNI_CMD(0x243)
-#define DPNI_CMDID_ADD_FS_ENT DPNI_CMD_V2(0x244)
+#define DPNI_CMDID_ADD_FS_ENT_LEGACY DPNI_CMD_V2(0x244)
+#define DPNI_CMDID_ADD_FS_ENT DPNI_CMD_V3(0x244)
#define DPNI_CMDID_REMOVE_FS_ENT DPNI_CMD(0x245)
#define DPNI_CMDID_CLR_FS_ENT DPNI_CMD(0x246)
@@ -584,19 +588,18 @@ struct dpni_cmd_set_queue {
#define DPNI_DISCARD_ON_MISS_SIZE 1
#define DPNI_KEEP_QOS_ENTRIES_SHIFT 1
#define DPNI_KEEP_QOS_ENTRIES_SIZE 1
+#define DPNI_SET_DEFAULT_FLOW_ID_SHIFT 2
+#define DPNI_SET_DEFAULT_FLOW_ID_SIZE 1
struct dpni_cmd_set_qos_table {
- uint32_t pad;
+ uint16_t pad;
+ uint16_t default_flow_id;
uint8_t default_tc;
- /* only the LSB */
- uint8_t discard_on_miss;
+ uint8_t flags;
uint16_t pad1[21];
uint64_t key_cfg_iova;
};
-#define DPNI_QOS_OPT_SET_TC_ONLY 0x0
-#define DPNI_QOS_OPT_SET_FLOW_ID 0x1
-
struct dpni_cmd_add_qos_entry {
uint8_t flags;
uint8_t flow_id;
@@ -616,6 +619,10 @@ struct dpni_cmd_remove_qos_entry {
uint64_t mask_iova;
};
+#ifndef DPNI_FS_REDIR_MAX_NUM
+#define DPNI_FS_REDIR_MAX_NUM 8
+#endif
+
struct dpni_cmd_add_fs_entry {
uint16_t options;
uint8_t tc_id;
@@ -625,7 +632,8 @@ struct dpni_cmd_add_fs_entry {
uint64_t key_iova;
uint64_t mask_iova;
uint64_t flc;
- uint16_t redir_token;
+ uint16_t redir_tokens[DPNI_FS_REDIR_MAX_NUM];
+ uint8_t token_num;
};
struct dpni_cmd_remove_fs_entry {
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 36/45] net/dpaa2: enhance xstat implementation
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (34 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 35/45] net/dpaa2: update MC dpni QoS and flow steering API Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 37/45] net/dpaa2: rework flow engine Prashant Gupta
` (8 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang, Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
1) Adapt RTE xstat API to return proper length.
2) Alloc DMA buffer of mac xstat only once.
3) Get MAC counter via dpni only when this dpni is connected to DPMAC.
4) Get TC policer counter only when meter flow of TC is there.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 930 ++++++++++++++++++-------------
drivers/net/dpaa2/dpaa2_ethdev.h | 12 +-
drivers/net/dpaa2/mc/fsl_dpni.h | 259 +++++++--
3 files changed, 756 insertions(+), 445 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 58b4cedee3..1836226706 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -26,6 +26,7 @@
#include <dpaa2_hw_pvt.h>
#include <dpaa2_hw_mempool.h>
#include <dpaa2_hw_dpio.h>
+#include <fsl_dprc.h>
#include <mc/fsl_dpmng.h>
#include <mc/fsl_dpcon.h>
#include "dpaa2_ethdev.h"
@@ -108,116 +109,287 @@ dpaa2_dist_size_is_supported(uint16_t n)
int dpaa2_valid_dev;
struct rte_mempool *dpaa2_tx_sg_pool;
-struct rte_dpaa2_xstats_name_off {
- char name[RTE_ETH_XSTATS_NAME_SIZE];
- uint8_t page_id; /* dpni statistics page id */
- uint8_t stats_id; /* stats id in the given page */
+struct dpaa2_xstats_seq {
+ struct dpni_statistics_page_0 pg0;
+ struct dpni_statistics_page_1 pg1;
+ struct dpni_statistics_page_2 pg2;
+ struct dpni_statistics_page_3 pg3;
+ struct dpni_statistics_page_4 pg4;
+ struct dpni_statistics_page_5 pg5[MAX_TCS];
+ struct dpni_statistics_page_6 pg6;
+ struct dpni_dpmac_counters mac_cnt;
};
-static const struct rte_dpaa2_xstats_name_off dpaa2_xstats_strings[] = {
- {"ingress_multicast_frames", 0, 2},
- {"ingress_multicast_bytes", 0, 3},
- {"ingress_broadcast_frames", 0, 4},
- {"ingress_broadcast_bytes", 0, 5},
- {"egress_multicast_frames", 1, 2},
- {"egress_multicast_bytes", 1, 3},
- {"egress_broadcast_frames", 1, 4},
- {"egress_broadcast_bytes", 1, 5},
- {"ingress_filtered_frames", 2, 0},
- {"ingress_discarded_frames", 2, 1},
- {"ingress_nobuffer_discards", 2, 2},
- {"egress_discarded_frames", 2, 3},
- {"egress_confirmed_frames", 2, 4},
- {"cgr_reject_frames", 4, 0},
- {"cgr_reject_bytes", 4, 1},
- {"TC_0_policer_cnt_red", 5, 0},
- {"TC_0_policer_cnt_yellow", 5, 1},
- {"TC_0_policer_cnt_green", 5, 2},
- {"TC_0_policer_cnt_re_red", 5, 3},
- {"TC_0_policer_cnt_re_yellow", 5, 4},
- {"TC_1_policer_cnt_red", 6, 0},
- {"TC_1_policer_cnt_yellow", 6, 1},
- {"TC_1_policer_cnt_green", 6, 2},
- {"TC_1_policer_cnt_re_red", 6, 3},
- {"TC_1_policer_cnt_re_yellow", 6, 4},
- {"TC_2_policer_cnt_red", 7, 0},
- {"TC_2_policer_cnt_yellow", 7, 1},
- {"TC_2_policer_cnt_green", 7, 2},
- {"TC_2_policer_cnt_re_red", 7, 3},
- {"TC_2_policer_cnt_re_yellow", 7, 4},
- {"TC_3_policer_cnt_red", 8, 0},
- {"TC_3_policer_cnt_yellow", 8, 1},
- {"TC_3_policer_cnt_green", 8, 2},
- {"TC_3_policer_cnt_re_red", 8, 3},
- {"TC_3_policer_cnt_re_yellow", 8, 4},
- {"TC_4_policer_cnt_red", 9, 0},
- {"TC_4_policer_cnt_yellow", 9, 1},
- {"TC_4_policer_cnt_green", 9, 2},
- {"TC_4_policer_cnt_re_red", 9, 3},
- {"TC_4_policer_cnt_re_yellow", 9, 4},
- {"TC_5_policer_cnt_red", 10, 0},
- {"TC_5_policer_cnt_yellow", 10, 1},
- {"TC_5_policer_cnt_green", 10, 2},
- {"TC_5_policer_cnt_re_red", 10, 3},
- {"TC_5_policer_cnt_re_yellow", 10, 4},
- {"TC_6_policer_cnt_red", 11, 0},
- {"TC_6_policer_cnt_yellow", 11, 1},
- {"TC_6_policer_cnt_green", 11, 2},
- {"TC_6_policer_cnt_re_red", 11, 3},
- {"TC_6_policer_cnt_re_yellow", 11, 4},
- {"TC_7_policer_cnt_red", 12, 0},
- {"TC_7_policer_cnt_yellow", 12, 1},
- {"TC_7_policer_cnt_green", 12, 2},
- {"TC_7_policer_cnt_re_red", 12, 3},
- {"TC_7_policer_cnt_re_yellow", 12, 4},
- {"mac_rx_64 bytes", 0, 0},
- {"mac_rx_65-127 bytes", 0, 0},
- {"mac_rx_128-255 bytes", 0, 0},
- {"mac_rx_256-511 bytes", 0, 0},
- {"mac_rx_512-1023 bytes", 0, 0},
- {"mac_rx_1024-1518 bytes", 0, 0},
- {"mac_rx_1519-max bytes", 0, 0},
- {"mac_rx_frags", 0, 0},
- {"mac_rx_jabber", 0, 0},
- {"mac_rx_frame discards", 0, 0},
- {"mac_rx_align errors", 0, 0},
- {"mac_tx_undersized", 0, 0},
- {"mac_rx_oversized", 0, 0},
- {"mac_rx_pause", 0, 0},
- {"mac_tx_b-pause", 0, 0},
- {"mac_rx_bytes", 0, 0},
- {"mac_rx_m-cast", 0, 0},
- {"mac_rx_b-cast", 0, 0},
- {"mac_rx_all frames", 0, 0},
- {"mac_rx_u-cast", 0, 0},
- {"mac_rx_frame errors", 0, 0},
- {"mac_tx_bytes", 0, 0},
- {"mac_tx_m-cast", 0, 0},
- {"mac_tx_b-cast", 0, 0},
- {"mac_tx_u-cast", 0, 0},
- {"mac_tx_frame errors", 0, 0},
- {"mac_rx_frames ok", 0, 0},
- {"mac_tx_frames ok", 0, 0},
- {"mac_tx_64 bytes", 0, 0},
- {"mac_tx_65-127 bytes", 0, 0},
- {"mac_tx_128-255 bytes", 0, 0},
- {"mac_tx_256-511 bytes", 0, 0},
- {"mac_tx_512-1023 bytes", 0, 0},
- {"mac_tx_1024-1518 bytes", 0, 0},
- {"mac_tx_1519-max bytes", 0, 0},
- {"mac_rx_all_bytes", 0, 0},
- {"mac_rx_fcs_err", 0, 0},
- {"mac_rx_vlan_frame", 0, 0},
- {"mac_rx_undersized", 0, 0},
- {"mac_rx_control_frame", 0, 0},
- {"mac_rx_frame_discard_not_trunc", 0, 0},
- {"mac_tx_all_bytes", 0, 0},
- {"mac_tx_fcs_err", 0, 0},
- {"mac_tx_vlan_frame", 0, 0},
- {"mac_tx_all_frame", 0, 0},
- {"mac_tx_control_frame", 0, 0},
+#define DPAA2_XSTAT_MAX_NUM \
+ (sizeof(struct dpaa2_xstats_seq) / sizeof(uint64_t))
+
+#define DPAA2_DPNI_XSTAT_MAX_NUM \
+ (offsetof(struct dpaa2_xstats_seq, mac_cnt) / sizeof(uint64_t))
+
+#define DPAA2_MAC_XSTAT_MAX_NUM DPAA2_MAC_NUM_STATS
+
+static_assert((DPAA2_DPNI_XSTAT_MAX_NUM + DPAA2_MAC_XSTAT_MAX_NUM) == DPAA2_XSTAT_MAX_NUM);
+
+#define DPAA2_DPNI_STAT_SET_PAGE_PARAM(param) \
+({ \
+ int ret = false; \
+ \
+ if ((id) >= this_offset && (id) < (this_offset + pg_size)) { \
+ _stat_id = (id) - this_offset; \
+ _param = param; \
+ _size = pg_size; \
+ ret = true; \
+ } \
+ ret; \
+})
+
+#define DPAA2_DPNI_STAT_PAGE_PARAM(pid) \
+({ \
+ uint64_t this_offset = offsetof(struct dpaa2_xstats_seq, pg##pid) / sizeof(uint64_t); \
+ uint64_t pg_size = sizeof(struct dpni_statistics_page_##pid) / sizeof(uint64_t); \
+ int ret; \
+ \
+ ret = DPAA2_DPNI_STAT_SET_PAGE_PARAM(0); \
+ if (!ret) \
+ _page++; \
+ ret; \
+})
+
+#define DPAA2_DPNI_STAT_POLICER_PAGE_PARAM() \
+({ \
+ int ret = false; \
+ uint8_t tc_id; \
+ uint64_t this_offset; \
+ uint64_t pg_size = sizeof(struct dpni_statistics_page_5) / sizeof(uint64_t); \
+ \
+ for (tc_id = 0; tc_id < MAX_TCS; tc_id++) { \
+ this_offset = offsetof(struct dpaa2_xstats_seq, pg5[tc_id]) / sizeof(uint64_t); \
+ ret = DPAA2_DPNI_STAT_SET_PAGE_PARAM(tc_id); \
+ if (ret) \
+ break; \
+ } \
+ if (!ret) \
+ _page++; \
+ ret; \
+})
+
+static inline int
+dpaa2_xstats_id_parse(uint32_t id, uint8_t *page,
+ uint8_t *stat_id, uint16_t *param, uint16_t *size)
+{
+ uint8_t _page = DPNI_INGRESS_STATISTICS_PAGE_ID, _stat_id = 0;
+ uint16_t _param = 0, _size = 0;
+
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(0))
+ goto found_id;
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(1))
+ goto found_id;
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(2))
+ goto found_id;
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(3))
+ goto found_id;
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(4))
+ goto found_id;
+ if (DPAA2_DPNI_STAT_POLICER_PAGE_PARAM())
+ goto found_id;
+ if (DPAA2_DPNI_STAT_PAGE_PARAM(6))
+ goto found_id;
+
+ DPAA2_PMD_ERR("ID(%d) was not found in DPNI page statistics", id);
+ return -EINVAL;
+
+found_id:
+ if (page)
+ *page = _page;
+ if (stat_id)
+ *stat_id = _stat_id;
+ if (param)
+ *param = _param;
+ if (size)
+ *size = _size;
+
+ return 0;
+}
+
+/* The field name is used both as a designator and stringified, so it must
+ * appear twice. Only literal member names are ever passed in.
+ */
+#define DPAA2_XSTAT_STR_SET(pg, field) \
+ .pg.field = (uint64_t)RTE_STR(field)
+
+#define DPAA2_XSTAT_TC_STR_SET(pg, tc, field) \
+ .pg[tc].field = (uint64_t)("TC" RTE_STR(tc) "_" RTE_STR(field))
+
+#define MAC_STR(name) ("mac_" #name)
+#define MAC_STR_ADDR(name) ((size_t)MAC_STR(name))
+
+static const struct dpaa2_xstats_seq dpaa2_xstats_strings = {
+ DPAA2_XSTAT_STR_SET(pg0, ingress_all_frames),
+ DPAA2_XSTAT_STR_SET(pg0, ingress_all_bytes),
+ DPAA2_XSTAT_STR_SET(pg0, ingress_multicast_frames),
+ DPAA2_XSTAT_STR_SET(pg0, ingress_multicast_bytes),
+ DPAA2_XSTAT_STR_SET(pg0, ingress_broadcast_frames),
+ DPAA2_XSTAT_STR_SET(pg0, ingress_broadcast_bytes),
+
+ DPAA2_XSTAT_STR_SET(pg1, egress_all_frames),
+ DPAA2_XSTAT_STR_SET(pg1, egress_all_bytes),
+ DPAA2_XSTAT_STR_SET(pg1, egress_multicast_frames),
+ DPAA2_XSTAT_STR_SET(pg1, egress_multicast_bytes),
+ DPAA2_XSTAT_STR_SET(pg1, egress_broadcast_frames),
+ DPAA2_XSTAT_STR_SET(pg1, egress_broadcast_bytes),
+
+ DPAA2_XSTAT_STR_SET(pg2, ingress_filtered_frames),
+ DPAA2_XSTAT_STR_SET(pg2, ingress_discarded_frames),
+ DPAA2_XSTAT_STR_SET(pg2, ingress_nobuffer_discards),
+ DPAA2_XSTAT_STR_SET(pg2, egress_discarded_frames),
+ DPAA2_XSTAT_STR_SET(pg2, egress_confirmed_frames),
+
+ DPAA2_XSTAT_STR_SET(pg3, ceetm_dequeue_bytes),
+ DPAA2_XSTAT_STR_SET(pg3, ceetm_dequeue_frames),
+ DPAA2_XSTAT_STR_SET(pg3, ceetm_reject_bytes),
+ DPAA2_XSTAT_STR_SET(pg3, ceetm_reject_frames),
+
+ DPAA2_XSTAT_STR_SET(pg4, cgr_reject_frames),
+ DPAA2_XSTAT_STR_SET(pg4, cgr_reject_bytes),
+
+ DPAA2_XSTAT_TC_STR_SET(pg5, 0, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 0, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 0, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 0, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 0, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 1, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 1, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 1, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 1, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 1, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 2, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 2, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 2, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 2, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 2, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 3, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 3, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 3, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 3, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 3, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 4, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 4, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 4, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 4, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 4, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 5, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 5, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 5, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 5, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 5, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 6, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 6, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 6, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 6, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 6, policer_cnt_re_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 7, policer_cnt_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 7, policer_cnt_yellow),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 7, policer_cnt_green),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 7, policer_cnt_re_red),
+ DPAA2_XSTAT_TC_STR_SET(pg5, 7, policer_cnt_re_yellow),
+
+ DPAA2_XSTAT_STR_SET(pg6, tx_pending_frames_cnt),
+
+ .mac_cnt.rx_64_bytes = MAC_STR_ADDR(rx_64_bytes),
+ .mac_cnt.rx_65_127_bytes = MAC_STR_ADDR(rx_65_127_bytes),
+ .mac_cnt.rx_128_255_bytes = MAC_STR_ADDR(rx_128_255_bytes),
+ .mac_cnt.rx_256_511_bytes = MAC_STR_ADDR(rx_256_511_bytes),
+ .mac_cnt.rx_512_1023_bytes = MAC_STR_ADDR(rx_512_1023_bytes),
+ .mac_cnt.rx_1024_1518_bytes = MAC_STR_ADDR(rx_1024_1518_bytes),
+ .mac_cnt.rx_1519_max_bytes = MAC_STR_ADDR(rx_1519_max_bytes),
+
+ .mac_cnt.rx_fragments = MAC_STR_ADDR(rx_fragments),
+ .mac_cnt.rx_jabber = MAC_STR_ADDR(rx_jabber),
+ .mac_cnt.rx_drop_fifo = MAC_STR_ADDR(rx_drop_fifo),
+ .mac_cnt.rx_alignment_error = MAC_STR_ADDR(rx_alignment_error),
+
+ .mac_cnt.tx_undersize_good = MAC_STR_ADDR(tx_undersize_good),
+ .mac_cnt.rx_oversize_good = MAC_STR_ADDR(rx_oversize_good),
+
+ .mac_cnt.rx_pause = MAC_STR_ADDR(rx_pause),
+ .mac_cnt.tx_pause = MAC_STR_ADDR(tx_pause),
+
+ .mac_cnt.rx_good_bytes = MAC_STR_ADDR(rx_good_bytes),
+ .mac_cnt.rx_multicast = MAC_STR_ADDR(rx_multicast),
+ .mac_cnt.rx_broadcast = MAC_STR_ADDR(rx_broadcast),
+ .mac_cnt.rx_all_frames = MAC_STR_ADDR(rx_all_frames),
+ .mac_cnt.rx_unicast = MAC_STR_ADDR(rx_unicast),
+ .mac_cnt.rx_error = MAC_STR_ADDR(rx_error),
+
+ .mac_cnt.tx_good_bytes = MAC_STR_ADDR(tx_good_bytes),
+ .mac_cnt.tx_multicast = MAC_STR_ADDR(tx_multicast),
+ .mac_cnt.tx_broadcast = MAC_STR_ADDR(tx_broadcast),
+ .mac_cnt.tx_unicast = MAC_STR_ADDR(tx_unicast),
+ .mac_cnt.tx_error = MAC_STR_ADDR(tx_error),
+
+ .mac_cnt.rx_valid_frames = MAC_STR_ADDR(rx_valid_frames),
+ .mac_cnt.tx_valid_frames = MAC_STR_ADDR(tx_valid_frames),
+
+ .mac_cnt.tx_64_bytes = MAC_STR_ADDR(tx_64_bytes),
+ .mac_cnt.tx_65_127_bytes = MAC_STR_ADDR(tx_65_127_bytes),
+ .mac_cnt.tx_128_255_bytes = MAC_STR_ADDR(tx_128_255_bytes),
+ .mac_cnt.tx_256_511_bytes = MAC_STR_ADDR(tx_256_511_bytes),
+ .mac_cnt.tx_512_1023_bytes = MAC_STR_ADDR(tx_512_1023_bytes),
+ .mac_cnt.tx_1024_1518_bytes = MAC_STR_ADDR(tx_1024_1518_bytes),
+ .mac_cnt.tx_1519_max_bytes = MAC_STR_ADDR(tx_1519_max_bytes),
+
+ .mac_cnt.rx_bytes_all = MAC_STR_ADDR(rx_bytes_all),
+ .mac_cnt.rx_crc_error = MAC_STR_ADDR(rx_crc_error),
+ .mac_cnt.rx_vlan = MAC_STR_ADDR(rx_vlan),
+ .mac_cnt.rx_undersize_good = MAC_STR_ADDR(rx_undersize_good),
+ .mac_cnt.rx_ctrl_non_pause = MAC_STR_ADDR(rx_ctrl_non_pause),
+ .mac_cnt.rx_drop_full = MAC_STR_ADDR(rx_drop_full),
+
+ .mac_cnt.tx_bytes_all = MAC_STR_ADDR(tx_bytes_all),
+ .mac_cnt.tx_crc_error = MAC_STR_ADDR(tx_crc_error),
+ .mac_cnt.tx_vlan = MAC_STR_ADDR(tx_vlan),
+ .mac_cnt.tx_all_frames = MAC_STR_ADDR(tx_all_frames),
+ .mac_cnt.tx_ctrl_non_pause = MAC_STR_ADDR(tx_ctrl_non_pause),
+
+ .mac_cnt.rx_pfc_class[0] = MAC_STR_ADDR(rx_pfc_class_0),
+ .mac_cnt.rx_pfc_class[1] = MAC_STR_ADDR(rx_pfc_class_1),
+ .mac_cnt.rx_pfc_class[2] = MAC_STR_ADDR(rx_pfc_class_2),
+ .mac_cnt.rx_pfc_class[3] = MAC_STR_ADDR(rx_pfc_class_3),
+ .mac_cnt.rx_pfc_class[4] = MAC_STR_ADDR(rx_pfc_class_4),
+ .mac_cnt.rx_pfc_class[5] = MAC_STR_ADDR(rx_pfc_class_5),
+ .mac_cnt.rx_pfc_class[6] = MAC_STR_ADDR(rx_pfc_class_6),
+ .mac_cnt.rx_pfc_class[7] = MAC_STR_ADDR(rx_pfc_class_7),
+
+ .mac_cnt.tx_pfc_class[0] = MAC_STR_ADDR(tx_pfc_class_0),
+ .mac_cnt.tx_pfc_class[1] = MAC_STR_ADDR(tx_pfc_class_1),
+ .mac_cnt.tx_pfc_class[2] = MAC_STR_ADDR(tx_pfc_class_2),
+ .mac_cnt.tx_pfc_class[3] = MAC_STR_ADDR(tx_pfc_class_3),
+ .mac_cnt.tx_pfc_class[4] = MAC_STR_ADDR(tx_pfc_class_4),
+ .mac_cnt.tx_pfc_class[5] = MAC_STR_ADDR(tx_pfc_class_5),
+ .mac_cnt.tx_pfc_class[6] = MAC_STR_ADDR(tx_pfc_class_6),
+ .mac_cnt.tx_pfc_class[7] = MAC_STR_ADDR(tx_pfc_class_7)
+};
+
+#define DPAA2_DPNI_XSTAT_FIELD_ID(field) \
+ (offsetof(struct dpaa2_xstats_seq, field) / sizeof(uint64_t))
+
+static const struct rte_eth_stats dpaa2_stats_xstat_ids = {
+ .ipackets = DPAA2_DPNI_XSTAT_FIELD_ID(pg0.ingress_all_frames),
+ .opackets = DPAA2_DPNI_XSTAT_FIELD_ID(pg1.egress_all_frames),
+ .ibytes = DPAA2_DPNI_XSTAT_FIELD_ID(pg0.ingress_all_bytes),
+ .obytes = DPAA2_DPNI_XSTAT_FIELD_ID(pg1.egress_all_bytes),
+ .imissed = DPAA2_DPNI_XSTAT_FIELD_ID(pg2.ingress_filtered_frames),
+ .ierrors = DPAA2_DPNI_XSTAT_FIELD_ID(pg2.ingress_discarded_frames),
+ .oerrors = DPAA2_DPNI_XSTAT_FIELD_ID(pg2.egress_discarded_frames),
+ .rx_nombuf = DPAA2_DPNI_XSTAT_FIELD_ID(pg2.ingress_nobuffer_discards)
};
+#define DPAA2_STATS_XSTATS_NUM \
+ (sizeof(dpaa2_stats_xstat_ids) / sizeof(uint64_t))
+
+#define DPAA2_MAC_XSTATS_START_ID DPAA2_DPNI_XSTAT_MAX_NUM
+
+
static struct rte_dpaa2_driver rte_dpaa2_pmd;
static int dpaa2_dev_link_update(struct rte_eth_dev *dev,
int wait_to_complete);
@@ -1892,127 +2064,47 @@ dpaa2_dev_set_mac_addr(struct rte_eth_dev *dev,
}
static int
-dpaa2_dev_stats_get(struct rte_eth_dev *dev,
- struct rte_eth_stats *stats, struct eth_queue_stats *qstats)
+dpaa2_dev_xstat_mac_setup_mem(struct rte_eth_dev *dev)
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct fsl_mc_io *dpni = dev->process_private;
- int32_t retcode;
- uint8_t page0 = 0, page1 = 1, page2 = 2;
- union dpni_statistics value;
- int i;
- struct dpaa2_queue *dpaa2_rxq, *dpaa2_txq;
-
- memset(&value, 0, sizeof(union dpni_statistics));
-
- PMD_INIT_FUNC_TRACE();
-
- if (!dpni) {
- DPAA2_PMD_ERR("dpni is NULL");
- return -EINVAL;
- }
-
- if (!stats) {
- DPAA2_PMD_ERR("stats is NULL");
- return -EINVAL;
- }
-
- /*Get Counters from page_0*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- page0, 0, &value);
- if (retcode)
- goto err;
-
- stats->ipackets = value.page_0.ingress_all_frames;
- stats->ibytes = value.page_0.ingress_all_bytes;
-
- /*Get Counters from page_1*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- page1, 0, &value);
- if (retcode)
- goto err;
-
- stats->opackets = value.page_1.egress_all_frames;
- stats->obytes = value.page_1.egress_all_bytes;
-
- /*Get Counters from page_2*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- page2, 0, &value);
- if (retcode)
- goto err;
-
- /* Ingress drop frame count due to configured rules */
- stats->ierrors = value.page_2.ingress_filtered_frames;
- /* Ingress drop frame count due to error */
- stats->ierrors += value.page_2.ingress_discarded_frames;
-
- stats->oerrors = value.page_2.egress_discarded_frames;
- stats->imissed = value.page_2.ingress_nobuffer_discards;
-
- /* Fill in per queue stats */
- if (qstats != NULL) {
- for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
- (i < priv->nb_rx_queues || i < priv->nb_tx_queues); ++i) {
- dpaa2_rxq = priv->rx_vq[i];
- dpaa2_txq = priv->tx_vq[i];
- if (dpaa2_rxq)
- qstats->q_ipackets[i] = dpaa2_rxq->rx_pkts;
- if (dpaa2_txq)
- qstats->q_opackets[i] = dpaa2_txq->tx_pkts;
+ int ret = 0;
- /* Byte counting is not implemented */
- qstats->q_ibytes[i] = 0;
- qstats->q_obytes[i] = 0;
+ if (!priv->cnt_idx_dma_mem) {
+ priv->cnt_idx_dma_mem = rte_zmalloc(NULL,
+ DPAA2_MAC_STATS_INDEX_DMA_SIZE, RTE_CACHE_LINE_SIZE);
+ if (!priv->cnt_idx_dma_mem) {
+ ret = -ENOMEM;
+ DPAA2_PMD_ERR("Failure to allocate memory for mac index");
+ goto out;
}
- }
-
- return 0;
-
-err:
- DPAA2_PMD_ERR("Operation not completed:Error Code = %d", retcode);
- return retcode;
-};
-
-void
-dpaa2_dev_mac_setup_stats(struct rte_eth_dev *dev)
-{
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- uint32_t *cnt_idx;
- int i;
- priv->cnt_idx_dma_mem = rte_malloc(NULL, DPAA2_MAC_STATS_INDEX_DMA_SIZE,
- RTE_CACHE_LINE_SIZE);
- if (!priv->cnt_idx_dma_mem) {
- DPAA2_PMD_ERR("Failure to allocate memory for mac index");
- goto out;
+ priv->cnt_idx_iova = rte_mem_virt2iova(priv->cnt_idx_dma_mem);
+ if (priv->cnt_idx_iova == RTE_BAD_IOVA) {
+ ret = -ENOBUFS;
+ DPAA2_PMD_ERR("%s: No IOMMU map for count index dma mem(%p)",
+ __func__, priv->cnt_idx_dma_mem);
+ goto err_dma_map;
+ }
}
- priv->cnt_values_dma_mem = rte_malloc(NULL, DPAA2_MAC_STATS_VALUE_DMA_SIZE,
- RTE_CACHE_LINE_SIZE);
if (!priv->cnt_values_dma_mem) {
- DPAA2_PMD_ERR("Failure to allocate memory for mac values");
- goto err_alloc_values;
- }
-
- cnt_idx = priv->cnt_idx_dma_mem;
- for (i = 0; i < DPAA2_MAC_NUM_STATS; i++)
- *cnt_idx++ = rte_cpu_to_le_32((uint32_t)i);
-
- priv->cnt_idx_iova = rte_mem_virt2iova(priv->cnt_idx_dma_mem);
- if (priv->cnt_idx_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for count index dma mem(%p)",
- __func__, priv->cnt_idx_dma_mem);
- goto err_dma_map;
- }
-
- priv->cnt_values_iova = rte_mem_virt2iova(priv->cnt_values_dma_mem);
- if (priv->cnt_values_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for count values dma mem(%p)",
- __func__, priv->cnt_values_dma_mem);
- goto err_dma_map;
+ priv->cnt_values_dma_mem = rte_zmalloc(NULL,
+ DPAA2_MAC_STATS_VALUE_DMA_SIZE, RTE_CACHE_LINE_SIZE);
+ if (!priv->cnt_values_dma_mem) {
+ ret = -ENOMEM;
+ DPAA2_PMD_ERR("Failure to allocate memory for mac values");
+ goto err_alloc_values;
+ }
+ priv->cnt_values_iova = rte_mem_virt2iova(priv->cnt_values_dma_mem);
+ if (priv->cnt_values_iova == RTE_BAD_IOVA) {
+ ret = -ENOBUFS;
+ DPAA2_PMD_ERR("%s: No IOMMU map for count values dma mem(%p)",
+ __func__, priv->cnt_values_dma_mem);
+ goto err_dma_map;
+ }
}
- return;
+ return 0;
err_dma_map:
rte_free(priv->cnt_values_dma_mem);
@@ -2021,224 +2113,281 @@ dpaa2_dev_mac_setup_stats(struct rte_eth_dev *dev)
out:
priv->cnt_idx_dma_mem = NULL;
priv->cnt_values_dma_mem = NULL;
+
+ return ret;
}
-/*
- * dpaa2_dev_xstats_get(): Get counters of dpni and dpmac.
- * MAC (mac_*) counters are supported on MC version > 10.39.0
- * TC_x_policer_* counters are supported only when Policer is enable.
- */
static int
-dpaa2_dev_xstats_get(struct rte_eth_dev *dev,
- struct rte_eth_xstat *xstats, unsigned int n)
+dpaa2_dev_mac_xstats_avail(struct rte_eth_dev *dev)
{
- struct fsl_mc_io *dpni = (struct fsl_mc_io *)dev->process_private;
- unsigned int i = 0, j = 0, num = RTE_DIM(dpaa2_xstats_strings);
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- union dpni_statistics value[13] = {};
- struct dpni_rx_tc_policing_cfg cfg;
- uint8_t page_id, stats_id;
- uint64_t *cnt_values;
- int32_t retcode;
- int16_t tc;
-
- if (n < num)
- return num;
+ uint64_t rev_mac_support;
- if (!xstats)
- return 0;
- /* Get Counters from page_0*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 0, 0, &value[0]);
- if (retcode)
- goto err;
-
- /* Get Counters from page_1*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 1, 0, &value[1]);
- if (retcode)
- goto err;
-
- /* Get Counters from page_2*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 2, 0, &value[2]);
- if (retcode)
- goto err;
-
- for (j = 0; j < priv->max_cgs; j++) {
- if (!priv->cgid_in_use[j]) {
- /* Get Counters from page_4*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW,
- priv->token,
- 4, 0, &value[4]);
- if (retcode)
- goto err;
- break;
- }
- }
+ if (priv->ep_dev_type != DPAA2_MAC)
+ return false;
- for (tc = 0; tc < priv->num_rx_tc; tc++) {
- retcode = dpni_get_rx_tc_policing(dpni, CMD_PRI_LOW,
- priv->token, tc,
- &cfg);
- if (retcode) {
- DPAA2_PMD_ERR("Error to get the policer rule: %d", retcode);
- goto err;
- }
+ rev_mac_support = RTE_FSL_MC_REV(MC_VER_MAJOR, MC_VER_MINOR, 0);
+ if (priv->mc_rev >= rev_mac_support)
+ return true;
- /* get policer stats if policer is enabled */
- if (cfg.mode != 0) {
- /* Get Counters from page_5*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW,
- priv->token, 5, tc,
- &value[(tc + 5)]);
- if (retcode)
- goto err;
- }
- }
+ return false;
+}
- while (i < (num - DPAA2_MAC_NUM_STATS)) {
- xstats[i].id = i;
- page_id = dpaa2_xstats_strings[i].page_id;
- stats_id = dpaa2_xstats_strings[i].stats_id;
- xstats[i].value = value[page_id].raw.counter[stats_id];
- i++;
- }
+static int
+dpaa2_dev_dpni_xstats_avail(struct rte_eth_dev *dev,
+ uint8_t page_id, uint16_t param)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ uint8_t tc, i;
- if (priv->cnt_idx_dma_mem &&
- !dpni_get_mac_statistics(dpni, CMD_PRI_LOW, priv->token,
- priv->cnt_idx_iova,
- priv->cnt_values_iova,
- DPAA2_MAC_NUM_STATS)) {
- cnt_values = priv->cnt_values_dma_mem;
- while (i >= (num - DPAA2_MAC_NUM_STATS) && i < num) {
- xstats[i].id = i;
- xstats[i].value = rte_le_to_cpu_64(*cnt_values++);
- i++;
+ if (page_id == DPNI_CGR_STATISTICS_PAGE_ID) {
+ for (i = 0; i < priv->max_cgs; i++) {
+ if (priv->cgid_in_use[i])
+ return true;
}
- return i;
- }
-
- while (i >= (num - DPAA2_MAC_NUM_STATS) && i < num) {
- xstats[i].id = i;
- xstats[i].value = 0;
- i++;
+ } else if (page_id == DPNI_POLICER_STATISTICS_PAGE_ID) {
+ tc = param;
+ if (tc < MAX_TCS)
+ return true;
+ } else {
+ return true;
}
- return i;
-err:
- DPAA2_PMD_ERR("Error in obtaining extended stats (%d)", retcode);
- return retcode;
+ return false;
}
static int
-dpaa2_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
- struct rte_eth_xstat_name *xstats_names,
- unsigned int limit)
+dpaa2_dev_xstats_get_names(struct rte_eth_dev *dev,
+ struct rte_eth_xstat_name *xstats_names, uint32_t limit)
{
- unsigned int i, stat_cnt = RTE_DIM(dpaa2_xstats_strings);
+ uint16_t i, stat_cnt;
+ uint64_t xstat_str[DPAA2_XSTAT_MAX_NUM];
+
+ stat_cnt = DPAA2_DPNI_XSTAT_MAX_NUM;
+ if (dpaa2_dev_mac_xstats_avail(dev))
+ stat_cnt += DPAA2_MAC_XSTAT_MAX_NUM;
+
+ if (!limit)
+ return stat_cnt;
if (limit < stat_cnt)
+ stat_cnt = limit;
+
+ if (!xstats_names)
return stat_cnt;
- if (xstats_names != NULL)
- for (i = 0; i < stat_cnt; i++)
- strlcpy(xstats_names[i].name,
- dpaa2_xstats_strings[i].name,
- sizeof(xstats_names[i].name));
+ rte_memcpy(xstat_str, &dpaa2_xstats_strings, sizeof(xstat_str));
+
+ for (i = 0; i < stat_cnt; i++)
+ rte_strscpy(xstats_names[i].name, (char *)xstat_str[i], RTE_ETH_XSTATS_NAME_SIZE);
return stat_cnt;
}
static int
-dpaa2_xstats_get_by_id(struct rte_eth_dev *dev, const uint64_t *ids,
- uint64_t *values, unsigned int n)
+dpaa2_dev_xstats_get_by_id(struct rte_eth_dev *dev, const uint64_t *ids,
+ uint64_t *values, uint32_t n)
{
- unsigned int i, stat_cnt = RTE_DIM(dpaa2_xstats_strings);
- uint64_t values_copy[stat_cnt];
- uint8_t page_id, stats_id;
-
- if (!ids) {
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct fsl_mc_io *dpni = dev->process_private;
- int32_t retcode;
- union dpni_statistics value[5] = {};
-
- if (n < stat_cnt)
- return stat_cnt;
-
- if (!values)
- return 0;
+ uint16_t i, id, stat_cnt, mac_num = 0;
+ uint8_t page_id, stat_id;
+ uint16_t param;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct fsl_mc_io *dpni = dev->process_private;
+ bool page_fetched[DPNI_MAX_STATISTICS_PAGE_ID][DPNI_STAT_MAX_PARAM];
+ /* Fetch required MAC counters in a single MC command. */
+ int retcode;
+ uint32_t *mac_idx;
+ uint64_t *mac_val[DPAA2_MAC_XSTAT_MAX_NUM];
- /* Get Counters from page_0*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 0, 0, &value[0]);
- if (retcode)
- return 0;
+ retcode = dpaa2_dev_xstat_mac_setup_mem(dev);
+ if (retcode) {
+ DPAA2_PMD_ERR("%s: Failed(%d) to setup %s's MAC statistics!",
+ __func__, retcode, dev->data->name);
+ return retcode;
+ }
+ mac_idx = priv->cnt_idx_dma_mem;
- /* Get Counters from page_1*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 1, 0, &value[1]);
- if (retcode)
- return 0;
+ stat_cnt = DPAA2_DPNI_XSTAT_MAX_NUM;
+ if (dpaa2_dev_mac_xstats_avail(dev))
+ stat_cnt += DPAA2_MAC_XSTAT_MAX_NUM;
- /* Get Counters from page_2*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 2, 0, &value[2]);
- if (retcode)
- return 0;
+ memset(page_fetched, 0, sizeof(page_fetched));
- /* Get Counters from page_4*/
- retcode = dpni_get_statistics(dpni, CMD_PRI_LOW, priv->token,
- 4, 0, &value[4]);
+ for (i = 0; i < n; i++) {
+ if (ids && ids[i] >= stat_cnt) {
+ DPAA2_PMD_ERR("xstats id value isn't valid");
+ return -EINVAL;
+ }
+ id = ids ? ids[i] : i;
+ if (id >= DPAA2_MAC_XSTATS_START_ID) {
+ values[i] = 0;
+ if (dpaa2_dev_mac_xstats_avail(dev)) {
+ mac_idx[mac_num] = rte_cpu_to_le_32(id - DPAA2_MAC_XSTATS_START_ID);
+ mac_val[mac_num] = &values[i];
+ mac_num++;
+ }
+ continue;
+ }
+ page_id = 0;
+ stat_id = 0;
+ param = 0;
+ retcode = dpaa2_xstats_id_parse(id, &page_id, &stat_id, ¶m, NULL);
if (retcode)
- return 0;
-
- for (i = 0; i < stat_cnt; i++) {
- page_id = dpaa2_xstats_strings[i].page_id;
- stats_id = dpaa2_xstats_strings[i].stats_id;
- values[i] = value[page_id].raw.counter[stats_id];
+ return retcode;
+ if (!dpaa2_dev_dpni_xstats_avail(dev, page_id, param)) {
+ values[i] = 0;
+ continue;
}
- return stat_cnt;
+ if (page_fetched[page_id][param]) {
+ values[i] = priv->pg_xstats[page_id][param].raw.counter[stat_id];
+ continue;
+ }
+ /* Cache dpni page results: multiple xstats can share
+ * the same (page_id, param), so only fetch each pair once.
+ */
+ retcode = dpni_get_statistics(dpni, CMD_PRI_LOW,
+ priv->token, page_id, param,
+ &priv->pg_xstats[page_id][param]);
+ if (retcode) {
+ DPAA2_PMD_ERR("%s: Failed(%d) to get %s's statistcis of page%d!",
+ __func__, retcode, dev->data->name, page_id);
+ return retcode;
+ }
+ page_fetched[page_id][param] = true;
+ values[i] = priv->pg_xstats[page_id][param].raw.counter[stat_id];
}
- dpaa2_xstats_get_by_id(dev, NULL, values_copy, stat_cnt);
-
- for (i = 0; i < n; i++) {
- if (ids[i] >= stat_cnt) {
- DPAA2_PMD_ERR("xstats id value isn't valid");
- return -EINVAL;
+ if (mac_num > 0) {
+ retcode = dpni_get_mac_statistics(dpni, CMD_PRI_LOW,
+ priv->token, priv->cnt_idx_iova,
+ priv->cnt_values_iova, mac_num);
+ if (retcode) {
+ DPAA2_PMD_ERR("%s: Failed(%d) to get %s's %d MAC statistics!",
+ __func__, retcode, dev->data->name, mac_num);
+ return retcode;
}
- values[i] = values_copy[ids[i]];
+ for (i = 0; i < mac_num; i++)
+ *mac_val[i] = priv->cnt_values_dma_mem[i];
}
+
return n;
}
static int
-dpaa2_xstats_get_names_by_id(struct rte_eth_dev *dev,
- const uint64_t *ids,
- struct rte_eth_xstat_name *xstats_names,
- unsigned int limit)
+dpaa2_dev_xstats_get_names_by_id(struct rte_eth_dev *dev,
+ const uint64_t *ids, struct rte_eth_xstat_name *xstats_names,
+ uint32_t limit)
{
- unsigned int i, stat_cnt = RTE_DIM(dpaa2_xstats_strings);
- struct rte_eth_xstat_name xstats_names_copy[stat_cnt];
+ uint16_t i, stat_cnt;
+ uint64_t xstat_str[DPAA2_XSTAT_MAX_NUM];
+
+ stat_cnt = DPAA2_DPNI_XSTAT_MAX_NUM;
+ if (dpaa2_dev_mac_xstats_avail(dev))
+ stat_cnt += DPAA2_MAC_XSTAT_MAX_NUM;
if (!ids)
- return dpaa2_xstats_get_names(dev, xstats_names, limit);
+ return dpaa2_dev_xstats_get_names(dev, xstats_names, limit);
- dpaa2_xstats_get_names(dev, xstats_names_copy, limit);
+ rte_memcpy(xstat_str, &dpaa2_xstats_strings, sizeof(xstat_str));
for (i = 0; i < limit; i++) {
if (ids[i] >= stat_cnt) {
- DPAA2_PMD_ERR("xstats id value isn't valid");
- return -1;
+ DPAA2_PMD_ERR("xstats id[%d] value(%" PRIu64 ") >= max count(%d)",
+ i, ids[i], stat_cnt);
+ return -EINVAL;
}
- strcpy(xstats_names[i].name, xstats_names_copy[ids[i]].name);
+ rte_strscpy(xstats_names[i].name,
+ (char *)xstat_str[ids[i]], RTE_ETH_XSTATS_NAME_SIZE);
}
return limit;
}
+/*
+ * dpaa2_dev_xstats_get(): Get counters of dpni and dpmac.
+ * MAC (mac_*) counters are supported on MC version > 10.39.0
+ * TC_x_policer_* counters are supported only when Policer is enable.
+ */
+static int
+dpaa2_dev_xstats_get(struct rte_eth_dev *dev,
+ struct rte_eth_xstat *xstats, uint32_t n)
+{
+ int retcode;
+ uint32_t i;
+ uint64_t ids[DPAA2_XSTAT_MAX_NUM], vals[DPAA2_XSTAT_MAX_NUM];
+
+ if (!xstats || !n)
+ return 0;
+
+ if (n > DPAA2_XSTAT_MAX_NUM) {
+ DPAA2_PMD_WARN("%s: Expected xstat number(%d) > max(%zu)",
+ __func__, n, (size_t)DPAA2_XSTAT_MAX_NUM);
+ n = DPAA2_XSTAT_MAX_NUM;
+ }
+ for (i = 0; i < n; i++) {
+ ids[i] = i;
+ xstats[i].id = i;
+ xstats[i].value = 0;
+ }
+
+ retcode = dpaa2_dev_xstats_get_by_id(dev, ids, vals, n);
+ if (retcode < 0)
+ return retcode;
+
+ for (i = 0; i < (uint32_t)retcode; i++)
+ xstats[i].value = vals[i];
+
+ return retcode;
+}
+
+static int
+dpaa2_dev_stats_get(struct rte_eth_dev *dev,
+ struct rte_eth_stats *stats, struct eth_queue_stats *qstats)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int32_t retcode;
+ uint16_t i, num;
+ uint64_t ids[DPAA2_STATS_XSTATS_NUM], counts[DPAA2_STATS_XSTATS_NUM] = {0};
+ struct dpaa2_queue *dpaa2_q;
+
+ PMD_INIT_FUNC_TRACE();
+
+ rte_memcpy(ids, &dpaa2_stats_xstat_ids, sizeof(dpaa2_stats_xstat_ids));
+ retcode = dpaa2_dev_xstats_get_by_id(dev, ids, counts, DPAA2_STATS_XSTATS_NUM);
+ if (retcode != RTE_DIM(ids)) {
+ DPAA2_PMD_ERR("%s: Failed to get xstats (%d)counters by %zu IDs",
+ __func__, retcode, (size_t)DPAA2_STATS_XSTATS_NUM);
+ if (retcode >= 0)
+ retcode = -EINVAL;
+ return retcode;
+ }
+
+ rte_memcpy(stats, counts, sizeof(struct rte_eth_stats));
+ stats->imissed += stats->ierrors + stats->rx_nombuf;
+
+ if (qstats) {
+ /* Fill in per queue stats */
+ num = RTE_MIN(priv->nb_rx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS);
+ for (i = 0; i < num; i++) {
+ dpaa2_q = priv->rx_vq[i];
+ qstats->q_ipackets[i] = dpaa2_q->rx_pkts;
+
+ /* Byte counting is not implemented */
+ qstats->q_ibytes[i] = 0;
+ }
+ num = RTE_MIN(priv->nb_tx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS);
+ for (i = 0; i < num; i++) {
+ dpaa2_q = priv->tx_vq[i];
+ qstats->q_opackets[i] = dpaa2_q->tx_pkts;
+
+ /* Byte counting is not implemented */
+ qstats->q_obytes[i] = 0;
+ }
+ }
+
+ return 0;
+}
+
static int
dpaa2_dev_stats_reset(struct rte_eth_dev *dev)
{
@@ -2255,7 +2404,7 @@ dpaa2_dev_stats_reset(struct rte_eth_dev *dev)
return -EINVAL;
}
- retcode = dpni_reset_statistics(dpni, CMD_PRI_LOW, priv->token);
+ retcode = dpni_reset_statistics(dpni, CMD_PRI_LOW, priv->token);
if (retcode)
goto error;
@@ -3262,9 +3411,9 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.link_update = dpaa2_dev_link_update,
.stats_get = dpaa2_dev_stats_get,
.xstats_get = dpaa2_dev_xstats_get,
- .xstats_get_by_id = dpaa2_xstats_get_by_id,
- .xstats_get_names_by_id = dpaa2_xstats_get_names_by_id,
- .xstats_get_names = dpaa2_xstats_get_names,
+ .xstats_get_by_id = dpaa2_dev_xstats_get_by_id,
+ .xstats_get_names_by_id = dpaa2_dev_xstats_get_names_by_id,
+ .xstats_get_names = dpaa2_dev_xstats_get_names,
.stats_reset = dpaa2_dev_stats_reset,
.xstats_reset = dpaa2_dev_stats_reset,
.fw_version_get = dpaa2_fw_version_get,
@@ -3719,17 +3868,6 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
priv->speed_capa = dpaa2_dev_get_speed_capability(eth_dev);
- /* mac_statistics supported on MC version > 10.39.0 */
- {
- struct mc_version mc_ver_info = {0};
-
- if (!mc_get_version(dpni_dev, CMD_PRI_LOW, &mc_ver_info) &&
- mc_ver_info.major >= MC_VER_MAJOR &&
- mc_ver_info.minor >= MC_VER_MINOR &&
- mc_ver_info.revision > 0)
- dpaa2_dev_mac_setup_stats(eth_dev);
- }
-
return 0;
init_err:
dpaa2_dev_close(eth_dev);
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index e7490b1e0c..523c420c51 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -203,7 +203,8 @@
(DPAA2_PKT_TYPE_VLAN_1 | DPAA2_PKT_TYPE_VLAN_2)
/* mac counters */
-#define DPAA2_MAC_NUM_STATS (DPMAC_CNT_EGR_CONTROL_FRAME + 1)
+#define DPAA2_MAC_NUM_STATS \
+ (sizeof(struct dpni_dpmac_counters) / sizeof(uint64_t))
#define DPAA2_MAC_STATS_INDEX_DMA_SIZE (DPAA2_MAC_NUM_STATS * sizeof(uint32_t))
#define DPAA2_MAC_STATS_VALUE_DMA_SIZE (DPAA2_MAC_NUM_STATS * sizeof(uint64_t))
@@ -540,6 +541,12 @@ struct dpaa2_dev_priv {
uint64_t cnt_idx_iova, cnt_values_iova;
uint64_t mc_rev;
+ /* Cache one dpni_statistics result per (page_id, param) pair to avoid
+ * redundant MC commands when multiple xstats share the same page.
+ * Put it in heap to avoid allocating multi-KB stack in runtime.
+ */
+ union dpni_statistics pg_xstats[DPNI_MAX_STATISTICS_PAGE_ID][DPNI_STAT_MAX_PARAM];
+
struct rte_mempool *tx_sg_pool;
struct dpaa2_dev_flow *curr;
@@ -703,7 +710,4 @@ dpaa2_eth_sg_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd);
int dpaa2_mtr_ops_get(struct rte_eth_dev *dev, void *ops);
int dpaa2_soft_parser_loaded(void);
-void
-dpaa2_dev_mac_setup_stats(struct rte_eth_dev *dev);
-
#endif /* _DPAA2_ETHDEV_H */
diff --git a/drivers/net/dpaa2/mc/fsl_dpni.h b/drivers/net/dpaa2/mc/fsl_dpni.h
index f315bb7004..bc895234f1 100644
--- a/drivers/net/dpaa2/mc/fsl_dpni.h
+++ b/drivers/net/dpaa2/mc/fsl_dpni.h
@@ -688,50 +688,77 @@ int dpni_get_tx_data_offset(struct fsl_mc_io *mc_io,
* @page_6.tx_pending_frames_cnt: total number of frames pending in Tx queues
* @raw: raw statistics structure, used to index counters
*/
+enum dpni_statistics_page_id {
+ DPNI_INGRESS_STATISTICS_PAGE_ID = 0,
+ DPNI_EGRESS_STATISTICS_PAGE_ID = 1,
+ DPNI_ERR_STATISTICS_PAGE_ID = 2,
+ DPNI_CEETM_STATISTICS_PAGE_ID = 3,
+ DPNI_CGR_STATISTICS_PAGE_ID = 4,
+ DPNI_POLICER_STATISTICS_PAGE_ID = 5,
+ DPNI_TX_PENDING_STATISTICS_PAGE_ID = 6,
+ DPNI_MAX_STATISTICS_PAGE_ID
+};
+
+struct dpni_statistics_page_0 {
+ uint64_t ingress_all_frames;
+ uint64_t ingress_all_bytes;
+ uint64_t ingress_multicast_frames;
+ uint64_t ingress_multicast_bytes;
+ uint64_t ingress_broadcast_frames;
+ uint64_t ingress_broadcast_bytes;
+};
+
+struct dpni_statistics_page_1 {
+ uint64_t egress_all_frames;
+ uint64_t egress_all_bytes;
+ uint64_t egress_multicast_frames;
+ uint64_t egress_multicast_bytes;
+ uint64_t egress_broadcast_frames;
+ uint64_t egress_broadcast_bytes;
+};
+
+struct dpni_statistics_page_2 {
+ uint64_t ingress_filtered_frames;
+ uint64_t ingress_discarded_frames;
+ uint64_t ingress_nobuffer_discards;
+ uint64_t egress_discarded_frames;
+ uint64_t egress_confirmed_frames;
+};
+
+struct dpni_statistics_page_3 {
+ uint64_t ceetm_dequeue_bytes;
+ uint64_t ceetm_dequeue_frames;
+ uint64_t ceetm_reject_bytes;
+ uint64_t ceetm_reject_frames;
+};
+
+struct dpni_statistics_page_4 {
+ uint64_t cgr_reject_frames;
+ uint64_t cgr_reject_bytes;
+};
+
+struct dpni_statistics_page_5 {
+ uint64_t policer_cnt_red;
+ uint64_t policer_cnt_yellow;
+ uint64_t policer_cnt_green;
+ uint64_t policer_cnt_re_red;
+ uint64_t policer_cnt_re_yellow;
+};
+
+struct dpni_statistics_page_6 {
+ uint64_t tx_pending_frames_cnt;
+};
+
+#define DPNI_STAT_MAX_PARAM DPNI_MAX_TC
+
union dpni_statistics {
- struct {
- uint64_t ingress_all_frames;
- uint64_t ingress_all_bytes;
- uint64_t ingress_multicast_frames;
- uint64_t ingress_multicast_bytes;
- uint64_t ingress_broadcast_frames;
- uint64_t ingress_broadcast_bytes;
- } page_0;
- struct {
- uint64_t egress_all_frames;
- uint64_t egress_all_bytes;
- uint64_t egress_multicast_frames;
- uint64_t egress_multicast_bytes;
- uint64_t egress_broadcast_frames;
- uint64_t egress_broadcast_bytes;
- } page_1;
- struct {
- uint64_t ingress_filtered_frames;
- uint64_t ingress_discarded_frames;
- uint64_t ingress_nobuffer_discards;
- uint64_t egress_discarded_frames;
- uint64_t egress_confirmed_frames;
- } page_2;
- struct {
- uint64_t ceetm_dequeue_bytes;
- uint64_t ceetm_dequeue_frames;
- uint64_t ceetm_reject_bytes;
- uint64_t ceetm_reject_frames;
- } page_3;
- struct {
- uint64_t cgr_reject_frames;
- uint64_t cgr_reject_bytes;
- } page_4;
- struct {
- uint64_t policer_cnt_red;
- uint64_t policer_cnt_yellow;
- uint64_t policer_cnt_green;
- uint64_t policer_cnt_re_red;
- uint64_t policer_cnt_re_yellow;
- } page_5;
- struct {
- uint64_t tx_pending_frames_cnt;
- } page_6;
+ struct dpni_statistics_page_0 page_0;
+ struct dpni_statistics_page_1 page_1;
+ struct dpni_statistics_page_2 page_2;
+ struct dpni_statistics_page_3 page_3;
+ struct dpni_statistics_page_4 page_4;
+ struct dpni_statistics_page_5 page_5;
+ struct dpni_statistics_page_6 page_6;
struct {
uint64_t counter[DPNI_STATISTICS_CNT];
} raw;
@@ -2147,9 +2174,151 @@ int dpni_set_sp_profile(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t to
int dpni_sp_enable(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token,
uint8_t type, uint8_t en);
-int dpni_get_mac_statistics(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token,
- uint64_t iova_cnt, uint64_t iova_values, uint32_t num_cnt);
+struct dpni_dpmac_counters {
+ /** 0x00: Counts received 64-bytes frames, good or bad */
+ uint64_t rx_64_bytes;
+
+ /** 0x01: Counts received 65- to 127-bytes frames, good or bad */
+ uint64_t rx_65_127_bytes;
+
+ /** 0x02: Counts received 128- to 255-bytes frames, good or bad */
+ uint64_t rx_128_255_bytes;
+
+ /** 0x03: Counts received 256- to 511-bytes frames, good or bad */
+ uint64_t rx_256_511_bytes;
+
+ /** 0x04: Counts received 512- to 1023-bytes frames, good or bad */
+ uint64_t rx_512_1023_bytes;
+
+ /** 0x05: Counts received 1024- to 1518-bytes frames, good or bad */
+ uint64_t rx_1024_1518_bytes;
+
+ /** 0x06: Counts received frames >=1519 bytes */
+ uint64_t rx_1519_max_bytes;
+
+ /** 0x07: <64B frames with bad CRC */
+ uint64_t rx_fragments;
+
+ /** 0x08: >max length frames with bad FCS */
+ uint64_t rx_jabber;
+
+ /** 0x09: FIFO overflow drops (including truncated frames) */
+ uint64_t rx_drop_fifo;
+
+ /** 0x0A: Alignment error frames */
+ uint64_t rx_alignment_error;
+
+ /** 0x0B: TX <64B frames with good CRC */
+ uint64_t tx_undersize_good;
+ /** 0x0C: >max length frames with good FCS */
+ uint64_t rx_oversize_good;
+
+ /** 0x0D: Received pause frames (including PFC) */
+ uint64_t rx_pause;
+
+ /** 0x0E: Transmitted pause frames (including PFC) */
+ uint64_t tx_pause;
+
+ /** 0x0F: Received bytes (valid frames, exclude preamble) */
+ uint64_t rx_good_bytes;
+
+ /** 0x10: Received multicast frames */
+ uint64_t rx_multicast;
+
+ /** 0x11: Received broadcast frames */
+ uint64_t rx_broadcast;
+
+ /** 0x12: Received frames (good + bad) */
+ uint64_t rx_all_frames;
+
+ /** 0x13: Received unicast frames */
+ uint64_t rx_unicast;
+
+ /** 0x14: Received error frames (excluding fragments) */
+ uint64_t rx_error;
+
+ /** 0x15: Transmitted bytes (valid frames) */
+ uint64_t tx_good_bytes;
+
+ /** 0x16: Transmitted multicast frames */
+ uint64_t tx_multicast;
+
+ /** 0x17: Transmitted broadcast frames */
+ uint64_t tx_broadcast;
+
+ /** 0x18: Transmitted unicast frames */
+ uint64_t tx_unicast;
+
+ /** 0x19: Transmitted error frames */
+ uint64_t tx_error;
+
+ /** 0x1A: Received valid frames (no error) */
+ uint64_t rx_valid_frames;
+
+ /** 0x1B: Transmitted valid frames (no error) */
+ uint64_t tx_valid_frames;
+
+ /** 0x1C: TX 64-byte frames */
+ uint64_t tx_64_bytes;
+
+ /** 0x1D: TX 65-127-byte frames */
+ uint64_t tx_65_127_bytes;
+
+ /** 0x1E: TX 128-255-byte frames */
+ uint64_t tx_128_255_bytes;
+
+ /** 0x1F: TX 256-511-byte frames */
+ uint64_t tx_256_511_bytes;
+
+ /** 0x20: TX 512-1023-byte frames */
+ uint64_t tx_512_1023_bytes;
+
+ /** 0x21: TX 1024-1518-byte frames */
+ uint64_t tx_1024_1518_bytes;
+
+ /** 0x22: TX >=1519-byte frames */
+ uint64_t tx_1519_max_bytes;
+
+ /** 0x23: RX bytes (good + bad frames) */
+ uint64_t rx_bytes_all;
+
+ /** 0x24: RX CRC-32 error frames */
+ uint64_t rx_crc_error;
+
+ /** 0x25: Valid VLAN-tagged RX frames */
+ uint64_t rx_vlan;
+
+ /** 0x26: RX <64B frames with good CRC */
+ uint64_t rx_undersize_good;
+
+ /** 0x27: RX control frames (0x8808, non-pause) */
+ uint64_t rx_ctrl_non_pause;
+
+ /** 0x28: Fully dropped frames due to MAC client FIFO overflow */
+ uint64_t rx_drop_full;
+
+ /** 0x29: TX bytes (good + bad frames) */
+ uint64_t tx_bytes_all;
+
+ /** 0x2A: TX CRC-32 error frames */
+ uint64_t tx_crc_error;
+
+ /** 0x2B: Valid VLAN-tagged TX frames */
+ uint64_t tx_vlan;
+
+ /** 0x2C: All TX frames (good + bad) */
+ uint64_t tx_all_frames;
+
+ /** 0x2D: TX control frames (0x8808, non-pause) */
+ uint64_t tx_ctrl_non_pause;
+
+ /** 0x2E-0x35: RX PFC frames per priority (0-7) */
+ uint64_t rx_pfc_class[DPNI_MAX_TC];
+
+ /** 0x36-0x3D: TX PFC frames per priority (0-7) */
+ uint64_t tx_pfc_class[DPNI_MAX_TC];
+};
int dpni_get_mac_statistics(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token,
uint64_t iova_cnt, uint64_t iova_values, uint32_t num_cnt);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 37/45] net/dpaa2: rework flow engine
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (35 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 36/45] net/dpaa2: enhance xstat implementation Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 38/45] net/dpaa2: support Rx mempool per traffic class Prashant Gupta
` (7 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Rework the dpaa2 flow engine so that flow rules are programmed through
the updated QoS and flow steering (FS) table interface.
The distribution helpers in base/dpaa2_hw_dpni.c are removed and folded
into the ethdev and flow code, which removes a layer of indirection and
keeps the key extraction, RSS distribution and QoS/FS entry programming
together in one place.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 546 --
drivers/net/dpaa2/dpaa2_ethdev.c | 457 +-
drivers/net/dpaa2/dpaa2_ethdev.h | 478 +-
drivers/net/dpaa2/dpaa2_flow.c | 6906 ++++++++++++++----------
drivers/net/dpaa2/meson.build | 3 +-
5 files changed, 4719 insertions(+), 3671 deletions(-)
delete mode 100644 drivers/net/dpaa2/base/dpaa2_hw_dpni.c
diff --git a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
deleted file mode 100644
index fee96b3529..0000000000
--- a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
+++ /dev/null
@@ -1,546 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- *
- * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2021,2023-2025 NXP
- *
- */
-
-#include <time.h>
-#include <net/if.h>
-
-#include <eal_export.h>
-#include <rte_mbuf.h>
-#include <ethdev_driver.h>
-#include <rte_malloc.h>
-#include <rte_memcpy.h>
-#include <rte_string_fns.h>
-#include <rte_cycles.h>
-#include <rte_kvargs.h>
-#include <dev_driver.h>
-
-#include <dpaa2_pmd_logs.h>
-#include <dpaa2_hw_pvt.h>
-#include <dpaa2_hw_mempool.h>
-
-#include "../dpaa2_ethdev.h"
-
-int
-dpaa2_distset_to_dpkg_profile_cfg(
- uint64_t req_dist_set,
- struct dpkg_profile_cfg *kg_cfg);
-
-RTE_EXPORT_SYMBOL(rte_pmd_dpaa2_set_custom_hash)
-int
-rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
- uint16_t offset, uint8_t size)
-{
- struct rte_eth_dev *eth_dev = &rte_eth_devices[port_id];
- struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
- struct fsl_mc_io *dpni = priv->hw;
- struct dpni_rx_tc_dist_cfg tc_cfg;
- struct dpkg_profile_cfg kg_cfg;
- void *p_params;
- int ret, tc_index = 0;
-
- if (!rte_eth_dev_is_valid_port(port_id)) {
- DPAA2_PMD_WARN("Invalid port id %u", port_id);
- return -EINVAL;
- }
-
- if (strcmp(eth_dev->device->driver->name,
- RTE_STR(NET_DPAA2_PMD_DRIVER_NAME))) {
- DPAA2_PMD_WARN("Not a valid dpaa2 port");
- return -EINVAL;
- }
-
- p_params = rte_zmalloc(NULL,
- DIST_PARAM_IOVA_SIZE, RTE_CACHE_LINE_SIZE);
- if (!p_params) {
- DPAA2_PMD_ERR("Unable to allocate flow-dist parameters");
- return -ENOMEM;
- }
-
- memset(&kg_cfg, 0, sizeof(struct dpkg_profile_cfg));
- kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
- kg_cfg.extracts[0].extract.from_data.offset = offset;
- kg_cfg.extracts[0].extract.from_data.size = size;
- kg_cfg.extracts[0].num_of_byte_masks = 0;
- kg_cfg.num_extracts = 1;
-
- ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
- if (ret) {
- DPAA2_PMD_ERR("Unable to prepare extract parameters");
- rte_free(p_params);
- return ret;
- }
-
- memset(&tc_cfg, 0, sizeof(struct dpni_rx_tc_dist_cfg));
- tc_cfg.key_cfg_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(p_params,
- DIST_PARAM_IOVA_SIZE);
- if (tc_cfg.key_cfg_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for key cfg(%p)",
- __func__, p_params);
- rte_free(p_params);
- return -ENOBUFS;
- }
-
- tc_cfg.dist_size = eth_dev->data->nb_rx_queues;
- tc_cfg.dist_mode = DPNI_DIST_MODE_HASH;
-
- ret = dpni_set_rx_tc_dist(dpni, CMD_PRI_LOW, priv->token, tc_index,
- &tc_cfg);
- rte_free(p_params);
- if (ret) {
- DPAA2_PMD_ERR("Set RX TC dist failed(err=%d)", ret);
- return ret;
- }
-
- return 0;
-}
-
-int
-dpaa2_setup_flow_dist(struct rte_eth_dev *eth_dev,
- uint64_t req_dist_set, int tc_index)
-{
- struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
- int tc_dist_queues;
-
- /* TC distribution size is set with dist_queues or
- * (nb_rx_queues - tc_index*dist_queues) in order of TC priority index.
- */
- tc_dist_queues = eth_dev->data->nb_rx_queues -
- tc_index * priv->dist_queues;
- if (tc_dist_queues <= 0) {
- DPAA2_PMD_INFO("No distribution on TC%d", tc_index);
- return 0;
- }
-
- if (tc_dist_queues > priv->dist_queues)
- tc_dist_queues = priv->dist_queues;
-
- return dpaa2_setup_flow_dist_size(eth_dev, req_dist_set,
- tc_index, tc_dist_queues);
-}
-
-int
-dpaa2_setup_flow_dist_size(struct rte_eth_dev *eth_dev,
- uint64_t req_dist_set, int tc_index, uint16_t dist_size)
-{
- struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
- struct fsl_mc_io *dpni = eth_dev->process_private;
- struct dpni_rx_dist_cfg tc_cfg;
- struct dpkg_profile_cfg kg_cfg;
- void *p_params;
- int ret;
-
- if (dist_size == 0)
- return 0;
-
- p_params = rte_malloc(NULL,
- DIST_PARAM_IOVA_SIZE, RTE_CACHE_LINE_SIZE);
- if (!p_params) {
- DPAA2_PMD_ERR("Unable to allocate flow-dist parameters");
- return -ENOMEM;
- }
-
- memset(p_params, 0, DIST_PARAM_IOVA_SIZE);
- memset(&tc_cfg, 0, sizeof(struct dpni_rx_dist_cfg));
-
- ret = dpaa2_distset_to_dpkg_profile_cfg(req_dist_set, &kg_cfg);
- if (ret) {
- DPAA2_PMD_ERR("Given RSS Hash (%" PRIx64 ") not supported",
- req_dist_set);
- rte_free(p_params);
- return ret;
- }
-
- tc_cfg.key_cfg_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(p_params,
- DIST_PARAM_IOVA_SIZE);
- if (tc_cfg.key_cfg_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for key cfg(%p)",
- __func__, p_params);
- rte_free(p_params);
- return -ENOBUFS;
- }
-
- tc_cfg.dist_size = dist_size;
- tc_cfg.enable = true;
- tc_cfg.tc = tc_index;
-
- ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
- if (ret) {
- DPAA2_PMD_ERR("Unable to prepare extract parameters");
- rte_free(p_params);
- return ret;
- }
-
- ret = dpni_set_rx_hash_dist(dpni, CMD_PRI_LOW, priv->token, &tc_cfg);
- rte_free(p_params);
- if (ret) {
- DPAA2_PMD_ERR("RX Hash dist for failed(err=%d)", ret);
- return ret;
- }
-
- if (tc_index < MAX_TCS)
- priv->dist_size_cur[tc_index] = dist_size;
-
- return 0;
-}
-
-int
-dpaa2_remove_flow_dist(struct rte_eth_dev *eth_dev,
- uint8_t tc_index)
-{
- struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
- struct fsl_mc_io *dpni = priv->hw;
- struct dpni_rx_dist_cfg tc_cfg;
- struct dpkg_profile_cfg kg_cfg;
- void *p_params;
- int ret;
-
- p_params = rte_malloc(NULL,
- DIST_PARAM_IOVA_SIZE, RTE_CACHE_LINE_SIZE);
- if (!p_params) {
- DPAA2_PMD_ERR("Unable to allocate flow-dist parameters");
- return -ENOMEM;
- }
-
- memset(&tc_cfg, 0, sizeof(struct dpni_rx_dist_cfg));
- tc_cfg.dist_size = 0;
- tc_cfg.key_cfg_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(p_params,
- DIST_PARAM_IOVA_SIZE);
- if (tc_cfg.key_cfg_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for key cfg(%p)",
- __func__, p_params);
- rte_free(p_params);
- return -ENOBUFS;
- }
-
- tc_cfg.enable = true;
- tc_cfg.tc = tc_index;
-
- memset(p_params, 0, DIST_PARAM_IOVA_SIZE);
- kg_cfg.num_extracts = 0;
- ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
- if (ret) {
- DPAA2_PMD_ERR("Unable to prepare extract parameters");
- rte_free(p_params);
- return ret;
- }
-
- ret = dpni_set_rx_hash_dist(dpni, CMD_PRI_LOW, priv->token,
- &tc_cfg);
- rte_free(p_params);
- if (ret)
- DPAA2_PMD_ERR("RX hash dist failed(err=%d)", ret);
- return ret;
-}
-
-int
-dpaa2_distset_to_dpkg_profile_cfg(
- uint64_t req_dist_set,
- struct dpkg_profile_cfg *kg_cfg)
-{
- uint32_t loop = 0, i = 0;
- uint64_t dist_field = 0;
- int l2_configured = 0, l3_configured = 0;
- int l4_configured = 0;
- int mpls_configured = 0;
- int vlan_configured = 0;
- int esp_configured = 0;
- int ah_configured = 0;
- int pppoe_configured = 0;
-
- memset(kg_cfg, 0, sizeof(struct dpkg_profile_cfg));
- while (req_dist_set) {
- if (req_dist_set % 2 != 0) {
- dist_field = 1ULL << loop;
- switch (dist_field) {
- case RTE_ETH_RSS_L2_PAYLOAD:
- case RTE_ETH_RSS_ETH:
- if (l2_configured)
- break;
- l2_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_ETH;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_ETH_TYPE;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_PPPOE:
- if (pppoe_configured)
- break;
- pppoe_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_PPPOE;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_PPPOE_SID;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_ESP:
- if (esp_configured)
- break;
- esp_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IPSEC_ESP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IPSEC_ESP_SPI;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_AH:
- if (ah_configured)
- break;
- ah_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IPSEC_AH;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IPSEC_AH_SPI;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_C_VLAN:
- case RTE_ETH_RSS_S_VLAN:
- if (vlan_configured)
- break;
- vlan_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_VLAN;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_VLAN_TCI;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_MPLS:
-
- if (mpls_configured)
- break;
- mpls_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_MPLS;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_MPLS_MPLSL_1;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_MPLS;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_MPLS_MPLSL_2;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_MPLS;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_MPLS_MPLSL_N;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_IPV4:
- case RTE_ETH_RSS_FRAG_IPV4:
- case RTE_ETH_RSS_NONFRAG_IPV4_OTHER:
- case RTE_ETH_RSS_IPV6:
- case RTE_ETH_RSS_FRAG_IPV6:
- case RTE_ETH_RSS_NONFRAG_IPV6_OTHER:
- case RTE_ETH_RSS_IPV6_EX:
-
- if (l3_configured)
- break;
- l3_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_SRC;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_DST;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_PROTO;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- case RTE_ETH_RSS_NONFRAG_IPV4_TCP:
- case RTE_ETH_RSS_NONFRAG_IPV6_TCP:
- case RTE_ETH_RSS_NONFRAG_IPV4_UDP:
- case RTE_ETH_RSS_NONFRAG_IPV6_UDP:
- case RTE_ETH_RSS_IPV6_TCP_EX:
- case RTE_ETH_RSS_IPV6_UDP_EX:
- case RTE_ETH_RSS_NONFRAG_IPV4_SCTP:
- case RTE_ETH_RSS_NONFRAG_IPV6_SCTP:
-
- if (l4_configured)
- break;
- l4_configured = 1;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_TCP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_TCP_PORT_SRC;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_TCP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_TCP_PORT_DST;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
-
- default:
- DPAA2_PMD_WARN(
- "unsupported flow dist option 0x%" PRIx64,
- dist_field);
- return -EINVAL;
- }
- }
- req_dist_set = req_dist_set >> 1;
- loop++;
- }
- kg_cfg->num_extracts = i;
- return 0;
-}
-
-int
-dpaa2_attach_bp_list(struct dpaa2_dev_priv *priv,
- struct fsl_mc_io *dpni, void *blist)
-{
- /* Function to attach a DPNI with a buffer pool list. Buffer pool list
- * handle is passed in blist.
- */
- int32_t retcode;
- struct dpni_pools_cfg bpool_cfg;
- struct dpaa2_bp_list *bp_list = (struct dpaa2_bp_list *)blist;
- struct dpni_buffer_layout layout;
- int tot_size;
-
- /* ... rx buffer layout .
- * Check alignment for buffer layouts first
- */
-
- /* ... rx buffer layout ... */
- if (priv->flags & DPAA2_TX_PREFETCH_DYNAMIC_CONF) {
- int out_min_hdr_room, in_min_hdr_room;
- /** Additional headroom layout for IPSec with TX configure
- * dynamic enabled.
- */
- in_min_hdr_room = DPAA2_RX_MIN_FD_OFFSET +
- DPAA2_SEC_SIMPLE_FD_IB_MIN;
- out_min_hdr_room = DPAA2_DYN_TX_MIN_FD_OFFSET +
- DPAA2_SEC_SIMPLE_FD_OB_MIN;
- tot_size = RTE_MAX(in_min_hdr_room, out_min_hdr_room);
- if (tot_size < RTE_PKTMBUF_HEADROOM)
- tot_size = RTE_PKTMBUF_HEADROOM;
- } else {
- tot_size = RTE_PKTMBUF_HEADROOM;
- }
- tot_size = RTE_ALIGN_CEIL(tot_size, DPAA2_PACKET_LAYOUT_ALIGN);
-
- memset(&layout, 0, sizeof(struct dpni_buffer_layout));
- layout.options = DPNI_BUF_LAYOUT_OPT_DATA_HEAD_ROOM |
- DPNI_BUF_LAYOUT_OPT_FRAME_STATUS |
- DPNI_BUF_LAYOUT_OPT_PARSER_RESULT |
- DPNI_BUF_LAYOUT_OPT_DATA_ALIGN |
- DPNI_BUF_LAYOUT_OPT_TIMESTAMP |
- DPNI_BUF_LAYOUT_OPT_PRIVATE_DATA_SIZE;
-
- layout.pass_timestamp = true;
- layout.pass_frame_status = 1;
- layout.private_data_size = DPAA2_FD_PTA_SIZE;
- layout.pass_parser_result = 1;
- layout.data_align = DPAA2_PACKET_LAYOUT_ALIGN;
- layout.data_head_room = tot_size - DPAA2_FD_PTA_SIZE -
- DPAA2_MBUF_HW_ANNOTATION;
- retcode = dpni_set_buffer_layout(dpni, CMD_PRI_LOW, priv->token,
- DPNI_QUEUE_RX, &layout);
- if (retcode) {
- DPAA2_PMD_ERR("Error configuring buffer pool Rx layout (%d)",
- retcode);
- return retcode;
- }
-
- /*Attach buffer pool to the network interface as described by the user*/
- memset(&bpool_cfg, 0, sizeof(struct dpni_pools_cfg));
- bpool_cfg.num_dpbp = 1;
- bpool_cfg.pools[0].dpbp_id = bp_list->buf_pool.dpbp_node->dpbp_id;
- bpool_cfg.pools[0].backup_pool = 0;
- bpool_cfg.pools[0].buffer_size = RTE_ALIGN_CEIL(bp_list->buf_pool.size,
- DPAA2_PACKET_LAYOUT_ALIGN);
- bpool_cfg.pools[0].priority_mask = 0;
-
- retcode = dpni_set_pools(dpni, CMD_PRI_LOW, priv->token, &bpool_cfg);
- if (retcode != 0) {
- DPAA2_PMD_ERR("Error configuring buffer pool on interface."
- " bpid = %d error code = %d",
- bpool_cfg.pools[0].dpbp_id, retcode);
- return retcode;
- }
-
- priv->bp_list = bp_list;
- return 0;
-}
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 1836226706..a7c9bb678e 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -73,8 +73,6 @@ static uint64_t dev_tx_offloads_sup =
static uint64_t dev_tx_offloads_nodis =
RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
-bool dpaa2_print_parser_result;
-
/* Rx descriptor limit when DPNI loads PFDRs in PEB */
#define MAX_NB_RX_DESC_IN_PEB 11264
static int total_nb_rx_desc;
@@ -85,27 +83,6 @@ static int total_nb_rx_desc;
* patterns are inspected by dpaa2_dev_rss_reta_update().
*/
#define DPAA2_RETA_SIZE 64
-
-/* Values of dist_size accepted by the DPNI 'dpni_set_rx_hash_dist' MC command.
- * Source: fsl_dpni.h, "struct dpni_rx_dist_cfg::dist_size" documentation.
- * Used by dpaa2_dev_rss_reta_update() to validate user-requested patterns.
- */
-static const uint16_t dpaa2_dist_size_allowed[] = {
- 1, 2, 3, 4, 6, 7, 8, 12, 14, 16, 24, 28, 32, 48, 56, 64,
- 96, 112, 128, 192, 224, 256, 384, 448, 512, 768, 896, 1024,
-};
-
-static bool
-dpaa2_dist_size_is_supported(uint16_t n)
-{
- size_t i;
- for (i = 0; i < RTE_DIM(dpaa2_dist_size_allowed); i++) {
- if (dpaa2_dist_size_allowed[i] == n)
- return true;
- }
- return false;
-}
-
int dpaa2_valid_dev;
struct rte_mempool *dpaa2_tx_sg_pool;
@@ -396,6 +373,197 @@ static int dpaa2_dev_link_update(struct rte_eth_dev *dev,
static int dpaa2_dev_set_link_up(struct rte_eth_dev *dev);
static int dpaa2_dev_set_link_down(struct rte_eth_dev *dev);
static int dpaa2_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
+static int
+dpaa2_setup_table_miss_action(struct rte_eth_dev *eth_dev,
+ uint8_t tc_index)
+{
+ struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
+ struct rte_flow_group_attr attr;
+ struct rte_flow_action actions[2];
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+
+ memset(&attr, 0, sizeof(attr));
+ attr.ingress = 1;
+ if (tc_index < priv->num_rx_tc) {
+ tbl_profile = &priv->flow_profile.tc_profile[tc_index];
+ if (tbl_profile->default_drop) {
+ actions[0].type = RTE_FLOW_ACTION_TYPE_DROP;
+ } else if (tbl_profile->default_queue.index >= eth_dev->data->nb_rx_queues) {
+ DPAA2_PMD_DEBUG("%s-tc%d-default-rxq(%d) >= max rxq(%d), Force to drop.",
+ eth_dev->data->name, tc_index, tbl_profile->default_queue.index,
+ eth_dev->data->nb_rx_queues);
+ tbl_profile->default_drop = true;
+ actions[0].type = RTE_FLOW_ACTION_TYPE_DROP;
+ } else {
+ actions[0].type = RTE_FLOW_ACTION_TYPE_QUEUE;
+ actions[0].conf = &tbl_profile->default_queue;
+ }
+ } else {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ if (tbl_profile->default_drop) {
+ actions[0].type = RTE_FLOW_ACTION_TYPE_DROP;
+ } else if (tbl_profile->default_jump.group >= priv->num_rx_tc) {
+ DPAA2_PMD_DEBUG("%s-default-tc(%d) >= max tc(%d), Force to drop.",
+ eth_dev->data->name, tbl_profile->default_jump.group,
+ priv->num_rx_tc);
+ tbl_profile->default_drop = true;
+ actions[0].type = RTE_FLOW_ACTION_TYPE_DROP;
+ } else {
+ actions[0].type = RTE_FLOW_ACTION_TYPE_JUMP;
+ actions[0].conf = &tbl_profile->default_jump;
+ }
+ }
+ actions[1].type = RTE_FLOW_ACTION_TYPE_END;
+
+ return rte_flow_group_set_miss_actions(eth_dev->data->port_id,
+ tc_index, &attr, actions, NULL);
+}
+
+static int
+dpaa2_setup_flow_rss_dist(struct rte_eth_dev *eth_dev,
+ uint64_t req_dist_set, int tc_index)
+{
+ struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
+ int tc_dist_queues;
+ struct rte_flow_attr attr;
+ struct rte_flow_action_rss action_rss;
+ struct rte_flow_action actions[2];
+ struct rte_flow *rss_flow;
+
+ /*TC distribution size is set with dist_queues or
+ * nb_rx_queues % dist_queues in order of TC priority index.
+ * Calculating dist size for this tc_index:-
+ */
+ tc_dist_queues = eth_dev->data->nb_rx_queues -
+ tc_index * priv->dist_queues;
+ if (tc_dist_queues <= 0) {
+ DPAA2_PMD_DEBUG("No distribution on TC%d", tc_index);
+ return 0;
+ }
+
+ if (tc_dist_queues > priv->dist_queues)
+ tc_dist_queues = priv->dist_queues;
+
+ memset(&attr, 0, sizeof(attr));
+ action_rss.func = RTE_ETH_HASH_FUNCTION_DEFAULT;
+ action_rss.level = 0;
+ action_rss.types = req_dist_set;
+ action_rss.key_len = 0;
+ action_rss.key = NULL;
+ action_rss.queue = NULL;
+ action_rss.queue_num = tc_dist_queues;
+
+ actions[0].type = RTE_FLOW_ACTION_TYPE_RSS;
+ actions[0].conf = &action_rss;
+ actions[1].type = RTE_FLOW_ACTION_TYPE_END;
+ attr.group = tc_index;
+ attr.ingress = 1;
+ rss_flow = rte_flow_create(eth_dev->data->port_id, &attr, NULL, actions, NULL);
+ if (!rss_flow) {
+ DPAA2_PMD_WARN("%s: Set RSS flow dist on tc%d failed",
+ __func__, tc_index);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int
+dpaa2_setup_flow_dcb_dist(struct rte_eth_dev *eth_dev,
+ uint8_t prio, uint8_t tc)
+{
+ struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
+ struct rte_flow_attr attr;
+ struct rte_flow_action actions[2];
+ struct rte_flow_item_vlan vlan_item;
+ struct rte_flow_item_vlan vlan_mask;
+ struct rte_flow_item items[2];
+ struct rte_flow_action_jump action_jump;
+ struct rte_flow *qos_flow;
+
+ memset(&attr, 0, sizeof(struct rte_flow_attr));
+ attr.ingress = 1;
+
+ attr.group = priv->num_rx_tc;
+ attr.priority = prio;
+ memset(&vlan_item, 0, sizeof(struct rte_flow_item_vlan));
+ memset(&vlan_mask, 0, sizeof(struct rte_flow_item_vlan));
+ vlan_item.hdr.vlan_tci = rte_cpu_to_be_16(RTE_VLAN_TCI_MAKE(0, prio, 0));
+ vlan_mask.hdr.vlan_tci = rte_cpu_to_be_16(RTE_VLAN_PRI_MASK);
+ items[0].spec = &vlan_item;
+ items[0].mask = &vlan_mask;
+ items[0].type = RTE_FLOW_ITEM_TYPE_VLAN;
+ items[1].type = RTE_FLOW_ITEM_TYPE_END;
+
+ actions[0].type = RTE_FLOW_ACTION_TYPE_JUMP;
+ action_jump.group = tc;
+ actions[0].conf = &action_jump;
+ actions[1].type = RTE_FLOW_ACTION_TYPE_END;
+
+ qos_flow = rte_flow_create(eth_dev->data->port_id,
+ &attr, items, actions, NULL);
+ if (!qos_flow) {
+ DPAA2_PMD_WARN("Failed to direct vlan with prio(%d) to tc%d", prio, tc);
+ return -EIO;
+ }
+ priv->dcb_flow[prio] = qos_flow;
+
+ return 0;
+}
+
+static int
+dpaa2_remove_flow_rss_dist(struct rte_eth_dev *eth_dev,
+ uint8_t tc_index)
+{
+ struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ int ret;
+
+ tbl_profile = &priv->flow_profile.tc_profile[tc_index];
+ if (!tbl_profile->rss_flow || !tbl_profile->is_rss) {
+ DPAA2_PMD_WARN("%s'TC[%d] is not RSS distributed",
+ eth_dev->data->name, tc_index);
+ return 0;
+ }
+ ret = rte_flow_destroy(eth_dev->data->port_id, tbl_profile->rss_flow, NULL);
+ if (ret)
+ return ret;
+ tbl_profile->rss_flow = NULL;
+
+ return 0;
+}
+
+static int
+dpaa2_update_flow_rss_dist(struct rte_eth_dev *eth_dev,
+ uint64_t req_dist_set, int tc_index)
+{
+ struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct rte_flow_action_rss action_rss;
+ struct rte_flow_action actions[2];
+ int ret;
+
+ tbl_profile = &priv->flow_profile.tc_profile[tc_index];
+ if (!tbl_profile->rss_flow || !tbl_profile->is_rss) {
+ DPAA2_PMD_WARN("%s'TC[%d] is not RSS distributed",
+ eth_dev->data->name, tc_index);
+ return 0;
+ }
+ action_rss.func = RTE_ETH_HASH_FUNCTION_DEFAULT;
+ action_rss.level = 0;
+ action_rss.types = req_dist_set;
+ action_rss.key_len = 0;
+ action_rss.queue_num = tbl_profile->tc_cfg.dist_size;
+ action_rss.key = NULL;
+ action_rss.queue = NULL;
+ actions[0].type = RTE_FLOW_ACTION_TYPE_RSS;
+ actions[0].conf = &action_rss;
+ actions[1].type = RTE_FLOW_ACTION_TYPE_END;
+ ret = rte_flow_actions_update(eth_dev->data->port_id, tbl_profile->rss_flow,
+ actions, NULL);
+
+ return ret;
+}
static int
dpaa2_vlan_filter_set(struct rte_eth_dev *dev, uint16_t vlan_id, int on)
@@ -613,7 +781,7 @@ dpaa2_dev_info_get(struct rte_eth_dev *dev,
dev_rx_offloads_nodis;
dev_info->tx_offload_capa = dev_tx_offloads_sup |
dev_tx_offloads_nodis;
- dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+ dev_info->dev_capa = 0;
dev_info->max_hash_mac_addrs = 0;
dev_info->max_vfs = 0;
@@ -883,6 +1051,28 @@ dpaa2_free_rx_tx_queues(struct rte_eth_dev *dev)
}
}
+static int
+dpaa2_dev_dcb_info(struct rte_eth_dev *dev,
+ struct rte_eth_dcb_info *dcb_info)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int i;
+
+ if (!priv->nb_dcb_tcs)
+ return -ENOTSUP;
+
+ memset(dcb_info, 0, sizeof(struct rte_eth_dcb_info));
+ dcb_info->nb_tcs = priv->nb_dcb_tcs;
+ rte_memcpy(dcb_info->prio_tc, priv->prio_dcb_tc,
+ sizeof(uint8_t) * RTE_ETH_DCB_NUM_USER_PRIORITIES);
+ for (i = 0; i < dcb_info->nb_tcs; i++) {
+ dcb_info->tc_queue.tc_rxq[0][i].base = priv->dist_queues * i;
+ dcb_info->tc_queue.tc_rxq[0][i].nb_queue = priv->dist_queues;
+ }
+
+ return 0;
+}
+
static int
dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
{
@@ -895,8 +1085,12 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
int rx_l4_csum_offload = false;
int tx_l3_csum_offload = false;
int tx_l4_csum_offload = false;
- int ret, tc_index;
- uint32_t max_rx_pktlen;
+ int ret, tc_index, nb_tcs;
+ uint32_t max_rx_pktlen, base;
+ struct rte_eth_rss_conf *rss_conf;
+ struct rte_eth_dcb_rx_conf *dcb_rx_conf;
+ struct rte_dpaa2_default_action_conf *def_act_conf;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
PMD_INIT_FUNC_TRACE();
@@ -942,15 +1136,64 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
return -1;
}
- if (eth_conf->rxmode.mq_mode == RTE_ETH_MQ_RX_RSS) {
+ rss_conf = ð_conf->rx_adv_conf.rss_conf;
+ dcb_rx_conf = ð_conf->rx_adv_conf.dcb_rx_conf;
+ def_act_conf = eth_conf->rxmode.reserved_ptrs[0];
+
+ for (tc_index = 0; tc_index < priv->num_rx_tc; tc_index++) {
+ tbl_profile = &priv->flow_profile.tc_profile[tc_index];
+ base = priv->dist_queues * tc_index;
+ if (def_act_conf && tc_index < def_act_conf->max_tc) {
+ if (def_act_conf->default_flows[tc_index] >= priv->dist_queues) {
+ tbl_profile->default_drop = true;
+ } else {
+ tbl_profile->default_drop = false;
+ tbl_profile->default_queue.index = base +
+ def_act_conf->default_flows[tc_index];
+ }
+ }
+ if (priv->fs_entries) {
+ ret = dpaa2_setup_table_miss_action(dev, tc_index);
+ if (ret) {
+ DPAA2_PMD_ERR("Error(%d) to set miss action of %s-tc%d table",
+ ret, dev->data->name, tc_index);
+ }
+ }
+ }
+ if (def_act_conf) {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ if (def_act_conf->default_tc >= priv->num_rx_tc) {
+ tbl_profile->default_drop = true;
+ } else {
+ tbl_profile->default_drop = false;
+ tbl_profile->default_jump.group = def_act_conf->default_tc;
+ }
+ }
+ ret = dpaa2_setup_table_miss_action(dev, priv->num_rx_tc);
+ if (ret) {
+ DPAA2_PMD_ERR("Error(%d) to set miss action of %s-QoS table",
+ ret, dev->data->name);
+ }
+
+ if (eth_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_RSS) {
for (tc_index = 0; tc_index < priv->num_rx_tc; tc_index++) {
- ret = dpaa2_setup_flow_dist(dev,
- eth_conf->rx_adv_conf.rss_conf.rss_hf,
- tc_index);
+ ret = dpaa2_setup_flow_rss_dist(dev, rss_conf->rss_hf, tc_index);
if (ret) {
- DPAA2_PMD_ERR(
- "Unable to set flow distribution on tc%d."
- "Check queue config", tc_index);
+ DPAA2_PMD_ERR("RSS dist on tc%d err(%d)", tc_index, ret);
+ return ret;
+ }
+ }
+ }
+
+ if ((eth_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_DCB) && priv->qos_entries) {
+ nb_tcs = dcb_rx_conf->nb_tcs;
+ for (tc_index = 0; tc_index < nb_tcs; tc_index++) {
+ if (tc_index >= priv->num_rx_tc)
+ break;
+ ret = dpaa2_setup_flow_dcb_dist(dev, dcb_rx_conf->dcb_tc[tc_index],
+ tc_index);
+ if (ret) {
+ DPAA2_PMD_ERR("DCB direct to tc%d err(%d)", tc_index, ret);
return ret;
}
}
@@ -1097,7 +1340,6 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
struct dpni_taildrop taildrop;
uint8_t qopt = 0;
uint16_t flow_id;
- uint32_t bpid;
int ret;
PMD_INIT_FUNC_TRACE();
@@ -1116,18 +1358,6 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
nb_rx_desc);
}
- if (!priv->bp_list || priv->bp_list->mp != mb_pool) {
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- ret = rte_dpaa2_bpid_info_init(mb_pool);
- if (ret)
- return ret;
- }
- bpid = mempool_to_bpid(mb_pool);
- ret = dpaa2_attach_bp_list(priv, dpni,
- rte_dpaa2_bpid_info[bpid].bp_list);
- if (ret)
- return ret;
- }
dpaa2_q = priv->rx_vq[rx_queue_id];
if (dpaa2_q->fqid != DPAA2_INVALID_FQ_ID) {
DPAA2_PMD_WARN("%s: RXQ[%d] has been setup",
@@ -1797,6 +2027,7 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
struct fsl_mc_io *dpni = dev->process_private;
int i, ret;
struct rte_eth_link link;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
PMD_INIT_FUNC_TRACE();
@@ -1809,7 +2040,9 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
}
dpaa2_tm_deinit(dev);
- dpaa2_flow_clean(dev);
+ dpaa2_flow_clean(dev, MAX_TCS);
+ /** No matter dcb flows are created or not, they are destroyed in flow clean.*/
+ memset(priv->dcb_flow, 0, sizeof(void *) * RTE_ETH_DCB_NUM_USER_PRIORITIES);
/* Clean the device first */
ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
if (ret) {
@@ -1843,10 +2076,16 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
dev->process_private = NULL;
rte_free(dpni);
- for (i = 0; i < MAX_TCS; i++)
- rte_free(priv->extract.tc_extract_param[i]);
-
- rte_free(priv->extract.qos_extract_param);
+ for (i = 0; i < (MAX_TCS + 1); i++) {
+ if (i < MAX_TCS)
+ tbl_profile = &priv->flow_profile.tc_profile[i];
+ else
+ tbl_profile = &priv->flow_profile.qos_profile;
+ rte_free(tbl_profile->extract_param);
+ tbl_profile->extract_param = NULL;
+ rte_free(tbl_profile->entry_map);
+ tbl_profile->entry_map = NULL;
+ }
rte_free(priv->cnt_idx_dma_mem);
rte_free(priv->cnt_values_dma_mem);
@@ -2746,25 +2985,25 @@ dpaa2_dev_rss_hash_update(struct rte_eth_dev *dev,
if (rss_conf->rss_hf) {
for (tc_index = 0; tc_index < priv->num_rx_tc; tc_index++) {
- ret = dpaa2_setup_flow_dist(dev, rss_conf->rss_hf,
+ ret = dpaa2_update_flow_rss_dist(dev, rss_conf->rss_hf,
tc_index);
- if (ret) {
- DPAA2_PMD_ERR("Unable to set flow dist on tc%d",
- tc_index);
- return ret;
- }
+ if (ret)
+ break;
}
} else {
for (tc_index = 0; tc_index < priv->num_rx_tc; tc_index++) {
- ret = dpaa2_remove_flow_dist(dev, tc_index);
- if (ret) {
- DPAA2_PMD_ERR(
- "Unable to remove flow dist on tc%d",
- tc_index);
- return ret;
- }
+ ret = dpaa2_remove_flow_rss_dist(dev, tc_index);
+ if (ret)
+ break;
}
}
+ if (ret) {
+ DPAA2_PMD_ERR("%s: %s flow dist on tc%d err(%d)",
+ data->name, rss_conf->rss_hf ? "set" : "remove",
+ tc_index, ret);
+
+ return ret;
+ }
eth_conf->rx_adv_conf.rss_conf.rss_hf = rss_conf->rss_hf;
return 0;
}
@@ -2879,23 +3118,34 @@ dpaa2_dev_rss_reta_update(struct rte_eth_dev *dev,
}
}
- if (!dpaa2_dist_size_is_supported(n)) {
- DPAA2_PMD_ERR("dist_size %u not supported by HW. Allowed: "
- "1,2,3,4,6,7,8,12,14,16,24,28,32,48,56,64,...",
- n);
+ /* The requested distribution size cannot exceed the number of
+ * distribution queues provisioned per TC. Any HW-specific limitation
+ * on the exact value is enforced later by the rte_flow update.
+ */
+ if (n > priv->dist_queues) {
+ DPAA2_PMD_ERR("dist_size %u exceeds max distribution queues %u",
+ n, priv->dist_queues);
return -ENOTSUP;
}
- /* Apply on every configured RX TC, matching rss_hash_update behavior. */
+ /* Apply on every configured RX TC, matching rss_hash_update behavior.
+ * Program the new dist_size into each TC's RSS flow profile and push
+ * it to the live flow via rte_flow_actions_update().
+ */
for (tc_index = 0; tc_index < priv->num_rx_tc; tc_index++) {
- ret = dpaa2_setup_flow_dist_size(dev,
+ struct dpaa2_flow_tbl_profile *tbl_profile =
+ &priv->flow_profile.tc_profile[tc_index];
+
+ tbl_profile->tc_cfg.dist_size = n;
+ ret = dpaa2_update_flow_rss_dist(dev,
eth_conf->rx_adv_conf.rss_conf.rss_hf,
- tc_index, n);
+ tc_index);
if (ret) {
DPAA2_PMD_ERR("Failed to apply dist_size=%u on tc%d (err=%d)",
n, tc_index, ret);
return ret;
}
+ priv->dist_size_cur[tc_index] = n;
}
DPAA2_PMD_DEBUG("RETA updated: dist_size now %u on %u TC(s)",
@@ -3453,7 +3703,8 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.timesync_adjust_time = dpaa2_timesync_adjust_time,
.timesync_read_rx_timestamp = dpaa2_timesync_read_rx_timestamp,
.timesync_read_tx_timestamp = dpaa2_timesync_read_tx_timestamp,
- .mtr_ops_get = dpaa2_mtr_ops_get,
+ .mtr_ops_get = dpaa2_mtr_ops_get,
+ .get_dcb_info = dpaa2_dev_dcb_info
};
/* Populate the mac address from physically available (u-boot/firmware) and/or
@@ -3570,7 +3821,9 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
struct dpni_attr attr;
struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
struct dpni_buffer_layout layout;
- int ret, hw_id, i;
+ int ret, hw_id, i, entry_num;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ uint64_t iova;
PMD_INIT_FUNC_TRACE();
@@ -3668,8 +3921,6 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
priv->dist_queues = attr.num_queues;
priv->num_channels = attr.num_channels;
priv->channel_inuse = 0;
- priv->default_flow = RTE_MIN(priv->fs_entries,
- priv->dist_queues) - 1;
rte_spinlock_init(&priv->lpbk_qp_lock);
/* only if the custom CG is enabled */
@@ -3830,27 +4081,51 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->tx_pkt_burst = dpaa2_dev_tx;
/* Init fields w.r.t. classification */
- memset(&priv->extract.qos_key_extract, 0,
- sizeof(struct dpaa2_key_extract));
- priv->extract.qos_extract_param = rte_zmalloc(NULL,
- DPAA2_EXTRACT_PARAM_MAX_SIZE,
- RTE_CACHE_LINE_SIZE);
- if (!priv->extract.qos_extract_param) {
- DPAA2_PMD_ERR("Memory alloc failed");
- goto init_err;
- }
-
- for (i = 0; i < MAX_TCS; i++) {
- memset(&priv->extract.tc_key_extract[i], 0,
- sizeof(struct dpaa2_key_extract));
- priv->extract.tc_extract_param[i] = rte_zmalloc(NULL,
+ for (i = 0; i < (MAX_TCS + 1); i++) {
+ if (i < MAX_TCS) {
+ tbl_profile = &priv->flow_profile.tc_profile[i];
+ entry_num = priv->fs_entries;
+ } else {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ entry_num = priv->qos_entries;
+ }
+ memset(tbl_profile, 0, sizeof(struct dpaa2_flow_tbl_profile));
+ tbl_profile->extract_param = rte_zmalloc(NULL,
DPAA2_EXTRACT_PARAM_MAX_SIZE,
RTE_CACHE_LINE_SIZE);
- if (!priv->extract.tc_extract_param[i]) {
- DPAA2_PMD_ERR("Memory alloc failed");
+ if (!tbl_profile->extract_param)
goto init_err;
+ iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(tbl_profile->extract_param,
+ DPAA2_EXTRACT_PARAM_MAX_SIZE);
+ tbl_profile->default_drop = false;
+ if (i < MAX_TCS) {
+ tbl_profile->tc_cfg.dist_size = priv->dist_queues;
+ tbl_profile->tc_cfg.key_cfg_iova = iova;
+ tbl_profile->tc_cfg.tc = i;
+ /** First flow of TC as default flow, otherwise, may be dropped.*/
+ tbl_profile->default_queue.index = priv->dist_queues * i;
+ } else {
+ tbl_profile->qos_cfg.key_cfg_iova = iova;
+ tbl_profile->qos_cfg.keep_entries = true;
+ /** First TC as default TC, otherwise, may be dropped.*/
+ tbl_profile->default_jump.group = 0;
}
+ if (!entry_num)
+ continue;
+
+ tbl_profile->entry_map = rte_zmalloc(NULL, entry_num / 8 + 1, 0);
+ if (!tbl_profile->entry_map)
+ goto init_err;
+ }
+
+ for (i = 0; i < priv->num_rx_tc; i++) {
+ if (i >= RTE_ETH_DCB_NUM_USER_PRIORITIES)
+ break;
+ if (i >= priv->qos_entries)
+ break;
+ priv->prio_dcb_tc[i] = i;
}
+ priv->nb_dcb_tcs = i;
ret = dpni_set_max_frame_length(dpni_dev, CMD_PRI_LOW, priv->token,
RTE_ETHER_MAX_LEN - RTE_ETHER_CRC_LEN
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 523c420c51..5659abcd51 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved.
- * Copyright 2016-2025 NXP
+ * Copyright 2016-2026 NXP
*
*/
@@ -12,6 +12,7 @@
#include <rte_compat.h>
#include <rte_event_eth_rx_adapter.h>
#include <rte_pmd_dpaa2.h>
+#include <rte_string_fns.h>
#include <bus_fslmc_driver.h>
#include <dpaa2_hw_pvt.h>
@@ -24,6 +25,13 @@
#include "base/dpaa2_hw_dpni_annot.h"
#include "dpaa2_parser_decode.h"
+#define DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV RTE_FSL_MC_REV(10, 39, 106)
+#define DPAA2_FS_FLOW_HW_ACTION_UPDATE_MC_REV RTE_FSL_MC_REV(10, 39, 106)
+#define DPAA2_QOS_FLOW_HW_ACTION_UPDATE_MC_REV RTE_FSL_MC_REV(10, 39, 109)
+#define DPAA2_POLICER_SET_V2_MC_REV RTE_FSL_MC_REV(10, 39, 109)
+#define DPAA2_POLICER_NOT_RESET_COUNTER_MC_REV DPAA2_POLICER_SET_V2_MC_REV
+#define DPAA2_QOS_FLOW_TABLE_MISS_FLOW_ACTION_MC_REV RTE_FSL_MC_REV(10, 39, 109)
+#define DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV RTE_FSL_MC_REV(10, 39, 109)
#define DPAA2_MIN_RX_BUF_SIZE 512
#define DPAA2_MAX_RX_PKT_LEN 10240 /*WRIOP support*/
@@ -35,6 +43,10 @@
#define MAX_DPNI 8
#define DPAA2_MAX_CHANNELS 16
+#define DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv) \
+ container_of((((struct dpaa2_dev_priv *)priv)->eth_dev->device), \
+ struct rte_dpaa2_device, device)
+
#define DPAA2_EXTRACT_PARAM_MAX_SIZE \
RTE_ALIGN(sizeof(struct dpni_ext_set_rx_tc_dist), 256)
@@ -254,125 +266,9 @@ struct dpaa2_dyn_rx_protocol_pos {
#define DPAA2_FS_FLC_FLOW_OFFSET \
(DPAA2_FS_FLC_TC_OFFSET + DPAA2_FS_FLC_TC_BIT_SIZE)
-#define DPAA2_FAPR_SIZE \
- (sizeof(struct dpaa2_annot_hdr) - \
- offsetof(struct dpaa2_annot_hdr, word3))
-
-#define DPAA2_PR_NXTHDR_OFFSET 0
-
-#define DPAA2_FAFE_PSR_OFFSET 2
-#define DPAA2_FAFE_PSR_SIZE 2
-
-#define DPAA2_FAF_PSR_OFFSET 4
-#define DPAA2_FAF_PSR_SIZE 12
-
-#define DPAA2_FAF_TOTAL_SIZE \
- (DPAA2_FAFE_PSR_SIZE + DPAA2_FAF_PSR_SIZE)
-
-/* Just most popular Frame attribute flags (FAF) here.*/
-enum dpaa2_rx_faf_offset {
- /* Set by SP start*/
- FAFE_VXLAN_IN_VLAN_FRAM = 0,
- FAFE_VXLAN_IN_IPV4_FRAM = 1,
- FAFE_VXLAN_IN_IPV6_FRAM = 2,
- FAFE_VXLAN_IN_UDP_FRAM = 3,
- FAFE_VXLAN_IN_TCP_FRAM = 4,
-
- FAFE_ECPRI_FRAM = 7,
- /* Set by SP end*/
-
- FAF_GTP_PRIMED_FRAM = 1 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_PTP_FRAM = 3 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_VXLAN_FRAM = 4 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_ETH_FRAM = 10 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_LLC_SNAP_FRAM = 18 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_VLAN_FRAM = 21 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_PPPOE_PPP_FRAM = 25 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_MPLS_FRAM = 27 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_ARP_FRAM = 30 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IPV4_FRAM = 34 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IPV6_FRAM = 42 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IP_FRAM = 48 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IP_FRAG_FRAM = 50 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_ICMP_FRAM = 57 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IGMP_FRAM = 58 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_GRE_FRAM = 65 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_UDP_FRAM = 70 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_TCP_FRAM = 72 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IPSEC_FRAM = 77 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IPSEC_ESP_FRAM = 78 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_IPSEC_AH_FRAM = 79 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_SCTP_FRAM = 81 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_DCCP_FRAM = 83 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_GTP_FRAM = 87 + DPAA2_FAFE_PSR_SIZE * 8,
- FAF_ESP_FRAM = 89 + DPAA2_FAFE_PSR_SIZE * 8,
-};
-
-enum dpaa2_ecpri_fafe_type {
- ECPRI_FAFE_TYPE_0 = (8 - FAFE_ECPRI_FRAM),
- ECPRI_FAFE_TYPE_1 = (8 - FAFE_ECPRI_FRAM) | (1 << 1),
- ECPRI_FAFE_TYPE_2 = (8 - FAFE_ECPRI_FRAM) | (2 << 1),
- ECPRI_FAFE_TYPE_3 = (8 - FAFE_ECPRI_FRAM) | (3 << 1),
- ECPRI_FAFE_TYPE_4 = (8 - FAFE_ECPRI_FRAM) | (4 << 1),
- ECPRI_FAFE_TYPE_5 = (8 - FAFE_ECPRI_FRAM) | (5 << 1),
- ECPRI_FAFE_TYPE_6 = (8 - FAFE_ECPRI_FRAM) | (6 << 1),
- ECPRI_FAFE_TYPE_7 = (8 - FAFE_ECPRI_FRAM) | (7 << 1)
-};
-
-#define DPAA2_PR_ETH_OFF_OFFSET 19
-#define DPAA2_PR_TCI_OFF_OFFSET 21
-#define DPAA2_PR_LAST_ETYPE_OFFSET 23
-#define DPAA2_PR_L3_OFF_OFFSET 27
-#define DPAA2_PR_L4_OFF_OFFSET 30
-#define DPAA2_PR_L5_OFF_OFFSET 31
-#define DPAA2_PR_NXTHDR_OFF_OFFSET 34
-
#define DPAA2_ECPRI_MAX_EXTRACT_NB 8
-struct ipv4_sd_addr_extract_rule {
- uint32_t ipv4_src;
- uint32_t ipv4_dst;
-};
-
-struct ipv6_sd_addr_extract_rule {
- uint8_t ipv6_src[NH_FLD_IPV6_ADDR_SIZE];
- uint8_t ipv6_dst[NH_FLD_IPV6_ADDR_SIZE];
-};
-
-struct ipv4_ds_addr_extract_rule {
- uint32_t ipv4_dst;
- uint32_t ipv4_src;
-};
-
-struct ipv6_ds_addr_extract_rule {
- uint8_t ipv6_dst[NH_FLD_IPV6_ADDR_SIZE];
- uint8_t ipv6_src[NH_FLD_IPV6_ADDR_SIZE];
-};
-
-union ip_addr_extract_rule {
- struct ipv4_sd_addr_extract_rule ipv4_sd_addr;
- struct ipv6_sd_addr_extract_rule ipv6_sd_addr;
- struct ipv4_ds_addr_extract_rule ipv4_ds_addr;
- struct ipv6_ds_addr_extract_rule ipv6_ds_addr;
-};
-
-union ip_src_addr_extract_rule {
- uint32_t ipv4_src;
- uint8_t ipv6_src[NH_FLD_IPV6_ADDR_SIZE];
-};
-
-union ip_dst_addr_extract_rule {
- uint32_t ipv4_dst;
- uint8_t ipv6_dst[NH_FLD_IPV6_ADDR_SIZE];
-};
-
-enum ip_addr_extract_type {
- IP_NONE_ADDR_EXTRACT,
- IP_SRC_EXTRACT,
- IP_DST_EXTRACT,
- IP_SRC_DST_EXTRACT,
- IP_DST_SRC_EXTRACT
-};
+#define DPAA2_IBTH_MAX_EXTRACT_NB 4
enum key_prot_type {
/* HW extracts from standard protocol fields*/
@@ -389,9 +285,9 @@ struct key_prot_field {
uint32_t key_field;
};
-struct dpaa2_raw_region {
- uint8_t raw_start;
- uint8_t raw_size;
+struct dpaa2_ip_addr_extract {
+ uint32_t field;
+ uint8_t max_size;
};
struct dpaa2_key_profile {
@@ -399,35 +295,52 @@ struct dpaa2_key_profile {
uint8_t key_offset[DPKG_MAX_NUM_OF_EXTRACTS];
uint8_t key_size[DPKG_MAX_NUM_OF_EXTRACTS];
- enum ip_addr_extract_type ip_addr_type;
- uint8_t ip_addr_extract_pos;
- uint8_t ip_addr_extract_off;
-
- uint8_t raw_extract_pos;
- uint8_t raw_extract_off;
- uint8_t raw_extract_num;
+ struct dpaa2_ip_addr_extract ip_addr_extracts[2];
- uint8_t l4_src_port_present;
- uint8_t l4_src_port_pos;
- uint8_t l4_src_port_offset;
- uint8_t l4_dst_port_present;
- uint8_t l4_dst_port_pos;
- uint8_t l4_dst_port_offset;
+ uint8_t l4_sp_present;
+ uint8_t l4_sp_extract_idx;
+ uint8_t l4_sp_key_offset;
+ uint8_t l4_dp_present;
+ uint8_t l4_dp_extract_idx;
+ uint8_t l4_dp_key_offset;
struct key_prot_field prot_field[DPKG_MAX_NUM_OF_EXTRACTS];
uint16_t key_max_size;
- struct dpaa2_raw_region raw_region;
};
-struct dpaa2_key_extract {
+struct dpaa2_flow_tbl_profile {
struct dpkg_profile_cfg dpkg;
struct dpaa2_key_profile key_profile;
+ uint8_t *extract_param;
+ int entry_num;
+ uint8_t *entry_map;
+ int enabled;
+ int is_rss;
+ void *rss_flow;
+ union {
+ struct dpni_qos_tbl_cfg qos_cfg;
+ struct dpni_rx_dist_cfg tc_cfg;
+ };
+ int default_drop;
+ union {
+ struct rte_flow_action_jump default_jump;
+ struct rte_flow_action_queue default_queue;
+ };
};
-struct extract_s {
- struct dpaa2_key_extract qos_key_extract;
- struct dpaa2_key_extract tc_key_extract[MAX_TCS];
- uint8_t *qos_extract_param;
- uint8_t *tc_extract_param[MAX_TCS];
+struct dpaa2_flow_profile {
+ struct dpaa2_flow_tbl_profile qos_profile;
+ struct dpaa2_flow_tbl_profile tc_profile[MAX_TCS];
+ /** Meter per TC.*/
+ struct dpaa2_dev_meter_profile *tc_mtr_profile[MAX_TCS];
+ void *mtr_flow[MAX_TCS];
+ void *mempool[MAX_TCS];
+ uint8_t bp_idx[MAX_TCS];
+};
+
+struct rte_dpaa2_default_action_conf {
+ uint8_t default_tc;
+ uint8_t max_tc;
+ uint16_t default_flows[];
};
struct dpaa2_dev_meter_profile {
@@ -491,15 +404,12 @@ struct dpaa2_dev_priv {
/** RXQs in same TC share same cgid.*/
uint8_t cgid_in_use[MAX_TCS];
rte_spinlock_t meter_lock;
- /* Lowest priority FS flow id to receive flow steering miss frames. */
- uint16_t default_flow;
/* Current hash distribution size per RX TC, written by
* dpaa2_setup_flow_dist_size() and read by reta_query / reta_update.
* Zero means "use default" (= nb_rx_queues clamped to dist_queues).
*/
uint16_t dist_size_cur[MAX_TCS];
-
uint16_t dpni_ver_major;
uint16_t dpni_ver_minor;
uint32_t speed_capa;
@@ -508,7 +418,10 @@ struct dpaa2_dev_priv {
uint16_t ep_object_id; /**< Endpoint DPAA2 Object ID */
char ep_name[RTE_DEV_NAME_MAX_LEN];
- struct extract_s extract;
+ struct dpaa2_flow_profile flow_profile;
+ uint8_t nb_dcb_tcs;
+ uint8_t prio_dcb_tc[RTE_ETH_DCB_NUM_USER_PRIORITIES];
+ void *dcb_flow[RTE_ETH_DCB_NUM_USER_PRIORITIES];
uint16_t ss_offset;
uint64_t ss_iova;
@@ -528,6 +441,7 @@ struct dpaa2_dev_priv {
struct rte_eth_dev *eth_dev; /**< Pointer back to holding ethdev */
rte_spinlock_t lpbk_qp_lock;
+ bool enable_bp_flow_ctrl;
bool sp_protocol;
uint8_t channel_inuse;
/* Rx-interrupt mode latched at the first dev_configure */
@@ -549,7 +463,7 @@ struct dpaa2_dev_priv {
struct rte_mempool *tx_sg_pool;
- struct dpaa2_dev_flow *curr;
+ struct dpaa2_generic_flow *cur_flow;
LIST_HEAD(, dpaa2_dev_flow) flows;
LIST_HEAD(, dpaa2_dev_meter_profile) profiles;
LIST_HEAD(, dpaa2_dev_meter_policy) policies;
@@ -569,6 +483,8 @@ static inline int dpaa2_dev_cmp_dpni_ver(struct dpaa2_dev_priv *priv,
return priv->dpni_ver_major - ver_major;
}
+#define DPAA2_FLOW_DUMP dpaa2_dump_print
+
static inline void
dpaa2_dev_rx_print_parser_result(struct dpaa2_dev_priv *priv,
const struct qbman_fd *fd, const struct rte_mbuf *m)
@@ -619,21 +535,269 @@ dpaa2_timestamp_debug(struct dpaa2_dev_priv *priv,
prefix, timestamp, (int64_t)ts.tv_sec, (int64_t)ts.tv_nsec);
}
-int dpaa2_distset_to_dpkg_profile_cfg(uint64_t req_dist_set,
- struct dpkg_profile_cfg *kg_cfg);
+static inline void
+dpaa2_prot_field_string(uint32_t prot, uint32_t field,
+ char *string, size_t size)
+{
+ if (prot == NET_PROT_ETH) {
+ strlcpy(string, "eth", size);
+ if (field == NH_FLD_ETH_DA)
+ strlcat(string, ".dst", size);
+ else if (field == NH_FLD_ETH_SA)
+ strlcat(string, ".src", size);
+ else if (field == NH_FLD_ETH_TYPE)
+ strlcat(string, ".type", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_VLAN) {
+ strlcpy(string, "vlan", size);
+ if (field == NH_FLD_VLAN_TCI)
+ strlcat(string, ".tci", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_IP) {
+ strlcpy(string, "ip", size);
+ if (field == NH_FLD_IP_SRC)
+ strlcat(string, ".src", size);
+ else if (field == NH_FLD_IP_DST)
+ strlcat(string, ".dst", size);
+ else if (field == NH_FLD_IP_PROTO)
+ strlcat(string, ".proto", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_TCP) {
+ strlcpy(string, "tcp", size);
+ if (field == NH_FLD_TCP_PORT_SRC)
+ strlcat(string, ".src", size);
+ else if (field == NH_FLD_TCP_PORT_DST)
+ strlcat(string, ".dst", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_UDP) {
+ strlcpy(string, "udp", size);
+ if (field == NH_FLD_UDP_PORT_SRC)
+ strlcat(string, ".src", size);
+ else if (field == NH_FLD_UDP_PORT_DST)
+ strlcat(string, ".dst", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_ICMP) {
+ strlcpy(string, "icmp", size);
+ if (field == NH_FLD_ICMP_TYPE)
+ strlcat(string, ".type", size);
+ else if (field == NH_FLD_ICMP_CODE)
+ strlcat(string, ".code", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_SCTP) {
+ strlcpy(string, "sctp", size);
+ if (field == NH_FLD_SCTP_PORT_SRC)
+ strlcat(string, ".src", size);
+ else if (field == NH_FLD_SCTP_PORT_DST)
+ strlcat(string, ".dst", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_GRE) {
+ strlcpy(string, "gre", size);
+ if (field == NH_FLD_GRE_TYPE)
+ strlcat(string, ".type", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_GTP) {
+ strlcpy(string, "gtp", size);
+ if (field == NH_FLD_GTP_TEID)
+ strlcat(string, ".teid", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else if (prot == NET_PROT_IPSEC_ESP) {
+ strlcpy(string, "esp", size);
+ if (field == NH_FLD_IPSEC_ESP_SPI)
+ strlcat(string, ".spi", size);
+ else if (field == NH_FLD_IPSEC_ESP_SEQUENCE_NUM)
+ strlcat(string, ".seq", size);
+ else
+ strlcat(string, ".unknown field", size);
+ } else {
+ snprintf(string, size, "unknown protocol(%d)", prot);
+ }
+}
+
+static inline void
+dpaa2_dump_dpkg(const struct dpkg_profile_cfg *dpkg)
+{
+ int idx;
+ char string[32];
+ const struct dpkg_extract *extract;
+ enum dpkg_extract_type type;
+ enum net_prot prot;
+ uint32_t field;
+
+ for (idx = 0; idx < dpkg->num_extracts; idx++) {
+ extract = &dpkg->extracts[idx];
+ type = extract->type;
+ if (type == DPKG_EXTRACT_FROM_HDR) {
+ prot = extract->extract.from_hdr.prot;
+ field = extract->extract.from_hdr.field;
+ dpaa2_prot_field_string(prot, field, string, sizeof(string));
+ } else if (type == DPKG_EXTRACT_FROM_DATA) {
+ snprintf(string, sizeof(string), "raw offset/len: %d/%d",
+ extract->extract.from_data.offset,
+ extract->extract.from_data.size);
+ } else if (type == DPKG_EXTRACT_FROM_PARSE) {
+ snprintf(string, sizeof(string), "parse offset/len: %d/%d",
+ extract->extract.from_parse.offset,
+ extract->extract.from_parse.size);
+ }
+ DPAA2_FLOW_DUMP("%s", string);
+ if ((idx + 1) < dpkg->num_extracts)
+ DPAA2_FLOW_DUMP(" / ");
+ else
+ DPAA2_FLOW_DUMP("\r\n\n");
+ }
+}
+
+static inline int
+dpaa2_extract_prev_ip_addr_pos(const struct dpaa2_key_profile *profile)
+{
+ int idx = -ENXIO;
+
+ if (!profile->num)
+ return -ENXIO;
+
+ for (idx = profile->num - 1; idx >= 0; idx--) {
+ if (!(profile->prot_field[idx].type == DPAA2_NET_PROT_KEY &&
+ profile->prot_field[idx].prot == NET_PROT_IP &&
+ (profile->prot_field[idx].key_field == NH_FLD_IP_SRC ||
+ profile->prot_field[idx].key_field == NH_FLD_IP_DST)))
+ break;
+ }
+
+ if (idx >= 0)
+ return idx;
+ return -ENXIO;
+}
+
+static inline int
+dpaa2_extract_ip_addr_add(uint32_t field,
+ struct dpaa2_key_profile *key_profile, uint8_t size,
+ int *update, int *pos)
+{
+ struct dpaa2_ip_addr_extract *ip_addr_extracts;
+ uint8_t i = 0;
+ uint16_t max_size_save = key_profile->key_max_size;
+ char log_buf[128];
+
+ if (field != NH_FLD_IP_SRC && field != NH_FLD_IP_DST)
+ return -EINVAL;
+
+ max_size_save -= key_profile->ip_addr_extracts[0].max_size;
+ max_size_save -= key_profile->ip_addr_extracts[1].max_size;
+ ip_addr_extracts = key_profile->ip_addr_extracts;
+ while (i < 2) {
+ if (ip_addr_extracts[i].field == field) {
+ if (size > ip_addr_extracts[i].max_size)
+ ip_addr_extracts[i].max_size = size;
+ break;
+ }
+ if (!ip_addr_extracts[i].field) {
+ ip_addr_extracts[i].field = field;
+ ip_addr_extracts[i].max_size = size;
+ if (update)
+ *update = 1;
+ break;
+ }
+ i++;
+ }
+ if (i > 1) {
+ snprintf(log_buf, sizeof(log_buf),
+ "field[0](%d)/size[0](%d)/field[1](%d)/size[1](%d)",
+ ip_addr_extracts[0].field,
+ ip_addr_extracts[0].max_size,
+ ip_addr_extracts[1].field,
+ ip_addr_extracts[1].max_size);
+ DPAA2_FLOW_DUMP("Invalid IP address extracts:%s\n",
+ log_buf);
+ return -EINVAL;
+ }
+
+ if (pos)
+ *pos = i;
+
+ max_size_save += key_profile->ip_addr_extracts[0].max_size;
+ max_size_save += key_profile->ip_addr_extracts[1].max_size;
+ key_profile->key_max_size = max_size_save;
+
+ return 0;
+}
+
+static inline uint8_t
+dpaa2_profile_insert_no_ipaddr_extract(struct dpaa2_key_profile *profile,
+ uint8_t size, uint8_t *poffset, int *ppos,
+ const struct key_prot_field *prot)
+{
+ uint8_t idx, ip_addr_num = 0, offset;
+
+ if (profile->ip_addr_extracts[0].field &&
+ profile->ip_addr_extracts[1].field) {
+ idx = profile->num - 2;
+ ip_addr_num = 2;
+ } else if (profile->ip_addr_extracts[0].field) {
+ idx = profile->num - 1;
+ ip_addr_num = 1;
+ } else {
+ idx = profile->num;
+ }
-int dpaa2_setup_flow_dist(struct rte_eth_dev *eth_dev,
- uint64_t req_dist_set, int tc_index);
+ if (idx > 0)
+ offset = profile->key_offset[idx - 1] + profile->key_size[idx - 1];
+ else
+ offset = 0;
-int dpaa2_setup_flow_dist_size(struct rte_eth_dev *eth_dev,
- uint64_t req_dist_set, int tc_index, uint16_t dist_size);
+ if (idx > 0) {
+ profile->key_offset[idx] =
+ profile->key_offset[idx - 1] + profile->key_size[idx - 1];
+ } else {
+ profile->key_offset[idx] = 0;
+ }
+ if (ppos)
+ *ppos = profile->key_offset[idx];
+ profile->key_size[idx] = size;
+ profile->key_max_size += size;
+ profile->num++;
+
+ if (ip_addr_num > 0) {
+ memmove(&profile->prot_field[idx + 1],
+ &profile->prot_field[idx],
+ sizeof(struct key_prot_field) * ip_addr_num);
+ }
+ if (poffset)
+ *poffset = offset;
-int dpaa2_remove_flow_dist(struct rte_eth_dev *eth_dev,
- uint8_t tc_index);
+ if (prot) {
+ rte_memcpy(&profile->prot_field[idx], prot,
+ sizeof(struct key_prot_field));
+ }
-int dpaa2_attach_bp_list(struct dpaa2_dev_priv *priv,
- struct fsl_mc_io *dpni, void *blist);
+ return idx;
+}
+
+static inline void
+dpaa2_dpkg_insert_extract(struct dpkg_profile_cfg *kg_cfg,
+ int idx, const struct dpkg_extract *extract)
+{
+ int i;
+ if (idx != kg_cfg->num_extracts) {
+ /* Not the last extract index, must have IP address extract.*/
+ for (i = kg_cfg->num_extracts - 1; i >= idx; i--) {
+ rte_memcpy(&kg_cfg->extracts[i + 1],
+ &kg_cfg->extracts[i], sizeof(struct dpkg_extract));
+ }
+ }
+
+ rte_memcpy(&kg_cfg->extracts[idx], extract, sizeof(struct dpkg_extract));
+ kg_cfg->num_extracts++;
+}
__rte_internal
int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,
int eth_rx_queue_id,
@@ -678,7 +842,7 @@ uint16_t dpaa2_dev_tx_multi_txq_ordered(void **queue,
struct rte_mbuf **bufs, uint16_t nb_pkts);
void dpaa2_dev_free_eqresp_buf(uint16_t eqresp_ci, struct dpaa2_queue *dpaa2_q);
-void dpaa2_flow_clean(struct rte_eth_dev *dev);
+void dpaa2_flow_clean(struct rte_eth_dev *dev, uint8_t tc_id);
uint16_t dpaa2_dev_tx_conf(void *txq, int drain);
void
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index c616857e98..8456b3b80d 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2018-2022 NXP
+ * Copyright 2018-2026 NXP
*/
#include <sys/queue.h>
@@ -16,6 +16,7 @@
#include <rte_flow_driver.h>
#include <rte_tailq.h>
+#include <eal_export.h>
#include <fsl_dpni.h>
#include <fsl_dpkg.h>
@@ -23,7 +24,28 @@
#include <dpaa2_pmd_logs.h>
#include "dpaa2_parser_decode.h"
-static char *dpaa2_flow_control_log;
+#ifndef RTE_DPAA2_ONE_LEVEL_GROUP_FLOW
+#define RTE_DPAA2_ONE_LEVEL_GROUP_FLOW 0
+#endif
+#ifndef RTE_DPAA2_QOS_GROUP_FLOW
+#define RTE_DPAA2_QOS_GROUP_FLOW 1
+#endif
+#ifndef RTE_DPAA2_FS_GROUP_FLOW
+#define RTE_DPAA2_FS_GROUP_FLOW 2
+#endif
+
+#ifndef RTE_DPAA2_FLOW_GROUP_TYPE_GET
+#define RTE_DPAA2_FLOW_GROUP_TYPE_GET(group) \
+ ((void)(group), RTE_DPAA2_ONE_LEVEL_GROUP_FLOW)
+#endif
+
+#ifndef RTE_DPAA2_FLOW_GROUP_ID_GET
+#define RTE_DPAA2_FLOW_GROUP_ID_GET(group) (group)
+#endif
+
+static bool dpaa2_flow_control_log;
+
+#define DPAA2_FLOW_QOS_GROUP_ID 0xff
/* Default size of a key */
#define DPNI_DEFAULT_KEY_SIZE 24
@@ -34,31 +56,61 @@ enum dpaa2_flow_entry_size {
};
enum dpaa2_flow_dist_type {
- DPAA2_FLOW_QOS_TYPE = 1 << 0,
- DPAA2_FLOW_FS_TYPE = 1 << 1
+ DPAA2_FLOW_NULL_TYPE,
+ DPAA2_FLOW_QOS_TYPE,
+ DPAA2_FLOW_FS_TYPE
};
#define DPAA2_FLOW_RAW_OFFSET_FIELD_SHIFT 16
#define DPAA2_FLOW_MAX_KEY_SIZE 16
-#define DPAA2_PROT_FIELD_STRING_SIZE 16
+
#define VXLAN_HF_VNI 0x08
+#define DPAA2_MAX_ACTION_PER_FLOW_NUM 4
-struct dpaa2_dev_flow {
- LIST_ENTRY(dpaa2_dev_flow) next;
- struct dpni_rule_cfg qos_rule;
- uint8_t *qos_key_addr;
- uint8_t *qos_mask_addr;
- uint16_t qos_rule_size;
- struct dpni_rule_cfg fs_rule;
- uint8_t qos_real_key_size;
- uint8_t fs_real_key_size;
- uint8_t *fs_key_addr;
- uint8_t *fs_mask_addr;
- uint16_t fs_rule_size;
- uint8_t tc_id; /** Traffic Class ID. */
- uint8_t tc_index; /** index within this Traffic Class. */
+struct dpaa2_dev_flow_qos_action {
+ enum rte_flow_action_type action_jump_type;
+ struct rte_flow_action_jump action_jump_cfg;
+};
+
+struct dpaa2_dev_flow_fs_action {
enum rte_flow_action_type action_type;
struct dpni_fs_action_cfg fs_action_cfg;
+ char dst_name[DPNI_FS_REDIR_MAX_NUM][RTE_ETH_NAME_MAX_LEN];
+};
+
+union dpaa2_dev_flow_action {
+ struct dpaa2_dev_flow_qos_action qos_action;
+ struct dpaa2_dev_flow_fs_action fs_action;
+};
+
+struct dpaa2_generic_flow {
+ struct dpni_rule_cfg rule_cfg;
+ uint8_t *key_addr;
+ uint8_t *mask_addr;
+ uint16_t rule_size;
+ uint16_t entry_index;
+ /** This flow has IPv4 or IPv6 extracts.*/
+ enum net_prot ip_key;
+ /** This flow has IPv4 or IPv6 source extract.*/
+ enum net_prot ip_src;
+ /** This flow has IPv4 or IPv6 destination extract.*/
+ enum net_prot ip_dst;
+ struct dpaa2_dev_priv *priv;
+ union {
+ uint8_t tc_id; /** For FS flow only.*/
+ uint8_t flow_id; /** For QoS flow with direct_queue.*/
+ };
+ union dpaa2_dev_flow_action flow_action;
+ int is_rss;
+ int direct_queue;
+};
+
+struct dpaa2_dev_flow {
+ LIST_ENTRY(dpaa2_dev_flow) next;
+ struct dpaa2_generic_flow *qos_flow;
+ struct dpaa2_generic_flow *fs_flow;
+ struct dpaa2_dev_priv *priv;
+ int is_meter_flow;
};
struct rte_dpaa2_flow_item {
@@ -81,20 +133,18 @@ enum rte_flow_item_type dpaa2_hp_supported_pattern_type[] = {
RTE_FLOW_ITEM_TYPE_GTP,
RTE_FLOW_ITEM_TYPE_ESP,
RTE_FLOW_ITEM_TYPE_AH,
- RTE_FLOW_ITEM_TYPE_MPLS,
- RTE_FLOW_ITEM_TYPE_PPPOES,
- RTE_FLOW_ITEM_TYPE_RAW
+ RTE_FLOW_ITEM_TYPE_RAW,
+ /** Hardware parser supports vlan protocol only.*/
+ RTE_FLOW_ITEM_TYPE_VXLAN
};
static const
enum rte_flow_item_type dpaa2_sp_supported_pattern_type[] = {
- RTE_FLOW_ITEM_TYPE_VXLAN,
- RTE_FLOW_ITEM_TYPE_ECPRI
+ RTE_FLOW_ITEM_TYPE_ECPRI,
+ RTE_FLOW_ITEM_TYPE_GENEVE
};
-static const
-enum rte_flow_action_type dpaa2_supported_action_type[] = {
- RTE_FLOW_ACTION_TYPE_END,
+static const enum rte_flow_action_type dpaa2_supported_fs_action_type[] = {
RTE_FLOW_ACTION_TYPE_QUEUE,
RTE_FLOW_ACTION_TYPE_PORT_ID,
RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
@@ -102,27 +152,63 @@ enum rte_flow_action_type dpaa2_supported_action_type[] = {
RTE_FLOW_ACTION_TYPE_DROP
};
+static const enum rte_flow_action_type dpaa2_supported_qos_action_type[] = {
+ RTE_FLOW_ACTION_TYPE_QUEUE,
+ RTE_FLOW_ACTION_TYPE_JUMP,
+ RTE_FLOW_ACTION_TYPE_DROP
+};
+
+static const enum rte_flow_action_type dpaa2_supported_meter_action_type[] = {
+ RTE_FLOW_ACTION_TYPE_METER_MARK,
+ RTE_FLOW_ACTION_TYPE_METER
+};
+
+#define DPAA2_FLOW_HDR_HEX_DUMP_SIZE \
+ (RTE_MAX(sizeof(struct rte_flow_item_eth), \
+ RTE_MAX(sizeof(struct rte_flow_item_vlan), \
+ RTE_MAX(sizeof(struct rte_flow_item_ipv4), \
+ RTE_MAX(sizeof(struct rte_flow_item_ipv6), \
+ RTE_MAX(sizeof(struct rte_flow_item_icmp), \
+ RTE_MAX(sizeof(struct rte_flow_item_udp), \
+ RTE_MAX(sizeof(struct rte_flow_item_tcp), \
+ RTE_MAX(sizeof(struct rte_flow_item_sctp), \
+ RTE_MAX(sizeof(struct rte_flow_item_gre), \
+ RTE_MAX(sizeof(struct rte_flow_item_ah), \
+ RTE_MAX(sizeof(struct rte_flow_item_esp), \
+ RTE_MAX(sizeof(struct rte_flow_item_vxlan), \
+ RTE_MAX(sizeof(struct rte_flow_item_ecpri), \
+ sizeof(struct rte_flow_item_gtp)))))))))))))) * 10)
+
+
#ifndef __cplusplus
static const struct rte_flow_item_eth dpaa2_flow_item_eth_mask = {
- .hdr.dst_addr.addr_bytes = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
- .hdr.src_addr.addr_bytes = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
- .hdr.ether_type = RTE_BE16(0xffff),
+ .dst.addr_bytes = "\xff\xff\xff\xff\xff\xff",
+ .src.addr_bytes = "\xff\xff\xff\xff\xff\xff",
+ .type = RTE_BE16(0xffff),
};
static const struct rte_flow_item_vlan dpaa2_flow_item_vlan_mask = {
- .hdr.vlan_tci = RTE_BE16(0xffff),
+ .tci = RTE_BE16(0xffff),
};
static const struct rte_flow_item_ipv4 dpaa2_flow_item_ipv4_mask = {
.hdr.src_addr = RTE_BE32(0xffffffff),
.hdr.dst_addr = RTE_BE32(0xffffffff),
.hdr.next_proto_id = 0xff,
+ .hdr.packet_id = 0xffff,
+ .hdr.fragment_offset = 0xffff,
};
static const struct rte_flow_item_ipv6 dpaa2_flow_item_ipv6_mask = {
.hdr = {
- .src_addr = RTE_IPV6_MASK_FULL,
- .dst_addr = RTE_IPV6_MASK_FULL,
+ .src_addr = {
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ },
+ .dst_addr = {
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ },
.proto = 0xff
},
};
@@ -170,7 +256,12 @@ static const struct rte_flow_item_gre dpaa2_flow_item_gre_mask = {
static const struct rte_flow_item_vxlan dpaa2_flow_item_vxlan_mask = {
.flags = 0xff,
- .vni = { 0xff, 0xff, 0xff },
+ .vni = "\xff\xff\xff",
+};
+
+static const struct rte_flow_item_geneve dpaa2_flow_item_geneve_mask = {
+ .protocol = RTE_BE16(0xffff),
+ .vni = "\xff\xff\xff",
};
static const struct rte_flow_item_ecpri dpaa2_flow_item_ecpri_mask = {
@@ -184,253 +275,182 @@ static const struct rte_flow_item_gtp dpaa2_flow_item_gtp_mask = {
.teid = RTE_BE32(0xffffffff),
};
-static const struct rte_flow_item_mpls dpaa2_flow_item_mpls_mask = {
- .label_tc_s = { 0xff, 0xff, 0xff },
-};
-
-static const struct rte_flow_item_pppoe dpaa2_flow_item_pppoe_mask = {
- .session_id = RTE_BE16(0xffff),
-};
-
#endif
+static inline uint8_t
+dpaa2_flow_entry_map_get(const uint8_t *entry_map,
+ uint16_t index)
+{
+ uint16_t byte_pos = index / 8;
+ uint16_t bit_pos = index % 8;
-#define DPAA2_FLOW_DUMP printf
+ return entry_map[byte_pos] & (1 << bit_pos);
+}
static inline void
-dpaa2_prot_field_string(uint32_t prot, uint32_t field,
- char *string)
+dpaa2_flow_entry_map_set(uint8_t *entry_map,
+ uint16_t index, int set)
{
- if (!dpaa2_flow_control_log)
- return;
+ uint16_t byte_pos = index / 8;
+ uint16_t bit_pos = index % 8;
- if (prot == NET_PROT_ETH) {
- strcpy(string, "eth");
- if (field == NH_FLD_ETH_DA)
- strcat(string, ".dst");
- else if (field == NH_FLD_ETH_SA)
- strcat(string, ".src");
- else if (field == NH_FLD_ETH_TYPE)
- strcat(string, ".type");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_VLAN) {
- strcpy(string, "vlan");
- if (field == NH_FLD_VLAN_TCI)
- strcat(string, ".tci");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_IP) {
- strcpy(string, "ip");
- if (field == NH_FLD_IP_SRC)
- strcat(string, ".src");
- else if (field == NH_FLD_IP_DST)
- strcat(string, ".dst");
- else if (field == NH_FLD_IP_PROTO)
- strcat(string, ".proto");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_TCP) {
- strcpy(string, "tcp");
- if (field == NH_FLD_TCP_PORT_SRC)
- strcat(string, ".src");
- else if (field == NH_FLD_TCP_PORT_DST)
- strcat(string, ".dst");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_UDP) {
- strcpy(string, "udp");
- if (field == NH_FLD_UDP_PORT_SRC)
- strcat(string, ".src");
- else if (field == NH_FLD_UDP_PORT_DST)
- strcat(string, ".dst");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_ICMP) {
- strcpy(string, "icmp");
- if (field == NH_FLD_ICMP_TYPE)
- strcat(string, ".type");
- else if (field == NH_FLD_ICMP_CODE)
- strcat(string, ".code");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_SCTP) {
- strcpy(string, "sctp");
- if (field == NH_FLD_SCTP_PORT_SRC)
- strcat(string, ".src");
- else if (field == NH_FLD_SCTP_PORT_DST)
- strcat(string, ".dst");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_GRE) {
- strcpy(string, "gre");
- if (field == NH_FLD_GRE_TYPE)
- strcat(string, ".type");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_GTP) {
- rte_strscpy(string, "gtp", DPAA2_PROT_FIELD_STRING_SIZE);
- if (field == NH_FLD_GTP_TEID)
- strcat(string, ".teid");
- else
- strcat(string, ".unknown field");
- } else if (prot == NET_PROT_IPSEC_ESP) {
- rte_strscpy(string, "esp", DPAA2_PROT_FIELD_STRING_SIZE);
- if (field == NH_FLD_IPSEC_ESP_SPI)
- strcat(string, ".spi");
- else if (field == NH_FLD_IPSEC_ESP_SEQUENCE_NUM)
- strcat(string, ".seq");
- else
- strcat(string, ".unknown field");
- } else {
- sprintf(string, "unknown protocol(%d)", prot);
- }
+ if (set)
+ entry_map[byte_pos] |= (1 << bit_pos);
+ else
+ entry_map[byte_pos] &= ~(((uint8_t)1) << bit_pos);
}
static inline void
-dpaa2_flow_qos_extracts_log(const struct dpaa2_dev_priv *priv)
+dpaa2_flow_extracts_log(const struct dpaa2_dev_priv *priv,
+ const char *prefix, uint8_t tc_id)
{
- int idx;
- char string[32];
- const struct dpkg_profile_cfg *dpkg =
- &priv->extract.qos_key_extract.dpkg;
- const struct dpkg_extract *extract;
- enum dpkg_extract_type type;
- enum net_prot prot;
- uint32_t field;
+ char string[1024];
+ const struct dpaa2_flow_tbl_profile *tbl_profile;
+ int offset = 0;
if (!dpaa2_flow_control_log)
return;
- DPAA2_FLOW_DUMP("QoS table: %d extracts\r\n",
- dpkg->num_extracts);
- for (idx = 0; idx < dpkg->num_extracts; idx++) {
- extract = &dpkg->extracts[idx];
- type = extract->type;
- if (type == DPKG_EXTRACT_FROM_HDR) {
- prot = extract->extract.from_hdr.prot;
- field = extract->extract.from_hdr.field;
- dpaa2_prot_field_string(prot, field,
- string);
- } else if (type == DPKG_EXTRACT_FROM_DATA) {
- sprintf(string, "raw offset/len: %d/%d",
- extract->extract.from_data.offset,
- extract->extract.from_data.size);
- } else if (type == DPKG_EXTRACT_FROM_PARSE) {
- sprintf(string, "parse offset/len: %d/%d",
- extract->extract.from_parse.offset,
- extract->extract.from_parse.size);
- }
- DPAA2_FLOW_DUMP("%s", string);
- if ((idx + 1) < dpkg->num_extracts)
- DPAA2_FLOW_DUMP(" / ");
- }
- DPAA2_FLOW_DUMP("\r\n");
+ offset += sprintf(&string[offset],
+ "%s's", priv->eth_dev->data->name);
+ if (tc_id >= MAX_TCS) {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ offset += sprintf(&string[offset], " QoS");
+ } else {
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ offset += sprintf(&string[offset], " FS[%d]", tc_id);
+ }
+ offset += sprintf(&string[offset],
+ " table: %d extracts/%d entries\n",
+ tbl_profile->dpkg.num_extracts, tbl_profile->entry_num);
+ DPAA2_FLOW_DUMP("\n%s %s", prefix, string);
+ dpaa2_dump_dpkg(&tbl_profile->dpkg);
}
static inline void
-dpaa2_flow_fs_extracts_log(const struct dpaa2_dev_priv *priv,
- int tc_id)
+dpaa2_flow_qos_entry_log(const char *log_info,
+ const struct dpaa2_generic_flow *flow)
{
int idx;
- char string[32];
- const struct dpkg_profile_cfg *dpkg =
- &priv->extract.tc_key_extract[tc_id].dpkg;
- const struct dpkg_extract *extract;
- enum dpkg_extract_type type;
- enum net_prot prot;
- uint32_t field;
+ struct rte_eth_dev *dev = flow->priv->eth_dev;
if (!dpaa2_flow_control_log)
return;
- DPAA2_FLOW_DUMP("FS table: %d extracts in TC[%d]\r\n",
- dpkg->num_extracts, tc_id);
- for (idx = 0; idx < dpkg->num_extracts; idx++) {
- extract = &dpkg->extracts[idx];
- type = extract->type;
- if (type == DPKG_EXTRACT_FROM_HDR) {
- prot = extract->extract.from_hdr.prot;
- field = extract->extract.from_hdr.field;
- dpaa2_prot_field_string(prot, field,
- string);
- } else if (type == DPKG_EXTRACT_FROM_DATA) {
- sprintf(string, "raw offset/len: %d/%d",
- extract->extract.from_data.offset,
- extract->extract.from_data.size);
- } else if (type == DPKG_EXTRACT_FROM_PARSE) {
- sprintf(string, "parse offset/len: %d/%d",
- extract->extract.from_parse.offset,
- extract->extract.from_parse.size);
- }
- DPAA2_FLOW_DUMP("%s", string);
- if ((idx + 1) < dpkg->num_extracts)
- DPAA2_FLOW_DUMP(" / ");
- }
- DPAA2_FLOW_DUMP("\r\n");
+ DPAA2_FLOW_DUMP("%s: %s QoS entry[%d](size %d/%d) to select TC[%d]\n",
+ dev->data->name, log_info, flow->entry_index, flow->rule_size,
+ flow->rule_cfg.key_size,
+ flow->flow_action.qos_action.action_jump_cfg.group);
+
+ DPAA2_FLOW_DUMP("key:\r\n");
+ for (idx = 0; idx < flow->rule_size; idx++)
+ DPAA2_FLOW_DUMP("%02x ", flow->key_addr[idx]);
+
+ DPAA2_FLOW_DUMP("\r\nmask:\r\n");
+ for (idx = 0; idx < flow->rule_size; idx++)
+ DPAA2_FLOW_DUMP("%02x ", flow->mask_addr[idx]);
+ DPAA2_FLOW_DUMP("\r\n\n");
}
static inline void
-dpaa2_flow_qos_entry_log(const char *log_info,
- const struct dpaa2_dev_flow *flow, int qos_index)
+dpaa2_flow_fs_entry_log(const char *log_info,
+ const struct dpaa2_generic_flow *flow)
{
int idx;
- uint8_t *key, *mask;
+ struct rte_eth_dev *dev = flow->priv->eth_dev;
+ const struct dpaa2_dev_flow_fs_action *fs_action;
if (!dpaa2_flow_control_log)
return;
- if (qos_index >= 0) {
- DPAA2_FLOW_DUMP("%s QoS entry[%d](size %d/%d) for TC[%d]\r\n",
- log_info, qos_index, flow->qos_rule_size,
- flow->qos_rule.key_size,
- flow->tc_id);
- } else {
- DPAA2_FLOW_DUMP("%s QoS entry(size %d/%d) for TC[%d]\r\n",
- log_info, flow->qos_rule_size,
- flow->qos_rule.key_size,
- flow->tc_id);
- }
+ DPAA2_FLOW_DUMP("%s: %s FS[%d]/entry[%d](size %d/%d)\r\n",
+ dev->data->name, log_info, flow->tc_id,
+ flow->entry_index, flow->rule_size,
+ flow->rule_cfg.key_size);
- key = flow->qos_key_addr;
- mask = flow->qos_mask_addr;
+ fs_action = &flow->flow_action.fs_action;
DPAA2_FLOW_DUMP("key:\r\n");
- for (idx = 0; idx < flow->qos_rule_size; idx++)
- DPAA2_FLOW_DUMP("%02x ", key[idx]);
+ for (idx = 0; idx < flow->rule_size; idx++)
+ DPAA2_FLOW_DUMP("%02x ", flow->key_addr[idx]);
DPAA2_FLOW_DUMP("\r\nmask:\r\n");
- for (idx = 0; idx < flow->qos_rule_size; idx++)
- DPAA2_FLOW_DUMP("%02x ", mask[idx]);
- DPAA2_FLOW_DUMP("\r\n");
+ for (idx = 0; idx < flow->rule_size; idx++)
+ DPAA2_FLOW_DUMP("%02x ", flow->mask_addr[idx]);
+ DPAA2_FLOW_DUMP("\r\nAction: ");
+ if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ DPAA2_FLOW_DUMP("Receive to flow%d\r\n\n",
+ fs_action->fs_action_cfg.flow_id);
+ } else if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
+ fs_action->action_type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) {
+ DPAA2_FLOW_DUMP("Re-direct to port(s): ");
+ for (idx = 0; idx < fs_action->fs_action_cfg.num_tokens; idx++) {
+ DPAA2_FLOW_DUMP("%s", fs_action->dst_name[idx]);
+ if ((idx + 1) < fs_action->fs_action_cfg.num_tokens)
+ DPAA2_FLOW_DUMP(", ");
+ }
+ DPAA2_FLOW_DUMP("\r\n");
+ } else if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_DROP) {
+ DPAA2_FLOW_DUMP("Drop\r\n\n");
+ } else {
+ DPAA2_FLOW_DUMP("Un-supported action type(%d)\r\n\n",
+ fs_action->action_type);
+ }
}
static inline void
-dpaa2_flow_fs_entry_log(const char *log_info,
- const struct dpaa2_dev_flow *flow)
+dpaa2_dump_extract_map(const struct dpaa2_dev_priv *priv,
+ const char *prefix, enum dpaa2_flow_dist_type dist_type,
+ uint8_t group)
{
- int idx;
- uint8_t *key, *mask;
+ int idx, offset = 0;
+ char string[2048];
+ const struct dpaa2_flow_tbl_profile *tbl_profile;
if (!dpaa2_flow_control_log)
return;
- DPAA2_FLOW_DUMP("%s FS/TC entry[%d](size %d/%d) of TC[%d]\r\n",
- log_info, flow->tc_index,
- flow->fs_rule_size, flow->fs_rule.key_size,
- flow->tc_id);
+ DPAA2_FLOW_DUMP("%s: %s\n", priv->eth_dev->data->name, prefix);
- key = flow->fs_key_addr;
- mask = flow->fs_mask_addr;
+ if (dist_type == DPAA2_FLOW_QOS_TYPE) {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ offset += sprintf(&string[offset], "QoS entry map:");
+ if (!tbl_profile->entry_num) {
+ offset += sprintf(&string[offset], " empty\n");
+ goto qos_dump_end;
+ } else {
+ offset += sprintf(&string[offset], "\n");
+ }
+ for (idx = 0; idx < priv->qos_entries; idx++) {
+ offset += sprintf(&string[offset], "%d ",
+ dpaa2_flow_entry_map_get(tbl_profile->entry_map, idx) ?
+ 1 : 0);
+ if (!((idx + 1) % 16) || (idx + 1) == priv->qos_entries)
+ offset += sprintf(&string[offset], "\n");
+ }
- DPAA2_FLOW_DUMP("key:\r\n");
- for (idx = 0; idx < flow->fs_rule_size; idx++)
- DPAA2_FLOW_DUMP("%02x ", key[idx]);
+qos_dump_end:
+ DPAA2_FLOW_DUMP("%s\n", string);
- DPAA2_FLOW_DUMP("\r\nmask:\r\n");
- for (idx = 0; idx < flow->fs_rule_size; idx++)
- DPAA2_FLOW_DUMP("%02x ", mask[idx]);
- DPAA2_FLOW_DUMP("\r\n");
+ return;
+ }
+
+ tbl_profile = &priv->flow_profile.tc_profile[group];
+ if (!tbl_profile->entry_num) {
+ DPAA2_FLOW_DUMP("FS[%d] table is empty\n\n", group);
+ return;
+ }
+
+ offset = sprintf(&string[offset],
+ "FS[%d] %d entries:\n", group, tbl_profile->entry_num);
+ for (idx = 0; idx < priv->fs_entries; idx++) {
+ offset += sprintf(&string[offset], "%d ",
+ dpaa2_flow_entry_map_get(tbl_profile->entry_map, idx) ?
+ 1 : 0);
+ if (!((idx + 1) % 16) || (idx + 1) == priv->fs_entries)
+ offset += sprintf(&string[offset], "\n");
+ }
+ DPAA2_FLOW_DUMP("%s\n", string);
}
static int
@@ -487,272 +507,481 @@ dpaa2_flow_l4_dst_port_extract(enum net_prot prot,
return false;
}
+static int
+dpaa2_flow_rss_conf_to_item(uint64_t req_dist_set,
+ struct rte_flow_item *items, uint8_t *spec_buf,
+ uint32_t buf_max)
+{
+ uint32_t i = 0, offset = 0;
+ uint64_t l2_mask, vlan_mask, ipv4_mask, ipv6_mask;
+ uint64_t dist_supported, udp_mask, tcp_mask, sctp_mask;
+ struct rte_flow_item_eth *eth = NULL;
+ struct rte_flow_item_pppoe *pppoe = NULL;
+ struct rte_flow_item_esp *esp = NULL;
+ struct rte_flow_item_ah *ah = NULL;
+ struct rte_flow_item_vlan *vlan = NULL;
+ struct rte_flow_item_mpls *mpls = NULL;
+ struct rte_flow_item_gtp *gtpu = NULL;
+ struct rte_flow_item_udp *udp = NULL;
+ struct rte_flow_item_tcp *tcp = NULL;
+ struct rte_flow_item_sctp *sctp = NULL;
+ struct rte_flow_item_ipv4 *ipv4 = NULL;
+
+ l2_mask = RTE_ETH_RSS_L2_PAYLOAD | RTE_ETH_RSS_ETH;
+ vlan_mask = RTE_ETH_RSS_C_VLAN | RTE_ETH_RSS_S_VLAN;
+ ipv4_mask = RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4 |
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER;
+ ipv6_mask = RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6 |
+ RTE_ETH_RSS_IPV6_EX | RTE_ETH_RSS_NONFRAG_IPV6_OTHER;
+ udp_mask = RTE_ETH_RSS_NONFRAG_IPV4_UDP |
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP |
+ RTE_ETH_RSS_IPV6_UDP_EX;
+ tcp_mask = RTE_ETH_RSS_NONFRAG_IPV4_TCP |
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP |
+ RTE_ETH_RSS_IPV6_TCP_EX;
+ sctp_mask = RTE_ETH_RSS_NONFRAG_IPV4_SCTP |
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP;
+ dist_supported = l2_mask | vlan_mask | ipv4_mask |
+ ipv6_mask | udp_mask | tcp_mask | sctp_mask |
+ RTE_ETH_RSS_GTPU | RTE_ETH_RSS_PPPOE |
+ RTE_ETH_RSS_ESP | RTE_ETH_RSS_AH | RTE_ETH_RSS_MPLS;
+ if (~dist_supported & req_dist_set) {
+ DPAA2_PMD_ERR("%s: Unsupported dist type:0x%" PRIx64,
+ __func__, (~dist_supported) & req_dist_set);
+ return -ENOTSUP;
+ }
+
+ if (req_dist_set & l2_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_ETH;
+ eth = (void *)(spec_buf + offset);
+ memset(eth, 0, sizeof(struct rte_flow_item_eth));
+ memset(eth->dst.addr_bytes, 0xff, RTE_ETHER_ADDR_LEN);
+ memset(eth->src.addr_bytes, 0xff, RTE_ETHER_ADDR_LEN);
+ eth->type = RTE_BE16(0xffff);
+ items[i].spec = eth;
+ items[i].mask = eth;
+ i++;
+ offset += sizeof(struct rte_flow_item_eth);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & RTE_ETH_RSS_PPPOE) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_PPPOES;
+ pppoe = (void *)(spec_buf + offset);
+ memset(pppoe, 0, sizeof(struct rte_flow_item_pppoe));
+ pppoe->session_id = RTE_BE16(0xffff);
+ items[i].spec = pppoe;
+ items[i].mask = pppoe;
+ i++;
+ offset += sizeof(struct rte_flow_item_pppoe);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & RTE_ETH_RSS_ESP) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_ESP;
+ esp = (void *)(spec_buf + offset);
+ memset(esp, 0, sizeof(struct rte_flow_item_esp));
+ esp->hdr.spi = RTE_BE32(0xffffffff);
+ items[i].spec = esp;
+ items[i].mask = esp;
+ i++;
+ offset += sizeof(struct rte_flow_item_esp);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & RTE_ETH_RSS_AH) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_AH;
+ ah = (void *)(spec_buf + offset);
+ memset(ah, 0, sizeof(struct rte_flow_item_ah));
+ ah->spi = RTE_BE32(0xffffffff);
+ items[i].spec = ah;
+ items[i].mask = ah;
+ i++;
+ offset += sizeof(struct rte_flow_item_ah);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & vlan_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_VLAN;
+ vlan = (void *)(spec_buf + offset);
+ memset(vlan, 0, sizeof(struct rte_flow_item_vlan));
+ vlan->tci = RTE_BE16(0xffff);
+ items[i].spec = vlan;
+ items[i].mask = vlan;
+ i++;
+ offset += sizeof(struct rte_flow_item_vlan);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & RTE_ETH_RSS_MPLS) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_MPLS;
+ mpls = (void *)(spec_buf + offset);
+ memset(mpls, 0, sizeof(struct rte_flow_item_mpls));
+ memset(mpls->label_tc_s, 0xff, 3);
+ items[i].spec = mpls;
+ items[i].mask = mpls;
+ i++;
+ offset += sizeof(struct rte_flow_item_mpls);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & RTE_ETH_RSS_GTPU) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_GTPU;
+ gtpu = (void *)(spec_buf + offset);
+ memset(gtpu, 0, sizeof(struct rte_flow_item_gtp));
+ gtpu->teid = RTE_BE32(0xffffffff);
+ items[i].spec = gtpu;
+ items[i].mask = gtpu;
+ i++;
+ offset += sizeof(struct rte_flow_item_gtp);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & udp_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_UDP;
+ udp = (void *)(spec_buf + offset);
+ memset(udp, 0, sizeof(struct rte_flow_item_udp));
+ udp->hdr.src_port = RTE_BE16(0xffff);
+ udp->hdr.dst_port = RTE_BE16(0xffff);
+ items[i].spec = udp;
+ items[i].mask = udp;
+ i++;
+ offset += sizeof(struct rte_flow_item_udp);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & tcp_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_TCP;
+ tcp = (void *)(spec_buf + offset);
+ memset(tcp, 0, sizeof(struct rte_flow_item_tcp));
+ tcp->hdr.src_port = RTE_BE16(0xffff);
+ tcp->hdr.dst_port = RTE_BE16(0xffff);
+ items[i].spec = tcp;
+ items[i].mask = tcp;
+ i++;
+ offset += sizeof(struct rte_flow_item_tcp);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & sctp_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_SCTP;
+ sctp = (void *)(spec_buf + offset);
+ memset(sctp, 0, sizeof(struct rte_flow_item_sctp));
+ sctp->hdr.src_port = RTE_BE16(0xffff);
+ sctp->hdr.dst_port = RTE_BE16(0xffff);
+ items[i].spec = sctp;
+ items[i].mask = sctp;
+ i++;
+ offset += sizeof(struct rte_flow_item_sctp);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ if (req_dist_set & ipv4_mask) {
+ items[i].type = RTE_FLOW_ITEM_TYPE_IPV4;
+ ipv4 = (void *)(spec_buf + offset);
+ memset(ipv4, 0, sizeof(struct rte_flow_item_ipv4));
+ ipv4->hdr.src_addr = RTE_BE32(0xffffffff);
+ ipv4->hdr.dst_addr = RTE_BE32(0xffffffff);
+ ipv4->hdr.next_proto_id = 0xff;
+ items[i].spec = ipv4;
+ items[i].mask = ipv4;
+ i++;
+ offset += sizeof(struct rte_flow_item_ipv4);
+ offset = RTE_CACHE_LINE_ROUNDUP(offset);
+ if (offset > buf_max)
+ return -ENOMEM;
+ }
+
+ items[i].type = RTE_FLOW_ITEM_TYPE_END;
+
+ return i;
+}
+
static int
dpaa2_flow_add_qos_rule(struct dpaa2_dev_priv *priv,
- struct dpaa2_dev_flow *flow)
+ struct dpaa2_generic_flow *flow)
{
- uint16_t qos_index;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
int ret;
struct fsl_mc_io *dpni = priv->hw;
+ uint8_t flags = DPNI_QOS_OPT_SET_TC_ONLY, flow_id = 0;
- if (priv->num_rx_tc <= 1 &&
- flow->action_type != RTE_FLOW_ACTION_TYPE_RSS) {
- DPAA2_PMD_WARN("No QoS Table for FS");
+ if (flow->entry_index >= priv->qos_entries) {
+ DPAA2_PMD_ERR("QoS table full(%d >= %d)",
+ flow->entry_index, priv->qos_entries);
+ return -EINVAL;
+ }
+ tbl_profile = &priv->flow_profile.qos_profile;
+ if (dpaa2_flow_entry_map_get(tbl_profile->entry_map,
+ flow->entry_index)) {
+ DPAA2_PMD_ERR("QoS entry[%d] has been occupied",
+ flow->entry_index);
return -EINVAL;
}
- /* QoS entry added is only effective for multiple TCs.*/
- qos_index = flow->tc_id * priv->fs_entries + flow->tc_index;
- if (qos_index >= priv->qos_entries) {
- DPAA2_PMD_ERR("QoS table full(%d >= %d)",
- qos_index, priv->qos_entries);
+ if (flow->flow_action.qos_action.action_jump_type !=
+ RTE_FLOW_ACTION_TYPE_JUMP) {
+ DPAA2_PMD_ERR("QoS entry[%d]'jump action not set(%d)",
+ flow->entry_index,
+ flow->flow_action.qos_action.action_jump_type);
+ return -EINVAL;
+ }
+ if (flow->flow_action.qos_action.action_jump_cfg.group >=
+ priv->num_rx_tc) {
+ DPAA2_PMD_ERR("QoS entry[%d]'dest table ID(%d) >= max(%d)",
+ flow->entry_index,
+ flow->flow_action.qos_action.action_jump_cfg.group,
+ priv->num_rx_tc);
return -EINVAL;
}
- dpaa2_flow_qos_entry_log("Start add", flow, qos_index);
+ dpaa2_flow_qos_entry_log("Add", flow);
+ if (flow->direct_queue) {
+ flags |= DPNI_QOS_OPT_SET_FLOW_ID;
+ flow_id = flow->flow_id;
+ }
ret = dpni_add_qos_entry(dpni, CMD_PRI_LOW,
- priv->token, &flow->qos_rule,
- flow->tc_id, qos_index,
- 0, 0);
+ priv->token, &flow->rule_cfg,
+ flow->flow_action.qos_action.action_jump_cfg.group,
+ flow->entry_index, flags, flow_id);
if (ret < 0) {
DPAA2_PMD_ERR("Add entry(%d) to table(%d) failed",
- qos_index, flow->tc_id);
+ flow->entry_index,
+ flow->flow_action.qos_action.action_jump_cfg.group);
return ret;
}
+ dpaa2_flow_entry_map_set(tbl_profile->entry_map, flow->entry_index, 1);
+ tbl_profile->entry_num++;
return 0;
}
static int
dpaa2_flow_add_fs_rule(struct dpaa2_dev_priv *priv,
- struct dpaa2_dev_flow *flow)
+ struct dpaa2_generic_flow *flow)
{
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
int ret;
struct fsl_mc_io *dpni = priv->hw;
- if (flow->tc_index >= priv->fs_entries) {
- DPAA2_PMD_ERR("FS table full(%d >= %d)",
- flow->tc_index, priv->fs_entries);
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+
+ tbl_profile = &priv->flow_profile.tc_profile[flow->tc_id];
+ if (dpaa2_flow_entry_map_get(tbl_profile->entry_map,
+ flow->entry_index)) {
+ DPAA2_PMD_ERR("FS[%d].entry[%d] has been occupied",
+ flow->tc_id, flow->entry_index);
return -EINVAL;
}
- dpaa2_flow_fs_entry_log("Start add", flow);
+ dpaa2_flow_fs_entry_log("Add", flow);
- ret = dpni_add_fs_entry(dpni, CMD_PRI_LOW,
- priv->token, flow->tc_id,
- flow->tc_index, &flow->fs_rule,
- &flow->fs_action_cfg);
+ if (dpaa2_dev->bus_info->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV) {
+ ret = dpni_add_fs_entry(dpni, CMD_PRI_LOW,
+ priv->token, flow->tc_id, flow->entry_index,
+ &flow->rule_cfg, &flow->flow_action.fs_action.fs_action_cfg);
+ } else {
+ ret = dpni_add_fs_entry_legacy(dpni, CMD_PRI_LOW,
+ priv->token, flow->tc_id, flow->entry_index,
+ &flow->rule_cfg, &flow->flow_action.fs_action.fs_action_cfg);
+ }
if (ret < 0) {
DPAA2_PMD_ERR("Add rule(%d) to FS table(%d) failed",
- flow->tc_index, flow->tc_id);
+ flow->entry_index, flow->tc_id);
return ret;
}
+ dpaa2_flow_entry_map_set(tbl_profile->entry_map, flow->entry_index, 1);
+ tbl_profile->entry_num++;
+
return 0;
}
static int
-dpaa2_flow_rule_insert_hole(struct dpaa2_dev_flow *flow,
- int offset, int size,
- enum dpaa2_flow_dist_type dist_type)
+dpaa2_flow_update_qos_rule_action(struct dpaa2_dev_priv *priv,
+ struct dpaa2_generic_flow *flow)
{
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- if (offset < flow->qos_rule_size) {
- memmove(flow->qos_key_addr + offset + size,
- flow->qos_key_addr + offset,
- flow->qos_rule_size - offset);
- memset(flow->qos_key_addr + offset,
- 0, size);
-
- memmove(flow->qos_mask_addr + offset + size,
- flow->qos_mask_addr + offset,
- flow->qos_rule_size - offset);
- memset(flow->qos_mask_addr + offset,
- 0, size);
- flow->qos_rule_size += size;
- } else {
- flow->qos_rule_size = offset + size;
- }
+ int ret;
+ struct fsl_mc_io *dpni = priv->hw;
+ uint8_t flags = DPNI_QOS_OPT_UPDATE_IF_EXISTS, flow_id = 0;
+
+ dpaa2_flow_qos_entry_log("Update action", flow);
+
+ if (flow->direct_queue) {
+ flags |= DPNI_QOS_OPT_SET_FLOW_ID;
+ flow_id = flow->flow_id;
}
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- if (offset < flow->fs_rule_size) {
- memmove(flow->fs_key_addr + offset + size,
- flow->fs_key_addr + offset,
- flow->fs_rule_size - offset);
- memset(flow->fs_key_addr + offset,
- 0, size);
-
- memmove(flow->fs_mask_addr + offset + size,
- flow->fs_mask_addr + offset,
- flow->fs_rule_size - offset);
- memset(flow->fs_mask_addr + offset,
- 0, size);
- flow->fs_rule_size += size;
- } else {
- flow->fs_rule_size = offset + size;
- }
+ /** This option doesn't support legacy operation.*/
+ ret = dpni_add_qos_entry(dpni, CMD_PRI_LOW,
+ priv->token, &flow->rule_cfg,
+ flow->flow_action.qos_action.action_jump_cfg.group,
+ flow->entry_index,
+ flags, flow_id);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("Update rule(%d) in QOS table failed(%d)",
+ flow->entry_index, ret);
+ return ret;
}
return 0;
}
static int
-dpaa2_flow_rule_add_all(struct dpaa2_dev_priv *priv,
- enum dpaa2_flow_dist_type dist_type,
- uint16_t entry_size, uint8_t tc_id)
+dpaa2_flow_update_fs_rule_action(struct dpaa2_dev_priv *priv,
+ struct dpaa2_generic_flow *flow)
{
- struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
int ret;
+ struct fsl_mc_io *dpni = priv->hw;
+ struct dpni_fs_action_cfg *cfg;
- while (curr) {
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- if (priv->num_rx_tc > 1 ||
- curr->action_type ==
- RTE_FLOW_ACTION_TYPE_RSS) {
- curr->qos_rule.key_size = entry_size;
- ret = dpaa2_flow_add_qos_rule(priv, curr);
- if (ret)
- return ret;
- }
- }
- if (dist_type & DPAA2_FLOW_FS_TYPE &&
- curr->tc_id == tc_id) {
- curr->fs_rule.key_size = entry_size;
- ret = dpaa2_flow_add_fs_rule(priv, curr);
- if (ret)
- return ret;
- }
- curr = LIST_NEXT(curr, next);
+ dpaa2_flow_fs_entry_log("Update action", flow);
+
+ cfg = &flow->flow_action.fs_action.fs_action_cfg;
+ cfg->options |= DPNI_FS_OPT_UPDATE_IF_EXISTS;
+ /** This option doesn't support legacy operation.*/
+ ret = dpni_add_fs_entry(dpni, CMD_PRI_LOW,
+ priv->token, flow->tc_id, flow->entry_index,
+ &flow->rule_cfg, cfg);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("Update rule(%d) in FS table(%d) failed",
+ flow->entry_index, flow->tc_id);
+ return ret;
}
return 0;
}
static int
-dpaa2_flow_qos_rule_insert_hole(struct dpaa2_dev_priv *priv,
+_dpaa2_flow_rule_insert_hole(struct dpaa2_generic_flow *flow,
int offset, int size)
{
- struct dpaa2_dev_flow *curr;
- int ret;
-
- curr = priv->curr;
- if (!curr) {
- DPAA2_PMD_ERR("Current qos flow insert hole failed.");
- return -EINVAL;
+ if (offset < flow->rule_size) {
+ memmove(flow->key_addr + offset + size,
+ flow->key_addr + offset,
+ flow->rule_size - offset);
+ memset(flow->key_addr + offset, 0, size);
+
+ memmove(flow->mask_addr + offset + size,
+ flow->mask_addr + offset,
+ flow->rule_size - offset);
+ memset(flow->mask_addr + offset, 0, size);
+ flow->rule_size += size;
} else {
- ret = dpaa2_flow_rule_insert_hole(curr, offset, size,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- }
-
- curr = LIST_FIRST(&priv->flows);
- while (curr) {
- ret = dpaa2_flow_rule_insert_hole(curr, offset, size,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- curr = LIST_NEXT(curr, next);
+ flow->rule_size = offset + size;
}
return 0;
}
static int
-dpaa2_flow_fs_rule_insert_hole(struct dpaa2_dev_priv *priv,
- int offset, int size, int tc_id)
+dpaa2_flow_rule_add_all(struct dpaa2_dev_priv *priv,
+ enum dpaa2_flow_dist_type dist_type,
+ uint16_t entry_size, uint8_t tc_id)
{
- struct dpaa2_dev_flow *curr;
+ struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
+ struct dpaa2_generic_flow *qos_flow;
+ struct dpaa2_generic_flow *fs_flow;
int ret;
- curr = priv->curr;
- if (!curr || curr->tc_id != tc_id) {
- DPAA2_PMD_ERR("Current flow insert hole failed.");
- return -EINVAL;
- } else {
- ret = dpaa2_flow_rule_insert_hole(curr, offset, size,
- DPAA2_FLOW_FS_TYPE);
- if (ret)
- return ret;
- }
-
- curr = LIST_FIRST(&priv->flows);
-
while (curr) {
- if (curr->tc_id != tc_id) {
- curr = LIST_NEXT(curr, next);
- continue;
+ qos_flow = curr->qos_flow;
+ fs_flow = curr->fs_flow;
+ if (qos_flow && dist_type == DPAA2_FLOW_QOS_TYPE) {
+ qos_flow->rule_cfg.key_size = entry_size;
+ ret = dpaa2_flow_add_qos_rule(priv, qos_flow);
+ if (ret)
+ return ret;
+ } else if (dist_type == DPAA2_FLOW_FS_TYPE &&
+ fs_flow && fs_flow->tc_id == tc_id) {
+ fs_flow->rule_cfg.key_size = entry_size;
+ ret = dpaa2_flow_add_fs_rule(priv, fs_flow);
+ if (ret)
+ return ret;
}
- ret = dpaa2_flow_rule_insert_hole(curr, offset, size,
- DPAA2_FLOW_FS_TYPE);
- if (ret)
- return ret;
curr = LIST_NEXT(curr, next);
}
return 0;
}
+static void
+dpaa2_flow_rule_insert_hole(struct dpaa2_dev_priv *priv,
+ int offset, int size, int tc_id,
+ enum dpaa2_flow_dist_type dist_type)
+{
+ struct dpaa2_dev_flow *flow;
+ struct dpaa2_generic_flow *curr;
+
+ curr = priv->cur_flow;
+ if (curr)
+ _dpaa2_flow_rule_insert_hole(curr, offset, size);
+
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ if (dist_type == DPAA2_FLOW_QOS_TYPE)
+ curr = flow->qos_flow;
+ else
+ curr = flow->fs_flow;
+ flow = LIST_NEXT(flow, next);
+ if (!curr)
+ continue;
+ if (dist_type == DPAA2_FLOW_FS_TYPE &&
+ curr->tc_id != tc_id)
+ continue;
+ if (curr->ip_src || curr->ip_dst)
+ _dpaa2_flow_rule_insert_hole(curr, offset, size);
+ }
+}
+
static int
dpaa2_flow_faf_advance(struct dpaa2_dev_priv *priv,
int faf_byte, enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int offset, ret;
struct dpaa2_key_profile *key_profile;
- int num, pos;
+ uint8_t idx, offset = 0xff;
+ struct key_prot_field prot = {0};
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_profile = &priv->extract.qos_key_extract.key_profile;
+ key_profile = &priv->flow_profile.qos_profile.key_profile;
else
- key_profile = &priv->extract.tc_key_extract[tc_id].key_profile;
-
- num = key_profile->num;
+ key_profile = &priv->flow_profile.tc_profile[tc_id].key_profile;
- if (num >= DPKG_MAX_NUM_OF_EXTRACTS) {
+ if (key_profile->num >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- if (key_profile->ip_addr_type != IP_NONE_ADDR_EXTRACT) {
- offset = key_profile->ip_addr_extract_off;
- pos = key_profile->ip_addr_extract_pos;
- key_profile->ip_addr_extract_pos++;
- key_profile->ip_addr_extract_off++;
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_flow_qos_rule_insert_hole(priv,
- offset, 1);
- } else {
- ret = dpaa2_flow_fs_rule_insert_hole(priv,
- offset, 1, tc_id);
- }
- if (ret)
- return ret;
- } else {
- pos = num;
- }
-
- if (pos > 0) {
- key_profile->key_offset[pos] =
- key_profile->key_offset[pos - 1] +
- key_profile->key_size[pos - 1];
- } else {
- key_profile->key_offset[pos] = 0;
- }
-
- key_profile->key_size[pos] = 1;
- key_profile->prot_field[pos].type = DPAA2_FAF_KEY;
- key_profile->prot_field[pos].key_field = faf_byte;
- key_profile->num++;
+ prot.type = DPAA2_FAF_KEY;
+ prot.key_field = faf_byte;
+ idx = dpaa2_profile_insert_no_ipaddr_extract(key_profile,
+ 1, &offset, insert_offset, &prot);
+ dpaa2_flow_rule_insert_hole(priv, offset, 1, tc_id, dist_type);
- if (insert_offset)
- *insert_offset = key_profile->key_offset[pos];
-
- key_profile->key_max_size++;
-
- return pos;
+ return idx;
}
static int
@@ -761,60 +990,27 @@ dpaa2_flow_pr_advance(struct dpaa2_dev_priv *priv,
enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int offset, ret;
struct dpaa2_key_profile *key_profile;
- int num, pos;
+ uint8_t idx, offset = 0xff;
+ struct key_prot_field prot = {0};
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_profile = &priv->extract.qos_key_extract.key_profile;
+ key_profile = &priv->flow_profile.qos_profile.key_profile;
else
- key_profile = &priv->extract.tc_key_extract[tc_id].key_profile;
-
- num = key_profile->num;
+ key_profile = &priv->flow_profile.tc_profile[tc_id].key_profile;
- if (num >= DPKG_MAX_NUM_OF_EXTRACTS) {
+ if (key_profile->num >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- if (key_profile->ip_addr_type != IP_NONE_ADDR_EXTRACT) {
- offset = key_profile->ip_addr_extract_off;
- pos = key_profile->ip_addr_extract_pos;
- key_profile->ip_addr_extract_pos++;
- key_profile->ip_addr_extract_off += pr_size;
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_flow_qos_rule_insert_hole(priv,
- offset, pr_size);
- } else {
- ret = dpaa2_flow_fs_rule_insert_hole(priv,
- offset, pr_size, tc_id);
- }
- if (ret)
- return ret;
- } else {
- pos = num;
- }
-
- if (pos > 0) {
- key_profile->key_offset[pos] =
- key_profile->key_offset[pos - 1] +
- key_profile->key_size[pos - 1];
- } else {
- key_profile->key_offset[pos] = 0;
- }
-
- key_profile->key_size[pos] = pr_size;
- key_profile->prot_field[pos].type = DPAA2_PR_KEY;
- key_profile->prot_field[pos].key_field =
- (pr_offset << 16) | pr_size;
- key_profile->num++;
-
- if (insert_offset)
- *insert_offset = key_profile->key_offset[pos];
+ prot.type = DPAA2_PR_KEY;
+ prot.key_field = (pr_offset << 16) | pr_size;
+ idx = dpaa2_profile_insert_no_ipaddr_extract(key_profile,
+ pr_size, &offset, insert_offset, &prot);
+ dpaa2_flow_rule_insert_hole(priv, offset, pr_size, tc_id, dist_type);
- key_profile->key_max_size += pr_size;
-
- return pos;
+ return idx;
}
/* Move IPv4/IPv6 addresses to fill new extract previous IP address.
@@ -829,9 +1025,9 @@ dpaa2_flow_key_profile_advance(enum net_prot prot,
enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int offset, ret;
struct dpaa2_key_profile *key_profile;
- int num, pos;
+ uint8_t idx, offset = 0xff;
+ struct key_prot_field prot_field;
if (dpaa2_flow_ip_address_extract(prot, field)) {
DPAA2_PMD_ERR("%s only for none IP address extract",
@@ -840,66 +1036,35 @@ dpaa2_flow_key_profile_advance(enum net_prot prot,
}
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_profile = &priv->extract.qos_key_extract.key_profile;
+ key_profile = &priv->flow_profile.qos_profile.key_profile;
else
- key_profile = &priv->extract.tc_key_extract[tc_id].key_profile;
-
- num = key_profile->num;
+ key_profile = &priv->flow_profile.tc_profile[tc_id].key_profile;
- if (num >= DPKG_MAX_NUM_OF_EXTRACTS) {
+ if (key_profile->num >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- if (key_profile->ip_addr_type != IP_NONE_ADDR_EXTRACT) {
- offset = key_profile->ip_addr_extract_off;
- pos = key_profile->ip_addr_extract_pos;
- key_profile->ip_addr_extract_pos++;
- key_profile->ip_addr_extract_off += field_size;
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_flow_qos_rule_insert_hole(priv,
- offset, field_size);
- } else {
- ret = dpaa2_flow_fs_rule_insert_hole(priv,
- offset, field_size, tc_id);
- }
- if (ret)
- return ret;
- } else {
- pos = num;
- }
-
- if (pos > 0) {
- key_profile->key_offset[pos] =
- key_profile->key_offset[pos - 1] +
- key_profile->key_size[pos - 1];
- } else {
- key_profile->key_offset[pos] = 0;
- }
-
- key_profile->key_size[pos] = field_size;
- key_profile->prot_field[pos].type = DPAA2_NET_PROT_KEY;
- key_profile->prot_field[pos].prot = prot;
- key_profile->prot_field[pos].key_field = field;
- key_profile->num++;
-
- if (insert_offset)
- *insert_offset = key_profile->key_offset[pos];
+ prot_field.type = DPAA2_NET_PROT_KEY;
+ prot_field.prot = prot;
+ prot_field.key_field = field;
+ idx = dpaa2_profile_insert_no_ipaddr_extract(key_profile,
+ field_size, &offset, insert_offset, &prot_field);
+ dpaa2_flow_rule_insert_hole(priv, offset, field_size, tc_id, dist_type);
if (dpaa2_flow_l4_src_port_extract(prot, field)) {
- key_profile->l4_src_port_present = 1;
- key_profile->l4_src_port_pos = pos;
- key_profile->l4_src_port_offset =
- key_profile->key_offset[pos];
+ key_profile->l4_sp_present = 1;
+ key_profile->l4_sp_extract_idx = idx;
+ key_profile->l4_sp_key_offset =
+ key_profile->key_offset[idx];
} else if (dpaa2_flow_l4_dst_port_extract(prot, field)) {
- key_profile->l4_dst_port_present = 1;
- key_profile->l4_dst_port_pos = pos;
- key_profile->l4_dst_port_offset =
- key_profile->key_offset[pos];
+ key_profile->l4_dp_present = 1;
+ key_profile->l4_dp_extract_idx = idx;
+ key_profile->l4_dp_key_offset =
+ key_profile->key_offset[idx];
}
- key_profile->key_max_size += field_size;
- return pos;
+ return idx;
}
static int
@@ -908,44 +1073,34 @@ dpaa2_flow_faf_add_hdr(int faf_byte,
enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int pos, i, offset;
- struct dpaa2_key_extract *key_extract;
+ int extract_idx;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpkg_profile_cfg *dpkg;
- struct dpkg_extract *extracts;
+ struct dpkg_extract extract;
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- dpkg = &key_extract->dpkg;
- extracts = dpkg->extracts;
+ dpkg = &tbl_profile->dpkg;
if (dpkg->num_extracts >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- pos = dpaa2_flow_faf_advance(priv,
+ extract_idx = dpaa2_flow_faf_advance(priv,
faf_byte, dist_type, tc_id,
insert_offset);
- if (pos < 0)
- return pos;
-
- if (pos != dpkg->num_extracts) {
- /* Not the last pos, must have IP address extract.*/
- for (i = dpkg->num_extracts - 1; i >= pos; i--) {
- extracts[i + 1] = extracts[i];
- }
- }
-
- offset = DPAA2_FAFE_PSR_OFFSET + faf_byte;
-
- extracts[pos].type = DPKG_EXTRACT_FROM_PARSE;
- extracts[pos].extract.from_parse.offset = offset;
- extracts[pos].extract.from_parse.size = 1;
+ if (extract_idx < 0)
+ return extract_idx;
- dpkg->num_extracts++;
+ memset(&extract, 0, sizeof(extract));
+ extract.type = DPKG_EXTRACT_FROM_PARSE;
+ extract.extract.from_parse.offset = faf_byte;
+ extract.extract.from_parse.size = 1;
+ dpaa2_dpkg_insert_extract(dpkg, extract_idx, &extract);
return 0;
}
@@ -956,48 +1111,40 @@ dpaa2_flow_pr_add_hdr(uint32_t pr_offset,
enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int pos, i;
- struct dpaa2_key_extract *key_extract;
+ int extract_idx;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpkg_profile_cfg *dpkg;
- struct dpkg_extract *extracts;
+ struct dpkg_extract extract;
- if ((pr_offset + pr_size) > DPAA2_FAPR_SIZE) {
+ if ((pr_offset + pr_size) > DPAA2_PSR_RESULT_SIZE) {
DPAA2_PMD_ERR("PR extracts(%d:%d) overflow",
pr_offset, pr_size);
return -EINVAL;
}
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- dpkg = &key_extract->dpkg;
- extracts = dpkg->extracts;
+ dpkg = &tbl_profile->dpkg;
if (dpkg->num_extracts >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- pos = dpaa2_flow_pr_advance(priv,
+ extract_idx = dpaa2_flow_pr_advance(priv,
pr_offset, pr_size, dist_type, tc_id,
insert_offset);
- if (pos < 0)
- return pos;
+ if (extract_idx < 0)
+ return extract_idx;
- if (pos != dpkg->num_extracts) {
- /* Not the last pos, must have IP address extract.*/
- for (i = dpkg->num_extracts - 1; i >= pos; i--) {
- extracts[i + 1] = extracts[i];
- }
- }
-
- extracts[pos].type = DPKG_EXTRACT_FROM_PARSE;
- extracts[pos].extract.from_parse.offset = pr_offset;
- extracts[pos].extract.from_parse.size = pr_size;
-
- dpkg->num_extracts++;
+ memset(&extract, 0, sizeof(extract));
+ extract.type = DPKG_EXTRACT_FROM_PARSE;
+ extract.extract.from_parse.offset = pr_offset;
+ extract.extract.from_parse.size = pr_size;
+ dpaa2_dpkg_insert_extract(dpkg, extract_idx, &extract);
return 0;
}
@@ -1009,18 +1156,17 @@ dpaa2_flow_extract_add_hdr(enum net_prot prot,
enum dpaa2_flow_dist_type dist_type, int tc_id,
int *insert_offset)
{
- int pos, i;
- struct dpaa2_key_extract *key_extract;
+ int extract_idx;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpkg_profile_cfg *dpkg;
- struct dpkg_extract *extracts;
+ struct dpkg_extract extract;
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- dpkg = &key_extract->dpkg;
- extracts = dpkg->extracts;
+ dpkg = &tbl_profile->dpkg;
if (dpaa2_flow_ip_address_extract(prot, field)) {
DPAA2_PMD_ERR("%s only for none IP address extract",
@@ -1033,52 +1179,86 @@ dpaa2_flow_extract_add_hdr(enum net_prot prot,
return -EINVAL;
}
- pos = dpaa2_flow_key_profile_advance(prot,
+ extract_idx = dpaa2_flow_key_profile_advance(prot,
field, field_size, priv,
dist_type, tc_id,
insert_offset);
- if (pos < 0)
- return pos;
+ if (extract_idx < 0)
+ return extract_idx;
- if (pos != dpkg->num_extracts) {
- /* Not the last pos, must have IP address extract.*/
- for (i = dpkg->num_extracts - 1; i >= pos; i--) {
- extracts[i + 1] = extracts[i];
- }
+ memset(&extract, 0, sizeof(extract));
+ extract.type = DPKG_EXTRACT_FROM_HDR;
+ extract.extract.from_hdr.prot = prot;
+ extract.extract.from_hdr.type = DPKG_FULL_FIELD;
+ extract.extract.from_hdr.field = field;
+ dpaa2_dpkg_insert_extract(dpkg, extract_idx, &extract);
+
+ return 0;
+}
+
+static inline int
+dpaa2_flow_extract_search(struct dpaa2_key_profile *key_profile,
+ enum key_prot_type type, enum net_prot prot, uint32_t key_field)
+{
+ int extract_idx;
+ struct key_prot_field *prot_field;
+
+ if (dpaa2_flow_ip_address_extract(prot, key_field)) {
+ DPAA2_PMD_ERR("%s only for none IP address extract",
+ __func__);
+ return -EINVAL;
}
- extracts[pos].type = DPKG_EXTRACT_FROM_HDR;
- extracts[pos].extract.from_hdr.prot = prot;
- extracts[pos].extract.from_hdr.type = DPKG_FULL_FIELD;
- extracts[pos].extract.from_hdr.field = field;
+ prot_field = key_profile->prot_field;
+ for (extract_idx = 0; extract_idx < key_profile->num; extract_idx++) {
+ if (type == DPAA2_NET_PROT_KEY &&
+ prot_field[extract_idx].prot == prot &&
+ prot_field[extract_idx].key_field == key_field &&
+ prot_field[extract_idx].type == type)
+ return extract_idx;
+ else if (type == DPAA2_FAF_KEY &&
+ prot_field[extract_idx].key_field == key_field &&
+ prot_field[extract_idx].type == type)
+ return extract_idx;
+ else if (type == DPAA2_PR_KEY &&
+ prot_field[extract_idx].key_field == key_field &&
+ prot_field[extract_idx].type == type)
+ return extract_idx;
+ }
- dpkg->num_extracts++;
+ if (type == DPAA2_NET_PROT_KEY &&
+ dpaa2_flow_l4_src_port_extract(prot, key_field)) {
+ if (key_profile->l4_sp_present)
+ return key_profile->l4_sp_extract_idx;
+ } else if (type == DPAA2_NET_PROT_KEY &&
+ dpaa2_flow_l4_dst_port_extract(prot, key_field)) {
+ if (key_profile->l4_dp_present)
+ return key_profile->l4_dp_extract_idx;
+ }
- return 0;
+ return -ENXIO;
}
static int
-dpaa2_flow_extract_new_raw(struct dpaa2_dev_priv *priv,
+_dpaa2_flow_extract_add_raw(struct dpaa2_dev_priv *priv,
int offset, int size,
enum dpaa2_flow_dist_type dist_type, int tc_id)
{
- struct dpaa2_key_extract *key_extract;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpkg_profile_cfg *dpkg;
struct dpaa2_key_profile *key_profile;
- int last_extract_size, index, pos, item_size;
+ int last_extract_size, index, raw_idx, item_size;
uint8_t num_extracts;
uint32_t field;
+ int ret;
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- dpkg = &key_extract->dpkg;
- key_profile = &key_extract->key_profile;
-
- key_profile->raw_region.raw_start = 0;
- key_profile->raw_region.raw_size = 0;
+ dpkg = &tbl_profile->dpkg;
+ key_profile = &tbl_profile->key_profile;
last_extract_size = (size % DPAA2_FLOW_MAX_KEY_SIZE);
num_extracts = (size / DPAA2_FLOW_MAX_KEY_SIZE);
@@ -1087,6 +1267,13 @@ dpaa2_flow_extract_new_raw(struct dpaa2_dev_priv *priv,
else
last_extract_size = DPAA2_FLOW_MAX_KEY_SIZE;
+ if ((key_profile->num + num_extracts) >
+ DPKG_MAX_NUM_OF_EXTRACTS) {
+ DPAA2_PMD_ERR("%s Failed to expand raw extracts",
+ __func__);
+ return -EINVAL;
+ }
+
for (index = 0; index < num_extracts; index++) {
if (index == num_extracts - 1)
item_size = last_extract_size;
@@ -1095,25 +1282,22 @@ dpaa2_flow_extract_new_raw(struct dpaa2_dev_priv *priv,
field = offset << DPAA2_FLOW_RAW_OFFSET_FIELD_SHIFT;
field |= item_size;
- pos = dpaa2_flow_key_profile_advance(NET_PROT_PAYLOAD,
+ ret = dpaa2_flow_extract_search(key_profile,
+ DPAA2_NET_PROT_KEY, NET_PROT_PAYLOAD, field);
+ if (ret >= 0) {
+ offset += item_size;
+ continue;
+ }
+
+ raw_idx = dpaa2_flow_key_profile_advance(NET_PROT_PAYLOAD,
field, item_size, priv, dist_type,
tc_id, NULL);
- if (pos < 0)
- return pos;
-
- dpkg->extracts[pos].type = DPKG_EXTRACT_FROM_DATA;
- dpkg->extracts[pos].extract.from_data.size = item_size;
- dpkg->extracts[pos].extract.from_data.offset = offset;
-
- if (index == 0) {
- key_profile->raw_extract_pos = pos;
- key_profile->raw_extract_off =
- key_profile->key_offset[pos];
- key_profile->raw_region.raw_start = offset;
- }
- key_profile->raw_extract_num++;
- key_profile->raw_region.raw_size +=
- key_profile->key_size[pos];
+ if (raw_idx < 0)
+ return raw_idx;
+
+ dpkg->extracts[raw_idx].type = DPKG_EXTRACT_FROM_DATA;
+ dpkg->extracts[raw_idx].extract.from_data.size = item_size;
+ dpkg->extracts[raw_idx].extract.from_data.offset = offset;
offset += item_size;
dpkg->num_extracts++;
@@ -1127,247 +1311,16 @@ dpaa2_flow_extract_add_raw(struct dpaa2_dev_priv *priv,
int offset, int size, enum dpaa2_flow_dist_type dist_type,
int tc_id, int *recfg)
{
- struct dpaa2_key_profile *key_profile;
- struct dpaa2_raw_region *raw_region;
- int end = offset + size, ret = 0, extract_extended, sz_extend;
- int start_cmp, end_cmp, new_size, index, pos, end_pos;
- int last_extract_size, item_size, num_extracts, bk_num = 0;
- struct dpkg_extract extract_bk[DPKG_MAX_NUM_OF_EXTRACTS];
- uint8_t key_offset_bk[DPKG_MAX_NUM_OF_EXTRACTS];
- uint8_t key_size_bk[DPKG_MAX_NUM_OF_EXTRACTS];
- struct key_prot_field prot_field_bk[DPKG_MAX_NUM_OF_EXTRACTS];
- struct dpaa2_raw_region raw_hole;
- struct dpkg_profile_cfg *dpkg;
- enum net_prot prot;
- uint32_t field;
-
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- key_profile = &priv->extract.qos_key_extract.key_profile;
- dpkg = &priv->extract.qos_key_extract.dpkg;
- } else {
- key_profile = &priv->extract.tc_key_extract[tc_id].key_profile;
- dpkg = &priv->extract.tc_key_extract[tc_id].dpkg;
- }
+ int ret;
- raw_region = &key_profile->raw_region;
- if (!raw_region->raw_size) {
- /* New RAW region*/
- ret = dpaa2_flow_extract_new_raw(priv, offset, size,
+ ret = _dpaa2_flow_extract_add_raw(priv, offset, size,
dist_type, tc_id);
- if (!ret && recfg)
- (*recfg) |= dist_type;
-
- return ret;
- }
- start_cmp = raw_region->raw_start;
- end_cmp = raw_region->raw_start + raw_region->raw_size;
-
- if (offset >= start_cmp && end <= end_cmp)
- return 0;
-
- sz_extend = 0;
- new_size = raw_region->raw_size;
- if (offset < start_cmp) {
- sz_extend += start_cmp - offset;
- new_size += (start_cmp - offset);
- }
- if (end > end_cmp) {
- sz_extend += end - end_cmp;
- new_size += (end - end_cmp);
- }
-
- last_extract_size = (new_size % DPAA2_FLOW_MAX_KEY_SIZE);
- num_extracts = (new_size / DPAA2_FLOW_MAX_KEY_SIZE);
- if (last_extract_size)
- num_extracts++;
- else
- last_extract_size = DPAA2_FLOW_MAX_KEY_SIZE;
-
- if ((key_profile->num + num_extracts -
- key_profile->raw_extract_num) >=
- DPKG_MAX_NUM_OF_EXTRACTS) {
- DPAA2_PMD_ERR("%s Failed to expand raw extracts",
- __func__);
- return -EINVAL;
- }
-
- if (offset < start_cmp) {
- raw_hole.raw_start = key_profile->raw_extract_off;
- raw_hole.raw_size = start_cmp - offset;
- raw_region->raw_start = offset;
- raw_region->raw_size += start_cmp - offset;
-
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_flow_qos_rule_insert_hole(priv,
- raw_hole.raw_start,
- raw_hole.raw_size);
- if (ret)
- return ret;
- }
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- ret = dpaa2_flow_fs_rule_insert_hole(priv,
- raw_hole.raw_start,
- raw_hole.raw_size, tc_id);
- if (ret)
- return ret;
- }
- }
-
- if (end > end_cmp) {
- raw_hole.raw_start =
- key_profile->raw_extract_off +
- raw_region->raw_size;
- raw_hole.raw_size = end - end_cmp;
- raw_region->raw_size += end - end_cmp;
-
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_flow_qos_rule_insert_hole(priv,
- raw_hole.raw_start,
- raw_hole.raw_size);
- if (ret)
- return ret;
- }
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- ret = dpaa2_flow_fs_rule_insert_hole(priv,
- raw_hole.raw_start,
- raw_hole.raw_size, tc_id);
- if (ret)
- return ret;
- }
- }
-
- end_pos = key_profile->raw_extract_pos +
- key_profile->raw_extract_num;
- if (key_profile->num > end_pos) {
- bk_num = key_profile->num - end_pos;
- memcpy(extract_bk, &dpkg->extracts[end_pos],
- bk_num * sizeof(struct dpkg_extract));
- memcpy(key_offset_bk, &key_profile->key_offset[end_pos],
- bk_num * sizeof(uint8_t));
- memcpy(key_size_bk, &key_profile->key_size[end_pos],
- bk_num * sizeof(uint8_t));
- memcpy(prot_field_bk, &key_profile->prot_field[end_pos],
- bk_num * sizeof(struct key_prot_field));
-
- for (index = 0; index < bk_num; index++) {
- key_offset_bk[index] += sz_extend;
- prot = prot_field_bk[index].prot;
- field = prot_field_bk[index].key_field;
- if (dpaa2_flow_l4_src_port_extract(prot,
- field)) {
- key_profile->l4_src_port_present = 1;
- key_profile->l4_src_port_pos = end_pos + index;
- key_profile->l4_src_port_offset =
- key_offset_bk[index];
- } else if (dpaa2_flow_l4_dst_port_extract(prot,
- field)) {
- key_profile->l4_dst_port_present = 1;
- key_profile->l4_dst_port_pos = end_pos + index;
- key_profile->l4_dst_port_offset =
- key_offset_bk[index];
- }
- }
- }
-
- pos = key_profile->raw_extract_pos;
-
- for (index = 0; index < num_extracts; index++) {
- if (index == num_extracts - 1)
- item_size = last_extract_size;
- else
- item_size = DPAA2_FLOW_MAX_KEY_SIZE;
- field = offset << DPAA2_FLOW_RAW_OFFSET_FIELD_SHIFT;
- field |= item_size;
-
- if (pos > 0) {
- key_profile->key_offset[pos] =
- key_profile->key_offset[pos - 1] +
- key_profile->key_size[pos - 1];
- } else {
- key_profile->key_offset[pos] = 0;
- }
- key_profile->key_size[pos] = item_size;
- key_profile->prot_field[pos].type = DPAA2_NET_PROT_KEY;
- key_profile->prot_field[pos].prot = NET_PROT_PAYLOAD;
- key_profile->prot_field[pos].key_field = field;
-
- dpkg->extracts[pos].type = DPKG_EXTRACT_FROM_DATA;
- dpkg->extracts[pos].extract.from_data.size = item_size;
- dpkg->extracts[pos].extract.from_data.offset = offset;
- offset += item_size;
- pos++;
- }
-
- if (bk_num) {
- memcpy(&dpkg->extracts[pos], extract_bk,
- bk_num * sizeof(struct dpkg_extract));
- memcpy(&key_profile->key_offset[end_pos],
- key_offset_bk, bk_num * sizeof(uint8_t));
- memcpy(&key_profile->key_size[end_pos],
- key_size_bk, bk_num * sizeof(uint8_t));
- memcpy(&key_profile->prot_field[end_pos],
- prot_field_bk, bk_num * sizeof(struct key_prot_field));
- }
-
- extract_extended = num_extracts - key_profile->raw_extract_num;
- if (key_profile->ip_addr_type != IP_NONE_ADDR_EXTRACT) {
- key_profile->ip_addr_extract_pos += extract_extended;
- key_profile->ip_addr_extract_off += sz_extend;
- }
- key_profile->raw_extract_num = num_extracts;
- key_profile->num += extract_extended;
- key_profile->key_max_size += sz_extend;
-
- dpkg->num_extracts += extract_extended;
if (!ret && recfg)
- (*recfg) |= dist_type;
+ (*recfg) |= true;
return ret;
}
-static inline int
-dpaa2_flow_extract_search(struct dpaa2_key_profile *key_profile,
- enum key_prot_type type, enum net_prot prot, uint32_t key_field)
-{
- int pos;
- struct key_prot_field *prot_field;
-
- if (dpaa2_flow_ip_address_extract(prot, key_field)) {
- DPAA2_PMD_ERR("%s only for none IP address extract",
- __func__);
- return -EINVAL;
- }
-
- prot_field = key_profile->prot_field;
- for (pos = 0; pos < key_profile->num; pos++) {
- if (type == DPAA2_NET_PROT_KEY &&
- prot_field[pos].prot == prot &&
- prot_field[pos].key_field == key_field &&
- prot_field[pos].type == type)
- return pos;
- else if (type == DPAA2_FAF_KEY &&
- prot_field[pos].key_field == key_field &&
- prot_field[pos].type == type)
- return pos;
- else if (type == DPAA2_PR_KEY &&
- prot_field[pos].key_field == key_field &&
- prot_field[pos].type == type)
- return pos;
- }
-
- if (type == DPAA2_NET_PROT_KEY &&
- dpaa2_flow_l4_src_port_extract(prot, key_field)) {
- if (key_profile->l4_src_port_present)
- return key_profile->l4_src_port_pos;
- } else if (type == DPAA2_NET_PROT_KEY &&
- dpaa2_flow_l4_dst_port_extract(prot, key_field)) {
- if (key_profile->l4_dst_port_present)
- return key_profile->l4_dst_port_pos;
- }
-
- return -ENXIO;
-}
-
static inline int
dpaa2_flow_extract_key_offset(struct dpaa2_key_profile *key_profile,
enum key_prot_type type, enum net_prot prot, uint32_t key_field)
@@ -1383,71 +1336,45 @@ dpaa2_flow_extract_key_offset(struct dpaa2_key_profile *key_profile,
static int
dpaa2_flow_faf_add_rule(struct dpaa2_dev_priv *priv,
- struct dpaa2_dev_flow *flow,
- enum dpaa2_rx_faf_offset faf_bit_off,
- int group,
- enum dpaa2_flow_dist_type dist_type)
+ struct dpaa2_generic_flow *flow, uint32_t faf_bit_off,
+ int group, enum dpaa2_flow_dist_type dist_type)
{
int offset;
uint8_t *key_addr;
uint8_t *mask_addr;
- struct dpaa2_key_extract *key_extract;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_key_profile *key_profile;
uint8_t faf_byte = faf_bit_off / 8;
uint8_t faf_bit_in_byte = faf_bit_off % 8;
faf_bit_in_byte = 7 - faf_bit_in_byte;
-
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- key_extract = &priv->extract.qos_key_extract;
- key_profile = &key_extract->key_profile;
-
- offset = dpaa2_flow_extract_key_offset(key_profile,
- DPAA2_FAF_KEY, NET_PROT_NONE, faf_byte);
- if (offset < 0) {
- DPAA2_PMD_ERR("%s QoS key extract failed", __func__);
- return -EINVAL;
- }
- key_addr = flow->qos_key_addr + offset;
- mask_addr = flow->qos_mask_addr + offset;
-
- if (!(*key_addr) &&
- key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT &&
- offset >= flow->qos_rule_size)
- flow->qos_rule_size = offset + sizeof(uint8_t);
-
- *key_addr |= (1 << faf_bit_in_byte);
- *mask_addr |= (1 << faf_bit_in_byte);
+ if (dist_type == DPAA2_FLOW_QOS_TYPE)
+ tbl_profile = &priv->flow_profile.qos_profile;
+ else
+ tbl_profile = &priv->flow_profile.tc_profile[group];
+ key_profile = &tbl_profile->key_profile;
+ offset = dpaa2_flow_extract_key_offset(key_profile,
+ DPAA2_FAF_KEY, NET_PROT_NONE, faf_byte);
+ if (offset < 0) {
+ DPAA2_PMD_ERR("%s QoS key extract failed", __func__);
+ return -EINVAL;
}
+ key_addr = flow->key_addr + offset;
+ mask_addr = flow->mask_addr + offset;
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- key_extract = &priv->extract.tc_key_extract[group];
- key_profile = &key_extract->key_profile;
-
- offset = dpaa2_flow_extract_key_offset(key_profile,
- DPAA2_FAF_KEY, NET_PROT_NONE, faf_byte);
- if (offset < 0) {
- DPAA2_PMD_ERR("%s TC[%d] key extract failed",
- __func__, group);
- return -EINVAL;
- }
- key_addr = flow->fs_key_addr + offset;
- mask_addr = flow->fs_mask_addr + offset;
-
- if (!(*key_addr) &&
- key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT &&
- offset >= flow->fs_rule_size)
- flow->fs_rule_size = offset + sizeof(uint8_t);
+ if (!(*key_addr) &&
+ !(flow->ip_src || flow->ip_dst) &&
+ offset >= flow->rule_size)
+ flow->rule_size = offset + sizeof(uint8_t);
- *key_addr |= (1 << faf_bit_in_byte);
- *mask_addr |= (1 << faf_bit_in_byte);
- }
+ *key_addr |= (1 << faf_bit_in_byte);
+ *mask_addr |= (1 << faf_bit_in_byte);
return 0;
}
static inline int
-dpaa2_flow_pr_rule_data_set(struct dpaa2_dev_flow *flow,
+dpaa2_flow_pr_rule_data_set(struct dpaa2_generic_flow *flow,
struct dpaa2_key_profile *key_profile,
uint32_t pr_offset, uint32_t pr_size,
const void *key, const void *mask,
@@ -1467,39 +1394,20 @@ dpaa2_flow_pr_rule_data_set(struct dpaa2_dev_flow *flow,
sprintf(offset_info, "offset(%d)", offset);
sprintf(size_info, "size(%d)", pr_size);
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- sprintf(rule_size_info, "qos rule size(%d)",
- flow->qos_rule_size);
- memcpy((flow->qos_key_addr + offset), key, pr_size);
- memcpy((flow->qos_mask_addr + offset), mask, pr_size);
- if (key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT) {
- if (offset >= flow->qos_rule_size) {
- flow->qos_rule_size = offset + pr_size;
- } else if ((offset + pr_size) > flow->qos_rule_size) {
- DPAA2_PMD_ERR("%s < %s, but %s + %s > %s",
- offset_info, rule_size_info,
- offset_info, size_info,
- rule_size_info);
- return -EINVAL;
- }
- }
- }
-
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- sprintf(rule_size_info, "fs rule size(%d)",
- flow->fs_rule_size);
- memcpy((flow->fs_key_addr + offset), key, pr_size);
- memcpy((flow->fs_mask_addr + offset), mask, pr_size);
- if (key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT) {
- if (offset >= flow->fs_rule_size) {
- flow->fs_rule_size = offset + pr_size;
- } else if ((offset + pr_size) > flow->fs_rule_size) {
- DPAA2_PMD_ERR("%s < %s, but %s + %s > %s",
- offset_info, rule_size_info,
- offset_info, size_info,
- rule_size_info);
- return -EINVAL;
- }
+ sprintf(rule_size_info, "%s rule size(%d)",
+ dist_type == DPAA2_FLOW_QOS_TYPE ?
+ "QoS" : "FS", flow->rule_size);
+ memcpy((flow->key_addr + offset), key, pr_size);
+ memcpy((flow->mask_addr + offset), mask, pr_size);
+ if (!(flow->ip_src || flow->ip_dst)) {
+ if (offset >= flow->rule_size) {
+ flow->rule_size = offset + pr_size;
+ } else if ((offset + pr_size) > flow->rule_size) {
+ DPAA2_PMD_ERR("%s < %s, but %s + %s > %s",
+ offset_info, rule_size_info,
+ offset_info, size_info,
+ rule_size_info);
+ return -EINVAL;
}
}
@@ -1507,13 +1415,13 @@ dpaa2_flow_pr_rule_data_set(struct dpaa2_dev_flow *flow,
}
static inline int
-dpaa2_flow_hdr_rule_data_set(struct dpaa2_dev_flow *flow,
+dpaa2_flow_hdr_rule_data_set(struct dpaa2_generic_flow *flow,
struct dpaa2_key_profile *key_profile,
enum net_prot prot, uint32_t field, int size,
const void *key, const void *mask,
enum dpaa2_flow_dist_type dist_type)
{
- int offset;
+ int offset, i;
char offset_info[64], size_info[64], rule_size_info[64];
if (dpaa2_flow_ip_address_extract(prot, field)) {
@@ -1532,37 +1440,22 @@ dpaa2_flow_hdr_rule_data_set(struct dpaa2_dev_flow *flow,
sprintf(offset_info, "offset(%d)", offset);
sprintf(size_info, "size(%d)", size);
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- sprintf(rule_size_info, "qos rule size(%d)",
- flow->qos_rule_size);
- memcpy((flow->qos_key_addr + offset), key, size);
- memcpy((flow->qos_mask_addr + offset), mask, size);
- if (key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT) {
- if (offset >= flow->qos_rule_size) {
- flow->qos_rule_size = offset + size;
- } else if ((offset + size) > flow->qos_rule_size) {
- DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
- __func__, offset_info, rule_size_info,
- offset_info, size_info, rule_size_info);
- return -EINVAL;
- }
- }
- }
+ sprintf(rule_size_info, "%s rule size(%d)",
+ dist_type == DPAA2_FLOW_QOS_TYPE ? "QoS" : "FS",
+ flow->rule_size);
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- sprintf(rule_size_info, "fs rule size(%d)",
- flow->fs_rule_size);
- memcpy((flow->fs_key_addr + offset), key, size);
- memcpy((flow->fs_mask_addr + offset), mask, size);
- if (key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT) {
- if (offset >= flow->fs_rule_size) {
- flow->fs_rule_size = offset + size;
- } else if ((offset + size) > flow->fs_rule_size) {
- DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
- __func__, offset_info, rule_size_info,
- offset_info, size_info, rule_size_info);
- return -EINVAL;
- }
+ for (i = 0; i < size; i++) {
+ *(flow->key_addr + offset + i) |= *((const uint8_t *)key + i);
+ *(flow->mask_addr + offset + i) |= *((const uint8_t *)mask + i);
+ }
+ if (!(flow->ip_src || flow->ip_dst)) {
+ if (offset >= flow->rule_size) {
+ flow->rule_size = offset + size;
+ } else if ((offset + size) > flow->rule_size) {
+ DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
+ __func__, offset_info, rule_size_info,
+ offset_info, size_info, rule_size_info);
+ return -EINVAL;
}
}
@@ -1570,7 +1463,7 @@ dpaa2_flow_hdr_rule_data_set(struct dpaa2_dev_flow *flow,
}
static inline int
-dpaa2_flow_raw_rule_data_set(struct dpaa2_dev_flow *flow,
+dpaa2_flow_raw_rule_data_set(struct dpaa2_generic_flow *flow,
struct dpaa2_key_profile *key_profile,
uint32_t extract_offset, int size,
const void *key, const void *mask,
@@ -1593,34 +1486,19 @@ dpaa2_flow_raw_rule_data_set(struct dpaa2_dev_flow *flow,
sprintf(offset_info, "offset(%d)", offset);
sprintf(size_info, "size(%d)", size);
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- sprintf(rule_size_info, "qos rule size(%d)",
- flow->qos_rule_size);
- memcpy((flow->qos_key_addr + offset), key, size);
- memcpy((flow->qos_mask_addr + offset), mask, size);
- if (offset >= flow->qos_rule_size) {
- flow->qos_rule_size = offset + size;
- } else if ((offset + size) > flow->qos_rule_size) {
- DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
- __func__, offset_info, rule_size_info,
- offset_info, size_info, rule_size_info);
- return -EINVAL;
- }
- }
-
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- sprintf(rule_size_info, "fs rule size(%d)",
- flow->fs_rule_size);
- memcpy((flow->fs_key_addr + offset), key, size);
- memcpy((flow->fs_mask_addr + offset), mask, size);
- if (offset >= flow->fs_rule_size) {
- flow->fs_rule_size = offset + size;
- } else if ((offset + size) > flow->fs_rule_size) {
- DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
- __func__, offset_info, rule_size_info,
- offset_info, size_info, rule_size_info);
- return -EINVAL;
- }
+ sprintf(rule_size_info, "%s rule size(%d)",
+ dist_type == DPAA2_FLOW_QOS_TYPE ?
+ "QoS" : "FS", flow->rule_size);
+
+ memcpy((flow->key_addr + offset), key, size);
+ memcpy((flow->mask_addr + offset), mask, size);
+ if (offset >= flow->rule_size) {
+ flow->rule_size = offset + size;
+ } else if ((offset + size) > flow->rule_size) {
+ DPAA2_PMD_ERR("%s: %s < %s, but %s + %s > %s",
+ __func__, offset_info, rule_size_info,
+ offset_info, size_info, rule_size_info);
+ return -EINVAL;
}
return 0;
@@ -1691,13 +1569,9 @@ dpaa2_flow_extract_support(const uint8_t *mask_src,
mask_support = (const char *)&dpaa2_flow_item_gtp_mask;
size = sizeof(struct rte_flow_item_gtp);
break;
- case RTE_FLOW_ITEM_TYPE_MPLS:
- mask_support = (const char *)&dpaa2_flow_item_mpls_mask;
- size = sizeof(struct rte_flow_item_mpls);
- break;
- case RTE_FLOW_ITEM_TYPE_PPPOES:
- mask_support = (const char *)&dpaa2_flow_item_pppoe_mask;
- size = sizeof(struct rte_flow_item_pppoe);
+ case RTE_FLOW_ITEM_TYPE_GENEVE:
+ mask_support = (const char *)&dpaa2_flow_item_geneve_mask;
+ size = sizeof(struct rte_flow_item_geneve);
break;
default:
return -EINVAL;
@@ -1716,19 +1590,17 @@ dpaa2_flow_extract_support(const uint8_t *mask_src,
static int
dpaa2_flow_identify_by_faf(struct dpaa2_dev_priv *priv,
- struct dpaa2_dev_flow *flow,
- enum dpaa2_rx_faf_offset faf_off,
- enum dpaa2_flow_dist_type dist_type,
- int group, int *recfg)
+ struct dpaa2_generic_flow *flow, uint32_t faf_bit_off,
+ enum dpaa2_flow_dist_type dist_type, int group, int *recfg)
{
- int ret, index, local_cfg = 0;
- struct dpaa2_key_extract *extract;
+ int ret, index, local_cfg = false;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_key_profile *key_profile;
- uint8_t faf_byte = faf_off / 8;
+ uint8_t faf_byte = faf_bit_off / 8;
- if (dist_type & DPAA2_FLOW_QOS_TYPE) {
- extract = &priv->extract.qos_key_extract;
- key_profile = &extract->key_profile;
+ if (dist_type == DPAA2_FLOW_QOS_TYPE) {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ key_profile = &tbl_profile->key_profile;
index = dpaa2_flow_extract_search(key_profile,
DPAA2_FAF_KEY, NET_PROT_NONE, faf_byte);
@@ -1741,20 +1613,18 @@ dpaa2_flow_identify_by_faf(struct dpaa2_dev_priv *priv,
return -EINVAL;
}
- local_cfg |= DPAA2_FLOW_QOS_TYPE;
+ local_cfg = true;
}
- ret = dpaa2_flow_faf_add_rule(priv, flow, faf_off, group,
+ ret = dpaa2_flow_faf_add_rule(priv, flow, faf_bit_off, group,
DPAA2_FLOW_QOS_TYPE);
if (ret) {
DPAA2_PMD_ERR("QoS faf rule set failed");
- return -EINVAL;
+ return ret;
}
- }
-
- if (dist_type & DPAA2_FLOW_FS_TYPE) {
- extract = &priv->extract.tc_key_extract[group];
- key_profile = &extract->key_profile;
+ } else if (dist_type == DPAA2_FLOW_FS_TYPE) {
+ tbl_profile = &priv->flow_profile.tc_profile[group];
+ key_profile = &tbl_profile->key_profile;
index = dpaa2_flow_extract_search(key_profile,
DPAA2_FAF_KEY, NET_PROT_NONE, faf_byte);
@@ -1768,10 +1638,10 @@ dpaa2_flow_identify_by_faf(struct dpaa2_dev_priv *priv,
return -EINVAL;
}
- local_cfg |= DPAA2_FLOW_FS_TYPE;
+ local_cfg = true;
}
- ret = dpaa2_flow_faf_add_rule(priv, flow, faf_off, group,
+ ret = dpaa2_flow_faf_add_rule(priv, flow, faf_bit_off, group,
DPAA2_FLOW_FS_TYPE);
if (ret) {
DPAA2_PMD_ERR("FS[%d] faf rule set failed",
@@ -1781,29 +1651,29 @@ dpaa2_flow_identify_by_faf(struct dpaa2_dev_priv *priv,
}
if (recfg)
- *recfg |= local_cfg;
+ (*recfg) |= local_cfg;
return 0;
}
static int
-dpaa2_flow_add_pr_extract_rule(struct dpaa2_dev_flow *flow,
+dpaa2_flow_add_pr_extract_rule(struct dpaa2_generic_flow *flow,
uint32_t pr_offset, uint32_t pr_size,
const void *key, const void *mask,
struct dpaa2_dev_priv *priv, int tc_id, int *recfg,
enum dpaa2_flow_dist_type dist_type)
{
- int index, ret, local_cfg = 0;
- struct dpaa2_key_extract *key_extract;
+ int index, ret, local_cfg = false;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_key_profile *key_profile;
uint32_t pr_field = pr_offset << 16 | pr_size;
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- key_profile = &key_extract->key_profile;
+ key_profile = &tbl_profile->key_profile;
index = dpaa2_flow_extract_search(key_profile,
DPAA2_PR_KEY, NET_PROT_NONE, pr_field);
@@ -1817,7 +1687,7 @@ dpaa2_flow_add_pr_extract_rule(struct dpaa2_dev_flow *flow,
return ret;
}
- local_cfg |= dist_type;
+ local_cfg = true;
}
ret = dpaa2_flow_pr_rule_data_set(flow, key_profile,
@@ -1830,31 +1700,31 @@ dpaa2_flow_add_pr_extract_rule(struct dpaa2_dev_flow *flow,
}
if (recfg)
- *recfg |= local_cfg;
+ (*recfg) |= local_cfg;
return 0;
}
static int
-dpaa2_flow_add_hdr_extract_rule(struct dpaa2_dev_flow *flow,
+dpaa2_flow_add_hdr_extract_rule(struct dpaa2_generic_flow *flow,
enum net_prot prot, uint32_t field,
const void *key, const void *mask, int size,
struct dpaa2_dev_priv *priv, int tc_id, int *recfg,
enum dpaa2_flow_dist_type dist_type)
{
- int index, ret, local_cfg = 0;
- struct dpaa2_key_extract *key_extract;
+ int index, ret, local_cfg = false;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_key_profile *key_profile;
if (dpaa2_flow_ip_address_extract(prot, field))
return -EINVAL;
if (dist_type == DPAA2_FLOW_QOS_TYPE)
- key_extract = &priv->extract.qos_key_extract;
+ tbl_profile = &priv->flow_profile.qos_profile;
else
- key_extract = &priv->extract.tc_key_extract[tc_id];
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
- key_profile = &key_extract->key_profile;
+ key_profile = &tbl_profile->key_profile;
index = dpaa2_flow_extract_search(key_profile,
DPAA2_NET_PROT_KEY, prot, field);
@@ -1868,7 +1738,7 @@ dpaa2_flow_add_hdr_extract_rule(struct dpaa2_dev_flow *flow,
return ret;
}
- local_cfg |= dist_type;
+ local_cfg = true;
}
ret = dpaa2_flow_hdr_rule_data_set(flow, key_profile,
@@ -1881,44 +1751,57 @@ dpaa2_flow_add_hdr_extract_rule(struct dpaa2_dev_flow *flow,
}
if (recfg)
- *recfg |= local_cfg;
+ (*recfg) |= local_cfg;
return 0;
}
static int
-dpaa2_flow_add_ipaddr_extract_rule(struct dpaa2_dev_flow *flow,
+dpaa2_flow_add_ipaddr_extract_rule(struct dpaa2_generic_flow *flow,
enum net_prot prot, uint32_t field,
const void *key, const void *mask, int size,
struct dpaa2_dev_priv *priv, int tc_id, int *recfg,
enum dpaa2_flow_dist_type dist_type)
{
- int local_cfg = 0, num, ipaddr_extract_len = 0;
- struct dpaa2_key_extract *key_extract;
+ int local_cfg = false, update = 0, ret, pos = 0;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_key_profile *key_profile;
struct dpkg_profile_cfg *dpkg;
- uint8_t *key_addr, *mask_addr;
- union ip_addr_extract_rule *ip_addr_data;
- union ip_addr_extract_rule *ip_addr_mask;
- enum net_prot orig_prot;
- uint32_t orig_field;
-
- if (prot != NET_PROT_IPV4 && prot != NET_PROT_IPV6)
- return -EINVAL;
+ uint8_t *key_addr, *mask_addr, num;
+ uint8_t ip_addr_offset = 0;
- if (prot == NET_PROT_IPV4 && field != NH_FLD_IPV4_SRC_IP &&
- field != NH_FLD_IPV4_DST_IP) {
+ if (prot != NET_PROT_IPV4 && prot != NET_PROT_IPV6) {
+ DPAA2_PMD_ERR("%s: Invalid protocol(%d)",
+ __func__, prot);
return -EINVAL;
}
- if (prot == NET_PROT_IPV6 && field != NH_FLD_IPV6_SRC_IP &&
- field != NH_FLD_IPV6_DST_IP) {
- return -EINVAL;
+ if (prot == NET_PROT_IPV4) {
+ if (field != NH_FLD_IPV4_SRC_IP &&
+ field != NH_FLD_IPV4_DST_IP) {
+ DPAA2_PMD_ERR("%s: Invalid ipv4 filed(%d)",
+ __func__, field);
+ return -EINVAL;
+ }
+ if (size != sizeof(rte_be32_t)) {
+ DPAA2_PMD_ERR("%s: Invalid ipv4 address size(%d)",
+ __func__, size);
+ return -EINVAL;
+ }
+ } else {
+ if (field != NH_FLD_IPV6_SRC_IP &&
+ field != NH_FLD_IPV6_DST_IP) {
+ DPAA2_PMD_ERR("%s: Invalid ipv6 filed(%d)",
+ __func__, field);
+ return -EINVAL;
+ }
+ if (size != NH_FLD_IPV6_ADDR_SIZE) {
+ DPAA2_PMD_ERR("%s: Invalid ipv6 address size(%d)",
+ __func__, size);
+ return -EINVAL;
+ }
}
- orig_prot = prot;
- orig_field = field;
-
if (prot == NET_PROT_IPV4 &&
field == NH_FLD_IPV4_SRC_IP) {
prot = NET_PROT_IP;
@@ -1941,182 +1824,111 @@ dpaa2_flow_add_ipaddr_extract_rule(struct dpaa2_dev_flow *flow,
return -EINVAL;
}
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- key_extract = &priv->extract.qos_key_extract;
- key_profile = &key_extract->key_profile;
- dpkg = &key_extract->dpkg;
- num = key_profile->num;
- key_addr = flow->qos_key_addr;
- mask_addr = flow->qos_mask_addr;
- } else {
- key_extract = &priv->extract.tc_key_extract[tc_id];
- key_profile = &key_extract->key_profile;
- dpkg = &key_extract->dpkg;
- num = key_profile->num;
- key_addr = flow->fs_key_addr;
- mask_addr = flow->fs_mask_addr;
- }
+ if (dist_type == DPAA2_FLOW_QOS_TYPE)
+ tbl_profile = &priv->flow_profile.qos_profile;
+ else
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+
+ key_profile = &tbl_profile->key_profile;
+ dpkg = &tbl_profile->dpkg;
+ num = key_profile->num;
+ key_addr = flow->key_addr;
+ mask_addr = flow->mask_addr;
if (num >= DPKG_MAX_NUM_OF_EXTRACTS) {
DPAA2_PMD_ERR("Number of extracts overflows");
return -EINVAL;
}
- if (key_profile->ip_addr_type == IP_NONE_ADDR_EXTRACT) {
- if (field == NH_FLD_IP_SRC)
- key_profile->ip_addr_type = IP_SRC_EXTRACT;
- else
- key_profile->ip_addr_type = IP_DST_EXTRACT;
- ipaddr_extract_len = size;
-
- key_profile->ip_addr_extract_pos = num;
- if (num > 0) {
- key_profile->ip_addr_extract_off =
- key_profile->key_offset[num - 1] +
- key_profile->key_size[num - 1];
- } else {
- key_profile->ip_addr_extract_off = 0;
- }
- key_profile->key_max_size += NH_FLD_IPV6_ADDR_SIZE;
- } else if (key_profile->ip_addr_type == IP_SRC_EXTRACT) {
- if (field == NH_FLD_IP_SRC) {
- ipaddr_extract_len = size;
- goto rule_configure;
- }
- key_profile->ip_addr_type = IP_SRC_DST_EXTRACT;
- ipaddr_extract_len = size * 2;
- key_profile->key_max_size += NH_FLD_IPV6_ADDR_SIZE;
- } else if (key_profile->ip_addr_type == IP_DST_EXTRACT) {
- if (field == NH_FLD_IP_DST) {
- ipaddr_extract_len = size;
- goto rule_configure;
- }
- key_profile->ip_addr_type = IP_DST_SRC_EXTRACT;
- ipaddr_extract_len = size * 2;
- key_profile->key_max_size += NH_FLD_IPV6_ADDR_SIZE;
+ pos = dpaa2_extract_prev_ip_addr_pos(key_profile);
+ if (pos >= 0) {
+ ip_addr_offset = key_profile->key_offset[pos] +
+ key_profile->key_size[pos];
}
- key_profile->num++;
- key_profile->prot_field[num].type = DPAA2_NET_PROT_KEY;
- dpkg->extracts[num].extract.from_hdr.prot = prot;
- dpkg->extracts[num].extract.from_hdr.field = field;
- dpkg->extracts[num].extract.from_hdr.type = DPKG_FULL_FIELD;
- dpkg->num_extracts++;
+ ret = dpaa2_extract_ip_addr_add(field,
+ key_profile, size, &update, &pos);
+ if (ret) {
+ DPAA2_PMD_ERR("Add IP address extract failed(%d)", ret);
+ return ret;
+ }
+ if (pos > 1) {
+ DPAA2_PMD_ERR("Invalid IP address extract position(%d)", pos);
+ return -EINVAL;
+ }
+ if (update) {
+ key_profile->num++;
+ key_profile->prot_field[num].type = DPAA2_NET_PROT_KEY;
+ key_profile->prot_field[num].prot = prot;
+ key_profile->prot_field[num].key_field = field;
- if (dist_type == DPAA2_FLOW_QOS_TYPE)
- local_cfg = DPAA2_FLOW_QOS_TYPE;
- else
- local_cfg = DPAA2_FLOW_FS_TYPE;
-
-rule_configure:
- key_addr += key_profile->ip_addr_extract_off;
- ip_addr_data = (union ip_addr_extract_rule *)key_addr;
- mask_addr += key_profile->ip_addr_extract_off;
- ip_addr_mask = (union ip_addr_extract_rule *)mask_addr;
-
- if (orig_prot == NET_PROT_IPV4 &&
- orig_field == NH_FLD_IPV4_SRC_IP) {
- if (key_profile->ip_addr_type == IP_SRC_EXTRACT ||
- key_profile->ip_addr_type == IP_SRC_DST_EXTRACT) {
- memcpy(&ip_addr_data->ipv4_sd_addr.ipv4_src,
- key, size);
- memcpy(&ip_addr_mask->ipv4_sd_addr.ipv4_src,
- mask, size);
- } else {
- memcpy(&ip_addr_data->ipv4_ds_addr.ipv4_src,
- key, size);
- memcpy(&ip_addr_mask->ipv4_ds_addr.ipv4_src,
- mask, size);
- }
- } else if (orig_prot == NET_PROT_IPV4 &&
- orig_field == NH_FLD_IPV4_DST_IP) {
- if (key_profile->ip_addr_type == IP_DST_EXTRACT ||
- key_profile->ip_addr_type == IP_DST_SRC_EXTRACT) {
- memcpy(&ip_addr_data->ipv4_ds_addr.ipv4_dst,
- key, size);
- memcpy(&ip_addr_mask->ipv4_ds_addr.ipv4_dst,
- mask, size);
- } else {
- memcpy(&ip_addr_data->ipv4_sd_addr.ipv4_dst,
- key, size);
- memcpy(&ip_addr_mask->ipv4_sd_addr.ipv4_dst,
- mask, size);
- }
- } else if (orig_prot == NET_PROT_IPV6 &&
- orig_field == NH_FLD_IPV6_SRC_IP) {
- if (key_profile->ip_addr_type == IP_SRC_EXTRACT ||
- key_profile->ip_addr_type == IP_SRC_DST_EXTRACT) {
- memcpy(ip_addr_data->ipv6_sd_addr.ipv6_src,
- key, size);
- memcpy(ip_addr_mask->ipv6_sd_addr.ipv6_src,
- mask, size);
- } else {
- memcpy(ip_addr_data->ipv6_ds_addr.ipv6_src,
- key, size);
- memcpy(ip_addr_mask->ipv6_ds_addr.ipv6_src,
- mask, size);
- }
- } else if (orig_prot == NET_PROT_IPV6 &&
- orig_field == NH_FLD_IPV6_DST_IP) {
- if (key_profile->ip_addr_type == IP_DST_EXTRACT ||
- key_profile->ip_addr_type == IP_DST_SRC_EXTRACT) {
- memcpy(ip_addr_data->ipv6_ds_addr.ipv6_dst,
- key, size);
- memcpy(ip_addr_mask->ipv6_ds_addr.ipv6_dst,
- mask, size);
- } else {
- memcpy(ip_addr_data->ipv6_sd_addr.ipv6_dst,
- key, size);
- memcpy(ip_addr_mask->ipv6_sd_addr.ipv6_dst,
- mask, size);
- }
+ dpkg->extracts[num].type = DPKG_EXTRACT_FROM_HDR;
+ dpkg->extracts[num].extract.from_hdr.prot = prot;
+ dpkg->extracts[num].extract.from_hdr.field = field;
+ dpkg->extracts[num].extract.from_hdr.type = DPKG_FULL_FIELD;
+ dpkg->num_extracts++;
+
+ local_cfg = true;
}
- if (dist_type == DPAA2_FLOW_QOS_TYPE) {
- flow->qos_rule_size =
- key_profile->ip_addr_extract_off + ipaddr_extract_len;
+ key_addr += ip_addr_offset;
+ mask_addr += ip_addr_offset;
+
+ if (pos == 0) {
+ rte_memcpy(key_addr, key, size);
+ rte_memcpy(mask_addr, mask, size);
} else {
- flow->fs_rule_size =
- key_profile->ip_addr_extract_off + ipaddr_extract_len;
+ rte_memcpy(key_addr + size, key, size);
+ rte_memcpy(mask_addr + size, mask, size);
}
+ flow->rule_size = ip_addr_offset + size * (pos + 1);
+
if (recfg)
- *recfg |= local_cfg;
+ (*recfg) |= local_cfg;
return 0;
}
+static void dpaa2_flow_hdr_hexdump(char *dump_buf,
+ const uint8_t *hdr, uint32_t size)
+{
+ uint32_t i;
+
+ for (i = 0; i < size; i++)
+ sprintf(&dump_buf[i], "%02x ", hdr[i]);
+}
+
static int
-dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_tunnel_eth_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
- const struct rte_flow_item *pattern,
- int *device_configured)
+ const struct rte_flow_item *pattern, int *extract_cfg,
+ enum dpaa2_flow_dist_type dist_type)
{
- int ret, local_cfg = 0;
+ int ret, local_cfg = false;
uint32_t group;
const struct rte_flow_item_eth *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const char zero_cmp[RTE_ETHER_ADDR_LEN] = {0};
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
+
+ if (!pattern->spec)
+ return 0;
group = attr->group;
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
- pattern->mask : &dpaa2_flow_item_eth_mask;
-
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
- if (!spec)
- return 0;
+ pattern->mask : &dpaa2_flow_item_eth_mask;
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_ETH);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of ethernet failed");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_eth));
+ DPAA2_PMD_WARN("Extract ethernet(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -2128,40 +1940,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_SADDR0_OFFSET,
1, &spec->src.addr_bytes[0],
&mask->src.addr_bytes[0],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*SRC[1:2]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_SADDR1_OFFSET,
- 2, &spec->src.addr_bytes[1],
- &mask->src.addr_bytes[1],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*SRC[3:1]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_SADDR3_OFFSET,
- 1, &spec->src.addr_bytes[3],
- &mask->src.addr_bytes[3],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*SRC[4:2]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_SADDR4_OFFSET,
- 2, &spec->src.addr_bytes[4],
- &mask->src.addr_bytes[4],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- /*SRC[0:1]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_SADDR0_OFFSET,
- 1, &spec->src.addr_bytes[0],
- &mask->src.addr_bytes[0],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*SRC[1:2]*/
@@ -2169,7 +1948,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_SADDR1_OFFSET,
2, &spec->src.addr_bytes[1],
&mask->src.addr_bytes[1],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*SRC[3:1]*/
@@ -2177,7 +1956,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_SADDR3_OFFSET,
1, &spec->src.addr_bytes[3],
&mask->src.addr_bytes[3],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*SRC[4:2]*/
@@ -2185,7 +1964,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_SADDR4_OFFSET,
2, &spec->src.addr_bytes[4],
&mask->src.addr_bytes[4],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -2197,40 +1976,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_DADDR0_OFFSET,
1, &spec->dst.addr_bytes[0],
&mask->dst.addr_bytes[0],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*DST[1:1]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_DADDR1_OFFSET,
- 1, &spec->dst.addr_bytes[1],
- &mask->dst.addr_bytes[1],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*DST[2:3]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_DADDR2_OFFSET,
- 3, &spec->dst.addr_bytes[2],
- &mask->dst.addr_bytes[2],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- /*DST[5:1]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_DADDR5_OFFSET,
- 1, &spec->dst.addr_bytes[5],
- &mask->dst.addr_bytes[5],
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- /*DST[0:1]*/
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_DADDR0_OFFSET,
- 1, &spec->dst.addr_bytes[0],
- &mask->dst.addr_bytes[0],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*DST[1:1]*/
@@ -2238,7 +1984,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_DADDR1_OFFSET,
1, &spec->dst.addr_bytes[1],
&mask->dst.addr_bytes[1],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*DST[2:3]*/
@@ -2246,7 +1992,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_DADDR2_OFFSET,
3, &spec->dst.addr_bytes[2],
&mask->dst.addr_bytes[2],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
/*DST[5:1]*/
@@ -2254,7 +2000,7 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
DPAA2_VXLAN_IN_DADDR5_OFFSET,
1, &spec->dst.addr_bytes[5],
&mask->dst.addr_bytes[5],
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -2264,76 +2010,65 @@ dpaa2_configure_flow_tunnel_eth(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_pr_extract_rule(flow,
DPAA2_VXLAN_IN_TYPE_OFFSET,
sizeof(rte_be16_t), &spec->type, &mask->type,
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_TYPE_OFFSET,
- sizeof(rte_be16_t), &spec->type, &mask->type,
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_eth(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_eth_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_eth *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const char zero_cmp[RTE_ETHER_ADDR_LEN] = {0};
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
if (dpaa2_pattern->in_tunnel) {
- return dpaa2_configure_flow_tunnel_eth(flow,
- dev, attr, pattern, device_configured);
+ return dpaa2_flow_tunnel_eth_extract_rule_set(flow,
+ attr, pattern, extract_cfg,
+ dist_type);
}
- group = attr->group;
-
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
- pattern->mask : &dpaa2_flow_item_eth_mask;
-
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
+ pattern->mask : &dpaa2_flow_item_eth_mask;
if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_ETH_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_MAC_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_ETH_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
+ if (extract_cfg && !ret)
+ (*extract_cfg) |= local_cfg;
- (*device_configured) |= local_cfg;
- return 0;
+ return ret;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_ETH);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of ethernet failed");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_eth));
+ DPAA2_PMD_WARN("Extract ethernet(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -2343,14 +2078,7 @@ dpaa2_configure_flow_eth(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
NH_FLD_ETH_SA, &spec->src.addr_bytes,
&mask->src.addr_bytes, RTE_ETHER_ADDR_LEN,
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
- NH_FLD_ETH_SA, &spec->src.addr_bytes,
- &mask->src.addr_bytes, RTE_ETHER_ADDR_LEN,
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -2360,14 +2088,7 @@ dpaa2_configure_flow_eth(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
NH_FLD_ETH_DA, &spec->dst.addr_bytes,
&mask->dst.addr_bytes, RTE_ETHER_ADDR_LEN,
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
- NH_FLD_ETH_DA, &spec->dst.addr_bytes,
- &mask->dst.addr_bytes, RTE_ETHER_ADDR_LEN,
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -2377,34 +2098,28 @@ dpaa2_configure_flow_eth(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
NH_FLD_ETH_TYPE, &spec->type,
&mask->type, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ETH,
- NH_FLD_ETH_TYPE, &spec->type,
- &mask->type, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_tunnel_vlan(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_tunnel_vlan_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
- const struct rte_flow_item *pattern,
- int *device_configured)
+ const struct rte_flow_item *pattern, int *extract_cfg,
+ enum dpaa2_flow_dist_type dist_type)
{
- int ret, local_cfg = 0;
- uint32_t group;
+ int ret, local_cfg = false;
+ uint32_t group, bit_offset;
const struct rte_flow_item_vlan *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
group = attr->group;
@@ -2413,33 +2128,30 @@ dpaa2_configure_flow_tunnel_vlan(struct dpaa2_dev_flow *flow,
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_vlan_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_VLAN_FRAM,
- DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TUNNEL_VLAN_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_VLAN_FRAM,
- DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_VLAN);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of vlan not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_vlan));
+ DPAA2_PMD_WARN("Extract vlan(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -2450,74 +2162,64 @@ dpaa2_configure_flow_tunnel_vlan(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_pr_extract_rule(flow,
DPAA2_VXLAN_IN_TCI_OFFSET,
sizeof(rte_be16_t), &spec->tci, &mask->tci,
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_IN_TCI_OFFSET,
- sizeof(rte_be16_t), &spec->tci, &mask->tci,
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, group, &local_cfg, dist_type);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_vlan(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_vlan_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_vlan *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
if (dpaa2_pattern->in_tunnel) {
- return dpaa2_configure_flow_tunnel_vlan(flow,
- dev, attr, pattern, device_configured);
+ return dpaa2_flow_tunnel_vlan_extract_rule_set(flow,
+ attr, pattern, extract_cfg,
+ dist_type);
}
- group = attr->group;
-
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ? pattern->mask : &dpaa2_flow_item_vlan_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
- if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_VLAN_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_VLAN_ID);
+ if (ret)
+ return ret;
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_VLAN_FRAM,
- DPAA2_FLOW_FS_TYPE, group,
- &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type, attr->group, &local_cfg);
+ if (ret)
+ return ret;
- (*device_configured) |= local_cfg;
+ if (!spec) {
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_VLAN);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of vlan not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_vlan));
+ DPAA2_PMD_WARN("Extract vlan(%s) failed(%d)",
+ hex_dump, ret);
+
return ret;
}
@@ -2527,45 +2229,34 @@ dpaa2_configure_flow_vlan(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_VLAN,
NH_FLD_VLAN_TCI, &spec->tci,
&mask->tci, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_VLAN,
- NH_FLD_VLAN_TCI, &spec->tci,
- &mask->tci, sizeof(rte_be16_t),
- priv, group, &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_ipv4(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_ipv4_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_ipv4 *spec_ipv4 = 0, *mask_ipv4 = 0;
const void *key, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
int size;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec_ipv4 = pattern->spec;
mask_ipv4 = pattern->mask ?
- pattern->mask : &dpaa2_flow_item_ipv4_mask;
+ pattern->mask : &dpaa2_flow_item_ipv4_mask;
if (dpaa2_pattern->in_tunnel) {
if (spec_ipv4) {
@@ -2573,44 +2264,48 @@ dpaa2_configure_flow_ipv4(struct dpaa2_dev_flow *flow,
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_IPV4_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TUNNEL_IPV4_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_IPV4_FRAM,
- DPAA2_FLOW_FS_TYPE, group,
- &local_cfg);
- return ret;
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
+ if (ret)
+ return ret;
}
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_IPV4_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_IPV4_ID);
if (ret)
return ret;
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_IPV4_FRAM,
- DPAA2_FLOW_FS_TYPE, group, &local_cfg);
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
if (ret)
return ret;
if (!spec_ipv4) {
- (*device_configured) |= local_cfg;
+ if (flow->ip_key == NET_PROT_IPV6) {
+ DPAA2_PMD_ERR("IPv6 flow has been configured");
+ return -EINVAL;
+ }
+ flow->ip_key = NET_PROT_IPV4;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask_ipv4,
RTE_FLOW_ITEM_TYPE_IPV4);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of IPv4 not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask_ipv4,
+ sizeof(struct rte_flow_item_ipv4));
+ DPAA2_PMD_WARN("Extract IPv4(%s) failed(%d)",
+ hex_dump, ret);
+
return ret;
}
@@ -2620,20 +2315,11 @@ dpaa2_configure_flow_ipv4(struct dpaa2_dev_flow *flow,
size = sizeof(rte_be32_t);
ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV4,
- NH_FLD_IPV4_SRC_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV4,
- NH_FLD_IPV4_SRC_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ NH_FLD_IPV4_SRC_IP, key, mask, size, priv,
+ attr->group, &local_cfg, dist_type);
if (ret)
return ret;
+ flow->ip_src = NET_PROT_IPV4;
}
if (mask_ipv4->hdr.dst_addr) {
@@ -2642,19 +2328,41 @@ dpaa2_configure_flow_ipv4(struct dpaa2_dev_flow *flow,
size = sizeof(rte_be32_t);
ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV4,
- NH_FLD_IPV4_DST_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
+ NH_FLD_IPV4_DST_IP, key, mask, size, priv,
+ attr->group, &local_cfg, dist_type);
if (ret)
return ret;
- ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV4,
- NH_FLD_IPV4_DST_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ flow->ip_dst = NET_PROT_IPV4;
+ }
+
+ if (mask_ipv4->hdr.packet_id) {
+ key = &spec_ipv4->hdr.packet_id;
+ mask = &mask_ipv4->hdr.packet_id;
+ size = sizeof(rte_be16_t);
+ ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IP,
+ NH_FLD_IP_ID, key, mask, size,
+ priv, attr->group, &local_cfg, dist_type);
+ if (ret)
+ return ret;
+ }
+
+ if (mask_ipv4->hdr.fragment_offset) {
+ key = &spec_ipv4->hdr.fragment_offset;
+ mask = &mask_ipv4->hdr.fragment_offset;
+ size = sizeof(rte_be16_t);
+
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_IP_FRAG_ID);
+ if (ret)
+ return ret;
+
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
+
+ if (spec_ipv4->hdr.fragment_offset)
+ DPAA2_PMD_WARN("Unsupport Extract of frag offset");
}
if (mask_ipv4->hdr.next_proto_id) {
@@ -2663,139 +2371,117 @@ dpaa2_configure_flow_ipv4(struct dpaa2_dev_flow *flow,
size = sizeof(uint8_t);
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IP,
- NH_FLD_IP_PROTO, key,
- mask, size, priv, group,
- &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IP,
- NH_FLD_IP_PROTO, key,
- mask, size, priv, group,
- &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ NH_FLD_IP_PROTO, key, mask, size, priv, attr->group,
+ &local_cfg, dist_type);
if (ret)
return ret;
}
+ if (flow->ip_key == NET_PROT_IPV6) {
+ DPAA2_PMD_ERR("IPv6 flow has been configured");
+ return -EINVAL;
+ }
+ flow->ip_key = NET_PROT_IPV4;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_ipv6(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_ipv6_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_ipv6 *spec_ipv6 = 0, *mask_ipv6 = 0;
const void *key, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const char zero_cmp[NH_FLD_IPV6_ADDR_SIZE] = {0};
int size;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec_ipv6 = pattern->spec;
mask_ipv6 = pattern->mask ? pattern->mask : &dpaa2_flow_item_ipv6_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
if (spec_ipv6) {
DPAA2_PMD_ERR("Tunnel-IPv6 distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_IPV6_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TUNNEL_IPV6_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_IPV6_FRAM,
- DPAA2_FLOW_FS_TYPE, group,
- &local_cfg);
- return ret;
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
+ if (ret)
+ return ret;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_IPV6_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_IPV6_ID);
if (ret)
return ret;
- ret = dpaa2_flow_identify_by_faf(priv, flow, FAF_IPV6_FRAM,
- DPAA2_FLOW_FS_TYPE, group, &local_cfg);
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec_ipv6) {
- (*device_configured) |= local_cfg;
+ if (flow->ip_key == NET_PROT_IPV4) {
+ DPAA2_PMD_ERR("IPv4 flow has been configured");
+ return -EINVAL;
+ }
+ flow->ip_key = NET_PROT_IPV6;
+
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask_ipv6,
RTE_FLOW_ITEM_TYPE_IPV6);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of IPv6 not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask_ipv6,
+ sizeof(struct rte_flow_item_ipv6));
+ DPAA2_PMD_WARN("Extract IPv6(%s) failed(%d)",
+ hex_dump, ret);
+
return ret;
}
- if (memcmp((const char *)&mask_ipv6->hdr.src_addr, zero_cmp, NH_FLD_IPV6_ADDR_SIZE)) {
+ if (memcmp((const void *)&mask_ipv6->hdr.src_addr, zero_cmp, NH_FLD_IPV6_ADDR_SIZE)) {
key = &spec_ipv6->hdr.src_addr;
mask = &mask_ipv6->hdr.src_addr;
size = NH_FLD_IPV6_ADDR_SIZE;
ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV6,
- NH_FLD_IPV6_SRC_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV6,
- NH_FLD_IPV6_SRC_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ NH_FLD_IPV6_SRC_IP, key, mask, size, priv,
+ attr->group, &local_cfg, dist_type);
if (ret)
return ret;
+ flow->ip_src = NET_PROT_IPV6;
}
- if (memcmp((const char *)&mask_ipv6->hdr.dst_addr, zero_cmp, NH_FLD_IPV6_ADDR_SIZE)) {
+ if (memcmp((const void *)&mask_ipv6->hdr.dst_addr, zero_cmp, NH_FLD_IPV6_ADDR_SIZE)) {
key = &spec_ipv6->hdr.dst_addr;
mask = &mask_ipv6->hdr.dst_addr;
size = NH_FLD_IPV6_ADDR_SIZE;
ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV6,
- NH_FLD_IPV6_DST_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_ipaddr_extract_rule(flow, NET_PROT_IPV6,
- NH_FLD_IPV6_DST_IP,
- key, mask, size, priv,
- group, &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ NH_FLD_IPV6_DST_IP, key, mask, size, priv,
+ attr->group, &local_cfg, dist_type);
if (ret)
return ret;
+ flow->ip_dst = NET_PROT_IPV6;
}
if (mask_ipv6->hdr.proto) {
@@ -2804,78 +2490,68 @@ dpaa2_configure_flow_ipv6(struct dpaa2_dev_flow *flow,
size = sizeof(uint8_t);
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IP,
- NH_FLD_IP_PROTO, key,
- mask, size, priv, group,
- &local_cfg,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IP,
- NH_FLD_IP_PROTO, key,
- mask, size, priv, group,
- &local_cfg,
- DPAA2_FLOW_FS_TYPE);
+ NH_FLD_IP_PROTO, key, mask, size, priv, attr->group,
+ &local_cfg, dist_type);
if (ret)
return ret;
}
+ if (flow->ip_key == NET_PROT_IPV4) {
+ DPAA2_PMD_ERR("IPv4 flow has been configured");
+ return -EINVAL;
+ }
+ flow->ip_key = NET_PROT_IPV6;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_icmp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_icmp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_icmp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_icmp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-ICMP distribution not support");
return -ENOTSUP;
}
if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_ICMP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_ICMP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_ICMP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_ICMP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of ICMP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_icmp));
+ DPAA2_PMD_WARN("Extract ICMP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -2884,14 +2560,7 @@ dpaa2_configure_flow_icmp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ICMP,
NH_FLD_ICMP_TYPE, &spec->hdr.icmp_type,
&mask->hdr.icmp_type, sizeof(uint8_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ICMP,
- NH_FLD_ICMP_TYPE, &spec->hdr.icmp_type,
- &mask->hdr.icmp_type, sizeof(uint8_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -2900,90 +2569,75 @@ dpaa2_configure_flow_icmp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ICMP,
NH_FLD_ICMP_CODE, &spec->hdr.icmp_code,
&mask->hdr.icmp_code, sizeof(uint8_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_ICMP,
- NH_FLD_ICMP_CODE, &spec->hdr.icmp_code,
- &mask->hdr.icmp_code, sizeof(uint8_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_udp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_udp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_udp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_udp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
if (spec) {
DPAA2_PMD_ERR("Tunnel-UDP distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_UDP_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TUNNEL_UDP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_UDP_FRAM,
- DPAA2_FLOW_FS_TYPE, group,
- &local_cfg);
- return ret;
+ bit_offset, dist_type, attr->group, &local_cfg);
+ if (ret)
+ return ret;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_UDP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_UDP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_UDP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec) {
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_UDP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of UDP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_udp));
+ DPAA2_PMD_ERR("Extract UDP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -2992,14 +2646,7 @@ dpaa2_configure_flow_udp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_UDP,
NH_FLD_UDP_PORT_SRC, &spec->hdr.src_port,
&mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_UDP,
- NH_FLD_UDP_PORT_SRC, &spec->hdr.src_port,
- &mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -3008,90 +2655,75 @@ dpaa2_configure_flow_udp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_UDP,
NH_FLD_UDP_PORT_DST, &spec->hdr.dst_port,
&mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_UDP,
- NH_FLD_UDP_PORT_DST, &spec->hdr.dst_port,
- &mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_tcp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_tcp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_tcp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_tcp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
if (spec) {
DPAA2_PMD_ERR("Tunnel-TCP distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_TCP_FRAM,
- DPAA2_FLOW_QOS_TYPE, group,
- &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TUNNEL_TCP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_VXLAN_IN_TCP_FRAM,
- DPAA2_FLOW_FS_TYPE, group,
- &local_cfg);
- return ret;
+ bit_offset, dist_type, attr->group, &local_cfg);
+ if (ret)
+ return ret;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_TCP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_TCP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_TCP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec) {
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_TCP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of TCP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_tcp));
+ DPAA2_PMD_ERR("Extract TCP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3100,14 +2732,7 @@ dpaa2_configure_flow_tcp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_TCP,
NH_FLD_TCP_PORT_SRC, &spec->hdr.src_port,
&mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_TCP,
- NH_FLD_TCP_PORT_SRC, &spec->hdr.src_port,
- &mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -3116,76 +2741,64 @@ dpaa2_configure_flow_tcp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_TCP,
NH_FLD_TCP_PORT_DST, &spec->hdr.dst_port,
&mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_TCP,
- NH_FLD_TCP_PORT_DST, &spec->hdr.dst_port,
- &mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_esp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_esp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_esp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_esp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-ESP distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_IPSEC_ESP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_IPSEC_ESP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_IPSEC_ESP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec) {
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_ESP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of ESP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_esp));
+ DPAA2_PMD_ERR("Extract IPSEC ESP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3194,14 +2807,7 @@ dpaa2_configure_flow_esp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_ESP,
NH_FLD_IPSEC_ESP_SPI, &spec->hdr.spi,
&mask->hdr.spi, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_ESP,
- NH_FLD_IPSEC_ESP_SPI, &spec->hdr.spi,
- &mask->hdr.spi, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -3210,76 +2816,64 @@ dpaa2_configure_flow_esp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_ESP,
NH_FLD_IPSEC_ESP_SEQUENCE_NUM, &spec->hdr.seq,
&mask->hdr.seq, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_ESP,
- NH_FLD_IPSEC_ESP_SEQUENCE_NUM, &spec->hdr.seq,
- &mask->hdr.seq, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_ah(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_ah_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_ah *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_ah_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-AH distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_IPSEC_AH_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_IPSEC_AH_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_IPSEC_AH_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec) {
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_AH);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of AH not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_ah));
+ DPAA2_PMD_ERR("Extract IPSEC AH(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3288,14 +2882,7 @@ dpaa2_configure_flow_ah(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_AH,
NH_FLD_IPSEC_AH_SPI, &spec->spi,
&mask->spi, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_IPSEC_AH,
- NH_FLD_IPSEC_AH_SPI, &spec->spi,
- &mask->spi, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -3305,63 +2892,58 @@ dpaa2_configure_flow_ah(struct dpaa2_dev_flow *flow,
return -ENOTSUP;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_sctp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_sctp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_sctp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_sctp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-SCTP distribution not support");
return -ENOTSUP;
}
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_SCTP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_SCTP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_SCTP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
if (!spec) {
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_SCTP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of SCTP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_sctp));
+ DPAA2_PMD_ERR("Extract SCTP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3370,14 +2952,7 @@ dpaa2_configure_flow_sctp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_SCTP,
NH_FLD_SCTP_PORT_SRC, &spec->hdr.src_port,
&mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_SCTP,
- NH_FLD_SCTP_PORT_SRC, &spec->hdr.src_port,
- &mask->hdr.src_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
@@ -3386,80 +2961,63 @@ dpaa2_configure_flow_sctp(struct dpaa2_dev_flow *flow,
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_SCTP,
NH_FLD_SCTP_PORT_DST, &spec->hdr.dst_port,
&mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_SCTP,
- NH_FLD_SCTP_PORT_DST, &spec->hdr.dst_port,
- &mask->hdr.dst_port, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_gre(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_gre_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_gre *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_gre_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-GRE distribution not support");
return -ENOTSUP;
}
if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_GRE_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_GRE_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_GRE_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
- if (!priv->sp_protocol) {
- DPAA2_PMD_ERR("vXLAN flow with spec is not supported without SP.");
- return -ENOTSUP;
- }
-
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_GRE);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of GRE not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_gre));
+ DPAA2_PMD_ERR("Extract GRE(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3468,81 +3026,65 @@ dpaa2_configure_flow_gre(struct dpaa2_dev_flow *flow,
return 0;
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_GRE,
- NH_FLD_GRE_TYPE, &spec->protocol,
- &mask->protocol, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_GRE,
- NH_FLD_GRE_TYPE, &spec->protocol,
- &mask->protocol, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ NH_FLD_GRE_TYPE, &spec->protocol, &mask->protocol,
+ sizeof(rte_be16_t), priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_vxlan(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_vxlan_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_vxlan *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
-
- if (!priv->sp_protocol) {
- DPAA2_PMD_ERR("eCPRI flow is not supported without SP.");
- return -ENOTSUP;
- }
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_vxlan_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-VXLAN distribution not support");
return -ENOTSUP;
}
- if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_VXLAN_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_VXLAN_ID);
+ if (ret)
+ return ret;
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_VXLAN_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type, attr->group, &local_cfg);
+ if (ret)
+ return ret;
- (*device_configured) |= local_cfg;
- return 0;
+ if (!spec)
+ goto quit;
+
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("vXLAN flow with spec is not supported without SP.");
+ return -ENOTSUP;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_VXLAN);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of VXLAN not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_vxlan));
+ DPAA2_PMD_ERR("Extract vXLAN(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3562,55 +3104,45 @@ dpaa2_configure_flow_vxlan(struct dpaa2_dev_flow *flow,
if (mask->vni[0] || mask->vni[1] || mask->vni[2]) {
ret = dpaa2_flow_add_pr_extract_rule(flow,
DPAA2_VXLAN_VNI_OFFSET,
- sizeof(mask->vni), spec->vni,
- mask->vni,
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- DPAA2_VXLAN_VNI_OFFSET,
- sizeof(mask->vni), spec->vni,
- mask->vni,
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ sizeof(mask->vni), spec->vni, mask->vni,
+ priv, attr->group, &local_cfg, dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+quit:
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_ecpri(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_ecpri_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_ecpri *spec, *mask;
struct rte_flow_item_ecpri local_mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
- uint8_t extract_nb = 0, i;
+ int extract_nb, i;
uint64_t rule_data[DPAA2_ECPRI_MAX_EXTRACT_NB];
uint64_t mask_data[DPAA2_ECPRI_MAX_EXTRACT_NB];
uint8_t extract_size[DPAA2_ECPRI_MAX_EXTRACT_NB];
uint8_t extract_off[DPAA2_ECPRI_MAX_EXTRACT_NB];
+ union dpaa2_sp_fafe_parse fafe;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
if (!priv->sp_protocol) {
- DPAA2_PMD_ERR("ROCEV2 flow is not supported without SP.");
+ DPAA2_PMD_ERR("eCPRI flow is not supported without SP.");
return -ENOTSUP;
}
- group = attr->group;
-
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
if (pattern->mask) {
@@ -3623,353 +3155,111 @@ dpaa2_configure_flow_ecpri(struct dpaa2_dev_flow *flow,
mask = &dpaa2_flow_item_ecpri_mask;
}
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-ECPRI distribution not support");
return -ENOTSUP;
}
- if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_ECPRI_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_ECPRI_ID);
+ if (ret)
+ return ret;
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAFE_ECPRI_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type,
+ attr->group, &local_cfg);
+ if (ret)
+ return ret;
- (*device_configured) |= local_cfg;
- return 0;
- }
+ if (!spec)
+ goto quit;
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_ECPRI);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of ECPRI not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_ecpri));
+ DPAA2_PMD_ERR("Extract eCPRI(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
- if (mask->hdr.common.type != 0xff) {
- DPAA2_PMD_WARN("ECPRI header type not specified.");
-
- return -EINVAL;
- }
+ memset(&fafe, 0, sizeof(union dpaa2_sp_fafe_parse));
+ extract_nb = dpaa2_parser_ecpri_extract(spec, mask,
+ rule_data, mask_data, extract_size, extract_off,
+ &fafe, DPAA2_ECPRI_MAX_EXTRACT_NB);
+ if (extract_nb < 0) {
+ DPAA2_PMD_ERR("Extract eCPRI from spec/mask failed(%d)",
+ extract_nb);
- if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_IQ_DATA) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_0;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type0.pc_id) {
- rule_data[extract_nb] = spec->hdr.type0.pc_id;
- mask_data[extract_nb] = mask->hdr.type0.pc_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_iq_data, pc_id);
- extract_nb++;
- }
- if (mask->hdr.type0.seq_id) {
- rule_data[extract_nb] = spec->hdr.type0.seq_id;
- mask_data[extract_nb] = mask->hdr.type0.seq_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_iq_data, seq_id);
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_BIT_SEQ) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_1;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type1.pc_id) {
- rule_data[extract_nb] = spec->hdr.type1.pc_id;
- mask_data[extract_nb] = mask->hdr.type1.pc_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_bit_seq, pc_id);
- extract_nb++;
- }
- if (mask->hdr.type1.seq_id) {
- rule_data[extract_nb] = spec->hdr.type1.seq_id;
- mask_data[extract_nb] = mask->hdr.type1.seq_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_bit_seq, seq_id);
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RTC_CTRL) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_2;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type2.rtc_id) {
- rule_data[extract_nb] = spec->hdr.type2.rtc_id;
- mask_data[extract_nb] = mask->hdr.type2.rtc_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_rtc_ctrl, rtc_id);
- extract_nb++;
- }
- if (mask->hdr.type2.seq_id) {
- rule_data[extract_nb] = spec->hdr.type2.seq_id;
- mask_data[extract_nb] = mask->hdr.type2.seq_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_rtc_ctrl, seq_id);
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_GEN_DATA) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_3;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type3.pc_id || mask->hdr.type3.seq_id)
- DPAA2_PMD_WARN("Extract type3 msg not support.");
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RM_ACC) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_4;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type4.rma_id) {
- rule_data[extract_nb] = spec->hdr.type4.rma_id;
- mask_data[extract_nb] = mask->hdr.type4.rma_id;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET + 0;
- /** Compiler not support to take address
- * of bit-field
- * offsetof(struct rte_ecpri_msg_rm_access,
- * rma_id);
- */
- extract_nb++;
- }
- if (mask->hdr.type4.ele_id) {
- rule_data[extract_nb] = spec->hdr.type4.ele_id;
- mask_data[extract_nb] = mask->hdr.type4.ele_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET + 2;
- /** Compiler not support to take address
- * of bit-field
- * offsetof(struct rte_ecpri_msg_rm_access,
- * ele_id);
- */
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_DLY_MSR) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_5;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type5.msr_id) {
- rule_data[extract_nb] = spec->hdr.type5.msr_id;
- mask_data[extract_nb] = mask->hdr.type5.msr_id;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_delay_measure,
- msr_id);
- extract_nb++;
- }
- if (mask->hdr.type5.act_type) {
- rule_data[extract_nb] = spec->hdr.type5.act_type;
- mask_data[extract_nb] = mask->hdr.type5.act_type;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_delay_measure,
- act_type);
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_RMT_RST) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_6;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type6.rst_id) {
- rule_data[extract_nb] = spec->hdr.type6.rst_id;
- mask_data[extract_nb] = mask->hdr.type6.rst_id;
- extract_size[extract_nb] = sizeof(rte_be16_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_remote_reset,
- rst_id);
- extract_nb++;
- }
- if (mask->hdr.type6.rst_op) {
- rule_data[extract_nb] = spec->hdr.type6.rst_op;
- mask_data[extract_nb] = mask->hdr.type6.rst_op;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_remote_reset,
- rst_op);
- extract_nb++;
- }
- } else if (spec->hdr.common.type == RTE_ECPRI_MSG_TYPE_EVT_IND) {
- rule_data[extract_nb] = ECPRI_FAFE_TYPE_7;
- mask_data[extract_nb] = 0xff;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] = DPAA2_FAFE_PSR_OFFSET;
- extract_nb++;
-
- if (mask->hdr.type7.evt_id) {
- rule_data[extract_nb] = spec->hdr.type7.evt_id;
- mask_data[extract_nb] = mask->hdr.type7.evt_id;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_event_ind,
- evt_id);
- extract_nb++;
- }
- if (mask->hdr.type7.evt_type) {
- rule_data[extract_nb] = spec->hdr.type7.evt_type;
- mask_data[extract_nb] = mask->hdr.type7.evt_type;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_event_ind,
- evt_type);
- extract_nb++;
- }
- if (mask->hdr.type7.seq) {
- rule_data[extract_nb] = spec->hdr.type7.seq;
- mask_data[extract_nb] = mask->hdr.type7.seq;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_event_ind,
- seq);
- extract_nb++;
- }
- if (mask->hdr.type7.number) {
- rule_data[extract_nb] = spec->hdr.type7.number;
- mask_data[extract_nb] = mask->hdr.type7.number;
- extract_size[extract_nb] = sizeof(uint8_t);
- extract_off[extract_nb] =
- DPAA2_ECPRI_MSG_OFFSET +
- offsetof(struct rte_ecpri_msg_event_ind,
- number);
- extract_nb++;
- }
- } else {
- DPAA2_PMD_ERR("Invalid ecpri header type(%d)",
- spec->hdr.common.type);
- return -EINVAL;
+ return extract_nb;
}
for (i = 0; i < extract_nb; i++) {
ret = dpaa2_flow_add_pr_extract_rule(flow,
- extract_off[i],
- extract_size[i], &rule_data[i], &mask_data[i],
- priv, group,
- device_configured,
- DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_pr_extract_rule(flow,
- extract_off[i],
- extract_size[i], &rule_data[i], &mask_data[i],
- priv, group,
- device_configured,
- DPAA2_FLOW_FS_TYPE);
+ extract_off[i], extract_size[i],
+ &rule_data[i], &mask_data[i],
+ priv, attr->group, extract_cfg,
+ dist_type);
if (ret)
return ret;
}
- (*device_configured) |= local_cfg;
+quit:
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_gtp(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_gtp_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
+ uint32_t bit_offset;
const struct rte_flow_item_gtp *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_priv *priv = flow->priv;
const struct rte_flow_item *pattern =
&dpaa2_pattern->generic_item;
-
- if (!priv->sp_protocol) {
- DPAA2_PMD_ERR("GENEVE flow is not supported without SP.");
- return -ENOTSUP;
- }
-
- group = attr->group;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
pattern->mask : &dpaa2_flow_item_gtp_mask;
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
if (dpaa2_pattern->in_tunnel) {
DPAA2_PMD_ERR("Tunnel-GTP distribution not support");
return -ENOTSUP;
}
if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_GTP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_GTP_ID);
if (ret)
return ret;
ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_GTP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
ret = dpaa2_flow_extract_support((const uint8_t *)mask,
RTE_FLOW_ITEM_TYPE_GTP);
if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of GTP not support.");
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_gtp));
+ DPAA2_PMD_ERR("Extract GTP(%s) failed(%d)",
+ hex_dump, ret);
return ret;
}
@@ -3978,298 +3268,236 @@ dpaa2_configure_flow_gtp(struct dpaa2_dev_flow *flow,
return 0;
ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_GTP,
- NH_FLD_GTP_TEID, &spec->teid,
- &mask->teid, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_GTP,
- NH_FLD_GTP_TEID, &spec->teid,
- &mask->teid, sizeof(rte_be32_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ NH_FLD_GTP_TEID, &spec->teid, &mask->teid,
+ sizeof(rte_be32_t), priv, attr->group,
+ &local_cfg, dist_type);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_mpls(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_raw_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
- int ret, local_cfg = 0;
- uint32_t group;
- const struct rte_flow_item_mpls *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- const struct rte_flow_item *pattern =
- &dpaa2_pattern->generic_item;
-
- group = attr->group;
-
- /* Parse pattern list to get the matching parameters */
- spec = pattern->spec;
- mask = pattern->mask ?
- pattern->mask : &dpaa2_flow_item_mpls_mask;
-
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
+ struct dpaa2_dev_priv *priv = flow->priv;
+ int local_cfg = 0, ret;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
+ const struct rte_flow_item_raw *spec = pattern->spec;
+ const struct rte_flow_item_raw *mask = pattern->mask;
- if (dpaa2_pattern->in_tunnel) {
- DPAA2_PMD_ERR("Tunnel-MPLS distribution not support");
- return -ENOTSUP;
+ /* Need both spec and mask */
+ if (!spec || !mask) {
+ DPAA2_PMD_ERR("spec or mask not present.");
+ return -EINVAL;
}
- if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_MPLS_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_MPLS_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
+ if (spec->relative) {
+ /* TBD: relative offset support.
+ * To support relative offset of previous L3 protocol item,
+ * extracts should be expanded to identify if the frame is:
+ * vlan or none-vlan.
+ *
+ * To support relative offset of previous L4 protocol item,
+ * extracts should be expanded to identify if the frame is:
+ * vlan/IPv4 or vlan/IPv6 or none-vlan/IPv4 or none-vlan/IPv6.
+ */
+ DPAA2_PMD_ERR("relative not supported.");
+ return -EINVAL;
+ }
- (*device_configured) |= local_cfg;
- return 0;
+ if (spec->search) {
+ DPAA2_PMD_ERR("search not supported.");
+ return -EINVAL;
}
- ret = dpaa2_flow_extract_support((const uint8_t *)mask,
- RTE_FLOW_ITEM_TYPE_MPLS);
- if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of MPLS not support.");
- return ret;
+ /* Spec len and mask len should be same */
+ if (spec->length != mask->length) {
+ DPAA2_PMD_ERR("Spec len and mask len mismatch.");
+ return -EINVAL;
}
- if (!mask->label_tc_s[0] && !mask->label_tc_s[1] &&
- !mask->label_tc_s[2])
- return 0;
+ if (dist_type == DPAA2_FLOW_QOS_TYPE)
+ tbl_profile = &priv->flow_profile.qos_profile;
+ else
+ tbl_profile = &priv->flow_profile.tc_profile[attr->group];
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_MPLS,
- NH_FLD_MPLS_MPLSL_1, spec->label_tc_s,
- mask->label_tc_s, sizeof(mask->label_tc_s),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
- if (ret)
+ ret = dpaa2_flow_extract_add_raw(priv,
+ spec->offset, spec->length, dist_type,
+ attr->group, &local_cfg);
+ if (ret) {
+ DPAA2_PMD_ERR("QoS Extract RAW add failed(%d).", ret);
return ret;
+ }
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_MPLS,
- NH_FLD_MPLS_MPLSL_1, spec->label_tc_s,
- mask->label_tc_s, sizeof(mask->label_tc_s),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
- if (ret)
- return ret;
+ if (dist_type == DPAA2_FLOW_QOS_TYPE ||
+ dist_type == DPAA2_FLOW_FS_TYPE) {
+ ret = dpaa2_flow_raw_rule_data_set(flow,
+ &tbl_profile->key_profile,
+ spec->offset, spec->length,
+ spec->pattern, mask->pattern,
+ dist_type);
+ if (ret) {
+ DPAA2_PMD_ERR("QoS RAW rule data set failed(%d)", ret);
+ return ret;
+ }
+ }
- (*device_configured) |= local_cfg;
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
return 0;
}
static int
-dpaa2_configure_flow_pppoe(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
+dpaa2_flow_geneve_extract_rule_set(struct dpaa2_generic_flow *flow,
const struct rte_flow_attr *attr,
const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
+ int *extract_cfg, enum dpaa2_flow_dist_type dist_type)
{
int ret, local_cfg = 0;
- uint32_t group;
- const struct rte_flow_item_pppoe *spec, *mask;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- const struct rte_flow_item *pattern =
- &dpaa2_pattern->generic_item;
+ uint32_t bit_offset;
+ const struct rte_flow_item_geneve *spec, *mask;
+ struct dpaa2_dev_priv *priv = flow->priv;
+ const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
+ char hex_dump[DPAA2_FLOW_HDR_HEX_DUMP_SIZE];
- group = attr->group;
+ if (!priv->sp_protocol) {
+ DPAA2_PMD_ERR("GENEVE flow is not supported without SP.");
+ return -ENOTSUP;
+ }
/* Parse pattern list to get the matching parameters */
spec = pattern->spec;
mask = pattern->mask ?
- pattern->mask : &dpaa2_flow_item_pppoe_mask;
-
- /* Get traffic class index and flow id to be configured */
- flow->tc_id = group;
- flow->tc_index = attr->priority;
+ pattern->mask : &dpaa2_flow_item_geneve_mask;
if (dpaa2_pattern->in_tunnel) {
- DPAA2_PMD_ERR("Tunnel-PPPoE distribution not support");
+ DPAA2_PMD_ERR("Tunnel-GENEVE distribution not support");
return -ENOTSUP;
}
- if (!spec) {
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_PPPOE_PPP_FRAM, DPAA2_FLOW_QOS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
-
- ret = dpaa2_flow_identify_by_faf(priv, flow,
- FAF_PPPOE_PPP_FRAM, DPAA2_FLOW_FS_TYPE,
- group, &local_cfg);
- if (ret)
- return ret;
-
- (*device_configured) |= local_cfg;
- return 0;
- }
-
- ret = dpaa2_flow_extract_support((const uint8_t *)mask,
- RTE_FLOW_ITEM_TYPE_PPPOES);
- if (ret) {
- DPAA2_PMD_WARN("Extract field(s) of PPPoE not support.");
- return ret;
- }
-
- if (!mask->session_id)
- return 0;
-
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_PPPOE,
- NH_FLD_PPPOE_SID, &spec->session_id,
- &mask->session_id, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_QOS_TYPE);
+ ret = dpaa2_protocol_psr_bit_offset(&bit_offset,
+ DPAA2_PARSER_GENEVE_ID);
if (ret)
return ret;
- ret = dpaa2_flow_add_hdr_extract_rule(flow, NET_PROT_PPPOE,
- NH_FLD_PPPOE_SID, &spec->session_id,
- &mask->session_id, sizeof(rte_be16_t),
- priv, group, &local_cfg, DPAA2_FLOW_FS_TYPE);
+ ret = dpaa2_flow_identify_by_faf(priv, flow,
+ bit_offset, dist_type, attr->group, &local_cfg);
if (ret)
return ret;
- (*device_configured) |= local_cfg;
-
- return 0;
-}
+ if (!spec)
+ goto quit;
-static int
-dpaa2_configure_flow_raw(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_dpaa2_flow_item *dpaa2_pattern,
- const struct rte_flow_action actions[] __rte_unused,
- struct rte_flow_error *error __rte_unused,
- int *device_configured)
-{
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- int local_cfg = 0, ret;
- uint32_t group;
- struct dpaa2_key_extract *qos_key_extract;
- struct dpaa2_key_extract *tc_key_extract;
- const struct rte_flow_item *pattern = &dpaa2_pattern->generic_item;
- const struct rte_flow_item_raw *spec = pattern->spec;
- const struct rte_flow_item_raw *mask = pattern->mask;
+ ret = dpaa2_flow_extract_support((const uint8_t *)mask,
+ RTE_FLOW_ITEM_TYPE_GENEVE);
+ if (ret) {
+ dpaa2_flow_hdr_hexdump(hex_dump, (const uint8_t *)mask,
+ sizeof(struct rte_flow_item_geneve));
+ DPAA2_PMD_ERR("Extract GENEVE(%s) failed(%d)",
+ hex_dump, ret);
- /* Need both spec and mask */
- if (!spec || !mask) {
- DPAA2_PMD_ERR("spec or mask not present.");
- return -EINVAL;
+ return ret;
}
- if (spec->relative) {
- /* TBD: relative offset support.
- * To support relative offset of previous L3 protocol item,
- * extracts should be expanded to identify if the frame is:
- * vlan or none-vlan.
- *
- * To support relative offset of previous L4 protocol item,
- * extracts should be expanded to identify if the frame is:
- * vlan/IPv4 or vlan/IPv6 or none-vlan/IPv4 or none-vlan/IPv6.
- */
- DPAA2_PMD_ERR("relative not supported.");
+ if (mask->protocol && mask->protocol != 0xffff) {
+ DPAA2_PMD_ERR("Not support to extract geneve protocol.");
return -EINVAL;
}
- if (spec->search) {
- DPAA2_PMD_ERR("search not supported.");
- return -EINVAL;
+ if (mask->protocol) {
+ ret = dpaa2_flow_add_pr_extract_rule(flow,
+ DPAA2_GENEVE_PROTOCOL_OFFSET,
+ sizeof(mask->protocol), &spec->protocol,
+ &mask->protocol, priv, attr->group, &local_cfg,
+ dist_type);
+ if (ret)
+ return ret;
}
- /* Spec len and mask len should be same */
- if (spec->length != mask->length) {
- DPAA2_PMD_ERR("Spec len and mask len mismatch.");
- return -EINVAL;
+ if (mask->vni[0] || mask->vni[1] || mask->vni[2]) {
+ ret = dpaa2_flow_add_pr_extract_rule(flow,
+ DPAA2_GENEVE_VNI_OFFSET,
+ sizeof(mask->vni), spec->vni,
+ mask->vni,
+ priv, attr->group, &local_cfg, dist_type);
+ if (ret)
+ return ret;
}
- /* Get traffic class index and flow id to be configured */
- group = attr->group;
- flow->tc_id = group;
- flow->tc_index = attr->priority;
-
- qos_key_extract = &priv->extract.qos_key_extract;
- tc_key_extract = &priv->extract.tc_key_extract[group];
-
- ret = dpaa2_flow_extract_add_raw(priv,
- spec->offset, spec->length,
- DPAA2_FLOW_QOS_TYPE, 0, &local_cfg);
- if (ret) {
- DPAA2_PMD_ERR("QoS Extract RAW add failed.");
- return -EINVAL;
- }
+quit:
+ if (extract_cfg)
+ (*extract_cfg) |= local_cfg;
- ret = dpaa2_flow_extract_add_raw(priv,
- spec->offset, spec->length,
- DPAA2_FLOW_FS_TYPE, group, &local_cfg);
- if (ret) {
- DPAA2_PMD_ERR("FS[%d] Extract RAW add failed.",
- group);
- return -EINVAL;
- }
+ return 0;
+}
- ret = dpaa2_flow_raw_rule_data_set(flow,
- &qos_key_extract->key_profile,
- spec->offset, spec->length,
- spec->pattern, mask->pattern,
- DPAA2_FLOW_QOS_TYPE);
- if (ret) {
- DPAA2_PMD_ERR("QoS RAW rule data set failed");
- return -EINVAL;
- }
+static inline int
+dpaa2_flow_verify_entry(struct dpaa2_dev_priv *priv,
+ uint16_t tc, uint16_t entry_idx,
+ enum dpaa2_flow_dist_type dist_type)
+{
+ struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
- ret = dpaa2_flow_raw_rule_data_set(flow,
- &tc_key_extract->key_profile,
- spec->offset, spec->length,
- spec->pattern, mask->pattern,
- DPAA2_FLOW_FS_TYPE);
- if (ret) {
- DPAA2_PMD_ERR("FS RAW rule data set failed");
- return -EINVAL;
+ while (curr) {
+ if (dist_type == DPAA2_FLOW_QOS_TYPE &&
+ curr->qos_flow &&
+ curr->qos_flow->entry_index == entry_idx) {
+ DPAA2_PMD_ERR("Flow QoS.entry[%d] exists",
+ curr->qos_flow->entry_index);
+ return -EEXIST;
+ }
+ if (dist_type == DPAA2_FLOW_FS_TYPE &&
+ curr->fs_flow && curr->fs_flow->tc_id == tc &&
+ curr->fs_flow->entry_index == entry_idx) {
+ DPAA2_PMD_ERR("Flow TC[%d].entry[%d] exists",
+ tc, curr->fs_flow->entry_index);
+ return -EEXIST;
+ }
+ curr = LIST_NEXT(curr, next);
}
- (*device_configured) |= local_cfg;
-
return 0;
}
static inline int
-dpaa2_flow_verify_attr(struct dpaa2_dev_priv *priv,
- const struct rte_flow_attr *attr)
+dpaa2_flow_acquire_entry_idx(struct dpaa2_dev_priv *priv,
+ uint16_t idx, enum dpaa2_flow_dist_type dist_type,
+ uint8_t group)
{
- struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
+ int occupied = -1;
+ uint16_t max_entries;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
- while (curr) {
- if (curr->tc_id == attr->group &&
- curr->tc_index == attr->priority) {
- DPAA2_PMD_ERR("Flow(TC[%d].entry[%d] exists",
- attr->group, attr->priority);
+ if (dist_type == DPAA2_FLOW_QOS_TYPE) {
+ tbl_profile = &priv->flow_profile.qos_profile;
+ max_entries = priv->qos_entries;
+ } else {
+ tbl_profile = &priv->flow_profile.tc_profile[group];
+ max_entries = priv->fs_entries;
+ }
- return -EINVAL;
+ if (!dpaa2_flow_entry_map_get(tbl_profile->entry_map, idx))
+ return idx;
+
+ idx = 0;
+ while (idx < max_entries) {
+ if (!dpaa2_flow_entry_map_get(tbl_profile->entry_map, idx)) {
+ occupied = idx;
+ break;
}
- curr = LIST_NEXT(curr, next);
+ idx++;
}
- return 0;
+ return occupied;
}
static inline struct rte_eth_dev *
@@ -4306,7 +3534,7 @@ dpaa2_flow_redirect_dev(struct dpaa2_dev_priv *priv,
}
static inline int
-dpaa2_flow_verify_action(struct dpaa2_dev_priv *priv,
+dpaa2_flow_verify_fs_action(struct dpaa2_dev_priv *priv,
const struct rte_flow_attr *attr,
const struct rte_flow_action actions[])
{
@@ -4332,46 +3560,51 @@ dpaa2_flow_verify_action(struct dpaa2_dev_priv *priv,
rxq->tc_index, rxq->flow_id,
attr->group);
- return -ENOTSUP;
+ return -EINVAL;
}
break;
case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
case RTE_FLOW_ACTION_TYPE_PORT_ID:
if (!dpaa2_flow_redirect_dev(priv, &actions[j])) {
DPAA2_PMD_ERR("Invalid port id of action");
- return -ENOTSUP;
+ return -EINVAL;
}
break;
case RTE_FLOW_ACTION_TYPE_RSS:
- rss_conf = (const struct rte_flow_action_rss *)
- (actions[j].conf);
+ rss_conf = actions[j].conf;
if (rss_conf->queue_num > priv->dist_queues) {
DPAA2_PMD_ERR("RSS number too large");
- return -ENOTSUP;
+ return -EINVAL;
}
- for (i = 0; i < (int)rss_conf->queue_num; i++) {
- if (rss_conf->queue[i] >= priv->nb_rx_queues) {
- DPAA2_PMD_ERR("RSS queue not in range");
- return -ENOTSUP;
- }
- rxq = priv->rx_vq[rss_conf->queue[i]];
- if (rxq->tc_index != attr->group) {
- DPAA2_PMD_ERR("RSS queue not in group");
- return -ENOTSUP;
+ if (rss_conf->queue) {
+ for (i = 0; i < (int)rss_conf->queue_num; i++) {
+ if (rss_conf->queue[i] >= priv->nb_rx_queues) {
+ DPAA2_PMD_ERR("RSS queue not in range");
+ return -EINVAL;
+ }
+ rxq = priv->rx_vq[rss_conf->queue[i]];
+ if (rxq->tc_index != attr->group) {
+ DPAA2_PMD_ERR("RSS queue not in group");
+ return -EINVAL;
+ }
}
}
break;
case RTE_FLOW_ACTION_TYPE_PF:
- /* Skip this action, have to add for vxlan */
+ /* Skip this action, have to add for vxlan*/
case RTE_FLOW_ACTION_TYPE_DROP:
break;
+ case RTE_FLOW_ACTION_TYPE_METER_MARK:
+ break;
+ case RTE_FLOW_ACTION_TYPE_METER:
+ break;
case RTE_FLOW_ACTION_TYPE_END:
end_of_list = 1;
break;
default:
DPAA2_PMD_ERR("Invalid action type");
- return -ENOTSUP;
+ return -EINVAL;
}
j++;
}
@@ -4379,40 +3612,74 @@ dpaa2_flow_verify_action(struct dpaa2_dev_priv *priv,
return 0;
}
+static struct dpaa2_queue *
+dpaa2_flow_queue_action_to_queue(struct dpaa2_dev_priv *priv,
+ uint8_t tc_id, uint16_t queue_id)
+{
+ struct dpaa2_queue *dest_q;
+
+ if (queue_id >= MAX_RX_QUEUES ||
+ !priv->rx_vq[queue_id]) {
+ DPAA2_PMD_ERR("Invalid FSQ index(%d)", queue_id);
+
+ return NULL;
+ }
+ dest_q = priv->rx_vq[queue_id];
+ if (tc_id != dest_q->tc_index) {
+ DPAA2_PMD_ERR("RXQ[%d](%d.%d) not in TC[%d]",
+ queue_id,
+ dest_q->tc_index, dest_q->flow_id,
+ tc_id);
+
+ return NULL;
+ }
+
+ return dest_q;
+}
+
static int
-dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
- struct dpaa2_dev_flow *flow,
+dpaa2_flow_fs_action_config(struct dpaa2_dev_priv *priv,
+ struct dpaa2_generic_flow *flow,
const struct rte_flow_action *rte_action)
{
+ struct rte_dpaa2_device *dpaa2_dev;
struct rte_eth_dev *dest_dev;
struct dpaa2_dev_priv *dest_priv;
const struct rte_flow_action_queue *dest_queue;
struct dpaa2_queue *dest_q;
uint64_t flc = 0;
-
- memset(&flow->fs_action_cfg, 0,
- sizeof(struct dpni_fs_action_cfg));
- flow->action_type = rte_action->type;
-
- if (flow->action_type == RTE_FLOW_ACTION_TYPE_QUEUE) {
- dest_queue = rte_action->conf;
- if (dest_queue->index >= MAX_RX_QUEUES ||
- !priv->rx_vq[dest_queue->index]) {
- DPAA2_PMD_ERR("Invalid FSQ index(%d)",
- dest_queue->index);
-
+ uint16_t num_tokens;
+ struct dpaa2_dev_flow_fs_action *fs_action;
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+
+ fs_action = &flow->flow_action.fs_action;
+ if (fs_action->action_type != RTE_FLOW_ACTION_TYPE_END) {
+ if (dpaa2_dev->bus_info->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV &&
+ (fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
+ fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR)) {
+ if (rte_action->type != RTE_FLOW_ACTION_TYPE_PORT_ID &&
+ rte_action->type != RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR) {
+ DPAA2_PMD_ERR("Redirect action can't mix with other action(%d)",
+ rte_action->type);
+ return -EINVAL;
+ }
+ } else {
+ DPAA2_PMD_ERR("Single action support only with action(%d)",
+ fs_action->action_type);
return -EINVAL;
}
- dest_q = priv->rx_vq[dest_queue->index];
- if (flow->tc_id != dest_q->tc_index) {
- DPAA2_PMD_ERR("RXQ[%d](%d.%d) not in TC[%d]",
- dest_queue->index,
- dest_q->tc_index, dest_q->flow_id,
- flow->tc_id);
+ }
+ fs_action->action_type = rte_action->type;
+ if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ dest_queue = rte_action->conf;
+ dest_q = dpaa2_flow_queue_action_to_queue(priv,
+ flow->tc_id, dest_queue->index);
+ if (!dest_q)
return -EINVAL;
- }
- flow->fs_action_cfg.options =
+
+ fs_action->fs_action_cfg.options =
DPNI_FS_OPT_SET_FLC | DPNI_FS_OPT_SET_STASH_CONTROL;
if (dest_q->data_stashing_off) {
@@ -4430,10 +3697,10 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
flc |= ((uint64_t)1) << DPAA2_FS_FLC_FS_MARK_OFFSET;
flc |= ((uint64_t)dest_q->tc_index) << DPAA2_FS_FLC_TC_OFFSET;
flc |= ((uint64_t)dest_q->flow_id) << DPAA2_FS_FLC_FLOW_OFFSET;
- flow->fs_action_cfg.flc = flc;
- flow->fs_action_cfg.flow_id = dest_q->flow_id;
- } else if (flow->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
- flow->action_type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) {
+ fs_action->fs_action_cfg.flc = flc;
+ fs_action->fs_action_cfg.flow_id = dest_q->flow_id;
+ } else if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
+ fs_action->action_type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) {
dest_dev = dpaa2_flow_redirect_dev(priv, rte_action);
if (!dest_dev) {
DPAA2_PMD_ERR("Invalid device to redirect");
@@ -4441,15 +3708,34 @@ dpaa2_configure_flow_fs_action(struct dpaa2_dev_priv *priv,
}
dest_priv = dest_dev->data->dev_private;
- dest_q = dest_priv->tx_vq[0];
- flow->fs_action_cfg.options =
- DPNI_FS_OPT_REDIRECT_TO_DPNI_TX;
- flow->fs_action_cfg.num_tokens = 1;
- flow->fs_action_cfg.redir_tokens[0] =
- dest_priv->token;
- flow->fs_action_cfg.flow_id = dest_q->flow_id;
- } else if (flow->action_type == RTE_FLOW_ACTION_TYPE_DROP) {
- flow->fs_action_cfg.options = DPNI_FS_OPT_DISCARD;
+ num_tokens = fs_action->fs_action_cfg.num_tokens;
+ if (num_tokens >= DPNI_FS_REDIR_MAX_NUM)
+ return -EINVAL;
+ fs_action->fs_action_cfg.redir_tokens[num_tokens] = dest_priv->token;
+ strlcpy(fs_action->dst_name[num_tokens],
+ dest_priv->eth_dev->data->name,
+ sizeof(fs_action->dst_name[num_tokens]));
+ fs_action->fs_action_cfg.num_tokens++;
+ if (fs_action->fs_action_cfg.num_tokens > 1) {
+ if (!(priv->options & DPNI_OPT_V8_HAS_REPLICATION)) {
+ DPAA2_PMD_ERR("0x%08x missed in DPNI creating options(0x%08x)",
+ DPNI_OPT_V8_HAS_REPLICATION, priv->options);
+ return -ENOTSUP;
+ }
+ fs_action->fs_action_cfg.options =
+ DPNI_FS_OPT_REDIRECT_TO_MULTIPLE_DPNI_TX;
+ } else {
+ fs_action->fs_action_cfg.options =
+ DPNI_FS_OPT_REDIRECT_TO_DPNI_TX;
+ }
+ } else if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_DROP) {
+ fs_action->fs_action_cfg.options = DPNI_FS_OPT_DISCARD;
+ } else if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_RSS) {
+ /** Do nothing...*/
+ } else {
+ DPAA2_PMD_ERR("Flow action(%d) not supported!",
+ fs_action->action_type);
+ return -ENOTSUP;
}
return 0;
@@ -4477,41 +3763,104 @@ dpaa2_flow_clear_fs_table(struct dpaa2_dev_priv *priv,
uint8_t tc_id)
{
struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
- int need_clear = 0, ret;
+ int fs_num = 0, ret, rss_num = 0;
struct fsl_mc_io *dpni = priv->hw;
while (curr) {
- if (curr->tc_id == tc_id) {
- need_clear = 1;
- break;
+ if (curr->fs_flow && curr->fs_flow->tc_id == tc_id) {
+ if (curr->fs_flow->is_rss)
+ rss_num++;
+ else
+ fs_num++;
}
curr = LIST_NEXT(curr, next);
}
- if (need_clear) {
+ if (rss_num > 1) {
+ DPAA2_PMD_ERR("TC[%d] should have one RSS flow at most!",
+ tc_id);
+ return -EINVAL;
+ }
+ if (rss_num && fs_num) {
+ DPAA2_PMD_ERR("TC[%d] has RSS flow and %d FS flow(s)!",
+ tc_id, fs_num);
+ return -EINVAL;
+ }
+
+ if (fs_num) {
ret = dpni_clear_fs_entries(dpni, CMD_PRI_LOW,
priv->token, tc_id);
if (ret) {
DPAA2_PMD_ERR("TC[%d] clear failed", tc_id);
return ret;
}
+ priv->flow_profile.tc_profile[tc_id].entry_num = 0;
+ memset(priv->flow_profile.tc_profile[tc_id].entry_map,
+ 0, priv->fs_entries);
+ }
+
+ return 0;
+}
+
+static int
+dpaa2_flow_fs_table_set_default(struct dpaa2_dev_priv *priv,
+ uint8_t tc_id, int discard, uint16_t default_queue)
+{
+ int ret;
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpni_rx_dist_cfg *tc_cfg;
+ struct fsl_mc_io *dpni = priv->hw;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct dpaa2_queue *queue;
+ char mc_rev[1024];
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ snprintf(mc_rev, 1024, "MC rev(%d.%d.%d)",
+ RTE_FSL_MC_REV_MAJOR(dpaa2_dev->bus_info->mc_rev),
+ RTE_FSL_MC_REV_MINOR(dpaa2_dev->bus_info->mc_rev),
+ RTE_FSL_MC_REV_REVISION(dpaa2_dev->bus_info->mc_rev));
+ if (!tbl_profile->dpkg.num_extracts &&
+ dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
+ DPAA2_PMD_DEBUG("%s can't set miss action of FS table indepentently.",
+ mc_rev);
+ return 0;
+ }
+ tc_cfg = &tbl_profile->tc_cfg;
+ tc_cfg->enable = true;
+ if (discard) {
+ tbl_profile->default_drop = true;
+ tc_cfg->fs_miss_flow_id = DPNI_FS_MISS_ACTION_DROP;
+ } else {
+ queue = dpaa2_flow_queue_action_to_queue(priv, tc_id, default_queue);
+ if (!queue)
+ return -EINVAL;
+ tbl_profile->default_drop = false;
+ tbl_profile->default_queue.index = default_queue;
+ tc_cfg->fs_miss_flow_id = queue->flow_id;
+ }
+ ret = dpni_set_rx_fs_dist(dpni, CMD_PRI_LOW, priv->token, tc_cfg);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("%s: Failed(%d) to set default action of TC[%d]",
+ __func__, ret, tc_id);
+ return ret;
}
return 0;
}
static int
-dpaa2_configure_fs_rss_table(struct dpaa2_dev_priv *priv,
- uint8_t tc_id, uint16_t dist_size, int rss_dist)
+dpaa2_flow_fs_rss_table_config(struct dpaa2_dev_priv *priv,
+ uint8_t tc_id, int rss_dist)
{
- struct dpaa2_key_extract *tc_extract;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
uint8_t *key_cfg_buf;
- uint64_t key_cfg_iova;
int ret;
- struct dpni_rx_dist_cfg tc_cfg;
+ struct dpni_rx_dist_cfg *tc_cfg;
struct fsl_mc_io *dpni = priv->hw;
uint16_t entry_size;
uint16_t key_max_size;
+ struct dpaa2_queue *queue;
ret = dpaa2_flow_clear_fs_table(priv, tc_id);
if (ret < 0) {
@@ -4519,38 +3868,30 @@ dpaa2_configure_fs_rss_table(struct dpaa2_dev_priv *priv,
return ret;
}
- tc_extract = &priv->extract.tc_key_extract[tc_id];
- key_cfg_buf = priv->extract.tc_extract_param[tc_id];
- key_cfg_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(key_cfg_buf,
- DPAA2_EXTRACT_PARAM_MAX_SIZE);
- if (key_cfg_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for key cfg(%p)",
- __func__, key_cfg_buf);
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ if (!tbl_profile->dpkg.num_extracts)
+ return 0;
- return -ENOBUFS;
- }
+ key_cfg_buf = priv->flow_profile.tc_profile[tc_id].extract_param;
- key_max_size = tc_extract->key_profile.key_max_size;
+ key_max_size = tbl_profile->key_profile.key_max_size;
entry_size = dpaa2_flow_entry_size(key_max_size);
- dpaa2_flow_fs_extracts_log(priv, tc_id);
- ret = dpkg_prepare_key_cfg(&tc_extract->dpkg,
+ dpaa2_flow_extracts_log(priv, "Configure", tc_id);
+ ret = dpkg_prepare_key_cfg(&tbl_profile->dpkg,
key_cfg_buf);
if (ret < 0) {
DPAA2_PMD_ERR("TC[%d] prepare key failed", tc_id);
return ret;
}
- memset(&tc_cfg, 0, sizeof(struct dpni_rx_dist_cfg));
- tc_cfg.dist_size = dist_size;
- tc_cfg.key_cfg_iova = key_cfg_iova;
+ tc_cfg = &tbl_profile->tc_cfg;
if (rss_dist)
- tc_cfg.enable = true;
+ tc_cfg->enable = true;
else
- tc_cfg.enable = false;
- tc_cfg.tc = tc_id;
+ tc_cfg->enable = false;
ret = dpni_set_rx_hash_dist(dpni, CMD_PRI_LOW,
- priv->token, &tc_cfg);
+ priv->token, tc_cfg);
if (ret < 0) {
if (rss_dist) {
DPAA2_PMD_ERR("RSS TC[%d] set failed",
@@ -4563,13 +3904,27 @@ dpaa2_configure_fs_rss_table(struct dpaa2_dev_priv *priv,
return ret;
}
- if (rss_dist)
- return 0;
+ if (rss_dist) {
+ tbl_profile->is_rss = true;
+ goto successful_config;
+ }
- tc_cfg.enable = true;
- tc_cfg.fs_miss_flow_id = priv->default_flow;
- ret = dpni_set_rx_fs_dist(dpni, CMD_PRI_LOW,
- priv->token, &tc_cfg);
+ tbl_profile->is_rss = false;
+
+ tc_cfg->enable = true;
+ if (tbl_profile->default_drop) {
+ tc_cfg->fs_miss_flow_id = DPNI_FS_MISS_ACTION_DROP;
+ } else {
+ queue = dpaa2_flow_queue_action_to_queue(priv, tc_id,
+ tbl_profile->default_queue.index);
+ if (!queue) {
+ DPAA2_PMD_ERR("TC[%d] invalid default queue(%d)",
+ tc_id, tbl_profile->default_queue.index);
+ return -EINVAL;
+ }
+ tc_cfg->fs_miss_flow_id = queue->flow_id;
+ }
+ ret = dpni_set_rx_fs_dist(dpni, CMD_PRI_LOW, priv->token, tc_cfg);
if (ret < 0) {
DPAA2_PMD_ERR("TC[%d] FS configured failed", tc_id);
return ret;
@@ -4580,74 +3935,145 @@ dpaa2_configure_fs_rss_table(struct dpaa2_dev_priv *priv,
if (ret)
return ret;
+successful_config:
+ tbl_profile->enabled = true;
+
+ return 0;
+}
+
+static int
+dpaa2_flow_qos_table_set_default(struct dpaa2_dev_priv *priv,
+ int discard, uint8_t default_tc, uint16_t default_flow)
+{
+ int ret;
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpni_qos_tbl_cfg qos_cfg;
+ struct fsl_mc_io *dpni = priv->hw;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ char mc_rev[1024];
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ tbl_profile = &priv->flow_profile.qos_profile;
+ snprintf(mc_rev, 1024, "MC rev(%d.%d.%d)",
+ RTE_FSL_MC_REV_MAJOR(dpaa2_dev->bus_info->mc_rev),
+ RTE_FSL_MC_REV_MINOR(dpaa2_dev->bus_info->mc_rev),
+ RTE_FSL_MC_REV_REVISION(dpaa2_dev->bus_info->mc_rev));
+ if (!tbl_profile->dpkg.num_extracts &&
+ dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
+ DPAA2_PMD_DEBUG("%s can't set miss action of QoS table indepentently.",
+ mc_rev);
+ return 0;
+ }
+ if (default_flow < priv->dist_queues &&
+ dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_MISS_FLOW_ACTION_MC_REV) {
+ DPAA2_PMD_WARN("%s can't direct miss traffic to TC%d-flow%d by QoS table only.",
+ mc_rev, default_tc, default_flow);
+ return 0;
+ }
+
+ rte_memcpy(&qos_cfg, &tbl_profile->qos_cfg, sizeof(struct dpni_qos_tbl_cfg));
+ qos_cfg.key_cfg_iova = 0;
+ qos_cfg.default_tc = default_tc;
+ qos_cfg.default_flow_id = default_flow;
+ qos_cfg.discard_on_miss = discard ? true : false;
+ qos_cfg.set_default_flow_id = default_flow < priv->dist_queues ? true : false;
+
+ ret = dpni_set_qos_table(dpni, CMD_PRI_LOW, priv->token, &qos_cfg);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("%s: Failed(%d) to set default action of QoS",
+ __func__, ret);
+ return ret;
+ }
+ if (discard) {
+ tbl_profile->default_drop = true;
+ } else {
+ tbl_profile->default_drop = false;
+ tbl_profile->default_jump.group = default_tc;
+ }
+ rte_memcpy(&tbl_profile->qos_cfg.discard_on_miss,
+ &qos_cfg.discard_on_miss,
+ sizeof(struct dpni_qos_tbl_cfg) -
+ offsetof(struct dpni_qos_tbl_cfg, discard_on_miss));
+
return 0;
}
static int
-dpaa2_configure_qos_table(struct dpaa2_dev_priv *priv,
+dpaa2_flow_qos_table_config(struct dpaa2_dev_priv *priv,
int rss_dist)
{
- struct dpaa2_key_extract *qos_extract;
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
uint8_t *key_cfg_buf;
- uint64_t key_cfg_iova;
int ret;
- struct dpni_qos_tbl_cfg qos_cfg;
+ struct dpni_qos_tbl_cfg *qos_cfg;
struct fsl_mc_io *dpni = priv->hw;
uint16_t entry_size;
uint16_t key_max_size;
+ struct dpaa2_dev_flow *flow;
if (!rss_dist && priv->num_rx_tc <= 1) {
/* QoS table is effecitive for FS multiple TCs or RSS.*/
return 0;
}
- if (LIST_FIRST(&priv->flows)) {
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ if (!flow->qos_flow) {
+ flow = LIST_NEXT(flow, next);
+ continue;
+ }
+ /** Clear existing QoS table if there is qos flow configured.*/
ret = dpni_clear_qos_table(dpni, CMD_PRI_LOW,
priv->token);
if (ret < 0) {
- DPAA2_PMD_ERR("QoS table clear failed");
+ DPAA2_PMD_ERR("QoS table clear failed(%d)", ret);
return ret;
}
+ priv->flow_profile.qos_profile.entry_num = 0;
+ memset(priv->flow_profile.qos_profile.entry_map,
+ 0, priv->qos_entries);
+ break;
}
- qos_extract = &priv->extract.qos_key_extract;
- key_cfg_buf = priv->extract.qos_extract_param;
- key_cfg_iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(key_cfg_buf,
- DPAA2_EXTRACT_PARAM_MAX_SIZE);
- if (key_cfg_iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for key cfg(%p)",
- __func__, key_cfg_buf);
+ tbl_profile = &priv->flow_profile.qos_profile;
+ if (!tbl_profile->dpkg.num_extracts)
+ return 0;
- return -ENOBUFS;
- }
+ key_cfg_buf = priv->flow_profile.qos_profile.extract_param;
- key_max_size = qos_extract->key_profile.key_max_size;
+ key_max_size = tbl_profile->key_profile.key_max_size;
entry_size = dpaa2_flow_entry_size(key_max_size);
- dpaa2_flow_qos_extracts_log(priv);
+ dpaa2_flow_extracts_log(priv, "Configure", MAX_TCS);
- ret = dpkg_prepare_key_cfg(&qos_extract->dpkg,
+ ret = dpkg_prepare_key_cfg(&tbl_profile->dpkg,
key_cfg_buf);
if (ret < 0) {
DPAA2_PMD_ERR("QoS prepare extract failed");
return ret;
}
- memset(&qos_cfg, 0, sizeof(struct dpni_qos_tbl_cfg));
- qos_cfg.keep_entries = true;
- qos_cfg.key_cfg_iova = key_cfg_iova;
+ qos_cfg = &tbl_profile->qos_cfg;
if (rss_dist) {
- qos_cfg.discard_on_miss = true;
+ qos_cfg->discard_on_miss = true;
} else {
- qos_cfg.discard_on_miss = false;
- qos_cfg.default_tc = 0;
+ qos_cfg->default_tc = tbl_profile->default_jump.group;
+ if (tbl_profile->default_drop)
+ qos_cfg->discard_on_miss = true;
+ else
+ qos_cfg->discard_on_miss = false;
}
- ret = dpni_set_qos_table(dpni, CMD_PRI_LOW,
- priv->token, &qos_cfg);
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ if (dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV)
+ ret = dpni_set_qos_table_v2(dpni, CMD_PRI_LOW, priv->token, qos_cfg);
+ else
+ ret = dpni_set_qos_table(dpni, CMD_PRI_LOW, priv->token, qos_cfg);
if (ret < 0) {
- DPAA2_PMD_ERR("QoS table set failed");
+ DPAA2_PMD_ERR("QoS table set failed(%d)", ret);
return ret;
}
+ tbl_profile->enabled = true;
ret = dpaa2_flow_rule_add_all(priv, DPAA2_FLOW_QOS_TYPE,
entry_size, 0);
@@ -4697,207 +4123,468 @@ dpaa2_flow_item_convert(const struct rte_flow_item pattern[],
}
static int
-dpaa2_generic_flow_set(struct dpaa2_dev_flow *flow,
- struct rte_eth_dev *dev,
- const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error)
+dpaa2_flow_table_update(struct dpaa2_dev_priv *priv,
+ enum dpaa2_flow_dist_type dist_type, uint8_t tc_id, int is_rss)
{
- const struct rte_flow_action_rss *rss_conf;
- int is_keycfg_configured = 0, end_of_list = 0;
- int ret = 0, i = 0, j = 0;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct dpaa2_dev_flow *curr = LIST_FIRST(&priv->flows);
- uint16_t dist_size, key_size;
- struct dpaa2_key_extract *qos_key_extract;
- struct dpaa2_key_extract *tc_key_extract;
- struct rte_dpaa2_flow_item *dpaa2_pattern = NULL;
-
- ret = dpaa2_flow_verify_attr(priv, attr);
- if (ret)
- return ret;
+ int ret;
- ret = dpaa2_flow_verify_action(priv, attr, actions);
- if (ret)
- return ret;
+ if (dist_type == DPAA2_FLOW_FS_TYPE) {
+ ret = dpaa2_flow_fs_rss_table_config(priv, tc_id, is_rss);
+ if (ret)
+ return ret;
+ } else if (dist_type == DPAA2_FLOW_QOS_TYPE) {
+ ret = dpaa2_flow_qos_table_config(priv, false);
+ if (ret)
+ return ret;
+ } else {
+ DPAA2_PMD_ERR("Invalid dist type(%d)!", dist_type);
+ return -EINVAL;
+ }
- ret = dpaa2_flow_item_convert(pattern, &dpaa2_pattern,
- priv->sp_protocol);
- if (ret)
- return ret;
+ return 0;
+}
- /* Parse pattern list to get the matching parameters */
- while (!end_of_list) {
- switch (pattern[i].type) {
+static int
+dpaa2_flow_action_meter_mark_init(struct dpaa2_dev_priv *priv,
+ uint32_t mtr_id, struct rte_flow_action_meter_mark *meter_mark)
+{
+ struct dpaa2_dev_meter *meter;
+ struct dpaa2_dev_meter_profile *profile;
+ struct dpaa2_dev_meter_policy *policy;
+ int found = 0;
+
+ meter = LIST_FIRST(&priv->meters);
+ while (meter) {
+ if (meter->meter_id == mtr_id) {
+ found = 1;
+ break;
+ }
+ meter = LIST_NEXT(meter, next);
+ }
+
+ if (!found) {
+ DPAA2_PMD_ERR("Meter ID(%d) is not found!", mtr_id);
+ return -ENXIO;
+ }
+
+ found = 0;
+ profile = LIST_FIRST(&priv->profiles);
+ while (profile) {
+ if (profile->profile_id == meter->profile_id) {
+ found = 1;
+ break;
+ }
+ profile = LIST_NEXT(profile, next);
+ }
+ if (!found) {
+ DPAA2_PMD_ERR("Meter ID(%d)'s profile(%d) not exist!",
+ mtr_id, meter->profile_id);
+ return -ENXIO;
+ }
+
+ found = 0;
+ policy = LIST_FIRST(&priv->policies);
+ while (policy) {
+ if (policy->policy_id == meter->policy_id) {
+ found = 1;
+ break;
+ }
+ policy = LIST_NEXT(policy, next);
+ }
+ if (!found) {
+ /** Option.*/
+ DPAA2_PMD_WARN("Meter ID(%d)'s policy(%d) not exist!",
+ mtr_id, meter->policy_id);
+ policy = NULL;
+ }
+
+ meter_mark->profile = (void *)profile;
+ meter_mark->policy = (void *)policy;
+ meter_mark->color_mode = 1;
+
+ return 0;
+}
+
+static int
+dpaa2_flow_set_police_action(struct dpaa2_dev_priv *priv,
+ uint8_t tc_id, const struct rte_flow_action_meter_mark *meter_mark)
+{
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpni_rx_tc_policing_cfg policing_cfg = {0};
+ const struct dpaa2_dev_meter_profile *dpaa2_profile;
+ const struct dpaa2_dev_meter_policy *dpaa2_policy = NULL;
+ int ret;
+
+ if (!(priv->options & DPNI_OPT_HAS_POLICING)) {
+ DPAA2_PMD_ERR("POLICING(0x%08x) was not added in DPNI creating options(0x%08x)",
+ DPNI_OPT_HAS_POLICING, priv->options);
+ return -ENOTSUP;
+ }
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+
+ dpaa2_profile = (void *)meter_mark->profile;
+ if (!dpaa2_profile) {
+ DPAA2_PMD_ERR("Meter profile not specified!");
+ return -EINVAL;
+ }
+
+ /** Blind as default.*/
+ policing_cfg.options = 0;
+ if (meter_mark->color_mode)
+ policing_cfg.options = DPNI_POLICER_OPT_COLOR_AWARE;
+ if (meter_mark->policy)
+ dpaa2_policy = (void *)meter_mark->policy;
+ if (dpaa2_policy && dpaa2_policy->red_drop) {
+ policing_cfg.options |= DPNI_POLICER_OPT_DISCARD_RED;
+ } else if (!dpaa2_policy) {
+ /** Default: Red is discarded if no policy specified.*/
+ policing_cfg.options |= DPNI_POLICER_OPT_DISCARD_RED;
+ }
+
+ if (priv->flow_profile.mtr_flow[tc_id]) {
+ /** Update existing policer.*/
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ policing_cfg.options |= DPNI_POLICER_OPT_DO_NOT_RESET_COUNTERS;
+ if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_NOT_RESET_COUNTER_MC_REV)
+ DPAA2_PMD_WARN("The existing policer's counters will be cleaned.");
+ }
+
+ policing_cfg.mode = dpaa2_profile->mode;
+ if (policing_cfg.mode < DPNI_POLICER_MODE_NONE ||
+ policing_cfg.mode > DPNI_POLICER_MODE_RFC_4115) {
+ DPAA2_PMD_ERR("Invalid policer mode(%d)",
+ policing_cfg.mode);
+ return -EINVAL;
+ }
+ policing_cfg.units = dpaa2_profile->policer_unit;
+ if (policing_cfg.units < DPNI_POLICER_UNIT_BYTES_L3 ||
+ policing_cfg.units > DPNI_POLICER_UNIT_BYTES_L2_WITHOUT_FCS) {
+ DPAA2_PMD_ERR("Invalid policer units(%d)",
+ policing_cfg.units);
+ return -EINVAL;
+ }
+ policing_cfg.cir = dpaa2_profile->cir;
+ policing_cfg.cbs = dpaa2_profile->cbs;
+ policing_cfg.eir = dpaa2_profile->pir;
+ policing_cfg.ebs = dpaa2_profile->pbs;
+
+ if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
+ ret = dpni_set_rx_tc_policing_v1(priv->hw, CMD_PRI_LOW,
+ priv->token, tc_id, &policing_cfg);
+ } else {
+ ret = dpni_set_rx_tc_policing(priv->hw, CMD_PRI_LOW,
+ priv->token, tc_id, &policing_cfg);
+ }
+ DPAA2_PMD_INFO("%s RX TC%d policer configure %s.",
+ priv->eth_dev->data->name, tc_id,
+ ret ? "failed" : "successfully");
+ if (!ret)
+ priv->flow_profile.tc_mtr_profile[tc_id] = (void *)meter_mark->profile;
+
+ return ret;
+}
+
+static int
+dpaa2_flow_action_single_type_check(const struct rte_flow_action actions[],
+ enum rte_flow_action_type type)
+{
+ if (actions[0].type == type &&
+ actions[1].type == RTE_FLOW_ACTION_TYPE_END)
+ return true;
+
+ return false;
+}
+
+static int
+dpaa2_flow_qos_action_update(struct dpaa2_dev_priv *priv,
+ struct dpaa2_generic_flow *flow,
+ const struct rte_flow_action actions[], int new_flow)
+{
+ int end_of_list = 0, ret = 0, i = 0, set_jump = 0;
+ struct dpaa2_dev_flow_qos_action *qos_action;
+
+ RTE_SET_USED(priv);
+ qos_action = &flow->flow_action.qos_action;
+
+ while (!end_of_list) {
+ switch (actions[i].type) {
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ qos_action->action_jump_type = actions[i].type;
+ rte_memcpy(&qos_action->action_jump_cfg, actions[i].conf,
+ sizeof(struct rte_flow_action_jump));
+ set_jump++;
+ break;
+ case RTE_FLOW_ACTION_TYPE_END:
+ end_of_list = 1;
+ break;
+ default:
+ DPAA2_PMD_ERR("Invalid action[%d]'s type(%d)",
+ i, actions[i].type);
+ ret = -ENOTSUP;
+ goto end_action_set;
+ }
+ i++;
+ }
+
+end_action_set:
+ if (!set_jump && !ret && new_flow)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+dpaa2_flow_fs_action_update(struct dpaa2_dev_priv *priv,
+ struct dpaa2_generic_flow *fs_flow,
+ const struct rte_flow_action actions[])
+{
+ int end_of_list = 0, ret = 0, i = 0;
+ const struct rte_flow_action_meter *meter;
+ struct rte_flow_action_meter_mark meter_mark;
+ union dpaa2_dev_flow_action flow_action_rollback;
+
+ rte_memcpy(&flow_action_rollback, &fs_flow->flow_action,
+ sizeof(union dpaa2_dev_flow_action));
+ memset(&fs_flow->flow_action, 0, sizeof(union dpaa2_dev_flow_action));
+
+ while (!end_of_list) {
+ switch (actions[i].type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ ret = dpaa2_flow_fs_action_config(priv, fs_flow, &actions[i]);
+ if (ret)
+ goto end_action_set;
+
+ break;
+ case RTE_FLOW_ACTION_TYPE_METER_MARK:
+ rte_memcpy(&meter_mark, actions[i].conf,
+ sizeof(meter_mark));
+ ret = dpaa2_flow_set_police_action(priv,
+ fs_flow->tc_id, &meter_mark);
+ if (ret)
+ goto end_action_set;
+ break;
+ case RTE_FLOW_ACTION_TYPE_METER:
+ meter = actions[i].conf;
+ ret = dpaa2_flow_action_meter_mark_init(priv,
+ meter->mtr_id, &meter_mark);
+ if (ret)
+ goto end_action_set;
+ ret = dpaa2_flow_set_police_action(priv,
+ fs_flow->tc_id, &meter_mark);
+ if (ret)
+ goto end_action_set;
+
+ break;
+ case RTE_FLOW_ACTION_TYPE_PF:
+ /* Skip this action, have to add for vxlan*/
+ break;
+ case RTE_FLOW_ACTION_TYPE_END:
+ end_of_list = 1;
+ break;
+ default:
+ DPAA2_PMD_ERR("Invalid action[%d]'s type(%d)",
+ i, actions[i].type);
+ ret = -ENOTSUP;
+ goto end_action_set;
+ }
+ i++;
+ }
+
+end_action_set:
+ if (ret || i < 1) {
+ rte_memcpy(&fs_flow->flow_action, &flow_action_rollback,
+ sizeof(union dpaa2_dev_flow_action));
+ }
+ if (ret)
+ return ret;
+
+ if (i < 1) {
+ DPAA2_PMD_ERR("No action is set!");
+ return -EINVAL;
+ }
+ if (i > 2) {
+ DPAA2_PMD_WARN("Only last of %d actions is applied",
+ i - 1);
+ }
+
+ return 0;
+}
+
+static int
+dpaa2_flow_generic_extract_rule_set(struct dpaa2_generic_flow *flow,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[], int is_rss,
+ enum dpaa2_flow_dist_type dist_type, int update)
+{
+ int extract_cfg = 0, end_of_list = 0;
+ int ret = 0, i = 0;
+ struct dpaa2_dev_priv *priv = flow->priv;
+ uint16_t key_size;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct rte_dpaa2_flow_item *dpaa2_pattern = NULL;
+
+ ret = dpaa2_flow_item_convert(pattern, &dpaa2_pattern,
+ priv->sp_protocol);
+ if (ret)
+ return ret;
+
+ flow->ip_key = NET_PROT_NONE;
+ flow->ip_src = NET_PROT_NONE;
+ flow->ip_dst = NET_PROT_NONE;
+
+ if (dist_type == DPAA2_FLOW_QOS_TYPE)
+ tbl_profile = &priv->flow_profile.qos_profile;
+ else
+ tbl_profile = &priv->flow_profile.tc_profile[attr->group];
+
+ if (is_rss || (!is_rss && tbl_profile->is_rss)) {
+ RTE_ASSERT(dist_type == DPAA2_FLOW_FS_TYPE);
+ if (!update) {
+ dpaa2_flow_clean(priv->eth_dev, attr->group);
+ tbl_profile->rss_flow = NULL;
+ }
+ if (!is_rss)
+ tbl_profile->tc_cfg.dist_size = priv->dist_queues;
+ memset(&tbl_profile->dpkg, 0,
+ sizeof(struct dpkg_profile_cfg));
+ memset(&tbl_profile->key_profile, 0,
+ sizeof(struct dpaa2_key_profile));
+ }
+
+ /* Parse pattern list to get the matching parameters */
+ while (!end_of_list) {
+ switch (pattern[i].type) {
case RTE_FLOW_ITEM_TYPE_ETH:
- ret = dpaa2_configure_flow_eth(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_eth_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("ETH flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_VLAN:
- ret = dpaa2_configure_flow_vlan(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_vlan_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("vLan flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
- ret = dpaa2_configure_flow_ipv4(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_ipv4_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("IPV4 flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
- ret = dpaa2_configure_flow_ipv6(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_ipv6_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("IPV6 flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_ICMP:
- ret = dpaa2_configure_flow_icmp(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_icmp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("ICMP flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_UDP:
- ret = dpaa2_configure_flow_udp(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_udp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("UDP flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_TCP:
- ret = dpaa2_configure_flow_tcp(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_tcp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("TCP flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_SCTP:
- ret = dpaa2_configure_flow_sctp(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_sctp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("SCTP flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_ESP:
- ret = dpaa2_configure_flow_esp(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_esp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("ESP flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_AH:
- ret = dpaa2_configure_flow_ah(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_ah_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("AH flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_GRE:
- ret = dpaa2_configure_flow_gre(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_gre_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("GRE flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_VXLAN:
- ret = dpaa2_configure_flow_vxlan(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_vxlan_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("VXLAN flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_ECPRI:
- ret = dpaa2_configure_flow_ecpri(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_ecpri_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("ECPRI flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_GTP:
- ret = dpaa2_configure_flow_gtp(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_gtp_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("GTP flow config failed!");
- goto end_flow_set;
- }
- break;
- case RTE_FLOW_ITEM_TYPE_MPLS:
- ret = dpaa2_configure_flow_mpls(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
- if (ret) {
- DPAA2_PMD_ERR("MPLS flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
- case RTE_FLOW_ITEM_TYPE_PPPOES:
- ret = dpaa2_configure_flow_pppoe(flow,
- dev, attr, &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ case RTE_FLOW_ITEM_TYPE_GENEVE:
+ ret = dpaa2_flow_geneve_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
- DPAA2_PMD_ERR("PPPoE flow config failed!");
- goto end_flow_set;
+ DPAA2_PMD_ERR("GENEVE flow config failed!");
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_RAW:
- ret = dpaa2_configure_flow_raw(flow, dev, attr,
- &dpaa2_pattern[i],
- actions, error,
- &is_keycfg_configured);
+ ret = dpaa2_flow_raw_extract_rule_set(flow, attr,
+ &dpaa2_pattern[i], &extract_cfg, dist_type);
if (ret) {
DPAA2_PMD_ERR("RAW flow config failed!");
- goto end_flow_set;
+ goto end_extract_set;
}
break;
case RTE_FLOW_ITEM_TYPE_END:
@@ -4907,164 +4594,104 @@ dpaa2_generic_flow_set(struct dpaa2_dev_flow *flow,
DPAA2_PMD_ERR("Invalid flow item[%d] type(%d)",
i, pattern[i].type);
ret = -ENOTSUP;
- break;
+ goto end_extract_set;
}
i++;
}
- qos_key_extract = &priv->extract.qos_key_extract;
- key_size = qos_key_extract->key_profile.key_max_size;
- flow->qos_rule.key_size = dpaa2_flow_entry_size(key_size);
-
- tc_key_extract = &priv->extract.tc_key_extract[flow->tc_id];
- key_size = tc_key_extract->key_profile.key_max_size;
- flow->fs_rule.key_size = dpaa2_flow_entry_size(key_size);
-
- /* Let's parse action on matching traffic */
- end_of_list = 0;
- while (!end_of_list) {
- switch (actions[j].type) {
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
- case RTE_FLOW_ACTION_TYPE_PORT_ID:
- case RTE_FLOW_ACTION_TYPE_DROP:
- ret = dpaa2_configure_flow_fs_action(priv, flow,
- &actions[j]);
- if (ret)
- goto end_flow_set;
-
- /* Configure FS table first*/
- dist_size = priv->nb_rx_queues / priv->num_rx_tc;
- if (is_keycfg_configured & DPAA2_FLOW_FS_TYPE) {
- ret = dpaa2_configure_fs_rss_table(priv,
- flow->tc_id,
- dist_size,
- false);
- if (ret)
- goto end_flow_set;
- }
-
- /* Configure QoS table then.*/
- if (is_keycfg_configured & DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_configure_qos_table(priv, false);
- if (ret)
- goto end_flow_set;
- }
-
- if (priv->num_rx_tc > 1) {
- ret = dpaa2_flow_add_qos_rule(priv, flow);
- if (ret)
- goto end_flow_set;
- }
+ key_size = tbl_profile->key_profile.key_max_size;
+ flow->rule_cfg.key_size = dpaa2_flow_entry_size(key_size);
- if (flow->tc_index >= priv->fs_entries) {
- DPAA2_PMD_ERR("FS table with %d entries full",
- priv->fs_entries);
- return -1;
- }
+ if (!extract_cfg)
+ goto end_extract_set;
- ret = dpaa2_flow_add_fs_rule(priv, flow);
- if (ret)
- goto end_flow_set;
+ ret = dpaa2_flow_table_update(priv, dist_type, attr->group, is_rss);
- break;
- case RTE_FLOW_ACTION_TYPE_RSS:
- rss_conf = actions[j].conf;
- flow->action_type = RTE_FLOW_ACTION_TYPE_RSS;
-
- ret = dpaa2_distset_to_dpkg_profile_cfg(rss_conf->types,
- &tc_key_extract->dpkg);
- if (ret < 0) {
- DPAA2_PMD_ERR("TC[%d] distset RSS failed",
- flow->tc_id);
- goto end_flow_set;
- }
+end_extract_set:
+ if (dpaa2_pattern)
+ rte_free(dpaa2_pattern);
- dist_size = rss_conf->queue_num;
- if (is_keycfg_configured & DPAA2_FLOW_FS_TYPE) {
- ret = dpaa2_configure_fs_rss_table(priv,
- flow->tc_id,
- dist_size,
- true);
- if (ret)
- goto end_flow_set;
- }
+ return ret;
+}
- if (is_keycfg_configured & DPAA2_FLOW_QOS_TYPE) {
- ret = dpaa2_configure_qos_table(priv, true);
- if (ret)
- goto end_flow_set;
- }
+static inline int
+dpaa2_flow_verify_attr(struct dpaa2_dev_priv *priv,
+ const struct rte_flow_attr *attr, uint32_t *pgroup_type,
+ uint32_t *pgroup_id)
+{
+ int ret = 0;
+ uint32_t group_id, group_type;
- ret = dpaa2_flow_add_qos_rule(priv, flow);
- if (ret)
- goto end_flow_set;
+ group_type = RTE_DPAA2_FLOW_GROUP_TYPE_GET(attr->group);
+ group_id = RTE_DPAA2_FLOW_GROUP_ID_GET(attr->group);
+ if (group_id >= priv->num_rx_tc &&
+ group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW) {
+ group_type = RTE_DPAA2_QOS_GROUP_FLOW;
+ group_id = DPAA2_FLOW_QOS_GROUP_ID;
+ }
- ret = dpaa2_flow_add_fs_rule(priv, flow);
- if (ret)
- goto end_flow_set;
+ if (group_type == RTE_DPAA2_FS_GROUP_FLOW && !priv->fs_entries) {
+ DPAA2_PMD_ERR("No FS table created to support FS flow!");
+ ret = -EINVAL;
+ goto complete_verify;
+ }
- break;
- case RTE_FLOW_ACTION_TYPE_PF:
- /* Skip this action, have to add for vxlan */
- break;
- case RTE_FLOW_ACTION_TYPE_END:
- end_of_list = 1;
- break;
- default:
- DPAA2_PMD_ERR("Invalid action type");
+ if (group_type == RTE_DPAA2_QOS_GROUP_FLOW) {
+ if (attr->priority >= priv->qos_entries) {
+ DPAA2_PMD_ERR("Prior(%d) is out of range(%d)",
+ attr->priority, priv->qos_entries);
ret = -ENOTSUP;
- break;
+ goto complete_verify;
}
- j++;
+ goto complete_verify_group;
}
-end_flow_set:
- if (!ret) {
- /* New rules are inserted. */
- if (!curr) {
- LIST_INSERT_HEAD(&priv->flows, flow, next);
- } else {
- while (LIST_NEXT(curr, next))
- curr = LIST_NEXT(curr, next);
- LIST_INSERT_AFTER(curr, flow, next);
+ if (group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW ||
+ group_type == RTE_DPAA2_FS_GROUP_FLOW) {
+ if (group_id >= priv->num_rx_tc) {
+ DPAA2_PMD_ERR("Group(%d) is out of range(%d)",
+ group_id, priv->num_rx_tc);
+ ret = -ENOTSUP;
+ goto complete_verify;
+ }
+ if (group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW &&
+ (attr->priority + priv->fs_entries * group_id) >= priv->qos_entries) {
+ DPAA2_PMD_ERR("Prior(%d) + fs entries(%d) * group(%d) is out of range(%d)",
+ attr->priority, priv->fs_entries, group_id,
+ priv->qos_entries);
+ ret = -ENOTSUP;
+ goto complete_verify;
}
+ } else {
+ DPAA2_PMD_ERR("Invalid group type(%d)", group_type);
+ ret = -EINVAL;
+ goto complete_verify;
}
- rte_free(dpaa2_pattern);
-
- return ret;
-}
-
-static inline int
-dpaa2_dev_verify_attr(struct dpni_attr *dpni_attr,
- const struct rte_flow_attr *attr)
-{
- int ret = 0;
-
- if (unlikely(attr->group >= dpni_attr->num_rx_tcs)) {
- DPAA2_PMD_ERR("Group/TC(%d) is out of range(%d)",
- attr->group, dpni_attr->num_rx_tcs);
- ret = -ENOTSUP;
- }
- if (unlikely(attr->priority >= dpni_attr->fs_entries)) {
- DPAA2_PMD_ERR("Priority(%d) within group is out of range(%d)",
- attr->priority, dpni_attr->fs_entries);
- ret = -ENOTSUP;
- }
+complete_verify_group:
if (unlikely(attr->egress)) {
DPAA2_PMD_ERR("Egress flow configuration is not supported");
ret = -ENOTSUP;
+ goto complete_verify;
}
if (unlikely(!attr->ingress)) {
DPAA2_PMD_ERR("Ingress flag must be configured");
ret = -EINVAL;
}
+
+complete_verify:
+ if (!ret) {
+ if (pgroup_type)
+ *pgroup_type = group_type;
+ if (pgroup_id)
+ *pgroup_id = group_id;
+ }
+
return ret;
}
static inline int
-dpaa2_dev_verify_patterns(const struct rte_flow_item pattern[],
+dpaa2_flow_verify_patterns(const struct rte_flow_item pattern[],
int sp_support)
{
unsigned int i, j, is_found = 0;
@@ -5109,29 +4736,32 @@ dpaa2_dev_verify_patterns(const struct rte_flow_item pattern[],
}
static inline int
-dpaa2_dev_verify_actions(const struct rte_flow_action actions[])
+dpaa2_flow_check_actions_support(const struct rte_flow_action actions[],
+ const enum rte_flow_action_type supported[], uint16_t len)
{
unsigned int i, j, is_found = 0;
- int ret = 0;
for (j = 0; actions[j].type != RTE_FLOW_ACTION_TYPE_END; j++) {
- for (i = 0; i < RTE_DIM(dpaa2_supported_action_type); i++) {
- if (dpaa2_supported_action_type[i] == actions[j].type) {
+ is_found = 0;
+ for (i = 0; i < len; i++) {
+ if (supported[i] == actions[j].type) {
is_found = 1;
break;
}
}
- if (!is_found) {
- ret = -ENOTSUP;
- break;
- }
+ if (!is_found)
+ return false;
}
for (j = 0; actions[j].type != RTE_FLOW_ACTION_TYPE_END; j++) {
if (actions[j].type != RTE_FLOW_ACTION_TYPE_DROP &&
- !actions[j].conf)
- ret = -EINVAL;
+ !actions[j].conf) {
+ DPAA2_PMD_ERR("No config for actions[%d].type(%d)",
+ j, actions[j].type);
+ return false;
+ }
}
- return ret;
+
+ return true;
}
static int
@@ -5142,238 +4772,1649 @@ dpaa2_flow_validate(struct rte_eth_dev *dev,
struct rte_flow_error *error)
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct dpni_attr dpni_attr;
- struct fsl_mc_io *dpni = (struct fsl_mc_io *)priv->hw;
- uint16_t token = priv->token;
+ uint16_t supported_len = 0;
int ret = 0;
+ uint32_t group_type;
+ const enum rte_flow_action_type *supported = NULL;
+ const char *err_str = NULL;
+ const void *cause = NULL;
+ enum rte_flow_error_type err_type = RTE_FLOW_ERROR_TYPE_NONE;
- memset(&dpni_attr, 0, sizeof(struct dpni_attr));
- ret = dpni_get_attributes(dpni, CMD_PRI_LOW, token, &dpni_attr);
- if (ret < 0) {
- DPAA2_PMD_ERR("Get dpni@%d attribute failed(%d)",
- priv->hw_id, ret);
- rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_ATTR,
- flow_attr, "invalid");
- return ret;
+ /* Verify input attributes */
+ ret = dpaa2_flow_verify_attr(priv, flow_attr, &group_type, NULL);
+ if (ret) {
+ err_str = "Invalid attributes are given";
+ cause = flow_attr;
+ err_type = RTE_FLOW_ERROR_TYPE_ATTR;
+ goto invalid_params;
}
- /* Verify input attributes */
- ret = dpaa2_dev_verify_attr(&dpni_attr, flow_attr);
- if (ret < 0) {
- DPAA2_PMD_ERR("Invalid attributes are given");
- rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_ATTR,
- flow_attr, "invalid");
- goto not_valid_params;
- }
- /* Verify input pattern list */
- ret = dpaa2_dev_verify_patterns(pattern, priv->sp_protocol);
- if (ret < 0) {
- DPAA2_PMD_ERR("Invalid pattern list is given");
- rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_ITEM,
- pattern, "invalid");
- goto not_valid_params;
+ if (pattern) {
+ /* Verify input pattern list */
+ ret = dpaa2_flow_verify_patterns(pattern, priv->sp_protocol);
+ if (ret) {
+ err_str = "Invalid pattern list is given";
+ cause = pattern;
+ err_type = RTE_FLOW_ERROR_TYPE_ITEM;
+ goto invalid_params;
+ }
+ }
+
+ supported = dpaa2_supported_meter_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_meter_action_type);
+ if (dpaa2_flow_check_actions_support(actions, supported, supported_len)) {
+ DPAA2_PMD_DEBUG("This is meter flow.");
+ if (group_type == RTE_DPAA2_QOS_GROUP_FLOW) {
+ err_str = "Meter flow's type can't be QoS flow.";
+ cause = actions;
+ err_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ ret = -EPERM;
+ goto invalid_params;
+ }
+ if (actions[1].type != RTE_FLOW_ACTION_TYPE_END) {
+ err_str = "Meter flow can't support multi-actions.";
+ cause = actions;
+ err_type = RTE_FLOW_ERROR_TYPE_ACTION_NUM;
+ ret = -EPERM;
+ goto invalid_params;
+ }
+ if (pattern)
+ DPAA2_PMD_WARN("Meter flow ingores flow items!");
+ return 0;
}
+
/* Verify input action list */
- ret = dpaa2_dev_verify_actions(actions);
- if (ret < 0) {
- DPAA2_PMD_ERR("Invalid action list is given");
- rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_ACTION,
- actions, "invalid");
- goto not_valid_params;
+ if (group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW ||
+ group_type == RTE_DPAA2_FS_GROUP_FLOW) {
+ supported = dpaa2_supported_fs_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_fs_action_type);
+ } else {
+ supported = dpaa2_supported_qos_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_qos_action_type);
+ }
+ if (!dpaa2_flow_check_actions_support(actions, supported, supported_len)) {
+ err_str = "Invalid action list is given";
+ cause = actions;
+ err_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ ret = -ENOTSUP;
+ goto invalid_params;
}
-not_valid_params:
+
+invalid_params:
+ if (err_str)
+ DPAA2_PMD_ERR("%s: %s", __func__, err_str);
+ rte_flow_error_set(error, -ret, err_type, cause, err_str);
+
return ret;
}
-static struct rte_flow *
-dpaa2_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
- const struct rte_flow_item pattern[],
- const struct rte_flow_action actions[],
- struct rte_flow_error *error)
+static int
+dpaa2_flow_is_ip_addr_extract(const struct dpkg_extract *extract)
{
- struct dpaa2_dev_flow *flow = NULL;
- struct dpaa2_dev_priv *priv = dev->data->dev_private;
- int ret;
- uint64_t iova;
+ enum dpkg_extract_type type = extract->type;
+ enum net_prot prot = extract->extract.from_hdr.prot;
+ uint32_t field = extract->extract.from_hdr.field;
- dpaa2_flow_control_log =
- getenv("DPAA2_FLOW_CONTROL_LOG");
+ if (type == DPKG_EXTRACT_FROM_HDR && prot == NET_PROT_IP &&
+ (field == NH_FLD_IP_SRC || field == NH_FLD_IP_DST))
+ return true;
- flow = rte_zmalloc(NULL, sizeof(struct dpaa2_dev_flow),
- RTE_CACHE_LINE_SIZE);
- if (!flow) {
- DPAA2_PMD_ERR("Failure to allocate memory for flow");
- goto mem_failure;
- }
+ return false;
+}
- /* Allocate DMA'ble memory to write the qos rules */
- flow->qos_key_addr = rte_zmalloc(NULL,
- DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE, RTE_CACHE_LINE_SIZE);
- if (!flow->qos_key_addr) {
- DPAA2_PMD_ERR("Memory allocation failed");
- goto mem_failure;
- }
- iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->qos_key_addr,
- DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
- if (iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for qos key(%p)",
- __func__, flow->qos_key_addr);
- goto mem_failure;
+static int
+dpaa2_flow_ip_addr_extract_pos(uint32_t field,
+ const struct dpaa2_key_profile *profile)
+{
+ if (field != NH_FLD_IP_SRC && field != NH_FLD_IP_DST)
+ return -EINVAL;
+ if (profile->ip_addr_extracts[0].field == field)
+ return 0;
+ else if (profile->ip_addr_extracts[1].field == field)
+ return 1;
+ else
+ return -ENXIO;
+}
+
+static int
+dpaa2_flow_key_offset_size(struct dpaa2_dev_flow *flow,
+ struct dpaa2_flow_tbl_profile *tbl_profile, uint8_t idx,
+ uint8_t *offset, uint8_t *size,
+ enum dpaa2_flow_dist_type type)
+{
+ struct dpkg_profile_cfg *dpkg = &tbl_profile->dpkg;
+ struct dpaa2_key_profile *profile = &tbl_profile->key_profile;
+ int pos, prev_ip_addr_pos;
+ uint32_t field;
+ uint8_t ip_addr_size = 0, ip_addr_offset = 0;
+ struct dpaa2_generic_flow *_flow;
+
+ _flow = type == DPAA2_FLOW_QOS_TYPE ? flow->qos_flow : flow->fs_flow;
+
+ if (_flow->ip_key == NET_PROT_IPV4)
+ ip_addr_size = NH_FLD_IPV4_ADDR_SIZE;
+ else if (_flow->ip_key == NET_PROT_IPV6)
+ ip_addr_size = NH_FLD_IPV6_ADDR_SIZE;
+
+ field = dpkg->extracts[idx].extract.from_hdr.field;
+ prev_ip_addr_pos = dpaa2_extract_prev_ip_addr_pos(profile);
+ if (prev_ip_addr_pos >= 0) {
+ ip_addr_offset =
+ profile->key_offset[prev_ip_addr_pos] +
+ profile->key_size[prev_ip_addr_pos];
}
- flow->qos_rule.key_iova = iova;
- flow->qos_mask_addr = rte_zmalloc(NULL,
- DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE, RTE_CACHE_LINE_SIZE);
- if (!flow->qos_mask_addr) {
- DPAA2_PMD_ERR("Memory allocation failed");
- goto mem_failure;
+ if (dpaa2_flow_is_ip_addr_extract(&dpkg->extracts[idx])) {
+ pos = dpaa2_flow_ip_addr_extract_pos(field, profile);
+ if (!ip_addr_size)
+ return -EINVAL;
+ *offset = ip_addr_offset + ip_addr_size * pos;
+ *size = ip_addr_size;
+ } else {
+ *offset = profile->key_offset[idx];
+ *size = profile->key_size[idx];
}
- iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->qos_mask_addr,
- DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
- if (iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for qos mask(%p)",
- __func__, flow->qos_mask_addr);
+
+ return 0;
+}
+
+static int
+dpaa2_flow_rule_data_valid(struct dpaa2_dev_flow *flow,
+ uint8_t key_offset, uint8_t key_size,
+ enum dpaa2_flow_dist_type type)
+{
+ uint8_t *mask, i;
+ int valid = 0;
+ struct dpaa2_generic_flow *_flow;
+
+ _flow = type == DPAA2_FLOW_QOS_TYPE ? flow->qos_flow : flow->fs_flow;
+ mask = _flow->mask_addr + key_offset;
+
+ for (i = 0; i < key_size; i++) {
+ if (mask[i]) {
+ valid = 1;
+ break;
+ }
+ }
+
+ return valid;
+}
+
+static void
+dpaa2_flow_remove_invalid_rule_data(struct dpaa2_dev_priv *priv,
+ enum dpaa2_flow_dist_type type, uint8_t tc_id,
+ uint8_t key_offset, uint8_t key_size)
+{
+ struct dpaa2_dev_flow *flow, *next;
+ struct dpaa2_generic_flow *_flow;
+
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ next = LIST_NEXT(flow, next);
+ if (type == DPAA2_FLOW_QOS_TYPE) {
+ _flow = flow->qos_flow;
+ if (!_flow)
+ goto skip_update_flow;
+ } else {
+ _flow = flow->fs_flow;
+ if (!_flow)
+ goto skip_update_flow;
+ if (_flow->tc_id != tc_id)
+ goto skip_update_flow;
+ }
+
+ if (key_offset < _flow->rule_size) {
+ memmove(_flow->key_addr + key_offset,
+ _flow->key_addr + key_offset + key_size,
+ _flow->rule_size - (key_offset + key_size));
+ memmove(_flow->mask_addr + key_offset,
+ _flow->mask_addr + key_offset + key_size,
+ _flow->rule_size - (key_offset + key_size));
+ _flow->rule_size -= key_size;
+ }
+ if (_flow->rule_cfg.key_size > _flow->rule_size) {
+ memset(_flow->key_addr + _flow->rule_size, 0,
+ _flow->rule_cfg.key_size - _flow->rule_size);
+ memset(_flow->mask_addr + _flow->rule_size, 0,
+ _flow->rule_cfg.key_size - _flow->rule_size);
+ }
+
+skip_update_flow:
+ flow = next;
+ }
+}
+
+static int
+dpaa2_flow_remove_invalid_extract(struct rte_eth_dev *dev,
+ enum dpaa2_flow_dist_type type, uint8_t tc_id)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_flow *flow, *next;
+ uint8_t i, key_offset, key_size, j, remove_conut = 0;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct dpkg_profile_cfg *dpkg;
+ struct dpaa2_key_profile *key_profile;
+ int valid[DPKG_MAX_NUM_OF_EXTRACTS], update = 0, ret, ipaddr;
+ enum net_prot prot;
+ uint32_t field, start, end;
+ uint32_t ip_src_size, ip_dst_size;
+ struct dpaa2_generic_flow *_flow;
+
+ if (type == DPAA2_FLOW_QOS_TYPE &&
+ priv->num_rx_tc <= 1) {
+ /** No QoS table.*/
+ return 0;
+ }
+
+ if (type == DPAA2_FLOW_QOS_TYPE)
+ tbl_profile = &priv->flow_profile.qos_profile;
+ else
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+
+ dpkg = &tbl_profile->dpkg;
+ key_profile = &tbl_profile->key_profile;
+ for (i = 0; i < key_profile->num; i++) {
+ valid[i] = 0;
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ next = LIST_NEXT(flow, next);
+ _flow = type == DPAA2_FLOW_QOS_TYPE ?
+ flow->qos_flow : flow->fs_flow;
+ if (!_flow)
+ goto skip_validation;
+
+ if (type == DPAA2_FLOW_FS_TYPE &&
+ _flow->tc_id != tc_id)
+ goto skip_validation;
+
+ key_offset = 0;
+ key_size = 0;
+
+ ret = dpaa2_flow_key_offset_size(flow, tbl_profile, i,
+ &key_offset, &key_size, type);
+ if (ret)
+ goto skip_validation;
+
+ if (dpaa2_flow_rule_data_valid(flow,
+ key_offset, key_size, type)) {
+ valid[i] = 1;
+ break;
+ }
+skip_validation:
+ flow = next;
+ }
+ if (!valid[i])
+ remove_conut++;
+ }
+
+ if (remove_conut > 0) {
+ if (type == DPAA2_FLOW_QOS_TYPE) {
+ dpaa2_flow_extracts_log(priv,
+ "Before removing extract(s)", MAX_TCS);
+ } else {
+ dpaa2_flow_extracts_log(priv,
+ "Before removing extract(s)", tc_id);
+ }
+ } else {
+ return 0;
+ }
+
+ i = key_profile->num - 1;
+ while (remove_conut) {
+ if (valid[i]) {
+ if (i == 0) {
+ DPAA2_PMD_ERR("Fatal: %d key(s) not removed!",
+ remove_conut);
+ return -EINVAL;
+ }
+ i--;
+ continue;
+ }
+ key_size = key_profile->key_size[i];
+ key_offset = key_profile->key_offset[i];
+
+ for (j = i + 1; j < key_profile->num; j++)
+ key_profile->key_offset[j] -= key_size;
+
+ ipaddr = 0;
+ if (dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR) {
+ prot = dpkg->extracts[i].extract.from_hdr.prot;
+ field = dpkg->extracts[i].extract.from_hdr.field;
+ ipaddr = dpaa2_flow_ip_address_extract(prot, field);
+ }
+ if (!ipaddr)
+ update += key_size;
+
+ if ((i + 1) < key_profile->num) {
+ memmove(&key_profile->key_offset[i],
+ &key_profile->key_offset[i + 1],
+ key_profile->num - i - 1);
+ memmove(&key_profile->key_size[i],
+ &key_profile->key_size[i + 1],
+ key_profile->num - i - 1);
+ memmove(&key_profile->prot_field[i],
+ &key_profile->prot_field[i + 1],
+ (key_profile->num - i - 1) *
+ sizeof(struct key_prot_field));
+
+ memmove(&dpkg->extracts[i], &dpkg->extracts[i + 1],
+ (key_profile->num - i - 1) *
+ sizeof(struct dpkg_extract));
+ }
+
+ dpaa2_flow_remove_invalid_rule_data(priv, type, tc_id,
+ key_offset, key_size);
+
+ key_profile->num--;
+ dpkg->num_extracts--;
+ i--;
+ remove_conut--;
+ };
+
+ key_profile->key_max_size -= update;
+ key_profile->key_max_size -=
+ (key_profile->ip_addr_extracts[0].max_size +
+ key_profile->ip_addr_extracts[1].max_size);
+
+ key_profile->l4_sp_present = 0;
+ key_profile->l4_dp_present = 0;
+ memset(key_profile->ip_addr_extracts, 0,
+ sizeof(struct dpaa2_ip_addr_extract) * 2);
+
+ if (key_profile->num >= 2)
+ start = key_profile->num - 2;
+ else if (key_profile->num >= 1)
+ start = key_profile->num - 1;
+ else
+ goto skip_ip_addr_extract;
+ end = key_profile->num;
+
+ j = 0;
+ for (i = start; i < end; i++) {
+ if (dpaa2_flow_is_ip_addr_extract(&dpkg->extracts[i])) {
+ key_profile->ip_addr_extracts[j].field =
+ dpkg->extracts[i].extract.from_hdr.field;
+ key_profile->ip_addr_extracts[j].max_size =
+ sizeof(rte_be32_t);
+ j++;
+ }
+ }
+
+ ip_src_size = sizeof(rte_be32_t);
+ ip_dst_size = sizeof(rte_be32_t);
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ next = LIST_NEXT(flow, next);
+ _flow = type == DPAA2_FLOW_QOS_TYPE ?
+ flow->qos_flow : flow->fs_flow;
+ if (!_flow)
+ goto next_flow;
+
+ if (type == DPAA2_FLOW_FS_TYPE &&
+ _flow->tc_id != tc_id)
+ goto next_flow;
+
+ if (_flow->ip_key == NET_PROT_IPV6) {
+ if (_flow->ip_src == NET_PROT_IPV6)
+ ip_src_size = NH_FLD_IPV6_ADDR_SIZE;
+ if (_flow->ip_dst == NET_PROT_IPV6)
+ ip_dst_size = NH_FLD_IPV6_ADDR_SIZE;
+ }
+
+next_flow:
+ flow = next;
+ }
+ if (key_profile->ip_addr_extracts[0].field == NH_FLD_IP_SRC)
+ key_profile->ip_addr_extracts[0].max_size = ip_src_size;
+ else if (key_profile->ip_addr_extracts[0].field == NH_FLD_IP_DST)
+ key_profile->ip_addr_extracts[0].max_size = ip_dst_size;
+
+ if (key_profile->ip_addr_extracts[1].field == NH_FLD_IP_SRC)
+ key_profile->ip_addr_extracts[1].max_size = ip_src_size;
+ else if (key_profile->ip_addr_extracts[1].field == NH_FLD_IP_DST)
+ key_profile->ip_addr_extracts[1].max_size = ip_dst_size;
+
+ key_profile->key_max_size +=
+ (key_profile->ip_addr_extracts[0].max_size +
+ key_profile->ip_addr_extracts[1].max_size);
+
+skip_ip_addr_extract:
+ for (i = 0; i < dpkg->num_extracts; i++) {
+ if ((dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_TCP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_TCP_PORT_SRC) ||
+ (dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_UDP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_UDP_PORT_SRC) ||
+ (dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_SCTP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_SCTP_PORT_SRC)) {
+ key_profile->l4_sp_present = 1;
+ key_profile->l4_sp_extract_idx = i;
+ key_profile->l4_sp_key_offset =
+ key_profile->key_offset[i];
+ } else if ((dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_TCP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_TCP_PORT_DST) ||
+ (dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_UDP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_UDP_PORT_DST) ||
+ (dpkg->extracts[i].type == DPKG_EXTRACT_FROM_HDR &&
+ dpkg->extracts[i].extract.from_hdr.prot ==
+ NET_PROT_SCTP &&
+ dpkg->extracts[i].extract.from_hdr.field ==
+ NH_FLD_SCTP_PORT_DST)) {
+ key_profile->l4_dp_present = 1;
+ key_profile->l4_dp_extract_idx = i;
+ key_profile->l4_dp_key_offset =
+ key_profile->key_offset[i];
+ }
+ }
+
+ if (type == DPAA2_FLOW_QOS_TYPE) {
+ dpaa2_flow_extracts_log(priv,
+ "After removing extract(s)", MAX_TCS);
+ } else {
+ dpaa2_flow_extracts_log(priv,
+ "After removing extract(s)", tc_id);
+ }
+
+ return update;
+}
+
+static int
+dpaa2_flow_remove_generic_entry(struct rte_eth_dev *dev,
+ struct dpaa2_generic_flow *flow, enum dpaa2_flow_dist_type type)
+{
+ int ret = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct fsl_mc_io *dpni = priv->hw;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct dpaa2_dev_flow_fs_action *fs_action;
+ uint8_t tc_id;
+
+ if (type == DPAA2_FLOW_QOS_TYPE)
+ goto remove_qos_flow;
+
+ fs_action = &flow->flow_action.fs_action;
+ tc_id = flow->tc_id;
+ switch (fs_action->action_type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ /* Then remove entry from FS table */
+ ret = dpni_remove_fs_entry(dpni, CMD_PRI_LOW, priv->token,
+ flow->tc_id, &flow->rule_cfg);
+ if (ret) {
+ DPAA2_PMD_ERR("Remove entry from FS[%d] failed(%d)",
+ flow->tc_id, ret);
+ dpaa2_flow_fs_entry_log("Delete failed", flow);
+ } else {
+ dpaa2_flow_fs_entry_log("Delete success", flow);
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ tbl_profile->entry_num--;
+ dpaa2_flow_entry_map_set(tbl_profile->entry_map,
+ flow->entry_index, 0);
+ }
+ break;
+ case RTE_FLOW_ACTION_TYPE_RSS:
+ /** Remove QoS entry later.*/
+ break;
+ default:
+ DPAA2_PMD_ERR("Action(%d) not supported",
+ fs_action->action_type);
+ ret = -ENOTSUP;
+ break;
+ }
+
+ return ret;
+
+remove_qos_flow:
+ if (priv->num_rx_tc > 1) {
+ /* Remove entry from QoS table first */
+ ret = dpni_remove_qos_entry(dpni, CMD_PRI_LOW,
+ priv->token, &flow->rule_cfg);
+ if (ret) {
+ DPAA2_PMD_ERR("Remove QoS entry failed(%d)", ret);
+ dpaa2_flow_qos_entry_log("Delete failed", flow);
+ /** Will not remove FS entry.*/
+ } else {
+ dpaa2_flow_qos_entry_log("Delete success", flow);
+ tbl_profile = &priv->flow_profile.qos_profile;
+ tbl_profile->entry_num--;
+ dpaa2_flow_entry_map_set(tbl_profile->entry_map,
+ flow->entry_index, 0);
+ }
+ }
+
+ return ret;
+}
+
+static int
+dpaa2_flow_generic_flow_destroy(struct rte_eth_dev *dev,
+ struct dpaa2_generic_flow *flow, enum dpaa2_flow_dist_type type)
+{
+ int ret = 0, update;
+ uint8_t tc_id = 0;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+
+ dpaa2_dump_extract_map(priv, "Start destroying flow", type,
+ type == DPAA2_FLOW_FS_TYPE ? flow->tc_id : 0);
+
+ ret = dpaa2_flow_remove_generic_entry(dev, flow, type);
+ if (ret)
+ return ret;
+
+ if (type == DPAA2_FLOW_FS_TYPE)
+ tc_id = flow->tc_id;
+
+ update = dpaa2_flow_remove_invalid_extract(dev, type, tc_id);
+ if (update > 0) {
+ if (type == DPAA2_FLOW_FS_TYPE) {
+ ret = dpaa2_flow_fs_rss_table_config(priv,
+ flow->tc_id, false);
+ if (ret)
+ DPAA2_PMD_ERR("Re-configure FS table failed(%d)", ret);
+ } else {
+ ret = dpaa2_flow_qos_table_config(priv, false);
+ if (ret)
+ DPAA2_PMD_ERR("Re-configure QoS table failed(%d)", ret);
+ }
+ }
+
+ if (!ret) {
+ dpaa2_dump_extract_map(priv, "End destroying flow",
+ type, tc_id);
+ }
+
+ return ret;
+}
+
+static int
+dpaa2_flow_qos_fs_action_set(const struct rte_flow_action actions[],
+ struct rte_flow_action qos_actions[],
+ struct rte_flow_action fs_actions[],
+ uint8_t *qos_action_num, uint8_t *fs_action_num)
+{
+ uint8_t qos_act_num = 0, fs_act_num = 0;
+ int end_of_list = 0, i = 0, is_qos_act, is_fs_act;
+ struct rte_flow_action action_check[2];
+
+ action_check[1].type = RTE_FLOW_ACTION_TYPE_END;
+ while (!end_of_list) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_END) {
+ if (qos_action_num)
+ *qos_action_num = qos_act_num;
+ if (fs_action_num)
+ *fs_action_num = fs_act_num;
+ qos_actions[qos_act_num].type = actions[i].type;
+ fs_actions[fs_act_num].type = actions[i].type;
+ end_of_list = 1;
+ continue;
+ }
+ action_check[0].type = actions[i].type;
+ action_check[0].conf = actions[i].conf;
+ is_qos_act = dpaa2_flow_check_actions_support(action_check,
+ dpaa2_supported_qos_action_type,
+ RTE_DIM(dpaa2_supported_qos_action_type));
+ is_fs_act = dpaa2_flow_check_actions_support(action_check,
+ dpaa2_supported_fs_action_type,
+ RTE_DIM(dpaa2_supported_fs_action_type));
+ if (!is_qos_act && !is_fs_act)
+ return -EINVAL;
+ if (is_qos_act && is_fs_act)
+ is_qos_act = false;
+ if (is_qos_act) {
+ qos_actions[qos_act_num].type = actions[i].type;
+ qos_actions[qos_act_num].conf = actions[i].conf;
+ qos_act_num++;
+ }
+ if (is_fs_act) {
+ fs_actions[fs_act_num].type = actions[i].type;
+ fs_actions[fs_act_num].conf = actions[i].conf;
+ fs_act_num++;
+ }
+ i++;
+ }
+
+ return 0;
+}
+
+static struct dpaa2_generic_flow *
+dpaa2_flow_generic_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error, enum dpaa2_flow_dist_type type,
+ int mix_extract, int is_rss)
+{
+ struct dpaa2_generic_flow *flow = NULL;
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int ret, idx = -1;
+ uint64_t iova, mc_rev;
+ uint8_t qos_action_num = 0, fs_action_num = 0;
+ struct rte_flow_action qos_actions[DPAA2_MAX_ACTION_PER_FLOW_NUM];
+ struct rte_flow_action fs_actions[DPAA2_MAX_ACTION_PER_FLOW_NUM];
+ struct rte_flow_action_jump action_jump;
+ const struct rte_flow_action_rss *action_rss;
+ const struct rte_flow_action_queue *action_q;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ struct dpaa2_queue *dir_rxq = NULL;
+
+ if (type != DPAA2_FLOW_QOS_TYPE && type != DPAA2_FLOW_FS_TYPE)
+ return NULL;
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ mc_rev = dpaa2_dev->bus_info->mc_rev;
+
+ if (actions) {
+ ret = dpaa2_flow_qos_fs_action_set(actions, qos_actions,
+ fs_actions, &qos_action_num, &fs_action_num);
+ if (ret) {
+ DPAA2_PMD_ERR("Failed(%d) to set qos/fs actions!", ret);
+ goto creation_error;
+ }
+ if (type == DPAA2_FLOW_QOS_TYPE &&
+ dpaa2_flow_action_single_type_check(fs_actions,
+ RTE_FLOW_ACTION_TYPE_QUEUE)) {
+ action_q = fs_actions[0].conf;
+ if (action_q->index >= dev->data->nb_rx_queues) {
+ DPAA2_PMD_ERR("direct queue index(%d) >= max(%d)",
+ action_q->index,
+ dev->data->nb_rx_queues);
+ ret = -EINVAL;
+ goto creation_error;
+ }
+ if ((mc_rev < DPAA2_QOS_FLOW_HW_ACTION_UPDATE_MC_REV ||
+ mc_rev < DPAA2_QOS_FLOW_TABLE_MISS_FLOW_ACTION_MC_REV) &&
+ mix_extract) {
+ /** We don't recommend steering flow by QoS table here
+ * because this version doesn't support QoS flow HW update
+ * and QoS table miss flow.
+ */
+ dir_rxq = NULL;
+ } else {
+ dir_rxq = dev->data->rx_queues[action_q->index];
+ }
+ }
+ }
+
+ if (type == DPAA2_FLOW_FS_TYPE && attr) {
+ if (attr->group >= priv->num_rx_tc) {
+ DPAA2_PMD_ERR("Flow group(%d) >= max(%d)",
+ attr->group, priv->num_rx_tc);
+ return NULL;
+ }
+ if (attr->priority >= priv->fs_entries && !is_rss) {
+ DPAA2_PMD_ERR("FS[%d].entry(%d) >= max(%d)",
+ attr->group, attr->priority, priv->fs_entries);
+ return NULL;
+ }
+ }
+
+ if (dir_rxq) {
+ qos_actions[0].type = RTE_FLOW_ACTION_TYPE_JUMP;
+ qos_actions[0].conf = &action_jump;
+ qos_actions[1].type = RTE_FLOW_ACTION_TYPE_END;
+ /** Ignore attr->group*/
+ if ((mix_extract || type == DPAA2_FLOW_FS_TYPE) &&
+ attr->group < priv->num_rx_tc &&
+ attr->group != dir_rxq->tc_index) {
+ DPAA2_PMD_ERR("Attribute group(%d) conflicts with dest queue's TC(%d)",
+ attr->group, dir_rxq->tc_index);
+ return NULL;
+ }
+ action_jump.group = dir_rxq->tc_index;
+ if (attr->group >= priv->num_rx_tc)
+ idx = attr->priority;
+ else if (!priv->fs_entries)
+ idx = action_jump.group * priv->dist_queues + attr->priority;
+ else
+ idx = action_jump.group * priv->fs_entries + attr->priority;
+ } else if (mix_extract && type == DPAA2_FLOW_QOS_TYPE) {
+ /** Move end action.*/
+ rte_memcpy(&qos_actions[qos_action_num + 1],
+ &qos_actions[qos_action_num],
+ sizeof(struct rte_flow_action));
+ /** Jump to FS[group]*/
+ qos_actions[qos_action_num].type = RTE_FLOW_ACTION_TYPE_JUMP;
+ qos_actions[qos_action_num].conf = &action_jump;
+ action_jump.group = attr->group;
+ idx = action_jump.group * priv->fs_entries + attr->priority;
+ qos_action_num++;
+ } else if (!mix_extract && type == DPAA2_FLOW_QOS_TYPE) {
+ if (!qos_action_num) {
+ DPAA2_PMD_ERR("No QoS flow's action!");
+ return NULL;
+ }
+ if (!is_rss) {
+ idx = dpaa2_flow_acquire_entry_idx(priv, attr->priority,
+ DPAA2_FLOW_QOS_TYPE, attr->group);
+ }
+ } else if (type == DPAA2_FLOW_FS_TYPE) {
+ ret = dpaa2_flow_verify_fs_action(priv, attr, actions);
+ if (ret)
+ return NULL;
+ if (!is_rss) {
+ idx = dpaa2_flow_acquire_entry_idx(priv, attr->priority,
+ DPAA2_FLOW_FS_TYPE, attr->group);
+ }
+ } else {
+ DPAA2_PMD_ERR("Invalid flow type(%d) or extract type(%d)",
+ type, mix_extract);
+ return NULL;
+ }
+ if (idx < 0 && !is_rss)
+ return NULL;
+ if (idx >= 0 && idx < RTE_ETH_DCB_NUM_USER_PRIORITIES && !is_rss &&
+ type == DPAA2_FLOW_QOS_TYPE && priv->dcb_flow[idx]) {
+ DPAA2_PMD_ERR("QoS flow[%d] added conflicts with dcb flow", idx);
+ return NULL;
+ }
+
+ if (!is_rss) {
+ ret = dpaa2_flow_verify_entry(priv, attr->group, idx, type);
+ if (ret)
+ return NULL;
+ if ((type == DPAA2_FLOW_QOS_TYPE && idx >= priv->qos_entries) ||
+ (type == DPAA2_FLOW_FS_TYPE && idx >= priv->fs_entries)) {
+ DPAA2_PMD_ERR("%s flow[%d] added more than max entries(%d)",
+ type == DPAA2_FLOW_QOS_TYPE ? "QoS" : "FS", idx,
+ type == DPAA2_FLOW_QOS_TYPE ? priv->qos_entries : priv->fs_entries);
+ return NULL;
+ }
+ }
+
+ dpaa2_dump_extract_map(priv, "Start creating flow",
+ type, attr->group);
+
+ flow = rte_zmalloc(NULL, sizeof(struct dpaa2_generic_flow),
+ RTE_CACHE_LINE_SIZE);
+ if (!flow) {
+ DPAA2_PMD_ERR("Failure to allocate memory for flow");
goto mem_failure;
}
- flow->qos_rule.mask_iova = iova;
+ flow->priv = priv;
- /* Allocate DMA'ble memory to write the FS rules */
- flow->fs_key_addr = rte_zmalloc(NULL,
+ /* Allocate DMA'ble memory to write the qos rules */
+ flow->key_addr = rte_zmalloc(NULL,
DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE, RTE_CACHE_LINE_SIZE);
- if (!flow->fs_key_addr) {
+ if (!flow->key_addr) {
DPAA2_PMD_ERR("Memory allocation failed");
goto mem_failure;
}
- iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->fs_key_addr,
+ iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->key_addr,
DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
if (iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for fs key(%p)",
- __func__, flow->fs_key_addr);
+ DPAA2_PMD_ERR("%s: No IOMMU map for key(%p)",
+ __func__, flow->key_addr);
goto mem_failure;
}
- flow->fs_rule.key_iova = iova;
+ flow->rule_cfg.key_iova = iova;
- flow->fs_mask_addr = rte_zmalloc(NULL,
+ flow->mask_addr = rte_zmalloc(NULL,
DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE, RTE_CACHE_LINE_SIZE);
- if (!flow->fs_mask_addr) {
+ if (!flow->mask_addr) {
DPAA2_PMD_ERR("Memory allocation failed");
goto mem_failure;
}
- iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->fs_mask_addr,
- DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
+ iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(flow->mask_addr,
+ DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
if (iova == RTE_BAD_IOVA) {
- DPAA2_PMD_ERR("%s: No IOMMU map for fs mask(%p)",
- __func__, flow->fs_mask_addr);
+ DPAA2_PMD_ERR("%s: No IOMMU map for mask(%p)",
+ __func__, flow->mask_addr);
goto mem_failure;
}
- flow->fs_rule.mask_iova = iova;
+ flow->rule_cfg.mask_iova = iova;
- priv->curr = flow;
+ flow->ip_key = NET_PROT_NONE;
- ret = dpaa2_generic_flow_set(flow, dev, attr, pattern, actions, error);
+ priv->cur_flow = flow;
+ if (is_rss) {
+ action_rss = actions[0].conf;
+ tbl_profile = &priv->flow_profile.tc_profile[attr->group];
+ tbl_profile->tc_cfg.dist_size = action_rss->queue_num;
+ }
+ ret = dpaa2_flow_generic_extract_rule_set(flow,
+ attr, pattern, is_rss, type, false);
if (ret < 0) {
- if (error && error->type > RTE_FLOW_ERROR_TYPE_ACTION)
+ if (error && error->type > RTE_FLOW_ERROR_TYPE_ACTION) {
rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- attr, "unknown");
- DPAA2_PMD_ERR("Create flow failed (%d)", ret);
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ attr, "unknown");
+ }
+ DPAA2_PMD_ERR("Create %s flow failed (%d)",
+ type == DPAA2_FLOW_QOS_TYPE ?
+ "QoS" : "FS", ret);
goto creation_error;
}
- priv->curr = NULL;
- return (struct rte_flow *)flow;
+ flow->entry_index = idx;
+ if (type == DPAA2_FLOW_QOS_TYPE) {
+ ret = dpaa2_flow_qos_action_update(priv, flow, qos_actions, true);
+ if (ret)
+ goto creation_error;
+ if (dir_rxq) {
+ flow->direct_queue = true;
+ flow->flow_id = dir_rxq->flow_id;
+ }
+ ret = dpaa2_flow_add_qos_rule(priv, flow);
+ if (ret)
+ goto creation_error;
+ } else {
+ flow->tc_id = attr->group;
+ ret = dpaa2_flow_fs_action_update(priv, flow, fs_actions);
+ if (ret)
+ goto creation_error;
+ if (!is_rss) {
+ ret = dpaa2_flow_add_fs_rule(priv, flow);
+ if (ret)
+ goto creation_error;
+ } else {
+ flow->is_rss = true;
+ }
+ }
+ priv->cur_flow = NULL;
+
+ dpaa2_dump_extract_map(priv, "Complete creating flow",
+ type, attr->group);
+
+ return flow;
mem_failure:
rte_flow_error_set(error, EPERM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
- "memory alloc");
+ "memory alloc");
creation_error:
if (flow) {
- rte_free(flow->qos_key_addr);
- rte_free(flow->qos_mask_addr);
- rte_free(flow->fs_key_addr);
- rte_free(flow->fs_mask_addr);
+ if (flow->key_addr)
+ rte_free(flow->key_addr);
+ if (flow->mask_addr)
+ rte_free(flow->mask_addr);
rte_free(flow);
}
- priv->curr = NULL;
+
+ return NULL;
+}
+
+static struct rte_flow *
+dpaa2_flow_create_meter_flow(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_action meter_action[])
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_flow *flow = NULL;
+ struct dpaa2_generic_flow *fs_flow = NULL;
+ int ret;
+
+ if (attr->group >= priv->num_rx_tc)
+ return NULL;
+
+ if (priv->flow_profile.mtr_flow[attr->group])
+ return NULL;
+
+ fs_flow = rte_zmalloc(NULL, sizeof(struct dpaa2_generic_flow),
+ RTE_CACHE_LINE_SIZE);
+ if (!fs_flow) {
+ DPAA2_PMD_ERR("Failure to allocate memory for flow");
+ return NULL;
+ }
+ flow = rte_zmalloc(NULL, sizeof(struct dpaa2_dev_flow),
+ RTE_CACHE_LINE_SIZE);
+ if (!flow) {
+ rte_free(fs_flow);
+ DPAA2_PMD_ERR("Failure to allocate memory for flow");
+ return NULL;
+ }
+ fs_flow->tc_id = attr->group;
+ fs_flow->priv = priv;
+ flow->fs_flow = fs_flow;
+ flow->priv = priv;
+
+ ret = dpaa2_flow_fs_action_update(priv, fs_flow, meter_action);
+ if (ret) {
+ rte_free(fs_flow);
+ rte_free(flow);
+ return NULL;
+ }
+ flow->is_meter_flow = true;
+ priv->flow_profile.mtr_flow[fs_flow->tc_id] = flow;
+
+ return (struct rte_flow *)flow;
+}
+
+static struct rte_flow *
+dpaa2_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct dpaa2_dev_flow *flow = NULL, *curr;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int ret, is_rss = false;
+ struct dpaa2_generic_flow *qos_flow = NULL;
+ struct dpaa2_generic_flow *fs_flow = NULL;
+ uint32_t group_id, group_type;
+ const struct rte_flow_action_rss *rss_conf;
+ struct rte_flow_item items[DPKG_MAX_NUM_OF_EXTRACTS + 1];
+ int rss_item = false, err_code = 0;
+ uint8_t spec_buf[1024];
+ enum rte_flow_error_type error_type = RTE_FLOW_ERROR_TYPE_NONE;
+ const char *err_str = NULL;
+ struct rte_flow_attr local_attr;
+
+ if (getenv("DPAA2_FLOW_CONTROL_LOG"))
+ dpaa2_flow_control_log = 1;
+
+ group_type = RTE_DPAA2_FLOW_GROUP_TYPE_GET(attr->group);
+ group_id = RTE_DPAA2_FLOW_GROUP_ID_GET(attr->group);
+ if (group_id >= priv->num_rx_tc &&
+ group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW) {
+ group_type = RTE_DPAA2_QOS_GROUP_FLOW;
+ group_id = DPAA2_FLOW_QOS_GROUP_ID;
+ }
+ rte_memcpy(&local_attr, attr, sizeof(struct rte_flow_attr));
+ local_attr.group = group_id;
+
+ if (actions) {
+ is_rss = dpaa2_flow_action_single_type_check(actions,
+ RTE_FLOW_ACTION_TYPE_RSS);
+ }
+ if (is_rss && group_type == RTE_DPAA2_QOS_GROUP_FLOW) {
+ err_str = "RSS doesn't supports QoS table!";
+ error_type = RTE_FLOW_ERROR_TYPE_ATTR;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ if (is_rss && group_type != RTE_DPAA2_FS_GROUP_FLOW) {
+ DPAA2_PMD_WARN("RSS distribution in TC%d.", group_id);
+ group_type = RTE_DPAA2_FS_GROUP_FLOW;
+ }
+ if (is_rss && !pattern) {
+ rss_conf = actions[0].conf;
+ ret = dpaa2_flow_rss_conf_to_item(rss_conf->types, items,
+ spec_buf, 1024);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("TC[%d] converts to RSS items failed(%d)",
+ group_id, ret);
+ error_type = RTE_FLOW_ERROR_TYPE_ITEM;
+ err_code = ret;
+ err_str = "Failed to converts RSS config type to RSS items!";
+ goto flow_failure;
+ }
+ rss_item = true;
+ }
+
+ if (dpaa2_flow_check_actions_support(actions,
+ dpaa2_supported_meter_action_type,
+ RTE_DIM(dpaa2_supported_meter_action_type))) {
+ /** Assume it's meter flow per TC.*/
+ if (group_type == RTE_DPAA2_QOS_GROUP_FLOW) {
+ error_type = RTE_FLOW_ERROR_TYPE_ATTR_GROUP;
+ err_code = -EINVAL;
+ err_str = "Failed to converts RSS config type to RSS items!";
+ goto flow_failure;
+ }
+ if (actions[1].type != RTE_FLOW_ACTION_TYPE_END) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION_NUM;
+ err_code = -EPERM;
+ err_str = "Meter flow can't support multi-actions.";
+ goto flow_failure;
+ }
+ flow = (void *)dpaa2_flow_create_meter_flow(dev, &local_attr, actions);
+ if (flow)
+ return (struct rte_flow *)flow;
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "Failed to create meter flow!";
+ goto flow_failure;
+ }
+
+ DPAA2_PMD_DEBUG("Port %s-%s: group type:%d, group id:%d, total RX TCs:%d",
+ dev->data->name, __func__, group_type, group_id, priv->num_rx_tc);
+ if (group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW) {
+ if (priv->qos_entries > 0) {
+ qos_flow = dpaa2_flow_generic_flow_create(dev, &local_attr, pattern,
+ actions, error, DPAA2_FLOW_QOS_TYPE, true, is_rss);
+ if (!qos_flow) {
+ err_str = "Failed to create QoS flow!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ }
+ if (priv->fs_entries > 0 && !(qos_flow && qos_flow->direct_queue)) {
+ fs_flow = dpaa2_flow_generic_flow_create(dev, &local_attr, pattern,
+ actions, error, DPAA2_FLOW_FS_TYPE, true, is_rss);
+ if (!fs_flow) {
+ if (qos_flow) {
+ ret = dpaa2_flow_generic_flow_destroy(dev, qos_flow,
+ DPAA2_FLOW_QOS_TYPE);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: QoS flow destroy failed(%d)!",
+ __func__, ret);
+ }
+ }
+ err_str = "Failed to create FS flow!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ }
+ } else if (group_type == RTE_DPAA2_QOS_GROUP_FLOW) {
+ if (!priv->qos_entries) {
+ err_str = "No QoS entry available!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ qos_flow = dpaa2_flow_generic_flow_create(dev, &local_attr, pattern,
+ actions, error, DPAA2_FLOW_QOS_TYPE, false, is_rss);
+ if (!qos_flow) {
+ err_str = "QoS flow create failed!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ } else if (group_type == RTE_DPAA2_FS_GROUP_FLOW) {
+ if (!priv->fs_entries && !is_rss) {
+ err_str = "No FS entry available!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ fs_flow = dpaa2_flow_generic_flow_create(dev, &local_attr,
+ rss_item ? items : pattern,
+ actions, error, DPAA2_FLOW_FS_TYPE, false, is_rss);
+ if (!fs_flow) {
+ err_str = "FS flow create failed!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ } else {
+ err_str = "Invalid flow attribut parameter!";
+ error_type = RTE_FLOW_ERROR_TYPE_ATTR;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ if (!qos_flow && !fs_flow) {
+ err_str = "Both QoS flow and FS flow are NULL!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -EINVAL;
+ goto flow_failure;
+ }
+ flow = rte_zmalloc(NULL, sizeof(struct dpaa2_dev_flow),
+ RTE_CACHE_LINE_SIZE);
+ if (!flow) {
+ err_str = "Failure to allocate memory for flow!";
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_code = -ENOMEM;
+ goto flow_failure;
+ }
+ flow->qos_flow = qos_flow;
+ flow->fs_flow = fs_flow;
+ flow->priv = priv;
+ if (fs_flow && fs_flow->is_rss)
+ priv->flow_profile.tc_profile[group_id].rss_flow = flow;
+
+ /* New rules are inserted. */
+ curr = LIST_FIRST(&priv->flows);
+ if (!curr) {
+ LIST_INSERT_HEAD(&priv->flows, flow, next);
+ } else {
+ while (LIST_NEXT(curr, next))
+ curr = LIST_NEXT(curr, next);
+ LIST_INSERT_AFTER(curr, flow, next);
+ }
+
+ return (struct rte_flow *)flow;
+
+flow_failure:
+ if (err_str)
+ DPAA2_PMD_ERR("%s: %s", __func__, err_str);
+ rte_flow_error_set(error, -err_code, error_type, NULL, err_str);
+
+ if (flow) {
+ if (flow->qos_flow && flow->qos_flow->key_addr)
+ rte_free(flow->qos_flow->key_addr);
+ if (flow->qos_flow && flow->qos_flow->mask_addr)
+ rte_free(flow->qos_flow->mask_addr);
+ if (flow->fs_flow && flow->fs_flow->key_addr)
+ rte_free(flow->fs_flow->key_addr);
+ if (flow->fs_flow && flow->fs_flow->mask_addr)
+ rte_free(flow->fs_flow->mask_addr);
+ if (flow->qos_flow)
+ rte_free(flow->qos_flow);
+ if (flow->fs_flow)
+ rte_free(flow->fs_flow);
+ rte_free(flow);
+ }
+ priv->cur_flow = NULL;
return NULL;
}
static int
-dpaa2_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *_flow,
- struct rte_flow_error *error)
+dpaa2_flow_destroy_meter_flow(struct rte_eth_dev *dev,
+ struct dpaa2_dev_flow *flow)
{
- int ret = 0;
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpni_rx_tc_policing_cfg cfg;
+ int ret;
+ uint8_t tc_id;
+
+ RTE_ASSERT(!flow->qos_flow && flow->fs_flow);
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+ tc_id = flow->fs_flow->tc_id;
+ RTE_ASSERT(priv->flow_profile.mtr_flow[tc_id] == flow);
+ memset(&cfg, 0, sizeof(cfg));
+ cfg.mode = DPNI_POLICER_MODE_NONE;
+ if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
+ ret = dpni_set_rx_tc_policing_v1(priv->hw, CMD_PRI_LOW,
+ priv->token, tc_id, &cfg);
+ } else {
+ ret = dpni_set_rx_tc_policing(priv->hw, CMD_PRI_LOW,
+ priv->token, tc_id, &cfg);
+ }
+ if (ret)
+ return ret;
+ priv->flow_profile.tc_mtr_profile[tc_id] = NULL;
+ priv->flow_profile.mtr_flow[tc_id] = NULL;
+ rte_free(flow->fs_flow);
+ rte_free(flow);
+
+ return 0;
+}
+
+static int
+dpaa2_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *_flow, struct rte_flow_error *error)
+{
+ int qos_ret = 0, fs_ret = 0;
struct dpaa2_dev_flow *flow;
struct dpaa2_dev_priv *priv = dev->data->dev_private;
- struct fsl_mc_io *dpni = priv->hw;
+ struct dpaa2_flow_tbl_profile *tbl_profile;
+ uint8_t tc_id;
+
+ RTE_SET_USED(error);
flow = (struct dpaa2_dev_flow *)_flow;
+ if (flow->is_meter_flow)
+ return dpaa2_flow_destroy_meter_flow(dev, flow);
- switch (flow->action_type) {
- case RTE_FLOW_ACTION_TYPE_QUEUE:
- case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT:
- case RTE_FLOW_ACTION_TYPE_PORT_ID:
- case RTE_FLOW_ACTION_TYPE_DROP:
- if (priv->num_rx_tc > 1) {
- /* Remove entry from QoS table first */
- ret = dpni_remove_qos_entry(dpni, CMD_PRI_LOW,
- priv->token,
- &flow->qos_rule);
- if (ret < 0) {
- DPAA2_PMD_ERR("Remove FS QoS entry failed");
- dpaa2_flow_qos_entry_log("Delete failed", flow,
- -1);
- abort();
- goto error;
+ LIST_REMOVE(flow, next);
+
+ if (flow->qos_flow) {
+ qos_ret = dpaa2_flow_generic_flow_destroy(dev, flow->qos_flow,
+ DPAA2_FLOW_QOS_TYPE);
+ if (qos_ret) {
+ DPAA2_PMD_ERR("%s: destroy QoS flow failed(%d)",
+ __func__, qos_ret);
+ }
+ if (flow->qos_flow->key_addr)
+ rte_free(flow->qos_flow->key_addr);
+ if (flow->qos_flow->mask_addr)
+ rte_free(flow->qos_flow->mask_addr);
+ rte_free(flow->qos_flow);
+ flow->qos_flow = NULL;
+ }
+ if (flow->fs_flow) {
+ fs_ret = dpaa2_flow_generic_flow_destroy(dev, flow->fs_flow,
+ DPAA2_FLOW_FS_TYPE);
+ if (fs_ret) {
+ DPAA2_PMD_ERR("%s: destroy FS flow failed(%d)",
+ __func__, fs_ret);
+ }
+ if (flow->fs_flow->key_addr)
+ rte_free(flow->fs_flow->key_addr);
+ if (flow->fs_flow->mask_addr)
+ rte_free(flow->fs_flow->mask_addr);
+ if (flow->fs_flow->is_rss) {
+ tc_id = flow->fs_flow->tc_id;
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ tbl_profile->rss_flow = NULL;
+ }
+ rte_free(flow->fs_flow);
+ flow->fs_flow = NULL;
+ }
+ rte_free(flow);
+
+ if (!qos_ret && !fs_ret)
+ return 0;
+
+ return qos_ret ? qos_ret : fs_ret;
+}
+
+static int
+dpaa2_flow_set_miss_actions(struct rte_eth_dev *dev,
+ uint32_t group, const struct rte_flow_group_attr *attr,
+ const struct rte_flow_action actions[], struct rte_flow_error *err)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ enum dpaa2_flow_dist_type flow_type = DPAA2_FLOW_NULL_TYPE;
+ int end_of_list = 0, i = 0, discard = false, err_code = 0;
+ const struct rte_flow_action_jump *action_jump = NULL;
+ const struct rte_flow_action_queue *dest_queue = NULL;
+ uint32_t group_id, group_type;
+ enum rte_flow_error_type error_type = RTE_FLOW_ERROR_TYPE_NONE;
+ const char *err_str = NULL;
+ struct dpaa2_queue *miss_rxq = NULL;
+ uint8_t qos_tc = 0xff;
+ uint16_t qos_flow = 0xffff;
+
+ RTE_SET_USED(attr);
+
+ group_type = RTE_DPAA2_FLOW_GROUP_TYPE_GET(group);
+ group_id = RTE_DPAA2_FLOW_GROUP_ID_GET(group);
+ if (group_id >= priv->num_rx_tc &&
+ group_type == RTE_DPAA2_ONE_LEVEL_GROUP_FLOW) {
+ group_type = RTE_DPAA2_QOS_GROUP_FLOW;
+ group_id = 0;
+ }
+ if (group_type == RTE_DPAA2_QOS_GROUP_FLOW)
+ flow_type = DPAA2_FLOW_QOS_TYPE;
+ else if (group_type == RTE_DPAA2_FS_GROUP_FLOW)
+ flow_type = DPAA2_FLOW_FS_TYPE;
+ else if (group >= priv->num_rx_tc)
+ flow_type = DPAA2_FLOW_QOS_TYPE;
+ else
+ flow_type = DPAA2_FLOW_FS_TYPE;
+
+ while (!end_of_list) {
+ switch (actions[i].type) {
+ case RTE_FLOW_ACTION_TYPE_QUEUE:
+ dest_queue = actions[i].conf;
+ if (dest_queue->index >= dev->data->nb_rx_queues) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION_CONF;
+ err_code = -EINVAL;
+ err_str = "Queue index overflows";
+ goto failure_to_set_miss_actions;
}
+ miss_rxq = priv->rx_vq[dest_queue->index];
+ break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ action_jump = actions[i].conf;
+ break;
+ case RTE_FLOW_ACTION_TYPE_DROP:
+ discard = true;
+ break;
+ case RTE_FLOW_ACTION_TYPE_END:
+ end_of_list = 1;
+ break;
+ default:
+ DPAA2_PMD_WARN("Invalid default action type[%d]:(%d)",
+ i, actions[i].type);
+ break;
}
+ i++;
+ }
- /* Then remove entry from FS table */
- ret = dpni_remove_fs_entry(dpni, CMD_PRI_LOW, priv->token,
- flow->tc_id, &flow->fs_rule);
- if (ret < 0) {
- DPAA2_PMD_ERR("Remove entry from FS[%d] failed",
- flow->tc_id);
- goto error;
+ if (flow_type == DPAA2_FLOW_QOS_TYPE) {
+ if (!priv->qos_entries) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "No QoS table available!";
+ goto failure_to_set_miss_actions;
}
- break;
- case RTE_FLOW_ACTION_TYPE_RSS:
- if (priv->num_rx_tc > 1) {
- ret = dpni_remove_qos_entry(dpni, CMD_PRI_LOW,
- priv->token,
- &flow->qos_rule);
- if (ret < 0) {
- DPAA2_PMD_ERR("Remove RSS QoS entry failed");
- goto error;
+ if (action_jump && miss_rxq) {
+ if (action_jump->group != miss_rxq->tc_index) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "Jump group conflicts to miss queue's TC";
+ goto failure_to_set_miss_actions;
}
}
- break;
- default:
- DPAA2_PMD_ERR("Action(%d) not supported", flow->action_type);
- ret = -ENOTSUP;
- break;
+ if (action_jump)
+ qos_tc = action_jump->group;
+ if (miss_rxq) {
+ qos_tc = miss_rxq->tc_index;
+ qos_flow = miss_rxq->flow_id;
+ }
+ if ((!discard && qos_tc == 0xff) ||
+ (discard && qos_tc != 0xff)) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "Invalid miss action set for QoS table";
+ goto failure_to_set_miss_actions;
+ }
+ err_code = dpaa2_flow_qos_table_set_default(priv, discard, qos_tc, qos_flow);
+ if (err_code) {
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_str = "Failed to set miss action for QoS table";
+ goto failure_to_set_miss_actions;
+ }
+
+ return 0;
}
- LIST_REMOVE(flow, next);
- rte_free(flow->qos_key_addr);
- rte_free(flow->qos_mask_addr);
- rte_free(flow->fs_key_addr);
- rte_free(flow->fs_mask_addr);
- /* Now free the flow */
- rte_free(flow);
+ if (!priv->fs_entries) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "No FS table available!";
+ goto failure_to_set_miss_actions;
+ }
+ if (!discard && !miss_rxq) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_code = -EINVAL;
+ err_str = "Invalid miss action set for FS table";
+ goto failure_to_set_miss_actions;
+ }
+ if (miss_rxq && miss_rxq->tc_index != group_id) {
+ error_type = RTE_FLOW_ERROR_TYPE_ATTR_GROUP;
+ err_code = -EINVAL;
+ err_str = "Group conflicts with dest queue's TC ID";
+ goto failure_to_set_miss_actions;
+ }
+ err_code = dpaa2_flow_fs_table_set_default(priv, group_id, discard,
+ dest_queue ? dest_queue->index : 0);
+ if (err_code) {
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_str = "Failed to set miss action for FS table";
+ }
-error:
- if (ret)
- rte_flow_error_set(error, EPERM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "unknown");
- return ret;
+failure_to_set_miss_actions:
+ if (err_str)
+ DPAA2_PMD_ERR("%s: %s", __func__, err_str);
+ rte_flow_error_set(err, -err_code, error_type, NULL, err_str);
+
+ return err_code;
+}
+
+static int
+dpaa2_flow_actions_update(struct rte_eth_dev *dev,
+ struct rte_flow *_flow,
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct rte_dpaa2_device *dpaa2_dev;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_dev_flow *flow;
+ struct dpaa2_flow_tbl_profile *tbl_profile = NULL;
+ int ret = 0, is_rss = false, hw_update = false, err_code = 0, qos_direct;
+ struct dpaa2_dev_flow_fs_action *fs_action;
+ uint8_t qos_action_num = 0, fs_action_num = 0, tc_id;
+ struct rte_flow_action qos_actions[DPAA2_MAX_ACTION_PER_FLOW_NUM];
+ struct rte_flow_action fs_actions[DPAA2_MAX_ACTION_PER_FLOW_NUM];
+ const struct rte_flow_action_rss *rss_conf;
+ struct rte_flow_action_jump jump_conf;
+ struct rte_flow_item items[DPKG_MAX_NUM_OF_EXTRACTS + 1];
+ uint8_t spec_buf[1024];
+ struct rte_flow_attr attr;
+ const enum rte_flow_action_type *supported = NULL;
+ uint16_t supported_len = 0;
+ enum rte_flow_error_type error_type = RTE_FLOW_ERROR_TYPE_NONE;
+ const char *err_str = NULL;
+ const struct rte_flow_action_queue *action_q;
+ struct dpaa2_queue *rxq = NULL;
+
+ dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
+
+ /* check for the valid flow */
+ flow = (void *)_flow;
+ if (flow->is_meter_flow) {
+ RTE_ASSERT(flow->fs_flow);
+ tc_id = flow->fs_flow->tc_id;
+ RTE_ASSERT(priv->flow_profile.mtr_flow[tc_id] == flow);
+ supported = dpaa2_supported_meter_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_meter_action_type);
+ if (!dpaa2_flow_check_actions_support(actions, supported, supported_len)) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to verify meter action!";
+ err_code = -ENOTSUP;
+ goto quit;
+ }
+ ret = dpaa2_flow_fs_action_update(priv, flow->fs_flow, actions);
+ if (ret) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to update meter action!";
+ err_code = ret;
+ }
+ goto quit;
+ }
+ LIST_FOREACH(flow, &priv->flows, next) {
+ if ((struct rte_flow *)flow == _flow)
+ goto action_update;
+ }
+ DPAA2_PMD_ERR("%s: Flow(%p) is not in %s's flow list",
+ __func__, flow, dev->data->name);
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_str = "Invalid flow for action update!";
+ err_code = -EINVAL;
+
+ goto quit;
+
+action_update:
+ if (flow->fs_flow) {
+ supported = dpaa2_supported_fs_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_fs_action_type);
+ } else {
+ supported = dpaa2_supported_qos_action_type;
+ supported_len = RTE_DIM(dpaa2_supported_qos_action_type);
+ }
+ if (!dpaa2_flow_check_actions_support(actions, supported, supported_len)) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to verify QoS/FS action!";
+ err_code = -ENOTSUP;
+ goto quit;
+ }
+ ret = dpaa2_flow_qos_fs_action_set(actions, qos_actions, fs_actions,
+ &qos_action_num, &fs_action_num);
+ if (ret) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to set QoS/FS action!";
+ err_code = ret;
+ goto quit;
+ }
+ flow = (struct dpaa2_dev_flow *)_flow;
+ qos_direct = false;
+ if (flow->qos_flow && flow->qos_flow->direct_queue)
+ qos_direct = true;
+ if (flow->fs_flow && flow->qos_flow && qos_action_num > 0) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION_NUM;
+ err_str = "One-level flow can't update QoS action!";
+ err_code = -ENOTSUP;
+ goto quit;
+ }
+ if ((!flow->fs_flow && fs_action_num > 0 && !qos_direct) ||
+ (!flow->qos_flow && qos_action_num > 0)) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Flow type and flow action don't match!";
+ err_code = -EINVAL;
+ goto quit;
+ }
+ if (fs_action_num > 0 &&
+ dpaa2_flow_action_single_type_check(fs_actions,
+ RTE_FLOW_ACTION_TYPE_QUEUE)) {
+ action_q = fs_actions[0].conf;
+ if (action_q->index >= dev->data->nb_rx_queues) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Queue index is too large!";
+ err_code = -EINVAL;
+ goto quit;
+ }
+ rxq = dev->data->rx_queues[action_q->index];
+ if (flow->fs_flow && flow->fs_flow->tc_id != rxq->tc_index) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Update queue is not in this TC!";
+ err_code = -EINVAL;
+ goto quit;
+ }
+ }
+
+ if (!flow->fs_flow)
+ goto qos_action_update;
+ fs_action = &flow->fs_flow->flow_action.fs_action;
+ if (fs_action->action_type == RTE_FLOW_ACTION_TYPE_RSS) {
+ is_rss = true;
+ tc_id = flow->fs_flow->tc_id;
+ tbl_profile = &priv->flow_profile.tc_profile[tc_id];
+ if (tbl_profile->rss_flow != flow) {
+ DPAA2_PMD_ERR("%s: RSS flow(%p) != TC[%d]'s rss flow(%p)",
+ __func__, flow, flow->fs_flow->tc_id, tbl_profile->rss_flow);
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "The flow to update is not specified TC's RSS flow!";
+ err_code = -EINVAL;
+ goto quit;
+ }
+ }
+ if (fs_action_num > 0 &&
+ fs_actions[0].type != RTE_FLOW_ACTION_TYPE_PORT_ID &&
+ fs_actions[0].type != RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT &&
+ dpaa2_dev->bus_info->mc_rev >= DPAA2_FS_FLOW_HW_ACTION_UPDATE_MC_REV) {
+ /** Action HW update doesn't support redirecting frames to other DPNIs.*/
+ hw_update = true;
+ goto skip_remove_fs_entry;
+ }
+
+ if (!is_rss) {
+ ret = dpaa2_flow_remove_generic_entry(dev, flow->fs_flow,
+ DPAA2_FLOW_FS_TYPE);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: remove flow fs entry failed(%d)",
+ __func__, ret);
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_str = "Failed to remove flow in flow action update!";
+ err_code = ret;
+
+ goto quit;
+ }
+ }
+
+skip_remove_fs_entry:
+ if (is_rss) {
+ rss_conf = fs_actions[0].conf;
+ ret = dpaa2_flow_rss_conf_to_item(rss_conf->types, items,
+ spec_buf, 1024);
+ if (ret < 0) {
+ DPAA2_PMD_ERR("TC[%d] converts to RSS items failed(%d)",
+ flow->fs_flow->tc_id, ret);
+ error_type = RTE_FLOW_ERROR_TYPE_UNSPECIFIED;
+ err_str = "Failed to gen rss items in flow action update!";
+ err_code = ret;
+ goto quit;
+ } else if (!ret) {
+ DPAA2_PMD_ERR("No item is generated by rss type(0x%" PRIx64 ")",
+ rss_conf->types);
+ error_type = RTE_FLOW_ERROR_TYPE_ITEM;
+ err_str = "No item is generated by rss type!";
+ err_code = -ENOTSUP;
+ goto quit;
+ }
+ memset(&attr, 0, sizeof(attr));
+ attr.group = flow->fs_flow->tc_id;
+ attr.ingress = 1;
+ tbl_profile->tc_cfg.dist_size = rss_conf->queue_num;
+ memset(&flow->fs_flow->rule_cfg, 0, sizeof(struct dpni_rule_cfg));
+ memset(flow->fs_flow->key_addr, 0, DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
+ memset(flow->fs_flow->mask_addr, 0, DPAA2_EXTRACT_ALLOC_KEY_MAX_SIZE);
+ flow->fs_flow->rule_size = 0;
+ flow->fs_flow->ip_key = NET_PROT_NONE;
+ flow->fs_flow->ip_src = NET_PROT_NONE;
+ flow->fs_flow->ip_dst = NET_PROT_NONE;
+ ret = dpaa2_flow_generic_extract_rule_set(flow->fs_flow, &attr,
+ items, true, DPAA2_FLOW_FS_TYPE, true);
+ if (ret) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "RSS action update failed!";
+ err_code = ret;
+ goto quit;
+ }
+ } else {
+ ret = dpaa2_flow_fs_action_update(priv, flow->fs_flow, fs_actions);
+ if (ret) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "FS action update failed!";
+ err_code = ret;
+ goto quit;
+ }
+ if (hw_update)
+ ret = dpaa2_flow_update_fs_rule_action(priv, flow->fs_flow);
+ else
+ ret = dpaa2_flow_add_fs_rule(priv, flow->fs_flow);
+ if (ret) {
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "FS rule action update failed!";
+ err_code = ret;
+ goto quit;
+ }
+ }
+qos_action_update:
+ if (!flow->qos_flow)
+ goto quit;
+
+ if (rxq) {
+ jump_conf.group = rxq->tc_index;
+ qos_actions[qos_action_num].type = RTE_FLOW_ACTION_TYPE_JUMP;
+ qos_actions[qos_action_num].conf = &jump_conf;
+ qos_actions[qos_action_num + 1].type = RTE_FLOW_ACTION_TYPE_END;
+ }
+
+ hw_update = false;
+ if (dpaa2_dev->bus_info->mc_rev >= DPAA2_QOS_FLOW_HW_ACTION_UPDATE_MC_REV) {
+ /** Action HW update doesn't support redirecting frames to other DPNIs.*/
+ hw_update = true;
+ goto skip_remove_qos_entry;
+ }
+
+ ret = dpaa2_flow_remove_generic_entry(dev, flow->qos_flow,
+ DPAA2_FLOW_QOS_TYPE);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: remove flow qos entry failed(%d)",
+ __func__, ret);
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to remove QoS entry in action update!";
+ err_code = ret;
+
+ goto quit;
+ }
+
+skip_remove_qos_entry:
+ ret = dpaa2_flow_qos_action_update(priv, flow->qos_flow, qos_actions,
+ false);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: QoS action update failed(%d)",
+ __func__, ret);
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to QoS action update!";
+ err_code = ret;
+
+ goto quit;
+ }
+ if (rxq) {
+ flow->qos_flow->direct_queue = true;
+ flow->qos_flow->flow_id = rxq->flow_id;
+ }
+ if (hw_update)
+ ret = dpaa2_flow_update_qos_rule_action(priv, flow->qos_flow);
+ else
+ ret = dpaa2_flow_add_qos_rule(priv, flow->qos_flow);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: %s qos flow entry failed(%d)",
+ __func__, hw_update ? "update" : "add", ret);
+ error_type = RTE_FLOW_ERROR_TYPE_ACTION;
+ err_str = "Failed to update QoS entry action to HW!";
+ err_code = ret;
+ }
+
+quit:
+ if (err_str)
+ DPAA2_PMD_ERR("%s: %s", __func__, err_str);
+ return rte_flow_error_set(error, -err_code, error_type, NULL, err_str);
}
/**
@@ -5390,24 +6431,113 @@ dpaa2_flow_flush(struct rte_eth_dev *dev,
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
struct dpaa2_dev_flow *flow = LIST_FIRST(&priv->flows);
+ int ret = 0;
while (flow) {
struct dpaa2_dev_flow *next = LIST_NEXT(flow, next);
- dpaa2_flow_destroy(dev, (struct rte_flow *)flow, error);
+ ret = dpaa2_flow_destroy(dev, (struct rte_flow *)flow, error);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: Remove flow failed(%d)",
+ __func__, ret);
+ }
+
flow = next;
}
- return 0;
+
+ return ret;
}
static int
-dpaa2_flow_query(struct rte_eth_dev *dev __rte_unused,
- struct rte_flow *_flow __rte_unused,
- const struct rte_flow_action *actions __rte_unused,
- void *data __rte_unused,
- struct rte_flow_error *error __rte_unused)
+dpaa2_flow_query(struct rte_eth_dev *dev,
+ struct rte_flow *_flow, const struct rte_flow_action *actions,
+ void *data, struct rte_flow_error *error)
{
- return 0;
+ struct dpaa2_dev_flow *flow;
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ uint16_t num = 0, num_rsp, i, tc_id = 0;
+ int ret = 0, found = false;
+ size_t size, iova;
+ struct dpni_dump_table_rsp *rsp = NULL;
+
+ RTE_SET_USED(actions);
+ RTE_SET_USED(data);
+ RTE_SET_USED(error);
+
+ flow = LIST_FIRST(&priv->flows);
+ while (flow) {
+ num++;
+ if (flow == (struct dpaa2_dev_flow *)_flow)
+ found = true;
+ flow = LIST_NEXT(flow, next);
+ }
+ if (!num || !found)
+ return -ENXIO;
+ num = num * 2;/** Max: FS entry + QoS entry.*/
+ flow = (void *)_flow;
+ size = sizeof(struct dpni_dump_table_header) +
+ num * sizeof(struct dpni_dump_table_entry);
+ rsp = rte_zmalloc(NULL, size, RTE_CACHE_LINE_SIZE);
+ if (!rsp)
+ return -ENOMEM;
+ iova = DPAA2_VADDR_TO_IOVA_AND_CHECK(rsp, size);
+ if (iova == RTE_BAD_IOVA) {
+ DPAA2_PMD_ERR("%s: No IOMMU map for dump table result!",
+ __func__);
+ ret = -EIO;
+ goto quit;
+ }
+
+ if (priv->num_rx_tc <= 1)
+ goto dump_fs;
+
+ ret = dpni_dump_table(priv->hw, CMD_PRI_LOW,
+ priv->token, DPNI_QOS_TABLE, 0/**Any*/,
+ iova, size, &num_rsp);
+ if (ret) {
+ DPAA2_PMD_ERR("dump %s's QoS table failed(%d)",
+ dev->data->name, ret);
+ goto quit;
+ }
+
+ if (!flow->qos_flow)
+ goto dump_fs;
+
+ for (i = 0; i < num_rsp; i++) {
+ if (!memcmp(rsp->entry[i].key, flow->qos_flow->key_addr,
+ flow->qos_flow->rule_size) &&
+ !memcmp(rsp->entry[i].mask,
+ flow->qos_flow->mask_addr, flow->qos_flow->rule_size))
+ break;
+ }
+
+dump_fs:
+ if (!flow->fs_flow)
+ goto quit;
+
+ tc_id = flow->fs_flow->tc_id;
+ ret = dpni_dump_table(priv->hw, CMD_PRI_LOW,
+ priv->token, DPNI_FS_TABLE, tc_id,
+ iova, size, &num_rsp);
+ if (ret) {
+ DPAA2_PMD_ERR("dump %s's FS%d table failed(%d)",
+ dev->data->name, tc_id, ret);
+ goto quit;
+ }
+ for (i = 0; i < num_rsp; i++) {
+ if (!memcmp(rsp->entry[i].key, flow->fs_flow->key_addr,
+ flow->fs_flow->rule_size) &&
+ !memcmp(rsp->entry[i].mask,
+ flow->fs_flow->mask_addr, flow->fs_flow->rule_size)) {
+ break;
+ }
+ }
+
+quit:
+ if (rsp)
+ rte_free(rsp);
+
+ return ret;
}
/**
@@ -5420,19 +6550,45 @@ dpaa2_flow_query(struct rte_eth_dev *dev __rte_unused,
* Pointer to private structure.
*/
void
-dpaa2_flow_clean(struct rte_eth_dev *dev)
+dpaa2_flow_clean(struct rte_eth_dev *dev, uint8_t tc_id)
{
- struct dpaa2_dev_flow *flow;
+ struct dpaa2_dev_flow *flow, *next_flow;
struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ int ret;
- while ((flow = LIST_FIRST(&priv->flows)))
- dpaa2_flow_destroy(dev, (struct rte_flow *)flow, NULL);
+ flow = LIST_FIRST(&priv->flows);
+ if (tc_id >= MAX_TCS)
+ goto clean_all;
+ while (flow) {
+ next_flow = LIST_NEXT(flow, next);
+ if (flow->fs_flow && flow->fs_flow->tc_id == tc_id) {
+ ret = dpaa2_flow_destroy(dev, (struct rte_flow *)flow, NULL);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: Remove flow failed(%d)",
+ __func__, ret);
+ }
+ }
+ flow = next_flow;
+ }
+ return;
+
+clean_all:
+ while (flow) {
+ ret = dpaa2_flow_destroy(dev, (struct rte_flow *)flow, NULL);
+ if (ret) {
+ DPAA2_PMD_ERR("%s: Remove flow failed(%d)",
+ __func__, ret);
+ }
+ flow = LIST_FIRST(&priv->flows);
+ }
}
const struct rte_flow_ops dpaa2_flow_ops = {
.create = dpaa2_flow_create,
.validate = dpaa2_flow_validate,
.destroy = dpaa2_flow_destroy,
+ .group_set_miss_actions = dpaa2_flow_set_miss_actions,
+ .actions_update = dpaa2_flow_actions_update,
.flush = dpaa2_flow_flush,
.query = dpaa2_flow_query,
};
diff --git a/drivers/net/dpaa2/meson.build b/drivers/net/dpaa2/meson.build
index 4f774f7c82..3f8bee9146 100644
--- a/drivers/net/dpaa2/meson.build
+++ b/drivers/net/dpaa2/meson.build
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: BSD-3-Clause
-# Copyright 2018-2021 NXP
+# Copyright 2018-2026 NXP
if not is_linux
build = false
@@ -8,7 +8,6 @@ endif
deps += ['mempool_dpaa2']
sources = files(
- 'base/dpaa2_hw_dpni.c',
'base/dpaa2_tlu_hash.c',
'dpaa2_tm.c',
'dpaa2_meter.c',
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 38/45] net/dpaa2: support Rx mempool per traffic class
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (36 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 37/45] net/dpaa2: rework flow engine Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 39/45] drivers: consume dpaa2 DQRR entries in batches Prashant Gupta
` (6 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Support configuring RX mempool per TC.
This patch isolates mempools by traffic class to avoid
individual TC's performance being impacted by lack of memory.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 115 ++++++++++++++++++++++++++++++-
drivers/net/dpaa2/dpaa2_ethdev.h | 2 +-
2 files changed, 115 insertions(+), 2 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index a7c9bb678e..a33d6c5db6 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -565,6 +565,97 @@ dpaa2_update_flow_rss_dist(struct rte_eth_dev *eth_dev,
return ret;
}
+static int
+dpaa2_attach_bp_list(struct dpaa2_dev_priv *priv,
+ struct fsl_mc_io *dpni, void *blist, uint8_t tc_id)
+{
+ /* Function to attach a DPNI with a buffer pool list. Buffer pool list
+ * handle is passed in blist.
+ */
+ int32_t retcode;
+ struct dpni_pools_cfg *bpool_cfg = &priv->pools_cfg;
+ struct dpaa2_bp_list *bp_list = blist;
+ struct dpni_buffer_layout layout;
+ int tot_size, out_min_hdr_room, in_min_hdr_room;
+ uint8_t bp_idx;
+ struct rte_mempool *mp = bp_list->mp;
+
+ if (priv->flow_profile.mempool[tc_id] != mp) {
+ if (!priv->flow_profile.mempool[tc_id]) {
+ bp_idx = bpool_cfg->num_dpbp;
+ bpool_cfg->num_dpbp++;
+ priv->flow_profile.bp_idx[tc_id] = bp_idx;
+ } else {
+ bp_idx = priv->flow_profile.bp_idx[tc_id];
+ }
+ priv->flow_profile.mempool[tc_id] = mp;
+ } else {
+ bp_idx = priv->flow_profile.bp_idx[tc_id];
+ }
+
+ /* ... rx buffer layout .
+ * Check alignment for buffer layouts first
+ */
+
+ /* ... rx buffer layout ... */
+ if (priv->tx_conf_type == DPAA2_TX_DYNAMIC_CONF) {
+ /** Additional headroom layout for IPSec with TX configure
+ * dynamic enabled.
+ */
+ in_min_hdr_room = DPAA2_RX_MIN_FD_OFFSET +
+ DPAA2_SEC_SIMPLE_FD_IB_MIN;
+ out_min_hdr_room = DPAA2_DYN_TX_MIN_FD_OFFSET +
+ DPAA2_SEC_SIMPLE_FD_OB_MIN;
+ tot_size = RTE_MAX(in_min_hdr_room, out_min_hdr_room);
+ if (tot_size < RTE_PKTMBUF_HEADROOM)
+ tot_size = RTE_PKTMBUF_HEADROOM;
+ } else {
+ tot_size = RTE_PKTMBUF_HEADROOM;
+ }
+ tot_size = RTE_ALIGN_CEIL(tot_size, DPAA2_PACKET_LAYOUT_ALIGN);
+
+ memset(&layout, 0, sizeof(struct dpni_buffer_layout));
+ layout.options = DPNI_BUF_LAYOUT_OPT_DATA_HEAD_ROOM |
+ DPNI_BUF_LAYOUT_OPT_FRAME_STATUS |
+ DPNI_BUF_LAYOUT_OPT_PARSER_RESULT |
+ DPNI_BUF_LAYOUT_OPT_DATA_ALIGN |
+ DPNI_BUF_LAYOUT_OPT_TIMESTAMP |
+ DPNI_BUF_LAYOUT_OPT_PRIVATE_DATA_SIZE;
+
+ layout.pass_timestamp = true;
+ layout.pass_frame_status = 1;
+ layout.private_data_size = DPAA2_FD_PTA_SIZE;
+ layout.pass_parser_result = 1;
+ layout.data_align = DPAA2_PACKET_LAYOUT_ALIGN;
+ layout.data_head_room = tot_size - DPAA2_FD_PTA_SIZE -
+ DPAA2_MBUF_HW_ANNOTATION;
+ retcode = dpni_set_buffer_layout(dpni, CMD_PRI_LOW, priv->token,
+ DPNI_QUEUE_RX, &layout);
+ if (retcode) {
+ DPAA2_PMD_ERR("Error configuring buffer pool Rx layout (%d)",
+ retcode);
+ return retcode;
+ }
+
+ /*Attach buffer pool to the network interface as described by the user*/
+ bpool_cfg->pools[bp_idx].dpbp_id = bp_list->buf_pool.dpbp_node->dpbp_id;
+ bpool_cfg->pools[bp_idx].backup_pool = 0;
+ bpool_cfg->pools[bp_idx].buffer_size = RTE_ALIGN_CEIL(bp_list->buf_pool.size,
+ DPAA2_PACKET_LAYOUT_ALIGN);
+ bpool_cfg->pools[bp_idx].priority_mask = tc_id;
+
+ retcode = dpni_set_pools(dpni, CMD_PRI_LOW, priv->token, bpool_cfg);
+ if (retcode) {
+ DPAA2_PMD_ERR("Error(%d) configuring pools[%d](id=%d) on %s.",
+ retcode, bp_idx, bpool_cfg->pools[bp_idx].dpbp_id,
+ priv->eth_dev->data->name);
+ return retcode;
+ }
+
+ priv->bp_list = bp_list;
+ return 0;
+}
+
static int
dpaa2_vlan_filter_set(struct rte_eth_dev *dev, uint16_t vlan_id, int on)
{
@@ -1340,7 +1431,8 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
struct dpni_taildrop taildrop;
uint8_t qopt = 0;
uint16_t flow_id;
- int ret;
+ uint32_t bpid;
+ int ret, ops_idx;
PMD_INIT_FUNC_TRACE();
@@ -1365,6 +1457,27 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
dev->data->rx_queues[rx_queue_id] = dpaa2_q;
return 0;
}
+
+ ops_idx = rte_dpaa2_mpool_get_ops_idx();
+ if (ops_idx != mb_pool->ops_index) {
+ DPAA2_PMD_ERR("MP(%s)'s ops index(%d) != %d",
+ mb_pool->name, mb_pool->ops_index, ops_idx);
+ return -EINVAL;
+ }
+
+ if (!priv->bp_list || priv->bp_list->mp != mb_pool) {
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ ret = rte_dpaa2_bpid_info_init(mb_pool);
+ if (ret)
+ return ret;
+ }
+ bpid = mempool_to_bpid(mb_pool);
+ ret = dpaa2_attach_bp_list(priv, dpni,
+ rte_dpaa2_bpid_info[bpid].bp_list,
+ dpaa2_q->tc_index);
+ if (ret)
+ return ret;
+ }
cfg = rte_zmalloc(NULL, sizeof(struct dpni_queue), 0);
if (!cfg)
return -ENOMEM;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 5659abcd51..7d93b75be3 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -404,7 +404,7 @@ struct dpaa2_dev_priv {
/** RXQs in same TC share same cgid.*/
uint8_t cgid_in_use[MAX_TCS];
rte_spinlock_t meter_lock;
-
+ struct dpni_pools_cfg pools_cfg;
/* Current hash distribution size per RX TC, written by
* dpaa2_setup_flow_dist_size() and read by reta_query / reta_update.
* Zero means "use default" (= nb_rx_queues clamped to dist_queues).
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 39/45] drivers: consume dpaa2 DQRR entries in batches
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (37 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 38/45] net/dpaa2: support Rx mempool per traffic class Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 40/45] drivers: resolve dpaa2 endpoint in the net driver Prashant Gupta
` (5 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
Each DQRR entry consumed by the software portal costs a write to the
cache-invalidate DCAP register. The DCAP register also supports a vector
mode where a bitmap of DQRR indices is consumed with a single write, so
accumulate the indices and flush the vector once half the ring is
pending. A smaller threshold gives little benefit, and a threshold close
to the ring size starves WRIOP of DQRR space.
Because entries are now released in batches, a caller that stops polling
a portal must flush the pending vector explicitly. Pass a NULL dequeue
result to qbman_swp_dqrr_consume() to do that when the eventdev dequeue
finds the portal dry, when the net PMD finishes draining a portal, and
when a portal is released back to the bus.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 4 ++
.../fslmc/qbman/include/fsl_qbman_portal.h | 5 +-
drivers/bus/fslmc/qbman/qbman_portal.c | 63 +++++++++++++++++--
drivers/bus/fslmc/qbman/qbman_portal.h | 11 +++-
drivers/event/dpaa2/dpaa2_eventdev.c | 4 ++
drivers/net/dpaa2/dpaa2_ethdev.c | 2 +
6 files changed, 81 insertions(+), 8 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
index 20d1bb2a66..cfd9548fb0 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
@@ -305,6 +305,10 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id, bool ethrx
static void dpaa2_put_qbman_swp(struct dpaa2_dpio_dev *dpio_dev)
{
if (dpio_dev) {
+ /* consume indices may still be pending in the portal's DQRR
+ * consume vector; flush them before the portal is released.
+ */
+ qbman_swp_dqrr_consume(dpio_dev->sw_portal, NULL);
/* rx-queue interrupts (net PMD) can arm a portal without the
* event driver; tear it down unconditionally. Safe when never
* armed: intr_deinit returns early if intr is not enabled.
diff --git a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
index b9e7789766..73e57bcdc1 100644
--- a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
+++ b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
@@ -451,7 +451,10 @@ void qbman_swp_prefetch_dqrr_next(struct qbman_swp *s);
* qbman_swp_dqrr_consume() - Consume DQRR entries previously returned from
* qbman_swp_dqrr_next().
* @s: the software portal object.
- * @dq: the DQRR entry to be consumed.
+ * @dq: the DQRR entry to be consumed, NULL to consume the entries accumulated
+ * so far. Entries are accumulated and consumed in batches, so a caller which
+ * stops polling the portal must call this with a NULL dq to release the
+ * entries it has already processed.
*/
__rte_internal
void qbman_swp_dqrr_consume(struct qbman_swp *s, const struct qbman_result *dq);
diff --git a/drivers/bus/fslmc/qbman/qbman_portal.c b/drivers/bus/fslmc/qbman/qbman_portal.c
index c93bec5dd3..c95c62baa7 100644
--- a/drivers/bus/fslmc/qbman/qbman_portal.c
+++ b/drivers/bus/fslmc/qbman/qbman_portal.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
- * Copyright 2018-2020,2023-2024 NXP
+ * Copyright 2018-2020,2023-2024,2026 NXP
*
*/
@@ -9,13 +9,20 @@
#include "qbman_portal.h"
#include <eal_export.h>
+#include <rte_bitops.h>
/* QBMan portal management command codes */
#define QBMAN_MC_ACQUIRE 0x30
#define QBMAN_WQCHAN_CONFIGURE 0x46
/* Reverse mapping of QBMAN_CENA_SWP_DQRR() */
-#define QBMAN_IDX_FROM_DQRR(p) (((unsigned long)p & 0x1ff) >> 6)
+#define QBMAN_IDX_FROM_DQRR(p) (((unsigned long)(p) & 0x1ff) >> 6)
+
+/* DCAP consume-index-vector mode: the vector of DQRR indices to consume starts
+ * at bit 16 of DCAP and bit 8 selects vector mode over single-index mode.
+ */
+#define DQRR_DCAP_CI_VEC_OFFSET 16
+#define DQRR_DCAP_CI_VEC_SELECT 0x100
/* QBMan FQ management command codes */
#define QBMAN_FQ_SCHEDULE 0x48
@@ -284,6 +291,13 @@ struct qbman_swp *qbman_swp_init(const struct qbman_swp_desc *d)
p->dqrr.dqrr_size = 8;
p->dqrr.reset_bug = 0;
}
+ /* Consume DQRR entries in vector mode by default and flush the vector
+ * once half the ring is pending. A smaller threshold gives little
+ * benefit; a threshold close to the ring size starves WRIOP of DQRR
+ * space and slows its enqueues down.
+ */
+ p->dqrr.ci_vec_en = true;
+ p->dqrr.ci_flush_th = p->dqrr.dqrr_size / 2;
ret = qbman_swp_sys_init(&p->sys, d, p->dqrr.dqrr_size);
if (ret) {
@@ -2225,13 +2239,47 @@ const struct qbman_result *qbman_swp_dqrr_next_mem_back(struct qbman_swp *s)
return p;
}
-/* Consume DQRR entries previously returned from qbman_swp_dqrr_next(). */
+/* Write the pending consume vector to DCAP if it holds at least "threshold"
+ * indices, consuming all of them with a single register access.
+ */
+static inline void
+qbman_swp_dqrr_vec_flush(struct qbman_swp *s, uint8_t threshold)
+{
+ if (s->dqrr.ci_count < threshold)
+ return;
+
+ qbman_cinh_write(&s->sys, QBMAN_CINH_SWP_DCAP,
+ s->dqrr.ci_vector | DQRR_DCAP_CI_VEC_SELECT);
+ s->dqrr.ci_vector = 0;
+ s->dqrr.ci_count = 0;
+}
+
+/* Add a DQRR index to the pending consume vector, flushing it once the
+ * configured threshold is reached.
+ */
+static inline void
+qbman_swp_dqrr_vec_consume(struct qbman_swp *s, uint8_t idx)
+{
+ s->dqrr.ci_vector |= RTE_BIT32(idx + DQRR_DCAP_CI_VEC_OFFSET);
+ s->dqrr.ci_count++;
+ qbman_swp_dqrr_vec_flush(s, s->dqrr.ci_flush_th);
+}
+
+/* Consume DQRR entries previously returned from qbman_swp_dqrr_next().
+ * A NULL dq flushes any indices still pending in the consume vector, which
+ * the caller must do before it stops polling the portal.
+ */
RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_dqrr_consume)
void qbman_swp_dqrr_consume(struct qbman_swp *s,
const struct qbman_result *dq)
{
- qbman_cinh_write(&s->sys,
- QBMAN_CINH_SWP_DCAP, QBMAN_IDX_FROM_DQRR(dq));
+ if (unlikely(dq == NULL))
+ qbman_swp_dqrr_vec_flush(s, 1);
+ else if (s->dqrr.ci_vec_en)
+ qbman_swp_dqrr_vec_consume(s, QBMAN_IDX_FROM_DQRR(dq));
+ else
+ qbman_cinh_write(&s->sys, QBMAN_CINH_SWP_DCAP,
+ QBMAN_IDX_FROM_DQRR(dq));
}
/* Consume DQRR entries previously returned from qbman_swp_dqrr_next(). */
@@ -2239,7 +2287,10 @@ RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_dqrr_idx_consume)
void qbman_swp_dqrr_idx_consume(struct qbman_swp *s,
uint8_t dqrr_index)
{
- qbman_cinh_write(&s->sys, QBMAN_CINH_SWP_DCAP, dqrr_index);
+ if (s->dqrr.ci_vec_en)
+ qbman_swp_dqrr_vec_consume(s, dqrr_index);
+ else
+ qbman_cinh_write(&s->sys, QBMAN_CINH_SWP_DCAP, dqrr_index);
}
/*********************************/
diff --git a/drivers/bus/fslmc/qbman/qbman_portal.h b/drivers/bus/fslmc/qbman/qbman_portal.h
index 1cf7918309..db74db5fe1 100644
--- a/drivers/bus/fslmc/qbman/qbman_portal.h
+++ b/drivers/bus/fslmc/qbman/qbman_portal.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
*
* Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
- * Copyright 2018-2020 NXP
+ * Copyright 2018-2020, 2026 NXP
*
*/
@@ -93,6 +93,15 @@ struct qbman_swp {
uint32_t valid_bit;
uint8_t dqrr_size;
int reset_bug;
+ /* Consume index vector: instead of writing DCAP once per
+ * consumed entry, the indices are accumulated in ci_vector
+ * and written with a single DCAP access once ci_count
+ * reaches ci_flush_th.
+ */
+ bool ci_vec_en;
+ uint8_t ci_count;
+ uint8_t ci_flush_th;
+ uint32_t ci_vector;
} dqrr;
struct {
uint32_t pi;
diff --git a/drivers/event/dpaa2/dpaa2_eventdev.c b/drivers/event/dpaa2/dpaa2_eventdev.c
index 95d9154b11..5d32bd9fd6 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.c
+++ b/drivers/event/dpaa2/dpaa2_eventdev.c
@@ -320,6 +320,10 @@ dpaa2_eventdev_dequeue_burst(void *port, struct rte_event ev[],
do {
dq = qbman_swp_dqrr_next(swp);
if (!dq) {
+ /* portal is dry: consume whatever is still pending in
+ * the DQRR consume vector before leaving it idle.
+ */
+ qbman_swp_dqrr_consume(swp, NULL);
if (!num_pkts && timeout_ticks) {
dpaa2_eventdev_dequeue_wait(timeout_ticks);
timeout_ticks = 0;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index a33d6c5db6..34f1e00b12 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3533,6 +3533,8 @@ dpaa2_napi_drain_portal(struct dpaa2_dpio_dev *dpio)
while ((dq = qbman_swp_dqrr_next(dpio->sw_portal)))
qbman_swp_dqrr_consume(dpio->sw_portal, dq);
+ /* flush the indices still pending in the DQRR consume vector */
+ qbman_swp_dqrr_consume(dpio->sw_portal, NULL);
qbman_swp_interrupt_clear_status(dpio->sw_portal, 0xffffffff);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 40/45] drivers: resolve dpaa2 endpoint in the net driver
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (38 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 39/45] drivers: consume dpaa2 DQRR entries in batches Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 41/45] drivers: align dpaa2 event port depths with hardware rings Prashant Gupta
` (4 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
The DPRC object driver walked every device on the bus and, for each DPNI,
issued dprc_get_connection() to find the object it is wired to. Two
things are wrong with that. The bus keeps the result in
struct rte_dpaa2_device even though only the net driver consumes it, and
a failure to resolve one DPNI's endpoint aborted the creation of the
whole container. The endpoint name was also formatted from endpoint2
outside the DPNI branch, so non-DPNI devices got a name built from
uninitialised or stale data.
Move the lookup to dpaa2_dev_init(), which already knows it is dealing
with a DPNI and can fail the probe of that one port. This also fills in
priv->ep_dev_type, priv->ep_object_id and priv->ep_name, which the net
driver reads for MAC-level operations, recycle/loopback configuration and
rte_pmd_dpaa2_ep_name(), but which nothing assigned. dpsw endpoints are
now recognised, and dpdmux/dpsw names carry the interface index because
those endpoints are per-interface.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
drivers/bus/fslmc/bus_fslmc_driver.h | 3 --
drivers/bus/fslmc/portal/dpaa2_hw_dprc.c | 45 +++---------------
drivers/net/dpaa2/dpaa2_ethdev.c | 58 +++++++++++++++++++++++-
drivers/net/dpaa2/dpaa2_recycle.c | 15 +++---
4 files changed, 70 insertions(+), 51 deletions(-)
diff --git a/drivers/bus/fslmc/bus_fslmc_driver.h b/drivers/bus/fslmc/bus_fslmc_driver.h
index b4a5a86318..0d431fdd19 100644
--- a/drivers/bus/fslmc/bus_fslmc_driver.h
+++ b/drivers/bus/fslmc/bus_fslmc_driver.h
@@ -104,10 +104,7 @@ struct rte_dpaa2_device {
struct rte_device device; /**< Inherit core device */
enum rte_dpaa2_dev_type dev_type; /**< Device Type */
uint16_t object_id; /**< DPAA2 Object ID */
- enum rte_dpaa2_dev_type ep_dev_type; /**< Endpoint Device Type */
struct dpaa2_dprc_dev *container;
- uint16_t ep_object_id; /**< Endpoint DPAA2 Object ID */
- char ep_name[RTE_DEV_NAME_MAX_LEN];
struct rte_intr_handle *intr_handle; /**< Interrupt handle */
char name[FSLMC_OBJECT_MAX_LEN]; /**< DPAA2 Object name*/
struct rte_dma_dev *dmadev; /**< DMA device */
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c b/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c
index 868ed646af..6441297ecf 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c
@@ -27,7 +27,6 @@ rte_dpaa2_create_dprc_device(int vdev_fd __rte_unused,
struct rte_dpaa2_device *obj)
{
struct dpaa2_dprc_dev *dprc_node;
- struct dprc_endpoint endpoint1, endpoint2;
struct rte_dpaa2_device *dev;
int ret, dprc_id = obj->object_id;
@@ -49,45 +48,13 @@ rte_dpaa2_create_dprc_device(int vdev_fd __rte_unused,
return ret;
}
- RTE_BUS_FOREACH_DEV(dev, &rte_fslmc_bus) {
- /** DPRC is always created before it's children are created.*/
+ /* DPRC is always created before its children are created, so every
+ * device scanned so far belongs to this container. Endpoint lookup is
+ * left to the object drivers, which know how many interfaces their
+ * object has and can report a failure to their own caller.
+ */
+ RTE_BUS_FOREACH_DEV(dev, &rte_fslmc_bus)
dev->container = dprc_node;
- if (dev->dev_type == DPAA2_ETH) {
- int link_state;
-
- memset(&endpoint1, 0, sizeof(struct dprc_endpoint));
- memset(&endpoint2, 0, sizeof(struct dprc_endpoint));
- strcpy(endpoint1.type, "dpni");
- endpoint1.id = dev->object_id;
- ret = dprc_get_connection(&dprc_node->dprc,
- CMD_PRI_LOW,
- dprc_node->token,
- &endpoint1, &endpoint2,
- &link_state);
- if (ret) {
- DPAA2_BUS_ERR("dpni.%d connection failed!",
- dev->object_id);
- dprc_close(&dprc_node->dprc, CMD_PRI_LOW,
- dprc_node->token);
- rte_free(dprc_node);
- return ret;
- }
-
- if (!strcmp(endpoint2.type, "dpmac"))
- dev->ep_dev_type = DPAA2_MAC;
- else if (!strcmp(endpoint2.type, "dpni"))
- dev->ep_dev_type = DPAA2_ETH;
- else if (!strcmp(endpoint2.type, "dpdmux"))
- dev->ep_dev_type = DPAA2_MUX;
- else
- dev->ep_dev_type = DPAA2_UNKNOWN;
-
- dev->ep_object_id = endpoint2.id;
- } else {
- dev->ep_dev_type = DPAA2_UNKNOWN;
- }
- sprintf(dev->ep_name, "%s.%d", endpoint2.type, endpoint2.id);
- }
TAILQ_INSERT_TAIL(&dprc_dev_list, dprc_node, next);
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 34f1e00b12..7cd38ddce6 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3927,6 +3927,55 @@ dpaa2_get_devargs(struct rte_devargs *devargs, const char *key)
return 1;
}
+/* Resolve the object this DPNI is wired to inside its DPRC container. The
+ * endpoint decides whether MAC-level operations (link, promisc, flow control)
+ * are available, and it is reported to the application by
+ * rte_pmd_dpaa2_ep_name().
+ */
+static int
+dpaa2_dev_ep_init(struct rte_dpaa2_device *dpaa2_dev,
+ struct dpaa2_dev_priv *priv)
+{
+ struct dpaa2_dprc_dev *dprc_node = dpaa2_dev->container;
+ struct dprc_endpoint endpoint1, endpoint2;
+ int link_state, ret;
+
+ memset(&endpoint1, 0, sizeof(endpoint1));
+ memset(&endpoint2, 0, sizeof(endpoint2));
+ strlcpy(endpoint1.type, "dpni", sizeof(endpoint1.type));
+ endpoint1.id = dpaa2_dev->object_id;
+ ret = dprc_get_connection(&dprc_node->dprc, CMD_PRI_LOW,
+ dprc_node->token, &endpoint1, &endpoint2,
+ &link_state);
+ if (ret != 0)
+ return ret;
+
+ if (strcmp(endpoint2.type, "dpmac") == 0)
+ priv->ep_dev_type = DPAA2_MAC;
+ else if (strcmp(endpoint2.type, "dpni") == 0)
+ priv->ep_dev_type = DPAA2_ETH;
+ else if (strcmp(endpoint2.type, "dpdmux") == 0)
+ priv->ep_dev_type = DPAA2_MUX;
+ else if (strcmp(endpoint2.type, "dpsw") == 0)
+ priv->ep_dev_type = DPAA2_SW;
+ else
+ priv->ep_dev_type = DPAA2_UNKNOWN;
+
+ priv->ep_object_id = endpoint2.id;
+
+ /* dpdmux and dpsw endpoints are per-interface, so the interface index
+ * is part of the name.
+ */
+ if (priv->ep_dev_type == DPAA2_MUX || priv->ep_dev_type == DPAA2_SW)
+ snprintf(priv->ep_name, sizeof(priv->ep_name), "%s.%d.%d",
+ endpoint2.type, endpoint2.id, endpoint2.if_id);
+ else
+ snprintf(priv->ep_name, sizeof(priv->ep_name), "%s.%d",
+ endpoint2.type, endpoint2.id);
+
+ return 0;
+}
+
static int
dpaa2_dev_init(struct rte_eth_dev *eth_dev)
{
@@ -4020,6 +4069,13 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
goto init_err;
}
+ ret = dpaa2_dev_ep_init(dpaa2_dev, priv);
+ if (ret) {
+ DPAA2_PMD_ERR("Failure in get dpni@%d endpoint, err code %d",
+ hw_id, ret);
+ goto init_err;
+ }
+
ret = dpni_get_api_version(dpni_dev, CMD_PRI_LOW, &priv->dpni_ver_major,
&priv->dpni_ver_minor);
if (ret) {
@@ -4254,7 +4310,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
priv->sp_protocol = dpaa2_dev->bus_info->sp_protocol;
DPAA2_PMD_INFO("%s: netdev created, connected to %s",
- eth_dev->data->name, dpaa2_dev->ep_name);
+ eth_dev->data->name, priv->ep_name);
priv->speed_capa = dpaa2_dev_get_speed_capability(eth_dev);
diff --git a/drivers/net/dpaa2/dpaa2_recycle.c b/drivers/net/dpaa2/dpaa2_recycle.c
index ed038c204b..acf4a8b320 100644
--- a/drivers/net/dpaa2/dpaa2_recycle.c
+++ b/drivers/net/dpaa2/dpaa2_recycle.c
@@ -628,18 +628,18 @@ dpaa2_dev_recycle_config(struct rte_eth_dev *eth_dev)
return 0;
}
- if (dpaa2_dev->ep_dev_type == DPAA2_MAC) {
+ if (priv->ep_dev_type == DPAA2_MAC) {
/** For dpmac-dpni connection,
* try setting serdes loopback as recycle device at first.
*/
if (dpaa2_svr_family == SVR_LS2088A) {
- ret = ls_serdes_eth_lpbk(dpaa2_dev->ep_object_id, 1);
+ ret = ls_serdes_eth_lpbk(priv->ep_object_id, 1);
if (!ret) {
priv->flags |= DPAA2_TX_SERDES_LOOPBACK_MODE;
return 0;
}
} else if (dpaa2_svr_family == SVR_LX2160A) {
- ret = lx_serdes_eth_lpbk(dpaa2_dev->ep_object_id, 1);
+ ret = lx_serdes_eth_lpbk(priv->ep_object_id, 1);
if (!ret) {
priv->flags |= DPAA2_TX_SERDES_LOOPBACK_MODE;
return 0;
@@ -668,8 +668,8 @@ dpaa2_dev_recycle_config(struct rte_eth_dev *eth_dev)
return 0;
}
- if (dpaa2_dev->ep_dev_type == DPAA2_ETH &&
- dpaa2_dev->object_id == dpaa2_dev->ep_object_id) {
+ if (priv->ep_dev_type == DPAA2_ETH &&
+ dpaa2_dev->object_id == priv->ep_object_id) {
priv->flags |= DPAA2_TX_DPNI_LOOPBACK_MODE;
return 0;
@@ -682,7 +682,6 @@ int
dpaa2_dev_recycle_deconfig(struct rte_eth_dev *eth_dev)
{
struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
- struct rte_dpaa2_device *dpaa2_dev = RTE_CLASS_TO_BUS_DEVICE(eth_dev, *dpaa2_dev);
struct fsl_mc_io *dpni_dev = eth_dev->process_private;
struct dpni_port_cfg port_cfg;
int ret = 0;
@@ -692,7 +691,7 @@ dpaa2_dev_recycle_deconfig(struct rte_eth_dev *eth_dev)
if (priv->flags & DPAA2_TX_SERDES_LOOPBACK_MODE) {
if (dpaa2_svr_family == SVR_LS2088A) {
- ret = ls_serdes_eth_lpbk(dpaa2_dev->ep_object_id, 0);
+ ret = ls_serdes_eth_lpbk(priv->ep_object_id, 0);
if (ret) {
DPAA2_PMD_WARN("Error(%d) to disable Serdes loopback",
ret);
@@ -700,7 +699,7 @@ dpaa2_dev_recycle_deconfig(struct rte_eth_dev *eth_dev)
priv->flags &= ~DPAA2_TX_SERDES_LOOPBACK_MODE;
}
} else if (dpaa2_svr_family == SVR_LX2160A) {
- ret = lx_serdes_eth_lpbk(dpaa2_dev->ep_object_id, 0);
+ ret = lx_serdes_eth_lpbk(priv->ep_object_id, 0);
if (ret) {
DPAA2_PMD_WARN("Error(%d) to disable Serdes loopback",
ret);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 41/45] drivers: align dpaa2 event port depths with hardware rings
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (39 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 40/45] drivers: resolve dpaa2 endpoint in the net driver Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 42/45] net/dpaa2: read MC version from device private data Prashant Gupta
` (3 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Jun Yang
From: Jun Yang <jun.yang@nxp.com>
The event port dequeue and enqueue depths were fixed at 8. A dequeue
drains the portal DQRR and an enqueue fills the EQCR, so the real limits
are the sizes of those two rings, which differ per SoC and are known at
run time. Report them instead, adding qbman_swp_portal_dqrr_size() to
query the DQRR size (the EQCR size is already available as
dpaa2_eqcr_size).
The Tx adapter enqueue helpers sized their on-stack mbuf arrays from the
advertised enqueue depth, which no longer bounds the burst the adapter
may hand them, so bound the copy by the array itself and transmit in
chunks.
Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
.../fslmc/qbman/include/fsl_qbman_portal.h | 7 ++
drivers/bus/fslmc/qbman/qbman_portal.c | 15 ++++
drivers/event/dpaa2/dpaa2_eventdev.c | 72 ++++++++++++-------
drivers/event/dpaa2/dpaa2_eventdev.h | 4 +-
4 files changed, 69 insertions(+), 29 deletions(-)
diff --git a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
index 73e57bcdc1..fb1c6046ae 100644
--- a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
+++ b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
@@ -1222,6 +1222,13 @@ __rte_internal
int qbman_swp_acquire(struct qbman_swp *s, uint16_t bpid, uint64_t *buffers,
unsigned int num_buffers);
+/**
+ * qbman_swp_portal_dqrr_size() - Number of entries in a portal's DQRR.
+ * @p: the software portal object, NULL to query the SoC's DQRR size.
+ */
+__rte_internal
+uint32_t qbman_swp_portal_dqrr_size(struct qbman_swp *p);
+
/*****************/
/* FQ management */
/*****************/
diff --git a/drivers/bus/fslmc/qbman/qbman_portal.c b/drivers/bus/fslmc/qbman/qbman_portal.c
index c95c62baa7..a5f92e27e7 100644
--- a/drivers/bus/fslmc/qbman/qbman_portal.c
+++ b/drivers/bus/fslmc/qbman/qbman_portal.c
@@ -73,6 +73,11 @@ enum qbman_sdqcr_fc {
static struct qbman_swp *portal_idx_map[MAX_QBMAN_PORTALS];
uint32_t qman_version;
+/* DQRR size of the portals initialised so far. All portals of a SoC have the
+ * same size, so this doubles as the SoC's DQRR size for callers that have no
+ * portal at hand.
+ */
+static uint32_t dpaa2_portal_dqrr_size = 8;
/* Internal Function declaration */
static int
@@ -239,6 +244,15 @@ static int (*qbman_swp_release_ptr)(struct qbman_swp *s,
const uint64_t *buffers, unsigned int num_buffers)
= qbman_swp_release_direct;
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_portal_dqrr_size)
+uint32_t qbman_swp_portal_dqrr_size(struct qbman_swp *p)
+{
+ if (p != NULL)
+ return p->dqrr.dqrr_size;
+
+ return dpaa2_portal_dqrr_size;
+}
+
/*********************************/
/* Portal constructor/destructor */
/*********************************/
@@ -298,6 +312,7 @@ struct qbman_swp *qbman_swp_init(const struct qbman_swp_desc *d)
*/
p->dqrr.ci_vec_en = true;
p->dqrr.ci_flush_th = p->dqrr.dqrr_size / 2;
+ dpaa2_portal_dqrr_size = p->dqrr.dqrr_size;
ret = qbman_swp_sys_init(&p->sys, d, p->dqrr.dqrr_size);
if (ret) {
diff --git a/drivers/event/dpaa2/dpaa2_eventdev.c b/drivers/event/dpaa2/dpaa2_eventdev.c
index 5d32bd9fd6..ddcf7b3678 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.c
+++ b/drivers/event/dpaa2/dpaa2_eventdev.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017,2019-2022 NXP
+ * Copyright 2017,2019-2022,2026 NXP
*/
#include <assert.h>
@@ -389,10 +389,11 @@ dpaa2_eventdev_info_get(struct rte_eventdev *dev,
/* we only support dpio up to number of cores */
if (dev_info->max_event_ports > rte_lcore_count())
dev_info->max_event_ports = rte_lcore_count();
- dev_info->max_event_port_dequeue_depth =
- DPAA2_EVENT_MAX_PORT_DEQUEUE_DEPTH;
- dev_info->max_event_port_enqueue_depth =
- DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH;
+ /* A dequeue drains the portal DQRR and an enqueue fills the EQCR, so
+ * the hardware ring sizes are the real limits.
+ */
+ dev_info->max_event_port_dequeue_depth = qbman_swp_portal_dqrr_size(NULL);
+ dev_info->max_event_port_enqueue_depth = dpaa2_eqcr_size;
dev_info->max_num_events = DPAA2_EVENT_MAX_NUM_EVENTS;
dev_info->event_dev_cap = RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED |
RTE_EVENT_DEV_CAP_ATOMIC |
@@ -527,12 +528,9 @@ dpaa2_eventdev_port_def_conf(struct rte_eventdev *dev, uint8_t port_id,
RTE_SET_USED(dev);
RTE_SET_USED(port_id);
- port_conf->new_event_threshold =
- DPAA2_EVENT_MAX_NUM_EVENTS;
- port_conf->dequeue_depth =
- DPAA2_EVENT_MAX_PORT_DEQUEUE_DEPTH;
- port_conf->enqueue_depth =
- DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH;
+ port_conf->new_event_threshold = DPAA2_EVENT_MAX_NUM_EVENTS;
+ port_conf->dequeue_depth = qbman_swp_portal_dqrr_size(NULL);
+ port_conf->enqueue_depth = dpaa2_eqcr_size;
port_conf->event_port_cfg = 0;
}
@@ -977,18 +975,31 @@ dpaa2_eventdev_txa_enqueue_same_dest(void *port,
struct rte_event ev[],
uint16_t nb_events)
{
- struct rte_mbuf *m[DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH], *m0;
- uint8_t qid, i;
+ struct rte_mbuf *m[MAX_TX_RING_SLOTS], *m0;
+ uint16_t sent = 0, burst, i, port_id;
+ uint8_t qid;
RTE_SET_USED(port);
- m0 = (struct rte_mbuf *)ev[0].mbuf;
+ m0 = ev[0].mbuf;
qid = rte_event_eth_tx_adapter_txq_get(m0);
+ port_id = m0->port;
+
+ /* The advertised enqueue depth is the EQCR size, which may exceed the
+ * on-stack burst array, so transmit in ring-sized chunks.
+ */
+ while (sent < nb_events) {
+ burst = RTE_MIN(nb_events - sent, (uint16_t)RTE_DIM(m));
+ for (i = 0; i < burst; i++)
+ m[i] = ev[sent + i].mbuf;
+
+ i = rte_eth_tx_burst(port_id, qid, m, burst);
+ sent += i;
+ if (i < burst)
+ break;
+ }
- for (i = 0; i < nb_events; i++)
- m[i] = (struct rte_mbuf *)ev[i].mbuf;
-
- return rte_eth_tx_burst(m0->port, qid, m, nb_events);
+ return sent;
}
static uint16_t
@@ -996,19 +1007,28 @@ dpaa2_eventdev_txa_enqueue(void *port,
struct rte_event ev[],
uint16_t nb_events)
{
- void *txq[DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH];
- struct rte_mbuf *m[DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH];
- uint8_t qid, i;
+ void *txq[MAX_TX_RING_SLOTS];
+ struct rte_mbuf *m[MAX_TX_RING_SLOTS];
+ uint16_t sent = 0, burst, i;
+ uint8_t qid;
RTE_SET_USED(port);
- for (i = 0; i < nb_events; i++) {
- m[i] = (struct rte_mbuf *)ev[i].mbuf;
- qid = rte_event_eth_tx_adapter_txq_get(m[i]);
- txq[i] = rte_eth_devices[m[i]->port].data->tx_queues[qid];
+ while (sent < nb_events) {
+ burst = RTE_MIN(nb_events - sent, (uint16_t)RTE_DIM(m));
+ for (i = 0; i < burst; i++) {
+ m[i] = ev[sent + i].mbuf;
+ qid = rte_event_eth_tx_adapter_txq_get(m[i]);
+ txq[i] = rte_eth_devices[m[i]->port].data->tx_queues[qid];
+ }
+
+ i = dpaa2_dev_tx_multi_txq_ordered(txq, m, burst);
+ sent += i;
+ if (i < burst)
+ break;
}
- return dpaa2_dev_tx_multi_txq_ordered(txq, m, nb_events);
+ return sent;
}
static struct eventdev_ops dpaa2_eventdev_ops = {
diff --git a/drivers/event/dpaa2/dpaa2_eventdev.h b/drivers/event/dpaa2/dpaa2_eventdev.h
index 3883a26c91..a054dfdbd7 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.h
+++ b/drivers/event/dpaa2/dpaa2_eventdev.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017 NXP
+ * Copyright 2017,2026 NXP
*/
#ifndef __DPAA2_EVENTDEV_H__
@@ -22,8 +22,6 @@
#define DPAA2_EVENT_MAX_QUEUE_FLOWS 2048
#define DPAA2_EVENT_MAX_QUEUE_PRIORITY_LEVELS 8
#define DPAA2_EVENT_MAX_EVENT_PRIORITY_LEVELS 0
-#define DPAA2_EVENT_MAX_PORT_DEQUEUE_DEPTH 8
-#define DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH 8
#define DPAA2_EVENT_MAX_NUM_EVENTS (INT32_MAX - 1)
#define DPAA2_EVENT_QUEUE_ATOMIC_FLOWS 2048
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 42/45] net/dpaa2: read MC version from device private data
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (40 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 41/45] drivers: align dpaa2 event port depths with hardware rings Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 43/45] net/dpaa2: do not overwrite mbuf hash with drop priority Prashant Gupta
` (2 subsequent siblings)
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev
The flow engine reached back to the bus device to read the MC firmware
version. The bus_info pointer behind it is only attached by the primary
process, so a secondary process that creates a flow dereferences a
pointer it never set up.
The version is already cached in priv->mc_rev at probe time and the rest
of the driver reads it from there, so use that here too. This removes the
last user of DPAA2_DEV_PRIV_TO_DPAA2_DEV(), so drop the macro.
Signed-off-by: Prashant Gupta <prashant.gupta_3@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.h | 4 ---
drivers/net/dpaa2/dpaa2_flow.c | 59 ++++++++++----------------------
2 files changed, 18 insertions(+), 45 deletions(-)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 7d93b75be3..e02fa9a622 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -43,10 +43,6 @@
#define MAX_DPNI 8
#define DPAA2_MAX_CHANNELS 16
-#define DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv) \
- container_of((((struct dpaa2_dev_priv *)priv)->eth_dev->device), \
- struct rte_dpaa2_device, device)
-
#define DPAA2_EXTRACT_PARAM_MAX_SIZE \
RTE_ALIGN(sizeof(struct dpni_ext_set_rx_tc_dist), 256)
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 8456b3b80d..187381c5dd 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -781,13 +781,10 @@ static int
dpaa2_flow_add_fs_rule(struct dpaa2_dev_priv *priv,
struct dpaa2_generic_flow *flow)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct dpaa2_flow_tbl_profile *tbl_profile;
int ret;
struct fsl_mc_io *dpni = priv->hw;
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
-
tbl_profile = &priv->flow_profile.tc_profile[flow->tc_id];
if (dpaa2_flow_entry_map_get(tbl_profile->entry_map,
flow->entry_index)) {
@@ -798,7 +795,7 @@ dpaa2_flow_add_fs_rule(struct dpaa2_dev_priv *priv,
dpaa2_flow_fs_entry_log("Add", flow);
- if (dpaa2_dev->bus_info->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV) {
+ if (priv->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV) {
ret = dpni_add_fs_entry(dpni, CMD_PRI_LOW,
priv->token, flow->tc_id, flow->entry_index,
&flow->rule_cfg, &flow->flow_action.fs_action.fs_action_cfg);
@@ -3642,7 +3639,6 @@ dpaa2_flow_fs_action_config(struct dpaa2_dev_priv *priv,
struct dpaa2_generic_flow *flow,
const struct rte_flow_action *rte_action)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct rte_eth_dev *dest_dev;
struct dpaa2_dev_priv *dest_priv;
const struct rte_flow_action_queue *dest_queue;
@@ -3651,11 +3647,9 @@ dpaa2_flow_fs_action_config(struct dpaa2_dev_priv *priv,
uint16_t num_tokens;
struct dpaa2_dev_flow_fs_action *fs_action;
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
-
fs_action = &flow->flow_action.fs_action;
if (fs_action->action_type != RTE_FLOW_ACTION_TYPE_END) {
- if (dpaa2_dev->bus_info->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV &&
+ if (priv->mc_rev >= DPAA2_FLOW_FRM_REPLICATION_ACTION_MC_REV &&
(fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_ID ||
fs_action->action_type == RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR)) {
if (rte_action->type != RTE_FLOW_ACTION_TYPE_PORT_ID &&
@@ -3807,21 +3801,19 @@ dpaa2_flow_fs_table_set_default(struct dpaa2_dev_priv *priv,
uint8_t tc_id, int discard, uint16_t default_queue)
{
int ret;
- struct rte_dpaa2_device *dpaa2_dev;
struct dpni_rx_dist_cfg *tc_cfg;
struct fsl_mc_io *dpni = priv->hw;
struct dpaa2_flow_tbl_profile *tbl_profile;
struct dpaa2_queue *queue;
char mc_rev[1024];
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
tbl_profile = &priv->flow_profile.tc_profile[tc_id];
snprintf(mc_rev, 1024, "MC rev(%d.%d.%d)",
- RTE_FSL_MC_REV_MAJOR(dpaa2_dev->bus_info->mc_rev),
- RTE_FSL_MC_REV_MINOR(dpaa2_dev->bus_info->mc_rev),
- RTE_FSL_MC_REV_REVISION(dpaa2_dev->bus_info->mc_rev));
+ RTE_FSL_MC_REV_MAJOR(priv->mc_rev),
+ RTE_FSL_MC_REV_MINOR(priv->mc_rev),
+ RTE_FSL_MC_REV_REVISION(priv->mc_rev));
if (!tbl_profile->dpkg.num_extracts &&
- dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
+ priv->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
DPAA2_PMD_DEBUG("%s can't set miss action of FS table indepentently.",
mc_rev);
return 0;
@@ -3946,26 +3938,24 @@ dpaa2_flow_qos_table_set_default(struct dpaa2_dev_priv *priv,
int discard, uint8_t default_tc, uint16_t default_flow)
{
int ret;
- struct rte_dpaa2_device *dpaa2_dev;
struct dpni_qos_tbl_cfg qos_cfg;
struct fsl_mc_io *dpni = priv->hw;
struct dpaa2_flow_tbl_profile *tbl_profile;
char mc_rev[1024];
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
tbl_profile = &priv->flow_profile.qos_profile;
snprintf(mc_rev, 1024, "MC rev(%d.%d.%d)",
- RTE_FSL_MC_REV_MAJOR(dpaa2_dev->bus_info->mc_rev),
- RTE_FSL_MC_REV_MINOR(dpaa2_dev->bus_info->mc_rev),
- RTE_FSL_MC_REV_REVISION(dpaa2_dev->bus_info->mc_rev));
+ RTE_FSL_MC_REV_MAJOR(priv->mc_rev),
+ RTE_FSL_MC_REV_MINOR(priv->mc_rev),
+ RTE_FSL_MC_REV_REVISION(priv->mc_rev));
if (!tbl_profile->dpkg.num_extracts &&
- dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
+ priv->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV) {
DPAA2_PMD_DEBUG("%s can't set miss action of QoS table indepentently.",
mc_rev);
return 0;
}
if (default_flow < priv->dist_queues &&
- dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_MISS_FLOW_ACTION_MC_REV) {
+ priv->mc_rev < DPAA2_QOS_FLOW_TABLE_MISS_FLOW_ACTION_MC_REV) {
DPAA2_PMD_WARN("%s can't direct miss traffic to TC%d-flow%d by QoS table only.",
mc_rev, default_tc, default_flow);
return 0;
@@ -4002,7 +3992,6 @@ static int
dpaa2_flow_qos_table_config(struct dpaa2_dev_priv *priv,
int rss_dist)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct dpaa2_flow_tbl_profile *tbl_profile;
uint8_t *key_cfg_buf;
int ret;
@@ -4064,8 +4053,7 @@ dpaa2_flow_qos_table_config(struct dpaa2_dev_priv *priv,
qos_cfg->discard_on_miss = false;
}
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
- if (dpaa2_dev->bus_info->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV)
+ if (priv->mc_rev < DPAA2_QOS_FLOW_TABLE_SET_V3_MC_REV)
ret = dpni_set_qos_table_v2(dpni, CMD_PRI_LOW, priv->token, qos_cfg);
else
ret = dpni_set_qos_table(dpni, CMD_PRI_LOW, priv->token, qos_cfg);
@@ -4209,7 +4197,6 @@ static int
dpaa2_flow_set_police_action(struct dpaa2_dev_priv *priv,
uint8_t tc_id, const struct rte_flow_action_meter_mark *meter_mark)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct dpni_rx_tc_policing_cfg policing_cfg = {0};
const struct dpaa2_dev_meter_profile *dpaa2_profile;
const struct dpaa2_dev_meter_policy *dpaa2_policy = NULL;
@@ -4221,8 +4208,6 @@ dpaa2_flow_set_police_action(struct dpaa2_dev_priv *priv,
return -ENOTSUP;
}
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
-
dpaa2_profile = (void *)meter_mark->profile;
if (!dpaa2_profile) {
DPAA2_PMD_ERR("Meter profile not specified!");
@@ -4244,9 +4229,8 @@ dpaa2_flow_set_police_action(struct dpaa2_dev_priv *priv,
if (priv->flow_profile.mtr_flow[tc_id]) {
/** Update existing policer.*/
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
policing_cfg.options |= DPNI_POLICER_OPT_DO_NOT_RESET_COUNTERS;
- if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_NOT_RESET_COUNTER_MC_REV)
+ if (priv->mc_rev < DPAA2_POLICER_NOT_RESET_COUNTER_MC_REV)
DPAA2_PMD_WARN("The existing policer's counters will be cleaned.");
}
@@ -4269,7 +4253,7 @@ dpaa2_flow_set_police_action(struct dpaa2_dev_priv *priv,
policing_cfg.eir = dpaa2_profile->pir;
policing_cfg.ebs = dpaa2_profile->pbs;
- if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
+ if (priv->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
ret = dpni_set_rx_tc_policing_v1(priv->hw, CMD_PRI_LOW,
priv->token, tc_id, &policing_cfg);
} else {
@@ -5400,7 +5384,6 @@ dpaa2_flow_generic_flow_create(struct rte_eth_dev *dev,
int mix_extract, int is_rss)
{
struct dpaa2_generic_flow *flow = NULL;
- struct rte_dpaa2_device *dpaa2_dev;
struct dpaa2_dev_priv *priv = dev->data->dev_private;
int ret, idx = -1;
uint64_t iova, mc_rev;
@@ -5416,8 +5399,7 @@ dpaa2_flow_generic_flow_create(struct rte_eth_dev *dev,
if (type != DPAA2_FLOW_QOS_TYPE && type != DPAA2_FLOW_FS_TYPE)
return NULL;
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
- mc_rev = dpaa2_dev->bus_info->mc_rev;
+ mc_rev = priv->mc_rev;
if (actions) {
ret = dpaa2_flow_qos_fs_action_set(actions, qos_actions,
@@ -5912,19 +5894,17 @@ static int
dpaa2_flow_destroy_meter_flow(struct rte_eth_dev *dev,
struct dpaa2_dev_flow *flow)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct dpaa2_dev_priv *priv = dev->data->dev_private;
struct dpni_rx_tc_policing_cfg cfg;
int ret;
uint8_t tc_id;
RTE_ASSERT(!flow->qos_flow && flow->fs_flow);
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
tc_id = flow->fs_flow->tc_id;
RTE_ASSERT(priv->flow_profile.mtr_flow[tc_id] == flow);
memset(&cfg, 0, sizeof(cfg));
cfg.mode = DPNI_POLICER_MODE_NONE;
- if (dpaa2_dev->bus_info->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
+ if (priv->mc_rev < DPAA2_POLICER_SET_V2_MC_REV) {
ret = dpni_set_rx_tc_policing_v1(priv->hw, CMD_PRI_LOW,
priv->token, tc_id, &cfg);
} else {
@@ -6141,7 +6121,6 @@ dpaa2_flow_actions_update(struct rte_eth_dev *dev,
const struct rte_flow_action actions[],
struct rte_flow_error *error)
{
- struct rte_dpaa2_device *dpaa2_dev;
struct dpaa2_dev_priv *priv = dev->data->dev_private;
struct dpaa2_dev_flow *flow;
struct dpaa2_flow_tbl_profile *tbl_profile = NULL;
@@ -6162,8 +6141,6 @@ dpaa2_flow_actions_update(struct rte_eth_dev *dev,
const struct rte_flow_action_queue *action_q;
struct dpaa2_queue *rxq = NULL;
- dpaa2_dev = DPAA2_DEV_PRIV_TO_DPAA2_DEV(priv);
-
/* check for the valid flow */
flow = (void *)_flow;
if (flow->is_meter_flow) {
@@ -6275,7 +6252,7 @@ dpaa2_flow_actions_update(struct rte_eth_dev *dev,
if (fs_action_num > 0 &&
fs_actions[0].type != RTE_FLOW_ACTION_TYPE_PORT_ID &&
fs_actions[0].type != RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT &&
- dpaa2_dev->bus_info->mc_rev >= DPAA2_FS_FLOW_HW_ACTION_UPDATE_MC_REV) {
+ priv->mc_rev >= DPAA2_FS_FLOW_HW_ACTION_UPDATE_MC_REV) {
/** Action HW update doesn't support redirecting frames to other DPNIs.*/
hw_update = true;
goto skip_remove_fs_entry;
@@ -6365,7 +6342,7 @@ dpaa2_flow_actions_update(struct rte_eth_dev *dev,
}
hw_update = false;
- if (dpaa2_dev->bus_info->mc_rev >= DPAA2_QOS_FLOW_HW_ACTION_UPDATE_MC_REV) {
+ if (priv->mc_rev >= DPAA2_QOS_FLOW_HW_ACTION_UPDATE_MC_REV) {
/** Action HW update doesn't support redirecting frames to other DPNIs.*/
hw_update = true;
goto skip_remove_qos_entry;
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 43/45] net/dpaa2: do not overwrite mbuf hash with drop priority
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (41 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 42/45] net/dpaa2: read MC version from device private data Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 44/45] bus/fslmc: reduce probe-time logging and MC traffic Prashant Gupta
2026-09-03 13:53 ` [PATCH 45/45] net/dpaa2: reject Rx queue deferred start Prashant Gupta
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Gagandeep Singh
From: Gagandeep Singh <g.singh@nxp.com>
dpaa2_eth_fd_to_mbuf() wrote the frame descriptor's drop priority into
mbuf->hash.sched.color unconditionally. hash.sched shares the union with
hash.rss and hash.fdir, so this clobbered whatever the parse step had
placed there. The scheduler color is only meaningful when the frame came
out of a flow-steering table, and dpaa2_dev_rx_mbuf_sched_set() already
sets the whole sched field (queue, traffic class and colour) in that
case, so drop the unconditional write.
Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
drivers/net/dpaa2/dpaa2_rxtx.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 5c6a77d2f8..f2137bddd4 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -495,7 +495,6 @@ dpaa2_eth_fd_to_mbuf(struct dpaa2_dev_priv *priv, const struct qbman_fd *fd)
mbuf->pkt_len = mbuf->data_len;
mbuf->port = priv->eth_dev->data->port_id;
mbuf->next = NULL;
- mbuf->hash.sched.color = DPAA2_GET_FD_DROPP(fd);
rte_mbuf_refcnt_set(mbuf, 1);
#ifdef RTE_LIBRTE_MEMPOOL_DEBUG
rte_mempool_check_cookies(rte_mempool_from_obj((void *)mbuf),
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 44/45] bus/fslmc: reduce probe-time logging and MC traffic
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (42 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 43/45] net/dpaa2: do not overwrite mbuf hash with drop priority Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
2026-09-03 13:53 ` [PATCH 45/45] net/dpaa2: reject Rx queue deferred start Prashant Gupta
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Every DMA map logged its virtual, IOVA and physical addresses at NOTICE.
An application that maps many memory segments prints one such line per
segment during initialisation, which drowns out messages the operator
actually needs; the neighbouring mapping paths already use debug level.
fslmc_vfio_get_bus_info() also issued mc_get_soc_version() twice in a row
into the same structure, so the second command only overwrote the first
result with an identical one. Keep a single query.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/bus/fslmc/fslmc_vfio.c | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
index 4eb8552696..5ccb457a75 100644
--- a/drivers/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -281,13 +281,6 @@ fslmc_vfio_get_bus_info(void *regs, struct rte_fslmc_bus_info *bus_info)
return ret;
}
- memset(&mc_plat_info, 0, sizeof(struct mc_soc_version));
- ret = mc_get_soc_version(&mc_io, CMD_PRI_LOW, &mc_plat_info);
- if (ret) {
- DPAA2_BUS_ERR("Unable to get SoC version err:%d", ret);
- return ret;
- }
-
for (i = 0; i < RTE_DIM(s_soc_type); i++) {
if ((mc_plat_info.svr & DPAA2_SVR_MASK) == s_soc_type[i].svr) {
dpaa2_svr_family = s_soc_type[i].svr;
@@ -1041,8 +1034,7 @@ fslmc_map_dma(uint64_t vaddr, rte_iova_t iovaddr, size_t len)
fslmc_mem_va2iova = RTE_BAD_IOVA;
TAILQ_INSERT_TAIL(&fslmc_memsegs, dmaseg, next);
}
- DPAA2_BUS_LOG(NOTICE,
- "%s(%zx): VA(%" PRIx64 "):IOVA(%" PRIx64 "):PHY(%" PRIx64 ")",
+ DPAA2_BUS_DEBUG("%s(%zx): VA(%" PRIx64 "):IOVA(%" PRIx64 "):PHY(%" PRIx64 ")",
is_io ? "DMA I/O map size" : "DMA MEM map size",
len, vaddr, iovaddr, phy);
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH 45/45] net/dpaa2: reject Rx queue deferred start
2026-09-03 13:53 [PATCH 00/45] net/dpaa2: features and fixes for NXP DPAA2 drivers Prashant Gupta
` (43 preceding siblings ...)
2026-09-03 13:53 ` [PATCH 44/45] bus/fslmc: reduce probe-time logging and MC traffic Prashant Gupta
@ 2026-09-03 13:53 ` Prashant Gupta
44 siblings, 0 replies; 47+ messages in thread
From: Prashant Gupta @ 2026-09-03 13:53 UTC (permalink / raw)
To: stephen, dev; +Cc: Hemant Agrawal
From: Hemant Agrawal <hemant.agrawal@nxp.com>
The driver has no way to hold an Rx queue down while the port is started,
and dpaa2_rxq_info_get() already reports rx_deferred_start as 0. The Tx
side rejects the request; do the same for Rx instead of silently
accepting a configuration that will not be honoured.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 7cd38ddce6..596806c18a 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1439,6 +1439,13 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
DPAA2_PMD_DEBUG("dev =%p, queue =%d, pool = %p, conf =%p",
dev, rx_queue_id, mb_pool, rx_conf);
+ /* Rx deferred start is not supported */
+ if (rx_conf->rx_deferred_start) {
+ DPAA2_PMD_ERR("%s: Rx deferred start not supported",
+ dev->data->name);
+ return -EINVAL;
+ }
+
total_nb_rx_desc += nb_rx_desc;
if (total_nb_rx_desc > MAX_NB_RX_DESC_IN_PEB &&
(priv->options & DPNI_OPT_V1_PFDR_IN_PEB)) {
--
2.43.0
^ permalink raw reply related [flat|nested] 47+ messages in thread