* Re: [PATCH] app: remove use of strncpy
From: Thomas Monjalon @ 2026-07-11 22:57 UTC (permalink / raw)
To: Bruce Richardson
Cc: dev, stable, Kai Ji, Cheng Jiang, Chengwen Feng, Jerin Jacob,
Ori Kam, Aman Singh, Michal Kobylinski, Piotr Azarewicz,
Marcin Kerlin, Slawomir Mrozowicz, Declan Doherty,
Aleksander Gajewski, Pablo de Lara, Guduri Prathyusha,
Harry van Haaren, Morten Brørup, Jiayu Hu, Yuan Wang,
Anoob Joseph, Xiaoyu Min, Xueming Li, Yuval Avnery
In-Reply-To: <20260623145028.711712-1-bruce.richardson@intel.com>
23/06/2026 16:50, Bruce Richardson:
> Use of strncpy is not recommended, so replace it with strlcpy or memcpy
> as appropriate.
>
> Fixes: f8be1786b1b8 ("app/crypto-perf: introduce performance test application")
> Fixes: 8ecd4048ba5d ("app/crypto-perf: fix string not null terminated")
> Fixes: 0add6c27cd7c ("app/testeventdev: define the test options")
> Fixes: 623dc9364dc6 ("app/dma-perf: introduce DMA performance test")
> Fixes: 1e8a4e97b057 ("app/testpmd: add flow dump command")
> Fixes: de06137cb295 ("app/regex: add RegEx test application")
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH] app/flow-perf: replace RTE_BE32/16 with rte_cpu_to_be_32/16 for variables
From: Thomas Monjalon @ 2026-07-11 22:45 UTC (permalink / raw)
To: Harold Huang, dev; +Cc: Wisam Jaddo
In-Reply-To: <20230312020009.288-1-baymaxhuang@gmail.com>
> In DPDK, the macros RTE_BE32 or RTE_BE16 are usually used for
> constant values. And functions such as rte_cpu_to_be_32 or
> rte_cpu_to_be_16 are optimized for variables.
>
> Signed-off-by: Harold Huang <baymaxhuang@gmail.com>
Sorry this patch was forgotten.
Applied, thanks.
^ permalink raw reply
* [PATCH] net/ice: fix missing return on unsupported range in FDIR
From: sandeep.penigalapati @ 2026-07-12 3:26 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, stable, Sandeep Penigalapati
From: Sandeep Penigalapati <sandeep.penigalapati@intel.com>
In ice_fdir_parse_pattern(), a flow item that specifies a 'last'
member (a range) is only allowed for IPV4 and IPV6_FRAG_EXT item
types. For other item types the code set an rte_flow error but did
not return, so parsing continued from an inconsistent state.
Add the missing return so an unsupported range aborts parsing.
Fixes: b7e8781de768 ("net/ice: support flow director for IP fragment packet")
Cc: stable@dpdk.org
Signed-off-by: Sandeep Penigalapati <sandeep.penigalapati@intel.com>
---
drivers/net/intel/ice/ice_fdir_filter.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/net/intel/ice/ice_fdir_filter.c b/drivers/net/intel/ice/ice_fdir_filter.c
index fc848a33db..d7ddd32f90 100644
--- a/drivers/net/intel/ice/ice_fdir_filter.c
+++ b/drivers/net/intel/ice/ice_fdir_filter.c
@@ -1929,6 +1929,7 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM, item,
"Not support range");
+ return -rte_errno;
}
input_set = (tunnel_type && !is_outer) ?
--
2.27.0
^ permalink raw reply related
* Re: [PATCH] examples/l3fwd-power: fix missing return on latency get
From: Thomas Monjalon @ 2026-07-11 14:15 UTC (permalink / raw)
To: Huisong Li
Cc: anatoly.burakov, sivaprasad.tummala, dev, stephen, fengchengwen,
yangxingui, zhanjie9
In-Reply-To: <20260512010900.340129-1-lihuisong@huawei.com>
12/05/2026 03:09, Huisong Li:
> Add return value when get cpu_resume_latency failed.
>
> Fixes: 4d23d39fd06e ("examples/l3fwd-power: add PM QoS configuration")
> Cc: stable@dpdk.org
>
> Signed-off-by: Huisong Li <lihuisong@huawei.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH v4 0/9] power: centralize lcore ID validation
From: Thomas Monjalon @ 2026-07-11 14:06 UTC (permalink / raw)
To: Huisong Li
Cc: anatoly.burakov, sivaprasad.tummala, dev, stephen, yangxingui,
zhanjie9, fengchengwen, Macnamara, Chris
In-Reply-To: <2a191a7f-5439-407b-b2cd-44dae6ab0cbc@huawei.com>
15/06/2026 11:22, fengchengwen:
> On 6/15/2026 3:30 PM, Huisong Li wrote:
> > This series centralizes the lcore ID verification in the power cpufreq
> > framework, replacing the per-driver range checks with a common validation.
> >
> > Background
> > ----------
> > Currently, various cpufreq drivers implement their own lcore ID checks,
> > which are limited to simple range validation against RTE_MAX_LCORE and
> > involve significant code duplication across 12+ functions per driver.
> > The checks are duplicated across all drivers — any change requires
> > updating 5+ drivers identically. Moreover, these checks do not verify
> > whether the lcore is actually managed by the application. So it is better
> > to verify lcore ID in cpufreq core.
> >
> > For cpufreq-related APIs, although service cores do not typically invoke
> > these APIs, they may operate in polling modes where power management is
> > required. To maintain compatibility with applications using service cores,
> > the validation logic now explicitly accepts both ROLE_RTE and ROLE_SERVICE.
> >
> > The usage of power QoS APIs are similar to that of cpufreq. They also can
> > accepts ROLE_RTE and ROLE_SERVICE.
> >
> > For PMD power management APIs, the lcore must be ROLE_RTE because these
> > are used together with the data plane of ethdev PMD. Hence,
> > rte_lcore_is_enabled() is used for validation.
>
> LGTM
> Series-acked-by: Chengwen Feng <fengchengwen@huawei.com>
Applied, thanks.
There are other patches for the power library which require reviews.
Please all, review each other to unblock patches.
^ permalink raw reply
* Re: [PATCH 1/2] bpf: default log level should be NOTICE
From: Thomas Monjalon @ 2026-07-11 12:43 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev@dpdk.org, Konstantin Ananyev, Marat Khalili
In-Reply-To: <35e932e0908a4326bc4430ad947fbcfb@huawei.com>
> > BPF is used by other services now and the default log
> > level should match other libraries.
> >
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>
> Acked-by: Marat Khalili <marat.khalili@huawei.com>
Series applied, thanks.
^ permalink raw reply
* Re: [PATCH 2/2] bpf: silence noisy message
From: Thomas Monjalon @ 2026-07-11 12:37 UTC (permalink / raw)
To: Stephen Hemminger, Konstantin Ananyev, Marat Khalili; +Cc: dev
In-Reply-To: <5745cb1553bb4e0e870d36e36755187e@huawei.com>
02/07/2026 18:34, Marat Khalili:
> > https://github.com/DPDK/dpdk/blob/09e4f37ddd0f7d5cc7c696741f31f1582c73a716/lib/bpf/bpf_load_elf.c#L331
> > -L333
> > >
> > > (In fact I tried to eliminate it, but had to but back at the insistence of AI reviewer
> > > and even broke the build in the process...)
> > >
> > > So naturally I support downgrading this message. With the commit message corrected,
> > >
> > > Acked-by: Marat Khalili <marat.khalili@huawei.com>
> >
> > Looks like you added that message in:
> > 55192fe168b9 ("bpf: introduce extensible load API")
>
> It was moved from a different file, see above.
> Maybe it applied to fewer code paths since they are now all merged.
>
> Anyway it's not really important unless you think it's worth backporting.
> Feel free to downgrade or remove this message.
Fixes: 5dba93ae5f2d ("bpf: add ability to load eBPF program from ELF object file")
Cc: stable@dpdk.org
^ permalink raw reply
* Re: [PATCH v2 0/6] crypto: use timing-safe digest comparison
From: Thomas Monjalon @ 2026-07-11 11:06 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
In-Reply-To: <20260629190027.2071745-1-stephen@networkplumber.org>
> Stephen Hemminger (6):
> eal: take experimental flag off of rte_memeq_timingsafe
> crypto/uadk: use timing-safe digest comparison
> crypto/ccp: use timing-safe digest comparison
> crypto/armv8: use timing-safe digest comparison
> crypto/cnxk: use timing-safe digest comparison
> crypto/octeontx: use timing-safe RSA signature verification
Applied, thanks.
^ permalink raw reply
* Re: [PATCH v4 0/6] net/dpaa2: NAPI-style Rx queue interrupts
From: Thomas Monjalon @ 2026-07-11 6:47 UTC (permalink / raw)
To: Maxime Leroy; +Cc: dev, Hemant Agrawal, Jun Yang
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
> Maxime Leroy (6):
> net/dpaa2: fix Rx queue count for primary process
> bus/fslmc: move DPCON management from event driver to bus
> bus/fslmc/dpio: make the portal DQRI epoll optional
> bus/fslmc/mc: implement DPCON set notification
> net/dpaa2: support Rx queue interrupts
> net/dpaa2: pin Rx queue interrupt to the polling core
Applied, thanks.
^ permalink raw reply
* Re: [PATCH v7 05/16] bus/dpaa: shutdown DPAA FQ by fq descriptor
From: Thomas Monjalon @ 2026-07-11 6:40 UTC (permalink / raw)
To: Jun Yang, Hemant Agrawal; +Cc: stephen, david.marchand, dev
In-Reply-To: <20260703124950.1895871-6-hemant.agrawal@nxp.com>
03/07/2026 14:49, Hemant Agrawal:
> From: Jun Yang <jun.yang@nxp.com>
>
> Pass the full FQ descriptor to qman_shutdown_fq() instead of
> just the fqid, so that channel-affine portals can be correctly
> accessed when shutting down push-mode Rx queues.
>
> Signed-off-by: Jun Yang <jun.yang@nxp.com>
> ---
> drivers/bus/dpaa/base/qbman/qman.c | 9 +++++----
> drivers/bus/dpaa/include/fsl_qman.h | 11 ++++++++++-
> 2 files changed, 15 insertions(+), 5 deletions(-)
>
> --- a/drivers/bus/dpaa/base/qbman/qman.c
> +++ b/drivers/bus/dpaa/base/qbman/qman.c
> @@ -2781,18 +2781,19 @@ qm_mc_result_timeout(struct qm_portal *portal,
>
> RTE_EXPORT_INTERNAL_SYMBOL(qman_shutdown_fq)
> int
> -qman_shutdown_fq(u32 fqid)
> +qman_shutdown_fq(struct qman_fq *fq)
You did not change the call:
drivers/net/dpaa/dpaa_ethdev.c:1124:29: error:
passing argument 1 of 'qman_shutdown_fq' makes pointer from integer without a cast [-Wint-conversion]
1124 | qman_shutdown_fq(rxq->fqid);
^ permalink raw reply
* [PATCH v6 2/2] dts: add build arguments to test run configuration
From: Koushik Bhargav Nimoji @ 2026-07-11 0:02 UTC (permalink / raw)
To: luca.vizzarro, patrickrobb1997
Cc: dev, abailey, ahassick, lylavoie, Koushik Bhargav Nimoji
In-Reply-To: <20260711000225.1836433-1-knimoji@iol.unh.edu>
This patch adds the ability to specify build arguments when building DPDK
through DTS. Doing so allows users to build DPDK with the desired build
arguments, which allows for a more configurable DTS run.
Signed-off-by: Koushik Bhargav Nimoji <knimoji@iol.unh.edu>
---
dts/configurations/test_run.example.yaml | 13 +++++++++++++
dts/framework/config/test_run.py | 2 ++
dts/framework/remote_session/dpdk.py | 16 ++++++++++------
dts/framework/utils.py | 21 ++++++++++++++++++++-
4 files changed, 45 insertions(+), 7 deletions(-)
diff --git a/dts/configurations/test_run.example.yaml b/dts/configurations/test_run.example.yaml
index 51f9ab8237..182ee377aa 100644
--- a/dts/configurations/test_run.example.yaml
+++ b/dts/configurations/test_run.example.yaml
@@ -16,6 +16,8 @@
# `precompiled_build_dir` or `build_options` can be defined, but not both.
# `compiler_wrapper`:
# Optional, adds a compiler wrapper if present.
+# `build_args`:
+# The additional build arguments to be used when building DPDK.
# `func_traffic_generator` & `perf_traffic_generator`:
# Define `func_traffic_generator` when `func` set to true.
# Define `perf_traffic_generator` when `perf` set to true.
@@ -46,6 +48,17 @@ dpdk:
# the combination of the following two makes CC="ccache gcc"
compiler: gcc
compiler_wrapper: ccache # see `Optional Fields`
+ # arguments to be used when building DPDK
+ # build_args:
+ # c_args:
+ # - O3
+ # - g
+ # b_coverage:
+ # - "true"
+ # buildtype:
+ # - release
+ # flags:
+ # - strip
func_traffic_generator:
type: SCAPY
# perf_traffic_generator:
diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test_run.py
index 81630df77d..cf6569ea0c 100644
--- a/dts/framework/config/test_run.py
+++ b/dts/framework/config/test_run.py
@@ -191,6 +191,8 @@ class DPDKBuildOptionsConfiguration(FrozenModel):
#: This string will be put in front of the compiler when executing the build. Useful for adding
#: wrapper commands, such as ``ccache``.
compiler_wrapper: str = ""
+ #: The build arguments to build dpdk with
+ build_args: dict[str, list[str]] = {}
class DPDKUncompiledBuildConfiguration(BaseDPDKBuildConfiguration):
diff --git a/dts/framework/remote_session/dpdk.py b/dts/framework/remote_session/dpdk.py
index 683bc3470e..58d20efebe 100644
--- a/dts/framework/remote_session/dpdk.py
+++ b/dts/framework/remote_session/dpdk.py
@@ -109,8 +109,8 @@ def setup(self) -> None:
"Cannot create code coverage report using a precompiled build directory."
)
self._set_remote_dpdk_build_dir(build_dir)
- case DPDKUncompiledBuildConfiguration(build_options=build_options):
- self._configure_dpdk_build(build_options)
+ case DPDKUncompiledBuildConfiguration():
+ self._configure_dpdk_build(self.config.build_options)
self._build_dpdk()
def teardown(self) -> None:
@@ -295,17 +295,21 @@ def _build_dpdk(self) -> None:
`remote_dpdk_tree_path` has already been set on the SUT node.
"""
ctx = get_ctx()
+ build_options = getattr(self.config, "build_options")
# If the SUT is an ice driver device, make sure to build with 16B descriptors.
if (
ctx.topology.type is not LinkTopology.NO_LINK
and ctx.topology.sut_port_ingress
and ctx.topology.sut_port_ingress.config.os_driver == "ice"
):
- meson_args = MesonArgs(
- default_library="static", libdir="lib", c_args="-DRTE_NET_INTEL_USE_16BYTE_DESC"
- )
+ if "c_args" in build_options.build_args:
+ build_options.build_args["c_args"].append("DRTE_NET_INTEL_USE_16BYTE_DESC")
+ else:
+ build_options.build_args["c_args"] = ["DRTE_NET_INTEL_USE_16BYTE_DESC"]
+
+ meson_args = MesonArgs(build_options.build_args, default_library="static", libdir="lib")
else:
- meson_args = MesonArgs(default_library="static", libdir="lib")
+ meson_args = MesonArgs(build_options.build_args, default_library="static", libdir="lib")
if SETTINGS.code_coverage:
meson_args._add_arg("-Db_coverage=true")
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 55d6dfc337..0e41f3f48c 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -101,10 +101,16 @@ class MesonArgs:
_default_library: str
- def __init__(self, default_library: str | None = None, **dpdk_args: str | bool):
+ def __init__(
+ self,
+ dpdk_build_args: dict[str, list[str]],
+ default_library: str | None = None,
+ **dpdk_args: str | bool,
+ ):
"""Initialize the meson arguments.
Args:
+ dpdk_build_args: The DPDK build arguments specified in the test run configuration file.
default_library: The default library type, Meson supports ``shared``, ``static`` and
``both``. Defaults to :data:`None`, in which case the argument won't be used.
dpdk_args: The arguments found in ``meson_options.txt`` in root DPDK directory.
@@ -123,6 +129,19 @@ def __init__(self, default_library: str | None = None, **dpdk_args: str | bool):
)
)
+ arguments = []
+ for option, value in dpdk_build_args.items():
+ if option == "c_args":
+ values = " ".join(f"-{val}" for val in value)
+ arguments.append(f'-D{option}="{values}"')
+ elif option == "flags":
+ values = " ".join(f"--{val}" for val in value)
+ arguments.append(values)
+ else:
+ arguments.append(f" -D{option}={value[0]}")
+
+ self._dpdk_args = " ".join(f"{self._dpdk_args} {' '.join(arguments)}".split())
+
def __str__(self) -> str:
"""The actual args."""
return " ".join(f"{self._default_library} {self._dpdk_args}".split())
--
2.54.0
^ permalink raw reply related
* [PATCH v6 1/2] dts: add code coverage reporting to DTS
From: Koushik Bhargav Nimoji @ 2026-07-11 0:02 UTC (permalink / raw)
To: luca.vizzarro, patrickrobb1997
Cc: dev, abailey, ahassick, lylavoie, Koushik Bhargav Nimoji
In-Reply-To: <20260522154637.952588-1-knimoji@iol.unh.edu>
Previously, DTS had no code coverage. This patch adds a command line
argument in order to build DPDK with code coverage enabled. This allows
users to create and view code coverage reports of what code and functions
were called during a DTS run.
Signed-off-by: Koushik Bhargav Nimoji <knimoji@iol.unh.edu>
---
v2:
*Fixed error in lcov/gcov tool detection
v3:
*Fixed type hints and error message typos
v4:
*Fixed documentation and docstring comments
*Added a check to make sure code coverage is
not enabled on a DTS run with a precompiled
build directory
v5:
*Fixed issue where teardown fails when code
coverage report generation fails
v6:
*Fixed error reporting when coverage report fails to generate
and resolved error where coverage report wouldnt generate due
to insuffucuent priviledges
---
.mailmap | 1 +
doc/guides/tools/dts.rst | 18 ++++++++++
dts/README.md | 5 +++
dts/framework/remote_session/dpdk.py | 36 +++++++++++++++++++
.../remote_session/remote_session.py | 5 ++-
dts/framework/settings.py | 10 ++++++
dts/framework/testbed_model/os_session.py | 10 ++++++
dts/framework/testbed_model/posix_session.py | 24 +++++++++++++
dts/framework/utils.py | 8 +++++
9 files changed, 116 insertions(+), 1 deletion(-)
diff --git a/.mailmap b/.mailmap
index 05a55c0bd6..ed1553c6c3 100644
--- a/.mailmap
+++ b/.mailmap
@@ -887,6 +887,7 @@ Klaus Degner <kd@allegro-packets.com>
Kommula Shiva Shankar <kshankar@marvell.com>
Konstantin Ananyev <konstantin.ananyev@huawei.com> <konstantin.v.ananyev@yandex.ru>
Konstantin Ananyev <konstantin.ananyev@huawei.com> <konstantin.ananyev@intel.com>
+Koushik Bhargav Nimoji <knimoji@iol.unh.edu>
Krishna Murthy <krishna.j.murthy@intel.com>
Krzysztof Galazka <krzysztof.galazka@intel.com>
Krzysztof Kanas <kkanas@marvell.com> <krzysztof.kanas@caviumnetworks.com>
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 5b9a348016..0ffebdc713 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -352,6 +352,10 @@ DTS is run with ``main.py`` located in the ``dts`` directory using the ``poetry
--precompiled-build-dir DIR_NAME
[DTS_PRECOMPILED_BUILD_DIR] Define the subdirectory under the DPDK tree root directory or tarball where the pre-
compiled binaries are located. (default: None)
+ --code-coverage Builds DPDK on the SUT node with code coverage enabled. Generates a code coverage report which can be found on
+ the DTS execution hosts local filesystem at dts/output/coverage_reports/meson-logs/coveragereport/index.html,
+ or the specified output directory. To use code coverage, please ensure lcov v1.15 and gcov v8.0 or higher
+ (included in gcc package) are installed on the SUT node.
The brackets contain the names of environment variables that set the same thing.
@@ -367,6 +371,20 @@ Results are stored in the output dir by default
which be changed with the ``--output-dir`` command line argument.
The results contain basic statistics of passed/failed test cases and DPDK version.
+Code Coverage
+~~~~~~~~~~~~~
+
+DTS has the ablilty to track code usage during test runs, and generate an HTML
+coverage report which shows the coverage percentage for the various DPDK
+libraries and drivers utilized during execution. The DPDK build directory must
+be compiled on the SUT node, as a pre-built build directory may not be properly
+configured for code coverage. Code coverage can be enabled by using the
+"--code-coverage" CLI parameter when running DTS.
+
+To use code coverage, please make sure the following dependencies are available
+on the SUT node:
+- lcov v1.15 or greater
+- gcov v8.0 or greater (included in gcc package)
Contributing to DTS
-------------------
diff --git a/dts/README.md b/dts/README.md
index d257b7a167..51f824e077 100644
--- a/dts/README.md
+++ b/dts/README.md
@@ -64,6 +64,11 @@ $ poetry run ./main.py
These commands will give you a bash shell inside a docker container
with all DTS Python dependencies installed.
+# Code Coverage
+
+To generate code coverage reports, ensure the SUT has lcov v1.15 and gcov v8.0 or greater
+installed, and that DTS is run using the '--code-coverage' argument.
+
## Visual Studio Code
Usage of VScode devcontainers is NOT required for developing on DTS and running DTS,
diff --git a/dts/framework/remote_session/dpdk.py b/dts/framework/remote_session/dpdk.py
index e43e1f2123..683bc3470e 100644
--- a/dts/framework/remote_session/dpdk.py
+++ b/dts/framework/remote_session/dpdk.py
@@ -30,6 +30,7 @@
from framework.logger import DTSLogger, get_dts_logger
from framework.params.eal import EalParams
from framework.remote_session.remote_session import CommandResult
+from framework.settings import SETTINGS
from framework.testbed_model.cpu import LogicalCore, LogicalCoreCount, LogicalCoreList, lcore_filter
from framework.testbed_model.node import Node
from framework.testbed_model.os_session import OSSession
@@ -81,6 +82,10 @@ def setup(self) -> None:
DPDK setup includes setting all internals needed for the build, the copying of DPDK
sources and then building DPDK or using the exist ones from the `dpdk_location`. The drivers
are bound to those that DPDK needs.
+
+ Raises:
+ ConfigurationError: When DTS is run with code coverage enabled, but is also provided
+ a precompiled build directory.
"""
if not isinstance(self.config.dpdk_location, RemoteDPDKTreeLocation):
self._node.main_session.create_directory(self.remote_dpdk_tree_path)
@@ -99,6 +104,10 @@ def setup(self) -> None:
match self.config:
case DPDKPrecompiledBuildConfiguration(precompiled_build_dir=build_dir):
+ if SETTINGS.code_coverage:
+ raise ConfigurationError(
+ "Cannot create code coverage report using a precompiled build directory."
+ )
self._set_remote_dpdk_build_dir(build_dir)
case DPDKUncompiledBuildConfiguration(build_options=build_options):
self._configure_dpdk_build(build_options)
@@ -108,7 +117,31 @@ def teardown(self) -> None:
"""Teardown the DPDK build on the target node.
Removes the DPDK tree and/or build directory/tarball depending on the configuration.
+ If code coverage is enabled, the coverage report and .info file are generated and
+ copied onto the local filesystem before teardown.
"""
+ try:
+ if SETTINGS.code_coverage:
+ report_folder = PurePath(self.remote_dpdk_build_dir / "meson-logs")
+ output_dir = SETTINGS.output_dir
+ Path(output_dir).mkdir(parents=True, exist_ok=True)
+
+ coverage_status = self._session.generate_coverage_report(self.remote_dpdk_build_dir)
+ if coverage_status:
+ self._session.copy_dir_from(report_folder, output_dir)
+ self._logger.info(
+ "Coverage HTML report generated, "
+ f"available at {output_dir}/meson-logs/coveragereport/index.html"
+ )
+ self._session.send_command(
+ f"rm -r {self.remote_dpdk_build_dir}/meson-logs/", privileged=True
+ )
+ else:
+ self._logger.info("Failed to generate code coverage report")
+
+ except Exception as e:
+ self._logger.info(f"Unable to create code coverage report due to an error: {e}")
+
match self.config.dpdk_location:
case LocalDPDKTreeLocation():
self._node.main_session.remove_remote_dir(self.remote_dpdk_tree_path)
@@ -274,6 +307,9 @@ def _build_dpdk(self) -> None:
else:
meson_args = MesonArgs(default_library="static", libdir="lib")
+ if SETTINGS.code_coverage:
+ meson_args._add_arg("-Db_coverage=true")
+
self._session.build_dpdk(
self._env_vars,
meson_args,
diff --git a/dts/framework/remote_session/remote_session.py b/dts/framework/remote_session/remote_session.py
index fb5f6fedf5..cc1f1f6a4f 100644
--- a/dts/framework/remote_session/remote_session.py
+++ b/dts/framework/remote_session/remote_session.py
@@ -250,7 +250,10 @@ def copy_from(self, source_file: str | PurePath, destination_dir: str | Path) ->
destination_dir: The directory path on the local filesystem where the `source_file`
will be saved.
"""
- self.session.get(str(source_file), str(destination_dir))
+ source_file = PurePath(source_file)
+ destination_dir = Path(destination_dir)
+ local_path = destination_dir / source_file.name
+ self.session.get(str(source_file), str(local_path))
def copy_to(self, source_file: str | Path, destination_dir: str | PurePath) -> None:
"""Copy a file from local filesystem to the remote Node.
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index f329677804..6e2fc78b78 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -159,6 +159,8 @@ class Settings:
re_run: int = 0
#:
random_seed: int | None = None
+ #:
+ code_coverage: bool = False
SETTINGS: Settings = Settings()
@@ -489,6 +491,14 @@ def _get_parser() -> _DTSArgumentParser:
)
_add_env_var_to_action(action)
+ action = parser.add_argument(
+ "--code-coverage",
+ action="store_true",
+ default=False,
+ help="Used to build DPDK with code coverage enabled.",
+ )
+ _add_env_var_to_action(action)
+
return parser
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index f88427a53d..4551c7fffe 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -480,6 +480,16 @@ def build_dpdk(
timeout: Wait at most this long in seconds for the build execution to complete.
"""
+ @abstractmethod
+ def generate_coverage_report(self, remote_build_dir: PurePath | None) -> bool:
+ """Generates a code coverage report for a DTS run.
+
+ Args:
+ remote_build_dir: The remote DPDK build directory
+ Returns:
+ Whether the coverage report was able to be created or not.
+ """
+
@abstractmethod
def get_dpdk_version(self, version_path: str | PurePath) -> str:
"""Inspect the DPDK version on the remote node.
diff --git a/dts/framework/testbed_model/posix_session.py b/dts/framework/testbed_model/posix_session.py
index dec952685a..e7b3e78333 100644
--- a/dts/framework/testbed_model/posix_session.py
+++ b/dts/framework/testbed_model/posix_session.py
@@ -295,6 +295,30 @@ def build_dpdk(
except RemoteCommandExecutionError as e:
raise DPDKBuildError(f"DPDK build failed when doing '{e.command}'.")
+ def generate_coverage_report(self, remote_build_dir: PurePath | None) -> bool:
+ """Overrides :meth:`~.os_session.OSSession.generate_coverage_report`."""
+ command_result = self.send_command(r"lcov --version | grep -oP '\d+\.\d+'")
+ lcov_version = float(
+ command_result.stdout if command_result.return_code == 0 and command_result else -1
+ )
+ command_result = self.send_command(
+ r"gcov --version | head -n 1 | grep -oP '\d+\.\d+' | tail -n 1"
+ )
+ gcov_version = float(
+ command_result.stdout if command_result.return_code == 0 and command_result else -1
+ )
+
+ if lcov_version < 1.15 or gcov_version < 8.0:
+ self._logger.info(
+ "lcov/gcov version mismatch, please ensure at least lcov v1.15 and gcov v8.0"
+ )
+ return False
+
+ coverage_command = self.send_command(
+ f"ninja -C {remote_build_dir} coverage-html", timeout=600, privileged=True
+ )
+ return coverage_command.return_code == 0
+
def get_dpdk_version(self, build_dir: str | PurePath) -> str:
"""Overrides :meth:`~.os_session.OSSession.get_dpdk_version`."""
out = self.send_command(f"cat {self.join_remote_path(build_dir, 'VERSION')}", verify=True)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 5753c1b7fe..55d6dfc337 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -127,6 +127,14 @@ def __str__(self) -> str:
"""The actual args."""
return " ".join(f"{self._default_library} {self._dpdk_args}".split())
+ def _add_arg(self, arg: str):
+ """Adds an argument to the meson setup command.
+
+ Args:
+ arg: The meson build argument to be added.
+ """
+ self._dpdk_args = self._dpdk_args + " " + arg
+
class TarCompressionFormat(StrEnum):
"""Compression formats that tar can use.
--
2.54.0
^ permalink raw reply related
* Re: [PATCH] examples/flow_filtering: fix null derefs and memory leaks
From: Thomas Monjalon @ 2026-07-10 22:22 UTC (permalink / raw)
To: Shani Peretz; +Cc: dev, Ori Kam
In-Reply-To: <20260331125347.346362-1-shperetz@nvidia.com>
31/03/2026 14:53, Shani Peretz:
> Add missing return statements after failed calloc NULL checks to
> prevent null pointer dereferences.
>
> Free previously allocated memory on error paths to prevent
> leaked storage when a subsequent allocation fails.
>
> Fixes: 3ebb8789136a ("examples/flow_filtering: add more snippets")
>
> Signed-off-by: Shani Peretz <shperetz@nvidia.com>
Applied, thanks.
^ permalink raw reply
* [PATCH v2 2/2] net/af_xdp: add read_clock support to AF_XDP PMD
From: Mark Blasko @ 2026-07-10 22:10 UTC (permalink / raw)
To: dev, Ciara Loftus, Maryam Tahhan; +Cc: joshwash, jtranoleary, blasko
In-Reply-To: <20260710221013.3913447-1-blasko@google.com>
Implement the ethdev read_clock operation in the AF_XDP Poll Mode
Driver. This allows DPDK applications to query the current time of the
NIC hardware clock.
At device start, the PMD queries ethtool for the interface's PTP
Hardware Clock index and opens the corresponding PTP device node. The
read_clock operation queries the current time via clock_gettime.
Signed-off-by: Mark Blasko <blasko@google.com>
Reviewed-by: Joshua Washington <joshwash@google.com>
---
v2:
- New patch introduced in v2 to support read_clock ethdev operation.
---
drivers/net/af_xdp/rte_eth_af_xdp.c | 127 ++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 2f9ad7d180..410be570a0 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -15,6 +15,7 @@
#include <linux/if_link.h>
#include <linux/ethtool.h>
#include <linux/sockios.h>
+#include <linux/net_tstamp.h>
#include "af_xdp_deps.h"
#include <rte_ethdev.h>
@@ -28,6 +29,7 @@
#include <dev_driver.h>
#include <rte_eal.h>
#include <rte_ether.h>
+#include <rte_time.h>
#include <rte_lcore.h>
#include <rte_log.h>
#include <rte_memory.h>
@@ -62,6 +64,9 @@
#define PF_XDP AF_XDP
#endif
+#include <fcntl.h>
+#include <time.h>
+
static int timestamp_dynfield_offset = -1;
static uint64_t timestamp_dynflag;
@@ -194,6 +199,7 @@ struct pmd_internals {
int rx_timestamp_offset;
int rx_timestamp_valid_offset;
uint8_t rx_timestamp_valid_mask;
+ int ptp_fd;
};
struct pmd_process_private {
@@ -795,9 +801,72 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
#endif
}
+static int
+eth_af_xdp_enable_hw_timestamping(const char *if_name)
+{
+ struct hwtstamp_config config = {0};
+ struct ifreq ifr = {0};
+ int fd, ret;
+
+ fd = socket(AF_INET, SOCK_DGRAM, 0);
+ if (fd < 0)
+ return -1;
+
+ ifr.ifr_data = (caddr_t)&config;
+ strlcpy(ifr.ifr_name, if_name, IFNAMSIZ);
+
+ ret = ioctl(fd, SIOCGHWTSTAMP, &ifr);
+ if (ret == 0) {
+ if (config.rx_filter != HWTSTAMP_FILTER_NONE) {
+ close(fd);
+ return 0;
+ }
+ }
+
+ config.flags = 0;
+ config.tx_type = HWTSTAMP_TX_OFF;
+ config.rx_filter = HWTSTAMP_FILTER_ALL;
+
+ ret = ioctl(fd, SIOCSHWTSTAMP, &ifr);
+ close(fd);
+
+ if (ret < 0)
+ return -errno;
+
+ if (config.rx_filter == HWTSTAMP_FILTER_NONE)
+ return -ENOTSUP;
+
+ return 0;
+}
+
+static int
+eth_af_xdp_get_ptp_index(const char *if_name)
+{
+ struct ethtool_ts_info info = {0};
+ struct ifreq ifr = {0};
+ int fd, ret;
+
+ fd = socket(AF_INET, SOCK_DGRAM, 0);
+ if (fd < 0)
+ return -1;
+
+ ifr.ifr_data = (caddr_t)&info;
+ info.cmd = ETHTOOL_GET_TS_INFO;
+ strlcpy(ifr.ifr_name, if_name, IFNAMSIZ);
+
+ ret = ioctl(fd, SIOCETHTOOL, &ifr);
+ close(fd);
+
+ if (ret < 0)
+ return -1;
+
+ return info.phc_index;
+}
+
static int
eth_dev_start(struct rte_eth_dev *dev)
{
+ struct pmd_internals *internals = dev->data->dev_private;
uint16_t i;
if (dev->data->dev_conf.rxmode.offloads &
@@ -814,6 +883,25 @@ eth_dev_start(struct rte_eth_dev *dev)
AF_XDP_LOG_LINE(INFO,
"Registered mbuf timestamp field, offset: %d",
timestamp_dynfield_offset);
+
+ rc = eth_af_xdp_enable_hw_timestamping(internals->if_name);
+ if (rc < 0) {
+ AF_XDP_LOG_LINE(WARNING,
+ "Could not enable HW timestamping on %s: %s",
+ internals->if_name, strerror(-rc));
+ }
+
+ int phc_index = eth_af_xdp_get_ptp_index(internals->if_name);
+ if (phc_index >= 0) {
+ char ptp_dev[32];
+ snprintf(ptp_dev, sizeof(ptp_dev), "/dev/ptp%d", phc_index);
+ internals->ptp_fd = open(ptp_dev, O_RDONLY);
+ if (internals->ptp_fd >= 0) {
+ AF_XDP_LOG_LINE(INFO,
+ "Opened PTP device %s for read_clock",
+ ptp_dev);
+ }
+ }
}
dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
@@ -829,6 +917,7 @@ eth_dev_start(struct rte_eth_dev *dev)
static int
eth_dev_stop(struct rte_eth_dev *dev)
{
+ struct pmd_internals *internals = dev->data->dev_private;
uint16_t i;
dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
@@ -837,6 +926,11 @@ eth_dev_stop(struct rte_eth_dev *dev)
dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
}
+ if (internals->ptp_fd >= 0) {
+ close(internals->ptp_fd);
+ internals->ptp_fd = -1;
+ }
+
return 0;
}
@@ -1170,6 +1264,11 @@ eth_dev_close(struct rte_eth_dev *dev)
}
}
+ if (internals->ptp_fd >= 0) {
+ close(internals->ptp_fd);
+ internals->ptp_fd = -1;
+ }
+
out:
rte_free(dev->process_private);
@@ -2039,6 +2138,31 @@ eth_dev_promiscuous_disable(struct rte_eth_dev *dev)
return eth_dev_change_flags(internals->if_name, 0, ~IFF_PROMISC);
}
+/*
+ * In Linux, dynamic POSIX clock IDs from file descriptors (such as /dev/ptpX)
+ * are encoded with CLOCKFD (3) in the lower 3 bits and ~fd in the upper bits.
+ * As this is not defined in user-space UAPI headers, define the macro here.
+ */
+#define CLOCKFD 3
+#define FD_TO_CLOCKID(fd) ((clockid_t)(~(unsigned int)(fd) << 3 | CLOCKFD))
+
+static int
+eth_af_xdp_read_clock(struct rte_eth_dev *dev, uint64_t *timestamp)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+ struct timespec ts;
+
+ if (internals->ptp_fd < 0)
+ return -ENOTSUP;
+
+ clockid_t clkid = FD_TO_CLOCKID(internals->ptp_fd);
+ if (clock_gettime(clkid, &ts) < 0)
+ return -1;
+
+ *timestamp = rte_timespec_to_ns(&ts);
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -2054,6 +2178,7 @@ static const struct eth_dev_ops ops = {
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
.get_monitor_addr = eth_get_monitor_addr,
+ .read_clock = eth_af_xdp_read_clock,
};
/* AF_XDP Device Plugin option works in unprivileged
@@ -2075,6 +2200,7 @@ static const struct eth_dev_ops ops_afxdp_dp = {
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
.get_monitor_addr = eth_get_monitor_addr,
+ .read_clock = eth_af_xdp_read_clock,
};
/** parse busy_budget argument */
@@ -2399,6 +2525,7 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
internals->rx_timestamp_offset = rx_timestamp_offset;
internals->rx_timestamp_valid_offset = rx_timestamp_valid_offset;
internals->rx_timestamp_valid_mask = (uint8_t)rx_timestamp_valid_mask;
+ internals->ptp_fd = -1;
if (xdp_get_channels_info(if_name, &internals->max_queue_cnt,
&internals->configured_queue_cnt)) {
--
2.55.0.141.g00534a21ce-goog
^ permalink raw reply related
* [PATCH v2 1/2] net/af_xdp: add af_xdp rx metadata and dynamic timestamping support
From: Mark Blasko @ 2026-07-10 22:10 UTC (permalink / raw)
To: dev, Ciara Loftus, Maryam Tahhan; +Cc: joshwash, jtranoleary, blasko
In-Reply-To: <20260710221013.3913447-1-blasko@google.com>
Enable dynamic RX timestamping in the AF_XDP Poll Mode Driver using
layout-agnostic metadata offsets and validity verification.
This introduces three new vdev devargs to describe the metadata layout:
1. xdp_meta_rx_ts_offset: byte offset of the RX timestamp in metadata.
2. xdp_meta_valid_hint_offset: byte offset of the validity flag byte.
3. xdp_meta_rx_ts_valid_mask: bitmask to verify timestamp validity.
If validation parameters are configured, the PMD verifies the validity
bit at the target offset on a per-packet basis before copying the
timestamp to the standard EAL dynamic field. Otherwise, the timestamp
is assumed valid if metadata space is available.
Signed-off-by: Mark Blasko <blasko@google.com>
Reviewed-by: Joshua Washington <joshwash@google.com>
---
v2:
- Replace static metadata struct assumption with layout-agnostic devargs
(xdp_meta_rx_ts_offset, xdp_meta_valid_hint_offset, and
xdp_meta_rx_ts_valid_mask).
- Support per-packet timestamp validity verification via metadata hint flag.
- Reject PMD probe if validity mask is zero when validity offset is set.
---
drivers/net/af_xdp/rte_eth_af_xdp.c | 139 +++++++++++++++++++++++++++-
1 file changed, 134 insertions(+), 5 deletions(-)
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 2cdb533276..2f9ad7d180 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -62,6 +62,10 @@
#define PF_XDP AF_XDP
#endif
+
+static int timestamp_dynfield_offset = -1;
+static uint64_t timestamp_dynflag;
+
RTE_LOG_REGISTER_DEFAULT(af_xdp_logtype, NOTICE);
#define RTE_LOGTYPE_NET_AF_XDP af_xdp_logtype
@@ -144,6 +148,10 @@ struct pkt_rx_queue {
struct pollfd fds[1];
int xsk_queue_idx;
int busy_budget;
+ bool rx_timestamp_enabled;
+ int rx_timestamp_offset;
+ int rx_timestamp_valid_offset;
+ uint8_t rx_timestamp_valid_mask;
};
struct tx_stats {
@@ -183,6 +191,9 @@ struct pmd_internals {
struct pkt_rx_queue *rx_queues;
struct pkt_tx_queue *tx_queues;
+ int rx_timestamp_offset;
+ int rx_timestamp_valid_offset;
+ uint8_t rx_timestamp_valid_mask;
};
struct pmd_process_private {
@@ -200,6 +211,9 @@ struct pmd_process_private {
#define ETH_AF_XDP_USE_PINNED_MAP_ARG "use_pinned_map"
#define ETH_AF_XDP_DP_PATH_ARG "dp_path"
#define ETH_AF_XDP_MODE_ARG "mode"
+#define ETH_AF_XDP_RX_TIMESTAMP_OFFSET_ARG "xdp_meta_rx_ts_offset"
+#define ETH_AF_XDP_RX_TIMESTAMP_VALID_OFFSET_ARG "xdp_meta_valid_hint_offset"
+#define ETH_AF_XDP_RX_TIMESTAMP_VALID_MASK_ARG "xdp_meta_rx_ts_valid_mask"
/* Define different modes for af_xdp prog to attach */
#define ETH_AF_XDP_DRV_MODE_ARG "drv"
@@ -232,6 +246,9 @@ static const char * const valid_arguments[] = {
ETH_AF_XDP_USE_PINNED_MAP_ARG,
ETH_AF_XDP_DP_PATH_ARG,
ETH_AF_XDP_MODE_ARG,
+ ETH_AF_XDP_RX_TIMESTAMP_OFFSET_ARG,
+ ETH_AF_XDP_RX_TIMESTAMP_VALID_OFFSET_ARG,
+ ETH_AF_XDP_RX_TIMESTAMP_VALID_MASK_ARG,
NULL
};
@@ -398,6 +415,27 @@ af_xdp_rx_zc(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
rte_pktmbuf_pkt_len(bufs[i]) = len;
rte_pktmbuf_data_len(bufs[i]) = len;
+
+ if (rxq->rx_timestamp_enabled &&
+ timestamp_dynfield_offset >= 0) {
+ /*
+ * Copy timestamp if validity offset is not defined or
+ * flag is valid.
+ */
+ if (rxq->rx_timestamp_valid_offset < 0 ||
+ (*(uint8_t *)((char *)rte_pktmbuf_mtod(bufs[i], void *) -
+ rxq->rx_timestamp_valid_offset) &
+ rxq->rx_timestamp_valid_mask)) {
+ uint64_t *ts_ptr = (uint64_t *)
+ ((char *)rte_pktmbuf_mtod(bufs[i], void *) -
+ rxq->rx_timestamp_offset);
+ *RTE_MBUF_DYNFIELD(bufs[i],
+ timestamp_dynfield_offset,
+ uint64_t *) = *ts_ptr;
+ bufs[i]->ol_flags |= timestamp_dynflag;
+ }
+ }
+
rx_bytes += len;
}
@@ -457,6 +495,25 @@ af_xdp_rx_cp(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
len = desc->len;
pkt = xsk_umem__get_data(rxq->umem->mz->addr, addr);
+ if (rxq->rx_timestamp_enabled &&
+ timestamp_dynfield_offset >= 0) {
+ /*
+ * Copy timestamp if validity offset is not defined or
+ * flag is valid.
+ */
+ if (rxq->rx_timestamp_valid_offset < 0 ||
+ (*(uint8_t *)((char *)pkt -
+ rxq->rx_timestamp_valid_offset) &
+ rxq->rx_timestamp_valid_mask)) {
+ uint64_t *ts_ptr = (uint64_t *)((char *)pkt -
+ rxq->rx_timestamp_offset);
+ *RTE_MBUF_DYNFIELD(mbufs[i],
+ timestamp_dynfield_offset,
+ uint64_t *) = *ts_ptr;
+ mbufs[i]->ol_flags |= timestamp_dynflag;
+ }
+ }
+
rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len);
rte_ring_enqueue(umem->buf_ring, (void *)addr);
rte_pktmbuf_pkt_len(mbufs[i]) = len;
@@ -743,6 +800,22 @@ eth_dev_start(struct rte_eth_dev *dev)
{
uint16_t i;
+ if (dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ int rc;
+
+ rc = rte_mbuf_dyn_rx_timestamp_register(×tamp_dynfield_offset,
+ ×tamp_dynflag);
+ if (rc) {
+ AF_XDP_LOG_LINE(ERR,
+ "Failed to register mbuf timestamp field");
+ return rc;
+ }
+ AF_XDP_LOG_LINE(INFO,
+ "Registered mbuf timestamp field, offset: %d",
+ timestamp_dynfield_offset);
+ }
+
dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
for (i = 0; i < dev->data->nb_rx_queues; i++) {
dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -870,6 +943,8 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_rx_queues = internals->queue_cnt;
dev_info->max_tx_queues = internals->queue_cnt;
+ dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_TIMESTAMP;
+
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
#if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
dev_info->max_rx_pktlen = getpagesize() -
@@ -1873,7 +1948,12 @@ eth_rx_queue_setup(struct rte_eth_dev *dev,
process_private->rxq_xsk_fds[rx_queue_id] = rxq->fds[0].fd;
rxq->port = dev->data->port_id;
-
+ rxq->rx_timestamp_offset = internals->rx_timestamp_offset;
+ rxq->rx_timestamp_valid_offset = internals->rx_timestamp_valid_offset;
+ rxq->rx_timestamp_valid_mask = internals->rx_timestamp_valid_mask;
+ rxq->rx_timestamp_enabled = (rxq->rx_timestamp_offset >= 0) &&
+ !!(dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP);
dev->data->rx_queues[rx_queue_id] = rxq;
return 0;
@@ -2032,6 +2112,22 @@ parse_integer_arg(const char *key __rte_unused,
return 0;
}
+static int
+parse_mask_arg(const char *key __rte_unused,
+ const char *value, void *extra_args)
+{
+ int *i = (int *)extra_args;
+ char *end;
+
+ *i = strtol(value, &end, 0);
+ if (*i < 0) {
+ AF_XDP_LOG_LINE(ERR, "Argument has to be positive.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
/** parse name argument */
static int
parse_name_arg(const char *key __rte_unused,
@@ -2146,7 +2242,9 @@ static int
parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue,
int *queue_cnt, int *shared_umem, char *prog_path,
int *busy_budget, int *force_copy, int *use_cni,
- int *use_pinned_map, char *dp_path, uint32_t *xdp_mode)
+ int *use_pinned_map, char *dp_path, uint32_t *xdp_mode,
+ int *rx_timestamp_offset, int *rx_timestamp_valid_offset,
+ int *rx_timestamp_valid_mask)
{
int ret;
@@ -2209,6 +2307,21 @@ parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue,
if (ret < 0)
goto free_kvlist;
+ ret = rte_kvargs_process(kvlist, ETH_AF_XDP_RX_TIMESTAMP_OFFSET_ARG,
+ &parse_integer_arg, rx_timestamp_offset);
+ if (ret < 0)
+ goto free_kvlist;
+
+ ret = rte_kvargs_process(kvlist, ETH_AF_XDP_RX_TIMESTAMP_VALID_OFFSET_ARG,
+ &parse_integer_arg, rx_timestamp_valid_offset);
+ if (ret < 0)
+ goto free_kvlist;
+
+ ret = rte_kvargs_process(kvlist, ETH_AF_XDP_RX_TIMESTAMP_VALID_MASK_ARG,
+ &parse_mask_arg, rx_timestamp_valid_mask);
+ if (ret < 0)
+ goto free_kvlist;
+
free_kvlist:
rte_kvargs_free(kvlist);
return ret;
@@ -2248,7 +2361,9 @@ static struct rte_eth_dev *
init_internals(struct rte_vdev_device *dev, const char *if_name,
int start_queue_idx, int queue_cnt, int shared_umem,
const char *prog_path, int busy_budget, int force_copy,
- int use_cni, int use_pinned_map, const char *dp_path, uint32_t xdp_mode)
+ int use_cni, int use_pinned_map, const char *dp_path, uint32_t xdp_mode,
+ int rx_timestamp_offset, int rx_timestamp_valid_offset,
+ int rx_timestamp_valid_mask)
{
const char *name = rte_vdev_device_name(dev);
const unsigned int numa_node = dev->device.numa_node;
@@ -2281,6 +2396,9 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
internals->use_pinned_map = use_pinned_map;
internals->mode_flag = XDP_FLAGS_UPDATE_IF_NOEXIST | xdp_mode;
strlcpy(internals->dp_path, dp_path, PATH_MAX);
+ internals->rx_timestamp_offset = rx_timestamp_offset;
+ internals->rx_timestamp_valid_offset = rx_timestamp_valid_offset;
+ internals->rx_timestamp_valid_mask = (uint8_t)rx_timestamp_valid_mask;
if (xdp_get_channels_info(if_name, &internals->max_queue_cnt,
&internals->configured_queue_cnt)) {
@@ -2474,6 +2592,9 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
int use_pinned_map = 0;
uint32_t xdp_mode = 0;
char dp_path[PATH_MAX] = {'\0'};
+ int rx_timestamp_offset = -1;
+ int rx_timestamp_valid_offset = -1;
+ int rx_timestamp_valid_mask = 0;
struct rte_eth_dev *eth_dev = NULL;
const char *name = rte_vdev_device_name(dev);
@@ -2517,11 +2638,18 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx,
&xsk_queue_cnt, &shared_umem, prog_path,
&busy_budget, &force_copy, &use_cni, &use_pinned_map,
- dp_path, &xdp_mode) < 0) {
+ dp_path, &xdp_mode, &rx_timestamp_offset,
+ &rx_timestamp_valid_offset, &rx_timestamp_valid_mask) < 0) {
AF_XDP_LOG_LINE(ERR, "Invalid kvargs value");
return -EINVAL;
}
+ if (rx_timestamp_valid_offset >= 0 && rx_timestamp_valid_mask == 0) {
+ AF_XDP_LOG_LINE(ERR,
+ "Validity mask cannot be zero when validity offset is configured");
+ return -EINVAL;
+ }
+
if (use_cni && use_pinned_map) {
AF_XDP_LOG_LINE(ERR, "When '%s' parameter is used, '%s' parameter is not valid",
ETH_AF_XDP_USE_CNI_ARG, ETH_AF_XDP_USE_PINNED_MAP_ARG);
@@ -2585,7 +2713,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
eth_dev = init_internals(dev, if_name, xsk_start_queue_idx,
xsk_queue_cnt, shared_umem, prog_path,
busy_budget, force_copy, use_cni, use_pinned_map,
- dp_path, xdp_mode);
+ dp_path, xdp_mode, rx_timestamp_offset,
+ rx_timestamp_valid_offset, rx_timestamp_valid_mask);
if (eth_dev == NULL) {
AF_XDP_LOG_LINE(ERR, "Failed to init internals");
return -1;
--
2.55.0.141.g00534a21ce-goog
^ permalink raw reply related
* [PATCH v2 0/2] net/af_xdp: add Rx timestamping and read_clock support
From: Mark Blasko @ 2026-07-10 22:10 UTC (permalink / raw)
To: dev; +Cc: joshwash, jtranoleary, blasko
In-Reply-To: <20260623215325.814776-1-blasko@google.com>
This patch series introduces support for dynamic RX timestamping and
clock querying in the AF_XDP Poll Mode Driver.
The first patch introduces three new vdev devargs to specify
layout-agnostic metadata offsets and bitmasks for extracting hardware
RX timestamps from XDP metadata into the mbuf dynamic timestamp field.
The second patch implements the read_clock ethdev operation, querying
ethtool for the interface's PTP Hardware Clock index at start and using
clock_gettime to query the NIC hardware clock time.
---
v2:
- Patch 1:
- Replace static metadata struct assumption with configurable vdev devargs
for layout-agnostic timestamp offset extraction and validity verification.
- Patch 2:
- New patch introduced in v2 to support read_clock ethdev operation.
---
Mark Blasko (2):
net/af_xdp: add af_xdp rx metadata and dynamic timestamping support
net/af_xdp: add read_clock support to AF_XDP PMD
drivers/net/af_xdp/rte_eth_af_xdp.c | 266 +++++++++++++++++++++++++++-
1 file changed, 261 insertions(+), 5 deletions(-)
--
2.55.0.141.g00534a21ce-goog
^ permalink raw reply
* Re: [PATCH] app/testpmd: fix flow cfg double-free after reattach
From: Thomas Monjalon @ 2026-07-10 21:45 UTC (permalink / raw)
To: Shani Peretz, Maayan Kashani
Cc: dev, stable, rasland, Dariusz Sosnowski, Aman Singh
In-Reply-To: <20260706115248.188551-1-mkashani@nvidia.com>
06/07/2026 13:52, Maayan Kashani:
> From: Shani Peretz <shperetz@nvidia.com>
>
> When a port is closed via close_port(), port_free_job_list() frees the
> job_list but does not NULL the pointer or reset queue_nb. If the port
> is later reattached and "flow configure" is issued, port_flow_configure()
> frees the dangling job_list pointer, causing a double-free.
>
> Fix this by setting job_list to NULL and resetting queue_nb to zero in
> port_free_job_list() after freeing.
>
> Fixes: 7fadc8039588 ("app/testpmd: fix memory leak in port flow configure")
> Cc: stable@dpdk.org
>
> Signed-off-by: Shani Peretz <shperetz@nvidia.com>
> Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH v4 0/9] Stability fixes for GVE
From: Thomas Monjalon @ 2026-07-10 21:34 UTC (permalink / raw)
To: Joshua Washington; +Cc: dev
In-Reply-To: <20260709040739.3184955-1-joshwash@google.com>
> Joshua Washington (9):
> net/gve: clear out shared memory region for stats report
> net/gve: delay adding mbuf head to software ring
> net/gve: copy data to QPL buffer when mbuf read does not
> net/gve: validate buf ID before processing Rx packet
> net/gve: set mbuf to null in software ring after use
> net/gve: free ctx mbuf if packet dropped after first segment
> net/gve: increase range of DMA memzone ids to 64 bits
> net/gve: don't reset ring size bounds to default on reset
> net/gve: restrict max ring size in GQ QPL to 2K
Applied, thanks.
^ permalink raw reply
* [PATCH v4 6/6] net/dpaa2: pin Rx queue interrupt to the polling core
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
A CDAN wakes the worker through the portal's VFIO eventfd, which is
signalled regardless of the core that took the MSI, so the wake is
correct either way. By default the MSI affinity is spread across the
housekeeping cores, so the wake crosses cores and adds a little latency.
Pin each DPIO's MSI to the lcore that arms it (as the event driver does)
so the wake lands on the worker's own core. This is a latency
optimisation, not a correctness fix.
Split dpaa2_affine_dpio_intr_to_respective_core out of the
RTE_EVENT_DPAA2 guard, expose dpaa2_dpio_affine_intr_to_core, and call it
from the rx-queue interrupt arm path. The pin is best-effort: it writes
/proc/irq/<n>/smp_affinity (needs privilege) and irqbalance may move it
back.
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 17 ++++++++++++++---
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 4 ++++
drivers/net/dpaa2/dpaa2_ethdev.c | 4 ++++
3 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
index 9dad78eba4..e17050b625 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
@@ -149,7 +149,6 @@ dpaa2_core_cluster_sdest(int cpu_id)
return dpaa2_core_cluster_base + x;
}
-#ifdef RTE_EVENT_DPAA2
static void
dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)
{
@@ -204,7 +203,19 @@ dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)
fclose(file);
}
-#endif /* RTE_EVENT_DPAA2 */
+
+/* Pin the dpio's MSI IRQ to the calling lcore's core (best-effort), so a
+ * worker sleeping on this portal's eventfd is woken on its own core.
+ */
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dpio_affine_intr_to_core)
+void
+dpaa2_dpio_affine_intr_to_core(int32_t dpio_id)
+{
+ int cpu_id = dpaa2_get_core_id();
+
+ if (cpu_id >= 0)
+ dpaa2_affine_dpio_intr_to_respective_core(dpio_id, cpu_id);
+}
/* threshold: DQRR fill raising DQRI (< ring depth); timeout: holdoff in ITP units.
* Per-mode values from the caller (eventdev vs rx-queue intr).
@@ -324,8 +335,8 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id, bool ethrx
DPAA2_BUS_ERR("Interrupt registration failed for dpio");
return -1;
}
+ dpaa2_affine_dpio_intr_to_respective_core(dpio_dev->hw_id, cpu_id);
}
- dpaa2_affine_dpio_intr_to_respective_core(dpio_dev->hw_id, cpu_id);
#else
RTE_SET_USED(ethrx);
#endif
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
index 1f6e521341..7e2271cc35 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
@@ -58,6 +58,10 @@ int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, int threshold,
__rte_internal
void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev);
+/* pin a DPIO's MSI IRQ to the calling lcore's core (rx-queue interrupt mode) */
+__rte_internal
+void dpaa2_dpio_affine_intr_to_core(int32_t dpio_id);
+
/* allocate memory for FQ - dq storage */
__rte_internal
int
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 9d3634d75f..53ea060a4c 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3170,6 +3170,10 @@ dpaa2_napi_subscribe(struct rte_eth_dev *dev, uint16_t queue_id,
}
rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio, dpio,
rte_memory_order_release);
+ /* pin this portal's MSI to the worker's core: a CDAN wake then lands on
+ * the same core the worker sleeps on, instead of the default spread.
+ */
+ dpaa2_dpio_affine_intr_to_core(dpio->hw_id);
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v4 5/6] net/dpaa2: support Rx queue interrupts
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
Implement .rx_queue_intr_enable / .rx_queue_intr_disable so a worker can
sleep on a queue's data-availability notification instead of busy-polling,
through the generic rte_eth_dev_rx_intr_* API.
A worker wakes on its software portal's DQRI, which fires when the
portal's DQRR is non-empty. To wake on a specific Rx FQ, the FQ is
scheduled to its own DPCON channel, and that channel is told to post a
CDAN (Channel Dequeue Available Notification) to the worker's portal when
it goes non-empty. rx_queue_setup schedules each Rx FQ to its DPCON
(DEST_DPCON) once, when the channel is first grabbed; a worker later, in
rx_queue_intr_enable, points the channel's CDAN at its own DPIO
(dpcon_set_notification), enables it on its portal and arms the DQRI.
The FQ stays scheduled to its DPCON for the life of the port and is
never re-bound: the MC refuses to move a scheduled FQ back to parked at
runtime, and re-issuing DEST_DPCON on a non-empty FQ would strand it (no
empty -> non-empty edge, so no CDAN); only the probe/close dpni_reset
un-schedules it. A Rx queue count change (dev_stop, dev_configure,
rx_queue_setup, dev_start) is therefore supported: growing binds only the
new queues, shrinking leaves the excess FQs scheduled but idle. The
interrupt handle is allocated once at dev_configure, sized to the queue
ceiling, and kept for the life of the port so the application's rx-intr
epoll registration stays valid across a reconfigure. Switching the
interrupt mode on or off is refused after the first configure (it cannot
be undone at runtime); close the port to change it.
One DPCON per FQ (drawn from the shared pool the DPCON move to the
fslmc bus feeds), so each queue keeps its own notification and can be
re-homed to another lcore without a set_queue or a port stop. Unlike the
event PMD, which concentrates N FQs onto one DPCON and lets the QBMan
scheduler load-balance across cores, affinity here is static and there is
no scheduling.
A CDAN only notifies; it does not carry frames. On wake the worker drains
each queue with a volatile dequeue on its DPCON channel
(dpaa2_dev_prefetch_rx_channel, the prefetch poll path retargeted from
the FQ to the channel) and consumes the CDAN from the DQRR. The
notification is one-shot, re-armed on the next idle->sleep.
The DQRI fires immediately on the first CDAN: the arm sets the ITP
holdoff to 0, so the interrupt is raised as soon as the DQRR goes
non-empty; with a zero holdoff the DQRR threshold is immaterial.
wire
|
[ DPMAC ]
|
[ DPNI ] (1)
|
TC0: FQ0 FQ1 FQ2 FQ3 (2)
| | | | (3)
[DPCON][DPCON][DPCON][DPCON]
\ | | / (4)
[ DPIO A ] [ DPIO B ] (5)
| |
DQRR DQRR (6)
| |
DQRI DQRI (7)
| |
eventfd eventfd (8)
| |
rte_epoll_wait rte_epoll_wait (9)
|
dpaa2_dev_prefetch_rx_channel (10)
|
frames -> store
(1) WRIOP picks a TC (QoS), then RSS-hashes within the TC to an FQ
(2) FQ0..FQ3 are the rte_eth Rx queues
(3) dpni_set_queue(DEST_DPCON): one DPCON per FQ (FQ scheduled to it)
(4) the lcore portal points each DPCON's CDAN at itself
(dpcon_set_notification) and enables it; its dedicated channel
static-dequeues the notifications
(5) one QBMan software portal per lcore
(6) a non-empty FQ makes its DPCON post a CDAN (a notification, not the
frames) into the portal DQRR
(7) DQRI is raised when the DQRR is non-empty (a CDAN is pending)
(8) a portal's queues share one fd (its DQRI eventfd)
(9) worker sleeps here when all its queues are idle
(10) on wake the worker pulls each notified queue's channel; frames land
in the per-queue store, the CDAN is consumed from the DQRR
The DQRI and eventfd are portal-wide: a queue's eventfd is its portal's
DQRI fd, and the inhibit bit is refcounted by armed queues so disabling
one queue never masks a sibling.
The PMD does not disarm from stop or close: the CDAN and DQRI operations
drive the single-owner QBMan software portal and must run on the polling
lcore, so stop or close can only warn if the application skips
rte_eth_dev_rx_intr_disable().
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Tested-by: Jun Yang <jun.yang@nxp.com>
---
doc/guides/nics/dpaa2.rst | 28 ++
doc/guides/nics/features/dpaa2.ini | 1 +
doc/guides/rel_notes/release_26_07.rst | 1 +
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 76 +++-
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 8 +
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 9 +
.../fslmc/qbman/include/fsl_qbman_portal.h | 5 +
drivers/bus/fslmc/qbman/qbman_portal.c | 5 +
drivers/net/dpaa2/dpaa2_ethdev.c | 409 +++++++++++++++++-
drivers/net/dpaa2/dpaa2_ethdev.h | 7 +
drivers/net/dpaa2/dpaa2_rxtx.c | 114 ++++-
11 files changed, 613 insertions(+), 50 deletions(-)
diff --git a/doc/guides/nics/dpaa2.rst b/doc/guides/nics/dpaa2.rst
index 2d70bd0ab9..b10b7c5c77 100644
--- a/doc/guides/nics/dpaa2.rst
+++ b/doc/guides/nics/dpaa2.rst
@@ -406,6 +406,7 @@ Features of the DPAA2 PMD are:
- Jumbo frames
- Link flow control
- Scattered and gather for TX and RX
+- Rx queue interrupts
- :ref:`Traffic Management API <dptmapi>`
@@ -553,6 +554,33 @@ is fixed and cannot be changed. So, even when the ``rxmode.mtu``
member of ``struct rte_eth_conf`` is set to a value lower than 10240, frames
up to 10240 bytes can still reach the host interface.
+Rx queue interrupts
+~~~~~~~~~~~~~~~~~~~
+
+Rx queue interrupts (``rte_eth_dev_rx_intr_*``) let a worker sleep on a queue
+instead of busy-polling. Note:
+
+- Each Rx queue needs its own DPCON (notification channel), drawn from the
+ shared fslmc bus pool. The DPRC must provision at least as many DPCON
+ objects as Rx queues (these are shared with the event PMD); queue setup
+ fails with ``-ENODEV`` if the pool is exhausted.
+- ``rte_eth_dev_rx_intr_enable()`` / ``rte_eth_dev_rx_intr_disable()`` drive the
+ per-lcore QBMan software portal and must be called from the lcore that polls
+ the queue. The application must call ``rte_eth_dev_rx_intr_disable()`` on that
+ lcore before stopping or closing the port.
+- The queue's interrupt fd is bound when the queue is armed (it follows the
+ polling lcore's portal), not at queue setup. The application must therefore
+ arm a queue with ``rte_eth_dev_rx_intr_enable()`` before adding it to an
+ epoll set with ``rte_eth_dev_rx_intr_ctl_q()``; registering before arming
+ would watch an unbound fd. Several queues pinned to the same lcore share one
+ portal fd, so ``rte_eth_dev_rx_intr_ctl_q()`` returns ``-EEXIST`` after the
+ first; treat it as success.
+- The Rx error queue (``drv_err_queue`` devarg) is not drained while a queue
+ runs in interrupt mode.
+- The DPIO MSI is pinned to the polling core as a best-effort latency
+ optimisation (writes ``/proc/irq/<n>/smp_affinity``, needs privilege, and
+ irqbalance may move it back); wakeups are delivered correctly regardless.
+
Other Limitations
~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/nics/features/dpaa2.ini b/doc/guides/nics/features/dpaa2.ini
index 5def653d1d..4788afef8d 100644
--- a/doc/guides/nics/features/dpaa2.ini
+++ b/doc/guides/nics/features/dpaa2.ini
@@ -8,6 +8,7 @@ Speed capabilities = Y
Link status = Y
Link status event = Y
Burst mode info = Y
+Rx interrupt = Y
Queue start/stop = Y
Scattered Rx = Y
MTU update = Y
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index 8b1bdada1a..d3268ab867 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -167,6 +167,7 @@ New Features
``RTE_ETH_RX_OFFLOAD_VLAN_STRIP`` is no longer advertised,
as no hardware strip backs it.
An application that needs the tag removed must now strip it itself.
+ * Added Rx queue interrupt support (rte_eth_dev_rx_intr_*).
* **Updated NXP enetc ethernet driver.**
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
index 05645d667a..9dad78eba4 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
@@ -204,12 +204,23 @@ dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)
fclose(file);
}
+#endif /* RTE_EVENT_DPAA2 */
-static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, bool build_epoll)
+/* threshold: DQRR fill raising DQRI (< ring depth); timeout: holdoff in ITP units.
+ * Per-mode values from the caller (eventdev vs rx-queue intr).
+ * The DQRI config is portal-wide and this is idempotent: the first caller to
+ * arm a portal wins, a later caller's values are ignored (a portal normally
+ * serves a single mode).
+ */
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dpio_intr_init)
+int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, int threshold,
+ int timeout, bool build_epoll)
{
- struct epoll_event epoll_ev;
int eventfd, dpio_epoll_fd, ret;
- int threshold = 0x3, timeout = 0xFF;
+ struct epoll_event epoll_ev;
+
+ if (dpio_dev->intr_enabled)
+ return 0;
ret = rte_dpaa2_intr_enable(dpio_dev->intr_handle, 0);
if (ret) {
@@ -217,12 +228,6 @@ static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, bool build_epol
return -1;
}
- if (getenv("DPAA2_PORTAL_INTR_THRESHOLD"))
- threshold = atoi(getenv("DPAA2_PORTAL_INTR_THRESHOLD"));
-
- if (getenv("DPAA2_PORTAL_INTR_TIMEOUT"))
- sscanf(getenv("DPAA2_PORTAL_INTR_TIMEOUT"), "%x", &timeout);
-
qbman_swp_interrupt_set_trigger(dpio_dev->sw_portal,
QBMAN_SWP_INTERRUPT_DQRI);
qbman_swp_interrupt_clear_status(dpio_dev->sw_portal, 0xffffffff);
@@ -233,9 +238,9 @@ static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, bool build_epol
dpio_dev->epoll_fd = -1;
/* The event PMD dequeues by sleeping on a private epoll instance owned
- * by the portal, so build it here. A caller that waits on another
- * epoll (the net rx-queue-interrupt path uses the application's) skips
- * this.
+ * by the portal, so build it here. The net rx-queue-interrupt path
+ * exposes the raw eventfd through the generic ethdev API and waits on
+ * the application's own epoll instead, so it skips this.
*/
if (build_epoll) {
dpio_epoll_fd = epoll_create1(EPOLL_CLOEXEC);
@@ -259,13 +264,19 @@ static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, bool build_epol
dpio_dev->epoll_fd = dpio_epoll_fd;
}
+ dpio_dev->intr_enabled = 1;
+
return 0;
}
-static void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev)
+RTE_EXPORT_INTERNAL_SYMBOL(dpaa2_dpio_intr_deinit)
+void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev)
{
int ret;
+ if (!dpio_dev->intr_enabled)
+ return;
+
ret = rte_dpaa2_intr_disable(dpio_dev->intr_handle, 0);
if (ret)
DPAA2_BUS_ERR("DPIO interrupt disable failed");
@@ -274,11 +285,11 @@ static void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev)
close(dpio_dev->epoll_fd);
dpio_dev->epoll_fd = -1;
}
+ dpio_dev->intr_enabled = 0;
}
-#endif
static int
-dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id)
+dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id, bool ethrx)
{
int sdest, ret;
@@ -297,11 +308,26 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id)
}
#ifdef RTE_EVENT_DPAA2
- if (dpaa2_dpio_intr_init(dpio_dev, true)) {
- DPAA2_BUS_ERR("Interrupt registration failed for dpio");
- return -1;
+ /* Only the event PMD's portal is set up here (coalesced DQRI, private
+ * epoll). The net rx-interrupt portal (ethrx) is configured by the net
+ * PMD in rx_queue_intr_enable, so do not touch it here.
+ */
+ if (!ethrx) {
+ int threshold = 3, timeout = 0xFF;
+
+ if (getenv("DPAA2_PORTAL_INTR_THRESHOLD"))
+ threshold = atoi(getenv("DPAA2_PORTAL_INTR_THRESHOLD"));
+ if (getenv("DPAA2_PORTAL_INTR_TIMEOUT"))
+ sscanf(getenv("DPAA2_PORTAL_INTR_TIMEOUT"), "%x", &timeout);
+
+ if (dpaa2_dpio_intr_init(dpio_dev, threshold, timeout, true)) {
+ DPAA2_BUS_ERR("Interrupt registration failed for dpio");
+ return -1;
+ }
}
dpaa2_affine_dpio_intr_to_respective_core(dpio_dev->hw_id, cpu_id);
+#else
+ RTE_SET_USED(ethrx);
#endif
return 0;
@@ -310,14 +336,16 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id)
static void dpaa2_put_qbman_swp(struct dpaa2_dpio_dev *dpio_dev)
{
if (dpio_dev) {
-#ifdef RTE_EVENT_DPAA2
+ /* rx-queue interrupts (net PMD) can arm a portal without the
+ * event driver; tear it down unconditionally. Safe when never
+ * armed: intr_deinit returns early if intr is not enabled.
+ */
dpaa2_dpio_intr_deinit(dpio_dev);
-#endif
rte_atomic16_clear(&dpio_dev->ref_count);
}
}
-static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(void)
+static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(bool ethrx)
{
struct dpaa2_dpio_dev *dpio_dev = NULL;
int cpu_id;
@@ -343,7 +371,7 @@ static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(void)
if (dpaa2_svr_family != SVR_LX2160A)
qbman_swp_update(dpio_dev->sw_portal, 1);
} else {
- ret = dpaa2_configure_stashing(dpio_dev, cpu_id);
+ ret = dpaa2_configure_stashing(dpio_dev, cpu_id, ethrx);
if (ret) {
DPAA2_BUS_ERR("dpaa2_configure_stashing failed");
rte_atomic16_clear(&dpio_dev->ref_count);
@@ -370,7 +398,7 @@ dpaa2_affine_qbman_swp(void)
/* Populate the dpaa2_io_portal structure */
if (!RTE_PER_LCORE(_dpaa2_io).dpio_dev) {
- dpio_dev = dpaa2_get_qbman_swp();
+ dpio_dev = dpaa2_get_qbman_swp(false);
if (!dpio_dev) {
DPAA2_BUS_ERR("Error in software portal allocation");
return -1;
@@ -392,7 +420,7 @@ dpaa2_affine_qbman_ethrx_swp(void)
/* Populate the dpaa2_io_portal structure */
if (!RTE_PER_LCORE(_dpaa2_io).ethrx_dpio_dev) {
- dpio_dev = dpaa2_get_qbman_swp();
+ dpio_dev = dpaa2_get_qbman_swp(true);
if (!dpio_dev) {
DPAA2_BUS_ERR("Error in software portal allocation");
return -1;
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
index 328e1e788a..1f6e521341 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h
@@ -50,6 +50,14 @@ int dpaa2_affine_qbman_swp(void);
__rte_internal
int dpaa2_affine_qbman_ethrx_swp(void);
+/* set up / tear down a DPIO portal's DQRI interrupt (rx-queue interrupt mode) */
+__rte_internal
+int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, int threshold,
+ int timeout, bool build_epoll);
+
+__rte_internal
+void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev);
+
/* allocate memory for FQ - dq storage */
__rte_internal
int
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index 2a6d970101..ef65e7895a 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -133,6 +133,8 @@ struct dpaa2_dpio_dev {
struct rte_intr_handle *intr_handle; /* Interrupt related info */
int32_t epoll_fd; /**< File descriptor created for interrupt polling */
int32_t hw_id; /**< An unique ID of this DPIO device instance */
+ uint8_t intr_enabled; /**< DQRI portal interrupt already set up */
+ uint16_t ethrx_intr_refcnt; /**< rx queues currently armed on this portal */
struct dpaa2_portal_dqrr dpaa2_held_bufs;
};
@@ -195,6 +197,13 @@ struct __rte_cache_aligned dpaa2_queue {
uint64_t offloads;
uint64_t lpbk_cntx;
uint8_t data_stashing_off;
+ /* NAPI rx-interrupt: per-queue DPCON bound to this FQ at dev_start
+ * (DEST_DPCON, static); the polling worker points the channel's CDAN at
+ * its ethrx portal, arms the DQRI, and drains by channel volatile pull.
+ */
+ struct dpaa2_dpcon_dev *napi_dpcon; /*!< notif channel, NULL = napi off */
+ RTE_ATOMIC(struct dpaa2_dpio_dev *) napi_sub_dpio; /*!< subscribed portal or NULL */
+ uint8_t napi_armed; /*!< this queue requests DQRI wakeups */
};
struct swp_active_dqs {
diff --git a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
index 5375ea386d..b9e7789766 100644
--- a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
+++ b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h
@@ -189,6 +189,7 @@ int qbman_swp_interrupt_get_inhibit(struct qbman_swp *p);
* @p: the given software portal object.
* @mask: The value to set in SWP_IIR register.
*/
+__rte_internal
void qbman_swp_interrupt_set_inhibit(struct qbman_swp *p, int inhibit);
/************/
@@ -400,6 +401,7 @@ void qbman_pull_desc_set_wq(struct qbman_pull_desc *d, uint32_t wqid,
* @chid: the channel id to be dequeued.
* @dct: the dequeue command type.
*/
+__rte_internal
void qbman_pull_desc_set_channel(struct qbman_pull_desc *d, uint32_t chid,
enum qbman_pull_type_e dct);
@@ -1298,6 +1300,7 @@ int qbman_swp_fq_xoff(struct qbman_swp *s, uint32_t fqid);
*
* Return 0 for success, or negative error code for failure.
*/
+__rte_internal
int qbman_swp_CDAN_set_context(struct qbman_swp *s, uint16_t channelid,
uint64_t ctx);
@@ -1308,6 +1311,7 @@ int qbman_swp_CDAN_set_context(struct qbman_swp *s, uint16_t channelid,
*
* Return 0 for success, or negative error code for failure.
*/
+__rte_internal
int qbman_swp_CDAN_enable(struct qbman_swp *s, uint16_t channelid);
/**
@@ -1317,6 +1321,7 @@ int qbman_swp_CDAN_enable(struct qbman_swp *s, uint16_t channelid);
*
* Return 0 for success, or negative error code for failure.
*/
+__rte_internal
int qbman_swp_CDAN_disable(struct qbman_swp *s, uint16_t channelid);
/**
diff --git a/drivers/bus/fslmc/qbman/qbman_portal.c b/drivers/bus/fslmc/qbman/qbman_portal.c
index 84853924e7..c93bec5dd3 100644
--- a/drivers/bus/fslmc/qbman/qbman_portal.c
+++ b/drivers/bus/fslmc/qbman/qbman_portal.c
@@ -448,6 +448,7 @@ int qbman_swp_interrupt_get_inhibit(struct qbman_swp *p)
return qbman_cinh_read(&p->sys, QBMAN_CINH_SWP_IIR);
}
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_interrupt_set_inhibit)
void qbman_swp_interrupt_set_inhibit(struct qbman_swp *p, int inhibit)
{
qbman_cinh_write(&p->sys, QBMAN_CINH_SWP_IIR,
@@ -1878,6 +1879,7 @@ void qbman_pull_desc_set_wq(struct qbman_pull_desc *d, uint32_t wqid,
d->pull.dq_src = wqid;
}
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_pull_desc_set_channel)
void qbman_pull_desc_set_channel(struct qbman_pull_desc *d, uint32_t chid,
enum qbman_pull_type_e dct)
{
@@ -2921,6 +2923,7 @@ static int qbman_swp_CDAN_set(struct qbman_swp *s, uint16_t channelid,
return 0;
}
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_CDAN_set_context)
int qbman_swp_CDAN_set_context(struct qbman_swp *s, uint16_t channelid,
uint64_t ctx)
{
@@ -2929,6 +2932,7 @@ int qbman_swp_CDAN_set_context(struct qbman_swp *s, uint16_t channelid,
0, ctx);
}
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_CDAN_enable)
int qbman_swp_CDAN_enable(struct qbman_swp *s, uint16_t channelid)
{
return qbman_swp_CDAN_set(s, channelid,
@@ -2936,6 +2940,7 @@ int qbman_swp_CDAN_enable(struct qbman_swp *s, uint16_t channelid)
1, 0);
}
+RTE_EXPORT_INTERNAL_SYMBOL(qbman_swp_CDAN_disable)
int qbman_swp_CDAN_disable(struct qbman_swp *s, uint16_t channelid)
{
return qbman_swp_CDAN_set(s, channelid,
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 61122b323f..9d3634d75f 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -5,6 +5,8 @@
#include <time.h>
#include <net/if.h>
+#include <unistd.h>
+#include <errno.h>
#include <eal_export.h>
#include <rte_mbuf.h>
@@ -25,6 +27,7 @@
#include <dpaa2_hw_mempool.h>
#include <dpaa2_hw_dpio.h>
#include <mc/fsl_dpmng.h>
+#include <mc/fsl_dpcon.h>
#include "dpaa2_ethdev.h"
#include "dpaa2_sparser.h"
#include <fsl_qbman_debug.h>
@@ -658,6 +661,8 @@ dpaa2_clear_queue_active_dps(struct dpaa2_queue *q, int num_lcores)
}
}
+static void dpaa2_dev_rx_queue_intr_unbind(struct dpaa2_queue *dpaa2_q);
+
static void
dpaa2_free_rx_tx_queues(struct rte_eth_dev *dev)
{
@@ -675,6 +680,13 @@ dpaa2_free_rx_tx_queues(struct rte_eth_dev *dev)
/* cleaning up queue storage */
for (i = 0; i < priv->nb_rx_queues; i++) {
dpaa2_q = priv->rx_vq[i];
+ if (dpaa2_q->napi_dpcon) { /* release the rx-intr channel */
+ dpaa2_dev_rx_queue_intr_unbind(dpaa2_q);
+ rte_dpaa2_free_dpcon_dev(dpaa2_q->napi_dpcon);
+ dpaa2_q->napi_dpcon = NULL;
+ rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio,
+ NULL, rte_memory_order_release);
+ }
dpaa2_clear_queue_active_dps(dpaa2_q,
RTE_MAX_LCORE);
dpaa2_queue_storage_free(dpaa2_q,
@@ -728,6 +740,15 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
PMD_INIT_FUNC_TRACE();
+ /* interrupt mode is fixed once set; the queue count may still change */
+ if (priv->intr_mode_set &&
+ priv->intr_mode != eth_conf->intr_conf.rxq) {
+ DPAA2_PMD_ERR("Rx interrupt mode is fixed after configure; close the port to change it");
+ return -ENOTSUP;
+ }
+ priv->intr_mode = eth_conf->intr_conf.rxq;
+ priv->intr_mode_set = 1;
+
/* Rx offloads which are enabled by default */
if (dev_rx_offloads_nodis & ~rx_offloads) {
DPAA2_PMD_INFO(
@@ -880,6 +901,28 @@ dpaa2_eth_dev_configure(struct rte_eth_dev *dev)
}
}
+ /* allocate the intr handle once, sized to the queue ceiling; kept across
+ * reconfigure and freed at dev_close so the app epoll registration holds
+ */
+ if (dev->data->dev_conf.intr_conf.rxq && dev->intr_handle == NULL) {
+ dev->intr_handle = rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE);
+ if (!dev->intr_handle ||
+ rte_intr_vec_list_alloc(dev->intr_handle, "rxq_intr",
+ priv->nb_rx_queues) ||
+ rte_intr_nb_efd_set(dev->intr_handle, priv->nb_rx_queues) ||
+ rte_intr_type_set(dev->intr_handle, RTE_INTR_HANDLE_EXT)) {
+ DPAA2_PMD_ERR("Failed to set up rx-queue interrupts");
+ /* capture the error before cleanup may clobber rte_errno */
+ ret = rte_errno ? -rte_errno : -EIO;
+ if (dev->intr_handle) {
+ rte_intr_vec_list_free(dev->intr_handle);
+ rte_intr_instance_free(dev->intr_handle);
+ dev->intr_handle = NULL;
+ }
+ return ret;
+ }
+ }
+
dpaa2_tm_init(dev);
return 0;
@@ -898,6 +941,7 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
{
struct dpaa2_dev_priv *priv = dev->data->dev_private;
struct fsl_mc_io *dpni = dev->process_private;
+ bool dpcon_allocated = false;
struct dpaa2_queue *dpaa2_q;
struct dpni_queue cfg;
uint8_t options = 0;
@@ -938,6 +982,29 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
dpaa2_q->bp_array = rte_dpaa2_bpid_info;
dpaa2_q->offloads = rx_conf->offloads;
+ /* NAPI: grab a DPCON channel so dev_start can bind this FQ statically.
+ * The DQRR burst replaces the poll path for every queue at once, so a
+ * missing channel is fatal rather than a silent per-queue fallback.
+ */
+ rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio, NULL,
+ rte_memory_order_release);
+ if (dev->data->dev_conf.intr_conf.rxq && !dpaa2_q->napi_dpcon) {
+ dpaa2_q->napi_dpcon = rte_dpaa2_alloc_dpcon_dev();
+ if (!dpaa2_q->napi_dpcon) {
+ DPAA2_PMD_ERR("rxq %d: no DPCON for rx-queue interrupts",
+ rx_queue_id);
+ return -ENODEV;
+ }
+ dpcon_allocated = true;
+ /* the DPCON must be enabled to generate CDAN notifications */
+ ret = dpcon_enable(&dpaa2_q->napi_dpcon->dpcon, CMD_PRI_LOW,
+ dpaa2_q->napi_dpcon->token);
+ if (ret) {
+ DPAA2_PMD_ERR("rxq %d: dpcon_enable: %d", rx_queue_id, ret);
+ goto err_free_dpcon;
+ }
+ }
+
/*Get the flow id from given VQ id*/
flow_id = dpaa2_q->flow_id;
memset(&cfg, 0, sizeof(struct dpni_queue));
@@ -945,6 +1012,15 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
options = options | DPNI_QUEUE_OPT_USER_CTX;
cfg.user_context = (size_t)(dpaa2_q);
+ /* napi: schedule the FQ to its DPCON once, when the channel is first
+ * grabbed; never re-bound (the MC refuses scheduled->parked at runtime)
+ */
+ if (dpcon_allocated) {
+ options |= DPNI_QUEUE_OPT_DEST;
+ cfg.destination.type = DPNI_DEST_DPCON;
+ cfg.destination.id = dpaa2_q->napi_dpcon->dpcon_id;
+ }
+
/* check if a private cgr available. */
for (i = 0; i < priv->max_cgs; i++) {
if (!priv->cgid_in_use[i]) {
@@ -985,7 +1061,7 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
dpaa2_q->tc_index, flow_id, options, &cfg);
if (ret) {
DPAA2_PMD_ERR("Error in setting the rx flow: = %d", ret);
- return ret;
+ goto err_free_dpcon;
}
dpaa2_q->nb_desc = nb_rx_desc;
@@ -1026,7 +1102,7 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
if (ret) {
DPAA2_PMD_ERR("Error in setting taildrop. err=(%d)",
ret);
- return ret;
+ goto err_free_dpcon;
}
} else { /* Disable tail Drop */
struct dpni_taildrop taildrop = {0};
@@ -1046,12 +1122,22 @@ dpaa2_dev_rx_queue_setup(struct rte_eth_dev *dev,
if (ret) {
DPAA2_PMD_ERR("Error in setting taildrop. err=(%d)",
ret);
- return ret;
+ goto err_free_dpcon;
}
}
dev->data->rx_queues[rx_queue_id] = dpaa2_q;
return 0;
+
+err_free_dpcon:
+ /* free only the DPCON this call allocated; a pre-existing one belongs to
+ * an earlier setup and is released at dev_close
+ */
+ if (dpcon_allocated) {
+ rte_dpaa2_free_dpcon_dev(dpaa2_q->napi_dpcon);
+ dpaa2_q->napi_dpcon = NULL;
+ }
+ return ret;
}
static int
@@ -1210,6 +1296,28 @@ dpaa2_dev_tx_queue_setup(struct rte_eth_dev *dev,
return 0;
}
+/* drop a queue's worker portal reference; the FQ stays scheduled to its DPCON
+ * (only dpni_reset at close un-schedules it). Teardown only.
+ */
+static void
+dpaa2_dev_rx_queue_intr_unbind(struct dpaa2_queue *dpaa2_q)
+{
+ if (!dpaa2_q || !dpaa2_q->napi_dpcon)
+ return;
+
+ /* the portal (CDAN, DQRI) can only be torn down on the polling lcore;
+ * warn if the app skipped rte_eth_dev_rx_intr_disable() before stop/close
+ */
+ if (dpaa2_q->napi_armed)
+ DPAA2_PMD_WARN("rxq flow %u freed while armed; call "
+ "rte_eth_dev_rx_intr_disable() on the polling "
+ "lcore before stop/close", dpaa2_q->flow_id);
+
+ /* CDAN and DQRI teardown is the worker's job via rte_eth_dev_rx_intr_disable() */
+ rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio, NULL,
+ rte_memory_order_release);
+}
+
static void
dpaa2_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
@@ -1239,6 +1347,10 @@ dpaa2_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t rx_queue_id)
priv->cgid_in_use[dpaa2_q->cgid] = 0;
dpaa2_q->cgid = DPAA2_INVALID_CGID;
}
+
+ /* keep the DPCON on the FQ across a reconfigure; freed at dev_close */
+ if (dpaa2_q->napi_dpcon)
+ dpaa2_dev_rx_queue_intr_unbind(dpaa2_q);
}
static int
@@ -1289,7 +1401,9 @@ dpaa2_supported_ptypes_get(struct rte_eth_dev *dev, size_t *no_of_elements)
if (dev->rx_pkt_burst == dpaa2_dev_prefetch_rx ||
dev->rx_pkt_burst == dpaa2_dev_rx ||
- dev->rx_pkt_burst == dpaa2_dev_loopback_rx) {
+ dev->rx_pkt_burst == dpaa2_dev_loopback_rx ||
+ dev->rx_pkt_burst == dpaa2_dev_rx_channel ||
+ dev->rx_pkt_burst == dpaa2_dev_prefetch_rx_channel) {
*no_of_elements = RTE_DIM(ptypes);
return ptypes;
}
@@ -1389,6 +1503,27 @@ dpaa2_dev_start(struct rte_eth_dev *dev)
intr_handle = dpaa2_dev->intr_handle;
PMD_INIT_FUNC_TRACE();
+
+ /* napi: FQs are scheduled to their DPCON at rx_queue_setup, not here; just
+ * select the channel burst (the worker subscribes its portal later)
+ */
+ if (dev->data->dev_conf.intr_conf.rxq) {
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ dpaa2_q = data->rx_queues[i];
+ /* the channel burst below derefs napi_dpcon, so every
+ * rx queue must own a channel
+ */
+ if (!dpaa2_q->napi_dpcon) {
+ DPAA2_PMD_ERR("napi: rxq %d has no DPCON channel", i);
+ return -ENODEV;
+ }
+ }
+ if (priv->flags & DPAA2_NO_PREFETCH_RX)
+ dev->rx_pkt_burst = dpaa2_dev_rx_channel;
+ else
+ dev->rx_pkt_burst = dpaa2_dev_prefetch_rx_channel;
+ }
+
ret = dpni_enable(dpni, CMD_PRI_LOW, priv->token);
if (ret) {
DPAA2_PMD_ERR("Failure in enabling dpni %d device: err=%d",
@@ -1569,6 +1704,15 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
/* Free private queues memory */
dpaa2_free_rx_tx_queues(dev);
+ /* release the rx-queue interrupt vector and instance allocated at
+ * dev_configure (configure/close pair); the per-queue CDAN and portal
+ * teardown is the worker's job via rte_eth_dev_rx_intr_disable().
+ */
+ if (dev->intr_handle) {
+ rte_intr_vec_list_free(dev->intr_handle);
+ rte_intr_instance_free(dev->intr_handle);
+ dev->intr_handle = NULL;
+ }
/* Close the device at underlying layer*/
ret = dpni_close(dpni, CMD_PRI_LOW, priv->token);
if (ret) {
@@ -2897,6 +3041,258 @@ rte_pmd_dpaa2_thread_init(void)
}
}
+/* Arm rx-queue interrupts on the worker lcore: subscribe its ethrx portal to
+ * the queue's DPCON channel (one-shot per-portal MC) and unmask the portal DQRI
+ * (pure QBMan).
+ *
+ * Affinity is static queue-to-lcore; a lcore may own several rx queues. The
+ * DQRI and the eventfd are portal-wide; on wake the worker polls each armed
+ * queue's channel (one FQ per channel, no per-frame demux) and the portal's
+ * inhibit bit is reference-counted by the number of its
+ * queues currently armed (ethrx_intr_refcnt) -- disabling one queue must not
+ * mask wakeups still wanted by its siblings. napi_armed and ethrx_intr_refcnt
+ * are plain (not atomic): these ops run on the queue's owner lcore against its
+ * own portal (one portal per lcore), so per-portal isolation keeps them from
+ * racing, not control-plane serialization.
+ *
+ * A re-home reclaims the channel by poking the old portal, so the caller must
+ * have quiesced the previous owner and disabled the queue there; napi_armed is
+ * then 0 and only the new portal is counted.
+ */
+/* Drain the portal DQRR (consuming pending CDANs) then clear SWPn_ISR.
+ * Clearing while the ring is non-empty re-asserts the DQRI at once, so drain
+ * first. Shared by arm and disarm; the caller sets the inhibit bit.
+ */
+static void
+dpaa2_napi_drain_portal(struct dpaa2_dpio_dev *dpio)
+{
+ const struct qbman_result *dq;
+
+ while ((dq = qbman_swp_dqrr_next(dpio->sw_portal)))
+ qbman_swp_dqrr_consume(dpio->sw_portal, dq);
+ qbman_swp_interrupt_clear_status(dpio->sw_portal, 0xffffffff);
+}
+
+/* Quiesce this queue's in-flight prefetch VDQ before a sleep: a pull left in
+ * flight strands the FQ (no empty->non-empty edge, no CDAN). Returns -EAGAIN if
+ * the completed pull captured a frame -> abort the sleep and keep polling.
+ */
+static int
+dpaa2_napi_quiesce_vdq(struct dpaa2_queue *dpaa2_q)
+{
+ struct queue_storage_info_t *qs = dpaa2_q->q_storage[rte_lcore_id()];
+ struct qbman_result *dq;
+
+ if (!qs || !qs->active_dqs)
+ return 0;
+ dq = qs->active_dqs;
+ while (!qbman_check_command_complete(dq))
+ ;
+ /* a completed pull holding a frame must not be dropped: leave it. Inspect
+ * without qbman_check_new_result(), which zeroes the token the next burst
+ * needs to detect completion.
+ */
+ if (!qbman_result_DQ_is_pull_complete(dq) ||
+ (qbman_result_DQ_flags(dq) & QBMAN_DQ_STAT_VALIDFRAME)) {
+ return -EAGAIN;
+ }
+ /* empty pull-complete: consume the token, drop the storage for a fresh pull */
+ while (!qbman_check_new_result(dq))
+ ;
+ clear_swp_active_dqs(qs->active_dpio_id);
+ qs->active_dqs = NULL;
+ return 0;
+}
+
+/* Route this DPCON's CDAN to the worker's DPIO at MC level (re-issued on
+ * re-home), with the queue as the wake-demux context.
+ */
+static int
+dpaa2_napi_set_cdan_dest(struct dpaa2_queue *dpaa2_q, struct dpaa2_dpio_dev *dpio)
+{
+ struct dpcon_notification_cfg ncfg = {
+ .dpio_id = dpio->hw_id,
+ .priority = 0,
+ .user_ctx = (uint64_t)(uintptr_t)dpaa2_q,
+ };
+
+ return dpcon_set_notification(&dpaa2_q->napi_dpcon->dpcon,
+ CMD_PRI_LOW, dpaa2_q->napi_dpcon->token, &ncfg);
+}
+
+/* (Re)subscribe the queue's DPCON channel to dpio: set the CDAN context on the
+ * new portal, route notifications there, and wire the queue's eventfd into the
+ * generic rx-intr epoll. CDAN is not enabled here; the arm path is the sole
+ * arming point. Runs on the initial arm and on an lcore re-home.
+ */
+static int
+dpaa2_napi_subscribe(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct dpaa2_dpio_dev *dpio, struct dpaa2_dpio_dev *old)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_queue *dpaa2_q = priv->rx_vq[queue_id];
+ uint16_t chid = dpaa2_q->napi_dpcon->qbman_ch_id;
+ int ret;
+
+ if (old) {
+ /* re-home: CDAN already off old (-EBUSY ensures it was disabled
+ * first); just drop the sub, never touch a foreign portal.
+ */
+ rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio, NULL,
+ rte_memory_order_release);
+ }
+ /* push_set ORs the channel-0 bit (idempotent across queues sharing the
+ * portal); CDAN context = the queue, so the wake path can demux it.
+ */
+ qbman_swp_push_set(dpio->sw_portal, 0, 1);
+ /* context only; the arm path is the sole place that enables the CDAN */
+ ret = qbman_swp_CDAN_set_context(dpio->sw_portal, chid,
+ (uint64_t)(uintptr_t)dpaa2_q);
+ if (ret) {
+ DPAA2_PMD_ERR("napi: CDAN set_context rxq %d: %d", queue_id, ret);
+ return ret;
+ }
+ ret = dpaa2_napi_set_cdan_dest(dpaa2_q, dpio);
+ if (ret) {
+ DPAA2_PMD_ERR("napi: dpcon_set_notification rxq %d: %d",
+ queue_id, ret);
+ return ret; /* CDAN not enabled here -> nothing to unwind */
+ }
+ DPAA2_PMD_DEBUG("napi-cdan rxq=%u dpio=%u chid=%u",
+ queue_id, dpio->index, chid);
+ /* point the queue's eventfd at the portal DQRI fd for the generic epoll */
+ if (rte_intr_vec_list_index_set(dev->intr_handle, queue_id,
+ queue_id + RTE_INTR_VEC_RXTX_OFFSET) ||
+ rte_intr_efds_index_set(dev->intr_handle, queue_id,
+ rte_intr_fd_get(dpio->intr_handle))) {
+ DPAA2_PMD_ERR("napi: efd wiring rxq %d", queue_id);
+ return -EIO;
+ }
+ rte_atomic_store_explicit(&dpaa2_q->napi_sub_dpio, dpio,
+ rte_memory_order_release);
+ return 0;
+}
+
+static int
+dpaa2_dev_rx_queue_intr_enable(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_queue *dpaa2_q = priv->rx_vq[queue_id];
+ struct dpaa2_dpio_dev *dpio, *old;
+ int ret;
+
+ /* the generic API has no dev_started guard (unlike the burst dummy ops),
+ * so it can reach us after a stop; do not arm a stopped port, let the
+ * caller fall back to polling.
+ */
+ if (!dev->data->dev_started)
+ return -EIO;
+ if (!dpaa2_q->napi_dpcon)
+ return -ENOTSUP; /* no channel -> caller keeps polling */
+
+ if (dpaa2_affine_qbman_ethrx_swp())
+ return -EIO;
+ dpio = DPAA2_PER_LCORE_ETHRX_DPIO;
+
+ old = rte_atomic_load_explicit(&dpaa2_q->napi_sub_dpio, rte_memory_order_acquire);
+ if (old && old != dpio && dpaa2_q->napi_armed) {
+ DPAA2_PMD_ERR("rxq %d still armed on another portal; disable it first",
+ queue_id);
+ return -EBUSY;
+ }
+
+ ret = dpaa2_napi_quiesce_vdq(dpaa2_q);
+ if (ret)
+ return ret;
+
+ /* holdoff 0: the DQRI fires immediately on the first entry, so the
+ * threshold is moot (1 here). build_epoll=false: the generic rx-intr API
+ * waits on the application epoll, not the portal's private one. Idempotent
+ * per dpio.
+ */
+ ret = dpaa2_dpio_intr_init(dpio, 1, 0, false);
+ if (ret)
+ return ret;
+
+ if (old != dpio) {
+ ret = dpaa2_napi_subscribe(dev, queue_id, dpio, old);
+ if (ret)
+ return ret;
+ }
+
+ /* first arm on this portal: mask + drain stale CDANs, so the unmask on
+ * the 0 -> 1 edge below is the sole DQRI open (independent of prior state)
+ */
+ if (!dpaa2_q->napi_armed && dpio->ethrx_intr_refcnt == 0) {
+ qbman_swp_interrupt_set_inhibit(dpio->sw_portal, 1);
+ dpaa2_napi_drain_portal(dpio);
+ }
+
+ /* CDAN is one-shot and the sole arming point, re-armed every sleep.
+ * Publish armed/refcnt only on a successful initial enable.
+ */
+ ret = qbman_swp_CDAN_enable(dpio->sw_portal, dpaa2_q->napi_dpcon->qbman_ch_id);
+ if (ret) {
+ DPAA2_PMD_DEBUG("napi: CDAN arm rxq %d: %d", queue_id, ret);
+ /* initial arm: nothing published; re-arm: state kept. Don't sleep. */
+ return -EAGAIN;
+ }
+
+ if (!dpaa2_q->napi_armed) {
+ dpaa2_q->napi_armed = 1;
+ /* unmask the portal DQRI once, on the 0 -> 1 armed-queue edge */
+ if (dpio->ethrx_intr_refcnt++ == 0)
+ qbman_swp_interrupt_set_inhibit(dpio->sw_portal, 0);
+ }
+
+ return 0;
+}
+
+/* Disarm rx-queue interrupts for this queue. The portal DQRI is masked only
+ * once the last of its queues disarms; act on the portal the queue is actually
+ * subscribed to, not the caller's current portal.
+ */
+static int
+dpaa2_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct dpaa2_dev_priv *priv = dev->data->dev_private;
+ struct dpaa2_queue *dpaa2_q = priv->rx_vq[queue_id];
+ struct dpaa2_dpio_dev *dpio;
+ uint64_t ev;
+ ssize_t nb;
+
+ dpio = rte_atomic_load_explicit(&dpaa2_q->napi_sub_dpio, rte_memory_order_acquire);
+ if (dpio && dpaa2_q->napi_armed) {
+ dpaa2_q->napi_armed = 0;
+ /* Silence this queue's CDAN on the portal the worker owns. Runs on
+ * the polling lcore, so this is the only safe place to drive the
+ * single-owner software portal. It also returns the channel to the
+ * pool clean (CDAN disabled) and lets a sibling queue keep its own
+ * interrupt (per-queue, unlike the portal-wide DQRI mask below).
+ */
+ qbman_swp_CDAN_disable(dpio->sw_portal,
+ dpaa2_q->napi_dpcon->qbman_ch_id);
+ if (dpio->ethrx_intr_refcnt > 0 &&
+ --dpio->ethrx_intr_refcnt == 0) {
+ /* last queue on the shared portal: drain + clear so the next
+ * arm sees a clean portal (once per wake), then mask.
+ */
+ dpaa2_napi_drain_portal(dpio);
+ qbman_swp_interrupt_set_inhibit(dpio->sw_portal, 1);
+ /* drain the VFIO eventfd: it is EPOLLET and the EAL leaves
+ * RTE_INTR_HANDLE_EXT undrained, so a stale count suppresses the
+ * next wake edge. Same as dpaa/mlx4/mlx5 do on disarm.
+ */
+ nb = read(rte_intr_fd_get(dpio->intr_handle), &ev, sizeof(ev));
+ if (nb != (ssize_t)sizeof(ev) && errno != EAGAIN)
+ DPAA2_PMD_DEBUG("napi: eventfd drain rxq %d (%zd)",
+ queue_id, nb);
+ }
+ }
+
+ return 0;
+}
+
static struct eth_dev_ops dpaa2_ethdev_ops = {
.dev_configure = dpaa2_eth_dev_configure,
.dev_start = dpaa2_dev_start,
@@ -2925,6 +3321,11 @@ static struct eth_dev_ops dpaa2_ethdev_ops = {
.vlan_tpid_set = dpaa2_vlan_tpid_set,
.rx_queue_setup = dpaa2_dev_rx_queue_setup,
.rx_queue_release = dpaa2_dev_rx_queue_release,
+ /* arm/disarm drive the per-lcore QBMan portal: the app must call
+ * rte_eth_dev_rx_intr_disable() from the polling lcore before stop/close.
+ */
+ .rx_queue_intr_enable = dpaa2_dev_rx_queue_intr_enable,
+ .rx_queue_intr_disable = dpaa2_dev_rx_queue_intr_disable,
.tx_queue_setup = dpaa2_dev_tx_queue_setup,
.rx_burst_mode_get = dpaa2_dev_rx_burst_mode_get,
.tx_burst_mode_get = dpaa2_dev_tx_burst_mode_get,
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 3f224c654e..149954fb6b 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -444,6 +444,9 @@ struct dpaa2_dev_priv {
rte_spinlock_t lpbk_qp_lock;
uint8_t channel_inuse;
+ /* Rx-interrupt mode latched at the first dev_configure */
+ uint8_t intr_mode;
+ uint8_t intr_mode_set;
/* Stores correction offset for one step timestamping */
uint16_t ptp_correction_offset;
/* for mac counters */
@@ -500,6 +503,10 @@ uint16_t dpaa2_dev_loopback_rx(void *queue, struct rte_mbuf **bufs,
uint16_t dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs,
uint16_t nb_pkts);
+uint16_t dpaa2_dev_prefetch_rx_channel(void *queue, struct rte_mbuf **bufs,
+ uint16_t nb_pkts);
+uint16_t dpaa2_dev_rx_channel(void *queue, struct rte_mbuf **bufs,
+ uint16_t nb_pkts);
void dpaa2_dev_process_parallel_event(struct qbman_swp *swp,
const struct qbman_fd *fd,
const struct qbman_result *dq,
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 884cea43c9..c31d6556ad 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -764,13 +764,13 @@ dump_err_pkts(struct dpaa2_queue *dpaa2_q)
* It will return the packets as requested in previous call without honoring
* the current nb_pkts or bufs space.
*/
-uint16_t
-dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+static __rte_always_inline uint16_t
+dpaa2_dev_prefetch_rx_common(void *queue, struct rte_mbuf **bufs,
+ uint16_t nb_pkts, bool by_channel)
{
/* Function receive frames for a given device and VQ*/
struct dpaa2_queue *dpaa2_q = queue;
struct qbman_result *dq_storage, *dq_storage1 = NULL;
- uint32_t fqid = dpaa2_q->fqid;
int ret, num_rx = 0, pull_size;
uint8_t pending, status;
struct qbman_swp *swp;
@@ -778,12 +778,18 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
struct qbman_pull_desc pulldesc;
struct queue_storage_info_t *q_storage;
struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
- struct dpaa2_dev_priv *priv = eth_data->dev_private;
q_storage = dpaa2_q->q_storage[rte_lcore_id()];
- if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
- dump_err_pkts(priv->rx_err_vq);
+ /* the error-queue drain runs on the regular portal (DPAA2_PER_LCORE_PORTAL);
+ * the channel/interrupt path owns the ethrx portal, so skip it there
+ */
+ if (!by_channel) {
+ struct dpaa2_dev_priv *priv = eth_data->dev_private;
+
+ if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
+ dump_err_pkts(priv->rx_err_vq);
+ }
if (unlikely(!DPAA2_PER_LCORE_ETHRX_DPIO)) {
ret = dpaa2_affine_qbman_ethrx_swp();
@@ -793,7 +799,7 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
}
}
- if (unlikely(!rte_dpaa2_bpid_info &&
+ if (!by_channel && unlikely(!rte_dpaa2_bpid_info &&
rte_eal_process_type() == RTE_PROC_SECONDARY))
rte_dpaa2_bpid_info = dpaa2_q->bp_array;
@@ -806,7 +812,12 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
qbman_pull_desc_clear(&pulldesc);
qbman_pull_desc_set_numframes(&pulldesc,
q_storage->last_num_pkts);
- qbman_pull_desc_set_fq(&pulldesc, fqid);
+ if (by_channel)
+ qbman_pull_desc_set_channel(&pulldesc,
+ dpaa2_q->napi_dpcon->qbman_ch_id,
+ qbman_pull_type_active);
+ else
+ qbman_pull_desc_set_fq(&pulldesc, dpaa2_q->fqid);
qbman_pull_desc_set_storage(&pulldesc, dq_storage,
(uint64_t)(DPAA2_VADDR_TO_IOVA(dq_storage)), 1);
if (check_swp_active_dqs(DPAA2_PER_LCORE_ETHRX_DPIO->index)) {
@@ -842,7 +853,12 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
dq_storage1 = q_storage->dq_storage[q_storage->toggle];
qbman_pull_desc_clear(&pulldesc);
qbman_pull_desc_set_numframes(&pulldesc, pull_size);
- qbman_pull_desc_set_fq(&pulldesc, fqid);
+ if (by_channel)
+ qbman_pull_desc_set_channel(&pulldesc,
+ dpaa2_q->napi_dpcon->qbman_ch_id,
+ qbman_pull_type_active);
+ else
+ qbman_pull_desc_set_fq(&pulldesc, dpaa2_q->fqid);
qbman_pull_desc_set_storage(&pulldesc, dq_storage1,
(uint64_t)(DPAA2_VADDR_TO_IOVA(dq_storage1)), 1);
@@ -885,6 +901,8 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
bufs[num_rx] = eth_fd_to_mbuf(fd, eth_data->port_id);
#if defined(RTE_LIBRTE_IEEE1588)
if (bufs[num_rx]->ol_flags & RTE_MBUF_F_RX_IEEE1588_TMST) {
+ struct dpaa2_dev_priv *priv = eth_data->dev_private;
+
priv->rx_timestamp =
*dpaa2_timestamp_dynfield(bufs[num_rx]);
}
@@ -918,6 +936,27 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
return num_rx;
}
+uint16_t __rte_hot
+dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+{
+ return dpaa2_dev_prefetch_rx_common(queue, bufs, nb_pkts, false);
+}
+
+/* rx interrupt/CDAN path: each Rx FQ is scheduled to its own DPCON channel,
+ * which a CDAN notifies on empty->non-empty. A VDQ on a scheduled FQ never
+ * completes, so the channel is drained by a volatile pull on the CHANNEL (not
+ * the FQID); one FQ per channel, so no demux is needed. Same prefetch pipeline
+ * as dpaa2_dev_prefetch_rx (one VDQ kept in flight to mask the ~700 cycle
+ * latency); the portal's single in-flight VDQ (global_active_dqs) must be
+ * quiesced before an interrupt sleep.
+ */
+uint16_t __rte_hot
+dpaa2_dev_prefetch_rx_channel(void *queue, struct rte_mbuf **bufs,
+ uint16_t nb_pkts)
+{
+ return dpaa2_dev_prefetch_rx_common(queue, bufs, nb_pkts, true);
+}
+
void __rte_hot
dpaa2_dev_process_parallel_event(struct qbman_swp *swp,
const struct qbman_fd *fd,
@@ -997,8 +1036,9 @@ dpaa2_dev_process_ordered_event(struct qbman_swp *swp,
qbman_swp_dqrr_consume(swp, dq);
}
-uint16_t
-dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+static __rte_always_inline uint16_t
+dpaa2_dev_rx_common(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts,
+ bool by_channel)
{
/* Function receive frames for a given device and VQ */
struct dpaa2_queue *dpaa2_q = queue;
@@ -1012,24 +1052,39 @@ dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
struct dpaa2_dev_priv *priv = eth_data->dev_private;
- if (unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
+ if (!by_channel && unlikely(priv->flags & DPAAX_RX_ERROR_QUEUE_FLAG))
dump_err_pkts(priv->rx_err_vq);
- if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
- ret = dpaa2_affine_qbman_swp();
- if (ret) {
- DPAA2_PMD_ERR(
- "Failed to allocate IO portal, tid: %d",
- rte_gettid());
- return 0;
+ if (by_channel) {
+ if (unlikely(!DPAA2_PER_LCORE_ETHRX_DPIO)) {
+ ret = dpaa2_affine_qbman_ethrx_swp();
+ if (ret) {
+ DPAA2_PMD_ERR("Failure in affining portal");
+ return 0;
+ }
+ }
+ swp = DPAA2_PER_LCORE_ETHRX_PORTAL;
+ } else {
+ if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
+ ret = dpaa2_affine_qbman_swp();
+ if (ret) {
+ DPAA2_PMD_ERR("Failed to allocate IO portal, tid: %d",
+ rte_gettid());
+ return 0;
+ }
}
+ swp = DPAA2_PER_LCORE_PORTAL;
}
- swp = DPAA2_PER_LCORE_PORTAL;
do {
dq_storage = dpaa2_q->q_storage[0]->dq_storage[0];
qbman_pull_desc_clear(&pulldesc);
- qbman_pull_desc_set_fq(&pulldesc, fqid);
+ if (by_channel)
+ qbman_pull_desc_set_channel(&pulldesc,
+ dpaa2_q->napi_dpcon->qbman_ch_id,
+ qbman_pull_type_active);
+ else
+ qbman_pull_desc_set_fq(&pulldesc, fqid);
qbman_pull_desc_set_storage(&pulldesc, dq_storage,
(size_t)(DPAA2_VADDR_TO_IOVA(dq_storage)), 1);
@@ -1108,6 +1163,22 @@ dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
return num_rx;
}
+uint16_t
+dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+{
+ return dpaa2_dev_rx_common(queue, bufs, nb_pkts, false);
+}
+
+/* rx-interrupt path: synchronous volatile dequeue on the queue's DPCON channel
+ * (no prefetch pipeline, so nothing is left in flight across an interrupt
+ * sleep, and a channel VDQ -- unlike an FQ VDQ -- completes on a scheduled FQ).
+ */
+uint16_t __rte_hot
+dpaa2_dev_rx_channel(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
+{
+ return dpaa2_dev_rx_common(queue, bufs, nb_pkts, true);
+}
+
uint16_t dpaa2_dev_tx_conf(void *queue)
{
/* Function receive frames for a given device and VQ */
@@ -1500,7 +1571,6 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
}
skip_tx:
dpaa2_q->tx_pkts += num_tx;
-
for (loop = 0; loop < free_count; loop++) {
if (buf_to_free[loop].pkt_id < num_tx)
rte_pktmbuf_free_seg(buf_to_free[loop].seg);
--
2.43.0
^ permalink raw reply related
* [PATCH v4 4/6] bus/fslmc/mc: implement DPCON set notification
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
The DPCON_CMDID_SET_NOTIFICATION command id and its wire struct were
already present in the flib, but the dpcon_set_notification() function and
the public dpcon_notification_cfg were never implemented. Add them so a
DPCON channel can be pointed at a DPIO for CDAN delivery, which the
net/dpaa2 Rx-queue-interrupt path needs.
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/bus/fslmc/mc/dpcon.c | 31 +++++++++++++++++++++++++++++++
drivers/bus/fslmc/mc/fsl_dpcon.h | 18 ++++++++++++++++++
2 files changed, 49 insertions(+)
diff --git a/drivers/bus/fslmc/mc/dpcon.c b/drivers/bus/fslmc/mc/dpcon.c
index cd909fcac4..c01d7adabb 100644
--- a/drivers/bus/fslmc/mc/dpcon.c
+++ b/drivers/bus/fslmc/mc/dpcon.c
@@ -338,3 +338,34 @@ int dpcon_get_api_version(struct fsl_mc_io *mc_io,
return 0;
}
+
+/**
+ * dpcon_set_notification() - Set the DPCON notification destination
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPCON object
+ * @cfg: Notification parameters (DPIO, priority, user context)
+ *
+ * Return: '0' on Success; Error code otherwise
+ */
+RTE_EXPORT_INTERNAL_SYMBOL(dpcon_set_notification)
+int dpcon_set_notification(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags,
+ uint16_t token,
+ const struct dpcon_notification_cfg *cfg)
+{
+ struct dpcon_cmd_set_notification *dpcon_cmd;
+ struct mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPCON_CMDID_SET_NOTIFICATION,
+ cmd_flags,
+ token);
+ dpcon_cmd = (struct dpcon_cmd_set_notification *)cmd.params;
+ dpcon_cmd->dpio_id = cpu_to_le32(cfg->dpio_id);
+ dpcon_cmd->priority = cfg->priority;
+ dpcon_cmd->user_ctx = cpu_to_le64(cfg->user_ctx);
+
+ /* Send command to MC. */
+ return mc_send_command(mc_io, &cmd);
+}
diff --git a/drivers/bus/fslmc/mc/fsl_dpcon.h b/drivers/bus/fslmc/mc/fsl_dpcon.h
index f2668da1b1..3bef97f6f5 100644
--- a/drivers/bus/fslmc/mc/fsl_dpcon.h
+++ b/drivers/bus/fslmc/mc/fsl_dpcon.h
@@ -95,4 +95,22 @@ int dpcon_get_api_version(struct fsl_mc_io *mc_io,
uint16_t *major_ver,
uint16_t *minor_ver);
+/**
+ * struct dpcon_notification_cfg - Notification parameters
+ * @dpio_id: DPIO object id that receives the channel's CDAN
+ * @priority: Priority within the DPIO channel (0-7)
+ * @user_ctx: User context provided with each CDAN message
+ */
+struct dpcon_notification_cfg {
+ int dpio_id;
+ uint8_t priority;
+ uint64_t user_ctx;
+};
+
+__rte_internal
+int dpcon_set_notification(struct fsl_mc_io *mc_io,
+ uint32_t cmd_flags,
+ uint16_t token,
+ const struct dpcon_notification_cfg *cfg);
+
#endif /* __FSL_DPCON_H */
--
2.43.0
^ permalink raw reply related
* [PATCH v4 3/6] bus/fslmc/dpio: make the portal DQRI epoll optional
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
dpaa2_dpio_intr_init() builds a private epoll instance the event PMD
sleeps on. The upcoming net rx-queue-interrupt path waits on the
application's own epoll instead, so that instance would be built but
never used.
Add a build_epoll parameter: pass true to build it (event PMD), false
to skip the epoll_create/epoll_ctl. epoll_fd is set to -1 when none is
built and closed in intr_deinit only when valid. The sole caller passes
true: no functional change.
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 44 +++++++++++++++++-------
1 file changed, 32 insertions(+), 12 deletions(-)
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
index 2a9e519668..05645d667a 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c
@@ -205,13 +205,12 @@ dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)
fclose(file);
}
-static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev)
+static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, bool build_epoll)
{
struct epoll_event epoll_ev;
int eventfd, dpio_epoll_fd, ret;
int threshold = 0x3, timeout = 0xFF;
- dpio_epoll_fd = epoll_create(1);
ret = rte_dpaa2_intr_enable(dpio_dev->intr_handle, 0);
if (ret) {
DPAA2_BUS_ERR("Interrupt registration failed");
@@ -231,16 +230,34 @@ static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev)
qbman_swp_dqrr_thrshld_write(dpio_dev->sw_portal, threshold);
qbman_swp_intr_timeout_write(dpio_dev->sw_portal, timeout);
- eventfd = rte_intr_fd_get(dpio_dev->intr_handle);
- epoll_ev.events = EPOLLIN | EPOLLPRI | EPOLLET;
- epoll_ev.data.fd = eventfd;
+ dpio_dev->epoll_fd = -1;
- ret = epoll_ctl(dpio_epoll_fd, EPOLL_CTL_ADD, eventfd, &epoll_ev);
- if (ret < 0) {
- DPAA2_BUS_ERR("epoll_ctl failed");
- return -1;
+ /* The event PMD dequeues by sleeping on a private epoll instance owned
+ * by the portal, so build it here. A caller that waits on another
+ * epoll (the net rx-queue-interrupt path uses the application's) skips
+ * this.
+ */
+ if (build_epoll) {
+ dpio_epoll_fd = epoll_create1(EPOLL_CLOEXEC);
+ if (dpio_epoll_fd < 0) {
+ DPAA2_BUS_ERR("epoll_create failed");
+ rte_dpaa2_intr_disable(dpio_dev->intr_handle, 0);
+ return -1;
+ }
+
+ eventfd = rte_intr_fd_get(dpio_dev->intr_handle);
+ epoll_ev.events = EPOLLIN | EPOLLPRI | EPOLLET;
+ epoll_ev.data.fd = eventfd;
+
+ ret = epoll_ctl(dpio_epoll_fd, EPOLL_CTL_ADD, eventfd, &epoll_ev);
+ if (ret < 0) {
+ DPAA2_BUS_ERR("epoll_ctl failed");
+ rte_dpaa2_intr_disable(dpio_dev->intr_handle, 0);
+ close(dpio_epoll_fd);
+ return -1;
+ }
+ dpio_dev->epoll_fd = dpio_epoll_fd;
}
- dpio_dev->epoll_fd = dpio_epoll_fd;
return 0;
}
@@ -253,7 +270,10 @@ static void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev)
if (ret)
DPAA2_BUS_ERR("DPIO interrupt disable failed");
- close(dpio_dev->epoll_fd);
+ if (dpio_dev->epoll_fd >= 0) {
+ close(dpio_dev->epoll_fd);
+ dpio_dev->epoll_fd = -1;
+ }
}
#endif
@@ -277,7 +297,7 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id)
}
#ifdef RTE_EVENT_DPAA2
- if (dpaa2_dpio_intr_init(dpio_dev)) {
+ if (dpaa2_dpio_intr_init(dpio_dev, true)) {
DPAA2_BUS_ERR("Interrupt registration failed for dpio");
return -1;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v4 2/6] bus/fslmc: move DPCON management from event driver to bus
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
The DPCON allocation helpers (rte_dpaa2_alloc_dpcon_dev /
rte_dpaa2_free_dpcon_dev) lived in the event driver, but a notification
channel is a generic QBMan resource. Move dpaa2_hw_dpcon.c to the fslmc
bus and export the helpers as internal symbols so both the event PMD and
the net driver's rx-queue interrupt path can draw channels from the same
pool. No functional change.
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
drivers/bus/fslmc/meson.build | 1 +
.../dpaa2 => bus/fslmc/portal}/dpaa2_hw_dpcon.c | 16 +++++++---------
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 6 ++++++
drivers/event/dpaa2/dpaa2_eventdev.h | 3 ---
drivers/event/dpaa2/meson.build | 1 -
5 files changed, 14 insertions(+), 13 deletions(-)
rename drivers/{event/dpaa2 => bus/fslmc/portal}/dpaa2_hw_dpcon.c (90%)
diff --git a/drivers/bus/fslmc/meson.build b/drivers/bus/fslmc/meson.build
index ceae1c6c11..50d9e91a37 100644
--- a/drivers/bus/fslmc/meson.build
+++ b/drivers/bus/fslmc/meson.build
@@ -22,6 +22,7 @@ sources = files(
'mc/mc_sys.c',
'portal/dpaa2_hw_dpbp.c',
'portal/dpaa2_hw_dpci.c',
+ 'portal/dpaa2_hw_dpcon.c',
'portal/dpaa2_hw_dpio.c',
'portal/dpaa2_hw_dprc.c',
'qbman/qbman_portal.c',
diff --git a/drivers/event/dpaa2/dpaa2_hw_dpcon.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpcon.c
similarity index 90%
rename from drivers/event/dpaa2/dpaa2_hw_dpcon.c
rename to drivers/bus/fslmc/portal/dpaa2_hw_dpcon.c
index ea5b0d4b85..6fd96ec0b9 100644
--- a/drivers/event/dpaa2/dpaa2_hw_dpcon.c
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpcon.c
@@ -18,13 +18,12 @@
#include <rte_cycles.h>
#include <rte_kvargs.h>
#include <dev_driver.h>
-#include <ethdev_driver.h>
+#include <eal_export.h>
#include <bus_fslmc_driver.h>
#include <mc/fsl_dpcon.h>
#include <portal/dpaa2_hw_pvt.h>
-#include "dpaa2_eventdev.h"
-#include "dpaa2_eventdev_logs.h"
+#include <fslmc_logs.h>
TAILQ_HEAD(dpcon_dev_list, dpaa2_dpcon_dev);
static struct dpcon_dev_list dpcon_dev_list
@@ -55,8 +54,7 @@ rte_dpaa2_create_dpcon_device(int dev_fd __rte_unused,
/* Allocate DPAA2 dpcon handle */
dpcon_node = rte_malloc(NULL, sizeof(struct dpaa2_dpcon_dev), 0);
if (!dpcon_node) {
- DPAA2_EVENTDEV_ERR(
- "Memory allocation failed for dpcon device");
+ DPAA2_BUS_ERR("Memory allocation failed for dpcon device");
return -1;
}
@@ -65,8 +63,7 @@ rte_dpaa2_create_dpcon_device(int dev_fd __rte_unused,
ret = dpcon_open(&dpcon_node->dpcon,
CMD_PRI_LOW, dpcon_id, &dpcon_node->token);
if (ret) {
- DPAA2_EVENTDEV_ERR("Unable to open dpcon device: err(%d)",
- ret);
+ DPAA2_BUS_ERR("Unable to open dpcon device: err(%d)", ret);
rte_free(dpcon_node);
return -1;
}
@@ -75,8 +72,7 @@ rte_dpaa2_create_dpcon_device(int dev_fd __rte_unused,
ret = dpcon_get_attributes(&dpcon_node->dpcon,
CMD_PRI_LOW, dpcon_node->token, &attr);
if (ret != 0) {
- DPAA2_EVENTDEV_ERR("dpcon attribute fetch failed: err(%d)",
- ret);
+ DPAA2_BUS_ERR("dpcon attribute fetch failed: err(%d)", ret);
rte_free(dpcon_node);
return -1;
}
@@ -92,6 +88,7 @@ rte_dpaa2_create_dpcon_device(int dev_fd __rte_unused,
return 0;
}
+RTE_EXPORT_INTERNAL_SYMBOL(rte_dpaa2_alloc_dpcon_dev)
struct dpaa2_dpcon_dev *rte_dpaa2_alloc_dpcon_dev(void)
{
struct dpaa2_dpcon_dev *dpcon_dev = NULL;
@@ -105,6 +102,7 @@ struct dpaa2_dpcon_dev *rte_dpaa2_alloc_dpcon_dev(void)
return dpcon_dev;
}
+RTE_EXPORT_INTERNAL_SYMBOL(rte_dpaa2_free_dpcon_dev)
void rte_dpaa2_free_dpcon_dev(struct dpaa2_dpcon_dev *dpcon)
{
struct dpaa2_dpcon_dev *dpcon_dev = NULL;
diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
index e625a5c035..2a6d970101 100644
--- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
+++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h
@@ -274,6 +274,12 @@ struct dpaa2_dpcon_dev {
uint8_t channel_index;
};
+/* DPCON channel pool, shared by the net rx-interrupt path and the event PMD */
+__rte_internal
+struct dpaa2_dpcon_dev *rte_dpaa2_alloc_dpcon_dev(void);
+__rte_internal
+void rte_dpaa2_free_dpcon_dev(struct dpaa2_dpcon_dev *dpcon);
+
/* Refer to Table 7-3 in SEC BG */
#define QBMAN_FLE_WORD4_FMT_SBF 0x0 /* Single buffer frame */
#define QBMAN_FLE_WORD4_FMT_SGE 0x2 /* Scatter gather frame */
diff --git a/drivers/event/dpaa2/dpaa2_eventdev.h b/drivers/event/dpaa2/dpaa2_eventdev.h
index bb87bdbab2..3883a26c91 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.h
+++ b/drivers/event/dpaa2/dpaa2_eventdev.h
@@ -85,9 +85,6 @@ struct dpaa2_eventdev {
uint32_t event_dev_cfg;
};
-struct dpaa2_dpcon_dev *rte_dpaa2_alloc_dpcon_dev(void);
-void rte_dpaa2_free_dpcon_dev(struct dpaa2_dpcon_dev *dpcon);
-
int test_eventdev_dpaa2(void);
#endif /* __DPAA2_EVENTDEV_H__ */
diff --git a/drivers/event/dpaa2/meson.build b/drivers/event/dpaa2/meson.build
index dd5063af43..62b8507652 100644
--- a/drivers/event/dpaa2/meson.build
+++ b/drivers/event/dpaa2/meson.build
@@ -7,7 +7,6 @@ if not is_linux
endif
deps += ['bus_vdev', 'net_dpaa2', 'crypto_dpaa2_sec']
sources = files(
- 'dpaa2_hw_dpcon.c',
'dpaa2_eventdev.c',
'dpaa2_eventdev_selftest.c',
)
--
2.43.0
^ permalink raw reply related
* [PATCH v4 1/6] net/dpaa2: fix Rx queue count for primary process
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy, stable
In-Reply-To: <20260710210920.2402309-1-maxime@leroys.fr>
The rx_queue_count callback was only assigned on the secondary process
path of dpaa2_dev_init(), leaving eth_dev->rx_queue_count NULL for the
primary process. The fast-path rte_eth_rx_queue_count() performs an
unguarded indirect call in non-debug builds, so invoking it on a
primary-process dpaa2 port dereferences a NULL function pointer and
crashes.
Assign the callback on the primary-process path as well, so both the
primary and secondary paths set it.
Fixes: cbfc6111b557 ("ethdev: move inline device operations")
Cc: stable@dpdk.org
Signed-off-by: Maxime Leroy <maxime@leroys.fr>
---
drivers/net/dpaa2/dpaa2_ethdev.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index a68404ee5e..61122b323f 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -3276,6 +3276,7 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
}
eth_dev->dev_ops = &dpaa2_ethdev_ops;
+ eth_dev->rx_queue_count = dpaa2_dev_rx_queue_count;
if (dpaa2_get_devargs(dev->devargs, DRIVER_LOOPBACK_MODE)) {
eth_dev->rx_pkt_burst = dpaa2_dev_loopback_rx;
--
2.43.0
^ permalink raw reply related
* [PATCH v4 0/6] net/dpaa2: NAPI-style Rx queue interrupts
From: Maxime Leroy @ 2026-07-10 21:09 UTC (permalink / raw)
To: dev; +Cc: Hemant Agrawal, Jun Yang, Maxime Leroy
In-Reply-To: <20260630144329.457643-1-maxime@leroys.fr>
This series lets a dpaa2 worker sleep on a queue's data-availability
notification instead of busy-polling, exposed through the generic
rte_eth_dev_rx_intr_* API (NAPI-style: poll while frames keep coming,
arm the interrupt and sleep when the queue runs dry).
Why it is not a trivial .rx_queue_intr_enable
----------------------------------------------
A worker wakes on its software portal's DQRI, which fires when the
portal's DQRR holds frames. The default dpaa2 Rx burst pulls frames
from the FQ with a volatile dequeue and cannot be interrupt-driven; to
wake on the DQRI the FQ must instead be pushed to the portal's DQRR.
The natural dpni_set_queue with a notification destination would have to
target the worker's portal, but that portal is only known once a worker
affines, after dev_start, and that MC command holds the global MC lock
long enough to wedge the firmware while traffic runs. So the bind cannot
be done late, against the polling lcore.
Design
------
Each Rx FQ is bound to its own DPCON channel, statically, at
rx_queue_setup while the dpni is still disabled (no knowledge of the
polling lcore). It stays scheduled to that channel for the life of the
port and is never re-bound, because the MC refuses to move a scheduled
FQ back to parked at runtime; a queue-count reconfigure (stop, configure,
setup, start) is therefore supported without a port close. A worker later
subscribes its own ethrx portal to the channel and arms the DQRI in
rx_queue_intr_enable, a one-shot per-portal op, never the wedging
set_queue. On a wakeup the worker drains each of its queues by a volatile
dequeue on the queue's own DPCON channel (one FQ per channel, so no
per-frame demux); it polls all its queues, the same scheduling contract
as plain DPDK polling. A queue can be re-homed to another lcore at
runtime with no set_queue and no port stop.
This reuses the event PMD's pushed/DQRR model but with one DPCON per FQ
and static affinity (no QBMan scheduling), so the DPCON allocator is
moved from the event driver to the fslmc bus and shared.
Patch 1 fixes an existing dpaa2 bug the interrupt path relies on: the
rx_queue_count callback was wired only for the secondary process, so the
generic arm-then-verify (rte_eth_rx_queue_count) misbehaves on a primary
process. Patches 2 to 4 move the DPCON allocator to the fslmc bus, make
the portal DQRI epoll optional, and add the DPCON set-notification MC
command. Patch 5 adds the interrupt support proper; patch 6 pins each
DPIO's MSI to the lcore that arms it, a latency optimisation.
Tested on LX2160A (lx2160acex7).
v4:
- Addressed Hemant's v3 review:
- Dropped the "event/dpaa2: disable channel before closing it" patch.
Pooled DPAA2 objects reprogram on borrow (the sibling DPCI close has
no dpci_disable either), so an enabled DPCON returned to the pool is
inert; the patch was an unnecessary outlier.
- Allocate the rx-interrupt handle once and keep it, instead of calling
rte_intr_vec_list_alloc() again on the existing handle at a second
dev_configure (which leaked the vector list and could size-mismatch).
- Store napi_sub_dpio with rte_atomic_store_explicit() at rx_queue_setup
instead of a plain assignment, for weakly-ordered architectures.
- Re-added the "fix Rx queue count for primary process" patch (patch 1).
v3 dropped it as applied to main, but only the ethdev dummy/keep-fast-
path fixes are upstream; the dpaa2 primary-process wiring is not, and
rte_eth_rx_queue_count returns 0 (defeating arm-then-verify) without it.
- Made the Rx queue count changeable at runtime in interrupt mode: the FQ
is scheduled to its DPCON once, at rx_queue_setup, and never re-bound
(the handle kept above lets the application's rx-intr epoll registration
stay valid across a reconfigure). Switching the interrupt mode on or off
is refused after the first configure.
- Advertise the supported packet types in interrupt mode too:
dpaa2_supported_ptypes_get returned NULL for the channel Rx bursts
(dpaa2_dev_rx_channel, dpaa2_dev_prefetch_rx_channel), so an application
saw no ptypes although they are parsed; the bursts are now recognised.
- Documented the arm-before-register contract (the interrupt fd is bound
at arm time) in doc/guides/nics/dpaa2.rst.
- Rebased on next-net/for-main (v3 was on an older main snapshot).
- Reworded the DPCON set-notification headline to drop the underscore.
v3:
- Reworked the Rx drain. Both versions bind one DPCON per FQ, but v2
drained the shared portal DQRR and demuxed frames to their FQ by
fqd_ctx, stashing foreign frames in a per-queue FIFO. v3 drains each
queue with a volatile dequeue on its own channel (one FQ per channel),
which drops the demux and stash code.
- Dropped the rx_queue_count fix; it is applied to main.
- Dropped the software VLAN strip patch; an independent net/dpaa2 cleanup,
sent standalone and now applied to next-net.
- Dropped the Depends-on: the ethdev fast-path ops fix is now in main.
- Split the dpcon_set_notification MC command into its own patch.
- Added an event/dpaa2 fix to disable the DPCON channel before close,
needed once the allocator is shared.
- Dropped the DQRI holdoff-tuning patch; the immediate-DQRI holdoff is now
set inline in the arm path.
- Added a patch reusing the event driver's MSI-affinity helper (exposed
from its RTE_EVENT_DPAA2 guard) to pin the portal MSI to the lcore that
arms it, so a CDAN wake lands on the worker's own core.
v2:
- Dropped the RSS RETA patch, an independent net/dpaa2 change the
interrupt path does not require; it will be sent as its own series.
- Dropped the ethdev fast-path ops fix; it is now a standalone series.
- Dropped the eal/interrupts -EEXIST fix, applied to main by David
Marchand.
- Declared qbman_swp_interrupt_set_inhibit and qbman_swp_dqrr_size
__rte_internal (David Marchand).
- Minor formatting cleanup in the Rx interrupt setup.
Maxime Leroy (6):
net/dpaa2: fix Rx queue count for primary process
bus/fslmc: move DPCON management from event driver to bus
bus/fslmc/dpio: make the portal DQRI epoll optional
bus/fslmc/mc: implement DPCON set notification
net/dpaa2: support Rx queue interrupts
net/dpaa2: pin Rx queue interrupt to the polling core
doc/guides/nics/dpaa2.rst | 28 ++
doc/guides/nics/features/dpaa2.ini | 1 +
doc/guides/rel_notes/release_26_07.rst | 1 +
drivers/bus/fslmc/mc/dpcon.c | 31 ++
drivers/bus/fslmc/mc/fsl_dpcon.h | 18 +
drivers/bus/fslmc/meson.build | 1 +
.../fslmc/portal}/dpaa2_hw_dpcon.c | 16 +-
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 125 ++++--
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 12 +
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 15 +
.../fslmc/qbman/include/fsl_qbman_portal.h | 5 +
drivers/bus/fslmc/qbman/qbman_portal.c | 5 +
drivers/event/dpaa2/dpaa2_eventdev.h | 3 -
drivers/event/dpaa2/meson.build | 1 -
drivers/net/dpaa2/dpaa2_ethdev.c | 414 +++++++++++++++++-
drivers/net/dpaa2/dpaa2_ethdev.h | 7 +
drivers/net/dpaa2/dpaa2_rxtx.c | 114 ++++-
17 files changed, 725 insertions(+), 72 deletions(-)
rename drivers/{event/dpaa2 => bus/fslmc/portal}/dpaa2_hw_dpcon.c (90%)
--
2.43.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox