[RFC 0/1] ethdev: per-packet Tx timestamp slot management

[RFC 0/1] ethdev: per-packet Tx timestamp slot management

[Rajesh Kumar]

The current DPDK ethdev time synchronization framework is architected
around a single, shared hardware latch. The existing API,
`rte_eth_timesync_read_tx_timestamp()`, assumes a serialization model
where only one TX timestamp is outstanding at any given time.

This model creates severe limitations for modern high-throughput network
interface cards (NICs). When multiple packets requiring precise
transmit timestamps are sent concurrently, the shared latch becomes a
race-condition bottleneck. It makes timestamp retrieval unreliable and
drops accuracy. Furthermore, Poll Mode Drivers (PMDs) backed by
hardware that supports independent, per-packet timestamping slots have
no way to expose this capability to the user.

To solve this, this RFC introduces a formal slot-based lifecycle API
for per-packet Tx timestamp management. The API decouples timestamp
tracking from the global latch model, enabling true asynchronous,
parallel hardware timestamping.

Key Components of the Proposal:
================================
1. **Slot Lifecycle Management APIs**:
   - `rte_eth_timesync_tx_timestamp_slot_alloc()`: Allocates and locks
     a unique hardware slot prior to frame transmission.
   - `rte_eth_timesync_tx_timestamp_stamp_mbuf()`: Embeds the allocated
     slot handle into an mbuf dynamic field, allowing the PMD to
     program the specific hardware descriptor during the Tx burst.
   - `rte_eth_timesync_read_tx_timestamp_slot()`: Asynchronously polls
     a specific slot for its captured value, safely returning `-EAGAIN`
     if the hardware has not yet written back the timestamp.
   - `rte_eth_timesync_tx_timestamp_slot_release()`: Recycles the
     hardware slot back to the PMD resource pool after successful
     retrieval or an application timeout.

2. **Dual-Domain Timing Mechanics**:
   - Introduces `struct rte_eth_timesync_dual_domain_timestamp` to
     simultaneously capture both the adjusted PTP Hardware Clock (PHC)
     domain (`adjusted_ns`) and the raw, free-running cycles domain
     (`cycles_ns`).
   - A dedicated `valid_mask` tracks the validity of each time domain
     independently, giving applications granular telemetry options.

Scope of this RFC & Open Questions for the Community:
======================================================
This RFC establishes the structural definition of the APIs and driver
interfaces. Before proceeding to a full `v1` patch with active PMD
implementations, we would highly appreciate the community's feedback
on the following design choices:

- **Mbuf Dynamic Field Layout**: Is the encapsulation of the slot
  handle into an mbuf dynamic field using standard dynamic registration
  the preferred mechanism for passing steering context to the Tx
  descriptor path?
- **Dual-Domain Structure**: Does tracking both the adjusted PHC and
  the cycle counter within `rte_eth_timesync_dual_domain_timestamp`
  cover the needs of other hardware vendors supporting concurrent
  domains?
- **Error States**: Is `-EAGAIN` an acceptable return code for
  non-blocking polling of incomplete timestamps, or should we consider
  an explicit bitmask status?

Please review the proposed design. We welcome your feedback, design
suggestions, and critique.

Rajesh Kumar (1):
  ethdev: add per-packet Tx timestamp slot APIs

 lib/ethdev/ethdev_driver.h |  19 ++++++
 lib/ethdev/rte_ethdev.c    | 121 +++++++++++++++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h    | 108 +++++++++++++++++++++++++++++++++
 3 files changed, 248 insertions(+)

-- 
2.54.0

[RFC 1/1] ethdev: add per-packet Tx timestamp slot APIs

[Rajesh Kumar]

Add ethdev public and driver-facing APIs for per-packet Tx
hardware timestamp slot management.

The existing `rte_eth_timesync_read_tx_timestamp()` exposes a
single shared latch, making it unreliable when multiple Tx
timestamps are outstanding concurrently. PMDs with per-packet
slot hardware cannot be exploited through this interface.

Introduce a slot lifecycle API:
- `rte_eth_timesync_tx_timestamp_slot_alloc()`: reserve a
  hardware slot before transmit
- `rte_eth_timesync_tx_timestamp_stamp_mbuf()`: embed slot
  handle into mbuf dynfield for per-packet NIC steering
- `rte_eth_timesync_read_tx_timestamp_slot()`: poll slot for
  a captured timestamp; returns -EAGAIN if not ready
- `rte_eth_timesync_tx_timestamp_slot_release()`: return slot
  to PMD after readback or timeout

Introduce `rte_eth_timesync_dual_domain_timestamp` to carry
both the adjusted PHC time (`adjusted_ns`) and free-running
cycles-domain time (`cycles_ns`), each with an individual
validity bit in `valid_mask`.

Signed-off-by: Rajesh Kumar <rajesh3.kumar@intel.com>
---
 lib/ethdev/ethdev_driver.h |  19 ++++++
 lib/ethdev/rte_ethdev.c    | 121 +++++++++++++++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h    | 108 +++++++++++++++++++++++++++++++++
 3 files changed, 248 insertions(+)

diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
index 0f336f9567..61dadcea49 100644
--- a/lib/ethdev/ethdev_driver.h
+++ b/lib/ethdev/ethdev_driver.h
@@ -795,6 +795,19 @@ typedef int (*eth_timesync_read_rx_timestamp_t)(struct rte_eth_dev *dev,
 typedef int (*eth_timesync_read_tx_timestamp_t)(struct rte_eth_dev *dev,
 						struct timespec *timestamp);
 
+/** @internal Allocate a per-packet TX timestamp slot handle. */
+typedef int (*eth_timesync_tx_timestamp_slot_alloc_t)(struct rte_eth_dev *dev,
+		uint16_t tx_queue_id, uint32_t *slot_id);
+
+/** @internal Read TX timestamp by slot handle. */
+typedef int (*eth_timesync_read_tx_timestamp_slot_t)(struct rte_eth_dev *dev,
+		uint32_t slot_id,
+		struct rte_eth_timesync_dual_domain_timestamp *timestamp);
+
+/** @internal Release a previously allocated TX timestamp slot handle. */
+typedef int (*eth_timesync_tx_timestamp_slot_release_t)(struct rte_eth_dev *dev,
+		uint32_t slot_id);
+
 /** @internal Function used to adjust the device clock. */
 typedef int (*eth_timesync_adjust_time)(struct rte_eth_dev *dev, int64_t);
 
@@ -1561,6 +1574,12 @@ struct eth_dev_ops {
 	eth_timesync_read_rx_timestamp_t timesync_read_rx_timestamp;
 	/** Read the IEEE1588/802.1AS Tx timestamp */
 	eth_timesync_read_tx_timestamp_t timesync_read_tx_timestamp;
+	/** Allocate a TX timestamp slot handle */
+	eth_timesync_tx_timestamp_slot_alloc_t timesync_tx_timestamp_slot_alloc;
+	/** Read a TX timestamp using a slot handle */
+	eth_timesync_read_tx_timestamp_slot_t timesync_read_tx_timestamp_slot;
+	/** Release a TX timestamp slot handle */
+	eth_timesync_tx_timestamp_slot_release_t timesync_tx_timestamp_slot_release;
 	/** Adjust the device clock */
 	eth_timesync_adjust_time   timesync_adjust_time;
 	/** Adjust the clock frequency */
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 9efeaf77cb..b955784594 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -6699,6 +6699,127 @@ rte_eth_timesync_read_tx_timestamp(uint16_t port_id,
 
 }
 
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_timesync_tx_timestamp_slot_alloc, 26.11)
+int
+rte_eth_timesync_tx_timestamp_slot_alloc(uint16_t port_id,
+					 uint16_t tx_queue_id,
+					 uint32_t *slot_id)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	if (slot_id == NULL) {
+		RTE_ETHDEV_LOG_LINE(ERR,
+			"Cannot allocate ethdev port %u Tx timestamp slot to NULL",
+			port_id);
+		return -EINVAL;
+	}
+
+	if (dev->dev_ops->timesync_tx_timestamp_slot_alloc == NULL)
+		return -ENOTSUP;
+
+	return eth_err(port_id,
+			dev->dev_ops->timesync_tx_timestamp_slot_alloc(dev,
+				tx_queue_id, slot_id));
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_timesync_read_tx_timestamp_slot, 26.11)
+int
+rte_eth_timesync_read_tx_timestamp_slot(uint16_t port_id,
+					uint32_t slot_id,
+					struct rte_eth_timesync_dual_domain_timestamp *timestamp)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	if (timestamp == NULL) {
+		RTE_ETHDEV_LOG_LINE(ERR,
+			"Cannot read ethdev port %u Tx timestamp slot to NULL",
+			port_id);
+		return -EINVAL;
+	}
+
+	if (dev->dev_ops->timesync_read_tx_timestamp_slot == NULL)
+		return -ENOTSUP;
+
+	return eth_err(port_id,
+			dev->dev_ops->timesync_read_tx_timestamp_slot(dev,
+				slot_id, timestamp));
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_timesync_tx_timestamp_slot_release, 26.11)
+int
+rte_eth_timesync_tx_timestamp_slot_release(uint16_t port_id, uint32_t slot_id)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->dev_ops->timesync_tx_timestamp_slot_release == NULL)
+		return -ENOTSUP;
+
+	return eth_err(port_id,
+			dev->dev_ops->timesync_tx_timestamp_slot_release(dev,
+				slot_id));
+}
+
+static int rte_eth_timesync_tx_slot_dynfield_offset = -1;
+static uint64_t rte_eth_timesync_tx_slot_dynflag;
+
+static int
+rte_eth_timesync_tx_slot_dynfield_register(void)
+{
+	const struct rte_mbuf_dynfield slot_dynfield = {
+		.name  = RTE_ETH_TIMESYNC_TX_SLOT_DYNFIELD_NAME,
+		.size  = sizeof(uint32_t),
+		.align = alignof(uint32_t),
+	};
+
+	if (rte_eth_timesync_tx_slot_dynfield_offset >= 0)
+		return 0;
+
+	rte_eth_timesync_tx_slot_dynfield_offset =
+			rte_mbuf_dynfield_register(&slot_dynfield);
+	if (rte_eth_timesync_tx_slot_dynfield_offset < 0)
+		rte_eth_timesync_tx_slot_dynfield_offset =
+				rte_mbuf_dynfield_lookup(
+					RTE_ETH_TIMESYNC_TX_SLOT_DYNFIELD_NAME, NULL);
+	if (rte_eth_timesync_tx_slot_dynfield_offset < 0)
+		return -ENOTSUP;
+
+	{
+		int flag_bit = rte_mbuf_dynflag_register(
+			&(const struct rte_mbuf_dynflag){
+				.name = RTE_ETH_TIMESYNC_TX_SLOT_DYNFIELD_NAME "_flag"});
+		if (flag_bit < 0)
+			flag_bit = rte_mbuf_dynflag_lookup(
+				RTE_ETH_TIMESYNC_TX_SLOT_DYNFIELD_NAME "_flag", NULL);
+		if (flag_bit >= 0)
+			rte_eth_timesync_tx_slot_dynflag = RTE_BIT64(flag_bit);
+	}
+	return 0;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_timesync_tx_timestamp_stamp_mbuf, 26.11)
+int
+rte_eth_timesync_tx_timestamp_stamp_mbuf(uint16_t port_id __rte_unused,
+					 uint32_t slot_id, struct rte_mbuf *m)
+{
+	if (m == NULL)
+		return -EINVAL;
+	if (rte_eth_timesync_tx_slot_dynfield_register() != 0)
+		return -ENOTSUP;
+	*RTE_MBUF_DYNFIELD(m, rte_eth_timesync_tx_slot_dynfield_offset,
+			   uint32_t *) = slot_id;
+	m->ol_flags |= rte_eth_timesync_tx_slot_dynflag;
+	return 0;
+}
+
 RTE_EXPORT_SYMBOL(rte_eth_timesync_adjust_time)
 int
 rte_eth_timesync_adjust_time(uint16_t port_id, int64_t delta)
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index ee400b386f..339c39fcdd 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -5528,6 +5528,114 @@ int rte_eth_timesync_read_rx_timestamp(uint16_t port_id,
 int rte_eth_timesync_read_tx_timestamp(uint16_t port_id,
 		struct timespec *timestamp);
 
+/** Valid bit for rte_eth_timesync_dual_domain_timestamp.adjusted_ns. */
+#define RTE_ETH_TIMESYNC_DUAL_DOMAIN_TIMESTAMP_ADJUSTED_VALID	RTE_BIT32(0)
+/** Valid bit for rte_eth_timesync_dual_domain_timestamp.cycles_ns. */
+#define RTE_ETH_TIMESYNC_DUAL_DOMAIN_TIMESTAMP_CYCLES_VALID	RTE_BIT32(1)
+
+/**
+ * Dual-domain TX timestamp payload.
+ */
+struct rte_eth_timesync_dual_domain_timestamp {
+	int64_t adjusted_ns;
+	int64_t cycles_ns;
+	uint32_t valid_mask;
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Allocate a TX timestamp slot handle for per-packet correlation.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ * @param tx_queue_id
+ *   TX queue used by the packet to be transmitted.
+ * @param slot_id
+ *   Output handle identifying the allocated slot.
+ *
+ * @return
+ *   - 0: Success.
+ *   - -ENODEV: The port ID is invalid.
+ *   - -EIO: if device is removed.
+ *   - -ENOTSUP: The function is not supported by the Ethernet driver.
+ *   - -EINVAL: Invalid parameters.
+ */
+__rte_experimental
+int rte_eth_timesync_tx_timestamp_slot_alloc(uint16_t port_id,
+		uint16_t tx_queue_id, uint32_t *slot_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Read a per-packet TX timestamp using a previously allocated slot handle.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ * @param slot_id
+ *   Slot handle returned by rte_eth_timesync_tx_timestamp_slot_alloc().
+ * @param timestamp
+ *   Output dual-domain timestamp payload.
+ *
+ * @return
+ *   - 0: Success.
+ *   - -EAGAIN: Timestamp is not ready yet.
+ *   - -ENODEV: The port ID is invalid.
+ *   - -EIO: if device is removed.
+ *   - -ENOTSUP: The function is not supported by the Ethernet driver.
+ *   - -EINVAL: Invalid parameters.
+ */
+__rte_experimental
+int rte_eth_timesync_read_tx_timestamp_slot(uint16_t port_id,
+		uint32_t slot_id,
+		struct rte_eth_timesync_dual_domain_timestamp *timestamp);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Release a previously allocated TX timestamp slot handle.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ * @param slot_id
+ *   Slot handle to release.
+ *
+ * @return
+ *   - 0: Success.
+ *   - -ENODEV: The port ID is invalid.
+ *   - -EIO: if device is removed.
+ *   - -ENOTSUP: The function is not supported by the Ethernet driver.
+ *   - -EINVAL: Invalid parameters.
+ */
+__rte_experimental
+int rte_eth_timesync_tx_timestamp_slot_release(uint16_t port_id,
+		uint32_t slot_id);
+
+/** Mbuf dynfield name for the TX timestamp slot handle. */
+#define RTE_ETH_TIMESYNC_TX_SLOT_DYNFIELD_NAME "rte_eth_timesync_tx_slot"
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Stamp an mbuf with a TX timestamp slot handle so the TX path steers the
+ * NIC to capture the timestamp in the correct per-packet slot.
+ *
+ * Must be called after rte_eth_timesync_tx_timestamp_slot_alloc() and before
+ * rte_eth_tx_burst(). Safe for concurrent callers — slot is stored per-mbuf.
+ *
+ * @param port_id  The port identifier (unused; reserved for future PMD use).
+ * @param slot_id  Slot handle from rte_eth_timesync_tx_timestamp_slot_alloc().
+ * @param m        Mbuf to stamp.
+ * @return 0 on success, -ENOTSUP if dynfield registration failed.
+ */
+__rte_experimental
+int rte_eth_timesync_tx_timestamp_stamp_mbuf(uint16_t port_id,
+		uint32_t slot_id, struct rte_mbuf *m);
+
 /**
  * Adjust the timesync clock on an Ethernet device.
  *
-- 
2.54.0

Re: [RFC 0/1] ethdev: per-packet Tx timestamp slot management

[Stephen Hemminger]

On Tue, 18 Aug 2026 00:54:14 +0530
Rajesh Kumar <rajesh3.kumar@intel.com> wrote:

> The current DPDK ethdev time synchronization framework is architected
> around a single, shared hardware latch. The existing API,
> `rte_eth_timesync_read_tx_timestamp()`, assumes a serialization model
> where only one TX timestamp is outstanding at any given time.
> 
> This model creates severe limitations for modern high-throughput network
> interface cards (NICs). When multiple packets requiring precise
> transmit timestamps are sent concurrently, the shared latch becomes a
> race-condition bottleneck. It makes timestamp retrieval unreliable and
> drops accuracy. Furthermore, Poll Mode Drivers (PMDs) backed by
> hardware that supports independent, per-packet timestamping slots have
> no way to expose this capability to the user.
> 
> To solve this, this RFC introduces a formal slot-based lifecycle API
> for per-packet Tx timestamp management. The API decouples timestamp
> tracking from the global latch model, enabling true asynchronous,
> parallel hardware timestamping.


Lots of reasonable AI feedback to the design.

Review of the RFC. Design issues first since that's what they're asking for, then code defects.

Design

    No capability discovery or exhaustion semantics. Nothing reports how many slots exist, whether they're per-port or per-queue, and slot_alloc() doesn't document what it returns when slots run out (-ENOSPC? -EAGAIN?). That's the first thing an application hits. Needs a rte_eth_dev_info field or query, and a defined out-of-slots errno.
    Queue asymmetry: alloc() takes tx_queue_id but read() and release() don't. Either slot_id is port-global (then why does alloc need the queue?) or it's per-queue (then read/release are ambiguous). Pick one and document it. Also tx_queue_id is never validated against nb_tx_queues in the ethdev layer.
    Interaction with the existing mechanism is undefined. Does the app still set RTE_MBUF_F_TX_IEEE1588_TMST? Can the legacy latch API and the slot API coexist on one port? PMDs today key tx timestamping off that flag; the RFC needs to say what supersedes what.
    Fast-path cost contradicts the stated motivation. The cover letter argues high-throughput concurrent timestamping, but the lifecycle is three dev_ops indirect calls plus a dynfield write per packet, all through the slow path. Fine for PTP rates; if the claim is more than that, alloc/release want burst variants or the intended rate should be stated.
    cycles_ns is self-contradictory: is it raw counter cycles or nanoseconds from the free-running clock? If cycles, drop the _ns and expose the frequency; if ns, call it raw_ns or free_ns. Also this struct switches to int64 ns while every other timesync call uses struct timespec; probably the right move but justify it in the cover letter.
    PMDs can't consume the dynfield as written. The offset and flag are static in rte_ethdev.c and not exposed to drivers. A PMD has to re-lookup by name, and the dynflag name only exists as a string concat inside the .c file, so drivers would hardcode "..._flag". Define the flag name macro in the header and provide a lookup helper, following the RTE_MBUF_DYNFIELD_TIMESTAMP_NAME pattern.
    Naming: rte_eth_timesync_tx_timestamp_stamp_mbuf stutters. ..._tx_slot_set_mbuf or similar.

Defects

    Silent dynflag failure in rte_eth_timesync_tx_slot_dynfield_register(). If both rte_mbuf_dynflag_register() and the lookup fail, rte_eth_timesync_tx_slot_dynflag stays 0, the function returns 0, and stamp_mbuf() ORs 0 into ol_flags and reports success. The PMD never sees the request. Must return error. Worse, the early return on offset >= 0 means the flag is never retried on subsequent calls, so one transient failure is permanent.
    Likely doesn't compile as posted: the diff adds no includes, but uses struct rte_mbuf_dynfield, rte_mbuf_dynflag_register() (needs rte_mbuf_dyn.h) and alignof (needs stdalign.h pre-C23). Check whether rte_ethdev.c already pulls those in; I don't believe it does.
    stamp_mbuf() takes port_id and ignores it, documented as "reserved for future PMD use". Either validate it or drop it; a parameter whose semantics arrive later is an API trap. Dropping it also removes the false implication that the call is port-scoped.

Nits

    stamp_mbuf() doxygen deviates from the file's param style, omits the -EINVAL return the code actually produces, and contains an em-dash. The dual_domain_timestamp struct fields lack doxygen comments.
    v1 needs rel_notes and prog_guide (ptp section) updates; RFC is fine without.