All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stephen Hemminger <stephen@networkplumber.org>
To: "Kumar, Rajesh" <rajesh3.kumar@intel.com>
Cc: <dev@dpdk.org>, <thomas@monjalon.net>,
	<bruce.richardson@intel.com>, <andrew.rybchenko@oktetlabs.ru>,
	<aman.deep.singh@intel.com>
Subject: Re: [RFC PATCH v5 0/5] ethdev: add Tx timestamp slot APIs
Date: Tue, 6 Oct 2026 07:16:48 -0700	[thread overview]
Message-ID: <20261006071648.4bb37e4f@phoenix.local> (raw)
In-Reply-To: <7624cfc9-c624-43e2-a046-250d332abe7e@intel.com>

On Tue, 6 Oct 2026 15:58:29 +0530
"Kumar, Rajesh" <rajesh3.kumar@intel.com> wrote:

> Hi Stephen,
> I wanted to quickly check if you have any further comments or feedback 
> on this v5 series?
> If you are satisfied with the current changes, we can move forward to 
> the next steps for this patch.
> 
> Regards,
> 
> Rajesh
> 
> On 21-09-2026

Lots of open issues from AI review.
The biggest issue is that this overlaps with existing metering API
and seems  specific to Intel NIC's. I strongly discourage API's that
can only be used on one vendor NIC.


Series: [RFC PATCH v5 0/5] ethdev: add Tx timestamp slot APIs

Applied to main 49bb9a5 (release_26_11.rst conflict only). Each
commit builds with -Dwerror=true (net/intel/ice, test-pmd); no bisect
issues.

Series-level:

The main problem is in the ice implementation. Slot state is never
cleaned up before a slot is handed out, so the read path can return
a timestamp that belongs to an earlier packet as if it were the
current one (patch 4, Error).

The legacy static index and the slot allocator also overlap on ice.

On the API side, the caller has to set two separate mbuf flags for
one action. The caps query returns -ENOTSUP on every PMD except ice,
so it cannot be used to choose a mechanism as the doc says.

The patch 1 commit message and the cover letter describe an
unregister / "process-local reset" API that is not in this version.


Patch 1/5 ethdev: add Tx timestamp slot management APIs

Warning:

- The commit message says "Add APIs to register and unregister the
  mbuf dynamic field", and the cover letter lists "process-local
  reset". Only rte_eth_timesync_tx_slot_dynfield_register() exists.

- rte_eth_timesync_tx_slot_dynfield_register() falls back to
  rte_mbuf_dynfield_lookup() / rte_mbuf_dynflag_lookup() after a
  registration failure. Register already returns the existing offset
  when the name and params match, including in a secondary process.
  The only extra case the lookup covers is EEXIST with different
  params, and there it accepts a field of the wrong size or
  alignment. It also returns -ENOTSUP for every failure, losing
  rte_errno (ENOMEM, EEXIST). Drop both lookups and return
  -rte_errno.

- The register doc says "Must be called before the first
  rte_pktmbuf_pool_create()". rte_mbuf_dyn.h says "The registration
  can be done at any moment"; pool creation has no bearing on it.
  The same wrong claim appears in the patch 3 note.

- rte_eth_timesync_tx_slot_caps() returns -ENOTSUP for every in-tree
  PMD except ice, including all PMDs that implement
  timesync_read_tx_timestamp. Nothing ever reports
  RTE_ETH_TIMESYNC_TX_SLOT_SINGLE_REG, so "Applications use this to
  select the mechanism supported by the port" does not hold. Either
  have ethdev report SINGLE_REG when timesync_tx_slot_get_caps is
  NULL and timesync_read_tx_timestamp is set, or drop SINGLE_REG.

- rte_eth_timesync_tx_slot_stamp() sets the dynflag but not
  RTE_MBUF_F_TX_IEEE1588_TMST. The patch 3 example has to set TMST
  separately. If the application forgets it, ice takes the
  "else if (ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST)" path, no
  timestamp is requested, and the slot read returns -EAGAIN forever.
  stamp() should set both flags.

- The release note says the new APIs "support both shared-register
  and slot-bank usage models through the rte_eth_timesync_tx_slot_*
  interface family". The shared-register model is still served only
  by rte_eth_timesync_read_tx_timestamp().

- There is no functional test. dynfield_register() and stamp() are
  pure software and can be tested in app/test without hardware.

Info:

- The read and release doxygen says "is undefined behaviour - PMDs
  should return -EINVAL"; pick one. Those two lines (rte_ethdev.h
  5448, 5472) also use non-ASCII em-dashes. The release doc still
  refers to "stamp_mbuf", the old function name.

- Drop "This function does not make the PMD hardware lifecycle safe
  by itself." It gives the reader nothing to act on (also in
  patch 2).

- The new wrappers have no trace points. The existing timesync calls
  do, e.g. rte_eth_trace_timesync_read_tx_timestamp().

- Unrelated changes: a blank line removed before
  rte_eth_add_rx_callback, and the log text reworded in
  rte_eth_timesync_read_tx_timestamp().

- The definitions of alloc, release, dynfield_register and stamp put
  the return type on the same line as the name. caps and read in the
  same patch follow DPDK style.

- Nothing in the series sets raw_ns or
  RTE_ETH_TIMESYNC_DUAL_DOMAIN_TIMESTAMP_RAW_VALID. Consider leaving
  them out until a PMD produces them.

- The dynfield name "rte_eth_timesync_tx_slot" does not follow the
  rte_<libname>_dynfield_<name> convention in rte_mbuf_dyn.h.


Patch 2/5 doc: describe ethdev timesync clock and Rx timestamp API

Info:

- The commit message says the patch documents existing behaviour
  only. The "must ensure no Tx timestamp operations are in flight
  before disabling" requirement is new in patch 1; move it to
  patch 3.

- The flags argument of rte_eth_timesync_read_rx_timestamp() is not
  explained. ice uses it as the Rx queue index.


Patch 3/5 doc: describe ethdev Tx timestamp slot API

Warning:

- "* **Single Shared Register** (...):" and "* **Adjusted Domain**"
  are term/description bullet lists. Use RST definition lists. The
  same applies to the numbered "1. **Clock Control & Adjustment**:"
  list in patch 2.

Info:

- Indentation is inconsistent in the step 1 and step 4 code
  examples.

- Document how late a slot read may happen. ice extends a 32-bit ns
  value against the current PHC time, so the result is only correct
  within about 2.1 s of capture (see patch 4).


Patch 4/5 net/ice: support per-packet Tx timestamp slots

Error:

- Stale timestamps are returned as valid.

  E810 has no ready register: ice_get_phy_tx_tstamp_ready_e810()
  does "*tstamp_ready = 0xFFFFFFFFFFFFFFFF;". In
  ice_ptp_read_tx_dual_timestamp() the ready-bit test therefore
  never fires, and "if (ret || tstamp == 0)" is the only readiness
  check.

  Nothing clears a slot before it is handed out.
  ice_ptp_alloc_tx_slot() only sets the bitmap bit.
  ice_timesync_enable() does not reset the PHY timestamp memory.
  ice_timesync_disable() clears index 0 only:
  "ice_clear_phy_tstamp(hw, lport, 0);".

  So a slot can still hold a value from a process that exited
  without releasing it, or one the hardware wrote after the
  application timed out and released the slot. The next owner reads
  that value as its own, before its packet is even sent.

  On E822 and ETH56G, release does not touch the PHY at all
  ("if (hw->phy_model == ICE_PHY_E810)"). A timestamp that arrives
  after a timeout leaves the ready bit set for the next owner.

  Fix: in timesync_enable, reset this PF's slot range in PHY memory
  and clear ts_slot_bitmap. In alloc, discard any entry already
  present in the chosen slot.

Warning:

- The legacy index overlaps the allocator. ice_ptp_init_info() sets
  "ad->ptp_tx_index = 0;" on E810, E830 and ETH56G, and base_slot on
  E822. "slot = (uint8_t)rte_ctz64(free_in_range);" therefore hands
  out the legacy slot first.

  A TMST packet without the dynflag uses ad->ptp_tx_index and lands
  in that same slot. On E810, ice_timesync_read_tx_timestamp() now
  also clears that slot after reading it. Reserve the legacy bit in
  ts_slot_bitmap at init.

- "caps->max_slots = 64;" is unconditional, but the allocator limits
  E822 to "64 / ppq" slots per PF. The debug log "all 64 timestamp
  slots busy" is wrong on E822 for the same reason. Compute the
  range once, store it in the adapter, and use it for caps, alloc,
  read and release.

- Read and release only check "slot_id > 63".

  Releasing an unallocated or already-released slot clears a bit
  that another thread may now own, and on E810 zeroes its PHY entry.
  The ethdev header asks PMDs to return -EINVAL in that case.

  On E822, reading a slot outside this PF's range reads another PF's
  quad entry. Per the ice_clear_phy_tstamp_e822() comment, that read
  auto-clears the entry, so this PF steals the other PF's timestamp.

  Check the slot against the PF range and the bitmap.
  rte_atomic_fetch_and_explicit() returns the old value for the
  ownership test.

- "if (ret) return -EAGAIN;" after ice_get_phy_tx_tstamp_ready() and
  ice_read_phy_tstamp() turns sideband and register failures into
  "retry". The caller then polls forever. Return -EIO.

- ice_timesync_tx_slot_alloc() does not check ad->ptp_ena. With
  timesync disabled, or with txpp_ena set (ice_timesync_enable()
  returns 0 without enabling), alloc succeeds and read returns
  -EAGAIN forever.

- The E810 ice_clear_phy_tstamp() call added to the timeout branch
  of ice_timesync_read_tx_timestamp() is unreachable. tstamp_ready
  is all ones on E810, so
  "!(tstamp_ready & BIT_ULL(ad->ptp_tx_index))" is never true. The
  "clear ... after ... a timeout" fix claimed in the commit message
  does not happen.

- No release note entry under "Updated Intel ice driver".

- ice_timesync_tx_slot_alloc() -> ice_get_next_tx_desc_idx() ->
  ice_ptp_alloc_tx_slot() repeats the dev, slot_id and dual NULL
  checks at every level, after ethdev has already validated them.
  ice_get_next_tx_desc_idx is also misnamed; it allocates a slot,
  not a descriptor index. Collapse the layers.

Info:

- get_context_desc() evaluates
  "if (ol_flags & rte_eth_timesync_tx_slot_dynflag)" for every
  packet in the scalar path, before the ice_calc_context_desc() == 0
  early return. That is an exported-variable load (via the GOT in a
  shared build) per packet. Move the lookup into the TMST branch.

- Pre-existing: TSYN is in the "else if" after TSO, so TMST is
  ignored on TSO packets. With slots, the allocated slot then stays
  at -EAGAIN.

- Pre-existing: the vector Tx paths, including the AVX2 and AVX512
  ctx offload paths, never set CI_TX_CTX_DESC_TSYN.

  Tx path selection does not consider ptp_ena, while Rx does
  (ice_set_rx_function). caps still reports PER_PACKET, so a stamped
  packet on a vector path never gets a timestamp.

- Pre-existing: ice_tstamp_convert_32b_64b(hw, ad, 1, ...) writes the
  shared ad->time_hw without synchronization. The slot read adds a
  concurrent writer alongside the Rx path.

- On E810 the legacy read now hits "Failed to read phy timestamp" at
  ERR on every poll until the hardware writes the slot. ptpclient
  and ieee1588fwd both poll in loops, so this floods the log. Log at
  DEBUG and return -EAGAIN.

- "(uint32_t)(uint16_t)idx" double cast. Magic 64 and 63 should be a
  define. Missing space in "RTE_ATOMIC(uint64_t)ts_slot_bitmap;".


Patch 5/5 app/testpmd: add Tx timestamp capabilities command

Warning:

- The "show" command calls rte_eth_timesync_tx_slot_alloc() and
  release(). A show command should not change device state; on E810
  the release zeroes a PHY entry. If a forwarding engine holds every
  slot, the output is "Alloc test: failed (-28)". Drop the
  round-trip.

- testpmd_funcs.rst and the testpmd help text are not updated.

- stamp, read and dynfield_register cannot be reached from testpmd.
  ieee1588fwd is the natural place to use the slot API when the port
  reports PER_PACKET.

Info:

- There is no port_id_is_invalid(res->port_id, ENABLED_WARN) check.
  Print errors with strerror(-ret).

  reply	other threads:[~2026-10-06 14:16 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-17 19:24 [RFC 0/1] ethdev: per-packet Tx timestamp slot management Rajesh Kumar
2026-08-17 19:24 ` [RFC 1/1] ethdev: add per-packet Tx timestamp slot APIs Rajesh Kumar
2026-08-20  4:51   ` Naga Harish K, S V
2026-08-18  2:23 ` [RFC 0/1] ethdev: per-packet Tx timestamp slot management Stephen Hemminger
2026-08-20  4:41 ` Naga Harish K, S V
2026-08-27 11:09   ` Kumar, Rajesh
2026-08-27 12:13 ` [RFC PATCH v2 0/1] ethdev: add Tx timestamp slot APIs Rajesh Kumar
2026-08-27 12:13   ` [RFC PATCH v3 1/1] ethdev: add Tx timestamp slot management APIs Rajesh Kumar
2026-08-27 12:18 ` [RFC PATCH v3 0/1] ethdev: add Tx timestamp slot APIs Rajesh Kumar
2026-08-27 12:21 ` Rajesh Kumar
2026-08-27 12:21   ` [RFC PATCH v3 1/1] ethdev: add Tx timestamp slot management APIs Rajesh Kumar
2026-08-27 21:45     ` Stephen Hemminger
2026-09-02  5:51     ` [RFC PATCH v4 0/3] ethdev: add Tx timestamp slot APIs Rajesh Kumar
2026-09-02  5:51       ` [RFC PATCH v4 1/3] ethdev: add Tx timestamp slot management APIs Rajesh Kumar
2026-09-02 14:13         ` Stephen Hemminger
2026-09-08  7:25           ` Kumar, Rajesh
2026-09-02  5:51       ` [RFC PATCH v4 2/3] net/ice: support per-packet Tx timestamp slots Rajesh Kumar
2026-09-02  5:51       ` [RFC PATCH v4 3/3] app/testpmd: add Tx timestamp capabilities command Rajesh Kumar
2026-09-08  7:32     ` [RFC PATCH v5 0/5] ethdev: add Tx timestamp slot APIs Rajesh Kumar
2026-09-08  7:32       ` [RFC PATCH v5 1/5] ethdev: add Tx timestamp slot management APIs Rajesh Kumar
2026-09-08  7:32       ` [RFC PATCH v5 2/5] doc: describe ethdev timesync clock and Rx timestamp API Rajesh Kumar
2026-09-08  7:32       ` [RFC PATCH v5 3/5] doc: describe ethdev Tx timestamp slot API Rajesh Kumar
2026-09-08  7:32       ` [RFC PATCH v5 4/5] net/ice: support per-packet Tx timestamp slots Rajesh Kumar
2026-09-08  7:32       ` [RFC PATCH v5 5/5] app/testpmd: add Tx timestamp capabilities command Rajesh Kumar
2026-09-21 15:00       ` [RFC PATCH v5 0/5] ethdev: add Tx timestamp slot APIs Kumar, Rajesh
2026-10-06 10:28         ` Kumar, Rajesh
2026-10-06 14:16           ` Stephen Hemminger [this message]
2026-08-27 12:34 ` [RFC PATCH v2 0/1] " Rajesh Kumar
2026-08-27 12:34   ` [RFC PATCH v2 1/1] ethdev: add Tx timestamp slot management APIs Rajesh Kumar
2026-09-02 14:16   ` [RFC PATCH v2 0/1] ethdev: add Tx timestamp slot APIs Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261006071648.4bb37e4f@phoenix.local \
    --to=stephen@networkplumber.org \
    --cc=aman.deep.singh@intel.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=rajesh3.kumar@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.