[PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Tariq Toukan]

Hi,

This series by Carolina adds support in ptp and usage in mlx5 for
exposing the raw free-running cycle counter of PTP hardware clocks.

Find detailed description by Carolina below [1].

Regards,
Tariq


[1]
This patch series introduces support for exposing the raw free-running
cycle counter of PTP hardware clocks. Some telemetry and low-level
logging use cycle counter timestamps rather than nanoseconds.
Currently, there is no generic interface to correlate these raw values
with system time.

To address this, the series introduces two new ioctl commands that
allow userspace to query the device's raw cycle counter together with
host time:

 - PTP_SYS_OFFSET_PRECISE_CYCLES

 - PTP_SYS_OFFSET_EXTENDED_CYCLES

These commands work like their existing counterparts but return the
device timestamp in cycle units instead of real-time nanoseconds.

This can also be useful in the XDP fast path: if a driver inserts the
raw cycle value into metadata instead of a real-time timestamp, it can
avoid the overhead of converting cycles to time in the kernel. Then
userspace can resolve the cycle-to-time mapping using this ioctl when
needed.

Adds the new PTP ioctls and integrates support in ptp_ioctl():
- ptp: Add ioctl commands to expose raw cycle counter values

Support for exposing raw cycles in mlx5:
- net/mlx5: Extract MTCTR register read logic into helper function
- net/mlx5: Support getcyclesx and getcrosscycles


Carolina Jubran (3):
  ptp: Add ioctl commands to expose raw cycle counter values
  net/mlx5: Extract MTCTR register read logic into helper function
  net/mlx5: Support getcyclesx and getcrosscycles

 .../ethernet/mellanox/mlx5/core/lib/clock.c   | 113 ++++++++++++++++--
 drivers/ptp/ptp_chardev.c                     |  34 ++++--
 include/uapi/linux/ptp_clock.h                |   4 +
 3 files changed, 130 insertions(+), 21 deletions(-)


base-commit: 06baf9bfa6ca8db7d5f32e12e27d1dc1b7cb3a8a
-- 
2.31.1

[PATCH net-next 1/3] ptp: Add ioctl commands to expose raw cycle counter values

[Tariq Toukan]

From: Carolina Jubran <cjubran@nvidia.com>

Introduce two new ioctl commands, PTP_SYS_OFFSET_PRECISE_CYCLES and
PTP_SYS_OFFSET_EXTENDED_CYCLES, to allow user space to access the
raw free-running cycle counter from PTP devices.

These ioctls are variants of the existing PRECISE and EXTENDED
offset queries, but instead of returning device time in realtime,
they return the raw cycle counter value.

Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 drivers/ptp/ptp_chardev.c      | 34 ++++++++++++++++++++++++++--------
 include/uapi/linux/ptp_clock.h |  4 ++++
 2 files changed, 30 insertions(+), 8 deletions(-)

diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index 4ca5a464a46a..e9719f365aab 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -285,17 +285,21 @@ static long ptp_enable_pps(struct ptp_clock *ptp, bool enable)
 		return ops->enable(ops, &req, enable);
 }
 
-static long ptp_sys_offset_precise(struct ptp_clock *ptp, void __user *arg)
+typedef int (*ptp_crosststamp_fn)(struct ptp_clock_info *,
+				  struct system_device_crosststamp *);
+
+static long ptp_sys_offset_precise(struct ptp_clock *ptp, void __user *arg,
+				   ptp_crosststamp_fn crosststamp_fn)
 {
 	struct ptp_sys_offset_precise precise_offset;
 	struct system_device_crosststamp xtstamp;
 	struct timespec64 ts;
 	int err;
 
-	if (!ptp->info->getcrosststamp)
+	if (!crosststamp_fn)
 		return -EOPNOTSUPP;
 
-	err = ptp->info->getcrosststamp(ptp->info, &xtstamp);
+	err = crosststamp_fn(ptp->info, &xtstamp);
 	if (err)
 		return err;
 
@@ -313,12 +317,17 @@ static long ptp_sys_offset_precise(struct ptp_clock *ptp, void __user *arg)
 	return copy_to_user(arg, &precise_offset, sizeof(precise_offset)) ? -EFAULT : 0;
 }
 
-static long ptp_sys_offset_extended(struct ptp_clock *ptp, void __user *arg)
+typedef int (*ptp_gettimex_fn)(struct ptp_clock_info *,
+			       struct timespec64 *,
+			       struct ptp_system_timestamp *);
+
+static long ptp_sys_offset_extended(struct ptp_clock *ptp, void __user *arg,
+				    ptp_gettimex_fn gettimex_fn)
 {
 	struct ptp_sys_offset_extended *extoff __free(kfree) = NULL;
 	struct ptp_system_timestamp sts;
 
-	if (!ptp->info->gettimex64)
+	if (!gettimex_fn)
 		return -EOPNOTSUPP;
 
 	extoff = memdup_user(arg, sizeof(*extoff));
@@ -346,7 +355,7 @@ static long ptp_sys_offset_extended(struct ptp_clock *ptp, void __user *arg)
 		struct timespec64 ts;
 		int err;
 
-		err = ptp->info->gettimex64(ptp->info, &ts, &sts);
+		err = gettimex_fn(ptp->info, &ts, &sts);
 		if (err)
 			return err;
 
@@ -497,11 +506,13 @@ long ptp_ioctl(struct posix_clock_context *pccontext, unsigned int cmd,
 
 	case PTP_SYS_OFFSET_PRECISE:
 	case PTP_SYS_OFFSET_PRECISE2:
-		return ptp_sys_offset_precise(ptp, argptr);
+		return ptp_sys_offset_precise(ptp, argptr,
+					      ptp->info->getcrosststamp);
 
 	case PTP_SYS_OFFSET_EXTENDED:
 	case PTP_SYS_OFFSET_EXTENDED2:
-		return ptp_sys_offset_extended(ptp, argptr);
+		return ptp_sys_offset_extended(ptp, argptr,
+					       ptp->info->gettimex64);
 
 	case PTP_SYS_OFFSET:
 	case PTP_SYS_OFFSET2:
@@ -523,6 +534,13 @@ long ptp_ioctl(struct posix_clock_context *pccontext, unsigned int cmd,
 	case PTP_MASK_EN_SINGLE:
 		return ptp_mask_en_single(pccontext->private_clkdata, argptr);
 
+	case PTP_SYS_OFFSET_PRECISE_CYCLES:
+		return ptp_sys_offset_precise(ptp, argptr,
+					      ptp->info->getcrosscycles);
+
+	case PTP_SYS_OFFSET_EXTENDED_CYCLES:
+		return ptp_sys_offset_extended(ptp, argptr,
+					       ptp->info->getcyclesx64);
 	default:
 		return -ENOTTY;
 	}
diff --git a/include/uapi/linux/ptp_clock.h b/include/uapi/linux/ptp_clock.h
index 18eefa6d93d6..65f187b5f0d0 100644
--- a/include/uapi/linux/ptp_clock.h
+++ b/include/uapi/linux/ptp_clock.h
@@ -245,6 +245,10 @@ struct ptp_pin_desc {
 	_IOWR(PTP_CLK_MAGIC, 18, struct ptp_sys_offset_extended)
 #define PTP_MASK_CLEAR_ALL  _IO(PTP_CLK_MAGIC, 19)
 #define PTP_MASK_EN_SINGLE  _IOW(PTP_CLK_MAGIC, 20, unsigned int)
+#define PTP_SYS_OFFSET_PRECISE_CYCLES \
+	_IOWR(PTP_CLK_MAGIC, 21, struct ptp_sys_offset_precise)
+#define PTP_SYS_OFFSET_EXTENDED_CYCLES \
+	_IOWR(PTP_CLK_MAGIC, 22, struct ptp_sys_offset_extended)
 
 struct ptp_extts_event {
 	struct ptp_clock_time t; /* Time event occurred. */
-- 
2.31.1

[PATCH net-next 2/3] net/mlx5: Extract MTCTR register read logic into helper function

[Tariq Toukan]

From: Carolina Jubran <cjubran@nvidia.com>

Refactor the MTCTR register reading logic into a dedicated helper to
lay the groundwork for the next patch.

Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../ethernet/mellanox/mlx5/core/lib/clock.c   | 39 +++++++++++++------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
index cec18efadc73..b1e2deeefc0c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
@@ -247,27 +247,24 @@ static bool mlx5_is_ptm_source_time_available(struct mlx5_core_dev *dev)
 	return !!MLX5_GET(mtptm_reg, out, psta);
 }
 
-static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
-				     struct system_counterval_t *sys_counterval,
-				     void *ctx)
+static int mlx5_mtctr_read(struct mlx5_core_dev *mdev,
+			   bool real_time_mode,
+			   struct system_counterval_t *sys_counterval,
+			   u64 *device)
 {
 	u32 out[MLX5_ST_SZ_DW(mtctr_reg)] = {0};
 	u32 in[MLX5_ST_SZ_DW(mtctr_reg)] = {0};
-	struct mlx5_core_dev *mdev = ctx;
-	bool real_time_mode;
-	u64 host, device;
+	u64 host;
 	int err;
 
-	real_time_mode = mlx5_real_time_mode(mdev);
-
 	MLX5_SET(mtctr_reg, in, first_clock_timestamp_request,
 		 MLX5_MTCTR_REQUEST_PTM_ROOT_CLOCK);
 	MLX5_SET(mtctr_reg, in, second_clock_timestamp_request,
 		 real_time_mode ? MLX5_MTCTR_REQUEST_REAL_TIME_CLOCK :
-		 MLX5_MTCTR_REQUEST_FREE_RUNNING_COUNTER);
+				  MLX5_MTCTR_REQUEST_FREE_RUNNING_COUNTER);
 
-	err = mlx5_core_access_reg(mdev, in, sizeof(in), out, sizeof(out), MLX5_REG_MTCTR,
-				   0, 0);
+	err = mlx5_core_access_reg(mdev, in, sizeof(in), out, sizeof(out),
+				   MLX5_REG_MTCTR, 0, 0);
 	if (err)
 		return err;
 
@@ -281,8 +278,26 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
 			.cs_id = CSID_X86_ART,
 			.use_nsecs = true,
 	};
+	*device = MLX5_GET64(mtctr_reg, out, second_clock_timestamp);
+
+	return 0;
+}
+
+static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
+				     struct system_counterval_t *sys_counterval,
+				     void *ctx)
+{
+	struct mlx5_core_dev *mdev = ctx;
+	bool real_time_mode;
+	u64 device;
+	int err;
+
+	real_time_mode = mlx5_real_time_mode(mdev);
+
+	err = mlx5_mtctr_read(mdev, real_time_mode, sys_counterval, &device);
+	if (err)
+		return err;
 
-	device = MLX5_GET64(mtctr_reg, out, second_clock_timestamp);
 	if (real_time_mode)
 		*device_time = ns_to_ktime(REAL_TIME_TO_NS(device >> 32, device & U32_MAX));
 	else
-- 
2.31.1

[PATCH net-next 3/3] net/mlx5: Support getcyclesx and getcrosscycles

[Tariq Toukan]

From: Carolina Jubran <cjubran@nvidia.com>

Implement the getcyclesx64 and getcrosscycles callbacks in ptp_info to
expose the device’s raw free-running counter.

Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../ethernet/mellanox/mlx5/core/lib/clock.c   | 74 ++++++++++++++++++-
 1 file changed, 73 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
index b1e2deeefc0c..2f75726674a9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
@@ -306,6 +306,23 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
 	return 0;
 }
 
+static int
+mlx5_mtctr_syncdevicecyclestime(ktime_t *device_time,
+				struct system_counterval_t *sys_counterval,
+				void *ctx)
+{
+	struct mlx5_core_dev *mdev = ctx;
+	u64 device;
+	int err;
+
+	err = mlx5_mtctr_read(mdev, false, sys_counterval, &device);
+	if (err)
+		return err;
+	*device_time = ns_to_ktime(device);
+
+	return 0;
+}
+
 static int mlx5_ptp_getcrosststamp(struct ptp_clock_info *ptp,
 				   struct system_device_crosststamp *cts)
 {
@@ -330,6 +347,32 @@ static int mlx5_ptp_getcrosststamp(struct ptp_clock_info *ptp,
 	mlx5_clock_unlock(clock);
 	return err;
 }
+
+static int mlx5_ptp_getcrosscycles(struct ptp_clock_info *ptp,
+				   struct system_device_crosststamp *cts)
+{
+	struct mlx5_clock *clock =
+		container_of(ptp, struct mlx5_clock, ptp_info);
+	struct system_time_snapshot history_begin = {0};
+	struct mlx5_core_dev *mdev;
+	int err;
+
+	mlx5_clock_lock(clock);
+	mdev = mlx5_clock_mdev_get(clock);
+
+	if (!mlx5_is_ptm_source_time_available(mdev)) {
+		err = -EBUSY;
+		goto unlock;
+	}
+
+	ktime_get_snapshot(&history_begin);
+
+	err = get_device_system_crosststamp(mlx5_mtctr_syncdevicecyclestime,
+					    mdev, &history_begin, cts);
+unlock:
+	mlx5_clock_unlock(clock);
+	return err;
+}
 #endif /* CONFIG_X86 */
 
 static u64 mlx5_read_time(struct mlx5_core_dev *dev,
@@ -528,6 +571,24 @@ static int mlx5_ptp_gettimex(struct ptp_clock_info *ptp, struct timespec64 *ts,
 	return 0;
 }
 
+static int mlx5_ptp_getcyclesx(struct ptp_clock_info *ptp,
+			       struct timespec64 *ts,
+			       struct ptp_system_timestamp *sts)
+{
+	struct mlx5_clock *clock = container_of(ptp, struct mlx5_clock,
+						ptp_info);
+	struct mlx5_core_dev *mdev;
+	u64 cycles;
+
+	mlx5_clock_lock(clock);
+	mdev = mlx5_clock_mdev_get(clock);
+
+	cycles = mlx5_read_time(mdev, sts, false);
+	*ts = ns_to_timespec64(cycles);
+	mlx5_clock_unlock(clock);
+	return 0;
+}
+
 static int mlx5_ptp_adjtime_real_time(struct mlx5_core_dev *mdev, s64 delta)
 {
 	u32 in[MLX5_ST_SZ_DW(mtutc_reg)] = {};
@@ -1244,6 +1305,7 @@ static void mlx5_init_timer_max_freq_adjustment(struct mlx5_core_dev *mdev)
 static void mlx5_init_timer_clock(struct mlx5_core_dev *mdev)
 {
 	struct mlx5_clock *clock = mdev->clock;
+	bool expose_cycles;
 
 	/* Configure the PHC */
 	clock->ptp_info = mlx5_ptp_clock_info;
@@ -1251,12 +1313,22 @@ static void mlx5_init_timer_clock(struct mlx5_core_dev *mdev)
 	if (MLX5_CAP_MCAM_REG(mdev, mtutc))
 		mlx5_init_timer_max_freq_adjustment(mdev);
 
+	expose_cycles = !MLX5_CAP_GEN(mdev, disciplined_fr_counter) ||
+			!mlx5_real_time_mode(mdev);
+
 #ifdef CONFIG_X86
 	if (MLX5_CAP_MCAM_REG3(mdev, mtptm) &&
-	    MLX5_CAP_MCAM_REG3(mdev, mtctr) && boot_cpu_has(X86_FEATURE_ART))
+	    MLX5_CAP_MCAM_REG3(mdev, mtctr) && boot_cpu_has(X86_FEATURE_ART)) {
 		clock->ptp_info.getcrosststamp = mlx5_ptp_getcrosststamp;
+		if (expose_cycles)
+			clock->ptp_info.getcrosscycles =
+				mlx5_ptp_getcrosscycles;
+	}
 #endif /* CONFIG_X86 */
 
+	if (expose_cycles)
+		clock->ptp_info.getcyclesx64 = mlx5_ptp_getcyclesx;
+
 	mlx5_timecounter_init(mdev);
 	mlx5_init_clock_info(mdev);
 	mlx5_init_overflow_period(mdev);
-- 
2.31.1

Re: [PATCH net-next 3/3] net/mlx5: Support getcyclesx and getcrosscycles

[Paolo Abeni]

On 7/15/25 7:15 AM, Tariq Toukan wrote:
> From: Carolina Jubran <cjubran@nvidia.com>
> 
> Implement the getcyclesx64 and getcrosscycles callbacks in ptp_info to
> expose the device’s raw free-running counter.
> 
> Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  .../ethernet/mellanox/mlx5/core/lib/clock.c   | 74 ++++++++++++++++++-
>  1 file changed, 73 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
> index b1e2deeefc0c..2f75726674a9 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
> @@ -306,6 +306,23 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
>  	return 0;
>  }
>  
> +static int
> +mlx5_mtctr_syncdevicecyclestime(ktime_t *device_time,
> +				struct system_counterval_t *sys_counterval,
> +				void *ctx)
> +{
> +	struct mlx5_core_dev *mdev = ctx;
> +	u64 device;
> +	int err;
> +
> +	err = mlx5_mtctr_read(mdev, false, sys_counterval, &device);
> +	if (err)
> +		return err;
> +	*device_time = ns_to_ktime(device);

If the goal is providing a raw cycle counter, why still using a timespec
to the user space? A plain u64 would possibly be less ambiguous.

/P

Re: [PATCH net-next 3/3] net/mlx5: Support getcyclesx and getcrosscycles

[Carolina Jubran]

On 17/07/2025 13:55, Paolo Abeni wrote:
> On 7/15/25 7:15 AM, Tariq Toukan wrote:
>> From: Carolina Jubran <cjubran@nvidia.com>
>>
>> Implement the getcyclesx64 and getcrosscycles callbacks in ptp_info to
>> expose the device’s raw free-running counter.
>>
>> Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
>> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
>> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
>> ---
>>   .../ethernet/mellanox/mlx5/core/lib/clock.c   | 74 ++++++++++++++++++-
>>   1 file changed, 73 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>> index b1e2deeefc0c..2f75726674a9 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>> @@ -306,6 +306,23 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
>>   	return 0;
>>   }
>>   
>> +static int
>> +mlx5_mtctr_syncdevicecyclestime(ktime_t *device_time,
>> +				struct system_counterval_t *sys_counterval,
>> +				void *ctx)
>> +{
>> +	struct mlx5_core_dev *mdev = ctx;
>> +	u64 device;
>> +	int err;
>> +
>> +	err = mlx5_mtctr_read(mdev, false, sys_counterval, &device);
>> +	if (err)
>> +		return err;
>> +	*device_time = ns_to_ktime(device);
> 
> If the goal is providing a raw cycle counter, why still using a timespec
> to the user space? A plain u64 would possibly be less ambiguous.
> 
> /P
> 

getcycles64 and getcrosscycles already return the cycle counter in a 
timespec64/ktime format, so I kept the new ioctls consistent with them.

Re: [PATCH net-next 3/3] net/mlx5: Support getcyclesx and getcrosscycles

[Paolo Abeni]

On 7/17/25 5:56 PM, Carolina Jubran wrote:
> On 17/07/2025 13:55, Paolo Abeni wrote:
>> On 7/15/25 7:15 AM, Tariq Toukan wrote:
>>> From: Carolina Jubran <cjubran@nvidia.com>
>>>
>>> Implement the getcyclesx64 and getcrosscycles callbacks in ptp_info to
>>> expose the device’s raw free-running counter.
>>>
>>> Signed-off-by: Carolina Jubran <cjubran@nvidia.com>
>>> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
>>> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
>>> ---
>>>   .../ethernet/mellanox/mlx5/core/lib/clock.c   | 74 ++++++++++++++++++-
>>>   1 file changed, 73 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>>> index b1e2deeefc0c..2f75726674a9 100644
>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
>>> @@ -306,6 +306,23 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,
>>>   	return 0;
>>>   }
>>>   
>>> +static int
>>> +mlx5_mtctr_syncdevicecyclestime(ktime_t *device_time,
>>> +				struct system_counterval_t *sys_counterval,
>>> +				void *ctx)
>>> +{
>>> +	struct mlx5_core_dev *mdev = ctx;
>>> +	u64 device;
>>> +	int err;
>>> +
>>> +	err = mlx5_mtctr_read(mdev, false, sys_counterval, &device);
>>> +	if (err)
>>> +		return err;
>>> +	*device_time = ns_to_ktime(device);
>>
>> If the goal is providing a raw cycle counter, why still using a timespec
>> to the user space? A plain u64 would possibly be less ambiguous.
>>
>> /P
>>
> 
> getcycles64 and getcrosscycles already return the cycle counter in a 
> timespec64/ktime format, so I kept the new ioctls consistent with them.

Ah, sorry I missed that context. Looks good to me than.

Thanks,

Paolo

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Jakub Kicinski]

On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
> This patch series introduces support for exposing the raw free-running
> cycle counter of PTP hardware clocks. Some telemetry and low-level
> logging use cycle counter timestamps rather than nanoseconds.
> Currently, there is no generic interface to correlate these raw values
> with system time.
> 
> To address this, the series introduces two new ioctl commands that
> allow userspace to query the device's raw cycle counter together with
> host time:
> 
>  - PTP_SYS_OFFSET_PRECISE_CYCLES
> 
>  - PTP_SYS_OFFSET_EXTENDED_CYCLES
> 
> These commands work like their existing counterparts but return the
> device timestamp in cycle units instead of real-time nanoseconds.
> 
> This can also be useful in the XDP fast path: if a driver inserts the
> raw cycle value into metadata instead of a real-time timestamp, it can
> avoid the overhead of converting cycles to time in the kernel. Then
> userspace can resolve the cycle-to-time mapping using this ioctl when
> needed.
> 
> Adds the new PTP ioctls and integrates support in ptp_ioctl():
> - ptp: Add ioctl commands to expose raw cycle counter values
> 
> Support for exposing raw cycles in mlx5:
> - net/mlx5: Extract MTCTR register read logic into helper function
> - net/mlx5: Support getcyclesx and getcrosscycles

It'd be great to an Ack from Thomas or Richard on this (or failing that
at least other vendors?) Seems like we have a number of parallel
efforts to extend the PTP uAPI, I'm not sure how they all square
against each other, TBH.

Full thread for folks I CCed in:
https://lore.kernel.org/all/1752556533-39218-1-git-send-email-tariqt@nvidia.com/

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Thomas Gleixner]

On Fri, Jul 18 2025 at 16:29, Jakub Kicinski wrote:
> On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
>> This patch series introduces support for exposing the raw free-running
>> cycle counter of PTP hardware clocks. Some telemetry and low-level
>> logging use cycle counter timestamps rather than nanoseconds.
>> Currently, there is no generic interface to correlate these raw values
>> with system time.
>> 
>> To address this, the series introduces two new ioctl commands that
>> allow userspace to query the device's raw cycle counter together with
>> host time:
>> 
>>  - PTP_SYS_OFFSET_PRECISE_CYCLES
>> 
>>  - PTP_SYS_OFFSET_EXTENDED_CYCLES
>> 
>> These commands work like their existing counterparts but return the
>> device timestamp in cycle units instead of real-time nanoseconds.
>> 
>> This can also be useful in the XDP fast path: if a driver inserts the
>> raw cycle value into metadata instead of a real-time timestamp, it can
>> avoid the overhead of converting cycles to time in the kernel. Then
>> userspace can resolve the cycle-to-time mapping using this ioctl when
>> needed.
>> 
>> Adds the new PTP ioctls and integrates support in ptp_ioctl():
>> - ptp: Add ioctl commands to expose raw cycle counter values
>> 
>> Support for exposing raw cycles in mlx5:
>> - net/mlx5: Extract MTCTR register read logic into helper function
>> - net/mlx5: Support getcyclesx and getcrosscycles
>
> It'd be great to an Ack from Thomas or Richard on this (or failing that
> at least other vendors?) Seems like we have a number of parallel
> efforts to extend the PTP uAPI, I'm not sure how they all square
> against each other, TBH.

I don't see a conflict vs. the aux clock support. These are orthogonal
issues and from a conceptual point it makes sense to me to expose the
raw cycles for the purposes Tariq described.

Thanks,

        tglx

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Jacob Keller]

[-- Attachment #1.1: Type: text/plain, Size: 3097 bytes --]



On 7/18/2025 4:29 PM, Jakub Kicinski wrote:
> On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
>> This patch series introduces support for exposing the raw free-running
>> cycle counter of PTP hardware clocks. Some telemetry and low-level
>> logging use cycle counter timestamps rather than nanoseconds.
>> Currently, there is no generic interface to correlate these raw values
>> with system time.
>>
>> To address this, the series introduces two new ioctl commands that
>> allow userspace to query the device's raw cycle counter together with
>> host time:
>>
>>  - PTP_SYS_OFFSET_PRECISE_CYCLES
>>
>>  - PTP_SYS_OFFSET_EXTENDED_CYCLES
>>
>> These commands work like their existing counterparts but return the
>> device timestamp in cycle units instead of real-time nanoseconds.
>>
>> This can also be useful in the XDP fast path: if a driver inserts the
>> raw cycle value into metadata instead of a real-time timestamp, it can
>> avoid the overhead of converting cycles to time in the kernel. Then
>> userspace can resolve the cycle-to-time mapping using this ioctl when
>> needed.
>>
>> Adds the new PTP ioctls and integrates support in ptp_ioctl():
>> - ptp: Add ioctl commands to expose raw cycle counter values
>>
>> Support for exposing raw cycles in mlx5:
>> - net/mlx5: Extract MTCTR register read logic into helper function
>> - net/mlx5: Support getcyclesx and getcrosscycles
> 
> It'd be great to an Ack from Thomas or Richard on this (or failing that
> at least other vendors?) Seems like we have a number of parallel
> efforts to extend the PTP uAPI, I'm not sure how they all square
> against each other, TBH.
> 
> Full thread for folks I CCed in:
> https://lore.kernel.org/all/1752556533-39218-1-git-send-email-tariqt@nvidia.com/
> 

I agree with Jakub about the need to properly explain the use cases and
goals in the commit and cover letter. AFAIK there are no current public
APIs for reporting cycles to userspace, so this really only makes sense
with something like DPDK. Even the XDP related helpers expect nanosecond
units now. Its unclear if we will need other parts of the APIs to also
handle cycles, or if simple ability to get the current cycles is sufficient.

The API also doesn't directly provide a way to query the expected or
nominal relationship between cycles and clock time.

If you try to just use PTP_SYS_OFFSET_EXTENDED_CYCLES to compare a
cycles value to a clock value to adjust a timestamp, that requires that
some other process is keeping CLOCK_REALTIME and the PHC clock
synchronized. When handled within the driver, the software typically has
an assumption about the relationship based on expected frequencies.
Thus, a conversion from cycles to time uses this relationship.

You don't appear to expose that relationship through the API, which
means you can only infer it either by knowing the device, or by assuming
CLOCK_REALTIME is already synchronized with the PHC?

I guess userspace could also simply build its own equivalent of the
struct timecounter using this API.. hmm.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Carolina Jubran]

On 30/07/2025 2:33, Jacob Keller wrote:
> 
> 
> On 7/18/2025 4:29 PM, Jakub Kicinski wrote:
>> On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
>>> This patch series introduces support for exposing the raw free-running
>>> cycle counter of PTP hardware clocks. Some telemetry and low-level
>>> logging use cycle counter timestamps rather than nanoseconds.
>>> Currently, there is no generic interface to correlate these raw values
>>> with system time.
>>>
>>> To address this, the series introduces two new ioctl commands that
>>> allow userspace to query the device's raw cycle counter together with
>>> host time:
>>>
>>>   - PTP_SYS_OFFSET_PRECISE_CYCLES
>>>
>>>   - PTP_SYS_OFFSET_EXTENDED_CYCLES
>>>
>>> These commands work like their existing counterparts but return the
>>> device timestamp in cycle units instead of real-time nanoseconds.
>>>
>>> This can also be useful in the XDP fast path: if a driver inserts the
>>> raw cycle value into metadata instead of a real-time timestamp, it can
>>> avoid the overhead of converting cycles to time in the kernel. Then
>>> userspace can resolve the cycle-to-time mapping using this ioctl when
>>> needed.
>>>
>>> Adds the new PTP ioctls and integrates support in ptp_ioctl():
>>> - ptp: Add ioctl commands to expose raw cycle counter values
>>>
>>> Support for exposing raw cycles in mlx5:
>>> - net/mlx5: Extract MTCTR register read logic into helper function
>>> - net/mlx5: Support getcyclesx and getcrosscycles
>>
>> It'd be great to an Ack from Thomas or Richard on this (or failing that
>> at least other vendors?) Seems like we have a number of parallel
>> efforts to extend the PTP uAPI, I'm not sure how they all square
>> against each other, TBH.
>>
>> Full thread for folks I CCed in:
>> https://lore.kernel.org/all/1752556533-39218-1-git-send-email-tariqt@nvidia.com/
>>
> 
> I agree with Jakub about the need to properly explain the use cases and
> goals in the commit and cover letter. AFAIK there are no current public
> APIs for reporting cycles to userspace, so this really only makes sense
> with something like DPDK. Even the XDP related helpers expect nanosecond
> units now. Its unclear if we will need other parts of the APIs to also
> handle cycles, or if simple ability to get the current cycles is sufficient.
> 
> The API also doesn't directly provide a way to query the expected or
> nominal relationship between cycles and clock time.
> 
> If you try to just use PTP_SYS_OFFSET_EXTENDED_CYCLES to compare a
> cycles value to a clock value to adjust a timestamp, that requires that
> some other process is keeping CLOCK_REALTIME and the PHC clock
> synchronized. When handled within the driver, the software typically has
> an assumption about the relationship based on expected frequencies.
> Thus, a conversion from cycles to time uses this relationship.
> 
> You don't appear to expose that relationship through the API, which
> means you can only infer it either by knowing the device, or by assuming
> CLOCK_REALTIME is already synchronized with the PHC?
> 
> I guess userspace could also simply build its own equivalent of the
> struct timecounter using this API.. hmm.

Hi Jacob,

You’re right I’m not trying to reason about the nominal frequency.
The goal is to collect (cycle, system time) pairs and use them to 
correlate raw device timestamps with host time. This doesn’t require the 
PHC to be synchronized to CLOCK_REALTIME, but it does assume the user 
can estimate the drift or nominal frequency from the ioctl data.

I’ll clarify this in v2.

Thanks,
Carolina

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Jakub Kicinski]

On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
> This patch series introduces support for exposing the raw free-running
> cycle counter of PTP hardware clocks. 

Could you say more about use cases? I realized when massaging the cover
letter to apply the series that all the use cases are vague and
hypothetical.

> Some telemetry and low-level logging use cycle counter timestamps
> rather than nanoseconds.

What is that "some telemetry"?

> Currently, there is no generic interface to
> correlate these raw values with system time.
> 
> To address this, the series introduces two new ioctl commands that
> allow userspace to query the device's raw cycle counter together with
> host time:
> 
>  - PTP_SYS_OFFSET_PRECISE_CYCLES
> 
>  - PTP_SYS_OFFSET_EXTENDED_CYCLES
> 
> These commands work like their existing counterparts but return the
> device timestamp in cycle units instead of real-time nanoseconds.
> 
> This can also be useful in the XDP fast path: if a driver inserts the
> raw cycle value into metadata instead of a real-time timestamp, it can
> avoid the overhead of converting cycles to time in the kernel. Then
> userspace can resolve the cycle-to-time mapping using this ioctl when
> needed.

There is no API to achieve that today, right? The XDP access helpers
are supposed to return converted time. Are you planning to add new
callbacks?

If there are solid networking use cases for this I'd prefer we fully
iron them out before merging this uAPI. If there are RDMA use cases
please spell them out in more detail.
-- 
pw-bot: cr

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Carolina Jubran]

On 22/07/2025 3:09, Jakub Kicinski wrote:
> On Tue, 15 Jul 2025 08:15:30 +0300 Tariq Toukan wrote:
>> This patch series introduces support for exposing the raw free-running
>> cycle counter of PTP hardware clocks.
> 
> Could you say more about use cases? I realized when massaging the cover
> letter to apply the series that all the use cases are vague and
> hypothetical.
> 
>> Some telemetry and low-level logging use cycle counter timestamps
>> rather than nanoseconds.
> 
> What is that "some telemetry"?
> 
>> Currently, there is no generic interface to
>> correlate these raw values with system time.
>>
>> To address this, the series introduces two new ioctl commands that
>> allow userspace to query the device's raw cycle counter together with
>> host time:
>>
>>   - PTP_SYS_OFFSET_PRECISE_CYCLES
>>
>>   - PTP_SYS_OFFSET_EXTENDED_CYCLES
>>
>> These commands work like their existing counterparts but return the
>> device timestamp in cycle units instead of real-time nanoseconds.
>>
>> This can also be useful in the XDP fast path: if a driver inserts the
>> raw cycle value into metadata instead of a real-time timestamp, it can
>> avoid the overhead of converting cycles to time in the kernel. Then
>> userspace can resolve the cycle-to-time mapping using this ioctl when
>> needed.
> 
> There is no API to achieve that today, right? The XDP access helpers
> are supposed to return converted time. Are you planning to add new
> callbacks?
>  > If there are solid networking use cases for this I'd prefer we fully
> iron them out before merging this uAPI. If there are RDMA use cases
> please spell them out in more detail.

Hi Jakub

Thanks for the feedback.

One concrete use case is monitoring the frequency stability of the 
device clock in FreeRunning mode. User space can periodically sample the 
(cycle, time) pairs returned by the new ioctl to estimate the clock’s 
frequency and detect anomalies, for example, drift caused by temperature 
changes. This is especially useful in holdover scenarios.

Another practical case is with DPDK. When the hardware is in FreeRunning 
mode, the CQE contains raw cycle counter values. DPDK returns these 
values directly to user space without converting them. The new ioctl 
provides a generic and consistent way to translate those raw values to 
host time.

As for XDP, you’re right that it doesn’t expose raw cycles today. The 
point here is more future-looking: if drivers ever choose to emit raw 
cycles into metadata for performance, this API gives user space a clean 
way to interpret those timestamps.

Carolina

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Jakub Kicinski]

On Tue, 29 Jul 2025 09:57:13 +0300 Carolina Jubran wrote:
> One concrete use case is monitoring the frequency stability of the 
> device clock in FreeRunning mode. User space can periodically sample the 
> (cycle, time) pairs returned by the new ioctl to estimate the clock’s 
> frequency and detect anomalies, for example, drift caused by temperature 
> changes. This is especially useful in holdover scenarios.

Because the servo running on the host doesn't know the stability?
Seems like your real use case is the one below.

> Another practical case is with DPDK. When the hardware is in FreeRunning 
> mode, the CQE contains raw cycle counter values. DPDK returns these 
> values directly to user space without converting them. The new ioctl 
> provides a generic and consistent way to translate those raw values to 
> host time.
> 
> As for XDP, you’re right that it doesn’t expose raw cycles today. The 
> point here is more future-looking: if drivers ever choose to emit raw 
> cycles into metadata for performance, this API gives user space a clean 
> way to interpret those timestamps.

Got it, I can see how DPDK / kernel bypass may need this.

Please include this justification in the commit message for v2 
and let's see if anyone merges it.

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Carolina Jubran]

On 30/07/2025 1:40, Jakub Kicinski wrote:
> On Tue, 29 Jul 2025 09:57:13 +0300 Carolina Jubran wrote:
>> One concrete use case is monitoring the frequency stability of the
>> device clock in FreeRunning mode. User space can periodically sample the
>> (cycle, time) pairs returned by the new ioctl to estimate the clock’s
>> frequency and detect anomalies, for example, drift caused by temperature
>> changes. This is especially useful in holdover scenarios.
> 
> Because the servo running on the host doesn't know the stability?
> Seems like your real use case is the one below.
> 
>> Another practical case is with DPDK. When the hardware is in FreeRunning
>> mode, the CQE contains raw cycle counter values. DPDK returns these
>> values directly to user space without converting them. The new ioctl
>> provides a generic and consistent way to translate those raw values to
>> host time.
>>
>> As for XDP, you’re right that it doesn’t expose raw cycles today. The
>> point here is more future-looking: if drivers ever choose to emit raw
>> cycles into metadata for performance, this API gives user space a clean
>> way to interpret those timestamps.
> 
> Got it, I can see how DPDK / kernel bypass may need this.
> 
> Please include this justification in the commit message for v2
> and let's see if anyone merges it.

Thanks, I’ll include the DPDK/kernel bypass justification clearly in the 
v2 commit message and cover letter.

Additionally, I wanted to mention another relevant use case that wasn’t 
brought up earlier: fwctl can expose event records tagged with raw cycle 
counter timestamps. When the device is in free-running mode, correlating 
those with host time becomes difficult unless user space has access to 
both cycle and system time snapshots.

Re: [PATCH net-next 0/3] Support exposing raw cycle counters in PTP and mlx5

[Jakub Kicinski]

On Thu, 31 Jul 2025 22:03:02 +0300 Carolina Jubran wrote:
> Additionally, I wanted to mention another relevant use case that wasn’t 
> brought up earlier: fwctl can expose event records tagged with raw cycle 
> counter timestamps. When the device is in free-running mode, correlating 
> those with host time becomes difficult unless user space has access to 
> both cycle and system time snapshots.

Okay, so DPDK and DOCA, got it.