[PATCH v2 0/8] Arm CoreSight: Support AUX pause and resume

[PATCH v2 0/8] Arm CoreSight: Support AUX pause and resume

[Leo Yan]

This series is to enable AUX pause and resume on Arm CoreSight.

The first patch extracts the trace unit controlling operations to two
functions.  These two functions will be used by AUX pause and resume.

Patches 02 and 03 change the ETMv4 driver to prepare callback functions
for AUX pause and resume.

Patch 04 changes the ETM perf layer to support AUX pause and resume in a
perf session.  The patches 05 and 06 offers an extra feature for
updating buffer on AUX pause occasion, which can mitigate the trace data
lose issue.

Patch 07 documents the AUX pause usages with Arm CoreSight.  The last
patch syncs headers between user space and the kernel.

This patch set has been verified on the Hikey960 board and TC platform.
The previous one uses ETR and the later uses TRBE as sink.

It is suggested to disable CPUIdle (add `nohlt` option in Linux command
line) when verifying this series.  ETM and funnel drivers are found
issues during CPU suspend and resume which will be addressed separately.

Changes from v1:
- Added validation function pointers in pause and resume APIs (Mike).


Leo Yan (8):
  coresight: etm4x: Extract the trace unit controlling
  coresight: Introduce pause and resume APIs for source
  coresight: etm4x: Hook pause and resume callbacks
  coresight: perf: Support AUX trace pause and resume
  coresight: etm: Add an attribute for updating buffer
  coresight: perf: Update buffer on AUX pause
  Documentation: coresight: Document AUX pause and resume
  perf cs-etm: Sync kernel coresight-pmu.h header

 .../trace/coresight/coresight-perf.rst        |  50 ++++++
 drivers/hwtracing/coresight/coresight-core.c  |  20 +++
 .../hwtracing/coresight/coresight-etm-perf.c  |  94 +++++++++-
 .../hwtracing/coresight/coresight-etm-perf.h  |   2 +
 .../coresight/coresight-etm4x-core.c          | 166 ++++++++++++------
 drivers/hwtracing/coresight/coresight-etm4x.h |   2 +
 drivers/hwtracing/coresight/coresight-priv.h  |   2 +
 include/linux/coresight-pmu.h                 |   1 +
 include/linux/coresight.h                     |   4 +
 tools/include/linux/coresight-pmu.h           |   1 +
 10 files changed, 289 insertions(+), 53 deletions(-)

-- 
2.34.1

[PATCH v2 1/8] coresight: etm4x: Extract the trace unit controlling

[Leo Yan]

The trace unit is controlled in the ETM hardware enabling and disabling.
The sequential changes for support AUX pause and resume will reuse the
same operations.

Extract the operations in the etm4_{enable|disable}_trace_unit()
functions.  A minor improvement in etm4_enable_trace_unit() is for
returning the timeout error to callers.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../coresight/coresight-etm4x-core.c          | 126 ++++++++++--------
 1 file changed, 74 insertions(+), 52 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index 2c1a60577728..71be566bd117 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -428,6 +428,78 @@ static void etm4_check_arch_features(struct etmv4_drvdata *drvdata,
 }
 #endif /* CONFIG_ETM4X_IMPDEF_FEATURE */
 
+static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata)
+{
+	struct coresight_device *csdev = drvdata->csdev;
+	struct device *etm_dev = &csdev->dev;
+	struct csdev_access *csa = &csdev->access;
+
+	/*
+	 * ETE mandates that the TRCRSR is written to before
+	 * enabling it.
+	 */
+	if (etm4x_is_ete(drvdata))
+		etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
+
+	etm4x_allow_trace(drvdata);
+	/* Enable the trace unit */
+	etm4x_relaxed_write32(csa, 1, TRCPRGCTLR);
+
+	/* Synchronize the register updates for sysreg access */
+	if (!csa->io_mem)
+		isb();
+
+	/* wait for TRCSTATR.IDLE to go back down to '0' */
+	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_IDLE_BIT, 0)) {
+		dev_err(etm_dev,
+			"timeout while waiting for Idle Trace Status\n");
+		return -ETIME;
+	}
+
+	/*
+	 * As recommended by section 4.3.7 ("Synchronization when using the
+	 * memory-mapped interface") of ARM IHI 0064D
+	 */
+	dsb(sy);
+	isb();
+
+	return 0;
+}
+
+static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata)
+{
+	u32 control;
+	struct coresight_device *csdev = drvdata->csdev;
+	struct device *etm_dev = &csdev->dev;
+	struct csdev_access *csa = &csdev->access;
+
+	control = etm4x_relaxed_read32(csa, TRCPRGCTLR);
+
+	/* EN, bit[0] Trace unit enable bit */
+	control &= ~0x1;
+
+	/*
+	 * If the CPU supports v8.4 Trace filter Control,
+	 * set the ETM to trace prohibited region.
+	 */
+	etm4x_prohibit_trace(drvdata);
+	/*
+	 * Make sure everything completes before disabling, as recommended
+	 * by section 7.3.77 ("TRCVICTLR, ViewInst Main Control Register,
+	 * SSTATUS") of ARM IHI 0064D
+	 */
+	dsb(sy);
+	isb();
+	/* Trace synchronization barrier, is a nop if not supported */
+	tsb_csync();
+	etm4x_relaxed_write32(csa, control, TRCPRGCTLR);
+
+	/* wait for TRCSTATR.PMSTABLE to go to '1' */
+	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_PMSTABLE_BIT, 1))
+		dev_err(etm_dev,
+			"timeout while waiting for PM stable Trace Status\n");
+}
+
 static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
 {
 	int i, rc;
@@ -536,33 +608,7 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
 		etm4x_relaxed_write32(csa, trcpdcr | TRCPDCR_PU, TRCPDCR);
 	}
 
-	/*
-	 * ETE mandates that the TRCRSR is written to before
-	 * enabling it.
-	 */
-	if (etm4x_is_ete(drvdata))
-		etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
-
-	etm4x_allow_trace(drvdata);
-	/* Enable the trace unit */
-	etm4x_relaxed_write32(csa, 1, TRCPRGCTLR);
-
-	/* Synchronize the register updates for sysreg access */
-	if (!csa->io_mem)
-		isb();
-
-	/* wait for TRCSTATR.IDLE to go back down to '0' */
-	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_IDLE_BIT, 0))
-		dev_err(etm_dev,
-			"timeout while waiting for Idle Trace Status\n");
-
-	/*
-	 * As recommended by section 4.3.7 ("Synchronization when using the
-	 * memory-mapped interface") of ARM IHI 0064D
-	 */
-	dsb(sy);
-	isb();
-
+	rc = etm4_enable_trace_unit(drvdata);
 done:
 	etm4_cs_lock(drvdata, csa);
 
@@ -906,7 +952,6 @@ static void etm4_disable_hw(void *info)
 	struct etmv4_drvdata *drvdata = info;
 	struct etmv4_config *config = &drvdata->config;
 	struct coresight_device *csdev = drvdata->csdev;
-	struct device *etm_dev = &csdev->dev;
 	struct csdev_access *csa = &csdev->access;
 	int i;
 
@@ -920,31 +965,8 @@ static void etm4_disable_hw(void *info)
 		etm4x_relaxed_write32(csa, control, TRCPDCR);
 	}
 
-	control = etm4x_relaxed_read32(csa, TRCPRGCTLR);
+	etm4_disable_trace_unit(drvdata);
 
-	/* EN, bit[0] Trace unit enable bit */
-	control &= ~0x1;
-
-	/*
-	 * If the CPU supports v8.4 Trace filter Control,
-	 * set the ETM to trace prohibited region.
-	 */
-	etm4x_prohibit_trace(drvdata);
-	/*
-	 * Make sure everything completes before disabling, as recommended
-	 * by section 7.3.77 ("TRCVICTLR, ViewInst Main Control Register,
-	 * SSTATUS") of ARM IHI 0064D
-	 */
-	dsb(sy);
-	isb();
-	/* Trace synchronization barrier, is a nop if not supported */
-	tsb_csync();
-	etm4x_relaxed_write32(csa, control, TRCPRGCTLR);
-
-	/* wait for TRCSTATR.PMSTABLE to go to '1' */
-	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_PMSTABLE_BIT, 1))
-		dev_err(etm_dev,
-			"timeout while waiting for PM stable Trace Status\n");
 	/* read the status of the single shot comparators */
 	for (i = 0; i < drvdata->nr_ss_cmp; i++) {
 		config->ss_status[i] =
-- 
2.34.1

Re: [PATCH v2 1/8] coresight: etm4x: Extract the trace unit controlling

[Suzuki K Poulose]

Hi Leo

On 10/03/2025 10:49, Leo Yan wrote:
> The trace unit is controlled in the ETM hardware enabling and disabling.
> The sequential changes for support AUX pause and resume will reuse the
> same operations.
> 
> Extract the operations in the etm4_{enable|disable}_trace_unit()
> functions.  A minor improvement in etm4_enable_trace_unit() is for
> returning the timeout error to callers.
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>

fyi, this area of code has changes significantly and will cause
conflicts, due to the introduction of synchronization after
each access to the TRCSTATR. Please rebase your next version on
the coresight next branch to avoid the conflict.

The changes as such look good to me.

Suzuki



> ---
>   .../coresight/coresight-etm4x-core.c          | 126 ++++++++++--------
>   1 file changed, 74 insertions(+), 52 deletions(-)
> 
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> index 2c1a60577728..71be566bd117 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> @@ -428,6 +428,78 @@ static void etm4_check_arch_features(struct etmv4_drvdata *drvdata,
>   }
>   #endif /* CONFIG_ETM4X_IMPDEF_FEATURE */
>   
> +static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata)
> +{
> +	struct coresight_device *csdev = drvdata->csdev;
> +	struct device *etm_dev = &csdev->dev;
> +	struct csdev_access *csa = &csdev->access;
> +
> +	/*
> +	 * ETE mandates that the TRCRSR is written to before
> +	 * enabling it.
> +	 */
> +	if (etm4x_is_ete(drvdata))
> +		etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
> +
> +	etm4x_allow_trace(drvdata);
> +	/* Enable the trace unit */
> +	etm4x_relaxed_write32(csa, 1, TRCPRGCTLR);
> +
> +	/* Synchronize the register updates for sysreg access */
> +	if (!csa->io_mem)
> +		isb();
> +
> +	/* wait for TRCSTATR.IDLE to go back down to '0' */
> +	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_IDLE_BIT, 0)) {
> +		dev_err(etm_dev,
> +			"timeout while waiting for Idle Trace Status\n");
> +		return -ETIME;
> +	}
> +
> +	/*
> +	 * As recommended by section 4.3.7 ("Synchronization when using the
> +	 * memory-mapped interface") of ARM IHI 0064D
> +	 */
> +	dsb(sy);
> +	isb();
> +
> +	return 0;
> +}
> +
> +static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata)
> +{
> +	u32 control;
> +	struct coresight_device *csdev = drvdata->csdev;
> +	struct device *etm_dev = &csdev->dev;
> +	struct csdev_access *csa = &csdev->access;
> +
> +	control = etm4x_relaxed_read32(csa, TRCPRGCTLR);
> +
> +	/* EN, bit[0] Trace unit enable bit */
> +	control &= ~0x1;
> +
> +	/*
> +	 * If the CPU supports v8.4 Trace filter Control,
> +	 * set the ETM to trace prohibited region.
> +	 */
> +	etm4x_prohibit_trace(drvdata);
> +	/*
> +	 * Make sure everything completes before disabling, as recommended
> +	 * by section 7.3.77 ("TRCVICTLR, ViewInst Main Control Register,
> +	 * SSTATUS") of ARM IHI 0064D
> +	 */
> +	dsb(sy);
> +	isb();
> +	/* Trace synchronization barrier, is a nop if not supported */
> +	tsb_csync();
> +	etm4x_relaxed_write32(csa, control, TRCPRGCTLR);
> +
> +	/* wait for TRCSTATR.PMSTABLE to go to '1' */
> +	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_PMSTABLE_BIT, 1))
> +		dev_err(etm_dev,
> +			"timeout while waiting for PM stable Trace Status\n");
> +}
> +
>   static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
>   {
>   	int i, rc;
> @@ -536,33 +608,7 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
>   		etm4x_relaxed_write32(csa, trcpdcr | TRCPDCR_PU, TRCPDCR);
>   	}
>   
> -	/*
> -	 * ETE mandates that the TRCRSR is written to before
> -	 * enabling it.
> -	 */
> -	if (etm4x_is_ete(drvdata))
> -		etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
> -
> -	etm4x_allow_trace(drvdata);
> -	/* Enable the trace unit */
> -	etm4x_relaxed_write32(csa, 1, TRCPRGCTLR);
> -
> -	/* Synchronize the register updates for sysreg access */
> -	if (!csa->io_mem)
> -		isb();
> -
> -	/* wait for TRCSTATR.IDLE to go back down to '0' */
> -	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_IDLE_BIT, 0))
> -		dev_err(etm_dev,
> -			"timeout while waiting for Idle Trace Status\n");
> -
> -	/*
> -	 * As recommended by section 4.3.7 ("Synchronization when using the
> -	 * memory-mapped interface") of ARM IHI 0064D
> -	 */
> -	dsb(sy);
> -	isb();
> -
> +	rc = etm4_enable_trace_unit(drvdata);
>   done:
>   	etm4_cs_lock(drvdata, csa);
>   
> @@ -906,7 +952,6 @@ static void etm4_disable_hw(void *info)
>   	struct etmv4_drvdata *drvdata = info;
>   	struct etmv4_config *config = &drvdata->config;
>   	struct coresight_device *csdev = drvdata->csdev;
> -	struct device *etm_dev = &csdev->dev;
>   	struct csdev_access *csa = &csdev->access;
>   	int i;
>   
> @@ -920,31 +965,8 @@ static void etm4_disable_hw(void *info)
>   		etm4x_relaxed_write32(csa, control, TRCPDCR);
>   	}
>   
> -	control = etm4x_relaxed_read32(csa, TRCPRGCTLR);
> +	etm4_disable_trace_unit(drvdata);
>   
> -	/* EN, bit[0] Trace unit enable bit */
> -	control &= ~0x1;
> -
> -	/*
> -	 * If the CPU supports v8.4 Trace filter Control,
> -	 * set the ETM to trace prohibited region.
> -	 */
> -	etm4x_prohibit_trace(drvdata);
> -	/*
> -	 * Make sure everything completes before disabling, as recommended
> -	 * by section 7.3.77 ("TRCVICTLR, ViewInst Main Control Register,
> -	 * SSTATUS") of ARM IHI 0064D
> -	 */
> -	dsb(sy);
> -	isb();
> -	/* Trace synchronization barrier, is a nop if not supported */
> -	tsb_csync();
> -	etm4x_relaxed_write32(csa, control, TRCPRGCTLR);
> -
> -	/* wait for TRCSTATR.PMSTABLE to go to '1' */
> -	if (coresight_timeout(csa, TRCSTATR, TRCSTATR_PMSTABLE_BIT, 1))
> -		dev_err(etm_dev,
> -			"timeout while waiting for PM stable Trace Status\n");
>   	/* read the status of the single shot comparators */
>   	for (i = 0; i < drvdata->nr_ss_cmp; i++) {
>   		config->ss_status[i] =

Re: [PATCH v2 1/8] coresight: etm4x: Extract the trace unit controlling

[Leo Yan]

Hi Suzuki,

On Mon, Mar 10, 2025 at 10:57:25AM +0000, Suzuki Kuruppassery Poulose wrote:
> Hi Leo
> 
> On 10/03/2025 10:49, Leo Yan wrote:
> > The trace unit is controlled in the ETM hardware enabling and disabling.
> > The sequential changes for support AUX pause and resume will reuse the
> > same operations.
> > 
> > Extract the operations in the etm4_{enable|disable}_trace_unit()
> > functions.  A minor improvement in etm4_enable_trace_unit() is for
> > returning the timeout error to callers.
> > 
> > Signed-off-by: Leo Yan <leo.yan@arm.com>
> 
> fyi, this area of code has changes significantly and will cause
> conflicts, due to the introduction of synchronization after
> each access to the TRCSTATR. Please rebase your next version on
> the coresight next branch to avoid the conflict.

Sure.  Will rebase on CoreSight next branch [1].

Thanks for reminding.

Leo

[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/coresight/linux.git/log/?//h=next

> The changes as such look good to me.
> 
> Suzuki

[PATCH v2 2/8] coresight: Introduce pause and resume APIs for source

[Leo Yan]

Introduce APIs for pausing and resuming trace source and export as GPL
symbols.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 drivers/hwtracing/coresight/coresight-core.c | 20 ++++++++++++++++++++
 drivers/hwtracing/coresight/coresight-priv.h |  2 ++
 include/linux/coresight.h                    |  4 ++++
 3 files changed, 26 insertions(+)

diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c
index 0a9380350fb5..eb7b83a7bfa2 100644
--- a/drivers/hwtracing/coresight/coresight-core.c
+++ b/drivers/hwtracing/coresight/coresight-core.c
@@ -365,6 +365,26 @@ void coresight_disable_source(struct coresight_device *csdev, void *data)
 }
 EXPORT_SYMBOL_GPL(coresight_disable_source);
 
+void coresight_pause_source(struct coresight_device *csdev)
+{
+	if (!csdev || !coresight_is_percpu_source(csdev))
+		return;
+
+	if (source_ops(csdev)->pause)
+		source_ops(csdev)->pause(csdev);
+}
+EXPORT_SYMBOL_GPL(coresight_pause_source);
+
+void coresight_resume_source(struct coresight_device *csdev)
+{
+	if (!csdev || !coresight_is_percpu_source(csdev))
+		return;
+
+	if (source_ops(csdev)->resume)
+		source_ops(csdev)->resume(csdev);
+}
+EXPORT_SYMBOL_GPL(coresight_resume_source);
+
 /*
  * coresight_disable_path_from : Disable components in the given path beyond
  * @nd in the list. If @nd is NULL, all the components, except the SOURCE are
diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h
index 76403530f33e..a9f14c075e91 100644
--- a/drivers/hwtracing/coresight/coresight-priv.h
+++ b/drivers/hwtracing/coresight/coresight-priv.h
@@ -247,5 +247,7 @@ void coresight_add_helper(struct coresight_device *csdev,
 void coresight_set_percpu_sink(int cpu, struct coresight_device *csdev);
 struct coresight_device *coresight_get_percpu_sink(int cpu);
 void coresight_disable_source(struct coresight_device *csdev, void *data);
+void coresight_pause_source(struct coresight_device *csdev);
+void coresight_resume_source(struct coresight_device *csdev);
 
 #endif
diff --git a/include/linux/coresight.h b/include/linux/coresight.h
index 17276965ff1d..703e1b8dbe22 100644
--- a/include/linux/coresight.h
+++ b/include/linux/coresight.h
@@ -385,6 +385,8 @@ struct coresight_ops_link {
  *		is associated to.
  * @enable:	enables tracing for a source.
  * @disable:	disables tracing for a source.
+ * @resume:	resumes tracing for a source.
+ * @pause:	pauses tracing for a source.
  */
 struct coresight_ops_source {
 	int (*cpu_id)(struct coresight_device *csdev);
@@ -392,6 +394,8 @@ struct coresight_ops_source {
 		      enum cs_mode mode, struct coresight_trace_id_map *id_map);
 	void (*disable)(struct coresight_device *csdev,
 			struct perf_event *event);
+	int (*resume)(struct coresight_device *csdev);
+	void (*pause)(struct coresight_device *csdev);
 };
 
 /**
-- 
2.34.1

Re: [PATCH v2 2/8] coresight: Introduce pause and resume APIs for source

[Suzuki K Poulose]

On 10/03/2025 10:49, Leo Yan wrote:
> Introduce APIs for pausing and resuming trace source and export as GPL
> symbols.
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---
>   drivers/hwtracing/coresight/coresight-core.c | 20 ++++++++++++++++++++
>   drivers/hwtracing/coresight/coresight-priv.h |  2 ++
>   include/linux/coresight.h                    |  4 ++++
>   3 files changed, 26 insertions(+)
> 
> diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c
> index 0a9380350fb5..eb7b83a7bfa2 100644
> --- a/drivers/hwtracing/coresight/coresight-core.c
> +++ b/drivers/hwtracing/coresight/coresight-core.c
> @@ -365,6 +365,26 @@ void coresight_disable_source(struct coresight_device *csdev, void *data)
>   }
>   EXPORT_SYMBOL_GPL(coresight_disable_source);
>   
> +void coresight_pause_source(struct coresight_device *csdev)
> +{
> +	if (!csdev || !coresight_is_percpu_source(csdev))

You may remove the !csdev check, coresight_is_percpu_source() already 
checks that.


> +		return;
> +
> +	if (source_ops(csdev)->pause)
> +		source_ops(csdev)->pause(csdev);
> +}
> +EXPORT_SYMBOL_GPL(coresight_pause_source);
> +
> +void coresight_resume_source(struct coresight_device *csdev)
> +{
> +	if (!csdev || !coresight_is_percpu_source(csdev))
> +		return;
> +

Same as above.

> +	if (source_ops(csdev)->resume)
> +		source_ops(csdev)->resume(csdev);
> +}
> +EXPORT_SYMBOL_GPL(coresight_resume_source);
> +
>   /*
>    * coresight_disable_path_from : Disable components in the given path beyond
>    * @nd in the list. If @nd is NULL, all the components, except the SOURCE are
> diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h
> index 76403530f33e..a9f14c075e91 100644
> --- a/drivers/hwtracing/coresight/coresight-priv.h
> +++ b/drivers/hwtracing/coresight/coresight-priv.h
> @@ -247,5 +247,7 @@ void coresight_add_helper(struct coresight_device *csdev,
>   void coresight_set_percpu_sink(int cpu, struct coresight_device *csdev);
>   struct coresight_device *coresight_get_percpu_sink(int cpu);
>   void coresight_disable_source(struct coresight_device *csdev, void *data);
> +void coresight_pause_source(struct coresight_device *csdev);
> +void coresight_resume_source(struct coresight_device *csdev);
>   
>   #endif
> diff --git a/include/linux/coresight.h b/include/linux/coresight.h
> index 17276965ff1d..703e1b8dbe22 100644
> --- a/include/linux/coresight.h
> +++ b/include/linux/coresight.h
> @@ -385,6 +385,8 @@ struct coresight_ops_link {
>    *		is associated to.
>    * @enable:	enables tracing for a source.
>    * @disable:	disables tracing for a source.
> + * @resume:	resumes tracing for a source.
> + * @pause:	pauses tracing for a source.
>    */
>   struct coresight_ops_source {
>   	int (*cpu_id)(struct coresight_device *csdev);
> @@ -392,6 +394,8 @@ struct coresight_ops_source {
>   		      enum cs_mode mode, struct coresight_trace_id_map *id_map);
>   	void (*disable)(struct coresight_device *csdev,
>   			struct perf_event *event);
> +	int (*resume)(struct coresight_device *csdev);
> +	void (*pause)(struct coresight_device *csdev);

Given these callbacks are for perf mode, does it make sense  to pass the 
parameters similar to enable() ? Also, should we rename these as :

resume_perf
pause_perf

To explicitly mark them perf mode specific ?

Suzuki

>   };
>   
>   /**

Re: [PATCH v2 2/8] coresight: Introduce pause and resume APIs for source

[Leo Yan]

On Mon, Mar 10, 2025 at 12:01:25PM +0000, Suzuki Kuruppassery Poulose wrote:
> On 10/03/2025 10:49, Leo Yan wrote:
> > Introduce APIs for pausing and resuming trace source and export as GPL
> > symbols.
> > 
> > Signed-off-by: Leo Yan <leo.yan@arm.com>
> > ---
> >   drivers/hwtracing/coresight/coresight-core.c | 20 ++++++++++++++++++++
> >   drivers/hwtracing/coresight/coresight-priv.h |  2 ++
> >   include/linux/coresight.h                    |  4 ++++
> >   3 files changed, 26 insertions(+)
> > 
> > diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c
> > index 0a9380350fb5..eb7b83a7bfa2 100644
> > --- a/drivers/hwtracing/coresight/coresight-core.c
> > +++ b/drivers/hwtracing/coresight/coresight-core.c
> > @@ -365,6 +365,26 @@ void coresight_disable_source(struct coresight_device *csdev, void *data)
> >   }
> >   EXPORT_SYMBOL_GPL(coresight_disable_source);
> > +void coresight_pause_source(struct coresight_device *csdev)
> > +{
> > +	if (!csdev || !coresight_is_percpu_source(csdev))
> 
> You may remove the !csdev check, coresight_is_percpu_source() already checks
> that.

Will do.

> > +		return;
> > +
> > +	if (source_ops(csdev)->pause)
> > +		source_ops(csdev)->pause(csdev);
> > +}
> > +EXPORT_SYMBOL_GPL(coresight_pause_source);
> > +
> > +void coresight_resume_source(struct coresight_device *csdev)
> > +{
> > +	if (!csdev || !coresight_is_percpu_source(csdev))
> > +		return;
> > +
> 
> Same as above.

Will do.

> > +	if (source_ops(csdev)->resume)
> > +		source_ops(csdev)->resume(csdev);
> > +}
> > +EXPORT_SYMBOL_GPL(coresight_resume_source);
> > +
> >   /*
> >    * coresight_disable_path_from : Disable components in the given path beyond
> >    * @nd in the list. If @nd is NULL, all the components, except the SOURCE are
> > diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h
> > index 76403530f33e..a9f14c075e91 100644
> > --- a/drivers/hwtracing/coresight/coresight-priv.h
> > +++ b/drivers/hwtracing/coresight/coresight-priv.h
> > @@ -247,5 +247,7 @@ void coresight_add_helper(struct coresight_device *csdev,
> >   void coresight_set_percpu_sink(int cpu, struct coresight_device *csdev);
> >   struct coresight_device *coresight_get_percpu_sink(int cpu);
> >   void coresight_disable_source(struct coresight_device *csdev, void *data);
> > +void coresight_pause_source(struct coresight_device *csdev);
> > +void coresight_resume_source(struct coresight_device *csdev);
> >   #endif
> > diff --git a/include/linux/coresight.h b/include/linux/coresight.h
> > index 17276965ff1d..703e1b8dbe22 100644
> > --- a/include/linux/coresight.h
> > +++ b/include/linux/coresight.h
> > @@ -385,6 +385,8 @@ struct coresight_ops_link {
> >    *		is associated to.
> >    * @enable:	enables tracing for a source.
> >    * @disable:	disables tracing for a source.
> > + * @resume:	resumes tracing for a source.
> > + * @pause:	pauses tracing for a source.
> >    */
> >   struct coresight_ops_source {
> >   	int (*cpu_id)(struct coresight_device *csdev);
> > @@ -392,6 +394,8 @@ struct coresight_ops_source {
> >   		      enum cs_mode mode, struct coresight_trace_id_map *id_map);
> >   	void (*disable)(struct coresight_device *csdev,
> >   			struct perf_event *event);
> > +	int (*resume)(struct coresight_device *csdev);
> > +	void (*pause)(struct coresight_device *csdev);
> 
> Given these callbacks are for perf mode, does it make sense to pass the
> parameters similar to enable() ?

It is not necessary to pass perf event handle to resume() and pause()
callbacks.

The main reason is perf related operations have been processed in the
coresight-etm-perf.c layer, the low level's callbacks (pause/resume)
should simply take actions on hardware logic.

> Also, should we rename these as :
> 
> resume_perf
> pause_perf
> 
> To explicitly mark them perf mode specific ?

This makes sense to me.  The naming "resume" might be confused with
CPUIdle's terminology (CPU resume).  pause_perf/resume_perf can
perfectly avoid any confusion.

Thanks,
Leo

[PATCH v2 3/8] coresight: etm4x: Hook pause and resume callbacks

[Leo Yan]

Add callbacks for pausing and resuming the tracer.

A "paused" flag in the driver data indicates whether the tracer is
paused.  If the flag is set, the driver will skip starting the hardware
trace.  The flag is always set to false for the sysfs mode, meaning the
tracer will never be paused in the case.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../coresight/coresight-etm4x-core.c          | 42 ++++++++++++++++++-
 drivers/hwtracing/coresight/coresight-etm4x.h |  2 +
 2 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index 71be566bd117..3a97d81b79db 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -608,7 +608,8 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
 		etm4x_relaxed_write32(csa, trcpdcr | TRCPDCR_PU, TRCPDCR);
 	}
 
-	rc = etm4_enable_trace_unit(drvdata);
+	if (!drvdata->paused)
+		rc = etm4_enable_trace_unit(drvdata);
 done:
 	etm4_cs_lock(drvdata, csa);
 
@@ -866,6 +867,9 @@ static int etm4_enable_perf(struct coresight_device *csdev,
 	}
 	drvdata->trcid = (u8)trace_id;
 
+	/* Populate pause state */
+	drvdata->paused = !!READ_ONCE(event->hw.aux_paused);
+
 	/* And enable it */
 	ret = etm4_enable_hw(drvdata);
 
@@ -895,6 +899,9 @@ static int etm4_enable_sysfs(struct coresight_device *csdev)
 	if (ret < 0)
 		goto unlock_sysfs_enable;
 
+	/* Tracer will never be paused in sysfs mode */
+	drvdata->paused = false;
+
 	/*
 	 * Executing etm4_enable_hw on the cpu whose ETM is being enabled
 	 * ensures that register writes occur when cpu is powered.
@@ -1082,10 +1089,43 @@ static void etm4_disable(struct coresight_device *csdev,
 		coresight_set_mode(csdev, CS_MODE_DISABLED);
 }
 
+static int etm4_resume(struct coresight_device *csdev)
+{
+	struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+	struct csdev_access *csa = &csdev->access;
+
+	if (coresight_get_mode(csdev) != CS_MODE_PERF)
+		return -EINVAL;
+
+	etm4_cs_unlock(drvdata, csa);
+	etm4_enable_trace_unit(drvdata);
+	etm4_cs_lock(drvdata, csa);
+
+	drvdata->paused = false;
+	return 0;
+}
+
+static void etm4_pause(struct coresight_device *csdev)
+{
+	struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+	struct csdev_access *csa = &csdev->access;
+
+	if (coresight_get_mode(csdev) != CS_MODE_PERF)
+		return;
+
+	etm4_cs_unlock(drvdata, csa);
+	etm4_disable_trace_unit(drvdata);
+	etm4_cs_lock(drvdata, csa);
+
+	drvdata->paused = true;
+}
+
 static const struct coresight_ops_source etm4_source_ops = {
 	.cpu_id		= etm4_cpu_id,
 	.enable		= etm4_enable,
 	.disable	= etm4_disable,
+	.resume		= etm4_resume,
+	.pause		= etm4_pause,
 };
 
 static const struct coresight_ops etm4_cs_ops = {
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h
index 1119762b5cec..c3389fdb5d67 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x.h
+++ b/drivers/hwtracing/coresight/coresight-etm4x.h
@@ -983,6 +983,7 @@ struct etmv4_save_state {
  * @state_needs_restore: True when there is context to restore after PM exit
  * @skip_power_up: Indicates if an implementation can skip powering up
  *		   the trace unit.
+ * @paused:	Indicates if the trace unit is paused.
  * @arch_features: Bitmap of arch features of etmv4 devices.
  */
 struct etmv4_drvdata {
@@ -1036,6 +1037,7 @@ struct etmv4_drvdata {
 	struct etmv4_save_state		*save_state;
 	bool				state_needs_restore;
 	bool				skip_power_up;
+	bool				paused;
 	DECLARE_BITMAP(arch_features, ETM4_IMPDEF_FEATURE_MAX);
 };
 
-- 
2.34.1

[PATCH v2 4/8] coresight: perf: Support AUX trace pause and resume

[Leo Yan]

This commit supports AUX trace pause and resume in a perf session for
Arm CoreSight.

First, we need to decide which flag can indicate the CoreSight PMU event
has started.  The 'event->hw.state' cannot be used for this purpose
because its initial value and the value after hardware trace enabling
are both 0.

On the other hand, the context value 'ctxt->event_data' stores the ETM
private info.  This pointer is valid only when the PMU event has been
enabled. It is safe to permit AUX trace pause and resume operations only
when it is not a NULL pointer.

To achieve fine-grained control of the pause and resume, only the tracer
is disabled and enabled.  This avoids the unnecessary complexity and
latency caused by manipulating the entire link path.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../hwtracing/coresight/coresight-etm-perf.c  | 40 ++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
index ad6a8f4b70b6..29d52386ffbb 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.c
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
@@ -366,6 +366,18 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
 			continue;
 		}
 
+		/*
+		 * If AUX pause feature is enabled but the ETM driver does not
+		 * support the operations, clear this CPU from the mask and
+		 * continue to next one.
+		 */
+		if (event->attr.aux_start_paused &&
+		    (!source_ops(csdev)->pause || !source_ops(csdev)->resume)) {
+			dev_err_once(&csdev->dev, "AUX pause is not supported.\n");
+			cpumask_clear_cpu(cpu, mask);
+			continue;
+		}
+
 		/*
 		 * No sink provided - look for a default sink for all the ETMs,
 		 * where this event can be scheduled.
@@ -451,6 +463,15 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
 	goto out;
 }
 
+static void etm_event_resume(struct coresight_device *csdev,
+			     struct etm_ctxt *ctxt)
+{
+	if (!ctxt->event_data)
+		return;
+
+	coresight_resume_source(csdev);
+}
+
 static void etm_event_start(struct perf_event *event, int flags)
 {
 	int cpu = smp_processor_id();
@@ -465,6 +486,9 @@ static void etm_event_start(struct perf_event *event, int flags)
 	if (!csdev)
 		goto fail;
 
+	if (flags & PERF_EF_RESUME)
+		return etm_event_resume(csdev, ctxt);
+
 	/* Have we messed up our tracking ? */
 	if (WARN_ON(ctxt->event_data))
 		goto fail;
@@ -550,6 +574,16 @@ static void etm_event_start(struct perf_event *event, int flags)
 	return;
 }
 
+static void etm_event_pause(struct coresight_device *csdev,
+			    struct etm_ctxt *ctxt)
+{
+	if (!ctxt->event_data)
+		return;
+
+	/* Stop tracer */
+	coresight_pause_source(csdev);
+}
+
 static void etm_event_stop(struct perf_event *event, int mode)
 {
 	int cpu = smp_processor_id();
@@ -560,6 +594,9 @@ static void etm_event_stop(struct perf_event *event, int mode)
 	struct etm_event_data *event_data;
 	struct list_head *path;
 
+	if (mode & PERF_EF_PAUSE)
+		return etm_event_pause(csdev, ctxt);
+
 	/*
 	 * If we still have access to the event_data via handle,
 	 * confirm that we haven't messed up the tracking.
@@ -904,7 +941,8 @@ int __init etm_perf_init(void)
 	int ret;
 
 	etm_pmu.capabilities		= (PERF_PMU_CAP_EXCLUSIVE |
-					   PERF_PMU_CAP_ITRACE);
+					   PERF_PMU_CAP_ITRACE |
+					   PERF_PMU_CAP_AUX_PAUSE);
 
 	etm_pmu.attr_groups		= etm_pmu_attr_groups;
 	etm_pmu.task_ctx_nr		= perf_sw_context;
-- 
2.34.1

[PATCH v2 5/8] coresight: etm: Add an attribute for updating buffer

[Leo Yan]

Add an attribute for updating buffer when the AUX trace is paused.  And
populate the value to the 'update_buf_on_pause' flag during the AUX
setting up.

If the AUX pause operation is attached to a PMU counter, when the
counter is overflow and if the PMU interrupt in an NMI, then AUX pause
operation will be triggered in the NMI context.  On the other hand, the
per CPU sink has its own interrupt handling.  Thus, there will be a race
condition between the updating buffer in NMI and sink's interrupt
handler.

To avoid the race condition, this commit disallows updating buffer on
AUX pause for the per CPU sink.  Currently, this is only applied for
TRBE.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../hwtracing/coresight/coresight-etm-perf.c  | 20 +++++++++++++++++++
 .../hwtracing/coresight/coresight-etm-perf.h  |  2 ++
 include/linux/coresight-pmu.h                 |  1 +
 3 files changed, 23 insertions(+)

diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
index 29d52386ffbb..d759663a1f7d 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.c
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
@@ -62,6 +62,8 @@ PMU_FORMAT_ATTR(contextid1,	"config:" __stringify(ETM_OPT_CTXTID));
 PMU_FORMAT_ATTR(contextid2,	"config:" __stringify(ETM_OPT_CTXTID2));
 PMU_FORMAT_ATTR(timestamp,	"config:" __stringify(ETM_OPT_TS));
 PMU_FORMAT_ATTR(retstack,	"config:" __stringify(ETM_OPT_RETSTK));
+PMU_FORMAT_ATTR(update_buf_on_pause,
+		"config:" __stringify(ETM_OPT_UPDATE_BUF_ON_PAUSE));
 /* preset - if sink ID is used as a configuration selector */
 PMU_FORMAT_ATTR(preset,		"config:0-3");
 /* Sink ID - same for all ETMs */
@@ -103,6 +105,7 @@ static struct attribute *etm_config_formats_attr[] = {
 	&format_attr_configid.attr,
 	&format_attr_branch_broadcast.attr,
 	&format_attr_cc_threshold.attr,
+	&format_attr_update_buf_on_pause.attr,
 	NULL,
 };
 
@@ -434,6 +437,23 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
 	if (!sink)
 		goto err;
 
+	/* Populate the flag for updating buffer on AUX pause */
+	event_data->update_buf_on_pause =
+		!!(event->attr.config & BIT(ETM_OPT_UPDATE_BUF_ON_PAUSE));
+
+	if (event_data->update_buf_on_pause) {
+		/*
+		 * The per CPU sink has own interrupt handling, it might have
+		 * race condition with updating buffer on AUX trace pause if
+		 * it is invoked from NMI.  To avoid the race condition,
+		 * disallows updating buffer for the per CPU sink case.
+		 */
+		if (coresight_is_percpu_sink(sink)) {
+			dev_err(&sink->dev, "update_buf_on_pause is not permitted.\n");
+			goto err;
+		}
+	}
+
 	/* If we don't have any CPUs ready for tracing, abort */
 	cpu = cpumask_first(mask);
 	if (cpu >= nr_cpu_ids)
diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.h b/drivers/hwtracing/coresight/coresight-etm-perf.h
index 744531158d6b..52b9385f8c11 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.h
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.h
@@ -51,6 +51,7 @@ struct etm_filters {
  * @aux_hwid_done:	Whether a CPU has emitted the TraceID packet or not.
  * @snk_config:		The sink configuration.
  * @cfg_hash:		The hash id of any coresight config selected.
+ * @update_buf_on_pause: The flag to indicate updating buffer on AUX pause.
  * @path:		An array of path, each slot for one CPU.
  */
 struct etm_event_data {
@@ -59,6 +60,7 @@ struct etm_event_data {
 	cpumask_t aux_hwid_done;
 	void *snk_config;
 	u32 cfg_hash;
+	bool update_buf_on_pause;
 	struct list_head * __percpu *path;
 };
 
diff --git a/include/linux/coresight-pmu.h b/include/linux/coresight-pmu.h
index 89b0ac0014b0..04147e30c2f2 100644
--- a/include/linux/coresight-pmu.h
+++ b/include/linux/coresight-pmu.h
@@ -35,6 +35,7 @@
 #define ETM_OPT_CTXTID2		15
 #define ETM_OPT_TS		28
 #define ETM_OPT_RETSTK		29
+#define ETM_OPT_UPDATE_BUF_ON_PAUSE	30
 
 /* ETMv4 CONFIGR programming bits for the ETM OPTs */
 #define ETM4_CFG_BIT_BB         3
-- 
2.34.1

Re: [PATCH v2 5/8] coresight: etm: Add an attribute for updating buffer

[Suzuki K Poulose]

Hi Leo

On 10/03/2025 10:49, Leo Yan wrote:
> Add an attribute for updating buffer when the AUX trace is paused.  And
> populate the value to the 'update_buf_on_pause' flag during the AUX
> setting up.

Do we need this attribute in the uAPI ? Could we do this by default for
sinks without interrupt ? This definitely improves the quality of trace
collected for such sinks and the driver can transparently do this.

Cheers
Suzuki

> 
> If the AUX pause operation is attached to a PMU counter, when the
> counter is overflow and if the PMU interrupt in an NMI, then AUX pause
> operation will be triggered in the NMI context.  On the other hand, the
> per CPU sink has its own interrupt handling.  Thus, there will be a race
> condition between the updating buffer in NMI and sink's interrupt
> handler.
> 
> To avoid the race condition, this commit disallows updating buffer on
> AUX pause for the per CPU sink.  Currently, this is only applied for
> TRBE.
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---
>   .../hwtracing/coresight/coresight-etm-perf.c  | 20 +++++++++++++++++++
>   .../hwtracing/coresight/coresight-etm-perf.h  |  2 ++
>   include/linux/coresight-pmu.h                 |  1 +
>   3 files changed, 23 insertions(+)
> 
> diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
> index 29d52386ffbb..d759663a1f7d 100644
> --- a/drivers/hwtracing/coresight/coresight-etm-perf.c
> +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
> @@ -62,6 +62,8 @@ PMU_FORMAT_ATTR(contextid1,	"config:" __stringify(ETM_OPT_CTXTID));
>   PMU_FORMAT_ATTR(contextid2,	"config:" __stringify(ETM_OPT_CTXTID2));
>   PMU_FORMAT_ATTR(timestamp,	"config:" __stringify(ETM_OPT_TS));
>   PMU_FORMAT_ATTR(retstack,	"config:" __stringify(ETM_OPT_RETSTK));
> +PMU_FORMAT_ATTR(update_buf_on_pause,
> +		"config:" __stringify(ETM_OPT_UPDATE_BUF_ON_PAUSE));
>   /* preset - if sink ID is used as a configuration selector */
>   PMU_FORMAT_ATTR(preset,		"config:0-3");
>   /* Sink ID - same for all ETMs */
> @@ -103,6 +105,7 @@ static struct attribute *etm_config_formats_attr[] = {
>   	&format_attr_configid.attr,
>   	&format_attr_branch_broadcast.attr,
>   	&format_attr_cc_threshold.attr,
> +	&format_attr_update_buf_on_pause.attr,
>   	NULL,
>   };
>   
> @@ -434,6 +437,23 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
>   	if (!sink)
>   		goto err;
>   
> +	/* Populate the flag for updating buffer on AUX pause */
> +	event_data->update_buf_on_pause =
> +		!!(event->attr.config & BIT(ETM_OPT_UPDATE_BUF_ON_PAUSE));
> +
> +	if (event_data->update_buf_on_pause) {
> +		/*
> +		 * The per CPU sink has own interrupt handling, it might have
> +		 * race condition with updating buffer on AUX trace pause if
> +		 * it is invoked from NMI.  To avoid the race condition,
> +		 * disallows updating buffer for the per CPU sink case.
> +		 */
> +		if (coresight_is_percpu_sink(sink)) {
> +			dev_err(&sink->dev, "update_buf_on_pause is not permitted.\n");
> +			goto err;
> +		}
> +	}
> +
>   	/* If we don't have any CPUs ready for tracing, abort */
>   	cpu = cpumask_first(mask);
>   	if (cpu >= nr_cpu_ids)
> diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.h b/drivers/hwtracing/coresight/coresight-etm-perf.h
> index 744531158d6b..52b9385f8c11 100644
> --- a/drivers/hwtracing/coresight/coresight-etm-perf.h
> +++ b/drivers/hwtracing/coresight/coresight-etm-perf.h
> @@ -51,6 +51,7 @@ struct etm_filters {
>    * @aux_hwid_done:	Whether a CPU has emitted the TraceID packet or not.
>    * @snk_config:		The sink configuration.
>    * @cfg_hash:		The hash id of any coresight config selected.
> + * @update_buf_on_pause: The flag to indicate updating buffer on AUX pause.
>    * @path:		An array of path, each slot for one CPU.
>    */
>   struct etm_event_data {
> @@ -59,6 +60,7 @@ struct etm_event_data {
>   	cpumask_t aux_hwid_done;
>   	void *snk_config;
>   	u32 cfg_hash;
> +	bool update_buf_on_pause;
>   	struct list_head * __percpu *path;
>   };
>   
> diff --git a/include/linux/coresight-pmu.h b/include/linux/coresight-pmu.h
> index 89b0ac0014b0..04147e30c2f2 100644
> --- a/include/linux/coresight-pmu.h
> +++ b/include/linux/coresight-pmu.h
> @@ -35,6 +35,7 @@
>   #define ETM_OPT_CTXTID2		15
>   #define ETM_OPT_TS		28
>   #define ETM_OPT_RETSTK		29
> +#define ETM_OPT_UPDATE_BUF_ON_PAUSE	30
>   
>   /* ETMv4 CONFIGR programming bits for the ETM OPTs */
>   #define ETM4_CFG_BIT_BB         3

Re: [PATCH v2 5/8] coresight: etm: Add an attribute for updating buffer

[Leo Yan]

On Mon, Mar 10, 2025 at 01:29:26PM +0000, Suzuki Kuruppassery Poulose wrote:
> Hi Leo
> 
> On 10/03/2025 10:49, Leo Yan wrote:
> > Add an attribute for updating buffer when the AUX trace is paused.  And
> > populate the value to the 'update_buf_on_pause' flag during the AUX
> > setting up.
> 
> Do we need this attribute in the uAPI ?

This uAPI allows users to perform AUX pause and resume without the long
latency caused by copying hardware trace data.

E.g., a user can specify a large AUX buffer size using option "-m,128M".
If the buffer is considered large enough to accommodate hardware trace
data for a small program, the 'update_buf_on_pause' flag can be set to
false, the copying will be deferred until the end of the perf session.

I am bias to keep this uAPI.  If you prefer to remove it, I am also
fine with that.

> Could we do this by default for
> sinks without interrupt ? This definitely improves the quality of trace
> collected for such sinks and the driver can transparently do this.

How about we dynamically set the default flag in the Perf tool?

- If users set explictly the 'update_buf_on_pause' flag, then the
  setting will be respected.
- If users don't set the flag, perf tool detects it is TRBE sinks,
  then it can set 'update_buf_on_pause' flag as false.
- If users don't set the flag, perf tool detects it is ETF/ETB/ETR
  sinks, it sets the flag as true.

Thanks,
Leo

Re: [PATCH v2 5/8] coresight: etm: Add an attribute for updating buffer

[Suzuki K Poulose]

On 10/03/2025 15:50, Leo Yan wrote:
> On Mon, Mar 10, 2025 at 01:29:26PM +0000, Suzuki Kuruppassery Poulose wrote:
>> Hi Leo
>>
>> On 10/03/2025 10:49, Leo Yan wrote:
>>> Add an attribute for updating buffer when the AUX trace is paused.  And
>>> populate the value to the 'update_buf_on_pause' flag during the AUX
>>> setting up.
>>
>> Do we need this attribute in the uAPI ?
> 
> This uAPI allows users to perform AUX pause and resume without the long
> latency caused by copying hardware trace data.
> 
> E.g., a user can specify a large AUX buffer size using option "-m,128M".
> If the buffer is considered large enough to accommodate hardware trace
> data for a small program, the 'update_buf_on_pause' flag can be set to
> false, the copying will be deferred until the end of the perf session.
> 
> I am bias to keep this uAPI.  If you prefer to remove it, I am also
> fine with that.
> 
>> Could we do this by default for
>> sinks without interrupt ? This definitely improves the quality of trace
>> collected for such sinks and the driver can transparently do this.
> 
> How about we dynamically set the default flag in the Perf tool?
> 
> - If users set explictly the 'update_buf_on_pause' flag, then the
>    setting will be respected.
> - If users don't set the flag, perf tool detects it is TRBE sinks,
>    then it can set 'update_buf_on_pause' flag as false.

Not really possible. There could be systems with mixed sinks. e.g.  TRBE
for some CPU and ETR for others (say due to a non-functioning TRBE).

> - If users don't set the flag, perf tool detects it is ETF/ETB/ETR
>    sinks, it sets the flag as true.

And in the cases above, perf event cannot run on all the CPUs, because
some sinks don't support it.

Why do we need a flag, when the effect is not user (read, perf decoder)
  visible and at the same time improves some scenarios (read non-TRBE 
cases) ?

I would say, let the driver always update on pause, depending on the
sink.

Suzuki


> 
> Thanks,
> Leo

Re: [PATCH v2 5/8] coresight: etm: Add an attribute for updating buffer

[Leo Yan]

On Mon, Mar 10, 2025 at 04:37:44PM +0000, Suzuki Kuruppassery Poulose wrote:

[...]

> > How about we dynamically set the default flag in the Perf tool?
> > 
> > - If users set explictly the 'update_buf_on_pause' flag, then the
> >    setting will be respected.
> > - If users don't set the flag, perf tool detects it is TRBE sinks,
> >    then it can set 'update_buf_on_pause' flag as false.
> 
> Not really possible. There could be systems with mixed sinks. e.g.  TRBE
> for some CPU and ETR for others (say due to a non-functioning TRBE).
> 
> > - If users don't set the flag, perf tool detects it is ETF/ETB/ETR
> >    sinks, it sets the flag as true.
> 
> And in the cases above, perf event cannot run on all the CPUs, because
> some sinks don't support it.
> 
> Why do we need a flag, when the effect is not user (read, perf decoder)
>  visible and at the same time improves some scenarios (read non-TRBE cases)
> ?

Indeed in this case the flag is redundant.

> I would say, let the driver always update on pause, depending on the
> sink.

It is fine for me.  I will move towards this direction.

Thanks,
Leo

[PATCH v2 6/8] coresight: perf: Update buffer on AUX pause

[Leo Yan]

Due to sinks like ETR and ETB don't support interrupt handling, the
hardware trace data might be lost for continuous running tasks.

This commit takes advantage of the AUX pause for updating trace buffer
to mitigate the trace data losing issue.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../hwtracing/coresight/coresight-etm-perf.c  | 38 ++++++++++++++++++-
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
index d759663a1f7d..667110429de9 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.c
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
@@ -594,14 +594,48 @@ static void etm_event_start(struct perf_event *event, int flags)
 	return;
 }
 
-static void etm_event_pause(struct coresight_device *csdev,
+static void etm_event_pause(struct perf_event *event,
+			    struct coresight_device *csdev,
 			    struct etm_ctxt *ctxt)
 {
+	int cpu = smp_processor_id();
+	struct coresight_device *sink;
+	struct perf_output_handle *handle = &ctxt->handle;
+	struct list_head *path;
+	unsigned long size;
+
 	if (!ctxt->event_data)
 		return;
 
 	/* Stop tracer */
 	coresight_pause_source(csdev);
+
+	/* Bail out if no need update buffer */
+	if (!ctxt->event_data->update_buf_on_pause)
+		return;
+
+	if (WARN_ON_ONCE(handle->event != event))
+		return;
+
+	path = etm_event_cpu_path(ctxt->event_data, cpu);
+	sink = coresight_get_sink(path);
+	if (WARN_ON_ONCE(!sink))
+		return;
+
+	if (!sink_ops(sink)->update_buffer)
+		return;
+
+	size = sink_ops(sink)->update_buffer(sink, handle,
+					     ctxt->event_data->snk_config);
+	if (READ_ONCE(handle->event)) {
+		if (!size)
+			return;
+
+		perf_aux_output_end(handle, size);
+		perf_aux_output_begin(handle, event);
+	} else {
+		WARN_ON_ONCE(size);
+	}
 }
 
 static void etm_event_stop(struct perf_event *event, int mode)
@@ -615,7 +649,7 @@ static void etm_event_stop(struct perf_event *event, int mode)
 	struct list_head *path;
 
 	if (mode & PERF_EF_PAUSE)
-		return etm_event_pause(csdev, ctxt);
+		return etm_event_pause(event, csdev, ctxt);
 
 	/*
 	 * If we still have access to the event_data via handle,
-- 
2.34.1

[PATCH v2 7/8] Documentation: coresight: Document AUX pause and resume

[Leo Yan]

This adds description for AUX pause and resume.  It gives introduction
for what's AUX pause and resume and how to use the configuration terms.
As last, it records several usage examples.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 .../trace/coresight/coresight-perf.rst        | 50 +++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/Documentation/trace/coresight/coresight-perf.rst b/Documentation/trace/coresight/coresight-perf.rst
index d087aae7d492..c49aa05c51eb 100644
--- a/Documentation/trace/coresight/coresight-perf.rst
+++ b/Documentation/trace/coresight/coresight-perf.rst
@@ -78,6 +78,56 @@ enabled like::
 
 Please refer to the kernel configuration help for more information.
 
+Fine-grained tracing with AUX pause and resume
+----------------------------------------------
+
+Arm CoreSight may generate a large amount of hardware trace data, which
+will lead to overhead in recording and distract users when reviewing
+profiling result. To mitigate the issue of excessive trace data, Perf
+provides AUX pause and resume functionality for fine-grained tracing.
+
+The AUX pause and resume can be triggered by associated events. These
+events can be ftrace tracepoints (including static and dynamic
+tracepoints) or PMU events (e.g. CPU PMU cycle event). To create a perf
+session with AUX pause / resume, three configuration terms are
+introduced:
+
+- "aux-action=start-paused": it is specified for the cs_etm PMU event to
+  launch in a paused state.
+- "aux-action=pause": an associated event is specified with this term
+  to pause AUX trace.
+- "aux-action=resume": an associated event is specified with this term
+  to resume AUX trace.
+
+Some Arm CoreSight sinks (e.g., ETR, ETB) do not support interrupt
+mechanism. As a result, trace data might be lost if it cannot be copied
+to the user space tool in a timely manner. To mitigate trace data lose,
+a configuration term "update_buf_on_pause" can be used for the cs_etm
+PMU event. As its name suggests, it directs the driver to update the
+buffer when the AUX trace is paused. By default, this feature is
+disabled until users explicitly specify "update_buf_on_pause=1".
+
+TRBE has interrupt handling.  To avoid the race condition between the
+AUX pause or resume invoked in NMI and TRBE interrupt handler, the
+driver reports a failure if users try to use "update_buf_on_pause=1" on
+TRBE.
+
+Example for triggering AUX pause and resume with ftrace tracepoints::
+
+  perf record -e cs_etm/aux-action=start-paused/k,syscalls:sys_enter_openat/aux-action=resume/,syscalls:sys_exit_openat/aux-action=pause/ ls
+
+Example for triggering AUX pause and resume with PMU event::
+
+  perf record -a -e cs_etm/aux-action=start-paused/k \
+        -e cycles/aux-action=pause,period=10000000/ \
+        -e cycles/aux-action=resume,period=1050000/ -- sleep 1
+
+Example for updating buffer on AUX pause::
+
+  perf record -a -e cs_etm/aux-action=start-paused,update_buf_on_pause=1/ \
+        -e cycles/aux-action=pause,period=10000000/ \
+        -e cycles/aux-action=resume,period=1050000/ -- sleep 1
+
 Perf test - Verify kernel and userspace perf CoreSight work
 -----------------------------------------------------------
 
-- 
2.34.1

[PATCH v2 8/8] perf cs-etm: Sync kernel coresight-pmu.h header

[Leo Yan]

Update the header in the tools to align with the kernel header.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/include/linux/coresight-pmu.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
index 89b0ac0014b0..04147e30c2f2 100644
--- a/tools/include/linux/coresight-pmu.h
+++ b/tools/include/linux/coresight-pmu.h
@@ -35,6 +35,7 @@
 #define ETM_OPT_CTXTID2		15
 #define ETM_OPT_TS		28
 #define ETM_OPT_RETSTK		29
+#define ETM_OPT_UPDATE_BUF_ON_PAUSE	30
 
 /* ETMv4 CONFIGR programming bits for the ETM OPTs */
 #define ETM4_CFG_BIT_BB         3
-- 
2.34.1