Re: [PATCH v8 0/8] Add metrics for Arm CMN

Re: [PATCH v8 0/8] Add metrics for Arm CMN

[Arnaldo Carvalho de Melo]

Em Thu, Sep 07, 2023 at 07:58:25PM +0800, Jing Zhang escreveu:
> Changes since v7:
> - Optimized pmu_uncore_identifier_match().
> - Added all missing "event=0" in JSON file.
> - Rebase this series onto the latest perf-tools-next.
> - Link: https://lore.kernel.org/all/1692606977-92009-1-git-send-email-renyu.zj@linux.alibaba.com/

Can you please refresh on top of what is in the tmp.perf-tools-next
branch at:

git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git

patch 3/8 has some fuzz and b4 is not finding it in the mailing list,
probably the patch is too large? Do you have some git tree somewhere I
can use to pull from?

- Arnaldo
 
> Jing Zhang (8):
>   perf pmu: "Compat" supports matching multiple identifiers
>   perf metric: "Compat" supports matching multiple identifiers
>   perf vendor events: Supplement the omitted EventCode
>   perf jevents: Support more event fields
>   perf test: Make matching_pmu effective
>   perf test: Add pmu-event test for "Compat" and new event_field.
>   perf jevents: Add support for Arm CMN PMU aliasing
>   perf vendor events: Add JSON metrics for Arm CMN
> 
>  .../pmu-events/arch/arm64/arm/cmn/sys/cmn.json     | 266 +++++++++++++++++++++
>  .../pmu-events/arch/arm64/arm/cmn/sys/metric.json  |  74 ++++++
>  .../pmu-events/arch/test/test_soc/sys/uncore.json  |   8 +
>  .../pmu-events/arch/x86/alderlake/pipeline.json    |   9 +
>  .../pmu-events/arch/x86/alderlaken/pipeline.json   |   3 +
>  .../pmu-events/arch/x86/broadwell/pipeline.json    |   4 +
>  .../pmu-events/arch/x86/broadwellde/pipeline.json  |   4 +
>  .../arch/x86/broadwellde/uncore-cache.json         |   2 +
>  .../arch/x86/broadwellde/uncore-interconnect.json  |   1 +
>  .../arch/x86/broadwellde/uncore-memory.json        |   1 +
>  .../arch/x86/broadwellde/uncore-power.json         |   1 +
>  .../pmu-events/arch/x86/broadwellx/pipeline.json   |   4 +
>  .../arch/x86/broadwellx/uncore-cache.json          |   2 +
>  .../arch/x86/broadwellx/uncore-interconnect.json   |  13 +
>  .../arch/x86/broadwellx/uncore-memory.json         |   2 +
>  .../arch/x86/broadwellx/uncore-power.json          |   1 +
>  .../pmu-events/arch/x86/cascadelakex/pipeline.json |   4 +
>  .../arch/x86/cascadelakex/uncore-cache.json        |   2 +
>  .../arch/x86/cascadelakex/uncore-interconnect.json |   1 +
>  .../arch/x86/cascadelakex/uncore-io.json           |   1 +
>  .../arch/x86/cascadelakex/uncore-memory.json       |   1 +
>  .../arch/x86/cascadelakex/uncore-power.json        |   1 +
>  .../pmu-events/arch/x86/elkhartlake/pipeline.json  |   2 +
>  .../pmu-events/arch/x86/goldmont/pipeline.json     |   3 +
>  .../pmu-events/arch/x86/goldmontplus/pipeline.json |   3 +
>  .../pmu-events/arch/x86/grandridge/pipeline.json   |   3 +
>  .../arch/x86/graniterapids/pipeline.json           |   4 +
>  .../perf/pmu-events/arch/x86/haswell/pipeline.json |   4 +
>  .../pmu-events/arch/x86/haswellx/pipeline.json     |   4 +
>  .../pmu-events/arch/x86/haswellx/uncore-cache.json |   2 +
>  .../arch/x86/haswellx/uncore-interconnect.json     |  14 ++
>  .../arch/x86/haswellx/uncore-memory.json           |   2 +
>  .../pmu-events/arch/x86/haswellx/uncore-power.json |   1 +
>  .../perf/pmu-events/arch/x86/icelake/pipeline.json |   5 +
>  .../pmu-events/arch/x86/icelakex/pipeline.json     |   5 +
>  .../pmu-events/arch/x86/icelakex/uncore-cache.json |   1 +
>  .../arch/x86/icelakex/uncore-interconnect.json     |   1 +
>  .../arch/x86/icelakex/uncore-memory.json           |   1 +
>  .../pmu-events/arch/x86/icelakex/uncore-power.json |   1 +
>  .../pmu-events/arch/x86/ivybridge/pipeline.json    |   3 +
>  .../perf/pmu-events/arch/x86/ivytown/pipeline.json |   4 +
>  .../pmu-events/arch/x86/ivytown/uncore-cache.json  |   2 +
>  .../arch/x86/ivytown/uncore-interconnect.json      |  11 +
>  .../pmu-events/arch/x86/ivytown/uncore-memory.json |   1 +
>  .../pmu-events/arch/x86/ivytown/uncore-power.json  |   1 +
>  .../pmu-events/arch/x86/jaketown/pipeline.json     |   4 +
>  .../pmu-events/arch/x86/jaketown/uncore-cache.json |   2 +
>  .../arch/x86/jaketown/uncore-interconnect.json     |  12 +
>  .../arch/x86/jaketown/uncore-memory.json           |   1 +
>  .../pmu-events/arch/x86/jaketown/uncore-power.json |   2 +
>  .../arch/x86/knightslanding/pipeline.json          |   3 +
>  .../arch/x86/knightslanding/uncore-cache.json      |   1 +
>  .../arch/x86/knightslanding/uncore-memory.json     |   4 +
>  .../pmu-events/arch/x86/meteorlake/pipeline.json   |   9 +
>  .../pmu-events/arch/x86/rocketlake/pipeline.json   |   3 +
>  .../pmu-events/arch/x86/sandybridge/pipeline.json  |   4 +
>  .../arch/x86/sapphirerapids/pipeline.json          |   5 +
>  .../pmu-events/arch/x86/sierraforest/pipeline.json |   4 +
>  .../pmu-events/arch/x86/silvermont/pipeline.json   |   3 +
>  .../perf/pmu-events/arch/x86/skylake/pipeline.json |   4 +
>  .../pmu-events/arch/x86/skylakex/pipeline.json     |   4 +
>  .../pmu-events/arch/x86/skylakex/uncore-cache.json |   2 +
>  .../arch/x86/skylakex/uncore-interconnect.json     |   1 +
>  .../pmu-events/arch/x86/skylakex/uncore-io.json    |   1 +
>  .../arch/x86/skylakex/uncore-memory.json           |   1 +
>  .../pmu-events/arch/x86/skylakex/uncore-power.json |   1 +
>  .../pmu-events/arch/x86/snowridgex/pipeline.json   |   2 +
>  .../arch/x86/snowridgex/uncore-cache.json          |   1 +
>  .../arch/x86/snowridgex/uncore-interconnect.json   |   1 +
>  .../arch/x86/snowridgex/uncore-memory.json         |   1 +
>  .../arch/x86/snowridgex/uncore-power.json          |   1 +
>  .../pmu-events/arch/x86/tigerlake/pipeline.json    |   5 +
>  tools/perf/pmu-events/empty-pmu-events.c           |   8 +
>  tools/perf/pmu-events/jevents.py                   |  21 +-
>  tools/perf/tests/pmu-events.c                      |  65 ++++-
>  tools/perf/util/metricgroup.c                      |   2 +-
>  tools/perf/util/pmu.c                              |  28 ++-
>  tools/perf/util/pmu.h                              |   1 +
>  78 files changed, 681 insertions(+), 9 deletions(-)
>  create mode 100644 tools/perf/pmu-events/arch/arm64/arm/cmn/sys/cmn.json
>  create mode 100644 tools/perf/pmu-events/arch/arm64/arm/cmn/sys/metric.json
> 
> -- 
> 1.8.3.1
> 

-- 

- Arnaldo

Re: [PATCH v8 0/8] Add metrics for Arm CMN

[Jing Zhang]

在 2023/9/8 上午12:06, Arnaldo Carvalho de Melo 写道:
> Em Thu, Sep 07, 2023 at 07:58:25PM +0800, Jing Zhang escreveu:
>> Changes since v7:
>> - Optimized pmu_uncore_identifier_match().
>> - Added all missing "event=0" in JSON file.
>> - Rebase this series onto the latest perf-tools-next.
>> - Link: https://lore.kernel.org/all/1692606977-92009-1-git-send-email-renyu.zj@linux.alibaba.com/
> 
> Can you please refresh on top of what is in the tmp.perf-tools-next
> branch at:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git
> 

Yes, this series is already based on the latest tmp.perf-tools-next.

> patch 3/8 has some fuzz and b4 is not finding it in the mailing list,
> probably the patch is too large? Do you have some git tree somewhere I
> can use to pull from?
> 

Patch 3/8 is too large and it is being held until the list moderator can
review it for approval.

Perhaps you can pull it from my repository. Sorry for the inconvenience.
https://github.com/ZhangJing-hub/linux/commit/c33234e62aa210cc7dea0be3c749af1a8737b88d

Thanks,
Jing

> - Arnaldo
>  
>> Jing Zhang (8):
>>   perf pmu: "Compat" supports matching multiple identifiers
>>   perf metric: "Compat" supports matching multiple identifiers
>>   perf vendor events: Supplement the omitted EventCode
>>   perf jevents: Support more event fields
>>   perf test: Make matching_pmu effective
>>   perf test: Add pmu-event test for "Compat" and new event_field.
>>   perf jevents: Add support for Arm CMN PMU aliasing
>>   perf vendor events: Add JSON metrics for Arm CMN
>>
>>  .../pmu-events/arch/arm64/arm/cmn/sys/cmn.json     | 266 +++++++++++++++++++++
>>  .../pmu-events/arch/arm64/arm/cmn/sys/metric.json  |  74 ++++++
>>  .../pmu-events/arch/test/test_soc/sys/uncore.json  |   8 +
>>  .../pmu-events/arch/x86/alderlake/pipeline.json    |   9 +
>>  .../pmu-events/arch/x86/alderlaken/pipeline.json   |   3 +
>>  .../pmu-events/arch/x86/broadwell/pipeline.json    |   4 +
>>  .../pmu-events/arch/x86/broadwellde/pipeline.json  |   4 +
>>  .../arch/x86/broadwellde/uncore-cache.json         |   2 +
>>  .../arch/x86/broadwellde/uncore-interconnect.json  |   1 +
>>  .../arch/x86/broadwellde/uncore-memory.json        |   1 +
>>  .../arch/x86/broadwellde/uncore-power.json         |   1 +
>>  .../pmu-events/arch/x86/broadwellx/pipeline.json   |   4 +
>>  .../arch/x86/broadwellx/uncore-cache.json          |   2 +
>>  .../arch/x86/broadwellx/uncore-interconnect.json   |  13 +
>>  .../arch/x86/broadwellx/uncore-memory.json         |   2 +
>>  .../arch/x86/broadwellx/uncore-power.json          |   1 +
>>  .../pmu-events/arch/x86/cascadelakex/pipeline.json |   4 +
>>  .../arch/x86/cascadelakex/uncore-cache.json        |   2 +
>>  .../arch/x86/cascadelakex/uncore-interconnect.json |   1 +
>>  .../arch/x86/cascadelakex/uncore-io.json           |   1 +
>>  .../arch/x86/cascadelakex/uncore-memory.json       |   1 +
>>  .../arch/x86/cascadelakex/uncore-power.json        |   1 +
>>  .../pmu-events/arch/x86/elkhartlake/pipeline.json  |   2 +
>>  .../pmu-events/arch/x86/goldmont/pipeline.json     |   3 +
>>  .../pmu-events/arch/x86/goldmontplus/pipeline.json |   3 +
>>  .../pmu-events/arch/x86/grandridge/pipeline.json   |   3 +
>>  .../arch/x86/graniterapids/pipeline.json           |   4 +
>>  .../perf/pmu-events/arch/x86/haswell/pipeline.json |   4 +
>>  .../pmu-events/arch/x86/haswellx/pipeline.json     |   4 +
>>  .../pmu-events/arch/x86/haswellx/uncore-cache.json |   2 +
>>  .../arch/x86/haswellx/uncore-interconnect.json     |  14 ++
>>  .../arch/x86/haswellx/uncore-memory.json           |   2 +
>>  .../pmu-events/arch/x86/haswellx/uncore-power.json |   1 +
>>  .../perf/pmu-events/arch/x86/icelake/pipeline.json |   5 +
>>  .../pmu-events/arch/x86/icelakex/pipeline.json     |   5 +
>>  .../pmu-events/arch/x86/icelakex/uncore-cache.json |   1 +
>>  .../arch/x86/icelakex/uncore-interconnect.json     |   1 +
>>  .../arch/x86/icelakex/uncore-memory.json           |   1 +
>>  .../pmu-events/arch/x86/icelakex/uncore-power.json |   1 +
>>  .../pmu-events/arch/x86/ivybridge/pipeline.json    |   3 +
>>  .../perf/pmu-events/arch/x86/ivytown/pipeline.json |   4 +
>>  .../pmu-events/arch/x86/ivytown/uncore-cache.json  |   2 +
>>  .../arch/x86/ivytown/uncore-interconnect.json      |  11 +
>>  .../pmu-events/arch/x86/ivytown/uncore-memory.json |   1 +
>>  .../pmu-events/arch/x86/ivytown/uncore-power.json  |   1 +
>>  .../pmu-events/arch/x86/jaketown/pipeline.json     |   4 +
>>  .../pmu-events/arch/x86/jaketown/uncore-cache.json |   2 +
>>  .../arch/x86/jaketown/uncore-interconnect.json     |  12 +
>>  .../arch/x86/jaketown/uncore-memory.json           |   1 +
>>  .../pmu-events/arch/x86/jaketown/uncore-power.json |   2 +
>>  .../arch/x86/knightslanding/pipeline.json          |   3 +
>>  .../arch/x86/knightslanding/uncore-cache.json      |   1 +
>>  .../arch/x86/knightslanding/uncore-memory.json     |   4 +
>>  .../pmu-events/arch/x86/meteorlake/pipeline.json   |   9 +
>>  .../pmu-events/arch/x86/rocketlake/pipeline.json   |   3 +
>>  .../pmu-events/arch/x86/sandybridge/pipeline.json  |   4 +
>>  .../arch/x86/sapphirerapids/pipeline.json          |   5 +
>>  .../pmu-events/arch/x86/sierraforest/pipeline.json |   4 +
>>  .../pmu-events/arch/x86/silvermont/pipeline.json   |   3 +
>>  .../perf/pmu-events/arch/x86/skylake/pipeline.json |   4 +
>>  .../pmu-events/arch/x86/skylakex/pipeline.json     |   4 +
>>  .../pmu-events/arch/x86/skylakex/uncore-cache.json |   2 +
>>  .../arch/x86/skylakex/uncore-interconnect.json     |   1 +
>>  .../pmu-events/arch/x86/skylakex/uncore-io.json    |   1 +
>>  .../arch/x86/skylakex/uncore-memory.json           |   1 +
>>  .../pmu-events/arch/x86/skylakex/uncore-power.json |   1 +
>>  .../pmu-events/arch/x86/snowridgex/pipeline.json   |   2 +
>>  .../arch/x86/snowridgex/uncore-cache.json          |   1 +
>>  .../arch/x86/snowridgex/uncore-interconnect.json   |   1 +
>>  .../arch/x86/snowridgex/uncore-memory.json         |   1 +
>>  .../arch/x86/snowridgex/uncore-power.json          |   1 +
>>  .../pmu-events/arch/x86/tigerlake/pipeline.json    |   5 +
>>  tools/perf/pmu-events/empty-pmu-events.c           |   8 +
>>  tools/perf/pmu-events/jevents.py                   |  21 +-
>>  tools/perf/tests/pmu-events.c                      |  65 ++++-
>>  tools/perf/util/metricgroup.c                      |   2 +-
>>  tools/perf/util/pmu.c                              |  28 ++-
>>  tools/perf/util/pmu.h                              |   1 +
>>  78 files changed, 681 insertions(+), 9 deletions(-)
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/arm/cmn/sys/cmn.json
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/arm/cmn/sys/metric.json
>>
>> -- 
>> 1.8.3.1
>>
> 

Re: [PATCH v8 1/8] perf pmu: "Compat" supports matching multiple identifiers

[Ian Rogers]

On Thu, Sep 7, 2023 at 4:58 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>
> The jevent "Compat" is used for uncore PMU alias or metric definitions.
>
> The same PMU driver has different PMU identifiers due to different
> hardware versions and types, but they may have some common PMU event.
> Since a Compat value can only match one identifier, when adding the
> same event alias to PMUs with different identifiers, each identifier
> needs to be defined once, which is not streamlined enough.
>
> So let "Compat" supports matching multiple identifiers for uncore PMU
> alias. For example, the Compat value {43401;436*} can match the PMU
> identifier "43401", that is, CMN600_r0p0, and the PMU identifier with
> the prefix "436", that is, all CMN650, where "*" is a wildcard.
> Tokens in Unit field are delimited by ';' with no spaces.
>
> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
> Reviewed-by: John Garry <john.g.garry@oracle.com>
> ---
>  tools/perf/util/pmu.c | 28 ++++++++++++++++++++++++++--
>  tools/perf/util/pmu.h |  1 +
>  2 files changed, 27 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
> index e215985..c3c3818 100644
> --- a/tools/perf/util/pmu.c
> +++ b/tools/perf/util/pmu.c
> @@ -875,6 +875,30 @@ static bool pmu_uncore_alias_match(const char *pmu_name, const char *name)
>         return res;
>  }
>
> +bool pmu_uncore_identifier_match(const char *id, const char *compat)
> +{
> +       char *tmp = NULL, *tok, *str;
> +       bool res = false;
> +
> +       /*
> +        * The strdup() call is necessary here because "compat" is a const str*
> +        * type and cannot be used as an argument to strtok_r().
> +        */
> +       str = strdup(compat);
> +       if (!str)
> +               return false;
> +
> +       tok = strtok_r(str, ";", &tmp);

Did the comma vs semicolon difference get explained? It seems to add
inconsistency to use a semicolon.

Thanks,
Ian

> +       for (; tok; tok = strtok_r(NULL, ";", &tmp)) {
> +               if (!fnmatch(tok, id, FNM_CASEFOLD)) {
> +                       res = true;
> +                       break;
> +               }
> +       }
> +       free(str);
> +       return res;
> +}
> +
>  static int pmu_add_cpu_aliases_map_callback(const struct pmu_event *pe,
>                                         const struct pmu_events_table *table __maybe_unused,
>                                         void *vdata)
> @@ -915,8 +939,8 @@ static int pmu_add_sys_aliases_iter_fn(const struct pmu_event *pe,
>         if (!pe->compat || !pe->pmu)
>                 return 0;
>
> -       if (!strcmp(pmu->id, pe->compat) &&
> -           pmu_uncore_alias_match(pe->pmu, pmu->name)) {
> +       if (pmu_uncore_alias_match(pe->pmu, pmu->name) &&
> +                       pmu_uncore_identifier_match(pmu->id, pe->compat)) {
>                 perf_pmu__new_alias(pmu,
>                                 pe->name,
>                                 pe->desc,
> diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
> index bd5d804..1bf5cf1 100644
> --- a/tools/perf/util/pmu.h
> +++ b/tools/perf/util/pmu.h
> @@ -240,6 +240,7 @@ void pmu_add_cpu_aliases_table(struct perf_pmu *pmu,
>  char *perf_pmu__getcpuid(struct perf_pmu *pmu);
>  const struct pmu_events_table *pmu_events_table__find(void);
>  const struct pmu_metrics_table *pmu_metrics_table__find(void);
> +bool pmu_uncore_identifier_match(const char *id, const char *compat);
>
>  int perf_pmu__convert_scale(const char *scale, char **end, double *sval);
>
> --
> 1.8.3.1
>

Re: [PATCH v8 1/8] perf pmu: "Compat" supports matching multiple identifiers

[Jing Zhang]

在 2023/9/9 上午5:33, Ian Rogers 写道:
> On Thu, Sep 7, 2023 at 4:58 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>
>> The jevent "Compat" is used for uncore PMU alias or metric definitions.
>>
>> The same PMU driver has different PMU identifiers due to different
>> hardware versions and types, but they may have some common PMU event.
>> Since a Compat value can only match one identifier, when adding the
>> same event alias to PMUs with different identifiers, each identifier
>> needs to be defined once, which is not streamlined enough.
>>
>> So let "Compat" supports matching multiple identifiers for uncore PMU
>> alias. For example, the Compat value {43401;436*} can match the PMU
>> identifier "43401", that is, CMN600_r0p0, and the PMU identifier with
>> the prefix "436", that is, all CMN650, where "*" is a wildcard.
>> Tokens in Unit field are delimited by ';' with no spaces.
>>
>> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
>> Reviewed-by: John Garry <john.g.garry@oracle.com>
>> ---
>>  tools/perf/util/pmu.c | 28 ++++++++++++++++++++++++++--
>>  tools/perf/util/pmu.h |  1 +
>>  2 files changed, 27 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
>> index e215985..c3c3818 100644
>> --- a/tools/perf/util/pmu.c
>> +++ b/tools/perf/util/pmu.c
>> @@ -875,6 +875,30 @@ static bool pmu_uncore_alias_match(const char *pmu_name, const char *name)
>>         return res;
>>  }
>>
>> +bool pmu_uncore_identifier_match(const char *id, const char *compat)
>> +{
>> +       char *tmp = NULL, *tok, *str;
>> +       bool res = false;
>> +
>> +       /*
>> +        * The strdup() call is necessary here because "compat" is a const str*
>> +        * type and cannot be used as an argument to strtok_r().
>> +        */
>> +       str = strdup(compat);
>> +       if (!str)
>> +               return false;
>> +
>> +       tok = strtok_r(str, ";", &tmp);
> 
> Did the comma vs semicolon difference get explained? It seems to add
> inconsistency to use a semicolon.
> 

Hi Ian,

Yes, I explained the reason for using semicolons instead of commas in v7.

I use a semicolon instead of a comma because I want to distinguish it from the function
of the comma in "Unit" and avoid confusion between the use of commas in "Unit" and "Compat".
Because in Unit, commas act as wildcards, and in “Compat”, the semicolon means “or”. So
I think semicolons are more appropriate.

John also raised this issue earlier, and we finally agreed to use semicolons.
What do you think?


Thanks,
Jing

> Thanks,
> Ian
> 
>> +       for (; tok; tok = strtok_r(NULL, ";", &tmp)) {
>> +               if (!fnmatch(tok, id, FNM_CASEFOLD)) {
>> +                       res = true;
>> +                       break;
>> +               }
>> +       }
>> +       free(str);
>> +       return res;
>> +}
>> +
>>  static int pmu_add_cpu_aliases_map_callback(const struct pmu_event *pe,
>>                                         const struct pmu_events_table *table __maybe_unused,
>>                                         void *vdata)
>> @@ -915,8 +939,8 @@ static int pmu_add_sys_aliases_iter_fn(const struct pmu_event *pe,
>>         if (!pe->compat || !pe->pmu)
>>                 return 0;
>>
>> -       if (!strcmp(pmu->id, pe->compat) &&
>> -           pmu_uncore_alias_match(pe->pmu, pmu->name)) {
>> +       if (pmu_uncore_alias_match(pe->pmu, pmu->name) &&
>> +                       pmu_uncore_identifier_match(pmu->id, pe->compat)) {
>>                 perf_pmu__new_alias(pmu,
>>                                 pe->name,
>>                                 pe->desc,
>> diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
>> index bd5d804..1bf5cf1 100644
>> --- a/tools/perf/util/pmu.h
>> +++ b/tools/perf/util/pmu.h
>> @@ -240,6 +240,7 @@ void pmu_add_cpu_aliases_table(struct perf_pmu *pmu,
>>  char *perf_pmu__getcpuid(struct perf_pmu *pmu);
>>  const struct pmu_events_table *pmu_events_table__find(void);
>>  const struct pmu_metrics_table *pmu_metrics_table__find(void);
>> +bool pmu_uncore_identifier_match(const char *id, const char *compat);
>>
>>  int perf_pmu__convert_scale(const char *scale, char **end, double *sval);
>>
>> --
>> 1.8.3.1
>>

Re: [PATCH v8 1/8] perf pmu: "Compat" supports matching multiple identifiers

[Ian Rogers]

On Sun, Sep 10, 2023 at 7:32 PM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>
>
>
> 在 2023/9/9 上午5:33, Ian Rogers 写道:
> > On Thu, Sep 7, 2023 at 4:58 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
> >>
> >> The jevent "Compat" is used for uncore PMU alias or metric definitions.
> >>
> >> The same PMU driver has different PMU identifiers due to different
> >> hardware versions and types, but they may have some common PMU event.
> >> Since a Compat value can only match one identifier, when adding the
> >> same event alias to PMUs with different identifiers, each identifier
> >> needs to be defined once, which is not streamlined enough.
> >>
> >> So let "Compat" supports matching multiple identifiers for uncore PMU
> >> alias. For example, the Compat value {43401;436*} can match the PMU
> >> identifier "43401", that is, CMN600_r0p0, and the PMU identifier with
> >> the prefix "436", that is, all CMN650, where "*" is a wildcard.
> >> Tokens in Unit field are delimited by ';' with no spaces.
> >>
> >> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
> >> Reviewed-by: John Garry <john.g.garry@oracle.com>
> >> ---
> >>  tools/perf/util/pmu.c | 28 ++++++++++++++++++++++++++--
> >>  tools/perf/util/pmu.h |  1 +
> >>  2 files changed, 27 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
> >> index e215985..c3c3818 100644
> >> --- a/tools/perf/util/pmu.c
> >> +++ b/tools/perf/util/pmu.c
> >> @@ -875,6 +875,30 @@ static bool pmu_uncore_alias_match(const char *pmu_name, const char *name)
> >>         return res;
> >>  }
> >>
> >> +bool pmu_uncore_identifier_match(const char *id, const char *compat)
> >> +{
> >> +       char *tmp = NULL, *tok, *str;
> >> +       bool res = false;
> >> +
> >> +       /*
> >> +        * The strdup() call is necessary here because "compat" is a const str*
> >> +        * type and cannot be used as an argument to strtok_r().
> >> +        */
> >> +       str = strdup(compat);
> >> +       if (!str)
> >> +               return false;
> >> +
> >> +       tok = strtok_r(str, ";", &tmp);
> >
> > Did the comma vs semicolon difference get explained? It seems to add
> > inconsistency to use a semicolon.
> >
>
> Hi Ian,
>
> Yes, I explained the reason for using semicolons instead of commas in v7.
>
> I use a semicolon instead of a comma because I want to distinguish it from the function
> of the comma in "Unit" and avoid confusion between the use of commas in "Unit" and "Compat".
> Because in Unit, commas act as wildcards, and in “Compat”, the semicolon means “or”. So
> I think semicolons are more appropriate.
>
> John also raised this issue earlier, and we finally agreed to use semicolons.
> What do you think?

I'm okay with it, but thanks for capturing the why of this. I'd like
at some point to make the wildcarding of things less ad hoc. For
example, on x86 we use regular expressions to match cpuid:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/arch/x86/mapfile.csv?h=perf-tools-next
but file name style matching for pmus:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/pmu.c?h=perf-tools-next#n1974
Given that we're okay with regular expressions then I don't see why
everything shouldn't be a regular expression. This could, for example,
make matching PMUs more specific than just adding a star and doing a
file name match. For an example of why this is weird, on my laptop:
```
$ perf stat -e i/actual-frequency/ true

Performance counter stats for 'system wide':

                0      i/actual-frequency/

      0.001168195 seconds time elapsed
```
The PMU I used here as 'i' is /sys/devices/i915 as we allow arbitrary
numbers after a PMU name for cases of multiple uncore PMUs.

My feeling longer term is that the matching distinction of Unit and
Compat, comma and semi-colon, would be better captured with regular
expressions as I think they show the intent in the matching more
clearly.

Thanks,
Ian


> Thanks,
> Jing
>
> > Thanks,
> > Ian
> >
> >> +       for (; tok; tok = strtok_r(NULL, ";", &tmp)) {
> >> +               if (!fnmatch(tok, id, FNM_CASEFOLD)) {
> >> +                       res = true;
> >> +                       break;
> >> +               }
> >> +       }
> >> +       free(str);
> >> +       return res;
> >> +}
> >> +
> >>  static int pmu_add_cpu_aliases_map_callback(const struct pmu_event *pe,
> >>                                         const struct pmu_events_table *table __maybe_unused,
> >>                                         void *vdata)
> >> @@ -915,8 +939,8 @@ static int pmu_add_sys_aliases_iter_fn(const struct pmu_event *pe,
> >>         if (!pe->compat || !pe->pmu)
> >>                 return 0;
> >>
> >> -       if (!strcmp(pmu->id, pe->compat) &&
> >> -           pmu_uncore_alias_match(pe->pmu, pmu->name)) {
> >> +       if (pmu_uncore_alias_match(pe->pmu, pmu->name) &&
> >> +                       pmu_uncore_identifier_match(pmu->id, pe->compat)) {
> >>                 perf_pmu__new_alias(pmu,
> >>                                 pe->name,
> >>                                 pe->desc,
> >> diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
> >> index bd5d804..1bf5cf1 100644
> >> --- a/tools/perf/util/pmu.h
> >> +++ b/tools/perf/util/pmu.h
> >> @@ -240,6 +240,7 @@ void pmu_add_cpu_aliases_table(struct perf_pmu *pmu,
> >>  char *perf_pmu__getcpuid(struct perf_pmu *pmu);
> >>  const struct pmu_events_table *pmu_events_table__find(void);
> >>  const struct pmu_metrics_table *pmu_metrics_table__find(void);
> >> +bool pmu_uncore_identifier_match(const char *id, const char *compat);
> >>
> >>  int perf_pmu__convert_scale(const char *scale, char **end, double *sval);
> >>
> >> --
> >> 1.8.3.1
> >>

Re: [PATCH v8 1/8] perf pmu: "Compat" supports matching multiple identifiers

[Jing Zhang]

在 2023/9/12 上午1:32, Ian Rogers 写道:
> On Sun, Sep 10, 2023 at 7:32 PM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>
>>
>>
>> 在 2023/9/9 上午5:33, Ian Rogers 写道:
>>> On Thu, Sep 7, 2023 at 4:58 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>>>
>>>> The jevent "Compat" is used for uncore PMU alias or metric definitions.
>>>>
>>>> The same PMU driver has different PMU identifiers due to different
>>>> hardware versions and types, but they may have some common PMU event.
>>>> Since a Compat value can only match one identifier, when adding the
>>>> same event alias to PMUs with different identifiers, each identifier
>>>> needs to be defined once, which is not streamlined enough.
>>>>
>>>> So let "Compat" supports matching multiple identifiers for uncore PMU
>>>> alias. For example, the Compat value {43401;436*} can match the PMU
>>>> identifier "43401", that is, CMN600_r0p0, and the PMU identifier with
>>>> the prefix "436", that is, all CMN650, where "*" is a wildcard.
>>>> Tokens in Unit field are delimited by ';' with no spaces.
>>>>
>>>> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
>>>> Reviewed-by: John Garry <john.g.garry@oracle.com>
>>>> ---
>>>>  tools/perf/util/pmu.c | 28 ++++++++++++++++++++++++++--
>>>>  tools/perf/util/pmu.h |  1 +
>>>>  2 files changed, 27 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
>>>> index e215985..c3c3818 100644
>>>> --- a/tools/perf/util/pmu.c
>>>> +++ b/tools/perf/util/pmu.c
>>>> @@ -875,6 +875,30 @@ static bool pmu_uncore_alias_match(const char *pmu_name, const char *name)
>>>>         return res;
>>>>  }
>>>>
>>>> +bool pmu_uncore_identifier_match(const char *id, const char *compat)
>>>> +{
>>>> +       char *tmp = NULL, *tok, *str;
>>>> +       bool res = false;
>>>> +
>>>> +       /*
>>>> +        * The strdup() call is necessary here because "compat" is a const str*
>>>> +        * type and cannot be used as an argument to strtok_r().
>>>> +        */
>>>> +       str = strdup(compat);
>>>> +       if (!str)
>>>> +               return false;
>>>> +
>>>> +       tok = strtok_r(str, ";", &tmp);
>>>
>>> Did the comma vs semicolon difference get explained? It seems to add
>>> inconsistency to use a semicolon.
>>>
>>
>> Hi Ian,
>>
>> Yes, I explained the reason for using semicolons instead of commas in v7.
>>
>> I use a semicolon instead of a comma because I want to distinguish it from the function
>> of the comma in "Unit" and avoid confusion between the use of commas in "Unit" and "Compat".
>> Because in Unit, commas act as wildcards, and in “Compat”, the semicolon means “or”. So
>> I think semicolons are more appropriate.
>>
>> John also raised this issue earlier, and we finally agreed to use semicolons.
>> What do you think?
> 
> I'm okay with it, but thanks for capturing the why of this. I'd like
> at some point to make the wildcarding of things less ad hoc. For
> example, on x86 we use regular expressions to match cpuid:
> https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/arch/x86/mapfile.csv?h=perf-tools-next

Thank you for the example. I was not aware that regular expressions were
already being used for matching in tools/perf.

> but file name style matching for pmus:
> https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/pmu.c?h=perf-tools-next#n1974
> Given that we're okay with regular expressions then I don't see why
> everything shouldn't be a regular expression. This could, for example,
> make matching PMUs more specific than just adding a star and doing a
> file name match. For an example of why this is weird, on my laptop:
> ```
> $ perf stat -e i/actual-frequency/ true
> 
> Performance counter stats for 'system wide':
> 
>                 0      i/actual-frequency/
> 
>       0.001168195 seconds time elapsed
> ```
> The PMU I used here as 'i' is /sys/devices/i915 as we allow arbitrary
> numbers after a PMU name for cases of multiple uncore PMUs.
> 
> My feeling longer term is that the matching distinction of Unit and
> Compat, comma and semi-colon, would be better captured with regular
> expressions as I think they show the intent in the matching more
> clearly.
> 

Yes, using regular expressions is indeed a better choice for consistency and clarity,
and I will try using regular expressions for Compat matching in the next version.

Thanks,
Jing

> Thanks,
> Ian
> 
> 
>> Thanks,
>> Jing
>>
>>> Thanks,
>>> Ian
>>>
>>>> +       for (; tok; tok = strtok_r(NULL, ";", &tmp)) {
>>>> +               if (!fnmatch(tok, id, FNM_CASEFOLD)) {
>>>> +                       res = true;
>>>> +                       break;
>>>> +               }
>>>> +       }
>>>> +       free(str);
>>>> +       return res;
>>>> +}
>>>> +
>>>>  static int pmu_add_cpu_aliases_map_callback(const struct pmu_event *pe,
>>>>                                         const struct pmu_events_table *table __maybe_unused,
>>>>                                         void *vdata)
>>>> @@ -915,8 +939,8 @@ static int pmu_add_sys_aliases_iter_fn(const struct pmu_event *pe,
>>>>         if (!pe->compat || !pe->pmu)
>>>>                 return 0;
>>>>
>>>> -       if (!strcmp(pmu->id, pe->compat) &&
>>>> -           pmu_uncore_alias_match(pe->pmu, pmu->name)) {
>>>> +       if (pmu_uncore_alias_match(pe->pmu, pmu->name) &&
>>>> +                       pmu_uncore_identifier_match(pmu->id, pe->compat)) {
>>>>                 perf_pmu__new_alias(pmu,
>>>>                                 pe->name,
>>>>                                 pe->desc,
>>>> diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
>>>> index bd5d804..1bf5cf1 100644
>>>> --- a/tools/perf/util/pmu.h
>>>> +++ b/tools/perf/util/pmu.h
>>>> @@ -240,6 +240,7 @@ void pmu_add_cpu_aliases_table(struct perf_pmu *pmu,
>>>>  char *perf_pmu__getcpuid(struct perf_pmu *pmu);
>>>>  const struct pmu_events_table *pmu_events_table__find(void);
>>>>  const struct pmu_metrics_table *pmu_metrics_table__find(void);
>>>> +bool pmu_uncore_identifier_match(const char *id, const char *compat);
>>>>
>>>>  int perf_pmu__convert_scale(const char *scale, char **end, double *sval);
>>>>
>>>> --
>>>> 1.8.3.1
>>>>

Re: [PATCH v8 3/8] perf vendor events: Supplement the omitted EventCode

[Jing Zhang]

在 2023/9/13 上午4:34, Arnaldo Carvalho de Melo 写道:
> Em Mon, Sep 11, 2023 at 10:41:16AM +0800, Jing Zhang escreveu:
>>
>> 在 2023/9/9 上午5:18, Ian Rogers 写道:
>>> On Thu, Sep 7, 2023 at 4:58 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>>> If there is an "event=0" in the event description, the EventCode can
>>>> be omitted in the JSON file, and jevent.py will automatically fill in
>>>> "event=0" during parsing.
>>>>
>>>> However, for some events where EventCode and ConfigCode are missing,
>>>> it is not necessary to automatically fill in "event=0", such as the
>>>> CMN event description which is typically "type=xxx, eventid=xxx".
>>>>
>>>> Therefore, before modifying jevent.py to prevent it from automatically
>>>> adding "event=0" by default, it is necessary to fill in all omitted
>>>> EventCodes first.
>>>>
>>>> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
>>> I thought you were going to change the behavior in jevents.py so this
>>> change would be unnecessary. The next time the json is generated by
>>> the script:
>>> https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py
>>> then this will break. It seems easier to workaround the issue in jevents.py.
>>>
>> Okay, I will workaround the issue in jevents.py. Thank you!
> So this means you will resubmit the whole 8-patches long series or
> should we merge this one and then get a followup patch?
> 

Please wait for v9 as there are still some unresolved issues in this series.

Thanks,
Jing

> - Arnaldo
>  
>>> Thanks,
>>> Ian