All of lore.kernel.org
 help / color / mirror / Atom feed
From: TAKANO Mitsuhiro <takano32@gmail.com>
To: linux-perf-users@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>
Cc: Ian Rogers <irogers@google.com>,
	James Clark <james.clark@linaro.org>,
	John Garry <john.garry@linux.dev>,
	linux-arm-kernel@lists.infradead.org,
	Ilkka Koskinen <ilkka@os.amperecomputing.com>,
	linux-kernel@vger.kernel.org,
	TAKANO Mitsuhiro <takano32@gmail.com>,
	Will Deacon <will@kernel.org>, Mike Leach <mike.leach@arm.com>,
	Leo Yan <leo.yan@linux.dev>, Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>,
	Adrian Hunter <adrian.hunter@intel.com>
Subject: [PATCH v11 2/2] perf jevents: Fail the build on a key the tables may not carry
Date: Fri, 20 Nov 2026 06:09:31 +0000	[thread overview]
Message-ID: <20261120060931.711483-3-takano32@gmail.com> (raw)
In-Reply-To: <20261120060931.711483-1-takano32@gmail.com>

jevents.py takes the fields it wants out of each JSON object by name and
ignores the rest, so a key that is spelled differently is dropped without
a word and the object is built without whatever that field held.

Compare the keys of each object with _valid_keys, the keys a table may
carry, and fail on anything else.  The error is one line naming the file
and the key:

  jevents.py: <path>: unknown key(s): ['BriefDescript6ion']

Written with Hermes Agent, an LLM assistant; the author reviewed it and
ran the generator on every arch directory, with and without a key the
tables may not carry.

Signed-off-by: TAKANO Mitsuhiro <takano32@gmail.com>
---
 tools/perf/pmu-events/jevents.py          | 70 ++++++++++++++++++-----
 tools/perf/tests/shell/jevents_bad_key.sh | 43 ++++++++++++++
 2 files changed, 98 insertions(+), 15 deletions(-)
 create mode 100755 tools/perf/tests/shell/jevents_bad_key.sh

diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
index 080e04b51b8c..65e51dafd42b 100755
--- a/tools/perf/pmu-events/jevents.py
+++ b/tools/perf/pmu-events/jevents.py
@@ -63,6 +63,27 @@ _json_metric_attributes = [
 # Attributes that are bools or enum int values, encoded as '0', '1',...
 _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg',
                          'default_show_events']
+# Keys a table under arch/ may carry.  jevents.py takes the fields it wants by
+# name and drops the rest without a word, so a key that is spelled differently
+# is only visible as a field that went missing; a table carrying anything else
+# is an error.  The list holds the keys jevents.py reads, plus the ones the
+# tables carry and it does not read: Counter, CountersNumFixed and
+# CountersNumGeneric (read by nothing) and Experimental (read by metric.py).
+_valid_keys: Set[str] = {
+    'AggregationMode', 'AnyThread', 'ArchStdEvent', 'BriefDescription',
+    'Compat', 'ConfigCode', 'Counter', 'CounterMask',
+    'CountersNumFixed', 'CountersNumGeneric', 'Data_LA', 'DefaultMetricgroupName',
+    'DefaultShowEvents', 'Deprecated', 'EdgeDetect', 'EnAllCores',
+    'EnAllSlices', 'Errata', 'EventCode', 'EventidCode',
+    'EventName', 'Experimental', 'ExtSel', 'FCMask',
+    'Filter', 'Invert', 'LegacyCacheCode', 'LegacyConfigCode',
+    'MetricConstraint', 'MetricExpr', 'MetricGroup', 'MetricgroupNoGroup',
+    'MetricName', 'MetricThreshold', 'MSRIndex', 'MSRValue',
+    'NodeType', 'PEBS', 'PerPkg', 'PortMask',
+    'PublicDescription', 'RdWrMask', 'RetirementLatencyMax', 'RetirementLatencyMean',
+    'RetirementLatencyMin', 'SampleAfterValue', 'ScaleUnit', 'SliceId',
+    'ThreadMask', 'UMask', 'Unit',
+}
 
 def file_name_to_table_name(prefix: str, parents: Sequence[str],
                             dirname: str) -> str:
@@ -196,6 +217,9 @@ class JsonEvent:
   def __init__(self, jd: dict):
     """Constructor passed the dictionary of parsed json values."""
 
+    if extra := set(jd) - _valid_keys:
+      raise ValueError(f'unknown key(s): {sorted(extra)}')
+
     def llx(x: int) -> str:
       """Convert an int to a string similar to a printf modifier of %#llx."""
       return str(x) if x >= 0 and x < 10 else hex(x)
@@ -459,6 +483,8 @@ def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]:
   try:
     with open(path, encoding='utf-8') as f:
       events = json.load(f, object_hook=JsonEvent)
+  except ValueError as exc:
+    raise ValueError(f'{path}: {exc}') from exc
   except BaseException:
     print(f"Exception processing {path}")
     raise
@@ -492,6 +518,8 @@ def preprocess_arch_std_files(archpath: str) -> None:
           _arch_std_events[event.name.lower()] = event
         if event.metric_name:
           _arch_std_events[event.metric_name.lower()] = event
+    except ValueError:
+        raise
     except Exception as e:
         raise RuntimeError(f'Failure processing \'{item.name}\' in \'{archpath}\'') from e
 
@@ -643,7 +671,10 @@ def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
   assert _bcs is not None
   if item.name.endswith('metricgroups.json'):
     with open(item.path, encoding='utf-8') as f:
-      metricgroup_descriptions = json.load(f)
+      try:
+        metricgroup_descriptions = json.load(f)
+      except ValueError as exc:
+        raise ValueError(f'{item.path}: {exc}') from exc
     for mgroup in metricgroup_descriptions:
       assert len(mgroup) > 1, parents
       description = f"{metricgroup_descriptions[mgroup]}\\000"
@@ -1473,6 +1504,8 @@ def main() -> None:
             continue
       try:
         action(parents, item)
+      except ValueError:
+        raise
       except Exception as e:
         raise RuntimeError(f'Action failure for \'{item.name}\' in {parents}') from e
       if item.is_dir():
@@ -1536,20 +1569,27 @@ struct pmu_table_entry {
     if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'):
       tasks.append((item.path, get_topic(item.name)))
 
-  for arch in archs:
-    arch_path = f'{_args.starting_dir}/{arch}'
-    preprocess_arch_std_files(arch_path)
-    ftw(arch_path, [], collect_json)
-
-  with concurrent.futures.ProcessPoolExecutor(max_workers=_parallel_worker_count(len(tasks)),
-                                              initializer=_init_worker,
-                                              initargs=(_arch_std_events,)) as executor:
-    for path, topic, events in executor.map(_parallel_read_json_events, tasks):
-      _json_cache[(path, topic)] = events
-
-  for arch in archs:
-    arch_path = f'{_args.starting_dir}/{arch}'
-    ftw(arch_path, [], preprocess_one_file)
+  try:
+    for arch in archs:
+      arch_path = f'{_args.starting_dir}/{arch}'
+      preprocess_arch_std_files(arch_path)
+      ftw(arch_path, [], collect_json)
+
+    with concurrent.futures.ProcessPoolExecutor(max_workers=_parallel_worker_count(len(tasks)),
+                                                initializer=_init_worker,
+                                                initargs=(_arch_std_events,)) as executor:
+      for path, topic, events in executor.map(_parallel_read_json_events, tasks):
+        _json_cache[(path, topic)] = events
+
+    for arch in archs:
+      arch_path = f'{_args.starting_dir}/{arch}'
+      ftw(arch_path, [], preprocess_one_file)
+  except ValueError as exc:
+    # A worker's traceback is three levels of ProcessPoolExecutor noise around
+    # the one line that matters, so say it here and stop.  make deletes the
+    # target it failed to build (.DELETE_ON_ERROR), so the next run reads the
+    # JSON again rather than a half-written table.
+    sys.exit(f'jevents.py: {exc}')
 
   assert _bcs is not None
   _bcs.compute()
diff --git a/tools/perf/tests/shell/jevents_bad_key.sh b/tools/perf/tests/shell/jevents_bad_key.sh
new file mode 100755
index 000000000000..7848e60b8b4a
--- /dev/null
+++ b/tools/perf/tests/shell/jevents_bad_key.sh
@@ -0,0 +1,43 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check jevents.py reports unknown keys as errors (no_mlock)
+#
+# A key that is spelled differently is dropped without a word, so jevents.py
+# compares the keys of each object in a table against the keys it reads and
+# fails on anything else.
+
+jevents=$(dirname "$0")/../../pmu-events/jevents.py
+python=${PYTHON:-python3}
+
+# An installed perf has the test but not the generator, and a python that is
+# not there or too old to run it is a skip, not a failure of the check.
+[ -r "${jevents}" ] || exit 2
+[ "$("${python}" -c 'import sys; print(sys.version_info >= (3, 9))' 2>/dev/null)" = True ] \
+  || exit 2
+
+tmp=$(mktemp -d) || exit 2
+trap 'rm -rf "${tmp}"' EXIT HUP INT TERM
+tables="${tmp}/arch/x86/model"
+mkdir -p "${tables}" "${tmp}/arch/common"
+printf '[]\n' > "${tmp}/arch/common/empty.json"
+printf 'Family-model,Version,Filename,EventType\nGenuineIntel-6-97,v1.0,model,core\n' \
+  > "${tmp}/arch/x86/mapfile.csv"
+printf '[{"EventName": "a", "NotAKey": "x"}]\n' > "${tables}/events.json"
+
+if "${python}" -B "${jevents}" x86 all "${tmp}/arch" \
+    "${tmp}/pmu-events.c" "${tmp}/pmu-events-string.c" 2> "${tmp}/err"; then
+  echo "jevents.py accepted a key it does not read" >&2
+  exit 1
+fi
+grep -qF -- "NotAKey" "${tmp}/err" || {
+  echo "jevents.py failed without naming the key" >&2
+  cat "${tmp}/err" >&2
+  exit 1
+}
+grep -qF -- "${tables}/events.json" "${tmp}/err" || {
+  echo "jevents.py failed without naming the file" >&2
+  cat "${tmp}/err" >&2
+  exit 1
+}
+
+exit 0
-- 
2.53.0



  parent reply	other threads:[~2026-11-20  6:09 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 22:22 [PATCH 0/3] perf jevents: Report the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-10 22:22 ` [PATCH 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-10 22:44   ` sashiko-bot
2026-11-10 22:22 ` [PATCH 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-10 22:43   ` sashiko-bot
2026-11-10 22:22 ` [PATCH 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-10 22:49   ` sashiko-bot
2026-11-11  2:33     ` TAKANO Mitsuhiro
2026-11-11  2:34 ` [PATCH v2 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  2:34   ` [PATCH v2 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  3:05     ` sashiko-bot
2026-11-11  2:34   ` [PATCH v2 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  3:15     ` sashiko-bot
2026-11-11  2:34   ` [PATCH v2 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  3:31     ` sashiko-bot
2026-11-11  3:43   ` [PATCH v3 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  3:43     ` [PATCH v3 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  3:54       ` sashiko-bot
2026-11-11  3:43     ` [PATCH v3 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  3:50       ` sashiko-bot
2026-11-11  3:43     ` [PATCH v3 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  3:59       ` sashiko-bot
2026-11-11  4:10     ` [PATCH v4 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  4:10       ` [PATCH v4 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  4:22         ` sashiko-bot
2026-11-11  4:10       ` [PATCH v4 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  4:16         ` sashiko-bot
2026-11-11  4:10       ` [PATCH v4 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  4:19         ` sashiko-bot
2026-11-11 13:26       ` [PATCH v5 0/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-11 13:26         ` [PATCH v5 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11 14:11           ` sashiko-bot
2026-11-11 13:26         ` [PATCH v5 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11 14:05           ` sashiko-bot
2026-11-11 13:26         ` [PATCH v5 3/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-11 14:12           ` sashiko-bot
2026-11-11 15:09         ` [PATCH v6 0/3] " TAKANO Mitsuhiro
2026-11-11 15:09           ` [PATCH v6 1/3] perf jevents: Name the metric and the file when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11 16:17             ` sashiko-bot
2026-11-11 15:09           ` [PATCH v6 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11 16:28             ` sashiko-bot
2026-11-11 15:09           ` [PATCH v6 3/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-11 16:36             ` sashiko-bot
2026-11-12 16:28           ` [PATCH v7 0/1] perf jevents: Check the keys of the JSON tables TAKANO Mitsuhiro
2026-11-12 16:28             ` [PATCH v7 1/1] " TAKANO Mitsuhiro
2026-11-12 16:38               ` sashiko-bot
2026-11-12 17:32             ` [PATCH v8 0/1] " TAKANO Mitsuhiro
2026-11-12 17:32               ` [PATCH v8 1/1] " TAKANO Mitsuhiro
2026-11-12 17:46                 ` sashiko-bot
2026-11-12 22:25               ` [PATCH v9 0/1] " TAKANO Mitsuhiro
2026-11-12 22:25                 ` [PATCH v9 1/1] " TAKANO Mitsuhiro
2026-11-12 22:35                   ` sashiko-bot
2026-11-13  3:29                 ` [PATCH v10 0/1] " TAKANO Mitsuhiro
2026-11-13  3:29                   ` [PATCH v10 1/1] " TAKANO Mitsuhiro
2026-11-13  3:44                     ` sashiko-bot
2026-11-19 14:20                     ` James Clark
2026-11-20  6:09                   ` [PATCH v11 0/2] perf jevents: Fail the build on a key the tables may not carry TAKANO Mitsuhiro
2026-11-20  6:09                     ` [PATCH v11 1/2] perf vendor events: Fix misspelled keys in the event tables TAKANO Mitsuhiro
2026-11-20  6:12                       ` sashiko-bot
2026-11-20  6:09                     ` TAKANO Mitsuhiro [this message]
2026-11-20  6:14                       ` [PATCH v11 2/2] perf jevents: Fail the build on a key the tables may not carry sashiko-bot
2026-11-20 10:06                       ` James Clark
2026-11-20 13:42                     ` [PATCH v12 0/2] " TAKANO Mitsuhiro
2026-11-20 13:42                       ` [PATCH v12 1/2] perf vendor events: Fix misspelled keys in the event tables TAKANO Mitsuhiro
2026-11-20 13:44                         ` sashiko-bot
2026-11-20 13:42                       ` [PATCH v12 2/2] perf jevents: Fail the build on a key the tables may not carry TAKANO Mitsuhiro
2026-11-20 13:47                         ` sashiko-bot
2026-11-20 14:58                         ` James Clark

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120060931.711483-3-takano32@gmail.com \
    --to=takano32@gmail.com \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=ilkka@os.amperecomputing.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=john.garry@linux.dev \
    --cc=jolsa@kernel.org \
    --cc=leo.yan@linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mike.leach@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.