[PATCH v5 00/21] Python module cleanup

[PATCH v5 00/21] Python module cleanup

[Ian Rogers]

This patch:
 - removes workarounds for Python 2 module support due to long
   deprecation and challenges in developing new code;
 - constifies variables and parameters to functions;
 - removes python.c stub code which existed due to missing functions
   that are defined in the builtin-* files, in general the builtin-*
   code is moved into util;
 - remove bench and test perf C code from the python module;
 - adds parse_events to the python perf module.
 - improves upon some of the existing python perf module functins.

v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
    caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
    arch_fetch_insn in the process, which was incorrectly using an
    empty weak symbol stub.
v4. Rebase. Fix the parse events evsel to be embedded in a
    pyrf_evsel. Add __str__/__repr__ functions to evlist and
    evsel. Throw an exception for a bad evlist index.
v3. Move is_directory_at to patch 6 rather than patch 7, respond to
    review feedback on the list from Namhyung.
v2. Add the bottom 4 bullet points - 13 more patches.

Ian Rogers (21):
  perf python: Remove python 2 scripting support
  perf python: Constify variables and parameters
  perf python: Remove unused #include
  perf script: Move scripting_max_stack out of builtin
  perf kvm: Move functions used in util out of builtin
  perf script: Move find_scripts to browser/scripts.c
  perf stat: Move stat_config into config.c
  perf script: Move script_spec code to trace-event-scripting.c
  perf script: Move script_fetch_insn to trace-event-scripting.c
  perf script: Move perf_sample__sprintf_flags to
    trace-event-scripting.c
  perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
  perf intel-pt: Remove stale build comment
  perf env: Move arch errno function to only use in env
  perf lock: Move common lock contention code to new file
  perf bench: Remove reference to cmd_inject
  perf kwork: Make perf_kwork_add_work a callback
  perf build: Remove test library from python shared object
  perf python: Add parse_events function
  perf python: Add __str__ and __repr__ functions to evlist
  perf python: Add __str__ and __repr__ functions to evsel
  perf python: Correctly throw IndexError

 tools/perf/Makefile.perf                      |   7 +-
 tools/perf/arch/x86/util/Build                |   2 +-
 tools/perf/bench/inject-buildid.c             |  13 +-
 tools/perf/builtin-kvm.c                      |  61 ----
 tools/perf/builtin-kwork.c                    |   3 +-
 tools/perf/builtin-lock.c                     | 137 +------
 tools/perf/builtin-script.c                   | 304 +---------------
 tools/perf/builtin-stat.c                     |  27 --
 tools/perf/builtin-trace.c                    |   1 -
 tools/perf/builtin.h                          |   6 -
 .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
 tools/perf/tests/stat.c                       |  16 +-
 tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
 tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
 tools/perf/util/Build                         |   4 +-
 tools/perf/util/bpf_kwork.c                   |   2 +-
 tools/perf/util/bpf_kwork_top.c               |   2 +-
 tools/perf/util/bpf_lock_contention.c         |   2 +-
 tools/perf/util/cgroup.c                      |   2 +-
 tools/perf/util/config.c                      |  27 ++
 tools/perf/util/dlfilter.c                    |   3 +-
 tools/perf/util/env.c                         |   4 +
 tools/perf/util/env.h                         |   2 -
 tools/perf/util/evsel.c                       |  19 +-
 tools/perf/util/evsel.h                       |   2 +-
 tools/perf/util/intel-pt-decoder/Build        |  18 +-
 .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
 tools/perf/util/kvm-stat.c                    |  70 ++++
 tools/perf/util/kvm-stat.h                    |   3 +
 tools/perf/util/kwork.h                       |   6 +-
 tools/perf/util/lock-contention.c             | 170 +++++++++
 tools/perf/util/lock-contention.h             |  37 +-
 tools/perf/util/path.c                        |  10 +
 tools/perf/util/path.h                        |   1 +
 tools/perf/util/python.c                      | 338 ++++++++----------
 .../scripting-engines/trace-event-python.c    |  63 +---
 tools/perf/util/stat.h                        |   3 +-
 tools/perf/util/trace-event-scripting.c       | 177 +++++++++
 tools/perf/util/trace-event.h                 |   5 +-
 39 files changed, 872 insertions(+), 878 deletions(-)
 create mode 100644 tools/perf/util/kvm-stat.c
 create mode 100644 tools/perf/util/lock-contention.c

-- 
2.47.0.163.g1226f6d8fa-goog

[PATCH v5 01/21] perf python: Remove python 2 scripting support

[Ian Rogers]

Python2 was deprecated 4 years ago, remove support and workarounds.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 .../scripts/python/Perf-Trace-Util/Context.c  | 18 -----
 tools/perf/util/python.c                      | 73 +++----------------
 .../scripting-engines/trace-event-python.c    | 63 +---------------
 3 files changed, 15 insertions(+), 139 deletions(-)

diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
index 01f54d6724a5..d742daaa5d5a 100644
--- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
+++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
@@ -24,16 +24,6 @@
 #include "../../../util/srcline.h"
 #include "../../../util/srccode.h"
 
-#if PY_MAJOR_VERSION < 3
-#define _PyCapsule_GetPointer(arg1, arg2) \
-  PyCObject_AsVoidPtr(arg1)
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
-  PyString_FromStringAndSize((arg1), (arg2))
-#define _PyUnicode_AsUTF8(arg) \
-  PyString_AsString(arg)
-
-PyMODINIT_FUNC initperf_trace_context(void);
-#else
 #define _PyCapsule_GetPointer(arg1, arg2) \
   PyCapsule_GetPointer((arg1), (arg2))
 #define _PyBytes_FromStringAndSize(arg1, arg2) \
@@ -42,7 +32,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
   PyUnicode_AsUTF8(arg)
 
 PyMODINIT_FUNC PyInit_perf_trace_context(void);
-#endif
 
 static struct scripting_context *get_args(PyObject *args, const char *name, PyObject **arg2)
 {
@@ -213,12 +202,6 @@ static PyMethodDef ContextMethods[] = {
 	{ NULL, NULL, 0, NULL}
 };
 
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf_trace_context(void)
-{
-	(void) Py_InitModule("perf_trace_context", ContextMethods);
-}
-#else
 PyMODINIT_FUNC PyInit_perf_trace_context(void)
 {
 	static struct PyModuleDef moduledef = {
@@ -240,4 +223,3 @@ PyMODINIT_FUNC PyInit_perf_trace_context(void)
 
 	return mod;
 }
-#endif
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 31a223eaf8e6..02279ab4967c 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -25,40 +25,14 @@
 #include <internal/lib.h>
 #include "../builtin.h"
 
-#if PY_MAJOR_VERSION < 3
-#define _PyUnicode_FromString(arg) \
-  PyString_FromString(arg)
-#define _PyUnicode_AsString(arg) \
-  PyString_AsString(arg)
-#define _PyUnicode_FromFormat(...) \
-  PyString_FromFormat(__VA_ARGS__)
-#define _PyLong_FromLong(arg) \
-  PyInt_FromLong(arg)
-
-#else
-
 #define _PyUnicode_FromString(arg) \
   PyUnicode_FromString(arg)
 #define _PyUnicode_FromFormat(...) \
   PyUnicode_FromFormat(__VA_ARGS__)
 #define _PyLong_FromLong(arg) \
   PyLong_FromLong(arg)
-#endif
 
-#ifndef Py_TYPE
-#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
-#endif
-
-/* Define PyVarObject_HEAD_INIT for python 2.5 */
-#ifndef PyVarObject_HEAD_INIT
-# define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size,
-#endif
-
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf(void);
-#else
 PyMODINIT_FUNC PyInit_perf(void);
-#endif
 
 #define member_def(type, member, ptype, help) \
 	{ #member, ptype, \
@@ -116,7 +90,7 @@ static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent)
 		     pevent->event.mmap.pgoff, pevent->event.mmap.filename) < 0) {
 		ret = PyErr_NoMemory();
 	} else {
-		ret = _PyUnicode_FromString(s);
+		ret = PyUnicode_FromString(s);
 		free(s);
 	}
 	return ret;
@@ -147,7 +121,7 @@ static PyMemberDef pyrf_task_event__members[] = {
 
 static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent)
 {
-	return _PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
+	return PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
 				   "ptid: %u, time: %" PRI_lu64 "}",
 				   pevent->event.header.type == PERF_RECORD_FORK ? "fork" : "exit",
 				   pevent->event.fork.pid,
@@ -180,7 +154,7 @@ static PyMemberDef pyrf_comm_event__members[] = {
 
 static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent)
 {
-	return _PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
+	return PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
 				   pevent->event.comm.pid,
 				   pevent->event.comm.tid,
 				   pevent->event.comm.comm);
@@ -211,7 +185,7 @@ static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent)
 {
 	struct perf_record_throttle *te = (struct perf_record_throttle *)(&pevent->event.header + 1);
 
-	return _PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
+	return PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
 				   ", stream_id: %" PRI_lu64 " }",
 				   pevent->event.header.type == PERF_RECORD_THROTTLE ? "" : "un",
 				   te->time, te->id, te->stream_id);
@@ -246,7 +220,7 @@ static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
 		     pevent->event.lost.id, pevent->event.lost.lost) < 0) {
 		ret = PyErr_NoMemory();
 	} else {
-		ret = _PyUnicode_FromString(s);
+		ret = PyUnicode_FromString(s);
 		free(s);
 	}
 	return ret;
@@ -273,7 +247,7 @@ static PyMemberDef pyrf_read_event__members[] = {
 
 static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
 {
-	return _PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
+	return PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
 				   pevent->event.read.pid,
 				   pevent->event.read.tid);
 	/*
@@ -308,7 +282,7 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
 	if (asprintf(&s, "{ type: sample }") < 0) {
 		ret = PyErr_NoMemory();
 	} else {
-		ret = _PyUnicode_FromString(s);
+		ret = PyUnicode_FromString(s);
 		free(s);
 	}
 	return ret;
@@ -342,7 +316,7 @@ tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
 		}
 		if (field->flags & TEP_FIELD_IS_STRING &&
 		    is_printable_array(data + offset, len)) {
-			ret = _PyUnicode_FromString((char *)data + offset);
+			ret = PyUnicode_FromString((char *)data + offset);
 		} else {
 			ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
 			field->flags &= ~TEP_FIELD_IS_STRING;
@@ -431,7 +405,7 @@ static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent)
 		     !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT)) < 0) {
 		ret = PyErr_NoMemory();
 	} else {
-		ret = _PyUnicode_FromString(s);
+		ret = PyUnicode_FromString(s);
 		free(s);
 	}
 	return ret;
@@ -917,17 +891,8 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,
 
 	for (i = 0; i < evlist->core.pollfd.nr; ++i) {
 		PyObject *file;
-#if PY_MAJOR_VERSION < 3
-		FILE *fp = fdopen(evlist->core.pollfd.entries[i].fd, "r");
-
-		if (fp == NULL)
-			goto free_list;
-
-		file = PyFile_FromFile(fp, "perf", "r", NULL);
-#else
 		file = PyFile_FromFd(evlist->core.pollfd.entries[i].fd, "perf", "r", -1,
 				     NULL, NULL, NULL, 0);
-#endif
 		if (file == NULL)
 			goto free_list;
 
@@ -1233,9 +1198,9 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
 
 	tp_format = trace_event__tp_format(sys, name);
 	if (IS_ERR(tp_format))
-		return _PyLong_FromLong(-1);
+		return PyLong_FromLong(-1);
 
-	return _PyLong_FromLong(tp_format->id);
+	return PyLong_FromLong(tp_format->id);
 #endif // HAVE_LIBTRACEEVENT
 }
 
@@ -1249,18 +1214,11 @@ static PyMethodDef perf__methods[] = {
 	{ .ml_name = NULL, }
 };
 
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf(void)
-#else
 PyMODINIT_FUNC PyInit_perf(void)
-#endif
 {
 	PyObject *obj;
 	int i;
 	PyObject *dict;
-#if PY_MAJOR_VERSION < 3
-	PyObject *module = Py_InitModule("perf", perf__methods);
-#else
 	static struct PyModuleDef moduledef = {
 		PyModuleDef_HEAD_INIT,
 		"perf",			/* m_name */
@@ -1273,7 +1231,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 		NULL,			/* m_free */
 	};
 	PyObject *module = PyModule_Create(&moduledef);
-#endif
 
 	if (module == NULL ||
 	    pyrf_event__setup_types() < 0 ||
@@ -1281,11 +1238,7 @@ PyMODINIT_FUNC PyInit_perf(void)
 	    pyrf_evsel__setup_types() < 0 ||
 	    pyrf_thread_map__setup_types() < 0 ||
 	    pyrf_cpu_map__setup_types() < 0)
-#if PY_MAJOR_VERSION < 3
-		return;
-#else
 		return module;
-#endif
 
 	/* The page_size is placed in util object. */
 	page_size = sysconf(_SC_PAGE_SIZE);
@@ -1334,7 +1287,7 @@ PyMODINIT_FUNC PyInit_perf(void)
 		goto error;
 
 	for (i = 0; perf__constants[i].name != NULL; i++) {
-		obj = _PyLong_FromLong(perf__constants[i].value);
+		obj = PyLong_FromLong(perf__constants[i].value);
 		if (obj == NULL)
 			goto error;
 		PyDict_SetItemString(dict, perf__constants[i].name, obj);
@@ -1344,9 +1297,7 @@ PyMODINIT_FUNC PyInit_perf(void)
 error:
 	if (PyErr_Occurred())
 		PyErr_SetString(PyExc_ImportError, "perf: Init failed!");
-#if PY_MAJOR_VERSION >= 3
 	return module;
-#endif
 }
 
 
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index d7183134b669..f1d461d47d73 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -58,22 +58,6 @@
 #include "mem-events.h"
 #include "util/perf_regs.h"
 
-#if PY_MAJOR_VERSION < 3
-#define _PyUnicode_FromString(arg) \
-  PyString_FromString(arg)
-#define _PyUnicode_FromStringAndSize(arg1, arg2) \
-  PyString_FromStringAndSize((arg1), (arg2))
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
-  PyString_FromStringAndSize((arg1), (arg2))
-#define _PyLong_FromLong(arg) \
-  PyInt_FromLong(arg)
-#define _PyLong_AsLong(arg) \
-  PyInt_AsLong(arg)
-#define _PyCapsule_New(arg1, arg2, arg3) \
-  PyCObject_FromVoidPtr((arg1), (arg2))
-
-PyMODINIT_FUNC initperf_trace_context(void);
-#else
 #define _PyUnicode_FromString(arg) \
   PyUnicode_FromString(arg)
 #define _PyUnicode_FromStringAndSize(arg1, arg2) \
@@ -88,7 +72,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
   PyCapsule_New((arg1), (arg2), (arg3))
 
 PyMODINIT_FUNC PyInit_perf_trace_context(void);
-#endif
 
 #ifdef HAVE_LIBTRACEEVENT
 #define TRACE_EVENT_TYPE_MAX				\
@@ -181,17 +164,7 @@ static int get_argument_count(PyObject *handler)
 {
 	int arg_count = 0;
 
-	/*
-	 * The attribute for the code object is func_code in Python 2,
-	 * whereas it is __code__ in Python 3.0+.
-	 */
-	PyObject *code_obj = PyObject_GetAttrString(handler,
-		"func_code");
-	if (PyErr_Occurred()) {
-		PyErr_Clear();
-		code_obj = PyObject_GetAttrString(handler,
-			"__code__");
-	}
+	PyObject *code_obj = code_obj = PyObject_GetAttrString(handler, "__code__");
 	PyErr_Clear();
 	if (code_obj) {
 		PyObject *arg_count_obj = PyObject_GetAttrString(code_obj,
@@ -1899,12 +1872,6 @@ static void set_table_handlers(struct tables *tables)
 	tables->synth_handler = get_handler("synth_data");
 }
 
-#if PY_MAJOR_VERSION < 3
-static void _free_command_line(const char **command_line, int num)
-{
-	free(command_line);
-}
-#else
 static void _free_command_line(wchar_t **command_line, int num)
 {
 	int i;
@@ -1912,7 +1879,6 @@ static void _free_command_line(wchar_t **command_line, int num)
 		PyMem_RawFree(command_line[i]);
 	free(command_line);
 }
-#endif
 
 
 /*
@@ -1922,30 +1888,12 @@ static int python_start_script(const char *script, int argc, const char **argv,
 			       struct perf_session *session)
 {
 	struct tables *tables = &tables_global;
-#if PY_MAJOR_VERSION < 3
-	const char **command_line;
-#else
 	wchar_t **command_line;
-#endif
-	/*
-	 * Use a non-const name variable to cope with python 2.6's
-	 * PyImport_AppendInittab prototype
-	 */
-	char buf[PATH_MAX], name[19] = "perf_trace_context";
+	char buf[PATH_MAX];
 	int i, err = 0;
 	FILE *fp;
 
 	scripting_context->session = session;
-#if PY_MAJOR_VERSION < 3
-	command_line = malloc((argc + 1) * sizeof(const char *));
-	if (!command_line)
-		return -1;
-
-	command_line[0] = script;
-	for (i = 1; i < argc + 1; i++)
-		command_line[i] = argv[i - 1];
-	PyImport_AppendInittab(name, initperf_trace_context);
-#else
 	command_line = malloc((argc + 1) * sizeof(wchar_t *));
 	if (!command_line)
 		return -1;
@@ -1953,15 +1901,10 @@ static int python_start_script(const char *script, int argc, const char **argv,
 	command_line[0] = Py_DecodeLocale(script, NULL);
 	for (i = 1; i < argc + 1; i++)
 		command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
-	PyImport_AppendInittab(name, PyInit_perf_trace_context);
-#endif
+	PyImport_AppendInittab("perf_trace_context", PyInit_perf_trace_context);
 	Py_Initialize();
 
-#if PY_MAJOR_VERSION < 3
-	PySys_SetArgv(argc + 1, (char **)command_line);
-#else
 	PySys_SetArgv(argc + 1, command_line);
-#endif
 
 	fp = fopen(script, "r");
 	if (!fp) {
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 01/21] perf python: Remove python 2 scripting support

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:32PM -0700, Ian Rogers wrote:
> Python2 was deprecated 4 years ago, remove support and workarounds.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  .../scripts/python/Perf-Trace-Util/Context.c  | 18 -----
>  tools/perf/util/python.c                      | 73 +++----------------
>  .../scripting-engines/trace-event-python.c    | 63 +---------------
>  3 files changed, 15 insertions(+), 139 deletions(-)
> 
> diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> index 01f54d6724a5..d742daaa5d5a 100644
> --- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> +++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> @@ -24,16 +24,6 @@
>  #include "../../../util/srcline.h"
>  #include "../../../util/srccode.h"
>  
> -#if PY_MAJOR_VERSION < 3
> -#define _PyCapsule_GetPointer(arg1, arg2) \
> -  PyCObject_AsVoidPtr(arg1)
> -#define _PyBytes_FromStringAndSize(arg1, arg2) \
> -  PyString_FromStringAndSize((arg1), (arg2))
> -#define _PyUnicode_AsUTF8(arg) \
> -  PyString_AsString(arg)
> -
> -PyMODINIT_FUNC initperf_trace_context(void);
> -#else
>  #define _PyCapsule_GetPointer(arg1, arg2) \
>    PyCapsule_GetPointer((arg1), (arg2))
>  #define _PyBytes_FromStringAndSize(arg1, arg2) \
> @@ -42,7 +32,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
>    PyUnicode_AsUTF8(arg)
>  
>  PyMODINIT_FUNC PyInit_perf_trace_context(void);
> -#endif
>  
>  static struct scripting_context *get_args(PyObject *args, const char *name, PyObject **arg2)
>  {
> @@ -213,12 +202,6 @@ static PyMethodDef ContextMethods[] = {
>  	{ NULL, NULL, 0, NULL}
>  };
>  
> -#if PY_MAJOR_VERSION < 3
> -PyMODINIT_FUNC initperf_trace_context(void)
> -{
> -	(void) Py_InitModule("perf_trace_context", ContextMethods);
> -}
> -#else
>  PyMODINIT_FUNC PyInit_perf_trace_context(void)
>  {
>  	static struct PyModuleDef moduledef = {
> @@ -240,4 +223,3 @@ PyMODINIT_FUNC PyInit_perf_trace_context(void)
>  
>  	return mod;
>  }
> -#endif
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 31a223eaf8e6..02279ab4967c 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -25,40 +25,14 @@
>  #include <internal/lib.h>
>  #include "../builtin.h"
>  
> -#if PY_MAJOR_VERSION < 3
> -#define _PyUnicode_FromString(arg) \
> -  PyString_FromString(arg)
> -#define _PyUnicode_AsString(arg) \
> -  PyString_AsString(arg)
> -#define _PyUnicode_FromFormat(...) \
> -  PyString_FromFormat(__VA_ARGS__)
> -#define _PyLong_FromLong(arg) \
> -  PyInt_FromLong(arg)
> -
> -#else
> -
>  #define _PyUnicode_FromString(arg) \
>    PyUnicode_FromString(arg)
>  #define _PyUnicode_FromFormat(...) \
>    PyUnicode_FromFormat(__VA_ARGS__)
>  #define _PyLong_FromLong(arg) \
>    PyLong_FromLong(arg)
> -#endif
>  
> -#ifndef Py_TYPE
> -#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
> -#endif
> -
> -/* Define PyVarObject_HEAD_INIT for python 2.5 */
> -#ifndef PyVarObject_HEAD_INIT
> -# define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size,
> -#endif
> -
> -#if PY_MAJOR_VERSION < 3
> -PyMODINIT_FUNC initperf(void);
> -#else
>  PyMODINIT_FUNC PyInit_perf(void);
> -#endif
>  
>  #define member_def(type, member, ptype, help) \
>  	{ #member, ptype, \
> @@ -116,7 +90,7 @@ static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent)
>  		     pevent->event.mmap.pgoff, pevent->event.mmap.filename) < 0) {
>  		ret = PyErr_NoMemory();
>  	} else {
> -		ret = _PyUnicode_FromString(s);
> +		ret = PyUnicode_FromString(s);
>  		free(s);
>  	}
>  	return ret;
> @@ -147,7 +121,7 @@ static PyMemberDef pyrf_task_event__members[] = {
>  
>  static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent)
>  {
> -	return _PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
> +	return PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
>  				   "ptid: %u, time: %" PRI_lu64 "}",
>  				   pevent->event.header.type == PERF_RECORD_FORK ? "fork" : "exit",
>  				   pevent->event.fork.pid,
> @@ -180,7 +154,7 @@ static PyMemberDef pyrf_comm_event__members[] = {
>  
>  static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent)
>  {
> -	return _PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
> +	return PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
>  				   pevent->event.comm.pid,
>  				   pevent->event.comm.tid,
>  				   pevent->event.comm.comm);
> @@ -211,7 +185,7 @@ static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent)
>  {
>  	struct perf_record_throttle *te = (struct perf_record_throttle *)(&pevent->event.header + 1);
>  
> -	return _PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
> +	return PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
>  				   ", stream_id: %" PRI_lu64 " }",
>  				   pevent->event.header.type == PERF_RECORD_THROTTLE ? "" : "un",
>  				   te->time, te->id, te->stream_id);
> @@ -246,7 +220,7 @@ static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
>  		     pevent->event.lost.id, pevent->event.lost.lost) < 0) {
>  		ret = PyErr_NoMemory();
>  	} else {
> -		ret = _PyUnicode_FromString(s);
> +		ret = PyUnicode_FromString(s);
>  		free(s);
>  	}
>  	return ret;
> @@ -273,7 +247,7 @@ static PyMemberDef pyrf_read_event__members[] = {
>  
>  static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
>  {
> -	return _PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
> +	return PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
>  				   pevent->event.read.pid,
>  				   pevent->event.read.tid);
>  	/*
> @@ -308,7 +282,7 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
>  	if (asprintf(&s, "{ type: sample }") < 0) {
>  		ret = PyErr_NoMemory();
>  	} else {
> -		ret = _PyUnicode_FromString(s);
> +		ret = PyUnicode_FromString(s);
>  		free(s);
>  	}
>  	return ret;
> @@ -342,7 +316,7 @@ tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
>  		}
>  		if (field->flags & TEP_FIELD_IS_STRING &&
>  		    is_printable_array(data + offset, len)) {
> -			ret = _PyUnicode_FromString((char *)data + offset);
> +			ret = PyUnicode_FromString((char *)data + offset);
>  		} else {
>  			ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
>  			field->flags &= ~TEP_FIELD_IS_STRING;
> @@ -431,7 +405,7 @@ static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent)
>  		     !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT)) < 0) {
>  		ret = PyErr_NoMemory();
>  	} else {
> -		ret = _PyUnicode_FromString(s);
> +		ret = PyUnicode_FromString(s);
>  		free(s);
>  	}
>  	return ret;
> @@ -917,17 +891,8 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,
>  
>  	for (i = 0; i < evlist->core.pollfd.nr; ++i) {
>  		PyObject *file;
> -#if PY_MAJOR_VERSION < 3
> -		FILE *fp = fdopen(evlist->core.pollfd.entries[i].fd, "r");
> -
> -		if (fp == NULL)
> -			goto free_list;
> -
> -		file = PyFile_FromFile(fp, "perf", "r", NULL);
> -#else
>  		file = PyFile_FromFd(evlist->core.pollfd.entries[i].fd, "perf", "r", -1,
>  				     NULL, NULL, NULL, 0);
> -#endif
>  		if (file == NULL)
>  			goto free_list;
>  
> @@ -1233,9 +1198,9 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
>  
>  	tp_format = trace_event__tp_format(sys, name);
>  	if (IS_ERR(tp_format))
> -		return _PyLong_FromLong(-1);
> +		return PyLong_FromLong(-1);
>  
> -	return _PyLong_FromLong(tp_format->id);
> +	return PyLong_FromLong(tp_format->id);
>  #endif // HAVE_LIBTRACEEVENT
>  }
>  
> @@ -1249,18 +1214,11 @@ static PyMethodDef perf__methods[] = {
>  	{ .ml_name = NULL, }
>  };
>  
> -#if PY_MAJOR_VERSION < 3
> -PyMODINIT_FUNC initperf(void)
> -#else
>  PyMODINIT_FUNC PyInit_perf(void)
> -#endif
>  {
>  	PyObject *obj;
>  	int i;
>  	PyObject *dict;
> -#if PY_MAJOR_VERSION < 3
> -	PyObject *module = Py_InitModule("perf", perf__methods);
> -#else
>  	static struct PyModuleDef moduledef = {
>  		PyModuleDef_HEAD_INIT,
>  		"perf",			/* m_name */
> @@ -1273,7 +1231,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  		NULL,			/* m_free */
>  	};
>  	PyObject *module = PyModule_Create(&moduledef);
> -#endif
>  
>  	if (module == NULL ||
>  	    pyrf_event__setup_types() < 0 ||
> @@ -1281,11 +1238,7 @@ PyMODINIT_FUNC PyInit_perf(void)
>  	    pyrf_evsel__setup_types() < 0 ||
>  	    pyrf_thread_map__setup_types() < 0 ||
>  	    pyrf_cpu_map__setup_types() < 0)
> -#if PY_MAJOR_VERSION < 3
> -		return;
> -#else
>  		return module;
> -#endif
>  
>  	/* The page_size is placed in util object. */
>  	page_size = sysconf(_SC_PAGE_SIZE);
> @@ -1334,7 +1287,7 @@ PyMODINIT_FUNC PyInit_perf(void)
>  		goto error;
>  
>  	for (i = 0; perf__constants[i].name != NULL; i++) {
> -		obj = _PyLong_FromLong(perf__constants[i].value);
> +		obj = PyLong_FromLong(perf__constants[i].value);
>  		if (obj == NULL)
>  			goto error;
>  		PyDict_SetItemString(dict, perf__constants[i].name, obj);
> @@ -1344,9 +1297,7 @@ PyMODINIT_FUNC PyInit_perf(void)
>  error:
>  	if (PyErr_Occurred())
>  		PyErr_SetString(PyExc_ImportError, "perf: Init failed!");
> -#if PY_MAJOR_VERSION >= 3
>  	return module;
> -#endif
>  }
>  
>  
> diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
> index d7183134b669..f1d461d47d73 100644
> --- a/tools/perf/util/scripting-engines/trace-event-python.c
> +++ b/tools/perf/util/scripting-engines/trace-event-python.c
> @@ -58,22 +58,6 @@
>  #include "mem-events.h"
>  #include "util/perf_regs.h"
>  
> -#if PY_MAJOR_VERSION < 3
> -#define _PyUnicode_FromString(arg) \
> -  PyString_FromString(arg)
> -#define _PyUnicode_FromStringAndSize(arg1, arg2) \
> -  PyString_FromStringAndSize((arg1), (arg2))
> -#define _PyBytes_FromStringAndSize(arg1, arg2) \
> -  PyString_FromStringAndSize((arg1), (arg2))
> -#define _PyLong_FromLong(arg) \
> -  PyInt_FromLong(arg)
> -#define _PyLong_AsLong(arg) \
> -  PyInt_AsLong(arg)
> -#define _PyCapsule_New(arg1, arg2, arg3) \
> -  PyCObject_FromVoidPtr((arg1), (arg2))
> -
> -PyMODINIT_FUNC initperf_trace_context(void);
> -#else
>  #define _PyUnicode_FromString(arg) \
>    PyUnicode_FromString(arg)
>  #define _PyUnicode_FromStringAndSize(arg1, arg2) \
> @@ -88,7 +72,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
>    PyCapsule_New((arg1), (arg2), (arg3))
>  
>  PyMODINIT_FUNC PyInit_perf_trace_context(void);
> -#endif
>  
>  #ifdef HAVE_LIBTRACEEVENT
>  #define TRACE_EVENT_TYPE_MAX				\
> @@ -181,17 +164,7 @@ static int get_argument_count(PyObject *handler)
>  {
>  	int arg_count = 0;
>  
> -	/*
> -	 * The attribute for the code object is func_code in Python 2,
> -	 * whereas it is __code__ in Python 3.0+.
> -	 */
> -	PyObject *code_obj = PyObject_GetAttrString(handler,
> -		"func_code");
> -	if (PyErr_Occurred()) {
> -		PyErr_Clear();
> -		code_obj = PyObject_GetAttrString(handler,
> -			"__code__");
> -	}
> +	PyObject *code_obj = code_obj = PyObject_GetAttrString(handler, "__code__");
>  	PyErr_Clear();
>  	if (code_obj) {
>  		PyObject *arg_count_obj = PyObject_GetAttrString(code_obj,
> @@ -1899,12 +1872,6 @@ static void set_table_handlers(struct tables *tables)
>  	tables->synth_handler = get_handler("synth_data");
>  }
>  
> -#if PY_MAJOR_VERSION < 3
> -static void _free_command_line(const char **command_line, int num)
> -{
> -	free(command_line);
> -}
> -#else
>  static void _free_command_line(wchar_t **command_line, int num)
>  {
>  	int i;
> @@ -1912,7 +1879,6 @@ static void _free_command_line(wchar_t **command_line, int num)
>  		PyMem_RawFree(command_line[i]);
>  	free(command_line);
>  }
> -#endif
>  
>  
>  /*
> @@ -1922,30 +1888,12 @@ static int python_start_script(const char *script, int argc, const char **argv,
>  			       struct perf_session *session)
>  {
>  	struct tables *tables = &tables_global;
> -#if PY_MAJOR_VERSION < 3
> -	const char **command_line;
> -#else
>  	wchar_t **command_line;
> -#endif
> -	/*
> -	 * Use a non-const name variable to cope with python 2.6's
> -	 * PyImport_AppendInittab prototype
> -	 */
> -	char buf[PATH_MAX], name[19] = "perf_trace_context";
> +	char buf[PATH_MAX];
>  	int i, err = 0;
>  	FILE *fp;
>  
>  	scripting_context->session = session;
> -#if PY_MAJOR_VERSION < 3
> -	command_line = malloc((argc + 1) * sizeof(const char *));
> -	if (!command_line)
> -		return -1;
> -
> -	command_line[0] = script;
> -	for (i = 1; i < argc + 1; i++)
> -		command_line[i] = argv[i - 1];
> -	PyImport_AppendInittab(name, initperf_trace_context);
> -#else
>  	command_line = malloc((argc + 1) * sizeof(wchar_t *));
>  	if (!command_line)
>  		return -1;
> @@ -1953,15 +1901,10 @@ static int python_start_script(const char *script, int argc, const char **argv,
>  	command_line[0] = Py_DecodeLocale(script, NULL);
>  	for (i = 1; i < argc + 1; i++)
>  		command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
> -	PyImport_AppendInittab(name, PyInit_perf_trace_context);
> -#endif
> +	PyImport_AppendInittab("perf_trace_context", PyInit_perf_trace_context);
>  	Py_Initialize();
>  
> -#if PY_MAJOR_VERSION < 3
> -	PySys_SetArgv(argc + 1, (char **)command_line);
> -#else
>  	PySys_SetArgv(argc + 1, command_line);
> -#endif
>  
>  	fp = fopen(script, "r");
>  	if (!fp) {
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 02/21] perf python: Constify variables and parameters

[Ian Rogers]

Opportunistically constify variables and parameters when possible.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/python.c | 55 +++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 26 deletions(-)

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 02279ab4967c..13dad27169a0 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -62,7 +62,7 @@ struct pyrf_event {
 	sample_member_def(sample_period, period, T_ULONGLONG, "event period"),		 \
 	sample_member_def(sample_cpu, cpu, T_UINT, "event cpu"),
 
-static char pyrf_mmap_event__doc[] = PyDoc_STR("perf mmap event object.");
+static const char pyrf_mmap_event__doc[] = PyDoc_STR("perf mmap event object.");
 
 static PyMemberDef pyrf_mmap_event__members[] = {
 	sample_members
@@ -77,7 +77,7 @@ static PyMemberDef pyrf_mmap_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_mmap_event__repr(const struct pyrf_event *pevent)
 {
 	PyObject *ret;
 	char *s;
@@ -106,7 +106,7 @@ static PyTypeObject pyrf_mmap_event__type = {
 	.tp_repr	= (reprfunc)pyrf_mmap_event__repr,
 };
 
-static char pyrf_task_event__doc[] = PyDoc_STR("perf task (fork/exit) event object.");
+static const char pyrf_task_event__doc[] = PyDoc_STR("perf task (fork/exit) event object.");
 
 static PyMemberDef pyrf_task_event__members[] = {
 	sample_members
@@ -119,7 +119,7 @@ static PyMemberDef pyrf_task_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_task_event__repr(const struct pyrf_event *pevent)
 {
 	return PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
 				   "ptid: %u, time: %" PRI_lu64 "}",
@@ -141,7 +141,7 @@ static PyTypeObject pyrf_task_event__type = {
 	.tp_repr	= (reprfunc)pyrf_task_event__repr,
 };
 
-static char pyrf_comm_event__doc[] = PyDoc_STR("perf comm event object.");
+static const char pyrf_comm_event__doc[] = PyDoc_STR("perf comm event object.");
 
 static PyMemberDef pyrf_comm_event__members[] = {
 	sample_members
@@ -152,7 +152,7 @@ static PyMemberDef pyrf_comm_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_comm_event__repr(const struct pyrf_event *pevent)
 {
 	return PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
 				   pevent->event.comm.pid,
@@ -170,7 +170,7 @@ static PyTypeObject pyrf_comm_event__type = {
 	.tp_repr	= (reprfunc)pyrf_comm_event__repr,
 };
 
-static char pyrf_throttle_event__doc[] = PyDoc_STR("perf throttle event object.");
+static const char pyrf_throttle_event__doc[] = PyDoc_STR("perf throttle event object.");
 
 static PyMemberDef pyrf_throttle_event__members[] = {
 	sample_members
@@ -181,9 +181,10 @@ static PyMemberDef pyrf_throttle_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_throttle_event__repr(const struct pyrf_event *pevent)
 {
-	struct perf_record_throttle *te = (struct perf_record_throttle *)(&pevent->event.header + 1);
+	const struct perf_record_throttle *te = (const struct perf_record_throttle *)
+		(&pevent->event.header + 1);
 
 	return PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
 				   ", stream_id: %" PRI_lu64 " }",
@@ -201,7 +202,7 @@ static PyTypeObject pyrf_throttle_event__type = {
 	.tp_repr	= (reprfunc)pyrf_throttle_event__repr,
 };
 
-static char pyrf_lost_event__doc[] = PyDoc_STR("perf lost event object.");
+static const char pyrf_lost_event__doc[] = PyDoc_STR("perf lost event object.");
 
 static PyMemberDef pyrf_lost_event__members[] = {
 	sample_members
@@ -210,7 +211,7 @@ static PyMemberDef pyrf_lost_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_lost_event__repr(const struct pyrf_event *pevent)
 {
 	PyObject *ret;
 	char *s;
@@ -236,7 +237,7 @@ static PyTypeObject pyrf_lost_event__type = {
 	.tp_repr	= (reprfunc)pyrf_lost_event__repr,
 };
 
-static char pyrf_read_event__doc[] = PyDoc_STR("perf read event object.");
+static const char pyrf_read_event__doc[] = PyDoc_STR("perf read event object.");
 
 static PyMemberDef pyrf_read_event__members[] = {
 	sample_members
@@ -245,7 +246,7 @@ static PyMemberDef pyrf_read_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_read_event__repr(const struct pyrf_event *pevent)
 {
 	return PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
 				   pevent->event.read.pid,
@@ -266,7 +267,7 @@ static PyTypeObject pyrf_read_event__type = {
 	.tp_repr	= (reprfunc)pyrf_read_event__repr,
 };
 
-static char pyrf_sample_event__doc[] = PyDoc_STR("perf sample event object.");
+static const char pyrf_sample_event__doc[] = PyDoc_STR("perf sample event object.");
 
 static PyMemberDef pyrf_sample_event__members[] = {
 	sample_members
@@ -274,7 +275,7 @@ static PyMemberDef pyrf_sample_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_sample_event__repr(const struct pyrf_event *pevent)
 {
 	PyObject *ret;
 	char *s;
@@ -289,13 +290,13 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
 }
 
 #ifdef HAVE_LIBTRACEEVENT
-static bool is_tracepoint(struct pyrf_event *pevent)
+static bool is_tracepoint(const struct pyrf_event *pevent)
 {
 	return pevent->evsel->core.attr.type == PERF_TYPE_TRACEPOINT;
 }
 
 static PyObject*
-tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
+tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)
 {
 	struct tep_handle *pevent = field->event->tep;
 	void *data = pe->sample.raw_data;
@@ -384,7 +385,7 @@ static PyTypeObject pyrf_sample_event__type = {
 	.tp_getattro	= (getattrofunc) pyrf_sample_event__getattro,
 };
 
-static char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
+static const char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
 
 static PyMemberDef pyrf_context_switch_event__members[] = {
 	sample_members
@@ -394,7 +395,7 @@ static PyMemberDef pyrf_context_switch_event__members[] = {
 	{ .name = NULL, },
 };
 
-static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent)
+static PyObject *pyrf_context_switch_event__repr(const struct pyrf_event *pevent)
 {
 	PyObject *ret;
 	char *s;
@@ -474,7 +475,7 @@ static PyTypeObject *pyrf_event__type[] = {
 	[PERF_RECORD_SWITCH_CPU_WIDE]  = &pyrf_context_switch_event__type,
 };
 
-static PyObject *pyrf_event__new(union perf_event *event)
+static PyObject *pyrf_event__new(const union perf_event *event)
 {
 	struct pyrf_event *pevent;
 	PyTypeObject *ptype;
@@ -542,7 +543,7 @@ static PySequenceMethods pyrf_cpu_map__sequence_methods = {
 	.sq_item   = pyrf_cpu_map__item,
 };
 
-static char pyrf_cpu_map__doc[] = PyDoc_STR("cpu map object.");
+static const char pyrf_cpu_map__doc[] = PyDoc_STR("cpu map object.");
 
 static PyTypeObject pyrf_cpu_map__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
@@ -611,7 +612,7 @@ static PySequenceMethods pyrf_thread_map__sequence_methods = {
 	.sq_item   = pyrf_thread_map__item,
 };
 
-static char pyrf_thread_map__doc[] = PyDoc_STR("thread map object.");
+static const char pyrf_thread_map__doc[] = PyDoc_STR("thread map object.");
 
 static PyTypeObject pyrf_thread_map__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
@@ -795,7 +796,7 @@ static PyMethodDef pyrf_evsel__methods[] = {
 	{ .ml_name = NULL, }
 };
 
-static char pyrf_evsel__doc[] = PyDoc_STR("perf event selector list object.");
+static const char pyrf_evsel__doc[] = PyDoc_STR("perf event selector list object.");
 
 static PyTypeObject pyrf_evsel__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
@@ -1078,7 +1079,7 @@ static PySequenceMethods pyrf_evlist__sequence_methods = {
 	.sq_item   = pyrf_evlist__item,
 };
 
-static char pyrf_evlist__doc[] = PyDoc_STR("perf event selector list object.");
+static const char pyrf_evlist__doc[] = PyDoc_STR("perf event selector list object.");
 
 static PyTypeObject pyrf_evlist__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
@@ -1100,10 +1101,12 @@ static int pyrf_evlist__setup_types(void)
 
 #define PERF_CONST(name) { #name, PERF_##name }
 
-static struct {
+struct perf_constant {
 	const char *name;
 	int	    value;
-} perf__constants[] = {
+};
+
+static const struct perf_constant perf__constants[] = {
 	PERF_CONST(TYPE_HARDWARE),
 	PERF_CONST(TYPE_SOFTWARE),
 	PERF_CONST(TYPE_TRACEPOINT),
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 02/21] perf python: Constify variables and parameters

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:33PM -0700, Ian Rogers wrote:
> Opportunistically constify variables and parameters when possible.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/python.c | 55 +++++++++++++++++++++-------------------
>  1 file changed, 29 insertions(+), 26 deletions(-)
> 
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 02279ab4967c..13dad27169a0 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -62,7 +62,7 @@ struct pyrf_event {
>  	sample_member_def(sample_period, period, T_ULONGLONG, "event period"),		 \
>  	sample_member_def(sample_cpu, cpu, T_UINT, "event cpu"),
>  
> -static char pyrf_mmap_event__doc[] = PyDoc_STR("perf mmap event object.");
> +static const char pyrf_mmap_event__doc[] = PyDoc_STR("perf mmap event object.");
>  
>  static PyMemberDef pyrf_mmap_event__members[] = {
>  	sample_members
> @@ -77,7 +77,7 @@ static PyMemberDef pyrf_mmap_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_mmap_event__repr(const struct pyrf_event *pevent)
>  {
>  	PyObject *ret;
>  	char *s;
> @@ -106,7 +106,7 @@ static PyTypeObject pyrf_mmap_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_mmap_event__repr,
>  };
>  
> -static char pyrf_task_event__doc[] = PyDoc_STR("perf task (fork/exit) event object.");
> +static const char pyrf_task_event__doc[] = PyDoc_STR("perf task (fork/exit) event object.");
>  
>  static PyMemberDef pyrf_task_event__members[] = {
>  	sample_members
> @@ -119,7 +119,7 @@ static PyMemberDef pyrf_task_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_task_event__repr(const struct pyrf_event *pevent)
>  {
>  	return PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
>  				   "ptid: %u, time: %" PRI_lu64 "}",
> @@ -141,7 +141,7 @@ static PyTypeObject pyrf_task_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_task_event__repr,
>  };
>  
> -static char pyrf_comm_event__doc[] = PyDoc_STR("perf comm event object.");
> +static const char pyrf_comm_event__doc[] = PyDoc_STR("perf comm event object.");
>  
>  static PyMemberDef pyrf_comm_event__members[] = {
>  	sample_members
> @@ -152,7 +152,7 @@ static PyMemberDef pyrf_comm_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_comm_event__repr(const struct pyrf_event *pevent)
>  {
>  	return PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
>  				   pevent->event.comm.pid,
> @@ -170,7 +170,7 @@ static PyTypeObject pyrf_comm_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_comm_event__repr,
>  };
>  
> -static char pyrf_throttle_event__doc[] = PyDoc_STR("perf throttle event object.");
> +static const char pyrf_throttle_event__doc[] = PyDoc_STR("perf throttle event object.");
>  
>  static PyMemberDef pyrf_throttle_event__members[] = {
>  	sample_members
> @@ -181,9 +181,10 @@ static PyMemberDef pyrf_throttle_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_throttle_event__repr(const struct pyrf_event *pevent)
>  {
> -	struct perf_record_throttle *te = (struct perf_record_throttle *)(&pevent->event.header + 1);
> +	const struct perf_record_throttle *te = (const struct perf_record_throttle *)
> +		(&pevent->event.header + 1);
>  
>  	return PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
>  				   ", stream_id: %" PRI_lu64 " }",
> @@ -201,7 +202,7 @@ static PyTypeObject pyrf_throttle_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_throttle_event__repr,
>  };
>  
> -static char pyrf_lost_event__doc[] = PyDoc_STR("perf lost event object.");
> +static const char pyrf_lost_event__doc[] = PyDoc_STR("perf lost event object.");
>  
>  static PyMemberDef pyrf_lost_event__members[] = {
>  	sample_members
> @@ -210,7 +211,7 @@ static PyMemberDef pyrf_lost_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_lost_event__repr(const struct pyrf_event *pevent)
>  {
>  	PyObject *ret;
>  	char *s;
> @@ -236,7 +237,7 @@ static PyTypeObject pyrf_lost_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_lost_event__repr,
>  };
>  
> -static char pyrf_read_event__doc[] = PyDoc_STR("perf read event object.");
> +static const char pyrf_read_event__doc[] = PyDoc_STR("perf read event object.");
>  
>  static PyMemberDef pyrf_read_event__members[] = {
>  	sample_members
> @@ -245,7 +246,7 @@ static PyMemberDef pyrf_read_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_read_event__repr(const struct pyrf_event *pevent)
>  {
>  	return PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
>  				   pevent->event.read.pid,
> @@ -266,7 +267,7 @@ static PyTypeObject pyrf_read_event__type = {
>  	.tp_repr	= (reprfunc)pyrf_read_event__repr,
>  };
>  
> -static char pyrf_sample_event__doc[] = PyDoc_STR("perf sample event object.");
> +static const char pyrf_sample_event__doc[] = PyDoc_STR("perf sample event object.");
>  
>  static PyMemberDef pyrf_sample_event__members[] = {
>  	sample_members
> @@ -274,7 +275,7 @@ static PyMemberDef pyrf_sample_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_sample_event__repr(const struct pyrf_event *pevent)
>  {
>  	PyObject *ret;
>  	char *s;
> @@ -289,13 +290,13 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
>  }
>  
>  #ifdef HAVE_LIBTRACEEVENT
> -static bool is_tracepoint(struct pyrf_event *pevent)
> +static bool is_tracepoint(const struct pyrf_event *pevent)
>  {
>  	return pevent->evsel->core.attr.type == PERF_TYPE_TRACEPOINT;
>  }
>  
>  static PyObject*
> -tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
> +tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)
>  {
>  	struct tep_handle *pevent = field->event->tep;
>  	void *data = pe->sample.raw_data;
> @@ -384,7 +385,7 @@ static PyTypeObject pyrf_sample_event__type = {
>  	.tp_getattro	= (getattrofunc) pyrf_sample_event__getattro,
>  };
>  
> -static char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
> +static const char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
>  
>  static PyMemberDef pyrf_context_switch_event__members[] = {
>  	sample_members
> @@ -394,7 +395,7 @@ static PyMemberDef pyrf_context_switch_event__members[] = {
>  	{ .name = NULL, },
>  };
>  
> -static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent)
> +static PyObject *pyrf_context_switch_event__repr(const struct pyrf_event *pevent)
>  {
>  	PyObject *ret;
>  	char *s;
> @@ -474,7 +475,7 @@ static PyTypeObject *pyrf_event__type[] = {
>  	[PERF_RECORD_SWITCH_CPU_WIDE]  = &pyrf_context_switch_event__type,
>  };
>  
> -static PyObject *pyrf_event__new(union perf_event *event)
> +static PyObject *pyrf_event__new(const union perf_event *event)
>  {
>  	struct pyrf_event *pevent;
>  	PyTypeObject *ptype;
> @@ -542,7 +543,7 @@ static PySequenceMethods pyrf_cpu_map__sequence_methods = {
>  	.sq_item   = pyrf_cpu_map__item,
>  };
>  
> -static char pyrf_cpu_map__doc[] = PyDoc_STR("cpu map object.");
> +static const char pyrf_cpu_map__doc[] = PyDoc_STR("cpu map object.");
>  
>  static PyTypeObject pyrf_cpu_map__type = {
>  	PyVarObject_HEAD_INIT(NULL, 0)
> @@ -611,7 +612,7 @@ static PySequenceMethods pyrf_thread_map__sequence_methods = {
>  	.sq_item   = pyrf_thread_map__item,
>  };
>  
> -static char pyrf_thread_map__doc[] = PyDoc_STR("thread map object.");
> +static const char pyrf_thread_map__doc[] = PyDoc_STR("thread map object.");
>  
>  static PyTypeObject pyrf_thread_map__type = {
>  	PyVarObject_HEAD_INIT(NULL, 0)
> @@ -795,7 +796,7 @@ static PyMethodDef pyrf_evsel__methods[] = {
>  	{ .ml_name = NULL, }
>  };
>  
> -static char pyrf_evsel__doc[] = PyDoc_STR("perf event selector list object.");
> +static const char pyrf_evsel__doc[] = PyDoc_STR("perf event selector list object.");
>  
>  static PyTypeObject pyrf_evsel__type = {
>  	PyVarObject_HEAD_INIT(NULL, 0)
> @@ -1078,7 +1079,7 @@ static PySequenceMethods pyrf_evlist__sequence_methods = {
>  	.sq_item   = pyrf_evlist__item,
>  };
>  
> -static char pyrf_evlist__doc[] = PyDoc_STR("perf event selector list object.");
> +static const char pyrf_evlist__doc[] = PyDoc_STR("perf event selector list object.");
>  
>  static PyTypeObject pyrf_evlist__type = {
>  	PyVarObject_HEAD_INIT(NULL, 0)
> @@ -1100,10 +1101,12 @@ static int pyrf_evlist__setup_types(void)
>  
>  #define PERF_CONST(name) { #name, PERF_##name }
>  
> -static struct {
> +struct perf_constant {
>  	const char *name;
>  	int	    value;
> -} perf__constants[] = {
> +};
> +
> +static const struct perf_constant perf__constants[] = {
>  	PERF_CONST(TYPE_HARDWARE),
>  	PERF_CONST(TYPE_SOFTWARE),
>  	PERF_CONST(TYPE_TRACEPOINT),
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 03/21] perf python: Remove unused #include

[Ian Rogers]

Remove unused #include of bpf-filter.h.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/python.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 13dad27169a0..974578a04099 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -16,7 +16,6 @@
 #include "thread_map.h"
 #include "trace-event.h"
 #include "mmap.h"
-#include "util/bpf-filter.h"
 #include "util/env.h"
 #include "util/kvm-stat.h"
 #include "util/kwork.h"
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 03/21] perf python: Remove unused #include

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:34PM -0700, Ian Rogers wrote:
> Remove unused #include of bpf-filter.h.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/python.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 13dad27169a0..974578a04099 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -16,7 +16,6 @@
>  #include "thread_map.h"
>  #include "trace-event.h"
>  #include "mmap.h"
> -#include "util/bpf-filter.h"
>  #include "util/env.h"
>  #include "util/kvm-stat.h"
>  #include "util/kwork.h"
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 04/21] perf script: Move scripting_max_stack out of builtin

[Ian Rogers]

scripting_max_stack is used in util code which is linked into the
python module. Move the variable declaration to
util/trace-event-scripting.c to avoid conditional compilation.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c             | 2 --
 tools/perf/util/python.c                | 2 --
 tools/perf/util/trace-event-scripting.c | 3 +++
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 6b6d4472db6e..5d5a1a06d8c6 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -92,8 +92,6 @@ static struct dlfilter		*dlfilter;
 static int			dlargc;
 static char			**dlargv;
 
-unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
-
 enum perf_output_field {
 	PERF_OUTPUT_COMM            = 1ULL << 0,
 	PERF_OUTPUT_TID             = 1ULL << 1,
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 974578a04099..298f43981fc5 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1306,8 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
-
 bool kvm_entry_event(struct evsel *evsel __maybe_unused)
 {
 	return false;
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index bd0000300c77..8abb7a7b6888 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -16,9 +16,12 @@
 #include "debug.h"
 #include "trace-event.h"
 #include "evsel.h"
+#include <linux/perf_event.h>
 #include <linux/zalloc.h>
 #include "util/sample.h"
 
+unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
+
 struct scripting_context *scripting_context;
 
 void scripting_context__update(struct scripting_context *c,
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 04/21] perf script: Move scripting_max_stack out of builtin

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:35PM -0700, Ian Rogers wrote:
> scripting_max_stack is used in util code which is linked into the
> python module. Move the variable declaration to
> util/trace-event-scripting.c to avoid conditional compilation.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-script.c             | 2 --
>  tools/perf/util/python.c                | 2 --
>  tools/perf/util/trace-event-scripting.c | 3 +++
>  3 files changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index 6b6d4472db6e..5d5a1a06d8c6 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -92,8 +92,6 @@ static struct dlfilter		*dlfilter;
>  static int			dlargc;
>  static char			**dlargv;
>  
> -unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
> -
>  enum perf_output_field {
>  	PERF_OUTPUT_COMM            = 1ULL << 0,
>  	PERF_OUTPUT_TID             = 1ULL << 1,
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 974578a04099..298f43981fc5 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1306,8 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
> -
>  bool kvm_entry_event(struct evsel *evsel __maybe_unused)
>  {
>  	return false;
> diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
> index bd0000300c77..8abb7a7b6888 100644
> --- a/tools/perf/util/trace-event-scripting.c
> +++ b/tools/perf/util/trace-event-scripting.c
> @@ -16,9 +16,12 @@
>  #include "debug.h"
>  #include "trace-event.h"
>  #include "evsel.h"
> +#include <linux/perf_event.h>
>  #include <linux/zalloc.h>
>  #include "util/sample.h"
>  
> +unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
> +
>  struct scripting_context *scripting_context;
>  
>  void scripting_context__update(struct scripting_context *c,
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 05/21] perf kvm: Move functions used in util out of builtin

[Ian Rogers]

The util library code is used by the python module but doesn't have
access to the builtin files. Make a util/kvm-stat.c to match the
kvm-stat.h file that declares the functions and move the functions
there.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-kvm.c   | 61 ---------------------------------
 tools/perf/util/Build      |  1 +
 tools/perf/util/kvm-stat.c | 70 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/kvm-stat.h |  3 ++
 tools/perf/util/python.c   | 30 ----------------
 5 files changed, 74 insertions(+), 91 deletions(-)
 create mode 100644 tools/perf/util/kvm-stat.c

diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index 099ce3ebf67c..e50cd04e3725 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -615,67 +615,6 @@ static const char *get_filename_for_perf_kvm(void)
 
 #if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
 
-void exit_event_get_key(struct evsel *evsel,
-			struct perf_sample *sample,
-			struct event_key *key)
-{
-	key->info = 0;
-	key->key  = evsel__intval(evsel, sample, kvm_exit_reason);
-}
-
-bool kvm_exit_event(struct evsel *evsel)
-{
-	return evsel__name_is(evsel, kvm_exit_trace);
-}
-
-bool exit_event_begin(struct evsel *evsel,
-		      struct perf_sample *sample, struct event_key *key)
-{
-	if (kvm_exit_event(evsel)) {
-		exit_event_get_key(evsel, sample, key);
-		return true;
-	}
-
-	return false;
-}
-
-bool kvm_entry_event(struct evsel *evsel)
-{
-	return evsel__name_is(evsel, kvm_entry_trace);
-}
-
-bool exit_event_end(struct evsel *evsel,
-		    struct perf_sample *sample __maybe_unused,
-		    struct event_key *key __maybe_unused)
-{
-	return kvm_entry_event(evsel);
-}
-
-static const char *get_exit_reason(struct perf_kvm_stat *kvm,
-				   struct exit_reasons_table *tbl,
-				   u64 exit_code)
-{
-	while (tbl->reason != NULL) {
-		if (tbl->exit_code == exit_code)
-			return tbl->reason;
-		tbl++;
-	}
-
-	pr_err("unknown kvm exit code:%lld on %s\n",
-		(unsigned long long)exit_code, kvm->exit_reasons_isa);
-	return "UNKNOWN";
-}
-
-void exit_event_decode_key(struct perf_kvm_stat *kvm,
-			   struct event_key *key,
-			   char *decode)
-{
-	const char *exit_reason = get_exit_reason(kvm, key->exit_reasons,
-						  key->key);
-
-	scnprintf(decode, KVM_EVENT_NAME_LEN, "%s", exit_reason);
-}
-
 static bool register_kvm_events_ops(struct perf_kvm_stat *kvm)
 {
 	struct kvm_reg_events_ops *events_ops = kvm_reg_events_ops;
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 1eedead5f2f2..650974413849 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -120,6 +120,7 @@ perf-util-y += spark.o
 perf-util-y += topdown.o
 perf-util-y += iostat.o
 perf-util-y += stream.o
+perf-util-y += kvm-stat.o
 perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
 perf-util-$(CONFIG_AUXTRACE) += intel-pt-decoder/
 perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
diff --git a/tools/perf/util/kvm-stat.c b/tools/perf/util/kvm-stat.c
new file mode 100644
index 000000000000..38ace736db5c
--- /dev/null
+++ b/tools/perf/util/kvm-stat.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "debug.h"
+#include "evsel.h"
+#include "kvm-stat.h"
+
+#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
+
+bool kvm_exit_event(struct evsel *evsel)
+{
+	return evsel__name_is(evsel, kvm_exit_trace);
+}
+
+void exit_event_get_key(struct evsel *evsel,
+			struct perf_sample *sample,
+			struct event_key *key)
+{
+	key->info = 0;
+	key->key  = evsel__intval(evsel, sample, kvm_exit_reason);
+}
+
+
+bool exit_event_begin(struct evsel *evsel,
+		      struct perf_sample *sample, struct event_key *key)
+{
+	if (kvm_exit_event(evsel)) {
+		exit_event_get_key(evsel, sample, key);
+		return true;
+	}
+
+	return false;
+}
+
+bool kvm_entry_event(struct evsel *evsel)
+{
+	return evsel__name_is(evsel, kvm_entry_trace);
+}
+
+bool exit_event_end(struct evsel *evsel,
+		    struct perf_sample *sample __maybe_unused,
+		    struct event_key *key __maybe_unused)
+{
+	return kvm_entry_event(evsel);
+}
+
+static const char *get_exit_reason(struct perf_kvm_stat *kvm,
+				   struct exit_reasons_table *tbl,
+				   u64 exit_code)
+{
+	while (tbl->reason != NULL) {
+		if (tbl->exit_code == exit_code)
+			return tbl->reason;
+		tbl++;
+	}
+
+	pr_err("unknown kvm exit code:%lld on %s\n",
+		(unsigned long long)exit_code, kvm->exit_reasons_isa);
+	return "UNKNOWN";
+}
+
+void exit_event_decode_key(struct perf_kvm_stat *kvm,
+			   struct event_key *key,
+			   char *decode)
+{
+	const char *exit_reason = get_exit_reason(kvm, key->exit_reasons,
+						  key->key);
+
+	scnprintf(decode, KVM_EVENT_NAME_LEN, "%s", exit_reason);
+}
+
+#endif
diff --git a/tools/perf/util/kvm-stat.h b/tools/perf/util/kvm-stat.h
index 3e9ac754c3d1..4249542544bb 100644
--- a/tools/perf/util/kvm-stat.h
+++ b/tools/perf/util/kvm-stat.h
@@ -115,6 +115,8 @@ struct kvm_reg_events_ops {
 	struct kvm_events_ops *ops;
 };
 
+#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
+
 void exit_event_get_key(struct evsel *evsel,
 			struct perf_sample *sample,
 			struct event_key *key);
@@ -127,6 +129,7 @@ bool exit_event_end(struct evsel *evsel,
 void exit_event_decode_key(struct perf_kvm_stat *kvm,
 			   struct event_key *key,
 			   char *decode);
+#endif
 
 bool kvm_exit_event(struct evsel *evsel);
 bool kvm_entry_event(struct evsel *evsel);
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 298f43981fc5..ab67abf3b607 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1306,36 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-bool kvm_entry_event(struct evsel *evsel __maybe_unused)
-{
-	return false;
-}
-
-bool kvm_exit_event(struct evsel *evsel __maybe_unused)
-{
-	return false;
-}
-
-bool exit_event_begin(struct evsel *evsel __maybe_unused,
-		      struct perf_sample *sample  __maybe_unused,
-		      struct event_key *key  __maybe_unused)
-{
-	return false;
-}
-
-bool exit_event_end(struct evsel *evsel __maybe_unused,
-		    struct perf_sample *sample __maybe_unused,
-		    struct event_key *key __maybe_unused)
-{
-	return false;
-}
-
-void exit_event_decode_key(struct perf_kvm_stat *kvm __maybe_unused,
-			   struct event_key *key __maybe_unused,
-			   char *decode __maybe_unused)
-{
-}
-
 int find_scripts(char **scripts_array  __maybe_unused, char **scripts_path_array  __maybe_unused,
 		int num  __maybe_unused, int pathlen __maybe_unused)
 {
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 05/21] perf kvm: Move functions used in util out of builtin

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:36PM -0700, Ian Rogers wrote:
> The util library code is used by the python module but doesn't have
> access to the builtin files. Make a util/kvm-stat.c to match the
> kvm-stat.h file that declares the functions and move the functions
> there.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-kvm.c   | 61 ---------------------------------
>  tools/perf/util/Build      |  1 +
>  tools/perf/util/kvm-stat.c | 70 ++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/kvm-stat.h |  3 ++
>  tools/perf/util/python.c   | 30 ----------------
>  5 files changed, 74 insertions(+), 91 deletions(-)
>  create mode 100644 tools/perf/util/kvm-stat.c
> 
> diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
> index 099ce3ebf67c..e50cd04e3725 100644
> --- a/tools/perf/builtin-kvm.c
> +++ b/tools/perf/builtin-kvm.c
> @@ -615,67 +615,6 @@ static const char *get_filename_for_perf_kvm(void)
>  
>  #if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
>  
> -void exit_event_get_key(struct evsel *evsel,
> -			struct perf_sample *sample,
> -			struct event_key *key)
> -{
> -	key->info = 0;
> -	key->key  = evsel__intval(evsel, sample, kvm_exit_reason);
> -}
> -
> -bool kvm_exit_event(struct evsel *evsel)
> -{
> -	return evsel__name_is(evsel, kvm_exit_trace);
> -}
> -
> -bool exit_event_begin(struct evsel *evsel,
> -		      struct perf_sample *sample, struct event_key *key)
> -{
> -	if (kvm_exit_event(evsel)) {
> -		exit_event_get_key(evsel, sample, key);
> -		return true;
> -	}
> -
> -	return false;
> -}
> -
> -bool kvm_entry_event(struct evsel *evsel)
> -{
> -	return evsel__name_is(evsel, kvm_entry_trace);
> -}
> -
> -bool exit_event_end(struct evsel *evsel,
> -		    struct perf_sample *sample __maybe_unused,
> -		    struct event_key *key __maybe_unused)
> -{
> -	return kvm_entry_event(evsel);
> -}
> -
> -static const char *get_exit_reason(struct perf_kvm_stat *kvm,
> -				   struct exit_reasons_table *tbl,
> -				   u64 exit_code)
> -{
> -	while (tbl->reason != NULL) {
> -		if (tbl->exit_code == exit_code)
> -			return tbl->reason;
> -		tbl++;
> -	}
> -
> -	pr_err("unknown kvm exit code:%lld on %s\n",
> -		(unsigned long long)exit_code, kvm->exit_reasons_isa);
> -	return "UNKNOWN";
> -}
> -
> -void exit_event_decode_key(struct perf_kvm_stat *kvm,
> -			   struct event_key *key,
> -			   char *decode)
> -{
> -	const char *exit_reason = get_exit_reason(kvm, key->exit_reasons,
> -						  key->key);
> -
> -	scnprintf(decode, KVM_EVENT_NAME_LEN, "%s", exit_reason);
> -}
> -
>  static bool register_kvm_events_ops(struct perf_kvm_stat *kvm)
>  {
>  	struct kvm_reg_events_ops *events_ops = kvm_reg_events_ops;
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 1eedead5f2f2..650974413849 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -120,6 +120,7 @@ perf-util-y += spark.o
>  perf-util-y += topdown.o
>  perf-util-y += iostat.o
>  perf-util-y += stream.o
> +perf-util-y += kvm-stat.o
>  perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
>  perf-util-$(CONFIG_AUXTRACE) += intel-pt-decoder/
>  perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
> diff --git a/tools/perf/util/kvm-stat.c b/tools/perf/util/kvm-stat.c
> new file mode 100644
> index 000000000000..38ace736db5c
> --- /dev/null
> +++ b/tools/perf/util/kvm-stat.c
> @@ -0,0 +1,70 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include "debug.h"
> +#include "evsel.h"
> +#include "kvm-stat.h"
> +
> +#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
> +
> +bool kvm_exit_event(struct evsel *evsel)
> +{
> +	return evsel__name_is(evsel, kvm_exit_trace);
> +}
> +
> +void exit_event_get_key(struct evsel *evsel,
> +			struct perf_sample *sample,
> +			struct event_key *key)
> +{
> +	key->info = 0;
> +	key->key  = evsel__intval(evsel, sample, kvm_exit_reason);
> +}
> +
> +
> +bool exit_event_begin(struct evsel *evsel,
> +		      struct perf_sample *sample, struct event_key *key)
> +{
> +	if (kvm_exit_event(evsel)) {
> +		exit_event_get_key(evsel, sample, key);
> +		return true;
> +	}
> +
> +	return false;
> +}
> +
> +bool kvm_entry_event(struct evsel *evsel)
> +{
> +	return evsel__name_is(evsel, kvm_entry_trace);
> +}
> +
> +bool exit_event_end(struct evsel *evsel,
> +		    struct perf_sample *sample __maybe_unused,
> +		    struct event_key *key __maybe_unused)
> +{
> +	return kvm_entry_event(evsel);
> +}
> +
> +static const char *get_exit_reason(struct perf_kvm_stat *kvm,
> +				   struct exit_reasons_table *tbl,
> +				   u64 exit_code)
> +{
> +	while (tbl->reason != NULL) {
> +		if (tbl->exit_code == exit_code)
> +			return tbl->reason;
> +		tbl++;
> +	}
> +
> +	pr_err("unknown kvm exit code:%lld on %s\n",
> +		(unsigned long long)exit_code, kvm->exit_reasons_isa);
> +	return "UNKNOWN";
> +}
> +
> +void exit_event_decode_key(struct perf_kvm_stat *kvm,
> +			   struct event_key *key,
> +			   char *decode)
> +{
> +	const char *exit_reason = get_exit_reason(kvm, key->exit_reasons,
> +						  key->key);
> +
> +	scnprintf(decode, KVM_EVENT_NAME_LEN, "%s", exit_reason);
> +}
> +
> +#endif
> diff --git a/tools/perf/util/kvm-stat.h b/tools/perf/util/kvm-stat.h
> index 3e9ac754c3d1..4249542544bb 100644
> --- a/tools/perf/util/kvm-stat.h
> +++ b/tools/perf/util/kvm-stat.h
> @@ -115,6 +115,8 @@ struct kvm_reg_events_ops {
>  	struct kvm_events_ops *ops;
>  };
>  
> +#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
> +
>  void exit_event_get_key(struct evsel *evsel,
>  			struct perf_sample *sample,
>  			struct event_key *key);
> @@ -127,6 +129,7 @@ bool exit_event_end(struct evsel *evsel,
>  void exit_event_decode_key(struct perf_kvm_stat *kvm,
>  			   struct event_key *key,
>  			   char *decode);
> +#endif
>  
>  bool kvm_exit_event(struct evsel *evsel);
>  bool kvm_entry_event(struct evsel *evsel);
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 298f43981fc5..ab67abf3b607 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1306,36 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -bool kvm_entry_event(struct evsel *evsel __maybe_unused)
> -{
> -	return false;
> -}
> -
> -bool kvm_exit_event(struct evsel *evsel __maybe_unused)
> -{
> -	return false;
> -}
> -
> -bool exit_event_begin(struct evsel *evsel __maybe_unused,
> -		      struct perf_sample *sample  __maybe_unused,
> -		      struct event_key *key  __maybe_unused)
> -{
> -	return false;
> -}
> -
> -bool exit_event_end(struct evsel *evsel __maybe_unused,
> -		    struct perf_sample *sample __maybe_unused,
> -		    struct event_key *key __maybe_unused)
> -{
> -	return false;
> -}
> -
> -void exit_event_decode_key(struct perf_kvm_stat *kvm __maybe_unused,
> -			   struct event_key *key __maybe_unused,
> -			   char *decode __maybe_unused)
> -{
> -}
> -
>  int find_scripts(char **scripts_array  __maybe_unused, char **scripts_path_array  __maybe_unused,
>  		int num  __maybe_unused, int pathlen __maybe_unused)
>  {
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

The only use of find_scripts is in browser/scripts.c but the
definition in builtin causes linking problems requiring a stub in
python.c. Move the function to allow the stub to be removed.

Rewrite the directory iteration to use openat so that large character
arrays aren't needed. The arrays are warned about potential buffer
overflows by GCC now that all the code exists in a single C file.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c      | 138 ------------------------
 tools/perf/builtin.h             |   6 --
 tools/perf/ui/browsers/scripts.c | 177 ++++++++++++++++++++++++++++++-
 tools/perf/util/path.c           |  10 ++
 tools/perf/util/path.h           |   1 +
 tools/perf/util/python.c         |   6 --
 6 files changed, 186 insertions(+), 152 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 5d5a1a06d8c6..e9ec74056f71 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -3521,144 +3521,6 @@ static void free_dlarg(void)
 	free(dlargv);
 }
 
-/*
- * Some scripts specify the required events in their "xxx-record" file,
- * this function will check if the events in perf.data match those
- * mentioned in the "xxx-record".
- *
- * Fixme: All existing "xxx-record" are all in good formats "-e event ",
- * which is covered well now. And new parsing code should be added to
- * cover the future complex formats like event groups etc.
- */
-static int check_ev_match(char *dir_name, char *scriptname,
-			struct perf_session *session)
-{
-	char filename[MAXPATHLEN], evname[128];
-	char line[BUFSIZ], *p;
-	struct evsel *pos;
-	int match, len;
-	FILE *fp;
-
-	scnprintf(filename, MAXPATHLEN, "%s/bin/%s-record", dir_name, scriptname);
-
-	fp = fopen(filename, "r");
-	if (!fp)
-		return -1;
-
-	while (fgets(line, sizeof(line), fp)) {
-		p = skip_spaces(line);
-		if (*p == '#')
-			continue;
-
-		while (strlen(p)) {
-			p = strstr(p, "-e");
-			if (!p)
-				break;
-
-			p += 2;
-			p = skip_spaces(p);
-			len = strcspn(p, " \t");
-			if (!len)
-				break;
-
-			snprintf(evname, len + 1, "%s", p);
-
-			match = 0;
-			evlist__for_each_entry(session->evlist, pos) {
-				if (evsel__name_is(pos, evname)) {
-					match = 1;
-					break;
-				}
-			}
-
-			if (!match) {
-				fclose(fp);
-				return -1;
-			}
-		}
-	}
-
-	fclose(fp);
-	return 0;
-}
-
-/*
- * Return -1 if none is found, otherwise the actual scripts number.
- *
- * Currently the only user of this function is the script browser, which
- * will list all statically runnable scripts, select one, execute it and
- * show the output in a perf browser.
- */
-int find_scripts(char **scripts_array, char **scripts_path_array, int num,
-		 int pathlen)
-{
-	struct dirent *script_dirent, *lang_dirent;
-	char scripts_path[MAXPATHLEN], lang_path[MAXPATHLEN];
-	DIR *scripts_dir, *lang_dir;
-	struct perf_session *session;
-	struct perf_data data = {
-		.path = input_name,
-		.mode = PERF_DATA_MODE_READ,
-	};
-	char *temp;
-	int i = 0;
-
-	session = perf_session__new(&data, NULL);
-	if (IS_ERR(session))
-		return PTR_ERR(session);
-
-	snprintf(scripts_path, MAXPATHLEN, "%s/scripts", get_argv_exec_path());
-
-	scripts_dir = opendir(scripts_path);
-	if (!scripts_dir) {
-		perf_session__delete(session);
-		return -1;
-	}
-
-	for_each_lang(scripts_path, scripts_dir, lang_dirent) {
-		scnprintf(lang_path, MAXPATHLEN, "%s/%s", scripts_path,
-			  lang_dirent->d_name);
-#ifndef HAVE_LIBPERL_SUPPORT
-		if (strstr(lang_path, "perl"))
-			continue;
-#endif
-#ifndef HAVE_LIBPYTHON_SUPPORT
-		if (strstr(lang_path, "python"))
-			continue;
-#endif
-
-		lang_dir = opendir(lang_path);
-		if (!lang_dir)
-			continue;
-
-		for_each_script(lang_path, lang_dir, script_dirent) {
-			/* Skip those real time scripts: xxxtop.p[yl] */
-			if (strstr(script_dirent->d_name, "top."))
-				continue;
-			if (i >= num)
-				break;
-			snprintf(scripts_path_array[i], pathlen, "%s/%s",
-				lang_path,
-				script_dirent->d_name);
-			temp = strchr(script_dirent->d_name, '.');
-			snprintf(scripts_array[i],
-				(temp - script_dirent->d_name) + 1,
-				"%s", script_dirent->d_name);
-
-			if (check_ev_match(lang_path,
-					scripts_array[i], session))
-				continue;
-
-			i++;
-		}
-		closedir(lang_dir);
-	}
-
-	closedir(scripts_dir);
-	perf_session__delete(session);
-	return i;
-}
-
 static char *get_script_path(const char *script_root, const char *suffix)
 {
 	struct dirent *script_dirent, *lang_dirent;
diff --git a/tools/perf/builtin.h b/tools/perf/builtin.h
index 94f4b3769bf7..a07e93c53848 100644
--- a/tools/perf/builtin.h
+++ b/tools/perf/builtin.h
@@ -2,10 +2,6 @@
 #ifndef BUILTIN_H
 #define BUILTIN_H
 
-#include <stddef.h>
-#include <linux/compiler.h>
-#include <tools/config.h>
-
 struct feature_status {
 	const char *name;
 	const char *macro;
@@ -56,6 +52,4 @@ int cmd_ftrace(int argc, const char **argv);
 int cmd_daemon(int argc, const char **argv);
 int cmd_kwork(int argc, const char **argv);
 
-int find_scripts(char **scripts_array, char **scripts_path_array, int num,
-		 int pathlen);
 #endif
diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
index e437d7889de6..2d04ece833aa 100644
--- a/tools/perf/ui/browsers/scripts.c
+++ b/tools/perf/ui/browsers/scripts.c
@@ -1,16 +1,18 @@
 // SPDX-License-Identifier: GPL-2.0
-#include "../../builtin.h"
-#include "../../perf.h"
 #include "../../util/util.h" // perf_exe()
 #include "../util.h"
+#include "../../util/evlist.h"
 #include "../../util/hist.h"
 #include "../../util/debug.h"
+#include "../../util/session.h"
 #include "../../util/symbol.h"
 #include "../browser.h"
 #include "../libslang.h"
 #include "config.h"
+#include <linux/err.h>
 #include <linux/string.h>
 #include <linux/zalloc.h>
+#include <subcmd/exec-cmd.h>
 #include <stdlib.h>
 
 #define SCRIPT_NAMELEN	128
@@ -77,6 +79,177 @@ static int scripts_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
+/*
+ * Some scripts specify the required events in their "xxx-record" file,
+ * this function will check if the events in perf.data match those
+ * mentioned in the "xxx-record".
+ *
+ * Fixme: All existing "xxx-record" are all in good formats "-e event ",
+ * which is covered well now. And new parsing code should be added to
+ * cover the future complex formats like event groups etc.
+ */
+static int check_ev_match(int dir_fd, const char *scriptname, struct perf_session *session)
+{
+	char line[BUFSIZ];
+	FILE *fp;
+
+	{
+		char filename[FILENAME_MAX + 5];
+		int fd;
+
+		scnprintf(filename, sizeof(filename), "bin/%s-record", scriptname);
+		fd = openat(dir_fd, filename, O_RDONLY);
+		if (fd == -1)
+			return -1;
+		fp = fdopen(fd, "r");
+		if (!fp)
+			return -1;
+	}
+
+	while (fgets(line, sizeof(line), fp)) {
+		char *p = skip_spaces(line);
+
+		if (*p == '#')
+			continue;
+
+		while (strlen(p)) {
+			int match, len;
+			struct evsel *pos;
+			char evname[128];
+
+			p = strstr(p, "-e");
+			if (!p)
+				break;
+
+			p += 2;
+			p = skip_spaces(p);
+			len = strcspn(p, " \t");
+			if (!len)
+				break;
+
+			snprintf(evname, len + 1, "%s", p);
+
+			match = 0;
+			evlist__for_each_entry(session->evlist, pos) {
+				if (evsel__name_is(pos, evname)) {
+					match = 1;
+					break;
+				}
+			}
+
+			if (!match) {
+				fclose(fp);
+				return -1;
+			}
+		}
+	}
+
+	fclose(fp);
+	return 0;
+}
+
+/*
+ * Return -1 if none is found, otherwise the actual scripts number.
+ *
+ * Currently the only user of this function is the script browser, which
+ * will list all statically runnable scripts, select one, execute it and
+ * show the output in a perf browser.
+ */
+static int find_scripts(char **scripts_array, char **scripts_path_array, int num,
+		 int pathlen)
+{
+	struct dirent *script_dirent, *lang_dirent;
+	int scripts_dir_fd, lang_dir_fd;
+	DIR *scripts_dir, *lang_dir;
+	struct perf_session *session;
+	struct perf_data data = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
+	char *temp;
+	int i = 0;
+	const char *exec_path = get_argv_exec_path();
+
+	session = perf_session__new(&data, NULL);
+	if (IS_ERR(session))
+		return PTR_ERR(session);
+
+	{
+		char scripts_path[PATH_MAX];
+
+		snprintf(scripts_path, sizeof(scripts_path), "%s/scripts", exec_path);
+		scripts_dir_fd = open(scripts_path, O_DIRECTORY);
+		pr_err("Failed to open directory '%s'", scripts_path);
+		if (scripts_dir_fd == -1) {
+			perf_session__delete(session);
+			return -1;
+		}
+	}
+	scripts_dir = fdopendir(scripts_dir_fd);
+	if (!scripts_dir) {
+		close(scripts_dir_fd);
+		perf_session__delete(session);
+		return -1;
+	}
+
+	while ((lang_dirent = readdir(scripts_dir)) != NULL) {
+		if (lang_dirent->d_type != DT_DIR &&
+		    (lang_dirent->d_type == DT_UNKNOWN &&
+		     !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
+			continue;
+		if (!strcmp(lang_dirent->d_name, ".") || !strcmp(lang_dirent->d_name, ".."))
+			continue;
+
+#ifndef HAVE_LIBPERL_SUPPORT
+		if (strstr(lang_dirent->d_name, "perl"))
+			continue;
+#endif
+#ifndef HAVE_LIBPYTHON_SUPPORT
+		if (strstr(lang_dirent->d_name, "python"))
+			continue;
+#endif
+
+		lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name, O_DIRECTORY);
+		if (lang_dir_fd == -1)
+			continue;
+		lang_dir = fdopendir(lang_dir_fd);
+		if (!lang_dir) {
+			close(lang_dir_fd);
+			continue;
+		}
+		while ((script_dirent = readdir(lang_dir)) != NULL) {
+			if (script_dirent->d_type == DT_DIR)
+				continue;
+			if (script_dirent->d_type == DT_UNKNOWN &&
+			    is_directory_at(lang_dir_fd, script_dirent->d_name))
+				continue;
+			/* Skip those real time scripts: xxxtop.p[yl] */
+			if (strstr(script_dirent->d_name, "top."))
+				continue;
+			if (i >= num)
+				break;
+			scnprintf(scripts_path_array[i], pathlen, "%s/scripts/%s/%s",
+				exec_path,
+				lang_dirent->d_name,
+				script_dirent->d_name);
+			temp = strchr(script_dirent->d_name, '.');
+			snprintf(scripts_array[i],
+				(temp - script_dirent->d_name) + 1,
+				"%s", script_dirent->d_name);
+
+			if (check_ev_match(lang_dir_fd, scripts_array[i], session))
+				continue;
+
+			i++;
+		}
+		closedir(lang_dir);
+	}
+
+	closedir(scripts_dir);
+	perf_session__delete(session);
+	return i;
+}
+
 /*
  * When success, will copy the full path of the selected script
  * into  the buffer pointed by script_name, and return 0.
diff --git a/tools/perf/util/path.c b/tools/perf/util/path.c
index 00adf872bf00..9712466c51e2 100644
--- a/tools/perf/util/path.c
+++ b/tools/perf/util/path.c
@@ -68,6 +68,16 @@ bool is_directory(const char *base_path, const struct dirent *dent)
 	return S_ISDIR(st.st_mode);
 }
 
+bool is_directory_at(int dir_fd, const char *path)
+{
+	struct stat st;
+
+	if (fstatat(dir_fd, path, &st, /*flags=*/0))
+		return false;
+
+	return S_ISDIR(st.st_mode);
+}
+
 bool is_executable_file(const char *base_path, const struct dirent *dent)
 {
 	char path[PATH_MAX];
diff --git a/tools/perf/util/path.h b/tools/perf/util/path.h
index d94902c22222..fbafbe7015dd 100644
--- a/tools/perf/util/path.h
+++ b/tools/perf/util/path.h
@@ -12,6 +12,7 @@ int path__join3(char *bf, size_t size, const char *path1, const char *path2, con
 
 bool is_regular_file(const char *file);
 bool is_directory(const char *base_path, const struct dirent *dent);
+bool is_directory_at(int dir_fd, const char *path);
 bool is_executable_file(const char *base_path, const struct dirent *dent);
 
 #endif /* _PERF_PATH_H */
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index ab67abf3b607..5f11ae88943d 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1306,12 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-int find_scripts(char **scripts_array  __maybe_unused, char **scripts_path_array  __maybe_unused,
-		int num  __maybe_unused, int pathlen __maybe_unused)
-{
-	return -1;
-}
-
 void perf_stat__set_no_csv_summary(int set __maybe_unused)
 {
 }
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> The only use of find_scripts is in browser/scripts.c but the
> definition in builtin causes linking problems requiring a stub in
> python.c. Move the function to allow the stub to be removed.
> 
> Rewrite the directory iteration to use openat so that large character
> arrays aren't needed. The arrays are warned about potential buffer
> overflows by GCC now that all the code exists in a single C file.

Introducing is_directory_at() should be done as a prep patch, as the
rest of the patch below could end up being reverted after some other
patch used it, making the process more difficult.

I mentioned cases like this in the past, so doing it again just for the
record.

- Arnaldo
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-script.c      | 138 ------------------------
>  tools/perf/builtin.h             |   6 --
>  tools/perf/ui/browsers/scripts.c | 177 ++++++++++++++++++++++++++++++-
>  tools/perf/util/path.c           |  10 ++
>  tools/perf/util/path.h           |   1 +
>  tools/perf/util/python.c         |   6 --
>  6 files changed, 186 insertions(+), 152 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index 5d5a1a06d8c6..e9ec74056f71 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -3521,144 +3521,6 @@ static void free_dlarg(void)
>  	free(dlargv);
>  }
>  
> -/*
> - * Some scripts specify the required events in their "xxx-record" file,
> - * this function will check if the events in perf.data match those
> - * mentioned in the "xxx-record".
> - *
> - * Fixme: All existing "xxx-record" are all in good formats "-e event ",
> - * which is covered well now. And new parsing code should be added to
> - * cover the future complex formats like event groups etc.
> - */
> -static int check_ev_match(char *dir_name, char *scriptname,
> -			struct perf_session *session)
> -{
> -	char filename[MAXPATHLEN], evname[128];
> -	char line[BUFSIZ], *p;
> -	struct evsel *pos;
> -	int match, len;
> -	FILE *fp;
> -
> -	scnprintf(filename, MAXPATHLEN, "%s/bin/%s-record", dir_name, scriptname);
> -
> -	fp = fopen(filename, "r");
> -	if (!fp)
> -		return -1;
> -
> -	while (fgets(line, sizeof(line), fp)) {
> -		p = skip_spaces(line);
> -		if (*p == '#')
> -			continue;
> -
> -		while (strlen(p)) {
> -			p = strstr(p, "-e");
> -			if (!p)
> -				break;
> -
> -			p += 2;
> -			p = skip_spaces(p);
> -			len = strcspn(p, " \t");
> -			if (!len)
> -				break;
> -
> -			snprintf(evname, len + 1, "%s", p);
> -
> -			match = 0;
> -			evlist__for_each_entry(session->evlist, pos) {
> -				if (evsel__name_is(pos, evname)) {
> -					match = 1;
> -					break;
> -				}
> -			}
> -
> -			if (!match) {
> -				fclose(fp);
> -				return -1;
> -			}
> -		}
> -	}
> -
> -	fclose(fp);
> -	return 0;
> -}
> -
> -/*
> - * Return -1 if none is found, otherwise the actual scripts number.
> - *
> - * Currently the only user of this function is the script browser, which
> - * will list all statically runnable scripts, select one, execute it and
> - * show the output in a perf browser.
> - */
> -int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> -		 int pathlen)
> -{
> -	struct dirent *script_dirent, *lang_dirent;
> -	char scripts_path[MAXPATHLEN], lang_path[MAXPATHLEN];
> -	DIR *scripts_dir, *lang_dir;
> -	struct perf_session *session;
> -	struct perf_data data = {
> -		.path = input_name,
> -		.mode = PERF_DATA_MODE_READ,
> -	};
> -	char *temp;
> -	int i = 0;
> -
> -	session = perf_session__new(&data, NULL);
> -	if (IS_ERR(session))
> -		return PTR_ERR(session);
> -
> -	snprintf(scripts_path, MAXPATHLEN, "%s/scripts", get_argv_exec_path());
> -
> -	scripts_dir = opendir(scripts_path);
> -	if (!scripts_dir) {
> -		perf_session__delete(session);
> -		return -1;
> -	}
> -
> -	for_each_lang(scripts_path, scripts_dir, lang_dirent) {
> -		scnprintf(lang_path, MAXPATHLEN, "%s/%s", scripts_path,
> -			  lang_dirent->d_name);
> -#ifndef HAVE_LIBPERL_SUPPORT
> -		if (strstr(lang_path, "perl"))
> -			continue;
> -#endif
> -#ifndef HAVE_LIBPYTHON_SUPPORT
> -		if (strstr(lang_path, "python"))
> -			continue;
> -#endif
> -
> -		lang_dir = opendir(lang_path);
> -		if (!lang_dir)
> -			continue;
> -
> -		for_each_script(lang_path, lang_dir, script_dirent) {
> -			/* Skip those real time scripts: xxxtop.p[yl] */
> -			if (strstr(script_dirent->d_name, "top."))
> -				continue;
> -			if (i >= num)
> -				break;
> -			snprintf(scripts_path_array[i], pathlen, "%s/%s",
> -				lang_path,
> -				script_dirent->d_name);
> -			temp = strchr(script_dirent->d_name, '.');
> -			snprintf(scripts_array[i],
> -				(temp - script_dirent->d_name) + 1,
> -				"%s", script_dirent->d_name);
> -
> -			if (check_ev_match(lang_path,
> -					scripts_array[i], session))
> -				continue;
> -
> -			i++;
> -		}
> -		closedir(lang_dir);
> -	}
> -
> -	closedir(scripts_dir);
> -	perf_session__delete(session);
> -	return i;
> -}
> -
>  static char *get_script_path(const char *script_root, const char *suffix)
>  {
>  	struct dirent *script_dirent, *lang_dirent;
> diff --git a/tools/perf/builtin.h b/tools/perf/builtin.h
> index 94f4b3769bf7..a07e93c53848 100644
> --- a/tools/perf/builtin.h
> +++ b/tools/perf/builtin.h
> @@ -2,10 +2,6 @@
>  #ifndef BUILTIN_H
>  #define BUILTIN_H
>  
> -#include <stddef.h>
> -#include <linux/compiler.h>
> -#include <tools/config.h>
> -
>  struct feature_status {
>  	const char *name;
>  	const char *macro;
> @@ -56,6 +52,4 @@ int cmd_ftrace(int argc, const char **argv);
>  int cmd_daemon(int argc, const char **argv);
>  int cmd_kwork(int argc, const char **argv);
>  
> -int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> -		 int pathlen);
>  #endif
> diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
> index e437d7889de6..2d04ece833aa 100644
> --- a/tools/perf/ui/browsers/scripts.c
> +++ b/tools/perf/ui/browsers/scripts.c
> @@ -1,16 +1,18 @@
>  // SPDX-License-Identifier: GPL-2.0
> -#include "../../builtin.h"
> -#include "../../perf.h"
>  #include "../../util/util.h" // perf_exe()
>  #include "../util.h"
> +#include "../../util/evlist.h"
>  #include "../../util/hist.h"
>  #include "../../util/debug.h"
> +#include "../../util/session.h"
>  #include "../../util/symbol.h"
>  #include "../browser.h"
>  #include "../libslang.h"
>  #include "config.h"
> +#include <linux/err.h>
>  #include <linux/string.h>
>  #include <linux/zalloc.h>
> +#include <subcmd/exec-cmd.h>
>  #include <stdlib.h>
>  
>  #define SCRIPT_NAMELEN	128
> @@ -77,6 +79,177 @@ static int scripts_config(const char *var, const char *value, void *data)
>  	return 0;
>  }
>  
> +/*
> + * Some scripts specify the required events in their "xxx-record" file,
> + * this function will check if the events in perf.data match those
> + * mentioned in the "xxx-record".
> + *
> + * Fixme: All existing "xxx-record" are all in good formats "-e event ",
> + * which is covered well now. And new parsing code should be added to
> + * cover the future complex formats like event groups etc.
> + */
> +static int check_ev_match(int dir_fd, const char *scriptname, struct perf_session *session)
> +{
> +	char line[BUFSIZ];
> +	FILE *fp;
> +
> +	{
> +		char filename[FILENAME_MAX + 5];
> +		int fd;
> +
> +		scnprintf(filename, sizeof(filename), "bin/%s-record", scriptname);
> +		fd = openat(dir_fd, filename, O_RDONLY);
> +		if (fd == -1)
> +			return -1;
> +		fp = fdopen(fd, "r");
> +		if (!fp)
> +			return -1;
> +	}
> +
> +	while (fgets(line, sizeof(line), fp)) {
> +		char *p = skip_spaces(line);
> +
> +		if (*p == '#')
> +			continue;
> +
> +		while (strlen(p)) {
> +			int match, len;
> +			struct evsel *pos;
> +			char evname[128];
> +
> +			p = strstr(p, "-e");
> +			if (!p)
> +				break;
> +
> +			p += 2;
> +			p = skip_spaces(p);
> +			len = strcspn(p, " \t");
> +			if (!len)
> +				break;
> +
> +			snprintf(evname, len + 1, "%s", p);
> +
> +			match = 0;
> +			evlist__for_each_entry(session->evlist, pos) {
> +				if (evsel__name_is(pos, evname)) {
> +					match = 1;
> +					break;
> +				}
> +			}
> +
> +			if (!match) {
> +				fclose(fp);
> +				return -1;
> +			}
> +		}
> +	}
> +
> +	fclose(fp);
> +	return 0;
> +}
> +
> +/*
> + * Return -1 if none is found, otherwise the actual scripts number.
> + *
> + * Currently the only user of this function is the script browser, which
> + * will list all statically runnable scripts, select one, execute it and
> + * show the output in a perf browser.
> + */
> +static int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> +		 int pathlen)
> +{
> +	struct dirent *script_dirent, *lang_dirent;
> +	int scripts_dir_fd, lang_dir_fd;
> +	DIR *scripts_dir, *lang_dir;
> +	struct perf_session *session;
> +	struct perf_data data = {
> +		.path = input_name,
> +		.mode = PERF_DATA_MODE_READ,
> +	};
> +	char *temp;
> +	int i = 0;
> +	const char *exec_path = get_argv_exec_path();
> +
> +	session = perf_session__new(&data, NULL);
> +	if (IS_ERR(session))
> +		return PTR_ERR(session);
> +
> +	{
> +		char scripts_path[PATH_MAX];
> +
> +		snprintf(scripts_path, sizeof(scripts_path), "%s/scripts", exec_path);
> +		scripts_dir_fd = open(scripts_path, O_DIRECTORY);
> +		pr_err("Failed to open directory '%s'", scripts_path);
> +		if (scripts_dir_fd == -1) {
> +			perf_session__delete(session);
> +			return -1;
> +		}
> +	}
> +	scripts_dir = fdopendir(scripts_dir_fd);
> +	if (!scripts_dir) {
> +		close(scripts_dir_fd);
> +		perf_session__delete(session);
> +		return -1;
> +	}
> +
> +	while ((lang_dirent = readdir(scripts_dir)) != NULL) {
> +		if (lang_dirent->d_type != DT_DIR &&
> +		    (lang_dirent->d_type == DT_UNKNOWN &&
> +		     !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
> +			continue;
> +		if (!strcmp(lang_dirent->d_name, ".") || !strcmp(lang_dirent->d_name, ".."))
> +			continue;
> +
> +#ifndef HAVE_LIBPERL_SUPPORT
> +		if (strstr(lang_dirent->d_name, "perl"))
> +			continue;
> +#endif
> +#ifndef HAVE_LIBPYTHON_SUPPORT
> +		if (strstr(lang_dirent->d_name, "python"))
> +			continue;
> +#endif
> +
> +		lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name, O_DIRECTORY);
> +		if (lang_dir_fd == -1)
> +			continue;
> +		lang_dir = fdopendir(lang_dir_fd);
> +		if (!lang_dir) {
> +			close(lang_dir_fd);
> +			continue;
> +		}
> +		while ((script_dirent = readdir(lang_dir)) != NULL) {
> +			if (script_dirent->d_type == DT_DIR)
> +				continue;
> +			if (script_dirent->d_type == DT_UNKNOWN &&
> +			    is_directory_at(lang_dir_fd, script_dirent->d_name))
> +				continue;
> +			/* Skip those real time scripts: xxxtop.p[yl] */
> +			if (strstr(script_dirent->d_name, "top."))
> +				continue;
> +			if (i >= num)
> +				break;
> +			scnprintf(scripts_path_array[i], pathlen, "%s/scripts/%s/%s",
> +				exec_path,
> +				lang_dirent->d_name,
> +				script_dirent->d_name);
> +			temp = strchr(script_dirent->d_name, '.');
> +			snprintf(scripts_array[i],
> +				(temp - script_dirent->d_name) + 1,
> +				"%s", script_dirent->d_name);
> +
> +			if (check_ev_match(lang_dir_fd, scripts_array[i], session))
> +				continue;
> +
> +			i++;
> +		}
> +		closedir(lang_dir);
> +	}
> +
> +	closedir(scripts_dir);
> +	perf_session__delete(session);
> +	return i;
> +}
> +
>  /*
>   * When success, will copy the full path of the selected script
>   * into  the buffer pointed by script_name, and return 0.
> diff --git a/tools/perf/util/path.c b/tools/perf/util/path.c
> index 00adf872bf00..9712466c51e2 100644
> --- a/tools/perf/util/path.c
> +++ b/tools/perf/util/path.c
> @@ -68,6 +68,16 @@ bool is_directory(const char *base_path, const struct dirent *dent)
>  	return S_ISDIR(st.st_mode);
>  }
>  
> +bool is_directory_at(int dir_fd, const char *path)
> +{
> +	struct stat st;
> +
> +	if (fstatat(dir_fd, path, &st, /*flags=*/0))
> +		return false;
> +
> +	return S_ISDIR(st.st_mode);
> +}
> +
>  bool is_executable_file(const char *base_path, const struct dirent *dent)
>  {
>  	char path[PATH_MAX];
> diff --git a/tools/perf/util/path.h b/tools/perf/util/path.h
> index d94902c22222..fbafbe7015dd 100644
> --- a/tools/perf/util/path.h
> +++ b/tools/perf/util/path.h
> @@ -12,6 +12,7 @@ int path__join3(char *bf, size_t size, const char *path1, const char *path2, con
>  
>  bool is_regular_file(const char *file);
>  bool is_directory(const char *base_path, const struct dirent *dent);
> +bool is_directory_at(int dir_fd, const char *path);
>  bool is_executable_file(const char *base_path, const struct dirent *dent);
>  
>  #endif /* _PERF_PATH_H */
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index ab67abf3b607..5f11ae88943d 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1306,12 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -int find_scripts(char **scripts_array  __maybe_unused, char **scripts_path_array  __maybe_unused,
> -		int num  __maybe_unused, int pathlen __maybe_unused)
> -{
> -	return -1;
> -}
> -
>  void perf_stat__set_no_csv_summary(int set __maybe_unused)
>  {
>  }
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > The only use of find_scripts is in browser/scripts.c but the
> > definition in builtin causes linking problems requiring a stub in
> > python.c. Move the function to allow the stub to be removed.
> >
> > Rewrite the directory iteration to use openat so that large character
> > arrays aren't needed. The arrays are warned about potential buffer
> > overflows by GCC now that all the code exists in a single C file.
>
> Introducing is_directory_at() should be done as a prep patch, as the
> rest of the patch below could end up being reverted after some other
> patch used it, making the process more difficult.
>
> I mentioned cases like this in the past, so doing it again just for the
> record.

This is highlighted in the commit message:
```
Rewrite the directory iteration to use openat so that large character
arrays aren't needed. The arrays are warned about potential buffer
overflows by GCC now that all the code exists in a single C file.
```
so without the change the code wouldn't build. The new is_directory_at
function is effectively 2 statements fstatat and S_ISDIR on the
result, it is put next to is_directory for consistency but could have
been a static function in the only C file to use it.

For the record, patches introducing 2 line long functions can be
excessively noisy, especially in a 21 patch series. There is always
the declared but not used build error to worry about - here things
couldn't just be simply moved due to triggering a different build
error. Given the simplicity of the function here I made a decision not
to split up the work - the commit message would likely be longer than
the function. The work never intended to introduce is_directory_at but
was forced into it through a desire not to disable compiler warnings.

Thanks,
Ian

>
> - Arnaldo
>
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/builtin-script.c      | 138 ------------------------
> >  tools/perf/builtin.h             |   6 --
> >  tools/perf/ui/browsers/scripts.c | 177 ++++++++++++++++++++++++++++++-
> >  tools/perf/util/path.c           |  10 ++
> >  tools/perf/util/path.h           |   1 +
> >  tools/perf/util/python.c         |   6 --
> >  6 files changed, 186 insertions(+), 152 deletions(-)
> >
> > diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> > index 5d5a1a06d8c6..e9ec74056f71 100644
> > --- a/tools/perf/builtin-script.c
> > +++ b/tools/perf/builtin-script.c
> > @@ -3521,144 +3521,6 @@ static void free_dlarg(void)
> >       free(dlargv);
> >  }
> >
> > -/*
> > - * Some scripts specify the required events in their "xxx-record" file,
> > - * this function will check if the events in perf.data match those
> > - * mentioned in the "xxx-record".
> > - *
> > - * Fixme: All existing "xxx-record" are all in good formats "-e event ",
> > - * which is covered well now. And new parsing code should be added to
> > - * cover the future complex formats like event groups etc.
> > - */
> > -static int check_ev_match(char *dir_name, char *scriptname,
> > -                     struct perf_session *session)
> > -{
> > -     char filename[MAXPATHLEN], evname[128];
> > -     char line[BUFSIZ], *p;
> > -     struct evsel *pos;
> > -     int match, len;
> > -     FILE *fp;
> > -
> > -     scnprintf(filename, MAXPATHLEN, "%s/bin/%s-record", dir_name, scriptname);
> > -
> > -     fp = fopen(filename, "r");
> > -     if (!fp)
> > -             return -1;
> > -
> > -     while (fgets(line, sizeof(line), fp)) {
> > -             p = skip_spaces(line);
> > -             if (*p == '#')
> > -                     continue;
> > -
> > -             while (strlen(p)) {
> > -                     p = strstr(p, "-e");
> > -                     if (!p)
> > -                             break;
> > -
> > -                     p += 2;
> > -                     p = skip_spaces(p);
> > -                     len = strcspn(p, " \t");
> > -                     if (!len)
> > -                             break;
> > -
> > -                     snprintf(evname, len + 1, "%s", p);
> > -
> > -                     match = 0;
> > -                     evlist__for_each_entry(session->evlist, pos) {
> > -                             if (evsel__name_is(pos, evname)) {
> > -                                     match = 1;
> > -                                     break;
> > -                             }
> > -                     }
> > -
> > -                     if (!match) {
> > -                             fclose(fp);
> > -                             return -1;
> > -                     }
> > -             }
> > -     }
> > -
> > -     fclose(fp);
> > -     return 0;
> > -}
> > -
> > -/*
> > - * Return -1 if none is found, otherwise the actual scripts number.
> > - *
> > - * Currently the only user of this function is the script browser, which
> > - * will list all statically runnable scripts, select one, execute it and
> > - * show the output in a perf browser.
> > - */
> > -int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> > -              int pathlen)
> > -{
> > -     struct dirent *script_dirent, *lang_dirent;
> > -     char scripts_path[MAXPATHLEN], lang_path[MAXPATHLEN];
> > -     DIR *scripts_dir, *lang_dir;
> > -     struct perf_session *session;
> > -     struct perf_data data = {
> > -             .path = input_name,
> > -             .mode = PERF_DATA_MODE_READ,
> > -     };
> > -     char *temp;
> > -     int i = 0;
> > -
> > -     session = perf_session__new(&data, NULL);
> > -     if (IS_ERR(session))
> > -             return PTR_ERR(session);
> > -
> > -     snprintf(scripts_path, MAXPATHLEN, "%s/scripts", get_argv_exec_path());
> > -
> > -     scripts_dir = opendir(scripts_path);
> > -     if (!scripts_dir) {
> > -             perf_session__delete(session);
> > -             return -1;
> > -     }
> > -
> > -     for_each_lang(scripts_path, scripts_dir, lang_dirent) {
> > -             scnprintf(lang_path, MAXPATHLEN, "%s/%s", scripts_path,
> > -                       lang_dirent->d_name);
> > -#ifndef HAVE_LIBPERL_SUPPORT
> > -             if (strstr(lang_path, "perl"))
> > -                     continue;
> > -#endif
> > -#ifndef HAVE_LIBPYTHON_SUPPORT
> > -             if (strstr(lang_path, "python"))
> > -                     continue;
> > -#endif
> > -
> > -             lang_dir = opendir(lang_path);
> > -             if (!lang_dir)
> > -                     continue;
> > -
> > -             for_each_script(lang_path, lang_dir, script_dirent) {
> > -                     /* Skip those real time scripts: xxxtop.p[yl] */
> > -                     if (strstr(script_dirent->d_name, "top."))
> > -                             continue;
> > -                     if (i >= num)
> > -                             break;
> > -                     snprintf(scripts_path_array[i], pathlen, "%s/%s",
> > -                             lang_path,
> > -                             script_dirent->d_name);
> > -                     temp = strchr(script_dirent->d_name, '.');
> > -                     snprintf(scripts_array[i],
> > -                             (temp - script_dirent->d_name) + 1,
> > -                             "%s", script_dirent->d_name);
> > -
> > -                     if (check_ev_match(lang_path,
> > -                                     scripts_array[i], session))
> > -                             continue;
> > -
> > -                     i++;
> > -             }
> > -             closedir(lang_dir);
> > -     }
> > -
> > -     closedir(scripts_dir);
> > -     perf_session__delete(session);
> > -     return i;
> > -}
> > -
> >  static char *get_script_path(const char *script_root, const char *suffix)
> >  {
> >       struct dirent *script_dirent, *lang_dirent;
> > diff --git a/tools/perf/builtin.h b/tools/perf/builtin.h
> > index 94f4b3769bf7..a07e93c53848 100644
> > --- a/tools/perf/builtin.h
> > +++ b/tools/perf/builtin.h
> > @@ -2,10 +2,6 @@
> >  #ifndef BUILTIN_H
> >  #define BUILTIN_H
> >
> > -#include <stddef.h>
> > -#include <linux/compiler.h>
> > -#include <tools/config.h>
> > -
> >  struct feature_status {
> >       const char *name;
> >       const char *macro;
> > @@ -56,6 +52,4 @@ int cmd_ftrace(int argc, const char **argv);
> >  int cmd_daemon(int argc, const char **argv);
> >  int cmd_kwork(int argc, const char **argv);
> >
> > -int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> > -              int pathlen);
> >  #endif
> > diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
> > index e437d7889de6..2d04ece833aa 100644
> > --- a/tools/perf/ui/browsers/scripts.c
> > +++ b/tools/perf/ui/browsers/scripts.c
> > @@ -1,16 +1,18 @@
> >  // SPDX-License-Identifier: GPL-2.0
> > -#include "../../builtin.h"
> > -#include "../../perf.h"
> >  #include "../../util/util.h" // perf_exe()
> >  #include "../util.h"
> > +#include "../../util/evlist.h"
> >  #include "../../util/hist.h"
> >  #include "../../util/debug.h"
> > +#include "../../util/session.h"
> >  #include "../../util/symbol.h"
> >  #include "../browser.h"
> >  #include "../libslang.h"
> >  #include "config.h"
> > +#include <linux/err.h>
> >  #include <linux/string.h>
> >  #include <linux/zalloc.h>
> > +#include <subcmd/exec-cmd.h>
> >  #include <stdlib.h>
> >
> >  #define SCRIPT_NAMELEN       128
> > @@ -77,6 +79,177 @@ static int scripts_config(const char *var, const char *value, void *data)
> >       return 0;
> >  }
> >
> > +/*
> > + * Some scripts specify the required events in their "xxx-record" file,
> > + * this function will check if the events in perf.data match those
> > + * mentioned in the "xxx-record".
> > + *
> > + * Fixme: All existing "xxx-record" are all in good formats "-e event ",
> > + * which is covered well now. And new parsing code should be added to
> > + * cover the future complex formats like event groups etc.
> > + */
> > +static int check_ev_match(int dir_fd, const char *scriptname, struct perf_session *session)
> > +{
> > +     char line[BUFSIZ];
> > +     FILE *fp;
> > +
> > +     {
> > +             char filename[FILENAME_MAX + 5];
> > +             int fd;
> > +
> > +             scnprintf(filename, sizeof(filename), "bin/%s-record", scriptname);
> > +             fd = openat(dir_fd, filename, O_RDONLY);
> > +             if (fd == -1)
> > +                     return -1;
> > +             fp = fdopen(fd, "r");
> > +             if (!fp)
> > +                     return -1;
> > +     }
> > +
> > +     while (fgets(line, sizeof(line), fp)) {
> > +             char *p = skip_spaces(line);
> > +
> > +             if (*p == '#')
> > +                     continue;
> > +
> > +             while (strlen(p)) {
> > +                     int match, len;
> > +                     struct evsel *pos;
> > +                     char evname[128];
> > +
> > +                     p = strstr(p, "-e");
> > +                     if (!p)
> > +                             break;
> > +
> > +                     p += 2;
> > +                     p = skip_spaces(p);
> > +                     len = strcspn(p, " \t");
> > +                     if (!len)
> > +                             break;
> > +
> > +                     snprintf(evname, len + 1, "%s", p);
> > +
> > +                     match = 0;
> > +                     evlist__for_each_entry(session->evlist, pos) {
> > +                             if (evsel__name_is(pos, evname)) {
> > +                                     match = 1;
> > +                                     break;
> > +                             }
> > +                     }
> > +
> > +                     if (!match) {
> > +                             fclose(fp);
> > +                             return -1;
> > +                     }
> > +             }
> > +     }
> > +
> > +     fclose(fp);
> > +     return 0;
> > +}
> > +
> > +/*
> > + * Return -1 if none is found, otherwise the actual scripts number.
> > + *
> > + * Currently the only user of this function is the script browser, which
> > + * will list all statically runnable scripts, select one, execute it and
> > + * show the output in a perf browser.
> > + */
> > +static int find_scripts(char **scripts_array, char **scripts_path_array, int num,
> > +              int pathlen)
> > +{
> > +     struct dirent *script_dirent, *lang_dirent;
> > +     int scripts_dir_fd, lang_dir_fd;
> > +     DIR *scripts_dir, *lang_dir;
> > +     struct perf_session *session;
> > +     struct perf_data data = {
> > +             .path = input_name,
> > +             .mode = PERF_DATA_MODE_READ,
> > +     };
> > +     char *temp;
> > +     int i = 0;
> > +     const char *exec_path = get_argv_exec_path();
> > +
> > +     session = perf_session__new(&data, NULL);
> > +     if (IS_ERR(session))
> > +             return PTR_ERR(session);
> > +
> > +     {
> > +             char scripts_path[PATH_MAX];
> > +
> > +             snprintf(scripts_path, sizeof(scripts_path), "%s/scripts", exec_path);
> > +             scripts_dir_fd = open(scripts_path, O_DIRECTORY);
> > +             pr_err("Failed to open directory '%s'", scripts_path);
> > +             if (scripts_dir_fd == -1) {
> > +                     perf_session__delete(session);
> > +                     return -1;
> > +             }
> > +     }
> > +     scripts_dir = fdopendir(scripts_dir_fd);
> > +     if (!scripts_dir) {
> > +             close(scripts_dir_fd);
> > +             perf_session__delete(session);
> > +             return -1;
> > +     }
> > +
> > +     while ((lang_dirent = readdir(scripts_dir)) != NULL) {
> > +             if (lang_dirent->d_type != DT_DIR &&
> > +                 (lang_dirent->d_type == DT_UNKNOWN &&
> > +                  !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
> > +                     continue;
> > +             if (!strcmp(lang_dirent->d_name, ".") || !strcmp(lang_dirent->d_name, ".."))
> > +                     continue;
> > +
> > +#ifndef HAVE_LIBPERL_SUPPORT
> > +             if (strstr(lang_dirent->d_name, "perl"))
> > +                     continue;
> > +#endif
> > +#ifndef HAVE_LIBPYTHON_SUPPORT
> > +             if (strstr(lang_dirent->d_name, "python"))
> > +                     continue;
> > +#endif
> > +
> > +             lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name, O_DIRECTORY);
> > +             if (lang_dir_fd == -1)
> > +                     continue;
> > +             lang_dir = fdopendir(lang_dir_fd);
> > +             if (!lang_dir) {
> > +                     close(lang_dir_fd);
> > +                     continue;
> > +             }
> > +             while ((script_dirent = readdir(lang_dir)) != NULL) {
> > +                     if (script_dirent->d_type == DT_DIR)
> > +                             continue;
> > +                     if (script_dirent->d_type == DT_UNKNOWN &&
> > +                         is_directory_at(lang_dir_fd, script_dirent->d_name))
> > +                             continue;
> > +                     /* Skip those real time scripts: xxxtop.p[yl] */
> > +                     if (strstr(script_dirent->d_name, "top."))
> > +                             continue;
> > +                     if (i >= num)
> > +                             break;
> > +                     scnprintf(scripts_path_array[i], pathlen, "%s/scripts/%s/%s",
> > +                             exec_path,
> > +                             lang_dirent->d_name,
> > +                             script_dirent->d_name);
> > +                     temp = strchr(script_dirent->d_name, '.');
> > +                     snprintf(scripts_array[i],
> > +                             (temp - script_dirent->d_name) + 1,
> > +                             "%s", script_dirent->d_name);
> > +
> > +                     if (check_ev_match(lang_dir_fd, scripts_array[i], session))
> > +                             continue;
> > +
> > +                     i++;
> > +             }
> > +             closedir(lang_dir);
> > +     }
> > +
> > +     closedir(scripts_dir);
> > +     perf_session__delete(session);
> > +     return i;
> > +}
> > +
> >  /*
> >   * When success, will copy the full path of the selected script
> >   * into  the buffer pointed by script_name, and return 0.
> > diff --git a/tools/perf/util/path.c b/tools/perf/util/path.c
> > index 00adf872bf00..9712466c51e2 100644
> > --- a/tools/perf/util/path.c
> > +++ b/tools/perf/util/path.c
> > @@ -68,6 +68,16 @@ bool is_directory(const char *base_path, const struct dirent *dent)
> >       return S_ISDIR(st.st_mode);
> >  }
> >
> > +bool is_directory_at(int dir_fd, const char *path)
> > +{
> > +     struct stat st;
> > +
> > +     if (fstatat(dir_fd, path, &st, /*flags=*/0))
> > +             return false;
> > +
> > +     return S_ISDIR(st.st_mode);
> > +}
> > +
> >  bool is_executable_file(const char *base_path, const struct dirent *dent)
> >  {
> >       char path[PATH_MAX];
> > diff --git a/tools/perf/util/path.h b/tools/perf/util/path.h
> > index d94902c22222..fbafbe7015dd 100644
> > --- a/tools/perf/util/path.h
> > +++ b/tools/perf/util/path.h
> > @@ -12,6 +12,7 @@ int path__join3(char *bf, size_t size, const char *path1, const char *path2, con
> >
> >  bool is_regular_file(const char *file);
> >  bool is_directory(const char *base_path, const struct dirent *dent);
> > +bool is_directory_at(int dir_fd, const char *path);
> >  bool is_executable_file(const char *base_path, const struct dirent *dent);
> >
> >  #endif /* _PERF_PATH_H */
> > diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> > index ab67abf3b607..5f11ae88943d 100644
> > --- a/tools/perf/util/python.c
> > +++ b/tools/perf/util/python.c
> > @@ -1306,12 +1306,6 @@ PyMODINIT_FUNC PyInit_perf(void)
> >  /* The following are stubs to avoid dragging in builtin-* objects. */
> >  /* TODO: move the code out of the builtin-* file into util. */
> >
> > -int find_scripts(char **scripts_array  __maybe_unused, char **scripts_path_array  __maybe_unused,
> > -             int num  __maybe_unused, int pathlen __maybe_unused)
> > -{
> > -     return -1;
> > -}
> > -
> >  void perf_stat__set_no_csv_summary(int set __maybe_unused)
> >  {
> >  }
> > --
> > 2.47.0.163.g1226f6d8fa-goog
> >

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Namhyung Kim]

On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > The only use of find_scripts is in browser/scripts.c but the
> > > definition in builtin causes linking problems requiring a stub in
> > > python.c. Move the function to allow the stub to be removed.
> > >
> > > Rewrite the directory iteration to use openat so that large character
> > > arrays aren't needed. The arrays are warned about potential buffer
> > > overflows by GCC now that all the code exists in a single C file.
> >
> > Introducing is_directory_at() should be done as a prep patch, as the
> > rest of the patch below could end up being reverted after some other
> > patch used it, making the process more difficult.
> >
> > I mentioned cases like this in the past, so doing it again just for the
> > record.
> 
> This is highlighted in the commit message:
> ```
> Rewrite the directory iteration to use openat so that large character
> arrays aren't needed. The arrays are warned about potential buffer
> overflows by GCC now that all the code exists in a single C file.
> ```
> so without the change the code wouldn't build. The new is_directory_at
> function is effectively 2 statements fstatat and S_ISDIR on the
> result, it is put next to is_directory for consistency but could have
> been a static function in the only C file to use it.
> 
> For the record, patches introducing 2 line long functions can be
> excessively noisy, especially in a 21 patch series. There is always
> the declared but not used build error to worry about - here things
> couldn't just be simply moved due to triggering a different build
> error. Given the simplicity of the function here I made a decision not
> to split up the work - the commit message would likely be longer than
> the function. The work never intended to introduce is_directory_at but
> was forced into it through a desire not to disable compiler warnings.

This patch does more than just moving the code which can be easy to miss
something in the middle.  I think you can move the code as is without
introducing build errors and then add new changes like using openat() on
top (you may separate the change out of this series).  I think it's
ok to have a small change if it clearly has different semantics.

Thanks,
Namhyung

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Mon, Nov 4, 2024 at 11:47 AM Namhyung Kim <namhyung@kernel.org> wrote:
>
> On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> > On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> > <acme@kernel.org> wrote:
> > >
> > > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > > The only use of find_scripts is in browser/scripts.c but the
> > > > definition in builtin causes linking problems requiring a stub in
> > > > python.c. Move the function to allow the stub to be removed.
> > > >
> > > > Rewrite the directory iteration to use openat so that large character
> > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > overflows by GCC now that all the code exists in a single C file.
> > >
> > > Introducing is_directory_at() should be done as a prep patch, as the
> > > rest of the patch below could end up being reverted after some other
> > > patch used it, making the process more difficult.
> > >
> > > I mentioned cases like this in the past, so doing it again just for the
> > > record.
> >
> > This is highlighted in the commit message:
> > ```
> > Rewrite the directory iteration to use openat so that large character
> > arrays aren't needed. The arrays are warned about potential buffer
> > overflows by GCC now that all the code exists in a single C file.
> > ```
> > so without the change the code wouldn't build. The new is_directory_at
> > function is effectively 2 statements fstatat and S_ISDIR on the
> > result, it is put next to is_directory for consistency but could have
> > been a static function in the only C file to use it.
> >
> > For the record, patches introducing 2 line long functions can be
> > excessively noisy, especially in a 21 patch series. There is always
> > the declared but not used build error to worry about - here things
> > couldn't just be simply moved due to triggering a different build
> > error. Given the simplicity of the function here I made a decision not
> > to split up the work - the commit message would likely be longer than
> > the function. The work never intended to introduce is_directory_at but
> > was forced into it through a desire not to disable compiler warnings.
>
> This patch does more than just moving the code which can be easy to miss
> something in the middle.  I think you can move the code as is without
> introducing build errors and then add new changes like using openat() on
> top (you may separate the change out of this series).  I think it's
> ok to have a small change if it clearly has different semantics.

If you are trying to bisect to find something that broke a build,
having commits that knowingly break the build will cause the bisect to
fail. The bisect will falsely fail on the known to be broken commit.
It should be unacceptable to knowingly break the build in a commit for
this reason.

Thanks,
Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Arnaldo Carvalho de Melo]

On Mon, Nov 04, 2024 at 12:34:47PM -0800, Ian Rogers wrote:
> On Mon, Nov 4, 2024 at 11:47 AM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> > > On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> > > <acme@kernel.org> wrote:
> > > >
> > > > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > > > The only use of find_scripts is in browser/scripts.c but the
> > > > > definition in builtin causes linking problems requiring a stub in
> > > > > python.c. Move the function to allow the stub to be removed.
> > > > >
> > > > > Rewrite the directory iteration to use openat so that large character
> > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > overflows by GCC now that all the code exists in a single C file.
> > > >
> > > > Introducing is_directory_at() should be done as a prep patch, as the
> > > > rest of the patch below could end up being reverted after some other
> > > > patch used it, making the process more difficult.
> > > >
> > > > I mentioned cases like this in the past, so doing it again just for the
> > > > record.
> > >
> > > This is highlighted in the commit message:
> > > ```
> > > Rewrite the directory iteration to use openat so that large character
> > > arrays aren't needed. The arrays are warned about potential buffer
> > > overflows by GCC now that all the code exists in a single C file.
> > > ```
> > > so without the change the code wouldn't build. The new is_directory_at
> > > function is effectively 2 statements fstatat and S_ISDIR on the
> > > result, it is put next to is_directory for consistency but could have
> > > been a static function in the only C file to use it.
> > >
> > > For the record, patches introducing 2 line long functions can be
> > > excessively noisy, especially in a 21 patch series. There is always
> > > the declared but not used build error to worry about - here things
> > > couldn't just be simply moved due to triggering a different build
> > > error. Given the simplicity of the function here I made a decision not
> > > to split up the work - the commit message would likely be longer than
> > > the function. The work never intended to introduce is_directory_at but
> > > was forced into it through a desire not to disable compiler warnings.
> >
> > This patch does more than just moving the code which can be easy to miss
> > something in the middle.  I think you can move the code as is without
> > introducing build errors and then add new changes like using openat() on
> > top (you may separate the change out of this series).  I think it's
> > ok to have a small change if it clearly has different semantics.
> 
> If you are trying to bisect to find something that broke a build,
> having commits that knowingly break the build will cause the bisect to
> fail. The bisect will falsely fail on the known to be broken commit.

I'm not understanding, AFAIK nobody is advocating for breaking
bisection, just to first instroduce a function, then use it to avoid:

1) Introduce function foo() and use it for feature bar()
2) Somebody else uses function foo()
3) We find a justification to revert 1) but can't, since it will break
   2) so we need to add 4) that removes bar() from 1).

- Arnaldo

> It should be unacceptable to knowingly break the build in a commit for
> this reason.
> 
> Thanks,
> Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Mon, Nov 4, 2024 at 12:39 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> On Mon, Nov 04, 2024 at 12:34:47PM -0800, Ian Rogers wrote:
> > On Mon, Nov 4, 2024 at 11:47 AM Namhyung Kim <namhyung@kernel.org> wrote:
> > >
> > > On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> > > > On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> > > > <acme@kernel.org> wrote:
> > > > >
> > > > > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > > > > The only use of find_scripts is in browser/scripts.c but the
> > > > > > definition in builtin causes linking problems requiring a stub in
> > > > > > python.c. Move the function to allow the stub to be removed.
> > > > > >
> > > > > > Rewrite the directory iteration to use openat so that large character
> > > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > > overflows by GCC now that all the code exists in a single C file.
> > > > >
> > > > > Introducing is_directory_at() should be done as a prep patch, as the
> > > > > rest of the patch below could end up being reverted after some other
> > > > > patch used it, making the process more difficult.
> > > > >
> > > > > I mentioned cases like this in the past, so doing it again just for the
> > > > > record.
> > > >
> > > > This is highlighted in the commit message:
> > > > ```
> > > > Rewrite the directory iteration to use openat so that large character
> > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > overflows by GCC now that all the code exists in a single C file.
> > > > ```
> > > > so without the change the code wouldn't build. The new is_directory_at
> > > > function is effectively 2 statements fstatat and S_ISDIR on the
> > > > result, it is put next to is_directory for consistency but could have
> > > > been a static function in the only C file to use it.
> > > >
> > > > For the record, patches introducing 2 line long functions can be
> > > > excessively noisy, especially in a 21 patch series. There is always
> > > > the declared but not used build error to worry about - here things
> > > > couldn't just be simply moved due to triggering a different build
> > > > error. Given the simplicity of the function here I made a decision not
> > > > to split up the work - the commit message would likely be longer than
> > > > the function. The work never intended to introduce is_directory_at but
> > > > was forced into it through a desire not to disable compiler warnings.
> > >
> > > This patch does more than just moving the code which can be easy to miss
> > > something in the middle.  I think you can move the code as is without
> > > introducing build errors and then add new changes like using openat() on
> > > top (you may separate the change out of this series).  I think it's
> > > ok to have a small change if it clearly has different semantics.
> >
> > If you are trying to bisect to find something that broke a build,
> > having commits that knowingly break the build will cause the bisect to
> > fail. The bisect will falsely fail on the known to be broken commit.
>
> I'm not understanding, AFAIK nobody is advocating for breaking
> bisection, just to first instroduce a function, then use it to avoid:
>
> 1) Introduce function foo() and use it for feature bar()
> 2) Somebody else uses function foo()
> 3) We find a justification to revert 1) but can't, since it will break
>    2) so we need to add 4) that removes bar() from 1).

Namhyung was asking that the c&p of code be 1 patch then "add new
changes like using openat() on top". That is:

patch 1: add is_directory_at - introduce the 2 line helper function
patch 2: move the code
patch 3: update the code to use is_directory_at

patch 2 is known broken as patch 3 is fixing it.

Hopefully this is clear.

Thanks,
Ian


>
> > It should be unacceptable to knowingly break the build in a commit for
> > this reason.
> >
> > Thanks,
> > Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Namhyung Kim]

On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> On Mon, Nov 4, 2024 at 12:39 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2024 at 12:34:47PM -0800, Ian Rogers wrote:
> > > On Mon, Nov 4, 2024 at 11:47 AM Namhyung Kim <namhyung@kernel.org> wrote:
> > > >
> > > > On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> > > > > On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> > > > > <acme@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > > > > > The only use of find_scripts is in browser/scripts.c but the
> > > > > > > definition in builtin causes linking problems requiring a stub in
> > > > > > > python.c. Move the function to allow the stub to be removed.
> > > > > > >
> > > > > > > Rewrite the directory iteration to use openat so that large character
> > > > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > > > overflows by GCC now that all the code exists in a single C file.
> > > > > >
> > > > > > Introducing is_directory_at() should be done as a prep patch, as the
> > > > > > rest of the patch below could end up being reverted after some other
> > > > > > patch used it, making the process more difficult.
> > > > > >
> > > > > > I mentioned cases like this in the past, so doing it again just for the
> > > > > > record.
> > > > >
> > > > > This is highlighted in the commit message:
> > > > > ```
> > > > > Rewrite the directory iteration to use openat so that large character
> > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > overflows by GCC now that all the code exists in a single C file.
> > > > > ```
> > > > > so without the change the code wouldn't build. The new is_directory_at
> > > > > function is effectively 2 statements fstatat and S_ISDIR on the
> > > > > result, it is put next to is_directory for consistency but could have
> > > > > been a static function in the only C file to use it.
> > > > >
> > > > > For the record, patches introducing 2 line long functions can be
> > > > > excessively noisy, especially in a 21 patch series. There is always
> > > > > the declared but not used build error to worry about - here things
> > > > > couldn't just be simply moved due to triggering a different build
> > > > > error. Given the simplicity of the function here I made a decision not
> > > > > to split up the work - the commit message would likely be longer than
> > > > > the function. The work never intended to introduce is_directory_at but
> > > > > was forced into it through a desire not to disable compiler warnings.
> > > >
> > > > This patch does more than just moving the code which can be easy to miss
> > > > something in the middle.  I think you can move the code as is without
> > > > introducing build errors and then add new changes like using openat() on
> > > > top (you may separate the change out of this series).  I think it's
> > > > ok to have a small change if it clearly has different semantics.
> > >
> > > If you are trying to bisect to find something that broke a build,
> > > having commits that knowingly break the build will cause the bisect to
> > > fail. The bisect will falsely fail on the known to be broken commit.
> >
> > I'm not understanding, AFAIK nobody is advocating for breaking
> > bisection, just to first instroduce a function, then use it to avoid:
> >
> > 1) Introduce function foo() and use it for feature bar()
> > 2) Somebody else uses function foo()
> > 3) We find a justification to revert 1) but can't, since it will break
> >    2) so we need to add 4) that removes bar() from 1).
> 
> Namhyung was asking that the c&p of code be 1 patch then "add new
> changes like using openat() on top". That is:
> 
> patch 1: add is_directory_at - introduce the 2 line helper function
> patch 2: move the code
> patch 3: update the code to use is_directory_at
> 
> patch 2 is known broken as patch 3 is fixing it.
> 
> Hopefully this is clear.

Actually I don't care about the patch ordering.  My request is not
to break build and just to separate different changes out. :)

Thanks,
Namhyung

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > On Mon, Nov 4, 2024 at 12:39 PM Arnaldo Carvalho de Melo
> > <acme@kernel.org> wrote:
> > >
> > > On Mon, Nov 04, 2024 at 12:34:47PM -0800, Ian Rogers wrote:
> > > > On Mon, Nov 4, 2024 at 11:47 AM Namhyung Kim <namhyung@kernel.org> wrote:
> > > > >
> > > > > On Thu, Oct 31, 2024 at 01:51:36PM -0700, Ian Rogers wrote:
> > > > > > On Thu, Oct 31, 2024 at 12:18 PM Arnaldo Carvalho de Melo
> > > > > > <acme@kernel.org> wrote:
> > > > > > >
> > > > > > > On Wed, Oct 30, 2024 at 06:42:37PM -0700, Ian Rogers wrote:
> > > > > > > > The only use of find_scripts is in browser/scripts.c but the
> > > > > > > > definition in builtin causes linking problems requiring a stub in
> > > > > > > > python.c. Move the function to allow the stub to be removed.
> > > > > > > >
> > > > > > > > Rewrite the directory iteration to use openat so that large character
> > > > > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > > > > overflows by GCC now that all the code exists in a single C file.
> > > > > > >
> > > > > > > Introducing is_directory_at() should be done as a prep patch, as the
> > > > > > > rest of the patch below could end up being reverted after some other
> > > > > > > patch used it, making the process more difficult.
> > > > > > >
> > > > > > > I mentioned cases like this in the past, so doing it again just for the
> > > > > > > record.
> > > > > >
> > > > > > This is highlighted in the commit message:
> > > > > > ```
> > > > > > Rewrite the directory iteration to use openat so that large character
> > > > > > arrays aren't needed. The arrays are warned about potential buffer
> > > > > > overflows by GCC now that all the code exists in a single C file.
> > > > > > ```
> > > > > > so without the change the code wouldn't build. The new is_directory_at
> > > > > > function is effectively 2 statements fstatat and S_ISDIR on the
> > > > > > result, it is put next to is_directory for consistency but could have
> > > > > > been a static function in the only C file to use it.
> > > > > >
> > > > > > For the record, patches introducing 2 line long functions can be
> > > > > > excessively noisy, especially in a 21 patch series. There is always
> > > > > > the declared but not used build error to worry about - here things
> > > > > > couldn't just be simply moved due to triggering a different build
> > > > > > error. Given the simplicity of the function here I made a decision not
> > > > > > to split up the work - the commit message would likely be longer than
> > > > > > the function. The work never intended to introduce is_directory_at but
> > > > > > was forced into it through a desire not to disable compiler warnings.
> > > > >
> > > > > This patch does more than just moving the code which can be easy to miss
> > > > > something in the middle.  I think you can move the code as is without
> > > > > introducing build errors and then add new changes like using openat() on
> > > > > top (you may separate the change out of this series).  I think it's
> > > > > ok to have a small change if it clearly has different semantics.
> > > >
> > > > If you are trying to bisect to find something that broke a build,
> > > > having commits that knowingly break the build will cause the bisect to
> > > > fail. The bisect will falsely fail on the known to be broken commit.
> > >
> > > I'm not understanding, AFAIK nobody is advocating for breaking
> > > bisection, just to first instroduce a function, then use it to avoid:
> > >
> > > 1) Introduce function foo() and use it for feature bar()
> > > 2) Somebody else uses function foo()
> > > 3) We find a justification to revert 1) but can't, since it will break
> > >    2) so we need to add 4) that removes bar() from 1).
> >
> > Namhyung was asking that the c&p of code be 1 patch then "add new
> > changes like using openat() on top". That is:
> >
> > patch 1: add is_directory_at - introduce the 2 line helper function
> > patch 2: move the code
> > patch 3: update the code to use is_directory_at
> >
> > patch 2 is known broken as patch 3 is fixing it.
> >
> > Hopefully this is clear.
>
> Actually I don't care about the patch ordering.  My request is not
> to break build and just to separate different changes out. :)

So, patch 2 can't be separated from patch 3 - are we agreed? So we
squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
bar of a meaningful change, so we squash that. We end up with this
patch. If there's a later revert and a dependence of the 2 liner, just
don't revert that part of the change. We've never had such a revert so
it is hard to see why we need to generate so much churn because of it.

Thanks,
Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Namhyung Kim]

On Mon, Nov 04, 2024 at 01:06:35PM -0800, Ian Rogers wrote:
> On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > > Namhyung was asking that the c&p of code be 1 patch then "add new
> > > changes like using openat() on top". That is:
> > >
> > > patch 1: add is_directory_at - introduce the 2 line helper function
> > > patch 2: move the code
> > > patch 3: update the code to use is_directory_at
> > >
> > > patch 2 is known broken as patch 3 is fixing it.
> > >
> > > Hopefully this is clear.
> >
> > Actually I don't care about the patch ordering.  My request is not
> > to break build and just to separate different changes out. :)
> 
> So, patch 2 can't be separated from patch 3 - are we agreed? So we
> squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
> bar of a meaningful change, so we squash that. We end up with this
> patch. If there's a later revert and a dependence of the 2 liner, just
> don't revert that part of the change. We've never had such a revert so
> it is hard to see why we need to generate so much churn because of it.

As I said the patch 1 should be the c&p and no need to introduce
is_directory_at() before that.  Why not doing

 patch1: move the code
 patch2: add and use is_directory_at() + openat()

?

Thanks,
Namhyung

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Mon, Nov 4, 2024 at 2:09 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> On Mon, Nov 04, 2024 at 01:06:35PM -0800, Ian Rogers wrote:
> > On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > >
> > > On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > > > Namhyung was asking that the c&p of code be 1 patch then "add new
> > > > changes like using openat() on top". That is:
> > > >
> > > > patch 1: add is_directory_at - introduce the 2 line helper function
> > > > patch 2: move the code
> > > > patch 3: update the code to use is_directory_at
> > > >
> > > > patch 2 is known broken as patch 3 is fixing it.
> > > >
> > > > Hopefully this is clear.
> > >
> > > Actually I don't care about the patch ordering.  My request is not
> > > to break build and just to separate different changes out. :)
> >
> > So, patch 2 can't be separated from patch 3 - are we agreed? So we
> > squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
> > bar of a meaningful change, so we squash that. We end up with this
> > patch. If there's a later revert and a dependence of the 2 liner, just
> > don't revert that part of the change. We've never had such a revert so
> > it is hard to see why we need to generate so much churn because of it.
>
> As I said the patch 1 should be the c&p and no need to introduce
> is_directory_at() before that.  Why not doing
>
>  patch1: move the code
>  patch2: add and use is_directory_at() + openat()
>
> ?

Because placing all the code in 1 file expands GCC's analysis and the
build fails. In the commit message I describe this:
"The arrays are warned about potential buffer overflows by GCC now
that all the code exists in a single C file."
A standard unsound workaround to this is to change "sizeof(...)" to
"sizeof(...) - 1", as it is ugly I added is_directory_at to not suffer
the problem as the arrays are gone.

Thanks,
Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Namhyung Kim]

On Mon, Nov 04, 2024 at 02:20:30PM -0800, Ian Rogers wrote:
> On Mon, Nov 4, 2024 at 2:09 PM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2024 at 01:06:35PM -0800, Ian Rogers wrote:
> > > On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > >
> > > > On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > > > > Namhyung was asking that the c&p of code be 1 patch then "add new
> > > > > changes like using openat() on top". That is:
> > > > >
> > > > > patch 1: add is_directory_at - introduce the 2 line helper function
> > > > > patch 2: move the code
> > > > > patch 3: update the code to use is_directory_at
> > > > >
> > > > > patch 2 is known broken as patch 3 is fixing it.
> > > > >
> > > > > Hopefully this is clear.
> > > >
> > > > Actually I don't care about the patch ordering.  My request is not
> > > > to break build and just to separate different changes out. :)
> > >
> > > So, patch 2 can't be separated from patch 3 - are we agreed? So we
> > > squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
> > > bar of a meaningful change, so we squash that. We end up with this
> > > patch. If there's a later revert and a dependence of the 2 liner, just
> > > don't revert that part of the change. We've never had such a revert so
> > > it is hard to see why we need to generate so much churn because of it.
> >
> > As I said the patch 1 should be the c&p and no need to introduce
> > is_directory_at() before that.  Why not doing
> >
> >  patch1: move the code
> >  patch2: add and use is_directory_at() + openat()
> >
> > ?
> 
> Because placing all the code in 1 file expands GCC's analysis and the
> build fails. In the commit message I describe this:
> "The arrays are warned about potential buffer overflows by GCC now
> that all the code exists in a single C file."
> A standard unsound workaround to this is to change "sizeof(...)" to
> "sizeof(...) - 1", as it is ugly I added is_directory_at to not suffer
> the problem as the arrays are gone.

Ok, it's strange that this type of analysis depends on the placement.
Anyway it seems there's a problem in the code already.  Then we can fix
it first and then move.  How about this?

 patch1: add and use is_directory_at() + openat()
 patch2: move the code

Thanks,
Namhyung

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Ian Rogers]

On Mon, Nov 4, 2024 at 3:22 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> On Mon, Nov 04, 2024 at 02:20:30PM -0800, Ian Rogers wrote:
> > On Mon, Nov 4, 2024 at 2:09 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > >
> > > On Mon, Nov 04, 2024 at 01:06:35PM -0800, Ian Rogers wrote:
> > > > On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > > >
> > > > > On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > > > > > Namhyung was asking that the c&p of code be 1 patch then "add new
> > > > > > changes like using openat() on top". That is:
> > > > > >
> > > > > > patch 1: add is_directory_at - introduce the 2 line helper function
> > > > > > patch 2: move the code
> > > > > > patch 3: update the code to use is_directory_at
> > > > > >
> > > > > > patch 2 is known broken as patch 3 is fixing it.
> > > > > >
> > > > > > Hopefully this is clear.
> > > > >
> > > > > Actually I don't care about the patch ordering.  My request is not
> > > > > to break build and just to separate different changes out. :)
> > > >
> > > > So, patch 2 can't be separated from patch 3 - are we agreed? So we
> > > > squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
> > > > bar of a meaningful change, so we squash that. We end up with this
> > > > patch. If there's a later revert and a dependence of the 2 liner, just
> > > > don't revert that part of the change. We've never had such a revert so
> > > > it is hard to see why we need to generate so much churn because of it.
> > >
> > > As I said the patch 1 should be the c&p and no need to introduce
> > > is_directory_at() before that.  Why not doing
> > >
> > >  patch1: move the code
> > >  patch2: add and use is_directory_at() + openat()
> > >
> > > ?
> >
> > Because placing all the code in 1 file expands GCC's analysis and the
> > build fails. In the commit message I describe this:
> > "The arrays are warned about potential buffer overflows by GCC now
> > that all the code exists in a single C file."
> > A standard unsound workaround to this is to change "sizeof(...)" to
> > "sizeof(...) - 1", as it is ugly I added is_directory_at to not suffer
> > the problem as the arrays are gone.
>
> Ok, it's strange that this type of analysis depends on the placement.
> Anyway it seems there's a problem in the code already.  Then we can fix
> it first and then move.  How about this?
>
>  patch1: add and use is_directory_at() + openat()
>  patch2: move the code

I'm happy if the maintainers do that.

Thanks,
Ian

Re: [PATCH v5 06/21] perf script: Move find_scripts to browser/scripts.c

[Namhyung Kim]

On Mon, Nov 04, 2024 at 03:28:15PM -0800, Ian Rogers wrote:
> On Mon, Nov 4, 2024 at 3:22 PM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Mon, Nov 04, 2024 at 02:20:30PM -0800, Ian Rogers wrote:
> > > On Mon, Nov 4, 2024 at 2:09 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > >
> > > > On Mon, Nov 04, 2024 at 01:06:35PM -0800, Ian Rogers wrote:
> > > > > On Mon, Nov 4, 2024 at 1:00 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > > > >
> > > > > > On Mon, Nov 04, 2024 at 12:48:01PM -0800, Ian Rogers wrote:
> > > > > > > Namhyung was asking that the c&p of code be 1 patch then "add new
> > > > > > > changes like using openat() on top". That is:
> > > > > > >
> > > > > > > patch 1: add is_directory_at - introduce the 2 line helper function
> > > > > > > patch 2: move the code
> > > > > > > patch 3: update the code to use is_directory_at
> > > > > > >
> > > > > > > patch 2 is known broken as patch 3 is fixing it.
> > > > > > >
> > > > > > > Hopefully this is clear.
> > > > > >
> > > > > > Actually I don't care about the patch ordering.  My request is not
> > > > > > to break build and just to separate different changes out. :)
> > > > >
> > > > > So, patch 2 can't be separated from patch 3 - are we agreed? So we
> > > > > squash patch 2 with patch 3. Patch 1 is trivial and fails to meet the
> > > > > bar of a meaningful change, so we squash that. We end up with this
> > > > > patch. If there's a later revert and a dependence of the 2 liner, just
> > > > > don't revert that part of the change. We've never had such a revert so
> > > > > it is hard to see why we need to generate so much churn because of it.
> > > >
> > > > As I said the patch 1 should be the c&p and no need to introduce
> > > > is_directory_at() before that.  Why not doing
> > > >
> > > >  patch1: move the code
> > > >  patch2: add and use is_directory_at() + openat()
> > > >
> > > > ?
> > >
> > > Because placing all the code in 1 file expands GCC's analysis and the
> > > build fails. In the commit message I describe this:
> > > "The arrays are warned about potential buffer overflows by GCC now
> > > that all the code exists in a single C file."
> > > A standard unsound workaround to this is to change "sizeof(...)" to
> > > "sizeof(...) - 1", as it is ugly I added is_directory_at to not suffer
> > > the problem as the arrays are gone.
> >
> > Ok, it's strange that this type of analysis depends on the placement.
> > Anyway it seems there's a problem in the code already.  Then we can fix
> > it first and then move.  How about this?
> >
> >  patch1: add and use is_directory_at() + openat()
> >  patch2: move the code
> 
> I'm happy if the maintainers do that.

It's probably not gonna happen anytime soon and I'd be happy if you
could do that.

Thanks,
Namhyung

[PATCH v5 07/21] perf stat: Move stat_config into config.c

[Ian Rogers]

stat_config is accessed by config.c via helper functions, but declared
in builtin-stat. Move to util/config.c so that stub functions aren't
needed in python.c which doesn't link against the builtin files.

To avoid name conflicts change builtin-script to use the same
stat_config as builtin-stat. Rename local variables in tests to avoid
shadow declaration warnings.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c |  1 -
 tools/perf/builtin-stat.c   | 27 ---------------------------
 tools/perf/tests/stat.c     | 16 +++++++++-------
 tools/perf/util/config.c    | 27 +++++++++++++++++++++++++++
 tools/perf/util/python.c    |  9 ---------
 tools/perf/util/stat.h      |  3 ++-
 6 files changed, 38 insertions(+), 45 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index e9ec74056f71..62e851fdf5ca 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -85,7 +85,6 @@ static bool			system_wide;
 static bool			print_flags;
 static const char		*cpu_list;
 static DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
-static struct perf_stat_config	stat_config;
 static int			max_blocks;
 static bool			native_arch;
 static struct dlfilter		*dlfilter;
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index c12158d99353..a1625443e5a5 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -112,8 +112,6 @@ static struct target target = {
 	.uid	= UINT_MAX,
 };
 
-#define METRIC_ONLY_LEN 20
-
 static volatile sig_atomic_t	child_pid			= -1;
 static int			detailed_run			=  0;
 static bool			transaction_run;
@@ -151,21 +149,6 @@ static struct perf_stat		perf_stat;
 
 static volatile sig_atomic_t done = 0;
 
-static struct perf_stat_config stat_config = {
-	.aggr_mode		= AGGR_GLOBAL,
-	.aggr_level		= MAX_CACHE_LVL + 1,
-	.scale			= true,
-	.unit_width		= 4, /* strlen("unit") */
-	.run_count		= 1,
-	.metric_only_len	= METRIC_ONLY_LEN,
-	.walltime_nsecs_stats	= &walltime_nsecs_stats,
-	.ru_stats		= &ru_stats,
-	.big_num		= true,
-	.ctl_fd			= -1,
-	.ctl_fd_ack		= -1,
-	.iostat_run		= false,
-};
-
 /* Options set from the command line. */
 struct opt_aggr_mode {
 	bool node, socket, die, cluster, cache, core, thread, no_aggr;
@@ -1071,16 +1054,6 @@ static void sig_atexit(void)
 	kill(getpid(), signr);
 }
 
-void perf_stat__set_big_num(int set)
-{
-	stat_config.big_num = (set != 0);
-}
-
-void perf_stat__set_no_csv_summary(int set)
-{
-	stat_config.no_csv_summary = (set != 0);
-}
-
 static int stat__set_big_num(const struct option *opt __maybe_unused,
 			     const char *s __maybe_unused, int unset)
 {
diff --git a/tools/perf/tests/stat.c b/tools/perf/tests/stat.c
index 6468cc0d0204..d60983657bad 100644
--- a/tools/perf/tests/stat.c
+++ b/tools/perf/tests/stat.c
@@ -27,7 +27,7 @@ static int process_stat_config_event(const struct perf_tool *tool __maybe_unused
 				     struct machine *machine __maybe_unused)
 {
 	struct perf_record_stat_config *config = &event->stat_config;
-	struct perf_stat_config stat_config = {};
+	struct perf_stat_config test_stat_config = {};
 
 #define HAS(term, val) \
 	has_term(config, PERF_STAT_CONFIG_TERM__##term, val)
@@ -39,25 +39,27 @@ static int process_stat_config_event(const struct perf_tool *tool __maybe_unused
 
 #undef HAS
 
-	perf_event__read_stat_config(&stat_config, config);
+	perf_event__read_stat_config(&test_stat_config, config);
 
-	TEST_ASSERT_VAL("wrong aggr_mode", stat_config.aggr_mode == AGGR_CORE);
-	TEST_ASSERT_VAL("wrong scale",     stat_config.scale == 1);
-	TEST_ASSERT_VAL("wrong interval",  stat_config.interval == 1);
+	TEST_ASSERT_VAL("wrong aggr_mode", test_stat_config.aggr_mode == AGGR_CORE);
+	TEST_ASSERT_VAL("wrong scale",     test_stat_config.scale == 1);
+	TEST_ASSERT_VAL("wrong interval",  test_stat_config.interval == 1);
 	return 0;
 }
 
 static int test__synthesize_stat_config(struct test_suite *test __maybe_unused,
 					int subtest __maybe_unused)
 {
-	struct perf_stat_config stat_config = {
+	struct perf_stat_config test_stat_config = {
 		.aggr_mode	= AGGR_CORE,
 		.scale		= 1,
 		.interval	= 1,
 	};
 
 	TEST_ASSERT_VAL("failed to synthesize stat_config",
-		!perf_event__synthesize_stat_config(NULL, &stat_config, process_stat_config_event, NULL));
+		!perf_event__synthesize_stat_config(NULL, &test_stat_config,
+						    process_stat_config_event,
+						    NULL));
 
 	return 0;
 }
diff --git a/tools/perf/util/config.c b/tools/perf/util/config.c
index 68f9407ca74b..2d07c9257a1a 100644
--- a/tools/perf/util/config.c
+++ b/tools/perf/util/config.c
@@ -13,6 +13,7 @@
 #include <sys/param.h>
 #include "cache.h"
 #include "callchain.h"
+#include "header.h"
 #include <subcmd/exec-cmd.h>
 #include "util/event.h"  /* proc_map_timeout */
 #include "util/hist.h"  /* perf_hist_config */
@@ -34,6 +35,22 @@
 
 #define DEBUG_CACHE_DIR ".debug"
 
+#define METRIC_ONLY_LEN 20
+
+struct perf_stat_config stat_config = {
+	.aggr_mode		= AGGR_GLOBAL,
+	.aggr_level		= MAX_CACHE_LVL + 1,
+	.scale			= true,
+	.unit_width		= 4, /* strlen("unit") */
+	.run_count		= 1,
+	.metric_only_len	= METRIC_ONLY_LEN,
+	.walltime_nsecs_stats	= &walltime_nsecs_stats,
+	.ru_stats		= &ru_stats,
+	.big_num		= true,
+	.ctl_fd			= -1,
+	.ctl_fd_ack		= -1,
+	.iostat_run		= false,
+};
 
 char buildid_dir[MAXPATHLEN]; /* root dir for buildid, binary cache */
 
@@ -455,6 +472,16 @@ static int perf_ui_config(const char *var, const char *value)
 	return 0;
 }
 
+void perf_stat__set_big_num(int set)
+{
+	stat_config.big_num = (set != 0);
+}
+
+static void perf_stat__set_no_csv_summary(int set)
+{
+	stat_config.no_csv_summary = (set != 0);
+}
+
 static int perf_stat_config(const char *var, const char *value)
 {
 	if (!strcmp(var, "stat.big-num"))
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 5f11ae88943d..0fa8e27769be 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -17,7 +17,6 @@
 #include "trace-event.h"
 #include "mmap.h"
 #include "util/env.h"
-#include "util/kvm-stat.h"
 #include "util/kwork.h"
 #include "util/sample.h"
 #include "util/lock-contention.h"
@@ -1306,14 +1305,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-void perf_stat__set_no_csv_summary(int set __maybe_unused)
-{
-}
-
-void perf_stat__set_big_num(int set __maybe_unused)
-{
-}
-
 int script_spec_register(const char *spec __maybe_unused, struct scripting_ops *ops __maybe_unused)
 {
 	return -1;
diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
index 6f8cff3cd39a..2fda9acd7374 100644
--- a/tools/perf/util/stat.h
+++ b/tools/perf/util/stat.h
@@ -117,8 +117,9 @@ struct perf_stat_config {
 	unsigned int		topdown_level;
 };
 
+extern struct perf_stat_config stat_config;
+
 void perf_stat__set_big_num(int set);
-void perf_stat__set_no_csv_summary(int set);
 
 void update_stats(struct stats *stats, u64 val);
 double avg_stats(struct stats *stats);
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 07/21] perf stat: Move stat_config into config.c

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:38PM -0700, Ian Rogers wrote:
> stat_config is accessed by config.c via helper functions, but declared
> in builtin-stat. Move to util/config.c so that stub functions aren't
> needed in python.c which doesn't link against the builtin files.
> 
> To avoid name conflicts change builtin-script to use the same
> stat_config as builtin-stat. Rename local variables in tests to avoid
> shadow declaration warnings.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-script.c |  1 -
>  tools/perf/builtin-stat.c   | 27 ---------------------------
>  tools/perf/tests/stat.c     | 16 +++++++++-------
>  tools/perf/util/config.c    | 27 +++++++++++++++++++++++++++
>  tools/perf/util/python.c    |  9 ---------
>  tools/perf/util/stat.h      |  3 ++-
>  6 files changed, 38 insertions(+), 45 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index e9ec74056f71..62e851fdf5ca 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -85,7 +85,6 @@ static bool			system_wide;
>  static bool			print_flags;
>  static const char		*cpu_list;
>  static DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
> -static struct perf_stat_config	stat_config;
>  static int			max_blocks;
>  static bool			native_arch;
>  static struct dlfilter		*dlfilter;
> diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
> index c12158d99353..a1625443e5a5 100644
> --- a/tools/perf/builtin-stat.c
> +++ b/tools/perf/builtin-stat.c
> @@ -112,8 +112,6 @@ static struct target target = {
>  	.uid	= UINT_MAX,
>  };
>  
> -#define METRIC_ONLY_LEN 20
> -
>  static volatile sig_atomic_t	child_pid			= -1;
>  static int			detailed_run			=  0;
>  static bool			transaction_run;
> @@ -151,21 +149,6 @@ static struct perf_stat		perf_stat;
>  
>  static volatile sig_atomic_t done = 0;
>  
> -static struct perf_stat_config stat_config = {
> -	.aggr_mode		= AGGR_GLOBAL,
> -	.aggr_level		= MAX_CACHE_LVL + 1,
> -	.scale			= true,
> -	.unit_width		= 4, /* strlen("unit") */
> -	.run_count		= 1,
> -	.metric_only_len	= METRIC_ONLY_LEN,
> -	.walltime_nsecs_stats	= &walltime_nsecs_stats,
> -	.ru_stats		= &ru_stats,
> -	.big_num		= true,
> -	.ctl_fd			= -1,
> -	.ctl_fd_ack		= -1,
> -	.iostat_run		= false,
> -};
> -
>  /* Options set from the command line. */
>  struct opt_aggr_mode {
>  	bool node, socket, die, cluster, cache, core, thread, no_aggr;
> @@ -1071,16 +1054,6 @@ static void sig_atexit(void)
>  	kill(getpid(), signr);
>  }
>  
> -void perf_stat__set_big_num(int set)
> -{
> -	stat_config.big_num = (set != 0);
> -}
> -
> -void perf_stat__set_no_csv_summary(int set)
> -{
> -	stat_config.no_csv_summary = (set != 0);
> -}
> -
>  static int stat__set_big_num(const struct option *opt __maybe_unused,
>  			     const char *s __maybe_unused, int unset)
>  {
> diff --git a/tools/perf/tests/stat.c b/tools/perf/tests/stat.c
> index 6468cc0d0204..d60983657bad 100644
> --- a/tools/perf/tests/stat.c
> +++ b/tools/perf/tests/stat.c
> @@ -27,7 +27,7 @@ static int process_stat_config_event(const struct perf_tool *tool __maybe_unused
>  				     struct machine *machine __maybe_unused)
>  {
>  	struct perf_record_stat_config *config = &event->stat_config;
> -	struct perf_stat_config stat_config = {};
> +	struct perf_stat_config test_stat_config = {};
>  
>  #define HAS(term, val) \
>  	has_term(config, PERF_STAT_CONFIG_TERM__##term, val)
> @@ -39,25 +39,27 @@ static int process_stat_config_event(const struct perf_tool *tool __maybe_unused
>  
>  #undef HAS
>  
> -	perf_event__read_stat_config(&stat_config, config);
> +	perf_event__read_stat_config(&test_stat_config, config);
>  
> -	TEST_ASSERT_VAL("wrong aggr_mode", stat_config.aggr_mode == AGGR_CORE);
> -	TEST_ASSERT_VAL("wrong scale",     stat_config.scale == 1);
> -	TEST_ASSERT_VAL("wrong interval",  stat_config.interval == 1);
> +	TEST_ASSERT_VAL("wrong aggr_mode", test_stat_config.aggr_mode == AGGR_CORE);
> +	TEST_ASSERT_VAL("wrong scale",     test_stat_config.scale == 1);
> +	TEST_ASSERT_VAL("wrong interval",  test_stat_config.interval == 1);
>  	return 0;
>  }
>  
>  static int test__synthesize_stat_config(struct test_suite *test __maybe_unused,
>  					int subtest __maybe_unused)
>  {
> -	struct perf_stat_config stat_config = {
> +	struct perf_stat_config test_stat_config = {
>  		.aggr_mode	= AGGR_CORE,
>  		.scale		= 1,
>  		.interval	= 1,
>  	};
>  
>  	TEST_ASSERT_VAL("failed to synthesize stat_config",
> -		!perf_event__synthesize_stat_config(NULL, &stat_config, process_stat_config_event, NULL));
> +		!perf_event__synthesize_stat_config(NULL, &test_stat_config,
> +						    process_stat_config_event,
> +						    NULL));
>  
>  	return 0;
>  }
> diff --git a/tools/perf/util/config.c b/tools/perf/util/config.c
> index 68f9407ca74b..2d07c9257a1a 100644
> --- a/tools/perf/util/config.c
> +++ b/tools/perf/util/config.c
> @@ -13,6 +13,7 @@
>  #include <sys/param.h>
>  #include "cache.h"
>  #include "callchain.h"
> +#include "header.h"
>  #include <subcmd/exec-cmd.h>
>  #include "util/event.h"  /* proc_map_timeout */
>  #include "util/hist.h"  /* perf_hist_config */
> @@ -34,6 +35,22 @@
>  
>  #define DEBUG_CACHE_DIR ".debug"
>  
> +#define METRIC_ONLY_LEN 20
> +
> +struct perf_stat_config stat_config = {
> +	.aggr_mode		= AGGR_GLOBAL,
> +	.aggr_level		= MAX_CACHE_LVL + 1,
> +	.scale			= true,
> +	.unit_width		= 4, /* strlen("unit") */
> +	.run_count		= 1,
> +	.metric_only_len	= METRIC_ONLY_LEN,
> +	.walltime_nsecs_stats	= &walltime_nsecs_stats,
> +	.ru_stats		= &ru_stats,
> +	.big_num		= true,
> +	.ctl_fd			= -1,
> +	.ctl_fd_ack		= -1,
> +	.iostat_run		= false,
> +};
>  
>  char buildid_dir[MAXPATHLEN]; /* root dir for buildid, binary cache */
>  
> @@ -455,6 +472,16 @@ static int perf_ui_config(const char *var, const char *value)
>  	return 0;
>  }
>  
> +void perf_stat__set_big_num(int set)
> +{
> +	stat_config.big_num = (set != 0);
> +}
> +
> +static void perf_stat__set_no_csv_summary(int set)
> +{
> +	stat_config.no_csv_summary = (set != 0);
> +}
> +
>  static int perf_stat_config(const char *var, const char *value)
>  {
>  	if (!strcmp(var, "stat.big-num"))
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 5f11ae88943d..0fa8e27769be 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -17,7 +17,6 @@
>  #include "trace-event.h"
>  #include "mmap.h"
>  #include "util/env.h"
> -#include "util/kvm-stat.h"
>  #include "util/kwork.h"
>  #include "util/sample.h"
>  #include "util/lock-contention.h"
> @@ -1306,14 +1305,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -void perf_stat__set_no_csv_summary(int set __maybe_unused)
> -{
> -}
> -
> -void perf_stat__set_big_num(int set __maybe_unused)
> -{
> -}
> -
>  int script_spec_register(const char *spec __maybe_unused, struct scripting_ops *ops __maybe_unused)
>  {
>  	return -1;
> diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
> index 6f8cff3cd39a..2fda9acd7374 100644
> --- a/tools/perf/util/stat.h
> +++ b/tools/perf/util/stat.h
> @@ -117,8 +117,9 @@ struct perf_stat_config {
>  	unsigned int		topdown_level;
>  };
>  
> +extern struct perf_stat_config stat_config;
> +
>  void perf_stat__set_big_num(int set);
> -void perf_stat__set_no_csv_summary(int set);
>  
>  void update_stats(struct stats *stats, u64 val);
>  double avg_stats(struct stats *stats);
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 08/21] perf script: Move script_spec code to trace-event-scripting.c

[Ian Rogers]

The script_spec code is referenced in util/trace-event-scripting but
the list was in builtin-script, accessed via a function that required
a stub function in python.c. Move all the logic to
trace-event-scripting, with lookup and foreach functions exposed for
builtin-script's benefit.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c             | 67 +---------------------
 tools/perf/util/python.c                |  5 --
 tools/perf/util/trace-event-scripting.c | 75 +++++++++++++++++++++++++
 tools/perf/util/trace-event.h           |  3 +-
 4 files changed, 80 insertions(+), 70 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 62e851fdf5ca..11c0ee8c1afc 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -2956,79 +2956,18 @@ static int __cmd_script(struct perf_script *script)
 	return ret;
 }
 
-struct script_spec {
-	struct list_head	node;
-	struct scripting_ops	*ops;
-	char			spec[];
-};
-
-static LIST_HEAD(script_specs);
-
-static struct script_spec *script_spec__new(const char *spec,
-					    struct scripting_ops *ops)
+static int list_available_languages_cb(struct scripting_ops *ops, const char *spec)
 {
-	struct script_spec *s = malloc(sizeof(*s) + strlen(spec) + 1);
-
-	if (s != NULL) {
-		strcpy(s->spec, spec);
-		s->ops = ops;
-	}
-
-	return s;
-}
-
-static void script_spec__add(struct script_spec *s)
-{
-	list_add_tail(&s->node, &script_specs);
-}
-
-static struct script_spec *script_spec__find(const char *spec)
-{
-	struct script_spec *s;
-
-	list_for_each_entry(s, &script_specs, node)
-		if (strcasecmp(s->spec, spec) == 0)
-			return s;
-	return NULL;
-}
-
-int script_spec_register(const char *spec, struct scripting_ops *ops)
-{
-	struct script_spec *s;
-
-	s = script_spec__find(spec);
-	if (s)
-		return -1;
-
-	s = script_spec__new(spec, ops);
-	if (!s)
-		return -1;
-	else
-		script_spec__add(s);
-
+	fprintf(stderr, "  %-42s [%s]\n", spec, ops->name);
 	return 0;
 }
 
-static struct scripting_ops *script_spec__lookup(const char *spec)
-{
-	struct script_spec *s = script_spec__find(spec);
-	if (!s)
-		return NULL;
-
-	return s->ops;
-}
-
 static void list_available_languages(void)
 {
-	struct script_spec *s;
-
 	fprintf(stderr, "\n");
 	fprintf(stderr, "Scripting language extensions (used in "
 		"perf script -s [spec:]script.[spec]):\n\n");
-
-	list_for_each_entry(s, &script_specs, node)
-		fprintf(stderr, "  %-42s [%s]\n", s->spec, s->ops->name);
-
+	script_spec__for_each(&list_available_languages_cb);
 	fprintf(stderr, "\n");
 }
 
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 0fa8e27769be..3d938fe2de6a 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1305,11 +1305,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-int script_spec_register(const char *spec __maybe_unused, struct scripting_ops *ops __maybe_unused)
-{
-	return -1;
-}
-
 arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch __maybe_unused)
 {
 	return NULL;
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index 8abb7a7b6888..ad62d8e5a368 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -24,6 +24,81 @@ unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
 
 struct scripting_context *scripting_context;
 
+struct script_spec {
+	struct list_head	node;
+	struct scripting_ops	*ops;
+	char			spec[];
+};
+
+static LIST_HEAD(script_specs);
+
+static struct script_spec *script_spec__new(const char *spec,
+					    struct scripting_ops *ops)
+{
+	struct script_spec *s = malloc(sizeof(*s) + strlen(spec) + 1);
+
+	if (s != NULL) {
+		strcpy(s->spec, spec);
+		s->ops = ops;
+	}
+
+	return s;
+}
+
+static void script_spec__add(struct script_spec *s)
+{
+	list_add_tail(&s->node, &script_specs);
+}
+
+static struct script_spec *script_spec__find(const char *spec)
+{
+	struct script_spec *s;
+
+	list_for_each_entry(s, &script_specs, node)
+		if (strcasecmp(s->spec, spec) == 0)
+			return s;
+	return NULL;
+}
+
+static int script_spec_register(const char *spec, struct scripting_ops *ops)
+{
+	struct script_spec *s;
+
+	s = script_spec__find(spec);
+	if (s)
+		return -1;
+
+	s = script_spec__new(spec, ops);
+	if (!s)
+		return -1;
+
+	script_spec__add(s);
+	return 0;
+}
+
+struct scripting_ops *script_spec__lookup(const char *spec)
+{
+	struct script_spec *s = script_spec__find(spec);
+
+	if (!s)
+		return NULL;
+
+	return s->ops;
+}
+
+int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec))
+{
+	struct script_spec *s;
+	int ret = 0;
+
+	list_for_each_entry(s, &script_specs, node) {
+		ret = cb(s->ops, s->spec);
+		if (ret)
+			break;
+	}
+	return ret;
+}
+
 void scripting_context__update(struct scripting_context *c,
 			       union perf_event *event,
 			       struct perf_sample *sample,
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index bbf8b26bc8da..81fceaf297ba 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -116,7 +116,8 @@ struct scripting_ops {
 
 extern unsigned int scripting_max_stack;
 
-int script_spec_register(const char *spec, struct scripting_ops *ops);
+struct scripting_ops *script_spec__lookup(const char *spec);
+int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec));
 
 void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
 		       struct machine *machine);
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 08/21] perf script: Move script_spec code to trace-event-scripting.c

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:39PM -0700, Ian Rogers wrote:
> The script_spec code is referenced in util/trace-event-scripting but
> the list was in builtin-script, accessed via a function that required
> a stub function in python.c. Move all the logic to
> trace-event-scripting, with lookup and foreach functions exposed for
> builtin-script's benefit.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-script.c             | 67 +---------------------
>  tools/perf/util/python.c                |  5 --
>  tools/perf/util/trace-event-scripting.c | 75 +++++++++++++++++++++++++
>  tools/perf/util/trace-event.h           |  3 +-
>  4 files changed, 80 insertions(+), 70 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index 62e851fdf5ca..11c0ee8c1afc 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -2956,79 +2956,18 @@ static int __cmd_script(struct perf_script *script)
>  	return ret;
>  }
>  
> -struct script_spec {
> -	struct list_head	node;
> -	struct scripting_ops	*ops;
> -	char			spec[];
> -};
> -
> -static LIST_HEAD(script_specs);
> -
> -static struct script_spec *script_spec__new(const char *spec,
> -					    struct scripting_ops *ops)
> +static int list_available_languages_cb(struct scripting_ops *ops, const char *spec)
>  {
> -	struct script_spec *s = malloc(sizeof(*s) + strlen(spec) + 1);
> -
> -	if (s != NULL) {
> -		strcpy(s->spec, spec);
> -		s->ops = ops;
> -	}
> -
> -	return s;
> -}
> -
> -static void script_spec__add(struct script_spec *s)
> -{
> -	list_add_tail(&s->node, &script_specs);
> -}
> -
> -static struct script_spec *script_spec__find(const char *spec)
> -{
> -	struct script_spec *s;
> -
> -	list_for_each_entry(s, &script_specs, node)
> -		if (strcasecmp(s->spec, spec) == 0)
> -			return s;
> -	return NULL;
> -}
> -
> -int script_spec_register(const char *spec, struct scripting_ops *ops)
> -{
> -	struct script_spec *s;
> -
> -	s = script_spec__find(spec);
> -	if (s)
> -		return -1;
> -
> -	s = script_spec__new(spec, ops);
> -	if (!s)
> -		return -1;
> -	else
> -		script_spec__add(s);
> -
> +	fprintf(stderr, "  %-42s [%s]\n", spec, ops->name);
>  	return 0;
>  }
>  
> -static struct scripting_ops *script_spec__lookup(const char *spec)
> -{
> -	struct script_spec *s = script_spec__find(spec);
> -	if (!s)
> -		return NULL;
> -
> -	return s->ops;
> -}
> -
>  static void list_available_languages(void)
>  {
> -	struct script_spec *s;
> -
>  	fprintf(stderr, "\n");
>  	fprintf(stderr, "Scripting language extensions (used in "
>  		"perf script -s [spec:]script.[spec]):\n\n");
> -
> -	list_for_each_entry(s, &script_specs, node)
> -		fprintf(stderr, "  %-42s [%s]\n", s->spec, s->ops->name);
> -
> +	script_spec__for_each(&list_available_languages_cb);
>  	fprintf(stderr, "\n");
>  }
>  
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 0fa8e27769be..3d938fe2de6a 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1305,11 +1305,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -int script_spec_register(const char *spec __maybe_unused, struct scripting_ops *ops __maybe_unused)
> -{
> -	return -1;
> -}
> -
>  arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch __maybe_unused)
>  {
>  	return NULL;
> diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
> index 8abb7a7b6888..ad62d8e5a368 100644
> --- a/tools/perf/util/trace-event-scripting.c
> +++ b/tools/perf/util/trace-event-scripting.c
> @@ -24,6 +24,81 @@ unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
>  
>  struct scripting_context *scripting_context;
>  
> +struct script_spec {
> +	struct list_head	node;
> +	struct scripting_ops	*ops;
> +	char			spec[];
> +};
> +
> +static LIST_HEAD(script_specs);
> +
> +static struct script_spec *script_spec__new(const char *spec,
> +					    struct scripting_ops *ops)
> +{
> +	struct script_spec *s = malloc(sizeof(*s) + strlen(spec) + 1);
> +
> +	if (s != NULL) {
> +		strcpy(s->spec, spec);
> +		s->ops = ops;
> +	}
> +
> +	return s;
> +}
> +
> +static void script_spec__add(struct script_spec *s)
> +{
> +	list_add_tail(&s->node, &script_specs);
> +}
> +
> +static struct script_spec *script_spec__find(const char *spec)
> +{
> +	struct script_spec *s;
> +
> +	list_for_each_entry(s, &script_specs, node)
> +		if (strcasecmp(s->spec, spec) == 0)
> +			return s;
> +	return NULL;
> +}
> +
> +static int script_spec_register(const char *spec, struct scripting_ops *ops)
> +{
> +	struct script_spec *s;
> +
> +	s = script_spec__find(spec);
> +	if (s)
> +		return -1;
> +
> +	s = script_spec__new(spec, ops);
> +	if (!s)
> +		return -1;
> +
> +	script_spec__add(s);
> +	return 0;
> +}
> +
> +struct scripting_ops *script_spec__lookup(const char *spec)
> +{
> +	struct script_spec *s = script_spec__find(spec);
> +
> +	if (!s)
> +		return NULL;
> +
> +	return s->ops;
> +}
> +
> +int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec))
> +{
> +	struct script_spec *s;
> +	int ret = 0;
> +
> +	list_for_each_entry(s, &script_specs, node) {
> +		ret = cb(s->ops, s->spec);
> +		if (ret)
> +			break;
> +	}
> +	return ret;
> +}
> +
>  void scripting_context__update(struct scripting_context *c,
>  			       union perf_event *event,
>  			       struct perf_sample *sample,
> diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
> index bbf8b26bc8da..81fceaf297ba 100644
> --- a/tools/perf/util/trace-event.h
> +++ b/tools/perf/util/trace-event.h
> @@ -116,7 +116,8 @@ struct scripting_ops {
>  
>  extern unsigned int scripting_max_stack;
>  
> -int script_spec_register(const char *spec, struct scripting_ops *ops);
> +struct scripting_ops *script_spec__lookup(const char *spec);
> +int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec));
>  
>  void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
>  		       struct machine *machine);
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 09/21] perf script: Move script_fetch_insn to trace-event-scripting.c

[Ian Rogers]

Add native_arch as a parameter to script_fetch_insn rather than
relying on the builtin-script value that won't be initialized for the
dlfilter and python Context use cases. Assume both of those cases are
running natively.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c                       | 15 +--------------
 .../perf/scripts/python/Perf-Trace-Util/Context.c |  2 +-
 tools/perf/util/dlfilter.c                        |  3 ++-
 tools/perf/util/python.c                          |  6 ------
 tools/perf/util/trace-event-scripting.c           | 14 ++++++++++++++
 tools/perf/util/trace-event.h                     |  2 +-
 6 files changed, 19 insertions(+), 23 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 11c0ee8c1afc..22d78a9d8f27 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -1586,19 +1586,6 @@ static int perf_sample__fprintf_callindent(struct perf_sample *sample,
 	return len + dlen;
 }
 
-__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
-			    struct thread *thread __maybe_unused,
-			    struct machine *machine __maybe_unused)
-{
-}
-
-void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
-		       struct machine *machine)
-{
-	if (sample->insn_len == 0 && native_arch)
-		arch_fetch_insn(sample, thread, machine);
-}
-
 static int perf_sample__fprintf_insn(struct perf_sample *sample,
 				     struct evsel *evsel,
 				     struct perf_event_attr *attr,
@@ -1608,7 +1595,7 @@ static int perf_sample__fprintf_insn(struct perf_sample *sample,
 {
 	int printed = 0;
 
-	script_fetch_insn(sample, thread, machine);
+	script_fetch_insn(sample, thread, machine, native_arch);
 
 	if (PRINT_FIELD(INSNLEN))
 		printed += fprintf(fp, " ilen: %d", sample->insn_len);
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
index d742daaa5d5a..60dcfe56d4d9 100644
--- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
+++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
@@ -93,7 +93,7 @@ static PyObject *perf_sample_insn(PyObject *obj, PyObject *args)
 	if (c->sample->ip && !c->sample->insn_len && thread__maps(c->al->thread)) {
 		struct machine *machine =  maps__machine(thread__maps(c->al->thread));
 
-		script_fetch_insn(c->sample, c->al->thread, machine);
+		script_fetch_insn(c->sample, c->al->thread, machine, /*native_arch=*/true);
 	}
 	if (!c->sample->insn_len)
 		Py_RETURN_NONE; /* N.B. This is a return statement */
diff --git a/tools/perf/util/dlfilter.c b/tools/perf/util/dlfilter.c
index 7d180bdaedbc..ddacef881af2 100644
--- a/tools/perf/util/dlfilter.c
+++ b/tools/perf/util/dlfilter.c
@@ -234,7 +234,8 @@ static const __u8 *dlfilter__insn(void *ctx, __u32 *len)
 			struct machine *machine = maps__machine(thread__maps(al->thread));
 
 			if (machine)
-				script_fetch_insn(d->sample, al->thread, machine);
+				script_fetch_insn(d->sample, al->thread, machine,
+						  /*native_arch=*/true);
 		}
 	}
 
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 3d938fe2de6a..22edadd64e5f 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1317,12 +1317,6 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
 	return NULL;
 }
 
-void script_fetch_insn(struct perf_sample *sample __maybe_unused,
-		struct thread *thread __maybe_unused,
-		struct machine *machine __maybe_unused)
-{
-}
-
 int perf_sample__sprintf_flags(u32 flags __maybe_unused, char *str __maybe_unused,
 			size_t sz __maybe_unused)
 {
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index ad62d8e5a368..beac456260ae 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -13,6 +13,7 @@
 #include <traceevent/event-parse.h>
 #endif
 
+#include "archinsn.h"
 #include "debug.h"
 #include "trace-event.h"
 #include "evsel.h"
@@ -269,3 +270,16 @@ void setup_perl_scripting(void)
 }
 #endif
 #endif
+
+__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
+		     struct thread *thread __maybe_unused,
+		     struct machine *machine __maybe_unused)
+{
+}
+
+void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
+		       struct machine *machine, bool native_arch)
+{
+	if (sample->insn_len == 0 && native_arch)
+		arch_fetch_insn(sample, thread, machine);
+}
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index 81fceaf297ba..e0bb42e6509e 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -120,7 +120,7 @@ struct scripting_ops *script_spec__lookup(const char *spec);
 int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec));
 
 void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
-		       struct machine *machine);
+		       struct machine *machine, bool native_arch);
 
 void setup_perl_scripting(void);
 void setup_python_scripting(void);
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 09/21] perf script: Move script_fetch_insn to trace-event-scripting.c

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:40PM -0700, Ian Rogers wrote:
> Add native_arch as a parameter to script_fetch_insn rather than
> relying on the builtin-script value that won't be initialized for the
> dlfilter and python Context use cases. Assume both of those cases are
> running natively.

Adrian, can you take a look? Maybe have some machine__native_arch() that
would use the perf_env struct, uname, etc?

So, as you noticed builtin-script.c does it, sets a global variable

        uname(&uts);
        if (data.is_pipe) { /* Assume pipe_mode indicates native_arch */
                native_arch = true;
        } else if (session->header.env.arch) {
                if (!strcmp(uts.machine, session->header.env.arch))
                        native_arch = true;
                else if (!strcmp(uts.machine, "x86_64") &&
                         !strcmp(session->header.env.arch, "i386"))
                        native_arch = true;
        }

So instead of doing it we should move that global variable to
env->native_arch and then use it?

Your patch doesn't introduce problems and shows this is a limitation, so
no problem with it being merged:

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>

But maybe this would be the time to try to fix this? Adrian,
cross-platform use of dlfilter seems not to be something you would be
interested on, but using a python script to process something from
another arch is interesting, as we advertise cross-platorm perf.data
file analysis as a feature for perf.

- Arnaldo
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-script.c                       | 15 +--------------
>  .../perf/scripts/python/Perf-Trace-Util/Context.c |  2 +-
>  tools/perf/util/dlfilter.c                        |  3 ++-
>  tools/perf/util/python.c                          |  6 ------
>  tools/perf/util/trace-event-scripting.c           | 14 ++++++++++++++
>  tools/perf/util/trace-event.h                     |  2 +-
>  6 files changed, 19 insertions(+), 23 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index 11c0ee8c1afc..22d78a9d8f27 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -1586,19 +1586,6 @@ static int perf_sample__fprintf_callindent(struct perf_sample *sample,
>  	return len + dlen;
>  }
>  
> -__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
> -			    struct thread *thread __maybe_unused,
> -			    struct machine *machine __maybe_unused)
> -{
> -}
> -
> -void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
> -		       struct machine *machine)
> -{
> -	if (sample->insn_len == 0 && native_arch)
> -		arch_fetch_insn(sample, thread, machine);
> -}
> -
>  static int perf_sample__fprintf_insn(struct perf_sample *sample,
>  				     struct evsel *evsel,
>  				     struct perf_event_attr *attr,
> @@ -1608,7 +1595,7 @@ static int perf_sample__fprintf_insn(struct perf_sample *sample,
>  {
>  	int printed = 0;
>  
> -	script_fetch_insn(sample, thread, machine);
> +	script_fetch_insn(sample, thread, machine, native_arch);
>  
>  	if (PRINT_FIELD(INSNLEN))
>  		printed += fprintf(fp, " ilen: %d", sample->insn_len);
> diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> index d742daaa5d5a..60dcfe56d4d9 100644
> --- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> +++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
> @@ -93,7 +93,7 @@ static PyObject *perf_sample_insn(PyObject *obj, PyObject *args)
>  	if (c->sample->ip && !c->sample->insn_len && thread__maps(c->al->thread)) {
>  		struct machine *machine =  maps__machine(thread__maps(c->al->thread));
>  
> -		script_fetch_insn(c->sample, c->al->thread, machine);
> +		script_fetch_insn(c->sample, c->al->thread, machine, /*native_arch=*/true);
>  	}
>  	if (!c->sample->insn_len)
>  		Py_RETURN_NONE; /* N.B. This is a return statement */
> diff --git a/tools/perf/util/dlfilter.c b/tools/perf/util/dlfilter.c
> index 7d180bdaedbc..ddacef881af2 100644
> --- a/tools/perf/util/dlfilter.c
> +++ b/tools/perf/util/dlfilter.c
> @@ -234,7 +234,8 @@ static const __u8 *dlfilter__insn(void *ctx, __u32 *len)
>  			struct machine *machine = maps__machine(thread__maps(al->thread));
>  
>  			if (machine)
> -				script_fetch_insn(d->sample, al->thread, machine);
> +				script_fetch_insn(d->sample, al->thread, machine,
> +						  /*native_arch=*/true);
>  		}
>  	}
>  
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 3d938fe2de6a..22edadd64e5f 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1317,12 +1317,6 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
>  	return NULL;
>  }
>  
> -void script_fetch_insn(struct perf_sample *sample __maybe_unused,
> -		struct thread *thread __maybe_unused,
> -		struct machine *machine __maybe_unused)
> -{
> -}
> -
>  int perf_sample__sprintf_flags(u32 flags __maybe_unused, char *str __maybe_unused,
>  			size_t sz __maybe_unused)
>  {
> diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
> index ad62d8e5a368..beac456260ae 100644
> --- a/tools/perf/util/trace-event-scripting.c
> +++ b/tools/perf/util/trace-event-scripting.c
> @@ -13,6 +13,7 @@
>  #include <traceevent/event-parse.h>
>  #endif
>  
> +#include "archinsn.h"
>  #include "debug.h"
>  #include "trace-event.h"
>  #include "evsel.h"
> @@ -269,3 +270,16 @@ void setup_perl_scripting(void)
>  }
>  #endif
>  #endif
> +
> +__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
> +		     struct thread *thread __maybe_unused,
> +		     struct machine *machine __maybe_unused)
> +{
> +}
> +
> +void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
> +		       struct machine *machine, bool native_arch)
> +{
> +	if (sample->insn_len == 0 && native_arch)
> +		arch_fetch_insn(sample, thread, machine);
> +}
> diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
> index 81fceaf297ba..e0bb42e6509e 100644
> --- a/tools/perf/util/trace-event.h
> +++ b/tools/perf/util/trace-event.h
> @@ -120,7 +120,7 @@ struct scripting_ops *script_spec__lookup(const char *spec);
>  int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec));
>  
>  void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
> -		       struct machine *machine);
> +		       struct machine *machine, bool native_arch);
>  
>  void setup_perl_scripting(void);
>  void setup_python_scripting(void);
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 10/21] perf script: Move perf_sample__sprintf_flags to trace-event-scripting.c

[Ian Rogers]

perf_sample__sprintf_flags is used in the python C code and so needs
to be in the util library rather than a builtin.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-script.c             | 81 ------------------------
 tools/perf/util/python.c                |  6 --
 tools/perf/util/trace-event-scripting.c | 83 +++++++++++++++++++++++++
 3 files changed, 83 insertions(+), 87 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 22d78a9d8f27..1eda065dcb2b 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -1693,87 +1693,6 @@ static int perf_sample__fprintf_bts(struct perf_sample *sample,
 	return printed;
 }
 
-static struct {
-	u32 flags;
-	const char *name;
-} sample_flags[] = {
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL, "call"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN, "return"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CONDITIONAL, "jcc"},
-	{PERF_IP_FLAG_BRANCH, "jmp"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT, "int"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT, "iret"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_SYSCALLRET, "syscall"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_SYSCALLRET, "sysret"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_ASYNC, "async"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_ASYNC |	PERF_IP_FLAG_INTERRUPT, "hw int"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TX_ABORT, "tx abrt"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_BEGIN, "tr strt"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END, "tr end"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMENTRY, "vmentry"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMEXIT, "vmexit"},
-	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_BRANCH_MISS, "br miss"},
-	{0, NULL}
-};
-
-static const char *sample_flags_to_name(u32 flags)
-{
-	int i;
-
-	for (i = 0; sample_flags[i].name ; i++) {
-		if (sample_flags[i].flags == flags)
-			return sample_flags[i].name;
-	}
-
-	return NULL;
-}
-
-int perf_sample__sprintf_flags(u32 flags, char *str, size_t sz)
-{
-	u32 xf = PERF_IP_FLAG_IN_TX | PERF_IP_FLAG_INTR_DISABLE |
-		 PERF_IP_FLAG_INTR_TOGGLE;
-	const char *chars = PERF_IP_FLAG_CHARS;
-	const size_t n = strlen(PERF_IP_FLAG_CHARS);
-	const char *name = NULL;
-	size_t i, pos = 0;
-	char xs[16] = {0};
-
-	if (flags & xf)
-		snprintf(xs, sizeof(xs), "(%s%s%s)",
-			 flags & PERF_IP_FLAG_IN_TX ? "x" : "",
-			 flags & PERF_IP_FLAG_INTR_DISABLE ? "D" : "",
-			 flags & PERF_IP_FLAG_INTR_TOGGLE ? "t" : "");
-
-	name = sample_flags_to_name(flags & ~xf);
-	if (name)
-		return snprintf(str, sz, "%-15s%6s", name, xs);
-
-	if (flags & PERF_IP_FLAG_TRACE_BEGIN) {
-		name = sample_flags_to_name(flags & ~(xf | PERF_IP_FLAG_TRACE_BEGIN));
-		if (name)
-			return snprintf(str, sz, "tr strt %-7s%6s", name, xs);
-	}
-
-	if (flags & PERF_IP_FLAG_TRACE_END) {
-		name = sample_flags_to_name(flags & ~(xf | PERF_IP_FLAG_TRACE_END));
-		if (name)
-			return snprintf(str, sz, "tr end  %-7s%6s", name, xs);
-	}
-
-	for (i = 0; i < n; i++, flags >>= 1) {
-		if ((flags & 1) && pos < sz)
-			str[pos++] = chars[i];
-	}
-	for (; i < 32; i++, flags >>= 1) {
-		if ((flags & 1) && pos < sz)
-			str[pos++] = '?';
-	}
-	if (pos < sz)
-		str[pos] = 0;
-
-	return pos;
-}
-
 static int perf_sample__fprintf_flags(u32 flags, FILE *fp)
 {
 	char str[SAMPLE_FLAGS_BUF_SIZE];
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 22edadd64e5f..77e02fcc51dd 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1317,12 +1317,6 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
 	return NULL;
 }
 
-int perf_sample__sprintf_flags(u32 flags __maybe_unused, char *str __maybe_unused,
-			size_t sz __maybe_unused)
-{
-	return -1;
-}
-
 bool match_callstack_filter(struct machine *machine __maybe_unused, u64 *callstack __maybe_unused)
 {
 	return false;
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index beac456260ae..2e9da0b089ef 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -15,6 +15,7 @@
 
 #include "archinsn.h"
 #include "debug.h"
+#include "event.h"
 #include "trace-event.h"
 #include "evsel.h"
 #include <linux/perf_event.h>
@@ -283,3 +284,85 @@ void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
 	if (sample->insn_len == 0 && native_arch)
 		arch_fetch_insn(sample, thread, machine);
 }
+
+static const struct {
+	u32 flags;
+	const char *name;
+} sample_flags[] = {
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL, "call"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN, "return"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CONDITIONAL, "jcc"},
+	{PERF_IP_FLAG_BRANCH, "jmp"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT, "int"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT, "iret"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_SYSCALLRET, "syscall"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_SYSCALLRET, "sysret"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_ASYNC, "async"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_ASYNC |	PERF_IP_FLAG_INTERRUPT,
+	 "hw int"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TX_ABORT, "tx abrt"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_BEGIN, "tr strt"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END, "tr end"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMENTRY, "vmentry"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMEXIT, "vmexit"},
+	{PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_BRANCH_MISS, "br miss"},
+	{0, NULL}
+};
+
+static const char *sample_flags_to_name(u32 flags)
+{
+	int i;
+
+	for (i = 0; sample_flags[i].name ; i++) {
+		if (sample_flags[i].flags == flags)
+			return sample_flags[i].name;
+	}
+
+	return NULL;
+}
+
+int perf_sample__sprintf_flags(u32 flags, char *str, size_t sz)
+{
+	u32 xf = PERF_IP_FLAG_IN_TX | PERF_IP_FLAG_INTR_DISABLE |
+		 PERF_IP_FLAG_INTR_TOGGLE;
+	const char *chars = PERF_IP_FLAG_CHARS;
+	const size_t n = strlen(PERF_IP_FLAG_CHARS);
+	const char *name = NULL;
+	size_t i, pos = 0;
+	char xs[16] = {0};
+
+	if (flags & xf)
+		snprintf(xs, sizeof(xs), "(%s%s%s)",
+			 flags & PERF_IP_FLAG_IN_TX ? "x" : "",
+			 flags & PERF_IP_FLAG_INTR_DISABLE ? "D" : "",
+			 flags & PERF_IP_FLAG_INTR_TOGGLE ? "t" : "");
+
+	name = sample_flags_to_name(flags & ~xf);
+	if (name)
+		return snprintf(str, sz, "%-15s%6s", name, xs);
+
+	if (flags & PERF_IP_FLAG_TRACE_BEGIN) {
+		name = sample_flags_to_name(flags & ~(xf | PERF_IP_FLAG_TRACE_BEGIN));
+		if (name)
+			return snprintf(str, sz, "tr strt %-7s%6s", name, xs);
+	}
+
+	if (flags & PERF_IP_FLAG_TRACE_END) {
+		name = sample_flags_to_name(flags & ~(xf | PERF_IP_FLAG_TRACE_END));
+		if (name)
+			return snprintf(str, sz, "tr end  %-7s%6s", name, xs);
+	}
+
+	for (i = 0; i < n; i++, flags >>= 1) {
+		if ((flags & 1) && pos < sz)
+			str[pos++] = chars[i];
+	}
+	for (; i < 32; i++, flags >>= 1) {
+		if ((flags & 1) && pos < sz)
+			str[pos++] = '?';
+	}
+	if (pos < sz)
+		str[pos] = 0;
+
+	return pos;
+}
-- 
2.47.0.163.g1226f6d8fa-goog

[PATCH v5 11/21] perf x86: Define arch_fetch_insn in NO_AUXTRACE builds

[Ian Rogers]

archinsn.c containing arch_fetch_insn was only enabled with
CONFIG_AUXTRACE, but this meant that a NO_AUXTRACE build on x86 would
use the empty weak version of arch_fetch_insn - weak symbols are a
frequent source of errors like this and are outside of the C
specification. Change it so that archinsn.c is always built on x86 and
make the weak symbol empty version of arch_fetch_insn a strong one
guarded by ifdefs.

arch_fetch_insn on x86 depends on insn_decode which is a function
included then built into
intel-pt-insn-decoder.c. intel-pt-insn-decoder.c isn't built in a
NO_AUXTRACE=1 build. Separate the insn_decode function from
intel-pt-insn-decoder.c by just directly compiling the relevant
file. Guard this compilation to be for either always on x86 (because
of the use in arch_fetch_insn) or when auxtrace is enabled. Apply the
CFLAGS overrides as necessary, reducing the amount of code where
warnings are disabled.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/arch/x86/util/Build                 |  2 +-
 tools/perf/util/Build                          |  2 +-
 tools/perf/util/intel-pt-decoder/Build         | 18 ++++++++++++++----
 .../intel-pt-decoder/intel-pt-insn-decoder.c   |  3 ---
 tools/perf/util/trace-event-scripting.c        |  4 +++-
 5 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/tools/perf/arch/x86/util/Build b/tools/perf/arch/x86/util/Build
index bc56a8e70f34..c5df4a2cf180 100644
--- a/tools/perf/arch/x86/util/Build
+++ b/tools/perf/arch/x86/util/Build
@@ -18,6 +18,6 @@ perf-util-$(CONFIG_LOCAL_LIBUNWIND)    += unwind-libunwind.o
 perf-util-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
 
 perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
-perf-util-$(CONFIG_AUXTRACE) += archinsn.o
+perf-util-y += archinsn.o
 perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
 perf-util-$(CONFIG_AUXTRACE) += intel-bts.o
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 650974413849..340544a6f5ec 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -122,7 +122,7 @@ perf-util-y += iostat.o
 perf-util-y += stream.o
 perf-util-y += kvm-stat.o
 perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
-perf-util-$(CONFIG_AUXTRACE) += intel-pt-decoder/
+perf-util-y += intel-pt-decoder/
 perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
 perf-util-$(CONFIG_AUXTRACE) += intel-bts.o
 perf-util-$(CONFIG_AUXTRACE) += arm-spe.o
diff --git a/tools/perf/util/intel-pt-decoder/Build b/tools/perf/util/intel-pt-decoder/Build
index 30793d08c6d4..f99d150059b9 100644
--- a/tools/perf/util/intel-pt-decoder/Build
+++ b/tools/perf/util/intel-pt-decoder/Build
@@ -9,14 +9,24 @@ $(OUTPUT)util/intel-pt-decoder/inat-tables.c: $(inat_tables_script) $(inat_table
 
 # Busybox's diff doesn't have -I, avoid warning in the case
 
-$(OUTPUT)util/intel-pt-decoder/intel-pt-insn-decoder.o: util/intel-pt-decoder/intel-pt-insn-decoder.c $(OUTPUT)util/intel-pt-decoder/inat-tables.c
+ifeq ($(SRCARCH),x86)
+  perf-util-y += inat.o insn.o
+else
+  perf-util-$(CONFIG_AUXTRACE) += inat.o insn.o
+endif
+
+$(OUTPUT)util/intel-pt-decoder/inat.o: $(srctree)/tools/arch/x86/lib/inat.c $(OUTPUT)util/intel-pt-decoder/inat-tables.c
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
 
-CFLAGS_intel-pt-insn-decoder.o += -I$(OUTPUT)util/intel-pt-decoder
+CFLAGS_inat.o += -I$(OUTPUT)util/intel-pt-decoder
+
+$(OUTPUT)util/intel-pt-decoder/insn.o: $(srctree)/tools/arch/x86/lib/insn.c
+	$(call rule_mkdir)
+	$(call if_changed_dep,cc_o_c)
 
 ifeq ($(CC_NO_CLANG), 1)
-  CFLAGS_intel-pt-insn-decoder.o += -Wno-override-init
+  CFLAGS_insn.o += -Wno-override-init
 endif
 
-CFLAGS_intel-pt-insn-decoder.o += -Wno-packed
+CFLAGS_insn.o += -Wno-packed
diff --git a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
index 47cf35799a4d..8fabddc1c0da 100644
--- a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
+++ b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
@@ -11,9 +11,6 @@
 #include <byteswap.h>
 #include "../../../arch/x86/include/asm/insn.h"
 
-#include "../../../arch/x86/lib/inat.c"
-#include "../../../arch/x86/lib/insn.c"
-
 #include "event.h"
 
 #include "intel-pt-insn-decoder.h"
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index 2e9da0b089ef..8d71998a9ecd 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -272,11 +272,13 @@ void setup_perl_scripting(void)
 #endif
 #endif
 
-__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
+#if !defined(__i386__) && !defined(__x86_64__)
+void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
 		     struct thread *thread __maybe_unused,
 		     struct machine *machine __maybe_unused)
 {
 }
+#endif
 
 void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
 		       struct machine *machine, bool native_arch)
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 11/21] perf x86: Define arch_fetch_insn in NO_AUXTRACE builds

[Adrian Hunter]

On 31/10/24 03:42, Ian Rogers wrote:
> archinsn.c containing arch_fetch_insn was only enabled with
> CONFIG_AUXTRACE, but this meant that a NO_AUXTRACE build on x86 would
> use the empty weak version of arch_fetch_insn - weak symbols are a
> frequent source of errors like this and are outside of the C
> specification. Change it so that archinsn.c is always built on x86 and
> make the weak symbol empty version of arch_fetch_insn a strong one
> guarded by ifdefs.
> 
> arch_fetch_insn on x86 depends on insn_decode which is a function
> included then built into
> intel-pt-insn-decoder.c. intel-pt-insn-decoder.c isn't built in a
> NO_AUXTRACE=1 build. Separate the insn_decode function from
> intel-pt-insn-decoder.c by just directly compiling the relevant
> file. Guard this compilation to be for either always on x86 (because
> of the use in arch_fetch_insn) or when auxtrace is enabled. Apply the
> CFLAGS overrides as necessary, reducing the amount of code where
> warnings are disabled.
> 
> Signed-off-by: Ian Rogers <irogers@google.com>

For Intel PT

Tested-by: Adrian Hunter <adrian.hunter@intel.com>

> ---
>  tools/perf/arch/x86/util/Build                 |  2 +-
>  tools/perf/util/Build                          |  2 +-
>  tools/perf/util/intel-pt-decoder/Build         | 18 ++++++++++++++----
>  .../intel-pt-decoder/intel-pt-insn-decoder.c   |  3 ---
>  tools/perf/util/trace-event-scripting.c        |  4 +++-
>  5 files changed, 19 insertions(+), 10 deletions(-)
> 
> diff --git a/tools/perf/arch/x86/util/Build b/tools/perf/arch/x86/util/Build
> index bc56a8e70f34..c5df4a2cf180 100644
> --- a/tools/perf/arch/x86/util/Build
> +++ b/tools/perf/arch/x86/util/Build
> @@ -18,6 +18,6 @@ perf-util-$(CONFIG_LOCAL_LIBUNWIND)    += unwind-libunwind.o
>  perf-util-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
>  
>  perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
> -perf-util-$(CONFIG_AUXTRACE) += archinsn.o
> +perf-util-y += archinsn.o
>  perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
>  perf-util-$(CONFIG_AUXTRACE) += intel-bts.o
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 650974413849..340544a6f5ec 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -122,7 +122,7 @@ perf-util-y += iostat.o
>  perf-util-y += stream.o
>  perf-util-y += kvm-stat.o
>  perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
> -perf-util-$(CONFIG_AUXTRACE) += intel-pt-decoder/
> +perf-util-y += intel-pt-decoder/
>  perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
>  perf-util-$(CONFIG_AUXTRACE) += intel-bts.o
>  perf-util-$(CONFIG_AUXTRACE) += arm-spe.o
> diff --git a/tools/perf/util/intel-pt-decoder/Build b/tools/perf/util/intel-pt-decoder/Build
> index 30793d08c6d4..f99d150059b9 100644
> --- a/tools/perf/util/intel-pt-decoder/Build
> +++ b/tools/perf/util/intel-pt-decoder/Build
> @@ -9,14 +9,24 @@ $(OUTPUT)util/intel-pt-decoder/inat-tables.c: $(inat_tables_script) $(inat_table
>  
>  # Busybox's diff doesn't have -I, avoid warning in the case
>  
> -$(OUTPUT)util/intel-pt-decoder/intel-pt-insn-decoder.o: util/intel-pt-decoder/intel-pt-insn-decoder.c $(OUTPUT)util/intel-pt-decoder/inat-tables.c
> +ifeq ($(SRCARCH),x86)
> +  perf-util-y += inat.o insn.o
> +else
> +  perf-util-$(CONFIG_AUXTRACE) += inat.o insn.o
> +endif
> +
> +$(OUTPUT)util/intel-pt-decoder/inat.o: $(srctree)/tools/arch/x86/lib/inat.c $(OUTPUT)util/intel-pt-decoder/inat-tables.c
>  	$(call rule_mkdir)
>  	$(call if_changed_dep,cc_o_c)
>  
> -CFLAGS_intel-pt-insn-decoder.o += -I$(OUTPUT)util/intel-pt-decoder
> +CFLAGS_inat.o += -I$(OUTPUT)util/intel-pt-decoder
> +
> +$(OUTPUT)util/intel-pt-decoder/insn.o: $(srctree)/tools/arch/x86/lib/insn.c
> +	$(call rule_mkdir)
> +	$(call if_changed_dep,cc_o_c)
>  
>  ifeq ($(CC_NO_CLANG), 1)
> -  CFLAGS_intel-pt-insn-decoder.o += -Wno-override-init
> +  CFLAGS_insn.o += -Wno-override-init
>  endif
>  
> -CFLAGS_intel-pt-insn-decoder.o += -Wno-packed
> +CFLAGS_insn.o += -Wno-packed
> diff --git a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
> index 47cf35799a4d..8fabddc1c0da 100644
> --- a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
> +++ b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
> @@ -11,9 +11,6 @@
>  #include <byteswap.h>
>  #include "../../../arch/x86/include/asm/insn.h"
>  
> -#include "../../../arch/x86/lib/inat.c"
> -#include "../../../arch/x86/lib/insn.c"
> -
>  #include "event.h"
>  
>  #include "intel-pt-insn-decoder.h"
> diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
> index 2e9da0b089ef..8d71998a9ecd 100644
> --- a/tools/perf/util/trace-event-scripting.c
> +++ b/tools/perf/util/trace-event-scripting.c
> @@ -272,11 +272,13 @@ void setup_perl_scripting(void)
>  #endif
>  #endif
>  
> -__weak void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
> +#if !defined(__i386__) && !defined(__x86_64__)
> +void arch_fetch_insn(struct perf_sample *sample __maybe_unused,
>  		     struct thread *thread __maybe_unused,
>  		     struct machine *machine __maybe_unused)
>  {
>  }
> +#endif
>  
>  void script_fetch_insn(struct perf_sample *sample, struct thread *thread,
>  		       struct machine *machine, bool native_arch)

[PATCH v5 12/21] perf intel-pt: Remove stale build comment

[Ian Rogers]

Commit 00a263902ac3 ("perf intel-pt: Use shared x86 insn decoder")
removed the use of diff, so remove stale busybox comment.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/intel-pt-decoder/Build | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tools/perf/util/intel-pt-decoder/Build b/tools/perf/util/intel-pt-decoder/Build
index f99d150059b9..5b8f0149167d 100644
--- a/tools/perf/util/intel-pt-decoder/Build
+++ b/tools/perf/util/intel-pt-decoder/Build
@@ -7,8 +7,6 @@ $(OUTPUT)util/intel-pt-decoder/inat-tables.c: $(inat_tables_script) $(inat_table
 	$(call rule_mkdir)
 	@$(call echo-cmd,gen)$(AWK) -f $(inat_tables_script) $(inat_tables_maps) > $@ || rm -f $@
 
-# Busybox's diff doesn't have -I, avoid warning in the case
-
 ifeq ($(SRCARCH),x86)
   perf-util-y += inat.o insn.o
 else
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 12/21] perf intel-pt: Remove stale build comment

[Adrian Hunter]

On 31/10/24 03:42, Ian Rogers wrote:
> Commit 00a263902ac3 ("perf intel-pt: Use shared x86 insn decoder")
> removed the use of diff, so remove stale busybox comment.
> 
> Signed-off-by: Ian Rogers <irogers@google.com>

Acked-by: Adrian Hunter <adrian.hunter@intel.com>

> ---
>  tools/perf/util/intel-pt-decoder/Build | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/tools/perf/util/intel-pt-decoder/Build b/tools/perf/util/intel-pt-decoder/Build
> index f99d150059b9..5b8f0149167d 100644
> --- a/tools/perf/util/intel-pt-decoder/Build
> +++ b/tools/perf/util/intel-pt-decoder/Build
> @@ -7,8 +7,6 @@ $(OUTPUT)util/intel-pt-decoder/inat-tables.c: $(inat_tables_script) $(inat_table
>  	$(call rule_mkdir)
>  	@$(call echo-cmd,gen)$(AWK) -f $(inat_tables_script) $(inat_tables_maps) > $@ || rm -f $@
>  
> -# Busybox's diff doesn't have -I, avoid warning in the case
> -
>  ifeq ($(SRCARCH),x86)
>    perf-util-y += inat.o insn.o
>  else

[PATCH v5 13/21] perf env: Move arch errno function to only use in env

[Ian Rogers]

Move arch_syscalls__strerrno_function out of builtin-trace.c to env.c
so that there isn't a util to builtin function call. This allows the
python.c stub to be removed. Also, remove declaration/prototype from
env.h and make static to reduce scope. The include is moved inside
ifdefs to avoid, "defined but unused warnings".

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-trace.c                  | 1 -
 tools/perf/trace/beauty/arch_errno_names.sh | 3 ++-
 tools/perf/util/env.c                       | 4 ++++
 tools/perf/util/env.h                       | 2 --
 tools/perf/util/python.c                    | 6 ------
 5 files changed, 6 insertions(+), 10 deletions(-)

diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 748b061f8678..5d0345480140 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -1108,7 +1108,6 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _
 	    .strtoul	= STUL_STRARRAY_FLAGS, \
 	    .parm	= &strarray__##array, }
 
-#include "trace/beauty/arch_errno_names.c"
 #include "trace/beauty/eventfd.c"
 #include "trace/beauty/futex_op.c"
 #include "trace/beauty/futex_val3.c"
diff --git a/tools/perf/trace/beauty/arch_errno_names.sh b/tools/perf/trace/beauty/arch_errno_names.sh
index 30d3889b2957..b22890b8d272 100755
--- a/tools/perf/trace/beauty/arch_errno_names.sh
+++ b/tools/perf/trace/beauty/arch_errno_names.sh
@@ -57,7 +57,8 @@ create_arch_errno_table_func()
 	archlist="$1"
 	default="$2"
 
-	printf 'arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch)\n'
+	printf 'static arch_syscalls__strerrno_t *\n'
+	printf 'arch_syscalls__strerrno_function(const char *arch)\n'
 	printf '{\n'
 	for arch in $archlist; do
 		arch_str=$(arch_string "$arch")
diff --git a/tools/perf/util/env.c b/tools/perf/util/env.c
index ccb464026642..e58f56412fbb 100644
--- a/tools/perf/util/env.c
+++ b/tools/perf/util/env.c
@@ -470,6 +470,10 @@ const char *perf_env__arch(struct perf_env *env)
 	return normalize_arch(arch_name);
 }
 
+#if defined(HAVE_SYSCALL_TABLE_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
+#include "trace/beauty/arch_errno_names.c"
+#endif
+
 const char *perf_env__arch_strerrno(struct perf_env *env __maybe_unused, int err __maybe_unused)
 {
 #if defined(HAVE_SYSCALL_TABLE_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
diff --git a/tools/perf/util/env.h b/tools/perf/util/env.h
index ae604c4edbb7..e9db89395dab 100644
--- a/tools/perf/util/env.h
+++ b/tools/perf/util/env.h
@@ -56,8 +56,6 @@ struct pmu_caps {
 
 typedef const char *(arch_syscalls__strerrno_t)(int err);
 
-arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch);
-
 struct perf_env {
 	char			*hostname;
 	char			*os_release;
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 77e02fcc51dd..35d84a96dbec 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -16,7 +16,6 @@
 #include "thread_map.h"
 #include "trace-event.h"
 #include "mmap.h"
-#include "util/env.h"
 #include "util/kwork.h"
 #include "util/sample.h"
 #include "util/lock-contention.h"
@@ -1305,11 +1304,6 @@ PyMODINIT_FUNC PyInit_perf(void)
 /* The following are stubs to avoid dragging in builtin-* objects. */
 /* TODO: move the code out of the builtin-* file into util. */
 
-arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch __maybe_unused)
-{
-	return NULL;
-}
-
 struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
 				       struct kwork_class *class __maybe_unused,
 				       struct kwork_work *key  __maybe_unused)
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 13/21] perf env: Move arch errno function to only use in env

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:44PM -0700, Ian Rogers wrote:
> Move arch_syscalls__strerrno_function out of builtin-trace.c to env.c
> so that there isn't a util to builtin function call. This allows the
> python.c stub to be removed. Also, remove declaration/prototype from
> env.h and make static to reduce scope. The include is moved inside
> ifdefs to avoid, "defined but unused warnings".

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-trace.c                  | 1 -
>  tools/perf/trace/beauty/arch_errno_names.sh | 3 ++-
>  tools/perf/util/env.c                       | 4 ++++
>  tools/perf/util/env.h                       | 2 --
>  tools/perf/util/python.c                    | 6 ------
>  5 files changed, 6 insertions(+), 10 deletions(-)
> 
> diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
> index 748b061f8678..5d0345480140 100644
> --- a/tools/perf/builtin-trace.c
> +++ b/tools/perf/builtin-trace.c
> @@ -1108,7 +1108,6 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _
>  	    .strtoul	= STUL_STRARRAY_FLAGS, \
>  	    .parm	= &strarray__##array, }
>  
> -#include "trace/beauty/arch_errno_names.c"
>  #include "trace/beauty/eventfd.c"
>  #include "trace/beauty/futex_op.c"
>  #include "trace/beauty/futex_val3.c"
> diff --git a/tools/perf/trace/beauty/arch_errno_names.sh b/tools/perf/trace/beauty/arch_errno_names.sh
> index 30d3889b2957..b22890b8d272 100755
> --- a/tools/perf/trace/beauty/arch_errno_names.sh
> +++ b/tools/perf/trace/beauty/arch_errno_names.sh
> @@ -57,7 +57,8 @@ create_arch_errno_table_func()
>  	archlist="$1"
>  	default="$2"
>  
> -	printf 'arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch)\n'
> +	printf 'static arch_syscalls__strerrno_t *\n'
> +	printf 'arch_syscalls__strerrno_function(const char *arch)\n'
>  	printf '{\n'
>  	for arch in $archlist; do
>  		arch_str=$(arch_string "$arch")
> diff --git a/tools/perf/util/env.c b/tools/perf/util/env.c
> index ccb464026642..e58f56412fbb 100644
> --- a/tools/perf/util/env.c
> +++ b/tools/perf/util/env.c
> @@ -470,6 +470,10 @@ const char *perf_env__arch(struct perf_env *env)
>  	return normalize_arch(arch_name);
>  }
>  
> +#if defined(HAVE_SYSCALL_TABLE_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
> +#include "trace/beauty/arch_errno_names.c"
> +#endif
> +
>  const char *perf_env__arch_strerrno(struct perf_env *env __maybe_unused, int err __maybe_unused)
>  {
>  #if defined(HAVE_SYSCALL_TABLE_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
> diff --git a/tools/perf/util/env.h b/tools/perf/util/env.h
> index ae604c4edbb7..e9db89395dab 100644
> --- a/tools/perf/util/env.h
> +++ b/tools/perf/util/env.h
> @@ -56,8 +56,6 @@ struct pmu_caps {
>  
>  typedef const char *(arch_syscalls__strerrno_t)(int err);
>  
> -arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch);
> -
>  struct perf_env {
>  	char			*hostname;
>  	char			*os_release;
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 77e02fcc51dd..35d84a96dbec 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -16,7 +16,6 @@
>  #include "thread_map.h"
>  #include "trace-event.h"
>  #include "mmap.h"
> -#include "util/env.h"
>  #include "util/kwork.h"
>  #include "util/sample.h"
>  #include "util/lock-contention.h"
> @@ -1305,11 +1304,6 @@ PyMODINIT_FUNC PyInit_perf(void)
>  /* The following are stubs to avoid dragging in builtin-* objects. */
>  /* TODO: move the code out of the builtin-* file into util. */
>  
> -arch_syscalls__strerrno_t *arch_syscalls__strerrno_function(const char *arch __maybe_unused)
> -{
> -	return NULL;
> -}
> -
>  struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
>  				       struct kwork_class *class __maybe_unused,
>  				       struct kwork_work *key  __maybe_unused)
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 14/21] perf lock: Move common lock contention code to new file

[Ian Rogers]

Avoid references from util code to builtin-lock that require python
stubs. Move the functions and related variables to
util/lock-contention.c. Add max_stack_depth parameter to
match_callstack_filter to avoid sharing a global variable.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-lock.c             | 137 +--------------------
 tools/perf/util/Build                 |   1 +
 tools/perf/util/bpf_lock_contention.c |   2 +-
 tools/perf/util/lock-contention.c     | 170 ++++++++++++++++++++++++++
 tools/perf/util/lock-contention.h     |  37 ++----
 tools/perf/util/python.c              |  17 ---
 6 files changed, 185 insertions(+), 179 deletions(-)
 create mode 100644 tools/perf/util/lock-contention.c

diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c
index 062e2b56a2ab..f66948b1fbed 100644
--- a/tools/perf/builtin-lock.c
+++ b/tools/perf/builtin-lock.c
@@ -46,15 +46,6 @@
 static struct perf_session *session;
 static struct target target;
 
-/* based on kernel/lockdep.c */
-#define LOCKHASH_BITS		12
-#define LOCKHASH_SIZE		(1UL << LOCKHASH_BITS)
-
-static struct hlist_head *lockhash_table;
-
-#define __lockhashfn(key)	hash_long((unsigned long)key, LOCKHASH_BITS)
-#define lockhashentry(key)	(lockhash_table + __lockhashfn((key)))
-
 static struct rb_root		thread_stats;
 
 static bool combine_locks;
@@ -67,24 +58,13 @@ static unsigned long bpf_map_entries = MAX_ENTRIES;
 static int max_stack_depth = CONTENTION_STACK_DEPTH;
 static int stack_skip = CONTENTION_STACK_SKIP;
 static int print_nr_entries = INT_MAX / 2;
-static LIST_HEAD(callstack_filters);
 static const char *output_name = NULL;
 static FILE *lock_output;
 
-struct callstack_filter {
-	struct list_head list;
-	char name[];
-};
-
 static struct lock_filter filters;
 
 static enum lock_aggr_mode aggr_mode = LOCK_AGGR_ADDR;
 
-static bool needs_callstack(void)
-{
-	return !list_empty(&callstack_filters);
-}
-
 static struct thread_stat *thread_stat_find(u32 tid)
 {
 	struct rb_node *node;
@@ -477,93 +457,6 @@ static struct lock_stat *pop_from_result(void)
 	return container_of(node, struct lock_stat, rb);
 }
 
-struct lock_stat *lock_stat_find(u64 addr)
-{
-	struct hlist_head *entry = lockhashentry(addr);
-	struct lock_stat *ret;
-
-	hlist_for_each_entry(ret, entry, hash_entry) {
-		if (ret->addr == addr)
-			return ret;
-	}
-	return NULL;
-}
-
-struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags)
-{
-	struct hlist_head *entry = lockhashentry(addr);
-	struct lock_stat *ret, *new;
-
-	hlist_for_each_entry(ret, entry, hash_entry) {
-		if (ret->addr == addr)
-			return ret;
-	}
-
-	new = zalloc(sizeof(struct lock_stat));
-	if (!new)
-		goto alloc_failed;
-
-	new->addr = addr;
-	new->name = strdup(name);
-	if (!new->name) {
-		free(new);
-		goto alloc_failed;
-	}
-
-	new->flags = flags;
-	new->wait_time_min = ULLONG_MAX;
-
-	hlist_add_head(&new->hash_entry, entry);
-	return new;
-
-alloc_failed:
-	pr_err("memory allocation failed\n");
-	return NULL;
-}
-
-bool match_callstack_filter(struct machine *machine, u64 *callstack)
-{
-	struct map *kmap;
-	struct symbol *sym;
-	u64 ip;
-	const char *arch = perf_env__arch(machine->env);
-
-	if (list_empty(&callstack_filters))
-		return true;
-
-	for (int i = 0; i < max_stack_depth; i++) {
-		struct callstack_filter *filter;
-
-		/*
-		 * In powerpc, the callchain saved by kernel always includes
-		 * first three entries as the NIP (next instruction pointer),
-		 * LR (link register), and the contents of LR save area in the
-		 * second stack frame. In certain scenarios its possible to have
-		 * invalid kernel instruction addresses in either LR or the second
-		 * stack frame's LR. In that case, kernel will store that address as
-		 * zero.
-		 *
-		 * The below check will continue to look into callstack,
-		 * incase first or second callstack index entry has 0
-		 * address for powerpc.
-		 */
-		if (!callstack || (!callstack[i] && (strcmp(arch, "powerpc") ||
-						(i != 1 && i != 2))))
-			break;
-
-		ip = callstack[i];
-		sym = machine__find_kernel_symbol(machine, ip, &kmap);
-		if (sym == NULL)
-			continue;
-
-		list_for_each_entry(filter, &callstack_filters, list) {
-			if (strstr(sym->name, filter->name))
-				return true;
-		}
-	}
-	return false;
-}
-
 struct trace_lock_handler {
 	/* it's used on CONFIG_LOCKDEP */
 	int (*acquire_event)(struct evsel *evsel,
@@ -1165,7 +1058,7 @@ static int report_lock_contention_begin_event(struct evsel *evsel,
 		if (callstack == NULL)
 			return -ENOMEM;
 
-		if (!match_callstack_filter(machine, callstack)) {
+		if (!match_callstack_filter(machine, callstack, max_stack_depth)) {
 			free(callstack);
 			return 0;
 		}
@@ -2449,34 +2342,6 @@ static int parse_lock_addr(const struct option *opt __maybe_unused, const char *
 	return ret;
 }
 
-static int parse_call_stack(const struct option *opt __maybe_unused, const char *str,
-			   int unset __maybe_unused)
-{
-	char *s, *tmp, *tok;
-	int ret = 0;
-
-	s = strdup(str);
-	if (s == NULL)
-		return -1;
-
-	for (tok = strtok_r(s, ", ", &tmp); tok; tok = strtok_r(NULL, ", ", &tmp)) {
-		struct callstack_filter *entry;
-
-		entry = malloc(sizeof(*entry) + strlen(tok) + 1);
-		if (entry == NULL) {
-			pr_err("Memory allocation failure\n");
-			free(s);
-			return -1;
-		}
-
-		strcpy(entry->name, tok);
-		list_add_tail(&entry->list, &callstack_filters);
-	}
-
-	free(s);
-	return ret;
-}
-
 static int parse_output(const struct option *opt __maybe_unused, const char *str,
 			int unset __maybe_unused)
 {
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 340544a6f5ec..3c6cd8d81d88 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -121,6 +121,7 @@ perf-util-y += topdown.o
 perf-util-y += iostat.o
 perf-util-y += stream.o
 perf-util-y += kvm-stat.o
+perf-util-y += lock-contention.o
 perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
 perf-util-y += intel-pt-decoder/
 perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c
index 41a1ad087895..37e17c56f106 100644
--- a/tools/perf/util/bpf_lock_contention.c
+++ b/tools/perf/util/bpf_lock_contention.c
@@ -458,7 +458,7 @@ int lock_contention_read(struct lock_contention *con)
 		if (con->save_callstack) {
 			bpf_map_lookup_elem(stack, &key.stack_id, stack_trace);
 
-			if (!match_callstack_filter(machine, stack_trace)) {
+			if (!match_callstack_filter(machine, stack_trace, con->max_stack)) {
 				con->nr_filtered += data.count;
 				goto next;
 			}
diff --git a/tools/perf/util/lock-contention.c b/tools/perf/util/lock-contention.c
new file mode 100644
index 000000000000..841bb18b1f06
--- /dev/null
+++ b/tools/perf/util/lock-contention.c
@@ -0,0 +1,170 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "debug.h"
+#include "env.h"
+#include "lock-contention.h"
+#include "machine.h"
+#include "symbol.h"
+
+#include <limits.h>
+#include <string.h>
+
+#include <linux/hash.h>
+#include <linux/zalloc.h>
+
+#define __lockhashfn(key)	hash_long((unsigned long)key, LOCKHASH_BITS)
+#define lockhashentry(key)	(lockhash_table + __lockhashfn((key)))
+
+struct callstack_filter {
+	struct list_head list;
+	char name[];
+};
+
+static LIST_HEAD(callstack_filters);
+struct hlist_head *lockhash_table;
+
+int parse_call_stack(const struct option *opt __maybe_unused, const char *str,
+		     int unset __maybe_unused)
+{
+	char *s, *tmp, *tok;
+	int ret = 0;
+
+	s = strdup(str);
+	if (s == NULL)
+		return -1;
+
+	for (tok = strtok_r(s, ", ", &tmp); tok; tok = strtok_r(NULL, ", ", &tmp)) {
+		struct callstack_filter *entry;
+
+		entry = malloc(sizeof(*entry) + strlen(tok) + 1);
+		if (entry == NULL) {
+			pr_err("Memory allocation failure\n");
+			free(s);
+			return -1;
+		}
+
+		strcpy(entry->name, tok);
+		list_add_tail(&entry->list, &callstack_filters);
+	}
+
+	free(s);
+	return ret;
+}
+
+bool needs_callstack(void)
+{
+	return !list_empty(&callstack_filters);
+}
+
+struct lock_stat *lock_stat_find(u64 addr)
+{
+	struct hlist_head *entry = lockhashentry(addr);
+	struct lock_stat *ret;
+
+	hlist_for_each_entry(ret, entry, hash_entry) {
+		if (ret->addr == addr)
+			return ret;
+	}
+	return NULL;
+}
+
+struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags)
+{
+	struct hlist_head *entry = lockhashentry(addr);
+	struct lock_stat *ret, *new;
+
+	hlist_for_each_entry(ret, entry, hash_entry) {
+		if (ret->addr == addr)
+			return ret;
+	}
+
+	new = zalloc(sizeof(struct lock_stat));
+	if (!new)
+		goto alloc_failed;
+
+	new->addr = addr;
+	new->name = strdup(name);
+	if (!new->name) {
+		free(new);
+		goto alloc_failed;
+	}
+
+	new->flags = flags;
+	new->wait_time_min = ULLONG_MAX;
+
+	hlist_add_head(&new->hash_entry, entry);
+	return new;
+
+alloc_failed:
+	pr_err("memory allocation failed\n");
+	return NULL;
+}
+
+bool match_callstack_filter(struct machine *machine, u64 *callstack, int max_stack_depth)
+{
+	struct map *kmap;
+	struct symbol *sym;
+	u64 ip;
+	const char *arch = perf_env__arch(machine->env);
+
+	if (list_empty(&callstack_filters))
+		return true;
+
+	for (int i = 0; i < max_stack_depth; i++) {
+		struct callstack_filter *filter;
+
+		/*
+		 * In powerpc, the callchain saved by kernel always includes
+		 * first three entries as the NIP (next instruction pointer),
+		 * LR (link register), and the contents of LR save area in the
+		 * second stack frame. In certain scenarios its possible to have
+		 * invalid kernel instruction addresses in either LR or the second
+		 * stack frame's LR. In that case, kernel will store that address as
+		 * zero.
+		 *
+		 * The below check will continue to look into callstack,
+		 * incase first or second callstack index entry has 0
+		 * address for powerpc.
+		 */
+		if (!callstack || (!callstack[i] && (strcmp(arch, "powerpc") ||
+						(i != 1 && i != 2))))
+			break;
+
+		ip = callstack[i];
+		sym = machine__find_kernel_symbol(machine, ip, &kmap);
+		if (sym == NULL)
+			continue;
+
+		list_for_each_entry(filter, &callstack_filters, list) {
+			if (strstr(sym->name, filter->name))
+				return true;
+		}
+	}
+	return false;
+}
+
+#ifndef HAVE_BPF_SKEL
+int lock_contention_prepare(struct lock_contention *con __maybe_unused)
+{
+	return 0;
+}
+
+int lock_contention_start(void)
+{
+	return 0;
+}
+
+int lock_contention_stop(void)
+{
+	return 0;
+}
+
+int lock_contention_finish(struct lock_contention *con __maybe_unused)
+{
+	return 0;
+}
+
+int lock_contention_read(struct lock_contention *con __maybe_unused)
+{
+	return 0;
+}
+#endif  /* !HAVE_BPF_SKEL */
diff --git a/tools/perf/util/lock-contention.h b/tools/perf/util/lock-contention.h
index 1a7248ff3889..bfa5c7db0a5d 100644
--- a/tools/perf/util/lock-contention.h
+++ b/tools/perf/util/lock-contention.h
@@ -67,10 +67,11 @@ struct lock_stat {
  */
 #define MAX_LOCK_DEPTH 48
 
-struct lock_stat *lock_stat_find(u64 addr);
-struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags);
+/* based on kernel/lockdep.c */
+#define LOCKHASH_BITS		12
+#define LOCKHASH_SIZE		(1UL << LOCKHASH_BITS)
 
-bool match_callstack_filter(struct machine *machine, u64 *callstack);
+extern struct hlist_head *lockhash_table;
 
 /*
  * struct lock_seq_stat:
@@ -148,7 +149,14 @@ struct lock_contention {
 	bool save_callstack;
 };
 
-#ifdef HAVE_BPF_SKEL
+struct option;
+int parse_call_stack(const struct option *opt, const char *str, int unset);
+bool needs_callstack(void);
+
+struct lock_stat *lock_stat_find(u64 addr);
+struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags);
+
+bool match_callstack_filter(struct machine *machine, u64 *callstack, int max_stack_depth);
 
 int lock_contention_prepare(struct lock_contention *con);
 int lock_contention_start(void);
@@ -156,25 +164,4 @@ int lock_contention_stop(void);
 int lock_contention_read(struct lock_contention *con);
 int lock_contention_finish(struct lock_contention *con);
 
-#else  /* !HAVE_BPF_SKEL */
-
-static inline int lock_contention_prepare(struct lock_contention *con __maybe_unused)
-{
-	return 0;
-}
-
-static inline int lock_contention_start(void) { return 0; }
-static inline int lock_contention_stop(void) { return 0; }
-static inline int lock_contention_finish(struct lock_contention *con __maybe_unused)
-{
-	return 0;
-}
-
-static inline int lock_contention_read(struct lock_contention *con __maybe_unused)
-{
-	return 0;
-}
-
-#endif  /* HAVE_BPF_SKEL */
-
 #endif  /* PERF_LOCK_CONTENTION_H */
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 35d84a96dbec..91fd444615cd 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -18,7 +18,6 @@
 #include "mmap.h"
 #include "util/kwork.h"
 #include "util/sample.h"
-#include "util/lock-contention.h"
 #include <internal/lib.h>
 #include "../builtin.h"
 
@@ -1311,22 +1310,6 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
 	return NULL;
 }
 
-bool match_callstack_filter(struct machine *machine __maybe_unused, u64 *callstack __maybe_unused)
-{
-	return false;
-}
-
-struct lock_stat *lock_stat_find(u64 addr __maybe_unused)
-{
-	return NULL;
-}
-
-struct lock_stat *lock_stat_findnew(u64 addr __maybe_unused, const char *name __maybe_unused,
-				int flags __maybe_unused)
-{
-	return NULL;
-}
-
 int cmd_inject(int argc __maybe_unused, const char *argv[] __maybe_unused)
 {
 	return -1;
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 14/21] perf lock: Move common lock contention code to new file

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:45PM -0700, Ian Rogers wrote:
> Avoid references from util code to builtin-lock that require python
> stubs. Move the functions and related variables to
> util/lock-contention.c. Add max_stack_depth parameter to
> match_callstack_filter to avoid sharing a global variable.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/builtin-lock.c             | 137 +--------------------
>  tools/perf/util/Build                 |   1 +
>  tools/perf/util/bpf_lock_contention.c |   2 +-
>  tools/perf/util/lock-contention.c     | 170 ++++++++++++++++++++++++++
>  tools/perf/util/lock-contention.h     |  37 ++----
>  tools/perf/util/python.c              |  17 ---
>  6 files changed, 185 insertions(+), 179 deletions(-)
>  create mode 100644 tools/perf/util/lock-contention.c
> 
> diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c
> index 062e2b56a2ab..f66948b1fbed 100644
> --- a/tools/perf/builtin-lock.c
> +++ b/tools/perf/builtin-lock.c
> @@ -46,15 +46,6 @@
>  static struct perf_session *session;
>  static struct target target;
>  
> -/* based on kernel/lockdep.c */
> -#define LOCKHASH_BITS		12
> -#define LOCKHASH_SIZE		(1UL << LOCKHASH_BITS)
> -
> -static struct hlist_head *lockhash_table;
> -
> -#define __lockhashfn(key)	hash_long((unsigned long)key, LOCKHASH_BITS)
> -#define lockhashentry(key)	(lockhash_table + __lockhashfn((key)))
> -
>  static struct rb_root		thread_stats;
>  
>  static bool combine_locks;
> @@ -67,24 +58,13 @@ static unsigned long bpf_map_entries = MAX_ENTRIES;
>  static int max_stack_depth = CONTENTION_STACK_DEPTH;
>  static int stack_skip = CONTENTION_STACK_SKIP;
>  static int print_nr_entries = INT_MAX / 2;
> -static LIST_HEAD(callstack_filters);
>  static const char *output_name = NULL;
>  static FILE *lock_output;
>  
> -struct callstack_filter {
> -	struct list_head list;
> -	char name[];
> -};
> -
>  static struct lock_filter filters;
>  
>  static enum lock_aggr_mode aggr_mode = LOCK_AGGR_ADDR;
>  
> -static bool needs_callstack(void)
> -{
> -	return !list_empty(&callstack_filters);
> -}
> -
>  static struct thread_stat *thread_stat_find(u32 tid)
>  {
>  	struct rb_node *node;
> @@ -477,93 +457,6 @@ static struct lock_stat *pop_from_result(void)
>  	return container_of(node, struct lock_stat, rb);
>  }
>  
> -struct lock_stat *lock_stat_find(u64 addr)
> -{
> -	struct hlist_head *entry = lockhashentry(addr);
> -	struct lock_stat *ret;
> -
> -	hlist_for_each_entry(ret, entry, hash_entry) {
> -		if (ret->addr == addr)
> -			return ret;
> -	}
> -	return NULL;
> -}
> -
> -struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags)
> -{
> -	struct hlist_head *entry = lockhashentry(addr);
> -	struct lock_stat *ret, *new;
> -
> -	hlist_for_each_entry(ret, entry, hash_entry) {
> -		if (ret->addr == addr)
> -			return ret;
> -	}
> -
> -	new = zalloc(sizeof(struct lock_stat));
> -	if (!new)
> -		goto alloc_failed;
> -
> -	new->addr = addr;
> -	new->name = strdup(name);
> -	if (!new->name) {
> -		free(new);
> -		goto alloc_failed;
> -	}
> -
> -	new->flags = flags;
> -	new->wait_time_min = ULLONG_MAX;
> -
> -	hlist_add_head(&new->hash_entry, entry);
> -	return new;
> -
> -alloc_failed:
> -	pr_err("memory allocation failed\n");
> -	return NULL;
> -}
> -
> -bool match_callstack_filter(struct machine *machine, u64 *callstack)
> -{
> -	struct map *kmap;
> -	struct symbol *sym;
> -	u64 ip;
> -	const char *arch = perf_env__arch(machine->env);
> -
> -	if (list_empty(&callstack_filters))
> -		return true;
> -
> -	for (int i = 0; i < max_stack_depth; i++) {
> -		struct callstack_filter *filter;
> -
> -		/*
> -		 * In powerpc, the callchain saved by kernel always includes
> -		 * first three entries as the NIP (next instruction pointer),
> -		 * LR (link register), and the contents of LR save area in the
> -		 * second stack frame. In certain scenarios its possible to have
> -		 * invalid kernel instruction addresses in either LR or the second
> -		 * stack frame's LR. In that case, kernel will store that address as
> -		 * zero.
> -		 *
> -		 * The below check will continue to look into callstack,
> -		 * incase first or second callstack index entry has 0
> -		 * address for powerpc.
> -		 */
> -		if (!callstack || (!callstack[i] && (strcmp(arch, "powerpc") ||
> -						(i != 1 && i != 2))))
> -			break;
> -
> -		ip = callstack[i];
> -		sym = machine__find_kernel_symbol(machine, ip, &kmap);
> -		if (sym == NULL)
> -			continue;
> -
> -		list_for_each_entry(filter, &callstack_filters, list) {
> -			if (strstr(sym->name, filter->name))
> -				return true;
> -		}
> -	}
> -	return false;
> -}
> -
>  struct trace_lock_handler {
>  	/* it's used on CONFIG_LOCKDEP */
>  	int (*acquire_event)(struct evsel *evsel,
> @@ -1165,7 +1058,7 @@ static int report_lock_contention_begin_event(struct evsel *evsel,
>  		if (callstack == NULL)
>  			return -ENOMEM;
>  
> -		if (!match_callstack_filter(machine, callstack)) {
> +		if (!match_callstack_filter(machine, callstack, max_stack_depth)) {
>  			free(callstack);
>  			return 0;
>  		}
> @@ -2449,34 +2342,6 @@ static int parse_lock_addr(const struct option *opt __maybe_unused, const char *
>  	return ret;
>  }
>  
> -static int parse_call_stack(const struct option *opt __maybe_unused, const char *str,
> -			   int unset __maybe_unused)
> -{
> -	char *s, *tmp, *tok;
> -	int ret = 0;
> -
> -	s = strdup(str);
> -	if (s == NULL)
> -		return -1;
> -
> -	for (tok = strtok_r(s, ", ", &tmp); tok; tok = strtok_r(NULL, ", ", &tmp)) {
> -		struct callstack_filter *entry;
> -
> -		entry = malloc(sizeof(*entry) + strlen(tok) + 1);
> -		if (entry == NULL) {
> -			pr_err("Memory allocation failure\n");
> -			free(s);
> -			return -1;
> -		}
> -
> -		strcpy(entry->name, tok);
> -		list_add_tail(&entry->list, &callstack_filters);
> -	}
> -
> -	free(s);
> -	return ret;
> -}
> -
>  static int parse_output(const struct option *opt __maybe_unused, const char *str,
>  			int unset __maybe_unused)
>  {
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 340544a6f5ec..3c6cd8d81d88 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -121,6 +121,7 @@ perf-util-y += topdown.o
>  perf-util-y += iostat.o
>  perf-util-y += stream.o
>  perf-util-y += kvm-stat.o
> +perf-util-y += lock-contention.o
>  perf-util-$(CONFIG_AUXTRACE) += auxtrace.o
>  perf-util-y += intel-pt-decoder/
>  perf-util-$(CONFIG_AUXTRACE) += intel-pt.o
> diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c
> index 41a1ad087895..37e17c56f106 100644
> --- a/tools/perf/util/bpf_lock_contention.c
> +++ b/tools/perf/util/bpf_lock_contention.c
> @@ -458,7 +458,7 @@ int lock_contention_read(struct lock_contention *con)
>  		if (con->save_callstack) {
>  			bpf_map_lookup_elem(stack, &key.stack_id, stack_trace);
>  
> -			if (!match_callstack_filter(machine, stack_trace)) {
> +			if (!match_callstack_filter(machine, stack_trace, con->max_stack)) {
>  				con->nr_filtered += data.count;
>  				goto next;
>  			}
> diff --git a/tools/perf/util/lock-contention.c b/tools/perf/util/lock-contention.c
> new file mode 100644
> index 000000000000..841bb18b1f06
> --- /dev/null
> +++ b/tools/perf/util/lock-contention.c
> @@ -0,0 +1,170 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include "debug.h"
> +#include "env.h"
> +#include "lock-contention.h"
> +#include "machine.h"
> +#include "symbol.h"
> +
> +#include <limits.h>
> +#include <string.h>
> +
> +#include <linux/hash.h>
> +#include <linux/zalloc.h>
> +
> +#define __lockhashfn(key)	hash_long((unsigned long)key, LOCKHASH_BITS)
> +#define lockhashentry(key)	(lockhash_table + __lockhashfn((key)))
> +
> +struct callstack_filter {
> +	struct list_head list;
> +	char name[];
> +};
> +
> +static LIST_HEAD(callstack_filters);
> +struct hlist_head *lockhash_table;
> +
> +int parse_call_stack(const struct option *opt __maybe_unused, const char *str,
> +		     int unset __maybe_unused)
> +{
> +	char *s, *tmp, *tok;
> +	int ret = 0;
> +
> +	s = strdup(str);
> +	if (s == NULL)
> +		return -1;
> +
> +	for (tok = strtok_r(s, ", ", &tmp); tok; tok = strtok_r(NULL, ", ", &tmp)) {
> +		struct callstack_filter *entry;
> +
> +		entry = malloc(sizeof(*entry) + strlen(tok) + 1);
> +		if (entry == NULL) {
> +			pr_err("Memory allocation failure\n");
> +			free(s);
> +			return -1;
> +		}
> +
> +		strcpy(entry->name, tok);
> +		list_add_tail(&entry->list, &callstack_filters);
> +	}
> +
> +	free(s);
> +	return ret;
> +}
> +
> +bool needs_callstack(void)
> +{
> +	return !list_empty(&callstack_filters);
> +}
> +
> +struct lock_stat *lock_stat_find(u64 addr)
> +{
> +	struct hlist_head *entry = lockhashentry(addr);
> +	struct lock_stat *ret;
> +
> +	hlist_for_each_entry(ret, entry, hash_entry) {
> +		if (ret->addr == addr)
> +			return ret;
> +	}
> +	return NULL;
> +}
> +
> +struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags)
> +{
> +	struct hlist_head *entry = lockhashentry(addr);
> +	struct lock_stat *ret, *new;
> +
> +	hlist_for_each_entry(ret, entry, hash_entry) {
> +		if (ret->addr == addr)
> +			return ret;
> +	}
> +
> +	new = zalloc(sizeof(struct lock_stat));
> +	if (!new)
> +		goto alloc_failed;
> +
> +	new->addr = addr;
> +	new->name = strdup(name);
> +	if (!new->name) {
> +		free(new);
> +		goto alloc_failed;
> +	}
> +
> +	new->flags = flags;
> +	new->wait_time_min = ULLONG_MAX;
> +
> +	hlist_add_head(&new->hash_entry, entry);
> +	return new;
> +
> +alloc_failed:
> +	pr_err("memory allocation failed\n");
> +	return NULL;
> +}
> +
> +bool match_callstack_filter(struct machine *machine, u64 *callstack, int max_stack_depth)
> +{
> +	struct map *kmap;
> +	struct symbol *sym;
> +	u64 ip;
> +	const char *arch = perf_env__arch(machine->env);
> +
> +	if (list_empty(&callstack_filters))
> +		return true;
> +
> +	for (int i = 0; i < max_stack_depth; i++) {
> +		struct callstack_filter *filter;
> +
> +		/*
> +		 * In powerpc, the callchain saved by kernel always includes
> +		 * first three entries as the NIP (next instruction pointer),
> +		 * LR (link register), and the contents of LR save area in the
> +		 * second stack frame. In certain scenarios its possible to have
> +		 * invalid kernel instruction addresses in either LR or the second
> +		 * stack frame's LR. In that case, kernel will store that address as
> +		 * zero.
> +		 *
> +		 * The below check will continue to look into callstack,
> +		 * incase first or second callstack index entry has 0
> +		 * address for powerpc.
> +		 */
> +		if (!callstack || (!callstack[i] && (strcmp(arch, "powerpc") ||
> +						(i != 1 && i != 2))))
> +			break;
> +
> +		ip = callstack[i];
> +		sym = machine__find_kernel_symbol(machine, ip, &kmap);
> +		if (sym == NULL)
> +			continue;
> +
> +		list_for_each_entry(filter, &callstack_filters, list) {
> +			if (strstr(sym->name, filter->name))
> +				return true;
> +		}
> +	}
> +	return false;
> +}
> +
> +#ifndef HAVE_BPF_SKEL
> +int lock_contention_prepare(struct lock_contention *con __maybe_unused)
> +{
> +	return 0;
> +}
> +
> +int lock_contention_start(void)
> +{
> +	return 0;
> +}
> +
> +int lock_contention_stop(void)
> +{
> +	return 0;
> +}
> +
> +int lock_contention_finish(struct lock_contention *con __maybe_unused)
> +{
> +	return 0;
> +}
> +
> +int lock_contention_read(struct lock_contention *con __maybe_unused)
> +{
> +	return 0;
> +}
> +#endif  /* !HAVE_BPF_SKEL */
> diff --git a/tools/perf/util/lock-contention.h b/tools/perf/util/lock-contention.h
> index 1a7248ff3889..bfa5c7db0a5d 100644
> --- a/tools/perf/util/lock-contention.h
> +++ b/tools/perf/util/lock-contention.h
> @@ -67,10 +67,11 @@ struct lock_stat {
>   */
>  #define MAX_LOCK_DEPTH 48
>  
> -struct lock_stat *lock_stat_find(u64 addr);
> -struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags);
> +/* based on kernel/lockdep.c */
> +#define LOCKHASH_BITS		12
> +#define LOCKHASH_SIZE		(1UL << LOCKHASH_BITS)
>  
> -bool match_callstack_filter(struct machine *machine, u64 *callstack);
> +extern struct hlist_head *lockhash_table;
>  
>  /*
>   * struct lock_seq_stat:
> @@ -148,7 +149,14 @@ struct lock_contention {
>  	bool save_callstack;
>  };
>  
> -#ifdef HAVE_BPF_SKEL
> +struct option;
> +int parse_call_stack(const struct option *opt, const char *str, int unset);
> +bool needs_callstack(void);
> +
> +struct lock_stat *lock_stat_find(u64 addr);
> +struct lock_stat *lock_stat_findnew(u64 addr, const char *name, int flags);
> +
> +bool match_callstack_filter(struct machine *machine, u64 *callstack, int max_stack_depth);
>  
>  int lock_contention_prepare(struct lock_contention *con);
>  int lock_contention_start(void);
> @@ -156,25 +164,4 @@ int lock_contention_stop(void);
>  int lock_contention_read(struct lock_contention *con);
>  int lock_contention_finish(struct lock_contention *con);
>  
> -#else  /* !HAVE_BPF_SKEL */
> -
> -static inline int lock_contention_prepare(struct lock_contention *con __maybe_unused)
> -{
> -	return 0;
> -}
> -
> -static inline int lock_contention_start(void) { return 0; }
> -static inline int lock_contention_stop(void) { return 0; }
> -static inline int lock_contention_finish(struct lock_contention *con __maybe_unused)
> -{
> -	return 0;
> -}
> -
> -static inline int lock_contention_read(struct lock_contention *con __maybe_unused)
> -{
> -	return 0;
> -}
> -
> -#endif  /* HAVE_BPF_SKEL */
> -
>  #endif  /* PERF_LOCK_CONTENTION_H */
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 35d84a96dbec..91fd444615cd 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -18,7 +18,6 @@
>  #include "mmap.h"
>  #include "util/kwork.h"
>  #include "util/sample.h"
> -#include "util/lock-contention.h"
>  #include <internal/lib.h>
>  #include "../builtin.h"
>  
> @@ -1311,22 +1310,6 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
>  	return NULL;
>  }
>  
> -bool match_callstack_filter(struct machine *machine __maybe_unused, u64 *callstack __maybe_unused)
> -{
> -	return false;
> -}
> -
> -struct lock_stat *lock_stat_find(u64 addr __maybe_unused)
> -{
> -	return NULL;
> -}
> -
> -struct lock_stat *lock_stat_findnew(u64 addr __maybe_unused, const char *name __maybe_unused,
> -				int flags __maybe_unused)
> -{
> -	return NULL;
> -}
> -
>  int cmd_inject(int argc __maybe_unused, const char *argv[] __maybe_unused)
>  {
>  	return -1;
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 15/21] perf bench: Remove reference to cmd_inject

[Ian Rogers]

Avoid `perf bench internals inject-build-id` referencing the
cmd_inject sub-command that requires perf-bench to backward reference
internals of builtins. Replace the reference to cmd_inject with a call
to main. To avoid python.c needing to link with something providing
main, drop the libperf-bench library from the python shared object.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.perf          |  7 +++++--
 tools/perf/bench/inject-buildid.c | 13 +++++++------
 tools/perf/util/python.c          |  6 ------
 3 files changed, 12 insertions(+), 14 deletions(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index b4dee7c20ed1..ef1b39b5042d 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -487,6 +487,9 @@ endif
 EXTLIBS := $(call filter-out,$(EXCLUDE_EXTLIBS),$(EXTLIBS))
 LIBS = -Wl,--whole-archive $(PERFLIBS) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
 
+PERFLIBS_PY := $(call filter-out,$(LIBPERF_BENCH),$(PERFLIBS))
+LIBS_PY = -Wl,--whole-archive $(PERFLIBS_PY) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
+
 export INSTALL SHELL_PATH
 
 ### Build rules
@@ -735,9 +738,9 @@ all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS)
 # Create python binding output directory if not already present
 $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
 
-$(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): util/python.c util/setup.py $(PERFLIBS)
+$(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): util/python.c util/setup.py $(PERFLIBS_PY)
 	$(QUIET_GEN)LDSHARED="$(CC) -pthread -shared" \
-        CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS) $(LIBS)' \
+        CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS) $(LIBS_PY)' \
 	  $(PYTHON_WORD) util/setup.py \
 	  --quiet build_ext; \
 	cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
diff --git a/tools/perf/bench/inject-buildid.c b/tools/perf/bench/inject-buildid.c
index a759eb2328be..f55c07e4be94 100644
--- a/tools/perf/bench/inject-buildid.c
+++ b/tools/perf/bench/inject-buildid.c
@@ -52,7 +52,7 @@ struct bench_dso {
 static int nr_dsos;
 static struct bench_dso *dsos;
 
-extern int cmd_inject(int argc, const char *argv[]);
+extern int main(int argc, const char **argv);
 
 static const struct option options[] = {
 	OPT_UINTEGER('i', "iterations", &iterations,
@@ -294,7 +294,7 @@ static int setup_injection(struct bench_data *data, bool build_id_all)
 
 	if (data->pid == 0) {
 		const char **inject_argv;
-		int inject_argc = 2;
+		int inject_argc = 3;
 
 		close(data->input_pipe[1]);
 		close(data->output_pipe[0]);
@@ -318,15 +318,16 @@ static int setup_injection(struct bench_data *data, bool build_id_all)
 		if (inject_argv == NULL)
 			exit(1);
 
-		inject_argv[0] = strdup("inject");
-		inject_argv[1] = strdup("-b");
+		inject_argv[0] = strdup("perf");
+		inject_argv[1] = strdup("inject");
+		inject_argv[2] = strdup("-b");
 		if (build_id_all)
-			inject_argv[2] = strdup("--buildid-all");
+			inject_argv[3] = strdup("--buildid-all");
 
 		/* signal that we're ready to go */
 		close(ready_pipe[1]);
 
-		cmd_inject(inject_argc, inject_argv);
+		main(inject_argc, inject_argv);
 
 		exit(0);
 	}
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 91fd444615cd..c52da509ae58 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -19,7 +19,6 @@
 #include "util/kwork.h"
 #include "util/sample.h"
 #include <internal/lib.h>
-#include "../builtin.h"
 
 #define _PyUnicode_FromString(arg) \
   PyUnicode_FromString(arg)
@@ -1309,8 +1308,3 @@ struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
 {
 	return NULL;
 }
-
-int cmd_inject(int argc __maybe_unused, const char *argv[] __maybe_unused)
-{
-	return -1;
-}
-- 
2.47.0.163.g1226f6d8fa-goog

[PATCH v5 16/21] perf kwork: Make perf_kwork_add_work a callback

[Ian Rogers]

perf_kwork_add_work is declared in builtin-kwork, whereas much kwork
code is in util. To avoid needing to stub perf_kwork_add_work in
python.c, add a callback to struct perf_kwork and initialize it in
builtin-kwork to perf_kwork_add_work - this is the only struct
perf_kwork. This removes the need for the stub in python.c.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-kwork.c      |  3 ++-
 tools/perf/util/bpf_kwork.c     |  2 +-
 tools/perf/util/bpf_kwork_top.c |  2 +-
 tools/perf/util/kwork.h         |  6 ++++--
 tools/perf/util/python.c        | 12 ------------
 5 files changed, 8 insertions(+), 17 deletions(-)

diff --git a/tools/perf/builtin-kwork.c b/tools/perf/builtin-kwork.c
index c1daf82c9b92..fabfcb74800b 100644
--- a/tools/perf/builtin-kwork.c
+++ b/tools/perf/builtin-kwork.c
@@ -1846,7 +1846,7 @@ static void process_skipped_events(struct perf_kwork *kwork,
 	}
 }
 
-struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork,
+static struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork,
 				       struct kwork_class *class,
 				       struct kwork_work *key)
 {
@@ -2344,6 +2344,7 @@ int cmd_kwork(int argc, const char **argv)
 		.all_runtime         = 0,
 		.all_count           = 0,
 		.nr_skipped_events   = { 0 },
+		.add_work            = perf_kwork_add_work,
 	};
 	static const char default_report_sort_order[] = "runtime, max, count";
 	static const char default_latency_sort_order[] = "avg, max, count";
diff --git a/tools/perf/util/bpf_kwork.c b/tools/perf/util/bpf_kwork.c
index 6c7126b7670d..5cff755c71fa 100644
--- a/tools/perf/util/bpf_kwork.c
+++ b/tools/perf/util/bpf_kwork.c
@@ -285,7 +285,7 @@ static int add_work(struct perf_kwork *kwork,
 	    (bpf_trace->get_work_name(key, &tmp.name)))
 		return -1;
 
-	work = perf_kwork_add_work(kwork, tmp.class, &tmp);
+	work = kwork->add_work(kwork, tmp.class, &tmp);
 	if (work == NULL)
 		return -1;
 
diff --git a/tools/perf/util/bpf_kwork_top.c b/tools/perf/util/bpf_kwork_top.c
index 7261cad43468..b6f187dd9136 100644
--- a/tools/perf/util/bpf_kwork_top.c
+++ b/tools/perf/util/bpf_kwork_top.c
@@ -255,7 +255,7 @@ static int add_work(struct perf_kwork *kwork, struct work_key *key,
 	bpf_trace = kwork_class_bpf_supported_list[type];
 	tmp.class = bpf_trace->class;
 
-	work = perf_kwork_add_work(kwork, tmp.class, &tmp);
+	work = kwork->add_work(kwork, tmp.class, &tmp);
 	if (!work)
 		return -1;
 
diff --git a/tools/perf/util/kwork.h b/tools/perf/util/kwork.h
index 76fe2a821bcf..29352ca1d497 100644
--- a/tools/perf/util/kwork.h
+++ b/tools/perf/util/kwork.h
@@ -251,12 +251,14 @@ struct perf_kwork {
 	 * perf kwork top data
 	 */
 	struct kwork_top_stat top_stat;
-};
 
-struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork,
+	/* Add work callback. */
+	struct kwork_work *(*add_work)(struct perf_kwork *kwork,
 				       struct kwork_class *class,
 				       struct kwork_work *key);
 
+};
+
 #ifdef HAVE_BPF_SKEL
 
 int perf_kwork__trace_prepare_bpf(struct perf_kwork *kwork);
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index c52da509ae58..6e2ff0076daa 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -16,7 +16,6 @@
 #include "thread_map.h"
 #include "trace-event.h"
 #include "mmap.h"
-#include "util/kwork.h"
 #include "util/sample.h"
 #include <internal/lib.h>
 
@@ -1297,14 +1296,3 @@ PyMODINIT_FUNC PyInit_perf(void)
 		PyErr_SetString(PyExc_ImportError, "perf: Init failed!");
 	return module;
 }
-
-
-/* The following are stubs to avoid dragging in builtin-* objects. */
-/* TODO: move the code out of the builtin-* file into util. */
-
-struct kwork_work *perf_kwork_add_work(struct perf_kwork *kwork __maybe_unused,
-				       struct kwork_class *class __maybe_unused,
-				       struct kwork_work *key  __maybe_unused)
-{
-	return NULL;
-}
-- 
2.47.0.163.g1226f6d8fa-goog

[PATCH v5 17/21] perf build: Remove test library from python shared object

[Ian Rogers]

With the attr.c code moved to a shell test, there is no need to link
the test code into the python dso to avoid a missing reference to
test_attr__open. Drop the test code from the python library.

With the bench and test code removed from the python library on my x86
debian derived laptop the python library is reduced in size by 508,712
bytes or nearly 5%.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.perf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index ef1b39b5042d..2059f1c4b78d 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -487,7 +487,7 @@ endif
 EXTLIBS := $(call filter-out,$(EXCLUDE_EXTLIBS),$(EXTLIBS))
 LIBS = -Wl,--whole-archive $(PERFLIBS) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
 
-PERFLIBS_PY := $(call filter-out,$(LIBPERF_BENCH),$(PERFLIBS))
+PERFLIBS_PY := $(call filter-out,$(LIBPERF_BENCH) $(LIBPERF_TEST),$(PERFLIBS))
 LIBS_PY = -Wl,--whole-archive $(PERFLIBS_PY) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
 
 export INSTALL SHELL_PATH
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 17/21] perf build: Remove test library from python shared object

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:48PM -0700, Ian Rogers wrote:
> With the attr.c code moved to a shell test, there is no need to link
> the test code into the python dso to avoid a missing reference to
> test_attr__open. Drop the test code from the python library.
> 
> With the bench and test code removed from the python library on my x86
> debian derived laptop the python library is reduced in size by 508,712
> bytes or nearly 5%.

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/Makefile.perf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index ef1b39b5042d..2059f1c4b78d 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -487,7 +487,7 @@ endif
>  EXTLIBS := $(call filter-out,$(EXCLUDE_EXTLIBS),$(EXTLIBS))
>  LIBS = -Wl,--whole-archive $(PERFLIBS) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
>  
> -PERFLIBS_PY := $(call filter-out,$(LIBPERF_BENCH),$(PERFLIBS))
> +PERFLIBS_PY := $(call filter-out,$(LIBPERF_BENCH) $(LIBPERF_TEST),$(PERFLIBS))
>  LIBS_PY = -Wl,--whole-archive $(PERFLIBS_PY) $(EXTRA_PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
>  
>  export INSTALL SHELL_PATH
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 18/21] perf python: Add parse_events function

[Ian Rogers]

Add basic parse_events function that takes a string and returns an
evlist. As the python evlist is embedded in a pyrf_evlist, and the
evsels are embedded in pyrf_evsels, copy the parsed data into those
structs and update evsel__clone to enable this.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/cgroup.c |  2 +-
 tools/perf/util/evsel.c  | 19 ++++++++-----
 tools/perf/util/evsel.h  |  2 +-
 tools/perf/util/python.c | 61 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 75 insertions(+), 9 deletions(-)

diff --git a/tools/perf/util/cgroup.c b/tools/perf/util/cgroup.c
index 0f759dd96db7..fbcc0626f9ce 100644
--- a/tools/perf/util/cgroup.c
+++ b/tools/perf/util/cgroup.c
@@ -473,7 +473,7 @@ int evlist__expand_cgroup(struct evlist *evlist, const char *str,
 
 		leader = NULL;
 		evlist__for_each_entry(orig_list, pos) {
-			evsel = evsel__clone(pos);
+			evsel = evsel__clone(/*dest=*/NULL, pos);
 			if (evsel == NULL)
 				goto out_err;
 
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index f780e30aa259..4213ffacdc3c 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -453,7 +453,7 @@ static int evsel__copy_config_terms(struct evsel *dst, struct evsel *src)
  * The assumption is that @orig is not configured nor opened yet.
  * So we only care about the attributes that can be set while it's parsed.
  */
-struct evsel *evsel__clone(struct evsel *orig)
+struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig)
 {
 	struct evsel *evsel;
 
@@ -466,7 +466,11 @@ struct evsel *evsel__clone(struct evsel *orig)
 	if (orig->bpf_obj)
 		return NULL;
 
-	evsel = evsel__new(&orig->core.attr);
+	if (dest)
+		evsel = dest;
+	else
+		evsel = evsel__new(&orig->core.attr);
+
 	if (evsel == NULL)
 		return NULL;
 
@@ -511,11 +515,12 @@ struct evsel *evsel__clone(struct evsel *orig)
 	evsel->core.leader = orig->core.leader;
 
 	evsel->max_events = orig->max_events;
-	free((char *)evsel->unit);
-	evsel->unit = strdup(orig->unit);
-	if (evsel->unit == NULL)
-		goto out_err;
-
+	zfree(&evsel->unit);
+	if (orig->unit) {
+		evsel->unit = strdup(orig->unit);
+		if (evsel->unit == NULL)
+			goto out_err;
+	}
 	evsel->scale = orig->scale;
 	evsel->snapshot = orig->snapshot;
 	evsel->per_pkg = orig->per_pkg;
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 04934a7af174..7f41d6f8cf6f 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -241,7 +241,7 @@ static inline struct evsel *evsel__new(struct perf_event_attr *attr)
 	return evsel__new_idx(attr, 0);
 }
 
-struct evsel *evsel__clone(struct evsel *orig);
+struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig);
 
 int copy_config_terms(struct list_head *dst, struct list_head *src);
 void free_config_terms(struct list_head *config_terms);
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 6e2ff0076daa..1a53a97c513d 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -13,6 +13,7 @@
 #include "evsel.h"
 #include "event.h"
 #include "print_binary.h"
+#include "strbuf.h"
 #include "thread_map.h"
 #include "trace-event.h"
 #include "mmap.h"
@@ -1201,6 +1202,60 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
 #endif // HAVE_LIBTRACEEVENT
 }
 
+static PyObject *pyrf_evsel__from_evsel(struct evsel *evsel)
+{
+	struct pyrf_evsel *pevsel = PyObject_New(struct pyrf_evsel, &pyrf_evsel__type);
+
+	if (!pevsel)
+		return NULL;
+
+	memset(&pevsel->evsel, 0, sizeof(pevsel->evsel));
+	evsel__init(&pevsel->evsel, &evsel->core.attr, evsel->core.idx);
+
+	evsel__clone(&pevsel->evsel, evsel);
+	return (PyObject *)pevsel;
+}
+
+static PyObject *pyrf_evlist__from_evlist(struct evlist *evlist)
+{
+	struct pyrf_evlist *pevlist = PyObject_New(struct pyrf_evlist, &pyrf_evlist__type);
+	struct evsel *pos;
+
+	if (!pevlist)
+		return NULL;
+
+	memset(&pevlist->evlist, 0, sizeof(pevlist->evlist));
+	evlist__init(&pevlist->evlist, evlist->core.all_cpus, evlist->core.threads);
+	evlist__for_each_entry(evlist, pos) {
+		struct pyrf_evsel *pevsel = (void *)pyrf_evsel__from_evsel(pos);
+
+		evlist__add(&pevlist->evlist, &pevsel->evsel);
+	}
+	return (PyObject *)pevlist;
+}
+
+static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)
+{
+	const char *input;
+	struct evlist evlist = {};
+	struct parse_events_error err;
+	PyObject *result;
+
+	if (!PyArg_ParseTuple(args, "s", &input))
+		return NULL;
+
+	parse_events_error__init(&err);
+	evlist__init(&evlist, NULL, NULL);
+	if (parse_events(&evlist, input, &err)) {
+		parse_events_error__print(&err, input);
+		PyErr_SetFromErrno(PyExc_OSError);
+		return NULL;
+	}
+	result = pyrf_evlist__from_evlist(&evlist);
+	evlist__exit(&evlist);
+	return result;
+}
+
 static PyMethodDef perf__methods[] = {
 	{
 		.ml_name  = "tracepoint",
@@ -1208,6 +1263,12 @@ static PyMethodDef perf__methods[] = {
 		.ml_flags = METH_VARARGS | METH_KEYWORDS,
 		.ml_doc	  = PyDoc_STR("Get tracepoint config.")
 	},
+	{
+		.ml_name  = "parse_events",
+		.ml_meth  = (PyCFunction) pyrf__parse_events,
+		.ml_flags = METH_VARARGS,
+		.ml_doc	  = PyDoc_STR("Parse a string of events and return an evlist.")
+	},
 	{ .ml_name = NULL, }
 };
 
-- 
2.47.0.163.g1226f6d8fa-goog

[PATCH v5 19/21] perf python: Add __str__ and __repr__ functions to evlist

[Ian Rogers]

This allows the values in the evlist to be shown in the REPL like:
```
Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0,'/tmp/perf/python')
>>> import perf
>>> perf.parse_events('cycles,data_read')
evlist([cycles,uncore_imc_free_running_0/data_read/,uncore_imc_free_running_1/data_read/])
```

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/python.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 1a53a97c513d..a71bad3418da 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1069,6 +1069,30 @@ static PyObject *pyrf_evlist__item(PyObject *obj, Py_ssize_t i)
 	return Py_BuildValue("O", container_of(pos, struct pyrf_evsel, evsel));
 }
 
+static PyObject *pyrf_evlist__str(PyObject *self)
+{
+	struct pyrf_evlist *pevlist = (void *)self;
+	struct evsel *pos;
+	struct strbuf sb = STRBUF_INIT;
+	bool first = true;
+	PyObject *result;
+
+	strbuf_addstr(&sb, "evlist([");
+	evlist__for_each_entry(&pevlist->evlist, pos) {
+		if (!first)
+			strbuf_addch(&sb, ',');
+		if (!pos->pmu)
+			strbuf_addstr(&sb, evsel__name(pos));
+		else
+			strbuf_addf(&sb, "%s/%s/", pos->pmu->name, evsel__name(pos));
+		first = false;
+	}
+	strbuf_addstr(&sb, "])");
+	result = PyUnicode_FromString(sb.buf);
+	strbuf_release(&sb);
+	return result;
+}
+
 static PySequenceMethods pyrf_evlist__sequence_methods = {
 	.sq_length = pyrf_evlist__length,
 	.sq_item   = pyrf_evlist__item,
@@ -1086,6 +1110,8 @@ static PyTypeObject pyrf_evlist__type = {
 	.tp_doc		= pyrf_evlist__doc,
 	.tp_methods	= pyrf_evlist__methods,
 	.tp_init	= (initproc)pyrf_evlist__init,
+	.tp_repr        = pyrf_evlist__str,
+	.tp_str         = pyrf_evlist__str,
 };
 
 static int pyrf_evlist__setup_types(void)
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 19/21] perf python: Add __str__ and __repr__ functions to evlist

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:50PM -0700, Ian Rogers wrote:
> This allows the values in the evlist to be shown in the REPL like:
> ```
> Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.path.insert(0,'/tmp/perf/python')
> >>> import perf
> >>> perf.parse_events('cycles,data_read')
> evlist([cycles,uncore_imc_free_running_0/data_read/,uncore_imc_free_running_1/data_read/])
> ```

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/python.c | 26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 1a53a97c513d..a71bad3418da 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1069,6 +1069,30 @@ static PyObject *pyrf_evlist__item(PyObject *obj, Py_ssize_t i)
>  	return Py_BuildValue("O", container_of(pos, struct pyrf_evsel, evsel));
>  }
>  
> +static PyObject *pyrf_evlist__str(PyObject *self)
> +{
> +	struct pyrf_evlist *pevlist = (void *)self;
> +	struct evsel *pos;
> +	struct strbuf sb = STRBUF_INIT;
> +	bool first = true;
> +	PyObject *result;
> +
> +	strbuf_addstr(&sb, "evlist([");
> +	evlist__for_each_entry(&pevlist->evlist, pos) {
> +		if (!first)
> +			strbuf_addch(&sb, ',');
> +		if (!pos->pmu)
> +			strbuf_addstr(&sb, evsel__name(pos));
> +		else
> +			strbuf_addf(&sb, "%s/%s/", pos->pmu->name, evsel__name(pos));
> +		first = false;
> +	}
> +	strbuf_addstr(&sb, "])");
> +	result = PyUnicode_FromString(sb.buf);
> +	strbuf_release(&sb);
> +	return result;
> +}
> +
>  static PySequenceMethods pyrf_evlist__sequence_methods = {
>  	.sq_length = pyrf_evlist__length,
>  	.sq_item   = pyrf_evlist__item,
> @@ -1086,6 +1110,8 @@ static PyTypeObject pyrf_evlist__type = {
>  	.tp_doc		= pyrf_evlist__doc,
>  	.tp_methods	= pyrf_evlist__methods,
>  	.tp_init	= (initproc)pyrf_evlist__init,
> +	.tp_repr        = pyrf_evlist__str,
> +	.tp_str         = pyrf_evlist__str,
>  };
>  
>  static int pyrf_evlist__setup_types(void)
> -- 
> 2.47.0.163.g1226f6d8fa-goog

[PATCH v5 20/21] perf python: Add __str__ and __repr__ functions to evsel

[Ian Rogers]

This allows evsel to be shown in the REPL like:
```
Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, '/tmp/perf/python')
>>> import perf
>>> x=perf.parse_events('cycles,data_read')
>>> print(x)
evlist([cycles,uncore_imc_free_running_0/data_read/,uncore_imc_free_running_1/data_read/])
>>> x[0]
evsel(cycles)
>>> x[1]
evsel(uncore_imc_free_running_0/data_read/)
>>> x[2]
evsel(uncore_imc_free_running_1/data_read/)
```

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/python.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index a71bad3418da..0d71ec673aa3 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -781,6 +781,17 @@ static PyObject *pyrf_evsel__open(struct pyrf_evsel *pevsel,
 	return Py_None;
 }
 
+static PyObject *pyrf_evsel__str(PyObject *self)
+{
+	struct pyrf_evsel *pevsel = (void *)self;
+	struct evsel *evsel = &pevsel->evsel;
+
+	if (!evsel->pmu)
+		return PyUnicode_FromFormat("evsel(%s)", evsel__name(evsel));
+
+	return PyUnicode_FromFormat("evsel(%s/%s/)", evsel->pmu->name, evsel__name(evsel));
+}
+
 static PyMethodDef pyrf_evsel__methods[] = {
 	{
 		.ml_name  = "open",
@@ -802,6 +813,8 @@ static PyTypeObject pyrf_evsel__type = {
 	.tp_doc		= pyrf_evsel__doc,
 	.tp_methods	= pyrf_evsel__methods,
 	.tp_init	= (initproc)pyrf_evsel__init,
+	.tp_str         = pyrf_evsel__str,
+	.tp_repr        = pyrf_evsel__str,
 };
 
 static int pyrf_evsel__setup_types(void)
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 20/21] perf python: Add __str__ and __repr__ functions to evsel

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:51PM -0700, Ian Rogers wrote:
> This allows evsel to be shown in the REPL like:
> ```
> Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.path.insert(0, '/tmp/perf/python')
> >>> import perf
> >>> x=perf.parse_events('cycles,data_read')
> >>> print(x)
> evlist([cycles,uncore_imc_free_running_0/data_read/,uncore_imc_free_running_1/data_read/])
> >>> x[0]
> evsel(cycles)
> >>> x[1]
> evsel(uncore_imc_free_running_0/data_read/)
> >>> x[2]
> evsel(uncore_imc_free_running_1/data_read/)
> ```
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/python.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index a71bad3418da..0d71ec673aa3 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -781,6 +781,17 @@ static PyObject *pyrf_evsel__open(struct pyrf_evsel *pevsel,
>  	return Py_None;
>  }
>  
> +static PyObject *pyrf_evsel__str(PyObject *self)
> +{
> +	struct pyrf_evsel *pevsel = (void *)self;
> +	struct evsel *evsel = &pevsel->evsel;
> +
> +	if (!evsel->pmu)
> +		return PyUnicode_FromFormat("evsel(%s)", evsel__name(evsel));
> +
> +	return PyUnicode_FromFormat("evsel(%s/%s/)", evsel->pmu->name, evsel__name(evsel));
> +}
> +
>  static PyMethodDef pyrf_evsel__methods[] = {
>  	{
>  		.ml_name  = "open",
> @@ -802,6 +813,8 @@ static PyTypeObject pyrf_evsel__type = {
>  	.tp_doc		= pyrf_evsel__doc,
>  	.tp_methods	= pyrf_evsel__methods,
>  	.tp_init	= (initproc)pyrf_evsel__init,
> +	.tp_str         = pyrf_evsel__str,
> +	.tp_repr        = pyrf_evsel__str,
>  };
>  
>  static int pyrf_evsel__setup_types(void)
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

[PATCH v5 21/21] perf python: Correctly throw IndexError

[Ian Rogers]

Correctly throw IndexError for out-of-bound accesses to evlist:
```
Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, '/tmp/perf/python')
>>> import perf
>>> x=perf.parse_events('cycles')
>>> print(x)
evlist([cycles])
>>> x[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: Index out of range
```

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/python.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 0d71ec673aa3..25114dcadd21 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -1071,8 +1071,10 @@ static PyObject *pyrf_evlist__item(PyObject *obj, Py_ssize_t i)
 	struct pyrf_evlist *pevlist = (void *)obj;
 	struct evsel *pos;
 
-	if (i >= pevlist->evlist.core.nr_entries)
+	if (i >= pevlist->evlist.core.nr_entries) {
+		PyErr_SetString(PyExc_IndexError, "Index out of range");
 		return NULL;
+	}
 
 	evlist__for_each_entry(&pevlist->evlist, pos) {
 		if (i-- == 0)
-- 
2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 21/21] perf python: Correctly throw IndexError

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:52PM -0700, Ian Rogers wrote:
> Correctly throw IndexError for out-of-bound accesses to evlist:
> ```
> Python 3.11.9 (main, Jun 19 2024, 00:38:48) [GCC 13.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.path.insert(0, '/tmp/perf/python')
> >>> import perf
> >>> x=perf.parse_events('cycles')
> >>> print(x)
> evlist([cycles])
> >>> x[2]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> IndexError: Index out of range
> ```

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/python.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index 0d71ec673aa3..25114dcadd21 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -1071,8 +1071,10 @@ static PyObject *pyrf_evlist__item(PyObject *obj, Py_ssize_t i)
>  	struct pyrf_evlist *pevlist = (void *)obj;
>  	struct evsel *pos;
>  
> -	if (i >= pevlist->evlist.core.nr_entries)
> +	if (i >= pevlist->evlist.core.nr_entries) {
> +		PyErr_SetString(PyExc_IndexError, "Index out of range");
>  		return NULL;
> +	}
>  
>  	evlist__for_each_entry(&pevlist->evlist, pos) {
>  		if (i-- == 0)
> -- 
> 2.47.0.163.g1226f6d8fa-goog

Re: [PATCH v5 00/21] Python module cleanup

[Arnaldo Carvalho de Melo]

On Wed, Oct 30, 2024 at 06:42:31PM -0700, Ian Rogers wrote:
> This patch:
>  - removes workarounds for Python 2 module support due to long
>    deprecation and challenges in developing new code;
>  - constifies variables and parameters to functions;
>  - removes python.c stub code which existed due to missing functions
>    that are defined in the builtin-* files, in general the builtin-*
>    code is moved into util;
>  - remove bench and test perf C code from the python module;
>  - adds parse_events to the python perf module.
>  - improves upon some of the existing python perf module functins.
> 
> v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
>     caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
>     arch_fetch_insn in the process, which was incorrectly using an
>     empty weak symbol stub.


I created a 'perf test' shell entry to test the python binding and ran
it after each relevant patch in this series, as well as doing 'perf
script -' to list available scripts, all seem to work.

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

I'll send the perf test entry now.

- Arnaldo


> v4. Rebase. Fix the parse events evsel to be embedded in a
>     pyrf_evsel. Add __str__/__repr__ functions to evlist and
>     evsel. Throw an exception for a bad evlist index.
> v3. Move is_directory_at to patch 6 rather than patch 7, respond to
>     review feedback on the list from Namhyung.
> v2. Add the bottom 4 bullet points - 13 more patches.
> 
> Ian Rogers (21):
>   perf python: Remove python 2 scripting support
>   perf python: Constify variables and parameters
>   perf python: Remove unused #include
>   perf script: Move scripting_max_stack out of builtin
>   perf kvm: Move functions used in util out of builtin
>   perf script: Move find_scripts to browser/scripts.c
>   perf stat: Move stat_config into config.c
>   perf script: Move script_spec code to trace-event-scripting.c
>   perf script: Move script_fetch_insn to trace-event-scripting.c
>   perf script: Move perf_sample__sprintf_flags to
>     trace-event-scripting.c
>   perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
>   perf intel-pt: Remove stale build comment
>   perf env: Move arch errno function to only use in env
>   perf lock: Move common lock contention code to new file
>   perf bench: Remove reference to cmd_inject
>   perf kwork: Make perf_kwork_add_work a callback
>   perf build: Remove test library from python shared object
>   perf python: Add parse_events function
>   perf python: Add __str__ and __repr__ functions to evlist
>   perf python: Add __str__ and __repr__ functions to evsel
>   perf python: Correctly throw IndexError
> 
>  tools/perf/Makefile.perf                      |   7 +-
>  tools/perf/arch/x86/util/Build                |   2 +-
>  tools/perf/bench/inject-buildid.c             |  13 +-
>  tools/perf/builtin-kvm.c                      |  61 ----
>  tools/perf/builtin-kwork.c                    |   3 +-
>  tools/perf/builtin-lock.c                     | 137 +------
>  tools/perf/builtin-script.c                   | 304 +---------------
>  tools/perf/builtin-stat.c                     |  27 --
>  tools/perf/builtin-trace.c                    |   1 -
>  tools/perf/builtin.h                          |   6 -
>  .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
>  tools/perf/tests/stat.c                       |  16 +-
>  tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
>  tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
>  tools/perf/util/Build                         |   4 +-
>  tools/perf/util/bpf_kwork.c                   |   2 +-
>  tools/perf/util/bpf_kwork_top.c               |   2 +-
>  tools/perf/util/bpf_lock_contention.c         |   2 +-
>  tools/perf/util/cgroup.c                      |   2 +-
>  tools/perf/util/config.c                      |  27 ++
>  tools/perf/util/dlfilter.c                    |   3 +-
>  tools/perf/util/env.c                         |   4 +
>  tools/perf/util/env.h                         |   2 -
>  tools/perf/util/evsel.c                       |  19 +-
>  tools/perf/util/evsel.h                       |   2 +-
>  tools/perf/util/intel-pt-decoder/Build        |  18 +-
>  .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
>  tools/perf/util/kvm-stat.c                    |  70 ++++
>  tools/perf/util/kvm-stat.h                    |   3 +
>  tools/perf/util/kwork.h                       |   6 +-
>  tools/perf/util/lock-contention.c             | 170 +++++++++
>  tools/perf/util/lock-contention.h             |  37 +-
>  tools/perf/util/path.c                        |  10 +
>  tools/perf/util/path.h                        |   1 +
>  tools/perf/util/python.c                      | 338 ++++++++----------
>  .../scripting-engines/trace-event-python.c    |  63 +---
>  tools/perf/util/stat.h                        |   3 +-
>  tools/perf/util/trace-event-scripting.c       | 177 +++++++++
>  tools/perf/util/trace-event.h                 |   5 +-
>  39 files changed, 872 insertions(+), 878 deletions(-)
>  create mode 100644 tools/perf/util/kvm-stat.c
>  create mode 100644 tools/perf/util/lock-contention.c
> 
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

Re: [PATCH v5 00/21] Python module cleanup

[Ian Rogers]

On Thu, Oct 31, 2024 at 12:40 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> On Wed, Oct 30, 2024 at 06:42:31PM -0700, Ian Rogers wrote:
> > This patch:
> >  - removes workarounds for Python 2 module support due to long
> >    deprecation and challenges in developing new code;
> >  - constifies variables and parameters to functions;
> >  - removes python.c stub code which existed due to missing functions
> >    that are defined in the builtin-* files, in general the builtin-*
> >    code is moved into util;
> >  - remove bench and test perf C code from the python module;
> >  - adds parse_events to the python perf module.
> >  - improves upon some of the existing python perf module functins.
> >
> > v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
> >     caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
> >     arch_fetch_insn in the process, which was incorrectly using an
> >     empty weak symbol stub.
>
>
> I created a 'perf test' shell entry to test the python binding and ran
> it after each relevant patch in this series, as well as doing 'perf
> script -' to list available scripts, all seem to work.
>
> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

There is coverage of just "import perf" in:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/python-use.c?h=perf-tools-next
Which I'd been using throughout creating the series.

Thanks,
Ian

> I'll send the perf test entry now.
>
> - Arnaldo
>
>
> > v4. Rebase. Fix the parse events evsel to be embedded in a
> >     pyrf_evsel. Add __str__/__repr__ functions to evlist and
> >     evsel. Throw an exception for a bad evlist index.
> > v3. Move is_directory_at to patch 6 rather than patch 7, respond to
> >     review feedback on the list from Namhyung.
> > v2. Add the bottom 4 bullet points - 13 more patches.
> >
> > Ian Rogers (21):
> >   perf python: Remove python 2 scripting support
> >   perf python: Constify variables and parameters
> >   perf python: Remove unused #include
> >   perf script: Move scripting_max_stack out of builtin
> >   perf kvm: Move functions used in util out of builtin
> >   perf script: Move find_scripts to browser/scripts.c
> >   perf stat: Move stat_config into config.c
> >   perf script: Move script_spec code to trace-event-scripting.c
> >   perf script: Move script_fetch_insn to trace-event-scripting.c
> >   perf script: Move perf_sample__sprintf_flags to
> >     trace-event-scripting.c
> >   perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
> >   perf intel-pt: Remove stale build comment
> >   perf env: Move arch errno function to only use in env
> >   perf lock: Move common lock contention code to new file
> >   perf bench: Remove reference to cmd_inject
> >   perf kwork: Make perf_kwork_add_work a callback
> >   perf build: Remove test library from python shared object
> >   perf python: Add parse_events function
> >   perf python: Add __str__ and __repr__ functions to evlist
> >   perf python: Add __str__ and __repr__ functions to evsel
> >   perf python: Correctly throw IndexError
> >
> >  tools/perf/Makefile.perf                      |   7 +-
> >  tools/perf/arch/x86/util/Build                |   2 +-
> >  tools/perf/bench/inject-buildid.c             |  13 +-
> >  tools/perf/builtin-kvm.c                      |  61 ----
> >  tools/perf/builtin-kwork.c                    |   3 +-
> >  tools/perf/builtin-lock.c                     | 137 +------
> >  tools/perf/builtin-script.c                   | 304 +---------------
> >  tools/perf/builtin-stat.c                     |  27 --
> >  tools/perf/builtin-trace.c                    |   1 -
> >  tools/perf/builtin.h                          |   6 -
> >  .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
> >  tools/perf/tests/stat.c                       |  16 +-
> >  tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
> >  tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
> >  tools/perf/util/Build                         |   4 +-
> >  tools/perf/util/bpf_kwork.c                   |   2 +-
> >  tools/perf/util/bpf_kwork_top.c               |   2 +-
> >  tools/perf/util/bpf_lock_contention.c         |   2 +-
> >  tools/perf/util/cgroup.c                      |   2 +-
> >  tools/perf/util/config.c                      |  27 ++
> >  tools/perf/util/dlfilter.c                    |   3 +-
> >  tools/perf/util/env.c                         |   4 +
> >  tools/perf/util/env.h                         |   2 -
> >  tools/perf/util/evsel.c                       |  19 +-
> >  tools/perf/util/evsel.h                       |   2 +-
> >  tools/perf/util/intel-pt-decoder/Build        |  18 +-
> >  .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
> >  tools/perf/util/kvm-stat.c                    |  70 ++++
> >  tools/perf/util/kvm-stat.h                    |   3 +
> >  tools/perf/util/kwork.h                       |   6 +-
> >  tools/perf/util/lock-contention.c             | 170 +++++++++
> >  tools/perf/util/lock-contention.h             |  37 +-
> >  tools/perf/util/path.c                        |  10 +
> >  tools/perf/util/path.h                        |   1 +
> >  tools/perf/util/python.c                      | 338 ++++++++----------
> >  .../scripting-engines/trace-event-python.c    |  63 +---
> >  tools/perf/util/stat.h                        |   3 +-
> >  tools/perf/util/trace-event-scripting.c       | 177 +++++++++
> >  tools/perf/util/trace-event.h                 |   5 +-
> >  39 files changed, 872 insertions(+), 878 deletions(-)
> >  create mode 100644 tools/perf/util/kvm-stat.c
> >  create mode 100644 tools/perf/util/lock-contention.c
> >
> > --
> > 2.47.0.163.g1226f6d8fa-goog
> >