[PATCH 0/2] tracing/boot: Support eprobe, fprobe, and tprobe events and add test framework

[PATCH 0/2] tracing/boot: Support eprobe, fprobe, and tprobe events and add test framework

[Masami Hiramatsu (Google)]

Hi,

This series extends kernel boot-time tracing to support newly added
dynamic event types (eprobes, fprobes, and tracepoint probes) and introduces
an automated QEMU-based test suite under tools/tracing/boottime/.
(Note: this requires busybox binary for making initramfs.)

Patch [1/2] updates boot time tracing to handle eprobe, fprobe and
tracepoint/tprobe events from bootconfig.
Patch [2/2] introduces an automated boot-time tracing test framework in
tools/testing/selftests/ftrace/boottime. It creates a minimal initramfs
with busybox, attaches bootconfig sample files, boots QEMU, and runs
corresponding tracefs verification scripts. This test is currently working
on x86 (it should be easy to support other arch, but I don't have such host
environment.)

Thanks,

---
base-commit: ea694cdd92b6bfa123b3f5c563e47b4cf968046a

Masami Hiramatsu (Google) (2):
      tracing/boot: Add support for eprobe, fprobe, and tprobe events
      selftests/ftrace: Add boot-time tracing test framework


 Documentation/trace/boottime-trace.rst             |   18 +-
 kernel/trace/trace_boot.c                          |  113 ++++++++++
 kernel/trace/trace_dynevent.c                      |    5 
 tools/testing/selftests/ftrace/Makefile            |    2 
 tools/testing/selftests/ftrace/boottime/Makefile   |   10 +
 tools/testing/selftests/ftrace/boottime/README     |   58 +++++
 .../ftrace/boottime/bootconfigs/01-kprobe.bconf    |    5 
 .../ftrace/boottime/bootconfigs/02-synth.bconf     |    4 
 .../ftrace/boottime/bootconfigs/03-eprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/04-fprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/05-tprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/06-instance.bconf  |    5 
 .../selftests/ftrace/boottime/run_boottime_test.sh |  218 ++++++++++++++++++++
 .../selftests/ftrace/boottime/tests/01-kprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/02-synth.sh    |   18 ++
 .../selftests/ftrace/boottime/tests/03-eprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/04-fprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/05-tprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/06-instance.sh |   23 ++
 tools/testing/selftests/ftrace/config              |    4 
 20 files changed, 563 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/ftrace/boottime/Makefile
 create mode 100644 tools/testing/selftests/ftrace/boottime/README
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf
 create mode 100755 tools/testing/selftests/ftrace/boottime/run_boottime_test.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/02-synth.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/06-instance.sh

--
Masami Hiramatsu (Google) <mhiramat@kernel.org>

[PATCH 1/2] tracing/boot: Add support for eprobe, fprobe, and tprobe events

[Masami Hiramatsu (Google)]

From: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Boot-time tracing currently supports kprobe-events and synthetic-events
under per-event configuration options.

Extend boot-time tracing to support newly added dynamic probe types:
- event probes (eprobe) under the "eprobes" event group
- function probes (fprobe) under the "fprobes" event group
- tracepoint probes (tprobe) under the "tracepoints" or "tprobes"
  event group

To support this cleanly, update dyn_event_create() in trace_dynevent.c
so that passing NULL as the type parameter delegates to
create_dyn_event(), allowing generic creation of any registered
dynamic event type from a raw command string.

Update Documentation/trace/boottime-trace.rst accordingly to describe
the new per-event bootconfig options.

Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
 Documentation/trace/boottime-trace.rst |   18 ++++-
 kernel/trace/trace_boot.c              |  113 ++++++++++++++++++++++++++++++++
 kernel/trace/trace_dynevent.c          |    5 +
 3 files changed, 133 insertions(+), 3 deletions(-)

diff --git a/Documentation/trace/boottime-trace.rst b/Documentation/trace/boottime-trace.rst
index 651f3a2c01de..2c3c1fedb92c 100644
--- a/Documentation/trace/boottime-trace.rst
+++ b/Documentation/trace/boottime-trace.rst
@@ -121,9 +121,21 @@ ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
    Defines new synthetic event with FIELDs. Each field should be
    "type varname".
 
-Note that kprobe and synthetic event definitions can be written under
-instance node, but those are also visible from other instances. So please
-take care for event name conflict.
+ftrace.[instance.INSTANCE.]event.eprobes.EVENT.probes = PROBE[, PROBE2[...]]
+   Defines new event probe based on PROBEs. This option is available only
+   for the event which group name is "eprobes".
+
+ftrace.[instance.INSTANCE.]event.fprobes.EVENT.probes = PROBE[, PROBE2[...]]
+   Defines new fprobe event based on PROBEs. This option is available only
+   for the event which group name is "fprobes".
+
+ftrace.[instance.INSTANCE.]event.tracepoints.EVENT.probes = PROBE[, PROBE2[...]]
+   Defines new tracepoint probe based on PROBEs. This option is available only
+   for the event which group name is "tracepoints" or "tprobes".
+
+Note that dynamic event definitions can be written under instance node, but
+those are also visible from other instances. So please take care for event
+name conflict.
 
 Ftrace Histogram Options
 ------------------------
diff --git a/kernel/trace/trace_boot.c b/kernel/trace/trace_boot.c
index 2ca2541c8a58..6ecf1a035caa 100644
--- a/kernel/trace/trace_boot.c
+++ b/kernel/trace/trace_boot.c
@@ -18,6 +18,7 @@
 #include <linux/trace_events.h>
 
 #include "trace.h"
+#include "trace_dynevent.h"
 
 #define MAX_BUF_LEN 256
 
@@ -172,6 +173,109 @@ trace_boot_add_synth_event(struct xbc_node *node, const char *event)
 }
 #endif
 
+#ifdef CONFIG_EPROBE_EVENTS
+static int __init
+trace_boot_add_eprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	struct xbc_node *anode;
+	char buf[MAX_BUF_LEN];
+	const char *val;
+	int ret = 0;
+
+	xbc_node_for_each_array_value(node, "probes", anode, val) {
+		if (val[0] == 'e' && (val[1] == ':' || isdigit(val[1])))
+			strscpy(buf, val, MAX_BUF_LEN);
+		else
+			snprintf(buf, MAX_BUF_LEN, "e:%s/%s %s", group, event, val);
+
+		ret = dyn_event_create(buf, NULL);
+		if (ret) {
+			pr_err("Failed to add eprobe: %s\n", buf);
+			break;
+		}
+	}
+
+	return ret;
+}
+#else
+static inline int __init
+trace_boot_add_eprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	pr_err("Event probe is not supported.\n");
+	return -EOPNOTSUPP;
+}
+#endif
+
+#ifdef CONFIG_FPROBE_EVENTS
+static int __init
+trace_boot_add_fprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	struct xbc_node *anode;
+	char buf[MAX_BUF_LEN];
+	const char *val;
+	int ret = 0;
+
+	xbc_node_for_each_array_value(node, "probes", anode, val) {
+		if (val[0] == 'f' && (val[1] == ':' || isdigit(val[1])))
+			strscpy(buf, val, MAX_BUF_LEN);
+		else
+			snprintf(buf, MAX_BUF_LEN, "f:%s/%s %s", group, event, val);
+
+		ret = dyn_event_create(buf, NULL);
+		if (ret) {
+			pr_err("Failed to add fprobe: %s\n", buf);
+			break;
+		}
+	}
+
+	return ret;
+}
+
+static int __init
+trace_boot_add_tprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	struct xbc_node *anode;
+	char buf[MAX_BUF_LEN];
+	const char *val;
+	int ret = 0;
+
+	xbc_node_for_each_array_value(node, "probes", anode, val) {
+		if (val[0] == 't' && (val[1] == ':' || isdigit(val[1])))
+			strscpy(buf, val, MAX_BUF_LEN);
+		else
+			snprintf(buf, MAX_BUF_LEN, "t:%s/%s %s", group, event, val);
+
+		ret = dyn_event_create(buf, NULL);
+		if (ret) {
+			pr_err("Failed to add tprobe: %s\n", buf);
+			break;
+		}
+	}
+
+	return ret;
+}
+#else
+static inline int __init
+trace_boot_add_fprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	pr_err("Fprobe event is not supported.\n");
+	return -EOPNOTSUPP;
+}
+
+static inline int __init
+trace_boot_add_tprobe_event(struct xbc_node *node, const char *group,
+			    const char *event)
+{
+	pr_err("Tracepoint probe is not supported.\n");
+	return -EOPNOTSUPP;
+}
+#endif
+
 #ifdef CONFIG_HIST_TRIGGERS
 static int __init __printf(3, 4)
 append_printf(char **bufp, char *end, const char *fmt, ...)
@@ -477,6 +581,15 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,
 	if (!strcmp(group, "synthetic"))
 		if (trace_boot_add_synth_event(enode, event) < 0)
 			return;
+	if (!strcmp(group, "eprobes"))
+		if (trace_boot_add_eprobe_event(enode, group, event) < 0)
+			return;
+	if (!strcmp(group, "fprobes"))
+		if (trace_boot_add_fprobe_event(enode, group, event) < 0)
+			return;
+	if (!strcmp(group, "tracepoints") || !strcmp(group, "tprobes"))
+		if (trace_boot_add_tprobe_event(enode, group, event) < 0)
+			return;
 
 	mutex_lock(&event_mutex);
 	file = find_event_file(tr, group, event);
diff --git a/kernel/trace/trace_dynevent.c b/kernel/trace/trace_dynevent.c
index c4dfbc293bae..6f2e39b797e5 100644
--- a/kernel/trace/trace_dynevent.c
+++ b/kernel/trace/trace_dynevent.c
@@ -116,6 +116,8 @@ int dyn_event_release(const char *raw_command, struct dyn_event_operations *type
 	return ret;
 }
 
+static int create_dyn_event(const char *raw_command);
+
 /*
  * Locked version of event creation. The event creation must be protected by
  * dyn_event_ops_mutex because of protecting trace_probe_log.
@@ -124,6 +126,9 @@ int dyn_event_create(const char *raw_command, struct dyn_event_operations *type)
 {
 	int ret;
 
+	if (!type)
+		return create_dyn_event(raw_command);
+
 	mutex_lock(&dyn_event_ops_mutex);
 	ret = type->create(raw_command);
 	mutex_unlock(&dyn_event_ops_mutex);

[PATCH 2/2] selftests/ftrace: Add boot-time tracing test framework

[Masami Hiramatsu (Google)]

From: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Add an automated test framework under
tools/testing/selftests/ftrace/boottime/ to verify boot-time tracing
configuration parsing.

The test runner (run_boottime_test.sh) sources ktap_helpers.sh to
generate output in TAP version 13 format. It builds a minimal
initramfs with busybox, attaches bootconfig examples using
tools/bootconfig/bootconfig, boots QEMU, and runs corresponding
tracefs checker scripts.

Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
 tools/testing/selftests/ftrace/Makefile            |    2 
 tools/testing/selftests/ftrace/boottime/Makefile   |   10 +
 tools/testing/selftests/ftrace/boottime/README     |   58 +++++
 .../ftrace/boottime/bootconfigs/01-kprobe.bconf    |    5 
 .../ftrace/boottime/bootconfigs/02-synth.bconf     |    4 
 .../ftrace/boottime/bootconfigs/03-eprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/04-fprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/05-tprobe.bconf    |    4 
 .../ftrace/boottime/bootconfigs/06-instance.bconf  |    5 
 .../selftests/ftrace/boottime/run_boottime_test.sh |  218 ++++++++++++++++++++
 .../selftests/ftrace/boottime/tests/01-kprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/02-synth.sh    |   18 ++
 .../selftests/ftrace/boottime/tests/03-eprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/04-fprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/05-tprobe.sh   |   18 ++
 .../selftests/ftrace/boottime/tests/06-instance.sh |   23 ++
 tools/testing/selftests/ftrace/config              |    4 
 17 files changed, 430 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/ftrace/boottime/Makefile
 create mode 100644 tools/testing/selftests/ftrace/boottime/README
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf
 create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf
 create mode 100755 tools/testing/selftests/ftrace/boottime/run_boottime_test.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/02-synth.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh
 create mode 100755 tools/testing/selftests/ftrace/boottime/tests/06-instance.sh

diff --git a/tools/testing/selftests/ftrace/Makefile b/tools/testing/selftests/ftrace/Makefile
index 7c12263f8260..8d8881744149 100644
--- a/tools/testing/selftests/ftrace/Makefile
+++ b/tools/testing/selftests/ftrace/Makefile
@@ -3,7 +3,7 @@ all:
 
 TEST_PROGS_EXTENDED := ftracetest
 TEST_PROGS := ftracetest-ktap
-TEST_FILES := test.d settings
+TEST_FILES := test.d settings boottime
 EXTRA_CLEAN := $(OUTPUT)/logs/*
 
 TEST_GEN_FILES := poll
diff --git a/tools/testing/selftests/ftrace/boottime/Makefile b/tools/testing/selftests/ftrace/boottime/Makefile
new file mode 100644
index 000000000000..fd0975350f10
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+
+all:
+
+run_tests:
+	@./run_boottime_test.sh
+
+clean:
+
+.PHONY: all run_tests clean
diff --git a/tools/testing/selftests/ftrace/boottime/README b/tools/testing/selftests/ftrace/boottime/README
new file mode 100644
index 000000000000..a6f930a6e44c
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/README
@@ -0,0 +1,58 @@
+Boot-Time Tracing Test Suite
+============================
+
+This directory contains an automated test framework to verify kernel boot-time
+tracing configuration parsing and tracefs application during early boot.
+
+Overview
+--------
+The test runner (`run_boottime_test.sh`) constructs a lightweight initramfs with
+busybox and a tracefs checker script from `tests/`, attaches a bootconfig example
+from `bootconfigs/` using the `bootconfig` tool, boots the target kernel image under
+QEMU, and reports results according to the TAP version 13 format.
+
+Required Tools
+--------------
+- Host Architecture: x86 (x86_64/i686/i386/x86; tests skip on non-x86 hosts)
+- `qemu-system-x86_64` (or custom QEMU executable)
+- `busybox` (host binary at `/usr/bin/busybox` or custom path)
+- `bootconfig` (compiled from `tools/bootconfig/bootconfig` or in PATH)
+- `cpio` (standard archiver)
+
+Required Kernel Configurations
+------------------------------
+The kernel binary under test must be compiled with:
+- `CONFIG_BOOT_CONFIG=y`
+- `CONFIG_BOOTTIME_TRACING=y`
+
+Additional feature-specific config options enable respective testcases:
+- `CONFIG_KPROBE_EVENTS=y` (for `01-kprobe`)
+- `CONFIG_SYNTH_EVENTS=y`  (for `02-synth`)
+- `CONFIG_EPROBE_EVENTS=y` (for `03-eprobe`)
+- `CONFIG_FPROBE_EVENTS=y` (for `04-fprobe` and `05-tprobe`)
+- `CONFIG_FUNCTION_TRACER=y` (for tracer options in `06-instance`)
+
+Usage
+-----
+To run all test cases with a compiled kernel image:
+
+  $ ./run_boottime_test.sh -k /path/to/bzImage
+
+To run a specific test case (e.g. `01-kprobe`):
+
+  $ ./run_boottime_test.sh -k /path/to/bzImage -t 01-kprobe
+
+Customizing Tool Paths:
+
+  $ ./run_boottime_test.sh -b /path/to/bootconfig \
+                         -k /path/to/bzImage \
+                         -q /path/to/qemu-system-x86_64 \
+                         -B /path/to/busybox
+
+Environment Variables
+---------------------
+Tool locations and kernel paths can also be set via environment variables:
+- `BOOTCONFIG`: Path to the `bootconfig` tool
+- `KERNEL`: Path to the `bzImage` kernel binary
+- `QEMU`: Path to the QEMU binary
+- `BUSYBOX`: Path to the `busybox` binary
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf
new file mode 100644
index 000000000000..f607840e3858
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf
@@ -0,0 +1,5 @@
+ftrace.event.kprobes.vfs_read {
+	probes = "vfs_read $arg1 $arg2"
+	filter = "common_pid < 200"
+	enable
+}
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf
new file mode 100644
index 000000000000..b1f37713f33c
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf
@@ -0,0 +1,4 @@
+ftrace.event.synthetic.boot_lat {
+	fields = "unsigned long id", "u64 delta"
+	enable
+}
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf
new file mode 100644
index 000000000000..f954aa7fb8bd
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf
@@ -0,0 +1,4 @@
+ftrace.event.eprobes.ep_read {
+	probes = "syscalls/sys_enter_openat"
+	enable
+}
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf
new file mode 100644
index 000000000000..82ae2ac0629f
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf
@@ -0,0 +1,4 @@
+ftrace.event.fprobes.fp_read {
+	probes = "vfs_read"
+	enable
+}
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf
new file mode 100644
index 000000000000..02a2803c770a
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf
@@ -0,0 +1,4 @@
+ftrace.event.tracepoints.tp_sched {
+	probes = "sched_switch"
+	enable
+}
diff --git a/tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf b/tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf
new file mode 100644
index 000000000000..31f81da570ea
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf
@@ -0,0 +1,5 @@
+ftrace.instance.foo {
+	event.sched.sched_switch.enable
+	buffer_size = 1MB
+	tracing_on = 1
+}
diff --git a/tools/testing/selftests/ftrace/boottime/run_boottime_test.sh b/tools/testing/selftests/ftrace/boottime/run_boottime_test.sh
new file mode 100755
index 000000000000..0bf95a47c5d6
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/run_boottime_test.sh
@@ -0,0 +1,218 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Boot-time tracing test runner
+# Builds a lightweight initramfs, attaches a bootconfig, boots QEMU,
+# and verifies tracefs configuration using test scripts.
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+KERNEL_SRC="$(cd "$SCRIPT_DIR/../../../../.." && pwd)"
+
+# Source KTAP helpers
+KSELFTEST_DIR="$(cd "$SCRIPT_DIR/../../kselftest" && pwd)"
+if [ -f "$KSELFTEST_DIR/ktap_helpers.sh" ]; then
+	. "$KSELFTEST_DIR/ktap_helpers.sh"
+else
+	echo "Error: ktap_helpers.sh not found at $KSELFTEST_DIR"
+	exit 1
+fi
+
+BOOTCONFIG="${BOOTCONFIG:-"$KERNEL_SRC/tools/bootconfig/bootconfig"}"
+KERNEL="${KERNEL:-"$KERNEL_SRC/arch/x86/boot/bzImage"}"
+QEMU="${QEMU:-"qemu-system-x86_64"}"
+BUSYBOX="${BUSYBOX:-"$(which busybox 2>/dev/null || echo "/usr/bin/busybox")"}"
+TARGET_TEST="all"
+
+usage() {
+	echo "Usage: $0 [options]"
+	echo "Options:"
+	echo "  -b, --bootconfig PATH   Path to bootconfig tool (default: $BOOTCONFIG)"
+	echo "  -k, --kernel PATH       Path to kernel image (default: $KERNEL)"
+	echo "  -q, --qemu PATH         Path to QEMU executable (default: $QEMU)"
+	echo "  -B, --busybox PATH      Path to busybox binary (default: $BUSYBOX)"
+	echo "  -t, --test NAME         Run specific test (e.g. 01-kprobe) or 'all'"
+	echo "  -h, --help              Show this help message"
+	exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+	case "$1" in
+		-b|--bootconfig)
+			BOOTCONFIG="$2"
+			shift 2
+			;;
+		-k|--kernel)
+			KERNEL="$2"
+			shift 2
+			;;
+		-q|--qemu)
+			QEMU="$2"
+			shift 2
+			;;
+		-B|--busybox)
+			BUSYBOX="$2"
+			shift 2
+			;;
+		-t|--test)
+			TARGET_TEST="$2"
+			shift 2
+			;;
+		-h|--help)
+			usage
+			;;
+		*)
+			echo "Unknown option: $1"
+			usage
+			;;
+	esac
+done
+
+# Check host architecture
+HOST_ARCH="$(uname -m)"
+case "$HOST_ARCH" in
+	x86_64|i686|i386|x86)
+		;;
+	*)
+		ktap_print_header
+		ktap_skip_all \
+			"Boot-time tracing tests currently support x86 host architecture only ($HOST_ARCH)"
+		exit 0
+		;;
+esac
+
+# Ensure bootconfig tool is built if Makefile exists
+if [ ! -x "$BOOTCONFIG" ]; then
+	if [ -f "$KERNEL_SRC/tools/bootconfig/Makefile" ]; then
+		make -C "$KERNEL_SRC/tools/bootconfig" > /dev/null 2>&1 || true
+	fi
+	if [ ! -x "$BOOTCONFIG" ]; then
+		BOOTCONFIG="$(which bootconfig 2>/dev/null || true)"
+	fi
+fi
+
+if [ ! -x "$BOOTCONFIG" ]; then
+	ktap_print_header
+	ktap_skip_all "bootconfig tool not found at $BOOTCONFIG"
+	exit 0
+fi
+
+if [ ! -f "$KERNEL" ]; then
+	ktap_print_header
+	ktap_skip_all "Kernel image not found at $KERNEL"
+	exit 0
+fi
+
+if ! command -v "$QEMU" >/dev/null 2>&1; then
+	ktap_print_header
+	ktap_skip_all "QEMU binary not found: $QEMU"
+	exit 0
+fi
+
+if [ ! -x "$BUSYBOX" ]; then
+	ktap_print_header
+	ktap_skip_all "busybox binary not found at $BUSYBOX"
+	exit 0
+fi
+
+run_single_test() {
+	local bconf_file="$1"
+	local test_script="$2"
+	local name="$(basename "$bconf_file" .bconf)"
+
+	local workdir="$(mktemp -d)"
+	trap 'rm -rf "$workdir"' EXIT
+
+	local rootfs="$workdir/rootfs"
+	mkdir -p "$rootfs"/{bin,sbin,etc,proc,sys,dev,tmp}
+
+	cp "$BUSYBOX" "$rootfs/bin/busybox"
+	chmod +x "$rootfs/bin/busybox"
+	(cd "$rootfs/bin" && for applet in sh cat mount umount echo sync reboot \
+		poweroff cpio grep; do ln -s busybox $applet; done)
+
+	cat << 'EOF' > "$rootfs/init"
+#!/bin/sh
+mount -t proc proc /proc 2>/dev/null
+mount -t sysfs sys /sys 2>/dev/null
+mount -t tracefs nodev /sys/kernel/tracing 2>/dev/null
+
+/bin/check_test.sh
+RET=$?
+
+if [ $RET -eq 0 ]; then
+	echo "TEST RESULT: PASS"
+else
+	echo "TEST RESULT: FAIL"
+fi
+
+sync
+echo o > /proc/sysrq-trigger 2>/dev/null || reboot -f 2>/dev/null || poweroff -f 2>/dev/null
+EOF
+	chmod +x "$rootfs/init"
+
+	cp "$test_script" "$rootfs/bin/check_test.sh"
+	chmod +x "$rootfs/bin/check_test.sh"
+
+	local initramfs="$workdir/initramfs.cpio"
+	(cd "$rootfs" && find . | cpio -o -H newc --quiet) > "$initramfs"
+
+	# Attach bootconfig
+	"$BOOTCONFIG" -a "$bconf_file" "$initramfs" > /dev/null
+
+	# Run QEMU
+	local logfile="$workdir/qemu.log"
+	"$QEMU" -kernel "$KERNEL" \
+		-initrd "$initramfs" \
+		-append "bootconfig console=ttyS0 quiet panic=-1" \
+		-nographic \
+		-no-reboot > "$logfile" 2>&1 || true
+
+	if grep -q "TEST RESULT: PASS" "$logfile"; then
+		ktap_test_pass "$name"
+		rm -rf "$workdir"
+		trap - EXIT
+		return 0
+	else
+		ktap_test_fail "$name"
+		ktap_print_msg "QEMU Console Output for $name:"
+		while IFS= read -r line; do
+			ktap_print_msg "$line"
+		done < "$logfile"
+		rm -rf "$workdir"
+		trap - EXIT
+		return 1
+	fi
+}
+
+TEST_FILES=("$SCRIPT_DIR/bootconfigs"/*.bconf)
+TOTAL_TESTS=0
+for bconf in "${TEST_FILES[@]}"; do
+	name="$(basename "$bconf" .bconf)"
+	if [ "$TARGET_TEST" != "all" ] && [ "$TARGET_TEST" != "$name" ]; then
+		continue
+	fi
+	TOTAL_TESTS=$((TOTAL_TESTS + 1))
+done
+
+ktap_print_header
+ktap_set_plan "$TOTAL_TESTS"
+
+for bconf in "${TEST_FILES[@]}"; do
+	name="$(basename "$bconf" .bconf)"
+	test_script="$SCRIPT_DIR/tests/$name.sh"
+
+	if [ "$TARGET_TEST" != "all" ] && [ "$TARGET_TEST" != "$name" ]; then
+		continue
+	fi
+
+	if [ ! -f "$test_script" ]; then
+		ktap_test_skip "$name (test script missing)"
+		continue
+	fi
+
+	run_single_test "$bconf" "$test_script" || true
+done
+
+ktap_finished
diff --git a/tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh b/tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh
new file mode 100755
index 000000000000..a20e7e82d158
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check kprobe bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/events/kprobes/vfs_read" ]; then
+	echo "FAIL: kprobe event kprobes/vfs_read does not exist"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/events/kprobes/vfs_read/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: kprobe event kprobes/vfs_read is not enabled ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 01-kprobe"
+exit 0
diff --git a/tools/testing/selftests/ftrace/boottime/tests/02-synth.sh b/tools/testing/selftests/ftrace/boottime/tests/02-synth.sh
new file mode 100755
index 000000000000..46d3b152cbea
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/02-synth.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check synthetic event bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/events/synthetic/boot_lat" ]; then
+	echo "FAIL: synthetic event synthetic/boot_lat does not exist"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/events/synthetic/boot_lat/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: synthetic event synthetic/boot_lat is not enabled ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 02-synth"
+exit 0
diff --git a/tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh b/tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh
new file mode 100755
index 000000000000..108f9ba31c06
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check eprobe event bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/events/eprobes/ep_read" ]; then
+	echo "FAIL: eprobe event eprobes/ep_read does not exist"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/events/eprobes/ep_read/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: eprobe event eprobes/ep_read is not enabled ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 03-eprobe"
+exit 0
diff --git a/tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh b/tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh
new file mode 100755
index 000000000000..c0a316bf7dde
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check fprobe event bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/events/fprobes/fp_read" ]; then
+	echo "FAIL: fprobe event fprobes/fp_read does not exist"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/events/fprobes/fp_read/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: fprobe event fprobes/fp_read is not enabled ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 04-fprobe"
+exit 0
diff --git a/tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh b/tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh
new file mode 100755
index 000000000000..b29119b81e07
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check tracepoint probe event bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/events/tracepoints/tp_sched" ]; then
+	echo "FAIL: tracepoint probe event tracepoints/tp_sched does not exist"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/events/tracepoints/tp_sched/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: tracepoint probe event tracepoints/tp_sched is not enabled ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 05-tprobe"
+exit 0
diff --git a/tools/testing/selftests/ftrace/boottime/tests/06-instance.sh b/tools/testing/selftests/ftrace/boottime/tests/06-instance.sh
new file mode 100755
index 000000000000..a95fdddaa6a6
--- /dev/null
+++ b/tools/testing/selftests/ftrace/boottime/tests/06-instance.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Check instance bootconfig settings on tracefs
+TRACEDIR="/sys/kernel/tracing"
+
+if [ ! -d "$TRACEDIR/instances/foo" ]; then
+	echo "FAIL: trace instance foo does not exist"
+	exit 1
+fi
+
+if [ ! -d "$TRACEDIR/instances/foo/events/sched/sched_switch" ]; then
+	echo "FAIL: event sched_switch does not exist in instance foo"
+	exit 1
+fi
+
+ENABLE=$(cat "$TRACEDIR/instances/foo/events/sched/sched_switch/enable")
+if [ "$ENABLE" != "1" ]; then
+	echo "FAIL: event sched_switch is not enabled in instance foo ($ENABLE)"
+	exit 1
+fi
+
+echo "PASS: 06-instance"
+exit 0
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index 544de0db5f58..fec41c6b290e 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -1,3 +1,5 @@
+CONFIG_BOOT_CONFIG=y
+CONFIG_BOOTTIME_TRACING=y
 CONFIG_BPF_SYSCALL=y
 CONFIG_DEBUG_INFO_BTF=y
 CONFIG_DEBUG_INFO_DWARF4=y
@@ -8,6 +10,7 @@ CONFIG_FTRACE=y
 CONFIG_FTRACE_SYSCALLS=y
 CONFIG_FUNCTION_GRAPH_RETVAL=y
 CONFIG_FUNCTION_PROFILER=y
+CONFIG_FUNCTION_TRACER=y
 CONFIG_HIST_TRIGGERS=y
 CONFIG_IRQSOFF_TRACER=y
 CONFIG_KALLSYMS_ALL=y
@@ -24,6 +27,7 @@ CONFIG_SAMPLE_TRACE_EVENTS=m
 CONFIG_SAMPLE_TRACE_PRINTK=m
 CONFIG_SCHED_TRACER=y
 CONFIG_STACK_TRACER=y
+CONFIG_SYNTH_EVENTS=y
 CONFIG_TRACER_SNAPSHOT=y
 CONFIG_UPROBES=y
 CONFIG_UPROBE_EVENTS=y

Re: [PATCH 1/2] tracing/boot: Add support for eprobe, fprobe, and tprobe events

[Masami Hiramatsu]

On Thu,  6 Aug 2026 10:07:01 +0900
"Masami Hiramatsu (Google)" <mhiramat@kernel.org> wrote:

> From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> 
> Boot-time tracing currently supports kprobe-events and synthetic-events
> under per-event configuration options.
> 
> Extend boot-time tracing to support newly added dynamic probe types:
> - event probes (eprobe) under the "eprobes" event group
> - function probes (fprobe) under the "fprobes" event group
> - tracepoint probes (tprobe) under the "tracepoints" or "tprobes"
>   event group
> 
> To support this cleanly, update dyn_event_create() in trace_dynevent.c
> so that passing NULL as the type parameter delegates to
> create_dyn_event(), allowing generic creation of any registered
> dynamic event type from a raw command string.
> 
> Update Documentation/trace/boottime-trace.rst accordingly to describe
> the new per-event bootconfig options.
> 
> Assisted-by: Antigravity:gemini-3.6-flash
> Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> ---
>  Documentation/trace/boottime-trace.rst |   18 ++++-
>  kernel/trace/trace_boot.c              |  113 ++++++++++++++++++++++++++++++++
>  kernel/trace/trace_dynevent.c          |    5 +
>  3 files changed, 133 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/trace/boottime-trace.rst b/Documentation/trace/boottime-trace.rst
> index 651f3a2c01de..2c3c1fedb92c 100644
> --- a/Documentation/trace/boottime-trace.rst
> +++ b/Documentation/trace/boottime-trace.rst
> @@ -121,9 +121,21 @@ ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
>     Defines new synthetic event with FIELDs. Each field should be
>     "type varname".
>  
> -Note that kprobe and synthetic event definitions can be written under
> -instance node, but those are also visible from other instances. So please
> -take care for event name conflict.
> +ftrace.[instance.INSTANCE.]event.eprobes.EVENT.probes = PROBE[, PROBE2[...]]
> +   Defines new event probe based on PROBEs. This option is available only
> +   for the event which group name is "eprobes".
> +
> +ftrace.[instance.INSTANCE.]event.fprobes.EVENT.probes = PROBE[, PROBE2[...]]
> +   Defines new fprobe event based on PROBEs. This option is available only
> +   for the event which group name is "fprobes".
> +
> +ftrace.[instance.INSTANCE.]event.tracepoints.EVENT.probes = PROBE[, PROBE2[...]]
> +   Defines new tracepoint probe based on PROBEs. This option is available only
> +   for the event which group name is "tracepoints" or "tprobes".
> +
> +Note that dynamic event definitions can be written under instance node, but
> +those are also visible from other instances. So please take care for event
> +name conflict.
>  
>  Ftrace Histogram Options
>  ------------------------
> diff --git a/kernel/trace/trace_boot.c b/kernel/trace/trace_boot.c
> index 2ca2541c8a58..6ecf1a035caa 100644
> --- a/kernel/trace/trace_boot.c
> +++ b/kernel/trace/trace_boot.c
> @@ -18,6 +18,7 @@
>  #include <linux/trace_events.h>
>  
>  #include "trace.h"
> +#include "trace_dynevent.h"
>  
>  #define MAX_BUF_LEN 256
>  
> @@ -172,6 +173,109 @@ trace_boot_add_synth_event(struct xbc_node *node, const char *event)
>  }
>  #endif
>  
> +#ifdef CONFIG_EPROBE_EVENTS
> +static int __init
> +trace_boot_add_eprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	struct xbc_node *anode;
> +	char buf[MAX_BUF_LEN];
> +	const char *val;
> +	int ret = 0;
> +
> +	xbc_node_for_each_array_value(node, "probes", anode, val) {
> +		if (val[0] == 'e' && (val[1] == ':' || isdigit(val[1])))

Oops, I missed it, this isdigit is only required kretprobes...

Let me fix it.

Thank you,

> +			strscpy(buf, val, MAX_BUF_LEN);
> +		else
> +			snprintf(buf, MAX_BUF_LEN, "e:%s/%s %s", group, event, val);
> +
> +		ret = dyn_event_create(buf, NULL);
> +		if (ret) {
> +			pr_err("Failed to add eprobe: %s\n", buf);
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +#else
> +static inline int __init
> +trace_boot_add_eprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	pr_err("Event probe is not supported.\n");
> +	return -EOPNOTSUPP;
> +}
> +#endif
> +
> +#ifdef CONFIG_FPROBE_EVENTS
> +static int __init
> +trace_boot_add_fprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	struct xbc_node *anode;
> +	char buf[MAX_BUF_LEN];
> +	const char *val;
> +	int ret = 0;
> +
> +	xbc_node_for_each_array_value(node, "probes", anode, val) {
> +		if (val[0] == 'f' && (val[1] == ':' || isdigit(val[1])))
> +			strscpy(buf, val, MAX_BUF_LEN);
> +		else
> +			snprintf(buf, MAX_BUF_LEN, "f:%s/%s %s", group, event, val);
> +
> +		ret = dyn_event_create(buf, NULL);
> +		if (ret) {
> +			pr_err("Failed to add fprobe: %s\n", buf);
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +static int __init
> +trace_boot_add_tprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	struct xbc_node *anode;
> +	char buf[MAX_BUF_LEN];
> +	const char *val;
> +	int ret = 0;
> +
> +	xbc_node_for_each_array_value(node, "probes", anode, val) {
> +		if (val[0] == 't' && (val[1] == ':' || isdigit(val[1])))
> +			strscpy(buf, val, MAX_BUF_LEN);
> +		else
> +			snprintf(buf, MAX_BUF_LEN, "t:%s/%s %s", group, event, val);
> +
> +		ret = dyn_event_create(buf, NULL);
> +		if (ret) {
> +			pr_err("Failed to add tprobe: %s\n", buf);
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +#else
> +static inline int __init
> +trace_boot_add_fprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	pr_err("Fprobe event is not supported.\n");
> +	return -EOPNOTSUPP;
> +}
> +
> +static inline int __init
> +trace_boot_add_tprobe_event(struct xbc_node *node, const char *group,
> +			    const char *event)
> +{
> +	pr_err("Tracepoint probe is not supported.\n");
> +	return -EOPNOTSUPP;
> +}
> +#endif
> +
>  #ifdef CONFIG_HIST_TRIGGERS
>  static int __init __printf(3, 4)
>  append_printf(char **bufp, char *end, const char *fmt, ...)
> @@ -477,6 +581,15 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,
>  	if (!strcmp(group, "synthetic"))
>  		if (trace_boot_add_synth_event(enode, event) < 0)
>  			return;
> +	if (!strcmp(group, "eprobes"))
> +		if (trace_boot_add_eprobe_event(enode, group, event) < 0)
> +			return;
> +	if (!strcmp(group, "fprobes"))
> +		if (trace_boot_add_fprobe_event(enode, group, event) < 0)
> +			return;
> +	if (!strcmp(group, "tracepoints") || !strcmp(group, "tprobes"))
> +		if (trace_boot_add_tprobe_event(enode, group, event) < 0)
> +			return;
>  
>  	mutex_lock(&event_mutex);
>  	file = find_event_file(tr, group, event);
> diff --git a/kernel/trace/trace_dynevent.c b/kernel/trace/trace_dynevent.c
> index c4dfbc293bae..6f2e39b797e5 100644
> --- a/kernel/trace/trace_dynevent.c
> +++ b/kernel/trace/trace_dynevent.c
> @@ -116,6 +116,8 @@ int dyn_event_release(const char *raw_command, struct dyn_event_operations *type
>  	return ret;
>  }
>  
> +static int create_dyn_event(const char *raw_command);
> +
>  /*
>   * Locked version of event creation. The event creation must be protected by
>   * dyn_event_ops_mutex because of protecting trace_probe_log.
> @@ -124,6 +126,9 @@ int dyn_event_create(const char *raw_command, struct dyn_event_operations *type)
>  {
>  	int ret;
>  
> +	if (!type)
> +		return create_dyn_event(raw_command);
> +
>  	mutex_lock(&dyn_event_ops_mutex);
>  	ret = type->create(raw_command);
>  	mutex_unlock(&dyn_event_ops_mutex);
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>