Linux Trace Kernel
 help / color / mirror / Atom feed
* [PATCH v3 1/5] tracing: Fix checking of freed trace_event_file for hist files
From: Petr Pavlu @ 2026-02-19 16:27 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Mathieu Desnoyers, Tom Zanussi, linux-kernel, linux-trace-kernel,
	Petr Pavlu
In-Reply-To: <20260219162737.314231-1-petr.pavlu@suse.com>

The event_hist_open() and event_hist_poll() functions currently retrieve
a trace_event_file pointer from a file struct by invoking
event_file_data(), which simply returns file->f_inode->i_private. The
functions then check if the pointer is NULL to determine whether the event
is still valid. This approach is flawed because i_private is assigned when
an eventfs inode is allocated and remains set throughout its lifetime.
Instead, the code should call event_file_file(), which checks for
EVENT_FILE_FL_FREED. Using the incorrect access function may result in the
code potentially opening a hist file for an event that is being removed or
becoming stuck while polling on this file.

Correct the access method to event_file_file() in both functions.

Fixes: 1bd13edbbed6 ("tracing/hist: Add poll(POLLIN) support on hist file")
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
 kernel/trace/trace_events_hist.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index e6f449f53afc..768df987419e 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -5784,7 +5784,7 @@ static __poll_t event_hist_poll(struct file *file, struct poll_table_struct *wai
 
 	guard(mutex)(&event_mutex);
 
-	event_file = event_file_data(file);
+	event_file = event_file_file(file);
 	if (!event_file)
 		return EPOLLERR;
 
@@ -5822,7 +5822,7 @@ static int event_hist_open(struct inode *inode, struct file *file)
 
 	guard(mutex)(&event_mutex);
 
-	event_file = event_file_data(file);
+	event_file = event_file_file(file);
 	if (!event_file) {
 		ret = -ENODEV;
 		goto err;
-- 
2.53.0


^ permalink raw reply related
* [PATCH v3 2/5] tracing: Wake up poll waiters for hist files when removing an event
From: Petr Pavlu @ 2026-02-19 16:27 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Mathieu Desnoyers, Tom Zanussi, linux-kernel, linux-trace-kernel,
	Petr Pavlu
In-Reply-To: <20260219162737.314231-1-petr.pavlu@suse.com>

The event_hist_poll() function attempts to verify whether an event file is
being removed, but this check may not occur or could be unnecessarily
delayed. This happens because hist_poll_wakeup() is currently invoked only
from event_hist_trigger() when a hist command is triggered. If the event
file is being removed, no associated hist command will be triggered and a
waiter will be woken up only after an unrelated hist command is triggered.

Fix the issue by adding a call to hist_poll_wakeup() in
remove_event_file_dir() after setting the EVENT_FILE_FL_FREED flag. This
ensures that a task polling on a hist file is woken up and receives
EPOLLERR.

Fixes: 1bd13edbbed6 ("tracing/hist: Add poll(POLLIN) support on hist file")
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
 include/linux/trace_events.h | 5 +++++
 kernel/trace/trace_events.c  | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 0a2b8229b999..37eb2f0f3dd8 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -683,6 +683,11 @@ static inline void hist_poll_wakeup(void)
 
 #define hist_poll_wait(file, wait)	\
 	poll_wait(file, &hist_poll_wq, wait)
+
+#else
+static inline void hist_poll_wakeup(void)
+{
+}
 #endif
 
 #define __TRACE_EVENT_FLAGS(name, value)				\
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 61fe01dce7a6..b659653dc03a 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -1311,6 +1311,9 @@ static void remove_event_file_dir(struct trace_event_file *file)
 	free_event_filter(file->filter);
 	file->flags |= EVENT_FILE_FL_FREED;
 	event_file_put(file);
+
+	/* Wake up hist poll waiters to notice the EVENT_FILE_FL_FREED flag. */
+	hist_poll_wakeup();
 }
 
 /*
-- 
2.53.0


^ permalink raw reply related
* [PATCH v3 3/5] tracing: Remove unnecessary check for EVENT_FILE_FL_FREED
From: Petr Pavlu @ 2026-02-19 16:27 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Mathieu Desnoyers, Tom Zanussi, linux-kernel, linux-trace-kernel,
	Petr Pavlu
In-Reply-To: <20260219162737.314231-1-petr.pavlu@suse.com>

The event_filter_write() function calls event_file_file() to retrieve
a trace_event_file associated with a given file struct. If a non-NULL
pointer is returned, the function then checks whether the trace_event_file
instance has the EVENT_FILE_FL_FREED flag set. This check is redundant
because event_file_file() already performs this validation and returns NULL
if the flag is set. The err value is also already initialized to -ENODEV.

Remove the unnecessary check for EVENT_FILE_FL_FREED in
event_filter_write().

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
---
 kernel/trace/trace_events.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index b659653dc03a..90915e1286da 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -2245,12 +2245,8 @@ event_filter_write(struct file *filp, const char __user *ubuf, size_t cnt,
 
 	mutex_lock(&event_mutex);
 	file = event_file_file(filp);
-	if (file) {
-		if (file->flags & EVENT_FILE_FL_FREED)
-			err = -ENODEV;
-		else
-			err = apply_event_filter(file, buf);
-	}
+	if (file)
+		err = apply_event_filter(file, buf);
 	mutex_unlock(&event_mutex);
 
 	kfree(buf);
-- 
2.53.0


^ permalink raw reply related
* [PATCH v3 4/5] tracing: Clean up access to trace_event_file from a file pointer
From: Petr Pavlu @ 2026-02-19 16:27 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Mathieu Desnoyers, Tom Zanussi, linux-kernel, linux-trace-kernel,
	Petr Pavlu
In-Reply-To: <20260219162737.314231-1-petr.pavlu@suse.com>

The tracing code provides two functions event_file_file() and
event_file_data() to obtain a trace_event_file pointer from a file struct.
The primary method to use is event_file_file(), as it checks for the
EVENT_FILE_FL_FREED flag to determine whether the event is being removed.
The second function event_file_data() is an optimization for retrieving the
same data when the event_mutex is still held.

In the past, when removing an event directory in remove_event_file_dir(),
the code set i_private to NULL for all event files and readers were
expected to check for this state to recognize that the event is being
removed. In the case of event_id_read(), the value was read using
event_file_data() without acquiring the event_mutex. This required
event_file_data() to use READ_ONCE() when retrieving the i_private data.

With the introduction of eventfs, i_private is assigned when an eventfs
inode is allocated and remains set throughout its lifetime.

Remove the now unnecessary READ_ONCE() access to i_private in both
event_file_file() and event_file_data(). Inline the access to i_private in
remove_event_file_dir(), which allows event_file_data() to handle i_private
solely as a trace_event_file pointer. Add a check in event_file_data() to
ensure that the event_mutex is held and that file->flags doesn't have the
EVENT_FILE_FL_FREED flag set. Finally, move event_file_data() immediately
after event_file_code() since the latter provides a comment explaining how
both functions should be used together.

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
---
 kernel/trace/trace.h        | 17 +++++++++++------
 kernel/trace/trace_events.c |  6 +++---
 2 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index b8f3804586a0..7db78a62f786 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1802,11 +1802,6 @@ extern struct trace_event_file *find_event_file(struct trace_array *tr,
 						const char *system,
 						const char *event);
 
-static inline void *event_file_data(struct file *filp)
-{
-	return READ_ONCE(file_inode(filp)->i_private);
-}
-
 extern struct mutex event_mutex;
 extern struct list_head ftrace_events;
 
@@ -1827,12 +1822,22 @@ static inline struct trace_event_file *event_file_file(struct file *filp)
 	struct trace_event_file *file;
 
 	lockdep_assert_held(&event_mutex);
-	file = READ_ONCE(file_inode(filp)->i_private);
+	file = file_inode(filp)->i_private;
 	if (!file || file->flags & EVENT_FILE_FL_FREED)
 		return NULL;
 	return file;
 }
 
+static inline void *event_file_data(struct file *filp)
+{
+	struct trace_event_file *file;
+
+	lockdep_assert_held(&event_mutex);
+	file = file_inode(filp)->i_private;
+	WARN_ON(!file || file->flags & EVENT_FILE_FL_FREED);
+	return file;
+}
+
 extern const struct file_operations event_trigger_fops;
 extern const struct file_operations event_hist_fops;
 extern const struct file_operations event_hist_debug_fops;
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 90915e1286da..c52135cd9265 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -2182,12 +2182,12 @@ static int trace_format_open(struct inode *inode, struct file *file)
 static ssize_t
 event_id_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos)
 {
-	int id = (long)event_file_data(filp);
+	/* id is directly in i_private and available for inode's lifetime. */
+	int id = (long)file_inode(filp)->i_private;
 	char buf[32];
 	int len;
 
-	if (unlikely(!id))
-		return -ENODEV;
+	WARN_ON(!id);
 
 	len = sprintf(buf, "%d\n", id);
 
-- 
2.53.0


^ permalink raw reply related
* [PATCH v3 5/5] tracing: Free up file->private_data for use by individual events
From: Petr Pavlu @ 2026-02-19 16:27 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Mathieu Desnoyers, Tom Zanussi, linux-kernel, linux-trace-kernel,
	Petr Pavlu
In-Reply-To: <20260219162737.314231-1-petr.pavlu@suse.com>

The tracing_open_file_tr() function currently copies the trace_event_file
pointer from inode->i_private to file->private_data when the file is
successfully opened. This duplication is not particularly useful, as all
event code should utilize event_file_file() or event_file_data() to
retrieve a trace_event_file pointer from a file struct and these access
functions read file->f_inode->i_private. Moreover, this setup requires the
code for opening hist files to explicitly clear file->private_data before
calling single_open(), since this function expects the private_data member
to be set to NULL and uses it to store a pointer to a seq_file.

Remove the unnecessary setting of file->private_data in
tracing_open_file_tr() and simplify the hist code.

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
---
 kernel/trace/trace.c             | 2 --
 kernel/trace/trace_events_hist.c | 4 ----
 2 files changed, 6 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 2f6fbf9e7caf..a51c675be0f4 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4051,8 +4051,6 @@ int tracing_open_file_tr(struct inode *inode, struct file *filp)
 		event_file_get(file);
 	}
 
-	filp->private_data = inode->i_private;
-
 	return 0;
 }
 
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 768df987419e..57c76d40be2d 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -5837,8 +5837,6 @@ static int event_hist_open(struct inode *inode, struct file *file)
 	hist_file->file = file;
 	hist_file->last_act = get_hist_hit_count(event_file);
 
-	/* Clear private_data to avoid warning in single_open() */
-	file->private_data = NULL;
 	ret = single_open(file, hist_show, hist_file);
 	if (ret) {
 		kfree(hist_file);
@@ -6127,8 +6125,6 @@ static int event_hist_debug_open(struct inode *inode, struct file *file)
 	if (ret)
 		return ret;
 
-	/* Clear private_data to avoid warning in single_open() */
-	file->private_data = NULL;
 	ret = single_open(file, hist_debug_show, file);
 	if (ret)
 		tracing_release_file_tr(inode, file);
-- 
2.53.0


^ permalink raw reply related
* Re: [PATCH v12 05/30] tracing: Introduce trace remotes
From: Steven Rostedt @ 2026-02-19 17:55 UTC (permalink / raw)
  To: Vincent Donnefort
  Cc: mhiramat, mathieu.desnoyers, linux-trace-kernel, maz,
	oliver.upton, joey.gouly, suzuki.poulose, yuzenghui, kvmarm,
	linux-arm-kernel, jstultz, qperret, will, aneesh.kumar,
	kernel-team, linux-kernel
In-Reply-To: <20260219150307.14538-6-vdonnefort@google.com>

On Thu, 19 Feb 2026 15:02:42 +0000
Vincent Donnefort <vdonnefort@google.com> wrote:

> A trace remote relies on ring-buffer remotes to read and control
> compatible tracing buffers, written by entity such as firmware or
> hypervisor.
> 
> Add a Tracefs directory remotes/ that contains all instances of trace
> remotes. Each instance follows the same hierarchy as any other to ease
> the support by existing user-space tools.
> 
> This currently does not provide any event support, which will come
> later.
> 
> Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve

^ permalink raw reply
* Re: [PATCH v12 07/30] tracing: Add non-consuming read to trace remotes
From: Steven Rostedt @ 2026-02-19 17:55 UTC (permalink / raw)
  To: Vincent Donnefort
  Cc: mhiramat, mathieu.desnoyers, linux-trace-kernel, maz,
	oliver.upton, joey.gouly, suzuki.poulose, yuzenghui, kvmarm,
	linux-arm-kernel, jstultz, qperret, will, aneesh.kumar,
	kernel-team, linux-kernel
In-Reply-To: <20260219150307.14538-8-vdonnefort@google.com>

On Thu, 19 Feb 2026 15:02:44 +0000
Vincent Donnefort <vdonnefort@google.com> wrote:

> Allow reading the trace file for trace remotes. This performs a
> non-consuming read of the trace buffer.
> 
> Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve

^ permalink raw reply
* Re: [PATCH v12 15/30] tracing: selftests: Add trace remote tests
From: Steven Rostedt @ 2026-02-19 17:56 UTC (permalink / raw)
  To: Vincent Donnefort
  Cc: mhiramat, mathieu.desnoyers, linux-trace-kernel, maz,
	oliver.upton, joey.gouly, suzuki.poulose, yuzenghui, kvmarm,
	linux-arm-kernel, jstultz, qperret, will, aneesh.kumar,
	kernel-team, linux-kernel, Shuah Khan, linux-kselftest
In-Reply-To: <20260219150307.14538-16-vdonnefort@google.com>

On Thu, 19 Feb 2026 15:02:52 +0000
Vincent Donnefort <vdonnefort@google.com> wrote:

> Exercise the tracefs interface for trace remote with a set of tests to
> check:
> 
>   * loading/unloading (unloading.tc)
>   * reset (reset.tc)
>   * size changes (buffer_size.tc)
>   * consuming read (trace_pipe.tc)
>   * non-consuming read (trace.tc)
> 
> Cc: Shuah Khan <skhan@linuxfoundation.org>
> Cc: linux-kselftest@vger.kernel.org
> Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve

^ permalink raw reply
* Re: [PATCH v12 00/30] Tracefs support for pKVM
From: Steven Rostedt @ 2026-02-19 18:02 UTC (permalink / raw)
  To: Vincent Donnefort
  Cc: mhiramat, mathieu.desnoyers, linux-trace-kernel, maz,
	oliver.upton, joey.gouly, suzuki.poulose, yuzenghui, kvmarm,
	linux-arm-kernel, jstultz, qperret, will, aneesh.kumar,
	kernel-team, linux-kernel
In-Reply-To: <20260219150307.14538-1-vdonnefort@google.com>

On Thu, 19 Feb 2026 15:02:37 +0000
Vincent Donnefort <vdonnefort@google.com> wrote:

> The growing set of features supported by the hypervisor in protected
> mode necessitates debugging and profiling tools. Tracefs is the
> ideal candidate for this task:
> 
>   * It is simple to use and to script.
> 
>   * It is supported by various tools, from the trace-cmd CLI to the
>     Android web-based perfetto.
> 
>   * The ring-buffer, where are stored trace events consists of linked
>     pages, making it an ideal structure for sharing between kernel and
>     hypervisor.
> 
> This series first introduces a new generic way of creating remote events and
> remote buffers. Then it adds support to the pKVM hypervisor.
> 

So I finished my review of all the tracing patches, but I just realized
that there's likely going to be some major conflicts with changes in the
tracing tree that's going to happen against this series.

To solve that, after -rc2 comes out, I'll apply the tracing portion of this
series to a branch in my tree directly on top of -rc2 and I will then use
that to base my changes for the next merge window.

Then the arm/kvm folks could start with that branch and add the arm/KVM
portion of this series on top of it. This will prevent major merge
conflicts in linux-next.

Does that sound OK?

-- Steve

^ permalink raw reply
* Re: [PATCH] pstore: fix ftrace dump, when ECC is enabled
From: Kees Cook @ 2026-02-19 18:39 UTC (permalink / raw)
  To: Tony Luck, Guilherme G. Piccoli, linux-hardening, linux-kernel,
	Steven Rostedt, Masami Hiramatsu, Mark Rutland,
	linux-trace-kernel, Andrey Skvortsov
  Cc: Kees Cook
In-Reply-To: <20260215185156.317394-1-andrej.skvortzov@gmail.com>

On Sun, 15 Feb 2026 21:51:55 +0300, Andrey Skvortsov wrote:
> total_size is sum of record->size and record->ecc_notice_size (ECC: No
> errors detected). When ECC is not used, then there is no problem.
> When ECC is enabled, then ftrace dump is decoded incorrectly after
> restart.
> 
> First this affects starting offset calculation, that breaks
> reading of all ftrace records.
> 
> [...]

Applied to for-next/pstore, thanks!

[1/1] pstore: fix ftrace dump, when ECC is enabled
      https://git.kernel.org/kees/c/07b7ece8aca0

Take care,

-- 
Kees Cook


^ permalink raw reply
* Re: [PATCH 1/2] x86: Prevent syscall hooking
From: Kees Cook @ 2026-02-19 18:45 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Dave Hansen, Elly I. Esparza, linux-kernel, luto, tglx, mingo, bp,
	dave.hansen, x86, hpa, Naveen N Rao, David S. Miller,
	Masami Hiramatsu, linux-trace-kernel
In-Reply-To: <20260218105204.3af7251e@gandalf.local.home>

On Wed, Feb 18, 2026 at 10:52:04AM -0500, Steven Rostedt wrote:
> Honesty, if you are worried about this, just run LOCKDOWN on tracing, and
> prevent *ALL* kprobes. Because yes, there's a 1000 ways to get this
> information once you have kprobes enabled and have root access. This patch
> is hurting legitimate debugging of running systems more than it is limiting
> rootkits from hacking the kernel.

Yeah, I agree. If kprobes is available, there is a lot of harm an
attacker can already do. If a bright line between root/ring-0 is
desired, a system needs to be configured to be using lockdown or similar
things to turn off the interfaces that let root write to kernel state.

-- 
Kees Cook

^ permalink raw reply
* Re: [PATCH v12 00/30] Tracefs support for pKVM
From: Marc Zyngier @ 2026-02-19 19:11 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Vincent Donnefort, mhiramat, mathieu.desnoyers,
	linux-trace-kernel, oliver.upton, joey.gouly, suzuki.poulose,
	yuzenghui, kvmarm, linux-arm-kernel, jstultz, qperret, will,
	aneesh.kumar, kernel-team, linux-kernel
In-Reply-To: <20260219130220.063c8db3@gandalf.local.home>

On Thu, 19 Feb 2026 18:02:20 +0000,
Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> On Thu, 19 Feb 2026 15:02:37 +0000
> Vincent Donnefort <vdonnefort@google.com> wrote:
> 
> > The growing set of features supported by the hypervisor in protected
> > mode necessitates debugging and profiling tools. Tracefs is the
> > ideal candidate for this task:
> > 
> >   * It is simple to use and to script.
> > 
> >   * It is supported by various tools, from the trace-cmd CLI to the
> >     Android web-based perfetto.
> > 
> >   * The ring-buffer, where are stored trace events consists of linked
> >     pages, making it an ideal structure for sharing between kernel and
> >     hypervisor.
> > 
> > This series first introduces a new generic way of creating remote events and
> > remote buffers. Then it adds support to the pKVM hypervisor.
> > 
> 
> So I finished my review of all the tracing patches, but I just realized
> that there's likely going to be some major conflicts with changes in the
> tracing tree that's going to happen against this series.
> 
> To solve that, after -rc2 comes out, I'll apply the tracing portion of this
> series to a branch in my tree directly on top of -rc2 and I will then use
> that to base my changes for the next merge window.
> 
> Then the arm/kvm folks could start with that branch and add the arm/KVM
> portion of this series on top of it. This will prevent major merge
> conflicts in linux-next.
> 
> Does that sound OK?

That works. Just send us a link to the branch after -rc2 and we'll get
that sorted.

Thanks,

	M.

-- 
Jazz isn't dead. It just smells funny.

^ permalink raw reply
* Re: [PATCH 1/2] x86: Prevent syscall hooking
From: H. Peter Anvin @ 2026-02-19 21:51 UTC (permalink / raw)
  To: Peter Zijlstra, Dave Hansen
  Cc: Elly I. Esparza, linux-kernel, luto, tglx, mingo, bp, dave.hansen,
	x86, Naveen N Rao, David S. Miller, Masami Hiramatsu,
	linux-trace-kernel, Kees Cook
In-Reply-To: <20260218153244.GG1282955@noisy.programming.kicks-ass.net>

On February 18, 2026 7:32:44 AM PST, Peter Zijlstra <peterz@infradead.org> wrote:
>On Wed, Feb 18, 2026 at 07:18:25AM -0800, Dave Hansen wrote:
>> ... adding kprobes folks and Kees to cc
>> 
>> On 2/18/26 06:47, Elly I. Esparza wrote:
>> > Kprobes can be used by rootkits to find the address of x64_sys_call(),
>> > x32_sys_call() and ia32_sys_call(). This in turn allows for the rootkits
>> > to find an specific syscall handler and hook it.
>> > 
>> > Add x64_sys_call(), x32_sys_call() and ia32_sys_call() to the kprobes
>> > blacklist.
>> I'm an occasional, but not super regular kprobes user. Is this going to
>> hurt folks who are legitimately probing the syscall dispatch functions?
>> 
>> I'm a bit worried that the rootkits will just move on to something else
>> and this will become a never ending game of whack-a-mole where half the
>> kernel needs NOKPROBE_SYMBOL(). ;)
>
>So I really think this should be noinstr; pretty much all the code here
>is noinstr already, so why not include the syscall dispatch.
>
>Better still, noinstr ensures the spectre-v1 mitigation actually works.

Yes, and merging the x64 and x32 dispatches into one function actually enables a lot of code sharing.

^ permalink raw reply
* [PATCH] mm/tracing: rss_stat: Ensure curr is false from kthread context
From: Kalesh Singh @ 2026-02-19 23:36 UTC (permalink / raw)
  To: akpm, rostedt, joel
  Cc: kernel-team, android-mm, Kalesh Singh, David Hildenbrand (Arm),
	Lorenzo Stoakes, Minchan Kim, Suren Baghdasaryan, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Masami Hiramatsu,
	Mathieu Desnoyers, Jann Horn, Pedro Falcato, Martin Liu,
	David Rientjes, Zi Yan, Wander Lairson Costa, Petr Mladek,
	linux-mm, linux-kernel, linux-trace-kernel

The rss_stat trace event allows userspace tools, like Perfetto [1],
to inspect per-process RSS metric changes over time.

The curr field was introduced to rss_stat in commit e4dcad204d3a
("rss_stat: add support to detect RSS updates of external mm").
It's intent is to  indicate whether the RSS update is for the
mm_struct of the current execution context; and is set to false
when operating on a remote mm_struct (e.g., via kswapd or a
direct reclaimer).

However, an issue arises when a kernel thread temporarily adopts
a user process's mm_struct. Kernel threads do not have their own
mm_struct and normally have current->mm set to NULL. To operate
on user memory, they can "borrow" a memory context using
kthread_use_mm(), which sets current->mm to the user process's mm.

This can be observed, for example, in the USB Function Filesystem
(FFS) driver. The ffs_user_copy_worker() handles AIO completions
and uses kthread_use_mm() to copy data to a user-space buffer.
If a page fault occurs during this copy, the fault handler executes
in the kthread's context.

At this point, current is the kthread, but current->mm points to the
user process's mm. Since the rss_stat event (from the page fault)
is for that same mm, the condition current->mm == mm becomes true,
causing curr to be incorrectly set to true when the trace event is
emitted.

This is misleading because it suggests the mm belongs to the kthread,
confusing userspace tools that track per-process RSS changes and
corrupting their mm_id-to-process association.

Fix this by ensuring curr is always false when the trace event is
emitted from a kthread context by checking for the PF_KTHREAD flag.

[1] https://perfetto.dev/

Fixes: e4dcad204d3a ("rss_stat: add support to detect RSS updates of external mm")
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "David Hildenbrand (Arm)" <david@kernel.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
---
 include/trace/events/kmem.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h
index 7f93e754da5c..cd7920c81f85 100644
--- a/include/trace/events/kmem.h
+++ b/include/trace/events/kmem.h
@@ -440,7 +440,13 @@ TRACE_EVENT(rss_stat,
 
 	TP_fast_assign(
 		__entry->mm_id = mm_ptr_to_hash(mm);
-		__entry->curr = !!(current->mm == mm);
+		/*
+		 * curr is true if the mm matches the current task's mm_struct.
+		 * Since kthreads (PF_KTHREAD) have no mm_struct of their own
+		 * but can borrow one via kthread_use_mm(), we must filter them
+		 * out to avoid incorrectly attributing the RSS update to them.
+		 */
+		__entry->curr = current->mm == mm && !(current->flags & PF_KTHREAD);
 		__entry->member = member;
 		__entry->size = (percpu_counter_sum_positive(&mm->rss_stat[member])
 							    << PAGE_SHIFT);

base-commit: 8bf22c33e7a172fbc72464f4cc484d23a6b412ba
-- 
2.53.0.371.g1d285c8824-goog


^ permalink raw reply related
* Re: [PATCH] mm/tracing: rss_stat: Ensure curr is false from kthread context
From: Zi Yan @ 2026-02-20  0:52 UTC (permalink / raw)
  To: Kalesh Singh
  Cc: akpm, rostedt, joel, kernel-team, android-mm,
	David Hildenbrand (Arm), Lorenzo Stoakes, Minchan Kim,
	Suren Baghdasaryan, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Masami Hiramatsu, Mathieu Desnoyers,
	Jann Horn, Pedro Falcato, Martin Liu, David Rientjes,
	Wander Lairson Costa, Petr Mladek, linux-mm, linux-kernel,
	linux-trace-kernel
In-Reply-To: <20260219233708.1971199-1-kaleshsingh@google.com>

On 19 Feb 2026, at 18:36, Kalesh Singh wrote:

> The rss_stat trace event allows userspace tools, like Perfetto [1],
> to inspect per-process RSS metric changes over time.
>
> The curr field was introduced to rss_stat in commit e4dcad204d3a
> ("rss_stat: add support to detect RSS updates of external mm").
> It's intent is to  indicate whether the RSS update is for the
> mm_struct of the current execution context; and is set to false
> when operating on a remote mm_struct (e.g., via kswapd or a
> direct reclaimer).
>
> However, an issue arises when a kernel thread temporarily adopts
> a user process's mm_struct. Kernel threads do not have their own
> mm_struct and normally have current->mm set to NULL. To operate
> on user memory, they can "borrow" a memory context using
> kthread_use_mm(), which sets current->mm to the user process's mm.
>
> This can be observed, for example, in the USB Function Filesystem
> (FFS) driver. The ffs_user_copy_worker() handles AIO completions
> and uses kthread_use_mm() to copy data to a user-space buffer.
> If a page fault occurs during this copy, the fault handler executes
> in the kthread's context.
>
> At this point, current is the kthread, but current->mm points to the
> user process's mm. Since the rss_stat event (from the page fault)
> is for that same mm, the condition current->mm == mm becomes true,
> causing curr to be incorrectly set to true when the trace event is
> emitted.
>
> This is misleading because it suggests the mm belongs to the kthread,
> confusing userspace tools that track per-process RSS changes and
> corrupting their mm_id-to-process association.
>
> Fix this by ensuring curr is always false when the trace event is
> emitted from a kthread context by checking for the PF_KTHREAD flag.
>
> [1] https://perfetto.dev/
>
> Fixes: e4dcad204d3a ("rss_stat: add support to detect RSS updates of external mm")
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "David Hildenbrand (Arm)" <david@kernel.org>
> Cc: Joel Fernandes <joel@joelfernandes.org>
> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
> ---
>  include/trace/events/kmem.h | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
LGTM.

Acked-by: Zi Yan <ziy@nvidia.com>

Best Regards,
Yan, Zi

^ permalink raw reply
* Re: [PATCH] mm/tracing: rss_stat: Ensure curr is false from kthread context
From: SeongJae Park @ 2026-02-20  1:16 UTC (permalink / raw)
  To: Kalesh Singh
  Cc: SeongJae Park, akpm, rostedt, joel, kernel-team, android-mm,
	David Hildenbrand (Arm), Lorenzo Stoakes, Minchan Kim,
	Suren Baghdasaryan, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Masami Hiramatsu, Mathieu Desnoyers,
	Jann Horn, Pedro Falcato, Martin Liu, David Rientjes, Zi Yan,
	Wander Lairson Costa, Petr Mladek, linux-mm, linux-kernel,
	linux-trace-kernel
In-Reply-To: <20260219233708.1971199-1-kaleshsingh@google.com>

On Thu, 19 Feb 2026 15:36:56 -0800 Kalesh Singh <kaleshsingh@google.com> wrote:

> The rss_stat trace event allows userspace tools, like Perfetto [1],
> to inspect per-process RSS metric changes over time.
> 
> The curr field was introduced to rss_stat in commit e4dcad204d3a
> ("rss_stat: add support to detect RSS updates of external mm").
> It's intent is to  indicate whether the RSS update is for the
> mm_struct of the current execution context; and is set to false
> when operating on a remote mm_struct (e.g., via kswapd or a
> direct reclaimer).
> 
> However, an issue arises when a kernel thread temporarily adopts
> a user process's mm_struct. Kernel threads do not have their own
> mm_struct and normally have current->mm set to NULL. To operate
> on user memory, they can "borrow" a memory context using
> kthread_use_mm(), which sets current->mm to the user process's mm.
> 
> This can be observed, for example, in the USB Function Filesystem
> (FFS) driver. The ffs_user_copy_worker() handles AIO completions
> and uses kthread_use_mm() to copy data to a user-space buffer.
> If a page fault occurs during this copy, the fault handler executes
> in the kthread's context.
> 
> At this point, current is the kthread, but current->mm points to the
> user process's mm. Since the rss_stat event (from the page fault)
> is for that same mm, the condition current->mm == mm becomes true,
> causing curr to be incorrectly set to true when the trace event is
> emitted.
> 
> This is misleading because it suggests the mm belongs to the kthread,
> confusing userspace tools that track per-process RSS changes and
> corrupting their mm_id-to-process association.
> 
> Fix this by ensuring curr is always false when the trace event is
> emitted from a kthread context by checking for the PF_KTHREAD flag.
> 
> [1] https://perfetto.dev/
> 
> Fixes: e4dcad204d3a ("rss_stat: add support to detect RSS updates of external mm")

Sounds like the issue is not that critical, but user-visible?  Would it be
better to Cc stable@ ?

> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "David Hildenbrand (Arm)" <david@kernel.org>
> Cc: Joel Fernandes <joel@joelfernandes.org>
> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Signed-off-by: Kalesh Singh <kaleshsingh@google.com>

Acked-by: SeongJae Park <sj@kernel.org>


Thanks,
SJ

[...]

^ permalink raw reply
* Re: [PATCH 1/2] x86: Prevent syscall hooking
From: Masami Hiramatsu @ 2026-02-20  2:45 UTC (permalink / raw)
  To: Kees Cook
  Cc: Steven Rostedt, Dave Hansen, Elly I. Esparza, linux-kernel, luto,
	tglx, mingo, bp, dave.hansen, x86, hpa, Naveen N Rao,
	David S. Miller, Masami Hiramatsu, linux-trace-kernel
In-Reply-To: <202602191041.4CB9C4AAFD@keescook>

On Thu, 19 Feb 2026 10:45:02 -0800
Kees Cook <kees@kernel.org> wrote:

> On Wed, Feb 18, 2026 at 10:52:04AM -0500, Steven Rostedt wrote:
> > Honesty, if you are worried about this, just run LOCKDOWN on tracing, and
> > prevent *ALL* kprobes. Because yes, there's a 1000 ways to get this
> > information once you have kprobes enabled and have root access. This patch
> > is hurting legitimate debugging of running systems more than it is limiting
> > rootkits from hacking the kernel.
> 
> Yeah, I agree. If kprobes is available, there is a lot of harm an
> attacker can already do. If a bright line between root/ring-0 is
> desired, a system needs to be configured to be using lockdown or similar
> things to turn off the interfaces that let root write to kernel state.

Agreed. The blacklist (or blocklist) of kprobes is designed for preventing
nesting software breakpoint handling, not for security.

Thank you,

-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

^ permalink raw reply
* Re: [PATCH 0/5] uprobes: transition from kmap_atomic to kmap_local_page
From: patchwork-bot+linux-riscv @ 2026-02-20  4:10 UTC (permalink / raw)
  To: Keke Ming
  Cc: linux-riscv, mhiramat, oleg, peterz, linux, catalin.marinas, will,
	tsbogend, pjw, palmer, aou, akpm, linux-kernel,
	linux-trace-kernel, linux-mm, linux-arm-kernel, linux-mips
In-Reply-To: <20260103084243.195125-1-ming.jvle@gmail.com>

Hello:

This series was applied to riscv/linux.git (fixes)
by Peter Zijlstra <peterz@infradead.org>:

On Sat,  3 Jan 2026 16:42:38 +0800 you wrote:
> The use of kmap_atomic/kunmap_atomic is deprecated. The purpose of
> kmap-like functions is to create temporary mappings.
> 
> kmap_atomic() typically disables preemption, while kmap_local_page()
> allows preemption.
> 
> According to the documentation, kmap_atomic() is primarily necessary
> for contexts that cannot sleep.
> 
> [...]

Here is the summary with links:
  - [1/5] riscv/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
    https://git.kernel.org/riscv/c/a18dfb5dd332
  - [2/5] arm64/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
    https://git.kernel.org/riscv/c/094cc7bb5fc3
  - [3/5] mips/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
    https://git.kernel.org/riscv/c/e6eb9acc024c
  - [4/5] arm/uprobes: use kmap_local_page() in arch_uprobe_copy_ixol()
    https://git.kernel.org/riscv/c/1752a1ad43a1
  - [5/5] uprobes: use kmap_local_page() for temporary page mappings
    https://git.kernel.org/riscv/c/a491c02c2770

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



^ permalink raw reply
* Re: [PATCH] mm/tracing: rss_stat: Ensure curr is false from kthread context
From: Kalesh Singh @ 2026-02-20  5:17 UTC (permalink / raw)
  To: SeongJae Park
  Cc: akpm, rostedt, joel, kernel-team, android-mm,
	David Hildenbrand (Arm), Lorenzo Stoakes, Minchan Kim,
	Suren Baghdasaryan, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Masami Hiramatsu, Mathieu Desnoyers,
	Jann Horn, Pedro Falcato, Martin Liu, David Rientjes, Zi Yan,
	Wander Lairson Costa, Petr Mladek, linux-mm, linux-kernel,
	linux-trace-kernel, stable
In-Reply-To: <20260220011700.127763-1-sj@kernel.org>

On Thu, Feb 19, 2026 at 5:17 PM SeongJae Park <sj@kernel.org> wrote:
>
> On Thu, 19 Feb 2026 15:36:56 -0800 Kalesh Singh <kaleshsingh@google.com> wrote:
>
> > The rss_stat trace event allows userspace tools, like Perfetto [1],
> > to inspect per-process RSS metric changes over time.
> >
> > The curr field was introduced to rss_stat in commit e4dcad204d3a
> > ("rss_stat: add support to detect RSS updates of external mm").
> > It's intent is to  indicate whether the RSS update is for the
> > mm_struct of the current execution context; and is set to false
> > when operating on a remote mm_struct (e.g., via kswapd or a
> > direct reclaimer).
> >
> > However, an issue arises when a kernel thread temporarily adopts
> > a user process's mm_struct. Kernel threads do not have their own
> > mm_struct and normally have current->mm set to NULL. To operate
> > on user memory, they can "borrow" a memory context using
> > kthread_use_mm(), which sets current->mm to the user process's mm.
> >
> > This can be observed, for example, in the USB Function Filesystem
> > (FFS) driver. The ffs_user_copy_worker() handles AIO completions
> > and uses kthread_use_mm() to copy data to a user-space buffer.
> > If a page fault occurs during this copy, the fault handler executes
> > in the kthread's context.
> >
> > At this point, current is the kthread, but current->mm points to the
> > user process's mm. Since the rss_stat event (from the page fault)
> > is for that same mm, the condition current->mm == mm becomes true,
> > causing curr to be incorrectly set to true when the trace event is
> > emitted.
> >
> > This is misleading because it suggests the mm belongs to the kthread,
> > confusing userspace tools that track per-process RSS changes and
> > corrupting their mm_id-to-process association.
> >
> > Fix this by ensuring curr is always false when the trace event is
> > emitted from a kthread context by checking for the PF_KTHREAD flag.
> >
> > [1] https://perfetto.dev/
> >
> > Fixes: e4dcad204d3a ("rss_stat: add support to detect RSS updates of external mm")
>
> Sounds like the issue is not that critical, but user-visible?  Would it be
> better to Cc stable@ ?

Thanks for the reviews, SJ and Zi.

I didn't add stable initially because it isn't functionally critical.
However, it would be nice to get it backported, as without it,
observability is much more difficult.

I believe the patch should apply cleanly to stable with minimal risk.
Andrew, if it isn't too much trouble, would you mind folding the
following tag into the staged patch?

Cc: stable@vger.kernel.org # 5.10+

Thanks,
Kalesh

>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: "David Hildenbrand (Arm)" <david@kernel.org>
> > Cc: Joel Fernandes <joel@joelfernandes.org>
> > Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> > Cc: Minchan Kim <minchan@kernel.org>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > Cc: Suren Baghdasaryan <surenb@google.com>
> > Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
>
> Acked-by: SeongJae Park <sj@kernel.org>
>
>
> Thanks,
> SJ
>
> [...]

^ permalink raw reply
* [BUG] suspicious RCU usage in event_filter_pid_sched_process_fork()
From: Masami Hiramatsu @ 2026-02-20  8:35 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: linux-kernel, Linux Trace Kernel, Mathieu Desnoyers

Hi,

On the latest tracing/for-next branch, I hit the following
suspicious RCU usage in event_filter_pid_sched_process_fork().
It seems that kernel_clone() calls back this event without
holding rcu_read_lock().

[37] event tracing - restricts events based on pid notrace filtering[  156.217609] 
[  156.218272] =============================
[  156.219569] WARNING: suspicious RCU usage
[  156.220887] 6.19.0-09924-g9678e53179aa #38 Not tainted
[  156.222336] -----------------------------
[  156.223481] kernel/trace/trace_events.c:1057 suspicious rcu_dereference_check() usage!
[  156.225660] 
[  156.225660] other info that might help us debug this:
[  156.225660] 
[  156.227958] 
[  156.227958] rcu_scheduler_active = 2, debug_locks = 1
[  156.229815] no locks held by ftracetest/4360.
[  156.231103] 
[  156.231103] stack backtrace:
[  156.232415] CPU: 1 UID: 0 PID: 4360 Comm: ftracetest Not tainted 6.19.0-09924-g9678e53179aa #38 PREEMPT(lazy) 
[  156.232418] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
[  156.232419] Call Trace:
[  156.232422]  <TASK>
[  156.232425]  dump_stack_lvl+0x67/0x90
[  156.232430]  lockdep_rcu_suspicious+0x154/0x1a0
[  156.232436]  event_filter_pid_sched_process_fork+0x9a/0xd0
[  156.232440]  kernel_clone+0x367/0x3a0
[  156.232448]  __x64_sys_clone+0x116/0x140
[  156.232454]  do_syscall_64+0x158/0x460
[  156.232457]  ? entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  156.232459]  ? trace_irq_disable+0x1d/0xc0
[  156.232464]  entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  156.232466] RIP: 0033:0x4697c3
[  156.232470] Code: 1f 84 00 00 00 00 00 64 48 8b 04 25 10 00 00 00 45 31 c0 31 d2 31 f6 bf 11 00 20 01 4c 8d 90 d0 02 00 00 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 35 89 c2 85 c0 75 2c 64 48 8b 04 25 10 00 00
[  156.232472] RSP: 002b:00007ffd8a0c1d58 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
[  156.232474] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004697c3
[  156.232476] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
[  156.232477] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000026970990
[  156.232478] R10: 000000002696e690 R11: 0000000000000246 R12: 0000000000000001
[  156.232479] R13: 0000000026989103 R14: 0000000000000001 R15: 0000000000000000
[  156.232484]  </TASK>
[  156.232495] 
[  156.268667] =============================
[  156.269872] WARNING: suspicious RCU usage
[  156.271089] 6.19.0-09924-g9678e53179aa #38 Not tainted
[  156.272544] -----------------------------
[  156.273704] kernel/trace/trace_events.c:1060 suspicious rcu_dereference_check() usage!
[  156.275903] 
[  156.275903] other info that might help us debug this:
[  156.275903] 
[  156.278122] 
[  156.278122] rcu_scheduler_active = 2, debug_locks = 1
[  156.279962] no locks held by ftracetest/4360.
[  156.281216] 
[  156.281216] stack backtrace:
[  156.282551] CPU: 1 UID: 0 PID: 4360 Comm: ftracetest Not tainted 6.19.0-09924-g9678e53179aa #38 PREEMPT(lazy) 
[  156.282553] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
[  156.282555] Call Trace:
[  156.282558]  <TASK>
[  156.282560]  dump_stack_lvl+0x67/0x90
[  156.282566]  lockdep_rcu_suspicious+0x154/0x1a0
[  156.282572]  event_filter_pid_sched_process_fork+0xcd/0xd0
[  156.282576]  kernel_clone+0x367/0x3a0
[  156.282584]  __x64_sys_clone+0x116/0x140
[  156.282589]  do_syscall_64+0x158/0x460
[  156.282592]  ? entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  156.282594]  ? trace_irq_disable+0x1d/0xc0
[  156.282600]  entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  156.282602] RIP: 0033:0x4697c3
[  156.282606] Code: 1f 84 00 00 00 00 00 64 48 8b 04 25 10 00 00 00 45 31 c0 31 d2 31 f6 bf 11 00 20 01 4c 8d 90 d0 02 00 00 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 35 89 c2 85 c0 75 2c 64 48 8b 04 25 10 00 00
[  156.282608] RSP: 002b:00007ffd8a0c1d58 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
[  156.282610] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004697c3
[  156.282611] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
[  156.282612] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000026970990
[  156.282613] R10: 000000002696e690 R11: 0000000000000246 R12: 0000000000000001
[  156.282615] R13: 0000000026989103 R14: 0000000000000001 R15: 0000000000000000
[  156.282621]  </TASK>

Thanks,

-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

^ permalink raw reply
* Re: [RFC PATCH v2 00/37] guest_memfd: In-place conversion support
From: Lisa Wang @ 2026-02-20  9:09 UTC (permalink / raw)
  To: Ackerley Tng
  Cc: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
	x86, aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
	chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
	ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
	mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
	oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
	rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
	steven.price, tabba, tglx, vannapurve, vbabka, willy, yan.y.zhao
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>

On Mon, Feb 02, 2026 at 02:36:37PM -0800, Ackerley Tng wrote:
> (resending to fix Message-ID)
> 
> Here's a second revision of guest_memfd In-place conversion support.
> 
> In this version, other than addressing comments from RFCv1 [1], the largest
> change is that guest_memfd now does not avoid participation in LRU; it
> participates in LRU by joining the unevictable list (no change from before this
> series).
> 
> While checking for elevated refcounts during shared to private conversions,
> guest_memfd will now do an lru_add_drain_all() if elevated refcounts were found,
> before concluding that there are true users of the shared folio and erroring
> out.
> 
> I'd still like feedback on these points, if any:
> 
> 1. Having private/shared status stored in a maple tree (Thanks Michael for your
>    support of using maple trees over xarrays for performance! [5]).
> 2. Having a new guest_memfd ioctl (not a vm ioctl) that performs conversions.
> 3. Using ioctls/structs/input attribute similar to the existing vm ioctl
>    KVM_SET_MEMORY_ATTRIBUTES to perform conversions.
> 4. Storing requested attributes directly in the maple tree.
> 5. Using a KVM module-wide param to toggle between setting memory attributes via
>    vm and guest_memfd ioctls (making them mututally exclusive - a single loaded
>    KVM module can only do one of the two.).
> 
> [...snip...]
>
> 
> --
> 2.53.0.rc1.225.gd81095ad13-goog

I’ve tested memory failure handling after applying this series and here’s what
memory_failure() does:

Shared memory: In line with other in-memory filesystems, the memory_failure()
handler unmaps the page if it is currently mapped, and issues a SIGBUS
  - if memory failure was injected with MF_ACTION_REQUIRED or
  - if the test process’s memory corruption kill policy is PR_MCE_KILL_EARLY

Here’s the above, in table form:

| MF_ACTION_REQUIRED | Kill Policy         | Mapped | Dirty | Result: SIGBUS |
|--------------------|---------------------|--------|-------|----------------|
| false              | PR_MCE_KILL_EARLY   | true   | true  | true           |
| false              | PR_MCE_KILL_EARLY   | true   | false | false          |
| false              | PR_MCE_KILL_EARLY   | false  | true  | false          |
| false              | PR_MCE_KILL_EARLY   | false  | false | false          |
| false              | PR_MCE_KILL_LATE    | true   | true  | false          |
| false              | PR_MCE_KILL_LATE    | true   | false | false          |
| false              | PR_MCE_KILL_LATE    | false  | true  | false          |
| false              | PR_MCE_KILL_LATE    | false  | false | false          |
| true               | Any Policy          | true   | true  | true           |
| true               | Any Policy          | true   | false | false          |

(I used MADV_HWPOISON to inject memory failures with MF_ACTION_REQUIRED set, and
there was no way to use MADV_HWPOISON without first mapping the page in. To
inject memory failures without MF_ACTION_REQUIRED set, I used debugfs’
hwpoison/corrupt-pfn.)

Private memory: The handler unmaps the page for the stage 2 page table and does
not issue a SIGBUS - the page is never mapped to the host, since it is private
to the guest.

| MF_ACTION_REQUIRED | Kill Policy         | Mapped | Dirty | Result: SIGBUS |
|--------------------|---------------------|--------|-------|----------------|
| false              | PR_MCE_KILL_EARLY   | false  | true  | false          |
| false              | PR_MCE_KILL_EARLY   | false  | false | false          |
| false              | PR_MCE_KILL_LATE    | false  | true  | false          |
| false              | PR_MCE_KILL_LATE    | false  | false | false          |

(I couldn’t use MADV_HWPOISON since private memory could not be mapped and hence
will not have a userspace address)

I’ll post updated memory failure tests together with the next revision of this
series [1] to fix MF_DELAYED handling on memory failure.

[1] https://lore.kernel.org/all/cover.1760551864.git.wyihan@google.com/T/


^ permalink raw reply
* [PATCH bpf-next 00/17] bpf: tracing_multi link
From: Jiri Olsa @ 2026-02-20 10:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, linux-trace-kernel, Martin KaFai Lau, Eduard Zingerman,
	Song Liu, Yonghong Song, Menglong Dong, Steven Rostedt

hi,
adding tracing_multi link support that allows fast attachment
of tracing program to many functions.

RFC version: https://lore.kernel.org/bpf/20260203093819.2105105-1-jolsa@kernel.org/

Changes to RFC:
- added ftrace_hash_count as wrapper for hash_count [Steven]
- added trampoline mutex pool [Andrii]
- reworked 'struct bpf_tramp_node' separatoin [Andrii]
  - the 'struct bpf_tramp_node' now holds pointer to bpf_link,
    which is similar to what we do for uprobe_multi;
    I understand it's not a fundamental change compared to previous
    version which used bpf_prog pointer instead, but I don't see better
    way of doing this.. I'm happy to discuss this further if there's
    better idea
- reworked 'struct bpf_fsession_link' based on bpf_tramp_node
- made btf__find_by_glob_kind function internal helper [Andrii]
- many small assorted fixes [Andrii,CI]
- added session support [Leon Hwang]
- added cookies support
- added more tests

Note I plan to send linkinfo/fdinfo support separately.

TODO: add rollback tests, add f*.multi.s tests, add trigger bench

---
Jiri Olsa (17):
      ftrace: Add ftrace_hash_count function
      bpf: Use mutex lock pool for bpf trampolines
      bpf: Add struct bpf_trampoline_ops object
      bpf: Add struct bpf_tramp_node object
      bpf: Factor fsession link to use struct bpf_tramp_node
      bpf: Add multi tracing attach types
      bpf: Add bpf_trampoline_multi_attach/detach functions
      bpf: Add support for tracing multi link
      bpf: Add support for tracing_multi link cookies
      bpf: Add support for tracing_multi link session
      libbpf: Add support to create tracing multi link
      selftests/bpf: Add tracing multi skel/pattern/ids attach tests
      selftests/bpf: Add tracing multi intersect tests
      selftests/bpf: Add tracing multi cookies test
      selftests/bpf: Add tracing multi session test
      selftests/bpf: Add tracing multi attach fails test
      selftests/bpf: Add tracing multi attach benchmark test

 arch/arm64/net/bpf_jit_comp.c                                      |  58 ++++-----
 arch/s390/net/bpf_jit_comp.c                                       |  42 +++---
 arch/x86/net/bpf_jit_comp.c                                        |  54 ++++----
 include/linux/bpf.h                                                |  87 +++++++++----
 include/linux/bpf_types.h                                          |   1 +
 include/linux/ftrace.h                                             |   1 +
 include/linux/trace_events.h                                       |   6 +
 include/uapi/linux/bpf.h                                           |   9 ++
 kernel/bpf/bpf_struct_ops.c                                        |  27 ++--
 kernel/bpf/btf.c                                                   |   4 +
 kernel/bpf/syscall.c                                               |  91 ++++++++-----
 kernel/bpf/trampoline.c                                            | 457 ++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 kernel/bpf/verifier.c                                              |  26 +++-
 kernel/trace/bpf_trace.c                                           | 120 ++++++++++++++++-
 kernel/trace/ftrace.c                                              |   7 +-
 net/bpf/bpf_dummy_struct_ops.c                                     |  14 +-
 net/bpf/test_run.c                                                 |   3 +
 tools/include/uapi/linux/bpf.h                                     |  10 ++
 tools/lib/bpf/bpf.c                                                |   9 ++
 tools/lib/bpf/bpf.h                                                |   5 +
 tools/lib/bpf/libbpf.c                                             | 200 ++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h                                             |  15 +++
 tools/lib/bpf/libbpf.map                                           |   1 +
 tools/testing/selftests/bpf/Makefile                               |   7 +-
 tools/testing/selftests/bpf/prog_tests/tracing_multi.c             | 595 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/tracing_multi_attach.c           |  26 ++++
 tools/testing/selftests/bpf/progs/tracing_multi_bench.c            |  13 ++
 tools/testing/selftests/bpf/progs/tracing_multi_check.c            | 165 ++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/tracing_multi_intersect_attach.c |  42 ++++++
 tools/testing/selftests/bpf/progs/tracing_multi_session_attach.c   |  27 ++++
 30 files changed, 1874 insertions(+), 248 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_multi.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_bench.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_check.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_intersect_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_session_attach.c

^ permalink raw reply
* [PATCH bpf-next 01/17] ftrace: Add ftrace_hash_count function
From: Jiri Olsa @ 2026-02-20 10:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, linux-trace-kernel, Martin KaFai Lau, Eduard Zingerman,
	Song Liu, Yonghong Song, Menglong Dong, Steven Rostedt
In-Reply-To: <20260220100649.628307-1-jolsa@kernel.org>

Adding external ftrace_hash_count function so we could get hash
count outside of ftrace object.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/linux/ftrace.h | 1 +
 kernel/trace/ftrace.c  | 7 ++++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 1a4d36fc9085..a1ea6ab29407 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -415,6 +415,7 @@ struct ftrace_hash *alloc_ftrace_hash(int size_bits);
 void free_ftrace_hash(struct ftrace_hash *hash);
 struct ftrace_func_entry *add_ftrace_hash_entry_direct(struct ftrace_hash *hash,
 						       unsigned long ip, unsigned long direct);
+unsigned long ftrace_hash_count(struct ftrace_hash *hash);
 
 /* The hash used to know what functions callbacks trace */
 struct ftrace_ops_hash {
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 1ce17c8af409..dd1844f882cd 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -6288,11 +6288,16 @@ int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
 }
 EXPORT_SYMBOL_GPL(modify_ftrace_direct);
 
-static unsigned long hash_count(struct ftrace_hash *hash)
+static inline unsigned long hash_count(struct ftrace_hash *hash)
 {
 	return hash ? hash->count : 0;
 }
 
+unsigned long ftrace_hash_count(struct ftrace_hash *hash)
+{
+	return hash_count(hash);
+}
+
 /**
  * hash_add - adds two struct ftrace_hash and returns the result
  * @a: struct ftrace_hash object
-- 
2.52.0


^ permalink raw reply related
* [PATCH bpf-next 02/17] bpf: Use mutex lock pool for bpf trampolines
From: Jiri Olsa @ 2026-02-20 10:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, linux-trace-kernel, Martin KaFai Lau, Eduard Zingerman,
	Song Liu, Yonghong Song, Menglong Dong, Steven Rostedt
In-Reply-To: <20260220100649.628307-1-jolsa@kernel.org>

Adding mutex lock pool that replaces bpf trampolines mutex.

For tracing_multi link coming in following changes we need to lock all
the involved trampolines during the attachment. This could mean thousands
of mutex locks, which is not convenient.

As suggested by Andrii we can replace bpf trampolines mutex with mutex
pool, where each trampoline is hash-ed to one of the locks from the pool.

It's better to lock all the pool mutexes (64 at the moment) than
thousands of them.

Removing the mutex_is_locked in bpf_trampoline_put, because we removed
the mutex from bpf_trampoline.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/linux/bpf.h     |  2 --
 kernel/bpf/trampoline.c | 74 +++++++++++++++++++++++++++++++----------
 2 files changed, 56 insertions(+), 20 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index cd9b96434904..46bf3d86bdb2 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1335,8 +1335,6 @@ struct bpf_trampoline {
 	/* hlist for trampoline_ip_table */
 	struct hlist_node hlist_ip;
 	struct ftrace_ops *fops;
-	/* serializes access to fields of this trampoline */
-	struct mutex mutex;
 	refcount_t refcnt;
 	u32 flags;
 	u64 key;
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 952cd7932461..05dc0358654d 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -30,6 +30,45 @@ static struct hlist_head trampoline_ip_table[TRAMPOLINE_TABLE_SIZE];
 /* serializes access to trampoline tables */
 static DEFINE_MUTEX(trampoline_mutex);
 
+#define TRAMPOLINE_LOCKS_BITS 6
+#define TRAMPOLINE_LOCKS_TABLE_SIZE (1 << TRAMPOLINE_LOCKS_BITS)
+
+static struct {
+	struct mutex mutex;
+	struct lock_class_key key;
+} *trampoline_locks;
+
+static struct mutex *trampoline_locks_lookup(struct bpf_trampoline *tr)
+{
+	return &trampoline_locks[hash_64((u64) tr, TRAMPOLINE_LOCKS_BITS)].mutex;
+}
+
+static void trampoline_lock(struct bpf_trampoline *tr)
+{
+	mutex_lock(trampoline_locks_lookup(tr));
+}
+
+static void trampoline_unlock(struct bpf_trampoline *tr)
+{
+	mutex_unlock(trampoline_locks_lookup(tr));
+}
+
+static int __init trampoline_locks_init(void)
+{
+	int i;
+
+	trampoline_locks = kmalloc_array(TRAMPOLINE_LOCKS_TABLE_SIZE,
+					 sizeof(trampoline_locks[0]), GFP_KERNEL);
+	if (!trampoline_locks)
+		return -ENOMEM;
+
+	for (i = 0; i < TRAMPOLINE_LOCKS_TABLE_SIZE; i++) {
+		lockdep_register_key(&trampoline_locks[i].key);
+		mutex_init_with_key(&trampoline_locks[i].mutex, &trampoline_locks[i].key);
+	}
+	return 0;
+}
+
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
 static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mutex);
 
@@ -71,7 +110,7 @@ static int bpf_tramp_ftrace_ops_func(struct ftrace_ops *ops, unsigned long ip,
 		/* This is called inside register_ftrace_direct_multi(), so
 		 * tr->mutex is already locked.
 		 */
-		lockdep_assert_held_once(&tr->mutex);
+		lockdep_assert_held_once(trampoline_locks_lookup(tr));
 
 		/* Instead of updating the trampoline here, we propagate
 		 * -EAGAIN to register_ftrace_direct(). Then we can
@@ -102,7 +141,7 @@ static int bpf_tramp_ftrace_ops_func(struct ftrace_ops *ops, unsigned long ip,
 	 * mutex_trylock(&tr->mutex) to avoid deadlock in race condition
 	 * (something else is making changes to this same trampoline).
 	 */
-	if (!mutex_trylock(&tr->mutex)) {
+	if (!mutex_trylock(trampoline_locks_lookup(tr))) {
 		/* sleep 1 ms to make sure whatever holding tr->mutex makes
 		 * some progress.
 		 */
@@ -129,7 +168,7 @@ static int bpf_tramp_ftrace_ops_func(struct ftrace_ops *ops, unsigned long ip,
 		break;
 	}
 
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 	return ret;
 }
 #endif
@@ -359,7 +398,6 @@ static struct bpf_trampoline *bpf_trampoline_lookup(u64 key, unsigned long ip)
 	head = &trampoline_ip_table[hash_64(tr->ip, TRAMPOLINE_HASH_BITS)];
 	hlist_add_head(&tr->hlist_ip, head);
 	refcount_set(&tr->refcnt, 1);
-	mutex_init(&tr->mutex);
 	for (i = 0; i < BPF_TRAMP_MAX; i++)
 		INIT_HLIST_HEAD(&tr->progs_hlist[i]);
 out:
@@ -844,9 +882,9 @@ int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
 {
 	int err;
 
-	mutex_lock(&tr->mutex);
+	trampoline_lock(tr);
 	err = __bpf_trampoline_link_prog(link, tr, tgt_prog);
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 	return err;
 }
 
@@ -887,9 +925,9 @@ int bpf_trampoline_unlink_prog(struct bpf_tramp_link *link,
 {
 	int err;
 
-	mutex_lock(&tr->mutex);
+	trampoline_lock(tr);
 	err = __bpf_trampoline_unlink_prog(link, tr, tgt_prog);
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 	return err;
 }
 
@@ -999,14 +1037,15 @@ int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog,
 	if (!tr)
 		return  -ENOMEM;
 
-	mutex_lock(&tr->mutex);
+	trampoline_lock(tr);
 
 	shim_link = cgroup_shim_find(tr, bpf_func);
 	if (shim_link) {
 		/* Reusing existing shim attached by the other program. */
 		bpf_link_inc(&shim_link->link.link);
 
-		mutex_unlock(&tr->mutex);
+		trampoline_unlock(tr);
+
 		bpf_trampoline_put(tr); /* bpf_trampoline_get above */
 		return 0;
 	}
@@ -1026,11 +1065,11 @@ int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog,
 	shim_link->trampoline = tr;
 	/* note, we're still holding tr refcnt from above */
 
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 
 	return 0;
 err:
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 
 	if (shim_link)
 		bpf_link_put(&shim_link->link.link);
@@ -1056,9 +1095,9 @@ void bpf_trampoline_unlink_cgroup_shim(struct bpf_prog *prog)
 	if (WARN_ON_ONCE(!tr))
 		return;
 
-	mutex_lock(&tr->mutex);
+	trampoline_lock(tr);
 	shim_link = cgroup_shim_find(tr, bpf_func);
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 
 	if (shim_link)
 		bpf_link_put(&shim_link->link.link);
@@ -1076,14 +1115,14 @@ struct bpf_trampoline *bpf_trampoline_get(u64 key,
 	if (!tr)
 		return NULL;
 
-	mutex_lock(&tr->mutex);
+	trampoline_lock(tr);
 	if (tr->func.addr)
 		goto out;
 
 	memcpy(&tr->func.model, &tgt_info->fmodel, sizeof(tgt_info->fmodel));
 	tr->func.addr = (void *)tgt_info->tgt_addr;
 out:
-	mutex_unlock(&tr->mutex);
+	trampoline_unlock(tr);
 	return tr;
 }
 
@@ -1096,7 +1135,6 @@ void bpf_trampoline_put(struct bpf_trampoline *tr)
 	mutex_lock(&trampoline_mutex);
 	if (!refcount_dec_and_test(&tr->refcnt))
 		goto out;
-	WARN_ON_ONCE(mutex_is_locked(&tr->mutex));
 
 	for (i = 0; i < BPF_TRAMP_MAX; i++)
 		if (WARN_ON_ONCE(!hlist_empty(&tr->progs_hlist[i])))
@@ -1382,6 +1420,6 @@ static int __init init_trampolines(void)
 		INIT_HLIST_HEAD(&trampoline_key_table[i]);
 	for (i = 0; i < TRAMPOLINE_TABLE_SIZE; i++)
 		INIT_HLIST_HEAD(&trampoline_ip_table[i]);
-	return 0;
+	return trampoline_locks_init();
 }
 late_initcall(init_trampolines);
-- 
2.52.0


^ permalink raw reply related
* [PATCH bpf-next 03/17] bpf: Add struct bpf_trampoline_ops object
From: Jiri Olsa @ 2026-02-20 10:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, linux-trace-kernel, Martin KaFai Lau, Eduard Zingerman,
	Song Liu, Yonghong Song, Menglong Dong, Steven Rostedt
In-Reply-To: <20260220100649.628307-1-jolsa@kernel.org>

In following changes we will need to override ftrace direct attachment
behaviour. In order to do that we are adding struct bpf_trampoline_ops
object that defines callbacks for ftrace direct attachment:

   register_fentry
   unregister_fentry
   modify_fentry

The new struct bpf_trampoline_ops object is passed as an argument to
__bpf_trampoline_link/unlink_prog functions.

At the moment the default trampoline_ops is set to the current ftrace
direct attachment functions, so there's no functional change for the
current code.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/bpf/trampoline.c | 54 +++++++++++++++++++++++++++++------------
 1 file changed, 39 insertions(+), 15 deletions(-)

diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 05dc0358654d..e9f0152289a4 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -69,6 +69,14 @@ static int __init trampoline_locks_init(void)
 	return 0;
 }
 
+struct bpf_trampoline_ops {
+	int (*register_fentry)(struct bpf_trampoline *tr, void *new_addr, void *data);
+	int (*unregister_fentry)(struct bpf_trampoline *tr, u32 orig_flags, void *old_addr,
+				 void *data);
+	int (*modify_fentry)(struct bpf_trampoline *tr, u32 orig_flags, void *old_addr,
+			     void *new_addr, bool lock_direct_mutex, void *data);
+};
+
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
 static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mutex);
 
@@ -425,7 +433,7 @@ static int bpf_trampoline_update_fentry(struct bpf_trampoline *tr, u32 orig_flag
 }
 
 static int unregister_fentry(struct bpf_trampoline *tr, u32 orig_flags,
-			     void *old_addr)
+			     void *old_addr, void *data)
 {
 	int ret;
 
@@ -439,7 +447,7 @@ static int unregister_fentry(struct bpf_trampoline *tr, u32 orig_flags,
 
 static int modify_fentry(struct bpf_trampoline *tr, u32 orig_flags,
 			 void *old_addr, void *new_addr,
-			 bool lock_direct_mutex)
+			 bool lock_direct_mutex, void *data __maybe_unused)
 {
 	int ret;
 
@@ -453,7 +461,7 @@ static int modify_fentry(struct bpf_trampoline *tr, u32 orig_flags,
 }
 
 /* first time registering */
-static int register_fentry(struct bpf_trampoline *tr, void *new_addr)
+static int register_fentry(struct bpf_trampoline *tr, void *new_addr, void *data __maybe_unused)
 {
 	void *ip = tr->func.addr;
 	unsigned long faddr;
@@ -475,6 +483,12 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr)
 	return ret;
 }
 
+static struct bpf_trampoline_ops trampoline_ops = {
+	.register_fentry   = register_fentry,
+	.unregister_fentry = unregister_fentry,
+	.modify_fentry     = modify_fentry,
+};
+
 static struct bpf_tramp_links *
 bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_arg)
 {
@@ -642,7 +656,8 @@ static struct bpf_tramp_image *bpf_tramp_image_alloc(u64 key, int size)
 	return ERR_PTR(err);
 }
 
-static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mutex)
+static int bpf_trampoline_update_ops(struct bpf_trampoline *tr, bool lock_direct_mutex,
+				     struct bpf_trampoline_ops *ops, void *data)
 {
 	struct bpf_tramp_image *im;
 	struct bpf_tramp_links *tlinks;
@@ -655,7 +670,7 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
 		return PTR_ERR(tlinks);
 
 	if (total == 0) {
-		err = unregister_fentry(tr, orig_flags, tr->cur_image->image);
+		err = ops->unregister_fentry(tr, orig_flags, tr->cur_image->image, data);
 		bpf_tramp_image_put(tr->cur_image);
 		tr->cur_image = NULL;
 		goto out;
@@ -726,11 +741,11 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
 	WARN_ON(tr->cur_image && total == 0);
 	if (tr->cur_image)
 		/* progs already running at this address */
-		err = modify_fentry(tr, orig_flags, tr->cur_image->image,
-				    im->image, lock_direct_mutex);
+		err = ops->modify_fentry(tr, orig_flags, tr->cur_image->image,
+					 im->image, lock_direct_mutex, data);
 	else
 		/* first time registering */
-		err = register_fentry(tr, im->image);
+		err = ops->register_fentry(tr, im->image, data);
 
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
 	if (err == -EAGAIN) {
@@ -760,6 +775,11 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
 	goto out;
 }
 
+static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mutex)
+{
+	return bpf_trampoline_update_ops(tr, lock_direct_mutex, &trampoline_ops, NULL);
+}
+
 static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(struct bpf_prog *prog)
 {
 	switch (prog->expected_attach_type) {
@@ -804,7 +824,9 @@ static int bpf_freplace_check_tgt_prog(struct bpf_prog *tgt_prog)
 
 static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link,
 				      struct bpf_trampoline *tr,
-				      struct bpf_prog *tgt_prog)
+				      struct bpf_prog *tgt_prog,
+				      struct bpf_trampoline_ops *ops,
+				      void *data)
 {
 	struct bpf_fsession_link *fslink = NULL;
 	enum bpf_tramp_prog_type kind;
@@ -862,7 +884,7 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link,
 	} else {
 		tr->progs_cnt[kind]++;
 	}
-	err = bpf_trampoline_update(tr, true /* lock_direct_mutex */);
+	err = bpf_trampoline_update_ops(tr, true /* lock_direct_mutex */, ops, data);
 	if (err) {
 		hlist_del_init(&link->tramp_hlist);
 		if (kind == BPF_TRAMP_FSESSION) {
@@ -883,14 +905,16 @@ int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
 	int err;
 
 	trampoline_lock(tr);
-	err = __bpf_trampoline_link_prog(link, tr, tgt_prog);
+	err = __bpf_trampoline_link_prog(link, tr, tgt_prog, &trampoline_ops, NULL);
 	trampoline_unlock(tr);
 	return err;
 }
 
 static int __bpf_trampoline_unlink_prog(struct bpf_tramp_link *link,
 					struct bpf_trampoline *tr,
-					struct bpf_prog *tgt_prog)
+					struct bpf_prog *tgt_prog,
+					struct bpf_trampoline_ops *ops,
+					void *data)
 {
 	enum bpf_tramp_prog_type kind;
 	int err;
@@ -915,7 +939,7 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_link *link,
 	}
 	hlist_del_init(&link->tramp_hlist);
 	tr->progs_cnt[kind]--;
-	return bpf_trampoline_update(tr, true /* lock_direct_mutex */);
+	return bpf_trampoline_update_ops(tr, true /* lock_direct_mutex */, ops, data);
 }
 
 /* bpf_trampoline_unlink_prog() should never fail. */
@@ -926,7 +950,7 @@ int bpf_trampoline_unlink_prog(struct bpf_tramp_link *link,
 	int err;
 
 	trampoline_lock(tr);
-	err = __bpf_trampoline_unlink_prog(link, tr, tgt_prog);
+	err = __bpf_trampoline_unlink_prog(link, tr, tgt_prog, &trampoline_ops, NULL);
 	trampoline_unlock(tr);
 	return err;
 }
@@ -1058,7 +1082,7 @@ int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog,
 		goto err;
 	}
 
-	err = __bpf_trampoline_link_prog(&shim_link->link, tr, NULL);
+	err = __bpf_trampoline_link_prog(&shim_link->link, tr, NULL, &trampoline_ops, NULL);
 	if (err)
 		goto err;
 
-- 
2.52.0


^ permalink raw reply related
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox