Linux Trace Kernel
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org
Subject: Re: [PATCH v20 08/11] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers
Date: Mon, 16 Nov 2026 06:20:28 +0000	[thread overview]
Message-ID: <sashiko-outbox-216266@kernel.org> (raw)
In-Reply-To: <179480684514.38065.14028970872476288842.stgit@devnote2>

Thank you for your contribution! Sashiko AI review found 9 potential
issues to consider:
- [Medium] tracing/wprobe: stale work_pending after cancel_work_sync()
  wedges triggers
- [Medium] tracing/wprobe: race and UAF on bp_event in
  wprobe_trigger_free() teardown
- [Medium] tracing/wprobe: broken trace_probe_is_enabled() check in
  trigger parse
- [Medium] tracing/wprobe: TASK_SIZE in trace_wprobe_is_valid_addr()
  allows user addr
- [Medium] tracing/wprobe: unrestored tw->addr after set_wprobe breaks
  show and match
- [Medium] tracing/wprobe: stale breakpoint hits fire triggers in
  wprobe_trace_handler()
- [Medium] tracing/wprobe: self-deadlock on tw->lock in wprobe_trigger()
- [Low] tracing/wprobe: in-place NUL mutation of glob truncates trigger
  error_log
- [Low] tracing/wprobe: unlocked double read of tw->addr in
  trace_wprobe_show()
--

commit 86f0de972cafa3458ae22f00d7d3014fc1fc6164
Author: Masami Hiramatsu (Google) <mhiramat@kernel.org>

tracing/wprobe: Add set_wprobe and clear_wprobe event triggers

This commit adds set_wprobe and clear_wprobe event triggers to dynamically
configure and clear hardware watchpoint addresses from trace event fields.

Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/

> diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
> index 644183cf5155e..d2b41aa69cfdf 100644
> --- a/kernel/trace/trace_wprobe.c
> +++ b/kernel/trace/trace_wprobe.c

[ ... ]

> @@ -217,28 +241,143 @@ static int __register_trace_wprobe(struct trace_wprobe *tw)
>  	attr.bp_len = tw->len;
>  	attr.bp_type = tw->type;
>  
> -	tw->bp_event = register_wide_hw_breakpoint(&attr, wprobe_perf_handler, tw);
> +	cpus_read_lock();
> +	tw->bp_event = register_wide_hw_breakpoint_cpuslocked(&attr,
> +							      wprobe_perf_handler,
> +							      tw);
>  	if (IS_ERR_PCPU(tw->bp_event)) {
>  		int ret = PTR_ERR_PCPU(tw->bp_event);
>  
>  		tw->bp_event = NULL;
> +		cpus_read_unlock();
>  		return ret;
>  	}
>  
> +	mutex_lock(&wprobe_mutex);
> +	list_add(&tw->active_list, &active_wprobes);
> +	mutex_unlock(&wprobe_mutex);
> +	cpus_read_unlock();
> +
>  	return 0;
>  }
>  
>  static void __unregister_trace_wprobe(struct trace_wprobe *tw)
>  {
>  	if (tw->bp_event) {
> +		mutex_lock(&wprobe_mutex);
> +		list_del_init(&tw->active_list);
> +		mutex_unlock(&wprobe_mutex);
> +
> +		irq_work_sync(&tw->irq_work);
> +		cancel_work_sync(&tw->work);
>  		unregister_wide_hw_breakpoint(tw->bp_event);
>  		tw->bp_event = NULL;
>  	}
>  }

[Severity: Medium]
Does cancelling tw->work in __unregister_trace_wprobe() leave
tw->work_pending stuck at 1?

wprobe_trigger() sets tw->work_pending to 1 before queueing tw->irq_work,
and only wprobe_work_func() resets it to 0.  If __unregister_trace_wprobe()
runs after wprobe_irq_work_func() schedules tw->work but before
wprobe_work_func() executes, cancel_work_sync(&tw->work) cancels the work
item without clearing tw->work_pending.  Every later trigger evaluation on
that wprobe then sees atomic_read(&tw->work_pending) != 0 in
__wprobe_count_func(), increments tw->missed, and drops all future updates.

[ ... ]

> @@ -788,6 +945,10 @@ static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
>  				   tw->offset, len);
>  		else
>  			seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len);
> +#ifdef CONFIG_WPROBE_TRIGGERS
> +	} else if (tw->addr == WPROBE_DEFAULT_CLEAR_ADDRESS) {\
> +		seq_printf(m, " %s@-1:%d", type_str, len);
> +#endif
>  	} else {
>  		seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len);
>  	}

[Severity: Low]
Can trace_wprobe_show() print the raw address of
wprobe_trigger_clear_target instead of -1?

wprobe_trigger() updates tw->addr concurrently under tw->lock without
holding dyn_event_ops_mutex.  Because trace_wprobe_show() reads tw->addr a
second time in the else branch instead of reading it once with READ_ONCE()
into a local variable, a concurrent clear_wprobe trigger setting tw->addr
to WPROBE_DEFAULT_CLEAR_ADDRESS between the comparison and seq_printf()
causes dynamic_events to print the address of wprobe_trigger_clear_target.

> @@ -800,9 +961,635 @@ static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)

[ ... ]

> +static bool __wprobe_count_func(struct event_trigger_data *data,
> +				unsigned long addr)
> +{
> +	struct wprobe_trigger_data *wprobe_data = data->private_data;
> +	struct trace_wprobe *tw = wprobe_data->tw;
> +	unsigned long cur_addr;
> +
> +	if (atomic_read(&tw->work_pending))
> +		return false;
> +
> +	cur_addr = READ_ONCE(tw->addr);
> +
> +	if (!wprobe_data->clear) {
> +		if (cur_addr != WPROBE_DEFAULT_CLEAR_ADDRESS)
> +			return false;
> +		if (!trace_wprobe_is_valid_addr(addr, tw->len))
> +			return false;

[Severity: Medium]
Can trace_wprobe_is_valid_addr() allow a set_wprobe trigger to install a
user-space watchpoint address when a 32-bit compat task is running?

kernel/trace/trace_wprobe.c:trace_wprobe_is_valid_addr() {
	if (addr < TASK_SIZE)
		return false;
    ...
}

On 64-bit architectures with 32-bit compatibility mode (such as x86_64 with
CONFIG_IA32_EMULATION), TASK_SIZE depends on the current task's personality
and evaluates to IA32_PAGE_OFFSET (~4 GiB) when a 32-bit compat task is
running.  Because modify_local_hw_breakpoint_addr() bypasses
arch_check_bp_in_kernelspace() (which checks TASK_SIZE_MAX), a set_wprobe
trigger firing in the context of a 32-bit compat task accepts any aligned
64-bit user-space address in [IA32_PAGE_OFFSET, TASK_SIZE_MAX) and programs
it into the CPU-wide hardware debug registers.  Using TASK_SIZE_MAX (or
arch_check_bp_in_kernelspace()) avoids depending on current.

[ ... ]

> +static void wprobe_trigger(struct event_trigger_data *data,
> +			   struct trace_buffer *buffer,  void *rec,
> +			   struct ring_buffer_event *event)
> +{
> +	struct wprobe_trigger_data *wprobe_data = data->private_data;
> +	struct trace_wprobe *tw = wprobe_data->tw;
> +	unsigned long target_addr, addr = 0, flags;
> +
> +	if (in_nmi())
> +		return;
> +
> +	if (wprobe_data->field) {
> +		if (!rec)
> +			return;
> +		addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
> +		addr += wprobe_data->adjust;
> +	}
> +
> +	raw_spin_lock_irqsave(&tw->lock, flags);
> +
> +	if (!__wprobe_count_func(data, addr)) {
> +		atomic_inc(&tw->missed);
> +		goto out;
> +	}
> +
> +	/*
> +	 * For counted triggers, verify data->count is still available under
> +	 * tw->lock to prevent concurrent executions from exceeding the limit.
> +	 */
> +	if ((data->flags & EVENT_TRIGGER_FL_COUNT) && !data->count)
> +		goto out;
> +
> +	target_addr = wprobe_data->clear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr;
> +
> +	/*
> +	 * Try updating the hardware breakpoint on the local CPU first.
> +	 * If this fails, skip updating tw->addr and do not queue irq_work.
> +	 * Note that this wprobe will not be traced until tw_addr is updated.
> +	 * (e.g. NMI happens on this CPU.)
> +	 */
> +	if (trace_wprobe_update_local(tw, target_addr)) {
> +		atomic_inc(&tw->missed);
> +		goto out;
> +	}
> +
> +	/*
> +	 * Decrement data->count here under tw->lock only after the local
> +	 * hardware breakpoint has been successfully updated. This avoids:
> +	 * 1) Consuming count on trigger attempts that failed/missed.
> +	 * 2) Race windows where concurrent CPUs decrement count and underflow
> +	 *    to -1 (unlimited).
> +	 */
> +	if (data->flags & EVENT_TRIGGER_FL_COUNT) {
> +		if (data->count > 0)
> +			(data->count)--;
> +	}
> +
> +	WRITE_ONCE(tw->addr, target_addr);
> +	if (!wprobe_data->clear)
> +		clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
> +	else
> +		set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
> +
> +	/*
> +	 * For other CPUs, kick irq_work to asynchronously propagate
> +	 * the new address.
> +	 */
> +	if (num_online_cpus() > 1) {
> +		atomic_set(&tw->work_pending, 1);
> +		irq_work_queue(&tw->irq_work);
> +	}
> +
> +out:
> +	raw_spin_unlock_irqrestore(&tw->lock, flags);
> +}

[Severity: Medium]
Can wprobe_trigger() self-deadlock on tw->lock if a nested event fires
while tw->lock is held?

wprobe_trigger()
  raw_spin_lock_irqsave(&tw->lock, flags)
  irq_work_queue(&tw->irq_work)
    ...
      native_write_msr()
        trace_write_msr()
          ...
            wprobe_trigger()
              raw_spin_lock_irqsave(&tw->lock, flags)  <-- spins forever

While wprobe_trigger() holds tw->lock, irq_work_queue() on x86 with x2APIC
raises a self-IPI via native_write_msr(), firing the msr:write_msr
tracepoint, and functions called by trace_wprobe_update_local() (such as
modify_local_hw_breakpoint_addr()) can also be traced by fprobe events.  If
a wprobe trigger targeting the same trace_wprobe is attached to one of
those nested events, the inner wprobe_trigger() runs on the same CPU with
in_nmi() == false and spins forever on tw->lock.

[Severity: Medium]
Can a stale breakpoint hit on a remote CPU fire unconditional triggers
attached to the wprobe event after clear_wprobe or set_wprobe updates
tw->addr?

CPU0                              CPU1
wprobe_trigger()
  WRITE_ONCE(tw->addr,
    WPROBE_DEFAULT_CLEAR_ADDRESS)
  set_bit(SOFT_DISABLED_BIT, ...)
  irq_work_queue(&tw->irq_work)
                                  wprobe_perf_handler()
                                    wprobe_trace_handler(tw, old_addr, ...)
                                      trace_trigger_soft_disabled(trace_file)
                                        event_triggers_call(...)
                                      if (READ_ONCE(tw->addr) != addr)
                                        return;

kernel/trace/trace_wprobe.c:wprobe_trace_handler() {
    ...
	if (trace_trigger_soft_disabled(trace_file))
		return;

	if (READ_ONCE(tw->addr) != addr)
		return;
    ...
}

Because remote CPUs update their hardware debug registers asynchronously via
tw->work, a remote CPU can still take a breakpoint exception for the old
address after tw->addr has changed.  In wprobe_trace_handler(),
trace_trigger_soft_disabled() runs before the READ_ONCE(tw->addr) != addr
check, which invokes __trace_trigger_soft_disabled() ->
event_triggers_call() and fires any unconditional triggers (such as
traceoff, snapshot, or stacktrace) attached to the wprobe event.  Checking
READ_ONCE(tw->addr) != addr before trace_trigger_soft_disabled() avoids
firing triggers on stale hits.

[ ... ]

> +static void release_private_wprobe_trigger_data(struct event_trigger_data *data)
> +{
> +	struct wprobe_trigger_data *wprobe_data = data->private_data;
> +
> +	/* The file may already be freed here, only the call is kept alive */
> +	trace_event_put_ref(trace_probe_event_call(&wprobe_data->tw->tp));
> +	free_wprobe_trigger_data(wprobe_data);
> +}

[ ... ]

> +static void wprobe_trigger_free(struct event_trigger_data *data)
> +{
> +	struct wprobe_trigger_data *wprobe_data = data->private_data;
> +
> +	if (WARN_ON_ONCE(data->ref <= 0))
> +		return;
> +
> +	data->ref--;
> +	if (!data->ref) {
> +		/* Remove the SOFT_MODE flag */
> +		trace_event_enable_disable(wprobe_data->file, 0, 1);
> +		data->private_data_free = release_private_wprobe_trigger_data;
> +		trigger_data_free(data);
> +	}
> +}

[Severity: Medium]
Can wprobe_trigger_free() unregister and free tw->bp_event while an
in-flight wprobe_trigger() is still using it?

CPU0                              CPU1
clear_event_triggers()
  list_del_rcu(&data->list)
  wprobe_trigger_free(data)
    trace_event_enable_disable()
      ...
        __unregister_trace_wprobe(tw)
          unregister_wide_hw_breakpoint(
            tw->bp_event)
                                  wprobe_trigger()
                                    trace_wprobe_update_local(tw, ...)
                                      bp = *this_cpu_ptr(tw->bp_event)
                                      modify_local_hw_breakpoint_addr(bp, ...)
                                        <-- use-after-free
          tw->bp_event = NULL
    trigger_data_free(data)
      /* tracepoint_synchronize_unregister() happens later in kthread */

When a trace instance is removed, clear_event_triggers() calls
data->cmd_ops->free(data) immediately after list_del_rcu() without waiting
for tracepoint_synchronize_unregister().  Because wprobe_trigger_free()
calls trace_event_enable_disable() before trigger_data_free() waits for
in-flight tracepoints, __unregister_trace_wprobe() frees the per-CPU
perf_event breakpoints and tw->bp_event before clearing tw->bp_event = NULL
and without synchronizing with tw->lock, while a concurrent wprobe_trigger()
on another CPU dereferences tw->bp_event and queues tw->irq_work after
irq_work_sync() has already returned.

Conversely, when a user removes a trigger via wprobe_unregister_trigger(),
tracepoint_synchronize_unregister() has already completed, but deferring
trace_event_put_ref() to release_private_wprobe_trigger_data() on
trigger_kthread means an immediate dynamic_events removal of the wprobe can
race with the kthread and fail with -EBUSY.

> +static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
> +				    struct trace_event_file *file,
> +				    char *glob, char *cmd,
> +				    char *param_and_filter)
> +{
> +	/*
> +	 * set_wprobe:EVENT:FIELD[+OFFS]
> +	 * clear_wprobe:EVENT[:FIELD[+OFFS]]
> +	 */
> +	struct wprobe_trigger_data *wprobe_data = NULL;
> +	struct event_trigger_data *trigger_data = NULL;
> +	struct trace_event_file *wprobe_file;
> +	struct trace_array *tr = file->tr;
> +	char *event_str, *field_str, *comment;
> +	struct ftrace_event_field *field;
> +	struct trace_event_call *event;
> +	bool remove, clear = false;
> +	unsigned long orig_addr = 0;
> +	struct trace_wprobe *tw;
> +	char *param, *filter;
> +	int ret;
> +
> +	remove = event_trigger_check_remove(glob);
> +
> +	if (!strcmp(cmd, CLEAR_WPROBE_STR))
> +		clear = true;
> +
> +	if (param_and_filter) {
> +		/* Recover original trigger string to show the error log correctly. */
> +		if (*(param_and_filter - 1) == '\0')
> +			*(param_and_filter - 1) = ':';
> +		comment = strchr(param_and_filter, '#');
> +		if (comment)
> +			*comment = '\0';
> +	}
> +
> +	if (event_trigger_empty_param(param_and_filter)) {
> +		wprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND);
> +		return -EINVAL;
> +	}
> +
> +	ret = event_trigger_separate_filter(param_and_filter, &param, &filter, true);
> +	if (ret)
> +		return ret;
> +
> +	if (file->event_call->flags & TRACE_EVENT_FL_KPROBE) {
> +		wprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE);
> +		return -EOPNOTSUPP;
> +	}
> +
> +	event_str = strsep(&param, ":");

[Severity: Low]
Does mutating param_and_filter in place truncate the command string logged
by wprobe_trigger_log_err()?

param_and_filter points into the glob buffer.  Although the code restores
the colon at *(param_and_filter - 1), subsequent calls to
event_trigger_separate_filter(), strsep(&param, ":"), and *offs = '\0'
insert NUL terminators directly into glob.  Any later
wprobe_trigger_log_err(file, glob, ...) call for a bad field, offset, or
count logs only the truncated prefix of glob while passing an error caret
offset past the end of the logged string.  Also, when param_and_filter is
NULL (for example, writing "set_wprobe" with no colon), strlen(cmd) + 1
points one byte past the NUL terminator of glob.

> +	/* Find target wprobe */
> +	tw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);
> +	if (!tw) {
> +		wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
> +		return -ENOENT;
> +	}
> +	/* The target wprobe must not be used (unless clear) */
> +	if (!remove && !clear && trace_probe_is_enabled(&tw->tp)) {
> +		wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY);
> +		return -EBUSY;
> +	}

[Severity: Medium]
Does checking trace_probe_is_enabled(&tw->tp) here conflate manual enabling
with trigger soft-enabling?

Both set_wprobe and clear_wprobe call trace_event_enable_disable(wprobe_file,
1, 1), which sets TP_FLAG_TRACE on tw->tp and makes
trace_probe_is_enabled(&tw->tp) return true.  As a result, registering
clear_wprobe before set_wprobe, re-adding set_wprobe after removing it while
clear_wprobe is still attached, or attaching set_wprobe to a second event
all fail with -EBUSY.  When clear_wprobe is registered first, tw->addr is
also not initialized to WPROBE_DEFAULT_CLEAR_ADDRESS because that assignment
is gated on !clear.

Conversely, because clear_wprobe skips this busy check, it can be attached
to a wprobe that was manually enabled by the user (where sm_ref == 0); when
clear_wprobe fires or is later removed, it soft-disables or unregisters the
manually enabled wprobe.

[ ... ]

> +	if (!clear) {
> +		orig_addr = tw->addr;
> +		WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS);
> +	}
> +
> +	ret = trace_event_enable_disable(wprobe_file, 1, 1);

[Severity: Medium]
Does overwriting tw->addr without restoring it on trigger teardown break
trace_wprobe_show() and dynamic_events deletion?

When set_wprobe is registered and fires, tw->addr is overwritten first with
WPROBE_DEFAULT_CLEAR_ADDRESS and then with the runtime target address, and
removing the trigger leaves tw->addr at that runtime value.  For a wprobe
created with rw@-1:8, removing set_wprobe after it fires causes
trace_wprobe_show() to print the runtime address instead of -1, and
deleting the wprobe with "!w:watch rw@-1:8 ..." fails in
trace_wprobe_match_command_head() because tw->addr no longer equals
WPROBE_DEFAULT_CLEAR_ADDRESS.  For a symbol-based wprobe, tw->symbol remains
set while tw->addr is left clobbered, so trace_wprobe_show() still shows
@symbol while re-enabling the wprobe watches the wrong address.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/179480675130.38065.3086970856119381693.stgit@devnote2?part=8

  reply	other threads:[~2026-11-16  6:20 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  5:25 [PATCH v20 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
2026-11-16  5:26 ` [PATCH v20 01/11] x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API Masami Hiramatsu (Google)
2026-11-16  5:47   ` sashiko-bot
2026-11-16  5:26 ` [PATCH v20 02/11] HWBP: Add modify_local_hw_breakpoint_addr() API Masami Hiramatsu (Google)
2026-11-16  5:26 ` [PATCH v20 03/11] perf/hw_breakpoint: Add register_wide_hw_breakpoint_cpuslocked() API Masami Hiramatsu (Google)
2026-11-16  5:26 ` [PATCH v20 04/11] tracing/wprobe: Add wprobe (watchpoint probe) trace event support Masami Hiramatsu (Google)
2026-11-16  5:56   ` sashiko-bot
2026-11-16  5:26 ` [PATCH v20 05/11] x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires Masami Hiramatsu (Google)
2026-11-16  5:27 ` [PATCH v20 06/11] selftests: tracing: Add a basic testcase for wprobe Masami Hiramatsu (Google)
2026-11-16  5:27 ` [PATCH v20 07/11] selftests: tracing: Add syntax " Masami Hiramatsu (Google)
2026-11-16  5:27 ` [PATCH v20 08/11] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers Masami Hiramatsu (Google)
2026-11-16  6:20   ` sashiko-bot [this message]
2026-11-16  5:27 ` [PATCH v20 09/11] selftests: tracing: Add wprobe trigger testcases Masami Hiramatsu (Google)
2026-11-16  5:52   ` sashiko-bot
2026-11-16  5:27 ` [PATCH v20 10/11] tracing/wprobe: Support BTF typecast in fetchargs Masami Hiramatsu (Google)
2026-11-16  5:27 ` [PATCH v20 11/11] tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger Masami Hiramatsu (Google)
2026-11-16  5:50   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-216266@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=mhiramat@kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox