[RFC] ptwrite uprobes

[RFC] ptwrite uprobes

[Andi Kleen]

uprobes currently always require entering the kernel to log anything.
While that works well, it is rather slow.
    
Modern Intel CPUs have the ptwrite instruction, which can log data to
the Processor Trace buffer without entering the kernel. 

This patch adds support in uprobes to patch in ptwrites instead of
the normal probes. If a user collects Processor Trace with perf
the logged data will appear in the PT log, otherwise the instructions
will be nops. 
    
The benefit is much faster logging, but it also has a lot of
limitations. There is little filtering (other than what perf or
PT can do), no EBPF, there are restrictions on what can be logged,
and of course it depends on PT being recorded.

Still I find it useful.

For more details and performance numbers see the Documentation patch,
but it's multiple orders of magnitude faster than classic uprobes.

This patchkit touches a variety of areas: perf tools, x86 generic,
uprobes. It currently relies on a separately posted bug fix ("RCU safety
for maple tree iterators"). The patchkit is on the larger
side, and maybe it should be split up. But I wanted to keep it together
at least for the first post. Some of the earlier patches are generic
fixes for uprobes.

Comments appreciated.

-Andi Kleen

[RFC v1 01/19] uprobes: guard trace cleanup against error pointers

[Andi Kleen]

Sashiko pointed out the some of the scope cleanups for free_uprobe
could get an error pointer. Handle this case in free_uprobe
to prevent a crash.

On the other hand the macro doesn't need the guard because
free_uprobe itself already does the check.

Assisted-by: omp:gpt-5.6-luna sashiko
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 kernel/trace/trace_uprobe.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index 861d857adadb..22cc3c8181b8 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -368,7 +368,7 @@ alloc_trace_uprobe(const char *group, const char *event, int nargs, bool is_ret)
 
 static void free_trace_uprobe(struct trace_uprobe *tu)
 {
-	if (!tu)
+	if (IS_ERR_OR_NULL(tu))
 		return;
 
 	path_put(&tu->path);
@@ -533,7 +533,7 @@ static int register_trace_uprobe(struct trace_uprobe *tu)
 	return ret;
 }
 
-DEFINE_FREE(free_trace_uprobe, struct trace_uprobe *, if (_T) free_trace_uprobe(_T))
+DEFINE_FREE(free_trace_uprobe, struct trace_uprobe *, free_trace_uprobe(_T))
 
 /*
  * Argument syntax:
-- 
2.54.0

[RFC v1 02/19] uprobes: Correctly reject anonymous VMAs for breakpoint installation

[Andi Kleen]

Checking for anonymous VMAs is supposed to use this special function,
not just check vm_file.

Assisted-by: omp:gpt-5.6-luna sashiko
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 kernel/events/uprobes.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 7709ea882477..4287c4ff4c0f 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -139,7 +139,8 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)
 	if (is_register)
 		flags |= VM_WRITE;
 
-	return vma->vm_file && (vma->vm_flags & flags) == VM_MAYEXEC;
+	return !vma_is_anonymous(vma) && vma->vm_file &&
+		(vma->vm_flags & flags) == VM_MAYEXEC;
 }
 
 static unsigned long offset_to_vaddr(struct vm_area_struct *vma, loff_t offset)
-- 
2.54.0

[RFC v1 03/19] uprobes: Print warning for missing breakpoint install

[Andi Kleen]

When installing a uprobe break point fails the error is currently
silently ignored. There is no obvious place to return it to, but
at least print a rate-limited warning to make it easier to debug.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 kernel/events/uprobes.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 4287c4ff4c0f..941b52c47858 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1630,7 +1630,13 @@ int uprobe_mmap(struct vm_area_struct *vma)
 		if (!fatal_signal_pending(current) &&
 		    filter_chain(uprobe, vma->vm_mm)) {
 			unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
-			install_breakpoint(uprobe, vma, vaddr);
+			int err = install_breakpoint(uprobe, vma, vaddr);
+
+			if (err)
+				pr_warn_ratelimited(
+					"uprobes: probe %pD+0x%llx failed to install (%d)\n",
+					vma->vm_file,
+					(unsigned long long)uprobe->offset, err);
 		}
 		put_uprobe(uprobe);
 	}
-- 
2.54.0

[RFC v1 04/19] ptwrite uprobes: Add infrastructure for ptwrite uprobes

[Andi Kleen]

uprobes currently always require entering the kernel to log anything.
While that works well, it is rather slow.

Modern Intel CPUs have the ptwrite instruction, which can log data to
the Processor Trace buffer. This patch adds support in uprobes
to patch in ptwrites instead of the normal probes. If a user collects
Processor Trace with perf the logged data will appear in the PT log,
otherwise the instructions will be nops. ptwrite is a 5 byte
instruction, here it can be only patched into 5 byte nops.

The benefit is much faster logging, but it also has a lot of
limitations. There is no filtering, no EBPF, there are restrictions on
what can be logged, and of course it depends on PT being recorded.

The instrumentation is similar to normal uprobes. Add a trampoline
page. Replace the original instruction (5 byte nop) with a jump
to the trampoline. The trampoline does ptwrites and then jumps back.

This is the high level infrastruture without any x86-64 specific
parts (except for one data structure). Add register/unregister, basic data
structures and high level hooks. Add weak stubs to handle the no uprobes
or different architectures case.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 include/linux/uprobes.h |  81 +++++++++++++++++++-
 kernel/events/uprobes.c | 166 +++++++++++++++++++++++++++++++++++++++-
 kernel/fork.c           |   1 +
 mm/mmap.c               |   8 +-
 4 files changed, 251 insertions(+), 5 deletions(-)

diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index d34dbc0fbbfe..0c422f6d9e7f 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -23,9 +23,11 @@ struct uprobe;
 struct vm_area_struct;
 struct mm_struct;
 struct inode;
+struct file;
 struct notifier_block;
 struct page;
 struct srcu_ctr;
+struct uprobe_ptwrite_desc;
 
 /*
  * Allowed return values from uprobe consumer's handler callback
@@ -187,6 +189,37 @@ struct xol_area;
 
 struct uprobes_state {
 	struct xol_area		*xol_area;
+#ifdef CONFIG_X86_64
+	struct hlist_head	head_ptwrite;
+	/* Ptwrite pages and metadata use the mm mmap write lock. */
+#endif
+};
+
+#define UPROBE_PTWRITE_MAX_ARGS	8
+
+/*
+ * Header word: event_id<<48 | nargs<<40 | UPROBE_PTW_HDR_MAGIC (bits 39..0).
+ */
+#define UPROBE_PTW_HDR_MAGIC	0x5054525731UL	/* "PTRW1" */
+
+enum uprobe_ptwrite_src {
+	UPROBE_PTW_SRC_REG,	/* value = live GPR (index in .reg) */
+	UPROBE_PTW_SRC_IMM,	/* value = constant (.val), stored in stub data slot */
+};
+
+struct uprobe_ptwrite_arg {
+	u8	src;		/* enum uprobe_ptwrite_src */
+	u8	reg;		/* x86-64 GPR index (0=rax..15=r15) for SRC_REG */
+	u8	size;		/* declared type size 1/2/4/8 (decoder hint) */
+	u8	reserved;
+	u64	val;		/* SRC_IMM: constant; SRC_REG: unused */
+};
+
+struct uprobe_ptwrite_desc {
+	u16	event_id;	/* identifier carried in the header word */
+	u8	nargs;
+	u8	flags;
+	struct uprobe_ptwrite_arg args[UPROBE_PTWRITE_MAX_ARGS];
 };
 
 typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vaddr,
@@ -205,6 +238,37 @@ extern int uprobe_write(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
 			uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, bool is_register, bool do_update_ref_ctr,
 			void *data);
 extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, loff_t ref_ctr_offset, struct uprobe_consumer *uc);
+extern struct uprobe *uprobe_register_ptwrite(struct inode *inode,
+					      struct file *file, loff_t offset,
+					      struct uprobe_consumer *uc,
+					      const struct uprobe_ptwrite_desc *desc);
+extern bool arch_uprobe_ptwrite_supported(void);
+extern int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
+				       const struct uprobe_ptwrite_desc *desc);
+extern int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
+				       struct vm_area_struct *vma,
+				       unsigned long vaddr);
+extern int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
+					 struct vm_area_struct *vma,
+					 unsigned long vaddr);
+
+enum uprobe_ptwrite_fetch_kind {
+	UPROBE_PTW_FETCH_REG,	/* live GPR */
+	UPROBE_PTW_FETCH_STACKP,/* stack pointer value ($stack) */
+	UPROBE_PTW_FETCH_STACKN,/* [SP + imm] ($stackN, imm pre-scaled) */
+	UPROBE_PTW_FETCH_MEMREG,/* [GPR + imm] (imm = disp32) */
+	UPROBE_PTW_FETCH_IMM,	/* constant */
+};
+
+struct uprobe_ptwrite_fetch {
+	enum uprobe_ptwrite_fetch_kind	kind;
+	unsigned int			reg;	/* pt_regs member offset */
+	u64				imm;	/* IMM value / MEMREG disp / STACKN off */
+};
+
+extern int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
+				     const struct uprobe_ptwrite_fetch *f);
+
 extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc, bool);
 extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_consumer *uc);
 extern void uprobe_unregister_sync(void);
@@ -212,7 +276,7 @@ extern int uprobe_mmap(struct vm_area_struct *vma);
 extern void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end);
 extern void uprobe_start_dup_mmap(void);
 extern void uprobe_end_dup_mmap(void);
-extern void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
+extern int uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
 extern void uprobe_free_utask(struct task_struct *t);
 extern void uprobe_copy_process(struct task_struct *t, u64 flags);
 extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
@@ -236,6 +300,9 @@ extern void uprobe_handle_trampoline(struct pt_regs *regs);
 extern void *arch_uretprobe_trampoline(unsigned long *psize);
 extern unsigned long uprobe_get_trampoline_vaddr(void);
 extern void uprobe_copy_from_page(struct page *page, unsigned long vaddr, void *dst, int len);
+extern void arch_uprobe_clear_state(struct mm_struct *mm);
+extern void arch_uprobe_init_state(struct mm_struct *mm);
+extern int arch_uprobe_dup_ptwrite(struct mm_struct *oldmm, struct mm_struct *newmm);
 extern void handle_syscall_uprobe(struct pt_regs *regs, unsigned long bp_vaddr);
 extern void arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned long vaddr);
 extern unsigned long arch_uprobe_get_xol_area(void);
@@ -254,6 +321,13 @@ uprobe_register(struct inode *inode, loff_t offset, loff_t ref_ctr_offset, struc
 {
 	return ERR_PTR(-ENOSYS);
 }
+static inline struct uprobe *
+uprobe_register_ptwrite(struct inode *inode, struct file *file, loff_t offset,
+			struct uprobe_consumer *uc,
+			const struct uprobe_ptwrite_desc *desc)
+{
+	return ERR_PTR(-ENOSYS);
+}
 static inline int
 uprobe_apply(struct uprobe* uprobe, struct uprobe_consumer *uc, bool add)
 {
@@ -280,9 +354,10 @@ static inline void uprobe_start_dup_mmap(void)
 static inline void uprobe_end_dup_mmap(void)
 {
 }
-static inline void
-uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
+static inline int uprobe_dup_mmap(struct mm_struct *oldmm,
+				  struct mm_struct *newmm)
 {
+	return 0;
 }
 static inline void uprobe_notify_resume(struct pt_regs *regs)
 {
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 941b52c47858..23202df2b51a 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -59,6 +59,8 @@ DEFINE_STATIC_SRCU_FAST_UPDOWN(uretprobes_srcu);
 /* Have a copy of original instruction */
 #define UPROBE_COPY_INSN	0
 
+#define UPROBE_PTWRITE		1
+
 struct uprobe {
 	struct rb_node		rb_node;	/* node in the rb tree */
 	refcount_t		ref;
@@ -1163,6 +1165,18 @@ static int install_breakpoint(struct uprobe *uprobe, struct vm_area_struct *vma,
 	if (ret)
 		return ret;
 
+	if (test_bit(UPROBE_PTWRITE, &uprobe->flags)) {
+		first_uprobe = !mm_flags_test(MMF_HAS_UPROBES, mm);
+		if (first_uprobe)
+			mm_flags_set(MMF_HAS_UPROBES, mm);
+
+		ret = arch_uprobe_install_ptwrite(&uprobe->arch, vma, vaddr);
+		if (!ret)
+			mm_flags_clear(MMF_RECALC_UPROBES, mm);
+		else if (first_uprobe)
+			mm_flags_clear(MMF_HAS_UPROBES, mm);
+		return ret;
+	}
 	/*
 	 * set MMF_HAS_UPROBES in advance for uprobe_pre_sstep_notifier(),
 	 * the task can hit this breakpoint right after __replace_page().
@@ -1186,6 +1200,9 @@ static int remove_breakpoint(struct uprobe *uprobe, struct vm_area_struct *vma,
 	struct mm_struct *mm = vma->vm_mm;
 
 	mm_flags_set(MMF_RECALC_UPROBES, mm);
+	if (test_bit(UPROBE_PTWRITE, &uprobe->flags))
+		return arch_uprobe_uninstall_ptwrite(&uprobe->arch, vma, vaddr);
+
 	return set_orig_insn(&uprobe->arch, vma, vaddr);
 }
 
@@ -1424,6 +1441,17 @@ struct uprobe *uprobe_register(struct inode *inode,
 		return uprobe;
 
 	down_write(&uprobe->register_rwsem);
+	/*
+	 * A dying deferred-removal ptwrite uprobe can make reuse temporarily
+	 * busy.
+	*/
+	if (test_bit(UPROBE_PTWRITE, &uprobe->flags)) {
+		ret = -EBUSY;
+		up_write(&uprobe->register_rwsem);
+		put_uprobe(uprobe);
+		return ERR_PTR(ret);
+	}
+
 	consumer_add(uprobe, uc);
 	ret = register_for_each_vma(uprobe, uc);
 	up_write(&uprobe->register_rwsem);
@@ -1443,6 +1471,138 @@ struct uprobe *uprobe_register(struct inode *inode,
 }
 EXPORT_SYMBOL_GPL(uprobe_register);
 
+/*
+ * Architecture state hooks and ptwrite hooks: weak defaults so the
+ * generic core builds on any architecture.
+ */
+void __weak arch_uprobe_init_state(struct mm_struct *mm)
+{
+}
+
+void __weak arch_uprobe_clear_state(struct mm_struct *mm)
+{
+}
+
+/*
+ * ptwrite arch hooks: weak defaults so the generic core builds on any
+ * architecture.
+ */
+bool __weak arch_uprobe_ptwrite_supported(void)
+{
+	return false;
+}
+
+int __weak arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
+				       const struct uprobe_ptwrite_desc *desc)
+{
+	return -EOPNOTSUPP;
+}
+
+int __weak arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
+				       struct vm_area_struct *vma,
+				       unsigned long vaddr)
+{
+	return -EOPNOTSUPP;
+}
+
+int __weak arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
+					  struct vm_area_struct *vma,
+					  unsigned long vaddr)
+{
+	return 0;
+}
+
+int __weak arch_uprobe_dup_ptwrite(struct mm_struct *oldmm, struct mm_struct *newmm)
+{
+	return 0;
+}
+
+int __weak arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
+					 const struct uprobe_ptwrite_fetch *f)
+{
+	return -EOPNOTSUPP;
+}
+
+/**
+ * uprobe_register_ptwrite - register a PTWRITE uprobe
+ * @inode: the probed file's inode
+ * @file: open file used while populating the instruction page cache
+ * @offset: offset from the start of the file
+ * @uc: consumer (handler is never invoked: no kernel entry at probe hit)
+ * @desc: requested values to emit
+ */
+struct uprobe *uprobe_register_ptwrite(struct inode *inode, struct file *file,
+				       loff_t offset, struct uprobe_consumer *uc,
+				       const struct uprobe_ptwrite_desc *desc)
+{
+	struct uprobe *uprobe;
+	int ret;
+
+	if (!file || (!uc->handler && !uc->ret_handler))
+		return ERR_PTR(-EINVAL);
+
+	if (!arch_uprobe_ptwrite_supported())
+		return ERR_PTR(-EOPNOTSUPP);
+
+	if (!desc || desc->nargs == 0 || desc->nargs > UPROBE_PTWRITE_MAX_ARGS)
+		return ERR_PTR(-EINVAL);
+
+	if (!inode->i_mapping->a_ops->read_folio &&
+	    !shmem_mapping(inode->i_mapping))
+		return ERR_PTR(-EIO);
+
+	/* Racy, just to catch the obvious mistakes */
+	if (offset < 0)
+		return ERR_PTR(-EINVAL);
+	if (offset > i_size_read(inode))
+		return ERR_PTR(-EINVAL);
+	if (!IS_ALIGNED(offset, UPROBE_SWBP_INSN_SIZE))
+		return ERR_PTR(-EINVAL);
+
+	uprobe = alloc_uprobe(inode, offset, 0);
+	if (IS_ERR(uprobe))
+		return uprobe;
+
+	down_write(&uprobe->register_rwsem);
+
+	/*
+	 * A dying normal uprobe can make reuse temporarily busy; don't overwrite
+	 * it.
+	*/
+	if (!list_empty(&uprobe->consumers)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	/* Build the mm-independent stub template once, at registration. */
+	ret = arch_uprobe_ptwrite_prepare(&uprobe->arch, desc);
+	if (ret)
+		goto out;
+
+
+	set_bit(UPROBE_PTWRITE, &uprobe->flags);
+	consumer_add(uprobe, uc);
+	ret = register_for_each_vma(uprobe, uc);
+	up_write(&uprobe->register_rwsem);
+
+	if (ret) {
+		uprobe_unregister_nosync(uprobe, uc);
+		/*
+		 * Registration might have partially succeeded. Clean
+		 * everything up.
+		 */
+		uprobe_unregister_sync();
+		return ERR_PTR(ret);
+	}
+
+	return uprobe;
+out:
+	up_write(&uprobe->register_rwsem);
+	put_uprobe(uprobe);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(uprobe_register_ptwrite);
+
 /**
  * uprobe_apply - add or remove the breakpoints according to @uc->filter
  * @uprobe: uprobe which "owns" the breakpoint
@@ -1827,6 +1987,8 @@ void uprobe_clear_state(struct mm_struct *mm)
 	delayed_uprobe_remove(NULL, mm);
 	mutex_unlock(&delayed_uprobe_lock);
 
+	arch_uprobe_clear_state(mm);
+
 	if (!area)
 		return;
 
@@ -1845,13 +2007,15 @@ void uprobe_end_dup_mmap(void)
 	percpu_up_read(&dup_mmap_sem);
 }
 
-void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
+int uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
 {
 	if (mm_flags_test(MMF_HAS_UPROBES, oldmm)) {
 		mm_flags_set(MMF_HAS_UPROBES, newmm);
 		/* unconditionally, dup_mmap() skips VM_DONTCOPY vmas */
 		mm_flags_set(MMF_RECALC_UPROBES, newmm);
 	}
+
+	return arch_uprobe_dup_ptwrite(oldmm, newmm);
 }
 
 static unsigned long xol_get_slot_nr(struct xol_area *area)
diff --git a/kernel/fork.c b/kernel/fork.c
index 416758c8a3d4..7cda19be2877 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1076,6 +1076,7 @@ static void mm_init_uprobes_state(struct mm_struct *mm)
 {
 #ifdef CONFIG_UPROBES
 	mm->uprobes_state.xol_area = NULL;
+	arch_uprobe_init_state(mm);
 #endif
 }
 
diff --git a/mm/mmap.c b/mm/mmap.c
index 4bf26b0f1e6e..e10412160b32 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1716,7 +1716,6 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
 	if (mmap_write_lock_killable(oldmm))
 		return -EINTR;
 	flush_cache_dup_mm(oldmm);
-	uprobe_dup_mmap(oldmm, mm);
 	/*
 	 * Not linked in yet - no deadlock potential:
 	 */
@@ -1825,6 +1824,13 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
 	}
 	/* a new mm has just been created */
 	retval = arch_dup_mmap(oldmm, mm);
+	if (!retval) {
+		/*
+		 * The arch state follows the fully populated child maple tree. A
+		 * non-fatal allocation failure can leave a child ptwrite hit faulting.
+		 */
+		retval = uprobe_dup_mmap(oldmm, mm);
+	}
 loop_out:
 	vma_iter_free(&vmi);
 	if (!retval) {
-- 
2.54.0

[RFC v1 05/19] ptwrite uprobes: Add minimal low level support for x86

[Andi Kleen]

Add more data structures and the x86 machinery to generate the PTWRITE
instructions for a ptwrite uprobe. The probe executes PTWRITEs and then
jumps back to the original code. In this variant only patching
5 byte nops is supported.

The instructions are pre-generated to templates and then patched when
setting up the final user page.

The patching code uses 3 phase patching similar to int3_update.

The ptwrite stub emits a header with a magic value and the number of
arguments, and then the actual probed values.

There is no separate config option for ptwrite uprobes, it is just tied
to the main uprobes config.

Some limitations in the current implementation:
- The probed 5 byte area cannot cross a page.
- The allocated stubs in the user program are only freed on exit.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/uprobes.h |  24 ++
 arch/x86/kernel/uprobes.c      | 623 +++++++++++++++++++++++++++++++++
 2 files changed, 647 insertions(+)

diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 362210c79998..05f8a7ea93a6 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -23,10 +23,33 @@ typedef u8 uprobe_opcode_t;
 enum {
 	ARCH_UPROBE_FLAG_CAN_OPTIMIZE   = 0,
 	ARCH_UPROBE_FLAG_OPTIMIZE_FAIL  = 1,
+	ARCH_UPROBE_FLAG_PTWRITE        = 2,
 };
 
 struct uprobe_xol_ops;
 
+/*
+ * ptwrite probe state. The stub template (code + data slots) is built
+ * once at registration (mm-independent except the final jmp's rel32, patched
+ * per-mm at install). Block layout:
+ *   [ptwriteq hdr(%rip)] [arg emissions] [jmp probe+5] [u64 slots: header, imms]
+ */
+struct uprobe_ptwrite_arch {
+	u8	stub[256];
+	u8	stub_len;	/* code + data, whole block */
+	u8	jmp_off;	/* offset of the final jmp's rel32 field */
+	u8	ndata;		/* number of u64 data slots */
+	u8	orig[MAX_UINSN_BYTES];	/* pristine file bytes, before generic analysis */
+};
+
+/* Per-mm page holding generated ptwrite stub blocks (mirrors trampolines). */
+struct uprobe_ptwrite_page {
+	struct hlist_node	node;
+	struct page		*page;		/* stub blocks written via kmap */
+	unsigned long		vaddr;		/* mapping base */
+	u16			cursor;		/* next free block offset */
+};
+
 struct arch_uprobe {
 	union {
 		u8			insn[MAX_UINSN_BYTES];
@@ -51,6 +74,7 @@ struct arch_uprobe {
 		}			push;
 	};
 
+	struct uprobe_ptwrite_arch	ptwrite;
 	unsigned long flags;
 };
 
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 65a2de82ecd2..df652c56414b 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -15,11 +15,15 @@
 #include <linux/syscalls.h>
 
 #include <linux/kdebug.h>
+#include <linux/highmem.h>
+#include <linux/mm.h>
 #include <asm/processor.h>
 #include <asm/insn.h>
 #include <asm/insn-eval.h>
 #include <asm/mmu_context.h>
 #include <asm/nops.h>
+#include <asm/cpufeature.h>
+#include <asm/cpuid/api.h>
 
 /* Post-execution fixups. */
 
@@ -717,6 +721,68 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign
 	return _install_special_mapping(mm, vaddr, PAGE_SIZE,
 				VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
 				&tramp_mapping);
+
+
+}
+
+void arch_uprobe_init_state(struct mm_struct *mm)
+{
+	INIT_HLIST_HEAD(&mm->uprobes_state.head_ptwrite);
+}
+
+void arch_uprobe_clear_state(struct mm_struct *mm)
+{
+	struct uprobes_state *state = &mm->uprobes_state;
+	struct uprobe_ptwrite_page *ptw;
+	struct hlist_node *n;
+
+	hlist_for_each_entry_safe(ptw, n, &state->head_ptwrite, node) {
+		hlist_del_rcu(&ptw->node);
+		synchronize_rcu();
+		__free_page(ptw->page);
+		kfree(ptw);
+	}
+}
+
+int arch_uprobe_dup_ptwrite(struct mm_struct *oldmm, struct mm_struct *newmm)
+{
+	struct uprobes_state *old_state = &oldmm->uprobes_state;
+	struct uprobes_state *new_state = &newmm->uprobes_state;
+	struct uprobe_ptwrite_page *ptw, *new;
+
+	mmap_assert_write_locked(oldmm);
+	mmap_assert_write_locked(newmm);
+	hlist_for_each_entry(ptw, &old_state->head_ptwrite, node) {
+		void *src, *dst;
+
+		new = kzalloc_obj(*new);
+		if (!new)
+			goto fail;
+		new->page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!new->page) {
+			kfree(new);
+			goto fail;
+		}
+
+		src = kmap_local_page(ptw->page);
+		dst = kmap_local_page(new->page);
+		memcpy(dst, src, PAGE_SIZE);
+		kunmap_local(dst);
+		kunmap_local(src);
+		new->vaddr = ptw->vaddr;
+		new->cursor = ptw->cursor;
+		new->nblocks = ptw->nblocks;
+		memcpy(new->index, ptw->index, sizeof(new->index));
+		/* Publish the copied page fields before RCU readers can find it. */
+		smp_wmb();
+		hlist_add_head_rcu(&new->node, &new_state->head_ptwrite);
+	}
+
+	return 0;
+
+fail:
+	arch_uprobe_clear_state(newmm);
+	return -ENOMEM;
 }
 
 static bool __in_uprobe_trampoline(struct mm_struct *mm, unsigned long ip)
@@ -869,11 +935,13 @@ enum {
 	EXPECT_SWBP,
 	EXPECT_OPTIMIZED,
 	EXPECT_SWBP_OPTIMIZED,
+	EXPECT_BYTE,
 };
 
 struct write_opcode_ctx {
 	unsigned long base;
 	int expect;
+	u8 expect_byte;
 };
 
 /*
@@ -901,6 +969,10 @@ static int verify_insn(struct page *page, unsigned long vaddr, uprobe_opcode_t *
 		if (is_swbp_opt_insns(&old_opcode[0]))
 			return 1;
 		break;
+	case EXPECT_BYTE:
+		if (old_opcode[0] == ctx->expect_byte)
+			return 1;
+		break;
 	}
 
 	return -1;
@@ -1064,6 +1136,55 @@ static int int3_update_unoptimize(struct arch_uprobe *auprobe, struct vm_area_st
 	return 0;
 }
 
+/*
+ * Modify a five-byte instruction by using INT3 breakpoints on SMP.
+ * The caller supplies the byte expected before the update and controls
+ * whether the anonymous page and reference counter are updated on the
+ * final write.
+ */
+static int text_poke_5byte(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
+				   unsigned long vaddr, u8 *new5, u8 expect_byte,
+				   bool skip_int3, bool is_register, bool final_is_register,
+				   bool do_update_ref_ctr, bool *first_phase_done)
+{
+	uprobe_opcode_t int3 = UPROBE_SWBP_INSN;
+	struct write_opcode_ctx ctx = {
+		.base = vaddr,
+		.expect = EXPECT_BYTE,
+		.expect_byte = expect_byte,
+	};
+	int err;
+
+	if (first_phase_done)
+		*first_phase_done = skip_int3;
+	if (!skip_int3) {
+		err = uprobe_write(auprobe, vma, vaddr, &int3, 1, verify_insn,
+				   is_register, false, &ctx);
+		if (err)
+			return err;
+		if (first_phase_done)
+			*first_phase_done = true;
+	}
+
+	smp_text_poke_sync_each_cpu();
+
+	ctx.expect = EXPECT_SWBP;
+	err = uprobe_write(auprobe, vma, vaddr + 1, new5 + 1, 4, verify_insn,
+			   is_register, false, &ctx);
+	if (err)
+		return err;
+
+	smp_text_poke_sync_each_cpu();
+
+	err = uprobe_write(auprobe, vma, vaddr, new5, 1, verify_insn,
+			   final_is_register, do_update_ref_ctr, &ctx);
+	if (err)
+		return err;
+
+	smp_text_poke_sync_each_cpu();
+	return 0;
+}
+
 static int swbp_optimize(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
 			 unsigned long vaddr, unsigned long tramp)
 {
@@ -1102,6 +1223,508 @@ static int copy_from_vaddr(struct mm_struct *mm, unsigned long vaddr, void *dst,
 	return 0;
 }
 
+/*
+ * ptwrite uprobes: trap-free user-mode instrumentation.
+ *
+ * Block layout (mm-independent template, built at registration):
+ *   ptwriteq hdr(%rip)      ; header: event_id<<48 | nargs<<40 | magic
+ *   ptwriteq %reg / imm(%rip)   ; one per arg
+ *   jmp probe+5             ; rel32 patched per-mm at install
+ *   [u64 slots: header, imm values]
+ */
+
+static int ptwrite_emit_reg(u8 *p, u8 reg)
+{
+	/* ptwriteq %reg : F3 REX.W[.B] 0F AE /4, modrm = 11 100 rrr */
+	*p++ = 0xf3;
+	*p++ = (reg >= 8) ? 0x49 : 0x48;	/* REX.W, +REX.B for r8-r15 */
+	*p++ = 0x0f;
+	*p++ = 0xae;
+	*p++ = 0xe0 | (reg & 7);
+	return 5;
+}
+
+static int ptwrite_emit_riprel(u8 *p, s32 disp)
+{
+	/*
+	 * ptwriteq disp32(%rip) : F3 48 0F AE 25 <disp32> (9 bytes)
+	 * modrm 0x25 = mod 00, reg 100 (/4, PTWRITE), rm 101 (RIP-relative).
+	 */
+	*p++ = 0xf3;
+	*p++ = 0x48;
+	*p++ = 0x0f;
+	*p++ = 0xae;
+	*p++ = 0x25;
+	memcpy(p, &disp, 4);
+	return 9;
+}
+
+bool arch_uprobe_ptwrite_supported(void)
+{
+	u32 eax, ebx, ecx, edx;
+
+	if (!boot_cpu_has(X86_FEATURE_INTEL_PT))
+		return false;
+	if (boot_cpu_data.cpuid_level < 0x14)
+		return false;
+
+	/* CPUID.(EAX=14H, ECX=0):EBX[4] = PTWRITE */
+	cpuid_count(0x14, 0, &eax, &ebx, &ecx, &edx);
+	return ebx & BIT(4);
+}
+
+/*
+ * x86-64 pt_regs member offset -> GPR index (0=rax..15=r15), matching
+ * the uprobe_ptwrite_arg.reg convention used by the stub generator.
+ * The offsets are what the generic trace-probe register parser
+ * (regs_query_register_offset) puts into FETCH_OP_REG.params.
+ */
+static const struct {
+	unsigned int off;
+	u8 idx;
+} ptwrite_reg_map[] = {
+	{ offsetof(struct pt_regs, ax), 0 }, { offsetof(struct pt_regs, cx), 1 },
+	{ offsetof(struct pt_regs, dx), 2 }, { offsetof(struct pt_regs, bx), 3 },
+	{ offsetof(struct pt_regs, sp), 4 }, { offsetof(struct pt_regs, bp), 5 },
+	{ offsetof(struct pt_regs, si), 6 }, { offsetof(struct pt_regs, di), 7 },
+	{ offsetof(struct pt_regs, r8), 8 }, { offsetof(struct pt_regs, r9), 9 },
+	{ offsetof(struct pt_regs, r10), 10 }, { offsetof(struct pt_regs, r11), 11 },
+	{ offsetof(struct pt_regs, r12), 12 }, { offsetof(struct pt_regs, r13), 13 },
+	{ offsetof(struct pt_regs, r14), 14 }, { offsetof(struct pt_regs, r15), 15 },
+};
+
+/* Compile the register, stack-pointer, and immediate fetch forms. */
+int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
+			      const struct uprobe_ptwrite_fetch *f)
+{
+	int j, idx = -1;
+
+	switch (f->kind) {
+	case UPROBE_PTW_FETCH_REG:
+		for (j = 0; j < ARRAY_SIZE(ptwrite_reg_map); j++)
+			if (ptwrite_reg_map[j].off == f->reg) {
+				idx = ptwrite_reg_map[j].idx;
+				break;
+			}
+		if (idx < 0)
+			return -EINVAL;
+		a->src = UPROBE_PTW_SRC_REG;
+		a->reg = idx;
+		break;
+	case UPROBE_PTW_FETCH_STACKP:
+		a->src = UPROBE_PTW_SRC_REG;
+		a->reg = 4; /* rsp */
+		break;
+	case UPROBE_PTW_FETCH_IMM:
+		a->src = UPROBE_PTW_SRC_IMM;
+		a->val = f->imm;
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
+				const struct uprobe_ptwrite_desc *desc)
+{
+	struct uprobe_ptwrite_arch *ptw = &auprobe->ptwrite;
+	u8 *code = ptw->stub, *p = ptw->stub;
+	u16 imm_off[UPROBE_PTWRITE_MAX_ARGS];
+	unsigned int data_off;
+	unsigned int hdr_off = 0;
+	unsigned int imm_idx = 0, n_imm = 0;
+	u64 hdr;
+	int i;
+
+	if (!desc || desc->nargs == 0)
+		return -EINVAL;
+	if (desc->nargs > UPROBE_PTWRITE_MAX_ARGS)
+		return -E2BIG;
+	if (desc->flags)
+		return -EINVAL;
+
+	/* The generic registration path copied these bytes before this hook. */
+	memcpy(ptw->orig, auprobe->insn, sizeof(ptw->orig));
+
+	for (i = 0; i < desc->nargs; i++) {
+		switch (desc->args[i].src) {
+		case UPROBE_PTW_SRC_REG:
+			if (desc->args[i].reg > 15)
+				return -EINVAL;
+			break;
+		case UPROBE_PTW_SRC_IMM:
+			if (n_imm >= ARRAY_SIZE(imm_off))
+				return -E2BIG;
+			n_imm++;
+			break;
+		default:
+			return -EINVAL;
+		}
+	}
+
+	/* header word emission (disp32 patched below) */
+	p += ptwrite_emit_riprel(p, 0);
+
+	for (i = 0; i < desc->nargs; i++) {
+		if (desc->args[i].src == UPROBE_PTW_SRC_REG) {
+			p += ptwrite_emit_reg(p, desc->args[i].reg);
+		} else {
+			imm_off[imm_idx++] = p - code;
+			p += ptwrite_emit_riprel(p, 0);
+		}
+	}
+
+	/* final jmp back to probe+5; rel32 patched per-mm at install */
+	*p++ = 0xe9;
+	if (p - code > U8_MAX)
+		return -E2BIG;
+	ptw->jmp_off = p - code;
+	p += 4;
+
+	data_off = (p - code + 7) & ~7UL;
+	if (data_off + 8 * (1 + n_imm) > sizeof(ptw->stub))
+		return -E2BIG;
+
+	/* data slots: header, then imm values in emission order */
+	hdr = ((u64)desc->event_id << 48) | ((u64)desc->nargs << 40);
+	*(u64 *)(code + data_off) = hdr;
+
+	/* patch the header's disp32: hdr slot - end of header insn */
+	*(s32 *)(code + hdr_off + 5) = (s32)(data_off - (hdr_off + 9));
+
+	imm_idx = 0;
+	for (i = 0; i < desc->nargs; i++) {
+		if (desc->args[i].src != UPROBE_PTW_SRC_IMM)
+			continue;
+		*(s32 *)(code + imm_off[imm_idx] + 5) =
+			(s32)((data_off + 8 * (1 + imm_idx)) - (imm_off[imm_idx] + 9));
+		*(u64 *)(code + data_off + 8 * (1 + imm_idx)) = desc->args[i].val;
+		imm_idx++;
+	}
+
+	ptw->stub_len = data_off + 8 * (1 + n_imm);
+	ptw->ndata = 1 + n_imm;
+	return 0;
+}
+#undef PTW_NEED
+
+static vm_fault_t ptwrite_fault(const struct vm_special_mapping *sm,
+				struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+	struct uprobes_state *state = &vma->vm_mm->uprobes_state;
+	struct uprobe_ptwrite_page *ptw;
+
+	rcu_read_lock();
+	hlist_for_each_entry_rcu(ptw, &state->head_ptwrite, node) {
+		if (ptw->vaddr == vma->vm_start) {
+			vmf->page = ptw->page;
+			get_page(vmf->page);
+			rcu_read_unlock();
+			return 0;
+		}
+	}
+	rcu_read_unlock();
+	return VM_FAULT_SIGBUS;
+}
+
+static int ptwrite_mremap(const struct vm_special_mapping *sm,
+			  struct vm_area_struct *new_vma)
+{
+	return -EPERM;
+}
+
+static const struct vm_special_mapping ptwrite_mapping = {
+	.name	= "[uprobes-ptwrite]",
+	.fault	= ptwrite_fault,
+	.mremap	= ptwrite_mremap,
+};
+
+static bool __in_uprobe_ptwrite(struct mm_struct *mm, unsigned long ip)
+{
+	struct vm_area_struct *vma = vma_lookup(mm, ip);
+
+	return vma && vma_is_special_mapping(vma, &ptwrite_mapping);
+}
+
+
+/*
+ * Find a free PAGE_SIZE area in @mm within +/-2GB of the probe (so the jmp
+ * rel32 at the probe can reach the stub). Caller holds mmap_write_lock(mm).
+ */
+static unsigned long find_ptwrite_page_area(struct mm_struct *mm,
+					    unsigned long vaddr)
+{
+	VMA_ITERATOR(vmi, mm, 0);
+	struct vm_area_struct *vma;
+	unsigned long low, high, prev, call_end;
+	const unsigned long call_range = (unsigned long)INT_MAX + 1;
+
+	mmap_assert_write_locked(mm);
+	if (check_add_overflow(vaddr, 5UL, &call_end))
+		return -ENOMEM;
+	if (call_end < call_range)
+		low = PAGE_SIZE;
+	else
+		low = call_end - call_range;
+	if (low < PAGE_SIZE)
+		low = PAGE_SIZE;
+	if (low > ULONG_MAX - (PAGE_SIZE - 1))
+		return -ENOMEM;
+	low = PAGE_ALIGN(low);
+
+	if (check_add_overflow(call_end, (unsigned long)INT_MAX, &high))
+		high = ULONG_MAX;
+	high = min(high, TASK_SIZE_MAX);
+	if (low >= high)
+		return -ENOMEM;
+
+	prev = low;
+	for_each_vma(vmi, vma) {
+		if (vma->vm_start >= high)
+			break;
+		if (vma->vm_end <= prev)
+			continue;
+		if (vma->vm_start > prev && vma->vm_start - prev >= PAGE_SIZE)
+			return prev;
+		if (vma->vm_end > prev) {
+			if (vma->vm_end > ULONG_MAX - (PAGE_SIZE - 1))
+				return -ENOMEM;
+			prev = PAGE_ALIGN(vma->vm_end);
+			if (prev >= high)
+				return -ENOMEM;
+		}
+	}
+	if (prev < high && high - prev >= PAGE_SIZE)
+		return prev;
+	return -ENOMEM;
+}
+
+static struct uprobe_ptwrite_page *
+create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
+{
+	struct uprobe_ptwrite_page *ptw;
+	struct vm_area_struct *vma;
+	unsigned long area;
+
+	area = find_ptwrite_page_area(mm, vaddr);
+	if (IS_ERR_VALUE(area))
+		return NULL;
+
+	mmap_assert_write_locked(mm);
+
+	ptw = kzalloc_obj(*ptw);
+	if (!ptw)
+		return NULL;
+
+	ptw->page = alloc_page(GFP_HIGHUSER | __GFP_ZERO);
+	if (!ptw->page) {
+		kfree(ptw);
+		return NULL;
+	}
+	ptw->vaddr = area;
+
+	vma = _install_special_mapping(mm, area, PAGE_SIZE,
+			VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
+			&ptwrite_mapping);
+	if (IS_ERR(vma)) {
+		__free_page(ptw->page);
+		kfree(ptw);
+		return NULL;
+	}
+	return ptw;
+}
+static struct uprobe_ptwrite_page *
+get_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr,
+			unsigned int len)
+{
+	struct uprobes_state *state = &mm->uprobes_state;
+	struct uprobe_ptwrite_page *ptw;
+	mmap_assert_write_locked(mm);
+
+	/* a block larger than a page can never be placed */
+	if (len > PAGE_SIZE)
+		return NULL;
+
+	hlist_for_each_entry(ptw, &state->head_ptwrite, node)
+		if (is_reachable_by_call(ptw->vaddr + ptw->cursor, vaddr) &&
+		    ptw->cursor + len <= PAGE_SIZE)
+			return ptw;
+
+	/* no reachable page with room: allocate a fresh one (cursor 0) */
+	ptw = create_uprobe_ptwrite_page(mm, vaddr);
+	if (!ptw)
+		return NULL;
+	/* Order page initialization before publishing the page on the RCU list. */
+	smp_wmb();
+
+	hlist_add_head_rcu(&ptw->node, &state->head_ptwrite);
+	return ptw;
+}
+
+/* Probe site must be a 5-byte NOP that does not cross a page boundary. */
+static int ptwrite_validate_site(const u8 *orig, unsigned long vaddr)
+{
+	struct insn insn;
+	int ret;
+	int off = 0;
+
+	/*
+	 * The 5 displaced bytes must be NOPs: either one 5-byte NOP
+	 * (nopl 0x0(%rax,%rax,1)) or a run of shorter NOPs summing to
+	 * exactly 5 (gcc -fpatchable-function-entry=5 emits 5 x 0x90 on
+	 * modern toolchains). Any non-NOP byte, or a NOP crossing the
+	 * 5-byte window, is rejected.
+	 */
+	while (off < 5) {
+		ret = insn_decode(&insn, orig + off, 5 - off, INSN_MODE_64);
+		if (ret < 0)
+			return -EINVAL;
+		if (insn.length < 1 || insn.length > 5 - off ||
+		    !insn_is_nop(&insn))
+			return -EINVAL;
+		off += insn.length;
+	}
+	if (off != 5)
+		return -EINVAL;
+	if (PAGE_SIZE - (vaddr & ~PAGE_MASK) < 5)
+		return -EINVAL;
+	return 0;
+}
+
+static bool ptwrite_rel32(unsigned long from, unsigned long to, s32 *rel)
+{
+	s64 delta = (s64)to - (s64)from;
+
+	if (delta < INT_MIN || delta > INT_MAX)
+		return false;
+	*rel = (s32)delta;
+	return true;
+}
+
+static bool ptwrite_is_installed(struct mm_struct *mm, unsigned long vaddr,
+				 const u8 *insn5)
+{
+	struct __packed __arch_relative_insn {
+		u8 op;
+		s32 raddr;
+	} *jmp = (struct __arch_relative_insn *)insn5;
+	s64 target;
+
+	if (jmp->op != 0xe9)
+		return false;
+	target = (s64)vaddr + 5 + (s64)jmp->raddr;
+	if (target < PAGE_SIZE || target >= TASK_SIZE_MAX)
+		return false;
+	return __in_uprobe_ptwrite(mm, (unsigned long)target);
+}
+
+/*
+ * Install a JMP rel32 at the probe site using the 3-phase SMP-safe poke.
+ * On failure, restores the original instruction so the site is never
+ * left half-poked.
+ */
+static int ptwrite_text_poke(struct arch_uprobe *auprobe,
+			     struct vm_area_struct *vma, unsigned long vaddr,
+			     unsigned long stub_addr)
+{
+	u8 jmp5[5] = { 0xe9, 0, 0, 0, 0 };
+	s32 rel;
+	int err;
+
+	if (!ptwrite_rel32(vaddr + 5, stub_addr, &rel))
+		return -ERANGE;
+	memcpy(jmp5 + 1, &rel, 4);
+
+	{
+		bool first_phase_done;
+
+		err = text_poke_5byte(auprobe, vma, vaddr, jmp5,
+				      auprobe->ptwrite.orig[0], false, true, true,
+				      false, &first_phase_done);
+		if (err && first_phase_done) {
+			int restore_err;
+
+			/* Restore only after INT3 was successfully installed. */
+			restore_err = text_poke_5byte(auprobe, vma, vaddr,
+					auprobe->ptwrite.orig, UPROBE_SWBP_INSN,
+					true, true, true, false, NULL);
+			if (restore_err)
+				return restore_err;
+		}
+	}
+	return err;
+}
+
+int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
+		struct vm_area_struct *vma, unsigned long vaddr)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	struct uprobe_ptwrite_page *ptw;
+	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
+	unsigned long block_off, stub_addr;
+	u8 *kaddr, orig[5];
+	s32 rel;
+	int ret;
+
+	if (!is_64bit_mm(mm))
+		return -EOPNOTSUPP;
+	/* A three-phase poke and this single-page read both require one page. */
+	if (PAGE_SIZE - (vaddr & ~PAGE_MASK) < 5)
+		return -EINVAL;
+	mmap_assert_write_locked(mm);
+
+	ret = copy_from_vaddr(mm, vaddr, orig, sizeof(orig));
+	if (ret)
+		return ret;
+	if (ptwrite_is_installed(mm, vaddr, orig))
+		return 0;
+
+	ret = ptwrite_validate_site(orig, vaddr);
+	if (ret)
+		return ret;
+
+	ptw = get_uprobe_ptwrite_page(mm, vaddr, ptw_a->stub_len);
+	if (!ptw)
+		return -ENOMEM;
+
+	block_off = ptw->cursor;
+	if (block_off > PAGE_SIZE ||
+	    ptw_a->stub_len > PAGE_SIZE - block_off)
+		return -ENOMEM;
+	stub_addr = ptw->vaddr + block_off;
+	if (!ptwrite_rel32(stub_addr + ptw_a->jmp_off + 4,
+			   vaddr + 5, &rel))
+		return -ERANGE;
+
+	kaddr = kmap_local_page(ptw->page);
+	memcpy(kaddr + block_off, ptw_a->stub, ptw_a->stub_len);
+	/* ptwrite_mapping rejects mremap, so this per-mm rel32 remains valid. */
+	memcpy(kaddr + block_off + ptw_a->jmp_off, &rel, sizeof(rel));
+	kunmap_local(kaddr);
+
+	ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
+	if (!ret)
+		ptw->cursor = block_off + ptw_a->stub_len;
+	return ret;
+}
+
+int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
+		struct vm_area_struct *vma, unsigned long vaddr)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	u8 cur[5];
+
+	mmap_assert_write_locked(mm);
+	if (copy_from_vaddr(mm, vaddr, cur, sizeof(cur)) ||
+	    !ptwrite_is_installed(mm, vaddr, cur))
+		return;
+
+	text_poke_5byte(auprobe, vma, vaddr, auprobe->ptwrite.orig,
+			UPROBE_SWBP_INSN, false, false, false, false, NULL);
+}
+
+
 static bool __is_optimized(struct mm_struct *mm, uprobe_opcode_t *insn, unsigned long vaddr)
 {
 	struct __packed __arch_relative_insn {
-- 
2.54.0

[RFC v1 06/19] ptwrite uprobes: Add a sample module to exercise interface

[Andi Kleen]

Add a basic test module and a test program for uprobes ptwrite.

The module allows to configure and register a ptwrite uprobe in a
executable.

This for testing and not intended as a production interface.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 samples/Kconfig                              |  13 ++
 samples/Makefile                             |   1 +
 samples/uprobe-ptwrite/Makefile              |  13 ++
 samples/uprobe-ptwrite/test_prog.c           |  35 ++++
 samples/uprobe-ptwrite/uprobe_ptwrite_test.c | 163 +++++++++++++++++++
 5 files changed, 225 insertions(+)
 create mode 100644 samples/uprobe-ptwrite/Makefile
 create mode 100644 samples/uprobe-ptwrite/test_prog.c
 create mode 100644 samples/uprobe-ptwrite/uprobe_ptwrite_test.c

diff --git a/samples/Kconfig b/samples/Kconfig
index a75e8e78330d..47bbef5d3e76 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -322,6 +322,19 @@ config SAMPLE_HUNG_TASK
 	  Reading these files with multiple processes triggers hung task
 	  detection by holding locks for a long time (256 seconds).
 
+config SAMPLE_UPROBE_PTWRITE
+	tristate "Build ptwrite uprobe test module -- loadable module only"
+	depends on UPROBES && X86_64 && m
+	help
+	  Builds the minimal ptwrite uprobe prototype driver. It replaces
+	  a 5-byte NOP at a user-specified file offset with a jmp to a per-mm
+	  user-mode stub that emits the requested live registers / immediates
+	  with the PTWRITE instruction into an externally configured Intel PT
+	  stream (no kernel entry, no syscall, no single-step at probe-hit
+	  time). The companion userspace target is test_prog.c in the same
+	  directory; the module params (path/offset/args/event_id) select the
+	  probe site and the emitted values.
+
 source "samples/rust/Kconfig"
 
 source "samples/damon/Kconfig"
diff --git a/samples/Makefile b/samples/Makefile
index 07641e177bd8..0c970d46653e 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -44,4 +44,5 @@ obj-$(CONFIG_SAMPLE_DAMON_WSSE)		+= damon/
 obj-$(CONFIG_SAMPLE_DAMON_PRCL)		+= damon/
 obj-$(CONFIG_SAMPLE_DAMON_MTIER)	+= damon/
 obj-$(CONFIG_SAMPLE_HUNG_TASK)		+= hung_task/
+obj-$(CONFIG_SAMPLE_UPROBE_PTWRITE)	+= uprobe-ptwrite/
 obj-$(CONFIG_SAMPLE_TSM_MR)		+= tsm-mr/
diff --git a/samples/uprobe-ptwrite/Makefile b/samples/uprobe-ptwrite/Makefile
new file mode 100644
index 000000000000..086d84be6ad2
--- /dev/null
+++ b/samples/uprobe-ptwrite/Makefile
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# ptwrite uprobe prototype sample.
+#
+# Module build (in-tree or via make M=samples/uprobe-ptwrite):
+#   obj-$(CONFIG_SAMPLE_UPROBE_PTWRITE) += uprobe_ptwrite_test.o
+#
+# The userspace test target (test_prog) is built by the out-of-tree test
+# harness (see /home/oc/uprobe-ptwrite-test); the harness computes the file
+# offset of target()'s 5-byte NOP and passes it to the module's offset
+# parameter.
+
+obj-$(CONFIG_SAMPLE_UPROBE_PTWRITE) += uprobe_ptwrite_test.o
diff --git a/samples/uprobe-ptwrite/test_prog.c b/samples/uprobe-ptwrite/test_prog.c
new file mode 100644
index 000000000000..813841ce5020
--- /dev/null
+++ b/samples/uprobe-ptwrite/test_prog.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * test_prog - userspace target for the ptwrite uprobe test.
+ *
+ * target() carries a 5-byte NOP (nopl 0(%rax,%rax,1) = 0f 1f 44 00 00) at
+ * its entry. A probe installed at that file offset emits rdi/rsi (a, b)
+ * via PTWRITE. In the test loop a == i and b == i + 1.
+ */
+#ifndef noinline
+#define noinline __attribute__((noinline))
+#endif
+
+#include <stdio.h>
+
+static noinline unsigned long
+target(unsigned long a, unsigned long b)
+{
+	/* 5-byte NOP: nopl 0(%rax,%rax,1) = 0f 1f 44 00 00.
+	 * Emitted as raw bytes: the assembler would otherwise shrink
+	 * "nopl (%rax,%rax,1)" to the 4-byte form (0f 1f 04 00), which
+	 * the probe site validation rejects (jmp rel32 needs 5 bytes).
+	 */
+	asm volatile(".byte 0x0f, 0x1f, 0x44, 0x00, 0x00");
+	return a * 31 + b;
+}
+
+int main(void)
+{
+	unsigned long i, acc = 0;
+
+	for (i = 0; i < 2000; i++)
+		acc += target(i, i + 1);
+	printf("acc=%lu\n", acc);
+	return 0;
+}
diff --git a/samples/uprobe-ptwrite/uprobe_ptwrite_test.c b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
new file mode 100644
index 000000000000..c3b9dd6bec16
--- /dev/null
+++ b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
@@ -0,0 +1,163 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * uprobe_ptwrite_test - minimal ptwrite uprobe prototype driver.
+ *
+ * Registers a trap-free ptwrite uprobe at a user-specified file offset and
+ * emits the requested live registers / immediates into an externally
+ * configured Intel PT stream (no kernel entry at probe-hit time):
+ *
+ *   perf record -e intel_pt/ptw=1,fup_on_ptw=1//u -o perf.data ./test_prog
+ *
+ * Usage (module params):
+ *   path=/path/to/prog   file to probe
+ *   offset=0xADDR        file offset of the probe site
+ *   args="r0,r1,i0x42"             comma-separated;
+ *                        r<N> = x86-64 GPR index 0..15,
+ *                        i<hex> = immediate constant,
+ *   event_id=0x1234      identifier carried in the PTW header word
+ */
+#include <linux/module.h>
+#include <linux/uprobes.h>
+#include <linux/fs.h>
+#include <linux/shmem_fs.h>
+
+static char *path = "/nonexistent";
+module_param(path, charp, 0444);
+MODULE_PARM_DESC(path, "path of the binary to probe");
+
+static ulong offset;
+module_param(offset, ulong, 0444);
+MODULE_PARM_DESC(offset, "file offset of the 5-byte NOP to probe");
+
+static ushort event_id = 0x1234;
+module_param(event_id, ushort, 0444);
+MODULE_PARM_DESC(event_id, "event id carried in the PTW header word");
+
+static char *args = "r0";
+module_param(args, charp, 0444);
+MODULE_PARM_DESC(args, "comma-separated args: r<N> GPR, i<hex> immediate, m<N>[:disp][:4|8] memory");
+
+static struct file *probe_file;
+static struct uprobe *probe;
+static struct uprobe_consumer consumer;
+static struct uprobe_ptwrite_desc desc;
+
+/* Never invoked: ptwrite probes do not trap. Satisfies the core contract. */
+static int noop_handler(struct uprobe_consumer *self, struct pt_regs *regs,
+			__u64 *data)
+{
+	return 0;
+}
+
+static int parse_probe_args(void)
+{
+	char *s, *p, *tok;
+	unsigned int n = 0;
+
+	s = kstrdup(args, GFP_KERNEL);
+	if (!s)
+		return -ENOMEM;
+
+	p = s;
+	while ((tok = strsep(&p, ",")) != NULL) {
+		struct uprobe_ptwrite_arg *a;
+
+		if (n >= UPROBE_PTWRITE_MAX_ARGS) {
+			pr_err("uprobe_ptwrite_test: too many args\n");
+			goto err;
+		}
+		a = &desc.args[n];
+
+		a->size = 8;
+		if (tok[0] == 'r') {
+			unsigned long reg;
+
+			if (kstrtoul(tok + 1, 10, &reg) || reg > 15) {
+				pr_err("uprobe_ptwrite_test: bad reg '%s'\n", tok);
+				goto err;
+			}
+			a->src = UPROBE_PTW_SRC_REG;
+			a->reg = reg;
+		} else if (tok[0] == 'i') {
+			unsigned long long v;
+
+			if (kstrtoull(tok + 1, 0, &v)) {
+				pr_err("uprobe_ptwrite_test: bad imm '%s'\n", tok);
+				goto err;
+			}
+			a->src = UPROBE_PTW_SRC_IMM;
+			a->val = v;
+		} else {
+			pr_err("uprobe_ptwrite_test: bad arg '%s'\n", tok);
+			goto err;
+		}
+		n++;
+	}
+	if (!n || n > UPROBE_PTWRITE_MAX_ARGS) {
+		pr_err("uprobe_ptwrite_test: need 1..%d args\n",
+		       UPROBE_PTWRITE_MAX_ARGS);
+		goto err;
+	}
+	desc.nargs = n;
+	kfree(s);
+	return 0;
+err:
+	kfree(s);
+	return -EINVAL;
+}
+
+static int __init uprobe_ptwrite_test_init(void)
+{
+	struct inode *inode;
+	int ret;
+
+	desc.event_id = event_id;
+	ret = parse_probe_args();
+	if (ret)
+		return ret;
+
+	probe_file = filp_open(path, O_RDONLY, 0);
+	if (IS_ERR(probe_file))
+		return PTR_ERR(probe_file);
+
+	inode = file_inode(probe_file);
+	if (!inode->i_mapping->a_ops->read_folio &&
+	    !shmem_mapping(inode->i_mapping)) {
+		pr_err("uprobe_ptwrite_test: unsupported mapping\n");
+		ret = -EIO;
+		goto out_file;
+	}
+
+	consumer.handler = noop_handler;
+	consumer.ret_handler = NULL;
+	consumer.filter = NULL;
+
+	probe = uprobe_register_ptwrite(inode, probe_file, offset, &consumer, &desc);
+	if (IS_ERR(probe)) {
+		ret = PTR_ERR(probe);
+		pr_err("uprobe_ptwrite_test: register failed: %d\n", ret);
+		goto out_file;
+	}
+
+	pr_info("uprobe_ptwrite_test: probe %s+0x%lx, %u args, event_id=0x%x\n",
+		path, offset, desc.nargs, desc.event_id);
+	return 0;
+
+out_file:
+	fput(probe_file);
+	return ret;
+}
+
+static void __exit uprobe_ptwrite_test_exit(void)
+{
+	uprobe_unregister_nosync(probe, &consumer);
+	uprobe_unregister_sync();
+	fput(probe_file);
+	pr_info("uprobe_ptwrite_test: unregistered\n");
+}
+
+module_init(uprobe_ptwrite_test_init);
+module_exit(uprobe_ptwrite_test_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("ptwrite uprobes testing driver");
-- 
2.54.0

[RFC v1 07/19] ptwrite uprobes: Add support to tracing infrastructure

[Andi Kleen]

Hook up the low level x86 ptwrite uprobes code to the generic trace
uprobes events parser, so that the new probes can be set up. The
interface is similar to classic probes, but there is new ptw: syntax
and various restrictions.

Add minimal docs.

Architectures without the ptwrite backend are handled by weak stubs.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 Documentation/trace/uprobetracer.rst |  61 +++++++-
 arch/x86/Kconfig                     |   1 +
 arch/x86/include/asm/uprobes.h       |  12 +-
 include/linux/uprobes.h              |   4 +
 kernel/trace/Kconfig                 |   6 +
 kernel/trace/trace_uprobe.c          | 221 +++++++++++++++++++++++++--
 6 files changed, 288 insertions(+), 17 deletions(-)

diff --git a/Documentation/trace/uprobetracer.rst b/Documentation/trace/uprobetracer.rst
index 01f6a780fb04..a373957c9f23 100644
--- a/Documentation/trace/uprobetracer.rst
+++ b/Documentation/trace/uprobetracer.rst
@@ -19,8 +19,8 @@ However unlike kprobe-event tracer, the uprobe event interface expects the
 user to calculate the offset of the probepoint in the object.
 
 You can also use /sys/kernel/tracing/dynamic_events instead of
-uprobe_events. That interface will provide unified access to other
-dynamic events too.
+uprobe_events. That interface provides unified access to other event types
+too.
 
 Synopsis of uprobe_tracer
 -------------------------
@@ -29,6 +29,7 @@ Synopsis of uprobe_tracer
   p[:[GRP/][EVENT]] PATH:OFFSET [FETCHARGS] : Set a uprobe
   r[:[GRP/][EVENT]] PATH:OFFSET [FETCHARGS] : Set a return uprobe (uretprobe)
   p[:[GRP/][EVENT]] PATH:OFFSET%return [FETCHARGS] : Set a return uprobe (uretprobe)
+  ptw[:[GRP/][EVENT]] PATH:OFFSET [FETCHARGS] : Set a trap-free ptwrite uprobe
   -:[GRP/][EVENT]                           : Clear uprobe or uretprobe event
 
   GRP           : Group name. If omitted, "uprobes" is the default value.
@@ -76,6 +77,62 @@ offset, and container-size (usually 32). The syntax is::
 For $comm, the default type is "string"; any other type is invalid.
 
 
+ptwrite uprobes (ptw:)
+--------------------------
+
+See ptwrite-uprobes.rst for more details.
+
+A ``ptw:`` probe emits values into
+an Intel Processor Trace data stream. It is faster
+than standard uprobes, but has restrictions.
+The Intel Processor Trace recording must be configured
+separately.
+
+Requirements and restrictions:
+
+- Intel CPU with PTWRITE support (``/sys/devices/intel_pt/format/ptw`` must exist)
+- Entry probes only: ``r:``/``%return`` and the SDT reference counter
+  ``(REF)`` are rejected.
+- The probe site must be a 5-byte NOP or a punnable instruction (see
+  ptwrite-uprobes.rst). For five ``0x90`` bytes from GCC's
+  ``-fpatchable-function-entry=5``, append ``%multinop`` to the offset.
+- FETCHARGS: register names (``%di``, ``%r8``, ...), ``$stack`` (the
+  stack pointer value) and immediates (``\IMM``), memory sources:
+  ``$stackN`` (the Nth stack slot, ``[%rsp + 8N]``) and ``+off(FETCHARG)``
+  dereferences (e.g. ``+8(%di)`` = ``[%rdi + 8]``). ``u64`` sources use
+  ``ptwriteq``; ``u32``/``s32``/``x32`` sources use ``ptwritel`` and read
+  four bytes. Strings, bitfields, and indirect dereferences are not
+  supported.
+- Memory sources execute a user-mode load in the stub. A bad base is
+  fixed up on the fault path to emit the fault word (0).
+  The probed task does not receive the fault, and the wire format
+  (``nargs`` words) is unchanged. uffd-managed pages are resolved by the
+  app's handler first and read their actual value; only a failed or
+  interrupted resolution degrades to the fault word.
+- The event does not produce ring-buffer records. It provides a type registry
+  (``events/GRP/EVENT/format``) and the wire ``event_id``
+  (``events/GRP/EVENT/id``) for an external decoder. Filters and perf
+  attach are rejected because filtering would need kernel entry.
+
+Wire format: each probe hit emits a 64-bit header word ``event_id<<48 |
+nargs<<40 | 0x5054525731`` followed by ``nargs`` PTWRITE payloads. Perf
+exposes each payload through its existing ``u64`` field.  Decode with
+``perf script`` and ``tools/perf/scripts/python/uprobe-ptwrite-decode.py``:
+
+::
+
+  perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o perf.data -- ./app
+  perf script --itrace=qwe -s uprobe-ptwrite-decode.py
+
+``fup_on_ptw=1`` increases the overhead, but also decoding reliability
+because the exact IP of each probe is logged.
+
+Example::
+
+  echo 'ptw:t /bin/app:0x1234 %rdi %rsi \0x42' > /sys/kernel/tracing/uprobe_events
+  echo 1 > /sys/kernel/tracing/events/uprobes/t/enable
+
+
 Event Profiling
 ---------------
 You can check the total number of probe hits per event via
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 15fd9ec5ecac..14ef8a97b496 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -422,6 +422,7 @@ config HAVE_INTEL_TXT
 
 config ARCH_SUPPORTS_UPROBES
 	def_bool y
+	select UPROBE_EVENTS_PTWRITE if X86_64 && UPROBE_EVENTS
 
 config FIX_EARLYCON_MEM
 	def_bool y
diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 05f8a7ea93a6..5cc870e857a5 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -28,6 +28,14 @@ enum {
 
 struct uprobe_xol_ops;
 
+/*
+ * Stub block array size. Worst case = 250 B (8 MEM args, rsp bases, fault
+ * table); 288 leaves 38 B slack. A file-scope static_assert in
+ * arch/x86/kernel/uprobes.c re-derives the worst case; prepare() also
+ * enforces it with -E2BIG at runtime.
+ */
+#define UPROBE_PTWRITE_STUB_SIZE	288
+
 /*
  * ptwrite probe state. The stub template (code + data slots) is built
  * once at registration (mm-independent except the final jmp's rel32, patched
@@ -35,8 +43,8 @@ struct uprobe_xol_ops;
  *   [ptwriteq hdr(%rip)] [arg emissions] [jmp probe+5] [u64 slots: header, imms]
  */
 struct uprobe_ptwrite_arch {
-	u8	stub[256];
-	u8	stub_len;	/* code + data, whole block */
+	u8	stub[UPROBE_PTWRITE_STUB_SIZE];
+	u16	stub_len;	/* code + data + fault table, whole block */
 	u8	jmp_off;	/* offset of the final jmp's rel32 field */
 	u8	ndata;		/* number of u64 data slots */
 	u8	orig[MAX_UINSN_BYTES];	/* pristine file bytes, before generic analysis */
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index 0c422f6d9e7f..6d2a430f88ca 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -205,8 +205,12 @@ struct uprobes_state {
 enum uprobe_ptwrite_src {
 	UPROBE_PTW_SRC_REG,	/* value = live GPR (index in .reg) */
 	UPROBE_PTW_SRC_IMM,	/* value = constant (.val), stored in stub data slot */
+	UPROBE_PTW_SRC_MEM,	/* value = [.reg + disp32] (v2); requires ALLOW_MEM */
 };
 
+/* uprobe_ptwrite_desc.flags */
+#define UPROBE_PTWRITE_FL_ALLOW_MEM	BIT(0) /* SRC_MEM args enabled */
+
 struct uprobe_ptwrite_arg {
 	u8	src;		/* enum uprobe_ptwrite_src */
 	u8	reg;		/* x86-64 GPR index (0=rax..15=r15) for SRC_REG */
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 0ab5916575a9..edbae5aa8dd3 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -848,6 +848,12 @@ config UPROBE_EVENTS
 	  This option is required if you plan to use perf-probe subcommand
 	  of perf tools on user space applications.
 
+config UPROBE_EVENTS_PTWRITE
+	bool
+	depends on UPROBE_EVENTS
+	help
+	  ptwrite uprobes ("ptw:" tracefs event type).
+
 config EPROBE_EVENTS
 	bool "Enable event-based dynamic events"
 	depends on TRACING
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index 22cc3c8181b8..c457c89afd73 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -18,6 +18,7 @@
 #include <linux/security.h>
 #include <linux/string.h>
 #include <linux/uaccess.h>
+#include <linux/fs.h>
 #include <linux/uprobes.h>
 
 #include "trace.h"
@@ -66,6 +67,9 @@ struct trace_uprobe {
 	unsigned long			offset;
 	unsigned long			ref_ctr_offset;
 	unsigned long __percpu		*nhits;
+	bool				is_ptwrite;
+	struct uprobe_ptwrite_desc	ptwrite_desc;
+	/* tp.args[] is a flex array and must remain the last member */
 	struct trace_probe		tp;
 };
 
@@ -275,7 +279,7 @@ static bool trace_uprobe_is_busy(struct dyn_event *ev)
 {
 	struct trace_uprobe *tu = to_trace_uprobe(ev);
 
-	return trace_probe_is_enabled(&tu->tp);
+	return trace_probe_is_enabled(&tu->tp) || tu->uprobe;
 }
 
 static bool trace_uprobe_match_command_head(struct trace_uprobe *tu,
@@ -510,7 +514,8 @@ static int register_trace_uprobe(struct trace_uprobe *tu)
 	old_tu = find_probe_event(trace_probe_name(&tu->tp),
 				  trace_probe_group_name(&tu->tp));
 	if (old_tu) {
-		if (is_ret_probe(tu) != is_ret_probe(old_tu)) {
+		if (is_ret_probe(tu) != is_ret_probe(old_tu) ||
+		    tu->is_ptwrite != old_tu->is_ptwrite) {
 			trace_probe_log_set_index(0);
 			trace_probe_log_err(0, DIFF_PROBE_TYPE);
 			return -EEXIST;
@@ -535,9 +540,95 @@ static int register_trace_uprobe(struct trace_uprobe *tu)
 
 DEFINE_FREE(free_trace_uprobe, struct trace_uprobe *, free_trace_uprobe(_T))
 
+/*
+ * ptwrite probes never dispatch, but provide a dummy handler
+ * to keep the core happy.
+ */
+static int ptwrite_noop_handler(struct uprobe_consumer *con,
+				struct pt_regs *regs, __u64 *data)
+{
+	return 0;
+}
+
+/*
+ * Compile one parsed fetch arg into a ptwrite descriptor entry. The
+ * arch-independent part: decode the fetch chain, reject shapes the
+ * scratch-free stub cannot emit, and hand the rest to the arch hook.
+ */
+static int ptwrite_compile_arg(struct trace_uprobe *tu, int i)
+{
+	struct fetch_insn *code = tu->tp.args[i].code;
+	struct uprobe_ptwrite_arg *a = &tu->ptwrite_desc.args[i];
+	struct uprobe_ptwrite_fetch f;
+
+	if (code[1].op == FETCH_OP_ST_MEM || code[1].op == FETCH_OP_ST_UMEM) {
+		if (code[2].op != FETCH_OP_END)
+			return -EINVAL;
+		if (code[0].op != FETCH_OP_REG) {
+			/*
+			 * +off($stackN): the STACK op derefs [rsp+8N] to a
+			 * POINTER, and ST_MEM derefs that pointer (load-of-
+			 * load). The scratch-free stub has no register to
+			 * hold the intermediate pointer, so reject.
+			 */
+			return -EINVAL;
+		}
+		if (tu->tp.args[i].type->size != 4 &&
+		    tu->tp.args[i].type->size != 8)
+			return -EINVAL;	/* memory derefs are u32 or u64 */
+		f.kind = UPROBE_PTW_FETCH_MEMREG;
+		f.reg = code[0].param;
+		f.imm = code[1].offset;
+		goto compile;
+	}
+
+	/* $stackN: [STACK, ST_RAW, END], the deref is folded inside the
+	 * STACK op (get_user_stack_nth reads [rsp + 8N]).
+	 */
+	if (code[0].op == FETCH_OP_STACK &&
+	    code[1].op == FETCH_OP_ST_RAW && code[2].op == FETCH_OP_END) {
+		if (tu->tp.args[i].type->size != 4 &&
+		    tu->tp.args[i].type->size != 8)
+			return -EINVAL;	/* stack slots are u32 or u64 */
+		f.kind = UPROBE_PTW_FETCH_STACKN;
+		f.imm = 8L * code[0].param;
+		goto compile;
+	}
+
+	if (code[1].op != FETCH_OP_ST_RAW || code[2].op != FETCH_OP_END)
+		return -EINVAL;
+
+	switch (code->op) {
+	case FETCH_OP_REG:	/* %reg */
+		f.kind = UPROBE_PTW_FETCH_REG;
+		f.reg = code->param;
+		break;
+	case FETCH_OP_STACKP:	/* $stack: SP value, never faults */
+		f.kind = UPROBE_PTW_FETCH_STACKP;
+		break;
+	case FETCH_OP_IMM:	/* \IMM */
+		f.kind = UPROBE_PTW_FETCH_IMM;
+		f.imm = code->immediate;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+compile:
+	if (f.kind == UPROBE_PTW_FETCH_MEMREG ||
+	    f.kind == UPROBE_PTW_FETCH_STACKN)
+		tu->ptwrite_desc.flags |= UPROBE_PTWRITE_FL_ALLOW_MEM;
+	if (arch_uprobe_ptwrite_fetch(a, &f))
+		return -EINVAL;
+	a->size = tu->tp.args[i].type->size;
+	return 0;
+}
+
+
 /*
  * Argument syntax:
  *  - Add uprobe: p|r[:[GRP/][EVENT]] PATH:OFFSET[%return][(REF)] [FETCHARGS]
+ *  - Add ptwrite uprobe: ptw[:[GRP/][EVENT]] PATH:OFFSET [FETCHARGS]
  */
 static int __trace_uprobe_create(int argc, const char **argv)
 {
@@ -553,10 +644,17 @@ static int __trace_uprobe_create(int argc, const char **argv)
 	char *buf __free(kfree) = NULL;
 	enum probe_print_type ptype;
 	bool is_return = false;
-	int i, ret;
+	bool is_ptwrite = false;
+	int i, ret, arg_start = 2;
 
 	ref_ctr_offset = 0;
 
+	if (!strncmp(argv[0], "ptw:", 4)) {
+		if (!IS_ENABLED(CONFIG_UPROBE_EVENTS_PTWRITE))
+			return -EOPNOTSUPP;	/* no arch backend configured */
+		is_ptwrite = true;
+	}
+
 	switch (argv[0][0]) {
 	case 'r':
 		is_return = true;
@@ -572,13 +670,16 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	trlog = trace_probe_log_init("trace_uprobe", argc, argv);
 
-	if (argc - 2 > MAX_TRACE_ARGS) {
+	if (argc - 2 > MAX_TRACE_ARGS ||
+	    (is_ptwrite && argc - 2 > UPROBE_PTWRITE_MAX_ARGS)) {
 		trace_probe_log_set_index(2);
 		trace_probe_log_err(0, TOO_MANY_ARGS);
 		return -E2BIG;
 	}
 
-	if (argv[0][1] == ':')
+	if (is_ptwrite)
+		event = argv[0][4] ? &argv[0][4] : NULL;	/* after "ptw:" */
+	else if (argv[0][1] == ':')
 		event = &argv[0][2];
 
 	if (!strchr(argv[1], '/'))
@@ -608,6 +709,10 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	/* Parse reference counter offset if specified. */
 	rctr = strchr(arg, '(');
+	if (rctr && is_ptwrite) {
+		trace_probe_log_err(rctr - filename, BAD_REFCNT);
+		return -EINVAL;	/* SDT ref-counting needs kernel updates */
+	}
 	if (rctr) {
 		rctr_end = strchr(rctr, ')');
 		if (!rctr_end) {
@@ -632,7 +737,10 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	/* Check if there is %return suffix */
 	tmp = strchr(arg, '%');
-	if (tmp) {
+	if (tmp && is_ptwrite) {
+		trace_probe_log_err(tmp - filename, BAD_ADDR_SUFFIX);
+		return -EINVAL;
+	} else if (tmp) {
 		if (!strcmp(tmp, "%return")) {
 			*tmp = '\0';
 			is_return = true;
@@ -677,7 +785,8 @@ static int __trace_uprobe_create(int argc, const char **argv)
 		buf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
 		if (!buf)
 			return -ENOMEM;
-		snprintf(buf, MAX_EVENT_NAME_LEN, "%c_%s_0x%lx", 'p', tail, offset);
+		snprintf(buf, MAX_EVENT_NAME_LEN, "%c_%s_0x%lx",
+			 is_ptwrite ? 't' : 'p', tail, offset);
 		event = buf;
 		kfree(tail);
 	}
@@ -712,6 +821,26 @@ static int __trace_uprobe_create(int argc, const char **argv)
 			return ret;
 	}
 
+	if (is_ptwrite) {
+		if (!argc) {
+			trace_probe_log_set_index(2);
+			trace_probe_log_err(0, NO_ARG_BODY);
+			return -EINVAL;	/* core rejects desc->nargs == 0 */
+		}
+		tu->is_ptwrite = true;
+		tu->ptwrite_desc.nargs = argc;
+		tu->ptwrite_desc.flags = 0;
+		for (i = 0; i < argc; i++) {
+			ret = ptwrite_compile_arg(tu, i);
+			if (ret) {
+				trace_probe_log_set_index(i + arg_start);
+				trace_probe_log_err(0, BAD_FETCH_ARG);
+				return ret;
+			}
+		}
+		tu->consumer.handler = ptwrite_noop_handler;
+	}
+
 	ptype = is_ret_probe(tu) ? PROBE_PRINT_RETURN : PROBE_PRINT_NORMAL;
 	ret = traceprobe_set_print_fmt(&tu->tp, ptype);
 	if (ret < 0)
@@ -754,9 +883,16 @@ static int trace_uprobe_show(struct seq_file *m, struct dyn_event *ev)
 	char c = is_ret_probe(tu) ? 'r' : 'p';
 	int i;
 
-	seq_printf(m, "%c:%s/%s %s:0x%0*lx", c, trace_probe_group_name(&tu->tp),
-			trace_probe_name(&tu->tp), tu->filename,
-			(int)(sizeof(void *) * 2), tu->offset);
+	if (tu->is_ptwrite) {
+		seq_printf(m, "ptw:%s/%s %s:0x%0*lx",
+			   trace_probe_group_name(&tu->tp),
+			   trace_probe_name(&tu->tp), tu->filename,
+			   (int)(sizeof(void *) * 2), tu->offset);
+	} else
+		seq_printf(m, "%c:%s/%s %s:0x%0*lx", c,
+			   trace_probe_group_name(&tu->tp),
+			   trace_probe_name(&tu->tp), tu->filename,
+			   (int)(sizeof(void *) * 2), tu->offset);
 
 	if (tu->ref_ctr_offset)
 		seq_printf(m, "(0x%lx)", tu->ref_ctr_offset);
@@ -1107,9 +1243,24 @@ static int trace_uprobe_enable(struct trace_uprobe *tu, filter_func_t filter)
 {
 	struct inode *inode = d_real_inode(tu->path.dentry);
 	struct uprobe *uprobe;
-
-	tu->consumer.filter = filter;
-	uprobe = uprobe_register(inode, tu->offset, tu->ref_ctr_offset, &tu->consumer);
+	struct file *file;
+
+	if (tu->is_ptwrite) {
+		if (filter)
+			return -EINVAL; /* no kernel entry to evaluate it */
+		file = dentry_open(&tu->path, O_RDONLY, current_cred());
+		if (IS_ERR(file))
+			return PTR_ERR(file);
+		tu->ptwrite_desc.event_id =
+			trace_probe_event_call(&tu->tp)->event.type;
+		uprobe = uprobe_register_ptwrite(inode, file, tu->offset,
+						 &tu->consumer, &tu->ptwrite_desc);
+		fput(file);
+	} else {
+		tu->consumer.filter = filter;
+		uprobe = uprobe_register(inode, tu->offset,
+					 tu->ref_ctr_offset, &tu->consumer);
+	}
 	if (IS_ERR(uprobe))
 		return PTR_ERR(uprobe);
 
@@ -1148,6 +1299,28 @@ static int probe_event_enable(struct trace_event_call *call,
 	tp = trace_probe_primary_from_call(call);
 	if (WARN_ON_ONCE(!tp))
 		return -ENODEV;
+	tu = container_of(tp, struct trace_uprobe, tp);
+
+	if (tu->is_ptwrite) {
+		if (filter || !file || file->filter)
+			return -EINVAL;
+		enabled = trace_probe_is_enabled(tp);
+		ret = trace_probe_add_file(tp, file);
+		if (ret < 0)
+			return ret;
+		if (enabled)
+			return 0;
+		list_for_each_entry(tu, trace_probe_probe_list(tp), tp.list) {
+			ret = trace_uprobe_enable(tu, NULL);
+			if (ret) {
+				__probe_event_disable(tp);
+				trace_probe_remove_file(tp, file);
+				return ret;
+			}
+		}
+		return 0;
+	}
+
 	enabled = trace_probe_is_enabled(tp);
 
 	/* This may also change "enabled" state */
@@ -1201,11 +1374,22 @@ static void probe_event_disable(struct trace_event_call *call,
 				struct trace_event_file *file)
 {
 	struct trace_probe *tp;
+	struct trace_uprobe *tu;
 
 	tp = trace_probe_primary_from_call(call);
 	if (WARN_ON_ONCE(!tp))
 		return;
 
+	tu = container_of(tp, struct trace_uprobe, tp);
+	if (tu->is_ptwrite) {
+		if (trace_probe_remove_file(tp, file) < 0)
+			return;
+		if (trace_probe_is_enabled(tp))
+			return;	/* other instances still enabled */
+		__probe_event_disable(tp);
+		return;
+	}
+
 	if (!trace_probe_is_enabled(tp))
 		return;
 
@@ -1493,6 +1677,13 @@ int bpf_get_uprobe_info(const struct perf_event *event, u32 *fd_type,
 }
 #endif	/* CONFIG_PERF_EVENTS */
 
+static bool ptwrite_event(struct trace_event_call *call)
+{
+	struct trace_uprobe *tu = trace_uprobe_primary_from_call(call);
+
+	return tu && tu->is_ptwrite;
+}
+
 static int
 trace_uprobe_register(struct trace_event_call *event, enum trace_reg type,
 		      void *data)
@@ -1516,9 +1707,13 @@ trace_uprobe_register(struct trace_event_call *event, enum trace_reg type,
 		return 0;
 
 	case TRACE_REG_PERF_OPEN:
+		if (ptwrite_event(event))
+			return -EINVAL;	/* no perf attach to ptwrite events */
 		return uprobe_perf_open(event, data);
 
 	case TRACE_REG_PERF_CLOSE:
+		if (ptwrite_event(event))
+			return -EINVAL;
 		return uprobe_perf_close(event, data);
 
 #endif
-- 
2.54.0

[RFC v1 08/19] ptwrite uprobes / x86: Add a user fault notifier chain

[Andi Kleen]

The ptwrite uprobes need to catch faults in the user probes, otherwise a
bad probe could crash the program. For classic probes that is handled in
the kernel, but with these new kinds of probes the crash happens in ring 3
code.

The existing die chain cannot be used for this because it only handles
kernel level faults. Add a new user fault notifier chain that is supported
for #GP, #PF, #SS. It is only called before a signal would be delivered, so
it doesn't slow down any hot paths. The fault handler can then handle the
fault and prevent the signal.

Add register code and the hooks for the chain.

Some of the existing fault hardware workarounds could be converted to this
in the future (not done yet)

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/traps.h | 17 +++++++++++++++++
 arch/x86/kernel/traps.c      | 32 ++++++++++++++++++++++++++++++++
 arch/x86/mm/fault.c          |  6 ++++++
 3 files changed, 55 insertions(+)

diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h
index 3f24cc472ce9..c3cdfde3c7b0 100644
--- a/arch/x86/include/asm/traps.h
+++ b/arch/x86/include/asm/traps.h
@@ -4,6 +4,7 @@
 
 #include <linux/context_tracking_state.h>
 #include <linux/kprobes.h>
+#include <linux/notifier.h>
 
 #include <asm/debugreg.h>
 #include <asm/idtentry.h>
@@ -59,4 +60,20 @@ static inline void cond_local_irq_disable(struct pt_regs *regs)
 		local_irq_disable();
 }
 
+/*
+ * User-mode fault notifier chain, called before a user exception is
+ * about to become a signal. NOTIFY_STOP consumes the fault.
+ */
+struct x86_user_fault_args {
+	struct pt_regs	*regs;
+	unsigned long	error_code;
+	unsigned long	address;	/* #PF: faulting address */
+	unsigned int	trap;
+};
+
+extern int register_x86_user_fault_notifier(struct notifier_block *nb);
+extern void unregister_x86_user_fault_notifier(struct notifier_block *nb);
+extern int notify_x86_user_fault(struct pt_regs *regs, unsigned long error_code,
+				 unsigned long address, unsigned int trap);
+
 #endif /* _ASM_X86_TRAPS_H */
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 30aa8369957e..5259a205b837 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -517,6 +517,11 @@ DEFINE_IDTENTRY_ERRORCODE(exc_segment_not_present)
 
 DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment)
 {
+	if (user_mode(regs) &&
+	    notify_x86_user_fault(regs, error_code, 0, X86_TRAP_SS) ==
+			NOTIFY_STOP)
+		return;
+
 	do_error_trap(regs, error_code, "stack segment", X86_TRAP_SS, SIGBUS,
 		      0, NULL);
 }
@@ -911,6 +916,30 @@ static void gp_user_force_sig_segv(struct pt_regs *regs, int trapnr,
 	force_sig(SIGSEGV);
 }
 
+static ATOMIC_NOTIFIER_HEAD(x86_user_fault_chain);
+
+int register_x86_user_fault_notifier(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_register(&x86_user_fault_chain, nb);
+}
+void unregister_x86_user_fault_notifier(struct notifier_block *nb)
+{
+	atomic_notifier_chain_unregister(&x86_user_fault_chain, nb);
+}
+
+int notify_x86_user_fault(struct pt_regs *regs, unsigned long error_code,
+			  unsigned long address, unsigned int trap)
+{
+	struct x86_user_fault_args args = {
+		.regs = regs,
+		.error_code = error_code,
+		.address = address,
+		.trap = trap,
+	};
+
+	return atomic_notifier_call_chain(&x86_user_fault_chain, 0, &args);
+}
+
 DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 {
 	char desc[sizeof(GPFSTR) + 50 + 2*sizeof(unsigned long) + 1] = GPFSTR;
@@ -942,6 +971,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 		if (emulate_vsyscall_gp(regs))
 			goto exit;
 
+		if (notify_x86_user_fault(regs, error_code, 0, X86_TRAP_GP) == NOTIFY_STOP)
+			goto exit;
+
 		gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc);
 		goto exit;
 	}
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index aa88370ce739..897165f960b8 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -821,6 +821,9 @@ __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
 	if (fixup_vdso_exception(regs, X86_TRAP_PF, error_code, address))
 		return;
 
+	if (notify_x86_user_fault(regs, error_code, address, X86_TRAP_PF) == NOTIFY_STOP)
+		return;
+
 	if (likely(show_unhandled_signals))
 		show_signal_msg(regs, error_code, address, tsk);
 
@@ -950,6 +953,9 @@ do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address,
 		return;
 	}
 #endif
+	if (notify_x86_user_fault(regs, error_code, address, X86_TRAP_PF) == NOTIFY_STOP)
+		return;
+
 	force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *)address);
 }
 
-- 
2.54.0

[RFC v1 09/19] ptwrite uprobes: Factor file-backed instruction reads

[Andi Kleen]

Refactor copy_insn into a more generic uprobe_copy_from_file.
The existing copy_insn is still there, but uses the generic
version now. The generic version will be used in later patches.

No semantic change intended.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 include/linux/uprobes.h |  2 ++
 kernel/events/uprobes.c | 56 ++++++++++++++++++++++++++++-------------
 2 files changed, 41 insertions(+), 17 deletions(-)

diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index 6d2a430f88ca..c0d65ea5353e 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -304,6 +304,8 @@ extern void uprobe_handle_trampoline(struct pt_regs *regs);
 extern void *arch_uretprobe_trampoline(unsigned long *psize);
 extern unsigned long uprobe_get_trampoline_vaddr(void);
 extern void uprobe_copy_from_page(struct page *page, unsigned long vaddr, void *dst, int len);
+extern int uprobe_copy_from_file(struct inode *inode, struct file *file,
+					 loff_t offset, void *buf, int size);
 extern void arch_uprobe_clear_state(struct mm_struct *mm);
 extern void arch_uprobe_init_state(struct mm_struct *mm);
 extern int arch_uprobe_dup_ptwrite(struct mm_struct *oldmm, struct mm_struct *newmm);
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 23202df2b51a..20fa16ed8519 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1073,30 +1073,52 @@ static int __copy_insn(struct address_space *mapping, struct file *filp,
 	return 0;
 }
 
-static int copy_insn(struct uprobe *uprobe, struct file *filp)
+/**
+ * uprobe_copy_from_file - read bytes from a file's page cache
+ * @inode: the file's inode
+ * @file: file used by the filesystem's read_folio callback
+ * @offset: byte offset into the file
+ * @buf: destination buffer
+ * @size: number of bytes to read (may cross page boundaries)
+ *
+ * Handles page-crossing reads transparently.  The return value is the number
+ * of bytes copied, or a negative error code.  Callers that require a full
+ * instruction must check that the requested size was copied.
+ */
+int uprobe_copy_from_file(struct inode *inode, struct file *file,
+			  loff_t offset, void *buf, int size)
 {
-	struct address_space *mapping = uprobe->inode->i_mapping;
-	loff_t offs = uprobe->offset;
-	void *insn = &uprobe->arch.insn;
-	int size = sizeof(uprobe->arch.insn);
-	int len, err = -EIO;
+	struct address_space *mapping = inode->i_mapping;
+	loff_t file_size;
+	int len, copied = 0, err;
 
-	/* Copy only available bytes, -EIO if nothing was read */
-	do {
-		if (offs >= i_size_read(uprobe->inode))
+	if (offset < 0 || size < 0)
+		return -EINVAL;
+	while (copied < size) {
+		file_size = i_size_read(inode);
+		if (offset >= file_size)
 			break;
 
-		len = min_t(int, size, PAGE_SIZE - (offs & ~PAGE_MASK));
-		err = __copy_insn(mapping, filp, insn, len, offs);
+		len = min_t(loff_t, size - copied, file_size - offset);
+		len = min_t(int, len, PAGE_SIZE - (offset & ~PAGE_MASK));
+		err = __copy_insn(mapping, file, buf + copied, len, offset);
 		if (err)
-			break;
+			return err;
 
-		insn += len;
-		offs += len;
-		size -= len;
-	} while (size);
+		copied += len;
+		offset += len;
+	}
+	return copied;
+}
 
-	return err;
+static int copy_insn(struct uprobe *uprobe, struct file *filp)
+{
+	int ret;
+
+	ret = uprobe_copy_from_file(uprobe->inode, filp, uprobe->offset,
+				   &uprobe->arch.insn,
+				   sizeof(uprobe->arch.insn));
+	return ret < 0 ? ret : ret == sizeof(uprobe->arch.insn) ? 0 : -EIO;
 }
 
 static int prepare_uprobe(struct uprobe *uprobe, struct file *file,
-- 
2.54.0

[RFC v1 10/19] ptwrite uprobes: Minimal memory references and fault handling

[Andi Kleen]

Add support for memory references. Currently this is only
simple cases, no indirect memory references or strings,
that would require saving/restoring registers. Only 8 and 4 byte
memory references are supported.

This requires fault handling using the fault notifier hook added
earlier. When a fault happens inside a probe return to a specially
generated tail that logs zeroes.

It increases the in memory overhead of uprobe_ptwrite_page somewhat
(to roughly a KB). I tried to find alternatives, but they all
complicated the code, and 1KB should be still acceptable.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/uprobes.h               |  14 +
 arch/x86/kernel/uprobes.c                    | 308 +++++++++++++++++--
 include/linux/uprobes.h                      |   4 +-
 samples/uprobe-ptwrite/uprobe_ptwrite_test.c |  41 ++-
 4 files changed, 344 insertions(+), 23 deletions(-)

diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 5cc870e857a5..efffdc44f00a 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -48,6 +48,8 @@ struct uprobe_ptwrite_arch {
 	u8	jmp_off;	/* offset of the final jmp's rel32 field */
 	u8	ndata;		/* number of u64 data slots */
 	u8	orig[MAX_UINSN_BYTES];	/* pristine file bytes, before generic analysis */
+	u16	ft_off;	/* fault table offset within the block (0 if none) */
+	u8	nft;		/* number of fault entries */
 };
 
 /* Per-mm page holding generated ptwrite stub blocks (mirrors trampolines). */
@@ -56,6 +58,18 @@ struct uprobe_ptwrite_page {
 	struct page		*page;		/* stub blocks written via kmap */
 	unsigned long		vaddr;		/* mapping base */
 	u16			cursor;		/* next free block offset */
+	u16			nblocks;
+	struct {
+		u16 off;	/* block offset in the page */
+		u16 len;	/* generated block length */
+		u16 ft_off;	/* fault table offset within the block */
+		u8  orig0;	/* original site byte 0 (pun restore) */
+		u8  pun;	/* instruction-pun mechanism (single-byte poke) */
+		u8  site_len;	/* original instruction length (pun identity) */
+		s32 site_off;	/* probe site - page base (idempotent reinstall) */
+		u8  site_insn[MAX_UINSN_BYTES];	/* original bytes (pun identity) */
+	} index[PAGE_SIZE / 32];	/* exact: min block = 32 B (nargs >= 1), */
+					/* so <= 128 blocks fit a page */
 };
 
 struct arch_uprobe {
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index df652c56414b..90e702a4a8e9 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -24,6 +24,7 @@
 #include <asm/nops.h>
 #include <asm/cpufeature.h>
 #include <asm/cpuid/api.h>
+#include <asm/traps.h>
 
 /* Post-execution fixups. */
 
@@ -727,6 +728,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign
 
 void arch_uprobe_init_state(struct mm_struct *mm)
 {
+
 	INIT_HLIST_HEAD(&mm->uprobes_state.head_ptwrite);
 }
 
@@ -923,9 +925,12 @@ asm (
 
 extern u8 uprobe_trampoline_entry[];
 
+static struct notifier_block uprobe_user_fault_nb;
+
 static int __init arch_uprobes_init(void)
 {
 	tramp_mapping_pages[0] = virt_to_page(uprobe_trampoline_entry);
+	register_x86_user_fault_notifier(&uprobe_user_fault_nb);
 	return 0;
 }
 
@@ -1259,6 +1264,20 @@ static int ptwrite_emit_riprel(u8 *p, s32 disp)
 	return 9;
 }
 
+static int ptwrite_emit_riprel32(u8 *p, s32 disp)
+{
+	/*
+	 * ptwritel disp32(%rip) : F3 0F AE 25 <disp32> (8 bytes)
+	 * modrm 0x25 = mod 00, reg 100 (/4, PTWRITE), rm 101 (RIP-relative).
+	 */
+	*p++ = 0xf3;
+	*p++ = 0x0f;
+	*p++ = 0xae;
+	*p++ = 0x25;
+	memcpy(p, &disp, 4);
+	return 8;
+}
+
 bool arch_uprobe_ptwrite_supported(void)
 {
 	u32 eax, ebx, ecx, edx;
@@ -1293,7 +1312,11 @@ static const struct {
 	{ offsetof(struct pt_regs, r14), 14 }, { offsetof(struct pt_regs, r15), 15 },
 };
 
-/* Compile the register, stack-pointer, and immediate fetch forms. */
+
+/*
+ * Compile one tracefs fetch arg (arch-neutral form, see
+ * uprobe_ptwrite_fetch) into a ptwrite descriptor entry.
+ */
 int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
 			      const struct uprobe_ptwrite_fetch *f)
 {
@@ -1301,21 +1324,30 @@ int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
 
 	switch (f->kind) {
 	case UPROBE_PTW_FETCH_REG:
+	case UPROBE_PTW_FETCH_MEMREG:
 		for (j = 0; j < ARRAY_SIZE(ptwrite_reg_map); j++)
 			if (ptwrite_reg_map[j].off == f->reg) {
 				idx = ptwrite_reg_map[j].idx;
 				break;
 			}
 		if (idx < 0)
-			return -EINVAL;
-		a->src = UPROBE_PTW_SRC_REG;
+			return -EINVAL;	/* not an x86-64 GPR */
+		a->src = f->kind == UPROBE_PTW_FETCH_REG ?
+			 UPROBE_PTW_SRC_REG : UPROBE_PTW_SRC_MEM;
 		a->reg = idx;
+		if (f->kind == UPROBE_PTW_FETCH_MEMREG)
+			a->val = (u64)(s32)f->imm;
 		break;
-	case UPROBE_PTW_FETCH_STACKP:
+	case UPROBE_PTW_FETCH_STACKP:	/* $stack: SP value, never faults */
 		a->src = UPROBE_PTW_SRC_REG;
 		a->reg = 4; /* rsp */
 		break;
-	case UPROBE_PTW_FETCH_IMM:
+	case UPROBE_PTW_FETCH_STACKN:	/* [rsp + imm] */
+		a->src = UPROBE_PTW_SRC_MEM;
+		a->reg = 4;	/* rsp */
+		a->val = f->imm;
+		break;
+	case UPROBE_PTW_FETCH_IMM:	/* \IMM */
 		a->src = UPROBE_PTW_SRC_IMM;
 		a->val = f->imm;
 		break;
@@ -1325,15 +1357,43 @@ int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
 	return 0;
 }
 
+
+/*
+ * Worst-case stub block: header ptwriteq (9) + max args of the largest form
+ * (MEM: 5 opcode + SIB + 4 disp + 2 short jmp + 9 fixup = 21 B) + final jmp
+ * (5) -> code; data: header + fault-word slots (16); fault table
+ * [nft][{start,end,fixup} x nft] (2 + 6*nft). Must fit UPROBE_PTWRITE_STUB_SIZE;
+ * prepare() also enforces it with -E2BIG at runtime.
+ */
+static_assert((((9 + UPROBE_PTWRITE_MAX_ARGS * 21 + 5 + 7) & ~7) +
+	       16 + 2 + 6 * UPROBE_PTWRITE_MAX_ARGS) <= UPROBE_PTWRITE_STUB_SIZE,
+	       "worst-case ptwrite stub block exceeds UPROBE_PTWRITE_STUB_SIZE");
+
+static bool ptwrite_has_room(const u8 *base, const u8 *p, size_t len)
+{
+	return p >= base && (size_t)(p - base) <=
+		       sizeof(((struct uprobe_ptwrite_arch *)0)->stub) - len;
+}
+
+#define PTW_NEED(_len) do { \
+		if (!ptwrite_has_room(code, p, (_len))) \
+			return -E2BIG; \
+	} while (0)
+
+
+
 int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 				const struct uprobe_ptwrite_desc *desc)
 {
 	struct uprobe_ptwrite_arch *ptw = &auprobe->ptwrite;
 	u8 *code = ptw->stub, *p = ptw->stub;
 	u16 imm_off[UPROBE_PTWRITE_MAX_ARGS];
-	unsigned int data_off;
+	u8 fixup_len[UPROBE_PTWRITE_MAX_ARGS];
+	struct { u16 start, end, fixup; } __packed mft[UPROBE_PTWRITE_MAX_ARGS];
+	u16 mdisp[UPROBE_PTWRITE_MAX_ARGS];
+	unsigned int data_off, flt_off, ft_off;
 	unsigned int hdr_off = 0;
-	unsigned int imm_idx = 0, n_imm = 0;
+	unsigned int imm_idx = 0, n_imm = 0, mem_idx = 0, n_mem = 0;
 	u64 hdr;
 	int i;
 
@@ -1341,7 +1401,7 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 		return -EINVAL;
 	if (desc->nargs > UPROBE_PTWRITE_MAX_ARGS)
 		return -E2BIG;
-	if (desc->flags)
+	if (desc->flags & ~UPROBE_PTWRITE_FL_ALLOW_MEM)
 		return -EINVAL;
 
 	/* The generic registration path copied these bytes before this hook. */
@@ -1358,24 +1418,89 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 				return -E2BIG;
 			n_imm++;
 			break;
+		case UPROBE_PTW_SRC_MEM:
+			if (!(desc->flags & UPROBE_PTWRITE_FL_ALLOW_MEM))
+				return -EINVAL;
+			if (desc->args[i].reg > 15)
+				return -EINVAL;
+			/*
+			 * u64 args use ptwriteq (8-byte load); u32/s32/x32
+			 * args use ptwritel (4-byte load). Any other size
+			 * would read the wrong width.
+			 */
+			if (desc->args[i].size != 4 &&
+			    desc->args[i].size != 8)
+				return -EINVAL;
+			if (n_mem >= ARRAY_SIZE(mft))
+				return -E2BIG;
+			n_mem++;
+			break;
 		default:
 			return -EINVAL;
 		}
 	}
 
 	/* header word emission (disp32 patched below) */
+	PTW_NEED(9);
 	p += ptwrite_emit_riprel(p, 0);
 
 	for (i = 0; i < desc->nargs; i++) {
-		if (desc->args[i].src == UPROBE_PTW_SRC_REG) {
+		switch (desc->args[i].src) {
+		case UPROBE_PTW_SRC_REG:
+			PTW_NEED(5);
 			p += ptwrite_emit_reg(p, desc->args[i].reg);
-		} else {
+			break;
+		case UPROBE_PTW_SRC_IMM:
+			if (imm_idx >= ARRAY_SIZE(imm_off))
+				return -E2BIG;
+			PTW_NEED(9);
 			imm_off[imm_idx++] = p - code;
 			p += ptwrite_emit_riprel(p, 0);
+			break;
+		case UPROBE_PTW_SRC_MEM: {
+			/*
+			 * ptwrite[q|l] disp32(%reg), short jump, and fault
+			 * fixup. The largest form is 21 bytes.
+			 */
+			u8 reg = desc->args[i].reg;
+			bool wide = desc->args[i].size == 8;
+			unsigned int arg_len = (wide ? 9 : 8) +
+				((reg & 7) == 4) + 2 + (wide ? 9 : 8);
+			int start;
+
+			if (mem_idx >= ARRAY_SIZE(mft))
+				return -E2BIG;
+			PTW_NEED(arg_len);
+			start = p - code;
+			*p++ = 0xf3;
+			if (wide)
+				*p++ = (reg & 8) ? 0x49 : 0x48; /* REX.W */
+			else if (reg & 8)
+				*p++ = 0x41; /* REX.B only (32-bit operand) */
+			*p++ = 0x0f;
+			*p++ = 0xae;
+			*p++ = 0xa0 | (reg & 7); /* mod 10, reg /4, rm reg */
+			if ((reg & 7) == 4) /* SIB escape: base rsp/esp/r12 */
+				*p++ = 0x24;
+			mft[mem_idx].start = start;
+			mdisp[mem_idx] = p - code;
+			p += 4;
+			mft[mem_idx].end = p - code;
+			*p++ = 0xeb;
+			*p++ = wide ? 9 : 8;
+			mft[mem_idx].fixup = p - code;
+			fixup_len[mem_idx] = wide ?
+				ptwrite_emit_riprel(p, 0) :
+				ptwrite_emit_riprel32(p, 0);
+			p += fixup_len[mem_idx];
+			mem_idx++;
+			break;
+		}
 		}
 	}
 
 	/* final jmp back to probe+5; rel32 patched per-mm at install */
+	PTW_NEED(5);
 	*p++ = 0xe9;
 	if (p - code > U8_MAX)
 		return -E2BIG;
@@ -1383,11 +1508,23 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 	p += 4;
 
 	data_off = (p - code + 7) & ~7UL;
-	if (data_off + 8 * (1 + n_imm) > sizeof(ptw->stub))
+	/* data: header + imm slots + one shared fault-word slot (0) */
+	if (data_off + 8 * (1 + n_imm + (n_mem ? 1 : 0)) > sizeof(ptw->stub))
 		return -E2BIG;
+	flt_off = data_off + 8 * (1 + n_imm);
 
-	/* data slots: header, then imm values in emission order */
-	hdr = ((u64)desc->event_id << 48) | ((u64)desc->nargs << 40);
+	/* fault table: [u16 nft][{start,end,fixup} x nft], block-relative */
+	ft_off = (flt_off + 8 * (n_mem ? 1 : 0) + 7) & ~7UL;
+	if (ft_off + 2 + 6 * n_mem > sizeof(ptw->stub))
+		return -E2BIG;
+	if (n_mem) {
+		*(u16 *)(code + ft_off) = n_mem;
+		memcpy(code + ft_off + 2, mft, 6 * n_mem);
+	}
+
+	/* data slots: header, imm values in emission order, fault word */
+	hdr = ((u64)desc->event_id << 48) | ((u64)desc->nargs << 40) |
+	      UPROBE_PTW_HDR_MAGIC;
 	*(u64 *)(code + data_off) = hdr;
 
 	/* patch the header's disp32: hdr slot - end of header insn */
@@ -1403,8 +1540,33 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 		imm_idx++;
 	}
 
-	ptw->stub_len = data_off + 8 * (1 + n_imm);
-	ptw->ndata = 1 + n_imm;
+	/* memory arg disp32s (absolute vs the base reg) + fixup disp32s */
+	mem_idx = 0;
+	for (i = 0; i < desc->nargs; i++) {
+		s32 disp;
+
+		if (desc->args[i].src != UPROBE_PTW_SRC_MEM)
+			continue;
+		disp = (s32)desc->args[i].val;
+		*(s32 *)(code + mdisp[mem_idx]) = disp;
+		/*
+		 * fixup's RIP-relative disp: flt slot - end of fixup insn.
+		 * disp32 field starts at flen - 4 in both forms
+		 */
+		*(s32 *)(code + mft[mem_idx].fixup + fixup_len[mem_idx] - 4) =
+			(s32)(flt_off - (mft[mem_idx].fixup +
+					fixup_len[mem_idx]));
+		mem_idx++;
+	}
+
+	/* the shared fault word: failed reads emit 0 */
+	if (n_mem)
+		*(u64 *)(code + flt_off) = 0;
+
+	ptw->stub_len = n_mem ? ft_off + 2 + 6 * n_mem : data_off + 8 * (1 + n_imm);
+	ptw->ndata = 1 + n_imm + (n_mem ? 1 : 0);
+	ptw->ft_off = n_mem ? ft_off : 0;
+	ptw->nft = n_mem;
 	return 0;
 }
 #undef PTW_NEED
@@ -1664,6 +1826,7 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
 	unsigned long block_off, stub_addr;
 	u8 *kaddr, orig[5];
+	s64 site_delta;
 	s32 rel;
 	int ret;
 
@@ -1689,13 +1852,22 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 		return -ENOMEM;
 
 	block_off = ptw->cursor;
-	if (block_off > PAGE_SIZE ||
-	    ptw_a->stub_len > PAGE_SIZE - block_off)
+	if (block_off > PAGE_SIZE || ptw_a->stub_len > PAGE_SIZE - block_off)
+		return -ENOMEM;
+	if (ptw->nblocks >= ARRAY_SIZE(ptw->index))
 		return -ENOMEM;
 	stub_addr = ptw->vaddr + block_off;
 	if (!ptwrite_rel32(stub_addr + ptw_a->jmp_off + 4,
 			   vaddr + 5, &rel))
 		return -ERANGE;
+	site_delta = (s64)vaddr - (s64)ptw->vaddr;
+	if (site_delta < INT_MIN || site_delta > INT_MAX)
+		return -ERANGE;
+
+	ptw->index[ptw->nblocks].off = block_off;
+	ptw->index[ptw->nblocks].len = ptw_a->stub_len;
+	ptw->index[ptw->nblocks].ft_off = ptw_a->ft_off;
+	ptw->nblocks++;
 
 	kaddr = kmap_local_page(ptw->page);
 	memcpy(kaddr + block_off, ptw_a->stub, ptw_a->stub_len);
@@ -1704,9 +1876,13 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	kunmap_local(kaddr);
 
 	ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
-	if (!ret)
-		ptw->cursor = block_off + ptw_a->stub_len;
-	return ret;
+	if (ret)
+		/* Publish rollback before readers use the reduced block count. */
+		smp_store_release(&ptw->nblocks, ptw->nblocks - 1);
+		return ret;
+	}
+	ptw->cursor = block_off + ptw_a->stub_len;
+	return 0;
 }
 
 int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
@@ -1724,6 +1900,98 @@ int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
 			UPROBE_SWBP_INSN, false, false, false, false, NULL);
 }
 
+/*
+ * Ptwrite memory faults are fixed up for fault classes routed through the
+ * user-fault notifier. #AC is intentionally not handled: alignment checking
+ * is normally disabled for user processes and is not a supported ptwrite mode.
+ */
+static bool uprobe_ptwrite_handle_fault(struct pt_regs *regs)
+{
+	struct mm_struct *mm = current->mm;
+	struct uprobes_state *state;
+	struct uprobe_ptwrite_page *ptw;
+	unsigned long ip = instruction_pointer(regs);
+	int b;
+
+	if (!mm)
+		return false;
+	state = &mm->uprobes_state;
+
+	rcu_read_lock();
+	hlist_for_each_entry_rcu(ptw, &state->head_ptwrite, node) {
+		unsigned long boff;
+		u16 nblocks;
+
+		if (ip < ptw->vaddr || ip >= ptw->vaddr + PAGE_SIZE)
+			continue;
+		boff = ip - ptw->vaddr;
+		/* Acquire published metadata before scanning blocks in the fault path. */
+		nblocks = smp_load_acquire(&ptw->nblocks);
+		for (b = 0; b < nblocks; b++) {
+			u16 off = ptw->index[b].off;
+			u16 len = ptw->index[b].len;
+			u16 fto = ptw->index[b].ft_off;
+			u8 *kaddr;
+			u16 nft, i;
+
+			/* no fault table, or IP outside this block: not ours */
+			if (!fto || boff < off || boff >= off + len)
+				continue;
+			if (off >= PAGE_SIZE || len > PAGE_SIZE - off ||
+			    len < sizeof(nft) ||
+			    fto > len - sizeof(nft)) {
+				rcu_read_unlock();
+				return false;
+			}
+			kaddr = kmap_local_page(ptw->page);
+			nft = *(u16 *)(kaddr + off + fto);
+			if (nft > UPROBE_PTWRITE_MAX_ARGS ||
+			    nft > (len - fto - sizeof(nft)) / 6) {
+				kunmap_local(kaddr);
+				rcu_read_unlock();
+				return false;
+			}
+			for (i = 0; i < nft; i++) {
+				u16 *e = (u16 *)(kaddr + off + fto +
+						 sizeof(nft) + i * 6);
+
+				if (e[0] >= e[1] || e[1] > len || e[2] >= len) {
+					kunmap_local(kaddr);
+					rcu_read_unlock();
+					return false;
+				}
+				if (boff >= off + e[0] && boff < off + e[1]) {
+					regs->ip = ptw->vaddr + off + e[2];
+					kunmap_local(kaddr);
+					rcu_read_unlock();
+					return true;
+				}
+			}
+			kunmap_local(kaddr);
+			rcu_read_unlock();
+			return false;
+		}
+	}
+	rcu_read_unlock();
+	return false;
+}
+
+static int uprobe_user_fault_notify(struct notifier_block *self,
+				    unsigned long val, void *data)
+{
+	struct x86_user_fault_args *args = data;
+
+	if (!args || !args->regs)
+		return NOTIFY_DONE;
+
+	if (uprobe_ptwrite_handle_fault(args->regs))
+		return NOTIFY_STOP;
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block uprobe_user_fault_nb = {
+	.notifier_call = uprobe_user_fault_notify,
+};
 
 static bool __is_optimized(struct mm_struct *mm, uprobe_opcode_t *insn, unsigned long vaddr)
 {
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index c0d65ea5353e..c0d189bef8bd 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -213,10 +213,10 @@ enum uprobe_ptwrite_src {
 
 struct uprobe_ptwrite_arg {
 	u8	src;		/* enum uprobe_ptwrite_src */
-	u8	reg;		/* x86-64 GPR index (0=rax..15=r15) for SRC_REG */
+	u8	reg;		/* x86-64 GPR index (0=rax..15=r15) for SRC_REG/SRC_MEM */
 	u8	size;		/* declared type size 1/2/4/8 (decoder hint) */
 	u8	reserved;
-	u64	val;		/* SRC_IMM: constant; SRC_REG: unused */
+	u64	val;		/* SRC_IMM: constant, SRC_MEM: disp32 (low 32 bits) */
 };
 
 struct uprobe_ptwrite_desc {
diff --git a/samples/uprobe-ptwrite/uprobe_ptwrite_test.c b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
index c3b9dd6bec16..5eafc26104ad 100644
--- a/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
+++ b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
@@ -11,9 +11,11 @@
  * Usage (module params):
  *   path=/path/to/prog   file to probe
  *   offset=0xADDR        file offset of the probe site
- *   args="r0,r1,i0x42"             comma-separated;
+ *   args="r0,r1,i0x42,m3,m2:0x8,m4:0x10:4"   comma-separated;
  *                        r<N> = x86-64 GPR index 0..15,
  *                        i<hex> = immediate constant,
+ *                        m<N>[:<disp>][:<size>] = memory arg [reg + disp32],
+ *                        size 4 (u32 load) or 8 (u64 load, default)
  *   event_id=0x1234      identifier carried in the PTW header word
  */
 #include <linux/module.h>
@@ -87,6 +89,43 @@ static int parse_probe_args(void)
 			}
 			a->src = UPROBE_PTW_SRC_IMM;
 			a->val = v;
+		} else if (tok[0] == 'm') {
+			/*
+			 * m<R>[:<disp>][:<size>]: memory arg [reg + disp32],
+			 * size 4 (u32 load) or 8 (u64 load, default)
+			 */
+			char *colon = strchr(tok, ':');
+			char *szs = NULL;
+			unsigned long reg;
+			long long disp = 0;
+			unsigned long size = 8;
+
+			if (colon) {
+				*colon = '\0';
+				szs = strchr(colon + 1, ':');
+				if (szs)
+					*szs++ = '\0';
+			}
+			if (kstrtoul(tok + 1, 10, &reg) || reg > 15) {
+				pr_err("uprobe_ptwrite_test: bad mem reg '%s'\n", tok);
+				goto err;
+			}
+			if (colon && kstrtoll(colon + 1, 0, &disp)) {
+				pr_err("uprobe_ptwrite_test: bad mem disp '%s'\n",
+				       colon + 1);
+				goto err;
+			}
+			if (szs && (kstrtoul(szs, 10, &size) ||
+				   (size != 4 && size != 8))) {
+				pr_err("uprobe_ptwrite_test: bad mem size '%s'\n",
+				       szs);
+				goto err;
+			}
+			a->src = UPROBE_PTW_SRC_MEM;
+			a->reg = reg;
+			a->val = (u64)(s32)disp;
+			a->size = size;
+			desc.flags |= UPROBE_PTWRITE_FL_ALLOW_MEM;
 		} else {
 			pr_err("uprobe_ptwrite_test: bad arg '%s'\n", tok);
 			goto err;
-- 
2.54.0

[RFC v1 11/19] ptwrite uprobes: Add multinop support

[Andi Kleen]

GCC's -fpatchable-function-entry=5 may emit five one-byte NOPs. Normally
that's not safe to patch because some might jump into a later nop.
But for the gcc case it's safe because nobody jumps into the nops.
Add a %multinop that allows the user opting into patching these sites.
This way patching for the gcc instrumentation works.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/uprobes.h               |  1 +
 arch/x86/kernel/uprobes.c                    |  4 +-
 include/linux/uprobes.h                      |  1 +
 kernel/trace/trace_uprobe.c                  | 42 ++++++++++++++------
 samples/uprobe-ptwrite/uprobe_ptwrite_test.c |  6 +++
 5 files changed, 40 insertions(+), 14 deletions(-)

diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index efffdc44f00a..4fc98eafbd5a 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -50,6 +50,7 @@ struct uprobe_ptwrite_arch {
 	u8	orig[MAX_UINSN_BYTES];	/* pristine file bytes, before generic analysis */
 	u16	ft_off;	/* fault table offset within the block (0 if none) */
 	u8	nft;		/* number of fault entries */
+	u8	allow_nop_run;	/* accept a five-byte run of 0x90 */
 };
 
 /* Per-mm page holding generated ptwrite stub blocks (mirrors trampolines). */
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 90e702a4a8e9..e17e0397eaae 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -1401,7 +1401,8 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 		return -EINVAL;
 	if (desc->nargs > UPROBE_PTWRITE_MAX_ARGS)
 		return -E2BIG;
-	if (desc->flags & ~UPROBE_PTWRITE_FL_ALLOW_MEM)
+	if (desc->flags & ~(UPROBE_PTWRITE_FL_ALLOW_MEM |
+			     UPROBE_PTWRITE_FL_ALLOW_NOP_RUN))
 		return -EINVAL;
 
 	/* The generic registration path copied these bytes before this hook. */
@@ -1567,6 +1568,7 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 	ptw->ndata = 1 + n_imm + (n_mem ? 1 : 0);
 	ptw->ft_off = n_mem ? ft_off : 0;
 	ptw->nft = n_mem;
+	ptw->allow_nop_run = !!(desc->flags & UPROBE_PTWRITE_FL_ALLOW_NOP_RUN);
 	return 0;
 }
 #undef PTW_NEED
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index c0d189bef8bd..b93ab24173c7 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -210,6 +210,7 @@ enum uprobe_ptwrite_src {
 
 /* uprobe_ptwrite_desc.flags */
 #define UPROBE_PTWRITE_FL_ALLOW_MEM	BIT(0) /* SRC_MEM args enabled */
+#define UPROBE_PTWRITE_FL_ALLOW_NOP_RUN	BIT(2) /* accept five 1-byte NOPs */
 
 struct uprobe_ptwrite_arg {
 	u8	src;		/* enum uprobe_ptwrite_src */
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index c457c89afd73..b54cc2057329 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -645,6 +645,7 @@ static int __trace_uprobe_create(int argc, const char **argv)
 	enum probe_print_type ptype;
 	bool is_return = false;
 	bool is_ptwrite = false;
+	bool is_nop_run = false;
 	int i, ret, arg_start = 2;
 
 	ref_ctr_offset = 0;
@@ -670,12 +671,6 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	trlog = trace_probe_log_init("trace_uprobe", argc, argv);
 
-	if (argc - 2 > MAX_TRACE_ARGS ||
-	    (is_ptwrite && argc - 2 > UPROBE_PTWRITE_MAX_ARGS)) {
-		trace_probe_log_set_index(2);
-		trace_probe_log_err(0, TOO_MANY_ARGS);
-		return -E2BIG;
-	}
 
 	if (is_ptwrite)
 		event = argv[0][4] ? &argv[0][4] : NULL;	/* after "ptw:" */
@@ -738,8 +733,13 @@ static int __trace_uprobe_create(int argc, const char **argv)
 	/* Check if there is %return suffix */
 	tmp = strchr(arg, '%');
 	if (tmp && is_ptwrite) {
-		trace_probe_log_err(tmp - filename, BAD_ADDR_SUFFIX);
-		return -EINVAL;
+		if (!strcmp(tmp, "%multinop")) {
+			*tmp = '\0';
+			is_nop_run = true;
+		} else {
+			trace_probe_log_err(tmp - filename, BAD_ADDR_SUFFIX);
+			return -EINVAL;
+		}
 	} else if (tmp) {
 		if (!strcmp(tmp, "%return")) {
 			*tmp = '\0';
@@ -756,6 +756,19 @@ static int __trace_uprobe_create(int argc, const char **argv)
 		trace_probe_log_err(arg - filename, BAD_UPROBE_OFFS);
 		return ret;
 	}
+	if (is_ptwrite) {
+		while (arg_start < argc && !strcmp(argv[arg_start], "%multinop")) {
+			is_nop_run = true;
+			arg_start++;
+		}
+	}
+
+	if (argc - arg_start > MAX_TRACE_ARGS ||
+	    (is_ptwrite && argc - arg_start > UPROBE_PTWRITE_MAX_ARGS)) {
+		trace_probe_log_set_index(arg_start);
+		trace_probe_log_err(0, TOO_MANY_ARGS);
+		return -E2BIG;
+	}
 
 	/* setup a probe */
 	trace_probe_log_set_index(0);
@@ -791,8 +804,8 @@ static int __trace_uprobe_create(int argc, const char **argv)
 		kfree(tail);
 	}
 
-	argc -= 2;
-	argv += 2;
+	argc -= arg_start;
+	argv += arg_start;
 
 	tu = alloc_trace_uprobe(group, event, argc, is_return);
 	if (IS_ERR(tu)) {
@@ -815,7 +828,7 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	/* parse arguments */
 	for (i = 0; i < argc; i++) {
-		trace_probe_log_set_index(i + 2);
+		trace_probe_log_set_index(i + arg_start);
 		ret = traceprobe_parse_probe_arg(&tu->tp, i, argv[i], ctx);
 		if (ret)
 			return ret;
@@ -823,13 +836,14 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	if (is_ptwrite) {
 		if (!argc) {
-			trace_probe_log_set_index(2);
+			trace_probe_log_set_index(arg_start);
 			trace_probe_log_err(0, NO_ARG_BODY);
 			return -EINVAL;	/* core rejects desc->nargs == 0 */
 		}
 		tu->is_ptwrite = true;
 		tu->ptwrite_desc.nargs = argc;
-		tu->ptwrite_desc.flags = 0;
+		tu->ptwrite_desc.flags = is_nop_run ?
+			UPROBE_PTWRITE_FL_ALLOW_NOP_RUN : 0;
 		for (i = 0; i < argc; i++) {
 			ret = ptwrite_compile_arg(tu, i);
 			if (ret) {
@@ -888,6 +902,8 @@ static int trace_uprobe_show(struct seq_file *m, struct dyn_event *ev)
 			   trace_probe_group_name(&tu->tp),
 			   trace_probe_name(&tu->tp), tu->filename,
 			   (int)(sizeof(void *) * 2), tu->offset);
+		if (tu->ptwrite_desc.flags & UPROBE_PTWRITE_FL_ALLOW_NOP_RUN)
+			seq_puts(m, "%multinop");
 	} else
 		seq_printf(m, "%c:%s/%s %s:0x%0*lx", c,
 			   trace_probe_group_name(&tu->tp),
diff --git a/samples/uprobe-ptwrite/uprobe_ptwrite_test.c b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
index 5eafc26104ad..b09521e5cd16 100644
--- a/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
+++ b/samples/uprobe-ptwrite/uprobe_ptwrite_test.c
@@ -17,6 +17,7 @@
  *                        m<N>[:<disp>][:<size>] = memory arg [reg + disp32],
  *                        size 4 (u32 load) or 8 (u64 load, default)
  *   event_id=0x1234      identifier carried in the PTW header word
+ *   allow_nop_run=1      accept five one-byte NOPs at the site
  */
 #include <linux/module.h>
 #include <linux/uprobes.h>
@@ -35,6 +36,10 @@ static ushort event_id = 0x1234;
 module_param(event_id, ushort, 0444);
 MODULE_PARM_DESC(event_id, "event id carried in the PTW header word");
 
+static bool allow_nop_run;
+module_param(allow_nop_run, bool, 0444);
+MODULE_PARM_DESC(allow_nop_run, "accept five one-byte NOPs at the site");
+
 static char *args = "r0";
 module_param(args, charp, 0444);
 MODULE_PARM_DESC(args, "comma-separated args: r<N> GPR, i<hex> immediate, m<N>[:disp][:4|8] memory");
@@ -151,6 +156,7 @@ static int __init uprobe_ptwrite_test_init(void)
 	int ret;
 
 	desc.event_id = event_id;
+	desc.flags = allow_nop_run ? UPROBE_PTWRITE_FL_ALLOW_NOP_RUN : 0;
 	ret = parse_probe_args();
 	if (ret)
 		return ret;
-- 
2.54.0

[RFC v1 12/19] ptwrite uprobes: Add pacing to the probes

[Andi Kleen]

When PTWRITEs are too tightly spaced they can lose data. While the decoder
makes some effort to recover from this it can be still annoying for the
user. This patch adds LFENCEs between the individual instructions to
mimimize (mostly avoid) this problem. It can still happen with parallel
branch collection or if a high frequency of timing packets are configured.
The extend also depends on the core-type.

It can be still disabled with %nopace. This is useful when the user knows
the probes are not too tightly spaced. It is usually still needed
with many arguments.

This patch generates LFENCEs for the probes unless disabled.

Compared with %nopace, default pacing increases probe cost by roughly 272%
with PT off, 227% with full tracing, and 48% in snapshot mode on Alder Lake.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/uprobes.h | 21 ++++++++---
 arch/x86/kernel/uprobes.c      | 64 +++++++++++++++++++++++++++++-----
 include/linux/uprobes.h        |  1 +
 kernel/trace/trace_uprobe.c    | 22 ++++++++++++
 4 files changed, 94 insertions(+), 14 deletions(-)

diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 4fc98eafbd5a..2800372b2f5a 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -29,12 +29,23 @@ enum {
 struct uprobe_xol_ops;
 
 /*
- * Stub block array size. Worst case = 250 B (8 MEM args, rsp bases, fault
- * table); 288 leaves 38 B slack. A file-scope static_assert in
- * arch/x86/kernel/uprobes.c re-derives the worst case; prepare() also
- * enforces it with -E2BIG at runtime.
+ * Stub block size. Conservative worst case is 298 bytes: 9-byte header,
+ * one lead fence, eight 21-byte memory forms with one 3-byte fence each,
+ * a 16-byte original-instruction copy, a 5-byte return jump, alignment,
+ * and 66 bytes of data/fault metadata. 384 leaves room. A static_assert in
+ * arch/x86/kernel/uprobes.c checks the bound; prepare() also checks it with
+ * -E2BIG.
  */
-#define UPROBE_PTWRITE_STUB_SIZE	288
+#define UPROBE_PTWRITE_STUB_SIZE	384
+
+/* the out-of-line original-instruction copy slot (x86 max insn length) */
+#define UPROBE_PTWRITE_COPY_SIZE	MAX_UINSN_BYTES
+
+/*
+ * Word pacing: insert this many LFENCEs between emitted ptwrite words and
+ * before the first word, unless UPROBE_PTWRITE_FL_NO_LEAD_PACE is requested.
+ */
+#define UPROBE_PTWRITE_SERIALIZE_LFENCES	1	/* LFENCEs per word gap */
 
 /*
  * ptwrite probe state. The stub template (code + data slots) is built
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index e17e0397eaae..20557423690f 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -1278,6 +1278,25 @@ static int ptwrite_emit_riprel32(u8 *p, s32 disp)
 	return 8;
 }
 
+static int ptwrite_emit_lfence(u8 *p)
+{
+	*p++ = 0x0f;
+	*p++ = 0xae;
+	*p++ = 0xe8;	/* lfence */
+	return 3;
+}
+
+/* the default pacing: one or more fences per word gap. */
+static int ptwrite_emit_lfences(u8 *p)
+{
+	int i;
+
+	for (i = 0; i < UPROBE_PTWRITE_SERIALIZE_LFENCES; i++)
+		p += ptwrite_emit_lfence(p);
+	return UPROBE_PTWRITE_SERIALIZE_LFENCES * 3;
+}
+
+
 bool arch_uprobe_ptwrite_supported(void)
 {
 	u32 eax, ebx, ecx, edx;
@@ -1359,15 +1378,19 @@ int arch_uprobe_ptwrite_fetch(struct uprobe_ptwrite_arg *a,
 
 
 /*
- * Worst-case stub block: header ptwriteq (9) + max args of the largest form
- * (MEM: 5 opcode + SIB + 4 disp + 2 short jmp + 9 fixup = 21 B) + final jmp
- * (5) -> code; data: header + fault-word slots (16); fault table
- * [nft][{start,end,fixup} x nft] (2 + 6*nft). Must fit UPROBE_PTWRITE_STUB_SIZE;
- * prepare() also enforces it with -E2BIG at runtime.
+ * Conservative worst-case stub block: code is 9-byte header plus one lead
+ * fence, eight 21-byte memory forms plus one fence per argument, the
+ * 16-byte original-instruction copy, and a 5-byte return jump, rounded up;
+ * data and metadata add 16 + 2 + 6 * 8 bytes. This is 298 bytes with the
+ * current fence count and must remain below UPROBE_PTWRITE_STUB_SIZE.
  */
-static_assert((((9 + UPROBE_PTWRITE_MAX_ARGS * 21 + 5 + 7) & ~7) +
-	       16 + 2 + 6 * UPROBE_PTWRITE_MAX_ARGS) <= UPROBE_PTWRITE_STUB_SIZE,
-	       "worst-case ptwrite stub block exceeds UPROBE_PTWRITE_STUB_SIZE");
+static_assert((((9 + UPROBE_PTWRITE_SERIALIZE_LFENCES * 3 +
+		UPROBE_PTWRITE_MAX_ARGS * (21 +
+			UPROBE_PTWRITE_SERIALIZE_LFENCES * 3) +
+		UPROBE_PTWRITE_COPY_SIZE + 5 + 7) & ~7) +
+		16 + 2 + 6 * UPROBE_PTWRITE_MAX_ARGS) <=
+		UPROBE_PTWRITE_STUB_SIZE,
+		"worst-case ptwrite stub block exceeds UPROBE_PTWRITE_STUB_SIZE");
 
 static bool ptwrite_has_room(const u8 *base, const u8 *p, size_t len)
 {
@@ -1375,6 +1398,7 @@ static bool ptwrite_has_room(const u8 *base, const u8 *p, size_t len)
 		       sizeof(((struct uprobe_ptwrite_arch *)0)->stub) - len;
 }
 
+
 #define PTW_NEED(_len) do { \
 		if (!ptwrite_has_room(code, p, (_len))) \
 			return -E2BIG; \
@@ -1394,6 +1418,7 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 	unsigned int data_off, flt_off, ft_off;
 	unsigned int hdr_off = 0;
 	unsigned int imm_idx = 0, n_imm = 0, mem_idx = 0, n_mem = 0;
+	bool paced = false;
 	u64 hdr;
 	int i;
 
@@ -1441,9 +1466,26 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 		}
 	}
 
+	/* Slow down the probes to avoid PT overflow. */
+#define PTW_NEED(_len) do { \
+		if (!ptwrite_has_room(code, p, (_len))) \
+			return -E2BIG; \
+	} while (0)
+
+	paced = !(desc->flags & UPROBE_PTWRITE_FL_NO_LEAD_PACE);
+	if (paced) {
+		PTW_NEED(UPROBE_PTWRITE_SERIALIZE_LFENCES * 3);
+		p += ptwrite_emit_lfences(p);
+	}
+
 	/* header word emission (disp32 patched below) */
 	PTW_NEED(9);
+	hdr_off = p - code;
 	p += ptwrite_emit_riprel(p, 0);
+	if (paced) {
+		PTW_NEED(UPROBE_PTWRITE_SERIALIZE_LFENCES * 3);
+		p += ptwrite_emit_lfences(p);
+	}
 
 	for (i = 0; i < desc->nargs; i++) {
 		switch (desc->args[i].src) {
@@ -1498,9 +1540,13 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 			break;
 		}
 		}
+		if (paced && i + 1 < desc->nargs) {
+			PTW_NEED(UPROBE_PTWRITE_SERIALIZE_LFENCES * 3);
+			p += ptwrite_emit_lfences(p);
+		}
 	}
 
-	/* final jmp back to probe+5; rel32 patched per-mm at install */
+	/* final jmp back to probe+len; rel32 patched per-mm at install */
 	PTW_NEED(5);
 	*p++ = 0xe9;
 	if (p - code > U8_MAX)
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index b93ab24173c7..b39abd1b057d 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -210,6 +210,7 @@ enum uprobe_ptwrite_src {
 
 /* uprobe_ptwrite_desc.flags */
 #define UPROBE_PTWRITE_FL_ALLOW_MEM	BIT(0) /* SRC_MEM args enabled */
+#define UPROBE_PTWRITE_FL_NO_LEAD_PACE	BIT(1) /* don't slow down probes */
 #define UPROBE_PTWRITE_FL_ALLOW_NOP_RUN	BIT(2) /* accept five 1-byte NOPs */
 
 struct uprobe_ptwrite_arg {
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index b54cc2057329..7625ee8c7efc 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -645,6 +645,7 @@ static int __trace_uprobe_create(int argc, const char **argv)
 	enum probe_print_type ptype;
 	bool is_return = false;
 	bool is_ptwrite = false;
+	bool is_nopace = false;
 	bool is_nop_run = false;
 	int i, ret, arg_start = 2;
 
@@ -732,6 +733,11 @@ static int __trace_uprobe_create(int argc, const char **argv)
 
 	/* Check if there is %return suffix */
 	tmp = strchr(arg, '%');
+	if (tmp && is_ptwrite && !strcmp(tmp, "%nopace")) {
+		*tmp = '\0';
+		is_nopace = true;
+		tmp = NULL;
+	}
 	if (tmp && is_ptwrite) {
 		if (!strcmp(tmp, "%multinop")) {
 			*tmp = '\0';
@@ -756,11 +762,20 @@ static int __trace_uprobe_create(int argc, const char **argv)
 		trace_probe_log_err(arg - filename, BAD_UPROBE_OFFS);
 		return ret;
 	}
+	if (is_ptwrite && arg_start < argc &&
+	    !strcmp(argv[arg_start], "%nopace")) {
+		is_nopace = true;
+		arg_start++;
+	}
 	if (is_ptwrite) {
 		while (arg_start < argc && !strcmp(argv[arg_start], "%multinop")) {
 			is_nop_run = true;
 			arg_start++;
 		}
+		if (arg_start < argc && !strcmp(argv[arg_start], "%nopace")) {
+			is_nopace = true;
+			arg_start++;
+		}
 	}
 
 	if (argc - arg_start > MAX_TRACE_ARGS ||
@@ -844,6 +859,8 @@ static int __trace_uprobe_create(int argc, const char **argv)
 		tu->ptwrite_desc.nargs = argc;
 		tu->ptwrite_desc.flags = is_nop_run ?
 			UPROBE_PTWRITE_FL_ALLOW_NOP_RUN : 0;
+		if (is_nopace)
+			tu->ptwrite_desc.flags |= UPROBE_PTWRITE_FL_NO_LEAD_PACE;
 		for (i = 0; i < argc; i++) {
 			ret = ptwrite_compile_arg(tu, i);
 			if (ret) {
@@ -902,6 +919,11 @@ static int trace_uprobe_show(struct seq_file *m, struct dyn_event *ev)
 			   trace_probe_group_name(&tu->tp),
 			   trace_probe_name(&tu->tp), tu->filename,
 			   (int)(sizeof(void *) * 2), tu->offset);
+		if (tu->ptwrite_desc.flags & UPROBE_PTWRITE_FL_NO_LEAD_PACE) {
+			seq_puts(m, "%nopace");
+			if (tu->ptwrite_desc.flags & UPROBE_PTWRITE_FL_ALLOW_NOP_RUN)
+				seq_putc(m, ' ');
+		}
 		if (tu->ptwrite_desc.flags & UPROBE_PTWRITE_FL_ALLOW_NOP_RUN)
 			seq_puts(m, "%multinop");
 	} else
-- 
2.54.0

[RFC v1 13/19] ptwrite uprobes: Support instruction puning

[Andi Kleen]

The previous ptwrite instrumentation only worked on 5 byte+ nops because
it needs to patch in a 5 byte branch. But the instruction may be shorter
and the kernel cannot prove that nobody jumps to the next instruction.

However that is somewhat limiting because it means most code cannot
be probed. This patch implements a simplified variant of the instruction
puning technique from Chamith et.al. "Instruction Punning: Lightweight
instrumentation for x86-64". The basic idea is to only patch in the
one byte opcode for the branch and reuse the existing instruction
bytes in the code as the branch target.

If someone branches to the remaining bytes they are still executed
in the original way because they didn't change.

This requires placing a target trampoline page at the right address. If
the area is not available or points to kernel space it doesn't work.

In general it is somewhat unreliable for non PIE executables because
the target is often negative and ends up in kernel space. However
on modern Linux distributions near all binaries are PIE and high
up in the address space with ample gaps around them, which makes
puning have a high success rate.

For example trying to probe every instruction in a PIE linked
Debian 13 /bin/bash:

bash has around 210k instructions, of which around 10k were directly
patchable 5 byte nops. Running it for 100 times with ASLR 76.4% of
the instructions were always pun probeable, with 23% of the instruction
never being punnable (and the rest sometimes). So punning is not
perfect, but still works most of the time, and is much better than just
nops.

The original paper used various fallback techniques to improve success,
but I found things work well enough anyways with PIE and PIC.

If the probing fails it's possible for the harness to move the probe
around until it finds a better target. Sometimes you're just
lucky on rerun with ASLR. Or alternatively just use a classic
uprobe.

I made sure to give all instruction error cases unique errnos so
that the harness can make informed decisions.

This patch adds the low level machinery for puning. Classify
the instruction, poke the target and map trampolines to the right place.
One difference to the nop probing is that the previous instruction
needs to be copied and fixed up (analogous to classic uprobes)

32bit relative ranches cannot be probed because their target is not
available by definition. For 8bit relative branches there are also cases
where it might need a second trampoline, or APX JMPABS support,
to reach the true target, so these are rejected too for now.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/include/asm/uprobes.h |  11 +-
 arch/x86/kernel/uprobes.c      | 480 ++++++++++++++++++++++++++++++---
 include/linux/uprobes.h        |   4 +-
 kernel/events/uprobes.c        |  18 +-
 4 files changed, 470 insertions(+), 43 deletions(-)

diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 2800372b2f5a..ba751af4204b 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -49,14 +49,19 @@ struct uprobe_xol_ops;
 
 /*
  * ptwrite probe state. The stub template (code + data slots) is built
- * once at registration (mm-independent except the final jmp's rel32, patched
- * per-mm at install). Block layout:
- *   [ptwriteq hdr(%rip)] [arg emissions] [jmp probe+5] [u64 slots: header, imms]
+ * once at registration. Only the final jmp's rel32 and the copy's
+ * disp/rel fields are patched per-mm at install. Block layout:
+ *   [ptwriteq hdr(%rip)] [arg emissions] [orig-insn copy]
+ *   [jmp probe+len] [u64 slots: header, imms]
  */
 struct uprobe_ptwrite_arch {
 	u8	stub[UPROBE_PTWRITE_STUB_SIZE];
 	u16	stub_len;	/* code + data + fault table, whole block */
 	u8	jmp_off;	/* offset of the final jmp's rel32 field */
+	u8	copy_off;	/* offset of the out-of-line instruction copy */
+	u8	len;		/* copy length (0 = drop); back-jmp = vaddr+len */
+	u8	disp_off;	/* rip-relative disp32 offset in the copy (0 = none) */
+	s32	disp;		/* original disp32 (delta-patched per-mm) */
 	u8	ndata;		/* number of u64 data slots */
 	u8	orig[MAX_UINSN_BYTES];	/* pristine file bytes, before generic analysis */
 	u16	ft_off;	/* fault table offset within the block (0 if none) */
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 20557423690f..806e40f7b0ab 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -958,8 +958,17 @@ static int verify_insn(struct page *page, unsigned long vaddr, uprobe_opcode_t *
 {
 	struct write_opcode_ctx *ctx = data;
 	uprobe_opcode_t old_opcode[OPT_INSN_SIZE];
+	int len;
 
-	uprobe_copy_from_page(page, ctx->base, old_opcode, OPT_INSN_SIZE);
+	/*
+	 * Byte-state checks need only the first byte. Optimized-state checks
+	 * inspect the complete ten-byte instruction.
+	 */
+	len = ctx->expect == EXPECT_OPTIMIZED ||
+		ctx->expect == EXPECT_SWBP_OPTIMIZED ? OPT_INSN_SIZE : 1;
+	if (PAGE_SIZE - (ctx->base & ~PAGE_MASK) < len)
+		return -1;
+	uprobe_copy_from_page(page, ctx->base, old_opcode, len);
 
 	switch (ctx->expect) {
 	case EXPECT_SWBP:
@@ -1398,6 +1407,11 @@ static bool ptwrite_has_room(const u8 *base, const u8 *p, size_t len)
 		       sizeof(((struct uprobe_ptwrite_arch *)0)->stub) - len;
 }
 
+static bool pun_site_is_nop(const u8 *orig, bool allow_nop_run);
+static int pun_classify_insn(struct insn *insn, u8 *disp_off, s32 *disp);
+static int pun_decode_site(struct inode *inode, struct file *file,
+			       loff_t offset, u8 *copy,
+			       u8 *disp_off, s32 *disp, bool allow_nop_run);
 
 #define PTW_NEED(_len) do { \
 		if (!ptwrite_has_room(code, p, (_len))) \
@@ -1407,6 +1421,8 @@ static bool ptwrite_has_room(const u8 *base, const u8 *p, size_t len)
 
 
 int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
+				struct inode *inode, struct file *file,
+				loff_t offset,
 				const struct uprobe_ptwrite_desc *desc)
 {
 	struct uprobe_ptwrite_arch *ptw = &auprobe->ptwrite;
@@ -1416,17 +1432,18 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 	struct { u16 start, end, fixup; } __packed mft[UPROBE_PTWRITE_MAX_ARGS];
 	u16 mdisp[UPROBE_PTWRITE_MAX_ARGS];
 	unsigned int data_off, flt_off, ft_off;
-	unsigned int hdr_off = 0;
 	unsigned int imm_idx = 0, n_imm = 0, mem_idx = 0, n_mem = 0;
+	unsigned int hdr_off = 0;
 	bool paced = false;
 	u64 hdr;
-	int i;
+	int i, ret;
 
 	if (!desc || desc->nargs == 0)
 		return -EINVAL;
 	if (desc->nargs > UPROBE_PTWRITE_MAX_ARGS)
 		return -E2BIG;
 	if (desc->flags & ~(UPROBE_PTWRITE_FL_ALLOW_MEM |
+			     UPROBE_PTWRITE_FL_NO_LEAD_PACE |
 			     UPROBE_PTWRITE_FL_ALLOW_NOP_RUN))
 		return -EINVAL;
 
@@ -1546,6 +1563,13 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 		}
 	}
 
+	/* the out-of-line original-instruction copy slot (patched per-mm) */
+	PTW_NEED(UPROBE_PTWRITE_COPY_SIZE);
+	if (p - code > U8_MAX)
+		return -E2BIG;
+	ptw->copy_off = p - code;
+	p += UPROBE_PTWRITE_COPY_SIZE;
+
 	/* final jmp back to probe+len; rel32 patched per-mm at install */
 	PTW_NEED(5);
 	*p++ = 0xe9;
@@ -1615,6 +1639,15 @@ int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
 	ptw->ft_off = n_mem ? ft_off : 0;
 	ptw->nft = n_mem;
 	ptw->allow_nop_run = !!(desc->flags & UPROBE_PTWRITE_FL_ALLOW_NOP_RUN);
+
+	ret = pun_decode_site(inode, file, offset, code + ptw->copy_off,
+				  &ptw->disp_off, &ptw->disp,
+				  ptw->allow_nop_run);
+	if (ret < 0)
+		return ret;
+	ptw->len = ret;
+	memset(code + ptw->copy_off + ptw->len, 0x90,
+	       UPROBE_PTWRITE_COPY_SIZE - ptw->len);
 	return 0;
 }
 #undef PTW_NEED
@@ -1711,15 +1744,10 @@ static unsigned long find_ptwrite_page_area(struct mm_struct *mm,
 }
 
 static struct uprobe_ptwrite_page *
-create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
+create_uprobe_ptwrite_page_at(struct mm_struct *mm, unsigned long area)
 {
 	struct uprobe_ptwrite_page *ptw;
 	struct vm_area_struct *vma;
-	unsigned long area;
-
-	area = find_ptwrite_page_area(mm, vaddr);
-	if (IS_ERR_VALUE(area))
-		return NULL;
 
 	mmap_assert_write_locked(mm);
 
@@ -1744,6 +1772,17 @@ create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
 	}
 	return ptw;
 }
+
+static struct uprobe_ptwrite_page *
+create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
+{
+	unsigned long area = find_ptwrite_page_area(mm, vaddr);
+
+	if (IS_ERR_VALUE(area))
+		return NULL;
+	return create_uprobe_ptwrite_page_at(mm, area);
+}
+
 static struct uprobe_ptwrite_page *
 get_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr,
 			unsigned int len)
@@ -1772,31 +1811,127 @@ get_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr,
 	return ptw;
 }
 
-/* Probe site must be a 5-byte NOP that does not cross a page boundary. */
-static int ptwrite_validate_site(const u8 *orig, unsigned long vaddr)
+/*
+ * A run of short NOPs is accepted only when requested. This validation does
+ * not make the three-phase poke safe for threads that already passed byte 0.
+ */
+static bool pun_site_is_nop(const u8 *orig, bool allow_nop_run)
 {
 	struct insn insn;
 	int ret;
-	int off = 0;
 
-	/*
-	 * The 5 displaced bytes must be NOPs: either one 5-byte NOP
-	 * (nopl 0x0(%rax,%rax,1)) or a run of shorter NOPs summing to
-	 * exactly 5 (gcc -fpatchable-function-entry=5 emits 5 x 0x90 on
-	 * modern toolchains). Any non-NOP byte, or a NOP crossing the
-	 * 5-byte window, is rejected.
-	 */
-	while (off < 5) {
-		ret = insn_decode(&insn, orig + off, 5 - off, INSN_MODE_64);
-		if (ret < 0)
-			return -EINVAL;
-		if (insn.length < 1 || insn.length > 5 - off ||
-		    !insn_is_nop(&insn))
-			return -EINVAL;
-		off += insn.length;
+	ret = insn_decode(&insn, orig, 5, INSN_MODE_64);
+	if (ret < 0)
+		return false;
+	if (insn.length == 5 && insn_is_nop(&insn))
+		return true;
+	if (!allow_nop_run)
+		return false;
+	return orig[0] == 0x90 && orig[1] == 0x90 &&
+		orig[2] == 0x90 && orig[3] == 0x90 && orig[4] == 0x90;
+}
+
+/*
+ * Classify the site's single instruction for out-of-line execution.
+ * Returns the length, or 0 when it cannot run safely out of line.
+ */
+static int pun_classify_insn(struct insn *insn, u8 *disp_off, s32 *disp)
+{
+	u8 op = insn->opcode.bytes[0];
+	switch (op) {
+	case 0xcc:	/* int3 */
+	case 0xcd:	/* int imm8 */
+	case 0xce:	/* into */
+	case 0xcf:	/* iret */
+	case 0xf1:	/* int1 */
+	case 0xea:	/* jmp far */
+	case 0x9a:	/* call far */
+	/* Could be handled with special case code. */
+	case 0xe8:	/* call rel32 */
+	case 0xe0:	/* loopne rel8: cannot run out of line */
+	case 0xe1:	/* loope rel8 */
+	case 0xe2:	/* loop rel8 */
+	case 0xe3:	/* jecxz/jrcxz */
+	/* These two could be handled if the offsets fit */
+	case 0xe9:	/* jmp rel32 */
+	case 0xeb:	/* jmp rel8 */
+	case 0x70 ... 0x7f:	/* jcc rel8 */
+		return -EOPNOTSUPP;
 	}
-	if (off != 5)
+	/* XBEGIN's rel32 abort target is IP-relative, not RIP-relative. */
+	if (op == 0xc7 && insn->modrm.nbytes &&
+	    X86_MODRM_MOD(insn->modrm.value) == 3 &&
+	    X86_MODRM_REG(insn->modrm.value) == 7 &&
+	    X86_MODRM_RM(insn->modrm.value) == 0)
+		return -EOPNOTSUPP;
+	if (op == 0x0f) {
+		switch (insn->opcode.bytes[1]) {
+		case 0x05:	/* syscall */
+		case 0x34:	/* sysenter */
+		case 0x35:	/* sysexit */
+			return -EOPNOTSUPP;
+		}
+		/* jcc rel32: could be handled if offsets fit */
+		if (insn->opcode.bytes[1] >= 0x80 &&
+		    insn->opcode.bytes[1] <= 0x8f)
+			return -EOPNOTSUPP;
+		/* Allow endbranch because this is incompatible with CET anyways */
+	}
+	if (op == 0xff) {
+		u8 reg = X86_MODRM_REG(insn->modrm.value);
+
+		/* call/lcall/jmp-far indirect */
+		if (reg == 2 || reg == 3 || reg == 5)
+			return -EOPNOTSUPP;
+	}
+
+	if (insn_rip_relative(insn)) {
+		*disp_off = insn_offset_displacement(insn);
+		insn_get_displacement(insn);
+		*disp = insn->displacement.value;
+	}
+	return insn->length;
+}
+
+/*
+ * Read the site's instruction bytes from the file and classify them.
+ * The bytes are identical in every mm, so the copy is mm-independent.
+ */
+static int pun_decode_site(struct inode *inode, struct file *file,
+			       loff_t offset, u8 *copy,
+			       u8 *disp_off, s32 *disp, bool allow_nop_run)
+{
+	u8 buf[MAX_UINSN_BYTES];
+	struct insn insn;
+	int ret;
+
+	ret = uprobe_copy_from_file(inode, file, offset, buf,
+				    MAX_UINSN_BYTES);
+	if (ret < 0)
+		return ret;
+	if (ret != MAX_UINSN_BYTES)
+		return -EIO;
+
+	if (pun_site_is_nop(buf, allow_nop_run))
+		return 0;
+
+	/* Check single instruction */
+	if (insn_decode(&insn, buf, MAX_UINSN_BYTES, INSN_MODE_64))
+		return -EINVAL;
+	if (insn.length < 1 || insn.length > MAX_UINSN_BYTES)
 		return -EINVAL;
+
+	/* the original bytes verbatim; classify only validates them */
+	memcpy(copy, buf, insn.length);
+	return pun_classify_insn(&insn, disp_off, disp);
+}
+
+static int ptwrite_validate_site(const u8 *orig, unsigned long vaddr)
+{
+	/*
+	 * Site installation reads and patches five bytes from one user page.
+	 * Do not let those page-relative copies cross into an unpinned page.
+	 */
 	if (PAGE_SIZE - (vaddr & ~PAGE_MASK) < 5)
 		return -EINVAL;
 	return 0;
@@ -1866,10 +2001,210 @@ static int ptwrite_text_poke(struct arch_uprobe *auprobe,
 	return err;
 }
 
+/*
+ * Replace an aligned five-byte NOP run with a JMP in one eight-byte store.
+ * The trailing three bytes are read from the existing text. We assume
+ * nobody else is changing it. This is covered by the Intel/AMD "aligned store"
+ * cross modifying guarantee.
+ */
+static int ptwrite_multinop_text_poke(struct arch_uprobe *auprobe,
+				      struct vm_area_struct *vma,
+				      unsigned long vaddr,
+				      unsigned long stub_addr)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	struct write_opcode_ctx ctx = {
+		.base = vaddr,
+		.expect = EXPECT_BYTE,
+		.expect_byte = 0x90,
+	};
+	u8 patch[8];
+	s32 rel;
+	int err;
+
+	if (vaddr & 7)
+		return -EINVAL;
+	if (!ptwrite_rel32(vaddr + 5, stub_addr, &rel))
+		return -ERANGE;
+	err = copy_from_vaddr(mm, vaddr, patch, sizeof(patch));
+	if (err)
+		return err;
+	patch[0] = 0xe9;
+	memcpy(&patch[1], &rel, sizeof(rel));
+	err = uprobe_write(auprobe, vma, vaddr, patch, sizeof(patch),
+			   verify_insn, true, false, &ctx);
+	if (!err)
+		smp_text_poke_sync_each_cpu();
+	return err;
+}
+
+static int pun_text_poke(struct arch_uprobe *auprobe,
+				 struct vm_area_struct *vma,
+				 unsigned long vaddr, u8 e9,
+				 struct write_opcode_ctx *ctx)
+{
+	int err;
+
+	err = uprobe_write(auprobe, vma, vaddr, &e9, 1, verify_insn,
+			   true, false, ctx);
+	if (err)
+		return err;
+	smp_text_poke_sync_each_cpu();
+	return 0;
+}
+
+static int pun_install(struct arch_uprobe *auprobe,
+			       struct vm_area_struct *vma, unsigned long vaddr,
+			       const u8 *orig)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	struct uprobe_ptwrite_page *ptw;
+	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
+	struct uprobes_state *state = &mm->uprobes_state;
+	struct write_opcode_ctx ctx = {
+		.base = vaddr,
+		.expect = EXPECT_BYTE,
+		.expect_byte = orig[0],
+	};
+	unsigned long t, page_base, block_off, stub_addr;
+	s64 site_delta;
+	s32 jump_rel, disp32, orig_rel;
+	u8 site_len;
+	bool found = false;
+	bool nop_fallback = ptwrite_site_is_multinop(orig,
+						     ptw_a->allow_nop_run) &&
+			    (vaddr & 7);
+	u8 *kaddr;
+	int b, ret;
+
+	mmap_assert_write_locked(mm);
+	if (nop_fallback) {
+		hlist_for_each_entry(ptw, &state->head_ptwrite, node) {
+			site_delta = (s64)vaddr - (s64)ptw->vaddr;
+			if (site_delta < INT_MIN || site_delta > INT_MAX)
+				continue;
+			for (b = 0; b < smp_load_acquire(&ptw->nblocks); b++)
+				if (!ptw->index[b].pun &&
+				    ptw->index[b].site_off == (s32)site_delta &&
+				    ptw->index[b].site_len == 5 &&
+				    !memcmp(ptw->index[b].site_insn, orig, 5))
+					break;
+			if (b >= smp_load_acquire(&ptw->nblocks))
+				continue;
+			if (!__in_uprobe_ptwrite(mm, ptw->vaddr))
+				continue;
+			return ptwrite_text_poke(auprobe, vma, vaddr,
+						 ptw->vaddr + ptw->index[b].off);
+		}
+		ptw = get_uprobe_ptwrite_page(mm, vaddr, ptw_a->stub_len);
+		if (!ptw)
+			return -ENOMEM;
+		block_off = ptw->cursor;
+	} else {
+		memcpy(&orig_rel, orig + 1, sizeof(orig_rel));
+		t = (unsigned long)((s64)vaddr + 5 + (s64)orig_rel);
+		if ((s64)vaddr + 5 + (s64)orig_rel < PAGE_SIZE ||
+		    (s64)vaddr + 5 + (s64)orig_rel >= TASK_SIZE_MAX)
+			return -EADDRNOTAVAIL;
+		page_base = t & PAGE_MASK;
+		block_off = t & (PAGE_SIZE - 1);
+		if (block_off + ptw_a->stub_len > PAGE_SIZE)
+			return -ENOSPC;
+
+		/* reuse an existing ptwrite page at the target, else map a new one */
+		hlist_for_each_entry(ptw, &state->head_ptwrite, node) {
+			if (ptw->vaddr == page_base) {
+				found = true;
+				break;
+			}
+		}
+		if (!found) {
+			if (vma_lookup(mm, page_base))
+				return -EADDRNOTAVAIL;	/* target page occupied */
+			ptw = create_uprobe_ptwrite_page_at(mm, page_base);
+			if (!ptw)
+				return -ENOMEM;
+			/* Order page initialization before publishing it to fault readers. */
+			smp_wmb();
+			hlist_add_head_rcu(&ptw->node, &state->head_ptwrite);
+		}
+	}
+
+	site_delta = (s64)vaddr - (s64)ptw->vaddr;
+	if (site_delta < INT_MIN || site_delta > INT_MAX)
+		return -ERANGE;
+
+	for (b = 0; b < ptw->nblocks; b++) {
+		if (ptw->index[b].off != block_off || !ptw->index[b].pun)
+			continue;
+		if (ptw->index[b].site_off != (s32)site_delta ||
+		    ptw->index[b].site_len != ptw_a->len ||
+		    memcmp(ptw->index[b].site_insn, ptw_a->orig,
+			   ptw_a->len))
+			return -EADDRNOTAVAIL;
+		return pun_text_poke(auprobe, vma, vaddr, 0xe9, &ctx);
+	}
+	if (block_off < ptw->cursor)
+		return -ENOSPC;
+	if (ptw->nblocks >= ARRAY_SIZE(ptw->index))
+		return -ENOMEM;
+
+	stub_addr = ptw->vaddr + block_off;
+	if (!ptwrite_rel32(stub_addr + ptw_a->jmp_off + 4,
+			   vaddr + (ptw_a->len ? ptw_a->len : 5), &jump_rel))
+		return -ERANGE;
+	if (ptw_a->len && ptw_a->disp_off) {
+		s64 d = (s64)ptw_a->disp + (s64)vaddr -
+			(s64)(stub_addr + ptw_a->copy_off);
+
+		if (d < INT_MIN || d > INT_MAX)
+			return -ERANGE;
+		disp32 = (s32)d;
+	}
+
+	/*
+	 * A NOP fallback needs a synthetic rel32 at the site, so it uses
+	 * the full five-byte poke and restore path rather than punning.
+	 */
+	site_len = nop_fallback ? 5 : ptw_a->len;
+	ptw->index[ptw->nblocks].off = block_off;
+	ptw->index[ptw->nblocks].len = ptw_a->stub_len;
+	ptw->index[ptw->nblocks].ft_off = ptw_a->ft_off;
+	ptw->index[ptw->nblocks].pun = !nop_fallback;
+	ptw->index[ptw->nblocks].orig0 = orig[0];
+	ptw->index[ptw->nblocks].site_len = site_len;
+	ptw->index[ptw->nblocks].site_off = (s32)site_delta;
+	memcpy(ptw->index[ptw->nblocks].site_insn, orig, site_len);
+	smp_store_release(&ptw->nblocks, ptw->nblocks + 1);
+
+	kaddr = kmap_local_page(ptw->page);
+	memcpy(kaddr + block_off, ptw_a->stub, ptw_a->stub_len);
+	memcpy(kaddr + block_off + ptw_a->jmp_off, &jump_rel, sizeof(jump_rel));
+	if (ptw_a->len && ptw_a->disp_off)
+		memcpy(kaddr + block_off + ptw_a->copy_off + ptw_a->disp_off,
+		       &disp32, sizeof(disp32));
+	kunmap_local(kaddr);
+
+	if (nop_fallback)
+		ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
+	else
+		ret = pun_text_poke(auprobe, vma, vaddr, 0xe9, &ctx);
+	if (ret) {
+		/* Publish rollback before readers observe the reduced block count. */
+		smp_store_release(&ptw->nblocks, ptw->nblocks - 1);
+		return ret;
+	}
+
+	set_bit(ARCH_UPROBE_FLAG_PTWRITE, &auprobe->flags);
+	ptw->cursor = block_off + ptw_a->stub_len;
+	return 0;
+}
+
 int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 		struct vm_area_struct *vma, unsigned long vaddr)
 {
 	struct mm_struct *mm = vma->vm_mm;
+	struct uprobes_state *state = &mm->uprobes_state;
 	struct uprobe_ptwrite_page *ptw;
 	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
 	unsigned long block_off, stub_addr;
@@ -1877,6 +2212,7 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	s64 site_delta;
 	s32 rel;
 	int ret;
+	int b;
 
 	if (!is_64bit_mm(mm))
 		return -EOPNOTSUPP;
@@ -1894,7 +2230,29 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	ret = ptwrite_validate_site(orig, vaddr);
 	if (ret)
 		return ret;
+	if (!pun_site_is_nop(orig, ptw_a->allow_nop_run))
+		return pun_install(auprobe, vma, vaddr, orig);
 
+	hlist_for_each_entry(ptw, &state->head_ptwrite, node) {
+		site_delta = (s64)vaddr - (s64)ptw->vaddr;
+		if (site_delta < INT_MIN || site_delta > INT_MAX)
+			continue;
+		/* Acquire the published count before reading block metadata. */
+		for (b = 0; b < smp_load_acquire(&ptw->nblocks); b++)
+			if (!ptw->index[b].pun &&
+			    ptw->index[b].site_off == (s32)site_delta &&
+			    ptw->index[b].site_len == sizeof(orig) &&
+			    !memcmp(ptw->index[b].site_insn, orig, sizeof(orig)))
+				break;
+		/* Recheck the published count with acquire ordering. */
+		if (b >= smp_load_acquire(&ptw->nblocks))
+			continue;
+		if (!__in_uprobe_ptwrite(mm, ptw->vaddr))
+			continue;
+		ret = ptwrite_text_poke(auprobe, vma, vaddr,
+					ptw->vaddr + ptw->index[b].off);
+		goto out;
+	}
 	ptw = get_uprobe_ptwrite_page(mm, vaddr, ptw_a->stub_len);
 	if (!ptw)
 		return -ENOMEM;
@@ -1915,7 +2273,13 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	ptw->index[ptw->nblocks].off = block_off;
 	ptw->index[ptw->nblocks].len = ptw_a->stub_len;
 	ptw->index[ptw->nblocks].ft_off = ptw_a->ft_off;
-	ptw->nblocks++;
+	ptw->index[ptw->nblocks].pun = 0;
+	ptw->index[ptw->nblocks].orig0 = orig[0];
+	ptw->index[ptw->nblocks].site_len = sizeof(orig);
+	ptw->index[ptw->nblocks].site_off = (s32)site_delta;
+	memcpy(ptw->index[ptw->nblocks].site_insn, orig, sizeof(orig));
+	/* Publish initialized metadata before exposing the probe jump. */
+	smp_store_release(&ptw->nblocks, ptw->nblocks + 1);
 
 	kaddr = kmap_local_page(ptw->page);
 	memcpy(kaddr + block_off, ptw_a->stub, ptw_a->stub_len);
@@ -1937,15 +2301,63 @@ int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
 		struct vm_area_struct *vma, unsigned long vaddr)
 {
 	struct mm_struct *mm = vma->vm_mm;
+	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
+	struct uprobes_state *state = &mm->uprobes_state;
 	u8 cur[5];
+	int b;
+	struct write_opcode_ctx ctx = {
+		.base = vaddr,
+		.expect = EXPECT_BYTE,
+		.expect_byte = 0xe9,
+	};
 
 	mmap_assert_write_locked(mm);
-	if (copy_from_vaddr(mm, vaddr, cur, sizeof(cur)) ||
-	    !ptwrite_is_installed(mm, vaddr, cur))
-		return;
+	{
+		struct uprobe_ptwrite_page *ptw;
+		struct uprobe_ptwrite_page *fpw = NULL;
+		s32 rel;
+		s64 target;
+		unsigned long page_base, boff;
+		int ret;
+
+		ret = copy_from_vaddr(mm, vaddr, cur, sizeof(cur));
+		if (ret)
+			return ret;
+		if (!ptwrite_is_installed(mm, vaddr, cur))
+			return 0;
+
+		memcpy(&rel, cur + 1, sizeof(rel));
+		target = (s64)vaddr + 5 + (s64)rel;
+		if (target < PAGE_SIZE || target >= TASK_SIZE_MAX)
+			return text_poke_5byte(auprobe, vma, vaddr, ptw_a->orig,
+					0xe9, false, false, false, false, NULL);
+		page_base = (unsigned long)target & PAGE_MASK;
+		boff = (unsigned long)target & (PAGE_SIZE - 1);
+		hlist_for_each_entry(ptw, &state->head_ptwrite, node)
+			if (ptw->vaddr == page_base) {
+				fpw = ptw;
+				break;
+			}
+		if (fpw)
+			/* Acquire the published count before reading pun metadata. */
+			for (b = 0; b < smp_load_acquire(&fpw->nblocks); b++)
+				if (fpw->index[b].off == boff)
+					break;
+		if (fpw && b < smp_load_acquire(&fpw->nblocks) &&
+		    fpw->index[b].pun) {
+			u8 orig0 = fpw->index[b].orig0;
+
+			ret = uprobe_write(auprobe, vma, vaddr, &orig0, 1,
+					   verify_insn, false, false, &ctx);
+			if (ret)
+				return ret;
+			smp_text_poke_sync_each_cpu();
+			return 0;
+		}
+	}
 
-	text_poke_5byte(auprobe, vma, vaddr, auprobe->ptwrite.orig,
-			UPROBE_SWBP_INSN, false, false, false, false, NULL);
+	return text_poke_5byte(auprobe, vma, vaddr, ptw_a->orig, 0xe9,
+			false, false, false, false, NULL);
 }
 
 /*
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index b39abd1b057d..917989b018f3 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -250,7 +250,9 @@ extern struct uprobe *uprobe_register_ptwrite(struct inode *inode,
 					      const struct uprobe_ptwrite_desc *desc);
 extern bool arch_uprobe_ptwrite_supported(void);
 extern int arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
-				       const struct uprobe_ptwrite_desc *desc);
+					       struct inode *inode, struct file *file,
+					       loff_t offset,
+					       const struct uprobe_ptwrite_desc *desc);
 extern int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 				       struct vm_area_struct *vma,
 				       unsigned long vaddr);
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 20fa16ed8519..c273ec2709e1 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1515,6 +1515,8 @@ bool __weak arch_uprobe_ptwrite_supported(void)
 }
 
 int __weak arch_uprobe_ptwrite_prepare(struct arch_uprobe *auprobe,
+				       struct inode *inode, struct file *file,
+				       loff_t offset,
 				       const struct uprobe_ptwrite_desc *desc)
 {
 	return -EOPNOTSUPP;
@@ -1595,13 +1597,19 @@ struct uprobe *uprobe_register_ptwrite(struct inode *inode, struct file *file,
 		ret = -EBUSY;
 		goto out;
 	}
-
-	/* Build the mm-independent stub template once, at registration. */
-	ret = arch_uprobe_ptwrite_prepare(&uprobe->arch, desc);
+	/*
+	 * Prepare the immutable PTWRITE stub before exposing the uprobe. The
+	 * copy-instruction flag also keeps the normal XOL preparation path out.
+	 */
+	ret = copy_insn(uprobe, file);
 	if (ret)
 		goto out;
-
-
+	ret = arch_uprobe_ptwrite_prepare(&uprobe->arch, inode, file, offset,
+					  desc);
+	if (ret)
+		goto out;
+	smp_wmb();
+	set_bit(UPROBE_COPY_INSN, &uprobe->flags);
 	set_bit(UPROBE_PTWRITE, &uprobe->flags);
 	consumer_add(uprobe, uc);
 	ret = register_for_each_vma(uprobe, uc);
-- 
2.54.0

[RFC v1 14/19] ptwrite uprobes: Use atomic patching for multinop sites

[Andi Kleen]

The earlier multinop patching is not quite safe because the cross
modified CPU could be already executing on a later nop when the
cross patching occurs. The Intel SDM allows cross modification
by larger stores as long as they are aligned. AMD has a similar
guarantee.

The motivation for multinop is mainly to support the gcc
function entry patch sites and these are always aligned.

So enforce 8 bytes alignment of the multinop and use a safe RMW 8 byte store
ot overwrite the 5 byte sequence. This assumes that the code is not
changing in parallel, but if that happens cross modification safety
is probably the smallest of the issues.

Assisted-by: omp:gpt-5.6-luna sashiko
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 arch/x86/kernel/uprobes.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 806e40f7b0ab..8d9dadc2b1fc 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -1812,8 +1812,9 @@ get_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr,
 }
 
 /*
- * A run of short NOPs is accepted only when requested. This validation does
- * not make the three-phase poke safe for threads that already passed byte 0.
+ * A run of short NOPs is accepted only when requested. It is patched with
+ * an aligned eight-byte read-modify-write, preserving the following bytes;
+ * code is not expected to change concurrently.
  */
 static bool pun_site_is_nop(const u8 *orig, bool allow_nop_run)
 {
@@ -1831,6 +1832,13 @@ static bool pun_site_is_nop(const u8 *orig, bool allow_nop_run)
 		orig[2] == 0x90 && orig[3] == 0x90 && orig[4] == 0x90;
 }
 
+/* Identify the explicitly opted-in run of five one-byte NOPs. */
+static bool ptwrite_site_is_multinop(const u8 *orig, bool allow_nop_run)
+{
+	return allow_nop_run && orig[0] == 0x90 && orig[1] == 0x90 &&
+		orig[2] == 0x90 && orig[3] == 0x90 && orig[4] == 0x90;
+}
+
 /*
  * Classify the site's single instruction for out-of-line execution.
  * Returns the length, or 0 when it cannot run safely out of line.
@@ -2224,6 +2232,9 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	ret = copy_from_vaddr(mm, vaddr, orig, sizeof(orig));
 	if (ret)
 		return ret;
+	if (ptwrite_site_is_multinop(orig, ptw_a->allow_nop_run) &&
+	    (vaddr & 7))
+		return pun_install(auprobe, vma, vaddr, orig);
 	if (ptwrite_is_installed(mm, vaddr, orig))
 		return 0;
 
@@ -2249,9 +2260,13 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 			continue;
 		if (!__in_uprobe_ptwrite(mm, ptw->vaddr))
 			continue;
-		ret = ptwrite_text_poke(auprobe, vma, vaddr,
-					ptw->vaddr + ptw->index[b].off);
-		goto out;
+		if (ptwrite_site_is_multinop(orig, ptw_a->allow_nop_run))
+			ret = ptwrite_multinop_text_poke(auprobe, vma, vaddr,
+							ptw->vaddr + ptw->index[b].off);
+		else
+			ret = ptwrite_text_poke(auprobe, vma, vaddr,
+						ptw->vaddr + ptw->index[b].off);
+		return ret;
 	}
 	ptw = get_uprobe_ptwrite_page(mm, vaddr, ptw_a->stub_len);
 	if (!ptw)
@@ -2287,8 +2302,11 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
 	memcpy(kaddr + block_off + ptw_a->jmp_off, &rel, sizeof(rel));
 	kunmap_local(kaddr);
 
-	ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
-	if (ret)
+	if (ptwrite_site_is_multinop(orig, ptw_a->allow_nop_run))
+		ret = ptwrite_multinop_text_poke(auprobe, vma, vaddr, stub_addr);
+	else
+		ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
+	if (ret) {
 		/* Publish rollback before readers use the reduced block count. */
 		smp_store_release(&ptw->nblocks, ptw->nblocks - 1);
 		return ret;
-- 
2.54.0

[RFC v1 15/19] ptwrite uprobes: Add a tutorial and overview documentation

[Andi Kleen]

No code changes.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 Documentation/trace/index.rst           |   1 +
 Documentation/trace/ptwrite-uprobes.rst | 390 ++++++++++++++++++++++++
 2 files changed, 391 insertions(+)
 create mode 100644 Documentation/trace/ptwrite-uprobes.rst

diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst
index f4058e8e92e3..4ae7b158804b 100644
--- a/Documentation/trace/index.rst
+++ b/Documentation/trace/index.rst
@@ -90,6 +90,7 @@ interactions.
 .. toctree::
    :maxdepth: 1
 
+   ptwrite-uprobes
    user_events
    uprobetracer
 
diff --git a/Documentation/trace/ptwrite-uprobes.rst b/Documentation/trace/ptwrite-uprobes.rst
new file mode 100644
index 000000000000..79ab35824b93
--- /dev/null
+++ b/Documentation/trace/ptwrite-uprobes.rst
@@ -0,0 +1,390 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+ptwrite uprobes
+===============
+
+.. contents:: :local:
+
+Introduction
+============
+
+A classic uprobe enters the kernel for each probe. That causes overhead
+when the probe is executed frequently.
+
+ptwrite uprobes instead rely on hardware tracing that doesn't enter
+the kernel. It uses the ``PTWRITE`` instruction available on modern
+Intel CPUs to log data to the Processor Trace buffer. Processor
+Trace is configured and recorded by Linux perf.
+
+There are some limitations of the scheme (see below)
+but it is a lot faster than classic uprobes.
+
+Performance
+===========
+
+Measured on the development kernel in a KVM guest (2 vCPUs) running
+on a AlderLake laptop with a probe on a hot function called in a
+tight loop.
+
+    +---------------------------------------+----------------------+--------------------+------------------------+
+    | mode (2-arg probe)                    | PT off (% of classic)| full (% of classic)| snapshot (% of classic)|
+    +=======================================+======================+====================+========================+
+    | classic uprobe (tracefs)              | 100%                 | 100%               | 100%                   |
+    | classic uprobe (perf probe, trace r.) | 104%                 |                    |                        |
+    | classic uprobe (perf probe, perf ring)|                      | 156%               |                        |
+    | ptwrite %nopace                       | 2%                   | 2%                 | 8%                     |
+    | ptwrite default                       | 7%                   | 7%                 | 11%                    |
+    | perf probe ``--ptwrite``              | 7%                   | 7%                 | 12%                    |
+    +---------------------------------------+----------------------+--------------------+------------------------+
+
+The percentages are normalized to the classic tracefs uprobe in each
+recording mode; lower values represent lower cost per hit.
+
+A ``%nopace`` probe costs roughly **2%** as much as a classic uprobe
+with PT off (about 98% less). The default pacing (on unless ``%nopace``
+is given) costs roughly 7% as much (about 93% less) in the same mode.
+The default pacing slows down the probes to avoid data loss when they are
+too tightly spaced.
+
+``snapshot`` refers to ``perf record`` snapshot mode (``-S``) which doesn't
+save the PT ring buffer constantly.
+
+
+Requirements
+============
+
+- An Intel CPU with Intel PT and PTWRITE. When running as a guest Intel PT
+  needs to be exposed to the guest.
+  PT/PTWRITE are available when ``/sys/devices/intel_pt/format/ptw`` exists.
+- A kernel with ``CONFIG_UPROBE_EVENTS`` enabled.
+
+Quick start (tracefs)
+=====================
+
+Pick a probe site, register a probe at its file offset, enable it, run the
+program under PT, decode.
+
+Example 1: probe an existing instruction (punning)
+----------------------------------------------------
+
+Build a small program and probe the entry of ``main``::
+
+    $ cat > t.c <<'EOF'
+    #include <stdio.h>
+
+    __attribute__((noinline, noipa)) static unsigned long
+    target(unsigned long a, unsigned long b)
+    {
+        return a * 31 + b;
+    }
+
+    int main(void)
+    {
+        unsigned long i, acc = 0;
+        for (i = 0; i < 100; i++)
+            acc += target(i, i + 1);
+        printf("acc=%lu\n", acc);
+        return 0;
+    }
+    EOF
+    $ gcc -O2 -no-pie -fno-inline -o t t.c
+
+``objdump -F`` prints the file offset of every instruction::
+
+    $ objdump -d -F t | sed -n "/<main> (File Offset/,+1p"
+    0000000000401040 <main> (File Offset: 0x1040):
+      401040:	55			push   %rbp
+
+``1040`` is the file offset of ``main``'s first instruction, exactly
+what the probe line needs. Register the probe there, enable it, run
+the program under PT and decode::
+
+    # echo "ptw:e t:0x1040 %di %si" > /sys/kernel/tracing/uprobe_events
+    # echo 1 > /sys/kernel/tracing/events/uprobes/e/enable
+    # perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o perf.data ./t
+    # perf script --itrace=qwe -s uprobe-ptwrite-decode.py -i perf.data
+    record 1: event=uprobes/e id=0x6c2 args=[1, 140728356930600]
+    summary: records=1 dropped=0 stray=0 unknown=0 errors=0
+
+Since it isn't a nop the instruction is "punned": byte 0 is changed
+to a jump to a trampoline that logs the data and returns to the
+previous execution.
+
+Punning is a probabilistic method that depends on the existing
+instruction bytes and the placement of the executable in memory.
+It has a high chance of success on PIE/PIC binaries, but tends
+to work poorly on non PIE main executables.
+
+When punning is not possible the probe is rejected at install
+time. Options in this case:
+- Move the probe site to a different instruction which may work.
+- Rebuild with -fPIE if it's a main problem not using PIE.
+- Enable or disable /proc/sys/kernel/randomize_va_space. If the
+  randomization is enabled it may also just work on a rerun of
+  the program.
+- Fall back to a classic uprobes
+- Insert a 5 byte nop which is always supported (see below)
+
+Example 2: an explicit 5-byte NOP (inline assembly)
+---------------------------------------------------
+
+Add a 5-byte NOP at the probe point::
+
+    $ cat > t.c <<'EOF'
+    #include <stdio.h>
+
+    __attribute__((noinline)) static unsigned long
+    target(unsigned long a, unsigned long b)
+    {
+        asm volatile(".byte 0x0f, 0x1f, 0x44, 0x00, 0x00"); /* nopl */
+        return a * 31 + b;
+    }
+
+    int main(void)
+    {
+        unsigned long i, acc = 0;
+        for (i = 0; i < 100; i++)
+            acc += target(i, i + 1);
+        printf("acc=%lu\n", acc);
+        return 0;
+    }
+    EOF
+    $ gcc -O2 -no-pie -o t t.c
+
+    $ objdump -d -F t | sed -n "/<target> (File Offset/,+1p"
+    0000000000401170 <target> (File Offset: 0x1170):
+      401170:	0f 1f 44 00 00		nopl   0x0(%rax,%rax,1)
+
+Probe it exactly like example 1::
+
+    # echo "ptw:e t:0x1170 %di %si" > /sys/kernel/tracing/uprobe_events
+    # echo 1 > /sys/kernel/tracing/events/uprobes/e/enable
+    # perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o perf.data ./t
+    # perf script --itrace=qwe -s uprobe-ptwrite-decode.py -i perf.data
+    record 99: event=uprobes/e id=0x6c2 args=[98, 99]
+    record 100: event=uprobes/e id=0x6c2 args=[99, 100]
+    summary: records=100 dropped=0 stray=0 unknown=0 errors=0
+
+Configuring ptwrite uprobes
+===========================
+
+ptwrite uprobes is configured like normal uprobes by writing
+commands to ``/sys/kernel/tracing/uprobe_events``.
+
+  ptw[:[GRP/][EVENT]] PATH:OFFSET [FETCHARGS] : set a ptwrite probe
+  -:[GRP/][EVENT]                             : clear a probe
+
+  GRP      : group name. If omitted, "uprobes" is the default (the
+             event appears under events/uprobes/).
+  EVENT    : event name. If omitted, one is generated from PATH+OFFSET.
+  PATH     : path to an executable or a library.
+  OFFSET   : file offset of the probe site (0x-prefixed hex, see above).
+  FETCHARGS: probe arguments, up to 8 (see "Argument syntax" below).
+
+After creating the ptwrite uprobe it becomes available with its name
+in ``/sys/kernel/tracing/uprobe_events``. There it can be enabled
+by writing 1 to its enable field. However it only logs data
+when a Linux perf PT recording session with ptw=1 is active.
+
+perf probe
+----------
+
+``perf probe --ptwrite -x <file>`` creates ptwrite uprobes instead of
+the classic trap-based ones. The example below uses SDT probes.
+
+(this requires installing systemtap-devel or an equivalent package)
+
+    $ cat > t.c <<'EOF'
+    #include <stdio.h>
+    #include <sys/sdt.h>
+    __attribute__((noinline, noclone)) static unsigned long
+    target(unsigned long a, unsigned long b)
+    {
+        unsigned long local = a * 2;
+        STAP_PROBE1(test, rarg, a);
+        STAP_PROBE1(test, carg, 42);
+        STAP_PROBE2(test, marg, &local, b);
+        return a * 31 + b;
+    }
+    int main(void)
+    {
+        unsigned long i, acc = 0;
+        for (i = 0; i < 20; i++) {
+            acc += target(i, i + 1);
+            asm volatile("pause");
+        }
+        printf("acc=%lu\n", acc);
+        return 0;
+    }
+    EOF
+    $ gcc -O2 -no-pie -o t t.c
+
+The first probe point (``rarg``) is a nop 9 bytes into ``target``::
+
+    $ objdump -d t | sed -n "/<target>:/,+3p"
+    0000000000401180 <target>:
+      401180:	48 8d 04 3f		lea    (%rdi,%rdi,1),%rax
+      401184:	48 89 44 24 f8		mov    %rax,-0x8(%rsp)
+      401189:	90			nop
+
+Probe it with ``perf probe --ptwrite`` using the function+offset
+form, then enable, capture and delete it like any ptwrite probe::
+
+    $ perf probe --ptwrite -x ./t --add "target+9 %di %si"
+    Added new event:
+      probe_t:target      (on target+9 in ./t with %di %si)
+
+    # the tracefs line it wrote:
+    # ptw:probe_t/target ./t:0x1189 arg1=%di arg2=%si
+
+    # echo 1 > /sys/kernel/tracing/events/probe_t/target/enable
+    # perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o perf.data ./t
+    # perf script --itrace=qwe -s uprobe-ptwrite-decode.py -i perf.data
+    record 1: event=probe_t/target id=0x6a9 args=[0, 1]
+    record 20: event=probe_t/target id=0x6a9 args=[19, 20]
+    summary: records=20 dropped=0 stray=0 unknown=0 errors=0
+    # perf probe -d probe_t:target
+
+A ``nop`` instruction, as used by SDT probes, is not guaranteed to
+be ptwrite patchable. It needs a 5-byte NOP, but it can
+often be punned. If punning fails, the kernel reports
+``failed to install`` and the probe has to be moved to another site.
+
+GCC's ``-fpatchable-function-entry=5`` may emit five one-byte NOPs.
+To use that site, add ``%multinop`` to the tracefs probe offset::
+
+    # echo "ptw:e t:0x1170%multinop %di %si" > /sys/kernel/tracing/uprobe_events
+
+The five-byte run must start at an 8-byte-aligned address because it is
+patched with an atomic eight-byte store. An unaligned ``%multinop`` site is
+rejected; without ``%multinop``, the run is treated as a pun and may not
+always succeed.
+
+perf probe uses the standard argument syntax for the ptwrite subset
+(registers, ``$stack``/``$stackN``, ``+disp(%reg)`` memory reads, and
+``\0x2a``-style constants). Strings, arrays and typed suffixes are not
+supported by the ptwrite stub and are rejected by the kernel.
+``%return`` is refused (ptwrite probes are entry-only), and the mode
+requires ``-x``. The probes are enabled, captured and deleted like
+classic probes (``perf probe -l``, ``perf probe -d``).
+They carry the default (LFENCE) pacing. ``%nopace`` cannot be selected
+through perf probe. Write the tracefs line by hand for that.
+
+Argument syntax
+---------------
+
+ptwrite uprobes only support a limited number of argument types
+compared to classic uprobes.
+
+``ptw:<name> <path>:<offset> <arg> ... [options]`` where each ``<arg>`` is one
+of:
+
+- ``%di``, ``%si``, ``%ax`` ...: a live register.
+- ``\IMM``: a fixed constant (stored in the stub), e.g. ``\0x42``.
+- ``$stack``: the stack pointer value (never faults).
+- ``$stackN``: the Nth stack slot (``[%rsp + 8N]``). ``u64`` uses an
+  8-byte load on the fault-fixup path; ``u32``/``s32``/``x32`` use a 4-byte
+  load.
+- ``+<disp>(%reg)``: read memory at ``[reg + disp]``. ``u64`` uses an
+  8-byte load; ``u32``/``s32``/``x32`` use a 4-byte load (``ptwritel``).
+  A bad address writes ``0``.
+
+Options
+-------
+
+``%nopace`` disables artificial slowdown of the probes. This can cause
+data loss when they are tightly spaced or have many arguments, but
+speeds up the probes (see the benchmark section above)
+
+``%multinop`` lets users probe a 5-byte nop sequence that is not one
+instruction. A program could jump to a later nop, which would break when
+the probe rewrites the site.
+
+However there is a common case where gcc's -fpatchable-function-entry=5
+generates 5 nops for each function that are convenient points
+for patching, and nobody jumps into the middle of them.
+
+The 5-byte single nop sequence must be aligned to 8 bytes.
+
+
+The encoding format
+===================
+
+Each probe writes a header and the arguments to the PT stream.
+
+The header is a 64-bit word. Each argument is one PTWRITE payload exposed by
+perf as a ``u64`` value.
+
+    header word:      bits 63..48  event id (matches the tracefs id in sysfs)
+                      bits 47..40  number of argument words
+                      bits 39..0   fixed magic 0x5054525731 ("PTRW1")
+    arguments:        one PTWRITE payload per FETCHARG
+
+If the program itself also executes own ``PTWRITE``, those values mix with the
+uprobe output in the stream. The decoder uses the header magic to identify
+uprobe records. Other values are printed as ``manual ptwrite:`` lines (with
+their IP when ``fup_on_ptw`` is set) and counted in the summary's ``stray``
+field.
+
+To also print the decoded branch stream alongside the records, add
+``b`` to the itrace options and drop the ``q``
+
+    # ``perf script --itrace=web -s uprobe-ptwrite-decode.py -i perf.data``
+
+Each decoded branch prints as a ``branch:`` line (from => to, with
+symbols where resolvable), interleaved with the probe records and any
+manual ptwrites in delivery order.
+
+Other events in the recording, including classic uprobes, tracepoints,
+and sample events, are printed as ``event:`` lines unless disabled
+by the decoder.
+
+Unsupported instructions for probes
+===================================
+
+The following instructions are always refused for instrumentation::
+
+- Traps: ``int3``, ``int1``, ``int imm8``, ``into``, ``iret``
+  because they save the IP.
+- System instructions: ``syscall``, ``sysenter``, ``sysexit``
+  for similar reasons.
+- Far control flow: ``jmp far``, ``call far``, and the indirect far
+  forms (call-far, jmp-far).
+- Relative branches: ``jmp rel8/rel32``, ``jcc rel8/rel32``,
+  ``loop*``, ``jecxz/jrcxz`` (target-inside-window, see below), and
+  ``call rel32`` (its return address would point into the stub).
+- Indirect ``call``: the return-address problem applies
+  to the register/memory forms too.
+- Relative branches (``jmp``/``jcc``/``loop`` with a rel8/rel32
+  displacement, and ``call rel32``) are refused because the trampoline
+  may not be able to reach the target.
+- If the 5 byte area of the instruction crosses a page boundary it
+  currently cannot be probed (this applies to nop probes too).
+
+Other restrictions
+==================
+
+- Each probe needs 4K of process memory and roughly 1K extra in the kernel.
+- The probe pages are currently only freed on process exit.
+- Return probes (``%return``/``r:``): ptwrite probes are entry-only
+  and ``%return`` is refused.
+- The SDT reference counter (``(REF)``).
+- EBPF, perf actions, filters, event predicates, histograms, triggers,
+  profiling and similar advanced trace features are all not supported
+  since they would require a kernel entry. However some basic filtering
+  is possible at the perf recording level, for example limit the scope
+  to a CPU or to a process. PT also supports address filter ranges
+  that allow filtering by IP.
+- More than one probe at the same site
+- Only 4 and 8 byte memory references are supported.
+- Fetch argument variety: classic probes fetch strings
+  (``:string``/``:ustring``), arrays, bitfields, nested derefs,
+  ``$retval``, ``$comm`` and ``$argN``. Ptwrite probes only take live
+  registers, ``\IMM`` constants, ``$stack``/``$stackN`` and
+  ``+disp(%reg)`` memory reads. Memory reads are 8-byte words for ``u64``
+  and 4-byte words for ``u32``/``s32``/``x32``.
+  (some of this could be relaxed, but it would require a writable stack)
+- Like normal uprobes one byte of the instruction stream is overwritten
+  (or 5 bytes for the nop case). If the program reads its own code
+  it might see different values.
-- 
2.54.0

[RFC v1 16/19] ptwrite uprobes / perf tools pt: Improve FUP error handling for ptwrite

[Andi Kleen]

The PT decoder can't see the patched code generated by ptwrite uprobes.

Without ptw_on_fup the ptwrite stubs are invisible to PT branch tracing
(other than the ptwrite packet itself) because they don't contain any
indirect or conditional branches, so there is no problem with the PT
decoder.

However when ptw_on_fup is enabled there is a FUP (Flow Update Packet)
reporting the IP of each ptwrite after the PTW packets. The decoder tries
to resolve this FUP packet to the code, but it errors out because it can't
see the uprobes generated code.

Normally this is not a problem because we just use 'q' mode which doesn't
walk instructions, but still reports on the ptwrites and their FUPs.

Also it's possible to disable fup_on_ptw, however that reduces the
tolerance to data loss in the uprobes ptwrite decoder.

When full instruction tracing is desired the PTW+FUP errors cause data
loss.

Special case this in the decoder instead. When the FUP is associated with a
ptwrite don't error out on missing instructions pages. Just report the
ptwrite with its IP and continue.

An alternative would be to define a metadata event for the JITed code and
let the decoder understand it. That may be desirable in the future so that
the PT users sees all the code executed. But for now this simple change is
good enough.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 .../util/intel-pt-decoder/intel-pt-decoder.c  | 23 ++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c b/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
index e733f6b1f7ac..bd31d65dbe03 100644
--- a/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
+++ b/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
@@ -1440,8 +1440,29 @@ static int intel_pt_walk_fup(struct intel_pt_decoder *decoder)
 			return -EAGAIN;
 		}
 		decoder->set_fup_tx_flags = false;
-		if (err)
+		if (err) {
+			/*
+			 * A ptwrite's FUP can target an address whose
+			 * instruction cannot be resolved (e.g. the
+			 * [uprobes-ptwrite] stub is an anonymous special
+			 * mapping invisible to the machine). The FUP is
+			 * still the ptwrite's IP: report it rather than
+			 * failing the whole walk.
+			 */
+			if (decoder->set_fup_ptw) {
+				decoder->set_fup_ptw = false;
+				decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
+				decoder->state.type &= ~INTEL_PT_BRANCH;
+				decoder->state.type |= INTEL_PT_PTW;
+				decoder->state.flags |= INTEL_PT_FUP_IP;
+				decoder->state.from_ip = decoder->ip;
+				decoder->state.to_ip = 0;
+				decoder->state.ptw_payload =
+							decoder->fup_ptw_payload;
+				return 0;
+			}
 			return err;
+		}
 
 		if (intel_pt_insn.branch == INTEL_PT_BR_INDIRECT) {
 			intel_pt_log_at("ERROR: Unexpected indirect branch",
-- 
2.54.0

[RFC v1 17/19] ptwrite uprobes / perf tools probe: Add support of ptwrite probes

[Andi Kleen]

Add a --ptwrite option to perf probe to enable ptwrite probes. It is mainly
identical to the normal uprobes support, but knows about the new ptw:
probe syntax.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 tools/perf/Documentation/perf-probe.txt |  6 ++++++
 tools/perf/builtin-probe.c              | 17 ++++++++++++++++-
 tools/perf/util/probe-event.c           | 18 +++++++++++++-----
 tools/perf/util/probe-event.h           |  2 ++
 4 files changed, 37 insertions(+), 6 deletions(-)

diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt
index 2e5790325430..859f7ec42422 100644
--- a/tools/perf/Documentation/perf-probe.txt
+++ b/tools/perf/Documentation/perf-probe.txt
@@ -136,6 +136,12 @@ OPTIONS
 --max-probes=NUM::
 	Set the maximum number of probe points for an event. Default is 128.
 
+--ptwrite::
+	Create a ptwrite uprobe (with -x). The probe writes its values
+	into the Intel PT trace stream instead of entering the kernel,
+	which is faster. Requires an Intel PT/PTWRITE CPU and a kernel
+	with ptwrite uprobe support.
+
 --target-ns=PID:
 	Obtain mount namespace information from the target pid.  This is
 	used when creating a uprobe for a process that resides in a
diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index a67b565278ae..f6c90bf9cd0d 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -40,6 +40,7 @@ static struct {
 	int command;	/* Command short_name */
 	bool list_events;
 	bool uprobes;
+	bool ptwrite;
 	bool target_used;
 	int nevents;
 	struct perf_probe_event events[MAX_PROBES];
@@ -62,6 +63,7 @@ static int parse_probe_event(const char *str)
 	}
 
 	pev->uprobes = params->uprobes;
+	pev->ptwrite = params->ptwrite;
 	if (params->target) {
 		pev->target = strdup(params->target);
 		if (!pev->target)
@@ -73,9 +75,15 @@ static int parse_probe_event(const char *str)
 
 	/* Parse a perf-probe command into event */
 	ret = parse_perf_probe_command(str, pev);
+	if (ret < 0)
+		return ret;
+	if (pev->ptwrite && pev->point.retprobe) {
+		pr_err("Error: ptwrite probes are entry-only (no %%return).\n");
+		return -EINVAL;
+	}
 	pr_debug("%d arguments\n", pev->nargs);
 
-	return ret;
+	return 0;
 }
 
 static int params_add_filter(const char *str)
@@ -532,6 +540,8 @@ __cmd_probe(int argc, const char **argv)
 		    "be more verbose (show parsed arguments, etc)"),
 	OPT_BOOLEAN('q', "quiet", &quiet,
 		    "be quiet (do not show any warnings or messages)"),
+	OPT_BOOLEAN(0, "ptwrite", &params->ptwrite,
+		    "create trap-free ptwrite uprobes (requires -x)"),
 	OPT_CALLBACK_DEFAULT('l', "list", NULL, "[GROUP:]EVENT",
 			     "list up probe events",
 			     opt_set_filter_with_command, DEFAULT_LIST_FILTER),
@@ -733,6 +743,11 @@ __cmd_probe(int argc, const char **argv)
 			parse_options_usage(NULL, options, "x", true);
 			return -EINVAL;
 		}
+		if (params->ptwrite && !params->uprobes) {
+			pr_err("  Error: --ptwrite requires -x.\n");
+			parse_options_usage(NULL, options, "x", true);
+			return -EINVAL;
+		}
 
 		ret = perf_add_probe_events(params->events, params->nevents);
 		if (ret < 0) {
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 11ae4a09412c..0417fa5ccfbe 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -1930,6 +1930,7 @@ int parse_probe_trace_command(const char *cmd, struct probe_trace_event *tev)
 		ret = -EINVAL;
 		goto out;
 	}
+	tev->ptwrite = !strcmp(fmt1_str, "ptw");
 	pr = fmt1_str[0];
 	tev->group = strdup(fmt2_str);
 	tev->event = strdup(fmt3_str);
@@ -1937,7 +1938,8 @@ int parse_probe_trace_command(const char *cmd, struct probe_trace_event *tev)
 		ret = -ENOMEM;
 		goto out;
 	}
-	pr_debug("Group:%s Event:%s probe:%c\n", tev->group, tev->event, pr);
+	pr_debug("Group:%s Event:%s probe:%c%s\n", tev->group, tev->event, pr,
+		 tev->ptwrite ? " (ptwrite)" : "");
 
 	tp->retprobe = (pr == 'r');
 
@@ -2260,9 +2262,11 @@ char *synthesize_probe_trace_command(struct probe_trace_event *tev)
 	if (strbuf_init(&buf, 32) < 0)
 		return NULL;
 
-	if (strbuf_addf(&buf, "%c:%s/%s ", tp->retprobe ? 'r' : 'p',
-			tev->group, tev->event) < 0)
-		goto error;
+	if (tev->ptwrite)
+		err = strbuf_addf(&buf, "ptw:%s/%s ", tev->group, tev->event);
+	else
+		err = strbuf_addf(&buf, "%c:%s/%s ", tp->retprobe ? 'r' : 'p',
+				  tev->group, tev->event);
 
 	if (tev->uprobes)
 		err = synthesize_uprobe_trace_def(tp, &buf);
@@ -2274,7 +2278,6 @@ char *synthesize_probe_trace_command(struct probe_trace_event *tev)
 
 	if (err >= 0)
 		ret = strbuf_detach(&buf, NULL);
-error:
 	strbuf_release(&buf);
 	return ret;
 }
@@ -2996,6 +2999,7 @@ static int __add_probe_trace_events(struct perf_probe_event *pev,
 	ret = 0;
 	for (i = 0; i < ntevs; i++) {
 		tev = &tevs[i];
+		tev->ptwrite = pev->ptwrite;
 		up = tev->uprobes ? 1 : 0;
 		if (fd[up] == -1) {	/* Open the kprobe/uprobe_events */
 			fd[up] = __open_probe_file_and_namelist(up,
@@ -3610,6 +3614,8 @@ int convert_perf_probe_events(struct perf_probe_event *pevs, int npevs)
 
 	/* Loop 1: convert all events */
 	for (i = 0; i < npevs; i++) {
+		int j;
+
 		/* Init kprobe blacklist if needed */
 		if (!pevs[i].uprobes)
 			kprobe_blacklist__init();
@@ -3618,6 +3624,8 @@ int convert_perf_probe_events(struct perf_probe_event *pevs, int npevs)
 		if (ret < 0)
 			return ret;
 		pevs[i].ntevs = ret;
+		for (j = 0; j < pevs[i].ntevs; j++)
+			pevs[i].tevs[j].ptwrite = pevs[i].ptwrite;
 	}
 	/* This just release blacklist only if allocated */
 	kprobe_blacklist__release();
diff --git a/tools/perf/util/probe-event.h b/tools/perf/util/probe-event.h
index 71905ede0207..5fbed6c6a78a 100644
--- a/tools/perf/util/probe-event.h
+++ b/tools/perf/util/probe-event.h
@@ -60,6 +60,7 @@ struct probe_trace_event {
 	int				nargs;	/* Number of args */
 	int				lang;	/* Dwarf language code */
 	bool				uprobes;	/* uprobes only */
+	bool				ptwrite;	/* ptwrite uprobe (trap-free) */
 	struct probe_trace_arg		*args;	/* Arguments */
 };
 
@@ -99,6 +100,7 @@ struct perf_probe_event {
 	int			nargs;	/* Number of arguments */
 	bool			sdt;	/* SDT/cached event flag */
 	bool			uprobes;	/* Uprobe event flag */
+	bool			ptwrite;	/* ptwrite uprobe (trap-free) */
 	char			*target;	/* Target binary */
 	struct perf_probe_arg	*args;	/* Arguments */
 	struct probe_trace_event *tevs;
-- 
2.54.0

[RFC v1 18/19] ptwrite uprobes / perf tools script: Add ptwrite uprobes decoder

[Andi Kleen]

Add a decoder for the PTWRITE records generated by ptwrite uprobes.
This runs as a python script in perf script.

The main tricky part is how to detect and recover data loss or interleaving
with other ptwrites. The decoder uses a magic value in the header
to synchronize, and also the IP from the FUP when available (fup_on_ptw=1)

It can display other events (including branches) when they are available.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 .../scripts/python/uprobe-ptwrite-decode.py   | 416 ++++++++++++++++++
 1 file changed, 416 insertions(+)
 create mode 100755 tools/perf/scripts/python/uprobe-ptwrite-decode.py

diff --git a/tools/perf/scripts/python/uprobe-ptwrite-decode.py b/tools/perf/scripts/python/uprobe-ptwrite-decode.py
new file mode 100755
index 000000000000..22f21cb168e1
--- /dev/null
+++ b/tools/perf/scripts/python/uprobe-ptwrite-decode.py
@@ -0,0 +1,416 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""
+uprobe-ptwrite-decode.py - decode ptwrite-uprobe trace events out of
+PT logs.
+
+Usage (perf script):
+  perf script --itrace=qwe -s uprobe-ptwrite-decode.py -i perf.data
+"""
+import os
+import re
+import struct
+import sys
+
+TRACEFS = "/sys/kernel/tracing"
+
+# type name -> (size bytes, signed, hex, char)
+_TYPES = {
+    "u8":  (1, False, False, False),
+    "u16": (2, False, False, False),
+    "u32": (4, False, False, False),
+    "u64": (8, False, False, False),
+    "s8":  (1, True, False, False),
+    "s16": (2, True, False, False),
+    "s32": (4, True, False, False),
+    "s64": (8, True, False, False),
+    "x8":  (1, False, True, False),
+    "x16": (2, False, True, False),
+    "x32": (4, False, True, False),
+    "x64": (8, False, True, False),
+    "char": (1, False, False, True),
+}
+
+_FIELD_RE = re.compile(r'^\s*field:(\S+)\s+(arg\d+);')
+
+HDR_MAGIC = 0x5054525731   # "PTRW1"
+HDR_MASK = (1 << 40) - 1
+
+def load_events(root=TRACEFS):
+    """Scan tracefs for (event_id -> (name, [(arg name, type)]))."""
+    events = {}
+    try:
+        groups = os.listdir(root + "/events")
+    except OSError:
+        return events
+    for g in groups:
+        gdir = root + "/events/" + g
+        if not os.path.isdir(gdir):
+            continue
+        for ev in os.listdir(gdir):
+            edir = gdir + "/" + ev
+            if not os.path.isdir(edir):
+                continue
+            try:
+                with open(edir + "/id") as f:
+                    eid = int(f.read().strip())
+                with open(edir + "/format") as f:
+                    fields = []
+                    for line in f:
+                        m = _FIELD_RE.match(line)
+                        if m:
+                            fields.append((m.group(2), m.group(1)))
+                events[eid] = (g + "/" + ev, fields)
+            except (OSError, ValueError):
+                continue
+    return events
+
+def type_info(t):
+    """('u64', size, signed, hex, char) for a format-file type name."""
+    if t in _TYPES:
+        return (t,) + _TYPES[t]
+    return (t, 8, False, False, False)
+
+def fmt_value(v, t):
+    name, size, signed, ishex, ischar = type_info(t)
+    mask = (1 << (8 * size)) - 1
+    v &= mask
+    if ischar:
+        return repr(chr(v))
+    if signed:
+        sign = 1 << (8 * size - 1)
+        if v & sign:
+            v -= 1 << (8 * size)
+    if ishex:
+        return "0x%x" % v
+    return str(v)
+
+def is_header(word, events):
+    if (word & HDR_MASK) != HDR_MAGIC:
+        return None
+    eid = (word >> 48) & 0xffff
+    nargs = (word >> 40) & 0xff
+    if not (1 <= nargs <= 8):
+        return None
+    if events and eid not in events:
+        return None
+    return (eid, nargs)
+
+def decode_words(words, events):
+    """Walk the ptwrite stream."""
+    records = dropped = stray = unknown = 0
+    cur = None      # (event_id, nargs, [args])
+    rec_drop = False
+    learned = {}
+    arg_off = {}
+    lines = []
+    for w in words:
+        if isinstance(w, tuple):
+            word, ip, key = (w + (0, 0))[:3] if len(w) < 3 else (w[0], w[1], w[2])
+        else:
+            word, ip, key = w, 0, 0
+        off = (ip & 0xfff) if ip else 0
+        hdr = is_header(word, events)
+        if hdr is not None:
+            if cur is not None:
+                dropped += cur[1] - len(cur[2])
+            cur = (hdr[0], hdr[1], [])
+            rec_drop = False
+            continue
+        if cur is None:
+            # no record open: this is a raw ptwrite from the program
+            lines.append((key, "manual ptwrite: payload=0x%x ip=0x%x"
+                          % (word, ip)))
+            stray += 1
+            continue
+        eid, nargs, args = cur
+        if len(args) >= nargs:
+            cur = None
+            lines.append((key, "manual ptwrite: payload=0x%x ip=0x%x"
+                          % (word, ip)))
+            stray += 1
+            continue
+        if ip and learned.get(eid):
+            offs = arg_off[eid]
+            expect = offs[len(args)]
+            if off != expect:
+                j = next((k for k in range(len(args) + 1, nargs)
+                          if off == offs[k]), None)
+                if j is not None:
+                    dropped += j - len(args)
+                    args.extend([None] * (j - len(args)))
+                    rec_drop = True
+                else:
+                    unknown += 1
+                    continue
+        if ip and not learned.get(eid):
+            offsets = arg_off.setdefault(eid, set())
+            if len(offsets) < nargs:
+                offsets.add(off)
+        args.append(word)
+        if len(args) == nargs:
+            if ip and not learned.get(eid):
+                offs = sorted(set(arg_off.get(eid, [])))
+                if len(offs) == nargs:
+                    arg_off[eid] = offs
+                    learned[eid] = True
+            if not rec_drop:
+                ev = (events or {}).get(eid)
+                name = ev[0] if ev else "?"
+                fields = ev[1] if ev else []
+                parts = []
+                for i, value in enumerate(args[:nargs]):
+                    field_name = None
+                    field_type = "u64"
+                    if i < len(fields):
+                        field_name, field_type = fields[i]
+                    text = fmt_value(value, field_type)
+                    parts.append("%s=%s" % (field_name, text)
+                                 if field_name else text)
+                fmt = ", ".join(parts)
+                lines.append((key, "record %d: event=%s id=0x%x args=[%s]"
+                             % (records + 1, name, eid, fmt)))
+            records += 1
+            cur = None
+    if cur is not None:
+        dropped += cur[1] - len(cur[2])
+
+    return records, dropped, stray, unknown, lines
+
+def _emit_record(eid, nargs, args, events, record_drop):
+    global _records
+    if record_drop:
+        return
+    ev = (events or {}).get(eid)
+    name = ev[0] if ev else "?"
+    fields = ev[1] if ev else []
+    parts = []
+    for i, value in enumerate(args[:nargs]):
+        field_name = None
+        field_type = "u64"
+        if i < len(fields):
+            field_name, field_type = fields[i]
+        text = fmt_value(value, field_type)
+        parts.append("%s=%s" % (field_name, text)
+                     if field_name else text)
+    fmt = ", ".join(parts)
+    print("record %d: event=%s id=0x%x args=[%s]" %
+          (_records + 1, name, eid, fmt))
+
+def _decode_stream_word(word, ip, events, stream_key):
+    global _records, _dropped, _stray, _unknown
+    state = _streams.setdefault(stream_key, {"cur": None, "drop": False})
+    off = (ip & 0xfff) if ip else 0
+    hdr = is_header(word, events)
+    if hdr is not None:
+        if state["cur"] is not None:
+            _dropped += state["cur"][1] - len(state["cur"][2])
+        state["cur"] = (hdr[0], hdr[1], [])
+        state["drop"] = False
+        return
+    cur = state["cur"]
+    if cur is None:
+        print("manual ptwrite: payload=0x%x ip=0x%x" % (word, ip))
+        _stray += 1
+        return
+    eid, nargs, args = cur
+    if len(args) >= nargs:
+        state["cur"] = None
+        print("manual ptwrite: payload=0x%x ip=0x%x" % (word, ip))
+        _stray += 1
+        return
+    if ip and _learned.get(eid):
+        offs = _arg_off[eid]
+        expect = offs[len(args)]
+        if off != expect:
+            j = next((k for k in range(len(args) + 1, nargs)
+                      if off == offs[k]), None)
+            if j is not None:
+                _dropped += j - len(args)
+                args.extend([None] * (j - len(args)))
+                state["drop"] = True
+            else:
+                _unknown += 1
+                return
+    if ip and not _learned.get(eid):
+        offsets = _arg_off.setdefault(eid, set())
+        if len(offsets) < nargs:
+            offsets.add(off)
+    args.append(word)
+    if len(args) == nargs:
+        if ip and not _learned.get(eid):
+            offs = sorted(set(_arg_off.get(eid, set())))
+            if len(offs) == nargs:
+                _arg_off[eid] = offs
+                _learned[eid] = True
+        _emit_record(eid, nargs, args, events, state["drop"])
+        _records += 1
+        state["cur"] = None
+
+def decode_buf(raw_buf, events, ip=0, key=0, stream_key=None):
+    """Decode one 12-byte perf raw sample without retaining prior samples."""
+    if len(raw_buf) < 12:
+        return
+    flags = struct.unpack_from("<I", raw_buf, 0)[0]
+    payload = struct.unpack_from("<Q", raw_buf, 4)[0]
+    if not flags & 1:
+        ip = 0          # no FUP: the IP is not recoverable
+    if stream_key is None:
+        stream_key = 0
+    _decode_stream_word(payload, ip, events, stream_key)
+
+_events = {}
+_streams = {}
+_learned = {}
+_arg_off = {}
+_records = _dropped = _stray = _unknown = 0
+_other_count = 0
+_show_branches = "--no-branches" not in sys.argv
+
+def _sample_stream_key(sample):
+    if not sample:
+        return (None, None, None)
+    return (sample.get("cpu"), sample.get("pid"), sample.get("tid"))
+
+def auxtrace_error(*args):
+    global _errors
+    if len(args) >= 8:
+        _errors += 1
+        print("ptwrite-decode: error type=%d code=%d cpu=%d pid=%d tid=%d"
+              " ip=0x%x msg=%s" % (args[0], args[1], args[2], args[3],
+                                   args[4], args[5], args[7]))
+
+def trace_begin():
+    global _events, _records, _dropped, _stray, _unknown, _errors, _other_count
+    if not _events:
+        _events = load_events()
+    _streams.clear()
+    _learned.clear()
+    _arg_off.clear()
+    _records = _dropped = _stray = _unknown = 0
+    _errors = _other_count = 0
+
+def branch_line(param_dict):
+    sample = param_dict.get("sample") or {}
+    frm = sample.get("ip") or 0
+    to = sample.get("addr") or 0
+    frm_sym = param_dict.get("symbol") or "[unknown]"
+    to_sym = sample.get("addr_symbol") or "[unknown]"
+    frm_off = param_dict.get("symoff") or 0
+    to_off = sample.get("addr_symoff") or 0
+    frm_dso = param_dict.get("dso") or "[unknown]"
+    to_dso = sample.get("addr_dso") or "[unknown]"
+    fs = "+0x%x" % frm_off if frm_off else ""
+    ts_ = "+0x%x" % to_off if to_off else ""
+    return ("branch: 0x%x %s%s (%s) => 0x%x %s%s (%s)"
+            % (frm, frm_sym, fs, frm_dso, to, to_sym, ts_, to_dso))
+
+def other_line(name, sample, comm, sym=None, off=0, dso=None):
+    """Format a non-ptwrite, non-branch event (classic uprobe,
+    tracepoint, sample) for interleaved printing."""
+    pid = sample.get("pid")
+    tid = sample.get("tid")
+    ip = sample.get("ip") or 0
+    loc = " %s+0x%x (%s)" % (sym, off, dso) if sym else ""
+    return ("event: %s comm=%s pid=%s tid=%s ip=0x%x%s"
+            % (name, comm, pid, tid, ip, loc))
+
+def trace_unhandled(handler_name, context, fields, sample=None):
+    """Print tracepoint-class events immediately in delivery order."""
+    global _other_count
+    fields = fields or {}
+    name = handler_name.replace("__", ":")
+    comm = fields.get("common_comm") or ""
+    pid = fields.get("common_pid")
+    cpu = fields.get("common_cpu")
+    print("event: %s comm=%s pid=%s cpu=%s" %
+          (name, comm, pid, cpu))
+    _other_count += 1
+
+def process_event(param_dict):
+    global _other_count
+    sample = param_dict.get("sample") or {}
+    name = param_dict.get("ev_name") or ""
+    if name == "ptwrite":
+        raw = param_dict.get("raw_buf")
+        if raw:
+            decode_buf(raw, _events, sample.get("ip") or 0,
+                       stream_key=_sample_stream_key(sample))
+    elif name.startswith("branches"):
+        if _show_branches:
+            print(branch_line(param_dict))
+    else:
+        print(other_line(name, sample,
+                         param_dict.get("comm") or "",
+                         param_dict.get("symbol"),
+                         param_dict.get("symoff") or 0,
+                         param_dict.get("dso")))
+        _other_count += 1
+
+def trace_end():
+    global _dropped
+    for state in _streams.values():
+        cur = state["cur"]
+        if cur is not None:
+            _dropped += cur[1] - len(cur[2])
+    print("summary: records=%d dropped=%d stray=%d unknown=%d errors=%d other=%d"
+          % (_records, _dropped, _stray, _unknown, _errors, _other_count))
+    _streams.clear()
+
+def main():
+    args = sys.argv[1:]
+    wordfile = None
+    eid_override = None
+    types_override = None
+    i = 0
+    while i < len(args):
+        a = args[i]
+        if a == "--words":
+            if i + 1 < len(args) and not args[i + 1].startswith("--"):
+                wordfile = args[i + 1]
+                i += 1
+        elif a == "--event-id":
+            if i + 1 >= len(args) or args[i + 1].startswith("--"):
+                print("--event-id requires a value", file=sys.stderr)
+                return 2
+            try:
+                eid_override = int(args[i + 1], 0)
+            except ValueError:
+                print("--event-id requires an integer", file=sys.stderr)
+                return 2
+            i += 1
+        elif a == "--types":
+            if i + 1 >= len(args) or args[i + 1].startswith("--"):
+                print("--types requires a value", file=sys.stderr)
+                return 2
+            types_override = args[i + 1].split(",")
+            i += 1
+        i += 1
+
+    events = dict(_events)
+    if eid_override is not None and types_override is not None:
+        events[eid_override] = ("override",
+                                [(None, t) for t in types_override])
+
+    words = []
+    if wordfile:
+        with open(wordfile) as f:
+            for line in f:
+                line = line.strip()
+                if line.startswith(("0x", "0X")) or line.isdigit():
+                    words.append(int(line, 16))
+    else:
+        for line in sys.stdin:
+            line = line.strip()
+            if line.startswith(("0x", "0X")) or line.isdigit():
+                words.append(int(line, 16))
+
+    records, dropped, stray, unknown, lines = decode_words(words, events)
+    for _, text in lines:
+        print(text)
+    print("summary: records=%d dropped=%d stray=%d unknown=%d"
+          % (records, dropped, stray, unknown))
+    return 0 if (records > 0 and dropped == 0 and unknown == 0) else 1
+
+if "--words" in sys.argv:
+    sys.exit(main())
-- 
2.54.0

[RFC v1 19/19] ptwrite uprobes: Add self tests

[Andi Kleen]

Regression test various cases with ptwrite uprobes. It uses both the
main probes and the perf decoder.

Assisted-by: omp:gpt-5.6-luna
Signed-off-by: Andi Kleen <ak@kernel.org>
---
 tools/testing/selftests/Makefile              |   1 +
 .../test.d/kprobe/uprobe_syntax_errors.tc     |  44 ++++
 tools/testing/selftests/uprobes/Makefile      |  15 ++
 tools/testing/selftests/uprobes/ptw_probe.c   | 156 ++++++++++++
 tools/testing/selftests/uprobes/run_decode.sh | 177 ++++++++++++++
 tools/testing/selftests/uprobes/run_ptw.sh    | 226 ++++++++++++++++++
 6 files changed, 619 insertions(+)
 create mode 100644 tools/testing/selftests/uprobes/Makefile
 create mode 100644 tools/testing/selftests/uprobes/ptw_probe.c
 create mode 100755 tools/testing/selftests/uprobes/run_decode.sh
 create mode 100755 tools/testing/selftests/uprobes/run_ptw.sh

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 2d960626750e..02f36cbf98ae 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -135,6 +135,7 @@ TARGETS += tpm2
 TARGETS += tty
 TARGETS += ublk
 TARGETS += uevent
+TARGETS += uprobes
 TARGETS += user_events
 TARGETS += vDSO
 TARGETS += mm
diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc
index e12dc967ec76..b48fbe7603d9 100644
--- a/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc
+++ b/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc
@@ -33,4 +33,48 @@ if grep -q "\$current.*" README; then
 check_error 'p /bin/sh:10 ^$current:u8'	# BAD_VAR
 fi
 
+# ptwrite options may be written as an offset suffix or as separate tokens.
+# Use /bin/sh's executable entry so registration reaches the parser options.
+ptw_off=
+if command -v readelf >/dev/null 2>&1; then
+	ptw_entry=$(readelf -hW /bin/sh |
+		awk '/Entry point address:/{print $NF; exit}')
+	ptw_load_off=$(readelf -lW /bin/sh |
+		awk '$1 == "LOAD" && $0 ~ / R E/ {print $2; exit}')
+	ptw_load_vaddr=$(readelf -lW /bin/sh |
+		awk '$1 == "LOAD" && $0 ~ / R E/ {print $3; exit}')
+	if [ -n "$ptw_entry" ] && [ -n "$ptw_load_off" ] &&
+		[ -n "$ptw_load_vaddr" ]; then
+		ptw_off=$(( $(printf "%d" "$ptw_load_off") +
+			$(printf "%d" "$ptw_entry") -
+			$(printf "%d" "$ptw_load_vaddr") ))
+	fi
+fi
+if [ "$(uname -m)" = x86_64 ] &&
+	[ -e /sys/devices/intel_pt/format/ptw ] && [ -n "$ptw_off" ]; then
+check_good_ptw() {
+	local ret
+	echo > uprobe_events
+	echo "$1" > uprobe_events
+	if grep -q 'ptw:uprobes/ptw_parser' uprobe_events; then
+		ret=0
+	else
+		ret=1
+	fi
+	echo "-:ptw_parser" > uprobe_events
+	return "$ret"
+}
+
+check_good_ptw "ptw:ptw_parser /bin/sh:$ptw_off%multinop %di" || exit 1
+check_good_ptw "ptw:ptw_parser /bin/sh:$ptw_off%nopace %multinop %di" || exit 1
+check_good_ptw "ptw:ptw_parser /bin/sh:$ptw_off %multinop %nopace %di" || exit 1
+
+check_error "ptw:ptw_parser /bin/sh:$ptw_off^%return %di"	# BAD_ADDR_SUFFIX
+check_error "ptw:ptw_parser /bin/sh:$ptw_off^%unknown %di"	# BAD_ADDR_SUFFIX
+if grep -q '\$comm' README; then
+	check_error "ptw:ptw_parser /bin/sh:$ptw_off %multinop %di ^\$comm"	# BAD_FETCH_ARG
+fi
+echo > uprobe_events
+fi
+
 exit 0
diff --git a/tools/testing/selftests/uprobes/Makefile b/tools/testing/selftests/uprobes/Makefile
new file mode 100644
index 000000000000..9fc65c1f04c8
--- /dev/null
+++ b/tools/testing/selftests/uprobes/Makefile
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+# ptwrite uprobe selftests (x86-64).
+ARCH ?= $(shell uname -m 2>/dev/null || echo not)
+CFLAGS += -O2 -Wall -no-pie
+
+TEST_GEN_FILES := ptw_probe
+TEST_PROGS := run_ptw.sh run_module.sh run_perfprobe.sh run_decode.sh
+
+ifneq ($(filter x86 x86_64,$(ARCH)),)
+TEST_GEN_FILES := ptw_probe
+else
+TEST_GEN_FILES :=
+endif
+
+include ../lib.mk
diff --git a/tools/testing/selftests/uprobes/ptw_probe.c b/tools/testing/selftests/uprobes/ptw_probe.c
new file mode 100644
index 000000000000..4f1740c26411
--- /dev/null
+++ b/tools/testing/selftests/uprobes/ptw_probe.c
@@ -0,0 +1,156 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ptw_probe - ptwrite uprobe selftest target.
+ */
+#include <stdio.h>
+#include <stdint.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+static __attribute__((noipa)) uint64_t
+punfn(uint64_t a)
+{
+	uint32_t v;
+
+	asm volatile("mov $0xfff10000, %%eax\n\tmovl %%eax, %0"
+		     : "=r"(v) : : "rax");
+	return v ^ (a * 0x9e3779b97f4a7c15ULL);
+}
+
+static __attribute__((noipa)) uint64_t
+jcc8(uint64_t a)
+{
+	asm volatile("jne 1f\n\tmovabs $0x1111111111111111, %%rax\n\t"
+		     "1:" : "+a"(a) : : "cc");
+	return a;
+}
+
+static __attribute__((noipa)) uint64_t
+faultfn(uint64_t *p)
+{
+	asm volatile("nop" ::: "memory");	/* the probe site (mem arg) */
+	return (p ? *p : 0) * 31 + 7;
+}
+
+static __attribute__((noipa)) uint64_t
+nopfn(uint64_t a)
+{
+	asm volatile("nop\n\t"
+		     ".globl nopfn_site\n\t"
+		     "nopfn_site:\n\t"
+		     "nop\n\tnop\n\tnop\n\tnop\n\tnop" ::: "memory");
+	return a * 31 + 7;
+}
+
+extern const uint8_t nopfn_site[];
+
+static __attribute__((noipa)) uint64_t
+nop5(uint64_t a)
+{
+	asm volatile(".byte 0x0f, 0x1f, 0x44, 0x00, 0x00" ::: "memory");
+	return a * 7 + 3;
+}
+
+static __attribute__((noipa)) uint64_t
+rzfn(uint64_t a)
+{
+	uint64_t v;
+
+	asm volatile("movq %1, -8(%%rsp)\n\tmovq -8(%%rsp), %0"
+		     : "=r"(v) : "r"(a) : "memory");
+	asm volatile("nop\n\tnop\n\tnop\n\tnop\n\tnop" ::: "memory");
+	return v ^ 0x55;
+}
+
+static uint8_t load_site_byte(const uint8_t *p)
+{
+	return __atomic_load_n(p, __ATOMIC_RELAXED);
+}
+
+static void dump_site(const char *name, const uint8_t *p)
+{
+	printf("SITE %s %02x%02x%02x%02x%02x\n", name,
+	       load_site_byte(p + 0), load_site_byte(p + 1),
+	       load_site_byte(p + 2), load_site_byte(p + 3),
+	       load_site_byte(p + 4));
+}
+
+static int check_installed(const char *name, const uint8_t *p, uint64_t vaddr)
+{
+	uint32_t rel_u;
+	int32_t rel;
+	uint64_t target, s, e;
+	FILE *f;
+	char line[256];
+	int found = 0;
+
+	if (load_site_byte(p + 0) != 0xe9)
+		return 1;	/* not installed */
+	rel_u = (uint32_t)load_site_byte(p + 1) |
+		((uint32_t)load_site_byte(p + 2) << 8) |
+		((uint32_t)load_site_byte(p + 3) << 16) |
+		((uint32_t)load_site_byte(p + 4) << 24);
+	rel = (int32_t)rel_u;
+	target = vaddr + 5 + (int64_t)rel;
+	f = fopen("/proc/self/maps", "r");
+	if (!f)
+		return -1;
+	while (fgets(line, sizeof(line), f)) {
+		if (!strstr(line, "[uprobes-ptwrite]"))
+			continue;
+		if (sscanf(line, "%lx-%lx", &s, &e) == 2 &&
+		    target >= s && target < e) {
+			found = 1;
+			break;
+		}
+	}
+	fclose(f);
+	printf("INSTALL %s %s (target %llx)\n", name,
+	       found ? "ok" : "BAD-TARGET", (unsigned long long)target);
+	return found ? 0 : 2;
+}
+
+int main(int argc, char **argv)
+{
+	uint64_t acc = 0x1122334455667788ULL;
+	uint8_t *guard;
+	uint64_t *faultp;
+	int i, r, bad = 0;
+
+	guard = mmap(NULL, 2 * 4096, PROT_READ | PROT_WRITE,
+		     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (guard == MAP_FAILED)
+		return 2;
+	mprotect(guard + 4096, 4096, PROT_NONE);
+	faultp = (uint64_t *)(guard + 4096 - 8);
+
+	for (i = 0; i < 100; i++)
+		acc = nopfn(acc + i);
+	for (i = 0; i < 100; i++)
+		acc = nop5(acc + i);
+	for (i = 0; i < 100; i++)
+		acc = rzfn(acc + i);
+	for (i = 0; i < 100; i++)
+		acc = punfn(acc + i);
+	for (i = 0; i < 100; i++)
+		acc = jcc8(acc + i);
+	acc += faultfn(faultp);
+
+	dump_site("punfn", (const uint8_t *)&punfn);
+	dump_site("jcc8", (const uint8_t *)&jcc8);
+	dump_site("faultfn", (const uint8_t *)&faultfn);
+	dump_site("nopfn", nopfn_site);
+	dump_site("nop5", (const uint8_t *)&nop5);
+
+	r = check_installed("punfn", (const uint8_t *)&punfn, (uint64_t)&punfn);
+	bad |= r == 2;
+	r = check_installed("nopfn", nopfn_site, (uint64_t)nopfn_site);
+	bad |= r == 2;
+	r = check_installed("nop5", (const uint8_t *)&nop5, (uint64_t)&nop5);
+	bad |= r == 2;
+
+	printf("PTW-PROBE acc=%llx %s\n", (unsigned long long)acc,
+	       bad ? "INSTALL-BAD" : "ok");
+	return bad ? 1 : 0;
+}
diff --git a/tools/testing/selftests/uprobes/run_decode.sh b/tools/testing/selftests/uprobes/run_decode.sh
new file mode 100755
index 000000000000..b427442041aa
--- /dev/null
+++ b/tools/testing/selftests/uprobes/run_decode.sh
@@ -0,0 +1,177 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# run_decode.sh - ptwrite decoder selftests.
+# Exercises the decoder CLI and PT/perf integration.
+# Root + tracefs + perf + gcc + a PTWRITE-capable CPU required.
+set -u
+DIR=$(dirname "$(readlink -f "$0")")
+SRC=${1:-"$DIR/manual_ptw.c"}
+DEC=${2:-}
+if [ -z "$DEC" ]; then
+	for candidate in \
+		"$DIR/../../../../tools/perf/scripts/python/uprobe-ptwrite-decode.py" \
+		"/usr/lib/linux-tools/$(uname -r)/scripts/python/uprobe-ptwrite-decode.py" \
+		"/usr/share/linux-tools/scripts/python/uprobe-ptwrite-decode.py"; do
+		if [ -f "$candidate" ]; then
+			DEC=$candidate
+			break
+		fi
+	done
+fi
+TR=/sys/kernel/tracing
+EV="$TR/uprobe_events"
+TMP=$(mktemp -d)
+BIN="$TMP/manual_ptw"
+fails=0
+
+t() { # t <num> <ok|not> <msg>
+	if [ "$2" = ok ]; then echo "ok $1 - $3"
+	else echo "not ok $1 - $3"; fails=$((fails + 1)); fi
+}
+
+
+cleanup() {
+	{ echo 0 > "$TR/events/uprobes/e/enable"; } 2>/dev/null
+	{ echo "-:e" > "$EV"; } 2>/dev/null
+	{ echo "-:classic" > "$EV"; } 2>/dev/null
+	rm -rf "$TMP"
+}
+trap cleanup EXIT
+
+if [ ! -e /sys/devices/intel_pt/format/ptw ]; then
+	echo "1..0 # SKIP PTWRITE unavailable"
+	exit 0
+fi
+
+if [ "$(id -u)" != 0 ] || [ ! -f "$SRC" ] || [ ! -f "$DEC" ] ||
+	! command -v perf >/dev/null 2>&1 || ! command -v gcc >/dev/null 2>&1; then
+	echo "1..0 # SKIP missing root, source, decoder, perf, or gcc"
+	exit 0
+fi
+
+if ! gcc -O2 -no-pie -o "$BIN" "$SRC" 2>/dev/null; then
+	echo "1..0 # SKIP test program build failed"
+	exit 0
+fi
+
+TV=$(objdump -d "$BIN" 2>/dev/null |
+	awk '/^[0-9a-f]+ <target>:/{print $1;exit}' | tr -d ':')
+LV=$(readelf -l "$BIN" 2>/dev/null |
+	awk '/LOAD/{if ($1=="LOAD") {print $3; exit}}' | sed 's/^0x//')
+LO=$(readelf -l "$BIN" 2>/dev/null |
+	awk '/LOAD/{if ($1=="LOAD") {print $2; exit}}' | sed 's/^0x//')
+OFF=$((0x$TV - 0x$LV + 0x$LO))
+echo "1..6"
+
+# 1: missing --event-id/--types values must return a controlled error
+cli_ok=1
+python3 "$DEC" --words --event-id >/dev/null 2>&1
+rc=$?
+[ "$rc" -eq 2 ] || cli_ok=0
+python3 "$DEC" --words --types >/dev/null 2>&1
+rc=$?
+[ "$rc" -eq 2 ] || cli_ok=0
+if [ "$cli_ok" -eq 1 ]; then
+	t 1 ok "decoder option bounds checks"
+else
+	t 1 not "decoder option bounds checks"
+fi
+
+# 2: manual ptwrites only (no probe): every word must print as a
+# manual ptwrite line, and no false records may appear
+perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o "$TMP/m.data" \
+	"$BIN" >/dev/null 2>&1
+out=$(perf script --itrace=qwe -s "$DEC" -i "$TMP/m.data" 2>/dev/null)
+manual=$(printf '%s' "$out" | grep -c "manual ptwrite:")
+recs=$(printf '%s' "$out" | grep -c "^record ")
+if [ "$manual" -ge 100 ] && [ "$recs" -eq 0 ]; then
+	t 2 ok "manual ptwrites detected ($manual words, 0 records)"
+else
+	t 2 not "manual ptwrites: $manual manual, $recs records"
+fi
+
+# 3: probe + manual words in one stream
+if ! echo "ptw:e $BIN:$OFF %di %si" > "$EV" 2>/dev/null ||
+   ! echo 1 > "$TR/events/uprobes/e/enable" 2>/dev/null; then
+	t 3 not "probe create/enable failed (setup)"
+else
+	perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o "$TMP/x.data" \
+		"$BIN" >/dev/null 2>&1
+	echo 0 > "$TR/events/uprobes/e/enable" 2>/dev/null
+	out=$(perf script --itrace=qwe -s "$DEC" -i "$TMP/x.data" 2>/dev/null)
+	recs=$(printf '%s' "$out" | grep -c "^record ")
+	manual=$(printf '%s' "$out" | grep -c "manual ptwrite:")
+	if [ "$recs" -eq 100 ] && [ "$manual" -ge 100 ]; then
+		t 3 ok "probe records + manual words mixed ($recs records, $manual manual)"
+	else
+		t 3 not "mixed stream: $recs records, $manual manual"
+	fi
+	echo "-:e" > "$EV" 2>/dev/null
+fi
+
+# 4: branches: with 'b' in --itrace the decoder prints the decoded
+# branch stream interleaved with the records
+if ! echo "ptw:e $BIN:$OFF %di %si" > "$EV" 2>/dev/null ||
+   ! echo 1 > "$TR/events/uprobes/e/enable" 2>/dev/null; then
+	t 4 not "probe create/enable failed (setup)"
+else
+	perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o "$TMP/b.data" \
+		"$BIN" >/dev/null 2>&1
+	echo 0 > "$TR/events/uprobes/e/enable" 2>/dev/null
+	out=$(perf script --itrace=qweb -s "$DEC" -i "$TMP/b.data" 2>/dev/null)
+	br=$(printf '%s' "$out" | grep -c "^branch:")
+	recs=$(printf '%s' "$out" | grep -c "^record ")
+	if [ "$br" -ge 100 ] && [ "$recs" -ge 100 ]; then
+		t 4 ok "branches + records interleaved ($br branches, $recs records)"
+	else
+		t 4 not "branch stream: $br branches, $recs records"
+	fi
+	echo "-:e" > "$EV" 2>/dev/null
+fi
+
+# 5: other event classes
+if ! echo "ptw:e $BIN:$OFF %di %si" > "$EV" 2>/dev/null ||
+   ! echo 1 > "$TR/events/uprobes/e/enable" 2>/dev/null ||
+   ! echo "p:classic $BIN:$((OFF+5)) %di %si" >> "$EV" 2>/dev/null; then
+	t 5 not "mixed-class probe create failed (setup)"
+else
+	perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -e uprobes:classic \
+		-e sched:sched_process_exec -o "$TMP/c.data" "$BIN" >/dev/null 2>&1
+	echo 0 > "$TR/events/uprobes/e/enable" 2>/dev/null
+	out=$(perf script --itrace=qwe -s "$DEC" -i "$TMP/c.data" 2>/dev/null)
+	recs=$(printf '%s' "$out" | grep -c "^record ")
+	manual=$(printf '%s' "$out" | grep -c "manual ptwrite:")
+	cl=$(printf '%s' "$out" | grep -c "^event:.*uprobes:classic")
+	tp=$(printf '%s' "$out" | grep -c "^event:.*sched:sched_process_exec")
+	if [ "$recs" -ge 100 ] && [ "$manual" -ge 100 ] && \
+	   [ "$cl" -ge 100 ] && [ "$tp" -ge 1 ]; then
+		t 5 ok "classic uprobe + tracepoint interleaved \
+($recs recs, $manual manual, $cl classic, $tp exec)"
+	else
+		t 5 not "mixed classes: $recs recs, $manual manual, $cl classic, $tp exec"
+	fi
+	echo "-:e" > "$EV" 2>/dev/null
+	echo "-:classic" > "$EV" 2>/dev/null
+fi
+
+# 6: --no-branches suppresses the branch stream
+if ! echo "ptw:e $BIN:$OFF %di %si" > "$EV" 2>/dev/null ||
+   ! echo 1 > "$TR/events/uprobes/e/enable" 2>/dev/null; then
+	t 6 not "probe create/enable failed (setup)"
+else
+	perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o "$TMP/n.data" \
+		"$BIN" >/dev/null 2>&1
+	echo 0 > "$TR/events/uprobes/e/enable" 2>/dev/null
+	out=$(perf script --itrace=qweb -s "$DEC" -i "$TMP/n.data" \
+		-- --no-branches 2>/dev/null)
+	br=$(printf '%s' "$out" | grep -c "^branch:")
+	recs=$(printf '%s' "$out" | grep -c "^record ")
+	if [ "$br" -eq 0 ] && [ "$recs" -ge 100 ]; then
+		t 6 ok "--no-branches suppresses branches ($br branches, $recs records)"
+	else
+		t 6 not "--no-branches: $br branches, $recs records"
+	fi
+	echo "-:e" > "$EV" 2>/dev/null
+fi
+
+[ $fails -eq 0 ] || exit 1
diff --git a/tools/testing/selftests/uprobes/run_ptw.sh b/tools/testing/selftests/uprobes/run_ptw.sh
new file mode 100755
index 000000000000..fb23ff83c384
--- /dev/null
+++ b/tools/testing/selftests/uprobes/run_ptw.sh
@@ -0,0 +1,226 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# run_ptw.sh - ptwrite uprobes selftests.
+# Root + tracefs + an x86-64 CPU with PTWRITE required
+DIR=$(dirname "$(readlink -f "$0")")
+BIN="$DIR/ptw_probe"
+TR=/sys/kernel/tracing
+EV="$TR/uprobe_events"
+PTW=/sys/devices/intel_pt/format/ptw
+
+cleanup() {
+	if [ -e "$TR/events/uprobes/pw/enable" ]; then
+		echo 0 > "$TR/events/uprobes/pw/enable" 2>/dev/null
+	fi
+	echo "-:pw" > "$EV" 2>/dev/null
+	echo "-:bad" > "$EV" 2>/dev/null
+	[ -z "$TMP" ] || rm -rf "$TMP"
+}
+
+if [ ! -e "$PTW" ]; then
+	echo "1..0 # SKIP PTWRITE unavailable"
+	exit 0
+fi
+if [ ! -d "$TR" ] || [ "$(id -u)" != 0 ] || [ ! -x "$BIN" ]; then
+	echo "1..0 # SKIP missing tracefs, root, or ptw_probe"
+	exit 0
+fi
+TMP=$(mktemp -d "${TMPDIR:-/tmp}/ptw.XXXXXX") || {
+	echo "1..0 # SKIP unable to create secure temporary directory"
+	exit 0
+}
+PERF_DATA="$TMP/ptw-perf.data"
+trap cleanup EXIT
+
+# the probe sites: the entry instructions
+elf_off() {
+	local v=$1
+	local base=$(readelf -l "$BIN" 2>/dev/null |
+		awk '/LOAD/{if ($1=="LOAD") {print $3; exit}}')
+	[ -n "$base" ] && printf "0x%x" $((v - base))
+}
+
+PUN_V=$(objdump -d "$BIN" | awk '/^[0-9a-f]+ <punfn>:/{print $1;exit}' | tr -d ':')
+JCC_V=$(objdump -d "$BIN" |
+	awk '/^[0-9a-f]+ <jcc8>:/ {f=1; next} f&&/jne/{print $1; exit}' |
+	tr -d ':')
+FLT_V=$(objdump -d "$BIN" | awk '/^[0-9a-f]+ <faultfn>:/{print $1;exit}' | tr -d ':')
+NOP_V=$(objdump -d "$BIN" | awk '/^[0-9a-f]+ <nopfn_site>:/{print $1;exit}' | tr -d ':')
+NOP5_V=$(objdump -d "$BIN" | awk '/^[0-9a-f]+ <nop5>:/{print $1;exit}' | tr -d ':')
+RZ_V=$(objdump -d "$BIN" | awk '/^[0-9a-f]+ <rzfn>:/{print $1;exit}' | tr -d ':')
+PUN_OFF=$(elf_off 0x$PUN_V)
+JCC_OFF=$(elf_off 0x$JCC_V)
+FLT_OFF=$(elf_off 0x$FLT_V)
+NOP_OFF=$(elf_off 0x$NOP_V)
+NOP5_OFF=$(elf_off 0x$NOP5_V)
+RZ_OFF=$(elf_off 0x$RZ_V)
+
+echo "1..12"
+failures=0
+
+# baseline (unprobed)
+base_out=$("$BIN"); base_rc=$?
+base=$(printf '%s' "$base_out" | sed -n 's/.*acc=\([0-9a-f]*\).*/\1/p')
+base_sites=$(printf '%s' "$base_out" | grep '^SITE ')
+[ -z "$base" ] && base=0
+
+# run one probed invocation: $run_rc = exit code, $probe = the acc
+run_one() {
+	out=$("$BIN")
+	run_rc=$?
+	probe=$(printf '%s' "$out" | sed -n 's/.*acc=\([0-9a-f]*\).*/\1/p')
+}
+
+# the site bytes of a fresh invocation must equal the baseline
+sites_match() {
+	[ "$(printf '%s' "$base_sites")" = \
+	  "$("$BIN" | grep '^SITE ')" ]
+}
+
+# emit the TAP line and count failures (tap <num> <ok|not|skip> <desc>)
+tap() {
+	if [ "$2" = ok ]; then
+		echo "ok $1 - $3"
+	elif [ "$2" = skip ]; then
+		echo "ok $1 - $3 # SKIP"
+	else
+		echo "not ok $1 - $3"
+		failures=$((failures + 1))
+	fi
+}
+
+# Install a probe at the site, run the probed binary once, and check the
+# run against the baseline (acc, exit, restored site bytes). A
+# create/enable failure is fatal.
+# Usage: probe_run <num> <offset> <args> <desc>
+probe_run() {
+	local num=$1 off=$2 args=$3 desc=$4
+
+	if ! echo "ptw:pw $BIN:$off $args" > "$EV" 2>/dev/null; then
+		tap "$num" not "$desc (probe create failed)"
+		exit 1
+	fi
+	if ! echo 1 > "$TR/events/uprobes/pw/enable" 2>/dev/null; then
+		tap "$num" not "$desc (probe enable failed)"
+		exit 1
+	fi
+	run_one
+	echo 0 > "$TR/events/uprobes/pw/enable" 2>/dev/null
+	echo "-:pw" > "$EV" 2>/dev/null
+	if [ "$probe" = "$base" ] && [ "$run_rc" -eq 0 ] && sites_match; then
+		tap "$num" ok "$desc"
+	else
+		tap "$num" not "$desc (base $base probed $probe rc $run_rc)"
+	fi
+}
+
+# 1: pun out-of-line execution preserves the site instruction's effect
+probe_run 1 $PUN_OFF "%di %si" \
+	"pun out-of-line execution preserves the instruction effect"
+
+# 2: a relative branch site must be refused at enable
+if ! echo "ptw:pw $BIN:$JCC_OFF %di %si" > "$EV" 2>/dev/null; then
+	tap 2 ok "rel8 jcc site rejected at create"
+else
+	if echo 1 > "$TR/events/uprobes/pw/enable" 2>/dev/null; then
+		echo 0 > "$TR/events/uprobes/pw/enable" 2>/dev/null
+		tap 2 not "rel8 jcc site enabled (expected rejection)"
+	else
+		tap 2 ok "rel8 jcc site rejected (no re-encode)"
+	fi
+	echo "-:pw" > "$EV" 2>/dev/null
+fi
+
+# 3: memory-arg fault fixup (a bad base fixes up to word 0)
+probe_run 3 $FLT_OFF "+8(%di) %si" \
+	"memory-arg fault fixup (child survived, acc unchanged)"
+
+# 4: ptwrite stream decode (words must capture + decode cleanly)
+if ! command -v perf >/dev/null 2>&1; then
+	tap 4 skip "ptwrite decode smoke (no perf)"
+elif ! { echo "ptw:pw $BIN:$PUN_OFF %di %si" > "$EV" &&
+	echo 1 > "$TR/events/uprobes/pw/enable" &&
+	perf record -e intel_pt/ptw=1,fup_on_ptw=1/u -o "$PERF_DATA" \
+		"$BIN" >/dev/null 2>&1 &&
+	echo 0 > "$TR/events/uprobes/pw/enable"; }; then
+	tap 4 not "perf record failed"
+else
+	words=$(perf script --itrace=qwe -i "$PERF_DATA" 2>/dev/null |
+		grep -c "ptwrite:")
+	if [ "${words:-0}" -gt 0 ]; then
+		tap 4 ok "ptwrite stream decode ($words words)"
+	else
+		tap 4 not "no ptwrite words decoded"
+	fi
+fi
+echo "-:pw" > "$EV" 2>/dev/null
+
+# 5: mini-stress (50 fork/execs survive)
+if ! echo "ptw:pw $BIN:$PUN_OFF %di %si" > "$EV" 2>/dev/null ||
+   ! echo 1 > "$TR/events/uprobes/pw/enable" 2>/dev/null; then
+	tap 5 not "churn mini-stress setup failed"
+else
+	fails=0
+	for i in $(seq 1 50); do
+		"$BIN" >/dev/null 2>&1 || fails=$((fails + 1))
+	done
+	echo 0 > "$TR/events/uprobes/pw/enable" 2>/dev/null
+	if [ "$fails" -eq 0 ] && sites_match; then
+		tap 5 ok "churn mini-stress (50 execs, 0 failures)"
+	else
+		tap 5 not "churn mini-stress ($fails/50 failed)"
+	fi
+fi
+echo "-:pw" > "$EV" 2>/dev/null
+
+# 6: a 5x1-byte NOP run takes the pun path (site restored)
+probe_run 6 "${NOP_OFF}%multinop" "%di %si" \
+	"misaligned NOP-composition install (probe fires, target valid, site restored)"
+
+# 7: the single 5-byte NOP keeps the classic 3-phase poke
+probe_run 7 $NOP5_OFF "%di %si" \
+	"single 5-byte-NOP 3-phase install (probe fires, target valid, site restored)"
+
+# 8: enable/disable flip loop (50 re-installs stay correct)
+if ! echo "ptw:pw $BIN:$NOP5_OFF %di %si" > "$EV" 2>/dev/null ||
+   [ ! -e "$TR/events/uprobes/pw/enable" ]; then
+	tap 8 not "flip loop setup failed"
+else
+	fails=0
+	for i in $(seq 1 50); do
+		echo 1 > "$TR/events/uprobes/pw/enable" 2>/dev/null ||
+			fails=$((fails + 1))
+		"$BIN" >/dev/null 2>&1 || fails=$((fails + 1))
+		echo 0 > "$TR/events/uprobes/pw/enable" 2>/dev/null ||
+			fails=$((fails + 1))
+	done
+	if [ "$fails" -eq 0 ] && sites_match; then
+		tap 8 ok "enable/disable flip loop (50 flips, 0 failures, site restored)"
+	else
+		tap 8 not "flip loop ($fails failures)"
+	fi
+fi
+echo "-:pw" > "$EV" 2>/dev/null
+
+# 9: a 4-arg paced probe at a site with a stack-local sentinel
+probe_run 9 $RZ_OFF "%di %si %dx %r8" \
+	"4-arg paced probe"
+
+# 10: %nopace attached to the offset remains accepted
+probe_run 10 "${PUN_OFF}%nopace" "%di %si" \
+	"%nopace offset suffix is accepted"
+
+# 11: %nopace as a separate option remains accepted
+probe_run 11 "$PUN_OFF" "%nopace %di %si" \
+	"%nopace separate option is accepted"
+
+# 12: unknown ptwrite options must be rejected by tracefs
+if echo "ptw:bad $BIN:${PUN_OFF}%unknown %di %si" > "$EV" 2>/dev/null; then
+	echo "-:bad" > "$EV" 2>/dev/null
+	tap 12 not "unknown ptwrite option accepted"
+else
+	tap 12 ok "unknown ptwrite option rejected"
+fi
+
+# the kselftest runner uses only the exit code
+[ "$failures" -eq 0 ] || exit 1
-- 
2.54.0

Re: [RFC v1 01/19] uprobes: guard trace cleanup against error pointers

[sashiko-bot]

> Sashiko pointed out the some of the scope cleanups for free_uprobe
> could get an error pointer. Handle this case in free_uprobe
> to prevent a crash.
> 
> On the other hand the macro doesn't need the guard because
> free_uprobe itself already does the check.
> 
> Assisted-by: omp:gpt-5.6-luna sashiko
> Signed-off-by: Andi Kleen <ak@kernel.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=1

Re: [RFC v1 02/19] uprobes: Correctly reject anonymous VMAs for breakpoint installation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Page tables are modified under mmap_write_lock without calling vma_start_write(vma), risking a race with concurrent per-VMA locked operations.
- [High] Several functions access vma->vm_file or use it for classification without first checking !vma_is_anonymous(vma), leaving them vulnerable to anonymous VMAs that have a vm_file set.
--

commit e85d99904cf4af95b805f715ec281558955b0733
Author: Andi Kleen <ak@kernel.org>

uprobes: Correctly reject anonymous VMAs for breakpoint installation

This patch updates valid_vma() to correctly check for anonymous VMAs 
using vma_is_anonymous() instead of relying only on vm_file.

> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> index 7709ea8824778..4287c4ff4c0fd 100644
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
> @@ -139,7 +139,8 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)
>  	if (is_register)
>  		flags |= VM_WRITE;
>  
> -	return vma->vm_file && (vma->vm_flags & flags) == VM_MAYEXEC;
> +	return !vma_is_anonymous(vma) && vma->vm_file &&
> +		(vma->vm_flags & flags) == VM_MAYEXEC;
>  }

[Severity: High]
This isn't a bug introduced by this patch, but since you are fixing the
anonymous VMA check here in valid_vma(), do we need to apply this same
!vma_is_anonymous(vma) check to other functions in this file?

Several other functions, such as valid_ref_ctr_vma(), 
find_active_uprobe_speculative(), find_active_uprobe_rcu(), and 
uprobe_mmap(), appear to rely directly on vma->vm_file to determine if 
a VMA is file-backed. 

For example, valid_ref_ctr_vma() checks:

    return uprobe->ref_ctr_offset &&
           vma->vm_file &&
           file_inode(vma->vm_file) == uprobe->inode;

If an anonymous VMA happens to have a backing file (e.g., from a private 
mapping of /dev/zero or a driver mapping using vma_set_anonymous()), could 
this incorrectly match the inode and update reference counters in anonymous
memory, or compute incorrect file offsets in the speculative lookups?

[Severity: High]
This is a pre-existing issue, but while reviewing the VMA handling in
uprobes, does this file also need to call vma_start_write(vma) when
modifying page tables?

Paths like register_for_each_vma() and unapply_uprobe() take
mmap_write_lock() and eventually call uprobe_write() to modify page
tables via folio_walk_start() in __uprobe_write(). 

Because mmap_write_lock() does not drain per-VMA read locks, a concurrent 
page table operation (like MADV_DONTNEED) under a per-VMA lock could 
clear a PMD and free the PTE page while uprobe_write() is walking it.

Could this race lead to a use-after-free or page table corruption when
tracers manipulate the memory layout concurrently without the protection of 
vma_start_write() draining concurrent VMA-lock readers?

>  
>  static unsigned long offset_to_vaddr(struct vm_area_struct *vma, loff_t offset)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=2

Re: [RFC v1 03/19] uprobes: Print warning for missing breakpoint install

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing VMA write lock during uprobe registration exposes page table modifications to races
--

commit c4ab2afca2dc57fb6a2f6789af30c2bec5a5064c
Author: Andi Kleen <ak@kernel.org>

uprobes: Print warning for missing breakpoint install
    
When a uprobe breakpoint installation fails, the error is currently silently
ignored. This commit adds a rate-limited warning to log these failures and
make debugging easier.

> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> index 4287c4ff4c0fd..941b52c478585 100644
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
> @@ -1630,7 +1630,13 @@ int uprobe_mmap(struct vm_area_struct *vma)
>  		if (!fatal_signal_pending(current) &&
>  		    filter_chain(uprobe, vma->vm_mm)) {
>  			unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
> -			install_breakpoint(uprobe, vma, vaddr);
> +			int err = install_breakpoint(uprobe, vma, vaddr);
                        ^^^^

[Severity: High]
This isn't a bug introduced by this patch, but modifying install_breakpoint()
reminded me of a pre-existing issue where it is called from
register_for_each_vma() without isolating the VMA.

In register_for_each_vma(), the code acquires mmap_write_lock(mm) and
dispatches install_breakpoint() or remove_breakpoint():

kernel/events/uprobes.c:register_for_each_vma() {
    ...
		mmap_write_lock(mm);
    ...
		if (is_register) {
			/* consult only the "caller", new consumer. */
			if (consumer_filter(new, mm))
				err = install_breakpoint(uprobe, vma, info->vaddr);
		} else if (mm_flags_test(MMF_HAS_UPROBES, mm)) {
			if (!filter_chain(uprobe, mm))
				err |= remove_breakpoint(uprobe, vma, info->vaddr);
		}
    ...
}

Both install_breakpoint() and remove_breakpoint() ultimately call
uprobe_write() to clear PTEs and collapse PMDs via __uprobe_write() and
collapse_pte_mapped_thp().

However, as per VMA lock invariants, mmap_write_lock alone does not exclude
concurrent per-VMA read lock holders (like page faults) that acquired the lock
prior to the write lock.

Should register_for_each_vma() call vma_start_write(vma) before dispatching
these functions? Failing to drain per-VMA lock holders could allow readers to
concurrently walk the VMA's page tables while uprobe_write() clears them,
potentially leading to use-after-free of PTE pages or stale PMD reads.

> +
> +			if (err)
> +				pr_warn_ratelimited(
> +					"uprobes: probe %pD+0x%llx failed to install (%d)\n",
> +					vma->vm_file,
> +					(unsigned long long)uprobe->offset, err);
>  		}
[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=3

Re: [RFC v1 04/19] ptwrite uprobes: Add infrastructure for ptwrite uprobes

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] handle_swbp() does not check for UPROBE_PTWRITE, leading to a kernel crash (NULL pointer dereference) if a thread traps at a PTWRITE uprobe address.
- [High] uprobe_register_ptwrite incorrectly reuses dying normal uprobes, leading to race conditions and SIGILL.
- [Low] uprobe_register_ptwrite requires a non-NULL handler despite documenting that it is never invoked.
--

commit 25466f8679470946e2456212a51efd54e55bf02c
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Add infrastructure for ptwrite uprobes

uprobes currently always require entering the kernel to log anything.
While that works well, it is rather slow.
Modern Intel CPUs have the ptwrite instruction, which can log data to
the Processor Trace buffer. This patch adds support in uprobes
to patch in ptwrites instead of the normal probes.

> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> index 941b52c478585..23202df2b51ae 100644
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
> @@ -59,6 +59,8 @@ DEFINE_STATIC_SRCU_FAST_UPDOWN(uretprobes_srcu);
>  /* Have a copy of original instruction */
>  #define UPROBE_COPY_INSN	0
>  
> +#define UPROBE_PTWRITE		1
> +

[Severity: Critical]
Since PTWRITE uprobes do not have an initialized uprobe->arch.ops pointer,
what happens if an unprivileged user attaches with ptrace and inserts a
breakpoint at the exact address of the PTWRITE uprobe?

If the process hits the breakpoint, it enters the kernel's handle_swbp(),
looks up the uprobe, and proceeds to call arch_uprobe_skip_sstep().
Since handle_swbp() doesn't check for the UPROBE_PTWRITE flag, wouldn't
this unconditionally dereference the uninitialized auprobe->ops->emulate,
resulting in a kernel panic (null pointer dereference)?

[ ... ]
> +struct uprobe *uprobe_register_ptwrite(struct inode *inode, struct file *file,
> +				       loff_t offset, struct uprobe_consumer *uc,
> +				       const struct uprobe_ptwrite_desc *desc)
> +{
> +	struct uprobe *uprobe;
> +	int ret;
> +
> +	if (!file || (!uc->handler && !uc->ret_handler))
> +		return ERR_PTR(-EINVAL);

[Severity: Low]
Is this validation check correct for a PTWRITE uprobe? The comment above
states "(handler is never invoked: no kernel entry at probe hit)", but this
check requires the caller to supply a non-null handler anyway.

[ ... ]
> +	down_write(&uprobe->register_rwsem);
> +
> +	/*
> +	 * A dying normal uprobe can make reuse temporarily busy; don't overwrite
> +	 * it.
> +	*/
> +	if (!list_empty(&uprobe->consumers)) {
> +		ret = -EBUSY;
> +		goto out;
> +	}

[Severity: High]
Does this logic safely handle dying normal uprobes?

If a normal uprobe is unregistered, its consumers list becomes empty, but
threads might still be hitting the breakpoint and executing handle_swbp().
Concurrently, uprobe_register_ptwrite() called for the same address will
reuse the dying uprobe because its consumers list is empty, and then mutate
its arch state to PTWRITE, installing a 5-byte jump.

Won't the threads finishing XOL for the normal uprobe return to
bp_vaddr + ilen, which now points into the middle of the 5-byte jump,
resulting in an invalid opcode exception for the user-space process?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=4

Re: [RFC v1 05/19] ptwrite uprobes: Add minimal low level support for x86

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Calling synchronize_rcu() inside a loop in arch_uprobe_clear_state() causes O(N) grace period delays while holding mmap_write_lock.
- [High] find_ptwrite_page_area bypasses mmap_min_addr security protections.
- [High] Child processes execute the parent's ptwrite page due to missing VM_WIPEONFORK/VM_DONTCOPY on the special mapping.
--

commit eb94d55c50bf768d31e8cac28283dd4258bc8a40
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Add minimal low level support for x86

This commit adds low-level x86 support for generating and executing
PTWRITE instructions for uprobes. It pre-generates instructions into
templates and patches them when setting up the user page.

> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index 65a2de82ecd29..df652c56414b5 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c
[ ... ]
> @@ -717,6 +721,68 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign
[ ... ]
> +void arch_uprobe_clear_state(struct mm_struct *mm)
> +{
> +	struct uprobes_state *state = &mm->uprobes_state;
> +	struct uprobe_ptwrite_page *ptw;
> +	struct hlist_node *n;
> +
> +	hlist_for_each_entry_safe(ptw, n, &state->head_ptwrite, node) {
> +		hlist_del_rcu(&ptw->node);
> +		synchronize_rcu();

[Severity: High]
Does calling synchronize_rcu() inside this loop cause an O(N) grace period
delay while holding the mmap_write_lock?

As shown in arch_uprobe_dup_ptwrite() below, arch_uprobe_clear_state() is
called on the failure path while both the parent and child's mmap_write_lock
are held by dup_mmap(). An attacker could potentially fork and intentionally
cause memory allocation failures to stall the parent's mmap_write_lock for
several RCU grace periods, leading to a Denial of Service.

> +		__free_page(ptw->page);
> +		kfree(ptw);
> +	}
> +}
> +
> +int arch_uprobe_dup_ptwrite(struct mm_struct *oldmm, struct mm_struct *newmm)
> +{
> +	struct uprobes_state *old_state = &oldmm->uprobes_state;
> +	struct uprobes_state *new_state = &newmm->uprobes_state;
> +	struct uprobe_ptwrite_page *ptw, *new;
> +
> +	mmap_assert_write_locked(oldmm);
> +	mmap_assert_write_locked(newmm);
[ ... ]
> +fail:
> +	arch_uprobe_clear_state(newmm);
> +	return -ENOMEM;
> +}
[ ... ]
> @@ -1102,6 +1223,508 @@ static int copy_from_vaddr(struct mm_struct *mm, unsigned long vaddr, void *dst,
[ ... ]
> +static unsigned long find_ptwrite_page_area(struct mm_struct *mm,
> +					    unsigned long vaddr)
> +{
> +	VMA_ITERATOR(vmi, mm, 0);
> +	struct vm_area_struct *vma;
> +	unsigned long low, high, prev, call_end;
> +	const unsigned long call_range = (unsigned long)INT_MAX + 1;
> +
> +	mmap_assert_write_locked(mm);
> +	if (check_add_overflow(vaddr, 5UL, &call_end))
> +		return -ENOMEM;
> +	if (call_end < call_range)
> +		low = PAGE_SIZE;
> +	else
> +		low = call_end - call_range;
> +	if (low < PAGE_SIZE)
> +		low = PAGE_SIZE;

[Severity: High]
Does hardcoding the lower bound to PAGE_SIZE bypass mmap_min_addr security
protections?

If a traced user-space process installs a probe near the bottom of its
address space, find_ptwrite_page_area() will search starting at PAGE_SIZE
(0x1000). If a gap is found, _install_special_mapping() could map the
executable ptwrite VMA near NULL, which undermines the sysctl protection
against kernel NULL-pointer dereference exploits.

> +	if (low > ULONG_MAX - (PAGE_SIZE - 1))
> +		return -ENOMEM;
> +	low = PAGE_ALIGN(low);
[ ... ]
> +static struct uprobe_ptwrite_page *
> +create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
> +{
> +	struct uprobe_ptwrite_page *ptw;
> +	struct vm_area_struct *vma;
> +	unsigned long area;
> +
> +	area = find_ptwrite_page_area(mm, vaddr);
> +	if (IS_ERR_VALUE(area))
> +		return NULL;
> +
> +	mmap_assert_write_locked(mm);
> +
> +	ptw = kzalloc_obj(*ptw);
> +	if (!ptw)
> +		return NULL;
> +
> +	ptw->page = alloc_page(GFP_HIGHUSER | __GFP_ZERO);
> +	if (!ptw->page) {
> +		kfree(ptw);
> +		return NULL;
> +	}
> +	ptw->vaddr = area;
> +
> +	vma = _install_special_mapping(mm, area, PAGE_SIZE,
> +			VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
> +			&ptwrite_mapping);

[Severity: High]
Can child processes incorrectly execute the parent's ptwrite page because
VM_WIPEONFORK or VM_DONTCOPY are missing here?

When a traced process forks, the child's VMA will receive PTEs pointing to
the parent's physical ptwrite page. Later, arch_uprobe_dup_ptwrite()
allocates a new page for the child's metadata, but the VMA still executes
the parent's page. This can cause the child to execute invalid instructions
or zeroes.

> +	if (IS_ERR(vma)) {
> +		__free_page(ptw->page);
> +		kfree(ptw);
> +		return NULL;
> +	}
> +	return ptw;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=5

Re: [RFC v1 06/19] ptwrite uprobes: Add a sample module to exercise interface

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Hardcoded local developer path in Makefile comment.
--

commit 6f339cbb02cf7327f16d076027733bac7f4b2900
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Add a sample module to exercise interface

This commit adds a basic test module and a test program for uprobes ptwrite.
It provides an interface to configure and register a ptwrite uprobe in an
executable for testing purposes.

> diff --git a/samples/uprobe-ptwrite/Makefile b/samples/uprobe-ptwrite/Makefile
> new file mode 100644
> index 0000000000000..086d84be6ad2e
> --- /dev/null
> +++ b/samples/uprobe-ptwrite/Makefile
> @@ -0,0 +1,13 @@
[ ... ]
> +#
> +# The userspace test target (test_prog) is built by the out-of-tree test
> +# harness (see /home/oc/uprobe-ptwrite-test); the harness computes the file

[Severity: Low]
This isn't a bug, but is this local path (/home/oc/uprobe-ptwrite-test)
intended to be included in the upstream kernel source? It appears to be a
local developer directory rather than a path that would be accessible to
other developers.

> +# offset of target()'s 5-byte NOP and passes it to the module's offset
> +# parameter.
> +
> +obj-$(CONFIG_SAMPLE_UPROBE_PTWRITE) += uprobe_ptwrite_test.o

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=6

Re: [RFC v1 07/19] ptwrite uprobes: Add support to tracing infrastructure

[sashiko-bot]

> Hook up the low level x86 ptwrite uprobes code to the generic trace
> uprobes events parser, so that the new probes can be set up. The
> interface is similar to classic probes, but there is new ptw: syntax
> and various restrictions.
> 
> Add minimal docs.
> 
> Architectures without the ptwrite backend are handled by weak stubs.
> 
> Assisted-by: omp:gpt-5.6-luna
> Signed-off-by: Andi Kleen <ak@kernel.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=7

Re: [RFC v1 08/19] ptwrite uprobes / x86: Add a user fault notifier chain

[sashiko-bot]

> The ptwrite uprobes need to catch faults in the user probes, otherwise a
> bad probe could crash the program. For classic probes that is handled in
> the kernel, but with these new kinds of probes the crash happens in ring 3
> code.
> 
> The existing die chain cannot be used for this because it only handles
> kernel level faults. Add a new user fault notifier chain that is supported
> for #GP, #PF, #SS. It is only called before a signal would be delivered, so
> it doesn't slow down any hot paths. The fault handler can then handle the
> fault and prevent the signal.
> 
> Add register code and the hooks for the chain.
> 
> Some of the existing fault hardware workarounds could be converted to this
> in the future (not done yet)
> 
> Assisted-by: omp:gpt-5.6-luna
> Signed-off-by: Andi Kleen <ak@kernel.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=8

Re: [RFC v1 09/19] ptwrite uprobes: Factor file-backed instruction reads

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] copy_insn() unconditionally returns -EIO for partial reads, breaking uprobes placed near the end of an executable.
--

commit 357913ae591e42b1fd264a2ff741868c174c2724
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Factor file-backed instruction reads

This commit refactors copy_insn() into a more generic uprobe_copy_from_file()
function. The existing copy_insn() is updated to use this generic version,
which is intended to be used by later patches.

> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> index 23202df2b51ae..20fa16ed85198 100644
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
[ ... ]
> +int uprobe_copy_from_file(struct inode *inode, struct file *file,
> +			  loff_t offset, void *buf, int size)
>  {
[ ... ]
> +	while (copied < size) {
> +		file_size = i_size_read(inode);
> +		if (offset >= file_size)
>  			break;
> 
> -		len = min_t(int, size, PAGE_SIZE - (offs & ~PAGE_MASK));
> -		err = __copy_insn(mapping, filp, insn, len, offs);
> +		len = min_t(loff_t, size - copied, file_size - offset);
[ ... ]
> +static int copy_insn(struct uprobe *uprobe, struct file *filp)
> +{
> +	int ret;
> +
> +	ret = uprobe_copy_from_file(uprobe->inode, filp, uprobe->offset,
> +				   &uprobe->arch.insn,
> +				   sizeof(uprobe->arch.insn));
> +	return ret < 0 ? ret : ret == sizeof(uprobe->arch.insn) ? 0 : -EIO;

[Severity: High]
Does this unconditionally return -EIO for partial reads near the end of the
file?

When registering a uprobe on a target file where the instruction offset is
close to the end of the file (where file_size - offset is smaller than
sizeof(uprobe->arch.insn)), uprobe_copy_from_file() will stop at the file
boundary and return a byte count strictly less than the requested size.

The original copy_insn() loop stopped reading at EOF but still returned 0 as
long as no explicit error occurred, allowing partial reads to succeed.

By enforcing a strict ret == sizeof(...) check here, will this fail uprobe
registration for valid instructions that happen to reside near the end of small
binaries or scripts?

>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=9

Re: [RFC v1 10/19] ptwrite uprobes: Minimal memory references and fault handling

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Premature publication of `ptw->nblocks` without memory barriers exposes uninitialized memory to concurrent page fault handlers.
- [Medium] Rolling back `ptw->nblocks` on error allows in-place reuse of array slots without an RCU grace period, causing torn reads in concurrent fault handlers.
--

commit 925690411c45029e0f0a323ac69035b5e1cc8c87
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Minimal memory references and fault handling

Add support for memory references. Currently this is only simple cases,
no indirect memory references or strings. This requires fault handling
using the fault notifier hook added earlier.

> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index df652c56414b5..90e702a4a8e99 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c
[ ... ]
> @@ -1689,13 +1852,22 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
>  		return -ENOMEM;
>  
>  	block_off = ptw->cursor;
> -	if (block_off > PAGE_SIZE ||
> -	    ptw_a->stub_len > PAGE_SIZE - block_off)
> +	if (block_off > PAGE_SIZE || ptw_a->stub_len > PAGE_SIZE - block_off)
> +		return -ENOMEM;
> +	if (ptw->nblocks >= ARRAY_SIZE(ptw->index))
>  		return -ENOMEM;
>  	stub_addr = ptw->vaddr + block_off;
>  	if (!ptwrite_rel32(stub_addr + ptw_a->jmp_off + 4,
>  			   vaddr + 5, &rel))
>  		return -ERANGE;
> +	site_delta = (s64)vaddr - (s64)ptw->vaddr;
> +	if (site_delta < INT_MIN || site_delta > INT_MAX)
> +		return -ERANGE;
> +
> +	ptw->index[ptw->nblocks].off = block_off;
> +	ptw->index[ptw->nblocks].len = ptw_a->stub_len;
> +	ptw->index[ptw->nblocks].ft_off = ptw_a->ft_off;
> +	ptw->nblocks++;
>  
>  	kaddr = kmap_local_page(ptw->page);
>  	memcpy(kaddr + block_off, ptw_a->stub, ptw_a->stub_len);

[Severity: Medium]
Does this code expose uninitialized memory to concurrent page fault handlers?

Since ptw->nblocks is incremented before memcpy() initializes the stub
block memory, a concurrent lockless reader in uprobe_ptwrite_handle_fault()
might observe the incremented nblocks and read the uninitialized fault table
while memcpy() is actively writing to it.

> @@ -1704,9 +1876,13 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
>  	kunmap_local(kaddr);
>  
>  	ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
> -	if (!ret)
> -		ptw->cursor = block_off + ptw_a->stub_len;
> -	return ret;
> +	if (ret)
> +		/* Publish rollback before readers use the reduced block count. */
> +		smp_store_release(&ptw->nblocks, ptw->nblocks - 1);
> +		return ret;
> +	}

[Severity: Medium]
Can this rollback of ptw->nblocks cause use-after-free or torn reads for
concurrent readers?

If ptwrite_text_poke() fails, ptw->nblocks is rolled back. However, a
lockless reader executing uprobe_ptwrite_handle_fault() under rcu_read_lock()
might have already read the old ptw->nblocks and be actively iterating over
this slot. A subsequent probe installation could then immediately reuse this
array slot and block_off memory without waiting for a synchronize_rcu()
grace period, causing the lockless reader to observe torn data.

> +	ptw->cursor = block_off + ptw_a->stub_len;
> +	return 0;
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=10

Re: [RFC v1 11/19] ptwrite uprobes: Add multinop support

[sashiko-bot]

> GCC's -fpatchable-function-entry=5 may emit five one-byte NOPs. Normally
> that's not safe to patch because some might jump into a later nop.
> But for the gcc case it's safe because nobody jumps into the nops.
> Add a %multinop that allows the user opting into patching these sites.
> This way patching for the gcc instrumentation works.
> 
> Assisted-by: omp:gpt-5.6-luna
> Signed-off-by: Andi Kleen <ak@kernel.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=11

Re: [RFC v1 12/19] ptwrite uprobes: Add pacing to the probes

[sashiko-bot]

> When PTWRITEs are too tightly spaced they can lose data. While the decoder
> makes some effort to recover from this it can be still annoying for the
> user. This patch adds LFENCEs between the individual instructions to
> mimimize (mostly avoid) this problem. It can still happen with parallel
> branch collection or if a high frequency of timing packets are configured.
> The extend also depends on the core-type.
> 
> It can be still disabled with %nopace. This is useful when the user knows
> the probes are not too tightly spaced. It is usually still needed
> with many arguments.
> 
> This patch generates LFENCEs for the probes unless disabled.
> 
> Compared with %nopace, default pacing increases probe cost by roughly 272%
> with PT off, 227% with full tracing, and 48% in snapshot mode on Alder Lake.
> 
> Assisted-by: omp:gpt-5.6-luna
> Signed-off-by: Andi Kleen <ak@kernel.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=12

Re: [RFC v1 13/19] ptwrite uprobes: Support instruction puning

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Stack out-of-bounds read in `pun_install` when copying the original instruction.
- [High] Uninstalling a punned uprobe fails and permanently corrupts process memory if the user modifies the instruction's subsequent bytes.
--

commit bfefc3cb6738d81a0fc8093e66776a8584e4deed
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Support instruction puning

The previous ptwrite instrumentation only worked on 5 byte+ nops because
it needs to patch in a 5 byte branch. But the instruction may be shorter
and the kernel cannot prove that nobody jumps to the next instruction.

[ ... ]

> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index 20557423690f9..806e40f7b0ab3 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c
> @@ -1866,10 +2001,210 @@ static int ptwrite_text_poke(struct arch_uprobe *auprobe,
>  	return err;
>  }
>  
> +/*
> + * Replace an aligned five-byte NOP run with a JMP in one eight-byte store.
> + * The trailing three bytes are read from the existing text. We assume
> + * nobody else is changing it. This is covered by the Intel/AMD "aligned store"
> + * cross modifying guarantee.
> + */

[ ... ]

> +static int pun_install(struct arch_uprobe *auprobe,
> +			       struct vm_area_struct *vma, unsigned long vaddr,
> +			       const u8 *orig)
> +{
> +	struct mm_struct *mm = vma->vm_mm;
> +	struct uprobe_ptwrite_page *ptw;
> +	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
> +	struct uprobes_state *state = &mm->uprobes_state;
> +	struct write_opcode_ctx ctx = {
> +		.base = vaddr,
> +		.expect = EXPECT_BYTE,
> +		.expect_byte = orig[0],
> +	};
> +	unsigned long t, page_base, block_off, stub_addr;
> +	s64 site_delta;
> +	s32 jump_rel, disp32, orig_rel;
> +	u8 site_len;
> +	bool found = false;

[ ... ]

> +	/*
> +	 * A NOP fallback needs a synthetic rel32 at the site, so it uses
> +	 * the full five-byte poke and restore path rather than punning.
> +	 */
> +	site_len = nop_fallback ? 5 : ptw_a->len;
> +	ptw->index[ptw->nblocks].off = block_off;
> +	ptw->index[ptw->nblocks].len = ptw_a->stub_len;
> +	ptw->index[ptw->nblocks].ft_off = ptw_a->ft_off;
> +	ptw->index[ptw->nblocks].pun = !nop_fallback;
> +	ptw->index[ptw->nblocks].orig0 = orig[0];
> +	ptw->index[ptw->nblocks].site_len = site_len;
> +	ptw->index[ptw->nblocks].site_off = (s32)site_delta;
> +	memcpy(ptw->index[ptw->nblocks].site_insn, orig, site_len);

[Severity: High]
Does this memcpy() read out-of-bounds? 

The original instruction is passed down via the orig parameter from
arch_uprobe_install_ptwrite(). In arch_uprobe_install_ptwrite(), orig is
declared as a 5-byte array on the stack.

However, site_len here can be up to 15 bytes (MAX_UINSN_BYTES) for generic
instructions when punning is active. If site_len is greater than 5, this will
copy uninitialized kernel stack memory into the permanent site_insn array.

Later, when attempting to reinstall the uprobe, the memcmp() against the true
original instruction will fail due to the garbage bytes. This breaks the
collision check and could cause a trampoline slot leak on every reinstall.

> +	smp_store_release(&ptw->nblocks, ptw->nblocks + 1);

[ ... ]

> @@ -1937,15 +2301,63 @@ int arch_uprobe_uninstall_ptwrite(struct arch_uprobe *auprobe,
>  		struct vm_area_struct *vma, unsigned long vaddr)
>  {
>  	struct mm_struct *mm = vma->vm_mm;
> +	struct uprobe_ptwrite_arch *ptw_a = &auprobe->ptwrite;
> +	struct uprobes_state *state = &mm->uprobes_state;
>  	u8 cur[5];
> +	int b;
> +	struct write_opcode_ctx ctx = {
> +		.base = vaddr,
> +		.expect = EXPECT_BYTE,
> +		.expect_byte = 0xe9,
> +	};
>  
>  	mmap_assert_write_locked(mm);
> -	if (copy_from_vaddr(mm, vaddr, cur, sizeof(cur)) ||
> -	    !ptwrite_is_installed(mm, vaddr, cur))
> -		return;
> +	{
> +		struct uprobe_ptwrite_page *ptw;
> +		struct uprobe_ptwrite_page *fpw = NULL;
> +		s32 rel;
> +		s64 target;
> +		unsigned long page_base, boff;
> +		int ret;
> +
> +		ret = copy_from_vaddr(mm, vaddr, cur, sizeof(cur));
> +		if (ret)
> +			return ret;
> +		if (!ptwrite_is_installed(mm, vaddr, cur))
> +			return 0;

[Severity: High]
Does this validation incorrectly abandon punned uprobes?

If a punned uprobe is installed on an instruction with a relative offset,
bytes 1-4 are mutable user code. The user application (e.g., a JIT compiler
or dynamic linker) might legitimately update the relative offset in these
subsequent bytes.

When the uprobe is uninstalled, ptwrite_is_installed() strictly requires the
JMP target to still point to the trampoline. Since the user modified the
relative displacement, ptwrite_is_installed() returns false.

This assumes the probe was entirely overwritten and aborts the uninstall
prematurely, returning 0. As a result, the injected 0xe9 (JMP) opcode is left
permanently in the user's execution stream, which can corrupt the instruction
stream and cause crashes or erratic behavior.

> +
> +		memcpy(&rel, cur + 1, sizeof(rel));
> +		target = (s64)vaddr + 5 + (s64)rel;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=13

Re: [RFC v1 14/19] ptwrite uprobes: Use atomic patching for multinop sites

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [High] Structurally unsafe patching of multinop sequences in user space due to thread preemption.
- [High] Unaligned multinops silently fall back to the unsafe INT3 patching method, violating the patch's safety guarantees.
- [Medium] Lockless Read-Modify-Write (RMW) of user memory resulting in a TOCTOU race and lost updates.

Pre-existing issues:
- [High] O(N) sequential synchronize_rcu() calls inside an RCU teardown loop causing severe process exit delays.
--

commit 5a4cb6048cbb1f8b37fe6998acf1c5e1ac8b2670
Author: Andi Kleen <ak@kernel.org>

    ptwrite uprobes: Use atomic patching for multinop sites

This commit modifies the patching mechanism for 5-byte NOP sequences
(multinop) in ptwrite uprobes. It enforces 8-byte alignment for these sites
and replaces the earlier INT3 patching method with an atomic 8-byte
read-modify-write store.

> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index 806e40f7b0ab3..8d9dadc2b1fc6 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c

[ ... ]

> @@ -2224,6 +2232,9 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
>  	ret = copy_from_vaddr(mm, vaddr, orig, sizeof(orig));
>  	if (ret)
>  		return ret;
> +	if (ptwrite_site_is_multinop(orig, ptw_a->allow_nop_run) &&
> +	    (vaddr & 7))
> +		return pun_install(auprobe, vma, vaddr, orig);

[Severity: High]
Does this silently apply the unsafe INT3 patching method to unaligned NOP
sequences?

The commit message states that the earlier multinop patching is unsafe and
enforces 8-byte alignment. However, when an unaligned 5-byte NOP sequence
is requested (vaddr & 7 is true), the code delegates to pun_install().

Looking at pun_install(), this triggers the fallback path:

arch/x86/kernel/uprobes.c:pun_install() {
    ...
	bool nop_fallback = ptwrite_site_is_multinop(orig,
						     ptw_a->allow_nop_run) &&
			    (vaddr & 7);
    ...
	if (nop_fallback)
		ret = ptwrite_text_poke(auprobe, vma, vaddr, stub_addr);
    ...
}

Will this expose the traced process to the same user-space crash issues
that this patch intends to fix?

>  	if (ptwrite_is_installed(mm, vaddr, orig))
>  		return 0;

[ ... ]

> @@ -2249,9 +2260,13 @@ int arch_uprobe_install_ptwrite(struct arch_uprobe *auprobe,
>  			continue;
>  		if (!__in_uprobe_ptwrite(mm, ptw->vaddr))
>  			continue;
> -		ret = ptwrite_text_poke(auprobe, vma, vaddr,
> -					ptw->vaddr + ptw->index[b].off);
> -		goto out;
> +		if (ptwrite_site_is_multinop(orig, ptw_a->allow_nop_run))
> +			ret = ptwrite_multinop_text_poke(auprobe, vma, vaddr,
> +							ptw->vaddr + ptw->index[b].off);

[Severity: High]
Is this structural patching safe against thread preemption?

If a user thread executes the first byte of a 5-byte NOP sequence (such as
one generated by -fpatchable-function-entry) and is preempted, the kernel
will patch the sequence with a 5-byte JMP.

When the thread resumes at vaddr + 1, won't it execute the relative offset
bytes of the JMP as garbage instructions and crash with SIGILL or SIGSEGV?
While an atomic store prevents fetch anomalies, does it prevent a preempted
thread from resuming execution in the middle of the patched instructions?

[Severity: Medium]
Can this lockless Read-Modify-Write result in a TOCTOU race and lost
updates?

Looking at ptwrite_multinop_text_poke():

arch/x86/kernel/uprobes.c:ptwrite_multinop_text_poke() {
    ...
	err = copy_from_vaddr(mm, vaddr, patch, sizeof(patch));
	if (err)
		return err;
	patch[0] = 0xe9;
	memcpy(&patch[1], &rel, sizeof(rel));
	err = uprobe_write(auprobe, vma, vaddr, patch, sizeof(patch),
			   verify_insn, true, false, &ctx);
    ...
}

If a user process concurrently modifies the trailing bytes (bytes 5-7) of
the 8-byte aligned word between the copy_from_vaddr() read and the
uprobe_write(), won't those modifications be silently overwritten and lost?

[Severity: High]
This is a pre-existing issue, but does the process cleanup path force an
extremely slow O(N) grace-period wait?

Looking at arch_uprobe_clear_state():

arch/x86/kernel/uprobes.c:arch_uprobe_clear_state() {
    ...
	hlist_for_each_entry_safe(ptw, n, &state->head_ptwrite, node) {
		hlist_del_rcu(&ptw->node);
		synchronize_rcu();
		__free_page(ptw->page);
		kfree(ptw);
	}
    ...
}

If a traced process has many ptwrite uprobes and exits, it will block
sequentially for an RCU grace period for every single page. Could this
cause severe process exit delays or hung task timeouts?

> +		else
> +			ret = ptwrite_text_poke(auprobe, vma, vaddr,
> +						ptw->vaddr + ptw->index[b].off);
> +		return ret;
>  	}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=14

Re: [RFC v1 15/19] ptwrite uprobes: Add a tutorial and overview documentation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Low] Missing blank line before a bullet list in reStructuredText.
- [Low] Typos in the 'Options in this case' list.
- [Low] Formatting inconsistency with double backticks in indented block.
--

commit e96dac3fdefb27a2867c9e268822b99d3e4c7aa7
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Add a tutorial and overview documentation

This patch adds a new documentation file explaining how to use ptwrite
uprobes for hardware tracing. It covers performance benefits, requirements,
and provides examples for configuring probes via tracefs and perf probe.

> diff --git a/Documentation/trace/ptwrite-uprobes.rst b/Documentation/trace/ptwrite-uprobes.rst
> new file mode 100644
> index 0000000000000..79ab35824b93c
> --- /dev/null
> +++ b/Documentation/trace/ptwrite-uprobes.rst

[ ... ]

> +Punning is a probabilistic method that depends on the existing
> +instruction bytes and the placement of the executable in memory.
> +It has a high chance of success on PIE/PIC binaries, but tends
> +to work poorly on non PIE main executables.
> +
> +When punning is not possible the probe is rejected at install
> +time. Options in this case:
> +- Move the probe site to a different instruction which may work.

[Severity: Low]
Does this list render correctly in Sphinx without a blank line before it?
In reStructuredText, bullet lists generally must be preceded by a blank
line, otherwise the list items may get merged into the preceding paragraph
or trigger build warnings.

> +- Rebuild with -fPIE if it's a main problem not using PIE.
> +- Enable or disable /proc/sys/kernel/randomize_va_space. If the
> +  randomization is enabled it may also just work on a rerun of
> +  the program.
> +- Fall back to a classic uprobes

[Severity: Low]
There are a couple of minor typos in this section. Could "main problem" 
be corrected to "main program", and "a classic uprobes" be changed to 
"a classic uprobe" or "classic uprobes"?

[ ... ]

> +If the program itself also executes own ``PTWRITE``, those values mix with the
> +uprobe output in the stream. The decoder uses the header magic to identify
> +uprobe records. Other values are printed as ``manual ptwrite:`` lines (with
> +their IP when ``fup_on_ptw`` is set) and counted in the summary's ``stray``
> +field.
> +
> +To also print the decoded branch stream alongside the records, add
> +``b`` to the itrace options and drop the ``q``
> +
> +    # ``perf script --itrace=web -s uprobe-ptwrite-decode.py -i perf.data``

[Severity: Low]
Are the double backticks intended here? Because this line is already indented
as a literal block, the double backticks will likely be rendered verbatim in
the output instead of being parsed as inline literal formatting.

> +
> +Each decoded branch prints as a ``branch:`` line (from => to, with
> +symbols where resolvable), interleaved with the probe records and any

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=15

Re: [RFC v1 16/19] ptwrite uprobes / perf tools pt: Improve FUP error handling for ptwrite

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Incorrect ptwrite IP reporting and failure to prevent trace abort in full instruction tracing mode.
- [Medium] Bypassing `intel_pt_fup_event()` causes incomplete manual state reconstruction, leaking instruction metadata and asynchronous FUP flags.
--

commit f901af3d0d6f6651e120a5aaaf5502b7aa3f6612
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes / perf tools pt: Improve FUP error handling for ptwrite

Special case the decoder to handle ptwrite FUPs targeting unmapped code
like uprobe stubs. When the decoder can't resolve the instruction page,
report the ptwrite with its IP instead of erroring out and failing the walk.

> diff --git a/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c b/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
> index e733f6b1f7ac5..bd31d65dbe035 100644
> --- a/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
> +++ b/tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
> @@ -1440,8 +1440,29 @@ static int intel_pt_walk_fup(struct intel_pt_decoder *decoder)
>  			return -EAGAIN;
>  		}
>  		decoder->set_fup_tx_flags = false;
> -		if (err)
> +		if (err) {
> +			/*
> +			 * A ptwrite's FUP can target an address whose
> +			 * instruction cannot be resolved (e.g. the
> +			 * [uprobes-ptwrite] stub is an anonymous special
> +			 * mapping invisible to the machine). The FUP is
> +			 * still the ptwrite's IP: report it rather than
> +			 * failing the whole walk.
> +			 */
> +			if (decoder->set_fup_ptw) {
> +				decoder->set_fup_ptw = false;
> +				decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
> +				decoder->state.type &= ~INTEL_PT_BRANCH;
> +				decoder->state.type |= INTEL_PT_PTW;

[Severity: Medium]
Will bypassing intel_pt_fup_event() here leak instruction metadata and
asynchronous FUP flags?

Because intel_pt_walk_insn() returned an error early, decoder->state.insn_op
and insn_len retain the metadata from the last successfully decoded
instruction. By skipping intel_pt_fup_event() which usually clears these
fields and handles asynchronous flags, could stale instruction op/len and
pending event flags be incorrectly attributed to subsequent events?

> +				decoder->state.flags |= INTEL_PT_FUP_IP;
> +				decoder->state.from_ip = decoder->ip;

[Severity: High]
Does this code assign the wrong IP for the PTWRITE event and fail to prevent
the trace abort it intends to fix?

When intel_pt_walk_insn() encounters an unmapped region like the uprobe stub,
it returns an error without advancing decoder->ip. The true PTWRITE IP from
the hardware FUP packet is passed as the ip parameter (from
decoder->last_ip) to intel_pt_walk_insn().

Assigning decoder->state.from_ip = decoder->ip instead of using the ip
parameter attributes the event to the stuck failure address. Additionally,
because decoder->ip is never advanced out of the unmapped region, subsequent
packets could trigger functions like intel_pt_walk_tip() to call
intel_pt_walk_insn(..., 0) starting from the same stuck decoder->ip, failing
again and aborting the trace.

> +				decoder->state.to_ip = 0;
> +				decoder->state.ptw_payload =
> +							decoder->fup_ptw_payload;
> +				return 0;
> +			}
>  			return err;
> +		}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=16

Re: [RFC v1 17/19] ptwrite uprobes / perf tools probe: Add support of ptwrite probes

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The error code from `strbuf_addf` is silently overwritten, hiding memory allocation or formatting failures.
--

commit 1e80616b3ae889174ea132bb4a1470dd0a9429ba
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes / perf tools probe: Add support of ptwrite probes

This commit adds a --ptwrite option to perf probe to enable trap-free ptwrite
uprobes. These probes write their values into the Intel PT trace stream
instead of entering the kernel.

> diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
> --- a/tools/perf/util/probe-event.c
> +++ b/tools/perf/util/probe-event.c
[ ... ]
> @@ -2260,9 +2262,11 @@ char *synthesize_probe_trace_command(struct probe_trace_event *tev)
>  	if (strbuf_init(&buf, 32) < 0)
>  		return NULL;
>  
> -	if (strbuf_addf(&buf, "%c:%s/%s ", tp->retprobe ? 'r' : 'p',
> -			tev->group, tev->event) < 0)
> -		goto error;
> +	if (tev->ptwrite)
> +		err = strbuf_addf(&buf, "ptw:%s/%s ", tev->group, tev->event);
> +	else
> +		err = strbuf_addf(&buf, "%c:%s/%s ", tp->retprobe ? 'r' : 'p',
> +				  tev->group, tev->event);
>  
>  	if (tev->uprobes)
>  		err = synthesize_uprobe_trace_def(tp, &buf);

[Severity: Medium]
Does this code silently overwrite the error code from strbuf_addf()?

It appears that if memory allocation fails inside the first strbuf_addf(),
the function continues without checking err. The negative error code is
then overwritten by the return value of the subsequent
synthesize_uprobe_trace_def() or synthesize_kprobe_trace_def() call.

Could this lead to a truncated or malformed trace command being synthesized
and sent to the kernel?

> @@ -2274,7 +2278,6 @@ char *synthesize_probe_trace_command(struct probe_trace_event *tev)
>  
>  	if (err >= 0)
>  		ret = strbuf_detach(&buf, NULL);
> -error:
>  	strbuf_release(&buf);
>  	return ret;
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=17

Re: [RFC v1 18/19] ptwrite uprobes / perf tools script: Add ptwrite uprobes decoder

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The script queries the local host's `/sys/kernel/tracing` to map dynamic event IDs, which silently breaks offline and cross-platform analysis of `perf.data` files.
--

commit cc31146d83897ea76d8b39eb7c2282185b7e63ac
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes / perf tools script: Add ptwrite uprobes decoder

Add a decoder for the PTWRITE records generated by ptwrite uprobes.
This runs as a python script in perf script.

> diff --git a/tools/perf/scripts/python/uprobe-ptwrite-decode.py b/tools/perf/scripts/python/uprobe-ptwrite-decode.py
> new file mode 100755
> index 0000000000000..22f21cb168e12
> --- /dev/null
> +++ b/tools/perf/scripts/python/uprobe-ptwrite-decode.py
> @@ -0,0 +1,416 @@
[ ... ]
> +TRACEFS = "/sys/kernel/tracing"
[ ... ]
> +def load_events(root=TRACEFS):
> +    """Scan tracefs for (event_id -> (name, [(arg name, type)]))."""
> +    events = {}
> +    try:
> +        groups = os.listdir(root + "/events")

[Severity: Medium]
Does coupling the event decoding logic to the host machine's live tracefs
state break offline cross-platform analysis of perf.data files?

Trace event IDs are dynamically assigned by the kernel and can differ across
machines or reboots. When a perf.data file is analyzed offline on a different
machine, the embedded ptwrite payload event IDs (which were generated on the
recording machine) will be incorrectly mapped to whatever unrelated events
happen to share those IDs in the local host's /sys/kernel/tracing. This can
lead to garbage decoding output.

Could the script be modified to extract event definitions from the perf.data
header instead of silently defaulting to the host's tracefs?

> +    except OSError:
> +        return events
> +    for g in groups:
> +        gdir = root + "/events/" + g
> +        if not os.path.isdir(gdir):
> +            continue

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=18

Re: [RFC v1 19/19] ptwrite uprobes: Add self tests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Missing test scripts and source files cause selftests build failures and unconditional skips.
--

commit cdbf7a2bf57258522e6b6951a74593c1f59e7dd4
Author: Andi Kleen <ak@kernel.org>

ptwrite uprobes: Add self tests

This commit adds selftests for ptwrite uprobes. It tests various cases using 
both the main probes and the perf decoder.

> diff --git a/tools/testing/selftests/uprobes/Makefile b/tools/testing/selftests/uprobes/Makefile
> new file mode 100644
> index 0000000000000..9fc65c1f04c84
> --- /dev/null
> +++ b/tools/testing/selftests/uprobes/Makefile
> @@ -0,0 +1,15 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# ptwrite uprobe selftests (x86-64).
> +ARCH ?= $(shell uname -m 2>/dev/null || echo not)
> +CFLAGS += -O2 -Wall -no-pie
> +
> +TEST_GEN_FILES := ptw_probe
> +TEST_PROGS := run_ptw.sh run_module.sh run_perfprobe.sh run_decode.sh

[Severity: Medium]
Will this break the selftests build during a `make install`? The Makefile
lists run_module.sh and run_perfprobe.sh in TEST_PROGS, but these files do
not appear to be included in this patch.

[ ... ]

> diff --git a/tools/testing/selftests/uprobes/run_decode.sh b/tools/testing/selftests/uprobes/run_decode.sh
> new file mode 100755
> index 0000000000000..b427442041aa9
> --- /dev/null
> +++ b/tools/testing/selftests/uprobes/run_decode.sh
> @@ -0,0 +1,177 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# run_decode.sh - ptwrite decoder selftests.
> +# Exercises the decoder CLI and PT/perf integration.
> +# Root + tracefs + perf + gcc + a PTWRITE-capable CPU required.
> +set -u
> +DIR=$(dirname "$(readlink -f "$0")")
> +SRC=${1:-"$DIR/manual_ptw.c"}

[Severity: Medium]
Does this cause the test to unconditionally skip? The SRC variable defaults
to manual_ptw.c, but this source file seems to be missing from the commit. 
If the file is added in a subsequent version, should it also be included in
TEST_FILES in the Makefile to ensure it is installed correctly during
`make install`?

[ ... ]

> +if ! gcc -O2 -no-pie -o "$BIN" "$SRC" 2>/dev/null; then
> +	echo "1..0 # SKIP test program build failed"
> +	exit 0
> +fi
> +
> +TV=$(objdump -d "$BIN" 2>/dev/null |
> +	awk '/^[0-9a-f]+ <target>:/{print $1;exit}' | tr -d ':')

[Severity: Medium]
Is manual_ptw.c missing from the commit? This logic expects the compiled
test binary to contain a `target` function, which further suggests that the
omitted C file is strictly required for this test to function.

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831150651.1134594-1-ak@kernel.org?part=19

Re: [RFC v1 01/19] uprobes: guard trace cleanup against error pointers

[Masami Hiramatsu]

On Mon, 31 Aug 2026 08:04:37 -0700
Andi Kleen <ak@kernel.org> wrote:

> Sashiko pointed out the some of the scope cleanups for free_uprobe
> could get an error pointer. Handle this case in free_uprobe
> to prevent a crash.
> 
> On the other hand the macro doesn't need the guard because
> free_uprobe itself already does the check.
> 
> Assisted-by: omp:gpt-5.6-luna sashiko
> Signed-off-by: Andi Kleen <ak@kernel.org>

This looks good to me. Let me pick this to probes/fixes independently.

Thanks!

> ---
>  kernel/trace/trace_uprobe.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
> index 861d857adadb..22cc3c8181b8 100644
> --- a/kernel/trace/trace_uprobe.c
> +++ b/kernel/trace/trace_uprobe.c
> @@ -368,7 +368,7 @@ alloc_trace_uprobe(const char *group, const char *event, int nargs, bool is_ret)
>  
>  static void free_trace_uprobe(struct trace_uprobe *tu)
>  {
> -	if (!tu)
> +	if (IS_ERR_OR_NULL(tu))
>  		return;
>  
>  	path_put(&tu->path);
> @@ -533,7 +533,7 @@ static int register_trace_uprobe(struct trace_uprobe *tu)
>  	return ret;
>  }
>  
> -DEFINE_FREE(free_trace_uprobe, struct trace_uprobe *, if (_T) free_trace_uprobe(_T))
> +DEFINE_FREE(free_trace_uprobe, struct trace_uprobe *, free_trace_uprobe(_T))
>  
>  /*
>   * Argument syntax:
> -- 
> 2.54.0
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [RFC v1 05/19] ptwrite uprobes: Add minimal low level support for x86

[Lorenzo Stoakes (ARM)]

NAK.

This is broken as described in [0] and causes use-after-frees.

[0]:https://lore.kernel.org/all/aphJ7olxr-_VhDKt@gremlin/

On Mon, Aug 31, 2026 at 08:04:41AM -0700, Andi Kleen wrote:
> Add more data structures and the x86 machinery to generate the PTWRITE
> instructions for a ptwrite uprobe. The probe executes PTWRITEs and then
> jumps back to the original code. In this variant only patching
> 5 byte nops is supported.
>
> The instructions are pre-generated to templates and then patched when
> setting up the final user page.
>
> The patching code uses 3 phase patching similar to int3_update.
>
> The ptwrite stub emits a header with a magic value and the number of
> arguments, and then the actual probed values.
>
> There is no separate config option for ptwrite uprobes, it is just tied
> to the main uprobes config.
>
> Some limitations in the current implementation:
> - The probed 5 byte area cannot cross a page.
> - The allocated stubs in the user program are only freed on exit.
>
> Assisted-by: omp:gpt-5.6-luna

I suggest looking into a model more suited to complex kernel
development. Googling it, Luna is described thusly:

'GPT-5.6 Luna is OpenAI's fastest and most budget-friendly AI model tier,
built specifically for high-volume, latency-sensitive tasks'

Which doesn't strike me as ideal for this kind of work, especially when you
are submitting things to the mailing list and asking people to dedicate
their own time (and better models) to assessing it.

> Signed-off-by: Andi Kleen <ak@kernel.org>
> ---
>  arch/x86/include/asm/uprobes.h |  24 ++
>  arch/x86/kernel/uprobes.c      | 623 +++++++++++++++++++++++++++++++++
>  2 files changed, 647 insertions(+)

(That's a huge diffstat for one patch and your cover letter is missing a
full diffstat also...)

> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index 65a2de82ecd2..df652c56414b 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c
> @@ -15,11 +15,15 @@
>  #include <linux/syscalls.h>

...

> +static struct uprobe_ptwrite_page *
> +create_uprobe_ptwrite_page(struct mm_struct *mm, unsigned long vaddr)
> +{
> +	struct uprobe_ptwrite_page *ptw;
> +	struct vm_area_struct *vma;
> +	unsigned long area;
> +
> +	area = find_ptwrite_page_area(mm, vaddr);
> +	if (IS_ERR_VALUE(area))
> +		return NULL;
> +
> +	mmap_assert_write_locked(mm);
> +
> +	ptw = kzalloc_obj(*ptw);
> +	if (!ptw)
> +		return NULL;
> +
> +	ptw->page = alloc_page(GFP_HIGHUSER | __GFP_ZERO);
> +	if (!ptw->page) {
> +		kfree(ptw);
> +		return NULL;
> +	}
> +	ptw->vaddr = area;
> +
> +	vma = _install_special_mapping(mm, area, PAGE_SIZE,
> +			VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
> +			&ptwrite_mapping);

You need to trampoline this (as the existing code does...) to avoid the
issue described in [0].

--
Cheers, Lorenzo