* [PATCH v3 1/10] X86 instruction analysis: Move Macro W to insn.h
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
@ 2010-05-06 18:01 ` Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 2/10] User Space Breakpoint Assistance Layer Srikar Dronamraju
` (10 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:01 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, Andrew Morton, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, LKML, Paul E. McKenney
move_W_to_insn.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Move Macro W to asm/insn.h
Macro W used to know if the instructions are valid for
user-space/kernel space. This macro is used by kprobes and
user_bkpt. (i.e user space breakpoint assistance layer.) So moving it
to a common header file asm/insn.h.
TODO: replace macro W with bits in inat table. (suggested by Masami Hiramatsu)
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
arch/x86/include/asm/insn.h | 7 +++++++
arch/x86/kernel/kprobes.c | 7 -------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/insn.h b/arch/x86/include/asm/insn.h
index 96c2e0a..8586820 100644
--- a/arch/x86/include/asm/insn.h
+++ b/arch/x86/include/asm/insn.h
@@ -23,6 +23,13 @@
/* insn_attr_t is defined in inat.h */
#include <asm/inat.h>
+#define W(row, b0, b1, b2, b3, b4, b5, b6, b7, b8, b9, ba, bb, bc, bd, be, bf)\
+ (((b0##UL << 0x0)|(b1##UL << 0x1)|(b2##UL << 0x2)|(b3##UL << 0x3) | \
+ (b4##UL << 0x4)|(b5##UL << 0x5)|(b6##UL << 0x6)|(b7##UL << 0x7) | \
+ (b8##UL << 0x8)|(b9##UL << 0x9)|(ba##UL << 0xa)|(bb##UL << 0xb) | \
+ (bc##UL << 0xc)|(bd##UL << 0xd)|(be##UL << 0xe)|(bf##UL << 0xf)) \
+ << (row % 32))
+
struct insn_field {
union {
insn_value_t value;
diff --git a/arch/x86/kernel/kprobes.c b/arch/x86/kernel/kprobes.c
index b43bbae..4379b40 100644
--- a/arch/x86/kernel/kprobes.c
+++ b/arch/x86/kernel/kprobes.c
@@ -66,12 +66,6 @@ DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
#define stack_addr(regs) ((unsigned long *)kernel_stack_pointer(regs))
-#define W(row, b0, b1, b2, b3, b4, b5, b6, b7, b8, b9, ba, bb, bc, bd, be, bf)\
- (((b0##UL << 0x0)|(b1##UL << 0x1)|(b2##UL << 0x2)|(b3##UL << 0x3) | \
- (b4##UL << 0x4)|(b5##UL << 0x5)|(b6##UL << 0x6)|(b7##UL << 0x7) | \
- (b8##UL << 0x8)|(b9##UL << 0x9)|(ba##UL << 0xa)|(bb##UL << 0xb) | \
- (bc##UL << 0xc)|(bd##UL << 0xd)|(be##UL << 0xe)|(bf##UL << 0xf)) \
- << (row % 32))
/*
* Undefined/reserved opcodes, conditional jump, Opcode Extension
* Groups, and some special opcodes can not boost.
@@ -98,7 +92,6 @@ static const u32 twobyte_is_boostable[256 / 32] = {
/* ----------------------------------------------- */
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
};
-#undef W
struct kretprobe_blackpoint kretprobe_blacklist[] = {
{"__switch_to", }, /* This function switches only current task, but
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 2/10] User Space Breakpoint Assistance Layer
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
2010-05-06 18:01 ` [PATCH v3 1/10] X86 instruction analysis: Move Macro W to insn.h Srikar Dronamraju
@ 2010-05-06 18:02 ` Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 3/10] x86 support for User space breakpoint assistance Srikar Dronamraju
` (9 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:02 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
user_bkpt_core.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
User Space Breakpoint Assistance Layer (USER_BKPT)
Changelog from v2: (addressing comments from Oleg)
* Use access_process_vm instead of replace_page based
background page replacement.
Changelog from v1:
* Use k(un)map_atomic instead of k(un)map.
* Remove BUG_ON.
* Few parameter changes to be more consistent with sparse.
* Added kernel-doc comments whereever necessary.
* Introduce a check to detect if post_xol can sleep.
Currently there is no mechanism in kernel to insert/remove breakpoints.
This patch implements user space breakpoint assistance layer provides
kernel subsystems with architecture independent interface to establish
breakpoints in user applications. This patch provides core
implementation of user_bkpt and also wrappers for architecture dependent
methods.
USER_BKPT currently supports both single stepping inline and execution
out of line strategies. Two different probepoints in the same process
can have two different strategies. It handles pre-processing and
post-processing of singlestep after a breakpoint hit.
Single stepping inline strategy is the traditional method where original
instructions replace the breakpointed instructions on a breakpoint hit.
This method works well with single threaded applications. However its
racy with multithreaded applications.
Execution out of line strategy single steps on a copy of the
instruction. This method works well for both single-threaded and
multithreaded applications.
There could be other strategies like emulating an instruction. However
they are currently not implemented.
Insertion and removal of breakpoints is by "Background page
replacement". i.e make a copy of the page, modify its the contents, set
the pagetable and flush the tlbs. This page uses enhanced replace_page
to cow the page. Modified page is only reflected for the interested
process. Others sharing the page will still see the old copy.
You need to follow this up with the USER_BKPT patch for your
architecture.
Uprobes uses this facility to insert/remove breakpoint.
TODO: Merge user_bkpt layer with uprobes.
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
arch/Kconfig | 13 +
include/linux/user_bkpt.h | 305 ++++++++++++++++++++++++++
kernel/Makefile | 1
kernel/user_bkpt.c | 536 +++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 855 insertions(+), 0 deletions(-)
create mode 100644 include/linux/user_bkpt.h
create mode 100644 kernel/user_bkpt.c
diff --git a/arch/Kconfig b/arch/Kconfig
index e5eb133..116f0ea 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -52,6 +52,16 @@ config OPTPROBES
This option will allow kprobes to optimize breakpoint to
a jump for reducing its overhead.
+config USER_BKPT
+ bool "User-space breakpoint assistance (EXPERIMENTAL)"
+ depends on MODULES
+ depends on HAVE_USER_BKPT
+ help
+ User_bkpt (User-space breakpoint assistance layer) enables
+ kernel subsystems to establish breakpoints in user applications.
+ This service is used by components such as uprobes.
+ If in doubt, say "N".
+
config HAVE_EFFICIENT_UNALIGNED_ACCESS
bool
help
@@ -85,6 +95,9 @@ config USER_RETURN_NOTIFIER
Provide a kernel-internal notification when a cpu is about to
switch to user mode.
+config HAVE_USER_BKPT
+ def_bool n
+
config HAVE_IOREMAP_PROT
bool
diff --git a/include/linux/user_bkpt.h b/include/linux/user_bkpt.h
new file mode 100644
index 0000000..4773541
--- /dev/null
+++ b/include/linux/user_bkpt.h
@@ -0,0 +1,305 @@
+#ifndef _LINUX_USER_BKPT_H
+#define _LINUX_USER_BKPT_H
+/*
+ * User-space BreakPoint support (user_bkpt)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+#include <asm/user_bkpt.h>
+struct task_struct;
+struct pt_regs;
+
+/**
+ * Strategy hints:
+ *
+ * %USER_BKPT_HNT_INLINE: Specifies that the instruction must
+ * be single-stepped inline. Can be set by the caller of
+ * @arch->analyze_insn() -- e.g., if caller is out of XOL slots --
+ * or by @arch->analyze_insn() if there's no viable XOL strategy
+ * for that instruction. Set in arch->strategies if the architecture
+ * doesn't implement XOL.
+ *
+ * %USER_BKPT_HNT_PERMSL: Specifies that the instruction slot whose
+ * address is @user_bkpt->xol_vaddr is assigned to @user_bkpt for the life of
+ * the process. Can be used by @arch->analyze_insn() to simplify
+ * XOL in some cases. Ignored in @arch->strategies.
+ *
+ * %USER_BKPT_HNT_TSKINFO: Set in @arch->strategies if the architecture's
+ * XOL handling requires the preservation of special
+ * task-specific info between the calls to @arch->pre_xol()
+ * and @arch->post_xol(). (E.g., XOL of x86_64 rip-relative
+ * instructions uses a scratch register, whose value is saved
+ * by pre_xol() and restored by post_xol().) The caller
+ * of @arch->analyze_insn() should set %USER_BKPT_HNT_TSKINFO in
+ * @user_bkpt->strategy if it's set in @arch->strategies and the caller
+ * can maintain a @user_bkpt_task_arch_info object for each probed task.
+ * @arch->analyze_insn() should leave this flag set in @user_bkpt->strategy
+ * if it needs to use the per-task @user_bkpt_task_arch_info object.
+ */
+#define USER_BKPT_HNT_INLINE 0x1 /* Single-step this insn inline. */
+#define USER_BKPT_HNT_TSKINFO 0x2 /* XOL requires user_bkpt_task_arch_info */
+#define USER_BKPT_HNT_PERMSL 0x4 /* XOL slot assignment is permanent */
+
+#define USER_BKPT_HNT_MASK 0x7
+
+/**
+ * struct user_bkpt - user-space breakpoint/probepoint
+ *
+ * @vaddr: virtual address of probepoint
+ * @xol_vaddr: virtual address of XOL slot assigned to this probepoint
+ * @opcode: copy of opcode at @vaddr
+ * @insn: typically a copy of the instruction at @vaddr. More
+ * precisely, this is the instruction (stream) that will be
+ * executed in place of the original instruction.
+ * @strategy: hints about how this instruction will be executed
+ * @fixups: set of fixups to be executed by @arch->post_xol()
+ * @arch_info: architecture-specific info about this probepoint
+ */
+struct user_bkpt {
+ unsigned long vaddr;
+ unsigned long xol_vaddr;
+ user_bkpt_opcode_t opcode;
+ u8 insn[USER_BKPT_XOL_SLOT_BYTES];
+ u16 strategy;
+ u16 fixups;
+ struct bkpt_arch_info arch_info;
+};
+
+/* Post-execution fixups. Some architectures may define others. */
+
+/* No fixup needed */
+#define USER_BKPT_FIX_NONE 0x0
+/* Adjust IP back to vicinity of actual insn */
+#define USER_BKPT_FIX_IP 0x1
+/* Adjust the return address of a call insn */
+#define USER_BKPT_FIX_CALL 0x2
+/* Might sleep while doing Fixup */
+#define USER_BKPT_FIX_SLEEPY 0x4
+
+#ifndef USER_BKPT_FIX_DEFAULT
+#define USER_BKPT_FIX_DEFAULT USER_BKPT_FIX_IP
+#endif
+
+#ifdef CONFIG_USER_BKPT
+extern int user_bkpt_init(u16 *strategies);
+extern int user_bkpt_insert_bkpt(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt);
+extern unsigned long user_bkpt_get_bkpt_addr(struct pt_regs *regs);
+extern int user_bkpt_pre_sstep(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs);
+extern int user_bkpt_post_sstep(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs);
+extern int user_bkpt_cancel_xol(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt);
+extern int user_bkpt_remove_bkpt(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt);
+extern int user_bkpt_validate_insn_addr(struct task_struct *tsk,
+ unsigned long vaddr);
+extern void user_bkpt_set_ip(struct pt_regs *regs, unsigned long vaddr);
+extern bool user_bkpt_resume_can_sleep(struct user_bkpt *user_bkpt);
+#else /* CONFIG_USER_BKPT */
+static inline int user_bkpt_init(u16 *strategies)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_insert_bkpt(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt)
+{
+ return -ENOSYS;
+}
+static inline unsigned long user_bkpt_get_bkpt_addr(struct pt_regs *regs)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_pre_sstep(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt, struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_post_sstep(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt, struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_cancel_xol(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_remove_bkpt(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt)
+{
+ return -ENOSYS;
+}
+static inline int user_bkpt_validate_insn_addr(struct task_struct *tsk,
+ unsigned long vaddr)
+{
+ return -ENOSYS;
+}
+static inline void user_bkpt_set_ip(struct pt_regs *regs, unsigned long vaddr)
+{
+}
+static inline bool user_bkpt_resume_can_sleep(struct user_bkpt *user_bkpt)
+{
+ return false;
+}
+#endif /* CONFIG_USER_BKPT */
+
+/**
+ * struct user_bkpt_arch_info - architecture-specific parameters and
+ * functions
+ *
+ * Most architectures can use the default versions of @read_opcode(),
+ * @set_bkpt(), @set_orig_insn(), and @is_bkpt_insn(); ia64 is an
+ * exception. All functions (including @validate_address()) can assume
+ * that the caller has verified that the probepoint's virtual address
+ * resides in an executable VM area.
+ *
+ * @bkpt_insn:
+ * The architecture's breakpoint instruction. This is used by
+ * the default versions of @set_bkpt(), @set_orig_insn(), and
+ * @is_bkpt_insn().
+ * @ip_advancement_by_bkpt_insn:
+ * The number of bytes the instruction pointer is advanced by
+ * this architecture's breakpoint instruction. For example, after
+ * the powerpc trap instruction executes, the ip still points to the
+ * breakpoint instruction (ip_advancement_by_bkpt_insn = 0); but the
+ * x86 int3 instruction (1 byte) advances the ip past the int3
+ * (ip_advancement_by_bkpt_insn = 1).
+ * @max_insn_bytes:
+ * The maximum length, in bytes, of an instruction in this
+ * architecture. This must be <= USER_BKPT_XOL_SLOT_BYTES;
+ * @strategies:
+ * Bit-map of %USER_BKPT_HNT_* values recognized by this architecture.
+ * Include %USER_BKPT_HNT_INLINE iff this architecture doesn't support
+ * execution out of line. Include %USER_BKPT_HNT_TSKINFO if
+ * XOL of at least some instructions requires communication of
+ * per-task state between @pre_xol() and @post_xol().
+ * @set_ip:
+ * Set the instruction pointer in @regs to @vaddr.
+ * @validate_address:
+ * Return 0 if @vaddr is a valid instruction address, or a negative
+ * errno (typically -%EINVAL) otherwise. If you don't provide
+ * @validate_address(), any address will be accepted. Caller
+ * guarantees that @vaddr is in an executable VM area. This
+ * function typically just enforces arch-specific instruction
+ * alignment.
+ * @read_opcode:
+ * For task @tsk, read the opcode at @vaddr and store it in
+ * @opcode. Return 0 (success) or a negative errno. Defaults to
+ * @user_bkpt_read_opcode().
+ * @set_bkpt:
+ * For task @tsk, store @bkpt_insn at @user_bkpt->vaddr. Return 0
+ * (success) or a negative errno. Defaults to @user_bkpt_set_bkpt().
+ * @set_orig_insn:
+ * For task @tsk, restore the original opcode (@user_bkpt->opcode) at
+ * @user_bkpt->vaddr. If @check is true, first verify that there's
+ * actually a breakpoint instruction there. Return 0 (success) or
+ * a negative errno. Defaults to @user_bkpt_set_orig_insn().
+ * @is_bkpt_insn:
+ * Return %true if @user_bkpt->opcode is @bkpt_insn. Defaults to
+ * @user_bkpt_is_bkpt_insn(), which just tests (user_bkpt->opcode ==
+ * arch->bkpt_insn).
+ * @analyze_insn:
+ * Analyze @user_bkpt->insn. Return 0 if @user_bkpt->insn is an
+ * instruction you can probe, or a negative errno (typically -%EPERM)
+ * otherwise. The caller sets @user_bkpt->strategy to
+ * %USER_BKPT_HNT_INLINE to suppress XOL for this instruction (e.g.,
+ * because we're out of XOL slots). If the instruction can be probed
+ * but can't be executed out of line, set @user_bkpt->strategy to
+ * %USER_BKPT_HNT_INLINE. Otherwise, determine what sort of
+ * XOL-related fixups @post_xol() (and possibly @pre_xol()) will need
+ * to do for this instruction, and annotate @user_bkpt accordingly.
+ * You may modify @user_bkpt->insn (e.g., the x86_64 port does this
+ * for rip-relative instructions), but if you do so, you should
+ * retain a copy in @user_bkpt->arch_info in case you have to revert
+ * to single-stepping inline (see @cancel_xol()).
+ * @pre_xol:
+ * Called just before executing the instruction associated
+ * with @user_bkpt out of line. @user_bkpt->xol_vaddr is the address
+ * in @tsk's virtual address space where @user_bkpt->insn has been
+ * copied. @pre_xol() should at least set the instruction pointer in
+ * @regs to @user_bkpt->xol_vaddr -- which is what the default,
+ * @user_bkpt_pre_xol(), does. If @user_bkpt->strategy includes the
+ * %USER_BKPT_HNT_TSKINFO flag, then @tskinfo points to a per-task
+ * copy of struct user_bkpt_task_arch_info.
+ * @post_xol:
+ * Called after executing the instruction associated with
+ * @user_bkpt out of line. @post_xol() should perform the fixups
+ * specified in @user_bkpt->fixups, which includes ensuring that the
+ * instruction pointer in @regs points at the next instruction in
+ * the probed instruction stream. @tskinfo is as for @pre_xol().
+ * You must provide this function.
+ * @cancel_xol:
+ * The instruction associated with @user_bkpt cannot be executed
+ * out of line after all. (This can happen when XOL slots
+ * are lazily assigned, and we run out of slots before we
+ * hit this breakpoint. This function should never be called
+ * if @analyze_insn() was previously called for @user_bkpt with a
+ * non-zero value of @user_bkpt->xol_vaddr and with
+ * %USER_BKPT_HNT_PERMSL set in @user_bkpt->strategy.) Adjust
+ * @user_bkpt as needed so it can be single-stepped inline. Omit this
+ * function if you don't need it.
+ */
+
+struct user_bkpt_arch_info {
+ user_bkpt_opcode_t bkpt_insn;
+ u8 ip_advancement_by_bkpt_insn;
+ u8 max_insn_bytes;
+ u16 strategies;
+ void (*set_ip)(struct pt_regs *regs, unsigned long vaddr);
+ int (*validate_address)(struct task_struct *tsk, unsigned long vaddr);
+ int (*read_opcode)(struct task_struct *tsk, unsigned long vaddr,
+ user_bkpt_opcode_t *opcode);
+ int (*set_bkpt)(struct task_struct *tsk, struct user_bkpt *user_bkpt);
+ int (*set_orig_insn)(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt, bool check);
+ bool (*is_bkpt_insn)(struct user_bkpt *user_bkpt);
+ int (*analyze_insn)(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt);
+ int (*pre_xol)(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs);
+ int (*post_xol)(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs);
+ void (*cancel_xol)(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt);
+};
+
+/* Unexported functions & macros for use by arch-specific code */
+#define user_bkpt_opcode_sz (sizeof(user_bkpt_opcode_t))
+extern unsigned long user_bkpt_read_vm(struct task_struct *tsk,
+ void __user *vaddr, void *kbuf,
+ unsigned long nbytes);
+extern unsigned long user_bkpt_write_data(struct task_struct *tsk,
+ void __user *vaddr, const void *kbuf,
+ unsigned long nbytes);
+
+extern struct user_bkpt_arch_info user_bkpt_arch_info;
+
+#endif /* _LINUX_USER_BKPT_H */
diff --git a/kernel/Makefile b/kernel/Makefile
index a987aa1..36a35b0 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -105,6 +105,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_event.o
obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
obj-$(CONFIG_USER_RETURN_NOTIFIER) += user-return-notifier.o
obj-$(CONFIG_PADATA) += padata.o
+obj-$(CONFIG_USER_BKPT) += user_bkpt.o
ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
# According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
diff --git a/kernel/user_bkpt.c b/kernel/user_bkpt.c
new file mode 100644
index 0000000..b61647f
--- /dev/null
+++ b/kernel/user_bkpt.c
@@ -0,0 +1,536 @@
+/*
+ * User-space BreakPoint support (user_bkpt)
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ * Ananth N Mavinakayanahalli
+ */
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/ptrace.h>
+#include <linux/mm.h>
+#include <linux/user_bkpt.h>
+#include <linux/uaccess.h>
+#include <linux/highmem.h>
+#include <linux/pagemap.h>
+
+static struct user_bkpt_arch_info *arch = &user_bkpt_arch_info;
+
+static bool uses_xol_strategy(u16 strategy)
+{
+ return !(strategy & USER_BKPT_HNT_INLINE);
+}
+
+static bool validate_strategy(u16 strategy, u16 valid_bits)
+{
+ return ((strategy & (~valid_bits)) == 0);
+}
+
+/**
+ * user_bkpt_init - initialize the user_bkpt data structures
+ * @strategies indicates which breakpoint-related strategies are
+ * supported by the client:
+ * %USER_BKPT_HNT_INLINE: Client supports only single-stepping inline.
+ * Otherwise client must provide an instruction slot
+ * (USER_BKPT_XOL_SLOT_BYTES bytes) in the probed process's address
+ * space for each instruction to be executed out of line.
+ * %USER_BKPT_HNT_TSKINFO: Client can provide and maintain one
+ * @user_bkpt_task_arch_info object for each probed task. (Failure to
+ * support this will prevent XOL of rip-relative instructions on
+ * x86_64, at least.)
+ * Upon return, @strategies is updated to reflect those strategies
+ * required by this particular architecture's implementation of user_bkpt:
+ * %USER_BKPT_HNT_INLINE: Architecture or client supports only
+ * single-stepping inline.
+ * %USER_BKPT_HNT_TSKINFO: Architecture uses @user_bkpt_task_arch_info,
+ * and will expect it to be passed to @user_bkpt_pre_sstep() and
+ * @user_bkpt_post_sstep() as needed (see @user_bkpt_insert_bkpt()).
+ * Possible errors:
+ * -%ENOSYS: user_bkpt not supported for this architecture.
+ * -%EINVAL: unrecognized flags in @strategies
+ */
+int user_bkpt_init(u16 *strategies)
+{
+ u16 inline_bit, tskinfo_bit;
+ u16 client_strategies = *strategies;
+
+ if (!validate_strategy(client_strategies,
+ USER_BKPT_HNT_INLINE | USER_BKPT_HNT_TSKINFO))
+ return -EINVAL;
+
+ inline_bit = (client_strategies | arch->strategies) &
+ USER_BKPT_HNT_INLINE;
+ tskinfo_bit = (client_strategies & arch->strategies) &
+ USER_BKPT_HNT_TSKINFO;
+ *strategies = (inline_bit | tskinfo_bit);
+ return 0;
+}
+
+/**
+ * user_bkpt_read_vm - Read @nbytes at @vaddr from @tsk into @kbuf.
+ * @tsk: The probed task
+ * @vaddr: Source address, in user space to be read.
+ * @kbuf: Destination address, in kernel space.
+ *
+ * Context: This function may sleep.
+ *
+ * Returns number of bytes that could be copied.
+ */
+unsigned long user_bkpt_read_vm(struct task_struct *tsk, void __user *vaddr,
+ void *kbuf, unsigned long nbytes)
+{
+ if (tsk == current) {
+ unsigned long nleft = copy_from_user(kbuf, vaddr, nbytes);
+ return nbytes - nleft;
+ } else
+ return access_process_vm(tsk, (unsigned long) vaddr, kbuf,
+ nbytes, 0);
+}
+
+/**
+ * user_bkpt_write_data - Write @nbytes from @kbuf at @vaddr in @tsk.
+ * Can be used to write to stack or data VM areas, but not instructions.
+ * Not exported, but available for use by arch-specific user_bkpt code.
+ * @tsk: The probed task
+ * @vaddr: Destination address, in user space.
+ * @kbuf: Source address, in kernel space to be read.
+ *
+ * Context: This function may sleep.
+ *
+ * Return number of bytes written.
+ */
+unsigned long user_bkpt_write_data(struct task_struct *tsk,
+ void __user *vaddr, const void *kbuf,
+ unsigned long nbytes)
+{
+ unsigned long nleft;
+
+ if (tsk == current) {
+ nleft = copy_to_user(vaddr, kbuf, nbytes);
+ return nbytes - nleft;
+ } else
+ return access_process_vm(tsk, (unsigned long) vaddr,
+ (void *) kbuf, nbytes, 1);
+}
+
+/*
+ * Given an address, get its pte. Very similar to get_locked_pte except
+ * there is no spinlock involved.
+ */
+static pte_t *get_pte(struct mm_struct *mm, unsigned long vaddr)
+{
+ pgd_t *pgd;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pgd = pgd_offset(mm, vaddr);
+ pud = pud_alloc(mm, pgd, vaddr);
+ if (!pud)
+ return NULL;
+ pmd = pmd_alloc(mm, pud, vaddr);
+ if (!pmd)
+ return NULL;
+ pte = pte_alloc_map(mm, pmd, vaddr);
+ return pte;
+}
+
+static int write_opcode(struct task_struct *tsk, unsigned long vaddr,
+ user_bkpt_opcode_t opcode)
+{
+ int ret;
+
+ if (!tsk)
+ return -EINVAL;
+
+ ret = access_process_vm(tsk, vaddr, &opcode, user_bkpt_opcode_sz, 1);
+ return (ret == user_bkpt_opcode_sz ? 0 : -EFAULT);
+}
+
+/* Default implementation of arch->read_opcode */
+static int read_opcode(struct task_struct *tsk, unsigned long vaddr,
+ user_bkpt_opcode_t *opcode)
+{
+ unsigned long bytes_read;
+
+ bytes_read = user_bkpt_read_vm(tsk, (void __user *) vaddr, opcode,
+ user_bkpt_opcode_sz);
+ return (bytes_read == user_bkpt_opcode_sz ? 0 : -EFAULT);
+}
+
+/* Default implementation of arch->set_bkpt */
+static int set_bkpt(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ return write_opcode(tsk, user_bkpt->vaddr, arch->bkpt_insn);
+}
+
+/* Default implementation of arch->set_orig_insn */
+static int set_orig_insn(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt, bool check)
+{
+ if (check) {
+ user_bkpt_opcode_t opcode;
+ int result = arch->read_opcode(tsk, user_bkpt->vaddr,
+ &opcode);
+ if (result)
+ return result;
+ if (opcode != arch->bkpt_insn)
+ return -EINVAL;
+ }
+ return write_opcode(tsk, user_bkpt->vaddr, user_bkpt->opcode);
+}
+
+/* Return 0 if vaddr is in an executable VM area, or -EINVAL otherwise. */
+static inline int check_vma(struct task_struct *tsk, unsigned long vaddr)
+{
+ struct vm_area_struct *vma;
+ struct mm_struct *mm;
+ int ret = -EINVAL;
+
+ mm = get_task_mm(tsk);
+ if (!mm)
+ return -EINVAL;
+ down_read(&mm->mmap_sem);
+ vma = find_vma(mm, vaddr);
+ if (vma && vaddr >= vma->vm_start && (vma->vm_flags & VM_EXEC))
+ ret = 0;
+ up_read(&mm->mmap_sem);
+ mmput(mm);
+ return ret;
+}
+
+/**
+ * user_bkpt_validate_insn_addr - Validate if the instruction is an
+ * executable vma.
+ * Returns 0 if the vaddr is a valid instruction address.
+ * @tsk: the probed task
+ * @vaddr: virtual address of the instruction to be verified.
+ *
+ * Possible errors:
+ * -%EINVAL: Instruction passed is not a valid instruction address.
+ */
+int user_bkpt_validate_insn_addr(struct task_struct *tsk, unsigned long vaddr)
+{
+ int result;
+
+ result = check_vma(tsk, vaddr);
+ if (result != 0)
+ return result;
+ if (arch->validate_address)
+ result = arch->validate_address(tsk, vaddr);
+ return result;
+}
+
+static void print_insert_fail(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt, const char *why)
+{
+ printk(KERN_ERR "Can't place breakpoint at pid %d vaddr %#lx: %s\n",
+ tsk->pid, user_bkpt->vaddr, why);
+}
+
+/**
+ * user_bkpt_insert_bkpt - insert breakpoint
+ * Insert a breakpoint into the process that includes @tsk, at the
+ * virtual address @user_bkpt->vaddr.
+ *
+ * @user_bkpt->strategy affects how this breakpoint will be handled:
+ * %USER_BKPT_HNT_INLINE: Probed instruction will be single-stepped inline.
+ * %USER_BKPT_HNT_TSKINFO: As above.
+ * %USER_BKPT_HNT_PERMSL: An XOL instruction slot in the probed process's
+ * address space has been allocated to this probepoint, and will
+ * remain so allocated as long as it's needed. @user_bkpt->xol_vaddr is
+ * its address. (This slot can be reallocated if
+ * @user_bkpt_insert_bkpt() fails.) The client is NOT required to
+ * allocate an instruction slot before calling @user_bkpt_insert_bkpt().
+ * @user_bkpt_insert_bkpt() updates @user_bkpt->strategy as needed:
+ * %USER_BKPT_HNT_INLINE: Architecture or client cannot do XOL for this
+ * probepoint.
+ * %USER_BKPT_HNT_TSKINFO: @user_bkpt_task_arch_info will be used for this
+ * probepoint.
+ *
+ * All threads of the probed process must be stopped while
+ * @user_bkpt_insert_bkpt() runs.
+ *
+ * Possible errors:
+ * -%ENOSYS: user_bkpt not supported for this architecture
+ * -%EINVAL: unrecognized/invalid strategy flags
+ * -%EINVAL: invalid instruction address
+ * -%EEXIST: breakpoint instruction already exists at that address
+ * -%EPERM: cannot probe this instruction
+ * -%EFAULT: failed to insert breakpoint instruction
+ * [TBD: Validate xol_vaddr?]
+ */
+int user_bkpt_insert_bkpt(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ int result;
+ unsigned long len;
+
+ if (!validate_strategy(user_bkpt->strategy, USER_BKPT_HNT_MASK))
+ return -EINVAL;
+
+ result = user_bkpt_validate_insn_addr(tsk, user_bkpt->vaddr);
+ if (result != 0)
+ return result;
+
+ /*
+ * If user_bkpt_read_vm() transfers fewer bytes than the maximum
+ * instruction size, assume that the probed instruction is smaller
+ * than the max and near the end of the last page of instructions.
+ * But there must be room at least for a breakpoint-size instruction.
+ */
+ len = user_bkpt_read_vm(tsk, (void __user *) user_bkpt->vaddr,
+ user_bkpt->insn, arch->max_insn_bytes);
+ if (len < user_bkpt_opcode_sz) {
+ print_insert_fail(tsk, user_bkpt,
+ "error reading original instruction");
+ return -EFAULT;
+ }
+ memcpy(&user_bkpt->opcode, user_bkpt->insn, user_bkpt_opcode_sz);
+ if (arch->is_bkpt_insn(user_bkpt)) {
+ print_insert_fail(tsk, user_bkpt,
+ "bkpt already exists at that addr");
+ return -EEXIST;
+ }
+
+ result = arch->analyze_insn(tsk, user_bkpt);
+ if (result < 0) {
+ print_insert_fail(tsk, user_bkpt,
+ "instruction type cannot be probed");
+ return result;
+ }
+
+ result = arch->set_bkpt(tsk, user_bkpt);
+ if (result < 0) {
+ print_insert_fail(tsk, user_bkpt,
+ "failed to insert bkpt instruction");
+ return result;
+ }
+ return 0;
+}
+
+/**
+ * user_bkpt_pre_sstep - prepare to single-step the probed instruction
+ * @tsk: the probed task
+ * @user_bkpt: the probepoint information, as returned by
+ * @user_bkpt_insert_bkpt(). Unless the %USER_BKPT_HNT_INLINE flag is
+ * set in @user_bkpt->strategy, @user_bkpt->xol_vaddr must be the
+ * address of an XOL instruction slot that is allocated to this
+ * probepoint at least until after the completion of
+ * @user_bkpt_post_sstep(), and populated with the contents of
+ * @user_bkpt->insn. [Need to be more precise here to account for
+ * untimely exit or USER_BKPT_HNT_BOOSTED.]
+ * @tskinfo: points to a @user_bkpt_task_arch_info object for @tsk, if
+ * the %USER_BKPT_HNT_TSKINFO flag is set in @user_bkpt->strategy.
+ * @regs: reflects the saved user state of @tsk. @user_bkpt_pre_sstep()
+ * adjusts this. In particular, the instruction pointer is set
+ * to the instruction to be single-stepped.
+ * Possible errors:
+ * -%EFAULT: Failed to read or write @tsk's address space as needed.
+ *
+ * The client must ensure that the contents of @user_bkpt are not
+ * changed during the single-step operation -- i.e., between when
+ * @user_bkpt_pre_sstep() is called and when @user_bkpt_post_sstep() returns.
+ * Additionally, if single-stepping inline is used for this probepoint,
+ * the client must serialize the single-step operation (so multiple
+ * threads don't step on each other while the opcode replacement is
+ * taking place).
+ */
+int user_bkpt_pre_sstep(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo, struct pt_regs *regs)
+{
+ int result;
+
+ if (uses_xol_strategy(user_bkpt->strategy))
+ return arch->pre_xol(tsk, user_bkpt, tskinfo, regs);
+
+ /*
+ * Single-step this instruction inline. Replace the breakpoint
+ * with the original opcode.
+ */
+ result = arch->set_orig_insn(tsk, user_bkpt, false);
+ if (result == 0)
+ arch->set_ip(regs, user_bkpt->vaddr);
+ return result;
+}
+
+/**
+ * user_bkpt_post_sstep - prepare to resume execution after single-step
+ * @tsk: the probed task
+ * @user_bkpt: the probepoint information, as with @user_bkpt_pre_sstep()
+ * @tskinfo: the @user_bkpt_task_arch_info object, if any, passed to
+ * @user_bkpt_pre_sstep()
+ * @regs: reflects the saved state of @tsk after the single-step
+ * operation. @user_bkpt_post_sstep() adjusts @tsk's state as needed,
+ * including pointing the instruction pointer at the instruction
+ * following the probed instruction.
+ * Possible errors:
+ * -%EFAULT: Failed to read or write @tsk's address space as needed.
+ */
+int user_bkpt_post_sstep(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo, struct pt_regs *regs)
+{
+ if (uses_xol_strategy(user_bkpt->strategy))
+ return arch->post_xol(tsk, user_bkpt, tskinfo, regs);
+
+ /*
+ * Single-stepped this instruction inline. Put the breakpoint
+ * instruction back.
+ */
+ return arch->set_bkpt(tsk, user_bkpt);
+}
+
+/**
+ * user_bkpt_cancel_xol - cancel XOL for this probepoint
+ * @tsk: a task in the probed process
+ * @user_bkpt: the probepoint information
+ * Switch @user_bkpt's single-stepping strategy from out-of-line to inline.
+ * If the client employs lazy XOL-slot allocation, it can call this function
+ * if it determines that it can't provide an XOL slot for @user_bkpt.
+ * @user_bkpt_cancel_xol() adjusts @user_bkpt appropriately.
+ *
+ * @user_bkpt_cancel_xol()'s behavior is undefined if @user_bkpt_pre_sstep()
+ * has already been called for @user_bkpt.
+ *
+ * Possible errors:
+ * Can't think of any yet.
+ */
+int user_bkpt_cancel_xol(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ if (arch->cancel_xol)
+ arch->cancel_xol(tsk, user_bkpt);
+ user_bkpt->strategy |= USER_BKPT_HNT_INLINE;
+ return 0;
+}
+
+/**
+ * user_bkpt_get_bkpt_addr - compute address of bkpt given post-bkpt regs
+ * @regs: Reflects the saved state of the task after it has hit a breakpoint
+ * instruction.
+ * Return the address of the breakpoint instruction.
+ */
+unsigned long user_bkpt_get_bkpt_addr(struct pt_regs *regs)
+{
+ return instruction_pointer(regs) - arch->ip_advancement_by_bkpt_insn;
+}
+
+/**
+ * user_bkpt_remove_bkpt - remove breakpoint
+ * For the process that includes @tsk, remove the breakpoint specified
+ * by @user_bkpt, restoring the original opcode.
+ *
+ * Possible errors:
+ * -%EINVAL: @user_bkpt->vaddr is not a valid instruction address.
+ * -%ENOENT: There is no breakpoint instruction at @user_bkpt->vaddr.
+ * -%EFAULT: Failed to read/write @tsk's address space as needed.
+ */
+int user_bkpt_remove_bkpt(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ if (user_bkpt_validate_insn_addr(tsk, user_bkpt->vaddr) != 0)
+ return -EINVAL;
+ return arch->set_orig_insn(tsk, user_bkpt, true);
+}
+
+/**
+ * user_bkpt_set_ip - Set instruction pointer
+ * @regs: Reflects the saved state of the task
+ */
+void user_bkpt_set_ip(struct pt_regs *regs, unsigned long vaddr)
+{
+ arch->set_ip(regs, vaddr);
+}
+
+/**
+ * user_bkpt_resume_can_sleep - Check if fixup might result in sleep.
+ * @user_bkpt: the probepoint information.
+ *
+ * Returns true if fixup might result in sleep.
+ */
+bool user_bkpt_resume_can_sleep(struct user_bkpt *user_bkpt)
+{
+ return user_bkpt->fixups & USER_BKPT_FIX_SLEEPY;
+}
+
+/* Default implementation of arch->is_bkpt_insn */
+static bool is_bkpt_insn(struct user_bkpt *user_bkpt)
+{
+ return (user_bkpt->opcode == arch->bkpt_insn);
+}
+
+/* Default implementation of arch->pre_xol */
+static int pre_xol(struct task_struct *tsk,
+ struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo,
+ struct pt_regs *regs)
+{
+ arch->set_ip(regs, user_bkpt->xol_vaddr);
+ return 0;
+}
+
+/* Validate arch-specific info during user_bkpt initialization. */
+static int bad_arch_param(const char *param_name, int value)
+{
+ printk(KERN_ERR "user_bkpt: bad value %d/%#x for parameter %s"
+ " in user_bkpt_arch_info\n", value, value, param_name);
+ return -ENOSYS;
+}
+
+static int missing_arch_func(const char *func_name)
+{
+ printk(KERN_ERR "user_bkpt: user_bkpt_arch_info lacks required "
+ "function: %s\n", func_name);
+ return -ENOSYS;
+}
+
+static int __init init_user_bkpt(void)
+{
+ int result = 0;
+
+ /* Accept any value of bkpt_insn. */
+ if (arch->max_insn_bytes < 1)
+ result = bad_arch_param("max_insn_bytes",
+ arch->max_insn_bytes);
+ if (arch->ip_advancement_by_bkpt_insn > arch->max_insn_bytes)
+ result = bad_arch_param("ip_advancement_by_bkpt_insn",
+ arch->ip_advancement_by_bkpt_insn);
+ /* Accept any value of strategies. */
+ if (!arch->set_ip)
+ result = missing_arch_func("set_ip");
+ /* Null validate_address() is OK. */
+ if (!arch->read_opcode)
+ arch->read_opcode = read_opcode;
+ if (!arch->set_bkpt)
+ arch->set_bkpt = set_bkpt;
+ if (!arch->set_orig_insn)
+ arch->set_orig_insn = set_orig_insn;
+ if (!arch->is_bkpt_insn)
+ arch->is_bkpt_insn = is_bkpt_insn;
+ if (!arch->analyze_insn)
+ result = missing_arch_func("analyze_insn");
+ if (!arch->pre_xol)
+ arch->pre_xol = pre_xol;
+ if (uses_xol_strategy(arch->strategies) && !arch->post_xol)
+ result = missing_arch_func("post_xol");
+ /* Null cancel_xol() is OK. */
+ return result;
+}
+
+module_init(init_user_bkpt);
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 3/10] x86 support for User space breakpoint assistance
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
2010-05-06 18:01 ` [PATCH v3 1/10] X86 instruction analysis: Move Macro W to insn.h Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 2/10] User Space Breakpoint Assistance Layer Srikar Dronamraju
@ 2010-05-06 18:02 ` Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 4/10] Slot allocation for execution out of line (XOL) Srikar Dronamraju
` (8 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:02 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, Andrew Morton, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, LKML, Paul E. McKenney
x86_support_for_user_bkpt.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
x86 support for user breakpoint Infrastructure
Changelog:
set USER_BKPT_FIX_SLEEPY if post_xol might sleep.
This patch provides x86 specific userspace breakpoint assistance
implementation details. It uses the "x86: instruction decoder API" patch
to do validate and analyze the instructions. This analysis is used at
the time of post-processing of breakpoint hit to do the necessary
fix-ups.
Almost all instructions are handled for traditional strategy and
execution out of line strategy. Instruction handled include the RIP
relative instructions.
This patch requires "x86: instruction decoder API" patch.
http://lkml.org/lkml/2009/6/1/459
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
arch/x86/Kconfig | 1
arch/x86/include/asm/user_bkpt.h | 43 +++
arch/x86/kernel/Makefile | 2
arch/x86/kernel/user_bkpt.c | 572 ++++++++++++++++++++++++++++++++++++++
4 files changed, 618 insertions(+), 0 deletions(-)
create mode 100644 arch/x86/include/asm/user_bkpt.h
create mode 100644 arch/x86/kernel/user_bkpt.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 9458685..7dce185 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -53,6 +53,7 @@ config X86
select HAVE_KERNEL_LZMA
select HAVE_KERNEL_LZO
select HAVE_HW_BREAKPOINT
+ select HAVE_USER_BKPT
select PERF_EVENTS
select ANON_INODES
select HAVE_ARCH_KMEMCHECK
diff --git a/arch/x86/include/asm/user_bkpt.h b/arch/x86/include/asm/user_bkpt.h
new file mode 100644
index 0000000..df8a4a0
--- /dev/null
+++ b/arch/x86/include/asm/user_bkpt.h
@@ -0,0 +1,43 @@
+#ifndef _ASM_USER_BKPT_H
+#define _ASM_USER_BKPT_H
+/*
+ * User-space BreakPoint support (user_bkpt) for x86
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+typedef u8 user_bkpt_opcode_t;
+#define MAX_UINSN_BYTES 16
+#define USER_BKPT_XOL_SLOT_BYTES (MAX_UINSN_BYTES)
+
+#ifdef CONFIG_X86_64
+struct bkpt_arch_info {
+ unsigned long rip_target_address;
+ u8 orig_insn[MAX_UINSN_BYTES];
+};
+struct user_bkpt_task_arch_info {
+ unsigned long saved_scratch_register;
+};
+#else
+struct bkpt_arch_info {};
+struct user_bkpt_task_arch_info {};
+#endif
+
+#endif /* _ASM_USER_BKPT_H */
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 4c58352..98c74b4 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -117,6 +117,8 @@ obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
+obj-$(CONFIG_USER_BKPT) += user_bkpt.o
+
###
# 64 bit specific files
ifeq ($(CONFIG_X86_64),y)
diff --git a/arch/x86/kernel/user_bkpt.c b/arch/x86/kernel/user_bkpt.c
new file mode 100644
index 0000000..eb74a8d
--- /dev/null
+++ b/arch/x86/kernel/user_bkpt.c
@@ -0,0 +1,572 @@
+/*
+ * User-space BreakPoint support (user_bkpt) for x86
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/ptrace.h>
+#include <linux/user_bkpt.h>
+#include <asm/insn.h>
+
+#ifdef CONFIG_X86_32
+#define is_32bit_app(tsk) 1
+#else
+#define is_32bit_app(tsk) (test_tsk_thread_flag(tsk, TIF_IA32))
+#endif
+
+#define USER_BKPT_FIX_RIP_AX 0x8000
+#define USER_BKPT_FIX_RIP_CX 0x4000
+
+/* Adaptations for mhiramat x86 decoder v14. */
+#define OPCODE1(insn) ((insn)->opcode.bytes[0])
+#define OPCODE2(insn) ((insn)->opcode.bytes[1])
+#define OPCODE3(insn) ((insn)->opcode.bytes[2])
+#define MODRM_REG(insn) X86_MODRM_REG(insn->modrm.value)
+
+static void set_ip(struct pt_regs *regs, unsigned long vaddr)
+{
+ regs->ip = vaddr;
+}
+
+#ifdef CONFIG_X86_64
+static bool is_riprel_insn(struct user_bkpt *user_bkpt)
+{
+ return ((user_bkpt->fixups &
+ (USER_BKPT_FIX_RIP_AX | USER_BKPT_FIX_RIP_CX)) != 0);
+}
+
+static void cancel_xol(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ if (is_riprel_insn(user_bkpt)) {
+ /*
+ * We rewrote user_bkpt->insn to use indirect addressing rather
+ * than rip-relative addressing for XOL. For
+ * single-stepping inline, put back the original instruction.
+ */
+ memcpy(user_bkpt->insn, user_bkpt->arch_info.orig_insn,
+ MAX_UINSN_BYTES);
+ user_bkpt->strategy &= ~USER_BKPT_HNT_TSKINFO;
+ }
+}
+#endif /* CONFIG_X86_64 */
+
+static const u32 good_insns_64[256 / 32] = {
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+ /* ---------------------------------------------- */
+ W(0x00, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) | /* 00 */
+ W(0x10, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) , /* 10 */
+ W(0x20, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) | /* 20 */
+ W(0x30, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) , /* 30 */
+ W(0x40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) | /* 40 */
+ W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
+ W(0x60, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* 60 */
+ W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 70 */
+ W(0x80, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
+ W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
+ W(0xa0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* a0 */
+ W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
+ W(0xc0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0) | /* c0 */
+ W(0xd0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
+ W(0xe0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* e0 */
+ W(0xf0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1) /* f0 */
+ /* ---------------------------------------------- */
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+};
+
+/* Good-instruction tables for 32-bit apps -- copied from i386 uprobes */
+
+static const u32 good_insns_32[256 / 32] = {
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+ /* ---------------------------------------------- */
+ W(0x00, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0) | /* 00 */
+ W(0x10, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0) , /* 10 */
+ W(0x20, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1) | /* 20 */
+ W(0x30, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1) , /* 30 */
+ W(0x40, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 40 */
+ W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
+ W(0x60, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* 60 */
+ W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 70 */
+ W(0x80, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
+ W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
+ W(0xa0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* a0 */
+ W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
+ W(0xc0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0) | /* c0 */
+ W(0xd0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
+ W(0xe0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* e0 */
+ W(0xf0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1) /* f0 */
+ /* ---------------------------------------------- */
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+};
+
+/* Using this for both 64-bit and 32-bit apps */
+static const u32 good_2byte_insns[256 / 32] = {
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+ /* ---------------------------------------------- */
+ W(0x00, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1) | /* 00 */
+ W(0x10, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1) , /* 10 */
+ W(0x20, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1) | /* 20 */
+ W(0x30, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) , /* 30 */
+ W(0x40, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 40 */
+ W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
+ W(0x60, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 60 */
+ W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1) , /* 70 */
+ W(0x80, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
+ W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
+ W(0xa0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1) | /* a0 */
+ W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
+ W(0xc0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* c0 */
+ W(0xd0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
+ W(0xe0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* e0 */
+ W(0xf0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0) /* f0 */
+ /* ---------------------------------------------- */
+ /* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
+};
+
+/*
+ * opcodes we'll probably never support:
+ * 6c-6d, e4-e5, ec-ed - in
+ * 6e-6f, e6-e7, ee-ef - out
+ * cc, cd - int3, int
+ * cf - iret
+ * d6 - illegal instruction
+ * f1 - int1/icebp
+ * f4 - hlt
+ * fa, fb - cli, sti
+ * 0f - lar, lsl, syscall, clts, sysret, sysenter, sysexit, invd, wbinvd, ud2
+ *
+ * invalid opcodes in 64-bit mode:
+ * 06, 0e, 16, 1e, 27, 2f, 37, 3f, 60-62, 82, c4-c5, d4-d5
+ *
+ * 63 - we support this opcode in x86_64 but not in i386.
+ *
+ * opcodes we may need to refine support for:
+ * 0f - 2-byte instructions: For many of these instructions, the validity
+ * depends on the prefix and/or the reg field. On such instructions, we
+ * just consider the opcode combination valid if it corresponds to any
+ * valid instruction.
+ * 8f - Group 1 - only reg = 0 is OK
+ * c6-c7 - Group 11 - only reg = 0 is OK
+ * d9-df - fpu insns with some illegal encodings
+ * f2, f3 - repnz, repz prefixes. These are also the first byte for
+ * certain floating-point instructions, such as addsd.
+ * fe - Group 4 - only reg = 0 or 1 is OK
+ * ff - Group 5 - only reg = 0-6 is OK
+ *
+ * others -- Do we need to support these?
+ * 0f - (floating-point?) prefetch instructions
+ * 07, 17, 1f - pop es, pop ss, pop ds
+ * 26, 2e, 36, 3e - es:, cs:, ss:, ds: segment prefixes --
+ * but 64 and 65 (fs: and gs:) seem to be used, so we support them
+ * 67 - addr16 prefix
+ * ce - into
+ * f0 - lock prefix
+ */
+
+/*
+ * TODO:
+ * - Where necessary, examine the modrm byte and allow only valid instructions
+ * in the different Groups and fpu instructions.
+ */
+
+static bool is_prefix_bad(struct insn *insn)
+{
+ int i;
+
+ for (i = 0; i < insn->prefixes.nbytes; i++) {
+ switch (insn->prefixes.bytes[i]) {
+ case 0x26: /*INAT_PFX_ES */
+ case 0x2E: /*INAT_PFX_CS */
+ case 0x36: /*INAT_PFX_DS */
+ case 0x3E: /*INAT_PFX_SS */
+ case 0xF0: /*INAT_PFX_LOCK */
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static void report_bad_prefix(void)
+{
+ printk(KERN_ERR "user_bkpt does not currently support probing "
+ "instructions with any of the following prefixes: "
+ "cs:, ds:, es:, ss:, lock:\n");
+}
+
+static void report_bad_1byte_opcode(int mode, user_bkpt_opcode_t op)
+{
+ printk(KERN_ERR "In %d-bit apps, "
+ "user_bkpt does not currently support probing "
+ "instructions whose first byte is 0x%2.2x\n", mode, op);
+}
+
+static void report_bad_2byte_opcode(user_bkpt_opcode_t op)
+{
+ printk(KERN_ERR "user_bkpt does not currently support probing "
+ "instructions with the 2-byte opcode 0x0f 0x%2.2x\n", op);
+}
+
+static int validate_insn_32bits(struct user_bkpt *user_bkpt, struct insn *insn)
+{
+ insn_init(insn, user_bkpt->insn, false);
+
+ /* Skip good instruction prefixes; reject "bad" ones. */
+ insn_get_opcode(insn);
+ if (is_prefix_bad(insn)) {
+ report_bad_prefix();
+ return -EPERM;
+ }
+ if (test_bit(OPCODE1(insn), (unsigned long *) good_insns_32))
+ return 0;
+ if (insn->opcode.nbytes == 2) {
+ if (test_bit(OPCODE2(insn),
+ (unsigned long *) good_2byte_insns))
+ return 0;
+ report_bad_2byte_opcode(OPCODE2(insn));
+ } else
+ report_bad_1byte_opcode(32, OPCODE1(insn));
+ return -EPERM;
+}
+
+static int validate_insn_64bits(struct user_bkpt *user_bkpt, struct insn *insn)
+{
+ insn_init(insn, user_bkpt->insn, true);
+
+ /* Skip good instruction prefixes; reject "bad" ones. */
+ insn_get_opcode(insn);
+ if (is_prefix_bad(insn)) {
+ report_bad_prefix();
+ return -EPERM;
+ }
+ if (test_bit(OPCODE1(insn), (unsigned long *) good_insns_64))
+ return 0;
+ if (insn->opcode.nbytes == 2) {
+ if (test_bit(OPCODE2(insn),
+ (unsigned long *) good_2byte_insns))
+ return 0;
+ report_bad_2byte_opcode(OPCODE2(insn));
+ } else
+ report_bad_1byte_opcode(64, OPCODE1(insn));
+ return -EPERM;
+}
+
+/*
+ * Figure out which fixups post_xol() will need to perform, and annotate
+ * user_bkpt->fixups accordingly. To start with, user_bkpt->fixups is
+ * either zero or it reflects rip-related fixups.
+ */
+static void prepare_fixups(struct user_bkpt *user_bkpt, struct insn *insn)
+{
+ bool fix_ip = true, fix_call = false; /* defaults */
+ insn_get_opcode(insn); /* should be a nop */
+
+ switch (OPCODE1(insn)) {
+ case 0xc3: /* ret/lret */
+ case 0xcb:
+ case 0xc2:
+ case 0xca:
+ /* ip is correct */
+ fix_ip = false;
+ break;
+ case 0xe8: /* call relative - Fix return addr */
+ fix_call = true;
+ break;
+ case 0x9a: /* call absolute - Fix return addr, not ip */
+ fix_call = true;
+ fix_ip = false;
+ break;
+ case 0xff:
+ {
+ int reg;
+ insn_get_modrm(insn);
+ reg = MODRM_REG(insn);
+ if (reg == 2 || reg == 3) {
+ /* call or lcall, indirect */
+ /* Fix return addr; ip is correct. */
+ fix_call = true;
+ fix_ip = false;
+ } else if (reg == 4 || reg == 5) {
+ /* jmp or ljmp, indirect */
+ /* ip is correct. */
+ fix_ip = false;
+ }
+ break;
+ }
+ case 0xea: /* jmp absolute -- ip is correct */
+ fix_ip = false;
+ break;
+ default:
+ break;
+ }
+ if (fix_ip)
+ user_bkpt->fixups |= USER_BKPT_FIX_IP;
+ if (fix_call)
+ user_bkpt->fixups |=
+ (USER_BKPT_FIX_CALL | USER_BKPT_FIX_SLEEPY);
+}
+
+#ifdef CONFIG_X86_64
+static int handle_riprel_insn(struct user_bkpt *user_bkpt, struct insn *insn);
+#endif
+
+static int analyze_insn(struct task_struct *tsk, struct user_bkpt *user_bkpt)
+{
+ int ret;
+ struct insn insn;
+
+ user_bkpt->fixups = 0;
+#ifdef CONFIG_X86_64
+ user_bkpt->arch_info.rip_target_address = 0x0;
+#endif
+
+ if (is_32bit_app(tsk))
+ ret = validate_insn_32bits(user_bkpt, &insn);
+ else
+ ret = validate_insn_64bits(user_bkpt, &insn);
+ if (ret != 0)
+ return ret;
+ if (user_bkpt->strategy & USER_BKPT_HNT_INLINE)
+ return 0;
+#ifdef CONFIG_X86_64
+ ret = handle_riprel_insn(user_bkpt, &insn);
+ if (ret == -1)
+ /* rip-relative; can't XOL */
+ return 0;
+ else if (ret == 0)
+ /* not rip-relative */
+ user_bkpt->strategy &= ~USER_BKPT_HNT_TSKINFO;
+#endif
+ prepare_fixups(user_bkpt, &insn);
+ return 0;
+}
+
+#ifdef CONFIG_X86_64
+/*
+ * If user_bkpt->insn doesn't use rip-relative addressing, return 0. Otherwise,
+ * rewrite the instruction so that it accesses its memory operand
+ * indirectly through a scratch register. Set user_bkpt->fixups and
+ * user_bkpt->arch_info.rip_target_address accordingly. (The contents of the
+ * scratch register will be saved before we single-step the modified
+ * instruction, and restored afterward.) Return 1.
+ *
+ * (... except if the client doesn't support our USER_BKPT_HNT_TSKINFO strategy,
+ * we must suppress XOL for rip-relative instructions: return -1.)
+ *
+ * We do this because a rip-relative instruction can access only a
+ * relatively small area (+/- 2 GB from the instruction), and the XOL
+ * area typically lies beyond that area. At least for instructions
+ * that store to memory, we can't execute the original instruction
+ * and "fix things up" later, because the misdirected store could be
+ * disastrous.
+ *
+ * Some useful facts about rip-relative instructions:
+ * - There's always a modrm byte.
+ * - There's never a SIB byte.
+ * - The displacement is always 4 bytes.
+ */
+static int handle_riprel_insn(struct user_bkpt *user_bkpt, struct insn *insn)
+{
+ u8 *cursor;
+ u8 reg;
+
+ if (!insn_rip_relative(insn))
+ return 0;
+
+ /*
+ * We have a rip-relative instruction. To allow this instruction
+ * to be single-stepped out of line, the client must provide us
+ * with a per-task user_bkpt_task_arch_info object.
+ */
+ if (!(user_bkpt->strategy & USER_BKPT_HNT_TSKINFO)) {
+ user_bkpt->strategy |= USER_BKPT_HNT_INLINE;
+ return -1;
+ }
+ memcpy(user_bkpt->arch_info.orig_insn, user_bkpt->insn,
+ MAX_UINSN_BYTES);
+
+ /*
+ * Point cursor at the modrm byte. The next 4 bytes are the
+ * displacement. Beyond the displacement, for some instructions,
+ * is the immediate operand.
+ */
+ cursor = user_bkpt->insn + insn->prefixes.nbytes
+ + insn->rex_prefix.nbytes + insn->opcode.nbytes;
+ insn_get_length(insn);
+
+ /*
+ * Convert from rip-relative addressing to indirect addressing
+ * via a scratch register. Change the r/m field from 0x5 (%rip)
+ * to 0x0 (%rax) or 0x1 (%rcx), and squeeze out the offset field.
+ */
+ reg = MODRM_REG(insn);
+ if (reg == 0) {
+ /*
+ * The register operand (if any) is either the A register
+ * (%rax, %eax, etc.) or (if the 0x4 bit is set in the
+ * REX prefix) %r8. In any case, we know the C register
+ * is NOT the register operand, so we use %rcx (register
+ * #1) for the scratch register.
+ */
+ user_bkpt->fixups = USER_BKPT_FIX_RIP_CX;
+ /* Change modrm from 00 000 101 to 00 000 001. */
+ *cursor = 0x1;
+ } else {
+ /* Use %rax (register #0) for the scratch register. */
+ user_bkpt->fixups = USER_BKPT_FIX_RIP_AX;
+ /* Change modrm from 00 xxx 101 to 00 xxx 000 */
+ *cursor = (reg << 3);
+ }
+
+ /* Target address = address of next instruction + (signed) offset */
+ user_bkpt->arch_info.rip_target_address = (long) user_bkpt->vaddr +
+ insn->length + insn->displacement.value;
+ /* Displacement field is gone; slide immediate field (if any) over. */
+ if (insn->immediate.nbytes) {
+ cursor++;
+ memmove(cursor, cursor + insn->displacement.nbytes,
+ insn->immediate.nbytes);
+ }
+ return 1;
+}
+
+/*
+ * If we're emulating a rip-relative instruction, save the contents
+ * of the scratch register and store the target address in that register.
+ */
+static int pre_xol(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo, struct pt_regs *regs)
+{
+ BUG_ON(!user_bkpt->xol_vaddr);
+ regs->ip = user_bkpt->xol_vaddr;
+ if (user_bkpt->fixups & USER_BKPT_FIX_RIP_AX) {
+ tskinfo->saved_scratch_register = regs->ax;
+ regs->ax = user_bkpt->arch_info.rip_target_address;
+ } else if (user_bkpt->fixups & USER_BKPT_FIX_RIP_CX) {
+ tskinfo->saved_scratch_register = regs->cx;
+ regs->cx = user_bkpt->arch_info.rip_target_address;
+ }
+ return 0;
+}
+#endif
+
+/*
+ * Called by post_xol() to adjust the return address pushed by a call
+ * instruction executed out of line.
+ */
+static int adjust_ret_addr(struct task_struct *tsk, unsigned long sp,
+ long correction)
+{
+ int rasize, ncopied;
+ long ra = 0;
+
+ if (is_32bit_app(tsk))
+ rasize = 4;
+ else
+ rasize = 8;
+ ncopied = user_bkpt_read_vm(tsk, (void __user *) sp, &ra, rasize);
+ if (unlikely(ncopied != rasize))
+ goto fail;
+ ra += correction;
+ ncopied = user_bkpt_write_data(tsk, (void __user *) sp, &ra, rasize);
+ if (unlikely(ncopied != rasize))
+ goto fail;
+ return 0;
+
+fail:
+ printk(KERN_ERR
+ "user_bkpt: Failed to adjust return address after"
+ " single-stepping call instruction;"
+ " pid=%d, sp=%#lx\n", tsk->pid, sp);
+ return -EFAULT;
+}
+
+/*
+ * Called after single-stepping. user_bkpt->vaddr is the address of the
+ * instruction whose first byte has been replaced by the "int3"
+ * instruction. To avoid the SMP problems that can occur when we
+ * temporarily put back the original opcode to single-step, we
+ * single-stepped a copy of the instruction. The address of this
+ * copy is user_bkpt->xol_vaddr.
+ *
+ * This function prepares to resume execution after the single-step.
+ * We have to fix things up as follows:
+ *
+ * Typically, the new ip is relative to the copied instruction. We need
+ * to make it relative to the original instruction (FIX_IP). Exceptions
+ * are return instructions and absolute or indirect jump or call instructions.
+ *
+ * If the single-stepped instruction was a call, the return address that
+ * is atop the stack is the address following the copied instruction. We
+ * need to make it the address following the original instruction (FIX_CALL).
+ *
+ * If the original instruction was a rip-relative instruction such as
+ * "movl %edx,0xnnnn(%rip)", we have instead executed an equivalent
+ * instruction using a scratch register -- e.g., "movl %edx,(%rax)".
+ * We need to restore the contents of the scratch register and adjust
+ * the ip, keeping in mind that the instruction we executed is 4 bytes
+ * shorter than the original instruction (since we squeezed out the offset
+ * field). (FIX_RIP_AX or FIX_RIP_CX)
+ */
+static int post_xol(struct task_struct *tsk, struct user_bkpt *user_bkpt,
+ struct user_bkpt_task_arch_info *tskinfo, struct pt_regs *regs)
+{
+ /* Typically, the XOL vma is at a high addr, so correction < 0. */
+ long correction = (long) (user_bkpt->vaddr - user_bkpt->xol_vaddr);
+ int result = 0;
+
+#ifdef CONFIG_X86_64
+ if (is_riprel_insn(user_bkpt)) {
+ if (user_bkpt->fixups & USER_BKPT_FIX_RIP_AX)
+ regs->ax = tskinfo->saved_scratch_register;
+ else
+ regs->cx = tskinfo->saved_scratch_register;
+ /*
+ * The original instruction includes a displacement, and so
+ * is 4 bytes longer than what we've just single-stepped.
+ * Fall through to handle stuff like "jmpq *...(%rip)" and
+ * "callq *...(%rip)".
+ */
+ correction += 4;
+ }
+#endif
+ if (user_bkpt->fixups & USER_BKPT_FIX_IP)
+ regs->ip += correction;
+ if (user_bkpt->fixups & USER_BKPT_FIX_CALL)
+ result = adjust_ret_addr(tsk, regs->sp, correction);
+ return result;
+}
+
+struct user_bkpt_arch_info user_bkpt_arch_info = {
+ .bkpt_insn = 0xcc,
+ .ip_advancement_by_bkpt_insn = 1,
+ .max_insn_bytes = MAX_UINSN_BYTES,
+#ifdef CONFIG_X86_32
+ .strategies = 0x0,
+#else
+ /* rip-relative instructions require special handling. */
+ .strategies = USER_BKPT_HNT_TSKINFO,
+ .pre_xol = pre_xol,
+ .cancel_xol = cancel_xol,
+#endif
+ .set_ip = set_ip,
+ .analyze_insn = analyze_insn,
+ .post_xol = post_xol,
+};
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 4/10] Slot allocation for execution out of line (XOL)
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (2 preceding siblings ...)
2010-05-06 18:02 ` [PATCH v3 3/10] x86 support for User space breakpoint assistance Srikar Dronamraju
@ 2010-05-06 18:02 ` Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 5/10] Uprobes Implementation Srikar Dronamraju
` (7 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:02 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
user_bkpt_xol.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Changelog from v2: (addressing Oleg's comments)
* Removed code in !CONFIG_USER_BKPT_XOL
* Functions now pass pointer to user_bkpt_xol_area instead of pointer
to void.
Slot allocation for Execution out of line strategy(XOL)
This patch provides slot allocation mechanism for execution out of
line strategy for use with user space breakpoint infrastructure.
Traditional method of replacing the original instructions on breakpoint
hit are racy when used on multithreaded applications.
Alternatives for the traditional method include:
- Emulating the breakpointed instruction.
- Execution out of line.
Emulating the instruction:
This approach would use a in-kernel instruction emulator to
emulate the breakpointed instruction. This approach could be looked in
at a later point of time.
Execution out of line:
In execution out of line strategy, a new vma is injected into
the target process, a copy of the instructions which are breakpointed is
stored in one of the slots. On breakpoint hit, the copy of the
instruction is single-stepped leaving the breakpoint instruction as is.
This method is architecture independent.
This method is useful while handling multithreaded processes.
This patch allocates one page per process for slots to be used to copy the
breakpointed instructions.
Current slot allocation mechanism:
1. Allocate one dedicated slot per user breakpoint. Each slot is big
enuf to accomodate the biggest instruction for that architecture. (16
bytes for x86).
2. We currently allocate only one page for slots. Hence the number of
slots is limited to active breakpoint hits on that process.
3. Bitmap to track used slots.
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
arch/Kconfig | 4 +
include/linux/user_bkpt_xol.h | 40 ++++++
kernel/Makefile | 1
kernel/user_bkpt_xol.c | 284 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 329 insertions(+), 0 deletions(-)
create mode 100644 include/linux/user_bkpt_xol.h
create mode 100644 kernel/user_bkpt_xol.c
diff --git a/arch/Kconfig b/arch/Kconfig
index 116f0ea..424494e 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -98,6 +98,10 @@ config USER_RETURN_NOTIFIER
config HAVE_USER_BKPT
def_bool n
+config USER_BKPT_XOL
+ def_bool y
+ depends on USER_BKPT
+
config HAVE_IOREMAP_PROT
bool
diff --git a/include/linux/user_bkpt_xol.h b/include/linux/user_bkpt_xol.h
new file mode 100644
index 0000000..47e8d62
--- /dev/null
+++ b/include/linux/user_bkpt_xol.h
@@ -0,0 +1,40 @@
+#ifndef _LINUX_XOL_H
+#define _LINUX_XOL_H
+/*
+ * User-space BreakPoint support (user_bkpt) -- Allocation of instruction
+ * slots for execution out of line (XOL)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2009, 2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+#define UINSNS_PER_PAGE (PAGE_SIZE/USER_BKPT_XOL_SLOT_BYTES)
+#define MAX_USER_BKPT_XOL_SLOTS UINSNS_PER_PAGE
+struct user_bkpt_xol_area;
+
+extern unsigned long xol_get_insn_slot(struct user_bkpt *user_bkpt,
+ struct user_bkpt_xol_area *xol_area);
+extern void xol_free_insn_slot(unsigned long,
+ struct user_bkpt_xol_area *xol_area);
+extern int xol_validate_vaddr(struct pid *pid, unsigned long vaddr,
+ struct user_bkpt_xol_area *xol_area);
+extern struct user_bkpt_xol_area *xol_alloc_area(void);
+extern void xol_free_area(struct user_bkpt_xol_area *xol_area);
+
+#endif /* _LINUX_XOL_H */
diff --git a/kernel/Makefile b/kernel/Makefile
index 36a35b0..e404aa0 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -106,6 +106,7 @@ obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
obj-$(CONFIG_USER_RETURN_NOTIFIER) += user-return-notifier.o
obj-$(CONFIG_PADATA) += padata.o
obj-$(CONFIG_USER_BKPT) += user_bkpt.o
+obj-$(CONFIG_USER_BKPT_XOL) += user_bkpt_xol.o
ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
# According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
diff --git a/kernel/user_bkpt_xol.c b/kernel/user_bkpt_xol.c
new file mode 100644
index 0000000..6d3e5bd
--- /dev/null
+++ b/kernel/user_bkpt_xol.c
@@ -0,0 +1,284 @@
+/*
+ * User-space BreakPoint support (user_bkpt) -- Allocation of instruction
+ * slots for execution out of line (XOL)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2009, 2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+/*
+ * Every probepoint gets its own slot. Once it's assigned a slot, it
+ * keeps that slot until the probepoint goes away. Only definite number
+ * of slots are allocated.
+ */
+#include <linux/mm.h>
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/mman.h>
+#include <linux/file.h>
+#include <linux/pid.h>
+#include <linux/sched.h>
+#include <linux/user_bkpt.h>
+#include <linux/user_bkpt_xol.h>
+#include <linux/slab.h>
+
+struct user_bkpt_xol_area {
+ spinlock_t lock; /* protects bitmap and slot (de)allocation*/
+ unsigned long *bitmap; /* 0 = free slot */
+
+ /*
+ * We keep the vma's vm_start rather than a pointer to the vma
+ * itself. The probed process or a naughty kernel module could make
+ * the vma go away, and we must handle that reasonably gracefully.
+ */
+ unsigned long vaddr; /* Page(s) of instruction slots */
+};
+
+static int xol_add_vma(struct user_bkpt_xol_area *area)
+{
+ struct vm_area_struct *vma;
+ struct mm_struct *mm;
+ struct file *file;
+ unsigned long addr;
+
+ mm = get_task_mm(current);
+ if (!mm)
+ return -ESRCH;
+
+ down_write(&mm->mmap_sem);
+ /*
+ * Find the end of the top mapping and skip a page.
+ * If there is no space for PAGE_SIZE above
+ * that, mmap will ignore our address hint.
+ *
+ * We allocate a "fake" unlinked shmem file because
+ * anonymous memory might not be granted execute
+ * permission when the selinux security hooks have
+ * their way.
+ */
+ vma = rb_entry(rb_last(&mm->mm_rb), struct vm_area_struct, vm_rb);
+ addr = vma->vm_end + PAGE_SIZE;
+ file = shmem_file_setup("uprobes/xol", PAGE_SIZE, VM_NORESERVE);
+ if (!file) {
+ printk(KERN_ERR "user_bkpt_xol failed to setup shmem_file "
+ "while allocating vma for pid/tgid %d/%d for "
+ "single-stepping out of line.\n",
+ current->pid, current->tgid);
+ goto fail;
+ }
+ addr = do_mmap_pgoff(file, addr, PAGE_SIZE, PROT_EXEC, MAP_PRIVATE, 0);
+ fput(file);
+
+ if (addr & ~PAGE_MASK) {
+ printk(KERN_ERR "user_bkpt_xol failed to allocate a vma for "
+ "pid/tgid %d/%d for single-stepping out of "
+ "line.\n", current->pid, current->tgid);
+ goto fail;
+ }
+ vma = find_vma(mm, addr);
+
+ /* Don't expand vma on mremap(). */
+ vma->vm_flags |= VM_DONTEXPAND | VM_DONTCOPY;
+ area->vaddr = vma->vm_start;
+ up_write(&mm->mmap_sem);
+ mmput(mm);
+ return 0;
+
+fail:
+ up_write(&mm->mmap_sem);
+ mmput(mm);
+ return -ENOMEM;
+}
+
+/**
+ * xol_alloc_area - Allocate process's user_bkpt_xol_area.
+ * This area will be used for storing instructions for execution out of
+ * line.
+ */
+struct user_bkpt_xol_area *xol_alloc_area(void)
+{
+ struct user_bkpt_xol_area *area = NULL;
+
+ area = kzalloc(sizeof(*area), GFP_USER);
+ if (unlikely(!area))
+ return NULL;
+
+ area->bitmap = kzalloc(BITS_TO_LONGS(UINSNS_PER_PAGE) * sizeof(long),
+ GFP_USER);
+
+ if (!area->bitmap)
+ goto fail;
+ if (xol_add_vma(area)) {
+ kfree(area->bitmap);
+ goto fail;
+ }
+ spin_lock_init(&area->lock);
+ return area;
+
+fail:
+ kfree(area);
+ return NULL;
+}
+
+void xol_free_area(struct user_bkpt_xol_area *xol_area)
+{
+ kfree(xol_area->bitmap);
+ kfree(xol_area);
+}
+
+/*
+ * Find a slot
+ * - searching in existing vmas for a free slot.
+ * - If no free slot in existing vmas, return 0;
+ * called with lock acquired.
+ */
+static unsigned long xol_take_insn_slot(struct user_bkpt_xol_area *area)
+{
+ unsigned long slot_addr;
+ int slot_nr;
+
+ slot_nr = find_first_zero_bit(area->bitmap, UINSNS_PER_PAGE);
+ if (slot_nr < UINSNS_PER_PAGE) {
+ set_bit(slot_nr, area->bitmap);
+ slot_addr = area->vaddr +
+ (slot_nr * USER_BKPT_XOL_SLOT_BYTES);
+ return slot_addr;
+ }
+
+ return 0;
+}
+
+/**
+ * xol_get_insn_slot - If user_bkpt was not allocated a slot, then
+ * allocate a slot. If user_bkpt_insert_bkpt is already called, (i.e
+ * user_bkpt.vaddr != 0) then copy the instruction into the slot.
+ * Returns the allocated slot address or 0.
+ * @user_bkpt: probepoint information
+ * @xol_area refers the unique per process user_bkpt_xol_area for
+ * this process.
+ */
+unsigned long xol_get_insn_slot(struct user_bkpt *user_bkpt,
+ struct user_bkpt_xol_area *xol_area)
+{
+ unsigned long flags;
+ int len;
+
+ if (unlikely(!xol_area))
+ return 0;
+
+ if (!user_bkpt->xol_vaddr) {
+ spin_lock_irqsave(&xol_area->lock, flags);
+ if (likely(!user_bkpt->xol_vaddr))
+ user_bkpt->xol_vaddr = xol_take_insn_slot(xol_area);
+ spin_unlock_irqrestore(&xol_area->lock, flags);
+
+ /*
+ * Initialize the slot if user_bkpt->vaddr points to valid
+ * instruction slot.
+ */
+ if (likely(user_bkpt->xol_vaddr) && user_bkpt->vaddr) {
+ len = access_process_vm(current, user_bkpt->xol_vaddr,
+ user_bkpt->insn, USER_BKPT_XOL_SLOT_BYTES, 1);
+ if (unlikely(len < USER_BKPT_XOL_SLOT_BYTES))
+ printk(KERN_ERR "Failed to copy instruction"
+ " at %#lx len = %d\n",
+ user_bkpt->vaddr, len);
+ }
+ }
+ return user_bkpt->xol_vaddr;
+}
+
+/**
+ * xol_free_insn_slot - If slot was earlier allocated by
+ * @xol_get_insn_slot(), make the slot available for
+ * subsequent requests.
+ * @slot_addr: slot address as returned by
+ * @xol_get_insn_area().
+ * @xol_area refers the unique per process user_bkpt_xol_area for
+ * this process.
+ */
+void xol_free_insn_slot(unsigned long slot_addr,
+ struct user_bkpt_xol_area *xol_area)
+{
+ unsigned long vma_end;
+ int found = 0;
+
+ if (unlikely(!slot_addr || IS_ERR_VALUE(slot_addr)))
+ return;
+
+ if (unlikely(!xol_area))
+ return;
+
+ vma_end = xol_area->vaddr + PAGE_SIZE;
+ if (xol_area->vaddr <= slot_addr && slot_addr < vma_end) {
+ int slot_nr;
+ unsigned long offset = slot_addr - xol_area->vaddr;
+ unsigned long flags;
+
+ BUG_ON(offset % USER_BKPT_XOL_SLOT_BYTES);
+
+ slot_nr = offset / USER_BKPT_XOL_SLOT_BYTES;
+ BUG_ON(slot_nr >= UINSNS_PER_PAGE);
+
+ spin_lock_irqsave(&xol_area->lock, flags);
+ clear_bit(slot_nr, xol_area->bitmap);
+ spin_unlock_irqrestore(&xol_area->lock, flags);
+ found = 1;
+ }
+
+ if (!found)
+ printk(KERN_ERR "%s: no XOL vma for slot address %#lx\n",
+ __func__, slot_addr);
+}
+
+/**
+ * xol_validate_vaddr - Verify if the specified address is in an
+ * executable vma, but not in an XOL vma.
+ * - Return 0 if the specified virtual address is in an
+ * executable vma, but not in an XOL vma.
+ * - Return 1 if the specified virtual address is in an
+ * XOL vma.
+ * - Return -EINTR otherwise.(i.e non executable vma, or
+ * not a valid address
+ * @pid: the probed process
+ * @vaddr: virtual address of the instruction to be validated.
+ * @xol_area refers the unique per process user_bkpt_xol_area for
+ * this process.
+ */
+int xol_validate_vaddr(struct pid *pid, unsigned long vaddr,
+ struct user_bkpt_xol_area *xol_area)
+{
+ struct task_struct *tsk;
+ unsigned long vma_end;
+ int result;
+
+ tsk = pid_task(pid, PIDTYPE_PID);
+ result = user_bkpt_validate_insn_addr(tsk, vaddr);
+ if (result != 0)
+ return result;
+
+ if (unlikely(!xol_area))
+ return 0;
+
+ vma_end = xol_area->vaddr + PAGE_SIZE;
+ if (xol_area->vaddr <= vaddr && vaddr < vma_end)
+ result = 1;
+
+ return result;
+}
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 5/10] Uprobes Implementation
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (3 preceding siblings ...)
2010-05-06 18:02 ` [PATCH v3 4/10] Slot allocation for execution out of line (XOL) Srikar Dronamraju
@ 2010-05-06 18:02 ` Srikar Dronamraju
2010-05-06 18:02 ` [PATCH v3 6/10] x86 support for Uprobes Srikar Dronamraju
` (6 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:02 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, Andrew Morton, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, LKML, Paul E. McKenney
uprobes_core.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Uprobes Implementation
Changelog from v2:
- Introduce TIF_UPROBED flag.
- uprobes hooks now in fork/exec/exit paths instead of tracehooks.
- uprobe_process is now part of the mm struct and is shared between
processes that share the mm.
- per thread information is now allocated on the fly.
* Hence allocation and freeing of this information is lockless.
- For now run the handler only in task context. The reasons for this
change being:
* utask (per-task meta-data structure) is now allocated on the
fly. Hence first request on the thread and first request for the
breakpoint have to be anyway allocated in task context.
* Measurements showed task based handler had negligible
overhead over interrupt based handlers.
* Feedback from Oleg and few others.
* Feedback at LFCS.
* Simplicity atleast till uprobes stabilizes.
( However we may introduce interrupt based handlers at a later time.)
Here are sample overhead figures (in usec) for x86 architecture.
i686: Intel(R) Xeon(TM) CPU 2.40GHz
Without probe module.
100000 interations in 0.000650 sec i.e 0.006500 usec per iteration
With probes and handler run in interrupt context.
100000 interations in 0.340774 sec i.e 3.407740 usec per iteration
probe overhead is 3.401240 usec per probe hit.
With probes and handler run in task context.
100000 interations in 0.365589 sec i.e 3.655890 usec per iteration
probe overhead is 3.649390 usec per probe hit.
x86_64: Intel(R) Xeon(R) CPU X7350 @ 2.93GHz
Without probe module.
100000 interations in 0.000468 sec i.e 0.004680 usec per iteration
With probes and handler run in interrupt context.
100000 interations in 0.120369 sec i.e 1.203690 usec per iteration
Probe overhead is 1.199010 usec per probe hit.
With probes and handler run in task context.
100000 interations in 0.130685 sec i.e 1.306850 usec per iteration
Probe overhead is 1.302170 usec per probe hit.
- find_probept() takes the spinlock; unlike previously when it was
expected that the spinlock was taken before calling it.
Changelog from v1:
- If fixup might sleep; then do the post singlestep
processing in task context.
The uprobes infrastructure enables a user to dynamically establish
probepoints in user applications and collect information by executing a
handler function when a probepoint is hit.
The user specifies the virtual address and the pid of the process of
interest along with the action to be performed (handler). The handle
Uprobes is implemented on the user-space breakpoint assistance layer
and uses the execution out of line strategy. Uprobes follows lazy slot
allocation. I.e, on the first probe hit for that process, a new vma (to
hold the probed instructions for execution out of line) is allocated.
Once allocated, this vma remains for the life of the process, and is
reused as needed for subsequent probes. A slot in the vma is allocated
for a probepoint when it is first hit.
A slot is marked for reuse when the probe gets unregistered and no
threads are using that slot.
In a multithreaded process, a probepoint once registered is active for
all threads of a process. If a thread specific action for a probepoint
is required then the handler should be implemented to do the same.
If a breakpoint already exists at a particular address (irrespective of
who inserted the breakpoint including uprobes), uprobes will refuse to
register any more probes at that address.
You need to follow this up with the uprobes patch for your
architecture.
For more information: please refer to Documentation/uprobes.txt
TODO:
1. Perf/trace events interface for uprobes.
2. Allow multiple probes at a probepoint.
3. Booster probes.
4. Allow probes to be inherited across fork.
5. probing function returns.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
---
arch/Kconfig | 13 +
arch/x86/include/asm/thread_info.h | 2
arch/x86/kernel/signal.c | 5
fs/exec.c | 4
include/linux/mm_types.h | 4
include/linux/sched.h | 3
include/linux/uprobes.h | 169 +++++++++
kernel/Makefile | 1
kernel/fork.c | 20 +
kernel/uprobes.c | 681 ++++++++++++++++++++++++++++++++++++
10 files changed, 902 insertions(+), 0 deletions(-)
create mode 100644 include/linux/uprobes.h
create mode 100644 kernel/uprobes.c
diff --git a/arch/Kconfig b/arch/Kconfig
index 424494e..34ce376 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -62,6 +62,16 @@ config USER_BKPT
This service is used by components such as uprobes.
If in doubt, say "N".
+config UPROBES
+ bool "User-space probes (EXPERIMENTAL)"
+ depends on MODULES && USER_BKPT_XOL
+ depends on HAVE_UPROBES
+ help
+ Uprobes enables kernel modules to establish probepoints
+ in user applications and execute handler functions when
+ the probepoints are hit. For more information, refer to
+ Documentation/uprobes.txt. If in doubt, say "N".
+
config HAVE_EFFICIENT_UNALIGNED_ACCESS
bool
help
@@ -113,6 +123,9 @@ config HAVE_KRETPROBES
config HAVE_OPTPROBES
bool
+
+config HAVE_UPROBES
+ def_bool n
#
# An arch should select this if it provides all these things:
#
diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index e0d2890..5258e69 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -84,6 +84,7 @@ struct thread_info {
#define TIF_SECCOMP 8 /* secure computing */
#define TIF_MCE_NOTIFY 10 /* notify userspace of an MCE */
#define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */
+#define TIF_UPROBE 12 /* breakpointed or singlestepping */
#define TIF_NOTSC 16 /* TSC is not accessible in userland */
#define TIF_IA32 17 /* 32bit process */
#define TIF_FORK 18 /* ret_from_fork */
@@ -108,6 +109,7 @@ struct thread_info {
#define _TIF_SECCOMP (1 << TIF_SECCOMP)
#define _TIF_MCE_NOTIFY (1 << TIF_MCE_NOTIFY)
#define _TIF_USER_RETURN_NOTIFY (1 << TIF_USER_RETURN_NOTIFY)
+#define _TIF_UPROBE (1 << TIF_UPROBE)
#define _TIF_NOTSC (1 << TIF_NOTSC)
#define _TIF_IA32 (1 << TIF_IA32)
#define _TIF_FORK (1 << TIF_FORK)
diff --git a/arch/x86/kernel/signal.c b/arch/x86/kernel/signal.c
index 4fd173c..851bc8d 100644
--- a/arch/x86/kernel/signal.c
+++ b/arch/x86/kernel/signal.c
@@ -848,6 +848,11 @@ do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
if (thread_info_flags & _TIF_SIGPENDING)
do_signal(regs);
+ if (thread_info_flags & _TIF_UPROBE) {
+ clear_thread_flag(TIF_UPROBE);
+ uprobe_notify_resume(regs);
+ }
+
if (thread_info_flags & _TIF_NOTIFY_RESUME) {
clear_thread_flag(TIF_NOTIFY_RESUME);
tracehook_notify_resume(regs);
diff --git a/fs/exec.c b/fs/exec.c
index 49cdaa1..a4b6dba 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -1048,6 +1048,10 @@ void setup_new_exec(struct linux_binprm * bprm)
flush_signal_handlers(current, 0);
flush_old_files(current->files);
+#ifdef CONFIG_UPROBES
+ if (unlikely(current->utask))
+ uprobe_free_utask(current);
+#endif
}
EXPORT_SYMBOL(setup_new_exec);
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index b8bb9a6..b80ffb3 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -14,6 +14,7 @@
#include <linux/page-debug-flags.h>
#include <asm/page.h>
#include <asm/mmu.h>
+#include <linux/uprobes.h>
#ifndef AT_VECTOR_SIZE_ARCH
#define AT_VECTOR_SIZE_ARCH 0
@@ -310,6 +311,9 @@ struct mm_struct {
#ifdef CONFIG_MMU_NOTIFIER
struct mmu_notifier_mm *mmu_notifier_mm;
#endif
+#ifdef CONFIG_UPROBES
+ struct uprobe_process *uproc; /* per mm uprobes meta-data */
+#endif
};
/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index dad7f66..2d2433a 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1506,6 +1506,9 @@ struct task_struct {
unsigned long memsw_bytes; /* uncharged mem+swap usage */
} memcg_batch;
#endif
+#ifdef CONFIG_UPROBES
+ struct uprobe_task *utask; /* per-thread uprobes meta-data */
+#endif
};
/* Future-safe accessor for struct task_struct's cpus_allowed. */
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
new file mode 100644
index 0000000..07444e1
--- /dev/null
+++ b/include/linux/uprobes.h
@@ -0,0 +1,169 @@
+#ifndef _LINUX_UPROBES_H
+#define _LINUX_UPROBES_H
+/*
+ * Userspace Probes (UProbes)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/rcupdate.h>
+#include <linux/mutex.h>
+#include <linux/wait.h>
+#include <linux/spinlock_types.h>
+#include <asm/atomic.h>
+#include <linux/user_bkpt.h>
+#include <linux/user_bkpt_xol.h>
+
+struct task_struct;
+struct pid;
+struct pt_regs;
+
+/* This is what the user supplies us. */
+struct uprobe {
+ /*
+ * The pid of the probed process. Currently, this can be the
+ * thread ID (task->pid) of any active thread in the process.
+ */
+ pid_t pid;
+
+ /* Location of the probepoint */
+ unsigned long vaddr;
+
+ /* Handler to run when the probepoint is hit */
+ void (*handler)(struct uprobe*, struct pt_regs*);
+};
+
+/*
+ * uprobe_process -- not a user-visible struct.
+ * A uprobe_process represents a probed process. A process can have
+ * multiple probepoints (each represented by a uprobe_probept) and
+ * one or more threads (each represented by a uprobe_task).
+ *
+ * All processes/threads that share a mm share the same uprobe_process.
+ */
+struct uprobe_process {
+ /*
+ * mutex locked for any change to the uprobe_process's
+ * graph (including uprobe_probept, taking a slot in xol_area) --
+ * e.g., due to probe [un]registration or special events like exit.
+ */
+ struct mutex mutex;
+
+ /* Table of uprobe_probepts registered for this process */
+ struct list_head uprobe_list;
+
+ atomic_t refcount;
+
+ /* lock held while traversing/modifying uprobe_list and n_ppts */
+ spinlock_t pptlist_lock; /* protects uprobe_list */
+
+ /* number of probept allocated for this process */
+ int n_ppts;
+
+ /*
+ * Manages slots for instruction-copies to be single-stepped
+ * out of line.
+ */
+ void *xol_area;
+};
+
+/*
+ * uprobe_probept -- not a user-visible struct.
+ * A uprobe_probept represents a probepoint.
+ * Guarded by uproc->lock.
+ */
+struct uprobe_probept {
+ /* breakpoint/XOL details */
+ struct user_bkpt user_bkpt;
+
+ /*
+ * ppt goes in the uprobe_process->uprobe_table when registered --
+ * even before the breakpoint has been inserted.
+ */
+ struct list_head ut_node;
+
+ atomic_t refcount;
+
+ /* The parent uprobe_process */
+ struct uprobe_process *uproc;
+
+ struct uprobe *uprobe;
+};
+
+enum uprobe_task_state {
+ UTASK_RUNNING,
+ UTASK_BP_HIT,
+ UTASK_SSTEP
+};
+
+/*
+ * uprobe_utask -- not a user-visible struct.
+ * Corresponds to a thread in a probed process.
+ * Guarded by uproc->mutex.
+ */
+struct uprobe_task {
+ struct user_bkpt_task_arch_info arch_info;
+
+ enum uprobe_task_state state;
+
+ struct uprobe_probept *active_ppt;
+};
+
+#ifdef CONFIG_UPROBES
+extern int uprobes_exception_notify(struct notifier_block *self,
+ unsigned long val, void *data);
+extern int uprobe_bkpt_notifier(struct pt_regs *regs);
+extern int uprobe_post_notifier(struct pt_regs *regs);
+extern void uprobe_notify_resume(struct pt_regs *regs);
+extern void arch_uprobe_enable_sstep(struct pt_regs *regs);
+extern void arch_uprobe_disable_sstep(struct pt_regs *regs);
+extern int register_uprobe(struct uprobe *u);
+extern void unregister_uprobe(struct uprobe *u);
+extern void uprobe_free_utask(struct task_struct *tsk);
+extern void uprobe_handle_fork(struct task_struct *child);
+extern void uprobe_put_uprocess(struct mm_struct *mm);
+#else /* CONFIG_UPROBES */
+
+/*
+ * Only register_uprobe() and unregister_uprobe() are part of
+ * the client API.
+ */
+static inline int register_uprobe(struct uprobe *u)
+{
+ return -ENOSYS;
+}
+static inline void unregister_uprobe(struct uprobe *u)
+{
+}
+static inline void uprobe_free_utask(void)
+{
+}
+static inline void uprobe_handle_fork(struct task_struct *child)
+{
+}
+static inline void uprobe_notify_resume(struct pt_regs *regs)
+{
+}
+static void uprobe_put_uprocess(struct mm_struct *mm)
+{
+}
+#endif /* CONFIG_UPROBES */
+#endif /* _LINUX_UPROBES_H */
diff --git a/kernel/Makefile b/kernel/Makefile
index e404aa0..f0cfb02 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -107,6 +107,7 @@ obj-$(CONFIG_USER_RETURN_NOTIFIER) += user-return-notifier.o
obj-$(CONFIG_PADATA) += padata.o
obj-$(CONFIG_USER_BKPT) += user_bkpt.o
obj-$(CONFIG_USER_BKPT_XOL) += user_bkpt_xol.o
+obj-$(CONFIG_UPROBES) += uprobes.o
ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
# According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
diff --git a/kernel/fork.c b/kernel/fork.c
index 44b0791..035ab06 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -173,6 +173,10 @@ void __put_task_struct(struct task_struct *tsk)
exit_creds(tsk);
delayacct_tsk_free(tsk);
+#ifdef CONFIG_UPROBES
+ if (unlikely(tsk->utask))
+ uprobe_free_utask(tsk);
+#endif
if (!profile_handoff_task(tsk))
free_task(tsk);
@@ -509,6 +513,10 @@ void __mmdrop(struct mm_struct *mm)
mm_free_pgd(mm);
destroy_context(mm);
mmu_notifier_mm_destroy(mm);
+#ifdef CONFIG_UPROBES
+ if (unlikely(mm->uproc))
+ uprobe_put_uprocess(mm);
+#endif
free_mm(mm);
}
EXPORT_SYMBOL_GPL(__mmdrop);
@@ -667,6 +675,9 @@ struct mm_struct *dup_mm(struct task_struct *tsk)
if (mm->binfmt && !try_module_get(mm->binfmt->module))
goto free_pt;
+#ifdef CONFIG_UPROBES
+ mm->uproc = NULL;
+#endif
return mm;
free_pt:
@@ -1183,6 +1194,9 @@ static struct task_struct *copy_process(unsigned long clone_flags,
INIT_LIST_HEAD(&p->pi_state_list);
p->pi_state_cache = NULL;
#endif
+#ifdef CONFIG_UPROBES
+ p->utask = NULL;
+#endif
/*
* sigaltstack should be cleared when sharing the same VM
*/
@@ -1281,6 +1295,12 @@ static struct task_struct *copy_process(unsigned long clone_flags,
proc_fork_connector(p);
cgroup_post_fork(p);
perf_event_fork(p);
+#ifdef CONFIG_UPROBES
+ if ((current->mm) && !(clone_flags & CLONE_VM)) {
+ if (unlikely(current->mm->uproc))
+ uprobe_handle_fork(p);
+ }
+#endif
return p;
bad_fork_free_pid:
diff --git a/kernel/uprobes.c b/kernel/uprobes.c
new file mode 100644
index 0000000..c40c848
--- /dev/null
+++ b/kernel/uprobes.c
@@ -0,0 +1,681 @@
+/*
+ * Userspace Probes (UProbes)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+#include <linux/types.h>
+#include <linux/hash.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/err.h>
+#include <linux/uprobes.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <linux/errno.h>
+#include <linux/kdebug.h>
+#include <linux/slab.h>
+
+static u16 user_bkpt_strategies;
+
+struct notifier_block uprobes_exception_nb = {
+ .notifier_call = uprobes_exception_notify,
+ .priority = 0x7ffffff0,
+};
+
+typedef void (*uprobe_handler_t)(struct uprobe*, struct pt_regs*);
+
+/* Guards lookup, creation, and deletion of uproc. */
+static DEFINE_MUTEX(uprobe_mutex);
+
+static inline void get_probept(struct uprobe_probept *ppt)
+{
+ atomic_inc(&ppt->refcount);
+}
+
+/*
+ * Creates a uprobe_probept and connects it to uprobe and uproc.
+ * Runs with uproc->mutex locked.
+ */
+static struct uprobe_probept *add_probept(struct uprobe *u,
+ struct uprobe_process *uproc)
+{
+ struct uprobe_probept *ppt;
+
+ ppt = kzalloc(sizeof *ppt, GFP_USER);
+ if (unlikely(ppt == NULL))
+ return ERR_PTR(-ENOMEM);
+
+ ppt->user_bkpt.vaddr = u->vaddr;
+ ppt->uprobe = u;
+ ppt->user_bkpt.xol_vaddr = 0;
+
+ ppt->user_bkpt.strategy = user_bkpt_strategies;
+
+ ppt->uproc = uproc;
+ INIT_LIST_HEAD(&ppt->ut_node);
+ spin_lock(&uproc->pptlist_lock);
+ list_add(&ppt->ut_node, &uproc->uprobe_list);
+ uproc->n_ppts++;
+ spin_unlock(&uproc->pptlist_lock);
+ atomic_set(&ppt->refcount, 1);
+ return ppt;
+}
+
+static void put_probept(struct uprobe_probept *ppt)
+{
+ struct uprobe_process *uproc;
+
+ uproc = ppt->uproc;
+ if (atomic_dec_and_lock(&ppt->refcount, &uproc->pptlist_lock)) {
+ list_del(&ppt->ut_node);
+ uproc->n_ppts--;
+ xol_free_insn_slot(ppt->user_bkpt.xol_vaddr, uproc->xol_area);
+ spin_unlock(&uproc->pptlist_lock);
+ kfree(ppt);
+ }
+}
+
+/*
+ * In the given uproc's hash table of probepoints, find the one with the
+ * specified virtual address.
+ * Called with uproc->pptlist_lock acquired.
+ */
+static struct uprobe_probept *find_probept(struct uprobe_process *uproc,
+ unsigned long vaddr)
+{
+ struct uprobe_probept *ppt;
+
+ spin_lock(&uproc->pptlist_lock);
+ list_for_each_entry(ppt, &uproc->uprobe_list, ut_node) {
+ if (ppt->user_bkpt.vaddr == vaddr) {
+ spin_unlock(&uproc->pptlist_lock);
+ return ppt;
+ }
+ }
+ spin_unlock(&uproc->pptlist_lock);
+ return NULL;
+}
+
+/*
+ * Save a copy of the original instruction (so it can be single-stepped
+ * out of line), insert the breakpoint instruction.
+ * Runs with uproc->mutex locked.
+ */
+static int insert_bkpt(struct uprobe_probept *ppt, struct task_struct *tsk)
+{
+ int result;
+
+ if (tsk)
+ result = user_bkpt_insert_bkpt(tsk, &ppt->user_bkpt);
+ else
+ /* No surviving tasks associated with ppt->uproc */
+ result = -ESRCH;
+ return result;
+}
+
+ /* Runs with uproc->mutex locked. */
+static void remove_bkpt(struct uprobe_probept *ppt, struct task_struct *tsk)
+{
+ if (!tsk)
+ return;
+
+ if (user_bkpt_remove_bkpt(tsk, &ppt->user_bkpt) != 0) {
+ printk(KERN_ERR "Error removing uprobe at pid %d vaddr %#lx:"
+ " can't restore original instruction\n",
+ tsk->tgid, ppt->user_bkpt.vaddr);
+ /*
+ * This shouldn't happen, since we were previously able
+ * to write the breakpoint at that address. There's not
+ * much we can do besides let the process die with a
+ * SIGTRAP the next time the breakpoint is hit.
+ */
+ }
+}
+
+/* Runs with the uprobe_mutex held. */
+static struct uprobe_process *find_uprocess(struct pid *tg_leader)
+{
+ struct uprobe_process *uproc = NULL;
+ struct task_struct *tsk;
+
+ rcu_read_lock();
+ tsk = pid_task(tg_leader, PIDTYPE_PID);
+ if (!tsk || !tsk->mm)
+ goto end;
+
+ uproc = tsk->mm->uproc;
+ if (uproc)
+ atomic_inc(&uproc->refcount);
+
+end:
+ rcu_read_unlock();
+ return uproc;
+}
+
+/*
+ * uproc's process is exiting or exec-ing.
+ * The last thread of uproc's process is about to die, and its
+ * mm_struct is about to be released.
+ * Hence do the cleanup without holding locks.
+ *
+ * Called with no locks held.
+ */
+static int free_uprocess(struct uprobe_process *uproc)
+{
+ struct uprobe_probept *ppt, *pnode;
+
+ list_for_each_entry_safe(ppt, pnode, &uproc->uprobe_list, ut_node) {
+ put_probept(ppt);
+ }
+ if (uproc->xol_area)
+ xol_free_area(uproc->xol_area);
+
+ kfree(uproc);
+ return 0;
+}
+
+/* Called with no locks held */
+static void put_uprocess(struct uprobe_process *uproc)
+{
+ if (atomic_dec_and_test(&uproc->refcount))
+ free_uprocess(uproc);
+}
+
+/*
+ * Called with no locks held.
+ * Called in context of a exiting or a exec-ing thread.
+ */
+void uprobe_free_utask(struct task_struct *tsk)
+{
+ if (!tsk->utask)
+ return;
+
+ if (tsk->utask->active_ppt)
+ put_probept(tsk->utask->active_ppt);
+ kfree(tsk->utask);
+ tsk->utask = NULL;
+}
+
+/*
+ * Callback from mmput() when mm->users count reduces to zero.
+ */
+void uprobe_put_uprocess(struct mm_struct *mm)
+{
+ put_uprocess(mm->uproc);
+ mm->uproc = NULL;
+}
+
+/*
+ * Allocate a uprobe_task object for the task.
+ * Called with t "got" and uprobe_mutex locked.
+ * Called when the thread hits a breakpoint for the first time.
+ *
+ * Returns:
+ * - pointer to new uprobe_task on success
+ * - negative errno otherwise
+ */
+static struct uprobe_task *add_utask(struct uprobe_process *uproc)
+{
+ struct uprobe_task *utask;
+
+ utask = kzalloc(sizeof *utask, GFP_KERNEL);
+ if (unlikely(utask == NULL))
+ return ERR_PTR(-ENOMEM);
+
+ utask->active_ppt = NULL;
+ current->utask = utask;
+ atomic_inc(&uproc->refcount);
+
+ return utask;
+}
+
+/* Runs with uprobe_mutex held; */
+static struct uprobe_process *create_uprocess(struct pid *tg_leader)
+{
+ struct uprobe_process *uproc = ERR_PTR(-ENOMEM);
+ struct task_struct *tsk;
+ struct mm_struct *mm = NULL;
+
+ tsk = get_pid_task(tg_leader, PIDTYPE_PID);
+ if (tsk)
+ mm = get_task_mm(tsk);
+ if (!mm) {
+ if (tsk)
+ put_task_struct(tsk);
+ return ERR_PTR(-ESRCH);
+ }
+
+ uproc = kzalloc(sizeof *uproc, GFP_KERNEL);
+ if (unlikely(uproc == NULL)) {
+ uproc = ERR_PTR(-ENOMEM);
+ goto end;
+ }
+
+ /* Initialize fields */
+ mutex_init(&uproc->mutex);
+ spin_lock_init(&uproc->pptlist_lock);
+ atomic_set(&uproc->refcount, 1);
+ INIT_LIST_HEAD(&uproc->uprobe_list);
+
+ BUG_ON(mm->uproc);
+ mm->uproc = uproc;
+
+ /*
+ * Incrementing the refcount saves us from calling find_uprocess
+ * in register_uprobe path.
+ */
+ atomic_inc(&uproc->refcount);
+
+end:
+ put_task_struct(tsk);
+ mmput(mm);
+ return uproc;
+}
+
+/*
+ * Given a numeric thread ID, return a ref-counted struct pid for the
+ * task-group-leader thread.
+ */
+static struct pid *get_tg_leader(pid_t p)
+{
+ struct pid *pid = NULL;
+
+ rcu_read_lock();
+ pid = find_vpid(p);
+ if (pid) {
+ struct task_struct *t = pid_task(pid, PIDTYPE_PID);
+
+ if (!t)
+ pid = NULL;
+ else
+ pid = get_pid(task_tgid(t));
+ }
+ rcu_read_unlock();
+ return pid;
+}
+
+/* See Documentation/uprobes.txt. */
+int register_uprobe(struct uprobe *u)
+{
+ struct uprobe_process *uproc;
+ struct uprobe_probept *ppt;
+ struct pid *p;
+ int ret = 0;
+
+ if (!u || !u->handler)
+ return -EINVAL;
+
+ p = get_tg_leader(u->pid);
+ if (!p)
+ return -ESRCH;
+
+ /* Get the uprobe_process for this pid, or make a new one. */
+ mutex_lock(&uprobe_mutex);
+ uproc = find_uprocess(p);
+
+ if (!uproc) {
+ uproc = create_uprocess(p);
+ if (IS_ERR(uproc)) {
+ ret = (int) PTR_ERR(uproc);
+ mutex_unlock(&uprobe_mutex);
+ goto fail_tsk;
+ }
+ }
+ mutex_unlock(&uprobe_mutex);
+ mutex_lock(&uproc->mutex);
+
+ if (uproc->n_ppts >= MAX_USER_BKPT_XOL_SLOTS)
+ goto fail_uproc;
+
+ ret = xol_validate_vaddr(p, u->vaddr, uproc->xol_area);
+ if (ret < 0)
+ goto fail_uproc;
+
+ /* See if we already have a probepoint at the vaddr. */
+ ppt = find_probept(uproc, u->vaddr);
+ if (ppt) {
+ /*
+ * A uprobe already exists at that address.
+ */
+ ret = -EALREADY;
+ goto fail_uproc;
+ } else {
+ ppt = add_probept(u, uproc);
+ if (IS_ERR(ppt)) {
+ ret = (int) PTR_ERR(ppt);
+ goto fail_uproc;
+ }
+ ret = insert_bkpt(ppt, pid_task(p, PIDTYPE_PID));
+ if (ret != 0)
+ goto fail_uproc;
+ }
+
+fail_uproc:
+ mutex_unlock(&uproc->mutex);
+ put_uprocess(uproc);
+
+fail_tsk:
+ put_pid(p);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(register_uprobe);
+
+/* See Documentation/uprobes.txt. */
+void unregister_uprobe(struct uprobe *u)
+{
+ struct uprobe_process *uproc;
+ struct uprobe_probept *ppt;
+ struct pid *p;
+
+ if (!u)
+ return;
+ p = get_tg_leader(u->pid);
+ if (!p)
+ return;
+
+ /* Get the uprobe_process for this pid. */
+ mutex_lock(&uprobe_mutex);
+ uproc = find_uprocess(p);
+ mutex_unlock(&uprobe_mutex);
+ if (!uproc) {
+ put_pid(p);
+ return;
+ }
+
+ /*
+ * Lock uproc before walking the graph, in case the process
+ * we're probing is exiting.
+ */
+ mutex_lock(&uproc->mutex);
+
+ ppt = find_probept(uproc, u->vaddr);
+ if (!ppt)
+ /*
+ * This probe was never successfully registered, or
+ * has already been unregistered.
+ */
+ goto done;
+
+ if (ppt->uprobe != u)
+ /*
+ * unregister request doesnt correspond to successful
+ * register request.
+ */
+ goto done;
+
+ remove_bkpt(ppt, pid_task(p, PIDTYPE_PID));
+
+ /*
+ * Breakpoint is removed; however a thread could have hit the
+ * same breakpoint and yet to find its corresponding probepoint.
+ * Before we remove the probepoint, give the breakpointed thread a
+ * chance to find the probepoint.
+ */
+ mutex_unlock(&uproc->mutex);
+ synchronize_sched();
+ mutex_lock(&uproc->mutex);
+ put_probept(ppt);
+
+done:
+ mutex_unlock(&uproc->mutex);
+ put_uprocess(uproc);
+ put_pid(p);
+}
+EXPORT_SYMBOL_GPL(unregister_uprobe);
+
+/* Prepare to single-step ppt's probed instruction out of line. */
+static int pre_ssout(struct uprobe_probept *ppt, struct pt_regs *regs)
+{
+ struct uprobe_process *uproc = current->mm->uproc;
+
+ if (unlikely(!ppt->user_bkpt.xol_vaddr)) {
+ mutex_lock(&uproc->mutex);
+ if (unlikely(!uproc->xol_area))
+ uproc->xol_area = xol_alloc_area();
+ if (uproc->xol_area && !ppt->user_bkpt.xol_vaddr)
+ xol_get_insn_slot(&ppt->user_bkpt, uproc->xol_area);
+ if (unlikely(!ppt->user_bkpt.xol_vaddr))
+ goto fail;
+ mutex_unlock(&uproc->mutex);
+ }
+ user_bkpt_pre_sstep(current, &ppt->user_bkpt,
+ ¤t->utask->arch_info, regs);
+ user_bkpt_set_ip(regs, ppt->user_bkpt.xol_vaddr);
+ return 0;
+
+/*
+ * We failed to execute out of line.
+ * reset the instruction pointer and remove the breakpoint.
+ */
+fail:
+ remove_bkpt(ppt, current);
+ mutex_unlock(&uproc->mutex);
+ user_bkpt_set_ip(regs, ppt->user_bkpt.vaddr);
+ put_probept(ppt);
+ return -1;
+}
+
+/* Prepare to continue execution after single-stepping out of line. */
+static int post_ssout(struct uprobe_probept *ppt, struct pt_regs *regs)
+{
+ return user_bkpt_post_sstep(current, &ppt->user_bkpt,
+ ¤t->utask->arch_info, regs);
+}
+
+/*
+ * Verify from Instruction Pointer if singlestep has indeed occurred.
+ * If Singlestep has occurred, then do post singlestep fix-ups.
+ */
+static bool sstep_complete(struct pt_regs *regs,
+ struct uprobe_probept *ppt)
+{
+ unsigned long vaddr = instruction_pointer(regs);
+
+ /*
+ * If we have executed out of line, Instruction pointer
+ * cannot be same as virtual address of XOL slot.
+ */
+ if (vaddr == ppt->user_bkpt.xol_vaddr)
+ return false;
+ post_ssout(ppt, regs);
+ return true;
+}
+
+/*
+ * Fork callback: The current task has spawned a process.
+ * NOTE: For now, we don't pass on uprobes from the parent to the
+ * child. We now do the necessary clearing of breakpoints in the
+ * child's address space.
+ * This function handles the case where vm is not shared between
+ * the parent and the child.
+ *
+ * TODO:
+ * - Provide option for child to inherit uprobes.
+ */
+void uprobe_handle_fork(struct task_struct *child)
+{
+ struct uprobe_process *uproc;
+ struct uprobe_probept *ppt;
+ int ret;
+
+ uproc = current->mm->uproc;
+
+ /*
+ * New process spawned by parent but not sharing the same mm.
+ * Remove the probepoints in the child's text.
+ *
+ * We also hold the uproc->mutex for the parent - so no
+ * new uprobes will be registered 'til we return.
+ */
+ mutex_lock(&uproc->mutex);
+ list_for_each_entry(ppt, &uproc->uprobe_list, ut_node) {
+ ret = user_bkpt_remove_bkpt(child, &ppt->user_bkpt);
+ if (ret && ret != -EINVAL) {
+ /* Ratelimit this? */
+ printk(KERN_ERR "Pid %d forked %d; failed to"
+ " remove probepoint at %#lx in child\n",
+ current->pid, child->pid,
+ ppt->user_bkpt.vaddr);
+ }
+ }
+ mutex_unlock(&uproc->mutex);
+}
+
+/*
+ * uprobe_notify_resume gets called in task context just before returning
+ * to userspace.
+ *
+ * If its the first time the probepoint is hit, slot gets allocated here.
+ * If its the first time the thread hit a breakpoint, utask gets
+ * allocated here.
+ */
+void uprobe_notify_resume(struct pt_regs *regs)
+{
+ struct uprobe_process *uproc;
+ struct uprobe_probept *ppt;
+ struct uprobe_task *utask;
+ struct uprobe *u;
+ unsigned long probept;
+
+ utask = current->utask;
+ uproc = current->mm->uproc;
+ if (unlikely(!utask)) {
+ utask = add_utask(uproc);
+
+ /* Failed to allocate utask for the current task. */
+ BUG_ON(!utask);
+ probept = user_bkpt_get_bkpt_addr(regs);
+ ppt = find_probept(uproc, probept);
+
+ /*
+ * The probept was refcounted in uprobe_bkpt_notifier;
+ * Hence it would be mysterious to miss ppt now
+ */
+ WARN_ON(!ppt);
+ utask->active_ppt = ppt;
+ utask->state = UTASK_BP_HIT;
+ } else
+ ppt = utask->active_ppt;
+
+ if (utask->state == UTASK_BP_HIT) {
+ utask->state = UTASK_SSTEP;
+ u = ppt->uprobe;
+ if (u && u->handler)
+ u->handler(u, regs);
+
+ if (!pre_ssout(ppt, regs))
+ arch_uprobe_enable_sstep(regs);
+ } else if (utask->state == UTASK_SSTEP) {
+ if (sstep_complete(regs, ppt)) {
+ put_probept(ppt);
+ utask->active_ppt = NULL;
+ utask->state = UTASK_RUNNING;
+ arch_uprobe_disable_sstep(regs);
+ }
+ }
+}
+
+/*
+ * uprobe_bkpt_notifier gets called from interrupt context
+ * it gets a reference to the ppt and sets TIF_UPROBE flag,
+ */
+int uprobe_bkpt_notifier(struct pt_regs *regs)
+{
+ struct uprobe_process *uproc;
+ struct uprobe_probept *ppt;
+ struct uprobe_task *utask;
+ unsigned long probept;
+
+ if (!current->mm || !current->mm->uproc)
+ /* task is currently not uprobed */
+ return 0;
+
+ uproc = current->mm->uproc;
+ utask = current->utask;
+ probept = user_bkpt_get_bkpt_addr(regs);
+ ppt = find_probept(uproc, probept);
+ if (!ppt)
+ return 0;
+ get_probept(ppt);
+ if (utask) {
+ utask->active_ppt = ppt;
+ utask->state = UTASK_BP_HIT;
+ }
+ set_thread_flag(TIF_UPROBE);
+ return 1;
+}
+
+/*
+ * uprobe_post_notifier gets called in interrupt context.
+ * It completes the single step operation.
+ */
+int uprobe_post_notifier(struct pt_regs *regs)
+{
+ struct uprobe_probept *ppt;
+ struct uprobe_task *utask;
+
+ if (!current->mm || !current->mm->uproc || !current->utask)
+ /* task is currently not uprobed */
+ return 0;
+
+ utask = current->utask;
+
+ ppt = utask->active_ppt;
+ if (!ppt)
+ return 0;
+
+ if (user_bkpt_resume_can_sleep(&ppt->user_bkpt)) {
+ set_thread_flag(TIF_UPROBE);
+ return 1;
+ }
+ if (sstep_complete(regs, ppt)) {
+ put_probept(ppt);
+ arch_uprobe_disable_sstep(regs);
+ utask->active_ppt = NULL;
+ utask->state = UTASK_RUNNING;
+ return 1;
+ }
+ return 0;
+}
+
+
+static int __init init_uprobes(void)
+{
+ int ret;
+
+ user_bkpt_strategies = USER_BKPT_HNT_TSKINFO;
+ ret = user_bkpt_init(&user_bkpt_strategies);
+ if (ret != 0) {
+ printk(KERN_ERR "Can't start uprobes: user_bkpt_init() returned %d\n",
+ ret);
+ return ret;
+ }
+
+ register_die_notifier(&uprobes_exception_nb);
+ return 0;
+}
+
+static void __exit exit_uprobes(void)
+{
+}
+
+module_init(init_uprobes);
+module_exit(exit_uprobes);
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 6/10] x86 support for Uprobes
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (4 preceding siblings ...)
2010-05-06 18:02 ` [PATCH v3 5/10] Uprobes Implementation Srikar Dronamraju
@ 2010-05-06 18:02 ` Srikar Dronamraju
2010-05-06 18:03 ` [PATCH v3 7/10] samples: Uprobes samples Srikar Dronamraju
` (5 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:02 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
uprobes_x86.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
X86 support for Uprobes
This patch provides x86 specific details for uprobes.
This includes interrupt notifier for uprobes, enabling/disabling
singlestep.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
---
arch/x86/Kconfig | 1 +
arch/x86/kernel/Makefile | 1 +
arch/x86/kernel/signal.c | 12 +++++++
arch/x86/kernel/uprobes.c | 77 +++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 91 insertions(+), 0 deletions(-)
create mode 100644 arch/x86/kernel/uprobes.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 7dce185..57be1fa 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -54,6 +54,7 @@ config X86
select HAVE_KERNEL_LZO
select HAVE_HW_BREAKPOINT
select HAVE_USER_BKPT
+ select HAVE_UPROBES
select PERF_EVENTS
select ANON_INODES
select HAVE_ARCH_KMEMCHECK
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 98c74b4..bfa48f0 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -118,6 +118,7 @@ obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_USER_BKPT) += user_bkpt.o
+obj-$(CONFIG_UPROBES) += uprobes.o
###
# 64 bit specific files
diff --git a/arch/x86/kernel/signal.c b/arch/x86/kernel/signal.c
index 851bc8d..dde9a42 100644
--- a/arch/x86/kernel/signal.c
+++ b/arch/x86/kernel/signal.c
@@ -850,7 +850,19 @@ do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
if (thread_info_flags & _TIF_UPROBE) {
clear_thread_flag(TIF_UPROBE);
+#ifdef CONFIG_X86_32
+ /*
+ * On x86_32, do_notify_resume() gets called with
+ * interrupts disabled. Hence enable interrupts if they
+ * are still disabled.
+ */
+ native_irq_enable();
+#endif
uprobe_notify_resume(regs);
+
+#ifdef CONFIG_X86_32
+ native_irq_disable();
+#endif
}
if (thread_info_flags & _TIF_NOTIFY_RESUME) {
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
new file mode 100644
index 0000000..5643b96
--- /dev/null
+++ b/arch/x86/kernel/uprobes.c
@@ -0,0 +1,77 @@
+/*
+ * Userspace Probes (UProbes)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2010
+ * Authors:
+ * Srikar Dronamraju
+ * Ananth N Mavinakayanahalli
+ */
+
+#include <linux/sched.h>
+#include <linux/kdebug.h>
+#include <linux/uprobes.h>
+
+/*
+ * Wrapper routine for handling exceptions.
+ */
+int uprobes_exception_notify(struct notifier_block *self,
+ unsigned long val, void *data)
+{
+ struct die_args *args = data;
+ struct pt_regs *regs = args->regs;
+ int ret = NOTIFY_DONE;
+
+ /* We are only interested in userspace traps */
+ if (regs && !user_mode_vm(regs))
+ return NOTIFY_DONE;
+
+ switch (val) {
+ case DIE_INT3:
+ /* Run your handler here */
+ if (uprobe_bkpt_notifier(regs))
+ ret = NOTIFY_STOP;
+ break;
+ case DIE_DEBUG:
+ if (uprobe_post_notifier(regs))
+ ret = NOTIFY_STOP;
+ default:
+ break;
+ }
+ return ret;
+}
+
+void arch_uprobe_enable_sstep(struct pt_regs *regs)
+{
+ /*
+ * Enable single-stepping by
+ * - Set TF on stack
+ * - Set TIF_SINGLESTEP: Guarantees that TF is set when
+ * returning to user mode.
+ * - Indicate that TF is set by us.
+ */
+ regs->flags |= X86_EFLAGS_TF;
+ set_thread_flag(TIF_SINGLESTEP);
+ set_thread_flag(TIF_FORCED_TF);
+}
+
+void arch_uprobe_disable_sstep(struct pt_regs *regs)
+{
+ /* Disable single-stepping by clearing what we set */
+ clear_thread_flag(TIF_SINGLESTEP);
+ clear_thread_flag(TIF_FORCED_TF);
+ regs->flags &= ~X86_EFLAGS_TF;
+}
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 7/10] samples: Uprobes samples
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (5 preceding siblings ...)
2010-05-06 18:02 ` [PATCH v3 6/10] x86 support for Uprobes Srikar Dronamraju
@ 2010-05-06 18:03 ` Srikar Dronamraju
2010-05-06 18:03 ` [PATCH v3 8/10] Uprobes documentation Srikar Dronamraju
` (4 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:03 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, Andrew Morton, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, LKML, Paul E. McKenney
samples.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Uprobes Samples
This provides an example uprobes module in the samples directory.
To run this module run (as root)
insmod uprobe_example.ko vaddr=<vaddr> pid=<pid>
Where <vaddr> is the address where we want to place the probe.
<pid> is the pid of the process we are interested to probe.
example: -
# cd samples/uprobes
[get the virtual address to place the probe.]
# vaddr=0x$(objdump -T /bin/bash |awk '/echo_builtin/ {print $1}')
[Run a bash shell in the background; have it echo 4 lines.]
# (sleep 10; echo 1; echo 2; echo 3; echo 4) &
[Probe calls echo_builtin() in the background bash process.]
# insmod uprobe_example.ko vaddr=$vaddr pid=$!
# sleep 10
# rmmod uprobe_example
# dmesg | tail -n 3
Registering uprobe on pid 10875, vaddr 0x45aa30
Unregistering uprobe on pid 10875, vaddr 0x45aa30
Probepoint was hit 4 times
#
[ Output shows that echo_builtin function was hit 4 times. ]
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
samples/Kconfig | 7 +++
samples/Makefile | 2 -
samples/uprobes/Makefile | 17 ++++++++
samples/uprobes/uprobe_example.c | 83 ++++++++++++++++++++++++++++++++++++++
4 files changed, 108 insertions(+), 1 deletions(-)
create mode 100644 samples/uprobes/Makefile
create mode 100644 samples/uprobes/uprobe_example.c
diff --git a/samples/Kconfig b/samples/Kconfig
index 8924f72..50b8b1c 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -44,4 +44,11 @@ config SAMPLE_HW_BREAKPOINT
help
This builds kernel hardware breakpoint example modules.
+config SAMPLE_UPROBES
+ tristate "Build uprobes example -- loadable module only"
+ depends on UPROBES && m
+ help
+ This builds uprobes example module.
+
+
endif # SAMPLES
diff --git a/samples/Makefile b/samples/Makefile
index 0f15e6d..c8fe6f9 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -1,4 +1,4 @@
# Makefile for Linux samples code
obj-$(CONFIG_SAMPLES) += kobject/ kprobes/ tracepoints/ trace_events/ \
- hw_breakpoint/
+ hw_breakpoint/ uprobes/
diff --git a/samples/uprobes/Makefile b/samples/uprobes/Makefile
new file mode 100644
index 0000000..f535f6f
--- /dev/null
+++ b/samples/uprobes/Makefile
@@ -0,0 +1,17 @@
+# builds the uprobes example kernel modules;
+# then to use one (as root):
+# insmod <module_name.ko> vaddr=<vaddr> pid=<pid>
+#
+#
+# example: -
+# vaddr=0x$(objdump -T /bin/bash |awk '/echo_builtin/ print $1}')
+# (sleep 10; echo 1; echo 2; echo 3; echo 4) &
+# insmod uprobe_example.ko vaddr=$vaddr pid=$!
+# sleep 10
+# rmmod uprobe_example
+# dmesg | tail -n 3
+# Registering uprobe on pid 3920, vaddr 0x45aa30
+# Unregistering uprobe on pid 3920, vaddr 0x45aa30
+# Probepoint was hit 4 times
+
+obj-$(CONFIG_SAMPLE_UPROBES) += uprobe_example.o
diff --git a/samples/uprobes/uprobe_example.c b/samples/uprobes/uprobe_example.c
new file mode 100644
index 0000000..f625bae
--- /dev/null
+++ b/samples/uprobes/uprobe_example.c
@@ -0,0 +1,83 @@
+/*
+ * Uprobes Example
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2008-2010
+ * Authors:
+ * Srikar Dronamraju
+ * Jim Keniston
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/uprobes.h>
+
+/*
+ * Usage: insmod uprobe_example.ko pid=<pid> vaddr=<address> [verbose=0]
+ * where <pid> identifies the probed process and <address> is the virtual
+ * address of the probed instruction.
+ */
+
+static int pid;
+module_param(pid, int, 0);
+MODULE_PARM_DESC(pid, "pid");
+
+static int verbose;
+module_param(verbose, int, 0);
+MODULE_PARM_DESC(verbose, "verbose");
+
+static long vaddr;
+module_param(vaddr, long, 0);
+MODULE_PARM_DESC(vaddr, "vaddr");
+
+static int nhits;
+static struct uprobe usp;
+
+static void uprobe_handler(struct uprobe *u, struct pt_regs *regs)
+{
+ nhits++;
+ if (verbose)
+ printk(KERN_INFO "Hit #%d on probepoint at %#lx\n",
+ nhits, u->vaddr);
+}
+
+int __init init_module(void)
+{
+ int ret;
+ usp.pid = pid;
+ usp.vaddr = vaddr;
+ usp.handler = uprobe_handler;
+ printk(KERN_INFO "Registering uprobe on pid %d, vaddr %#lx\n",
+ usp.pid, usp.vaddr);
+ ret = register_uprobe(&usp);
+ if (ret != 0) {
+ printk(KERN_ERR "register_uprobe() failed, returned %d\n", ret);
+ printk(KERN_ERR "Usage: insmod uprobe_example.ko pid=<pid> "
+ "vaddr=<address>\n");
+ return ret;
+ }
+ return 0;
+}
+
+void __exit cleanup_module(void)
+{
+ printk(KERN_INFO "Unregistering uprobe on pid %d, vaddr %#lx\n",
+ usp.pid, usp.vaddr);
+ printk(KERN_INFO "Probepoint was hit %d times\n", nhits);
+ unregister_uprobe(&usp);
+}
+MODULE_LICENSE("GPL");
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 8/10] Uprobes documentation.
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (6 preceding siblings ...)
2010-05-06 18:03 ` [PATCH v3 7/10] samples: Uprobes samples Srikar Dronamraju
@ 2010-05-06 18:03 ` Srikar Dronamraju
2010-05-06 18:03 ` [PATCH v3 9/10] trace: uprobes trace_event interface Srikar Dronamraju
` (3 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:03 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
uprobes-documentation.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Uprobes documentation.
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
Documentation/uprobes.txt | 236 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 236 insertions(+), 0 deletions(-)
create mode 100644 Documentation/uprobes.txt
diff --git a/Documentation/uprobes.txt b/Documentation/uprobes.txt
new file mode 100644
index 0000000..1608fc2
--- /dev/null
+++ b/Documentation/uprobes.txt
@@ -0,0 +1,236 @@
+Title : User-Space Probes (Uprobes)
+Authors : Jim Keniston <jkenisto@us.ibm.com>
+ : Srikar Dronamraju <srikar@linux.vnet.ibm.com>
+
+CONTENTS
+
+1. Concepts: Uprobes
+2. Architectures Supported
+3. Configuring Uprobes
+4. API Reference
+5. Uprobes Features and Limitations
+6. Probe Overhead
+7. TODO
+8. Uprobes Team
+9. Uprobes Example
+
+1. Concepts: Uprobes
+
+Uprobes enables you to dynamically break into any routine in a
+user application and collect debugging and performance information
+non-disruptively. You can trap at any code address, specifying a
+kernel handler routine to be invoked when the breakpoint is hit.
+
+A uprobe can be inserted on any instruction in the application's
+virtual address space. The registration function register_uprobe()
+specifies which process is to be probed, where the probe is to be
+inserted, and what handler is to be called when the probe is hit.
+
+Uprobes-based instrumentation can be packaged as a kernel
+module. In the simplest case, the module's init function installs
+("registers") one or more probes, and the exit function unregisters
+them.
+
+1.1 How Does a Uprobe Work?
+
+When a uprobe is registered, Uprobes makes a copy of the probed
+instruction, stops the probed application, replaces the first byte(s)
+of the probed instruction with a breakpoint instruction (e.g., int3
+on i386 and x86_64), and allows the probed application to continue.
+(When inserting the breakpoint, Uprobes uses background page
+replacement mechanism, so that the breakpoint affects only that
+process, and not any other process running that program. This is
+true even if the probed instruction is in a shared library.)
+
+When a CPU hits the breakpoint instruction, a trap occurs, the CPU's
+user-mode registers are saved, and uprobes notifier code finds the
+associated uprobe. It then executes the handler associated with the
+uprobe, passing the handler the addresses of the uprobe struct and the
+saved registers. The handler can run either in interrupt context or in
+task context; this is specified by the user at the time of registration.
+When run in task context, the handler may block, but keep in mind that
+the probed thread remains stopped while your handler runs.
+
+Next, Uprobes single-steps its copy of the probed instruction and
+resumes execution of the probed process at the instruction following
+the probepoint. (It would be simpler to single-step the actual
+instruction in place, but then Uprobes would have to temporarily
+remove the breakpoint instruction. This would create problems in a
+multithreaded application. For example, it would open a time window
+when another thread could sail right past the probepoint.)
+
+Instruction copies to be single-stepped are stored in a per-process
+"execution out of line (XOL) area," which is a little VM area
+created by Uprobes in each probed process's address space.
+
+Uprobes handles interesting events in the lifetime of the probed
+process, such as fork, clone, exec, and exit.
+
+1.2 Multithreaded Applications
+
+Uprobes supports the probing of multithreaded applications. Uprobes
+imposes no limit on the number of threads in a probed application.
+All threads in a process use the same text pages, so every probe
+in a process affects all threads; of course, each thread hits the
+probepoint (and runs the handler) independently. Multiple threads
+may run the same handler simultaneously. If you want a particular
+thread or set of threads to run a particular handler, your handler
+should check current or current->pid to determine which thread has
+hit the probepoint.
+
+When a process clones a new thread, that thread automatically shares
+all current and future probes established for that process.
+
+2. Architectures Supported
+
+This user_bkpt based version of Uprobes is implemented on the following
+architectures:
+
+- x86
+
+3. Configuring Uprobes
+
+When configuring the kernel using make menuconfig/xconfig/oldconfig,
+ensure that CONFIG_UPROBES is set to "y". Under "General setup" select
+"User-space breakpoint assistance" then select "User-space probes".
+
+So that you can load and unload Uprobes-based instrumentation modules,
+make sure "Loadable module support" (CONFIG_MODULES) and "Module
+unloading" (CONFIG_MODULE_UNLOAD) are set to "y".
+
+4. API Reference
+
+The Uprobes API includes a "register" function and an "unregister"
+function for uprobes. Here are terse, mini-man-page specifications for
+these functions and the associated probe handlers that you'll write.
+See the latter half of this document for examples.
+
+4.1 register_uprobe
+
+#include <linux/uprobes.h>
+int register_uprobe(struct uprobe *u);
+
+Sets a breakpoint at virtual address u->vaddr in the process whose
+pid is u->pid. When the breakpoint is hit, Uprobes calls u->handler.
+If u->handler_in_interrupt is set, the handler runs in interrupt
+context. Otherwise it runs in task context.
+
+register_uprobe() returns 0 on success, or a negative errno
+otherwise.
+
+User's handler (u->handler):
+#include <linux/uprobes.h>
+#include <linux/ptrace.h>
+void handler(struct uprobe *u, struct pt_regs *regs);
+
+Called with u pointing to the uprobe associated with the breakpoint,
+and regs pointing to the struct containing the registers saved when
+the breakpoint was hit.
+
+4.2 unregister_uprobe
+
+#include <linux/uprobes.h>
+void unregister_uprobe(struct uprobe *u);
+
+Removes the specified probe. The unregister function can be called
+at any time after the probe has been registered, and can be called
+from a uprobe handler.
+
+5. Uprobes Features and Limitations
+
+The user is expected to assign values to the following members
+of struct uprobe: pid, vaddr, handler, and handler_in_interrupt.
+Uprobes may produce unexpected results if you:
+- change the contents of a uprobe object while it is registered; or
+- attempt to register a uprobe that is already registered.
+
+In this implementation, Uprobes allows only one uprobe at a particular
+address.
+
+Any number of kernel modules may probe a particular process
+simultaneously, and a particular module may probe any number of
+processes simultaneously.
+
+Probes are shared by all threads in a process (including newly
+created threads).
+
+If a probed process exits or execs, Uprobes automatically
+unregisters all uprobes associated with that process. Subsequent
+attempts to unregister these probes will be treated as no-ops.
+
+On the other hand, if a probed memory area is removed from the
+process's virtual memory map (e.g., via dlclose(3) or munmap(2)),
+it's currently up to you to unregister the probes first.
+
+There is no way to specify that probes should be inherited across fork;
+Uprobes removes all probepoints in the newly created child process.
+
+To avoid interfering with interactive debuggers, Uprobes will refuse
+to insert a probepoint where a breakpoint instruction already exists.
+Some architectures may refuse to insert probes on other types of
+instructions.
+
+If you install a probe in an inline-able function, Uprobes makes
+no attempt to chase down all inline instances of the function and
+install probes there. gcc may inline a function without being asked,
+so keep this in mind if you're not seeing the probe hits you expect.
+
+A probe handler can modify the environment of the probed function
+-- e.g., by modifying data structures, or by modifying the
+contents of the pt_regs struct (which are restored to the registers
+upon return from the breakpoint). So Uprobes can be used, for example,
+to install a bug fix or to inject faults for testing. Uprobes, of
+course, has no way to distinguish the deliberately injected faults
+from the accidental ones. Don't drink and probe.
+
+When Uprobes establishes a probepoint on a previous unprobed page
+of text, Linux creates a new copy of the page via its copy-on-write
+mechanism. When probepoints are removed, Uprobes makes no attempt
+to consolidate identical copies of the same page. This could affect
+memory availability if you probe many, many pages in many, many
+long-running processes.
+
+6. Probe Overhead
+
+Probe overhead is measured on a benchmark that hits the same probepoint
+repeatedly, firing a simple handler each time. Probe overhead
+changes with different cpus/archs/probe handlers and the number of
+iterations.
+
+Here are sample overhead figures (in usec) for x86 architecture.
+
+i686: Intel(R) Xeon(TM) CPU 2.40GHz
+Without probe module.
+100000 interations in 0.000650 sec i.e 0.006500 usec per iteration
+
+With probes
+100000 interations in 0.365589 sec i.e 3.655890 usec per iteration
+probe overhead is 3.649390 usec per probe hit.
+
+x86_64: Intel(R) Xeon(R) CPU X7350 @ 2.93GHz
+Without probe module.
+100000 interations in 0.000468 sec i.e 0.004680 usec per iteration
+
+With probes
+100000 interations in 0.130685 sec i.e 1.306850 usec per iteration
+Probe overhead is 1.302170 usec per probe hit.
+
+7. TODO
+
+a. Support for other architectures.
+b. Support for multiple probes at the same address.
+c. Support for boosted probes.
+d. Support return probes.
+
+8. Uprobes Team
+
+The following people have made major contributions to Uprobes:
+Jim Keniston - jkenisto@us.ibm.com
+Srikar Dronamraju - srikar@linux.vnet.ibm.com
+Ananth Mavinakayanahalli - ananth@in.ibm.com
+Prasanna Panchamukhi - prasanna@in.ibm.com
+Dave Wilder - dwilder@us.ibm.com
+
+9. Uprobes Example
+
+samples/uprobes/uprobe_example.c
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 9/10] trace: uprobes trace_event interface
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (7 preceding siblings ...)
2010-05-06 18:03 ` [PATCH v3 8/10] Uprobes documentation Srikar Dronamraju
@ 2010-05-06 18:03 ` Srikar Dronamraju
2010-05-06 18:03 ` [PATCH v3 10/10] perf: perf interface for uprobes Srikar Dronamraju
` (2 subsequent siblings)
11 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:03 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, Andrew Morton, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, LKML, Paul E. McKenney
uprobes_trace_events.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
The following patch implements trace_event support for uprobes. In its
current form it can be used to put probes at a specified text address
in a process and dump the required registers when the code flow reaches
the probed address.
This is based on trace_events for kprobes to the extent that it may
resemble that file on 2.6.34-rc3.
The following example shows how to dump the instruction pointer and %ax a
register at the probed text address.
Start a process to trace. Get the address to trace.
[Here pid is asssumed as 3548]
[Address to trace is 0x0000000000446420]
[Registers to be dumped are %ip and %ax]
# cd /sys/kernel/debug/tracing/
# echo 'p 3548:0x0000000000446420 %ip %ax' > uprobe_events
# cat uprobe_events
p:uprobes/p_3548_0x0000000000446420 3548:0x0000000000446420 %ip=%ip %ax=%ax
# cat events/uprobes/p_3548_0x0000000000446420/enable
0
[enable the event]
# echo 1 > events/uprobes/p_3548_0x0000000000446420/enable
# cat events/uprobes/p_3548_0x0000000000446420/enable
1
# #### do some activity on the program so that it hits the breakpoint
# cat uprobe_profile
3548 p_3548_0x0000000000446420 234
# tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
<...>-3548 [001] 294.285812: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
<...>-3548 [001] 294.285884: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
<...>-3548 [001] 294.285894: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
<...>-3548 [001] 294.285903: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
<...>-3548 [001] 294.285912: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
<...>-3548 [001] 294.285922: p_3548_0x0000000000446420: (0x446420) %ip=446421 %ax=1
TODO: Documentation/trace/uprobetrace.txt
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
kernel/trace/Kconfig | 8
kernel/trace/Makefile | 1
kernel/trace/trace.h | 12 +
kernel/trace/trace_uprobe.c | 927 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 948 insertions(+), 0 deletions(-)
create mode 100644 kernel/trace/trace_uprobe.c
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 13e13d4..1435e09 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -557,6 +557,14 @@ config RING_BUFFER_BENCHMARK
If unsure, say N.
+config UPROBE_EVENT
+ depends on UPROBES
+ bool "Enable uprobes-based dynamic events"
+ select TRACING
+ default y
+ help
+ This allows the user to add tracing events (similar to tracepoints)
+ on the fly via the traceevents interface.
endif # FTRACE
endif # TRACING_SUPPORT
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index 78edc64..8911959 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -58,5 +58,6 @@ obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o
obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o
obj-$(CONFIG_KSYM_TRACER) += trace_ksym.o
obj-$(CONFIG_EVENT_TRACING) += power-traces.o
+obj-$(CONFIG_UPROBE_EVENT) += trace_uprobe.o
libftrace-y := ftrace.o
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 2825ef2..9fe02ab 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -126,6 +126,18 @@ struct kretprobe_trace_entry {
(offsetof(struct kretprobe_trace_entry, args) + \
(sizeof(unsigned long) * (n)))
+struct uprobe_trace_entry {
+ struct trace_entry ent;
+ pid_t pid;
+ unsigned long ip;
+ int nargs;
+ unsigned long args[];
+};
+
+#define SIZEOF_UPROBE_TRACE_ENTRY(n) \
+ (offsetof(struct uprobe_trace_entry, args) + \
+ (sizeof(unsigned long) * (n)))
+
/*
* trace_flag_type is an enumeration that holds different
* states when a trace occurs. These are:
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
new file mode 100644
index 0000000..8c48818
--- /dev/null
+++ b/kernel/trace/trace_uprobe.c
@@ -0,0 +1,927 @@
+/*
+ * Uprobes-based tracing events
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ * Copyright (C) IBM Corporation, 2010
+ * Author: Srikar Dronamraju
+ */
+
+#include <linux/module.h>
+#include <linux/uaccess.h>
+#include <linux/uprobes.h>
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/ctype.h>
+#include <linux/ptrace.h>
+#include <linux/perf_event.h>
+#include <linux/slab.h>
+
+#include "trace.h"
+#include "trace_output.h"
+
+#define MAX_TRACE_ARGS 128
+#define MAX_ARGSTR_LEN 63
+#define MAX_EVENT_NAME_LEN 64
+#define UPROBE_EVENT_SYSTEM "uprobes"
+
+/* Reserved field names */
+#define FIELD_STRING_IP "__probe_ip"
+#define FIELD_STRING_NARGS "__probe_nargs"
+#define FIELD_STRING_PID "__probe_pid"
+
+const char *ureserved_field_names[] = {
+ "common_type",
+ "common_flags",
+ "common_preempt_count",
+ "common_pid",
+ "common_tgid",
+ "common_lock_depth",
+ FIELD_STRING_IP,
+ FIELD_STRING_NARGS,
+ FIELD_STRING_PID,
+};
+
+struct fetch_func {
+ unsigned long (*func)(struct pt_regs *, void *);
+ void *data;
+};
+
+static unsigned long call_fetch(struct fetch_func *f,
+ struct pt_regs *regs)
+{
+ return f->func(regs, f->data);
+}
+
+/* fetch handlers */
+static unsigned long fetch_register(struct pt_regs *regs,
+ void *offset)
+{
+ return regs_get_register(regs, (unsigned int)((unsigned long)offset));
+}
+
+/**
+ * Uprobe event core functions
+ */
+
+/* Flags for trace_probe */
+#define TP_FLAG_TRACE 1
+#define TP_FLAG_PROFILE 2
+#define UPROBE_ENABLED 4
+
+struct probe_arg {
+ struct fetch_func fetch;
+ const char *name;
+};
+
+struct trace_uprobe {
+ struct list_head list;
+ struct uprobe up;
+ unsigned long nhit;
+ unsigned int flags; /* For TP_FLAG_* */
+ struct ftrace_event_call call;
+ struct trace_event event;
+ unsigned int nr_args;
+ struct probe_arg args[];
+};
+
+#define SIZEOF_TRACE_UPROBE(n) \
+ (offsetof(struct trace_uprobe, args) + \
+ (sizeof(struct probe_arg) * (n)))
+
+static int probe_arg_string(char *buf, size_t n, struct fetch_func *ff)
+{
+ int ret = -EINVAL;
+
+ if (ff->func == fetch_register) {
+ const char *name;
+ name = regs_query_register_name((unsigned int)((long)ff->data));
+ ret = snprintf(buf, n, "%%%s", name);
+ }
+ if (ret >= n)
+ return -ENOSPC;
+ return ret;
+}
+
+static int register_uprobe_event(struct trace_uprobe *tp);
+static void unregister_uprobe_event(struct trace_uprobe *tp);
+
+static DEFINE_MUTEX(uprobe_lock);
+static LIST_HEAD(uprobe_list);
+
+static void uprobe_dispatcher(struct uprobe *up, struct pt_regs *regs);
+
+/* Check the name is good for event/group */
+static int check_event_name(const char *name)
+{
+ if (!isalpha(*name) && *name != '_')
+ return 0;
+ while (*++name != '\0') {
+ if (!isalpha(*name) && !isdigit(*name) && *name != '_')
+ return 0;
+ }
+ return 1;
+}
+
+/*
+ * Allocate new trace_uprobe and initialize it (including uprobes).
+ */
+static struct trace_uprobe *alloc_trace_uprobe(const char * group,
+ const char *event,
+ void *addr,
+ pid_t pid, int nargs)
+{
+ struct trace_uprobe *tp;
+ int ret = -ENOMEM;
+
+ if (!event || !check_event_name(event))
+ return ERR_PTR(-EINVAL);
+
+ if (!group || !check_event_name(group))
+ return ERR_PTR(-EINVAL);
+
+ tp = kzalloc(SIZEOF_TRACE_UPROBE(nargs), GFP_KERNEL);
+ if (!tp)
+ return ERR_PTR(ret);
+
+ tp->up.vaddr = (unsigned long)addr;
+ tp->up.pid = pid;
+ tp->up.handler = uprobe_dispatcher;
+
+ tp->call.name = kstrdup(event, GFP_KERNEL);
+ if (!tp->call.name)
+ goto error;
+
+ tp->call.system = kstrdup(group, GFP_KERNEL);
+ if (!tp->call.system)
+ goto error;
+
+ INIT_LIST_HEAD(&tp->list);
+ return tp;
+error:
+ kfree(tp->call.name);
+ kfree(tp);
+ return ERR_PTR(ret);
+}
+
+static void free_probe_arg(struct probe_arg *arg)
+{
+ kfree(arg->name);
+}
+
+static void free_trace_uprobe(struct trace_uprobe *tp)
+{
+ int i;
+
+ for (i = 0; i < tp->nr_args; i++)
+ free_probe_arg(&tp->args[i]);
+
+ kfree(tp->call.system);
+ kfree(tp->call.name);
+ kfree(tp);
+}
+
+static struct trace_uprobe *find_probe_event(const char *event,
+ const char *group)
+{
+ struct trace_uprobe *tp;
+
+ list_for_each_entry(tp, &uprobe_list, list)
+ if (strcmp(tp->call.name, event) == 0 &&
+ strcmp(tp->call.system, group) == 0)
+ return tp;
+ return NULL;
+}
+
+/* Unregister a trace_uprobe and probe_event: call with locking uprobe_lock */
+static void unregister_trace_uprobe(struct trace_uprobe *tp)
+{
+ if (tp->flags & UPROBE_ENABLED)
+ unregister_uprobe(&tp->up);
+ list_del(&tp->list);
+ unregister_uprobe_event(tp);
+}
+
+/* Register a trace_uprobe and probe_event */
+static int register_trace_uprobe(struct trace_uprobe *tp)
+{
+ struct trace_uprobe *old_tp;
+ int ret;
+
+ mutex_lock(&uprobe_lock);
+
+ /* register as an event */
+ old_tp = find_probe_event(tp->call.name, tp->call.system);
+ if (old_tp) {
+ /* delete old event */
+ unregister_trace_uprobe(old_tp);
+ free_trace_uprobe(old_tp);
+ }
+ ret = register_uprobe_event(tp);
+ if (ret) {
+ pr_warning("Faild to register probe event(%d)\n", ret);
+ goto end;
+ }
+
+ list_add_tail(&tp->list, &uprobe_list);
+end:
+ mutex_unlock(&uprobe_lock);
+ return ret;
+}
+
+#define PARAM_MAX_ARGS 16
+#define PARAM_MAX_STACK (THREAD_SIZE / sizeof(unsigned long))
+
+/* Recursive argument parser */
+static int __parse_probe_arg(char *arg, struct fetch_func *ff)
+{
+ int ret = 0;
+
+ switch (arg[0]) {
+ case '%': /* named register */
+ ret = regs_query_register_offset(arg + 1);
+ if (ret >= 0) {
+ ff->func = fetch_register;
+ ff->data = (void *)(unsigned long)ret;
+ ret = 0;
+ }
+ break;
+ default:
+ /* TODO: support custom handler */
+ ret = -EINVAL;
+ }
+ return ret;
+}
+
+/* String length checking wrapper */
+static int parse_probe_arg(char *arg, struct fetch_func *ff)
+{
+ if (strlen(arg) > MAX_ARGSTR_LEN) {
+ pr_info("Argument is too long.: %s\n", arg);
+ return -ENOSPC;
+ }
+ return __parse_probe_arg(arg, ff);
+}
+
+/* Return 1 if name is reserved or already used by another argument */
+static int conflict_field_name(const char *name,
+ struct probe_arg *args, int narg)
+{
+ int i;
+ for (i = 0; i < ARRAY_SIZE(ureserved_field_names); i++)
+ if (strcmp(ureserved_field_names[i], name) == 0)
+ return 1;
+ for (i = 0; i < narg; i++)
+ if (strcmp(args[i].name, name) == 0)
+ return 1;
+ return 0;
+}
+
+static int create_trace_uprobe(int argc, char **argv)
+{
+ /*
+ * Argument syntax:
+ * - Add uprobe: p[:[GRP/]EVENT] VADDR@PID [%REG]
+ *
+ * - Remove uprobe: -:[GRP/]EVENT
+ */
+ struct trace_uprobe *tp;
+ int i, ret = 0;
+ int is_delete = 0;
+ char *arg = NULL, *event = NULL, *group = NULL;
+ void *addr = NULL;
+ pid_t pid = 0;
+ char buf[MAX_EVENT_NAME_LEN];
+
+ /* argc must be >= 1 */
+ if (argv[0][0] == '-')
+ is_delete = 1;
+ else if (argv[0][0] != 'p') {
+ pr_info("Probe definition must be started with 'p', 'r' or"
+ " '-'.\n");
+ return -EINVAL;
+ }
+
+ if (argv[0][1] == ':') {
+ event = &argv[0][2];
+ if (strchr(event, '/')) {
+ group = event;
+ event = strchr(group, '/') + 1;
+ event[-1] = '\0';
+ if (strlen(group) == 0) {
+ pr_info("Group name is not specified\n");
+ return -EINVAL;
+ }
+ }
+ if (strlen(event) == 0) {
+ pr_info("Event name is not specified\n");
+ return -EINVAL;
+ }
+ }
+ if (!group)
+ group = UPROBE_EVENT_SYSTEM;
+
+ if (is_delete) {
+ if (!event) {
+ pr_info("Delete command needs an event name.\n");
+ return -EINVAL;
+ }
+ tp = find_probe_event(event, group);
+ if (!tp) {
+ pr_info("Event %s/%s doesn't exist.\n", group, event);
+ return -ENOENT;
+ }
+ /* delete an event */
+ unregister_trace_uprobe(tp);
+ free_trace_uprobe(tp);
+ return 0;
+ }
+
+ if (argc < 2) {
+ pr_info("Probe point is not specified.\n");
+ return -EINVAL;
+ }
+ if (isdigit(argv[1][0])) {
+ /* an address specified */
+ arg = strchr(argv[1], ':');
+ if (!arg)
+ goto fail_address_parse;
+
+ *arg++ = '\0';
+ ret = strict_strtoul(&argv[1][0], 0, (unsigned long *)&pid);
+ if (ret)
+ goto fail_address_parse;
+
+ ret = strict_strtoul(arg, 0, (unsigned long *)&addr);
+ if (ret)
+ goto fail_address_parse;
+ }
+ argc -= 2; argv += 2;
+
+ /* setup a probe */
+ if (!event) {
+ snprintf(buf, MAX_EVENT_NAME_LEN, "%c_%d_0x%p", 'p',
+ pid, addr);
+ event = buf;
+ }
+ tp = alloc_trace_uprobe(group, event, addr, pid, argc);
+ if (IS_ERR(tp)) {
+ pr_info("Failed to allocate trace_uprobe.(%d)\n",
+ (int)PTR_ERR(tp));
+ return PTR_ERR(tp);
+ }
+
+ /* parse arguments */
+ ret = 0;
+ for (i = 0; i < argc && i < MAX_TRACE_ARGS; i++) {
+ /* Parse argument name */
+ arg = strchr(argv[i], '=');
+ if (arg)
+ *arg++ = '\0';
+ else
+ arg = argv[i];
+
+ if (conflict_field_name(argv[i], tp->args, i)) {
+ pr_info("Argument%d name '%s' conflicts with "
+ "another field.\n", i, argv[i]);
+ ret = -EINVAL;
+ goto error;
+ }
+
+ tp->args[i].name = kstrdup(argv[i], GFP_KERNEL);
+ if (!tp->args[i].name) {
+ pr_info("Failed to allocate argument%d name '%s'.\n",
+ i, argv[i]);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ /* Parse fetch argument */
+ ret = parse_probe_arg(arg, &tp->args[i].fetch);
+ if (ret) {
+ pr_info("Parse error at argument%d. (%d)\n", i, ret);
+ kfree(tp->args[i].name);
+ goto error;
+ }
+
+ tp->nr_args++;
+ }
+
+ ret = register_trace_uprobe(tp);
+ if (ret)
+ goto error;
+ return 0;
+
+error:
+ free_trace_uprobe(tp);
+ return ret;
+
+fail_address_parse:
+ pr_info("Failed to parse address.\n");
+ return ret;
+}
+
+static void cleanup_all_probes(void)
+{
+ struct trace_uprobe *tp;
+
+ mutex_lock(&uprobe_lock);
+ /* TODO: Use batch unregistration */
+ while (!list_empty(&uprobe_list)) {
+ tp = list_entry(uprobe_list.next, struct trace_uprobe, list);
+ unregister_trace_uprobe(tp);
+ free_trace_uprobe(tp);
+ }
+ mutex_unlock(&uprobe_lock);
+}
+
+
+/* Probes listing interfaces */
+static void *probes_seq_start(struct seq_file *m, loff_t *pos)
+{
+ mutex_lock(&uprobe_lock);
+ return seq_list_start(&uprobe_list, *pos);
+}
+
+static void *probes_seq_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ return seq_list_next(v, &uprobe_list, pos);
+}
+
+static void probes_seq_stop(struct seq_file *m, void *v)
+{
+ mutex_unlock(&uprobe_lock);
+}
+
+static int probes_seq_show(struct seq_file *m, void *v)
+{
+ struct trace_uprobe *tp = v;
+ int i, ret;
+ char buf[MAX_ARGSTR_LEN + 1];
+
+ seq_printf(m, "%c", 'p');
+ seq_printf(m, ":%s/%s", tp->call.system, tp->call.name);
+
+ seq_printf(m, " %d:0x%p", tp->up.pid, (void *)tp->up.vaddr);
+
+ for (i = 0; i < tp->nr_args; i++) {
+ ret = probe_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i].fetch);
+ if (ret < 0) {
+ pr_warning("Argument%d decoding error(%d).\n", i, ret);
+ return ret;
+ }
+ seq_printf(m, " %s=%s", tp->args[i].name, buf);
+ }
+ seq_printf(m, "\n");
+ return 0;
+}
+
+static const struct seq_operations probes_seq_op = {
+ .start = probes_seq_start,
+ .next = probes_seq_next,
+ .stop = probes_seq_stop,
+ .show = probes_seq_show
+};
+
+static int probes_open(struct inode *inode, struct file *file)
+{
+ if ((file->f_mode & FMODE_WRITE) &&
+ (file->f_flags & O_TRUNC))
+ cleanup_all_probes();
+
+ return seq_open(file, &probes_seq_op);
+}
+
+static int command_trace_uprobe(const char *buf)
+{
+ char **argv;
+ int argc = 0, ret = 0;
+
+ argv = argv_split(GFP_KERNEL, buf, &argc);
+ if (!argv)
+ return -ENOMEM;
+
+ if (argc)
+ ret = create_trace_uprobe(argc, argv);
+
+ argv_free(argv);
+ return ret;
+}
+
+#define WRITE_BUFSIZE 128
+
+static ssize_t probes_write(struct file *file, const char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ char *kbuf, *tmp;
+ int ret;
+ size_t done;
+ size_t size;
+
+ kbuf = kmalloc(WRITE_BUFSIZE, GFP_KERNEL);
+ if (!kbuf)
+ return -ENOMEM;
+
+ ret = done = 0;
+ while (done < count) {
+ size = count - done;
+ if (size >= WRITE_BUFSIZE)
+ size = WRITE_BUFSIZE - 1;
+ if (copy_from_user(kbuf, buffer + done, size)) {
+ ret = -EFAULT;
+ goto out;
+ }
+ kbuf[size] = '\0';
+ tmp = strchr(kbuf, '\n');
+ if (tmp) {
+ *tmp = '\0';
+ size = tmp - kbuf + 1;
+ } else if (done + size < count) {
+ pr_warning("Line length is too long: "
+ "Should be less than %d.", WRITE_BUFSIZE);
+ ret = -EINVAL;
+ goto out;
+ }
+ done += size;
+ /* Remove comments */
+ tmp = strchr(kbuf, '#');
+ if (tmp)
+ *tmp = '\0';
+
+ ret = command_trace_uprobe(kbuf);
+ if (ret)
+ goto out;
+ }
+ ret = done;
+out:
+ kfree(kbuf);
+ return ret;
+}
+
+static const struct file_operations uprobe_events_ops = {
+ .owner = THIS_MODULE,
+ .open = probes_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release,
+ .write = probes_write,
+};
+
+/* Probes profiling interfaces */
+static int probes_profile_seq_show(struct seq_file *m, void *v)
+{
+ struct trace_uprobe *tp = v;
+
+ seq_printf(m, " %d %-44s %15lu\n", tp->up.pid, tp->call.name,
+ tp->nhit);
+ return 0;
+}
+
+static const struct seq_operations profile_seq_op = {
+ .start = probes_seq_start,
+ .next = probes_seq_next,
+ .stop = probes_seq_stop,
+ .show = probes_profile_seq_show
+};
+
+static int profile_open(struct inode *inode, struct file *file)
+{
+ return seq_open(file, &profile_seq_op);
+}
+
+static const struct file_operations uprobe_profile_ops = {
+ .owner = THIS_MODULE,
+ .open = profile_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release,
+};
+
+/* Uprobe handler */
+static void uprobe_trace_func(struct uprobe *up, struct pt_regs *regs)
+{
+ struct trace_uprobe *tp = container_of(up, struct trace_uprobe, up);
+ struct uprobe_trace_entry *entry;
+ struct ring_buffer_event *event;
+ struct ring_buffer *buffer;
+ int size, i, pc;
+ unsigned long irq_flags;
+ struct ftrace_event_call *call = &tp->call;
+
+ tp->nhit++;
+
+ local_save_flags(irq_flags);
+ pc = preempt_count();
+
+ size = SIZEOF_UPROBE_TRACE_ENTRY(tp->nr_args);
+
+ event = trace_current_buffer_lock_reserve(&buffer, call->id, size,
+ irq_flags, pc);
+ if (!event)
+ return;
+
+ entry = ring_buffer_event_data(event);
+ entry->nargs = tp->nr_args;
+ entry->ip = (unsigned long)up->vaddr;
+ for (i = 0; i < tp->nr_args; i++)
+ entry->args[i] = call_fetch(&tp->args[i].fetch, regs);
+
+ if (!filter_current_check_discard(buffer, call, entry, event))
+ trace_nowake_buffer_unlock_commit(buffer, event, irq_flags, pc);
+}
+
+/* Event entry printers */
+enum print_line_t
+print_uprobe_event(struct trace_iterator *iter, int flags)
+{
+ struct uprobe_trace_entry *field;
+ struct trace_seq *s = &iter->seq;
+ struct trace_event *event;
+ struct trace_uprobe *tp;
+ int i;
+
+ field = (struct uprobe_trace_entry *)iter->ent;
+ event = ftrace_find_event(field->ent.type);
+ tp = container_of(event, struct trace_uprobe, event);
+
+ if (!trace_seq_printf(s, "%s: (", tp->call.name))
+ goto partial;
+
+ if (!seq_print_ip_sym(s, field->ip, flags | TRACE_ITER_SYM_OFFSET))
+ goto partial;
+
+ if (!trace_seq_puts(s, ")"))
+ goto partial;
+
+ for (i = 0; i < field->nargs; i++)
+ if (!trace_seq_printf(s, " %s=%lx",
+ tp->args[i].name, field->args[i]))
+ goto partial;
+
+ if (!trace_seq_puts(s, "\n"))
+ goto partial;
+
+ return TRACE_TYPE_HANDLED;
+partial:
+ return TRACE_TYPE_PARTIAL_LINE;
+}
+
+
+static int probe_event_enable(struct ftrace_event_call *call)
+{
+ int ret = 0;
+ struct trace_uprobe *tp = (struct trace_uprobe *)call->data;
+
+ if (!(tp->flags & UPROBE_ENABLED)) {
+ ret = register_uprobe(&tp->up);
+ if (!ret)
+ tp->flags |= (UPROBE_ENABLED | TP_FLAG_TRACE);
+ }
+ return ret;
+}
+
+static void probe_event_disable(struct ftrace_event_call *call)
+{
+ struct trace_uprobe *tp = (struct trace_uprobe *)call->data;
+
+ if (tp->flags & UPROBE_ENABLED) {
+ unregister_uprobe(&tp->up);
+ tp->flags &= ~(UPROBE_ENABLED | TP_FLAG_TRACE);
+ }
+}
+
+static int probe_event_raw_init(struct ftrace_event_call *event_call)
+{
+ INIT_LIST_HEAD(&event_call->fields);
+
+ return 0;
+}
+
+#undef DEFINE_FIELD
+#define DEFINE_FIELD(type, item, name, is_signed) \
+ do { \
+ ret = trace_define_field(event_call, #type, name, \
+ offsetof(typeof(field), item), \
+ sizeof(field.item), is_signed, \
+ FILTER_OTHER); \
+ if (ret) \
+ return ret; \
+ } while (0)
+
+static int uprobe_event_define_fields(struct ftrace_event_call *event_call)
+{
+ int ret, i;
+ struct uprobe_trace_entry field;
+ struct trace_uprobe *tp = (struct trace_uprobe *)event_call->data;
+
+ DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+ DEFINE_FIELD(int, nargs, FIELD_STRING_NARGS, 1);
+ DEFINE_FIELD(pid_t, pid, FIELD_STRING_PID, 2);
+ /* Set argument names as fields */
+ for (i = 0; i < tp->nr_args; i++)
+ DEFINE_FIELD(unsigned long, args[i], tp->args[i].name, 0);
+ return 0;
+}
+
+static int __set_print_fmt(struct trace_uprobe *tp, char *buf, int len)
+{
+ int i;
+ int pos = 0;
+
+ const char *fmt, *arg;
+
+ fmt = "(%lx)";
+ arg = "REC->" FIELD_STRING_IP;
+
+ /* When len=0, we just calculate the needed length */
+#define LEN_OR_ZERO (len ? len - pos : 0)
+
+ pos += snprintf(buf + pos, LEN_OR_ZERO, "\"%s", fmt);
+ for (i = 0; i < tp->nr_args; i++)
+ pos += snprintf(buf + pos, LEN_OR_ZERO, " %s=%%lx",
+ tp->args[i].name);
+
+ pos += snprintf(buf + pos, LEN_OR_ZERO, "\", %s", arg);
+ for (i = 0; i < tp->nr_args; i++)
+ pos += snprintf(buf + pos, LEN_OR_ZERO, ", REC->%s",
+ tp->args[i].name);
+
+#undef LEN_OR_ZERO
+
+ /* return the length of print_fmt */
+ return pos;
+}
+
+static int set_print_fmt(struct trace_uprobe *tp)
+{
+ int len;
+ char *print_fmt;
+
+ /* First: called with 0 length to calculate the needed length */
+ len = __set_print_fmt(tp, NULL, 0);
+ print_fmt = kmalloc(len + 1, GFP_KERNEL);
+ if (!print_fmt)
+ return -ENOMEM;
+
+ /* Second: actually write the @print_fmt */
+ __set_print_fmt(tp, print_fmt, len + 1);
+ tp->call.print_fmt = print_fmt;
+
+ return 0;
+}
+
+#ifdef CONFIG_PERF_EVENTS
+
+/* Uprobe profile handler */
+static void uprobe_perf_func(struct uprobe *up,
+ struct pt_regs *regs)
+{
+ struct trace_uprobe *tp = container_of(up, struct trace_uprobe, up);
+ struct ftrace_event_call *call = &tp->call;
+ struct uprobe_trace_entry *entry;
+ int size, __size, i;
+ unsigned long irq_flags;
+ int rctx;
+
+ __size = SIZEOF_UPROBE_TRACE_ENTRY(tp->nr_args);
+ size = ALIGN(__size + sizeof(u32), sizeof(u64));
+ size -= sizeof(u32);
+ if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
+ "profile buffer not large enough"))
+ return;
+
+ entry = perf_trace_buf_prepare(size, call->id, &rctx, &irq_flags);
+ if (!entry)
+ return;
+
+ entry->nargs = tp->nr_args;
+ entry->ip = (unsigned long)up->vaddr;
+ for (i = 0; i < tp->nr_args; i++)
+ entry->args[i] = call_fetch(&tp->args[i].fetch, regs);
+
+ perf_trace_buf_submit(entry, size, rctx, entry->ip, 1, irq_flags, regs);
+}
+
+static int probe_perf_enable(struct ftrace_event_call *call)
+{
+ int ret = 0;
+ struct trace_uprobe *tp = (struct trace_uprobe *)call->data;
+
+ if (!(tp->flags & UPROBE_ENABLED)) {
+ ret = register_uprobe(&tp->up);
+ if (!ret)
+ tp->flags |= (UPROBE_ENABLED | TP_FLAG_PROFILE);
+ }
+ return ret;
+}
+
+static void probe_perf_disable(struct ftrace_event_call *call)
+{
+ struct trace_uprobe *tp = (struct trace_uprobe *)call->data;
+
+ if (tp->flags & UPROBE_ENABLED) {
+ unregister_uprobe(&tp->up);
+ tp->flags &= ~(UPROBE_ENABLED | TP_FLAG_PROFILE);
+ }
+}
+#endif /* CONFIG_PERF_EVENTS */
+
+
+static void uprobe_dispatcher(struct uprobe *up, struct pt_regs *regs)
+{
+ struct trace_uprobe *tp = container_of(up, struct trace_uprobe, up);
+
+ if (tp->flags & TP_FLAG_TRACE)
+ uprobe_trace_func(up, regs);
+#ifdef CONFIG_PERF_EVENTS
+ if (tp->flags & TP_FLAG_PROFILE)
+ uprobe_perf_func(up, regs);
+#endif
+}
+
+
+static int register_uprobe_event(struct trace_uprobe *tp)
+{
+ struct ftrace_event_call *call = &tp->call;
+ int ret;
+
+ /* Initialize ftrace_event_call */
+ tp->event.trace = print_uprobe_event;
+ call->raw_init = probe_event_raw_init;
+ call->define_fields = uprobe_event_define_fields;
+ if (set_print_fmt(tp) < 0)
+ return -ENOMEM;
+
+ call->event = &tp->event;
+ call->id = register_ftrace_event(&tp->event);
+ if (!call->id) {
+ kfree(call->print_fmt);
+ return -ENODEV;
+ }
+
+ call->enabled = 0;
+ call->regfunc = probe_event_enable;
+ call->unregfunc = probe_event_disable;
+
+#ifdef CONFIG_PERF_EVENTS
+ call->perf_event_enable = probe_perf_enable;
+ call->perf_event_disable = probe_perf_disable;
+#endif
+ call->data = tp;
+ ret = trace_add_event_call(call);
+ if (ret) {
+ pr_info("Failed to register uprobe event: %s\n", call->name);
+ kfree(call->print_fmt);
+ unregister_ftrace_event(&tp->event);
+ }
+ return ret;
+}
+
+static void unregister_uprobe_event(struct trace_uprobe *tp)
+{
+ /* tp->event is unregistered in trace_remove_event_call() */
+ trace_remove_event_call(&tp->call);
+ kfree(tp->call.print_fmt);
+}
+
+/* Make a debugfs interface for controling probe points */
+static __init int init_uprobe_trace(void)
+{
+ struct dentry *d_tracer;
+ struct dentry *entry;
+
+ d_tracer = tracing_init_dentry();
+ if (!d_tracer)
+ return 0;
+
+ entry = debugfs_create_file("uprobe_events", 0644, d_tracer,
+ NULL, &uprobe_events_ops);
+
+ /* Event list interface */
+ if (!entry)
+ pr_warning("Could not create debugfs "
+ "'uprobe_events' entry\n");
+
+ /* Profile interface */
+ entry = debugfs_create_file("uprobe_profile", 0444, d_tracer,
+ NULL, &uprobe_profile_ops);
+
+ if (!entry)
+ pr_warning("Could not create debugfs "
+ "'uprobe_profile' entry\n");
+ return 0;
+}
+fs_initcall(init_uprobe_trace);
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH v3 10/10] perf: perf interface for uprobes.
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (8 preceding siblings ...)
2010-05-06 18:03 ` [PATCH v3 9/10] trace: uprobes trace_event interface Srikar Dronamraju
@ 2010-05-06 18:03 ` Srikar Dronamraju
2010-05-06 23:13 ` Masami Hiramatsu
2010-05-11 20:59 ` [PATCH v3 0/10] Uprobes v3 Peter Zijlstra
2010-05-12 10:38 ` Frederic Weisbecker
11 siblings, 1 reply; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-06 18:03 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar
Cc: Masami Hiramatsu, Mel Gorman, Srikar Dronamraju, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
perf-events.patch
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
This patch enhances perf probe to accept pid and user vaddr.
This patch provides very basic support for uprobes.
TODO:
Update perf-probes.txt.
Global tracing.
Here is a terminal snapshot of placing, using and removing a probe on a
process with pid 3329 (corresponding to zsh)
[ Probing a function in the executable using function name ]
-------------------------------------------------------------
[root@ABCD]# perf probe -p 3329 zfree@zsh
Added new event:
probe:p_3329_zfree (on 0x446420)
You can now use it on all perf tools, such as:
perf record -e probe:p_3329_zfree -a sleep 1
[root@ABCD]# perf probe --list
probe:p_3329_zfree (on 3329:0x0000000000446420)
[root@ABCD]# cat /sys/kernel/debug/tracing/uprobe_events
p:probe/p_3329_zfree 3329:0x0000000000446420
[root@ABCD]# perf record -f -e probe:p_3329_zfree -a sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.039 MB perf.data (~1716 samples) ]
[root@ABCD]# perf probe -p 3329 --del probe:p_3329_zfree
Remove event: probe:p_3329_zfree
[root@ABCD]# perf report
# Samples: 447
#
# Overhead Command Shared Object Symbol
# ........ ............... ............. ......
#
100.00% zsh zsh [.] zfree
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[ Probing a function + offset ]
-------------------------------
[root@ABCD]# perf probe -p 3329 zfree@zsh+5
Added new event:
probe:p_3329_zfree (on 0x446425)
You can now use it on all perf tools, such as:
perf record -e probe:p_3329_zfree -a sleep 1
[root@ABCD]# perf probe --list
probe:p_3329_zfree (on 3329:0x0000000000446425)
[root@ABCD]# cat /sys/kernel/debug/tracing/uprobe_events
p:probe/p_3329_zfree 3329:0x0000000000446425
[root@ABCD]# perf record -f -e probe:p_3329_zfree -a sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.036 MB perf.data (~1590 samples) ]
[root@ABCD]# perf probe -p 3329 --del probe:p_3329_zfree
Remove event: probe:p_3329_zfree
[root@ABCD]# perf report
# Samples: 18
#
# Overhead Command Shared Object Symbol
# ........ ............... ............. ......
#
100.00% zsh zsh [.] zfree
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[ Probing a library function using function name ]
--------------------------------------------------
[root@ABCD]# perf probe -p 3329 write@libc-2.5.so
Added new event:
probe:p_3329_write (on 0x36010c6060)
You can now use it on all perf tools, such as:
perf record -e probe:p_3329_write -a sleep 1
[root@ABCD]# perf probe --list
probe:p_3329_write (on 3329:0x00000036010c6060)
[root@ABCD]# cat /sys/kernel/debug/tracing/uprobe_events
p:probe/p_3329_write 3329:0x00000036010c6060
[root@ABCD]# perf record -f -e probe:p_3329_write -a sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.040 MB perf.data (~1738 samples) ]
[root@ABCD]# perf probe -p 3329 --del probe:p_3329_write
Remove event: probe:p_3329_write
[root@ABCD]# perf report
# Samples: 11
#
# Overhead Command Shared Object Symbol
# ........ ............... .................. ......
#
100.00% zsh libc-2.5.so [.] __GI___libc_write
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
Adding a probe for a process
[ Probing using vaddr 0x0000000000446420 (corresponding to zfree)]
-------------------------------------------------------------------
[root@ABCD]# perf probe -p 3329 0x0000000000446420
Added new event:
probe:p_3329_0x0000000000446420 (on 0x0000000000446420)
You can now use it on all perf tools, such as:
perf record -e probe:p_3329_0x0000000000446420 -a sleep 1
[root@ABCD]# perf record -e probe:p_3329_0x0000000000446420 -a sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.041 MB perf.data (~1797 samples) ]
[root@ABCD]# perf report
#
# Samples: 628
#
# Overhead Command Shared Object Symbol
# ........ ............... ............. ......
#
100.00% zsh zsh [.] zfree
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[root@ABCD]# perf report --sort comm,dso
# Samples: 628
#
# Overhead Command Shared Object
# ........ ............... .............
#
100.00% zsh zsh
[root@ABCD]# perf probe --list
probe:p_3329_0x0000000000446420 (on 3329:0x0000000000446420)
[root@ABCD]# perf list | grep probe
probe:p_3329_0x0000000000446420 [Tracepoint event]
[root@ABCD]# perf probe -p 3329 --del probe:p_3329_0x0000000000446420
Remove event: probe:p_3329_0x0000000000446420
[root@ABCD]#
Another example for a shared library: write stub in libc. (corresponds to
0x00000036010c6060)
on a vaddr
[ Probing a libc vaddr 0x00000036010c6060 (corresponding to write) ]
[root@ABCD]# perf probe -p 3329 0x00000036010c6060
dded new event:
probe:p_3329_0x00000036010c6060 (on 0x00000036010c6060)
You can now use it on all perf tools, such as:
perf record -e probe:p_3329_0x00000036010c6060 -a sleep 1
[root@ABCD]# perf record -f -e probe:p_3329_0x00000036010c6060 -a sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.040 MB perf.data (~1748 samples) ]
[root@ABCD]# perf report
# Samples: 24
#
# Overhead Command Shared Object Symbol
# ........ ............... .................. ......
#
100.00% zsh libc-2.5.so [.] __GI___libc_write
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[root@ABCD]#
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
tools/perf/builtin-probe.c | 34 +++++--
tools/perf/builtin-top.c | 20 ----
tools/perf/util/event.c | 20 ++++
tools/perf/util/event.h | 1
tools/perf/util/probe-event.c | 207 ++++++++++++++++++++++++++++++++--------
tools/perf/util/probe-event.h | 9 +-
tools/perf/util/probe-finder.h | 1
tools/perf/util/symbol.c | 6 +
8 files changed, 224 insertions(+), 74 deletions(-)
diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index 152d6c9..9f867ad 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -55,6 +55,7 @@ static struct {
bool force_add;
bool show_lines;
int nr_probe;
+ pid_t pid;
struct probe_point probes[MAX_PROBES];
struct strlist *dellist;
struct map_groups kmap_groups;
@@ -73,7 +74,7 @@ static void parse_probe_event(const char *str)
die("Too many probes (> %d) are specified.", MAX_PROBES);
/* Parse perf-probe event into probe_point */
- parse_perf_probe_event(str, pp, &session.need_dwarf);
+ parse_perf_probe_event(str, pp, &session.need_dwarf, session.pid);
pr_debug("%d arguments\n", pp->nr_args);
}
@@ -203,6 +204,8 @@ static const struct option options[] = {
"FUNC[:RLN[+NUM|:RLN2]]|SRC:ALN[+NUM|:ALN2]",
"Show source code lines.", opt_show_lines),
#endif
+ OPT_INTEGER('p', "pid", &session.pid,
+ "specify a pid for a uprobes based probe"),
OPT_END()
};
@@ -263,7 +266,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
}
#ifndef NO_DWARF_SUPPORT
- if (session.show_lines) {
+ if (session.show_lines && !session.pid) {
if (session.nr_probe != 0 || session.dellist) {
pr_warning(" Error: Don't use --line with"
" --add/--del.\n");
@@ -283,12 +286,19 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
#endif
if (session.dellist) {
- del_trace_kprobe_events(session.dellist);
+ if (session.pid)
+ del_trace_uprobe_events(session.dellist);
+ else
+ del_trace_kprobe_events(session.dellist);
+
strlist__delete(session.dellist);
if (session.nr_probe == 0)
return 0;
}
+ if (session.pid)
+ goto end_dwarf;
+
/* Add probes */
init_vmlinux();
@@ -334,18 +344,19 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
break;
}
close(fd);
-
-end_dwarf:
#endif /* !NO_DWARF_SUPPORT */
+end_dwarf:
/* Synthesize probes without dwarf */
for (i = 0; i < session.nr_probe; i++) {
pp = &session.probes[i];
if (pp->found) /* This probe is already found. */
continue;
- evaluate_probe_point(pp);
- ret = synthesize_trace_kprobe_event(pp);
+ if (!session.pid)
+ evaluate_probe_point(pp);
+
+ ret = synthesize_trace_probe_event(pp);
if (ret == -E2BIG)
die("probe point definition becomes too long.");
else if (ret < 0)
@@ -353,8 +364,11 @@ end_dwarf:
}
/* Settng up probe points */
- add_trace_kprobe_events(session.probes, session.nr_probe,
- session.force_add);
+ if (session.pid)
+ add_trace_uprobe_events(session.probes, session.nr_probe,
+ session.force_add, session.pid);
+ else
+ add_trace_kprobe_events(session.probes, session.nr_probe,
+ session.force_add);
return 0;
}
-
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 1f52932..430d910 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1005,26 +1005,6 @@ static void event__process_sample(const event_t *self,
}
}
-static int event__process(event_t *event, struct perf_session *session)
-{
- switch (event->header.type) {
- case PERF_RECORD_COMM:
- event__process_comm(event, session);
- break;
- case PERF_RECORD_MMAP:
- event__process_mmap(event, session);
- break;
- case PERF_RECORD_FORK:
- case PERF_RECORD_EXIT:
- event__process_task(event, session);
- break;
- default:
- break;
- }
-
- return 0;
-}
-
struct mmap_data {
int counter;
void *base;
diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c
index 705ec63..e5190f2 100644
--- a/tools/perf/util/event.c
+++ b/tools/perf/util/event.c
@@ -432,6 +432,26 @@ int event__process_task(event_t *self, struct perf_session *session)
return 0;
}
+int event__process(event_t *event, struct perf_session *session)
+{
+ switch (event->header.type) {
+ case PERF_RECORD_COMM:
+ event__process_comm(event, session);
+ break;
+ case PERF_RECORD_MMAP:
+ event__process_mmap(event, session);
+ break;
+ case PERF_RECORD_FORK:
+ case PERF_RECORD_EXIT:
+ event__process_task(event, session);
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
void thread__find_addr_map(struct thread *self,
struct perf_session *session, u8 cpumode,
enum map_type type, u64 addr,
diff --git a/tools/perf/util/event.h b/tools/perf/util/event.h
index a33b949..282486b 100644
--- a/tools/perf/util/event.h
+++ b/tools/perf/util/event.h
@@ -128,6 +128,7 @@ int event__process_comm(event_t *self, struct perf_session *session);
int event__process_lost(event_t *self, struct perf_session *session);
int event__process_mmap(event_t *self, struct perf_session *session);
int event__process_task(event_t *self, struct perf_session *session);
+int event__process(event_t *event, struct perf_session *session);
struct addr_location;
int event__preprocess_sample(const event_t *self, struct perf_session *session,
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 7c004b6..743a72b 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -41,6 +41,7 @@
#include "color.h"
#include "parse-events.h" /* For debugfs_path */
#include "probe-event.h"
+#include "session.h"
#define MAX_CMDLEN 256
#define MAX_PROBE_ARGS 128
@@ -112,6 +113,64 @@ static bool check_event_name(const char *name)
return true;
}
+/*
+ * uprobe_events only accepts address:
+ * Convert function and any offset to address
+ */
+static void convert_name_to_addr(struct probe_point *pp)
+{
+ struct perf_session *session;
+ struct thread *thread;
+ struct symbol *sym;
+ struct map *map;
+ char *name = pp->file;
+ unsigned long long vaddr;
+
+ /* check if user has specifed a virtual address */
+ vaddr = strtoul(pp->function, NULL, 0);
+ if (vaddr)
+ return;
+
+ session = perf_session__new(NULL, O_WRONLY, false);
+ DIE_IF(session == NULL);
+ symbol_conf.sort_by_name = true;
+ symbol_conf.try_vmlinux_path = false;
+ if (symbol__init() < 0)
+ semantic_error("Cannot initialize symbols.");
+
+ event__synthesize_thread(pp->upid, event__process, session);
+
+ thread = perf_session__findnew(session, pp->upid);
+ DIE_IF(thread == NULL);
+
+ if (!name)
+ /* Lets find the function in the executable. */
+ name = thread->comm;
+ DIE_IF(name == NULL);
+
+ map = map_groups__find_by_name(&thread->mg, MAP__FUNCTION, name);
+ if (!map)
+ semantic_error("Cannot find appropriate DSO.");
+
+ sym = map__find_symbol_by_name(map, pp->function, NULL);
+ if (!sym)
+ semantic_error("Cannot find appropriate Symbol.");
+
+ if (map->start > sym->start)
+ vaddr = map->start;
+ vaddr += sym->start + pp->offset + map->pgoff;
+ pp->offset = 0;
+
+ if (!pp->event)
+ pp->event = pp->function;
+ else
+ free(pp->function);
+ pp->function = zalloc(sizeof(char *) * 20);
+ if (!pp->function)
+ die("Failed to allocate memory by zalloc.");
+ e_snprintf(pp->function, 20, "0x%llx", vaddr);
+}
+
/* Parse probepoint definition. */
static void parse_perf_probe_probepoint(char *arg, struct probe_point *pp)
{
@@ -166,7 +225,8 @@ static void parse_perf_probe_probepoint(char *arg, struct probe_point *pp)
*ptr++ = '\0';
}
switch (c) {
- case ':': /* Line number */
+ case ':':
+ /* Line number */
pp->line = strtoul(arg, &tmp, 0);
if (*tmp != '\0')
semantic_error("There is non-digit char"
@@ -216,6 +276,9 @@ static void parse_perf_probe_probepoint(char *arg, struct probe_point *pp)
if (pp->retprobe && !pp->function)
semantic_error("Return probe requires an entry function.");
+ if (pp->upid && !pp->function)
+ semantic_error("No function specified for uprobes");
+
if ((pp->offset || pp->line || pp->lazy_line) && pp->retprobe)
semantic_error("Offset/Line/Lazy pattern can't be used with "
"return probe.");
@@ -223,11 +286,14 @@ static void parse_perf_probe_probepoint(char *arg, struct probe_point *pp)
pr_debug("symbol:%s file:%s line:%d offset:%d return:%d lazy:%s\n",
pp->function, pp->file, pp->line, pp->offset, pp->retprobe,
pp->lazy_line);
+
+ if (pp->upid)
+ convert_name_to_addr(pp);
}
/* Parse perf-probe event definition */
void parse_perf_probe_event(const char *str, struct probe_point *pp,
- bool *need_dwarf)
+ bool *need_dwarf, pid_t pid)
{
char **argv;
int argc, i;
@@ -241,6 +307,7 @@ void parse_perf_probe_event(const char *str, struct probe_point *pp,
semantic_error("Too many arguments");
/* Parse probe point */
+ pp->upid = pid;
parse_perf_probe_probepoint(argv[0], pp);
if (pp->file || pp->line || pp->lazy_line)
*need_dwarf = true;
@@ -263,15 +330,15 @@ void parse_perf_probe_event(const char *str, struct probe_point *pp,
argv_free(argv);
}
-/* Parse kprobe_events event into struct probe_point */
-void parse_trace_kprobe_event(const char *str, struct probe_point *pp)
+/* Parse kprobe_events (uprobe_events) event into struct probe_point */
+void parse_trace_probe_event(const char *str, struct probe_point *pp)
{
char pr;
char *p;
int ret, i, argc;
char **argv;
- pr_debug("Parsing kprobe_events: %s\n", str);
+ pr_debug("Parsing probe_events: %s\n", str);
argv = argv_split(str, &argc);
if (!argv)
die("argv_split failed.");
@@ -375,7 +442,7 @@ error:
return ret;
}
-int synthesize_trace_kprobe_event(struct probe_point *pp)
+int synthesize_trace_probe_event(struct probe_point *pp)
{
char *buf;
int i, len, ret;
@@ -383,7 +450,11 @@ int synthesize_trace_kprobe_event(struct probe_point *pp)
pp->probes[0] = buf = zalloc(MAX_CMDLEN);
if (!buf)
die("Failed to allocate memory by zalloc.");
- ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
+ if (pp->offset)
+ ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function,
+ pp->offset);
+ else
+ ret = e_snprintf(buf, MAX_CMDLEN, "%s", pp->function);
if (ret <= 0)
goto error;
len = ret;
@@ -426,8 +497,8 @@ static int open_kprobe_events(int flags, int mode)
return ret;
}
-/* Get raw string list of current kprobe_events */
-static struct strlist *get_trace_kprobe_event_rawlist(int fd)
+/* Get raw string list of current kprobe_events or uprobe_events */
+static struct strlist *get_trace_probe_event_rawlist(int fd)
{
int ret, idx;
FILE *fp;
@@ -455,6 +526,27 @@ static struct strlist *get_trace_kprobe_event_rawlist(int fd)
return sl;
}
+static int open_uprobe_events(int flags, int mode)
+{
+ char buf[PATH_MAX];
+ int ret;
+
+ ret = e_snprintf(buf, PATH_MAX, "%s/../uprobe_events", debugfs_path);
+ if (ret < 0)
+ die("Failed to make uprobe_events path.");
+
+ ret = open(buf, flags, mode);
+ if (ret < 0) {
+ if (errno == ENOENT)
+ die("uprobe_events file does not exist -"
+ " please rebuild with CONFIG_UPROBE_EVENT.");
+ else
+ die("Could not open uprobe_events file: %s",
+ strerror(errno));
+ }
+ return ret;
+}
+
/* Free and zero clear probe_point */
static void clear_probe_point(struct probe_point *pp)
{
@@ -500,9 +592,8 @@ static void show_perf_probe_event(const char *event, const char *place,
}
/* List up current perf-probe events */
-void show_perf_probe_events(void)
+static void __show_perf_probe_events(int fd)
{
- int fd;
struct probe_point pp;
struct strlist *rawlist;
struct str_node *ent;
@@ -510,22 +601,30 @@ void show_perf_probe_events(void)
setup_pager();
memset(&pp, 0, sizeof(pp));
- fd = open_kprobe_events(O_RDONLY, 0);
- rawlist = get_trace_kprobe_event_rawlist(fd);
- close(fd);
-
+ rawlist = get_trace_probe_event_rawlist(fd);
strlist__for_each(ent, rawlist) {
- parse_trace_kprobe_event(ent->s, &pp);
+ parse_trace_probe_event(ent->s, &pp);
/* Synthesize only event probe point */
synthesize_perf_probe_point(&pp);
/* Show an event */
show_perf_probe_event(pp.event, pp.probes[0], &pp);
clear_probe_point(&pp);
}
-
strlist__delete(rawlist);
}
+void show_perf_probe_events(void)
+{
+ int fd;
+ fd = open_kprobe_events(O_RDONLY, 0);
+ __show_perf_probe_events(fd);
+ close(fd);
+
+ fd = open_uprobe_events(O_RDONLY, 0);
+ __show_perf_probe_events(fd);
+ close(fd);
+}
+
/* Get current perf-probe event names */
static struct strlist *get_perf_event_names(int fd, bool include_group)
{
@@ -535,11 +634,11 @@ static struct strlist *get_perf_event_names(int fd, bool include_group)
struct probe_point pp;
memset(&pp, 0, sizeof(pp));
- rawlist = get_trace_kprobe_event_rawlist(fd);
+ rawlist = get_trace_probe_event_rawlist(fd);
sl = strlist__new(true, NULL);
strlist__for_each(ent, rawlist) {
- parse_trace_kprobe_event(ent->s, &pp);
+ parse_trace_probe_event(ent->s, &pp);
if (include_group) {
if (e_snprintf(buf, 128, "%s:%s", pp.group,
pp.event) < 0)
@@ -555,7 +654,7 @@ static struct strlist *get_perf_event_names(int fd, bool include_group)
return sl;
}
-static void write_trace_kprobe_event(int fd, const char *buf)
+static void write_trace_probe_event(int fd, const char *buf)
{
int ret;
@@ -566,12 +665,15 @@ static void write_trace_kprobe_event(int fd, const char *buf)
}
static void get_new_event_name(char *buf, size_t len, const char *base,
- struct strlist *namelist, bool allow_suffix)
+ struct strlist *namelist, bool allow_suffix, pid_t pid)
{
int i, ret;
/* Try no suffix */
- ret = e_snprintf(buf, len, "%s", base);
+ if (pid)
+ ret = e_snprintf(buf, len, "p_%d_%s", pid, base);
+ else
+ ret = e_snprintf(buf, len, "%s", base);
if (ret < 0)
die("snprintf() failed: %s", strerror(-ret));
if (!strlist__has_entry(namelist, buf))
@@ -595,22 +697,25 @@ static void get_new_event_name(char *buf, size_t len, const char *base,
die("Too many events are on the same function.");
}
-void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
- bool force_add)
+static void add_trace_probe_events(int fd, struct probe_point *probes,
+ int nr_probes, bool force_add, pid_t pid)
{
- int i, j, fd;
+ int i, j;
struct probe_point *pp;
char buf[MAX_CMDLEN];
+ char tempbuf[MAX_CMDLEN];
char event[64];
struct strlist *namelist;
bool allow_suffix;
- fd = open_kprobe_events(O_RDWR, O_APPEND);
/* Get current event names */
namelist = get_perf_event_names(fd, false);
for (j = 0; j < nr_probes; j++) {
pp = probes + j;
+ pp->upid = pid;
+ if (pid)
+ snprintf(tempbuf, MAX_CMDLEN, "%d:", pid);
if (!pp->event)
pp->event = strdup(pp->function);
if (!pp->group)
@@ -621,12 +726,13 @@ void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
for (i = 0; i < pp->found; i++) {
/* Get an unused new event name */
get_new_event_name(event, 64, pp->event, namelist,
- allow_suffix);
- snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s\n",
+ allow_suffix, pid);
+ snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s%s\n",
pp->retprobe ? 'r' : 'p',
pp->group, event,
+ pp->upid ? tempbuf : " ",
pp->probes[i]);
- write_trace_kprobe_event(fd, buf);
+ write_trace_probe_event(fd, buf);
printf("Added new event:\n");
/* Get the first parameter (probe-point) */
sscanf(pp->probes[i], "%s", buf);
@@ -650,7 +756,21 @@ void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
close(fd);
}
-static void __del_trace_kprobe_event(int fd, struct str_node *ent)
+void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
+ bool force_add)
+{
+ int fd = open_kprobe_events(O_RDWR, O_APPEND);
+ add_trace_probe_events(fd, probes, nr_probes, force_add, 0);
+}
+
+void add_trace_uprobe_events(struct probe_point *probes, int nr_probes,
+ bool force_add, pid_t pid)
+{
+ int fd = open_uprobe_events(O_RDWR, O_APPEND);
+ add_trace_probe_events(fd, probes, nr_probes, force_add, pid);
+}
+
+static void __del_trace_probe_event(int fd, struct str_node *ent)
{
char *p;
char buf[128];
@@ -663,11 +783,11 @@ static void __del_trace_kprobe_event(int fd, struct str_node *ent)
die("Internal error: %s should have ':' but not.", ent->s);
*p = '/';
- write_trace_kprobe_event(fd, buf);
+ write_trace_probe_event(fd, buf);
printf("Remove event: %s\n", ent->s);
}
-static void del_trace_kprobe_event(int fd, const char *group,
+static void del_trace_probe_event(int fd, const char *group,
const char *event, struct strlist *namelist)
{
char buf[128];
@@ -681,14 +801,14 @@ static void del_trace_kprobe_event(int fd, const char *group,
strlist__for_each_safe(ent, n, namelist)
if (strglobmatch(ent->s, buf)) {
found++;
- __del_trace_kprobe_event(fd, ent);
+ __del_trace_probe_event(fd, ent);
strlist__remove(namelist, ent);
}
} else {
ent = strlist__find(namelist, buf);
if (ent) {
found++;
- __del_trace_kprobe_event(fd, ent);
+ __del_trace_probe_event(fd, ent);
strlist__remove(namelist, ent);
}
}
@@ -696,16 +816,13 @@ static void del_trace_kprobe_event(int fd, const char *group,
pr_info("Info: event \"%s\" does not exist, could not remove it.\n", buf);
}
-void del_trace_kprobe_events(struct strlist *dellist)
+static void del_trace_probe_events(int fd, struct strlist *dellist)
{
- int fd;
const char *group, *event;
char *p, *str;
struct str_node *ent;
struct strlist *namelist;
- fd = open_kprobe_events(O_RDWR, O_APPEND);
- /* Get current event names */
namelist = get_perf_event_names(fd, true);
strlist__for_each(ent, dellist) {
@@ -723,13 +840,25 @@ void del_trace_kprobe_events(struct strlist *dellist)
event = str;
}
pr_debug("Group: %s, Event: %s\n", group, event);
- del_trace_kprobe_event(fd, group, event, namelist);
+ del_trace_probe_event(fd, group, event, namelist);
free(str);
}
strlist__delete(namelist);
close(fd);
}
+void del_trace_kprobe_events(struct strlist *dellist)
+{
+ int fd = open_kprobe_events(O_RDWR, O_APPEND);
+ del_trace_probe_events(fd, dellist);
+}
+
+void del_trace_uprobe_events(struct strlist *dellist)
+{
+ int fd = open_uprobe_events(O_RDWR, O_APPEND);
+ del_trace_probe_events(fd, dellist);
+}
+
#define LINEBUF_SIZE 256
#define NR_ADDITIONAL_LINES 2
diff --git a/tools/perf/util/probe-event.h b/tools/perf/util/probe-event.h
index 711287d..bcb7ab2 100644
--- a/tools/perf/util/probe-event.h
+++ b/tools/perf/util/probe-event.h
@@ -7,14 +7,17 @@
extern void parse_line_range_desc(const char *arg, struct line_range *lr);
extern void parse_perf_probe_event(const char *str, struct probe_point *pp,
- bool *need_dwarf);
+ bool *need_dwarf, pid_t pid);
extern int synthesize_perf_probe_point(struct probe_point *pp);
extern int synthesize_perf_probe_event(struct probe_point *pp);
-extern void parse_trace_kprobe_event(const char *str, struct probe_point *pp);
-extern int synthesize_trace_kprobe_event(struct probe_point *pp);
+extern int synthesize_trace_probe_event(struct probe_point *pp);
+extern void parse_trace_probe_event(const char *str, struct probe_point *pp);
extern void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
bool force_add);
extern void del_trace_kprobe_events(struct strlist *dellist);
+extern void add_trace_uprobe_events(struct probe_point *probes, int nr_probes,
+ bool force_add, pid_t pid);
+extern void del_trace_uprobe_events(struct strlist *dellist);
extern void show_perf_probe_events(void);
extern void show_line_range(struct line_range *lr);
diff --git a/tools/perf/util/probe-finder.h b/tools/perf/util/probe-finder.h
index 21f7354..b4d8ebc 100644
--- a/tools/perf/util/probe-finder.h
+++ b/tools/perf/util/probe-finder.h
@@ -33,6 +33,7 @@ struct probe_point {
/* Output */
int found; /* Number of found probe points */
+ pid_t upid; /* uprobes only */
char *probes[MAX_PROBES]; /* Output buffers (will be allocated)*/
};
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index c458c4a..7267050 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -958,12 +958,14 @@ static int dso__load_sym(struct dso *self, struct map *map, const char *name,
nr_syms = shdr.sh_size / shdr.sh_entsize;
memset(&sym, 0, sizeof(sym));
- if (!self->kernel) {
+ if (self->kernel || symbol_conf.sort_by_name)
+ self->adjust_symbols = 0;
+ else {
self->adjust_symbols = (ehdr.e_type == ET_EXEC ||
elf_section_by_name(elf, &ehdr, &shdr,
".gnu.prelink_undo",
NULL) != NULL);
- } else self->adjust_symbols = 0;
+ }
elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
struct symbol *f;
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH v3 10/10] perf: perf interface for uprobes.
2010-05-06 18:03 ` [PATCH v3 10/10] perf: perf interface for uprobes Srikar Dronamraju
@ 2010-05-06 23:13 ` Masami Hiramatsu
2010-05-07 2:24 ` Srikar Dronamraju
0 siblings, 1 reply; 28+ messages in thread
From: Masami Hiramatsu @ 2010-05-06 23:13 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Peter Zijlstra, Ingo Molnar, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
Srikar Dronamraju wrote:
> perf-events.patch
>
> From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>
> This patch enhances perf probe to accept pid and user vaddr.
> This patch provides very basic support for uprobes.
Good work! Thank you!
> TODO:
> Update perf-probes.txt.
> Global tracing.
Could you also rebase your patch on the latest tip tree? :)
(and remove die() too)
> Here is a terminal snapshot of placing, using and removing a probe on a
> process with pid 3329 (corresponding to zsh)
>
> [ Probing a function in the executable using function name ]
> -------------------------------------------------------------
> [root@ABCD]# perf probe -p 3329 zfree@zsh
Hmm, I'm not sure why we have to specify both of PID and exec name.
Is that possible to give a symbol as below? (omit exec name)
# perf probe -p 3329 zfree
> Added new event:
> probe:p_3329_zfree (on 0x446420)
Hmm, the event name should be simpler, as like as zfree_3329.
or, we might better make a new event group for each process, as like as
'probe_3329:zfree'
> You can now use it on all perf tools, such as:
>
> perf record -e probe:p_3329_zfree -a sleep 1
> [root@ABCD]# perf probe --list
> probe:p_3329_zfree (on 3329:0x0000000000446420)
> [root@ABCD]# cat /sys/kernel/debug/tracing/uprobe_events
> p:probe/p_3329_zfree 3329:0x0000000000446420
> [root@ABCD]# perf record -f -e probe:p_3329_zfree -a sleep 10
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.039 MB perf.data (~1716 samples) ]
> [root@ABCD]# perf probe -p 3329 --del probe:p_3329_zfree
> Remove event: probe:p_3329_zfree
> [root@ABCD]# perf report
> # Samples: 447
> #
> # Overhead Command Shared Object Symbol
> # ........ ............... ............. ......
> #
> 100.00% zsh zsh [.] zfree
>
>
> #
> # (For a higher level overview, try: perf report --sort comm,dso)
> #
>
> [ Probing a function + offset ]
> -------------------------------
Looks great! :D
[...]
>
> Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> ---
>
> tools/perf/builtin-probe.c | 34 +++++--
> tools/perf/builtin-top.c | 20 ----
> tools/perf/util/event.c | 20 ++++
> tools/perf/util/event.h | 1
> tools/perf/util/probe-event.c | 207 ++++++++++++++++++++++++++++++++--------
> tools/perf/util/probe-event.h | 9 +-
> tools/perf/util/probe-finder.h | 1
> tools/perf/util/symbol.c | 6 +
> 8 files changed, 224 insertions(+), 74 deletions(-)
Some comments on the patch.
[...]
>
> diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
> index 152d6c9..9f867ad 100644
> --- a/tools/perf/builtin-probe.c
> +++ b/tools/perf/builtin-probe.c
> @@ -55,6 +55,7 @@ static struct {
> bool force_add;
> bool show_lines;
> int nr_probe;
> + pid_t pid;
> struct probe_point probes[MAX_PROBES];
> struct strlist *dellist;
> struct map_groups kmap_groups;
> @@ -73,7 +74,7 @@ static void parse_probe_event(const char *str)
> die("Too many probes (> %d) are specified.", MAX_PROBES);
>
> /* Parse perf-probe event into probe_point */
> - parse_perf_probe_event(str, pp, &session.need_dwarf);
> + parse_perf_probe_event(str, pp, &session.need_dwarf, session.pid);
>
> pr_debug("%d arguments\n", pp->nr_args);
> }
> @@ -203,6 +204,8 @@ static const struct option options[] = {
> "FUNC[:RLN[+NUM|:RLN2]]|SRC:ALN[+NUM|:ALN2]",
> "Show source code lines.", opt_show_lines),
> #endif
> + OPT_INTEGER('p', "pid", &session.pid,
> + "specify a pid for a uprobes based probe"),
> OPT_END()
> };
>
> @@ -263,7 +266,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
> }
>
> #ifndef NO_DWARF_SUPPORT
> - if (session.show_lines) {
> + if (session.show_lines && !session.pid) {
> if (session.nr_probe != 0 || session.dellist) {
> pr_warning(" Error: Don't use --line with"
> " --add/--del.\n");
Hmm, if user gave --list with -p, what happened?
We need to check a bad combination and warn it.
> @@ -283,12 +286,19 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
> #endif
>
> if (session.dellist) {
> - del_trace_kprobe_events(session.dellist);
> + if (session.pid)
> + del_trace_uprobe_events(session.dellist);
> + else
> + del_trace_kprobe_events(session.dellist);
> +
> strlist__delete(session.dellist);
> if (session.nr_probe == 0)
> return 0;
> }
>
> + if (session.pid)
> + goto end_dwarf;
> +
Again, it must be checked that the combination of -p option and dwarf requirement.
The latest code has perf_probe_event_need_dwarf(), so you can check it easier.
[...]
> @@ -112,6 +113,64 @@ static bool check_event_name(const char *name)
> return true;
> }
>
> +/*
> + * uprobe_events only accepts address:
> + * Convert function and any offset to address
> + */
> +static void convert_name_to_addr(struct probe_point *pp)
> +{
> + struct perf_session *session;
> + struct thread *thread;
> + struct symbol *sym;
> + struct map *map;
> + char *name = pp->file;
> + unsigned long long vaddr;
> +
> + /* check if user has specifed a virtual address */
> + vaddr = strtoul(pp->function, NULL, 0);
> + if (vaddr)
> + return;
> +
> + session = perf_session__new(NULL, O_WRONLY, false);
> + DIE_IF(session == NULL);
> + symbol_conf.sort_by_name = true;
> + symbol_conf.try_vmlinux_path = false;
> + if (symbol__init() < 0)
> + semantic_error("Cannot initialize symbols.");
> +
> + event__synthesize_thread(pp->upid, event__process, session);
> +
> + thread = perf_session__findnew(session, pp->upid);
> + DIE_IF(thread == NULL);
> +
> + if (!name)
> + /* Lets find the function in the executable. */
> + name = thread->comm;
> + DIE_IF(name == NULL);
> +
> + map = map_groups__find_by_name(&thread->mg, MAP__FUNCTION, name);
> + if (!map)
> + semantic_error("Cannot find appropriate DSO.");
> +
> + sym = map__find_symbol_by_name(map, pp->function, NULL);
> + if (!sym)
> + semantic_error("Cannot find appropriate Symbol.");
> +
> + if (map->start > sym->start)
> + vaddr = map->start;
> + vaddr += sym->start + pp->offset + map->pgoff;
> + pp->offset = 0;
> +
> + if (!pp->event)
> + pp->event = pp->function;
> + else
> + free(pp->function);
> + pp->function = zalloc(sizeof(char *) * 20);
> + if (!pp->function)
> + die("Failed to allocate memory by zalloc.");
> + e_snprintf(pp->function, 20, "0x%llx", vaddr);
> +}
Well, after rebasing, you'll need to remove die()s from here and
make it returns errors.
[...]
> @@ -383,7 +450,11 @@ int synthesize_trace_kprobe_event(struct probe_point *pp)
> pp->probes[0] = buf = zalloc(MAX_CMDLEN);
> if (!buf)
> die("Failed to allocate memory by zalloc.");
> - ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
> + if (pp->offset)
> + ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function,
> + pp->offset);
> + else
> + ret = e_snprintf(buf, MAX_CMDLEN, "%s", pp->function);
> if (ret <= 0)
> goto error;
> len = ret;
Isn't it a cleanup ?
[...]
> @@ -595,22 +697,25 @@ static void get_new_event_name(char *buf, size_t len, const char *base,
> die("Too many events are on the same function.");
> }
>
> -void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
> - bool force_add)
> +static void add_trace_probe_events(int fd, struct probe_point *probes,
> + int nr_probes, bool force_add, pid_t pid)
> {
> - int i, j, fd;
> + int i, j;
> struct probe_point *pp;
> char buf[MAX_CMDLEN];
> + char tempbuf[MAX_CMDLEN];
> char event[64];
> struct strlist *namelist;
> bool allow_suffix;
>
> - fd = open_kprobe_events(O_RDWR, O_APPEND);
> /* Get current event names */
> namelist = get_perf_event_names(fd, false);
>
> for (j = 0; j < nr_probes; j++) {
> pp = probes + j;
> + pp->upid = pid;
> + if (pid)
> + snprintf(tempbuf, MAX_CMDLEN, "%d:", pid);
> if (!pp->event)
> pp->event = strdup(pp->function);
> if (!pp->group)
> @@ -621,12 +726,13 @@ void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
> for (i = 0; i < pp->found; i++) {
> /* Get an unused new event name */
> get_new_event_name(event, 64, pp->event, namelist,
> - allow_suffix);
> - snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s\n",
> + allow_suffix, pid);
> + snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s%s\n",
> pp->retprobe ? 'r' : 'p',
> pp->group, event,
> + pp->upid ? tempbuf : " ",
> pp->probes[i]);
> - write_trace_kprobe_event(fd, buf);
> + write_trace_probe_event(fd, buf);
> printf("Added new event:\n");
> /* Get the first parameter (probe-point) */
> sscanf(pp->probes[i], "%s", buf);
> @@ -650,7 +756,21 @@ void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
> close(fd);
> }
>
> -static void __del_trace_kprobe_event(int fd, struct str_node *ent)
> +void add_trace_kprobe_events(struct probe_point *probes, int nr_probes,
> + bool force_add)
> +{
> + int fd = open_kprobe_events(O_RDWR, O_APPEND);
> + add_trace_probe_events(fd, probes, nr_probes, force_add, 0);
> +}
> +
> +void add_trace_uprobe_events(struct probe_point *probes, int nr_probes,
> + bool force_add, pid_t pid)
> +{
> + int fd = open_uprobe_events(O_RDWR, O_APPEND);
> + add_trace_probe_events(fd, probes, nr_probes, force_add, pid);
> +}
Please close fd in the function which opened it.
> @@ -696,16 +816,13 @@ static void del_trace_kprobe_event(int fd, const char *group,
> pr_info("Info: event \"%s\" does not exist, could not remove it.\n", buf);
> }
>
> -void del_trace_kprobe_events(struct strlist *dellist)
> +static void del_trace_probe_events(int fd, struct strlist *dellist)
> {
> - int fd;
> const char *group, *event;
> char *p, *str;
> struct str_node *ent;
> struct strlist *namelist;
>
> - fd = open_kprobe_events(O_RDWR, O_APPEND);
> - /* Get current event names */
> namelist = get_perf_event_names(fd, true);
>
> strlist__for_each(ent, dellist) {
> @@ -723,13 +840,25 @@ void del_trace_kprobe_events(struct strlist *dellist)
> event = str;
> }
> pr_debug("Group: %s, Event: %s\n", group, event);
> - del_trace_kprobe_event(fd, group, event, namelist);
> + del_trace_probe_event(fd, group, event, namelist);
> free(str);
> }
> strlist__delete(namelist);
> close(fd);
> }
>
> +void del_trace_kprobe_events(struct strlist *dellist)
> +{
> + int fd = open_kprobe_events(O_RDWR, O_APPEND);
> + del_trace_probe_events(fd, dellist);
> +}
> +
> +void del_trace_uprobe_events(struct strlist *dellist)
> +{
> + int fd = open_uprobe_events(O_RDWR, O_APPEND);
> + del_trace_probe_events(fd, dellist);
> +}
Ditto.
[...]
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index c458c4a..7267050 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
> @@ -958,12 +958,14 @@ static int dso__load_sym(struct dso *self, struct map *map, const char *name,
> nr_syms = shdr.sh_size / shdr.sh_entsize;
>
> memset(&sym, 0, sizeof(sym));
> - if (!self->kernel) {
> + if (self->kernel || symbol_conf.sort_by_name)
> + self->adjust_symbols = 0;
> + else {
> self->adjust_symbols = (ehdr.e_type == ET_EXEC ||
> elf_section_by_name(elf, &ehdr, &shdr,
> ".gnu.prelink_undo",
> NULL) != NULL);
> - } else self->adjust_symbols = 0;
> + }
>
> elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
> struct symbol *f;
This one changes the symbol.c, so I think you'd better make a
separate patch for this change.
Thank you,
--
Masami Hiramatsu
e-mail: mhiramat@redhat.com
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 10/10] perf: perf interface for uprobes.
2010-05-06 23:13 ` Masami Hiramatsu
@ 2010-05-07 2:24 ` Srikar Dronamraju
2010-05-07 17:53 ` Masami Hiramatsu
0 siblings, 1 reply; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-07 2:24 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Peter Zijlstra, Ingo Molnar, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
>
> Could you also rebase your patch on the latest tip tree? :)
> (and remove die() too)
I was aware that I have to rebase the patch to the latest tip.
However I wanted to get a working set out.
Please expect the rebased version in the next version of the posting.
>
> > Here is a terminal snapshot of placing, using and removing a probe on a
> > process with pid 3329 (corresponding to zsh)
> >
> > [ Probing a function in the executable using function name ]
> > -------------------------------------------------------------
> > [root@ABCD]# perf probe -p 3329 zfree@zsh
>
> Hmm, I'm not sure why we have to specify both of PID and exec name.
> Is that possible to give a symbol as below? (omit exec name)
>
> # perf probe -p 3329 zfree
Yes, If the symbol name belongs to th execfile then we can omit the
execfile part. However for symbols outside the execname, the
execfile(library file) is a must(atleast for now).
>
> > Added new event:
> > probe:p_3329_zfree (on 0x446420)
>
> Hmm, the event name should be simpler, as like as zfree_3329.
> or, we might better make a new event group for each process, as like as
> 'probe_3329:zfree'
>
Either zfree_3329 or having a new event group for each process is
possible.
>
>
> Hmm, if user gave --list with -p, what happened?
> We need to check a bad combination and warn it.
>
Currently it would ignore -p option. i.e --list overrides.
We could either warn, or error out. What do you suggest?
> >
> > + if (session.pid)
> > + goto end_dwarf;
> > +
>
> Again, it must be checked that the combination of -p option and dwarf requirement.
> The latest code has perf_probe_event_need_dwarf(), so you can check it easier.
>
Okay,
> [...]
> > + else
> > + free(pp->function);
> > + pp->function = zalloc(sizeof(char *) * 20);
> > + if (!pp->function)
> > + die("Failed to allocate memory by zalloc.");
> > + e_snprintf(pp->function, 20, "0x%llx", vaddr);
> > +}
>
> Well, after rebasing, you'll need to remove die()s from here and
> make it returns errors.
>
Okay.
> [...]
> > @@ -383,7 +450,11 @@ int synthesize_trace_kprobe_event(struct probe_point *pp)
> > pp->probes[0] = buf = zalloc(MAX_CMDLEN);
> > if (!buf)
> > die("Failed to allocate memory by zalloc.");
> > - ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
> > + if (pp->offset)
> > + ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function,
> > + pp->offset);
> > + else
> > + ret = e_snprintf(buf, MAX_CMDLEN, "%s", pp->function);
> > if (ret <= 0)
> > goto error;
> > len = ret;
>
> Isn't it a cleanup ?
Can you please elaborate?
>
> > +void add_trace_uprobe_events(struct probe_point *probes, int nr_probes,
> > + bool force_add, pid_t pid)
> > +{
> > + int fd = open_uprobe_events(O_RDWR, O_APPEND);
> > + add_trace_probe_events(fd, probes, nr_probes, force_add, pid);
> > +}
>
> Please close fd in the function which opened it.
>
Okay.
>
> > memset(&sym, 0, sizeof(sym));
> > - if (!self->kernel) {
> > + if (self->kernel || symbol_conf.sort_by_name)
> > + self->adjust_symbols = 0;
> > + else {
> > self->adjust_symbols = (ehdr.e_type == ET_EXEC ||
> > elf_section_by_name(elf, &ehdr, &shdr,
> > ".gnu.prelink_undo",
> > NULL) != NULL);
> > - } else self->adjust_symbols = 0;
> > + }
> >
> > elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
> > struct symbol *f;
>
> This one changes the symbol.c, so I think you'd better make a
> separate patch for this change.
Okay, I will separate it in the next version.
Thanks Masami for the review.
--
Thanks and Regards
Srikar
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 10/10] perf: perf interface for uprobes.
2010-05-07 2:24 ` Srikar Dronamraju
@ 2010-05-07 17:53 ` Masami Hiramatsu
2010-05-09 11:18 ` Srikar Dronamraju
0 siblings, 1 reply; 28+ messages in thread
From: Masami Hiramatsu @ 2010-05-07 17:53 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Peter Zijlstra, Ingo Molnar, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
Srikar Dronamraju wrote:
>>
>> Could you also rebase your patch on the latest tip tree? :)
>> (and remove die() too)
>
> I was aware that I have to rebase the patch to the latest tip.
> However I wanted to get a working set out.
> Please expect the rebased version in the next version of the posting.
>
>>
>>> Here is a terminal snapshot of placing, using and removing a probe on a
>>> process with pid 3329 (corresponding to zsh)
>>>
>>> [ Probing a function in the executable using function name ]
>>> -------------------------------------------------------------
>>> [root@ABCD]# perf probe -p 3329 zfree@zsh
>>
>> Hmm, I'm not sure why we have to specify both of PID and exec name.
>> Is that possible to give a symbol as below? (omit exec name)
>>
>> # perf probe -p 3329 zfree
>
> Yes, If the symbol name belongs to th execfile then we can omit the
> execfile part. However for symbols outside the execname, the
> execfile(library file) is a must(atleast for now).
Good!:)
>>> Added new event:
>>> probe:p_3329_zfree (on 0x446420)
>>
>> Hmm, the event name should be simpler, as like as zfree_3329.
>> or, we might better make a new event group for each process, as like as
>> 'probe_3329:zfree'
>>
>
> Either zfree_3329 or having a new event group for each process is
> possible.
I think adding new groups make controlling those event easier.
(e.g. deleting all probes on exited process)
>> Hmm, if user gave --list with -p, what happened?
>> We need to check a bad combination and warn it.
>>
>
> Currently it would ignore -p option. i.e --list overrides.
> We could either warn, or error out. What do you suggest?
>
>>>
>>> + if (session.pid)
>>> + goto end_dwarf;
>>> +
>>
>> Again, it must be checked that the combination of -p option and dwarf requirement.
>> The latest code has perf_probe_event_need_dwarf(), so you can check it easier.
>>
>
> Okay,
>
>> [...]
>>> + else
>>> + free(pp->function);
>>> + pp->function = zalloc(sizeof(char *) * 20);
>>> + if (!pp->function)
>>> + die("Failed to allocate memory by zalloc.");
>>> + e_snprintf(pp->function, 20, "0x%llx", vaddr);
>>> +}
>>
>> Well, after rebasing, you'll need to remove die()s from here and
>> make it returns errors.
>>
>
> Okay.
>
>
>> [...]
>>> @@ -383,7 +450,11 @@ int synthesize_trace_kprobe_event(struct probe_point *pp)
>>> pp->probes[0] = buf = zalloc(MAX_CMDLEN);
>>> if (!buf)
>>> die("Failed to allocate memory by zalloc.");
>>> - ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
>>> + if (pp->offset)
>>> + ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function,
>>> + pp->offset);
>>> + else
>>> + ret = e_snprintf(buf, MAX_CMDLEN, "%s", pp->function);
>>> if (ret <= 0)
>>> goto error;
>>> len = ret;
>>
>> Isn't it a cleanup ?
>
> Can you please elaborate?
symbol+offset is always allowed even if offset == 0,
so I think this change isn't needed. I guessed it is just for cleanup.
>>> +void add_trace_uprobe_events(struct probe_point *probes, int nr_probes,
>>> + bool force_add, pid_t pid)
>>> +{
>>> + int fd = open_uprobe_events(O_RDWR, O_APPEND);
>>> + add_trace_probe_events(fd, probes, nr_probes, force_add, pid);
>>> +}
>>
>> Please close fd in the function which opened it.
>>
>
> Okay.
>
>>
>>> memset(&sym, 0, sizeof(sym));
>>> - if (!self->kernel) {
>>> + if (self->kernel || symbol_conf.sort_by_name)
>>> + self->adjust_symbols = 0;
>>> + else {
>>> self->adjust_symbols = (ehdr.e_type == ET_EXEC ||
>>> elf_section_by_name(elf, &ehdr, &shdr,
>>> ".gnu.prelink_undo",
>>> NULL) != NULL);
>>> - } else self->adjust_symbols = 0;
>>> + }
>>>
>>> elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
>>> struct symbol *f;
>>
>> This one changes the symbol.c, so I think you'd better make a
>> separate patch for this change.
>
> Okay, I will separate it in the next version.
> Thanks Masami for the review.
Thank you!
--
Masami Hiramatsu
e-mail: mhiramat@redhat.com
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 10/10] perf: perf interface for uprobes.
2010-05-07 17:53 ` Masami Hiramatsu
@ 2010-05-09 11:18 ` Srikar Dronamraju
0 siblings, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-09 11:18 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Peter Zijlstra, Ingo Molnar, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
> >>
> >
> > Either zfree_3329 or having a new event group for each process is
> > possible.
>
> I think adding new groups make controlling those event easier.
> (e.g. deleting all probes on exited process)
>
Okay..
>
> >> [...]
> >>> @@ -383,7 +450,11 @@ int synthesize_trace_kprobe_event(struct probe_point *pp)
> >>> pp->probes[0] = buf = zalloc(MAX_CMDLEN);
> >>> if (!buf)
> >>> die("Failed to allocate memory by zalloc.");
> >>> - ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
> >>> + if (pp->offset)
> >>> + ret = e_snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function,
> >>> + pp->offset);
> >>> + else
> >>> + ret = e_snprintf(buf, MAX_CMDLEN, "%s", pp->function);
> >>> if (ret <= 0)
> >>> goto error;
> >>> len = ret;
> >>
> >> Isn't it a cleanup ?
> >
> > Can you please elaborate?
>
> symbol+offset is always allowed even if offset == 0,
> so I think this change isn't needed. I guessed it is just for cleanup.
>
kprobes has an advantage that it can make sense of kernel symbols.
However uprobes doesnt have the advantage. It cannot make any sense of
user space symbols. Hence unlike kprobes, uprobes doesnt have
symbol_name and offset fields. To overcome this the perf probe for
uprobes resolves the symbol_name, offset and pass the plain vaddr to
uprobes.
--
Thanks and Regards
Srikar Dronamraju
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (9 preceding siblings ...)
2010-05-06 18:03 ` [PATCH v3 10/10] perf: perf interface for uprobes Srikar Dronamraju
@ 2010-05-11 20:59 ` Peter Zijlstra
2010-05-12 10:25 ` Srikar Dronamraju
2010-05-12 10:38 ` Frederic Weisbecker
11 siblings, 1 reply; 28+ messages in thread
From: Peter Zijlstra @ 2010-05-11 20:59 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Paul E. McKenney
On Thu, 2010-05-06 at 23:31 +0530, Srikar Dronamraju wrote:
> - Addressed comments from Oleg, including removal of interrupt context
> handlers, reverting background page replacement in favour of
> access_process_vm().
> +static int write_opcode(struct task_struct *tsk, unsigned long vaddr,
> + user_bkpt_opcode_t opcode)
> +{
> + int ret;
> +
> + if (!tsk)
> + return -EINVAL;
> +
> + ret = access_process_vm(tsk, vaddr, &opcode, user_bkpt_opcode_sz, 1);
> + return (ret == user_bkpt_opcode_sz ? 0 : -EFAULT);
> +}
Why!
That's not not the atomic sequence outlined.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 0/10] Uprobes v3
2010-05-11 20:59 ` [PATCH v3 0/10] Uprobes v3 Peter Zijlstra
@ 2010-05-12 10:25 ` Srikar Dronamraju
2010-05-12 12:13 ` Peter Zijlstra
0 siblings, 1 reply; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-12 10:25 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Andrea Arcangeli, Hugh Dickins,
Rik van Riel, Paul E. McKenney
* Peter Zijlstra <peterz@infradead.org> [2010-05-11 22:59:45]:
> On Thu, 2010-05-06 at 23:31 +0530, Srikar Dronamraju wrote:
> > - Addressed comments from Oleg, including removal of interrupt context
> > handlers, reverting background page replacement in favour of
> > access_process_vm().
>
>
> > +static int write_opcode(struct task_struct *tsk, unsigned long vaddr,
> > + user_bkpt_opcode_t opcode)
> > +{
> > + int ret;
> > +
> > + if (!tsk)
> > + return -EINVAL;
> > +
> > + ret = access_process_vm(tsk, vaddr, &opcode, user_bkpt_opcode_sz, 1);
> > + return (ret == user_bkpt_opcode_sz ? 0 : -EFAULT);
> > +}
>
> Why!
>
> That's not not the atomic sequence outlined.
>
Yes, we had moved away from access_process_vm to background page
replacement in Version 1 and Version 2.
One of the reasons being Mathieu suggesting to Jim in LFCS that
for almost all architectures insertion of a breakpoint instruction on a
user page is an atomic operation, as far as the CPU is concerned.
Can you and other VM experts tell me if access_process_vm isnt going to
be atomic with respect to inserting/deleting a breakpoint instruction?
Oleg had few questions which I didnt have answers. (Most of
which you have already answered yesterday). One thing that's still
missing is
[ snipping from Oleg's mail: ]
-----
But suppose that the application does mprotect(PROT_WRITE)
after register_uprobe() installs the bp, now unregister_uprobe/etc
can't restore the original insn?
---
Also I tried a write_opcode that uses background page replacement which
addressed some of Oleg's comments. The pseudo-code is here:
write_opcode()
{
down_read(mmap_sem);
get_user_pages(tsk, mm, vaddr, .. &old_page, &vma);
anon_vma_prepare(vma);
new_page=alloc_page_vma(.., vma, vaddr);
copy_user_page(new_page, old_page);
kmap_atomic(new_page,...);
memcpy(vaddr,..);
kunmap_atomic(..);
lock_page(new_page);
old_pte = get_pte(mm,vaddr);
replace_page(vma, new_page, old_page, old_pte);
unlock_page(new_page);
put_page(new_page);
put_page(old_page);
up_read(mmap);
}
Will this work?
The Other VM quieries that I had were:
Is there any thing else needed for the parent process to pass on the anon_vma to
the child process. (I inserted a breakpoint in the parent and tried
removing the breakpoint in the child.
However page_address_in_vma() (called by replace_page() returned
EFAULT because "vma->anon_vma != page_anon_vma(page)"
Do we need to take care of mem_cgroups?
Do we need to update mm counters?
--
Thanks and Regards
Srikar
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 10:25 ` Srikar Dronamraju
@ 2010-05-12 12:13 ` Peter Zijlstra
2010-05-12 13:27 ` Ananth N Mavinakayanahalli
2010-05-13 12:07 ` Srikar Dronamraju
0 siblings, 2 replies; 28+ messages in thread
From: Peter Zijlstra @ 2010-05-12 12:13 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Andrea Arcangeli, Hugh Dickins,
Rik van Riel, Paul E. McKenney
On Wed, 2010-05-12 at 15:55 +0530, Srikar Dronamraju wrote:
> * Peter Zijlstra <peterz@infradead.org> [2010-05-11 22:59:45]:
>
> > On Thu, 2010-05-06 at 23:31 +0530, Srikar Dronamraju wrote:
> > > - Addressed comments from Oleg, including removal of interrupt context
> > > handlers, reverting background page replacement in favour of
> > > access_process_vm().
> >
> >
> > > +static int write_opcode(struct task_struct *tsk, unsigned long vaddr,
> > > + user_bkpt_opcode_t opcode)
> > > +{
> > > + int ret;
> > > +
> > > + if (!tsk)
> > > + return -EINVAL;
> > > +
> > > + ret = access_process_vm(tsk, vaddr, &opcode, user_bkpt_opcode_sz, 1);
> > > + return (ret == user_bkpt_opcode_sz ? 0 : -EFAULT);
> > > +}
> >
> > Why!
> >
> > That's not not the atomic sequence outlined.
> >
>
>
> Yes, we had moved away from access_process_vm to background page
> replacement in Version 1 and Version 2.
>
> One of the reasons being Mathieu suggesting to Jim in LFCS that
> for almost all architectures insertion of a breakpoint instruction on a
> user page is an atomic operation, as far as the CPU is concerned.
That is true, however when restoring the old instruction you do need to
take care, so your usage from set_orig_insn() is dubious.
> Can you and other VM experts tell me if access_process_vm isnt going to
> be atomic with respect to inserting/deleting a breakpoint instruction?
Well, clearly not, since it simply does a gup(.force=1), if whatever
page is there is writable it will simply poke at it.
Now writing the INT3 is special, but restoring the old insn is not and
might confuse another CPU that is currently trying to execute code near
there.
All in all, doing the page flip thing isn't hard at all and does avoid
any and all potential CPU decoder/i$ races.
> Oleg had few questions which I didnt have answers. (Most of
> which you have already answered yesterday). One thing that's still
> missing is
>
> [ snipping from Oleg's mail: ]
> -----
> But suppose that the application does mprotect(PROT_WRITE)
> after register_uprobe() installs the bp, now unregister_uprobe/etc
> can't restore the original insn?
I wouldn't see why not. The mprotect() will only allow the application
to poke at its own private copy of the page (the one you COWed for it).
Of course the app, a ptrace-client and uprobes could go fighting over
who owns what INT3 (all 3 poking an INT3 at the same place, and one of
them removing it will confuse matters, but that's unavoidable).
But a nicely written flip will be atomic wrt concurrent writes (the
below is not).
> Also I tried a write_opcode that uses background page replacement which
> addressed some of Oleg's comments. The pseudo-code is here:
> write_opcode()
> {
> down_read(mmap_sem);
> get_user_pages(tsk, mm, vaddr, .. &old_page, &vma);
> anon_vma_prepare(vma);
> new_page=alloc_page_vma(.., vma, vaddr);
> copy_user_page(new_page, old_page);
> kmap_atomic(new_page,...);
> memcpy(vaddr,..);
> kunmap_atomic(..);
> lock_page(new_page);
> old_pte = get_pte(mm,vaddr);
> replace_page(vma, new_page, old_page, old_pte);
> unlock_page(new_page);
> put_page(new_page);
> put_page(old_page);
> up_read(mmap);
> }
>
>
> Will this work?
No, that's full of races. you need to copy and modify the original page
while holding the pte-lock and after you cleared the original pte.
You could possible skip the whole COW break logic by cheating and making
gup() do that, it would be a little extra overhead, but a lot less to
worry about.
Hrmm, you could do something like the above and use write_protect_page()
along with that lock_page() you do to sync against concurrent faults. It
won't be the fastest code around but it should work I guess.
Something like try_to_merge_one_page() in the KSM code:
write_opcode(struct mm_struct *mm, unsigned long address, void *insn,
int len)
{
down_read(mm->mmap_sem);
/* let gup() deal with COW and VM accounting */
nr = get_user_pages(tsk, mm, address, 1, 1, 1, &old_page, &vma);
if (!nr) goto fail;
/* get ourselves a new page */
new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address);
if (!new_page) goto fail;
/* lock page will serialize against do_wp_page()'s PageAnon() handling */
lock_page(old_page);
/* mark the page RO so any concurrent access will end up in do_wp_page() */
if (write_protect_page(vma, old_page, &orig_pte)) goto fail_locked;
/* copy the page now that we've got it stable */
vaddr_old = kmap_atomic(old_page);
vaddr_new = kmap_atomic(new_page);
memcpy(vaddr_new, vaddr_old, PAGE_SIZE);
/* poke the new insn in, ASSUMES we don't cross page boundary */
memcpy(vaddr_new + (address & PAGE_MASK), insn, len);
kunmap_atomic(vaddr_new);
kunmap_atomic(vaddr_old);
/* XXX: frob mlock state */
/* flip pages, do_wp_page() will fail pte_same() and bail */
err = replace_page(vma, new_page, old_page, orig_pte)
if (err) goto fail_more;
/* we're done, let 'em rip */
unlock_page(old_page);
put_page(old_page);
return;
fail:
/* XXX: fixup stuff */
}
Note that this can fail for fun reasons like O_DIRECT IO, but that
should be very rare indeed on text pages.
> The Other VM quieries that I had were:
>
> Is there any thing else needed for the parent process to pass on the anon_vma to
> the child process. (I inserted a breakpoint in the parent and tried
> removing the breakpoint in the child.
> However page_address_in_vma() (called by replace_page() returned
> EFAULT because "vma->anon_vma != page_anon_vma(page)"
Yeah, anon_vma is a bit of a pain after Rik changed it, you need to
compare the oldest entries on the list or something.
> Do we need to take care of mem_cgroups?
Dunno, I've so far successfully avoided looking at them, but yeah, I
think you need to do like the regular COW path does when breaking COW.
> Do we need to update mm counters?
When breaking COW, yes, but the above code cheats and lets gup() worry
about that.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 12:13 ` Peter Zijlstra
@ 2010-05-12 13:27 ` Ananth N Mavinakayanahalli
2010-05-12 13:39 ` Peter Zijlstra
2010-05-13 12:07 ` Srikar Dronamraju
1 sibling, 1 reply; 28+ messages in thread
From: Ananth N Mavinakayanahalli @ 2010-05-12 13:27 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Srikar Dronamraju, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Oleg Nesterov, Mark Wielaard, Mathieu Desnoyers, LKML,
Jim Keniston, Frederic Weisbecker, Frank Ch. Eigler,
Andrew Morton, Andrea Arcangeli, Hugh Dickins, Rik van Riel,
Paul E. McKenney
On Wed, May 12, 2010 at 02:13:05PM +0200, Peter Zijlstra wrote:
> On Wed, 2010-05-12 at 15:55 +0530, Srikar Dronamraju wrote:
> > * Peter Zijlstra <peterz@infradead.org> [2010-05-11 22:59:45]:
> >
> > > On Thu, 2010-05-06 at 23:31 +0530, Srikar Dronamraju wrote:
> > > > - Addressed comments from Oleg, including removal of interrupt context
> > > > handlers, reverting background page replacement in favour of
> > > > access_process_vm().
> > >
> > >
> > > > +static int write_opcode(struct task_struct *tsk, unsigned long vaddr,
> > > > + user_bkpt_opcode_t opcode)
> > > > +{
> > > > + int ret;
> > > > +
> > > > + if (!tsk)
> > > > + return -EINVAL;
> > > > +
> > > > + ret = access_process_vm(tsk, vaddr, &opcode, user_bkpt_opcode_sz, 1);
> > > > + return (ret == user_bkpt_opcode_sz ? 0 : -EFAULT);
> > > > +}
> > >
> > > Why!
> > >
> > > That's not not the atomic sequence outlined.
> > >
> >
> >
> > Yes, we had moved away from access_process_vm to background page
> > replacement in Version 1 and Version 2.
> >
> > One of the reasons being Mathieu suggesting to Jim in LFCS that
> > for almost all architectures insertion of a breakpoint instruction on a
> > user page is an atomic operation, as far as the CPU is concerned.
>
> That is true, however when restoring the old instruction you do need to
> take care, so your usage from set_orig_insn() is dubious.
>
> > Can you and other VM experts tell me if access_process_vm isnt going to
> > be atomic with respect to inserting/deleting a breakpoint instruction?
>
> Well, clearly not, since it simply does a gup(.force=1), if whatever
> page is there is writable it will simply poke at it.
>
> Now writing the INT3 is special, but restoring the old insn is not and
> might confuse another CPU that is currently trying to execute code near
> there.
Yes, this helps for breakpoint insertion, but...
The question is whether only INT3 special or single-byte changes are
also guaranteed to be atomic. In http://lkml.org/lkml/2010/1/27/275
Peter Anvin states 'specific case of a more generic rule'.
For restoring the old instruction, we technically need to put back just
one byte, irrespective of the actual length of the underlying
instruction. Now, as long as we have the housekeeping code to handle the
possibility of a thread hitting the said breakpoint when its being
removed, is it safe to assume atomicity for replacing one byte of
possibly a longer instruction?
Ananth
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 13:27 ` Ananth N Mavinakayanahalli
@ 2010-05-12 13:39 ` Peter Zijlstra
2010-05-12 14:04 ` Ananth N Mavinakayanahalli
0 siblings, 1 reply; 28+ messages in thread
From: Peter Zijlstra @ 2010-05-12 13:39 UTC (permalink / raw)
To: ananth
Cc: Srikar Dronamraju, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Oleg Nesterov, Mark Wielaard, Mathieu Desnoyers, LKML,
Jim Keniston, Frederic Weisbecker, Frank Ch. Eigler,
Andrew Morton, Andrea Arcangeli, Hugh Dickins, Rik van Riel,
Paul E. McKenney
On Wed, 2010-05-12 at 18:57 +0530, Ananth N Mavinakayanahalli wrote:
> Now, as long as we have the housekeeping code to handle the
> possibility of a thread hitting the said breakpoint when its being
> removed, is it safe to assume atomicity for replacing one byte of
> possibly a longer instruction?
Dunno I'm not a hardware guy, but the issue is so simple to side-step
I'm not sure why you're arguing for relying on these special semantics.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 13:39 ` Peter Zijlstra
@ 2010-05-12 14:04 ` Ananth N Mavinakayanahalli
2010-05-12 14:46 ` Mathieu Desnoyers
0 siblings, 1 reply; 28+ messages in thread
From: Ananth N Mavinakayanahalli @ 2010-05-12 14:04 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Srikar Dronamraju, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Oleg Nesterov, Mark Wielaard, Mathieu Desnoyers, LKML,
Jim Keniston, Frederic Weisbecker, Frank Ch. Eigler,
Andrew Morton, Andrea Arcangeli, Hugh Dickins, Rik van Riel,
Paul E. McKenney
On Wed, May 12, 2010 at 03:39:20PM +0200, Peter Zijlstra wrote:
> On Wed, 2010-05-12 at 18:57 +0530, Ananth N Mavinakayanahalli wrote:
> > Now, as long as we have the housekeeping code to handle the
> > possibility of a thread hitting the said breakpoint when its being
> > removed, is it safe to assume atomicity for replacing one byte of
> > possibly a longer instruction?
>
> Dunno I'm not a hardware guy, but the issue is so simple to side-step
> I'm not sure why you're arguing for relying on these special semantics.
Yes we know what to do, but I am just trying to get clarity if its
possible at all, since Mathieu was pretty sure that the hoops aren't
necessary...
Ananth
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 14:04 ` Ananth N Mavinakayanahalli
@ 2010-05-12 14:46 ` Mathieu Desnoyers
2010-05-12 16:55 ` H. Peter Anvin
0 siblings, 1 reply; 28+ messages in thread
From: Mathieu Desnoyers @ 2010-05-12 14:46 UTC (permalink / raw)
To: Ananth N Mavinakayanahalli
Cc: Peter Zijlstra, Srikar Dronamraju, Ingo Molnar, Masami Hiramatsu,
Mel Gorman, Randy Dunlap, Linus Torvalds, Roland McGrath,
H. Peter Anvin, Oleg Nesterov, Mark Wielaard, LKML, Jim Keniston,
Frederic Weisbecker, Frank Ch. Eigler, Andrew Morton,
Andrea Arcangeli, Hugh Dickins, Rik van Riel, Paul E. McKenney
* Ananth N Mavinakayanahalli (ananth@in.ibm.com) wrote:
> On Wed, May 12, 2010 at 03:39:20PM +0200, Peter Zijlstra wrote:
> > On Wed, 2010-05-12 at 18:57 +0530, Ananth N Mavinakayanahalli wrote:
> > > Now, as long as we have the housekeeping code to handle the
> > > possibility of a thread hitting the said breakpoint when its being
> > > removed, is it safe to assume atomicity for replacing one byte of
> > > possibly a longer instruction?
> >
> > Dunno I'm not a hardware guy, but the issue is so simple to side-step
> > I'm not sure why you're arguing for relying on these special semantics.
>
> Yes we know what to do, but I am just trying to get clarity if its
> possible at all, since Mathieu was pretty sure that the hoops aren't
> necessary...
>From what I gathered in the Intel code modification documentation, setting a
breakpoint instruction is both atomic and OK with respect to concurrent CPUs
executing the code. The breakpoint instruction itself deals with serialization
of remote cores when it is hit.
Placing back _the original instruction_ is considered as safe too, as it is both
atomic and brings back the remote cores to the original code state. Basically,
these cores either observe the int3, or the original code. If they observe the
int3, they perform core serialization. If they observe the original code, they
either have never observed the int3 version (which is fine, it is as if the code
had never changed), or if they did observe the int3, they executed the
serialization instruction at the int3, so it's OK to put back the original
instruction.
Now the tricky case is the sequence: instruction A -> int3 -> instruction B,
because a core can only see "instruction A -> instruction B" without any
core synchronization whatsoever, and may not see the int3. That's where the
djprobes logic (with IPIs to all cores) comes into play. But as long as we stick
to "insn A -> int3 -> insn A", things can be done very simply.
By the way, kprobes rely on the assumption that it is OK to put a breakpoint
atomically and to put back the original instruction afterward.
Thanks,
Mathieu
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 14:46 ` Mathieu Desnoyers
@ 2010-05-12 16:55 ` H. Peter Anvin
2010-05-12 17:59 ` Mathieu Desnoyers
0 siblings, 1 reply; 28+ messages in thread
From: H. Peter Anvin @ 2010-05-12 16:55 UTC (permalink / raw)
To: Mathieu Desnoyers
Cc: Ananth N Mavinakayanahalli, Peter Zijlstra, Srikar Dronamraju,
Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, Oleg Nesterov, Mark Wielaard,
LKML, Jim Keniston, Frederic Weisbecker, Frank Ch. Eigler,
Andrew Morton, Andrea Arcangeli, Hugh Dickins, Rik van Riel,
Paul E. McKenney
On 05/12/2010 07:46 AM, Mathieu Desnoyers wrote:
>
> Now the tricky case is the sequence: instruction A -> int3 -> instruction B,
> because a core can only see "instruction A -> instruction B" without any
> core synchronization whatsoever, and may not see the int3. That's where the
> djprobes logic (with IPIs to all cores) comes into play. But as long as we stick
> to "insn A -> int3 -> insn A", things can be done very simply.
>
> By the way, kprobes rely on the assumption that it is OK to put a breakpoint
> atomically and to put back the original instruction afterward.
>
Keep in mind the following corner case, though:
insnA -> int3@A -> insnA
insnB -> int3@B -> insnB
It is now possible for the core to hit int3@A, without the int3@B being
there. The int3 handler *has* to be able to handle any of the int3's
put in place, quite possibly out of order, until a core serialization is
performed.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 16:55 ` H. Peter Anvin
@ 2010-05-12 17:59 ` Mathieu Desnoyers
0 siblings, 0 replies; 28+ messages in thread
From: Mathieu Desnoyers @ 2010-05-12 17:59 UTC (permalink / raw)
To: H. Peter Anvin
Cc: Ananth N Mavinakayanahalli, Peter Zijlstra, Srikar Dronamraju,
Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, Oleg Nesterov, Mark Wielaard,
LKML, Jim Keniston, Frederic Weisbecker, Frank Ch. Eigler,
Andrew Morton, Andrea Arcangeli, Hugh Dickins, Rik van Riel,
Paul E. McKenney
* H. Peter Anvin (hpa@zytor.com) wrote:
> On 05/12/2010 07:46 AM, Mathieu Desnoyers wrote:
> >
> > Now the tricky case is the sequence: instruction A -> int3 -> instruction B,
> > because a core can only see "instruction A -> instruction B" without any
> > core synchronization whatsoever, and may not see the int3. That's where the
> > djprobes logic (with IPIs to all cores) comes into play. But as long as we stick
> > to "insn A -> int3 -> insn A", things can be done very simply.
> >
> > By the way, kprobes rely on the assumption that it is OK to put a breakpoint
> > atomically and to put back the original instruction afterward.
> >
>
> Keep in mind the following corner case, though:
>
> insnA -> int3@A -> insnA
> insnB -> int3@B -> insnB
>
> It is now possible for the core to hit int3@A, without the int3@B being
> there. The int3 handler *has* to be able to handle any of the int3's
> put in place, quite possibly out of order, until a core serialization is
> performed.
Indeed. I would add that this out-of-order observation of breakpoint
modification is already expected, because other cores may be executing arbitrary
code when we add the two breakpoints (either insnA, B, or somewhere in between,
or somewhere else entirely).
Basically, we can make no assumption about the order in which the two stores
will be seen by other CPUs. But I can't imagine a debugger expecting these two
stores to be observed in order.
If we ever want to provide a _guarantee_ that starting from a certain point all
CPUs will see a set of breakpoint, then we need to issue a core serializing
instruction on all CPUs. This would determine the "quiescent state" of the CPUs,
starting from which all of them see the breakpoints.
Thanks,
Mathieu
>
> -hpa
>
> --
> H. Peter Anvin, Intel Open Source Technology Center
> I work for Intel. I don't speak on their behalf.
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 12:13 ` Peter Zijlstra
2010-05-12 13:27 ` Ananth N Mavinakayanahalli
@ 2010-05-13 12:07 ` Srikar Dronamraju
1 sibling, 0 replies; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-13 12:07 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Ingo Molnar, Masami Hiramatsu, Mel Gorman, Randy Dunlap,
Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frederic Weisbecker,
Frank Ch. Eigler, Andrew Morton, Andrea Arcangeli, Hugh Dickins,
Rik van Riel, Paul E. McKenney
>
> Hrmm, you could do something like the above and use write_protect_page()
> along with that lock_page() you do to sync against concurrent faults. It
> won't be the fastest code around but it should work I guess.
Okay, thanks for the code.
>
> Something like try_to_merge_one_page() in the KSM code:
>
> write_opcode(struct mm_struct *mm, unsigned long address, void *insn,
> int len)
> {
> down_read(mm->mmap_sem);
[ snipped]
>
> kunmap_atomic(vaddr_new);
> kunmap_atomic(vaddr_old);
>
> /* XXX: frob mlock state */
I think you are referring to mlock_vma_page() and munlock_vma_page that
we have in try_to_merge_one_page(). However mlock_vma_page and
munlock_vma_page are not exposed outside mm as they are declared inside
mm/internal.h. So do you suggest moving the declarations out of
mm/internal.h or have a wrapper function defined in mm/ that could be
used by both KSM and uprobes.
>
> /* flip pages, do_wp_page() will fail pte_same() and bail */
> err = replace_page(vma, new_page, old_page, orig_pte)
> if (err) goto fail_more;
>
> /* we're done, let 'em rip */
> unlock_page(old_page);
> put_page(old_page);
>
> return;
>
> fail:
> /* XXX: fixup stuff */
> }
>
> Note that this can fail for fun reasons like O_DIRECT IO, but that
> should be very rare indeed on text pages.
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-06 18:01 [PATCH v3 0/10] Uprobes v3 Srikar Dronamraju
` (10 preceding siblings ...)
2010-05-11 20:59 ` [PATCH v3 0/10] Uprobes v3 Peter Zijlstra
@ 2010-05-12 10:38 ` Frederic Weisbecker
2010-05-12 13:39 ` Srikar Dronamraju
11 siblings, 1 reply; 28+ messages in thread
From: Frederic Weisbecker @ 2010-05-12 10:38 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Peter Zijlstra, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frank Ch. Eigler,
Andrew Morton, Paul E. McKenney
On Thu, May 06, 2010 at 11:31:39PM +0530, Srikar Dronamraju wrote:
> intro.patch
>
> From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>
> Uprobes Patches
I can't find the trace event patch inside this set.
May be you missed it somehow? (or I did).
Thanks.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 10:38 ` Frederic Weisbecker
@ 2010-05-12 13:39 ` Srikar Dronamraju
2010-05-12 14:53 ` Frederic Weisbecker
0 siblings, 1 reply; 28+ messages in thread
From: Srikar Dronamraju @ 2010-05-12 13:39 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Peter Zijlstra, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frank Ch. Eigler,
Andrew Morton, Paul E. McKenney
* Frederic Weisbecker <fweisbec@gmail.com> [2010-05-12 12:38:35]:
> On Thu, May 06, 2010 at 11:31:39PM +0530, Srikar Dronamraju wrote:
> > intro.patch
> >
> > From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> >
> > Uprobes Patches
>
>
>
> I can't find the trace event patch inside this set.
> May be you missed it somehow? (or I did).
>
> Thanks.
>
http://lkml.org/lkml/2010/5/6/284
However there were no changes from the patchset in version 2.
So your comments hold good for this version too.
--
Thanks and Regards
Srikar
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 0/10] Uprobes v3
2010-05-12 13:39 ` Srikar Dronamraju
@ 2010-05-12 14:53 ` Frederic Weisbecker
0 siblings, 0 replies; 28+ messages in thread
From: Frederic Weisbecker @ 2010-05-12 14:53 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Peter Zijlstra, Ingo Molnar, Masami Hiramatsu, Mel Gorman,
Randy Dunlap, Linus Torvalds, Roland McGrath, H. Peter Anvin,
Ananth N Mavinakayanahalli, Oleg Nesterov, Mark Wielaard,
Mathieu Desnoyers, LKML, Jim Keniston, Frank Ch. Eigler,
Andrew Morton, Paul E. McKenney
On Wed, May 12, 2010 at 07:09:52PM +0530, Srikar Dronamraju wrote:
> * Frederic Weisbecker <fweisbec@gmail.com> [2010-05-12 12:38:35]:
>
> > On Thu, May 06, 2010 at 11:31:39PM +0530, Srikar Dronamraju wrote:
> > > intro.patch
> > >
> > > From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> > >
> > > Uprobes Patches
> >
> >
> >
> > I can't find the trace event patch inside this set.
> > May be you missed it somehow? (or I did).
> >
> > Thanks.
> >
>
> http://lkml.org/lkml/2010/5/6/284
I missed it somehow.
Nevermind.
Thanks.
^ permalink raw reply [flat|nested] 28+ messages in thread