* [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints
@ 2025-07-22 17:38 Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 1/6] riscv: Add insn.c, consolidate instruction decoding Jesse Taube
` (5 more replies)
0 siblings, 6 replies; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
This patchset adds initial support for hardware breakpoints and
watchpoints to the RISC-V architecture. The framework is built on
top of perf subsystem and SBI debug trigger extension.
Currently following features are not supported and are in works:
- Ptrace regset support
- Virtualization of debug triggers
The SBI debug trigger extension can be found at:
https://github.com/riscv-non-isa/riscv-sbi-doc/blob/master/src/ext-debug-triggers.adoc
The Sdtrig ISA is part of RISC-V debug specification which can be
found at:
https://github.com/riscv/riscv-debug-spec
based off the original RFC by Himanshu Chauhan here:
https://lore.kernel.org/lkml/20240222125059.13331-1-hchauhan@ventanamicro.com/
Himanshu Chauhan (2):
riscv: Add SBI debug trigger extension and function ids
riscv: Introduce support for hardware break/watchpoints
Jesse Taube (4):
riscv: Add insn.c, consolidate instruction decoding
riscv: insn: __read_insn use copy_from_X_nofault
riscv: hw_breakpoint: Use icount for single stepping
riscv: ptrace: Add hw breakpoint support
arch/riscv/Kconfig | 12 +
arch/riscv/include/asm/bug.h | 12 -
arch/riscv/include/asm/hw_breakpoint.h | 60 +++
arch/riscv/include/asm/insn.h | 131 ++++-
arch/riscv/include/asm/kdebug.h | 3 +-
arch/riscv/include/asm/processor.h | 4 +
arch/riscv/include/asm/sbi.h | 33 +-
arch/riscv/include/uapi/asm/ptrace.h | 3 +-
arch/riscv/kernel/Makefile | 2 +
arch/riscv/kernel/hw_breakpoint.c | 681 +++++++++++++++++++++++++
arch/riscv/kernel/insn.c | 153 ++++++
arch/riscv/kernel/kgdb.c | 102 +---
arch/riscv/kernel/probes/kprobes.c | 1 +
arch/riscv/kernel/process.c | 4 +
arch/riscv/kernel/ptrace.c | 93 ++++
arch/riscv/kernel/traps.c | 11 +-
arch/riscv/kernel/traps_misaligned.c | 93 +---
17 files changed, 1208 insertions(+), 190 deletions(-)
create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
create mode 100644 arch/riscv/kernel/hw_breakpoint.c
create mode 100644 arch/riscv/kernel/insn.c
--
2.43.0
^ permalink raw reply [flat|nested] 18+ messages in thread
* [RFC PATCH 1/6] riscv: Add insn.c, consolidate instruction decoding
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 2/6] riscv: Add SBI debug trigger extension and function ids Jesse Taube
` (4 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
Various parts of the kernel decode and read instruction from memory.
Functions like get_insn, GET_INSN_LENGTH and riscv_insn_is_c are defined
in multiple places. Consolidate these functions into the insn.h and the
newly added insn.c.
Signed-off-by: Jesse Taube <jesse@rivosinc.com>
---
arch/riscv/include/asm/bug.h | 12 ---
arch/riscv/include/asm/insn.h | 131 ++++++++++++++++++++++-
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/insn.c | 151 +++++++++++++++++++++++++++
arch/riscv/kernel/kgdb.c | 102 +-----------------
arch/riscv/kernel/probes/kprobes.c | 1 +
arch/riscv/kernel/traps.c | 5 +-
arch/riscv/kernel/traps_misaligned.c | 93 ++++-------------
8 files changed, 309 insertions(+), 187 deletions(-)
create mode 100644 arch/riscv/kernel/insn.c
diff --git a/arch/riscv/include/asm/bug.h b/arch/riscv/include/asm/bug.h
index 1aaea81fb141..a2777eb67ad1 100644
--- a/arch/riscv/include/asm/bug.h
+++ b/arch/riscv/include/asm/bug.h
@@ -12,21 +12,9 @@
#include <asm/asm.h>
-#define __INSN_LENGTH_MASK _UL(0x3)
-#define __INSN_LENGTH_32 _UL(0x3)
-#define __COMPRESSED_INSN_MASK _UL(0xffff)
-
#define __BUG_INSN_32 _UL(0x00100073) /* ebreak */
#define __BUG_INSN_16 _UL(0x9002) /* c.ebreak */
-#define GET_INSN_LENGTH(insn) \
-({ \
- unsigned long __len; \
- __len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? \
- 4UL : 2UL; \
- __len; \
-})
-
typedef u32 bug_insn_t;
#ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS
diff --git a/arch/riscv/include/asm/insn.h b/arch/riscv/include/asm/insn.h
index 09fde95a5e8f..ba74e5b8262c 100644
--- a/arch/riscv/include/asm/insn.h
+++ b/arch/riscv/include/asm/insn.h
@@ -64,6 +64,7 @@
#define RVG_RS2_OPOFF 20
#define RVG_RD_OPOFF 7
#define RVG_RS1_MASK GENMASK(4, 0)
+#define RVG_RS2_MASK GENMASK(4, 0)
#define RVG_RD_MASK GENMASK(4, 0)
/* The bit field of immediate value in RVC J instruction */
@@ -121,17 +122,27 @@
#define RVC_C0_RS1_OPOFF 7
#define RVC_C0_RS2_OPOFF 2
#define RVC_C0_RD_OPOFF 2
+#define RVC_C0_RS1_MASK GENMASK(2, 0)
+#define RVC_C0_RS2_MASK GENMASK(2, 0)
+#define RVC_C0_RD_MASK GENMASK(2, 0)
+#define RVC_C0_REG_OFFSET 8
/* The register offset in RVC op=C1 instruction */
#define RVC_C1_RS1_OPOFF 7
#define RVC_C1_RS2_OPOFF 2
#define RVC_C1_RD_OPOFF 7
+#define RVC_C1_RS1_MASK GENMASK(2, 0)
+#define RVC_C1_RS2_MASK GENMASK(2, 0)
+#define RVC_C1_RD_MASK GENMASK(2, 0)
+#define RVC_C1_REG_OFFSET 8
/* The register offset in RVC op=C2 instruction */
#define RVC_C2_RS1_OPOFF 7
#define RVC_C2_RS2_OPOFF 2
#define RVC_C2_RD_OPOFF 7
#define RVC_C2_RS1_MASK GENMASK(4, 0)
+#define RVC_C2_RS2_MASK GENMASK(4, 0)
+#define RVC_C2_RD_MASK GENMASK(4, 0)
/* parts of opcode for RVG*/
#define RVG_OPCODE_FENCE 0x0f
@@ -226,12 +237,26 @@
#define RVC_MASK_C_EBREAK 0xffff
#define RVG_MASK_EBREAK 0xffffffff
#define RVG_MASK_SRET 0xffffffff
+#define RVC_MASK_C GENMASK(15, 0)
#define __INSN_LENGTH_MASK _UL(0x3)
#define __INSN_LENGTH_GE_32 _UL(0x3)
#define __INSN_OPCODE_MASK _UL(0x7F)
#define __INSN_BRANCH_OPCODE _UL(RVG_OPCODE_BRANCH)
+#define GET_INSN_LENGTH(insn) \
+({ \
+ unsigned long __len; \
+ __len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_GE_32) ? \
+ 4UL : 2UL; \
+ __len; \
+})
+
+static __always_inline bool riscv_insn_is_c(u32 code)
+{
+ return (code & (__INSN_LENGTH_MASK)) != (__INSN_LENGTH_GE_32);
+}
+
#define __RISCV_INSN_FUNCS(name, mask, val) \
static __always_inline bool riscv_insn_is_##name(u32 code) \
{ \
@@ -260,7 +285,7 @@ __RISCV_INSN_FUNCS(c_bnez, RVC_MASK_C_BNEZ, RVC_MATCH_C_BNEZ)
__RISCV_INSN_FUNCS(c_ebreak, RVC_MASK_C_EBREAK, RVC_MATCH_C_EBREAK)
__RISCV_INSN_FUNCS(ebreak, RVG_MASK_EBREAK, RVG_MATCH_EBREAK)
__RISCV_INSN_FUNCS(sret, RVG_MASK_SRET, RVG_MATCH_SRET)
-__RISCV_INSN_FUNCS(fence, RVG_MASK_FENCE, RVG_MATCH_FENCE);
+__RISCV_INSN_FUNCS(fence, RVG_MASK_FENCE, RVG_MATCH_FENCE)
/* special case to catch _any_ system instruction */
static __always_inline bool riscv_insn_is_system(u32 code)
@@ -295,6 +320,10 @@ static __always_inline bool riscv_insn_is_c_jalr(u32 code)
({typeof(x) x_ = (x); \
(RV_X(x_, RVG_RS1_OPOFF, RVG_RS1_MASK)); })
+#define RV_EXTRACT_RS2_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RV_X(x_, RVG_RS2_OPOFF, RVG_RS2_MASK)); })
+
#define RV_EXTRACT_RD_REG(x) \
({typeof(x) x_ = (x); \
(RV_X(x_, RVG_RD_OPOFF, RVG_RD_MASK)); })
@@ -322,9 +351,41 @@ static __always_inline bool riscv_insn_is_c_jalr(u32 code)
(RV_X(x_, RV_B_IMM_11_OPOFF, RV_B_IMM_11_MASK) << RV_B_IMM_11_OFF) | \
(RV_IMM_SIGN(x_) << RV_B_IMM_SIGN_OFF); })
+#define RVC_EXTRACT_C0_RS1_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C0_RS1_OPOFF, RVC_C0_RS1_MASK)); })
+
+#define RVC_EXTRACT_C0_RS2_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C0_RS2_OPOFF, RVC_C0_RS2_MASK)); })
+
+#define RVC_EXTRACT_C0_RD_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C0_RD_OPOFF, RVC_C0_RD_MASK)); })
+
+#define RVC_EXTRACT_C1_RS1_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C1_RS1_OPOFF, RVC_C1_RS1_MASK)); })
+
+#define RVC_EXTRACT_C1_RS2_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C1_RS2_OPOFF, RVC_C1_RS2_MASK)); })
+
+#define RVC_EXTRACT_C1_RD_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C1_RD_OPOFF, RVC_C1_RD_MASK)); })
+
#define RVC_EXTRACT_C2_RS1_REG(x) \
({typeof(x) x_ = (x); \
- (RV_X(x_, RVC_C2_RS1_OPOFF, RVC_C2_RS1_MASK)); })
+ (RVC_X(x_, RVC_C2_RS1_OPOFF, RVC_C2_RS1_MASK)); })
+
+#define RVC_EXTRACT_C2_RS2_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C2_RS2_OPOFF, RVC_C2_RS2_MASK)); })
+
+#define RVC_EXTRACT_C2_RD_REG(x) \
+ ({typeof(x) x_ = (x); \
+ (RVC_X(x_, RVC_C2_RD_OPOFF, RVC_C2_RD_MASK)); })
#define RVC_EXTRACT_JTYPE_IMM(x) \
({typeof(x) x_ = (x); \
@@ -354,6 +415,66 @@ static __always_inline bool riscv_insn_is_c_jalr(u32 code)
#define RVV_EXRACT_VL_VS_WIDTH(x) RVFDQ_EXTRACT_FL_FS_WIDTH(x)
+/*
+ * Get the rs1 register number from RV or RVC instruction.
+ *
+ * @insn: instruction to process
+ * Return: rs1 register
+ */
+static inline unsigned int riscv_insn_extract_rs1_reg(u32 insn)
+{
+ switch (RVC_INSN_OPCODE_MASK & insn) {
+ case RVC_OPCODE_C0:
+ return RVC_EXTRACT_C0_RS1_REG(insn) + RVC_C0_REG_OFFSET;
+ case RVC_OPCODE_C1:
+ return RVC_EXTRACT_C1_RS1_REG(insn) + RVC_C1_REG_OFFSET;
+ case RVC_OPCODE_C2:
+ return RVC_EXTRACT_C2_RS1_REG(insn);
+ default:
+ return RV_EXTRACT_RS1_REG(insn);
+ }
+}
+
+/*
+ * Get the rs2 register number from RV or RVC instruction.
+ *
+ * @insn: instruction to process
+ * Return: rs2 register
+ */
+static inline unsigned int riscv_insn_extract_rs2_reg(u32 insn)
+{
+ switch (RVC_INSN_OPCODE_MASK & insn) {
+ case RVC_OPCODE_C0:
+ return RVC_EXTRACT_C0_RS2_REG(insn) + RVC_C0_REG_OFFSET;
+ case RVC_OPCODE_C1:
+ return RVC_EXTRACT_C1_RS2_REG(insn) + RVC_C1_REG_OFFSET;
+ case RVC_OPCODE_C2:
+ return RVC_EXTRACT_C2_RS2_REG(insn);
+ default:
+ return RV_EXTRACT_RS2_REG(insn);
+ }
+}
+
+/*
+ * Get the rd register number from RV or RVC instruction.
+ *
+ * @insn: instruction to process
+ * Return: rd register
+ */
+static inline unsigned int riscv_insn_extract_rd_reg(u32 insn)
+{
+ switch (RVC_INSN_OPCODE_MASK & insn) {
+ case RVC_OPCODE_C0:
+ return RVC_EXTRACT_C0_RD_REG(insn) + RVC_C0_REG_OFFSET;
+ case RVC_OPCODE_C1:
+ return RVC_EXTRACT_C1_RD_REG(insn) + RVC_C1_REG_OFFSET;
+ case RVC_OPCODE_C2:
+ return RVC_EXTRACT_C2_RD_REG(insn);
+ default:
+ return RV_EXTRACT_RD_REG(insn);
+ }
+}
+
/*
* Get the immediate from a J-type instruction.
*
@@ -428,4 +549,10 @@ static inline void riscv_insn_insert_utype_itype_imm(u32 *utype_insn, u32 *itype
*utype_insn |= (imm & RV_U_IMM_31_12_MASK) + ((imm & BIT(11)) << 1);
*itype_insn |= ((imm & RV_I_IMM_11_0_MASK) << RV_I_IMM_11_0_OPOFF);
}
+
+#include <asm/ptrace.h>
+
+int get_insn(struct pt_regs *regs, ulong epc, ulong *r_insn);
+unsigned long get_step_address(struct pt_regs *regs, u32 code);
+
#endif /* _ASM_RISCV_INSN_H */
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index f7480c9c6f8d..4f719b09e5ad 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -51,6 +51,7 @@ obj-$(CONFIG_RISCV_ALTERNATIVE) += alternative.o
obj-y += cpu.o
obj-y += cpufeature.o
obj-y += entry.o
+obj-y += insn.o
obj-y += irq.o
obj-y += process.o
obj-y += ptrace.o
diff --git a/arch/riscv/kernel/insn.c b/arch/riscv/kernel/insn.c
new file mode 100644
index 000000000000..dd2a6ef9fd25
--- /dev/null
+++ b/arch/riscv/kernel/insn.c
@@ -0,0 +1,151 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2025 Rivos, Inc
+ */
+#include <asm/insn.h>
+#include <asm/ptrace.h>
+#include <asm/uaccess.h>
+
+#define __read_insn(regs, insn, insn_addr, type) \
+({ \
+ int __ret; \
+ \
+ if (user_mode(regs)) { \
+ __ret = get_user(insn, (type __user *) insn_addr); \
+ } else { \
+ insn = *(type *)insn_addr; \
+ __ret = 0; \
+ } \
+ \
+ __ret; \
+})
+
+/*
+ * Update a set of two instructions (U-type + I-type) with an immediate value.
+ *
+ * Used for example in auipc+jalrs pairs the U-type instructions contains
+ * a 20bit upper immediate representing bits[31:12], while the I-type
+ * instruction contains a 12bit immediate representing bits[11:0].
+ *
+ * This also takes into account that both separate immediates are
+ * considered as signed values, so if the I-type immediate becomes
+ * negative (BIT(11) set) the U-type part gets adjusted.
+ *
+ * @regs: pointer to the utype instruction of the pair
+ * @epc: pointer to the itype instruction of the pair
+ * @r_insn: the immediate to insert into the two instructions
+ * Return: combined immediate
+ */
+int get_insn(struct pt_regs *regs, ulong epc, ulong *r_insn)
+{
+ ulong insn = 0;
+
+ if (epc & 0x2) {
+ ulong tmp = 0;
+
+ if (__read_insn(regs, insn, epc, u16))
+ return -EFAULT;
+ /* __get_user() uses regular "lw" which sign extend the loaded
+ * value make sure to clear higher order bits in case we "or" it
+ * below with the upper 16 bits half.
+ */
+ insn &= RVC_MASK_C;
+ if (riscv_insn_is_c(insn)) {
+ *r_insn = insn;
+ return 0;
+ }
+ epc += sizeof(u16);
+ if (__read_insn(regs, tmp, epc, u16))
+ return -EFAULT;
+ *r_insn = (tmp << 16) | insn;
+
+ return 0;
+ } else {
+ if (__read_insn(regs, insn, epc, u32))
+ return -EFAULT;
+ if (!riscv_insn_is_c(insn)) {
+ *r_insn = insn;
+ return 0;
+ }
+ insn &= RVC_MASK_C;
+ *r_insn = insn;
+
+ return 0;
+ }
+}
+
+/* Calculate the new address for after a step */
+unsigned long get_step_address(struct pt_regs *regs, u32 code)
+{
+ unsigned long pc = regs->epc;
+ unsigned int rs1_num, rs2_num;
+
+ if ((code & __INSN_LENGTH_MASK) != __INSN_LENGTH_GE_32) {
+ if (riscv_insn_is_c_jalr(code) ||
+ riscv_insn_is_c_jr(code)) {
+ rs1_num = riscv_insn_extract_rs1_reg(code);
+ return regs_get_register(regs, rs1_num);
+ } else if (riscv_insn_is_c_j(code) ||
+ riscv_insn_is_c_jal(code)) {
+ return RVC_EXTRACT_JTYPE_IMM(code) + pc;
+ } else if (riscv_insn_is_c_beqz(code)) {
+ rs1_num = riscv_insn_extract_rs1_reg(code);
+ if (!rs1_num || regs_get_register(regs, rs1_num) == 0)
+ return RVC_EXTRACT_BTYPE_IMM(code) + pc;
+ else
+ return pc + 2;
+ } else if (riscv_insn_is_c_bnez(code)) {
+ rs1_num = riscv_insn_extract_rs1_reg(RVC_C1_RS1_OPOFF);
+ if (rs1_num && regs_get_register(regs, rs1_num) != 0)
+ return RVC_EXTRACT_BTYPE_IMM(code) + pc;
+ else
+ return pc + 2;
+ } else {
+ return pc + 2;
+ }
+ } else {
+ if ((code & __INSN_OPCODE_MASK) == __INSN_BRANCH_OPCODE) {
+ bool result = false;
+ long imm = RV_EXTRACT_BTYPE_IMM(code);
+ unsigned long rs1_val = 0, rs2_val = 0;
+
+ rs1_num = riscv_insn_extract_rs1_reg(code);
+ rs2_num = riscv_insn_extract_rs2_reg(code);
+ if (rs1_num)
+ rs1_val = regs_get_register(regs, rs1_num);
+ if (rs2_num)
+ rs2_val = regs_get_register(regs, rs2_num);
+
+ if (riscv_insn_is_beq(code))
+ result = (rs1_val == rs2_val) ? true : false;
+ else if (riscv_insn_is_bne(code))
+ result = (rs1_val != rs2_val) ? true : false;
+ else if (riscv_insn_is_blt(code))
+ result =
+ ((long)rs1_val <
+ (long)rs2_val) ? true : false;
+ else if (riscv_insn_is_bge(code))
+ result =
+ ((long)rs1_val >=
+ (long)rs2_val) ? true : false;
+ else if (riscv_insn_is_bltu(code))
+ result = (rs1_val < rs2_val) ? true : false;
+ else if (riscv_insn_is_bgeu(code))
+ result = (rs1_val >= rs2_val) ? true : false;
+ if (result)
+ return imm + pc;
+ else
+ return pc + 4;
+ } else if (riscv_insn_is_jal(code)) {
+ return RV_EXTRACT_JTYPE_IMM(code) + pc;
+ } else if (riscv_insn_is_jalr(code)) {
+ rs1_num = riscv_insn_extract_rs1_reg(code);
+ return RV_EXTRACT_ITYPE_IMM(code) +
+ (rs1_num ? regs_get_register(regs, rs1_num) : 0);
+ } else if (riscv_insn_is_sret(code)) {
+ return pc;
+ } else {
+ return pc + 4;
+ }
+ }
+}
diff --git a/arch/riscv/kernel/kgdb.c b/arch/riscv/kernel/kgdb.c
index 9f3db3503dab..aafc1424fc81 100644
--- a/arch/riscv/kernel/kgdb.c
+++ b/arch/riscv/kernel/kgdb.c
@@ -23,111 +23,19 @@ enum {
static unsigned long stepped_address;
static unsigned int stepped_opcode;
-static int decode_register_index(unsigned long opcode, int offset)
-{
- return (opcode >> offset) & 0x1F;
-}
-
-static int decode_register_index_short(unsigned long opcode, int offset)
-{
- return ((opcode >> offset) & 0x7) + 8;
-}
-
-/* Calculate the new address for after a step */
-static int get_step_address(struct pt_regs *regs, unsigned long *next_addr)
-{
- unsigned long pc = regs->epc;
- unsigned long *regs_ptr = (unsigned long *)regs;
- unsigned int rs1_num, rs2_num;
- int op_code;
-
- if (get_kernel_nofault(op_code, (void *)pc))
- return -EINVAL;
- if ((op_code & __INSN_LENGTH_MASK) != __INSN_LENGTH_GE_32) {
- if (riscv_insn_is_c_jalr(op_code) ||
- riscv_insn_is_c_jr(op_code)) {
- rs1_num = decode_register_index(op_code, RVC_C2_RS1_OPOFF);
- *next_addr = regs_ptr[rs1_num];
- } else if (riscv_insn_is_c_j(op_code) ||
- riscv_insn_is_c_jal(op_code)) {
- *next_addr = RVC_EXTRACT_JTYPE_IMM(op_code) + pc;
- } else if (riscv_insn_is_c_beqz(op_code)) {
- rs1_num = decode_register_index_short(op_code,
- RVC_C1_RS1_OPOFF);
- if (!rs1_num || regs_ptr[rs1_num] == 0)
- *next_addr = RVC_EXTRACT_BTYPE_IMM(op_code) + pc;
- else
- *next_addr = pc + 2;
- } else if (riscv_insn_is_c_bnez(op_code)) {
- rs1_num =
- decode_register_index_short(op_code, RVC_C1_RS1_OPOFF);
- if (rs1_num && regs_ptr[rs1_num] != 0)
- *next_addr = RVC_EXTRACT_BTYPE_IMM(op_code) + pc;
- else
- *next_addr = pc + 2;
- } else {
- *next_addr = pc + 2;
- }
- } else {
- if ((op_code & __INSN_OPCODE_MASK) == __INSN_BRANCH_OPCODE) {
- bool result = false;
- long imm = RV_EXTRACT_BTYPE_IMM(op_code);
- unsigned long rs1_val = 0, rs2_val = 0;
-
- rs1_num = decode_register_index(op_code, RVG_RS1_OPOFF);
- rs2_num = decode_register_index(op_code, RVG_RS2_OPOFF);
- if (rs1_num)
- rs1_val = regs_ptr[rs1_num];
- if (rs2_num)
- rs2_val = regs_ptr[rs2_num];
-
- if (riscv_insn_is_beq(op_code))
- result = (rs1_val == rs2_val) ? true : false;
- else if (riscv_insn_is_bne(op_code))
- result = (rs1_val != rs2_val) ? true : false;
- else if (riscv_insn_is_blt(op_code))
- result =
- ((long)rs1_val <
- (long)rs2_val) ? true : false;
- else if (riscv_insn_is_bge(op_code))
- result =
- ((long)rs1_val >=
- (long)rs2_val) ? true : false;
- else if (riscv_insn_is_bltu(op_code))
- result = (rs1_val < rs2_val) ? true : false;
- else if (riscv_insn_is_bgeu(op_code))
- result = (rs1_val >= rs2_val) ? true : false;
- if (result)
- *next_addr = imm + pc;
- else
- *next_addr = pc + 4;
- } else if (riscv_insn_is_jal(op_code)) {
- *next_addr = RV_EXTRACT_JTYPE_IMM(op_code) + pc;
- } else if (riscv_insn_is_jalr(op_code)) {
- rs1_num = decode_register_index(op_code, RVG_RS1_OPOFF);
- if (rs1_num)
- *next_addr = ((unsigned long *)regs)[rs1_num];
- *next_addr += RV_EXTRACT_ITYPE_IMM(op_code);
- } else if (riscv_insn_is_sret(op_code)) {
- *next_addr = pc;
- } else {
- *next_addr = pc + 4;
- }
- }
- return 0;
-}
-
static int do_single_step(struct pt_regs *regs)
{
/* Determine where the target instruction will send us to */
- unsigned long addr = 0;
- int error = get_step_address(regs, &addr);
+ unsigned long addr, insn;
+ int error = get_insn(regs, regs->epc, &insn);
if (error)
return error;
+ addr = get_step_address(regs, insn);
+
/* Store the op code in the stepped address */
- error = get_kernel_nofault(stepped_opcode, (void *)addr);
+ error = get_insn(regs, addr, stepped_opcode);
if (error)
return error;
diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c
index c0738d6c6498..6a9cfb0b664a 100644
--- a/arch/riscv/kernel/probes/kprobes.c
+++ b/arch/riscv/kernel/probes/kprobes.c
@@ -12,6 +12,7 @@
#include <asm/sections.h>
#include <asm/cacheflush.h>
#include <asm/bug.h>
+#include <asm/insn.h>
#include <asm/text-patching.h>
#include "decode-insn.h"
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 9c83848797a7..938a8b841f94 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -26,6 +26,7 @@
#include <asm/bug.h>
#include <asm/cfi.h>
#include <asm/csr.h>
+#include <asm/insn.h>
#include <asm/processor.h>
#include <asm/ptrace.h>
#include <asm/syscall.h>
@@ -409,10 +410,10 @@ int is_valid_bugaddr(unsigned long pc)
return 0;
if (get_kernel_nofault(insn, (bug_insn_t *)pc))
return 0;
- if ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32)
+ if ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_GE_32)
return (insn == __BUG_INSN_32);
else
- return ((insn & __COMPRESSED_INSN_MASK) == __BUG_INSN_16);
+ return ((insn & RVC_MASK_C) == __BUG_INSN_16);
}
#endif /* CONFIG_GENERIC_BUG */
diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c
index 77c788660223..42a50e21b1d2 100644
--- a/arch/riscv/kernel/traps_misaligned.c
+++ b/arch/riscv/kernel/traps_misaligned.c
@@ -10,12 +10,13 @@
#include <linux/irq.h>
#include <linux/stringify.h>
-#include <asm/processor.h>
-#include <asm/ptrace.h>
+#include <asm/cpufeature.h>
#include <asm/csr.h>
#include <asm/entry-common.h>
#include <asm/hwprobe.h>
-#include <asm/cpufeature.h>
+#include <asm/insn.h>
+#include <asm/processor.h>
+#include <asm/ptrace.h>
#include <asm/vector.h>
#define INSN_MATCH_LB 0x3
@@ -112,25 +113,22 @@
#define SH_RS2 20
#define SH_RS2C 2
-#define RV_X(x, s, n) (((x) >> (s)) & ((1 << (n)) - 1))
-#define RVC_LW_IMM(x) ((RV_X(x, 6, 1) << 2) | \
- (RV_X(x, 10, 3) << 3) | \
- (RV_X(x, 5, 1) << 6))
-#define RVC_LD_IMM(x) ((RV_X(x, 10, 3) << 3) | \
- (RV_X(x, 5, 2) << 6))
-#define RVC_LWSP_IMM(x) ((RV_X(x, 4, 3) << 2) | \
+#define RVC_LW_IMM(x) ((RV_X(x, 6, 0x1) << 2) | \
+ (RV_X(x, 10, 0x7) << 3) | \
+ (RV_X(x, 5, 0x1) << 6))
+#define RVC_LD_IMM(x) ((RV_X(x, 10, 0x7) << 3) | \
+ (RV_X(x, 5, 0x3) << 6))
+#define RVC_LWSP_IMM(x) ((RV_X(x, 4, 0x7) << 2) | \
+ (RV_X(x, 12, 0x1) << 5) | \
+ (RV_X(x, 2, 0x3) << 6))
+#define RVC_LDSP_IMM(x) ((RV_X(x, 5, 0x3) << 3) | \
(RV_X(x, 12, 1) << 5) | \
- (RV_X(x, 2, 2) << 6))
-#define RVC_LDSP_IMM(x) ((RV_X(x, 5, 2) << 3) | \
- (RV_X(x, 12, 1) << 5) | \
- (RV_X(x, 2, 3) << 6))
-#define RVC_SWSP_IMM(x) ((RV_X(x, 9, 4) << 2) | \
- (RV_X(x, 7, 2) << 6))
-#define RVC_SDSP_IMM(x) ((RV_X(x, 10, 3) << 3) | \
- (RV_X(x, 7, 3) << 6))
-#define RVC_RS1S(insn) (8 + RV_X(insn, SH_RD, 3))
-#define RVC_RS2S(insn) (8 + RV_X(insn, SH_RS2C, 3))
-#define RVC_RS2(insn) RV_X(insn, SH_RS2C, 5)
+ (RV_X(x, 2, 0x7) << 6))
+#define RVC_SWSP_IMM(x) ((RV_X(x, 9, 0xf) << 2) | \
+ (RV_X(x, 7, 0x3) << 6))
+#define RVC_SDSP_IMM(x) ((RV_X(x, 10, 0x7) << 3) | \
+ (RV_X(x, 7, 0x7) << 6))
+#define RVC_RS2S(insn) (8 + RV_X(insn, SH_RS2C, 0x7))
#define SHIFT_RIGHT(x, y) \
((y) < 0 ? ((x) << -(y)) : ((x) >> (y)))
@@ -146,7 +144,6 @@
#define GET_RS1(insn, regs) (*REG_PTR(insn, SH_RS1, regs))
#define GET_RS2(insn, regs) (*REG_PTR(insn, SH_RS2, regs))
-#define GET_RS1S(insn, regs) (*REG_PTR(RVC_RS1S(insn), 0, regs))
#define GET_RS2S(insn, regs) (*REG_PTR(RVC_RS2S(insn), 0, regs))
#define GET_RS2C(insn, regs) (*REG_PTR(insn, SH_RS2C, regs))
#define GET_SP(regs) (*REG_PTR(2, 0, regs))
@@ -270,58 +267,6 @@ static unsigned long get_f32_rs(unsigned long insn, u8 fp_reg_offset,
#define GET_F32_RS2C(insn, regs) (get_f32_rs(insn, 2, regs))
#define GET_F32_RS2S(insn, regs) (get_f32_rs(RVC_RS2S(insn), 0, regs))
-#define __read_insn(regs, insn, insn_addr, type) \
-({ \
- int __ret; \
- \
- if (user_mode(regs)) { \
- __ret = get_user(insn, (type __user *) insn_addr); \
- } else { \
- insn = *(type *)insn_addr; \
- __ret = 0; \
- } \
- \
- __ret; \
-})
-
-static inline int get_insn(struct pt_regs *regs, ulong epc, ulong *r_insn)
-{
- ulong insn = 0;
-
- if (epc & 0x2) {
- ulong tmp = 0;
-
- if (__read_insn(regs, insn, epc, u16))
- return -EFAULT;
- /* __get_user() uses regular "lw" which sign extend the loaded
- * value make sure to clear higher order bits in case we "or" it
- * below with the upper 16 bits half.
- */
- insn &= GENMASK(15, 0);
- if ((insn & __INSN_LENGTH_MASK) != __INSN_LENGTH_32) {
- *r_insn = insn;
- return 0;
- }
- epc += sizeof(u16);
- if (__read_insn(regs, tmp, epc, u16))
- return -EFAULT;
- *r_insn = (tmp << 16) | insn;
-
- return 0;
- } else {
- if (__read_insn(regs, insn, epc, u32))
- return -EFAULT;
- if ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) {
- *r_insn = insn;
- return 0;
- }
- insn &= GENMASK(15, 0);
- *r_insn = insn;
-
- return 0;
- }
-}
-
union reg_data {
u8 data_bytes[8];
ulong data_ulong;
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH 2/6] riscv: Add SBI debug trigger extension and function ids
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 1/6] riscv: Add insn.c, consolidate instruction decoding Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault Jesse Taube
` (3 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
From: Himanshu Chauhan <hchauhan@ventanamicro.com>
Debug trigger extension is a proposed SBI extension to support
native debugging in S-mode and VS-mode.
The proposal for the extension can be found at:
https://lists.riscv.org/g/sig-hypervisors/message/361
This patch adds the extension and the function IDs defined
by the extension.
Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
---
arch/riscv/include/asm/sbi.h | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 3d250824178b..be2ca8e8a49e 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -35,6 +35,7 @@ enum sbi_ext_id {
SBI_EXT_DBCN = 0x4442434E,
SBI_EXT_STA = 0x535441,
SBI_EXT_NACL = 0x4E41434C,
+ SBI_EXT_DBTR = 0x44425452,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -402,6 +403,34 @@ enum sbi_ext_nacl_feature {
#define SBI_NACL_SHMEM_SRET_X(__i) ((__riscv_xlen / 8) * (__i))
#define SBI_NACL_SHMEM_SRET_X_LAST 31
+/* SBI debug triggers function IDs */
+enum sbi_ext_dbtr_fid {
+ SBI_EXT_DBTR_NUM_TRIGGERS = 0,
+ SBI_EXT_DBTR_SETUP_SHMEM,
+ SBI_EXT_DBTR_TRIG_READ,
+ SBI_EXT_DBTR_TRIG_INSTALL,
+ SBI_EXT_DBTR_TRIG_UPDATE,
+ SBI_EXT_DBTR_TRIG_UNINSTALL,
+ SBI_EXT_DBTR_TRIG_ENABLE,
+ SBI_EXT_DBTR_TRIG_DISABLE,
+};
+
+struct sbi_dbtr_data_msg {
+ unsigned long tstate;
+ unsigned long tdata1;
+ unsigned long tdata2;
+ unsigned long tdata3;
+};
+
+struct sbi_dbtr_id_msg {
+ unsigned long idx;
+};
+
+union sbi_dbtr_shmem_entry {
+ struct sbi_dbtr_data_msg data;
+ struct sbi_dbtr_id_msg id;
+};
+
/* SBI spec version fields */
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 1/6] riscv: Add insn.c, consolidate instruction decoding Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 2/6] riscv: Add SBI debug trigger extension and function ids Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-08-04 7:44 ` Clément Léger
2025-07-22 17:38 ` [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints Jesse Taube
` (2 subsequent siblings)
5 siblings, 1 reply; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
__read_insn was using get_user for user space reads and direct
dereferencing for kernel space reads.
Update to use copy_from_user_nofault, copy_from_kernel_nofault for this
as get_user is user context only and may sleep.
Signed-off-by: Jesse Taube <jesse@rivosinc.com>
---
Squash with previous commit as it's breaks bisectability.
Separated as i'm unsure if copy_from_user_nofault is an acceptable
replacement.
---
arch/riscv/kernel/insn.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/kernel/insn.c b/arch/riscv/kernel/insn.c
index dd2a6ef9fd25..b8e5202ddced 100644
--- a/arch/riscv/kernel/insn.c
+++ b/arch/riscv/kernel/insn.c
@@ -2,6 +2,9 @@
/*
* Copyright 2025 Rivos, Inc
*/
+
+#include <linux/uaccess.h>
+
#include <asm/insn.h>
#include <asm/ptrace.h>
#include <asm/uaccess.h>
@@ -11,10 +14,9 @@
int __ret; \
\
if (user_mode(regs)) { \
- __ret = get_user(insn, (type __user *) insn_addr); \
+ __ret = copy_from_user_nofault(&insn, (const type __user *) insn_addr, sizeof(type)); \
} else { \
- insn = *(type *)insn_addr; \
- __ret = 0; \
+ __ret = copy_from_kernel_nofault(&insn, (const type *) insn_addr, sizeof(type)); \
} \
\
__ret; \
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
` (2 preceding siblings ...)
2025-07-22 17:38 ` [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-07-23 2:49 ` Deepak Gupta
2025-08-04 8:01 ` Clément Léger
2025-07-22 17:38 ` [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support Jesse Taube
5 siblings, 2 replies; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
From: Himanshu Chauhan <hchauhan@ventanamicro.com>
RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
SBI debug trigger extension to install/uninstall/update/enable/disable hardware
triggers as specified in Sdtrig ISA extension.
Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
Signed-off-by: Jesse Taube <jesse@rivosinc.com>
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/hw_breakpoint.h | 60 +++
arch/riscv/include/asm/kdebug.h | 3 +-
arch/riscv/include/asm/sbi.h | 4 +-
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
arch/riscv/kernel/traps.c | 6 +
7 files changed, 693 insertions(+), 2 deletions(-)
create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
create mode 100644 arch/riscv/kernel/hw_breakpoint.c
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index bbec87b79309..95d3047cab10 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -163,6 +163,7 @@ config RISCV
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_GCC_PLUGINS
select HAVE_GENERIC_VDSO if MMU && 64BIT
+ select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
new file mode 100644
index 000000000000..8efa3921c535
--- /dev/null
+++ b/arch/riscv/include/asm/hw_breakpoint.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024 Ventana Micro Systems Inc.
+ */
+
+#ifndef __RISCV_HW_BREAKPOINT_H
+#define __RISCV_HW_BREAKPOINT_H
+
+struct task_struct;
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+
+#include <uapi/linux/hw_breakpoint.h>
+
+#if __riscv_xlen == 64
+#define cpu_to_le cpu_to_le64
+#define le_to_cpu le64_to_cpu
+#elif __riscv_xlen == 32
+#define cpu_to_le cpu_to_le32
+#define le_to_cpu le32_to_cpu
+#else
+#error "Unexpected __riscv_xlen"
+#endif
+
+struct arch_hw_breakpoint {
+ unsigned long address;
+ unsigned long len;
+
+ /* Callback info */
+ unsigned long next_addr;
+ bool in_callback;
+
+
+ /* Trigger configuration data */
+ unsigned long tdata1;
+ unsigned long tdata2;
+ unsigned long tdata3;
+};
+
+/* Maximum number of hardware breakpoints supported */
+#define RV_MAX_TRIGGERS 32
+
+struct perf_event_attr;
+struct notifier_block;
+struct perf_event;
+struct pt_regs;
+
+int hw_breakpoint_slots(int type);
+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
+int hw_breakpoint_arch_parse(struct perf_event *bp,
+ const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw);
+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data);
+int arch_install_hw_breakpoint(struct perf_event *bp);
+void arch_uninstall_hw_breakpoint(struct perf_event *bp);
+void hw_breakpoint_pmu_read(struct perf_event *bp);
+
+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
+#endif /* __RISCV_HW_BREAKPOINT_H */
diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
index 85ac00411f6e..53e989781aa1 100644
--- a/arch/riscv/include/asm/kdebug.h
+++ b/arch/riscv/include/asm/kdebug.h
@@ -6,7 +6,8 @@
enum die_val {
DIE_UNUSED,
DIE_TRAP,
- DIE_OOPS
+ DIE_OOPS,
+ DIE_DEBUG
};
#endif
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index be2ca8e8a49e..64fa7a82aa45 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -282,7 +282,9 @@ struct sbi_sta_struct {
u8 pad[47];
} __packed;
-#define SBI_SHMEM_DISABLE -1
+#define SBI_SHMEM_DISABLE (-1UL)
+#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
+#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
enum sbi_ext_nacl_fid {
SBI_EXT_NACL_PROBE_FEATURE = 0x0,
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 4f719b09e5ad..3e72505734bd 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
+obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
ifeq ($(CONFIG_RISCV_SBI), y)
obj-$(CONFIG_SMP) += sbi-ipi.o
diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
new file mode 100644
index 000000000000..9e3a3b82d300
--- /dev/null
+++ b/arch/riscv/kernel/hw_breakpoint.c
@@ -0,0 +1,620 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Ventana Micro Systems Inc.
+ */
+
+#include <linux/hw_breakpoint.h>
+#include <linux/perf_event.h>
+#include <linux/spinlock.h>
+#include <linux/percpu.h>
+#include <linux/kdebug.h>
+#include <linux/bitops.h>
+#include <linux/bitfield.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+
+#include <asm/insn.h>
+#include <asm/sbi.h>
+
+#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
+#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
+
+#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
+#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
+
+#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
+#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
+#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
+#define DBTR_TDATA1_MCONTROL6_U BIT(3)
+#define DBTR_TDATA1_MCONTROL6_S BIT(4)
+#define DBTR_TDATA1_MCONTROL6_M BIT(6)
+#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
+#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
+#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
+#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
+
+#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
+#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
+#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
+#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
+
+#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
+#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
+#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
+#define DBTR_TDATA1_MCONTROL_U BIT(3)
+#define DBTR_TDATA1_MCONTROL_S BIT(4)
+#define DBTR_TDATA1_MCONTROL_M BIT(6)
+#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
+#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
+#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
+
+#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
+#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
+#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
+/* value of 5 split across HI and LO */
+#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
+#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
+
+/* Registered per-cpu bp/wp */
+static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
+static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
+static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
+
+/* Per-cpu shared memory between S and M mode */
+static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
+
+/* number of debug triggers on this cpu . */
+static int dbtr_total_num __ro_after_init;
+static unsigned long dbtr_type __ro_after_init;
+static unsigned long dbtr_init __ro_after_init;
+
+static int arch_smp_setup_sbi_shmem(unsigned int cpu)
+{
+ union sbi_dbtr_shmem_entry *dbtr_shmem;
+ unsigned long shmem_pa;
+ struct sbiret ret;
+ int rc;
+
+ dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
+ if (!dbtr_shmem) {
+ pr_err("Invalid per-cpu shared memory for debug triggers\n");
+ return -ENODEV;
+ }
+
+ shmem_pa = virt_to_phys(dbtr_shmem);
+
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
+ SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
+
+ if (ret.error) {
+ switch (ret.error) {
+ case SBI_ERR_DENIED:
+ pr_warn("%s: Access denied for shared memory at %lx\n",
+ __func__, shmem_pa);
+ rc = -EPERM;
+ break;
+
+ case SBI_ERR_INVALID_PARAM:
+ case SBI_ERR_INVALID_ADDRESS:
+ pr_warn("%s: Invalid address parameter (%ld)\n",
+ __func__, ret.error);
+ rc = -EINVAL;
+ break;
+
+ case SBI_ERR_ALREADY_AVAILABLE:
+ pr_warn("%s: Shared memory is already set\n",
+ __func__);
+ rc = -EADDRINUSE;
+ break;
+
+ case SBI_ERR_FAILURE:
+ pr_err("%s: Internal sdtrig state error\n",
+ __func__);
+ rc = -ENXIO;
+ break;
+
+ default:
+ pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
+ rc = -ENXIO;
+ break;
+ }
+ }
+
+ pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
+
+ return rc;
+}
+
+static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
+{
+ struct sbiret ret;
+
+ /* Disable shared memory */
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
+ SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
+
+ if (ret.error) {
+ switch (ret.error) {
+ case SBI_ERR_DENIED:
+ pr_err("%s: Access denied for shared memory.\n",
+ __func__);
+ break;
+
+ case SBI_ERR_INVALID_PARAM:
+ case SBI_ERR_INVALID_ADDRESS:
+ pr_err("%s: Invalid address parameter (%lu)\n",
+ __func__, ret.error);
+ break;
+
+ case SBI_ERR_ALREADY_AVAILABLE:
+ pr_err("%s: Shared memory is already set\n",
+ __func__);
+ break;
+ case SBI_ERR_FAILURE:
+ pr_err("%s: Internal sdtrig state error\n",
+ __func__);
+ break;
+ default:
+ pr_err("%s: Unknown error %lu\n", __func__, ret.error);
+ break;
+ }
+ }
+
+ pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
+
+ return 0;
+}
+
+static void init_sbi_dbtr(void)
+{
+ struct sbiret ret;
+
+ /*
+ * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
+ * Only proceed if this is the first CPU to reach this code.
+ */
+ if (test_and_set_bit(0, &dbtr_init))
+ return;
+
+ if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
+ pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
+ dbtr_total_num = 0;
+ return;
+ }
+
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
+ DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
+ if (ret.error) {
+ pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
+ __func__, ret.error);
+ } else if (!ret.value) {
+ pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
+ } else {
+ dbtr_total_num = ret.value;
+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
+ return;
+ }
+
+ /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
+ DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
+ if (ret.error) {
+ pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
+ __func__, ret.error);
+ } else if (!ret.value) {
+ pr_err("%s: No mcontrol triggers available.\n", __func__);
+ dbtr_total_num = 0;
+ } else {
+ dbtr_total_num = ret.value;
+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
+ }
+}
+
+int hw_breakpoint_slots(int type)
+{
+ /*
+ * We can be called early, so don't rely on
+ * static variables being initialised.
+ */
+ init_sbi_dbtr();
+
+ return dbtr_total_num;
+}
+
+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
+{
+ unsigned int len;
+ unsigned long va;
+
+ va = hw->address;
+ len = hw->len;
+
+ return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
+}
+
+static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw)
+{
+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
+
+ switch (attr->bp_type) {
+ case HW_BREAKPOINT_X:
+ tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
+ break;
+ case HW_BREAKPOINT_R:
+ tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
+ break;
+ case HW_BREAKPOINT_W:
+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
+ break;
+ case HW_BREAKPOINT_RW:
+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (attr->bp_len) {
+ case HW_BREAKPOINT_LEN_1:
+ hw->len = 1;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
+ DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
+ break;
+ case HW_BREAKPOINT_LEN_2:
+ hw->len = 2;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
+ DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
+ break;
+ case HW_BREAKPOINT_LEN_4:
+ hw->len = 4;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
+ DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
+ break;
+#if __riscv_xlen >= 64
+ case HW_BREAKPOINT_LEN_8:
+ hw->len = 8;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
+ DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
+ FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
+ DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
+ break;
+#endif
+ default:
+ return -EINVAL;
+ }
+
+ tdata1 |= DBTR_TDATA1_MCONTROL_U;
+
+ hw->tdata1 = tdata1;
+
+ return 0;
+}
+
+static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw)
+{
+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
+
+ switch (attr->bp_type) {
+ case HW_BREAKPOINT_X:
+ tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
+ break;
+ case HW_BREAKPOINT_R:
+ tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
+ break;
+ case HW_BREAKPOINT_W:
+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
+ break;
+ case HW_BREAKPOINT_RW:
+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (attr->bp_len) {
+ case HW_BREAKPOINT_LEN_1:
+ hw->len = 1;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
+ DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
+ break;
+ case HW_BREAKPOINT_LEN_2:
+ hw->len = 2;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
+ DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
+ break;
+ case HW_BREAKPOINT_LEN_4:
+ hw->len = 4;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
+ DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
+ break;
+ case HW_BREAKPOINT_LEN_8:
+ hw->len = 8;
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
+ DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ tdata1 |= DBTR_TDATA1_MCONTROL6_U;
+
+ hw->tdata1 = tdata1;
+
+ return 0;
+}
+
+int hw_breakpoint_arch_parse(struct perf_event *bp,
+ const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw)
+{
+ int ret;
+
+ /* Breakpoint address */
+ hw->address = attr->bp_addr;
+ hw->tdata2 = attr->bp_addr;
+ hw->tdata3 = 0x0;
+ hw->next_addr = 0x0;
+ hw->in_callback = false;
+
+ switch (dbtr_type) {
+ case DBTR_TDATA1_TYPE_MCONTROL:
+ ret = rv_init_mcontrol_trigger(attr, hw);
+ break;
+ case DBTR_TDATA1_TYPE_MCONTROL6:
+ ret = rv_init_mcontrol6_trigger(attr, hw);
+ break;
+ default:
+ pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
+ ret = -EOPNOTSUPP;
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Set breakpoint to next insruction after breakpoint.
+ * Returns 0 if success
+ * Returns < 0 on error
+ */
+static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
+{
+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
+ struct arch_hw_breakpoint old_hw_bp;
+ struct perf_event_attr bp_insn;
+ unsigned long next_addr, insn;
+ int ret;
+
+ /* Remove breakpoint even if return error as not to loop */
+ arch_uninstall_hw_breakpoint(event);
+
+ ret = get_insn(regs, regs->epc, &insn);
+ if (ret < 0)
+ return ret;
+
+ next_addr = get_step_address(regs, insn);
+
+ ret = get_insn(regs, next_addr, &insn);
+ if (ret < 0)
+ return ret;
+
+ bp_insn.bp_type = HW_BREAKPOINT_X;
+ bp_insn.bp_addr = next_addr;
+ /* Get the size of the intruction */
+ bp_insn.bp_len = GET_INSN_LENGTH(insn);
+
+ ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
+ if (ret)
+ return ret;
+
+ ret = arch_install_hw_breakpoint(event);
+ if (ret)
+ return ret;
+
+ bp->in_callback = true;
+ bp->next_addr = next_addr;
+ return 0;
+}
+
+/*
+ * HW Breakpoint/watchpoint handler
+ */
+static int hw_breakpoint_handler(struct pt_regs *regs)
+{
+ struct perf_event *event;
+ struct arch_hw_breakpoint *bp;
+ int ret, i;
+
+ for (i = 0; i < dbtr_total_num; i++) {
+ event = this_cpu_read(pcpu_hw_bp_events[i]);
+ if (!event)
+ continue;
+
+ bp = counter_arch_bp(event);
+ if (bp->in_callback) {
+ /* Reset changed breakpoint data */
+ bp->in_callback = false;
+ if (regs->epc == bp->next_addr) {
+ arch_uninstall_hw_breakpoint(event);
+ /* Restore original breakpoint */
+ if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
+ return NOTIFY_DONE;
+ if (arch_install_hw_breakpoint(event))
+ return NOTIFY_DONE;
+ return NOTIFY_STOP;
+ }
+
+ pr_err("%s: in_callback was set, but epc(%lx) was not next "
+ "address(%lx).\n", __func__, regs->epc, bp->next_addr);
+ bp->next_addr = 0x0;
+ return NOTIFY_DONE;
+ }
+
+ switch (event->attr.bp_type) {
+ /* Breakpoint */
+ case HW_BREAKPOINT_X:
+ if (event->attr.bp_addr == regs->epc) {
+ ret = setup_singlestep(event, regs);
+ if (ret < 0) {
+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
+ return NOTIFY_DONE;
+ }
+
+ perf_bp_event(event, regs);
+ return NOTIFY_STOP;
+ }
+ break;
+
+ /* Watchpoint */
+ case HW_BREAKPOINT_W:
+ case HW_BREAKPOINT_R:
+ case HW_BREAKPOINT_RW:
+ if (event->attr.bp_addr == regs->badaddr) {
+ ret = setup_singlestep(event, regs);
+ if (ret < 0) {
+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
+ return NOTIFY_DONE;
+ }
+
+ perf_bp_event(event, regs);
+ return NOTIFY_STOP;
+ }
+ break;
+
+ default:
+ pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
+ break;
+ }
+ }
+
+ return NOTIFY_DONE;
+}
+
+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data)
+{
+ struct die_args *args = data;
+
+ if (val != DIE_DEBUG)
+ return NOTIFY_DONE;
+
+ return hw_breakpoint_handler(args->regs);
+}
+
+/* atomic: counter->ctx->lock is held */
+int arch_install_hw_breakpoint(struct perf_event *event)
+{
+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
+ union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
+ struct sbi_dbtr_data_msg *xmit;
+ struct sbi_dbtr_id_msg *recv;
+ struct perf_event **slot;
+ unsigned long idx;
+ struct sbiret ret;
+ int err = 0;
+
+ raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
+ *this_cpu_ptr(&ecall_lock_flags));
+
+ xmit = &shmem->data;
+ recv = &shmem->id;
+ xmit->tdata1 = cpu_to_le(bp->tdata1);
+ xmit->tdata2 = cpu_to_le(bp->tdata2);
+ xmit->tdata3 = cpu_to_le(bp->tdata3);
+
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
+ 1, 0, 0, 0, 0, 0);
+
+ if (ret.error) {
+ pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
+ err = sbi_err_map_linux_errno(ret.error);
+ goto done;
+ }
+
+ idx = le_to_cpu(recv->idx);
+ if (idx >= dbtr_total_num) {
+ pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
+ err = -EINVAL;
+ goto done;
+ }
+
+ slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
+ if (*slot) {
+ pr_warn("%s: slot %lu is in use\n", __func__, idx);
+ err = -EBUSY;
+ goto done;
+ }
+
+ pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
+
+ /* Save the event - to be looked up in handler */
+ *slot = event;
+
+done:
+ raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
+ *this_cpu_ptr(&ecall_lock_flags));
+ return err;
+}
+
+/* atomic: counter->ctx->lock is held */
+void arch_uninstall_hw_breakpoint(struct perf_event *event)
+{
+ struct sbiret ret;
+ int i;
+
+ for (i = 0; i < dbtr_total_num; i++) {
+ struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
+
+ if (*slot == event) {
+ *slot = NULL;
+ break;
+ }
+ }
+
+
+ if (i == dbtr_total_num) {
+ pr_warn("%s: Breakpoint not installed.\n", __func__);
+ return;
+ }
+
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
+ i, 1, 0, 0, 0, 0);
+ if (ret.error)
+ pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
+}
+
+void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
+
+void hw_breakpoint_pmu_read(struct perf_event *bp) { }
+
+static int __init arch_hw_breakpoint_init(void)
+{
+ unsigned int cpu;
+ int rc = 0;
+
+ for_each_possible_cpu(cpu)
+ raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
+
+ init_sbi_dbtr();
+
+ if (dbtr_total_num) {
+ pr_debug("%s: total number of type %lu triggers: %u\n",
+ __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
+ } else {
+ pr_debug("%s: No hardware triggers available\n", __func__);
+ return rc;
+ }
+
+ /* Hotplug handler to register/unregister shared memory with SBI */
+ rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "riscv/hw_breakpoint:prepare",
+ arch_smp_setup_sbi_shmem,
+ arch_smp_teardown_sbi_shmem);
+
+ if (rc < 0)
+ pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
+
+ return rc;
+}
+arch_initcall(arch_hw_breakpoint_init);
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 938a8b841f94..2ac471ec79a8 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
if (probe_breakpoint_handler(regs))
return;
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
+ == NOTIFY_STOP)
+ return;
+#endif
+
current->thread.bad_cause = regs->cause;
if (user_mode(regs))
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
` (3 preceding siblings ...)
2025-07-22 17:38 ` [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-08-04 8:15 ` Clément Léger
2025-07-22 17:38 ` [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support Jesse Taube
5 siblings, 1 reply; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
The Sdtrig RISC-V ISA extension does not have a resume flag for
returning to and executing the instruction at the breakpoint.
To avoid skipping the instruction or looping, it is necessary to remove
the hardware breakpoint and single step. Use the icount feature of
Sdtrig to accomplish this. Use icount as default with an option to allow
software-based single stepping when hardware or SBI does not have
icount functionality, as it may cause unwanted side effects when reading
the instruction from memory.
Signed-off-by: Jesse Taube <jesse@rivosinc.com>
---
arch/riscv/Kconfig | 11 +++++
arch/riscv/kernel/hw_breakpoint.c | 81 +++++++++++++++++++++++++------
2 files changed, 76 insertions(+), 16 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 95d3047cab10..bbde5e118470 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -1105,6 +1105,17 @@ config PARAVIRT_TIME_ACCOUNTING
If in doubt, say N here.
+config HW_BREAKPOINT_COMPUTE_STEP
+ bool "Allow computing hardware breakpoint step address"
+ default n
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Select this option if hardware breakpoints are desired, but
+ hardware or SBI does not have icount functionality. This may cause
+ unwanted side affects when reading the instruction from memory.
+
+ If unsure, say N.
+
config RELOCATABLE
bool "Build a relocatable kernel"
depends on !XIP_KERNEL
diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
index 9e3a3b82d300..437fd82b9590 100644
--- a/arch/riscv/kernel/hw_breakpoint.c
+++ b/arch/riscv/kernel/hw_breakpoint.c
@@ -20,6 +20,7 @@
#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
+#define DBTR_TDATA1_TYPE_ICOUNT (3UL << DBTR_TDATA1_TYPE_SHIFT)
#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
@@ -55,6 +56,14 @@
#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
+#define DBTR_TDATA1_ICOUNT_U BIT(6)
+#define DBTR_TDATA1_ICOUNT_S BIT(7)
+#define DBTR_TDATA1_ICOUNT_PENDING BIT(8)
+#define DBTR_TDATA1_ICOUNT_M BIT(9)
+#define DBTR_TDATA1_ICOUNT_COUNT_FIELD GENMASK(23, 10)
+#define DBTR_TDATA1_ICOUNT_VU BIT(25)
+#define DBTR_TDATA1_ICOUNT_VS BIT(26)
+
/* Registered per-cpu bp/wp */
static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
@@ -65,6 +74,7 @@ static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
/* number of debug triggers on this cpu . */
static int dbtr_total_num __ro_after_init;
+static bool have_icount __ro_after_init;
static unsigned long dbtr_type __ro_after_init;
static unsigned long dbtr_init __ro_after_init;
@@ -168,6 +178,7 @@ static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
static void init_sbi_dbtr(void)
{
struct sbiret ret;
+ unsigned long dbtr_count = 0;
/*
* Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
@@ -182,6 +193,25 @@ static void init_sbi_dbtr(void)
return;
}
+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
+ DBTR_TDATA1_TYPE_ICOUNT, 0, 0, 0, 0, 0);
+ if (ret.error) {
+ pr_warn("%s: failed to detect icount triggers. error: %ld.\n",
+ __func__, ret.error);
+ } else if (!ret.value) {
+ if (IS_ENABLED(CONFIG_HW_BREAKPOINT_COMPUTE_STEP)) {
+ pr_warn("%s: No icount triggers available. "
+ "Falling-back to computing single step address.\n", __func__);
+ } else {
+ pr_err("%s: No icount triggers available.\n", __func__);
+ dbtr_total_num = 0;
+ return;
+ }
+ } else {
+ dbtr_count = ret.value;
+ have_icount = true;
+ }
+
ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
if (ret.error) {
@@ -190,7 +220,7 @@ static void init_sbi_dbtr(void)
} else if (!ret.value) {
pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
} else {
- dbtr_total_num = ret.value;
+ dbtr_total_num = min_not_zero((unsigned long)ret.value, dbtr_count);
dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
return;
}
@@ -205,7 +235,7 @@ static void init_sbi_dbtr(void)
pr_err("%s: No mcontrol triggers available.\n", __func__);
dbtr_total_num = 0;
} else {
- dbtr_total_num = ret.value;
+ dbtr_total_num = min_not_zero((unsigned long)ret.value, dbtr_count);
dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
}
}
@@ -344,6 +374,21 @@ static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
return 0;
}
+static int rv_init_icount_trigger(struct arch_hw_breakpoint *hw)
+{
+ unsigned long tdata1 = DBTR_TDATA1_TYPE_ICOUNT;
+
+ /* Step one instruction */
+ tdata1 |= FIELD_PREP(DBTR_TDATA1_ICOUNT_COUNT_FIELD, 1);
+
+ tdata1 |= DBTR_TDATA1_ICOUNT_U;
+
+ hw->tdata1 = tdata1;
+ hw->tdata2 = 0;
+
+ return 0;
+}
+
int hw_breakpoint_arch_parse(struct perf_event *bp,
const struct perf_event_attr *attr,
struct arch_hw_breakpoint *hw)
@@ -389,24 +434,28 @@ static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
/* Remove breakpoint even if return error as not to loop */
arch_uninstall_hw_breakpoint(event);
- ret = get_insn(regs, regs->epc, &insn);
- if (ret < 0)
- return ret;
+ if (have_icount) {
+ rv_init_icount_trigger(bp);
+ } else {
+ ret = get_insn(regs, regs->epc, &insn);
+ if (ret < 0)
+ return ret;
- next_addr = get_step_address(regs, insn);
+ next_addr = get_step_address(regs, insn);
- ret = get_insn(regs, next_addr, &insn);
- if (ret < 0)
- return ret;
+ ret = get_insn(regs, next_addr, &insn);
+ if (ret < 0)
+ return ret;
- bp_insn.bp_type = HW_BREAKPOINT_X;
- bp_insn.bp_addr = next_addr;
- /* Get the size of the intruction */
- bp_insn.bp_len = GET_INSN_LENGTH(insn);
+ bp_insn.bp_type = HW_BREAKPOINT_X;
+ bp_insn.bp_addr = next_addr;
+ /* Get the size of the intruction */
+ bp_insn.bp_len = GET_INSN_LENGTH(insn);
- ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
- if (ret)
- return ret;
+ ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
+ if (ret)
+ return ret;
+ }
ret = arch_install_hw_breakpoint(event);
if (ret)
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
` (4 preceding siblings ...)
2025-07-22 17:38 ` [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping Jesse Taube
@ 2025-07-22 17:38 ` Jesse Taube
2025-07-23 4:18 ` Deepak Gupta
2025-08-04 8:16 ` Clément Léger
5 siblings, 2 replies; 18+ messages in thread
From: Jesse Taube @ 2025-07-22 17:38 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Jesse Taube, Himanshu Chauhan, Charlie Jenkins,
Samuel Holland, Deepak Gupta, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
Add ability to setup hw breakpoints to ptrace. Call defines a new
structure of (ulong[3]){bp_addr, bp_len, bp_type} with
bp_type being one of HW_BREAKPOINT_LEN_X and
bp_len being one of HW_BREAKPOINT_X with a value of
zero dissabling the breakpoint.
Signed-off-by: Jesse Taube <jesse@rivosinc.com>
---
arch/riscv/include/asm/processor.h | 4 ++
arch/riscv/include/uapi/asm/ptrace.h | 3 +-
arch/riscv/kernel/hw_breakpoint.c | 14 ++++-
arch/riscv/kernel/process.c | 4 ++
arch/riscv/kernel/ptrace.c | 93 ++++++++++++++++++++++++++++
5 files changed, 116 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 5f56eb9d114a..488d956a951f 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -12,6 +12,7 @@
#include <vdso/processor.h>
+#include <asm/hw_breakpoint.h>
#include <asm/ptrace.h>
#define arch_get_mmap_end(addr, len, flags) \
@@ -108,6 +109,9 @@ struct thread_struct {
struct __riscv_v_ext_state vstate;
unsigned long align_ctl;
struct __riscv_v_ext_state kernel_vstate;
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ struct perf_event *ptrace_bps[RV_MAX_TRIGGERS];
+#endif
#ifdef CONFIG_SMP
/* Flush the icache on migration */
bool force_icache_flush;
diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
index a38268b19c3d..a7998ed41913 100644
--- a/arch/riscv/include/uapi/asm/ptrace.h
+++ b/arch/riscv/include/uapi/asm/ptrace.h
@@ -14,7 +14,8 @@
#define PTRACE_GETFDPIC_EXEC 0
#define PTRACE_GETFDPIC_INTERP 1
-
+#define PTRACE_GETHBPREGS 2
+#define PTRACE_SETHBPREGS 3
/*
* User-mode register state for core dumps, ptrace, sigcontext
*
diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
index 437fd82b9590..c58145464539 100644
--- a/arch/riscv/kernel/hw_breakpoint.c
+++ b/arch/riscv/kernel/hw_breakpoint.c
@@ -633,7 +633,19 @@ void arch_uninstall_hw_breakpoint(struct perf_event *event)
pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
}
-void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
+/*
+ * Release the user breakpoints used by ptrace
+ */
+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
+{
+ int i;
+ struct thread_struct *t = &tsk->thread;
+
+ for (i = 0; i < dbtr_total_num; i++) {
+ unregister_hw_breakpoint(t->ptrace_bps[i]);
+ t->ptrace_bps[i] = NULL;
+ }
+}
void hw_breakpoint_pmu_read(struct perf_event *bp) { }
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 15d8f75902f8..9cf07ecfb523 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -9,6 +9,7 @@
#include <linux/bitfield.h>
#include <linux/cpu.h>
+#include <linux/hw_breakpoint.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/sched/debug.h>
@@ -164,6 +165,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
void flush_thread(void)
{
+ flush_ptrace_hw_breakpoint(current);
#ifdef CONFIG_FPU
/*
* Reset FPU state and context
@@ -218,6 +220,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
set_bit(MM_CONTEXT_LOCK_PMLEN, &p->mm->context.flags);
memset(&p->thread.s, 0, sizeof(p->thread.s));
+ if (IS_ENABLED(CONFIG_HAVE_HW_BREAKPOINT))
+ memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
/* p->thread holds context to be restored by __switch_to() */
if (unlikely(args->fn)) {
diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
index ea67e9fb7a58..b78cfb0f1c0e 100644
--- a/arch/riscv/kernel/ptrace.c
+++ b/arch/riscv/kernel/ptrace.c
@@ -9,11 +9,13 @@
#include <asm/vector.h>
#include <asm/ptrace.h>
+#include <asm/hw_breakpoint.h>
#include <asm/syscall.h>
#include <asm/thread_info.h>
#include <asm/switch_to.h>
#include <linux/audit.h>
#include <linux/compat.h>
+#include <linux/hw_breakpoint.h>
#include <linux/ptrace.h>
#include <linux/elf.h>
#include <linux/regset.h>
@@ -336,12 +338,103 @@ void ptrace_disable(struct task_struct *child)
{
}
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+static void ptrace_hbptriggered(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
+ int num = 0;
+
+ force_sig_ptrace_errno_trap(num, (void __user *)bkpt->address);
+}
+
+/*
+ * idx selects the breakpoint index.
+ * Both PTRACE_GETHBPREGS and PTRACE_SETHBPREGS transfer three 32-bit words:
+ * address (0), length (1), type (2).
+ * Instruction breakpoint length is one of HW_BREAKPOINT_LEN_X or 0. 0 will
+ * disable the breakpoint.
+ * Instruction breakpoint type is one of HW_BREAKPOINT_X.
+ */
+
+static long ptrace_gethbpregs(struct task_struct *child, unsigned long idx,
+ unsigned long __user *datap)
+{
+ struct perf_event *bp;
+ unsigned long user_data[3] = {0};
+
+ if (idx >= RV_MAX_TRIGGERS)
+ return -EINVAL;
+
+ bp = child->thread.ptrace_bps[idx];
+
+ if (!IS_ERR_OR_NULL(bp)) {
+ user_data[0] = bp->attr.bp_addr;
+ user_data[1] = bp->attr.disabled ? 0 : bp->attr.bp_len;
+ user_data[2] = bp->attr.bp_type;
+ }
+
+ if (copy_to_user(datap, user_data, sizeof(user_data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static long ptrace_sethbpregs(struct task_struct *child, unsigned long idx,
+ unsigned long __user *datap)
+{
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+ unsigned long user_data[3];
+
+ if (idx >= RV_MAX_TRIGGERS)
+ return -EINVAL;
+
+ if (copy_from_user(user_data, datap, sizeof(user_data)))
+ return -EFAULT;
+
+ bp = child->thread.ptrace_bps[idx];
+ if (IS_ERR_OR_NULL(bp))
+ attr = bp->attr;
+ else
+ ptrace_breakpoint_init(&attr);
+
+ attr.bp_addr = user_data[0];
+ attr.bp_len = user_data[1];
+ attr.bp_type = user_data[2];
+ attr.disabled = !attr.bp_len;
+
+ if (IS_ERR_OR_NULL(bp)) {
+ bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL,
+ child);
+ if (IS_ERR(bp))
+ return PTR_ERR(bp);
+
+ child->thread.ptrace_bps[idx] = bp;
+ return 0;
+ } else {
+ return modify_user_hw_breakpoint(bp, &attr);
+ }
+}
+#endif
+
long arch_ptrace(struct task_struct *child, long request,
unsigned long addr, unsigned long data)
{
long ret = -EIO;
+ unsigned long __user *datap = (unsigned long __user *) data;
switch (request) {
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ case PTRACE_GETHBPREGS:
+ ret = ptrace_gethbpregs(child, addr, datap);
+ break;
+
+ case PTRACE_SETHBPREGS:
+ ret = ptrace_sethbpregs(child, addr, datap);
+ break;
+#endif
default:
ret = ptrace_request(child, request, addr, data);
break;
--
2.43.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-07-22 17:38 ` [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints Jesse Taube
@ 2025-07-23 2:49 ` Deepak Gupta
2025-07-23 17:02 ` Jesse Taube
2025-08-04 8:01 ` Clément Léger
1 sibling, 1 reply; 18+ messages in thread
From: Deepak Gupta @ 2025-07-23 2:49 UTC (permalink / raw)
To: Jesse Taube
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Tue, Jul 22, 2025 at 10:38:27AM -0700, Jesse Taube wrote:
>From: Himanshu Chauhan <hchauhan@ventanamicro.com>
>
>RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
>SBI debug trigger extension to install/uninstall/update/enable/disable hardware
>triggers as specified in Sdtrig ISA extension.
>
>Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
>Signed-off-by: Jesse Taube <jesse@rivosinc.com>
>---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/hw_breakpoint.h | 60 +++
> arch/riscv/include/asm/kdebug.h | 3 +-
> arch/riscv/include/asm/sbi.h | 4 +-
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
> arch/riscv/kernel/traps.c | 6 +
> 7 files changed, 693 insertions(+), 2 deletions(-)
> create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
> create mode 100644 arch/riscv/kernel/hw_breakpoint.c
>
>diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>index bbec87b79309..95d3047cab10 100644
>--- a/arch/riscv/Kconfig
>+++ b/arch/riscv/Kconfig
>@@ -163,6 +163,7 @@ config RISCV
> select HAVE_FUNCTION_ERROR_INJECTION
> select HAVE_GCC_PLUGINS
> select HAVE_GENERIC_VDSO if MMU && 64BIT
>+ select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
> select HAVE_IRQ_TIME_ACCOUNTING
> select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
> select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
>diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
>new file mode 100644
>index 000000000000..8efa3921c535
>--- /dev/null
>+++ b/arch/riscv/include/asm/hw_breakpoint.h
>@@ -0,0 +1,60 @@
>+/* SPDX-License-Identifier: GPL-2.0-only */
>+/*
>+ * Copyright (C) 2024 Ventana Micro Systems Inc.
>+ */
>+
>+#ifndef __RISCV_HW_BREAKPOINT_H
>+#define __RISCV_HW_BREAKPOINT_H
>+
>+struct task_struct;
>+
>+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>+
>+#include <uapi/linux/hw_breakpoint.h>
>+
>+#if __riscv_xlen == 64
>+#define cpu_to_le cpu_to_le64
>+#define le_to_cpu le64_to_cpu
>+#elif __riscv_xlen == 32
>+#define cpu_to_le cpu_to_le32
>+#define le_to_cpu le32_to_cpu
>+#else
>+#error "Unexpected __riscv_xlen"
>+#endif
>+
>+struct arch_hw_breakpoint {
>+ unsigned long address;
>+ unsigned long len;
>+
>+ /* Callback info */
>+ unsigned long next_addr;
>+ bool in_callback;
>+
>+
>+ /* Trigger configuration data */
>+ unsigned long tdata1;
>+ unsigned long tdata2;
>+ unsigned long tdata3;
>+};
>+
>+/* Maximum number of hardware breakpoints supported */
>+#define RV_MAX_TRIGGERS 32
>+
>+struct perf_event_attr;
>+struct notifier_block;
>+struct perf_event;
>+struct pt_regs;
>+
>+int hw_breakpoint_slots(int type);
>+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
>+int hw_breakpoint_arch_parse(struct perf_event *bp,
>+ const struct perf_event_attr *attr,
>+ struct arch_hw_breakpoint *hw);
>+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
>+ unsigned long val, void *data);
>+int arch_install_hw_breakpoint(struct perf_event *bp);
>+void arch_uninstall_hw_breakpoint(struct perf_event *bp);
>+void hw_breakpoint_pmu_read(struct perf_event *bp);
>+
>+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
>+#endif /* __RISCV_HW_BREAKPOINT_H */
>diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
>index 85ac00411f6e..53e989781aa1 100644
>--- a/arch/riscv/include/asm/kdebug.h
>+++ b/arch/riscv/include/asm/kdebug.h
>@@ -6,7 +6,8 @@
> enum die_val {
> DIE_UNUSED,
> DIE_TRAP,
>- DIE_OOPS
>+ DIE_OOPS,
>+ DIE_DEBUG
> };
>
> #endif
>diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
>index be2ca8e8a49e..64fa7a82aa45 100644
>--- a/arch/riscv/include/asm/sbi.h
>+++ b/arch/riscv/include/asm/sbi.h
>@@ -282,7 +282,9 @@ struct sbi_sta_struct {
> u8 pad[47];
> } __packed;
>
>-#define SBI_SHMEM_DISABLE -1
>+#define SBI_SHMEM_DISABLE (-1UL)
>+#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
>+#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
>
> enum sbi_ext_nacl_fid {
> SBI_EXT_NACL_PROBE_FEATURE = 0x0,
>diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
>index 4f719b09e5ad..3e72505734bd 100644
>--- a/arch/riscv/kernel/Makefile
>+++ b/arch/riscv/kernel/Makefile
>@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
>
> obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
> obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
>+obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
> obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
> ifeq ($(CONFIG_RISCV_SBI), y)
> obj-$(CONFIG_SMP) += sbi-ipi.o
>diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
>new file mode 100644
>index 000000000000..9e3a3b82d300
>--- /dev/null
>+++ b/arch/riscv/kernel/hw_breakpoint.c
>@@ -0,0 +1,620 @@
>+// SPDX-License-Identifier: GPL-2.0-only
>+/*
>+ * Copyright (C) 2024 Ventana Micro Systems Inc.
>+ */
>+
>+#include <linux/hw_breakpoint.h>
>+#include <linux/perf_event.h>
>+#include <linux/spinlock.h>
>+#include <linux/percpu.h>
>+#include <linux/kdebug.h>
>+#include <linux/bitops.h>
>+#include <linux/bitfield.h>
>+#include <linux/cpu.h>
>+#include <linux/cpuhotplug.h>
>+
>+#include <asm/insn.h>
>+#include <asm/sbi.h>
>+
>+#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
>+#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
>+
>+#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
>+#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
>+
>+#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
>+#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
>+#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
>+#define DBTR_TDATA1_MCONTROL6_U BIT(3)
>+#define DBTR_TDATA1_MCONTROL6_S BIT(4)
>+#define DBTR_TDATA1_MCONTROL6_M BIT(6)
>+#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
>+#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
>+#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
>+#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
>+
>+#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
>+#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
>+#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
>+#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
>+
>+#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
>+#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
>+#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
>+#define DBTR_TDATA1_MCONTROL_U BIT(3)
>+#define DBTR_TDATA1_MCONTROL_S BIT(4)
>+#define DBTR_TDATA1_MCONTROL_M BIT(6)
>+#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
>+#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
>+#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
>+
>+#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
>+#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
>+#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
>+/* value of 5 split across HI and LO */
>+#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
>+#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
>+
>+/* Registered per-cpu bp/wp */
>+static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
>+static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
>+static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
>+
>+/* Per-cpu shared memory between S and M mode */
>+static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
>+
>+/* number of debug triggers on this cpu . */
>+static int dbtr_total_num __ro_after_init;
>+static unsigned long dbtr_type __ro_after_init;
>+static unsigned long dbtr_init __ro_after_init;
>+
>+static int arch_smp_setup_sbi_shmem(unsigned int cpu)
>+{
>+ union sbi_dbtr_shmem_entry *dbtr_shmem;
>+ unsigned long shmem_pa;
>+ struct sbiret ret;
>+ int rc;
>+
>+ dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
>+ if (!dbtr_shmem) {
>+ pr_err("Invalid per-cpu shared memory for debug triggers\n");
>+ return -ENODEV;
>+ }
>+
>+ shmem_pa = virt_to_phys(dbtr_shmem);
>+
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
>+ SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
>+
>+ if (ret.error) {
>+ switch (ret.error) {
>+ case SBI_ERR_DENIED:
>+ pr_warn("%s: Access denied for shared memory at %lx\n",
>+ __func__, shmem_pa);
>+ rc = -EPERM;
>+ break;
>+
>+ case SBI_ERR_INVALID_PARAM:
>+ case SBI_ERR_INVALID_ADDRESS:
>+ pr_warn("%s: Invalid address parameter (%ld)\n",
>+ __func__, ret.error);
>+ rc = -EINVAL;
>+ break;
>+
>+ case SBI_ERR_ALREADY_AVAILABLE:
>+ pr_warn("%s: Shared memory is already set\n",
>+ __func__);
>+ rc = -EADDRINUSE;
>+ break;
>+
>+ case SBI_ERR_FAILURE:
>+ pr_err("%s: Internal sdtrig state error\n",
>+ __func__);
>+ rc = -ENXIO;
>+ break;
>+
>+ default:
>+ pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
>+ rc = -ENXIO;
>+ break;
>+ }
>+ }
>+
>+ pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
>+
>+ return rc;
>+}
>+
>+static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
>+{
>+ struct sbiret ret;
>+
>+ /* Disable shared memory */
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
>+ SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
>+
>+ if (ret.error) {
>+ switch (ret.error) {
>+ case SBI_ERR_DENIED:
>+ pr_err("%s: Access denied for shared memory.\n",
>+ __func__);
>+ break;
>+
>+ case SBI_ERR_INVALID_PARAM:
>+ case SBI_ERR_INVALID_ADDRESS:
>+ pr_err("%s: Invalid address parameter (%lu)\n",
>+ __func__, ret.error);
>+ break;
>+
>+ case SBI_ERR_ALREADY_AVAILABLE:
>+ pr_err("%s: Shared memory is already set\n",
>+ __func__);
>+ break;
>+ case SBI_ERR_FAILURE:
>+ pr_err("%s: Internal sdtrig state error\n",
>+ __func__);
>+ break;
>+ default:
>+ pr_err("%s: Unknown error %lu\n", __func__, ret.error);
>+ break;
>+ }
>+ }
>+
>+ pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
>+
>+ return 0;
>+}
>+
>+static void init_sbi_dbtr(void)
>+{
>+ struct sbiret ret;
>+
>+ /*
>+ * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
>+ * Only proceed if this is the first CPU to reach this code.
>+ */
>+ if (test_and_set_bit(0, &dbtr_init))
>+ return;
>+
>+ if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
>+ pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
>+ dbtr_total_num = 0;
>+ return;
>+ }
>+
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
>+ DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
>+ if (ret.error) {
>+ pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
>+ __func__, ret.error);
>+ } else if (!ret.value) {
>+ pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
>+ } else {
>+ dbtr_total_num = ret.value;
>+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
>+ return;
>+ }
>+
>+ /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
>+ DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
>+ if (ret.error) {
>+ pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
>+ __func__, ret.error);
>+ } else if (!ret.value) {
>+ pr_err("%s: No mcontrol triggers available.\n", __func__);
>+ dbtr_total_num = 0;
>+ } else {
>+ dbtr_total_num = ret.value;
>+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
>+ }
>+}
`dbtr_total_num` is static and thus would be initialized to zero.
Although above code is setting it to zero in some error conditions.
And not setting to zero in other error conditions.
It'll be better if function starts out setting `dbtr_total_num` as setting to 0.
Then set correct value in success conditions.
>+
>+int hw_breakpoint_slots(int type)
>+{
>+ /*
>+ * We can be called early, so don't rely on
>+ * static variables being initialised.
>+ */
>+ init_sbi_dbtr();
>+
>+ return dbtr_total_num;
>+}
>+
>+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
>+{
>+ unsigned int len;
>+ unsigned long va;
>+
>+ va = hw->address;
>+ len = hw->len;
>+
>+ return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
>+}
>+
>+static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
>+ struct arch_hw_breakpoint *hw)
>+{
>+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
>+
>+ switch (attr->bp_type) {
>+ case HW_BREAKPOINT_X:
>+ tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
>+ break;
>+ case HW_BREAKPOINT_R:
>+ tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
>+ break;
>+ case HW_BREAKPOINT_W:
>+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
>+ break;
>+ case HW_BREAKPOINT_RW:
>+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
>+ break;
>+ default:
>+ return -EINVAL;
>+ }
>+
>+ switch (attr->bp_len) {
>+ case HW_BREAKPOINT_LEN_1:
>+ hw->len = 1;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>+ DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
>+ break;
>+ case HW_BREAKPOINT_LEN_2:
>+ hw->len = 2;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>+ DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
>+ break;
>+ case HW_BREAKPOINT_LEN_4:
>+ hw->len = 4;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>+ DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
>+ break;
>+#if __riscv_xlen >= 64
>+ case HW_BREAKPOINT_LEN_8:
>+ hw->len = 8;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>+ DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
>+ FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
>+ DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
>+ break;
>+#endif
>+ default:
>+ return -EINVAL;
>+ }
>+
>+ tdata1 |= DBTR_TDATA1_MCONTROL_U;
Assuming this function could be re-used for kernel data breakpoints too
Shouldn't this be based on some flag to select U v/s S.
Same comment for setting up tdata1 with mcontrol6.
>+
>+ hw->tdata1 = tdata1;
>+
>+ return 0;
>+}
>+
>+static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
>+ struct arch_hw_breakpoint *hw)
>+{
>+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
>+
>+ switch (attr->bp_type) {
>+ case HW_BREAKPOINT_X:
>+ tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
>+ break;
>+ case HW_BREAKPOINT_R:
>+ tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
>+ break;
>+ case HW_BREAKPOINT_W:
>+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
>+ break;
>+ case HW_BREAKPOINT_RW:
>+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
>+ break;
>+ default:
>+ return -EINVAL;
>+ }
>+
>+ switch (attr->bp_len) {
>+ case HW_BREAKPOINT_LEN_1:
>+ hw->len = 1;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>+ DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
>+ break;
>+ case HW_BREAKPOINT_LEN_2:
>+ hw->len = 2;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>+ DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
>+ break;
>+ case HW_BREAKPOINT_LEN_4:
>+ hw->len = 4;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>+ DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
>+ break;
>+ case HW_BREAKPOINT_LEN_8:
>+ hw->len = 8;
>+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>+ DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
>+ break;
>+ default:
>+ return -EINVAL;
>+ }
>+
>+ tdata1 |= DBTR_TDATA1_MCONTROL6_U;
>+
>+ hw->tdata1 = tdata1;
>+
>+ return 0;
>+}
>+
>+int hw_breakpoint_arch_parse(struct perf_event *bp,
>+ const struct perf_event_attr *attr,
>+ struct arch_hw_breakpoint *hw)
>+{
>+ int ret;
>+
>+ /* Breakpoint address */
>+ hw->address = attr->bp_addr;
>+ hw->tdata2 = attr->bp_addr;
>+ hw->tdata3 = 0x0;
>+ hw->next_addr = 0x0;
>+ hw->in_callback = false;
>+
>+ switch (dbtr_type) {
>+ case DBTR_TDATA1_TYPE_MCONTROL:
>+ ret = rv_init_mcontrol_trigger(attr, hw);
>+ break;
>+ case DBTR_TDATA1_TYPE_MCONTROL6:
>+ ret = rv_init_mcontrol6_trigger(attr, hw);
>+ break;
>+ default:
>+ pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
>+ ret = -EOPNOTSUPP;
>+ break;
>+ }
>+
>+ return ret;
>+}
>+
>+/*
>+ * Set breakpoint to next insruction after breakpoint.
>+ * Returns 0 if success
>+ * Returns < 0 on error
>+ */
>+static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
>+{
>+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
>+ struct arch_hw_breakpoint old_hw_bp;
>+ struct perf_event_attr bp_insn;
>+ unsigned long next_addr, insn;
>+ int ret;
>+
>+ /* Remove breakpoint even if return error as not to loop */
>+ arch_uninstall_hw_breakpoint(event);
>+
>+ ret = get_insn(regs, regs->epc, &insn);
>+ if (ret < 0)
>+ return ret;
>+
>+ next_addr = get_step_address(regs, insn);
>+
>+ ret = get_insn(regs, next_addr, &insn);
>+ if (ret < 0)
>+ return ret;
>+
>+ bp_insn.bp_type = HW_BREAKPOINT_X;
>+ bp_insn.bp_addr = next_addr;
>+ /* Get the size of the intruction */
>+ bp_insn.bp_len = GET_INSN_LENGTH(insn);
>+
>+ ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
>+ if (ret)
>+ return ret;
>+
>+ ret = arch_install_hw_breakpoint(event);
>+ if (ret)
>+ return ret;
>+
>+ bp->in_callback = true;
>+ bp->next_addr = next_addr;
>+ return 0;
>+}
>+
>+/*
>+ * HW Breakpoint/watchpoint handler
>+ */
>+static int hw_breakpoint_handler(struct pt_regs *regs)
>+{
>+ struct perf_event *event;
>+ struct arch_hw_breakpoint *bp;
>+ int ret, i;
>+
>+ for (i = 0; i < dbtr_total_num; i++) {
>+ event = this_cpu_read(pcpu_hw_bp_events[i]);
>+ if (!event)
>+ continue;
>+
>+ bp = counter_arch_bp(event);
>+ if (bp->in_callback) {
>+ /* Reset changed breakpoint data */
>+ bp->in_callback = false;
>+ if (regs->epc == bp->next_addr) {
>+ arch_uninstall_hw_breakpoint(event);
>+ /* Restore original breakpoint */
>+ if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
>+ return NOTIFY_DONE;
>+ if (arch_install_hw_breakpoint(event))
>+ return NOTIFY_DONE;
>+ return NOTIFY_STOP;
>+ }
>+
>+ pr_err("%s: in_callback was set, but epc(%lx) was not next "
>+ "address(%lx).\n", __func__, regs->epc, bp->next_addr);
>+ bp->next_addr = 0x0;
>+ return NOTIFY_DONE;
>+ }
>+
>+ switch (event->attr.bp_type) {
>+ /* Breakpoint */
>+ case HW_BREAKPOINT_X:
>+ if (event->attr.bp_addr == regs->epc) {
>+ ret = setup_singlestep(event, regs);
>+ if (ret < 0) {
>+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
>+ return NOTIFY_DONE;
>+ }
>+
>+ perf_bp_event(event, regs);
>+ return NOTIFY_STOP;
>+ }
>+ break;
>+
>+ /* Watchpoint */
>+ case HW_BREAKPOINT_W:
>+ case HW_BREAKPOINT_R:
>+ case HW_BREAKPOINT_RW:
>+ if (event->attr.bp_addr == regs->badaddr) {
>+ ret = setup_singlestep(event, regs);
>+ if (ret < 0) {
>+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
>+ return NOTIFY_DONE;
>+ }
>+
>+ perf_bp_event(event, regs);
>+ return NOTIFY_STOP;
>+ }
>+ break;
>+
>+ default:
>+ pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
>+ break;
>+ }
>+ }
>+
>+ return NOTIFY_DONE;
>+}
>+
>+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
>+ unsigned long val, void *data)
>+{
>+ struct die_args *args = data;
>+
>+ if (val != DIE_DEBUG)
>+ return NOTIFY_DONE;
>+
>+ return hw_breakpoint_handler(args->regs);
>+}
>+
>+/* atomic: counter->ctx->lock is held */
>+int arch_install_hw_breakpoint(struct perf_event *event)
>+{
>+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
>+ union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
>+ struct sbi_dbtr_data_msg *xmit;
>+ struct sbi_dbtr_id_msg *recv;
>+ struct perf_event **slot;
>+ unsigned long idx;
>+ struct sbiret ret;
>+ int err = 0;
>+
>+ raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
>+ *this_cpu_ptr(&ecall_lock_flags));
>+
>+ xmit = &shmem->data;
>+ recv = &shmem->id;
>+ xmit->tdata1 = cpu_to_le(bp->tdata1);
>+ xmit->tdata2 = cpu_to_le(bp->tdata2);
>+ xmit->tdata3 = cpu_to_le(bp->tdata3);
>+
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
>+ 1, 0, 0, 0, 0, 0);
>+
>+ if (ret.error) {
>+ pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
>+ err = sbi_err_map_linux_errno(ret.error);
>+ goto done;
>+ }
>+
>+ idx = le_to_cpu(recv->idx);
>+ if (idx >= dbtr_total_num) {
>+ pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
>+ err = -EINVAL;
>+ goto done;
>+ }
>+
>+ slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
>+ if (*slot) {
>+ pr_warn("%s: slot %lu is in use\n", __func__, idx);
>+ err = -EBUSY;
>+ goto done;
>+ }
>+
>+ pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
>+
>+ /* Save the event - to be looked up in handler */
>+ *slot = event;
>+
>+done:
>+ raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
>+ *this_cpu_ptr(&ecall_lock_flags));
>+ return err;
>+}
>+
>+/* atomic: counter->ctx->lock is held */
>+void arch_uninstall_hw_breakpoint(struct perf_event *event)
>+{
>+ struct sbiret ret;
>+ int i;
>+
>+ for (i = 0; i < dbtr_total_num; i++) {
>+ struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
>+
>+ if (*slot == event) {
>+ *slot = NULL;
>+ break;
>+ }
>+ }
>+
>+
>+ if (i == dbtr_total_num) {
>+ pr_warn("%s: Breakpoint not installed.\n", __func__);
>+ return;
>+ }
>+
>+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
>+ i, 1, 0, 0, 0, 0);
>+ if (ret.error)
>+ pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
>+}
>+
>+void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
>+
>+void hw_breakpoint_pmu_read(struct perf_event *bp) { }
>+
>+static int __init arch_hw_breakpoint_init(void)
>+{
>+ unsigned int cpu;
>+ int rc = 0;
>+
>+ for_each_possible_cpu(cpu)
>+ raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
>+
>+ init_sbi_dbtr();
>+
>+ if (dbtr_total_num) {
>+ pr_debug("%s: total number of type %lu triggers: %u\n",
>+ __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
>+ } else {
>+ pr_debug("%s: No hardware triggers available\n", __func__);
>+ return rc;
>+ }
>+
>+ /* Hotplug handler to register/unregister shared memory with SBI */
>+ rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
>+ "riscv/hw_breakpoint:prepare",
>+ arch_smp_setup_sbi_shmem,
>+ arch_smp_teardown_sbi_shmem);
>+
>+ if (rc < 0)
>+ pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
>+
>+ return rc;
>+}
>+arch_initcall(arch_hw_breakpoint_init);
>diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
>index 938a8b841f94..2ac471ec79a8 100644
>--- a/arch/riscv/kernel/traps.c
>+++ b/arch/riscv/kernel/traps.c
>@@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
> if (probe_breakpoint_handler(regs))
> return;
>
>+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>+ if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
>+ == NOTIFY_STOP)
>+ return;
>+#endif
>+
> current->thread.bad_cause = regs->cause;
>
> if (user_mode(regs))
>--
>2.43.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support
2025-07-22 17:38 ` [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support Jesse Taube
@ 2025-07-23 4:18 ` Deepak Gupta
2025-07-23 16:55 ` Jesse Taube
2025-08-04 8:16 ` Clément Léger
1 sibling, 1 reply; 18+ messages in thread
From: Deepak Gupta @ 2025-07-23 4:18 UTC (permalink / raw)
To: Jesse Taube
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Tue, Jul 22, 2025 at 10:38:29AM -0700, Jesse Taube wrote:
>Add ability to setup hw breakpoints to ptrace. Call defines a new
>structure of (ulong[3]){bp_addr, bp_len, bp_type} with
>bp_type being one of HW_BREAKPOINT_LEN_X and
>bp_len being one of HW_BREAKPOINT_X with a value of
>zero dissabling the breakpoint.
>
>Signed-off-by: Jesse Taube <jesse@rivosinc.com>
>---
> arch/riscv/include/asm/processor.h | 4 ++
> arch/riscv/include/uapi/asm/ptrace.h | 3 +-
> arch/riscv/kernel/hw_breakpoint.c | 14 ++++-
> arch/riscv/kernel/process.c | 4 ++
> arch/riscv/kernel/ptrace.c | 93 ++++++++++++++++++++++++++++
> 5 files changed, 116 insertions(+), 2 deletions(-)
>
>diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
>index 5f56eb9d114a..488d956a951f 100644
>--- a/arch/riscv/include/asm/processor.h
>+++ b/arch/riscv/include/asm/processor.h
>@@ -12,6 +12,7 @@
>
> #include <vdso/processor.h>
>
>+#include <asm/hw_breakpoint.h>
> #include <asm/ptrace.h>
>
> #define arch_get_mmap_end(addr, len, flags) \
>@@ -108,6 +109,9 @@ struct thread_struct {
> struct __riscv_v_ext_state vstate;
> unsigned long align_ctl;
> struct __riscv_v_ext_state kernel_vstate;
>+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>+ struct perf_event *ptrace_bps[RV_MAX_TRIGGERS];
>+#endif
> #ifdef CONFIG_SMP
> /* Flush the icache on migration */
> bool force_icache_flush;
>diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
>index a38268b19c3d..a7998ed41913 100644
>--- a/arch/riscv/include/uapi/asm/ptrace.h
>+++ b/arch/riscv/include/uapi/asm/ptrace.h
>@@ -14,7 +14,8 @@
>
> #define PTRACE_GETFDPIC_EXEC 0
> #define PTRACE_GETFDPIC_INTERP 1
>-
>+#define PTRACE_GETHBPREGS 2
>+#define PTRACE_SETHBPREGS 3
Why not use `PTRACE_GETREGSET` `PTRACE_SETREGSET` ?
> /*
> * User-mode register state for core dumps, ptrace, sigcontext
> *
>diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
>index 437fd82b9590..c58145464539 100644
>--- a/arch/riscv/kernel/hw_breakpoint.c
>+++ b/arch/riscv/kernel/hw_breakpoint.c
>@@ -633,7 +633,19 @@ void arch_uninstall_hw_breakpoint(struct perf_event *event)
> pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> }
>
>-void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
>+/*
>+ * Release the user breakpoints used by ptrace
>+ */
>+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
>+{
>+ int i;
>+ struct thread_struct *t = &tsk->thread;
>+
>+ for (i = 0; i < dbtr_total_num; i++) {
>+ unregister_hw_breakpoint(t->ptrace_bps[i]);
>+ t->ptrace_bps[i] = NULL;
>+ }
>+}
>
> void hw_breakpoint_pmu_read(struct perf_event *bp) { }
>
>diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
>index 15d8f75902f8..9cf07ecfb523 100644
>--- a/arch/riscv/kernel/process.c
>+++ b/arch/riscv/kernel/process.c
>@@ -9,6 +9,7 @@
>
> #include <linux/bitfield.h>
> #include <linux/cpu.h>
>+#include <linux/hw_breakpoint.h>
> #include <linux/kernel.h>
> #include <linux/sched.h>
> #include <linux/sched/debug.h>
>@@ -164,6 +165,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
>
> void flush_thread(void)
> {
>+ flush_ptrace_hw_breakpoint(current);
> #ifdef CONFIG_FPU
> /*
> * Reset FPU state and context
>@@ -218,6 +220,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> set_bit(MM_CONTEXT_LOCK_PMLEN, &p->mm->context.flags);
>
> memset(&p->thread.s, 0, sizeof(p->thread.s));
>+ if (IS_ENABLED(CONFIG_HAVE_HW_BREAKPOINT))
>+ memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
>
> /* p->thread holds context to be restored by __switch_to() */
> if (unlikely(args->fn)) {
>diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
>index ea67e9fb7a58..b78cfb0f1c0e 100644
>--- a/arch/riscv/kernel/ptrace.c
>+++ b/arch/riscv/kernel/ptrace.c
>@@ -9,11 +9,13 @@
>
> #include <asm/vector.h>
> #include <asm/ptrace.h>
>+#include <asm/hw_breakpoint.h>
> #include <asm/syscall.h>
> #include <asm/thread_info.h>
> #include <asm/switch_to.h>
> #include <linux/audit.h>
> #include <linux/compat.h>
>+#include <linux/hw_breakpoint.h>
> #include <linux/ptrace.h>
> #include <linux/elf.h>
> #include <linux/regset.h>
>@@ -336,12 +338,103 @@ void ptrace_disable(struct task_struct *child)
> {
> }
>
>+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>+static void ptrace_hbptriggered(struct perf_event *bp,
>+ struct perf_sample_data *data,
>+ struct pt_regs *regs)
>+{
>+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
>+ int num = 0;
>+
>+ force_sig_ptrace_errno_trap(num, (void __user *)bkpt->address);
>+}
>+
>+/*
>+ * idx selects the breakpoint index.
>+ * Both PTRACE_GETHBPREGS and PTRACE_SETHBPREGS transfer three 32-bit words:
>+ * address (0), length (1), type (2).
>+ * Instruction breakpoint length is one of HW_BREAKPOINT_LEN_X or 0. 0 will
>+ * disable the breakpoint.
>+ * Instruction breakpoint type is one of HW_BREAKPOINT_X.
>+ */
>+
>+static long ptrace_gethbpregs(struct task_struct *child, unsigned long idx,
>+ unsigned long __user *datap)
>+{
>+ struct perf_event *bp;
>+ unsigned long user_data[3] = {0};
>+
>+ if (idx >= RV_MAX_TRIGGERS)
>+ return -EINVAL;
>+
>+ bp = child->thread.ptrace_bps[idx];
>+
>+ if (!IS_ERR_OR_NULL(bp)) {
>+ user_data[0] = bp->attr.bp_addr;
>+ user_data[1] = bp->attr.disabled ? 0 : bp->attr.bp_len;
>+ user_data[2] = bp->attr.bp_type;
>+ }
>+
>+ if (copy_to_user(datap, user_data, sizeof(user_data)))
>+ return -EFAULT;
>+
>+ return 0;
>+}
>+
>+static long ptrace_sethbpregs(struct task_struct *child, unsigned long idx,
>+ unsigned long __user *datap)
>+{
>+ struct perf_event *bp;
>+ struct perf_event_attr attr;
>+ unsigned long user_data[3];
>+
>+ if (idx >= RV_MAX_TRIGGERS)
>+ return -EINVAL;
>+
>+ if (copy_from_user(user_data, datap, sizeof(user_data)))
>+ return -EFAULT;
>+
>+ bp = child->thread.ptrace_bps[idx];
>+ if (IS_ERR_OR_NULL(bp))
Why not only check for NULL?
IS_ERR_VALUE will always expand to be true. right?
>+ attr = bp->attr;
>+ else
>+ ptrace_breakpoint_init(&attr);
>+
>+ attr.bp_addr = user_data[0];
>+ attr.bp_len = user_data[1];
>+ attr.bp_type = user_data[2];
>+ attr.disabled = !attr.bp_len;
Is it okay to not have any sanitization on inputs?
Can these inputs be controlled by user to give kernel address and kernel
breakpoint?
>+
>+ if (IS_ERR_OR_NULL(bp)) {
>+ bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL,
>+ child);
>+ if (IS_ERR(bp))
>+ return PTR_ERR(bp);
>+
>+ child->thread.ptrace_bps[idx] = bp;
>+ return 0;
>+ } else {
>+ return modify_user_hw_breakpoint(bp, &attr);
>+ }
>+}
>+#endif
>+
> long arch_ptrace(struct task_struct *child, long request,
> unsigned long addr, unsigned long data)
> {
> long ret = -EIO;
>+ unsigned long __user *datap = (unsigned long __user *) data;
>
> switch (request) {
>+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>+ case PTRACE_GETHBPREGS:
>+ ret = ptrace_gethbpregs(child, addr, datap);
>+ break;
>+
>+ case PTRACE_SETHBPREGS:
>+ ret = ptrace_sethbpregs(child, addr, datap);
>+ break;
>+#endif
> default:
> ret = ptrace_request(child, request, addr, data);
> break;
>--
>2.43.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support
2025-07-23 4:18 ` Deepak Gupta
@ 2025-07-23 16:55 ` Jesse Taube
2025-07-23 17:23 ` Deepak Gupta
0 siblings, 1 reply; 18+ messages in thread
From: Jesse Taube @ 2025-07-23 16:55 UTC (permalink / raw)
To: Deepak Gupta
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Tue, Jul 22, 2025 at 9:18 PM Deepak Gupta <debug@rivosinc.com> wrote:
>
> On Tue, Jul 22, 2025 at 10:38:29AM -0700, Jesse Taube wrote:
> >Add ability to setup hw breakpoints to ptrace. Call defines a new
> >structure of (ulong[3]){bp_addr, bp_len, bp_type} with
> >bp_type being one of HW_BREAKPOINT_LEN_X and
> >bp_len being one of HW_BREAKPOINT_X with a value of
> >zero dissabling the breakpoint.
> >
> >Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> >---
> > arch/riscv/include/asm/processor.h | 4 ++
> > arch/riscv/include/uapi/asm/ptrace.h | 3 +-
> > arch/riscv/kernel/hw_breakpoint.c | 14 ++++-
> > arch/riscv/kernel/process.c | 4 ++
> > arch/riscv/kernel/ptrace.c | 93 ++++++++++++++++++++++++++++
> > 5 files changed, 116 insertions(+), 2 deletions(-)
> >
> >diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> >index 5f56eb9d114a..488d956a951f 100644
> >--- a/arch/riscv/include/asm/processor.h
> >+++ b/arch/riscv/include/asm/processor.h
> >@@ -12,6 +12,7 @@
> >
> > #include <vdso/processor.h>
> >
> >+#include <asm/hw_breakpoint.h>
> > #include <asm/ptrace.h>
> >
> > #define arch_get_mmap_end(addr, len, flags) \
> >@@ -108,6 +109,9 @@ struct thread_struct {
> > struct __riscv_v_ext_state vstate;
> > unsigned long align_ctl;
> > struct __riscv_v_ext_state kernel_vstate;
> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
> >+ struct perf_event *ptrace_bps[RV_MAX_TRIGGERS];
> >+#endif
> > #ifdef CONFIG_SMP
> > /* Flush the icache on migration */
> > bool force_icache_flush;
> >diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
> >index a38268b19c3d..a7998ed41913 100644
> >--- a/arch/riscv/include/uapi/asm/ptrace.h
> >+++ b/arch/riscv/include/uapi/asm/ptrace.h
> >@@ -14,7 +14,8 @@
> >
> > #define PTRACE_GETFDPIC_EXEC 0
> > #define PTRACE_GETFDPIC_INTERP 1
> >-
> >+#define PTRACE_GETHBPREGS 2
> >+#define PTRACE_SETHBPREGS 3
>
> Why not use `PTRACE_GETREGSET` `PTRACE_SETREGSET` ?
Because it was easier to implement this first, and REGSET will be
another commit ontop of this one.
Unless there is a reason to not have this version.
>
> > /*
> > * User-mode register state for core dumps, ptrace, sigcontext
> > *
> >diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> >index 437fd82b9590..c58145464539 100644
> >--- a/arch/riscv/kernel/hw_breakpoint.c
> >+++ b/arch/riscv/kernel/hw_breakpoint.c
> >@@ -633,7 +633,19 @@ void arch_uninstall_hw_breakpoint(struct perf_event *event)
> > pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> > }
> >
> >-void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
> >+/*
> >+ * Release the user breakpoints used by ptrace
> >+ */
> >+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
> >+{
> >+ int i;
> >+ struct thread_struct *t = &tsk->thread;
> >+
> >+ for (i = 0; i < dbtr_total_num; i++) {
> >+ unregister_hw_breakpoint(t->ptrace_bps[i]);
> >+ t->ptrace_bps[i] = NULL;
> >+ }
> >+}
> >
> > void hw_breakpoint_pmu_read(struct perf_event *bp) { }
> >
> >diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> >index 15d8f75902f8..9cf07ecfb523 100644
> >--- a/arch/riscv/kernel/process.c
> >+++ b/arch/riscv/kernel/process.c
> >@@ -9,6 +9,7 @@
> >
> > #include <linux/bitfield.h>
> > #include <linux/cpu.h>
> >+#include <linux/hw_breakpoint.h>
> > #include <linux/kernel.h>
> > #include <linux/sched.h>
> > #include <linux/sched/debug.h>
> >@@ -164,6 +165,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
> >
> > void flush_thread(void)
> > {
> >+ flush_ptrace_hw_breakpoint(current);
> > #ifdef CONFIG_FPU
> > /*
> > * Reset FPU state and context
> >@@ -218,6 +220,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> > set_bit(MM_CONTEXT_LOCK_PMLEN, &p->mm->context.flags);
> >
> > memset(&p->thread.s, 0, sizeof(p->thread.s));
> >+ if (IS_ENABLED(CONFIG_HAVE_HW_BREAKPOINT))
> >+ memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
> >
> > /* p->thread holds context to be restored by __switch_to() */
> > if (unlikely(args->fn)) {
> >diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
> >index ea67e9fb7a58..b78cfb0f1c0e 100644
> >--- a/arch/riscv/kernel/ptrace.c
> >+++ b/arch/riscv/kernel/ptrace.c
> >@@ -9,11 +9,13 @@
> >
> > #include <asm/vector.h>
> > #include <asm/ptrace.h>
> >+#include <asm/hw_breakpoint.h>
> > #include <asm/syscall.h>
> > #include <asm/thread_info.h>
> > #include <asm/switch_to.h>
> > #include <linux/audit.h>
> > #include <linux/compat.h>
> >+#include <linux/hw_breakpoint.h>
> > #include <linux/ptrace.h>
> > #include <linux/elf.h>
> > #include <linux/regset.h>
> >@@ -336,12 +338,103 @@ void ptrace_disable(struct task_struct *child)
> > {
> > }
> >
> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
> >+static void ptrace_hbptriggered(struct perf_event *bp,
> >+ struct perf_sample_data *data,
> >+ struct pt_regs *regs)
> >+{
> >+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
> >+ int num = 0;
> >+
> >+ force_sig_ptrace_errno_trap(num, (void __user *)bkpt->address);
> >+}
> >+
> >+/*
> >+ * idx selects the breakpoint index.
> >+ * Both PTRACE_GETHBPREGS and PTRACE_SETHBPREGS transfer three 32-bit words:
> >+ * address (0), length (1), type (2).
> >+ * Instruction breakpoint length is one of HW_BREAKPOINT_LEN_X or 0. 0 will
> >+ * disable the breakpoint.
> >+ * Instruction breakpoint type is one of HW_BREAKPOINT_X.
> >+ */
> >+
> >+static long ptrace_gethbpregs(struct task_struct *child, unsigned long idx,
> >+ unsigned long __user *datap)
> >+{
> >+ struct perf_event *bp;
> >+ unsigned long user_data[3] = {0};
> >+
> >+ if (idx >= RV_MAX_TRIGGERS)
> >+ return -EINVAL;
> >+
> >+ bp = child->thread.ptrace_bps[idx];
> >+
> >+ if (!IS_ERR_OR_NULL(bp)) {
> >+ user_data[0] = bp->attr.bp_addr;
> >+ user_data[1] = bp->attr.disabled ? 0 : bp->attr.bp_len;
> >+ user_data[2] = bp->attr.bp_type;
> >+ }
> >+
> >+ if (copy_to_user(datap, user_data, sizeof(user_data)))
> >+ return -EFAULT;
> >+
> >+ return 0;
> >+}
> >+
> >+static long ptrace_sethbpregs(struct task_struct *child, unsigned long idx,
> >+ unsigned long __user *datap)
> >+{
> >+ struct perf_event *bp;
> >+ struct perf_event_attr attr;
> >+ unsigned long user_data[3];
> >+
> >+ if (idx >= RV_MAX_TRIGGERS)
> >+ return -EINVAL;
> >+
> >+ if (copy_from_user(user_data, datap, sizeof(user_data)))
> >+ return -EFAULT;
> >+
> >+ bp = child->thread.ptrace_bps[idx];
> >+ if (IS_ERR_OR_NULL(bp))
>
> Why not only check for NULL?
> IS_ERR_VALUE will always expand to be true. right?
Because im dumb and thought i was setting bp to an error code, but i'm not.
Yes if (!bp) is right.
>
> >+ attr = bp->attr;
> >+ else
> >+ ptrace_breakpoint_init(&attr);
> >+
> >+ attr.bp_addr = user_data[0];
> >+ attr.bp_len = user_data[1];
> >+ attr.bp_type = user_data[2];
> >+ attr.disabled = !attr.bp_len;
>
> Is it okay to not have any sanitization on inputs?
>
> Can these inputs be controlled by user to give kernel address and kernel
> breakpoint?
modify_user_hw_breakpoint calls modify_user_hw_breakpoint_check, which
eventually checks if we have CAP_SYS_ADMIN.
Same for register. type and len are also checked by the
_user_hw_breakpoint_check functions and again in the riscv code.
it would be nice if this could be double checked, but it does seem
other architectures don't check addr aswell.
Thanks,
Jesse Taube
>
> >+
> >+ if (IS_ERR_OR_NULL(bp)) {
> >+ bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL,
> >+ child);
> >+ if (IS_ERR(bp))
> >+ return PTR_ERR(bp);
> >+
> >+ child->thread.ptrace_bps[idx] = bp;
> >+ return 0;
> >+ } else {
> >+ return modify_user_hw_breakpoint(bp, &attr);
> >+ }
> >+}
> >+#endif
> >+
> > long arch_ptrace(struct task_struct *child, long request,
> > unsigned long addr, unsigned long data)
> > {
> > long ret = -EIO;
> >+ unsigned long __user *datap = (unsigned long __user *) data;
> >
> > switch (request) {
> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
> >+ case PTRACE_GETHBPREGS:
> >+ ret = ptrace_gethbpregs(child, addr, datap);
> >+ break;
> >+
> >+ case PTRACE_SETHBPREGS:
> >+ ret = ptrace_sethbpregs(child, addr, datap);
> >+ break;
> >+#endif
> > default:
> > ret = ptrace_request(child, request, addr, data);
> > break;
> >--
> >2.43.0
> >
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-07-23 2:49 ` Deepak Gupta
@ 2025-07-23 17:02 ` Jesse Taube
2025-07-23 17:25 ` Deepak Gupta
0 siblings, 1 reply; 18+ messages in thread
From: Jesse Taube @ 2025-07-23 17:02 UTC (permalink / raw)
To: Deepak Gupta
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Tue, Jul 22, 2025 at 7:49 PM Deepak Gupta <debug@rivosinc.com> wrote:
>
> On Tue, Jul 22, 2025 at 10:38:27AM -0700, Jesse Taube wrote:
> >From: Himanshu Chauhan <hchauhan@ventanamicro.com>
> >
> >RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
> >SBI debug trigger extension to install/uninstall/update/enable/disable hardware
> >triggers as specified in Sdtrig ISA extension.
> >
> >Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
> >Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> >---
> > arch/riscv/Kconfig | 1 +
> > arch/riscv/include/asm/hw_breakpoint.h | 60 +++
> > arch/riscv/include/asm/kdebug.h | 3 +-
> > arch/riscv/include/asm/sbi.h | 4 +-
> > arch/riscv/kernel/Makefile | 1 +
> > arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
> > arch/riscv/kernel/traps.c | 6 +
> > 7 files changed, 693 insertions(+), 2 deletions(-)
> > create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
> > create mode 100644 arch/riscv/kernel/hw_breakpoint.c
> >
> >diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >index bbec87b79309..95d3047cab10 100644
> >--- a/arch/riscv/Kconfig
> >+++ b/arch/riscv/Kconfig
> >@@ -163,6 +163,7 @@ config RISCV
> > select HAVE_FUNCTION_ERROR_INJECTION
> > select HAVE_GCC_PLUGINS
> > select HAVE_GENERIC_VDSO if MMU && 64BIT
> >+ select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
> > select HAVE_IRQ_TIME_ACCOUNTING
> > select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
> > select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
> >diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
> >new file mode 100644
> >index 000000000000..8efa3921c535
> >--- /dev/null
> >+++ b/arch/riscv/include/asm/hw_breakpoint.h
> >@@ -0,0 +1,60 @@
> >+/* SPDX-License-Identifier: GPL-2.0-only */
> >+/*
> >+ * Copyright (C) 2024 Ventana Micro Systems Inc.
> >+ */
> >+
> >+#ifndef __RISCV_HW_BREAKPOINT_H
> >+#define __RISCV_HW_BREAKPOINT_H
> >+
> >+struct task_struct;
> >+
> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
> >+
> >+#include <uapi/linux/hw_breakpoint.h>
> >+
> >+#if __riscv_xlen == 64
> >+#define cpu_to_le cpu_to_le64
> >+#define le_to_cpu le64_to_cpu
> >+#elif __riscv_xlen == 32
> >+#define cpu_to_le cpu_to_le32
> >+#define le_to_cpu le32_to_cpu
> >+#else
> >+#error "Unexpected __riscv_xlen"
> >+#endif
> >+
> >+struct arch_hw_breakpoint {
> >+ unsigned long address;
> >+ unsigned long len;
> >+
> >+ /* Callback info */
> >+ unsigned long next_addr;
> >+ bool in_callback;
> >+
> >+
> >+ /* Trigger configuration data */
> >+ unsigned long tdata1;
> >+ unsigned long tdata2;
> >+ unsigned long tdata3;
> >+};
> >+
> >+/* Maximum number of hardware breakpoints supported */
> >+#define RV_MAX_TRIGGERS 32
> >+
> >+struct perf_event_attr;
> >+struct notifier_block;
> >+struct perf_event;
> >+struct pt_regs;
> >+
> >+int hw_breakpoint_slots(int type);
> >+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
> >+int hw_breakpoint_arch_parse(struct perf_event *bp,
> >+ const struct perf_event_attr *attr,
> >+ struct arch_hw_breakpoint *hw);
> >+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> >+ unsigned long val, void *data);
> >+int arch_install_hw_breakpoint(struct perf_event *bp);
> >+void arch_uninstall_hw_breakpoint(struct perf_event *bp);
> >+void hw_breakpoint_pmu_read(struct perf_event *bp);
> >+
> >+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
> >+#endif /* __RISCV_HW_BREAKPOINT_H */
> >diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
> >index 85ac00411f6e..53e989781aa1 100644
> >--- a/arch/riscv/include/asm/kdebug.h
> >+++ b/arch/riscv/include/asm/kdebug.h
> >@@ -6,7 +6,8 @@
> > enum die_val {
> > DIE_UNUSED,
> > DIE_TRAP,
> >- DIE_OOPS
> >+ DIE_OOPS,
> >+ DIE_DEBUG
> > };
> >
> > #endif
> >diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> >index be2ca8e8a49e..64fa7a82aa45 100644
> >--- a/arch/riscv/include/asm/sbi.h
> >+++ b/arch/riscv/include/asm/sbi.h
> >@@ -282,7 +282,9 @@ struct sbi_sta_struct {
> > u8 pad[47];
> > } __packed;
> >
> >-#define SBI_SHMEM_DISABLE -1
> >+#define SBI_SHMEM_DISABLE (-1UL)
> >+#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
> >+#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
> >
> > enum sbi_ext_nacl_fid {
> > SBI_EXT_NACL_PROBE_FEATURE = 0x0,
> >diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> >index 4f719b09e5ad..3e72505734bd 100644
> >--- a/arch/riscv/kernel/Makefile
> >+++ b/arch/riscv/kernel/Makefile
> >@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
> >
> > obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
> > obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
> >+obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
> > obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
> > ifeq ($(CONFIG_RISCV_SBI), y)
> > obj-$(CONFIG_SMP) += sbi-ipi.o
> >diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> >new file mode 100644
> >index 000000000000..9e3a3b82d300
> >--- /dev/null
> >+++ b/arch/riscv/kernel/hw_breakpoint.c
> >@@ -0,0 +1,620 @@
> >+// SPDX-License-Identifier: GPL-2.0-only
> >+/*
> >+ * Copyright (C) 2024 Ventana Micro Systems Inc.
> >+ */
> >+
> >+#include <linux/hw_breakpoint.h>
> >+#include <linux/perf_event.h>
> >+#include <linux/spinlock.h>
> >+#include <linux/percpu.h>
> >+#include <linux/kdebug.h>
> >+#include <linux/bitops.h>
> >+#include <linux/bitfield.h>
> >+#include <linux/cpu.h>
> >+#include <linux/cpuhotplug.h>
> >+
> >+#include <asm/insn.h>
> >+#include <asm/sbi.h>
> >+
> >+#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
> >+#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
> >+
> >+#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
> >+#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
> >+
> >+#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
> >+#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
> >+#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
> >+#define DBTR_TDATA1_MCONTROL6_U BIT(3)
> >+#define DBTR_TDATA1_MCONTROL6_S BIT(4)
> >+#define DBTR_TDATA1_MCONTROL6_M BIT(6)
> >+#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
> >+#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
> >+#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
> >+#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
> >+
> >+#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
> >+#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
> >+#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
> >+#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
> >+
> >+#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
> >+#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
> >+#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
> >+#define DBTR_TDATA1_MCONTROL_U BIT(3)
> >+#define DBTR_TDATA1_MCONTROL_S BIT(4)
> >+#define DBTR_TDATA1_MCONTROL_M BIT(6)
> >+#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
> >+#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
> >+#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
> >+
> >+#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
> >+#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
> >+#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
> >+/* value of 5 split across HI and LO */
> >+#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
> >+#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
> >+
> >+/* Registered per-cpu bp/wp */
> >+static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
> >+static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
> >+static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
> >+
> >+/* Per-cpu shared memory between S and M mode */
> >+static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
> >+
> >+/* number of debug triggers on this cpu . */
> >+static int dbtr_total_num __ro_after_init;
> >+static unsigned long dbtr_type __ro_after_init;
> >+static unsigned long dbtr_init __ro_after_init;
> >+
> >+static int arch_smp_setup_sbi_shmem(unsigned int cpu)
> >+{
> >+ union sbi_dbtr_shmem_entry *dbtr_shmem;
> >+ unsigned long shmem_pa;
> >+ struct sbiret ret;
> >+ int rc;
> >+
> >+ dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
> >+ if (!dbtr_shmem) {
> >+ pr_err("Invalid per-cpu shared memory for debug triggers\n");
> >+ return -ENODEV;
> >+ }
> >+
> >+ shmem_pa = virt_to_phys(dbtr_shmem);
> >+
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> >+ SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
> >+
> >+ if (ret.error) {
> >+ switch (ret.error) {
> >+ case SBI_ERR_DENIED:
> >+ pr_warn("%s: Access denied for shared memory at %lx\n",
> >+ __func__, shmem_pa);
> >+ rc = -EPERM;
> >+ break;
> >+
> >+ case SBI_ERR_INVALID_PARAM:
> >+ case SBI_ERR_INVALID_ADDRESS:
> >+ pr_warn("%s: Invalid address parameter (%ld)\n",
> >+ __func__, ret.error);
> >+ rc = -EINVAL;
> >+ break;
> >+
> >+ case SBI_ERR_ALREADY_AVAILABLE:
> >+ pr_warn("%s: Shared memory is already set\n",
> >+ __func__);
> >+ rc = -EADDRINUSE;
> >+ break;
> >+
> >+ case SBI_ERR_FAILURE:
> >+ pr_err("%s: Internal sdtrig state error\n",
> >+ __func__);
> >+ rc = -ENXIO;
> >+ break;
> >+
> >+ default:
> >+ pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
> >+ rc = -ENXIO;
> >+ break;
> >+ }
> >+ }
> >+
> >+ pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
> >+
> >+ return rc;
> >+}
> >+
> >+static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
> >+{
> >+ struct sbiret ret;
> >+
> >+ /* Disable shared memory */
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> >+ SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
> >+
> >+ if (ret.error) {
> >+ switch (ret.error) {
> >+ case SBI_ERR_DENIED:
> >+ pr_err("%s: Access denied for shared memory.\n",
> >+ __func__);
> >+ break;
> >+
> >+ case SBI_ERR_INVALID_PARAM:
> >+ case SBI_ERR_INVALID_ADDRESS:
> >+ pr_err("%s: Invalid address parameter (%lu)\n",
> >+ __func__, ret.error);
> >+ break;
> >+
> >+ case SBI_ERR_ALREADY_AVAILABLE:
> >+ pr_err("%s: Shared memory is already set\n",
> >+ __func__);
> >+ break;
> >+ case SBI_ERR_FAILURE:
> >+ pr_err("%s: Internal sdtrig state error\n",
> >+ __func__);
> >+ break;
> >+ default:
> >+ pr_err("%s: Unknown error %lu\n", __func__, ret.error);
> >+ break;
> >+ }
> >+ }
> >+
> >+ pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
> >+
> >+ return 0;
> >+}
> >+
> >+static void init_sbi_dbtr(void)
> >+{
> >+ struct sbiret ret;
> >+
> >+ /*
> >+ * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
> >+ * Only proceed if this is the first CPU to reach this code.
> >+ */
> >+ if (test_and_set_bit(0, &dbtr_init))
> >+ return;
> >+
> >+ if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
> >+ pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
> >+ dbtr_total_num = 0;
> >+ return;
> >+ }
> >+
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> >+ DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
> >+ if (ret.error) {
> >+ pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
> >+ __func__, ret.error);
> >+ } else if (!ret.value) {
> >+ pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
> >+ } else {
> >+ dbtr_total_num = ret.value;
> >+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
> >+ return;
> >+ }
> >+
> >+ /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> >+ DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
> >+ if (ret.error) {
> >+ pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
> >+ __func__, ret.error);
> >+ } else if (!ret.value) {
> >+ pr_err("%s: No mcontrol triggers available.\n", __func__);
> >+ dbtr_total_num = 0;
> >+ } else {
> >+ dbtr_total_num = ret.value;
> >+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
> >+ }
> >+}
>
> `dbtr_total_num` is static and thus would be initialized to zero.
> Although above code is setting it to zero in some error conditions.
> And not setting to zero in other error conditions.
>
> It'll be better if function starts out setting `dbtr_total_num` as setting to 0.
> Then set correct value in success conditions.
I would do that, but `dbtr_total_num` is marked as `__ro_after_init`
and I don't want to
write to it twice, and i wanted to explicitly set it to zero as that
means breakpoints are disabled.
I can just remove the assignments to zero though.
>
> >+
> >+int hw_breakpoint_slots(int type)
> >+{
> >+ /*
> >+ * We can be called early, so don't rely on
> >+ * static variables being initialised.
> >+ */
> >+ init_sbi_dbtr();
> >+
> >+ return dbtr_total_num;
> >+}
> >+
> >+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
> >+{
> >+ unsigned int len;
> >+ unsigned long va;
> >+
> >+ va = hw->address;
> >+ len = hw->len;
> >+
> >+ return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
> >+}
> >+
> >+static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
> >+ struct arch_hw_breakpoint *hw)
> >+{
> >+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> >+
> >+ switch (attr->bp_type) {
> >+ case HW_BREAKPOINT_X:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
> >+ break;
> >+ case HW_BREAKPOINT_R:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
> >+ break;
> >+ case HW_BREAKPOINT_W:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
> >+ break;
> >+ case HW_BREAKPOINT_RW:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
> >+ break;
> >+ default:
> >+ return -EINVAL;
> >+ }
> >+
> >+ switch (attr->bp_len) {
> >+ case HW_BREAKPOINT_LEN_1:
> >+ hw->len = 1;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> >+ DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
> >+ break;
> >+ case HW_BREAKPOINT_LEN_2:
> >+ hw->len = 2;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> >+ DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
> >+ break;
> >+ case HW_BREAKPOINT_LEN_4:
> >+ hw->len = 4;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> >+ DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
> >+ break;
> >+#if __riscv_xlen >= 64
> >+ case HW_BREAKPOINT_LEN_8:
> >+ hw->len = 8;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> >+ DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
> >+ FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
> >+ DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
> >+ break;
> >+#endif
> >+ default:
> >+ return -EINVAL;
> >+ }
> >+
> >+ tdata1 |= DBTR_TDATA1_MCONTROL_U;
>
> Assuming this function could be re-used for kernel data breakpoints too
> Shouldn't this be based on some flag to select U v/s S.
> Same comment for setting up tdata1 with mcontrol6.
Yes I can add a mode flag, so when kernel breakpoints or virtualized breakpoints
are added it will be easier.
Thanks,
Jesse Taube
>
> >+
> >+ hw->tdata1 = tdata1;
> >+
> >+ return 0;
> >+}
> >+
> >+static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
> >+ struct arch_hw_breakpoint *hw)
> >+{
> >+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> >+
> >+ switch (attr->bp_type) {
> >+ case HW_BREAKPOINT_X:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
> >+ break;
> >+ case HW_BREAKPOINT_R:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
> >+ break;
> >+ case HW_BREAKPOINT_W:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
> >+ break;
> >+ case HW_BREAKPOINT_RW:
> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
> >+ break;
> >+ default:
> >+ return -EINVAL;
> >+ }
> >+
> >+ switch (attr->bp_len) {
> >+ case HW_BREAKPOINT_LEN_1:
> >+ hw->len = 1;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> >+ DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
> >+ break;
> >+ case HW_BREAKPOINT_LEN_2:
> >+ hw->len = 2;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> >+ DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
> >+ break;
> >+ case HW_BREAKPOINT_LEN_4:
> >+ hw->len = 4;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> >+ DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
> >+ break;
> >+ case HW_BREAKPOINT_LEN_8:
> >+ hw->len = 8;
> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> >+ DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
> >+ break;
> >+ default:
> >+ return -EINVAL;
> >+ }
> >+
> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_U;
> >+
> >+ hw->tdata1 = tdata1;
> >+
> >+ return 0;
> >+}
> >+
> >+int hw_breakpoint_arch_parse(struct perf_event *bp,
> >+ const struct perf_event_attr *attr,
> >+ struct arch_hw_breakpoint *hw)
> >+{
> >+ int ret;
> >+
> >+ /* Breakpoint address */
> >+ hw->address = attr->bp_addr;
> >+ hw->tdata2 = attr->bp_addr;
> >+ hw->tdata3 = 0x0;
> >+ hw->next_addr = 0x0;
> >+ hw->in_callback = false;
> >+
> >+ switch (dbtr_type) {
> >+ case DBTR_TDATA1_TYPE_MCONTROL:
> >+ ret = rv_init_mcontrol_trigger(attr, hw);
> >+ break;
> >+ case DBTR_TDATA1_TYPE_MCONTROL6:
> >+ ret = rv_init_mcontrol6_trigger(attr, hw);
> >+ break;
> >+ default:
> >+ pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
> >+ ret = -EOPNOTSUPP;
> >+ break;
> >+ }
> >+
> >+ return ret;
> >+}
> >+
> >+/*
> >+ * Set breakpoint to next insruction after breakpoint.
> >+ * Returns 0 if success
> >+ * Returns < 0 on error
> >+ */
> >+static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
> >+{
> >+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> >+ struct arch_hw_breakpoint old_hw_bp;
> >+ struct perf_event_attr bp_insn;
> >+ unsigned long next_addr, insn;
> >+ int ret;
> >+
> >+ /* Remove breakpoint even if return error as not to loop */
> >+ arch_uninstall_hw_breakpoint(event);
> >+
> >+ ret = get_insn(regs, regs->epc, &insn);
> >+ if (ret < 0)
> >+ return ret;
> >+
> >+ next_addr = get_step_address(regs, insn);
> >+
> >+ ret = get_insn(regs, next_addr, &insn);
> >+ if (ret < 0)
> >+ return ret;
> >+
> >+ bp_insn.bp_type = HW_BREAKPOINT_X;
> >+ bp_insn.bp_addr = next_addr;
> >+ /* Get the size of the intruction */
> >+ bp_insn.bp_len = GET_INSN_LENGTH(insn);
> >+
> >+ ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
> >+ if (ret)
> >+ return ret;
> >+
> >+ ret = arch_install_hw_breakpoint(event);
> >+ if (ret)
> >+ return ret;
> >+
> >+ bp->in_callback = true;
> >+ bp->next_addr = next_addr;
> >+ return 0;
> >+}
> >+
> >+/*
> >+ * HW Breakpoint/watchpoint handler
> >+ */
> >+static int hw_breakpoint_handler(struct pt_regs *regs)
> >+{
> >+ struct perf_event *event;
> >+ struct arch_hw_breakpoint *bp;
> >+ int ret, i;
> >+
> >+ for (i = 0; i < dbtr_total_num; i++) {
> >+ event = this_cpu_read(pcpu_hw_bp_events[i]);
> >+ if (!event)
> >+ continue;
> >+
> >+ bp = counter_arch_bp(event);
> >+ if (bp->in_callback) {
> >+ /* Reset changed breakpoint data */
> >+ bp->in_callback = false;
> >+ if (regs->epc == bp->next_addr) {
> >+ arch_uninstall_hw_breakpoint(event);
> >+ /* Restore original breakpoint */
> >+ if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
> >+ return NOTIFY_DONE;
> >+ if (arch_install_hw_breakpoint(event))
> >+ return NOTIFY_DONE;
> >+ return NOTIFY_STOP;
> >+ }
> >+
> >+ pr_err("%s: in_callback was set, but epc(%lx) was not next "
> >+ "address(%lx).\n", __func__, regs->epc, bp->next_addr);
> >+ bp->next_addr = 0x0;
> >+ return NOTIFY_DONE;
> >+ }
> >+
> >+ switch (event->attr.bp_type) {
> >+ /* Breakpoint */
> >+ case HW_BREAKPOINT_X:
> >+ if (event->attr.bp_addr == regs->epc) {
> >+ ret = setup_singlestep(event, regs);
> >+ if (ret < 0) {
> >+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> >+ return NOTIFY_DONE;
> >+ }
> >+
> >+ perf_bp_event(event, regs);
> >+ return NOTIFY_STOP;
> >+ }
> >+ break;
> >+
> >+ /* Watchpoint */
> >+ case HW_BREAKPOINT_W:
> >+ case HW_BREAKPOINT_R:
> >+ case HW_BREAKPOINT_RW:
> >+ if (event->attr.bp_addr == regs->badaddr) {
> >+ ret = setup_singlestep(event, regs);
> >+ if (ret < 0) {
> >+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> >+ return NOTIFY_DONE;
> >+ }
> >+
> >+ perf_bp_event(event, regs);
> >+ return NOTIFY_STOP;
> >+ }
> >+ break;
> >+
> >+ default:
> >+ pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
> >+ break;
> >+ }
> >+ }
> >+
> >+ return NOTIFY_DONE;
> >+}
> >+
> >+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> >+ unsigned long val, void *data)
> >+{
> >+ struct die_args *args = data;
> >+
> >+ if (val != DIE_DEBUG)
> >+ return NOTIFY_DONE;
> >+
> >+ return hw_breakpoint_handler(args->regs);
> >+}
> >+
> >+/* atomic: counter->ctx->lock is held */
> >+int arch_install_hw_breakpoint(struct perf_event *event)
> >+{
> >+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> >+ union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
> >+ struct sbi_dbtr_data_msg *xmit;
> >+ struct sbi_dbtr_id_msg *recv;
> >+ struct perf_event **slot;
> >+ unsigned long idx;
> >+ struct sbiret ret;
> >+ int err = 0;
> >+
> >+ raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
> >+ *this_cpu_ptr(&ecall_lock_flags));
> >+
> >+ xmit = &shmem->data;
> >+ recv = &shmem->id;
> >+ xmit->tdata1 = cpu_to_le(bp->tdata1);
> >+ xmit->tdata2 = cpu_to_le(bp->tdata2);
> >+ xmit->tdata3 = cpu_to_le(bp->tdata3);
> >+
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
> >+ 1, 0, 0, 0, 0, 0);
> >+
> >+ if (ret.error) {
> >+ pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
> >+ err = sbi_err_map_linux_errno(ret.error);
> >+ goto done;
> >+ }
> >+
> >+ idx = le_to_cpu(recv->idx);
> >+ if (idx >= dbtr_total_num) {
> >+ pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
> >+ err = -EINVAL;
> >+ goto done;
> >+ }
> >+
> >+ slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
> >+ if (*slot) {
> >+ pr_warn("%s: slot %lu is in use\n", __func__, idx);
> >+ err = -EBUSY;
> >+ goto done;
> >+ }
> >+
> >+ pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
> >+
> >+ /* Save the event - to be looked up in handler */
> >+ *slot = event;
> >+
> >+done:
> >+ raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
> >+ *this_cpu_ptr(&ecall_lock_flags));
> >+ return err;
> >+}
> >+
> >+/* atomic: counter->ctx->lock is held */
> >+void arch_uninstall_hw_breakpoint(struct perf_event *event)
> >+{
> >+ struct sbiret ret;
> >+ int i;
> >+
> >+ for (i = 0; i < dbtr_total_num; i++) {
> >+ struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
> >+
> >+ if (*slot == event) {
> >+ *slot = NULL;
> >+ break;
> >+ }
> >+ }
> >+
> >+
> >+ if (i == dbtr_total_num) {
> >+ pr_warn("%s: Breakpoint not installed.\n", __func__);
> >+ return;
> >+ }
> >+
> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
> >+ i, 1, 0, 0, 0, 0);
> >+ if (ret.error)
> >+ pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> >+}
> >+
> >+void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
> >+
> >+void hw_breakpoint_pmu_read(struct perf_event *bp) { }
> >+
> >+static int __init arch_hw_breakpoint_init(void)
> >+{
> >+ unsigned int cpu;
> >+ int rc = 0;
> >+
> >+ for_each_possible_cpu(cpu)
> >+ raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
> >+
> >+ init_sbi_dbtr();
> >+
> >+ if (dbtr_total_num) {
> >+ pr_debug("%s: total number of type %lu triggers: %u\n",
> >+ __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
> >+ } else {
> >+ pr_debug("%s: No hardware triggers available\n", __func__);
> >+ return rc;
> >+ }
> >+
> >+ /* Hotplug handler to register/unregister shared memory with SBI */
> >+ rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> >+ "riscv/hw_breakpoint:prepare",
> >+ arch_smp_setup_sbi_shmem,
> >+ arch_smp_teardown_sbi_shmem);
> >+
> >+ if (rc < 0)
> >+ pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
> >+
> >+ return rc;
> >+}
> >+arch_initcall(arch_hw_breakpoint_init);
> >diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> >index 938a8b841f94..2ac471ec79a8 100644
> >--- a/arch/riscv/kernel/traps.c
> >+++ b/arch/riscv/kernel/traps.c
> >@@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
> > if (probe_breakpoint_handler(regs))
> > return;
> >
> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
> >+ if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
> >+ == NOTIFY_STOP)
> >+ return;
> >+#endif
> >+
> > current->thread.bad_cause = regs->cause;
> >
> > if (user_mode(regs))
> >--
> >2.43.0
> >
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support
2025-07-23 16:55 ` Jesse Taube
@ 2025-07-23 17:23 ` Deepak Gupta
0 siblings, 0 replies; 18+ messages in thread
From: Deepak Gupta @ 2025-07-23 17:23 UTC (permalink / raw)
To: Jesse Taube
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Wed, Jul 23, 2025 at 09:55:25AM -0700, Jesse Taube wrote:
>On Tue, Jul 22, 2025 at 9:18 PM Deepak Gupta <debug@rivosinc.com> wrote:
>>
>> On Tue, Jul 22, 2025 at 10:38:29AM -0700, Jesse Taube wrote:
>> >Add ability to setup hw breakpoints to ptrace. Call defines a new
>> >structure of (ulong[3]){bp_addr, bp_len, bp_type} with
>> >bp_type being one of HW_BREAKPOINT_LEN_X and
>> >bp_len being one of HW_BREAKPOINT_X with a value of
>> >zero dissabling the breakpoint.
>> >
>> >Signed-off-by: Jesse Taube <jesse@rivosinc.com>
>> >---
>> > arch/riscv/include/asm/processor.h | 4 ++
>> > arch/riscv/include/uapi/asm/ptrace.h | 3 +-
>> > arch/riscv/kernel/hw_breakpoint.c | 14 ++++-
>> > arch/riscv/kernel/process.c | 4 ++
>> > arch/riscv/kernel/ptrace.c | 93 ++++++++++++++++++++++++++++
>> > 5 files changed, 116 insertions(+), 2 deletions(-)
>> >
>> >diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
>> >index 5f56eb9d114a..488d956a951f 100644
>> >--- a/arch/riscv/include/asm/processor.h
>> >+++ b/arch/riscv/include/asm/processor.h
>> >@@ -12,6 +12,7 @@
>> >
>> > #include <vdso/processor.h>
>> >
>> >+#include <asm/hw_breakpoint.h>
>> > #include <asm/ptrace.h>
>> >
>> > #define arch_get_mmap_end(addr, len, flags) \
>> >@@ -108,6 +109,9 @@ struct thread_struct {
>> > struct __riscv_v_ext_state vstate;
>> > unsigned long align_ctl;
>> > struct __riscv_v_ext_state kernel_vstate;
>> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>> >+ struct perf_event *ptrace_bps[RV_MAX_TRIGGERS];
>> >+#endif
>> > #ifdef CONFIG_SMP
>> > /* Flush the icache on migration */
>> > bool force_icache_flush;
>> >diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
>> >index a38268b19c3d..a7998ed41913 100644
>> >--- a/arch/riscv/include/uapi/asm/ptrace.h
>> >+++ b/arch/riscv/include/uapi/asm/ptrace.h
>> >@@ -14,7 +14,8 @@
>> >
>> > #define PTRACE_GETFDPIC_EXEC 0
>> > #define PTRACE_GETFDPIC_INTERP 1
>> >-
>> >+#define PTRACE_GETHBPREGS 2
>> >+#define PTRACE_SETHBPREGS 3
>>
>> Why not use `PTRACE_GETREGSET` `PTRACE_SETREGSET` ?
>
>Because it was easier to implement this first, and REGSET will be
>another commit ontop of this one.
>Unless there is a reason to not have this version.
>
Yeah I don't have strong feelings on this. Whatever is best for the tools
(gdb, etc) who will consume this interface, lets do that.
>>
>> > /*
>> > * User-mode register state for core dumps, ptrace, sigcontext
>> > *
>> >diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
>> >index 437fd82b9590..c58145464539 100644
>> >--- a/arch/riscv/kernel/hw_breakpoint.c
>> >+++ b/arch/riscv/kernel/hw_breakpoint.c
>> >@@ -633,7 +633,19 @@ void arch_uninstall_hw_breakpoint(struct perf_event *event)
>> > pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
>> > }
>> >
>> >-void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
>> >+/*
>> >+ * Release the user breakpoints used by ptrace
>> >+ */
>> >+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
>> >+{
>> >+ int i;
>> >+ struct thread_struct *t = &tsk->thread;
>> >+
>> >+ for (i = 0; i < dbtr_total_num; i++) {
>> >+ unregister_hw_breakpoint(t->ptrace_bps[i]);
>> >+ t->ptrace_bps[i] = NULL;
>> >+ }
>> >+}
>> >
>> > void hw_breakpoint_pmu_read(struct perf_event *bp) { }
>> >
>> >diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
>> >index 15d8f75902f8..9cf07ecfb523 100644
>> >--- a/arch/riscv/kernel/process.c
>> >+++ b/arch/riscv/kernel/process.c
>> >@@ -9,6 +9,7 @@
>> >
>> > #include <linux/bitfield.h>
>> > #include <linux/cpu.h>
>> >+#include <linux/hw_breakpoint.h>
>> > #include <linux/kernel.h>
>> > #include <linux/sched.h>
>> > #include <linux/sched/debug.h>
>> >@@ -164,6 +165,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
>> >
>> > void flush_thread(void)
>> > {
>> >+ flush_ptrace_hw_breakpoint(current);
>> > #ifdef CONFIG_FPU
>> > /*
>> > * Reset FPU state and context
>> >@@ -218,6 +220,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
>> > set_bit(MM_CONTEXT_LOCK_PMLEN, &p->mm->context.flags);
>> >
>> > memset(&p->thread.s, 0, sizeof(p->thread.s));
>> >+ if (IS_ENABLED(CONFIG_HAVE_HW_BREAKPOINT))
>> >+ memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
>> >
>> > /* p->thread holds context to be restored by __switch_to() */
>> > if (unlikely(args->fn)) {
>> >diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
>> >index ea67e9fb7a58..b78cfb0f1c0e 100644
>> >--- a/arch/riscv/kernel/ptrace.c
>> >+++ b/arch/riscv/kernel/ptrace.c
>> >@@ -9,11 +9,13 @@
>> >
>> > #include <asm/vector.h>
>> > #include <asm/ptrace.h>
>> >+#include <asm/hw_breakpoint.h>
>> > #include <asm/syscall.h>
>> > #include <asm/thread_info.h>
>> > #include <asm/switch_to.h>
>> > #include <linux/audit.h>
>> > #include <linux/compat.h>
>> >+#include <linux/hw_breakpoint.h>
>> > #include <linux/ptrace.h>
>> > #include <linux/elf.h>
>> > #include <linux/regset.h>
>> >@@ -336,12 +338,103 @@ void ptrace_disable(struct task_struct *child)
>> > {
>> > }
>> >
>> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>> >+static void ptrace_hbptriggered(struct perf_event *bp,
>> >+ struct perf_sample_data *data,
>> >+ struct pt_regs *regs)
>> >+{
>> >+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
>> >+ int num = 0;
>> >+
>> >+ force_sig_ptrace_errno_trap(num, (void __user *)bkpt->address);
>> >+}
>> >+
>> >+/*
>> >+ * idx selects the breakpoint index.
>> >+ * Both PTRACE_GETHBPREGS and PTRACE_SETHBPREGS transfer three 32-bit words:
>> >+ * address (0), length (1), type (2).
>> >+ * Instruction breakpoint length is one of HW_BREAKPOINT_LEN_X or 0. 0 will
>> >+ * disable the breakpoint.
>> >+ * Instruction breakpoint type is one of HW_BREAKPOINT_X.
>> >+ */
>> >+
>> >+static long ptrace_gethbpregs(struct task_struct *child, unsigned long idx,
>> >+ unsigned long __user *datap)
>> >+{
>> >+ struct perf_event *bp;
>> >+ unsigned long user_data[3] = {0};
>> >+
>> >+ if (idx >= RV_MAX_TRIGGERS)
>> >+ return -EINVAL;
>> >+
>> >+ bp = child->thread.ptrace_bps[idx];
>> >+
>> >+ if (!IS_ERR_OR_NULL(bp)) {
>> >+ user_data[0] = bp->attr.bp_addr;
>> >+ user_data[1] = bp->attr.disabled ? 0 : bp->attr.bp_len;
>> >+ user_data[2] = bp->attr.bp_type;
>> >+ }
>> >+
>> >+ if (copy_to_user(datap, user_data, sizeof(user_data)))
>> >+ return -EFAULT;
>> >+
>> >+ return 0;
>> >+}
>> >+
>> >+static long ptrace_sethbpregs(struct task_struct *child, unsigned long idx,
>> >+ unsigned long __user *datap)
>> >+{
>> >+ struct perf_event *bp;
>> >+ struct perf_event_attr attr;
>> >+ unsigned long user_data[3];
>> >+
>> >+ if (idx >= RV_MAX_TRIGGERS)
>> >+ return -EINVAL;
>> >+
>> >+ if (copy_from_user(user_data, datap, sizeof(user_data)))
>> >+ return -EFAULT;
>> >+
>> >+ bp = child->thread.ptrace_bps[idx];
>> >+ if (IS_ERR_OR_NULL(bp))
>>
>> Why not only check for NULL?
>> IS_ERR_VALUE will always expand to be true. right?
>
>Because im dumb and thought i was setting bp to an error code, but i'm not.
>Yes if (!bp) is right.
>
>>
>> >+ attr = bp->attr;
>> >+ else
>> >+ ptrace_breakpoint_init(&attr);
>> >+
>> >+ attr.bp_addr = user_data[0];
>> >+ attr.bp_len = user_data[1];
>> >+ attr.bp_type = user_data[2];
>> >+ attr.disabled = !attr.bp_len;
>>
>> Is it okay to not have any sanitization on inputs?
>>
>> Can these inputs be controlled by user to give kernel address and kernel
>> breakpoint?
>
>modify_user_hw_breakpoint calls modify_user_hw_breakpoint_check, which
>eventually checks if we have CAP_SYS_ADMIN.
>Same for register. type and len are also checked by the
>_user_hw_breakpoint_check functions and again in the riscv code.
>
>it would be nice if this could be double checked, but it does seem
>other architectures don't check addr aswell.
I see. That's interesting.
>
>Thanks,
>Jesse Taube
>
>>
>> >+
>> >+ if (IS_ERR_OR_NULL(bp)) {
>> >+ bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL,
>> >+ child);
>> >+ if (IS_ERR(bp))
>> >+ return PTR_ERR(bp);
>> >+
>> >+ child->thread.ptrace_bps[idx] = bp;
>> >+ return 0;
>> >+ } else {
>> >+ return modify_user_hw_breakpoint(bp, &attr);
>> >+ }
>> >+}
>> >+#endif
>> >+
>> > long arch_ptrace(struct task_struct *child, long request,
>> > unsigned long addr, unsigned long data)
>> > {
>> > long ret = -EIO;
>> >+ unsigned long __user *datap = (unsigned long __user *) data;
>> >
>> > switch (request) {
>> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>> >+ case PTRACE_GETHBPREGS:
>> >+ ret = ptrace_gethbpregs(child, addr, datap);
>> >+ break;
>> >+
>> >+ case PTRACE_SETHBPREGS:
>> >+ ret = ptrace_sethbpregs(child, addr, datap);
>> >+ break;
>> >+#endif
>> > default:
>> > ret = ptrace_request(child, request, addr, data);
>> > break;
>> >--
>> >2.43.0
>> >
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-07-23 17:02 ` Jesse Taube
@ 2025-07-23 17:25 ` Deepak Gupta
0 siblings, 0 replies; 18+ messages in thread
From: Deepak Gupta @ 2025-07-23 17:25 UTC (permalink / raw)
To: Jesse Taube
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Andrew Jones, Atish Patra,
Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen,
Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados,
Clément Léger, Celeste Liu, Evan Green, Nylon Chen
On Wed, Jul 23, 2025 at 10:02:34AM -0700, Jesse Taube wrote:
>On Tue, Jul 22, 2025 at 7:49 PM Deepak Gupta <debug@rivosinc.com> wrote:
>>
>> On Tue, Jul 22, 2025 at 10:38:27AM -0700, Jesse Taube wrote:
>> >From: Himanshu Chauhan <hchauhan@ventanamicro.com>
>> >
>> >RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
>> >SBI debug trigger extension to install/uninstall/update/enable/disable hardware
>> >triggers as specified in Sdtrig ISA extension.
>> >
>> >Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
>> >Signed-off-by: Jesse Taube <jesse@rivosinc.com>
>> >---
>> > arch/riscv/Kconfig | 1 +
>> > arch/riscv/include/asm/hw_breakpoint.h | 60 +++
>> > arch/riscv/include/asm/kdebug.h | 3 +-
>> > arch/riscv/include/asm/sbi.h | 4 +-
>> > arch/riscv/kernel/Makefile | 1 +
>> > arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
>> > arch/riscv/kernel/traps.c | 6 +
>> > 7 files changed, 693 insertions(+), 2 deletions(-)
>> > create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
>> > create mode 100644 arch/riscv/kernel/hw_breakpoint.c
>> >
>> >diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> >index bbec87b79309..95d3047cab10 100644
>> >--- a/arch/riscv/Kconfig
>> >+++ b/arch/riscv/Kconfig
>> >@@ -163,6 +163,7 @@ config RISCV
>> > select HAVE_FUNCTION_ERROR_INJECTION
>> > select HAVE_GCC_PLUGINS
>> > select HAVE_GENERIC_VDSO if MMU && 64BIT
>> >+ select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
>> > select HAVE_IRQ_TIME_ACCOUNTING
>> > select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
>> > select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
>> >diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
>> >new file mode 100644
>> >index 000000000000..8efa3921c535
>> >--- /dev/null
>> >+++ b/arch/riscv/include/asm/hw_breakpoint.h
>> >@@ -0,0 +1,60 @@
>> >+/* SPDX-License-Identifier: GPL-2.0-only */
>> >+/*
>> >+ * Copyright (C) 2024 Ventana Micro Systems Inc.
>> >+ */
>> >+
>> >+#ifndef __RISCV_HW_BREAKPOINT_H
>> >+#define __RISCV_HW_BREAKPOINT_H
>> >+
>> >+struct task_struct;
>> >+
>> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>> >+
>> >+#include <uapi/linux/hw_breakpoint.h>
>> >+
>> >+#if __riscv_xlen == 64
>> >+#define cpu_to_le cpu_to_le64
>> >+#define le_to_cpu le64_to_cpu
>> >+#elif __riscv_xlen == 32
>> >+#define cpu_to_le cpu_to_le32
>> >+#define le_to_cpu le32_to_cpu
>> >+#else
>> >+#error "Unexpected __riscv_xlen"
>> >+#endif
>> >+
>> >+struct arch_hw_breakpoint {
>> >+ unsigned long address;
>> >+ unsigned long len;
>> >+
>> >+ /* Callback info */
>> >+ unsigned long next_addr;
>> >+ bool in_callback;
>> >+
>> >+
>> >+ /* Trigger configuration data */
>> >+ unsigned long tdata1;
>> >+ unsigned long tdata2;
>> >+ unsigned long tdata3;
>> >+};
>> >+
>> >+/* Maximum number of hardware breakpoints supported */
>> >+#define RV_MAX_TRIGGERS 32
>> >+
>> >+struct perf_event_attr;
>> >+struct notifier_block;
>> >+struct perf_event;
>> >+struct pt_regs;
>> >+
>> >+int hw_breakpoint_slots(int type);
>> >+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
>> >+int hw_breakpoint_arch_parse(struct perf_event *bp,
>> >+ const struct perf_event_attr *attr,
>> >+ struct arch_hw_breakpoint *hw);
>> >+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
>> >+ unsigned long val, void *data);
>> >+int arch_install_hw_breakpoint(struct perf_event *bp);
>> >+void arch_uninstall_hw_breakpoint(struct perf_event *bp);
>> >+void hw_breakpoint_pmu_read(struct perf_event *bp);
>> >+
>> >+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
>> >+#endif /* __RISCV_HW_BREAKPOINT_H */
>> >diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
>> >index 85ac00411f6e..53e989781aa1 100644
>> >--- a/arch/riscv/include/asm/kdebug.h
>> >+++ b/arch/riscv/include/asm/kdebug.h
>> >@@ -6,7 +6,8 @@
>> > enum die_val {
>> > DIE_UNUSED,
>> > DIE_TRAP,
>> >- DIE_OOPS
>> >+ DIE_OOPS,
>> >+ DIE_DEBUG
>> > };
>> >
>> > #endif
>> >diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
>> >index be2ca8e8a49e..64fa7a82aa45 100644
>> >--- a/arch/riscv/include/asm/sbi.h
>> >+++ b/arch/riscv/include/asm/sbi.h
>> >@@ -282,7 +282,9 @@ struct sbi_sta_struct {
>> > u8 pad[47];
>> > } __packed;
>> >
>> >-#define SBI_SHMEM_DISABLE -1
>> >+#define SBI_SHMEM_DISABLE (-1UL)
>> >+#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
>> >+#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
>> >
>> > enum sbi_ext_nacl_fid {
>> > SBI_EXT_NACL_PROBE_FEATURE = 0x0,
>> >diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
>> >index 4f719b09e5ad..3e72505734bd 100644
>> >--- a/arch/riscv/kernel/Makefile
>> >+++ b/arch/riscv/kernel/Makefile
>> >@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
>> >
>> > obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
>> > obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
>> >+obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
>> > obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
>> > ifeq ($(CONFIG_RISCV_SBI), y)
>> > obj-$(CONFIG_SMP) += sbi-ipi.o
>> >diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
>> >new file mode 100644
>> >index 000000000000..9e3a3b82d300
>> >--- /dev/null
>> >+++ b/arch/riscv/kernel/hw_breakpoint.c
>> >@@ -0,0 +1,620 @@
>> >+// SPDX-License-Identifier: GPL-2.0-only
>> >+/*
>> >+ * Copyright (C) 2024 Ventana Micro Systems Inc.
>> >+ */
>> >+
>> >+#include <linux/hw_breakpoint.h>
>> >+#include <linux/perf_event.h>
>> >+#include <linux/spinlock.h>
>> >+#include <linux/percpu.h>
>> >+#include <linux/kdebug.h>
>> >+#include <linux/bitops.h>
>> >+#include <linux/bitfield.h>
>> >+#include <linux/cpu.h>
>> >+#include <linux/cpuhotplug.h>
>> >+
>> >+#include <asm/insn.h>
>> >+#include <asm/sbi.h>
>> >+
>> >+#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
>> >+#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
>> >+
>> >+#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
>> >+#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
>> >+
>> >+#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
>> >+#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
>> >+#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
>> >+#define DBTR_TDATA1_MCONTROL6_U BIT(3)
>> >+#define DBTR_TDATA1_MCONTROL6_S BIT(4)
>> >+#define DBTR_TDATA1_MCONTROL6_M BIT(6)
>> >+#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
>> >+#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
>> >+#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
>> >+#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
>> >+
>> >+#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
>> >+#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
>> >+#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
>> >+#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
>> >+
>> >+#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
>> >+#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
>> >+#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
>> >+#define DBTR_TDATA1_MCONTROL_U BIT(3)
>> >+#define DBTR_TDATA1_MCONTROL_S BIT(4)
>> >+#define DBTR_TDATA1_MCONTROL_M BIT(6)
>> >+#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
>> >+#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
>> >+#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
>> >+
>> >+#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
>> >+#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
>> >+#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
>> >+/* value of 5 split across HI and LO */
>> >+#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
>> >+#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
>> >+
>> >+/* Registered per-cpu bp/wp */
>> >+static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
>> >+static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
>> >+static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
>> >+
>> >+/* Per-cpu shared memory between S and M mode */
>> >+static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
>> >+
>> >+/* number of debug triggers on this cpu . */
>> >+static int dbtr_total_num __ro_after_init;
>> >+static unsigned long dbtr_type __ro_after_init;
>> >+static unsigned long dbtr_init __ro_after_init;
>> >+
>> >+static int arch_smp_setup_sbi_shmem(unsigned int cpu)
>> >+{
>> >+ union sbi_dbtr_shmem_entry *dbtr_shmem;
>> >+ unsigned long shmem_pa;
>> >+ struct sbiret ret;
>> >+ int rc;
>> >+
>> >+ dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
>> >+ if (!dbtr_shmem) {
>> >+ pr_err("Invalid per-cpu shared memory for debug triggers\n");
>> >+ return -ENODEV;
>> >+ }
>> >+
>> >+ shmem_pa = virt_to_phys(dbtr_shmem);
>> >+
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
>> >+ SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
>> >+
>> >+ if (ret.error) {
>> >+ switch (ret.error) {
>> >+ case SBI_ERR_DENIED:
>> >+ pr_warn("%s: Access denied for shared memory at %lx\n",
>> >+ __func__, shmem_pa);
>> >+ rc = -EPERM;
>> >+ break;
>> >+
>> >+ case SBI_ERR_INVALID_PARAM:
>> >+ case SBI_ERR_INVALID_ADDRESS:
>> >+ pr_warn("%s: Invalid address parameter (%ld)\n",
>> >+ __func__, ret.error);
>> >+ rc = -EINVAL;
>> >+ break;
>> >+
>> >+ case SBI_ERR_ALREADY_AVAILABLE:
>> >+ pr_warn("%s: Shared memory is already set\n",
>> >+ __func__);
>> >+ rc = -EADDRINUSE;
>> >+ break;
>> >+
>> >+ case SBI_ERR_FAILURE:
>> >+ pr_err("%s: Internal sdtrig state error\n",
>> >+ __func__);
>> >+ rc = -ENXIO;
>> >+ break;
>> >+
>> >+ default:
>> >+ pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
>> >+ rc = -ENXIO;
>> >+ break;
>> >+ }
>> >+ }
>> >+
>> >+ pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
>> >+
>> >+ return rc;
>> >+}
>> >+
>> >+static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
>> >+{
>> >+ struct sbiret ret;
>> >+
>> >+ /* Disable shared memory */
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
>> >+ SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
>> >+
>> >+ if (ret.error) {
>> >+ switch (ret.error) {
>> >+ case SBI_ERR_DENIED:
>> >+ pr_err("%s: Access denied for shared memory.\n",
>> >+ __func__);
>> >+ break;
>> >+
>> >+ case SBI_ERR_INVALID_PARAM:
>> >+ case SBI_ERR_INVALID_ADDRESS:
>> >+ pr_err("%s: Invalid address parameter (%lu)\n",
>> >+ __func__, ret.error);
>> >+ break;
>> >+
>> >+ case SBI_ERR_ALREADY_AVAILABLE:
>> >+ pr_err("%s: Shared memory is already set\n",
>> >+ __func__);
>> >+ break;
>> >+ case SBI_ERR_FAILURE:
>> >+ pr_err("%s: Internal sdtrig state error\n",
>> >+ __func__);
>> >+ break;
>> >+ default:
>> >+ pr_err("%s: Unknown error %lu\n", __func__, ret.error);
>> >+ break;
>> >+ }
>> >+ }
>> >+
>> >+ pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
>> >+
>> >+ return 0;
>> >+}
>> >+
>> >+static void init_sbi_dbtr(void)
>> >+{
>> >+ struct sbiret ret;
>> >+
>> >+ /*
>> >+ * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
>> >+ * Only proceed if this is the first CPU to reach this code.
>> >+ */
>> >+ if (test_and_set_bit(0, &dbtr_init))
>> >+ return;
>> >+
>> >+ if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
>> >+ pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
>> >+ dbtr_total_num = 0;
>> >+ return;
>> >+ }
>> >+
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
>> >+ DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
>> >+ if (ret.error) {
>> >+ pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
>> >+ __func__, ret.error);
>> >+ } else if (!ret.value) {
>> >+ pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
>> >+ } else {
>> >+ dbtr_total_num = ret.value;
>> >+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
>> >+ return;
>> >+ }
>> >+
>> >+ /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
>> >+ DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
>> >+ if (ret.error) {
>> >+ pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
>> >+ __func__, ret.error);
>> >+ } else if (!ret.value) {
>> >+ pr_err("%s: No mcontrol triggers available.\n", __func__);
>> >+ dbtr_total_num = 0;
>> >+ } else {
>> >+ dbtr_total_num = ret.value;
>> >+ dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
>> >+ }
>> >+}
>>
>> `dbtr_total_num` is static and thus would be initialized to zero.
>> Although above code is setting it to zero in some error conditions.
>> And not setting to zero in other error conditions.
>>
>> It'll be better if function starts out setting `dbtr_total_num` as setting to 0.
>> Then set correct value in success conditions.
>
>I would do that, but `dbtr_total_num` is marked as `__ro_after_init`
>and I don't want to
>write to it twice, and i wanted to explicitly set it to zero as that
>means breakpoints are disabled.
>I can just remove the assignments to zero though.
Make sense.
>
>>
>> >+
>> >+int hw_breakpoint_slots(int type)
>> >+{
>> >+ /*
>> >+ * We can be called early, so don't rely on
>> >+ * static variables being initialised.
>> >+ */
>> >+ init_sbi_dbtr();
>> >+
>> >+ return dbtr_total_num;
>> >+}
>> >+
>> >+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
>> >+{
>> >+ unsigned int len;
>> >+ unsigned long va;
>> >+
>> >+ va = hw->address;
>> >+ len = hw->len;
>> >+
>> >+ return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
>> >+}
>> >+
>> >+static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
>> >+ struct arch_hw_breakpoint *hw)
>> >+{
>> >+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
>> >+
>> >+ switch (attr->bp_type) {
>> >+ case HW_BREAKPOINT_X:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
>> >+ break;
>> >+ case HW_BREAKPOINT_R:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
>> >+ break;
>> >+ case HW_BREAKPOINT_W:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
>> >+ break;
>> >+ case HW_BREAKPOINT_RW:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
>> >+ break;
>> >+ default:
>> >+ return -EINVAL;
>> >+ }
>> >+
>> >+ switch (attr->bp_len) {
>> >+ case HW_BREAKPOINT_LEN_1:
>> >+ hw->len = 1;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>> >+ DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
>> >+ break;
>> >+ case HW_BREAKPOINT_LEN_2:
>> >+ hw->len = 2;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>> >+ DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
>> >+ break;
>> >+ case HW_BREAKPOINT_LEN_4:
>> >+ hw->len = 4;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>> >+ DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
>> >+ break;
>> >+#if __riscv_xlen >= 64
>> >+ case HW_BREAKPOINT_LEN_8:
>> >+ hw->len = 8;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
>> >+ DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
>> >+ FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
>> >+ DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
>> >+ break;
>> >+#endif
>> >+ default:
>> >+ return -EINVAL;
>> >+ }
>> >+
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL_U;
>>
>> Assuming this function could be re-used for kernel data breakpoints too
>> Shouldn't this be based on some flag to select U v/s S.
>> Same comment for setting up tdata1 with mcontrol6.
>
>Yes I can add a mode flag, so when kernel breakpoints or virtualized breakpoints
>are added it will be easier.
>
>Thanks,
>Jesse Taube
>
>>
>> >+
>> >+ hw->tdata1 = tdata1;
>> >+
>> >+ return 0;
>> >+}
>> >+
>> >+static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
>> >+ struct arch_hw_breakpoint *hw)
>> >+{
>> >+ unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
>> >+
>> >+ switch (attr->bp_type) {
>> >+ case HW_BREAKPOINT_X:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
>> >+ break;
>> >+ case HW_BREAKPOINT_R:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
>> >+ break;
>> >+ case HW_BREAKPOINT_W:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
>> >+ break;
>> >+ case HW_BREAKPOINT_RW:
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
>> >+ break;
>> >+ default:
>> >+ return -EINVAL;
>> >+ }
>> >+
>> >+ switch (attr->bp_len) {
>> >+ case HW_BREAKPOINT_LEN_1:
>> >+ hw->len = 1;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>> >+ DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
>> >+ break;
>> >+ case HW_BREAKPOINT_LEN_2:
>> >+ hw->len = 2;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>> >+ DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
>> >+ break;
>> >+ case HW_BREAKPOINT_LEN_4:
>> >+ hw->len = 4;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>> >+ DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
>> >+ break;
>> >+ case HW_BREAKPOINT_LEN_8:
>> >+ hw->len = 8;
>> >+ tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
>> >+ DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
>> >+ break;
>> >+ default:
>> >+ return -EINVAL;
>> >+ }
>> >+
>> >+ tdata1 |= DBTR_TDATA1_MCONTROL6_U;
>> >+
>> >+ hw->tdata1 = tdata1;
>> >+
>> >+ return 0;
>> >+}
>> >+
>> >+int hw_breakpoint_arch_parse(struct perf_event *bp,
>> >+ const struct perf_event_attr *attr,
>> >+ struct arch_hw_breakpoint *hw)
>> >+{
>> >+ int ret;
>> >+
>> >+ /* Breakpoint address */
>> >+ hw->address = attr->bp_addr;
>> >+ hw->tdata2 = attr->bp_addr;
>> >+ hw->tdata3 = 0x0;
>> >+ hw->next_addr = 0x0;
>> >+ hw->in_callback = false;
>> >+
>> >+ switch (dbtr_type) {
>> >+ case DBTR_TDATA1_TYPE_MCONTROL:
>> >+ ret = rv_init_mcontrol_trigger(attr, hw);
>> >+ break;
>> >+ case DBTR_TDATA1_TYPE_MCONTROL6:
>> >+ ret = rv_init_mcontrol6_trigger(attr, hw);
>> >+ break;
>> >+ default:
>> >+ pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
>> >+ ret = -EOPNOTSUPP;
>> >+ break;
>> >+ }
>> >+
>> >+ return ret;
>> >+}
>> >+
>> >+/*
>> >+ * Set breakpoint to next insruction after breakpoint.
>> >+ * Returns 0 if success
>> >+ * Returns < 0 on error
>> >+ */
>> >+static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
>> >+{
>> >+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
>> >+ struct arch_hw_breakpoint old_hw_bp;
>> >+ struct perf_event_attr bp_insn;
>> >+ unsigned long next_addr, insn;
>> >+ int ret;
>> >+
>> >+ /* Remove breakpoint even if return error as not to loop */
>> >+ arch_uninstall_hw_breakpoint(event);
>> >+
>> >+ ret = get_insn(regs, regs->epc, &insn);
>> >+ if (ret < 0)
>> >+ return ret;
>> >+
>> >+ next_addr = get_step_address(regs, insn);
>> >+
>> >+ ret = get_insn(regs, next_addr, &insn);
>> >+ if (ret < 0)
>> >+ return ret;
>> >+
>> >+ bp_insn.bp_type = HW_BREAKPOINT_X;
>> >+ bp_insn.bp_addr = next_addr;
>> >+ /* Get the size of the intruction */
>> >+ bp_insn.bp_len = GET_INSN_LENGTH(insn);
>> >+
>> >+ ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
>> >+ if (ret)
>> >+ return ret;
>> >+
>> >+ ret = arch_install_hw_breakpoint(event);
>> >+ if (ret)
>> >+ return ret;
>> >+
>> >+ bp->in_callback = true;
>> >+ bp->next_addr = next_addr;
>> >+ return 0;
>> >+}
>> >+
>> >+/*
>> >+ * HW Breakpoint/watchpoint handler
>> >+ */
>> >+static int hw_breakpoint_handler(struct pt_regs *regs)
>> >+{
>> >+ struct perf_event *event;
>> >+ struct arch_hw_breakpoint *bp;
>> >+ int ret, i;
>> >+
>> >+ for (i = 0; i < dbtr_total_num; i++) {
>> >+ event = this_cpu_read(pcpu_hw_bp_events[i]);
>> >+ if (!event)
>> >+ continue;
>> >+
>> >+ bp = counter_arch_bp(event);
>> >+ if (bp->in_callback) {
>> >+ /* Reset changed breakpoint data */
>> >+ bp->in_callback = false;
>> >+ if (regs->epc == bp->next_addr) {
>> >+ arch_uninstall_hw_breakpoint(event);
>> >+ /* Restore original breakpoint */
>> >+ if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
>> >+ return NOTIFY_DONE;
>> >+ if (arch_install_hw_breakpoint(event))
>> >+ return NOTIFY_DONE;
>> >+ return NOTIFY_STOP;
>> >+ }
>> >+
>> >+ pr_err("%s: in_callback was set, but epc(%lx) was not next "
>> >+ "address(%lx).\n", __func__, regs->epc, bp->next_addr);
>> >+ bp->next_addr = 0x0;
>> >+ return NOTIFY_DONE;
>> >+ }
>> >+
>> >+ switch (event->attr.bp_type) {
>> >+ /* Breakpoint */
>> >+ case HW_BREAKPOINT_X:
>> >+ if (event->attr.bp_addr == regs->epc) {
>> >+ ret = setup_singlestep(event, regs);
>> >+ if (ret < 0) {
>> >+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
>> >+ return NOTIFY_DONE;
>> >+ }
>> >+
>> >+ perf_bp_event(event, regs);
>> >+ return NOTIFY_STOP;
>> >+ }
>> >+ break;
>> >+
>> >+ /* Watchpoint */
>> >+ case HW_BREAKPOINT_W:
>> >+ case HW_BREAKPOINT_R:
>> >+ case HW_BREAKPOINT_RW:
>> >+ if (event->attr.bp_addr == regs->badaddr) {
>> >+ ret = setup_singlestep(event, regs);
>> >+ if (ret < 0) {
>> >+ pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
>> >+ return NOTIFY_DONE;
>> >+ }
>> >+
>> >+ perf_bp_event(event, regs);
>> >+ return NOTIFY_STOP;
>> >+ }
>> >+ break;
>> >+
>> >+ default:
>> >+ pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
>> >+ break;
>> >+ }
>> >+ }
>> >+
>> >+ return NOTIFY_DONE;
>> >+}
>> >+
>> >+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
>> >+ unsigned long val, void *data)
>> >+{
>> >+ struct die_args *args = data;
>> >+
>> >+ if (val != DIE_DEBUG)
>> >+ return NOTIFY_DONE;
>> >+
>> >+ return hw_breakpoint_handler(args->regs);
>> >+}
>> >+
>> >+/* atomic: counter->ctx->lock is held */
>> >+int arch_install_hw_breakpoint(struct perf_event *event)
>> >+{
>> >+ struct arch_hw_breakpoint *bp = counter_arch_bp(event);
>> >+ union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
>> >+ struct sbi_dbtr_data_msg *xmit;
>> >+ struct sbi_dbtr_id_msg *recv;
>> >+ struct perf_event **slot;
>> >+ unsigned long idx;
>> >+ struct sbiret ret;
>> >+ int err = 0;
>> >+
>> >+ raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
>> >+ *this_cpu_ptr(&ecall_lock_flags));
>> >+
>> >+ xmit = &shmem->data;
>> >+ recv = &shmem->id;
>> >+ xmit->tdata1 = cpu_to_le(bp->tdata1);
>> >+ xmit->tdata2 = cpu_to_le(bp->tdata2);
>> >+ xmit->tdata3 = cpu_to_le(bp->tdata3);
>> >+
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
>> >+ 1, 0, 0, 0, 0, 0);
>> >+
>> >+ if (ret.error) {
>> >+ pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
>> >+ err = sbi_err_map_linux_errno(ret.error);
>> >+ goto done;
>> >+ }
>> >+
>> >+ idx = le_to_cpu(recv->idx);
>> >+ if (idx >= dbtr_total_num) {
>> >+ pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
>> >+ err = -EINVAL;
>> >+ goto done;
>> >+ }
>> >+
>> >+ slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
>> >+ if (*slot) {
>> >+ pr_warn("%s: slot %lu is in use\n", __func__, idx);
>> >+ err = -EBUSY;
>> >+ goto done;
>> >+ }
>> >+
>> >+ pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
>> >+
>> >+ /* Save the event - to be looked up in handler */
>> >+ *slot = event;
>> >+
>> >+done:
>> >+ raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
>> >+ *this_cpu_ptr(&ecall_lock_flags));
>> >+ return err;
>> >+}
>> >+
>> >+/* atomic: counter->ctx->lock is held */
>> >+void arch_uninstall_hw_breakpoint(struct perf_event *event)
>> >+{
>> >+ struct sbiret ret;
>> >+ int i;
>> >+
>> >+ for (i = 0; i < dbtr_total_num; i++) {
>> >+ struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
>> >+
>> >+ if (*slot == event) {
>> >+ *slot = NULL;
>> >+ break;
>> >+ }
>> >+ }
>> >+
>> >+
>> >+ if (i == dbtr_total_num) {
>> >+ pr_warn("%s: Breakpoint not installed.\n", __func__);
>> >+ return;
>> >+ }
>> >+
>> >+ ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
>> >+ i, 1, 0, 0, 0, 0);
>> >+ if (ret.error)
>> >+ pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
>> >+}
>> >+
>> >+void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
>> >+
>> >+void hw_breakpoint_pmu_read(struct perf_event *bp) { }
>> >+
>> >+static int __init arch_hw_breakpoint_init(void)
>> >+{
>> >+ unsigned int cpu;
>> >+ int rc = 0;
>> >+
>> >+ for_each_possible_cpu(cpu)
>> >+ raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
>> >+
>> >+ init_sbi_dbtr();
>> >+
>> >+ if (dbtr_total_num) {
>> >+ pr_debug("%s: total number of type %lu triggers: %u\n",
>> >+ __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
>> >+ } else {
>> >+ pr_debug("%s: No hardware triggers available\n", __func__);
>> >+ return rc;
>> >+ }
>> >+
>> >+ /* Hotplug handler to register/unregister shared memory with SBI */
>> >+ rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
>> >+ "riscv/hw_breakpoint:prepare",
>> >+ arch_smp_setup_sbi_shmem,
>> >+ arch_smp_teardown_sbi_shmem);
>> >+
>> >+ if (rc < 0)
>> >+ pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
>> >+
>> >+ return rc;
>> >+}
>> >+arch_initcall(arch_hw_breakpoint_init);
>> >diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
>> >index 938a8b841f94..2ac471ec79a8 100644
>> >--- a/arch/riscv/kernel/traps.c
>> >+++ b/arch/riscv/kernel/traps.c
>> >@@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
>> > if (probe_breakpoint_handler(regs))
>> > return;
>> >
>> >+#ifdef CONFIG_HAVE_HW_BREAKPOINT
>> >+ if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
>> >+ == NOTIFY_STOP)
>> >+ return;
>> >+#endif
>> >+
>> > current->thread.bad_cause = regs->cause;
>> >
>> > if (user_mode(regs))
>> >--
>> >2.43.0
>> >
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault
2025-07-22 17:38 ` [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault Jesse Taube
@ 2025-08-04 7:44 ` Clément Léger
0 siblings, 0 replies; 18+ messages in thread
From: Clément Léger @ 2025-08-04 7:44 UTC (permalink / raw)
To: Jesse Taube, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Himanshu Chauhan, Charlie Jenkins, Samuel Holland,
Deepak Gupta, Andrew Jones, Atish Patra, Anup Patel,
Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen, Nam Cao,
Andrew Morton, Mike Rapoport (Microsoft), Luis Chamberlain,
Yunhui Cui, Joel Granados, Celeste Liu, Evan Green, Nylon Chen
On 22/07/2025 19:38, Jesse Taube wrote:
> __read_insn was using get_user for user space reads and direct
> dereferencing for kernel space reads.
> Update to use copy_from_user_nofault, copy_from_kernel_nofault for this
> as get_user is user context only and may sleep.
>
> Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> ---
> Squash with previous commit as it's breaks bisectability.
> Separated as i'm unsure if copy_from_user_nofault is an acceptable
> replacement.
> ---
> arch/riscv/kernel/insn.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/arch/riscv/kernel/insn.c b/arch/riscv/kernel/insn.c
> index dd2a6ef9fd25..b8e5202ddced 100644
> --- a/arch/riscv/kernel/insn.c
> +++ b/arch/riscv/kernel/insn.c
> @@ -2,6 +2,9 @@
> /*
> * Copyright 2025 Rivos, Inc
> */
> +
> +#include <linux/uaccess.h>
> +
> #include <asm/insn.h>
> #include <asm/ptrace.h>
> #include <asm/uaccess.h>
> @@ -11,10 +14,9 @@
> int __ret; \
> \
> if (user_mode(regs)) { \
> - __ret = get_user(insn, (type __user *) insn_addr); \
> + __ret = copy_from_user_nofault(&insn, (const type __user *) insn_addr, sizeof(type)); \
> } else { \
> - insn = *(type *)insn_addr; \
> - __ret = 0; \
> + __ret = copy_from_kernel_nofault(&insn, (const type *) insn_addr, sizeof(type)); \
> } \
Hi Jesse,
This can not be used for misaligned accesses handling. As pointed out by
Samuel in a previous misaligned access handling patch, using the
no_fault() version would lead to access failure if the user memory is
swapped out. For that reason it was swapped from the no_fault() version
to the standard version.
Thanks,
Clément
> \
> __ret; \
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-07-22 17:38 ` [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints Jesse Taube
2025-07-23 2:49 ` Deepak Gupta
@ 2025-08-04 8:01 ` Clément Léger
2025-08-04 15:03 ` Jesse Taube
1 sibling, 1 reply; 18+ messages in thread
From: Clément Léger @ 2025-08-04 8:01 UTC (permalink / raw)
To: Jesse Taube, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Himanshu Chauhan, Charlie Jenkins, Samuel Holland,
Deepak Gupta, Andrew Jones, Atish Patra, Anup Patel,
Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen, Nam Cao,
Andrew Morton, Mike Rapoport (Microsoft), Luis Chamberlain,
Yunhui Cui, Joel Granados, Celeste Liu, Evan Green, Nylon Chen
On 22/07/2025 19:38, Jesse Taube wrote:
> From: Himanshu Chauhan <hchauhan@ventanamicro.com>
>
> RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
> SBI debug trigger extension to install/uninstall/update/enable/disable hardware
> triggers as specified in Sdtrig ISA extension.
>
> Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
> Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/hw_breakpoint.h | 60 +++
> arch/riscv/include/asm/kdebug.h | 3 +-
> arch/riscv/include/asm/sbi.h | 4 +-
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
> arch/riscv/kernel/traps.c | 6 +
> 7 files changed, 693 insertions(+), 2 deletions(-)
> create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
> create mode 100644 arch/riscv/kernel/hw_breakpoint.c
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index bbec87b79309..95d3047cab10 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -163,6 +163,7 @@ config RISCV
> select HAVE_FUNCTION_ERROR_INJECTION
> select HAVE_GCC_PLUGINS
> select HAVE_GENERIC_VDSO if MMU && 64BIT
> + select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
> select HAVE_IRQ_TIME_ACCOUNTING
> select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
> select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
> diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
> new file mode 100644
> index 000000000000..8efa3921c535
> --- /dev/null
> +++ b/arch/riscv/include/asm/hw_breakpoint.h
> @@ -0,0 +1,60 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2024 Ventana Micro Systems Inc.
> + */
> +
> +#ifndef __RISCV_HW_BREAKPOINT_H
> +#define __RISCV_HW_BREAKPOINT_H
> +
> +struct task_struct;
> +
> +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> +
> +#include <uapi/linux/hw_breakpoint.h>
> +
> +#if __riscv_xlen == 64
> +#define cpu_to_le cpu_to_le64
> +#define le_to_cpu le64_to_cpu
> +#elif __riscv_xlen == 32
> +#define cpu_to_le cpu_to_le32
> +#define le_to_cpu le32_to_cpu
> +#else
> +#error "Unexpected __riscv_xlen"
> +#endif
> +
> +struct arch_hw_breakpoint {
> + unsigned long address;
> + unsigned long len;
> +
> + /* Callback info */
> + unsigned long next_addr;
> + bool in_callback;
> +
> +
> + /* Trigger configuration data */
> + unsigned long tdata1;
> + unsigned long tdata2;
> + unsigned long tdata3;
> +};
> +
> +/* Maximum number of hardware breakpoints supported */
> +#define RV_MAX_TRIGGERS 32
> +
> +struct perf_event_attr;
> +struct notifier_block;
> +struct perf_event;
> +struct pt_regs;
> +
> +int hw_breakpoint_slots(int type);
> +int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
> +int hw_breakpoint_arch_parse(struct perf_event *bp,
> + const struct perf_event_attr *attr,
> + struct arch_hw_breakpoint *hw);
> +int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> + unsigned long val, void *data);
> +int arch_install_hw_breakpoint(struct perf_event *bp);
> +void arch_uninstall_hw_breakpoint(struct perf_event *bp);
> +void hw_breakpoint_pmu_read(struct perf_event *bp);
> +
> +#endif /* CONFIG_HAVE_HW_BREAKPOINT */
> +#endif /* __RISCV_HW_BREAKPOINT_H */
> diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
> index 85ac00411f6e..53e989781aa1 100644
> --- a/arch/riscv/include/asm/kdebug.h
> +++ b/arch/riscv/include/asm/kdebug.h
> @@ -6,7 +6,8 @@
> enum die_val {
> DIE_UNUSED,
> DIE_TRAP,
> - DIE_OOPS
> + DIE_OOPS,
> + DIE_DEBUG
> };
>
> #endif
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index be2ca8e8a49e..64fa7a82aa45 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -282,7 +282,9 @@ struct sbi_sta_struct {
> u8 pad[47];
> } __packed;
>
> -#define SBI_SHMEM_DISABLE -1
> +#define SBI_SHMEM_DISABLE (-1UL)
> +#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
> +#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
>
> enum sbi_ext_nacl_fid {
> SBI_EXT_NACL_PROBE_FEATURE = 0x0,
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index 4f719b09e5ad..3e72505734bd 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
>
> obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
> obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
> +obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
> obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
> ifeq ($(CONFIG_RISCV_SBI), y)
> obj-$(CONFIG_SMP) += sbi-ipi.o
> diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> new file mode 100644
> index 000000000000..9e3a3b82d300
> --- /dev/null
> +++ b/arch/riscv/kernel/hw_breakpoint.c
> @@ -0,0 +1,620 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2024 Ventana Micro Systems Inc.
> + */
> +
> +#include <linux/hw_breakpoint.h>
> +#include <linux/perf_event.h>
> +#include <linux/spinlock.h>
> +#include <linux/percpu.h>
> +#include <linux/kdebug.h>
> +#include <linux/bitops.h>
> +#include <linux/bitfield.h>
> +#include <linux/cpu.h>
> +#include <linux/cpuhotplug.h>
> +
> +#include <asm/insn.h>
> +#include <asm/sbi.h>
> +
> +#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
> +#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
> +
> +#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
> +#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
> +
> +#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
> +#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
> +#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
> +#define DBTR_TDATA1_MCONTROL6_U BIT(3)
> +#define DBTR_TDATA1_MCONTROL6_S BIT(4)
> +#define DBTR_TDATA1_MCONTROL6_M BIT(6)
> +#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
> +#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
> +#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
> +#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
> +
> +#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
> +#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
> +#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
> +#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
> +
> +#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
> +#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
> +#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
> +#define DBTR_TDATA1_MCONTROL_U BIT(3)
> +#define DBTR_TDATA1_MCONTROL_S BIT(4)
> +#define DBTR_TDATA1_MCONTROL_M BIT(6)
> +#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
> +#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
> +#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
> +
> +#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
> +#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
> +#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
> +/* value of 5 split across HI and LO */
> +#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
> +#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
> +
> +/* Registered per-cpu bp/wp */
> +static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
> +static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
> +static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
> +
> +/* Per-cpu shared memory between S and M mode */
> +static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
> +
> +/* number of debug triggers on this cpu . */
> +static int dbtr_total_num __ro_after_init;
> +static unsigned long dbtr_type __ro_after_init;
> +static unsigned long dbtr_init __ro_after_init;
> +
> +static int arch_smp_setup_sbi_shmem(unsigned int cpu)
> +{
> + union sbi_dbtr_shmem_entry *dbtr_shmem;
> + unsigned long shmem_pa;
> + struct sbiret ret;
> + int rc;
> +
> + dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
> + if (!dbtr_shmem) {
> + pr_err("Invalid per-cpu shared memory for debug triggers\n");
> + return -ENODEV;
> + }
> +
> + shmem_pa = virt_to_phys(dbtr_shmem);
> +
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> + SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
> +
> + if (ret.error) {
> + switch (ret.error) {
> + case SBI_ERR_DENIED:
> + pr_warn("%s: Access denied for shared memory at %lx\n",
> + __func__, shmem_pa);
> + rc = -EPERM;
> + break;
> +
> + case SBI_ERR_INVALID_PARAM:
> + case SBI_ERR_INVALID_ADDRESS:
> + pr_warn("%s: Invalid address parameter (%ld)\n",
> + __func__, ret.error);
> + rc = -EINVAL;
> + break;
> +
> + case SBI_ERR_ALREADY_AVAILABLE:
> + pr_warn("%s: Shared memory is already set\n",
> + __func__);
> + rc = -EADDRINUSE;
> + break;
> +
> + case SBI_ERR_FAILURE:
> + pr_err("%s: Internal sdtrig state error\n",
> + __func__);
> + rc = -ENXIO;
> + break;
> +
> + default:
> + pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
> + rc = -ENXIO;
> + break;
> + }
> + }
Hi Jesse,
Is there a reason not to use sbi_err_map_linux_errno() ?
> +
> + pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
> +
> + return rc;
> +}
> +
> +static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
> +{
> + struct sbiret ret;
> +
> + /* Disable shared memory */
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> + SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
> +
> + if (ret.error) {
> + switch (ret.error) {
> + case SBI_ERR_DENIED:
> + pr_err("%s: Access denied for shared memory.\n",
> + __func__);
> + break;
> +
> + case SBI_ERR_INVALID_PARAM:
> + case SBI_ERR_INVALID_ADDRESS:
> + pr_err("%s: Invalid address parameter (%lu)\n",
> + __func__, ret.error);
> + break;
> +
> + case SBI_ERR_ALREADY_AVAILABLE:
> + pr_err("%s: Shared memory is already set\n",
> + __func__);
> + break;
> + case SBI_ERR_FAILURE:
> + pr_err("%s: Internal sdtrig state error\n",
> + __func__);
> + break;
> + default:
> + pr_err("%s: Unknown error %lu\n", __func__, ret.error);
> + break;
> + }
> + }
Ditto
> +
> + pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
> +
> + return 0;
> +}
> +
> +static void init_sbi_dbtr(void)
> +{
> + struct sbiret ret;
> +
> + /*
> + * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
> + * Only proceed if this is the first CPU to reach this code.
> + */
> + if (test_and_set_bit(0, &dbtr_init))
> + return;
> +
> + if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
> + pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
> + dbtr_total_num = 0;
> + return;
> + }
> +
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> + DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
> + if (ret.error) {
> + pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
> + __func__, ret.error);
> + } else if (!ret.value) {
> + pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
> + } else {
> + dbtr_total_num = ret.value;
> + dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
> + return;
> + }
> +
> + /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
Nit: Since all other comments starts with a capital letter, I'd suggest
using a capital F (Fallback).
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> + DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
> + if (ret.error) {
> + pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
> + __func__, ret.error);
> + } else if (!ret.value) {
> + pr_err("%s: No mcontrol triggers available.\n", __func__);
> + dbtr_total_num = 0;
> + } else {
> + dbtr_total_num = ret.value;
> + dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
> + }
> +}
> +
> +int hw_breakpoint_slots(int type)
> +{
> + /*
> + * We can be called early, so don't rely on
> + * static variables being initialised.
> + */
> + init_sbi_dbtr();
> +
> + return dbtr_total_num;
> +}
> +
> +int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
> +{
> + unsigned int len;
> + unsigned long va;
> +
> + va = hw->address;
> + len = hw->len;
> +
> + return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
> +}
> +
> +static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
> + struct arch_hw_breakpoint *hw)
> +{
> + unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> +
> + switch (attr->bp_type) {
> + case HW_BREAKPOINT_X:
> + tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
> + break;
> + case HW_BREAKPOINT_R:
> + tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
> + break;
> + case HW_BREAKPOINT_W:
> + tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
> + break;
> + case HW_BREAKPOINT_RW:
> + tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + switch (attr->bp_len) {
> + case HW_BREAKPOINT_LEN_1:
> + hw->len = 1;
You can probably add a define for that to avoid
#define TDATA1(lo, hi) \
FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD, \
DBTR_TDATA1_MCONTROL_SIZELO_64BIT) | \
FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD, \
DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> + DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_8BIT, 0);
> + break;
> + case HW_BREAKPOINT_LEN_2:
> + hw->len = 2;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> + DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_16BIT, 0);
> + break;
> + case HW_BREAKPOINT_LEN_4:
> + hw->len = 4;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> + DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_32BIT, 0);
> + break;
> +#if __riscv_xlen >= 64
> + case HW_BREAKPOINT_LEN_8:
> + hw->len = 8;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> + DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
> + FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
> + DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_64BIT,
DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
> + break;
> +#endif
> + default:
> + return -EINVAL;
> + }
> +
> + tdata1 |= DBTR_TDATA1_MCONTROL_U;
> +
> + hw->tdata1 = tdata1;
> +
> + return 0;
> +}
> +
> +static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
> + struct arch_hw_breakpoint *hw)
> +{
> + unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> +
> + switch (attr->bp_type) {
> + case HW_BREAKPOINT_X:
> + tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
> + break;
> + case HW_BREAKPOINT_R:
> + tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
> + break;
> + case HW_BREAKPOINT_W:
> + tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
> + break;
> + case HW_BREAKPOINT_RW:
> + tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + switch (attr->bp_len) {
> + case HW_BREAKPOINT_LEN_1:
> + hw->len = 1;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> + DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
> + break;
> + case HW_BREAKPOINT_LEN_2:
> + hw->len = 2;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> + DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
> + break;
> + case HW_BREAKPOINT_LEN_4:
> + hw->len = 4;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> + DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
> + break;
> + case HW_BREAKPOINT_LEN_8:
> + hw->len = 8;
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> + DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + tdata1 |= DBTR_TDATA1_MCONTROL6_U;
> +
> + hw->tdata1 = tdata1;
> +
> + return 0;
> +}
> +
> +int hw_breakpoint_arch_parse(struct perf_event *bp,
> + const struct perf_event_attr *attr,
> + struct arch_hw_breakpoint *hw)
> +{
> + int ret;
> +
> + /* Breakpoint address */
> + hw->address = attr->bp_addr;
> + hw->tdata2 = attr->bp_addr;
> + hw->tdata3 = 0x0;
> + hw->next_addr = 0x0;
> + hw->in_callback = false;
> +
> + switch (dbtr_type) {
> + case DBTR_TDATA1_TYPE_MCONTROL:
> + ret = rv_init_mcontrol_trigger(attr, hw);
> + break;
> + case DBTR_TDATA1_TYPE_MCONTROL6:
> + ret = rv_init_mcontrol6_trigger(attr, hw);
> + break;
> + default:
> + pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
> + ret = -EOPNOTSUPP;
> + break;
> + }
> +
> + return ret;
> +}
> +
> +/*
> + * Set breakpoint to next insruction after breakpoint.
Typo: instruction
> + * Returns 0 if success
> + * Returns < 0 on error
> + */
> +static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
> +{
> + struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> + struct arch_hw_breakpoint old_hw_bp;
> + struct perf_event_attr bp_insn;
> + unsigned long next_addr, insn;
> + int ret;
> +
> + /* Remove breakpoint even if return error as not to loop */
> + arch_uninstall_hw_breakpoint(event);
> +
> + ret = get_insn(regs, regs->epc, &insn);
> + if (ret < 0)
> + return ret;
> +
> + next_addr = get_step_address(regs, insn);
> +
> + ret = get_insn(regs, next_addr, &insn);
> + if (ret < 0)
> + return ret;
> +
> + bp_insn.bp_type = HW_BREAKPOINT_X;
> + bp_insn.bp_addr = next_addr;
> + /* Get the size of the intruction */
> + bp_insn.bp_len = GET_INSN_LENGTH(insn);
> +
> + ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
> + if (ret)
> + return ret;
> +
> + ret = arch_install_hw_breakpoint(event);
> + if (ret)
> + return ret;
> +
> + bp->in_callback = true;
> + bp->next_addr = next_addr;
> + return 0;
> +}
> +
> +/*
> + * HW Breakpoint/watchpoint handler
> + */
> +static int hw_breakpoint_handler(struct pt_regs *regs)
> +{
> + struct perf_event *event;
> + struct arch_hw_breakpoint *bp;
> + int ret, i;
> +
> + for (i = 0; i < dbtr_total_num; i++) {
> + event = this_cpu_read(pcpu_hw_bp_events[i]);
> + if (!event)
> + continue;
> +
> + bp = counter_arch_bp(event);
> + if (bp->in_callback) {
> + /* Reset changed breakpoint data */
> + bp->in_callback = false;
> + if (regs->epc == bp->next_addr) {
> + arch_uninstall_hw_breakpoint(event);
> + /* Restore original breakpoint */
> + if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
> + return NOTIFY_DONE;
> + if (arch_install_hw_breakpoint(event))
> + return NOTIFY_DONE;
> + return NOTIFY_STOP;
> + }
> +
> + pr_err("%s: in_callback was set, but epc(%lx) was not next "
> + "address(%lx).\n", __func__, regs->epc, bp->next_addr);
> + bp->next_addr = 0x0;
> + return NOTIFY_DONE;
> + }
> +
> + switch (event->attr.bp_type) {
> + /* Breakpoint */
> + case HW_BREAKPOINT_X:
> + if (event->attr.bp_addr == regs->epc) {
> + ret = setup_singlestep(event, regs);
> + if (ret < 0) {
> + pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> + return NOTIFY_DONE;
> + }
> +
> + perf_bp_event(event, regs);
> + return NOTIFY_STOP;
> + }
> + break;
> +
> + /* Watchpoint */
> + case HW_BREAKPOINT_W:
> + case HW_BREAKPOINT_R:
> + case HW_BREAKPOINT_RW:
> + if (event->attr.bp_addr == regs->badaddr) {
> + ret = setup_singlestep(event, regs);
> + if (ret < 0) {
> + pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> + return NOTIFY_DONE;
> + }
> +
> + perf_bp_event(event, regs);
> + return NOTIFY_STOP;
> + }
> + break;
> +
> + default:
> + pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
> + break;
> + }
> + }
> +
> + return NOTIFY_DONE;
> +}
> +
> +int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> + unsigned long val, void *data)
> +{
> + struct die_args *args = data;
> +
> + if (val != DIE_DEBUG)
> + return NOTIFY_DONE;
> +
> + return hw_breakpoint_handler(args->regs);
> +}
> +
> +/* atomic: counter->ctx->lock is held */
> +int arch_install_hw_breakpoint(struct perf_event *event)
> +{
> + struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> + union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
> + struct sbi_dbtr_data_msg *xmit;
> + struct sbi_dbtr_id_msg *recv;
> + struct perf_event **slot;
> + unsigned long idx;
> + struct sbiret ret;
> + int err = 0;
> +
> + raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
> + *this_cpu_ptr(&ecall_lock_flags));
> +
> + xmit = &shmem->data;
> + recv = &shmem->id;
> + xmit->tdata1 = cpu_to_le(bp->tdata1);
> + xmit->tdata2 = cpu_to_le(bp->tdata2);
> + xmit->tdata3 = cpu_to_le(bp->tdata3);
> +
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
> + 1, 0, 0, 0, 0, 0);
> +
> + if (ret.error) {
> + pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
> + err = sbi_err_map_linux_errno(ret.error);
> + goto done;
> + }
> +
> + idx = le_to_cpu(recv->idx);
> + if (idx >= dbtr_total_num) {
> + pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
> + err = -EINVAL;
> + goto done;
> + }
> +
> + slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
> + if (*slot) {
> + pr_warn("%s: slot %lu is in use\n", __func__, idx);
> + err = -EBUSY;
> + goto done;
> + }
> +
> + pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
> +
> + /* Save the event - to be looked up in handler */
> + *slot = event;
> +
> +done:
> + raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
> + *this_cpu_ptr(&ecall_lock_flags));
> + return err;
> +}
> +
> +/* atomic: counter->ctx->lock is held */
> +void arch_uninstall_hw_breakpoint(struct perf_event *event)
> +{
> + struct sbiret ret;
> + int i;
> +
> + for (i = 0; i < dbtr_total_num; i++) {
> + struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
> +
> + if (*slot == event) {
> + *slot = NULL;
> + break;
> + }
> + }
> +
> +
> + if (i == dbtr_total_num) {
> + pr_warn("%s: Breakpoint not installed.\n", __func__);
> + return;
> + }
> +
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
> + i, 1, 0, 0, 0, 0);
> + if (ret.error)
> + pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> +}
> +
> +void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
Is there nothing to do to remove the registered breakpoints ? All other
architecture seems to implement it.
> +
> +void hw_breakpoint_pmu_read(struct perf_event *bp) { }
> +
> +static int __init arch_hw_breakpoint_init(void)
> +{
> + unsigned int cpu;
> + int rc = 0;
> +
> + for_each_possible_cpu(cpu)
> + raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
> +
> + init_sbi_dbtr();
> +
> + if (dbtr_total_num) {
> + pr_debug("%s: total number of type %lu triggers: %u\n",
> + __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
> + } else {
> + pr_debug("%s: No hardware triggers available\n", __func__);
> + return rc;
> + }
> +
> + /* Hotplug handler to register/unregister shared memory with SBI */
> + rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> + "riscv/hw_breakpoint:prepare",
> + arch_smp_setup_sbi_shmem,
> + arch_smp_teardown_sbi_shmem);
> +
> + if (rc < 0)
> + pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
> +
> + return rc;
> +}
> +arch_initcall(arch_hw_breakpoint_init);
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index 938a8b841f94..2ac471ec79a8 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
> if (probe_breakpoint_handler(regs))
> return;
>
> +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> + if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
> + == NOTIFY_STOP)
> + return;
> +#endif
> +
> current->thread.bad_cause = regs->cause;
>
> if (user_mode(regs))
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping
2025-07-22 17:38 ` [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping Jesse Taube
@ 2025-08-04 8:15 ` Clément Léger
0 siblings, 0 replies; 18+ messages in thread
From: Clément Léger @ 2025-08-04 8:15 UTC (permalink / raw)
To: Jesse Taube, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Himanshu Chauhan, Charlie Jenkins, Samuel Holland,
Deepak Gupta, Andrew Jones, Atish Patra, Anup Patel,
Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen, Nam Cao,
Andrew Morton, Mike Rapoport (Microsoft), Luis Chamberlain,
Yunhui Cui, Joel Granados, Celeste Liu, Evan Green, Nylon Chen
On 22/07/2025 19:38, Jesse Taube wrote:
> The Sdtrig RISC-V ISA extension does not have a resume flag for
> returning to and executing the instruction at the breakpoint.
> To avoid skipping the instruction or looping, it is necessary to remove
> the hardware breakpoint and single step. Use the icount feature of
> Sdtrig to accomplish this. Use icount as default with an option to allow
> software-based single stepping when hardware or SBI does not have
> icount functionality, as it may cause unwanted side effects when reading
> the instruction from memory.
>
> Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> ---
> arch/riscv/Kconfig | 11 +++++
> arch/riscv/kernel/hw_breakpoint.c | 81 +++++++++++++++++++++++++------
> 2 files changed, 76 insertions(+), 16 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 95d3047cab10..bbde5e118470 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -1105,6 +1105,17 @@ config PARAVIRT_TIME_ACCOUNTING
>
> If in doubt, say N here.
>
> +config HW_BREAKPOINT_COMPUTE_STEP
> + bool "Allow computing hardware breakpoint step address"
> + default n
> + depends on HAVE_HW_BREAKPOINT
> + help
> + Select this option if hardware breakpoints are desired, but
> + hardware or SBI does not have icount functionality. This may cause
> + unwanted side affects when reading the instruction from memory.
Hi Jesse,
Typo: s/affects/effects
Thanks,
Clément
> +
> + If unsure, say N.
> +
> config RELOCATABLE
> bool "Build a relocatable kernel"
> depends on !XIP_KERNEL
> diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> index 9e3a3b82d300..437fd82b9590 100644
> --- a/arch/riscv/kernel/hw_breakpoint.c
> +++ b/arch/riscv/kernel/hw_breakpoint.c
> @@ -20,6 +20,7 @@
> #define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
>
> #define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
> +#define DBTR_TDATA1_TYPE_ICOUNT (3UL << DBTR_TDATA1_TYPE_SHIFT)
> #define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
>
> #define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
> @@ -55,6 +56,14 @@
> #define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
> #define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
>
> +#define DBTR_TDATA1_ICOUNT_U BIT(6)
> +#define DBTR_TDATA1_ICOUNT_S BIT(7)
> +#define DBTR_TDATA1_ICOUNT_PENDING BIT(8)
> +#define DBTR_TDATA1_ICOUNT_M BIT(9)
> +#define DBTR_TDATA1_ICOUNT_COUNT_FIELD GENMASK(23, 10)
> +#define DBTR_TDATA1_ICOUNT_VU BIT(25)
> +#define DBTR_TDATA1_ICOUNT_VS BIT(26)
> +
> /* Registered per-cpu bp/wp */
> static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
> static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
> @@ -65,6 +74,7 @@ static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
>
> /* number of debug triggers on this cpu . */
> static int dbtr_total_num __ro_after_init;
> +static bool have_icount __ro_after_init;
> static unsigned long dbtr_type __ro_after_init;
> static unsigned long dbtr_init __ro_after_init;
>
> @@ -168,6 +178,7 @@ static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
> static void init_sbi_dbtr(void)
> {
> struct sbiret ret;
> + unsigned long dbtr_count = 0;
>
> /*
> * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
> @@ -182,6 +193,25 @@ static void init_sbi_dbtr(void)
> return;
> }
>
> + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> + DBTR_TDATA1_TYPE_ICOUNT, 0, 0, 0, 0, 0);
> + if (ret.error) {
> + pr_warn("%s: failed to detect icount triggers. error: %ld.\n",
> + __func__, ret.error);
> + } else if (!ret.value) {
> + if (IS_ENABLED(CONFIG_HW_BREAKPOINT_COMPUTE_STEP)) {
> + pr_warn("%s: No icount triggers available. "
> + "Falling-back to computing single step address.\n", __func__);
> + } else {
> + pr_err("%s: No icount triggers available.\n", __func__);
> + dbtr_total_num = 0;
> + return;
> + }
> + } else {
> + dbtr_count = ret.value;
> + have_icount = true;
> + }
> +
> ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
> if (ret.error) {
> @@ -190,7 +220,7 @@ static void init_sbi_dbtr(void)
> } else if (!ret.value) {
> pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
> } else {
> - dbtr_total_num = ret.value;
> + dbtr_total_num = min_not_zero((unsigned long)ret.value, dbtr_count);
> dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
> return;
> }
> @@ -205,7 +235,7 @@ static void init_sbi_dbtr(void)
> pr_err("%s: No mcontrol triggers available.\n", __func__);
> dbtr_total_num = 0;
> } else {
> - dbtr_total_num = ret.value;
> + dbtr_total_num = min_not_zero((unsigned long)ret.value, dbtr_count);
> dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
> }
> }
> @@ -344,6 +374,21 @@ static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
> return 0;
> }
>
> +static int rv_init_icount_trigger(struct arch_hw_breakpoint *hw)
> +{
> + unsigned long tdata1 = DBTR_TDATA1_TYPE_ICOUNT;
> +
> + /* Step one instruction */
> + tdata1 |= FIELD_PREP(DBTR_TDATA1_ICOUNT_COUNT_FIELD, 1);
> +
> + tdata1 |= DBTR_TDATA1_ICOUNT_U;
> +
> + hw->tdata1 = tdata1;
> + hw->tdata2 = 0;
> +
> + return 0;
> +}
> +
> int hw_breakpoint_arch_parse(struct perf_event *bp,
> const struct perf_event_attr *attr,
> struct arch_hw_breakpoint *hw)
> @@ -389,24 +434,28 @@ static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
> /* Remove breakpoint even if return error as not to loop */
> arch_uninstall_hw_breakpoint(event);
>
> - ret = get_insn(regs, regs->epc, &insn);
> - if (ret < 0)
> - return ret;
> + if (have_icount) {
> + rv_init_icount_trigger(bp);
> + } else {
> + ret = get_insn(regs, regs->epc, &insn);
> + if (ret < 0)
> + return ret;
>
> - next_addr = get_step_address(regs, insn);
> + next_addr = get_step_address(regs, insn);
>
> - ret = get_insn(regs, next_addr, &insn);
> - if (ret < 0)
> - return ret;
> + ret = get_insn(regs, next_addr, &insn);
> + if (ret < 0)
> + return ret;
>
> - bp_insn.bp_type = HW_BREAKPOINT_X;
> - bp_insn.bp_addr = next_addr;
> - /* Get the size of the intruction */
> - bp_insn.bp_len = GET_INSN_LENGTH(insn);
> + bp_insn.bp_type = HW_BREAKPOINT_X;
> + bp_insn.bp_addr = next_addr;
> + /* Get the size of the intruction */
> + bp_insn.bp_len = GET_INSN_LENGTH(insn);
>
> - ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
> - if (ret)
> - return ret;
> + ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
> + if (ret)
> + return ret;
> + }
>
> ret = arch_install_hw_breakpoint(event);
> if (ret)
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support
2025-07-22 17:38 ` [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support Jesse Taube
2025-07-23 4:18 ` Deepak Gupta
@ 2025-08-04 8:16 ` Clément Léger
1 sibling, 0 replies; 18+ messages in thread
From: Clément Léger @ 2025-08-04 8:16 UTC (permalink / raw)
To: Jesse Taube, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Oleg Nesterov, Himanshu Chauhan, Charlie Jenkins, Samuel Holland,
Deepak Gupta, Andrew Jones, Atish Patra, Anup Patel,
Mayuresh Chitale, Conor Dooley, WangYuli, Huacai Chen, Nam Cao,
Andrew Morton, Mike Rapoport (Microsoft), Luis Chamberlain,
Yunhui Cui, Joel Granados, Celeste Liu, Evan Green, Nylon Chen
On 22/07/2025 19:38, Jesse Taube wrote:
> Add ability to setup hw breakpoints to ptrace. Call defines a new
> structure of (ulong[3]){bp_addr, bp_len, bp_type} with
> bp_type being one of HW_BREAKPOINT_LEN_X and
> bp_len being one of HW_BREAKPOINT_X with a value of
> zero dissabling the breakpoint.
>
> Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> ---
> arch/riscv/include/asm/processor.h | 4 ++
> arch/riscv/include/uapi/asm/ptrace.h | 3 +-
> arch/riscv/kernel/hw_breakpoint.c | 14 ++++-
> arch/riscv/kernel/process.c | 4 ++
> arch/riscv/kernel/ptrace.c | 93 ++++++++++++++++++++++++++++
> 5 files changed, 116 insertions(+), 2 deletions(-)
>
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index 5f56eb9d114a..488d956a951f 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,7 @@
>
> #include <vdso/processor.h>
>
> +#include <asm/hw_breakpoint.h>
> #include <asm/ptrace.h>
>
> #define arch_get_mmap_end(addr, len, flags) \
> @@ -108,6 +109,9 @@ struct thread_struct {
> struct __riscv_v_ext_state vstate;
> unsigned long align_ctl;
> struct __riscv_v_ext_state kernel_vstate;
> +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> + struct perf_event *ptrace_bps[RV_MAX_TRIGGERS];
> +#endif
> #ifdef CONFIG_SMP
> /* Flush the icache on migration */
> bool force_icache_flush;
> diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
> index a38268b19c3d..a7998ed41913 100644
> --- a/arch/riscv/include/uapi/asm/ptrace.h
> +++ b/arch/riscv/include/uapi/asm/ptrace.h
> @@ -14,7 +14,8 @@
>
> #define PTRACE_GETFDPIC_EXEC 0
> #define PTRACE_GETFDPIC_INTERP 1
> -
> +#define PTRACE_GETHBPREGS 2
> +#define PTRACE_SETHBPREGS 3
> /*
> * User-mode register state for core dumps, ptrace, sigcontext
> *
> diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> index 437fd82b9590..c58145464539 100644
> --- a/arch/riscv/kernel/hw_breakpoint.c
> +++ b/arch/riscv/kernel/hw_breakpoint.c
> @@ -633,7 +633,19 @@ void arch_uninstall_hw_breakpoint(struct perf_event *event)
> pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> }
>
> -void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
> +/*
> + * Release the user breakpoints used by ptrace
> + */
> +void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
> +{
> + int i;
> + struct thread_struct *t = &tsk->thread;
> +
> + for (i = 0; i < dbtr_total_num; i++) {
> + unregister_hw_breakpoint(t->ptrace_bps[i]);
> + t->ptrace_bps[i] = NULL;
> + }
> +}
Hi Jesse,
You can discard my comment in the previous patch about implementing this
function ;)
Thanks,
Clément
>
> void hw_breakpoint_pmu_read(struct perf_event *bp) { }
>
> diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> index 15d8f75902f8..9cf07ecfb523 100644
> --- a/arch/riscv/kernel/process.c
> +++ b/arch/riscv/kernel/process.c
> @@ -9,6 +9,7 @@
>
> #include <linux/bitfield.h>
> #include <linux/cpu.h>
> +#include <linux/hw_breakpoint.h>
> #include <linux/kernel.h>
> #include <linux/sched.h>
> #include <linux/sched/debug.h>
> @@ -164,6 +165,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
>
> void flush_thread(void)
> {
> + flush_ptrace_hw_breakpoint(current);
> #ifdef CONFIG_FPU
> /*
> * Reset FPU state and context
> @@ -218,6 +220,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> set_bit(MM_CONTEXT_LOCK_PMLEN, &p->mm->context.flags);
>
> memset(&p->thread.s, 0, sizeof(p->thread.s));
> + if (IS_ENABLED(CONFIG_HAVE_HW_BREAKPOINT))
> + memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
>
> /* p->thread holds context to be restored by __switch_to() */
> if (unlikely(args->fn)) {
> diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
> index ea67e9fb7a58..b78cfb0f1c0e 100644
> --- a/arch/riscv/kernel/ptrace.c
> +++ b/arch/riscv/kernel/ptrace.c
> @@ -9,11 +9,13 @@
>
> #include <asm/vector.h>
> #include <asm/ptrace.h>
> +#include <asm/hw_breakpoint.h>
> #include <asm/syscall.h>
> #include <asm/thread_info.h>
> #include <asm/switch_to.h>
> #include <linux/audit.h>
> #include <linux/compat.h>
> +#include <linux/hw_breakpoint.h>
> #include <linux/ptrace.h>
> #include <linux/elf.h>
> #include <linux/regset.h>
> @@ -336,12 +338,103 @@ void ptrace_disable(struct task_struct *child)
> {
> }
>
> +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> +static void ptrace_hbptriggered(struct perf_event *bp,
> + struct perf_sample_data *data,
> + struct pt_regs *regs)
> +{
> + struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
> + int num = 0;
> +
> + force_sig_ptrace_errno_trap(num, (void __user *)bkpt->address);
> +}
> +
> +/*
> + * idx selects the breakpoint index.
> + * Both PTRACE_GETHBPREGS and PTRACE_SETHBPREGS transfer three 32-bit words:
> + * address (0), length (1), type (2).
> + * Instruction breakpoint length is one of HW_BREAKPOINT_LEN_X or 0. 0 will
> + * disable the breakpoint.
> + * Instruction breakpoint type is one of HW_BREAKPOINT_X.
> + */
> +
> +static long ptrace_gethbpregs(struct task_struct *child, unsigned long idx,
> + unsigned long __user *datap)
> +{
> + struct perf_event *bp;
> + unsigned long user_data[3] = {0};
> +
> + if (idx >= RV_MAX_TRIGGERS)
> + return -EINVAL;
> +
> + bp = child->thread.ptrace_bps[idx];
> +
> + if (!IS_ERR_OR_NULL(bp)) {
> + user_data[0] = bp->attr.bp_addr;
> + user_data[1] = bp->attr.disabled ? 0 : bp->attr.bp_len;
> + user_data[2] = bp->attr.bp_type;
> + }
> +
> + if (copy_to_user(datap, user_data, sizeof(user_data)))
> + return -EFAULT;
> +
> + return 0;
> +}
> +
> +static long ptrace_sethbpregs(struct task_struct *child, unsigned long idx,
> + unsigned long __user *datap)
> +{
> + struct perf_event *bp;
> + struct perf_event_attr attr;
> + unsigned long user_data[3];
> +
> + if (idx >= RV_MAX_TRIGGERS)
> + return -EINVAL;
> +
> + if (copy_from_user(user_data, datap, sizeof(user_data)))
> + return -EFAULT;
> +
> + bp = child->thread.ptrace_bps[idx];
> + if (IS_ERR_OR_NULL(bp))
> + attr = bp->attr;
> + else
> + ptrace_breakpoint_init(&attr);
> +
> + attr.bp_addr = user_data[0];
> + attr.bp_len = user_data[1];
> + attr.bp_type = user_data[2];
> + attr.disabled = !attr.bp_len;
> +
> + if (IS_ERR_OR_NULL(bp)) {
> + bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL,
> + child);
> + if (IS_ERR(bp))
> + return PTR_ERR(bp);
> +
> + child->thread.ptrace_bps[idx] = bp;
> + return 0;
> + } else {
> + return modify_user_hw_breakpoint(bp, &attr);
> + }
> +}
> +#endif
> +
> long arch_ptrace(struct task_struct *child, long request,
> unsigned long addr, unsigned long data)
> {
> long ret = -EIO;
> + unsigned long __user *datap = (unsigned long __user *) data;
>
> switch (request) {
> +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> + case PTRACE_GETHBPREGS:
> + ret = ptrace_gethbpregs(child, addr, datap);
> + break;
> +
> + case PTRACE_SETHBPREGS:
> + ret = ptrace_sethbpregs(child, addr, datap);
> + break;
> +#endif
> default:
> ret = ptrace_request(child, request, addr, data);
> break;
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints
2025-08-04 8:01 ` Clément Léger
@ 2025-08-04 15:03 ` Jesse Taube
0 siblings, 0 replies; 18+ messages in thread
From: Jesse Taube @ 2025-08-04 15:03 UTC (permalink / raw)
To: Clément Léger
Cc: linux-riscv, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Oleg Nesterov, Himanshu Chauhan,
Charlie Jenkins, Samuel Holland, Deepak Gupta, Andrew Jones,
Atish Patra, Anup Patel, Mayuresh Chitale, Conor Dooley, WangYuli,
Huacai Chen, Nam Cao, Andrew Morton, Mike Rapoport (Microsoft),
Luis Chamberlain, Yunhui Cui, Joel Granados, Celeste Liu,
Evan Green, Nylon Chen
On Mon, Aug 4, 2025 at 1:01 AM Clément Léger <cleger@rivosinc.com> wrote:
>
>
>
> On 22/07/2025 19:38, Jesse Taube wrote:
> > From: Himanshu Chauhan <hchauhan@ventanamicro.com>
> >
> > RISC-V hardware breakpoint framework is built on top of perf subsystem and uses
> > SBI debug trigger extension to install/uninstall/update/enable/disable hardware
> > triggers as specified in Sdtrig ISA extension.
> >
> > Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
> > Signed-off-by: Jesse Taube <jesse@rivosinc.com>
> > ---
> > arch/riscv/Kconfig | 1 +
> > arch/riscv/include/asm/hw_breakpoint.h | 60 +++
> > arch/riscv/include/asm/kdebug.h | 3 +-
> > arch/riscv/include/asm/sbi.h | 4 +-
> > arch/riscv/kernel/Makefile | 1 +
> > arch/riscv/kernel/hw_breakpoint.c | 620 +++++++++++++++++++++++++
> > arch/riscv/kernel/traps.c | 6 +
> > 7 files changed, 693 insertions(+), 2 deletions(-)
> > create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
> > create mode 100644 arch/riscv/kernel/hw_breakpoint.c
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index bbec87b79309..95d3047cab10 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -163,6 +163,7 @@ config RISCV
> > select HAVE_FUNCTION_ERROR_INJECTION
> > select HAVE_GCC_PLUGINS
> > select HAVE_GENERIC_VDSO if MMU && 64BIT
> > + select HAVE_HW_BREAKPOINT if PERF_EVENTS && RISCV_SBI
> > select HAVE_IRQ_TIME_ACCOUNTING
> > select HAVE_KERNEL_BZIP2 if !XIP_KERNEL && !EFI_ZBOOT
> > select HAVE_KERNEL_GZIP if !XIP_KERNEL && !EFI_ZBOOT
> > diff --git a/arch/riscv/include/asm/hw_breakpoint.h b/arch/riscv/include/asm/hw_breakpoint.h
> > new file mode 100644
> > index 000000000000..8efa3921c535
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/hw_breakpoint.h
> > @@ -0,0 +1,60 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2024 Ventana Micro Systems Inc.
> > + */
> > +
> > +#ifndef __RISCV_HW_BREAKPOINT_H
> > +#define __RISCV_HW_BREAKPOINT_H
> > +
> > +struct task_struct;
> > +
> > +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> > +
> > +#include <uapi/linux/hw_breakpoint.h>
> > +
> > +#if __riscv_xlen == 64
> > +#define cpu_to_le cpu_to_le64
> > +#define le_to_cpu le64_to_cpu
> > +#elif __riscv_xlen == 32
> > +#define cpu_to_le cpu_to_le32
> > +#define le_to_cpu le32_to_cpu
> > +#else
> > +#error "Unexpected __riscv_xlen"
> > +#endif
> > +
> > +struct arch_hw_breakpoint {
> > + unsigned long address;
> > + unsigned long len;
> > +
> > + /* Callback info */
> > + unsigned long next_addr;
> > + bool in_callback;
> > +
> > +
> > + /* Trigger configuration data */
> > + unsigned long tdata1;
> > + unsigned long tdata2;
> > + unsigned long tdata3;
> > +};
> > +
> > +/* Maximum number of hardware breakpoints supported */
> > +#define RV_MAX_TRIGGERS 32
> > +
> > +struct perf_event_attr;
> > +struct notifier_block;
> > +struct perf_event;
> > +struct pt_regs;
> > +
> > +int hw_breakpoint_slots(int type);
> > +int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
> > +int hw_breakpoint_arch_parse(struct perf_event *bp,
> > + const struct perf_event_attr *attr,
> > + struct arch_hw_breakpoint *hw);
> > +int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> > + unsigned long val, void *data);
> > +int arch_install_hw_breakpoint(struct perf_event *bp);
> > +void arch_uninstall_hw_breakpoint(struct perf_event *bp);
> > +void hw_breakpoint_pmu_read(struct perf_event *bp);
> > +
> > +#endif /* CONFIG_HAVE_HW_BREAKPOINT */
> > +#endif /* __RISCV_HW_BREAKPOINT_H */
> > diff --git a/arch/riscv/include/asm/kdebug.h b/arch/riscv/include/asm/kdebug.h
> > index 85ac00411f6e..53e989781aa1 100644
> > --- a/arch/riscv/include/asm/kdebug.h
> > +++ b/arch/riscv/include/asm/kdebug.h
> > @@ -6,7 +6,8 @@
> > enum die_val {
> > DIE_UNUSED,
> > DIE_TRAP,
> > - DIE_OOPS
> > + DIE_OOPS,
> > + DIE_DEBUG
> > };
> >
> > #endif
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index be2ca8e8a49e..64fa7a82aa45 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -282,7 +282,9 @@ struct sbi_sta_struct {
> > u8 pad[47];
> > } __packed;
> >
> > -#define SBI_SHMEM_DISABLE -1
> > +#define SBI_SHMEM_DISABLE (-1UL)
> > +#define SBI_SHMEM_LO(pa) ((unsigned long)lower_32_bits(pa))
> > +#define SBI_SHMEM_HI(pa) ((unsigned long)upper_32_bits(pa))
> >
> > enum sbi_ext_nacl_fid {
> > SBI_EXT_NACL_PROBE_FEATURE = 0x0,
> > diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> > index 4f719b09e5ad..3e72505734bd 100644
> > --- a/arch/riscv/kernel/Makefile
> > +++ b/arch/riscv/kernel/Makefile
> > @@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
> >
> > obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
> > obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
> > +obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
> > obj-$(CONFIG_RISCV_SBI) += sbi.o sbi_ecall.o
> > ifeq ($(CONFIG_RISCV_SBI), y)
> > obj-$(CONFIG_SMP) += sbi-ipi.o
> > diff --git a/arch/riscv/kernel/hw_breakpoint.c b/arch/riscv/kernel/hw_breakpoint.c
> > new file mode 100644
> > index 000000000000..9e3a3b82d300
> > --- /dev/null
> > +++ b/arch/riscv/kernel/hw_breakpoint.c
> > @@ -0,0 +1,620 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * Copyright (C) 2024 Ventana Micro Systems Inc.
> > + */
> > +
> > +#include <linux/hw_breakpoint.h>
> > +#include <linux/perf_event.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/percpu.h>
> > +#include <linux/kdebug.h>
> > +#include <linux/bitops.h>
> > +#include <linux/bitfield.h>
> > +#include <linux/cpu.h>
> > +#include <linux/cpuhotplug.h>
> > +
> > +#include <asm/insn.h>
> > +#include <asm/sbi.h>
> > +
> > +#define DBTR_TDATA1_TYPE_SHIFT (__riscv_xlen - 4)
> > +#define DBTR_TDATA1_DMODE BIT_UL(__riscv_xlen - 5)
> > +
> > +#define DBTR_TDATA1_TYPE_MCONTROL (2UL << DBTR_TDATA1_TYPE_SHIFT)
> > +#define DBTR_TDATA1_TYPE_MCONTROL6 (6UL << DBTR_TDATA1_TYPE_SHIFT)
> > +
> > +#define DBTR_TDATA1_MCONTROL6_LOAD BIT(0)
> > +#define DBTR_TDATA1_MCONTROL6_STORE BIT(1)
> > +#define DBTR_TDATA1_MCONTROL6_EXECUTE BIT(2)
> > +#define DBTR_TDATA1_MCONTROL6_U BIT(3)
> > +#define DBTR_TDATA1_MCONTROL6_S BIT(4)
> > +#define DBTR_TDATA1_MCONTROL6_M BIT(6)
> > +#define DBTR_TDATA1_MCONTROL6_SIZE_FIELD GENMASK(18, 16)
> > +#define DBTR_TDATA1_MCONTROL6_SELECT BIT(21)
> > +#define DBTR_TDATA1_MCONTROL6_VU BIT(23)
> > +#define DBTR_TDATA1_MCONTROL6_VS BIT(24)
> > +
> > +#define DBTR_TDATA1_MCONTROL6_SIZE_8BIT 1
> > +#define DBTR_TDATA1_MCONTROL6_SIZE_16BIT 2
> > +#define DBTR_TDATA1_MCONTROL6_SIZE_32BIT 3
> > +#define DBTR_TDATA1_MCONTROL6_SIZE_64BIT 5
> > +
> > +#define DBTR_TDATA1_MCONTROL_LOAD BIT(0)
> > +#define DBTR_TDATA1_MCONTROL_STORE BIT(1)
> > +#define DBTR_TDATA1_MCONTROL_EXECUTE BIT(2)
> > +#define DBTR_TDATA1_MCONTROL_U BIT(3)
> > +#define DBTR_TDATA1_MCONTROL_S BIT(4)
> > +#define DBTR_TDATA1_MCONTROL_M BIT(6)
> > +#define DBTR_TDATA1_MCONTROL_SIZELO_FIELD GENMASK(17, 16)
> > +#define DBTR_TDATA1_MCONTROL_SELECT BIT(19)
> > +#define DBTR_TDATA1_MCONTROL_SIZEHI_FIELD GENMASK(22, 21)
> > +
> > +#define DBTR_TDATA1_MCONTROL_SIZELO_8BIT 1
> > +#define DBTR_TDATA1_MCONTROL_SIZELO_16BIT 2
> > +#define DBTR_TDATA1_MCONTROL_SIZELO_32BIT 3
> > +/* value of 5 split across HI and LO */
> > +#define DBTR_TDATA1_MCONTROL_SIZELO_64BIT 1
> > +#define DBTR_TDATA1_MCONTROL_SIZEHI_64BIT 1
> > +
> > +/* Registered per-cpu bp/wp */
> > +static DEFINE_PER_CPU(struct perf_event *, pcpu_hw_bp_events[RV_MAX_TRIGGERS]);
> > +static DEFINE_PER_CPU(unsigned long, ecall_lock_flags);
> > +static DEFINE_PER_CPU(raw_spinlock_t, ecall_lock);
> > +
> > +/* Per-cpu shared memory between S and M mode */
> > +static DEFINE_PER_CPU(union sbi_dbtr_shmem_entry, sbi_dbtr_shmem);
> > +
> > +/* number of debug triggers on this cpu . */
> > +static int dbtr_total_num __ro_after_init;
> > +static unsigned long dbtr_type __ro_after_init;
> > +static unsigned long dbtr_init __ro_after_init;
> > +
> > +static int arch_smp_setup_sbi_shmem(unsigned int cpu)
> > +{
> > + union sbi_dbtr_shmem_entry *dbtr_shmem;
> > + unsigned long shmem_pa;
> > + struct sbiret ret;
> > + int rc;
> > +
> > + dbtr_shmem = per_cpu_ptr(&sbi_dbtr_shmem, cpu);
> > + if (!dbtr_shmem) {
> > + pr_err("Invalid per-cpu shared memory for debug triggers\n");
> > + return -ENODEV;
> > + }
> > +
> > + shmem_pa = virt_to_phys(dbtr_shmem);
> > +
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> > + SBI_SHMEM_LO(shmem_pa), SBI_SHMEM_HI(shmem_pa), 0, 0, 0, 0);
> > +
> > + if (ret.error) {
> > + switch (ret.error) {
> > + case SBI_ERR_DENIED:
> > + pr_warn("%s: Access denied for shared memory at %lx\n",
> > + __func__, shmem_pa);
> > + rc = -EPERM;
> > + break;
> > +
> > + case SBI_ERR_INVALID_PARAM:
> > + case SBI_ERR_INVALID_ADDRESS:
> > + pr_warn("%s: Invalid address parameter (%ld)\n",
> > + __func__, ret.error);
> > + rc = -EINVAL;
> > + break;
> > +
> > + case SBI_ERR_ALREADY_AVAILABLE:
> > + pr_warn("%s: Shared memory is already set\n",
> > + __func__);
> > + rc = -EADDRINUSE;
> > + break;
> > +
> > + case SBI_ERR_FAILURE:
> > + pr_err("%s: Internal sdtrig state error\n",
> > + __func__);
> > + rc = -ENXIO;
> > + break;
> > +
> > + default:
> > + pr_warn("%s: Unknown error %lu\n", __func__, ret.error);
> > + rc = -ENXIO;
> > + break;
> > + }
> > + }
>
> Hi Jesse,
>
> Is there a reason not to use sbi_err_map_linux_errno() ?
No, I kept this from Himanshu Chauhan's original RFC.
I changed it to:
if (ret.error) {
pr_warn("%s: failed to setup shared memory. error: %ld\n",
__func__, ret.error);
return sbi_err_map_linux_errno(ret.error);
}
>
> > +
> > + pr_debug("CPU %d: HW Breakpoint shared memory registered.\n", cpu);
> > +
> > + return rc;
> > +}
> > +
> > +static int arch_smp_teardown_sbi_shmem(unsigned int cpu)
> > +{
> > + struct sbiret ret;
> > +
> > + /* Disable shared memory */
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_SETUP_SHMEM,
> > + SBI_SHMEM_DISABLE, SBI_SHMEM_DISABLE, 0, 0, 0, 0);
> > +
> > + if (ret.error) {
> > + switch (ret.error) {
> > + case SBI_ERR_DENIED:
> > + pr_err("%s: Access denied for shared memory.\n",
> > + __func__);
> > + break;
> > +
> > + case SBI_ERR_INVALID_PARAM:
> > + case SBI_ERR_INVALID_ADDRESS:
> > + pr_err("%s: Invalid address parameter (%lu)\n",
> > + __func__, ret.error);
> > + break;
> > +
> > + case SBI_ERR_ALREADY_AVAILABLE:
> > + pr_err("%s: Shared memory is already set\n",
> > + __func__);
> > + break;
> > + case SBI_ERR_FAILURE:
> > + pr_err("%s: Internal sdtrig state error\n",
> > + __func__);
> > + break;
> > + default:
> > + pr_err("%s: Unknown error %lu\n", __func__, ret.error);
> > + break;
> > + }
> > + }
>
> Ditto
>
> > +
> > + pr_debug("CPU %d: HW Breakpoint shared memory disabled.\n", cpu);
> > +
> > + return 0;
> > +}
> > +
> > +static void init_sbi_dbtr(void)
> > +{
> > + struct sbiret ret;
> > +
> > + /*
> > + * Called by hw_breakpoint_slots and arch_hw_breakpoint_init.
> > + * Only proceed if this is the first CPU to reach this code.
> > + */
> > + if (test_and_set_bit(0, &dbtr_init))
> > + return;
> > +
> > + if (sbi_probe_extension(SBI_EXT_DBTR) <= 0) {
> > + pr_debug("%s: SBI_EXT_DBTR is not supported\n", __func__);
> > + dbtr_total_num = 0;
> > + return;
> > + }
> > +
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> > + DBTR_TDATA1_TYPE_MCONTROL6, 0, 0, 0, 0, 0);
> > + if (ret.error) {
> > + pr_warn("%s: failed to detect mcontrol6 triggers. error: %ld.\n",
> > + __func__, ret.error);
> > + } else if (!ret.value) {
> > + pr_warn("%s: No mcontrol6 triggers available.\n", __func__);
> > + } else {
> > + dbtr_total_num = ret.value;
> > + dbtr_type = DBTR_TDATA1_TYPE_MCONTROL6;
> > + return;
> > + }
> > +
> > + /* fallback to legacy mcontrol triggers if mcontrol6 is not available */
>
> Nit: Since all other comments starts with a capital letter, I'd suggest
> using a capital F (Fallback).
>
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_NUM_TRIGGERS,
> > + DBTR_TDATA1_TYPE_MCONTROL, 0, 0, 0, 0, 0);
> > + if (ret.error) {
> > + pr_warn("%s: failed to detect mcontrol triggers. error: %ld.\n",
> > + __func__, ret.error);
> > + } else if (!ret.value) {
> > + pr_err("%s: No mcontrol triggers available.\n", __func__);
> > + dbtr_total_num = 0;
> > + } else {
> > + dbtr_total_num = ret.value;
> > + dbtr_type = DBTR_TDATA1_TYPE_MCONTROL;
> > + }
> > +}
> > +
> > +int hw_breakpoint_slots(int type)
> > +{
> > + /*
> > + * We can be called early, so don't rely on
> > + * static variables being initialised.
> > + */
> > + init_sbi_dbtr();
> > +
> > + return dbtr_total_num;
> > +}
> > +
> > +int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
> > +{
> > + unsigned int len;
> > + unsigned long va;
> > +
> > + va = hw->address;
> > + len = hw->len;
> > +
> > + return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
> > +}
> > +
> > +static int rv_init_mcontrol_trigger(const struct perf_event_attr *attr,
> > + struct arch_hw_breakpoint *hw)
> > +{
> > + unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> > +
> > + switch (attr->bp_type) {
> > + case HW_BREAKPOINT_X:
> > + tdata1 |= DBTR_TDATA1_MCONTROL_EXECUTE;
> > + break;
> > + case HW_BREAKPOINT_R:
> > + tdata1 |= DBTR_TDATA1_MCONTROL_LOAD;
> > + break;
> > + case HW_BREAKPOINT_W:
> > + tdata1 |= DBTR_TDATA1_MCONTROL_STORE;
> > + break;
> > + case HW_BREAKPOINT_RW:
> > + tdata1 |= DBTR_TDATA1_MCONTROL_STORE | DBTR_TDATA1_MCONTROL_LOAD;
> > + break;
> > + default:
> > + return -EINVAL;
> > + }
> > +
> > + switch (attr->bp_len) {
> > + case HW_BREAKPOINT_LEN_1:
> > + hw->len = 1;
> You can probably add a define for that to avoid
>
> #define TDATA1(lo, hi) \
> FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD, \
> DBTR_TDATA1_MCONTROL_SIZELO_64BIT) | \
> FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD, \
> DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
Good idea! I changed the name to TDATA1_MCTRL_SZ and also added
TDATA1_MCTRL6_SZ
>
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> > + DBTR_TDATA1_MCONTROL_SIZELO_8BIT);
>
> tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_8BIT, 0);
>
> > + break;
> > + case HW_BREAKPOINT_LEN_2:
> > + hw->len = 2;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> > + DBTR_TDATA1_MCONTROL_SIZELO_16BIT);
>
> tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_16BIT, 0);
>
> > + break;
> > + case HW_BREAKPOINT_LEN_4:
> > + hw->len = 4;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> > + DBTR_TDATA1_MCONTROL_SIZELO_32BIT);
>
> tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_32BIT, 0);
>
> > + break;
> > +#if __riscv_xlen >= 64
> > + case HW_BREAKPOINT_LEN_8:
> > + hw->len = 8;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZELO_FIELD,
> > + DBTR_TDATA1_MCONTROL_SIZELO_64BIT) |
> > + FIELD_PREP(DBTR_TDATA1_MCONTROL_SIZEHI_FIELD,
> > + DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
>
>
> tdata1 |= TDATA1(DBTR_TDATA1_MCONTROL_SIZELO_64BIT,
> DBTR_TDATA1_MCONTROL_SIZEHI_64BIT);
>
> > + break;
> > +#endif
> > + default:
> > + return -EINVAL;
> > + }
> > +
> > + tdata1 |= DBTR_TDATA1_MCONTROL_U;
> > +
> > + hw->tdata1 = tdata1;
> > +
> > + return 0;
> > +}
> > +
> > +static int rv_init_mcontrol6_trigger(const struct perf_event_attr *attr,
> > + struct arch_hw_breakpoint *hw)
> > +{
> > + unsigned long tdata1 = DBTR_TDATA1_TYPE_MCONTROL;
> > +
> > + switch (attr->bp_type) {
> > + case HW_BREAKPOINT_X:
> > + tdata1 |= DBTR_TDATA1_MCONTROL6_EXECUTE;
> > + break;
> > + case HW_BREAKPOINT_R:
> > + tdata1 |= DBTR_TDATA1_MCONTROL6_LOAD;
> > + break;
> > + case HW_BREAKPOINT_W:
> > + tdata1 |= DBTR_TDATA1_MCONTROL6_STORE;
> > + break;
> > + case HW_BREAKPOINT_RW:
> > + tdata1 |= DBTR_TDATA1_MCONTROL6_STORE | DBTR_TDATA1_MCONTROL6_LOAD;
> > + break;
> > + default:
> > + return -EINVAL;
> > + }
> > +
> > + switch (attr->bp_len) {
> > + case HW_BREAKPOINT_LEN_1:
> > + hw->len = 1;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> > + DBTR_TDATA1_MCONTROL6_SIZE_8BIT);
> > + break;
> > + case HW_BREAKPOINT_LEN_2:
> > + hw->len = 2;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> > + DBTR_TDATA1_MCONTROL6_SIZE_16BIT);
> > + break;
> > + case HW_BREAKPOINT_LEN_4:
> > + hw->len = 4;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> > + DBTR_TDATA1_MCONTROL6_SIZE_32BIT);
> > + break;
> > + case HW_BREAKPOINT_LEN_8:
> > + hw->len = 8;
> > + tdata1 |= FIELD_PREP(DBTR_TDATA1_MCONTROL6_SIZE_FIELD,
> > + DBTR_TDATA1_MCONTROL6_SIZE_64BIT);
> > + break;
> > + default:
> > + return -EINVAL;
> > + }
> > +
> > + tdata1 |= DBTR_TDATA1_MCONTROL6_U;
> > +
> > + hw->tdata1 = tdata1;
> > +
> > + return 0;
> > +}
> > +
> > +int hw_breakpoint_arch_parse(struct perf_event *bp,
> > + const struct perf_event_attr *attr,
> > + struct arch_hw_breakpoint *hw)
> > +{
> > + int ret;
> > +
> > + /* Breakpoint address */
> > + hw->address = attr->bp_addr;
> > + hw->tdata2 = attr->bp_addr;
> > + hw->tdata3 = 0x0;
> > + hw->next_addr = 0x0;
> > + hw->in_callback = false;
> > +
> > + switch (dbtr_type) {
> > + case DBTR_TDATA1_TYPE_MCONTROL:
> > + ret = rv_init_mcontrol_trigger(attr, hw);
> > + break;
> > + case DBTR_TDATA1_TYPE_MCONTROL6:
> > + ret = rv_init_mcontrol6_trigger(attr, hw);
> > + break;
> > + default:
> > + pr_warn("Unsupported trigger type %lu.\n", dbtr_type >> DBTR_TDATA1_TYPE_SHIFT);
> > + ret = -EOPNOTSUPP;
> > + break;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +/*
> > + * Set breakpoint to next insruction after breakpoint.
>
> Typo: instruction
>
> > + * Returns 0 if success
> > + * Returns < 0 on error
> > + */
> > +static int setup_singlestep(struct perf_event *event, struct pt_regs *regs)
> > +{
> > + struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> > + struct arch_hw_breakpoint old_hw_bp;
> > + struct perf_event_attr bp_insn;
> > + unsigned long next_addr, insn;
> > + int ret;
> > +
> > + /* Remove breakpoint even if return error as not to loop */
> > + arch_uninstall_hw_breakpoint(event);
> > +
> > + ret = get_insn(regs, regs->epc, &insn);
> > + if (ret < 0)
> > + return ret;
> > +
> > + next_addr = get_step_address(regs, insn);
> > +
> > + ret = get_insn(regs, next_addr, &insn);
> > + if (ret < 0)
> > + return ret;
> > +
> > + bp_insn.bp_type = HW_BREAKPOINT_X;
> > + bp_insn.bp_addr = next_addr;
> > + /* Get the size of the intruction */
> > + bp_insn.bp_len = GET_INSN_LENGTH(insn);
> > +
> > + ret = hw_breakpoint_arch_parse(NULL, &bp_insn, bp);
> > + if (ret)
> > + return ret;
> > +
> > + ret = arch_install_hw_breakpoint(event);
> > + if (ret)
> > + return ret;
> > +
> > + bp->in_callback = true;
> > + bp->next_addr = next_addr;
> > + return 0;
> > +}
> > +
> > +/*
> > + * HW Breakpoint/watchpoint handler
> > + */
> > +static int hw_breakpoint_handler(struct pt_regs *regs)
> > +{
> > + struct perf_event *event;
> > + struct arch_hw_breakpoint *bp;
> > + int ret, i;
> > +
> > + for (i = 0; i < dbtr_total_num; i++) {
> > + event = this_cpu_read(pcpu_hw_bp_events[i]);
> > + if (!event)
> > + continue;
> > +
> > + bp = counter_arch_bp(event);
> > + if (bp->in_callback) {
> > + /* Reset changed breakpoint data */
> > + bp->in_callback = false;
> > + if (regs->epc == bp->next_addr) {
> > + arch_uninstall_hw_breakpoint(event);
> > + /* Restore original breakpoint */
> > + if (hw_breakpoint_arch_parse(NULL, &event->attr, bp))
> > + return NOTIFY_DONE;
> > + if (arch_install_hw_breakpoint(event))
> > + return NOTIFY_DONE;
> > + return NOTIFY_STOP;
> > + }
> > +
> > + pr_err("%s: in_callback was set, but epc(%lx) was not next "
> > + "address(%lx).\n", __func__, regs->epc, bp->next_addr);
> > + bp->next_addr = 0x0;
> > + return NOTIFY_DONE;
> > + }
> > +
> > + switch (event->attr.bp_type) {
> > + /* Breakpoint */
> > + case HW_BREAKPOINT_X:
> > + if (event->attr.bp_addr == regs->epc) {
> > + ret = setup_singlestep(event, regs);
> > + if (ret < 0) {
> > + pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> > + return NOTIFY_DONE;
> > + }
> > +
> > + perf_bp_event(event, regs);
> > + return NOTIFY_STOP;
> > + }
> > + break;
> > +
> > + /* Watchpoint */
> > + case HW_BREAKPOINT_W:
> > + case HW_BREAKPOINT_R:
> > + case HW_BREAKPOINT_RW:
> > + if (event->attr.bp_addr == regs->badaddr) {
> > + ret = setup_singlestep(event, regs);
> > + if (ret < 0) {
> > + pr_err("%s: setup_singlestep failed %d.\n", __func__, ret);
> > + return NOTIFY_DONE;
> > + }
> > +
> > + perf_bp_event(event, regs);
> > + return NOTIFY_STOP;
> > + }
> > + break;
> > +
> > + default:
> > + pr_warn("%s: Unknown type: %u\n", __func__, event->attr.bp_type);
> > + break;
> > + }
> > + }
> > +
> > + return NOTIFY_DONE;
> > +}
> > +
> > +int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
> > + unsigned long val, void *data)
> > +{
> > + struct die_args *args = data;
> > +
> > + if (val != DIE_DEBUG)
> > + return NOTIFY_DONE;
> > +
> > + return hw_breakpoint_handler(args->regs);
> > +}
> > +
> > +/* atomic: counter->ctx->lock is held */
> > +int arch_install_hw_breakpoint(struct perf_event *event)
> > +{
> > + struct arch_hw_breakpoint *bp = counter_arch_bp(event);
> > + union sbi_dbtr_shmem_entry *shmem = this_cpu_ptr(&sbi_dbtr_shmem);
> > + struct sbi_dbtr_data_msg *xmit;
> > + struct sbi_dbtr_id_msg *recv;
> > + struct perf_event **slot;
> > + unsigned long idx;
> > + struct sbiret ret;
> > + int err = 0;
> > +
> > + raw_spin_lock_irqsave(this_cpu_ptr(&ecall_lock),
> > + *this_cpu_ptr(&ecall_lock_flags));
> > +
> > + xmit = &shmem->data;
> > + recv = &shmem->id;
> > + xmit->tdata1 = cpu_to_le(bp->tdata1);
> > + xmit->tdata2 = cpu_to_le(bp->tdata2);
> > + xmit->tdata3 = cpu_to_le(bp->tdata3);
> > +
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_INSTALL,
> > + 1, 0, 0, 0, 0, 0);
> > +
> > + if (ret.error) {
> > + pr_warn("%s: failed to install trigger. error: %ld\n", __func__, ret.error);
> > + err = sbi_err_map_linux_errno(ret.error);
> > + goto done;
> > + }
> > +
> > + idx = le_to_cpu(recv->idx);
> > + if (idx >= dbtr_total_num) {
> > + pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
> > + err = -EINVAL;
> > + goto done;
> > + }
> > +
> > + slot = this_cpu_ptr(&pcpu_hw_bp_events[idx]);
> > + if (*slot) {
> > + pr_warn("%s: slot %lu is in use\n", __func__, idx);
> > + err = -EBUSY;
> > + goto done;
> > + }
> > +
> > + pr_debug("Trigger 0x%lx installed at index 0x%lx\n", bp->tdata2, idx);
> > +
> > + /* Save the event - to be looked up in handler */
> > + *slot = event;
> > +
> > +done:
> > + raw_spin_unlock_irqrestore(this_cpu_ptr(&ecall_lock),
> > + *this_cpu_ptr(&ecall_lock_flags));
> > + return err;
> > +}
> > +
> > +/* atomic: counter->ctx->lock is held */
> > +void arch_uninstall_hw_breakpoint(struct perf_event *event)
> > +{
> > + struct sbiret ret;
> > + int i;
> > +
> > + for (i = 0; i < dbtr_total_num; i++) {
> > + struct perf_event **slot = this_cpu_ptr(&pcpu_hw_bp_events[i]);
> > +
> > + if (*slot == event) {
> > + *slot = NULL;
> > + break;
> > + }
> > + }
> > +
> > +
> > + if (i == dbtr_total_num) {
> > + pr_warn("%s: Breakpoint not installed.\n", __func__);
> > + return;
> > + }
> > +
> > + ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIG_UNINSTALL,
> > + i, 1, 0, 0, 0, 0);
> > + if (ret.error)
> > + pr_warn("%s: Failed to uninstall trigger %d. error: %ld\n", __func__, i, ret.error);
> > +}
> > +
> > +void flush_ptrace_hw_breakpoint(struct task_struct *tsk) { }
>
> Is there nothing to do to remove the registered breakpoints ? All other
> architecture seems to implement it.
As you noticed in the next patch I did add it. This function is only
necessary if adding ptrace support.
Thanks,
Jesse Taube
>
> > +
> > +void hw_breakpoint_pmu_read(struct perf_event *bp) { }
> > +
> > +static int __init arch_hw_breakpoint_init(void)
> > +{
> > + unsigned int cpu;
> > + int rc = 0;
> > +
> > + for_each_possible_cpu(cpu)
> > + raw_spin_lock_init(&per_cpu(ecall_lock, cpu));
> > +
> > + init_sbi_dbtr();
> > +
> > + if (dbtr_total_num) {
> > + pr_debug("%s: total number of type %lu triggers: %u\n",
> > + __func__, dbtr_type >> DBTR_TDATA1_TYPE_SHIFT, dbtr_total_num);
> > + } else {
> > + pr_debug("%s: No hardware triggers available\n", __func__);
> > + return rc;
> > + }
> > +
> > + /* Hotplug handler to register/unregister shared memory with SBI */
> > + rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> > + "riscv/hw_breakpoint:prepare",
> > + arch_smp_setup_sbi_shmem,
> > + arch_smp_teardown_sbi_shmem);
> > +
> > + if (rc < 0)
> > + pr_warn("%s: Failed to setup CPU hotplug state\n", __func__);
> > +
> > + return rc;
> > +}
> > +arch_initcall(arch_hw_breakpoint_init);
> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > index 938a8b841f94..2ac471ec79a8 100644
> > --- a/arch/riscv/kernel/traps.c
> > +++ b/arch/riscv/kernel/traps.c
> > @@ -289,6 +289,12 @@ void handle_break(struct pt_regs *regs)
> > if (probe_breakpoint_handler(regs))
> > return;
> >
> > +#ifdef CONFIG_HAVE_HW_BREAKPOINT
> > + if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
> > + == NOTIFY_STOP)
> > + return;
> > +#endif
> > +
> > current->thread.bad_cause = regs->cause;
> >
> > if (user_mode(regs))
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2025-08-04 15:03 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-22 17:38 [RFC PATCH 0/6] riscv: add initial support for hardware breakpoints Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 1/6] riscv: Add insn.c, consolidate instruction decoding Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 2/6] riscv: Add SBI debug trigger extension and function ids Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 3/6] riscv: insn: __read_insn use copy_from_X_nofault Jesse Taube
2025-08-04 7:44 ` Clément Léger
2025-07-22 17:38 ` [RFC PATCH 4/6] riscv: Introduce support for hardware break/watchpoints Jesse Taube
2025-07-23 2:49 ` Deepak Gupta
2025-07-23 17:02 ` Jesse Taube
2025-07-23 17:25 ` Deepak Gupta
2025-08-04 8:01 ` Clément Léger
2025-08-04 15:03 ` Jesse Taube
2025-07-22 17:38 ` [RFC PATCH 5/6] riscv: hw_breakpoint: Use icount for single stepping Jesse Taube
2025-08-04 8:15 ` Clément Léger
2025-07-22 17:38 ` [RFC PATCH 6/6] riscv: ptrace: Add hw breakpoint support Jesse Taube
2025-07-23 4:18 ` Deepak Gupta
2025-07-23 16:55 ` Jesse Taube
2025-07-23 17:23 ` Deepak Gupta
2025-08-04 8:16 ` Clément Léger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).