* [PATCH v8 18/27] arm64/hw_breakpoint: Add arch_reinstall_hw_breakpoint
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Add arch_reinstall_hw_breakpoint() to restore a hardware breakpoint
in an atomic context. Unlike the full uninstall and reallocation
path, this lightweight function re-establishes an existing breakpoint
efficiently and safely.
This aligns ARM64 with x86 support for atomic breakpoint reinstalls.
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/hw_breakpoint.h | 1 +
arch/arm64/kernel/hw_breakpoint.c | 5 +++++
3 files changed, 7 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 6663ffd23f25..fa35dfa2f5cc 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -232,6 +232,7 @@ config ARM64
select HAVE_HARDLOCKUP_DETECTOR_PERF if PERF_EVENTS && \
HW_PERF_EVENTS && HAVE_PERF_EVENTS_NMI
select HAVE_HW_BREAKPOINT if PERF_EVENTS
+ select HAVE_REINSTALL_HW_BREAKPOINT if PERF_EVENTS
select HAVE_IOREMAP_PROT
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_LIVEPATCH
diff --git a/arch/arm64/include/asm/hw_breakpoint.h b/arch/arm64/include/asm/hw_breakpoint.h
index bd81cf17744a..6c98bbbc6aa6 100644
--- a/arch/arm64/include/asm/hw_breakpoint.h
+++ b/arch/arm64/include/asm/hw_breakpoint.h
@@ -119,6 +119,7 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
unsigned long val, void *data);
extern int arch_install_hw_breakpoint(struct perf_event *bp);
+extern int arch_reinstall_hw_breakpoint(struct perf_event *bp);
extern void arch_uninstall_hw_breakpoint(struct perf_event *bp);
extern void hw_breakpoint_pmu_read(struct perf_event *bp);
extern int hw_breakpoint_slots(int type);
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index ab76b36dce82..bd7d23d7893d 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -292,6 +292,11 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
return hw_breakpoint_control(bp, HW_BREAKPOINT_INSTALL);
}
+int arch_reinstall_hw_breakpoint(struct perf_event *bp)
+{
+ return hw_breakpoint_control(bp, HW_BREAKPOINT_RESTORE);
+}
+
void arch_uninstall_hw_breakpoint(struct perf_event *bp)
{
hw_breakpoint_control(bp, HW_BREAKPOINT_UNINSTALL);
--
2.43.0
^ permalink raw reply related
* [PATCH v8 17/27] mm/ksw: add KSTACKWATCH_PROFILING to measure probe cost
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
CONFIG_KSTACKWATCH_PROFILING enables runtime measurement of KStackWatch
probe latencies. When profiling is enabled, KStackWatch collects
entry/exit latencies in its probe callbacks. When KStackWatch is
disabled by clearing its config file, the previously collected statistics
are printed.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/Kconfig | 10 +++
mm/kstackwatch/stack.c | 185 ++++++++++++++++++++++++++++++++++++++---
2 files changed, 183 insertions(+), 12 deletions(-)
diff --git a/mm/kstackwatch/Kconfig b/mm/kstackwatch/Kconfig
index 496caf264f35..3c9385a15c33 100644
--- a/mm/kstackwatch/Kconfig
+++ b/mm/kstackwatch/Kconfig
@@ -12,3 +12,13 @@ config KSTACKWATCH
introduce minor overhead during runtime monitoring.
If unsure, say N.
+
+config KSTACKWATCH_PROFILING
+ bool "KStackWatch profiling"
+ depends on KSTACKWATCH
+ help
+ Measure probe latency and overhead in KStackWatch. It records
+ entry/exit probe times (ns and cycles) and shows statistics when
+ stopping. Useful for performance tuning, not for production use.
+
+ If unsure, say N.
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
index 3455d1e70db9..72ae2d3adeec 100644
--- a/mm/kstackwatch/stack.c
+++ b/mm/kstackwatch/stack.c
@@ -6,7 +6,10 @@
#include <linux/kprobes.h>
#include <linux/kstackwatch.h>
#include <linux/kstackwatch_types.h>
+#include <linux/ktime.h>
+#include <linux/percpu.h>
#include <linux/printk.h>
+#include <linux/timex.h>
#define MAX_CANARY_SEARCH_STEPS 128
static struct kprobe entry_probe;
@@ -15,6 +18,120 @@ static struct fprobe exit_probe;
static bool probe_enable;
static u16 probe_generation;
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+struct measure_data {
+ u64 total_entry_with_watch_ns;
+ u64 total_entry_with_watch_cycles;
+ u64 total_entry_without_watch_ns;
+ u64 total_entry_without_watch_cycles;
+ u64 total_exit_with_watch_ns;
+ u64 total_exit_with_watch_cycles;
+ u64 total_exit_without_watch_ns;
+ u64 total_exit_without_watch_cycles;
+ u64 entry_with_watch_count;
+ u64 entry_without_watch_count;
+ u64 exit_with_watch_count;
+ u64 exit_without_watch_count;
+};
+
+static DEFINE_PER_CPU(struct measure_data, measure_stats);
+
+struct measure_ctx {
+ u64 ns_start;
+ u64 cycles_start;
+};
+
+static __always_inline void measure_start(struct measure_ctx *ctx)
+{
+ ctx->ns_start = ktime_get_ns();
+ ctx->cycles_start = get_cycles();
+}
+
+static __always_inline void measure_end(struct measure_ctx *ctx, u64 *total_ns,
+ u64 *total_cycles, u64 *count)
+{
+ u64 ns_end = ktime_get_ns();
+ u64 c_end = get_cycles();
+
+ *total_ns += ns_end - ctx->ns_start;
+ *total_cycles += c_end - ctx->cycles_start;
+ (*count)++;
+}
+
+static void show_measure_stats(void)
+{
+ int cpu;
+ struct measure_data sum = {};
+
+ for_each_possible_cpu(cpu) {
+ struct measure_data *md = per_cpu_ptr(&measure_stats, cpu);
+
+ sum.total_entry_with_watch_ns += md->total_entry_with_watch_ns;
+ sum.total_entry_with_watch_cycles +=
+ md->total_entry_with_watch_cycles;
+ sum.total_entry_without_watch_ns +=
+ md->total_entry_without_watch_ns;
+ sum.total_entry_without_watch_cycles +=
+ md->total_entry_without_watch_cycles;
+
+ sum.total_exit_with_watch_ns += md->total_exit_with_watch_ns;
+ sum.total_exit_with_watch_cycles +=
+ md->total_exit_with_watch_cycles;
+ sum.total_exit_without_watch_ns +=
+ md->total_exit_without_watch_ns;
+ sum.total_exit_without_watch_cycles +=
+ md->total_exit_without_watch_cycles;
+
+ sum.entry_with_watch_count += md->entry_with_watch_count;
+ sum.entry_without_watch_count += md->entry_without_watch_count;
+ sum.exit_with_watch_count += md->exit_with_watch_count;
+ sum.exit_without_watch_count += md->exit_without_watch_count;
+ }
+
+#define AVG(ns, cnt) ((cnt) ? ((ns) / (cnt)) : 0ULL)
+
+ pr_info("entry (with watch): %llu ns, %llu cycles (%llu samples)\n",
+ AVG(sum.total_entry_with_watch_ns, sum.entry_with_watch_count),
+ AVG(sum.total_entry_with_watch_cycles,
+ sum.entry_with_watch_count),
+ sum.entry_with_watch_count);
+
+ pr_info("entry (without watch): %llu ns, %llu cycles (%llu samples)\n",
+ AVG(sum.total_entry_without_watch_ns,
+ sum.entry_without_watch_count),
+ AVG(sum.total_entry_without_watch_cycles,
+ sum.entry_without_watch_count),
+ sum.entry_without_watch_count);
+
+ pr_info("exit (with watch): %llu ns, %llu cycles (%llu samples)\n",
+ AVG(sum.total_exit_with_watch_ns, sum.exit_with_watch_count),
+ AVG(sum.total_exit_with_watch_cycles,
+ sum.exit_with_watch_count),
+ sum.exit_with_watch_count);
+
+ pr_info("exit (without watch): %llu ns, %llu cycles (%llu samples)\n",
+ AVG(sum.total_exit_without_watch_ns,
+ sum.exit_without_watch_count),
+ AVG(sum.total_exit_without_watch_cycles,
+ sum.exit_without_watch_count),
+ sum.exit_without_watch_count);
+}
+
+static void reset_measure_stats(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ struct measure_data *md = per_cpu_ptr(&measure_stats, cpu);
+
+ memset(md, 0, sizeof(*md));
+ }
+
+ pr_info("measure stats reset.\n");
+}
+
+#endif
+
static void ksw_reset_ctx(void)
{
struct ksw_ctx *ctx = ¤t->ksw_ctx;
@@ -159,25 +276,28 @@ static void ksw_stack_entry_handler(struct kprobe *p, struct pt_regs *regs,
unsigned long flags)
{
struct ksw_ctx *ctx = ¤t->ksw_ctx;
- ulong stack_pointer;
- ulong watch_addr;
+ ulong stack_pointer, watch_addr;
u16 watch_len;
int ret;
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ struct measure_ctx m;
+ struct measure_data *md = this_cpu_ptr(&measure_stats);
+ bool watched = false;
+
+ measure_start(&m);
+#endif
stack_pointer = kernel_stack_pointer(regs);
- /*
- * triggered more than once, may be in a loop
- */
if (ctx->wp && ctx->sp == stack_pointer)
- return;
+ goto out;
if (!ksw_stack_check_ctx(true))
- return;
+ goto out;
ret = ksw_watch_get(&ctx->wp);
if (ret)
- return;
+ goto out;
ret = ksw_stack_prepare_watch(regs, ksw_get_config(), &watch_addr,
&watch_len);
@@ -185,17 +305,32 @@ static void ksw_stack_entry_handler(struct kprobe *p, struct pt_regs *regs,
ksw_watch_off(ctx->wp);
ctx->wp = NULL;
pr_err("failed to prepare watch target: %d\n", ret);
- return;
+ goto out;
}
ret = ksw_watch_on(ctx->wp, watch_addr, watch_len);
if (ret) {
pr_err("failed to watch on depth:%d addr:0x%lx len:%u %d\n",
ksw_get_config()->depth, watch_addr, watch_len, ret);
- return;
+ goto out;
}
ctx->sp = stack_pointer;
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ watched = true;
+#endif
+
+out:
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ if (watched)
+ measure_end(&m, &md->total_entry_with_watch_ns,
+ &md->total_entry_with_watch_cycles,
+ &md->entry_with_watch_count);
+ else
+ measure_end(&m, &md->total_entry_without_watch_ns,
+ &md->total_entry_without_watch_cycles,
+ &md->entry_without_watch_count);
+#endif
}
static void ksw_stack_exit_handler(struct fprobe *fp, unsigned long ip,
@@ -203,15 +338,36 @@ static void ksw_stack_exit_handler(struct fprobe *fp, unsigned long ip,
struct ftrace_regs *regs, void *data)
{
struct ksw_ctx *ctx = ¤t->ksw_ctx;
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ struct measure_ctx m;
+ struct measure_data *md = this_cpu_ptr(&measure_stats);
+ bool watched = false;
+ measure_start(&m);
+#endif
if (!ksw_stack_check_ctx(false))
- return;
+ goto out;
if (ctx->wp) {
ksw_watch_off(ctx->wp);
ctx->wp = NULL;
ctx->sp = 0;
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ watched = true;
+#endif
}
+
+out:
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ if (watched)
+ measure_end(&m, &md->total_exit_with_watch_ns,
+ &md->total_exit_with_watch_cycles,
+ &md->exit_with_watch_count);
+ else
+ measure_end(&m, &md->total_exit_without_watch_ns,
+ &md->total_exit_without_watch_cycles,
+ &md->exit_without_watch_count);
+#endif
}
int ksw_stack_init(void)
@@ -239,7 +395,9 @@ int ksw_stack_init(void)
unregister_kprobe(&entry_probe);
return ret;
}
-
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ reset_measure_stats();
+#endif
WRITE_ONCE(probe_generation, READ_ONCE(probe_generation) + 1);
WRITE_ONCE(probe_enable, true);
@@ -252,4 +410,7 @@ void ksw_stack_exit(void)
WRITE_ONCE(probe_generation, READ_ONCE(probe_generation) + 1);
unregister_fprobe(&exit_probe);
unregister_kprobe(&entry_probe);
+#ifdef CONFIG_KSTACKWATCH_PROFILING
+ show_measure_stats();
+#endif
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 16/27] mm/ksw: manage probe and HWBP lifecycle via procfs
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Allow dynamic enabling/disabling of KStackWatch through user input of proc.
With this patch, the entire system becomes functional.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/kernel.c | 60 +++++++++++++++++++++++++++++++++++++++--
1 file changed, 58 insertions(+), 2 deletions(-)
diff --git a/mm/kstackwatch/kernel.c b/mm/kstackwatch/kernel.c
index 87fef139f494..a0e676e60692 100644
--- a/mm/kstackwatch/kernel.c
+++ b/mm/kstackwatch/kernel.c
@@ -14,6 +14,43 @@ static struct ksw_config *ksw_config;
static struct dentry *dbgfs_config;
static struct dentry *dbgfs_dir;
+static bool watching_active;
+
+static int ksw_start_watching(void)
+{
+ int ret;
+
+ /*
+ * Watch init will preallocate the HWBP,
+ * so it must happen before stack init
+ */
+ ret = ksw_watch_init();
+ if (ret) {
+ pr_err("ksw_watch_init ret: %d\n", ret);
+ return ret;
+ }
+
+ ret = ksw_stack_init();
+ if (ret) {
+ pr_err("ksw_stack_init ret: %d\n", ret);
+ ksw_watch_exit();
+ return ret;
+ }
+ watching_active = true;
+
+ pr_info("start watching: %s\n", ksw_config->user_input);
+ return 0;
+}
+
+static void ksw_stop_watching(void)
+{
+ ksw_stack_exit();
+ ksw_watch_exit();
+ watching_active = false;
+
+ pr_info("stop watching: %s\n", ksw_config->user_input);
+}
+
struct param_map {
const char *name; /* long name */
const char *short_name; /* short name (2 letters) */
@@ -119,8 +156,18 @@ static int ksw_parse_config(char *buf, struct ksw_config *config)
static ssize_t ksw_dbgfs_read(struct file *file, char __user *buf, size_t count,
loff_t *ppos)
{
- return simple_read_from_buffer(buf, count, ppos, ksw_config->user_input,
- ksw_config->user_input ? strlen(ksw_config->user_input) : 0);
+ const char *out;
+ size_t len;
+
+ if (watching_active && ksw_config->user_input) {
+ out = ksw_config->user_input;
+ len = strlen(out);
+ } else {
+ out = "not watching\n";
+ len = strlen(out);
+ }
+
+ return simple_read_from_buffer(buf, count, ppos, out, len);
}
static ssize_t ksw_dbgfs_write(struct file *file, const char __user *buffer,
@@ -135,6 +182,9 @@ static ssize_t ksw_dbgfs_write(struct file *file, const char __user *buffer,
if (copy_from_user(input, buffer, count))
return -EFAULT;
+ if (watching_active)
+ ksw_stop_watching();
+
input[count] = '\0';
strim(input);
@@ -149,6 +199,12 @@ static ssize_t ksw_dbgfs_write(struct file *file, const char __user *buffer,
return ret;
}
+ ret = ksw_start_watching();
+ if (ret) {
+ pr_err("Failed to start watching with %d\n", ret);
+ return ret;
+ }
+
return count;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 15/27] mm/ksw: limit canary search to current stack frame
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Use the compiler-provided frame pointer when CONFIG_FRAME_POINTER is
enabled to restrict the stack canary search range to the current
function frame. This prevents scanning beyond valid stack bounds and
improves reliability across architectures.
Also add explicit handling for missing CONFIG_STACKPROTECTOR and make
the failure message more visible.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/stack.c | 29 +++++++++++++++++++++--------
1 file changed, 21 insertions(+), 8 deletions(-)
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
index 60371b292915..3455d1e70db9 100644
--- a/mm/kstackwatch/stack.c
+++ b/mm/kstackwatch/stack.c
@@ -64,15 +64,32 @@ static unsigned long ksw_find_stack_canary_addr(struct pt_regs *regs)
unsigned long *stack_ptr, *stack_end, *stack_base;
unsigned long expected_canary;
unsigned int i;
+#ifdef CONFIG_FRAME_POINTER
+ unsigned long *fp = NULL;
+#endif
stack_ptr = (unsigned long *)kernel_stack_pointer(regs);
-
stack_base = (unsigned long *)(current->stack);
- // TODO: limit it to the current frame
stack_end = (unsigned long *)((char *)current->stack + THREAD_SIZE);
+#ifdef CONFIG_FRAME_POINTER
+ /*
+ * Use the compiler-provided frame pointer.
+ * Limit the search to the current frame
+ * Works on any arch that keeps FP when CONFIG_FRAME_POINTER=y.
+ */
+ fp = __builtin_frame_address(0);
+ if (fp > stack_ptr && fp < stack_end)
+ stack_end = fp;
+#endif
+
+#ifdef CONFIG_STACKPROTECTOR
expected_canary = current->stack_canary;
+#else
+ pr_err("no canary without CONFIG_STACKPROTECTOR\n");
+ return 0;
+#endif
if (stack_ptr < stack_base || stack_ptr >= stack_end) {
pr_err("Stack pointer 0x%lx out of bounds [0x%lx, 0x%lx)\n",
@@ -85,15 +102,11 @@ static unsigned long ksw_find_stack_canary_addr(struct pt_regs *regs)
if (&stack_ptr[i] >= stack_end)
break;
- if (stack_ptr[i] == expected_canary) {
- pr_debug("canary found i:%d 0x%lx\n", i,
- (unsigned long)&stack_ptr[i]);
+ if (stack_ptr[i] == expected_canary)
return (unsigned long)&stack_ptr[i];
- }
}
- pr_debug("canary not found in first %d steps\n",
- MAX_CANARY_SEARCH_STEPS);
+ pr_err("canary not found in first %d steps\n", MAX_CANARY_SEARCH_STEPS);
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 14/27] mm/ksw: resolve stack watch addr and len
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Add helpers to find the stack canary or a local variable addr and len
for the probed function based on ksw_get_config(). For canary search,
limits search to a fixed number of steps to avoid scanning the entire
stack. Validates that the computed address and length are within the
kernel stack.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/stack.c | 80 ++++++++++++++++++++++++++++++++++++++++--
1 file changed, 77 insertions(+), 3 deletions(-)
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
index 96014eb4cb12..60371b292915 100644
--- a/mm/kstackwatch/stack.c
+++ b/mm/kstackwatch/stack.c
@@ -8,6 +8,7 @@
#include <linux/kstackwatch_types.h>
#include <linux/printk.h>
+#define MAX_CANARY_SEARCH_STEPS 128
static struct kprobe entry_probe;
static struct fprobe exit_probe;
@@ -58,13 +59,86 @@ static bool ksw_stack_check_ctx(bool entry)
return false;
}
+static unsigned long ksw_find_stack_canary_addr(struct pt_regs *regs)
+{
+ unsigned long *stack_ptr, *stack_end, *stack_base;
+ unsigned long expected_canary;
+ unsigned int i;
+
+ stack_ptr = (unsigned long *)kernel_stack_pointer(regs);
+
+ stack_base = (unsigned long *)(current->stack);
+
+ // TODO: limit it to the current frame
+ stack_end = (unsigned long *)((char *)current->stack + THREAD_SIZE);
+
+ expected_canary = current->stack_canary;
+
+ if (stack_ptr < stack_base || stack_ptr >= stack_end) {
+ pr_err("Stack pointer 0x%lx out of bounds [0x%lx, 0x%lx)\n",
+ (unsigned long)stack_ptr, (unsigned long)stack_base,
+ (unsigned long)stack_end);
+ return 0;
+ }
+
+ for (i = 0; i < MAX_CANARY_SEARCH_STEPS; i++) {
+ if (&stack_ptr[i] >= stack_end)
+ break;
+
+ if (stack_ptr[i] == expected_canary) {
+ pr_debug("canary found i:%d 0x%lx\n", i,
+ (unsigned long)&stack_ptr[i]);
+ return (unsigned long)&stack_ptr[i];
+ }
+ }
+
+ pr_debug("canary not found in first %d steps\n",
+ MAX_CANARY_SEARCH_STEPS);
+ return 0;
+}
+
+static int ksw_stack_validate_addr(unsigned long addr, size_t size)
+{
+ unsigned long stack_start, stack_end;
+
+ if (!addr || !size)
+ return -EINVAL;
+
+ stack_start = (unsigned long)current->stack;
+ stack_end = stack_start + THREAD_SIZE;
+
+ if (addr < stack_start || (addr + size) > stack_end)
+ return -ERANGE;
+
+ return 0;
+}
+
static int ksw_stack_prepare_watch(struct pt_regs *regs,
const struct ksw_config *config,
ulong *watch_addr, u16 *watch_len)
{
- /* implement logic will be added in following patches */
- *watch_addr = 0;
- *watch_len = 0;
+ ulong addr;
+ u16 len;
+
+ if (ksw_get_config()->auto_canary) {
+ addr = ksw_find_stack_canary_addr(regs);
+ if (!addr)
+ return -EINVAL;
+ len = sizeof(ulong);
+ } else {
+ addr = kernel_stack_pointer(regs) + ksw_get_config()->sp_offset;
+ len = ksw_get_config()->watch_len;
+ if (!len)
+ len = sizeof(ulong);
+ }
+
+ if (ksw_stack_validate_addr(addr, len)) {
+ pr_err("invalid stack addr:0x%lx len :%u\n", addr, len);
+ return -EINVAL;
+ }
+
+ *watch_addr = addr;
+ *watch_len = len;
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 13/27] mm/ksw: add per-task ctx tracking
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Each task tracks its depth, stack pointer, and generation. A watchpoint is
enabled only when the configured depth is reached, and disabled on function
exit.
The context is reset when probes are disabled, generation changes, or exit
depth becomes inconsistent.
Duplicate arming on the same frame is skipped.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/stack.c | 67 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
index 3aa02f8370af..96014eb4cb12 100644
--- a/mm/kstackwatch/stack.c
+++ b/mm/kstackwatch/stack.c
@@ -11,6 +11,53 @@
static struct kprobe entry_probe;
static struct fprobe exit_probe;
+static bool probe_enable;
+static u16 probe_generation;
+
+static void ksw_reset_ctx(void)
+{
+ struct ksw_ctx *ctx = ¤t->ksw_ctx;
+
+ if (ctx->wp)
+ ksw_watch_off(ctx->wp);
+
+ ctx->wp = NULL;
+ ctx->sp = 0;
+ ctx->depth = 0;
+ ctx->generation = READ_ONCE(probe_generation);
+}
+
+static bool ksw_stack_check_ctx(bool entry)
+{
+ struct ksw_ctx *ctx = ¤t->ksw_ctx;
+ u16 cur_enable = READ_ONCE(probe_enable);
+ u16 cur_generation = READ_ONCE(probe_generation);
+ u16 cur_depth, target_depth = ksw_get_config()->depth;
+
+ if (!cur_enable) {
+ ksw_reset_ctx();
+ return false;
+ }
+
+ if (ctx->generation != cur_generation)
+ ksw_reset_ctx();
+
+ if (!entry && !ctx->depth) {
+ ksw_reset_ctx();
+ return false;
+ }
+
+ if (entry)
+ cur_depth = ctx->depth++;
+ else
+ cur_depth = --ctx->depth;
+
+ if (cur_depth == target_depth)
+ return true;
+ else
+ return false;
+}
+
static int ksw_stack_prepare_watch(struct pt_regs *regs,
const struct ksw_config *config,
ulong *watch_addr, u16 *watch_len)
@@ -25,10 +72,22 @@ static void ksw_stack_entry_handler(struct kprobe *p, struct pt_regs *regs,
unsigned long flags)
{
struct ksw_ctx *ctx = ¤t->ksw_ctx;
+ ulong stack_pointer;
ulong watch_addr;
u16 watch_len;
int ret;
+ stack_pointer = kernel_stack_pointer(regs);
+
+ /*
+ * triggered more than once, may be in a loop
+ */
+ if (ctx->wp && ctx->sp == stack_pointer)
+ return;
+
+ if (!ksw_stack_check_ctx(true))
+ return;
+
ret = ksw_watch_get(&ctx->wp);
if (ret)
return;
@@ -49,6 +108,7 @@ static void ksw_stack_entry_handler(struct kprobe *p, struct pt_regs *regs,
return;
}
+ ctx->sp = stack_pointer;
}
static void ksw_stack_exit_handler(struct fprobe *fp, unsigned long ip,
@@ -57,6 +117,8 @@ static void ksw_stack_exit_handler(struct fprobe *fp, unsigned long ip,
{
struct ksw_ctx *ctx = ¤t->ksw_ctx;
+ if (!ksw_stack_check_ctx(false))
+ return;
if (ctx->wp) {
ksw_watch_off(ctx->wp);
@@ -91,11 +153,16 @@ int ksw_stack_init(void)
return ret;
}
+ WRITE_ONCE(probe_generation, READ_ONCE(probe_generation) + 1);
+ WRITE_ONCE(probe_enable, true);
+
return 0;
}
void ksw_stack_exit(void)
{
+ WRITE_ONCE(probe_enable, false);
+ WRITE_ONCE(probe_generation, READ_ONCE(probe_generation) + 1);
unregister_fprobe(&exit_probe);
unregister_kprobe(&entry_probe);
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 12/27] mm/ksw: add entry kprobe and exit fprobe management
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Provide ksw_stack_init() and ksw_stack_exit() to manage entry and exit
probes for the target function from ksw_get_config().
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 4 ++
mm/kstackwatch/stack.c | 100 ++++++++++++++++++++++++++++++++++++
2 files changed, 104 insertions(+)
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
index d7ea89c8c6af..afedd9823de9 100644
--- a/include/linux/kstackwatch.h
+++ b/include/linux/kstackwatch.h
@@ -41,6 +41,10 @@ struct ksw_config {
// singleton, only modified in kernel.c
const struct ksw_config *ksw_get_config(void);
+/* stack management */
+int ksw_stack_init(void);
+void ksw_stack_exit(void);
+
/* watch management */
struct ksw_watchpoint {
struct perf_event *__percpu *event;
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
index cec594032515..3aa02f8370af 100644
--- a/mm/kstackwatch/stack.c
+++ b/mm/kstackwatch/stack.c
@@ -1 +1,101 @@
// SPDX-License-Identifier: GPL-2.0
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/atomic.h>
+#include <linux/fprobe.h>
+#include <linux/kprobes.h>
+#include <linux/kstackwatch.h>
+#include <linux/kstackwatch_types.h>
+#include <linux/printk.h>
+
+static struct kprobe entry_probe;
+static struct fprobe exit_probe;
+
+static int ksw_stack_prepare_watch(struct pt_regs *regs,
+ const struct ksw_config *config,
+ ulong *watch_addr, u16 *watch_len)
+{
+ /* implement logic will be added in following patches */
+ *watch_addr = 0;
+ *watch_len = 0;
+ return 0;
+}
+
+static void ksw_stack_entry_handler(struct kprobe *p, struct pt_regs *regs,
+ unsigned long flags)
+{
+ struct ksw_ctx *ctx = ¤t->ksw_ctx;
+ ulong watch_addr;
+ u16 watch_len;
+ int ret;
+
+ ret = ksw_watch_get(&ctx->wp);
+ if (ret)
+ return;
+
+ ret = ksw_stack_prepare_watch(regs, ksw_get_config(), &watch_addr,
+ &watch_len);
+ if (ret) {
+ ksw_watch_off(ctx->wp);
+ ctx->wp = NULL;
+ pr_err("failed to prepare watch target: %d\n", ret);
+ return;
+ }
+
+ ret = ksw_watch_on(ctx->wp, watch_addr, watch_len);
+ if (ret) {
+ pr_err("failed to watch on depth:%d addr:0x%lx len:%u %d\n",
+ ksw_get_config()->depth, watch_addr, watch_len, ret);
+ return;
+ }
+
+}
+
+static void ksw_stack_exit_handler(struct fprobe *fp, unsigned long ip,
+ unsigned long ret_ip,
+ struct ftrace_regs *regs, void *data)
+{
+ struct ksw_ctx *ctx = ¤t->ksw_ctx;
+
+
+ if (ctx->wp) {
+ ksw_watch_off(ctx->wp);
+ ctx->wp = NULL;
+ ctx->sp = 0;
+ }
+}
+
+int ksw_stack_init(void)
+{
+ int ret;
+ char *symbuf = NULL;
+
+ memset(&entry_probe, 0, sizeof(entry_probe));
+ entry_probe.symbol_name = ksw_get_config()->func_name;
+ entry_probe.offset = ksw_get_config()->func_offset;
+ entry_probe.post_handler = ksw_stack_entry_handler;
+ ret = register_kprobe(&entry_probe);
+ if (ret) {
+ pr_err("failed to register kprobe ret %d\n", ret);
+ return ret;
+ }
+
+ memset(&exit_probe, 0, sizeof(exit_probe));
+ exit_probe.exit_handler = ksw_stack_exit_handler;
+ symbuf = (char *)ksw_get_config()->func_name;
+
+ ret = register_fprobe_syms(&exit_probe, (const char **)&symbuf, 1);
+ if (ret < 0) {
+ pr_err("failed to register fprobe ret %d\n", ret);
+ unregister_kprobe(&entry_probe);
+ return ret;
+ }
+
+ return 0;
+}
+
+void ksw_stack_exit(void)
+{
+ unregister_fprobe(&exit_probe);
+ unregister_kprobe(&entry_probe);
+}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 11/27] sched/ksw: add per-task context
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Introduce struct ksw_ctx to enable lockless per-task state
tracking. This is required because KStackWatch operates in NMI context
(via kprobe handler) where traditional locking is unsafe.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch_types.h | 14 ++++++++++++++
include/linux/sched.h | 5 +++++
2 files changed, 19 insertions(+)
create mode 100644 include/linux/kstackwatch_types.h
diff --git a/include/linux/kstackwatch_types.h b/include/linux/kstackwatch_types.h
new file mode 100644
index 000000000000..8c4e9b0f0c6a
--- /dev/null
+++ b/include/linux/kstackwatch_types.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_KSTACKWATCH_TYPES_H
+#define _LINUX_KSTACKWATCH_TYPES_H
+#include <linux/types.h>
+
+struct ksw_watchpoint;
+struct ksw_ctx {
+ struct ksw_watchpoint *wp;
+ ulong sp;
+ u16 depth;
+ u16 generation;
+};
+
+#endif /* _LINUX_KSTACKWATCH_TYPES_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index b469878de25c..db49325428b3 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -22,6 +22,7 @@
#include <linux/sem_types.h>
#include <linux/shm.h>
#include <linux/kmsan_types.h>
+#include <linux/kstackwatch_types.h>
#include <linux/mutex_types.h>
#include <linux/plist_types.h>
#include <linux/hrtimer_types.h>
@@ -1487,6 +1488,10 @@ struct task_struct {
struct kmsan_ctx kmsan_ctx;
#endif
+#if IS_ENABLED(CONFIG_KSTACKWATCH)
+ struct ksw_ctx ksw_ctx;
+#endif
+
#if IS_ENABLED(CONFIG_KUNIT)
struct kunit *kunit_test;
#endif
--
2.43.0
^ permalink raw reply related
* [PATCH v8 10/27] mm/ksw: support CPU hotplug
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Register CPU online/offline callbacks via cpuhp_setup_state_nocalls()
so stack watches are installed/removed dynamically as CPUs come online
or go offline.
When a new CPU comes online, register a hardware breakpoint for the holder,
avoiding races with watch_on()/watch_off() that may run on another CPU. The
watch address will be updated the next time watch_on() is called.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/watch.c | 52 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 52 insertions(+)
diff --git a/mm/kstackwatch/watch.c b/mm/kstackwatch/watch.c
index f922b4164be5..99184f63d7e3 100644
--- a/mm/kstackwatch/watch.c
+++ b/mm/kstackwatch/watch.c
@@ -85,6 +85,48 @@ static void ksw_watch_on_local_cpu(void *info)
WARN(ret, "fail to reinstall HWBP on CPU%d ret %d", cpu, ret);
}
+static int ksw_watch_cpu_online(unsigned int cpu)
+{
+ struct perf_event_attr attr;
+ struct ksw_watchpoint *wp;
+ call_single_data_t *csd;
+ struct perf_event *bp;
+
+ mutex_lock(&all_wp_mutex);
+ list_for_each_entry(wp, &all_wp_list, list) {
+ attr = wp->attr;
+ attr.bp_addr = (u64)&holder;
+ bp = perf_event_create_kernel_counter(&attr, cpu, NULL,
+ ksw_watch_handler, wp);
+ if (IS_ERR(bp)) {
+ pr_warn("%s failed to create watch on CPU %d: %ld\n",
+ __func__, cpu, PTR_ERR(bp));
+ continue;
+ }
+
+ per_cpu(*wp->event, cpu) = bp;
+ csd = per_cpu_ptr(wp->csd, cpu);
+ INIT_CSD(csd, ksw_watch_on_local_cpu, wp);
+ }
+ mutex_unlock(&all_wp_mutex);
+ return 0;
+}
+
+static int ksw_watch_cpu_offline(unsigned int cpu)
+{
+ struct ksw_watchpoint *wp;
+ struct perf_event *bp;
+
+ mutex_lock(&all_wp_mutex);
+ list_for_each_entry(wp, &all_wp_list, list) {
+ bp = per_cpu(*wp->event, cpu);
+ if (bp)
+ unregister_hw_breakpoint(bp);
+ }
+ mutex_unlock(&all_wp_mutex);
+ return 0;
+}
+
static void ksw_watch_update(struct ksw_watchpoint *wp, ulong addr, u16 len)
{
call_single_data_t *csd;
@@ -206,6 +248,16 @@ int ksw_watch_init(void)
if (ret <= 0)
return -EBUSY;
+ ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
+ "kstackwatch:online",
+ ksw_watch_cpu_online,
+ ksw_watch_cpu_offline);
+ if (ret < 0) {
+ ksw_watch_free();
+ pr_err("Failed to register CPU hotplug notifier\n");
+ return ret;
+ }
+
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 09/27] mm/ksw: ignore false positives from exit trampolines
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Because trampolines run after the watched function returns but before the
exit_handler is called, and in the original stack frame, so the trampoline
code may overwrite the watched stack address.
These false positives should be ignored. is_ftrace_trampoline() does
not cover all trampolines, so add a local check to handle the remaining
cases.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
mm/kstackwatch/watch.c | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/mm/kstackwatch/watch.c b/mm/kstackwatch/watch.c
index 3817a172dc25..f922b4164be5 100644
--- a/mm/kstackwatch/watch.c
+++ b/mm/kstackwatch/watch.c
@@ -2,6 +2,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/cpuhotplug.h>
+#include <linux/ftrace.h>
#include <linux/hw_breakpoint.h>
#include <linux/irqflags.h>
#include <linux/kstackwatch.h>
@@ -14,10 +15,46 @@ static DEFINE_MUTEX(all_wp_mutex);
static ulong holder;
+#define TRAMPOLINE_NAME "return_to_handler"
+#define TRAMPOLINE_DEPTH 16
+
+/* Resolved once, then reused */
+static unsigned long tramp_start, tramp_end;
+
+static void ksw_watch_resolve_trampoline(void)
+{
+ unsigned long sz, off;
+
+ if (likely(tramp_start && tramp_end))
+ return;
+
+ tramp_start = kallsyms_lookup_name(TRAMPOLINE_NAME);
+ if (tramp_start && kallsyms_lookup_size_offset(tramp_start, &sz, &off))
+ tramp_end = tramp_start + sz;
+}
+
+static bool ksw_watch_in_trampoline(unsigned long ip)
+{
+ if (tramp_start && tramp_end && ip >= tramp_start && ip < tramp_end)
+ return true;
+ return false;
+}
static void ksw_watch_handler(struct perf_event *bp,
struct perf_sample_data *data,
struct pt_regs *regs)
{
+ unsigned long entries[TRAMPOLINE_DEPTH];
+ int i, nr = 0;
+
+ nr = stack_trace_save_regs(regs, entries, TRAMPOLINE_DEPTH, 0);
+ for (i = 0; i < nr; i++) {
+ //ignore trampoline
+ if (is_ftrace_trampoline(entries[i]))
+ return;
+ if (ksw_watch_in_trampoline(entries[i]))
+ return;
+ }
+
pr_err("========== KStackWatch: Caught stack corruption =======\n");
pr_err("config %s\n", ksw_get_config()->user_input);
dump_stack();
@@ -164,6 +201,7 @@ int ksw_watch_init(void)
{
int ret;
+ ksw_watch_resolve_trampoline();
ret = ksw_watch_alloc();
if (ret <= 0)
return -EBUSY;
--
2.43.0
^ permalink raw reply related
* [PATCH v8 08/27] mm/ksw: Add atomic watchpoint management api
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Add three functions for atomic lifecycle management of watchpoints:
- ksw_watch_get(): Acquires a watchpoint from a llist.
- ksw_watch_on(): Enables the watchpoint on all online CPUs.
- ksw_watch_off(): Disables the watchpoint and returns it to the llist.
For cross-CPU synchronization, updates are propagated using direct
modification on the local CPU and asynchronous IPIs for remote CPUs.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 4 ++
mm/kstackwatch/watch.c | 85 ++++++++++++++++++++++++++++++++++++-
2 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
index eb9f2b4f2109..d7ea89c8c6af 100644
--- a/include/linux/kstackwatch.h
+++ b/include/linux/kstackwatch.h
@@ -44,11 +44,15 @@ const struct ksw_config *ksw_get_config(void);
/* watch management */
struct ksw_watchpoint {
struct perf_event *__percpu *event;
+ call_single_data_t __percpu *csd;
struct perf_event_attr attr;
struct llist_node node; // for atomic watch_on and off
struct list_head list; // for cpu online and offline
};
int ksw_watch_init(void);
void ksw_watch_exit(void);
+int ksw_watch_get(struct ksw_watchpoint **out_wp);
+int ksw_watch_on(struct ksw_watchpoint *wp, ulong watch_addr, u16 watch_len);
+int ksw_watch_off(struct ksw_watchpoint *wp);
#endif /* _KSTACKWATCH_H */
diff --git a/mm/kstackwatch/watch.c b/mm/kstackwatch/watch.c
index 4947eac32c61..3817a172dc25 100644
--- a/mm/kstackwatch/watch.c
+++ b/mm/kstackwatch/watch.c
@@ -27,11 +27,83 @@ static void ksw_watch_handler(struct perf_event *bp,
panic("Stack corruption detected");
}
+static void ksw_watch_on_local_cpu(void *info)
+{
+ struct ksw_watchpoint *wp = info;
+ struct perf_event *bp;
+ ulong flags;
+ int cpu;
+ int ret;
+
+ local_irq_save(flags);
+ cpu = raw_smp_processor_id();
+ bp = per_cpu(*wp->event, cpu);
+ if (!bp) {
+ local_irq_restore(flags);
+ return;
+ }
+
+ ret = modify_wide_hw_breakpoint_local(bp, &wp->attr);
+ local_irq_restore(flags);
+ WARN(ret, "fail to reinstall HWBP on CPU%d ret %d", cpu, ret);
+}
+
+static void ksw_watch_update(struct ksw_watchpoint *wp, ulong addr, u16 len)
+{
+ call_single_data_t *csd;
+ int cur_cpu;
+ int cpu;
+
+ wp->attr.bp_addr = addr;
+ wp->attr.bp_len = len;
+
+ cur_cpu = raw_smp_processor_id();
+ for_each_online_cpu(cpu) {
+ /* remote cpu first */
+ if (cpu == cur_cpu)
+ continue;
+ csd = per_cpu_ptr(wp->csd, cpu);
+ smp_call_function_single_async(cpu, csd);
+ }
+ ksw_watch_on_local_cpu(wp);
+}
+
+int ksw_watch_get(struct ksw_watchpoint **out_wp)
+{
+ struct ksw_watchpoint *wp;
+ struct llist_node *node;
+
+ node = llist_del_first(&free_wp_list);
+ if (!node)
+ return -EBUSY;
+
+ wp = llist_entry(node, struct ksw_watchpoint, node);
+ WARN_ON_ONCE(wp->attr.bp_addr != (u64)&holder);
+
+ *out_wp = wp;
+ return 0;
+}
+int ksw_watch_on(struct ksw_watchpoint *wp, ulong watch_addr, u16 watch_len)
+{
+ ksw_watch_update(wp, watch_addr, watch_len);
+ return 0;
+}
+
+int ksw_watch_off(struct ksw_watchpoint *wp)
+{
+ WARN_ON_ONCE(wp->attr.bp_addr == (u64)&holder);
+ ksw_watch_update(wp, (ulong)&holder, sizeof(ulong));
+ llist_add(&wp->node, &free_wp_list);
+ return 0;
+}
+
static int ksw_watch_alloc(void)
{
int max_watch = ksw_get_config()->max_watch;
struct ksw_watchpoint *wp;
+ call_single_data_t *csd;
int success = 0;
+ int cpu;
int ret;
init_llist_head(&free_wp_list);
@@ -41,6 +113,16 @@ static int ksw_watch_alloc(void)
wp = kzalloc(sizeof(*wp), GFP_KERNEL);
if (!wp)
return success > 0 ? success : -EINVAL;
+ wp->csd = alloc_percpu(call_single_data_t);
+ if (!wp->csd) {
+ kfree(wp);
+ return success > 0 ? success : -EINVAL;
+ }
+
+ for_each_possible_cpu(cpu) {
+ csd = per_cpu_ptr(wp->csd, cpu);
+ INIT_CSD(csd, ksw_watch_on_local_cpu, wp);
+ }
hw_breakpoint_init(&wp->attr);
wp->attr.bp_addr = (ulong)&holder;
@@ -50,6 +132,7 @@ static int ksw_watch_alloc(void)
ksw_watch_handler, wp);
if (IS_ERR((void *)wp->event)) {
ret = PTR_ERR((void *)wp->event);
+ free_percpu(wp->csd);
kfree(wp);
return success > 0 ? success : ret;
}
@@ -71,6 +154,7 @@ static void ksw_watch_free(void)
list_for_each_entry_safe(wp, tmp, &all_wp_list, list) {
list_del(&wp->list);
unregister_wide_hw_breakpoint(wp->event);
+ free_percpu(wp->csd);
kfree(wp);
}
mutex_unlock(&all_wp_mutex);
@@ -84,7 +168,6 @@ int ksw_watch_init(void)
if (ret <= 0)
return -EBUSY;
-
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 07/27] mm/ksw: add HWBP pre-allocation
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Pre-allocate per-CPU hardware breakpoints at init with a place holder
address, which will be retargeted dynamically in kprobe handler.
This avoids allocation in atomic context.
At most max_watch breakpoints are allocated (0 means no limit).
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 13 ++++++
mm/kstackwatch/watch.c | 93 +++++++++++++++++++++++++++++++++++++
2 files changed, 106 insertions(+)
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
index ada5ac64190c..eb9f2b4f2109 100644
--- a/include/linux/kstackwatch.h
+++ b/include/linux/kstackwatch.h
@@ -2,6 +2,9 @@
#ifndef _KSTACKWATCH_H
#define _KSTACKWATCH_H
+#include <linux/llist.h>
+#include <linux/percpu.h>
+#include <linux/perf_event.h>
#include <linux/types.h>
#define MAX_CONFIG_STR_LEN 128
@@ -38,4 +41,14 @@ struct ksw_config {
// singleton, only modified in kernel.c
const struct ksw_config *ksw_get_config(void);
+/* watch management */
+struct ksw_watchpoint {
+ struct perf_event *__percpu *event;
+ struct perf_event_attr attr;
+ struct llist_node node; // for atomic watch_on and off
+ struct list_head list; // for cpu online and offline
+};
+int ksw_watch_init(void);
+void ksw_watch_exit(void);
+
#endif /* _KSTACKWATCH_H */
diff --git a/mm/kstackwatch/watch.c b/mm/kstackwatch/watch.c
index cec594032515..4947eac32c61 100644
--- a/mm/kstackwatch/watch.c
+++ b/mm/kstackwatch/watch.c
@@ -1 +1,94 @@
// SPDX-License-Identifier: GPL-2.0
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/cpuhotplug.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/irqflags.h>
+#include <linux/kstackwatch.h>
+#include <linux/mutex.h>
+#include <linux/printk.h>
+
+static LLIST_HEAD(free_wp_list);
+static LIST_HEAD(all_wp_list);
+static DEFINE_MUTEX(all_wp_mutex);
+
+static ulong holder;
+
+static void ksw_watch_handler(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ pr_err("========== KStackWatch: Caught stack corruption =======\n");
+ pr_err("config %s\n", ksw_get_config()->user_input);
+ dump_stack();
+ pr_err("=================== KStackWatch End ===================\n");
+
+ if (ksw_get_config()->panic_hit)
+ panic("Stack corruption detected");
+}
+
+static int ksw_watch_alloc(void)
+{
+ int max_watch = ksw_get_config()->max_watch;
+ struct ksw_watchpoint *wp;
+ int success = 0;
+ int ret;
+
+ init_llist_head(&free_wp_list);
+
+ //max_watch=0 means at most
+ while (!max_watch || success < max_watch) {
+ wp = kzalloc(sizeof(*wp), GFP_KERNEL);
+ if (!wp)
+ return success > 0 ? success : -EINVAL;
+
+ hw_breakpoint_init(&wp->attr);
+ wp->attr.bp_addr = (ulong)&holder;
+ wp->attr.bp_len = sizeof(ulong);
+ wp->attr.bp_type = HW_BREAKPOINT_W;
+ wp->event = register_wide_hw_breakpoint(&wp->attr,
+ ksw_watch_handler, wp);
+ if (IS_ERR((void *)wp->event)) {
+ ret = PTR_ERR((void *)wp->event);
+ kfree(wp);
+ return success > 0 ? success : ret;
+ }
+ llist_add(&wp->node, &free_wp_list);
+ mutex_lock(&all_wp_mutex);
+ list_add(&wp->list, &all_wp_list);
+ mutex_unlock(&all_wp_mutex);
+ success++;
+ }
+
+ return success;
+}
+
+static void ksw_watch_free(void)
+{
+ struct ksw_watchpoint *wp, *tmp;
+
+ mutex_lock(&all_wp_mutex);
+ list_for_each_entry_safe(wp, tmp, &all_wp_list, list) {
+ list_del(&wp->list);
+ unregister_wide_hw_breakpoint(wp->event);
+ kfree(wp);
+ }
+ mutex_unlock(&all_wp_mutex);
+}
+
+int ksw_watch_init(void)
+{
+ int ret;
+
+ ret = ksw_watch_alloc();
+ if (ret <= 0)
+ return -EBUSY;
+
+
+ return 0;
+}
+
+void ksw_watch_exit(void)
+{
+ ksw_watch_free();
+}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 06/27] mm/ksw: add singleton debugfs interface
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Provide the debugfs config file to read or update the configuration.
Only a single process can open this file at a time, enforced using atomic
config_file_busy, to prevent concurrent access.
ksw_get_config() exposes the configuration pointer as const.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 3 ++
mm/kstackwatch/kernel.c | 103 ++++++++++++++++++++++++++++++++++--
2 files changed, 103 insertions(+), 3 deletions(-)
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
index dd00c4c8922e..ada5ac64190c 100644
--- a/include/linux/kstackwatch.h
+++ b/include/linux/kstackwatch.h
@@ -35,4 +35,7 @@ struct ksw_config {
char *user_input;
};
+// singleton, only modified in kernel.c
+const struct ksw_config *ksw_get_config(void);
+
#endif /* _KSTACKWATCH_H */
diff --git a/mm/kstackwatch/kernel.c b/mm/kstackwatch/kernel.c
index 50104e78cf3d..87fef139f494 100644
--- a/mm/kstackwatch/kernel.c
+++ b/mm/kstackwatch/kernel.c
@@ -1,13 +1,18 @@
// SPDX-License-Identifier: GPL-2.0
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/debugfs.h>
#include <linux/kstackwatch.h>
#include <linux/kstrtox.h>
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/string.h>
+#include <linux/uaccess.h>
+static atomic_t dbgfs_config_busy = ATOMIC_INIT(0);
static struct ksw_config *ksw_config;
+static struct dentry *dbgfs_config;
+static struct dentry *dbgfs_dir;
struct param_map {
const char *name; /* long name */
@@ -76,7 +81,7 @@ static int ksw_parse_param(struct ksw_config *config, const char *key,
* - sp_offset |so (u16) : offset from stack pointer at func_offset
* - watch_len |wl (u16) : watch length (1,2,4,8)
*/
-static int __maybe_unused ksw_parse_config(char *buf, struct ksw_config *config)
+static int ksw_parse_config(char *buf, struct ksw_config *config)
{
char *part, *key, *val;
int ret;
@@ -111,18 +116,110 @@ static int __maybe_unused ksw_parse_config(char *buf, struct ksw_config *config)
return 0;
}
+static ssize_t ksw_dbgfs_read(struct file *file, char __user *buf, size_t count,
+ loff_t *ppos)
+{
+ return simple_read_from_buffer(buf, count, ppos, ksw_config->user_input,
+ ksw_config->user_input ? strlen(ksw_config->user_input) : 0);
+}
+
+static ssize_t ksw_dbgfs_write(struct file *file, const char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ char input[MAX_CONFIG_STR_LEN];
+ int ret;
+
+ if (count == 0 || count >= sizeof(input))
+ return -EINVAL;
+
+ if (copy_from_user(input, buffer, count))
+ return -EFAULT;
+
+ input[count] = '\0';
+ strim(input);
+
+ if (!strlen(input)) {
+ pr_info("config cleared\n");
+ return count;
+ }
+
+ ret = ksw_parse_config(input, ksw_config);
+ if (ret) {
+ pr_err("Failed to parse config %d\n", ret);
+ return ret;
+ }
+
+ return count;
+}
+
+static int ksw_dbgfs_open(struct inode *inode, struct file *file)
+{
+ if (atomic_cmpxchg(&dbgfs_config_busy, 0, 1))
+ return -EBUSY;
+ return 0;
+}
+
+static int ksw_dbgfs_release(struct inode *inode, struct file *file)
+{
+ atomic_set(&dbgfs_config_busy, 0);
+ return 0;
+}
+
+static const struct file_operations kstackwatch_fops = {
+ .owner = THIS_MODULE,
+ .open = ksw_dbgfs_open,
+ .read = ksw_dbgfs_read,
+ .write = ksw_dbgfs_write,
+ .release = ksw_dbgfs_release,
+ .llseek = default_llseek,
+};
+
+const struct ksw_config *ksw_get_config(void)
+{
+ return ksw_config;
+}
+
static int __init kstackwatch_init(void)
{
+ int ret = 0;
+
ksw_config = kzalloc(sizeof(*ksw_config), GFP_KERNEL);
- if (!ksw_config)
- return -ENOMEM;
+ if (!ksw_config) {
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+
+ dbgfs_dir = debugfs_create_dir("kstackwatch", NULL);
+ if (!dbgfs_dir) {
+ ret = -ENOMEM;
+ goto err_dir;
+ }
+
+ dbgfs_config = debugfs_create_file("config", 0600, dbgfs_dir, NULL,
+ &kstackwatch_fops);
+ if (!dbgfs_config) {
+ ret = -ENOMEM;
+ goto err_file;
+ }
pr_info("module loaded\n");
return 0;
+
+err_file:
+ debugfs_remove_recursive(dbgfs_dir);
+ dbgfs_dir = NULL;
+err_dir:
+ kfree(ksw_config);
+ ksw_config = NULL;
+err_alloc:
+ return ret;
}
static void __exit kstackwatch_exit(void)
{
+ debugfs_remove_recursive(dbgfs_dir);
+ kfree(ksw_config->func_name);
+ kfree(ksw_config->user_input);
kfree(ksw_config);
pr_info("module unloaded\n");
--
2.43.0
^ permalink raw reply related
* [PATCH v8 05/27] mm/ksw: add ksw_config struct and parser
From: Jinchao Wang @ 2025-11-10 16:36 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Add struct ksw_config and ksw_parse_config() to parse user string.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 33 +++++++++++
mm/kstackwatch/kernel.c | 114 ++++++++++++++++++++++++++++++++++++
2 files changed, 147 insertions(+)
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
index 0273ef478a26..dd00c4c8922e 100644
--- a/include/linux/kstackwatch.h
+++ b/include/linux/kstackwatch.h
@@ -2,4 +2,37 @@
#ifndef _KSTACKWATCH_H
#define _KSTACKWATCH_H
+#include <linux/types.h>
+
+#define MAX_CONFIG_STR_LEN 128
+
+struct ksw_config {
+ char *func_name;
+ u16 depth;
+
+ /*
+ * watched variable info:
+ * - func_offset : instruction offset in the function, typically the
+ * assignment of the watched variable, where ksw
+ * registers a kprobe post-handler.
+ * - sp_offset : offset from stack pointer at func_offset. Usually 0.
+ * - watch_len : size of the watched variable (1, 2, 4, or 8 bytes).
+ */
+ u16 func_offset;
+ u16 sp_offset;
+ u16 watch_len;
+
+ /* max number of hwbps that can be used */
+ u16 max_watch;
+
+ /* search canary as watch target automatically */
+ u16 auto_canary;
+
+ /* panic on watchpoint hit */
+ u16 panic_hit;
+
+ /* save to show */
+ char *user_input;
+};
+
#endif /* _KSTACKWATCH_H */
diff --git a/mm/kstackwatch/kernel.c b/mm/kstackwatch/kernel.c
index 78f1d019225f..50104e78cf3d 100644
--- a/mm/kstackwatch/kernel.c
+++ b/mm/kstackwatch/kernel.c
@@ -1,16 +1,130 @@
// SPDX-License-Identifier: GPL-2.0
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/kstackwatch.h>
+#include <linux/kstrtox.h>
+#include <linux/slab.h>
#include <linux/module.h>
+#include <linux/string.h>
+
+static struct ksw_config *ksw_config;
+
+struct param_map {
+ const char *name; /* long name */
+ const char *short_name; /* short name (2 letters) */
+ size_t offset; /* offsetof(struct ksw_config, field) */
+ bool is_string; /* true for string */
+};
+
+/* macro generates both long and short name automatically */
+#define PMAP(field, short, is_str) \
+ { #field, #short, offsetof(struct ksw_config, field), is_str }
+
+static const struct param_map ksw_params[] = {
+ PMAP(func_name, fn, true),
+ PMAP(func_offset, fo, false),
+ PMAP(depth, dp, false),
+ PMAP(max_watch, mw, false),
+ PMAP(sp_offset, so, false),
+ PMAP(watch_len, wl, false),
+ PMAP(auto_canary, ac, false),
+ PMAP(panic_hit, ph, false),
+};
+
+static int ksw_parse_param(struct ksw_config *config, const char *key,
+ const char *val)
+{
+ const struct param_map *pm = NULL;
+ int ret;
+
+ for (int i = 0; i < ARRAY_SIZE(ksw_params); i++) {
+ if (strcmp(key, ksw_params[i].name) == 0 ||
+ strcmp(key, ksw_params[i].short_name) == 0) {
+ pm = &ksw_params[i];
+ break;
+ }
+ }
+
+ if (!pm)
+ return -EINVAL;
+
+ if (pm->is_string) {
+ char **dst = (char **)((char *)config + pm->offset);
+ *dst = kstrdup(val, GFP_KERNEL);
+ if (!*dst)
+ return -ENOMEM;
+ } else {
+ ret = kstrtou16(val, 0, (u16 *)((char *)config + pm->offset));
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Configuration string format:
+ * param_name=<value> [param_name=<value> ...]
+ *
+ * Required parameters:
+ * - func_name |fn (str) : target function name
+ * - func_offset|fo (u16) : instruction pointer offset
+ *
+ * Optional parameters:
+ * - depth |dp (u16) : recursion depth
+ * - max_watch |mw (u16) : maximum number of watchpoints
+ * - sp_offset |so (u16) : offset from stack pointer at func_offset
+ * - watch_len |wl (u16) : watch length (1,2,4,8)
+ */
+static int __maybe_unused ksw_parse_config(char *buf, struct ksw_config *config)
+{
+ char *part, *key, *val;
+ int ret;
+
+ kfree(config->func_name);
+ kfree(config->user_input);
+ memset(ksw_config, 0, sizeof(*ksw_config));
+
+ buf = strim(buf);
+ config->user_input = kstrdup(buf, GFP_KERNEL);
+ if (!config->user_input)
+ return -ENOMEM;
+
+ while ((part = strsep(&buf, " \t\n")) != NULL) {
+ if (*part == '\0')
+ continue;
+
+ key = strsep(&part, "=");
+ val = part;
+ if (!key || !val)
+ continue;
+ ret = ksw_parse_param(config, key, val);
+ if (ret)
+ pr_warn("unsupported param %s=%s", key, val);
+ }
+
+ if (!config->func_name) {
+ pr_err("Missing required parameters: function or func_offset\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
static int __init kstackwatch_init(void)
{
+ ksw_config = kzalloc(sizeof(*ksw_config), GFP_KERNEL);
+ if (!ksw_config)
+ return -ENOMEM;
+
pr_info("module loaded\n");
return 0;
}
static void __exit kstackwatch_exit(void)
{
+ kfree(ksw_config);
+
pr_info("module unloaded\n");
}
--
2.43.0
^ permalink raw reply related
* [PATCH v8 04/27] mm/ksw: add build system support
From: Jinchao Wang @ 2025-11-10 16:35 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Add Kconfig and Makefile infrastructure.
The implementation is located under `mm/kstackwatch/`.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
---
include/linux/kstackwatch.h | 5 +++++
mm/Kconfig | 1 +
mm/Makefile | 1 +
mm/kstackwatch/Kconfig | 14 ++++++++++++++
mm/kstackwatch/Makefile | 2 ++
mm/kstackwatch/kernel.c | 23 +++++++++++++++++++++++
mm/kstackwatch/stack.c | 1 +
mm/kstackwatch/watch.c | 1 +
8 files changed, 48 insertions(+)
create mode 100644 include/linux/kstackwatch.h
create mode 100644 mm/kstackwatch/Kconfig
create mode 100644 mm/kstackwatch/Makefile
create mode 100644 mm/kstackwatch/kernel.c
create mode 100644 mm/kstackwatch/stack.c
create mode 100644 mm/kstackwatch/watch.c
diff --git a/include/linux/kstackwatch.h b/include/linux/kstackwatch.h
new file mode 100644
index 000000000000..0273ef478a26
--- /dev/null
+++ b/include/linux/kstackwatch.h
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _KSTACKWATCH_H
+#define _KSTACKWATCH_H
+
+#endif /* _KSTACKWATCH_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index 0e26f4fc8717..61d4e6edadf2 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1373,5 +1373,6 @@ config FIND_NORMAL_PAGE
def_bool n
source "mm/damon/Kconfig"
+source "mm/kstackwatch/Kconfig"
endmenu
diff --git a/mm/Makefile b/mm/Makefile
index 21abb3353550..efc101816f00 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -92,6 +92,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o
obj-$(CONFIG_KASAN) += kasan/
obj-$(CONFIG_KFENCE) += kfence/
obj-$(CONFIG_KMSAN) += kmsan/
+obj-$(CONFIG_KSTACKWATCH) += kstackwatch/
obj-$(CONFIG_FAILSLAB) += failslab.o
obj-$(CONFIG_FAIL_PAGE_ALLOC) += fail_page_alloc.o
obj-$(CONFIG_MEMTEST) += memtest.o
diff --git a/mm/kstackwatch/Kconfig b/mm/kstackwatch/Kconfig
new file mode 100644
index 000000000000..496caf264f35
--- /dev/null
+++ b/mm/kstackwatch/Kconfig
@@ -0,0 +1,14 @@
+config KSTACKWATCH
+ bool "Kernel Stack Watch"
+ depends on HAVE_HW_BREAKPOINT && KPROBES && FPROBE && STACKTRACE
+ help
+ A lightweight real-time debugging tool to detect stack corruption
+ and abnormal stack usage patterns in the kernel. It monitors stack
+ boundaries and detects overwrites in real time using hardware
+ breakpoints and probe-based instrumentation.
+
+ This feature is intended for kernel developers or advanced users
+ diagnosing rare stack overflow or memory corruption bugs. It may
+ introduce minor overhead during runtime monitoring.
+
+ If unsure, say N.
diff --git a/mm/kstackwatch/Makefile b/mm/kstackwatch/Makefile
new file mode 100644
index 000000000000..c99c621eac02
--- /dev/null
+++ b/mm/kstackwatch/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_KSTACKWATCH) += kstackwatch.o
+kstackwatch-y := kernel.o stack.o watch.o
diff --git a/mm/kstackwatch/kernel.c b/mm/kstackwatch/kernel.c
new file mode 100644
index 000000000000..78f1d019225f
--- /dev/null
+++ b/mm/kstackwatch/kernel.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/module.h>
+
+static int __init kstackwatch_init(void)
+{
+ pr_info("module loaded\n");
+ return 0;
+}
+
+static void __exit kstackwatch_exit(void)
+{
+ pr_info("module unloaded\n");
+}
+
+module_init(kstackwatch_init);
+module_exit(kstackwatch_exit);
+
+MODULE_AUTHOR("Jinchao Wang");
+MODULE_DESCRIPTION("Kernel Stack Watch");
+MODULE_LICENSE("GPL");
+
diff --git a/mm/kstackwatch/stack.c b/mm/kstackwatch/stack.c
new file mode 100644
index 000000000000..cec594032515
--- /dev/null
+++ b/mm/kstackwatch/stack.c
@@ -0,0 +1 @@
+// SPDX-License-Identifier: GPL-2.0
diff --git a/mm/kstackwatch/watch.c b/mm/kstackwatch/watch.c
new file mode 100644
index 000000000000..cec594032515
--- /dev/null
+++ b/mm/kstackwatch/watch.c
@@ -0,0 +1 @@
+// SPDX-License-Identifier: GPL-2.0
--
2.43.0
^ permalink raw reply related
* [PATCH v8 03/27] HWBP: Add modify_wide_hw_breakpoint_local() API
From: Jinchao Wang @ 2025-11-10 16:35 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
From: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Add modify_wide_hw_breakpoint_local() arch-wide interface which allows
hwbp users to update watch address on-line. This is available if the
arch supports CONFIG_HAVE_REINSTALL_HW_BREAKPOINT.
Note that this allows to change the type only for compatible types,
because it does not release and reserve the hwbp slot based on type.
For instance, you can not change HW_BREAKPOINT_W to HW_BREAKPOINT_X.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
arch/Kconfig | 10 ++++++++++
arch/x86/Kconfig | 1 +
include/linux/hw_breakpoint.h | 6 ++++++
kernel/events/hw_breakpoint.c | 37 +++++++++++++++++++++++++++++++++++
4 files changed, 54 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index 61130b88964b..c45fe5366125 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -456,6 +456,16 @@ config HAVE_MIXED_BREAKPOINTS_REGS
Select this option if your arch implements breakpoints under the
latter fashion.
+config HAVE_REINSTALL_HW_BREAKPOINT
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Depending on the arch implementation of hardware breakpoints,
+ some of them are able to update the breakpoint configuration
+ without release and reserve the hardware breakpoint register.
+ What configuration is able to update depends on hardware and
+ software implementation.
+
config HAVE_USER_RETURN_NOTIFIER
bool
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index fa3b616af03a..4d2ef8a45681 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -245,6 +245,7 @@ config X86
select HAVE_FUNCTION_TRACER
select HAVE_GCC_PLUGINS
select HAVE_HW_BREAKPOINT
+ select HAVE_REINSTALL_HW_BREAKPOINT
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64
select HAVE_IRQ_TIME_ACCOUNTING
diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index db199d653dd1..ea373f2587f8 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context);
+extern int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr);
+
extern int register_perf_hw_breakpoint(struct perf_event *bp);
extern void unregister_hw_breakpoint(struct perf_event *bp);
extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);
@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context) { return NULL; }
static inline int
+modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr) { return -ENOSYS; }
+static inline int
register_perf_hw_breakpoint(struct perf_event *bp) { return -ENOSYS; }
static inline void unregister_hw_breakpoint(struct perf_event *bp) { }
static inline void
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index 8ec2cb688903..5ee1522a99c9 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -887,6 +887,43 @@ void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events)
}
EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint);
+/**
+ * modify_wide_hw_breakpoint_local - update breakpoint config for local CPU
+ * @bp: the hwbp perf event for this CPU
+ * @attr: the new attribute for @bp
+ *
+ * This does not release and reserve the slot of a HWBP; it just reuses the
+ * current slot on local CPU. So the users must update the other CPUs by
+ * themselves.
+ * Also, since this does not release/reserve the slot, this can not change the
+ * type to incompatible type of the HWBP.
+ * Return err if attr is invalid or the CPU fails to update debug register
+ * for new @attr.
+ */
+#ifdef CONFIG_HAVE_REINSTALL_HW_BREAKPOINT
+int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr)
+{
+ int ret;
+
+ if (find_slot_idx(bp->attr.bp_type) != find_slot_idx(attr->bp_type))
+ return -EINVAL;
+
+ ret = hw_breakpoint_arch_parse(bp, attr, counter_arch_bp(bp));
+ if (ret)
+ return ret;
+
+ return arch_reinstall_hw_breakpoint(bp);
+}
+#else
+int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+EXPORT_SYMBOL_GPL(modify_wide_hw_breakpoint_local);
+
/**
* hw_breakpoint_is_used - check if breakpoints are currently used
*
--
2.43.0
^ permalink raw reply related
* [PATCH v8 02/27] x86/hw_breakpoint: Add arch_reinstall_hw_breakpoint
From: Jinchao Wang @ 2025-11-10 16:35 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
The new arch_reinstall_hw_breakpoint() function can be used in an
atomic context, unlike the more expensive free and re-allocation path.
This allows callers to efficiently re-establish an existing breakpoint.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
arch/x86/include/asm/hw_breakpoint.h | 2 ++
arch/x86/kernel/hw_breakpoint.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index aa6adac6c3a2..c22cc4e87fc5 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -21,6 +21,7 @@ struct arch_hw_breakpoint {
enum bp_slot_action {
BP_SLOT_ACTION_INSTALL,
+ BP_SLOT_ACTION_REINSTALL,
BP_SLOT_ACTION_UNINSTALL,
};
@@ -65,6 +66,7 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_reinstall_hw_breakpoint(struct perf_event *bp);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index 3658ace4bd8d..29c9369264d4 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -99,6 +99,10 @@ static int manage_bp_slot(struct perf_event *bp, enum bp_slot_action action)
old_bp = NULL;
new_bp = bp;
break;
+ case BP_SLOT_ACTION_REINSTALL:
+ old_bp = bp;
+ new_bp = bp;
+ break;
case BP_SLOT_ACTION_UNINSTALL:
old_bp = bp;
new_bp = NULL;
@@ -187,6 +191,11 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
return arch_manage_bp(bp, BP_SLOT_ACTION_INSTALL);
}
+int arch_reinstall_hw_breakpoint(struct perf_event *bp)
+{
+ return arch_manage_bp(bp, BP_SLOT_ACTION_REINSTALL);
+}
+
void arch_uninstall_hw_breakpoint(struct perf_event *bp)
{
arch_manage_bp(bp, BP_SLOT_ACTION_UNINSTALL);
--
2.43.0
^ permalink raw reply related
* [PATCH v8 01/27] x86/hw_breakpoint: Unify breakpoint install/uninstall
From: Jinchao Wang @ 2025-11-10 16:35 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
In-Reply-To: <20251110163634.3686676-1-wangjinchao600@gmail.com>
Consolidate breakpoint management to reduce code duplication.
The diffstat was misleading, so the stripped code size is compared instead.
After refactoring, it is reduced from 11976 bytes to 11448 bytes on my
x86_64 system built with clang.
This also makes it easier to introduce arch_reinstall_hw_breakpoint().
In addition, including linux/types.h to fix a missing build dependency.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
arch/x86/include/asm/hw_breakpoint.h | 6 ++
arch/x86/kernel/hw_breakpoint.c | 141 +++++++++++++++------------
2 files changed, 84 insertions(+), 63 deletions(-)
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index 0bc931cd0698..aa6adac6c3a2 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -5,6 +5,7 @@
#include <uapi/asm/hw_breakpoint.h>
#define __ARCH_HW_BREAKPOINT_H
+#include <linux/types.h>
/*
* The name should probably be something dealt in
@@ -18,6 +19,11 @@ struct arch_hw_breakpoint {
u8 type;
};
+enum bp_slot_action {
+ BP_SLOT_ACTION_INSTALL,
+ BP_SLOT_ACTION_UNINSTALL,
+};
+
#include <linux/kdebug.h>
#include <linux/percpu.h>
#include <linux/list.h>
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index b01644c949b2..3658ace4bd8d 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -48,7 +48,6 @@ static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]);
*/
static DEFINE_PER_CPU(struct perf_event *, bp_per_reg[HBP_NUM]);
-
static inline unsigned long
__encode_dr7(int drnum, unsigned int len, unsigned int type)
{
@@ -85,96 +84,112 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *len, unsigned *type)
}
/*
- * Install a perf counter breakpoint.
- *
- * We seek a free debug address register and use it for this
- * breakpoint. Eventually we enable it in the debug control register.
- *
- * Atomic: we hold the counter->ctx->lock and we only handle variables
- * and registers local to this cpu.
+ * We seek a slot and change it or keep it based on the action.
+ * Returns slot number on success, negative error on failure.
+ * Must be called with IRQs disabled.
*/
-int arch_install_hw_breakpoint(struct perf_event *bp)
+static int manage_bp_slot(struct perf_event *bp, enum bp_slot_action action)
{
- struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long *dr7;
- int i;
-
- lockdep_assert_irqs_disabled();
+ struct perf_event *old_bp;
+ struct perf_event *new_bp;
+ int slot;
+
+ switch (action) {
+ case BP_SLOT_ACTION_INSTALL:
+ old_bp = NULL;
+ new_bp = bp;
+ break;
+ case BP_SLOT_ACTION_UNINSTALL:
+ old_bp = bp;
+ new_bp = NULL;
+ break;
+ default:
+ return -EINVAL;
+ }
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+ for (slot = 0; slot < HBP_NUM; slot++) {
+ struct perf_event **curr = this_cpu_ptr(&bp_per_reg[slot]);
- if (!*slot) {
- *slot = bp;
- break;
+ if (*curr == old_bp) {
+ *curr = new_bp;
+ return slot;
}
}
- if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
- return -EBUSY;
+ if (old_bp) {
+ WARN_ONCE(1, "Can't find matching breakpoint slot");
+ return -EINVAL;
+ }
+
+ WARN_ONCE(1, "No free breakpoint slots");
+ return -EBUSY;
+}
+
+static void setup_hwbp(struct arch_hw_breakpoint *info, int slot, bool enable)
+{
+ unsigned long dr7;
- set_debugreg(info->address, i);
- __this_cpu_write(cpu_debugreg[i], info->address);
+ set_debugreg(info->address, slot);
+ __this_cpu_write(cpu_debugreg[slot], info->address);
- dr7 = this_cpu_ptr(&cpu_dr7);
- *dr7 |= encode_dr7(i, info->len, info->type);
+ dr7 = this_cpu_read(cpu_dr7);
+ if (enable)
+ dr7 |= encode_dr7(slot, info->len, info->type);
+ else
+ dr7 &= ~__encode_dr7(slot, info->len, info->type);
/*
- * Ensure we first write cpu_dr7 before we set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
+ * Enabling:
+ * Ensure we first write cpu_dr7 before we set the DR7 register.
+ * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
*/
+ if (enable)
+ this_cpu_write(cpu_dr7, dr7);
+
barrier();
- set_debugreg(*dr7, 7);
+ set_debugreg(dr7, 7);
+
if (info->mask)
- amd_set_dr_addr_mask(info->mask, i);
+ amd_set_dr_addr_mask(enable ? info->mask : 0, slot);
- return 0;
+ /*
+ * Disabling:
+ * Ensure the write to cpu_dr7 is after we've set the DR7 register.
+ * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
+ */
+ if (!enable)
+ this_cpu_write(cpu_dr7, dr7);
}
/*
- * Uninstall the breakpoint contained in the given counter.
- *
- * First we search the debug address register it uses and then we disable
- * it.
- *
- * Atomic: we hold the counter->ctx->lock and we only handle variables
- * and registers local to this cpu.
+ * find suitable breakpoint slot and set it up based on the action
*/
-void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+static int arch_manage_bp(struct perf_event *bp, enum bp_slot_action action)
{
- struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long dr7;
- int i;
+ struct arch_hw_breakpoint *info;
+ int slot;
lockdep_assert_irqs_disabled();
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
-
- if (*slot == bp) {
- *slot = NULL;
- break;
- }
- }
-
- if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
- return;
+ slot = manage_bp_slot(bp, action);
+ if (slot < 0)
+ return slot;
- dr7 = this_cpu_read(cpu_dr7);
- dr7 &= ~__encode_dr7(i, info->len, info->type);
+ info = counter_arch_bp(bp);
+ setup_hwbp(info, slot, action != BP_SLOT_ACTION_UNINSTALL);
- set_debugreg(dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(0, i);
+ return 0;
+}
- /*
- * Ensure the write to cpu_dr7 is after we've set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
+int arch_install_hw_breakpoint(struct perf_event *bp)
+{
+ return arch_manage_bp(bp, BP_SLOT_ACTION_INSTALL);
+}
- this_cpu_write(cpu_dr7, dr7);
+void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+{
+ arch_manage_bp(bp, BP_SLOT_ACTION_UNINSTALL);
}
static int arch_bp_generic_len(int x86_len)
--
2.43.0
^ permalink raw reply related
* [PATCH v8 00/27] mm/ksw: Introduce KStackWatch debugging tool
From: Jinchao Wang @ 2025-11-10 16:35 UTC (permalink / raw)
To: Andrew Morton, Masami Hiramatsu (Google), Peter Zijlstra,
Randy Dunlap, Marco Elver, Mike Rapoport, Alexander Potapenko,
Adrian Hunter, Alexander Shishkin, Alice Ryhl, Andrey Konovalov,
Andrey Ryabinin, Andrii Nakryiko, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Ben Segall, Bill Wendling,
Borislav Petkov, Catalin Marinas, Dave Hansen, David Hildenbrand,
David Kaplan, David S. Miller, Dietmar Eggemann, Dmitry Vyukov,
H. Peter Anvin, Ian Rogers, Ingo Molnar, James Clark,
Jinchao Wang, Jinjie Ruan, Jiri Olsa, Jonathan Corbet, Juri Lelli,
Justin Stitt, kasan-dev, Kees Cook, Liam R. Howlett, Liang Kan,
Linus Walleij, linux-arm-kernel, linux-doc, linux-kernel,
linux-mm, linux-perf-users, linux-trace-kernel, llvm,
Lorenzo Stoakes, Mark Rutland, Masahiro Yamada, Mathieu Desnoyers,
Mel Gorman, Michal Hocko, Miguel Ojeda, Nam Cao, Namhyung Kim,
Nathan Chancellor, Naveen N Rao, Nick Desaulniers, Rong Xu,
Sami Tolvanen, Steven Rostedt, Suren Baghdasaryan,
Thomas Gleixner, Thomas Weißschuh, Valentin Schneider,
Vincent Guittot, Vincenzo Frascino, Vlastimil Babka, Will Deacon,
workflows, x86
Earlier this year, I debugged a stack corruption panic that revealed the
limitations of existing debugging tools. The bug persisted for 739 days
before being fixed (CVE-2025-22036), and my reproduction scenario
differed from the CVE report—highlighting how unpredictably these bugs
manifest.
The panic call trace:
<4>[89318.486564] <TASK>
<4>[89318.486570] dump_stack_lvl+0x48/0x70
<4>[89318.486580] dump_stack+0x10/0x20
<4>[89318.486586] panic+0x345/0x3a0
<4>[89318.486596] ? __blk_flush_plug+0x121/0x130
<4>[89318.486603] __stack_chk_fail+0x14/0x20
<4>[89318.486612] __blk_flush_plug+0x121/0x130
...27 other frames omitted
<4>[89318.486824] ksys_read+0x6b/0xf0
<4>[89318.486829] __x64_sys_read+0x19/0x30
<4>[89318.486834] x64_sys_call+0x1ada/0x25c0
<4>[89318.486840] do_syscall_64+0x7f/0x180
<4>[89318.486847] ? exc_page_fault+0x94/0x1b0
<4>[89318.486855] entry_SYSCALL_64_after_hwframe+0x73/0x7b
<4>[89318.486866] </TASK>
Initially, I enabled KASAN, but the bug did not reproduce. Reviewing the
code in __blk_flush_plug(), I found it difficult to trace all logic
paths due to indirect function calls through function pointers.
I added canary-locating code to obtain the canary address and value,
then inserted extensive debugging code to track canary modifications. I
observed the canary being corrupted between two unrelated assignments,
indicating corruption by another thread—a silent stack corruption bug.
I then added hardware breakpoint (hwbp) code, but still failed to catch
the corruption. After adding PID filters, function parameter filters,
and depth filters, I discovered the corruption occurred in
end_buffer_read_sync() via atomic_dec(&bh->b_count), where bh->b_count
overlapped with __blk_flush_plug()'s canary address. Tracing the bh
lifecycle revealed the root cause in exfat_get_block()—a function not
even present in the panic call trace.
This bug was later assigned CVE-2025-22036
(https://lore.kernel.org/all/2025041658-CVE-2025-22036-6469@gregkh/).
The vulnerability was introduced in commit 11a347fb6cef (March 13, 2023)
and fixed in commit 1bb7ff4204b6 (March 21, 2025)—persisting for 739
days. Notably, my reproduction scenario differed significantly from that
described in the CVE report, highlighting how these bugs manifest
unpredictably across different workloads.
This experience revealed how notoriously difficult stack corruption bugs
are to debug: KASAN cannot reproduce them, call traces are misleading,
and the actual culprit often lies outside the visible call chain. Manual
instrumentation with hardware breakpoints and filters was effective but
extremely time-consuming.
This motivated KStackWatch: automating the debugging workflow I manually
performed, making hardware breakpoint-based stack monitoring readily
available to all kernel developers facing similar issues.
KStackWatch is a lightweight debugging tool to detect kernel stack
corruption in real time. It installs a hardware breakpoint (watchpoint)
at a function's specified offset using kprobe.post_handler and removes
it in fprobe.exit_handler. This covers the full execution window and
reports corruption immediately with time, location, and a call stack.
Beyond automating proven debugging workflows, KStackWatch incorporates
robust engineering to handle complex scenarios like context switches,
recursion, and concurrent execution, making it suitable for broad
debugging use cases.
## Key Features
* Immediate and precise stack corruption detection
* Support for multiple concurrent watchpoints with configurable limits
* Lockless design, usable in any context
* Depth filter for recursive calls
* Low overhead of memory and CPU
* Flexible debugfs configuration with key=val syntax
* Architecture support: x86_64 and arm64
* Auto-canary detection to simplify configuration
## Architecture Support
KStackWatch currently supports x86_64 and arm64. The design is
architecture-agnostic, requiring only:
* Hardware breakpoint modification in atomic context
Arm64 support required only ~20 lines of code(patch 18,19). Future ports
to other architectures (e.g., riscv) should be straightforward for
developers familiar with their hardware breakpoint implementations.
## Performance Impact
Runtime overhead was measured on Intel Core Ultra 5 125H @ 3 GHz running
kernel 6.17, using test4 from patch 24:
Type | Time (ns) | Cycles
-----------------------------------------------
entry with watch | 10892 | 32620
entry without watch | 159 | 466
exit with watch | 12541 | 37556
exit without watch | 124 | 369
Comparation with other scenarios:
Mode | CPU Overhead (add) | Memory Overhead (add)
----------------------------+----------------------+-------------------------
Compiled but not enabled | None | ~20 B per task
Enabled, no function hit | None | ~few hundred B
Func hit, HWBP not toggled | ~140 ns per call | None
Func hit, HWBP toggled | ~11–12 µs per call | None
The overhead is minimal, making KStackWatch suitable for production
environments where stack corruption is suspected but kernel rebuilds are not feasible.
## Validation
To validate the approach, this series includes a self-contained test module and
a companion shell script. The module provides several test cases covering
scenarios such as canary overflow, recursive depth tracking, multi-threaded
silent corruption, retaddr overwriten. A detailed workflow example and usage
guide are provided in the documentation (patch 26).
While KStackWatch itself is a new tool and has not yet discovered production
bugs, it automates the exact methodology that I used to manually uncover
CVE-2025-22036. The tool is designed to make this powerful debugging technique
readily available to kernel developers, enabling them to efficiently detect and
diagnose similar stack corruption issues in the future.
---
Patches 1–3 of this series are also used in the wprobe work proposed by
Masami Hiramatsu, so there may be some overlap between our patches.
Patch 3 comes directly from Masami Hiramatsu (thanks).
---
Changelog:
v8:
* Add arm64 support
* Implement hwbp_reinstall() for arm64.
* Use single-step mode as default in ksw_watch_handler().
* Add latency measurements for probe handlers.
* Update configuration options
* Introduce explicit auto_canary parameter.
* Default watch_len to sizeof(unsigned long) when zero.
* Replace panic_on_catch with panic_hit ksw_config option.
* Enable KStackWatch in non-debug builds.
* Limit canary search range to the current stack frame when possible.
* Add automatic architecture detection for test parameters.
* Move kstackwatch.h to include/linux/.
* Relocate Kconfig fragments to the kstackwatch/ directory.
v7:
https://lore.kernel.org/all/20251009105650.168917-1-wangjinchao600@gmail.com/
* Fix maintainer entry to alphabetical position
v6:
https://lore.kernel.org/all/20250930024402.1043776-1-wangjinchao600@gmail.com/
* Replace procfs with debugfs interface
* Fix typos
v5:
https://lore.kernel.org/all/20250924115124.194940-1-wangjinchao600@gmail.com/
* Support key=value input format
* Support multiple watchpoints
* Support watching instruction inside loop
* Support recursion depth tracking with generation
* Ignore triggers from fprobe trampoline
* Split watch_on into watch_get and watch_on to fail fast
* Handle ksw_stack_prepare_watch error
* Rewrite silent corruption test
* Add multiple watchpoints test
* Add an example in documentation
v4:
https://lore.kernel.org/all/20250912101145.465708-1-wangjinchao600@gmail.com/
* Solve the lockdep issues with:
* per-task KStackWatch context to track depth
* atomic flag to protect watched_addr
* Use refactored version of arch_reinstall_hw_breakpoint
v3:
https://lore.kernel.org/all/20250910052335.1151048-1-wangjinchao600@gmail.com/
* Use modify_wide_hw_breakpoint_local() (from Masami)
* Add atomic flag to restrict /proc/kstackwatch to a single opener
* Protect stack probe with an atomic PID flag
* Handle CPU hotplug for watchpoints
* Add preempt_disable/enable in ksw_watch_on_local_cpu()
* Introduce const struct ksw_config *ksw_get_config(void) and use it
* Switch to global watch_attr, remove struct watch_info
* Validate local_var_len in parser()
* Handle case when canary is not found
* Use dump_stack() instead of show_regs() to allow module build
* Reduce logging and comments
* Format logs with KBUILD_MODNAME
* Remove unused headers
* Add new document
v2:
https://lore.kernel.org/all/20250904002126.1514566-1-wangjinchao600@gmail.com/
* Make hardware breakpoint and stack operations
architecture-independent.
v1:
https://lore.kernel.org/all/20250828073311.1116593-1-wangjinchao600@gmail.com/
* Replaced kretprobe with fprobe for function exit hooking, as
suggested by Masami Hiramatsu
* Introduced per-task depth logic to track recursion across scheduling
* Removed the use of workqueue for a more efficient corruption check
* Reordered patches for better logical flow
* Simplified and improved commit messages throughout the series
* Removed initial archcheck which should be improved later
* Replaced the multiple-thread test with silent corruption test
* Split self-tests into a separate patch to improve clarity.
* Added a new entry for KStackWatch to the MAINTAINERS file.
---
Jinchao Wang (26):
x86/hw_breakpoint: Unify breakpoint install/uninstall
x86/hw_breakpoint: Add arch_reinstall_hw_breakpoint
mm/ksw: add build system support
mm/ksw: add ksw_config struct and parser
mm/ksw: add singleton debugfs interface
mm/ksw: add HWBP pre-allocation
mm/ksw: Add atomic watchpoint management api
mm/ksw: ignore false positives from exit trampolines
mm/ksw: support CPU hotplug
sched/ksw: add per-task context
mm/ksw: add entry kprobe and exit fprobe management
mm/ksw: add per-task ctx tracking
mm/ksw: resolve stack watch addr and len
mm/ksw: limit canary search to current stack frame
mm/ksw: manage probe and HWBP lifecycle via procfs
mm/ksw: add KSTACKWATCH_PROFILING to measure probe cost
arm64/hw_breakpoint: Add arch_reinstall_hw_breakpoint
arm64/hwbp/ksw: integrate KStackWatch handler support
mm/ksw: add self-debug helpers
mm/ksw: add test module
mm/ksw: add stack overflow test
mm/ksw: add recursive depth test
mm/ksw: add multi-thread corruption test cases
tools/ksw: add arch-specific test script
docs: add KStackWatch document
MAINTAINERS: add entry for KStackWatch
Masami Hiramatsu (Google) (1):
HWBP: Add modify_wide_hw_breakpoint_local() API
Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kstackwatch.rst | 377 +++++++++++++++++++++
MAINTAINERS | 9 +
arch/Kconfig | 10 +
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/hw_breakpoint.h | 1 +
arch/arm64/kernel/hw_breakpoint.c | 12 +
arch/x86/Kconfig | 1 +
arch/x86/include/asm/hw_breakpoint.h | 8 +
arch/x86/kernel/hw_breakpoint.c | 148 +++++----
include/linux/hw_breakpoint.h | 6 +
include/linux/kstackwatch.h | 68 ++++
include/linux/kstackwatch_types.h | 14 +
include/linux/sched.h | 5 +
kernel/events/hw_breakpoint.c | 37 +++
mm/Kconfig | 1 +
mm/Makefile | 1 +
mm/kstackwatch/Kconfig | 34 ++
mm/kstackwatch/Makefile | 8 +
mm/kstackwatch/kernel.c | 295 +++++++++++++++++
mm/kstackwatch/stack.c | 416 ++++++++++++++++++++++++
mm/kstackwatch/test.c | 345 ++++++++++++++++++++
mm/kstackwatch/watch.c | 309 ++++++++++++++++++
tools/kstackwatch/kstackwatch_test.sh | 85 +++++
24 files changed, 2130 insertions(+), 62 deletions(-)
create mode 100644 Documentation/dev-tools/kstackwatch.rst
create mode 100644 include/linux/kstackwatch.h
create mode 100644 include/linux/kstackwatch_types.h
create mode 100644 mm/kstackwatch/Kconfig
create mode 100644 mm/kstackwatch/Makefile
create mode 100644 mm/kstackwatch/kernel.c
create mode 100644 mm/kstackwatch/stack.c
create mode 100644 mm/kstackwatch/test.c
create mode 100644 mm/kstackwatch/watch.c
create mode 100755 tools/kstackwatch/kstackwatch_test.sh
-*
2.43.0
^ permalink raw reply
* Re: [PATCH 1/2] mm/khugepaged: do synchronous writeback for MADV_COLLAPSE
From: Lorenzo Stoakes @ 2025-11-10 16:06 UTC (permalink / raw)
To: Garg, Shivank
Cc: Andrew Morton, David Hildenbrand, Zi Yan, Baolin Wang,
Liam R . Howlett, Nico Pache, Ryan Roberts, Dev Jain, Barry Song,
Lance Yang, Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers,
Zach O'Keefe, linux-mm, linux-kernel, linux-trace-kernel,
Branden Moore
In-Reply-To: <bc07aa10-d8d1-435f-9393-6c4ab63cc179@lucifer.local>
On Mon, Nov 10, 2025 at 01:22:16PM +0000, Lorenzo Stoakes wrote:
> On Mon, Nov 10, 2025 at 06:37:58PM +0530, Garg, Shivank wrote:
> >
> >
> > On 11/10/2025 5:31 PM, Lorenzo Stoakes wrote:
> > > On Mon, Nov 10, 2025 at 11:32:53AM +0000, Shivank Garg wrote:
> > >> When MADV_COLLAPSE is called on file-backed mappings (e.g., executable
> >
> > >> ---
> > >> Applies cleanly on:
> > >> 6.18-rc5
> > >> mm-stable:e9a6fb0bc
> > >
> > > Please base on mm-unstable. mm-stable is usually out of date until very close to
> > > merge window.
> >
> > I'm observing issues when testing with kselftest on mm-unstable and mm-new branches that prevent
> > proper testing for my patches:
> >
> > On mm-unstable (without my patches):
> >
> > # # running ./transhuge-stress -d 20
> > # # --------------------------------
> > # # TAP version 13
> > # # 1..1
> > # # transhuge-stress: allocate 220271 transhuge pages, using 440543 MiB virtual memory and 1720 MiB of ram
> >
> >
> > [ 367.225667] RIP: 0010:swap_cache_get_folio+0x2d/0xc0
> > [ 367.230635] Code: 00 00 48 89 f9 49 89 f9 48 89 fe 48 c1 e1 06 49 c1 e9 3a 48 c1 e9 0f 48 c1 e1 05 4a 8b 04 cd c0 2e 5b 99 48 8b 78 60 48 01 cf <48> 8b 47 08 48 85 c0 74 20 48 89 f2 81 e2 ff 01 00 00 48 8d 04 d0
> > [ 367.249378] RSP: 0000:ffffcde32943fba8 EFLAGS: 00010282
> > [ 367.254605] RAX: ffff8bd1668fdc00 RBX: 00007ffc15df5000 RCX: 00003fffffffffe0
> > [ 367.261736] RDX: ffffffff995cb530 RSI: 0003ffffffffffff RDI: ffffcbd1560dffe0
> > [ 367.268862] RBP: 0003ffffffffffff R08: ffffcde32943fc47 R09: 0000000000000000
> > [ 367.275994] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> > [ 367.283129] R13: 0000000000000000 R14: ffff8bd1668fdc00 R15: 0000000000100cca
> > [ 367.290260] FS: 00007ff600af5b80(0000) GS:ffff8c4e9ec7e000(0000) knlGS:0000000000000000
> > [ 367.298344] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 367.304083] CR2: ffffcbd1560dffe8 CR3: 00000001280e9001 CR4: 0000000000770ef0
> > [ 367.311216] PKRU: 55555554
> > [ 367.313929] Call Trace:
> > [ 367.316375] <TASK>
> > [ 367.318479] __read_swap_cache_async+0x8e/0x1b0
> > [ 367.323014] swap_vma_readahead+0x23d/0x430
> > [ 367.327198] swapin_readahead+0xb0/0xc0
> > [ 367.331039] do_swap_page+0x5bc/0x1260
> > [ 367.334789] ? rseq_ip_fixup+0x6f/0x190
> > [ 367.338631] ? __pfx_default_wake_function+0x10/0x10
> > [ 367.343596] __handle_mm_fault+0x49a/0x760
> > [ 367.347696] handle_mm_fault+0x188/0x300
> > [ 367.351620] do_user_addr_fault+0x15b/0x6c0
> > [ 367.355807] exc_page_fault+0x60/0x100
> > [ 367.359562] asm_exc_page_fault+0x22/0x30
> > [ 367.363574] RIP: 0033:0x7ff60091ba99
> > [ 367.367153] Code: f7 d8 64 89 02 b8 ff ff ff ff eb bd e8 40 c4 01 00 f3 0f 1e fa 80 3d b5 f5 0e 00 00 74 13 31 c0 0f 05 48 3d 00 f0 ff ff 77 4f <c3> 66 0f 1f 44 00 00 55 48 89 e5 48 83 ec 20 48 89 55 e8 48 89 75
> > [ 367.385897] RSP: 002b:00007ffc15df1118 EFLAGS: 00010203
> > [ 367.391124] RAX: 0000000000000001 RBX: 000055941fb672a0 RCX: 00007ff60091ba91
> > [ 367.398256] RDX: 0000000000000001 RSI: 000055941fb813e0 RDI: 0000000000000000
> > [ 367.405387] RBP: 00007ffc15df21e0 R08: 0000000000000000 R09: 0000000000000007
> > [ 367.412513] R10: 000055941fb97cb0 R11: 0000000000000246 R12: 000055941fb813e0
> > [ 367.419646] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> > [ 367.426781] </TASK>
> > [ 367.428970] Modules linked in: xfrm_user xfrm_algo xt_addrtype xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_reject_ipv4 nft_compat nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables overlay bridge stp llc cfg80211 rfkill binfmt_misc ipmi_ssif amd_atl intel_rapl_msr intel_rapl_common wmi_bmof amd64_edac edac_mce_amd mgag200 rapl drm_client_lib i2c_algo_bit drm_shmem_helper drm_kms_helper acpi_cpufreq i2c_piix4 ptdma k10temp i2c_smbus wmi acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg dm_multipath drm fuse dm_mod nfnetlink ext4 crc16 mbcache jbd2 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 kvm_amd sd_mod ahci nvme libahci kvm libata nvme_core tg3 ccp megaraid_sas irqbypass
> > [ 367.497528] CR2: ffffcbd1560dffe8
> > [ 367.500846] ---[ end trace 0000000000000000 ]---
>
> Yikes, oopsies!
>
> I'll try running tests locally on threadripper, but ran tests against yours
> previously and seemed fine, strange. Maybe fixed since but let me try, maybe
> because swap is not enabled locally for me?
>
> Likely this actually...
I have tried on swap-enabled setup and no issue with mm-unstable.
So this is odd, I know you have limited time (_totally sympathise_) but is it at
all possible if you get a moment to bisect against tip mm-unstable/mm-new?
Obviously we want to make sure buggy swap code doesn't get merged to mainline!
>
> >
> >
> >
> > -----------------
> > On mm-new (without my patches):
> >
> > [ 394.144770] get_swap_device: Bad swap offset entry 3ffffffffffff
> >
> > dmesg | grep "get_swap_device: Bad swap offset entry" | wc -l
> > 359
> >
> >
> > Additionally, kexec triggers an oops and crash during swapoff:
> >
> >
> > Deactivating swap 704.854238] BUG: unable to handle page fault for address: ffffcbe2de8dffe8
> > [ 704.861524] #PF: supervisor read access in kernel mode
> > ;39mswap.img.swa[ 704.866666] #PF: error_code(0x0000) - not-present page
> > [ 704.873253] PGD 0 P4D 0
> > p - /swap.im[ 704.875790] Oops: Oops: 0000 [#1] SMP NOPTI
> > g...
> > [ 704.881354] CPU: 122 UID: 0 PID: 107680 Comm: swapoff Kdump: loaded Not tainted 6.18.0-rc5+ #11 NONE
> > [ 704.891283] Hardware name: Dell Inc. PowerEdge R6525/024PW1, BIOS 2.16.2 07/09/2024
> > [ 704.898930] RIP: 0010:swap_cache_get_folio+0x2d/0xc0
> > [ 704.903907] Code: 00 00 48 89 f9 49 89 f9 48 89 fe 48 c1 e1 06 49 c1 e9 3a 48 c1 e9 0f 48 c1 e1 05 4a 8b 04 cd c0 2e 7b 95 48 8b 78 60 48 01 cf <48> 8b 47 08 48 85 c0 74 20 48 89 f2 81 e2 ff 01 00 00 48 8d 04 d0
> > [ 704.922720] RSP: 0018:ffffcf1227b1fc08 EFLAGS: 00010282
> > [ 704.928035] RAX: ffff8be2cefb3c00 RBX: 0000555c65a5c000 RCX: 00003fffffffffe0
> > [ 704.928036] RDX: 0003ffffffffffff RSI: 0003ffffffffffff RDI: ffffcbe2de8dffe0
> > [ 704.928037] RBP: 0000000000000000 R08: ffff8be2de8e0520 R09: 0000000000000000
> > Unmount[ 704.928038] R10: 000000000000ffff R11: ffffcf12236f4000 R12: ffff8be2d5b8d968
> > [ 704.928039] R13: 0003ffffffffffff R14: fffff3eec85eb000 R15: 0000555c65a51000
> > [ 704.928039] FS: 00007f41fcab3800(0000) GS:ffff8c602b6fe000(0000) knlGS:0000000000000000
> > [ 704.928040] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 704.928041] CR2: ffffcbe2de8dffe8 CR3: 00000074981af004 CR4: 0000000000770ef0
> > [ 704.928041] PKRU: 55555554
> > [ 704.928042] Call Trace:
> > [ 704.928043] <TASK>
> > [ 704.928044] unuse_pte_range+0x10b/0x290
> > [ 704.928047] unuse_pud_range.isra.0+0x149/0x190
> > [ 704.928048] unuse_vma+0x1a6/0x220
> > [ 704.928050] unuse_mm+0x9b/0x110
> > [ 704.928052] try_to_unuse+0xc5/0x260
> > [ 704.928053] __do_sys_swapoff+0x244/0x670
> > ing boo[ 705.016662] do_syscall_64+0x67/0xc50
> > [ 705.016667] ? do_user_addr_fault+0x15b/0x6c0
> > t.mount - /b[ 705.026100] ? exc_page_fault+0x60/0x100
> > [ 705.031498] ? irqentry_exit_to_user_mode+0x20/0xe0
> > oot...
> > [ 705.036377] entry_SYSCALL_64_after_hwframe+0x76/0x7e
> > [ 705.042200] RIP: 0033:0x7f41fc9271bb
> > [ 705.045780] Code: 0f 1e fa 48 83 fe 01 48 8b 15 59 bc 0d 00 19 c0 83 e0 f0 83 c0 26 64 89 02 b8 ff ff ff ff c3 f3 0f 1e fa b8 a8 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 2d bc 0d 00 f7 d8 64 89 01 48
> > [ 705.064807] RSP: 002b:00007ffd14b5b6e8 EFLAGS: 00000202 ORIG_RAX: 00000000000000a8
> > [ 705.064809] RAX: ffffffffffffffda RBX: 00007ffd14b5cf30 RCX: 00007f41fc9271bb
> > [ 705.064810] RDX: 0000000000000001 RSI: 0000000000000c00 RDI: 000055d48f533a40
> > [ 705.064810] RBP: 00007ffd14b5b750 R08: 00007f41fca03b20 R09: 0000000000000000
> > [ 705.064811] R10: 0000000000000001 R11: 0000000000000202 R12: 0000000000000000
> > [ 705.064811] R13: 0000000000000000 R14: 000055d4584f1479 R15: 000055d4584f2b20
> > [ 705.064813] </TASK>
> > [ 705.064814] Modules linked in: xfrm_user xfrm_algo xt_addrtype xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_reject_ipv4 nft_compat nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables overlay bridge stp llc cfg80211 rfkill binfmt_misc ipmi_ssif amd_atl intel_rapl_msr intel_rapl_common wmi_bmof amd64_edac edac_mce_amd rapl mgag200 drm_client_lib i2c_algo_bit drm_shmem_helper drm_kms_helper acpi_cpufreq i2c_piix4 ptdma ipmi_si k10temp i2c_smbus acpi_power_meter wmi acpi_ipmi ipmi_msghandler sg dm_multipath fuse drm dm_mod nfnetlink ext4 crc16 mbcache jbd2 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 sd_mod kvm_amd ahci libahci kvm nvme tg3 libata ccp irqbypass nvme_core megaraid_sas [last unloaded: ipmi_devintf]
> > [ 705.180420] CR2: ffffcbe2de8dffe8
> > [ 705.183852] ---[ end trace 0000000000000000 ]---
> >
> >
> > I haven't had cycles to dig into this yet and been swamped with other things.
>
> Fully understand, I'm _very_ familiar with this situation :)
>
> I need more cores... ;)
Oh it's nice to have more :) I am bankrupt now, but it's nice to have more ;)
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v3 5/8] mm: set the VM_MAYBE_GUARD flag on guard region install
From: Vlastimil Babka @ 2025-11-10 16:17 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton
Cc: Jonathan Corbet, David Hildenbrand, Liam R . Howlett,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jann Horn, Pedro Falcato,
Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Lance Yang, linux-kernel, linux-fsdevel, linux-doc,
linux-mm, linux-trace-kernel, linux-kselftest, Andrei Vagin
In-Reply-To: <99ce6131e27592c92b43ac866da48b4c2a568298.1762531708.git.lorenzo.stoakes@oracle.com>
On 11/7/25 17:11, Lorenzo Stoakes wrote:
> Now we have established the VM_MAYBE_GUARD flag and added the capacity to
> set it atomically, do so upon MADV_GUARD_INSTALL.
>
> The places where this flag is used currently and matter are:
>
> * VMA merge - performed under mmap/VMA write lock, therefore excluding
> racing writes.
>
> * /proc/$pid/smaps - can race the write, however this isn't meaningful as
> the flag write is performed at the point of the guard region being
> established, and thus an smaps reader can't reasonably expect to avoid
> races. Due to atomicity, a reader will observe either the flag being set
> or not. Therefore consistency will be maintained.
>
> In all other cases the flag being set is irrelevant and atomicity
> guarantees other flags will be read correctly.
>
> Note that non-atomic updates of unrelated flags do not cause an issue with
> this flag being set atomically, as writes of other flags are performed
> under mmap/VMA write lock, and these atomic writes are performed under
> mmap/VMA read lock, which excludes the write, avoiding RMW races.
>
> Note that we do not encounter issues with KCSAN by adjusting this flag
> atomically, as we are only updating a single bit in the flag bitmap and
> therefore we do not need to annotate these changes.
>
> We intentionally set this flag in advance of actually updating the page
> tables, to ensure that any racing atomic read of this flag will only return
> false prior to page tables being updated, to allow for serialisation via
> page table locks.
>
> Note that we set vma->anon_vma for anonymous mappings. This is because the
> expectation for anonymous mappings is that an anon_vma is established
> should they possess any page table mappings. This is also consistent with
> what we were doing prior to this patch (unconditionally setting anon_vma on
> guard region installation).
>
> We also need to update retract_page_tables() to ensure that madvise(...,
> MADV_COLLAPSE) doesn't incorrectly collapse file-backed ranges contain
> guard regions.
>
> This was previously guarded by anon_vma being set to catch MAP_PRIVATE
> cases, but the introduction of VM_MAYBE_GUARD necessitates that we check
> this flag instead.
>
> We utilise vma_flag_test_atomic() to do so - we first perform an optimistic
> check, then after the PTE page table lock is held, we can check again
> safely, as upon guard marker install the flag is set atomically prior to
> the page table lock being taken to actually apply it.
>
> So if the initial check fails either:
>
> * Page table retraction acquires page table lock prior to VM_MAYBE_GUARD
> being set - guard marker installation will be blocked until page table
> retraction is complete.
>
> OR:
>
> * Guard marker installation acquires page table lock after setting
> VM_MAYBE_GUARD, which raced and didn't pick this up in the initial
> optimistic check, blocking page table retraction until the guard regions
> are installed - the second VM_MAYBE_GUARD check will prevent page table
> retraction.
>
> Either way we're safe.
>
> We refactor the retraction checks into a single
> file_backed_vma_is_retractable(), there doesn't seem to be any reason that
> the checks were separated as before.
>
> Note that VM_MAYBE_GUARD being set atomically remains correct as
> vma_needs_copy() is invoked with the mmap and VMA write locks held,
> excluding any race with madvise_guard_install().
>
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Small nit below:
> @@ -1778,15 +1801,16 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)
> spin_lock_nested(ptl, SINGLE_DEPTH_NESTING);
>
> /*
> - * Huge page lock is still held, so normally the page table
> - * must remain empty; and we have already skipped anon_vma
> - * and userfaultfd_wp() vmas. But since the mmap_lock is not
> - * held, it is still possible for a racing userfaultfd_ioctl()
> - * to have inserted ptes or markers. Now that we hold ptlock,
> - * repeating the anon_vma check protects from one category,
> - * and repeating the userfaultfd_wp() check from another.
> + * Huge page lock is still held, so normally the page table must
> + * remain empty; and we have already skipped anon_vma and
> + * userfaultfd_wp() vmas. But since the mmap_lock is not held,
> + * it is still possible for a racing userfaultfd_ioctl() or
> + * madvise() to have inserted ptes or markers. Now that we hold
> + * ptlock, repeating the anon_vma check protects from one
> + * category, and repeating the userfaultfd_wp() check from
> + * another.
The last part of the comment is unchanged and mentions anon_vma check and
userfaultfd_wp() check which were there explicitly originally, but now it's
a file_backed_vma_is_retractable() check that also includes the guard region
check, so maybe could be updated?
> */
> - if (likely(!vma->anon_vma && !userfaultfd_wp(vma))) {
> + if (likely(file_backed_vma_is_retractable(vma))) {
> pgt_pmd = pmdp_collapse_flush(vma, addr, pmd);
> pmdp_get_lockless_sync();
> success = true;
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 67bdfcb315b3..de918b107cfc 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -1139,15 +1139,21 @@ static long madvise_guard_install(struct madvise_behavior *madv_behavior)
> return -EINVAL;
>
> /*
> - * If we install guard markers, then the range is no longer
> - * empty from a page table perspective and therefore it's
> - * appropriate to have an anon_vma.
> - *
> - * This ensures that on fork, we copy page tables correctly.
> + * Set atomically under read lock. All pertinent readers will need to
> + * acquire an mmap/VMA write lock to read it. All remaining readers may
> + * or may not see the flag set, but we don't care.
> + */
> + vma_flag_set_atomic(vma, VM_MAYBE_GUARD_BIT);
> +
> + /*
> + * If anonymous and we are establishing page tables the VMA ought to
> + * have an anon_vma associated with it.
> */
> - err = anon_vma_prepare(vma);
> - if (err)
> - return err;
> + if (vma_is_anonymous(vma)) {
> + err = anon_vma_prepare(vma);
> + if (err)
> + return err;
> + }
>
> /*
> * Optimistically try to install the guard marker pages first. If any
^ permalink raw reply
* Re: [PATCH v3 2/8] mm: add atomic VMA flags and set VM_MAYBE_GUARD as such
From: Vlastimil Babka @ 2025-11-10 15:51 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton
Cc: Jonathan Corbet, David Hildenbrand, Liam R . Howlett,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jann Horn, Pedro Falcato,
Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Lance Yang, linux-kernel, linux-fsdevel, linux-doc,
linux-mm, linux-trace-kernel, linux-kselftest, Andrei Vagin
In-Reply-To: <cda9d4c073d773ef6c2cf2939d66cf80544cff40.1762531708.git.lorenzo.stoakes@oracle.com>
On 11/7/25 17:11, Lorenzo Stoakes wrote:
> This patch adds the ability to atomically set VMA flags with only the mmap
> read/VMA read lock held.
>
> As this could be hugely problematic for VMA flags in general given that all
> other accesses are non-atomic and serialised by the mmap/VMA locks, we
> implement this with a strict allow-list - that is, only designated flags
> are allowed to do this.
>
> We make VM_MAYBE_GUARD one of these flags.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> ---
> include/linux/mm.h | 42 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 2a5516bff75a..699566c21ff7 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -518,6 +518,9 @@ extern unsigned int kobjsize(const void *objp);
> /* This mask represents all the VMA flag bits used by mlock */
> #define VM_LOCKED_MASK (VM_LOCKED | VM_LOCKONFAULT)
>
> +/* These flags can be updated atomically via VMA/mmap read lock. */
> +#define VM_ATOMIC_SET_ALLOWED VM_MAYBE_GUARD
> +
> /* Arch-specific flags to clear when updating VM flags on protection change */
> #ifndef VM_ARCH_CLEAR
> # define VM_ARCH_CLEAR VM_NONE
> @@ -860,6 +863,45 @@ static inline void vm_flags_mod(struct vm_area_struct *vma,
> __vm_flags_mod(vma, set, clear);
> }
>
> +static inline bool __vma_flag_atomic_valid(struct vm_area_struct *vma,
> + int bit)
> +{
> + const vm_flags_t mask = BIT(bit);
> +
> + /* Only specific flags are permitted */
> + if (WARN_ON_ONCE(!(mask & VM_ATOMIC_SET_ALLOWED)))
> + return false;
> +
> + return true;
> +}
> +
> +/*
> + * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific
> + * valid flags are allowed to do this.
> + */
> +static inline void vma_flag_set_atomic(struct vm_area_struct *vma, int bit)
> +{
> + /* mmap read lock/VMA read lock must be held. */
> + if (!rwsem_is_locked(&vma->vm_mm->mmap_lock))
> + vma_assert_locked(vma);
> +
> + if (__vma_flag_atomic_valid(vma, bit))
> + set_bit(bit, &vma->__vm_flags);
> +}
> +
> +/*
> + * Test for VMA flag atomically. Requires no locks. Only specific valid flags
> + * are allowed to do this.
> + *
> + * This is necessarily racey, so callers must ensure that serialisation is
> + * achieved through some other means, or that races are permissible.
> + */
> +static inline bool vma_flag_test_atomic(struct vm_area_struct *vma, int bit)
> +{
> + if (__vma_flag_atomic_valid(vma, bit))
> + return test_bit(bit, &vma->__vm_flags);
> +}
Hm clang is unhappy here.
./include/linux/mm.h:932:1: error: non-void function does not return a value in all control paths [-Werror,-Wreturn-type]
932 | }
| ^
1 error generated.
I don't have CONFIG_WERROR enabled though, so not sure why it's not just a
warning, as the function is unused until patch 5/8 which adds a "return
false" here. So it's just a potential bisection annoyance with clang.
Andrew could you move that hunk from to this patch? Thanks.
> +
> static inline void vma_set_anonymous(struct vm_area_struct *vma)
> {
> vma->vm_ops = NULL;
^ permalink raw reply
* [PATCH v2] tracing: Report wrong dynamic event command
From: Masami Hiramatsu (Google) @ 2025-11-10 15:48 UTC (permalink / raw)
To: Steven Rostedt, Masami Hiramatsu
Cc: Mathieu Desnoyers, linux-kernel, linux-trace-kernel
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Report wrong dynamic event type in the command via error_log.
-----
# echo "z hoge" > /sys/kernel/tracing/dynamic_events
sh: write error: Invalid argument
# cat /sys/kernel/tracing/error_log
[ 22.977022] dynevent: error: No matching dynamic event type
Command: z hoge
^
-----
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v2
- Use tracing_log_err() directly to avoid compile error.
- Update error message.
---
kernel/trace/trace_dynevent.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/kernel/trace/trace_dynevent.c b/kernel/trace/trace_dynevent.c
index d06854bd32b3..c4dfbc293bae 100644
--- a/kernel/trace/trace_dynevent.c
+++ b/kernel/trace/trace_dynevent.c
@@ -144,9 +144,16 @@ static int create_dyn_event(const char *raw_command)
if (!ret || ret != -ECANCELED)
break;
}
- mutex_unlock(&dyn_event_ops_mutex);
- if (ret == -ECANCELED)
+ if (ret == -ECANCELED) {
+ static const char *err_msg[] = {"No matching dynamic event type"};
+
+ /* Wrong dynamic event. Leave an error message. */
+ tracing_log_err(NULL, "dynevent", raw_command, err_msg,
+ 0, 0);
ret = -EINVAL;
+ }
+
+ mutex_unlock(&dyn_event_ops_mutex);
return ret;
}
^ permalink raw reply related
* Re: [PATCH 1/2] mm/khugepaged: do synchronous writeback for MADV_COLLAPSE
From: David Hildenbrand (Red Hat) @ 2025-11-10 14:54 UTC (permalink / raw)
To: Lorenzo Stoakes, Matthew Wilcox
Cc: Shivank Garg, Andrew Morton, Zi Yan, Baolin Wang,
Liam R . Howlett, Nico Pache, Ryan Roberts, Dev Jain, Barry Song,
Lance Yang, Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers,
Zach O'Keefe, linux-mm, linux-kernel, linux-trace-kernel,
Branden Moore
In-Reply-To: <2e6153c9-1729-463a-86b5-0973b3f56ea6@lucifer.local>
On 10.11.25 15:05, Lorenzo Stoakes wrote:
> On Mon, Nov 10, 2025 at 02:03:43PM +0000, Matthew Wilcox wrote:
>> On Mon, Nov 10, 2025 at 03:00:32PM +0100, David Hildenbrand (Red Hat) wrote:
>>> On 10.11.25 14:47, Matthew Wilcox wrote:
>>>> On Mon, Nov 10, 2025 at 11:32:53AM +0000, Shivank Garg wrote:
>>>>> When MADV_COLLAPSE is called on file-backed mappings (e.g., executable
>>>>> text sections), the pages may still be dirty from recent writes. The
>>>>
>>>> That explanation derails my brain entirely. Text isn't writable! How
>>>> can the pages be dirty and file-backed text?
>>>
>>> Files are writable :)
>>
>> Well, if you're root ...
>>
>> -rwxr-xr-x 1 root root 158632 Jun 4 11:14 /bin/ls
>>
>
> gcc foo.c -o foo && ./foo
>
> Is more the issue I think
Yeah, that's my understanding as well. I guess stuff like
package/container updates might similarly trigger it.
^ permalink raw reply
* Re: [PATCH 1/2] mm/khugepaged: do synchronous writeback for MADV_COLLAPSE
From: Matthew Wilcox @ 2025-11-10 14:28 UTC (permalink / raw)
To: Garg, Shivank
Cc: Lorenzo Stoakes, Andrew Morton, David Hildenbrand, Zi Yan,
Baolin Wang, Liam R . Howlett, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Lance Yang, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Zach O'Keefe, linux-mm, linux-kernel,
linux-trace-kernel, Branden Moore
In-Reply-To: <39c8a4ed-53b5-48e3-baea-f67cc1e8be4f@amd.com>
On Mon, Nov 10, 2025 at 07:50:17PM +0530, Garg, Shivank wrote:
> The issue is copying those binary to a freshly mounted filesystem.
> The page cache folios remain dirty until background writeback completes.
>
> Reproduces 100% for me: fresh XFS/EXT4 mount -> copy binary -> execute -> MADV_COLLAPSE fails.
Yes, but this is an uncommon thing to do. Really, it's the kind of
thing you do when you're testing something (like, whether ext4 supports
large folios, and whether that yields a performance improvement).
It's more reasonable to change userspace than the kernel to solve this
problem you're having.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox