From: Jinchao Wang <wangjinchao600@gmail.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Masami Hiramatsu <mhiramat@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Mike Rapoport <rppt@kernel.org>,
Alexander Potapenko <glider@google.com>,
Jonathan Corbet <corbet@lwn.net>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
"Liang, Kan" <kan.liang@linux.intel.com>,
David Hildenbrand <david@redhat.com>,
Lorenzo Stoakes <lorenzo.stoakes@oracle.com>,
"Liam R. Howlett" <Liam.Howlett@oracle.com>,
Vlastimil Babka <vbabka@suse.cz>,
Suren Baghdasaryan <surenb@google.com>,
Michal Hocko <mhocko@suse.com>,
Nathan Chancellor <nathan@kernel.org>,
Nick Desaulniers <nick.desaulniers+lkml@gmail.com>,
Bill Wendling <morbo@google.com>,
Justin Stitt <justinstitt@google.com>,
Kees Cook <kees@kernel.org>, Alice Ryhl <aliceryhl@google.com>,
Sami Tolvanen <samitolvanen@google.com>,
Miguel Ojeda <ojeda@kernel.org>,
Masahiro Yamada <masahiroy@kernel.org>, Rong Xu <xur@google.com>,
Naveen N Rao <naveen@kernel.org>,
David Kaplan <david.kaplan@amd.com>,
Andrii Nakryiko <andrii@kernel.org>,
Jinjie Ruan <ruanjinjie@huawei.com>,
Nam Cao <namcao@linutronix.de>,
workflows@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
linux-mm@kvack.org, llvm@lists.linux.dev,
Andrey Ryabinin <ryabinin.a.a@gmail.com>,
Andrey Konovalov <andreyknvl@gmail.com>,
Dmitry Vyukov <dvyukov@google.com>,
Vincenzo Frascino <vincenzo.frascino@arm.com>,
kasan-dev@googlegroups.com,
"David S. Miller" <davem@davemloft.net>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
linux-trace-kernel@vger.kernel.org
Cc: Jinchao Wang <wangjinchao600@gmail.com>
Subject: [PATCH v4 00/21] mm/ksw: Introduce real-time KStackWatch debugging tool
Date: Fri, 12 Sep 2025 18:11:10 +0800 [thread overview]
Message-ID: <20250912101145.465708-1-wangjinchao600@gmail.com> (raw)
This patch series introduces KStackWatch, a lightweight kernel debugging tool
for detecting kernel stack corruption in real time.
The motivation comes from scenarios where corruption occurs silently in one function
but manifests later as a crash in another. Using other tools may not reproduce the
issue due to its heavy overhead. with no direct call trace linking the two. Such bugs
are often extremely hard to debug with existing tools.
I demonstrate this scenario in test2 (silent corruption test).
KStackWatch works by combining a hardware breakpoint with kprobe and fprobe.
It can watch a stack canary or a selected local variable and detects the moment the
corruption actually occurs. This allows developers to pinpoint the real source rather
than only observing the final crash.
Key features include:
- Lightweight overhead with minimal impact on bug reproducibility
- Real-time detection of stack corruption
- Simple configuration through `/proc/kstackwatch`
- Support for recursive depth filter
To validate the approach, the patch includes a test module and a test script.
---
Changelog
V4:
* Solve the lockdep issues with:
* per-task KStackWatch context to track depth
* atomic flag to protect watched_addr
* Use refactored version of arch_reinstall_hw_breakpoint
Patches 1–3 of this series are also used in the wprobe work proposed by
Masami Hiramatsu, so there may be some overlap between our patches.
Patch 3 comes directly from Masami Hiramatsu (thanks).
V3:
Main changes:
* Use modify_wide_hw_breakpoint_local() (from Masami)
* Add atomic flag to restrict /proc/kstackwatch to a single opener
* Protect stack probe with an atomic PID flag
* Handle CPU hotplug for watchpoints
* Add preempt_disable/enable in ksw_watch_on_local_cpu()
* Introduce const struct ksw_config *ksw_get_config(void) and use it
* Switch to global watch_attr, remove struct watch_info
* Validate local_var_len in parser()
* Handle case when canary is not found
* Use dump_stack() instead of show_regs() to allow module build
Cleanups:
* Reduce logging and comments
* Format logs with KBUILD_MODNAME
* Remove unused headers
Documentation:
* Add new document
V2:
https://lore.kernel.org/all/20250904002126.1514566-1-wangjinchao600@gmail.com/
* Make hardware breakpoint and stack operations architecture-independent.
V1:
https://lore.kernel.org/all/20250828073311.1116593-1-wangjinchao600@gmail.com/
Core Implementation
* Replaced kretprobe with fprobe for function exit hooking, as suggested
by Masami Hiramatsu
* Introduced per-task depth logic to track recursion across scheduling
* Removed the use of workqueue for a more efficient corruption check
* Reordered patches for better logical flow
* Simplified and improved commit messages throughout the series
* Removed initial archcheck which should be improved later
Testing and Architecture
* Replaced the multiple-thread test with silent corruption test
* Split self-tests into a separate patch to improve clarity.
Maintenance
* Added a new entry for KStackWatch to the MAINTAINERS file.
RFC:
https://lore.kernel.org/lkml/20250818122720.434981-1-wangjinchao600@gmail.com/
---
The series is structured as follows:
Jinchao Wang (20):
x86/hw_breakpoint: Unify breakpoint install/uninstall
x86/hw_breakpoint: Add arch_reinstall_hw_breakpoint
mm/ksw: add build system support
mm/ksw: add ksw_config struct and parser
mm/ksw: add singleton /proc/kstackwatch interface
mm/ksw: add HWBP pre-allocation
mm/ksw: Add atomic ksw_watch_on() and ksw_watch_off()
mm/ksw: support CPU hotplug
sched: add per-task KStackWatch context
mm/ksw: add probe management helpers
mm/ksw: resolve stack watch addr and len
mm/ksw: manage probe and HWBP lifecycle via procfs
mm/ksw: add self-debug helpers
mm/ksw: add test module
mm/ksw: add stack overflow test
mm/ksw: add silent corruption test case
mm/ksw: add recursive stack corruption test
tools/ksw: add test script
docs: add KStackWatch document
MAINTAINERS: add entry for KStackWatch
Masami Hiramatsu (Google) (1):
HWBP: Add modify_wide_hw_breakpoint_local() API
Documentation/dev-tools/kstackwatch.rst | 94 +++++++++
MAINTAINERS | 8 +
arch/Kconfig | 10 +
arch/x86/Kconfig | 1 +
arch/x86/include/asm/hw_breakpoint.h | 8 +
arch/x86/kernel/hw_breakpoint.c | 148 +++++++------
include/linux/hw_breakpoint.h | 6 +
include/linux/kstackwatch_types.h | 13 ++
include/linux/sched.h | 5 +
kernel/events/hw_breakpoint.c | 36 ++++
mm/Kconfig.debug | 21 ++
mm/Makefile | 1 +
mm/kstackwatch/Makefile | 8 +
mm/kstackwatch/kernel.c | 239 +++++++++++++++++++++
mm/kstackwatch/kstackwatch.h | 53 +++++
mm/kstackwatch/stack.c | 194 ++++++++++++++++++
mm/kstackwatch/test.c | 262 ++++++++++++++++++++++++
mm/kstackwatch/watch.c | 181 ++++++++++++++++
tools/kstackwatch/kstackwatch_test.sh | 40 ++++
19 files changed, 1266 insertions(+), 62 deletions(-)
create mode 100644 Documentation/dev-tools/kstackwatch.rst
create mode 100644 include/linux/kstackwatch_types.h
create mode 100644 mm/kstackwatch/Makefile
create mode 100644 mm/kstackwatch/kernel.c
create mode 100644 mm/kstackwatch/kstackwatch.h
create mode 100644 mm/kstackwatch/stack.c
create mode 100644 mm/kstackwatch/test.c
create mode 100644 mm/kstackwatch/watch.c
create mode 100755 tools/kstackwatch/kstackwatch_test.sh
--
2.43.0
next reply other threads:[~2025-09-12 10:12 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-12 10:11 Jinchao Wang [this message]
2025-09-12 10:11 ` [PATCH v4 01/21] x86/hw_breakpoint: Unify breakpoint install/uninstall Jinchao Wang
2025-09-14 13:52 ` Masami Hiramatsu
2025-09-12 10:11 ` [PATCH v4 02/21] x86/hw_breakpoint: Add arch_reinstall_hw_breakpoint Jinchao Wang
2025-09-14 13:53 ` Masami Hiramatsu
2025-09-12 10:11 ` [PATCH v4 03/21] HWBP: Add modify_wide_hw_breakpoint_local() API Jinchao Wang
2025-09-13 4:13 ` Randy Dunlap
2025-09-14 13:02 ` Masami Hiramatsu
2025-09-12 10:11 ` [PATCH v4 04/21] mm/ksw: add build system support Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 05/21] mm/ksw: add ksw_config struct and parser Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 06/21] mm/ksw: add singleton /proc/kstackwatch interface Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 07/21] mm/ksw: add HWBP pre-allocation Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 08/21] mm/ksw: Add atomic ksw_watch_on() and ksw_watch_off() Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 09/21] mm/ksw: support CPU hotplug Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 10/21] sched: add per-task KStackWatch context Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 11/21] mm/ksw: add probe management helpers Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 12/21] mm/ksw: resolve stack watch addr and len Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 13/21] mm/ksw: manage probe and HWBP lifecycle via procfs Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 14/21] mm/ksw: add self-debug helpers Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 15/21] mm/ksw: add test module Jinchao Wang
2025-09-13 4:07 ` Randy Dunlap
2025-09-15 2:03 ` Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 16/21] mm/ksw: add stack overflow test Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 17/21] mm/ksw: add silent corruption test case Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 18/21] mm/ksw: add recursive stack corruption test Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 19/21] tools/ksw: add test script Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 20/21] docs: add KStackWatch document Jinchao Wang
2025-09-12 10:11 ` [PATCH v4 21/21] MAINTAINERS: add entry for KStackWatch Jinchao Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250912101145.465708-1-wangjinchao600@gmail.com \
--to=wangjinchao600@gmail.com \
--cc=Liam.Howlett@oracle.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=akpm@linux-foundation.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=aliceryhl@google.com \
--cc=andreyknvl@gmail.com \
--cc=andrii@kernel.org \
--cc=bp@alien8.de \
--cc=bsegall@google.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=davem@davemloft.net \
--cc=david.kaplan@amd.com \
--cc=david@redhat.com \
--cc=dietmar.eggemann@arm.com \
--cc=dvyukov@google.com \
--cc=glider@google.com \
--cc=hpa@zytor.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=juri.lelli@redhat.com \
--cc=justinstitt@google.com \
--cc=kan.liang@linux.intel.com \
--cc=kasan-dev@googlegroups.com \
--cc=kees@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=lorenzo.stoakes@oracle.com \
--cc=mark.rutland@arm.com \
--cc=masahiroy@kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mgorman@suse.de \
--cc=mhiramat@kernel.org \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=morbo@google.com \
--cc=namcao@linutronix.de \
--cc=namhyung@kernel.org \
--cc=nathan@kernel.org \
--cc=naveen@kernel.org \
--cc=nick.desaulniers+lkml@gmail.com \
--cc=ojeda@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=rppt@kernel.org \
--cc=ruanjinjie@huawei.com \
--cc=ryabinin.a.a@gmail.com \
--cc=samitolvanen@google.com \
--cc=surenb@google.com \
--cc=tglx@linutronix.de \
--cc=vbabka@suse.cz \
--cc=vincent.guittot@linaro.org \
--cc=vincenzo.frascino@arm.com \
--cc=vschneid@redhat.com \
--cc=workflows@vger.kernel.org \
--cc=x86@kernel.org \
--cc=xur@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).