[PATCH 0/2] mm/kmemleak: add min_unref_scans to suppress transient false positives

[PATCH 0/2] mm/kmemleak: add min_unref_scans to suppress transient false positives

[Breno Leitao]

I have kmemleak on some test tiers to find memory leaks in my fleet, with
the reports dumped to dmesg (CONFIG_DEBUG_KMEMLEAK_VERBOSE set). It works
super well.

The problem I have is some false positives that show up from time to time
and go away on a subsequent scan. Something transiently unreferenced --
whose only reference is briefly invisible during a concurrent RCU update,
e.g. a VMA moving between maple-tree nodes, or a page-cache xa_node -- is
seen as unreferenced for a single scan and reported as a leak that does
not exist.

This series adds a min_unref_scans module parameter requiring an object
to stay unreferenced across that many consecutive scans before it is
reported. Basically it is a trade-off between report latency and
reliability (false-positiveness?).

It defaults to 1 (the unchanged report-on-first-scan behaviour) and, set
to 2 or more, filters these single-scan races while still reporting
genuine leaks one scan later. Would it be acceptable upstream?

Patch 1 implements it; patch 2 adds an mm selftest that drives the
parameter via samples/kmemleak, in case this is useful.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
Breno Leitao (2):
      mm/kmemleak: report leaks only after N consecutive unreferenced scans
      selftests/mm: test kmemleak's N-consecutive-scan leak confirmation

 Documentation/dev-tools/kmemleak.rst               |   8 ++
 mm/kmemleak.c                                      |  14 ++-
 tools/testing/selftests/mm/Makefile                |   1 +
 .../testing/selftests/mm/ksft_kmemleak_confirm.sh  | 111 +++++++++++++++++++++
 4 files changed, 132 insertions(+), 2 deletions(-)
---
base-commit: 30ffa8de54e5cc80d93fd211ca134d1764a7011f
change-id: 20260626-kmemleak_twice-ed01218aeccb

Best regards,
-- 
Breno Leitao <leitao@debian.org>

[PATCH 1/2] mm/kmemleak: report leaks only after N consecutive unreferenced scans

[Breno Leitao]

kmemleak reports an object the first scan it is found unreferenced. Its
mark phase runs without stopping the rest of the kernel and without a
write barrier, so a live object whose only reference is briefly invisible
during a concurrent RCU update -- e.g. a VMA moved between maple tree
nodes, or a page-cache xa_node -- can be seen as unreferenced for that one
scan. Because an object is flagged as reported only once, such a transient
race turns into a permanent false positive.

Track how many consecutive scans each object has been seen unreferenced
and only report it once that reaches min_unref_scans, a new module
parameter. It defaults to 1, leaving the behaviour unchanged; setting it
higher (e.g. 2) still reports a genuine leak, one scan later, while an
object referenced again before the threshold restarts its run and is never
reported.

min_unref_scans can be set at boot with kmemleak.min_unref_scans=<n> or at
run-time via /sys/module/kmemleak/parameters/min_unref_scans.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 Documentation/dev-tools/kmemleak.rst |  8 ++++++++
 mm/kmemleak.c                        | 14 ++++++++++++--
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/Documentation/dev-tools/kmemleak.rst b/Documentation/dev-tools/kmemleak.rst
index 7d784e03f3f9d..a8a83bc69ceb8 100644
--- a/Documentation/dev-tools/kmemleak.rst
+++ b/Documentation/dev-tools/kmemleak.rst
@@ -198,6 +198,14 @@ systems, because of pointers temporarily stored in CPU registers or
 stacks. Kmemleak defines MSECS_MIN_AGE (defaulting to 1000) representing
 the minimum age of an object to be reported as a memory leak.
 
+The ``min_unref_scans`` module parameter (default 1) requires an object to
+be seen unreferenced in that many consecutive scans before it is reported.
+Keeping it at 1 preserves the historical behaviour; higher values filter
+the transient false positives described above, at the cost of delaying
+genuine reports by up to that many scans. It can be set at boot with
+``kmemleak.min_unref_scans=<n>`` or at run-time via
+``/sys/module/kmemleak/parameters/min_unref_scans``.
+
 Limitations and Drawbacks
 -------------------------
 
diff --git a/mm/kmemleak.c b/mm/kmemleak.c
index 7c7ba17ce7af0..5b14ccb36f95b 100644
--- a/mm/kmemleak.c
+++ b/mm/kmemleak.c
@@ -151,6 +151,8 @@ struct kmemleak_object {
 	int min_count;
 	/* the total number of pointers found pointing to this object */
 	int count;
+	/* consecutive scans the object has been seen unreferenced */
+	unsigned int unref_scans;
 	/* checksum for detecting modified objects */
 	u32 checksum;
 	depot_stack_handle_t trace_handle;
@@ -232,6 +234,9 @@ static unsigned long max_percpu_addr;
 static struct task_struct *scan_thread;
 /* used to avoid reporting of recently allocated objects */
 static unsigned long jiffies_min_age;
+/* consecutive scans an object must stay unreferenced before reporting */
+static unsigned int min_unref_scans = 1;
+module_param(min_unref_scans, uint, 0644);
 static unsigned long jiffies_last_scan;
 /* delay between automatic memory scannings */
 static unsigned long jiffies_scan_wait;
@@ -687,6 +692,7 @@ static struct kmemleak_object *__alloc_object(gfp_t gfp)
 	atomic_set(&object->use_count, 1);
 	object->excess_ref = 0;
 	object->count = 0;			/* white color initially */
+	object->unref_scans = 0;
 	object->checksum = 0;
 	object->del_state = 0;
 
@@ -1833,6 +1839,9 @@ static void kmemleak_scan(void)
 				__paint_it(object, KMEMLEAK_BLACK);
 		}
 
+		/* referenced last scan: restart the unreferenced run */
+		if (!color_white(object))
+			object->unref_scans = 0;
 		/* reset the reference count (whiten the object) */
 		object->count = 0;
 		if (color_gray(object) && get_object(object))
@@ -1968,8 +1977,9 @@ static void kmemleak_scan(void)
 		raw_spin_lock_irq(&object->lock);
 		trace_handle = 0;
 		dedup_print = false;
-		if (unreferenced_object(object) &&
-		    !(object->flags & OBJECT_REPORTED)) {
+		if (!(object->flags & OBJECT_REPORTED) &&
+		    unreferenced_object(object) &&
+		    ++object->unref_scans >= min_unref_scans) {
 			object->flags |= OBJECT_REPORTED;
 			if (kmemleak_verbose) {
 				trace_handle = object->trace_handle;

-- 
2.53.0-Meta

[PATCH 2/2] selftests/mm: test kmemleak's N-consecutive-scan leak confirmation

[Breno Leitao]

Add a functional test for the min_unref_scans kmemleak module parameter.
Using samples/kmemleak's helper module it checks that min_unref_scans=1
reports an orphan on the first scan, min_unref_scans=2 reports nothing on
the first scan but does on the second, and that the parameter reads back
what was written.

It counts only the helper module's own orphans (matched by their
[kmemleak_test] backtrace, with the module kept loaded so the symbols
resolve) so unrelated leaks already present on the system do not perturb
the result. The test skips when run as non-root, without
CONFIG_DEBUG_KMEMLEAK / CONFIG_SAMPLE_KMEMLEAK, on a kernel without the
parameter, or when the helper yields no detectable orphan.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 tools/testing/selftests/mm/Makefile                |   1 +
 .../testing/selftests/mm/ksft_kmemleak_confirm.sh  | 111 +++++++++++++++++++++
 2 files changed, 112 insertions(+)

diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index e6df968f0971c..84026b62a1ae5 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -150,6 +150,7 @@ TEST_PROGS += ksft_gup_test.sh
 TEST_PROGS += ksft_hmm.sh
 TEST_PROGS += ksft_hugetlb.sh
 TEST_PROGS += ksft_hugevm.sh
+TEST_PROGS += ksft_kmemleak_confirm.sh
 TEST_PROGS += ksft_kmemleak_dedup.sh
 TEST_PROGS += ksft_ksm.sh
 TEST_PROGS += ksft_ksm_numa.sh
diff --git a/tools/testing/selftests/mm/ksft_kmemleak_confirm.sh b/tools/testing/selftests/mm/ksft_kmemleak_confirm.sh
new file mode 100755
index 0000000000000..34ab64bc6948f
--- /dev/null
+++ b/tools/testing/selftests/mm/ksft_kmemleak_confirm.sh
@@ -0,0 +1,111 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Functional test for kmemleak's N-consecutive-scan leak confirmation
+# (the min_unref_scans module parameter).
+#
+# kmemleak only reports an object once it has stayed unreferenced for
+# min_unref_scans consecutive scans. The default of 1 reports on the first
+# scan (historical behaviour); higher values filter transient false
+# positives where a live object's only reference is briefly invisible to a
+# single scan (e.g. an RCU tree update in flight while the scan runs). The
+# test loads samples/kmemleak's helper module to create orphan allocations
+# and, counting only those orphans (matched by their [kmemleak_test]
+# backtrace so unrelated leaks already present on the system are ignored),
+# checks that:
+#   - min_unref_scans=1 reports them on the first scan,
+#   - min_unref_scans=2 reports nothing on the first scan but does on the
+#     second,
+#   - the parameter reads back what was written.
+#
+# The "nothing on the first scan" check is the core regression test: with
+# min_unref_scans=2 no object can be reported in fewer than two scans. Like
+# ksft_kmemleak_dedup.sh, if the module yields no detectable orphan at all
+# in the running environment the test skips rather than failing.
+#
+# Author: Breno Leitao <leitao@debian.org>
+
+ksft_skip=4
+KMEMLEAK=/sys/kernel/debug/kmemleak
+PARAM=/sys/module/kmemleak/parameters/min_unref_scans
+MODULE=kmemleak-test
+AGE=6		# seconds; must exceed kmemleak's 5s minimum object age
+
+skip() { echo "SKIP: $*"; exit $ksft_skip; }
+fail() { echo "FAIL: $*"; exit 1; }
+pass() { echo "PASS: $*"; exit 0; }
+
+[ "$(id -u)" -eq 0 ] || skip "must run as root"
+[ -r "$KMEMLEAK" ] || skip "no kmemleak debugfs (CONFIG_DEBUG_KMEMLEAK)"
+[ -w "$PARAM" ] || skip "min_unref_scans module parameter not present"
+modinfo "$MODULE" >/dev/null 2>&1 ||
+	skip "$MODULE not built (CONFIG_SAMPLE_KMEMLEAK)"
+
+# kmemleak can be present but disabled at runtime (kmemleak=off boot arg,
+# or it self-disabled after an internal error); a "scan" then returns
+# EPERM. Probe once and skip if so.
+echo scan > "$KMEMLEAK" 2>/dev/null ||
+	skip "kmemleak is disabled (check dmesg or kmemleak= boot arg)"
+
+prev=$(cat "$PARAM")
+# shellcheck disable=SC2317  # invoked indirectly via trap
+cleanup() {
+	echo "$prev" > "$PARAM" 2>/dev/null		# restore the parameter
+	echo scan=on > "$KMEMLEAK" 2>/dev/null		# re-enable auto scan
+	rmmod "$MODULE" 2>/dev/null
+	echo clear > "$KMEMLEAK" 2>/dev/null
+}
+trap cleanup EXIT
+
+# Stop the automatic scan thread: only our manual scans should advance an
+# object's consecutive-unreferenced run. An auto scan landing between two
+# manual scans would change the result and make the test flaky.
+echo scan=off > "$KMEMLEAK" 2>/dev/null
+
+# Create a fresh, aged set of orphan objects from the helper module's init
+# path (its kmalloc/vmalloc/percpu allocations are dropped right away).
+# Pre-existing reported leaks are greyed first ("clear") so only our
+# orphans are counted. The module is left loaded on purpose: once it is
+# unloaded its symbols are gone, so the orphan backtraces no longer resolve
+# to [kmemleak_test] and could not be matched below.
+gen_orphans() {
+	rmmod "$MODULE" 2>/dev/null
+	echo clear > "$KMEMLEAK"
+	modprobe "$MODULE" || skip "failed to load $MODULE"
+	sleep "$AGE"
+}
+
+scan() { echo scan > "$KMEMLEAK"; }
+
+# Number of helper-module orphans currently reported by kmemleak. Matching
+# the module's own backtrace ([kmemleak_test]) keeps the count immune to
+# unrelated leaks on the running system. kmemleak only lists an object here
+# once it has been reported, so this reflects the confirmation gating.
+count_orphans() {
+	c=$(grep -c '\[kmemleak_test\]' "$KMEMLEAK" 2>/dev/null)
+	echo "${c:-0}"
+}
+
+# 0) the parameter reads back what was written.
+echo 3 > "$PARAM"
+[ "$(cat "$PARAM")" = "3" ] || fail "min_unref_scans did not read back as 3"
+
+# 1) min_unref_scans=1 (default): orphans reported on the first scan. This
+#    also establishes that the helper produces detectable orphans here.
+echo 1 > "$PARAM"
+gen_orphans
+scan
+first=$(count_orphans)
+[ "$first" -gt 0 ] ||
+	skip "$MODULE produced no detectable orphans (cannot test min_unref_scans)"
+
+# 2) min_unref_scans=2: nothing reported after the first scan, reported
+#    after the second. The first-scan-zero check is the core regression.
+echo 2 > "$PARAM"
+gen_orphans
+scan; s1=$(count_orphans)
+scan; s2=$(count_orphans)
+[ "$s1" -eq 0 ] || fail "min_unref_scans=2: $s1 orphan(s) reported after the 1st scan (must be 0)"
+[ "$s2" -gt 0 ] || fail "min_unref_scans=2: no report on the 2nd scan (false negative)"
+
+pass "min_unref_scans=1 immediate; =2 gated to 2nd scan (counts $first/$s1/$s2); param read-back ok"

-- 
2.53.0-Meta

Re: [PATCH 1/2] mm/kmemleak: report leaks only after N consecutive unreferenced scans

[Catalin Marinas]

On Fri, Jun 26, 2026 at 08:52:02AM -0700, Breno Leitao wrote:
> kmemleak reports an object the first scan it is found unreferenced. Its
> mark phase runs without stopping the rest of the kernel and without a
> write barrier, so a live object whose only reference is briefly invisible
> during a concurrent RCU update -- e.g. a VMA moved between maple tree
> nodes, or a page-cache xa_node -- can be seen as unreferenced for that one
> scan. Because an object is flagged as reported only once, such a transient
> race turns into a permanent false positive.
> 
> Track how many consecutive scans each object has been seen unreferenced
> and only report it once that reaches min_unref_scans, a new module
> parameter. It defaults to 1, leaving the behaviour unchanged; setting it
> higher (e.g. 2) still reports a genuine leak, one scan later, while an
> object referenced again before the threshold restarts its run and is never
> reported.
> 
> min_unref_scans can be set at boot with kmemleak.min_unref_scans=<n> or at
> run-time via /sys/module/kmemleak/parameters/min_unref_scans.
> 
> Signed-off-by: Breno Leitao <leitao@debian.org>

It looks like a good addition to me. All objects require a second pass
initially to get their checksum updated but that's not sufficient when
they are moved between nodes without having their content changed (list
in a linked list).

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

> diff --git a/mm/kmemleak.c b/mm/kmemleak.c
> index 7c7ba17ce7af0..5b14ccb36f95b 100644
> --- a/mm/kmemleak.c
> +++ b/mm/kmemleak.c
> @@ -151,6 +151,8 @@ struct kmemleak_object {
>  	int min_count;
>  	/* the total number of pointers found pointing to this object */
>  	int count;
> +	/* consecutive scans the object has been seen unreferenced */
> +	unsigned int unref_scans;
>  	/* checksum for detecting modified objects */
>  	u32 checksum;
>  	depot_stack_handle_t trace_handle;
> @@ -232,6 +234,9 @@ static unsigned long max_percpu_addr;
>  static struct task_struct *scan_thread;
>  /* used to avoid reporting of recently allocated objects */
>  static unsigned long jiffies_min_age;
> +/* consecutive scans an object must stay unreferenced before reporting */
> +static unsigned int min_unref_scans = 1;
> +module_param(min_unref_scans, uint, 0644);

0644 is fine. Not sure why kmemleak_verbose was 0600.

-- 
Catalin

Re: [PATCH 2/2] selftests/mm: test kmemleak's N-consecutive-scan leak confirmation

[Catalin Marinas]

On Fri, Jun 26, 2026 at 08:52:03AM -0700, Breno Leitao wrote:
> --- /dev/null
> +++ b/tools/testing/selftests/mm/ksft_kmemleak_confirm.sh
> @@ -0,0 +1,111 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Functional test for kmemleak's N-consecutive-scan leak confirmation
> +# (the min_unref_scans module parameter).
> +#
> +# kmemleak only reports an object once it has stayed unreferenced for
> +# min_unref_scans consecutive scans. The default of 1 reports on the first
> +# scan (historical behaviour); higher values filter transient false
> +# positives where a live object's only reference is briefly invisible to a
> +# single scan (e.g. an RCU tree update in flight while the scan runs). The
> +# test loads samples/kmemleak's helper module to create orphan allocations
> +# and, counting only those orphans (matched by their [kmemleak_test]
> +# backtrace so unrelated leaks already present on the system are ignored),
> +# checks that:
> +#   - min_unref_scans=1 reports them on the first scan,
> +#   - min_unref_scans=2 reports nothing on the first scan but does on the
> +#     second,
> +#   - the parameter reads back what was written.
> +#
> +# The "nothing on the first scan" check is the core regression test: with
> +# min_unref_scans=2 no object can be reported in fewer than two scans. Like
> +# ksft_kmemleak_dedup.sh, if the module yields no detectable orphan at all
> +# in the running environment the test skips rather than failing.
> +#
> +# Author: Breno Leitao <leitao@debian.org>
> +
> +ksft_skip=4
> +KMEMLEAK=/sys/kernel/debug/kmemleak
> +PARAM=/sys/module/kmemleak/parameters/min_unref_scans
> +MODULE=kmemleak-test
> +AGE=6		# seconds; must exceed kmemleak's 5s minimum object age
> +
> +skip() { echo "SKIP: $*"; exit $ksft_skip; }
> +fail() { echo "FAIL: $*"; exit 1; }
> +pass() { echo "PASS: $*"; exit 0; }
> +
> +[ "$(id -u)" -eq 0 ] || skip "must run as root"
> +[ -r "$KMEMLEAK" ] || skip "no kmemleak debugfs (CONFIG_DEBUG_KMEMLEAK)"
> +[ -w "$PARAM" ] || skip "min_unref_scans module parameter not present"
> +modinfo "$MODULE" >/dev/null 2>&1 ||
> +	skip "$MODULE not built (CONFIG_SAMPLE_KMEMLEAK)"
> +
> +# kmemleak can be present but disabled at runtime (kmemleak=off boot arg,
> +# or it self-disabled after an internal error); a "scan" then returns
> +# EPERM. Probe once and skip if so.
> +echo scan > "$KMEMLEAK" 2>/dev/null ||
> +	skip "kmemleak is disabled (check dmesg or kmemleak= boot arg)"
> +
> +prev=$(cat "$PARAM")
> +# shellcheck disable=SC2317  # invoked indirectly via trap
> +cleanup() {
> +	echo "$prev" > "$PARAM" 2>/dev/null		# restore the parameter
> +	echo scan=on > "$KMEMLEAK" 2>/dev/null		# re-enable auto scan
> +	rmmod "$MODULE" 2>/dev/null
> +	echo clear > "$KMEMLEAK" 2>/dev/null
> +}
> +trap cleanup EXIT
> +
> +# Stop the automatic scan thread: only our manual scans should advance an
> +# object's consecutive-unreferenced run. An auto scan landing between two
> +# manual scans would change the result and make the test flaky.
> +echo scan=off > "$KMEMLEAK" 2>/dev/null
> +
> +# Create a fresh, aged set of orphan objects from the helper module's init
> +# path (its kmalloc/vmalloc/percpu allocations are dropped right away).
> +# Pre-existing reported leaks are greyed first ("clear") so only our
> +# orphans are counted. The module is left loaded on purpose: once it is
> +# unloaded its symbols are gone, so the orphan backtraces no longer resolve
> +# to [kmemleak_test] and could not be matched below.
> +gen_orphans() {
> +	rmmod "$MODULE" 2>/dev/null
> +	echo clear > "$KMEMLEAK"
> +	modprobe "$MODULE" || skip "failed to load $MODULE"
> +	sleep "$AGE"
> +}
> +
> +scan() { echo scan > "$KMEMLEAK"; }
> +
> +# Number of helper-module orphans currently reported by kmemleak. Matching
> +# the module's own backtrace ([kmemleak_test]) keeps the count immune to
> +# unrelated leaks on the running system. kmemleak only lists an object here
> +# once it has been reported, so this reflects the confirmation gating.
> +count_orphans() {
> +	c=$(grep -c '\[kmemleak_test\]' "$KMEMLEAK" 2>/dev/null)
> +	echo "${c:-0}"
> +}
> +
> +# 0) the parameter reads back what was written.
> +echo 3 > "$PARAM"
> +[ "$(cat "$PARAM")" = "3" ] || fail "min_unref_scans did not read back as 3"
> +
> +# 1) min_unref_scans=1 (default): orphans reported on the first scan. This
> +#    also establishes that the helper produces detectable orphans here.
> +echo 1 > "$PARAM"
> +gen_orphans
> +scan
> +first=$(count_orphans)
> +[ "$first" -gt 0 ] ||
> +	skip "$MODULE produced no detectable orphans (cannot test min_unref_scans)"
> +
> +# 2) min_unref_scans=2: nothing reported after the first scan, reported
> +#    after the second. The first-scan-zero check is the core regression.
> +echo 2 > "$PARAM"
> +gen_orphans
> +scan; s1=$(count_orphans)
> +scan; s2=$(count_orphans)
> +[ "$s1" -eq 0 ] || fail "min_unref_scans=2: $s1 orphan(s) reported after the 1st scan (must be 0)"
> +[ "$s2" -gt 0 ] || fail "min_unref_scans=2: no report on the 2nd scan (false negative)"
> +
> +pass "min_unref_scans=1 immediate; =2 gated to 2nd scan (counts $first/$s1/$s2); param read-back ok"

Are these off by one? Kmemleak has a mechanism to detect live objects
via the checksum. A side effect is that on allocation, the checksum is 0
and only after the first scan the checksum is changed. On checksum
mismatch (i.e. the first scan), we mark the object gray temporarily and
won't increment unref_scans. So we already have an implicit two scans
required to report an object as unreferenced during its early life.

I think this test needs a priming scan to update the checksums followed
by the actual check for min_unref_scans (with scan=off, otherwise random
scanning will skew the results).

-- 
Catalin

Re: [PATCH 2/2] selftests/mm: test kmemleak's N-consecutive-scan leak confirmation

[Breno Leitao]

On Thu, Jul 02, 2026 at 09:41:14AM +0100, Catalin Marinas wrote:
> On Fri, Jun 26, 2026 at 08:52:03AM -0700, Breno Leitao wrote:
> > +pass "min_unref_scans=1 immediate; =2 gated to 2nd scan (counts $first/$s1/$s2); param read-back ok"
> 
> Are these off by one?

They seem to be OK, and I've tested it multiple times.

> Kmemleak has a mechanism to detect live objects
> via the checksum. A side effect is that on allocation, the checksum is 0
> and only after the first scan the checksum is changed.

I got the impression that checksum continues to be zero for these
objects during the whole life time? (weird). 

If you think this selftest brings value, let me investigate what the
heck is happening here.

> On checksum mismatch (i.e. the first scan), we mark the object gray
> temporarily and won't increment unref_scans. So we already have an
> implicit two scans required to report an object as unreferenced during
> its early life.
> 
> I think this test needs a priming scan to update the checksums
> followed by the actual check for min_unref_scans (with scan=off,
> otherwise random scanning will skew the results).

I tried a priming scan and it actually breaks the min_unref_scans=2 case