From: Breno Leitao <leitao@debian.org>
To: Miaohe Lin <linmiaohe@huawei.com>,
Andrew Morton <akpm@linux-foundation.org>,
David Hildenbrand <david@kernel.org>,
Lorenzo Stoakes <ljs@kernel.org>,
Vlastimil Babka <vbabka@kernel.org>,
Mike Rapoport <rppt@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Michal Hocko <mhocko@suse.com>, Shuah Khan <shuah@kernel.org>,
Naoya Horiguchi <nao.horiguchi@gmail.com>,
Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
"Liam R. Howlett" <liam@infradead.org>,
lance.yang@linux.dev, Steven Rostedt <rostedt@goodmis.org>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
linux-trace-kernel@vger.kernel.org, kernel-team@meta.com
Subject: Re: [PATCH v9 0/6] mm/memory-failure: add panic option for unrecoverable pages
Date: Wed, 17 Jun 2026 02:40:06 -0700 [thread overview]
Message-ID: <ajJq-uVRBH5SBkwK@gmail.com> (raw)
In-Reply-To: <20260609-ecc_panic-v9-0-432a74002e74@debian.org>
On Tue, Jun 09, 2026 at 03:56:54AM -0700, Breno Leitao wrote:
> A multi-bit ECC error on a kernel-owned page that the memory failure
> handler cannot recover is currently swallowed: PG_hwpoison is set, the
> event is logged, and the kernel keeps running. The corrupted memory
> remains accessible to the kernel and either drives silent data
> corruption or surfaces seconds-to-minutes later as an apparently
> unrelated crash. In a large fleet that delayed, unattributable crash
> turns into significant engineering effort to root-cause; in a kdump
> configuration, by the time the crash happens the original error
> context (faulting PFN, MCE/GHES record, page state) is long gone.
>
> This series adds an opt-in sysctl,
> vm.panic_on_unrecoverable_memory_failure, that converts an
> unrecoverable kernel-page hwpoison event into an immediate panic with
> a clean dmesg/vmcore that still contains the original failure
> context. The default is disabled so existing workloads see no
> change.
>
> There is a selftest that test different cases, and I tested it using
> the following variants:
>
> ┌─────────┬──────────┬───────────────────────────────────────────────────────────┐
> │ Variant │ PFN │ Result │
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ rodata │ 0x2600 │ Panic with "Memory failure: 0x2600: unrecoverable page" │
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ slab │ 0x100032 │ Panic with "Memory failure: 0x100032: unrecoverable page" │
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ pgtable │ 0x100000 │ Panic with "Memory failure: 0x100000: unrecoverable page" │
> └─────────┴──────────┴───────────────────────────────────────────────────────────┘
>
> Each one shows the same call trace, exactly the path the series builds:
>
> hard_offline_page_store
> → memory_failure
> → action_result
> → panic("Memory failure: %#lx: unrecoverable page")
Debugging another issue earlier today, just found a kernel crash that is
hitting a ignored page later in the day, and randomly misbehaving/crashing.
Memory failure: 0x140ae: unhandlable page.
Memory failure: 0x140ae: recovery action for get hwpoison page: Ignored <-- Ignored
loop0: detected capacity change from 0 to 15241056
EDAC MC0: 1 UE multi-bit ECC on LP5x_0 LP5x_0 (node:0 card:0 module:0 rank:0 bank:2 device:28 row:42700 column:96
{3}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 308
{3}[Hardware Error]: event severity: recoverable
{3}[Hardware Error]: imprecise tstamp: 2026-06-16 02:50:03
{3}[Hardware Error]: Error 0, type: recoverable
{3}[Hardware Error]: section_type: memory error
{3}[Hardware Error]: physical_address: 0x0000000aeccde180
{3}[Hardware Error]: physical_address_mask: 0xfffffffffffff000
{3}[Hardware Error]: node:0 card:0 module:0 rank:0 bank:2 device:28 row:42700 column:960 requestor_id:0x0000000
{3}[Hardware Error]: error_type: 3, multi-bit ECC
{3}[Hardware Error]: DIMM location: LP5x_0 LP5x_0
Memory failure: 0xaeccd: recovery action for dirty LRU page: Recovered
Internal error: synchronous external abort: 0000000096000410 [#1] SMP
Modules linked in: ghes_edac(E) squashfs(E) act_gact(E) sch_fq(E) tcp_diag(E) inet_diag(E) cls_bpf(E) evdev(E) sm
CPU: 51 UID: 0 PID: 1 Comm: systemd Kdump: loaded Tainted: G M OE K 6.16.1-0_fbk2_0_gf40efc324cc8 #1
Tainted: [M]=MACHINE_CHECK, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [K]=LIVEPATCH
pstate: 834010c9 (Nzcv daIF +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
pc : clear_inode+0x34/0x108
lr : proc_evict_inode.llvm.1771226604092943895+0x28/0x68
sp : ffff800083f6f8d0
x29: ffff800083f6f8e0 x28: 0000000000000011 x27: ffff0000c1378788
x26: ffffffffffffffff x25: ffff800082747de0 x24: ffff0000c0ae9898
x23: ffff8000819155f8 x22: ffff0000c0ae9888 x21: ffff0000c0ae9808
x20: ffff0000c0ae9818 x19: ffff0000c0ae9788 x18: 000000000000001c
x17: 0000000000000018 x16: 0000000000000040 x15: 0000000000000000
x14: 0000000000000001 x13: 0000000000000000 x12: 0000000000002710
x11: ffff0000c0ae9898 x10: ffff0000c1299b58 x9 : 0000000000000001
x8 : ffff0000c0ae9900 x7 : ffff8000828db000 x6 : 0000000000005040
x5 : ffffffffffffffff x4 : ffffffdfc05c8aa0 x3 : ffff000126470000
x2 : ffffffffffffffff x1 : 0000000000000000 x0 : ffff0000c0ae9788
Call trace:
clear_inode+0x34/0x108 (P)
proc_evict_inode.llvm.1771226604092943895+0x28/0x68
evict+0xec/0x328
iput+0xa8/0x310
dentry_unlink_inode+0xa4/0x188
__dentry_kill+0x74/0x358
shrink_dentry_list+0xc8/0x198
....
prev parent reply other threads:[~2026-06-17 9:40 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-09 10:56 [PATCH v9 0/6] mm/memory-failure: add panic option for unrecoverable pages Breno Leitao
2026-06-09 10:56 ` [PATCH v9 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages Breno Leitao
2026-06-09 10:56 ` [PATCH v9 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE Breno Leitao
2026-06-09 14:41 ` David Hildenbrand (Arm)
2026-06-09 16:15 ` Breno Leitao
2026-06-09 18:41 ` David Hildenbrand (Arm)
2026-06-10 9:53 ` Breno Leitao
2026-06-09 10:56 ` [PATCH v9 3/6] mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages Breno Leitao
2026-06-09 10:56 ` [PATCH v9 4/6] mm/memory-failure: add panic option for unrecoverable pages Breno Leitao
2026-06-09 10:56 ` [PATCH v9 5/6] Documentation: document panic_on_unrecoverable_memory_failure sysctl Breno Leitao
2026-06-09 10:57 ` [PATCH v9 6/6] selftests/mm: add hwpoison-panic destructive test Breno Leitao
2026-06-17 9:40 ` Breno Leitao [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ajJq-uVRBH5SBkwK@gmail.com \
--to=leitao@debian.org \
--cc=akpm@linux-foundation.org \
--cc=corbet@lwn.net \
--cc=david@kernel.org \
--cc=kernel-team@meta.com \
--cc=lance.yang@linux.dev \
--cc=liam@infradead.org \
--cc=linmiaohe@huawei.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=ljs@kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=mhocko@suse.com \
--cc=nao.horiguchi@gmail.com \
--cc=rostedt@goodmis.org \
--cc=rppt@kernel.org \
--cc=shuah@kernel.org \
--cc=skhan@linuxfoundation.org \
--cc=surenb@google.com \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.