From: Breno Leitao <leitao@debian.org>
To: Christian Brauner <brauner@kernel.org>
Cc: linux-fsdevel@vger.kernel.org,
Linus Torvalds <torvalds@linux-foundation.org>,
Chris Mason <mason@kernel.org>,
Thomas Gleixner <tglx@kernel.org>,
x86@kernel.org, kexec@lists.infradead.org,
linuxppc-dev@lists.ozlabs.org,
linux-arm-kernel@lists.infradead.org,
linux-riscv@lists.infradead.org, loongarch@lists.linux.dev,
linux-mips@vger.kernel.org, linux-sh@vger.kernel.org,
linux-kernel@vger.kernel.org,
David Hildenbrand <david@kernel.org>,
stable@vger.kernel.org
Subject: Re: [PATCH RFC 3/4] crash_dump: recover from a machine check in copy_oldmem_page()
Date: Fri, 13 Nov 2026 01:38:22 -0800 [thread overview]
Message-ID: <avbagwxWrmFrjrtV@gmail.com> (raw)
In-Reply-To: <20261112-work-procfs-oldmem-v1-3-ecdcafd99265@kernel.org>
On Thu, Nov 12, 2026 at 05:37:09PM +0100, Christian Brauner wrote:
> A capture kernel reads the memory of the crashed kernel, and that memory
> may well have uncorrectable errors. That is after all what the first
> kernel often died of. Every architecture that maps a page of the old
> kernel and copies it out does so with copy_to_iter(), and a machine
> check in that copy takes the capture kernel down.
>
> On x86 a machine check in the kernel is only recovered from if the
> instruction that took it has a machine check safe exception table entry
> or copies from userspace (cf. error_context()). The copy of an old page
> is neither. Its source is kernel memory, whether it goes to the buffer
> of a read() or, since commit 4a22fd20379c ("vmcore: convert
> __read_vmcore to use an iov_iter"), into a kernel page for splice(),
> sendfile() and copy_file_range(). So a read of /proc/vmcore that runs
> into a poisoned page panics the capture kernel, and so do vmcore's own
> reads of the ELF headers and notes. Breno sees the result on a fleet:
> the first kernel dies of a machine check and the capture kernel produces
> no usable vmcore at all (cf. [1]).
>
> On book3s64 a machine check in the kernel is recovered from if the
> instruction that took it has an exception table entry (cf.
> mce_common_process_ue()). The user copy has them, so a read() comes up
> short there and read_from_oldmem() returns -EFAULT. The copies into
> kernel pages have none and panic the same way. Other powerpc platforms
> don't recover from a machine check in such a copy at all.
>
> Copy with _copy_mc_to_iter() instead. x86_64 and PPC64 select
> ARCH_HAS_COPY_MC, and on x86_64 and book3s64 the machine check safe copy
> turns the machine check into a short copy, which read_from_oldmem()
> turns into -EFAULT like any other short copy. On x86 that holds for the
> fragile and the enhanced fast string variant, a CPU with neither gets
> the plain copy it has today. Everywhere else _copy_mc_to_iter() is
> _copy_to_iter(). Put the call into a helper, copy_oldmem_to_iter(), so
> that the copy of a page of the old kernel lives in one place and an
> architecture that gains a machine check safe copy later gets this for
> free. The helper does the same check_copy_size() as copy_to_iter(), so
> hardened usercopy keeps checking the copy.
>
> With a v7.4-rc1 capture kernel reading /proc/vmcore and a machine check
> injected into the copy of an old page the capture kernel is gone. A
> machine check panic prints no trace of its own, this one comes from
> panic_sys_info=all_bt:
>
> [ 0.580568] mce: [Hardware Error]: CPU 0: Machine Check Exception: 7 Bank 1: bd80000000000134
> [ 0.580574] mce: [Hardware Error]: RIP 10:<ffffffff81909b11> {_copy_to_iter+0x81/0x850}
> [ 0.580584] mce: [Hardware Error]: TSC 202f2117f ADDR 10000000 MISC 8c
> [ 0.580587] mce: [Hardware Error]: PROCESSOR 2:aa0f02 TIME 1794493617 SOCKET 0 APIC 0 microcode aa0021c
> [ 0.580589] mce: [Hardware Error]: Machine check: Uncorrected unrecoverable error in kernel context
> [ 0.580591] Kernel panic - not syncing: Fatal local machine check
> [ 0.582587] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 7.4.0-rc1-g6628e7d4c6b2 #1 PREEMPT(lazy)
> [ 0.583101] Call Trace:
> [ 0.583198] <#MC>
> [ 0.583279] dump_stack_lvl+0x4d/0x70
> [ 0.583419] nmi_cpu_backtrace.cold+0x18/0x6e
> [ 0.583584] ? __pfx_nmi_raise_cpu_backtrace+0x10/0x10
> [ 0.583772] nmi_trigger_cpumask_backtrace+0xba/0xf0
> [ 0.583955] sys_info+0x83/0xb0
> [ 0.584075] vpanic+0x1b4/0x303
> [ 0.584197] panic+0x66/0x70
> [ 0.584488] mce_panic+0x19e/0x290
> [ 0.584799] do_machine_check+0x627/0x9a0
> [ 0.584953] ? _copy_to_iter+0x81/0x850
> [ 0.585099] exc_machine_check+0x68/0xb0
> [ 0.585246] asm_exc_machine_check+0x1e/0x40
> [ 0.587747] </#MC>
> [ 0.587831] <TASK>
> [ 0.588092] ? iomem_map_sanity_check+0x98/0xb0
> [ 0.588430] ? __ioremap_caller.isra.0+0x1a9/0x300
> [ 0.588604] ? __copy_oldmem_page+0x3d/0xb0
> [ 0.588928] __copy_oldmem_page+0x5e/0xb0
> [ 0.589079] read_from_oldmem.part.0+0xe3/0x140
> [ 0.589247] read_vmcore+0x137/0x280
> [ 0.589381] proc_reg_read_iter+0x49/0x90
> [ 0.589533] vfs_read+0x348/0x490
> [ 0.589666] ksys_read+0x76/0x100
> [ 0.589792] do_syscall_64+0xde/0x4b0
> [ 0.590103] entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> With this patch the same injection ends the read() with -EFAULT, the
> capture kernel logs the machine check and keeps running.
>
> [1]: https://lore.kernel.org/all/avRXzzCSzT9Qj2FC@gmail.com/
>
> Reported-by: Chris Mason <mason@kernel.org>
> Suggested-by: Breno Leitao <leitao@debian.org>
> Fixes: 4ae362be5093 ("[PATCH] kdump: read previous kernel's memory")
> Fixes: 54c32021eb6f ("[PATCH] powerpc: Add arch-dependent copy_oldmem_page")
> Fixes: 4a22fd20379c ("vmcore: convert __read_vmcore to use an iov_iter")
> Cc: stable@vger.kernel.org
> Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
WARNING: multiple messages have this Message-ID (diff)
From: Breno Leitao <leitao@debian.org>
To: Christian Brauner <brauner@kernel.org>
Cc: linux-fsdevel@vger.kernel.org,
Linus Torvalds <torvalds@linux-foundation.org>,
Chris Mason <mason@kernel.org>,
Thomas Gleixner <tglx@kernel.org>,
x86@kernel.org, kexec@lists.infradead.org,
linuxppc-dev@lists.ozlabs.org,
linux-arm-kernel@lists.infradead.org,
linux-riscv@lists.infradead.org, loongarch@lists.linux.dev,
linux-mips@vger.kernel.org, linux-sh@vger.kernel.org,
linux-kernel@vger.kernel.org,
David Hildenbrand <david@kernel.org>,
stable@vger.kernel.org
Subject: Re: [PATCH RFC 3/4] crash_dump: recover from a machine check in copy_oldmem_page()
Date: Fri, 13 Nov 2026 01:38:22 -0800 [thread overview]
Message-ID: <avbagwxWrmFrjrtV@gmail.com> (raw)
In-Reply-To: <20261112-work-procfs-oldmem-v1-3-ecdcafd99265@kernel.org>
On Thu, Nov 12, 2026 at 05:37:09PM +0100, Christian Brauner wrote:
> A capture kernel reads the memory of the crashed kernel, and that memory
> may well have uncorrectable errors. That is after all what the first
> kernel often died of. Every architecture that maps a page of the old
> kernel and copies it out does so with copy_to_iter(), and a machine
> check in that copy takes the capture kernel down.
>
> On x86 a machine check in the kernel is only recovered from if the
> instruction that took it has a machine check safe exception table entry
> or copies from userspace (cf. error_context()). The copy of an old page
> is neither. Its source is kernel memory, whether it goes to the buffer
> of a read() or, since commit 4a22fd20379c ("vmcore: convert
> __read_vmcore to use an iov_iter"), into a kernel page for splice(),
> sendfile() and copy_file_range(). So a read of /proc/vmcore that runs
> into a poisoned page panics the capture kernel, and so do vmcore's own
> reads of the ELF headers and notes. Breno sees the result on a fleet:
> the first kernel dies of a machine check and the capture kernel produces
> no usable vmcore at all (cf. [1]).
>
> On book3s64 a machine check in the kernel is recovered from if the
> instruction that took it has an exception table entry (cf.
> mce_common_process_ue()). The user copy has them, so a read() comes up
> short there and read_from_oldmem() returns -EFAULT. The copies into
> kernel pages have none and panic the same way. Other powerpc platforms
> don't recover from a machine check in such a copy at all.
>
> Copy with _copy_mc_to_iter() instead. x86_64 and PPC64 select
> ARCH_HAS_COPY_MC, and on x86_64 and book3s64 the machine check safe copy
> turns the machine check into a short copy, which read_from_oldmem()
> turns into -EFAULT like any other short copy. On x86 that holds for the
> fragile and the enhanced fast string variant, a CPU with neither gets
> the plain copy it has today. Everywhere else _copy_mc_to_iter() is
> _copy_to_iter(). Put the call into a helper, copy_oldmem_to_iter(), so
> that the copy of a page of the old kernel lives in one place and an
> architecture that gains a machine check safe copy later gets this for
> free. The helper does the same check_copy_size() as copy_to_iter(), so
> hardened usercopy keeps checking the copy.
>
> With a v7.4-rc1 capture kernel reading /proc/vmcore and a machine check
> injected into the copy of an old page the capture kernel is gone. A
> machine check panic prints no trace of its own, this one comes from
> panic_sys_info=all_bt:
>
> [ 0.580568] mce: [Hardware Error]: CPU 0: Machine Check Exception: 7 Bank 1: bd80000000000134
> [ 0.580574] mce: [Hardware Error]: RIP 10:<ffffffff81909b11> {_copy_to_iter+0x81/0x850}
> [ 0.580584] mce: [Hardware Error]: TSC 202f2117f ADDR 10000000 MISC 8c
> [ 0.580587] mce: [Hardware Error]: PROCESSOR 2:aa0f02 TIME 1794493617 SOCKET 0 APIC 0 microcode aa0021c
> [ 0.580589] mce: [Hardware Error]: Machine check: Uncorrected unrecoverable error in kernel context
> [ 0.580591] Kernel panic - not syncing: Fatal local machine check
> [ 0.582587] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 7.4.0-rc1-g6628e7d4c6b2 #1 PREEMPT(lazy)
> [ 0.583101] Call Trace:
> [ 0.583198] <#MC>
> [ 0.583279] dump_stack_lvl+0x4d/0x70
> [ 0.583419] nmi_cpu_backtrace.cold+0x18/0x6e
> [ 0.583584] ? __pfx_nmi_raise_cpu_backtrace+0x10/0x10
> [ 0.583772] nmi_trigger_cpumask_backtrace+0xba/0xf0
> [ 0.583955] sys_info+0x83/0xb0
> [ 0.584075] vpanic+0x1b4/0x303
> [ 0.584197] panic+0x66/0x70
> [ 0.584488] mce_panic+0x19e/0x290
> [ 0.584799] do_machine_check+0x627/0x9a0
> [ 0.584953] ? _copy_to_iter+0x81/0x850
> [ 0.585099] exc_machine_check+0x68/0xb0
> [ 0.585246] asm_exc_machine_check+0x1e/0x40
> [ 0.587747] </#MC>
> [ 0.587831] <TASK>
> [ 0.588092] ? iomem_map_sanity_check+0x98/0xb0
> [ 0.588430] ? __ioremap_caller.isra.0+0x1a9/0x300
> [ 0.588604] ? __copy_oldmem_page+0x3d/0xb0
> [ 0.588928] __copy_oldmem_page+0x5e/0xb0
> [ 0.589079] read_from_oldmem.part.0+0xe3/0x140
> [ 0.589247] read_vmcore+0x137/0x280
> [ 0.589381] proc_reg_read_iter+0x49/0x90
> [ 0.589533] vfs_read+0x348/0x490
> [ 0.589666] ksys_read+0x76/0x100
> [ 0.589792] do_syscall_64+0xde/0x4b0
> [ 0.590103] entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> With this patch the same injection ends the read() with -EFAULT, the
> capture kernel logs the machine check and keeps running.
>
> [1]: https://lore.kernel.org/all/avRXzzCSzT9Qj2FC@gmail.com/
>
> Reported-by: Chris Mason <mason@kernel.org>
> Suggested-by: Breno Leitao <leitao@debian.org>
> Fixes: 4ae362be5093 ("[PATCH] kdump: read previous kernel's memory")
> Fixes: 54c32021eb6f ("[PATCH] powerpc: Add arch-dependent copy_oldmem_page")
> Fixes: 4a22fd20379c ("vmcore: convert __read_vmcore to use an iov_iter")
> Cc: stable@vger.kernel.org
> Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2026-11-13 9:38 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 16:37 [PATCH RFC 0/4] crash_dump: recover from machine checks in copy_oldmem_page() Christian Brauner
2026-11-12 16:37 ` Christian Brauner
2026-11-12 16:37 ` [PATCH RFC 1/4] powerpc/crash_dump: check ioremap_cache() " Christian Brauner
2026-11-12 16:37 ` Christian Brauner
2026-11-12 16:45 ` sashiko-bot
2026-11-12 19:10 ` Geert Uytterhoeven
2026-11-12 19:10 ` Geert Uytterhoeven
2026-11-12 16:37 ` [PATCH RFC 2/4] sh/crash_dump: check ioremap() " Christian Brauner
2026-11-12 16:37 ` Christian Brauner
2026-11-12 16:42 ` sashiko-bot
2026-11-12 19:07 ` Geert Uytterhoeven
2026-11-12 19:07 ` Geert Uytterhoeven
2026-11-12 16:37 ` [PATCH RFC 3/4] crash_dump: recover from a machine check " Christian Brauner
2026-11-12 16:37 ` Christian Brauner
2026-11-12 16:50 ` sashiko-bot
2026-11-13 9:38 ` Breno Leitao [this message]
2026-11-13 9:38 ` Breno Leitao
2026-11-12 16:37 ` [PATCH RFC 4/4] crash_dump: use copy_oldmem_to_iter() everywhere Christian Brauner
2026-11-12 16:37 ` Christian Brauner
2026-11-12 16:43 ` sashiko-bot
2026-11-12 16:40 ` [PATCH RFC 0/4] crash_dump: recover from machine checks in copy_oldmem_page() Christian Brauner
2026-11-12 16:40 ` Christian Brauner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avbagwxWrmFrjrtV@gmail.com \
--to=leitao@debian.org \
--cc=brauner@kernel.org \
--cc=david@kernel.org \
--cc=kexec@lists.infradead.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-sh@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=loongarch@lists.linux.dev \
--cc=mason@kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.