[PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Paweł Anikiel]

Calling core::fmt::write() from rust code while FineIBT is enabled
results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls
core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:
171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

This causes a Control Protection exception, because FineIBT has sealed
off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig
dependency that prevents FineIBT from getting turned on by default
if rust is enabled.

Signed-off-by: Paweł Anikiel <panikiel@google.com>
---
 arch/x86/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 4b9f378e05f6..b31811a92286 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2367,7 +2367,7 @@ config STRICT_SIGALTSTACK_SIZE
 
 config CFI_AUTO_DEFAULT
 	bool "Attempt to use FineIBT by default at boot time"
-	depends on FINEIBT
+	depends on FINEIBT && !RUST
 	default y
 	help
 	  Attempt to use FineIBT by default at boot time. If enabled,
-- 
2.49.0.504.g3bcea36a83-goog

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> Calling core::fmt::write() from rust code while FineIBT is enabled
> results in a kernel panic:
> 
> [ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
> [ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> [ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
> [ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
> [ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
> [ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
> [ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
> [ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
> [ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
> [ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
> [ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
> [ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
> [ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
> [ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
> [ 4614.336777] PKRU: 55555554
> [ 4614.339786] Call Trace:
> [ 4614.342524]  <TASK>
> [ 4614.344867]  ? __die_body+0x69/0xb0
> [ 4614.348786]  ? die+0xa9/0xd0
> [ 4614.352000]  ? do_trap+0x89/0x160
> [ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
> [ 4614.360413]  ? handle_invalid_op+0x69/0x90
> [ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
> [ 4614.369654]  ? exc_invalid_op+0x36/0x60
> [ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
> [ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
> [ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
> [ 4614.388005]  exc_control_protection+0x49/0x70
> [ 4614.392871]  asm_exc_control_protection+0x2b/0x60
> [ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
> [ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
> [ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
> [ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
> [ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
> [ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
> [ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
> [ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
> [ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
> [ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250
> 
> This happens because core::fmt::write() calls
> core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> 
> library/core/src/fmt/rt.rs:
> 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> 172     // it here is an explicit CFI violation.
> 173     #[allow(inline_no_sanitize)]
> 174     #[no_sanitize(cfi, kcfi)]
> 175     #[inline]
> 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> 
> This causes a Control Protection exception, because FineIBT has sealed
> off the original function's endbr64.
> 
> This makes rust currently incompatible with FineIBT. Add a Kconfig
> dependency that prevents FineIBT from getting turned on by default
> if rust is enabled.

Why ?!, what's wrong with removing that no_sanitize() instead?

How is it still compatible with kCFI and not with FineIBT?

NAK

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 02:36:02PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> > Calling core::fmt::write() from rust code while FineIBT is enabled
> > results in a kernel panic:
> > 
> > [ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
> > [ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> > [ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
> > [ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
> > [ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
> > [ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
> > [ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
> > [ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
> > [ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
> > [ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
> > [ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
> > [ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
> > [ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
> > [ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
> > [ 4614.336777] PKRU: 55555554
> > [ 4614.339786] Call Trace:
> > [ 4614.342524]  <TASK>
> > [ 4614.344867]  ? __die_body+0x69/0xb0
> > [ 4614.348786]  ? die+0xa9/0xd0
> > [ 4614.352000]  ? do_trap+0x89/0x160
> > [ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
> > [ 4614.360413]  ? handle_invalid_op+0x69/0x90
> > [ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
> > [ 4614.369654]  ? exc_invalid_op+0x36/0x60
> > [ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
> > [ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
> > [ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
> > [ 4614.388005]  exc_control_protection+0x49/0x70
> > [ 4614.392871]  asm_exc_control_protection+0x2b/0x60
> > [ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
> > [ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
> > [ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
> > [ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
> > [ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
> > [ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
> > [ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
> > [ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
> > [ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
> > [ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250
> > 
> > This happens because core::fmt::write() calls
> > core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> > 
> > library/core/src/fmt/rt.rs:
> > 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> > 172     // it here is an explicit CFI violation.
> > 173     #[allow(inline_no_sanitize)]
> > 174     #[no_sanitize(cfi, kcfi)]
> > 175     #[inline]
> > 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> > 
> > This causes a Control Protection exception, because FineIBT has sealed
> > off the original function's endbr64.
> > 
> > This makes rust currently incompatible with FineIBT. Add a Kconfig
> > dependency that prevents FineIBT from getting turned on by default
> > if rust is enabled.
> 
> Why ?!, what's wrong with removing that no_sanitize() instead?
> 
> How is it still compatible with kCFI and not with FineIBT?

FWIW, CFI violations of any kind are a no-no. They're not accepted in C
and they're not accepted in Rust. This is clearly a Rust bug that needs
to be fixed ASAP. Disabling CFI is not an option.

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 02:45:26PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 10, 2025 at 02:36:02PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> > > Calling core::fmt::write() from rust code while FineIBT is enabled
> > > results in a kernel panic:

> > > This happens because core::fmt::write() calls
> > > core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> > > 
> > > library/core/src/fmt/rt.rs:
> > > 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> > > 172     // it here is an explicit CFI violation.
> > > 173     #[allow(inline_no_sanitize)]
> > > 174     #[no_sanitize(cfi, kcfi)]
> > > 175     #[inline]
> > > 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> > > 

Miguel, I cannot find this code in the kernel tree. Is this again
because Rust is not free-standing and relies on external code?

Can you please fix that. Building against external code that is not
under our control is a problem.

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Paweł Anikiel]

On Thu, Apr 10, 2025 at 3:09 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 02:45:26PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 10, 2025 at 02:36:02PM +0200, Peter Zijlstra wrote:
> > > On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> > > > Calling core::fmt::write() from rust code while FineIBT is enabled
> > > > results in a kernel panic:
>
> > > > This happens because core::fmt::write() calls
> > > > core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> > > >
> > > > library/core/src/fmt/rt.rs:
> > > > 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> > > > 172     // it here is an explicit CFI violation.
> > > > 173     #[allow(inline_no_sanitize)]
> > > > 174     #[no_sanitize(cfi, kcfi)]
> > > > 175     #[inline]
> > > > 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> > > >
>
> Miguel, I cannot find this code in the kernel tree. Is this again
> because Rust is not free-standing and relies on external code?

The code I referenced is from here:
https://github.com/rust-lang/rust/blob/master/library/core/src/fmt/rt.rs#L139

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Thu, Apr 10, 2025 at 3:09 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 02:45:26PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 10, 2025 at 02:36:02PM +0200, Peter Zijlstra wrote:
> > > On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> > > > Calling core::fmt::write() from rust code while FineIBT is enabled
> > > > results in a kernel panic:
>
> > > > This happens because core::fmt::write() calls
> > > > core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> > > >
> > > > library/core/src/fmt/rt.rs:
> > > > 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> > > > 172     // it here is an explicit CFI violation.
> > > > 173     #[allow(inline_no_sanitize)]
> > > > 174     #[no_sanitize(cfi, kcfi)]
> > > > 175     #[inline]
> > > > 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> > > >
>
> Miguel, I cannot find this code in the kernel tree. Is this again
> because Rust is not free-standing and relies on external code?
>
> Can you please fix that. Building against external code that is not
> under our control is a problem.

It's from the core standard library, which is shipped together with
the compiler.

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 3:09 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Miguel, I cannot find this code in the kernel tree. Is this again
> because Rust is not free-standing and relies on external code?

Yes, this comes from the `core` standard library. If it requires a
change, then it needs to be fixed upstream.

> Can you please fix that. Building against external code that is not
> under our control is a problem.

The only way to fix it would be to vendor the standard library, for
each Rust compiler version -- and there is one new every 6 weeks.
Thus, quite a few copies of it. Or keep patches (one per version) that
we apply on the fly.

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:21:44PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 3:09 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > Miguel, I cannot find this code in the kernel tree. Is this again
> > because Rust is not free-standing and relies on external code?
> 
> Yes, this comes from the `core` standard library. If it requires a
> change, then it needs to be fixed upstream.
> 
> > Can you please fix that. Building against external code that is not
> > under our control is a problem.
> 
> The only way to fix it would be to vendor the standard library, for
> each Rust compiler version -- and there is one new every 6 weeks.
> Thus, quite a few copies of it. Or keep patches (one per version) that
> we apply on the fly.

New compiler can't build old core?

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 3:26 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> New compiler can't build old core?

No, that is not intended to work -- `core` is tied to the compiler.

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:27:55PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 3:26 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > New compiler can't build old core?
> 
> No, that is not intended to work -- `core` is tied to the compiler.

OMFG, you just gave me another reason to hate on this rust-bucket of a
language :-/ What language can't even build an old version of its own
runtime :-(

Anyway, given this core nonsense is now CFI unsound, it is no longer
suitable to build the kernel. So either refrain from using part of the
core that trips this, or get it fixed asap.

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 3:34 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Anyway, given this core nonsense is now CFI unsound, it is no longer
> suitable to build the kernel. So either refrain from using part of the
> core that trips this, or get it fixed asap.

We discussed it a bit yesterday. Still, until it gets resolved one way
or another, we should be avoiding people to build the kernel like
that, no?

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:54:32PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 3:34 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > Anyway, given this core nonsense is now CFI unsound, it is no longer
> > suitable to build the kernel. So either refrain from using part of the
> > core that trips this, or get it fixed asap.
> 
> We discussed it a bit yesterday. Still, until it gets resolved one way
> or another, we should be avoiding people to build the kernel like
> that, no?

Remove the offending Rust code? Afaict from this github issue, it is
just some formatting nonsense. Surely code can be adjusted to not use
that?

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 3:57 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Remove the offending Rust code? Afaict from this github issue, it is
> just some formatting nonsense. Surely code can be adjusted to not use
> that?

If you mean not using the formatting machinery from our side, then
that is a major change -- we should just fix it upstream, really.

If you mean fixing it upstream, definitely, but we should still
prevent people from building an invalid kernel, i.e. when Alice's PR
or similar lands upstream, then we can relax the `depends on` based on
the Rust version (which is something we have done for other bits).

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 04:05:54PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 3:57 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > Remove the offending Rust code? Afaict from this github issue, it is
> > just some formatting nonsense. Surely code can be adjusted to not use
> > that?
> 
> If you mean not using the formatting machinery from our side, then
> that is a major change -- we should just fix it upstream, really.

That is what I mean, yeah. I've no idea what is or is not common in
this rust code. It still looks like line noise to me :/

But given that FineIBT isn't at all new, I was under the assumption that
this was some shiny new rust driver that did something that hadn't been
done before. And it could now not do that.

> If you mean fixing it upstream, definitely, but we should still
> prevent people from building an invalid kernel, i.e. when Alice's PR
> or similar lands upstream, then we can relax the `depends on` based on
> the Rust version (which is something we have done for other bits).

So why wasn't any of this a problem when Rust enabled kCFI? Surely the
testing back then included FineIBT. That has been in longer than rust's
kcfi support (integer type confusion etc.).

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Thu, Apr 10, 2025 at 4:15 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 04:05:54PM +0200, Miguel Ojeda wrote:
> > On Thu, Apr 10, 2025 at 3:57 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > If you mean fixing it upstream, definitely, but we should still
> > prevent people from building an invalid kernel, i.e. when Alice's PR
> > or similar lands upstream, then we can relax the `depends on` based on
> > the Rust version (which is something we have done for other bits).
>
> So why wasn't any of this a problem when Rust enabled kCFI? Surely the
> testing back then included FineIBT. That has been in longer than rust's
> kcfi support (integer type confusion etc.).

I assume this has been a problem for a while. The testing we did with
FineIBT must not have triggered this codepath. But Rust drivers have
used the codepath for a long time.

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Thu, Apr 10, 2025 at 3:34 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 03:27:55PM +0200, Miguel Ojeda wrote:
> > On Thu, Apr 10, 2025 at 3:26 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > New compiler can't build old core?
> >
> > No, that is not intended to work -- `core` is tied to the compiler.
>
> OMFG, you just gave me another reason to hate on this rust-bucket of a
> language :-/ What language can't even build an old version of its own
> runtime :-(

I don't think there's anything unreasonable about having a place to
put logic that depends on compiler internals.

> Anyway, given this core nonsense is now CFI unsound, it is no longer
> suitable to build the kernel. So either refrain from using part of the
> core that trips this, or get it fixed asap.

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:59:53PM +0200, Alice Ryhl wrote:
> On Thu, Apr 10, 2025 at 3:34 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Thu, Apr 10, 2025 at 03:27:55PM +0200, Miguel Ojeda wrote:
> > > On Thu, Apr 10, 2025 at 3:26 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > > >
> > > > New compiler can't build old core?
> > >
> > > No, that is not intended to work -- `core` is tied to the compiler.
> >
> > OMFG, you just gave me another reason to hate on this rust-bucket of a
> > language :-/ What language can't even build an old version of its own
> > runtime :-(
> 
> I don't think there's anything unreasonable about having a place to
> put logic that depends on compiler internals.

Having compiler internals exposed to such an extend that programs can
make use of them _is_ weird. It creates the whole problem of where to
put it.

C++ doesn't have anything like this afaik. Esp. if you disable rtti and
exception muck, you get a very minimal runtime. And even if you don't,
it certainly doesn't break every 6 weeks. ABIs and all that.

Maybe I've been living in C/C++ land for too long, but you either expose
a full language feature and then everybody can use it, runtime,
libraries and user code alike, or you don't.

> > Anyway, given this core nonsense is now CFI unsound, it is no longer
> > suitable to build the kernel. So either refrain from using part of the
> > core that trips this, or get it fixed asap.
> 
> I submitted a PR that I believe should fix it:
> https://github.com/rust-lang/rust/pull/139632

Thanks!

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 4:08 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Maybe I've been living in C/C++ land for too long, but you either expose
> a full language feature and then everybody can use it, runtime,
> libraries and user code alike, or you don't.

I find it unlikely that C++ stdlibs happen to build and work properly
with compilers they are not tested for. Say, MS' STL with GCC or
libc++ with MSVC.

At the very least, you need to agree on builtins or `#ifdef` stuff
manually for magic bits, and you need to avoid to rely on any compiler
detail (or bug... :) anywhere (say, the non-conforming template
behavior from old MSVC), and so on. So you need an active effort to
make it work, I would expect.

And with new complex features like modules, I find it even more
unlikely the first implementations of a compiler's stdlib would happen
to work on the first implementation of the feature of another
compiler...

Now, it is true that Rust's `core` uses a lot of internal features,
precisely because they don't expect to be built by anything else that
the current (and current - 1) compiler, so they actually take
advantage of that.

It would perhaps be nice to split the "really requires magic" in
`core` from the rest somehow. In this case it wouldn't have helped
though, since the formatting machinery still uses builtins last time I
looked.

Relatedly, GCC Rust's goal is to build an old `core` at the moment, so
that they have a fixed set of things to solve.

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 04:54:20PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 4:08 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > Maybe I've been living in C/C++ land for too long, but you either expose
> > a full language feature and then everybody can use it, runtime,
> > libraries and user code alike, or you don't.
> 
> I find it unlikely that C++ stdlibs happen to build and work properly
> with compilers they are not tested for. Say, MS' STL with GCC or
> libc++ with MSVC.

Ignoring MSVC, which is a horrible joke of a compiler (or did Sutter
finally cluebat that team into building a sane compiler?), there are
definitely cross compiler STL implementations around.

> At the very least, you need to agree on builtins or `#ifdef` stuff
> manually for magic bits, and you need to avoid to rely on any compiler
> detail (or bug... :) anywhere (say, the non-conforming template
> behavior from old MSVC), and so on. So you need an active effort to
> make it work, I would expect.

Again, with the exception of MSVC, Boost builds on most C++ compilers
and is the staging ground for many new library features.

> And with new complex features like modules, I find it even more
> unlikely the first implementations of a compiler's stdlib would happen
> to work on the first implementation of the feature of another
> compiler...

Sure, shit happens, but... at least they try, it *should* work.

And the old STL (although STL really is far more a library than runtime,
it being fully optional) will most certainly build on a new compiler
from the same family.

And I know Linus hates on C++ something mighty, but in this parallel
universe where he doesn't, I would still recommend the kernel to not use
STL and instead build its own libraries (or borrow some nice pieces here
and there).

> Now, it is true that Rust's `core` uses a lot of internal features,
> precisely because they don't expect to be built by anything else that
> the current (and current - 1) compiler, so they actually take
> advantage of that.

So I don't think this is a good thing. Even builtin stuff should be
more stable than this.

> It would perhaps be nice to split the "really requires magic" in
> `core` from the rest somehow. In this case it wouldn't have helped
> though, since the formatting machinery still uses builtins last time I
> looked.

Right, to the point where we can carry a copy of the non-magic part
locally that matches the minimum version requirement. And no other bits
are to be used by in-tree rust code.

> Relatedly, GCC Rust's goal is to build an old `core` at the moment, so
> that they have a fixed set of things to solve.

Very sensible. The Rust project should have an stable core subset / variant
:-)

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 5:14 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Ignoring MSVC, which is a horrible joke of a compiler (or did Sutter
> finally cluebat that team into building a sane compiler?), there are
> definitely cross compiler STL implementations around.

Definitely, there are, e.g. libc++ and libstdc++, but my point was
that it doesn't happen by accident, i.e. it requires effort (perhaps
even on both sides).

[ As for MSVC, last time I used it (2020?), it was way better compared
to the VC++6 times -- at least they started to try to be conforming,
e.g. all the work behind `/permissive-`, including the two-phase name
lookup I mentioned. They also improved a bunch of thing for C to be
able to build projects out there. ]

> Again, with the exception of MSVC, Boost builds on most C++ compilers
> and is the staging ground for many new library features.

Hmm... I am not sure what you mean -- Boost works with MSVC, and Boost
tries to use standard C++. Even if it may rely on hacks or compiler
extensions at points, Boost does not need to implement e.g. a magic
`std::launder`, and I hope they don't use builtins etc.

> And the old STL (although STL really is far more a library than runtime,
> it being fully optional) will most certainly build on a new compiler
> from the same family.

[ To clarify, by "STL" above I was referring to the Microsoft's STL
project (i.e. they call it STL, but it is the full C++ standard
library). ]

The pure "STL" stuff, i.e. containers and algorithms and such, yeah,
that may not require any magic. Some Rust `core` types and functions
can also be reimplemented in normal Rust too.

> And I know Linus hates on C++ something mighty, but in this parallel
> universe where he doesn't, I would still recommend the kernel to not use
> STL and instead build its own libraries (or borrow some nice pieces here
> and there).

Yeah, we take that approach when needed; e.g. nowadays we don't use
`alloc`, which is another part of the Rust standard library that
contains things like the equivalent to `std::vector`, and we also have
our own equivalents of `core` types sometimes, e.g. our `CStr`.

And, yeah, I agree it would be ideal to be able to reuse/customize
some parts more.

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Thu, Apr 10, 2025 at 4:08 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 03:59:53PM +0200, Alice Ryhl wrote:
> > On Thu, Apr 10, 2025 at 3:34 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Thu, Apr 10, 2025 at 03:27:55PM +0200, Miguel Ojeda wrote:
> > > > On Thu, Apr 10, 2025 at 3:26 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > > > >
> > > > > New compiler can't build old core?
> > > >
> > > > No, that is not intended to work -- `core` is tied to the compiler.
> > >
> > > OMFG, you just gave me another reason to hate on this rust-bucket of a
> > > language :-/ What language can't even build an old version of its own
> > > runtime :-(
> >
> > I don't think there's anything unreasonable about having a place to
> > put logic that depends on compiler internals.
>
> Having compiler internals exposed to such an extend that programs can
> make use of them _is_ weird. It creates the whole problem of where to
> put it.
>
> C++ doesn't have anything like this afaik. Esp. if you disable rtti and
> exception muck, you get a very minimal runtime. And even if you don't,
> it certainly doesn't break every 6 weeks. ABIs and all that.
>
> Maybe I've been living in C/C++ land for too long, but you either expose
> a full language feature and then everybody can use it, runtime,
> libraries and user code alike, or you don't.

Well, I would say that Rust has just decided to expose such language
features in a different way from how C/C++ exposes them. Core is *how*
rust exposes the features. There are quite a few language features
that are exposed via a type or method in core that you use like any
other type or method. I think it works relatively well because you can
expose new features by adding things to core, which means that you
don't need any new syntax for the feature. It's just ordinary method
calls into code that you could not implement yourself.

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 4:00 PM Alice Ryhl <aliceryhl@google.com> wrote:
>
> I submitted a PR that I believe should fix it:
> https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Peter: are you OK with landing a patch like this? Well, modified to
look like this now that we know the version:

    depends on FINEIBT
    depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

Thanks!

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Tue, Apr 15, 2025 at 05:15:31PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 4:00 PM Alice Ryhl <aliceryhl@google.com> wrote:
> >
> > I submitted a PR that I believe should fix it:
> > https://github.com/rust-lang/rust/pull/139632
> 
> This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.
> 
> Peter: are you OK with landing a patch like this? Well, modified to
> look like this now that we know the version:
> 
>     depends on FINEIBT
>     depends on !RUST || RUSTC_VERSION >= 108800
> 
> (assuming we confirm the nightly build works properly)

With that change:
Reviewed-by: Alice Ryhl <aliceryhl@google.com>

I have tested locally that the upstream PR works together with CFI,
although I did it by backporting the PR to the rustc version I'm using
rather than by using the latest nightly. Not sure if a Tested-by on
*this* patch is warranted from that test ... but if you think so feel
free to add that too.

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Tue, Apr 15, 2025 at 05:15:31PM +0200, Miguel Ojeda wrote:
> On Thu, Apr 10, 2025 at 4:00 PM Alice Ryhl <aliceryhl@google.com> wrote:
> >
> > I submitted a PR that I believe should fix it:
> > https://github.com/rust-lang/rust/pull/139632
> 
> This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.
> 
> Peter: are you OK with landing a patch like this? Well, modified to
> look like this now that we know the version:
> 
>     depends on FINEIBT
>     depends on !RUST || RUSTC_VERSION >= 108800
> 
> (assuming we confirm the nightly build works properly)

I don't much like it -- disabling FineIBT at config time like this also
kills the CFI type rehash. Not to mention that FineIBT is a Spectre-BHI
mitigation, esp. with that arity thing on top.

I don't suppose we can simply mandate this rust version?

Speaking of which, I should go install this latest rust thing....

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Kees Cook]

On Wed, Apr 16, 2025 at 10:20:40PM +0200, Peter Zijlstra wrote:
> On Tue, Apr 15, 2025 at 05:15:31PM +0200, Miguel Ojeda wrote:
> > On Thu, Apr 10, 2025 at 4:00 PM Alice Ryhl <aliceryhl@google.com> wrote:
> > >
> > > I submitted a PR that I believe should fix it:
> > > https://github.com/rust-lang/rust/pull/139632
> > 
> > This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.
> > 
> > Peter: are you OK with landing a patch like this? Well, modified to
> > look like this now that we know the version:
> > 
> >     depends on FINEIBT
> >     depends on !RUST || RUSTC_VERSION >= 108800
> > 
> > (assuming we confirm the nightly build works properly)
> 
> I don't much like it -- disabling FineIBT at config time like this also
> kills the CFI type rehash.

This isn't disabling CONFIG_FINEIBT (which gates cfi_rand), it's making
FineIBT not enabled by default at boot time. This is actually when I
created CONFIG_CFI_AUTO_DEFAULT: to be able to have kCFI _with_ type
rehashing still enabled.

> Not to mention that FineIBT is a Spectre-BHI
> mitigation, esp. with that arity thing on top.

That's certainly true, but the needed overlapping corner cases seem to
make this tolerable?

> I don't suppose we can simply mandate this rust version?

Perhaps capture the needed version in the Kconfig change at least, so it
becomes self-documenting.

-Kees

-- 
Kees Cook

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Wed, Apr 16, 2025 at 02:51:55PM -0700, Kees Cook wrote:
> On Wed, Apr 16, 2025 at 10:20:40PM +0200, Peter Zijlstra wrote:
> > On Tue, Apr 15, 2025 at 05:15:31PM +0200, Miguel Ojeda wrote:
> > > On Thu, Apr 10, 2025 at 4:00 PM Alice Ryhl <aliceryhl@google.com> wrote:
> > > >
> > > > I submitted a PR that I believe should fix it:
> > > > https://github.com/rust-lang/rust/pull/139632
> > > 
> > > This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.
> > > 
> > > Peter: are you OK with landing a patch like this? Well, modified to
> > > look like this now that we know the version:
> > > 
> > >     depends on FINEIBT
> > >     depends on !RUST || RUSTC_VERSION >= 108800
> > > 
> > > (assuming we confirm the nightly build works properly)
> > 
> > I don't much like it -- disabling FineIBT at config time like this also
> > kills the CFI type rehash.
> 
> This isn't disabling CONFIG_FINEIBT (which gates cfi_rand), it's making
> FineIBT not enabled by default at boot time. This is actually when I
> created CONFIG_CFI_AUTO_DEFAULT: to be able to have kCFI _with_ type
> rehashing still enabled.

Urgh, yeah, reading hard :-/

OK, Ack

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

On Thu, Apr 17, 2025 at 10:18 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Urgh, yeah, reading hard :-/
>
> OK, Ack

Thanks Peter (and Kees for clarifying) -- not sure how you/others
prefer to route this patch or if you expect a v2, but I got the
following (attached). I converted your Ack.

Cheers,
Miguel

[-- Attachment #2: 0001-x86-Kconfig-make-CFI_AUTO_DEFAULT-depend-on-RUST-or-.patch --]
[-- Type: text/x-patch, Size: 5458 bytes --]

From 2c6636c82f8c6c18e18c085fe69d0cd3dbe3b991 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pawe=C5=82=20Anikiel?= <panikiel@google.com>
Date: Thu, 10 Apr 2025 11:54:20 +0000
Subject: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST or Rust >=
 1.88
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Calling core::fmt::write() from rust code while FineIBT is enabled
results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls
core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:
171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

This causes a Control Protection exception, because FineIBT has sealed
off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig
dependency that prevents FineIBT from getting turned on by default
if rust is enabled.

[ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
  and thus we relaxed the condition with Rust >= 1.88.

  When `objtool` lands checking for this with e.g. [2], the plan is
  to ideally run that in upstream Rust's CI to prevent regressions
  early [3], since we do not control `core`'s source code.

  Alice tested the Rust PR backported to an older compiler.

    - Miguel ]

Signed-off-by: Paweł Anikiel <panikiel@google.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Link: https://github.com/rust-lang/rust/pull/139632 [1]
Link: https://lore.kernel.org/rust-for-linux/20250410154556.GB9003@noisy.programming.kicks-ass.net/ [2]
Link: https://github.com/rust-lang/rust/pull/139632#issuecomment-2801950873 [3]
Link: https://lore.kernel.org/r/20250410115420.366349-1-panikiel@google.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
---
 arch/x86/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 4b9f378e05f6..5873c9e39919 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2368,6 +2368,7 @@ config STRICT_SIGALTSTACK_SIZE
 config CFI_AUTO_DEFAULT
 	bool "Attempt to use FineIBT by default at boot time"
 	depends on FINEIBT
+	depends on !RUST || RUSTC_VERSION >= 108800
 	default y
 	help
 	  Attempt to use FineIBT by default at boot time. If enabled,

base-commit: c1b4071ec3a6a594df6c49bf8f04a60a88072525
-- 
2.49.0

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 17, 2025 at 08:40:24PM +0200, Miguel Ojeda wrote:

> From 2c6636c82f8c6c18e18c085fe69d0cd3dbe3b991 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Pawe=C5=82=20Anikiel?= <panikiel@google.com>
> Date: Thu, 10 Apr 2025 11:54:20 +0000
> Subject: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST or Rust >=
>  1.88
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> Calling core::fmt::write() from rust code while FineIBT is enabled
> results in a kernel panic:
> 
> [ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
> [ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> [ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
> [ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
> [ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0

Might want to 'compress' the splat a bit, lots of useless bits on etc.

> [ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
> [ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
> [ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
> [ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
> [ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
> [ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
> [ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
> [ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
> [ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
> [ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250
> 
> This happens because core::fmt::write() calls
> core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> 
> library/core/src/fmt/rt.rs:
> 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> 172     // it here is an explicit CFI violation.
> 173     #[allow(inline_no_sanitize)]
> 174     #[no_sanitize(cfi, kcfi)]
> 175     #[inline]
> 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> 
> This causes a Control Protection exception, because FineIBT has sealed
> off the original function's endbr64.

Right, doing an indirect call without the CFI adornment to a CFI adorned
function is explicitly not supported.

> This makes rust currently incompatible with FineIBT. Add a Kconfig
> dependency that prevents FineIBT from getting turned on by default
> if rust is enabled.
> 
> [ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
>   and thus we relaxed the condition with Rust >= 1.88.
> 
>   When `objtool` lands checking for this with e.g. [2], the plan is
>   to ideally run that in upstream Rust's CI to prevent regressions
>   early [3], since we do not control `core`'s source code.

I'm still firmly of the opinion Rust should grow up and have a stable
core which we can pull into the kernel. Relying on that much out of tree
code is 'unfortunate'.

>   Alice tested the Rust PR backported to an older compiler.
> 
>     - Miguel ]
> 
> Signed-off-by: Paweł Anikiel <panikiel@google.com>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Acked-by: Peter Zijlstra <peterz@infradead.org>
> Link: https://github.com/rust-lang/rust/pull/139632 [1]
> Link: https://lore.kernel.org/rust-for-linux/20250410154556.GB9003@noisy.programming.kicks-ass.net/ [2]
> Link: https://github.com/rust-lang/rust/pull/139632#issuecomment-2801950873 [3]
> Link: https://lore.kernel.org/r/20250410115420.366349-1-panikiel@google.com
> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
> ---
>  arch/x86/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 4b9f378e05f6..5873c9e39919 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2368,6 +2368,7 @@ config STRICT_SIGALTSTACK_SIZE
>  config CFI_AUTO_DEFAULT
>  	bool "Attempt to use FineIBT by default at boot time"
>  	depends on FINEIBT
> +	depends on !RUST || RUSTC_VERSION >= 108800
>  	default y
>  	help
>  	  Attempt to use FineIBT by default at boot time. If enabled,
> 
> base-commit: c1b4071ec3a6a594df6c49bf8f04a60a88072525
> -- 
> 2.49.0
> 

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 17, 2025 at 8:40 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> Thanks Peter (and Kees for clarifying) -- not sure how you/others
> prefer to route this patch or if you expect a v2, but I got the
> following (attached). I converted your Ack.

Applied to `rust-fixes` -- thanks everyone!

(If someone else prefers to carry it, please shout)

    [ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
      and thus we relaxed the condition with Rust >= 1.88.

      When `objtool` lands checking for this with e.g. [2], the plan is
      to ideally run that in upstream Rust's CI to prevent regressions
      early [3], since we do not control `core`'s source code.

      Alice tested the Rust PR backported to an older compiler.

      Peter would like that Rust provides a stable `core` which can be
      pulled into the kernel: "Relying on that much out of tree code is
      'unfortunate'".

        - Miguel ]

    [ Reduced splat. - Miguel ]

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Wed, May 07, 2025 at 12:19:11AM +0200, Miguel Ojeda wrote:
> On Thu, Apr 17, 2025 at 8:40 PM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> >
> > Thanks Peter (and Kees for clarifying) -- not sure how you/others
> > prefer to route this patch or if you expect a v2, but I got the
> > following (attached). I converted your Ack.
> 
> Applied to `rust-fixes` -- thanks everyone!
> 
> (If someone else prefers to carry it, please shout)
> 
>     [ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
>       and thus we relaxed the condition with Rust >= 1.88.
> 
>       When `objtool` lands checking for this with e.g. [2], the plan is
>       to ideally run that in upstream Rust's CI to prevent regressions
>       early [3], since we do not control `core`'s source code.
> 
>       Alice tested the Rust PR backported to an older compiler.
> 
>       Peter would like that Rust provides a stable `core` which can be
>       pulled into the kernel: "Relying on that much out of tree code is
>       'unfortunate'".
> 
>         - Miguel ]
> 
>     [ Reduced splat. - Miguel ]

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right
approach.

Shouldn't the depends on clause go on `config FINEIBT` instead? After
all, the current patch just means that you can't make FineIBT the
default option. But you can still pass kcfi=fineibt on boot to enble
FineIBT which would result in the same crash.

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Fri, May 9, 2025 at 10:46 AM Alice Ryhl <aliceryhl@google.com> wrote:
>
> Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right
> approach.
>
> Shouldn't the depends on clause go on `config FINEIBT` instead? After
> all, the current patch just means that you can't make FineIBT the
> default option. But you can still pass kcfi=fineibt on boot to enble
> FineIBT which would result in the same crash.

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

If another approach is needed, then we can change it on top if that is
OK (I am about to submit the PR).

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Paweł Anikiel]

On Fri, May 9, 2025 at 11:04 AM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Fri, May 9, 2025 at 10:46 AM Alice Ryhl <aliceryhl@google.com> wrote:
> >
> > Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right
> > approach.
> >
> > Shouldn't the depends on clause go on `config FINEIBT` instead? After
> > all, the current patch just means that you can't make FineIBT the
> > default option. But you can still pass kcfi=fineibt on boot to enble
> > FineIBT which would result in the same crash.
>
> I thought the intention was to avoid disabling FineIBT at config time:
> https://lore.kernel.org/all/202504161442.66CE2596@keescook/

I think you mean enabling FineIBT - yes, this patch does exactly that,
it's still possible to enable it by hand with cfi=

> If another approach is needed, then we can change it on top if that is
> OK (I am about to submit the PR).

Putting it on CONFIG_FINEIBT was my initial idea, but it turns out
that this creates a dependency cycle in Kconfig:

error: recursive dependency detected!
        symbol RUST depends on CALL_PADDING
        symbol CALL_PADDING is selected by FINEIBT
        symbol FINEIBT depends on RUST

Not sure if it's possible to make it work.

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Alice Ryhl]

On Fri, May 09, 2025 at 11:11:48AM +0200, Paweł Anikiel wrote:
> On Fri, May 9, 2025 at 11:04 AM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> >
> > On Fri, May 9, 2025 at 10:46 AM Alice Ryhl <aliceryhl@google.com> wrote:
> > >
> > > Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right
> > > approach.
> > >
> > > Shouldn't the depends on clause go on `config FINEIBT` instead? After
> > > all, the current patch just means that you can't make FineIBT the
> > > default option. But you can still pass kcfi=fineibt on boot to enble
> > > FineIBT which would result in the same crash.
> >
> > I thought the intention was to avoid disabling FineIBT at config time:
> > https://lore.kernel.org/all/202504161442.66CE2596@keescook/
> 
> I think you mean enabling FineIBT - yes, this patch does exactly that,
> it's still possible to enable it by hand with cfi=
> 
> > If another approach is needed, then we can change it on top if that is
> > OK (I am about to submit the PR).
> 
> Putting it on CONFIG_FINEIBT was my initial idea, but it turns out
> that this creates a dependency cycle in Kconfig:
> 
> error: recursive dependency detected!
>         symbol RUST depends on CALL_PADDING
>         symbol CALL_PADDING is selected by FINEIBT
>         symbol FINEIBT depends on RUST
> 
> Not sure if it's possible to make it work.

To fix that, we probably have to do one of:

Option 1: Upgrade the rustc MSRV past 1.81 and drop the depends on
          CALL_PADDING.

Option 2: Split `config FINEIBT` into two options:

config HAVE_FINEIBT
	def_bool y
	depends on X86_KERNEL_IBT && CFI_CLANG && MITIGATION_RETPOLINE
	depends on !RUST || RUSTC_VERSION >= 108800

config FINEIBT
	bool "Accept cfi=fineibt option at boot"
	depends on HAVE_FINEIBT
	select CALL_PADDING

Alice

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Kees Cook]

On Fri, May 09, 2025 at 09:39:37AM +0000, Alice Ryhl wrote:
> On Fri, May 09, 2025 at 11:11:48AM +0200, Paweł Anikiel wrote:
> > On Fri, May 9, 2025 at 11:04 AM Miguel Ojeda
> > <miguel.ojeda.sandonis@gmail.com> wrote:
> > >
> > > On Fri, May 9, 2025 at 10:46 AM Alice Ryhl <aliceryhl@google.com> wrote:
> > > >
> > > > Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right
> > > > approach.
> > > >
> > > > Shouldn't the depends on clause go on `config FINEIBT` instead? After
> > > > all, the current patch just means that you can't make FineIBT the
> > > > default option. But you can still pass kcfi=fineibt on boot to enble
> > > > FineIBT which would result in the same crash.
> > >
> > > I thought the intention was to avoid disabling FineIBT at config time:
> > > https://lore.kernel.org/all/202504161442.66CE2596@keescook/
> > 
> > I think you mean enabling FineIBT - yes, this patch does exactly that,
> > it's still possible to enable it by hand with cfi=
> > 
> > > If another approach is needed, then we can change it on top if that is
> > > OK (I am about to submit the PR).
> > 
> > Putting it on CONFIG_FINEIBT was my initial idea, but it turns out
> > that this creates a dependency cycle in Kconfig:
> > 
> > error: recursive dependency detected!
> >         symbol RUST depends on CALL_PADDING
> >         symbol CALL_PADDING is selected by FINEIBT
> >         symbol FINEIBT depends on RUST
> > 
> > Not sure if it's possible to make it work.
> 
> To fix that, we probably have to do one of:
> 
> Option 1: Upgrade the rustc MSRV past 1.81 and drop the depends on
>           CALL_PADDING.
> 
> Option 2: Split `config FINEIBT` into two options:
> 
> config HAVE_FINEIBT
> 	def_bool y
> 	depends on X86_KERNEL_IBT && CFI_CLANG && MITIGATION_RETPOLINE
> 	depends on !RUST || RUSTC_VERSION >= 108800
> 
> config FINEIBT
> 	bool "Accept cfi=fineibt option at boot"
> 	depends on HAVE_FINEIBT
> 	select CALL_PADDING

Right -- if you want to make FINEIBT unselectable, this will need some
plumbing. Right now, it's kind of tied in with the hash shuffling, so
it'll need more than just a Kconfig change to properly split it.

-- 
Kees Cook

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Fri, May 9, 2025 at 11:12 AM Paweł Anikiel <panikiel@google.com> wrote:
>
> On Fri, May 9, 2025 at 11:04 AM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> >
> > I thought the intention was to avoid disabling FineIBT at config time:
> > https://lore.kernel.org/all/202504161442.66CE2596@keescook/
>
> I think you mean enabling FineIBT - yes, this patch does exactly that,
> it's still possible to enable it by hand with cfi=

No, I meant avoiding to disable it at config time due to the rehash
discussion between Peter & Kees in the Lore link I mentioned (I see
Kees has replied again -- thanks!)

Cheers,
Miguel

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Paweł Anikiel]

On Thu, Apr 10, 2025 at 2:45 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Apr 10, 2025 at 02:36:02PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 10, 2025 at 11:54:20AM +0000, Paweł Anikiel wrote:
> > > Calling core::fmt::write() from rust code while FineIBT is enabled
> > > results in a kernel panic:
> > >
> > > [ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
> > > [ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> > > [ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
> > > [ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
> > > [ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
> > > [ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
> > > [ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
> > > [ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
> > > [ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
> > > [ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
> > > [ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
> > > [ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
> > > [ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
> > > [ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
> > > [ 4614.336777] PKRU: 55555554
> > > [ 4614.339786] Call Trace:
> > > [ 4614.342524]  <TASK>
> > > [ 4614.344867]  ? __die_body+0x69/0xb0
> > > [ 4614.348786]  ? die+0xa9/0xd0
> > > [ 4614.352000]  ? do_trap+0x89/0x160
> > > [ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
> > > [ 4614.360413]  ? handle_invalid_op+0x69/0x90
> > > [ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
> > > [ 4614.369654]  ? exc_invalid_op+0x36/0x60
> > > [ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
> > > [ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
> > > [ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
> > > [ 4614.388005]  exc_control_protection+0x49/0x70
> > > [ 4614.392871]  asm_exc_control_protection+0x2b/0x60
> > > [ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
> > > [ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
> > > [ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
> > > [ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
> > > [ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
> > > [ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
> > > [ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
> > > [ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
> > > [ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
> > > [ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250
> > >
> > > This happens because core::fmt::write() calls
> > > core::fmt::rt::Argument::fmt(), which currently has CFI disabled:
> > >
> > > library/core/src/fmt/rt.rs:
> > > 171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
> > > 172     // it here is an explicit CFI violation.
> > > 173     #[allow(inline_no_sanitize)]
> > > 174     #[no_sanitize(cfi, kcfi)]
> > > 175     #[inline]
> > > 176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
> > >
> > > This causes a Control Protection exception, because FineIBT has sealed
> > > off the original function's endbr64.
> > >
> > > This makes rust currently incompatible with FineIBT. Add a Kconfig
> > > dependency that prevents FineIBT from getting turned on by default
> > > if rust is enabled.
> >
> > Why ?!, what's wrong with removing that no_sanitize() instead?

I'll let the rust maintainers answer this one.

> >
> > How is it still compatible with kCFI and not with FineIBT?

kCFI keeps the original function's endbr64. If a callsite disables
CFI, it simply jumps to that function (without any magic value
checks), the endbr64 is there, so no #CP.

With FineIBT, the original function's endbr64 gets replaced with a
nop, and the callsite is supposed to be modified to jump to the CFI
preamble. With CFI disabled, that modification doesn't happen
(cfi_rewrite_callers() quietly ignores that callsite), so we jump to
the nop, and we get a #CP.

>
> FWIW, CFI violations of any kind are a no-no. They're not accepted in C
> and they're not accepted in Rust. This is clearly a Rust bug that needs
> to be fixed ASAP. Disabling CFI is not an option.

This is a known issue, but it doesn't seem to have high priority:
https://github.com/rust-lang/rust/issues/115199

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:12:54PM +0200, Paweł Anikiel wrote:

> I'll let the rust maintainers answer this one.
> 
> > >
> > > How is it still compatible with kCFI and not with FineIBT?
> 
> kCFI keeps the original function's endbr64. If a callsite disables
> CFI, it simply jumps to that function (without any magic value
> checks), the endbr64 is there, so no #CP.
> 
> With FineIBT, the original function's endbr64 gets replaced with a
> nop, and the callsite is supposed to be modified to jump to the CFI
> preamble. With CFI disabled, that modification doesn't happen
> (cfi_rewrite_callers() quietly ignores that callsite), so we jump to
> the nop, and we get a #CP.

Oh, you're attempting to do a no-cfi indirect branch? Yeah, you don't
get to do that.

I should get objtool to warn about those. They undermine the point of
CFI.

> > FWIW, CFI violations of any kind are a no-no. They're not accepted in C
> > and they're not accepted in Rust. This is clearly a Rust bug that needs
> > to be fixed ASAP. Disabling CFI is not an option.
> 
> This is a known issue, but it doesn't seem to have high priority:
> https://github.com/rust-lang/rust/issues/115199

Now you get to fix it :) Because such code is unacceptable for the
kernel.

Just because some crazy Rust person sticks this in their runtime,
doesn't mean you get to use it in the kernel.

[PATCH] objtool: Detect __nocfi calls

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 03:25:22PM +0200, Peter Zijlstra wrote:

> I should get objtool to warn about those. They undermine the point of
> CFI.

---
Subject: objtool: Detect __nocfi calls

Detect and WARN about no_sanitize(kcfi) indirect calls.

Apparently there were a few in some Rust 'core' that got included in the
kernel and things went *bang*.

This is not a supported form for kernel code. So detect and warn about
it.

Adds an annotation for the two cases where we have to live with them:

 - EFI stubs;
 - kexec handover.

Notably, EFI calls fully disable IBT, as such using runtime EFI services
is a significant security issue. If you can exploit the kexec handover,
you get to keep it.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index cc73f9708464..67ba0db92272 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -442,6 +442,7 @@ void __nocfi machine_kexec(struct kimage *image)
 
 	__ftrace_enabled_restore(save_ftrace_enabled);
 }
+ANNOTATE_NOCFI_SYM(machine_kexec);
 
 /* arch-dependent functionality related to kexec file-based syscall */
 
diff --git a/arch/x86/platform/efi/efi_stub_64.S b/arch/x86/platform/efi/efi_stub_64.S
index 2206b8bc47b8..2bee139ae1ee 100644
--- a/arch/x86/platform/efi/efi_stub_64.S
+++ b/arch/x86/platform/efi/efi_stub_64.S
@@ -11,6 +11,7 @@
 #include <asm/nospec-branch.h>
 
 SYM_FUNC_START(__efi_call)
+	ANNOTATE_NOCFI
 	pushq %rbp
 	movq %rsp, %rbp
 	and $~0xf, %rsp
diff --git a/include/linux/objtool.h b/include/linux/objtool.h
index 366ad004d794..518daea19699 100644
--- a/include/linux/objtool.h
+++ b/include/linux/objtool.h
@@ -185,6 +185,8 @@
  */
 #define ANNOTATE_REACHABLE(label)	__ASM_ANNOTATE(label, ANNOTYPE_REACHABLE)
 
+#define ANNOTATE_NOCFI_SYM(sym)		asm(__ASM_ANNOTATE(sym, ANNOTYPE_NOCFI))
+
 #else
 #define ANNOTATE_NOENDBR		ANNOTATE type=ANNOTYPE_NOENDBR
 #define ANNOTATE_RETPOLINE_SAFE		ANNOTATE type=ANNOTYPE_RETPOLINE_SAFE
@@ -194,6 +196,7 @@
 #define ANNOTATE_INTRA_FUNCTION_CALL	ANNOTATE type=ANNOTYPE_INTRA_FUNCTION_CALL
 #define ANNOTATE_UNRET_BEGIN		ANNOTATE type=ANNOTYPE_UNRET_BEGIN
 #define ANNOTATE_REACHABLE		ANNOTATE type=ANNOTYPE_REACHABLE
+#define ANNOTATE_NOCFI			ANNOTATE type=ANNOTYPE_NOCFI
 #endif
 
 #if defined(CONFIG_NOINSTR_VALIDATION) && \
diff --git a/include/linux/objtool_types.h b/include/linux/objtool_types.h
index df5d9fa84dba..aceac94632c8 100644
--- a/include/linux/objtool_types.h
+++ b/include/linux/objtool_types.h
@@ -65,5 +65,6 @@ struct unwind_hint {
 #define ANNOTYPE_IGNORE_ALTS		6
 #define ANNOTYPE_INTRA_FUNCTION_CALL	7
 #define ANNOTYPE_REACHABLE		8
+#define ANNOTYPE_NOCFI			9
 
 #endif /* _LINUX_OBJTOOL_TYPES_H */
diff --git a/tools/include/linux/objtool_types.h b/tools/include/linux/objtool_types.h
index df5d9fa84dba..aceac94632c8 100644
--- a/tools/include/linux/objtool_types.h
+++ b/tools/include/linux/objtool_types.h
@@ -65,5 +65,6 @@ struct unwind_hint {
 #define ANNOTYPE_IGNORE_ALTS		6
 #define ANNOTYPE_INTRA_FUNCTION_CALL	7
 #define ANNOTYPE_REACHABLE		8
+#define ANNOTYPE_NOCFI			9
 
 #endif /* _LINUX_OBJTOOL_TYPES_H */
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 4a1f6c3169b3..868601760953 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -1382,6 +1382,26 @@ static int add_call_dest(struct objtool_file *file, struct instruction *insn,
 
 static int add_retpoline_call(struct objtool_file *file, struct instruction *insn)
 {
+	struct symbol *sym = insn->sym;
+
+	/*
+	 * kCFI call sites look like:
+	 *
+	 *     movl $(-0x12345678), %r10d
+	 *     addl -4(%r11), %r10d
+	 *     jz 1f
+	 *     ud2
+	 *  1: cs call __x86_indirect_thunk_r11
+	 *
+	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
+	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
+	 */
+	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
+		struct instruction *prev = prev_insn_same_sym(file, insn);
+		if (!prev || prev->type != INSN_BUG)
+			WARN_INSN(insn, "no-cfi indirect call!");
+	}
+
 	/*
 	 * Retpoline calls/jumps are really dynamic calls/jumps in disguise,
 	 * so convert them accordingly.
@@ -2334,6 +2354,8 @@ static int read_annotate(struct objtool_file *file,
 
 static int __annotate_early(struct objtool_file *file, int type, struct instruction *insn)
 {
+	struct symbol *sym;
+
 	switch (type) {
 
 	/* Must be before add_special_section_alts() */
@@ -2348,6 +2370,19 @@ static int __annotate_early(struct objtool_file *file, int type, struct instruct
 		insn->noendbr = 1;
 		break;
 
+	/*
+	 * Must be before add_{jump,call}_destination(), specifically any
+	 * add_retpoline_call().
+	 */
+	case ANNOTYPE_NOCFI:
+		sym = insn->sym;
+		if (!sym) {
+			WARN_INSN(insn, "dodgy NOCFI annotation");
+			break;
+		}
+		insn->sym->nocfi = 1;
+		break;
+
 	default:
 		break;
 	}
@@ -2428,6 +2463,10 @@ static int __annotate_late(struct objtool_file *file, int type, struct instructi
 		insn->dead_end = false;
 		break;
 
+	case ANNOTYPE_NOCFI:
+		/* early */
+		break;
+
 	default:
 		ERROR_INSN(insn, "Unknown annotation type: %d", type);
 		return -1;
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index c7c4e87ebe88..f60604d30793 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -70,6 +70,7 @@ struct symbol {
 	u8 local_label       : 1;
 	u8 frame_pointer     : 1;
 	u8 ignore	     : 1;
+	u8 nocfi             : 1;
 	struct list_head pv_target;
 	struct reloc *relocs;
 };

Re: [PATCH] objtool: Detect __nocfi calls

[Josh Poimboeuf]

On Thu, Apr 10, 2025 at 05:45:56PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 10, 2025 at 03:25:22PM +0200, Peter Zijlstra wrote:
> 
> > I should get objtool to warn about those. They undermine the point of
> > CFI.
> 
> ---
> Subject: objtool: Detect __nocfi calls

"Warn on indirect calls in __nocfi functions" ?

>  static int add_retpoline_call(struct objtool_file *file, struct instruction *insn)
>  {
> +	struct symbol *sym = insn->sym;
> +
> +	/*
> +	 * kCFI call sites look like:
> +	 *
> +	 *     movl $(-0x12345678), %r10d
> +	 *     addl -4(%r11), %r10d
> +	 *     jz 1f
> +	 *     ud2
> +	 *  1: cs call __x86_indirect_thunk_r11
> +	 *
> +	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
> +	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
> +	 */
> +	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
> +		struct instruction *prev = prev_insn_same_sym(file, insn);
> +		if (!prev || prev->type != INSN_BUG)
> +			WARN_INSN(insn, "no-cfi indirect call!");

Since this can break things pretty badly at runtime, this should
actually fail the build on CONFIG_OBJTOOL_WERROR.

The warning counts aren't plumbed in this early, so can this check be
done later?  validate_retpoline() or validate_call()?

-- 
Josh

Re: [PATCH] objtool: Detect __nocfi calls

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 12:09:02PM -0700, Josh Poimboeuf wrote:
> On Thu, Apr 10, 2025 at 05:45:56PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 10, 2025 at 03:25:22PM +0200, Peter Zijlstra wrote:
> > 
> > > I should get objtool to warn about those. They undermine the point of
> > > CFI.
> > 
> > ---
> > Subject: objtool: Detect __nocfi calls
> 
> "Warn on indirect calls in __nocfi functions" ?

Yeah, I suppose that's more accurate.

> >  static int add_retpoline_call(struct objtool_file *file, struct instruction *insn)
> >  {
> > +	struct symbol *sym = insn->sym;
> > +
> > +	/*
> > +	 * kCFI call sites look like:
> > +	 *
> > +	 *     movl $(-0x12345678), %r10d
> > +	 *     addl -4(%r11), %r10d
> > +	 *     jz 1f
> > +	 *     ud2
> > +	 *  1: cs call __x86_indirect_thunk_r11
> > +	 *
> > +	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
> > +	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
> > +	 */
> > +	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
> > +		struct instruction *prev = prev_insn_same_sym(file, insn);
> > +		if (!prev || prev->type != INSN_BUG)
> > +			WARN_INSN(insn, "no-cfi indirect call!");
> 
> Since this can break things pretty badly at runtime, this should
> actually fail the build on CONFIG_OBJTOOL_WERROR.

Oh right, I still got to adjust to the new world order here :-)

> The warning counts aren't plumbed in this early, so can this check be
> done later?  validate_retpoline() or validate_call()?

Hmm, let me have a poke around, see what can be done.

Re: [PATCH] objtool: Detect __nocfi calls

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 5:46 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> Detect and WARN about no_sanitize(kcfi) indirect calls.
>
> Apparently there were a few in some Rust 'core' that got included in the
> kernel and things went *bang*.
>
> This is not a supported form for kernel code. So detect and warn about
> it.

That was quick!

It seems to work -- I got:

      LD      vmlinux.o
    vmlinux.o: warning: objtool:
_RNvNtCs8DPF7ip8WBQ_4core3fmt5write+0x170: no-cfi indirect call!

Cheers,
Miguel

Re: [PATCH] objtool: Detect __nocfi calls

[Sami Tolvanen]

Hi Peter,

On Thu, Apr 10, 2025 at 05:45:56PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 10, 2025 at 03:25:22PM +0200, Peter Zijlstra wrote:
> 
> > I should get objtool to warn about those. They undermine the point of
> > CFI.
> 
> ---
> Subject: objtool: Detect __nocfi calls
> 
> Detect and WARN about no_sanitize(kcfi) indirect calls.
> 
> Apparently there were a few in some Rust 'core' that got included in the
> kernel and things went *bang*.
> 
> This is not a supported form for kernel code. So detect and warn about
> it.

Cool, this looks useful!

> Adds an annotation for the two cases where we have to live with them:
> 
>  - EFI stubs;
>  - kexec handover.
> 
> Notably, EFI calls fully disable IBT, as such using runtime EFI services
> is a significant security issue. If you can exploit the kexec handover,
> you get to keep it.

OK, with this applied I now see a warning about the __nocfi call in
Rust code:

vmlinux.o: warning: objtool: _RNvNtCsjWi3sh0wSlE_4core3fmt5write+0x170: no-cfi indirect call!

But an allmodconfig build reveals a few more warnings:

arch/x86/kvm/kvm.o: warning: objtool: x86_emulate_insn+0xaf7: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_das+0x290: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_imul_3op+0x15f: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_aam+0x21c: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_aad+0x1dc: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_loop+0x312: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_cmpxchg+0x329: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_bsf_c+0x1b7: no-cfi indirect call!
arch/x86/kvm/kvm.o: warning: objtool: em_bsr_c+0x1b7: no-cfi indirect call!
arch/x86/kvm/kvm-intel.o: warning: objtool: vmx_do_interrupt_irqoff+0xe: no-cfi indirect call!
drivers/misc/lkdtm/lkdtm.o: warning: objtool: execute_location+0x5a: no-cfi indirect call!
drivers/pci/controller/pci-hyperv.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call!
drivers/hv/hv_balloon.o: warning: objtool: hv_free_page_report+0x5da: no-cfi indirect call!
drivers/hv/hv_vmbus.o: warning: objtool: hv_post_message+0x457: no-cfi indirect call!
drivers/hv/hv_vmbus.o: warning: objtool: vmbus_set_event+0x2a2: no-cfi indirect call!
vmlinux.o: warning: objtool: hyperv_flush_tlb_multi+0xe96: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_do_hypercall+0x12b: no-cfi indirect call!
vmlinux.o: warning: objtool: hyperv_flush_guest_mapping+0x2e3: no-cfi indirect call!
vmlinux.o: warning: objtool: hyperv_flush_guest_mapping_range+0x36a: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_snp_boot_ap+0xb08: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_vtom_set_host_visibility+0x54a: no-cfi indirect call!
vmlinux.o: warning: objtool: __send_ipi_one+0x362: no-cfi indirect call!
vmlinux.o: warning: objtool: __send_ipi_mask_ex+0x655: no-cfi indirect call!
vmlinux.o: warning: objtool: __send_ipi_mask+0x635: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_query_ext_cap+0x175: no-cfi indirect call!
vmlinux.o: warning: objtool: get_vtl+0x38c: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_get_partition_id+0x224: no-cfi indirect call!

Sami

Re: [PATCH] objtool: Detect __nocfi calls

[Peter Zijlstra]

On Thu, Apr 10, 2025 at 07:43:34PM +0000, Sami Tolvanen wrote:

> But an allmodconfig build reveals a few more warnings:

Sigh, let me go look at all that and address Josh's feedback.

Thanks!

Re: [PATCH] objtool: Detect __nocfi calls

[Peter Zijlstra]

On Fri, Apr 11, 2025 at 08:44:49AM +0200, Peter Zijlstra wrote:
> On Thu, Apr 10, 2025 at 07:43:34PM +0000, Sami Tolvanen wrote:
> 
> > But an allmodconfig build reveals a few more warnings:
> 
> Sigh, let me go look at all that and address Josh's feedback.

Latest sits here:

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git x86/core

I'll have the robots chew on it over the weekend and then post on
Monday.

Re: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

[Miguel Ojeda]

On Thu, Apr 10, 2025 at 3:13 PM Paweł Anikiel <panikiel@google.com> wrote:
>
> I'll let the rust maintainers answer this one.
>
> This is a known issue, but it doesn't seem to have high priority:
> https://github.com/rust-lang/rust/issues/115199

Cc'ing Ramon who opened the issue -- do you know if there has been any
recent movement on it?

Cheers,
Miguel