Xen PV dom0 "tried to execute NX-protected page" when running nested in KVM - 6.15 regression

["Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>]

[-- Attachment #1: Type: text/plain, Size: 5975 bytes --]

Hi,

With Linux 6.15.2 I got a crash like below. It worked fine with Linux
6.14.11. Furthermore, the failure seems to be hardware-dependent. It
happens when running on Intel Core i9-13900H, but does not happen when
running on Intel Xeon E5-2620v4 (in both cases QEMU uses -cpu host).

The crash:
[    1.121608] ITS: Mitigation: Aligned branch/return thunks
[    1.122604] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    1.123656] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    1.124603] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    1.125603] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    1.126599] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
[    1.128391] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[    1.128391] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[    1.128391] BUG: unable to handle page fault for address: ffffffffc0000760
[    1.128391] #PF: supervisor instruction fetch in kernel mode
[    1.128391] #PF: error_code(0x0011) - permissions violation
[    1.128391] PGD 2433067 P4D 2433067 PUD 2435067 PMD 1002b7067 PTE 80100001002b6067
[    1.128391] Oops: Oops: 0011 [#1] SMP NOPTI
[    1.128391] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.15.2-1.qubes.fc41.x86_64 #1 PREEMPT(full) 
[    1.128391] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-2-gc13ff2cd-prebuilt.qemu.org 04/01/2014
[    1.128391] RIP: e030:0xffffffffc0000760
[    1.128391] Code: e0 cc ff e0 cc ff e0 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc <ff> e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff
[    1.128391] RSP: e02b:ffffc90040003b60 EFLAGS: 00010002
[    1.128391] RAX: ffffffff80f02190 RBX: ffffffff834da1b5 RCX: 00000000ffffffff
[    1.128391] RDX: ffffc90040003b78 RSI: 0000000000000001 RDI: ffff88810029f400
[    1.128391] RBP: ffffc90040003b78 R08: 0000000000000000 R09: 205d313933383231
[    1.128391] R10: 0000000000000029 R11: 000000006e72656b R12: 000000000000000a
[    1.128391] R13: ffffffff834da1b5 R14: 0000000000000000 R15: ffff88810029f400
[    1.128391] FS:  0000000000000000(0000) GS:ffff8881fc9c8000(0000) knlGS:0000000000000000
[    1.128391] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.128391] CR2: ffffffffc0000760 CR3: 000000000242e000 CR4: 0000000000050660
[    1.128391] Call Trace:
[    1.128391]  <IRQ>
[    1.128391]  ? vt_console_print+0x2e6/0x500
[    1.128391]  ? console_emit_next_record+0x110/0x1b0
[    1.128391]  ? console_flush_all+0x1d5/0x2a0
[    1.128391]  ? console_unlock+0x7c/0x140
[    1.128391]  ? vprintk_emit+0x278/0x2d0
[    1.128391]  ? _printk+0x6b/0x90
[    1.128391]  ? show_fault_oops+0x17c/0x1b0
[    1.128391]  ? page_fault_oops+0x11b/0x160
[    1.128391]  ? exc_page_fault+0x189/0x1a0
[    1.128391]  ? asm_exc_page_fault+0x26/0x30
[    1.128391]  ? __pfx_evtchn_fifo_clear_pending+0x10/0x10
[    1.128391]  ? handle_percpu_irq+0x30/0x60
[    1.128391]  ? generic_handle_irq+0x3c/0x60
[    1.128391]  ? __evtchn_fifo_handle_events+0x1df/0x2c0
[    1.128391]  ? xen_evtchn_do_upcall+0x6d/0xc0
[    1.128391]  ? __xen_pv_evtchn_do_upcall+0x26/0x40
[    1.128391]  ? xen_pv_evtchn_do_upcall+0x84/0xa0
[    1.128391]  </IRQ>
[    1.128391]  <TASK>
[    1.128391]  ? exc_xen_hypervisor_callback+0x8/0x20
[    1.128391]  ? print_bpf_insn+0x322/0xb70
[    1.128391]  ? xen_save_fl_direct+0xf/0x20
[    1.128391]  ? text_poke_early+0x35/0xa0
[    1.128391]  ? print_bpf_insn+0x322/0xb70
[    1.128391]  ? apply_retpolines+0x1ad/0x1d0
[    1.128391]  ? print_bpf_insn+0x322/0xb70
[    1.128391]  ? print_bpf_insn+0x331/0xb70
[    1.128391]  ? print_bpf_insn+0x328/0xb70
[    1.128391]  ? alternative_instructions+0x56/0x200
[    1.128391]  ? arch_cpu_finalize_init+0x80/0x120
[    1.128391]  ? start_kernel+0x3f5/0x490
[    1.128391]  ? x86_64_start_reservations+0x24/0x30
[    1.128391]  ? xen_start_kernel+0x6d7/0x6f0
[    1.128391]  ? startup_xen+0x1b/0x20
[    1.128391]  </TASK>
[    1.128391] Modules linked in:
[    1.128391] CR2: ffffffffc0000760
[    1.128391] ---[ end trace 0000000000000000 ]---
[    1.128391] RIP: e030:0xffffffffc0000760
[    1.128391] Code: e0 cc ff e0 cc ff e0 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc <ff> e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff e0 cc ff
[    1.128391] RSP: e02b:ffffc90040003b60 EFLAGS: 00010002
[    1.128391] RAX: ffffffff80f02190 RBX: ffffffff834da1b5 RCX: 00000000ffffffff
[    1.128391] RDX: ffffc90040003b78 RSI: 0000000000000001 RDI: ffff88810029f400
[    1.128391] RBP: ffffc90040003b78 R08: 0000000000000000 R09: 205d313933383231
[    1.128391] R10: 0000000000000029 R11: 000000006e72656b R12: 000000000000000a
[    1.128391] R13: ffffffff834da1b5 R14: 0000000000000000 R15: ffff88810029f400
[    1.128391] FS:  0000000000000000(0000) GS:ffff8881fc9c8000(0000) knlGS:0000000000000000
[    1.128391] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.128391] CR2: ffffffffc0000760 CR3: 000000000242e000 CR4: 0000000000050660
[    1.128391] Kernel panic - not syncing: Fatal exception in interrupt

Full console log: https://gist.github.com/marmarek/7a4ad628c7bf76339aed79ff4478f8ea

Full QEMU command (if relevant) can be seen at https://openqa.qubes-os.org/tests/143860/logfile?filename=autoinst-log.txt

At this point, I'm not even sure who to report it to... In an earlier
attempt I've got stack trace full of ext4 functions, which is unlikely
relevant (see revisions of the gist linked above). I'll try to bisect
this, but due to hardware-dependent nature it will take some time as I
have a bit limited access to that hardware.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]