[kvm-unit-tests PATCH v4 00/18] x86: Improve CET tests

[kvm-unit-tests PATCH v4 00/18] x86: Improve CET tests

[Sean Christopherson]

Hopefully the last version of this particular CET series.  Mathias, I owe you
like five beers for root causing and fixing all the gnarly edge cases.

v4:
 - Fixup the argumentes for the vmx_cet_test. [Mathias]
 - Drop "_test" from the vmx_cet config to match the other VMX testcases.
 - Enable NOTRACK instead of dodging jmp tables in exception_mnemonic() [Mathias].
 - Reset IBT state after (intentional) #CP. [Mathias]
 - Fix a changelog typo. [Mathias]
 - Document that ljmpq isn't supported on AMD. [Mathias]
 - Use ljmpl to make the 32-bit JMP FAR more obvious. [Mathias]

v3:
 - https://lore.kernel.org/all/20251114001258.1717007-1-seanjc@google.com
 - Run the test if only one of SHSTK or IBT is supported (e.g. to test
   SHSTK on AMD).
 - Rename the test from "intel_cet" to just "cet".
 - Add an endbr64 in the user_mode trampoline (the test was getting false
   passes without ever reaching cet_shstk_far_ret() due to getting the
   expected #CP).
 - Add testcases to verify KVM rejects emulation as expected.
 - Add a comment explaining the SHSTK PTE magic (I forgot about the magic
   and spent a long time trying to figure out how the user_mode trampoline
   was succeeding if the SHSTK wasn't writable, *sigh*)


Chao Gao (7):
  x86: cet: Remove unnecessary memory zeroing for shadow stack
  x86: cet: Directly check for #CP exception in run_in_user()
  x86: cet: Validate #CP error code
  x86: cet: Use report_skip()
  x86: cet: Drop unnecessary casting
  x86: cet: Validate writing unaligned values to SSP MSR causes #GP
  x86: cet: Validate CET states during VMX transitions

Mathias Krause (5):
  x86: cet: Make shadow stack less fragile
  x86: cet: Simplify IBT test
  x86: cet: Use symbolic values for the #CP error codes
  x86: cet: Test far returns too
  x86: Avoid top-most page for vmalloc on x86-64

Sean Christopherson (5):
  x86: cet: Run SHSTK and IBT tests as appropriate if either feature is
    supported
  x86: cet: Drop the "intel_" prefix from the CET testcase
  x86: cet: Enable NOTRACK handling for IBT tests
  x86: cet: Reset IBT tracker state on #CP violations
  x86: cet: Add testcases to verify KVM rejects emulation of CET
    instructions

Yang Weijiang (1):
  x86: cet: Pass virtual addresses to invlpg

 lib/x86/msr.h      |   1 +
 lib/x86/usermode.c |  16 ++-
 lib/x86/usermode.h |  13 +-
 lib/x86/vm.c       |   2 +
 x86/cet.c          | 308 ++++++++++++++++++++++++++++++++++++---------
 x86/lam.c          |  10 +-
 x86/unittests.cfg  |  10 +-
 x86/vmx.h          |   8 +-
 x86/vmx_tests.c    |  81 ++++++++++++
 9 files changed, 375 insertions(+), 74 deletions(-)


base-commit: c885c94f523eb4518dc30408fb5199fd23d4aa0a
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 01/18] x86: cet: Pass virtual addresses to invlpg

[Sean Christopherson]

From: Yang Weijiang <weijiang.yang@intel.com>

Correct the parameter passed to invlpg.

The invlpg instruction should take a virtual address instead of a physical
address when flushing TLBs. Using shstk_phys results in TLBs associated
with the virtual address (shstk_virt) not being flushed, and the virtual
address may not be treated as a shadow stack address if there is a stale
TLB. So, subsequent shadow stack accesses to shstk_virt may cause a #PF,
which terminates the test unexpectedly.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/x86/cet.c b/x86/cet.c
index 42d2b1fc..51a54a50 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -100,7 +100,7 @@ int main(int ac, char **av)
 	*ptep |= PT_DIRTY_MASK;
 
 	/* Flush the paging cache. */
-	invlpg((void *)shstk_phys);
+	invlpg((void *)shstk_virt);
 
 	/* Enable shadow-stack protection */
 	wrmsr(MSR_IA32_U_CET, ENABLE_SHSTK_BIT);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 02/18] x86: cet: Remove unnecessary memory zeroing for shadow stack

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

Skip mapping the shadow stack as a writable page and the redundant memory
zeroing.

Currently, the shadow stack is allocated using alloc_page(), then mapped as
a writable page, zeroed, and finally mapped as a shadow stack page. The
memory zeroing is redundant as alloc_page() already does that.

This also eliminates the need for invlpg, as the shadow stack is no
longer mapped writable.

Signed-off-by: Chao Gao <chao.gao@intel.com>
[mks: drop invlpg() as it's no longer needed, adapted changelog accordingly]
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
[sean: add a comment to explain the magic shadow stack protections]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index 51a54a50..e2681886 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -67,7 +67,6 @@ int main(int ac, char **av)
 {
 	char *shstk_virt;
 	unsigned long shstk_phys;
-	unsigned long *ptep;
 	pteval_t pte = 0;
 	bool rvc;
 
@@ -89,18 +88,14 @@ int main(int ac, char **av)
 	shstk_virt = alloc_vpage();
 	shstk_phys = (unsigned long)virt_to_phys(alloc_page());
 
-	/* Install the new page. */
-	pte = shstk_phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | PT_USER_MASK;
+	/*
+	 * Install a mapping for the shadow stack page.  Shadow stack pages are
+	 * denoted by an "impossible" combination of a !WRITABLE, DIRTY PTE
+	 * (writes from CPU for shadow stack operations are allowed, but writes
+	 * from software are not).
+	 */
+	pte = shstk_phys | PT_PRESENT_MASK | PT_USER_MASK | PT_DIRTY_MASK;
 	install_pte(current_page_table(), 1, shstk_virt, pte, 0);
-	memset(shstk_virt, 0x0, PAGE_SIZE);
-
-	/* Mark it as shadow-stack page. */
-	ptep = get_pte_level(current_page_table(), shstk_virt, 1);
-	*ptep &= ~PT_WRITABLE_MASK;
-	*ptep |= PT_DIRTY_MASK;
-
-	/* Flush the paging cache. */
-	invlpg((void *)shstk_virt);
 
 	/* Enable shadow-stack protection */
 	wrmsr(MSR_IA32_U_CET, ENABLE_SHSTK_BIT);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 03/18] x86: cet: Directly check for #CP exception in run_in_user()

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

Current CET tests validate if a #CP exception is raised by registering
a #CP handler. This handler counts the #CP exceptions and raises a #GP
exception, which is then caught by the run_in_user() infrastructure to
switch back to the kernel. This is convoluted.

Catch the #CP exception directly by run_in_user() to avoid the manual
counting of #CP exceptions and the #CP->#GP dance.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index e2681886..7635fe34 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -8,9 +8,6 @@
 #include "alloc_page.h"
 #include "fault_test.h"
 
-static int cp_count;
-static unsigned long invalid_offset = 0xffffffffffffff;
-
 static u64 cet_shstk_func(void)
 {
 	unsigned long *ret_addr, *ssp;
@@ -54,15 +51,6 @@ static u64 cet_ibt_func(void)
 #define ENABLE_SHSTK_BIT 0x1
 #define ENABLE_IBT_BIT   0x4
 
-static void handle_cp(struct ex_regs *regs)
-{
-	cp_count++;
-	printf("In #CP exception handler, error_code = 0x%lx\n",
-		regs->error_code);
-	/* Below jmp is expected to trigger #GP */
-	asm("jmpq *%0": :"m"(invalid_offset));
-}
-
 int main(int ac, char **av)
 {
 	char *shstk_virt;
@@ -70,7 +58,6 @@ int main(int ac, char **av)
 	pteval_t pte = 0;
 	bool rvc;
 
-	cp_count = 0;
 	if (!this_cpu_has(X86_FEATURE_SHSTK)) {
 		printf("SHSTK not enabled\n");
 		return report_summary();
@@ -82,7 +69,6 @@ int main(int ac, char **av)
 	}
 
 	setup_vm();
-	handle_exception(CP_VECTOR, handle_cp);
 
 	/* Allocate one page for shadow-stack. */
 	shstk_virt = alloc_vpage();
@@ -107,15 +93,14 @@ int main(int ac, char **av)
 	write_cr4(read_cr4() | X86_CR4_CET);
 
 	printf("Unit test for CET user mode...\n");
-	run_in_user((usermode_func)cet_shstk_func, GP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(cp_count == 1, "Completed shadow-stack protection test successfully.");
-	cp_count = 0;
+	run_in_user((usermode_func)cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
+	report(rvc, "Shadow-stack protection test.");
 
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
-	run_in_user((usermode_func)cet_ibt_func, GP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(cp_count == 1, "Completed Indirect-branch tracking test successfully.");
+	run_in_user((usermode_func)cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
+	report(rvc, "Indirect-branch tracking test.");
 
 	write_cr4(read_cr4() & ~X86_CR4_CET);
 	wrmsr(MSR_IA32_U_CET, 0);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 04/18] x86: cet: Validate #CP error code

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

The #CP exceptions include an error code that provides additional
information about how the exception occurred. Previously, CET tests simply
printed these error codes without validation.

Enhance the CET tests to validate the #CP error code.

This requires the run_in_user() infrastructure to catch the exception
vector, error code, and rflags, similar to what check_exception_table()
does.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 lib/x86/usermode.c | 4 ++++
 x86/cet.c          | 4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/x86/usermode.c b/lib/x86/usermode.c
index c3ec0ad7..f896e3bd 100644
--- a/lib/x86/usermode.c
+++ b/lib/x86/usermode.c
@@ -23,6 +23,10 @@ static void restore_exec_to_jmpbuf(void)
 
 static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)
 {
+	this_cpu_write_exception_vector(regs->vector);
+	this_cpu_write_exception_rflags_rf((regs->rflags >> 16) & 1);
+	this_cpu_write_exception_error_code(regs->error_code);
+
 	/* longjmp must happen after iret, so do not do it now.  */
 	regs->rip = (unsigned long)&restore_exec_to_jmpbuf;
 	regs->cs = KERNEL_CS;
diff --git a/x86/cet.c b/x86/cet.c
index 7635fe34..0452851d 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -94,13 +94,13 @@ int main(int ac, char **av)
 
 	printf("Unit test for CET user mode...\n");
 	run_in_user((usermode_func)cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(rvc, "Shadow-stack protection test.");
+	report(rvc && exception_error_code() == 1, "Shadow-stack protection test.");
 
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
 	run_in_user((usermode_func)cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(rvc, "Indirect-branch tracking test.");
+	report(rvc && exception_error_code() == 3, "Indirect-branch tracking test.");
 
 	write_cr4(read_cr4() & ~X86_CR4_CET);
 	wrmsr(MSR_IA32_U_CET, 0);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 05/18] x86: cet: Use report_skip()

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

report_skip() function is preferred for skipping inapplicable tests when
the necessary hardware features are unavailable. For example, with this
patch applied, the test output is as follows if IBT is not supported:

SKIP: IBT not enabled
SUMMARY: 1 tests, 1 skipped

Previously, it printed:

IBT not enabled
SUMMARY: 0 tests

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index 0452851d..d6ca5dd8 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -59,12 +59,12 @@ int main(int ac, char **av)
 	bool rvc;
 
 	if (!this_cpu_has(X86_FEATURE_SHSTK)) {
-		printf("SHSTK not enabled\n");
+		report_skip("SHSTK not enabled");
 		return report_summary();
 	}
 
 	if (!this_cpu_has(X86_FEATURE_IBT)) {
-		printf("IBT not enabled\n");
+		report_skip("IBT not enabled");
 		return report_summary();
 	}
 
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 06/18] x86: cet: Drop unnecessary casting

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

cet_shstk_func() and cet_ibt_func() have the same type as usermode_func.
So, remove the unnecessary casting.

Signed-off-by: Chao Gao <chao.gao@intel.com>
[mks: make the types really equal by using uint64_t]
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index d6ca5dd8..8c2cf8c6 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -8,7 +8,7 @@
 #include "alloc_page.h"
 #include "fault_test.h"
 
-static u64 cet_shstk_func(void)
+static uint64_t cet_shstk_func(void)
 {
 	unsigned long *ret_addr, *ssp;
 
@@ -31,7 +31,7 @@ static u64 cet_shstk_func(void)
 	return 0;
 }
 
-static u64 cet_ibt_func(void)
+static uint64_t cet_ibt_func(void)
 {
 	/*
 	 * In below assembly code, the first instruction at label 2 is not
@@ -93,13 +93,13 @@ int main(int ac, char **av)
 	write_cr4(read_cr4() | X86_CR4_CET);
 
 	printf("Unit test for CET user mode...\n");
-	run_in_user((usermode_func)cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
+	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == 1, "Shadow-stack protection test.");
 
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
-	run_in_user((usermode_func)cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
+	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == 3, "Indirect-branch tracking test.");
 
 	write_cr4(read_cr4() & ~X86_CR4_CET);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 07/18] x86: cet: Validate writing unaligned values to SSP MSR causes #GP

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

Validate that writing invalid values to SSP MSRs triggers a #GP exception.
This verifies that necessary validity checks are performed by the hardware
or the underlying VMM.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/x86/cet.c b/x86/cet.c
index 8c2cf8c6..80864fb1 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -56,6 +56,7 @@ int main(int ac, char **av)
 	char *shstk_virt;
 	unsigned long shstk_phys;
 	pteval_t pte = 0;
+	u8 vector;
 	bool rvc;
 
 	if (!this_cpu_has(X86_FEATURE_SHSTK)) {
@@ -105,5 +106,9 @@ int main(int ac, char **av)
 	write_cr4(read_cr4() & ~X86_CR4_CET);
 	wrmsr(MSR_IA32_U_CET, 0);
 
+	/* SSP should be 4-Byte aligned */
+	vector = wrmsr_safe(MSR_IA32_PL3_SSP, 0x1);
+	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
+
 	return report_summary();
 }
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 08/18] x86: cet: Validate CET states during VMX transitions

[Sean Christopherson]

From: Chao Gao <chao.gao@intel.com>

Add tests to verify that CET states are correctly handled during VMX
transitions.

The following behaviors are verified:

1. Host states are loaded from VMCS iff "Load CET" VM-exit control is set
2. Guest states are loaded from VMCS iff "Load CET" VM-entry control is set
3. Guest states are saved to VMCS during VM exits unconditionally
4. Invalid guest or host CET states leads to VM entry failures.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
[sean: drop "_test" from config name]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 lib/x86/msr.h     |  1 +
 x86/unittests.cfg |  8 +++++
 x86/vmx.h         |  8 +++--
 x86/vmx_tests.c   | 81 +++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 96 insertions(+), 2 deletions(-)

diff --git a/lib/x86/msr.h b/lib/x86/msr.h
index e586a8e9..7397809c 100644
--- a/lib/x86/msr.h
+++ b/lib/x86/msr.h
@@ -300,6 +300,7 @@
 #define MSR_IA32_FEATURE_CONTROL        0x0000003a
 #define MSR_IA32_TSC_ADJUST		0x0000003b
 #define MSR_IA32_U_CET                  0x000006a0
+#define MSR_IA32_S_CET                  0x000006a2
 #define MSR_IA32_PL3_SSP                0x000006a7
 #define MSR_IA32_PKRS			0x000006e1
 
diff --git a/x86/unittests.cfg b/x86/unittests.cfg
index acb8a8ba..ff537d3f 100644
--- a/x86/unittests.cfg
+++ b/x86/unittests.cfg
@@ -453,6 +453,14 @@ arch = x86_64
 groups = vmx nested_exception
 check = /sys/module/kvm_intel/parameters/allow_smaller_maxphyaddr=Y
 
+[vmx_cet]
+file = vmx.flat
+test_args = "vmx_cet_test"
+qemu_params = -cpu max,+vmx
+arch = x86_64
+groups = vmx
+timeout = 240
+
 [debug]
 file = debug.flat
 arch = x86_64
diff --git a/x86/vmx.h b/x86/vmx.h
index 9cd90488..33373bd1 100644
--- a/x86/vmx.h
+++ b/x86/vmx.h
@@ -356,6 +356,7 @@ enum Encoding {
 	GUEST_PENDING_DEBUG	= 0x6822ul,
 	GUEST_SYSENTER_ESP	= 0x6824ul,
 	GUEST_SYSENTER_EIP	= 0x6826ul,
+	GUEST_S_CET		= 0x6828ul,
 
 	/* Natural-Width Host State Fields */
 	HOST_CR0		= 0x6c00ul,
@@ -369,7 +370,8 @@ enum Encoding {
 	HOST_SYSENTER_ESP	= 0x6c10ul,
 	HOST_SYSENTER_EIP	= 0x6c12ul,
 	HOST_RSP		= 0x6c14ul,
-	HOST_RIP		= 0x6c16ul
+	HOST_RIP		= 0x6c16ul,
+	HOST_S_CET		= 0x6c18ul,
 };
 
 #define VMX_ENTRY_FAILURE	(1ul << 31)
@@ -449,6 +451,7 @@ enum Ctrl_exi {
 	EXI_SAVE_EFER		= 1UL << 20,
 	EXI_LOAD_EFER		= 1UL << 21,
 	EXI_SAVE_PREEMPT	= 1UL << 22,
+	EXI_LOAD_CET		= 1UL << 28,
 };
 
 enum Ctrl_ent {
@@ -457,7 +460,8 @@ enum Ctrl_ent {
 	ENT_LOAD_PERF		= 1UL << 13,
 	ENT_LOAD_PAT		= 1UL << 14,
 	ENT_LOAD_EFER		= 1UL << 15,
-	ENT_LOAD_BNDCFGS	= 1UL << 16
+	ENT_LOAD_BNDCFGS	= 1UL << 16,
+	ENT_LOAD_CET		= 1UL << 20,
 };
 
 enum Ctrl_pin {
diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index 5c8c9f31..5ffb80a3 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -11382,6 +11382,85 @@ static void vmx_posted_interrupts_test(void)
 	enter_guest();
 }
 
+static u64 guest_s_cet = -1;
+
+static void vmx_cet_test_guest(void)
+{
+	guest_s_cet = rdmsr(MSR_IA32_S_CET);
+	vmcall();
+}
+
+static void vmx_cet_test(void)
+{
+	struct vmcs *curr;
+	u64 val;
+
+	if (!(ctrl_exit_rev.clr & EXI_LOAD_CET)) {
+		report_skip("Load CET state exit control is not available");
+		return;
+	}
+
+	if (!(ctrl_enter_rev.clr & ENT_LOAD_CET)) {
+		report_skip("Load CET state entry control is not available");
+		return;
+	}
+
+	/* Allow the guest to read GUEST_S_CET directly */
+	msr_bmp_init();
+
+	/*
+	 * Check whether VMCS transitions load host and guest values
+	 * according to the settings of the relevant VM-entry and exit
+	 * controls.
+	 */
+	vmcs_write(HOST_S_CET, 2);
+	vmcs_write(GUEST_S_CET, 2);
+	test_set_guest(vmx_cet_test_guest);
+
+	enter_guest();
+	val = rdmsr(MSR_IA32_S_CET);
+
+	/* Validate both guest/host S_CET MSR have the default values */
+	report(val == 0 && guest_s_cet == 0, "Load CET state disabled");
+
+	/*
+	 * CPU supports the 1-setting of the 'load CET' VM-entry control,
+	 * the contents of the IA32_S_CET and IA32_INTERRUPT_SSP_TABLE_ADDR
+	 * MSRs are saved into the corresponding fields
+	 */
+	report(vmcs_read(GUEST_S_CET) == 0, "S_CET is unconditionally saved");
+
+	/* Enable load CET state entry/exit controls and retest */
+	vmcs_set_bits(EXI_CONTROLS, EXI_LOAD_CET);
+	vmcs_set_bits(ENT_CONTROLS, ENT_LOAD_CET);
+	vmcs_write(GUEST_S_CET, 2);
+	test_override_guest(vmx_cet_test_guest);
+
+	enter_guest();
+	val = rdmsr(MSR_IA32_S_CET);
+
+	/* Validate both guest/host S_CET MSR are loaded from VMCS */
+	report(val == 2 && guest_s_cet == 2, "Load CET state enabled");
+
+	/*
+	 * Validate that bit 10 (SUPPRESS) and Bit 11 (TRACKER) cannot be
+	 * both set
+	 */
+	val = BIT(10) | BIT(11);
+	vmcs_write(GUEST_S_CET, val);
+	test_guest_state("Load invalid guest CET state", true, val, "GUEST_S_CET");
+
+	/* Following test_vmx_vmlaunch() needs a "not launched" VMCS */
+	vmcs_save(&curr);
+	vmcs_clear(curr);
+	make_vmcs_current(curr);
+
+	vmcs_write(HOST_S_CET, val);
+	test_vmx_vmlaunch(VMXERR_ENTRY_INVALID_HOST_STATE_FIELD);
+
+	test_set_guest_finished();
+}
+
 #define TEST(name) { #name, .v2 = name }
 
 /* name/init/guest_main/exit_handler/syscall_handler/guest_regs */
@@ -11495,5 +11574,7 @@ struct vmx_test vmx_tests[] = {
 	TEST(vmx_pf_vpid_test),
 	TEST(vmx_exception_test),
 	TEST(vmx_canonical_test),
+	/* "Load CET" VM-entry/exit controls tests. */
+	TEST(vmx_cet_test),
 	{ NULL, NULL, NULL, NULL, NULL, {0} },
 };
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 09/18] x86: cet: Make shadow stack less fragile

[Sean Christopherson]

From: Mathias Krause <minipli@grsecurity.net>

The CET shadow stack test has certain assumptions about the code, namely
that it was compiled with frame pointers enabled and the return address
won't be 0xdeaddead.

Make the code less fragile by actually lifting these assumptions to (1)
explicitly mention the dependency to the frame pointer by making us of
__builtin_frame_address(0) and (2) modify the return address by toggling
bits instead of writing a fixed value. Also ensure that write will
actually be generated by the compiler by making it a 'volatile' write.

Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index 80864fb1..61059ef2 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -10,14 +10,14 @@
 
 static uint64_t cet_shstk_func(void)
 {
-	unsigned long *ret_addr, *ssp;
+	unsigned long *ret_addr = __builtin_frame_address(0) + sizeof(void *);
+	unsigned long *ssp;
 
 	/* rdsspq %rax */
 	asm volatile (".byte 0xf3, 0x48, 0x0f, 0x1e, 0xc8" : "=a"(ssp));
 
-	asm("movq %%rbp,%0" : "=r"(ret_addr));
 	printf("The return-address in shadow-stack = 0x%lx, in normal stack = 0x%lx\n",
-	       *ssp, *(ret_addr + 1));
+	       *ssp, *ret_addr);
 
 	/*
 	 * In below line, it modifies the return address, it'll trigger #CP
@@ -26,7 +26,7 @@ static uint64_t cet_shstk_func(void)
 	 * when HW detects the violation.
 	 */
 	printf("Try to temper the return-address, this causes #CP on returning...\n");
-	*(ret_addr + 1) = 0xdeaddead;
+	*(volatile unsigned long *)ret_addr ^= 0xdeaddead;
 
 	return 0;
 }
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 10/18] x86: cet: Simplify IBT test

[Sean Christopherson]

From: Mathias Krause <minipli@grsecurity.net>

The inline assembly of cet_ibt_func() does unnecessary things and
doesn't mention the clobbered registers.

Fix that by reducing the code to what's needed (an indirect jump to a
target lacking the ENDBR instruction) and passing and output register
variable for it.

Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index 61059ef2..a1643c83 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -33,18 +33,17 @@ static uint64_t cet_shstk_func(void)
 
 static uint64_t cet_ibt_func(void)
 {
+	unsigned long tmp;
 	/*
 	 * In below assembly code, the first instruction at label 2 is not
 	 * endbr64, it'll trigger #CP with error code 0x3, and the execution
 	 * is terminated when HW detects the violation.
 	 */
 	printf("No endbr64 instruction at jmp target, this triggers #CP...\n");
-	asm volatile ("movq $2, %rcx\n"
-		      "dec %rcx\n"
-		      "leaq 2f(%rip), %rax\n"
-		      "jmp *%rax \n"
-		      "2:\n"
-		      "dec %rcx\n");
+	asm volatile ("leaq 2f(%%rip), %0\n\t"
+		      "jmpq *%0\n\t"
+		      "2:"
+		      : "=r"(tmp));
 	return 0;
 }
 
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 11/18] x86: cet: Use symbolic values for the #CP error codes

[Sean Christopherson]

From: Mathias Krause <minipli@grsecurity.net>

Use symbolic names for the #CP exception error codes.

Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index a1643c83..f19ceb22 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -47,6 +47,13 @@ static uint64_t cet_ibt_func(void)
 	return 0;
 }
 
+#define CP_ERR_NEAR_RET	0x0001
+#define CP_ERR_FAR_RET	0x0002
+#define CP_ERR_ENDBR	0x0003
+#define CP_ERR_RSTORSSP	0x0004
+#define CP_ERR_SETSSBSY	0x0005
+#define CP_ERR_ENCL		BIT(15)
+
 #define ENABLE_SHSTK_BIT 0x1
 #define ENABLE_IBT_BIT   0x4
 
@@ -92,15 +99,17 @@ int main(int ac, char **av)
 	/* Enable CET master control bit in CR4. */
 	write_cr4(read_cr4() | X86_CR4_CET);
 
-	printf("Unit test for CET user mode...\n");
+	printf("Unit tests for CET user mode...\n");
 	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(rvc && exception_error_code() == 1, "Shadow-stack protection test.");
+	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
+	       "NEAR RET shadow-stack protection test");
 
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
 	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(rvc && exception_error_code() == 3, "Indirect-branch tracking test.");
+	report(rvc && exception_error_code() == CP_ERR_ENDBR,
+	       "Indirect-branch tracking test");
 
 	write_cr4(read_cr4() & ~X86_CR4_CET);
 	wrmsr(MSR_IA32_U_CET, 0);
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 12/18] x86: cet: Test far returns too

[Sean Christopherson]

From: Mathias Krause <minipli@grsecurity.net>

Add a test for far returns which has a dedicated error code.

Tested-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
[sean: use lretl instead of bare lret]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/x86/cet.c b/x86/cet.c
index f19ceb22..eeab5901 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -31,6 +31,34 @@ static uint64_t cet_shstk_func(void)
 	return 0;
 }
 
+static uint64_t cet_shstk_far_ret(void)
+{
+	struct far_pointer32 fp = {
+		.offset = (uintptr_t)&&far_func,
+		.selector = USER_CS,
+	};
+
+	if (fp.offset != (uintptr_t)&&far_func) {
+		printf("Code address too high.\n");
+		return -1;
+	}
+
+	printf("Try to temper the return-address of far-called function...\n");
+
+	/* The NOP isn't superfluous, the called function tries to skip it. */
+	asm goto ("lcall *%0; nop" : : "m" (fp) : : far_func);
+
+	printf("Uhm... how did we get here?! This should have #CP'ed!\n");
+
+	return 0;
+far_func:
+	asm volatile (/* mess with the ret addr, make it point past the NOP */
+		      "incq (%rsp)\n\t"
+		      /* 32-bit return, just as we have been called */
+		      "lretl");
+	__builtin_unreachable();
+}
+
 static uint64_t cet_ibt_func(void)
 {
 	unsigned long tmp;
@@ -104,6 +132,10 @@ int main(int ac, char **av)
 	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
 	       "NEAR RET shadow-stack protection test");
 
+	run_in_user(cet_shstk_far_ret, CP_VECTOR, 0, 0, 0, 0, &rvc);
+	report(rvc && exception_error_code() == CP_ERR_FAR_RET,
+	       "FAR RET shadow-stack protection test");
+
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 13/18] x86: Avoid top-most page for vmalloc on x86-64

[Sean Christopherson]

From: Mathias Krause <minipli@grsecurity.net>

The x86-64 implementation of setup_mmu() doesn't initialize 'vfree_top'
and leaves it at its zero-value. This isn't wrong per se, however, it
leads to odd configurations when the first vmalloc/vmap page gets
allocated. It'll be the very last page in the virtual address space --
which is an interesting corner case -- but its boundary will probably
wrap. It does so, for CET's shadow stack, at least, which loads the
shadow stack pointer with the base address of the mapped page plus its
size, i.e. 0xffffffff_fffff000 + 4096, which wraps to 0x0.

The CPU seems to handle such configurations just fine. However, it feels
odd to set the shadow stack pointer to "NULL".

To avoid the wrapping, ignore the top most page by initializing
'vfree_top' to just one page below.

Reviewed-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 lib/x86/vm.c |  2 ++
 x86/lam.c    | 10 +++++-----
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index 90f73fbb..27e7bb40 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -191,6 +191,8 @@ void *setup_mmu(phys_addr_t end_of_memory, void *opt_mask)
         end_of_memory = (1ul << 32);  /* map mmio 1:1 */
 
     setup_mmu_range(cr3, 0, end_of_memory);
+    /* skip the last page for out-of-bound and wrap-around reasons */
+    init_alloc_vpage((void *)(~(PAGE_SIZE - 1)));
 #else
     setup_mmu_range(cr3, 0, (2ul << 30));
     setup_mmu_range(cr3, 3ul << 30, (1ul << 30));
diff --git a/x86/lam.c b/x86/lam.c
index 1af6c5fd..87efc5dd 100644
--- a/x86/lam.c
+++ b/x86/lam.c
@@ -197,11 +197,11 @@ static void test_lam_sup(void)
 	int vector;
 
 	/*
-	 * KUT initializes vfree_top to 0 for X86_64, and each virtual address
-	 * allocation decreases the size from vfree_top. It's guaranteed that
-	 * the return value of alloc_vpage() is considered as kernel mode
-	 * address and canonical since only a small amount of virtual address
-	 * range is allocated in this test.
+	 * KUT initializes vfree_top to -PAGE_SIZE for X86_64, and each virtual
+	 * address allocation decreases the size from vfree_top. It's
+	 * guaranteed that the return value of alloc_vpage() is considered as
+	 * kernel mode address and canonical since only a small amount of
+	 * virtual address range is allocated in this test.
 	 */
 	vaddr = alloc_vpage();
 	vaddr_mmio = alloc_vpage();
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 14/18] x86: cet: Run SHSTK and IBT tests as appropriate if either feature is supported

[Sean Christopherson]

Run the SHSTK and IBT tests if their respective feature is supported, as
nothing in the architecture requires both features to be supported.
Decoupling the two features allows running the SHSTK test on AMD CPUs,
which support SHSTK but not IBT.

Reviewed-by: Mathias Krause <minipli@grsecurity.net>
Tested-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 50 +++++++++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 17 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index eeab5901..26cd1c9b 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -85,7 +85,7 @@ static uint64_t cet_ibt_func(void)
 #define ENABLE_SHSTK_BIT 0x1
 #define ENABLE_IBT_BIT   0x4
 
-int main(int ac, char **av)
+static void test_shstk(void)
 {
 	char *shstk_virt;
 	unsigned long shstk_phys;
@@ -94,17 +94,10 @@ int main(int ac, char **av)
 	bool rvc;
 
 	if (!this_cpu_has(X86_FEATURE_SHSTK)) {
-		report_skip("SHSTK not enabled");
-		return report_summary();
+		report_skip("SHSTK not supported");
+		return;
 	}
 
-	if (!this_cpu_has(X86_FEATURE_IBT)) {
-		report_skip("IBT not enabled");
-		return report_summary();
-	}
-
-	setup_vm();
-
 	/* Allocate one page for shadow-stack. */
 	shstk_virt = alloc_vpage();
 	shstk_phys = (unsigned long)virt_to_phys(alloc_page());
@@ -124,9 +117,6 @@ int main(int ac, char **av)
 	/* Store shadow-stack pointer. */
 	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
 
-	/* Enable CET master control bit in CR4. */
-	write_cr4(read_cr4() | X86_CR4_CET);
-
 	printf("Unit tests for CET user mode...\n");
 	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
@@ -136,19 +126,45 @@ int main(int ac, char **av)
 	report(rvc && exception_error_code() == CP_ERR_FAR_RET,
 	       "FAR RET shadow-stack protection test");
 
+	/* SSP should be 4-Byte aligned */
+	vector = wrmsr_safe(MSR_IA32_PL3_SSP, 0x1);
+	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
+}
+
+static void test_ibt(void)
+{
+	bool rvc;
+
+	if (!this_cpu_has(X86_FEATURE_IBT)) {
+		report_skip("IBT not supported");
+		return;
+	}
+
 	/* Enable indirect-branch tracking */
 	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
 
 	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_ENDBR,
 	       "Indirect-branch tracking test");
+}
+
+int main(int ac, char **av)
+{
+	if (!this_cpu_has(X86_FEATURE_SHSTK) && !this_cpu_has(X86_FEATURE_IBT)) {
+		report_skip("No CET features supported");
+		return report_summary();
+	}
+
+	setup_vm();
+
+	/* Enable CET global control bit in CR4. */
+	write_cr4(read_cr4() | X86_CR4_CET);
+
+	test_shstk();
+	test_ibt();
 
 	write_cr4(read_cr4() & ~X86_CR4_CET);
 	wrmsr(MSR_IA32_U_CET, 0);
 
-	/* SSP should be 4-Byte aligned */
-	vector = wrmsr_safe(MSR_IA32_PL3_SSP, 0x1);
-	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
-
 	return report_summary();
 }
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 15/18] x86: cet: Drop the "intel_" prefix from the CET testcase

[Sean Christopherson]

Now that the CET test supports both Intel and AMD, drop the "intel" prefix.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/unittests.cfg | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/x86/unittests.cfg b/x86/unittests.cfg
index ff537d3f..522318d3 100644
--- a/x86/unittests.cfg
+++ b/x86/unittests.cfg
@@ -510,7 +510,7 @@ file = tsx-ctrl.flat
 qemu_params = -cpu max
 groups = tsx-ctrl
 
-[intel_cet]
+[cet]
 file = cet.flat
 arch = x86_64
 smp = 2
-- 
2.52.0.rc1.455.g30608eb744-goog

[kvm-unit-tests PATCH v4 16/18] x86: cet: Enable NOTRACK handling for IBT tests

[Sean Christopherson]

gcc's jump table handling makes use of 'notrack' indirect jumps, causing
spurious #CP(3) exceptions.

Enable 'notrack' handling for the IBT tests instead of disabling jump
tables as we may want to make use of 'notrack' ourselves in future
tests.  This will allow using report() in IBT tests, as gcc likes to
generate a small jump table for exception_mnemonic():

 000000000040707c <exception_mnemonic>:
  40707c:       endbr64
  407080:       cmp    $0x1e,%edi
  407083:       ja     407117 <exception_mnemonic+0x9b>
  407089:       mov    %edi,%edi
  40708b:       notrack jmp *0x4107e0(,%rdi,8)
    ::
  4070b1:       mov    $0x411c7c,%eax	# <-- #CP(3) here

Link: https://lore.kernel.org/all/fc886a22-49f3-4627-8ba6-933099e7640d@grsecurity.net
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/x86/cet.c b/x86/cet.c
index 26cd1c9b..74d3f701 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -82,8 +82,9 @@ static uint64_t cet_ibt_func(void)
 #define CP_ERR_SETSSBSY	0x0005
 #define CP_ERR_ENCL		BIT(15)
 
-#define ENABLE_SHSTK_BIT 0x1
-#define ENABLE_IBT_BIT   0x4
+#define CET_ENABLE_SHSTK			BIT(0)
+#define CET_ENABLE_IBT				BIT(2)
+#define CET_ENABLE_NOTRACK			BIT(4)
 
 static void test_shstk(void)
 {
@@ -112,7 +113,7 @@ static void test_shstk(void)
 	install_pte(current_page_table(), 1, shstk_virt, pte, 0);
 
 	/* Enable shadow-stack protection */
-	wrmsr(MSR_IA32_U_CET, ENABLE_SHSTK_BIT);
+	wrmsr(MSR_IA32_U_CET, CET_ENABLE_SHSTK);
 
 	/* Store shadow-stack pointer. */
 	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
@@ -140,8 +141,8 @@ static void test_ibt(void)
 		return;
 	}
 
-	/* Enable indirect-branch tracking */
-	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
+	/* Enable indirect-branch tracking (notrack handling for jump tables) */
+	wrmsr(MSR_IA32_U_CET, CET_ENABLE_IBT | CET_ENABLE_NOTRACK);
 
 	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_ENDBR,
-- 
2.52.0.rc1.455.g30608eb744-goog

Re: [kvm-unit-tests PATCH v4 16/18] x86: cet: Enable NOTRACK handling for IBT tests

[Mathias Krause]

On 14.11.25 21:50, Sean Christopherson wrote:
> gcc's jump table handling makes use of 'notrack' indirect jumps, causing
> spurious #CP(3) exceptions.
> 

Missing a "From: Mathias Krause <minipli@grsecurity.net>", maybe?

> Enable 'notrack' handling for the IBT tests instead of disabling jump
> tables as we may want to make use of 'notrack' ourselves in future
> tests.  This will allow using report() in IBT tests, as gcc likes to
> generate a small jump table for exception_mnemonic():
> 
>  000000000040707c <exception_mnemonic>:
>   40707c:       endbr64
>   407080:       cmp    $0x1e,%edi
>   407083:       ja     407117 <exception_mnemonic+0x9b>
>   407089:       mov    %edi,%edi
>   40708b:       notrack jmp *0x4107e0(,%rdi,8)
>     ::
>   4070b1:       mov    $0x411c7c,%eax	# <-- #CP(3) here
> 
> Link: https://lore.kernel.org/all/fc886a22-49f3-4627-8ba6-933099e7640d@grsecurity.net
> Signed-off-by: Mathias Krause <minipli@grsecurity.net>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  x86/cet.c | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/x86/cet.c b/x86/cet.c
> index 26cd1c9b..74d3f701 100644
> --- a/x86/cet.c
> +++ b/x86/cet.c
> @@ -82,8 +82,9 @@ static uint64_t cet_ibt_func(void)
>  #define CP_ERR_SETSSBSY	0x0005
>  #define CP_ERR_ENCL		BIT(15)
>  
> -#define ENABLE_SHSTK_BIT 0x1
> -#define ENABLE_IBT_BIT   0x4
> +#define CET_ENABLE_SHSTK			BIT(0)
> +#define CET_ENABLE_IBT				BIT(2)
> +#define CET_ENABLE_NOTRACK			BIT(4)
>  
>  static void test_shstk(void)
>  {
> @@ -112,7 +113,7 @@ static void test_shstk(void)
>  	install_pte(current_page_table(), 1, shstk_virt, pte, 0);
>  
>  	/* Enable shadow-stack protection */
> -	wrmsr(MSR_IA32_U_CET, ENABLE_SHSTK_BIT);
> +	wrmsr(MSR_IA32_U_CET, CET_ENABLE_SHSTK);
>  
>  	/* Store shadow-stack pointer. */
>  	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
> @@ -140,8 +141,8 @@ static void test_ibt(void)
>  		return;
>  	}
>  
> -	/* Enable indirect-branch tracking */
> -	wrmsr(MSR_IA32_U_CET, ENABLE_IBT_BIT);
> +	/* Enable indirect-branch tracking (notrack handling for jump tables) */
> +	wrmsr(MSR_IA32_U_CET, CET_ENABLE_IBT | CET_ENABLE_NOTRACK);
>  
>  	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
>  	report(rvc && exception_error_code() == CP_ERR_ENDBR,

Otherwise, LGTM!

Thanks,
Mathias

[kvm-unit-tests PATCH v4 17/18] x86: cet: Reset IBT tracker state on #CP violations

[Sean Christopherson]

Reset the IBT tracker state back to IDLE on #CP violations to not
influence follow-up tests with a poisoned starting state.

Opportunistically rename "rvc" to "got_cp" to make it more obvious what
the flag tracks ("rvc" is presumably "raised vector CP"?).

Signed-off-by: Mathias Krause <minipli@grsecurity.net>
[sean: add helper, align indentation, use handler+callback instead of "extra"]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 lib/x86/usermode.c | 12 +++++++++---
 lib/x86/usermode.h | 13 ++++++++++---
 x86/cet.c          | 31 +++++++++++++++++++++++++++----
 3 files changed, 46 insertions(+), 10 deletions(-)

diff --git a/lib/x86/usermode.c b/lib/x86/usermode.c
index f896e3bd..b65c5378 100644
--- a/lib/x86/usermode.c
+++ b/lib/x86/usermode.c
@@ -21,12 +21,17 @@ static void restore_exec_to_jmpbuf(void)
 	longjmp(jmpbuf, 1);
 }
 
+static handler ex_callback;
+
 static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)
 {
 	this_cpu_write_exception_vector(regs->vector);
 	this_cpu_write_exception_rflags_rf((regs->rflags >> 16) & 1);
 	this_cpu_write_exception_error_code(regs->error_code);
 
+	if (ex_callback)
+		ex_callback(regs);
+
 	/* longjmp must happen after iret, so do not do it now.  */
 	regs->rip = (unsigned long)&restore_exec_to_jmpbuf;
 	regs->cs = KERNEL_CS;
@@ -35,9 +40,9 @@ static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)
 #endif
 }
 
-uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
-		uint64_t arg1, uint64_t arg2, uint64_t arg3,
-		uint64_t arg4, bool *raised_vector)
+uint64_t run_in_user_ex(usermode_func func, unsigned int fault_vector,
+			uint64_t arg1, uint64_t arg2, uint64_t arg3,
+			uint64_t arg4, bool *raised_vector, handler ex_handler)
 {
 	extern char ret_to_kernel;
 	volatile uint64_t rax = 0;
@@ -45,6 +50,7 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
 	handler old_ex;
 
 	*raised_vector = 0;
+	ex_callback = ex_handler;
 	set_idt_entry(RET_TO_KERNEL_IRQ, &ret_to_kernel, 3);
 	old_ex = handle_exception(fault_vector,
 				  restore_exec_to_jmpbuf_exception_handler);
diff --git a/lib/x86/usermode.h b/lib/x86/usermode.h
index 04e358e2..7eca9079 100644
--- a/lib/x86/usermode.h
+++ b/lib/x86/usermode.h
@@ -20,11 +20,18 @@ typedef uint64_t (*usermode_func)(void);
  * Supports running functions with up to 4 arguments.
  * fault_vector: exception vector that might get thrown during the function.
  * raised_vector: outputs true if exception occurred.
+ * ex_handler: optiona handler to call when handling @fault_vector exceptions
  *
  * returns: return value returned by function, or 0 if an exception occurred.
  */
-uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
-		uint64_t arg1, uint64_t arg2, uint64_t arg3,
-		uint64_t arg4, bool *raised_vector);
+uint64_t run_in_user_ex(usermode_func func, unsigned int fault_vector,
+			uint64_t arg1, uint64_t arg2, uint64_t arg3,
+			uint64_t arg4, bool *raised_vector, handler ex_handler);
 
+static inline uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
+				   uint64_t arg1, uint64_t arg2, uint64_t arg3,
+				   uint64_t arg4, bool *raised_vector)
+{
+	return run_in_user_ex(func, fault_vector, arg1, arg2, arg3, arg4, raised_vector, NULL);
+}
 #endif
diff --git a/x86/cet.c b/x86/cet.c
index 74d3f701..7ffe234b 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -1,4 +1,3 @@
-
 #include "libcflat.h"
 #include "x86/desc.h"
 #include "x86/processor.h"
@@ -85,6 +84,8 @@ static uint64_t cet_ibt_func(void)
 #define CET_ENABLE_SHSTK			BIT(0)
 #define CET_ENABLE_IBT				BIT(2)
 #define CET_ENABLE_NOTRACK			BIT(4)
+#define CET_IBT_SUPPRESS			BIT(10)
+#define CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH	BIT(11)
 
 static void test_shstk(void)
 {
@@ -132,9 +133,31 @@ static void test_shstk(void)
 	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
 }
 
+static void ibt_tracker_cp_fixup(struct ex_regs *regs)
+{
+	u64 cet_u = rdmsr(MSR_IA32_U_CET);
+
+	/*
+	 * Switch the IBT tracker state to IDLE to have a clean state for
+	 * following tests.
+	 */
+	if (cet_u & CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH) {
+		cet_u &= ~CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH;
+		printf("CET: suppressing IBT WAIT_FOR_ENDBRANCH state at RIP: %lx\n",
+		       regs->rip);
+		wrmsr(MSR_IA32_U_CET, cet_u);
+	}
+}
+
+static uint64_t ibt_run_in_user(usermode_func func, bool *got_cp)
+{
+	return run_in_user_ex(func, CP_VECTOR, 0, 0, 0, 0, got_cp,
+			      ibt_tracker_cp_fixup);
+}
+
 static void test_ibt(void)
 {
-	bool rvc;
+	bool got_cp;
 
 	if (!this_cpu_has(X86_FEATURE_IBT)) {
 		report_skip("IBT not supported");
@@ -144,8 +167,8 @@ static void test_ibt(void)
 	/* Enable indirect-branch tracking (notrack handling for jump tables) */
 	wrmsr(MSR_IA32_U_CET, CET_ENABLE_IBT | CET_ENABLE_NOTRACK);
 
-	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
-	report(rvc && exception_error_code() == CP_ERR_ENDBR,
+	ibt_run_in_user(cet_ibt_func, &got_cp);
+	report(got_cp && exception_error_code() == CP_ERR_ENDBR,
 	       "Indirect-branch tracking test");
 }
 
-- 
2.52.0.rc1.455.g30608eb744-goog

Re: [kvm-unit-tests PATCH v4 17/18] x86: cet: Reset IBT tracker state on #CP violations

[Mathias Krause]

On 14.11.25 21:50, Sean Christopherson wrote:
> Reset the IBT tracker state back to IDLE on #CP violations to not
> influence follow-up tests with a poisoned starting state.

Again, maybe missing a "From: Mathias Krause <minipli@grsecurity.net>"?

> 
> Opportunistically rename "rvc" to "got_cp" to make it more obvious what
> the flag tracks ("rvc" is presumably "raised vector CP"?).
> 
> Signed-off-by: Mathias Krause <minipli@grsecurity.net>
> [sean: add helper, align indentation, use handler+callback instead of "extra"]
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  lib/x86/usermode.c | 12 +++++++++---
>  lib/x86/usermode.h | 13 ++++++++++---
>  x86/cet.c          | 31 +++++++++++++++++++++++++++----
>  3 files changed, 46 insertions(+), 10 deletions(-)
> 
> diff --git a/lib/x86/usermode.c b/lib/x86/usermode.c
> index f896e3bd..b65c5378 100644
> --- a/lib/x86/usermode.c
> +++ b/lib/x86/usermode.c
> @@ -21,12 +21,17 @@ static void restore_exec_to_jmpbuf(void)
>  	longjmp(jmpbuf, 1);
>  }
>  
> +static handler ex_callback;
> +
>  static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)
>  {
>  	this_cpu_write_exception_vector(regs->vector);
>  	this_cpu_write_exception_rflags_rf((regs->rflags >> 16) & 1);
>  	this_cpu_write_exception_error_code(regs->error_code);
>  
> +	if (ex_callback)
> +		ex_callback(regs);
> +
>  	/* longjmp must happen after iret, so do not do it now.  */
>  	regs->rip = (unsigned long)&restore_exec_to_jmpbuf;
>  	regs->cs = KERNEL_CS;
> @@ -35,9 +40,9 @@ static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)
>  #endif
>  }
>  
> -uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
> -		uint64_t arg1, uint64_t arg2, uint64_t arg3,
> -		uint64_t arg4, bool *raised_vector)
> +uint64_t run_in_user_ex(usermode_func func, unsigned int fault_vector,
> +			uint64_t arg1, uint64_t arg2, uint64_t arg3,
> +			uint64_t arg4, bool *raised_vector, handler ex_handler)
>  {
>  	extern char ret_to_kernel;
>  	volatile uint64_t rax = 0;
> @@ -45,6 +50,7 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
>  	handler old_ex;
>  
>  	*raised_vector = 0;
> +	ex_callback = ex_handler;
>  	set_idt_entry(RET_TO_KERNEL_IRQ, &ret_to_kernel, 3);
>  	old_ex = handle_exception(fault_vector,
>  				  restore_exec_to_jmpbuf_exception_handler);
> diff --git a/lib/x86/usermode.h b/lib/x86/usermode.h
> index 04e358e2..7eca9079 100644
> --- a/lib/x86/usermode.h
> +++ b/lib/x86/usermode.h
> @@ -20,11 +20,18 @@ typedef uint64_t (*usermode_func)(void);
>   * Supports running functions with up to 4 arguments.
>   * fault_vector: exception vector that might get thrown during the function.
>   * raised_vector: outputs true if exception occurred.
> + * ex_handler: optiona handler to call when handling @fault_vector exceptions
                  ^^^^^^^ optional

>   *
>   * returns: return value returned by function, or 0 if an exception occurred.
>   */
> -uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
> -		uint64_t arg1, uint64_t arg2, uint64_t arg3,
> -		uint64_t arg4, bool *raised_vector);
> +uint64_t run_in_user_ex(usermode_func func, unsigned int fault_vector,
> +			uint64_t arg1, uint64_t arg2, uint64_t arg3,
> +			uint64_t arg4, bool *raised_vector, handler ex_handler);
>  
> +static inline uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
> +				   uint64_t arg1, uint64_t arg2, uint64_t arg3,
> +				   uint64_t arg4, bool *raised_vector)
> +{
> +	return run_in_user_ex(func, fault_vector, arg1, arg2, arg3, arg4, raised_vector, NULL);
> +}
>  #endif
> diff --git a/x86/cet.c b/x86/cet.c
> index 74d3f701..7ffe234b 100644
> --- a/x86/cet.c
> +++ b/x86/cet.c
> @@ -1,4 +1,3 @@
> -
>  #include "libcflat.h"
>  #include "x86/desc.h"
>  #include "x86/processor.h"
> @@ -85,6 +84,8 @@ static uint64_t cet_ibt_func(void)
>  #define CET_ENABLE_SHSTK			BIT(0)
>  #define CET_ENABLE_IBT				BIT(2)
>  #define CET_ENABLE_NOTRACK			BIT(4)
> +#define CET_IBT_SUPPRESS			BIT(10)

CET_IBT_SUPPRESS isn't needed (not used in the code), but doesn't hurt
either to have it.

> +#define CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH	BIT(11)
>  
>  static void test_shstk(void)
>  {
> @@ -132,9 +133,31 @@ static void test_shstk(void)
>  	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
>  }
>  
> +static void ibt_tracker_cp_fixup(struct ex_regs *regs)
> +{
> +	u64 cet_u = rdmsr(MSR_IA32_U_CET);
> +
> +	/*
> +	 * Switch the IBT tracker state to IDLE to have a clean state for
> +	 * following tests.
> +	 */
> +	if (cet_u & CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH) {
> +		cet_u &= ~CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH;
> +		printf("CET: suppressing IBT WAIT_FOR_ENDBRANCH state at RIP: %lx\n",
> +		       regs->rip);
> +		wrmsr(MSR_IA32_U_CET, cet_u);
> +	}
> +}
> +
> +static uint64_t ibt_run_in_user(usermode_func func, bool *got_cp)
> +{
> +	return run_in_user_ex(func, CP_VECTOR, 0, 0, 0, 0, got_cp,
> +			      ibt_tracker_cp_fixup);
> +}

Ahh, yeah. Nice indirection, helps making the code less cluttered :)

> +
>  static void test_ibt(void)
>  {
> -	bool rvc;
> +	bool got_cp;
>  
>  	if (!this_cpu_has(X86_FEATURE_IBT)) {
>  		report_skip("IBT not supported");
> @@ -144,8 +167,8 @@ static void test_ibt(void)
>  	/* Enable indirect-branch tracking (notrack handling for jump tables) */
>  	wrmsr(MSR_IA32_U_CET, CET_ENABLE_IBT | CET_ENABLE_NOTRACK);
>  
> -	run_in_user(cet_ibt_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
> -	report(rvc && exception_error_code() == CP_ERR_ENDBR,
> +	ibt_run_in_user(cet_ibt_func, &got_cp);
> +	report(got_cp && exception_error_code() == CP_ERR_ENDBR,
>  	       "Indirect-branch tracking test");
>  }
>  
LGTM!

Thanks,
Mathias

[kvm-unit-tests PATCH v4 18/18] x86: cet: Add testcases to verify KVM rejects emulation of CET instructions

[Sean Christopherson]

Add SHSTK and IBT testcases to verify that KVM rejects (forced) emulation
of instructions that interact with SHSTK and/or IBT state, as KVM doesn't
support emulating SHSTK or IBT (rejecting emulation is preferable to
compromising guest security).

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 x86/cet.c | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 124 insertions(+), 1 deletion(-)

diff --git a/x86/cet.c b/x86/cet.c
index 7ffe234b..e94ffb72 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -74,6 +74,116 @@ static uint64_t cet_ibt_func(void)
 	return 0;
 }
 
+#define __CET_TEST_UNSUPPORTED_INSTRUCTION(insn)			\
+({									\
+	struct far_pointer32 fp = {					\
+		.offset = 0,						\
+		.selector = USER_CS,					\
+	};								\
+									\
+	asm volatile ("push %%rax\n"					\
+		      ASM_TRY_FEP("1f") insn "\n\t"			\
+		      "1:"						\
+		      "pop %%rax\n"					\
+		      : : "m" (fp), "a" (NONCANONICAL) : "memory");	\
+									\
+	exception_vector();					\
+})
+
+#define SHSTK_TEST_UNSUPPORTED_INSTRUCTION(insn)			\
+do {									\
+	uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn);	\
+									\
+	report(vector == UD_VECTOR, "SHSTK: Wanted #UD on %s, got %s",	\
+	       insn, exception_mnemonic(vector));			\
+} while (0)
+
+/*
+ * Treat IRET as unsupported with IBT even though the minimal interactions with
+ * IBT _could_ be easily emulated by KVM, as KVM doesn't support emulating IRET
+ * outside of Real Mode.
+ */
+#define CET_TEST_UNSUPPORTED_INSTRUCTIONS(CET)				\
+do {									\
+	CET##_TEST_UNSUPPORTED_INSTRUCTION("callq *%%rax");		\
+	CET##_TEST_UNSUPPORTED_INSTRUCTION("lcall *%0");		\
+	CET##_TEST_UNSUPPORTED_INSTRUCTION("syscall");			\
+	CET##_TEST_UNSUPPORTED_INSTRUCTION("sysenter");			\
+	CET##_TEST_UNSUPPORTED_INSTRUCTION("iretq");			\
+} while (0)
+
+static uint64_t cet_shstk_emulation(void)
+{
+	CET_TEST_UNSUPPORTED_INSTRUCTIONS(SHSTK);
+
+	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("call 1f");
+	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq");
+	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq $10");
+	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq");
+	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq $10");
+
+	/* Do a handful of JMPs to verify they aren't impacted by SHSTK. */
+	asm volatile(KVM_FEP "jmp 1f\n\t"
+		     "1:\n\t"
+		     KVM_FEP "lea 2f(%%rip), %%rax\n\t"
+		     KVM_FEP "jmp *%%rax\n\t"
+		     "2:\n\t"
+		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
+		     KVM_FEP "lea 3f(%%rip), %%rax\n\t"
+		     KVM_FEP "push %%rax\n\t"
+		     /*
+		      * Manually encode ljmpq, which gas doesn't recognize due
+		      * to AMD not supporting the instruction (64-bit JMP FAR).
+		      */
+		     KVM_FEP ".byte 0x48\n\t"
+		     "ljmpl *(%%rsp)\n\t"
+		     "3:\n\t"
+		     KVM_FEP "pop %%rax\n\t"
+		     KVM_FEP "pop %%rax\n\t"
+		     ::: "eax");
+
+	return 0;
+}
+
+/*
+ * Don't invoke printf() or report() in the IBT testcase, as it will likely
+ * generate an indirect branch without an endbr64 annotation and thus #CP.
+ * Return the line number of the macro invocation to signal failure.
+ */
+#define IBT_TEST_UNSUPPORTED_INSTRUCTION(insn)				\
+do {									\
+	uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn);	\
+									\
+	report(vector == UD_VECTOR, "IBT: Wanted #UD on %s, got %s",	\
+	       insn, exception_mnemonic(vector));			\
+} while (0)
+
+static uint64_t cet_ibt_emulation(void)
+{
+	CET_TEST_UNSUPPORTED_INSTRUCTIONS(IBT);
+
+	IBT_TEST_UNSUPPORTED_INSTRUCTION("jmp *%%rax");
+	IBT_TEST_UNSUPPORTED_INSTRUCTION("ljmpl *%0");
+
+	/* Verify direct CALLs and JMPs, and all RETs aren't impacted by IBT. */
+	asm volatile(KVM_FEP "jmp 2f\n\t"
+		     "1: " KVM_FEP " ret\n\t"
+		     "2: " KVM_FEP " call 1b\n\t"
+		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
+		     KVM_FEP "lea 3f(%%rip), %%rax\n\t"
+		     KVM_FEP "push %%rax\n\t"
+		     KVM_FEP "lretq\n\t"
+		     "3:\n\t"
+		     KVM_FEP "push $0x55555555\n\t"
+		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
+		     KVM_FEP "lea 4f(%%rip), %%rax\n\t"
+		     KVM_FEP "push %%rax\n\t"
+		     KVM_FEP "lretq $8\n\t"
+		     "4:\n\t"
+		     ::: "eax");
+	return 0;
+}
+
 #define CP_ERR_NEAR_RET	0x0001
 #define CP_ERR_FAR_RET	0x0002
 #define CP_ERR_ENDBR	0x0003
@@ -119,7 +229,7 @@ static void test_shstk(void)
 	/* Store shadow-stack pointer. */
 	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
 
-	printf("Unit tests for CET user mode...\n");
+	printf("Running user mode Shadow Stack tests\n");
 	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
 	       "NEAR RET shadow-stack protection test");
@@ -128,6 +238,12 @@ static void test_shstk(void)
 	report(rvc && exception_error_code() == CP_ERR_FAR_RET,
 	       "FAR RET shadow-stack protection test");
 
+	if (is_fep_available &&
+	    (run_in_user(cet_shstk_emulation, CP_VECTOR, 0, 0, 0, 0, &rvc) || rvc))
+		report_fail("Forced emulation with SHSTK generated %s(%u)",
+			    exception_mnemonic(exception_vector()),
+			    exception_error_code());
+
 	/* SSP should be 4-Byte aligned */
 	vector = wrmsr_safe(MSR_IA32_PL3_SSP, 0x1);
 	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
@@ -158,6 +274,7 @@ static uint64_t ibt_run_in_user(usermode_func func, bool *got_cp)
 static void test_ibt(void)
 {
 	bool got_cp;
+	uint64_t l;
 
 	if (!this_cpu_has(X86_FEATURE_IBT)) {
 		report_skip("IBT not supported");
@@ -170,6 +287,12 @@ static void test_ibt(void)
 	ibt_run_in_user(cet_ibt_func, &got_cp);
 	report(got_cp && exception_error_code() == CP_ERR_ENDBR,
 	       "Indirect-branch tracking test");
+
+	if (is_fep_available &&
+	    ((l = ibt_run_in_user(cet_ibt_emulation, &got_cp)) || got_cp))
+		report_fail("Forced emulation with IBT generated %s(%u) at line %lu",
+			    exception_mnemonic(exception_vector()),
+			    exception_error_code(), l);
 }
 
 int main(int ac, char **av)
-- 
2.52.0.rc1.455.g30608eb744-goog

Re: [kvm-unit-tests PATCH v4 18/18] x86: cet: Add testcases to verify KVM rejects emulation of CET instructions

[Mathias Krause]

[-- Attachment #1: Type: text/plain, Size: 7068 bytes --]

On 14.11.25 21:51, Sean Christopherson wrote:
> Add SHSTK and IBT testcases to verify that KVM rejects (forced) emulation
> of instructions that interact with SHSTK and/or IBT state, as KVM doesn't
> support emulating SHSTK or IBT (rejecting emulation is preferable to
> compromising guest security).
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  x86/cet.c | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 124 insertions(+), 1 deletion(-)
> 
> diff --git a/x86/cet.c b/x86/cet.c
> index 7ffe234b..e94ffb72 100644
> --- a/x86/cet.c
> +++ b/x86/cet.c
> @@ -74,6 +74,116 @@ static uint64_t cet_ibt_func(void)
>  	return 0;
>  }
>  
> +#define __CET_TEST_UNSUPPORTED_INSTRUCTION(insn)			\
> +({									\
> +	struct far_pointer32 fp = {					\
> +		.offset = 0,						\
> +		.selector = USER_CS,					\
> +	};								\
> +									\
> +	asm volatile ("push %%rax\n"					\
> +		      ASM_TRY_FEP("1f") insn "\n\t"			\
> +		      "1:"						\
> +		      "pop %%rax\n"					\
> +		      : : "m" (fp), "a" (NONCANONICAL) : "memory");	\
> +									\
> +	exception_vector();					\
> +})
> +
> +#define SHSTK_TEST_UNSUPPORTED_INSTRUCTION(insn)			\
> +do {									\
> +	uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn);	\
> +									\
> +	report(vector == UD_VECTOR, "SHSTK: Wanted #UD on %s, got %s",	\
> +	       insn, exception_mnemonic(vector));			\
> +} while (0)
> +
> +/*
> + * Treat IRET as unsupported with IBT even though the minimal interactions with
> + * IBT _could_ be easily emulated by KVM, as KVM doesn't support emulating IRET
> + * outside of Real Mode.
> + */
> +#define CET_TEST_UNSUPPORTED_INSTRUCTIONS(CET)				\
> +do {									\
> +	CET##_TEST_UNSUPPORTED_INSTRUCTION("callq *%%rax");		\
> +	CET##_TEST_UNSUPPORTED_INSTRUCTION("lcall *%0");		\
                                            ^^^^^
I meant that this one should be "calll *%0" (three Ls). But yeah, could
easily be seen as a typo, so we can just leave it as-is.

> +	CET##_TEST_UNSUPPORTED_INSTRUCTION("syscall");			\
> +	CET##_TEST_UNSUPPORTED_INSTRUCTION("sysenter");			\
> +	CET##_TEST_UNSUPPORTED_INSTRUCTION("iretq");			\
> +} while (0)
> +
> +static uint64_t cet_shstk_emulation(void)
> +{
> +	CET_TEST_UNSUPPORTED_INSTRUCTIONS(SHSTK);
> +
> +	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("call 1f");
> +	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq");
> +	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq $10");
> +	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq");
> +	SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq $10");
> +
> +	/* Do a handful of JMPs to verify they aren't impacted by SHSTK. */
> +	asm volatile(KVM_FEP "jmp 1f\n\t"
> +		     "1:\n\t"
> +		     KVM_FEP "lea 2f(%%rip), %%rax\n\t"
> +		     KVM_FEP "jmp *%%rax\n\t"
> +		     "2:\n\t"
> +		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
> +		     KVM_FEP "lea 3f(%%rip), %%rax\n\t"
> +		     KVM_FEP "push %%rax\n\t"
> +		     /*
> +		      * Manually encode ljmpq, which gas doesn't recognize due

Well, it does, if explicitly told so:

$ echo 'ljmpq *(%rax)' | as
{standard input}: Assembler messages:
{standard input}:1: Error: invalid instruction suffix for `ljmp'
$ echo 'ljmpq *(%rax)' | as -mintel64 && objdump -d -Mintel64 | tail -1
   0:	48 ff 28             	ljmpq  *(%rax)

But lets not split hairs!

> +		      * to AMD not supporting the instruction (64-bit JMP FAR).
> +		      */
> +		     KVM_FEP ".byte 0x48\n\t"
> +		     "ljmpl *(%%rsp)\n\t"
> +		     "3:\n\t"
> +		     KVM_FEP "pop %%rax\n\t"
> +		     KVM_FEP "pop %%rax\n\t"
> +		     ::: "eax");
> +
> +	return 0;
> +}
> +

> +/*
> + * Don't invoke printf() or report() in the IBT testcase, as it will likely
> + * generate an indirect branch without an endbr64 annotation and thus #CP.
> + * Return the line number of the macro invocation to signal failure.
> + */

That comment is outdated and can simply be dropped now.

> +#define IBT_TEST_UNSUPPORTED_INSTRUCTION(insn)				\
> +do {									\
> +	uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn);	\
> +									\
> +	report(vector == UD_VECTOR, "IBT: Wanted #UD on %s, got %s",	\
> +	       insn, exception_mnemonic(vector));			\
> +} while (0)
> +
> +static uint64_t cet_ibt_emulation(void)
> +{
> +	CET_TEST_UNSUPPORTED_INSTRUCTIONS(IBT);
> +
> +	IBT_TEST_UNSUPPORTED_INSTRUCTION("jmp *%%rax");
> +	IBT_TEST_UNSUPPORTED_INSTRUCTION("ljmpl *%0");
> +
> +	/* Verify direct CALLs and JMPs, and all RETs aren't impacted by IBT. */
> +	asm volatile(KVM_FEP "jmp 2f\n\t"
> +		     "1: " KVM_FEP " ret\n\t"
> +		     "2: " KVM_FEP " call 1b\n\t"
> +		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
> +		     KVM_FEP "lea 3f(%%rip), %%rax\n\t"
> +		     KVM_FEP "push %%rax\n\t"
> +		     KVM_FEP "lretq\n\t"
> +		     "3:\n\t"
> +		     KVM_FEP "push $0x55555555\n\t"
> +		     KVM_FEP "push $" xstr(USER_CS) "\n\t"
> +		     KVM_FEP "lea 4f(%%rip), %%rax\n\t"
> +		     KVM_FEP "push %%rax\n\t"
> +		     KVM_FEP "lretq $8\n\t"
> +		     "4:\n\t"
> +		     ::: "eax");
> +	return 0;
> +}
> +
>  #define CP_ERR_NEAR_RET	0x0001
>  #define CP_ERR_FAR_RET	0x0002
>  #define CP_ERR_ENDBR	0x0003
> @@ -119,7 +229,7 @@ static void test_shstk(void)
>  	/* Store shadow-stack pointer. */
>  	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
>  
> -	printf("Unit tests for CET user mode...\n");
> +	printf("Running user mode Shadow Stack tests\n");
>  	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
>  	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
>  	       "NEAR RET shadow-stack protection test");
> @@ -128,6 +238,12 @@ static void test_shstk(void)
>  	report(rvc && exception_error_code() == CP_ERR_FAR_RET,
>  	       "FAR RET shadow-stack protection test");
>  
> +	if (is_fep_available &&
> +	    (run_in_user(cet_shstk_emulation, CP_VECTOR, 0, 0, 0, 0, &rvc) || rvc))
> +		report_fail("Forced emulation with SHSTK generated %s(%u)",
> +			    exception_mnemonic(exception_vector()),
> +			    exception_error_code());
> +
>  	/* SSP should be 4-Byte aligned */
>  	vector = wrmsr_safe(MSR_IA32_PL3_SSP, 0x1);
>  	report(vector == GP_VECTOR, "MSR_IA32_PL3_SSP alignment test.");
> @@ -158,6 +274,7 @@ static uint64_t ibt_run_in_user(usermode_func func, bool *got_cp)
>  static void test_ibt(void)
>  {
>  	bool got_cp;

> +	uint64_t l;

>  
>  	if (!this_cpu_has(X86_FEATURE_IBT)) {
>  		report_skip("IBT not supported");
> @@ -170,6 +287,12 @@ static void test_ibt(void)
>  	ibt_run_in_user(cet_ibt_func, &got_cp);
>  	report(got_cp && exception_error_code() == CP_ERR_ENDBR,
>  	       "Indirect-branch tracking test");
> +
> +	if (is_fep_available &&
> +	    ((l = ibt_run_in_user(cet_ibt_emulation, &got_cp)) || got_cp))
> +		report_fail("Forced emulation with IBT generated %s(%u) at line %lu",
> +			    exception_mnemonic(exception_vector()),
> +			    exception_error_code(), l);

cet_ibt_emulation() returns no line information any more, so that should
be simplified.

>  }
>  
>  int main(int ac, char **av)
With something like the attached diff as a fixup:
Reviewed-by: Mathias Krause <minipli@grsecurity.net>

Thanks,
Mathias

[-- Attachment #2: fep_fixup.diff --]
[-- Type: text/x-patch, Size: 1599 bytes --]

diff --git a/x86/cet.c b/x86/cet.c
index e94ffb725e88..9a5913f764ea 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -145,11 +145,6 @@ static uint64_t cet_shstk_emulation(void)
 	return 0;
 }
 
-/*
- * Don't invoke printf() or report() in the IBT testcase, as it will likely
- * generate an indirect branch without an endbr64 annotation and thus #CP.
- * Return the line number of the macro invocation to signal failure.
- */
 #define IBT_TEST_UNSUPPORTED_INSTRUCTION(insn)				\
 do {									\
 	uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn);	\
@@ -194,7 +189,6 @@ static uint64_t cet_ibt_emulation(void)
 #define CET_ENABLE_SHSTK			BIT(0)
 #define CET_ENABLE_IBT				BIT(2)
 #define CET_ENABLE_NOTRACK			BIT(4)
-#define CET_IBT_SUPPRESS			BIT(10)
 #define CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH	BIT(11)
 
 static void test_shstk(void)
@@ -274,7 +268,6 @@ static uint64_t ibt_run_in_user(usermode_func func, bool *got_cp)
 static void test_ibt(void)
 {
 	bool got_cp;
-	uint64_t l;
 
 	if (!this_cpu_has(X86_FEATURE_IBT)) {
 		report_skip("IBT not supported");
@@ -289,10 +282,10 @@ static void test_ibt(void)
 	       "Indirect-branch tracking test");
 
 	if (is_fep_available &&
-	    ((l = ibt_run_in_user(cet_ibt_emulation, &got_cp)) || got_cp))
-		report_fail("Forced emulation with IBT generated %s(%u) at line %lu",
+	    (ibt_run_in_user(cet_ibt_emulation, &got_cp) || got_cp))
+		report_fail("Forced emulation with IBT generated %s(%u)",
 			    exception_mnemonic(exception_vector()),
-			    exception_error_code(), l);
+			    exception_error_code());
 }
 
 int main(int ac, char **av)

Re: [kvm-unit-tests PATCH v4 18/18] x86: cet: Add testcases to verify KVM rejects emulation of CET instructions

[Mathias Krause]

On 14.11.25 21:51, Sean Christopherson wrote:
> Add SHSTK and IBT testcases to verify that KVM rejects (forced) emulation
> of instructions that interact with SHSTK and/or IBT state, as KVM doesn't
> support emulating SHSTK or IBT (rejecting emulation is preferable to
> compromising guest security).
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> [...]

Successfully tested on v6.18-rc6:

Tested-by: Mathias Krause <minipli@grsecurity.net>

Thanks,
Mathias

Re: [kvm-unit-tests PATCH v4 00/18] x86: Improve CET tests

[Sean Christopherson]

On Fri, 14 Nov 2025 12:50:42 -0800, Sean Christopherson wrote:
> Hopefully the last version of this particular CET series.  Mathias, I owe you
> like five beers for root causing and fixing all the gnarly edge cases.
> 
> v4:
>  - Fixup the argumentes for the vmx_cet_test. [Mathias]
>  - Drop "_test" from the vmx_cet config to match the other VMX testcases.
>  - Enable NOTRACK instead of dodging jmp tables in exception_mnemonic() [Mathias].
>  - Reset IBT state after (intentional) #CP. [Mathias]
>  - Fix a changelog typo. [Mathias]
>  - Document that ljmpq isn't supported on AMD. [Mathias]
>  - Use ljmpl to make the 32-bit JMP FAR more obvious. [Mathias]
> 
> [...]

Applied to kvm-x86 next, with fixups for the goofs reported by Mathias.

[01/18] x86: cet: Pass virtual addresses to invlpg
        https://github.com/kvm-x86/kvm-unit-tests/commit/d50f434d3221
[02/18] x86: cet: Remove unnecessary memory zeroing for shadow stack
        https://github.com/kvm-x86/kvm-unit-tests/commit/648aa2c9b31e
[03/18] x86: cet: Directly check for #CP exception in run_in_user()
        https://github.com/kvm-x86/kvm-unit-tests/commit/3194f87894d7
[04/18] x86: cet: Validate #CP error code
        https://github.com/kvm-x86/kvm-unit-tests/commit/2cafa419137c
[05/18] x86: cet: Use report_skip()
        https://github.com/kvm-x86/kvm-unit-tests/commit/7607229f204e
[06/18] x86: cet: Drop unnecessary casting
        https://github.com/kvm-x86/kvm-unit-tests/commit/9981a3a28ad8
[07/18] x86: cet: Validate writing unaligned values to SSP MSR causes #GP
        https://github.com/kvm-x86/kvm-unit-tests/commit/6133fbee3ef7
[08/18] x86: cet: Validate CET states during VMX transitions
        https://github.com/kvm-x86/kvm-unit-tests/commit/986bd380204f
[09/18] x86: cet: Make shadow stack less fragile
        https://github.com/kvm-x86/kvm-unit-tests/commit/00d5543c241b
[10/18] x86: cet: Simplify IBT test
        https://github.com/kvm-x86/kvm-unit-tests/commit/bc0cc3c28b31
[11/18] x86: cet: Use symbolic values for the #CP error codes
        https://github.com/kvm-x86/kvm-unit-tests/commit/5690621f59d6
[12/18] x86: cet: Test far returns too
        https://github.com/kvm-x86/kvm-unit-tests/commit/f472c43a4b64
[13/18] x86: Avoid top-most page for vmalloc on x86-64
        https://github.com/kvm-x86/kvm-unit-tests/commit/63e5bb9be210
[14/18] x86: cet: Run SHSTK and IBT tests as appropriate if either feature is supported
        https://github.com/kvm-x86/kvm-unit-tests/commit/57793a2090f8
[15/18] x86: cet: Drop the "intel_" prefix from the CET testcase
        https://github.com/kvm-x86/kvm-unit-tests/commit/09124b986c65
[16/18] x86: cet: Enable NOTRACK handling for IBT tests
        https://github.com/kvm-x86/kvm-unit-tests/commit/3ba7b9377a7b
[17/18] x86: cet: Reset IBT tracker state on #CP violations
        https://github.com/kvm-x86/kvm-unit-tests/commit/03722f15f4f5
[18/18] x86: cet: Add testcases to verify KVM rejects emulation of CET instructions
        https://github.com/kvm-x86/kvm-unit-tests/commit/cb2b118200c0

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next