[kvm-unit-tests PATCH v7 0/5] x86: Add test cases for LAM

[kvm-unit-tests PATCH v7 0/5] x86: Add test cases for LAM

[Binbin Wu]

Intel Linear-address masking (LAM) [1], modifies the checking that is applied to
*64-bit* linear addresses, allowing software to use of the untranslated address
bits for metadata.

The patch series add test cases for KVM LAM:

Patch 1 moves struct invpcid_desc to header file for new test cases.
Patch 2 makes change to allow setting of CR3 LAM bits in vmlaunch tests.
Patch 3~5 add test cases for LAM supervisor mode and user mode, including:
- For supervisor mode
  CR4.LAM_SUP toggle
  Memory/MMIO access with tagged pointer
  INVLPG
  INVPCID
  INVVPID (also used to cover VMX instruction VMExit path)
- For user mode
  CR3 LAM bits toggle 
  Memory/MMIO access with tagged pointer

[1] Intel ISE https://cdrdv2.intel.com/v1/dl/getContent/671368
    Chapter Linear Address Masking (LAM)
---
Changelog:
v7
- Move struct invpcid_desc to header file instead of defining a new copy in lam.c.
- Rename is_la57()/lam_sup_active() to is_la57_enabled()/is_lam_sup_enabled(),
  and move them to processor.h (Sean)
- Drop cr4_set_lam_sup()/cr4_clear_lam_sup() and use write_cr4_safe() instead. (Sean)
- Add get_lam_mask() to get lam status based on the address and vCPU state. (Sean)
- Drop the wrappers for INVLPG since INVLPG never faults. (Sean)
- Drop the wrapper for INVPCID and use invpcid_safe() instead. (Sean)
- Drop the check for X86_FEATURE_PCID. (Sean)
- Rename lam_{u48,u57}_active() to is_lam_{u48,u57}_enabled(), and move them to
  processor.h (Sean)
- Test LAM userspace address in kernel mode directly to simplify the interface
  of test_ptr() since LAM identifies a address as kernel or user only based on
  the address itself.
- Add comments about the virtualization hole of CR3 LAM bits.
- Drop the check of X86_FEATURE_LA57 when check LA57. (Sean)

v6
- https://lore.kernel.org/kvm/20240122085354.9510-1-binbin.wu@linux.intel.com/

v5
- https://lore.kernel.org/kvm/20230530024356.24870-1-binbin.wu@linux.intel.com/

Binbin Wu (4):
  x86: Move struct invpcid_desc to processor.h
  x86: Allow setting of CR3 LAM bits if LAM supported
  x86: Add test cases for LAM_{U48,U57}
  x86: Add test case for INVVPID with LAM

Robert Hoo (1):
  x86: Add test case for LAM_SUP

 lib/x86/processor.h |  41 +++++++
 x86/Makefile.x86_64 |   1 +
 x86/lam.c           | 281 ++++++++++++++++++++++++++++++++++++++++++++
 x86/pcid.c          |   6 -
 x86/unittests.cfg   |  10 ++
 x86/vmx_tests.c     |  51 +++++++-
 6 files changed, 382 insertions(+), 8 deletions(-)
 create mode 100644 x86/lam.c


base-commit: d301d0187f5db09531a1c2c7608997cc3b0a5c7d
-- 
2.43.2

[kvm-unit-tests PATCH v7 1/5] x86: Move struct invpcid_desc to processor.h

[Binbin Wu]

Move struct invpcid_desc to processor.h so that new test cases need
to do invpcid can use the definition.

Opportunistically add packed attribute, because according to C standard,
the allocation of storage unit for bit-field is implementation specific.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
---
v7:
- New added.
---
 lib/x86/processor.h | 6 ++++++
 x86/pcid.c          | 6 ------
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/x86/processor.h b/lib/x86/processor.h
index da1ed662..85a1781b 100644
--- a/lib/x86/processor.h
+++ b/lib/x86/processor.h
@@ -959,4 +959,10 @@ static inline void generate_cr0_em_nm(void)
 	fnop();
 }
 
+struct invpcid_desc {
+	u64 pcid : 12;
+	u64 rsv  : 52;
+	u64 addr : 64;
+} __attribute__((packed));
+
 #endif
diff --git a/x86/pcid.c b/x86/pcid.c
index c503efb8..7425e0fe 100644
--- a/x86/pcid.c
+++ b/x86/pcid.c
@@ -4,12 +4,6 @@
 #include "processor.h"
 #include "desc.h"
 
-struct invpcid_desc {
-    u64 pcid : 12;
-    u64 rsv  : 52;
-    u64 addr : 64;
-};
-
 static void test_pcid_enabled(void)
 {
     int passed = 0;
-- 
2.43.2

[kvm-unit-tests PATCH v7 2/5] x86: Allow setting of CR3 LAM bits if LAM supported

[Binbin Wu]

If LINEAR ADDRESS MASKING (LAM) is supported, VM entry allows CR3.LAM_U48
(bit 62) and CR3.LAM_U57 (bit 61) to be set in CR3 field.

Change the test result expectations when setting CR3.LAM_U48 or CR3.LAM_U57
on vmlaunch tests when LAM is supported.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
---
 lib/x86/processor.h | 3 +++
 x86/vmx_tests.c     | 6 +++++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/x86/processor.h b/lib/x86/processor.h
index 85a1781b..f7f2df50 100644
--- a/lib/x86/processor.h
+++ b/lib/x86/processor.h
@@ -68,6 +68,8 @@
 #define X86_CR0_PG		BIT(X86_CR0_PG_BIT)
 
 #define X86_CR3_PCID_MASK	GENMASK(11, 0)
+#define X86_CR3_LAM_U57_BIT	(61)
+#define X86_CR3_LAM_U48_BIT	(62)
 
 #define X86_CR4_VME_BIT		(0)
 #define X86_CR4_VME		BIT(X86_CR4_VME_BIT)
@@ -262,6 +264,7 @@ static inline bool is_intel(void)
 #define	X86_FEATURE_FLUSH_L1D		(CPUID(0x7, 0, EDX, 28))
 #define	X86_FEATURE_ARCH_CAPABILITIES	(CPUID(0x7, 0, EDX, 29))
 #define	X86_FEATURE_PKS			(CPUID(0x7, 0, ECX, 31))
+#define	X86_FEATURE_LAM			(CPUID(0x7, 1, EAX, 26))
 
 /*
  * KVM defined leafs
diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index ffe7064c..4b161c3c 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -7032,7 +7032,11 @@ static void test_host_ctl_regs(void)
 		cr3 = cr3_saved | (1ul << i);
 		vmcs_write(HOST_CR3, cr3);
 		report_prefix_pushf("HOST_CR3 %lx", cr3);
-		test_vmx_vmlaunch(VMXERR_ENTRY_INVALID_HOST_STATE_FIELD);
+		if (this_cpu_has(X86_FEATURE_LAM) &&
+		    ((i == X86_CR3_LAM_U57_BIT) || (i == X86_CR3_LAM_U48_BIT)))
+			test_vmx_vmlaunch(0);
+		else
+			test_vmx_vmlaunch(VMXERR_ENTRY_INVALID_HOST_STATE_FIELD);
 		report_prefix_pop();
 	}
 
-- 
2.43.2

[kvm-unit-tests PATCH v7 3/5] x86: Add test case for LAM_SUP

[Binbin Wu]

From: Robert Hoo <robert.hu@linux.intel.com>

This unit test covers:
1. CR4.LAM_SUP toggles.
2. Memory & MMIO access with supervisor mode address with LAM metadata.
3. INVLPG memory operand doesn't contain LAM meta data, if the address
   is non-canonical form then the INVLPG is the same as a NOP (no #GP).
4. INVPCID memory operand (descriptor pointer) could contain LAM meta data,
   however, the address in the descriptor should be canonical.

In x86/unittests.cfg, add 2 test cases/guest conf, with and without LAM.

LAM feature spec: https://cdrdv2.intel.com/v1/dl/getContent/671368,
Chapter LINEAR ADDRESS MASKING (LAM)

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Co-developed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
---
v7：
- Rename is_la57()/lam_sup_active() to is_la57_enabled()/is_lam_sup_enabled(),
  and move them to processor.h (Sean)
- Drop cr4_set_lam_sup()/cr4_clear_lam_sup() and use write_cr4_safe() instead. (Sean)
- Add get_lam_mask() to get lam status based on the address and vCPU state. (Sean)
- Drop the wrappers for INVLPG since INVLPG never faults. (Sean)
- Drop the wrapper for INVPCID and use invpcid_safe() instead. (Sean)
- Drop the check for X86_FEATURE_PCID. (Sean)
---
 lib/x86/processor.h |  20 +++++
 x86/Makefile.x86_64 |   1 +
 x86/lam.c           | 214 ++++++++++++++++++++++++++++++++++++++++++++
 x86/unittests.cfg   |  10 +++
 4 files changed, 245 insertions(+)
 create mode 100644 x86/lam.c

diff --git a/lib/x86/processor.h b/lib/x86/processor.h
index f7f2df50..a38f87ed 100644
--- a/lib/x86/processor.h
+++ b/lib/x86/processor.h
@@ -8,6 +8,14 @@
 #include <stdint.h>
 
 #define NONCANONICAL	0xaaaaaaaaaaaaaaaaull
+#define LAM57_MASK	GENMASK_ULL(62, 57)
+#define LAM48_MASK	GENMASK_ULL(62, 48)
+
+/* Set metadata with non-canonical pattern in mask bits of a linear address */
+static inline u64 set_la_non_canonical(u64 src, u64 mask)
+{
+	return (src & ~mask) | (NONCANONICAL & mask);
+}
 
 #ifdef __x86_64__
 #  define R "r"
@@ -120,6 +128,8 @@
 #define X86_CR4_CET		BIT(X86_CR4_CET_BIT)
 #define X86_CR4_PKS_BIT		(24)
 #define X86_CR4_PKS		BIT(X86_CR4_PKS_BIT)
+#define X86_CR4_LAM_SUP_BIT	(28)
+#define X86_CR4_LAM_SUP		BIT(X86_CR4_LAM_SUP_BIT)
 
 #define X86_EFLAGS_CF_BIT	(0)
 #define X86_EFLAGS_CF		BIT(X86_EFLAGS_CF_BIT)
@@ -968,4 +978,14 @@ struct invpcid_desc {
 	u64 addr : 64;
 } __attribute__((packed));
 
+static inline bool is_la57_enabled(void)
+{
+	return !!(read_cr4() & X86_CR4_LA57);
+}
+
+static inline bool is_lam_sup_enabled(void)
+{
+	return !!(read_cr4() & X86_CR4_LAM_SUP);
+}
+
 #endif
diff --git a/x86/Makefile.x86_64 b/x86/Makefile.x86_64
index 2771a6fa..e5db2365 100644
--- a/x86/Makefile.x86_64
+++ b/x86/Makefile.x86_64
@@ -38,6 +38,7 @@ tests += $(TEST_DIR)/rdpru.$(exe)
 tests += $(TEST_DIR)/pks.$(exe)
 tests += $(TEST_DIR)/pmu_lbr.$(exe)
 tests += $(TEST_DIR)/pmu_pebs.$(exe)
+tests += $(TEST_DIR)/lam.$(exe)
 
 ifeq ($(CONFIG_EFI),y)
 tests += $(TEST_DIR)/amd_sev.$(exe)
diff --git a/x86/lam.c b/x86/lam.c
new file mode 100644
index 00000000..2f95b6c9
--- /dev/null
+++ b/x86/lam.c
@@ -0,0 +1,214 @@
+/*
+ * Intel LAM unit test
+ *
+ * Copyright (C) 2023 Intel
+ *
+ * Author: Robert Hoo <robert.hu@linux.intel.com>
+ *         Binbin Wu <binbin.wu@linux.intel.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or
+ * later.
+ */
+
+#include "libcflat.h"
+#include "processor.h"
+#include "desc.h"
+#include "vmalloc.h"
+#include "alloc_page.h"
+#include "vm.h"
+#include "asm/io.h"
+#include "ioram.h"
+
+static void test_cr4_lam_set_clear(void)
+{
+	int vector;
+	bool has_lam = this_cpu_has(X86_FEATURE_LAM);
+
+	vector = write_cr4_safe(read_cr4() | X86_CR4_LAM_SUP);
+	report(has_lam ? !vector : vector == GP_VECTOR,
+	       "Expected CR4.LAM_SUP=1 to %s", has_lam ? "succeed" : "#GP");
+
+	vector = write_cr4_safe(read_cr4() & ~X86_CR4_LAM_SUP);
+	report(!vector, "Expected CR4.LAM_SUP=0 to succeed");
+}
+
+/* Refer to emulator.c */
+static void do_mov(void *mem)
+{
+	unsigned long t1, t2;
+
+	t1 = 0x123456789abcdefull & -1ul;
+	asm volatile("mov %[t1], (%[mem])\n\t"
+		     "mov (%[mem]), %[t2]"
+		     : [t2]"=r"(t2)
+		     : [t1]"r"(t1), [mem]"r"(mem)
+		     : "memory");
+	report(t1 == t2, "Mov result check");
+}
+
+static bool get_lam_mask(u64 address, u64* lam_mask)
+{
+	/*
+	 * Use LAM57_MASK as mask to construct non-canonical address if LAM is
+	 * not supported or enabled.
+	 */
+	*lam_mask = LAM57_MASK;
+
+	/*
+	 * Bit 63 determines if the address should be treated as a user address
+	 * or a supervisor address.
+	 */
+	if (address & BIT_ULL(63)) {
+		if (!(is_lam_sup_enabled()))
+			return false;
+
+		if (!is_la57_enabled())
+			*lam_mask = LAM48_MASK;
+		return true;
+	}
+
+	/* TODO: Get LAM mask for userspace address. */
+	return false;
+}
+
+
+static void test_ptr(u64* ptr, bool is_mmio)
+{
+	u64 lam_mask;
+	bool lam_active, fault;
+
+	lam_active = get_lam_mask((u64)ptr, &lam_mask);
+
+	fault = test_for_exception(GP_VECTOR, do_mov, ptr);
+	report(!fault, "Expected access to untagged address for %s to succeed",
+	       is_mmio ? "MMIO" : "memory");
+
+	ptr = (u64 *)set_la_non_canonical((u64)ptr, lam_mask);
+	fault = test_for_exception(GP_VECTOR, do_mov, ptr);
+	report(fault != lam_active, "Expected access to tagged address for %s %s LAM to %s",
+	       is_mmio ? "MMIO" : "memory", lam_active ? "with" : "without",
+	       lam_active ? "succeed" : "#GP");
+}
+
+/* invlpg with tagged address is same as NOP, no #GP expected. */
+static void test_invlpg(void *va, bool fep)
+{
+	u64 lam_mask;
+	u64 *ptr;
+
+	/*
+	 * The return value is not checked, invlpg should never faults no matter
+	 * LAM is supported or not.
+	 */
+	get_lam_mask((u64)va, &lam_mask);
+	ptr = (u64 *)set_la_non_canonical((u64)va, lam_mask);
+	if (fep)
+		asm volatile(KVM_FEP "invlpg (%0)" ::"r" (ptr) : "memory");
+	else
+		invlpg(ptr);
+
+	report(true, "Expected %sINVLPG with tagged addr to succeed", fep ? "fep: " : "");
+}
+
+/* LAM doesn't apply to the linear address in the descriptor of invpcid */
+static void test_invpcid(void *data)
+{
+	/*
+	 * Reuse the memory address for the descriptor since stack memory
+	 * address in KUT doesn't follow the kernel address space partitions.
+	 */
+	struct invpcid_desc *desc_ptr = (struct invpcid_desc *)data;
+	int vector;
+	u64 lam_mask;
+	bool lam_active;
+
+	if (!this_cpu_has(X86_FEATURE_INVPCID)) {
+		report_skip("INVPCID not supported");
+		return;
+	}
+
+	lam_active = get_lam_mask((u64)data, &lam_mask);
+
+	memset(desc_ptr, 0, sizeof(struct invpcid_desc));
+	desc_ptr->addr = (u64)data;
+
+	vector = invpcid_safe(0, desc_ptr);
+	report(!vector,
+	       "Expected INVPCID with untagged pointer + untagged addr to succeed");
+
+	desc_ptr->addr = set_la_non_canonical(desc_ptr->addr, lam_mask);
+	vector = invpcid_safe(0, desc_ptr);
+	report(vector==GP_VECTOR,
+	       "Expected INVPCID with untagged pointer + tagged addr to #GP");
+
+	desc_ptr = (struct invpcid_desc *)set_la_non_canonical((u64)desc_ptr,
+							       lam_mask);
+	vector = invpcid_safe(0, desc_ptr);
+	report(vector==GP_VECTOR,
+	       "Expected INVPCID with tagged pointer + tagged addr to #GP");
+
+	desc_ptr = (struct invpcid_desc *)data;
+	desc_ptr->addr = (u64)data;
+	desc_ptr = (struct invpcid_desc *)set_la_non_canonical((u64)desc_ptr,
+							       lam_mask);
+	vector = invpcid_safe(0, desc_ptr);
+	report(lam_active ? !vector : vector==GP_VECTOR,
+	       "Expected INVPCID with tagged pointer + untagged addr to %s",
+	       lam_active? "succeed" : "#GP");
+}
+
+static void test_lam_sup(void)
+{
+	void *vaddr, *vaddr_mmio;
+	phys_addr_t paddr;
+	unsigned long cr4 = read_cr4();
+	int vector;
+
+	/*
+	 * KUT initializes vfree_top to 0 for X86_64, and each virtual address
+	 * allocation decreases the size from vfree_top. It's guaranteed that
+	 * the return value of alloc_vpage() is considered as kernel mode
+	 * address and canonical since only a small mount virtual address range
+	 * is allocated in this test.
+	 */
+	vaddr = alloc_vpage();
+	vaddr_mmio = alloc_vpage();
+	paddr = virt_to_phys(alloc_page());
+	install_page(current_page_table(), paddr, vaddr);
+	install_page(current_page_table(), IORAM_BASE_PHYS, vaddr_mmio);
+
+	test_cr4_lam_set_clear();
+
+	/* Set for the following LAM_SUP tests. */
+	if (this_cpu_has(X86_FEATURE_LAM)) {
+		vector = write_cr4_safe(cr4 | X86_CR4_LAM_SUP);
+		report(!vector && is_lam_sup_enabled(),
+		       "Expected CR4.LAM_SUP=1 to succeed");
+	}
+
+	/* Test for normal memory. */
+	test_ptr(vaddr, false);
+	/* Test for MMIO to trigger instruction emulation. */
+	test_ptr(vaddr_mmio, true);
+	test_invpcid(vaddr);
+	test_invlpg(vaddr, false);
+	if (is_fep_available())
+		test_invlpg(vaddr, true);
+	else
+		report_skip("Skipping tests the forced emulation, "
+			    "use kvm.force_emulation_prefix=1 to enable\n");
+}
+
+int main(int ac, char **av)
+{
+	setup_vm();
+
+	if (!this_cpu_has(X86_FEATURE_LAM))
+		report_info("This CPU doesn't support LAM feature\n");
+	else
+		report_info("This CPU supports LAM feature\n");
+
+	test_lam_sup();
+
+	return report_summary();
+}
diff --git a/x86/unittests.cfg b/x86/unittests.cfg
index 7c1691a9..f1178edd 100644
--- a/x86/unittests.cfg
+++ b/x86/unittests.cfg
@@ -490,3 +490,13 @@ file = cet.flat
 arch = x86_64
 smp = 2
 extra_params = -enable-kvm -m 2048 -cpu host
+
+[intel-lam]
+file = lam.flat
+arch = x86_64
+extra_params = -enable-kvm -cpu host
+
+[intel-no-lam]
+file = lam.flat
+arch = x86_64
+extra_params = -enable-kvm -cpu host,-lam
-- 
2.43.2

[kvm-unit-tests PATCH v7 4/5] x86: Add test cases for LAM_{U48,U57}

[Binbin Wu]

This unit test covers:
1. CR3 LAM bits toggles.
2. Memory/MMIO access with user mode address containing LAM metadata.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
---
v7:
- Rename lam_{u48,u57}_active() to is_lam_{u48,u57}_enabled(), and move them to
  processor.h (Sean)
- Get lam status based on the address and vCPU state. (Sean)
- Test LAM userspace address in kernel mode directly to simplify the interface
  of test_ptr() since LAM identifies a address as kernel or user only based on
  the address itself.
- Add comments about the virtualization hole of CR3 LAM bits.
---
 lib/x86/processor.h | 12 ++++++++
 x86/lam.c           | 69 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/lib/x86/processor.h b/lib/x86/processor.h
index a38f87ed..c2cafb01 100644
--- a/lib/x86/processor.h
+++ b/lib/x86/processor.h
@@ -77,7 +77,9 @@ static inline u64 set_la_non_canonical(u64 src, u64 mask)
 
 #define X86_CR3_PCID_MASK	GENMASK(11, 0)
 #define X86_CR3_LAM_U57_BIT	(61)
+#define X86_CR3_LAM_U57		BIT_ULL(X86_CR3_LAM_U57_BIT)
 #define X86_CR3_LAM_U48_BIT	(62)
+#define X86_CR3_LAM_U48		BIT_ULL(X86_CR3_LAM_U48_BIT)
 
 #define X86_CR4_VME_BIT		(0)
 #define X86_CR4_VME		BIT(X86_CR4_VME_BIT)
@@ -988,4 +990,14 @@ static inline bool is_lam_sup_enabled(void)
 	return !!(read_cr4() & X86_CR4_LAM_SUP);
 }
 
+static inline bool is_lam_u48_enabled(void)
+{
+	return (read_cr3() & (X86_CR3_LAM_U48 | X86_CR3_LAM_U57)) == X86_CR3_LAM_U48;
+}
+
+static inline bool is_lam_u57_enabled(void)
+{
+	return !!(read_cr3() & X86_CR3_LAM_U57);
+}
+
 #endif
diff --git a/x86/lam.c b/x86/lam.c
index 2f95b6c9..40b8ecdd 100644
--- a/x86/lam.c
+++ b/x86/lam.c
@@ -67,7 +67,14 @@ static bool get_lam_mask(u64 address, u64* lam_mask)
 		return true;
 	}
 
-	/* TODO: Get LAM mask for userspace address. */
+	if(is_lam_u48_enabled()) {
+		*lam_mask = LAM48_MASK;
+		return true;
+	}
+
+	if(is_lam_u57_enabled())
+		return true;
+
 	return false;
 }
 
@@ -88,6 +95,17 @@ static void test_ptr(u64* ptr, bool is_mmio)
 	report(fault != lam_active, "Expected access to tagged address for %s %s LAM to %s",
 	       is_mmio ? "MMIO" : "memory", lam_active ? "with" : "without",
 	       lam_active ? "succeed" : "#GP");
+
+	/*
+	 * This test case is only triggered when LAM_U57 is active and 4-level
+	 * paging is used. For the case, bit[56:47] aren't all 0 triggers #GP.
+	 */
+	if (lam_active && (lam_mask == LAM57_MASK) && !is_la57_enabled()) {
+		ptr = (u64 *)set_la_non_canonical((u64)ptr, LAM48_MASK);
+		fault = test_for_exception(GP_VECTOR, do_mov, ptr);
+		report(fault, "Expected access to non-LAM-canonical address for %s to #GP",
+		       is_mmio ? "MMIO" : "memory");
+	}
 }
 
 /* invlpg with tagged address is same as NOP, no #GP expected. */
@@ -199,6 +217,54 @@ static void test_lam_sup(void)
 			    "use kvm.force_emulation_prefix=1 to enable\n");
 }
 
+static void test_lam_user(void)
+{
+	void* vaddr;
+	int vector;
+	unsigned long cr3 = read_cr3() & ~(X86_CR3_LAM_U48 | X86_CR3_LAM_U57);
+	bool has_lam = this_cpu_has(X86_FEATURE_LAM);
+
+	/*
+	 * The physical address of AREA_NORMAL is within 36 bits, so that using
+	 * identical mapping, the linear address will be considered as user mode
+	 * address from the view of LAM, and the metadata bits are not used as
+	 * address for both LAM48 and LAM57.
+	 */
+	vaddr = alloc_pages_flags(0, AREA_NORMAL);
+	_Static_assert((AREA_NORMAL_PFN & GENMASK(63, 47)) == 0UL,
+			"Identical mapping range check");
+
+	/*
+	 * Note, LAM doesn't have a global control bit to turn on/off LAM
+	 * completely, but purely depends on hardware's CPUID to determine it
+	 * can be enabled or not. That means, when EPT is on, even when KVM
+	 * doesn't expose LAM to guest, the guest can still set LAM control bits
+	 * in CR3 w/o causing problem. This is an unfortunate virtualization
+	 * hole. KVM doesn't choose to intercept CR3 in this case for
+	 * performance.
+	 * Only enable LAM CR3 bits when LAM feature is exposed.
+	 */
+	if (has_lam) {
+		vector = write_cr3_safe(cr3 | X86_CR3_LAM_U48);
+		report(!vector && is_lam_u48_enabled(), "Expected CR3.LAM_U48=1 to succeed");
+	}
+	/*
+	 * Physical memory & MMIO have already been identical mapped in
+	 * setup_mmu().
+	 */
+	test_ptr(vaddr, false);
+	test_ptr(phys_to_virt(IORAM_BASE_PHYS), true);
+
+	if (has_lam) {
+		vector = write_cr3_safe(cr3 | X86_CR3_LAM_U57);
+		report(!vector && is_lam_u57_enabled(), "Expected CR3.LAM_U57=1 to succeed");
+
+		/* If !has_lam, it has been tested above, no need to test again. */
+		test_ptr(vaddr, false);
+		test_ptr(phys_to_virt(IORAM_BASE_PHYS), true);
+	}
+}
+
 int main(int ac, char **av)
 {
 	setup_vm();
@@ -209,6 +275,7 @@ int main(int ac, char **av)
 		report_info("This CPU supports LAM feature\n");
 
 	test_lam_sup();
+	test_lam_user();
 
 	return report_summary();
 }
-- 
2.43.2

[kvm-unit-tests PATCH v7 5/5] x86: Add test case for INVVPID with LAM

[Binbin Wu]

LAM applies to the linear address of INVVPID operand, however,
it doesn't apply to the linear address in the INVVPID descriptor.

The added cases use tagged operand or tagged target invalidation
address to make sure the behaviors are expected when LAM is on.

Also, INVVPID case using tagged operand can be used as the common
test cases for VMX instruction VMExits.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
---
v7:
- Drop the check of X86_FEATURE_LA57 when check LA57. (Sean)
---
 x86/vmx_tests.c | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index 4b161c3c..758ab0d3 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -3233,6 +3233,47 @@ static void invvpid_test_not_in_vmx_operation(void)
 	TEST_ASSERT(!vmx_on());
 }
 
+/* LAM doesn't apply to the linear address inside the descriptor of invvpid */
+static void invvpid_test_lam(void)
+{
+	void *vaddr;
+	struct invvpid_operand *operand;
+	u64 lam_mask;
+	bool fault;
+
+	if (!this_cpu_has(X86_FEATURE_LAM)) {
+		report_skip("LAM is not supported, skip INVVPID with LAM");
+		return;
+	}
+
+	write_cr4(read_cr4() | X86_CR4_LAM_SUP);
+	lam_mask = is_la57_enabled() ? LAM57_MASK : LAM48_MASK;
+
+	vaddr = alloc_vpage();
+	install_page(current_page_table(), virt_to_phys(alloc_page()), vaddr);
+	/*
+	 * Since the stack memory address in KUT doesn't follow kernel address
+	 * space partition rule, reuse the memory address for descriptor and
+	 * the target address in the descriptor of invvpid.
+	 */
+	operand = (struct invvpid_operand *)vaddr;
+	operand->vpid = 0xffff;
+	operand->gla = (u64)vaddr;
+	operand = (struct invvpid_operand *)set_la_non_canonical((u64)operand,
+								 lam_mask);
+	fault = test_for_exception(GP_VECTOR, ds_invvpid, operand);
+	report(!fault, "Expected INVVPID with tagged operand when LAM is enabled to succeed");
+
+	/*
+	 * Verify that LAM doesn't apply to the address inside the descriptor
+	 * even when LAM is enabled. i.e., the address in the descriptor should
+	 * be canonical.
+	 */
+	try_invvpid(INVVPID_ADDR, 0xffff, (u64)operand);
+
+	write_cr4(read_cr4() & ~X86_CR4_LAM_SUP);
+}
+
 /*
  * This does not test real-address mode, virtual-8086 mode, protected mode,
  * or CPL > 0.
@@ -3282,8 +3323,10 @@ static void invvpid_test(void)
 	/*
 	 * The gla operand is only validated for single-address INVVPID.
 	 */
-	if (types & (1u << INVVPID_ADDR))
+	if (types & (1u << INVVPID_ADDR)) {
 		try_invvpid(INVVPID_ADDR, 0xffff, NONCANONICAL);
+		invvpid_test_lam();
+	}
 
 	invvpid_test_gp();
 	invvpid_test_ss();
-- 
2.43.2

Re: [kvm-unit-tests PATCH v7 0/5] x86: Add test cases for LAM

[Binbin Wu]

Ping...

On 7/1/2024 3:30 PM, Binbin Wu wrote:
> Intel Linear-address masking (LAM) [1], modifies the checking that is applied to
> *64-bit* linear addresses, allowing software to use of the untranslated address
> bits for metadata.
>
> The patch series add test cases for KVM LAM:
>
> Patch 1 moves struct invpcid_desc to header file for new test cases.
> Patch 2 makes change to allow setting of CR3 LAM bits in vmlaunch tests.
> Patch 3~5 add test cases for LAM supervisor mode and user mode, including:
> - For supervisor mode
>    CR4.LAM_SUP toggle
>    Memory/MMIO access with tagged pointer
>    INVLPG
>    INVPCID
>    INVVPID (also used to cover VMX instruction VMExit path)
> - For user mode
>    CR3 LAM bits toggle
>    Memory/MMIO access with tagged pointer
>
> [1] Intel ISE https://cdrdv2.intel.com/v1/dl/getContent/671368
>      Chapter Linear Address Masking (LAM)
> ---
> Changelog:
> v7
> - Move struct invpcid_desc to header file instead of defining a new copy in lam.c.
> - Rename is_la57()/lam_sup_active() to is_la57_enabled()/is_lam_sup_enabled(),
>    and move them to processor.h (Sean)
> - Drop cr4_set_lam_sup()/cr4_clear_lam_sup() and use write_cr4_safe() instead. (Sean)
> - Add get_lam_mask() to get lam status based on the address and vCPU state. (Sean)
> - Drop the wrappers for INVLPG since INVLPG never faults. (Sean)
> - Drop the wrapper for INVPCID and use invpcid_safe() instead. (Sean)
> - Drop the check for X86_FEATURE_PCID. (Sean)
> - Rename lam_{u48,u57}_active() to is_lam_{u48,u57}_enabled(), and move them to
>    processor.h (Sean)
> - Test LAM userspace address in kernel mode directly to simplify the interface
>    of test_ptr() since LAM identifies a address as kernel or user only based on
>    the address itself.
> - Add comments about the virtualization hole of CR3 LAM bits.
> - Drop the check of X86_FEATURE_LA57 when check LA57. (Sean)
>
> v6
> - https://lore.kernel.org/kvm/20240122085354.9510-1-binbin.wu@linux.intel.com/
>
> v5
> - https://lore.kernel.org/kvm/20230530024356.24870-1-binbin.wu@linux.intel.com/
>
> Binbin Wu (4):
>    x86: Move struct invpcid_desc to processor.h
>    x86: Allow setting of CR3 LAM bits if LAM supported
>    x86: Add test cases for LAM_{U48,U57}
>    x86: Add test case for INVVPID with LAM
>
> Robert Hoo (1):
>    x86: Add test case for LAM_SUP
>
>   lib/x86/processor.h |  41 +++++++
>   x86/Makefile.x86_64 |   1 +
>   x86/lam.c           | 281 ++++++++++++++++++++++++++++++++++++++++++++
>   x86/pcid.c          |   6 -
>   x86/unittests.cfg   |  10 ++
>   x86/vmx_tests.c     |  51 +++++++-
>   6 files changed, 382 insertions(+), 8 deletions(-)
>   create mode 100644 x86/lam.c
>
>
> base-commit: d301d0187f5db09531a1c2c7608997cc3b0a5c7d

Re: [kvm-unit-tests PATCH v7 0/5] x86: Add test cases for LAM

[Sean Christopherson]

On Mon, 01 Jul 2024 15:30:05 +0800, Binbin Wu wrote:
> Intel Linear-address masking (LAM) [1], modifies the checking that is applied to
> *64-bit* linear addresses, allowing software to use of the untranslated address
> bits for metadata.
> 
> The patch series add test cases for KVM LAM:
> 
> Patch 1 moves struct invpcid_desc to header file for new test cases.
> Patch 2 makes change to allow setting of CR3 LAM bits in vmlaunch tests.
> Patch 3~5 add test cases for LAM supervisor mode and user mode, including:
> - For supervisor mode
>   CR4.LAM_SUP toggle
>   Memory/MMIO access with tagged pointer
>   INVLPG
>   INVPCID
>   INVVPID (also used to cover VMX instruction VMExit path)
> - For user mode
>   CR3 LAM bits toggle
>   Memory/MMIO access with tagged pointer
> 
> [...]

Applied to kvm-x86 next (and now pulled by Paolo), except for patch 1, as a
previous commit from a different series moved the INVPCID helpers to common code.

Note, I deliberately dropped the "packed" addition.  The storage size may be
implementation specific, but the layout is not.  If the compiler does something
bizarre, e.g. allocates 32 instead of 16 bytes for the union, INVPCID will still
work correctly as the CPU won't be aware that the compiler reserved 16 extra
bytes.

[1/5] x86: Move struct invpcid_desc to processor.h
      (no commit info)
[2/5] x86: Allow setting of CR3 LAM bits if LAM supported
      https://github.com/kvm-x86/kvm-unit-tests/commit/0a6b8b7d601a
[3/5] x86: Add test case for LAM_SUP
      https://github.com/kvm-x86/kvm-unit-tests/commit/14520f8e9709
[4/5] x86: Add test cases for LAM_{U48,U57}
      https://github.com/kvm-x86/kvm-unit-tests/commit/0164d7595c85
[5/5] x86: Add test case for INVVPID with LAM
      https://github.com/kvm-x86/kvm-unit-tests/commit/a33a3ac8db56

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next