[PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[Kirill A. Shutemov]

Linear Address Space Separation (LASS) is a security feature that intends to
prevent malicious virtual address space accesses across user/kernel mode.

Such mode based access protection already exists today with paging and features
such as SMEP and SMAP. However, to enforce these protections, the processor
must traverse the paging structures in memory.  Malicious software can use
timing information resulting from this traversal to determine details about the
paging structures, and these details may also be used to determine the layout
of the kernel memory.

The LASS mechanism provides the same mode-based protections as paging but
without traversing the paging structures. Because the protections enforced by
LASS are applied before paging, software will not be able to derive
paging-based timing information from the various caching structures such as the
TLBs, mid-level caches, page walker, data caches, etc. LASS can avoid probing
using double page faults, TLB flush and reload, and SW prefetch instructions.
See [2], [3] and [4] for some research on the related attack vectors.

In addition, LASS prevents an attack vector described in a Spectre LAM (SLAM)
whitepaper [7].

LASS enforcement relies on the typical kernel implementation to divide the
64-bit virtual address space into two halves:
  Addr[63]=0 -> User address space
  Addr[63]=1 -> Kernel address space
Any data access or code execution across address spaces typically results in a
#GP fault.

Kernel accesses usually only happen to the kernel address space. However, there
are valid reasons for kernel to access memory in the user half. For these cases
(such as text poking and EFI runtime accesses), the kernel can temporarily
suspend the enforcement of LASS by toggling SMAP (Supervisor Mode Access
Prevention) using the stac()/clac() instructions and in one instance a downright
disabling LASS for an EFI runtime call.

User space cannot access any kernel address while LASS is enabled.
Unfortunately, legacy vsyscall functions are located in the address range
0xffffffffff600000 - 0xffffffffff601000 and emulated in kernel.  To avoid
breaking user applications when LASS is enabled, extend the vsyscall emulation
in execute (XONLY) mode to the #GP fault handler.

In contrast, the vsyscall EMULATE mode is deprecated and not expected to be
used by anyone.  Supporting EMULATE mode with LASS would need complex
instruction decoding in the #GP fault handler and is probably not worth the
hassle. Disable LASS in this rare case when someone absolutely needs and
enables vsyscall=emulate via the command line.

Changes from v5[9]:
- Report LASS violation as NULL pointer dereference if the address is in the
  first page frame;
- Provide helpful error message on #SS due to LASS violation;
- Fold patch for vsyscall=emulate documentation into patch
  that disables LASS with vsyscall=emulate;
- Rewrite __inline_memeset() and __inline_memcpy();
- Adjust comments and commit messages;

Changes from v4[8]:
- Added PeterZ's Originally-by and SoB to 2/16
- Added lass_clac()/lass_stac() to differentiate from SMAP necessitated
  clac()/stac() and to be NOPs on CPUs that don't support LASS
- Moved LASS enabling patch to the end to avoid rendering machines
  unbootable between until the patch that disables LASS around EFI
  initialization
- Reverted Pawan's LAM disabling commit

Changes from v3[6]:
- Made LAM dependent on LASS
- Moved EFI runtime initialization to x86 side of things
- Suspended LASS validation around EFI set_virtual_address_map call
- Added a message for the case of kernel side LASS violation
- Moved inline memset/memcpy versions to the common string.h

Changes from v2[5]:
- Added myself to the SoB chain

Changes from v1[1]:
- Emulate vsyscall violations in execute mode in the #GP fault handler
- Use inline memcpy and memset while patching alternatives
- Remove CONFIG_X86_LASS
- Make LASS depend on SMAP
- Dropped the minimal KVM enabling patch


[1] https://lore.kernel.org/lkml/20230110055204.3227669-1-yian.chen@intel.com/
[2] “Practical Timing Side Channel Attacks against Kernel Space ASLR”,
https://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf
[3] “Prefetch Side-Channel Attacks: Bypassing SMAP and Kernel ASLR”, http://doi.acm.org/10.1145/2976749.2978356
[4] “Harmful prefetch on Intel”, https://ioactive.com/harmful-prefetch-on-intel/ (H/T Anders)
[5] https://lore.kernel.org/all/20230530114247.21821-1-alexander.shishkin@linux.intel.com/
[6] https://lore.kernel.org/all/20230609183632.48706-1-alexander.shishkin@linux.intel.com/
[7] https://download.vusec.net/papers/slam_sp24.pdf
[8] https://lore.kernel.org/all/20240710160655.3402786-1-alexander.shishkin@linux.intel.com/
[9] https://lore.kernel.org/all/20241028160917.1380714-1-alexander.shishkin@linux.intel.com

Alexander Shishkin (6):
  x86/efi: Move runtime service initialization to arch/x86
  x86/cpu: Defer CR pinning setup until after EFI initialization
  efi: Disable LASS around set_virtual_address_map() EFI call
  x86/traps: Communicate a LASS violation in #GP message
  x86/traps: Handle LASS thrown #SS
  x86/cpu: Make LAM depend on LASS

Kirill A. Shutemov (2):
  x86/asm: Introduce inline memcpy and memset
  x86: Re-enable Linear Address Masking

Sohil Mehta (7):
  x86/cpu: Enumerate the LASS feature bits
  x86/alternatives: Disable LASS when patching kernel alternatives
  x86/vsyscall: Reorganize the #PF emulation code
  x86/traps: Consolidate user fixups in exc_general_protection()
  x86/vsyscall: Add vsyscall emulation for #GP
  x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE
  x86/cpu: Enable LASS during CPU initialization

Yian Chen (1):
  x86/cpu: Set LASS CR4 bit as pinning sensitive

 .../admin-guide/kernel-parameters.txt         |  4 +-
 arch/x86/Kconfig                              |  1 -
 arch/x86/Kconfig.cpufeatures                  |  4 ++
 arch/x86/entry/vsyscall/vsyscall_64.c         | 61 +++++++++++-----
 arch/x86/include/asm/cpufeatures.h            |  1 +
 arch/x86/include/asm/smap.h                   | 22 +++++-
 arch/x86/include/asm/string.h                 | 45 ++++++++++++
 arch/x86/include/asm/uaccess_64.h             | 37 +++-------
 arch/x86/include/asm/vsyscall.h               | 14 ++--
 arch/x86/include/uapi/asm/processor-flags.h   |  2 +
 arch/x86/kernel/alternative.c                 | 12 +++-
 arch/x86/kernel/cpu/common.c                  | 35 ++++++++--
 arch/x86/kernel/cpu/cpuid-deps.c              |  2 +
 arch/x86/kernel/traps.c                       | 70 ++++++++++++++-----
 arch/x86/lib/clear_page_64.S                  | 10 ++-
 arch/x86/mm/fault.c                           |  2 +-
 arch/x86/platform/efi/efi.c                   | 13 ++++
 init/main.c                                   |  5 --
 tools/arch/x86/include/asm/cpufeatures.h      |  1 +
 19 files changed, 253 insertions(+), 88 deletions(-)

-- 
2.47.2

[PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

Linear Address Space Separation (LASS) is a security feature that
intends to prevent malicious virtual address space accesses across
user/kernel mode.

Such mode based access protection already exists today with paging and
features such as SMEP and SMAP. However, to enforce these protections,
the processor must traverse the paging structures in memory.  Malicious
software can use timing information resulting from this traversal to
determine details about the paging structures, and these details may
also be used to determine the layout of the kernel memory.

The LASS mechanism provides the same mode-based protections as paging
but without traversing the paging structures. Because the protections
enforced by LASS are applied before paging, software will not be able to
derive paging-based timing information from the various caching
structures such as the TLBs, mid-level caches, page walker, data caches,
etc.

LASS enforcement relies on the typical kernel implementation to divide
the 64-bit virtual address space into two halves:
  Addr[63]=0 -> User address space
  Addr[63]=1 -> Kernel address space

Any data access or code execution across address spaces typically
results in a #GP fault.

The LASS enforcement for kernel data access is dependent on CR4.SMAP
being set. The enforcement can be disabled by toggling the RFLAGS.AC bit
similar to SMAP.

Define the CPU feature bits to enumerate this feature and include
feature dependencies to reflect the same.

Co-developed-by: Yian Chen <yian.chen@intel.com>
Signed-off-by: Yian Chen <yian.chen@intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/Kconfig.cpufeatures                |  4 ++++
 arch/x86/include/asm/cpufeatures.h          |  1 +
 arch/x86/include/asm/smap.h                 | 22 +++++++++++++++++++--
 arch/x86/include/uapi/asm/processor-flags.h |  2 ++
 arch/x86/kernel/cpu/cpuid-deps.c            |  1 +
 tools/arch/x86/include/asm/cpufeatures.h    |  1 +
 6 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
index 250c10627ab3..9574c198fc08 100644
--- a/arch/x86/Kconfig.cpufeatures
+++ b/arch/x86/Kconfig.cpufeatures
@@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
 	def_bool y
 	depends on !X86_64
 
+config X86_DISABLED_FEATURE_LASS
+	def_bool y
+	depends on !X86_64
+
 config X86_DISABLED_FEATURE_PKU
 	def_bool y
 	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index ee176236c2be..4473a6f7800b 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -313,6 +313,7 @@
 #define X86_FEATURE_SM4			(12*32+ 2) /* SM4 instructions */
 #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* "avx_vnni" AVX VNNI instructions */
 #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT16 instructions */
+#define X86_FEATURE_LASS		(12*32+ 6) /* "lass" Linear Address Space Separation */
 #define X86_FEATURE_CMPCCXADD           (12*32+ 7) /* CMPccXADD instructions */
 #define X86_FEATURE_ARCH_PERFMON_EXT	(12*32+ 8) /* Intel Architectural PerfMon Extension */
 #define X86_FEATURE_FZRM		(12*32+10) /* Fast zero-length REP MOVSB */
diff --git a/arch/x86/include/asm/smap.h b/arch/x86/include/asm/smap.h
index 4f84d421d1cf..1f36c5b26949 100644
--- a/arch/x86/include/asm/smap.h
+++ b/arch/x86/include/asm/smap.h
@@ -23,18 +23,36 @@
 
 #else /* __ASSEMBLER__ */
 
+/*
+ * The CLAC/STAC instructions toggle enforcement of X86_FEATURE_SMAP.
+ *
+ * X86_FEATURE_LASS requires flipping the AC flag when accessing the lower half
+ * of the virtual address space, regardless of the _PAGE_BIT_USER bit in the
+ * page tables. lass_clac/stac() should be used for these cases.
+ *
+ * Note: a barrier is implicit in alternative().
+ */
+
 static __always_inline void clac(void)
 {
-	/* Note: a barrier is implicit in alternative() */
 	alternative("", "clac", X86_FEATURE_SMAP);
 }
 
 static __always_inline void stac(void)
 {
-	/* Note: a barrier is implicit in alternative() */
 	alternative("", "stac", X86_FEATURE_SMAP);
 }
 
+static __always_inline void lass_clac(void)
+{
+	alternative("", "clac", X86_FEATURE_LASS);
+}
+
+static __always_inline void lass_stac(void)
+{
+	alternative("", "stac", X86_FEATURE_LASS);
+}
+
 static __always_inline unsigned long smap_save(void)
 {
 	unsigned long flags;
diff --git a/arch/x86/include/uapi/asm/processor-flags.h b/arch/x86/include/uapi/asm/processor-flags.h
index f1a4adc78272..81d0c8bf1137 100644
--- a/arch/x86/include/uapi/asm/processor-flags.h
+++ b/arch/x86/include/uapi/asm/processor-flags.h
@@ -136,6 +136,8 @@
 #define X86_CR4_PKE		_BITUL(X86_CR4_PKE_BIT)
 #define X86_CR4_CET_BIT		23 /* enable Control-flow Enforcement Technology */
 #define X86_CR4_CET		_BITUL(X86_CR4_CET_BIT)
+#define X86_CR4_LASS_BIT	27 /* enable Linear Address Space Separation support */
+#define X86_CR4_LASS		_BITUL(X86_CR4_LASS_BIT)
 #define X86_CR4_LAM_SUP_BIT	28 /* LAM for supervisor pointers */
 #define X86_CR4_LAM_SUP		_BITUL(X86_CR4_LAM_SUP_BIT)
 
diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c
index 46efcbd6afa4..98d0cdd82574 100644
--- a/arch/x86/kernel/cpu/cpuid-deps.c
+++ b/arch/x86/kernel/cpu/cpuid-deps.c
@@ -89,6 +89,7 @@ static const struct cpuid_dep cpuid_deps[] = {
 	{ X86_FEATURE_SHSTK,			X86_FEATURE_XSAVES    },
 	{ X86_FEATURE_FRED,			X86_FEATURE_LKGS      },
 	{ X86_FEATURE_SPEC_CTRL_SSBD,		X86_FEATURE_SPEC_CTRL },
+	{ X86_FEATURE_LASS,			X86_FEATURE_SMAP      },
 	{}
 };
 
diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h
index e02be2962a01..20df2ce5d339 100644
--- a/tools/arch/x86/include/asm/cpufeatures.h
+++ b/tools/arch/x86/include/asm/cpufeatures.h
@@ -313,6 +313,7 @@
 #define X86_FEATURE_SM4			(12*32+ 2) /* SM4 instructions */
 #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* "avx_vnni" AVX VNNI instructions */
 #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT16 instructions */
+#define X86_FEATURE_LASS		(12*32+ 6) /* "lass" Linear Address Space Separation */
 #define X86_FEATURE_CMPCCXADD           (12*32+ 7) /* CMPccXADD instructions */
 #define X86_FEATURE_ARCH_PERFMON_EXT	(12*32+ 8) /* Intel Architectural PerfMon Extension */
 #define X86_FEATURE_FZRM		(12*32+10) /* Fast zero-length REP MOVSB */
-- 
2.47.2

[PATCHv6 02/16] x86/asm: Introduce inline memcpy and memset

[Kirill A. Shutemov]

Extract memcpy and memset functions from copy_user_generic() and
__clear_user().

They can be used as inline memcpy and memset instead of the GCC builtins
whenever necessary. LASS requires them to handle text_poke.

Originally-by: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/all/20241029184840.GJ14555@noisy.programming.kicks-ass.net/
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/include/asm/string.h     | 45 +++++++++++++++++++++++++++++++
 arch/x86/include/asm/uaccess_64.h | 37 +++++++------------------
 arch/x86/lib/clear_page_64.S      | 10 +++++--
 3 files changed, 62 insertions(+), 30 deletions(-)

diff --git a/arch/x86/include/asm/string.h b/arch/x86/include/asm/string.h
index c3c2c1914d65..d75e965d1ce4 100644
--- a/arch/x86/include/asm/string.h
+++ b/arch/x86/include/asm/string.h
@@ -1,6 +1,51 @@
 /* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_STRING_H
+#define _ASM_X86_STRING_H
+
+#include <asm/asm.h>
+#include <asm/alternative.h>
+
 #ifdef CONFIG_X86_32
 # include <asm/string_32.h>
 #else
 # include <asm/string_64.h>
 #endif
+
+#ifdef CONFIG_X86_64
+#define ALT_64(orig, alt, feat) ALTERNATIVE(orig, alt, feat)
+#else
+#define ALT_64(orig, alt, feat) orig
+#endif
+
+static __always_inline void *__inline_memcpy(void *to, const void *from, size_t len)
+{
+	void *ret = to;
+
+	asm volatile("1:\n\t"
+		     ALT_64("rep movsb",
+			    "call rep_movs_alternative", ALT_NOT(X86_FEATURE_FSRM))
+		     "2:\n\t"
+		     _ASM_EXTABLE_UA(1b, 2b)
+		     :"+c" (len), "+D" (to), "+S" (from), ASM_CALL_CONSTRAINT
+		     : : "memory", _ASM_AX);
+
+	return ret + len;
+}
+
+static __always_inline void *__inline_memset(void *addr, int v, size_t len)
+{
+	void *ret = addr;
+
+	asm volatile("1:\n\t"
+		     ALT_64("rep stosb",
+			    "call rep_stos_alternative", ALT_NOT(X86_FEATURE_FSRM))
+		     "2:\n\t"
+		     _ASM_EXTABLE_UA(1b, 2b)
+		     : "+c" (len), "+D" (addr), ASM_CALL_CONSTRAINT
+		     : "a" ((uint8_t)v)
+		     : "memory", _ASM_SI);
+
+	return ret + len;
+}
+
+#endif /* _ASM_X86_STRING_H */
diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h
index c8a5ae35c871..30cc318eb3ed 100644
--- a/arch/x86/include/asm/uaccess_64.h
+++ b/arch/x86/include/asm/uaccess_64.h
@@ -13,6 +13,7 @@
 #include <asm/page.h>
 #include <asm/percpu.h>
 #include <asm/runtime-const.h>
+#include <asm/string.h>
 
 /*
  * Virtual variable: there's no actual backing store for this,
@@ -118,21 +119,12 @@ rep_movs_alternative(void *to, const void *from, unsigned len);
 static __always_inline __must_check unsigned long
 copy_user_generic(void *to, const void *from, unsigned long len)
 {
+	void *ret;
+
 	stac();
-	/*
-	 * If CPU has FSRM feature, use 'rep movs'.
-	 * Otherwise, use rep_movs_alternative.
-	 */
-	asm volatile(
-		"1:\n\t"
-		ALTERNATIVE("rep movsb",
-			    "call rep_movs_alternative", ALT_NOT(X86_FEATURE_FSRM))
-		"2:\n"
-		_ASM_EXTABLE_UA(1b, 2b)
-		:"+c" (len), "+D" (to), "+S" (from), ASM_CALL_CONSTRAINT
-		: : "memory", "rax");
+	ret = __inline_memcpy(to, from, len);
 	clac();
-	return len;
+	return ret - to;
 }
 
 static __always_inline __must_check unsigned long
@@ -178,25 +170,14 @@ rep_stos_alternative(void __user *addr, unsigned long len);
 
 static __always_inline __must_check unsigned long __clear_user(void __user *addr, unsigned long size)
 {
+	void *ret;
+
 	might_fault();
 	stac();
-
-	/*
-	 * No memory constraint because it doesn't change any memory gcc
-	 * knows about.
-	 */
-	asm volatile(
-		"1:\n\t"
-		ALTERNATIVE("rep stosb",
-			    "call rep_stos_alternative", ALT_NOT(X86_FEATURE_FSRS))
-		"2:\n"
-	       _ASM_EXTABLE_UA(1b, 2b)
-	       : "+c" (size), "+D" (addr), ASM_CALL_CONSTRAINT
-	       : "a" (0));
-
+	ret = __inline_memset(addr, 0, size);
 	clac();
 
-	return size;
+	return ret - addr;
 }
 
 static __always_inline unsigned long clear_user(void __user *to, unsigned long n)
diff --git a/arch/x86/lib/clear_page_64.S b/arch/x86/lib/clear_page_64.S
index a508e4a8c66a..ca94828def62 100644
--- a/arch/x86/lib/clear_page_64.S
+++ b/arch/x86/lib/clear_page_64.S
@@ -55,17 +55,23 @@ SYM_FUNC_END(clear_page_erms)
 EXPORT_SYMBOL_GPL(clear_page_erms)
 
 /*
- * Default clear user-space.
+ * Default memset.
  * Input:
  * rdi destination
+ * rsi scratch
  * rcx count
- * rax is zero
+ * al is value
  *
  * Output:
  * rcx: uncleared bytes or 0 if successful.
  */
 SYM_FUNC_START(rep_stos_alternative)
 	ANNOTATE_NOENDBR
+
+	movzbq %al, %rsi
+	movabs $0x0101010101010101, %rax
+	mulq %rsi
+
 	cmpq $64,%rcx
 	jae .Lunrolled
 
-- 
2.47.2

[PATCHv6 03/16] x86/alternatives: Disable LASS when patching kernel alternatives

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

For patching, the kernel initializes a temporary mm area in the lower
half of the address range. See commit 4fc19708b165 ("x86/alternatives:
Initialize temporary mm for patching").

Disable LASS enforcement during patching using the stac()/clac()
instructions to avoid triggering a #GP fault.

The objtool warns due to a call to a non-allowed function that exists
outside of the stac/clac guard, or references to any function with a
dynamic function pointer inside the guard. See the Objtool warnings
section #9 in the document tools/objtool/Documentation/objtool.txt.

Considering that patching is usually small, replace the memcpy and
memset functions in the text poking functions with their inline versions
respectively.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/alternative.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 9ae80fa904a2..0b9f41ed6af7 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -2447,16 +2447,24 @@ void __init_or_module text_poke_early(void *addr, const void *opcode,
 __ro_after_init struct mm_struct *text_poke_mm;
 __ro_after_init unsigned long text_poke_mm_addr;
 
+/*
+ * poking_init() initializes the text poking address from the lower half of the
+ * address space. Relax LASS enforcement when accessing the poking address.
+ */
 static void text_poke_memcpy(void *dst, const void *src, size_t len)
 {
-	memcpy(dst, src, len);
+	lass_stac();
+	__inline_memcpy(dst, src, len);
+	lass_clac();
 }
 
 static void text_poke_memset(void *dst, const void *src, size_t len)
 {
 	int c = *(const int *)src;
 
-	memset(dst, c, len);
+	lass_stac();
+	__inline_memset(dst, c, len);
+	lass_clac();
 }
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
-- 
2.47.2

[PATCHv6 04/16] x86/efi: Move runtime service initialization to arch/x86

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

The EFI call in start_kernel() is guarded by #ifdef CONFIG_X86. Move
the thing to the arch_cpu_finalize_init() path on x86 and get rid of
the #ifdef in start_kernel().

No functional change intended.

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Suggested-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/cpu/common.c | 7 +++++++
 init/main.c                  | 5 -----
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 8feb8fd2957a..4f430be285de 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -26,6 +26,7 @@
 #include <linux/pgtable.h>
 #include <linux/stackprotector.h>
 #include <linux/utsname.h>
+#include <linux/efi.h>
 
 #include <asm/alternative.h>
 #include <asm/cmdline.h>
@@ -2529,6 +2530,12 @@ void __init arch_cpu_finalize_init(void)
 	fpu__init_system();
 	fpu__init_cpu();
 
+	/*
+	 * This needs to follow the FPU initializtion, since EFI depends on it.
+	 */
+	if (efi_enabled(EFI_RUNTIME_SERVICES))
+		efi_enter_virtual_mode();
+
 	/*
 	 * Ensure that access to the per CPU representation has the initial
 	 * boot CPU configuration.
diff --git a/init/main.c b/init/main.c
index 225a58279acd..f9f401b6fdfb 100644
--- a/init/main.c
+++ b/init/main.c
@@ -53,7 +53,6 @@
 #include <linux/cpuset.h>
 #include <linux/memcontrol.h>
 #include <linux/cgroup.h>
-#include <linux/efi.h>
 #include <linux/tick.h>
 #include <linux/sched/isolation.h>
 #include <linux/interrupt.h>
@@ -1068,10 +1067,6 @@ void start_kernel(void)
 
 	pid_idr_init();
 	anon_vma_init();
-#ifdef CONFIG_X86
-	if (efi_enabled(EFI_RUNTIME_SERVICES))
-		efi_enter_virtual_mode();
-#endif
 	thread_stack_cache_init();
 	cred_init();
 	fork_init();
-- 
2.47.2

[PATCHv6 05/16] x86/cpu: Defer CR pinning setup until after EFI initialization

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

In order to map the EFI runtime services, set_virtual_address_map()
needs to be called, which resides in the lower half of the address
space. This means that LASS needs to be temporarily disabled around
this call. This can only be done before the CR pinning is set up.

Move CR pinning setup behind the EFI initialization.

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Suggested-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/cpu/common.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 4f430be285de..1a779e302872 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2081,7 +2081,6 @@ static __init void identify_boot_cpu(void)
 	enable_sep_cpu();
 #endif
 	cpu_detect_tlb(&boot_cpu_data);
-	setup_cr_pinning();
 
 	tsx_init();
 	tdx_init();
@@ -2532,10 +2531,21 @@ void __init arch_cpu_finalize_init(void)
 
 	/*
 	 * This needs to follow the FPU initializtion, since EFI depends on it.
+	 *
+	 * It also needs to precede the CR pinning setup, because the CR4.LASS
+	 * bit has to be cleared temporarily in order to execute the
+	 * set_virtual_address_map() EFI call, which resides in lower addresses
+	 * and would trip LASS if enabled.
+	 *
+	 * Wrapping efi_enter_virtual_mode() into lass_stac()/clac() is not
+	 * enough because AC flag gates data accesses, but not instruction
+	 * fetch. Clearing the CR4 bit is required.
 	 */
 	if (efi_enabled(EFI_RUNTIME_SERVICES))
 		efi_enter_virtual_mode();
 
+	setup_cr_pinning();
+
 	/*
 	 * Ensure that access to the per CPU representation has the initial
 	 * boot CPU configuration.
-- 
2.47.2

[PATCHv6 06/16] efi: Disable LASS around set_virtual_address_map() EFI call

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

Of all the EFI runtime services, set_virtual_address_map() is the only
one that is called at its lower mapping, which LASS prohibits regardless
of EFLAGS.AC setting. The only way to allow this to happen is to disable
LASS in the CR4 register.

Disable LASS around this low address EFI call.

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/platform/efi/efi.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 463b784499a8..94c335229697 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -825,11 +825,24 @@ static void __init __efi_enter_virtual_mode(void)
 
 	efi_sync_low_kernel_mappings();
 
+	/*
+	 * set_virtual_address_map is the only service located at lower
+	 * addresses, so we have to temporarily disable LASS around it.
+	 * Note that clearing EFLAGS.AC is not enough for this, the whole
+	 * LASS needs to be disabled.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_LASS))
+		cr4_clear_bits(X86_CR4_LASS);
+
 	status = efi_set_virtual_address_map(efi.memmap.desc_size * count,
 					     efi.memmap.desc_size,
 					     efi.memmap.desc_version,
 					     (efi_memory_desc_t *)pa,
 					     efi_systab_phys);
+
+	if (cpu_feature_enabled(X86_FEATURE_LASS))
+		cr4_set_bits(X86_CR4_LASS);
+
 	if (status != EFI_SUCCESS) {
 		pr_err("Unable to switch EFI into virtual mode (status=%lx)!\n",
 		       status);
-- 
2.47.2

[PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

Separate out the actual vsyscall emulation from the page fault specific
handling in preparation for the upcoming #GP fault emulation.

No functional change intended.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/entry/vsyscall/vsyscall_64.c | 42 +++++++++++++++------------
 arch/x86/include/asm/vsyscall.h       |  8 ++---
 arch/x86/mm/fault.c                   |  2 +-
 3 files changed, 29 insertions(+), 23 deletions(-)

diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index c9103a6fa06e..f18b08c03553 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -112,30 +112,13 @@ static bool write_ok_or_segv(unsigned long ptr, size_t size)
 	}
 }
 
-bool emulate_vsyscall(unsigned long error_code,
-		      struct pt_regs *regs, unsigned long address)
+static bool __emulate_vsyscall(struct pt_regs *regs, unsigned long address)
 {
 	unsigned long caller;
 	int vsyscall_nr, syscall_nr, tmp;
 	long ret;
 	unsigned long orig_dx;
 
-	/* Write faults or kernel-privilege faults never get fixed up. */
-	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
-		return false;
-
-	if (!(error_code & X86_PF_INSTR)) {
-		/* Failed vsyscall read */
-		if (vsyscall_mode == EMULATE)
-			return false;
-
-		/*
-		 * User code tried and failed to read the vsyscall page.
-		 */
-		warn_bad_vsyscall(KERN_INFO, regs, "vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround");
-		return false;
-	}
-
 	/*
 	 * No point in checking CS -- the only way to get here is a user mode
 	 * trap to a high address, which means that we're in 64-bit user code.
@@ -270,6 +253,29 @@ bool emulate_vsyscall(unsigned long error_code,
 	return true;
 }
 
+bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,
+			 unsigned long address)
+{
+	/* Write faults or kernel-privilege faults never get fixed up. */
+	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
+		return false;
+
+	if (!(error_code & X86_PF_INSTR)) {
+		/* Failed vsyscall read */
+		if (vsyscall_mode == EMULATE)
+			return false;
+
+		/*
+		 * User code tried and failed to read the vsyscall page.
+		 */
+		warn_bad_vsyscall(KERN_INFO, regs,
+				  "vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround");
+		return false;
+	}
+
+	return __emulate_vsyscall(regs, address);
+}
+
 /*
  * A pseudo VMA to allow ptrace access for the vsyscall page.  This only
  * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
diff --git a/arch/x86/include/asm/vsyscall.h b/arch/x86/include/asm/vsyscall.h
index 472f0263dbc6..214977f4fa11 100644
--- a/arch/x86/include/asm/vsyscall.h
+++ b/arch/x86/include/asm/vsyscall.h
@@ -14,12 +14,12 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);
  * Called on instruction fetch fault in vsyscall page.
  * Returns true if handled.
  */
-extern bool emulate_vsyscall(unsigned long error_code,
-			     struct pt_regs *regs, unsigned long address);
+extern bool emulate_vsyscall_pf(unsigned long error_code,
+				struct pt_regs *regs, unsigned long address);
 #else
 static inline void map_vsyscall(void) {}
-static inline bool emulate_vsyscall(unsigned long error_code,
-				    struct pt_regs *regs, unsigned long address)
+static inline bool emulate_vsyscall_pf(unsigned long error_code,
+				       struct pt_regs *regs, unsigned long address)
 {
 	return false;
 }
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 998bd807fc7b..fbcc2da75fd6 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1316,7 +1316,7 @@ void do_user_addr_fault(struct pt_regs *regs,
 	 * to consider the PF_PK bit.
 	 */
 	if (is_vsyscall_vaddr(address)) {
-		if (emulate_vsyscall(error_code, regs, address))
+		if (emulate_vsyscall_pf(error_code, regs, address))
 			return;
 	}
 #endif
-- 
2.47.2

[PATCHv6 08/16] x86/traps: Consolidate user fixups in exc_general_protection()

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

Move the UMIP exception fixup along with the other user mode fixups,
that is, under the common "if (user_mode(regs))" condition where the
rest of the fixups reside.

No functional change intended.

Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/traps.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index c5c897a86418..3eb2ef46f733 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -800,11 +800,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 
 	cond_local_irq_enable(regs);
 
-	if (static_cpu_has(X86_FEATURE_UMIP)) {
-		if (user_mode(regs) && fixup_umip_exception(regs))
-			goto exit;
-	}
-
 	if (v8086_mode(regs)) {
 		local_irq_enable();
 		handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code);
@@ -819,6 +814,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 		if (fixup_vdso_exception(regs, X86_TRAP_GP, error_code, 0))
 			goto exit;
 
+		if (cpu_feature_enabled(X86_FEATURE_UMIP) && fixup_umip_exception(regs))
+			goto exit;
+
 		gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc);
 		goto exit;
 	}
-- 
2.47.2

[PATCHv6 09/16] x86/vsyscall: Add vsyscall emulation for #GP

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

The legacy vsyscall page is mapped at a fixed address in the kernel
address range 0xffffffffff600000-0xffffffffff601000. Prior to LASS being
introduced, a legacy vsyscall page access from userspace would always
generate a page fault. The kernel emulates the execute (XONLY) accesses
in the page fault handler and returns back to userspace with the
appropriate register values.

Since LASS intercepts these accesses before the paging structures are
traversed it generates a general protection fault instead of a page
fault. The #GP fault doesn't provide much information in terms of the
error code. So, use the faulting RIP which is preserved in the user
registers to emulate the vsyscall access without going through complex
instruction decoding.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/entry/vsyscall/vsyscall_64.c | 11 ++++++++++-
 arch/x86/include/asm/vsyscall.h       |  6 ++++++
 arch/x86/kernel/traps.c               |  4 ++++
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index f18b08c03553..f07c1d087227 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -23,7 +23,7 @@
  * soon be no new userspace code that will ever use a vsyscall.
  *
  * The code in this file emulates vsyscalls when notified of a page
- * fault to a vsyscall address.
+ * fault or a general protection fault to a vsyscall address.
  */
 
 #include <linux/kernel.h>
@@ -276,6 +276,15 @@ bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,
 	return __emulate_vsyscall(regs, address);
 }
 
+bool emulate_vsyscall_gp(struct pt_regs *regs)
+{
+	/* Emulate only if the RIP points to the vsyscall address */
+	if (!is_vsyscall_vaddr(regs->ip))
+		return false;
+
+	return __emulate_vsyscall(regs, regs->ip);
+}
+
 /*
  * A pseudo VMA to allow ptrace access for the vsyscall page.  This only
  * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
diff --git a/arch/x86/include/asm/vsyscall.h b/arch/x86/include/asm/vsyscall.h
index 214977f4fa11..4eb8d3673223 100644
--- a/arch/x86/include/asm/vsyscall.h
+++ b/arch/x86/include/asm/vsyscall.h
@@ -16,6 +16,7 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);
  */
 extern bool emulate_vsyscall_pf(unsigned long error_code,
 				struct pt_regs *regs, unsigned long address);
+extern bool emulate_vsyscall_gp(struct pt_regs *regs);
 #else
 static inline void map_vsyscall(void) {}
 static inline bool emulate_vsyscall_pf(unsigned long error_code,
@@ -23,6 +24,11 @@ static inline bool emulate_vsyscall_pf(unsigned long error_code,
 {
 	return false;
 }
+
+static inline bool emulate_vsyscall_gp(struct pt_regs *regs)
+{
+	return false;
+}
 #endif
 
 /*
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 3eb2ef46f733..333c45df80fc 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -69,6 +69,7 @@
 #include <asm/tdx.h>
 #include <asm/cfi.h>
 #include <asm/msr.h>
+#include <asm/vsyscall.h>
 
 #ifdef CONFIG_X86_64
 #include <asm/x86_init.h>
@@ -817,6 +818,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 		if (cpu_feature_enabled(X86_FEATURE_UMIP) && fixup_umip_exception(regs))
 			goto exit;
 
+		if (cpu_feature_enabled(X86_FEATURE_LASS) && emulate_vsyscall_gp(regs))
+			goto exit;
+
 		gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc);
 		goto exit;
 	}
-- 
2.47.2

[PATCHv6 10/16] x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

The EMULATE mode of vsyscall maps the vsyscall page into user address
space which can be read directly by the user application. This mode has
been deprecated recently and can only be enabled from a special command
line parameter vsyscall=emulate. See commit bf00745e7791 ("x86/vsyscall:
Remove CONFIG_LEGACY_VSYSCALL_EMULATE")

Fixing the LASS violations during the EMULATE mode would need complex
instruction decoding since the resulting #GP fault does not include any
useful error information and the vsyscall address is not readily
available in the RIP.

At this point, no one is expected to be using the insecure and
deprecated EMULATE mode. The rare usages that need support probably
don't care much about security anyway. Disable LASS when EMULATE mode is
requested during command line parsing to avoid breaking user software.
LASS will be supported if vsyscall mode is set to XONLY or NONE.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 4 +++-
 arch/x86/entry/vsyscall/vsyscall_64.c           | 8 ++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f1f2c0874da9..796c987372df 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -7926,7 +7926,9 @@
 
 			emulate     Vsyscalls turn into traps and are emulated
 			            reasonably safely.  The vsyscall page is
-				    readable.
+				    readable.  This disables the Linear
+				    Address Space Separation (LASS) security
+				    feature and makes the system less secure.
 
 			xonly       [default] Vsyscalls turn into traps and are
 			            emulated reasonably safely.  The vsyscall
diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index f07c1d087227..77abe6aec71f 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -36,6 +36,7 @@
 #include <asm/vsyscall.h>
 #include <asm/unistd.h>
 #include <asm/fixmap.h>
+#include <asm/tlbflush.h>
 #include <asm/traps.h>
 #include <asm/paravirt.h>
 
@@ -63,6 +64,13 @@ static int __init vsyscall_setup(char *str)
 		else
 			return -EINVAL;
 
+		if (cpu_feature_enabled(X86_FEATURE_LASS) &&
+		    vsyscall_mode == EMULATE) {
+			cr4_clear_bits(X86_CR4_LASS);
+			setup_clear_cpu_cap(X86_FEATURE_LASS);
+			pr_warn_once("x86/cpu: Disabling LASS support due to vsyscall=emulate\n");
+		}
+
 		return 0;
 	}
 
-- 
2.47.2

[PATCHv6 11/16] x86/cpu: Set LASS CR4 bit as pinning sensitive

[Kirill A. Shutemov]

From: Yian Chen <yian.chen@intel.com>

Security features such as LASS are not expected to be disabled once
initialized. Add LASS to the CR4 pinned mask.

Signed-off-by: Yian Chen <yian.chen@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/cpu/common.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 1a779e302872..d12ce36a76f5 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -403,7 +403,8 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
-					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED;
+					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED |
+					     X86_CR4_LASS;
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
-- 
2.47.2

[PATCHv6 12/16] x86/traps: Communicate a LASS violation in #GP message

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

Provide a more helpful message on #GP when a kernel side LASS violation
is detected.

A NULL pointer dereference is reported if a LASS violation occurs due to
accessing the first page frame.

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/traps.c | 24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 333c45df80fc..0f558d3369a3 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -636,7 +636,16 @@ DEFINE_IDTENTRY(exc_bounds)
 enum kernel_gp_hint {
 	GP_NO_HINT,
 	GP_NON_CANONICAL,
-	GP_CANONICAL
+	GP_CANONICAL,
+	GP_LASS_VIOLATION,
+	GP_NULL_POINTER,
+};
+
+static const char *kernel_gp_hint_help[] = {
+	[GP_NON_CANONICAL]	= "probably for non-canonical address",
+	[GP_CANONICAL]		= "maybe for address",
+	[GP_LASS_VIOLATION]	= "LASS prevented access to address",
+	[GP_NULL_POINTER]	= "kernel NULL pointer dereference",
 };
 
 /*
@@ -672,6 +681,12 @@ static enum kernel_gp_hint get_kernel_gp_address(struct pt_regs *regs,
 	if (*addr < ~__VIRTUAL_MASK &&
 	    *addr + insn.opnd_bytes - 1 > __VIRTUAL_MASK)
 		return GP_NON_CANONICAL;
+	else if (*addr < ~__VIRTUAL_MASK &&
+		 cpu_feature_enabled(X86_FEATURE_LASS)) {
+		if (*addr < PAGE_SIZE)
+			return GP_NULL_POINTER;
+		return GP_LASS_VIOLATION;
+	}
 #endif
 
 	return GP_CANONICAL;
@@ -833,11 +848,10 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 	else
 		hint = get_kernel_gp_address(regs, &gp_addr);
 
-	if (hint != GP_NO_HINT)
+	if (hint != GP_NO_HINT) {
 		snprintf(desc, sizeof(desc), GPFSTR ", %s 0x%lx",
-			 (hint == GP_NON_CANONICAL) ? "probably for non-canonical address"
-						    : "maybe for address",
-			 gp_addr);
+			 kernel_gp_hint_help[hint], gp_addr);
+	}
 
 	/*
 	 * KASAN is interested only in the non-canonical case, clear it
-- 
2.47.2

[PATCHv6 13/16] x86/traps: Handle LASS thrown #SS

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

LASS throws a #GP for any violations except for stack register accesses,
in which case it throws a #SS instead. Handle this similarly to how other
LASS violations are handled.

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/traps.c | 34 ++++++++++++++++++++++++++++------
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 0f558d3369a3..bd8f7e72b238 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -418,12 +418,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_segment_not_present)
 		      SIGBUS, 0, NULL);
 }
 
-DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment)
-{
-	do_error_trap(regs, error_code, "stack segment", X86_TRAP_SS, SIGBUS,
-		      0, NULL);
-}
-
 DEFINE_IDTENTRY_ERRORCODE(exc_alignment_check)
 {
 	char *str = "alignment check";
@@ -866,6 +860,34 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 	cond_local_irq_disable(regs);
 }
 
+DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment)
+{
+	if (cpu_feature_enabled(X86_FEATURE_LASS)) {
+		enum kernel_gp_hint hint = GP_NO_HINT;
+		unsigned long gp_addr;
+
+		if (user_mode(regs)) {
+			gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, GPFSTR);
+			return;
+		}
+
+		hint = get_kernel_gp_address(regs, &gp_addr);
+		if (hint != GP_NO_HINT) {
+			printk(GPFSTR ", %s 0x%lx", kernel_gp_hint_help[hint],
+			       gp_addr);
+		}
+
+		if (hint != GP_NON_CANONICAL)
+			gp_addr = 0;
+
+		die_addr(GPFSTR, regs, error_code, gp_addr);
+		return;
+	}
+
+	do_error_trap(regs, error_code, "stack segment", X86_TRAP_SS, SIGBUS,
+		0, NULL);
+}
+
 static bool do_int3(struct pt_regs *regs)
 {
 	int res;
-- 
2.47.2

[PATCHv6 14/16] x86/cpu: Make LAM depend on LASS

[Kirill A. Shutemov]

From: Alexander Shishkin <alexander.shishkin@linux.intel.com>

To prevent exploits for Spectre based on LAM as demonstrated by the
whitepaper [1], make LAM depend on LASS, which avoids this type of
vulnerability.

[1] https://download.vusec.net/papers/slam_sp24.pdf

Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/cpu/cpuid-deps.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c
index 98d0cdd82574..11bb9ed40140 100644
--- a/arch/x86/kernel/cpu/cpuid-deps.c
+++ b/arch/x86/kernel/cpu/cpuid-deps.c
@@ -90,6 +90,7 @@ static const struct cpuid_dep cpuid_deps[] = {
 	{ X86_FEATURE_FRED,			X86_FEATURE_LKGS      },
 	{ X86_FEATURE_SPEC_CTRL_SSBD,		X86_FEATURE_SPEC_CTRL },
 	{ X86_FEATURE_LASS,			X86_FEATURE_SMAP      },
+	{ X86_FEATURE_LAM,			X86_FEATURE_LASS      },
 	{}
 };
 
-- 
2.47.2

[PATCHv6 15/16] x86/cpu: Enable LASS during CPU initialization

[Kirill A. Shutemov]

From: Sohil Mehta <sohil.mehta@intel.com>

Being a security feature, enable LASS by default if the platform
supports it.

While at it, get rid of the comment above the SMAP/SMEP/UMIP/LASS setup
instead of updating it to mention LASS as well, as the whole sequence is
quite self-explanatory.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/cpu/common.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index d12ce36a76f5..edb342214328 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -401,6 +401,12 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 	cr4_clear_bits(X86_CR4_UMIP);
 }
 
+static __always_inline void setup_lass(struct cpuinfo_x86 *c)
+{
+	if (cpu_feature_enabled(X86_FEATURE_LASS))
+		cr4_set_bits(X86_CR4_LASS);
+}
+
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
 					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED |
@@ -1975,10 +1981,10 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	/* Disable the PN if appropriate */
 	squash_the_stupid_serial_number(c);
 
-	/* Set up SMEP/SMAP/UMIP */
 	setup_smep(c);
 	setup_smap(c);
 	setup_umip(c);
+	setup_lass(c);
 
 	/* Enable FSGSBASE instructions if available. */
 	if (cpu_has(c, X86_FEATURE_FSGSBASE)) {
-- 
2.47.2

[PATCHv6 16/16] x86: Re-enable Linear Address Masking

[Kirill A. Shutemov]

This reverts commit 3267cb6d3a174ff83d6287dcd5b0047bbd912452.

LASS mitigates the Spectre based on LAM (SLAM) [1] and the previous
commit made LAM depend on LASS, so we no longer need to disable LAM at
compile time, so revert the commit that disables LAM.

Adjust USER_PTR_MAX if LAM enabled, allowing tag bits to be set for
userspace pointers. The value for the constant is defined in a way to
avoid overflow compiler warning on 32-bit config.

[1] https://download.vusec.net/papers/slam_sp24.pdf

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
---
 arch/x86/Kconfig             | 1 -
 arch/x86/kernel/cpu/common.c | 5 +----
 2 files changed, 1 insertion(+), 5 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 71019b3b54ea..2b48e916b754 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2181,7 +2181,6 @@ config RANDOMIZE_MEMORY_PHYSICAL_PADDING
 config ADDRESS_MASKING
 	bool "Linear Address Masking support"
 	depends on X86_64
-	depends on COMPILE_TEST || !CPU_MITIGATIONS # wait for LASS
 	help
 	  Linear Address Masking (LAM) modifies the checking that is applied
 	  to 64-bit linear addresses, allowing software to use of the
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index edb342214328..0adf1b4722b8 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2565,11 +2565,8 @@ void __init arch_cpu_finalize_init(void)
 	if (IS_ENABLED(CONFIG_X86_64)) {
 		unsigned long USER_PTR_MAX = TASK_SIZE_MAX;
 
-		/*
-		 * Enable this when LAM is gated on LASS support
 		if (cpu_feature_enabled(X86_FEATURE_LAM))
-			USER_PTR_MAX = (1ul << 63) - PAGE_SIZE;
-		 */
+			USER_PTR_MAX = (-1UL >> 1) & PAGE_MASK;
 		runtime_const_init(ptr, USER_PTR_MAX);
 
 		/*
-- 
2.47.2

Re: [PATCHv6 08/16] x86/traps: Consolidate user fixups in exc_general_protection()

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> @@ -819,6 +814,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
>  		if (fixup_vdso_exception(regs, X86_TRAP_GP, error_code, 0))
>  			goto exit;
>  
> +		if (cpu_feature_enabled(X86_FEATURE_UMIP) && fixup_umip_exception(regs))
> +			goto exit;

I'd move the cpu_feature_enabled() check into fixup_umip_exception()
just to make the exc_general_protection() code easier to look at. It'll
force an extra (cheap) CALL, but this is a super cold code path anyway.

With that:

Acked-by: Dave Hansen <dave.hansen@linux.intel.com>

Re: [PATCHv6 13/16] x86/traps: Handle LASS thrown #SS

[Xin Li]

On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
> From: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> 
> LASS throws a #GP for any violations except for stack register accesses,
> in which case it throws a #SS instead. Handle this similarly to how other
> LASS violations are handled.
> 
> Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> ---
>   arch/x86/kernel/traps.c | 34 ++++++++++++++++++++++++++++------
>   1 file changed, 28 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
> index 0f558d3369a3..bd8f7e72b238 100644
> --- a/arch/x86/kernel/traps.c
> +++ b/arch/x86/kernel/traps.c
> @@ -866,6 +860,34 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
>   	cond_local_irq_disable(regs);
>   }
>   
> +DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment)
> +{
> +	if (cpu_feature_enabled(X86_FEATURE_LASS)) {
> +		enum kernel_gp_hint hint = GP_NO_HINT;
> +		unsigned long gp_addr;
> +
> +		if (user_mode(regs)) {
> +			gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, GPFSTR);
> +			return;
> +		}

Please add the following FRED code:

		if (cpu_feature_enabled(X86_FEATURE_FRED) &&
		    fixup_exception(regs, X86_TRAP_SS, error_code, 0))
			return;

Because the following FRED fixup

  
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5105e7687ad3dffde77f6e4393b5530e83d672dc

is otherwise skipped, it is interpreted as a LASS #SS before reaching
the FRED fixup (which is indeed a userspace fault but triggered from
a specific kernel IP as described in the commit changelog).

The x86 selftest sigreturn_64 fails w/o the above fix.

> +
> +		hint = get_kernel_gp_address(regs, &gp_addr);
> +		if (hint != GP_NO_HINT) {
> +			printk(GPFSTR ", %s 0x%lx", kernel_gp_hint_help[hint],

GPFSTR?

#define GPFSTR "general protection fault"

It is misleading here, you need something like SSFSTR which tells
exactly which exception caused it.

Thanks!
     Xin

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> +static __always_inline void lass_clac(void)
> +{
> +	alternative("", "clac", X86_FEATURE_LASS);
> +}
> +
> +static __always_inline void lass_stac(void)
> +{
> +	alternative("", "stac", X86_FEATURE_LASS);
> +}

There are differing opinions on how to introduce code. Do you do it up
front and then use it later in the series? Or do you only introduce
infrastructure when you use it?

Doing what you have here (introduce first, use later) is fine. But it
does incur an increased burden of explaining things.

These two functions in particular mean that you have to at least allude
to how these get used later in the series.

Re: [PATCHv6 03/16] x86/alternatives: Disable LASS when patching kernel alternatives

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> +/*
> + * poking_init() initializes the text poking address from the lower half of the
> + * address space. Relax LASS enforcement when accessing the poking address.
> + */

This does not read quite right. I think there's some unnecessary
confusiuon about initializing the address versus the contents. But
either way, how about:

/*
 * Text poking creates and uses a mapping in the lower half of the
 * address space. Relax LASS enforcement when accessing the poking
 * address.
 */

>  static void text_poke_memset(void *dst, const void *src, size_t len)
>  {
>  	int c = *(const int *)src;
>  
> -	memset(dst, c, len);
> +	lass_stac();
> +	__inline_memset(dst, c, len);
> +	lass_clac();
>  }

Yeah, this patch small enough that lass_{stac,clac}() are definitely
misplaced in this series. Just move them into this patch.

Re: [PATCHv6 04/16] x86/efi: Move runtime service initialization to arch/x86

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> From: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> 
> The EFI call in start_kernel() is guarded by #ifdef CONFIG_X86. Move
> the thing to the arch_cpu_finalize_init() path on x86 and get rid of
> the #ifdef in start_kernel().

What does this have to do with LASS?

Is there any reason not to just pull this in to tip now as a cleanup? It
looks good regardless of what it does to LASS.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
> diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
> index 250c10627ab3..9574c198fc08 100644
> --- a/arch/x86/Kconfig.cpufeatures
> +++ b/arch/x86/Kconfig.cpufeatures
> @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
>   	def_bool y
>   	depends on !X86_64
>   
> +config X86_DISABLED_FEATURE_LASS
> +	def_bool y
> +	depends on !X86_64
> +
>   config X86_DISABLED_FEATURE_PKU
>   	def_bool y
>   	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS

You don't need to add X86_DISABLED_FEATURE_LASS, because the LASS code
is NOT optional at build time, i.e., you now don't have CONFIG_X86_LASS.

It was removed per the change log:

Changes from v1[1]:
- Remove CONFIG_X86_LASS

Thanks!
     Xin

Re: [PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> Linear Address Space Separation (LASS) is a security feature that intends to
> prevent malicious virtual address space accesses across user/kernel mode.

Most of the cover letter here reads like an Intel whitepaper. That's not
the worst thing in the world, but I think it glosses over one very
important point:

	Had it been available, LASS alone would have mitigated Meltdown.

Could we say this up front in a prominent place, please?

Re: [PATCHv6 05/16] x86/cpu: Defer CR pinning setup until after EFI initialization

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
>  	/*
>  	 * This needs to follow the FPU initializtion, since EFI depends on it.
> +	 *
> +	 * It also needs to precede the CR pinning setup, because the CR4.LASS
> +	 * bit has to be cleared temporarily in order to execute the
> +	 * set_virtual_address_map() EFI call, which resides in lower addresses
> +	 * and would trip LASS if enabled.
> +	 *
> +	 * Wrapping efi_enter_virtual_mode() into lass_stac()/clac() is not
> +	 * enough because AC flag gates data accesses, but not instruction
> +	 * fetch. Clearing the CR4 bit is required.
>  	 */
>  	if (efi_enabled(EFI_RUNTIME_SERVICES))
>  		efi_enter_virtual_mode();
>  
> +	setup_cr_pinning();

This is a _bit_ too much of a comment for me in this context. This would
be enough:

	/* EFI twiddles CR4.LASS. Do it before CR pinning: */
  	if (efi_enabled(EFI_RUNTIME_SERVICES))
  		efi_enter_virtual_mode();

	setup_cr_pinning();

I also dislike comments where code will do. Could
efi_enter_virtual_mode() actually check for CR4.LASS? Or check
cr4_pinned_bits() or 'cr_pinning.key'?

Also, the "lass_stac()/clac() is not enough" comment seems more like
changelog material to me than something for a comment.

Re: [PATCHv6 06/16] efi: Disable LASS around set_virtual_address_map() EFI call

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
> index 463b784499a8..94c335229697 100644
> --- a/arch/x86/platform/efi/efi.c
> +++ b/arch/x86/platform/efi/efi.c
> @@ -825,11 +825,24 @@ static void __init __efi_enter_virtual_mode(void)
>  
>  	efi_sync_low_kernel_mappings();
>  
> +	/*
> +	 * set_virtual_address_map is the only service located at lower
> +	 * addresses, so we have to temporarily disable LASS around it.
> +	 * Note that clearing EFLAGS.AC is not enough for this, the whole
> +	 * LASS needs to be disabled.
> +	 */
> +	if (cpu_feature_enabled(X86_FEATURE_LASS))
> +		cr4_clear_bits(X86_CR4_LASS);

Could we do it like this instead?

	unsigned long lass = cr4_read_shadow() & X86_FEATURE_LASS;
	...
	cr4_clear_bits(lass);

>  	status = efi_set_virtual_address_map(efi.memmap.desc_size * count,
>  					     efi.memmap.desc_size,
>  					     efi.memmap.desc_version,
>  					     (efi_memory_desc_t *)pa,
>  					     efi_systab_phys);
> +
> +	if (cpu_feature_enabled(X86_FEATURE_LASS))
> +		cr4_set_bits(X86_CR4_LASS);

and:

	cr4_set_bits(lass);

>  	if (status != EFI_SUCCESS) {
>  		pr_err("Unable to switch EFI into virtual mode (status=%lx)!\n",
>  		       status);

That way, neither the presence of X86_FEATURE_LASS nor the ordering of
setting up X86_CR4_LASS matters.

Let's say the CPU supports X86_FEATURE_LASS and this code gets called
before the kernel is ready for LASS. It would break as written above.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Randy Dunlap]

Hi--

On 6/20/25 6:53 AM, Kirill A. Shutemov wrote:
> From: Sohil Mehta <sohil.mehta@intel.com>
> 
> Linear Address Space Separation (LASS) is a security feature that
> intends to prevent malicious virtual address space accesses across
> user/kernel mode.
> 
> Such mode based access protection already exists today with paging and
> features such as SMEP and SMAP. However, to enforce these protections,
> the processor must traverse the paging structures in memory.  Malicious
> software can use timing information resulting from this traversal to
> determine details about the paging structures, and these details may
> also be used to determine the layout of the kernel memory.
> 
> The LASS mechanism provides the same mode-based protections as paging
> but without traversing the paging structures. Because the protections
> enforced by LASS are applied before paging, software will not be able to
> derive paging-based timing information from the various caching
> structures such as the TLBs, mid-level caches, page walker, data caches,
> etc.
> 
> LASS enforcement relies on the typical kernel implementation to divide
> the 64-bit virtual address space into two halves:
>   Addr[63]=0 -> User address space
>   Addr[63]=1 -> Kernel address space
> 
> Any data access or code execution across address spaces typically
> results in a #GP fault.
> 
> The LASS enforcement for kernel data access is dependent on CR4.SMAP
> being set. The enforcement can be disabled by toggling the RFLAGS.AC bit
> similar to SMAP.
> 
> Define the CPU feature bits to enumerate this feature and include
> feature dependencies to reflect the same.
> 
> Co-developed-by: Yian Chen <yian.chen@intel.com>
> Signed-off-by: Yian Chen <yian.chen@intel.com>
> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
> Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> ---
>  arch/x86/Kconfig.cpufeatures                |  4 ++++
>  arch/x86/include/asm/cpufeatures.h          |  1 +
>  arch/x86/include/asm/smap.h                 | 22 +++++++++++++++++++--
>  arch/x86/include/uapi/asm/processor-flags.h |  2 ++
>  arch/x86/kernel/cpu/cpuid-deps.c            |  1 +
>  tools/arch/x86/include/asm/cpufeatures.h    |  1 +
>  6 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
> index 250c10627ab3..9574c198fc08 100644
> --- a/arch/x86/Kconfig.cpufeatures
> +++ b/arch/x86/Kconfig.cpufeatures
> @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
>  	def_bool y
>  	depends on !X86_64
>  
> +config X86_DISABLED_FEATURE_LASS
> +	def_bool y
> +	depends on !X86_64

Please explain why this is   !X86_64.
Thanks.

> +
>  config X86_DISABLED_FEATURE_PKU
>  	def_bool y
>  	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS


-- 
~Randy

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/20/2025 9:02 AM, Randy Dunlap wrote:
>> +config X86_DISABLED_FEATURE_LASS
>> +	def_bool y
>> +	depends on !X86_64
> Please explain why this is   !X86_64.

When NOT on X86_64, the LASS code should not be compiled.

But first of all, as I replied earlier, X86_DISABLED_FEATURE_LASS is
completely not needed.

Thanks!
     Xin

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Randy Dunlap]

On 6/20/25 9:12 AM, Xin Li wrote:
> On 6/20/2025 9:02 AM, Randy Dunlap wrote:
>>> +config X86_DISABLED_FEATURE_LASS
>>> +    def_bool y
>>> +    depends on !X86_64
>> Please explain why this is   !X86_64.
> 
> When NOT on X86_64, the LASS code should not be compiled.

Ah yes, the double negative caught me.

> But first of all, as I replied earlier, X86_DISABLED_FEATURE_LASS is
> completely not needed.

That's good.

Thanks.

-- 
~Randy

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Fri, Jun 20, 2025 at 04:53:09PM +0300, Kirill A. Shutemov wrote:
> diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> index ee176236c2be..4473a6f7800b 100644
> --- a/arch/x86/include/asm/cpufeatures.h
> +++ b/arch/x86/include/asm/cpufeatures.h
> @@ -313,6 +313,7 @@
>  #define X86_FEATURE_SM4			(12*32+ 2) /* SM4 instructions */
>  #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* "avx_vnni" AVX VNNI instructions */
>  #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT16 instructions */
> +#define X86_FEATURE_LASS		(12*32+ 6) /* "lass" Linear Address Space Separation */

This needs to be in /proc/cpuinfo because...?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 03/16] x86/alternatives: Disable LASS when patching kernel alternatives

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:33:57AM -0700, Dave Hansen wrote:
> On 6/20/25 06:53, Kirill A. Shutemov wrote:
> > +/*
> > + * poking_init() initializes the text poking address from the lower half of the
> > + * address space. Relax LASS enforcement when accessing the poking address.
> > + */
> 
> This does not read quite right. I think there's some unnecessary
> confusiuon about initializing the address versus the contents. But
> either way, how about:
> 
> /*
>  * Text poking creates and uses a mapping in the lower half of the
>  * address space. Relax LASS enforcement when accessing the poking
>  * address.
>  */

Looks good. Will update, thanks!
> 
> >  static void text_poke_memset(void *dst, const void *src, size_t len)
> >  {
> >  	int c = *(const int *)src;
> >  
> > -	memset(dst, c, len);
> > +	lass_stac();
> > +	__inline_memset(dst, c, len);
> > +	lass_clac();
> >  }
> 
> Yeah, this patch small enough that lass_{stac,clac}() are definitely
> misplaced in this series. Just move them into this patch.

Okay, makes sense.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:36:30AM -0700, Xin Li wrote:
> On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
> > diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
> > index 250c10627ab3..9574c198fc08 100644
> > --- a/arch/x86/Kconfig.cpufeatures
> > +++ b/arch/x86/Kconfig.cpufeatures
> > @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
> >   	def_bool y
> >   	depends on !X86_64
> > +config X86_DISABLED_FEATURE_LASS
> > +	def_bool y
> > +	depends on !X86_64
> > +
> >   config X86_DISABLED_FEATURE_PKU
> >   	def_bool y
> >   	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS
> 
> You don't need to add X86_DISABLED_FEATURE_LASS, because the LASS code
> is NOT optional at build time, i.e., you now don't have CONFIG_X86_LASS.

Hmm. But it is optional. It depends on CONFIG_X86_64. I don't think we
want it to be advertised on 32-bit kernels.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 06:35:04PM +0200, Borislav Petkov wrote:
> On Fri, Jun 20, 2025 at 04:53:09PM +0300, Kirill A. Shutemov wrote:
> > diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> > index ee176236c2be..4473a6f7800b 100644
> > --- a/arch/x86/include/asm/cpufeatures.h
> > +++ b/arch/x86/include/asm/cpufeatures.h
> > @@ -313,6 +313,7 @@
> >  #define X86_FEATURE_SM4			(12*32+ 2) /* SM4 instructions */
> >  #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* "avx_vnni" AVX VNNI instructions */
> >  #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT16 instructions */
> > +#define X86_FEATURE_LASS		(12*32+ 6) /* "lass" Linear Address Space Separation */
> 
> This needs to be in /proc/cpuinfo because...?

What is current policy around it ?

I think it is useful to advertise security features in cpuinfo. LASS fits
next to SMAP/SMEP/UMIP.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 04/16] x86/efi: Move runtime service initialization to arch/x86

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:35:14AM -0700, Dave Hansen wrote:
> On 6/20/25 06:53, Kirill A. Shutemov wrote:
> > From: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> > 
> > The EFI call in start_kernel() is guarded by #ifdef CONFIG_X86. Move
> > the thing to the arch_cpu_finalize_init() path on x86 and get rid of
> > the #ifdef in start_kernel().
> 
> What does this have to do with LASS?

It is required to move CR pinning below it and allow to disable CR4.LASS
temporarily during EFI initialization.

> Is there any reason not to just pull this in to tip now as a cleanup? It
> looks good regardless of what it does to LASS.

I am totally fine with applying it right away.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 06/16] efi: Disable LASS around set_virtual_address_map() EFI call

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:55:29AM -0700, Dave Hansen wrote:
> On 6/20/25 06:53, Kirill A. Shutemov wrote:
> > diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
> > index 463b784499a8..94c335229697 100644
> > --- a/arch/x86/platform/efi/efi.c
> > +++ b/arch/x86/platform/efi/efi.c
> > @@ -825,11 +825,24 @@ static void __init __efi_enter_virtual_mode(void)
> >  
> >  	efi_sync_low_kernel_mappings();
> >  
> > +	/*
> > +	 * set_virtual_address_map is the only service located at lower
> > +	 * addresses, so we have to temporarily disable LASS around it.
> > +	 * Note that clearing EFLAGS.AC is not enough for this, the whole
> > +	 * LASS needs to be disabled.
> > +	 */
> > +	if (cpu_feature_enabled(X86_FEATURE_LASS))
> > +		cr4_clear_bits(X86_CR4_LASS);
> 
> Could we do it like this instead?
> 
> 	unsigned long lass = cr4_read_shadow() & X86_FEATURE_LASS;
> 	...
> 	cr4_clear_bits(lass);
> 
> >  	status = efi_set_virtual_address_map(efi.memmap.desc_size * count,
> >  					     efi.memmap.desc_size,
> >  					     efi.memmap.desc_version,
> >  					     (efi_memory_desc_t *)pa,
> >  					     efi_systab_phys);
> > +
> > +	if (cpu_feature_enabled(X86_FEATURE_LASS))
> > +		cr4_set_bits(X86_CR4_LASS);
> 
> and:
> 
> 	cr4_set_bits(lass);
> 
> >  	if (status != EFI_SUCCESS) {
> >  		pr_err("Unable to switch EFI into virtual mode (status=%lx)!\n",
> >  		       status);
> 
> That way, neither the presence of X86_FEATURE_LASS nor the ordering of
> setting up X86_CR4_LASS matters.
> 
> Let's say the CPU supports X86_FEATURE_LASS and this code gets called
> before the kernel is ready for LASS. It would break as written above.

Okay, will rework this.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 13/16] x86/traps: Handle LASS thrown #SS

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:20:12AM -0700, Xin Li wrote:
> On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
> > From: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> > 
> > LASS throws a #GP for any violations except for stack register accesses,
> > in which case it throws a #SS instead. Handle this similarly to how other
> > LASS violations are handled.
> > 
> > Signed-off-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > ---
> >   arch/x86/kernel/traps.c | 34 ++++++++++++++++++++++++++++------
> >   1 file changed, 28 insertions(+), 6 deletions(-)
> > 
> > diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
> > index 0f558d3369a3..bd8f7e72b238 100644
> > --- a/arch/x86/kernel/traps.c
> > +++ b/arch/x86/kernel/traps.c
> > @@ -866,6 +860,34 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
> >   	cond_local_irq_disable(regs);
> >   }
> > +DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment)
> > +{
> > +	if (cpu_feature_enabled(X86_FEATURE_LASS)) {
> > +		enum kernel_gp_hint hint = GP_NO_HINT;
> > +		unsigned long gp_addr;
> > +
> > +		if (user_mode(regs)) {
> > +			gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, GPFSTR);
> > +			return;
> > +		}
> 
> Please add the following FRED code:
> 
> 		if (cpu_feature_enabled(X86_FEATURE_FRED) &&
> 		    fixup_exception(regs, X86_TRAP_SS, error_code, 0))
> 			return;
> 
> Because the following FRED fixup
> 
> https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5105e7687ad3dffde77f6e4393b5530e83d672dc
> 
> is otherwise skipped, it is interpreted as a LASS #SS before reaching
> the FRED fixup (which is indeed a userspace fault but triggered from
> a specific kernel IP as described in the commit changelog).
> 
> The x86 selftest sigreturn_64 fails w/o the above fix.

Okay, will do.

> > +
> > +		hint = get_kernel_gp_address(regs, &gp_addr);
> > +		if (hint != GP_NO_HINT) {
> > +			printk(GPFSTR ", %s 0x%lx", kernel_gp_hint_help[hint],
> 
> GPFSTR?
> 
> #define GPFSTR "general protection fault"
> 
> It is misleading here, you need something like SSFSTR which tells
> exactly which exception caused it.

Yep, thanks for catching that!

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Sohil Mehta]

On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
>  
> +/*
> + * The CLAC/STAC instructions toggle enforcement of X86_FEATURE_SMAP.
> + *
> + * X86_FEATURE_LASS requires flipping the AC flag when accessing the lower half
> + * of the virtual address space, regardless of the _PAGE_BIT_USER bit in the
> + * page tables. lass_clac/stac() should be used for these cases.
> + *

Is this supposed to be "regardless" or only when the _PAGE_BIT_USER bit
it set? The way the sentence is worded it would seem that the kernel
could always use lass_clac()/stac() since the value in _PAGE_BIT_USER
doesn't matter.

Please correct me if I am wrong, but here is my understanding:

X86_FEATURE_SMAP and X86_FEATURE_LASS both complain when the kernel
tries to access the lower half of the virtual addresses.

SMAP flags an issue if _PAGE_BIT_USER is not set. LASS would #GP in both
cases with or without the _PAGE_BIT_USER being set.

However, in terms of usage, we want to use LASS specific stac()/clac()
only when _PAGE_BIT_USER is set. Since this won't be flagged by SMAP.

@Dave Hansen, you had suggested separating out the SMAP/LASS AC toggle
functions. But, the difference in usage between both of them seems very
subtle. Could this be easily misused?

For example, there is no failure that would happen if someone
incorrectly uses the SMAP specific clac()/stac() calls instead of the
LASS ones.

> + * Note: a barrier is implicit in alternative().
> + */
> +
>  static __always_inline void clac(void)
>  {
> -	/* Note: a barrier is implicit in alternative() */
>  	alternative("", "clac", X86_FEATURE_SMAP);
>  }
>  
>  static __always_inline void stac(void)
>  {
> -	/* Note: a barrier is implicit in alternative() */
>  	alternative("", "stac", X86_FEATURE_SMAP);
>  }
>  
> +static __always_inline void lass_clac(void)
> +{
> +	alternative("", "clac", X86_FEATURE_LASS);
> +}
> +
> +static __always_inline void lass_stac(void)
> +{
> +	alternative("", "stac", X86_FEATURE_LASS);
> +}
> +

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Dave Hansen]

On 6/20/25 11:14, Sohil Mehta wrote:
> @Dave Hansen, you had suggested separating out the SMAP/LASS AC toggle
> functions. But, the difference in usage between both of them seems very
> subtle. Could this be easily misused?

Logically there are two completely different things:

	1. Touching userspace
	2. Touching the lower half of the address space

If it's only userspace in the lower half of the address space, then
there's no controversy. But the problem obviously occurs when you want
to touch kernel mappings in the lower half of the address space.

I want to preserve the "stac/clas" meaning as just "touch userspace".

The new functions should be for "touch the lower half of the address
space" alone. Maybe it should be:

	lass_disable_enforcement()
	lass_enable_enforcement()

The only downside of not having stac/clac in the names is that it's not
obvious that they have an impact on SMAP because they're named to be
LASS-only. But I'm not super worried about this. If we have a
proliferation of call sites we have bigger problem on our hands.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Fri, Jun 20, 2025 at 08:33:36PM +0300, Kirill A. Shutemov wrote:
> What is current policy around it ?

Documentation/arch/x86/cpuinfo.rst

> I think it is useful to advertise security features in cpuinfo.

Because who's going to consume them?

Don't get me wrong - I am trying to see whether the whole use case for this is
well thought out. Because it becomes an ABI.

But if no one is going to use it, why bother?

Arguably, for this thing the argument would be - as it is put in that file
above:

"So, the current use of /proc/cpuinfo is to show features which the
kernel has *enabled* and *supports*."

as it has been enabled by machinery.

So that's ok. I'm just making sure we're on the same page and you're not
aiming at something completely different with this.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Dave Hansen]

On 6/20/25 06:53, Kirill A. Shutemov wrote:
> +bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,
> +			 unsigned long address)
> +{
> +	/* Write faults or kernel-privilege faults never get fixed up. */
> +	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
> +		return false;
> +
> +	if (!(error_code & X86_PF_INSTR)) {
> +		/* Failed vsyscall read */
> +		if (vsyscall_mode == EMULATE)
> +			return false;
> +
> +		/*
> +		 * User code tried and failed to read the vsyscall page.
> +		 */
> +		warn_bad_vsyscall(KERN_INFO, regs,
> +				  "vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround");
> +		return false;
> +	}
> +
> +	return __emulate_vsyscall(regs, address);
> +}

For this patch that just moves the code:

Acked-by: Dave Hansen <dave.hansen@linux.intel.com>

But, the resulting code is wonky. It needs to do something more like this:

	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
		return false;

	if (error_code & X86_PF_INSTR))
		return __emulate_vsyscall(regs, address);

	/* Failed vsyscall read */
	if (vsyscall_mode == EMULATE)
		return false;

	/*
	 * User code tried and failed to read the vsyscall page.
	 */
	warn_bad_vsyscall(KERN_INFO, regs, ...
	return false;

That's much more linear to read.

Re: [PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[Andrew Cooper]

> Most of the cover letter here reads like an Intel whitepaper. That's not
> the worst thing in the world, but I think it glosses over one very
> important point:
>
> 	Had it been available, LASS alone would have mitigated Meltdown.
>
> Could we say this up front in a prominent place, please?

I'm going to nitpick. :)

Yes, LASS would have made Meltdown a far less major problem than it was,
but I don't think that phrasing is fair.  As I recall, LASS was
literally invented as a "what would have been useful?" exercise in the
wake of Meltdown.

However, a less well known/researched area of Meltdown, which would not
be addressed by LASS, is the ability to pend stores to read-only memory
and proceed with the expectation that they'll success.

Other things which would have helped would have been PKS (and this *was*
asked for ahead of Skylake launching...)

The other important thing about LASS is that it does cut off a whole
class of sidechannels.  This halts definitely-rogue speculation, but is
useful for non-speculative security too.

~Andrew

Re: [PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[H. Peter Anvin]

On June 20, 2025 3:04:53 PM PDT, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> Most of the cover letter here reads like an Intel whitepaper. That's not
>> the worst thing in the world, but I think it glosses over one very
>> important point:
>>
>> 	Had it been available, LASS alone would have mitigated Meltdown.
>>
>> Could we say this up front in a prominent place, please?
>
>I'm going to nitpick. :)
>
>Yes, LASS would have made Meltdown a far less major problem than it was,
>but I don't think that phrasing is fair.  As I recall, LASS was
>literally invented as a "what would have been useful?" exercise in the
>wake of Meltdown.
>
>However, a less well known/researched area of Meltdown, which would not
>be addressed by LASS, is the ability to pend stores to read-only memory
>and proceed with the expectation that they'll success.
>
>Other things which would have helped would have been PKS (and this *was*
>asked for ahead of Skylake launching...)
>
>The other important thing about LASS is that it does cut off a whole
>class of sidechannels.  This halts definitely-rogue speculation, but is
>useful for non-speculative security too.
>
>~Andrew

Could you clarify what you mean with "pend stores to read-only memory?"

Re: [PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[H. Peter Anvin]

On June 20, 2025 3:04:53 PM PDT, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> Most of the cover letter here reads like an Intel whitepaper. That's not
>> the worst thing in the world, but I think it glosses over one very
>> important point:
>>
>> 	Had it been available, LASS alone would have mitigated Meltdown.
>>
>> Could we say this up front in a prominent place, please?
>
>I'm going to nitpick. :)
>
>Yes, LASS would have made Meltdown a far less major problem than it was,
>but I don't think that phrasing is fair.  As I recall, LASS was
>literally invented as a "what would have been useful?" exercise in the
>wake of Meltdown.
>
>However, a less well known/researched area of Meltdown, which would not
>be addressed by LASS, is the ability to pend stores to read-only memory
>and proceed with the expectation that they'll success.
>
>Other things which would have helped would have been PKS (and this *was*
>asked for ahead of Skylake launching...)
>
>The other important thing about LASS is that it does cut off a whole
>class of sidechannels.  This halts definitely-rogue speculation, but is
>useful for non-speculative security too.
>
>~Andrew

To some degree, the best way to describe LASS is "KPTI in hardware." No, it isn't an exact analogy – we don't switch the entire page table structure – but it does make the supervisor page table structure protected from user space *while still in memory*.

It is coarser-grained than software KPTI, but on the other hand the separation is absolute; there is to keep things like the kernel stack, descriptor tables and entry/exit code unprotected.

Re: [PATCHv6 00/16] x86: Enable Linear Address Space Separation support

[Dave Hansen]

On 6/20/25 15:04, Andrew Cooper wrote:
>> Most of the cover letter here reads like an Intel whitepaper. That's not
>> the worst thing in the world, but I think it glosses over one very
>> important point:
>>
>> 	Had it been available, LASS alone would have mitigated Meltdown.
>>
>> Could we say this up front in a prominent place, please?
> 
> I'm going to nitpick. :)
> 
> Yes, LASS would have made Meltdown a far less major problem than it was,
> but I don't think that phrasing is fair.  As I recall, LASS was
> literally invented as a "what would have been useful?" exercise in the
> wake of Meltdown.

While being concise, I was also trying to convey that LASS is pretty
powerful, rather than it being some uniquely genius move by the CPU
designers. It was absolutely conceived of in hindsight.

Do you feel like the changelog is sufficient as-is? Or do you think we
can beef the changelog up a bit? I just want to make sure folks reading
it have a clear understanding that it's not fixing purely theoretical
issues.

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Andrew Cooper]

> But, the resulting code is wonky. It needs to do something more like this:
>
> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
> 		return false;
>
> 	if (error_code & X86_PF_INSTR))
> 		return __emulate_vsyscall(regs, address);

To do this, LASS needs a proper interlink against NX || SMEP.

If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
meaning that fetches are reported as plain reads.

This leads to some fun corner cases in SMAP and now LASS too for virt.

~Andrew

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Sohil Mehta]

On 6/20/2025 11:24 AM, Dave Hansen wrote:
> On 6/20/25 11:14, Sohil Mehta wrote:
>> @Dave Hansen, you had suggested separating out the SMAP/LASS AC toggle
>> functions. But, the difference in usage between both of them seems very
>> subtle. Could this be easily misused?
> 
> Logically there are two completely different things:
> 
> 	1. Touching userspace
> 	2. Touching the lower half of the address space
> 
> If it's only userspace in the lower half of the address space, then
> there's no controversy. But the problem obviously occurs when you want
> to touch kernel mappings in the lower half of the address space.
> 

Makes sense. If we decide to go this way, we should capture some of this
thinking into the kernel comments. I find the current wording very
ambiguous.

> I want to preserve the "stac/clas" meaning as just "touch userspace".
> 
> The new functions should be for "touch the lower half of the address
> space" alone. Maybe it should be:
> 
> 	lass_disable_enforcement()
> 	lass_enable_enforcement()
> 
> The only downside of not having stac/clac in the names is that it's not
> obvious that they have an impact on SMAP because they're named to be
> LASS-only. 

Yeah, the reverse is also true, where the SMAP specific clac()/stac()
toggle LASS enforcement :)

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Sohil Mehta]

On 6/20/2025 4:08 PM, Andrew Cooper wrote:
>> But, the resulting code is wonky. It needs to do something more like this:
>>
>> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
>> 		return false;
>>
>> 	if (error_code & X86_PF_INSTR))
>> 		return __emulate_vsyscall(regs, address);
> 
> To do this, LASS needs a proper interlink against NX || SMEP.
> 
> If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
> meaning that fetches are reported as plain reads.
> 
> This leads to some fun corner cases in SMAP and now LASS too for virt.


Maybe I am missing something, but LASS works pre-paging so it wouldn't
generate a PF, right?

We have a new vsyscall emulation for #GP (patch 9) that wouldn't follow
this path.

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Dave Hansen]

On 6/20/25 16:08, Andrew Cooper wrote:
>> But, the resulting code is wonky. It needs to do something more like this:
>>
>> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
>> 		return false;
>>
>> 	if (error_code & X86_PF_INSTR))
>> 		return __emulate_vsyscall(regs, address);
> To do this, LASS needs a proper interlink against NX || SMEP.
> 
> If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
> meaning that fetches are reported as plain reads.
Interesting point.

I think the easiest way to do this is just make a cpuid_deps[] entry for
LASS and NX. If there's a CPU where LASS is available but where NX isn't
available, we have much bigger problems on our hands.

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Andrew Cooper]

On 21/06/2025 12:18 am, Sohil Mehta wrote:
> On 6/20/2025 4:08 PM, Andrew Cooper wrote:
>>> But, the resulting code is wonky. It needs to do something more like this:
>>>
>>> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
>>> 		return false;
>>>
>>> 	if (error_code & X86_PF_INSTR))
>>> 		return __emulate_vsyscall(regs, address);
>> To do this, LASS needs a proper interlink against NX || SMEP.
>>
>> If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
>> meaning that fetches are reported as plain reads.
>>
>> This leads to some fun corner cases in SMAP and now LASS too for virt.
> Maybe I am missing something, but LASS works pre-paging so it wouldn't
> generate a PF, right?

Oh right, yes.  This is a preexisting bug in vsyscall #PF handling.

It simply became obvious with Dave's suggested rearrangement.

~Andrew

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/20/2025 10:31 AM, Kirill A. Shutemov wrote:
> On Fri, Jun 20, 2025 at 08:36:30AM -0700, Xin Li wrote:
>> On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
>>> diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
>>> index 250c10627ab3..9574c198fc08 100644
>>> --- a/arch/x86/Kconfig.cpufeatures
>>> +++ b/arch/x86/Kconfig.cpufeatures
>>> @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
>>>    	def_bool y
>>>    	depends on !X86_64
>>> +config X86_DISABLED_FEATURE_LASS
>>> +	def_bool y
>>> +	depends on !X86_64
>>> +
>>>    config X86_DISABLED_FEATURE_PKU
>>>    	def_bool y
>>>    	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS
>>
>> You don't need to add X86_DISABLED_FEATURE_LASS, because the LASS code
>> is NOT optional at build time, i.e., you now don't have CONFIG_X86_LASS.
> 
> Hmm. But it is optional. It depends on CONFIG_X86_64. I don't think we
> want it to be advertised on 32-bit kernels.
> 

I kind of ignore 32-bit...

But I simply hate adding a disabled feature that depends on !X86_64;
x86_64 has a broad scope, and new CPU features are often intentionally
not enabled for 32-bit.

(X86_DISABLED_FEATURE_PCID is the only one before LASS)

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On June 20, 2025 4:46:21 PM PDT, Xin Li <xin@zytor.com> wrote:
>On 6/20/2025 10:31 AM, Kirill A. Shutemov wrote:
>> On Fri, Jun 20, 2025 at 08:36:30AM -0700, Xin Li wrote:
>>> On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
>>>> diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures
>>>> index 250c10627ab3..9574c198fc08 100644
>>>> --- a/arch/x86/Kconfig.cpufeatures
>>>> +++ b/arch/x86/Kconfig.cpufeatures
>>>> @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID
>>>>    	def_bool y
>>>>    	depends on !X86_64
>>>> +config X86_DISABLED_FEATURE_LASS
>>>> +	def_bool y
>>>> +	depends on !X86_64
>>>> +
>>>>    config X86_DISABLED_FEATURE_PKU
>>>>    	def_bool y
>>>>    	depends on !X86_INTEL_MEMORY_PROTECTION_KEYS
>>> 
>>> You don't need to add X86_DISABLED_FEATURE_LASS, because the LASS code
>>> is NOT optional at build time, i.e., you now don't have CONFIG_X86_LASS.
>> 
>> Hmm. But it is optional. It depends on CONFIG_X86_64. I don't think we
>> want it to be advertised on 32-bit kernels.
>> 
>
>I kind of ignore 32-bit...
>
>But I simply hate adding a disabled feature that depends on !X86_64;
>x86_64 has a broad scope, and new CPU features are often intentionally
>not enabled for 32-bit.
>
>(X86_DISABLED_FEATURE_PCID is the only one before LASS)
>
>

More importantly, it is wrong. 

The 32-bit build can depend on this feature not existing, therefore it SHOULD be listed as a disabled feature.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On 2025-06-20 17:45, H. Peter Anvin wrote:
>>
>> But I simply hate adding a disabled feature that depends on !X86_64;
>> x86_64 has a broad scope, and new CPU features are often intentionally
>> not enabled for 32-bit.
>>
>> (X86_DISABLED_FEATURE_PCID is the only one before LASS)
> 
> More importantly, it is wrong.
> 
> The 32-bit build can depend on this feature not existing, therefore it SHOULD be listed as a disabled feature.
> 

Ok, that was word salad. What I meant was that the original patch is 
correct, and we SHOULD have this as a disabled feature.

The reason is that it reduces the need to explicitly test for 32/64 bits 
for features that don't exist on 32 bits. When they are flagged as 
disabled, they get filtered out *at compile time*.

	-hpa

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[H. Peter Anvin]

On June 20, 2025 4:21:38 PM PDT, Dave Hansen <dave.hansen@intel.com> wrote:
>On 6/20/25 16:08, Andrew Cooper wrote:
>>> But, the resulting code is wonky. It needs to do something more like this:
>>>
>>> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
>>> 		return false;
>>>
>>> 	if (error_code & X86_PF_INSTR))
>>> 		return __emulate_vsyscall(regs, address);
>> To do this, LASS needs a proper interlink against NX || SMEP.
>> 
>> If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
>> meaning that fetches are reported as plain reads.
>Interesting point.
>
>I think the easiest way to do this is just make a cpuid_deps[] entry for
>LASS and NX. If there's a CPU where LASS is available but where NX isn't
>available, we have much bigger problems on our hands.

Indeed. There is exactly zero reason to support this case.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 08:29:43PM +0200, Borislav Petkov wrote:
> On Fri, Jun 20, 2025 at 08:33:36PM +0300, Kirill A. Shutemov wrote:
> > What is current policy around it ?
> 
> Documentation/arch/x86/cpuinfo.rst
> 
> > I think it is useful to advertise security features in cpuinfo.
> 
> Because who's going to consume them?
> 
> Don't get me wrong - I am trying to see whether the whole use case for this is
> well thought out. Because it becomes an ABI.
> 
> But if no one is going to use it, why bother?
> 
> Arguably, for this thing the argument would be - as it is put in that file
> above:
> 
> "So, the current use of /proc/cpuinfo is to show features which the
> kernel has *enabled* and *supports*."
> 
> as it has been enabled by machinery.
> 
> So that's ok. I'm just making sure we're on the same page and you're not
> aiming at something completely different with this.

What about this:

LASS provides protection against a class of speculative attacks, such as
SLAM[1]. Add the "lass" flag to /proc/cpuinfo to indicate that the feature
is supported by hardware and enabled by the kernel. This allows userspace
to determine if the setup is secure against such attacks.

[1] https://download.vusec.net/papers/slam_sp24.pdf

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Mon, Jun 23, 2025 at 11:17:02AM +0300, Kirill A. Shutemov wrote:
> What about this:
> 
> LASS provides protection against a class of speculative attacks, such as
> SLAM[1]. Add the "lass" flag to /proc/cpuinfo to indicate that the feature
> is supported by hardware and enabled by the kernel. This allows userspace
> to determine if the setup is secure against such attacks.

Yeah, thanks.

I'm still not fully on board with userspace determining whether they're
mitigated or not but that's a general problem with our mitigations.

Also, I haven't looked at the patchset yet but I think it should be also
adding code to bugs.c to make all those vulns which it addresses, report that
they're mitigated by LASS now in

grep -r . /sys/devices/system/cpu/vulnerabilities/

output.

Which makes your cpuinfo flag not really needed as we already have a special
method for the mitigations reporting.

But ok, it has gotten kernel enablement so stating so in cpuinfo is ok.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Kirill A. Shutemov]

On Fri, Jun 20, 2025 at 04:21:38PM -0700, Dave Hansen wrote:
> On 6/20/25 16:08, Andrew Cooper wrote:
> >> But, the resulting code is wonky. It needs to do something more like this:
> >>
> >> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
> >> 		return false;
> >>
> >> 	if (error_code & X86_PF_INSTR))
> >> 		return __emulate_vsyscall(regs, address);
> > To do this, LASS needs a proper interlink against NX || SMEP.
> > 
> > If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
> > meaning that fetches are reported as plain reads.
> Interesting point.
> 
> I think the easiest way to do this is just make a cpuid_deps[] entry for
> LASS and NX. If there's a CPU where LASS is available but where NX isn't
> available, we have much bigger problems on our hands.

I am not sure what I suppose to do here.

Sohil pointed out that with LASS we get #GP on vsyscall, not #PF and PFEC
is not relevant for LASS.

So, IIUC, that's dependency of vsyscall PF on NX. Do we want to disable
vsyscall on boot if NX is not available?

BTW, why do we even support !NX on X86_64? Is there such HW?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Andrew Cooper]

On 23/06/2025 1:41 pm, Kirill A. Shutemov wrote:
> On Fri, Jun 20, 2025 at 04:21:38PM -0700, Dave Hansen wrote:
>> On 6/20/25 16:08, Andrew Cooper wrote:
>>>> But, the resulting code is wonky. It needs to do something more like this:
>>>>
>>>> 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
>>>> 		return false;
>>>>
>>>> 	if (error_code & X86_PF_INSTR))
>>>> 		return __emulate_vsyscall(regs, address);
>>> To do this, LASS needs a proper interlink against NX || SMEP.
>>>
>>> If neither NX nor SMEP are active, the CPU does not report X86_PF_INSTR,
>>> meaning that fetches are reported as plain reads.
>> Interesting point.
>>
>> I think the easiest way to do this is just make a cpuid_deps[] entry for
>> LASS and NX. If there's a CPU where LASS is available but where NX isn't
>> available, we have much bigger problems on our hands.
> I am not sure what I suppose to do here.
>
> Sohil pointed out that with LASS we get #GP on vsyscall, not #PF and PFEC
> is not relevant for LASS.

Correct.  That was my mistake originally.

>
> So, IIUC, that's dependency of vsyscall PF on NX. Do we want to disable
> vsyscall on boot if NX is not available?
>
> BTW, why do we even support !NX on X86_64? Is there such HW?

Yes.  Early P4 steppings had no NX at all.

~Andrew

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

On Mon, Jun 23, 2025 at 12:21:05PM +0200, Borislav Petkov wrote:
> On Mon, Jun 23, 2025 at 11:17:02AM +0300, Kirill A. Shutemov wrote:
> > What about this:
> > 
> > LASS provides protection against a class of speculative attacks, such as
> > SLAM[1]. Add the "lass" flag to /proc/cpuinfo to indicate that the feature
> > is supported by hardware and enabled by the kernel. This allows userspace
> > to determine if the setup is secure against such attacks.
> 
> Yeah, thanks.
> 
> I'm still not fully on board with userspace determining whether they're
> mitigated or not but that's a general problem with our mitigations.
> 
> Also, I haven't looked at the patchset yet but I think it should be also
> adding code to bugs.c to make all those vulns which it addresses, report that
> they're mitigated by LASS now in
> 
> grep -r . /sys/devices/system/cpu/vulnerabilities/
> 
> output.
> 
> Which makes your cpuinfo flag not really needed as we already have a special
> method for the mitigations reporting.
> 
> But ok, it has gotten kernel enablement so stating so in cpuinfo is ok.

Due to SLAM, we decided to postpone LAM enabling, until LASS is landed.

I am not sure if we want to add static
/sys/devices/system/cpu/vulnerabilities/slam with "Mitigation: LASS".

There might be other yet-to-be-discovered speculative attacks that LASS
mitigates. Security features have to visible to userspace independently of
known vulnerabilities.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Dave Hansen]

[-- Attachment #1: Type: text/plain, Size: 1130 bytes --]

On 6/23/25 05:41, Kirill A. Shutemov wrote:
> So, IIUC, that's dependency of vsyscall PF on NX. Do we want to disable
> vsyscall on boot if NX is not available?

Well, vsyscall=none can break old userspace, so forcing it on old
hardware doesn't seem like a great idea.

But, either way, this doesn't really appear to be a LASS issue. This code:

>         if (!(error_code & X86_PF_INSTR)) {
>                 /* Failed vsyscall read */
>                 if (vsyscall_mode == EMULATE)
>                         return false;

Is really asking the question:

	Is this #PF from an instruction fetch in the vsyscall page?

That _should_ be able to be done by comparing CR2 and regs->rip. In
fact, that's done just below anyway:

	WARN_ON_ONCE(address != regs->ip);

So I think we can fix this up with something like the attached patch
which just drives the if() from regs->rip and make the warning NX-only.

But this code has been like this a long time and I'm 99% sure the x86
selftests poke at all these cases. I'm curious what they do on those old
P4's (or a 64-bit VM with NX turned off), but it's not super important
either way.

[-- Attachment #2: vsyscall-nx.patch --]
[-- Type: text/x-patch, Size: 1152 bytes --]

diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index c9103a6fa06e..0b0e0283994f 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -124,7 +124,8 @@ bool emulate_vsyscall(unsigned long error_code,
 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
 		return false;
 
-	if (!(error_code & X86_PF_INSTR)) {
+	/* Avoid emulation unless userspace was executing from vsyscall page: */
+	if (address != regs->ip) {
 		/* Failed vsyscall read */
 		if (vsyscall_mode == EMULATE)
 			return false;
@@ -136,13 +137,16 @@ bool emulate_vsyscall(unsigned long error_code,
 		return false;
 	}
 
+
+	/* X86_PF_INSTR is only set when NX is supported: */
+	if (cpu_feature_enabled(X86_FEATURE_NX))
+		WARN_ON_ONCE(!(error_code & X86_PF_INSTR));
+
 	/*
 	 * No point in checking CS -- the only way to get here is a user mode
 	 * trap to a high address, which means that we're in 64-bit user code.
 	 */
 
-	WARN_ON_ONCE(address != regs->ip);
-
 	if (vsyscall_mode == NONE) {
 		warn_bad_vsyscall(KERN_INFO, regs,
 				  "vsyscall attempted with vsyscall=none");

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Andrew Cooper]

On 23/06/2025 4:32 pm, Dave Hansen wrote:
> On 6/23/25 05:41, Kirill A. Shutemov wrote:
>> So, IIUC, that's dependency of vsyscall PF on NX. Do we want to disable
>> vsyscall on boot if NX is not available?
> Well, vsyscall=none can break old userspace, so forcing it on old
> hardware doesn't seem like a great idea.
>
> But, either way, this doesn't really appear to be a LASS issue. This code:
>
>>         if (!(error_code & X86_PF_INSTR)) {
>>                 /* Failed vsyscall read */
>>                 if (vsyscall_mode == EMULATE)
>>                         return false;
> Is really asking the question:
>
> 	Is this #PF from an instruction fetch in the vsyscall page?
>
> That _should_ be able to be done by comparing CR2 and regs->rip. In
> fact, that's done just below anyway:
>
> 	WARN_ON_ONCE(address != regs->ip);
>
> So I think we can fix this up with something like the attached patch
> which just drives the if() from regs->rip and make the warning NX-only.

Yeah, that looks good.  Furthermore, it means that the LASS #GP path
(patch 9) will be consistent with this path.  (i.e. both doing a
regs->rip check.)

Patch Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> if that
counts for anything.

~Andrew

RE: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Luck, Tony]

> > functions. But, the difference in usage between both of them seems very
> > subtle. Could this be easily misused?
>
> Logically there are two completely different things:
>
>       1. Touching userspace
>       2. Touching the lower half of the address space
>
> If it's only userspace in the lower half of the address space, then
> there's no controversy. But the problem obviously occurs when you want
> to touch kernel mappings in the lower half of the address space.

Why does the kernel create the mappings to poke kernel text
for ALTERNATIVE patching in the lower half of the address space?

Instead of special "we really to want to access the lower addresses"
code, wouldn't it be easier to map the "poke" virtual addresses in normal
kernel upper-half space?

-Tony

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Dave Hansen]

On 6/23/25 09:25, Luck, Tony wrote:
>>> functions. But, the difference in usage between both of them seems very
>>> subtle. Could this be easily misused?
>>
>> Logically there are two completely different things:
>>
>>       1. Touching userspace
>>       2. Touching the lower half of the address space
>>
>> If it's only userspace in the lower half of the address space, then
>> there's no controversy. But the problem obviously occurs when you want
>> to touch kernel mappings in the lower half of the address space.
> 
> Why does the kernel create the mappings to poke kernel text
> for ALTERNATIVE patching in the lower half of the address space?
> 
> Instead of special "we really to want to access the lower addresses"
> code, wouldn't it be easier to map the "poke" virtual addresses in normal
> kernel upper-half space?

The upper half of the address space is shared kernel space, right? Every
PGD has identical contents in the upper half. So if we create a mapping
there,everybody get access to it. Every mm can access it. Every
*process* can access it. It still has kernel permissions of course, but
it's still a place that everybody can get at.

The lower half is *ONLY* accessible to the local mm. In this case, only
the text poking mm. It's a natural, safe, place to create a mapping that
you want to be private and not be exploited.

So, doing it in the upper half is risky.

If we *wanted*, we could have a non-shared PGD entry in the top half of
the address space. But we'd need to reserve its address space and all
that jazz. I'm not sure it's any better than just disabling LASS
enforcement for a moment.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/20/2025 5:50 PM, H. Peter Anvin wrote:
> On 2025-06-20 17:45, H. Peter Anvin wrote:
>>>
>>> But I simply hate adding a disabled feature that depends on !X86_64;
>>> x86_64 has a broad scope, and new CPU features are often intentionally
>>> not enabled for 32-bit.
>>>
>>> (X86_DISABLED_FEATURE_PCID is the only one before LASS)
>>
>> More importantly, it is wrong.
>>
>> The 32-bit build can depend on this feature not existing, therefore it 
>> SHOULD be listed as a disabled feature.
>>
> 
> Ok, that was word salad. What I meant was that the original patch is 
> correct, and we SHOULD have this as a disabled feature.

Agreed!

> The reason is that it reduces the need to explicitly test for 32/64 bits 
> for features that don't exist on 32 bits. When they are flagged as 
> disabled, they get filtered out *at compile time*.

It's better to make it depend on X86_32 directly rather than !X86_64:

config X86_DISABLED_FEATURE_LASS
	def_bool y
	depends on X86_32


But the disabled feature list due to lack of 32-bit enabling will keep
growing until we remove 32-bit kernel code.

Wondering should we bother enforcing cpuid_deps[] on 32-bit?

IOW, turn off the feature when its dependency isn’t satisfied on 32b-it;
don’t just throw a warning and hope for the best.

Thanks!
     Xin

RE: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Luck, Tony]

> >> Logically there are two completely different things:
> >>
> >>       1. Touching userspace
> >>       2. Touching the lower half of the address space
> >>
> >> If it's only userspace in the lower half of the address space, then
> >> there's no controversy. But the problem obviously occurs when you want
> >> to touch kernel mappings in the lower half of the address space.
> >
> > Why does the kernel create the mappings to poke kernel text
> > for ALTERNATIVE patching in the lower half of the address space?
> >
> > Instead of special "we really to want to access the lower addresses"
> > code, wouldn't it be easier to map the "poke" virtual addresses in normal
> > kernel upper-half space?
>
> The upper half of the address space is shared kernel space, right? Every
> PGD has identical contents in the upper half. So if we create a mapping
> there,everybody get access to it. Every mm can access it. Every
> *process* can access it. It still has kernel permissions of course, but
> it's still a place that everybody can get at.
>
> The lower half is *ONLY* accessible to the local mm. In this case, only
> the text poking mm. It's a natural, safe, place to create a mapping that
> you want to be private and not be exploited.
>
> So, doing it in the upper half is risky.
>
> If we *wanted*, we could have a non-shared PGD entry in the top half of
> the address space. But we'd need to reserve its address space and all
> that jazz. I'm not sure it's any better than just disabling LASS
> enforcement for a moment.

Maybe it’s a thing to put on the list for "when x86 drops support for 32-bit".

Reserving a PGD entry in the kernel half of the address space for
local CPU use would be practical then. Perhaps there might be other
uses too.

-Tony

RE: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On June 23, 2025 4:13:34 PM PDT, "Luck, Tony" <tony.luck@intel.com> wrote:
>> >> Logically there are two completely different things:
>> >>
>> >>       1. Touching userspace
>> >>       2. Touching the lower half of the address space
>> >>
>> >> If it's only userspace in the lower half of the address space, then
>> >> there's no controversy. But the problem obviously occurs when you want
>> >> to touch kernel mappings in the lower half of the address space.
>> >
>> > Why does the kernel create the mappings to poke kernel text
>> > for ALTERNATIVE patching in the lower half of the address space?
>> >
>> > Instead of special "we really to want to access the lower addresses"
>> > code, wouldn't it be easier to map the "poke" virtual addresses in normal
>> > kernel upper-half space?
>>
>> The upper half of the address space is shared kernel space, right? Every
>> PGD has identical contents in the upper half. So if we create a mapping
>> there,everybody get access to it. Every mm can access it. Every
>> *process* can access it. It still has kernel permissions of course, but
>> it's still a place that everybody can get at.
>>
>> The lower half is *ONLY* accessible to the local mm. In this case, only
>> the text poking mm. It's a natural, safe, place to create a mapping that
>> you want to be private and not be exploited.
>>
>> So, doing it in the upper half is risky.
>>
>> If we *wanted*, we could have a non-shared PGD entry in the top half of
>> the address space. But we'd need to reserve its address space and all
>> that jazz. I'm not sure it's any better than just disabling LASS
>> enforcement for a moment.
>
>Maybe it’s a thing to put on the list for "when x86 drops support for 32-bit".
>
>Reserving a PGD entry in the kernel half of the address space for
>local CPU use would be practical then. Perhaps there might be other
>uses too.
>
>-Tony
>

Are we actually doing patching on more than one CPU at a time?

RE: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Luck, Tony]

> Are we actually doing patching on more than one CPU at a time?

We could call static_key_enable()/static_key_disable() for different
keys at the same time from multiple CPUs.

Do we actually do that? Probably not.

-Tony

RE: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On June 23, 2025 5:10:50 PM PDT, "Luck, Tony" <tony.luck@intel.com> wrote:
>> Are we actually doing patching on more than one CPU at a time?
>
>We could call static_key_enable()/static_key_disable() for different
>keys at the same time from multiple CPUs.
>
>Do we actually do that? Probably not.
>
>-Tony
>

But does it actually *work*?

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On June 23, 2025 10:40:59 AM PDT, Xin Li <xin@zytor.com> wrote:
>On 6/20/2025 5:50 PM, H. Peter Anvin wrote:
>> On 2025-06-20 17:45, H. Peter Anvin wrote:
>>>> 
>>>> But I simply hate adding a disabled feature that depends on !X86_64;
>>>> x86_64 has a broad scope, and new CPU features are often intentionally
>>>> not enabled for 32-bit.
>>>> 
>>>> (X86_DISABLED_FEATURE_PCID is the only one before LASS)
>>> 
>>> More importantly, it is wrong.
>>> 
>>> The 32-bit build can depend on this feature not existing, therefore it SHOULD be listed as a disabled feature.
>>> 
>> 
>> Ok, that was word salad. What I meant was that the original patch is correct, and we SHOULD have this as a disabled feature.
>
>Agreed!
>
>> The reason is that it reduces the need to explicitly test for 32/64 bits for features that don't exist on 32 bits. When they are flagged as disabled, they get filtered out *at compile time*.
>
>It's better to make it depend on X86_32 directly rather than !X86_64:
>
>config X86_DISABLED_FEATURE_LASS
>	def_bool y
>	depends on X86_32
>
>
>But the disabled feature list due to lack of 32-bit enabling will keep
>growing until we remove 32-bit kernel code.
>
>Wondering should we bother enforcing cpuid_deps[] on 32-bit?
>
>IOW, turn off the feature when its dependency isn’t satisfied on 32b-it;
>don’t just throw a warning and hope for the best.
>
>Thanks!
>    Xin
>

We should have the dependencies enforced; in fact, preferably we would enforce them at build time as well.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/23/2025 7:04 PM, H. Peter Anvin wrote:
> On June 23, 2025 10:40:59 AM PDT, Xin Li <xin@zytor.com> wrote:
>> On 6/20/2025 5:50 PM, H. Peter Anvin wrote:
>>> On 2025-06-20 17:45, H. Peter Anvin wrote:
>>>>>
>>>>> But I simply hate adding a disabled feature that depends on !X86_64;
>>>>> x86_64 has a broad scope, and new CPU features are often intentionally
>>>>> not enabled for 32-bit.
>>>>>
>>>>> (X86_DISABLED_FEATURE_PCID is the only one before LASS)
>>>>
>>>> More importantly, it is wrong.
>>>>
>>>> The 32-bit build can depend on this feature not existing, therefore it SHOULD be listed as a disabled feature.
>>>>
>>>
>>> Ok, that was word salad. What I meant was that the original patch is correct, and we SHOULD have this as a disabled feature.
>>
>> Agreed!
>>
>>> The reason is that it reduces the need to explicitly test for 32/64 bits for features that don't exist on 32 bits. When they are flagged as disabled, they get filtered out *at compile time*.
>>
>> It's better to make it depend on X86_32 directly rather than !X86_64:
>>
>> config X86_DISABLED_FEATURE_LASS
>> 	def_bool y
>> 	depends on X86_32
>>
>>
>> But the disabled feature list due to lack of 32-bit enabling will keep
>> growing until we remove 32-bit kernel code.
>>
>> Wondering should we bother enforcing cpuid_deps[] on 32-bit?
>>
>> IOW, turn off the feature when its dependency isn’t satisfied on 32b-it;
>> don’t just throw a warning and hope for the best.
>>
>> Thanks!
>>     Xin
>>
> 
> We should have the dependencies enforced; in fact, preferably we would enforce them at build time as well.
> 
> 

Yeah, sounds something we can do later :)

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Xin Li]

On 6/23/2025 9:57 PM, Xin Li wrote:
> On 6/23/2025 7:04 PM, H. Peter Anvin wrote:
>> On June 23, 2025 10:40:59 AM PDT, Xin Li <xin@zytor.com> wrote:
>>> On 6/20/2025 5:50 PM, H. Peter Anvin wrote:
>>>> On 2025-06-20 17:45, H. Peter Anvin wrote:
>>>>>>
>>>>>> But I simply hate adding a disabled feature that depends on !X86_64;
>>>>>> x86_64 has a broad scope, and new CPU features are often 
>>>>>> intentionally
>>>>>> not enabled for 32-bit.
>>>>>>
>>>>>> (X86_DISABLED_FEATURE_PCID is the only one before LASS)
>>>>>
>>>>> More importantly, it is wrong.
>>>>>
>>>>> The 32-bit build can depend on this feature not existing, therefore 
>>>>> it SHOULD be listed as a disabled feature.
>>>>>
>>>>
>>>> Ok, that was word salad. What I meant was that the original patch is 
>>>> correct, and we SHOULD have this as a disabled feature.
>>>
>>> Agreed!
>>>
>>>> The reason is that it reduces the need to explicitly test for 32/64 
>>>> bits for features that don't exist on 32 bits. When they are flagged 
>>>> as disabled, they get filtered out *at compile time*.
>>>
>>> It's better to make it depend on X86_32 directly rather than !X86_64:
>>>
>>> config X86_DISABLED_FEATURE_LASS
>>>     def_bool y
>>>     depends on X86_32
>>>
>>>
>>> But the disabled feature list due to lack of 32-bit enabling will keep
>>> growing until we remove 32-bit kernel code.
>>>
>>> Wondering should we bother enforcing cpuid_deps[] on 32-bit?
>>>
>>> IOW, turn off the feature when its dependency isn’t satisfied on 32b-it;
>>> don’t just throw a warning and hope for the best.
>>>
>>> Thanks!
>>>     Xin
>>>
>>
>> We should have the dependencies enforced; in fact, preferably we would 
>> enforce them at build time as well.
>>
>>
> 
> Yeah, sounds something we can do later :)
> 

We could introduce a new Kconfig file at

     arch/x86/Kconfig.cpufeatures.disabled_on_32bit

to track all features disabled due to lack of 32-bit support.

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Kirill A. Shutemov]

On Mon, Jun 23, 2025 at 08:32:53AM -0700, Dave Hansen wrote:
> On 6/23/25 05:41, Kirill A. Shutemov wrote:
> > So, IIUC, that's dependency of vsyscall PF on NX. Do we want to disable
> > vsyscall on boot if NX is not available?
> 
> Well, vsyscall=none can break old userspace, so forcing it on old
> hardware doesn't seem like a great idea.
> 
> But, either way, this doesn't really appear to be a LASS issue. This code:
> 
> >         if (!(error_code & X86_PF_INSTR)) {
> >                 /* Failed vsyscall read */
> >                 if (vsyscall_mode == EMULATE)
> >                         return false;
> 
> Is really asking the question:
> 
> 	Is this #PF from an instruction fetch in the vsyscall page?
> 
> That _should_ be able to be done by comparing CR2 and regs->rip. In
> fact, that's done just below anyway:
> 
> 	WARN_ON_ONCE(address != regs->ip);
> 
> So I think we can fix this up with something like the attached patch
> which just drives the if() from regs->rip and make the warning NX-only.

Looks good to me.

Do you want me to include it into this patchset or will you apply it
separately?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 07/16] x86/vsyscall: Reorganize the #PF emulation code

[Dave Hansen]

On 6/24/25 04:37, Kirill A. Shutemov wrote:
> Do you want me to include it into this patchset or will you apply it
> separately?

Actually, if you want to break it out and just submit it separately,
I'll probably just apply it.

[PATCH] x86/vsyscall: Do not require X86_PF_INSTR to emulate vsyscall

[Kirill A. Shutemov]

emulate_vsyscall() expects to see X86_PF_INSTR in PFEC on a vsyscall
page fault, but the CPU does not report X86_PF_INSTR if neither
X86_FEATURE_NX nor X86_FEATURE_SMEP are enabled.

X86_FEATURE_NX should be enabled on nearly all 64-bit CPUs, except for
early P4 processors that did not support this feature.

Instead of explicitly checking for X86_PF_INSTR, compare the fault
address to RIP.

On machines with X86_FEATURE_NX enabled, issue a warning if RIP is equal
to fault address but X86_PF_INSTR is absent.

Originally-by: Dave Hansen <dave.hansen@intel.com>
Link: https://lore.kernel.org/all/bd81a98b-f8d4-4304-ac55-d4151a1a77ab@intel.com
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 arch/x86/entry/vsyscall/vsyscall_64.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index c9103a6fa06e..0b0e0283994f 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -124,7 +124,8 @@ bool emulate_vsyscall(unsigned long error_code,
 	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
 		return false;
 
-	if (!(error_code & X86_PF_INSTR)) {
+	/* Avoid emulation unless userspace was executing from vsyscall page: */
+	if (address != regs->ip) {
 		/* Failed vsyscall read */
 		if (vsyscall_mode == EMULATE)
 			return false;
@@ -136,13 +137,16 @@ bool emulate_vsyscall(unsigned long error_code,
 		return false;
 	}
 
+
+	/* X86_PF_INSTR is only set when NX is supported: */
+	if (cpu_feature_enabled(X86_FEATURE_NX))
+		WARN_ON_ONCE(!(error_code & X86_PF_INSTR));
+
 	/*
 	 * No point in checking CS -- the only way to get here is a user mode
 	 * trap to a high address, which means that we're in 64-bit user code.
 	 */
 
-	WARN_ON_ONCE(address != regs->ip);
-
 	if (vsyscall_mode == NONE) {
 		warn_bad_vsyscall(KERN_INFO, regs,
 				  "vsyscall attempted with vsyscall=none");
-- 
2.47.2

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[H. Peter Anvin]

On June 20, 2025 11:14:56 AM PDT, Sohil Mehta <sohil.mehta@intel.com> wrote:
>On 6/20/2025 6:53 AM, Kirill A. Shutemov wrote:
>>  
>> +/*
>> + * The CLAC/STAC instructions toggle enforcement of X86_FEATURE_SMAP.
>> + *
>> + * X86_FEATURE_LASS requires flipping the AC flag when accessing the lower half
>> + * of the virtual address space, regardless of the _PAGE_BIT_USER bit in the
>> + * page tables. lass_clac/stac() should be used for these cases.
>> + *
>
>Is this supposed to be "regardless" or only when the _PAGE_BIT_USER bit
>it set? The way the sentence is worded it would seem that the kernel
>could always use lass_clac()/stac() since the value in _PAGE_BIT_USER
>doesn't matter.
>
>Please correct me if I am wrong, but here is my understanding:
>
>X86_FEATURE_SMAP and X86_FEATURE_LASS both complain when the kernel
>tries to access the lower half of the virtual addresses.
>
>SMAP flags an issue if _PAGE_BIT_USER is not set. LASS would #GP in both
>cases with or without the _PAGE_BIT_USER being set.
>
>However, in terms of usage, we want to use LASS specific stac()/clac()
>only when _PAGE_BIT_USER is set. Since this won't be flagged by SMAP.
>
>@Dave Hansen, you had suggested separating out the SMAP/LASS AC toggle
>functions. But, the difference in usage between both of them seems very
>subtle. Could this be easily misused?
>
>For example, there is no failure that would happen if someone
>incorrectly uses the SMAP specific clac()/stac() calls instead of the
>LASS ones.
>
>> + * Note: a barrier is implicit in alternative().
>> + */
>> +
>>  static __always_inline void clac(void)
>>  {
>> -	/* Note: a barrier is implicit in alternative() */
>>  	alternative("", "clac", X86_FEATURE_SMAP);
>>  }
>>  
>>  static __always_inline void stac(void)
>>  {
>> -	/* Note: a barrier is implicit in alternative() */
>>  	alternative("", "stac", X86_FEATURE_SMAP);
>>  }
>>  
>> +static __always_inline void lass_clac(void)
>> +{
>> +	alternative("", "clac", X86_FEATURE_LASS);
>> +}
>> +
>> +static __always_inline void lass_stac(void)
>> +{
>> +	alternative("", "stac", X86_FEATURE_LASS);
>> +}
>> +

"Regardless" is correct. LASS only considers which hemisphere the virtual address is located in, because it is explicitly designed to prevent walking the page tables in the "wrong" hemisphere and therefore speculative accesses that happen to form pointers into user space addresses will not cause TLB or cache fills that might be possible to probe.

The obvious exception is when the kernel is intentionally performing accesses on behalf of user space, which is exactly what SMAP tells the hardware already.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Mon, Jun 23, 2025 at 04:42:41PM +0300, Kirill A. Shutemov wrote:
> Due to SLAM, we decided to postpone LAM enabling, until LASS is landed.
> 
> I am not sure if we want to add static
> /sys/devices/system/cpu/vulnerabilities/slam with "Mitigation: LASS".
> 
> There might be other yet-to-be-discovered speculative attacks that LASS
> mitigates. Security features have to visible to userspace independently of
> known vulnerabilities.

... and the fact that a vuln is being mitigated by stating that in
/sys/devices/system/cpu/vulnerabilities/ needs to happen too.

I'm not talking about LAM enablement - I'm talking about adding a

SPECTRE_V1_MITIGATION_LASS

and setting that when X86_FEATURE_LASS is set so that luserspace gets told
that

"Spectre V1 : Mitigation: LASS"

or so.

Makes more sense?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Thu, Jun 26, 2025 at 05:18:37PM +0200, Borislav Petkov wrote:
> On Mon, Jun 23, 2025 at 04:42:41PM +0300, Kirill A. Shutemov wrote:
> > Due to SLAM, we decided to postpone LAM enabling, until LASS is landed.
> > 
> > I am not sure if we want to add static
> > /sys/devices/system/cpu/vulnerabilities/slam with "Mitigation: LASS".
> > 
> > There might be other yet-to-be-discovered speculative attacks that LASS
> > mitigates. Security features have to visible to userspace independently of
> > known vulnerabilities.
> 
> ... and the fact that a vuln is being mitigated by stating that in
> /sys/devices/system/cpu/vulnerabilities/ needs to happen too.
> 
> I'm not talking about LAM enablement - I'm talking about adding a
> 
> SPECTRE_V1_MITIGATION_LASS
> 
> and setting that when X86_FEATURE_LASS is set so that luserspace gets told
> that
> 
> "Spectre V1 : Mitigation: LASS"
> 
> or so.
> 
> Makes more sense?

I meant this crap, ofc:

        switch (bug) {
        case X86_BUG_CPU_MELTDOWN:
                if (boot_cpu_has(X86_FEATURE_PTI))
                        return sysfs_emit(buf, "Mitigation: PTI\n");

This should say "Mitigation: LASS" if LASS is enabled...

Which begs the question: how do LASS and PTI interact now?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Dave Hansen]

On 6/26/25 09:07, Borislav Petkov wrote:
>> Makes more sense?
> I meant this crap, ofc:
> 
>         switch (bug) {
>         case X86_BUG_CPU_MELTDOWN:
>                 if (boot_cpu_has(X86_FEATURE_PTI))
>                         return sysfs_emit(buf, "Mitigation: PTI\n");
> 
> This should say "Mitigation: LASS" if LASS is enabled...
> 
> Which begs the question: how do LASS and PTI interact now?

Maybe my babbling about LASS mitigation Meltdown was ill considered. It
seems that I've just muddied the waters.

All the real LASS-capable hardware also has RDCL_NO=1 which is the
_actual_ x86 Meltdown mitigation. Those systems are not vulnerable to
Meltdown in the first place.

They should say: "Not affected" for Meltdown, both before and after LASS.

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Kirill A. Shutemov]

On Thu, Jun 26, 2025 at 10:21:23AM -0700, Dave Hansen wrote:
> On 6/26/25 09:07, Borislav Petkov wrote:
> >> Makes more sense?
> > I meant this crap, ofc:
> > 
> >         switch (bug) {
> >         case X86_BUG_CPU_MELTDOWN:
> >                 if (boot_cpu_has(X86_FEATURE_PTI))
> >                         return sysfs_emit(buf, "Mitigation: PTI\n");
> > 
> > This should say "Mitigation: LASS" if LASS is enabled...
> > 
> > Which begs the question: how do LASS and PTI interact now?
> 
> Maybe my babbling about LASS mitigation Meltdown was ill considered. It
> seems that I've just muddied the waters.
> 
> All the real LASS-capable hardware also has RDCL_NO=1 which is the
> _actual_ x86 Meltdown mitigation. Those systems are not vulnerable to
> Meltdown in the first place.
> 
> They should say: "Not affected" for Meltdown, both before and after LASS.

Right. To best of my knowledge, SLAM is the only known vulnerability LASS
fixes directly so far.

So, we want an entry for SLAM?

I don't think it is very useful as we don't allow LAM if LASS is missing.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Borislav Petkov]

On Fri, Jun 27, 2025 at 01:25:12PM +0300, Kirill A. Shutemov wrote:
> So, we want an entry for SLAM?
> 
> I don't think it is very useful as we don't allow LAM if LASS is missing.

Nah, it is all clear now.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCHv6 01/16] x86/cpu: Enumerate the LASS feature bits

[Dave Hansen]

On 6/27/25 03:25, Kirill A. Shutemov wrote:
> So, we want an entry for SLAM?

SLAM wasn't ever a practical problem anywhere, so I don't think we need
to do anything on top of what we already did.