[RFC PATCH 0/7] arm64: Make EFI calls preemptible

[RFC PATCH 0/7] arm64: Make EFI calls preemptible

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

The arm64 port permits the use of the baseline FP/SIMD register file in
kernel mode, and no longer requires preemption to be disabled. Now that
the EFI spec is being clarified to state that EFI runtime services may
only use baseline FP/SIMD, the fact that EFI may code may use FP/SIMD
registers (while executing at the same privilege level as the kernel) is
no longer a reason to disable preemption when invoking them.

This means that the only remaining reason for disabling preemption is
the fact that the active mm is swapped out and replaced with efi_mm in a
way that is hidden from the scheduler, and so scheduling is not
supported currently. However, given that virtually all (*) EFI runtime
calls are made from the efi_rts_wq workqueue, the efi_mm can simply be
loaded into the workqueue worker kthread while the call is in progress,
and this does not require preemption to be disabled.

Note that this is only a partial solution in terms of RT guarantees,
given that the runtime services execute at the same privilege level as
the kernel, and can therefore disable interrupts (and therefore
preemption) directly. But it should prevent scheduling latency spikes
for EFI calls that simply take a long time to run to completion.

(*) only efi_reset_system() and EFI pstore invoke EFI runtime services
    without going through the workqueue, and the latter only when saving
    a kernel oops log to the EFI varstore

Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>

Ard Biesheuvel (7):
  efi: Add missing static initializer for efi_mm::cpus_allowed_lock
  efi/runtime: Return success/failure from arch_efi_call_virt_setup()
  efi/runtime: Deal with arch_efi_call_virt_setup() returning failure
  arm64/fpsimd: Don't warn when EFI execution context is preemptible
  arm64/efi: Use a semaphore to protect the EFI stack and FP/SIMD state
  arm64/efi: Move uaccess en/disable out of efi_set_pgd()
  arm64/efi: Call EFI runtime services without disabling preemption

 arch/arm/include/asm/efi.h              |  2 +-
 arch/arm64/include/asm/efi.h            |  5 +--
 arch/arm64/kernel/efi.c                 | 34 ++++++++++++++++----
 arch/arm64/kernel/fpsimd.c              |  4 +--
 arch/loongarch/include/asm/efi.h        |  2 +-
 arch/riscv/include/asm/efi.h            |  2 +-
 arch/x86/include/asm/efi.h              |  2 +-
 arch/x86/platform/efi/efi_32.c          |  3 +-
 arch/x86/platform/efi/efi_64.c          |  3 +-
 arch/x86/platform/uv/bios_uv.c          |  3 +-
 drivers/firmware/efi/efi.c              |  3 ++
 drivers/firmware/efi/riscv-runtime.c    |  3 +-
 drivers/firmware/efi/runtime-wrappers.c | 20 ++++++++----
 include/linux/efi.h                     |  8 ++---
 14 files changed, 63 insertions(+), 31 deletions(-)


base-commit: e04796c8b5980700c78f2fd1b29724afd80dcc62
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 1/7] efi: Add missing static initializer for efi_mm::cpus_allowed_lock

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

Initialize the cpus_allowed_lock struct member of efi_mm.

Cc: <stable@vger.kernel.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 drivers/firmware/efi/efi.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 7309394b8fc9..59a56661937c 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -73,6 +73,9 @@ struct mm_struct efi_mm = {
 	.page_table_lock	= __SPIN_LOCK_UNLOCKED(efi_mm.page_table_lock),
 	.mmlist			= LIST_HEAD_INIT(efi_mm.mmlist),
 	.cpu_bitmap		= { [BITS_TO_LONGS(NR_CPUS)] = 0},
+#ifdef CONFIG_SCHED_MM_CID
+	.cpus_allowed_lock	= __RAW_SPIN_LOCK_UNLOCKED(efi_mm.cpus_allowed_lock),
+#endif
 };
 
 struct workqueue_struct *efi_rts_wq;
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 2/7] efi/runtime: Return success/failure from arch_efi_call_virt_setup()

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

Permit the arch glue to signal failure from arch_efi_call_virt_setup().
This permits the use of sleeping locks in the call wrappers, and this
will allow EFI runtime services to be invoked without the need for
disabling preemption.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm/include/asm/efi.h           | 2 +-
 arch/arm64/include/asm/efi.h         | 2 +-
 arch/arm64/kernel/efi.c              | 3 ++-
 arch/loongarch/include/asm/efi.h     | 2 +-
 arch/riscv/include/asm/efi.h         | 2 +-
 arch/x86/include/asm/efi.h           | 2 +-
 arch/x86/platform/efi/efi_32.c       | 3 ++-
 arch/x86/platform/efi/efi_64.c       | 3 ++-
 drivers/firmware/efi/riscv-runtime.c | 3 ++-
 9 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/arch/arm/include/asm/efi.h b/arch/arm/include/asm/efi.h
index e408399d5f0e..0809a69bb579 100644
--- a/arch/arm/include/asm/efi.h
+++ b/arch/arm/include/asm/efi.h
@@ -23,7 +23,7 @@ void arm_efi_init(void);
 int efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md);
 int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md, bool);
 
-#define arch_efi_call_virt_setup()	efi_virtmap_load()
+#define arch_efi_call_virt_setup()	(efi_virtmap_load(), true)
 #define arch_efi_call_virt_teardown()	efi_virtmap_unload()
 
 #ifdef CONFIG_CPU_TTBR0_PAN
diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index bcd5622aa096..decf87777f57 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -37,7 +37,7 @@ int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md,
 extern u64 *efi_rt_stack_top;
 efi_status_t __efi_rt_asm_wrapper(void *, const char *, ...);
 
-void arch_efi_call_virt_setup(void);
+bool arch_efi_call_virt_setup(void);
 void arch_efi_call_virt_teardown(void);
 
 /*
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index 250e9d7c08a7..44ad5e759af4 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -166,11 +166,12 @@ asmlinkage efi_status_t efi_handle_corrupted_x18(efi_status_t s, const char *f)
 
 static DEFINE_RAW_SPINLOCK(efi_rt_lock);
 
-void arch_efi_call_virt_setup(void)
+bool arch_efi_call_virt_setup(void)
 {
 	efi_virtmap_load();
 	raw_spin_lock(&efi_rt_lock);
 	__efi_fpsimd_begin();
+	return true;
 }
 
 void arch_efi_call_virt_teardown(void)
diff --git a/arch/loongarch/include/asm/efi.h b/arch/loongarch/include/asm/efi.h
index eddc8e79b3fa..84cf2151123f 100644
--- a/arch/loongarch/include/asm/efi.h
+++ b/arch/loongarch/include/asm/efi.h
@@ -14,7 +14,7 @@ void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
 
 #define ARCH_EFI_IRQ_FLAGS_MASK  0x00000004  /* Bit 2: CSR.CRMD.IE */
 
-#define arch_efi_call_virt_setup()
+#define arch_efi_call_virt_setup()	true
 #define arch_efi_call_virt_teardown()
 
 #define EFI_ALLOC_ALIGN		SZ_64K
diff --git a/arch/riscv/include/asm/efi.h b/arch/riscv/include/asm/efi.h
index 46a355913b27..a7b4d719e7be 100644
--- a/arch/riscv/include/asm/efi.h
+++ b/arch/riscv/include/asm/efi.h
@@ -40,7 +40,7 @@ static inline unsigned long efi_get_kimg_min_align(void)
 
 #define EFI_KIMG_PREFERRED_ADDRESS	efi_get_kimg_min_align()
 
-void arch_efi_call_virt_setup(void);
+bool arch_efi_call_virt_setup(void);
 void arch_efi_call_virt_teardown(void);
 
 unsigned long stext_offset(void);
diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index f227a70ac91f..879c8402e024 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -140,7 +140,7 @@ extern void efi_delete_dummy_variable(void);
 extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr);
 extern void efi_free_boot_services(void);
 
-void arch_efi_call_virt_setup(void);
+bool arch_efi_call_virt_setup(void);
 void arch_efi_call_virt_teardown(void);
 
 extern u64 efi_setup;
diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c
index b2cc7b4552a1..215f16ce84ab 100644
--- a/arch/x86/platform/efi/efi_32.c
+++ b/arch/x86/platform/efi/efi_32.c
@@ -141,10 +141,11 @@ void __init efi_runtime_update_mappings(void)
 	}
 }
 
-void arch_efi_call_virt_setup(void)
+bool arch_efi_call_virt_setup(void)
 {
 	efi_fpu_begin();
 	firmware_restrict_branch_speculation_start();
+	return true;
 }
 
 void arch_efi_call_virt_teardown(void)
diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index ac57259a432b..023368e9698a 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -445,12 +445,13 @@ static void efi_leave_mm(void)
 	switch_mm(&efi_mm, efi_prev_mm, NULL);
 }
 
-void arch_efi_call_virt_setup(void)
+bool arch_efi_call_virt_setup(void)
 {
 	efi_sync_low_kernel_mappings();
 	efi_fpu_begin();
 	firmware_restrict_branch_speculation_start();
 	efi_enter_mm();
+	return true;
 }
 
 void arch_efi_call_virt_teardown(void)
diff --git a/drivers/firmware/efi/riscv-runtime.c b/drivers/firmware/efi/riscv-runtime.c
index fa71cd898120..07e04b8f982a 100644
--- a/drivers/firmware/efi/riscv-runtime.c
+++ b/drivers/firmware/efi/riscv-runtime.c
@@ -142,10 +142,11 @@ static void efi_virtmap_unload(void)
 	preempt_enable();
 }
 
-void arch_efi_call_virt_setup(void)
+bool arch_efi_call_virt_setup(void)
 {
 	sync_kernel_mappings(efi_mm.pgd);
 	efi_virtmap_load();
+	return true;
 }
 
 void arch_efi_call_virt_teardown(void)
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 3/7] efi/runtime: Deal with arch_efi_call_virt_setup() returning failure

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

Deal with arch_efi_call_virt_setup() returning failure, by giving up and
returning an appropriate error code to the caller.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/x86/platform/uv/bios_uv.c          |  3 ++-
 drivers/firmware/efi/runtime-wrappers.c | 20 +++++++++++++-------
 include/linux/efi.h                     |  8 ++++----
 3 files changed, 19 insertions(+), 12 deletions(-)

diff --git a/arch/x86/platform/uv/bios_uv.c b/arch/x86/platform/uv/bios_uv.c
index bf31af3d32d6..a442bbe5b1c2 100644
--- a/arch/x86/platform/uv/bios_uv.c
+++ b/arch/x86/platform/uv/bios_uv.c
@@ -32,7 +32,8 @@ static s64 __uv_bios_call(enum uv_bios_cmd which, u64 a1, u64 a2, u64 a3,
 		 */
 		return BIOS_STATUS_UNIMPLEMENTED;
 
-	ret = efi_call_virt_pointer(tab, function, (u64)which, a1, a2, a3, a4, a5);
+	ret = efi_call_virt_pointer(tab, function, BIOS_STATUS_UNIMPLEMENTED,
+				    (u64)which, a1, a2, a3, a4, a5);
 
 	return ret;
 }
diff --git a/drivers/firmware/efi/runtime-wrappers.c b/drivers/firmware/efi/runtime-wrappers.c
index 708b777857d3..82a27b414485 100644
--- a/drivers/firmware/efi/runtime-wrappers.c
+++ b/drivers/firmware/efi/runtime-wrappers.c
@@ -219,7 +219,10 @@ static void __nocfi efi_call_rts(struct work_struct *work)
 	efi_status_t status = EFI_NOT_FOUND;
 	unsigned long flags;
 
-	arch_efi_call_virt_setup();
+	if (!arch_efi_call_virt_setup()) {
+		status = EFI_NOT_READY;
+		goto out;
+	}
 	flags = efi_call_virt_save_flags();
 
 	switch (efi_rts_work.efi_rts_id) {
@@ -308,6 +311,7 @@ static void __nocfi efi_call_rts(struct work_struct *work)
 	efi_call_virt_check_flags(flags, efi_rts_work.caller);
 	arch_efi_call_virt_teardown();
 
+out:
 	efi_rts_work.status = status;
 	complete(&efi_rts_work.efi_rts_comp);
 }
@@ -444,8 +448,8 @@ virt_efi_set_variable_nb(efi_char16_t *name, efi_guid_t *vendor, u32 attr,
 	if (down_trylock(&efi_runtime_lock))
 		return EFI_NOT_READY;
 
-	status = efi_call_virt_pointer(efi.runtime, set_variable, name, vendor,
-				       attr, data_size, data);
+	status = efi_call_virt_pointer(efi.runtime, set_variable, EFI_NOT_READY,
+				       name, vendor, attr, data_size, data);
 	up(&efi_runtime_lock);
 	return status;
 }
@@ -481,9 +485,9 @@ virt_efi_query_variable_info_nb(u32 attr, u64 *storage_space,
 	if (down_trylock(&efi_runtime_lock))
 		return EFI_NOT_READY;
 
-	status = efi_call_virt_pointer(efi.runtime, query_variable_info, attr,
-				       storage_space, remaining_space,
-				       max_variable_size);
+	status = efi_call_virt_pointer(efi.runtime, query_variable_info,
+				       EFI_NOT_READY, attr, storage_space,
+				       remaining_space, max_variable_size);
 	up(&efi_runtime_lock);
 	return status;
 }
@@ -509,12 +513,14 @@ virt_efi_reset_system(int reset_type, efi_status_t status,
 		return;
 	}
 
-	arch_efi_call_virt_setup();
+	if (!arch_efi_call_virt_setup())
+		goto out;
 	efi_rts_work.efi_rts_id = EFI_RESET_SYSTEM;
 	arch_efi_call_virt(efi.runtime, reset_system, reset_type, status,
 			   data_size, data);
 	arch_efi_call_virt_teardown();
 
+out:
 	up(&efi_runtime_lock);
 }
 
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 7d63d1d75f22..13aff30be3a9 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1179,19 +1179,19 @@ static inline void efi_check_for_embedded_firmwares(void) { }
  *    Restores the usual kernel environment once the call has returned.
  */
 
-#define efi_call_virt_pointer(p, f, args...)				\
+#define efi_call_virt_pointer(p, f, busy, args...)			\
 ({									\
-	typeof((p)->f(args)) __s;					\
+	typeof((p)->f(args)) __s = (busy);				\
 	unsigned long __flags;						\
 									\
-	arch_efi_call_virt_setup();					\
+	if (!arch_efi_call_virt_setup()) goto __out;			\
 									\
 	__flags = efi_call_virt_save_flags();				\
 	__s = arch_efi_call_virt(p, f, args);				\
 	efi_call_virt_check_flags(__flags, NULL);			\
 									\
 	arch_efi_call_virt_teardown();					\
-									\
+__out:									\
 	__s;								\
 })
 
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 4/7] arm64/fpsimd: Don't warn when EFI execution context is preemptible

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

Kernel mode FP/SIMD no longer requires preemption to be disabled, so
only warn on uses of FP/SIMD from preemptible context if the fallback
path is taken for cases where kernel mode NEON would not be allowed
otherwise.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/kernel/fpsimd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index ae2ea0196030..47a8706e26b7 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -1975,11 +1975,11 @@ void __efi_fpsimd_begin(void)
 	if (!system_supports_fpsimd())
 		return;
 
-	WARN_ON(preemptible());
-
 	if (may_use_simd()) {
 		kernel_neon_begin();
 	} else {
+		WARN_ON(preemptible());
+
 		/*
 		 * If !efi_sve_state, SVE can't be in use yet and doesn't need
 		 * preserving:
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 5/7] arm64/efi: Use a semaphore to protect the EFI stack and FP/SIMD state

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

Replace the spinlock in the arm64 glue code with a semaphore, so that
the CPU can preempted while running the EFI runtime service.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/kernel/efi.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index 44ad5e759af4..d01ae156bb63 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -164,12 +164,19 @@ asmlinkage efi_status_t efi_handle_corrupted_x18(efi_status_t s, const char *f)
 	return s;
 }
 
-static DEFINE_RAW_SPINLOCK(efi_rt_lock);
+static DEFINE_SEMAPHORE(efi_rt_lock, 1);
 
 bool arch_efi_call_virt_setup(void)
 {
+	/*
+	 * This might be called from a non-sleepable context so try to take the
+	 * lock but don't block on it. This should never occur in practice, as
+	 * all EFI runtime calls are serialized under the efi_runtime_lock.
+	 */
+	if (WARN_ON(down_trylock(&efi_rt_lock)))
+		return false;
+
 	efi_virtmap_load();
-	raw_spin_lock(&efi_rt_lock);
 	__efi_fpsimd_begin();
 	return true;
 }
@@ -177,8 +184,8 @@ bool arch_efi_call_virt_setup(void)
 void arch_efi_call_virt_teardown(void)
 {
 	__efi_fpsimd_end();
-	raw_spin_unlock(&efi_rt_lock);
 	efi_virtmap_unload();
+	up(&efi_rt_lock);
 }
 
 asmlinkage u64 *efi_rt_stack_top __ro_after_init;
-- 
2.49.0.1101.gccaa498523-goog

[RFC PATCH 6/7] arm64/efi: Move uaccess en/disable out of efi_set_pgd()

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

efi_set_pgd() will no longer be called when invoking EFI runtime
services via the efi_rts_wq work queue, but the uaccess en/disable are
still needed when using PAN emulation using TTBR0 switching. So move
these into the callers.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/include/asm/efi.h | 3 ---
 arch/arm64/kernel/efi.c      | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index decf87777f57..abe9176a3a23 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -132,15 +132,12 @@ static inline void efi_set_pgd(struct mm_struct *mm)
 			 * exception when invoking the EFI run-time services.
 			 */
 			update_saved_ttbr0(current, mm);
-			uaccess_ttbr0_enable();
-			post_ttbr_update_workaround();
 		} else {
 			/*
 			 * Defer the switch to the current thread's TTBR0_EL1
 			 * until uaccess_enable(). Restore the current
 			 * thread's saved ttbr0 corresponding to its active_mm
 			 */
-			uaccess_ttbr0_disable();
 			update_saved_ttbr0(current, current->active_mm);
 		}
 	}
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index d01ae156bb63..5d188c6c44d7 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -177,6 +177,8 @@ bool arch_efi_call_virt_setup(void)
 		return false;
 
 	efi_virtmap_load();
+	uaccess_ttbr0_enable();
+	post_ttbr_update_workaround();
 	__efi_fpsimd_begin();
 	return true;
 }
@@ -185,6 +187,7 @@ void arch_efi_call_virt_teardown(void)
 {
 	__efi_fpsimd_end();
 	efi_virtmap_unload();
+	uaccess_ttbr0_disable();
 	up(&efi_rt_lock);
 }
 
-- 
2.49.0.1101.gccaa498523-goog

Re: [RFC PATCH 6/7] arm64/efi: Move uaccess en/disable out of efi_set_pgd()

[Will Deacon]

On Wed, May 14, 2025 at 07:43:46PM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb@kernel.org>
> 
> efi_set_pgd() will no longer be called when invoking EFI runtime
> services via the efi_rts_wq work queue, but the uaccess en/disable are
> still needed when using PAN emulation using TTBR0 switching. So move
> these into the callers.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
>  arch/arm64/include/asm/efi.h | 3 ---
>  arch/arm64/kernel/efi.c      | 3 +++
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> index decf87777f57..abe9176a3a23 100644
> --- a/arch/arm64/include/asm/efi.h
> +++ b/arch/arm64/include/asm/efi.h
> @@ -132,15 +132,12 @@ static inline void efi_set_pgd(struct mm_struct *mm)
>  			 * exception when invoking the EFI run-time services.
>  			 */
>  			update_saved_ttbr0(current, mm);
> -			uaccess_ttbr0_enable();
> -			post_ttbr_update_workaround();
>  		} else {
>  			/*
>  			 * Defer the switch to the current thread's TTBR0_EL1
>  			 * until uaccess_enable(). Restore the current
>  			 * thread's saved ttbr0 corresponding to its active_mm
>  			 */
> -			uaccess_ttbr0_disable();
>  			update_saved_ttbr0(current, current->active_mm);
>  		}
>  	}
> diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
> index d01ae156bb63..5d188c6c44d7 100644
> --- a/arch/arm64/kernel/efi.c
> +++ b/arch/arm64/kernel/efi.c
> @@ -177,6 +177,8 @@ bool arch_efi_call_virt_setup(void)
>  		return false;
>  
>  	efi_virtmap_load();
> +	uaccess_ttbr0_enable();
> +	post_ttbr_update_workaround();
>  	__efi_fpsimd_begin();
>  	return true;
>  }
> @@ -185,6 +187,7 @@ void arch_efi_call_virt_teardown(void)
>  {
>  	__efi_fpsimd_end();
>  	efi_virtmap_unload();
> +	uaccess_ttbr0_disable();

Moving this after updating the saved TTBR0 isn't great for SWPAN, as it
means that if we take an exception (e.g. an IRQ) before calling
uaccess_ttbr0_disable() then I think we'll end up running with the user
page-table installed briefly in TTBR0 which SWPAN is supposed to prevent
outside of genuine uaccess sections.

Will

Re: [RFC PATCH 6/7] arm64/efi: Move uaccess en/disable out of efi_set_pgd()

[Ard Biesheuvel]

On Fri, 11 Jul 2025 at 23:42, Will Deacon <will@kernel.org> wrote:
>
> On Wed, May 14, 2025 at 07:43:46PM +0200, Ard Biesheuvel wrote:
> > From: Ard Biesheuvel <ardb@kernel.org>
> >
> > efi_set_pgd() will no longer be called when invoking EFI runtime
> > services via the efi_rts_wq work queue, but the uaccess en/disable are
> > still needed when using PAN emulation using TTBR0 switching. So move
> > these into the callers.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > ---
> >  arch/arm64/include/asm/efi.h | 3 ---
> >  arch/arm64/kernel/efi.c      | 3 +++
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> > index decf87777f57..abe9176a3a23 100644
> > --- a/arch/arm64/include/asm/efi.h
> > +++ b/arch/arm64/include/asm/efi.h
> > @@ -132,15 +132,12 @@ static inline void efi_set_pgd(struct mm_struct *mm)
> >                        * exception when invoking the EFI run-time services.
> >                        */
> >                       update_saved_ttbr0(current, mm);
> > -                     uaccess_ttbr0_enable();
> > -                     post_ttbr_update_workaround();
> >               } else {
> >                       /*
> >                        * Defer the switch to the current thread's TTBR0_EL1
> >                        * until uaccess_enable(). Restore the current
> >                        * thread's saved ttbr0 corresponding to its active_mm
> >                        */
> > -                     uaccess_ttbr0_disable();
> >                       update_saved_ttbr0(current, current->active_mm);
> >               }
> >       }
> > diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
> > index d01ae156bb63..5d188c6c44d7 100644
> > --- a/arch/arm64/kernel/efi.c
> > +++ b/arch/arm64/kernel/efi.c
> > @@ -177,6 +177,8 @@ bool arch_efi_call_virt_setup(void)
> >               return false;
> >
> >       efi_virtmap_load();
> > +     uaccess_ttbr0_enable();
> > +     post_ttbr_update_workaround();
> >       __efi_fpsimd_begin();
> >       return true;
> >  }
> > @@ -185,6 +187,7 @@ void arch_efi_call_virt_teardown(void)
> >  {
> >       __efi_fpsimd_end();
> >       efi_virtmap_unload();
> > +     uaccess_ttbr0_disable();
>
> Moving this after updating the saved TTBR0 isn't great for SWPAN, as it
> means that if we take an exception (e.g. an IRQ) before calling
> uaccess_ttbr0_disable() then I think we'll end up running with the user
> page-table installed briefly in TTBR0 which SWPAN is supposed to prevent
> outside of genuine uaccess sections.
>

OK, I'll respin this to take that into account.

[RFC PATCH 7/7] arm64/efi: Call EFI runtime services without disabling preemption

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

The only remaining reason why EFI runtime services are invoked with
preemption disabled is the fact that the mm is swapped out behind the
back of the context switching code.

The kernel no longer disables preemption in kernel_neon_begin().
Furthermore, the EFI spec is being clarified to explicitly state that
only baseline FP/SIMD is permitted in EFI runtime service
implementations, and so the existing kernel mode NEON context switching
code is sufficient to preserve and restore the execution context of an
in-progress EFI runtime service call.

Most EFI calls are made from the efi_rts_wq, which is serviced by a
kthread. As kthreads never return to user space, they usually don't have
an mm, and so we can use the existing infrastructure to swap in the
efi_mm while the EFI call is in progress. This is visible to the
scheduler, which will therefore reactivate the selected mm when
switching out the kthread and back in again.

Given that the EFI spec explicitly permits runtime services to be called
with interrupts enabled, firmware code is already required to tolerate
interruptions. So rather than disable preemption, disable only migration
so that EFI runtime services are less likely to cause scheduling delays.

Note, though, that the firmware executes at the same privilege level as
the kernel, and is therefore able to disable interrupts altogether.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/kernel/efi.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index 5d188c6c44d7..1c86a891f6d7 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -10,6 +10,7 @@
 #include <linux/efi.h>
 #include <linux/init.h>
 #include <linux/kmemleak.h>
+#include <linux/kthread.h>
 #include <linux/screen_info.h>
 #include <linux/vmalloc.h>
 
@@ -176,7 +177,12 @@ bool arch_efi_call_virt_setup(void)
 	if (WARN_ON(down_trylock(&efi_rt_lock)))
 		return false;
 
-	efi_virtmap_load();
+	if (preemptible() && (current->flags & PF_KTHREAD)) {
+		migrate_disable();
+		kthread_use_mm(&efi_mm);
+	} else {
+		efi_virtmap_load();
+	}
 	uaccess_ttbr0_enable();
 	post_ttbr_update_workaround();
 	__efi_fpsimd_begin();
@@ -186,7 +192,12 @@ bool arch_efi_call_virt_setup(void)
 void arch_efi_call_virt_teardown(void)
 {
 	__efi_fpsimd_end();
-	efi_virtmap_unload();
+	if (preemptible() && (current->flags & PF_KTHREAD)) {
+		kthread_unuse_mm(&efi_mm);
+		migrate_enable();
+	} else {
+		efi_virtmap_unload();
+	}
 	uaccess_ttbr0_disable();
 	up(&efi_rt_lock);
 }
-- 
2.49.0.1101.gccaa498523-goog

Re: [RFC PATCH 7/7] arm64/efi: Call EFI runtime services without disabling preemption

[Peter Zijlstra]

On Wed, May 14, 2025 at 07:43:47PM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb@kernel.org>
> 
> The only remaining reason why EFI runtime services are invoked with
> preemption disabled is the fact that the mm is swapped out behind the
> back of the context switching code.
> 
> The kernel no longer disables preemption in kernel_neon_begin().
> Furthermore, the EFI spec is being clarified to explicitly state that
> only baseline FP/SIMD is permitted in EFI runtime service
> implementations, and so the existing kernel mode NEON context switching
> code is sufficient to preserve and restore the execution context of an
> in-progress EFI runtime service call.
> 
> Most EFI calls are made from the efi_rts_wq, which is serviced by a
> kthread. As kthreads never return to user space, they usually don't have
> an mm, and so we can use the existing infrastructure to swap in the
> efi_mm while the EFI call is in progress. This is visible to the
> scheduler, which will therefore reactivate the selected mm when
> switching out the kthread and back in again.
> 
> Given that the EFI spec explicitly permits runtime services to be called
> with interrupts enabled, firmware code is already required to tolerate
> interruptions. So rather than disable preemption, disable only migration
> so that EFI runtime services are less likely to cause scheduling delays.
> 
> Note, though, that the firmware executes at the same privilege level as
> the kernel, and is therefore able to disable interrupts altogether.

Is the migrate_disable() strictly required, or just paranoia?

> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
>  arch/arm64/kernel/efi.c | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
> index 5d188c6c44d7..1c86a891f6d7 100644
> --- a/arch/arm64/kernel/efi.c
> +++ b/arch/arm64/kernel/efi.c
> @@ -10,6 +10,7 @@
>  #include <linux/efi.h>
>  #include <linux/init.h>
>  #include <linux/kmemleak.h>
> +#include <linux/kthread.h>
>  #include <linux/screen_info.h>
>  #include <linux/vmalloc.h>
>  
> @@ -176,7 +177,12 @@ bool arch_efi_call_virt_setup(void)
>  	if (WARN_ON(down_trylock(&efi_rt_lock)))
>  		return false;
>  
> -	efi_virtmap_load();
> +	if (preemptible() && (current->flags & PF_KTHREAD)) {
> +		migrate_disable();
> +		kthread_use_mm(&efi_mm);
> +	} else {
> +		efi_virtmap_load();
> +	}
>  	uaccess_ttbr0_enable();
>  	post_ttbr_update_workaround();
>  	__efi_fpsimd_begin();
> @@ -186,7 +192,12 @@ bool arch_efi_call_virt_setup(void)
>  void arch_efi_call_virt_teardown(void)
>  {
>  	__efi_fpsimd_end();
> -	efi_virtmap_unload();
> +	if (preemptible() && (current->flags & PF_KTHREAD)) {
> +		kthread_unuse_mm(&efi_mm);
> +		migrate_enable();
> +	} else {
> +		efi_virtmap_unload();
> +	}
>  	uaccess_ttbr0_disable();
>  	up(&efi_rt_lock);
>  }
> -- 
> 2.49.0.1101.gccaa498523-goog
> 

Re: [RFC PATCH 7/7] arm64/efi: Call EFI runtime services without disabling preemption

[Ard Biesheuvel]

On Fri, 11 Jul 2025 at 23:48, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, May 14, 2025 at 07:43:47PM +0200, Ard Biesheuvel wrote:
> > From: Ard Biesheuvel <ardb@kernel.org>
> >
> > The only remaining reason why EFI runtime services are invoked with
> > preemption disabled is the fact that the mm is swapped out behind the
> > back of the context switching code.
> >
> > The kernel no longer disables preemption in kernel_neon_begin().
> > Furthermore, the EFI spec is being clarified to explicitly state that
> > only baseline FP/SIMD is permitted in EFI runtime service
> > implementations, and so the existing kernel mode NEON context switching
> > code is sufficient to preserve and restore the execution context of an
> > in-progress EFI runtime service call.
> >
> > Most EFI calls are made from the efi_rts_wq, which is serviced by a
> > kthread. As kthreads never return to user space, they usually don't have
> > an mm, and so we can use the existing infrastructure to swap in the
> > efi_mm while the EFI call is in progress. This is visible to the
> > scheduler, which will therefore reactivate the selected mm when
> > switching out the kthread and back in again.
> >
> > Given that the EFI spec explicitly permits runtime services to be called
> > with interrupts enabled, firmware code is already required to tolerate
> > interruptions. So rather than disable preemption, disable only migration
> > so that EFI runtime services are less likely to cause scheduling delays.
> >
> > Note, though, that the firmware executes at the same privilege level as
> > the kernel, and is therefore able to disable interrupts altogether.
>
> Is the migrate_disable() strictly required, or just paranoia?
>

Runtime services might be polling the secure firmware for an async
completion when they are interrupted, and so I don't think it is
generally safe to assume that an interrupted EFI runtime service can
be resumed on another CPU.

Re: [RFC PATCH 7/7] arm64/efi: Call EFI runtime services without disabling preemption

[Peter Zijlstra]

On Mon, Jul 14, 2025 at 12:20:30PM +1000, Ard Biesheuvel wrote:
> On Fri, 11 Jul 2025 at 23:48, Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Wed, May 14, 2025 at 07:43:47PM +0200, Ard Biesheuvel wrote:
> > > From: Ard Biesheuvel <ardb@kernel.org>
> > >
> > > The only remaining reason why EFI runtime services are invoked with
> > > preemption disabled is the fact that the mm is swapped out behind the
> > > back of the context switching code.
> > >
> > > The kernel no longer disables preemption in kernel_neon_begin().
> > > Furthermore, the EFI spec is being clarified to explicitly state that
> > > only baseline FP/SIMD is permitted in EFI runtime service
> > > implementations, and so the existing kernel mode NEON context switching
> > > code is sufficient to preserve and restore the execution context of an
> > > in-progress EFI runtime service call.
> > >
> > > Most EFI calls are made from the efi_rts_wq, which is serviced by a
> > > kthread. As kthreads never return to user space, they usually don't have
> > > an mm, and so we can use the existing infrastructure to swap in the
> > > efi_mm while the EFI call is in progress. This is visible to the
> > > scheduler, which will therefore reactivate the selected mm when
> > > switching out the kthread and back in again.
> > >
> > > Given that the EFI spec explicitly permits runtime services to be called
> > > with interrupts enabled, firmware code is already required to tolerate
> > > interruptions. So rather than disable preemption, disable only migration
> > > so that EFI runtime services are less likely to cause scheduling delays.
> > >
> > > Note, though, that the firmware executes at the same privilege level as
> > > the kernel, and is therefore able to disable interrupts altogether.
> >
> > Is the migrate_disable() strictly required, or just paranoia?
> >
> 
> Runtime services might be polling the secure firmware for an async
> completion when they are interrupted, and so I don't think it is
> generally safe to assume that an interrupted EFI runtime service can
> be resumed on another CPU.

Can we please get a comment with that migrate_disable() explaining this?

Thanks!

Re: [RFC PATCH 7/7] arm64/efi: Call EFI runtime services without disabling preemption

[Ard Biesheuvel]

On Mon, 14 Jul 2025 at 20:55, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Jul 14, 2025 at 12:20:30PM +1000, Ard Biesheuvel wrote:
> > On Fri, 11 Jul 2025 at 23:48, Peter Zijlstra <peterz@infradead.org> wrote:
...
> > > Is the migrate_disable() strictly required, or just paranoia?
> > >
> >
> > Runtime services might be polling the secure firmware for an async
> > completion when they are interrupted, and so I don't think it is
> > generally safe to assume that an interrupted EFI runtime service can
> > be resumed on another CPU.
>
> Can we please get a comment with that migrate_disable() explaining this?
>

Sure

Re: [RFC PATCH 0/7] arm64: Make EFI calls preemptible

[Will Deacon]

Hey Ard,

On Wed, May 14, 2025 at 07:43:40PM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb@kernel.org>
> 
> The arm64 port permits the use of the baseline FP/SIMD register file in
> kernel mode, and no longer requires preemption to be disabled. Now that
> the EFI spec is being clarified to state that EFI runtime services may
> only use baseline FP/SIMD, the fact that EFI may code may use FP/SIMD
> registers (while executing at the same privilege level as the kernel) is
> no longer a reason to disable preemption when invoking them.
> 
> This means that the only remaining reason for disabling preemption is
> the fact that the active mm is swapped out and replaced with efi_mm in a
> way that is hidden from the scheduler, and so scheduling is not
> supported currently. However, given that virtually all (*) EFI runtime
> calls are made from the efi_rts_wq workqueue, the efi_mm can simply be
> loaded into the workqueue worker kthread while the call is in progress,
> and this does not require preemption to be disabled.
> 
> Note that this is only a partial solution in terms of RT guarantees,
> given that the runtime services execute at the same privilege level as
> the kernel, and can therefore disable interrupts (and therefore
> preemption) directly. But it should prevent scheduling latency spikes
> for EFI calls that simply take a long time to run to completion.
> 
> (*) only efi_reset_system() and EFI pstore invoke EFI runtime services
>     without going through the workqueue, and the latter only when saving
>     a kernel oops log to the EFI varstore
> 
> Cc: Will Deacon <will@kernel.org>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> 
> Ard Biesheuvel (7):
>   efi: Add missing static initializer for efi_mm::cpus_allowed_lock
>   efi/runtime: Return success/failure from arch_efi_call_virt_setup()
>   efi/runtime: Deal with arch_efi_call_virt_setup() returning failure
>   arm64/fpsimd: Don't warn when EFI execution context is preemptible
>   arm64/efi: Use a semaphore to protect the EFI stack and FP/SIMD state
>   arm64/efi: Move uaccess en/disable out of efi_set_pgd()
>   arm64/efi: Call EFI runtime services without disabling preemption

I quite like the idea of this from the arm64 side (modulo a small comment
I've left on one of the later patches), however it would be great to hear
from one of the -rt folk to confirm that this actually solves a problem
for them.

Will