* [PATCH 1/6] x86/efi/mixed: Check CPU compatibility without relying on verify_cpu()
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 2/6] x86/efi/mixed: Remove dependency on legacy startup_32 code Ard Biesheuvel
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
In order for the EFI mixed mode startup code to be reusable in a context
where the legacy decompressor is not used, replace the call to
verify_cpu() [which performs an elaborate set of checks] with a simple
check against the 'long mode' bit in the appropriate CPUID leaf.
This is reasonable, given that EFI support is implied when booting in
this manner, and so there is no need to consider very old CPUs when
performing this check.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/efi_mixed.S | 22 ++++++++------------
1 file changed, 9 insertions(+), 13 deletions(-)
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S
index 876fc6d46a13..067e10eb7897 100644
--- a/arch/x86/boot/compressed/efi_mixed.S
+++ b/arch/x86/boot/compressed/efi_mixed.S
@@ -293,24 +293,20 @@ SYM_FUNC_END(efi32_entry)
* efi_system_table_32_t *sys_table)
*/
SYM_FUNC_START(efi32_pe_entry)
- pushl %ebp
- movl %esp, %ebp
pushl %ebx // save callee-save registers
- pushl %edi
-
- call verify_cpu // check for long mode support
- testl %eax, %eax
- movl $0x80000003, %eax // EFI_UNSUPPORTED
- jnz 2f
- movl 8(%ebp), %ecx // image_handle
- movl 12(%ebp), %edx // sys_table
+ /* Check whether the CPU supports long mode */
+ movl $0x80000001, %eax // assume extended info support
+ cpuid
+ btl $29, %edx // check long mode bit
+ jnc 1f
+ leal 8(%esp), %esp // preserve stack alignment
+ movl (%esp), %ecx // image_handle
+ movl 4(%esp), %edx // sys_table
jmp efi32_entry // pass %ecx, %edx
// no other registers remain live
-
-2: popl %edi // restore callee-save registers
+1: movl $0x80000003, %eax // EFI_UNSUPPORTED
popl %ebx
- leave
RET
SYM_FUNC_END(efi32_pe_entry)
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 2/6] x86/efi/mixed: Remove dependency on legacy startup_32 code
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 1/6] x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 3/6] x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors Ard Biesheuvel
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
The EFI mixed mode startup code calls into startup_32 in the legacy
decompressor with a mocked up boot_params struct, only to get it to set
up the 1:1 mapping of the lower 4 GiB of memory and switch to a GDT that
supports 64-bit mode.
In order to be able to reuse the EFI mixed mode startup code in EFI
zboot images, which do not incorporate the legacy decompressor code,
decouple it, by building the 1:1 map and installing the GDT directly.
This also removes the need to preserve and restore the stack pointer and
the arguments passed to the entrypoint, and instead, the firmware stack
can be retained, and the arguments can be loaded from it at the point
where they are needed, i.e., before calling into the EFI stub C code.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/efi_mixed.S | 143 ++++++++++++--------
arch/x86/boot/compressed/head_64.S | 7 -
2 files changed, 86 insertions(+), 64 deletions(-)
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S
index 067e10eb7897..c010aba7a0d0 100644
--- a/arch/x86/boot/compressed/efi_mixed.S
+++ b/arch/x86/boot/compressed/efi_mixed.S
@@ -15,15 +15,18 @@
*/
#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
#include <asm/msr.h>
#include <asm/page_types.h>
+#include <asm/pgtable_types.h>
#include <asm/processor-flags.h>
#include <asm/segment.h>
-#include <asm/setup.h>
.code64
.text
+ .balign 8
+SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff) /* __KERNEL_CS */
+ .set gdt_size, . - gdt
+
/*
* When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
* is the first thing that runs after switching to long mode. Depending on
@@ -35,30 +38,34 @@
* pointer is used to disambiguate.
*
* +--------------+
- * +------------------+ +------------+ +------>| efi_pe_entry |
- * | efi32_pe_entry |---->| | | +-----------+--+
- * +------------------+ | | +------+----------------+ |
- * | startup_32 |---->| startup_64_mixed_mode | |
- * +------------------+ | | +------+----------------+ |
- * | efi32_stub_entry |---->| | | |
- * +------------------+ +------------+ | |
+ * +------------------+ +-------------+ +------>| efi_pe_entry |
+ * | efi32_pe_entry |--->| | | +-----------+--+
+ * +------------------+ | | +------+----------------+ |
+ * | efi32_entry |---->| startup_64_mixed_mode | |
+ * +------------------+ | | +------+----------------+ |
+ * | efi32_stub_entry |--->| | | |
+ * +------------------+ +-------------+ | |
* V |
- * +------------+ +----------------+ |
- * | startup_64 |<----| efi_stub_entry |<--------+
- * +------------+ +----------------+
+ * +-------------+ +----------------+ |
+ * | vmlinux |<----| efi_stub_entry |<--------+
+ * +-------------+ +----------------+
*/
-SYM_FUNC_START(startup_64_mixed_mode)
- lea efi32_boot_args(%rip), %rdx
- mov 0(%rdx), %edi
- mov 4(%rdx), %esi
+SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
+ xorl %eax, %eax
+ movl %eax, %ds
+ movl %eax, %es
+ movl %eax, %ss
+ movl %eax, %fs
+ movl %eax, %gs
- /* Switch to the firmware's stack */
- movl efi32_boot_sp(%rip), %esp
- andl $~7, %esp
+ movl 0(%rsp), %ecx // MS calling convention
+ movl 4(%rsp), %edx
#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
- mov 8(%rdx), %edx // saved bootparams pointer
- test %edx, %edx
+ test %edi, %edi // struct boot_params provided?
+ movl %edx, %esi // SysV calling convention
+ cmovnzl %edi, %edx
+ movl %ecx, %edi
jnz efi_stub_entry
#endif
/*
@@ -69,8 +76,6 @@ SYM_FUNC_START(startup_64_mixed_mode)
* the correct stack alignment for entry.
*/
sub $40, %rsp
- mov %rdi, %rcx // MS calling convention
- mov %rsi, %rdx
jmp efi_pe_entry
SYM_FUNC_END(startup_64_mixed_mode)
@@ -151,7 +156,6 @@ SYM_FUNC_END(__efi64_thunk)
SYM_FUNC_START(efi32_stub_entry)
call 1f
1: popl %ecx
- leal (efi32_boot_args - 1b)(%ecx), %ebx
/* Clear BSS */
xorl %eax, %eax
@@ -163,10 +167,7 @@ SYM_FUNC_START(efi32_stub_entry)
rep stosl
add $0x4, %esp /* Discard return address */
- popl %ecx
- popl %edx
- popl %esi
- movl %esi, 8(%ebx)
+ movl 8(%esp), %edi /* struct boot_params pointer */
jmp efi32_entry
SYM_FUNC_END(efi32_stub_entry)
#endif
@@ -241,8 +242,9 @@ SYM_FUNC_END(efi_enter32)
/*
* This is the common EFI stub entry point for mixed mode.
*
- * Arguments: %ecx image handle
- * %edx EFI system table pointer
+ * Arguments: 0(%esp) image handle
+ * 4(%esp) EFI system table pointer
+ * %edi struct boot_params pointer (or NULL)
*
* Since this is the point of no return for ordinary execution, no registers
* are considered live except for the function parameters. [Note that the EFI
@@ -261,31 +263,58 @@ SYM_FUNC_START_LOCAL(efi32_entry)
/* Store firmware IDT descriptor */
sidtl (efi32_boot_idt - 1b)(%ebx)
- /* Store firmware stack pointer */
- movl %esp, (efi32_boot_sp - 1b)(%ebx)
-
- /* Store boot arguments */
- leal (efi32_boot_args - 1b)(%ebx), %ebx
- movl %ecx, 0(%ebx)
- movl %edx, 4(%ebx)
- movb $0x0, 12(%ebx) // efi_is64
-
- /*
- * Allocate some memory for a temporary struct boot_params, which only
- * needs the minimal pieces that startup_32() relies on.
- */
- subl $PARAM_SIZE, %esp
- movl %esp, %esi
- movl $PAGE_SIZE, BP_kernel_alignment(%esi)
- movl $_end - 1b, BP_init_size(%esi)
- subl $startup_32 - 1b, BP_init_size(%esi)
+ /* Record mixed mode entry */
+ movb $0x0, (efi_is64 - 1b)(%ebx)
/* Disable paging */
movl %cr0, %eax
btrl $X86_CR0_PG_BIT, %eax
movl %eax, %cr0
- jmp startup_32
+ /* Set up 1:1 mapping */
+ leal (pte - 1b)(%ebx), %eax
+ movl $_PAGE_PRESENT | _PAGE_RW | _PAGE_PSE, %ecx
+ leal (_PAGE_PRESENT | _PAGE_RW)(%eax), %edx
+2: movl %ecx, (%eax)
+ addl $8, %eax
+ addl $PMD_SIZE, %ecx
+ jnc 2b
+
+ xor %ecx, %ecx
+ movl $PAGE_SIZE, %esi
+3: movl %edx, (%eax,%ecx,8)
+ addl %esi, %edx
+ inc %ecx
+ cmp $4, %ecx
+ jl 3b
+
+ addl %esi, %eax
+ movl %edx, (%eax)
+ movl %eax, %cr3
+
+ movl %cr4, %eax
+ orl $X86_CR4_PAE, %eax
+ movl %eax, %cr4
+
+ movl $MSR_EFER, %ecx
+ rdmsr
+ btsl $_EFER_LME, %eax
+ wrmsr
+
+ leal (gdt - 1b)(%ebx), %ecx
+ pushl %ecx
+ pushw $gdt_size - 1
+ lgdtl (%esp)
+ lea 6(%esp), %esp
+
+ /* Enable paging and jump to long mode */
+ leal (startup_64_mixed_mode - 1b)(%ebx), %ecx
+ pushl $__KERNEL_CS
+ pushl %ecx
+ movl %cr0, %eax
+ btsl $X86_CR0_PG_BIT, %eax
+ movl %eax, %cr0
+ lret
SYM_FUNC_END(efi32_entry)
/*
@@ -301,10 +330,8 @@ SYM_FUNC_START(efi32_pe_entry)
btl $29, %edx // check long mode bit
jnc 1f
leal 8(%esp), %esp // preserve stack alignment
- movl (%esp), %ecx // image_handle
- movl 4(%esp), %edx // sys_table
- jmp efi32_entry // pass %ecx, %edx
- // no other registers remain live
+ xor %edi, %edi // no struct boot_params in EDI
+ jmp efi32_entry // only ESP and EDI remain live
1: movl $0x80000003, %eax // EFI_UNSUPPORTED
popl %ebx
RET
@@ -318,8 +345,10 @@ SYM_FUNC_START_NOALIGN(efi64_stub_entry)
SYM_FUNC_END(efi64_stub_entry)
#endif
- .data
- .balign 8
+ .bss
+ .balign PAGE_SIZE
+SYM_DATA_LOCAL(pte, .fill 6 * PAGE_SIZE, 1, 0)
+
SYM_DATA_START_LOCAL(efi32_boot_gdt)
.word 0
.quad 0
@@ -330,8 +359,8 @@ SYM_DATA_START_LOCAL(efi32_boot_idt)
.quad 0
SYM_DATA_END(efi32_boot_idt)
+ .data
+ .balign 4
SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
-SYM_DATA_LOCAL(efi32_boot_sp, .long 0)
-SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
SYM_DATA(efi_is64, .byte 1)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index 1dcb794c5479..5db6495a3bb9 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -263,13 +263,6 @@ SYM_FUNC_START(startup_32)
* used to perform that far jump.
*/
leal rva(startup_64)(%ebp), %eax
-#ifdef CONFIG_EFI_MIXED
- cmpb $1, rva(efi_is64)(%ebp)
- je 1f
- leal rva(startup_64_mixed_mode)(%ebp), %eax
-1:
-#endif
-
pushl $__KERNEL_CS
pushl %eax
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 3/6] x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 1/6] x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 2/6] x86/efi/mixed: Remove dependency on legacy startup_32 code Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 4/6] x86/efi/mixed: Simplify and document thunking logic Ard Biesheuvel
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
Commit
e10848a26a96 ("x86/efi: Preserve segment registers in mixed mode")
added code to preserve and restore the caller's data segment selector
values when temporarily switching back to the firmware's GDT and segment
layout to perform an EFI runtime call.
At that point, this code was shared between the boot-time and runtime
mixed mode implementations, and the issue this commit aimed to address
only affected the latter.
Since commit
96738c69a7fc ("x86/efi: Avoid triple faults during EFI mixed mode calls")
the runtime version of the mixed mode handling is entirely separate, and
relies on IA-32e compatibility mode to call into the 32-bit firmware
while retaining the OS's GDT/IDT and segment layout.
At boot-time, preserving/restoring the values is pointless, and the data
segment selectors can now simply be zeroed after (re)entering 64-bit
mode.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/efi_mixed.S | 24 ++++++--------------
1 file changed, 7 insertions(+), 17 deletions(-)
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S
index c010aba7a0d0..066f4365af4f 100644
--- a/arch/x86/boot/compressed/efi_mixed.S
+++ b/arch/x86/boot/compressed/efi_mixed.S
@@ -83,17 +83,10 @@ SYM_FUNC_START(__efi64_thunk)
push %rbp
push %rbx
- movl %ds, %eax
- push %rax
- movl %es, %eax
- push %rax
- movl %ss, %eax
- push %rax
-
/* Copy args passed on stack */
- movq 0x30(%rsp), %rbp
- movq 0x38(%rsp), %rbx
- movq 0x40(%rsp), %rax
+ movq 0x18(%rsp), %rbp
+ movq 0x20(%rsp), %rbx
+ movq 0x28(%rsp), %rax
/*
* Convert x86-64 ABI params to i386 ABI
@@ -135,14 +128,11 @@ SYM_FUNC_START(__efi64_thunk)
1: addq $64, %rsp
movq %rdi, %rax
- pop %rbx
- movl %ebx, %ss
- pop %rbx
- movl %ebx, %es
- pop %rbx
- movl %ebx, %ds
- /* Clear out 32-bit selector from FS and GS */
+ /* Clear out 32-bit segment selectors */
xorl %ebx, %ebx
+ movl %ebx, %ds
+ movl %ebx, %es
+ movl %ebx, %ss
movl %ebx, %fs
movl %ebx, %gs
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 4/6] x86/efi/mixed: Simplify and document thunking logic
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
` (2 preceding siblings ...)
2025-01-08 18:22 ` [PATCH 3/6] x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 5/6] x86/efi/mixed: Reduce padding by moving some code around Ard Biesheuvel
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
The current boot-time version of the thunk into 32-bit mode, to invoke
EFI services and protocols using the 32-bit calling convention, is a
jumble of argument marshalling code and GDT/IDT and segment register
handling, with an undocumented de facto calling convention that passes
the data segment descriptor, the return address and the GDT/IDT base
pointer for the return in registers EDX, EBP and EBX respectively.
Let's clean this up, and replace this with documented logic that
separates the handling of the segment registers into the 64-bit caller,
and the marshalling of the arguments into the 32-bit callee. Also,
replace the open coded far call with an actual one, removing the need to
pass the return address via a register.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/efi_mixed.S | 92 ++++++++++----------
1 file changed, 44 insertions(+), 48 deletions(-)
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S
index 066f4365af4f..a44e522bbbed 100644
--- a/arch/x86/boot/compressed/efi_mixed.S
+++ b/arch/x86/boot/compressed/efi_mixed.S
@@ -80,52 +80,36 @@ SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
SYM_FUNC_END(startup_64_mixed_mode)
SYM_FUNC_START(__efi64_thunk)
- push %rbp
push %rbx
- /* Copy args passed on stack */
- movq 0x18(%rsp), %rbp
- movq 0x20(%rsp), %rbx
- movq 0x28(%rsp), %rax
-
- /*
- * Convert x86-64 ABI params to i386 ABI
- */
- subq $64, %rsp
- movl %esi, 0x0(%rsp)
- movl %edx, 0x4(%rsp)
- movl %ecx, 0x8(%rsp)
- movl %r8d, 0xc(%rsp)
- movl %r9d, 0x10(%rsp)
- movl %ebp, 0x14(%rsp)
- movl %ebx, 0x18(%rsp)
- movl %eax, 0x1c(%rsp)
-
- leaq 0x20(%rsp), %rbx
- sgdt (%rbx)
- sidt 16(%rbx)
-
- leaq 1f(%rip), %rbp
+ /* Store live GDT and IDT descriptors */
+ subq $16, %rsp
+ sgdt (%rsp)
+ sidt 6(%rsp)
/*
* Switch to IDT and GDT with 32-bit segments. These are the firmware
* GDT and IDT that were installed when the kernel started executing.
* The pointers were saved by the efi32_entry() routine below.
- *
- * Pass the saved DS selector to the 32-bit code, and use far return to
- * restore the saved CS selector.
*/
lidt efi32_boot_idt(%rip)
lgdt efi32_boot_gdt(%rip)
- movzwl efi32_boot_ds(%rip), %edx
- movzwq efi32_boot_cs(%rip), %rax
- pushq %rax
- leaq efi_enter32(%rip), %rax
- pushq %rax
- lretq
+ /* Reload firmware's data segment selectors */
+ movw efi32_boot_ds(%rip), %bx
+ movl %ebx, %ds
+ movl %ebx, %es
+ movl %ebx, %ss
+ movl %ebx, %fs
+ movl %ebx, %gs
+
+ /* Move args #5 and #6 into 32-bit accessible registers */
+ movl %r8d, %eax
+ movl %r9d, %ebx
-1: addq $64, %rsp
+ lcalll *efi32_thunk(%rip)
+
+ addq $16, %rsp
movq %rdi, %rax
/* Clear out 32-bit segment selectors */
@@ -137,7 +121,6 @@ SYM_FUNC_START(__efi64_thunk)
movl %ebx, %gs
pop %rbx
- pop %rbp
RET
SYM_FUNC_END(__efi64_thunk)
@@ -163,17 +146,27 @@ SYM_FUNC_END(efi32_stub_entry)
#endif
/*
- * EFI service pointer must be in %edi.
+ * Called using a far call from 64-bit code, using the x86_64 SysV ABI (except
+ * for R8/R9 which are inaccessible to 32-bit code - EAX/EBX are used instead).
+ *
+ * The first argument (EDI) is a pointer to the boot service or protocol, to
+ * which the remaining arguments are passed, each truncated to 32 bits.
*
- * The stack should represent the 32-bit calling convention.
+ * Entered with ESP+40 pointing to the arguments passed via the stack, and with
+ * the 64-bit mode GDT and IDT descriptors at ESP+8 and ESP+14, respectively.
*/
SYM_FUNC_START_LOCAL(efi_enter32)
- /* Load firmware selector into data and stack segment registers */
- movl %edx, %ds
- movl %edx, %es
- movl %edx, %fs
- movl %edx, %gs
- movl %edx, %ss
+ /*
+ * Convert x86-64 SysV ABI params to i386 ABI
+ */
+ pushl 56(%esp) /* Up to 3 args passed via the caller's stack */
+ pushl 52(%esp)
+ pushl 48(%esp)
+ pushl %ebx /* R9 */
+ pushl %eax /* R8 */
+ pushl %ecx
+ pushl %edx
+ pushl %esi
/* Reload pgtables */
movl %cr3, %eax
@@ -201,8 +194,9 @@ SYM_FUNC_START_LOCAL(efi_enter32)
*/
cli
- lidtl 16(%ebx)
- lgdtl (%ebx)
+ addl $32, %esp
+ lidtl 14(%esp)
+ lgdtl 8(%esp)
movl %cr4, %eax
btsl $(X86_CR4_PAE_BIT), %eax
@@ -219,9 +213,6 @@ SYM_FUNC_START_LOCAL(efi_enter32)
xorl %eax, %eax
lldt %ax
- pushl $__KERNEL_CS
- pushl %ebp
-
/* Enable paging */
movl %cr0, %eax
btsl $X86_CR0_PG_BIT, %eax
@@ -250,6 +241,10 @@ SYM_FUNC_START_LOCAL(efi32_entry)
movw %cs, (efi32_boot_cs - 1b)(%ebx)
movw %ds, (efi32_boot_ds - 1b)(%ebx)
+ /* Fix up absolute reference */
+ leal (efi32_thunk - 1b)(%ebx), %eax
+ addl %eax, (%eax)
+
/* Store firmware IDT descriptor */
sidtl (efi32_boot_idt - 1b)(%ebx)
@@ -351,6 +346,7 @@ SYM_DATA_END(efi32_boot_idt)
.data
.balign 4
+SYM_DATA_LOCAL(efi32_thunk, .long efi_enter32 - .)
SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
SYM_DATA(efi_is64, .byte 1)
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 5/6] x86/efi/mixed: Reduce padding by moving some code around
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
` (3 preceding siblings ...)
2025-01-08 18:22 ` [PATCH 4/6] x86/efi/mixed: Simplify and document thunking logic Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-01-08 18:22 ` [PATCH 6/6] x86/efi/mixed: Move mixed mode startup code into libstub Ard Biesheuvel
2025-02-25 20:56 ` [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ingo Molnar
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
The 32-bit and 64-bit entrypoints for the EFI handover protocol need to
be exactly 0x200 bytes apart, and so the 64-bit version is placed at the
end, using padding to move it into place.
This padding can be reduced now, since the code in between has reduced
in size as well. So move efi32_stub_entry() to the beginning of the
source file.
Note that this code is outside of the compressed kernel image, and so
the padding is not compressed.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/efi_mixed.S | 51 ++++++++++----------
1 file changed, 26 insertions(+), 25 deletions(-)
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S
index a44e522bbbed..b5a46a4ce083 100644
--- a/arch/x86/boot/compressed/efi_mixed.S
+++ b/arch/x86/boot/compressed/efi_mixed.S
@@ -21,12 +21,32 @@
#include <asm/processor-flags.h>
#include <asm/segment.h>
- .code64
.text
.balign 8
SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff) /* __KERNEL_CS */
.set gdt_size, . - gdt
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
+ .code32
+SYM_FUNC_START_NOALIGN(efi32_stub_entry)
+ call 1f
+1: popl %ecx
+
+ /* Clear BSS */
+ xorl %eax, %eax
+ leal (_bss - 1b)(%ecx), %edi
+ leal (_ebss - 1b)(%ecx), %ecx
+ subl %edi, %ecx
+ shrl $2, %ecx
+ cld
+ rep stosl
+
+ add $0x4, %esp /* Discard return address */
+ movl 8(%esp), %edi /* struct boot_params pointer */
+ jmp efi32_entry
+SYM_FUNC_END(efi32_stub_entry)
+#endif
+
/*
* When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
* is the first thing that runs after switching to long mode. Depending on
@@ -50,6 +70,7 @@ SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff) /* __KERNEL_CS */
* | vmlinux |<----| efi_stub_entry |<--------+
* +-------------+ +----------------+
*/
+ .code64
SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
xorl %eax, %eax
movl %eax, %ds
@@ -79,7 +100,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
jmp efi_pe_entry
SYM_FUNC_END(startup_64_mixed_mode)
-SYM_FUNC_START(__efi64_thunk)
+SYM_FUNC_START_NOALIGN(__efi64_thunk)
push %rbx
/* Store live GDT and IDT descriptors */
@@ -125,26 +146,6 @@ SYM_FUNC_START(__efi64_thunk)
SYM_FUNC_END(__efi64_thunk)
.code32
-#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
-SYM_FUNC_START(efi32_stub_entry)
- call 1f
-1: popl %ecx
-
- /* Clear BSS */
- xorl %eax, %eax
- leal (_bss - 1b)(%ecx), %edi
- leal (_ebss - 1b)(%ecx), %ecx
- subl %edi, %ecx
- shrl $2, %ecx
- cld
- rep stosl
-
- add $0x4, %esp /* Discard return address */
- movl 8(%esp), %edi /* struct boot_params pointer */
- jmp efi32_entry
-SYM_FUNC_END(efi32_stub_entry)
-#endif
-
/*
* Called using a far call from 64-bit code, using the x86_64 SysV ABI (except
* for R8/R9 which are inaccessible to 32-bit code - EAX/EBX are used instead).
@@ -155,7 +156,7 @@ SYM_FUNC_END(efi32_stub_entry)
* Entered with ESP+40 pointing to the arguments passed via the stack, and with
* the 64-bit mode GDT and IDT descriptors at ESP+8 and ESP+14, respectively.
*/
-SYM_FUNC_START_LOCAL(efi_enter32)
+SYM_FUNC_START_LOCAL_NOALIGN(efi_enter32)
/*
* Convert x86-64 SysV ABI params to i386 ABI
*/
@@ -232,7 +233,7 @@ SYM_FUNC_END(efi_enter32)
* stub may still exit and return to the firmware using the Exit() EFI boot
* service.]
*/
-SYM_FUNC_START_LOCAL(efi32_entry)
+SYM_FUNC_START_LOCAL_NOALIGN(efi32_entry)
call 1f
1: pop %ebx
@@ -306,7 +307,7 @@ SYM_FUNC_END(efi32_entry)
* efi_status_t efi32_pe_entry(efi_handle_t image_handle,
* efi_system_table_32_t *sys_table)
*/
-SYM_FUNC_START(efi32_pe_entry)
+SYM_FUNC_START_NOALIGN(efi32_pe_entry)
pushl %ebx // save callee-save registers
/* Check whether the CPU supports long mode */
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 6/6] x86/efi/mixed: Move mixed mode startup code into libstub
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
` (4 preceding siblings ...)
2025-01-08 18:22 ` [PATCH 5/6] x86/efi/mixed: Reduce padding by moving some code around Ard Biesheuvel
@ 2025-01-08 18:22 ` Ard Biesheuvel
2025-02-25 20:56 ` [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ingo Molnar
6 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2025-01-08 18:22 UTC (permalink / raw)
To: linux-efi; +Cc: x86, kees, Ard Biesheuvel
From: Ard Biesheuvel <ardb@kernel.org>
The EFI mixed mode code has been decoupled from the legacy decompressor,
in order to be able to reuse it with generic EFI zboot images for x86.
Move the source file into the libstub source directory to facilitate
this.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/boot/compressed/Makefile | 1 -
drivers/firmware/efi/libstub/Makefile | 3 +++
arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 0
3 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index f2051644de94..fc5563704466 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -104,7 +104,6 @@ vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o $(obj)/td
vmlinux-objs-$(CONFIG_UNACCEPTED_MEMORY) += $(obj)/mem.o
vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
vmlinux-libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
$(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index e04285a7a6b9..f8cbd4557131 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -62,6 +62,8 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
# `-fdata-sections` flag from KBUILD_CFLAGS_KERNEL
KBUILD_CFLAGS_KERNEL := $(filter-out -fdata-sections, $(KBUILD_CFLAGS_KERNEL))
+KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
+
lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \
file.o mem.o random.o randomalloc.o pci.o \
skip_spaces.o lib-cmdline.o lib-ctype.o \
@@ -83,6 +85,7 @@ lib-$(CONFIG_EFI_GENERIC_STUB) += efi-stub.o string.o intrinsics.o systable.o \
lib-$(CONFIG_ARM) += arm32-stub.o
lib-$(CONFIG_ARM64) += kaslr.o arm64.o arm64-stub.o smbios.o
lib-$(CONFIG_X86) += x86-stub.o smbios.o
+lib-$(CONFIG_EFI_MIXED) += x86-mixed.o
lib-$(CONFIG_X86_64) += x86-5lvl.o
lib-$(CONFIG_RISCV) += kaslr.o riscv.o riscv-stub.o
lib-$(CONFIG_LOONGARCH) += loongarch.o loongarch-stub.o
diff --git a/arch/x86/boot/compressed/efi_mixed.S b/drivers/firmware/efi/libstub/x86-mixed.S
similarity index 100%
rename from arch/x86/boot/compressed/efi_mixed.S
rename to drivers/firmware/efi/libstub/x86-mixed.S
--
2.47.1.613.gc27f4b7a9f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor
2025-01-08 18:22 [PATCH 0/6] x86/efi/mixed: Decouple from legacy decompressor Ard Biesheuvel
` (5 preceding siblings ...)
2025-01-08 18:22 ` [PATCH 6/6] x86/efi/mixed: Move mixed mode startup code into libstub Ard Biesheuvel
@ 2025-02-25 20:56 ` Ingo Molnar
6 siblings, 0 replies; 8+ messages in thread
From: Ingo Molnar @ 2025-02-25 20:56 UTC (permalink / raw)
To: Ard Biesheuvel; +Cc: linux-efi, x86, kees, Ard Biesheuvel
* Ard Biesheuvel <ardb+git@google.com> wrote:
> From: Ard Biesheuvel <ardb@kernel.org>
>
> Since commit
>
> a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")
>
> booting via the EFI stub no longer relies on the legacy decompressor,
> and instead, the kernel proper is decompressed by code executing in the
> context of the EFI boot services, and subsequently invoked directly.
>
> The only remaining dependency is the EFI mixed mode startup code, which
> makes a detour via the legacy decompressor's 32-bit entrypoint, in order
> to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit
> execution on x86.
>
> This detour requires some fiddly setup on the part of the mixed mode
> startup code, which has to stash the firmware stack pointer and boot
> arguments in memory, and create a fake struct boot_params to trick the
> code in startup_32 to behave as intended.
>
> This dependency also impedes reuse of the EFI stub code in other
> contexts, such as generic EFI zboot, which will reuse the EFI stub but
> not the legacy decompressor.
>
> So remove this dependency, by replacing this detour with a minimal
> reimplementation of the 1:1 mapping code. With some further cleanup
> applied on top, the line count barely changes, but the resulting code
> can operate independently from the legacy decompressor, and is therefore
> moved out of arch/x86/boot/compressed and into the EFI libstub/
> directory.
>
> Ard Biesheuvel (6):
> x86/efi/mixed: Check CPU compatibility without relying on verify_cpu()
> x86/efi/mixed: Remove dependency on legacy startup_32 code
> x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors
> x86/efi/mixed: Simplify and document thunking logic
> x86/efi/mixed: Reduce padding by moving some code around
> x86/efi/mixed: Move mixed mode startup code into libstub
>
> arch/x86/boot/compressed/Makefile | 1 -
> arch/x86/boot/compressed/head_64.S | 7 -
> drivers/firmware/efi/libstub/Makefile | 3 +
> arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 320 ++++++++++----------
> 4 files changed, 169 insertions(+), 162 deletions(-)
> rename arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S (54%)
A belated:
Acked-by: Ingo Molnar <mingo@kernel.org>
Thanks,
Ingo
^ permalink raw reply [flat|nested] 8+ messages in thread